Network Working Group Keith Moore Internet-Draft University of Tennessee Expires: January 12, 2002 July 12, 2001 The Binary Low-Overhead Block Presentation Protocol draft-moore-rescap-blob-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This document is being submitted as a contribution to the IETF rescap working group. Comments regarding this internet-draft should be sent to the rescap mailing list at rescap@cs.utk.edu, or to the author at the address listed below. Requests to subscribe to the rescap mailing list should be sent to rescap-REQUEST@cs.utk.edu. This Internet-Draft will expire on January 12, 2001. Abstract This memo describes the Binary Low-Overhead Block (BLOB) protocol for on-the-wire presentation of data in the context of higher-level protocols. BLOB is designed to encode and decode data with low overhead on most CPUs, to be reasonably space-efficient, and for its representation to be sufficiently precise that it is suitable as a canonical format for digital signatures. Moore Expires January 12, 2002 [Page 1] BLOB Protocol Internet-Draft 12 July 2001 1. Introduction When designing applications-layer protocols there is sometimes a need to have an efficient means of encoding protocol elements or protocol data units. Existing solutions in this space may be deemed inadequate, for various reasons. For example: - ASN.1 [1] and BER [2] are baroque both in terms of the abstract syntax and available on-the-wire representations, and complex to implement. - ONC XDR [3] requires a stub generator and support libraries which are not easily available on all platforms, and there are subtle differences between the APIs provided by different implementations. XDR is large enough that it's not usually feasible to write your own implementation, and it's difficult to write portable code that can work with the various implementations that are deployed. Many XDR implementations have significant unnecessary processing overhead. This impairs performace of applications based on XDR and gives the protocol itself a worse reputation than it otherwise deserves. - The design of MIME [4] was heavily influenced by the need to be able to operate over existing text-based mail systems which imposed a number of constraints. This worked out well for email, but for other applications, MIME is neither efficient in terms of storage density nor easy to parse. - XML [5] is easier to parse than MIME, but still requires significant processing overhead. There is also a large and growing body of "culture" regarding how XML should be used, which paradoxically imposes a significant barrier to use of XML. (To be fair, MIME also has a fair amount of "culture" associated with it.) Finally, for small and regular data structures XML imposes a lot of overhead. BLOB was designed to serve as an alternative to these presentation layers for use in representing relatively simple strucutres, consisting of a limited set of primitive data types, and where the structures can reasonably be contained within a single protocol data unit. BLOB is designed with the following considerations: - It should be easy and efficient to generate the encoded form. - The encoded form should require minimal processing to decode, ideally being usable in-place (without allocating memory or copying) on most platforms. Moore Expires January 12, 2002 [Page 2] BLOB Protocol Internet-Draft 12 July 2001 - It should be easy to write programs which mainpulate and exchange BLOBs, without needing significant external support in the form of libraries or stub generators. - The structure should be easy and efficient to verify for internal consistency. - For any structure to be represented there should be a unique (canonical) on-the-wire encoding which is always used. - It should be reasonably space-efficient. However, this is secondary to minimizing processing overhead. The BLOB approach is more feasible now than in years past because data representations have become more uniform across different computing platforms. Essentially all widely-used computers now support 32-bit integers, can address 32-bit integers which are not aligned on any larger boundary, use word sizes which are a multiple of 8 bits, and can directly address strings of 8-bit characters which are not aligned on any boundary larger than an octet. Such computers are termed "well- behaved" with respect to BLOB. BLOB is designed to be usable on machines which do not have these characteristics, but such machines will necessarily incur more data conversion overhead. 2. Data Types A "blob" is a linear (octet-stream) encoding of a data structure (or "struct"), which is a sequence of heterogeneous data types. The data types which can appear as members of a struct are: - unsigned integer (32-bit) - string (variable-length sequence of arbitrary octets) - integer array (variable-length sequence of unsigned integers) - string array (variable-length sequence of strings) - struct (heterogeneous sequence of any of these types) These primitive types were chosen because they are directly usable on most hardware, they represent the vast majority of data types used in networking protocols, and because data types outside of this set are often specific to the higher-level protocol anyway. Having a limited set of data types allows for a more compact encoding, which is easier to decode, and which doesn't need separate marshalling routines for each individual structure. Moore Expires January 12, 2002 [Page 3] BLOB Protocol Internet-Draft 12 July 2001 "Variable-length" here means that the lengths of arrays need not be pre- determined by the protocol using BLOB. However the maximum lengths of strings and arrays are constrained by the use of a 32-bit integer for the length of the blob, and the representation of offsets within the blob as 32-bit integers. They may be further constrained by the higher- level protocol's choice of transmission medium - for instance, if the blob must fit into a UDP datagram. The number of members of a struct is considerably more constrained (as will become clear below), but should be adequate for most data structures encountered in protocols. When other data types are needed by a protocol, they must be represented in terms of the above primitive types. The higher-level protocol must choose the representation, and the conversion between the blob representation and the native format must be explicitly managed by the applications. For instance: - A signed 32-bit integer may be transmitted as an unsigned 32-bit integer by encoding the signed integer in twos-complement format. On most modern machines no conversion will be necessary; however on machines for which the smallest integer represenation is larger than 32 bits it will be necessary for the application to sign- extend the result. - A 64-bit integer may be transmitted as two consecutive 32-bit integers (with the most significant word first), which would require that the receiving application arrange those two integers according to its native byte ordering. Alternatively a 64-bit integer may be transmitted as eight consecutive octets within a string (most significant byte first), which would require that the receiving application re-arrange those octets according to its local byte ordering. - A multi-dimensional array may be represented as a single- dimensional array with the sizes of the dimensions passed as integer parameters; - Floating point numbers may be encoded in IEEE format and transmitted as either integers or strings (modulo sign-extension issues), - A small dense set may be represented as bits within an integer. Slightly larger dense sets may be encoded as bit offsets into an integer array. Larger or sparse sets may be represented by encoding them in a string. Moore Expires January 12, 2002 [Page 4] BLOB Protocol Internet-Draft 12 July 2001 3. BLOB Protocol The basic unit of BLOB encoding is a "struct". A "blob" is a sequence of octets which forms the on-the-wire representation of a struct. At the most basic level, the blob consists of an integer portion followed by a string portion. The integer portion consists of a header, an argument list, and an integer pool. Each of these is a sequence of integers, all of which are 4 octets in length and represented on-the- wire in network byte (big-endian) order. The string portion is a sequence of octets; order of octets is preserved within the string portion. The blob is separated into string and integer portions in order to facilitate easy decoding. In order for the blob to be usable on a little-endian cpu, each integer of the integer portion will need to have its octets reversed. By contrast, the string portion has the same representation on both big-endian and little-endian platforms. Thus on a "well-behaved" little-endian machine the blob can be converted from on-the-wire format to a format which is usable locally, merely by reversing the order of the octets within each of the first (string_pool_offset / 4) 32-bit integers of the blob. No conversion is necessary in order to use a blob on a "well-behaved" big-endian machine. Since a blob is the on-the-wire representation of a struct, if a blob contains one or more structs as components of the outer struct, they will themselves be represented as blobs. Those blobs will be stored in the string pool. Inner blobs must be explicitly decoded/converted by the receiving application; they are not automatically decoded when the outer blob is decoded. Moore Expires January 12, 2002 [Page 5] BLOB Protocol Internet-Draft 12 July 2001 3.1 Structure of a blob The structure of a blob is as follows: octet offset name 0 +--------------------------------+ \ | blob_length | | 4 +--------------------------------+ | | integer_pool_offset | | 8 +--------------------------------+ | | string_pool_offset | | 12 +--------------------------------+ | | argument_counts | | 16 +--------------------------------+ + integer portion : : | : argument list : | : : | integer_pool_offset +--------------------------------+ | : : | : integer_pool : | : : / string_pool_offset +--------------------------------+ \ : : | : string_pool : + string portion : : | blob length +--------------------------------+/ blob_length The blob_length is the length of the entire blob in octets. The length includes the space occupied by blob_length. integer_pool_offset The integer_pool_offset is the octet offset (relative to the start of the blob) of the integer_pool portion of the blob. integer_pool_offset must be a multiple of four, greater than or equal to 16, and less than or equal to string_pool_offset. If the length of integer_pool is zero, integer_pool_offset will be equal to string_pool_offset. string_pool_offset The string_pool_offset is the offset (relative to the start of the blob) of the string_pool portion of the blob. It must be greater than or equal to integer_pool_offset and less than or equal to blob_length. If the length of the string_pool is zero, string_pool_offset will be equal to blob_length. Moore Expires January 12, 2002 [Page 6] BLOB Protocol Internet-Draft 12 July 2001 argument_counts The argument_counts field indicates the number of each kind of argument. This field is calculated as follows: argument_counts = (num_int_args) + (num_int_array_args << 8) + (num_string_or_struct_args << 16) + (num_string_or_struct_array_args << 24) where num_xxx_args is the number of arguments of type xxx, and num_xxx_array_args is the number of arguments of type array of xxx. argument_list The argument_list contains a list of integers which represent the members of the struct. In order that the blob may be sanity checked for internal consistency without wasting lots of space, the arguments within the argument_list are arranged so that similar types of arguments are consecutive. Within the argument_list, the arguments appear in the following order: 1. int arguments 2. int array arguments 3. string or struct arguments 4. string or struct array arguments integer_pool The integer_pool contains integers in the following order: 1. The elements of integer arrays, in the order that these arrays appear in the argument list. 2. Offsets of strings and structs within string and struct arrays, in the order that the offsets of these arrays appear in the argument list. These offsets are offsets from the beginning of the blob, and point into the string_pool. string_pool The string_pool begins at string_pool_offset and contains strings and embedded structs which are referenced within the outer struct. The strings and structs appear in the following order: 1. Contents of strings or structs that are referenced in the argument list, in the order that those offsets appear in the argument list. Moore Expires January 12, 2002 [Page 7] BLOB Protocol Internet-Draft 12 July 2001 2. Contents of strings or structs that are elements of arrays, in the order that their offsets appear in the integer_pool. For compatibility with programming languages which terminate strings with a zero octet, a zero octet is automatically appended to each string in the string_pool. 3.2 Struct Member Encoding The members of a struct are encoded as follows: - An "int" is represented as a 32-bit integer in big-endian format. - An "int array" is represented as an integer offset relative to the beginning of the blob, which points to the elements of the array. The elements of the array are stored in the integer_pool, in increasing order, in big-endian format. The offset of an integer array must therefore be greater than or equal to integer_pool_offset and less than or equal to string_pool_offset. Consecutive int arrays are stored in consecutive locations within the integer_pool. Thus the length of an integer array N (where N is less than the number of integer arrays, minus 1) can be determined by subtracting the offset of integer array N from the offset of integer array N+1, and dividing the result by 4. The length of the last integer array can be determined by subtracting the offset of that integer array from the offset of the first string array, or if there are no string arrays, from string_pool_offset. - A "string" is represented as an integer offset relative to the beginning of the blob, which points to the contents of the string. The contents of the string are stored in the string_pool. The offset of any string must therefore be greater than or equal to the string_pool_offset and less than or equal to blob_length. String arguments, and elements of string arrays, are stored consecutively in the string pool. Each string is followed in the string_pool by a zero octet which is not part of the string. Thus the length of any string argument (other than the last) can be calculated by subtracting its offset from the offset of the subsequent string argument, minus 1. The length of the last string argument can be calculated by subtracting its offset from the offset of the first element of the first string array, or if there are no string arrays, from Moore Expires January 12, 2002 [Page 8] BLOB Protocol Internet-Draft 12 July 2001 blob_length. Strings can be of zero length, in which case the corresponding offset points to a zero octet which is immediately followed by the next string in the string_pool. Strings can also be 'missing' or NULL, in which case the offset is zero. - A "string array" is represented as an integer offset (relative to the beginning of the blob) which points to an array of integers (stored in the integer pool), each element of which points to the offset of a string (within the string pool). The length of any string array element (other than the last one in that array) can be calculated by subtracting its offset from the offset of the subsequent element, minus 1. The length of the last element in a string array (other than the last string array) can be calculated by subtracting its offset from the offset of the first element of the subsequent string array, minus 1. The length of the last element in the last string array can be calculated by subtracting its offset from blob_length, minus 1. - A "struct" is represented as an integer offset (relative to the beginning of the blob) which points to the beginning of an inner blob (stored in the string portion of the outer blob), which contains the inner struct. - A "struct array" is represented as an integer offset (relative to the beginning of the blob), which points to an array of integers (stored in the integer pool), each element of which points to the offset of a blob (within the string pool) that represents a struct. 4. Use of BLOBs by higher-level protocols Higher-level protocols using BLOB as an encoding mechanism need to define their protocol data units in terms of BLOB "structs". Since BLOB groups all similarly-typed data together within the blob (for ease of conversion), and since BLOB rigidly defines the order in which data must appear, applications generally cannot refer to protocol elements within a blob by a fixed offset. Instead, the application code references protocol elements in terms of "the second string parameter", "the third integer parameter" or "the second element of the fourth integer array parameter". Macros which allow these elements to be accessed from a decoded blob structure are easily constructed. It is possible to define a simple specification language which allows the elements of a struct to be specified in the order that makes the Moore Expires January 12, 2002 [Page 9] BLOB Protocol Internet-Draft 12 July 2001 most sense to an application, and which produces a list of macros which map from protocol data element names to routines which can access those data elements. This hides the details of BLOB's reordering from the application without significantly impairing efficiency. An example of such a language is given in Appendix B. If higher-level protocols employ data types other than the BLOB primitive data types, they must define how the application-specific data types are represented as one or more BLOB primitive types, and implementations of the protocol will be responsible for conversion. Applications which require a canonical form (say for signing) should specify the conversion from application data types to BLOB types so that there is exactly one possible representation of each application data type within BLOB. Since a single blobs cannot encode arbitrarily complex structures, and since nesting blobs add a bit of overhead, protocol designers should avoid deep nesting of structures. For instance, what to the application is conceptually an array of structs may be better represented within BLOB as a set of parallel arrays. At the same time, nesting of structs is useful when it is desired that an inner blob be opaque to the layer of a protocol that decodes the outer blob. 5. Encoding Issues Most blobs will contain at least one variable-length data structure. This implies that a program that encodes a blob will usually be unable to generate the elements of a blob in-place. Instead, the program will need to copy the elements of a blob from their various locations into a contiguous location in memory, in ther order prescribed by the BLOB specification. A sample implementation is given in Appendix C. 6. Decoding Issues On "well-behaved" machines it should be possible to use blobs in-place after converting the integer portion of the blob to the local byte order. The protocol elements within the blob can then be accessed with macros. It is necessary to check the blob for consistency before using it. In particular: - The blob_length must be consistent with the length of the PDU or buffer in which the blob was received. (For instance, it must not be less than the length of data received). - The blob_length must be at least 16 (which would be the length of an empty blob with no arguments). Moore Expires January 12, 2002 [Page 10] BLOB Protocol Internet-Draft 12 July 2001 - The integer_pool_offset must be equal to the the number of arguments (decoded from argument_counts) multiplied by 4, plus 16. - The string_pool_offset must be greater than or equal to integer_pool_offset. - The string_pool_offset must be less than or equal to blob_length. - The offset of each integer array must be a multiple of 4. - The offset of the first integer array (if any) must be equal to integer_pool_offset. - Each subsequent non-null integer array offset must be greater than or equal to the previous integer array offset, and less than string_pool_offset. - The offset of the first element of the first string array must be greater than or equal to the offset of the last non-null integer array. - The offset of the first element of each subsequent string array must be greater than or equal to the offset of the first element of the previous string array. - The first string argument must have an offset equal to string_pool. - Each subsequent non-null string argument must have an offset greater (by at least 1) than that of the previous string argument. - The first element of the first string array must have an offset greater (by at least 1) than the offset of the last string argument. - The first element of any subsequent string array must have an offset which is greater (by at least 1) than the last element of the previous string array. - Each element of a string array must have an offset greater (by at least 1) than the offset of the previous element in that array. - Except for the first string, there must be a zero octet preceding each offset of each non-null string argument or non-null string array element. - The last octet in the string_pool must be a zero. Moore Expires January 12, 2002 [Page 11] BLOB Protocol Internet-Draft 12 July 2001 A sample implementation is given in Appendix D. 7. Security Considerations It is believed that the BLOB encoding is unique and can serve as a useful 'canonical form' for a data structure. However, if higher-level protocols encode non-native data types as BLOB primitive types, they must also define a unique representation for each quantity to be stored in that data-type. In order to prevent possible attacks by transmission of blobs containing bogus offsets, it is essential to perform the bounds checks listed in section 6 while decoding blobs. While such attacks could not easily overwrite memory with data chosen by an attacker, they could cause a server to malfunction. 8. Author's Address Keith Moore University of Tennessee 1122 Volunteer Blvd, Suite 203 Knoxville TN 37996-3450 email: moore@cs.utk.edu 9. References [1] "Specification of Basic Encoding Rules for Abstract Syntax Notation One (ASN.1)", CCITT Recommendation X.209, January 1988. [2] "Specification of ASN.1 encoding rules: Basic, Canonical, and Distinguished Encoding Rules", ITU-T X.690, January 1994. [3] Srinivasan, R., "XDR: External Data Representation Standard", RFC 1832, August 1995. [4] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, November 1996. [5] "Extensible Markup Language (XML) 1.0 (Second Edition)", W3C Recommendation, October 2000, . [6] Crocker, D. (ed.), Overell, P. "Augmented BNF for Syntax Specifications: ABNF.". RFC 2234, November 1997. Moore Expires January 12, 2002 [Page 12] BLOB Protocol Internet-Draft 12 July 2001 Appendix A. ASCII-Art Picture of a BLOB This diagram attempts to illustrate the ordering of the various elements of a blob and the relationship of the offsets to the elements to which they point: octet offset name 0 +--------------------------------+ | blob_length | 4 +--------------------------------+ | integer_pool_offset | 8 +--------------------------------+ | string_pool_offset | 12 +--------------------------------+ | argument_counts | 16 +--------------------------------+ The argument list looks like this: 16 +--------------------------------+ | 1st scalar int arg | +--------------------------------+ | 2nd scalar int arg | +--------------------------------+ : : (16 + num_int_args / 4) +--------------------------------+ | offset of 1st int array arg |--+ +--------------------------------+ | | offset of 2nd int array arg |--|--+ +--------------------------------+ | | 16 + (num_int_args + : : | | num_int_array_args) / 4 +--------------------------------+ | | | offset of 1st struct/string arg|--|--|---+ +--------------------------------+ | | | | offset of 2nd string/string arg|--|--|---|-+ +--------------------------------+ | | | | : : | | | | 16 + (num_int_args + : : | | | | num_int_array_args + : : | | | | numstr_or_strct_args) / 4 +--------------------------------+ | | | | | offset of 1st str* array arg |--|--|-+ | | +--------------------------------+ | | | | | | offset of 2nd str* array arg | | | | | | +--------------------------------+ | | | | | | offset of 3rd str* array arg | | | | | | integer pool offset +--------------------------------+ | | | | | | | | | | Moore Expires January 12, 2002 [Page 13] BLOB Protocol Internet-Draft 12 July 2001 | | | | | | | | | | The integer pool looks like this: | | | | | | | | | | integer pool offset = | | | | | offset of 1st +--------------------------------+ <+ | | | | int array arg | 1st element of 1st array | | | | | +--------------------------------+ | | | | | 2nd element of 1st array | | | | | +--------------------------------+ | | | | : : | | | | : : | | | | : : | | | | offset of 2nd +--------------------------------+ <---+ | | | int array arg | 1st element of 2nd array | | | | +--------------------------------+ | | | : : | | | : : | | | offset of 1st +--------------------------------+ <-----+ | | str* array arg | offset of 1st elem of 1st str* | | | +--------------------------------+ | | | offset of 2nd elem of 1st str* | | | +--------------------------------+ | | : : | | offset of 2nd +--------------------------------+ | | str* array arg | offset of 1st elem of 2nd str* | | | +--------------------------------+ | | | offset of 2nd elem of 2nd str* | | | +--------------------------------+ | | | | | | The string pool looks like this: | | | | string pool offset = | | offset of first +--------------------------------+ <-------+ | string arg | S T R I | | +--------------------------------+ | | N G 1 \0 | | +--------------------------------+ <---------+ offset of second | S e c o | string arg +--------------------------------+ | n d \040 S | +--------------------------------+ | t r i n | +--------------------------------+ | g \0 | +----------------+ Moore Expires January 12, 2002 [Page 14] BLOB Protocol Internet-Draft 12 July 2001 Appendix B. Example Abstract Syntax This syntax used to describe BLOB structures is described below using the ABNF syntax from [6]: file = *(block / comment-line) block = "BEGIN" 1*space id [ 1*space comment ] CRLF *element END [ comment ] CRLF element = "int" 1*space identifier [ comment ] CRLF / "string" 1*space identifier [ comment ] CRLF / "int<>" 1*space identifier [ comment ] CRLF / "string<>" 1*space identifier [ comment ] CRLF / "struct" 1*space identifier [ comment ] CRLF comment = *space "#" *char comment-line = comment CRLF id = letter *(letter / digit / "_") letter = "A".."Z" / "a".."z" digit = "0".."9" space = %20 / %09 char = %01..%09 / %0B / %0C / %0E..%FF CRLF = 0*1%0D 0*1%0A Here is a simple awk program to interpret this syntax and produce a list of C #define macros. The macros are of the form #define structname_element_type number where 'structname' is the name of the structure, 'element' is the name of the element, and 'type' is a suffix indicating the type of the element (i = int, s = string/struct, ia = integer array, sa = struct/string array) for ease in visual type checking. This program is quite simplistic and performs no error checking. Moore Expires January 12, 2002 [Page 15] BLOB Protocol Internet-Draft 12 July 2001 #!/bin/sh # the sed line deletes comments sed -e 's/[ ]*#.*//' | awk ' $1 == "BEGIN" { current_id = $2; nint = nstr = ninta = nstra = 0; } $1 == "int" { inames[nint] = $2; nint++; next; } $1 == "string" { snames[nstr] = $2; nstr++; next; } $1 == "struct" { snames[nstr] = $2; nstr++; next; } $1 == "int<>" { ianames[ninta] = $2; ninta++; next; } $1 == "string<>" { sanames[nstra] = $2; nstra++; next; } $1 == "struct<>" { sanames[nstra] = $2; nstra++; next; } $1 == "END" { for (i = 0; i < nint; ++i) printf ("#define %s_%s_i %d\n", current_id, inames[i], i); for (i = 0; i < nstr; ++i) printf ("#define %s_%s_s %d\n", current_id, snames[i], i); for (i = 0; i < ninta; ++i) printf ("#define %s_%s_ia %d\n", current_id, ianames[i], i); for (i = 0; i < nstra; ++i) printf ("#define %s_%s_sa %d\n", current_id, sanames[i], i); next; }' Moore Expires January 12, 2002 [Page 16] BLOB Protocol Internet-Draft 12 July 2001 Appendix C. Example Encoding Code NB: due to deadline pressures this code has not been recently tested, and probably contains bugs. Check http://www.cs.utk.edu/~moore/blob for the latest version. struct preblob { int ni_args; /* number of integer arguments */ int i_args[256]; /* integer arguments */ int nia_args; /* number of integer array arguments */ int *ia_args[256]; /* bases of integer array arguments */ int lia_args[256]; /* num elements in each integer array */ int ns_args; /* number of string arguments */ char *s_args[256]; /* bases of string arguments */ int ls_args[256]; /* length of each string argument */ int nsa_args; /* number of string array arguments */ char **sa_args[256]; /* base of each string array */ int nlsa_args[256]; /* number of elements in each string array */ int *lsa_args[256]; /* lengths of strings in each string array */ char *blob; int blobsize; }; /* initialize a blob - this is called only once */ #define blob_init (p) \ memset (&(p), 0, sizeof (struct preblob)) /* reset the state of a blob without leaking any memory that it has allocated */ #define blob_reset (p) \ do { \ char *tblob = (p).blob; \ int tblobsize = (p).blobsize; \ blob_init (p); \ (p).blob = tblob; \ (p).blobsize = tblobsize; \ } while (0) /* set the number of integer parameters in a blob */ #define blob_set_nint (p, n) \ (p).ni_args = (n); Moore Expires January 12, 2002 [Page 17] BLOB Protocol Internet-Draft 12 July 2001 /* set the value of the nth integer parameter to x */ #define blob_set_int (p, n, x) \ (p).i_args[n] = (x) /* set the number of string parameters in a blob */ #define blob_set_nstr (p, n) \ (p).ns_args = (n); /* set the number of integer array parameters in a blob */ #define blob_set_ninta (p, n) \ (p).nia_args = (n); /* set the number of string array parameters in a blob */ #define blob_set_nstra (p, n) \ (p).nsa_args = (n); /* set the value of the nth string parameter to 'str' where 'str' is NUL-terminated */ #define blob_set_str0 (p, n, str) \ do { \ (p).s_args[n] = (str); \ (p).ls_args[n] = strlen(str); \ } while (0) /* set the value of the nth string parameter to 'str' where 'str' is 'len' bytes long */ #define blob_set_strl (p, n, str, len) \ do { \ (p).s_args[n] = (str); \ (p).ls_args[n] = (len); \ } while (0) /* set the value of the nth integer array to the in-core integer array starting at 'base' and containing 'nelem' elements */ #define blob_set_int_array (p, n, base, nelem) \ do { \ (p).ia_args[n] = (base); \ (p).lia_args[n] = (nelem); \ } while (0) Moore Expires January 12, 2002 [Page 18] BLOB Protocol Internet-Draft 12 July 2001 /* set the value of the nth string array to the in-core string array starting at 'bases' and containing 'nelem' strings, where each string is NUL-terminated */ #define blob_set_str0_array (p, n, bases, nelem) \ do { \ (p).sa_args[n] = (bases); \ (p).lsa_args[n] = NULL; \ (p).nlsa_args[n] = (nelem); \ } while (0) /* * set the value of the nth string array to the * in-core string array starting at 'bases' * with the lengths stored in integer array 'lengths' * where each array is 'nelem' long */ #define blob_set_strl_array (p, n, bases, lengths, nelem) \ do { \ (p).sa_args[n] = (bases); \ (p).lsa_args[n] = (lengths); \ (p).nlsa_args[n] = (nelem); \ } while (0) /* * encode an int 'x' in big-endian format at ptr 'p'. * this is designed to be portable, there are certainly more * efficient ways to do this on any specific machine * * it should be okay to assume that 'ptr' is aligned on a 4-byte * boundary. */ #define ENCODE_INT(ptr, x) \ do { \ *ptr++ = ((x) >> 24) & 0xff; \ *ptr++ = ((x) >> 16) & 0xff; \ *ptr++ = ((x) >> 8) & 0xff; \ *ptr++ = (x) & 0xff; \ } while (0) Moore Expires January 12, 2002 [Page 19] BLOB Protocol Internet-Draft 12 July 2001 /* * this routine encodes a blob pointed to by 'p' * and leaves the result at p->blob * with the size in p->blobsize */ int blob_encode (struct preblob *p) { int i; int size = 0; int ipoolsize = 0; int spoolsize = 0; int nargs; unsigned int argcounts; char *ptr; char *iptr; char *sptr; if ((p->ni_args > 255) || (p->nia_args > 255) || (p->ns_args > 255) || (p->nsa_args > 255)) return -1; /* too many arguments */ /* * calculate the amount of space needed */ nargs = p->ni_args + p->nia_args + p->ns_args + p->nsa_args; argcounts = p->ni_args + (p->nia_args << 8) + (p->ns_args << 16) + (p->nsa_args << 24); size = 16 + (4 * nargs); /* size of integer array arguments */ for (i = 0; i < p->nia_args; ++i) ipoolsize += p->lia_args[i] * 4; /* size of string arguments */ for (i = 0; i < p->ns_args; ++i) { if (p->s_args[i] != 0) spoolsize += p->ls_args[i] + 1; } /* size of string array arguments */ for (i = 0; i < p->nsa_args; ++i) { int j; int *lengths = p->lsa_args[i]; Moore Expires January 12, 2002 [Page 20] BLOB Protocol Internet-Draft 12 July 2001 ipoolsize += p->nlsa_args[i] * 4; for (j = 0; j < p->nlsa_args[i]; ++j) { if (p->sa_args[i][j] != 0) { if (lengths) spoolsize += lengths[j] + 1; else spoolsize += strlen (p->sa_args[i][j]) + 1; } } } size = size + ipoolsize + spoolsize; /* * make sure there's enough space allocated */ if (p->blobsize == 0) { p->blob = (char *) malloc (size); p->blobsize = size; } else { p->blob = (char *) realloc (p->blob, size); p->blobsize = size; } /* * now, encode things */ ptr = p->blob; iptr = p->blob + 16 + (nargs * 4); sptr = p->blob + 16 + (nargs * 4) + ipoolsize; /* header */ ENCODE_INT (ptr, size); ENCODE_INT (ptr, 16 + (nargs * 4)); ENCODE_INT (ptr, 16 + (nargs * 4) + ipoolsize); ENCODE_INT (ptr, argcounts); /* int arguments */ for (i = 0; i < p->ni_args; ++i) ENCODE_INT (ptr, p->i_args[i]); /* int array arguments */ for (i = 0; i < p->nia_args; ++i) { int j; ENCODE_INT (ptr, iptr - p->blob); for (j = 0; j < p->lia_args[i]; ++j) ENCODE_INT (iptr, p->ia_args[i][j]); Moore Expires January 12, 2002 [Page 21] BLOB Protocol Internet-Draft 12 July 2001 } /* string arguments */ for (i = 0; i < p->ns_args; ++i) { if (p->s_args[i] != 0) { ENCODE_INT (ptr, sptr - p->blob); memcpy (sptr, p->s_args[i], p->ls_args[i]); sptr[p->ls_args[i]] = '\0'; sptr += p->ls_args[i] + 1; } else ENCODE_INT (ptr, 0); } /* string array arguments */ for (i = 0; i < p->nsa_args; ++i) { int j; ENCODE_INT (ptr, iptr - p->blob); for (j = 0; j < p->nlsa_args[i]; ++j) { if (p->sa_args[i][j] != 0) { ENCODE_INT (iptr, sptr - p->blob); if (p->lsa_args[i]) { memcpy (sptr, p->sa_args[i][j], p->lsa_args[i][j]); sptr += p->lsa_args[i][j]; *sptr++ = '\0'; } else { char *src = p->sa_args[i][j]; while (*sptr++ = *src++); } } else ENCODE_INT (iptr, 0); } } } Appendix D: Example Decoding Code This code will be supplied in a later version of this document. Check http://www.cs.utk.edu/~moore/blob for availability. Moore Expires January 12, 2002 [Page 22]