Extensible XDR Discriminated Union Primitive Type

AFS-3 is a distributed file system that has its origins in the VICE project at the Carnegie Mellon University Information Technology Center , a joint venture between CMU and IBM. VICE later became AFS when CMU moved development to a new commercial venture called Transarc Corporation, which later became IBM Pittsburgh Labs. AFS-3 is a suite of un-standardized network protocols based on a remote procedure call (RPC) suite known as Rx . While de jure standards for AFS-3 fail to exist, the various AFS-3 implementations have agreed upon certain de facto standards, largely helped by the existence of an open source fork called OpenAFS that has served the role of reference implementation. In addition to using OpenAFS as a reference, IBM wrote and donated developer documentation that contains somewhat outdated specifications for the Rx protocol and all AFS-3 remote procedure calls, as well as a detailed description of the AFS-3 system architecture. The Rx RPC protocol utilizes XDR as its means of encoding RPC call and response payloads. XDR provides a discriminated union type. However, the semantics of the discriminated union base type do not lend themselves to evolution of the discriminant namespace: introduction of new discriminants--when there is no default leg--cause the remainder of the XDR octet stream to be un-parseable (due to the lack of a length field in the encoding) by older peers. This memo introduces a new XDR primitive type that is identical to the XDR discriminated union, except that: each leg contains a length field, and the default leg is disallowed.

Given that this design doubles the overhead from 4 to 8 octets (relative to the XDR discriminated union primitive type), it is ill-suited for use with small implied legs. Within the AFS-3 protocol suite, the primary use case for the extensible union type is to wrap large data structures, rather than small primitive types.

Historically, AFS stood for the Andrew File System; AFS no longer stands for anything Remote Procedure Call Rx RPC Interface Definition Language (fork of ONC RPC .x file format) The Remote Procedure Call mechanism utilized by AFS-3 eXternal Data Representation

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

The extensible discriminated union will contain a length field in every leg so that decoding peers can compute the offset of the next object in the XDR octet stream, regardless of whether the discriminant is recognized. For small legs, this will result in significant encoding inefficiency, but it is necessary to permit the union to evolve over time (without peers failing to decode the entire XDR octet stream).

The definition of the extensible discriminated union is derived from the XDR union defined in section 4.15 of the XDR specification . Unlike XDR discriminated unions, the XDR types mapped to each arm of the union need not be defined a priori. Instead, the length of the arm is always included in the wire encoding along with the discriminant value, thus permitting the decoder to continue decoding past an unknown discriminant in an XDR octet stream. How undefined discriminants are handled by the decoder is deliberately left unspecified by this document. Rather, this memo merely specifies which error conditions must be flagged to the caller (see ). The error handling semantics--for both length mismatches and unknown discriminants--are left up to the definition of any type built upon the ext-union primitive type. While this lends significant flexibility to the design, it also permits XDR decoding to continue when the expected implied arm length doesn't match the length on the wire. It is RECOMMENDED that implementations fail to decode the entire XDR stream when such a length mismatch is encountered, as such a length mismatch is indicative of either: a significant divergence in RPC-L definitions across the peers, or an undetected bit error in the XDR octet stream. Extensible discriminated unions are defined in RPC-L as follows:

Because the discriminant namespace of an extensible union must be capable of evolving over time, it is not possible to support a default leg. The max-unknown-leg-length optional parameter specifies the maximum permissible leg length for any union leg whose discriminant value is not known to the decoder. This is necessary to limit the scope of denial of service attacks: by permitting the decoder to detect inordinately large payloads--after only receiving the first few octets of the extensible union. The extensible discriminated union is encoded on the wire as: a 4-octet discriminant, followed by a 4-octet arm length, and finally the variable-length implied arm. The arm length field SHALL count the length of the implied arm in octets, and only the implied arm. In other words, the 8 octets occupied by the discriminant and arm length fields SHALL NOT be counted as part of the arm length value.

|<---4 octets-->| ]]> It should be noted that this design makes it convenient to implement extensible discriminated unions on top of existing XDR primitive types. Thus, in terms of the existing XDR primitives , we can describe an extensible discriminated union as follows:

; }; ]]>

In order to implement the above, the XDR grammar, as specified in Section 6.3 of , will need to be modified in the following ways: "type-specifier" will require a new production rule mapping to "ext-union-type-spec", and an "ext-union-type-spec" production rule will need to be defined. The "type-specifier" grammar will now include a new production rule for "ext-union-type-spec":

The new "ext-union-type-spec" production rule, and the production rule for its nonterminal symbol dependency "ext-union-body", are defined as follows:

It is RECOMMENDED that encoding of an AFS-3 extensible union proceed using the following algorithm: encode an XDR unsigned 32-bit integer (see Section 4.2 of ) containing the discrimant, XDR encode into temporary storage the implied leg (according to the type definition of the type specified for this discriminant value in the ext-union definition), and encode the implied leg using the XDR variable-length opaque specification (see Section 4.10 of ).

It is RECOMMENDED that decoding of an AFS-3 extensible union proceed using the following algorithm: XDR decode the 32-bit unsigned integer containing the discriminant; XDR decode the variable-length opaque blob into temporary storage; If this is a known discriminant: XDR decode the implied leg payload using the appropriate decoder for the discriminant value; Compare the length of the decoded XDR payload against the previously-decoded extensible union implied leg length. If the lengths do not match, then mark the union as failed to decode due to a length mismatch; However, if this is an unknown discriminant, then mark the union as failed to decode due to an unknown discriminant; XDR decoding continues at the current offset plus the length of the previously-decoded XDR variable-length opaque.

While the specific decoding algorithm used is left up to the implementor, error handling MUST be implemented as described in this section. When a decoder encounters an unknown discriminant, it MUST mark the discriminant as unknown, and SHOULD proceed to decoding the next element in the XDR stream by seeking past the length and implied leg fields. When a decoder encounters a length field that doesn't agree with the length expected for this discriminant, it MUST mark the discriminant as failed to decode due to a length mismatch, and SHOULD fail to decode the rest of the XDR octet stream. When the following conditions are satisfied: the max-unknown-leg-length value is specified in the extensible union type definition, and the discriminant value is unknown, and the implied leg length exceeds the max-unknown-leg-length specified in the type definition, then the decoder MUST mark the discriminant as failed to decode due to excessive length, and SHOULD fail to decode the rest of the XDR octet stream. In addition, an implementation MAY choose to mark a discriminant as failed to decode--and MAY fail to decode the rest of the XDR octet stream--when a known implied leg's length exceeds the max-unknown-leg-length in the extensible union type specification.

The author would like to thank Jeffrey Hutzelman for proposing standardization of a new XDR primitive type; and Matt Benjamin, Derrick Brashear, Andrew Deason, Steven Jenkins, Michael Meffie, Mark Vitale, and Simon Wilkinson for helping to refine the design of this extensible union type. Finally, the author would like to thank the attendees of the 2011 Pittsburgh AFS Hackathon for their review and comments regarding the -01 draft.

This memo includes no request to IANA.

This memo includes no request to the AFS Assigned Numbers Registrar.

Users of this extensible type should understand that any Rx XDR payload is only as secure as the security class bound to the Rx connection in question. This document merely standardizes a primitive type; it is up to the authors of standards defining new types (upon the "ext-union" primitive type) to ensure that the contents of their types are only marshalled over sufficiently-secure security classes. Decoders should take special care when encountering unexpected implied arm lengths. This could be indicative of serious errors, such as octet stream bit errors that were undetected by lower-layer checksums. At the very least, this error condition implies that the peers do not agree upon their ext-union type-to-discriminant mappings. It is RECOMMENDED that decoders treat this as a hard error and fail to decode the remainder of the XDR octet stream.