Asymmetric Extended Route Optimization (AERO)

The following sections specify the operation of IP over Asymmetric Extended Route Optimization (AERO) links:

R1 | |(X1->S1; X2->S2)| | default->R1 | | X1->C1 | | ASP A1 | | X2->C2 | +-------+------+ +--------+-------+ +------+-------+ | AERO Link | | X---+---+-------------------+-+----------------+---+---X | | | +-----+--------+ +----------+------+ +--------+-----+ |AERO Client C1| | AERO Proxy P1 | |AERO Client C2| | Nbr: S1 | |(Proxy Nbr Cache)| | Nbr: S2 | | default->S1 | +--------+--------+ | default->S2 | | ACP X1 | | | ACP X2 | +------+-------+ .--------+------. +-----+--------+ | (- Proxyed Clients -) | .-. `---------------' .-. ,-( _)-. ,-( _)-. .-(_ IP )-. +-------+ +-------+ .-(_ IP )-. (__ EUN )--|Host H1| |Host H2|--(__ EUN ) `-(______)-' +-------+ +-------+ `-(______)-' ]]> presents the AERO link reference model. In this model: AERO Relay R1 aggregates AERO Service Prefix (ASP) A1, acts as a default router for its associated Servers (S1 and S2), and connects the AERO link to the rest of the Internetwork. AERO Servers S1 and S2 associate with Relay R1 and also act as default routers for their associated Clients C1 and C2. AERO Clients C1 and C2 associate with Servers S1 and S2, respectively. They receive AERO Client Prefix (ACP) delegations X1 and X2, and also act as default routers for their associated physical or internal virtual EUNs. Simple hosts H1 and H2 attach to the EUNs served by Clients C1 and C2, respectively. AERO Proxy P1 provides proxy services for AERO Clients in secured enclaves that cannot associate directly with other AERO link neighbors. Each node on the AERO link maintains an AERO interface neighbor cache and an IP forwarding table the same as for any link. Although the figure shows a limited deployment, in common operational practice there may be many additional Relays, Servers, Clients and Proxies.

AERO Relays are standard IP routers that provide default forwarding services to AERO Servers. Each Relay also peers with Servers and other Relays in a dynamic routing protocol instance to discover the list of active ACPs (see ). Relays forward packets between neighbors connected to the same AERO link and also forward packets between the AERO link and the native Internetwork. Relays present the AERO link to the native Internetwork as a set of one or more AERO Service Prefixes (ASPs) and serve as a gateway between the AERO link and the Internetwork. Relays maintain tunnels with neighboring Servers, and maintain an IP forwarding table entry for each AERO Client Prefix (ACP). AERO Servers provide default forwarding services to AERO Clients. Each Server also peers with Relays in a dynamic routing protocol instance to advertise its list of associated ACPs (see ). Servers facilitate PD exchanges with Clients, where each delegated prefix becomes an ACP taken from an ASP. Servers forward packets between AERO interface neighbors, and maintain AERO interface neighbor cache entries for Relays. They also maintain both neighbor cache entries and IP forwarding table entries for each of their associated Clients. AERO Clients act as requesting routers to receive ACPs through PD exchanges with AERO Servers over the AERO link. Each Client can associate with a single Server or with multiple Servers, e.g., for fault tolerance, load balancing, etc. Each IPv6 Client receives at least a /64 IPv6 ACP, and may receive even shorter prefixes. Similarly, each IPv4 Client receives at least a /32 IPv4 ACP (i.e., a singleton IPv4 address), and may receive even shorter prefixes. Clients maintain an AERO interface neighbor cache entry for each of their associated Servers as well as for each of their correspondent Clients. AERO Proxies provide a conduit for AERO Clients connected to secured enclaves to associate with AERO link Servers. The Proxy can either be explicit or transparent. In the explicit case, the Client sends all of its control plane messages addressed to the Server to the link-layer address of the Proxy. In the transparent case, the Client sends all of its control plane messages to the Server's link-layer address and the Proxy intercepts them before they leave the secured enclave. In both cases, the Proxy forwards the Client's control and data plane messages to and from the Client's current Server(s). The Proxy may also discover a more direct route toward a target destination via AERO route optimization, in which case future outbound data packets would be forwarded via the more direct route. The Proxy function is specified in .

The AERO routing system comprises a private instance of the Border Gateway Protocol (BGP) that is coordinated between Relays and Servers and does not interact with either the public Internet BGP routing system or the native Internetwork routing system. Relays advertise only a small and unchanging set of ASPs to the native Internetwork routing system instead of the full dynamically changing set of ACPs. In a reference deployment, each AERO Server is configured as an Autonomous System Border Router (ASBR) for a stub Autonomous System (AS) using an AS Number (ASN) that is unique within the BGP instance, and each Server further uses eBGP to peer with one or more Relays but does not peer with other Servers. All Relays are members of the same hub AS using a common ASN, and use iBGP to maintain a consistent view of all active ACPs currently in service. Each Server maintains a working set of associated ACPs, and dynamically announces new ACPs and withdraws departed ACPs in its eBGP updates to Relays. Clients are expected to remain associated with their current Servers for extended timeframes, however Servers SHOULD selectively suppress updates for impatient Clients that repeatedly associate and disassociate with them in order to dampen routing churn. Each Relay configures a black-hole route for each of its ASPs. By black-holing the ASPs, the Relay will maintain forwarding table entries only for the ACPs that are currently active, and packets destined to all other ACPs will correctly incur Destination Unreachable messages due to the black hole route. Relays do not send eBGP updates for ACPs to Servers, but instead only originate a default route. In this way, Servers have only partial topology knowledge (i.e., they know only about the ACPs of their directly associated Clients) and they forward all other packets to Relays which have full topology knowledge. Scaling properties of the AERO routing system are limited by the number of BGP routes that can be carried by Relays. At the time of this writing, the global public Internet BGP routing system manages more than 500K routes with linear growth and no signs of router resource exhaustion . Network emulation studies have also shown that a single Relay can accommodate at least 1M dynamically changing BGP routes even on a lightweight virtual machine, i.e., and without requiring high-end dedicated router hardware. Therefore, assuming each Relay can carry 1M or more routes, this means that at least 1M Clients can be serviced by a single set of Relays. A means of increasing scaling would be to assign a different set of Relays for each set of ASPs. In that case, each Server still peers with one or more Relays, but the Server institutes route filters so that it only sends BGP updates to the specific set of Relays that aggregate the ASP. For example, if the ASP for the AERO link is 2001:db8::/32, a first set of Relays could service the ASP segment 2001:db8::/40, a second set of Relays could service 2001:db8:0100::/40, a third set could service 2001:db8:0200::/40, etc. Assuming up to 1K sets of Relays, the AERO routing system can then accommodate 1B or more ACPs with no additional overhead for Servers and Relays (for example, it should be possible to service 1B /64 ACPs taken from a /34 ASP and even more for shorter prefixes). In this way, each set of Relays services a specific set of ASPs that they advertise to the native Internetwork routing system, and each Server configures ASP-specific routes that list the correct set of Relays as next hops. This arrangement also allows for natural incremental deployment, and can support small scale initial deployments followed by dynamic deployment of additional Clients, Servers and Relays without disturbing the already-deployed base. Note that in an alternate routing arrangement each set of Relays could advertise an aggregated ASP for the link into the native Internetwork routing system even though each Relay services only smaller segments of the ASP. In that case, a Relay upon receiving a packet with a destination address covered by the ASP segment of another Relay can simply tunnel the packet to the other Relay. The tradeoff then is the penalty for Relay-to-Relay tunneling compared with reduced routing information in the native routing system. A full discussion of the BGP-based routing system used by AERO is found in .

AERO interface link-local address types include administratively-provisioned addresses and AERO addresses. Administratively-provisioned addresses are allocated from the range fe80::/96 and assigned to a Server's AERO interface. Administratively-provisioned addresses MUST be managed for uniqueness by the administrative authority for the AERO link. The address fe80:: is reserved as the IPv6 link-local Subnet Router Anycast address, and the address fe80::ffff:ffff is reserved as the "prefix-solicitation" address used by Clients to bootstrap AERO address autoconfiguration. These reserved addresses are therefore not available for general assignment. An AERO address is an IPv6 link-local address with an embedded prefix based on an ACP and associated with a Client's AERO interface. AERO addresses remain stable as the Client moves between topological locations, i.e., even if its link-layer addresses change. For IPv6, AERO addresses begin with the prefix fe80::/64 and include in the interface identifier (i.e., the lower 64 bits) a 64-bit prefix taken from one of the Client's IPv6 ACPs. For example, if the AERO Client receives the IPv6 ACP: 2001:db8:1000:2000::/56 it constructs its corresponding AERO addresses as: fe80::2001:db8:1000:2000 fe80::2001:db8:1000:2001 fe80::2001:db8:1000:2002 ... etc. ... fe80::2001:db8:1000:20ff For IPv4, AERO addresses are based on an IPv4-mapped IPv6 address formed from an IPv4 ACP and with a Prefix Length of 96 plus the ACP prefix length. For example, for the IPv4 ACP 192.0.2.32/28 the IPv4-mapped IPv6 ACP is: 0:0:0:0:0:FFFF:192.0.2.16/124 The Client then constructs its AERO addresses with the prefix fe80::/64 and with the lower 64 bits of the IPv4-mapped IPv6 address in the interface identifier as: fe80::FFFF:192.0.2.16 fe80::FFFF:192.0.2.17 fe80::FFFF:192.0.2.18 ... etc. ... fe80:FFFF:192.0.2.31 When the Server delegates ACPs to the Client, both the Server and Client use the lowest-numbered AERO address from the first ACP delegation as the "base" AERO address (for example, for the ACP 2001:db8:1000:2000::/56 the base AERO address is fe80::2001:db8:1000:2000). The Client then assigns the base AERO address to the AERO interface and uses it for the purpose of maintaining the neighbor cache entry. The Server likewise uses the AERO address as its index into the neighbor cache for this Client. If the Client has multiple AERO addresses (i.e., when there are multiple ACPs and/or ACPs with short prefix lengths), the Client originates ND messages using the base AERO address as the source address and accepts and responds to ND messages destined to any of its AERO addresses as equivalent to the base AERO address. In this way, the Client maintains a single neighbor cache entry that may be indexed by multiple AERO addresses. AERO addresses that embed an IPv6 prefix can be statelessly transformed into an IPv6 Subnet Router Anycast address. For example, for the AERO address fe80::2001:db8:2000:3000 the corresponding Subnet Router Anycast address is 2001:db8:2000:3000::. In the same way, for the IPv6 Subnet Router Anycast address 2001:db8:1:2:: the corresponding AERO address is fe80::2001:db8:1:2. In other words, the low-order 64 bits of an AERO address can be used as the high-order 64 bits of a Subnet Router Anycast address, and vice-versa. The IPv6 Subnet Router Anycast address corresponding to an AERO address is used as the destination address for forwarding route optimization control messages via a Relay acting as a standard IPv6 router. The source address used in these control messages is the AERO Server Subnet Router Anycast Address taken from a reserved Unique Local Address (ULA) prefix for the AERO link (for example, for the ULA prefix fd00:db8::/64 the AERO Server Subnet Router Anycast Address is fd00:db8::). See for further details.

AERO interfaces use encapsulation (see: ) to exchange packets with neighbors attached to the AERO link. AERO interfaces maintain a neighbor cache for tracking per-neighbor state the same as for any interface. AERO interfaces use ND messages including Neighbor Solicitation (NS), Neighbor Advertisement (NA), Router Solicitation (RS), Router Advertisement (RA) and Redirect for neighbor cache management. AERO interfaces use RS/RA messages with an embedded PD message (e.g., see: ). AERO interfaces include routing information in ND messages to support route optimization. AERO interface ND messages include one or more Source/Target Link-Layer Address Options (S/TLLAOs) formatted as shown in :

In this format: Type is set to '1' for SLLAO or '2' for TLLAO. Length is set to the constant value '5' (i.e., 5 units of 8 octets). X (proXy) is set to '1' in an S/TLLAO if the address corresponds to a Proxy; otherwise, X is set to '0'. Reserved is set to the value '0' on transmission and ignored on receipt. Interface ID is set to a 16-bit integer value corresponding to an underlying interface of the AERO node. The value 255 is reserved for Server-based route optimization (see: ). UDP Port Number and IP Address are set to the addresses used by the AERO node when it sends encapsulated packets over the specified underlying interface (or to '0' when the addresses are left unspecified). When UDP is not used as part of the encapsulation, UDP Port Number is set to '0'. When the encapsulation IP address family is IPv4, IP Address is formed as an IPv4-mapped IPv6 address as specified in . P(i) is a set of 64 Preference values that correspond to the 64 Differentiated Service Code Point (DSCP) values . Each P(i) is set to the value '0' ("disabled"), '1' ("low"), '2' ("medium") or '3' ("high") to indicate a QoS preference level for packet forwarding purposes. AERO interfaces may be configured over multiple underlying interface connections to underlying links. For example, common mobile handheld devices have both wireless local area network ("WLAN") and cellular wireless links. These links are typically used "one at a time" with low-cost WLAN preferred and highly-available cellular wireless as a standby. In a more complex example, aircraft frequently have many wireless data link types (e.g. satellite-based, cellular, terrestrial, air-to-air directional, etc.) with diverse performance and cost properties. A Client's underlying interfaces are classified as follows: Native interfaces connect to the open Internetwork, and have a global IP address that is reachable from any open Internetwork correspondent. NAT'ed interfaces connect to a closed network that is separated from the open Internetwork by a Network Address Translator (NAT). The NAT does not participate in any AERO control message signaling, but the AERO Server can issue AERO control messages on behalf of the Client. VPN'ed interfaces use security encapsulation over the Internetwork to a Virtual Private Network (VPN) gateway that also acts as an AERO Server. As with NAT'ed links, the AERO Server can issue control messages on behalf of the Client. Proxy'ed interfaces connect to a closed network that is separated from the open Internetwork by an AERO Proxy. Unlike NAT'ed and VPN'ed interfaces, the AERO Proxy (rather than the Server) can issue control message on behalf of the Client. Direct interfaces connect the Client directly to a peer without crossing any networked paths. An example is a line-of-sight link between a remote pilot and an unmanned aircraft. If a Client's multiple underlying interfaces are used "one at a time" (i.e., all other interfaces are in standby mode while one interface is active), then ND messages include only a single S/TLLAO with Interface ID set to a constant value. In that case, the Client would appear to have a single underlying interface but with a dynamically changing link-layer address. If the Client has multiple active underlying interfaces, then from the perspective of ND it would appear to have multiple link-layer addresses. In that case, ND messages MAY include multiple S/TLLAOs -- each with an Interface ID that corresponds to a specific underlying interface of the AERO node. When the Client includes an S/TLLAO for an underlying interface for which it is aware that there is a NAT or Proxy on the path to the Server, or when a node includes an S/TLLAO solely for the purpose of announcing new QoS preferences, the node sets both UDP Port Number and IP Address to 0 to indicate that the addresses are unspecified. When an ND message includes multiple S/TLLAOs, the first S/TLLAO MUST correspond to the AERO node's underlying interface used to transmit the message.

When a Relay enables an AERO interface, it first assigns an administratively-provisioned link-local address fe80::ID to the interface. Each fe80::ID address MUST be unique among all AERO nodes on the link. The Relay then engages in a dynamic routing protocol session with one or more Servers and all other Relays on the link (see: ), and advertises its assigned ASPs into the native Internetwork. Each Relay subsequently maintains an IP forwarding table entry for each active ACP covered by its ASP(s), and maintains neighbor cache entries for all Servers on the link.

When a Server enables an AERO interface, it assigns an administratively-provisioned link-local address fe80::ID the same as for Relays. The Server further configures a service to facilitate PD exchanges with AERO Clients. The Server maintains neighbor cache entries for one or more Relays on the link, and manages per-Client neighbor cache entries and IP forwarding table entries based on control message exchanges. Each Server also engages in a dynamic routing protocol with their neighboring Relays (see: ). When the Server receives an NS/RS message from a Client on the AERO interface it authenticates the message and returns an NA/RA message. The Server further provides a simple link-layer conduit between AERO interface neighbors. In particular, when a packet sent by a source Client arrives on the Server's AERO interface and is destined to another AERO node, the Server forwards the packet from within the AERO interface driver at the link layer without ever disturbing the network layer.

When a Client enables an AERO interface, it sends RS messages with PD "Solicit" options over an underlying interface using the prefix-solicitation address as the source network layer address and all-routers as the destination network layer address to obtain ACPs from one or more AERO Servers. Each Server processes the message and returns an RA message with a PD "Reply" option with the Server's link-layer address as the source and the base AERO address as the destination network layer addresses. In this way, the ND/PD control messages securely perform all autoconfiguration operations in a single request/response exchange. After the initial ND/PD message exchange, the Client can register additional underlying interfaces with the Server by sending an RS message over each underlying interface using its base AERO address as the source network layer address and without including a PD option. The Server will update its neighbor cache entry for the Client and return an RA message. The Client maintains a neighbor cache entry for each of its Servers and each of its active correspondent Clients. When the Client receives ND messages on the AERO interface it updates or creates neighbor cache entries, including link-layer address and QoS preferences.

When a Proxy enables an AERO interface, it maintains per-Client proxy neighbor cache entries based on control message exchanges. Proxies forward packets between their associated Clients and the Clients' associated Servers. When the Proxy receives an RS message from a Client in the secured enclave, it creates an incomplete proxy neighbor cache entry and forwards the message to a Server selected by the Client while using its own link-layer address as the source address. When the Server returns an RA message, the Proxy completes the proxy neighbor cache entry based on autoconfiguration information in the RA and forwards the RA to the Client while using its own link-layer address as the source address. The Client, Server and Proxy will then have the necessary state for managing the proxyed neighbor association.

Each AERO interface maintains a conceptual neighbor cache that includes an entry for each neighbor it communicates with on the AERO link, the same as for any IPv6 interface . AERO interface neighbor cache entries are said to be one of "permanent", "static", "proxy" or "dynamic". Permanent neighbor cache entries are created through explicit administrative action; they have no timeout values and remain in place until explicitly deleted. AERO Relays maintain permanent neighbor cache entries for Servers on the link, and AERO Servers maintain permanent neighbor cache entries for Relays. Each entry maintains the mapping between the neighbor's fe80::ID network-layer address and corresponding link-layer address. Static neighbor cache entries are created and maintained through ND/PD exchanges as specified in , and remain in place for durations bounded by ND/PD lifetimes. AERO Servers maintain static neighbor cache entries for each of their associated Clients, and AERO Clients maintain static neighbor cache entries for each of their associated Servers. Proxy neighbor cache entries are created and maintained by AERO Proxies by gleaning information from Client/Server ND/PD exchanges, and remain in place for durations bounded by ND/PD lifetimes. AERO Proxies maintain proxy neighbor cache entries for each of their associated Clients, and include pointers to the Client's current set of Servers. Dynamic neighbor cache entries are created or updated based on receipt of route optimization messages as specified in , and are garbage-collected when keepalive timers expire. AERO nodes maintain dynamic neighbor cache entries for each of their active correspondents with lifetimes based on ND messaging constants. When a target AERO node receives a valid NS message with an AERO source address, it returns an NA message and also creates or updates a dynamic neighbor cache entry for the source network-layer and link-layer addresses. The node then sets an "AcceptTime" variable in the neighbor cache entry to ACCEPT_TIME seconds and uses this value to determine whether packets received from the correspondent can be accepted. The node resets AcceptTime when it receives a new ND message, and otherwise decrements AcceptTime while no ND messages have been received. It is RECOMMENDED that ACCEPT_TIME be set to the default constant value 40 seconds to allow a 10 second window so that the AERO route optimization procedure can converge before AcceptTime decrements below FORWARD_TIME (see below). When a source AERO node receives a valid NA message with an AERO source address that matches its NS message, it creates or updates a dynamic neighbor cache entry for the target network-layer and link-layer addresses. The node then sets a "ForwardTime" variable in the neighbor cache entry to FORWARD_TIME seconds and uses this value to determine whether packets can be forwarded directly to the correspondent, i.e., instead of via a default route. The node resets ForwardTime when it receives a new NA, and otherwise decrements ForwardTime while no further NA messages have been received. It is RECOMMENDED that FORWARD_TIME be set to the default constant value 30 seconds to match the default REACHABLE_TIME value specified in . The node also sets a "MaxRetry" variable to MAX_RETRY to limit the number of keepalives sent when a correspondent may have gone unreachable. It is RECOMMENDED that MAX_RETRY be set to 3 the same as described for address resolution in Section 7.3.3 of . Different values for ACCEPT_TIME, FORWARD_TIME and MAX_RETRY MAY be administratively set, if necessary, to better match the AERO link's performance characteristics; however, if different values are chosen, all nodes on the link MUST consistently configure the same values. Most importantly, ACCEPT_TIME SHOULD be set to a value that is sufficiently longer than FORWARD_TIME to allow the AERO route optimization procedure to converge. When there may be a NAT between the Client and the Server, or if the path from the Client to the Server should be tested for reachability, the Client can send periodic RS messages to the Server without a PD option to receive RA replies. The RS/RA messaging will keep NAT state alive and test Server reachability without disturbing the PD service.

IP packets enter a node's AERO interface either from the network layer (i.e., from a local application or the IP forwarding system) or from the link layer (i.e., from the AERO tunnel virtual link). Packets that enter the AERO interface from the network layer are encapsulated and forwarded into the AERO link, i.e., they are tunneled to an AERO interface neighbor. Packets that enter the AERO interface from the link layer are either re-admitted into the AERO link or forwarded to the network layer where they are subject to either local delivery or IP forwarding. In all cases, the AERO interface itself MUST NOT decrement the network layer TTL/Hop-count since its forwarding actions occur below the network layer. AERO interfaces may have multiple underlying interfaces and/or neighbor cache entries for neighbors with multiple Interface ID registrations (see ). The AERO node uses each packet's DSCP value to select an outgoing underlying interface based on the node's own QoS preferences, and also to select a destination link-layer address based on the neighbor's underlying interface with the highest preference. If multiple outgoing interfaces and/or neighbor interfaces have a preference of "high", the AERO node sends one copy of the packet via each of the (outgoing / neighbor) interface pairs; otherwise, the node sends a single copy of the packet via the interface with the highest preference. AERO nodes keep track of which underlying interfaces are currently "reachable" or "unreachable", and only use "reachable" interfaces for forwarding purposes. The following sections discuss the AERO interface forwarding algorithms for Clients, Proxies, Servers and Relays. In the following discussion, a packet's destination address is said to "match" if it is a non-link-local address with a prefix covered by an ASP/ACP, or if it is an AERO address that embeds an ACP, or if it is the same as an administratively-provisioned link-local address.

When an IP packet enters a Client's AERO interface from the network layer the Client searches for a dynamic neighbor cache entry that matches the destination. If there is a match, the Client uses one or more "reachable" link-layer addresses in the entry as the link-layer addresses for encapsulation and admits the packet into the AERO link. Otherwise, the Client uses the link-layer address in a static neighbor cache entry for a Server as the encapsulation address (noting that there may be a Proxy on the path to the real Server). When an IP packet enters a Client's AERO interface from the link-layer, if the destination matches one of the Client's ACPs or link-local addresses the Client decapsulates the packet and delivers it to the network layer. Otherwise, the Client drops the packet and MAY return a network-layer ICMP Destination Unreachable message subject to rate limiting (see: ).

When the Proxy receives a packet from a Client within the secured enclave, the Proxy searches for a dynamic neighbor cache entry that matches the destination. If there is a match, the Proxy uses one or more "reachable" link-layer addresses in the entry as the link-layer addresses for encapsulation and admits the packet into the AERO link. Otherwise, the Proxy uses the link-layer address for one of the Client's Servers as the encapsulation address. When the Proxy receives a packet from an AERO interface neighbor, it searches for a proxy neighbor cache entry for a Client within the secured enclave that matches the destination. If there is a match, the Proxy forwards the packet to the Client. Otherwise, the Proxy returns the packet to the neighbor, i.e., by reversing the source and destination link-layer addresses.

When an IP packet enters a Server's AERO interface from the network layer, the Server searches for a static neighbor cache entry for a Client that matches the destination. If there is a match, the Server uses one or more link-layer addresses in the entry as the link-layer addresses for encapsulation and admits the packet into the AERO link. Otherwise, the Server uses the link-layer address in a permanent neighbor cache entry for a Relay (selected through longest-prefix match) as the link-layer address for encapsulation. When an IP packet enters a Server's AERO interface from the link layer, the Server processes the packet according to the network-layer destination address as follows: if the destination matches one of the Server's own addresses the Server decapsulates the packet and forwards it to the network layer for local delivery. else, if the destination matches a static neighbor cache entry for a Client the Server first determines whether the neighbor is the same as the one it received the packet from. If so, the Server drops the packet silently to avoid looping; otherwise, the Server uses the neighbor's link-layer address(es) as the destination for encapsulation and re-admits the packet into the AERO link. else, the Server uses the link-layer address in a neighbor cache entry for a Relay (selected through longest-prefix match) as the link-layer address for encapsulation.

When an IP packet enters a Relay's AERO interface from the network layer, the Relay searches its IP forwarding table for an ACP entry that matches the destination the same as for any IP router. If there is a match, the Relay uses the link-layer address in the corresponding neighbor cache entry as the link-layer address for encapsulation and forwards the packet to the AERO neighbor. Otherwise, the Relay drops the packet and returns a network-layer ICMP Destination Unreachable message subject to rate limiting (see: ). When an IP packet enters a Relay's AERO interface from the link-layer, the Relay processes the packet as follows: if the destination does not match an ASP, or if the destination matches one of the Relay's own addresses, the Relay decapsulates the packet and forwards it to the network layer where it will be subject to either IP forwarding or local delivery. else, if the destination matches an ACP entry in the IP forwarding table the Relay first determines whether the neighbor is the same as the one it received the packet from. If so the Relay MUST drop the packet silently to avoid looping; otherwise, the Relay uses the neighbor's link-layer address as the destination for encapsulation and re-admits the packet into the AERO link. else, the Relay drops the packet and returns an ICMP Destination Unreachable message subject to rate limiting (see: ).

When an AERO node receives a return packet such as generated by an AERO Proxy (see ), it proceeds according to the AERO link trust basis. Namely, the return packets have the same trust profile as for link-layer Destination Unreachable messages. If the node has sufficient trust basis to accept link-layer Destination Unreachable messages, it can then process the return packet as described in the following paragraph. Otherwise, the node SHOULD drop the packet and treat it as an indication that a path may be failing, and MAY use NUD to test the path for reachability. If the node has sufficient trust basis to accept return packets, it searches for a dynamic neighbor cache entry that matches the destination. If there is a match, the neighbor marks the corresponding link-layer address as "unreachable", selects the next-highest priority "reachable" link-layer address in the entry as the link-layer address for encapsulation then (re)admits the packet into the AERO link. If there are no "reachable" link-layer addresses, the neighbor instead sets FowardTime in the dynamic neighbor cache entry to 0. If the source address corresponds to one of the neighbor's own addresses, the neighbor also forwards the packet to the corresponding Server; otherwise, it drops the packet.

AERO interfaces encapsulate IP packets according to whether they are entering the AERO interface from the network layer or if they are being re-admitted into the same AERO link they arrived on. This latter form of encapsulation is known as "re-encapsulation". The AERO interface encapsulates packets per the Generic UDP Encapsulation (GUE) procedures in , or through an alternate encapsulation format (see: ). For packets entering the AERO interface from the network layer, the AERO interface copies the "TTL/Hop Limit", "Type of Service/Traffic Class" , "Flow Label" (for IPv6) and "Congestion Experienced" values in the packet's IP header into the corresponding fields in the encapsulation IP header. For packets undergoing re-encapsulation, the AERO interface instead copies these values from the original encapsulation IP header into the new encapsulation header, i.e., the values are transferred between encapsulation headers and *not* copied from the encapsulated packet's network-layer header. (Note especially that by copying the TTL/Hop Limit between encapsulation headers the value will eventually decrement to 0 if there is a (temporary) routing loop.) For IPv4 encapsulation/re-encapsulation, the AERO interface sets the DF bit as discussed in . When GUE encapsulation is used, the AERO interface next sets the UDP source port to a constant value that it will use in each successive packet it sends, and sets the UDP length field to the length of the encapsulated packet plus 8 bytes for the UDP header itself plus the length of the GUE header (or 0 if GUE direct IP encapsulation is used). For packets sent to a Server or Relay, the AERO interface sets the UDP destination port to 8060, i.e., the IANA-registered port number for AERO. For packets sent to a Client, the AERO interface sets the UDP destination port to the port value stored in the neighbor cache entry for this Client. The AERO interface then either includes or omits the UDP checksum according to the GUE specification. Clients normally use the IP address of the underlying interface as the encapsulation source address. If the underlying interface does not have an IP address, however, the Client uses an IP address taken from an ACP as the encapsulation source address (assuming the node has some way of injecting the ACP into the underlying network routing system). For IPv6 addresses, the Client normally uses the ACP Subnet Router Anycast address . Encapsulation between Servers and Relays can use standard mechanisms such as Generic Routing Encapsulation (GRE) and IPSec so that Relays can be standard IP routers. The encapsulation format used for Server-to-Client and Client-to-Client tunneling can therefore be different than that used for Server-to-Relay tunneling.

AERO interfaces decapsulate packets destined either to the AERO node itself or to a destination reached via an interface other than the AERO interface the packet was received on. Decapsulation is per the procedures specified for the appropriate encapsulation format.

AERO nodes employ simple data origin authentication procedures for encapsulated packets they receive from other nodes on the AERO link. In particular: AERO Relays and Servers accept encapsulated packets with a link-layer source address that matches a permanent neighbor cache entry. AERO Servers accept authentic encapsulated ND messages from Clients, and create or update a static neighbor cache entry for the Client based on the specific message type. AERO Clients and Servers accept encapsulated packets if there is a static neighbor cache entry with a link-layer address that matches the packet's link-layer source address. AERO Clients and Servers accept encapsulated packets if there is a dynamic neighbor cache entry with an AERO address that matches the packet's network-layer source address, with a link-layer address that matches the packet's link-layer source address, and with a non-zero AcceptTime. AERO Proxies accept encapsulated packets if there is a proxy neighbor cache entry that matches the packet's network-layer destination address (i.e., the address of the Client) and link-layer source address (i.e., the address of one of the Client's Servers). When the proxy is configured to accept packets originating from any address in the open Internetwork however (e.g., from another Proxy), it omits the source address check. Note that this simple data origin authentication is effective in environments in which link-layer addresses cannot be spoofed. In other environments, each AERO message must include a signature that the recipient can use to authenticate the message origin, e.g., as for common VPN systems such as OpenVPN . In environments where end systems use end-to-end security, however, it may be sufficient to require signatures only for ND and ICMP control plane messages and omit signatures for data plane messages.

The AERO interface is the node's attachment to the AERO link. The AERO interface acts as a tunnel ingress when it sends a packet to an AERO link neighbor and as a tunnel egress when it receives a packet from an AERO link neighbor. AERO interfaces observe the packet sizing considerations for tunnels discussed in and as specified below. The Internet Protocol expects that IP packets will either be delivered to the destination or a suitable Packet Too Big (PTB) message returned to support the process known as IP Path MTU Discovery (PMTUD) . However, PTB messages may be crafted for malicious purposes such as denial of service, or lost in the network . This can be especially problematic for tunnels, where a condition known as a PMTUD "black hole" can result. For these reasons, AERO interfaces employ operational procedures that avoid interactions with PMTUD, including the use of fragmentation when necessary. AERO interfaces observe two different types of fragmentation. Source fragmentation occurs when the AERO interface (acting as a tunnel ingress) fragments the encapsulated packet into multiple fragments before admitting each fragment into the tunnel. Network fragmentation occurs when an encapsulated packet admitted into the tunnel by the ingress is fragmented by an IPv4 router on the path to the egress. Note that a packet that incurs source fragmentation may also incur network fragmentation. IPv6 specifies a minimum link Maximum Transmission Unit (MTU) of 1280 bytes . Although IPv4 specifies a smaller minimum link MTU of 68 bytes , AERO interfaces also observe the IPv6 minimum for IPv4 even if encapsulated packets may incur network fragmentation. IPv6 specifies a minimum Maximum Reassembly Unit (MRU) of 1500 bytes , while the minimum MRU for IPv4 is only 576 bytes (note that common IPv6 over IPv4 tunnels already assume a larger MRU than the IPv4 minimum). AERO interfaces therefore configure an MTU that MUST NOT be smaller than 1280 bytes, MUST NOT be larger than the minimum MRU among all nodes on the AERO link minus the encapsulation overhead ("ENCAPS"), and SHOULD NOT be smaller than 1500 bytes. AERO interfaces also configure a Maximum Segment Unit (MSU) as the maximum-sized encapsulated packet that the ingress can inject into the tunnel without source fragmentation. The MSU value MUST NOT be larger than (MTU+ENCAPS) and MUST NOT be larger than 1280 bytes unless there is operational assurance that a larger size can traverse the link along all paths. All AERO nodes MUST configure the same MTU/MSU values for reasons cited in ; in particular, multicast support requires a common MTU value among all nodes on the link. All AERO nodes MUST configure an MRU large enough to reassemble packets up to (MTU+ENCAPS) bytes in length; nodes that cannot configure a large-enough MRU MUST NOT enable an AERO interface. The network layer proceeds as follow when it presents an IP packet to the AERO interface. For each IPv4 packet that is larger than the AERO interface MTU and with the DF bit set to 0, the network layer uses IPv4 fragmentation to break the packet into a minimum number of non-overlapping fragments where the first fragment is no larger than the MTU and the remaining fragments are no larger than the first. For all other IP packets, if the packet is larger than the AERO interface MTU, the network layer drops the packet and returns a PTB message to the original source. Otherwise, the network layer admits each IP packet or fragment into the AERO interface. For each IP packet admitted into the AERO interface, the interface (acting as a tunnel ingress) encapsulates the packet. If the encapsulated packet is larger than the AERO interface MSU the ingress source-fragments the encapsulated packet into a minimum number of non-overlapping fragments where the first fragment is no larger than the MSU and the remaining fragments are no larger than the first. The ingress then admits each encapsulated packet or fragment into the tunnel, and for IPv4 sets the DF bit to 0 in the IP encapsulation header in case any network fragmentation is necessary. The encapsulated packets will be delivered to the egress, which reassembles them into a whole packet if necessary. Several factors must be considered when fragmentation is needed. For AERO links over IPv4, the IP ID field is only 16 bits in length, meaning that fragmentation at high data rates could result in data corruption due to reassembly misassociations . For AERO links over both IPv4 and IPv6, studies have also shown that IP fragments are dropped unconditionally over some network paths [I-D.taylor-v6ops-fragdrop]. In environments where IP fragmentation issues could result in operational problems, the ingress SHOULD employ intermediate-layer source fragmentation (see: and ) before appending the outer encapsulation headers to each fragment. Since the encapsulation fragment header reduces the room available for packet data, but the original source has no way to control its insertion, the ingress MUST include the fragment header length in the ENCAPS length even for packets in which the header is absent.

When an AERO node admits encapsulated packets into the AERO interface, it may receive link-layer or network-layer error indications. A link-layer error indication is an ICMP error message generated by a router in the underlying network on the path to the neighbor or by the neighbor itself. The message includes an IP header with the address of the node that generated the error as the source address and with the link-layer address of the AERO node as the destination address. The IP header is followed by an ICMP header that includes an error Type, Code and Checksum. Valid type values include "Destination Unreachable", "Time Exceeded" and "Parameter Problem" . (AERO interfaces ignore all link-layer IPv4 "Fragmentation Needed" and IPv6 "Packet Too Big" messages since they only emit packets that are guaranteed to be no larger than the IP minimum link MTU as discussed in .) The ICMP header is followed by the leading portion of the packet that generated the error, also known as the "packet-in-error". For ICMPv6, specifies that the packet-in-error includes: "As much of invoking packet as possible without the ICMPv6 packet exceeding the minimum IPv6 MTU" (i.e., no more than 1280 bytes). For ICMPv4, specifies that the packet-in-error includes: "Internet Header + 64 bits of Original Data Datagram", however Section 4.3.2.3 updates this specification by stating: "the ICMP datagram SHOULD contain as much of the original datagram as possible without the length of the ICMP datagram exceeding 576 bytes". The link-layer error message format is shown in (where, "L2" and "L3" refer to link-layer and network-layer, respectively):

The AERO node rules for processing these link-layer error messages are as follows: When an AERO node receives a link-layer Parameter Problem message, it processes the message the same as described as for ordinary ICMP errors in the normative references . When an AERO node receives persistent link-layer Time Exceeded messages, the IP ID field may be wrapping before earlier fragments awaiting reassembly have been processed. In that case, the node SHOULD begin including integrity checks and/or institute rate limits for subsequent packets. When an AERO node receives persistent link-layer Destination Unreachable messages in response to encapsulated packets that it sends to one of its dynamic neighbor correspondents, the node SHOULD process the message as an indication that a path may be failing, and MAY initiate NUD over that path. If it receives Destination Unreachable messages on many or all paths, the node SHOULD set ForwardTime for the corresponding dynamic neighbor cache entry to 0 and allow future packets destined to the correspondent to flow through a default route. When an AERO Client receives persistent link-layer Destination Unreachable messages in response to encapsulated packets that it sends to one of its static neighbor Servers, the Client SHOULD math the path as unusable and use another path. If it receives Destination Unreachable messages on many or all paths, the Client SHOULD associate with a new Server and send a PD "Release" message to the old Server as specified in . When an AERO Server receives persistent link-layer Destination Unreachable messages in response to encapsulated packets that it sends to one of its static neighbor Clients, the Server SHOULD mark the path as unusable and use another path. If it receives Destination Unreachable messages on multiple paths, the Server should take no further actions unless it receives a PD "Release" message or if the PD lifetime expires. In that case, the Server MUST release the Client's delegated ACP, withdraw the ACP from the AERO routing system and delete the neighbor cache entry. When an AERO Relay or Server receives link-layer Destination Unreachable messages in response to an encapsulated packet that it sends to one of its permanent neighbors, it treats the messages as an indication that the path to the neighbor may be failing. However, the dynamic routing protocol should soon reconverge and correct the temporary outage. When an AERO Relay receives a packet for which the network-layer destination address is covered by an ASP, if there is no more-specific routing information for the destination the Relay drops the packet and returns a network-layer Destination Unreachable message subject to rate limiting. The Relay writes the network-layer source address of the original packet as the destination address and uses one of its non link-local addresses as the source address of the message. When an AERO node receives an encapsulated packet for which the reassembly buffer it too small, it drops the packet and returns a network-layer Packet Too Big (PTB) message. The node first writes the MRU value into the PTB message MTU field, writes the network-layer source address of the original packet as the destination address of the message and determines the next hop to the destination. If the next hop is reached via the AERO interface, the node uses the IPv6 address "::" or the IPv4 address "0.0.0.0" as the source address of the message, then encapsulates the message and forwards it to the next hop within the AERO interface. Otherwise, the node uses one of its non link-local addresses as the source address of the message and forwards it via a link outside the AERO interface. When an AERO node receives any network-layer error message via the AERO interface, it examines the network-layer destination address. If the next hop toward the destination is via the AERO interface, the node re-encapsulates and forwards the message to the next hop within the AERO interface. Otherwise, if the network-layer source address is the IPv6 address "::" or the IPv4 address "0.0.0.0", the node writes one of its non link-local addresses as the source address, recalculates the IP and/or ICMP checksums then forwards the message via a link outside the AERO interface.

AERO Router Discovery, Prefix Delegation and Autoconfiguration are coordinated as discussed in the following Sections.

Each AERO Server configures a PD service to facilitate Client requests. Each Server is provisioned with a database of ACP-to-Client ID mappings for all Clients enrolled in the AERO system, as well as any information necessary to authenticate each Client. The Client database is maintained by a central administrative authority for the AERO link and securely distributed to all Servers, e.g., via the Lightweight Directory Access Protocol (LDAP) , via static configuration, etc. Therefore, no Server-to-Server PD state synchronization is necessary, and Clients can optionally hold separate PDs for the same ACPs from multiple Servers. In this way, Clients can associate with multiple Servers, and can receive new PDs from new Servers before releasing PDs received from existing Servers. This provides the Client with a natural fault-tolerance and/or load balancing profile. AERO Clients and Servers use ND messages to maintain neighbor cache entries. AERO Servers configure their AERO interfaces as advertising interfaces, and therefore send unicast RA messages with configuration information in response to a Client's RS message. The RS/RA messaging is conducted in the same fashion as specified in . AERO Clients and Servers include PD messages as options in the RS/RA messages they exchange (see: ). Client-initiated PD options are included in RS messages, and Server-initiated PD options are included in RA messages. The unified ND/PD messages are exchanged between Client and Server according to the prefix management schedule determined by the PD service. The unified messages can be protected using SEcure Neighbor Discovery (SEND) . On Some AERO links, PD arrangements may be through some out-of-band service such as network management, static configuration, etc. In those cases, AERO nodes can use simple RS/RA message exchanges with no explicit PD options. Instead, the RS/RA messages use AERO addresses as a means of representing the delegated prefixes, e.g., if a message includes a source address of "fe80::2001:db8:1:2" then the recipient can infer that the sender holds the prefix delegation "2001:db8:1:2::/64". The following sections specify the Client and Server behavior.

AERO Clients discover the link-layer addresses of AERO Servers via static configuration (e.g., from a flat-file map of Server addresses and locations), or through an automated means such as Domain Name System (DNS) name resolution . In the absence of other information, the Client resolves the DNS Fully-Qualified Domain Name (FQDN) "linkupnetworks.[domainname]" where "linkupnetworks" is a constant text string and "[domainname]" is a DNS suffix for the Client's underlying interface (e.g., "example.com"). After discovering the link-layer addresses, the Client associates with one or more of the corresponding Servers. To associate with a Server, the Client acts as a requesting router to request ACPs through a combined ND/PD message exchange. The Client includes a PD "Solicit" message as an ND option in an RS message with the prefix-solicitation address as the IPv6 source address, all-routers multicast as the IPv6 destination address, the address of the Client's underlying interface as the link-layer source address and the link-layer address of the Server as the link-layer destination address. (If the Client's underlying interface does not have an IP address, the Client can use the ACP's Subnet Router Anycast address as the link-layer source address.) The Client next includes a "Client Identifier" and an "IA_PD" (i.e., prefix request) code in the PD "Solicit" message. If the Client is pre-provisioned with ACPs associated with the AERO service, it MAY also include the ACPs in the "IA_PD" option to indicate its preferences to the Server. The Client finally includes any additional PD codes (e.g., "Rapid Commit"). The Client next includes one or more SLLAOs in the RS message formatted as described in to register its link-layer address(es) with the Server. The first SLLAO MUST correspond to the underlying interface over which the Client will send the RS message. The Client MAY include additional SLLAOs specific to other underlying interfaces, but if so it MUST have assurance that there will be no NATs or Proxies on the paths to the Server via those interfaces. (Otherwise, the Client can register additional link-layer addresses with the Server by sending subsequent NS/RS messages via different underlying interfaces after the initial RS/RA exchange). The Client then sends the RS message to the AERO Server and waits for an RA message reply (see ) while retrying MAX_RETRY times until an RA is received. If no RA is received, or if it receives an RA with Router Lifetime set to 0 and/or a "Reply" with no ACPs, the Client SHOULD discontinue autoconfiguration attempts through this Server and try another Server. Otherwise, the Client processes the ACPs in the embedded "Reply" message. Next, the Client creates a static neighbor cache entry with the Server's link-local address as the network-layer address and the Server's encapsulation source address as the link-layer address. The Client then autoconfigures AERO addresses for each of the delegated ACPs and assigns them to the AERO interface. The Client next examines the P bit in the RA message flags field . If the P bit value was 1, the Client assumes that there is a NAT or Proxy on the path to the Server via the interface over which it sent the RS message. In that case, the Client sets UDP Port Number and IP Address to 0 in the S/TLLAOs of any subsequent ND messages it sends to the Server over that link. The Client also caches any ASPs included in Route Information Options (RIOs) as ASPs to associate with the AERO link, and assigns the MTU/MSU values in the MTU options to its AERO interface while configuring an appropriate MRU. This configuration information applies to the AERO link as a whole, and all AERO nodes will receive the same values. Following autoconfiguration, the Client sub-delegates the ACPs to its attached EUNs and/or the Client's own internal virtual interfaces as described in . The Client subsequently maintains its ACP delegations through each of its Servers by sending RS "Renew", "Rebind", and/or "Release" messages. The Server will in turn send RA "Reply" messages. After the Client registers its Interface IDs and their associated UDP/IP addresses and 'P(i)' values, it may wish to change one or more Interface ID registrations, e.g., if an underlying interface changes address or becomes unavailable, if QoS preferences change, etc. To do so, the Client prepares an unsolicited NA message to send over any available underlying interface. The source and target address of the NA message are set to the Client's AERO address, and the destination address is set to all-nodes multicast. The NA MUST include a TLLAO specific to the selected available underlying interface as the first TLLAO and MAY include any additional TLLAOs specific to other underlying interfaces. The Client includes fresh 'P(i)' values in each TLLAO to update the Server's neighbor cache entry. If the Client wishes to update 'P(i)' values without updating the link-layer address, it sets the UDP Port Number and IP Address fields to 0. If the Client wishes to disable the interface, it sets all 'P(i)' values to '0' ("disabled"). If the Client wishes to discontinue use of a Server it issues an RS "Release" message. When the Server processes the message, it releases the ACP, deletes its neighbor cache entry for the Client, withdraws the IP route from the routing system and returns an RA "Reply".

AERO Servers act as IPv6 routers and support a PD service on their AERO links. AERO Servers arrange to add their encapsulation layer IP addresses (i.e., their link-layer addresses) to a static map of Server addresses for the link and/or the DNS resource records for the FQDN "linkupnetworks.[domainname]" before entering service. When an AERO Server receives a prospective Client's RS "Solicit" message on its AERO interface, and the Server is too busy, it SHOULD return an immediate RA "Reply" message with no ACPs and with Router Lifetime set to 0. Otherwise, the Server authenticates the RS message and processes the embedded "Solicit" option. The Server first determines the correct ACPs to delegate to the Client by searching the Client database. When the Server delegates the ACPs, it also creates an IP forwarding table entry for each ACP so that the AERO BGP-based routing system will propagate the ACPs to the Relays that aggregate the corresponding ASP (see: ). Next, the Server prepares an RA "Reply" message that includes the delegated ACPs. For IPv4 ACPs, the ACP is in IPv4-mapped IPv6 address format and with prefix length set as specified in . The Server then prepares an RA "Reply" message using its link-local address (i.e., fe80::ID) as the network-layer source address, the Client's base AERO address from the first ACP as the network-layer destination address, the Server's link-layer address as the source link-layer address, and the source link-layer address of the RS message as the destination link-layer address. The Server next sets the P flag in the RA message flags field to 1 if the source link-layer address in the RS message was different than the address in the first SLLAO to indicate that there is a NAT or Proxy on the path; otherwise it sets P to 0. The Server then includes one or more RIOs that encode the ASPs for the AERO link. The Server also includes two MTU options - the first MTU option includes the MTU for the link and the second MTU option includes the MSU for the link (see ). The Server finally sends the RA "Reply" message to the Client. The Server next creates a static neighbor cache entry for the Client using the base AERO address as the network-layer address and with lifetime set to no more than the smallest PD lifetime. Next, the Server updates the neighbor cache entry link-layer address(es) by recording the information in each SLLAO option indexed by the Interface ID and including the UDP port number, IP address and P(i) values. For the first SLLAO in the list, however, the Server records the actual encapsulation source UDP and IP addresses instead of those that appear in the SLLAO in case there was a NAT or Proxy in the path. After the initial RS/RA exchange, the AERO Server maintains the neighbor cache entry for the Client until the PD lifetimes expire. If the Client issues an RS "Renew", the Server extends the PD lifetimes. If the Client issues an RS "Release", or if the Client does not issue a "Renew" before the lifetime expires, the Server deletes the neighbor cache entry for the Client and withdraws the IP routes from the AERO routing system. The Server processes these and any other Client PD messages, and returns an RA "Reply". The Server may also issue an unsolicited RA "Reconfigure" message to inform the Client that it needs to renegotiate its PDs.

When DHCPv6 is used as the PD service, AERO Clients and Servers are always on the same link (i.e., the AERO link) from the perspective of DHCPv6. However, in some implementations the DHCPv6 server and ND function may be located in separate modules. In that case, the Server's AERO interface driver module can act as a Lightweight DHCPv6 Relay Agent (LDRA) to relay PD messages to and from the DHCPv6 server module. When the LDRA receives an authentic RS message, it extracts the PD message option and wraps it in IPv6/UDP headers. It sets the IPv6 source address to the source address of the RS message, sets the IPv6 destination address to 'All_DHCP_Relay_Agents_and_Servers' and sets the UDP fields to values that will be understood by the DHCPv6 server. The LDRA then wraps the message in a Relay-Forward message header and includes an Interface-ID option that includes enough information to allow the LDRA to forward the resulting Reply message back to the Client (e.g., the Client's link-layer addresses, a security association identifier, etc.). The LDRA also wraps the information in all of the SLLAO options from the RS message into the Interface-ID option, then forwards the message to the DHCPv6 server. When the DHCPv6 server prepares a Reply message, it wraps the message in a Relay-Reply message and echoes the Interface-ID option. The DHCPv6 server then delivers the Relay-Reply message to the LDRA, which discards the Relay-Reply wrapper and IPv6/UDP headers, then delivers the DHCPv6 message to be wrapped into an RA response to the Client. The Server uses the information in the Interface ID option to prepare the RA message and to cache the link-layer addresses taken from the SLLAOs echoed in the Interface-ID option.

When a source Client forwards packets to a prospective correspondent Client within the same AERO link domain (i.e., one for which the packet's destination address is covered by an ASP), the source Client MAY initiate an AERO link route optimization procedure on behalf of any of its native underlying interfaces. The procedure is based on an exchange of IPv6 ND messages using a chain of AERO Servers and Relays as a trust basis. Although the Client is responsible for initiating route optimization, the Server is the policy enforcement point that determines whether route optimization is permitted. For example, on some AERO links route optimization would allow traffic to circumvent critical network-based traffic inspection points. In those cases, the Server can simply discard any route optimization messages instead of forwarding them. The following sections specify the AERO link route optimization procedure.

depicts the AERO link route optimization reference operational scenario, using IPv6 addressing as the example (while not shown, a corresponding example for IPv4 addressing can be easily constructed). The figure shows an AERO Relay ('R1'), two AERO Servers ('S1', 'S2'), two AERO Clients ('C1', 'C2') and two ordinary IPv6 hosts ('H1', 'H2'):

In , Relay ('R1') assigns the administratively-provisioned link-local address fe80::1 to its AERO interface with link-layer address L2(R1), Server ('S1') assigns the address fe80::2 with link-layer address L2(S1), and Server ('S2') assigns the address fe80::3 with link-layer address L2(S2). Servers ('S1') and ('S2') next arrange to add their link-layer addresses to a published list of valid Servers for the AERO link. AERO Client ('C1') receives the ACP 2001:db8:0::/48 in an ND/PD exchange via AERO Server ('S1') then assigns the address fe80::2001:db8:0:0 to its AERO interface with link-layer address L2(C1). Client ('C1') configures a default route and neighbor cache entry via the AERO interface with next-hop address fe80::2 and link-layer address L2(S1), then sub-delegates the ACP to its attached EUNs. IPv6 host ('H1') connects to the EUN, and configures the address 2001:db8:0::1. AERO Client ('C2') receives the ACP 2001:db8:1::/48 in an ND/PD exchange via AERO Server ('S2') then assigns the address fe80::2001:db8:1:0 to its AERO interface with link-layer address L2(C2). Client ('C2') configures a default route and neighbor cache entry via the AERO interface with next-hop address fe80::3 and link-layer address L2(S2), then sub-delegates the ACP to its attached EUNs. IPv6 host ('H2') connects to the EUN, and configures the address 2001:db8:1::1.

Again, with reference to , when source host ('H1') sends a packet to destination host ('H2'), the packet is first forwarded over the source host's attached EUN to Client ('C1'). Client ('C1') then forwards the packet via its AERO interface to Server ('S1') and also sends an NS message toward Client ('C2') via Server ('S1'). Server ('S1') then forwards both the packet and the NS message out the same AERO interface toward Client ('C2') via Relay ('R1'). When Relay ('R1') receives the packet and NS message, it consults its forwarding table to discover Server ('S2') as the next hop toward Client ('C2'). Relay ('R1') then forwards both the packet and the NS message to Server ('S2'), which then forwards them to Client ('C2'). After Client ('C2') receives the NS message, it process the message and creates or updates a dynamic neighbor cache entry for Client ('C1'), then sends the NA response to the link-layer address of Server ('S2'). When Server ('S2') receives the NA message it forwards the NA on to Relay ('R1'), which forwards the message on to Server ('S1') which forwards the message on to Client ('C1'). After Client ('C1') receives the NA message, it processes the message and creates or updates a dynamic neighbor cache entry for Client ('C2'). Thereafter, forwarding of packets from Client ('C1') to Client ('C2') without involving any intermediate nodes is enabled. The mechanisms that support this exchange are specified in the following sections.

When a Client forwards a packet with a source address from one of its ACPs toward a destination address covered by an ASP (i.e., toward another AERO Client connected to the same AERO link), the source Client MAY send an NS message forward toward the destination Client via the Server. In the reference operational scenario, when Client ('C1') forwards a packet toward Client ('C2'), it MAY also send an NS message forward toward Client ('C2'), subject to rate limiting (see Section 8.2 of ). Client ('C1') prepares the NS message as follows: the link-layer source address is set to 'L2(C1)' (i.e., the link-layer address of Client ('C1')). the link-layer destination address is set to 'L2(S1)' (i.e., the link-layer address of Server ('S1')). the network-layer source address is set to fe80::2001:db8:0:0 (i.e., the base AERO address of Client ('C1')). the network-layer destination address is set to the AERO address corresponding to the destination address of Client ('C2'). the Type is set to 135. the Target Address is set to the destination address of the packet that triggered route optimization. the message includes one or more SLLAOs set to appropriate values for Client ('C1')'s native underlying interfaces. the message includes one or more RIOs that include Client ('C1')'s ACPs . the message SHOULD include a Timestamp option and a Nonce option. Note that the act of sending NS messages is cited as "MAY", since Client ('C1') may have advanced knowledge that the direct path to Client ('C2') would be unusable or otherwise undesirable. If the direct path later becomes unusable after the initial route optimization, Client ('C1') simply allows packets to again flow through Server ('S1').

When Server ('S1') receives an NS message from Client ('C1'), it first verifies that the SLLAOs in the NS are a proper subset of the link-layer addresses in Client ('C1')'s neighbor cache entry. If the Client's SLLAOs are not acceptable, Server ('S1') discards the message. Otherwise, Server ('S1') verifies that Client ('C1') is authorized to use the ACPs encoded in the RIOs of the NS and discards the NS if verification fails. Server ('S1') then examines the network-layer destination address of the NS to determine the next hop toward Client ('C2') by searching for the AERO address in the neighbor cache. Since Client ('C2') is not one of its neighbors, Server ('S1') then inserts an additional layer of encapsulation between the outer IP header and the NS message body. This new "mid-layer" IP header uses the AERO Server Subnet Router Anycast address as the source address and the Subnet Router Anycast address corresponding to the Client ("C2")'s AERO address as the destination address (in this case, C2's Subnet Router Anycast address if 2001:db8:1:0::). The Server then relays this double-encapsulated NS message via Relay ('R1') by changing the link-layer source address of the message to 'L2(S1)' and changing the link-layer destination address to 'L2(R1)'. Server ('S1') finally forwards the re-encapsulated message to Relay ('R1') without decrementing the network-layer TTL/Hop Limit field. When Relay ('R1') receives the double-encapsulated NS message from Server ('S1') it discards the outer IP header and determines that Server ('S2') is the next hop toward Client ('C2') by consulting its standard IP forwarding table for the Subnet Router Anycast destination address. Relay ('R1') then encapsulates and forwards the message to Server ('S2') the same as for any IP router. When Server ('S2') receives the double-encapsulated NS message from Relay ('R1') it removes the mid-layer IP header and determines that Client ('C2') is a neighbor by consulting its neighbor cache for Client ('C2')'s AERO address. Server ('S2') then re-encapsulates the NS while changing the link-layer source address to 'L2(S2)' and changing the link-layer destination address to 'L2(C2)'. Server ('S2') then forwards the message to Client ('C2').

When Client ('C2') receives the NS message, it accepts the NS only if the message has a link-layer source address of one of its Servers (e.g., L2(S2)). Client ('C2') further accepts the message only if it is willing to serve as a route optimization target. In the reference operational scenario, when Client ('C2') receives a valid NS message, it either creates or updates a dynamic neighbor cache entry that stores the source address of the message as the network-layer address of Client ('C1') , stores the link-layer addresses found in the SLLAOs as the link-layer addresses of Client ('C1'), and stores the ACPs encoded in the RIOs of the NS as the ACPs for Client ('C1'). Client ('C2') then sets AcceptTime for the neighbor cache entry to ACCEPT_TIME. After processing the message, Client ('C2') prepares an NA message response as follows: the link-layer source address is set to 'L2(C2)' (i.e., the link-layer address of Client ('C2')). the link-layer destination address is set to 'L2(S2)' (i.e., the link-layer address of Server ('S2')). the network-layer source address is set to fe80::2001:db8:1:0 (i.e., the base AERO address of Client ('C2')). the network-layer destination address is set to fe80::2001:db8:0:0 (i.e., the base AERO address of Client ('C1')). the Type is set to 136. The Target Address is set to the Target Address field in the NS message. the message includes one or more TLLAOs set to appropriate values for Client ('C2')'s native underlying interfaces. the message includes one or more RIOs that include Client ('C2')'s ACPs . the message SHOULD include a Timestamp option and MUST echo the Nonce option received in the NS (i.e., if a Nonce option is included). Client ('C2') then sends the NA message to Server ('S2').

When Server ('S2') receives an NA message from Client ('C2'), it first verifies that the TLLAOs in the NA are a proper subset of the Interface IDs in Client ('C2')'s neighbor cache entry. If the Client's TLLAOs are not acceptable, Server ('S2') discards the message. Otherwise, Server ('S2') verifies that Client ('C2') is authorized to use the ACPs encoded in the RIOs of the NA message. If validation fails, Server ('S2') discards the NA. Server ('S2') then examines the network-layer destination address of the NA to determine the next hop toward Client ('C1') by searching for the AERO address in the neighbor cache. Since Client ('C1') is not one of its neighbors, Server ('S2') then inserts an additional layer of encapsulation between the outer IP header and the NA message body. This new "mid-layer" IP header uses the AERO Server Subnet Router Anycast address as the source address and the Subnet Router Anycast address corresponding to the Client ("C1")'s AERO address as the destination address (in this case, C1's Subnet Router Anycast address if 2001:db8:0:0::). The Server then relays this double-encapsulated NA message via Relay ('R1') by changing the link-layer source address of the message to 'L2(S2)' and changing the link-layer destination address to 'L2(R1)'. Server ('S2') finally forwards the re-encapsulated message to Relay ('R1') without decrementing the network-layer TTL/Hop Limit field. When Relay ('R1') receives the double-encapsulated NA message from Server ('S2') it discards the outer IP header and determines that Server ('S1') is the next hop toward Client ('C1') by consulting its standard IP forwarding table for the Subnet Router Anycast destination address. Relay ('R1') then encapsulates and forwards the message to Server ('S1') the same as for any IP router. When Server ('S1') receives the double-encapsulated NS message from Relay ('R1') it removes the mid-layer IP header and determines that Client ('C1') is a neighbor by consulting its neighbor cache for Client ('C1')'s AERO address. Server ('S1') then re-encapsulates the NA while changing the link-layer source address to 'L2(S1)' and changing the link-layer destination address to 'L2(C1)'. Server ('S1') then forwards the message to Client ('C1').

When Client ('C1') receives the NA message, it first verifies the Nonce value matches the value that it included in its NS message (if any). If the Nonce values match, Client ('C1') then processes the message as follows. In the reference operational scenario, when Client ('C1') receives the NA message, it either creates or updates a dynamic neighbor cache entry that stores the source address of the message as the network-layer address of Client ('C2'), stores the link-layer addresses found in the TLLAOs as the link-layer addresses of Client ('C2') and stores the ACPs encoded in the RIOs of the NA as the ACPs for Client ('C2'). Client ('C1') then sets ForwardTime for the neighbor cache entry to FORWARD_TIME. Now, Client ('C1') has a neighbor cache entry with a valid ForwardTime value, while Client ('C2') has a neighbor cache entry with a valid AcceptTime value. Thereafter, Client ('C1') may forward ordinary network-layer data packets directly to Client ('C2') without involving any intermediate nodes, and Client ('C2') can verify that the packets came from an acceptable source. (In order for Client ('C2') to forward packets to Client ('C1'), a corresponding NS/NA message exchange is required in the reverse direction; hence, the mechanism is asymmetric.)

Route optimization may be initiated by the source Client by sending NS messages with SLLAOs corresponding to its native underlying interfaces. Route optimization for the source Client's other interfaces may be initiated by Servers and/or Proxies. Each node initiates route optimization by sending NS messages with SLLAOs only for those underlying interfaces they are authoritative for. Each node MUST consistently use the same Interface ID values to denote the same interfaces. The Interface IDs are established and maintained by the source Client's RS/RA exchanges. The target Client's Server serves as a route optimization target if some or all of the target Client's underlying interfaces connect via NATs, Proxies and/or VPNs. In that case, when the source sends an NS message the target Server both forwards the NS toward native underlying interfaces of the target Client (if any) and prepares an NA response the same as if it were the target Client (see: ). (This means that the source may receive multiple NA messages - one from the target Server and additional messages from the target Client. The source must accept the union of the information from all messages.) For non-native underlying interfaces, the target Server includes a first TLLAO option in the NA with Interface ID set to 255 and includes any additional TLLAOs corresponding to the Client's NATed, Proxyed and/or VPNed underlying interfaces. The Server writes its own link-layer address in TLLAOs corresponding to NATed and VPNed underlying interfaces, and writes the link-layer address of the Proxy in TLLAOs corresponding to Proxyed underlying interfaces (while also setting the X flag). The Interface ID and QoS Preference values in the TLLAOs are those supplied by the Client during the initial RS/RA exchange and updated by any ensuing NS/NA messages. The target Server must then maintain a dynamic neighbor cache entry for the Client, but MUST NOT send BGP updates for Clients discovered through dynamic route optimization. Thereafter, if the target Client moves to a new Server, the old Server sends unsolicited NA messages with no TLLAOs (subject to rate limiting) back to the source in response to data packets received from a correspondent node while forwarding the packets themselves to a Relay. The Relay will then either forward the packets to the new Server if the target Client has moved, or drop the packets if the target Client is no longer in the network. The source then allows future packets destined to the target Client to again flow through its own Server (or Relay). Note however that the old Server retains the neighbor cache entry with its associated AcceptTime since there may be many packets in flight. AcceptTime will then eventually decrement to 0 once the correspondent node processes and acts on the unsolicited NAs. When the target Client (or Proxy) sends unsolicited NA messages to the target Server to update link-layer address and/or QoS preferences, the target Server repeats the messages to any of its dynamic neighbors while using its own link-layer and link-local addresses as the source addresses. In this way, the target Server acts as a link-scoped multicast repeater on behalf of the target Client (or Proxy). (Note that instead of serving as the route optimization target for Proxy interfaces, the target Server could instead forward the source's NS messages and allow the Proxies to return NA messages, i.e., the same as for Clients on native interfaces. That would mean that the source could receive multiple NA messages from multiple Proxies and, if some or all NA messages are lost, the source would not be able to determine the full picture of the Client's Proxy affiliations. If this alternate architecture is deemed appropriate in some use cases, then the AERO Proxies could be employed to serve as route optimization targets instead of depending on the Servers to do so.)

AERO nodes perform Neighbor Unreachability Detection (NUD) by sending NS messages to elicit solicited NA messages from neighbors the same as described in . NUD is performed either reactively in response to persistent link-layer errors (see ) or proactively to update neighbor cache entry timers and/or link-layer address information. (NS messages may include SLLAOs and NA messages may include TLLAOs in order to update link-layer address information.) When an AERO node sends an NS/NA message, it uses one of its link-local addresses as the IPv6 source address and a link-local address of the neighbor as the IPv6 destination address. When route optimization directs a source AERO node to a target AERO node, the source node SHOULD proactively test the direct path by sending an initial NS message to elicit a solicited NA response. While testing the path, the source node can optionally continue sending packets via its default router, maintain a small queue of packets until target reachability is confirmed, or (optimistically) allow packets to flow directly to the target. While data packets are still flowing, the source node thereafter periodically tests the direct path to the target node (see Section 7.3 of ) in order to keep dynamic neighbor cache entries alive. When the target node receives a valid NS message, it resets AcceptTime to ACCEPT_TIME and updates its cached link-layer addresses (if necessary). When the source node receives a solicited NA message, it resets ForwardTime to FORWARD_TIME and updates its cached link-layer addresses (if necessary). If the source node is unable to elicit a solicited NA response from the target node after MaxRetry attempts, it SHOULD set ForwardTime to 0. Otherwise, the source node considers the path usable and SHOULD thereafter process any link-layer errors as an indication that the direct path to the target node has either failed or has become intermittent. When ForwardTime for a dynamic neighbor cache entry expires, the source node resumes sending any subsequent packets via a Server (or Relay) and may (eventually) attempt to re-initiate the AERO route optimization process. When AcceptTime for a dynamic neighbor cache entry expires, the target node discards any subsequent packets received directly from the source node. When both ForwardTime and AcceptTime for a dynamic neighbor cache entry expire, the node deletes the neighbor cache entry. Note that an AERO node may have multiple underlying interface paths toward the target neighbor. In that case, the node SHOULD perform NUD over each underlying interface and only consider the neighbor unreachable if NUD fails over multiple underlying interface paths.

AERO is an example of a Distributed Mobility Management (DMM) service. Each AERO Server is responsible for only a subset of the Clients on the AERO link, as opposed to a Centralized Mobility Management (CMM) service where there is a single network service for all Clients. AERO Clients coordinate with their regional Servers via RS/RA exchanges to maintain the DMM profile, and the AERO routing system tracks the current AERO Client/Server peering relationships. AERO interfaces accommodate mobility management by sending unsolicited NA messages the same as for announcing link-layer address changes for any interface that implements IPv6 ND . (In environments where reliability is a concern, AERO interfaces can send immediate NS messages to receive solicited NA messages, i.e., they can skip the unreliable unsolicited NA messaging step and move directly to a reliable NS/NA exchange. This comes at a penalty of at least one round trip.) When a node sends an unsolicited NA message, it sets the IPv6 source to its own link-local address, sets the IPv6 destination address to all-nodes multicast, sets the link-layer source address to its own address and sets the link-layer destination address to either a multicast address or the unicast link-layer address of a neighbor. If the unsolicited NA message must be received by multiple neighbors, the node sends multiple copies of the NA using a different unicast link-layer destination address for each neighbor. Mobility management considerations are specified in the following sections.

When a Server receives packets with destination addresses that do not match one of its static neighbor cache Clients, it forwards the packets to a Relay and also returns an unsolicited NA message to the sender with no TLLAOs. The packets will be delivered to the target Client's new location, and the sender will realize that it needs to deprecate its routing information that associated the target with this Server.

When a Client needs to change its link-layer addresses, e.g., due to a mobility event, it sends unsolicited NAs to its neighbors using the new link-layer address as the source address and with TLLAOs that include the new Client UDP Port Number, IP Address and P(i) values. If the Client sends the NA solely for the purpose of updating QoS preferences without updating the link-layer address, the Client sets the UDP Port Number and IP Address to 0. The Client MAY send up to MaxRetry unsolicited NA messages in parallel with sending actual data packets in case one or more NAs are lost. If all NAs are lost, the neighbor will eventually invoke NUD by sending NS messages that include SLLAOs.

When a Client needs to bring new underlying interfaces into service (e.g., when it activates a new data link), it sends unsolicited NAs to its neighbors using the new link-layer address as the source address and with TLLAOs that include the new Client link-layer information.

When a Client needs to remove existing underlying interfaces from service (e.g., when it de-activates an existing data link), it sends unsolicited NAs to its neighbors with TLLAOs with all P(i) values set to 0. If the Client needs to send the unsolicited NAs over an underlying interface other than the one being removed from service, it MUST include a current TLLAO for the sending interface as the first TLLAO and include TLLAOs for any underlying interface being removed from service as additional TLLAOs.

AERO interface neighbors MAY provide a configuration option that allows them to perform implicit mobility management in which no ND messaging is used. In that case, the Client only transmits packets over a single interface at a time, and the neighbor always observes packets arriving from the Client from the same link-layer source address. If the Client's underlying interface address changes (either due to a readdressing of the original interface or switching to a new interface) the neighbor immediately updates the neighbor cache entry for the Client and begins accepting and sending packets to the Client's new link-layer address. This implicit mobility method applies to use cases such as cellphones with both WiFi and Cellular interfaces where only one of the interfaces is active at a given time, and the Client automatically switches over to the backup interface if the primary interface fails.

When a Client associates with a new Server, it performs the Client procedures specified in . When a Client disassociates with an existing Server, it sends an RS "Release" message via a new Server with its base AERO address as the network-layer source address and the (administratively-provisioned) link-local address of the old Server as the network-layer destination address. The new Server then caches the Client's AERO address and "Release" message parameters (e.g., "transaction ID") and writes its own administratively-provisioned link-local address as the network-layer source address. The new Server then forwards the message to a Relay, which forwards the message to the old Server. When the old Server receives the "Release", it releases the Client's ACP prefix delegations and routes. The old Server then deletes the Client's neighbor cache entry so that any in-flight packets will be forwarded via a Relay to the new Server, which will forward them to the Client. The old Server finally returns a "Reply" message via a Relay to the new Server, which will decapsulate the "Reply" message and forward it as an RA "Reply" to the Client. When the new Server forwards the "Reply" message, the Client can delete both the default route and the neighbor cache entry for the old Server. (Note that since messages may be lost in the network the Client SHOULD retry until it gets an RA "Reply" indicating that the RS "Release" was successful. If the Client does not receive a "Reply" after MaxRetry attempts, the old Server may have failed and the Client should discontinue its "Release" attempts.) Finally, Clients SHOULD NOT move rapidly between Servers in order to avoid causing excessive oscillations in the AERO routing system. Such oscillations could result in intermittent reachability for the Client itself, while causing little harm to the network. Examples of when a Client might wish to change to a different Server include a Server that has gone unreachable, topological movements of significant distance, etc.

When the underlying network does not support multicast, AERO Clients map link-scoped multicast addresses to the link-layer address of a Server, which acts as a multicast forwarding agent. The AERO Client also serves as an IGMP/MLD Proxy for its EUNs and/or hosted applications per while using the link-layer address of the Server as the link-layer address for all multicast packets. When the underlying network supports multicast, AERO nodes use the multicast address mapping specification found in for IPv4 underlying networks and use a TBD site-scoped multicast mapping for IPv6 underlying networks. In that case, border routers must ensure that the encapsulated site-scoped multicast packets do not leak outside of the site spanned by the AERO link.