<?xml version="1.0" encoding="US-ASCII"?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd">
<?rfc toc="yes"?>
<?rfc tocompact="yes"?>
<?rfc tocdepth="3"?>
<?rfc tocindent="yes"?>
<?rfc symrefs="yes"?>
<?rfc sortrefs="yes"?>
<?rfc comments="yes"?>
<?rfc inline="yes"?>
<?rfc compact="yes"?>
<?rfc subcompact="no"?>
<?rfc strict='yes'?>
<?rfc iprnotified='no'?>
<rfc category="info" docName="draft-ietf-rtgwg-atn-bgp-10" ipr="trust200902">
  <front>
    <title abbrev="BGP for ATN/IPS">A Simple BGP-based Mobile Routing System
    for the Aeronautical Telecommunications Network</title>

    <author fullname="Fred L. Templin" initials="F. L." role="editor"
            surname="Templin">
      <organization>Boeing Research &amp; Technology</organization>

      <address>
        <postal>
          <street>P.O. Box 3707</street>

          <city>Seattle</city>

          <region>WA</region>

          <code>98124</code>

          <country>USA</country>
        </postal>

        <email>fltemplin@acm.org</email>
      </address>
    </author>

    <author fullname="Greg Saccone" initials="G." surname="Saccone">
      <organization>Boeing Research &amp; Technology</organization>

      <address>
        <postal>
          <street>P.O. Box 3707</street>

          <city>Seattle</city>

          <region>WA</region>

          <code>98124</code>

          <country>USA</country>
        </postal>

        <email>gregory.t.saccone@boeing.com</email>
      </address>
    </author>

    <author fullname="Gaurav Dawra" initials="G." surname="Dawra">
      <organization>LinkedIn</organization>

      <address>
        <postal>
          <street/>

          <city/>

          <region/>

          <code/>

          <country>USA</country>
        </postal>

        <email>gdawra.ietf@gmail.com</email>
      </address>
    </author>

    <author fullname="Acee Lindem" initials="A." surname="Lindem">
      <organization>Cisco Systems, Inc.</organization>

      <address>
        <postal>
          <street/>

          <city/>

          <region/>

          <code/>

          <country>USA</country>
        </postal>

        <email>acee@cisco.com</email>
      </address>
    </author>

    <author fullname="Victor Moreno" initials="V." surname="Moreno">
      <organization>Cisco Systems, Inc.</organization>

      <address>
        <postal>
          <street/>

          <city/>

          <region/>

          <code/>

          <country>USA</country>
        </postal>

        <email>vimoreno@cisco.com</email>
      </address>
    </author>

    <date day="1" month="January" year="2021"/>

    <keyword>I-D</keyword>

    <keyword>Internet-Draft</keyword>

    <abstract>
      <t>The International Civil Aviation Organization (ICAO) is investigating
      mobile routing solutions for a worldwide Aeronautical Telecommunications
      Network with Internet Protocol Services (ATN/IPS). The ATN/IPS will
      eventually replace existing communication services with an IPv6-based
      service supporting pervasive Air Traffic Management (ATM) for Air
      Traffic Controllers (ATC), Airline Operations Controllers (AOC), and all
      commercial aircraft worldwide. This informational document describes a
      simple and extensible mobile routing service based on industry-standard
      BGP to address the ATN/IPS requirements.</t>
    </abstract>
  </front>

  <middle>
    <section anchor="intro" title="Introduction">
      <t>The worldwide Air Traffic Management (ATM) system today uses a
      service known as Aeronautical Telecommunications Network based on Open
      Systems Interconnection (ATN/OSI). The service is used to augment
      controller to pilot voice communications with rudimentary short text
      command and control messages. The service has seen successful deployment
      in a limited set of worldwide ATM domains.</t>

      <t>The International Civil Aviation Organization (ICAO) is now
      undertaking the development of a next-generation replacement for ATN/OSI
      known as Aeronautical Telecommunications Network with Internet Protocol
      Services (ATN/IPS) <xref target="ATN"/><xref target="ATN-IPS"/>. ATN/IPS
      will eventually provide an IPv6-based <xref target="RFC8200"/> service
      supporting pervasive ATM for Air Traffic Controllers (ATC), Airline
      Operations Controllers (AOC), and all commercial aircraft worldwide. As
      part of the ATN/IPS undertaking, a new mobile routing service will be
      needed. This document presents an approach based on the Border Gateway
      Protocol (BGP) <xref target="RFC4271"/>.</t>

      <t>Aircraft communicate via wireless aviation data links that typically
      support much lower data rates than terrestrial wireless and wired-line
      communications. For example, some Very High Frequency (VHF)-based data
      links only support data rates on the order of 32Kbps and an emerging
      L-Band data link that is expected to play a key role in future
      aeronautical communications only supports rates on the order of 1Mbps.
      Although satellite data links can provide much higher data rates during
      optimal conditions, like any other aviation data link they are subject
      to errors, delay, disruption, signal intermittence, degradation due to
      atmospheric conditions, etc. The well-connected ground domain ATN/IPS
      network should therefore treat each safety-of-flight critical packet
      produced by (or destined to) an aircraft as a precious commodity and
      strive for an optimized service that provides the highest possible
      degree of reliability.</t>

      <t>The ATN/IPS is an IPv6-based overlay network configured over one or
      more Internetworking underlays ("INETs") maintained by aeronautical
      network service providers such as ARINC, SITA and Inmarsat. The overlay
      provides an Overlay Multilink Network Interface (OMNI) virtual link
      layer <xref target="I-D.templin-6man-omni-interface"/> as a
      Non-Broadcast, Multiple Access (NBMA) virtual link that spans the entire
      ATN/IPS. Each aircraft connects to the OMNI link via an OMNI interface
      configured over the aircraft's underlying physical and/or virtual access
      network interfaces.</t>

      <t>Each underlying INET comprises one or more "partitions" where all
      nodes within a partition can exchange packets with all other nodes,
      i.e., the partition is connected internally. There is no requirement
      that any two INET partitions use the same IP protocol version nor have
      consistent IP addressing plans in comparison with other partitions.
      Instead, the OMNI link sees each partition as a "segment" of a
      link-layer topology manifested through a (virtual) bridging service
      based on IPv6 encapsulation <xref target="RFC2473"/> known as the OMNI
      Adaptation Layer (OAL) <xref
      target="I-D.templin-6man-omni-interface"/><xref
      target="I-D.templin-intarea-6706bis"/>.</t>

      <t>The IPv6 addressing architecture provides different classes of
      addresses, including Global Unicast Addresses (GUAs), Unique Local
      Addresses (ULAs) and Link-Local Addresses (LLAs) <xref
      target="RFC4291"/><xref target="RFC4193"/>. The ATN/IPS receives an IPv6
      GUA Mobility Service Prefix (MSP) from an Internet assigned numbers
      authority, and each aircraft will receive a Mobile Network Prefix (MNP)
      delegation from the MSP that accompanies the aircraft wherever it
      travels. ATCs and AOCs will likewise receive MNPs, but they would
      typically appear in static (not mobile) deployments such as air traffic
      control towers, airline headquarters, etc.</t>

      <t>The OAL uses ULAs in the source and destination addresses of IPv6
      encapsulation headers. Each ULA includes an MNP in the interface
      identifier ("MNP-ULA") as discussed in <xref
      target="I-D.templin-6man-omni-interface"/>. Due to MNP delegation
      policies and random MN mobility properties, MNP-ULAs are generally not
      aggregatable in the BGP routing service. In addition, OMNI link service
      nodes configure administratively-assigned ULAs ("ADM-ULA") that are
      statically-assigned and derived from a shorter ADM-ULA prefix assigned
      to their OMNI link partition <xref
      target="I-D.templin-6man-omni-interface"/>. Unlike MNP-ULAs, the
      ADM-ULAs are persistently present and unchanging in the routing system.
      The BGP routing services therefore perform forwarding based on these
      MNP-ULAs and ADM-ULAs instead of based on the GUA MNPs themselves.</t>

      <t>Connexion By Boeing <xref target="CBB"/> was an early aviation mobile
      routing service based on dynamic updates in the global public Internet
      BGP routing system. Practical experience with the approach has shown
      that frequent injections and withdrawals of prefixes in the Internet
      routing system can result in excessive BGP update messaging, slow
      routing table convergence times, and extended outages when no route is
      available. This is due to both conservative default BGP protocol timing
      parameters (see <xref target="limit"/>) and the complex peering
      interconnections of BGP routers within the global Internet
      infrastructure. The situation is further exacerbated by frequent
      aircraft mobility events that each result in BGP updates that must be
      propagated to all BGP routers in the Internet that carry a full routing
      table.</t>

      <t>We therefore consider an approach using a BGP overlay network routing
      system where a private BGP routing protocol instance is maintained
      between ATN/IPS Autonomous System (AS) Border Routers (ASBRs). The
      private BGP instance does not interact with the native BGP routing
      systems in underlying INETs, and BGP updates are unidirectional from
      "stub" ASBRs (s-ASBRs) to a small set of "core" ASBRs (c-ASBRs) in a
      hub-and-spokes topology. No extensions to the BGP protocol are
      necessary. BGP routing is based on the ULAs found in OAL headers, i.e.,
      it provides an adaptation layer forwarding service instead of a
      networking layer services.</t>

      <t>The s-ASBRs for each stub AS connect to a small number of c-ASBRs via
      dedicated high speed links and/or tunnels across the INET using
      industry-standard secured encapsulations (e.g., IPsec <xref
      target="RFC4301"/>, Wireguard, etc.). In particular, tunneling must be
      used when neighboring ASBRs are separated by multiple INET hops.</t>

      <t>The s-ASBRs engage in external BGP (eBGP) peerings with their
      respective c-ASBRs, and only maintain routing table entries for the
      MNP-ULAs currently active within the stub AS. The s-ASBRs send BGP
      updates for MNP-ULA injections or withdrawals to c-ASBRs but do not
      receive any BGP updates from c-ASBRs. Instead, the s-ASBRs maintain
      default routes with their c-ASBRs as the next hop, and therefore hold
      only partial topology information.</t>

      <t>The c-ASBRs connect to other c-ASBRs within the same partition using
      internal BGP (iBGP) peerings over which they collaboratively maintain a
      full routing table for all active MNP-ULAs currently in service within
      the partition. Therefore, only the c-ASBRs maintain a full BGP routing
      table and never send any BGP updates to s-ASBRs. This simple routing
      model therefore greatly reduces the number of BGP updates that need to
      be synchronized among peers, and the number is reduced further still
      when intradomain routing changes within stub ASes are processed within
      the AS instead of being propagated to the core. BGP Route Reflectors
      (RRs) <xref target="RFC4456"/> can also be used to support increased
      scaling properties.</t>

      <t>When there are multiple INET partitions, the c-ASBRs of each
      partition use eBGP to peer with the c-ASBRs of other partitions so that
      the full set of ULAs for all partitions are known globally among all of
      the c-ASBRs. Each c/s-ASBR further configures an ADM-ULA which is taken
      from a ADM-ULA prefix assigned to each partition, as well as static
      forwarding table entries for all other OMNI link partition prefixes.
      Both ADM-ULAs and MNP-ULAs are used by the OAL for nested encapsulation
      where the inner IPv6 packet is encapsulated in an IPv6 OAL header with
      ULA source and destination addresses, which is then encapsulated in an
      IP header specific to the INET partition.</t>

      <t>The remainder of this document discusses the proposed BGP-based
      ATN/IPS mobile routing service.</t>
    </section>

    <section anchor="terminology" title="Terminology">
      <t>The terms Autonomous System (AS) and Autonomous System Border Router
      (ASBR) are the same as defined in <xref target="RFC4271"/>.</t>

      <t>The following terms are defined for the purposes of this
      document:</t>

      <t><list style="hanging">
          <t hangText="Air Traffic Management (ATM)"><vspace/>The worldwide
          service for coordinating safe aviation operations.</t>

          <t hangText="Air Traffic Controller (ATC)"><vspace/>A government
          agent responsible for coordinating with aircraft within a defined
          operational region via voice and/or data Command and Control
          messaging.</t>

          <t hangText="Airline Operations Controller (AOC)"><vspace/>An
          airline agent responsible for tracking and coordinating with
          aircraft within their fleet.</t>

          <t
          hangText="Aeronautical Telecommunications Network with Internet Protocol Services (ATN/IPS)"><vspace/>A
          future aviation network for ATCs and AOCs to coordinate with all
          aircraft operating worldwide. The ATN/IPS will be an IPv6-based
          overlay network service that connects access networks via tunneling
          over one or more Internetworking underlays.</t>

          <t hangText="Internetworking underlay (&quot;INET&quot;)"><vspace/>A
          wide-area network that supports overlay network tunneling and
          connects Radio Access Networks to the rest of the ATN/IPS. Example
          INET service providers for civil aviation include ARINC, SITA and
          Inmarsat.</t>

          <t hangText="(Radio) Access Network (&quot;ANET&quot;)"><vspace/>An
          aviation radio data link service provider's network, including radio
          transmitters and receivers as well as supporting ground-domain
          infrastructure needed to convey a customer's data packets to outside
          INETs. The term ANET is intended in the same spirit as for
          radio-based Internet service provider networks (e.g., cellular
          operators), but can also refer to ground-domain networks that
          connect AOCs and ATCs.</t>

          <t hangText="partition (or &quot;segment&quot;)"><vspace/>A
          fully-connected internal subnetwork of an INET in which all nodes
          can communicate with all other nodes within the same partition using
          the same IP protocol version and addressing plan. Each INET consists
          of one or more partitions.</t>

          <t hangText="Overlay Multilink Network Interface (OMNI)"><vspace/>A
          virtual layer 2 bridging service that presents an ATN/IPS overlay
          unified link view even though the underlay may consist of multiple
          INET partitions. The OMNI virtual link is manifested through nested
          encapsulation in which GUA-addressed IPv6 packets from the ATN/IPS
          are first encapsulated in ULA-addressed IPv6 headers which are then
          forwarded to the next hop using INET encapsulation if necessary.
          Forwarding over the OMNI virtual link is therefore based on ULAs
          instead of GUAs. In this way, packets sent from a source can be
          conveyed over the OMNI virtual link even though there may be many
          underlying INET partitions in the path to the destination.</t>

          <t hangText="OMNI Adaptation Layer (OAL)"><vspace/>A middle layer
          below the IPv6 layer but above the INET layer that applies
          IPv6-in-IPv6 encapsulation prior to INET encapsulation. The IPv6
          encapsulation header inserted by the OAL uses ULAs instead of GUAs.
          Further details on OMNI and the OAL are found in <xref
          target="I-D.templin-6man-omni-interface"/>.</t>

          <t hangText="OAL Autonomous System"><vspace/>A "hub-of-hubs"
          autonomous system maintained through peerings between the core
          autonomous systems of different OMNI virtual link partitions.</t>

          <t
          hangText="Core Autonomous System Border Router (c-ASBR)"><vspace/>A
          BGP router located in the hub of the INET partition hub-and-spokes
          overlay network topology.</t>

          <t hangText="Core Autonomous System"><vspace/>The "hub" autonomous
          system maintained by all c-ASBRs within the same partition.</t>

          <t
          hangText="Stub Autonomous System Border Router (s-ASBR)"><vspace/>A
          BGP router configured as a spoke in the INET partition
          hub-and-spokes overlay network topology.</t>

          <t hangText="Stub Autonomous System"><vspace/>A logical grouping
          that includes all Clients currently associated with a given
          s-ASBR.</t>

          <t hangText="Client"><vspace/>An ATC, AOC or aircraft that connects
          to the ATN/IPS as a leaf node. The Client could be a singleton host,
          or a router that connects a mobile or fixed network.</t>

          <t hangText="Proxy"><vspace/>An ANET/INET border node that acts as a
          transparent intermediary between Clients and s-ASBRs. From the
          Client's perspective, the Proxy presents the appearance that the
          Client is communicating directly with the s-ASBR. From the s-ASBR's
          perspective, the Proxy presents the appearance that the s-ASBR is
          communicating directly with the Client.</t>

          <t hangText="Mobile Network Prefix (MNP)"><vspace/>An IPv6 prefix
          that is delegated to any ATN/IPS end system, including ATCs, AOCs,
          and aircraft.</t>

          <t hangText="Mobility Service Prefix (MSP)"><vspace/>An aggregated
          prefix assigned to the ATN/IPS by an Internet assigned numbers
          authority, and from which all MNPs are delegated (e.g., up to 2**32
          IPv6 /56 MNPs could be delegated from a /24 MSP).</t>
        </list></t>
    </section>

    <section anchor="scaling" title="ATN/IPS Routing System">
      <t>The ATN/IPS routing system comprises a private BGP instance
      coordinated in an overlay network via tunnels between neighboring ASBRs
      over one or more underlying INETs. The overlay does not interact with
      the underlying INET BGP routing systems, and only a small and unchanging
      set of MSPs are advertised externally instead of the full dynamically
      changing set of MNPs.</t>

      <t>Within each INET partition, each s-ASBRs connects a stub AS to the
      INET partition core using a distinct stub AS Number (ASN). Each s-ASBR
      further uses eBGP to peer with one or more c-ASBRs. All c-ASBRs are
      members of the INET partition core AS, and use a shared core ASN. Unique
      ASNs are assigned according to the standard 32-bit ASN format <xref
      target="RFC4271"/><xref target="RFC6793"/>. Since the BGP instance does
      not connect with any INET BGP routing systems, the ASNs assigned need
      not be coordinated with IANA and can in fact coincide with values that
      are assigned in other domains. The only requirement is that ASNs must
      not be duplicated within the ATN/IPS routing system itself.</t>

      <t>The c-ASBRs use iBGP to maintain a synchronized consistent view of
      all active MNP-ULAs currently in service within the INET partition.
      <xref target="BGPDEP"/> below represents the reference INET partition
      deployment. (Note that the figure shows details for only two s-ASBRs
      (s-ASBR1 and s-ASBR2) due to space constraints, but the other s-ASBRs
      should be understood to have similar Stub AS, MNP and eBGP peering
      arrangements.) The solution described in this document is flexible
      enough to extend to these topologies.</t>

      <t><figure anchor="BGPDEP" title="INET Partition Reference Deployment">
          <artwork><![CDATA[  ...........................................................
.                                                             .
.               (:::)-.  <- Stub ASes ->  (:::)-.             .
.   MNPs-> .-(:::::::::)             .-(:::::::::) <-MNPs     .
.            `-(::::)-'                `-(::::)-'             .
.             +-------+                +-------+              .
.             |s-ASBR1+-----+    +-----+s-ASBR2|              .
.             +--+----+ eBGP \  / eBGP +-----+-+              .
.                 \           \/            /                 .
.                  \eBGP      / \          /eBGP              .
.                   \        /   \        /                   .
.                    +-------+   +-------+                    .
.          eBGP+-----+c-ASBR |...|c-ASBR +-----+eBGP          .
.   +-------+ /      +--+----+   +-----+-+      \ +-------+   .
.   |s-ASBR +/       iBGP\   (:::)-.  /iBGP      \+s-ASBR |   .
.   +-------+            .-(::::::::)             +-------+   .
.       .            .-(::::::::::::::)-.             .       .
.       .           (::::  Core AS   :::)             .       .
.   +-------+         `-(:::::::::::::)-'         +-------+   .
.   |s-ASBR +\      iBGP/`-(:::::::-'\iBGP       /+s-ASBR |   .
.   +-------+ \      +-+-----+   +----+--+      / +-------+   .
.          eBGP+-----+c-ASBR |...|c-ASBR +-----+eBGP          .
.                    +-------+   +-------+                    .
.                   /                     \                   .
.                  /eBGP                   \eBGP              .
.                 /                         \                 .
.            +---+---+                 +-----+-+              .
.            |s-ASBR |                 |s-ASBR |              .
.            +-------+                 +-------+              .
.                                                             .
.                                                             .
.   <----------------- INET Partition  ------------------->   .
 ............................................................
]]></artwork>
        </figure>In the reference deployment, each s-ASBR maintains routes for
      active MNP-ULAs that currently belong to its stub AS. In response to
      "Inter-domain" mobility events, each s-ASBR will dynamically announces
      new MNP-ULAs and withdraws departed MNP-ULAs in its eBGP updates to
      c-ASBRs. Since ATN/IPS end systems are expected to remain within the
      same stub AS for extended timeframes, however, intra-domain mobility
      events (such as an aircraft handing off between cell towers) are handled
      within the stub AS instead of being propagated as inter-domain eBGP
      updates.</t>

      <t>Each c-ASBR configures a black-hole route for each of its MSPs. By
      black-holing the MSPs, the c-ASBR will maintain forwarding table entries
      only for the MNP-ULAs that are currently active, and packets destined to
      all other MNP-ULAs will correctly incur ICMPv6 Destination Unreachable
      messages <xref target="RFC4443"/> due to the black hole route. (This is
      the same behavior as for ordinary BGP routers in the Internet when they
      receive packets for which there is no route available.) The c-ASBRs do
      not send eBGP updates for MNP-ULAs to s-ASBRs, but instead originate a
      default route. In this way, s-ASBRs have only partial topology knowledge
      (i.e., they know only about the active MNP-ULAs currently within their
      stub ASes) and they forward all other packets to c-ASBRs which have full
      topology knowledge.</t>

      <t>Each s-ASBR and c-ASBR configures an ADM-ULA that is aggregatable
      within an INET partition, and each partition configures a unique ADM-ULA
      prefix that is permanently announced into the routing system. The core
      ASes of each INET partition are joined together through external BGP
      peerings. The c-ASBRs of each partition establish external peerings with
      the c-ASBRs of other partitions to form a "core-of-cores" OMNI link AS.
      The OMNI link AS contains the global knowledge of all MNP-ULAs deployed
      worldwide, and supports ATN/IPS overlay communications between nodes
      located in different INET partitions by virtue of OAL encapsulation.
      OMNI link nodes can then navigate to ASBRs by including an ADM-ULA or
      directly to an end system by including an MNP-ULA in the destination
      address of an OAL-encapsulated packet (see: <xref
      target="I-D.templin-intarea-6706bis"/>) . <xref target="the-span"/>
      shows a reference OAL topology.</t>

      <figure anchor="the-span" title="Spanning Partitions with the OAL">
        <artwork><![CDATA[              . . . . . . . . . . . . . . . . . . . . . . . . . 
            .                                                   .
            .              .-(::::::::)                          .
            .           .-(::::::::::::)-.   +------+            .
            .          (::: Partition 1 ::)--|c-ASBR|---+        .
            .           `-(::::::::::::)-'   +------+   |        .
            .              `-(::::::)-'                 |        .
            .                                           |        .
            .              .-(::::::::)                 |        .
            .           .-(::::::::::::)-.   +------+   |        .
            .          (::: Partition 2 ::)--|c-ASBR|---+        .
            .           `-(::::::::::::)-'   +------+   |        .
            .              `-(::::::)-'                 |        .
            .                                           |        .
            .              .-(::::::::)                 |        .
            .           .-(::::::::::::)-.   +------+   |        .
            .          (::: Partition 3 ::)--|c-ASBR|---+        .
            .           `-(::::::::::::)-'   +------+   |        .
            .              `-(::::::)-'                 |        .
            .                                           |        .
            .                ..(etc)..                  x        .
            .                                                    .
            .                                                    .
            .    <- ATN/IPS Overlay Bridged by the OAL AS ->     .
              . . . . . . . . . . . . . .. . . . . . . . . . . .  
]]></artwork>
      </figure>

      <t>Scaling properties of this ATN/IPS routing system are limited by the
      number of BGP routes that can be carried by the c-ASBRs. A 2015 study
      showed that BGP routers in the global public Internet at that time
      carried more than 500K routes with linear growth and no signs of router
      resource exhaustion <xref target="BGP"/>. A more recent network
      emulation study also showed that a single c-ASBR can accommodate at
      least 1M dynamically changing BGP routes even on a lightweight virtual
      machine. Commercially-available high-performance dedicated router
      hardware can support many millions of routes.</t>

      <t>Therefore, assuming each c-ASBR can carry 1M or more routes, this
      means that at least 1M ATN/IPS end system MNP-ULAs can be serviced by a
      single set of c-ASBRs and that number could be further increased by
      using RRs and/or more powerful routers. Another means of increasing
      scale would be to assign a different set of c-ASBRs for each set of
      MSPs. In that case, each s-ASBR still peers with one or more c-ASBRs
      from each set of c-ASBRs, but the s-ASBR institutes route filters so
      that it only sends BGP updates to the specific set of c-ASBRs that
      aggregate the MSP. In this way, each set of c-ASBRs maintains separate
      routing and forwarding tables so that scaling is distributed across
      multiple c-ASBR sets instead of concentrated in a single c-ASBR set. For
      example, a first c-ASBR set could aggregate an MSP segment A::/32, a
      second set could aggregate B::/32, a third could aggregate C::/32, etc.
      The union of all MSP segments would then constitute the collective
      MSP(s) for the entire ATN/IPS, with potential for supporting many
      millions of mobile networks or more.</t>

      <t>In this way, each set of c-ASBRs services a specific set of MSPs, and
      each s-ASBR configures MSP-specific routes that list the correct set of
      c-ASBRs as next hops. This design also allows for natural incremental
      deployment, and can support initial medium-scale deployments followed by
      dynamic deployment of additional ATN/IPS infrastructure elements without
      disturbing the already-deployed base. For example, a few more c-ASBRs
      could be added if the MNP service demand ever outgrows the initial
      deployment. For larger-scale applications (such as unmanned air vehicles
      and terrestrial vehicles) even larger scales can be accommodated by
      adding more c-ASBRs.</t>
    </section>

    <section anchor="intradomain"
             title="ATN/IPS (Radio) Access Network (ANET) Model">
      <t>(Radio) Access Networks (ANETs) connect end system Clients such as
      aircraft, ATCs, AOCs etc. to the ATN/IPS routing system. Clients may
      connect to multiple ANETs at once, for example, when they have both
      satellite and cellular data links activated simultaneously. Clients
      configure an Overlay Multilink Network (OMNI) Interface <xref
      target="I-D.templin-6man-omni-interface"/> over their underlying ANET
      interfaces as a connection to an NBMA virtual link (manifested by the
      OAL) that spans the entire ATN/IPS. Clients may further move between
      ANETs in a manner that is perceived as a network layer mobility event.
      Clients could therefore employ a multilink/mobility routing service such
      as those discussed in <xref target="maint"/>.</t>

      <t>Clients register all of their active data link connections with their
      serving s-ASBRs as discussed in <xref target="scaling"/>. Clients may
      connect to s-ASBRs either directly, or via a Proxy at the ANET/INET
      boundary.</t>

      <t><xref target="dsp_model"/> shows the ATN/IPS ANET model where Clients
      connect to ANETs via aviation data links. Clients register their ANET
      addresses with a nearby s-ASBR, where the registration process may be
      brokered by a Proxy at the edge of the ANET.</t>

      <figure anchor="dsp_model" title="ATN/IPS ANET Architecture">
        <artwork><![CDATA[                     +-----------------+
                     |     Client      |
      Data Link "A"  +-----------------+  Data Link "B"
              +----- |  OMNI Interface |--------+
             /       +-----------------+         \
            /                                     \
           /                                       \
        (:::)-.                                   (:::)-.
   .-(:::::::::)<- (Radio) Access Networks ->.-(:::::::::)
     `-(::::)-'                                `-(::::)-'
      +-------+                                +-------+
 ...  | Proxy |  ............................  | Proxy |  ...
.     +-------+                                +-------+     .
.         ^^                                      ^^         .
.         ||                                      ||         .
.         ||              +--------+              ||         .
.         ++============> | s-ASBR | <============++         .
.                         +--------+                         .
.                              | eBGP                        .
.                            (:::)-.                         .
.                        .-(::::::::)                        .
.                    .-(::: ATN/IPS :::)-.                   .
.                  (::::: BGP Routing ::::)                  .
.                     `-(:: System ::::)-'                   .
.                         `-(:::::::-'                       .
.                                                            .
.                                                            .
.  <------- OMNI virtual link bridged by the OAL -------->   .
 ............................................................
]]></artwork>
      </figure>

      <t>When a Client logs into an ANET it specifies a nearby s-ASBR that it
      has selected to connect to the ATN/IPS. (Selection of a nearby s-ASBR
      could be through consulting a geographically-keyed static host file,
      through a DNS lookup, through a network query response, etc.) The login
      process is transparently brokered by a Proxy at the border of the ANET,
      which then conveys the connection request to the s-ASBR via tunneling
      across the OMNI virtual link. The s-ASBR then registers the address of
      the Proxy as the address for the Client, and the Proxy forwards the
      s-ASBR's reply to the Client. If the Client connects to multiple ANETs,
      the s-ASBR will register the addresses of all Proxies as addresses
      through which the Client can be reached.</t>

      <t>The s-ASBR represents all of its active Clients as MNP-ULA routes in
      the ATN/IPS BGP routing system. The s-ASBR's stub AS therefore consists
      of the set of all of its active Clients (i.e., the stub AS is a logical
      construct and not a physical construct). The s-ASBR injects the MNP-ULAs
      of its active Clients and withdraws the MNP-ULAs of its departed Clients
      via BGP updates to c-ASBRs, which further propagate the MNP-ULAs to
      other c-ASBRs within the OAL AS. Since Clients are expected to remain
      associated with their current s-ASBR for extended periods, the level of
      MNP-ULA injections and withdrawals in the BGP routing system will be on
      the order of the numbers of network joins, leaves and s-ASBR handovers
      for aircraft operations (see: <xref target="limit"/>). It is important
      to observe that fine-grained events such as Client mobility and Quality
      of Service (QoS) signaling are coordinated only by Proxies and the
      Client's current s-ASBRs, and do not involve other ASBRs in the routing
      system. In this way, intradomain routing changes within the stub AS are
      not propagated into the rest of the ATN/IPS BGP routing system.</t>
    </section>

    <section anchor="optimize" title="ATN/IPS Route Optimization">
      <t>ATN/IPS end systems will frequently need to communicate with
      correspondents associated with other s-ASBRs. In the BGP peering
      topology discussed in <xref target="scaling"/>, this can initially only
      be accommodated by including multiple tunnel segments in the forwarding
      path. In many cases, it would be desirable to eliminate extraneous
      tunnel segments from this "dogleg" route so that packets can traverse a
      minimum number of tunneling hops across the OMNI virtual link. ATN/IPS
      end systems could therefore employ a route optimization service
      according to the mobility service employed (see: <xref
      target="maint"/>).</t>

      <t>A route optimization example is shown in <xref target="ro-dog"/> and
      <xref target="ro-opt"/> below. In the first figure, multiple tunneled
      segments between Proxys and ASBRs are necessary to convey packets
      between Clients associated with different s-ASBRs. In the second figure,
      the optimized route tunnels packets directly between Proxys without
      involving the ASBRs.</t>

      <t><figure anchor="ro-dog" title="Dogleg Route Before Optimization">
          <artwork><![CDATA[      +---------+                             +---------+
      | Client1 |                             | Client2 | 
      +---v-----+                             +-----^---+
          *                                         *
          *                                         *
        (:::)-.                                   (:::)-.
   .-(:::::::::)<- (Radio) Access Networks ->.-(:::::::::)
     `-(::::)-'                                `-(::::)-'
      +--------+                              +--------+
 ...  | Proxy1 |  ..........................  | Proxy2 |  ...
.     +--------+                              +--------+     .
.             **                               **            .
.              **                             **             .
.               **                           **              .
.           +---------+                  +---------+         .
.           | s-ASBR1 |                  | s-ASBR2 |         .
.           +--+------+                  +-----+---+         .
.                 \  **      Dogleg      **   /              .
.              eBGP\  **     Route      **   /eBGP           .
.                   \  **==============**   /                .
.                   +---------+   +---------+                .
.                   | c-ASBR1 |   | c-ASBR2 |                .
.                   +---+-----+   +----+----+                .
.                       +--------------+                     .
.                             iBGP                           .
.                                                            .
.  <------- OMNI virtual link bridged by the OAL -------->   .
 ............................................................
]]></artwork>
        </figure><figure anchor="ro-opt" title="Optimized Route">
          <artwork><![CDATA[      +---------+                             +---------+
      | Client1 |                             | Client2 | 
      +---v-----+                             +-----^---+
          *                                         *
          *                                         *
        (:::)-.                                   (:::)-.
   .-(:::::::::) <- (Radio) Access Networks ->.-(:::::::::)
     `-(::::)-'                                `-(::::)-'
      +--------+                              +--------+
 ...  | Proxy1 |  ..........................  | Proxy2 |  ...
.     +------v-+                              +--^-----+     .
.             *                                  *           .
.              *================================*            .
.                                                            .
.           +---------+                  +---------+         .
.           | s-ASBR1 |                  | s-ASBR2 |         .
.           +--+------+                  +-----+---+         .
.                 \                           /              .
.              eBGP\                         /eBGP           .
.                   \                       /                .
.                   +---------+   +---------+                .
.                   | c-ASBR1 |   | c-ASBR2 |                .
.                   +---+-----+   +----+----+                .
.                       +--------------+                     .
.                             iBGP                           .
.                                                            .
.  <------- OMNI virtual link bridged by the OAL -------->   .
 ............................................................
]]></artwork>
        </figure></t>
    </section>

    <section anchor="limit" title="BGP Protocol Considerations">
      <t>The number of eBGP peering sessions that each c-ASBR must service is
      proportional to the number of s-ASBRs in its local partition. Network
      emulations with lightweight virtual machines have shown that a single
      c-ASBR can service at least 100 eBGP peerings from s-ASBRs that each
      advertise 10K MNP-ULA routes (i.e., 1M total). It is expected that
      robust c-ASBRs can service many more peerings than this - possibly by
      multiple orders of magnitude. But even assuming a conservative limit,
      the number of s-ASBRs could be increased by also increasing the number
      of c-ASBRs. Since c-ASBRs also peer with each other using iBGP, however,
      larger-scale c-ASBR deployments may need to employ an adjunct facility
      such as BGP Route Reflectors (RRs)<xref target="RFC4456"> </xref>.</t>

      <t>The number of aircraft in operation at a given time worldwide is
      likely to be significantly less than 1M, but we will assume this number
      for a worst-case analysis. Assuming a worst-case average 1 hour flight
      profile from gate-to-gate with 10 service region transitions per flight,
      the entire system will need to service at most 10M BGP updates per hour
      (2778 updates per second). This number is within the realm of the peak
      BGP update messaging seen in the global public Internet today <xref
      target="BGP2"/>. Assuming a BGP update message size of 100 bytes
      (800bits), the total amount of BGP control message traffic to a single
      c-ASBR will be less than 2.5Mbps which is a nominal rate for modern data
      links.</t>

      <t>Industry standard BGP routers provide configurable parameters with
      conservative default values. For example, the default hold time is 90
      seconds, the default keepalive time is 1/3 of the hold time, and the
      default MinRouteAdvertisementinterval is 30 seconds for eBGP peers and 5
      seconds for iBGP peers (see Section 10 of <xref target="RFC4271"/>). For
      the simple mobile routing system described herein, these parameters can
      be set to more aggressive values to support faster neighbor/link failure
      detection and faster routing protocol convergence times. For example, a
      hold time of 3 seconds and a MinRouteAdvertisementinterval of 0 seconds
      for both iBGP and eBGP.</t>

      <t>Instead of adjusting BGP default time values, BGP routers can use the
      Bidirectional Forwarding Detection (BFD) protocol <xref
      target="RFC5880"/> to quickly detect link failures that don&rsquo;t
      result in interface state changes, BGP peer failures, and administrative
      state changes. BFD is important in environments where rapid response to
      failures is required for routing reconvergence and, hence,
      communications continuity.</t>

      <t>Each c-ASBR will be using eBGP both in the ATN/IPS and the INET with
      the ATN/IPS unicast IPv6 routes resolving over INET routes.
      Consequently, c-ASBRs and potentially s-ASBRs will need to support
      separate local ASes for the two BGP routing domains and routing policy
      or assure routes are not propagated between the two BGP routing domains.
      From a conceptual and operational standpoint, the implementation should
      provide isolation between the two BGP routing domains (e.g., separate
      BGP instances).</t>

      <t>ADM-ULAs and MNP-ULAs begin with fd00::/8 followed by a pseudo-random
      40-bit global ID to form the prefix [ULA]::/48, along with a 16-bit OMNI
      link identifier '*' to form the prefix [ULA*]::/64. Each individual
      address taken from [ULA*]::/64 includes additional routing information
      in the interface identifier. For example, for the MNP 2001:db8:1:0::/56,
      the resulting MNP-ULA is [ULA*]:2001:db8:1:0/120, and for the
      administrative address 1001:2002/16 the ADM-ULA is [ULA*]::1001:2002/112
      (see: <xref target="I-D.templin-6man-omni-interface"/> for further
      details). This gives rise to a BGP routing system that must accommodate
      large numbers of long and non-aggregatable MNP-ULA prefixes as well as
      moderate numbers of long and semi-aggregatable ADM-ULA prefixes. The
      system is kept stable and scalable through the s-ASBR / c-ASBR
      hub-and-spokes topology which ensures that mobility-related churn is not
      exposed to the core.</t>
    </section>

    <section anchor="maint" title="Stub AS Mobile Routing Services">
      <t>Stub ASes maintain intradomain routing information for mobile node
      clients, and are responsible for all localized mobility signaling
      without disturbing the BGP routing system. Clients can enlist the
      services of a candidate mobility service such as Mobile IPv6 (MIPv6)
      <xref target="RFC6275"/>, LISP <xref target="I-D.ietf-lisp-rfc6830bis"/>
      and AERO <xref target="I-D.templin-intarea-6706bis"/> according to the
      service offered by the stub AS. Further details of mobile routing
      services are out of scope for this document.</t>
    </section>

    <section anchor="implement" title="Implementation Status">
      <t>The BGP routing topology described in this document has been modeled
      in realistic network emulations showing that at least 1 million MNP-ULAs
      can be propagated to each c-ASBR even on lightweight virtual machines.
      No BGP routing protocol extensions need to be adopted.</t>
    </section>

    <section anchor="iana" title="IANA Considerations">
      <t>This document does not introduce any IANA considerations.</t>
    </section>

    <section anchor="secure" title="Security Considerations">
      <t>ATN/IPS ASBRs on the open Internet are susceptible to the same attack
      profiles as for any Internet nodes. For this reason, ASBRs should employ
      physical security and/or IP securing mechanisms such as IPsec <xref
      target="RFC4301"/>, TLS <xref target="RFC5246"/>, WireGuard, etc.</t>

      <t>ATN/IPS ASBRs present targets for Distributed Denial of Service
      (DDoS) attacks. This concern is no different than for any node on the
      open Internet, where attackers could send spoofed packets to the node at
      high data rates. This can be mitigated by connecting ATN/IPS ASBRs over
      dedicated links with no connections to the Internet and/or when ASBR
      connections to the Internet are only permitted through well-managed
      firewalls.</t>

      <t>ATN/IPS s-ASBRs should institute rate limits to protect low data rate
      aviation data links from receiving DDoS packet floods.</t>

      <t>BGP protocol message exchanges and control message exchanges used for
      route optimization must be secured to ensure the integrity of the
      system-wide routing information base.</t>

      <t>This document does not include any new specific requirements for
      mitigation of DDoS.</t>
    </section>

    <section anchor="ack" title="Acknowledgements">
      <t>This work is aligned with the FAA as per the SE2025 contract number
      DTFAWA-15-D-00030.</t>

      <t>This work is aligned with the NASA Safe Autonomous Systems Operation
      (SASO) program under NASA contract number NNA16BD84C.</t>

      <t>This work is aligned with the Boeing Commercial Airplanes (BCA)
      Internet of Things (IoT) and autonomy programs.</t>

      <t>This work is aligned with the Boeing Information Technology (BIT)
      MobileNet program.</t>

      <t>The following individuals contributed insights that have improved the
      document: Ahmad Amin, Erik Kline, Hubert Kuenig, Tony Li, Alexandre
      Petrescu, Pascal Thubert, Tony Whyman.</t>
    </section>
  </middle>

  <back>
    <references title="Normative References">
      <?rfc include="reference.RFC.8200"?>

      <?rfc include="reference.RFC.4291"?>

      <?rfc include="reference.RFC.4271"?>

      <?rfc include="reference.RFC.4456"?>

      <?rfc include="reference.RFC.4443"?>

      <?rfc include="reference.RFC.5880"?>

      <?rfc include="reference.RFC.4193"?>

      <?rfc include="reference.RFC.2473"?>
    </references>

    <references title="Informative References">
      <?rfc include="reference.I-D.templin-6man-omni-interface"?>

      <?rfc include="reference.RFC.4301"?>

      <?rfc include="reference.RFC.5246"?>

      <reference anchor="ATN">
        <front>
          <title>The OMNI Interface - An IPv6 Air/Ground Interface for Civil
          Aviation, IETF Liaison Statement #1676,
          https://datatracker.ietf.org/liaison/1676/</title>

          <author fullname="Vaughn Maiolla" initials="V." surname="Maiolla">
            <organization/>
          </author>

          <date day="3" month="March" year="2020"/>
        </front>
      </reference>

      <reference anchor="ATN-IPS">
        <front>
          <title>ICAO Document 9896 (Manual on the Aeronautical
          Telecommunication Network (ATN) using Internet Protocol Suite (IPS)
          Standards and Protocol), Draft Edition 3 (work-in-progress)</title>

          <author fullname="International Civil Aviation Organization"
                  initials="ICAO" surname="WG-I">
            <organization/>
          </author>

          <date day="10" month="December" year="2020"/>
        </front>
      </reference>

      <reference anchor="CBB">
        <front>
          <title>Global IP Network Mobility using Border Gateway Protocol
          (BGP),
          http://www.quark.net/docs/Global_IP_Network_Mobility_using_BGP.pdf</title>

          <author fullname="Andrew Dul" initials="A" surname="Dul">
            <organization/>
          </author>

          <date month="March" year="2006"/>
        </front>
      </reference>

      <reference anchor="BGP">
        <front>
          <title>BGP in 2015, http://potaroo.net</title>

          <author fullname="Geoff Huston" initials="G." surname="Huston">
            <organization/>
          </author>

          <date month="January" year="2016"/>
        </front>
      </reference>

      <reference anchor="BGP2">
        <front>
          <title>BGP Instability Report,
          http://bgpupdates.potaroo.net/instability/bgpupd.html</title>

          <author fullname="Geoff Huston" initials="G." surname="Huston">
            <organization/>
          </author>

          <date month="May" year="2017"/>
        </front>
      </reference>

      <?rfc include="reference.RFC.2784"?>

      <?rfc include="reference.RFC.6793"?>

      <?rfc include="reference.I-D.templin-intarea-6706bis"?>

      <?rfc include="reference.RFC.6275"?>

      <?rfc include="reference.I-D.ietf-lisp-rfc6830bis"?>
    </references>

    <section title="BGP Convergence Considerations">
      <t>Experimental evidence has shown that BGP convergence time required
      for when an MNP-ULA is asserted at a new location or withdrawn from an
      old location can be several hundred milliseconds even under optimal AS
      peering arrangements. This means that packets in flight destined to an
      MNP-ULA route that has recently been changed can be (mis)delivered to an
      old s-ASBR after a Client has moved to a new s-ASBR.</t>

      <t>To address this issue, the old s-ASBR can maintain temporary state
      for a "departed" Client that includes an OAL address for the new s-ASBR.
      The OAL address never changes since ASBRs are fixed infrastructure
      elements that never move. Hence, packets arriving at the old s-ASBR can
      be forwarded to the new s-ASBR while the BGP routing system is still
      undergoing reconvergence. Therefore, as long as the Client associates
      with the new s-ASBR before it departs from the old s-ASBR (while
      informing the old s-ASBR of its new location) packets in flight during
      the BGP reconvergence window are accommodated without loss.</t>
    </section>

    <section title="Change Log">
      <t>&lt;&lt; RFC Editor - remove prior to publication &gt;&gt;</t>

      <t>Changes from -05 to -06:<list style="symbols">
          <t>OMNI interface introduced</t>

          <t>Version and reference update.</t>
        </list>Changes from -04 to -05:<list style="symbols">
          <t>Version and reference update.</t>
        </list>Changes from -03 to -04:<list style="symbols">
          <t>added discussion of Bidirectional Forwarding Detection (BFD).</t>
        </list>Changes from -02 to -03:<list style="symbols">
          <t>added reference to ICAO A/G interface specification.</t>
        </list>Changes from -01 to -02:<list style="symbols">
          <t>introduced the SPAN and the concept of Internetwork
          partitioning</t>

          <t>new terms "ANET" (for (Radio) Access Network) and "INET" (for
          Internetworking underlay)</t>

          <t>new appendix on BGP convergence considerations</t>
        </list>Changes from -00 to -01:<list style="symbols">
          <t>incorporated clarifications due to list comments and
          questions.</t>

          <t>new section 7 on Stub AS Mobile Routing Services</t>

          <t>updated references, and included new reference for MIPv6 and
          LISP</t>
        </list></t>

      <t>Status as of 08/30/2018:</t>

      <t><list style="symbols">
          <t>'draft-templin-atn-bgp' becomes 'draft-ietf-rtgwg-atn-bgp'</t>
        </list></t>
    </section>
  </back>
</rfc>
