idnits 2.17.1 draft-ietf-lsvr-lsoe-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (December 24, 2018) is 1940 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '256' on line 389 -- Looks like a reference, but probably isn't: '4' on line 418 == Outdated reference: A later version (-18) exists of draft-ietf-idr-bgp-ls-segment-routing-ext-11 == Outdated reference: A later version (-19) exists of draft-ietf-idr-bgpls-segment-routing-epe-17 == Outdated reference: A later version (-29) exists of draft-ietf-lsvr-bgp-spf-04 ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) ** Obsolete normative reference: RFC 7752 (Obsoleted by RFC 9552) Summary: 2 errors (**), 0 flaws (~~), 5 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group R. Bush 3 Internet-Draft Arrcus & IIJ 4 Intended status: Standards Track R. Austein 5 Expires: June 27, 2019 K. Patel 6 Arrcus 7 December 24, 2018 9 Link State Over Ethernet 10 draft-ietf-lsvr-lsoe-00 12 Abstract 14 Used in Massive Data Centers (MDCs), BGP-SPF and similar protocols 15 need link neighbor discovery, link encapsulation data, and Layer 2 16 liveness. The Link State Over Ethernet protocol provides link 17 discovery, exchanges supported encapsulations (IPv4, IPv6, ...), 18 discovers encapsulation addresses (Layer 3 / MPLS identifiers) over 19 raw Ethernet, and provides layer 2 liveness checking. The interface 20 data are pushed directly to a BGP-LS API, obviating the need for 21 centralized controller architectures. This protocol is intended to 22 be more widely applicable to other upper layer routing protocols 23 which need link discovery and characterisation. 25 Requirements Language 27 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 28 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" are to 29 be interpreted as described in [RFC2119] only when they appear in all 30 upper case. They may also appear in lower or mixed case as English 31 words, without normative meaning. See [RFC8174]. 33 Status of This Memo 35 This Internet-Draft is submitted in full conformance with the 36 provisions of BCP 78 and BCP 79. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF). Note that other groups may also distribute 40 working documents as Internet-Drafts. The list of current Internet- 41 Drafts is at https://datatracker.ietf.org/drafts/current/. 43 Internet-Drafts are draft documents valid for a maximum of six months 44 and may be updated, replaced, or obsoleted by other documents at any 45 time. It is inappropriate to use Internet-Drafts as reference 46 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on June 27, 2019. 50 Copyright Notice 52 Copyright (c) 2018 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents 57 (https://trustee.ietf.org/license-info) in effect on the date of 58 publication of this document. Please review these documents 59 carefully, as they describe your rights and restrictions with respect 60 to this document. Code Components extracted from this document must 61 include Simplified BSD License text as described in Section 4.e of 62 the Trust Legal Provisions and are provided without warranty as 63 described in the Simplified BSD License. 65 Table of Contents 67 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 68 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 69 3. Background . . . . . . . . . . . . . . . . . . . . . . . . . 4 70 4. Top Level Overview . . . . . . . . . . . . . . . . . . . . . 5 71 5. Ethernet to Ethernet Protocols . . . . . . . . . . . . . . . 6 72 5.1. Inter-Link Ether Protocol Overview . . . . . . . . . . . 6 73 6. Transport Layer . . . . . . . . . . . . . . . . . . . . . . . 8 74 7. The Checksum . . . . . . . . . . . . . . . . . . . . . . . . 8 75 8. TLV PDUs . . . . . . . . . . . . . . . . . . . . . . . . . . 10 76 9. HELLO . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 77 10. OPEN . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 78 11. ACK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 79 11.1. Retransmission . . . . . . . . . . . . . . . . . . . . . 13 80 12. The Encapsulations . . . . . . . . . . . . . . . . . . . . . 13 81 12.1. The Encapsulation PDU Skeleton . . . . . . . . . . . . . 14 82 12.2. Prim/Loop Flags . . . . . . . . . . . . . . . . . . . . 15 83 12.3. IPv4 Encapsulation . . . . . . . . . . . . . . . . . . . 15 84 12.4. IPv6 Encapsulation . . . . . . . . . . . . . . . . . . . 16 85 12.5. MPLS Label List . . . . . . . . . . . . . . . . . . . . 16 86 12.6. MPLS IPv4 Encapsulation . . . . . . . . . . . . . . . . 16 87 12.7. MPLS IPv6 Encapsulation . . . . . . . . . . . . . . . . 17 88 13. KEEPALIVE - Layer 2 Liveness . . . . . . . . . . . . . . . . 18 89 14. Layers 2.5 and 3 Liveness . . . . . . . . . . . . . . . . . . 19 90 15. The North/South Protocol . . . . . . . . . . . . . . . . . . 19 91 15.1. Use BGP-LS as Much as Possible . . . . . . . . . . . . . 19 92 15.2. Extensions to BGP-LS . . . . . . . . . . . . . . . . . . 20 93 16. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 20 94 16.1. HELLO Discussion . . . . . . . . . . . . . . . . . . . . 20 95 16.2. HELLO versus KEEPALIVE . . . . . . . . . . . . . . . . . 20 96 17. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 21 97 18. Security Considerations . . . . . . . . . . . . . . . . . . . 21 98 19. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 99 20. IEEE Considerations . . . . . . . . . . . . . . . . . . . . . 22 100 21. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 22 101 22. References . . . . . . . . . . . . . . . . . . . . . . . . . 22 102 22.1. Normative References . . . . . . . . . . . . . . . . . . 22 103 22.2. Informative References . . . . . . . . . . . . . . . . . 23 104 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24 106 1. Introduction 108 The Massive Data Center (MDC) environment presents unusual problems 109 of scale, e.g. O(10,000) devices, while its homogeneity presents 110 opportunities for simple approaches. Approaches such as Jupiter 111 Rising [JUPITER] use a central controller to deal with scaling, while 112 BGP-SPF [I-D.ietf-lsvr-bgp-spf] provides massive scale-out without 113 centralization using a tried and tested scalable distributed control 114 plane, offering a scalable routing solution in Clos and similar 115 environments. But BGP-SPF and similar higher level device-spanning 116 protocols need link state and addressing data from the network to 117 build the routing topology. LLDP has scaling issues, e.g. in 118 extending a message beyond 1,500 bytes. 120 Link State Over Ethernet (LSOE) provides brutally simple mechanisms 121 for devices to 123 o Discover each other's Layer 2 (MAC) Addresses, 125 o Run Layer 2 keep-alive messages for liveness continuity, 127 o Discover each other's unique IDs (ASN, RouterID, ...), 129 o Discover mutually supported encapsulations, e.g. IP/MPLS, 131 o Discover Layer 3 and/or MPLS addressing of interfaces of the link 132 encapsulations, 134 o Enable layer 3 link liveness such as BFD, and finally 136 o Present these data, using a very restricted profile of a BGP-LS 137 [RFC7752] API, to BGP-SPF which computes the topology and builds 138 routing and forwarding tables. 140 This protocol may be more widely applicable to a range of routing and 141 similar protocols which need link discovery and characterisation. 143 2. Terminology 145 Even though it concentrates on the Ethernet layer, this document 146 relies heavily on routing terminology. The following are some 147 possibly confusing terms: 149 ASN: Autonomous System Number [RFC4271], a BGP identifier for 150 an originator of Layer 3 routes, particularly BGP 151 announcements. 152 BGP-LS: A mechanism by which link-state and TE information can be 153 collected from networks and shared with external 154 components using the BGP routing protocol. See [RFC7752]. 155 BGP-SPF A hybrid protocol using BGP transport but a Dijkstra SPF 156 decision process. See [I-D.ietf-lsvr-bgp-spf]. 157 Clos: A hierarchic subset of a crossbar switch topology commonly 158 used in data centers. 159 Datagram: The LSOE content of a single Ethernet frame. A full LSOE 160 PDU may be packaged in multiple Datagrams. 161 Encapsulation: Address Family Indicator and Subsequent Address 162 Family Indicator (AFI/SAFI). I.e. classes of addresses 163 such as IPv4, IPv6, MPLS, ... 164 Frame: An Ethernet Layer 2 packet. 165 MAC Address: Media Access Control Address, essentially an Ethernet 166 address, six octets. 167 MDC: Massive Data Center, commonly thousands of TORs. 168 PDU: Protocol Data Unit, an LSOE application layer message. A 169 PDU may need to be broken into multiple Datagrams to make 170 it through MTU or other restrictions. 171 RouterID: An 32-bit identifier unique in the current routing domain, 172 see [RFC4271] updated by [RFC6286]. 173 Session: An established, via OPEN PDUs, session between two LSOE 174 capable devices, 175 SPF: Shortest Path First, an algorithm for finding the shortest 176 paths between nodes in a graph; AKA Dijkstra's algorithm. 177 TOR: Top Of Rack switch, aggregates the servers in a rack and 178 connects to aggregation layers of the Clos tree, AKA the 179 Clos spine. 180 ZTP: Zero Touch Provisioning gives devices initial addresses, 181 credentials, etc. on boot/restart. 183 3. Background 185 LSOE assumes a datacenter scale and topology, but can accommodate 186 richer topologies which contain potential cycles. 188 While LSOE is designed for the MDC, there are no inherent reasons it 189 could not run on a WAN; though, as it is simply a discovery protocol, 190 it is not clear that this would be useful. The authentication and 191 authorisation needed to run safely on the WAN are not provided in 192 detail in this version of the protocol, although future versions/ 193 extensions could expend on them. 195 LSOE assumes a new IEEE assigned EtherType (TBD). 197 4. Top Level Overview 199 o Devices discover each other on Ethernet links 201 o MAC addresses and Link State are exchanged over Ethernet 203 o Layer 2 Liveness Checks are begun 205 o Encapsulation data are exchanged and IP-Level Liveness Checks done 207 o A BGP-like protocol is assumed to use these data to discover and 208 build a topology database 210 +-------------------+ +-------------------+ +-------------------+ 211 | Device | | Device | | Device | 212 | | | | | | 213 |+-----------------+| |+-----------------+| |+-----------------+| 214 || || || || || || 215 || BGP-SPF <+---+> BGP-SPF <+---+> BGP-SPF || 216 || || || || || || 217 |+--------^--------+| |+--------^--------+| |+--------^--------+| 218 | | | | | | | | | 219 | | | | | | | | | 220 |+--------+--------+| |+--------+--------+| |+--------+--------+| 221 || L2 Liveness || || L2 Liveness || || L2 Liveness || 222 || Encapsulations || || Encapsulations || || Encapsulations || 223 || Addresses || || Addresses || || Addresses || 224 |+--------^--------+| |+--------^--------+| |+--------^--------+| 225 | | | | | | | | | 226 | | | | | | | | | 227 |+--------v--------+| |+--------v--------+| |+--------v--------+| 228 || || || || || || 229 || Ether PDUs <+---+> Ether PDUs <+---+> Ether PDUs || 230 || || || || || || 231 |+-----------------+| |+-----------------+| |+-----------------+| 232 +-------------------+ +-------------------+ +-------------------+ 234 There are two protocols, the Ethernet discovery and the interface to 235 the upper level BGP-like protocol: 237 o Layer 2 Ethernet protocols are used to exchange Layer 2 data, i.e. 238 MAC addresses, and layer 2.5 and 3 identifiers (not payloads), 239 i.e. ASNs, Encapsulations, and interface addresses. 241 o A Link Layer to BGP API presents these data up the stack to a BGP 242 protocol or an other device-spanning upper layer protocol, 243 presenting them using the BGP-LS BGP-like data format. 245 The upper layer BGP family routing protocols cross all the devices, 246 though they are not part of these LSOE protocols. 248 To simplify this document, Layer 2 Ethernet framing is not shown. 250 5. Ethernet to Ethernet Protocols 252 Two devices discover each other and their respective MAC addresses by 253 sending multicast HELLO PDUs (Section 9). To allow discovery of new 254 devices coming up on a multi-link topology, devices send periodic 255 HELLOs forever, see Section 16.1. 257 Once a new device is recognized, both devices attempt to negotiate 258 and establish peering by sending unicast OPEN PDUs (Section 10). In 259 an established peering, Encapsulations (Section 12) may be announced 260 and modified. When two devices on a link have compatible 261 Encapsulations and addresses, i.e. the same AFI/SAFI and the same 262 subnet, the link is announced via the BGP-LS API. 264 5.1. Inter-Link Ether Protocol Overview 266 The HELLO, Section 9, is a priming message. It is an Ethernet 267 multicast frame with a small LSOE PDU with the simple goal of 268 discovering the Ethernet MAC address(es) of devices reachable via an 269 interface. 271 The HELLO and OPEN, Section 10, PDUs, which are used to discover and 272 exchange MAC address and IDs, are mandatory; other PDUs are optional; 273 though at least one encapsulation MUST be agreed at some point. 275 The following is a ladder-style sketch of the Ethernet protocol 276 exchanges: 278 | HELLO | MAC Address discovery 279 |---------------------------->| 280 | HELLO | Mandatory 281 |<----------------------------| 282 | | 283 | | 284 | OPEN | MACs, IDs, and Capabilities 285 |---------------------------->| 286 | OPEN | Mandatory 287 |<----------------------------| 288 | | 289 | | 290 | Interface IPv4 Addresses | Interface IPv4 Addresses 291 |---------------------------->| Optional 292 | ACK | 293 |<----------------------------| 294 | | 295 | Interface IPv4 Addresses | 296 |<----------------------------| 297 | ACK | 298 |---------------------------->| 299 | | 300 | | 301 | Interface IPv6 Addresses | Interface IPv6 Addresses 302 |---------------------------->| Optional 303 | ACK | 304 |<----------------------------| 305 | | 306 | Interface IPv6 Addresses | 307 |<----------------------------| 308 | ACK | 309 |---------------------------->| 310 | | 311 | | 312 | Interface MPLSv4 Labels | Interface MPLSv4 Labels 313 |---------------------------->| Optional 314 | ACK | 315 |<----------------------------| 316 | | 317 | Interface MPLSv4 Labels | Interface MPLSv4 Labels 318 |<----------------------------| Optional 319 | ACK | 320 |---------------------------->| 321 | | 322 | | 323 | Interface MPLSv6 Labels | Interface MPLSv6 Labels 324 |---------------------------->| Optional 325 | ACK | 326 |<----------------------------| 327 | | 328 | Interface MPLSv6 Labels | Interface MPLSv6 Labels 329 |<----------------------------| Optional 330 | ACK | 331 |---------------------------->| 332 | | 333 | | 334 | LSOE KEEPALIVE | Layer 2 Liveness 335 |---------------------------->| Optional 336 | LSOE KEEPALIVE | 337 |<----------------------------| 339 6. Transport Layer 341 LSOE PDU are carried by a simple transport layer which allows long 342 PDUs to occupy multiple Ethernet frames. The LSOE data in each frame 343 is referred to as a Datagram. 345 The LSOE Transport Layer encapsulates each Datagram using a common 346 transport header. 348 0 1 2 3 349 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 350 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 351 | Version |L|Datagram Number| Datagram Length | 352 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 353 | Checksum | 354 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 356 The fields of the LSOE Transport Header are as follows: 358 Version: Version number of the protocol, currently 0. Values other 359 than 0 are treated as failure. 361 Datagram Number: 0..255, a monotonically increasing value, modulo 362 256, see [RFC1982]. 364 L: A bit that set to 1 if this Datagram is the last Datagram of the 365 PDU. For a PDU which fits in only one Datagram, it is set to one. 367 PDU Length: Total number of octets in the Datagram including all 368 payloads and fields. 370 Checksum: A 32 bit hash over the Datagram to detect bit flips, see 371 Section 7. 373 7. The Checksum 375 There is a reason conservative folk use a checksum in UDP. And as 376 many operators stretch to jumbo frames (over 1,500 octets) longer 377 checksums are the conservative approach. 379 For the purpose of computing a checksum, the checksum field itself is 380 assumed to be zero. 382 Sum up 32-bit unsigned ints in a 64-bit long, then take the high- 383 order section, shift it right, rotate, add it in, repeat until zero. 385 #include 386 #include 388 /* The F table from Skipjack, and it would work for the S-Box. */ 389 static const uint8_t sbox[256] = { 390 0xa3,0xd7,0x09,0x83,0xf8,0x48,0xf6,0xf4,0xb3,0x21,0x15,0x78, 391 0x99,0xb1,0xaf,0xf9,0xe7,0x2d,0x4d,0x8a,0xce,0x4c,0xca,0x2e, 392 0x52,0x95,0xd9,0x1e,0x4e,0x38,0x44,0x28,0x0a,0xdf,0x02,0xa0, 393 0x17,0xf1,0x60,0x68,0x12,0xb7,0x7a,0xc3,0xe9,0xfa,0x3d,0x53, 394 0x96,0x84,0x6b,0xba,0xf2,0x63,0x9a,0x19,0x7c,0xae,0xe5,0xf5, 395 0xf7,0x16,0x6a,0xa2,0x39,0xb6,0x7b,0x0f,0xc1,0x93,0x81,0x1b, 396 0xee,0xb4,0x1a,0xea,0xd0,0x91,0x2f,0xb8,0x55,0xb9,0xda,0x85, 397 0x3f,0x41,0xbf,0xe0,0x5a,0x58,0x80,0x5f,0x66,0x0b,0xd8,0x90, 398 0x35,0xd5,0xc0,0xa7,0x33,0x06,0x65,0x69,0x45,0x00,0x94,0x56, 399 0x6d,0x98,0x9b,0x76,0x97,0xfc,0xb2,0xc2,0xb0,0xfe,0xdb,0x20, 400 0xe1,0xeb,0xd6,0xe4,0xdd,0x47,0x4a,0x1d,0x42,0xed,0x9e,0x6e, 401 0x49,0x3c,0xcd,0x43,0x27,0xd2,0x07,0xd4,0xde,0xc7,0x67,0x18, 402 0x89,0xcb,0x30,0x1f,0x8d,0xc6,0x8f,0xaa,0xc8,0x74,0xdc,0xc9, 403 0x5d,0x5c,0x31,0xa4,0x70,0x88,0x61,0x2c,0x9f,0x0d,0x2b,0x87, 404 0x50,0x82,0x54,0x64,0x26,0x7d,0x03,0x40,0x34,0x4b,0x1c,0x73, 405 0xd1,0xc4,0xfd,0x3b,0xcc,0xfb,0x7f,0xab,0xe6,0x3e,0x5b,0xa5, 406 0xad,0x04,0x23,0x9c,0x14,0x51,0x22,0xf0,0x29,0x79,0x71,0x7e, 407 0xff,0x8c,0x0e,0xe2,0x0c,0xef,0xbc,0x72,0x75,0x6f,0x37,0xa1, 408 0xec,0xd3,0x8e,0x62,0x8b,0x86,0x10,0xe8,0x08,0x77,0x11,0xbe, 409 0x92,0x4f,0x24,0xc5,0x32,0x36,0x9d,0xcf,0xf3,0xa6,0xbb,0xac, 410 0x5e,0x6c,0xa9,0x13,0x57,0x25,0xb5,0xe3,0xbd,0xa8,0x3a,0x01, 411 0x05,0x59,0x2a,0x46 412 }; 414 /* non-normative example C code, constant time even */ 416 uint32_t sbox_checksum_32(const uint8_t *b, const size_t n) 417 { 418 uint32_t sum[4] = {0, 0, 0, 0}; 419 uint64_t result = 0; 420 for (size_t i = 0; i < n; i++) 421 sum[i & 3] += sbox[*b++]; 422 for (int i = 0; i < sizeof(sum)/sizeof(*sum); i++) 423 result = (result << 8) + sum[i]; 424 result = (result >> 32) + (result & 0xFFFFFFFF); 425 result = (result >> 32) + (result & 0xFFFFFFFF); 426 return (uint32_t) result; 427 } 429 8. TLV PDUs 431 The basic LSOE application layer PDU is a typical TLV (Type Length 432 Value) PDU. It may be broken into multiple Datagrams, see Section 6 434 0 1 2 3 435 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 436 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 437 | Type | PDU Length | | 438 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 439 | Value ... | 440 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 442 The fields of the basic LSOE header are as follows: 444 Type: An integer differentiating PDU payload types 446 0 - HELLO 447 1 - OPEN 448 2 - KEEPALIVE 449 3 - ACK 450 4 - IPv4 Announcement 451 5 - IPv6 Announcement 452 6 - MPLS IPv4 Announcement 453 7 - MPLS IPv6 Announcement 454 8-255 Reserved 456 PDU Length: Total number of octets in the PDU including all payloads 457 and fields 459 Value: Any application layer content of the LSOE PDU beyond the 460 type. 462 9. HELLO 464 The HELLO PDU is unique in that it is a multicast Ethernet frame. It 465 solicits response(s) from other device(s) on the link. See 466 Section 16.1 for why multicast is used. 468 All other LSOE PDUs are unicast Ethernet frames, as the peer's MAC 469 Address is known after the HELLO exchange. 471 When an interface is turned up on a device, it SHOULD issue a HELLO 472 periodically. The interval is set by configuration. 474 0 1 2 3 475 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 476 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 477 | Type = 0 | PDU Length = 9 | | 478 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 479 | MyMAC Address | 480 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 481 | | 482 +-+-+-+-+-+-+-+-+ 484 If more than one device responds, one adjacency is formed for each 485 unique (MAC address) response. LSOE treats the adjacencies as 486 separate links. 488 When a HELLO is received from a MAC address where there is no 489 established LSOE adjacency, the receiver SHOULD respond with an OPEN 490 PDU. The two devices establish an LSOE adjacency by exchanging OPEN 491 PDUs. 493 The PDU Length is the octet count of the entire PDU, including the 494 Type, the Datagram Length field itself, and the MyMAC Address 495 payload. 497 A particular MAC address SHOULD arrive on frames from only one 498 interface. 500 10. OPEN 502 Each device has learned the other's MAC address from the HELLO 503 exchange, see Section 9. Therefore the OPEN and subsequent PDUs are 504 unicast, as opposed to the HELLO's multicast, Ethernet frames. 506 0 1 2 3 507 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 508 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 509 | Type = 1 | PDU Length | | 510 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 511 | | 512 + + 513 | Local ID | 514 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 515 | | | 516 +-+-+-+-+-+-+-+-+ + 517 | Remote ID (or Zero) | 518 + +-+-+-+-+-+-+-+-+ 519 | | AttrCount | 520 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 521 | Attribute List ... | Auth Length | 522 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 523 | ... | Authentication Data ... | 524 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 526 An ID can be an ASN with high order bits zero, a classic RouterID 527 with high order bits zero, a catenation of the two, a 80-bit ISO 528 System-ID, or any other identifier unique to a single device in the 529 current routing space. IDs are big-endian. 531 When the local device sends an OPEN without knowing the remote 532 device's ID, the Remote ID MUST be zero. The Local ID MUST NOT be 533 zero. 535 AttrCount is the number of attributes in the Attribute List. 536 Attributes are single octets whose semantics are user-defined. 538 A node may have zero or more user-defined attributes, e.g. spine, 539 leaf, backbone, route reflector, arabica, ... 541 Attribute syntax and semantics are local to an operator or 542 datacenter; hence there is no global registry. Nodes exchange their 543 attributes only in the OPEN PDU. 545 Auth Length is a 16-bit field denoting the length in octets of the 546 Authentication Data, not including the Auth Length itself. If there 547 are no Authentication Data, the Auth Length MUST BE zero. 549 The Authentication Data are specific to the operational environment. 550 A failure to authenticate is a failure to start the LSOE session, and 551 HELLOs MUST BE restarted. 553 Once two devices know each other's MAC addresses, and have ACKed 554 eachother's OPEN PDUs, Layer 2 KEEPALIVEs (see Section 13) SHOULD be 555 started to ensure Layer 2 liveness and keep the session semantics 556 alive. The timing and acceptable drop of the KEEPALIVE PDUs SHOULD 557 be configured. 559 If a properly authenticated OPEN arrives from a device with which the 560 receiving device believes it already has an LSOE session (OPENs have 561 already been exchanged), the receiver MUST assume that the sending 562 device has been reset. All discovered data MUST BE withdrawn via the 563 BGP-LS API and the recipient MUST respond with a new OPEN. 565 11. ACK 567 0 1 2 3 568 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 569 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 570 | Type = 3 | Length = 4 | PDU Type | 571 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 573 The ACK acknowledges receipt of an OPEN or an Encapsulation PDU. 575 The PDU Type is the Type of the PDU being acknowledged, OPEN or one 576 of the Encapsulations. 578 11.1. Retransmission 580 If a PDU sender expects an ACK, e.g. for an OPEN or an Encapsulation, 581 and does not receive the ACK for a configurable time (default one 582 second), the sender resends the PDU. This cycle MAY be repeated a 583 configurable number of times (default three) before it is considered 584 a failure. The session is considered closed in case of an ACK 585 failure. 587 12. The Encapsulations 589 Once the devices know each other's MAC addresses, know each other's 590 upper layer identities, have means to ensure link state, etc., the 591 LSOE session is considered established, and the devices SHOULD 592 announce their interface encapsulation, addresses, (and labels). 594 The Encapsulation types the peers exchange may be IPv4 Announcement 595 (Section 12.3), IPv6 Announcement (Section 12.4), MPLS IPv4 596 Announcement (Section 12.6), MPLS IPv6 Announcement (Section 12.7), 597 and/or possibly others not defined here. 599 The sender of an Encapsulation PDU MUST NOT assume that the peer is 600 capable of the same Encapsulation Type. An ACK (Section 11) merely 601 acknowledges receipt. Only if both peers have sent the same 602 Encapsulation Type is it safe to assume that they are compatible for 603 that type. 605 Further, to consider a link of a type to formally be established so 606 that it may be pushed up to upper layer protocols, the addressing for 607 the type must be compatible, i.e. on the same IPvX subnet. 609 12.1. The Encapsulation PDU Skeleton 611 The header for all encapsulation PDUs is as follows: 613 0 1 2 3 614 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 615 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 616 | Type | PDU Length | Count | 617 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 618 | ... | Encapsulation List... | 619 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 621 The 16-bit Count is the number of Encapsulations in the Encapsulation 622 list. 624 If the length of an Encapsulation PDU exceeds the Datagram size limit 625 on media, the PDU is broken into multiple Datagrams. See Section 8. 627 The Receiver MUST acknowledge the Encapsulation PDU with a Type=3, 628 ACK PDU (Section 11) with the Encapsulation Type being that of the 629 encapsulation being announced, see Section 11. 631 If the Sender does not receive an ACK in one second, they SHOULD 632 retransmit. After a user configurable number of failures, the LSOE 633 session should be considered dead and the OPEN process SHOULD be 634 restarted. 636 An Encapsulation PDU describes zero or more addresses of the 637 encapsulation type. 639 An Encapsulation PDU of Type T replaces all previous encapsulations 640 of Type T. 642 To remove all encapsulations of Type T, the sender uses a Count of 643 zero. 645 If an interface has multiple addresses for an encapsulation type, one 646 address SHOULD be marked as primary, see Section 12.2. 648 If a loopback address needs to be exposed, e.g. for iBGP peering, 649 then it should be marked as such, see Section 12.2. 651 If a sender has multiple links on the same interface, separate data, 652 ACKs, etc. must be kept for each peer. 654 Over time, multiple Encapsulation PDUs may be sent for an interface 655 as configuration changes. 657 12.2. Prim/Loop Flags 659 0 1 2 3 ... 7 660 +---------------+---------------+---------------+---------------+ 661 | Primary | Loopback | Reserved ... | | 662 +---------------+---------------+---------------+---------------+ 664 Each Encapsulation interface address MAY be marked as a primary 665 address, and/or a loopback, in which case the respective bit is set 666 to one. 668 Only one address MAY be marked as primary for an encapsulation type. 670 12.3. IPv4 Encapsulation 672 The IPv4 Encapsulation describes a device's ability to exchange IPv4 673 packets on one or more subnets. It does so by stating the 674 interface's address and the prefix length. 676 0 1 2 3 677 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 678 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 679 | Type = 4 | PDU Length | Count | 680 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 681 | ... | PrimLoop Flags| IPv4 Address | 682 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 683 | ... | PrefixLen | PrimLoop Flags| 684 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 685 | IPv4 Address | 686 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 687 | PrefixLen | more ... | 688 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 690 The 16-bit Count is the number of IPv4 Encapsulations. 692 12.4. IPv6 Encapsulation 694 The IPv6 Encapsulation describes a device's ability to exchange IPv6 695 packets on one or more subnets. It does so by stating the 696 interface's address and the prefix length. 698 0 1 2 3 699 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 700 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 701 | Type = 5 | PDU Length | Count | 702 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 703 | ... | PrimLoop Flags| | 704 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 705 | | 706 + + 707 | | 708 + + 709 | IPv6 Address | 710 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 711 | | PrefixLen | more ... | 712 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 714 The 16-bit Count is the number of IPv6 Encapsulations. 716 12.5. MPLS Label List 718 As an MPLS enabled interface may have a label stack, see [RFC3032], a 719 variable length list of labels is needed. 721 0 1 2 3 722 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 723 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 724 | Label Count | Label | Exp |S| 725 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 726 | Label | Exp |S| more ... | 727 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 729 A Label Count of zero is an implicit withdraw of all labels for that 730 prefix on that interface. 732 12.6. MPLS IPv4 Encapsulation 734 The MPLS IPv4 Encapsulation describes a device's ability to exchange 735 labeled IPv4 packets on one or more subnets. It does so by stating 736 the interface's address and the prefix length. 738 0 1 2 3 739 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 740 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 741 | Type = 6 | PDU Length | Count | 742 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 743 | ... | PrimLoop Flags| MPLS Label List ... | 744 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 745 | ... | IPv4 Address | 746 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 747 | ... | PrefixLen | PrimLoop Flags| 748 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 749 | MPLS Label List ... | 750 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 751 | IPv4 Address | 752 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 753 | Prefix Len | more ... | 754 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 756 The 16-bit Count is the number of MPLSv6 Encapsulations. 758 12.7. MPLS IPv6 Encapsulation 760 The MPLS IPv6 Encapsulation describes a device's ability to exchange 761 labeled IPv6 packets on one or more subnets. It does so by stating 762 the interface's address and the prefix length. 764 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 765 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 766 | Type = 7 | PDU Length | Count | 767 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 768 | ... | PrimLoop Flags| MPLS Label List ... | 769 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 770 | ... | | 771 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 772 | | 773 + + 774 | | 775 + + 776 | IPv6 Address | 777 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 778 | | Prefix Len | PrimLoop Flags| 779 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 780 | MPLS Label List ... | 781 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 782 | | 783 + + 784 | | 785 + IPv6 Address + 786 | | 787 + + 788 | | 789 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 790 | Prefix Len | more ... | 791 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 793 The 16-bit Count is the number of MPLSv6 Encapsulations. 795 13. KEEPALIVE - Layer 2 Liveness 797 LSOE devices MUST beacon occasional Layer 2 KEEPALIVE PDUs to ensure 798 session continuity. 800 They SHOULD be beaconed at a configured frequency. One per second is 801 the default. Layer 3 liveness, such as BFD, will likely be more 802 aggressive. 804 If a KEEPALIVE is not received from a peer with which a receiver has 805 an open session for a configurable time (default one minute), the 806 session SHOULD BE presumed closed. The devices MAY keep 807 configuration state until a new session is established and new 808 Encapsulation PDUs are received. 810 0 1 2 811 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 812 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 813 | Type = 2 | Length = 3 | 814 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 816 14. Layers 2.5 and 3 Liveness 818 Ethernet liveness is continuously tested by KEEPALIVE PDUs, see 819 Section 13. As layer 2.5 or layer 3 connectivity could still break, 820 liveness above layer 2 SHOULD be frequently tested using BFD 821 ([RFC5880]) or a similar technique. 823 This protocol assumes that one or more Encapsulation addresses will 824 be used to ping, BFD, or whatever the operator configures. 826 15. The North/South Protocol 828 Thus far, a one-hop point-to-point link discovery protocol has been 829 defined. 831 The nodes know the unique node identifiers (ASNs, RouterIDs, ...) and 832 Encapsulations on each link interface. 834 Full topology discovery is not appropriate at the Ethernet layer, so 835 Dijkstra a la IS-IS etc. is assumed to be done by higher level 836 protocols. 838 Therefore the node identifiers, link Encapsulations, and state 839 changes are pushed North via a small subset of the BGP-LS API. The 840 upper layer routing protocol(s), e.g. BGP-SPF, learn and maintain 841 the topology, run Dijkstra, and build the routing database(s). 843 For example, if a neighbor's IPv4 Encapsulation address changes, the 844 devices seeing the change push that change Northbound. 846 15.1. Use BGP-LS as Much as Possible 848 BGP-LS [RFC7752] defines BGP-like Datagrams describing link state 849 (links, nodes, link prefixes, and many other things), and a new BGP 850 path attribute providing Northbound transport, all of which can be 851 ingested by upper layer protocols such as BGP-SPF; see Section 4 of 852 [I-D.ietf-lsvr-bgp-spf]. 854 For IPv4 links, TLVs 259 and 260 are used. For IPv6 links, TLVs 261 855 and 262. If there are multiple addresses on a link, multiple TLV 856 pairs are pushed North, having the same ID pairs. 858 15.2. Extensions to BGP-LS 860 The Northbound protocol needs a few minor extensions to BGP-LS. 861 Luckily, others have needed the same extensions. 863 Similarly to BGP-SPF, the BGP protocol is used in the Protocol-ID 864 field specified in table 1 of 865 [I-D.ietf-idr-bgpls-segment-routing-epe]. The local and remote node 866 descriptors for all NLRI are the ID's described in Section 10. This 867 is equivalent to an adjacency SID or a node SID if the address is a 868 loopback address. 870 Label Sub-TLVs from [I-D.ietf-idr-bgp-ls-segment-routing-ext] 871 Section 2.1.1, are used to associate one or more MPLS Labels with a 872 link. 874 16. Discussion 876 This section explores some trade-offs taken and some considerations. 878 16.1. HELLO Discussion 880 There is the question of whether to allow an intermediate switch to 881 be transparent to discovery. We consider that an interface on a 882 device is a Layer 2 or a Layer 3 interface. In theory it could be a 883 Layer 3 interface with no encapsulation or Layer 3 addressing 884 currently configured. 886 A device with multiple Layer 2 interfaces, traditionally called a 887 switch, may be used to forward frames and therefore packets from 888 multiple devices to one interface, I, on an LSOE speaking device. 889 Interface I could discover a peer J across the switch. Later, a 890 prospective peer K could come up across the switch. If I was not 891 still sending and listening for HELLOs, the potential peering with K 892 could not be discovered. Therefore, interfaces MUST continue to send 893 HELLOs as long as they are turned up. 895 16.2. HELLO versus KEEPALIVE 897 Both HELLO and KEEPALIVE are periodic. KEEPALIVE might be eliminated 898 in favor of keeping only HELLOs. But currently KEEPALIVE is unicast, 899 has a checksum, is acknowledged, and thus more firmly verifies 900 session existence. 902 This warrants discussion. 904 17. Open Issues 906 VLANs/SVIs/Subinterfaces 908 18. Security Considerations 910 The protocol as is MUST NOT be used outside a datacenter or similarly 911 closed environment due to lack of formal definition of the 912 authentication and authorisation mechanism. These will be worked on 913 in a later effort, likely using credentials configured using ZTP or 914 similar configuration automation. 916 Many MDC operators have a strange belief that physical walls and 917 firewalls provide sufficient security. This is not credible. All 918 MDC protocols need to be examined for exposure and attack surface. 920 It is generally unwise to assume that on the wire Ethernet is secure. 921 Strange/unauthorized devices may plug into a port. Mis-wiring is 922 very common in datacenter installations. A poisoned laptop might be 923 plugged into a device's port. 925 Malicious nodes/devices could mis-announce addressing, form malicious 926 sessions, etc. 928 For these reasons, the OPEN PDU's authentication data exchange SHOULD 929 be used. [ A mandatory to implement authentication is in 930 development. ] 932 19. IANA Considerations 934 This document requests the IANA create a registry for LSOE PDU Type, 935 which may range from 0 to 255. The name of the registry should be 936 LSOE-PDU-Type. The policy for adding to the registry is RFC Required 937 per [RFC5226], either standards track or experimental. The initial 938 entries should be the following: 940 PDU 941 Code PDU Name 942 ---- ------------------- 943 0 HELLO 944 1 OPEN 945 2 KEEPALIVE 946 3 ACK 947 4 IPv4 Announce / Withdraw 948 5 IPv6 Announce / Withdraw 949 6 MPLS IPv4 Announce / Withdraw 950 7 MPLS IPv6 Announce / Withdraw 951 8-255 Reserved 953 This document requests the IANA create a registry for LSOE PL Flag 954 Bits, which may range from 0 to 7. The name of the registry should 955 be LSOE-PL-Flag-Bits. The policy for adding to the registry is RFC 956 Required per [RFC5226], either standards track or experimental. The 957 initial entries should be the following: 959 Bit Bit Name 960 ---- ------------------- 961 0 Primary 962 1 Loopback 963 2-7 Reserved 965 20. IEEE Considerations 967 This document requires a new EtherType. 969 21. Acknowledgments 971 The authors thank Cristel Pelsser for multiple reviews, Jeff Haas for 972 review and comments, Joe Clarke for a useful review, John Scudder 973 deeply serious review and comments, Larry Kreeger for a lot of layer 974 2 clue, Martijn Schmidt for his contribution, Russ Housley for 975 checksum discussion and sBox, and Steve Bellovin for checksum advice. 977 22. References 979 22.1. Normative References 981 [I-D.ietf-idr-bgp-ls-segment-routing-ext] 982 Previdi, S., Talaulikar, K., Filsfils, C., Gredler, H., 983 and M. Chen, "BGP Link-State extensions for Segment 984 Routing", draft-ietf-idr-bgp-ls-segment-routing-ext-11 985 (work in progress), October 2018. 987 [I-D.ietf-idr-bgpls-segment-routing-epe] 988 Previdi, S., Talaulikar, K., Filsfils, C., Patel, K., Ray, 989 S., and J. Dong, "BGP-LS extensions for Segment Routing 990 BGP Egress Peer Engineering", draft-ietf-idr-bgpls- 991 segment-routing-epe-17 (work in progress), October 2018. 993 [I-D.ietf-lsvr-bgp-spf] 994 Patel, K., Lindem, A., Zandi, S., and W. Henderickx, 995 "Shortest Path Routing Extensions for BGP Protocol", 996 draft-ietf-lsvr-bgp-spf-04 (work in progress), December 997 2018. 999 [RFC1982] Elz, R. and R. Bush, "Serial Number Arithmetic", RFC 1982, 1000 DOI 10.17487/RFC1982, August 1996, 1001 . 1003 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1004 Requirement Levels", BCP 14, RFC 2119, 1005 DOI 10.17487/RFC2119, March 1997, 1006 . 1008 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 1009 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 1010 Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, 1011 . 1013 [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A 1014 Border Gateway Protocol 4 (BGP-4)", RFC 4271, 1015 DOI 10.17487/RFC4271, January 2006, 1016 . 1018 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 1019 IANA Considerations Section in RFCs", RFC 5226, 1020 DOI 10.17487/RFC5226, May 2008, 1021 . 1023 [RFC5880] Katz, D. and D. Ward, "Bidirectional Forwarding Detection 1024 (BFD)", RFC 5880, DOI 10.17487/RFC5880, June 2010, 1025 . 1027 [RFC6286] Chen, E. and J. Yuan, "Autonomous-System-Wide Unique BGP 1028 Identifier for BGP-4", RFC 6286, DOI 10.17487/RFC6286, 1029 June 2011, . 1031 [RFC7752] Gredler, H., Ed., Medved, J., Previdi, S., Farrel, A., and 1032 S. Ray, "North-Bound Distribution of Link-State and 1033 Traffic Engineering (TE) Information Using BGP", RFC 7752, 1034 DOI 10.17487/RFC7752, March 2016, 1035 . 1037 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1038 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1039 May 2017, . 1041 22.2. Informative References 1043 [JUPITER] Singh, A., Germano, P., Kanagala, A., Liu, H., Provost, 1044 J., Simmons, J., Tanda, E., Wanderer, J., HAP.lzle, U., 1045 Stuart, S., Vahdat, A., Ong, J., Agarwal, A., Anderson, 1046 G., Armistead, A., Bannon, R., Boving, S., Desai, G., and 1047 B. Felderman, "Jupiter rising", Communications of the 1048 ACM Vol. 59, pp. 88-97, DOI 10.1145/2975159, August 2016. 1050 Authors' Addresses 1052 Randy Bush 1053 Arrcus & IIJ 1054 5147 Crystal Springs 1055 Bainbridge Island, WA 98110 1056 United States of America 1058 Email: randy@psg.com 1060 Rob Austein 1061 Arrcus, Inc 1063 Email: sra@hactrn.net 1065 Keyur Patel 1066 Arrcus 1067 2077 Gateway Place, Suite #400 1068 San Jose, CA 95119 1069 United States of America 1071 Email: keyur@arrcus.com