idnits 2.17.1 draft-ietf-idr-rs-bfd-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 1 instance of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (March 11, 2017) is 2603 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) No issues found here. Summary: 0 errors (**), 0 flaws (~~), 3 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group R. Bush 3 Internet-Draft Internet Initiative Japan 4 Intended status: Standards Track J. Haas 5 Expires: September 12, 2017 J. Scudder 6 Juniper Networks, Inc. 7 A. Nipper 8 T. King, Ed. 9 DE-CIX Management GmbH 10 March 11, 2017 12 Making Route Servers Aware of Data Link Failures at IXPs 13 draft-ietf-idr-rs-bfd-02 15 Abstract 17 When route servers are used, the data plane is not congruent with the 18 control plane. Therefore, the peers on the Internet exchange can 19 lose data connectivity without the control plane being aware of it, 20 and packets are dropped on the floor. This document proposes the use 21 of BFD between the two peering routers to detect a data plane 22 failure, and then uses a newly defined BGP SAFI to signal the state 23 of the data link to the route server(s). 25 Requirements Language 27 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 28 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" are to 29 be interpreted as described in [RFC2119] only when they appear in all 30 upper case. They may also appear in lower or mixed case as English 31 words, without normative meaning. 33 Status of This Memo 35 This Internet-Draft is submitted in full conformance with the 36 provisions of BCP 78 and BCP 79. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF). Note that other groups may also distribute 40 working documents as Internet-Drafts. The list of current Internet- 41 Drafts is at http://datatracker.ietf.org/drafts/current/. 43 Internet-Drafts are draft documents valid for a maximum of six months 44 and may be updated, replaced, or obsoleted by other documents at any 45 time. It is inappropriate to use Internet-Drafts as reference 46 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on September 12, 2017. 50 Copyright Notice 52 Copyright (c) 2017 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents 57 (http://trustee.ietf.org/license-info) in effect on the date of 58 publication of this document. Please review these documents 59 carefully, as they describe your rights and restrictions with respect 60 to this document. Code Components extracted from this document must 61 include Simplified BSD License text as described in Section 4.e of 62 the Trust Legal Provisions and are provided without warranty as 63 described in the Simplified BSD License. 65 Table of Contents 67 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 68 2. Operation . . . . . . . . . . . . . . . . . . . . . . . . . . 3 69 2.1. Mutual Discovery of Route Server Client Next-Hops . . . . 3 70 2.2. Tracking Connectivity . . . . . . . . . . . . . . . . . . 4 71 3. Advertising Client Router Connectivity to the Route Server . 5 72 4. Advertising NHIB state in BGP . . . . . . . . . . . . . . . . 5 73 4.1. Using the RS-Reachable SAFI to carry NHIB state . . . . . 6 74 4.2. Specific Procedures for Route Server Clients . . . . . . 6 75 4.3. The RS-Reachable Control Extended Community . . . . . . . 6 76 5. Processing NHIB State Changes . . . . . . . . . . . . . . . . 7 77 5.1. Route Server Client Procedures for NHIB Changes . . . . . 7 78 5.2. Route Server Procedures for NHIB Changes . . . . . . . . 8 79 6. Utilizing Next Hop Unreachability Information at Client 80 Routers . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 81 7. Recommendations for Using BFD . . . . . . . . . . . . . . . . 9 82 8. Bootstrapping . . . . . . . . . . . . . . . . . . . . . . . . 11 83 9. Other Considerations . . . . . . . . . . . . . . . . . . . . 11 84 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 85 11. Security Considerations . . . . . . . . . . . . . . . . . . . 11 86 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 12 87 12.1. Normative References . . . . . . . . . . . . . . . . . . 12 88 12.2. Informative References . . . . . . . . . . . . . . . . . 12 89 Appendix A. Summary of Adj-NHIB-In state . . . . . . . . . . . . 13 90 Appendix B. Summary of Document Changes . . . . . . . . . . . . 13 91 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 13 93 1. Introduction 95 In configurations (typically Internet Exchange Points (IXPs)) where 96 EBGP routing information is exchanged between client routers through 97 the agency of a route server [RFC7947], but traffic is exchanged 98 directly, operational issues can arise when partial data plane 99 connectivity exists among the route server client routers. Since the 100 data plane is not congruent with the control plane, the client 101 routers on the IXP can lose data connectivity without the control 102 plane - the route server - being aware of it, resulting in 103 significant data loss. 105 To remedy this, two basic problems need to be solved: 107 1. Client routers must have a means of verifying connectivity 108 amongst themselves, and 109 2. Client routers must have a means of communicating the knowledge 110 of the failure back to the route server. 112 The first can be solved by application of Bidirectional Forwarding 113 Detection [RFC5880]. The second can be solved by exchanging BGP 114 routes which use the RS-Reachable SAFI defined in this document. 116 Throughout this document, we generally assume that the route server 117 being discussed is able to represent different RIBs towards different 118 clients, as discussed in section 2.3.2.1. [RFC7947]. These 119 procedures (other than the use of BFD to track next hop reachability) 120 have limited value if this is not the case. 122 2. Operation 124 Below, we detail procedures where a route server tells its client 125 routers about other client nexthops by sending it RS-Reachable 126 routes, the client router verifies connectivity to those other client 127 routers using BFD and communicates its findings back to the route 128 server using RS-Reachable routes. The route server uses the received 129 routes with RS-Reachable SAFI as input to the route selection process 130 it performs on behalf of the client. 132 2.1. Mutual Discovery of Route Server Client Next-Hops 134 Strictly speaking, a route server client does not need to know of 135 other control-plane clients. For validation purposes, it only needs 136 to know the set of next hops the route server might choose to send to 137 it; i.e., to know all potential forwarding plane relationships. 139 This requirement amounts to knowing the BGP next hops the route 140 server is aware of for the particular per-client Loc-RIB (see section 141 2.3.2.1. [RFC7947]). We introduce a new table for each client to 142 store known next hops, their compatibility with this proposed 143 solution and their learned reachability. We call these tables per- 144 client Next Hop Information Base (NHIB). The NHIB is communicated to 145 the Route Server using RS-Reachable routes. 147 +--------------------------------------------------------+ 148 | +------------+ | 149 | | Per- | | 150 | .----------> Client |----------. | 151 | | | NHIB | | | 152 | | +------------+ | | 153 | +------+-----+ +-----v------+ | 154 | |Adj-NHIB-In | |Adj-NHIB-Out| | 155 | +------^-----+ Route Server +-----+------+ | 156 +----------|----------------------------------|----------+ 157 | | 158 | | 159 | | 160 | | 161 +----------|----------------------------------|----------+ 162 | +------+-----+ RS Client +-----v------+ | 163 | |Adj-NHIB-Out| |Adj-NHIB-In | | 164 | +------^-----+ +-----+------+ | 165 | | +------------+ | | 166 | | | | | | 167 | `----------+ NHIB <----------' | 168 | | | | 169 | +------------+ | 170 +--------------------------------------------------------+ 172 Figure 1: Route Server, RS Client, and NHIBs with In/Out Queues 174 The NHIB is not large; the set of routers in the ASs the client has 175 asked the RS to maintain in its view. 177 At the route server, the Adj-NHIB-Out for each client is populated 178 with the next hops from its Loc-RIB. If the BGP capabilities learned 179 during BGP session setup identify a next hop as compatible with this 180 proposal, this is reflected in the NHIB. Initially, it is assumed 181 that the client router is able to reach its next hops which is stored 182 in the NHIB. If a next hop is added to the NHIB for a particular 183 client, a route SHOULD be added to the router server's Adj-NHIB-Out. 185 A route server client SHOULD use BFD [RFC5880] (or other means beyond 186 the scope of this document) to track forwarding plane connectivity to 187 each next hop in its NHIB as received from the RS's Adj-NHIB-Out. 189 2.2. Tracking Connectivity 191 For each next hop in the NHIB received from the route server (called 192 Adj-NHIB-In), the client router SHOULD use some means to confirm that 193 data plane connectivity exists to that next hop. Here we assume BFD. 195 The client router maintains its own NHIB in order to keep track of 196 its (potential) next hops and their reachability. The NHIB is 197 updated according to the Adj-NHIB-In and client routers own tests to 198 verify connectivity to next hops. 200 For each next hop in the Adj-NHIB-In received from the route server, 201 the client router SHOULD attempt to establish a BFD session if one is 202 not already established, and track the reachability of this next hop. 204 For each nexthop that is determined to be reachable, an entry should 205 be added in the client router's Adj-NHIB-Out to be advertised to the 206 route server. Similarly, when that nexthop is determined to no 207 longer be reachable, the entry should be removed from the client 208 router's Adj-NHIB-Out. This may also be done as a result of policy 209 even if connectivity exists. 211 If the client can not establish a BFD session with an entry in its 212 NHIB, the next hop is put it in the Adj-NHIB-Out for backward 213 compatibility. 215 If the test of connectivity between one client router and another 216 client router fails, the client router detecting this failure should 217 perform the connectivity test for a configurable amount of time, 218 preferably 24 hours. If during this time no connectivity can be 219 restored no more testing is performed until manually changed or the 220 client router is rebooted. 222 3. Advertising Client Router Connectivity to the Route Server 224 As discussed above, a client router will advertise its Adj-NHIB-Out 225 to the route server. The route server SHOULD update the reachability 226 information of next hops in the client's NHIB table accordingly. 227 Furthermore, the route server SHOULD use reachability information 228 from the NHIB as input to its own decision process when computing the 229 Adj-RIB-Out for this client. This client-dependent Adj-RIB-Out is 230 then advertised to this client. In particular, the route server MUST 231 exclude any routes whose next hops the client has declared to be not 232 reachable. 234 4. Advertising NHIB state in BGP 236 Two distinct pieces of per-peer state have been identified in the 237 sections above: 239 o The set of next-hops for BGP routes received from the BGP speaker, 240 the Adj-NHIB-In. 241 o The set of next-hops the BGP speaker is advertising as reachable, 242 i.e., has potential connectivity to, the Adj-NHIB-Out. 244 4.1. Using the RS-Reachable SAFI to carry NHIB state 246 A new BGP SAFI, the RS-Reachable SAFI, is defined in this document. 247 It has been assigned a value TBD. A route server or a route server 248 client using the procedures in this document negotiate the RS- 249 Reachable SAFI for the IPv4 and/or IPv6 AFIs to carry NHIB entries. 251 NHIB entries are exchanged as host routes using the NLRI format 252 described in [RFC4271], section 4.3. If a NHIB entry for a given AFI 253 is received with an inappropriate prefix length, that NLRI MUST BE 254 ignored. 256 NHIB entries MUST NOT be propagated from one BGP peering session to 257 another; the routes are not transitive. To help enforce this 258 expected behavior, RS-Reachable routes MUST carry the NO_ADVERTISE 259 community [RFC1997]. RS-Reachable routes not carrying this community 260 MUST BE ignored. 262 If a NHIB entry is received from a BGP speaker and that entry is not 263 part of the sub-network for that BGP session, that NLRI MUST BE 264 ignored. This prevents erroneous BFD peering session being 265 provisioned outside of the IXP network. 267 4.2. Specific Procedures for Route Server Clients 269 A route server SHALL always create an entry in its Adj-NHIB-Out for 270 its clients that are peering with each other through the route 271 server, even if a next hop has not been received for this client. 272 This self-originated entry permits BFD sessions at the clients to be 273 provisioned even if the route exchange via the route server is 274 asymmetric and one router sends routes to the second router in the 275 route server view but not vice versa. 277 Route server clients are considered to be peering with each other if 278 the configuration of the route server permits routes from a given 279 pair of peers to be mutually exchanged through the route server. 281 4.3. The RS-Reachable Control Extended Community 283 0 1 2 3 284 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 285 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 286 | 0x43 | Sub-Type TBD1 | Reserved (Must be Zero) | 287 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 288 | Reserved (Must be Zero) | Flags |F| 289 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 290 The RS-Reachable Control Extended Community is used to signal 291 additional information in RS-Reachable NLRI. Currently, a two-octet 292 flag field is utilized for Flags. The remainder of the extended 293 community is currently reserved and its contents MUST be set to zero 294 when originated and SHOULD be ignored upon receipt. 296 A single flag is currently reserved in this proposal: 298 F: Flush received NHIB state. 300 5. Processing NHIB State Changes 302 5.1. Route Server Client Procedures for NHIB Changes 304 When entries are added to the a route server client's Adj-NHIB-In for 305 a route server peering session, it will then attempt to verify 306 connectivity to the BGP nexthop for that entry. The procedure 307 described in this specification utilizes BFD; other mechanisms are 308 permitted but are out of scope of this document. 310 If no existing BFD session exists to this nexthop, a BFD session is 311 provisioned to that IP address and the Adj-NHIB-In (In?) Reachable 312 state is set to Unknown. Since this session requires the remote BFD 313 session to also be provisioned, it may stay in the Down/AdminDown 314 state for a period of time. 316 If the client can not establish a BFD session with an entry in its 317 NHIB, the next hop is put it in the Adj-NHIB-Out as Reachable for 318 backward compatibility. 320 Once the BFD session moves to the Up state, the Adj-NHIB-In Reachable 321 state is set to Up. This NHIB entry is now eligible to be placed in 322 Adj-NHIB-Out table and distributed according to the procedures above. 323 Additionally, local BGP route selection may be impacted by this 324 state. See Section 6. 326 When the BFD session transitions out of the Up state to the Down 327 state, the Adj-NHIB-In Reachable state is set to Down. The NHIB 328 entry MUST be removed from the Adj-NHIB-Out table. This informs the 329 route server that the next hop is no longer reachable. 331 If the BFD session transitions out of the Up state to the AdminDown 332 state, the Adj-NHIB-In Reachable state is set to AdminDown. During 333 this transition, the NHIB entry is not be removed from the Adj-NHIB- 334 Out table. Instead, the RS-Reachable Extended Community is added to 335 the route with the F (flush) bit set. This signals the route server 336 should remove cached state for this entry. 338 The motivation for this behavior is that AdminDown could imply one of 339 two possible circumstances: 341 o The local BFD session has been deconfigured and BFD validation is 342 no longer possible. While the nexthop may still be usable, it is 343 no longer able to be determined using BFD whether that can happen. 344 Removing the entry from the Adj-NHIB-Out will inform the route 345 server that the next hop is no longer reachable and may adversely 346 impact the route server's view supplied to that route server 347 client. 348 o The remote BFD session has been deconfigured with similar impact. 350 An implementation of these procedures MUST provide an administrative 351 mechanism to clear such AdminDown entries from the Adj-NHIB-Out 352 table. 354 When entries are removed from the route server client's Adj-NHIB-In 355 for a route server peering session, the client MAY delay de- 356 provisioning the BFD peering session. If the client delays de- 357 provisioning the session, it should remove it if the BFD session 358 transitions to the Down or AdminDown states. The client should 359 remove the entry from its Adj-NHIB-Out table regardless of the state 360 of the BFD session. 362 5.2. Route Server Procedures for NHIB Changes 364 A route server is tracking two distinct types of next hop state for 365 its clients: 367 o The BGP next hops received from those clients' BGP routes. 368 o The Adj-NHIB-Out state from each client representing next hops to 369 which the clients believe they have connectivity. 371 The route-server will place the collection of received BGP next hops 372 from its clients into its per client Adj-NHIB-Out tables when at 373 least one of the route server peers that supports this procedure has 374 negotiated the RS-Reachable SAFI. It will then advertise them per 375 the procedures above. This informs the route server clients of the 376 available BGP nexthops visible to the route server supporting this 377 feature. 379 In the event that a given client that supports this feature does not 380 provide any routes containing BGP next hops that would be used to 381 populate an Adj-NHIB-Out entry, the route server SHOULD advertise an 382 entry for such a router using the provided self-originated entry. 383 This permits the provisioning of BFD peering sessions for continuity 384 check when route exchange via the route server is asymmetric and one 385 client has routes from a second client, but not vice-versa. 387 A route server will not generally delete NHIB entries learned in its 388 per client Adj-NHIB-In table when processing a withdraw from the 389 route server client. It derives the following information from the 390 presence and state, or absence, of an entry: 392 o When an NHIB entry is present, it means that the route server 393 client has noted the BGP next hop from the route server and has 394 validated connectivity to it. Such an entry has the Received 395 state of Active. 396 o When an entry is withdrawn but was previously present, it means 397 that the route server client previously had validated connectivity 398 to that next hop and NO LONGER has connectivity to it. Such an 399 entry has the Received state of Cached. The route server may 400 choose to adjust what routes are present in that client's view 401 (Adj-Rib-Out) based on that information according to local 402 capability and configuration. 403 o When an entry is missing, i.e. never has been seen, the route 404 server can't derive any information about the reachability of a 405 given next hop from the perspective of the route server client. 406 The route server SHOULD NOT negatively bias the client's view 407 according to this information. 409 However, if the route server receives an NHIB entry with the F 410 (flush) bit set the RS-Reachable Control Extended Community, it will 411 remove the entry from the Adj-NHIB-In table for that peer. 412 Similarly, if the entry is being removed because the peering session 413 with the client has closed, entries will also be removed. 415 6. Utilizing Next Hop Unreachability Information at Client Routers 417 A client router detecting an unreachable next hop signals this 418 information to the route server as described above. Also, it treats 419 the routes as unresolvable as per section 9.1.2.1 [RFC4271] and 420 proceeds with route selection as normal. 422 Changes in nexthop reachability via the above should apply mechanisms 423 to avoid unnecessary route flapping. Such mechanisms exist in IGP 424 implementations which should be applied to this scenario. 426 7. Recommendations for Using BFD 428 The RECOMMENDED way a client router can confirm the data plane 429 connectivity to its next hops is available, is the use of BFD in 430 asynchronous mode. Echo mode MAY be used if both client routers 431 running a BFD session support this. The use of authentication in BFD 432 is OPTIONAL as there is a certain level of trust between the 433 operators of the client routers at a particular IXP. If trust cannot 434 be assumed, it is recommended to use pair-wise keys (how this can be 435 achieved is outside the scope of this document). The ttl/hop limit 436 values as described in section 5 [RFC5881] MUST be obeyed in order to 437 shield BFD sessions against packets coming from outside the IXP. 439 There is interdependence between the functions described in this 440 document and BFD from an administrative point of view. To streamline 441 behaviour of different implementations the following are RECOMMENDED: 443 o If BFD is administratively shut down by the administrator of a 444 client router then the functions described in this document MUST 445 also be administratively shut down. 446 o If the administrator enables the functions described in this 447 document on a client router then BFD MUST be automatically 448 enabled. 450 The following values of the BFD configuration of client routers (see 451 section 6.8.1 [RFC5880]) are RECOMMENDED in order to allow fast 452 detection of lost data plane connectivity: 454 o DesiredMinTxInterval: 1,000,000 (microseconds) 455 o RequiredMinRxInterval: 1,000,000 (microseconds) 456 o DetectMult: 3 458 The configuration values above are a trade-off between fast detection 459 of data plane connectivity and the load client routers must handle 460 keeping up the BFD communication. Selecting smaller 461 DesiredMinTxInterval and RequiredMinRxInterval values generates 462 excessive BFD packets, especially at larger IXPs with many hundreds 463 of client routers. 465 The configuration values above were chosen to accept brief 466 interruptions in the data plane. Otherwise, if a BFD session detects 467 a brief data plane interruption to a particular client router, it 468 will signal to the route server that it should remove routes from 469 this client router and shortly thereafter to add the routes again. 470 This is disruptive and computationally expensive on the route server. 472 The configuration values above are also partially impacted by BGP 473 advertisement time in reaction to events from BFD. If the 474 configuration values are selected so that BFD detects data plane 475 interruptions faster than the BGP advertisement time, a data plane 476 connectivity flap could be detected by BFD but the route server is 477 not informed about it because BGP is not able to transport this 478 information quickly enough. 480 As discussed, finding good configuration values is hard, so a client 481 router administrator MAY select more appropriate values to meet the 482 special needs of a particular deployment. 484 8. Bootstrapping 486 During route server start-up, it does not know anything about 487 connectivity states between client routers. So, the route server 488 assumes optimistically that all client routers are able to reach each 489 other unless told otherwise. 491 9. Other Considerations 493 For purposes of routing stability, implementations may wish to apply 494 hysteresis ("holddown") to next hops that have transitioned from 495 reachable to unreachable and back. 497 10. IANA Considerations 499 IANA is requested to allocate a value from the Subsequent Address 500 Family Identifiers (SAFI) Parameters registry for this proposal. Its 501 Description in that registry shall bgp RS-Reachable with a Reference 502 of this RFC. 504 IANA is request to allocate a value from the Non-Transitive Opaque 505 Extended Community Sub-Types registry. Its Name will be "RS- 506 Reachable Control Extended Community" with a Reference of this RFC. 508 11. Security Considerations 510 The mechanism in this document permits route server clients to 511 influence the contents of the route server's Adj-Ribs-Out through its 512 reports of NHIB state using the Rs-Reachable SAFI. Since this state 513 is per-client, if a route server client is able to inject Rs- 514 Reachable routes for another route server's BGP session to a client, 515 it can cause the route server to select different forwarding than 516 otherwise expected. This issue may be mitigated using transport 517 security on its BGP session to route server clients. See [RFC4272]. 519 Should route server clients provision the RS-Reachable SAFI amongst 520 themselves, it would be an error but would have no undesired impact 521 on forwarding. It is incorrect provisioning for an IXP client which 522 is using a Route Server to have a BGP session with another IXP 523 client. Should they negotiate the RS-Reachable SAFI and send RS- 524 Reachable routes, this only serves to signal that BGP Speaker, when 525 not operating as a route server, to attempt to set verify 526 connectivity with the hosts in the received NLRI. While this may 527 potentially request a large number of sessions, the default BFD 528 timers prevent excess packets from being sent from inappropriately 529 provisioned sessions. 531 The reachability tests between route server clients themselves may be 532 a target for attack. Such attacks may include forcing a BFD session 533 Down through injecting false BFD state. A less likely attack 534 includes forcing a BFD session to stay Up when its real state is 535 Down. These attacks may be mitigated using the BFD security 536 mechanisms defined in [RFC5880]. 538 12. References 540 12.1. Normative References 542 [RFC1997] Chandra, R., Traina, P., and T. Li, "BGP Communities 543 Attribute", RFC 1997, DOI 10.17487/RFC1997, August 1996, 544 . 546 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 547 Requirement Levels", BCP 14, RFC 2119, 548 DOI 10.17487/RFC2119, March 1997, 549 . 551 [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A 552 Border Gateway Protocol 4 (BGP-4)", RFC 4271, 553 DOI 10.17487/RFC4271, January 2006, 554 . 556 [RFC5880] Katz, D. and D. Ward, "Bidirectional Forwarding Detection 557 (BFD)", RFC 5880, DOI 10.17487/RFC5880, June 2010, 558 . 560 [RFC5881] Katz, D. and D. Ward, "Bidirectional Forwarding Detection 561 (BFD) for IPv4 and IPv6 (Single Hop)", RFC 5881, 562 DOI 10.17487/RFC5881, June 2010, 563 . 565 [RFC7947] Jasinska, E., Hilliard, N., Raszuk, R., and N. Bakker, 566 "Internet Exchange BGP Route Server", RFC 7947, 567 DOI 10.17487/RFC7947, September 2016, 568 . 570 12.2. Informative References 572 [RFC4272] Murphy, S., "BGP Security Vulnerabilities Analysis", 573 RFC 4272, DOI 10.17487/RFC4272, January 2006, 574 . 576 Appendix A. Summary of Adj-NHIB-In state 578 The Adj-NHIB-In state is maintained per BGP peering session. It 579 consists of per-peer state and per-peer, per-nexthop state. 581 +-----------------------------------+----------------------------+ 582 | Client Role | (Route-Server | | 583 | | Route-Server-Client | 584 +-----------------------------------+----------------------------+ 585 Fig. 1 Per-peer Adj-NHIB-In Table State 587 +---------------------------+--------------------------------------+ 588 | NextHop |