idnits 2.17.1 draft-ietf-bier-te-arch-12.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 4 instances of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (January 2022) is 832 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '3' on line 1065 -- Looks like a reference, but probably isn't: '2' on line 1074 -- Looks like a reference, but probably isn't: '1' on line 1081 == Missing Reference: 'SI' is mentioned on line 2366, but not defined == Missing Reference: 'I' is mentioned on line 1134, but not defined == Missing Reference: 'VRF' is mentioned on line 2706, but not defined == Unused Reference: 'RFC6241' is defined on line 2931, but no explicit reference was found in the text == Unused Reference: 'RFC7950' is defined on line 2952, but no explicit reference was found in the text == Unused Reference: 'RFC8040' is defined on line 2961, but no explicit reference was found in the text == Outdated reference: A later version (-06) exists of draft-ietf-bier-te-yang-04 == Outdated reference: A later version (-27) exists of draft-ietf-teas-rfc3272bis-13 -- Obsolete informational reference (is this intentional?): RFC 7752 (Obsoleted by RFC 9552) Summary: 0 errors (**), 0 flaws (~~), 10 warnings (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group T.T.E. Eckert, Ed. 3 Internet-Draft Futurewei 4 Intended status: Standards Track M.M. Menth 5 Expires: 1 August 2022 University of Tuebingen 6 G.C. Cauchie 7 Bouygues Telecom 8 January 2022 10 Tree Engineering for Bit Index Explicit Replication (BIER-TE) 11 draft-ietf-bier-te-arch-12 13 Abstract 15 This memo describes per-packet stateless strict and loose path 16 steered replication and forwarding for "Bit Index Explicit 17 Replication" (BIER, RFC8279) packets. It is called BIER Tree 18 Engineering (BIER-TE) and is intended to be used as the path steering 19 mechanism for Traffic Engineering with BIER. 21 BIER-TE introduces a new semantic for "bit positions" (BP). They 22 indicate adjacencies of the network topology, as opposed to (non-TE) 23 BIER in which BPs indicate "Bit-Forwarding Egress Routers" (BFER). A 24 BIER-TE packets BitString therefore indicates the edges of the (loop- 25 free) tree that the packet is forwarded across by BIER-TE. BIER-TE 26 can leverage BIER forwarding engines with little changes. Co- 27 existence of BIER and BIER-TE forwarding in the same domain is 28 possible, for example by using separate BIER "sub-domains" (SDs). 29 Except for the optional routed adjacencies, BIER-TE does not require 30 a BIER routing underlay, and can therefore operate without depending 31 on an "Interior Gateway Routing protocol" (IGP). 33 As it operates on the same per-packet stateless forwarding 34 principles, BIER-TE can also be a good fit to support multicast path 35 steering in "Segment Routing" (SR) networks. 37 Status of This Memo 39 This Internet-Draft is submitted in full conformance with the 40 provisions of BCP 78 and BCP 79. 42 Internet-Drafts are working documents of the Internet Engineering 43 Task Force (IETF). Note that other groups may also distribute 44 working documents as Internet-Drafts. The list of current Internet- 45 Drafts is at https://datatracker.ietf.org/drafts/current/. 47 Internet-Drafts are draft documents valid for a maximum of six months 48 and may be updated, replaced, or obsoleted by other documents at any 49 time. It is inappropriate to use Internet-Drafts as reference 50 material or to cite them other than as "work in progress." 52 This Internet-Draft will expire on 5 July 2022. 54 Copyright Notice 56 Copyright (c) 2022 IETF Trust and the persons identified as the 57 document authors. All rights reserved. 59 This document is subject to BCP 78 and the IETF Trust's Legal 60 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 61 license-info) in effect on the date of publication of this document. 62 Please review these documents carefully, as they describe your rights 63 and restrictions with respect to this document. Code Components 64 extracted from this document must include Revised BSD License text as 65 described in Section 4.e of the Trust Legal Provisions and are 66 provided without warranty as described in the Revised BSD License. 68 Table of Contents 70 1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 3 71 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 5 72 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 5 73 2.1. Basic Examples . . . . . . . . . . . . . . . . . . . . . 5 74 2.2. BIER-TE Topology and adjacencies . . . . . . . . . . . . 8 75 2.3. Relationship to BIER . . . . . . . . . . . . . . . . . . 9 76 2.4. Accelerated/Hardware forwarding comparison . . . . . . . 11 77 3. Components . . . . . . . . . . . . . . . . . . . . . . . . . 11 78 3.1. The Multicast Flow Overlay . . . . . . . . . . . . . . . 12 79 3.2. The BIER-TE Control Plane . . . . . . . . . . . . . . . . 12 80 3.2.1. The BIER-TE Controller . . . . . . . . . . . . . . . 14 81 3.2.1.1. BIER-TE Topology discovery and creation . . . . . 14 82 3.2.1.2. Engineered Trees via BitStrings . . . . . . . . . 15 83 3.2.1.3. Changes in the network topology . . . . . . . . . 16 84 3.2.1.4. Link/Node Failures and Recovery . . . . . . . . . 16 85 3.3. The BIER-TE Forwarding Plane . . . . . . . . . . . . . . 16 86 3.4. The Routing Underlay . . . . . . . . . . . . . . . . . . 17 87 3.5. Traffic Engineering Considerations . . . . . . . . . . . 17 88 4. BIER-TE Forwarding . . . . . . . . . . . . . . . . . . . . . 18 89 4.1. The BIER-TE Bit Index Forwarding Table (BIFT) . . . . . . 18 90 4.2. Adjacency Types . . . . . . . . . . . . . . . . . . . . . 20 91 4.2.1. Forward Connected . . . . . . . . . . . . . . . . . . 21 92 4.2.2. Forward Routed . . . . . . . . . . . . . . . . . . . 21 93 4.2.3. ECMP . . . . . . . . . . . . . . . . . . . . . . . . 21 94 4.2.4. Local Decap(sulation) . . . . . . . . . . . . . . . . 22 96 4.3. Encapsulation / Co-existence with BIER . . . . . . . . . 22 97 4.4. BIER-TE Forwarding Pseudocode . . . . . . . . . . . . . . 23 98 4.5. Basic BIER-TE Forwarding Example . . . . . . . . . . . . 26 99 4.6. BFR Requirements for BIER-TE forwarding . . . . . . . . . 28 100 5. BIER-TE Controller Operational Considerations . . . . . . . . 29 101 5.1. Bit Position Assignments . . . . . . . . . . . . . . . . 29 102 5.1.1. P2P Links . . . . . . . . . . . . . . . . . . . . . . 29 103 5.1.2. BFER . . . . . . . . . . . . . . . . . . . . . . . . 29 104 5.1.3. Leaf BFERs . . . . . . . . . . . . . . . . . . . . . 29 105 5.1.4. LANs . . . . . . . . . . . . . . . . . . . . . . . . 31 106 5.1.5. Hub and Spoke . . . . . . . . . . . . . . . . . . . . 32 107 5.1.6. Rings . . . . . . . . . . . . . . . . . . . . . . . . 32 108 5.1.7. Equal Cost MultiPath (ECMP) . . . . . . . . . . . . . 33 109 5.1.8. Forward Routed adjacencies . . . . . . . . . . . . . 36 110 5.1.8.1. Reducing bit positions . . . . . . . . . . . . . 36 111 5.1.8.2. Supporting nodes without BIER-TE . . . . . . . . 37 112 5.1.9. Reuse of bit positions (without DNC) . . . . . . . . 37 113 5.1.10. Summary of BP optimizations . . . . . . . . . . . . . 38 114 5.2. Avoiding duplicates and loops . . . . . . . . . . . . . . 39 115 5.2.1. Loops . . . . . . . . . . . . . . . . . . . . . . . . 40 116 5.2.2. Duplicates . . . . . . . . . . . . . . . . . . . . . 40 117 5.3. Managing SI, sub-domains and BFR-ids . . . . . . . . . . 41 118 5.3.1. Why SI and sub-domains . . . . . . . . . . . . . . . 41 119 5.3.2. Assigning bits for the BIER-TE topology . . . . . . . 42 120 5.3.3. Assigning BFR-id with BIER-TE . . . . . . . . . . . . 43 121 5.3.4. Mapping from BFR to BitStrings with BIER-TE . . . . . 44 122 5.3.5. Assigning BFR-ids for BIER-TE . . . . . . . . . . . . 45 123 5.3.6. Example bit allocations . . . . . . . . . . . . . . . 45 124 5.3.6.1. With BIER . . . . . . . . . . . . . . . . . . . . 45 125 5.3.6.2. With BIER-TE . . . . . . . . . . . . . . . . . . 46 126 5.3.7. Summary . . . . . . . . . . . . . . . . . . . . . . . 47 127 6. Security Considerations . . . . . . . . . . . . . . . . . . . 48 128 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 49 129 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 49 130 9. Change log [RFC Editor: Please remove] . . . . . . . . . . . 50 131 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 62 132 10.1. Normative References . . . . . . . . . . . . . . . . . . 62 133 10.2. Informative References . . . . . . . . . . . . . . . . . 63 134 Appendix A. BIER-TE and Segment Routing . . . . . . . . . . . . 66 135 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 67 137 1. Overview 139 BIER-TE is based on the (non-TE) BIER architecture, terminology and 140 packet formats as described in [RFC8279] and [RFC8296]. This 141 document describes BIER-TE in the expectation that the reader is 142 familiar with these two documents. 144 BIER-TE introduces a new semantic for "bit positions" (BP). They 145 indicate adjacencies of the network topology, as opposed to (non-TE) 146 BIER in which BPs indicate "Bit-Forwarding Egress Routers" (BFER). A 147 BIER-TE packets BitString therefore indicates the edges of the (loop- 148 free) tree that the packet is forwarded across by BIER-TE. With 149 BIER-TE, the "Bit Index Forwarding Table" (BIFT) of each "Bit 150 Forwarding Router" (BFR) is only populated with BP that are adjacent 151 to the BFR in the BIER-TE Topology. Other BPs are empty in the BIFT. 152 The BFR replicate and forwards BIER packets to adjacent BPs that are 153 set in the packet. BPs are normally also cleared upon forwarding to 154 avoid duplicates and loops. 156 BIER-TE can leverage BIER forwarding engines with little or no 157 changes. It can also co-exist with BIER forwarding in the same 158 domain, for example by using separate BIER sub-domains. Except for 159 the optional routed adjacencies, BIER-TE does not require a BIER 160 routing underlay, and can therefore operate without depending on an 161 "Interior Gateway Routing protocol" (IGP). 163 As it operates on the same per-packet stateless forwarding 164 principles, BIER-TE can also be a good fit to support multicast path 165 steering in "Segment Routing" (SR) networks ([RFC8402]). 167 This document is structured as follows: 169 * Section 2 introduces BIER-TE with two forwarding examples, 170 followed by an introduction of the new concepts of the BIER-TE 171 (overlay) topology and finally a summary of the relationship 172 between BIER and BIER-TE and a discussion of accelerated hardware 173 forwarding. 175 * Section 3 describes the components of the BIER-TE architecture, 176 Flow overlay, BIER-TE layer with the BIER-TE control plane 177 (including the BIER-TE controller) and BIER-TE forwarding plane, 178 and the routing underlay. 180 * Section 4 specifies the behavior of the BIER-TE forwarding plane 181 with the different type of adjacencies and possible variations of 182 BIER-TE forwarding pseudocode, and finally the mandatory and 183 optional requirements. 185 * Section 5 describes operational considerations for the BIER-TE 186 controller, foremost how the BIER-TE controller can optimize the 187 use of BP by using specific type of BIER-TE adjacencies for 188 different type of topological situations, but also how to assign 189 bits to avoid loops and duplicates (which in BIER-TE does not come 190 for free), and finally how "Set Identifier" (SI), "sub-domain" 191 (SD) and BFR-ids can be managed by a BIER-TE controller, examples 192 and summary. 194 * Appendix A concludes the technology specific sections of the 195 document by further relating BIER-TE to SR. 197 Note that related work, [I-D.ietf-roll-ccast] uses Bloom filters 198 [Bloom70] to represent leaves or edges of the intended delivery tree. 199 Bloom filters in general can support larger trees/topologies with 200 fewer addressing bits than explicit BitStrings, but they introduce 201 the heuristic risk of false positives and cannot clear bits in the 202 BitString during forwarding to avoid loops. For these reasons, BIER- 203 TE uses explicit BitStrings like BIER. The explicit BitStrings of 204 BIER-TE can also be seen as a special type of Bloom filter, and this 205 is how related work [ICC] describes it. 207 1.1. Requirements Language 209 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 210 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 211 "OPTIONAL" in this document are to be interpreted as described in BCP 212 14 [RFC2119] [RFC8174] when, and only when, they appear in all 213 capitals, as shown here. 215 2. Introduction 217 2.1. Basic Examples 219 BIER-TE forwarding is best introduced with simple examples. These 220 examples use formal terms defined later in the document (Figure 4), 221 including forward_connected(), forward_routed() and local_decap(). 223 BIER-TE Topology: 225 Diagram: 227 p5 p6 228 --- BFR3 --- 229 p3/ p13 \p7 p15 230 BFR1 ---- BFR2 BFR5 ----- BFR6 231 p1 p2 p4\ p14 /p10 p11 p12 232 --- BFR4 --- 233 p8 p9 235 (simplified) BIER-TE Bit Index Forwarding Tables (BIFT): 237 BFR1: p1 -> local_decap() 238 p2 -> forward_connected() to BFR2 240 BFR2: p1 -> forward_connected() to BFR1 241 p5 -> forward_connected() to BFR3 242 p8 -> forward_connected() to BFR4 244 BFR3: p3 -> forward_connected() to BFR2 245 p7 -> forward_connected() to BFR5 246 p13 -> local_decap() 248 BFR4: p4 -> forward_connected() to BFR2 249 p10 -> forward_connected() to BFR5 250 p14 -> local_decap() 252 BFR5: p6 -> forward_connected() to BFR3 253 p9 -> forward_connected() to BFR4 254 p12 -> forward_connected() to BFR6 256 BFR6: p11 -> forward_connected() to BFR5 257 p15 -> local_decap() 259 Figure 1: BIER-TE basic example 261 Consider the simple network in the above BIER-TE overview example 262 picture with 6 BFRs. p1...p15 are the bit positions used. All BFRs 263 can act as an ingress BFR (BFIR), BFR1, BFR3, BFR4 and BFR6 can also 264 be BFERs. Forward_connected() is the name for adjacencies that are 265 representing subnet adjacencies of the network. Local_decap() is the 266 name of the adjacency to decapsulate BIER-TE packets and pass their 267 payload to higher layer processing. 269 Assume a packet from BFR1 should be sent via BFR4 to BFR6. This 270 requires a BitString (p2,p8,p10,p12,p15). When this packet is 271 examined by BIER-TE on BFR1, the only bit position from the BitString 272 that is also set in the BIFT is p2. This will cause BFR1 to send the 273 only copy of the packet to BFR2. Similarly, BFR2 will forward to 274 BFR4 because of p8, BFR4 to BFR5 because of p10 and BFR5 to BFR6 275 because of p12. p15 finally makes BFR6 receive and decapsulate the 276 packet. 278 To send a copy to BFR6 via BFR4 and also a copy to BFR3, the 279 BitString needs to be (p2,p5,p8,p10,p12,p13,p15). When this packet 280 is examined by BFR2, p5 causes one copy to be sent to BFR3 and p8 one 281 copy to BFR4. When BFR3 receives the packet, p13 will cause it to 282 receive and decapsulate the packet. 284 If instead the BitString was (p2,p6,p8,p10,p12,p13,p15), the packet 285 would be copied by BFR5 towards BFR3 because of p6 instead of being 286 copied by BFR2 to BFR3 because of p5 in the prior case. This is 287 showing the ability of the shown BIER-TE Topology to make the traffic 288 pass across any possible path and be replicated where desired. 290 BIER-TE has various options to minimize BP assignments, many of which 291 are based on out-of-band knowledge about the required multicast 292 traffic paths and bandwidth consumption in the network, such as from 293 pre-deployment planning. 295 Figure 2 shows a modified example, in which Rtr2 and Rtr5 are assumed 296 not to support BIER-TE, so traffic has to be unicast encapsulated 297 across them. To emphasize non-L2, but routed/tunneled forwarding of 298 BIER-TE packets, these adjacencies are called "forward_routed". 299 Otherwise, there is no difference in their processing over the 300 aforementioned forward_connected() adjacencies. 302 In addition, bits are saved in the following example by assuming that 303 BFR1 only needs to be BFIR but not BFER or transit BFR. 305 BIER-TE Topology: 307 Diagram: 309 p1 p3 p7 310 ....> BFR3 <.... p5 311 ........ ........> 312 BFR1 (Rtr2) (Rtr5) BFR6 313 ........ ........> p9 314 ....> BFR4 <.... p6 315 p2 p4 p8 317 (simplified) BIER-TE Bit Index Forwarding Tables (BIFT): 319 BFR1: p1 -> forward_routed() to BFR3 320 p2 -> forward_routed() to BFR4 322 BFR3: p3 -> local_decap() 323 p5 -> forward_routed() to BFR6 325 BFR4: p4 -> local_decap() 326 p6 -> forward_routed() to BFR6 328 BFR6: p7 -> forward_routed() to BFR3 329 p8 -> forward_routed() to BFR4 330 p9 -> local_decap() 332 Figure 2: BIER-TE basic overlay example 334 To send a BIER-TE packet from BFR1 via BFR3 to be received by BFR6, 335 the BitString is (p1,p5,p9). From BFR1 via BFR4 to be received by 336 BFR6, the BitString is (p2,p6,p9). A packet from BFR1 to be received 337 by BFR3,BFR4 and from BFR3 to be received by BFR6 uses 338 (p1,p2,p3,p4,p5,p9). A packet from BFR1 to be received by BFR3,BFR4 339 and from BFR4 to be received by BFR6 uses (p1,p2,p3,p4,p6,p9). A 340 packet from BFR1 to be received by BFR4, and from BFR4 to be received 341 by BFR6 and from there to be received by BFR3 uses 342 (p2,p3,p4,p6,p7,p9). A packet from BFR1 to be received by BFR3, and 343 from BFR3 to be received by BFR6 there to be received by BFR4 uses 344 (p1,p3,p4,p5,p8,p9). 346 2.2. BIER-TE Topology and adjacencies 348 The key new component in BIER-TE compared to (non-TE) BIER is the 349 BIER-TE topology as introduced through the two examples in 350 Section 2.1. It is used to control where replication can or should 351 happen and how to minimize the required number of BP for adjacencies. 353 The BIER-TE Topology consists of the BIFTs of all the BFR and can 354 also be expressed as a directed graph where the edges are the 355 adjacencies between the BFRs labelled with the BP used for the 356 adjacency. Adjacencies are naturally unidirectional. BP can be 357 reused across multiple adjacencies as long as this does not lead to 358 undesired duplicates or loops as explained in Section 5.2. 360 If the BIER-TE topology represents (a subset of) the underlying 361 (layer 2) topology of the network as shown in the first example, this 362 may be called a "native" BIER-TE topology. A topology consisting 363 only of "forward_routed" adjacencies as shown in the second example 364 may be called an "overlay" BIER-TE topology. A BIER-TE topology with 365 both forward_connected() and forward_routed() adjacencies may be 366 called a "hybrid" BIER-TE topology. 368 2.3. Relationship to BIER 370 BIER-TE is designed so that its forwarding plane is a simple 371 extension to the (non-TE) BIER forwarding plane, hence allowing for 372 it to be added to BIER deployments where it can be beneficial. 374 BIER-TE is also intended as an option to expand the BIER architecture 375 into deployments where (non-TE) BIER may not be the best fit, such as 376 statically provisioned networks with needs for path steering but 377 without desire for distributed routing protocols. 379 1. BIER-TE inherits the following aspects from BIER unchanged: 381 1. The fundamental purpose of per-packet signaled replication 382 and delivery via a BitString. 384 2. The overall architecture consisting of three layers, flow 385 overlay, BIER(-TE) layer and routing underlay. 387 3. The supported encapsulations [RFC8296]. 389 4. The semantic of all [RFC8296] header elements used by the 390 BIER-TE forwarding plane other than the semantic of the BP in 391 the BitString. 393 5. The BIER forwarding plane, except for how bits have to be 394 cleared during replication. 396 2. BIER-TE has the following key changes with respect to BIER: 398 1. In BIER, bits in the BitString of a BIER packet header 399 indicate a BFER and bits in the BIFT indicate the BIER 400 control plane calculated next-hop toward that BFER. In BIER- 401 TE, a bit in the BitString of a BIER packet header indicates 402 an adjacency in the BIER-TE topology, and only the BFR that 403 is the upstream of that adjacency has its BP populated with 404 the adjacency in its BIFT. 406 2. In BIER, the implied reference option for the core part of 407 the BIER layer control plane are the BIER extensions for 408 distributed routing protocols. This includes ISIS/OSPF 409 extensions for BIER, [RFC8401] and [RFC8444]. 411 3. The reference option for the core part of the BIER-TE control 412 plane is the BIER-TE controller. Nevertheless, both the BIER 413 and BIER-TE BIFTs forwarding plane state could equally be 414 populated by any mechanism. 416 4. Assuming the reference options for the control plane, BIER-TE 417 replaces in-network autonomous path calculation by explicit 418 paths calculated by the BIER-TE controller. 420 3. The following elements/functions described in the BIER 421 architecture are not required by the BIER-TE architecture: 423 1. "Bit Index Routing Tables" (BIRTs) are not required on BFRs 424 for BIER-TE when using a BIER-TE controller because the 425 controller can directly populate the BIFTs. In BIER, BIRTs 426 are populated by the distributed routing protocol support for 427 BIER, allowing BFRs to populate their BIFTs locally from 428 their BIRTs. Other BIER-TE control plane or management plane 429 options may introduce requirements for BIRTs for BIER-TE 430 BFRs. 432 2. The BIER-TE layer forwarding plane does not require BFRs to 433 have a unique BP and therefore also no unique BFR-id. See 434 Section 5.1.3. 436 3. Identification of BFRs by the BIER-TE control plane is 437 outside the scope of this specification. Whereas the BIER 438 control plane uses BFR-ids in its BFR to BFR signaling, a 439 BIER-TE controller may choose any form of identification 440 deemed appropriate. 442 4. BIER-TE forwarding does not require the BFIR-id field of the 443 BIER packet header. 445 4. Co-existence of BIER and BIER-TE in the same network requires the 446 following: 448 1. The BIER/BIER-TE packet header needs to allow addressing both 449 BIER and BIER-TE BIFTs. Depending on the encapsulation 450 option, the same SD may or may not be reusable across BIER 451 and BIER-TE. See Section 4.3. In either case, a packet is 452 always only forwarded end-to-end via BIER or via BIER-TE 453 (ships in the nights forwarding). 455 2. BIER-TE deployments will have to assign BFR-ids to BFRs and 456 insert them into the BFIR-id field of BIER packet headers as 457 BIER does, whenever the deployment uses (unchanged) 458 components developed for BIER that use BFR-id, such as 459 multicast flow overlays or BIER layer control plane elements. 460 See also Section 5.3.3. 462 2.4. Accelerated/Hardware forwarding comparison 464 BIER-TE forwarding rules, especially the BitString parsing are 465 designed to be as close as possible to those of BIER in the 466 expectation that this eases the programming of BIER-TE forwarding 467 code and/or BIER-TE forwarding hardware on platforms supporting BIER. 468 The pseudocode in Section 4.4 shows how existing (non-TE) BIER/BIFT 469 forwarding can be modified to support the required BIER-TE forwarding 470 functionality (Section 4.6), by using BIER BIFT's "Forwarding Bit 471 Mask" (F-BM): Only the clearing of bits to avoid duplicate packets to 472 a BFR's neighbor is skipped in BIER-TE forwarding because it is not 473 necessary and could not be done when using BIER F-BM. 475 Whether to use BIER or BIER-TE forwarding is simply a choice of the 476 mode of the BIFT indicated by the packet (BIER or BIER-TE BIFT). 477 This is determined by the BFR configuration for the encapsulation, 478 see Section 4.3. 480 3. Components 482 BIER-TE can be thought of being constituted from the same three 483 layers as BIER: The "multicast flow overlay", the "BIER layer" and 484 the "routing underlay". The following picture also shows how the 485 "BIER layer" is constituted from the "BIER-TE forwarding plane" and 486 the "BIER-TE control plane" represent by the "BIER-TE Controller". 488 <------BGP/PIM-----> 489 |<-IGMP/PIM-> multicast flow <-PIM/IGMP->| 490 overlay 492 BIER-TE [BIER-TE Controller] <=> [BIER-TE Topology] 493 control ^ ^ ^ 494 plane / | \ BIER-TE control protocol 495 | | | e.g. YANG/NETCONF/RESTCONF 496 | | | PCEP/... 497 v v v 498 Src -> Rtr1 -> BFIR-----BFR-----BFER -> Rtr2 -> Rcvr 500 |<----------------->| 501 BIER-TE forwarding plane 503 |<- BIER-TE domain->| 505 |<--------------------->| 506 Routing underlay 508 Figure 3: BIER-TE architecture 510 3.1. The Multicast Flow Overlay 512 The Multicast Flow Overlay has the same role as described for BIER in 513 [RFC8279], Section 4.3. See also Section 3.2.1.2. 515 When a BIER-TE controller is used, then the signaling for the 516 Multicast Flow Overlay may also be preferred to operate through a 517 central point of control. For BGP based overlay flow services such 518 as "Multicast VPN Using BIER" ([RFC8556]) this can be achieved by 519 making the BIER-TE controller operate as a BGP Route Reflector 520 ([RFC4456]) and combining it with signaling through BGP or a 521 different protocol for the BIER-TE controller calculated BitStrings. 522 See Section 3.2.1.2 and Section 5.3.4. 524 3.2. The BIER-TE Control Plane 526 In the (non-TE) BIER architecture [RFC8279], the BIER control plane 527 is not explicitly separated from the BIER forwarding plane, but 528 instead their functions are summarized together in Section 4.2. 529 Example standardized options for the BIER control plane include ISIS/ 530 OSPF extensions for BIER, [RFC8401] and [RFC8444]. 532 For BIER-TE, the control plane includes at minimum the following 533 functionality. 535 1. BIER-TE topology control: During initial provisioning of the 536 network and/or during modifications of its topology and/or 537 services, the protocols and/or procedures to establish BIER-TE 538 BIFTs: 540 1. Determine the desired BIER-TE topology for a BIER-TE sub- 541 domains: the native and/or overlay adjacencies that are 542 assigned to BPs. Topology discovery is discussed in 543 Section 3.2.1.1 and the various aspects of the BIER-TE 544 controllers determinations about the topology are discussed 545 throughout Section 5 547 2. Determine the per-BFR BIFT from the BIER-TE topology. This is 548 achieved by simply extracting the adjacencies of the BFR from 549 the BIER-TE topology and populating the BFRs BIFT with them. 551 3. Optionally assign BFR-ids to BFIRs for later insertion into 552 BIER headers on BFIRs as BFIR-id. Alternatively, BFIR-id in 553 BIER packet headers may be managed solely by the flow overlay 554 layer and/or be unused. This is discussed in Section 5.3.3. 556 4. Install/update the BIFTs into the BFRs and optionally BFR-ids 557 into BFIRs. This is discussed in Section 3.2.1.1. 559 2. BIER-TE tree control: During operations of the network, 560 protocols and/or procedures to support creation/change/removal of 561 overlay flows on BFIRs: 563 1. Process the BIER-TE requirements for the multicast overlay 564 flow: BFIR and BFERs of the flow as well as policies for the 565 path selection of the flow. This is discussed in Section 3.5. 567 2. Determine the BitStrings and optionally Entropy. This is 568 discussed in Section 3.2.1.2, Section 3.5 and Section 5.3.4. 570 3. Install state on the BFIR to impose the desired BIER packet 571 header(s) for packets of the overlay flow. Different aspects 572 of this and the next point are discussed throughout 573 Section 3.2.1 and in Section 4.3, but the main responsibility 574 of these two points is with the Multicast Flow Overlay 575 (Section 3.1), which is architecturally inherited from BIER. 577 4. Install the necessary state on the BFERs to decapsulate the 578 BIER packet header and properly dispatch its payload. 580 3.2.1. The BIER-TE Controller 582 [RFC-Editor: the following text has three references to anchors 583 topology-control, topology-control-1 and tree-control. 584 Unfortunately, XMLv2 does not offer any tagging that reasonable 585 references are generated (i had this problem already in RFCs last 586 year. Please make sure there are useful-to-read cross-references in 587 the RFC in these three places after you convert to XMLv3.] 589 This architecture describes the BIER-TE control plane as shown in 590 Figure 3 to consist of: 592 * A BIER-TE controller. 594 * BFR data-models and protocols to communicate between controller 595 and BFRs in support of BIER-TE topology control (Section 3.2), 596 such as YANG/NETCONF/RESTCONF ([RFC7950]/[RFC6241]/[RFC8040]). 598 * BFR data-models and protocols to communicate between controller 599 and BFIR in support of BIER-TE tree control (Section 3.2), such as 600 BIER-TE extensions for [RFC5440]. 602 The single, centralized BIER-TE controller is used in this document 603 as reference option for the BIER-TE control plane but other options 604 are equally feasible. The BIER-TE control plane could equally be 605 implemented without automated configuration/protocols, by an operator 606 via CLI on the BFRs. In that case, operator configured local policy 607 on the BFIR would have to determine how to set the appropriate BIER 608 header fields. The BIER-TE control plane could also be decentralized 609 and/or distributed, but this document does not consider any 610 additional protocols and/or procedures which would then be necessary 611 to coordinate its (distributed/decentralized) entities to achieve the 612 above described functionality. 614 3.2.1.1. BIER-TE Topology discovery and creation 616 The first item of BIER-TE topology control (Section 3.2, Paragraph 3, 617 Item 2.2.1) includes network topology discovery and BIER-TE topology 618 creation. The latter describes the process by which a Controller 619 determines which routers are to be configured as BFRs and the 620 adjacencies between them. 622 In statically managed networks, such as in industrial environments, 623 both discovery and creation can be a manual/offline process. 625 In other networks, topology discovery may rely on protocols including 626 extending a "Link-State-Protocol" based IGP into the BIER-TE 627 controller itself, [RFC7752] (BGP-LS) or [RFC8345] (YANG topology) as 628 well as BIER-TE specific methods, for example via 629 [I-D.ietf-bier-te-yang]. These options are non-exhaustive. 631 Dynamic creation of the BIER-TE topology can be as easy as mapping 632 the network topology 1:1 to the BIER-TE topology by assigning a BP 633 for every network subnet adjacency. In larger networks, it likely 634 involves more complex policy and optimization decisions including how 635 to minimize the number of BPs required and how to assign BPs across 636 different BitStrings to minimize the number of duplicate packets 637 across links when delivering an overlay flow to BFER using different 638 SIs/BitStrings. These topics are discussed in Section 5. 640 When the BIER-TE topology is determined, the BIER-TE Controller then 641 pushes the BitPositions/adjacencies to the BIFT of the BFRs. On each 642 BFR only those SI:BitPositions are populated that are adjacencies to 643 other BFRs in the BIER-TE topology. 645 Communications between the BIER-TE Controller and BFRs for both BIER- 646 TE topology control and BIER-TE tree control is ideally via 647 standardized protocols and data-models such as NETCONF/RESTCONF/YANG/ 648 PCEP. Vendor-specific CLI on the BFRs is also an option (as in many 649 other SDN solutions lacking definition of standardized data models). 651 3.2.1.2. Engineered Trees via BitStrings 653 In BIER, the same set of BFER in a single sub-domain is always 654 encoded as the same BitString. In BIER-TE, the BitString used to 655 reach the same set of BFER in the same sub-domain can be different 656 for different overlay flows because the BitString encodes the paths 657 towards the BFER, so the BitStrings from different BFIR to the same 658 set of BFER will often be different. Likewise, the BitString from 659 the same BFIR to the same set of BFER can be different for different 660 overlay flows for policy reasons such as shortest path trees, Steiner 661 trees (minimum cost trees), diverse path trees for redundancy and so 662 on. 664 See also [I-D.ietf-bier-multicast-http-response] for an application 665 leveraging BIER-TE engineered trees. 667 3.2.1.3. Changes in the network topology 669 If the network topology changes (not failure based) so that 670 adjacencies that are assigned to bit positions are no longer needed, 671 the BIER-TE Controller can re-use those bit positions for new 672 adjacencies. First, these bit positions need to be removed from any 673 BFIR flow state and BFR BIFT state, then they can be repopulated, 674 first into BIFT and then into the BFIR. 676 3.2.1.4. Link/Node Failures and Recovery 678 When link or nodes fail or recover in the topology, BIER-TE could 679 quickly respond with FRR procedures such as [I-D.eckert-bier-te-frr], 680 the details of which are out of scope for this document. It can also 681 more slowly react by recalculating the BitStrings of affected 682 multicast flows. This reaction is slower than the FRR procedure 683 because the BIER-TE Controller needs to receive link/node up/down 684 indications, recalculate the desired BitStrings and push them down 685 into the BFIRs. With FRR, this is all performed locally on a BFR 686 receiving the adjacency up/down notification. 688 3.3. The BIER-TE Forwarding Plane 690 [RFC-editor Q: "is constituted from" / "consists of" / "composed 691 from..." ???] 693 The BIER-TE Forwarding Plane is constituted from the following 694 components: 696 1. On a BFIR, imposition of the BIER header for packets from overlay 697 flows. This is driven by a combination of state established by 698 the BIER-TE control plane and/or the multicast flow overlay as 699 explained in Section 3.1. 701 2. On BFRs (including BFIR and BFER), forwarding/replication of BIER 702 packets according to their SD, SI, "BitStringLength" (BSL), 703 BitString and optionally Entropy fields as explained in 704 Section 4. Processing of other BIER header fields such as DSCP 705 is outside the scope of this document. 707 3. On BFERs, removal of the BIER header and dispatching of the 708 payload according to state created by the BIER-TE control plane 709 and/or overlay layer. 711 When the BIER-TE Forwarding Plane receives a packet, it simply looks 712 up the bit positions that are set in the BitString of the packet in 713 the BIFT that was populated by the BIER-TE Controller. For every BP 714 that is set in the BitString, and that has one or more adjacencies in 715 the BIFT, a copy is made according to the type of adjacencies for 716 that BP in the BIFT. Before sending any copy, the BFR clears all BPs 717 in the BitString of the packet for which the BFR has one or more 718 adjacencies in the BIFT. Clearing these bits inhibits packets from 719 looping when the BitStrings erroneously includes a forwarding loop. 720 When a forward_connected() adjacency has the "DoNotClear" (DNC) flag 721 set, then this BP is re-set for the packet copied to that adjacency. 722 See Section 4.2.1. 724 3.4. The Routing Underlay 726 For forward_connected() adjacencies, BIER-TE is sending BIER packets 727 to directly connected BIER-TE neighbors as L2 (unicasted) BIER 728 packets without requiring a routing underlay. For forward_routed() 729 adjacencies, BIER-TE forwarding encapsulates a copy of the BIER 730 packet so that it can be delivered by the forwarding plane of the 731 routing underlay to the routable destination address indicated in the 732 adjacency. See Section 4.2.2 for the adjacency definition. 734 BIER relies on the routing underlay to calculate paths towards BFERs 735 and derive next-hop BFR adjacencies for those paths. This commonly 736 relies on BIER specific extensions to the routing protocols of the 737 routing underlay but may also be established by a controller. In 738 BIER-TE, the next-hops of a packet are determined by the BitString 739 through the BIER-TE Controller established adjacencies on the BFR for 740 the BPs of the BitString. There is thus no need for BFR specific 741 routing underlay extensions to forward BIER packets with BIER-TE 742 semantics. 744 Encapsulation parameters can be provisioned by the BIER-TE controller 745 into the forward_connected() or forward_routed() adjacencies directly 746 without relying on a routing underlay. 748 If the BFR intends to support FRR for BIER-TE, then the BIER-TE 749 forwarding plane needs to receive fast adjacency up/down 750 notifications: Link up/down or neighbor up/down, e.g. from BFD. 751 Providing these notifications is considered to be part of the routing 752 underlay in this document. 754 3.5. Traffic Engineering Considerations 756 Traffic Engineering ([I-D.ietf-teas-rfc3272bis]) provides performance 757 optimization of operational IP networks while utilizing network 758 resources economically and reliably. The key elements needed to 759 effect TE are policy, path steering and resource management. These 760 elements require support at the control/controller level and within 761 the forwarding plane. 763 Policy decisions are made within the BIER-TE control plane, i.e., 764 within BIER-TE Controllers. Controllers use policy when composing 765 BitStrings and BFR BIFT state. The mapping of user/IP traffic to 766 specific BitStrings/BIER-TE flows is made based on policy. The 767 specific details of BIER-TE policies and how a controller uses them 768 are out of scope of this document. 770 Path steering is supported via the definition of a BitString. 771 BitStrings used in BIER-TE are composed based on policy and resource 772 management considerations. For example, when composing BIER-TE 773 BitStrings, a Controller must take into account the resources 774 available at each BFR and for each BP when it is providing 775 congestion-loss-free services such as Rate Controlled Service 776 Disciplines [RCSD94]. Resource availability could be provided for 777 example via routing protocol information, but may also be obtained 778 via a BIER-TE control protocol such as NETCONF or any other protocol 779 commonly used by a Controller to understand the resources of the 780 network it operates on. The resource usage of the BIER-TE traffic 781 admitted by the BIER-TE controller can be solely tracked on the BIER- 782 TE Controller based on local accounting as long as no 783 forward_routed() adjacencies are used (see Section 4.2.1 for the 784 definition of forward_routed() adjacencies). When forward_routed() 785 adjacencies are used, the paths selected by the underlying routing 786 protocol need to be tracked as well. 788 Resource management has implications on the forwarding plane beyond 789 the BIER-TE defined steering of packets. This includes allocation of 790 buffers to guarantee the worst case requirements of admitted RCSD 791 traffic and potentially policing and/or rate-shaping mechanisms, 792 typically done via various forms of queuing. This level of resource 793 control, while optional, is important in networks that wish to 794 support congestion management policies to control or regulate the 795 offered traffic to deliver different levels of service and alleviate 796 congestion problems, or those networks that wish to control latencies 797 experienced by specific traffic flows. 799 4. BIER-TE Forwarding 801 4.1. The BIER-TE Bit Index Forwarding Table (BIFT) 803 The BIER-TE BIFT is the equivalent to the BIER BIFT for (non-TE) 804 BIER. It exists on every BFR running BIER-TE. For every BIER sub- 805 domain (SD) in use for BIER-TE, it is a table as shown shown in 806 Figure 4. That example BIFT assumes a BSL of 8 bit positions (BPs) 807 in the packets BitString. As in [RFC8279] this BSL is purely used 808 for the example and not a BIER/BIER-TE supported BSL (minimum BSL is 809 64). 811 A BIER-TE BIFT compares to a BIER BIFT as shown in [RFC8279] as 812 follows. 814 In both BIER and BIER-TE, BIFT rows/entries are indexed in their 815 respective BIER pseudocode ([RFC8279] Section 6.5) and BIER-TE 816 pseudocode (Section 4.4) by the BIFT-index derived from the packets 817 SI, BSL and the one bit position of the packets BitString (BP) 818 addressing the BIFT row: BIFT-index = SI * BSL + BP - 1. BP within a 819 BitString are numbered from 1 to BSL, hence the - 1 offset when 820 converting to a BIFT-index. This document also uses the notion SI:BP 821 to indicate BIFT rows, [RFC8279] uses the equivalent notion 822 SI:BitString, where the BitString is filled with only the BP for the 823 BIFT row. 825 In BIER, each BIFT-index addresses one BFER by its BFR-id = BIFT- 826 index + 1 and is populated on each BFR with the next-hop "BFR 827 Neighbor" (BFR-NBR) towards that BFER. 829 In BIER-TE, each BIFT-index and therefore SI:BP indicates one or more 830 adjacencies between BFRs in the topology and is only populated with 831 those adjacencies forwarding entries on the BFR that is the upstream 832 for these adjacencies. The BIFT entry are empty on all other BFRs. 834 In BIER, each BIFT rows also requires a "Forwarding Bit Mask" (F-BM) 835 entry for BIER forwarding rules. In BIER-TE forwarding, F-BM is not 836 required, but can be used when implementing BIER-TE on forwarding 837 hardware derived from BIER forwarding, that must use F-BM. This is 838 discussed in the first BIER-TE forwarding pseudocode in Section 4.4. 840 ------------------------------------------------------------------ 841 | BIFT-index | | Adjacencies: | 842 | (SI:BP) |(FBM)| or one or more per entry | 843 ================================================================== 844 | BIFT indices for Packets with SI=0 | 845 ------------------------------------------------------------------ 846 | 0 (0:1) | ... | forward_connected(interface,neighbor{,DNC}) | 847 ------------------------------------------------------------------ 848 | 1 (0:2) | ... | forward_connected(interface,neighbor{,DNC}) | 849 | | ... | forward_connected(interface,neighbor{,DNC}) | 850 ------------------------------------------------------------------ 851 | ... | ... | ... | 852 ------------------------------------------------------------------ 853 | 4 (0:5) | ... | local_decap({VRF}) | 854 ------------------------------------------------------------------ 855 | 5 (0:6) | ... | forward_routed({VRF,}l3-neighbor) | 856 ------------------------------------------------------------------ 857 | 6 (0:7) | ... | | 858 ------------------------------------------------------------------ 859 | 7 (0:8) | ... | ECMP((adjacency1,...adjacencyN){,seed}) | 860 ----------------------------------------------------------------- 861 | BIFT indices for BitString/Packet with SI=1 | 862 ------------------------------------------------------------------ 863 | 9 (1:1) | | ... | 864 | ... |... | ... | 865 ------------------------------------------------------------------ 866 BIER-TE Bit Index Forwarding Table (BIFT) 868 Figure 4: BIER-TE BIFT with different adjacencies 870 The BIFT is configured for the BIER-TE data plane of a BFR by the 871 BIER-TE Controller through an appropriate protocol and data-model. 872 The BIFT is then used to forward packets, according to the rules 873 specified in the BIER-TE Forwarding Procedures. 875 Note that a BIFT index (SI:BP) may be populated in the BIFT of more 876 than one BFR to save BPs. See Section 5.1.6 for an example of how a 877 BIER-TE controller could assign BPs to (logical) adjacencies shared 878 across multiple BFRs, Section 5.1.3 for an example of assigning the 879 same BP to different adjacencies, and Section 5.1.9 for general 880 guidelines regarding re-use of BPs across different adjacencies. 882 {VRF} indicates the Virtual Routing and Forwarding context into which 883 the BIER payload is to be delivered. This is optional and depends on 884 the multicast flow overlay. 886 4.2. Adjacency Types 887 4.2.1. Forward Connected 889 A "forward_connected()" adjacency is towards a directly connected BFR 890 neighbor using an interface address of that BFR on the connecting 891 interface. A forward_connected() adjacency does not route packets 892 but only L2 forwards them to the neighbor. 894 Packets sent to an adjacency with "DoNotClear" (DNC) set in the BIFT 895 MUST NOT have the bit position for that adjacency cleared when the 896 BFR creates a copy for it. The bit position will still be cleared 897 for copies of the packet made towards other adjacencies. This can be 898 used for example in ring topologies as explained in Section 5.1.6. 900 For protection against loops from misconfiguration (see 901 Section 5.2.1), DNC is only permissible for forward_connected() 902 adjacencies. No need or benefit of DNC for other type of adjacencies 903 was identified and their risk was not analyzed. 905 4.2.2. Forward Routed 907 A "forward_routed()" adjacency is an adjacency towards a BFR that 908 uses a (tunneling) encapsulation which will cause the packet to be 909 forwarded by the routing underlay toward the adjacent BFR. This can 910 leverage any feasible encapsulation, such as MPLS or tunneling over 911 IP/IPv6, as long as the BIER-TE packet can be identified as a 912 payload. This identification can either rely on the BIER/BIER-TE co- 913 existence mechanisms described in Section 4.3, or by explicit support 914 for a BIER-TE payload type in the tunneling encapsulation. 916 forward_routed() adjacencies are necessary to pass BIER-TE traffic 917 across non BIER-TE capable routers or to minimize the number of 918 required BP by tunneling over (BIER-TE capable) routers on which 919 neither replication nor path-steering is desired, or simply to 920 leverage path redundancy and FRR of the routing underlay towards the 921 next BFR. They may also be useful to a multi-subnet adjacent BFR to 922 leverage the routing underlay ECMP independent of BIER-TE ECMP 923 (Section 4.2.3). 925 4.2.3. ECMP 927 (non-TE) BIER ECMP is tied to the BIER BIFT processing semantic and 928 is therefore not directly usable with BIER-TE. 930 A BIER-TE "Equal Cost Multipath" (ECMP()) adjacency as shown in 931 Figure 4 for BIFT-index 7 has a list of two or more non-ECMP 932 adjacencies as parameters and an optional seed parameter. When a 933 BIER-TE packet is copied onto such an ECMP() adjacency, an 934 implementation specific so-called hash function will select one out 935 of the list's adjacencies to which the packet is forwarded. If the 936 packet's encapsulation contains an entropy field, the entropy field 937 SHOULD be respected; two packets with the same value of the entropy 938 field SHOULD be sent on the same adjacency. The seed parameter 939 allows to design hash functions that are easy to implement at high 940 speed without running into polarization issues across multiple 941 consecutive ECMP hops. See Section 5.1.7 for more explanations. 943 4.2.4. Local Decap(sulation) 945 A "local_decap()" adjacency passes a copy of the payload of the BIER- 946 TE packet to the protocol ("NextProto") within the BFR (IPv4/IPv6, 947 Ethernet,...) responsible for that payload according to the packet 948 header fields. A local_decap() adjacency turns the BFR into a BFER 949 for matching packets. Local_decap() adjacencies require the BFER to 950 support routing or switching for NextProto to determine how to 951 further process the packet. 953 4.3. Encapsulation / Co-existence with BIER 955 Specifications for BIER-TE encapsulation are outside the scope of 956 this document. This section gives explanations and guidelines. 958 Like [RFC8279], handling of "Maximum Transmission Unit" (MTU) 959 limitations is outside the scope of this document and instead part of 960 the BIER-TE packet encapsulation and/or flow overlay. See for 961 example [RFC8296], Section 3. It applies equally to BIER-TE as it 962 does to BIER. 964 Because a BFR needs to interpret the BitString of a BIER-TE packet 965 differently from a (non-TE) BIER packet, it is necessary to 966 distinguish BIER from BIER-TE packets. In the BIER encapsulation 967 [RFC8296], the BIFT-id field of the packet indicates the BIFT of the 968 packet. BIER and BIER-TE can therefore be run simultaneously, when 969 the BIFT-id address space is shared across BIER BIFT and BIER-TE 970 BIFT. Partitioning the BIFT-id address space is subject to BIER-TE/ 971 BIER control plane procedures. 973 When [RFC8296] is used for BIER with MPLS, BIFT-id address ranges can 974 be dynamically allocated from MPLS label space only for the set of 975 actually used SD:BSL BIFT. This allows to also allocate non- 976 overlapping label ranges for BIFT-id that are to be used with BIER-TE 977 BIFTs. 979 With MPLS, it is also possible to reuse the same SD space for both 980 BIER-TE and BIER, so that the same SD has both a BIER BIFT with a 981 corresponding range of BIFT-ids and disjoint BIER-TE BIFTs with a 982 non-overlapping range of BIFT-ids. 984 When a fixed mapping from BSL, SD and SI to BIFT-id is used which 985 does not explicitly partition the BIFT-id space between BIER and 986 BIER-TE, such as proposed for non-MPLS forwarding with [RFC8296] 987 encapsulation in [I-D.ietf-bier-non-mpls-bift-encoding] revision 04, 988 section 5, then it is necessary to allocate disjoint SDs to BIER and 989 BIER-TE BIFTs so that both can be addressed by the BIFT-ids. The 990 encoding proposed in section 6. of the same document does not 991 statically encode BSL or SD into the BIFT-id, but allows for a 992 mapping, and hence could provide for the same freedom as when MPLS is 993 being used (same or different SD for BIER/BIER-TE). 995 forward_routed() requires an encapsulation that permits to direct 996 unicast encapsulated BIER-TE packets to a specific interface address 997 on a target BFR. With MPLS encapsulation, this can simply be done 998 via a label stack with that addresses label as the top label - 999 followed by the label assigned to the (BSL,SD,SI) BitString. With 1000 non-MPLS encapsulation, some form of IP encapsulation would be 1001 required (for example IP/GRE). 1003 The encapsulation used for forward_routed() adjacencies can equally 1004 support existing advanced adjacency information such as "loose source 1005 routes" via e.g. MPLS label stacks or appropriate header extensions 1006 (e.g. for IPv6). 1008 4.4. BIER-TE Forwarding Pseudocode 1010 The following pseudocode, Figure 5, for BIER-TE forwarding is based 1011 on the (non-TE) BIER forwarding pseudocode of [RFC8279], section 6.5 1012 with one modification. 1014 void ForwardBitMaskPacket_withTE (Packet) 1015 { 1016 SI=GetPacketSI(Packet); 1017 Offset=SI*BitStringLength; 1018 for (Index = GetFirstBitPosition(Packet->BitString); Index ; 1019 Index = GetNextBitPosition(Packet->BitString, Index)) { 1020 F-BM = BIFT[Index+Offset]->F-BM; 1021 if (!F-BM) continue; [3] 1022 BFR-NBR = BIFT[Index+Offset]->BFR-NBR; 1023 PacketCopy = Copy(Packet); 1024 PacketCopy->BitString &= F-BM; [2] 1025 PacketSend(PacketCopy, BFR-NBR); 1026 // The following must not be done for BIER-TE: 1027 // Packet->BitString &= ~F-BM; [1] 1028 } 1029 } 1030 Figure 5: BIER-TE Forwarding Pseudocode for required functions, 1031 based on BIER Pseudocode 1033 In step [2], the F-BM is used to clear bit(s) in PacketCopy. This 1034 step exists in both BIER and BIER-TE, but the F-BMs need to be 1035 populated differently for BIER-TE than for BIER for the desired 1036 clearing. 1038 In BIER, multiple bits of a BitString can have the same BFR-NBR. 1039 When a received packets BitString has more than one of those bits 1040 set, the BIER replication logic has to avoid that more than one 1041 PacketCopy is sent to that BFR-NBR ([1]). Likewise, the PacketCopy 1042 sent to a BFR-NBR must clear all bits in its BitString that are not 1043 routed across BFR-NBR. This protects against BIER replication on any 1044 possible further BFR to create duplicates ([2]). 1046 To solve both [1] and [2] for BIER, the F-BM of each bit index needs 1047 to have all bits set that this BFR wants to route across BFR-NBR. [2] 1048 clears all other bits in PacketCopy->BitString, and [1] clears those 1049 bits from Packet->BitString after the first PacketCopy. 1051 In BIER-TE, a BFR-NBR in this pseudocode is an adjacency, 1052 forward_connected(), forward_routed() or local_decap(). There is no 1053 need for [2] to suppress duplicates in the way BIER does because in 1054 general, different BP would never have the same adjacency. If a 1055 BIER-TE controller actually finds some optimization in which this 1056 would be desirable, then the controller is also responsible to ensure 1057 that only one of those bits is set in any Packet->BitString, unless 1058 the controller explicitly wants for duplicates to be created. 1060 The following points describe how the forwarding bit mask (F-BM) for 1061 each BP is configured in the BIFT and how this impacts the BitString 1062 of the packet being processed with that BIFT: 1064 1. The F-BMs of all BIFT BPs without an adjacency have all their 1065 bits clear. This will cause [3] to skip further processing of 1066 such a BP. 1068 2. All BIFT BPs with an adjacency (with DNC flag clear) have an F-BM 1069 that has only those BPs set for which this BFR does not have an 1070 adjacency. This causes [2] to clear all bits from 1071 PacketCopy->BitString for which this BFR does have an adjacency. 1073 3. [1] is not performed for BIER-TE. All bit clearing required by 1074 BIER-TE is performed by [2]. 1076 This Forwarding Pseudocode can support the required BIER-TE 1077 forwarding functions (see Section 4.6), forward_connected(), 1078 forward_routed() and local_decap(), but not the recommended functions 1079 DNC flag and multiple adjacencies per bit nor the optional function, 1080 ECMP() adjacencies. The DNC flag cannot be supported when using only 1081 [1] to mask bits. 1083 The modified and expanded Forwarding Pseudocode in Figure 6 specifies 1084 how to support all BIER-TE forwarding functions (required, 1085 recommended and optional): 1087 * This pseudocode eliminates per-bit F-BM, therefore reducing the 1088 size of BIFT state by BSL^2*SI and eliminating the need for per- 1089 packet-copy BitString masking operations except for adjacencies 1090 with the DNC flag set: 1092 - AdjacentBits[SI] are bit positions with a non-empty list of 1093 adjacencies in this BFR BIFT. This can be computed whenever 1094 the BIER-TE Controller updates (add/removes) adjacencies in the 1095 BIFT. 1097 - The BFR needs to create packet copies for these adjacent bits 1098 when they are set in the packets BitString. This set of bits 1099 is calculated in PktAdjacentBits. 1101 - All bit positions to which the BFR creates copies have to be 1102 cleared in packet copies to avoid loops. This is done by 1103 masking the BitString of the packet with ~AdjacentBits[SI]. 1104 When an adjacency has DNC set, this bit position is set again 1105 only for the packet copy towards that bit position. 1107 * BIFT entries may contain more than one adjacency in support of 1108 specific configurations such as Section 5.1.5. The code therefore 1109 includes a loop over these adjacencies. 1111 * The ECMP() adjacency is shown. Its parameters are a seed and a 1112 ListOfAdjacencies from which one is picked. 1114 * The forward_connected(), forward_routed(), local_decap() 1115 adjacencies are shown with their parameters. 1117 void ForwardBitMaskPacket_withTE (Packet) 1118 { 1119 SI = GetPacketSI(Packet); 1120 Offset = SI * BitStringLength; 1121 // Determine adjacent bits in the Packets BitString 1122 PktAdjacentBits = Packet->BitString & AdjacentBits[SI]; 1124 // Clear adjacent bits in Packet header to avoid loops 1125 Packet->BitString &= ~AdjacentBits[SI]; 1127 // Loop over PktAdjacentBits to create packet copies 1128 for (Index = GetFirstBitPosition(PktAdjacentBits); Index ; 1129 Index = GetNextBitPosition(PktAdjacentBits, Index)) { 1130 for adjacency in BIFT[Index+Offset]->Adjacencies { 1131 if(adjacency.type == ECMP(ListOfAdjacencies,seed) ) { 1132 I = ECMP_hash(sizeof(ListOfAdjacencies), 1133 Packet->Entropy,seed); 1134 adjacency = ListOfAdjacencies[I]; 1135 } 1136 PacketCopy = Copy(Packet); 1137 switch(adjacency.type) { 1138 case forward_connected(interface,neighbor,DNC): 1139 if(DNC) 1140 PacketCopy->BitString |= 1<<(Index-1); 1141 SendToL2Unicast(PacketCopy,interface,neighbor); 1143 case forward_routed({VRF,}l3-neighbor): 1144 SendToL3(PacketCopy,{VRF,}l3-neighbor); 1146 case local_decap({VRF},neighbor): 1147 DecapBierHeader(PacketCopy); 1148 PassTo(PacketCopy,{VRF,}Packet->NextProto); 1149 } 1150 } 1151 } 1152 } 1154 Figure 6: Complete BIER-TE Forwarding Pseudocode for required, 1155 recommended and optional functions 1157 4.5. Basic BIER-TE Forwarding Example 1159 [RFC Editor: remove this section.] 1161 THIS SECTION TO BE REMOVED IN RFC BECAUSE IT WAS SUPERSEDED BY 1162 SECTION 1.1 EXAMPLE - IN CASE FINAL REVIES FIND A GOOD REASON TO KEEP 1163 IT, BUT DOESN'T SEEM TO SHOW IMPORTANT NEW STUFF OVER INITIAL 1164 EXAMPLES. 1166 Step-by-step example of basic BIER-TE forwarding. This example does 1167 not use ECMP() or forward_routed() adjacencies nor does it try to 1168 minimize the number of required BitPositions for the topology. 1170 [BIER-TE Controller] 1171 / | \ 1172 v v v 1173 . . 1174 | p13 p1 | . 1175 +- BFIR2 --+ | . 1176 | . | p2 p6 | . LAN2 1177 | . +-- BFR3 --+ . | 1178 | . | | p7 p11 | 1179 Src -+ . +-- BFER1 --+ 1180 | . | p3 p8 | . | 1181 | . +-- BFR4 --+ . +-- Rcv1 1182 | . | | . | 1183 | . | . 1184 | p14 p4 | . 1185 +- BFIR1 --+ | . 1186 | . +-- BFR5 --+ p10 p12 | 1187 LAN1 . | p5 p9 +-- BFER2 --+ 1188 . | . +-- Rcv2 1189 . . | 1190 . . LAN3 1191 . . 1192 IP |..... BIER-TE network.....| IP 1194 Figure 7: BIER-TE Forwarding Example 1196 pXX indicate the BitPositions number assigned by the BIER-TE 1197 Controller to adjacencies in the BIER-TE topology. For example, p9 1198 is the adjacency towards BFR5 on the LAN connecting to BFER2. 1200 BIFT BFIR2: 1201 p13: local_decap 1202 p2: forward_connected(BFR3) 1204 BIFT BFR3: 1205 p1: forward_connected(BFIR2) 1206 p7: forward_connected(BFER1) 1207 p8: forward_connected(BFR4) 1209 BIFT BFER1: 1210 p11: local_decap 1211 p6: forward_connected(BFR3) 1212 p8: forward_connected(BFR4) 1213 Figure 8: BIER-TE Forwarding Example Adjacencies 1215 ...and so on. 1217 For example, we assume that some multicast traffic seen on LAN1 needs 1218 to be sent via BIER-TE by BFIR2 towards Rcv1 and Rcv2. The BIER-TE 1219 Controller determines it wants it to pass this traffic across the 1220 following paths: 1222 -> BFER1 ---------------> Rcv1 1223 BFIR2 -> BFR3 1224 -> BFR4 -> BFR5 -> BFER2 -> Rcv2 1226 Figure 9: BIER-TE Forwarding Example Paths 1228 These paths equal to the following BitString: p2, p5, p7, p8, p10, 1229 p11, p12. 1231 This BitString is assigned by BFIR2 to the example multicast traffic 1232 received from LAN1. 1234 Then BFIR2 forwards this multicast traffic with BIER-TE based on that 1235 BitString. The BIFT of BFIR2 has only p2 and p13 populated. Only p2 1236 is in the BitString and this is an adjacency towards BFR3. BFIR2 1237 therefore clears p2 in the BitString and sends a copy towards BFR2. 1239 BFR3 sees a BitString of p5,p7,p8,p10,p11,p12. For those BPs, it has 1240 only adjacencies for p7,p8. It creates a copy of the packet to BFER1 1241 (due to p7) and one to BFR4 (due to p8). It clears both p7 and p8 1242 before sending. 1244 BFER1 sees a BitString of p5,p10,p11,p12. For those BPs, it only has 1245 an adjacency for p11. p11 is a local_decap() adjacency installed by 1246 the BIER-TE Controller to receive a copy of the BIER packet - dispose 1247 of the BIER header and pass the payload to IP multicast. IP 1248 multicast will then forward the packet out to LAN2 because it did 1249 receive PIM or IGMP joins on LAN2 for the traffic. 1251 Further processing of the packet in BFR4, BFR5 and BFER2 accordingly. 1253 4.6. BFR Requirements for BIER-TE forwarding 1255 BFR that support BIER-TE and BIER MUST support configuration that 1256 enables BIER-TE instead of (non-TE) BIER forwarding rules for all 1257 BIFT of one or more BIER sub-domains. Every BP in a BIER-TE BIFT 1258 MUST support to have zero or one adjacency. BIER-TE forwarding MUST 1259 support the adjacency types forward_connected() with the DNC flag not 1260 set, forward_routed() and local_decap(). As explained in 1261 Section 4.4, these required BIER-TE forwarding functions can be 1262 implemented via the same Forwarding Pseudocode as BIER forwarding 1263 except for one modification (skipping one masking with F-BM). 1265 BIER-TE forwarding SHOULD support forward_connected() adjacencies 1266 with a set DNC flag, as this is highly useful to save bits in rings 1267 (see Section 5.1.6). 1269 BIER-TE forwarding SHOULD support more than one adjacency on a bit. 1270 This allows to save bits in hub and spoke scenarios (see 1271 Section 5.1.5). 1273 BIER-TE forwarding MAY support ECMP() adjacencies to save bits in 1274 ECMP scenarios, see Section 5.1.7 for an example. This is an 1275 optional requirement, because for ECMP deployments using BIER-TE one 1276 can also leverage ECMP of the routing underlay via forwarded_routed 1277 adjacencies and/or might prefer to have more explicit control of the 1278 path chosen via explicit BP/adjacencies for each ECMP path 1279 alternative. 1281 5. BIER-TE Controller Operational Considerations 1283 5.1. Bit Position Assignments 1285 This section describes how the BIER-TE Controller can use the 1286 different BIER-TE adjacency types to define the bit positions of a 1287 BIER-TE domain. 1289 Because the size of the BitString limits the size of the BIER-TE 1290 domain, many of the options described exist to support larger 1291 topologies with fewer bit positions. 1293 5.1.1. P2P Links 1295 On a P2P link that connects two BFRs, the same bit position can be 1296 used on both BFRs for the adjacency to the neighboring BFR. A P2P 1297 link requires therefore only one bit position. 1299 5.1.2. BFER 1301 Every non-Leaf BFER is given a unique bit position with a 1302 local_decap() adjacency. 1304 5.1.3. Leaf BFERs 1305 BFR1(P) BFR2(P) BFR1(P) BFR2(P) 1306 | \ / | | | 1307 | X | | | 1308 | / \ | | | 1309 BFER1(PE) BFER2(PE) BFER1(PE)----BFER2(PE) 1311 ^ U-turn link 1313 Leaf BFER / Non-Leaf BFER / 1314 PE-router PE-router 1316 Figure 10: Leaf vs. non-Leaf BFER Example 1318 A leaf BFER is one where incoming BIER-TE packets never need to be 1319 forwarded to another BFR but are only sent to the BFER to exit the 1320 BIER-TE domain. For example, in networks where Provider Edge (PE) 1321 router are spokes connected to Provider (P) routers, those PEs are 1322 Leaf BFERs unless there is a U-turn between two PEs. 1324 Consider how redundant disjoint traffic can reach BFER1/BFER2 in 1325 Figure 10: When BFER1/BFER2 are Non-Leaf BFER as shown on the right- 1326 hand side, one traffic copy would be forwarded to BFER1 from BFR1, 1327 but the other one could only reach BFER1 via BFER2, which makes BFER2 1328 a non-Leaf BFER. Likewise, BFER1 is a non-Leaf BFER when forwarding 1329 traffic to BFER2. Note that the BFERs in the left-hand picture are 1330 only guaranteed to be leaf-BFER by fitting routing configuration that 1331 prohibits transit traffic to pass through the BFERs, which is 1332 commonly applied in these topologies. 1334 In most situations, leaf-BFER that are to be addressed via the same 1335 BitString can share a single bit position for their local_decap() 1336 adjacency in that BitString and therefore save bit positions. On a 1337 non-leaf BFER, a received BIER-TE packet may only need to transit the 1338 BFER or it may need to also be decapsulated. Whether or not to 1339 decapsulate the packet therefore needs to be indicated by a unique 1340 bit position populated only on the BIFT of this BFER with a 1341 local_decap() adjacency. On a leaf-BFER, packets never need to pass 1342 through; any packet received is therefore usually intended to be 1343 decapsulated. This can be expressed by a single, shared bit position 1344 that is populated with a local_decap() adjacency on all leaf-BFER 1345 addressed by the BitString. 1347 The possible exception from this leaf-BFER bit position optimization 1348 can be cases where the bit position on the prior BIER-TE BFR (which 1349 created the packet copy for the leaf-BFER in question) is populated 1350 with multiple adjacencies as an optimization, such as in 1351 Section 5.1.4 or Section 5.1.5. With either of these two 1352 optimizations, the sender of the packet could only control explicitly 1353 whether the packet was to be decapsulated on the leaf-BFER in 1354 question, if the leaf-BFER has a unique bit position for its 1355 local_decap() adjacency. 1357 However, if the bit position is shared across leaf-BFER, and packets 1358 are therefore decapsulated potentially unnecessarily, this may still 1359 be appropriate if the decapsulated payload of the BIER-TE packet does 1360 indicate whether or not the packet needs to be further processed/ 1361 received. This is typically true for example if the payload is IP 1362 multicast because IP multicast on a BFER would know the membership 1363 state of the IP multicast payload and be able to discard it if the 1364 packet was delivered unnecessarily by the BIER-TE layer. If the 1365 payload has no such membership indication, and the BFIR wants to have 1366 explicit control about which BFER are to receive and decapsulate a 1367 packet, then these two optimizations can not be used together with 1368 shared bit positions optimization for leaf-BFER. 1370 5.1.4. LANs 1372 In a LAN, the adjacency to each neighboring BFR is given a unique bit 1373 position. The adjacency of this bit position is a 1374 forward_connected() adjacency towards the BFR and this bit position 1375 is populated into the BIFT of all the other BFRs on that LAN. 1377 BFR1 1378 |p1 1379 LAN1-+-+---+-----+ 1380 p3| p4| p2| 1381 BFR3 BFR4 BFR7 1383 Figure 11: LAN Example 1385 If Bandwidth on the LAN is not an issue and most BIER-TE traffic 1386 should be copied to all neighbors on a LAN, then bit positions can be 1387 saved by assigning just a single bit position to the LAN and 1388 populating the bit position of the BIFTs of each BFRs on the LAN with 1389 a list of forward_connected() adjacencies to all other neighbors on 1390 the LAN. 1392 This optimization does not work in the case of BFRs redundantly 1393 connected to more than one LAN with this optimization because these 1394 BFRs would receive duplicates and forward those duplicates into the 1395 opposite LANs. Adjacencies of such BFRs into their LAN still need a 1396 separate bit position. 1398 5.1.5. Hub and Spoke 1400 In a setup with a hub and multiple spokes connected via separate p2p 1401 links to the hub, all p2p adjacencies from the hub to the spokes 1402 links can share the same bit position. The bit position on the hub's 1403 BIFT is set up with a list of forward_connected() adjacencies, one 1404 for each Spoke. 1406 This option is similar to the bit position optimization in LANs: 1407 Redundantly connected spokes need their own bit positions, unless 1408 they are themselves Leaf-BFER. 1410 This type of optimized BP could be used for example when all traffic 1411 is "broadcast" traffic (very dense receiver set) such as live-TV or 1412 many-to-many telemetry including situation-awareness (SA). This BP 1413 optimization can then be used to explicitly steer different traffic 1414 flows across different ECMP paths in Data-Center or broadband- 1415 aggregation networks with minimal use of BPs. 1417 5.1.6. Rings 1419 In L3 rings, instead of assigning a single bit position for every p2p 1420 link in the ring, it is possible to save bit positions by setting the 1421 "DoNotClear" (DNC) flag on forward_connected() adjacencies. 1423 For the rings shown in Figure 12, a single bit position will suffice 1424 to forward traffic entering the ring at BFRa or BFRb all the way up 1425 to BFR1: 1427 On BFRa, BFRb, BFR30,... BFR3, the bit position is populated with a 1428 forward_connected() adjacency pointing to the clockwise neighbor on 1429 the ring and with DNC set. On BFR2, the adjacency also points to the 1430 clockwise neighbor BFR1, but without DNC set. 1432 Handling DNC this way ensures that copies forwarded from any BFR in 1433 the ring to a BFR outside the ring will not have the ring bit 1434 position set, therefore minimizing the chance to create loops. 1436 v v 1437 | | 1438 L1 | L2 | L3 1439 /-------- BFRa ---- BFRb --------------------\ 1440 | | 1441 \- BFR1 - BFR2 - BFR3 - ... - BFR29 - BFR30 -/ 1442 | | L4 | | 1443 p33| p15| 1444 BFRd BFRc 1445 Figure 12: Ring Example 1447 Note that this example only permits for packets intended to make it 1448 all the way around the ring to enter it at BFRa and BFRb, and that 1449 packets will always travel clockwise. If packets should be allowed 1450 to enter the ring at any ring BFR, then one would have to use two 1451 ring bit positions. One for each direction: clockwise and 1452 counterclockwise. 1454 Both would be set up to stop rotating on the same link, e.g. L1. 1455 When the ingress ring BFR creates the clockwise copy, it will clear 1456 the counterclockwise bit position because the DNC bit only applies to 1457 the bit for which the replication is done. Likewise for the 1458 clockwise bit position for the counterclockwise copy. As a result, 1459 the ring ingress BFR will send a copy in both directions, serving 1460 BFRs on either side of the ring up to L1. 1462 5.1.7. Equal Cost MultiPath (ECMP) 1464 [RFC-Editor: A reviewer (Lars Eggert) noted that the infinite "to 1465 use" in the following sentence is not correct. The same was also 1466 noted for several other similar instances. The following URL seems 1467 to indicate though that this is a per-case decision, which seems 1468 undefined: https://writingcenter.gmu.edu/guides/choosing-between- 1469 infinitive-and-gerund-to-do-or-doing. What exactly should be done 1470 about this ?]. 1472 An ECMP() adjacency allows to use just one BP to deliver packets to 1473 one one of N adjacencies instead of one BP for each adjacency. In 1474 the common example case Figure 13, a link-bundle of three links 1475 L1,L2,L3 connects BFR1 and BFR2, and only one BP is used instead of 1476 three BP to deliver packets from BFR1 to BFR2. 1478 --L1----- 1479 BFR1 --L2----- BFR2 1480 --L3----- 1482 BIFT entry in BFR1: 1483 ------------------------------------------------------------------ 1484 | Index | Adjacencies | 1485 ================================================================== 1486 | 0:6 | ECMP({forward_connected(L1, BFR2), | 1487 | | forward_connected(L2, BFR2), | 1488 | | forward_connected(L3, BFR2)}, seed) | 1489 ------------------------------------------------------------------ 1491 BIFT entry in BFR2: 1492 ------------------------------------------------------------------ 1493 | Index | Adjacencies | 1494 ================================================================== 1495 | 0:6 | ECMP({forward_connected(L1, BFR1), | 1496 | | forward_connected(L2, BFR1), | 1497 | | forward_connected(L3, BFR1)}, seed) | 1498 ------------------------------------------------------------------ 1500 Figure 13: ECMP Example 1502 This document does not standardize any ECMP algorithm because it is 1503 sufficient for implementations to document their freely chosen ECMP 1504 algorithm. Figure 14 shows an example ECMP algorithm, and would 1505 double as its documentation: A BIER-TE controller could determine 1506 which adjacency is chosen based on the seed and adjacencies 1507 parameters and the packet entropy. 1509 forward(packet, ECMP(adj(0), adj(1),... adj(N-1), seed)): 1510 i = (packet(bier-header-entropy) XOR seed) % N 1511 forward packet to adj(i) 1513 Figure 14: ECMP algorithm Example 1515 In the following example, all traffic from BFR1 towards BFR10 is 1516 intended to be ECMP load split equally across the topology. This 1517 example is not meant as a likely setup, but to illustrate that ECMP 1518 can be used to share BPs not only across link bundles, but also 1519 across alternative paths across different transit BFR, and it 1520 explains the use of the seed parameter. 1522 BFR1 (BFIR) 1523 /L11 \L12 1524 / \ 1525 BFR2 BFR3 1526 /L21 \L22 /L31 \L32 1527 / \ / \ 1528 BFR4 BFR5 BFR6 BFR7 1529 \ / \ / 1530 \ / \ / 1531 BFR8 BFR9 1532 \ / 1533 \ / 1534 BFR10 (BFER) 1536 BIFT entry in BFR1: 1537 ------------------------------------------------------------------ 1538 | 0:6 | ECMP({forward_connected(L11, BFR2), | 1539 | | forward_connected(L12, BFR3)}, seed1) | 1540 ------------------------------------------------------------------ 1542 BIFT entry in BFR2: 1543 ------------------------------------------------------------------ 1544 | 0:7 | ECMP({forward_connected(L21, BFR4), | 1545 | | forward_connected(L22, BFR5)}, seed1) | 1546 ------------------------------------------------------------------ 1548 BIFT entry in BFR3: 1549 ------------------------------------------------------------------ 1550 | 0:7 | ECMP({forward_connected(L31, BFR6), | 1551 | | forward_connected(L32, BFR7)}, seed1) | 1552 ------------------------------------------------------------------ 1554 BIFT entry in BFR4, BFR5: 1555 ------------------------------------------------------------------ 1556 | 0:8 | forward_connected(Lxx, BFR8) |xx differs on BFR4/BFR5| 1557 ------------------------------------------------------------------ 1559 BIFT entry in BFR6, BFR7: 1560 ------------------------------------------------------------------ 1561 | 0:8 | forward_connected(Lxx, BFR9) |xx differs on BFR6/BFR7| 1562 ------------------------------------------------------------------ 1564 BIFT entry in BFR8, BFR9: 1565 ------------------------------------------------------------------ 1566 | 0:9 | forward_connected(Lxx, BFR10) |xx differs on BFR8/BFR9| 1567 ------------------------------------------------------------------ 1569 Figure 15: Polarization Example 1571 Note that for the following discussion of ECMP, only the BIFT ECMP 1572 adjacencies on BFR1, BFR2, BFR3 are relevant. The re-use of BP 1573 across BFR in this example is further explained in Section 5.1.9 1574 below. 1576 With the setup of ECMP in the topology above, traffic would not be 1577 equally load-split. Instead, links L22 and L31 would see no traffic 1578 at all: BFR2 will only see traffic from BFR1 for which the ECMP hash 1579 in BFR1 selected the first adjacency in the list of 2 adjacencies 1580 given as parameters to the ECMP. It is link L11-to-BFR2. BFR2 1581 performs again ECMP with two adjacencies on that subset of traffic 1582 using the same seed1, and will therefore again select the first of 1583 its two adjacencies: L21-to-BFR4. And therefore L22 and BFR5 sees no 1584 traffic. Likewise for L31 and BFR6. 1586 This issue in BFR2/BFR3 is called polarization. It results from the 1587 re-use of the same hash function across multiple consecutive hops in 1588 topologies like these. To resolve this issue, the ECMP() adjacency 1589 on BFR1 can be set up with a different seed2 than the ECMP() 1590 adjacencies on BFR2/BFR3. BFR2/BFR3 can use the same hash because 1591 packets will not sequentially pass across both of them. Therefore, 1592 they can also use the same BP 0:7. 1594 Note that ECMP solutions outside of BIER often hide the seed by auto- 1595 selecting it from local entropy such as unique local or next-hop 1596 identifiers. Allowing the BIER-TE Controller to explicitly set the 1597 seed gives the ability for it to control same/different path 1598 selection across multiple consecutive ECMP hops. 1600 5.1.8. Forward Routed adjacencies 1602 5.1.8.1. Reducing bit positions 1604 Forward_routed() adjacencies can reduce the number of bit positions 1605 required when the path steering requirement is not hop-by-hop 1606 explicit path selection, but loose-hop selection. Forward_routed() 1607 adjacencies can also allow to operate BIER-TE across intermediate hop 1608 routers that do not support BIER-TE. 1610 ............... 1611 ...BFR1--... ...--L1-- BFR2... 1612 ... .Routers. ...--L2--/ 1613 ...BFR4--... ...--L3-- BFR3... 1614 ... ...--L4--/ | 1615 ............... | 1616 LO 1617 Network Area 1 1619 Figure 16: Forward Routed Adjacencies Example 1621 Assume the requirement in Figure 16 is to explicitly steer traffic 1622 flows that have arrived at BFR1 or BFR4 via a path in the routing 1623 underlay "Network Area 1" to one of the following three next 1624 segments: (1) BFR2 via link L1, (2) BFR2 via link L2, or (3) via BFR3 1625 and then nor caring whether the packet is forwarded via L3 or L4. 1627 To enable this, both BFR1 and BFR4 are set up with a forward_routed 1628 adjacency bit position towards an address of BFR2 on link L1, another 1629 forward_routed() bit position towards an address of BFR2 on link L2 1630 and a third forward_routed() bit position towards a node address LO 1631 of BFR3. 1633 5.1.8.2. Supporting nodes without BIER-TE 1635 Forward_routed() adjacencies also enable incremental deployment of 1636 BIER-TE. Only the nodes through which BIER-TE traffic needs to be 1637 steered - with or without replication - need to support BIER-TE. 1638 Where they are not directly connected to each other, forward_routed 1639 adjacencies are used to pass over non BIER-TE enabled nodes. 1641 5.1.9. Reuse of bit positions (without DNC) 1643 Bit positions can be re-used across multiple BFRs to minimize the 1644 number of BP needed. This happens when adjacencies on multiple BFRs 1645 use the DNC flag as described above, but it can also be done for non- 1646 DNC adjacencies. This section only discusses this non-DNC case. 1648 Because BP are cleared when passing a BFR with an adjacency for that 1649 BP, reuse of BP across multiple BFRs does not introduce any problems 1650 with duplicates or loops that do not also exist when every adjacency 1651 has a unique BP. Instead, the challenge when reusing BP is whether 1652 it allows to still achieve the desired Tree Engineering goals. 1654 BP cannot be reused across two BFRs that would need to be passed 1655 sequentially for some path: The first BFR will clear the BP, so those 1656 paths cannot be built. BP can be set across BFR that would (A) only 1657 occur across different paths or (B) across different branches of the 1658 same tree. 1660 An example of (A) was given in Figure 15, where BP 0:7, BP 0:8 and BP 1661 0:9 are each reused across multiple BFRs because a single packet/path 1662 would never be able to reach more than one BFR sharing the same BP. 1664 Assume the example was changed: BFR1 has no ECMP() adjacency for BP 1665 0:6, but instead BP 0:5 with forward_connected() to BFR2 and BP 0:6 1666 with forward_connected() to BFR3. Packets with both BP 0:5 and BP 1667 0:6 would now be able to reach both BFR2 and BFR3 and the still 1668 existing re-use of BP 0:7 between BFR2 and BFR3 is a case of (B) 1669 where reuse of BP is perfect because it does not limit the set of 1670 useful path choices: 1672 If instead of reusing BP 0:7, BFR3 used a separate BP 0:10 for its 1673 ECMP() adjacency, no useful additional path steering options would be 1674 enabled. If duplicates at BFR10 where undesirable, this would be 1675 done by not setting BP 0:5 and BP 0:6 for the same packet. If the 1676 duplicates where desirable (e.g.: resilient transmission), the 1677 additional BP 0:10 would also not render additional value. 1679 area1 1680 BFR1a BFR1b 1681 / \ 1682 .................................... 1683 . Core . 1684 .................................... 1685 | / \ / \ | 1686 BFR2a BFR2b BFR3a BFR3b BFR6a BFR6b 1687 /-------\ /---------\ /--------\ 1688 | area2 | | area3 | ... | area6 | 1689 | ring | | ring | | ring | 1690 \-------/ \---------/ \--------/ 1691 more BFR more BFR more BFR 1693 Figure 17: Reuse of BP 1695 Reuse may also save BPs in larger topologies. Consider the topology 1696 shown in Figure 17. A BFIR/sender (e.g.: video headend) is attached 1697 to area 1, and area 2...6 contain receivers/BFER. Assume each area 1698 had a distribution ring, each with two BPs to indicate the direction 1699 (as explained before). These two BPs could be reused across the 5 1700 areas. Packets would be replicated through other BPs for the Core to 1701 the desired subset of areas, and once a packet copy reaches the ring 1702 of the area, the two ring BPs come into play. This reuse is a case 1703 of (B), but it limits the topology choices: Packets can only flow 1704 around the same direction in the rings of all areas. This may or may 1705 not be acceptable based on the desired path steering options: If 1706 resilient transmission is the path engineering goal, then it is 1707 likely a good optimization, if the bandwidth of each ring was to be 1708 optimized separately, it would not be a good limitation. 1710 5.1.10. Summary of BP optimizations 1712 This section reviewed a range of techniques by which a BIER-TE 1713 Controller can create a BIER-TE topology in a way that minimizes the 1714 number of necessary BPs. 1716 Without any optimization, a BIER-TE Controller would attempt to map 1717 the network subnet topology 1:1 into the BIER-TE topology and every 1718 subnet adjacent neighbor requires a forward_connected() BP and every 1719 BFER requires a local_decap() BP. 1721 The optimizations described are then as follows: 1723 * P2P links require only one BP (Section 5.1.1). 1725 * All leaf-BFER can share a single local_decap() BP (Section 5.1.3). 1727 * A LAN with N BFR needs at most N BP (one for each BFR). It only 1728 needs one BP for all those BFR that are not redundantly connected 1729 to multiple LANs (Section 5.1.4). 1731 * A hub with p2p connections to multiple non-leaf-BFER spokes can 1732 share one BP to all spokes if traffic can be flooded to all 1733 spokes, e.g.: because of no bandwidth concerns or dense receiver 1734 sets (Section 5.1.5). 1736 * Rings of BFR can be built with just two BP (one for each 1737 direction) except for BFR with multiple ring connections - similar 1738 to LANs (Section 5.1.6). 1740 * ECMP() adjacencies to N neighbors can replace N BP with 1 BP. 1741 Multihop ECMP can avoid polarization through different seeds of 1742 the ECMP algorithm (Section 5.1.7). 1744 * Forward_routed() adjacencies allow to "tunnel" across non-BIER-TE 1745 capable routers and across BIER-TE capable routers where no 1746 traffic-steering or replications are required (Section 5.1.8). 1748 * BP can generally be reused across a set of nodes where it can be 1749 guaranteed that no path will ever need to traverse more than one 1750 node of the set. Depending on scenario, this may limit the 1751 feasible path steering options (Section 5.1.9). 1753 Note that the described list of optimizations is not exhaustive. 1754 Especially when the set of required path steering choices is limited 1755 and the set of possible subsets of BFERs that should be able to 1756 receive traffic is limited, further optimizations of BP are possible. 1757 The hub and spoke optimization is a simple example of such traffic 1758 pattern dependent optimizations. 1760 5.2. Avoiding duplicates and loops 1761 5.2.1. Loops 1763 Whenever BIER-TE creates a copy of a packet, the BitString of that 1764 copy will have all bit positions cleared that are associated with 1765 adjacencies on the BFR. This inhibits looping of packets. The only 1766 exception are adjacencies with DNC set. 1768 v v 1769 | | 1770 L1 | L2 | L3 1771 /-------- BFRa ---- BFRb ---------------------\ 1772 | . | 1773 | ...... Wrong link wiring | 1774 | . | 1775 \- BFR1 - BFR2 BFR3 - ... - BFR29 - BFR30 -/ 1776 | | L4 | | 1777 p33| p15| 1778 BFRd BFRc 1780 Figure 18: Miswired Ring Example 1782 With DNC set, looping can happen. Consider in Figure 18 that link L4 1783 from BFR3 is (inadvertently) plugged into the L1 interface of BFRa 1784 (instead of BFR2). This creates a loop where the rings clockwise bit 1785 position is never cleared for copies of the packets traveling 1786 clockwise around the ring. 1788 To inhibit looping in the face of such physical misconfiguration, 1789 only forward_connected() adjacencies are permitted to have DNC set, 1790 and the link layer port unique unicast destination address of the 1791 adjacency (e.g. MAC address) protects against closing the loop. 1792 Link layers without port unique link layer addresses should not be 1793 used with the DNC flag set. 1795 5.2.2. Duplicates 1797 BFIR1 1798 / \ 1799 / p2 \ p3 1800 BFR2 BFR3 1801 \ p4 / p5 1802 \ / 1803 BFER4 1805 Figure 19: Duplicates Example 1807 Duplicates happen when the graph expressed by a BitString is not a 1808 tree but redundantly connecting BFRs with each other. In Figure 19, 1809 a BitString of p2,p3,p4,p5 would result in duplicate packets to 1810 arrive on BFER4. The BIER-TE Controller must therefore ensure to 1811 only create BitStrings that are trees. 1813 When links are incorrectly physically re-connected before the BIER-TE 1814 Controller updates BitStrings in BFIRs, duplicates can happen. Like 1815 loops, these can be inhibited by link layer addressing in 1816 forward_connected() adjacencies. 1818 If interface or loopback addresses used in forward_routed() 1819 adjacencies are moved from one BFR to another, duplicates can equally 1820 happen. Such re-addressing operations must be coordinated with the 1821 BIER-TE Controller. 1823 5.3. Managing SI, sub-domains and BFR-ids 1825 When the number of bits required to represent the necessary hops in 1826 the topology and BFER exceeds the supported BitStringLength (BSL), 1827 multiple SIs and/or sub-domains must be used. This section discusses 1828 how. 1830 BIER-TE forwarding does not require the concept of BFR-id, but 1831 routing underlay, flow overlay and BIER headers may. This section 1832 also discusses how BFR-ids can be assigned to BFIR/BFER for BIER-TE. 1834 5.3.1. Why SI and sub-domains 1836 For (non-TE) BIER and BIER-TE forwarding, the most important result 1837 of using multiple SI and/or sub-domains is the same: Packets that 1838 need to be sent to BFERs in different SIs or sub-domains require 1839 different BIER packets: each one with a BitString for a different 1840 (SI,sub-domain) combination. Each such BitString uses one BSL sized 1841 SI block in the BIFT of the sub-domain. We call this a BIFT:SI 1842 (block). 1844 For BIER and BIER-TE forwarding themselves there is also no 1845 difference whether different SIs and/or sub-domains are chosen, but 1846 SI and sub-domain have different purposes in the BIER architecture 1847 shared by BIER-TE. This impacts how operators are managing them and 1848 how especially flow overlays will likely use them. 1850 By default, every possible BFIR/BFER in a BIER network would likely 1851 be given a BFR-id in sub-domain 0 (unless there are > 64k BFIR/BFER). 1853 If there are different flow services (or service instances) requiring 1854 replication to different subsets of BFERs, then it will likely not be 1855 possible to achieve the best replication efficiency for all of these 1856 service instances via sub-domain 0. Ideal replication efficiency for 1857 N BFER exists in a sub-domain if they are split over not more than 1858 ceiling(N/BitStringLength) SI. 1860 If service instances justify additional BIER:SI state in the network, 1861 additional sub-domains will be used: BFIR/BFER are assigned BFR-id in 1862 those sub-domains and each service instance is configured to use the 1863 most appropriate sub-domain. This results in improved replication 1864 efficiency for different services. 1866 Even if creation of sub-domains and assignment of BFR-id to BFIR/BFER 1867 in those sub-domains is automated, it is not expected that individual 1868 service instances can deal with BFER in different sub-domains. A 1869 service instance may only support configuration of a single sub- 1870 domain it should rely on. 1872 To be able to easily reuse (and modify as little as possible) 1873 existing BIER procedures including flow-overlay and routing underlay, 1874 when BIER-TE forwarding is added, we therefore reuse SI and sub- 1875 domain logically in the same way as they are used in BIER: All 1876 necessary BFIR/BFER for a service use a single BIER-TE BIFT and are 1877 split across as many SIs as necessary (see Section 5.3.2). Different 1878 services may use different sub-domains that primarily exist to 1879 provide more efficient replication (and for BIER-TE desirable path 1880 steering) for different subsets of BFIR/BFER. 1882 5.3.2. Assigning bits for the BIER-TE topology 1884 In BIER, BitStrings only need to carry bits for BFERs, which leads to 1885 the model that BFR-ids map 1:1 to each bit in a BitString. 1887 In BIER-TE, BitStrings need to carry bits to indicate not only the 1888 receiving BFER but also the intermediate hops/links across which the 1889 packet must be sent. The maximum number of BFER that can be 1890 supported in a single BitString or BIFT:SI depends on the number of 1891 bits necessary to represent the desired topology between them. 1893 "Desired" topology because it depends on the physical topology, and 1894 on the desire of the operator to allow for explicit path steering 1895 across every single hop (which requires more bits), or reducing the 1896 number of required bits by exploiting optimizations such as unicast 1897 (forward_routed()), ECMP() or flood (DNC) over "uninteresting" sub- 1898 parts of the topology - e.g. parts where different trees do not need 1899 to take different paths due to path steering reasons. 1901 The total number of bits to describe the topology vs. the number of 1902 BFERs in a BIFT:SI can range widely based on the size of the topology 1903 and the amount of alternative paths in it. In a BIER-TE topology 1904 crafted by a BIER-TE expert, the higher the percentage of non-BFER 1905 bits, the higher the likelihood, that those topology bits are not 1906 just BIER-TE overhead without additional benefit, but instead that 1907 they will allow to express desirable path steering alternatives. 1909 5.3.3. Assigning BFR-id with BIER-TE 1911 BIER-TE forwarding does not use the BFR-id, nor does it require for 1912 the BFIR-id field of the BIER header to be set to a particular value. 1913 However, other parts of a BIER-TE deployment may need a BFR-id, 1914 specifically multicast flow overlay signaling and multicast flow 1915 overlay packet disposition, and in that case BFRs need to also have 1916 BFR-ids for BIER-TE SDs. 1918 For example, for BIER overlay signaling, BFIRs need to have a BFR-id, 1919 because this BFIR BFR-id is carried in the BFIR-id field of the BIER 1920 header to indicate to the overlay signaling on the receiving BFER 1921 which BFIR originated the packet. 1923 In BIER, BFR-id = SI * BSL + BP, such that the SI and BP of a BFER 1924 can be calculated from the BFR-id and vice versa. This also means 1925 that every BFR with a BFR-id has a reserved BP in an SI, even if that 1926 is not necessary for BIER forwarding, because the BFR may never be a 1927 BFER but only a BFIR. 1929 In BIER-TE, for a non-leaf BFER, there is usually a single BP for 1930 that BFER with a local_decap() adjacency on the BFER. The BFR-id for 1931 such a BFER can therefore be determined using the same procedure as 1932 in (non-TE) BIER: BFR-id = SI * BSL + BP. 1934 As explained in Section 5.1.3, leaf BFERs do not need such a unique 1935 local_decap() adjacency. Likewise, BFIRs that are not also BFERs may 1936 not have a unique local_decap() adjacency either. For all those 1937 BFIRs and (leaf) BFERs, the controller needs to determine unique BFR- 1938 ids that do not collide with the BFR-ids derived from the non-leaf 1939 BFER local_decap() BPs. 1941 While this document defines no requirements on how to allocate such 1942 BFR-id, a simple option is to derive it from the (SI,BP) of an 1943 adjacency that is unique to the BFR in question. For a BFIR this can 1944 be the first adjacency only populated on this BFIR, for a leaf-BFER, 1945 this could be the first BP with an adjacency towards that BFER. 1947 5.3.4. Mapping from BFR to BitStrings with BIER-TE 1949 In BIER, applications of the flow overlay on a BFIR can calculate the 1950 (SI,BP) of a BFER from the BFR-id of the BFER and can therefore 1951 easily determine the BitStrings for a BIER packet to a set of BFERs 1952 with known BFR-ids. 1954 In BIER-TE this mapping needs to be equally supported for flow 1955 overlays. This section outlines two core options, based on what type 1956 of Tree Engineering the BIER-TE controller needs to performs for a 1957 particular application. 1959 "Independent branches": For a given flow overlay instance, the 1960 branches from a BFIR to every BFER are calculated by the BIER-TE 1961 controller to be independent of the branches to any other BFER. 1962 Shortest path trees are the most common examples of trees with 1963 independent branches. 1965 "Interdependent branches": When a BFER is added or deleted from a 1966 particular distribution tree, the BIER-TE controller has to 1967 recalculate the branches to other BFER, because they may need to 1968 change. Steiner trees are examples of interdependent branch trees. 1970 If "independent branches" are used, the BIER-TE Controller can signal 1971 to the BFIR flow overlay for every BFER an SI:BitString that 1972 represents the branch to that BFER. The flow overlay on the BIFR can 1973 then independently of the controller calculate the SI:BitString for 1974 all desired BFERs by OR'ing their BitStrings. This allows for flow 1975 overlay applications to operate independently of the controller 1976 whenever it needs to determine which subset of BFERs need to receive 1977 a particular packet. 1979 If "interdependent branches" are required, the application would need 1980 to inquire the SI:BitString for a given set of BFER whenever the set 1981 changes. 1983 Note that in either case (unlike in BIER), the bits may need to 1984 change upon link/node failure/recovery, network expansion and network 1985 resource consumption by other traffic as part of traffic engineering 1986 goals (e.g.: re-optimization of lower priority traffic flows). 1987 Interactions between such BFIR applications and the BIER-TE 1988 Controller do therefore need to support dynamic updates to the 1989 SI:BitStrings. 1991 Communications between the BFIR flow overlay and the BIER-TE 1992 controller requires some way to identify the BFER. If BFR-ids are 1993 used in the deployment, as outlined in Section 5.3.3, then those are 1994 the natural BFR identifier. If BFR-ids are not used, then any other 1995 unique identifier, such as the BFR-prefix of the BFR ([RFC8279]) 1996 could be used. 1998 5.3.5. Assigning BFR-ids for BIER-TE 2000 It is not currently determined if a single sub-domain could or should 2001 be allowed to forward both (non-TE) BIER and BIER-TE packets. If 2002 this should be supported, there are two options: 2004 A. BIER and BIER-TE have different BFR-id in the same sub-domain. 2005 This allows higher replication efficiency for BIER because their BFR- 2006 id can be assigned sequentially, while the BitStrings for BIER-TE 2007 will have also the additional bits for the topology. There is no 2008 relationship between a BFR BIER BFR-id and its BIER-TE BFR-id. 2010 B. BIER and BIER-TE share the same BFR-id. The BFR-ids are assigned 2011 as explained above for BIER-TE and simply reused for BIER. The 2012 replication efficiency for BIER will be as low as that for BIER-TE in 2013 this approach. 2015 5.3.6. Example bit allocations 2017 5.3.6.1. With BIER 2019 Consider a network setup with a BSL of 256 for a network topology as 2020 shown in Figure 20. The network has 6 areas, each with 170 BFERs, 2021 connecting via a core with 4 (core) BFRs. To address all BFERs with 2022 BIER, 4 SIs are required. To send a BIER packet to all BFER in the 2023 network, 4 copies need to be sent by the BFIR. On the BFIR it does 2024 not make a difference how the BFR-ids are allocated to BFER in the 2025 network, but for efficiency further down in the network it does make 2026 a difference. 2028 area1 area2 area3 2029 BFR1a BFR1b BFR2a BFR2b BFR3a BFR3b 2030 | \ / \ / | 2031 ................................ 2032 . Core . 2033 ................................ 2034 | / \ / \ | 2035 BFR4a BFR4b BFR5a BFR5b BFR6a BFR6b 2036 area4 area5 area6 2038 Figure 20: Scaling BIER-TE bits by reuse 2040 With random allocation of BFR-id to BFER, each receiving area would 2041 (most likely) have to receive all 4 copies of the BIER packet because 2042 there would be BFR-id for each of the 4 SIs in each of the areas. 2043 Only further towards each BFER would this duplication subside - when 2044 each of the 4 trees runs out of branches. 2046 If BFR-ids are allocated intelligently, then all the BFER in an area 2047 would be given BFR-id with as few as possible different SIs. Each 2048 area would only have to forward one or two packets instead of 4. 2050 Given how networks can grow over time, replication efficiency in an 2051 area will then also go down over time when BFR-ids are only allocated 2052 sequentially, network wide. An area that initially only has BFR-id 2053 in one SI might end up with many SIs over a longer period of growth. 2054 Allocating SIs to areas with initially sufficiently many spare bits 2055 for growths can help to alleviate this issue. Or renumber BFERs 2056 after network expansion. In this example one may consider to use 6 2057 SIs and assign one to each area. 2059 This example shows that intelligent BFR-id allocation within at least 2060 sub-domain 0 can even be helpful or even necessary in BIER. 2062 5.3.6.2. With BIER-TE 2064 In BIER-TE one needs to determine a subset of the physical topology 2065 and attached BFERs so that the "desired" representation of this 2066 topology and the BFER fit into a single BitString. This process 2067 needs to be repeated until the whole topology is covered. 2069 Once bits/SIs are assigned to topology and BFERs, BFR-id is just a 2070 derived set of identifiers from the operator/BIER-TE Controller as 2071 explained above. 2073 Every time that different sub-topologies have overlap, bits need to 2074 be repeated across the BitStrings, increasing the overall amount of 2075 bits required across all BitString/SIs. In the worst case, one 2076 assigns random subsets of BFERs to different SIs. This will result 2077 in an outcome much worse than in (non-TE) BIER: It maximizes the 2078 amount of unnecessary topology overlap across SI and therefore 2079 reduces the number of BFER that can be reached across each individual 2080 SI. Intelligent BFER to SI assignment and selecting specific 2081 "desired" subtopologies can minimize this problem. 2083 To set up BIER-TE efficiently for the topology of Figure 20, the 2084 following bit allocation method can be used. This method can easily 2085 be expanded to other, similarly structured larger topologies. 2087 Each area is allocated one or more SIs depending on the number of 2088 future expected BFERs and number of bits required for the topology in 2089 the area. In this example, 6 SIs, one per area. 2091 In addition, we use 4 bits in each SI: bia, bib, bea, beb: (b)it 2092 (i)ngress (a), (b)it (i)ngress (b), (b)it (e)gress (a), (b)it 2093 (e)gress (b). These bits will be used to pass BIER packets from any 2094 BFIR via any combination of ingress area a/b BFR and egress area a/b 2095 BFR into a specific target area. These bits are then set up with the 2096 right forward_routed() adjacencies on the BFIR and area edge BFR: 2098 On all BFIRs in an area j|j=1...6, bia in each BIFT:SI is populated 2099 with the same forward_routed(BFRja), and bib with 2100 forward_routed(BFRjb). On all area edge BFR, bea in 2101 BIFT:SI=k|k=1...6 is populated with forward_routed(BFRka) and beb in 2102 BIFT:SI=k with forward_routed(BFRkb). 2104 For BIER-TE forwarding of a packet to a subset of BFERs across all 2105 areas, a BFIR would create at most 6 copies, with SI=1...SI=6, In 2106 each packet, the bits indicate bits for topology and BFER in that 2107 topology plus the four bits to indicate whether to pass this packet 2108 via the ingress area a or b border BFR and the egress area a or b 2109 border BFR, therefore allowing path steering for those two "unicast" 2110 legs: 1) BFIR to ingress area edge and 2) core to egress area edge. 2111 Replication only happens inside the egress areas. For BFER in the 2112 same area as in the BFIR, these four bits are not used. 2114 5.3.7. Summary 2116 BIER-TE can, like BIER, support multiple SIs within a sub-domain. 2117 This allows to apply the mapping BFR-id = SI * BSL + BP. This allows 2118 to re-use the BIER architecture concept of BFR-id and therefore 2119 minimize BIER-TE specific functions in possible BIER layer control 2120 plane mechanisms with BIER-TE, including flow overlay methods and 2121 BIER header fields. 2123 The number of BFIR/BFER possible in a sub-domain is smaller than in 2124 BIER because BIER-TE uses additional bits for topology. 2126 Sub-domains (SDs) in BIER-TE can be used like in BIER to create more 2127 efficient replication to known subsets of BFERs. 2129 Assigning bits for BFERs intelligently into the right SI is more 2130 important in BIER-TE than in BIER because of replication efficiency 2131 and overall amount of bits required. 2133 6. Security Considerations 2135 If [RFC8296] is used, BIER-TE shares its security considerations. 2137 BIER-TE shares the security considerations of BIER, [RFC8279], with 2138 the following overriding or additional considerations. 2140 BIER-TE forwarding explicitly supports unicast "tunneling" of BIER 2141 packets via forward_routed() adjacencies. The BIER domain security 2142 model is based on a subset of interfaces on a BFR that connect to 2143 other BFRs of the same BIER domain. For BIER-TE, this security model 2144 equally applies to such unicast "tunneled" BIER packets. This does 2145 not only include the need to filter received unicast "tunneled" BIER 2146 packets to prohibit injection of such "tunneled" BIER packets from 2147 outside the BIER domain, but also prohibiting forward_routed() 2148 adjacencies to leak BIER packets from the BIER domain. It SHOULD be 2149 possible to configure interfaces to be part of a BIER domain solely 2150 for sending and receiving of unicast "tunneled" BIER packets even if 2151 the interface can not send/receive BIER encapsulated packets. 2153 In BIER, the standardized methods for the routing underlays are IGPs 2154 with extensions to distribute BFR-ids and BFR-prefixes. [RFC8401] 2155 specifies the extensions for IS-IS and [RFC8444] specifies the 2156 extensions for OSPF. Attacking the protocols for the BIER routing 2157 underlay or (non-TE) BIER layer control plane, or impairment of any 2158 BFR in a domain may lead to successful attacks against the results of 2159 the routing protocol, enabling DoS attacks against paths or the 2160 addressing (BFR-id, BFR-prefixes) used by BIER. 2162 The reference model for the BIER-TE layer control plane is a BIER-TE 2163 controller. When such a controller is used, impairment of an 2164 individual BFR in a domain causes no impairment of the BIER-TE 2165 control plane on other BFRs. If a routing protocol is used to 2166 support forward_routed() adjacencies, then this is still an attack 2167 vector as in BIER, but only for BIER-TE forward_routed() adjacencies, 2168 and not other adjacencies. 2170 Whereas IGP routing protocols are most often not well secured through 2171 cryptographic authentication and confidentiality, communications 2172 between controllers and routers such as those to be considered for 2173 the BIER-TE controller/control-plane can be and are much more 2174 commonly secured with those security properties, for example by using 2175 Secure SHell (SSH), [RFC4253] for NETCONF ([RFC6242]), or via 2176 Transport Layer Security (TLS), such as [RFC8253] for PCEP, 2177 [RFC5440], or [RFC7589] for NETCONF. BIER-TE controllers SHOULD use 2178 security equal to or better than these mechanisms. 2180 When any of these security mechanisms/protocols are used for 2181 communications between a BIER-TE controller and BFRs, their security 2182 considerations apply to BIER-TE. In addition, the security 2183 considerations of PCE, [RFC4655] apply. 2185 The most important attack vector in BIER-TE is misconfiguration, 2186 either on the BFR themselves or via the BIER-TE controller. 2187 Forwarding entries with DNC could be set up to create persistent 2188 loops, in which packets only expire because of TTL. To minimize the 2189 impact of such attacks (or more likely unintentional misconfiguration 2190 by operators and/or bad BIER-TE controller software), the BIER-TE 2191 forwarding rules are defined to be as strict in clearing bits as 2192 possible. The clearing of all bits with an adjacency on a BFR 2193 prohibits that a looping packet creates additional packet 2194 amplification through the misconfigured loop on the packet's second 2195 or further times around the loop, because all relevant adjacency bits 2196 would have been cleared on the first round through the loop. In 2197 result, BIER-TE has the same degree of looping packets as possible 2198 with unintentional or malicious loops in the routing underlay with 2199 BIER or even with unicast traffic. 2201 Deployments where BIER-TE would likely be beneficial may include 2202 operational models where actual configuration changes from the 2203 controller are only required during non-production phases of the 2204 network's life-cycle, such as in embedded networks or in 2205 manufacturing networks during e.g. plant reworking/repairs. In these 2206 type of deployments, configuration changes could be locked out when 2207 the network is in production state and could only be (re-)enabled 2208 through reverting the network/installation into non-production state. 2209 Such security designs would not only allow to provide additional 2210 layers of protection against configuration attacks, but would 2211 foremost protect the active production process from such 2212 configuration attacks. 2214 7. IANA Considerations 2216 This document requests no action by IANA. 2218 8. Acknowledgements 2220 The authors would like to thank Greg Shepherd, Ijsbrand Wijnands, 2221 Neale Ranns, Dirk Trossen, Sandy Zheng, Lou Berger, Jeffrey Zhang, 2222 Carsten Borman and Wolfgang Braun for their reviews and suggestions. 2224 Special thanks to Xuesong Geng for shepherding the document and for 2225 IESG review/suggestions by Alvaro Retana (responsible AD/RTG), 2226 Benjamin Kaduk (SEC), Tommy Pauly (TSV), Zaheduzzaman Sarker (TSV), 2227 Eric Vyncke (INT), Martin Vigoureux (RTG), Robert Wilton (OPS), Eric 2228 Kline (INT), Lars Eggert (GEN), Roman Danyliv (SEC), Ines Robles 2229 (RTGDIR), Robert Sparks (Gen-ART), Yingzhen Qu (RTGdir), Martin Duke 2230 (TSV). 2232 9. Change log [RFC Editor: Please remove] 2234 draft-ietf-bier-te-arch: 2236 12: 2238 AD review Alvaro Retana. 2240 Various textual/editorial nits including adding () to all 2241 instances of forwarding adjacency name instances. 2243 3.1 Added new paragraph outlining possible use of BGP as RR in 2244 BIER-TE controller as core of multicast flow overlay component of 2245 BIER-TE. 2247 3.2 added xref's to relevant sections to the listed control plane 2248 points. 2250 4.1 rewrote paragraphs of 4.1 leading up to Figure 4. to eliminate 2251 any confusion in how the BIFT work and how it compares to the 2252 notions in rfc8279, as well as better linking it to the 2253 Pseudocode. 2255 Moved SR section into appendix. 2257 TSV review Martin Duke. 2259 Text/editorial nits. 2261 4.4 improved text describing handling of F-BM. 2263 RTGdir review Yingzhen Qu. 2265 Various text/editorial nits. 2267 Added notion that BitStrings represent loop free tree for packet 2268 to abstract and intro. 2270 Various text nit and editorial improvements. 2272 Fixed some BFR-id field -> BFIR-id field mistakes. 2274 Capitalized NETCONF/RESTCONF/YANG, added RFC references. 2276 Improved Figure 16 with explicitly two links into BFR3 and 2277 explanatory text. 2279 Gen-ART review Robert Sparks. 2281 Various textual nits, editorial improvements. 2283 3.2 Introduced terms "BIER-TE topology control" and "BIER-TE tree 2284 control" for the two functional components of the control plane. 2286 3.2.1 - 3.2 change introduces the open RFC-editor issue of 2287 appropriate xrfs (to be resolved by RFC-editor). 2289 3.3 Rewrote last paragraph to better describe loop prevention 2290 through clearing of bits in BitString. 2292 4.1 Fixed up text/formula describing mapping between bfr-id, SI:BP 2293 and SI,BSL and BP. Fix offset bug. 2295 5.3.6.2 Improved description paragraph explaining overlap of 2296 topology for different SI. 2298 5.3.7 Improved first summary paragraph. 2300 7. Rephrased applicability statement of control plane protocol 2301 security considerations to BIER-TE security. 2303 RTGDIR review Ines Robles. 2305 Fixed up adjacencies in Example 2 and explanation text to be 2306 explicit about which BFR not only passes, but also receives the 2307 packet. 2309 7. (security considerations). Added paragraph about 2310 forward_routed() and prohibiting BIER packet leaking in/out of 2311 domain. 2313 IESG review Roman Danyliv (SEC). 2315 Several textual/sentence nits/editorials. 2317 IESG review Lars Eggert (GEN). 2319 Various good editorial word fixed. 2321 Pointer to non-false-positive bloom filter work that looks like it 2322 happened after our IETF discussions documented in this doc, so 2323 will not add it to doc, but here is URL for folks interested: 2324 https://ieeexplore.ieee.org/document/8486415. 2326 Did not change "native" to a different word for inclusivity 2327 because of my worry there is no estavblished single replacement 2328 word, making reading/searching/understanding more difficult. 2330 IESG review Martin Vigeureux (RTG). 2332 Added back reference to RFC8402. Textual fixes. 2334 IESG review Eric Kline (INT). 2336 2.1 Fixed typo in BFR* explanations. 2338 4.3 Added explanatio about MTU handling. 2340 IESG review Eric Vyncke (INT). 2342 Fixed up initial text to introduce various abbreviations. 2344 2.4 refined wording to "with the _intent_ to easily build common 2345 forwarding planes...". 2347 4.2.3 refined text about entropy in ECMP - now taken text from 2348 rfc8279. 2350 IESG review Zaheduzzaman Sarker (TSV). 2352 5.1.7 Refined text explaining documentation of ECMP algorithm. 2354 5.3.6.2. fixed range of areas/SI over which to build the example 2355 large network BPs - removed explanation of the large network shown 2356 to be only used for sources in area 1 (IPTV), because it was a 2357 stale explanation. 2359 IESG review Ben Kaduk (round 2): 2361 4.4 Advanced pseudocode still had one wrong "~". Root cause seems 2362 to have been day 0 problem in pseudocode written for -01, "~" was 2363 inserted in the wrong one of two code lines. Also enhanced 2364 textual description and comments in pseudocode, changed variable 2365 name AdjacentBits to PktAdjacentBits to avoid confusion with 2366 AdjacentBits[SI]. 2368 5.1.3 Rewrote last two paragraphs explaining the sharing of bit 2369 positions for lead-BFER hopefully better. Also detailled how it 2370 interacts with other optimizations and the type of payload BIER-TE 2371 packets may carry. 2373 4.4 (from Carsten Borman) changed spacing in pseudocode to be 2374 consistent. Fixed {VRF}, clarified pseudocode object syntax, 2375 typos. 2377 11: IESG review Ben Kaduk, summary: 2379 One discuss for bug in pseudocode. turned out to be one cahrcter 2380 typo. 2382 Added (non-TE) prefix in places where BIER by itsels had to be 2383 better disambiguated. 2385 enhanced text for hub-and-spoke to indicate we're only talking 2386 about hub to spoke traffic. 2388 long list ot language fixes/improvement (nits). Thanks a lot!. 2390 add suggestion to SHOULD use known confidentiality protocols 2391 between controller and BFR. 2393 10: AD review Alvaro Retana, summary: 2395 Note: rfcdiff shows more changes than actually exist because text 2396 moved around. 2398 Summary: 2400 1. restructuring: merged all controller sections under common 2401 controller ops main section, moved unfitting stuff out to 2402 other parts of doc. Split Intro section into Overview and 2403 Intro. Shortened Abstract, moved text into Overview, added 2404 sections overview. 2406 2. enhanced/rewrote: 2.3 Comparison with -> Relationship to BIER- 2407 TE 2409 3. enhanced/rewrote: 3.2 BIER-TE controller -> BIER-TE control 2410 plane, 3.2.1 BIER-TE controller, for consistency with rfc8279 2412 4. additional subsections for Alvaros asks 2414 5. added to: 3.3 BIER-TE forwarding plane (consistency with 2415 rfc8279) 2417 6. Enhanced description of 4.3/encap considerations to better 2418 explain how BIER/BIER-TE can run together. 2420 Notation: Markers (a),(b),... at end of points are references from 2421 the review discussion with Alvaro to the changes made. 2423 Details:. 2425 Throughout text: changed term spelling to rfc8279 - bit positions, 2426 sub-domain, ... (i). 2428 Reset changed to clear, also DNR changed to DNC (Do Not Clear) 2429 (q). 2431 Abstract: Shortened. Removed name explanation note (Tree 2432 Engineering), (a). 2434 1. Introduction -> Overview: Moved important explanation 2435 paragraph from abstract to Introduction. Fixed text, (a). 2437 Added bullet point list explanation of structure of document (e). 2439 Renamed to Overview because that is now more factually correct. 2441 1.1. Fixed bug in example adding bit p15.(l). 2443 2. (New - Introduction): Moved section 1.1 - 1.3 (examples, 2444 comparison with BIER-TE) from Introduction into new "Overview" 2445 section. Primarily so that "requirements language" section (at 2446 end of Introduction) is not in middle of document after all the 2447 Introduction. 2449 2.1 Removed discussion of encap, moved to 4.2.2 (m). 2451 2.2 enhanced paragraph suggesting native/overlay topology types, 2452 also sugest type hybrid (n). 2454 2.3 Overhauled comparison text BIER/BIER-TE, structured into 2455 common, different, not-required-by-te, integration-bier-bier-te. 2456 Changed title to "Relationship" to allow including last point. 2457 (f). 2459 2.4 moved Hardware forwarding comparison section into section 2 to 2460 allow coalescing of sections into section 5 about the controller 2461 operations (hardware forwarding was in the middle of it, wrong 2462 place). Shortened/improved third paragraph by pointing to BIFT as 2463 deciding element for selection between BIER/BIER-TE. Removed 2464 notion of experimentation (this now targets standard) (g). 2466 3. (Components): Aligned component name and descriptions better 2467 with RFC8279. Now describe exactly same three layers. BIER layer 2468 constituted from BIER-TE control plane and BIER-TE forwarding 2469 plane. BIER-TE controller is now simply component of BIER-TE 2470 control plane. (b). 2472 3.1. shortened/improved paragraph explaining use of SI:BP instead 2473 of also bfr-id as index into BIFT, rewrote paragraph talking about 2474 reuse of BPs(o). 2476 3.2. rewrote explanation of BIER-TE control plane in the style of 2477 RFC8729 Section 4.2 (BIER layer) with numbered points. Note that 2478 RFC8729 mixes control and forwarding plane bullet points (this doc 2479 does not). Merged text from old sections 2.2.1 and 2.2.3 into 2480 list. (b). 2482 3.2.1. Expanded/improved explanation of BIER-TE Controller (b). 2484 3.2.1.1. Added subsection for topology discovery and creation 2485 (d). 2487 3.2.1.2. Added subsection for engineered BitStrings as key novel 2488 aspect not found in BIER. (X). 2490 3.3. Added numbered list for components of BIER-TE forwarding 2491 plane (completing the comparable text from RFC8729 Section 4.2). 2493 3.4 Alvaro does not mind additional example, fixed bugs. 2495 3.5 Removed notion about using IGP BIER extensions for BIER-TE, 2496 such as BIFT address ranges. After -10 making use of BIFT 2497 clearer, it now looks to authors as if use of IGP extensions would 2498 not be beneficial, as long as we do need to use the BIER-TE 2499 controller, e.g. unlike in BIER, a BFR could not learn from the 2500 IGP information what traffic to send towards a particular BIFT-ID, 2501 but instead that is the core of what the controller needs to 2502 provide. 2504 4.2.2 Improved text to explain requirement to identify BIER-TE in 2505 the tunnel encap and compress description of use-cases (m). 2507 4.2.3 enhanced ECMP text (p). 2509 4.3. rewrote most of Encapsulation Considerations to better 2510 explain to Alvaros question re sharing or not sharing SD via BIER/ 2511 BIER-TE. Added reference to I-D.ietf-bier-non-mpls-bift-encoding 2512 as a very helpful example. (f). 2514 4.3 Renamed title to "...Co-Existence with BIER" as this is what 2515 it is about and to help finding it from abstract/intro ("co- 2516 exist") (j). 2518 4.4. Moved BIER-TE Forwarding Pseudocode here to coalesce text 2519 logically. Changed text to better compare with BIER pseudo 2520 forwarding code. Numerical list of how F-BM works for BIER-TE. 2521 Removed efficiency comparison with BIER (too difficult to provide 2522 sufficient justification, derails from focus of section) (j). 2524 4.6. (Requirements) Restructured: Removed notion of "basic" BIER- 2525 TE forwarding, simply referring to it now as "mandatory" BIER-TE 2526 forwarding. Cleaned up text to have requirements for different 2527 adjacencies in different paragraphs. (c). 2529 5. Created new main section "BIER-TE Controller operational 2530 considerations", coalesced old sections 4., 5., 7. into this new 2531 main section. No text changes. (k). 2533 5.1.9 Added new separate picture instead of referring to a picture 2534 later in text, adjusted text (r). 2536 5.3.2 Changed title to not include word "comparison" to avoid this 2537 being accounted against Alvaros concern about scattering 2538 comparison (IMHO text already has little comparison, so title was 2539 misleading) (h). 2541 co-authors internal review: 2543 4.4 Added xref to Figure 5. 2545 5.2.1 Duplicated ring picture, added visuals for described 2546 miswiring (s). 2548 5.2.2 replace "topology" with graph (wrong word). 2550 5.3.3 rewrote explanation of how to map BFR-id to SI:BP and assign 2551 them, clarified BFR-id is option. Retitled to better explain 2552 scope of section. 2554 5.3.4 Removed considerations in 5.3.4 for sharing BFR-id across 2555 BIER/BIER-TE (t), changed title to explain how BFIR/BIER-TE 2556 controller interactions need some form of identifying BFR but this 2557 does not have to be BFR-id. 2559 7. Added new security considerations (u). 2561 09: Incorporated fixes for feedback from Shepherd (Xuesong Geng). 2563 Added references for Bloom Filters and Rate Controlled Service 2564 Disciplines. 2566 1.1 Fixed numbering of example 1 topology explanation. Improved 2567 language on second example (less abbreviating to avoid confusion 2568 about meaning). 2570 1.2 Improved explanation of BIER-TE topology, fixed terminology of 2571 graphs (BIER-TE topology is a directed graph where the edges are 2572 the adjacencies). 2574 2.4 Fixed and amended routing underlay explanations: detailed why 2575 no need for BFER routing underlay routing protocol extensions, but 2576 potential to re-use BIER routing underlay routing protocol 2577 extensions for non-BFER related extensions. 2579 3.1 Added explanation for VRF and its use in adjacencies. 2581 08: Incorporated (with hopefully acceptable fixes) for Lou 2582 suggested section 2.5, TE considerations. 2584 Fixes are primarily to the point to a) emphasize that BIER-TE does 2585 not depend on the routing underlay unless forward_routed() 2586 adjacencies are used, and b) that the allocation and tracking of 2587 resources does not explicitly have to be tied to BPs, because they 2588 are just steering labels. Instead, it would ideally come from 2589 per-hop resource management that can be maintained only via local 2590 accounting in the controller. 2592 07: Further reworking text for Lou. 2594 Renamed BIER-PE to BIER-TE standing for "Tree Engineering" after 2595 votes from BIER WG. 2597 Removed section 1.1 (introduced by version 06) because not 2598 considered necessary in this doc by Lou (for framework doc). 2600 Added [RFC editor pls. remove] Section to explain name change to 2601 future reviewers. 2603 06: Concern by Lou Berger re. BIER-TE as full traffic engineering 2604 solution. 2606 Changed title "Traffic Engineering" to "Path Engineering" 2607 Added intro section of relationship BIER-PE to traffic 2608 engineering. 2610 Changed "traffic engineering" term in text" to "path engineering", 2611 where appropriate 2613 Other: 2615 Shortened "BIER-TE Controller Host" to "BIER-TE Controller". 2616 Fixed up all instances of controller to do this. 2618 05: Review Jeffrey Zhang. 2620 Part 2: 2622 4.3 added note about leaf-BFER being also a propery of routing 2623 setup. 2625 4.7 Added missing details from example to avoid confusion with 2626 routed adjacencies, also compressed explanatory text and better 2627 justification why seed is explicitly configured by controller. 2629 4.9 added section discussing generic reuse of BP methods. 2631 4.10 added section summarizing BP optimizations of section 4. 2633 6. Rewrote/compressed explanation of comparison BIER/BIER-TE 2634 forwarding difference. Explained benefit of BIER-TE per-BP 2635 forwarding being independent of forwarding for other BPs. 2637 Part 1: 2639 Explicitly ue forwarded_connected adjcency in ECMP adjcency 2640 examples to avoid confusion. 2642 4.3 Add picture as example for leav vs. non-leaf BFR in topology. 2643 Improved description. 2645 4.5 Exampe for traffic that can be broadcast -> for single BP in 2646 hub&spoke. 2648 4.8.1 Simplified example picture for routed adjacency, explanatory 2649 text. 2651 Review from Dirk Trossen: 2653 Fixed up explanation of ICC paper vs. bloom filter. 2655 04: spell check run. 2657 Addded remaining fixes for Sandys (Zhang Zheng) review: 2659 4.7 Enhance ECMP explanations: 2661 example ECMP algorithm, highlight that doc does not standardize 2662 ECMP algorithm. 2664 Review from Dirk Trossen: 2666 1. Added mentioning of prior work for traffic engineered paths 2667 with bloom filters. 2669 2. Changed title from layers to components and added "BIER-TE 2670 control plane" to "BIER-TE Controller" to make it clearer, what it 2671 does. 2673 2.2.3. Added reference to I-D.ietf-bier-multicast-http-response 2674 as an example solution. 2676 2.3. clarified sentence about resetting BPs before sending copies 2677 (also forgot to mention DNR here). 2679 3.4. Added text saying this section will be removed unless IESG 2680 review finds enough redeeming value in this example given how -03 2681 introduced section 1.1 with basic examples. 2683 7.2. Removed explicit numbers 20%/80% for number of topology bits 2684 in BIER-TE, replaced with more vague (high/low) description, 2685 because we do not have good reference material Added text saying 2686 this section will be removed unless IESG review finds enough 2687 redeeming value in this example given how -03 introduced section 2688 1.1 with basic examples. 2690 many typos fixed. Thanks a lot. 2692 03: Last call textual changes by authors to improve readability: 2694 removed Wolfgang Braun as co-authors (as requested). 2696 Improved abstract to be more explanatory. Removed mentioning of 2697 FRR (not concluded on so far). 2699 Added new text into Introduction section because the text was too 2700 difficult to jump into (too many forward pointers). This 2701 primarily consists of examples and the early introduction of the 2702 BIER-TE Topology concept enabled by these examples. 2704 Amended comparison to SR. 2706 Changed syntax from [VRF] to {VRF} to indicate its optional and to 2707 make idnits happy. 2709 Split references into normative / informative, added references. 2711 02: Refresh after IETF104 discussion: changed intended status back 2712 to standard. Reasoning: 2714 Tighter review of standards document == ensures arch will be 2715 better prepared for possible adoption by other WGs (e.g. DetNet) 2716 or std. bodies. 2718 Requirement against the degree of existing implementations is self 2719 defined by the WG. BIER WG seems to think it is not necessary to 2720 apply multiple interoperating implementations against an 2721 architecture level document at this time to make it qualify to go 2722 to standards track. Also, the levels of support introduced in -01 2723 rev. should allow all BIER forwarding engines to also be able to 2724 support the base level BIER-TE forwarding. 2726 01: Added note comparing BIER and SR to also hopefully clarify 2727 BIER-TE vs. BIER comparison re. SR. 2729 - added requirements section mandating only most basic BIER-TE 2730 forwarding features as MUST. 2732 - reworked comparison with BIER forwarding section to only 2733 summarize and point to pseudocode section. 2735 - reworked pseudocode section to have one pseudocode that mirrors 2736 the BIER forwarding pseudocode to make comparison easier and a 2737 second pseudocode that shows the complete set of BIER-TE 2738 forwarding options and simplification/optimization possible vs. 2739 BIER forwarding. Removed MyBitsOfInterest (was pure 2740 optimization). 2742 - Added captions to pictures. 2744 - Part of review feedback from Sandy (Zhang Zheng) integrated. 2746 00: Changed target state to experimental (WG conclusion), updated 2747 references, mod auth association. 2749 - Source now on https://www.github.com/toerless/bier-te-arch 2750 - Please open issues on the github for change/improvement requests 2751 to the document - in addition to posting them on the list 2752 (bier@ietf.). Thanks!. 2754 draft-eckert-bier-te-arch: 2756 06: Added overview of forwarding differences between BIER, BIER- 2757 TE. 2759 05: Author affiliation change only. 2761 04: Added comparison to Live-Live and BFIR to FRR section 2762 (Eckert). 2764 04: Removed FRR content into the new FRR draft [I-D.eckert-bier- 2765 te-frr] (Braun). 2767 - Linked FRR information to new draft in Overview/Introduction 2769 - Removed BTAFT/FRR from "Changes in the network topology" 2771 - Linked new draft in "Link/Node Failures and Recovery" 2773 - Removed FRR from "The BIER-TE Forwarding Layer" 2775 - Moved FRR section to new draft 2777 - Moved FRR parts of Pseudocode into new draft 2779 - Left only non FRR parts 2781 - removed FrrUpDown(..) and //FRR operations in 2782 ForwardBierTePacket(..) 2784 - New draft contains FrrUpDown(..) and ForwardBierTePacket(Packet) 2785 from bier-arch-03 2787 - Moved "BIER-TE and existing FRR to new draft 2789 - Moved "BIER-TE and Segment Routing" section one level up 2791 - Thus, removed "Further considerations" that only contained this 2792 section 2794 - Added Changes for version 04 2795 03: Updated the FRR section. Added examples for FRR key concepts. 2796 Added BIER-in-BIER tunneling as option for tunnels in backup 2797 paths. BIFT structure is expanded and contains an additional 2798 match field to support full node protection with BIER-TE FRR. 2800 03: Updated FRR section. Explanation how BIER-in-BIER 2801 encapsulation provides P2MP protection for node failures even 2802 though the routing underlay does not provide P2MP. 2804 02: Changed the definition of BIFT to be more inline with BIER. 2805 In revs. up to -01, the idea was that a BIFT has only entries for 2806 a single BitString, and every SI and sub-domain would be a 2807 separate BIFT. In BIER, each BIFT covers all SI. This is now 2808 also how we define it in BIER-TE. 2810 02: Added Section 5.3 to explain the use of SI, sub-domains and 2811 BFR-id in BIER-TE and to give an example how to efficiently assign 2812 bits for a large topology requiring multiple SI. 2814 02: Added further detailed for rings - how to support input from 2815 all ring nodes. 2817 01: Fixed BFIR -> BFER for section 4.3. 2819 01: Added explanation of SI, difference to BIER ECMP, 2820 consideration for Segment Routing, unicast FRR, considerations for 2821 encapsulation, explanations of BIER-TE Controller and CLI. 2823 00: Initial version. 2825 10. References 2827 10.1. Normative References 2829 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2830 Requirement Levels", BCP 14, RFC 2119, 2831 DOI 10.17487/RFC2119, March 1997, 2832 . 2834 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2835 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 2836 May 2017, . 2838 [RFC8279] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., 2839 Przygienda, T., and S. Aldrin, "Multicast Using Bit Index 2840 Explicit Replication (BIER)", RFC 8279, 2841 DOI 10.17487/RFC8279, November 2017, 2842 . 2844 [RFC8296] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., 2845 Tantsura, J., Aldrin, S., and I. Meilik, "Encapsulation 2846 for Bit Index Explicit Replication (BIER) in MPLS and Non- 2847 MPLS Networks", RFC 8296, DOI 10.17487/RFC8296, January 2848 2018, . 2850 10.2. Informative References 2852 [Bloom70] Bloom, B. H., "Space/time trade-offs in hash coding with 2853 allowable errors", Comm. ACM 13(7):422-6, July 1970, 2854 . 2856 [I-D.eckert-bier-te-frr] 2857 Eckert, T., Cauchie, G., Braun, W., and M. Menth, 2858 "Protection Methods for BIER-TE", Work in Progress, 2859 Internet-Draft, draft-eckert-bier-te-frr-03, 5 March 2018, 2860 . 2863 [I-D.ietf-bier-multicast-http-response] 2864 Trossen, D., Rahman, A., Wang, C., and T. Eckert, 2865 "Applicability of BIER Multicast Overlay for Adaptive 2866 Streaming Services", Work in Progress, Internet-Draft, 2867 draft-ietf-bier-multicast-http-response-06, 10 July 2021, 2868 . 2871 [I-D.ietf-bier-non-mpls-bift-encoding] 2872 Wijnands, I., Mishra, M., Xu, X., and H. Bidgoli, "An 2873 Optional Encoding of the BIFT-id Field in the non-MPLS 2874 BIER Encapsulation", Work in Progress, Internet-Draft, 2875 draft-ietf-bier-non-mpls-bift-encoding-04, 30 May 2021, 2876 . 2879 [I-D.ietf-bier-te-yang] 2880 Zhang, Z., Wang, C., Chen, R., Hu, F., Sivakumar, M., and 2881 H. Chen, "A YANG data model for Tree Engineering for Bit 2882 Index Explicit Replication (BIER-TE)", Work in Progress, 2883 Internet-Draft, draft-ietf-bier-te-yang-04, 7 November 2884 2021, . 2887 [I-D.ietf-roll-ccast] 2888 Bergmann, O., Bormann, C., Gerdes, S., and H. Chen, 2889 "Constrained-Cast: Source-Routed Multicast for RPL", Work 2890 in Progress, Internet-Draft, draft-ietf-roll-ccast-01, 30 2891 October 2017, . 2894 [I-D.ietf-teas-rfc3272bis] 2895 Farrel, A., "Overview and Principles of Internet Traffic 2896 Engineering", Work in Progress, Internet-Draft, draft- 2897 ietf-teas-rfc3272bis-13, 8 November 2021, 2898 . 2901 [ICC] Reed, M. J., Al-Naday, M., Thomos, N., Trossen, D., 2902 Petropoulos, G., and S. Spirou, "Stateless multicast 2903 switching in software defined networks", IEEE 2904 International Conference on Communications (ICC), Kuala 2905 Lumpur, Malaysia, 2016, May 2016, 2906 . 2908 [RCSD94] Zhang, H. and D. Domenico, "Rate-Controlled Service 2909 Disciplines", Journal of High-Speed Networks, 1994, May 2910 1994, . 2912 [RFC4253] Ylonen, T. and C. Lonvick, Ed., "The Secure Shell (SSH) 2913 Transport Layer Protocol", RFC 4253, DOI 10.17487/RFC4253, 2914 January 2006, . 2916 [RFC4456] Bates, T., Chen, E., and R. Chandra, "BGP Route 2917 Reflection: An Alternative to Full Mesh Internal BGP 2918 (IBGP)", RFC 4456, DOI 10.17487/RFC4456, April 2006, 2919 . 2921 [RFC4655] Farrel, A., Vasseur, J.-P., and J. Ash, "A Path 2922 Computation Element (PCE)-Based Architecture", RFC 4655, 2923 DOI 10.17487/RFC4655, August 2006, 2924 . 2926 [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation 2927 Element (PCE) Communication Protocol (PCEP)", RFC 5440, 2928 DOI 10.17487/RFC5440, March 2009, 2929 . 2931 [RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., 2932 and A. Bierman, Ed., "Network Configuration Protocol 2933 (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, 2934 . 2936 [RFC6242] Wasserman, M., "Using the NETCONF Protocol over Secure 2937 Shell (SSH)", RFC 6242, DOI 10.17487/RFC6242, June 2011, 2938 . 2940 [RFC7589] Badra, M., Luchuk, A., and J. Schoenwaelder, "Using the 2941 NETCONF Protocol over Transport Layer Security (TLS) with 2942 Mutual X.509 Authentication", RFC 7589, 2943 DOI 10.17487/RFC7589, June 2015, 2944 . 2946 [RFC7752] Gredler, H., Ed., Medved, J., Previdi, S., Farrel, A., and 2947 S. Ray, "North-Bound Distribution of Link-State and 2948 Traffic Engineering (TE) Information Using BGP", RFC 7752, 2949 DOI 10.17487/RFC7752, March 2016, 2950 . 2952 [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", 2953 RFC 7950, DOI 10.17487/RFC7950, August 2016, 2954 . 2956 [RFC7988] Rosen, E., Ed., Subramanian, K., and Z. Zhang, "Ingress 2957 Replication Tunnels in Multicast VPN", RFC 7988, 2958 DOI 10.17487/RFC7988, October 2016, 2959 . 2961 [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF 2962 Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, 2963 . 2965 [RFC8253] Lopez, D., Gonzalez de Dios, O., Wu, Q., and D. Dhody, 2966 "PCEPS: Usage of TLS to Provide a Secure Transport for the 2967 Path Computation Element Communication Protocol (PCEP)", 2968 RFC 8253, DOI 10.17487/RFC8253, October 2017, 2969 . 2971 [RFC8345] Clemm, A., Medved, J., Varga, R., Bahadur, N., 2972 Ananthakrishnan, H., and X. Liu, "A YANG Data Model for 2973 Network Topologies", RFC 8345, DOI 10.17487/RFC8345, March 2974 2018, . 2976 [RFC8401] Ginsberg, L., Ed., Przygienda, T., Aldrin, S., and Z. 2977 Zhang, "Bit Index Explicit Replication (BIER) Support via 2978 IS-IS", RFC 8401, DOI 10.17487/RFC8401, June 2018, 2979 . 2981 [RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L., 2982 Decraene, B., Litkowski, S., and R. Shakir, "Segment 2983 Routing Architecture", RFC 8402, DOI 10.17487/RFC8402, 2984 July 2018, . 2986 [RFC8444] Psenak, P., Ed., Kumar, N., Wijnands, IJ., Dolganow, A., 2987 Przygienda, T., Zhang, J., and S. Aldrin, "OSPFv2 2988 Extensions for Bit Index Explicit Replication (BIER)", 2989 RFC 8444, DOI 10.17487/RFC8444, November 2018, 2990 . 2992 [RFC8556] Rosen, E., Ed., Sivakumar, M., Przygienda, T., Aldrin, S., 2993 and A. Dolganow, "Multicast VPN Using Bit Index Explicit 2994 Replication (BIER)", RFC 8556, DOI 10.17487/RFC8556, April 2995 2019, . 2997 Appendix A. BIER-TE and Segment Routing 2999 SR (xref target="RFC8402"/>) aims to enable lightweight path steering 3000 via loose source routing. Compared to its more heavy-weight 3001 predecessor RSVP-TE, SR does for example not require per-path 3002 signaling to each of these hops. 3004 BIER-TE supports the same design philosophy for multicast. Like in 3005 SR, it relies on source-routing - via the definition of a BitString. 3006 Like SR, it only requires to consider the "hops" on which either 3007 replication has to happen, or across which the traffic should be 3008 steered (even without replication). Any other hops can be skipped 3009 via the use of routed adjacencies. 3011 BIER-TE bit position (BP) can be understood as the BIER-TE equivalent 3012 of "forwarding segments" in SR, but they have a different scope than 3013 SR forwarding segments. Whereas forwarding segments in SR are global 3014 or local, BPs in BIER-TE have a scope that is the group of BFR(s) 3015 that have adjacencies for this BP in their BIFT. This can be called 3016 "adjacency" scoped forwarding segments. 3018 Adjacency scope could be global, but then every BFR would need an 3019 adjacency for this BP, for example a forward_routed() adjacency with 3020 encapsulation to the global SR SID of the destination. Such a BP 3021 would always result in ingress replication though (as in [RFC7988]). 3022 The first BFR encountering this BP would directly replicate to it. 3023 Only by using non-global adjacency scope for BPs can traffic be 3024 steered and replicated on non-ingress BFR. 3026 SR can naturally be combined with BIER-TE and help to optimize it. 3027 For example, instead of defining bit positions for non-replicating 3028 hops, it is equally possible to use segment routing encapsulations 3029 (e.g. SR-MPLS label stacks) for the encapsulation of 3030 "forward_routed" adjacencies. 3032 Note that (non-TE) BIER itself can also be seen to be similar to SR. 3033 BIER BPs act as global destination Node-SIDs and the BIER BitString 3034 is simply a highly optimized mechanism to indicate multiple such SIDs 3035 and let the network take care of effectively replicating the packet 3036 hop-by-hop to each destination Node-SID. What BIER does not allow is 3037 to indicate intermediate hops, or in terms of SR the ability to 3038 indicate a sequence of SID to reach the destination. This is what 3039 BIER-TE and its adjacency scoped BP enables. 3041 Authors' Addresses 3043 Toerless Eckert (editor) 3044 Futurewei Technologies Inc. 3045 2330 Central Expy 3046 Santa Clara, 95050 3047 United States of America 3049 Email: tte+ietf@cs.fau.de 3051 Michael Menth 3052 University of Tuebingen 3054 Email: menth@uni-tuebingen.de 3056 Gregory Cauchie 3057 Bouygues Telecom 3059 Email: GCAUCHIE@bouyguestelecom.fr