< draft-ietf-bier-te-arch-09.txt   draft-ietf-bier-te-arch-10.txt >
Network Working Group T. Eckert, Ed. Network Working Group T. Eckert, Ed.
Internet-Draft Futurewei Internet-Draft Futurewei
Intended status: Standards Track G. Cauchie Intended status: Standards Track G. Cauchie
Expires: May 3, 2021 Bouygues Telecom Expires: January 10, 2022 Bouygues Telecom
M. Menth M. Menth
University of Tuebingen University of Tuebingen
Oct 30, 2020 July 9, 2021
Tree Engineering for Bit Index Explicit Replication (BIER-TE) Tree Engineering for Bit Index Explicit Replication (BIER-TE)
draft-ietf-bier-te-arch-09 draft-ietf-bier-te-arch-10
Abstract Abstract
This memo introduces per-packet stateless strict and loose path This memo describes per-packet stateless strict and loose path
steered replication and forwarding for Bit Index Explicit Replication steered replication and forwarding for Bit Index Explicit Replication
packets (RFC8279). This is called BIER Tree Engineering (BIER-TE). packets (RFC8279). It is called BIER Tree Engineering (BIER-TE) and
BIER-TE can be used as a path steering mechanism in future Traffic is intended to be used as the path steering mechanism for Traffic
Engineering solutions for BIER (BIER-TE). Engineering with BIER.
BIER-TE leverages RFC8279 and extends it with a new semantic for bits
in the bitstring. BIER-TE can leverage BIER forwarding engines with
little or no changes.
In BIER, the BitPositions (BP) of the packets bitstring indicate BIER
Forwarding Egress Routers (BFER), and hop-by-hop forwarding uses a
Routing Underlay such as an IGP.
In BIER-TE, BitPositions indicate adjacencies. The BIFT of each BFR
are only populated with BPs that are adjacent to the BFR in the BIER-
TE topology. The BIER-TE topology can consist of layer 2 or remote
(routed) adjacencies. The BFR then replicates and forwards BIER
packets to those adjacencies. This results in the aforementioned
strict and loose path steering and replications.
BIER-TE can co-exist with BIER forwarding in the same domain, for
example by using separate BIER sub-domains. In the absence of routed
adjacencies, BIER-TE does not require a BIER routing underlay, and
can then be operated without requiring an Interior Gateway Routing
protocol (IGP).
BIER-TE operates without explicit in-network tree-state and carries
the multicast distribution tree in the packet header. It can
therefore be a good fit to support multicast path steering in Segment
Routing (SR) networks.
Name explanation
[RFC-editor: This section to be removed before publication.]
Explanation for name change from BIER-TE to mean "Traffic
Engineering" to BIER-TE "Tree Engineering" in WG last-call (to
benefit IETF/IESG reviewers):
This document started by calling itself BIER-TE, "Traffic BIER-TE introduces a new semantic for bit positions (BP) that
Engineering" as it is a mode of BIER specifically beneficial for indicate adjacencies, as opposed to BIER in which BPs indicate Bit-
Traffic Engineering. It supports per-packet bitstring based policy Forwarding Egress Routers (BFER). BIER-TE can leverage BIER
steering and replication. BIER-TE technology itself does not provide forwarding engines with little changes. Co-existence of BIER and
a complete traffic engineering solution for BIER but would require BIER-TE forwarding in the same domain is possible, for example by
combination with other technologies for a full BIER based TE using separate BIER sub-domains (SDs). Except for the optional
solution, such as a PCE and queuing mechanisms to provide bandwidth routed adjacencies, BIER-TE does not require a BIER routing underlay,
and latency reservations. It is also not the only option to build a and can therefore operate without depending on an Interior Gateway
traffic engineering solution utilizing BIER, for example BIER trees Routing protocol (IGP).
could be steered through IGP metric engineering, such as through
Flex-Topologies. The architecure for Traffic Engineering with either
modes of BIER (BIER-TE/BIER) is intended to be defined in a separate
document, most likely in TEAs WG.
Because the name of such an overall solution is intended to be BIER- As it operates on the same per-packet stateless forwarding
TE, the expansion of BIER-TE was therefore changed to name this BIER principles, BIER-TE can also be a good fit to support multicast path
mode "Tree Engineering", so the overall solution can be distinguished steering in Segment Routing (SR) networks.
better from its tree building/engineering method without having to
change the long time well-established abbreviation BIER-TE.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 10, 2022.
This Internet-Draft will expire on May 3, 2021.
Copyright Notice Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the Copyright (c) 2021 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Basic Examples . . . . . . . . . . . . . . . . . . . . . 5 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 5
1.2. BIER-TE Topology and adjacencies . . . . . . . . . . . . 8 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3. Comparison with BIER . . . . . . . . . . . . . . . . . . 9 2.1. Basic Examples . . . . . . . . . . . . . . . . . . . . . 5
1.4. Requirements Language . . . . . . . . . . . . . . . . . . 9 2.2. BIER-TE Topology and adjacencies . . . . . . . . . . . . 8
2. Components . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.3. Relationship to BIER . . . . . . . . . . . . . . . . . . 9
2.1. The Multicast Flow Overlay . . . . . . . . . . . . . . . 10 2.4. Accelerated/Hardware forwarding comparison . . . . . . . 11
2.2. The BIER-TE Controller . . . . . . . . . . . . . . . . . 10 3. Components . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.1. Assignment of BitPositions to adjacencies of the 3.1. The Multicast Flow Overlay . . . . . . . . . . . . . . . 12
network topology . . . . . . . . . . . . . . . . . . 11 3.2. The BIER-TE Control Plane . . . . . . . . . . . . . . . . 12
2.2.2. Changes in the network topology . . . . . . . . . . . 11 3.2.1. The BIER-TE Controller . . . . . . . . . . . . . . . 13
2.2.3. Set up per-multicast flow BIER-TE state . . . . . . . 11 3.2.1.1. BIER-TE Topology discovery and creation . . . . . 14
2.2.4. Link/Node Failures and Recovery . . . . . . . . . . . 12 3.2.1.2. Engineered Trees via BitStrings . . . . . . . . . 14
2.3. The BIER-TE Forwarding Layer . . . . . . . . . . . . . . 12 3.2.1.3. Changes in the network topology . . . . . . . . . 15
2.4. The Routing Underlay . . . . . . . . . . . . . . . . . . 12 3.2.1.4. Link/Node Failures and Recovery . . . . . . . . . 15
2.5. Traffic Engineering Considerations . . . . . . . . . . . 13 3.3. The BIER-TE Forwarding Plane . . . . . . . . . . . . . . 15
3. BIER-TE Forwarding . . . . . . . . . . . . . . . . . . . . . 14 3.4. The Routing Underlay . . . . . . . . . . . . . . . . . . 16
3.1. The Bit Index Forwarding Table (BIFT) . . . . . . . . . . 14 3.5. Traffic Engineering Considerations . . . . . . . . . . . 16
3.2. Adjacency Types . . . . . . . . . . . . . . . . . . . . . 15 4. BIER-TE Forwarding . . . . . . . . . . . . . . . . . . . . . 17
3.2.1. Forward Connected . . . . . . . . . . . . . . . . . . 15 4.1. The Bit Index Forwarding Table (BIFT) . . . . . . . . . . 17
3.2.2. Forward Routed . . . . . . . . . . . . . . . . . . . 16 4.2. Adjacency Types . . . . . . . . . . . . . . . . . . . . . 18
3.2.3. ECMP . . . . . . . . . . . . . . . . . . . . . . . . 16 4.2.1. Forward Connected . . . . . . . . . . . . . . . . . . 18
3.2.4. Local Decap . . . . . . . . . . . . . . . . . . . . . 16 4.2.2. Forward_routed . . . . . . . . . . . . . . . . . . . 19
3.3. Encapsulation considerations . . . . . . . . . . . . . . 17 4.2.3. ECMP . . . . . . . . . . . . . . . . . . . . . . . . 19
3.4. Basic BIER-TE Forwarding Example . . . . . . . . . . . . 17 4.2.4. Local Decap(sulation) . . . . . . . . . . . . . . . . 19
3.5. Forwarding comparison with BIER . . . . . . . . . . . . . 19 4.3. Encapsulation / Co-existence with BIER . . . . . . . . . 20
3.6. Requirements . . . . . . . . . . . . . . . . . . . . . . 20 4.4. BIER-TE Forwarding Pseudocode . . . . . . . . . . . . . . 21
4. BIER-TE Controller BitPosition Assignments . . . . . . . . . 20 4.5. Basic BIER-TE Forwarding Example . . . . . . . . . . . . 24
4.1. P2P Links . . . . . . . . . . . . . . . . . . . . . . . . 21 4.6. BFR Requirements for BIER-TE forwarding . . . . . . . . . 26
4.2. BFER . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.3. Leaf BFERs . . . . . . . . . . . . . . . . . . . . . . . 21
4.4. LANs . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.5. Hub and Spoke . . . . . . . . . . . . . . . . . . . . . . 22
4.6. Rings . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.7. Equal Cost MultiPath (ECMP) . . . . . . . . . . . . . . . 24
4.8. Routed adjacencies . . . . . . . . . . . . . . . . . . . 26
4.8.1. Reducing BitPositions . . . . . . . . . . . . . . . . 26
4.8.2. Supporting nodes without BIER-TE . . . . . . . . . . 27
4.9. Reuse of BitPositions (without DNR) . . . . . . . . . . . 27
4.10. Summary of BP optimizations . . . . . . . . . . . . . . . 28
5. Avoiding duplicates and loops . . . . . . . . . . . . . . . . 29
5.1. Loops . . . . . . . . . . . . . . . . . . . . . . . . . . 29
5.2. Duplicates . . . . . . . . . . . . . . . . . . . . . . . 30
6. BIER-TE Forwarding Pseudocode . . . . . . . . . . . . . . . . 30
7. Managing SI, subdomains and BFR-ids . . . . . . . . . . . . . 33
7.1. Why SI and sub-domains . . . . . . . . . . . . . . . . . 34
7.2. Bit assignment comparison BIER and BIER-TE . . . . . . . 35
7.3. Using BFR-id with BIER-TE . . . . . . . . . . . . . . . . 35
7.4. Assigning BFR-ids for BIER-TE . . . . . . . . . . . . . . 36
7.5. Example bit allocations . . . . . . . . . . . . . . . . . 37
7.5.1. With BIER . . . . . . . . . . . . . . . . . . . . . . 37
7.5.2. With BIER-TE . . . . . . . . . . . . . . . . . . . . 38
7.6. Summary . . . . . . . . . . . . . . . . . . . . . . . . . 39
8. BIER-TE and Segment Routing . . . . . . . . . . . . . . . . . 39
9. Security Considerations . . . . . . . . . . . . . . . . . . . 40
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 41
11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 41
12. Change log [RFC Editor: Please remove] . . . . . . . . . . . 41
13. References . . . . . . . . . . . . . . . . . . . . . . . . . 47
13.1. Normative References . . . . . . . . . . . . . . . . . . 47
13.2. Informative References . . . . . . . . . . . . . . . . . 47
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 48
1. Introduction 5. BIER-TE Controller Operational Considerations . . . . . . . . 26
5.1. Bit position Assignments . . . . . . . . . . . . . . . . 26
5.1.1. P2P Links . . . . . . . . . . . . . . . . . . . . . . 27
5.1.2. BFER . . . . . . . . . . . . . . . . . . . . . . . . 27
5.1.3. Leaf BFERs . . . . . . . . . . . . . . . . . . . . . 27
5.1.4. LANs . . . . . . . . . . . . . . . . . . . . . . . . 28
5.1.5. Hub and Spoke . . . . . . . . . . . . . . . . . . . . 28
5.1.6. Rings . . . . . . . . . . . . . . . . . . . . . . . . 29
5.1.7. Equal Cost MultiPath (ECMP) . . . . . . . . . . . . . 30
5.1.8. Forward_routed adjacencies . . . . . . . . . . . . . 32
5.1.8.1. Reducing bit positions . . . . . . . . . . . . . 32
5.1.8.2. Supporting nodes without BIER-TE . . . . . . . . 33
5.1.9. Reuse of bit positions (without DNC) . . . . . . . . 33
5.1.10. Summary of BP optimizations . . . . . . . . . . . . . 35
5.2. Avoiding duplicates and loops . . . . . . . . . . . . . . 36
5.2.1. Loops . . . . . . . . . . . . . . . . . . . . . . . . 36
5.2.2. Duplicates . . . . . . . . . . . . . . . . . . . . . 36
5.3. Managing SI, sub-domains and BFR-ids . . . . . . . . . . 37
5.3.1. Why SI and sub-domains . . . . . . . . . . . . . . . 37
5.3.2. Assigning bits for the BIER-TE topology . . . . . . . 38
5.3.3. Assigning BFR-id with BIER-TE . . . . . . . . . . . . 39
5.3.4. Mapping from BFR to BitStrings with BIER-TE . . . . . 40
5.3.5. Assigning BFR-ids for BIER-TE . . . . . . . . . . . . 41
5.3.6. Example bit allocations . . . . . . . . . . . . . . . 41
5.3.6.1. With BIER . . . . . . . . . . . . . . . . . . . . 41
5.3.6.2. With BIER-TE . . . . . . . . . . . . . . . . . . 42
5.3.7. Summary . . . . . . . . . . . . . . . . . . . . . . . 43
6. BIER-TE and Segment Routing . . . . . . . . . . . . . . . . . 44
7. Security Considerations . . . . . . . . . . . . . . . . . . . 45
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 46
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 46
10. Change log [RFC Editor: Please remove] . . . . . . . . . . . 46
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 56
11.1. Normative References . . . . . . . . . . . . . . . . . . 56
11.2. Informative References . . . . . . . . . . . . . . . . . 56
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 58
BIER-TE shares architecture, terminology and packet formats with BIER 1. Overview
as described in [RFC8279] and [RFC8296]. This document describes
BIER-TE in the expectation that the reader is familiar with these two
documents.
In BIER-TE, BitPositions (BP) indicate adjacencies. The BIFT of each BIER-TE is based on architecture, terminology and packet formats with
BFR is only populated with BP that are adjacent to the BFR in the BIER as described in [RFC8279] and [RFC8296]. This document
BIER-TE Topology. Other BPs are left without adjacency. The BFR describes BIER-TE in the expectation that the reader is familiar with
replicate and forwards BIER packets to adjacent BPs that are set in these two documents.
the packet. BPs are normally also reset upon forwarding to avoid
duplicates and loops. This is detailed further below. BIER-TE introduces a new semantic for bit positions (BP) that
indicate adjacencies, as opposed to BIER in which BPs indicate Bit-
Forwarding Egress Routers (BFER). With BIER-TE, the BIFT of each BFR
is only populated with BP that are adjacent to the BFR in the BIER-TE
Topology. Other BPs are empty in the BIFT. The BFR replicate and
forwards BIER packets to adjacent BPs that are set in the packet.
BPs are normally also cleared upon forwarding to avoid duplicates and
loops. This is detailed further below.
BIER-TE can leverage BIER forwarding engines with little or no
changes. It can also co-exist with BIER forwarding in the same
domain, for example by using separate BIER sub-domains. Except for
the optional routed adjacencies, BIER-TE does not require a BIER
routing underlay, and can therefore operate without depending on an
Interior Gateway Routing protocol (IGP).
As it operates on the same per-packet stateless forwarding
principles, BIER-TE can also be a good fit to support multicast path
steering in Segment Routing (SR) networks.
This document is structured as follows:
o Section 2 introduces BIER-TE with two reference forwarding
examples, followed by an introduction of the new concepts of the
BIER-TE (overlay) topology and finally a summary of the
relationship between BIER and BIER-TE and a discussion of
accelerated hardware forwarding.
o Section 3 describes the components of the BIER-TE architecture,
Flow overlay, BIER-TE layer with the BIER-TE control plane
(including the BIER-TE controller) and BIER-TE forwarding plane,
and the routing underlay.
o Section 4 specifies the behavior of the BIER-TE forwarding plane
with the different type of adjacencies and possible variations of
BIER-TE forwarding pseudocode, and finally the mandatory and
optional requirements.
o Section 5 describes operational considerations for the BIER-TE
controller, foremost how the BIER-TE controller can optimize the
use of BP by using specific type of BIER-TE adjacencies for
different type of topological situations, but also how to assign
bits to avoid loops and duplicates (which in BIER-TE does not come
for free), and finally how SI, sub-domains and BFR-ids can be
managed by a BIER-TE controller, examples and summary.
o Section 6 concludes the technology specific sections of document
by further relating BIER-TE to Segment Routing (SR).
Note that related work, [I-D.ietf-roll-ccast] uses Bloom filters Note that related work, [I-D.ietf-roll-ccast] uses Bloom filters
[Bloom70] to represent leaves or edges of the intended delivery tree. [Bloom70] to represent leaves or edges of the intended delivery tree.
Bloom filters in general can support larger trees/topologies with Bloom filters in general can support larger trees/topologies with
fewer addressing bits than explicit bitstrings, but they introduce fewer addressing bits than explicit BitStrings, but they introduce
the heuristic risk of false positives and cannot reset bits in the the heuristic risk of false positives and cannot clear bits in the
bitstring during forwarding to avoid loops. For these reasons, BIER- BitString during forwarding to avoid loops. For these reasons, BIER-
TE uses explicit bitstrings like BIER. The explicit bitstrings of TE uses explicit BitStrings like BIER. The explicit BitStrings of
BIER-TE can also be seen as a special type of Bloom filter, and this BIER-TE can also be seen as a special type of Bloom filter, and this
is how related work [ICC] describes it. is how related work [ICC] describes it.
1.1. Basic Examples 1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119], [RFC8174] when, and only when, they appear in all
capitals, as shown here.
2. Introduction
2.1. Basic Examples
BIER-TE forwarding is best introduced with simple examples. BIER-TE forwarding is best introduced with simple examples.
BIER-TE Topology: BIER-TE Topology:
Diagram: Diagram:
p5 p6 p5 p6
--- BFR3 --- --- BFR3 ---
p3/ p13 \p7 p3/ p13 \p7 p15
BFR1 ---- BFR2 BFR5 ----- BFR6 BFR1 ---- BFR2 BFR5 ----- BFR6
p1 p2 p4\ p14 /p10 p11 p12 p1 p2 p4\ p14 /p10 p11 p12
--- BFR4 --- --- BFR4 ---
p8 p9 p8 p9
(simplified) BIER-TE Bit Index Forwarding Tables (BIFT): (simplified) BIER-TE Bit Index Forwarding Tables (BIFT):
BFR1: p1 -> local_decap BFR1: p1 -> local_decap
p2 -> forward_connected to BFR2 p2 -> forward_connected() to BFR2
BFR2: p1 -> forward_connected to BFR1 BFR2: p1 -> forward_connected() to BFR1
p5 -> forward_connected to BFR3 p5 -> forward_connected() to BFR3
p8 -> forward_connected to BFR4 p8 -> forward_connected() to BFR4
BFR3: p3 -> forward_connected to BFR2 BFR3: p3 -> forward_connected() to BFR2
p7 -> forward_connected to BFR5 p7 -> forward_connected() to BFR5
p13 -> local_decap p13 -> local_decap
BFR4: p4 -> forward_connected to BFR2 BFR4: p4 -> forward_connected() to BFR2
p10 -> forward_connected to BFR5 p10 -> forward_connected() to BFR5
p14 -> local_decap p14 -> local_decap
BFR5: p6 -> forward_connected to BFR3 BFR5: p6 -> forward_connected() to BFR3
p9 -> forward_connected to BFR4 p9 -> forward_connected() to BFR4
p12 -> forward_connected to BFR6 p12 -> forward_connected() to BFR6
BFR6: p11 -> forward_connected to BFR5 BFR6: p11 -> forward_connected() to BFR5
p12 -> local_decap p15 -> local_decap
Figure 1: BIER-TE basic example Figure 1: BIER-TE basic example
Consider the simple network in the above BIER-TE overview example Consider the simple network in the above BIER-TE overview example
picture with 6 BFRs. p1...p14 are the BitPositions (BP) used. All picture with 6 BFRs. p1...p14 are the bit positions (BP) used. All
BFRs can act as ingress BFR (BFIR), BFR1, BFR3, BFR4 and BFR6 can BFRs can act as an ingress BFR (BFIR), BFR1, BFR3, BFR4 and BFR6 can
also be egress BFR (BFER). Forward_connected is the name for also be egress BFRs (BFERs). Forward_connected() is the name for
adjacencies that are representing subnet adjacencies of the network. adjacencies that are representing subnet adjacencies of the network.
Local_decap is the name of the adjacency to decapsulate BIER-TE Local_decap() is the name of the adjacency to decapsulate BIER-TE
packets and pass their payload to higher layer processing. packets and pass their payload to higher layer processing.
Assume a packet from BFR1 should be sent via BFR4 to BFR6. This Assume a packet from BFR1 should be sent via BFR4 to BFR6. This
requires a bitstring (p2,p8,p10,p12). When this packet is examined requires a BitString (p2,p8,p10,p12,p15). When this packet is
by BIER-TE on BFR1, the only BitPosition from the bitstring that is examined by BIER-TE on BFR1, the only bit position from the BitString
also set in the BIFT is p2. This will cause BFR1 to send the only that is also set in the BIFT is p2. This will cause BFR1 to send the
copy of the packet to BFR2. Similarly, BFR2 will forward to BFR4 only copy of the packet to BFR2. Similarly, BFR2 will forward to
because of p8, BFR4 to BFR5 because of p10 and BFR5 to BFR6 because BFR4 because of p8, BFR4 to BFR5 because of p10 and BFR5 to BFR6
of p12. p12 also makes BFR6 receive and decapsulate the packet. because of p12. p15 finally makes BFR6 receive and decapsulate the
packet.
To send in addition to BFR6 via BFR4 also a copy to BFR3, the To send in addition to BFR6 via BFR4 also a copy to BFR3, the
bitstring needs to be (p2,p5,p8,p10,p12,p13). When this packet is BitString needs to be (p2,p5,p8,p10,p12,p13). When this packet is
examined by BFR2, p5 causes one copy to be sent to BFR3 and p8 one examined by BFR2, p5 causes one copy to be sent to BFR3 and p8 one
copy to BFR4. When BFR3 receives the packet, p13 will cause it to copy to BFR4. When BFR3 receives the packet, p13 will cause it to
receive and decapsulate the packet. receive and decapsulate the packet.
If instead the bitstring was (p2,p6,p8,p10,p12,p13), the packet would If instead the BitString was (p2,p6,p8,p10,p12,p13,p15), the packet
be copied by BFR5 towards BFR3 because of p6 instead of being copied would be copied by BFR5 towards BFR3 because of p6 instead of being
by BFR2 to BFR3 because of p5 in the prior case. This is showing the copied by BFR2 to BFR3 because of p5 in the prior case. This is
ability of the shown BIER-TE Topology to make the traffic pass across showing the ability of the shown BIER-TE Topology to make the traffic
any possible path and be replicated where desired. pass across any possible path and be replicated where desired.
BIER-TE has various options to minimize BP assignments, many of which BIER-TE has various options to minimize BP assignments, many of which
are based on assumptions about the required multicast traffic paths are based on assumptions about the required multicast traffic paths
and bandwidth consumption in the network. and bandwidth consumption in the network.
The following picture shows a modified example, in which Rtr2 and The following picture shows a modified example, in which Rtr2 and
Rtr5 are assumed not to support BIER-TE, so traffic has to be unicast Rtr5 are assumed not to support BIER-TE, so traffic has to be unicast
encapsulated across them. Unicast tunneling of BIER-TE packets can encapsulated across them. To emphasize non-L2, but routed/tunneled
leverage any feasible mechanism such as MPLS or IP, these forwarding of BIER-TE packets, these adjacencies are called
encapsulations are out of scope of this document. To emphasize non- "forward_routed". Otherwise there is no difference in their
native forwarding of BIER-TE packets, these adjacencies are called
"forward_routed", but otherwise there is no difference in their
processing over the aforementioned "forward_connected" adjacencies. processing over the aforementioned "forward_connected" adjacencies.
In addition, bits are saved in the following example by assuming that In addition, bits are saved in the following example by assuming that
BFR1 only needs to be BFIR but not BFER or transit BFR. BFR1 only needs to be BFIR but not BFER or transit BFR.
BIER-TE Topology: BIER-TE Topology:
Diagram: Diagram:
p1 p3 p7 p1 p3 p7
....> BFR3 <.... p5 ....> BFR3 <.... p5
........ ........> ........ ........>
BFR1 (Rtr2) (Rtr5) BFR6 BFR1 (Rtr2) (Rtr5) BFR6
........ ........> ........ ........>
....> BFR4 <.... p6 ....> BFR4 <.... p6
p2 p4 p8 p2 p4 p8
(simplified) BIER-TE Bit Index Forwarding Tables (BIFT): (simplified) BIER-TE Bit Index Forwarding Tables (BIFT):
BFR1: p1 -> forward_routed to BFR3 BFR1: p1 -> forward_routed() to BFR3
p2 -> forward_routed to BFR4 p2 -> forward_routed() to BFR4
BFR3: p3 -> local_decap BFR3: p3 -> local_decap
p5 -> forward_routed to BFR6 p5 -> forward_routed() to BFR6
BFR4: p4 -> local_decap BFR4: p4 -> local_decap
p6 -> forward_routed to BFR6 p6 -> forward_routed() to BFR6
BFR6: p5 -> local_decap BFR6: p5 -> local_decap
p6 -> local_decap p6 -> local_decap
p7 -> forward_routed to BFR3 p7 -> forward_routed() to BFR3
p8 -> forward_routed to BFR4 p8 -> forward_routed() to BFR4
Figure 2: BIER-TE basic overlay example Figure 2: BIER-TE basic overlay example
To send a BIER-TE packet from BFR1 via BFR3 to BFR6, the bitstring is To send a BIER-TE packet from BFR1 via BFR3 to BFR6, the BitString is
(p1,p5). From BFR1 via BFR4 to BFR6 it is (p2,p6). A packet from (p1,p5). From BFR1 via BFR4 to BFR6 it is (p2,p6). A packet from
BFR1 to BFR3,BFR4 and from BFR3 to BFR6 uses (p1,p2,p3,p4,p5). A BFR1 to BFR3,BFR4 and from BFR3 to BFR6 uses (p1,p2,p3,p4,p5). A
packet from BFR1 to BFR3,BFR4 and from BFR4 to BFR uses packet from BFR1 to BFR3,BFR4 and from BFR4 to BFR uses
(p1,p2,p3,p4,p6). A packet from BFR1 to BFR4, and from BFR4 to BFR6 (p1,p2,p3,p4,p6). A packet from BFR1 to BFR4, and from BFR4 to BFR6
and from BFR6 to BFR3 uses (p2,p3,p4,p6,p7). A packet from BFR1 to and from BFR6 to BFR3 uses (p2,p3,p4,p6,p7). A packet from BFR1 to
BFR3, and from BFR3 to BFR6 and from BFR6 to BFR4 uses BFR3, and from BFR3 to BFR6 and from BFR6 to BFR4 uses
(p1,p3,p4,p5,p8). (p1,p3,p4,p5,p8).
1.2. BIER-TE Topology and adjacencies 2.2. BIER-TE Topology and adjacencies
The key new component in BIER-TE compared to BIER is the BIER-TE The key new component in BIER-TE compared to BIER is the BIER-TE
topology as introduced through the two examples in Section 1.1. It topology as introduced through the two examples in Section 2.1. It
is used to control where replication can or should happen and how to is used to control where replication can or should happen and how to
minimize the required number of BP for adjacencies. minimize the required number of BP for adjacencies.
The BIER-TE Topology consists of the BIFT of all the BFR and can also The BIER-TE Topology consists of the BIFTs of all the BFR and can
be expressed as a directed graph where the edges are the adjacencies also be expressed as a directed graph where the edges are the
between the BFR labelled with the BP used for the adjacency. adjacencies between the BFR labelled with the BP used for the
Adjacencies are naturally unidirectional. BP can be reused across adjacency. Adjacencies are naturally unidirectional. BP can be
multiple adjacencies as long as this does not lead to undesired reused across multiple adjacencies as long as this does not lead to
duplicates or loops as explained further down in the text. undesired duplicates or loops as explained further down in the text.
If the BIER-TE topology represents the underlying (layer 2) topology If the BIER-TE topology represents (a subset of) the underlying
of the network, this is called "native" BIER-TE as shown in the first (layer 2) topology of the network as shown in the first example, this
example. This can be freely mixed with "overlay" BIER-TE, in may be called a "native" BIER-TE topology. A topology consisting
"forward_routed" adjacencies are used. only of "forward_routed" adjacencies as shown in the second example
may be called an "overlay" BIER-TE topology. A BIER-TE topology will
both "forward_connected" and "forward_routed" adjacencies may be
called a "hybrid" BIER-TE topology.
1.3. Comparison with BIER 2.3. Relationship to BIER
The key differences over BIER are: BIER-TE is designed so that is forwarding plane is a simple extension
to the BIER forwarding plane, hence allowing for it to be added to
BIER deployments where it can be beneficial.
o BIER-TE replaces in-network autonomous path calculation by BIER-TE is also intended as an option to expand the BIER architecture
explicit paths calculated by the BIER-TE Controller. into deployments where BIER may not be the best fit, such as
statically provisioned networks with needs for path steering but
without desire for distributed routing protocols.
o In BIER-TE every BitPosition of the BitString of a BIER-TE packet 1. BIER-TE inherits the following aspects from BIER unchanged:
indicates one or more adjacencies - instead of a BFER as in BIER.
o BIER-TE in each BFR has no routing table but only a BIER-TE 1. The fundamental purpose of per-packet signaled packet
Forwarding Table (BIFT) indexed by SI:BitPosition and populated replication and delivery via a BitString.
with only those adjacencies to which the BFR should replicate
packets to.
BIER-TE headers use the same format as BIER headers. 2. The overall architecture consisting of three layers, flow
overlay, BIER(-TE) layer and routing underlay.
BIER-TE forwarding does not require/use the BFIR-ID. The BFIR-ID can 3. The supportable encapsulations, [RFC8296] or other (future)
still be useful though for coordinated BFIR/BFER functions, such as encapsulations.
the context for upstream assigned labels for MPLS payloads in MVPN
over BIER-TE.
If the BIER-TE domain is also running BIER, then the BFIR-ID in BIER- 4. The semantic of all [RFC8296] header elements used by the
TE packets can be set to the same BFIR-ID as used with BIER packets. BIER-TE forwarding plane other than the semantic of the BP in
the BitString.
If the BIER-TE domain is not running full BIER or does not want to 5. The BIER forwarding plane, with the exception of how bits
reduce the need to allocate bits in BIER bitstrings for BFIR-ID have to be cleared during replication.
values, then the allocation of BFIR-ID values in BIER-TE packets can
be done through other mechanisms outside the scope of this document,
as long as this is appropriately agreed upon between all BFIR/BFER.
1.4. Requirements Language 2. BIER-TE has the following key changes with respect to BIER:
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 1. In BIER, bits in the BitString of a BIER packet header
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this indicate a BFER and bits in the BIFT indicate the BIER
document are to be interpreted as described in RFC 2119 [RFC2119]. control plane calculated next-hop toward that BFER. In BIER-
TE, bits in the BitString of a BIER packet header indicate an
adjacency in the BIER-TE topology, and only the BFRs that are
upstream of this adjacency have this bit populated with the
adjacency in their BIFT.
2. Components 2. In BIER, the implied reference option for the core part of
the BIER layer control plane is the BIER extension to
distributed routing protocol, such as standardized in ISIS/
OSPF extensions for BIER, [RFC8401] and [RFC8444]. The
reference option for the core part of the BIER-TE control
plane is the BIER-TE controller. Nevertheless, both BIER and
BIER-TE BIFT forwarding plane state could equally be
populated by any mechanism.
End to end BIER-TE operations consists of four mayor components: The 3. Assuming the reference options for the control plane, BIER-TE
"Multicast Flow Overlay", the "BIER-TE control plane" consisting of replaces in-network autonomous path calculation by explicit
the "BIER-TE Controller" and its signaling channels to the BFR, the paths calculated by the BIER-TE controller.
"Routing Underlay" and the "BIER-TE forwarding layer". The Bier-TE
Controller is the new architectural component in BIER-TE compared to
BIER.
Picture 2: Components of BIER-TE 3. The following element/functions described in the BIER
architecture are not required by the BIER-TE architecture:
1. BIRTs on BFR for BIER-TE are not required when using a BIER-
TE controller because the controller can directly populate
the BIFTs. In BIER, BIRTs are populated by the distributed
routing protocol support for BIER, allowing BFR to populate
their BIFTs locally from their BIRTs. Other BIER-TE control
plane or management plane options may introduce requirements
for BIRTs for BIER-TE BFR.
2. The BIER-TE layer forwarding plane does not require BFR to
have a unique BP and therefore also no unique BFR-id. See
for example See Section 5.1.3.
3. Identification of BFR by the BIER-TE control plane is outside
the scope of this specification. Whereas the BIER control
plane uses BFR-ids in its BFR to BFR signaling, a BIER-TE
controller may choose any form of identification deemed
appropriate.
4. BIER-TE forwarding does not use the BFR-id field of the BIER
packet header.
4. Co-existence of BIER and BIER-TE in the same network requires the
following:
1. The BIER/BIER-TE packet header needs to allow addressing both
BIER and BIER-TE BIFT. Depending on the encapsulation
option, the same SD may or may not be reusable across BIER
and BIER-TE. See Section 4.3. In either case, a packet is
always only forwarded end-to-end via BIER or via BIER-TE
(ships in the nights forwarding).
2. BIER-TE deployments will have to assign BFR-ids to BFR and
insert them into the BFR-id field of BIER packet headers as
BIER does, whenever the deployment uses (unchanged)
components developed for BIER that use BFR-id, such as
multicast flow overlays or BIER layer control plane elements.
See also Section 5.3.3.
2.4. Accelerated/Hardware forwarding comparison
Forwarding of BIER-TE is designed to easily build/program common
forwarding hardware with BIER. The pseudocode in Section 4.4 shows
how existing BIER/BIFT forwarding can be modified to support the
REQUIRED BIER-TE forwarding functionality, by using BIER BIFT's
"Forwarding Bit Mask" (F-BM): Only the clearing of bits to avoid
duplicate packets to a BFR neighbor is skipped in BIER-TE forwarding
because it is not necessary and could not be done when using BIER
F-BM.
Whether to use BIER or BIER-TE forwarding is simply a choice of the
mode of the BIFT indicated by the packet (BIER or BIER-TE BIFT).
This is determined by the BFR configuration for the encapsulation,
see Section 4.3.
3. Components
BIER-TE can be thought of being constituted from the same three
layers as BIER: The "multicast flow overlay", the "BIER layer" and
the "routing underlay". The following picture also shows how the
"BIER layer" is constituted from the "BIER-TE forwarding plane" and
the "BIER-TE control plane" represent by the "BIER-TE Controller".
<------BGP/PIM-----> <------BGP/PIM----->
|<-IGMP/PIM-> multicast flow <-PIM/IGMP->| |<-IGMP/PIM-> multicast flow <-PIM/IGMP->|
overlay overlay
[BIER-TE Controller] <=> [BIER-TE Topology] BIER-TE [BIER-TE Controller] <=> [BIER-TE Topology]
BIER-TE control plane control ^ ^ ^
^ ^ ^ plane / | \ BIER-TE control protocol
/ | \ BIER-TE control protocol | | | e.g. YANG/Netconf/RestConf
| | | e.g. Netconf/Restconf/Yang | | | PCEP/...
v v v v v v
Src -> Rtr1 -> BFIR-----BFR-----BFER -> Rtr2 -> Rcvr Src -> Rtr1 -> BFIR-----BFR-----BFER -> Rtr2 -> Rcvr
|<----------------->| |<----------------->|
BIER-TE forwarding layer BIER-TE forwarding plane
|<- BIER-TE domain->| |<- BIER-TE domain->|
|<--------------------->| |<--------------------->|
Routing underlay Routing underlay
Figure 3: BIER-TE architecture Figure 3: BIER-TE architecture
2.1. The Multicast Flow Overlay 3.1. The Multicast Flow Overlay
The Multicast Flow Overlay operates as in BIER. See [RFC8279]. The Multicast Flow Overlay has the same role as described for BIER in
Instead of interacting with the BIER forwarding layer (as in BIER), [RFC8279], Section 4.3. See also Section 3.2.1.2.
it interacts with the BIER-TE Controller.
2.2. The BIER-TE Controller 3.2. The BIER-TE Control Plane
The BIER-TE Controller is representing the control plane of BIER-TE. In the BIER architecture [RFC8279], the BIER control plane is not
It communicates two sets of information with BFRs: explicitly separated from the BIER forwarding plane, but instead
their functions are summarized together in Section 4.2. Example
standardized options for the BIER control plane include ISIS/OSPF
extensions for BIER, [RFC8401] and [RFC8444].
During initial provisioning or modifications of the network topology, For BIER-TE, the control plane includes at minimum the following
the BIER-TE Controller discovers the network topology and creates the functionality.
BIER-TE topology from it: determine which adjacencies are required/
desired and assign BitPositions to them. Then it signals the
resulting of BitPositions and their adjacencies to each BFR to set up
their BIER-TE BIFTs.
During day-to-day operations of the network, the BIER-TE Controller 1. During initial provisioning of the network and/or during
signals to BFIRs what multicast flows are mapped to what BitStrings. modifications of its topology and/or services: protocols and/or
procedures to establish BIER-TE BIFTs:
Communications between the BIER-TE Controller and BFRs is ideally via 1. Determine the desired BIER-TE topology for a BIER-TE sub-
standardized protocols and data-models such as Netconf/Restconf/Yang. domains: the native and/or overlay adjacencies that are
This is currently outside the scope of this document. Vendor- assigned to BPs.
specific CLI on the BFRs is also a possible stopgap option (as in
many other SDN solutions lacking definition of standardized data
model).
For simplicity, the procedures of the BIER-TE Controller are 2. Determine the per-BFR BIFT from the BIER-TE topology.
described in this document as if it is a single, centralized
automated entity, such as an SDN controller. It could equally be an
operator setting up CLI on the BFRs. Distribution of the functions
of the BIER-TE Controller is currently outside the scope of this
document.
2.2.1. Assignment of BitPositions to adjacencies of the network 3. Optionally assign BFR-id to BFIR for later insertion into
topology BIER-TE headers on BFIR. Alternatively, bfir-id in BIER
packet headers may be managed solely by the flow overlay
layer and/or be unused.
The BIER-TE Controller tracks the BFR topology of the BIER-TE domain. 4. Install/update the BIFTs into the BFRs and optionally BFR-id
It determines what adjacencies require BitPositions so that BIER-TE into BFIR.
explicit paths can be built through them as desired by operator
policy.
The BIER-TE Controller then pushes the BitPositions/adjacencies to 2. During operations of the network: Protocols and/or procedures to
the BIFT of the BFRs, populating only those SI:BitPositions to the support creation/change/removal of overlay flows on BFIR:
BIFT of each BFR to which that BFR should be able to send packets to
- adjacencies connecting to this BFR.
2.2.2. Changes in the network topology 1. Process the BIER-TE requirements for the multicast overlay
flow: BFIR and BFERs of the flow as well as policies for the
path selection of the flow.
If the network topology changes (not failure based) so that 2. Determine the BitStrings and optionally Entropy.
adjacencies that are assigned to BitPositions are no longer needed,
the BIER-TE Controller can re-use those BitPositions for new
adjacencies. First, these BitPositions need to be removed from any
BFIR flow state and BFR BIFT state, then they can be repopulated,
first into BIFT and then into the BFIR.
2.2.3. Set up per-multicast flow BIER-TE state 3. Install state on the BFIR to imposition the desired BIER
packet header(s) for packets of the overlay flow.
The BIER-TE Controller interacts with the multicast flow overlay to 4. Install the necessary state on the BFERs to decapsulate the
determine what multicast flow needs to be sent by a BFIR to which set BIER packet header and properly dispatch its payload.
of BFER. It calculates the desired distribution tree across the
BIER-TE domain based on algorithms outside the scope of this document
(e.g. CSFP, Steiner Tree, ...). It then pushes the calculated
BitString into the BFIR.
See [I-D.ietf-bier-multicast-http-response] for a solution describing 3.2.1. The BIER-TE Controller
this interaction.
2.2.4. Link/Node Failures and Recovery Notwithstanding other options, this architecture describes the BIER
control plane as shown in Figure 3 to consists of:
When link or nodes fail or recover in the topology, BIER-TE can o A single centralized BIER-TE controller.
quickly respond with the optional FRR procedures described in [I-
D.eckert-bier-te-frr]. It can also more slowly react by o Data-models and protocols to communicate between controller and
BFR in step 1, such YANG/Netconf/RestConf.
o Protocols to communicate between controller and BFIR in step 2,
such as BIER-TE extensions for [RFC5440].
The BIER control plane could equally be implemented without any
active dynamic components by an operator via CLI on the BFRs. In
that case, operator configured local policy on the BFIR would have to
determine how to set the appropriate BIER header fields. The BIER-TE
control plane could also be decentralized and/or distributed, but
this document does not consider any additional protocols and/or
procedures which would then be necessary to coordinate its entities
to achieve the above described functionality.
3.2.1.1. BIER-TE Topology discovery and creation
Step 1.1 includes network topology discovery and BIER-TE topology
creation. The latter describes the process by which a Controller
determines which routers are to be configured as BFR and the
adjacencies between them.
In statically managed networks, such as in industrial environments,
both discovery and creation can be a manual/offline process.
In other networks, topology discovery may rely on protocols including
extending a Link-State-Protocol (LSP) based IGP into the BIER-TE
controller itself, [RFC7752] (BGP-LS) or [RFC8345] (Yang topology) as
well as BIER-TE specific methods, for example via
[I-D.ietf-bier-te-yang]. These options are non-exhaustive.
Dynamic creation of the BIER-TE topology can be as easy as mapping
the network topology 1:1 to the BIER-TE topology by assigning a BP
for every network subnet adjacency. In larger networks, it likely
involves more complex policy and optimization decisions including how
to minimize the number of BP required and how to assign BP across
different BitStrings to minimize the number of duplicate packets
across links when delivering an overlay flow to BFER using different
SIs/BitStrings. These topics are discussed in Section 5.
When the topology is determined, the BIER-TE Controller then pushes
the BitPositions/adjacencies to the BIFT of the BFRs, populating only
those SI:BitPositions to the BIFT of each BFR to which that BFR
should be able to send packets to - adjacencies connecting to this
BFR.
Communications between the BIER-TE Controller and BFRs (beside
topology discovery) is ideally via standardized protocols and data-
models such as Netconf/RestConf/Yang/PCEP. Vendor-specific CLI on
the BFRs is also an option (as in many other SDN solutions lacking
definition of standardized data model).
3.2.1.2. Engineered Trees via BitStrings
In BIER, the same set of BFER in a single sub-domain is always
encoded as the same BitString. In BIER-TE, the BitString used to
reach the same set of BFER in the same sub-domain can be different
for different overlay flows because the BitString encodes the paths
towards the BFER, so the BitStrings from different BFIR to the same
set of BFER will often be different, and the BitString from the same
BFIR to the same set of BFER can different for different overlay
flows for policy reasons such as shortest path trees, Steiner trees
(minimum cost trees), diverse path trees for redundancy and so on.
See also [I-D.ietf-bier-multicast-http-response] for a solution
describing this interaction.
3.2.1.3. Changes in the network topology
If the network topology changes (not failure based) so that
adjacencies that are assigned to bit positions are no longer needed,
the BIER-TE Controller can re-use those bit positions for new
adjacencies. First, these bit positions need to be removed from any
BFIR flow state and BFR BIFT state, then they can be repopulated,
first into BIFT and then into the BFIR.
3.2.1.4. Link/Node Failures and Recovery
When link or nodes fail or recover in the topology, BIER-TE could
quickly respond with out-of-scope FRR procedures such as
[I-D.eckert-bier-te-frr]. It can also more slowly react by
recalculating the BitStrings of affected multicast flows. This recalculating the BitStrings of affected multicast flows. This
reaction is slower than the FRR procedure because the BIER-TE reaction is slower than the FRR procedure because the BIER-TE
Controller needs to receive link/node up/down indications, Controller needs to receive link/node up/down indications,
recalculate the desired BitStrings and push them down into the BFIRs. recalculate the desired BitStrings and push them down into the BFIRs.
With FRR, this is all performed locally on a BFR receiving the With FRR, this is all performed locally on a BFR receiving the
adjacency up/down notification. adjacency up/down notification.
2.3. The BIER-TE Forwarding Layer 3.3. The BIER-TE Forwarding Plane
When the BIER-TE Forwarding Layer receives a packet, it simply looks The BIER-TE Forwarding Plane constitutes of the following components:
up the BitPositions that are set in the BitString of the packet in
1. On BFIR imposition of BIER header for packets from overlay flows.
This is driven by a combination of state established by the BIER-
TE control plane and/or the multicast flow overlay as explained
in Section 3.1.
2. On BFR (including BFIR and BFER), forwarding/replication of BIER
packets according to their BitString as explained below and
optionally Entropy. Processing of other BIER header fields such
as DSCP is outside the scope of this document.
3. On BFER removal of BIER header and dispatching of the payload
according to state created by the BIER-TE control plane and/or
overlay layer.
When the BIER-TE Forwarding Plane receives a packet, it simply looks
up the bit positions that are set in the BitString of the packet in
the Bit Index Forwarding Table (BIFT) that was populated by the BIER- the Bit Index Forwarding Table (BIFT) that was populated by the BIER-
TE Controller. For every BP that is set in the BitString, and that TE Controller. For every BP that is set in the BitString, and that
has one or more adjacencies in the BIFT, a copy is made according to has one or more adjacencies in the BIFT, a copy is made according to
the type of adjacencies for that BP in the BIFT. Before sending any the type of adjacencies for that BP in the BIFT. Before sending any
copy, the BFR resets all BP in the BitString of the packet for which copy, the BFR clears all BPs in the BitString of the packet for which
the BFR has one or more adjacencies in the BIFT, except when the the BFR has one or more adjacencies in the BIFT, except when the
adjacency indicates "DoNotReset" (DNR, see Section 3.2.1). This is adjacency indicates "DoNotClear" (DNC, see Section 4.2.1). This is
done to inhibit that packets can loop. done to inhibit that packets can loop.
2.4. The Routing Underlay 3.4. The Routing Underlay
For forward_connected adjacencies, BIER-TE is sending BIER packets to For forward_connected() adjacencies, BIER-TE is sending BIER packets
directly connected BIER-TE neighbors as L2 (unicasted) BIER packets to directly connected BIER-TE neighbors as L2 (unicasted) BIER
without requiring a routing underlay. For forward_routed packets without requiring a routing underlay. For forward_routed()
adjacencies, BIER-TE forwarding encapsulates a copy of the BIER adjacencies, BIER-TE forwarding encapsulates a copy of the BIER
packet so that it can be delivered by the forwarding plane of the packet so that it can be delivered by the forwarding plane of the
routing underlay to the routable destination address indicated in the routing underlay to the routable destination address indicated in the
adjacency. See Section 3.2.2 for the adjacency definition. adjacency. See Section 4.2.2 for the adjacency definition.
BIER relies on the routing underlay to calculate paths towards BFER BIER relies on the routing underlay to calculate paths towards BFERs
and derive next-hop BFR adjacencies for those paths. This commonly and derive next-hop BFR adjacencies for those paths. This commonly
relies on BIER specific extensions to the routing protocols of the relies on BIER specific extensions to the routing protocols of the
routing underlay but may also be established by a controller. In routing underlay but may also be established by a controller. In
BIER-TE, the next-hops of a packet are determined by the bitstring BIER-TE, the next-hops of a packet are determined by the BitString
through the BIER-TE Controller established adjacencies on the BFR for through the BIER-TE Controller established adjacencies on the BFR for
the BPs of the bitsring. There is thus no need for BFER specific the BPs of the BitString. There is thus no need for BFER specific
routing underlay extensions to forward BIER packets with BIER-TE routing underlay extensions to forward BIER packets with BIER-TE
semantics. semantics.
BIER encapsulations may have BFER independent extensions in the Encapsulation parameters can be provisioned by the BIER-TE controller
routing underlay, such as the label range for BIER packets in the into the forward_connected() or forward_routed() adjacencies directly
BIER over MPLS encapsulation ([RFC8296]). These BIER specific without relying on a routing underlay.
functions of the routing underlay are equally useable by BIER-TE.
Alternatively, these encapsulation parameters can be provisioned by
the BIER-TE controller into the forward_connected or forward_routed
adjacencies directly without relying on a routing underlay.
If the BFR intends to support FRR for BIER-TE, then the BIER-TE If the BFR intends to support FRR for BIER-TE, then the BIER-TE
forwarding plane needs to receive fast adjacency up/down forwarding plane needs to receive fast adjacency up/down
notifications: Link up/down or neighbor up/down, e.g. from BFD. notifications: Link up/down or neighbor up/down, e.g. from BFD.
Providing these notifications is considered to be part of the routing Providing these notifications is considered to be part of the routing
underlay in this document. underlay in this document.
2.5. Traffic Engineering Considerations 3.5. Traffic Engineering Considerations
Traffic Engineering ([I-D.ietf-teas-rfc3272bis]) provides performance Traffic Engineering ([I-D.ietf-teas-rfc3272bis]) provides performance
optimization of operational IP networks while utilizing network optimization of operational IP networks while utilizing network
resources economically and reliably. The key elements needed to resources economically and reliably. The key elements needed to
effect TE are policy, path steering and resource management. These effect TE are policy, path steering and resource management. These
elements require support at the control/controller level and within elements require support at the control/controller level and within
the forwarding plane. the forwarding plane.
Policy decisions are made within the BIER-TE control plane, i.e., Policy decisions are made within the BIER-TE control plane, i.e.,
within BIER-TE Controllers. Controllers use policy when composing within BIER-TE Controllers. Controllers use policy when composing
BitStrings (BFR flow state) and BFR BIFT state. The mapping of user/ BitStrings and BFR BIFT state. The mapping of user/IP traffic to
IP traffic to specific BitStrings/BIER-TE flows is made based on specific BitStrings/BIER-TE flows is made based on policy. The
policy. The specifics details of BIER-TE policies and how a specific details of BIER-TE policies and how a controller uses them
controller uses such are out of scope of this document. are out of scope of this document.
Path steering is supported via the definition of a BitString. Path steering is supported via the definition of a BitString.
BitStrings used in BIER-TE are composed based on policy and resource BitStrings used in BIER-TE are composed based on policy and resource
management considerations. When composing BIER-TE BitStrings, a management considerations. For example, when composing BIER-TE
Controller MUST take into account the resources available at each BFR BitStrings, a Controller must take into account the resources
and for each BP when it is providing congestion loss free services available at each BFR and for each BP when it is providing congestion
such as Rate Controlled Service Disciplines [RCSD94]. Resource loss free services such as Rate Controlled Service Disciplines
availability could be provided for example via routing protocol [RCSD94]. Resource availability could be provided for example via
information, but may also be obtained via a BIER-TE control protocol routing protocol information, but may also be obtained via a BIER-TE
such as Netconf or any other protocol commonly used by a PCE to control protocol such as Netconf or any other protocol commonly used
understand the resources of the network it operates on. The resource by a PCE to understand the resources of the network it operates on.
usage of the BIER-TE traffic admitted by the BIER-TE controller can The resource usage of the BIER-TE traffic admitted by the BIER-TE
be solely tracked on the BIER-TE Controller based on local accounting controller can be solely tracked on the BIER-TE Controller based on
as long as no forward_routed adjacencies are used (see Section 3.2.1 local accounting as long as no forward_routed() adjacencies are used
for the definition of forward_routed adjacencies). When (see Section 4.2.1 for the definition of forward_routed()
forward_routed adjacencies are used, the paths selected by the adjacencies). When forward_routed() adjacencies are used, the paths
underlying routing protocol need to be tracked as well. selected by the underlying routing protocol need to be tracked as
well.
Resource management has implications on the forwarding plane beyond Resource management has implications on the forwarding plane beyond
the BIER-TE defined steering of packets. This includes allocation of the BIER-TE defined steering of packets. This includes allocation of
buffers to guarantee the worst case requirements of admitted RCSD buffers to guarantee the worst case requirements of admitted RCSD
trafic and potential policing and/or rate-shaping mechanisms, traffic and potential policing and/or rate-shaping mechanisms,
typically done via various forms of queuing. This level of resource typically done via various forms of queuing. This level of resource
control, while optional, is important in networks that wish to control, while optional, is important in networks that wish to
support congestion management policies to control or regulate the support congestion management policies to control or regulate the
offered traffic to deliver different levels of service and alleviate offered traffic to deliver different levels of service and alleviate
congestion problems, or those networks that wish to control latencies congestion problems, or those networks that wish to control latencies
experienced by specific traffic flows. experienced by specific traffic flows.
3. BIER-TE Forwarding 4. BIER-TE Forwarding
3.1. The Bit Index Forwarding Table (BIFT) 4.1. The Bit Index Forwarding Table (BIFT)
The Bit Index Forwarding Table (BIFT) exists in every BFR. For every The Bit Index Forwarding Table (BIFT) exists in every BFR. For every
subdomain in use, it is a table indexed by SI:BitPosition and is sub-domain in use, it is a table indexed by SI:bit position and is
populated by the BIER-TE control plane. Each index can be empty or populated by the BIER-TE control plane. Each index can be empty or
contain a list of one or more adjacencies. contain a list of one or more adjacencies.
BIER-TE can support multiple subdomains like BIER. Each one with a Like BIER, BIER-TE can support multiple sub-domains, each with a
separate BIFT separate BIFT.
In the BIER architecture, indices into the BIFT are explained to be In [RFC8279], Figure 2, indices into the BIFT are both SI:BitString
both BFR-id and SI:BitString (BitPosition). This is because there is and BFR-id, where BitString is indicating a BP: BFR-id = SI * 2^BSL +
a 1:1 relationship between BFR-id and SI:BitString - every bit in BP. As shown in Figure 4, in BIER-TE, only SI:BP are used as indices
every SI is/can be assigned to a BFIR/BFER. In BIER-TE there are into a BIFT because they identify adjacencies and not BFR.
more bits used in each BitString than there are BFIR/BFER assigned to
the bitstring. This is because of the bits required to express the
engineered path through the topology. The BIER-TE forwarding
definitions do therefore not use the term BFR-id at all. Instead,
BFR-ids are only used as required by routing underlay, flow overlay
of BIER headers. Please refer to Section 7 for explanations how to
deal with SI, subdomains and BFR-id in BIER-TE.
------------------------------------------------------------------ ------------------------------------------------------------------
| Index: | Adjacencies: | | Index: | Adjacencies: |
| SI:BitPosition | <empty> or one or more per entry | | SI:bit position | <empty> or one or more per entry |
================================================================== ==================================================================
| 0:1 | forward_connected(interface,neighbor{,DNR}) | | 0:1 | forward_connected(interface,neighbor{,DNC}) |
------------------------------------------------------------------ ------------------------------------------------------------------
| 0:2 | forward_connected(interface,neighbor{,DNR}) | | 0:2 | forward_connected(interface,neighbor{,DNC}) |
| | forward_connected(interface,neighbor{,DNR}) | | | forward_connected(interface,neighbor{,DNC}) |
------------------------------------------------------------------ ------------------------------------------------------------------
| 0:3 | local_decap({VRF}) | | 0:3 | local_decap({VRF}) |
------------------------------------------------------------------ ------------------------------------------------------------------
| 0:4 | forward_routed({VRF,}l3-neighbor) | | 0:4 | forward_routed({VRF,}l3-neighbor) |
------------------------------------------------------------------ ------------------------------------------------------------------
| 0:5 | <empty> | | 0:5 | <empty> |
------------------------------------------------------------------ ------------------------------------------------------------------
| 0:6 | ECMP({adjacency1,...adjacencyN}, seed) | | 0:6 | ECMP({adjacency1,...adjacencyN}, seed) |
------------------------------------------------------------------ ------------------------------------------------------------------
... ...
| BitStringLength | ... | | BitStringLength | ... |
------------------------------------------------------------------ ------------------------------------------------------------------
Bit Index Forwarding Table Bit Index Forwarding Table
Figure 4: BIFT adjacencies Figure 4: BIFT adjacencies
The BIFT is programmed into the data plane of BFRs by the BIER-TE The BIFT is programmed into the data plane of BFRs by the BIER-TE
Controller and used to forward packets, according to the rules Controller and used to forward packets, according to the rules
specified in the BIER-TE Forwarding Procedures. specified in the BIER-TE Forwarding Procedures.
Adjacencies for the same BP when populated in more than one BFR by Note that a BIFT index (SI:BP) may be populated in the BIFT of more
the BIER-TE Controller does not have to have the same adjacencies. than one BFR. See Section 5.1.6 for an example of how a BIER-TE
This is up to the BIER-TE Controller. BPs for p2p links are one case controller could assign BPs to (logical) adjacencies shared across
(see below). multiple BFRs, Section 5.1.3 for an example of assigning the same BP
to different adjacencies, and Section 5.1.9 for guidelines regarding
re-use of BPs across different adjacencies.
{VRF}indicates the Virtual Routing and Forwarding context into which {VRF} indicates the Virtual Routing and Forwarding context into which
the BIER payload is to be delivered. This is optional and depends on the BIER payload is to be delivered. This is optional and depends on
the multicast flow overlay. the multicast flow overlay.
3.2. Adjacency Types 4.2. Adjacency Types
3.2.1. Forward Connected 4.2.1. Forward Connected
A "forward_connected" adjacency is towards a directly connected BFR A "forward_connected" adjacency is towards a directly connected BFR
neighbor using an interface address of that BFR on the connecting neighbor using an interface address of that BFR on the connecting
interface. A forward_connected adjacency does not route packets but interface. A forward_connected() adjacency does not route packets
only L2 forwards them to the neighbor. but only L2 forwards them to the neighbor.
Packets sent to an adjacency with "DoNotReset" (DNR) set in the BIFT Packets sent to an adjacency with "DoNotClear" (DNC) set in the BIFT
will not have the BitPosition for that adjacency reset when the BFR MUST NOT have the bit position for that adjacency cleared when the
creates a copy for it. The BitPosition will still be reset for BFR creates a copy for it. The bit position will still be cleared
copies of the packet made towards other adjacencies. This can be for copies of the packet made towards other adjacencies. This can be
used for example in ring topologies as explained below. used for example in ring topologies as explained below.
3.2.2. Forward Routed 4.2.2. Forward_routed
A "forward_routed" adjacency is an adjacency towards a BFR that is
not a forward_connected adjacency: towards a loopback address of a
BFR or towards an interface address that is non-directly connected.
Forward_routed packets are forwarded via the Routing Underlay.
If the Routing Underlay has multiple paths for a forward_routed A "forward_routed" adjacency is an adjacency towards a BFR that uses
adjacency, it will perform ECMP independent of BIER-TE for packets a (tunneling) encapsulation which will cause the packet to be
forwarded across a forward_routed adjacency. This is independent of forwarded by the routing underlay toward the adjacent BFR. This can
BIER-TE ECMP described in Section 3.2.3. leverage any feasible encapsulation, such as MPLS or tunneling over
IP/IPv6, as long as the BIER-TE packet can be identified as a
payload. This identification can either rely on the BIER/BIER-TE co-
existence mechanisms described in Section 4.3, or by explicit support
for a BIER-TE payload type in the tunneling encapsulation.
If the Routing Underlay has FRR, it will perform FRR independent of "forward_routed" adjacencies are necessary to pass BIER-TE traffic
BIER-TE for packets forwarded across a forward_routed adjacency. across non BIER-TE capable routers or to minimize the number of
required BP by tunneling over (BIER-TE capable) routers on which
neither replication nor path-steering is desired, or simply to
leverage path redundancy and FRR of the routing underlay towards the
next BFR. They may also be useful to a multi-subnet adjacent BFR to
leverage the routing underlay ECMP independent of BIER-TE ECMP
(Section 4.2.3).
3.2.3. ECMP 4.2.3. ECMP
The ECMP mechanisms in BIER are tied to the BIER BIFT and are BIER ECMP is tied to the BIER BIFT processing semantic and are
therefore not directly useable with BIER-TE. The following therefore not directly usable with BIER-TE.
procedures describe ECMP for BIER-TE that we consider to be
lightweight but also well manageable. It leverages the existing
entropy parameter in the BIER header to keep packets of the flows on
the same path and it introduces a "seed" parameter to allow for
traffic flows to be polarized or randomized across multiple hops.
An "Equal Cost Multipath" (ECMP) adjacency has a list of two or more A BIER-TE "Equal Cost Multipath" (ECMP) adjacency has a list of two
adjacencies included in it. It copies the BIER-TE to one of those or more non-ECMP adjacencies and a seed parameter. When a BIER-TE
adjacencies based on the ECMP hash calculation. The BIER-TE ECMP packet is copied onto such an ECMP adjacency, an implementation
hash algorithm must select the same adjacency from that list for all specific so-called hash function will select one out of the lists
packets with the same "entropy" value in the BIER-TE header if the adjacencies to which the packet is forwarded. This ECMP hash
same number of adjacencies and same seed are given as parameters. function MUST select the same adjacency from that list for all
Further use of the seed parameter is explained below. packets with the same entropy parameter. The seed parameter allows
to design hash functions that are easy to implement at high speed
without running into polarization issues across multiple consecutive
ECMP hops. See Section 5.1.7 for more explanations.
3.2.4. Local Decap 4.2.4. Local Decap(sulation)
A "local_decap" adjacency passes a copy of the payload of the BIER-TE A "local_decap" adjacency passes a copy of the payload of the BIER-TE
packet to the packets NextProto within the BFR (IPv4/IPv6, packet to the protocol within the BFR (IPv4/IPv6, Ethernet,...)
Ethernet,...). A local_decap adjacency turns the BFR into a BFER for responsible for that payload according to the packet header fields.
matching packets. Local_decap adjacencies require the BFER to A local_decap() adjacency turns the BFR into a BFER for matching
support routing or switching for NextProto to determine how to packets. Local_decap() adjacencies require the BFER to support
further process the packet. routing or switching for NextProto to determine how to further
process the packet.
3.3. Encapsulation considerations 4.3. Encapsulation / Co-existence with BIER
Specifications for BIER-TE encapsulation are outside the scope of Specifications for BIER-TE encapsulation are outside the scope of
this document. This section gives explanations and guidelines. this document. This section gives explanations and guidelines.
Because a BFR needs to interpret the BitString of a BIER-TE packet Because a BFR needs to interpret the BitString of a BIER-TE packet
differently from a BIER packet, it is necessary to distinguish BIER differently from a BIER packet, it is necessary to distinguish BIER
from BIER-TE packets. This is subject to definitions in BIER from BIER-TE packets. In the BIER encapsulation [RFC8296], the BIFT-
encapsulation specifications. id field of the packet indicates the BIFT of the packet. BIER and
BIER-TE can therefore be run simultaneously, when the BIFT-id address
space is shared across BIER BIFT and BIER-TE BIFT. Partitioning the
BIFT-id address space is subject to BIER-TE/BIER control plane
procedures.
MPLS encapsulation [RFC8296] for example assigns one label by which When [RFC8296] is used for BIER with MPLS, BIFT-id address ranges can
BFRs recognizes BIER packets for every (SI,subdomain) combination. be dynamically allocated from MPLS label space only for the set of
If it is desirable that every subdomain can forward only BIER or actually used SD:BSL BIFT. This allows to also allocate non-
BIER-TE packets, then the label allocation could stay the same, and overlapping label ranges for BIFT-ids that are to be used with BIER-
only the forwarding model (BIER/BIER-TE) would have to be defined per TE BIFTs.
subdomain. If it is desirable to support both BIER and BIER-TE
forwarding in the same subdomain, then additional labels would need With MPLS, it is also possible to reuse the same SD space for both
to be assigned for BIER-TE forwarding. BIER-TE and BIER, so that the same SD has both a BIER BIFT and
according range of BIFT-ids and a disjoint BIER-TE BIFT and non-
overlapping range of BIFT-ids.
When a fixed mapping from BSL, SD, SI is used without specifically
distinguishing BIER and BIER-TE, such as proposed for non-MPLS
forwarding with [RFC8296] in [I-D.ietf-bier-non-mpls-bift-encoding]
revision 04, section 5., then it is necessary to allocate disjoint
SDs to BIER and BIER-TE BIFT so that both can be addressed by the
BIFT-ids. The encoding proposed in section 6. of the same document
does not statically encode BSL or SD into the BIFT-id, but allows for
a mapping, and hence could provide for the same freedom as when MPLS
is being used (same or different SD for BIER/BIER-TE).
"forward_routed" requires an encapsulation permitting to unicast "forward_routed" requires an encapsulation permitting to unicast
BIER-TE packets to a specific interface address on a target BFR. BIER-TE packets to a specific interface address on a target BFR.
With MPLS encapsulation, this can simply be done via a label stack With MPLS encapsulation, this can simply be done via a label stack
with that addresses label as the top label - followed by the label with that addresses label as the top label - followed by the label
assigned to (SI,subdomain) - and if necessary (see above) BIER-TE. assigned to the (BSL,SD,SI) BitString. With non-MPLS encapsulation,
With non-MPLS encapsulation, some form of IP encapsulation would be some form of IP encapsulation would be required (for example IP/GRE).
required (for example IP/GRE).
The encapsulation used for "forward_routed" adjacencies can equally The encapsulation used for "forward_routed" adjacencies can equally
support existing advanced adjacency information such as "loose source support existing advanced adjacency information such as "loose source
routes" via e.g. MPLS label stacks or appropriate header extensions routes" via e.g. MPLS label stacks or appropriate header extensions
(e.g. for IPv6). (e.g. for IPv6).
3.4. Basic BIER-TE Forwarding Example 4.4. BIER-TE Forwarding Pseudocode
The following pseudocode, Figure 5, for BIER-TE forwarding is based
on the BIER forwarding pseudocode of [RFC8279], section 6.5 with one
modification.
void ForwardBitMaskPacket_withTE (Packet)
{
SI=GetPacketSI(Packet);
Offset=SI*BitStringLength;
for (Index = GetFirstbit position(Packet->BitString); Index ;
Index = GetNextbit position(Packet->BitString, Index)) {
F-BM = BIFT[Index+Offset]->F-BM;
if (!F-BM) continue; [3]
BFR-NBR = BIFT[Index+Offset]->BFR-NBR;
PacketCopy = Copy(Packet);
PacketCopy->BitString &= F-BM; [2]
PacketSend(PacketCopy, BFR-NBR);
// The following must not be done for BIER-TE:
// Packet->BitString &= ~F-BM; [1]
}
}
Figure 5: BIER-TE Forwarding Pseudocode for required functions, based
on BIER Pseudocode
In step [2], the F-BM is used to clear bit(s) in PacketCopy. This
step exists in both BIER and BIER-TE, but the F-BMs need to be
populated differently for BIER-TE than for BIER for the desired
clearing.
In BIER, multiple bits of a BitString can have the same BFR-NBR.
When a received packets BitString has more than one of those bits
set, the BIER replication logic has to avoid that more than one
PacketCopy is sent to that BFR-NBR ([1]). Likewise, the PacketCopy
sent to a BFR-NBR must clear all bits in its BitString that are not
routed across BFR-NBR. This protects against BIER replication on any
possible further BFR to create duplicates ([2]).
To solve both [1] and [2] for BIER, the F-BM of each bit needs to
have all bits set that this BFR wants to route across BFR-NBR. [2]
clears all other bits in PacketCopy->BitString, and [1] clears those
bits from Packet->BitString after the first PacketCopy.
In BIER-TE, a BFR-NBR is an adjacency, forward_connected,
forward_routed or local_decap. There is no need for [2] to suppress
duplicates in the way BIER does because in general, different BP
would never have the same adjacency. If a BIER-TE controller
actually finds some optimization in which this would be desirable,
then the controller is also responsible to ensure that only one of
those bits is set in any Packet->BitString, unless the controller
explicitly wants for duplicates to be created.
For BIER-TE, F-BM is handled as follows:
1. The F-BM of all bits without an adjacency has all bits clear.
This will cause [3] to skip further processing of such a bit.
2. All bits with an adjacency (with DNC flag clear) have an F-BM
that has only those bits set for which this BFR does not have an
adjacency. This causes [2] to clear all bits from
PacketCopy->BitString for which this BFR does have an adjacency.
3. [1] is not performed for BIER-TE. All bit clearing required by
BIER-TE is performed by [2].
This Forwarding Pseudocode can support the REQUIRED BIER-TE
forwarding functions (see Section 4.6), forward_connected,
forward_routed() and local decap, but not the RECOMMENDED functions
DNC flag and multiple adjacencies per bit nor the OPTIONAL function,
ECMP adjacencies. The DNC flag cannot be supported when using only
[1] to mask bits.
The modified and expanded Forwarding Pseudocode in Figure 6 specifies
how to support all BIER-TE forwarding functions (required,
recommended and optional):
o This pseudocode eliminates per-bit F-BM, therefore reducing the
size of BIFT state by BitStringLength^2*SI and eliminating the
need for per-packet-copy masking operation except for adjacencies
with the DNC flag set:
* AdjacentBits[SI] are bits with a non-empty list of adjacencies.
This can be computed whenever the BIER-TE Controller updates
the adjacencies.
* Only the AdjacentBits need to be examined in the loop for
packet copies.
* The packets BitString is masked with those AdjacentBits before
the loop to avoid doing this repeatedly for every PacketCopy.
o The code loops over the adjacencies because there may be more than
one adjacency for a bit.
o When an adjacency has the DNC bit, the bit is set in the packet
copy (to save bits in rings for example).
o The ECMP adjacency is shown. Its parameters are a
ListOfAdjacencies from which one is picked.
o The forward_local, forward_routed, local_decap() adjacencies are
shown with their parameters.
void ForwardBitMaskPacket_withTE (Packet)
{
SI=GetPacketSI(Packet);
Offset=SI*BitStringLength;
AdjacentBits = Packet->BitString &= ~AdjacentBits[SI];
Packet->BitString &= AdjacentBits[SI];
for (Index = GetFirstbit position(AdjacentBits); Index ;
Index = GetNextbit position(AdjacentBits, Index)) {
foreach adjacency BIFT[Index+Offset] {
if(adjacency == ECMP(ListOfAdjacencies, seed) ) {
I = ECMP_hash(sizeof(ListOfAdjacencies),
Packet->Entropy, seed);
adjacency = ListOfAdjacencies[I];
}
PacketCopy = Copy(Packet);
switch(adjacency) {
case forward_connected(interface,neighbor,DNC):
if(DNC)
PacketCopy->BitString |= 1<<(Index-1);
SendToL2Unicast(PacketCopy,interface,neighbor);
case forward_routed({VRF},neighbor):
SendToL3(PacketCopy,{VRF,}l3-neighbor);
case local_decap({VRF},neighbor):
DecapBierHeader(PacketCopy);
PassTo(PacketCopy,{VRF,}Packet->NextProto);
}
}
}
}
Figure 6: Complete BIER-TE Forwarding Pseudocode for required,
recommended and optional functions
4.5. Basic BIER-TE Forwarding Example
[RFC Editor: remove this section.] [RFC Editor: remove this section.]
THIS SECTION TO BE REMOVED IN RFC BECAUSE IT WAS SUPERCEEDED BY THIS SECTION TO BE REMOVED IN RFC BECAUSE IT WAS SUPERCEEDED BY
SECTION 1.1 EXAMPLE - UNLESS REVIEWERS CHIME IN AND EXPRESS DESIRE TO SECTION 1.1 EXAMPLE - UNLESS REVIEWERS CHIME IN AND EXPRESS DESIRE TO
KEEP THIS ADDITIONAL EXAMPLE SECTION. KEEP THIS ADDITIONAL EXAMPLE SECTION. ALVARO RETANA DID NOT MIND
ANOTHER EXAMPLE.
Step by step example of basic BIER-TE forwarding. This does not use
ECMP or forward_routed adjacencies nor does it try to minimize the
number of required BitPositions for the topology.
[BIER-TE Controller]
/ | \
v v v
| p13 p1 | Step by step example of basic BIER-TE forwarding. This example does
+- BFIR2 --+ | not use ECMP or forward_routed() adjacencies nor does it try to
| | p2 p6 | LAN2 minimize the number of required BitPositions for the topology.
| +-- BFR3 --+ |
| | | p7 p11 |
Src -+ +-- BFER1 --+
| | p3 p8 | |
| +-- BFR4 --+ +-- Rcv1
| | | |
| |
| p14 p4 |
+- BFIR1 --+ |
| +-- BFR5 --+ p10 p12 |
LAN1 | p5 p9 +-- BFER2 --+
| +-- Rcv2
|
LAN3
IP |..... BIER-TE network......| IP [BIER-TE Controller]
/ | \
v v v
. .
| p13 p1 | .
+- BFIR2 --+ | .
| . | p2 p6 | . LAN2
| . +-- BFR3 --+ . |
| . | | p7 p11 |
Src -+ . +-- BFER1 --+
| . | p3 p8 | . |
| . +-- BFR4 --+ . +-- Rcv1
| . | | . |
| . | .
| p14 p4 | .
+- BFIR1 --+ | .
| . +-- BFR5 --+ p10 p12 |
LAN1 . | p5 p9 +-- BFER2 --+
. | . +-- Rcv2
. . |
. . LAN3
. .
IP |..... BIER-TE network.....| IP
Figure 5: BIER-TE Forwarding Example Figure 7: BIER-TE Forwarding Example
pXX indicate the BitPositions number assigned by the BIER-TE pXX indicate the BitPositions number assigned by the BIER-TE
Controller to adjacencies in the BIER-TE topology. For example, p9 Controller to adjacencies in the BIER-TE topology. For example, p9
is the adjacency towards BFR5 on the LAN connecting to BFER2. is the adjacency towards BFR5 on the LAN connecting to BFER2.
BIFT BFIR2: BIFT BFIR2:
p13: local_decap() p13: local_decap
p2: forward_connected(BFR3) p2: forward_connected(BFR3)
BIFT BFR3: BIFT BFR3:
p1: forward_connected(BFIR2) p1: forward_connected(BFIR2)
p7: forward_connected(BFER1) p7: forward_connected(BFER1)
p8: forward_connected(BFR4) p8: forward_connected(BFR4)
BIFT BFER1: BIFT BFER1:
p11: local_decap() p11: local_decap
p6: forward_connected(BFR3) p6: forward_connected(BFR3)
p8: forward_connected(BFR4) p8: forward_connected(BFR4)
Figure 6: BIER-TE Forwarding Example Adjacencies Figure 8: BIER-TE Forwarding Example Adjacencies
...and so on. ...and so on.
For example, we assume that some multicast traffic seen on LAN1 needs For example, we assume that some multicast traffic seen on LAN1 needs
to be sent via BIER-TE by BFIR2 towards Rcv1 and Rcv2. The BIER-TE to be sent via BIER-TE by BFIR2 towards Rcv1 and Rcv2. The BIER-TE
Controller determines it wants it to pass this traffic across the Controller determines it wants it to pass this traffic across the
following paths: following paths:
-> BFER1 ---------------> Rcv1 -> BFER1 ---------------> Rcv1
BFIR2 -> BFR3 BFIR2 -> BFR3
-> BFR4 -> BFR5 -> BFER2 -> Rcv2 -> BFR4 -> BFR5 -> BFER2 -> Rcv2
Figure 7: BIER-TE Forwarding Example Paths Figure 9: BIER-TE Forwarding Example Paths
These paths equal to the following BitString: p2, p5, p7, p8, p10, These paths equal to the following BitString: p2, p5, p7, p8, p10,
p11, p12. p11, p12.
This BitString is assigned by BFIR2 to the example multicast traffic This BitString is assigned by BFIR2 to the example multicast traffic
received from LAN1. received from LAN1.
Then BFIR2 forwards this multicast traffic with BIER-TE based on that Then BFIR2 forwards this multicast traffic with BIER-TE based on that
BitString. The BIFT of BFIR2 has only p2 and p13 populated. Only p2 BitString. The BIFT of BFIR2 has only p2 and p13 populated. Only p2
is in the BitString and this is an adjacency towards BFR3. BFIR2 is in the BitString and this is an adjacency towards BFR3. BFIR2
therefore resets p2 in the BitString and sends a copy towards BFR2. therefore clears p2 in the BitString and sends a copy towards BFR2.
BFR3 sees a BitString of p5,p7,p8,p10,p11,p12. It is only interested BFR3 sees a BitString of p5,p7,p8,p10,p11,p12. For those BPs, it has
in p1,p7,p8. It creates a copy of the packet to BFER1 (due to p7) only adjacencies for p7,p8. It creates a copy of the packet to BFER1
and one to BFR4 (due to p8). It resets p7, p8 before sending. (due to p7) and one to BFR4 (due to p8). It clears p7, p8 before
sending.
BFER1 sees a BitString of p5,p10,p11,p12. It is only interested in BFER1 sees a BitString of p5,p10,p11,p12. For those BPs, it only has
p6,p7,p8,p11 and therefore considers only p11. p11 is a "local_decap" an adjacency for p11. p11 is a "local_decap" adjacency installed by
adjacency installed by the BIER-TE Controller because BFER1 should the BIER-TE Controller to receive a copy of the BIER packet - dispose
pass packets to IP multicast. The local_decap adjacency instructs of the BIER header and pass the payload to IP multicast. IP
BFER1 to create a copy, decapsulate it from the BIER header and pass
it on to the NextProtocol, in this example IP multicast. IP
multicast will then forward the packet out to LAN2 because it did multicast will then forward the packet out to LAN2 because it did
receive PIM or IGMP joins on LAN2 for the traffic. receive PIM or IGMP joins on LAN2 for the traffic.
Further processing of the packet in BFR4, BFR5 and BFER2 accordingly. Further processing of the packet in BFR4, BFR5 and BFER2 accordingly.
3.5. Forwarding comparison with BIER 4.6. BFR Requirements for BIER-TE forwarding
Forwarding of BIER-TE is designed to allow common forwarding hardware
with BIER. In fact, one of the main goals of this document is to
encourage the building of forwarding hardware that can not only
support BIER, but also BIER-TE - to allow experimentation with BIER-
TE and support building of BIER-TE control plane code.
The pseudocode in Section 6 shows how existing BIER/BIFT forwarding
can be amended to support basic BIER-TE forwarding, by using BIER
BIFT's F-BM. Only the masking of bits due to avoid duplicates must
be skipped when forwarding is for BIER-TE.
Whether to use BIER or BIER-TE forwarding can simply be a configured BFR MUST support to configure the BIFT of sub-domains so that they
choice per subdomain and accordingly be set up by a BIER-TE use BIER-TE forwarding rules instead of BIER forwarding rules. Every
Controller. The BIER packet encapsulation [RFC8296] too can be BP in the BIFT MUST support to have zero or one adjacency.
reused without changes except that the currently defined BIER-TE ECMP Forwarding MUST support the adjacency types forward_connected() with
adjacency does not leverage the entropy field so that field would be clear DNC flag, forward_routed() and local_decap. As explained in
unused when BIER-TE forwarding is used. Section 4.4, these REQUIRED BIER-TE forwarding functions can be
implement via the same Forwarding Pseudocode as BIER forwarding
except for one modification (skipping one masking with F-BM).
3.6. Requirements BIER-TE forwarding SHOULD support forward_connected() adjacencies
with a set DNC flag, as this is highly useful to save bits in rings
(see Section 5.1.6).
Basic BIER-TE forwarding MUST support to configure Subdomains to use BIER-TE forwarding SHOULD support more than one adjacency on a bit.
basic BIER-TE forwarding rules (instead of BIER). With basic BIER-TE This allows to save bits in hub&spoke scenarios (see Section 5.1.5).
forwarding, every bit MUST support to have zero or one adjacency. It
MUST support the adjacency types forward_connected without DNR flag,
forward_routed and local_decap. All other BIER-TE forwarding
features are optional. These basic BIER-TE requirements make BIER-TE
forwarding exactly the same as BIER forwarding with the exception of
skipping the aforementioned F-BM masking on egress.
BIER-TE forwarding SHOULD support the DNR flag, as this is highly BIER-TE forwarding MAY support ECMP adjacencies to save bits in ECMP
useful to save bits in rings (see Section 4.6). scenarios, see Section 5.1.7 for an example. This is a MAY
requirement, because the deployment importance of ECMP adjacencies
for BIER-TE is unclear as one can also leverage ECMP of the routing
underlay via forwarded_routed adjacencies and/or might prefer to have
more explicit control of the path chosen via explicit BP/adjacencies
for each ECMP path alternative.
BIER-TE forwarding MAY support more than one adjacency on a bit and 5. BIER-TE Controller Operational Considerations
ECMP adjacencies. The importance of ECMP adjacencies is unclear when
traffic steering is used because it may be more desirable to
explicitly steer traffic across non-ECMP paths to make per-path
traffic calculation easier for BIER-TE Controllers. Having more than
one adjacency for a bit allows further savings of bits in hub&spoke
scenarios, but unlike rings it is less "natural" to flood traffic
across multiple links unconditional. Both ECMP and multiple
adjacencies are forwarding plane features that should be possible to
support later when needed as they do not impact the basic BIER-TE
replication loop. This is true because there is no inter-copy
dependency through resetting of F-BM as in BIER.
4. BIER-TE Controller BitPosition Assignments 5.1. Bit position Assignments
This section describes how the BIER-TE Controller can use the This section describes how the BIER-TE Controller can use the
different BIER-TE adjacency types to define the BitPositions of a different BIER-TE adjacency types to define the bit positions of a
BIER-TE domain. BIER-TE domain.
Because the size of the BitString is limiting the size of the BIER-TE Because the size of the BitString limits the size of the BIER-TE
domain, many of the options described exist to support larger domain, many of the options described exist to support larger
topologies with fewer BitPositions (4.1, 4.3, 4.4, 4.5, 4.6, 4.7, topologies with fewer bit positions (4.1, 4.3, 4.4, 4.5, 4.6, 4.7,
4.8). 4.8).
4.1. P2P Links 5.1.1. P2P Links
Each P2p link in the BIER-TE domain is assigned one unique On a P2P link that connects two BFR, the same bit position can be
BitPosition with a forward_connected adjacency pointing to the used on both BFR for the adjacency to the neighboring BFR. A P2P
neighbor on the p2p link. link requires therefore only one bit position.
4.2. BFER 5.1.2. BFER
Every non-Leaf BFER is given a unique BitPosition with a local_decap Every non-Leaf BFER is given a unique bit position with a local_decap
adjacency. adjacency.
4.3. Leaf BFERs 5.1.3. Leaf BFERs
BFR1(P) BFR2(P) BFR1(P) BFR2(P) BFR1(P) BFR2(P) BFR1(P) BFR2(P)
| \ / | | | | \ / | | |
| X | | | | X | | |
| / \ | | | | / \ | | |
BFER1(PE) BFER2(PE) BFER1(PE)----BFER2(PE) BFER1(PE) BFER2(PE) BFER1(PE)----BFER2(PE)
^ U-turn link
Leaf BFER / Non-Leaf BFER / Leaf BFER / Non-Leaf BFER /
PE-router PE-router PE-router PE-router
Figure 8: Leaf vs. non-Leaf BFER Example Figure 10: Leaf vs. non-Leaf BFER Example
Leaf BFERs are BFERs where incoming BIER-TE packets never need to be A leaf BFERs is one where incoming BIER-TE packets never need to be
forwarded to another BFR but are only sent to the BFER to exit the forwarded to another BFR but are only sent to the BFER to exit the
BIER-TE domain. For example, in networks where PEs are spokes BIER-TE domain. For example, in networks where Provider Edge (PE)
connected to P routers, those PEs are Leaf BFERs unless there is a router are spokes connected to Provider (P) routers, those PEs are
U-turn between two PEs. Consider how redundant disjoint traffic can Leaf BFERs unless there is a U-turn between two PEs.
reach BFER1/BFER2 in above picture: When BFER1/BFER2 are Non-Leaf
BFER as shown on the right hand side, one traffic copy would be
forwarded to BFER1 from BFR1, but the other one could only reach
BFER1 via BFER2, which makes BFER2 a non-Leaf BFER. Likewise BFER1
is a non-Leaf BFER when forwarding traffic to BFER2.
Note that the BFERs in the left hand picture are only guaranteed to Consider how redundant disjoint traffic can reach BFER1/BFER2 in
be leaf-BFER by fitting routing configuration that prohibits transit Figure 10: When BFER1/BFER2 are Non-Leaf BFER as shown on the right
traffic to pass through the BFERs, which is commonly applied in these hand side, one traffic copy would be forwarded to BFER1 from BFR1,
topologies. but the other one could only reach BFER1 via BFER2, which makes BFER2
a non-Leaf BFER. Likewise BFER1 is a non-Leaf BFER when forwarding
traffic to BFER2. Note that the BFERs in the left hand picture are
only guaranteed to be leaf-BFER by fitting routing configuration that
prohibits transit traffic to pass through the BFERs, which is
commonly applied in these topologies.
All leaf-BFER in a BIER-TE domain can share a single BitPosition. All leaf-BFERs in a BIER-TE domain can share a single bit position.
This is possible because the BitPosition for the adjacency to reach This is possible because the bit position for the adjacency to reach
the BFER can be used to distinguish whether or not packets should the BFER can be used to distinguish whether or not packets should
reach the BFER. reach the BFER.
This optimization will not work if an upstream interface of the BFER This optimization will not work if an upstream interface of the BFER
is using a BitPosition optimized as described in the following two is using a bit position optimized as described in the following two
sections (LAN, Hub and Spoke). sections (LAN, Hub and Spoke).
4.4. LANs 5.1.4. LANs
In a LAN, the adjacency to each neighboring BFR on the LAN is given a In a LAN, the adjacency to each neighboring BFR is given a unique bit
unique BitPosition. The adjacency of this BitPosition is a position. The adjacency of this bit position is a
forward_connected adjacency towards the BFR and this BitPosition is forward_connected() adjacency towards the BFR and this bit position
populated into the BIFT of all the other BFRs on that LAN. is populated into the BIFT of all the other BFRs on that LAN.
BFR1 BFR1
|p1 |p1
LAN1-+-+---+-----+ LAN1-+-+---+-----+
p3| p4| p2| p3| p4| p2|
BFR3 BFR4 BFR7 BFR3 BFR4 BFR7
Figure 9: LAN Example Figure 11: LAN Example
If Bandwidth on the LAN is not an issue and most BIER-TE traffic If Bandwidth on the LAN is not an issue and most BIER-TE traffic
should be copied to all neighbors on a LAN, then BitPositions can be should be copied to all neighbors on a LAN, then bit positions can be
saved by assigning just a single BitPosition to the LAN and saved by assigning just a single bit position to the LAN and
populating the BitPosition of the BIFTs of each BFRs on the LAN with populating the bit position of the BIFTs of each BFRs on the LAN with
a list of forward_connected adjacencies to all other neighbors on the a list of forward_connected() adjacencies to all other neighbors on
LAN. the LAN.
This optimization does not work in the case of BFRs redundantly This optimization does not work in the case of BFRs redundantly
connected to more than one LANs with this optimization because these connected to more than one LAN with this optimization because these
BFRs would receive duplicates and forward those duplicates into the BFRs would receive duplicates and forward those duplicates into the
opposite LANs. Adjacencies of such BFRs into their LANs still need a opposite LANs. Adjacencies of such BFRs into their LAN still need a
separate BitPosition. separate bit position.
4.5. Hub and Spoke 5.1.5. Hub and Spoke
In a setup with a hub and multiple spokes connected via separate p2p In a setup with a hub and multiple spokes connected via separate p2p
links to the hub, all p2p links can share the same BitPosition. The links to the hub, all p2p links can share the same bit position. The
BitPosition on the hub's BIFT is set up with a list of bit position on the hub's BIFT is set up with a list of
forward_connected adjacencies, one for each Spoke. forward_connected() adjacencies, one for each Spoke.
This option is similar to the BitPosition optimization in LANs: This option is similar to the bit position optimization in LANs:
Redundantly connected spokes need their own BitPositions. Redundantly connected spokes need their own bit positions, unless
they are themselves Leaf-BFER.
This type of optimized BP could be used for example when all traffic This type of optimized BP could be used for example when all traffic
is "broadcast" traffic (very dense receiver set) such as live-TV or is "broadcast" traffic (very dense receiver set) such as live-TV or
situation-awareness (SA). This BP optimization can then be used to situation-awareness (SA). This BP optimization can then be used to
explicitly steer different traffic flows across different ECMP paths explicitly steer different traffic flows across different ECMP paths
in Data-Center or broadband-aggregation networks with minimal use of in Data-Center or broadband-aggregation networks with minimal use of
BPs. BPs.
4.6. Rings 5.1.6. Rings
In L3 rings, instead of assigning a single BitPosition for every p2p In L3 rings, instead of assigning a single bit position for every p2p
link in the ring, it is possible to save BitPositions by setting the link in the ring, it is possible to save bit positions by setting the
"Do Not Reset" (DNR) flag on forward_connected adjacencies. "DoNotClear" (DNC) flag on forward_connected() adjacencies.
For the rings shown in the following picture, a single BitPosition For the rings shown in Figure 12, a single bit position will suffice
will suffice to forward traffic entering the ring at BFRa or BFRb all to forward traffic entering the ring at BFRa or BFRb all the way up
the way up to BFR1: to BFR1:
On BFRa, BFRb, BFR30,... BFR3, the BitPosition is populated with a On BFRa, BFRb, BFR30,... BFR3, the bit position is populated with a
forward_connected adjacency pointing to the clockwise neighbor on the forward_connected() adjacency pointing to the clockwise neighbor on
ring and with DNR set. On BFR2, the adjacency also points to the the ring and with DNC set. On BFR2, the adjacency also points to the
clockwise neighbor BFR1, but without DNR set. clockwise neighbor BFR1, but without DNC set.
Handling DNR this way ensures that copies forwarded from any BFR in Handling DNC this way ensures that copies forwarded from any BFR in
the ring to a BFR outside the ring will not have the ring BitPosition the ring to a BFR outside the ring will not have the ring bit
set, therefore minimizing the chance to create loops. position set, therefore minimizing the chance to create loops.
v v v v
| | | |
L1 | L2 | L3 L1 | L2 | L3
/-------- BFRa ---- BFRb --------------------\ /-------- BFRa ---- BFRb --------------------\
| | | |
\- BFR1 - BFR2 - BFR3 - ... - BFR29 - BFR30 -/ \- BFR1 - BFR2 - BFR3 - ... - BFR29 - BFR30 -/
| | L4 | | | | L4 | |
p33| p15| p33| p15|
BFRd BFRc BFRd BFRc
Figure 10: Ring Example Figure 12: Ring Example
Note that this example only permits for packets to enter the ring at Note that this example only permits for packets intended to make it
BFRa and BFRb, and that packets will always travel clockwise. If all the way around the ring to enter it at BFRa and BFRb, and that
packets should be allowed to enter the ring at any ring BFR, then one packets will always travel clockwise. If packets should be allowed
would have to use two ring BitPositions. One for clockwise, one for to enter the ring at any ring BFR, then one would have to use two
ring bit positions. One for each direction: clockwise and
counterclockwise. counterclockwise.
Both would be set up to stop rotating on the same link, e.g. L1. Both would be set up to stop rotating on the same link, e.g. L1.
When the ingress ring BFR creates the clockwise copy, it will reset When the ingress ring BFR creates the clockwise copy, it will clear
the counterclockwise BitPosition because the DNR bit only applies to the counterclockwise bit position because the DNC bit only applies to
the bit for which the replication is done. Likewise for the the bit for which the replication is done. Likewise for the
clockwise BitPosition for the counterclockwise copy. In result, the clockwise bit position for the counterclockwise copy. As a result,
ring ingress BFR will send a copy in both directions, serving BFRs on the ring ingress BFR will send a copy in both directions, serving
either side of the ring up to L1. BFRs on either side of the ring up to L1.
4.7. Equal Cost MultiPath (ECMP) 5.1.7. Equal Cost MultiPath (ECMP)
The ECMP adjacency allows to use just one BP per link bundle between The ECMP adjacency allows to use just one BP per link bundle between
two BFRs instead of one BP for each p2p member link of that link two BFRs instead of one BP for each p2p member link of that link
bundle. In the following picture, one BP is used across L1,L2,L3. bundle. In Figure 13, one BP is used across L1,L2,L3.
--L1----- --L1-----
BFR1 --L2----- BFR2 BFR1 --L2----- BFR2
--L3----- --L3-----
BIFT entry in BFR1: BIFT entry in BFR1:
------------------------------------------------------------------ ------------------------------------------------------------------
| Index | Adjacencies | | Index | Adjacencies |
================================================================== ==================================================================
| 0:6 | ECMP({forward_connected(L1, BFR2), | | 0:6 | ECMP({forward_connected(L1, BFR2), |
skipping to change at page 24, line 33 skipping to change at page 30, line 33
BIFT entry in BFR2: BIFT entry in BFR2:
------------------------------------------------------------------ ------------------------------------------------------------------
| Index | Adjacencies | | Index | Adjacencies |
================================================================== ==================================================================
| 0:6 | ECMP({forward_connected(L1, BFR1), | | 0:6 | ECMP({forward_connected(L1, BFR1), |
| | forward_connected(L2, BFR1), | | | forward_connected(L2, BFR1), |
| | forward_connected(L3, BFR1)}, seed) | | | forward_connected(L3, BFR1)}, seed) |
------------------------------------------------------------------ ------------------------------------------------------------------
Figure 11: ECMP Example Figure 13: ECMP Example
This document does not standardize any ECMP algorithm because it is This document does not standardize any ECMP algorithm because it is
sufficient for implementations to document their freely chosen ECMP sufficient for implementations to document their freely chosen ECMP
algorithm. This allows the BIER-TE Controller to calculate ECMP algorithm. This allows the BIER-TE Controller to calculate ECMP
paths and seeds. The following picture shows an example ECMP paths and seeds. Figure 14 shows an example ECMP algorithm:
algorithm:
forward(packet, ECMP(adj(0), adj(1),... adj(N-1), seed)): forward(packet, ECMP(adj(0), adj(1),... adj(N-1), seed)):
i = (packet(bier-header-entropy) XOR seed) % N i = (packet(bier-header-entropy) XOR seed) % N
forward packet to adj(i) forward packet to adj(i)
Figure 12: ECMP algorithm Example Figure 14: ECMP algorithm Example
In the following example, all traffic from BFR1 towards BFR10 is In the following example, all traffic from BFR1 towards BFR10 is
intended to be ECMP load split equally across the topology. This intended to be ECMP load split equally across the topology. This
example is not meant as a likely setup, but to illustrate that ECMP example is not meant as a likely setup, but to illustrate that ECMP
can be used to share BPs not only across link bundles, and it can be used to share BPs not only across link bundles, but also
across alternative paths across different transit BFR, and it
explains the use of the seed parameter. explains the use of the seed parameter.
BFR1 (BFIR) BFR1 (BFIR)
/L11 \L12 /L11 \L12
/ \ / \
BFR2 BFR3 BFR2 BFR3
/L21 \L22 /L31 \L32 /L21 \L22 /L31 \L32
/ \ / \ / \ / \
BFR4 BFR5 BFR6 BFR7 BFR4 BFR5 BFR6 BFR7
\ / \ / \ / \ /
skipping to change at page 26, line 6 skipping to change at page 32, line 6
BIFT entry in BFR6, BFR7: BIFT entry in BFR6, BFR7:
------------------------------------------------------------------ ------------------------------------------------------------------
| 0:8 | forward_connected(Lxx, BFR9) |xx differs on BFR6/BFR7| | 0:8 | forward_connected(Lxx, BFR9) |xx differs on BFR6/BFR7|
------------------------------------------------------------------ ------------------------------------------------------------------
BIFT entry in BFR8, BFR9: BIFT entry in BFR8, BFR9:
------------------------------------------------------------------ ------------------------------------------------------------------
| 0:9 | forward_connected(Lxx, BFR10) |xx differs on BFR8/BFR9| | 0:9 | forward_connected(Lxx, BFR10) |xx differs on BFR8/BFR9|
------------------------------------------------------------------ ------------------------------------------------------------------
Figure 13: Polarization Example Figure 15: Polarization Example
Note that for the following discussion of ECMP, only the BIFT ECMP Note that for the following discussion of ECMP, only the BIFT ECMP
adjacencies on BFR1, BFR2, BFR3 are relevant. The re-use of BP adjacencies on BFR1, BFR2, BFR3 are relevant. The re-use of BP
across BFR in this example is further explained in Section 4.9 below. across BFR in this example is further explained in Section 5.1.9
below.
With the setup of ECMP in above topology, traffic would not be With the setup of ECMP in the topology above, traffic would not be
equally load-split. Instead, links L22 and L31 would see no traffic equally load-split. Instead, links L22 and L31 would see no traffic
at all: BFR2 will only see traffic from BFR1 for which the ECMP hash at all: BFR2 will only see traffic from BFR1 for which the ECMP hash
in BFR1 selected the first adjacency in the list of 2 adjacencies in BFR1 selected the first adjacency in the list of 2 adjacencies
given as parameters to the ECMP. It is link L11-to-BFR2. BFR2 given as parameters to the ECMP. It is link L11-to-BFR2. BFR2
performs again ECMP with two adjacencies on that subset of traffic performs again ECMP with two adjacencies on that subset of traffic
using the same seed1, and will therefore again select the first of using the same seed1, and will therefore again select the first of
its two adjacencies: L21-to-BFR4. And therefore L22 and BFR5 sees no its two adjacencies: L21-to-BFR4. And therefore L22 and BFR5 sees no
traffic. Likewise for L31 and BFR6. traffic. Likewise for L31 and BFR6.
This issue in BFR2/BFR3 is called polarization. It results from the This issue in BFR2/BFR3 is called polarization. It results from the
re-use of the same hash function across multiple consecutive hops in re-use of the same hash function across multiple consecutive hops in
topologies like these. To resolve this issue, the ECMP adjacency on topologies like these. To resolve this issue, the ECMP adjacency on
BFR1 can be set up with a different seed2 than the ECMP adjacencies BFR1 can be set up with a different seed2 than the ECMP adjacencies
on BFR2/BFR3. BFR2/BFR3 can use the same hash because packets will on BFR2/BFR3. BFR2/BFR3 can use the same hash because packets will
not sequentially pass across both of them. Therefore, they can also not sequentially pass across both of them. Therefore, they can also
use the same BP 0:7. use the same BP 0:7.
Note that ECMP solutions outside of BIER often hide the seed by auto- Note that ECMP solutions outside of BIER often hide the seed by auto-
selecting it from local entropy such as unique local or next-hop selecting it from local entropy such as unique local or next-hop
identifiers. The solutions chosen for BIER-TE to allow the BIER-TE identifiers. Allowing the BIER-TE Controller to explicitly set the
Controller to explicitly set the seed maximizes the ability of the seed gives the ability for it to control same/different path
BIER-TE Controller to choose the seed, independent of such seed selection across multiple consecutive ECMP hops.
source that the BIER-TE Controller may not be able to control well,
and even calculate optimized seeds for multi-hop cases.
4.8. Routed adjacencies 5.1.8. Forward_routed adjacencies
4.8.1. Reducing BitPositions 5.1.8.1. Reducing bit positions
Routed adjacencies can reduce the number of BitPositions required Forward_routed() adjacencies can reduce the number of bit positions
when the path steering requirement is not hop-by-hop explicit path required when the path steering requirement is not hop-by-hop
selection, but loose-hop selection. Routed adjacencies can also explicit path selection, but loose-hop selection. Forward_routed()
allow to operate BIER-TE across intermediate hop routers that do not adjacencies can also allow to operate BIER-TE across intermediate hop
support BIER-TE. routers that do not support BIER-TE.
............... ...............
...BFR1--... ...--L1-- BFR2... ...BFR1--... ...--L1-- BFR2...
... .Routers. ...--L2--/ ... .Routers. ...--L2--/
...BFR4--... ...------ BFR3... ...BFR4--... ...------ BFR3...
............... | ............... |
LO LO
Network Area 1 Network Area 1
Figure 14: Routed Adjacencies Example Figure 16: Forward_routed Adjacencies Example
Assume the requirement in the above picture is to explicitly steer Assume the requirement in Figure 16 is to explicitly steer traffic
traffic flows that have arrived at BFR1 or BFR4 via a shortest path flows that have arrived at BFR1 or BFR4 via a shortest path in the
in the routing underlay "Network Area 1" to one of the following routing underlay "Network Area 1" to one of the following three next
three next segments: (1) BFR2 via link L1, (2) BFR2 via link L2, (3) segments: (1) BFR2 via link L1, (2) BFR2 via link L2, or (3) via
via BFR3. BFR3.
To enable this, both BFR1 and BFR4 are set up with a forward_routed To enable this, both BFR1 and BFR4 are set up with a forward_routed
adjacency BitPosition towards an address of BFR2 on link L1, another adjacency bit position towards an address of BFR2 on link L1, another
forward_routed BitPosition towards an address of BFR2 on link L2 and forward_routed() bit position towards an address of BFR2 on link L2
a third forward_routed Bitposition towards a node address LO of BFR3. and a third forward_routed() bit position towards a node address LO
of BFR3.
4.8.2. Supporting nodes without BIER-TE 5.1.8.2. Supporting nodes without BIER-TE
Routed adjacencies also enable incremental deployment of BIER-TE. Forward_routed() adjacencies also enable incremental deployment of
Only the nodes through which BIER-TE traffic needs to be steered - BIER-TE. Only the nodes through which BIER-TE traffic needs to be
with or without replication - need to support BIER-TE. Where they steered - with or without replication - need to support BIER-TE.
are not directly connected to each other, forward_routed adjacencies Where they are not directly connected to each other, forward_routed
are used to pass over non BIER-TE enabled nodes. adjacencies are used to pass over non BIER-TE enabled nodes.
4.9. Reuse of BitPositions (without DNR) 5.1.9. Reuse of bit positions (without DNC)
BitPositions can be re-used across multiple BFR to minimize the bit positions can be re-used across multiple BFR to minimize the
number of BP needed. This happens when adjacencies on multiple BFR number of BP needed. This happens when adjacencies on multiple BFR
use the DNR flag as described above, but it can also be done for non- use the DNC flag as described above, but it can also be done for non-
DNR adjacencies. This section only discussses this non-DNR case. DNC adjacencies. This section only discusses this non-DNC case.
Because BP are reset after passing a BFR with an adjacency for that Because BP are cleared when passing a BFR with an adjacency for that
BP, reuse of BP across multiple BFR does not introduce any problems BP, reuse of BP across multiple BFR does not introduce any problems
with duplicates or loops that do not also exist when every adjacency with duplicates or loops that do not also exist when every adjacency
has a unique BP: Instead of setting one BP in a BitString that is has a unique BP. Instead, the challenge when reusing BP is whether
reused in N-adjacencies, one would get the same or worse results if it allows to still achieve the desired Tree Engineering goals.
each of these adjacencies had a unique BP and all of them where set
in the BitString. Instead, based on the case, BPs can be reused
without limitation, or they introduce fewer path steering choices, or
they do not work.
BP cannot be reused across two BFR that would need to be passed BP cannot be reused across two BFR that would need to be passed
sequentially for some path: The first BFR will reset the BP, so those sequentially for some path: The first BFR will clear the BP, so those
paths cannot be built. BP can be set across BFR that would (A) only paths cannot be built. BP can be set across BFR that would (A) only
occur across different paths or (B) across different branches of the occur across different paths or (B) across different branches of the
same tree. same tree.
An example of (A) was given in Figure 13, where BP 0:7, BP 0:8 and BP An example of (A) was given in Figure 15, where BP 0:7, BP 0:8 and BP
0:9 are each reused across multiple BFR because a single packet/path 0:9 are each reused across multiple BFRs because a single packet/path
would never be able to reach more than one BFR sharing the same BP. would never be able to reach more than one BFR sharing the same BP.
Assume the example was changed: BFR1 has no ECMP adjacency for BP Assume the example was changed: BFR1 has no ECMP adjacency for BP
0:6, but instead BP 0:5 with forward_connected to BFR2 and BP 0:6 0:6, but instead BP 0:5 with forward_connected() to BFR2 and BP 0:6
with forward_connected to BFR3. Packets with both BP 0:5 and BP 0:6 with forward_connected() to BFR3. Packets with both BP 0:5 and BP
would now be able to reach both BFR2 and BFR3 and the still existing 0:6 would now be able to reach both BFR2 and BFR3 and the still
re-use of BP 0:7 between BFR2 and BFR3 is a case of (B) where reuse existing re-use of BP 0:7 between BFR2 and BFR3 is a case of (B)
of BP is perfect because it does not limit the set of useful path where reuse of BP is perfect because it does not limit the set of
choices: useful path choices:
If instead of reusing BP 0:7, BFR3 used a separate BP 0:10 for its If instead of reusing BP 0:7, BFR3 used a separate BP 0:10 for its
ECMP adjacency, no useful additional path steering options would be ECMP adjacency, no useful additional path steering options would be
enabled. If duplicates at BFR10 where undesirable, this would be enabled. If duplicates at BFR10 where undesirable, this would be
done by not setting BP 0:5 and BP 0:6 for the same packet. If the done by not setting BP 0:5 and BP 0:6 for the same packet. If the
duplicates where desirable (e.g.: resilient transmission), the duplicates where desirable (e.g.: resilient transmission), the
additional BP 0:10 would also not render additional value. additional BP 0:10 would also not render additional value.
area1
BFR1a BFR1b
/ \
....................................
. Core .
....................................
| / \ / \ |
BFR2a BFR2b BFR3a BFR3b BFR6a BFR6b
/-------\ /---------\ /--------\
| area2 | | area3 | ... | area6 |
| ring | | ring | | ring |
\-------/ \---------/ \--------/
more BFR more BFR more BFR
Figure 17: Reuse of BP
Reuse may also save BPs in larger topologies. Consider the topology Reuse may also save BPs in larger topologies. Consider the topology
shown in Figure 17, but only the following explanations: A BFIR/ shown in Figure 20. A BFIR/sender (e.g.: video headend) is attached
sender (e.g.: video headend) is attached to area 1, and area 2...6 to area 1, and area 2...6 contain receivers/BFER. Assume each area
contain receivers/BFER. Assume each area had a distribution ring, had a distribution ring, each with two BPs to indicate the direction
each with two BPs to indicate the direction (as explained in before). (as explained before). These two BPs could be reused across the 5
These two BPs could be reused across the 5 areas. Packets would be areas. Packets would be replicated through other BPs for the Core to
replicated through other BPs to the desired subset of areas, and once the desired subset of areas, and once a packet copy reaches the ring
a packet copy reaches the ring of the area, the two ring BPs come of the area, the two ring BPs come into play. This reuse is a case
into play. This reuse is a case of (B), but it limits the topology of (B), but it limits the topology choices: Packets can only flow
choices: Packets can only flow around the same direction in the rings around the same direction in the rings of all areas. This may or may
of all areas. This may or may not be acceptable based on the desired not be acceptable based on the desired path steering options: If
path steering options: If resilient transmission is the path resilient transmission is the path engineering goal, then it is
engineering goal, then it is likely a good optimization, if the likely a good optimization, if the bandwidth of each ring was to be
bandwidth of each ring was to be optimized separately, it would not optimized separately, it would not be a good limitation.
be a good limitation.
4.10. Summary of BP optimizations 5.1.10. Summary of BP optimizations
This section reviewed a range of techniques by which a BIER-TE This section reviewed a range of techniques by which a BIER-TE
Controller can create a BIER-TE topology in a way that minimizes the Controller can create a BIER-TE topology in a way that minimizes the
number of necessary BPs. number of necessary BPs.
Without any optimization, a BIER-TE Controller would attempt to map Without any optimization, a BIER-TE Controller would attempt to map
the network subnet topology 1:1 into the BIER-TE topology and every the network subnet topology 1:1 into the BIER-TE topology and every
subnet adjacent neighbor requires a forward_connected BP and every subnet adjacent neighbor requires a forward_connected() BP and every
BFER requires a local_decap BP. BFER requires a local_decap() BP.
The optimizations described are then as follows: The optimizations described are then as follows:
o P2p links require only one BP (Section 4.1). o P2P links require only one BP (Section 5.1.1).
o All leaf-BFER can share a single local_decap BP (Section 4.3). o All leaf-BFER can share a single local_decap() BP (Section 5.1.3).
o A LAN with N BFR needs at most N BP (one for each BFR). It only o A LAN with N BFR needs at most N BP (one for each BFR). It only
needs one BP for all those BFR tha are not redundanty connected to needs one BP for all those BFR that are not redundantly connected
multiple LANs (Section 4.4). to multiple LANs (Section 5.1.4).
o A hub with p2p connections to multiple non-leaf-BFER spokes can o A hub with p2p connections to multiple non-leaf-BFER spokes can
share one BP to all spokes if traffic can be flooded to all share one BP to all spokes if traffic can be flooded to all
spokes, e.g.: because of no bandwidth concerns or dense receiver spokes, e.g.: because of no bandwidth concerns or dense receiver
sets (Section 4.5). sets (Section 5.1.5).
o Rings of BFR can be built with just two BP (one for each o Rings of BFR can be built with just two BP (one for each
direction) except for BFR with multiple ring connections - similar direction) except for BFR with multiple ring connections - similar
to LANs (Section 4.6). to LANs (Section 5.1.6).
o ECMP adjacencies to N neighbors can replace N BP with 1 BP. o ECMP adjacencies to N neighbors can replace N BP with 1 BP.
Multihop ECMP can avoid polarization through different seeds of Multihop ECMP can avoid polarization through different seeds of
the ECMP algorithm (Section 4.7). the ECMP algorithm (Section 5.1.7).
o Routed adjacencies allow to "tunnel" across non-BIER-TE capable o Forward_routed() adjacencies allow to "tunnel" across non-BIER-TE
routers and across BIER-TE capable routers where no traffic- capable routers and across BIER-TE capable routers where no
steering or replications are required (Section 4.8). traffic-steering or replications are required (Section 5.1.8).
o BP can generally be reused across nodes that do not need to be o BP can generally be reused across nodes that do not need to be
consecutive in paths, but depending on scenario, this may limit consecutive in paths, but depending on scenario, this may limit
the feasible path steering options (Section 4.9). the feasible path steering options (Section 5.1.9).
Note that the described list of optimizations is not exhaustive. Note that the described list of optimizations is not exhaustive.
Especially when the set of required path steering choices is limited Especially when the set of required path steering choices is limited
and the set of possible subsets of BFER that should be able to and the set of possible subsets of BFERs that should be able to
receive traffic is limited, further optimizations of BP are possible. receive traffic is limited, further optimizations of BP are possible.
The hub & spoke optimization is a simple example of such traffic The hub & spoke optimization is a simple example of such traffic
pattern dependent optimizations. pattern dependent optimizations.
5. Avoiding duplicates and loops 5.2. Avoiding duplicates and loops
5.1. Loops 5.2.1. Loops
Whenever BIER-TE creates a copy of a packet, the BitString of that Whenever BIER-TE creates a copy of a packet, the BitString of that
copy will have all BitPositions cleared that are associated with copy will have all bit positions cleared that are associated with
adjacencies on the BFR. This inhibits looping of packets. The only adjacencies on the BFR. This inhibits looping of packets. The only
exception are adjacencies with DNR set. exception are adjacencies with DNC set.
With DNR set, looping can happen. Consider in the ring picture that v v
link L4 from BFR3 is plugged into the L1 interface of BFRa. This | |
creates a loop where the rings clockwise BitPosition is never reset L1 | L2 | L3
for copies of the packets traveling clockwise around the ring. /-------- BFRa ---- BFRb ---------------------\
| . |
| ...... Wrong link wiring |
| . |
\- BFR1 - BFR2 BFR3 - ... - BFR29 - BFR30 -/
| | L4 | |
p33| p15|
BFRd BFRc
Figure 18: Miswired Ring Example
With DNC set, looping can happen. Consider in Figure 18 that link L4
from BFR3 is (inadvertently) plugged into the L1 interface of BFRa
(instead of BFR2). This creates a loop where the rings clockwise bit
position is never cleared for copies of the packets traveling
clockwise around the ring.
To inhibit looping in the face of such physical misconfiguration, To inhibit looping in the face of such physical misconfiguration,
only forward_connected adjacencies are permitted to have DNR set, and only forward_connected() adjacencies are permitted to have DNC set,
the link layer port unique unicast destination address of the and the link layer port unique unicast destination address of the
adjacency (e.g. MAC address) protects against closing the loop. adjacency (e.g. MAC address) protects against closing the loop.
Link layers without port unique link layer addresses should not be Link layers without port unique link layer addresses should not be
used with the DNR flag set. used with the DNC flag set.
5.2. Duplicates 5.2.2. Duplicates
BFIR1
/ \
/ p2 \ p3
BFR2 BFR3
\ p4 / p5
\ /
BFER4
Duplicates happen when the topology of the BitString is not a tree Figure 19: Duplicates Example
but redundantly connecting BFRs with each other. The BIER-TE
Controller must therefore ensure to only create BitStrings that are Duplicates happen when the graph expressed by a BitString is not a
trees in the topology. tree but redundantly connecting BFRs with each other. In Figure 19,
a BitString of p2,p3,p4,p5 would result in duplicate packets to
arrive on BFER4. The BIER-TE Controller must therefore ensure to
only create BitStrings that are trees.
When links are incorrectly physically re-connected before the BIER-TE When links are incorrectly physically re-connected before the BIER-TE
Controller updates BitStrings in BFIRs, duplicates can happen. Like Controller updates BitStrings in BFIRs, duplicates can happen. Like
loops, these can be inhibited by link layer addressing in loops, these can be inhibited by link layer addressing in
forward_connected adjacencies. forward_connected() adjacencies.
If interface or loopback addresses used in forward_routed adjacencies
are moved from one BFR to another, duplicates can equally happen.
Such re-addressing operations must be coordinated with the BIER-TE
Controller.
6. BIER-TE Forwarding Pseudocode
The following simplified pseudocode for BIER-TE forwarding is using
BIER forwarding pseudocode of [RFC8279], section 6.5 with the one
modification necessary to support basic BIER-TE forwarding. Like the
BIER pseudo forwarding code, for simplicity it does hide the details
of the adjacency processing inside PacketSend() which can be
forward_connected, forward_routed or local_decap.
void ForwardBitMaskPacket_withTE (Packet)
{
SI=GetPacketSI(Packet);
Offset=SI*BitStringLength;
for (Index = GetFirstBitPosition(Packet->BitString); Index ;
Index = GetNextBitPosition(Packet->BitString, Index)) {
F-BM = BIFT[Index+Offset]->F-BM;
if (!F-BM) continue;
BFR-NBR = BIFT[Index+Offset]->BFR-NBR;
PacketCopy = Copy(Packet);
PacketCopy->BitString &= F-BM; [2]
PacketSend(PacketCopy, BFR-NBR);
// The following must not be done for BIER-TE:
// Packet->BitString &= ~F-BM; [1]
}
}
Figure 15: Simplified BIER-TE Forwarding Pseudocode
The difference is that in BIER-TE, step [1] must not be performed,
but is replaced with [2] (when the forwarding plane algorithm is
implemented verbatim as shown above).
In BIER, the F-BM of a BP has all BP set that are meant to be
forwarded via the same neighbor. It is used to reset those BP in the
packet after the first copy to this neighbor has been made to inhibit
multiple copies to the same neighbor.
In BIER-TE, the F-BM of a particular BP with an adjacency is the list
of all BPs with an adjacency on this BFR except the particular BP
itself if it has an adjacency with the DNR bit set. The F-BM is used
to reset the F-BM BPs before creating copies.
In BIER, the order of BPs impacts the result of forwarding because of
[1]. In BIER-TE, forwarding is not impacted by the order of BPs. It
is therefore possible to further optimize forwarding than in BIER.
For example, BIER-TE forwarding can be parallelized such that a
parallel instance (such as an egres linecard) can process any subset
of BPs without any considerations for the other BPs - and without any
prior, cross-BP shared processing.
The above simplified pseudocode is elaborated further as follows:
o This pseudocode eliminates per-bit F-BM, therefore reducing state
by BitStringLength^2*SI and eliminating the need for per-packet-
copy masking operation except for adjacencies with DNR flag set:
* AdjacentBits[SI] are bits with a non-empty list of adjacencies.
This can be computed whenever the BIER-TE Controller updates
the adjacencies.
* Only the AdjacentBits need to be examined in the loop for
packet copies.
* The packets BitString is masked with those AdjacentBits on
ingress to avoid packets looping.
o The code loops over the adjacencies because there may be more than
one adjacency for a bit.
o When an adjacency has the DNR bit, the bit is set in the packet
copy (to save bits in rings for example).
o The ECMP adjacency is shown. Its parameters are a
ListOfAdjacencies from which one is picked.
o The forward_local, forward_routed, local_decap adjacencies are
shown with their parameters.
void ForwardBitMaskPacket_withTE (Packet)
{
SI=GetPacketSI(Packet);
Offset=SI*BitStringLength;
AdjacentBitstring = Packet->BitString &= ~AdjacentBits[SI];
Packet->BitString &= AdjacentBits[SI];
for (Index = GetFirstBitPosition(AdjacentBits); Index ;
Index = GetNextBitPosition(AdjacentBits, Index)) {
foreach adjacency BIFT[Index+Offset] {
if(adjacency == ECMP(ListOfAdjacencies, seed) ) {
I = ECMP_hash(sizeof(ListOfAdjacencies),
Packet->Entropy, seed);
adjacency = ListOfAdjacencies[I];
}
PacketCopy = Copy(Packet);
switch(adjacency) {
case forward_connected(interface,neighbor,DNR):
if(DNR)
PacketCopy->BitString |= 2<<(Index-1);
SendToL2Unicast(PacketCopy,interface,neighbor);
case forward_routed({VRF},neighbor):
SendToL3(PacketCopy,{VRF,}l3-neighbor);
case local_decap({VRF},neighbor):
DecapBierHeader(PacketCopy);
PassTo(PacketCopy,{VRF,}Packet->NextProto);
}
}
}
}
Figure 16: BIER-TE Forwarding Pseudocode If interface or loopback addresses used in forward_routed()
adjacencies are moved from one BFR to another, duplicates can equally
happen. Such re-addressing operations must be coordinated with the
BIER-TE Controller.
7. Managing SI, subdomains and BFR-ids 5.3. Managing SI, sub-domains and BFR-ids
When the number of bits required to represent the necessary hops in When the number of bits required to represent the necessary hops in
the topology and BFER exceeds the supported bitstring length, the topology and BFER exceeds the supported BitStringLength (BSL),
multiple SI and/or subdomains must be used. This section discusses multiple SIs and/or sub-domains must be used. This section discusses
how. how.
BIER-TE forwarding does not require the concept of BFR-id, but BIER-TE forwarding does not require the concept of BFR-id, but
routing underlay, flow overlay and BIER headers may. This section routing underlay, flow overlay and BIER headers may. This section
also discusses how BFR-ids can be assigned to BFIR/BFER for BIER-TE. also discusses how BFR-ids can be assigned to BFIR/BFER for BIER-TE.
7.1. Why SI and sub-domains 5.3.1. Why SI and sub-domains
For BIER and BIER-TE forwarding, the most important result of using For BIER and BIER-TE forwarding, the most important result of using
multiple SI and/or subdomains is the same: Packets that need to be multiple SI and/or sub-domains is the same: Packets that need to be
sent to BFER in different SI or subdomains require different BIER sent to BFERs in different SIs or sub-domains require different BIER
packets: each one with a bitstring for a different (SI,subdomain) packets: each one with a BitString for a different (SI,sub-domain)
combination. Each such bitstring uses one bitstring length sized SI combination. Each such BitString uses one BSL sized SI block in the
block in the BIFT of the subdomain. We call this a BIFT:SI (block). BIFT of the sub-domain. We call this a BIFT:SI (block).
For BIER and BIER-TE forwarding itself there is also no difference For BIER and BIER-TE forwarding themselves there is also no
whether different SI and/or sub-domains are chosen, but SI and difference whether different SIs and/or sub-domains are chosen, but
subdomain have different purposes in the BIER architecture shared by SI and sub-domain have different purposes in the BIER architecture
BIER-TE. This impacts how operators are managing them and how shared by BIER-TE. This impacts how operators are managing them and
especially flow overlays will likely use them. how especially flow overlays will likely use them.
By default, every possible BFIR/BFER in a BIER network would likely By default, every possible BFIR/BFER in a BIER network would likely
be given a BFR-id in subdomain 0 (unless there are > 64k BFIR/BFER). be given a BFR-id in sub-domain 0 (unless there are > 64k BFIR/BFER).
If there are different flow services (or service instances) requiring If there are different flow services (or service instances) requiring
replication to different subsets of BFER, then it will likely not be replication to different subsets of BFERs, then it will likely not be
possible to achieve the best replication efficiency for all of these possible to achieve the best replication efficiency for all of these
service instances via subdomain 0. Ideal replication efficiency for service instances via sub-domain 0. Ideal replication efficiency for
N BFER exists in a subdomain if they are split over not more than N BFER exists in a sub-domain if they are split over not more than
ceiling(N/bitstring-length) SI. ceiling(N/BitStringLength) SI.
If service instances justify additional BIER:SI state in the network, If service instances justify additional BIER:SI state in the network,
additional subdomains will be used: BFIR/BFER are assigned BFIR-id in additional sub-domains will be used: BFIR/BFER are assigned BFR-id in
those subdomains and each service instance is configured to use the those sub-domains and each service instance is configured to use the
most appropriate subdomain. This results in improved replication most appropriate sub-domain. This results in improved replication
efficiency for different services. efficiency for different services.
Even if creation of subdomains and assignment of BFR-id to BFIR/BFER Even if creation of sub-domains and assignment of BFR-id to BFIR/BFER
in those subdomains is automated, it is not expected that individual in those sub-domains is automated, it is not expected that individual
service instances can deal with BFER in different subdomains. A service instances can deal with BFER in different sub-domains. A
service instance may only support configuration of a single subdomain service instance may only support configuration of a single sub-
it should rely on. domain it should rely on.
To be able to easily reuse (and modify as little as possible) To be able to easily reuse (and modify as little as possible)
existing BIER procedures including flow-overlay and routing underlay, existing BIER procedures including flow-overlay and routing underlay,
when BIER-TE forwarding is added, we therefore reuse SI and subdomain when BIER-TE forwarding is added, we therefore reuse SI and sub-
logically in the same way as they are used in BIER: All necessary domain logically in the same way as they are used in BIER: All
BFIR/BFER for a service use a single BIER-TE BIFT and are split necessary BFIR/BFER for a service use a single BIER-TE BIFT and are
across as many SI as necessary (see below). Different services may split across as many SIs as necessary (see Section 5.3.2). Different
use different subdomains that primarily exist to provide more services may use different sub-domains that primarily exist to
efficient replication (and for BIER-TE desirable path steering) for provide more efficient replication (and for BIER-TE desirable path
different subsets of BFIR/BFER. steering) for different subsets of BFIR/BFER.
7.2. Bit assignment comparison BIER and BIER-TE 5.3.2. Assigning bits for the BIER-TE topology
In BIER, bitstrings only need to carry bits for BFER, which leads to In BIER, BitStrings only need to carry bits for BFERs, which leads to
the model that BFR-ids map 1:1 to each bit in a bitstring. the model that BFR-ids map 1:1 to each bit in a BitString.
In BIER-TE, bitstrings need to carry bits to indicate not only the In BIER-TE, BitStrings need to carry bits to indicate not only the
receiving BFER but also the intermediate hops/links across which the receiving BFER but also the intermediate hops/links across which the
packet must be sent. The maximum number of BFER that can be packet must be sent. The maximum number of BFER that can be
supported in a single bitstring or BIFT:SI depends on the number of supported in a single BitString or BIFT:SI depends on the number of
bits necessary to represent the desired topology between them. bits necessary to represent the desired topology between them.
"Desired" topology because it depends on the physical topology, and "Desired" topology because it depends on the physical topology, and
on the desire of the operator to allow for explicit path steeering on the desire of the operator to allow for explicit path steering
across every single hop (which requires more bits), or reducing the across every single hop (which requires more bits), or reducing the
number of required bits by exploiting optimizations such as unicast number of required bits by exploiting optimizations such as unicast
(forward_route), ECMP or flood (DNR) over "uninteresting" sub-parts (forward_routed), ECMP or flood (DNC) over "uninteresting" sub-parts
of the topology - e.g. parts where different trees do not need to of the topology - e.g. parts where different trees do not need to
take different paths due to path steering reasons. take different paths due to path steering reasons.
The total number of bits to describe the topology vs. the BFER in a The total number of bits to describe the topology vs. the number of
BIFT:SI can range widely based on the size of the topology and the BFERs in a BIFT:SI can range widely based on the size of the topology
amount of alternative paths in it. The higher the percentage, the and the amount of alternative paths in it. The higher the percentage
higher the likelihood, that those topology bits are not just BIER-TE of non-BFER bits, the higher the likelihood, that those topology bits
overhead without additional benefit, but instead that they will allow are not just BIER-TE overhead without additional benefit, but instead
to express desirable path steering alternatives. that they will allow to express desirable path steering alternatives.
7.3. Using BFR-id with BIER-TE 5.3.3. Assigning BFR-id with BIER-TE
Because there is no 1:1 mapping between bits in the bitstring and BIER-TE forwarding does not use the BFR-id, not does it require for
BFER, BIER-TE cannot simply rely on the BIER 1:1 mapping between bits the BFR-id field of the BIER header to be set to a particular value.
in a bitstring and BFR-id. However, other parts of a BIER-TE deployment may need a BFR-id,
specifically overlay signaling, and in that case BFR need to also
have BFR-ids for BIER-TE SDs.
In BIER, automatic schemes could assign all possible BFR-ids For example, for BIER overlay signaling, BFIR need to have a BFR-id,
sequentially to BFERs. This will not work in BIER-TE. In BIER-TE, because this BFIR BFR-id is carried in the BFR-id field of the BIER
the operator or BIER-TE Controller has to determine a BFR-id for each header to indicate to the overlay signaling on the receiving BFER
BFER in each required subdomain. The BFR-id may or may not have a which BFIR originated the packet.
relationship with a bit in the bitstring. Suggestions are detailed
below. Once determined, the BFR-id can then be configured on the
BFER and used by flow overlay, routing underlay and the BIER header
almost the same as the BFR-id in BIER.
The one exception are application/flow-overlays that automatically In BIER, BFR-id = BSL * SI + BP, such that the SI and BP of a BFER
calculate the bitstring(s) of BIER packets by converting BFR-id to can be calculated from the BFR-id and vice versa. This also means
bits. In BIER-TE, this operation can be done in two ways: that every BFR with a BFR-id has a reserved BP in an SI, even if that
is not necessary for BIER forwarding, because the BFR may never be a
BFER but only a BFIR.
"Independent branches": For a given application or (set of) trees, In BIER-TE, for a non-leaf BFER, there is usually a single BP for
the branches from a BFIR to every BFER are independent of the that BFER with a local_decap() adjacency on the BFER. The BFR-id for
branches to any other BFER. For example, shortest part trees have such a BFER can therefore equally be determined as in BIER: BFR-id =
SI * BitStringLength + BP.
As explained in Section 5.1.3, leaf BFERs do not need such a unique
local_decap() adjacency, likewise, BFIR who are not also BFER may not
have a unique local_decap() adjacency either. For all those BFIR and
(leaf) BFER, the controller needs to determine unique BFR-ids that do
not collide with the BFR-ids derived from the non-leaf BFER
local_decap() BPs.
While this document defines no requirements how to allocate such BFR-
id, a simple option is to derive it from the (SI,BP) of an adjacency
that is unique to the BFR in question. For a BFIR this can be he
first adjacency only populated on this BFIR, for a leaf-BFER, this
could be the first BP with an adjacency towards that BFER.
5.3.4. Mapping from BFR to BitStrings with BIER-TE
In BIER, applications of the flow overlay on a BFIR can calculate the
(SI,BP) of a BFER from the BFR-id of the BFER and can therefore
easily determine the BitStrings for a BIER packet to a set of BFER
with known BFR-ids.
In BIER-TE this mapping needs to be equally supported for flow
overlays. This section outlines two core options, based on how
"complex" the Tree Engineering is that the BIER-TE controller
performs for a particular application.
"Independent branches": For a given flow overlay instance, the
branches from a BFIR to every BFER are calculated by the BIER-TE
controller to be independent of the branches to any other BFER.
Shortest path trees are the most common examples of trees with
independent branches. independent branches.
"Interdependent branches": When a BFER is added or deleted from a "Interdependent branches": When a BFER is added or deleted from a
particular distribution tree, branches to other BFER still in the particular distribution tree, the BIER-TE controller has to
tree may need to change. Steiner tree are examples of dependent recalculate the branches to other BFER, because they may need to
branch trees. change. Steiner trees are examples of interdependent branch trees.
If "independent branches" are sufficient, the BIER-TE Controller can If "independent branches" are used, the BIER-TE Controller can signal
provide to such applications for every BFR-id a SI:bitstring with the to the BFIR flow overlay for every BFER an SI:BitString that
BIER-TE bits for the branch towards that BFER. The application can represents the branch to that BFER. The flow overlay on the BIFR can
then independently calculate the SI:bitstring for all desired BFER by then independently of the controller calculate the SI:BitString for
OR'ing their bitstrings. all desired BFER by OR'ing their BitStrings. This allows for flow
overlay applications to operate independently from the controller
whenever it needs to determine which subset of BFERs need to receive
a particular packet.
If "interdependent branches" are required, the application could call If "interdependent branches" are required, the application would need
a BIER-TE Controller API with the list of required BFER-id and get to inquire the SI:BitString for a given set of BFER whenever the set
the required bitstring back. Whenever the set of BFER-id changes, changes.
this is repeated.
Note that in either case (unlike in BIER), the bits in BIER-TE may Note that in either case (unlike in BIER), the bits may need to
need to change upon link/node failure/recovery, network expansion and change upon link/node failure/recovery, network expansion and network
network resource consumption by other traffic as part of traffic resource consumption by other traffic as part of traffic engineering
engineering goals (e.g.: re-optimization of lower priority traffic goals (e.g.: re-optimization of lower priority traffic flows).
flows). Interactions between such BFIR applications and the BIER-TE Interactions between such BFIR applications and the BIER-TE
Controller do therefore need to support dynamic updates to the Controller do therefore need to support dynamic updates to the
bitstrings. SI:BitStrings.
7.4. Assigning BFR-ids for BIER-TE
For a non-leaf BFER, there is usually a single bit k for that BFER
with a local_decap() adjacency on the BFER. The BFR-id for such a
BFER is therefore most easily the one it would have in BIER: SI *
bitstring-length + k.
As explained earlier in the document, leaf BFERs do not need such a Communications between BFIR flow overlay and BIER-TE controller
separate bit because the fact alone that the BIER-TE packet is requires some way to identify BFER. If BFR-ids are used in the
forwarded to the leaf BFER indicates that the BFER should decapsulate deployment, as outlined in Section 5.3.3, then those are the natural
it. Such a BFER will have one or more bits for the links leading BFR identifier. If BFR-ids are not used, then any other unique
only to it. The BFR-id could therefore most easily be the BFR-id identifier, such as the BFR-prefix of the BFR as of [RFC8279] could
derived from the lowest bit for those links. be used.
These two rules are only recommendations for the operator or BIER-TE 5.3.5. Assigning BFR-ids for BIER-TE
Controller assigning the BFR-ids. Any allocation scheme can be used,
the BFR-ids just need to be unique across BFRs in each subdomain.
It is not currently determined if a single subdomain could or should It is not currently determined if a single sub-domain could or should
be allowed to forward both BIER and BIER-TE packets. If this should be allowed to forward both BIER and BIER-TE packets. If this should
be supported, there are two options: be supported, there are two options:
A. BIER and BIER-TE have different BFR-id in the same subdomain. A. BIER and BIER-TE have different BFR-id in the same sub-domain.
This allows higher replication efficiency for BIER because their BFR- This allows higher replication efficiency for BIER because their BFR-
id can be assigned sequentially, while the bitstrings for BIER-TE id can be assigned sequentially, while the BitStrings for BIER-TE
will have also the additional bits for the topology. There is no will have also the additional bits for the topology. There is no
relationship between a BFR BIER BFR-id and BIER-TE BFR-id. relationship between a BFR BIER BFR-id and BIER-TE BFR-id.
B. BIER and BIER-TE share the same BFR-id. The BFR-id are assigned B. BIER and BIER-TE share the same BFR-id. The BFR-ids are assigned
as explained above for BIER-TE and simply reused for BIER. The as explained above for BIER-TE and simply reused for BIER. The
replication efficiency for BIER will be as low as that for BIER-TE in replication efficiency for BIER will be as low as that for BIER-TE in
this approach. Depending on topology, only the same 20%..80% of bits this approach. Depending on topology, only the same 20%..80% of bits
as possible for BIER-TE can be used for BIER. as possible for BIER-TE can be used for BIER.
7.5. Example bit allocations 5.3.6. Example bit allocations
7.5.1. With BIER 5.3.6.1. With BIER
Consider a network setup with a bitstring length of 256 for a network Consider a network setup with a BSL of 256 for a network topology as
topology as shown in the picture below. The network has 6 areas, shown in Figure 20. The network has 6 areas, each with 170 BFRs,
each with ca. 170 BFR, connecting via a core with some larger (core) connecting via a core with 4 (core) BFRs. To address all BFERs with
BFR. To address all BFER with BIER, 4 SI are required. To send a BIER, 4 SIs are required. To send a BIER packet to all BFER in the
BIER packet to all BFER in the network, 4 copies need to be sent by network, 4 copies need to be sent by the BFIR. On the BFIR it does
the BFIR. On the BFIR it does not make a difference how the BFR-id not make a difference how the BFR-ids are allocated to BFER in the
are allocated to BFER in the network, but for efficiency further down network, but for efficiency further down in the network it does make
in the network it does make a difference. a difference.
area1 area2 area3 area1 area2 area3
BFR1a BFR1b BFR2a BFR2b BFR3a BFR3b BFR1a BFR1b BFR2a BFR2b BFR3a BFR3b
| \ / \ / | | \ / \ / |
................................ ................................
. Core . . Core .
................................ ................................
| / \ / \ | | / \ / \ |
BFR4a BFR4b BFR5a BFR5b BFR6a BFR6b BFR4a BFR4b BFR5a BFR5b BFR6a BFR6b
area4 area5 area6 area4 area5 area6
Figure 17: Scaling BIER-TE bits by reuse Figure 20: Scaling BIER-TE bits by reuse
With random allocation of BFR-id to BFER, each receiving area would With random allocation of BFR-id to BFER, each receiving area would
(most likely) have to receive all 4 copies of the BIER packet because (most likely) have to receive all 4 copies of the BIER packet because
there would be BFR-id for each of the 4 SI in each of the areas. there would be BFR-id for each of the 4 SIs in each of the areas.
Only further towards each BFER would this duplication subside - when Only further towards each BFER would this duplication subside - when
each of the 4 trees runs out of branches. each of the 4 trees runs out of branches.
If BFR-id are allocated intelligently, then all the BFER in an area If BFR-ids are allocated intelligently, then all the BFER in an area
would be given BFR-id with as few as possible different SI. Each would be given BFR-id with as few as possible different SIs. Each
area would only have to forward one or two packets instead of 4. area would only have to forward one or two packets instead of 4.
Given how networks can grow over time, replication efficiency in an Given how networks can grow over time, replication efficiency in an
area will also easily go down over time when BFR-id are network wide area will also easily go down over time when BFR-ids are network wide
allocated sequentially over time. An area that initially only has allocated sequentially over time. An area that initially only has
BFR-id in one SI might end up with many SI over a longer period of BFR-id in one SI might end up with many SIs over a longer period of
growth. Allocating SIs to areas with initially sufficiently many growth. Allocating SIs to areas with initially sufficiently many
spare bits for growths can help to alleviate this issue. Or renumber spare bits for growths can help to alleviate this issue. Or renumber
BFR-id after network expansion. In this example one may consider to BFERs after network expansion. In this example one may consider to
use 6 SI and assign one to each area. use 6 SIs and assign one to each area.
This example shows that intelligent BFR-id allocation within at least This example shows that intelligent BFR-id allocation within at least
subdomain 0 can even be helpful or even necessary in BIER. sub-domain 0 can even be helpful or even necessary in BIER.
7.5.2. With BIER-TE 5.3.6.2. With BIER-TE
In BIER-TE one needs to determine a subset of the physical topology In BIER-TE one needs to determine a subset of the physical topology
and attached BFER so that the "desired" representation of this and attached BFERs so that the "desired" representation of this
topology and the BFER fit into a single bitstring. This process topology and the BFER fit into a single BitString. This process
needs to be repeated until the whole topology is covered. needs to be repeated until the whole topology is covered.
Once bits/SIs are assigned to topology and BFER, BFR-id is just a Once bits/SIs are assigned to topology and BFERs, BFR-id is just a
derived set of identifiers from the operator/BIER-TE Controller as derived set of identifiers from the operator/BIER-TE Controller as
explained above. explained above.
Every time that different sub-topologies have overlap, bits need to Every time that different sub-topologies have overlap, bits need to
be repeated across the bitstrings, increasing the overall amount of be repeated across the BitStrings, increasing the overall amount of
bits required across all bitstring/SIs. In the worst case, random bits required across all BitString/SIs. In the worst case, random
subsets of BFER are assigned to different SI. This is much worse subsets of BFERs are assigned to different SIs. This is much worse
than in BIER because it not only reduces replication efficiency with than in BIER because it not only reduces replication efficiency with
the same number of overall bits, but even further - because more bits the same number of overall bits, but even further - because more bits
are required due to duplication of bits for topology across multiple are required due to duplication of bits for topology across multiple
SI. Intelligent BFER to SI assignment and selecting specific SIs. Intelligent BFER to SI assignment and selecting specific
"desired" subtopologies can minimize this problem. "desired" subtopologies can minimize this problem.
To set up BIER-TE efficiently for above topology, the following bit To set up BIER-TE efficiently for the above topology, the following
allocation methods can be used. This method can easily be expanded bit allocation method can be used. This method can easily be
to other, similarly structured larger topologies. expanded to other, similarly structured larger topologies.
Each area is allocated one or more SI depending on the number of Each area is allocated one or more SIs depending on the number of
future expected BFER and number of bits required for the topology in future expected BFERs and number of bits required for the topology in
the area. In this example, 6 SI, one per area. the area. In this example, 6 SIs, one per area.
In addition, we use 4 bits in each SI: bia, bib, bea, beb: bit In addition, we use 4 bits in each SI: bia, bib, bea, beb: bit
ingress a, bit ingress b, bit egress a, bit egress b. These bits ingress a, bit ingress b, bit egress a, bit egress b. These bits
will be used to pass BIER packets from any BFIR via any combination will be used to pass BIER packets from any BFIR via any combination
of ingress area a/b BFR and egress area a/b BFR into a specific of ingress area a/b BFR and egress area a/b BFR into a specific
target area. These bits are then set up with the right target area. These bits are then set up with the right
forward_routed adjacencies on the BFIR and area edge BFR: forward_routed() adjacencies on the BFIR and area edge BFR:
On all BFIR in an area j, bia in each BIFT:SI is populated with the On all BFIRs in an area j|j=2...6, bia in each BIFT:SI is populated
same forward_routed(BFRja), and bib with forward_routed(BFRjb). On with the same forward_routed(BFRja), and bib with
all area edge BFR, bea in BIFT:SI=k is populated with forward_routed(BFRjb). On all area edge BFR, bea in
forward_routed(BFRka) and beb in BIFT:SI=k with BIFT:SI=k|k=2...6 is populated with forward_routed(BFRka) and beb in
forward_routed(BFRkb). BIFT:SI=k with forward_routed(BFRkb).
For BIER-TE forwarding of a packet to some subset of BFER across all For BIER-TE forwarding of a packet to a subset of BFERs across all
areas, a BFIR would create at most 6 copies, with SI=1...SI=6, In areas, a BFIR would create at most 6 copies, with SI=1...SI=6, In
each packet, the bits indicate bits for topology and BFER in that each packet, the bits indicate bits for topology and BFER in that
topology plus the four bits to indicate whether to pass this packet topology plus the four bits to indicate whether to pass this packet
via the ingress area a or b border BFR and the egress area a or b via the ingress area a or b border BFR and the egress area a or b
border BFR, therefore allowing path steering for those two "unicast" border BFR, therefore allowing path steering for those two "unicast"
legs: 1) BFIR to ingress are edge and 2) core to egress area edge. legs: 1) BFIR to ingress are edge and 2) core to egress area edge.
Replication only happens inside the egress areas. For BFER in the Replication only happens inside the egress areas. For BFER in the
same area as in the BFIR, these four bits are not used. same area as in the BFIR, these four bits are not used.
7.6. Summary 5.3.7. Summary
BIER-TE can like BIER support multiple SI within a sub-domain to BIER-TE can, like BIER, support multiple SIs within a sub-domain to
allow re-using the concept of BFR-id and therefore minimize BIER-TE allow re-using the concept of BFR-id and therefore minimize BIER-TE
specific functions in underlay routing, flow overlay methods and BIER specific functions in any possible BIER layer control plane used in
headers. conjunction with BIER-TE, flow overlay methods and BIER headers.
The number of BFIR/BFER possible in a subdomain is smaller than in The number of BFIR/BFER possible in a sub-domain is smaller than in
BIER because BIER-TE uses additional bits for topology. BIER because BIER-TE uses additional bits for topology.
Subdomains can in BIER-TE be used like in BIER to create more Sub-domains (SDs) in BIER-TE can be used like in BIER to create more
efficient replication to known subsets of BFER. efficient replication to known subsets of BFERs.
Assigning bits for BFER intelligently into the right SI is more Assigning bits for BFERs intelligently into the right SI is more
important in BIER-TE than in BIER because of replication efficiency important in BIER-TE than in BIER because of replication efficiency
and overall amount of bits required. and overall amount of bits required.
8. BIER-TE and Segment Routing 6. BIER-TE and Segment Routing
SR aims to enable lightweight path steering via loose source routing. SR aims to enable lightweight path steering via loose source routing.
Compared to its more heavy-weight predecessor RSVP-TE, SR does for Compared to its more heavy-weight predecessor RSVP-TE, SR does for
example not require per-path signaling to each of these hops. example not require per-path signaling to each of these hops.
BIER-TE supports the same design philosophy for multicast. Like in BIER-TE supports the same design philosophy for multicast. Like in
SR, it relies on source-routing - via the definition of a BitString. SR, it relies on source-routing - via the definition of a BitString.
Like SR, it only requires to consider the "hops" on which either Like SR, it only requires to consider the "hops" on which either
replication has to happen, or across which the traffic should be replication has to happen, or across which the traffic should be
steered (even without replication). Any other hops can be skipped steered (even without replication). Any other hops can be skipped
via the use of routed adjacencies. via the use of routed adjacencies.
BIER-TE BitPosition (BP) can be understood as the BIER-TE equivalent BIER-TE bit position (BP) can be understood as the BIER-TE equivalent
of "forwarding segments" in SR, but they have a different scope than of "forwarding segments" in SR, but they have a different scope than
SR forwarding segments. Whereas forwarding segments in SR are global SR forwarding segments. Whereas forwarding segments in SR are global
or local, BPs in BIER-TE have a scope that is the group of BFR(s) or local, BPs in BIER-TE have a scope that is the group of BFR(s)
that have adjacencies for this BP in their BIFT. This can be called that have adjacencies for this BP in their BIFT. This can be called
"adjacency" scoped forwarding segments. "adjacency" scoped forwarding segments.
Adjacency scope could be global, but then every BFR would need an Adjacency scope could be global, but then every BFR would need an
adjacency for this BP, for example a forward_routed adjacency with adjacency for this BP, for example a forward_routed() adjacency with
encapsulation to the global SR SID of the destination. Such a BP encapsulation to the global SR SID of the destination. Such a BP
would always result in ingress replication though. The first BFR would always result in ingress replication though. The first BFR
encountering this BP would directly replicate to it. Only by using encountering this BP would directly replicate to it. Only by using
non-global adjacency scope for BPs can traffic be steered and non-global adjacency scope for BPs can traffic be steered and
replicated on non-ingress BFR. replicated on non-ingress BFR.
SR can naturally be combined with BIER-TE and help to optimize it. SR can naturally be combined with BIER-TE and help to optimize it.
For example, instead of defining BitPositions for non-replicating For example, instead of defining bit positions for non-replicating
hops, it is equally possible to use segment routing encapsulations hops, it is equally possible to use segment routing encapsulations
(eg: MPLS label stacks) for the encapsulation of "forward_routed" (e.g. SR-MPLS label stacks) for the encapsulation of
adjacencies. "forward_routed" adjacencies.
Note that BIER itself can also be seen to be similar to SR. BIER BPs Note that BIER itself can also be seen to be similar to SR. BIER BPs
act as global destination Node-SIDs and the BIER bitstring is simply act as global destination Node-SIDs and the BIER BitString is simply
a highly optimized mechanism to indicate multiple such SIDS and let a highly optimized mechanism to indicate multiple such SIDs and let
the network take care of effectively replicating the packet hop-by- the network take care of effectively replicating the packet hop-by-
hop to each destination Node-SID. What BIER does not allow is to hop to each destination Node-SID. What BIER does not allow is to
indicate intermediate hops, or terms of SR the ability to indicate a indicate intermediate hops, or terms of SR the ability to indicate a
sequence of SID to reach the destination. This is what BIER-TE and sequence of SID to reach the destination. This is what BIER-TE and
its adjacency scoped BP enables. its adjacency scoped BP enables.
Both BIER and BIER-TE allow BFIR to "opportunistically" copy packets Both BIER and BIER-TE allow BFIR to "opportunistically" copy packets
to a set of desired BFER on a packet-by-packet basis. In BIER, this to a set of desired BFER on a packet-by-packet basis. In BIER, this
is done by OR'ing the BP for the desired BFER. In BIER-TE this can is done by OR'ing the BP for the desired BFER. In BIER-TE this can
be done by OR'ing for each desired BFER a bitstring using the be done by OR'ing for each desired BFER a BitString using the
"independent branches" approach described in Section 7.3 and "independent branches" approach described in Section 5.3.3 and
therefore also indicating the engineered path towards each desired therefore also indicating the engineered path towards each desired
BFER. This is the approach that BFER. This is the approach that
[I-D.ietf-bier-multicast-http-response] relies on. [I-D.ietf-bier-multicast-http-response] relies on.
9. Security Considerations 7. Security Considerations
The security considerations are the same as for BIER with the If [RFC8296] is used, BIER-TE shares its security considerations.
following differences:
BFR-ids and BFR-prefixes are not used in BIER-TE, nor are procedures BIER-TE shares the security considerations of BIER, [RFC8279], with
for their distribution, so these are not attack vectors against BIER- the following overriding or additional considerations.
TE.
10. IANA Considerations In BIER, the standardized methods for the routing underlays as well
as to distribute BFR-ids and BFR-prefixes are IGPs such as specified
in [RFC8401] for IS-IS and in [RFC8444] for OSPF. Attacking the
protocols for the BIER routing underlay or BIER layer control plane,
or impairment of any BFR in a domain may lead to successful attacks
against the results of the routing protocol, enabling DoS attacks
against paths or addressing (BFR-id, BFR-prefixes) used by BIER.
The reference model for the BIER-TE layer control plane is a BIER-TE
controller. When such a controller is used, impairment of individual
BFR in a domain causes no impairment of the BIER-TE control plane on
other BFR. If a routing protocol is used to support forward_routed()
adjacencies, then this is still an attack vector as in BIER, but only
for BIER-TE forward_routed() adjacencies, and no other adjacencies.
Whereas IGP routing protocols are most often not well secured through
cryptographic authentication and confidentiality, communications
between controllers and routers such as those to be considered for
the BIER-TE controller/control-plane can and are much more commonly
secured with those security properties, for example by using Secure
SHell (SSH), [RFC4253] for NetConf ([RFC6241]), or via Transport
Layer Security (TLS), such as [RFC8253] for PCEP, [RFC5440], or
[RFC7589] for NetConf.
For additional, BIER-TE independent security considerations for the
use of a central BIER-TE controller, the security section of the
protocols and security options in the previous paragraph apply. In
addition, the security considerations of [RFC4655] apply.
The most important attack vector in BIER-TE is misconfiguration,
either on the BFR themselves or via the BIER-TE controller.
Forwarding entries with DNC could be set up to create persistent
loops, in which packets only expire because of TTL. To minimize the
impact of such attacks (or more likely unintentional misconfiguration
by operators and/or bad BIER-TE controller software), the BIER-TE
forwarding rules are defined to be as strict in clearing bits as they
are. The clearing of all bits with an adjacency on a BFR prohibits
that a looping packet creates additional packet amplification through
the misconfigured loop on the packets second or further times around
the loop, because all relevant adjacency bits would have been cleared
on the first round through the loop. In result, BIER-TE has the same
degree of looping packets as possible with unintentional or malicious
loops in the routing underlay with BIER or even with unicast traffic.
Deployments especially where BIER-TE would likely be beneficial may
include operational models where actual configuration changes from
the controller are only required during non-productive phases of the
networks life-cycle, such as in embedded networks or in manufacturing
networks during e.g. plant reworking/repairs. In these type of
deployments, configuration changes could be locked out when the
network is in production state and could only be (re-)enabled through
reverting the network/installation into non-productive state. Such
security designs would not only allow to provide additional layers of
protection against configuration attacks, but would foremost protect
the active production process from such configuration attacks.
8. IANA Considerations
This document requests no action by IANA. This document requests no action by IANA.
11. Acknowledgements 9. Acknowledgements
The authors would like to thank Greg Shepherd, Ijsbrand Wijnands, The authors would like to thank Greg Shepherd, Ijsbrand Wijnands,
Neale Ranns, Dirk Trossen, Sandy Zheng, Lou Berger and Jeffrey Zhang Neale Ranns, Dirk Trossen, Sandy Zheng, Lou Berger, Jeffrey Zhang,
for their reviews and suggestions. Alvaro Retana and Wolfgang Braun for their reviews and suggestions.
12. Change log [RFC Editor: Please remove] 10. Change log [RFC Editor: Please remove]
draft-ietf-bier-te-arch: draft-ietf-bier-te-arch:
10: AD review Alvaro Retana, summary:
Note: rfcdiff shows more changes than actually exist because text
moved around.
Summary:
1. restructuring: merged all controller sections under common
controller ops main section, moved unfitting stuff out to
other parts of doc. Split Intro section into Overview and
Intro. Shortened Abstract, moved text into Overview, added
sections overview.
2. enhanced/rewrote: 2.3 Comparison with -> Relationship to BIER-
TE
3. enhanced/rewrote: 3.2 BIER-TE controller -> BIER-TE control
plane, 3.2.1 BIER-TE controller, for consistency with rfc8279
4. additional subsections for Alvaros asks
5. added to: 3.3 BIER-TE forwarding plane (consistency with
rfc8279)
6. Enhanced description of 4.3/encap considerations to better
explain how BIER/BIER-TE can run together.
Notation: Markers (a),(b),... at end of points are references from
the review discussion with Alvaro to the changes made.
Details:.
Throughout text: changed term spelling to rfc8279 - bit positions,
sub-domain, ... (i).
Reset changed to clear, also DNR changed to DNC (Do Not Clear)
(q).
Abstract: Shortened. Removed name explanation note (Tree
Engineering), (a).
1. Introduction -> Overview: Moved important explanation
paragraph from abstract to Introduction. Fixed text, (a).
Added bullet point list explanation of structure of document (e).
Renamed to Overview because that is now more factually correct.
1.1. Fixed bug in example adding bit p15.(l).
2. (New - Introduction): Moved section 1.1 - 1.3 (examples,
comparison with BIER-TE) from Introduction into new "Overview"
section. Primarily so that "requirements language" section (at
end of Introduction) is not in middle of document after all the
Introduction.
2.1 Removed discussion of encap, moved to 4.2.2 (m).
2.2 enhanced paragraph suggesting native/overlay topology types,
also sugest type hybrid (n).
2.3 Overhauled comparison text BIER/BIER-TE, structured into
common, different, not-required-by-te, integration-bier-bier-te.
Changed title to "Relationship" to allow including last point.
(f).
2.4 moved Hardware forwarding comparison section into section 2 to
allow coalescing of sections into section 5 about the controller
operations (hardware forwarding was in the middle of it, wrong
place). Shortened/improved third paragraph by pointing to BIFT as
deciding element for selection between BIER/BIER-TE. Removed
notion of experimentation (this now targets standard) (g).
3. (Components): Aligned component name and descriptions better
with RFC8279. Now describe exactly same three layers. BIER layer
constituted from BIER-TE control plane and BIER-TE forwarding
plane. BIER-TE controller is now simply component of BIER-TE
control plane. (b).
3.1. shortened/improved paragraph explaining use of SI:BP instead
of also bfr-id as index into BIFT, rewrote paragraph talking about
reuse of BPs(o).
3.2. rewrote explanation of BIER-TE control plane in the style of
RFC8729 Section 4.2 (BIER layer) with numbered points. Note that
RFC8729 mixes control and forwarding plane bullet points (this doc
does not). Merged text from old sections 2.2.1 and 2.2.3 into
list. (b).
3.2.1. Expanded/improved explanation of BIER-TE Controller (b).
3.2.1.1. Added subsection for topology discovery and creation
(d).
3.2.1.2. Added subsection for engineered BitStrings as key novel
aspect not found in BIER. (X).
3.3. Added numbered list for components of BIER-TE forwarding
plane (completing the comparable text from RFC8729 Section 4.2).
3.4 Alvaro does not mind additional example, fixed bugs.
3.5 Removed notion about using IGP BIER extensions for BIER-TE,
such as BIFT address ranges. After -10 making use of BIFT
clearer, it now looks to authors as if use of IGP extensions would
not be beneficial, as long as we do need to use the BIER-TE
controller, e.g. unlike in BIER, a BFR could not learn from the
IGP information what traffic to send towards a particular BIFT-ID,
but instead that is the core of what the controller needs to
provide.
4.2.2 Improved text to explain requirement to identify BIER-TE in
the tunnel encap and compress description of use-cases (m).
4.2.3 enhanced ECMP text (p).
4.3. rewrote most of Encapsulation Considerations to better
explain to Alvaros question re sharing or not sharing SD via BIER/
BIER-TE. Added reference to I-D.ietf-bier-non-mpls-bift-encoding
as a very helpful example. (f).
4.3 Renamed title to "...Co-Existence with BIER" as this is what
it is about and to help finding it from abstract/intro ("co-
exist") (j).
4.4. Moved BIER-TE Forwarding Pseudocode here to coalesce text
logically. Changed text to better compare with BIER pseudo
forwarding code. Numerical list of how F-BM works for BIER-TE.
Removed efficiency comparison with BIER (too difficult to provide
sufficient justification, derails from focus of section) (j).
4.6. (Requirements) Restructured: Removed notion of "basic" BIER-
TE forwarding, simply referring to it now as "mandatory" BIER-TE
forwarding. Cleaned up text to have requirements for different
adjacencies in different paragraphs. (c).
5. Created new main section "BIER-TE Controller operational
considerations", coalesced old sections 4., 5., 7. into this new
main section. No text changes. (k).
5.1.9 Added new separate picture instead of referring to a picture
later in text, adjusted text (r).
5.3.2 Changed title to not include word "comparison" to avoid this
being accounted against Alvaros concern about scattering
comparison (IMHO text already has little comparison, so title was
misleading) (h).
co-authors internal review:
4.4 Added xref to Figure 5.
5.2.1 Duplicated ring picture, added visuals for described
miswiring (s).
5.2.2 replace "topology" with graph (wrong word).
5.3.3 rewrote explanation of how to map BFR-id to SI:BP and assign
them, clarified BFR-id is option. Retitled to better explain
scope of section.
5.3.4 Removed considerations in 5.3.4 for sharing BFR-id across
BIER/BIER-TE (t), changed title to explain how BFIR/BIER-TE
controller interactions need some form of identifying BFR but this
does not have to be BFR-id.
7. Added new security considerations (u).
09: Incorporated fixes for feedback from Shepherd (Xuesong Geng). 09: Incorporated fixes for feedback from Shepherd (Xuesong Geng).
Added references for Bloom Filders and Rate Controlled Service Added references for Bloom Filters and Rate Controlled Service
Disciplines. Disciplines.
1.1 Fixed numbering of example 1 topology explanation. Improved 1.1 Fixed numbering of example 1 topology explanation. Improved
language on second example (less abbreviating to avoid confusion language on second example (less abbreviating to avoid confusion
about meaning). about meaning).
1.2 Improved explanation of BIER-TE topology, fixed terminology of 1.2 Improved explanation of BIER-TE topology, fixed terminology of
graphs (BIER-TE topology is a directed graph where the edges are graphs (BIER-TE topology is a directed graph where the edges are
the adjacencies). the adjacencies).
2.4 Fixed and amended routing underlay explanations: detailled why 2.4 Fixed and amended routing underlay explanations: detailed why
no need for BFER routing underlay routing protocol etensions, but no need for BFER routing underlay routing protocol extensions, but
potential to re-use BIER routing underlay routing protocol potential to re-use BIER routing underlay routing protocol
extensions for non-BFER related extensions. extensions for non-BFER related extensions.
3.1 Added explanation for VRF and its use in adjacencies. 3.1 Added explanation for VRF and its use in adjacencies.
08: Incorporated (with hopefully acceptable fixes) for Lou 08: Incorporated (with hopefully acceptable fixes) for Lou
suggested section 2.5, TE considerations. suggested section 2.5, TE considerations.
Fixes are primarily to the point to a) emphasize that BIER-TE does Fixes are primarily to the point to a) emphasize that BIER-TE does
not depend on the routing underlay unless forward_routed not depend on the routing underlay unless forward_routed()
adjacencies are used, and b) that the allocation and tracking of adjacencies are used, and b) that the allocation and tracking of
resources does not explicitly have to be tied to BPs, because they resources does not explicitly have to be tied to BPs, because they
are just steering labels. Instead, it would ideally come from are just steering labels. Instead, it would ideally come from
per-hop resource management that can be maintained only via local per-hop resource management that can be maintained only via local
accounting in the controller. accounting in the controller.
07: Further reworking text for Lou. 07: Further reworking text for Lou.
Renamed BIER-PE to BIER-TE standing for "Tree Engineering" after Renamed BIER-PE to BIER-TE standing for "Tree Engineering" after
votes from BIER WG. votes from BIER WG.
skipping to change at page 46, line 27 skipping to change at page 55, line 30
Added BIER-in-BIER tunneling as option for tunnels in backup Added BIER-in-BIER tunneling as option for tunnels in backup
paths. BIFT structure is expanded and contains an additional paths. BIFT structure is expanded and contains an additional
match field to support full node protection with BIER-TE FRR. match field to support full node protection with BIER-TE FRR.
03: Updated FRR section. Explanation how BIER-in-BIER 03: Updated FRR section. Explanation how BIER-in-BIER
encapsulation provides P2MP protection for node failures even encapsulation provides P2MP protection for node failures even
though the routing underlay does not provide P2MP. though the routing underlay does not provide P2MP.
02: Changed the definition of BIFT to be more inline with BIER. 02: Changed the definition of BIFT to be more inline with BIER.
In revs. up to -01, the idea was that a BIFT has only entries for In revs. up to -01, the idea was that a BIFT has only entries for
a single bitstring, and every SI and subdomain would be a separate a single BitString, and every SI and sub-domain would be a
BIFT. In BIER, each BIFT covers all SI. This is now also how we separate BIFT. In BIER, each BIFT covers all SI. This is now
define it in BIER-TE. also how we define it in BIER-TE.
02: Added Section 7 to explain the use of SI, subdomains and BFR- 02: Added Section 5.3 to explain the use of SI, sub-domains and
id in BIER-TE and to give an example how to efficiently assign BFR-id in BIER-TE and to give an example how to efficiently assign
bits for a large topology requiring multiple SI. bits for a large topology requiring multiple SI.
02: Added further detailed for rings - how to support input from 02: Added further detailed for rings - how to support input from
all ring nodes. all ring nodes.
01: Fixed BFIR -> BFER for section 4.3. 01: Fixed BFIR -> BFER for section 4.3.
01: Added explanation of SI, difference to BIER ECMP, 01: Added explanation of SI, difference to BIER ECMP,
consideration for Segment Routing, unicast FRR, considerations for consideration for Segment Routing, unicast FRR, considerations for
encapsulation, explanations of BIER-TE Controller and CLI. encapsulation, explanations of BIER-TE Controller and CLI.
00: Initial version. 00: Initial version.
13. References 11. References
13.1. Normative References 11.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8279] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., [RFC8279] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A.,
Przygienda, T., and S. Aldrin, "Multicast Using Bit Index Przygienda, T., and S. Aldrin, "Multicast Using Bit Index
Explicit Replication (BIER)", RFC 8279, Explicit Replication (BIER)", RFC 8279,
DOI 10.17487/RFC8279, November 2017, DOI 10.17487/RFC8279, November 2017,
<https://www.rfc-editor.org/info/rfc8279>. <https://www.rfc-editor.org/info/rfc8279>.
[RFC8296] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., 11.2. Informative References
Tantsura, J., Aldrin, S., and I. Meilik, "Encapsulation
for Bit Index Explicit Replication (BIER) in MPLS and Non-
MPLS Networks", RFC 8296, DOI 10.17487/RFC8296, January
2018, <https://www.rfc-editor.org/info/rfc8296>.
13.2. Informative References
[Bloom70] Bloom, B., "Space/time trade-offs in hash coding with [Bloom70] Bloom, B., "Space/time trade-offs in hash coding with
allowable errors", Comm. ACM 13(7):422-6, July 1970. allowable errors", Comm. ACM 13(7):422-6, July 1970.
[I-D.eckert-bier-te-frr]
Eckert, T., Cauchie, G., Braun, W., and M. Menth,
"Protection Methods for BIER-TE", draft-eckert-bier-te-
frr-03 (work in progress), March 2018.
[I-D.ietf-bier-multicast-http-response] [I-D.ietf-bier-multicast-http-response]
Trossen, D., Rahman, A., Wang, C., and T. Eckert, Trossen, D., Rahman, A., Wang, C., and T. Eckert,
"Applicability of BIER Multicast Overlay for Adaptive "Applicability of BIER Multicast Overlay for Adaptive
Streaming Services", draft-ietf-bier-multicast-http- Streaming Services", draft-ietf-bier-multicast-http-
response-04 (work in progress), July 2020. response-05 (work in progress), January 2021.
[I-D.ietf-bier-non-mpls-bift-encoding]
Wijnands, I., Mishra, M., Xu, X., and H. Bidgoli, "An
Optional Encoding of the BIFT-id Field in the non-MPLS
BIER Encapsulation", draft-ietf-bier-non-mpls-bift-
encoding-03 (work in progress), November 2020.
[I-D.ietf-bier-te-yang]
Zhang, Z., Wang, C., Chen, R., Hu, F., Sivakumar, M., and
H. Chen, "A YANG data model for Traffic Engineering for
Bit Index Explicit Replication (BIER-TE)", draft-ietf-
bier-te-yang-02 (work in progress), August 2020.
[I-D.ietf-roll-ccast] [I-D.ietf-roll-ccast]
Bergmann, O., Bormann, C., Gerdes, S., and H. Chen, Bergmann, O., Bormann, C., Gerdes, S., and H. Chen,
"Constrained-Cast: Source-Routed Multicast for RPL", "Constrained-Cast: Source-Routed Multicast for RPL",
draft-ietf-roll-ccast-01 (work in progress), October 2017. draft-ietf-roll-ccast-01 (work in progress), October 2017.
[I-D.ietf-teas-rfc3272bis] [I-D.ietf-teas-rfc3272bis]
Farrel, A., "Overview and Principles of Internet Traffic Farrel, A., "Overview and Principles of Internet Traffic
Engineering", draft-ietf-teas-rfc3272bis-01 (work in Engineering", draft-ietf-teas-rfc3272bis-11 (work in
progress), July 2020. progress), April 2021.
[ICC] Reed, M., Al-Naday, M., Thomos, N., Trossen, D., [ICC] Reed, M., Al-Naday, M., Thomos, N., Trossen, D.,
Petropoulos, G., and S. Spirou, "Stateless multicast Petropoulos, G., and S. Spirou, "Stateless multicast
switching in software defined networks", IEEE switching in software defined networks", IEEE
International Conference on Communications (ICC), Kuala International Conference on Communications (ICC), Kuala
Lumpur, Malaysia, 2016, May 2016, Lumpur, Malaysia, 2016, May 2016,
<https://ieeexplore.ieee.org/document/7511036>. <https://ieeexplore.ieee.org/document/7511036>.
[RCSD94] Zhang, H. and D. Domenico, "Rate-Controlled Service [RCSD94] Zhang, H. and D. Domenico, "Rate-Controlled Service
Disciplines", Journal of High-Speed Networks, 1994, May Disciplines", Journal of High-Speed Networks, 1994, May
1994, <https://dl.acm.org/doi/10.5555/2692227.2692232>. 1994, <https://dl.acm.org/doi/10.5555/2692227.2692232>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC4253] Ylonen, T. and C. Lonvick, Ed., "The Secure Shell (SSH)
Requirement Levels", BCP 14, RFC 2119, Transport Layer Protocol", RFC 4253, DOI 10.17487/RFC4253,
DOI 10.17487/RFC2119, March 1997, January 2006, <https://www.rfc-editor.org/info/rfc4253>.
<https://www.rfc-editor.org/info/rfc2119>.
[RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation
Element (PCE)-Based Architecture", RFC 4655,
DOI 10.17487/RFC4655, August 2006,
<https://www.rfc-editor.org/info/rfc4655>.
[RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation
Element (PCE) Communication Protocol (PCEP)", RFC 5440,
DOI 10.17487/RFC5440, March 2009,
<https://www.rfc-editor.org/info/rfc5440>.
[RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed.,
and A. Bierman, Ed., "Network Configuration Protocol
(NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011,
<https://www.rfc-editor.org/info/rfc6241>.
[RFC7589] Badra, M., Luchuk, A., and J. Schoenwaelder, "Using the
NETCONF Protocol over Transport Layer Security (TLS) with
Mutual X.509 Authentication", RFC 7589,
DOI 10.17487/RFC7589, June 2015,
<https://www.rfc-editor.org/info/rfc7589>.
[RFC7752] Gredler, H., Ed., Medved, J., Previdi, S., Farrel, A., and
S. Ray, "North-Bound Distribution of Link-State and
Traffic Engineering (TE) Information Using BGP", RFC 7752,
DOI 10.17487/RFC7752, March 2016,
<https://www.rfc-editor.org/info/rfc7752>.
[RFC8253] Lopez, D., Gonzalez de Dios, O., Wu, Q., and D. Dhody,
"PCEPS: Usage of TLS to Provide a Secure Transport for the
Path Computation Element Communication Protocol (PCEP)",
RFC 8253, DOI 10.17487/RFC8253, October 2017,
<https://www.rfc-editor.org/info/rfc8253>.
[RFC8296] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A.,
Tantsura, J., Aldrin, S., and I. Meilik, "Encapsulation
for Bit Index Explicit Replication (BIER) in MPLS and Non-
MPLS Networks", RFC 8296, DOI 10.17487/RFC8296, January
2018, <https://www.rfc-editor.org/info/rfc8296>.
[RFC8345] Clemm, A., Medved, J., Varga, R., Bahadur, N.,
Ananthakrishnan, H., and X. Liu, "A YANG Data Model for
Network Topologies", RFC 8345, DOI 10.17487/RFC8345, March
2018, <https://www.rfc-editor.org/info/rfc8345>.
[RFC8401] Ginsberg, L., Ed., Przygienda, T., Aldrin, S., and Z.
Zhang, "Bit Index Explicit Replication (BIER) Support via
IS-IS", RFC 8401, DOI 10.17487/RFC8401, June 2018,
<https://www.rfc-editor.org/info/rfc8401>.
[RFC8444] Psenak, P., Ed., Kumar, N., Wijnands, IJ., Dolganow, A.,
Przygienda, T., Zhang, J., and S. Aldrin, "OSPFv2
Extensions for Bit Index Explicit Replication (BIER)",
RFC 8444, DOI 10.17487/RFC8444, November 2018,
<https://www.rfc-editor.org/info/rfc8444>.
Authors' Addresses Authors' Addresses
Toerless Eckert (editor) Toerless Eckert (editor)
Futurewei Technologies Inc. Futurewei Technologies Inc.
2330 Central Expy 2330 Central Expy
Santa Clara 95050 Santa Clara 95050
USA USA
Email: tte+ietf@cs.fau.de Email: tte+ietf@cs.fau.de
 End of changes. 302 change blocks. 
984 lines changed or deleted 1527 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/