<?xml version="1.0" encoding="UTF-8"?>
  <?xml-stylesheet type="text/xsl" href="rfc2629.xslt" ?>

<!DOCTYPE rfc SYSTEM "rfc2629.dtd" [
]>

<?xml-stylesheet type='text/xsl' href='rfc2629.xslt' ?>
<?rfc toc="yes"?>
<?rfc symrefs="yes"?>
<?rfc sortrefs="yes"?>
<?rfc iprnotified="no"?>
<?rfc strict="yes"?>
<?rfc compact="yes"?>
<?rfc subcompact="yes"?>
<rfc category="std"
     docName="draft-thubert-bier-replication-elimination-03"
     ipr="trust200902"
     submissionType="IETF">
  <front>
    <title abbrev="BIER-TE PREF/OAM">
     BIER-TE extensions for Packet Replication and Elimination Function (PREF) and OAM</title>

  
    <author initials="P." surname="Thubert" fullname="Pascal Thubert" role="editor">
      <organization abbrev="Cisco">Cisco Systems</organization>
      <address>
        <postal>
          <street>Village d'Entreprises Green Side</street> <street>400, Avenue de Roumanille</street> <street>Batiment T3</street>
          <city>Biot - Sophia Antipolis</city>
          <code>06410</code>
          <country>FRANCE</country>
        </postal>
        <phone>+33 4 97 23 26 34</phone>
        <email>pthubert@cisco.com</email>
      </address>
    </author>

   
    <author fullname="Toerless Eckert" initials="T.T.E" surname="Eckert">
       <organization abbrev="Huawei">Huawei USA - Futurewei Technologies Inc.</organization>
       <address>
        <postal>
          <street>2330 Central Expy</street>
          <city>Santa Clara</city>
          <code>95050</code>
          <country>USA</country>
        </postal>
        <email>tte+ietf@cs.fau.de</email>
      </address>
    </author>
    
     <author initials="Z." surname="Brodard" fullname="Zacharie Brodard" >
      <organization abbrev="Ecole Polytechnique">Ecole Polytechnique</organization>
      <address>
        <postal>
          <street>Route de Saclay</street>
          <city>Palaiseau</city>
          <code>91128</code>
          <country>FRANCE</country>
        </postal>
        <phone>+33 6 73 73 35 09</phone>
        <email>zacharie.brodard@polytechnique.edu</email>
      </address>
    </author>
    
       <author initials="H." surname="Jiang" fullname="Hao Jiang">
      <organization abbrev="Telecom Bretagne">Telecom Bretagne</organization>
      <address>
        <postal>
          <street>2, rue de la Châtaigneraie</street> 
          <city> Cesson-Sévigné</city>
          <code>35510</code>
          <country>FRANCE</country>
        </postal>
        <phone>+33 7 53 70 97 34</phone>
        <email>hao.jiang@telecom-bretagne.eu</email>
      </address>
    </author>

  <date />
  <workgroup>BIER</workgroup>

  <abstract>
  <t>
	This specification extends Bit Index Explicit Replication - Traffic
    Engineering (BIER-TE) forwarding to support in the data plane the DetNet 
Packet Replication and Elimination Functions (PREF). It also provides traceability of links/adjacencies where replication and loss happen, in a manner that is agnostic to the
	forwarding information (OAM). 
  </t>
  </abstract>

  
  </front>

 <middle>
 <section title="Introduction">
  <t>
    Deterministic Networking (DetNet) <xref
    target="I-D.ietf-detnet-problem-statement"/> provides a capability to carry
    unicast or multicast data flows for real-time applications with extremely
    low data loss rates and known upper bound maximum latency <xref
    target="I-D.ietf-detnet-architecture"/>. 
  </t>
  <t>DetNet applies to multiple environments where there is a desire to replace
     a point to point serial cable or a multidrop bus by a switched or routed
	 infrastucture, in order to scale, lower costs, and simplify management.
	 One classical use case is found in particular in the context of the
	 convergence of IT with Operational Technology (OT), also referred to as
	 the Industrial Internet. But there are many others use cases
	 <xref target="I-D.ietf-detnet-use-cases"/>, for instance in
	 in professional audio and video, automotive, radio fronthauls, etc..
 </t>

  <t>
    The <xref target="I-D.dt-detnet-dp-alt">DetNet data plane alternatives
	</xref> studies the applicability of existing and emerging dataplane 
	techniques that can be leveraged to
	enable DetNet properties in IP networks. One critical feature in 
	the dataplane is traceability, the capability to control the activity
    of intermediate nodes on a packet. For instance, if Replication and Elimination
	is applied to a packet, then it is desirable to determine which node performed
	a certain copy of that packet that is circulating in the network. Likewise,
        engineered paths are required to support redundant transmission across
        disjoint paths in support of DetNets PREF functios.
	</t>

	<t>
    Traceability belongs to Operations, Administration, and Maintenance (OAM) 
	which is the toolset for fault detection and isolation, and for performance
	measurement. More can be found on OAM Tools  in
	<xref target="I-D.ietf-opsawg-oam-overview">"An Overview of Operations,
      Administration  and Maintenance (OAM) Tools"</xref>.
  
  </t>
	<t>
	This document proposes a new set to mechanisms based on 
    <xref target="RFC8279"/> (BIER) and more specifically 
    <xref target="I-D.ietf-bier-te-arch">BIER Traffic Engineering</xref>
	(BIER-TE) to control the process or Packet Replication and Elimination Functions (PREF),
        and provide traceability of these operations, in the DetNet dataplane. An
    adjacency, which is represented by a bit in the BIER header, can correspond
    in the dataplane to an Ethernet hop, a Label Switched Path, or it can
    correspond to an IPv6 loose or strict source routed path. </t>

    <t>BIER-TE was primarily designed to carry multicast traffic, but there is
      nothing prohibiting for it to be used with unicast traffic, and the authors
      of this document think that for networks whose size requirement match the
      supportable bitstring length (BSL) in BIER, it can be a good choice as
      the forwarding plane specifically for DetNet type traffic for both multicast
      and unicast traffic because it would be a common solution for unicast and
      multicast (limiting the number of different technologies a DetNet solution
      requires) and likely provides the most flexible support for path engineering,
      replication and elimination (PREF) and the novel OAM method described in this document.</t>
  
 </section>

 <section title="Terminology">
  <t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in <xref
   target="RFC2119"></xref>.</t>
 </section>

 
 <section title="On BIER - Traffic Engineering"
          anchor="sec_alt_bier_te">

<!-- left in DT doc
  <t>
   Bit-Indexed Explicit Replication (BIER) uses bits to indicate destination,
   so called BFER - BIER Forwarding Egress Routers. The path to each of those
   receivers is calculated by the IGP in the network. In BIER-TE, bits of the
   bitstring represent adjacencies, and destinations are just one type of those
   adjacencies. More importantly, interfaces on nodes are also adjacencies, and
   this allows to express paths through the network and complete DAGs (Directed
   Acyclic Graphs) as a subset of bit of the network topology.
   See <xref target="I-D.ietf-bier-te-arch"> BIER Traffic Engineering (TE)</xref>.
   </t>

   <t>
   <t>DetNet resilient transmission is based on traffic flows identified through some
   "flod identification" of packets. Each packet then has a unique sequence number.
   these packet flows have to be replicated and forwarded over disjoint paths
   so that N-1 individual failures (link/node) would not impact packets of all
   N copies. On a node performing the so-called Elimination Function (EF), copies
   are merged by simply eliminating duplicate packets before forwarding - based
   on the sequence number of the packet.</t>

   <t>The resilience requirements against the network does not necessarily imply
   that all nodes and links used for different copies of the traffic need to be
   disjoint. Instead, nodes and links can be shared In the most simple & traditional
   example, the topology is a ring of nodes and the two copies of traffic flows
   are carried counterclockwise to each other across the ring.</t>
   
   <t>In the overall solution assumed by this document, the encapsulation/insertion
   of flow-identification and sequence number into packets is performed by some
   function outside the scope of this packet. A companion document 
   <xref target="I-D.huang-bier-te-encapsulation"/> proposes an encapsulation
   for BIER-TE and BIER that can provide flow-id and sequence-number ID. Other
   encapsulations can be used as well, as long as they provide these element - and
   are supported by the Elimination Function described in this document. In the
   remainder of this document we will call this the extended BIER encapsulation and
   assume that it is used when describing examples. Unless otherwise noted,
   we assume that the BFIR performs encapsulation of some data flow packets with
   an extended BIER header, indicates BIER-TE forwarding in it and fills in 
   flow-id and sequence number. It then fills in the bitstring with two (or more)
   alternative paths/DAGs and sends of the packets into the BIER-TE domain.</t>

   <t>The actual packet replication function is simply subject to the BIER-TE bitstring
   of packets, this document does not introduce novel functionality. In many other
   solutions, it is required that the (two) copies of a packet stream need to
   be distinguished in the packet header to make them subject to routing across
   disjoint paths. If for example BIER was used with the extended BIER header, 
   it would be necessary to assign to every destination (BFER) bits in two different
   BIER Subdomains (BIER-SD) so that traffic for these bits could be routed
   across disjoint paths. For example, the two SDs could be mapped to the two
   different path sets calculated by MRTs (<xref target="RFC7811"/>). This 
   use of multiple bits for destinations or SDs and mapping to different topologies
   is not necessary with the solution solution described here for BIER-TE, because
   the BIER-TE bits fully describe both paths through the topology - terminating
   both in the same destination (bits).
   
   <t>The BIER-TE extensions described in this document perform two functions.
   The is the Elimination Function. The second is the measurement (OAM) function
   that is examining the remaining bits of the BIER-TE bitstring to analyze the
   forwarding that did happen to the packet. Both functions are described as
   new BIER-TE adjacencies associated with bit(s) of the BIER-TE bitstring.</t>

   <t>For the OAM funcion described to be beneficial, additional signaling, for
   for example to the BIER-TE Controller Host is required. This is outside the
   scope of this document. For example the controller could take the measurement
   information, use it to discover path failures, caluculate new paths, which
   it then feeds back to the BFIR.</t>
   <t>The value of this OAM approach is that the replication can be controlled and monitored with the
   granularity of a packet and a adjacency in a control loops that involves an
   external controller.
  </t>
  
-->  
  
    <t>
   <xref target="RFC8279"/> (BIER)
   is a network plane replication technique that was initially
   intended as a new method for multicast distribution. In a nutshell, a BIER
   header includes a bitmap that explicitly signals the listeners that are
   intended for a particular packet, which means that 1) the sender is aware of
   the individual listeners and 2) the BIER control plane is a simple extension 
   of the unicast routing as opposed to a dedicated multicast data plane, which 
   represents a considerable reduction in OPEX. For this reason, the technology
   faces a lot of traction from Service Providers. 
  </t>
  <t>
   The simplicity of the BIER technology makes it very versatile as a network
   plane signaling protocol. Already, a new Traffic Engineering variation is
   emerging that uses bits to signal segments along a TE path.</t>

<!-- 

   TTE: I removed the following sentences because i don't think BIER-TE is
   "primarily a unicast technology. In fact, if you tried to compare how well
   BIER-TE would fare against SR, you would need label stacks of 12 * 20 bit
   MPLS labels to come up to the size of a 256 bit BIER-TE bitstring. So
   it would be arguable, if if BIER-TE would ever be more efficient for
   UNICAST path engineeering than SR. Of course it would be a lot easier to
   compete with SRv6 (256 / 64 = 4 label hops). See below for a proposed restatement.


   While  BIER is mainly a multicast technology that typically leverages a
   unicast distributed control plane through IGP extensions, BIER-TE
   <xref target="I-D.ietf-bier-te-arch"/> is mainly
   a unicast technology that leverages a central computation to setup path,
   compute segments and install the mapping in the intermediate nodes.

-->

  <t>While BIER-TE was like BIER primarily developed for multicast traffic, 
  the authors think that it can equally be attractive for unicast traffic
  requiring the DetNet resilience of multiple transitions. If the topology
  of the network can well be represented by standard BIER-TE bitstring sizes
  of e.g.: up to 256 bits, then this would allow for a single technology
  for both unicast and multicast.  </t>

  <t>
   BIER-TE supports a Traffic Engineered forwarding plane
   by explicit hop-by-hop forwarding and loose hop forwarding of packets. 
     </t><t>
      From the BIER-TE architecture, the key differences over BIER are:
      <list style="symbols">
      <t>BIER-TE replaces in-network autonomous path calculation by
         explicit paths calculated off path for example by a BIER-TE controller host.
      </t>
      <t>In BIER-TE every BitPosition of the BitString of a BIER-TE packet
         indicates one or more adjacencies - instead of a BFER as in BIER.
         processing packets as a destination (BFER) is one of the possible adjacency types.
      </t>
      <t>BIER-TE in each BFR has no routing table but only a BIER-TE
         Forwarding Table (BIFT) indexed by SI:BitPosition and populated
         with only those adjacencies to which the BFR should replicate
         packets to.
      </t>
      </list>        
   The generic view of an adjacency can be over a link, a tunnel or along a
   path segment. 
  </t>

<!--

   TTE: Removed the following SR explanation, because it looked to me as derailing/confusing,
   e.g.: was unclear why it was here. If its meant to explain what a path segment is,
   then merge with the previous paragraph as in... "A path segment according to .. SR.."
<t>
   With <xref target="I-D.ietf-spring-segment-routing">Segment Routing</xref> a
   segment can be signaled as an MPLS label, or an IPv6 routing header .  A
   segment may be loosely of strictly source routed, depending on the need for
   full non-congruence and the confidence that loose routing may still achieve
   that need. 
  </t>

-->
  
 </section>
  <section title="BIER-TE-based Replication and Elimination Control">

   <t>This document only needs to introduce new functionality to support the Elimination Function
   and OAM. Creation of appropriate BIER-TE packets is subject to to existing work.</t>

   <t>In the solution described below, the encapsulation/insertion
   of flow-identification and sequence number into packets is performed by a
   function on the BFIR outside the scope of this document.  A companion document 

   <!-- <xref target="I-D.huang-bier-te-encapsulation"/>
        Not yet in datatracker, use this xref once it is -->

   draft-huang-bier-te-encapsulation defines an encapsulation
   for BIER-TE and BIER that can support flow-id and sequence-number ID. Other
   encapsulations can be used as well, as long as they provide these signaling elements
   and are supported by the Elimination Function described in this document (e.g.: that
   the EF can read these fields and therefore remove duplicates). In the
   remainder of this document we will call this the extended BIER encapsulation and
   assume that it is used when describing examples. Unless otherwise noted,
   we assume that the BFIR performs encapsulation of some data flow packets with
   an extended BIER header, indicates BIER-TE forwarding in it and fills in 
   flow-id and sequence number. It then fills in the bitstring with two (or more)
   alternative paths/DAGs and sends off the packets into the BIER-TE domain, replicating
   it itself if so indicated by the bitstring.</t>

<t>
   In a nutshell, BIER-TE is used as follows:
     <list style="symbols">
     <t>
     A controller computes a complex path, sometimes called a track, which takes
     the general form of a ladder. The steps and the side rails between them
     are the adjacencies that can be activated on demand on a per-packet basis
     using bits in the BIER header. 
     </t>
     </list>
     </t>
<figure anchor="fig_ladder" align="center"
title="Ladder Shape with Replication and Elimination Points">
<artwork align="center"><![CDATA[
 
                 ===> (A) ====> (C) ==== 
               //     ^ |       ^ |     \\
   ingress (I)        | |       | |       (E) egress
               \\     | v       | v     //
                 ===> (B) ====> (D) ==== 
 
]]></artwork>
</figure>
     <t>      
     <list style="symbols">
     <t>      
     The controller assigns a BIER domain, and inside that domain, assigns bits
     to the adjacencies. The controller assigns each bit to a replication node
     that sends towards the adjacency, for instance the ingress router into a
     segment that will insert a routing header in the packet. A single bit may
     be used for a step in the ladder, indicating the other end of the step in
     both directions.
     </t>
     </list>
     </t>
<figure anchor="fig_track" align="center"
title="Assigning Bits">
<artwork align="center"><![CDATA[
 
                 ===> (A) ====> (C) ==== 
               // 1   ^ |  4    ^ |   7 \\
   ingress (I)        |2|       |6|       (E) egress
               \\ 3   | v  5    | v   8 //    
                 ===> (B) ====> (D) ==== 
 
]]></artwork>
</figure>
     <t>      
     <list style="symbols">
     <t>     
     The controller activates the replication by deciding the setting of the
     bits associated with the adjacencies. This decision can be modified at any
     time, but takes the latency of a controller round trip to effectively take
     place. Below is an example that uses Replication and Elimination to protect
     the A->C adjacency. The "(EF)" in the following pictures Owner column
     indicates the fact that that BFR will perform the "Elimination Function"
     for received BIER-TE packets before further processing/copying them.
     In this example, only C performs EF. A (1) in the Example Bitstring indicates
     that the bit is set, but that the actual adjacency is not used by packets
     because this bit is shared with another adjacency and the overall bitstring
     will make the packet only use that other adjacency. This applies to bits
     2 and 6.
     </t>
     </list>
     </t>
      
<texttable anchor="table_bit" title="Controlling Replication">
    <ttcol align='center'>Bit #</ttcol>
    <ttcol align='center'>Adjacency</ttcol>
    <ttcol align='center'>Owner</ttcol>
    <ttcol align='center'>Example Bitstring</ttcol>
    
    <c>1</c>
    <c>I->A</c>
    <c>I</c>
    <c>1</c>
    
    <c>2</c>
    <c>A->B</c>
    <c>A</c>
    <c>1</c>
    
    <c></c>
    <c>B->A</c>
    <c>B</c>
    <c>(1)</c>
    
    <c>3</c>
    <c>I->B</c>
    <c>I</c>
    <c>0</c>
    
    <c>4</c>
    <c>A->C</c>
    <c>A</c>
    <c>1</c>
    
    <c>5</c>
    <c>B->D</c>
    <c>B</c>
    <c>1</c>
    
    <c>6</c>
    <c>C->D</c>
    <c>C (EF)</c>
    <c>(1)</c>
    
    <c></c>
    <c>D->C</c>
    <c>D</c>
    <c></c>
    
    <c>7</c>
    <c>C->E</c>
    <c>C (EF)</c>
    <c>1</c>
    
    <c>8</c>
    <c>D->E</c>
    <c>D</c>
    <c>0</c>
    
    <postamble>Replication and Elimination Protecting A->C</postamble>
</texttable>

     <t>
     <list style="symbols">
     <t> 
     The BIER header with the controlling BitString , flow-id and sequence number
     is injected in the packet by the ingress node I (BFIR). That node may act as a
     replication point, in which case it may issue multiple copies of the packet,
     but for the purpose of this example it will not do it, so that the two paths
     used in this example only go from A to C, and therefore require explicit
     path engineering. For example, bandwidth I-A and I-B may be more limited and
     those paths being non long-haul may not warrant the dual transmission.
     </t>
     </list>
     </t>
<figure anchor="fig_track_prot" align="center"
title="Enabled Adjacencies">
<artwork align="center"><![CDATA[
 
              ====>  Repl ===> Elim ==== 
           //         |         ^        \\
   ingress            |         |           egress
                      v         |             
                     Fwd ====> Fwd      
 
]]></artwork>
</figure>
     <t>      
     <list style="symbols">
     <t>
     For each of its bits that is set in the BIER header, the owner replication
     point resets the bit used for a copy and transmits towards the associated adjacency;
     to achieve this, the replication point copies the packet and inserts the
     relevant data plane information, such as next-hop label, MAC-address or source route header (for a BIER-TE routed adjacency), towards the adjacency that corresponds to the bit 
     </t>
     </list>
     </t>
<texttable anchor="table_bit2" title="BIER-TE in Action">
    <ttcol align='center'>Adjacency</ttcol>
    <ttcol align='center'>BIER BitString</ttcol>
    
    <c>I->A</c>
    <c>01011110</c>
    <c>A->B</c>
    <c>00011110</c>
    <c>B->D</c>
    <c>00010110</c>
    <c>D->C</c>
    <c>00010010</c>
    <c>A->C</c>
    <c>01001110</c>
    <postamble>BitString in BIER Header as Packet Progresses</postamble>
</texttable>

     <t>
     <list style="symbols">
     <t> 
     Adversely, an elimination node on the path performs the Elimination Function
     which will remove duplicate packets (same flow-id, same sequence number) 
     and performs a bitwise AND on the BitStrings from
     the various copies of the packet that it has received, before it forwards the packet with the
     resulting BitString. Details of the Elimination Function are described below.
     </t>
     </list>
     </t>
<texttable anchor="table_bit3" title="BIER-TE in Action (cont.)">
    <ttcol align='center'>Operation</ttcol>
    <ttcol align='center'>BIER BitString</ttcol>
    <c>D->C</c>
    <c>00010010</c>
    <c>A->C</c>
    <c>01001110</c>
    <c> </c>
    <c>--------</c>
    <c>AND in C</c>
    <c>00000010</c>
    <c> </c>
    <c> </c>
    <c>C->E</c>
    <c>00000000</c>
    <postamble>BitString Processing at Elimination Point C</postamble>
</texttable>
     <t>
     <list style="symbols">
     <t> 
     In this example, all the transmissions succeeded and the BitString at
     arrival has all the bits reset - note that the egress may be an Elimination
     Point in which case this is evaluated after this node has performed its AND
     operation on the received BitStrings).
     </t>
     </list>
     </t>
<texttable anchor="table_bit4" title="BIER-TE in Action (cont.)">
    <ttcol align='center'>Failing Adjacency</ttcol>
    <ttcol align='center'>Egress BIER BitString</ttcol>
    <c>I->A</c>
    <c>Frame Lost</c>
    <c>I->B</c>
    <c>Not Tried</c>
    <c>A->C</c>
    <c>00010000</c>
    <c>A->B</c>
    <c>01001100</c>
    <c>B->D</c>
    <c>01001100</c>
    <c>D->C</c>
    <c>01001100</c>
    <c>C->E</c>
    <c>Frame Lost</c>
    <c>D->E</c>
    <c>Not Tried</c>
    <postamble>BitString indicating failures</postamble>
</texttable>
     <t>
     <list style="symbols">
     <t> 
     But if a transmission failed along the way, one (or more) bit is never
     cleared. <xref target="table_bit4"/> provides the possible outcomes of a 
     transmission. If the frame is lost, then it is probably due to a failure in
     either I->A or C->E, and the controller should enable I->B and D->E to
     find out. A BitString of 00010000 indicates unequivocally a transmission
     error on the A->C adjacency, and a BitString of 01001100 indicates a loss
     in either A->B, B->D or D->C; enabling D->E on the next packets may provide
     more information to sort things out.
     </t>
     </list>
     </t>

<t>In more details:   
  </t><t>   
   The BIER header is of variable size, and a DetNet network of a limited size
   can use a model with 64 bits if 64 adjacencies are enough, whereas a larger
   deployment may be able to signal up to 256 adjacencies for use 
   in very complex paths. The format of this header is common to
   BIER and BIER-TE.
  </t>
 <t>
   For the DetNet data plane, a replication point is an ingress point for more
   than one adjacency, and an elimination point is an egress point for more than
   one adjacency.
   </t><t>   
   A pre-populated state in a replication node indicates which bits are
   served by this node and to which adjacency each of these bits corresponds.
   With DetNet, the state is typically installed by a controller entity such as
   a PCE. 
   The way the adjacency is signaled in the packet is fully abstracted in the
   bit representation and must be provisioned to the replication nodes and 
   maintained as a local state, together with the timing or shaping information
   for the associated flow.
  </t><t>
   The DetNet data plane uses BIER-TE to control which adjacencies are used
   for a given packet. This is signaled from the path ingress, which sets the
   appropriate bits in the BIER BitString to indicate which replication must
   happen.
  </t><t>
   The replication point clears the bit associated to the adjacency where the
   replica is placed, and the elimination points perform a logical AND of the
   BitStrings of the copies that it gets before forwarding.   
  </t><t>     
   As is apparent in the examples above, clearing the bits enables to trace a
   packet to the replication points that made any particular copy. BIER-TE also
   enables to detect the failing adjacencies or sequences of adjacencies along a
   path and to activate additional replications to counter balance the failures.
    </t><t>
   Finally, using the same BIER-TE bit for both directions of the steps of the
   ladder enables to avoid replication in both directions along the crossing
   adjacencies. At the time of sending along the step of the ladder, the bit may
   have been already reset by performing the AND operation with the copy from
   the other side, in which case the transmission is not needed and does not
   occur (since the control bit is now off).
  </t>

  </section>
  
  
  
 
    <!-- section title="Analysis and Discussion">
      <t><list style="hanging">
          <t hangText="#? DetNet Service Interface (M)">
            <vspace blankLines="1"/> 
            To be defined
            <vspace blankLines="1"/> 
          </t>
          <t hangText="#1 Encapsulation and overhead (W/M)">
            <vspace blankLines="1"/>
            The size of the BIER header depends on the number of segments in the
            particular path. It is very concise considering the amount of
            information that is carried (control of replication, traceability,
            and measurement of the reliability of the segments).

            <vspace blankLines="1"/> 
          </t>
          <t hangText="#2 Flow identification (N)">
            <vspace blankLines="1"/>
           Some fields in the BIER header could be used to identify the flows
           but they are not the primary purpose, so it's probably not a good
           idea.
            <vspace blankLines="1"/> 
          </t>

          <t hangText="#4 Explicit routes (N)">
            <vspace blankLines="1"/>
            A separate procedure must be used to set up the paths and allocate
            the bits for the adjacencies. The bits should be distributed as a
            form of tag by the route setup protocol. This procedure requires
            more work and is separate from the dataplane method that is
            described here.
            <vspace blankLines="1"/> 
          </t>
          <t hangText="#5 Packet replication and deletion (M/W)">
            <vspace blankLines="1"/>
            The bitmap expresses in a very concise fashion which replication and elimination should take place for a given packet . It also enables
            to control that process on a per packet basis, depending on the loss
            that it enables to measure. The net result is that a complex path
            may be installed with all the possibilities and that the decision
            of which possibilities are used is controlled in the dataplane.
            <vspace blankLines="1"/> 
          </t>
          <t hangText="#6 Operations, Administration and Maintenance (W)">
            <vspace blankLines="1"/>
            The setting of the bits at arrival enables to determine which
            adjacencies worked and which did not, enabling a dynamic control of
            the replication and elimination process. This is a form of OAM that
            is in-band with the data stream as opposed to leveraging separate 
            packets, which is a more accurate information on the reliability of
            the link for the user.
            <vspace blankLines="1"/> 
          </t>
          <t hangText="#7 Time synchronization (N)">
            <vspace blankLines="1"/>
            Not provided
            <vspace blankLines="1"/> 
          </t>
          <t hangText="#8 Class and quality of service capabilities (N)">
            <vspace blankLines="1"/>
            BIER-TE does not signal that explicitly.
            <vspace blankLines="1"/> 
          </t>
          <t hangText="#9 Packet traceability (W)">
            <vspace blankLines="1"/>
            This is a strong point of the solution. The solution enables to
            determine which is the current segment that a given packet is
            expected to traverse, which node performed the replication and 
            which should perform the elimination if any
            <vspace blankLines="1"/> 
            
          </t>
          <t hangText="#10 Technical maturity (W/N)">
            <vspace blankLines="1"/>
            Some components of the technology are already proven, e.g. segment
            routing and BIER. Yet, the overall solution has never been deployed
            in Service Provider networks.             
            <vspace blankLines="1"/> 
          </t>
        </list>
      </t>

    </section-->
 
    <section title="Elimination Function (Normative)">

<t>This section defines the normative behavior of the Elimination Function with
optional OAM sub-function.</t>

<t>The Elimination Function is performed logically on reception of
BIER-TE packets. It is therefore not part of the adjacencies or otherwise
assigned to a specific bit. "Logically" means that this specification does
not constrain implementations, especially on multi-linecard/multi-chassis systems
to perform EF on a physcial egres module. It just implies that it has to
happen before replication to the bits in the bitstring.</t>

<t>TBD: In addition to being an ingres, EF could as well be modelled as a new
adjacency asigned to bits. The full adjacency of a bit could then be a
sequence of EF followed by one (or more) of existing adjacencies. This is currently
not considered by this document due to the lack of identified need to support
this option - e.g.: problems that can not be equally/better be solved with EF
logically on ingres.</t>

<t>The Elimination Function is more formally written as EF(OAM, BIFT, {flows}/*),
and is configured like BIFTs from the BIER-TE controller host and/or other future
mechanisms.</t>

<t>OAM is boolean and indicates whether OAM function of bitwise AND of received packet copies 
is performed. This OAM function requires additional memory/processing over EF without
OAM. Note that the OAM function does not change the effect of the Elimination Function
for BFR/receivers - they will continue to just receive the first copy of a packet. Instead,
it will continue to track further copies solely for the purpose of providing OAM
information. This also requires some timout or sequence number advancement to decide
when to terminate waiting for further copies of packets before considering the OAM
analysis of this packet to be complete.  BFR supporting this document SHOULD support the 
OAM sub-function.</t>

<t>BIFT indicates the &lt;SD,SI,BSL> for which to perform EF. Devices SHOULD support
enabling per EF. {flows}/* indicates the set of flows for which EF operates (using
the specified BIFT).  Duplicate elimination has to create per-flow state to remember
 which sequence number packets for this flow where already received. In the case of
OAM also what bits where set in that received prior copy of the packet.</t>

<t>When a device supports "*", then it will automatically
allocate such a flow-state for every new recognized flow and expire such flow state
after an operator determined timeout of activity - for example with a default of 10 seconds.
Dynamic allocation of flow-state may cause some inital duplicates before this state
is working and it makes the BFR more vulnerable to state DOS attacks, but it will
allows BIER applications to send flows with the benefit of EF without the help
of the controller having to know and program every flow.</t>

<t>In the {flows} option, control procedures (e.g.: BIER-TE controller host) indicate to
the BFR explicitly the set of flows for which it should install/operate the EF function.
Note that the flow-id in the extended BIER encapsulation is the combination of BFIR-ID and
entropy field of the BIER header.</t>

<t> BFR supporting this document MUST support the {flows} option and MAY support the "*" option.</t>

<t>The following picture explains the results of EF being performed on ingres in a typical example:</t>

<figure anchor="EFrings" align="center"
title="EF with Rings">
<artwork align="center"><![CDATA[
                     I1
                     | 
                     v 
         /---------- B1 ------------\
         |                          |
         \-- B3 -- B4 -- B5 -- B6 --/
              |     |    |     |
              |     |    |     |
             O1    O2    O3    O4
]]></artwork>
</figure>

<t> Consider a simple ring where BFIR I1 generates BIER-TE packets. The bitstring
indicates that the packet is sent hop-by-hop counterclockwise 
B1->B3->B4->B6 and counterclockwise B1->B6->B5->B4->B3. Bits for BFER O1, O2, O3 and
O4 are also set. B3,B4,B5,B6,B7 perform EF. The result
of this setup is that B2 creates two copies of the packets received from I1, one going
to B6, the other to B3. Assume B4 first received the counter-clockwise copy from
 B3 and B5 the clockwise copy from B6. They will both forward these packets to each
other because those where the first copies they saw, but the would block these
second copies. Therefore only the link B4&lt;->B5 will have carried the packet
copy twice (once in each direction). All the other ring links will only
carry one copy of the packet.</t>

<t>This is notably different from schemes where EF is not performed before
replication, but afterwards. In those schemes, both copies of the packets
would flow counterclockwise around (most of) the ring, ocupying more bandwidth.</t>

    </section>

    <section title="Summary">
      <t>
      With the addition of the functions of this document, BIER-TE becomes a potential option for the DetNet dataplane specifically beneficial when PREF (replication and elimination) is required for resilience (to reduce packet loss). For DetNet multicast but also DetNet unicast. 
      The unique capabilities of this approach areare:
      <list style="symbols">
      <t>Explicit per-packet path selection for packet. Multicast and Unicast.</t>
      <t>Control which replication take place on a per packet basis, so that
      replication points can be configured but not actually utilized</t>
      <t>Trace the replication activity and determine which node replicated a
      particular packet</t>
      <t>Measure the quality of transmission of the actual data packet along
      the replication segments and use that in a control loop to adapt the
      setting of the bits and maintain the reliability.</t>
      </list>
      </t>
    </section>

<section anchor="impl" title="Implementation Status">
<t> A research-stage implementation of the forwarding plane fir a 6TiSCH IOT use case
    was developed at Cisco's Paris Innovation Lab (PIRL) by Zacharie Brodard. 
	It was implemented on OpenWSN Open-source firmware and tested on the OpenMote-CC2538
	hardware. It implements the header types 15,16, 17, 18 and 19 (bit-by-bit encoding
	without group ID) in order to allow a BIER-TE protocol over IEE802.15.4e.

</t><t>
    This work was complemented with a Controller-based control loop by Hao Jiang. 
	The controller builds the complex paths (called Tracks in 6TiSCH) and decides
	the setting oif the BitStrings in real time in order to optimize the delivery ratio
	within a minimal energy budget. 
</t><t>
   Links: 
   <list>
   <t>
   github: https://github.com/zach-b/openwsn-fw/tree/BIER
   </t><t>  
   OpenWSN firmware: https://openwsn.atlassian.net/wiki/pages/viewpage.action?pageId=688187
   </t><t>  
   OpenMote hardware: http://www.openmote.com/ 
   </t>
   </list>
</t>   
</section>


<section title="Security considerations">
  <t>TBD.
  </t>
</section>


<section anchor="iana" title="IANA Considerations">
  <t>This document has no IANA considerations.
  </t>
</section>

<section anchor="acks" title="Acknowledgements">

  <t> The method presented in this document was discussed and worked out together with the DetNet Data Plane Design Team:
  <list style="bullets">
   <t>Jouni Korhonen</t>
   <t>J&aacute;nos Farkas</t>
   <t>Norman Finn</t>
   <t>Olivier Marce</t>
   <t>Gregory Mirsky</t>
   <t>Pascal Thubert</t>
   <t>Zhuangyan Zhuang</t>
  </list></t>
  <t>
   The authors also like to thank the DetNet chairs Lou Berger and Pat Thaler,
   as well as Thomas Watteyne, 6TiSCH co-chair, for their contributions
   and support to this work.</t>
</section>
</middle>

    <back>
    <references title='Normative References'>
	  <?rfc include="reference.RFC.2119"?>
     
    </references>
    <references title='Informative References'>

	   <!--?rfc include="reference.RFC.6282"?-->
      <!--?rfc include='reference.I-D.bergmann-bier-ccast'?-->
      <!--?rfc include='reference.I-D.thubert-6tisch-4detnet'?-->
      <!--?rfc include='reference.I-D.thubert-6lo-bier-dispatch'?-->
      <?rfc include='reference.I-D.ietf-detnet-architecture'?>
      <?rfc include='reference.RFC.8279'?>
      <?rfc include='reference.I-D.ietf-bier-te-arch'?>
	  <?rfc include='reference.I-D.ietf-opsawg-oam-overview'?>
	  <?rfc include='reference.I-D.ietf-detnet-problem-statement'?>
	  <?rfc include='reference.I-D.ietf-detnet-use-cases'?>
      <!--?rfc include='reference.I-D.ietf-6tisch-architecture'?-->
      <?rfc include='reference.I-D.dt-detnet-dp-alt'?>
	  <?rfc include='reference.I-D.ietf-spring-segment-routing'?>
      

    </references>
 </back>
</rfc>

