<?xml version="1.0" encoding="US-ASCII"?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd">
<?rfc toc="yes"?>
<?rfc tocompact="yes"?>
<?rfc tocdepth="3"?>
<?rfc tocindent="yes"?>
<?rfc symrefs="yes"?>
<?rfc sortrefs="yes"?>
<?rfc comments="yes"?>
<?rfc inline="yes"?>
<?rfc compact="yes"?>
<?rfc subcompact="no"?>
<rfc category="std" docName="draft-song-ippm-ioam-ipv6-support-02" ipr="trust200902">
  <front>
    <title abbrev="IOAM IPv6 Support">Approaches on Supporting IOAM in IPv6</title>

    <author fullname="Haoyu Song" initials="H." surname="Song">
      <organization>Futurewei</organization>
      <address>
        <postal>
          <street></street>
          <city></city>
          <code></code>
          <country>USA</country>
        </postal>
        <email>haoyu.song@futurewei.com</email>
      </address>
    </author>

    <author fullname="Zhenbin Li" initials="Z. " surname="Li">
      <organization>Huawei Technologies</organization>
      <address>
        <postal>
          <street></street>
          <city></city>
          <code></code>
          <country>China</country>
        </postal>
        <email>lizhenbin@huawei.com</email>
      </address>
    </author>

    <author fullname="Shuping Peng" initials="S. " surname="Peng">
      <organization>Huawei Technologies</organization>
      <address>
        <postal>
          <street></street>
          <city></city>
          <code></code>
          <country>China</country>
        </postal>
        <email>pengshuping@huawei.com</email>
      </address>
    </author>

	<author fullname="James Guichard" initials="J. " surname="Guichard">
      <organization>Futurewei</organization>
      <address>
        <postal>
          <street></street>
          <city></city>
          <code></code>
          <country>USA</country>
        </postal>
        <email>james.n.guichard@futurewei.com</email>
      </address>
    </author>

    <date day="4" month="January" year="2021"/>
	
	<area>OPS</area>

    <workgroup>IPPM</workgroup>

    <abstract>
      <t>It has been proposed to encapsulate IOAM tracing option data fields in IPv6 HbH options header. 
      However, due to size of the trace data and the extension header location in the IPv6 packets, 
	  the proposal creates practical challenges for implementation, especially when other extension headers,
	  such as a routing header, also exist and require in-network processing. 
	  We propose several alternative approaches to address this challenge, including  
	  separating the IOAM trace data to a different extension header, using the postcard-based telemetry (e.g., IOAM DEX) instead,
	  and applying the segment IOAM trace export scheme, based on the network scenario and application requirements.
	  We discuss the pros and cons of each approach and hope to foster standard convergence and industry adoption. </t> 
	  
    </abstract>

    <note title="Requirements Language">
      <t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
      "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
      document are to be interpreted as described in <xref
      target="RFC2119">RFC 2119</xref>.</t>
    </note>
  </front>

  <middle>
    <section title="Introduction">
      <t>In-situ OAM (IOAM) <xref target="I-D.ietf-ippm-ioam-data"/> defines two tracing options, pre-allocated tracing option and incremental tracing option,
	  which record hop-by-hop data along a packet's forwarding path. 
	  The draft <xref target="I-D.ietf-ippm-ioam-ipv6-options"/> proposes the method to encapsulate IOAM trace option data fields 
      in IPv6. Because the tracing options requires per hop processing, such options can only be encapsulated in IPv6 Hop-by-Hop (HbH)
      options header. The draft <xref target="I-D.ioametal-ippm-6man-ioam-ipv6-deployment"/> further describes some deployment approaches.</t>

	  <t><xref target="RFC8200"/> mandates that the HbH options header, if exists, must be the first extension header following the IPv6 header.
      However, the IOAM trace data can be large, which easily amount to tens to hundreds of bytes, making accessing other headers after it difficult
	  or even impossible.
      There are practical limitations on how far the hardware can reach into a packet in forwarding hardware. The IOAM tracing option cannot 
	  be applied if it makes other extension headers inaccessible. Even if the other headers can be
      reached, the deeper they are, the higher the cost to access and process them, and the lower the forwarding performance.	  
      A potentially more detrimental issue is that the incremental tracing option will expand the HbH header at each hop and push back all other headers, 
	  which keeps shifting the locations of the other extension headers, further complicating the hardware implementation and impeding the forwarding.</t>

      <t>The issue becomes more severe when SRv6 and IOAM coexist. The Segment Routing Extension Header (SRH) <xref
      target="I-D.ietf-6man-segment-routing-header"/> is encapsulated in a routing header which is after the HbH options header.
      SRH itself can be large. It requires read and write operations at each SRv6 node. If it is deeply embedded in a packet 
	  and its location keeps shifting, either it is beyond the reach of hardware or the forwarding performance degrades.</t>

      <t>We can avoid the problem by simply not using both at the same time, but apparently this is not ideal, because IOAM is an important OAM tool and it is
      even more wanted when SRv6 brings more operational complexity into IPv6 networks.</t>

      <t>Our second recourse is to limit the IOAM to SRv6 nodes only. That is, consider SRv6 as an overlay tunnel over IPv6 and apply the IOAM pipe mode as discussed in
	  <xref target="I-D.song-ippm-ioam-tunnel-mode" />, which only collects data at each SRv6 nodes. To realize this, <xref target="I-D.ali-spring-ioam-srv6" /> describes 
	  an approach that encapsulates the IOAM option data fields in an SRH TLV. <xref target="I-D.song-6man-srv6-pbt"/> and 
	  <xref target="I-D.ietf-6man-spring-srv6-oam"/> 
	  describe another approach to enable 
      postcard-based telemetry for SRv6 without needing IOAM option encapsulation. 
	  In either case, the SRH is close to the packet front and its location is fixed.
	  <xref target="I-D.song-spring-siam" /> proposes to support IOAM in the payload of the dedicated SRv6 probing packets only.  
	  While these approaches are useful for use cases that only need to monitor the segment end points, it fails to cover all the IPv6 nodes in a network.</t>   	  

      <t>So the proposition of this draft is, suppose we need to apply IOAM on all nodes in an SRv6 network, how we can amend the approach in
	  <xref target="I-D.ietf-ippm-ioam-ipv6-options"/> or use alternative approaches to circumvent the aforementioned issues. In this draft, we 
	  propose three such approaches:  
	  (1) separating the IOAM trace data to a different extension header, (2) using the postcard-based telemetry (e.g., IOAM DEX) instead,
	  and (3) applying the segment IOAM trace export scheme. 
	  We discuss the pros and cons of each approach and hope to foster standard convergence and industry adoption. </t>
   	  
    </section>

    <section title="IOAM Data Separate and Postpose">
      <t>An IOAM trace type data fields contain two parts: instruction and trace data. Although by convention the trace data part immediately follow 
      the instruction part, there is not fundamental reason why these two parts must stick together. This observation provides us an optimization 
	  opportunity to amend the original proposal in <xref target="I-D.ietf-ippm-ioam-ipv6-options"/>. </t>

  	  <t>We separate the IOAM trace type data fields into the instruction part and the trace data part. We encapsulate only the instruction part in the 
	  HbH options header, and encapsulate the trace data part in another metadata extension header after all the IPv6 extension headers
	  and before upper layer protocol headers. 
	  This arrangement allows high performance hardware implementation. When using the incremental data trace, even if the data trace size increases at
      each node, all IPv6 extension headers remain intact and new data is inserted at a fixed location.</t>

      <t>Figure 1 shows the HbH option format for IOAM trace type instruction. The field specification is identical to that in <xref target="RFC8200" /> and 
	  <xref target="I-D.ietf-ippm-ioam-data"/>. </t>
	  
        <t><figure anchor="figure_1" title="HbH Option Format for IOAM Trace Type Instruction">
            <artwork><![CDATA[	  
  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |  Option Type  |  Opt Data Len |   Reserved    |   IOAM Type   |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<------+
 |        Namespace-ID           |NodeLen  | Flags | RemainingLen| IOAM 
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Trace 
 |               IOAM-Trace-Type                 |  Reserved     | Type
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<Instr.+                                                                 
        ]]></artwork>
          </figure></t>
	  
	<t>Figure 2 shows the TLV option format for IOAM trace type data. The IOAM trace type data format is compliant with <xref target="I-D.ietf-ippm-ioam-data" />.</t>  
	  
	  <t><figure anchor="figure_2" title="Option Format for IOAM Trace Type Data">
            <artwork><![CDATA[	 
  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |   IOAM Type   |     Length    |            Reserved           |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |                                                               |
 ~                      IOAM Trace Type Data                     ~
 ~                                                               ~ 
 |                                                               |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+	
]]></artwork>
          </figure></t>
	
	  <section title="IOAM Trace Data Encapsulation">
	    <t>We have basically two methods to encapsulate the IOAM trace data. First, we can define a new IPv6 extension header which is dedicated 
           to metadata. Once standardized, this extension header can also be used to host potential metadata from other applications such as NSH for SFC
           <xref target="RFC8300" />. Second, this option can be carried as a TLV option in another existing extension header such as the destination header.
		   The only requirement is that this extension header should be the last one in the extension header chain. 
           The first method is cleaner but it requires extra standard effort; the second method is simpler but it needs to overcome the access constraints exerted
           by <xref target="RFC8300" />.</t>		   
	  </section>	
	</section>
		  
	<section title="Segment IOAM Data Export">
	   <t>If the overhead of the IOAM trace type data fields is under control, we may still manage to 
        encapsulate both instruction and data in HbH options header as in <xref target="I-D.ietf-ippm-ioam-ipv6-options"/>. To this end, we introduce
        two sub-approaches.</t>		
		
	   <section title="Independent of SRv6">
	      <t><xref target="I-D.song-ippm-segment-ioam"/> proposes an enhancement to IOAM trace type which can configure the allowable overhead of the IOAM
          trace type data fields. Once the trace data size is up to the limit at a network node (i.e., a segment or a fixed number of network nodes are 
		  traversed), the trace data will be stripped and exported 
		  so room is made to accommodate new trace data from nodes in the next segment of the forwarding path.</t>   		  

          <t>This approach requires some moderate updates to the IOAM trace type data fields, as described in <xref target="I-D.song-ippm-segment-ioam"/>.
		  Figure 3 shows the format of the HbH Option Header containing Segment IOAM trace type data fields. A flag bit (#23) in
          the Flags field is used to indicate the current header is a segment
          IOAM header.  In this context, the last octet in the IOAM header is
          partitioned into two 4-bit nibbles.  The first nibble (SSize) is used
          to save the segment size and the second nibble (RHop) is used to save
          the remaining hops. This limits the maximum segment size to 15.
		  </t>	

 <t><figure anchor="figure_3" title="HbH Option Format for Segment IOAM Trace Type Data Fields">
            <artwork><![CDATA[	 
   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |  Option Type  |  Opt Data Len |   Reserved    |   IOAM Type   |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<------+
  |       Namespace-ID            |NodeLen|Flags|1| SSize | RHop  | IOAM 
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Segment
  |               IOAM-Trace-Type                 |  Reserved     | Trace
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Type
  |                                                               | Data 
  |                  Node Data List []                            | Fields
  |                                                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<------+
]]></artwork>
          </figure></t>

    <t> At the beginning of each segment, the
   segment size (SSize) and the remaining hops (RHop) are initialized: RHop is set to equal to SSize.  At each
   hop, if RHop is not zero, the node data is added to the node data list
   and then RHop is decremented by 1.  If RHop
   is equal to 0 when receiving the packet, the node needs to remove (in
   incremental trace option) or clear (in pre-allocated trace option)
   the IOAM node data list and reset RHop to SSize.</t> 
   
   <t>In this case, if we use the IOAM pre-allocated trace type, the size and location of each IPv6 extension header is fixed and predictable, 
		  and the hardware capability and performance can be guaranteed.</t>
		  
	   </section>
	   
	   <section title="Export at SRv6 node">
	      <t>Whenever a packet with the IOAM option reaches a SRv6 node which needs to access the SRH, we can configure the node to export immediately the
          IOAM trace data accumulated so far. In this case, basically at each SRv6 node, the HbH header size is fixed and the header contains an IOAM option with only
          the instruction part. After the SRH processing, this node can add local IOAM trace data in the HbH option header before forwarding the packet.</t>

          <t>The incremental trace type can be used in this approach. In an extreme case when every node is also an SRv6 node, this approach regresses to a 
		  per-hop postcard-based telemetry approach as described in <xref target="I-D.song-ippm-postcard-based-telemetry"/>. In this case, the HbH option 
		  for IOAM can even be avoided altogether if we can find a way to simply mark the packet for postcard export.</t>   		  	   
       </section>			
	</section>
		  
	<section title="Direct Export Option">
	
	   <t>As an embodiment of the PBT-I approach introduced in <xref target="I-D.song-ippm-postcard-based-telemetry"/>, IOAM Direct Export (DEX) Option Type 
	   discussed in <xref target="I-D.ioamteam-ippm-ioam-direct-export" /> can be used to replace the IOAM trace type. IOAM DEX only needs to encapsulate
       a fix-size instruction header in the HbH option header. </t>
	   
	   <t>Figure 4 shows the HbH option format for IOAM DEX type fields. The field specification is identical to that in <xref target="RFC8200" /> and 
	  <xref target="I-D.ioamteam-ippm-ioam-direct-export" />. </t>
	  
 <t><figure anchor="figure_4" title="HbH Option Format for IOAM DEX Type Fields">
            <artwork><![CDATA[	  
    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Option Type  |  Opt Data Len |   Reserved    |   IOAM Type   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<------+
   |        Namespace-ID           |           Flags               |   
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ IOAM 
   |               IOAM-Trace-Type                 |  Reserved     | DEX
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Type
   |                         Flow ID (optional)                    | Fields
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     Sequence Number  (Optional)               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<------+
]]></artwork>
</figure></t>
	     
	
	</section>
	  
	<section title="Comparison">

<t>The following table compares the existing approach and the four other alternative approaches proposed in this draft. </t>
	
<t><figure anchor="figure_5" title="Comparison of Different Approaches">
            <artwork><![CDATA[
   +--------------+-------------------------+--------------------------+ 
   |  Approach    |   Pros                  |   Cons                   |  
   |              |                         |                          |  
   +--------------+-------------------------+--------------------------+ 
   |IOAM Trace    |Comply w/ IOAM Data Spec |Variable and long HbH     |  
   |in HbH        |                         |header impeding access of |
   |              |                         |later extension headers   |
   +--------------+-------------------------+--------------------------+     	  
   |IOAM Trace    |Fix-size and short HbH   |Need extra extension      |
   |Data Separate |header, good for later   |header to hold trace data |
   |and Postpose  |extension header access  |                          |
   |(Sec. 2)      |                         |                          |
   +--------------+-------------------------+--------------------------+ 
   |Segment IOAM  |Fix-size and controllable|Need to enhance IOAM trace|
   |Data Export   |HbH header size          |type data field spec.     | 
   |(Sec. 3.1)    |                         |                          |
   +--------------+-------------------------+--------------------------+ 
   |Trace Export  |Can be done through      |Specific to SRv6;         |
   |at SRv6 nodes |configuration            |No better than PB & IOAM  | 
   |(Sec. 3.2)    |                         |DEX in the worst case     |  
   +--------------+-------------------------+--------------------------+ 
   |IOAM Direct   |Comply w/ IOAM DEX Spec; |Need export data          |
   |Export in HbH |Fix-size and short HbH   |correlation               | 
   |(Sec. 4)      |                         |                          |
   +--------------+-------------------------+--------------------------+ 
]]></artwork>
	  </figure></t>
	
    </section>

    <section anchor="Security" title="Security Considerations">
      <t>TBD.</t>
    </section>
  </middle>

  <back>
    <references title="Normative References">
      <?rfc include="reference.RFC.2119"?>

      <?rfc include='reference.RFC.8300'?>

      <?rfc include='reference.RFC.8200'?>

      <?rfc include='reference.I-D.ietf-ippm-ioam-data'?>
	  
	  <?rfc include='reference.I-D.ioametal-ippm-6man-ioam-ipv6-deployment'?>

      <?rfc include='reference.I-D.ietf-ippm-ioam-ipv6-options'?>

      <?rfc include='reference.I-D.ietf-6man-segment-routing-header'?>
	  
	  <?rfc include='reference.I-D.ioamteam-ippm-ioam-direct-export'?>
	  
	  <?rfc include='reference.I-D.song-ippm-segment-ioam'?>
	  
	  <?rfc include='reference.I-D.song-ippm-postcard-based-telemetry'?>
	  
	  <?rfc include='reference.I-D.song-ippm-ioam-tunnel-mode'?>
	  
	  <?rfc include='reference.I-D.ali-spring-ioam-srv6'?>
	  
	  <?rfc include='reference.I-D.song-6man-srv6-pbt'?>
	  
	  <?rfc include='reference.I-D.ietf-6man-spring-srv6-oam'?>
	  
	  <?rfc include='reference.I-D.song-spring-siam'?>
	  
    </references>

  </back>
</rfc>
