idnits 2.17.1 draft-ietf-6man-spring-srv6-oam-07.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There is 1 instance of too long lines in the document, the longest one being 1 character in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 26, 2020) is 1369 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'SL' is mentioned on line 199, but not defined -- Looks like a reference, but probably isn't: '2' on line 200 -- Looks like a reference, but probably isn't: '1' on line 201 -- Looks like a reference, but probably isn't: '0' on line 201 == Outdated reference: A later version (-07) exists of draft-gandhi-spring-stamp-srpm-01 == Outdated reference: A later version (-11) exists of draft-gandhi-spring-twamp-srpm-09 == Outdated reference: A later version (-28) exists of draft-ietf-spring-srv6-network-programming-16 == Outdated reference: A later version (-15) exists of draft-matsushima-spring-srv6-deployment-status-07 Summary: 1 error (**), 0 flaws (~~), 6 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 6man Z. Ali 3 Internet-Draft C. Filsfils 4 Intended status: Standards Track Cisco Systems 5 Expires: January 27, 2021 S. Matsushima 6 Softbank 7 D. Voyer 8 Bell Canada 9 M. Chen 10 Huawei 11 July 26, 2020 13 Operations, Administration, and Maintenance (OAM) in Segment Routing 14 Networks with IPv6 Data plane (SRv6) 15 draft-ietf-6man-spring-srv6-oam-07 17 Abstract 19 This document describes how the existing IPv6 OAM mechanisms can be 20 used in an SRv6 network. The document also introduces enhancements 21 for OAM mechanisms for SRv6 networks. 23 Status of This Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at https://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on January 27, 2021. 40 Copyright Notice 42 Copyright (c) 2020 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (https://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 58 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 59 1.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 3 60 1.3. Terminology and Reference Topology . . . . . . . . . . . 3 61 2. OAM Mechanisms . . . . . . . . . . . . . . . . . . . . . . . 5 62 2.1. O-flag in Segment Routing Header . . . . . . . . . . . . 5 63 2.1.1. O-flag Processing . . . . . . . . . . . . . . . . . . 5 64 2.2. OAM Operations . . . . . . . . . . . . . . . . . . . . . 7 65 3. Illustrations . . . . . . . . . . . . . . . . . . . . . . . . 7 66 3.1. Ping in SRv6 Networks . . . . . . . . . . . . . . . . . . 8 67 3.1.1. Classic Ping . . . . . . . . . . . . . . . . . . . . 8 68 3.1.2. Pinging a SID . . . . . . . . . . . . . . . . . . . . 9 69 3.2. Traceroute . . . . . . . . . . . . . . . . . . . . . . . 10 70 3.2.1. Classic Traceroute . . . . . . . . . . . . . . . . . 10 71 3.2.2. Traceroute to a SID . . . . . . . . . . . . . . . . . 12 72 3.3. A Hybrid OAM Using O-flag . . . . . . . . . . . . . . . . 13 73 3.4. Monitoring of SRv6 Paths . . . . . . . . . . . . . . . . 16 74 4. Implementation Status . . . . . . . . . . . . . . . . . . . . 17 75 5. Security Considerations . . . . . . . . . . . . . . . . . . . 17 76 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17 77 6.1. Segment Routing Header Flags . . . . . . . . . . . . . . 17 78 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 18 79 8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 18 80 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 19 81 9.1. Normative References . . . . . . . . . . . . . . . . . . 19 82 9.2. Informative References . . . . . . . . . . . . . . . . . 19 83 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 21 85 1. Introduction 87 As Segment Routing with IPv6 data plane (SRv6) [RFC8402] simply adds 88 a new type of Routing Extension Header, existing IPv6 OAM mechanisms 89 can be used in an SRv6 network. This document describes how the 90 existing IPv6 mechanisms for ping and trace route can be used in an 91 SRv6 network. 93 The document also introduces enhancements for OAM mechanism for SRv6 94 networks. Specifically, the document describes an OAM mechanism for 95 performing controllable and predictable flow sampling from segment 96 endpoints using, e.g., IP Flow Information Export (IPFIX) protocol 98 [RFC7011]. The document also outlines how centralized OAM technique 99 in [RFC8403] can be extended for SRv6 to perform a path continuity 100 check between any nodes within an SRv6 domain from a centralized 101 monitoring system. 103 1.1. Requirements Language 105 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 106 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 107 document are to be interpreted as described in [RFC2119], [RFC8174]. 109 1.2. Abbreviations 111 The following abbreviations are used in this document: 113 SID: Segment ID. 115 SL: Segments Left. 117 SR: Segment Routing. 119 SRH: Segment Routing Header [RFC8754]. 121 SRv6: Segment Routing with IPv6 Data plane. 123 TC: Traffic Class. 125 ICMPv6: ICMPv6 Specification [RFC4443]. 127 1.3. Terminology and Reference Topology 129 Throughout the document, the following terminology and simple 130 topology is used for illustration. 132 +--------------------------| N100 |---------------------------------+ 133 | | 134 | ====== link1====== link3------ link5====== link9------ ====== | 135 ||N1||------||N2||------| N3 |------||N4||------| N5 |---||N7|| 136 || ||------|| ||------| |------|| ||------| |---|| || 137 ====== link2====== link4------ link6======link10------ ====== 138 | | | | 139 ---+-- | ------ | --+--- 140 |CE 1| +-------| N6 |---------+ |CE 2| 141 ------ link7 | | link8 ------ 142 ------ 144 Figure 1 Reference Topology 146 In the reference topology: 148 Node k has a classic IPv6 loopback address 2001:DB8:A:k::/128. 150 Nodes N1, N2, N4 and N7 are SRv6 capable nodes. 152 Nodes N3, N5 and N6 are IPv6 nodes that are not SRv6 capable. 153 Such nodes are referred as classic IPv6 nodes. 155 CE1 and CE2 are Customer Edge devices of any data plane capability 156 (e.g., IPv4, IPv6, L2, etc.). 158 A SID at node k with locator block 2001:DB8:B::/48 and function F 159 is represented by 2001:DB8:B:k:F::. 161 Node N100 is a controller. 163 The IPv6 address of the nth Link between node X and Y at the X 164 side is represented as 2001:DB8:X:Y:Xn::, e.g., the IPv6 address 165 of link6 (the 2nd link) between N3 and N4 at N3 in Figure 1 is 166 2001:DB8:3:4:32::. Similarly, the IPv6 address of link5 (the 1st 167 link between N3 and N4) at node 3 is 2001:DB8:3:4:31::. 169 2001:DB8:B:k:Cij:: is explicitly allocated as the END.X SID (refer 170 [I-D.ietf-spring-srv6-network-programming]) at node k towards 171 neighbor node i via jth Link between node i and node k. e.g., 172 2001:DB8:B:2:C31:: represents END.X at N2 towards N3 via link3 173 (the 1st link between N2 and N3). Similarly, 2001:DB8:B:4:C52:: 174 represents the END.X at N4 towards N5 via link10. 176 A SID list is represented as where S1 is the first 177 SID to visit, S2 is the second SID to visit and S3 is the last SID 178 to visit along the SR path. 180 (SA,DA) (S3, S2, S1; SL)(payload) represents an IPv6 packet with: 182 * IPv6 header with source address SA, destination addresses DA 183 and SRH as next-header 185 * SRH with SID list with SegmentsLeft = SL 187 * Note the difference between the < > and () symbols: represents a SID list where S1 is the first SID and S3 is 189 the last SID to traverse. (S3, S2, S1; SL) represents the same 190 SID list but encoded in the SRH format where the rightmost SID 191 in the SRH is the first SID and the leftmost SID in the SRH is 192 the last SID. When referring to an SR policy in a high-level 193 use-case, it is simpler to use the notation. When 194 referring to an illustration of the detailed packet behavior, 195 the (S3, S2, S1; SL) notation is more convenient. 197 * (payload) represents the the payload of the packet. 199 SRH[SL] represents the SID pointed by the SL field in the first 200 SRH. In our example SID list (S3, S2, S1; SL), SRH[2] represents 201 S1, SRH[1] represents S2 and SRH[0] represents S3. 203 2. OAM Mechanisms 205 This section defines OAM enhancement for the SRv6 networks. 207 2.1. O-flag in Segment Routing Header 209 [RFC8754] describes the Segment Routing Header (SRH) and how SR 210 capable nodes use it. The SRH contains an 8-bit "Flags" field. This 211 document defines the following bit in the SRH.Flags to carry the 212 O-flag: 214 0 1 2 3 4 5 6 7 215 +-+-+-+-+-+-+-+-+ 216 | |O| | 217 +-+-+-+-+-+-+-+-+ 219 Where: 221 O-flag: OAM flag. 223 The document does not define any other flag in the SRH.Flags and 224 meaning and processing of any other bit in SRH.Flags is outside of 225 the scope of this document. 227 2.1.1. O-flag Processing 229 The O-flag in SRH is used as a marking-bit in the user packets to 230 trigger the telemetry data collection and export at the segment 231 endpoints. 233 This document does not specify the data elements that needs to be 234 exported and the associated configurations. Similarly, this document 235 does not define any formats for exporting the data elements. 236 Nonetheless, without the loss of generality, this document assumes IP 237 Flow Information Export (IPFIX) protocol [RFC7011] is used for 238 exporting the traffic flow information from the network devices to a 239 controller for monitoring and analytics. Similarly, without the loss 240 of generality, this document assumes requested information elements 241 are configured by the management plane through data set templates 242 (e.g., as in IPFIX [RFC7012]). 244 Implementation of the O-flag is OPTIONAL. If a node does not support 245 the O-flag, then upon reception it simply ignores it. If a node 246 supports the O-flag, it can optionally advertise its potential via 247 control plan protocol(s). 249 When N receives a packet whose IPv6 DA is S and S is a local SID, the 250 line S01 of the pseudo-code associated with the SID S, as defined in 251 section 4.3.1.1 of [RFC8754], is modified as follows for the O-flag 252 processing. 254 S01.1. IF SRH.Flags.O-flag is set and local configuration permits 255 O-flag processing THEN 256 a. Make a copy of the packet. 257 b. Send the copied packet, along with a timestamp 258 to the OAM process for telemetry data collection 259 and export. ;; Ref1 260 Ref1: An implementation SHOULD copy and record the timestamp as 261 soon as possible during packet processing. Timestamp or any other 262 metadata is not 263 carried in the packet forwarded to the next hop. 265 Please note that the O-flag processing happens before execution of 266 regular processing of the local SID S. 268 Based on the requested information elements configured by the 269 management plane through data set templates [RFC7012], the OAM 270 process exports the requested information elements. The information 271 elements include parts of the packet header and/or parts of the 272 packet payload for flow identification. The OAM process uses 273 information elements defined in IPFIX [RFC7011] and PSAMP [RFC5476] 274 for exporting the requested sections of the mirrored packets. 276 If the telemetry data from the last node in the segment-list (egress 277 node) is desired, the ingress uses an Ultimate Segment Pop (USP) SID 278 advertised by the egress node. 280 The processing node SHOULD rate-limit the number of packets punted to 281 the OAM process to avoid hitting any performance impact. 283 The OAM process MUST NOT process the copy of the packet or respond to 284 any upper-layer header (like ICMP, UDP, etc.) payload to prevent 285 multiple evaluations of the datagram. 287 Specification of the OAM process or the external controller 288 operations are beyond the scope of this document. How to correlate 289 the data collected from different nodes at an external controller is 290 also outside the scope of the document. Section 3 illustrates use of 291 the SRH.Flags.O-flag for implementing a hybrid OAM mechanism, where 292 the "hybrid" classification is based on RFC7799 [RFC7799]. 294 2.2. OAM Operations 296 IPv6 OAM operations can be performed for any SRv6 SID whose behavior 297 allows Upper Layer Header processing for an applicable OAM payload 298 (e.g., ICMP, UDP). 300 Ping to a SID is used for SID connectivity checks and to validate the 301 availability of a SID. Traceroute to a SID is used for hop-by-hop 302 fault localization as well as path tracing to a SID. Section 3 303 illustrates the ICMPv6 based ping and the UDP based traceroute 304 mechanisms for ping and traceroute to an SRv6 SID. Although this 305 document only illustrates ICMP ping and UDP-based traceroute to an 306 SRv6 SID, the procedures are equally applicable to other IPv6 OAM 307 probing to an SRv6 SID (e.g., Bidirectional Forwarding Detection 308 (BFD) [RFC5880], Seamless BFD (SBFD) [RFC7880], TWAMP Light and STAMP 309 probe message processing as described in 310 [I-D.gandhi-spring-twamp-srpm] and [I-D.gandhi-spring-stamp-srpm], 311 respectively, etc.). Specifically, as long as local configuration 312 allows the Upper-layer Header processing of the applicable OAM payload 313 for SRv6 SIDs, the existing IPv6 OAM techniques can be used to target 314 a probe to a (remote) SID. 316 IPv6 OAM operations can be performed with the target SID in the IPv6 317 destination address without SRH or with SRH where the target SID is 318 the last segment. In general, OAM operations to a target SID may not 319 exercise all of its processing depending on its behavior definition. 320 For example, ping to an END.X SID (refer [I-D.ietf-spring-srv6- 321 network-programming]) at the target node only validates availability 322 of the SID and does not validate switching to the correct outgoing 323 interface. To exercise the behavior of a target SID, the OAM 324 operation SHOULD construct the probe in a manner similar to a data 325 packet that exercises the SID behavior, i.e. to include that SID as a 326 transit SID in either an SRH or IPv6 DA of an outer IPv6 header or as 327 appropriate based on the definition of the SID behavior. 329 3. Illustrations 331 This section shows how some of the existing IPv6 OAM mechanisms can 332 be used in an SRv6 network. It also illustrates an OAM mechanism for 333 performing controllable and predictable flow sampling from segment 334 endpoints. How centralized OAM technique in [RFC8403] can be 335 extended for SRv6 is also described in this Section. 337 3.1. Ping in SRv6 Networks 339 The following subsections outline some use cases of the ICMP ping in 340 the SRv6 networks. 342 3.1.1. Classic Ping 344 The existing mechanism to perform the connectivity checks, along the 345 shortest path, continues to work without any modification. The 346 initiator may be an SRv6 node or a classic IPv6 node. Similarly, the 347 egress or transit may be an SRv6 capable node or a classic IPv6 node. 349 If an SRv6 capable ingress node wants to ping an IPv6 address via an 350 arbitrary segment list , it needs to initiate ICMPv6 ping 351 with an SR header containing the SID list . This is 352 illustrated using the topology in Figure 1. Assume all the links 353 have IGP metric 10 except both links between node2 and node3, which 354 have IGP metric set to 100. User issues a ping from node N1 to a 355 loopback of node 5, via segment list <2001:DB8:B:2:C31::, 356 2001:DB8:B:4:C52::>. The SID behavior used in the example is End.X 357 SID (refer [I-D.ietf-spring-srv6-network-programming]) but the 358 procedure is equally applicable to any other (transit) SID type. 360 Figure 2 contains sample output for a ping request initiated at node 361 N1 to the loopback address of node N5 via a segment list 362 <2001:DB8:B:2:C31::, 2001:DB8:B:4:C52::>. 364 > ping 2001:DB8:A:5:: via segment-list 2001:DB8:B:2:C31::, 365 2001:DB8:B:4:C52:: 367 Sending 5, 100-byte ICMP Echos to B5::, timeout is 2 seconds: 368 !!!!! 369 Success rate is 100 percent (5/5), round-trip min/avg/max = 0.625 370 /0.749/0.931 ms 372 Figure 2 A sample ping output at an SRv6 capable node 374 All transit nodes process the echo request message like any other 375 data packet carrying SR header and hence do not require any change. 376 Similarly, the egress node (IPv6 classic or SRv6 capable) does not 377 require any change to process the ICMPv6 echo request. For example, 378 in the ping example of Figure 2: 380 o Node N1 initiates an ICMPv6 ping packet with SRH as follows 381 (2001:DB8:A:1::, 2001:DB8:B:2:C31::) (2001:DB8:A:5::, 382 2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::, SL=2, NH = ICMPv6)(ICMPv6 383 Echo Request). 385 o Node N2, which is an SRv6 capable node, performs the standard SRH 386 processing. Specifically, it executes the END.X behavior 387 (2001:DB8:B:2:C31::) and forwards the packet on link3 to N3. 389 o Node N3, which is a classic IPv6 node, performs the standard IPv6 390 processing. Specifically, it forwards the echo request based on 391 the DA 2001:DB8:B:4:C52:: in the IPv6 header. 393 o Node N4, which is an SRv6 capable node, performs the standard SRH 394 processing. Specifically, it observes the END.X behavior 395 (2001:DB8:B:4:C52::) and forwards the packet on link10 towards N5. 396 If 2001:DB8:B:4:C52:: is a PSP SID, The penultimate node (Node N4) 397 does not, should not and cannot differentiate between the data 398 packets and OAM probes. Specifically, if 2001:DB8:B:4:C52:: is a 399 PSP SID, node N4 executes the SID like any other data packet with 400 DA = 2001:DB8:B:4:C52:: and removes the SRH. 402 o The echo request packet at N5 arrives as an IPv6 packet with or 403 without an SRH. If N5 receives the packet with SRH, it skips SRH 404 processing (SL=0). In either case, Node N5 performs the standard 405 IPv6/ ICMPv6 processing on the echo request. 407 3.1.2. Pinging a SID 409 The classic ping described in the previous section applies equally to 410 perform SID connectivity checks and to validate the availability of a 411 remote SID. This is explained using an example in the following. 412 The example uses ping to an END SID (refer [I-D.ietf-spring-srv6- 413 network-programming]) but the procedure is equally applicable to ping 414 any other SID behaviors. 416 Consider the example where the user wants to ping a remote SID 417 2001:DB8:B:4::, via 2001:DB8:B:2:C31::, from node N1. The ICMPv6 418 echo request is processed at the individual nodes along the path as 419 follows: 421 o Node N1 initiates an ICMPv6 ping packet with SRH as follows 422 (2001:DB8:A:1::, 2001:DB8:B:2:C31::) (2001:DB8:B:4::, 423 2001:DB8:B:2:C31::; SL=1; NH=ICMPv6)(ICMPv6 Echo Request). 425 o Node N2, which is an SRv6 capable node, performs the standard SRH 426 processing. Specifically, it executes the END.X behavior 427 (2001:DB8:B:2:C31::) on the echo request packet. If 428 2001:DB8:B:2:C31:: is a PSP SID, node N4 executes the SID like any 429 other data packet with DA = 2001:DB8:B:2:C31:: and removes the 430 SRH. 432 o Node N3, which is a classic IPv6 node, performs the standard IPv6 433 processing. Specifically, it forwards the echo request based on 434 DA = 2001:DB8:B:4:: in the IPv6 header. 436 o When node N4 receives the packet, it processes the target SID 437 (2001:DB8:B:4::). 439 o If the target SID (2001:DB8:B:4::) is not locally instantiated, 440 the packet is discarded 442 o If the target SID (2001:DB8:B:4::) is locally instantiated, the 443 node processes the upper layer header. As part of the upper layer 444 header processing node N4 respond to the ICMPv6 echo request 445 message. 447 3.2. Traceroute 449 There is no hardware or software change required for traceroute 450 operation at the classic IPv6 nodes in an SRv6 network. That 451 includes the classic IPv6 node with ingress, egress or transit roles. 452 Furthermore, no protocol changes are required to the standard 453 traceroute operations. In other words, existing traceroute 454 mechanisms work seamlessly in the SRv6 networks. 456 The following subsections outline some use cases of the traceroute in 457 the SRv6 networks. 459 3.2.1. Classic Traceroute 461 The existing mechanism to traceroute a remote IP address, along the 462 shortest path, continues to work without any modification. The 463 initiator may be an SRv6 node or a classic IPv6 node. Similarly, the 464 egress or transit may be an SRv6 node or a classic IPv6 node. 466 If an SRv6 capable ingress node wants to traceroute to IPv6 address 467 via an arbitrary segment list , it needs to initiate 468 traceroute probe with an SR header containing the SID list . That is illustrated using the topology in Figure 1. Assume all 470 the links have IGP metric 10 except both links between node2 and 471 node3, which have IGP metric set to 100. User issues a traceroute 472 from node N1 to a loopback of node 5, via segment list 473 <2001:DB8:B:2:C31::, 2001:DB8:B:4:C52::>. The SID behavior used in 474 the example is End.X SID (refer [I-D.ietf-spring-srv6-network- 475 programming]) but the procedure is equally applicable to any other 476 (transit) SID type. Figure 3 contains sample output for the 477 traceroute request. 479 > traceroute 2001:DB8:A:5:: via segment-list 2001:DB8:B:2:C31::, 480 2001:DB8:B:4:C52:: 482 Tracing the route to 2001:DB8:A:5:: 483 1 2001:DB8:1:2:21:: 0.512 msec 0.425 msec 0.374 msec 484 DA: 2001:DB8:B:2:C31::, 485 SRH:(2001:DB8:A:5::, 2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::, SL=2) 486 2 2001:DB8:2:3:31:: 0.721 msec 0.810 msec 0.795 msec 487 DA: 2001:DB8:B:4:C52::, 488 SRH:(2001:DB8:A:5::, 2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::, SL=1) 489 3 2001:DB8:3:4::41:: 0.921 msec 0.816 msec 0.759 msec 490 DA: 2001:DB8:B:4:C52::, 491 SRH:(2001:DB8:A:5::, 2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::, SL=1) 492 4 2001:DB8:4:5::52:: 0.879 msec 0.916 msec 1.024 msec 493 DA: 2001:DB8:A:5:: 495 Figure 3 A sample traceroute output at an SRv6 capable node 497 Please note that information for hop2 is returned by N3, which is a 498 classic IPv6 node. Nonetheless, the ingress node is able to display 499 SR header contents as the packet travels through the IPv6 classic 500 node. This is because the "Time Exceeded Message" ICMPv6 message can 501 contain as much of the invoking packet as possible without the ICMPv6 502 packet exceeding the minimum IPv6 MTU [RFC4443]. The SR header is 503 also included in these ICMPv6 messages initiated by the classic IPv6 504 transit nodes that are not running SRv6 software. Specifically, a 505 node generating ICMPv6 message containing a copy of the invoking 506 packet does not need to understand the extension header(s) in the 507 invoking packet. 509 The segment list information returned for hop1 is returned by N2, 510 which is an SRv6 capable node. Just like for hop2, the ingress node 511 is able to display SR header contents for hop1. 513 There is no difference in processing of the traceroute probe at an 514 IPv6 classic node and an SRv6 capable node. Similarly, both IPv6 515 classic and SRv6 capable nodes may use the address of the interface 516 on which probe was received as the source address in the ICMPv6 517 response. ICMP extensions defined in [RFC5837] can be used to also 518 display information about the IP interface through which the datagram 519 would have been forwarded had it been forwardable, and the IP next 520 hop to which the datagram would have been forwarded, the IP interface 521 upon which a datagram arrived, the sub-IP component of an IP 522 interface upon which a datagram arrived. 524 The information about the IP address of the incoming interface on 525 which the traceroute probe was received by the reporting node is very 526 useful. This information can also be used to verify if SIDs 527 2001:DB8:B:2:C31:: and 2001:DB8:B:4:C52:: are executed correctly by 528 N2 and N4, respectively. Specifically, the information displayed for 529 hop2 contains the incoming interface address 2001:DB8:2:3:31:: at N3. 530 This matches with the expected interface bound to END.X behavior 531 2001:DB8:B:2:C31:: (link3). Similarly, the information displayed for 532 hop5 contains the incoming interface address 2001:DB8:4:5::52:: at 533 N5. This matches with the expected interface bound to the END.X 534 behavior 2001:DB8:B:4:C52:: (link10). 536 3.2.2. Traceroute to a SID 538 The classic traceroute described in the previous section applies 539 equally to traceroute a remote SID behavior, as explained using an 540 example in the following. The example uses traceroute to an END SID 541 (refer [I-D.ietf-spring-srv6-network-programming]) but the procedure 542 is equally applicable to tracerouting any other SID behaviors. 544 Please note that traceroute to a SID is exemplified using UDP probes. 545 However, the procedure is equally applicable to other implementations 546 of traceroute mechanism. 548 Consider the example where the user wants to traceroute a remote SID 549 2001:DB8:B:4::, via 2001:DB8:B:2:C31::, from node N1. The traceroute 550 probe is processed at the individual nodes along the path as follows: 552 o Node N1 initiates a traceroute probe packet with a monotonically 553 increasing value of hop count and SRH as follows (2001:DB8:A:1::, 554 2001:DB8:B:2:C31::) (2001:DB8:B:4::, 2001:DB8:B:2:C31::; SL=1; 555 NH=UDP)(Traceroute probe). 557 o When node N2 receives the packet with hop-count = 1, it processes 558 the hop count expiry. Specifically, the node N2 responses with 559 the ICMPv6 message (Type: "Time Exceeded", Code: "Time to Live 560 exceeded in Transit"). 562 o When Node N2 receives the packet with hop-count > 1, it performs 563 the standard SRH processing. Specifically, it executes the END.X 564 behavior (2001:DB8:B:2:C31::) on the traceroute probe. If 565 2001:DB8:B:2:C31:: is a PSP SID, node N4 executes the SID like any 566 other data packet with DA = 2001:DB8:B:2:C31:: and removes the 567 SRH. 569 o When node N3, which is a classic IPv6 node, receives the packet 570 with hop-count = 1, it processes the hop count expiry. 571 Specifically, the node N3 responses with the ICMPv6 message (Type: 572 "Time Exceeded", Code: "Time to Live exceeded in Transit"). 574 o When node N3, which is a classic IPv6 node, receives the packet 575 with hop-count > 1, it performs the standard IPv6 processing. 576 Specifically, it forwards the traceroute probe based on DA 577 2001:DB8:B:4:: in the IPv6 header. 579 o When node N4 receives the packet with DA set to the local SID 580 2001:DB8:B:4::, it processes the END SID. 582 o If the target SID (2001:DB8:B:4::) is not locally instantiated, 583 the packet is discarded. 585 o If the target SID (2001:DB8:B:4::) is locally instantiated, the 586 node processes the upper layer header. As part of the upper layer 587 header processing node N4 responses with the ICMPv6 message (Type: 588 Destination unreachable, Code: Port Unreachable). 590 Figure 4 displays a sample traceroute output for this example. 592 > traceroute 2001:DB8:B:4:C52:: via segment-list 2001:DB8:B:2:C31:: 594 Tracing the route to SID 2001:DB8:B:4:C52:: 595 1 2001:DB8:1:2:21:: 0.512 msec 0.425 msec 0.374 msec 596 DA: 2001:DB8:B:2:C31::, 597 SRH:(2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::; SL=1) 598 2 2001:DB8:2:3:31:: 0.721 msec 0.810 msec 0.795 msec 599 DA: 2001:DB8:B:4:C52::, 600 SRH:(2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::; SL=0) 601 3 2001:DB8:3:4:41:: 0.921 msec 0.816 msec 0.759 msec 602 DA: 2001:DB8:B:4:C52::, 603 SRH:(2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::; SL=0) 605 Figure 4 A sample output for hop-by-hop traceroute to a SID 607 3.3. A Hybrid OAM Using O-flag 609 This section illustrates a hybrid OAM mechanism using the the 610 SRH.Flags.O-flag. Without loss of the generality, the illustration 611 assumes N100 is a centralized controller. 613 The illustration is different than the In-situ OAM defined in [I.D- 614 draft-ietf-ippm-ioam-data]. This is because In-situ OAM records 615 operational and telemetry information in the packet as the packet 616 traverses a path between two points in the network [I.D-draft-ietf- 617 ippm-ioam-data]. The illustration in section 3 does not require the 618 recording of OAM data in the packet. 620 The illustration does not assume any formats for exporting the data 621 elements or the data elements that needs to be exported. 623 Consider the example where the user wants to monitor sampled IPv4 VPN 624 100 traffic going from CE1 to CE2 via a low latency SR policy P 625 installed at Node N1. To exercise a low latency path, the SR Policy 626 P forces the packet via segments 2001:DB8:B:2:C31:: and 627 2001:DB8:B:4:C52::. The VPN SID at N7 associated with VPN100 is 628 2001:DB8:B:7:DT100::. 2001:DB8:B:7:DT100:: is a USP SID. N1, N4, 629 and N7 are capable of processing SRH.Flags.O-flag but N2 is not 630 capable of processing SRH.Flags.O-flag. N100 is the centralized 631 controller capable of processing and correlating the copy of the 632 packets sent from nodes N1, N4, and N7. N100 is aware of 633 SRH.Flags.O-flag processing capabilities. Controller N100 with the 634 help from nodes N1, N4, N7 and implements a hybrid OAM mechanism 635 using the SRH.Flags.O-flag as follows: 637 o A packet P1:(IPv4 header)(payload) is sent from CE1 to Node N1. 639 o Node N1 steers the packet P1 through the Policy P. Based on a 640 local configuration, Node N1 also implements logic to sample 641 traffic steered through policy P for hybrid OAM purposes. 642 Specification for the sampling logic is beyond the scope of this 643 document. Consider the case where packet P1 is classified as a 644 packet to be monitored via the hybrid OAM. Node N1 sets 645 SRH.Flags.O-flag during encapsulation required by policy P. As 646 part of setting the SRH.Flags.O-flag, node N1 also send a 647 timestamped copy of the packet P1: (2001:DB8:A:1::, 648 2001:DB8:B:2:C31::) (2001:DB8:B:7:DT100::, 2001:DB8:B:4:C52::, 649 2001:DB8:B:2:C31::; SL=2; O-flag=1; NH=IPv4)(IPv4 header)(payload) 650 to a local OAM process. The local OAM process sends a full or 651 partial copy of the packet P1 to the controller N100. The OAM 652 process includes the recorded timestamp, additional OAM 653 information like incoming and outgoing interface, etc. along with 654 any applicable metadata. Node N1 forwards the original packet 655 towards the next segment 2001:DB8:B:2:C31::. 657 o When node N2 receives the packet with SRH.Flags.O-flag set, it 658 ignores the SRH.Flags.O-flag. This is because node N2 is not 659 capable of processing the O-flag. Node N2 performs the standard 660 SRv6 SID and SRH processing. Specifically, it executes the END.X 661 (refer [I-D.ietf-spring-srv6-network-programming]) behavior 662 (2001:DB8:B:2:C31::) and forwards the packet P1 (2001:DB8:A:1::, 663 2001:DB8:B:4:C52::) (2001:DB8:B:7:DT100::, 2001:DB8:B:4:C52::, 664 2001:DB8:B:2:C31::; SL=1; O-flag=1; NH=IPv4)(IPv4 header)(payload) 665 over link 3 towards Node N3. 667 o When node N3, which is a classic IPv6 node, receives the packet P1 668 , it performs the standard IPv6 processing. Specifically, it 669 forwards the packet P1 based on DA 2001:DB8:B:4:C52:: in the IPv6 670 header. 672 o When node N4 receives the packet P1 (2001:DB8:A:1::, 673 2001:DB8:B:4:C52::) (2001:DB8:B:7:DT100::, 2001:DB8:B:4:C52::, 674 2001:DB8:B:2:C31::; SL=1; O-flag=1; NH=IPv4)(IPv4 675 header)(payload), it processes the SRH.Flags.O-flag. As part of 676 processing the O-flag, it sends a timestamped copy of the packet 677 to a local OAM process. The local OAM process sends a full or 678 partial copy of the packet P1 to the controller N100. The OAM 679 process includes the recorded timestamp, additional OAM 680 information like incoming and outgoing interface, etc. along with 681 any applicable metadata. Node N4 performs the standard SRv6 SID 682 and SRH processing on the original packet P1. Specifically, it 683 executes the END.X behavior (2001:DB8:B:4:C52::) and forwards the 684 packet P1 (2001:DB8:A:1::, 2001:DB8:B:7:DT100::) 685 (2001:DB8:B:7:DT100::, 2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::; 686 SL=0; O-flag=1; NH=IPv4)(IPv4 header)(payload) over link 10 687 towards Node N5. 689 o When node N5, which is a classic IPv6 node, receives the packet 690 P1, it performs the standard IPv6 processing. Specifically, it 691 forwards the packet based on DA 2001:DB8:B:7:DT100:: in the IPv6 692 header. 694 o When node N7 receives the packet P1 (2001:DB8:A:1::, 695 2001:DB8:B:7:DT100::) (2001:DB8:B:7:DT100::, 2001:DB8:B:4:C52::, 696 2001:DB8:B:2:C31::; SL=0; O-flag=1; NH=IPv4)(IPv4 697 header)(payload), it processes the SRH.Flags.O-flag. As part of 698 processing the O-flag, it sends a timestamped copy of the packet 699 to a local OAM process. The local OAM process sends a full or 700 partial copy of the packet P1 to the controller N100. The OAM 701 process includes the recorded timestamp, additional OAM 702 information like incoming and outgoing interface, etc. along with 703 any applicable metadata. Node N4 performs the standard SRv6 SID 704 and SRH processing on the original packet P1. Specifically, it 705 executes the VPN SID (2001:DB8:B:7:DT100::) and based on lookup in 706 table 100 forwards the packet P1 (IPv4 header)(payload) towards CE 707 2. 709 o The controller N100 processes and correlates the copy of the 710 packets sent from nodes N1, N4 and N7 to find segment-by-segment 711 delays and provide other hybrid OAM information related to packet 712 P1. 714 o The process continues for any other sampled packets. 716 3.4. Monitoring of SRv6 Paths 718 In the recent past, network operators demonstrated interest in 719 performing network OAM functions in a centralized manner. [RFC8403] 720 describes such a centralized OAM mechanism. Specifically, the 721 document describes a procedure that can be used to perform path 722 continuity check between any nodes within an SR domain from a 723 centralized monitoring system. However, the document focuses on SR 724 networks with MPLS data plane. This document describes how the 725 concept can be used to perform path monitoring in an SRv6 network 726 from a centralized controller. 728 In the reference topology in Figure 1, N100 uses an IGP protocol like 729 OSPF or ISIS to get the topology view within the IGP domain. N100 730 can also use BGP-LS to get the complete view of an inter-domain 731 topology. The controller leverages the visibility of the topology to 732 monitor the paths between the various endpoints. 734 The controller N100 advertises an END (refer [I-D.ietf-spring-srv6- 735 network-programming]) SID 2001:DB8:B:100:1::. To monitor any 736 arbitrary SRv6 paths, the controller can create a loopback probe that 737 originates and terminates on Node N100. To distinguish between a 738 failure in the monitored path and loss of connectivity between the 739 controller and the network, Node N100 runs a suitable mechanism to 740 monitor its connectivity to the monitored network. 742 The loopback probes are exemplified using an example where controller 743 N100 needs to verify a segment list <2001:DB8:B:2:C31::, 744 2001:DB8:B:4:C52::>: 746 o N100 generates an OAM packet (2001:DB8:A:100::, 747 2001:DB8:B:2:C31::)(2001:DB8:B:100:1::, 2001:DB8:B:4:C52::, 748 2001:DB8:B:2:C31::, SL=2)(OAM Payload). The controller routes the 749 probe packet towards the first segment, which is 750 2001:DB8:B:2:C31::. 752 o Node N2 executes the END.X behavior (2001:DB8:B:2:C31::) and 753 forwards the packet (2001:DB8:A:100::, 754 2001:DB8:B:4:C52::)(2001:DB8:B:100:1::, 2001:DB8:B:4:C52::, 755 2001:DB8:B:2:C31::, SL=1)(OAM Payload) on link3 to N3. 757 o Node N3, which is a classic IPv6 node, performs the standard IPv6 758 processing. Specifically, it forwards the packet based on the DA 759 2001:DB8:B:4:C52:: in the IPv6 header. 761 o Node N4 executes the END.X behavior (2001:DB8:B:4:C52::) and 762 forwards the packet (2001:DB8:A:100::, 763 2001:DB8:B:100:1::)(2001:DB8:B:100:1::, 2001:DB8:B:4:C52::, 764 2001:DB8:B:2:C31::, SL=0)(OAM Payload) on link10 to N5. 766 o Node N5, which is a classic IPv6 node, performs the standard IPv6 767 processing. Specifically, it forwards the packet based on the DA 768 2001:DB8:B:100:1:: in the IPv6 header. 770 o Node N100 executes the standard SRv6 END behavior. It 771 decapsulates the header and consume the probe for OAM processing. 772 The information in the OAM payload is used to detect any missing 773 probes, round trip delay, etc. 775 The OAM payload type or the information carried in the OAM probe is a 776 local implementation decision at the controller and is outside the 777 scope of this document. 779 4. Implementation Status 781 This section is to be removed prior to publishing as an RFC. 783 See [I-D.matsushima-spring-srv6-deployment-status] for updated 784 deployment and interoperability reports. 786 5. Security Considerations 788 This document does not define any new protocol extensions and relies 789 on existing procedures defined for ICMP. This document does not 790 impose any additional security challenges to be considered beyond 791 security considerations described in [RFC4884], [RFC4443], [RFC0792], 792 and [RFC8754]. 794 6. IANA Considerations 796 6.1. Segment Routing Header Flags 798 This I-D requests to IANA to allocate bit position 2, within the 799 "Segment Routing Header Flags" registry defined in [RFC8754]. 801 7. Acknowledgements 803 The authors would like to thank Joel M. Halpern, Greg Mirsky, Bob 804 Hinden, Loa Andersson, Gaurav Naik, Ketan Talaulikar and Haoyu Song 805 for their review comments. 807 8. Contributors 809 The following people have contributed to this document: 811 Robert Raszuk 812 Bloomberg LP 813 Email: robert@raszuk.net 815 John Leddy 816 Individual 817 Email: john@leddy.net 819 Gaurav Dawra 820 LinkedIn 821 Email: gdawra.ietf@gmail.com 823 Bart Peirens 824 Proximus 825 Email: bart.peirens@proximus.com 827 Nagendra Kumar 828 Cisco Systems, Inc. 829 Email: naikumar@cisco.com 831 Carlos Pignataro 832 Cisco Systems, Inc. 833 Email: cpignata@cisco.com 835 Rakesh Gandhi 836 Cisco Systems, Inc. 837 Canada 838 Email: rgandhi@cisco.com 839 Frank Brockners 840 Cisco Systems, Inc. 841 Germany 842 Email: fbrockne@cisco.com 844 Darren Dukes 845 Cisco Systems, Inc. 846 Email: ddukes@cisco.com 848 Cheng Li 849 Huawei 850 Email: chengli13@huawei.com 852 Faisal Iqbal 853 Individual 854 Email: faisal.ietf@gmail.com 856 9. References 858 9.1. Normative References 860 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 861 Requirement Levels", BCP 14, RFC 2119, 862 DOI 10.17487/RFC2119, March 1997, 863 . 865 [RFC8754] Filsfils, C., Ed., Dukes, D., Ed., Previdi, S., Leddy, J., 866 Matsushima, S., and D. Voyer, "IPv6 Segment Routing Header 867 (SRH)", RFC 8754, DOI 10.17487/RFC8754, March 2020, 868 . 870 9.2. Informative References 872 [I-D.gandhi-spring-stamp-srpm] 873 Gandhi, R., Filsfils, C., Voyer, D., Chen, M., and B. 874 Janssens, "Performance Measurement Using STAMP for Segment 875 Routing Networks", draft-gandhi-spring-stamp-srpm-01 (work 876 in progress), June 2020. 878 [I-D.gandhi-spring-twamp-srpm] 879 Gandhi, R., Filsfils, C., Voyer, D., Chen, M., and B. 880 Janssens, "Performance Measurement Using TWAMP Light for 881 Segment Routing Networks", draft-gandhi-spring-twamp- 882 srpm-09 (work in progress), June 2020. 884 [I-D.ietf-spring-srv6-network-programming] 885 Filsfils, C., Camarillo, P., Leddy, J., Voyer, D., 886 Matsushima, S., and Z. Li, "SRv6 Network Programming", 887 draft-ietf-spring-srv6-network-programming-16 (work in 888 progress), June 2020. 890 [I-D.matsushima-spring-srv6-deployment-status] 891 Matsushima, S., Filsfils, C., Ali, Z., Li, Z., and K. 892 Rajaraman, "SRv6 Implementation and Deployment Status", 893 draft-matsushima-spring-srv6-deployment-status-07 (work in 894 progress), April 2020. 896 [RFC0792] Postel, J., "Internet Control Message Protocol", STD 5, 897 RFC 792, DOI 10.17487/RFC0792, September 1981, 898 . 900 [RFC4443] Conta, A., Deering, S., and M. Gupta, Ed., "Internet 901 Control Message Protocol (ICMPv6) for the Internet 902 Protocol Version 6 (IPv6) Specification", STD 89, 903 RFC 4443, DOI 10.17487/RFC4443, March 2006, 904 . 906 [RFC4884] Bonica, R., Gan, D., Tappan, D., and C. Pignataro, 907 "Extended ICMP to Support Multi-Part Messages", RFC 4884, 908 DOI 10.17487/RFC4884, April 2007, 909 . 911 [RFC5476] Claise, B., Ed., Johnson, A., and J. Quittek, "Packet 912 Sampling (PSAMP) Protocol Specifications", RFC 5476, 913 DOI 10.17487/RFC5476, March 2009, 914 . 916 [RFC5837] Atlas, A., Ed., Bonica, R., Ed., Pignataro, C., Ed., Shen, 917 N., and JR. Rivers, "Extending ICMP for Interface and 918 Next-Hop Identification", RFC 5837, DOI 10.17487/RFC5837, 919 April 2010, . 921 [RFC5880] Katz, D. and D. Ward, "Bidirectional Forwarding Detection 922 (BFD)", RFC 5880, DOI 10.17487/RFC5880, June 2010, 923 . 925 [RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, 926 "Specification of the IP Flow Information Export (IPFIX) 927 Protocol for the Exchange of Flow Information", STD 77, 928 RFC 7011, DOI 10.17487/RFC7011, September 2013, 929 . 931 [RFC7012] Claise, B., Ed. and B. Trammell, Ed., "Information Model 932 for IP Flow Information Export (IPFIX)", RFC 7012, 933 DOI 10.17487/RFC7012, September 2013, 934 . 936 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 937 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 938 May 2016, . 940 [RFC7880] Pignataro, C., Ward, D., Akiya, N., Bhatia, M., and S. 941 Pallagatti, "Seamless Bidirectional Forwarding Detection 942 (S-BFD)", RFC 7880, DOI 10.17487/RFC7880, July 2016, 943 . 945 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 946 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 947 May 2017, . 949 [RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L., 950 Decraene, B., Litkowski, S., and R. Shakir, "Segment 951 Routing Architecture", RFC 8402, DOI 10.17487/RFC8402, 952 July 2018, . 954 [RFC8403] Geib, R., Ed., Filsfils, C., Pignataro, C., Ed., and N. 955 Kumar, "A Scalable and Topology-Aware MPLS Data-Plane 956 Monitoring System", RFC 8403, DOI 10.17487/RFC8403, July 957 2018, . 959 Authors' Addresses 961 Zafar Ali 962 Cisco Systems 964 Email: zali@cisco.com 966 Clarence Filsfils 967 Cisco Systems 969 Email: cfilsfil@cisco.com 971 Satoru Matsushima 972 Softbank 974 Email: satoru.matsushima@g.softbank.co.jp 975 Daniel Voyer 976 Bell Canada 978 Email: daniel.voyer@bell.ca 980 Mach Chen 981 Huawei 983 Email: mach.chen@huawei.com