idnits 2.17.1 draft-ietf-6man-spring-srv6-oam-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 30, 2020) is 1274 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'SL' is mentioned on line 209, but not defined == Unused Reference: 'I-D.gandhi-spring-twamp-srpm' is defined on line 892, but no explicit reference was found in the text == Outdated reference: A later version (-07) exists of draft-gandhi-spring-stamp-srpm-03 == Outdated reference: A later version (-28) exists of draft-ietf-spring-srv6-network-programming-24 == Outdated reference: A later version (-15) exists of draft-matsushima-spring-srv6-deployment-status-08 Summary: 0 errors (**), 0 flaws (~~), 6 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 6man Z. Ali 2 Internet-Draft C. Filsfils 3 Intended status: Standards Track Cisco Systems 4 Expires: May 3, 2021 S. Matsushima 5 Softbank 6 D. Voyer 7 Bell Canada 8 M. Chen 9 Huawei 10 October 30, 2020 12 Operations, Administration, and Maintenance (OAM) in Segment Routing 13 Networks with IPv6 Data plane (SRv6) 14 draft-ietf-6man-spring-srv6-oam-08 16 Abstract 18 This document describes how the existing IPv6 mechanisms for ping and 19 traceroute can be used in an SRv6 network. The document also 20 specifies the OAM flag in the Segment Routing Header (SRH) for 21 performing controllable and predictable flow sampling from segment 22 endpoints. In addition, the document describes how a centralized 23 monitoring system performs a path continuity check between any nodes 24 within an SRv6 domain. 26 Status of This Memo 28 This Internet-Draft is submitted in full conformance with the 29 provisions of BCP 78 and BCP 79. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF). Note that other groups may also distribute 33 working documents as Internet-Drafts. The list of current Internet- 34 Drafts is at https://datatracker.ietf.org/drafts/current/. 36 Internet-Drafts are draft documents valid for a maximum of six months 37 and may be updated, replaced, or obsoleted by other documents at any 38 time. It is inappropriate to use Internet-Drafts as reference 39 material or to cite them other than as "work in progress." 41 This Internet-Draft will expire on May 3, 2021. 43 Copyright Notice 45 Copyright (c) 2020 IETF Trust and the persons identified as the 46 document authors. All rights reserved. 48 This document is subject to BCP 78 and the IETF Trust's Legal 49 Provisions Relating to IETF Documents 50 (https://trustee.ietf.org/license-info) in effect on the date of 51 publication of this document. Please review these documents 52 carefully, as they describe your rights and restrictions with respect 53 to this document. Code Components extracted from this document must 54 include Simplified BSD License text as described in Section 4.e of 55 the Trust Legal Provisions and are provided without warranty as 56 described in the Simplified BSD License. 58 Table of Contents 60 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 61 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 62 1.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 3 63 1.3. Terminology and Reference Topology . . . . . . . . . . . 3 64 2. OAM Mechanisms . . . . . . . . . . . . . . . . . . . . . . . 5 65 2.1. O-flag in Segment Routing Header . . . . . . . . . . . . 5 66 2.1.1. O-flag Processing . . . . . . . . . . . . . . . . . . 6 67 2.2. OAM Operations . . . . . . . . . . . . . . . . . . . . . 7 68 3. Illustrations . . . . . . . . . . . . . . . . . . . . . . . . 8 69 3.1. Ping in SRv6 Networks . . . . . . . . . . . . . . . . . . 8 70 3.1.1. Classic Ping . . . . . . . . . . . . . . . . . . . . 8 71 3.1.2. Pinging a SID . . . . . . . . . . . . . . . . . . . . 10 72 3.2. Traceroute . . . . . . . . . . . . . . . . . . . . . . . 10 73 3.2.1. Classic Traceroute . . . . . . . . . . . . . . . . . 11 74 3.2.2. Traceroute to a SID . . . . . . . . . . . . . . . . . 12 75 3.3. A Hybrid OAM Using O-flag . . . . . . . . . . . . . . . . 14 76 3.4. Monitoring of SRv6 Paths . . . . . . . . . . . . . . . . 16 77 4. Implementation Status . . . . . . . . . . . . . . . . . . . . 18 78 5. Security Considerations . . . . . . . . . . . . . . . . . . . 18 79 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18 80 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 18 81 8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 18 82 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 20 83 9.1. Normative References . . . . . . . . . . . . . . . . . . 20 84 9.2. Informative References . . . . . . . . . . . . . . . . . 20 85 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22 87 1. Introduction 89 As Segment Routing with IPv6 data plane (SRv6) [RFC8402] simply adds 90 a new type of Routing Extension Header, existing IPv6 OAM mechanisms 91 can be used in an SRv6 network. This document describes how the 92 existing IPv6 mechanisms for ping and trace route can be used in an 93 SRv6 network. This includes illustrations to pinging an SRv6 SID for 94 the SID connectivity checks and to validate the availability of a 95 SID. This also includes illustrations for tracerouting to an SRv6 96 SID for hop-by-hop fault localization as well as path tracing to a 97 SID. 99 The document also introduces enhancements for OAM mechanism for SRv6 100 networks for performing controllable and predictable flow sampling 101 from segment endpoints using, e.g., IP Flow Information Export 102 (IPFIX) protocol [RFC7011]. Specifically, the document specifies 103 O-flag in SRH as a marking-bit in the user packets to trigger the 104 telemetry data collection and export at the segment endpoints. 106 The document also outlines how centralized OAM technique in [RFC8403] 107 can be extended for SRv6 to perform a path continuity check between 108 any nodes within an SRv6 domain. Specifically, the document 109 illustrates how a centralized monitoring system can monitor arbitrary 110 SRv6 paths by creating the loopback probes that originates and 111 terminates at the centralized monitoring system. 113 1.1. Requirements Language 115 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 116 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 117 document are to be interpreted as described in [RFC2119], [RFC8174]. 119 1.2. Abbreviations 121 The following abbreviations are used in this document: 123 SID: Segment ID. 125 SL: Segments Left. 127 SR: Segment Routing. 129 SRH: Segment Routing Header [RFC8754]. 131 SRv6: Segment Routing with IPv6 Data plane. 133 TC: Traffic Class. 135 ICMPv6: ICMPv6 Specification [RFC4443]. 137 1.3. Terminology and Reference Topology 139 Throughout the document, the following terminology and simple 140 topology is used for illustration. 142 +--------------------------| N100 |---------------------------------+ 143 | | 144 | ====== link1====== link3------ link5====== link9------ ====== | 145 ||N1||------||N2||------| N3 |------||N4||------| N5 |---||N7|| 146 || ||------|| ||------| |------|| ||------| |---|| || 147 ====== link2====== link4------ link6======link10------ ====== 148 | | | | 149 ---+-- | ------ | --+--- 150 |CE 1| +-------| N6 |---------+ |CE 2| 151 ------ link7 | | link8 ------ 152 ------ 154 Figure 1 Reference Topology 156 In the reference topology: 158 Node k has a classic IPv6 loopback address 2001:DB8:A:k::/128. 160 Nodes N1, N2, N4 and N7 are SRv6 capable nodes. 162 Nodes N3, N5 and N6 are IPv6 nodes that are not SRv6 capable. 163 Such nodes are referred as classic IPv6 nodes. 165 CE1 and CE2 are Customer Edge devices of any data plane capability 166 (e.g., IPv4, IPv6, L2, etc.). 168 A SID at node k with locator block 2001:DB8:B::/48 and function F 169 is represented by 2001:DB8:B:k:F::. 171 Node N100 is a controller. 173 The IPv6 address of the nth Link between node X and Y at the X 174 side is represented as 2001:DB8:X:Y:Xn::, e.g., the IPv6 address 175 of link6 (the 2nd link) between N3 and N4 at N3 in Figure 1 is 176 2001:DB8:3:4:32::. Similarly, the IPv6 address of link5 (the 1st 177 link between N3 and N4) at node 3 is 2001:DB8:3:4:31::. 179 2001:DB8:B:k:Cij:: is explicitly allocated as the END.X SID (refer 180 [I-D.ietf-spring-srv6-network-programming]) at node k towards 181 neighbor node i via jth Link between node i and node k. e.g., 182 2001:DB8:B:2:C31:: represents END.X at N2 towards N3 via link3 183 (the 1st link between N2 and N3). Similarly, 2001:DB8:B:4:C52:: 184 represents the END.X at N4 towards N5 via link10. 186 A SID list is represented as where S1 is the first 187 SID to visit, S2 is the second SID to visit and S3 is the last SID 188 to visit along the SR path. 190 (SA,DA) (S3, S2, S1; SL)(payload) represents an IPv6 packet with: 192 * IPv6 header with source address SA, destination addresses DA 193 and SRH as next-header 195 * SRH with SID list with SegmentsLeft = SL 197 * Note the difference between the < > and () symbols: represents a SID list where S1 is the first SID and S3 is 199 the last SID to traverse. (S3, S2, S1; SL) represents the same 200 SID list but encoded in the SRH format where the rightmost SID 201 in the SRH is the first SID and the leftmost SID in the SRH is 202 the last SID. When referring to an SR policy in a high-level 203 use-case, it is simpler to use the notation. When 204 referring to an illustration of the detailed packet behavior, 205 the (S3, S2, S1; SL) notation is more convenient. 207 * (payload) represents the the payload of the packet. 209 SRH[SL] represents the SID pointed by the SL field in the first 210 SRH. In our example SID list (S3, S2, S1; SL), SRH(2) represents 211 S1, SRH(1) represents S2 and SRH(0) represents S3. 213 2. OAM Mechanisms 215 This section defines OAM enhancement for the SRv6 networks. 217 2.1. O-flag in Segment Routing Header 219 [RFC8754] describes the Segment Routing Header (SRH) and how SR 220 capable nodes use it. The SRH contains an 8-bit "Flags" field. This 221 document defines the following bit in the SRH.Flags to carry the 222 O-flag: 224 0 1 2 3 4 5 6 7 225 +-+-+-+-+-+-+-+-+ 226 | |O| | 227 +-+-+-+-+-+-+-+-+ 229 Where: 231 O-flag: OAM flag. 233 The document does not define any other flag in the SRH.Flags and 234 meaning and processing of any other bit in SRH.Flags is outside of 235 the scope of this document. 237 2.1.1. O-flag Processing 239 The O-flag in SRH is used as a marking-bit in the user packets to 240 trigger the telemetry data collection and export at the segment 241 endpoints. 243 This document does not specify the data elements that needs to be 244 exported and the associated configurations. Similarly, this document 245 does not define any formats for exporting the data elements. 246 Nonetheless, without the loss of generality, this document assumes IP 247 Flow Information Export (IPFIX) protocol [RFC7011] is used for 248 exporting the traffic flow information from the network devices to a 249 controller for monitoring and analytics. Similarly, without the loss 250 of generality, this document assumes requested information elements 251 are configured by the management plane through data set templates 252 (e.g., as in IPFIX [RFC7012]). 254 Implementation of the O-flag is OPTIONAL. If a node does not support 255 the O-flag, then upon reception it simply ignores it. If a node 256 supports the O-flag, it can optionally advertise its potential via 257 control plan protocol(s). 259 When N receives a packet whose IPv6 DA is S and S is a local SID, the 260 line S01 of the pseudo-code associated with the SID S, as defined in 261 section 4.3.1.1 of [RFC8754], is modified as follows for the O-flag 262 processing. 264 S01.1. IF SRH.Flags.O-flag is set and local configuration permits 265 O-flag processing THEN 266 a. Make a copy of the packet. 267 b. Send the copied packet, along with a timestamp 268 to the OAM process for telemetry data collection 269 and export. ;; Ref1 270 Ref1: An implementation SHOULD copy and record the timestamp as 271 soon as possible during packet processing. Timestamp or any other 272 metadata is not 273 carried in the packet forwarded to the next hop. 275 Please note that the O-flag processing happens before execution of 276 regular processing of the local SID S. 278 Based on the requested information elements configured by the 279 management plane through data set templates [RFC7012], the OAM 280 process exports the requested information elements. The information 281 elements include parts of the packet header and/or parts of the 282 packet payload for flow identification. The OAM process uses 283 information elements defined in IPFIX [RFC7011] and PSAMP [RFC5476] 284 for exporting the requested sections of the mirrored packets. 286 If the telemetry data from the last node in the segment-list (egress 287 node) is desired, the ingress uses an Ultimate Segment Pop (USP) SID 288 advertised by the egress node. 290 The processing node SHOULD rate-limit the number of packets punted to 291 the OAM process to avoid hitting any performance impact. 293 The OAM process MUST NOT process the copy of the packet or respond to 294 any upper-layer header (like ICMP, UDP, etc.) payload to prevent 295 multiple evaluations of the datagram. 297 Specification of the OAM process or the external controller 298 operations are beyond the scope of this document. How to correlate 299 the data collected from different nodes at an external controller is 300 also outside the scope of the document. Section 3 illustrates use of 301 the SRH.Flags.O-flag for implementing a hybrid OAM mechanism, where 302 the "hybrid" classification is based on RFC7799 [RFC7799]. 304 2.2. OAM Operations 306 IPv6 OAM operations can be performed for any SRv6 SID whose behavior 307 allows Upper Layer Header processing for an applicable OAM payload 308 (e.g., ICMP, UDP). 310 Ping to a SID is used for SID connectivity checks and to validate the 311 availability of a SID. Traceroute to a SID is used for hop-by-hop 312 fault localization as well as path tracing to a SID. Section 3 313 illustrates the ICMPv6 based ping and the UDP based traceroute 314 mechanisms for ping and traceroute to an SRv6 SID. Although this 315 document only illustrates ICMP ping and UDP-based traceroute to an 316 SRv6 SID, the procedures are equally applicable to other IPv6 OAM 317 probing to an SRv6 SID (e.g., Bidirectional Forwarding Detection 318 (BFD) [RFC5880], Seamless BFD (SBFD) [RFC7880], TWAMP Light and STAMP 319 probe message processing as described in [I-D.gandhi-spring-twamp- 320 srpm] and [I-D.gandhi-spring-stamp-srpm], respectively, etc.). 321 Specifically, as long as local configuration allows the Upper-layer 322 Header processing of the applicable OAM payload for SRv6 SIDs, the 323 existing IPv6 OAM techniques can be used to target a probe to a 324 (remote) SID. 326 IPv6 OAM operations can be performed with the target SID in the IPv6 327 destination address without SRH or with SRH where the target SID is 328 the last segment. In general, OAM operations to a target SID may not 329 exercise all of its processing depending on its behavior definition. 330 For example, ping to an END.X SID (refer [I-D.ietf-spring-srv6- 331 network-programming]) at the target node only validates availability 332 of the SID and does not validate switching to the correct outgoing 333 interface. To exercise the behavior of a target SID, the OAM 334 operation SHOULD construct the probe in a manner similar to a data 335 packet that exercises the SID behavior, i.e. to include that SID as a 336 transit SID in either an SRH or IPv6 DA of an outer IPv6 header or as 337 appropriate based on the definition of the SID behavior. 339 3. Illustrations 341 This section shows how some of the existing IPv6 OAM mechanisms can 342 be used in an SRv6 network. It also illustrates an OAM mechanism for 343 performing controllable and predictable flow sampling from segment 344 endpoints. How centralized OAM technique in [RFC8403] can be 345 extended for SRv6 is also described in this Section. 347 3.1. Ping in SRv6 Networks 349 The following subsections outline some use cases of the ICMP ping in 350 the SRv6 networks. 352 3.1.1. Classic Ping 354 The existing mechanism to perform the connectivity checks, along the 355 shortest path, continues to work without any modification. The 356 initiator may be an SRv6 node or a classic IPv6 node. Similarly, the 357 egress or transit may be an SRv6 capable node or a classic IPv6 node. 359 If an SRv6 capable ingress node wants to ping an IPv6 address via an 360 arbitrary segment list , it needs to initiate ICMPv6 ping 361 with an SR header containing the SID list . This is 362 illustrated using the topology in Figure 1. Assume all the links 363 have IGP metric 10 except both links between node2 and node3, which 364 have IGP metric set to 100. User issues a ping from node N1 to a 365 loopback of node 5, via segment list <2001:DB8:B:2:C31::, 366 2001:DB8:B:4:C52::>. The SID behavior used in the example is End.X 367 SID (refer [I-D.ietf-spring-srv6-network-programming]) but the 368 procedure is equally applicable to any other (transit) SID type. 370 Figure 2 contains sample output for a ping request initiated at node 371 N1 to the loopback address of node N5 via a segment list 372 <2001:DB8:B:2:C31::, 2001:DB8:B:4:C52::>. 374 > ping 2001:DB8:A:5:: via segment-list 2001:DB8:B:2:C31::, 375 2001:DB8:B:4:C52:: 377 Sending 5, 100-byte ICMP Echos to B5::, timeout is 2 seconds: 378 !!!!! 379 Success rate is 100 percent (5/5), round-trip min/avg/max = 0.625 380 /0.749/0.931 ms 382 Figure 2 A sample ping output at an SRv6 capable node 384 All transit nodes process the echo request message like any other 385 data packet carrying SR header and hence do not require any change. 386 Similarly, the egress node (IPv6 classic or SRv6 capable) does not 387 require any change to process the ICMPv6 echo request. For example, 388 in the ping example of Figure 2: 390 o Node N1 initiates an ICMPv6 ping packet with SRH as follows 391 (2001:DB8:A:1::, 2001:DB8:B:2:C31::) (2001:DB8:A:5::, 392 2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::, SL=2, NH = ICMPv6)(ICMPv6 393 Echo Request). 395 o Node N2, which is an SRv6 capable node, performs the standard SRH 396 processing. Specifically, it executes the END.X behavior 397 (2001:DB8:B:2:C31::) and forwards the packet on link3 to N3. 399 o Node N3, which is a classic IPv6 node, performs the standard IPv6 400 processing. Specifically, it forwards the echo request based on 401 the DA 2001:DB8:B:4:C52:: in the IPv6 header. 403 o Node N4, which is an SRv6 capable node, performs the standard SRH 404 processing. Specifically, it observes the END.X behavior 405 (2001:DB8:B:4:C52::) and forwards the packet on link10 towards N5. 406 If 2001:DB8:B:4:C52:: is a PSP SID, The penultimate node (Node N4) 407 does not, should not and cannot differentiate between the data 408 packets and OAM probes. Specifically, if 2001:DB8:B:4:C52:: is a 409 PSP SID, node N4 executes the SID like any other data packet with 410 DA = 2001:DB8:B:4:C52:: and removes the SRH. 412 o The echo request packet at N5 arrives as an IPv6 packet with or 413 without an SRH. If N5 receives the packet with SRH, it skips SRH 414 processing (SL=0). In either case, Node N5 performs the standard 415 IPv6/ ICMPv6 processing on the echo request. 417 3.1.2. Pinging a SID 419 The classic ping described in the previous section applies equally to 420 perform SID connectivity checks and to validate the availability of a 421 remote SID. This is explained using an example in the following. 422 The example uses ping to an END SID (refer [I-D.ietf-spring-srv6- 423 network-programming]) but the procedure is equally applicable to ping 424 any other SID behaviors. 426 Consider the example where the user wants to ping a remote SID 427 2001:DB8:B:4::, via 2001:DB8:B:2:C31::, from node N1. The ICMPv6 428 echo request is processed at the individual nodes along the path as 429 follows: 431 o Node N1 initiates an ICMPv6 ping packet with SRH as follows 432 (2001:DB8:A:1::, 2001:DB8:B:2:C31::) (2001:DB8:B:4::, 433 2001:DB8:B:2:C31::; SL=1; NH=ICMPv6)(ICMPv6 Echo Request). 435 o Node N2, which is an SRv6 capable node, performs the standard SRH 436 processing. Specifically, it executes the END.X behavior 437 (2001:DB8:B:2:C31::) on the echo request packet. If 438 2001:DB8:B:2:C31:: is a PSP SID, node N4 executes the SID like any 439 other data packet with DA = 2001:DB8:B:2:C31:: and removes the 440 SRH. 442 o Node N3, which is a classic IPv6 node, performs the standard IPv6 443 processing. Specifically, it forwards the echo request based on 444 DA = 2001:DB8:B:4:: in the IPv6 header. 446 o When node N4 receives the packet, it processes the target SID 447 (2001:DB8:B:4::). 449 o If the target SID (2001:DB8:B:4::) is not locally instantiated, 450 the packet is discarded 452 o If the target SID (2001:DB8:B:4::) is locally instantiated, the 453 node processes the upper layer header. As part of the upper layer 454 header processing node N4 respond to the ICMPv6 echo request 455 message. 457 3.2. Traceroute 459 There is no hardware or software change required for traceroute 460 operation at the classic IPv6 nodes in an SRv6 network. That 461 includes the classic IPv6 node with ingress, egress or transit roles. 462 Furthermore, no protocol changes are required to the standard 463 traceroute operations. In other words, existing traceroute 464 mechanisms work seamlessly in the SRv6 networks. 466 The following subsections outline some use cases of the traceroute in 467 the SRv6 networks. 469 3.2.1. Classic Traceroute 471 The existing mechanism to traceroute a remote IP address, along the 472 shortest path, continues to work without any modification. The 473 initiator may be an SRv6 node or a classic IPv6 node. Similarly, the 474 egress or transit may be an SRv6 node or a classic IPv6 node. 476 If an SRv6 capable ingress node wants to traceroute to IPv6 address 477 via an arbitrary segment list , it needs to initiate 478 traceroute probe with an SR header containing the SID list . That is illustrated using the topology in Figure 1. Assume all 480 the links have IGP metric 10 except both links between node2 and 481 node3, which have IGP metric set to 100. User issues a traceroute 482 from node N1 to a loopback of node 5, via segment list 483 <2001:DB8:B:2:C31::, 2001:DB8:B:4:C52::>. The SID behavior used in 484 the example is End.X SID (refer [I-D.ietf-spring-srv6-network- 485 programming]) but the procedure is equally applicable to any other 486 (transit) SID type. Figure 3 contains sample output for the 487 traceroute request. 489 > traceroute 2001:DB8:A:5:: via segment-list 2001:DB8:B:2:C31::, 490 2001:DB8:B:4:C52:: 492 Tracing the route to 2001:DB8:A:5:: 493 1 2001:DB8:1:2:21:: 0.512 msec 0.425 msec 0.374 msec 494 DA: 2001:DB8:B:2:C31::, 495 SRH:(2001:DB8:A:5::, 2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::, SL=2) 496 2 2001:DB8:2:3:31:: 0.721 msec 0.810 msec 0.795 msec 497 DA: 2001:DB8:B:4:C52::, 498 SRH:(2001:DB8:A:5::, 2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::, SL=1) 499 3 2001:DB8:3:4::41:: 0.921 msec 0.816 msec 0.759 msec 500 DA: 2001:DB8:B:4:C52::, 501 SRH:(2001:DB8:A:5::, 2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::, SL=1) 502 4 2001:DB8:4:5::52:: 0.879 msec 0.916 msec 1.024 msec 503 DA: 2001:DB8:A:5:: 505 Figure 3 A sample traceroute output at an SRv6 capable node 507 Please note that information for hop2 is returned by N3, which is a 508 classic IPv6 node. Nonetheless, the ingress node is able to display 509 SR header contents as the packet travels through the IPv6 classic 510 node. This is because the "Time Exceeded Message" ICMPv6 message can 511 contain as much of the invoking packet as possible without the ICMPv6 512 packet exceeding the minimum IPv6 MTU [RFC4443]. The SR header is 513 also included in these ICMPv6 messages initiated by the classic IPv6 514 transit nodes that are not running SRv6 software. Specifically, a 515 node generating ICMPv6 message containing a copy of the invoking 516 packet does not need to understand the extension header(s) in the 517 invoking packet. 519 The segment list information returned for hop1 is returned by N2, 520 which is an SRv6 capable node. Just like for hop2, the ingress node 521 is able to display SR header contents for hop1. 523 There is no difference in processing of the traceroute probe at an 524 IPv6 classic node and an SRv6 capable node. Similarly, both IPv6 525 classic and SRv6 capable nodes may use the address of the interface 526 on which probe was received as the source address in the ICMPv6 527 response. ICMP extensions defined in [RFC5837] can be used to also 528 display information about the IP interface through which the datagram 529 would have been forwarded had it been forwardable, and the IP next 530 hop to which the datagram would have been forwarded, the IP interface 531 upon which a datagram arrived, the sub-IP component of an IP 532 interface upon which a datagram arrived. 534 The information about the IP address of the incoming interface on 535 which the traceroute probe was received by the reporting node is very 536 useful. This information can also be used to verify if SIDs 537 2001:DB8:B:2:C31:: and 2001:DB8:B:4:C52:: are executed correctly by 538 N2 and N4, respectively. Specifically, the information displayed for 539 hop2 contains the incoming interface address 2001:DB8:2:3:31:: at N3. 540 This matches with the expected interface bound to END.X behavior 541 2001:DB8:B:2:C31:: (link3). Similarly, the information displayed for 542 hop5 contains the incoming interface address 2001:DB8:4:5::52:: at 543 N5. This matches with the expected interface bound to the END.X 544 behavior 2001:DB8:B:4:C52:: (link10). 546 3.2.2. Traceroute to a SID 548 The classic traceroute described in the previous section applies 549 equally to traceroute a remote SID behavior, as explained using an 550 example in the following. The example uses traceroute to an END SID 551 (refer [I-D.ietf-spring-srv6-network-programming]) but the procedure 552 is equally applicable to tracerouting any other SID behaviors. 554 Please note that traceroute to a SID is exemplified using UDP probes. 555 However, the procedure is equally applicable to other implementations 556 of traceroute mechanism. 558 Consider the example where the user wants to traceroute a remote SID 559 2001:DB8:B:4::, via 2001:DB8:B:2:C31::, from node N1. The traceroute 560 probe is processed at the individual nodes along the path as follows: 562 o Node N1 initiates a traceroute probe packet with a monotonically 563 increasing value of hop count and SRH as follows (2001:DB8:A:1::, 564 2001:DB8:B:2:C31::) (2001:DB8:B:4::, 2001:DB8:B:2:C31::; SL=1; 565 NH=UDP)(Traceroute probe). 567 o When node N2 receives the packet with hop-count = 1, it processes 568 the hop count expiry. Specifically, the node N2 responses with 569 the ICMPv6 message (Type: "Time Exceeded", Code: "Time to Live 570 exceeded in Transit"). 572 o When Node N2 receives the packet with hop-count > 1, it performs 573 the standard SRH processing. Specifically, it executes the END.X 574 behavior (2001:DB8:B:2:C31::) on the traceroute probe. If 575 2001:DB8:B:2:C31:: is a PSP SID, node N4 executes the SID like any 576 other data packet with DA = 2001:DB8:B:2:C31:: and removes the 577 SRH. 579 o When node N3, which is a classic IPv6 node, receives the packet 580 with hop-count = 1, it processes the hop count expiry. 581 Specifically, the node N3 responses with the ICMPv6 message (Type: 582 "Time Exceeded", Code: "Time to Live exceeded in Transit"). 584 o When node N3, which is a classic IPv6 node, receives the packet 585 with hop-count > 1, it performs the standard IPv6 processing. 586 Specifically, it forwards the traceroute probe based on DA 587 2001:DB8:B:4:: in the IPv6 header. 589 o When node N4 receives the packet with DA set to the local SID 590 2001:DB8:B:4::, it processes the END SID. 592 o If the target SID (2001:DB8:B:4::) is not locally instantiated, 593 the packet is discarded. 595 o If the target SID (2001:DB8:B:4::) is locally instantiated, the 596 node processes the upper layer header. As part of the upper layer 597 header processing node N4 responses with the ICMPv6 message (Type: 598 Destination unreachable, Code: Port Unreachable). 600 Figure 4 displays a sample traceroute output for this example. 602 > traceroute 2001:DB8:B:4:C52:: via segment-list 2001:DB8:B:2:C31:: 604 Tracing the route to SID 2001:DB8:B:4:C52:: 605 1 2001:DB8:1:2:21:: 0.512 msec 0.425 msec 0.374 msec 606 DA: 2001:DB8:B:2:C31::, 607 SRH:(2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::; SL=1) 608 2 2001:DB8:2:3:31:: 0.721 msec 0.810 msec 0.795 msec 609 DA: 2001:DB8:B:4:C52::, 610 SRH:(2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::; SL=0) 611 3 2001:DB8:3:4:41:: 0.921 msec 0.816 msec 0.759 msec 612 DA: 2001:DB8:B:4:C52::, 613 SRH:(2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::; SL=0) 615 Figure 4 A sample output for hop-by-hop traceroute to a SID 617 3.3. A Hybrid OAM Using O-flag 619 This section illustrates a hybrid OAM mechanism using the the 620 SRH.Flags.O-flag. Without loss of the generality, the illustration 621 assumes N100 is a centralized controller. 623 The illustration is different than the In-situ OAM defined in [I.D- 624 draft-ietf-ippm-ioam-data]. This is because In-situ OAM records 625 operational and telemetry information in the packet as the packet 626 traverses a path between two points in the network [I.D-draft-ietf- 627 ippm-ioam-data]. The illustration in section 3 does not require the 628 recording of OAM data in the packet. 630 The illustration does not assume any formats for exporting the data 631 elements or the data elements that needs to be exported. 633 Consider the example where the user wants to monitor sampled IPv4 VPN 634 100 traffic going from CE1 to CE2 via a low latency SR policy P 635 installed at Node N1. To exercise a low latency path, the SR Policy 636 P forces the packet via segments 2001:DB8:B:2:C31:: and 637 2001:DB8:B:4:C52::. The VPN SID at N7 associated with VPN100 is 638 2001:DB8:B:7:DT100::. 2001:DB8:B:7:DT100:: is a USP SID. N1, N4, 639 and N7 are capable of processing SRH.Flags.O-flag but N2 is not 640 capable of processing SRH.Flags.O-flag. N100 is the centralized 641 controller capable of processing and correlating the copy of the 642 packets sent from nodes N1, N4, and N7. N100 is aware of 643 SRH.Flags.O-flag processing capabilities. Controller N100 with the 644 help from nodes N1, N4, N7 and implements a hybrid OAM mechanism 645 using the SRH.Flags.O-flag as follows: 647 o A packet P1:(IPv4 header)(payload) is sent from CE1 to Node N1. 649 o Node N1 steers the packet P1 through the Policy P. Based on a 650 local configuration, Node N1 also implements logic to sample 651 traffic steered through policy P for hybrid OAM purposes. 652 Specification for the sampling logic is beyond the scope of this 653 document. Consider the case where packet P1 is classified as a 654 packet to be monitored via the hybrid OAM. Node N1 sets 655 SRH.Flags.O-flag during encapsulation required by policy P. As 656 part of setting the SRH.Flags.O-flag, node N1 also send a 657 timestamped copy of the packet P1: (2001:DB8:A:1::, 658 2001:DB8:B:2:C31::) (2001:DB8:B:7:DT100::, 2001:DB8:B:4:C52::, 659 2001:DB8:B:2:C31::; SL=2; O-flag=1; NH=IPv4)(IPv4 header)(payload) 660 to a local OAM process. The local OAM process sends a full or 661 partial copy of the packet P1 to the controller N100. The OAM 662 process includes the recorded timestamp, additional OAM 663 information like incoming and outgoing interface, etc. along with 664 any applicable metadata. Node N1 forwards the original packet 665 towards the next segment 2001:DB8:B:2:C31::. 667 o When node N2 receives the packet with SRH.Flags.O-flag set, it 668 ignores the SRH.Flags.O-flag. This is because node N2 is not 669 capable of processing the O-flag. Node N2 performs the standard 670 SRv6 SID and SRH processing. Specifically, it executes the END.X 671 (refer [I-D.ietf-spring-srv6-network-programming]) behavior 672 (2001:DB8:B:2:C31::) and forwards the packet P1 (2001:DB8:A:1::, 673 2001:DB8:B:4:C52::) (2001:DB8:B:7:DT100::, 2001:DB8:B:4:C52::, 674 2001:DB8:B:2:C31::; SL=1; O-flag=1; NH=IPv4)(IPv4 header)(payload) 675 over link 3 towards Node N3. 677 o When node N3, which is a classic IPv6 node, receives the packet P1 678 , it performs the standard IPv6 processing. Specifically, it 679 forwards the packet P1 based on DA 2001:DB8:B:4:C52:: in the IPv6 680 header. 682 o When node N4 receives the packet P1 (2001:DB8:A:1::, 683 2001:DB8:B:4:C52::) (2001:DB8:B:7:DT100::, 2001:DB8:B:4:C52::, 684 2001:DB8:B:2:C31::; SL=1; O-flag=1; NH=IPv4)(IPv4 685 header)(payload), it processes the SRH.Flags.O-flag. As part of 686 processing the O-flag, it sends a timestamped copy of the packet 687 to a local OAM process. The local OAM process sends a full or 688 partial copy of the packet P1 to the controller N100. The OAM 689 process includes the recorded timestamp, additional OAM 690 information like incoming and outgoing interface, etc. along with 691 any applicable metadata. Node N4 performs the standard SRv6 SID 692 and SRH processing on the original packet P1. Specifically, it 693 executes the END.X behavior (2001:DB8:B:4:C52::) and forwards the 694 packet P1 (2001:DB8:A:1::, 2001:DB8:B:7:DT100::) 695 (2001:DB8:B:7:DT100::, 2001:DB8:B:4:C52::, 2001:DB8:B:2:C31::; 696 SL=0; O-flag=1; NH=IPv4)(IPv4 header)(payload) over link 10 697 towards Node N5. 699 o When node N5, which is a classic IPv6 node, receives the packet 700 P1, it performs the standard IPv6 processing. Specifically, it 701 forwards the packet based on DA 2001:DB8:B:7:DT100:: in the IPv6 702 header. 704 o When node N7 receives the packet P1 (2001:DB8:A:1::, 705 2001:DB8:B:7:DT100::) (2001:DB8:B:7:DT100::, 2001:DB8:B:4:C52::, 706 2001:DB8:B:2:C31::; SL=0; O-flag=1; NH=IPv4)(IPv4 707 header)(payload), it processes the SRH.Flags.O-flag. As part of 708 processing the O-flag, it sends a timestamped copy of the packet 709 to a local OAM process. The local OAM process sends a full or 710 partial copy of the packet P1 to the controller N100. The OAM 711 process includes the recorded timestamp, additional OAM 712 information like incoming and outgoing interface, etc. along with 713 any applicable metadata. Node N4 performs the standard SRv6 SID 714 and SRH processing on the original packet P1. Specifically, it 715 executes the VPN SID (2001:DB8:B:7:DT100::) and based on lookup in 716 table 100 forwards the packet P1 (IPv4 header)(payload) towards CE 717 2. 719 o The controller N100 processes and correlates the copy of the 720 packets sent from nodes N1, N4 and N7 to find segment-by-segment 721 delays and provide other hybrid OAM information related to packet 722 P1. 724 o The process continues for any other sampled packets. 726 3.4. Monitoring of SRv6 Paths 728 In the recent past, network operators demonstrated interest in 729 performing network OAM functions in a centralized manner. [RFC8403] 730 describes such a centralized OAM mechanism. Specifically, the 731 document describes a procedure that can be used to perform path 732 continuity check between any nodes within an SR domain from a 733 centralized monitoring system. However, the document focuses on SR 734 networks with MPLS data plane. This document describes how the 735 concept can be used to perform path monitoring in an SRv6 network 736 from a centralized controller. 738 In the reference topology in Figure 1, N100 uses an IGP protocol like 739 OSPF or ISIS to get the topology view within the IGP domain. N100 740 can also use BGP-LS to get the complete view of an inter-domain 741 topology. The controller leverages the visibility of the topology to 742 monitor the paths between the various endpoints. 744 The controller N100 advertises an END (refer [I-D.ietf-spring-srv6- 745 network-programming]) SID 2001:DB8:B:100:1::. To monitor any 746 arbitrary SRv6 paths, the controller can create a loopback probe that 747 originates and terminates on Node N100. To distinguish between a 748 failure in the monitored path and loss of connectivity between the 749 controller and the network, Node N100 runs a suitable mechanism to 750 monitor its connectivity to the monitored network. 752 The loopback probes are exemplified using an example where controller 753 N100 needs to verify a segment list <2001:DB8:B:2:C31::, 754 2001:DB8:B:4:C52::>: 756 o N100 generates an OAM packet (2001:DB8:A:100::, 757 2001:DB8:B:2:C31::)(2001:DB8:B:100:1::, 2001:DB8:B:4:C52::, 758 2001:DB8:B:2:C31::, SL=2)(OAM Payload). The controller routes the 759 probe packet towards the first segment, which is 760 2001:DB8:B:2:C31::. 762 o Node N2 executes the END.X behavior (2001:DB8:B:2:C31::) and 763 forwards the packet (2001:DB8:A:100::, 764 2001:DB8:B:4:C52::)(2001:DB8:B:100:1::, 2001:DB8:B:4:C52::, 765 2001:DB8:B:2:C31::, SL=1)(OAM Payload) on link3 to N3. 767 o Node N3, which is a classic IPv6 node, performs the standard IPv6 768 processing. Specifically, it forwards the packet based on the DA 769 2001:DB8:B:4:C52:: in the IPv6 header. 771 o Node N4 executes the END.X behavior (2001:DB8:B:4:C52::) and 772 forwards the packet (2001:DB8:A:100::, 773 2001:DB8:B:100:1::)(2001:DB8:B:100:1::, 2001:DB8:B:4:C52::, 774 2001:DB8:B:2:C31::, SL=0)(OAM Payload) on link10 to N5. 776 o Node N5, which is a classic IPv6 node, performs the standard IPv6 777 processing. Specifically, it forwards the packet based on the DA 778 2001:DB8:B:100:1:: in the IPv6 header. 780 o Node N100 executes the standard SRv6 END behavior. It 781 decapsulates the header and consume the probe for OAM processing. 782 The information in the OAM payload is used to detect any missing 783 probes, round trip delay, etc. 785 The OAM payload type or the information carried in the OAM probe is a 786 local implementation decision at the controller and is outside the 787 scope of this document. 789 4. Implementation Status 791 This section is to be removed prior to publishing as an RFC. 793 See [I-D.matsushima-spring-srv6-deployment-status] for updated 794 deployment and interoperability reports. 796 5. Security Considerations 798 This document does not define any new protocol extensions and relies 799 on existing procedures defined for ICMP. This document does not 800 impose any additional security challenges to be considered beyond 801 security considerations described in [RFC4884], [RFC4443], [RFC0792], 802 and [RFC8754]. 804 6. IANA Considerations 806 This document requests that IANA allocate the following registrations 807 in the "Segment Routing Header Flags" sub-registry for the "Internet 808 Protocol Version 6 (IPv6) Parameters" registry maintained by IANA: 810 +-------+------------------------------+---------------+ 811 | Bit | Description | Reference | 812 +=======+==============================+===============+ 813 | 2 | O-flag | This document | 814 +-------+------------------------------+---------------+ 816 7. Acknowledgements 818 The authors would like to thank Joel M. Halpern, Greg Mirsky, Bob 819 Hinden, Loa Andersson, Gaurav Naik, Ketan Talaulikar and Haoyu Song 820 for their review comments. 822 8. Contributors 824 The following people have contributed to this document: 826 Robert Raszuk 827 Bloomberg LP 828 Email: robert@raszuk.net 829 John Leddy 830 Individual 831 Email: john@leddy.net 833 Gaurav Dawra 834 LinkedIn 835 Email: gdawra.ietf@gmail.com 837 Bart Peirens 838 Proximus 839 Email: bart.peirens@proximus.com 841 Nagendra Kumar 842 Cisco Systems, Inc. 843 Email: naikumar@cisco.com 845 Carlos Pignataro 846 Cisco Systems, Inc. 847 Email: cpignata@cisco.com 849 Rakesh Gandhi 850 Cisco Systems, Inc. 851 Canada 852 Email: rgandhi@cisco.com 854 Frank Brockners 855 Cisco Systems, Inc. 856 Germany 857 Email: fbrockne@cisco.com 859 Darren Dukes 860 Cisco Systems, Inc. 861 Email: ddukes@cisco.com 863 Cheng Li 864 Huawei 865 Email: chengli13@huawei.com 866 Faisal Iqbal 867 Individual 868 Email: faisal.ietf@gmail.com 870 9. References 872 9.1. Normative References 874 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 875 Requirement Levels", BCP 14, RFC 2119, 876 DOI 10.17487/RFC2119, March 1997, 877 . 879 [RFC8754] Filsfils, C., Ed., Dukes, D., Ed., Previdi, S., Leddy, J., 880 Matsushima, S., and D. Voyer, "IPv6 Segment Routing Header 881 (SRH)", RFC 8754, DOI 10.17487/RFC8754, March 2020, 882 . 884 9.2. Informative References 886 [I-D.gandhi-spring-stamp-srpm] 887 Gandhi, R., Filsfils, C., Voyer, D., Chen, M., and B. 888 Janssens, "Performance Measurement Using Simple TWAMP 889 (STAMP) for Segment Routing Networks", draft-gandhi- 890 spring-stamp-srpm-03 (work in progress), October 2020. 892 [I-D.gandhi-spring-twamp-srpm] 893 Gandhi, R., Filsfils, C., Voyer, D., Chen, M., and B. 894 Janssens, "Performance Measurement Using TWAMP Light for 895 Segment Routing Networks", draft-gandhi-spring-twamp- 896 srpm-11 (work in progress), October 2020. 898 [I-D.ietf-spring-srv6-network-programming] 899 Filsfils, C., Camarillo, P., Leddy, J., Voyer, D., 900 Matsushima, S., and Z. Li, "SRv6 Network Programming", 901 draft-ietf-spring-srv6-network-programming-24 (work in 902 progress), October 2020. 904 [I-D.matsushima-spring-srv6-deployment-status] 905 Matsushima, S., Filsfils, C., Ali, Z., Li, Z., and K. 906 Rajaraman, "SRv6 Implementation and Deployment Status", 907 draft-matsushima-spring-srv6-deployment-status-08 (work in 908 progress), October 2020. 910 [RFC0792] Postel, J., "Internet Control Message Protocol", STD 5, 911 RFC 792, DOI 10.17487/RFC0792, September 1981, 912 . 914 [RFC4443] Conta, A., Deering, S., and M. Gupta, Ed., "Internet 915 Control Message Protocol (ICMPv6) for the Internet 916 Protocol Version 6 (IPv6) Specification", STD 89, 917 RFC 4443, DOI 10.17487/RFC4443, March 2006, 918 . 920 [RFC4884] Bonica, R., Gan, D., Tappan, D., and C. Pignataro, 921 "Extended ICMP to Support Multi-Part Messages", RFC 4884, 922 DOI 10.17487/RFC4884, April 2007, 923 . 925 [RFC5476] Claise, B., Ed., Johnson, A., and J. Quittek, "Packet 926 Sampling (PSAMP) Protocol Specifications", RFC 5476, 927 DOI 10.17487/RFC5476, March 2009, 928 . 930 [RFC5837] Atlas, A., Ed., Bonica, R., Ed., Pignataro, C., Ed., Shen, 931 N., and JR. Rivers, "Extending ICMP for Interface and 932 Next-Hop Identification", RFC 5837, DOI 10.17487/RFC5837, 933 April 2010, . 935 [RFC5880] Katz, D. and D. Ward, "Bidirectional Forwarding Detection 936 (BFD)", RFC 5880, DOI 10.17487/RFC5880, June 2010, 937 . 939 [RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, 940 "Specification of the IP Flow Information Export (IPFIX) 941 Protocol for the Exchange of Flow Information", STD 77, 942 RFC 7011, DOI 10.17487/RFC7011, September 2013, 943 . 945 [RFC7012] Claise, B., Ed. and B. Trammell, Ed., "Information Model 946 for IP Flow Information Export (IPFIX)", RFC 7012, 947 DOI 10.17487/RFC7012, September 2013, 948 . 950 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 951 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 952 May 2016, . 954 [RFC7880] Pignataro, C., Ward, D., Akiya, N., Bhatia, M., and S. 955 Pallagatti, "Seamless Bidirectional Forwarding Detection 956 (S-BFD)", RFC 7880, DOI 10.17487/RFC7880, July 2016, 957 . 959 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 960 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 961 May 2017, . 963 [RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L., 964 Decraene, B., Litkowski, S., and R. Shakir, "Segment 965 Routing Architecture", RFC 8402, DOI 10.17487/RFC8402, 966 July 2018, . 968 [RFC8403] Geib, R., Ed., Filsfils, C., Pignataro, C., Ed., and N. 969 Kumar, "A Scalable and Topology-Aware MPLS Data-Plane 970 Monitoring System", RFC 8403, DOI 10.17487/RFC8403, July 971 2018, . 973 Authors' Addresses 975 Zafar Ali 976 Cisco Systems 978 Email: zali@cisco.com 980 Clarence Filsfils 981 Cisco Systems 983 Email: cfilsfil@cisco.com 985 Satoru Matsushima 986 Softbank 988 Email: satoru.matsushima@g.softbank.co.jp 990 Daniel Voyer 991 Bell Canada 993 Email: daniel.voyer@bell.ca 995 Mach Chen 996 Huawei 998 Email: mach.chen@huawei.com