idnits 2.17.1 draft-ietf-6man-spring-srv6-oam-11.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 2, 2021) is 1059 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'I-D.ietf-ippm-ioam-data' is defined on line 941, but no explicit reference was found in the text == Outdated reference: A later version (-07) exists of draft-gandhi-spring-stamp-srpm-06 == Outdated reference: A later version (-17) exists of draft-ietf-ippm-ioam-data-11 == Outdated reference: A later version (-15) exists of draft-matsushima-spring-srv6-deployment-status-11 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 6man Z. Ali 3 Internet-Draft C. Filsfils 4 Intended status: Standards Track Cisco Systems 5 Expires: December 4, 2021 S. Matsushima 6 Softbank 7 D. Voyer 8 Bell Canada 9 M. Chen 10 Huawei 11 June 2, 2021 13 Operations, Administration, and Maintenance (OAM) in Segment Routing 14 Networks with IPv6 Data plane (SRv6) 15 draft-ietf-6man-spring-srv6-oam-11 17 Abstract 19 This document describes how the existing IPv6 mechanisms for ping and 20 traceroute can be used in an SRv6 network. The document also 21 specifies the OAM flag in the Segment Routing Header (SRH) for 22 performing controllable and predictable flow sampling from segment 23 endpoints. In addition, the document describes how a centralized 24 monitoring system performs a path continuity check between any nodes 25 within an SRv6 domain. 27 Status of This Memo 29 This Internet-Draft is submitted in full conformance with the 30 provisions of BCP 78 and BCP 79. 32 Internet-Drafts are working documents of the Internet Engineering 33 Task Force (IETF). Note that other groups may also distribute 34 working documents as Internet-Drafts. The list of current Internet- 35 Drafts is at https://datatracker.ietf.org/drafts/current/. 37 Internet-Drafts are draft documents valid for a maximum of six months 38 and may be updated, replaced, or obsoleted by other documents at any 39 time. It is inappropriate to use Internet-Drafts as reference 40 material or to cite them other than as "work in progress." 42 This Internet-Draft will expire on December 4, 2021. 44 Copyright Notice 46 Copyright (c) 2021 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents 51 (https://trustee.ietf.org/license-info) in effect on the date of 52 publication of this document. Please review these documents 53 carefully, as they describe your rights and restrictions with respect 54 to this document. Code Components extracted from this document must 55 include Simplified BSD License text as described in Section 4.e of 56 the Trust Legal Provisions and are provided without warranty as 57 described in the Simplified BSD License. 59 Table of Contents 61 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 62 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 63 1.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 3 64 1.3. Terminology and Reference Topology . . . . . . . . . . . 4 65 2. OAM Mechanisms . . . . . . . . . . . . . . . . . . . . . . . 5 66 2.1. O-flag in Segment Routing Header . . . . . . . . . . . . 5 67 2.1.1. O-flag Processing . . . . . . . . . . . . . . . . . . 6 68 2.2. OAM Operations . . . . . . . . . . . . . . . . . . . . . 7 69 3. Illustrations . . . . . . . . . . . . . . . . . . . . . . . . 8 70 3.1. Ping in SRv6 Networks . . . . . . . . . . . . . . . . . . 8 71 3.1.1. Classic Ping . . . . . . . . . . . . . . . . . . . . 8 72 3.1.2. Pinging a SID . . . . . . . . . . . . . . . . . . . . 10 73 3.2. Traceroute . . . . . . . . . . . . . . . . . . . . . . . 11 74 3.2.1. Classic Traceroute . . . . . . . . . . . . . . . . . 11 75 3.2.2. Traceroute to a SID . . . . . . . . . . . . . . . . . 13 76 3.3. A Hybrid OAM Using O-flag . . . . . . . . . . . . . . . . 15 77 3.4. Monitoring of SRv6 Paths . . . . . . . . . . . . . . . . 17 78 4. Implementation Status . . . . . . . . . . . . . . . . . . . . 18 79 5. Security Considerations . . . . . . . . . . . . . . . . . . . 18 80 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19 81 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 19 82 8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 19 83 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 20 84 9.1. Normative References . . . . . . . . . . . . . . . . . . 21 85 9.2. Informative References . . . . . . . . . . . . . . . . . 21 86 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 23 88 1. Introduction 90 As Segment Routing with IPv6 data plane (SRv6) [RFC8402] simply adds 91 a new type of Routing Extension Header, existing IPv6 OAM mechanisms 92 can be used in an SRv6 network. This document describes how the 93 existing IPv6 mechanisms for ping and traceroute can be used in an 94 SRv6 network. This includes illustrations of pinging an SRv6 SID to 95 verify that the SID is reachable and is locally programmed at the 96 target node. This also includes illustrations for tracerouting to an 97 SRv6 SID for hop-by-hop fault localization as well as path tracing to 98 a SID. 100 The document also introduces enhancements for OAM mechanism for SRv6 101 networks for performing controllable and predictable flow sampling 102 from segment endpoints using, e.g., IP Flow Information Export 103 (IPFIX) protocol [RFC7011]. Specifically, the document specifies the 104 O-flag in SRH as a marking-bit in the user packets to trigger the 105 telemetry data collection and export at the segment endpoints. 107 The document also outlines how centralized OAM technique in [RFC8403] 108 can be extended for SRv6 to perform a path continuity check between 109 any nodes within an SRv6 domain. Specifically, the document 110 illustrates how a centralized monitoring system can monitor arbitrary 111 SRv6 paths by creating the loopback probes that originates and 112 terminates at the centralized monitoring system. 114 1.1. Requirements Language 116 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 117 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 118 "OPTIONAL" in this document are to be interpreted as described in BCP 119 14 [RFC2119] [RFC8174] when, and only when, they appear in all 120 capitals, as shown here. 122 1.2. Abbreviations 124 The following abbreviations are used in this document: 126 SID: Segment ID. 128 SL: Segments Left. 130 SR: Segment Routing. 132 SRH: Segment Routing Header [RFC8754]. 134 SRv6: Segment Routing with IPv6 Data plane. 136 TC: Traffic Class. 138 ICMPv6: ICMPv6 Specification [RFC4443]. 140 IS-IS: Intermediate System to Intermediate System 142 OSPF: Open Shortest Path First protocol [RFC2328] 144 IGP: Interior Gateway Protocols (e.g., OSPF, IS-IS). 146 BGP-LS: Border Gateway Protocol - Link State Extensions [RFC8571] 148 1.3. Terminology and Reference Topology 150 Throughout the document, the following terminology and simple 151 topology is used for illustration. 153 +--------------------------| N100 |---------------------------------+ 154 | | 155 | ====== link1====== link3------ link5====== link9------ ====== | 156 ||N1||------||N2||------| N3 |------||N4||------| N5 |---||N7|| 157 || ||------|| ||------| |------|| ||------| |---|| || 158 ====== link2====== link4------ link6======link10------ ====== 159 | | | | 160 ---+-- | ------ | --+--- 161 |CE 1| +-------| N6 |---------+ |CE 2| 162 ------ link7 | | link8 ------ 163 ------ 165 Figure 1 Reference Topology 167 In the reference topology: 169 Node k has a IPv6 loopback address 2001:db8::A:k::/128. 171 Nodes N1, N2, N4 and N7 are SRv6-capable nodes. 173 Nodes N3, N5 and N6 are IPv6 nodes that are not SRv6-capable. 174 Such nodes are referred as classic IPv6 nodes. 176 CE1 and CE2 are Customer Edge devices of any data plane capability 177 (e.g., IPv4, IPv6, L2, etc.). 179 A SID at node k with locator block 2001:db8:B::/48 and function F 180 is represented by 2001:db8:B:k:F::. 182 Node N100 is a controller. 184 The IPv6 address of the nth Link between node i and j at the i 185 side is represented as 2001:db8:i:j:in::, e.g., the IPv6 address 186 of link6 (the 2nd link) between N3 and N4 at N3 in Figure 1 is 187 2001:db8:3:4:32::. Similarly, the IPv6 address of link5 (the 1st 188 link between N3 and N4) at node 3 is 2001:db8:3:4:31::. 190 2001:db8:B:k:Cij:: is explicitly allocated as the END.X SID at 191 node k towards neighbor node i via jth Link between node i and 192 node k. e.g., 2001:db8:B:2:C31:: represents END.X at N2 towards 193 N3 via link3 (the 1st link between N2 and N3). Similarly, 194 2001:db8:B:4:C52:: represents the END.X at N4 towards N5 via 195 link10. Please refer to [RFC8986] for description of END.X SID. 197 A SID list is represented as where S1 is the first 198 SID to visit, S2 is the second SID to visit and S3 is the last SID 199 to visit along the SR path. 201 (SA,DA) (S3, S2, S1; SL)(payload) represents an IPv6 packet with: 203 * IPv6 header with source address SA, destination addresses DA 204 and SRH as next-header 206 * SRH with SID list with SegmentsLeft = SL 208 * Note the difference between the < > and () symbols: represents a SID list where S1 is the first SID and S3 is 210 the last SID to traverse. (S3, S2, S1; SL) represents the same 211 SID list but encoded in the SRH format where the rightmost SID 212 in the SRH is the first SID and the leftmost SID in the SRH is 213 the last SID. When referring to an SR policy in a high-level 214 use-case, it is simpler to use the notation. When 215 referring to an illustration of the detailed packet behavior, 216 the (S3, S2, S1; SL) notation is more convenient. 218 * (payload) represents the the payload of the packet. 220 2. OAM Mechanisms 222 This section defines OAM enhancement for the SRv6 networks. 224 2.1. O-flag in Segment Routing Header 226 [RFC8754] describes the Segment Routing Header (SRH) and how SR 227 capable nodes use it. The SRH contains an 8-bit "Flags" field. 229 This document defines the following bit in the SRH Flags field to 230 carry the O-flag: 232 0 1 2 3 4 5 6 7 233 +-+-+-+-+-+-+-+-+ 234 | |O| | 235 +-+-+-+-+-+-+-+-+ 237 Where: 239 O-flag: OAM flag in the SRH Flags field defined in [RFC8754] . 241 2.1.1. O-flag Processing 243 The O-flag in SRH is used as a marking-bit in the user packets to 244 trigger the telemetry data collection and export at the segment 245 endpoints. 247 This document does not specify the data elements that need to be 248 exported and the associated configurations. Similarly, this document 249 does not define any formats for exporting the data elements. 250 Nonetheless, without the loss of generality, this document assumes IP 251 Flow Information Export (IPFIX) protocol [RFC7011] is used for 252 exporting the traffic flow information from the network devices to a 253 controller for monitoring and analytics. Similarly, without the loss 254 of generality, this document assumes requested information elements 255 are configured by the management plane through data set templates 256 (e.g., as in IPFIX [RFC7012]). 258 Implementation of the O-flag is OPTIONAL. If a node does not support 259 the O-flag, then upon reception it simply ignores it. If a node 260 supports the O-flag, it can optionally advertise its potential via 261 control plane protocol(s). 263 When N receives a packet whose IPv6 DA is S and S is a local SID, the 264 line S01 of the pseudo-code associated with the SID S, as defined in 265 section 4.3.1.1 of [RFC8754], is appended as follows for the O-flag 266 processing. 268 S01.1. IF O-flag is set and local configuration permits 269 O-flag processing { 270 a. Make a copy of the packet. 271 b. Send the copied packet, along with a timestamp 272 to the OAM process for telemetry data collection 273 and export. ;; Ref1 274 } 275 Ref1: An implementation SHOULD copy and record the timestamp as 276 soon as possible during packet processing. Timestamp or any other 277 metadata is not 278 carried in the packet forwarded to the next hop. 280 Please note that the O-flag processing happens before execution of 281 regular processing of the local SID S. Specifically, the line S01.1 282 of the pseudo-code specified in this document is inserted between 283 line S01 and S02 of the pseudo-code defined in section 4.3.1.1 of 284 [RFC8754]. 286 Based on the requested information elements configured by the 287 management plane through data set templates [RFC7012], the OAM 288 process exports the requested information elements. The information 289 elements include parts of the packet header and/or parts of the 290 packet payload for flow identification. The OAM process uses 291 information elements defined in IPFIX [RFC7011] and PSAMP [RFC5476] 292 for exporting the requested sections of the mirrored packets. 294 If the telemetry data from the ultimate segment in the segment-list 295 is required, a penultimate segment SHOULD NOT be a Penultimate 296 Segment Pop (PSP) SID. When the penultimate segment is a PSP SID, 297 the SRH will be removed and the O-flag will not be processed at the 298 ultimate segment. 300 The processing node SHOULD rate-limit the number of packets punted to 301 the OAM process to a configurable rate. This is to avoid hitting any 302 performance impact on the OAM and the telemetry collection processes. 303 Failure in implementing the rate limit can lead to a denial-of- 304 service attack, as detailed in Section 5. 306 The OAM process MUST NOT process the copy of the packet or respond to 307 any upper-layer header (like ICMP, UDP, etc.) payload to prevent 308 multiple evaluations of the datagram. 310 The OAM process is expected to be located on the routing node 311 processing the packet. Although the specification of the OAM process 312 or the external controller operations are beyond the scope of this 313 document, the OAM process SHOULD NOT be topologically distant from 314 the routing node, as this is likely to create significant security 315 and congestion issues. How to correlate the data collected from 316 different nodes at an external controller is also outside the scope 317 of the document. Section 3 illustrates use of the O-flag for 318 implementing a hybrid OAM mechanism, where the "hybrid" 319 classification is based on RFC7799 [RFC7799]. 321 2.2. OAM Operations 323 IPv6 OAM operations can be performed for any SRv6 SID whose behavior 324 allows Upper Layer Header processing for an applicable OAM payload 325 (e.g., ICMP, UDP). 327 Ping to an SRv6 SID is used to verify that the SID is reachable and 328 is locally programmed at the target node. Traceroute to a SID is 329 used for hop-by-hop fault localization as well as path tracing to a 330 SID. Section 3 illustrates the ICMPv6 based ping and the UDP based 331 traceroute mechanisms for ping and traceroute to an SRv6 SID. 332 Although this document only illustrates ICMPv6 ping and UDP based 333 traceroute to an SRv6 SID, the procedures are equally applicable to 334 other IPv6 OAM probing to an SRv6 SID (e.g., Bidirectional Forwarding 335 Detection (BFD) [RFC5880], Seamless BFD (SBFD) [RFC7880], STAMP probe 336 message processing [I-D.gandhi-spring-stamp-srpm], etc.). 337 Specifically, as long as local configuration allows the Upper-layer 338 Header processing of the applicable OAM payload for SRv6 SIDs, the 339 existing IPv6 OAM techniques can be used to target a probe to a 340 (remote) SID. 342 IPv6 OAM operations can be performed with the target SID in the IPv6 343 destination address without SRH or with SRH where the target SID is 344 the last segment. In general, OAM operations to a target SID may not 345 exercise all of its processing depending on its behavior definition. 346 For example, ping to an END.X SID [RFC8986] only validates the SID is 347 locally programmed at the target node and does not validate switching 348 to the correct outgoing interface. To exercise the behavior of a 349 target SID, the OAM operation SHOULD construct the probe in a manner 350 similar to a data packet that exercises the SID behavior, i.e. to 351 include that SID as a transit SID in either an SRH or IPv6 DA of an 352 outer IPv6 header or as appropriate based on the definition of the 353 SID behavior. 355 3. Illustrations 357 This section shows how some of the existing IPv6 OAM mechanisms can 358 be used in an SRv6 network. It also illustrates an OAM mechanism for 359 performing controllable and predictable flow sampling from segment 360 endpoints. How centralized OAM technique in [RFC8403] can be 361 extended for SRv6 is also described in this Section. 363 3.1. Ping in SRv6 Networks 365 The following subsections outline some use cases of the ICMPv6 ping 366 in the SRv6 networks. 368 3.1.1. Classic Ping 370 The existing mechanism to perform the reachability checks, along the 371 shortest path, continues to work without any modification. The 372 initiator may be an SRv6 node or a classic IPv6 node. Similarly, the 373 egress or transit may be an SRv6-capable node or a classic IPv6 node. 375 If an SRv6-capable ingress node wants to ping an IPv6 address via an 376 arbitrary segment list , it needs to initiate ICMPv6 ping 377 with an SR header containing the SID list . This is 378 illustrated using the topology in Figure 1. Assume all the links 379 have IGP metric 10 except both links between N2 and N3, which have 380 IGP metric set to 100. User issues a ping from node N1 to a loopback 381 of node 5, via segment list <2001:db8:B:2:C31::, 2001:db8:B:4:C52::>. 382 The SID behavior used in the example is End.X SID, as described in 384 [RFC8986], but the procedure is equally applicable to any other 385 (transit) SID type. 387 Figure 2 contains sample output for a ping request initiated at node 388 N1 to the loopback address of node N5 via a segment list 389 <2001:db8:B:2:C31::, 2001:db8:B:4:C52::>. 391 > ping 2001:db8:A:5:: via segment-list 2001:db8:B:2:C31::, 392 2001:db8:B:4:C52:: 394 Sending 5, 100-byte ICMPv6 Echos to B5::, timeout is 2 seconds: 395 !!!!! 396 Success rate is 100 percent (5/5), round-trip min/avg/max = 0.625 397 /0.749/0.931 ms 399 Figure 2 A sample ping output at an SRv6-capable node 401 All transit nodes process the echo request message like any other 402 data packet carrying SR header and hence do not require any change. 403 Similarly, the egress node (IPv6 classic or SRv6-capable) does not 404 require any change to process the ICMPv6 echo request. For example, 405 in the ping example of Figure 2: 407 o Node N1 initiates an ICMPv6 ping packet with SRH as follows 408 (2001:db8:A:1::, 2001:db8:B:2:C31::) (2001:db8:A:5::, 409 2001:db8:B:4:C52::, 2001:db8:B:2:C31::, SL=2, NH = ICMPv6)(ICMPv6 410 Echo Request). 412 o Node N2, which is an SRv6-capable node, performs the standard SRH 413 processing. Specifically, it executes the END.X behavior 414 (2001:db8:B:2:C31::) and forwards the packet on link3 to N3. 416 o Node N3, which is a classic IPv6 node, performs the standard IPv6 417 processing. Specifically, it forwards the echo request based on 418 the DA 2001:db8:B:4:C52:: in the IPv6 header. 420 o Node N4, which is an SRv6-capable node, performs the standard SRH 421 processing. Specifically, it observes the END.X behavior 422 (2001:db8:B:4:C52::) and forwards the packet on link10 towards N5. 423 If 2001:db8:B:4:C52:: is a PSP SID, The penultimate node (Node N4) 424 does not, should not and cannot differentiate between the data 425 packets and OAM probes. Specifically, if 2001:db8:B:4:C52:: is a 426 PSP SID, node N4 executes the SID like any other data packet with 427 DA = 2001:db8:B:4:C52:: and removes the SRH. 429 o The echo request packet at N5 arrives as an IPv6 packet with or 430 without an SRH. If N5 receives the packet with SRH, it skips SRH 431 processing (SL=0). In either case, Node N5 performs the standard 432 ICMPv6 processing on the echo request and responds with the echo 433 reply message to N1. The echo reply message is IP routed. 435 3.1.2. Pinging a SID 437 The classic ping described in the previous section applies equally to 438 perform SID reachability check and to validate the SID is locally 439 programmed at the target node. This is explained using an example in 440 the following. The example uses ping to an END SID, as described in 441 [RFC8986], but the procedure is equally applicable to ping any other 442 SID behaviors. 444 Consider the example where the user wants to ping a remote SID 445 2001:db8:B:4::, via 2001:db8:B:2:C31::, from node N1. The ICMPv6 446 echo request is processed at the individual nodes along the path as 447 follows: 449 o Node N1 initiates an ICMPv6 ping packet with SRH as follows 450 (2001:db8:A:1::, 2001:db8:B:2:C31::) (2001:db8:B:4::, 451 2001:db8:B:2:C31::; SL=1; NH=ICMPv6)(ICMPv6 Echo Request). 453 o Node N2, which is an SRv6-capable node, performs the standard SRH 454 processing. Specifically, it executes the END.X behavior 455 (2001:db8:B:2:C31::) on the echo request packet. If 456 2001:db8:B:2:C31:: is a PSP SID, node N4 executes the SID like any 457 other data packet with DA = 2001:db8:B:2:C31:: and removes the 458 SRH. 460 o Node N3, which is a classic IPv6 node, performs the standard IPv6 461 processing. Specifically, it forwards the echo request based on 462 DA = 2001:db8:B:4:: in the IPv6 header. 464 o When node N4 receives the packet, it processes the target SID 465 (2001:db8:B:4::). 467 o If the target SID (2001:db8:B:4::) is not locally instantiated, 468 the packet is discarded 470 o If the target SID (2001:db8:B:4::) is locally instantiated, the 471 node processes the upper layer header. As part of the upper layer 472 header processing node N4 respond to the ICMPv6 echo request 473 message and responds with the echo reply message. The echo reply 474 message is IP routed. 476 3.2. Traceroute 478 There is no hardware or software change required for traceroute 479 operation at the classic IPv6 nodes in an SRv6 network. That 480 includes the classic IPv6 node with ingress, egress or transit roles. 481 Furthermore, no protocol changes are required to the standard 482 traceroute operations. In other words, existing traceroute 483 mechanisms work seamlessly in the SRv6 networks. 485 The following subsections outline some use cases of the traceroute in 486 the SRv6 networks. 488 3.2.1. Classic Traceroute 490 The existing mechanism to traceroute a remote IP address, along the 491 shortest path, continues to work without any modification. The 492 initiator may be an SRv6 node or a classic IPv6 node. Similarly, the 493 egress or transit may be an SRv6 node or a classic IPv6 node. 495 If an SRv6-capable ingress node wants to traceroute to IPv6 address 496 via an arbitrary segment list , it needs to initiate 497 traceroute probe with an SR header containing the SID list . That is illustrated using the topology in Figure 1. Assume all 499 the links have IGP metric 10 except both links between N2 and N3, 500 which have IGP metric set to 100. User issues a traceroute from node 501 N1 to a loopback of node 5, via segment list <2001:db8:B:2:C31::, 502 2001:db8:B:4:C52::>. The SID behavior used in the example is End.X 503 SID, as described in [RFC8986], but the procedure is equally 504 applicable to any other (transit) SID type. Figure 3 contains sample 505 output for the traceroute request. 507 > traceroute 2001:db8:A:5:: via segment-list 2001:db8:B:2:C31::, 508 2001:db8:B:4:C52:: 510 Tracing the route to 2001:db8:A:5:: 511 1 2001:db8:1:2:21:: 0.512 msec 0.425 msec 0.374 msec 512 DA: 2001:db8:B:2:C31::, 513 SRH:(2001:db8:A:5::, 2001:db8:B:4:C52::, 2001:db8:B:2:C31::, SL=2) 514 2 2001:db8:2:3:31:: 0.721 msec 0.810 msec 0.795 msec 515 DA: 2001:db8:B:4:C52::, 516 SRH:(2001:db8:A:5::, 2001:db8:B:4:C52::, 2001:db8:B:2:C31::, SL=1) 517 3 2001:db8:3:4::41:: 0.921 msec 0.816 msec 0.759 msec 518 DA: 2001:db8:B:4:C52::, 519 SRH:(2001:db8:A:5::, 2001:db8:B:4:C52::, 2001:db8:B:2:C31::, SL=1) 520 4 2001:db8:4:5::52:: 0.879 msec 0.916 msec 1.024 msec 521 DA: 2001:db8:A:5:: 523 Figure 3 A sample traceroute output at an SRv6-capable node 525 In the sample traceroute output, the information displayed at each 526 hop is obtained using the contents of the "Time Exceeded" or 527 "Destination Unreachable" ICMPv6 responses. These ICMPv6 responses 528 are IP routed. 530 In the sample traceroute output, the information for link3 is 531 returned by N3, which is a classic IPv6 node. Nonetheless, the 532 ingress node is able to display SR header contents as the packet 533 travels through the IPv6 classic node. This is because the "Time 534 Exceeded Message" ICMPv6 message can contain as much of the invoking 535 packet as possible without the ICMPv6 packet exceeding the minimum 536 IPv6 MTU [RFC4443]. The SR header is also included in these ICMPv6 537 messages initiated by the classic IPv6 transit nodes that are not 538 running SRv6 software. Specifically, a node generating ICMPv6 539 message containing a copy of the invoking packet does not need to 540 understand the extension header(s) in the invoking packet. 542 The segment list information returned for the first hop is returned 543 by N2, which is an SRv6-capable node. Just like for the second hop, 544 the ingress node is able to display SR header contents for the first 545 hop. 547 There is no difference in processing of the traceroute probe at an 548 IPv6 classic node and an SRv6-capable node. Similarly, both IPv6 549 classic and SRv6-capable nodes may use the address of the interface 550 on which probe was received as the source address in the ICMPv6 551 response. ICMPv6 extensions defined in [RFC5837] can be used to also 552 display information about the IP interface through which the datagram 553 would have been forwarded had it been forwardable, and the IP next 554 hop to which the datagram would have been forwarded, the IP interface 555 upon which a datagram arrived, the sub-IP component of an IP 556 interface upon which a datagram arrived. 558 The IP address of the interface on which the traceroute probe was 559 received is useful. This information can also be used to verify if 560 SIDs 2001:db8:B:2:C31:: and 2001:db8:B:4:C52:: are executed correctly 561 by N2 and N4, respectively. Specifically, the information displayed 562 for the second hop contains the incoming interface address 563 2001:db8:2:3:31:: at N3. This matches with the expected interface 564 bound to END.X behavior 2001:db8:B:2:C31:: (link3). Similarly, the 565 information displayed for hop5 contains the incoming interface 566 address 2001:db8:4:5::52:: at N5. This matches with the expected 567 interface bound to the END.X behavior 2001:db8:B:4:C52:: (link10). 569 3.2.2. Traceroute to a SID 571 The classic traceroute described in the previous section applies 572 equally to traceroute a remote SID behavior, as explained using an 573 example in the following. The example uses traceroute to an END SID, 574 as described in [RFC8986], but the procedure is equally applicable to 575 tracerouting any other SID behaviors. 577 Please note that traceroute to a SID is exemplified using UDP probes. 578 However, the procedure is equally applicable to other implementations 579 of traceroute mechanism. The UDP encoded message to traceroute a SID 580 uses the UDP ports assigned by IANA for "traceroute use". 582 Consider the example where the user wants to traceroute a remote SID 583 2001:db8:B:4::, via 2001:db8:B:2:C31::, from node N1. The traceroute 584 probe is processed at the individual nodes along the path as follows: 586 o Node N1 initiates a traceroute probe packet with a monotonically 587 increasing value of hop count and SRH as follows (2001:db8:A:1::, 588 2001:db8:B:2:C31::) (2001:db8:B:4::, 2001:db8:B:2:C31::; SL=1; 589 NH=UDP)(Traceroute probe). 591 o When node N2 receives the packet with hop-count = 1, it processes 592 the hop count expiry. Specifically, the node N2 responses with 593 the ICMPv6 message (Type: "Time Exceeded", Code: "Hop limit 594 exceeded in transit"). The ICMPv6 response is IP routed. 596 o When Node N2 receives the packet with hop-count > 1, it performs 597 the standard SRH processing. Specifically, it executes the END.X 598 behavior (2001:db8:B:2:C31::) on the traceroute probe. If 599 2001:db8:B:2:C31:: is a PSP SID, node N4 executes the SID like any 600 other data packet with DA = 2001:db8:B:2:C31:: and removes the 601 SRH. 603 o When node N3, which is a classic IPv6 node, receives the packet 604 with hop-count = 1, it processes the hop count expiry. 605 Specifically, the node N3 responses with the ICMPv6 message (Type: 606 "Time Exceeded", Code: "Hop limit exceeded in Transit"). The 607 ICMPv6 response is IP routed. 609 o When node N3, which is a classic IPv6 node, receives the packet 610 with hop-count > 1, it performs the standard IPv6 processing. 611 Specifically, it forwards the traceroute probe based on DA 612 2001:db8:B:4:: in the IPv6 header. 614 o When node N4 receives the packet with DA set to the local SID 615 2001:db8:B:4::, it processes the END SID. 617 o If the target SID (2001:db8:B:4::) is not locally instantiated, 618 the packet is discarded. 620 o If the target SID (2001:db8:B:4::) is locally instantiated, the 621 node processes the upper layer header. As part of the upper layer 622 header processing node N4 responses with the ICMPv6 message (Type: 623 Destination unreachable, Code: Port Unreachable). The ICMPv6 624 response is IP routed. 626 Figure 4 displays a sample traceroute output for this example. 628 > traceroute 2001:db8:B:4:C52:: via segment-list 2001:db8:B:2:C31:: 630 Tracing the route to SID 2001:db8:B:4:C52:: 631 1 2001:db8:1:2:21:: 0.512 msec 0.425 msec 0.374 msec 632 DA: 2001:db8:B:2:C31::, 633 SRH:(2001:db8:B:4:C52::, 2001:db8:B:2:C31::; SL=1) 634 2 2001:db8:2:3:31:: 0.721 msec 0.810 msec 0.795 msec 635 DA: 2001:db8:B:4:C52::, 636 SRH:(2001:db8:B:4:C52::, 2001:db8:B:2:C31::; SL=0) 637 3 2001:db8:3:4:41:: 0.921 msec 0.816 msec 0.759 msec 638 DA: 2001:db8:B:4:C52::, 639 SRH:(2001:db8:B:4:C52::, 2001:db8:B:2:C31::; SL=0) 641 Figure 4 A sample output for hop-by-hop traceroute to a SID 643 3.3. A Hybrid OAM Using O-flag 645 This section illustrates a hybrid OAM mechanism using the the O-flag. 646 Without loss of the generality, the illustration assumes N100 is a 647 centralized controller. 649 The illustration is different than the In-situ OAM defined in [I.D- 650 draft-ietf-ippm-ioam-data]. This is because In-situ OAM records 651 operational and telemetry information in the packet as the packet 652 traverses a path between two points in the network [I.D-draft-ietf- 653 ippm-ioam-data]. The illustration in this subsection does not 654 require the recording of OAM data in the packet. 656 The illustration does not assume any formats for exporting the data 657 elements or the data elements that need to be exported. 659 Consider the example where the user wants to monitor sampled IPv4 VPN 660 999 traffic going from CE1 to CE2 via a low latency SR policy P 661 installed at Node N1. To exercise a low latency path, the SR Policy 662 P forces the packet via segments 2001:db8:B:2:C31:: and 663 2001:db8:B:4:C52::. The VPN SID at N7 associated with VPN 999 is 664 2001:db8:B:7:DT999::. 2001:db8:B:7:DT999:: is a USP SID. N1, N4, 665 and N7 are capable of processing O-flag but N2 is not capable of 666 processing O-flag. N100 is the centralized controller capable of 667 processing and correlating the copy of the packets sent from nodes 668 N1, N4, and N7. N100 is aware of O-flag processing capabilities. 669 Controller N100 with the help from nodes N1, N4, N7 and implements a 670 hybrid OAM mechanism using the O-flag as follows: 672 o A packet P1:(IPv4 header)(payload) is sent from CE1 to Node N1. 674 o Node N1 steers the packet P1 through the Policy P. Based on a 675 local configuration, Node N1 also implements logic to sample 676 traffic steered through policy P for hybrid OAM purposes. 677 Specification for the sampling logic is beyond the scope of this 678 document. Consider the case where packet P1 is classified as a 679 packet to be monitored via the hybrid OAM. Node N1 sets O-flag 680 during encapsulation required by policy P. As part of setting the 681 O-flag, node N1 also sends a timestamped copy of the packet P1: 682 (2001:db8:A:1::, 2001:db8:B:2:C31::) (2001:db8:B:7:DT999::, 683 2001:db8:B:4:C52::, 2001:db8:B:2:C31::; SL=2; O-flag=1; 684 NH=IPv4)(IPv4 header)(payload) to a local OAM process. The local 685 OAM process sends a full or partial copy of the packet P1 to the 686 controller N100. The OAM process includes the recorded timestamp, 687 additional OAM information like incoming and outgoing interface, 688 etc. along with any applicable metadata. Node N1 forwards the 689 original packet towards the next segment 2001:db8:B:2:C31::. 691 o When node N2 receives the packet with O-flag set, it ignores the 692 O-flag. This is because node N2 is not capable of processing the 693 O-flag. Node N2 performs the standard SRv6 SID and SRH 694 processing. Specifically, it executes the END.X behavior 695 (2001:db8:B:2:C31::) as described in [RFC8986] and forwards the 696 packet P1 (2001:db8:A:1::, 2001:db8:B:4:C52::) 697 (2001:db8:B:7:DT999::, 2001:db8:B:4:C52::, 2001:db8:B:2:C31::; 698 SL=1; O-flag=1; NH=IPv4)(IPv4 header)(payload) over link 3 towards 699 Node N3. 701 o When node N3, which is a classic IPv6 node, receives the packet P1 702 , it performs the standard IPv6 processing. Specifically, it 703 forwards the packet P1 based on DA 2001:db8:B:4:C52:: in the IPv6 704 header. 706 o When node N4 receives the packet P1 (2001:db8:A:1::, 707 2001:db8:B:4:C52::) (2001:db8:B:7:DT999::, 2001:db8:B:4:C52::, 708 2001:db8:B:2:C31::; SL=1; O-flag=1; NH=IPv4)(IPv4 709 header)(payload), it processes the O-flag. As part of processing 710 the O-flag, it sends a timestamped copy of the packet to a local 711 OAM process. Based on a local configuration, the local OAM 712 process sends a full or partial copy of the packet P1 to the 713 controller N100. The OAM process includes the recorded timestamp, 714 additional OAM information like incoming and outgoing interface, 715 etc. along with any applicable metadata. Node N4 performs the 716 standard SRv6 SID and SRH processing on the original packet P1. 717 Specifically, it executes the END.X behavior (2001:db8:B:4:C52::) 718 and forwards the packet P1 (2001:db8:A:1::, 2001:db8:B:7:DT999::) 719 (2001:db8:B:7:DT999::, 2001:db8:B:4:C52::, 2001:db8:B:2:C31::; 720 SL=0; O-flag=1; NH=IPv4)(IPv4 header)(payload) over link 10 721 towards Node N5. 723 o When node N5, which is a classic IPv6 node, receives the packet 724 P1, it performs the standard IPv6 processing. Specifically, it 725 forwards the packet based on DA 2001:db8:B:7:DT999:: in the IPv6 726 header. 728 o When node N7 receives the packet P1 (2001:db8:A:1::, 729 2001:db8:B:7:DT999::) (2001:db8:B:7:DT999::, 2001:db8:B:4:C52::, 730 2001:db8:B:2:C31::; SL=0; O-flag=1; NH=IPv4)(IPv4 731 header)(payload), it processes the O-flag. As part of processing 732 the O-flag, it sends a timestamped copy of the packet to a local 733 OAM process. The local OAM process sends a full or partial copy 734 of the packet P1 to the controller N100. The OAM process includes 735 the recorded timestamp, additional OAM information like incoming 736 and outgoing interface, etc. along with any applicable metadata. 737 Node N4 performs the standard SRv6 SID and SRH processing on the 738 original packet P1. Specifically, it executes the VPN SID 739 (2001:db8:B:7:DT999::) and based on lookup in table 100 forwards 740 the packet P1 (IPv4 header)(payload) towards CE 2. 742 o The controller N100 processes and correlates the copy of the 743 packets sent from nodes N1, N4 and N7 to find segment-by-segment 744 delays and provide other hybrid OAM information related to packet 745 P1. 747 o The process continues for any other sampled packets. 749 3.4. Monitoring of SRv6 Paths 751 In the recent past, network operators demonstrated interest in 752 performing network OAM functions in a centralized manner. [RFC8403] 753 describes such a centralized OAM mechanism. Specifically, the 754 document describes a procedure that can be used to perform path 755 continuity check between any nodes within an SR domain from a 756 centralized monitoring system. However, the document focuses on SR 757 networks with MPLS data plane. This document describes how the 758 concept can be used to perform path monitoring in an SRv6 network 759 from a centralized controller. 761 In the reference topology in Figure 1, N100 uses an IGP protocol like 762 OSPF or ISIS to get the topology view within the IGP domain. N100 763 can also use BGP-LS to get the complete view of an inter-domain 764 topology. The controller leverages the visibility of the topology to 765 monitor the paths between the various endpoints. 767 The controller N100 advertises an END SID [RFC8986] 768 2001:db8:B:100:1::. To monitor any arbitrary SRv6 paths, the 769 controller can create a loopback probe that originates and terminates 770 on Node N100. To distinguish between a failure in the monitored path 771 and loss of connectivity between the controller and the network, Node 772 N100 runs a suitable mechanism to monitor its connectivity to the 773 monitored network. 775 The loopback probes are exemplified using an example where controller 776 N100 needs to verify a segment list <2001:db8:B:2:C31::, 777 2001:db8:B:4:C52::>: 779 o N100 generates an OAM packet (2001:db8:A:100::, 780 2001:db8:B:2:C31::)(2001:db8:B:100:1::, 2001:db8:B:4:C52::, 781 2001:db8:B:2:C31::, SL=2)(OAM Payload). The controller routes the 782 probe packet towards the first segment, which is 783 2001:db8:B:2:C31::. 785 o Node N2 executes the END.X behavior (2001:db8:B:2:C31::) and 786 forwards the packet (2001:db8:A:100::, 787 2001:db8:B:4:C52::)(2001:db8:B:100:1::, 2001:db8:B:4:C52::, 788 2001:db8:B:2:C31::, SL=1)(OAM Payload) on link3 to N3. 790 o Node N3, which is a classic IPv6 node, performs the standard IPv6 791 processing. Specifically, it forwards the packet based on the DA 792 2001:db8:B:4:C52:: in the IPv6 header. 794 o Node N4 executes the END.X behavior (2001:db8:B:4:C52::) and 795 forwards the packet (2001:db8:A:100::, 796 2001:db8:B:100:1::)(2001:db8:B:100:1::, 2001:db8:B:4:C52::, 797 2001:db8:B:2:C31::, SL=0)(OAM Payload) on link10 to N5. 799 o Node N5, which is a classic IPv6 node, performs the standard IPv6 800 processing. Specifically, it forwards the packet based on the DA 801 2001:db8:B:100:1:: in the IPv6 header. 803 o Node N100 executes the standard SRv6 END behavior. It 804 decapsulates the header and consume the probe for OAM processing. 805 The information in the OAM payload is used to detect any missing 806 probes, round trip delay, etc. 808 The OAM payload type or the information carried in the OAM probe is a 809 local implementation decision at the controller and is outside the 810 scope of this document. 812 4. Implementation Status 814 This section is to be removed prior to publishing as an RFC. 816 See [I-D.matsushima-spring-srv6-deployment-status] for updated 817 deployment and interoperability reports. 819 5. Security Considerations 821 This document does not define any new protocol extensions and relies 822 on existing procedures defined for ICMPv6. 824 [RFC8754] defines the notion of an SR domain and use of SRH within 825 the SR domain. The use of OAM procedures described in this document 826 is restricted to an SR domain. For example, similar to the SID 827 manipulation, O-flag manipulation is not considered as a threat 828 within the SR domain. Procedures for securing an SR domain are 829 defined the section 5.1 and section 7 of [RFC8754]. 831 As noted in section 7.1 of [RFC8754], compromised nodes within the SR 832 domain may mount attacks. The O-flag may be set by an attacking node 833 attempting a denial-of-service attack on the OAM process at the 834 segment endpoint node. An implementation correctly implementing the 835 rate limiting in section 2.1.1 is not susceptible to that denial-of- 836 service attack. Additionally, SRH Flags are protected by the HMAC 837 TLV, as described in Section 2.1.2.1 of [RFC8754]. 839 This document does not impose any additional security challenges to 840 be considered beyond security threats described in [RFC4884], 841 [RFC4443], [RFC0792], and [RFC8754]. 843 6. IANA Considerations 845 This document requests that IANA allocate the following registration 846 in the "Segment Routing Header Flags" sub-registry for the "Internet 847 Protocol Version 6 (IPv6) Parameters" registry maintained by IANA: 849 +-------+------------------------------+---------------+ 850 | Bit | Description | Reference | 851 +=======+==============================+===============+ 852 | 2 | O-flag | This document | 853 +-------+------------------------------+---------------+ 855 7. Acknowledgements 857 The authors would like to thank Joel M. Halpern, Greg Mirsky, Bob 858 Hinden, Loa Andersson, Gaurav Naik, Ketan Talaulikar and Haoyu Song 859 for their review comments. 861 8. Contributors 863 The following people have contributed to this document: 865 Robert Raszuk 866 Bloomberg LP 867 Email: robert@raszuk.net 869 John Leddy 870 Individual 871 Email: john@leddy.net 873 Gaurav Dawra 874 LinkedIn 875 Email: gdawra.ietf@gmail.com 876 Bart Peirens 877 Proximus 878 Email: bart.peirens@proximus.com 880 Nagendra Kumar 881 Cisco Systems, Inc. 882 Email: naikumar@cisco.com 884 Carlos Pignataro 885 Cisco Systems, Inc. 886 Email: cpignata@cisco.com 888 Rakesh Gandhi 889 Cisco Systems, Inc. 890 Canada 891 Email: rgandhi@cisco.com 893 Frank Brockners 894 Cisco Systems, Inc. 895 Germany 896 Email: fbrockne@cisco.com 898 Darren Dukes 899 Cisco Systems, Inc. 900 Email: ddukes@cisco.com 902 Cheng Li 903 Huawei 904 Email: chengli13@huawei.com 906 Faisal Iqbal 907 Individual 908 Email: faisal.ietf@gmail.com 910 9. References 911 9.1. Normative References 913 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 914 Requirement Levels", BCP 14, RFC 2119, 915 DOI 10.17487/RFC2119, March 1997, 916 . 918 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 919 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 920 May 2017, . 922 [RFC8754] Filsfils, C., Ed., Dukes, D., Ed., Previdi, S., Leddy, J., 923 Matsushima, S., and D. Voyer, "IPv6 Segment Routing Header 924 (SRH)", RFC 8754, DOI 10.17487/RFC8754, March 2020, 925 . 927 [RFC8986] Filsfils, C., Ed., Camarillo, P., Ed., Leddy, J., Voyer, 928 D., Matsushima, S., and Z. Li, "Segment Routing over IPv6 929 (SRv6) Network Programming", RFC 8986, 930 DOI 10.17487/RFC8986, February 2021, 931 . 933 9.2. Informative References 935 [I-D.gandhi-spring-stamp-srpm] 936 Gandhi, R., Filsfils, C., Voyer, D., Chen, M., Janssens, 937 B., and R. Foote, "Performance Measurement Using Simple 938 TWAMP (STAMP) for Segment Routing Networks", draft-gandhi- 939 spring-stamp-srpm-06 (work in progress), April 2021. 941 [I-D.ietf-ippm-ioam-data] 942 Brockners, F., Bhandari, S., and T. Mizrahi, "Data Fields 943 for In-situ OAM", draft-ietf-ippm-ioam-data-11 (work in 944 progress), November 2020. 946 [I-D.matsushima-spring-srv6-deployment-status] 947 Matsushima, S., Filsfils, C., Ali, Z., Li, Z., and K. 948 Rajaraman, "SRv6 Implementation and Deployment Status", 949 draft-matsushima-spring-srv6-deployment-status-11 (work in 950 progress), February 2021. 952 [RFC0792] Postel, J., "Internet Control Message Protocol", STD 5, 953 RFC 792, DOI 10.17487/RFC0792, September 1981, 954 . 956 [RFC2328] Moy, J., "OSPF Version 2", STD 54, RFC 2328, 957 DOI 10.17487/RFC2328, April 1998, 958 . 960 [RFC4443] Conta, A., Deering, S., and M. Gupta, Ed., "Internet 961 Control Message Protocol (ICMPv6) for the Internet 962 Protocol Version 6 (IPv6) Specification", STD 89, 963 RFC 4443, DOI 10.17487/RFC4443, March 2006, 964 . 966 [RFC4884] Bonica, R., Gan, D., Tappan, D., and C. Pignataro, 967 "Extended ICMP to Support Multi-Part Messages", RFC 4884, 968 DOI 10.17487/RFC4884, April 2007, 969 . 971 [RFC5476] Claise, B., Ed., Johnson, A., and J. Quittek, "Packet 972 Sampling (PSAMP) Protocol Specifications", RFC 5476, 973 DOI 10.17487/RFC5476, March 2009, 974 . 976 [RFC5837] Atlas, A., Ed., Bonica, R., Ed., Pignataro, C., Ed., Shen, 977 N., and JR. Rivers, "Extending ICMP for Interface and 978 Next-Hop Identification", RFC 5837, DOI 10.17487/RFC5837, 979 April 2010, . 981 [RFC5880] Katz, D. and D. Ward, "Bidirectional Forwarding Detection 982 (BFD)", RFC 5880, DOI 10.17487/RFC5880, June 2010, 983 . 985 [RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, 986 "Specification of the IP Flow Information Export (IPFIX) 987 Protocol for the Exchange of Flow Information", STD 77, 988 RFC 7011, DOI 10.17487/RFC7011, September 2013, 989 . 991 [RFC7012] Claise, B., Ed. and B. Trammell, Ed., "Information Model 992 for IP Flow Information Export (IPFIX)", RFC 7012, 993 DOI 10.17487/RFC7012, September 2013, 994 . 996 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 997 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 998 May 2016, . 1000 [RFC7880] Pignataro, C., Ward, D., Akiya, N., Bhatia, M., and S. 1001 Pallagatti, "Seamless Bidirectional Forwarding Detection 1002 (S-BFD)", RFC 7880, DOI 10.17487/RFC7880, July 2016, 1003 . 1005 [RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L., 1006 Decraene, B., Litkowski, S., and R. Shakir, "Segment 1007 Routing Architecture", RFC 8402, DOI 10.17487/RFC8402, 1008 July 2018, . 1010 [RFC8403] Geib, R., Ed., Filsfils, C., Pignataro, C., Ed., and N. 1011 Kumar, "A Scalable and Topology-Aware MPLS Data-Plane 1012 Monitoring System", RFC 8403, DOI 10.17487/RFC8403, July 1013 2018, . 1015 [RFC8571] Ginsberg, L., Ed., Previdi, S., Wu, Q., Tantsura, J., and 1016 C. Filsfils, "BGP - Link State (BGP-LS) Advertisement of 1017 IGP Traffic Engineering Performance Metric Extensions", 1018 RFC 8571, DOI 10.17487/RFC8571, March 2019, 1019 . 1021 Authors' Addresses 1023 Zafar Ali 1024 Cisco Systems 1026 Email: zali@cisco.com 1028 Clarence Filsfils 1029 Cisco Systems 1031 Email: cfilsfil@cisco.com 1033 Satoru Matsushima 1034 Softbank 1036 Email: satoru.matsushima@g.softbank.co.jp 1038 Daniel Voyer 1039 Bell Canada 1041 Email: daniel.voyer@bell.ca 1043 Mach Chen 1044 Huawei 1046 Email: mach.chen@huawei.com