idnits 2.17.1 draft-ietf-spring-oam-usecase-07.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 1, 2017) is 2484 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 spring R. Geib, Ed. 3 Internet-Draft Deutsche Telekom 4 Intended status: Informational C. Filsfils 5 Expires: January 2, 2018 C. Pignataro, Ed. 6 N. Kumar 7 Cisco Systems, Inc. 8 July 1, 2017 10 A Scalable and Topology-Aware MPLS Dataplane Monitoring System 11 draft-ietf-spring-oam-usecase-07 13 Abstract 15 This document describes features of a path monitoring system and 16 related use cases. Segment based routing enables a scalable and 17 simple method to monitor data plane liveliness of the complete set of 18 paths belonging to a single domain. The MPLS monitoring system adds 19 features to the traditional MPLS Ping and LSP Trace, in a very 20 complementary way. MPLS topology awareness reduces management and 21 control plane involvement of OAM measurements while enabling new OAM 22 features. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at http://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on January 2, 2018. 41 Copyright Notice 43 Copyright (c) 2017 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (http://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 59 2. Terminology and Acronyms . . . . . . . . . . . . . . . . . . 5 60 2.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 5 61 2.2. Acronyms . . . . . . . . . . . . . . . . . . . . . . . . 5 62 3. An MPLS Topology-Aware Path Monitoring System . . . . . . . . 6 63 4. SR-based Path Monitoring Use Case Illustration . . . . . . . 7 64 4.1. Use Case 1 - LSP Dataplane Monitoring . . . . . . . . . . 7 65 4.2. Use Case 2 - Monitoring a Remote Bundle . . . . . . . . . 10 66 4.3. Use Case 3 - Fault Localization . . . . . . . . . . . . . 11 67 5. Path Trace and Failure Notification . . . . . . . . . . . . . 11 68 6. Applying SR to Monitoring non-SR based LSPs (LDP and possibly 69 RSVP-TE) . . . . . . . . . . . . . . . . . . . . . . . . . . 12 70 7. PMS Monitoring of Different Segment ID Types . . . . . . . . 13 71 8. Connectivity Verification Using PMS . . . . . . . . . . . . . 13 72 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 73 10. Security Considerations . . . . . . . . . . . . . . . . . . . 14 74 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 15 75 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 16 76 12.1. Normative References . . . . . . . . . . . . . . . . . . 16 77 12.2. Informative References . . . . . . . . . . . . . . . . . 17 78 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 17 80 1. Introduction 82 Network operator need to be able to monitor the forwarding paths used 83 to transport user packets. Monitoring packets are expected to be 84 forwarded in dataplane in a similar way as user packets. Segment 85 Routing enables forwarding of packets along pre-defined paths and 86 segments and thus a Segment Routed monitoring packet can stay in 87 dataplane while passing along one or more segments to be monitored. 89 This document describes a system using MPLS data plane path 90 monitoring capabilities. The use cases introduced here are limited 91 to a single Interior Gateway Protocol (IGP) MPLS domain. 93 The system applies to monitoring of non Segment Routing Label 94 Switched Paths (LSP's) like LDP as well as to monitoring of Segment 95 Routed LSP's (section 7 offers some more information). As compared 96 to non Segment Routing approaches, Segment Routing is expected to 97 simplify such a monitoring system by enabling MPLS topology detection 98 based on IGP signaled segments. The MPLS topology should be detected 99 and correlated with the IGP topology, which is too detected by IGP 100 signaling. Thus a centralized and MPLS topology aware monitoring 101 unit can be realized in a Segment Routed domain. This topology 102 awareness can be used for Operation, Administration, and Maintenance 103 (OAM) purposes as described by this document. 105 Benefits offered by the system: 107 o The system described here allows to set up an SR domain wide 108 centralized connectivity validation. Many operators operators of 109 large networks regard centralized monitoring system as useful.. 111 o The MPLS Ping (or continuity check) packets never leave the MPLS 112 user data plane. 114 o SR allows to transport MPLS path trace or connectivity validation 115 packets for every Label Switched Path to all nodes of an SR 116 domain. This use case doesn't describe new path trace features. 117 The system described here allows to set up an SR domain wide 118 centralized connectivity validation, which is useful in large 119 network operator domains. 121 o The system sending the monitoring packet is also receiving it. 122 The payload of the monitoring packet may be chosen freely. This 123 allows sending probing packets which represent customer traffic, 124 possibly from multiple services (e.g., small Voice over IP packet, 125 larger HTTP packets) and embedding of useful monitoring data 126 (e.g., accurate time stamps since both sender and receiver have 127 the same clock, sequence numbers to ease the measurement...). 129 o Set up of a flexible MPLS monitoring system in terms of 130 deployment: from one single centralized one to a set of 131 distributed systems (e.g., on a per region or service base), and 132 in terms of redundancy from 1+1 to N+1. 134 In addition to monitoring paths, problem localization is required. 135 Topology awareness is an important feature of link state IGPs 136 deployed by operators of large networks. MPLS topology awareness 137 combined with IGP topology awareness enables a simple and scalable 138 data plane based monitoring mechanism. Faults can be localized: 140 o by capturing the Interior Gateway Protocol (IGP) topology and 141 analyzing IGP messages indicating changes of it. 143 o by correlation between different SR based monitoring probes. 145 o by setting up an MPLS traceroute packet for a path (or Segment) to 146 be tested and transporting it to a node to validate path 147 connectivity from that node on. 149 MPLS OAM offers flexible traceroute (connectivity verification) 150 features to detect and execute data paths of an MPLS domain. By 151 utilizing the Equal Cost Multipath (ECMP) related tool set offered, 152 e.g., by RFC 8029 [RFC8029], a SR based MPLS monitoring system can be 153 enabled to: 155 o detect how to route packets along different ECMP routed paths. 157 o construct Ping packets, which can be steered to paths whose 158 connectivity is to be checked, also if ECMP is present. 160 o limit the MPLS label stack of such a Ping packet checking 161 continuity of every single IGP-Segment to the maximum number of 3 162 labels. A smaller label stack may also be helpful, if any router 163 interprets a limited number of packet header bytes to determine an 164 ECMP path along which to route a packet. 166 Alternatively, any path may be executed by building suitable label 167 stacks. This allows path execution without ECMP awareness. 169 The MPLS Path Monitoring System may be any server residing at a 170 single interface of the domain to be monitored. The PMS doesn't need 171 to support the complete MPLS routing or control plane. It needs to 172 be capable to learn and maintain an accurate MPLS and IGP topology. 173 MPLS Ping and traceroute packets need to be set up and sent with the 174 correct segment stack. The PMS further must be able to receive and 175 decode returning Ping or Traceroute packets. Packets from a variety 176 of protocols can be used to check continuity. These include Internet 177 Control Message Protocol [RFC0792] [RFC4443] [RFC4884] [RFC4950], 178 Bidirectional Forwarding Detection (BFD) [RFC5884], Seamless 179 Bidirectional Forwarding Detection (S-BFD) [RFC7880] [RFC7881] (see 180 Section 3.4 of [RFC7882]), and MPLS LSP Ping [RFC8029]. They can 181 also have any other OAM format supported by the PMS. As long as the 182 packet used to check continuity returns back to the server while no 183 IGP change is detected, the monitored path can be considered as 184 validated. If monitoring requires pushing a large label stack, a 185 software based implementation is usually more flexible than an 186 hardware based one. Hence router label stack depth and label 187 composition limitations don't limit MPLS OAM choices. 189 [I-D.draft-ietf-mpls-spring-lsp-ping] discusses SR OAM applicability 190 and MPLS traceroute enhancements adding functionality to the use 191 cases described by this document. 193 2. Terminology and Acronyms 195 2.1. Terminology 197 Continuity Check 199 is defined in Section 2.2.7 of RFC 7276 [RFC7276]. 201 Connectivity Verification 203 is defined in Section 2.2.7 of RFC 7276 [RFC7276]. 205 MPLS topology 207 The MPLS topology of an MPLS domain is the complete set of MPLS- 208 and IP-address information and all routing and data plane 209 information required to address and utilize every MPLS path 210 within this domain from an MPLS Path Monitoring System attached 211 to this MPLS domain at an arbitrary access. This document 212 assumes availability of the MPLS topology (which can be detected 213 with available protocols and interfaces). None of the use cases 214 will describe how to set it up. 216 This document further adopts the terminology and framework described 217 in [I-D.ietf-spring-segment-routing]. 219 2.2. Acronyms 221 ECMP Equal-Cost Multi-Path 223 IGP Interior Gateway Protocol 225 LER Label Edge Router 227 LSP Label Switched Path 229 LSR Label Switching Router 231 OAM Operations, Administration, and Maintenance 233 PMS Path Monitoring System 235 RSVP-TE Resource ReserVation Protocol-Traffic Engineering 237 SID Segment Identifier 239 SR Segment Routing 240 SRGB Segment Routing Global Block 242 3. An MPLS Topology-Aware Path Monitoring System 244 Any node at least listening to the IGP of an SR domain is MPLS 245 topology aware (the node knows all related IP addresses, SR SIDs and 246 MPLS labels). An MPLS PMS which is able to learn the IGP LSDB 247 (including the SID's) is able to execute arbitrary chains of label 248 switched paths. To monitor an MPLS SR domain, a PMS needs to set up 249 a topology data base of the MPLS SR domain to be monitored. It may 250 be used to send ping type packets to only check continuity along such 251 a path chain based on the topology information only. In addition, 252 the PMS can be used to trace MPLS Label Switched Path and thus verify 253 their connectivity and correspondence between control and data plane, 254 respectively. The PMS can direct suitable MPLS traceroute packets to 255 any node along a path segment. 257 Let us describe how the PMS constructs a labels stack to transport a 258 packet to LER i, monitor its path to LER j and then receive the 259 packet back. 261 The PMS may do so by sending packets carrying the following MPLS 262 label stack information: 264 o Top Label: a path from PMS to LER i, which is expressed as Node 265 SID of LER i. 267 o Next Label: the path that needs to be monitored from LER i to LER 268 j. If this path is a single physical interface (or a bundle of 269 connected interfaces), it can be expressed by the related 270 Adjacency-SID. If the shortest path from LER i to LER j is 271 supposed to be monitored, the Node-SID (LER j) can be used. 272 Another option is to insert a list of segments expressing the 273 desired path (hop by hop as an extreme case). If LER i pushes a 274 stack of Labels based on a SR policy decision and this stack of 275 LSPs is to be monitored, the PMS needs an interface to collect the 276 information enabling it to address this SR created path. 278 o Next Label or address: the path back to the PMS. Likely, no 279 further segment/label is required here. Indeed, once the packet 280 reaches LER j, the 'steering' part of the solution is done and the 281 probe just needs to return to the PMS. This is best achieved by 282 popping the MPLS stack and revealing a probe packet with PMS as 283 destination address (note that in this case, the source and 284 destination addresses could be the same). If an IP address is 285 applied, no SID/label has to be assigned to the PMS (if it is a 286 host/server residing in an IP subnet outside the MPLS domain). 288 The PMS should be physically connected to a router which is part of 289 the SR domain. It must be able to send and receive MPLS packets via 290 this interface. As mentioned above, routing protocol support isn't 291 required and the PMS itself doesn't have to be involved in IGP or 292 MPLS routing. A static route will do. Further options, like 293 deployment of a PMS connecting to the MPLS domain by a tunnel only 294 require more thought, as this implies security aspects. MPLS so far 295 separates networks securely by avoiding tunnel access to MPLS 296 domains. 298 4. SR-based Path Monitoring Use Case Illustration 300 4.1. Use Case 1 - LSP Dataplane Monitoring 302 +---+ +----+ +-----+ 303 |PMS| |LSR1|-----|LER i| 304 +---+ +----+ +-----+ 305 | / \ / 306 | / \__/ 307 +-----+/ /| 308 |LER m| / | 309 +-----+\ / \ 310 \ / \ 311 \+----+ +-----+ 312 |LSR2|-----|LER j| 313 +----+ +-----+ 315 Example of a PMS based LSP dataplane monitoring 317 Figure 1 319 For the sake of simplicity, let's assume that all the nodes are 320 configured with the same SRGB [I-D.ietf-spring-segment-routing]. 322 Let's assign the following Node SIDs to the nodes of the figure: PMS 323 = 10, LER i = 20, LER j = 30. 325 The aim is to set up a continuity check of the path between LER i and 326 LER j. As has been said, the monitoring packets are to be sent and 327 received by the PMS. Let's assume the design aim is to be able to 328 work with the smallest possible SR label stack. In the given 329 topology, a fairly simple option is to perform an MPLS path trace, as 330 specified by RFC 8029 [RFC8029] (using the Downstream (Detailed) 331 Mapping information resulting from a path trace). The starting point 332 for the path trace is LER i and the PMS sends the MPLS path trace 333 packet to LER i. The MPLS echo reply of LER i should be sent to the 334 PMS. As a result, IP destination address choices are detected, which 335 are then used to target any one of the ECMP routed paths between LER 336 i and LER j by the MPLS ping packets to later check path continuity. 337 The Label stack of these ping packets doesn't need to consist of more 338 than 3 labels. Finally, the PMS sets up and sends packets to monitor 339 connectivity of the ECMP routed paths. The PMS does this by creating 340 a measurement packet with the following label stack (top to bottom): 341 20 - 30 - 10. The ping packets reliably use the monitored path, if 342 the IP-address information which has been detected by the MPLS trace 343 route is used as the IP destination address (note that this IP 344 address isn't used or required for any IP routing). 346 LER m forwards the packet received from the PMS to LSR1. Assuming 347 Pen-ultimate Hop Popping to be deployed, LSR1 pops the top label and 348 forwards the packet to LER i. There the top label has a value 30 and 349 LER i forwards it to LER j. This will be done transmitting the 350 packet via LSR1 or LSR2. The LSR will again pop the top label. LER 351 j will forward the packet now carrying the top label 10 to the PMS 352 (and it will pass a LSR and LER m). 354 A few observations on the example given in figure 1: 356 o The path PMS to LER i must be available (i.e., a continuity check 357 only along the path to LER i must succeed). If desired, an MPLS 358 trace route may be used to exactly detect the data plane path 359 taken for this MPLS Segment. It is usually sufficient to just 360 apply any of the existing Shortest Path routed paths. 362 o If ECMP is deployed, separate continuity checks monitoring all 363 possible paths which a packet may use between LER i and LER j may 364 be desired. This can be done by applying an MPLS trace route 365 between LER i and LER j. Another option is to use SR routing, but 366 this will likely require additional label information within the 367 label stack of the ping packet. Further, if multiple links are 368 deployed between two nodes, SR methods to address each individual 369 path require an Adj-SID to be assigned to each single interface. 370 This method is based on control plane information - a connectivity 371 verification based on MPLS traceroute seems to be a fairly good 372 option to deal with ECMP and validation of control and data plane 373 correlation. 375 o The path LER j to PMS must be available (i.e., a continuity check 376 only along the path from LER j to PMS must succeed). If desired, 377 an MPLS trace route may be used to exactly detect the data plane 378 path taken for this MPLS Segment. It is usually sufficient to 379 just apply any of the existing Shortest Path routed paths. 381 Once the MPLS paths (Node-SIDs) and the required information to deal 382 with ECMP have been detected, the path continuity between LER i and 383 LER j can be monitored by the PMS. Path continuity monitoring by 384 ping packets does not require RFC 8029 [RFC8029] MPLS OAM 385 functionality. All monitoring packets stay on dataplane, hence path 386 continuity monitoring does not require control plane interaction in 387 any LER or LSR of the domain. To ensure consistent interpretation of 388 the results, the PMS should be aware of any changes in IGP or MPLS 389 topology or ECMP routing. While the description given here 390 pronouncing path connectivity checking as a simple basic application, 391 others like checking continuity of underlying physical infrastructure 392 or delay measurements may be desired. In both cases, a change in 393 ECMP routing which is not caused by an IGP or MPLS topology change 394 may not be desirable. A PMS therefore should also periodically 395 verify connectivity of the SR paths which are monitored for 396 continuity. 398 Determining a path to be executed prior to a measurement may also be 399 done by setting up a label stack including all Node-SIDs along that 400 path (if LSR1 has Node SID 40 in the example and it should be passed 401 between LER i and LER j, the label stack is 20 - 40 - 30 - 10). The 402 advantage of this method is, that it does not involve RFC 8029 403 [RFC8029] connectivity verification and, if there's only one physical 404 connection between all nodes, the approach is independent of ECMP 405 functionalities. The method still is able to monitor all link 406 combinations of all paths of an MPLS domain. If correct forwarding 407 along the desired paths has to be checked, or multiple physical 408 connections exist between any two nodes, all Adj-SIDs along that path 409 should be part of the label stack. 411 In theory at least, a single PMS is able to monitor data plane 412 availability of all LSPs in the domain. The PMS may be a router, but 413 could also be dedicated monitoring system. If measurement system 414 reliability is an issue, more than a single PMS may be connected to 415 the MPLS domain. 417 Monitoring an MPLS domain by a PMS based on SR offers the option of 418 monitoring complete MPLS domains with limited effort and a unique 419 possibility to scale a flexible monitoring solution as required by 420 the operator (the number of PMS deployed is independent of the 421 locations of the origin and destination of the monitored paths). The 422 PMS can be enabled to send MPLS OAM packets with the label stacks and 423 address information identical to those of the monitoring packets to 424 any node of the MPLS domain. The routers of the monitored domain 425 should support MPLS LSP Ping RFC 8029 [RFC8029]. They may also 426 incorporate the additional enhancements defined in 427 [I-D.draft-ietf-mpls-spring-lsp-ping] to incorporate further MPLS 428 trace route features. ICMP Ping based continuity checks don't 429 require router control plane activity. Prior to monitoring a path, 430 MPLS OAM may be used to detect ECMP dependent forwarding of a packet. 431 A PMS may be designed to learn the IP address information required to 432 execute a particular ECMP routed path and interfaces along that path. 433 This allows to monitor these paths with label stacks reduced to a 434 limited number of Node-SIDs resulting from SPF routing. The PMS does 435 not require access to LSR / LER management- or data-plane information 436 to do so. 438 4.2. Use Case 2 - Monitoring a Remote Bundle 440 +---+ _ +--+ +-------+ 441 | | { } | |---991---L1---662---| | 442 |PMS|--{ }-|R1|---992---L2---663---|R2 (72)| 443 | | {_} | |---993---L3---664---| | 444 +---+ +--+ +-------+ 446 SR based probing of all the links of a remote bundle 448 Figure 2 450 In the figure, R1 addresses Link "x" Lx by the Adjacency SID 99x, 451 while R2 addresses Link Lx by the Adjacency SID 66(x+1). 453 In the above figure, the PMS needs to assess the dataplane 454 availability of all the links within a remote bundle connected to 455 routers R1 and R2. 457 The monitoring system retrieves the SID/Label information from the 458 IGP LSDB and appends the following segment list/label stack: {72, 459 662, 992, 664} on its IP probe (whose source and destination 460 addresses are the address of the PMS). 462 PMS sends the probe to its connected router. The MPLS/SR domain then 463 forwards the probe to R2 (72 is the Node SID of R2). R2 forwards the 464 probe to R1 over link L1 (Adjacency SID 662). R1 forwards the probe 465 to R2 over link L2 (Adjacency SID 992). R2 forwards the probe to R1 466 over link L3 (Adjacency SID 664). R1 then forwards the IP probe to 467 PMS as per classic IP forwarding. 469 As has been mentioned in section 5.1, the PMS must be able monitor 470 continuity of the path PMS to R2 (Node-SID 72) as well as continuity 471 from R1 to the PMS. If both are given and packets are lost, 472 forwarding on one of the three interfaces connecting R1 to R2 must be 473 disturbed. 475 4.3. Use Case 3 - Fault Localization 477 In the previous example, a uni-directional fault on the middle link 478 in direction of R2 to R1 would be localized by sending the following 479 two probes with respective segment lists: 481 o 72, 662, 992, 664 483 o 72, 663, 992, 664 485 The first probe would succeed while the second would fail. 486 Correlation of the measurements reveals that the only difference is 487 using the Adjacency SID 663 of the middle link from R2 to R1 in the 488 non successful measurement. Assuming the second probe has been 489 routed correctly, the problem is that for some (possibly unknown) 490 reason SR packets to be forwarded from R2 via the interface 491 identified by Adjacency SID 663 are lost. 493 The example above only illustrates a method to localize a fault by 494 correlated continuity checks. Any operational deployment requires a 495 well designed engineering to allow for the desired non ambiguous 496 diagnosis on the monitored section of the SR network. 'Section' here 497 could be a path, a single physical interface, the set of all links of 498 a bundle or an adjacency of two nodes, just to name a few. 500 5. Path Trace and Failure Notification 502 Sometimes forwarding along a single path indeed doesn't work, while 503 the control plane information is healthy. Such a situation may occur 504 after maintenance work within a domain. An operator may perform on 505 demand-tests, but execution of automated PMS path trace checks may be 506 set up too (scope may be limited to a subset of important end-to-end 507 paths crossing the router or network section after completion of the 508 maintenance work there). Upon detection of a path which can't be 509 used, the operator needs to be notified. A check ensuring that re- 510 routing event is differed from a path facing whose forwarding 511 behavior doesn't correspond to the control plane information is 512 necessary (but out of scope of this document). 514 Adding an automated problem solution to the PMS features only makes 515 sense, if the root cause of the symptom appears often, can be assumed 516 to be non-ambiguous by its symptoms, can be solved by a pre- 517 determined chain of commands and the automated PMS reaction not doing 518 any collateral damage. A closer analysis is out of scope of this 519 document. 521 The PMS is expected to check control plane liveliness after a path 522 repair effort was executed. It doesn't matter whether the path 523 repair was triggered manually or by an automated system. 525 6. Applying SR to Monitoring non-SR based LSPs (LDP and possibly RSVP- 526 TE) 528 The MPLS path monitoring system described by this document can be 529 realized with pre-Segment Routing (SR) based technology. Making such 530 a pre-SR MPLS monitoring system aware of a domain's complete MPLS 531 topology requires, e.g., management plane access to the routers of 532 the domain to be monitored or set up of a dedicated tLDP tunnel per 533 router to set up an LDP adjacency. To avoid the use of stale MPLS 534 label information, the IGP must be monitored and MPLS topology must 535 be timely aligned with IGP topology. Enhancing IGPs to exchange of 536 MPLS topology information as done by SR significantly simplifies and 537 stabilizes such an MPLS path monitoring system. 539 A SR based PMS connected to a MPLS domain consisting of LER and LSR 540 supporting SR and LDP or RSVP-TE in parallel in all nodes may use SR 541 paths to transmit packets to and from start and end points of non-SR 542 based LSP paths to be monitored. In the example given in figure 1, 543 the label stack top to bottom may be as follows, when sent by the 544 PMS: 546 o Top: SR based Node-SID of LER i at LER m. 548 o Next: LDP or RSVP-TE label identifying the path or tunnel, 549 respectively from LER i to LER j (at LER i). 551 o Bottom: SR based Node-SID identifying the path to the PMS at LER j 553 While the mixed operation shown here still requires the PMS to be 554 aware of the LER LDP-MPLS topology, the PMS may learn the SR MPLS 555 topology by IGP and use this information. 557 An implementation report on a PMS operating in an LDP domain is given 558 in [I-D.leipnitz-spring-pms-implementation-report]. In addition, 559 this report compares delays measured with a single PMS to the results 560 measured by three standard conformant Measurement Agents ([RFC6808] 561 connected to an MPLS domain at three different sites). The delay 562 measurements of the PMS and the IPPM Measurement Agents were compared 563 based on a statistical test in [RFC6576]. The Anderson Darling 564 k-sample test showed that the PMS round-trip delay measurements are 565 equal to those captured by an IPPM conformant IP measurement system 566 for 64 Byte measurement packets with 95% confidence. 568 The authors are not aware of similar deployment for RSVP-TE. 569 Identification of tunnel entry- and transit-nodes may add complexity. 570 They are not within scope of this document. 572 7. PMS Monitoring of Different Segment ID Types 574 MPLS SR topology awareness should allow the PMS to monitor liveliness 575 of SIDs related to interfaces within the SR and IGP domain, 576 respectively. Tracing a path where an SR capable node assigns an 577 Adj-SID for a non-SR-capable node may fail. This and other backward 578 compatibility with non Segment Routing devices are discussed by 579 [I-D.draft-ietf-mpls-spring-lsp-ping]. 581 To match control plane information with data plane information for 582 all relevant types of Segment IDs, 583 [I-D.draft-ietf-mpls-spring-lsp-ping]enhances MPLS OAM functions 584 defined by RFC 8029 [RFC8029]. 586 8. Connectivity Verification Using PMS 588 While the PMS based use cases explained in Section 5 are sufficient 589 to provide continuity check between LER i and LER j, it may not help 590 perform connectivity verification. 592 +---+ 593 |PMS| 594 +---+ 595 | 596 | 597 +----+ +----+ +-----+ 598 |LSRa|-----|LSR1|-----|LER i| 599 +----+ +----+ +-----+ 600 | / \ / 601 | / \__/ 602 +-----+/ /| 603 |LER m| / | 604 +-----+\ / \ 605 \ / \ 606 \+----+ +-----+ 607 |LSR2| |LER j| 608 +----+ +-----+ 610 Connectivity verification with a PMS 612 Figure 3 614 Let's assign the following Node SIDs to the nodes of the figure: PMS 615 = 10, LER i = 20, LER j = 30, LER m = 40. PMS is intended to 616 validate the path between LER m and LER j. In order to validate this 617 path, PMS will send the probe packet with label stack of (top to 618 bottom): {40} {30} {10}. Imagine any of the below forwarding entry 619 misprogrammed situation: 621 o LSRa receiving any packet with top label 40 will POP and forwards 622 to LSR1 instead of LER m. 624 o LSR1 receiving any packet with top label 30 will pop and forward 625 to LER i instead of LER j. 627 In any of these above situation, the probe packet will be delivered 628 back to PMS leading to a falsified path liveliness indication by the 629 PMS. 631 Connectivity Verification functions helps us to verify if the probe 632 is taking the expected path. For example, PMS can intermittently 633 send the probe packet with label stack of (top to bottom): 634 {40;ttl=255} {30;ttl=1} {10;ttl=255}. The probe packet may carry 635 information about LER m which could be carried in Target FEC Stack in 636 case of MPLS Echo Request or Discriminator in case of Seamless BFD. 637 When LER m receives the packet, it will punt due to TTL expiry and 638 sends a positive response. In the above mentioned misprogramming 639 situation, LSRa will forwards to LSR1 which will send a negative 640 response to PMS as the information in probe does not match the local 641 node. PMS can do the same for bottom label as well. This will help 642 perform connectivity verification and ensure that the path between 643 LER m and LER j is working as expected. 645 9. IANA Considerations 647 This memo includes no request to IANA. 649 10. Security Considerations 651 The PMS builds packets with intent of performing OAM tasks. It uses 652 address information based on topology information, rather than a 653 protocol. 655 The PMS allows to insert traffic into non-SR domains. This may be 656 required in the case of an LDP domain attached to the SR domain, but 657 it can be used to compromise security in the case of external IP 658 domains and MPLS based VPNs. 660 To avoid a PMS to insert traffic into an MPLS VPN domain, one or more 661 sets of label ranges may be reserved for service labels within an SR 662 domain. The PMS should be configured to reject usage of these 663 service label values. In the same way, misuse of IP destination 664 addresses is blocked if only IP-destination address values conforming 665 to RFC 8029 [RFC8029] are settable by the PMS. 667 To limit potential misuse, access to a PMS needs to be authorized and 668 should be logged. OAM supported by a PMS requires skilled personal 669 and hence only experts requiring PMS access should be allowed to 670 access such a system. It is recommended to directly attach a PMS to 671 an SR domain. Connecting a PMS to an SR domain is technically 672 possible, but adds further security issues. A tunnel based access of 673 a PMS to an SR domain is not recommended. 675 Use of stale MPLS or IGP routing information could cause a PMS 676 monitoring packet to leave the domain where it originated. PMS 677 monitoring packets should not be sent using stale MPLS or IGP routing 678 information. As it is necessary to know that the information is 679 stale is order to follow the instruction, as is the case with for 680 example convergence events that may be ongoing at the time of 681 diagnostic measurement. 683 Traffic insertion by a PMS may be unintended, especially if the IGP 684 or MPLS topology stored locally are in stale state. As soon as the 685 PMS has an indication, that its IGP or MPLS topology are stale, it 686 should stop operations involving network sections whose topology may 687 not be accurate. Note however that it is a task of an OAM system to 688 discover and locate network sections having where forwarding behavior 689 is not matching control plane state. As soon as a PMS or an operator 690 of a PMS has the impression, that the PMS topology information is 691 stale, measures need to be taken to refresh the topology information. 692 These measures should be part of the PMS design. Matching forwarding 693 and control plane state by periodically automated execution of RFC 694 8029 [RFC8029] mechanisms may be such a feature. Whenever network 695 maintenance tasks are performed by operators, the PMS topology 696 discovery should be started asynchronously after network maintenance 697 has been finished. 699 A PMS loosing network connectivity or crashing must remove all IGP 700 and MPLS topology information prior to restarting operation. 702 A PMS may operate routine measurements. If these are automated, care 703 must be taken to avoid unintended traffic insertion. On the other 704 hand, large scale operation based on operator configuration itself is 705 a source of unintended misconfigurations and should be avoided. 707 11. Acknowledgements 709 The authors would like to thank Nobo Akiya for his contribution. 710 Raik Leipnitz kindly provided an editorial review. The authors would 711 also like to thank Faisal Iqbal for an insightful review and a useful 712 set of comments and suggestions. Finally, Bruno Decraene's shepherd 713 review led to a clarified document. 715 12. References 717 12.1. Normative References 719 [I-D.ietf-spring-segment-routing] 720 IETF, "Segment Routing Architecture", IETF, 721 https://datatracker.ietf.org/doc/draft-ietf-spring- 722 segment-routing/, 2016. 724 [RFC0792] Postel, J., "Internet Control Message Protocol", STD 5, 725 RFC 792, DOI 10.17487/RFC0792, September 1981, 726 . 728 [RFC4443] Conta, A., Deering, S., and M. Gupta, Ed., "Internet 729 Control Message Protocol (ICMPv6) for the Internet 730 Protocol Version 6 (IPv6) Specification", RFC 4443, 731 DOI 10.17487/RFC4443, March 2006, 732 . 734 [RFC4884] Bonica, R., Gan, D., Tappan, D., and C. Pignataro, 735 "Extended ICMP to Support Multi-Part Messages", RFC 4884, 736 DOI 10.17487/RFC4884, April 2007, 737 . 739 [RFC4950] Bonica, R., Gan, D., Tappan, D., and C. Pignataro, "ICMP 740 Extensions for Multiprotocol Label Switching", RFC 4950, 741 DOI 10.17487/RFC4950, August 2007, 742 . 744 [RFC5884] Aggarwal, R., Kompella, K., Nadeau, T., and G. Swallow, 745 "Bidirectional Forwarding Detection (BFD) for MPLS Label 746 Switched Paths (LSPs)", RFC 5884, DOI 10.17487/RFC5884, 747 June 2010, . 749 [RFC7276] Mizrahi, T., Sprecher, N., Bellagamba, E., and Y. 750 Weingarten, "An Overview of Operations, Administration, 751 and Maintenance (OAM) Tools", RFC 7276, 752 DOI 10.17487/RFC7276, June 2014, 753 . 755 [RFC7880] Pignataro, C., Ward, D., Akiya, N., Bhatia, M., and S. 756 Pallagatti, "Seamless Bidirectional Forwarding Detection 757 (S-BFD)", RFC 7880, DOI 10.17487/RFC7880, July 2016, 758 . 760 [RFC7881] Pignataro, C., Ward, D., and N. Akiya, "Seamless 761 Bidirectional Forwarding Detection (S-BFD) for IPv4, IPv6, 762 and MPLS", RFC 7881, DOI 10.17487/RFC7881, July 2016, 763 . 765 [RFC8029] Kompella, K., Swallow, G., Pignataro, C., Ed., Kumar, N., 766 Aldrin, S., and M. Chen, "Detecting Multiprotocol Label 767 Switched (MPLS) Data-Plane Failures", RFC 8029, 768 DOI 10.17487/RFC8029, March 2017, 769 . 771 12.2. Informative References 773 [I-D.draft-ietf-mpls-spring-lsp-ping] 774 IETF, "Label Switched Path (LSP) Ping/Trace for Segment 775 Routing Networks Using MPLS Dataplane", IETF, 776 https://datatracker.ietf.org/doc/draft-ietf-mpls-spring- 777 lsp-ping/, 2016. 779 [I-D.leipnitz-spring-pms-implementation-report] 780 Leipnitz, R. and R. Geib, "A scalable and topology aware 781 MPLS data plane monitoring system", IETF, draft-leipnitz- 782 spring-pms-implementation-report-00, 2016. 784 [RFC6576] Geib, R., Ed., Morton, A., Fardid, R., and A. Steinmitz, 785 "IP Performance Metrics (IPPM) Standard Advancement 786 Testing", BCP 176, RFC 6576, DOI 10.17487/RFC6576, March 787 2012, . 789 [RFC6808] Ciavattone, L., Geib, R., Morton, A., and M. Wieser, "Test 790 Plan and Results Supporting Advancement of RFC 2679 on the 791 Standards Track", RFC 6808, DOI 10.17487/RFC6808, December 792 2012, . 794 [RFC7882] Aldrin, S., Pignataro, C., Mirsky, G., and N. Kumar, 795 "Seamless Bidirectional Forwarding Detection (S-BFD) Use 796 Cases", RFC 7882, DOI 10.17487/RFC7882, July 2016, 797 . 799 Authors' Addresses 800 Ruediger Geib (editor) 801 Deutsche Telekom 802 Heinrich Hertz Str. 3-7 803 Darmstadt 64295 804 Germany 806 Phone: +49 6151 5812747 807 Email: Ruediger.Geib@telekom.de 809 Clarence Filsfils 810 Cisco Systems, Inc. 811 Brussels 812 Belgium 814 Email: cfilsfil@cisco.com 816 Carlos Pignataro (editor) 817 Cisco Systems, Inc. 818 7200 Kit Creek Road 819 Research Triangle Park, NC 27709-4987 820 US 822 Email: cpignata@cisco.com 824 Nagendra Kumar 825 Cisco Systems, Inc. 826 7200 Kit Creek Road 827 Research Triangle Park, NC 27709 828 US 830 Email: naikumar@cisco.com