idnits 2.17.1 draft-ietf-spring-oam-usecase-10.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (December 18, 2017) is 2314 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-15) exists of draft-ietf-spring-segment-routing-13 Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 spring R. Geib, Ed. 3 Internet-Draft Deutsche Telekom 4 Intended status: Informational C. Filsfils 5 Expires: June 21, 2018 C. Pignataro, Ed. 6 N. Kumar 7 Cisco Systems, Inc. 8 December 18, 2017 10 A Scalable and Topology-Aware MPLS Dataplane Monitoring System 11 draft-ietf-spring-oam-usecase-10 13 Abstract 15 This document describes features of an MPLS path monitoring system 16 and related use cases. Segment based routing enables a scalable and 17 simple method to monitor data plane liveliness of the complete set of 18 paths belonging to a single domain. The MPLS monitoring system adds 19 features to the traditional MPLS Ping and LSP Trace, in a very 20 complementary way. MPLS topology awareness reduces management and 21 control plane involvement of OAM measurements while enabling new OAM 22 features. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at https://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on June 21, 2018. 41 Copyright Notice 43 Copyright (c) 2017 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (https://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 59 2. Terminology and Acronyms . . . . . . . . . . . . . . . . . . 5 60 2.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 5 61 2.2. Acronyms . . . . . . . . . . . . . . . . . . . . . . . . 5 62 3. An MPLS Topology-Aware Path Monitoring System . . . . . . . . 6 63 4. SR-based Path Monitoring Use Case Illustration . . . . . . . 7 64 4.1. Use Case 1 - LSP Dataplane Monitoring . . . . . . . . . . 7 65 4.2. Use Case 2 - Monitoring a Remote Bundle . . . . . . . . . 10 66 4.3. Use Case 3 - Fault Localization . . . . . . . . . . . . . 11 67 5. Path Trace and Failure Notification . . . . . . . . . . . . . 12 68 6. Applying SR to Monitoring non-SR based LSPs (LDP and possibly 69 RSVP-TE) . . . . . . . . . . . . . . . . . . . . . . . . . . 12 70 7. PMS Monitoring of Different Segment ID Types . . . . . . . . 13 71 8. Connectivity Verification Using PMS . . . . . . . . . . . . . 14 72 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15 73 10. Security Considerations . . . . . . . . . . . . . . . . . . . 15 74 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 16 75 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 16 76 12.1. Normative References . . . . . . . . . . . . . . . . . . 17 77 12.2. Informative References . . . . . . . . . . . . . . . . . 17 78 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 18 80 1. Introduction 82 Network operators need to be able to monitor the forwarding paths 83 used to transport user packets. Monitoring packets are expected to 84 be forwarded in dataplane in a similar way as user packets. Segment 85 Routing enables forwarding of packets along pre-defined paths and 86 segments and thus a Segment Routed monitoring packet can stay in 87 dataplane while passing along one or more segments to be monitored. 89 This document describes a system as a functional component called 90 (MPLS) Path Monitoring System, PMS. The PMS is using MPLS data plane 91 path monitoring capabilities. The use cases introduced here are 92 limited to a single Interior Gateway Protocol (IGP) MPLS domain. The 93 use cases of this document refer to the PMS system realised as a 94 separate node. Although many use cases depict the PMS as a physical 95 node, no assumption should be made, and the node could be virtual. 96 This system is defined as a functional component abstracted to have 97 many realizations. The terms PMS and system are used interchangeably 98 in the following. 100 The system applies to monitoring of non Segment Routing Label 101 Switched Paths (LSP's) like Label Distribution Protocol (LDP) as well 102 as to monitoring of Segment Routed LSP's (section 7 offers some more 103 information). As compared to non Segment Routing approaches, Segment 104 Routing is expected to simplify such a monitoring system by enabling 105 MPLS topology detection based on IGP signaled segments. The MPLS 106 topology should be detected and correlated with the IGP topology, 107 which is too detected by IGP signaling. Thus a centralized and MPLS 108 topology aware monitoring unit can be realized in a Segment Routed 109 domain. This topology awareness can be used for Operation, 110 Administration, and Maintenance (OAM) purposes as described by this 111 document. 113 Benefits offered by the system: 115 o The system described here allows to set up an SR domain wide 116 centralized connectivity validation. Many operators of large 117 networks regard centralized monitoring system as useful.. 119 o The MPLS Ping (or continuity check) packets never leave the MPLS 120 user data plane. 122 o SR allows the transport of MPLS path trace or connectivity 123 validation packets for every Label Switched Path to all nodes of 124 an SR domain. This use case doesn't describe new path trace 125 features. The system described here allows to set up an SR domain 126 wide centralized connectivity validation, which is useful in large 127 network operator domains. 129 o The system sending the monitoring packet is also receiving it. 130 The payload of the monitoring packet may be chosen freely. This 131 allows sending probing packets which represent customer traffic, 132 possibly from multiple services (e.g., small Voice over IP packet, 133 larger HTTP packets) and embedding of useful monitoring data 134 (e.g., accurate time stamps since both sender and receiver have 135 the same clock and sequence numbers to ease the measurement...). 137 o Set up of a flexible MPLS monitoring system in terms of 138 deployment: from one single centralized one to a set of 139 distributed systems (e.g., on a per region or service base), and 140 in terms of redundancy from 1+1 to N+1. 142 In addition to monitoring paths, problem localization is required. 143 Topology awareness is an important feature of link state IGPs 144 deployed by operators of large networks. MPLS topology awareness 145 combined with IGP topology awareness enables a simple and scalable 146 data plane based monitoring mechanism. Faults can be localized: 148 o by capturing the Interior Gateway Protocol (IGP) topology and 149 analyzing IGP messages indicating changes of it. 151 o by correlation between different SR based monitoring probes. 153 o by setting up an MPLS traceroute packet for a path (or Segment) to 154 be tested and transporting it to a node to validate path 155 connectivity from that node on. 157 MPLS OAM offers flexible traceroute (connectivity verification) 158 features to detect and execute data paths of an MPLS domain. By 159 utilizing the Equal Cost Multipath (ECMP) related tool set offered, 160 e.g., by RFC 8029 [RFC8029], a SR based MPLS monitoring system can be 161 enabled to: 163 o detect how to route packets along different ECMP routed paths. 165 o construct Ping packets, which can be steered to paths whose 166 connectivity is to be checked, also if ECMP is present. 168 o limit the MPLS label stack of such a Ping packet checking 169 continuity of every single IGP-Segment to the maximum number of 3 170 labels. A smaller label stack may also be helpful, if any router 171 interprets a limited number of packet header bytes to determine an 172 ECMP path along which to route a packet. 174 Alternatively, any path may be executed by building suitable label 175 stacks. This allows path execution without ECMP awareness. 177 The MPLS Path Monitoring System may be any server residing at a 178 single interface of the domain to be monitored. The PMS doesn't need 179 to support the complete MPLS routing or control plane. It needs to 180 be capable to learn and maintain an accurate MPLS and IGP topology. 181 MPLS Ping and traceroute packets need to be set up and sent with the 182 correct segment stack. The PMS further must be able to receive and 183 decode returning Ping or Traceroute packets. Packets from a variety 184 of protocols can be used to check continuity. These include Internet 185 Control Message Protocol [RFC0792] [RFC4443] [RFC4884] [RFC4950], 186 Bidirectional Forwarding Detection (BFD) [RFC5884], Seamless 187 Bidirectional Forwarding Detection (S-BFD) [RFC7880] [RFC7881] (see 188 Section 3.4 of [RFC7882]), and MPLS LSP Ping [RFC8029]. They can 189 also have any other OAM format supported by the PMS. As long as the 190 packet used to check continuity returns back to the server while no 191 IGP change is detected, the monitored path can be considered as 192 validated. If monitoring requires pushing a large label stack, a 193 software based implementation is usually more flexible than an 194 hardware based one. Hence router label stack depth and label 195 composition limitations don't limit MPLS OAM choices. 197 [I-D.ietf-mpls-spring-lsp-ping] discusses SR OAM applicability and 198 MPLS traceroute enhancements adding functionality to the use cases 199 described by this document. 201 The document describes both use cases and a standalone monitoring 202 framework. The monitoring system re-uses existing IETF OAM protocols 203 and leverage Segment Routing (Source Routing) to allow a single 204 device to send, have exercised, and receive its own probing packets. 205 As a consequence, there are no new interoperability considerations. 206 Standard Track is not required and Informational status is 207 appropriate 209 2. Terminology and Acronyms 211 2.1. Terminology 213 Continuity Check 215 is defined in Section 2.2.7 of RFC 7276 [RFC7276]. 217 Connectivity Verification 219 is defined in Section 2.2.7 of RFC 7276 [RFC7276]. 221 MPLS topology 223 The MPLS topology of an MPLS domain is the complete set of MPLS- 224 and IP-address information and all routing and data plane 225 information required to address and utilize every MPLS path 226 within this domain from an MPLS Path Monitoring System attached 227 to this MPLS domain at an arbitrary access. This document 228 assumes availability of the MPLS topology (which can be detected 229 with available protocols and interfaces). None of the use cases 230 will describe how to set it up. 232 This document further adopts the terminology and framework described 233 in [I-D.ietf-spring-segment-routing]. 235 2.2. Acronyms 237 ECMP Equal-Cost Multi-Path 239 IGP Interior Gateway Protocol 240 LER Label Edge Router 242 LSP Label Switched Path 244 LSR Label Switching Router 246 OAM Operations, Administration, and Maintenance 248 PMS Path Monitoring System 250 RSVP-TE Resource ReserVation Protocol-Traffic Engineering 252 SID Segment Identifier 254 SR Segment Routing 256 SRGB Segment Routing Global Block 258 3. An MPLS Topology-Aware Path Monitoring System 260 Any node at least listening to the IGP of an SR domain is MPLS 261 topology aware (the node knows all related IP addresses, SR SIDs and 262 MPLS labels). An MPLS PMS which is able to learn the IGP LSDB 263 (including the SID's) is able to execute arbitrary chains of label 264 switched paths. To monitor an MPLS SR domain, a PMS needs to set up 265 a topology data base of the MPLS SR domain to be monitored. It may 266 be used to send ping type packets to only check continuity along such 267 a path chain based on the topology information only. In addition, 268 the PMS can be used to trace MPLS Label Switched Path and thus verify 269 their connectivity and correspondence between control and data plane, 270 respectively. The PMS can direct suitable MPLS traceroute packets to 271 any node along a path segment. 273 Let us describe how the PMS constructs a labels stack to transport a 274 packet to LER i, monitor its path to LER j and then receive the 275 packet back. 277 The PMS may do so by sending packets carrying the following MPLS 278 label stack information: 280 o Top Label: a path from PMS to LER i, which is expressed as Node 281 SID of LER i. 283 o Next Label: the path that needs to be monitored from LER i to LER 284 j. If this path is a single physical interface (or a bundle of 285 connected interfaces), it can be expressed by the related 286 Adjacency-SID. If the shortest path from LER i to LER j is 287 supposed to be monitored, the Node-SID (LER j) can be used. 289 Another option is to insert a list of segments expressing the 290 desired path (hop by hop as an extreme case). If LER i pushes a 291 stack of Labels based on a SR policy decision and this stack of 292 LSPs is to be monitored, the PMS needs an interface to collect the 293 information enabling it to address this SR created path. 295 o Next Label or address: the path back to the PMS. Likely, no 296 further segment/label is required here. Indeed, once the packet 297 reaches LER j, the 'steering' part of the solution is done and the 298 probe just needs to return to the PMS. This is best achieved by 299 popping the MPLS stack and revealing a probe packet with PMS as 300 destination address (note that in this case, the source and 301 destination addresses could be the same). If an IP address is 302 applied, no SID/label has to be assigned to the PMS (if it is a 303 host/server residing in an IP subnet outside the MPLS domain). 305 The PMS should be physically connected to a router which is part of 306 the SR domain. It must be able to send and receive MPLS packets via 307 this interface. As mentioned above, routing protocol support isn't 308 required and the PMS itself doesn't have to be involved in IGP or 309 MPLS routing. A static route will do. The option to connect a PMS 310 to an MPLS domain by a tunnel may be attractive to some operators. 311 MPLS so far separates networks securely by avoiding tunnel access to 312 MPLS domains. Tunnel based access of a PMS to an MPLS domain is out 313 of scope of this document, as it implies additional security aspects. 315 4. SR-based Path Monitoring Use Case Illustration 317 4.1. Use Case 1 - LSP Dataplane Monitoring 319 Figure 1 shows an example of this functional component as a system, 320 which can be physical or virtual. 322 +---+ +----+ +-----+ 323 |PMS| |LSR1|-----|LER i| 324 +---+ +----+ +-----+ 325 | / \ / 326 | / \__/ 327 +-----+/ /| 328 |LER m| / | 329 +-----+\ / \ 330 \ / \ 331 \+----+ +-----+ 332 |LSR2|-----|LER j| 333 +----+ +-----+ 335 Example of a PMS based LSP dataplane monitoring 337 Figure 1 339 For the sake of simplicity, let's assume that all the nodes are 340 configured with the same SRGB [I-D.ietf-spring-segment-routing]. 342 Let's assign the following Node SIDs to the nodes of the figure: PMS 343 = 10, LER i = 20, LER j = 30. 345 The aim is to set up a continuity check of the path between LER i and 346 LER j. As has been said, the monitoring packets are to be sent and 347 received by the PMS. Let's assume the design aim is to be able to 348 work with the smallest possible SR label stack. In the given 349 topology, a fairly simple option is to perform an MPLS path trace, as 350 specified by RFC 8029 [RFC8029] (using the Downstream (Detailed) 351 Mapping information resulting from a path trace). The starting point 352 for the path trace is LER i and the PMS sends the MPLS path trace 353 packet to LER i. The MPLS echo reply of LER i should be sent to the 354 PMS. As a result, IP destination address choices are detected, which 355 are then used to target any one of the ECMP routed paths between LER 356 i and LER j by the MPLS ping packets to later check path continuity. 357 The Label stack of these ping packets doesn't need to consist of more 358 than 3 labels. Finally, the PMS sets up and sends packets to monitor 359 connectivity of the ECMP routed paths. The PMS does this by creating 360 a measurement packet with the following label stack (top to bottom): 361 20 - 30 - 10. The ping packets reliably use the monitored path, if 362 the IP-address information which has been detected by the MPLS trace 363 route is used as the IP destination address (note that this IP 364 address isn't used or required for any IP routing). 366 LER m forwards the packet received from the PMS to LSR1. Assuming 367 Pen-ultimate Hop Popping to be deployed, LSR1 pops the top label and 368 forwards the packet to LER i. There the top label has a value 30 and 369 LER i forwards it to LER j. This will be done transmitting the 370 packet via LSR1 or LSR2. The LSR will again pop the top label. LER 371 j will forward the packet now carrying the top label 10 to the PMS 372 (and it will pass a LSR and LER m). 374 A few observations on the example given in figure 1: 376 o The path PMS to LER i must be available (i.e., a continuity check 377 only along the path to LER i must succeed). If desired, an MPLS 378 trace route may be used to exactly detect the data plane path 379 taken for this MPLS Segment. It is usually sufficient to just 380 apply any of the existing Shortest Path routed paths. 382 o If ECMP is deployed, separate continuity checks monitoring all 383 possible paths which a packet may use between LER i and LER j may 384 be desired. This can be done by applying an MPLS trace route 385 between LER i and LER j. Another option is to use SR routing, but 386 this will likely require additional label information within the 387 label stack of the ping packet. Further, if multiple links are 388 deployed between two nodes, SR methods to address each individual 389 path require an Adj-SID to be assigned to each single interface. 390 This method is based on control plane information - a connectivity 391 verification based on MPLS traceroute seems to be a fairly good 392 option to deal with ECMP and validation of control and data plane 393 correlation. 395 o The path LER j to PMS must be available (i.e., a continuity check 396 only along the path from LER j to PMS must succeed). If desired, 397 an MPLS trace route may be used to exactly detect the data plane 398 path taken for this MPLS Segment. It is usually sufficient to 399 just apply any of the existing Shortest Path routed paths. 401 Once the MPLS paths (Node-SIDs) and the required information to deal 402 with ECMP have been detected, the path continuity between LER i and 403 LER j can be monitored by the PMS. Path continuity monitoring by 404 ping packets does not require RFC 8029 [RFC8029] MPLS OAM 405 functionality. All monitoring packets stay on dataplane, hence path 406 continuity monitoring does not require control plane interaction in 407 any LER or LSR of the domain. To ensure consistent interpretation of 408 the results, the PMS should be aware of any changes in IGP or MPLS 409 topology or ECMP routing. While the description given here 410 pronouncing path connectivity checking as a simple basic application, 411 others like checking continuity of underlying physical infrastructure 412 or delay measurements may be desired. In both cases, a change in 413 ECMP routing which is not caused by an IGP or MPLS topology change 414 may not be desirable. A PMS therefore should also periodically 415 verify connectivity of the SR paths which are monitored for 416 continuity. 418 Determining a path to be executed prior to a measurement may also be 419 done by setting up a label stack including all Node-SIDs along that 420 path (if LSR1 has Node SID 40 in the example and it should be passed 421 between LER i and LER j, the label stack is 20 - 40 - 30 - 10). The 422 advantage of this method is, that it does not involve RFC 8029 423 [RFC8029] connectivity verification and, if there's only one physical 424 connection between all nodes, the approach is independent of ECMP 425 functionalities. The method still is able to monitor all link 426 combinations of all paths of an MPLS domain. If correct forwarding 427 along the desired paths has to be checked, or multiple physical 428 connections exist between any two nodes, all Adj-SIDs along that path 429 should be part of the label stack. 431 While a single PMS can detect the complete MPLS control and data 432 plane topology, a reliable deployment requires two separated PMS. 433 Scalable permanent surveillance of a set of LSPs could require 434 deployment of several PMS. The PMS may be a router, but could also 435 be dedicated monitoring system. If measurement system reliability is 436 an issue, more than a single PMS may be connected to the MPLS domain. 438 Monitoring an MPLS domain by a PMS based on SR offers the option of 439 monitoring complete MPLS domains with limited effort and a unique 440 possibility to scale a flexible monitoring solution as required by 441 the operator (the number of PMS deployed is independent of the 442 locations of the origin and destination of the monitored paths). The 443 PMS can be enabled to send MPLS OAM packets with the label stacks and 444 address information identical to those of the monitoring packets to 445 any node of the MPLS domain. The routers of the monitored domain 446 should support MPLS LSP Ping RFC 8029 [RFC8029]. They may also 447 incorporate the additional enhancements defined in 448 [I-D.ietf-mpls-spring-lsp-ping] to incorporate further MPLS trace 449 route features. ICMP Ping based continuity checks don't require 450 router control plane activity. Prior to monitoring a path, MPLS OAM 451 may be used to detect ECMP dependent forwarding of a packet. A PMS 452 may be designed to learn the IP address information required to 453 execute a particular ECMP routed path and interfaces along that path. 454 This allows to monitor these paths with label stacks reduced to a 455 limited number of Node-SIDs resulting from SPF routing. The PMS does 456 not require access to LSR / LER management- or data-plane information 457 to do so. 459 4.2. Use Case 2 - Monitoring a Remote Bundle 460 +---+ _ +--+ +-------+ 461 | | { } | |---991---L1---662---| | 462 |PMS|--{ }-|R1|---992---L2---663---|R2 (72)| 463 | | {_} | |---993---L3---664---| | 464 +---+ +--+ +-------+ 466 SR based probing of all the links of a remote bundle 468 Figure 2 470 In the figure, R1 addresses Link "x" Lx by the Adjacency SID 99x, 471 while R2 addresses Link Lx by the Adjacency SID 66(x+1). 473 In the above figure, the PMS needs to assess the dataplane 474 availability of all the links within a remote bundle connected to 475 routers R1 and R2. 477 The monitoring system retrieves the SID/Label information from the 478 IGP LSDB and appends the following segment list/label stack: {72, 479 662, 992, 664} on its IP probe (whose source and destination 480 addresses are the address of the PMS). 482 PMS sends the probe to its connected router. The MPLS/SR domain then 483 forwards the probe to R2 (72 is the Node SID of R2). R2 forwards the 484 probe to R1 over link L1 (Adjacency SID 662). R1 forwards the probe 485 to R2 over link L2 (Adjacency SID 992). R2 forwards the probe to R1 486 over link L3 (Adjacency SID 664). R1 then forwards the IP probe to 487 PMS as per classic IP forwarding. 489 As has been mentioned in section 5.1, the PMS must be able monitor 490 continuity of the path PMS to R2 (Node-SID 72) as well as continuity 491 from R1 to the PMS. If both are given and packets are lost, 492 forwarding on one of the three interfaces connecting R1 to R2 must be 493 disturbed. 495 4.3. Use Case 3 - Fault Localization 497 In the previous example, a uni-directional fault on the middle link 498 in direction of R2 to R1 would be localized by sending the following 499 two probes with respective segment lists: 501 o 72, 662, 992, 664 503 o 72, 663, 992, 664 505 The first probe would succeed while the second would fail. 506 Correlation of the measurements reveals that the only difference is 507 using the Adjacency SID 663 of the middle link from R2 to R1 in the 508 non successful measurement. Assuming the second probe has been 509 routed correctly, the problem is that for some (possibly unknown) 510 reason SR packets to be forwarded from R2 via the interface 511 identified by Adjacency SID 663 are lost. 513 The example above only illustrates a method to localize a fault by 514 correlated continuity checks. Any operational deployment requires a 515 well designed engineering to allow for the desired non ambiguous 516 diagnosis on the monitored section of the SR network. 'Section' here 517 could be a path, a single physical interface, the set of all links of 518 a bundle or an adjacency of two nodes, just to name a few. 520 5. Path Trace and Failure Notification 522 Sometimes forwarding along a single path indeed doesn't work, while 523 the control plane information is healthy. Such a situation may occur 524 after maintenance work within a domain. An operator may perform on 525 demand-tests, but execution of automated PMS path trace checks may be 526 set up too (scope may be limited to a subset of important end-to-end 527 paths crossing the router or network section after completion of the 528 maintenance work there). Upon detection of a path which can't be 529 used, the operator needs to be notified. A check ensuring that re- 530 routing event is differed from a path facing whose forwarding 531 behavior doesn't correspond to the control plane information is 532 necessary (but out of scope of this document). 534 Adding an automated problem solution to the PMS features only makes 535 sense, if the root cause of the symptom appears often, can be assumed 536 to be non-ambiguous by its symptoms, can be solved by a pre- 537 determined chain of commands and the automated PMS reaction not doing 538 any collateral damage. A closer analysis is out of scope of this 539 document. 541 The PMS is expected to check control plane liveliness after a path 542 repair effort was executed. It doesn't matter whether the path 543 repair was triggered manually or by an automated system. 545 6. Applying SR to Monitoring non-SR based LSPs (LDP and possibly RSVP- 546 TE) 548 The MPLS path monitoring system described by this document can be 549 realized with non-Segment Routing (SR) based technology. Making such 550 a non-SR MPLS monitoring system aware of a domain's complete MPLS 551 topology requires, e.g., management plane access to the routers of 552 the domain to be monitored or set up of a dedicated tLDP tunnel per 553 router to set up an LDP adjacency. To avoid the use of stale MPLS 554 label information, the IGP must be monitored and MPLS topology must 555 be timely aligned with IGP topology. Enhancing IGPs to exchange of 556 MPLS topology information as done by SR significantly simplifies and 557 stabilizes such an MPLS path monitoring system. 559 A SR based PMS connected to a MPLS domain consisting of LER and LSR 560 supporting SR and LDP or RSVP-TE in parallel in all nodes may use SR 561 paths to transmit packets to and from start and end points of non-SR 562 based LSP paths to be monitored. In the example given in figure 1, 563 the label stack top to bottom may be as follows, when sent by the 564 PMS: 566 o Top: SR based Node-SID of LER i at LER m. 568 o Next: LDP or RSVP-TE label identifying the path or tunnel, 569 respectively from LER i to LER j (at LER i). 571 o Bottom: SR based Node-SID identifying the path to the PMS at LER j 573 While the mixed operation shown here still requires the PMS to be 574 aware of the LER LDP-MPLS topology, the PMS may learn the SR MPLS 575 topology by IGP and use this information. 577 An implementation report on a PMS operating in an LDP domain is given 578 in [I-D.leipnitz-spring-pms-implementation-report]. In addition, 579 this report compares delays measured with a single PMS to the results 580 measured by three standard conformant Measurement Agents ([RFC6808] 581 connected to an MPLS domain at three different sites). The delay 582 measurements of the PMS and the IPPM Measurement Agents were compared 583 based on a statistical test in [RFC6576]. The Anderson Darling 584 k-sample test showed that the PMS round-trip delay measurements are 585 equal to those captured by an IPPM conformant IP measurement system 586 for 64 Byte measurement packets with 95% confidence. 588 The authors are not aware of similar deployment for RSVP-TE. 589 Identification of tunnel entry- and transit-nodes may add complexity. 590 They are not within scope of this document. 592 7. PMS Monitoring of Different Segment ID Types 594 MPLS SR topology awareness should allow the PMS to monitor liveliness 595 of SIDs related to interfaces within the SR and IGP domain, 596 respectively. Tracing a path where an SR capable node assigns an 597 Adj-SID for a non-SR-capable node may fail. This and other backward 598 compatibility with non Segment Routing devices are discussed by 599 [I-D.ietf-mpls-spring-lsp-ping]. 601 To match control plane information with data plane information for 602 all relevant types of Segment IDs, [I-D.ietf-mpls-spring-lsp-ping] 603 enhances MPLS OAM functions defined by RFC 8029 [RFC8029]. 605 8. Connectivity Verification Using PMS 607 While the PMS based use cases explained in Section 5 are sufficient 608 to provide continuity check between LER i and LER j, it may not help 609 perform connectivity verification. 611 +---+ 612 |PMS| 613 +---+ 614 | 615 | 616 +----+ +----+ +-----+ 617 |LSRa|-----|LSR1|-----|LER i| 618 +----+ +----+ +-----+ 619 | / \ / 620 | / \__/ 621 +-----+/ /| 622 |LER m| / | 623 +-----+\ / \ 624 \ / \ 625 \+----+ +-----+ 626 |LSR2| |LER j| 627 +----+ +-----+ 629 Connectivity verification with a PMS 631 Figure 3 633 Let's assign the following Node SIDs to the nodes of the figure: PMS 634 = 10, LER i = 20, LER j = 30, LER m = 40. PMS is intended to 635 validate the path between LER m and LER j. In order to validate this 636 path, PMS will send the probe packet with label stack of (top to 637 bottom): {40} {30} {10}. Imagine any of the below forwarding entry 638 misprogrammed situation: 640 o LSRa receiving any packet with top label 40 will POP and forwards 641 to LSR1 instead of LER m. 643 o LSR1 receiving any packet with top label 30 will pop and forward 644 to LER i instead of LER j. 646 In any of these above situation, the probe packet will be delivered 647 back to PMS leading to a falsified path liveliness indication by the 648 PMS. 650 Connectivity Verification functions helps us to verify if the probe 651 is taking the expected path. For example, PMS can intermittently 652 send the probe packet with label stack of (top to bottom): 653 {40;ttl=255} {30;ttl=1} {10;ttl=255}. The probe packet may carry 654 information about LER m which could be carried in Target FEC Stack in 655 case of MPLS Echo Request or Discriminator in case of Seamless BFD. 656 When LER m receives the packet, it will punt due to TTL expiry and 657 sends a positive response. In the above mentioned misprogramming 658 situation, LSRa will forwards to LSR1 which will send a negative 659 response to PMS as the information in probe does not match the local 660 node. PMS can do the same for bottom label as well. This will help 661 perform connectivity verification and ensure that the path between 662 LER m and LER j is working as expected. 664 9. IANA Considerations 666 This memo includes no request to IANA. 668 10. Security Considerations 670 The PMS builds packets with intent of performing OAM tasks. It uses 671 address information based on topology information, rather than a 672 protocol. 674 The PMS allows the insertion of traffic into non-SR domains. This 675 may be required in the case of an LDP domain attached to the SR 676 domain, but it can be used to maliciously insert traffic in the case 677 of external IP domains and MPLS based VPNs. 679 To prevent a PMS from inserting traffic into an MPLS VPN domain, one 680 or more sets of label ranges may be reserved for service labels 681 within an SR domain. The PMS should be configured to reject usage of 682 these service label values. In the same way, misuse of IP 683 destination addresses is blocked if only IP-destination address 684 values conforming to RFC 8029 [RFC8029] are settable by the PMS. 686 To limit potential misuse, access to a PMS needs to be authorized and 687 should be logged. OAM supported by a PMS requires skilled personnel 688 and hence only experts requiring PMS access should be allowed to 689 access such a system. It is recommended to directly attach a PMS to 690 an SR domain. Connecting a PMS to an SR domain by a tunnel is 691 technically possible, but adds further security issues. A tunnel 692 based access of a PMS to an SR domain is not recommended. 694 Use of stale MPLS or IGP routing information could cause a PMS 695 monitoring packet to leave the domain where it originated. PMS 696 monitoring packets should not be sent using stale MPLS or IGP routing 697 information. To carry out a desired measurement properly, the PMS 698 must be aware of and respect the actual route changes, convergence 699 events, as well as the assignment of Segment IDs relevant for 700 measurements. At a minimum, the PMS must be able to listen to IGP 701 topology changes, or pull routing and segment information from 702 routers signaling topology changes. 704 Traffic insertion by a PMS may be unintended, especially if the IGP 705 or MPLS topology stored locally are in stale state. As soon as the 706 PMS has an indication, that its IGP or MPLS topology are stale, it 707 should stop operations involving network sections whose topology may 708 not be accurate. Note however that it is a task of an OAM system to 709 discover and locate network sections having where forwarding behavior 710 is not matching control plane state. As soon as a PMS or an operator 711 of a PMS has the impression that the PMS topology information is 712 stale, measures need to be taken to refresh the topology information. 713 These measures should be part of the PMS design. Matching forwarding 714 and control plane state by periodically automated execution of RFC 715 8029 [RFC8029] mechanisms may be such a feature. Whenever network 716 maintenance tasks are performed by operators, the PMS topology 717 discovery should be started asynchronously after network maintenance 718 has been finished. 720 A PMS loosing network connectivity or crashing must remove all IGP 721 and MPLS topology information prior to restarting operation. 723 A PMS may operate routine measurements on large scale. Care must be 724 taken to avoid unintended traffic insertion after topology changes 725 which result , e.g., in changes of label assignments to routes or 726 interfaces within a domain. If the labels concerned are part of the 727 label stack composed by the PMS for any measurement packet and their 728 state is stale, the measurement initially needs to be stopped. Set 729 up and operation of routine measurements may be automated. Secure 730 automated PMS operation requires a working automated detection and 731 recognition of stale routing state. 733 11. Acknowledgements 735 The authors would like to thank Nobo Akiya for his contribution. 736 Raik Leipnitz kindly provided an editorial review. The authors would 737 also like to thank Faisal Iqbal for an insightful review and a useful 738 set of comments and suggestions. Finally, Bruno Decraene's shepherd 739 review led to a clarified document. 741 12. References 742 12.1. Normative References 744 [I-D.ietf-spring-segment-routing] 745 Filsfils, C., Previdi, S., Ginsberg, L., Decraene, B., 746 Litkowski, S., and R. Shakir, "Segment Routing 747 Architecture", draft-ietf-spring-segment-routing-13 (work 748 in progress), October 2017. 750 [RFC7276] Mizrahi, T., Sprecher, N., Bellagamba, E., and Y. 751 Weingarten, "An Overview of Operations, Administration, 752 and Maintenance (OAM) Tools", RFC 7276, 753 DOI 10.17487/RFC7276, June 2014, 754 . 756 12.2. Informative References 758 [I-D.ietf-mpls-spring-lsp-ping] 759 Kumar, N., Pignataro, C., Swallow, G., Akiya, N., Kini, 760 S., and M. Chen, "Label Switched Path (LSP) Ping/ 761 Traceroute for Segment Routing IGP Prefix and Adjacency 762 SIDs with MPLS Data-plane", draft-ietf-mpls-spring-lsp- 763 ping-13 (work in progress), October 2017. 765 [I-D.leipnitz-spring-pms-implementation-report] 766 Leipnitz, R. and R. Geib, "A scalable and topology aware 767 MPLS data plane monitoring system", draft-leipnitz-spring- 768 pms-implementation-report-00 (work in progress), June 769 2016. 771 [RFC0792] Postel, J., "Internet Control Message Protocol", STD 5, 772 RFC 792, DOI 10.17487/RFC0792, September 1981, 773 . 775 [RFC4443] Conta, A., Deering, S., and M. Gupta, Ed., "Internet 776 Control Message Protocol (ICMPv6) for the Internet 777 Protocol Version 6 (IPv6) Specification", STD 89, 778 RFC 4443, DOI 10.17487/RFC4443, March 2006, 779 . 781 [RFC4884] Bonica, R., Gan, D., Tappan, D., and C. Pignataro, 782 "Extended ICMP to Support Multi-Part Messages", RFC 4884, 783 DOI 10.17487/RFC4884, April 2007, 784 . 786 [RFC4950] Bonica, R., Gan, D., Tappan, D., and C. Pignataro, "ICMP 787 Extensions for Multiprotocol Label Switching", RFC 4950, 788 DOI 10.17487/RFC4950, August 2007, 789 . 791 [RFC5884] Aggarwal, R., Kompella, K., Nadeau, T., and G. Swallow, 792 "Bidirectional Forwarding Detection (BFD) for MPLS Label 793 Switched Paths (LSPs)", RFC 5884, DOI 10.17487/RFC5884, 794 June 2010, . 796 [RFC6576] Geib, R., Ed., Morton, A., Fardid, R., and A. Steinmitz, 797 "IP Performance Metrics (IPPM) Standard Advancement 798 Testing", BCP 176, RFC 6576, DOI 10.17487/RFC6576, March 799 2012, . 801 [RFC6808] Ciavattone, L., Geib, R., Morton, A., and M. Wieser, "Test 802 Plan and Results Supporting Advancement of RFC 2679 on the 803 Standards Track", RFC 6808, DOI 10.17487/RFC6808, December 804 2012, . 806 [RFC7880] Pignataro, C., Ward, D., Akiya, N., Bhatia, M., and S. 807 Pallagatti, "Seamless Bidirectional Forwarding Detection 808 (S-BFD)", RFC 7880, DOI 10.17487/RFC7880, July 2016, 809 . 811 [RFC7881] Pignataro, C., Ward, D., and N. Akiya, "Seamless 812 Bidirectional Forwarding Detection (S-BFD) for IPv4, IPv6, 813 and MPLS", RFC 7881, DOI 10.17487/RFC7881, July 2016, 814 . 816 [RFC7882] Aldrin, S., Pignataro, C., Mirsky, G., and N. Kumar, 817 "Seamless Bidirectional Forwarding Detection (S-BFD) Use 818 Cases", RFC 7882, DOI 10.17487/RFC7882, July 2016, 819 . 821 [RFC8029] Kompella, K., Swallow, G., Pignataro, C., Ed., Kumar, N., 822 Aldrin, S., and M. Chen, "Detecting Multiprotocol Label 823 Switched (MPLS) Data-Plane Failures", RFC 8029, 824 DOI 10.17487/RFC8029, March 2017, 825 . 827 Authors' Addresses 829 Ruediger Geib (editor) 830 Deutsche Telekom 831 Heinrich Hertz Str. 3-7 832 Darmstadt 64295 833 Germany 835 Phone: +49 6151 5812747 836 Email: Ruediger.Geib@telekom.de 837 Clarence Filsfils 838 Cisco Systems, Inc. 839 Brussels 840 Belgium 842 Email: cfilsfil@cisco.com 844 Carlos Pignataro (editor) 845 Cisco Systems, Inc. 846 7200 Kit Creek Road 847 Research Triangle Park, NC 27709-4987 848 US 850 Email: cpignata@cisco.com 852 Nagendra Kumar 853 Cisco Systems, Inc. 854 7200 Kit Creek Road 855 Research Triangle Park, NC 27709 856 US 858 Email: naikumar@cisco.com