idnits 2.17.1 draft-tissa-nvo3-oam-fm-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There is 1 instance of too long lines in the document, the longest one being 1 character in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 133 has weird spacing: '...leading to...' == Line 927 has weird spacing: '... out of b...' == Line 1499 has weird spacing: '... no speci...' == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (May 5, 2017) is 2547 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'TRILLFM' is mentioned on line 159, but not defined == Missing Reference: 'TBDMIBD' is mentioned on line 670, but not defined == Missing Reference: 'TBDMIB' is mentioned on line 722, but not defined == Unused Reference: 'RFC4379' is defined on line 1550, but no explicit reference was found in the text == Unused Reference: 'RFC6291' is defined on line 1554, but no explicit reference was found in the text == Unused Reference: 'NVO3ARC' is defined on line 1563, but no explicit reference was found in the text == Unused Reference: 'NV03DPREQ' is defined on line 1570, but no explicit reference was found in the text -- Obsolete informational reference (is this intentional?): RFC 4379 (Obsoleted by RFC 8029) Summary: 1 error (**), 0 flaws (~~), 12 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 NV03 Working Group Tissa Senevirathne 3 Internet Draft Samer Salam 4 Intended Status: Informational Deepak Kumar 5 Norman Finn 7 Cisco 8 Donald Eastlake 9 Sam Aldrin 10 Huawei 11 Expires September 2017 May 5, 2017 13 NVO3 Fault Management 14 draft-tissa-nvo3-oam-fm-04.txt 16 Status of this Memo 18 This Internet-Draft is submitted in full conformance with the 19 provisions of BCP 78 and BCP 79. 21 Internet-Drafts are working documents of the Internet Engineering 22 Task Force (IETF), its areas, and its working groups. Note that 23 other groups may also distribute working documents as Internet- 24 Drafts. 26 Internet-Drafts are draft documents valid for a maximum of six months 27 and may be updated, replaced, or obsoleted by other documents at any 28 time. It is inappropriate to use Internet-Drafts as reference 29 material or to cite them other than as "work in progress." 31 The list of current Internet-Drafts can be accessed at 32 http://www.ietf.org/ietf/1id-abstracts.txt. 34 The list of Internet-Draft Shadow Directories can be accessed at 35 http://www.ietf.org/shadow.html. 37 This Internet-Draft will expire on November 6, 2017. 39 Copyright Notice 41 Copyright (c) 2017 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (http://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Abstract 56 This document specifies Fault Management solution for network 57 virtualization overlay networks. Methods in this document follow the 58 IEEE 802.1 CFM framework and reuse OAM tools where possible. 59 Additional messages and TLVs are defined for IETF overlay OAM 60 specific applications or where extensions beyond IEEE 802.1 CFM are 61 required. 63 Table of Contents 65 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 66 2. Conventions used in this document . . . . . . . . . . . . . . . 5 67 3. NV03 OAM Layers . . . . . . . . . . . . . . . . . . . . . . . . 6 68 4. General Format of NV03 OAM frames . . . . . . . . . . . . . . . 7 69 4.1. NV03 Shim . . . . . . . . . . . . . . . . . . . . . . . . . 8 70 4.2. Identification of OAM frames . . . . . . . . . . . . . . . 8 71 4.3 OAM Channel . . . . . . . . . . . . . . . . . . . . . . . . 10 72 4.3.1 OAM Channel Functionality Summary . . . . . . . . . . . 10 73 4.3.2 Opcode Implementation Recommendation . . . . . . . . . . 10 74 5. Maintenance Associations (MA) in NVO3 . . . . . . . . . . . . . 11 75 6. MEP Addressing . . . . . . . . . . . . . . . . . . . . . . . . 12 76 6.1. Use of MIP in NV03 . . . . . . . . . . . . . . . . . . . . 15 77 7. Continuity Check Message (CCM) . . . . . . . . . . . . . . . . 16 78 8. NVO3 OAM Message Channel . . . . . . . . . . . . . . . . . . . 18 79 8.1. NVO3 OAM Message header . . . . . . . . . . . . . . . . . . 18 80 8.2. IETF Overlay OAM Opcodes . . . . . . . . . . . . . . . . . 19 81 8.3. Format of IETF Overlay OAM TLV . . . . . . . . . . . . . . 19 82 8.4. IETF Overlay OAM TLVs . . . . . . . . . . . . . . . . . . . 20 83 8.4.1. Common TLVs between 8201Q CFM and IETF Overlay OAM . . 20 84 8.4.2. IETF Overaly OAM Specific TLVs . . . . . . . . . . . . 20 85 8.4.3. OAM Application Identifier TLV . . . . . . . . . . . . 21 86 8.4.4. Out Of Band Reply Address TLV . . . . . . . . . . . . . 22 87 8.4.5. Diagnostics Label TLV . . . . . . . . . . . . . . . . . 23 88 8.4.6. Original Data Payload TLV . . . . . . . . . . . . . . . 23 89 8.4.7. Flow Identifier (flow-id) TLV . . . . . . . . . . . . . 24 90 8.4.8. Reflector Entropy TLV . . . . . . . . . . . . . . . . . 24 91 9. Loopback Message . . . . . . . . . . . . . . . . . . . . . . . 25 92 9.2. Theory of Operation . . . . . . . . . . . . . . . . . . . . 26 93 9.2.1. Actions by Originator . . . . . . . . . . . . . . . . . 26 94 9.2.2. Intermediate Devices . . . . . . . . . . . . . . . . . 26 95 9.2.3. Destination Device . . . . . . . . . . . . . . . . . . 27 96 10. Path Trace Message . . . . . . . . . . . . . . . . . . . . . . 27 97 10.1. Theory of Operation . . . . . . . . . . . . . . . . . . . 28 98 10.1.1. Action by Originator Device . . . . . . . . . . . . . 28 99 10.1.2. Intermediate Device . . . . . . . . . . . . . . . . . 29 100 10.1.3. Destination Device . . . . . . . . . . . . . . . . . . 29 101 11. Link Trace Message . . . . . . . . . . . . . . . . . . . . . . 30 102 11.1 MEP and MIP . . . . . . . . . . . . . . . . . . . . . . . . 30 103 11.2 Initiator . . . . . . . . . . . . . . . . . . . . . . . . . 30 104 11.3 Intermediate Devices . . . . . . . . . . . . . . . . . . . 31 105 11.4 Terminating Device . . . . . . . . . . . . . . . . . . . . 31 106 11.5 Output . . . . . . . . . . . . . . . . . . . . . . . . . . 31 107 12. Application of Continuity Check Message (CCM) in NVO3 . . . . 32 108 12.1. CCM Error Notification . . . . . . . . . . . . . . . . . . 32 109 12.2. Theory of Operation . . . . . . . . . . . . . . . . . . . 34 110 12.2.1. Actions by Originator Device . . . . . . . . . . . . . 34 111 12.2.2. Intermediate Devices . . . . . . . . . . . . . . . . . 34 112 12.2.3. Destination Device . . . . . . . . . . . . . . . . . . 34 113 13. Security Considerations . . . . . . . . . . . . . . . . . . . 35 114 14. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 35 115 15. References . . . . . . . . . . . . . . . . . . . . . . . . . . 35 116 15.1. Normative References . . . . . . . . . . . . . . . . . . . 35 117 15.2. Informative References . . . . . . . . . . . . . . . . . . 35 118 16. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 36 119 Appendix A. Base Mode for NVO3 OAM . . . . . . . . 36 121 1. Introduction 123 Conceptually, NVO3 architecture contains four separate layers, 124 namely, Service (customer) Layer, Overlay Layer, Transport or 125 Underlay Layer and Media Layer. Figure 1 below depicts the 126 relationship between each of these layers. 128 Fault Monitoring, Fault Verification, Fault Isolation, Loss and delay 129 measurements are integral part of NVO3 [nvo3oamReq]. These need to 130 cover both unicast and multicast traffic streams. 132 For effective fault isolation, the overlay OAM solution should 133 complement the OAM functions at adjacent layers, thereby leading to 134 nested OAM model. Nested OAM allows operators to quickly and 135 effectively troubleshoot and isolate data plane failures using a 136 common OAM framework. 138 Common OAM message format and infrastructure makes it easier to 139 accomplish nested OAM. IEEE Connectivity Fault Management (CFM) 140 [8021Q] is widely used in the Ethernet world. The same technology has 141 been extended by ITU-T [Y1731], Metro Ethernet Forum (MEF) and TRILL 142 [TRILLFM]. 144 A common OAM message channel can be shared between different 145 technologies. This consistency between different OAM technologies 146 promotes nested fault monitoring and isolation between technologies 147 that share the same OAM framework. 149 This document uses the message format defined in IEEE 802.1ag 150 Connectivity Fault Management (CFM) [8021Q] as the basis for the OAM 151 messages. 153 This document presents the NVO3 OAM message structure, NVO3 frame 154 identification and NV03 OAM tools. The NVO3 OAM Management 155 Information Base (MIB) will be presented in a separate document. 157 The ITU-T Y.1731 [Y1731] standard utilizes the same messaging format 158 as [8021Q] and for OAM messages where applicable. This document takes 159 a similar stance and reuses [8021Q] and TRILL OAM [TRILLFM]. It is 160 assumed that readers are familiar with [8021Q] and [Y1731]. 162 |T |--|NVE|--|R|-|B|--|B|-|R|-|NVE|-|T| 164 --- X----o-----------------------o-----X [Service Layer] 165 | O | | | 166 | A | | | 167 | M | X------o-----------o----X [NVo3 Layer] 168 | | | | 169 | M | | | 170 | I | X--o-----o--X [Transport Layer] 171 | B | | | 172 | | | | 173 --- X--o--X [Link/Circuit Layer] 175 X - Maintenance End Point (MEP) 176 o - Maintenance Intermediate Point (MIP) 178 T - Tenant System NVE - Network Virtualization Edge 179 R - Router B - Bridge 181 Figure 1 Layered OAM Architecture 183 2. Conventions used in this document 185 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", 186 "SHALL NOT","SHOULD", "SHOULD NOT", "RECOMMENDED", 187 "MAY", and "OPTIONAL" in this 188 document are to be interpreted as described in RFC-2119 [RFC2119]. 190 Acronyms used in the document include the following: 192 MP - Maintenance Point [8021Q] 194 MEP - Maintenance End Point [8021Q] 195 MIP - Maintenance Intermediate Point [8021Q] 197 MA - Maintenance Association [8021Q] 199 MD - Maintenance Domain [8021Q] 201 CCM - Continuity Check Message [8021Q] 203 LBM - Loop Back Message [8021Q] 205 PTM - Path Trace Message 207 MTV - Multi-destination Tree Verification Message 209 ECMP - Equal Cost Multipath 211 ISS - Internal Sub Layer Service [8021Q] 213 VNI - Virtual Network Instance [NVO3FRM] 215 NVE - Network Virtual Edge [NVO3FRM] 217 SAP - Service Access Point [8021Q] 219 3. NV03 OAM Layers 221 Figure 1 above depicts different layers within NVO3. Each of these 222 layers has a unique scope within the common framework. In this 223 section, we define functionality of each of these layers 225 Service Layer: Service Layer carries customer or user traffic. 226 It is originated at Tenant systems and terminates at other 227 Tenant system(s) within the same NVO3 context (VNI). 229 NVO3 Layer: NVO3 Layer carries service Layer traffic 230 encapsulated in NVO3 format. NVO3 Layer originates at an 231 NVE and terminates at another NVE. 233 Transport Layer: Transport Layer carries NVO3 Layer traffic 234 encapsulated in its data format. The transport Layer 235 can be IP, MPLS or any other protocol. 237 Link Layer: This is also known as circuit layer. 238 It carries Transport Layer traffic in a specific manner 239 from one device to another. 240 Ethernet is an example of link layer. 242 4. General Format of NV03 OAM frames 244 For accurate monitoring and/or diagnostics, OAM Messages are required 245 to follow the same data path as corresponding user packets. 246 Additionally, NVEs are required to identify NVO3 frames and act on 247 them, in addition to preventing the overlay OAM packets from leaking 248 outside of the NVO3 domain [NVO3OAMREQ]. This document defines the 249 format of the NVO3 overlay OAM messages. 251 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 252 | | 253 . Circuit Header . (variable) 254 | | 255 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 256 | | 257 + Transport Header Technology dependent 258 | | 259 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 260 | NVO3 Shim | 8 bytes 261 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 262 | | 263 . Payload fragment . 96 bytes (optional) 264 . . 265 | | 266 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 267 | OAM Ether Type | 268 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 269 | | 270 . OAM Message Channel . Variable 271 . . 272 | | 273 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 274 | Link Trailer | Variable 275 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 277 Figure 2 Format of NVO3 Overlay OAM Messages 279 Link Header: Media-dependent header. For Ethernet, this includes 280 Destination MAC, Source MAC, VLAN (optional) and EtherType fields. 282 Transport Header: Header of the Transport Layer e.g. IP , MPLS, TRILL 283 etc. 285 NVO3 Shim: This is a fixed sized field (size TBD 8 bytes [vxLAN]) 286 that carries NVO3 specific information. Fields in this header will be 287 utilized to identify OAM frames and other NVO3 specific operations. 288 Please see section 4.2 below of NVO3 OAM specific operations. 290 Payload Fragment: This is an optional field. When included it has a 291 fixed size of 96 bytes. The least significant bits of the field MUST 292 be padded with zeros, up to 96 bytes, when the payload fragment is 293 less than 96 bytes. The Payload Fragment field starts with the 294 Inner.MacDA. 296 OAM Ether Type: OAM Ether Type is 16-bit EtherType that identifies 297 the OAM Message channel that follows. This document specifies using 298 the EtherType 0x8902 allocated for [8021Q] for this purpose. 299 Identifying the OAM Message Channel with a dedicated EtherType allows 300 the easy identification of the beginning of the OAM message channel 301 across multiple standards. 303 OAM Message Channel: This is a variable size section that carries OAM 304 related information. The message format defined in [8021Q] will be 305 reused. 307 Link Trailer: Media-dependent trailer. For Ethernet, this is the FCS 308 (Frame Check Sequence). 310 Note: In this draft we are proposing re-use of OAM Channel defined in 311 RFC7455 and RFC7456. 313 4.1. NV03 Shim 315 NVO3 Shim is an 8 octet vector that carries series of flags and 316 additional information [vxLAN]. Each of the flags identifies a 317 specific operation. 319 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 320 |R|R|R|R|I|R|R|R| Reserved | 321 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 322 | VXLAN Network Identifier (VNI) | Reserved | 323 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 325 Figure 3 NVO3 Shim 327 4.2. Identification of OAM frames 329 Implementations that comply with this document MUST utilize "O" flag 330 to identify NVO3 OAM frames. The "O" flag MUST NOT BE utilized for 331 forwarding decisions such as the selection of which ECMP path to use. 333 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 334 |R|R|R|R|I|R|R|O| Reserved | 335 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 336 | VXLAN Network Identifier (VNI) | Reserved | 337 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 339 Figure 4 NVO3 shim with the "O" Flag 341 O (1 bit) - Indicates this is a possible OAM frame and is subject to 342 specific handling as specified in this document. O bit is aligned 343 with Vxlan GPE. 345 Payload Fragment is optional and if hardware is capable of matching 346 OAM frame based on O bit then payload fragment handling doesn't 347 require extra bit by checking etype. OAM Frame may have 96 octets of 348 payload fragments immediately after the NVO3 shim or OAM Ethertype 349 0x8902 immediately follows the NVO3 shim after Inner DMAC and Inner 350 SMAC. This can be determined by matching 0x8902 at right position 351 after matching "O" bit, if it's not present, then look deeper after 352 96 bytes. 354 Payload Fragment is optional implementation but it's very important 355 to track actual data path in scenario described below. 357 SS1 SS2 (L3 VTEP) 358 / | \/ \ 359 / | /\ \ (ECMP paths) 360 / |/ \ \ 361 S1 S2 S3 362 || || || (Underlay ECMP) 363 || || || 364 L1 L2 L3 (L2 VTEP) 366 | | 367 H1 H2 369 Figure 5 Payload Fragment use case 371 For H1 and H2 we can have bridge traffic and routed traffic on SSx. 372 For Routed traffic inner header or payload Fragement is required to 373 be looked to find the exact ECMP path. 375 All other fields carry the same meaning as defined in [vXLAN]. 377 4.3 OAM Channel 379 OAM channel proposed to use is based of RFC7455 and RFC7456. 380 Advantage of re-using this channel is header is flexible to support 381 extensible functionality. It also provide backward compatibility mode 382 for hardware which were developed before OAM became standard to do 383 OAM functionality. 385 4.3.1 OAM Channel Functionality Summary 387 OAM Common Header is defined in section 8.1. MD-L allows multiple 388 level of OAM to be possible, for eg:- IP layer Overlay can be at 389 default level 3 and SFC layer OAM can be at Application Level 4. 391 Opcode provide hardware friendly and extensible extensions. Hardware 392 friendly messages are defined without TLVs. 394 Summary of Opcode(s) to be considered. 396 1. CCM - Proactive Fault Monitoring (one-way) 398 2. LBM/LBR - on-demand Fault Verification (LBR can be sent out of 399 band also and carry ICMP error code if required.) 401 3. PTM/PTR - on-demand Fault Isolation based on TTL expiry (works 402 with non OAM capable underlay, ip unnumbered underlay, and provide 403 Egress Interface to better isolate fault than traditional traceroute) 405 4. DMM/DMR - on-demand or pro-active Delay Measurement. 407 5. 1DM - On-demand or pro-active one way Delay Measurement. 409 6. LTM/LTR - on-demand Fault Isolation for scenario where all 410 underlay switches are OAM capable to provide path via hardware 411 forwarding without CPU intervention. 413 7. SLM/SLR - on-demand or pro-active Synthetic Delay Measurement. 415 8. 1SL - On-demand or pro-active one way Synthetic Delay Measurement. 417 Application Identifier TLV allow Return code and sub-code to carry 418 ICMP Errors, It allows out of band or in-band communication 419 flexibility and support OAM to be carried in multiple fragments if 420 data request is very large. 422 4.3.2 Opcode Implementation Recommendation 424 CCM - Optional LBM/LBR - Mandatory PTM/PTR - Mandatory DMM/DMR - 425 Optional (As it's performance function) 1DM - Optional LTM/LTR - 426 Optional SLM/SLR - Optional 1SL - Optional 428 5. Maintenance Associations (MA) in NVO3 430 [8021Q] defines a maintenance association as a logical relationship 431 between a group of nodes. Each Maintenance Association (MA) is 432 identified with a unique MAID of 48 bytes [8021Q]. CCM and other 433 related OAM functions operate within the scope of an MA. The 434 definition of MA is technology independent. MA is encoded in the 435 technology independent part of the OAM message Hence the MAID, as 436 defined in [8021Q], can be utilized for NVo3 OAM, without 437 modifications. This also allows us to utilize CCM and LBM messages 438 defined in [8021Q], as is. 440 In NVO3, an MA may contain two or more NVEs (MEPs). For unicast, it 441 is likely that the MA contains exactly two MEPs that are the two end- 442 points of the flow. For multicast, the MA may contain two or more 443 MEPs. 445 For NVO3, in addition to all of the standard CFM MIB [8021Q] 446 definitions, each MEP's MIB contains one or more flow definitions 447 corresponding to the set of flows that the MEP monitors. Flow entropy 448 specifies the VNI within the NVE. 450 MEPs can be created per VNI within the NVE. 452 We propose to augment the [8021Q] MIB to add NVO3 specific 453 information. Figure 5, below depicts the augmentation of the CFM MIB 454 to add the NVO3 specific Flow Entropy. 456 MA--- 457 | 458 --- MEP 459 | 460 . - Remote MEP List 461 . 462 | 463 --- MEP-A 464 | 465 --- MEP-B 466 . 468 | 469 . - Flow List { Augments IEEE8021-CFM-MIB} 471 | 472 --- (Flow-1) 473 | 474 --- (Flow-2) 475 | 476 . --- (Flow-n) 477 | 478 Other MIB entries 480 Figure 6 Correlation of NVO3 augmented MIB 482 The flow list contains multiple flows. Each flow contains a VNI and 483 optional data payload. There can be more than one flow for a given 484 VNI with different data payloads e.g. unicast vs. multicast or 485 unicast with different data payloads. 487 6. MEP Addressing 489 In IEEE 802.1ag [8021Q], OAM messages address the target MEP by 490 utilizing a unique MAC address. In NVO3, MEPs are created per VNI, 491 MEPs are addressed by combination of Transport Layer address of the 492 NVE and the VNI. 494 At the MEP, OAM packets go through a hierarchy of op-code de- 495 multiplexers. The op-code de-multiplexers channel the incoming OAM 496 packets to the appropriate message processor (e.g. LBM) The reader 497 may refer to Figure 6 below for a visual depiction of these different 498 de-multiplexers. 500 1. Identify the packets that need OAM processing at the Local Device 501 as specified in Section 4.2. 503 a. Identify the MEP that is associated with the VNI. 505 2. The MEP then validates the MD-LEVEL 507 a. Redirect to MD-LEVEL De-multiplexer 509 3. MD-LEVEL de-multiplexer compares the MD-Level of the packet 510 against the MD level of the local MEPs. (Note: there can be more 511 than one MEP at the same MD-Level but belonging to different MAs) 513 a. If the packet MD-LEVEL is equal to the configured MD-LEVEL 514 of the MEP, then pass to the Opcode de-multiplexer 516 b. If the packet MD-LEVEL is less than the configured MD-LEVEL 517 of the MEP, discard the packet 519 c. If the packet MD-LEVEL is greater than the configured MD- 520 LEVEL of the MEP, then pass on to the next higher MD-LEVEL 521 de-multiplexer, if available. Otherwise, if no such higher MD- 522 LEVEL de-multiplexer exists, then forward the packet as normal 523 data. 525 4. Opcode De-multiplexer compares the opcode in the packet with the 526 supported opcodes 528 a. If Op-code is CCM, LBM, LBR, PTM, PTR, MTVM, MTVR, then pass 529 on to the correct Processor 531 b. If Op-code is Unknown, then discard. 533 | 534 .CCM LBM PTM MTV LT 535 | | | | | 536 +-+-+-+-+-+-+-+-+-+-+-+-+ 537 | OP Code DE-Mux |--- Unknown 538 +-+-+-+-+-+-+-+-+-+-+-+-+ 539 ^ ^ ^ 540 MD==Li | | | 541 +-+-+ +-+-+ +-+-+ 542 | L |-->|L2 |-.- |Ln |---- > 543 +-+-+ +-+-+ +-+-+ | 544 | ^ | | | 545 MD
  • | T |----------------- >| M |--- > 552 + NVO3 OAM ---- + pass through OAM ---- 554 Figure 7 OAM De-Multiplexers at MEP for active SAP 556 Default MEPs are assumed to be created at NVE to handle OAM 557 functionality as per Appendix A. 559 T : Denotes Tap, that identifies OAM frames that need local 560 processing. These are the packets with OAM flag set AND OAM 561 Ether type is present after the flow entropy of the packet 563 M : Is the post processing merge, merges data and OAM messages 564 that are passed through. Additionally, the Merge component 565 ensures, as explained earlier, that OAM packets are not 566 forwarded out as native frames. 568 L : Denotes MD-Level processing. Packets with MD-Level less than 569 the Level will be dropped. Packets with equal MD-Level are 570 passed on to the opcode de-multiplexer. Others are passed on to 571 the next level MD processors or eventually to the merge point 572 (M). 574 NOTE: LBM, MTV and PT are not subject to MA de-multiplexers. 575 These packets do not have an MA encoded in the packet. Adequate 576 response can be generated to these packets, without loss of 577 functionality, by any of the MEPs. 579 6.1. Use of MIP in NV03 581 Maintenance Intermediate Points (MIP) are mainly used for fault 582 isolation. Link Trace Messages in [8021Q] utilize a well-known 583 multicast MAC address and MIPs generate responses to Link Trace 584 messages. Response to Link Trace messages or lack thereof can be 585 used for fault isolation in NVO3. 587 As explained in section 10. below, a TTL expiry approach will be 588 utilized for fault isolation and path tracing. The approach is very 589 similar to the well-known IP trace-route approach. Hence, explicit 590 addressing of MIPs is not required for the purpose of fault 591 isolation. 593 Any given NVO3 device can have multiple MIPs located within the 594 device. As such, a mechanism is required to identify which MIP 595 should respond to an incoming OAM message. 597 A similar approach to that presented above for MEPs can be used for 598 MIP processing. It is important to note that "M", the merge block of 599 a MIP, does not prevent OAM packets leaking out as native frames. On 600 edge interfaces, MEPs MUST be configured to prevent the leaking of 601 overlay OAM packets out of the NVO3 domain. 603 LT PT MTV 604 | | | 605 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 606 | OP Code De-Mux |->Unknown 607 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 608 ^ ^ ^ 609 MD==Li | | | 610 +-+-+ +-+-+ +-+-+ 611 | L |- >|L2 |-.- |Ln |------+ 612 +-+-+ +-+-+ +-+-+ | 613 ^ | 614 | | 615 Drop | | 616 MD not --- |NVO3 OAM | 617 Present | | 618 | v 619 NVO3 Data ---- NVO3 Data ----- 620 ------- >| T |------------------ >| M |---> 621 + NVO3 OAM ---- ---- 623 Figure 8 OAM De-Multiplexers at MIP for active SAP 625 T: TAP processing for MIP. All packets with OAM flag set are 626 captured. 628 L : MD Level Processing, Packet with matching MD Level are "copied" 629 to the Opcode de-multiplexer and original packet is passed on to the 630 next MD level processor. Other packets are simply passed on to the 631 next MD level processor, without copying to the OP code de- 632 multiplexer. 634 M : Merge processor, merge OAM packets to be forwarded along with 635 the data flow. 637 Packets that carry Path Trace Message (PTM) or Multi-destination 638 Tree Verification (MTV) OpCodes are passed on to the respective 639 processors. 641 Packets with unknown OpCodes are counted and discarded. 643 7. Continuity Check Message (CCM) 645 CCMs are used to monitor connectivity and configuration errors. 646 [8021Q] monitors connectivity by having a MEP listening to periodic 647 CCM messages received from its remote MEP partners in the MA. An 648 [8021Q] MEP identifies cross-connect errors by comparing the MAID in 649 the received CCM message with the MEP's local MAID. The MAID [8021Q] 650 is a 48-byte field that is technology independent. Similarly, the 651 MEPID is a 2-byte field that is independent of the technology. Given 652 this generic definition of CCM fields, CCM as defined in [8021Q] can 653 be utilized in NVO3 with no changes. NVO3 specific information may 654 be carried in CCMs when encoded using IETF overlay specific TLVs or 655 sub- TLVs. This is possible since CCMs are capable of carrying 656 optional TLVs. 658 Unlike classical Ethernet environments with spanning tree, NVO3 659 supports multipath forwarding. The path taken by a packet depends on 660 the Transport header and other parts of the packet. The Maintenance 661 Association identifies the interested end-points (MEPs) of a given 662 monitored path. For unicast there are only two MEPs per MA. For 663 multicast there can be two or more MEPs in the MA. The Flow (i.e VNI 664 and other parameters) values of the monitored flows are defined 665 within the MA. CCM transmit logic will utilize these flow entropy 666 values when constructing the CCM packets. Please see below for the 667 theory of operation of CCM. 669 The CFM MIB of [8021Q] will be augmented with the definition of 670 flow- entropy. Please see [TBDMIBD] for these and other NVO3 related 671 OAM MIB definitions. The figure below depicts the correlation 672 between MA, CCM and the flow. 674 CCM implementation is Optional. 676 MA--- 677 | 678 --- MEP 679 | 680 . - Remote MEP List 681 . 682 | 683 --- MEP-A 684 | 685 --- MEP-B 686 . 688 | 689 . - Flow List {Augments IEEE8021-CFM-MIB} 691 | 692 --- (Flow-1) {note we have to define 693 | destination NVE with 694 --- (Flow-2) the flow entropy discuss} 695 | 696 . ---(Flow-n) 697 | 698 . - CCM 699 | 700 --- (standard 8021ag entries) 701 | 702 --- (TTL) { Augments IEEE8021-CFM-MIB} 703 | 704 --- (Other TBD NVO3 OAM specific entries) 705 {Augmented} 706 | 707 . 708 | 709 - Other MIB entries 711 Figure 9 Augmentation of CCM MIB in NVO3 713 NOTE: Flow entropy field contain VNI and flow specific information 714 that affect the path selection and forwarding. 716 In a multi-pathing environment, a Flow - by definition - is 717 unidirectional. A question may arise as to what flow entropy should 718 be used in the response. CCMs are unidirectional and have no 719 explicit reply; as such, the issue of the response flow entropy does 720 not arise. In the transmitted CCM, each MEP reports local status 721 using the Remote Defect Indication (RDI) flag. Additionally, a MEP 722 may raise SNMP TRAPs [TBDMIB] as Alarms when a connectivity failure 723 occurs. 725 8. NVO3 OAM Message Channel 727 The NVO3 OAM Message Channel can be divided into two parts: NVO3 OAM 728 Message header and NVO3 OAM Message TLVs. Every OAM Message MUST 729 contain a single OAM message header and a set of one or more 730 specified OAM Message TLVs. 732 8.1. NVO3 OAM Message header 734 As discussed earlier, a common messaging framework between [8021Q], 735 NVO3, and other similar standards such as Y.1731 can be accomplished 736 by re-using the OAM message header defined in [8021Q]. 738 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 739 |MD-L | Version | OpCode | Flags |FirstTLVOffset | 740 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 741 | | 742 | | 743 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 744 | | 745 | | 746 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 748 Figure 10 OAM Message Format 750 o MD-L: Maintenance Domain Level (3 bits). Identifies the 751 maintenance domain level. For NVO3, in general, this field is 752 set to a single value. If using Base Mode operations as defined 753 in Appendix B, this field is set to 3. However, future 754 extensions of NVO3, for example to support hierarchy, may create 755 different MD-LEVELs and MD-L field must be appropriately set in 756 those scenarios. (Please refer to [8021Q] for the definition of 757 MD-Level) o Version: Indicates the version (5 bits), as specified 758 in [8021Q]. This document does not require changing the Version 759 defined in [8021Q]. 761 o Flags: Includes operational flags (1 byte). The definition of 762 flags is Opcode-specific and is covered in the applicable 763 sections. 765 o FirstTLVOffset: Defines the location of the first TLV, in 766 bytes, starting from the end of the FirstTLVOffset field (1 767 byte). (Refer to [8021Q] for the definition of the 768 FirstTLVOffset.) 770 MD-L, Version, Opcode, Flags and FirstTLVOffset fields collectively 771 are referred to as the OAM Message Header. 773 The Opcode specific information section of the OAM Message may 774 contain Session Identification number, time-stamp, etc. 776 8.2. IETF Overlay OAM Opcodes 778 The following Opcodes are defined for IETF Overlay OAM. Each of the 779 Opcodes indicates a separate type of OAM message. Details of the 780 messages are presented in the related sections. 782 IETF OAM Message Opcodes: 784 64 : Path Trace Reply 65 : Path Trace Message 66 : Multicast Tree 785 Verification Reply 67 : Multicast Tree Verification Message 787 8.3. Format of IETF Overlay OAM TLV 789 The same TLV format as defined in section 21.5.1 of [8021Q] is used 790 for IETF Overlay OAM. The following figure depicts the general 791 format of a IETF Overlay OAM TLV: 793 0 1 2 794 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 795 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 796 | Type | Length | 797 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 798 | | 799 | | 800 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 802 Figure 11 NVO3 OAM TLV 804 Type (1 octet): Specifies the Type of the TLV (see sections 8.4. 805 for TLV types). 807 Length (2 octets): Specifies the length of the 'Value' field in 808 octets. Length of the 'Value' field can be either zero or more 809 octets. 811 Value (variable): The length and the content of this field depend on 812 the type of the TLV. Please refer to applicable TLV definitions for 813 the details. 815 Semantics and usage of Type values allocated for TRILL OAM purpose 816 are defined by this document and other future related documents. 818 8.4. IETF Overlay OAM TLVs 820 Overlay related TLVs are defined in this section. Previously defined 821 [8021Q] TLVs are reused, where applicable. Block of 32 TLVs are 822 allocated for the purpose of IETF defined standards 824 8.4.1. Common TLVs between 8201Q CFM and IETF Overlay OAM 826 The following TLVs are defined in [8021Q]. We propose to re-use them 827 where applicable. The format and semantics of the TLVs are as 828 defined in [8021Q]. 830 Type Name of TLV in [8021Q] 831 ---- ------------- 832 0 End TLV 833 1 Sender ID TLV 834 2 Port Status TLV 835 3 Data TLV 836 4 Interface Status TLV 837 5 Reply Ingress TLV 838 6 Reply Egress TLV 839 7 LTM Egress Identifier TLV 840 8 LTR Egress Identifier TLV 841 9-30 Reserved 842 31 Organization Specific TLV 844 8.4.2. IETF Overaly OAM Specific TLVs 846 As indicated above, a block of 32 TLVs will be requested to be 847 reserved for IETF OAM purposes. Listed below is a summary of the 848 IETF Overlay OAM TLVs and their corresponding codes. Format and 849 semantics of OAM TLVs are defined in subsequent sections. 851 Type TLV Name 852 ----------- ---------------------- 853 64 OAM Application Identifier 854 65 Out of Band IP Address 855 66 Original Payload 856 67 Diagnostic VLAN 857 68 scope 858 69 Previous Device address 859 70 Next Hop Device List (ECMP) 860 71 Multicast Receiver Availability 861 72 Flow Identifier 862 73 Reflector Entropy 863 74 to 95 Reserved 865 8.4.3. OAM Application Identifier TLV 867 OAM Application Identifier TLV carries Overlay OAM application 868 specific information. The Overlay OAM Application Identifier TLV 869 MUST always be present and MUST be the first TLV in the OAM 870 messages. Messages that do not include the OAM Application 871 Identifier TLV as the first TLV MUST be discarded. 873 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 874 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 875 | Type | Length | Version | 876 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 877 | Return Code |Return sub-code| Protocol |Rsvd |F|C|O|I| 878 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 880 Figure 12 OAM Application Identifier TLV 882 Type (1 octet) = TBD-TLV-64 indicate that this is the IETF Overlay 883 OAM Application Identifier TLV 885 Length (2 octets) = 6 887 IETF Overlay OAM Version (1 Octet), currently set to zero. Indicates 888 the Overlay OAM version. IETF Overlay OAM version can be different 889 than the [8021Q] version. 891 Return Code (1 Octet): Set to zero on requests. Set to an 892 appropriate value in response messages. 894 Return sub-code (1 Octet): Return sub-code is set to zero on 895 transmission of request message. Return sub-code identifies 896 categories within a specific Return code. Return sub-code MUST be 897 interpreted within a Return code. 899 Protocol: This indicates the overlay protocol on which the OAM is 900 applied. In this document we cover NVO3 902 0 : TRILL 903 1 : NVO3 905 2 - 255 : reserved 907 F (1 bit): Final flag, when set, indicates this is the last 908 response. 910 C (1 bit): Label error, if set indicates that the label (VLAN) in 911 the flow entropy is different than the label included in the 912 diagnostic TLV. This field is ignored in request messages and MUST 913 only be interpreted in response messages. 915 O (1 bit): If set, indicates, OAM out-of-band response requested. 917 I (1 bit): If set, indicates, OAM in-band response requested. 919 NOTE: When both O and I bits are set to zero, indicates that no 920 response is required (silent mode). User MAY specify both O and I or 921 one of them or none. When both O and I bits are set response is sent 922 both in-band and out-of-band. 924 8.4.4. Out Of Band Reply Address TLV 926 Out of Band Reply Address TLV specifies the address to which an 927 out of band OAM reply message MUST be sent. When O bit in the 928 Application Identifier TLV is not set, Out of Band Reply Address 929 TLV is ignored. 931 1 2 3 932 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 933 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 934 | Type | Length | Address Type | 935 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 936 | Addr Length | | 937 +-+-+-+-+-+-+-+-+ | 938 | | 939 . Reply Address . 940 | | 941 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 943 Figure 13 Out of Band IP Address TLV 945 Type (1 octet) = TBD-TLV-65 947 Length (2 octets) = Variable. Minimum length is 2. 949 Address Type (1 Octet): 0 - IPv4. 1 - IPv6. All other values 950 reserved. 952 Addr Length (1 Octet). 4 - IPv4. 16 - IPv6, 954 Reply Address (variable): Address where the reply needed to be sent. 955 Length depends on the address specification. 957 8.4.5. Diagnostics Label TLV 959 Diagnostic label specifies the VNI which the OAM messages are 960 generated. Receiving device MUST compare the VNI of the NVO3 shim to 961 that of Diagnostic TLV. Label Error Flag in the response MUST be set 962 when the two VLANs do not match. 964 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 965 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 966 | Type | Length | L-Type | 967 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 968 | Reserved | Label(VLAN) | 969 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 971 Figure 14 Diagnostic TLV 973 Type (1 octet) = TBD-TLV-66 indicates that this is the Diagnostic 974 TLV 976 Length (2 octets) = 5 978 L-Type (Label type, 1 octet) 980 0- indicate 802.1Q 12 bit VLAN. 982 2 - 24 bit ISID 984 3 - VNI 24 bits 986 Label (24 bits): Either 12 bit VLAN or 24 bit ISID or 24 bit VNI. 988 Devices should perform Label error checking when Label TLV is not 989 included in the OAM message. 991 8.4.6. Original Data Payload TLV 993 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 994 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 995 | Type | Length | | 996 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 997 | | 998 | | 999 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1001 Figure 15 Original Data Payload TLV 1003 Type (1 Octet) = TBD-TLV-67 1005 Length (2 octets) = variable 1007 8.4.7. Flow Identifier (flow-id) TLV 1009 Flow Identifier (flow-id) uniquely identifies a specific flow. The 1010 flow-id value is unique per MEP and needs to be interpreted as such. 1012 1 2 3 1013 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1014 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1015 | Type | Length | Reserved | 1016 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1017 | MEP-ID | flow-id | 1018 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1020 Figure 16 Flow Identifier TLV 1022 Type (1 octet) = TBD-TLV-72 Length (2 octets) = 5. 1024 Reserved (1 octet) set to 0 on transmission and ignored on 1025 reception. 1027 MEP-ID (2 octets) = MEP-ID of the originator [8021Q]. 1029 Flow-id (2 octets) = uniquely identifies the flow per MEP. Different 1030 MEPs may allocate the same flow-id value. The {MEP-ID, flow-id} pair 1031 is globally unique. 1033 Inclusion of the MEP-ID in the flow-id TLV allows inclusion of MEP- 1034 ID for messages that do not contain MEP-ID in OAM header. 1035 Applications may use MEP-ID information for different types of 1036 troubleshooting. 1038 8.4.8. Reflector Entropy TLV 1040 Reflector Entropy TLV is an optional TLV. This TLV, when present, 1041 tells the responder to utilize the Reflector Entropy specified 1042 within the TLV as the flow-entropy of the response message. 1044 1 2 3 1045 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1046 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1047 | Type | Length | Reserved | 1048 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1049 | | 1050 . Reflector Entropy . 1051 | | 1052 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1054 Figure 17 Reflector Entropy TLV 1056 Type (1 octet) =TBD-TLV-73 Reflector Entropy TLV. 1058 Length (1 octet) =97. 1060 Reserved (1 octet) = set to zero on transmission and ignored by the 1061 recipient. 1063 Reflector Entropy (96-octet) = Flow Entropy to be used by the 1064 responder. May be padded with zero if the desired flow entropy is 1065 less than 96 octets. 1067 9. Loopback Message 1069 9.1. Loopback OAM Message format 1071 1 2 3 1072 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1073 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1074 |MD-L | Version | OpCode | Flags |FirstTLVOffset | 1075 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1076 | Loopback Transaction Identifier | 1077 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1078 | | 1079 . TLVs . 1080 | | 1081 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1083 Figure 18 Loopback OAM Message Format 1085 The above figure depicts the format of the Loopback Request and 1086 response messages as defined in [8021Q]. The Opcode for Loopback 1087 Message is set to 65 and the Opcode for the Reply Message is set to 1088 64. The Session Identification Number is a 32-bit integer that 1089 allows the requesting Device to uniquely identify the corresponding 1090 session. Responding Device, without modification, MUST echo the 1091 received "Loopback Transaction Identifier" number.. 1093 9.2. Theory of Operation 1095 9.2.1. Actions by Originator 1097 Identifies the destination NVE address based on user specification 1098 or based on the specified destination, VNI and MAC 1100 Constructs the flow entropy based on user specified parameters or 1101 implementation specific default parameters. 1103 Constructs the OAM header: sets the opcode to Loopback message type 1104 (3). Assign applicable Loopback Transaction Identifier number for 1105 the request. 1107 The Overlay OAM Version TLV MUST be included and with the flags set 1108 to applicable values. 1110 Include following OAM TLVs, where applicable 1112 o Out-of-band Reply address TLV 1114 o Diagnostic Label TLV 1116 o Sender ID TLV 1118 Specify the Transport Layer parameters based on user inputs or 1119 default settings. 1121 Dispatch the OAM frame for transmission. 1123 Devices may continue to retransmit the request at periodic 1124 intervals, until a response is received or the re-transmission count 1125 expires. At each transmission Session Identification number MUST be 1126 incremented. 1128 9.2.2. Intermediate Devices 1130 Intermediate Devices forward the frame as a normal data frame and no 1131 special handling is required. 1133 9.2.3. Destination Device 1135 If the Loopback message is addressed to the local Device and 1136 satisfies the OAM identification criteria specified in section 4.2. 1137 then, the Device data plane forwards the message to the CPU for 1138 further processing. 1140 The OAM application layer further validates the received OAM frame 1141 by checking for the presence of OAM-Ethertype and the MD Level. 1142 Frames that do not contain OAM-Ethertype MUST be discarded. 1144 Construction of the OAM response: 1146 OAM application encodes the received Transport header, NVO3 shim and 1147 payload fragment (when present) in the Original payload TLV and 1148 includes it in the OAM message. 1150 Set the Return Code and Return sub code to applicable values. Update 1151 the OAM opcode to 2 (Loopback Message Reply) 1153 Optionally, if the VNI identifier value of the received differs from 1154 the value specified in the diagnostic Label, set the Label Error 1155 Flag on OAM Application Identifier TLV. 1157 Include the sender ID TLV (1) 1159 If in-band response was requested, dispatch the frame to theNVO3 1160 data plane with request-originator Transport Layer address as the 1161 destination address. 1163 If out-of-band response was requested, dispatch the frame to the IP 1164 forwarding process. 1166 10. Path Trace Message 1168 The primary use of the Path Trace Message is for fault isolation. It 1169 may also be used for plotting the path taken from a given Device to 1170 another Device. 1172 [8021Q] accomplishes the objectives of the NVO3 Path Trace Message 1173 using Link Trace Messages. Link Trace Messages utilize a well-known 1174 multicast MAC address. However, in NOV3 the transport Layer can be 1175 different technologies such as IP or MPLS etc. Hence, a definition 1176 of a new Path Trace message format is required for Overlay OAM. The 1177 Path Trace message is defined for the purpose. 1179 The Path Trace Message has the same format as Loopback Message, but 1180 utilizes a different opcode set. Opcode for Path Trace Reply Message 1181 is 64 and Request Message is 65 1183 Operation of the Path Trace message is identical to the Loopback 1184 message except that it is first transmitted with a TTL field value 1185 of 1. The sending device expects a Time Expiry Return-Code from the 1186 next hop or a successful response. If a Time Expiry Return-code is 1187 received as the response, the originator Device records the 1188 information received from intermediate node that generated the Time 1189 Expiry message and resends the message by incrementing the previous 1190 TTL value by 1. This process is continued until, a response is 1191 received from the destination Device or Path Trace process timeout 1192 occur or TTL reaches a configured maximum value. 1194 10.1. Theory of Operation 1196 10.1.1. Action by Originator Device Identifies the destination NVE 1197 address based on user specification or based on the specified 1198 destination, VNI and MAC 1200 Constructs the NVO3 shim and the flow based on user specified 1201 parameters or implementation specific default parameters. 1203 Construct the OAM header: Set the opcode to Path Trace Request 1204 message type (TBD-65). Assign an applicable Session Identification 1205 number for the request. Return-code and sub-code MUST be set to 1206 zero. 1208 The OAM Application Identifier TLV MUST be included and set the 1209 flags to applicable values. 1211 Include following OAM TLVs, where applicable 1213 o Out-of-band IP address TLV 1215 o Diagnostic Label TLV 1217 o Include the Sender ID TLV 1219 Specify the TTL of the transport header as 1 for the first request. 1221 Dispatch the OAM frame to the data plane for transmission. 1223 A Device (MEP) may continue to retransmit the request at periodic 1224 intervals, until a response is received or the re-transmission count 1225 expires. At each new re-transmission, the Session Identification 1226 number MUST be incremented. Additionally, for responses received 1227 from intermediate devices (MIP), the device address and interface 1228 information MUST be recorded. 1230 10.1.2. Intermediate Device 1232 Path Trace Messages transit through Intermediate devices 1233 transparently, unless Hop-count has expired. OAM application layer 1234 further validates the received OAM frame by examining the presence 1235 of NVO3 OAM Flag and OAM-Ethertype and by examining the MD Level. 1236 Frames that do not contain OAM-Ethertype MUST be discarded. 1238 Construction of the OAM response: 1240 OAM application encodes the received Transport header and flow 1241 entropy in the Original payload TLV and include it in the OAM 1242 message. 1244 Set the Return Code to (2) "Time Expired" and Return sub code to 1245 zero (0). Update the OAM opcode to 64 (Path Trace Message Reply). 1247 If the VNI identifier value of the received OAM message differs from 1248 the value specified in the diagnostic Label, set the Label Error 1249 Flag on OAM Application Identifier TLV. 1251 Include following TLVs 1253 Reply Ingress TLV (5) 1255 Reply Egress TLV (6) 1257 Interface Status TLV (4) 1259 Sender ID TLV (1) 1261 If Label error detected, set C flag (Label error detected) in the 1262 version. 1264 If in-band response was requested, dispatch the frame to the NVO3 1265 data plane with request-originator Transport address as the 1266 destination address. 1268 If out-of-band response was requested, dispatch the frame to the 1269 standard IP forwarding process. 1271 10.1.3. Destination Device 1272 Processing is identical to Loop back response processing in section 1273 9.2.3. with the exception that OAM Opcode is set to Path Trace Reply 1274 (64). 1276 11. Link Trace Message 1278 Link Trace Message (LTM/LTR) procedure is defined in detail in 1279 [8021Q]. In this section I am covering the summary. 1281 Link Trace Message are to be used when operator network all switches 1282 are OAM capable. In this scenario we will recommend 8021Q port based 1283 model for Link Trace Message and Reply Procedure as Bridge Brain is 1284 same as Path Trace message. 1286 SS1 SS2 (L3 VTEP) 1287 / | \/ \ 1288 / | /\ \ (ECMP paths) 1289 / |/ \ \ 1290 S1 S2 S3 1291 || || || (Underlay ECMP) 1292 || || || 1293 L1 L2 L3 (L2 VTEP) 1295 | | 1296 H1 H2 1298 Figure 19 Payload Fragment use case 1300 In Figure 19, if all switches in operator network are OAM capable 1301 then hardware friendly and accurate OAM solution for Fault isolation 1302 will be Link Trace. 1304 11.1 MEP and MIP 1306 By default if network is OAM capable all switch has virtual default 1307 MEP created on the NVE and MIP created on all fabric facing 1308 interface. 1310 MEP can initiate and terminate the OAM message. MIP is stateless and 1311 passive and only Reply to OAM Message. 1313 11.2 Initiator 1315 Initiator Device generate Link Trace Message as per procedure 1316 described in [8021Q] and encapsulated as per the draft. 1318 As transit packet will hit the MIP on the egress port, a copy of 1319 packet is punted to cpu to generate Link Trace Reply (LTR) to the 1320 originating MEP and packet continue forwarding in hardware. 1322 11.3 Intermediate Devices 1324 Intermediate devices has MIP configured on all the fabric 1325 Interfaces. 1327 |-------------| 1328 . . 1329 | | 1330 MIP MIP 1331 ->--- Device --->- 1332 | 1333 |-------------| 1335 Figure 20 MIP configuration and traffic Flow 1337 In above figure traffic is entering from left side of device hitting 1338 first MIP and copy of packet is punted to cpu to generate Link Trace 1339 Reply and LTM continue forwarding in hardware. 1341 LTM hits right side MIP while exiting out of the device and copy of 1342 the packet is punted to cpu to generate Link Trace Reply and LTM 1343 continue forwarding. 1345 11.4 Terminating Device 1347 Terminating device has MIP and MEP configured, and on ingress 1348 interface packet will hit the MIP and copy of it's punted to 1349 generate LTR and packet continue forwarding in hardware. 1351 Packet then hits the terminating MEP and LTR is generated. 1353 11.5 Output 1355 If for example path was like below 1357 A Eth1 -- Eth2 B Eth3 --- Eth4 C 1359 Link trace Message is generated from A towards C. 1361 LTR will be received from 1362 Eth1 A 1363 Eth2 B 1364 Eth3 B 1365 Eth4 C 1366 MEP C 1368 In this way complete path is tracked in hardware forwarding. 1370 12. Application of Continuity Check Message (CCM) in NVO3 1372 Section 7. provides an overview of CCM Messages defined in [8021Q] 1373 and how they can be used within the NVO3 OAM. This section, presents 1374 the application and Theory of Operations of CCM within the NVO3 OAM 1375 framework. Readers are referred to [8021Q] for CCM message format 1376 and applicable TLV definitions and usages. Only the NVO3 specific 1377 aspects are explained below. 1379 In NVO3, between any two given MEPs there can be multiple potential 1380 paths. Whereas in [8021Q], there is always a single path between any 1381 two MEPs at any given time. It is important that solutions to have 1382 the ability to monitor continuity over one or more paths. 1384 CCM Messages are uni-directional, such that there is no explicit 1385 response to a received CCM message. Connectivity fault status is 1386 indicated by setting the applicable flags (e.g. RDI) of the CCM 1387 messages transmitted by an MEP. 1389 It is important that the solution presented in this document 1390 accomplishes the requirements specified in [NVO3OAMREQ] within the 1391 framework of [8021Q] in a straightforward manner and with minimum 1392 changes. Section 8 above defines multiple flows within the CCM 1393 object, each corresponding to a flow that a given MEP wishes to 1394 monitor. 1396 Receiving MEPs do not cross check whether a received CCM belongs to 1397 a specific flow from the originating MEP. Any attempt to track 1398 status of individual flows may explode the amount of state 1399 information that any given MEP has to maintain. 1401 The obvious question arises: How does the originating device know 1402 which flow or flows are at fault? 1404 This is accomplished with a combination of the RDI flag in the CCM 1405 header, flow-id TLV, and SNMP Notifications (Traps). Section 11.1 1406 below discusses the procedure. 1408 12.1. CCM Error Notification 1410 Each MEP transmits 4 CCM messages per each flow. ([8021Q] detects 1411 CCM fault when 3 consecutive CCM messages are lost). Each CCM 1412 Message has a unique sequence number and unique flow-identifier. The 1413 flow identifier is included in the OAM message via flow-id TLV. 1415 When an MEP notices a CCM timeout from a remote MEP ( MEP-A), it 1416 sets the RDI flag on the next CCM message it generates. 1417 Additionally, it logs and sends SNMP notification that contain the 1418 remote MEP Identification, flow-id and the Sequence Number of the 1419 last CCM message it received and if available, the flow-id and the 1420 Sequence Number of the first CCM message it received after the 1421 failure. Each MEP maintains a unique flow-id per each flow, hence 1422 the operator can easily identify flows that correspond to the 1423 specific flow-id. 1425 The following example illustrates the above. 1427 Assume there are two MEPs, MEP-A and MEP-B. 1429 Assume there are 3 flows between MEP-A and MEP-B. 1431 Let's assume MEP-A allocates sequence numbers as follows 1433 Flow-1 Sequence={1,2,3,4,13,14,15,16,.. } flow-id=(1) 1435 Flow-2 Sequence={5,6,7,8,17,18,19,20,.. } flow-id=(2) 1437 Flow-3 Sequence={9,10,12,11,21,22,23,24,.. } flow-id=(3) 1439 Let's Assume Flow-2 is at fault. 1441 MEP-B, receives CCM from MEP-A with sequence numbers 1,2,3,4, but 1442 did not receive 5,6,7,8. CCM timeout is set to 3 CCM intervals in 1443 [8021Q]. Hence MEP-B detects the error at the 8'th CCM message. At 1444 this time the sequence number of the last good CCM message MEP-B has 1445 received from MEP-A is 4 and flow-id of the last good CCM Message is 1446 (1). Hence MEP-B will generate a CCM error SNMP notification with 1447 MEP-A and Last good flow-id (1) and sequence number 4. 1449 When MEP-A switches to flow-3 after transmitting flow-2, MEP-B will 1450 start receiving CCM messages. In the foregoing example it will be 1451 CCM message with Sequence Numbers 9,10,11,12,21 and so on. When in 1452 receipt of a new CCM message from a specific MEP, after a CCM 1453 timeout, the NVO3 OAM will generate an SNMP Notification of CCM 1454 resume with remote MEP-ID and the first valid flow-id and the 1455 Sequence number after the CCM timeout. In the foregoing example, it 1456 is MEP-A, flow-id (3) and Sequence Number 9. 1458 The remote MEP list under the CCM MIB Object is augmented to contain 1459 "Last Sequence Number", flow-id and "CCM Timeout" variables. Last 1460 Sequence Number and flow-id are updated every time a CCM is received 1461 from a remote MEP. CCM Timeout variable is set when the CCM timeout 1462 occurs and is cleared when a CCM is received. 1464 12.2. Theory of Operation 1466 12.2.1. Actions by Originator Device 1468 Derive the VNI and entropy based on flow entropy specified in the 1469 CCM Management object. 1471 Construct the NVO3 CCM OAM header as specified in [8021Q]. 1473 OAM Version TLV MUST be included as the first TLV and set the flags 1474 to applicable values. 1476 Include other TLVs specified in [8021Q] 1478 Include the following optional OAM TLVs, where applicable 1480 o Sender ID TLV 1482 Specify the TTL of the NVO3 data frame per user specification or 1483 utilize an applicable Hop count value. 1485 Dispatch the OAM frame to the NVO3 data plane for transmission. 1487 An MEP transmits a total of 4 requests, each at CCM retransmission 1488 interval. At each transmission the Session Identification number 1489 MUST be incremented by one. 1491 At the 5'th retransmission interval, flow entropy of the CCM packet 1492 is updated to the next flow entropy specified in the CCM Management 1493 Object. If current flow entropy is the last flow entropy specified, 1494 move to the first flow entropy specified and continue the process. 1496 12.2.2. Intermediate Devices 1498 Intermediate devices forward the frame as a normal data frame and 1499 no special handling is required. 1501 12.2.3. Destination Device 1503 If the CCM Message is addressed to the local MEP or multicast and 1504 satisfies OAM identification methods specified in sections 4.2. 1505 then the data plane forwards the message to the CPU for further 1506 processing. 1508 The OAM application layer further validates the received OAM frame 1509 by examining the presence of OAM-Ethertype at the end of the flow 1510 entropy. Frames that do not contain OAM-Ethertype at the end of the 1511 flow entropy MUST be discarded. 1513 Validate the MD-LEVEL and pass the packet to the Opcode de- 1514 multiplexer. The Opcode de-multiplexer delivers CCM packets to the 1515 CCM process. 1517 The CCM Process performs processing specified in [8021Q]. 1519 Additionally the CCM process updates the CCM Management Object with 1520 the sequence number of the received CCM packet. Note: The last 1521 received CCM sequence number and CCM timeout are tracked per each 1522 remote MEP. 1524 If the CCM timeout is true for the sending remote MEP, then clear 1525 the CCM timeout in the CCM Management object and generate the SNMP 1526 notification as specified above. 1528 13. Security Considerations 1530 Specific security considerations related methods presented in this 1531 document are currently under investigation. 1533 14. IANA Considerations 1535 Re-use the Type and TLV from RFC7455. 1537 15. References 1539 15.1. Normative References 1541 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1542 Requirement Levels", BCP 14, RFC 2119, March 1997. 1544 [8021Q] IEEE, "Media Access Control (MAC) Bridges and Virtual 1545 Bridged Local Area Networks", IEEE Std 802.1Q-2011, August, 1546 2011. 1548 15.2. Informative References 1550 [RFC4379] Kompella, K. et.al, "Detecting Multi-Protocol Label 1551 Switched (MPLS) Data Plane Failures", RFC 4379, February 1552 2006. 1554 [RFC6291] Andersson, L., et.al., "Guidelines for the use of the 1555 "OAM" Acronym in the IETF" RFC 6291, June 2011. 1557 [Y1731] ITU, "OAM functions and mechanisms for Ethernet based 1558 networks", ITU-T G.8013/Y.1731, July, 2011. 1560 [NVO3FRM] Lasserre, M., et.al., "Framework for DC Network 1561 Virtualization", Work in Progress, January, 2014. 1563 [NVO3ARC] Black, D., et.al., "Architecture for Overlay Networks 1564 (NVO3)", Work in Progress, December, 2013. 1566 [NVO3OAMREQ] Ashwood-Smith, P., "NVO3 Operations, Administrations 1567 and Maintenance Requirements", work in progress, January, 1568 2014. 1570 [NV03DPREQ] Bitar, N., "NVO3 Data Plane Requirements", Work in 1571 Progress, November, 2013. 1573 16. Acknowledgments 1575 This document was prepared using 2-Word-v2.0.template.dot. 1577 Appendix A. Base Mode for NVO3 OAM 1579 CFM, as defined in [8021Q], requires configuration of several 1580 parameters before the protocol can be used. These parameters include 1581 MAID, Maintenance Domain Level (MD-LEVEL) and MEPIDs. The Base Mode 1582 for NVO3 OAM defined here facilitates ease of use and provides out 1583 of the box plug-and-play capabilities. 1585 All NVO3 compliant devices that support OAM specified in this 1586 document MUST support Base Mode operation. 1588 All NVO3 compliant devices that support OAM MUST create a default MA 1589 with MAID as specified herein. 1591 MAID [8021Q] has a flexible format and includes two parts: 1592 Maintenance Domain Name and Short MA name. In the Based Mode of 1593 operation, the value of the Maintenance Domain Name must be the 1594 character string "NVO3BaseMode" (excluding the quotes "). In Base 1595 Mode operation Short MA Name format is set to 2-octet integer format 1596 (value 3 in Short MA Format field) and Short MA name set to 65532 1597 (0xFFFC). 1599 The Default MA belongs to MD-LEVEL 3. 1601 In the Base Mode of operation, each NVO3 device creates a single UP 1602 MEP associated with a virtual OAM port located within the CPU with 1603 no physical layer (NULL PHY). The MEPID associated with this MEP is 1604 the 2-octet VNI Nickname. 1606 By default, with the base mode OAM in place, all NVO3 devices 1607 operating in the Base Mode for NVO3 OAM are able to initiate LBM, PT 1608 and other OAM tools with no configuration. 1610 Implementation MAY provide default flow to be included in OAM 1611 messages. Content of the default flow is outside the scope of this 1612 document. 1614 Figure 18, below depicts encoding of MAID within the CCM messages. 1616 +-+-+-+-+-+-+-+-+-+-+-+-+-+ 1617 |Field Name |Size | 1618 | | | 1619 +-+-+-+-+-+-+-+-+-+-+-+-+-+ 1620 |Maintenance | 1 | 1621 |Domain Format | | 1622 +-+-+-+-+-+-+-+-+-+-+-+-+-+ 1623 |Maintenance | 2 | 1624 |Domain Length | | 1625 +-+-+-+-+-+-+-+-+-+-+-+-+-+ 1626 |Maintenance | variable| 1627 |Domain Name | | 1628 +-+-+-+-+-+-+-+-+-+-+-+-+-+ 1629 |Short MA | 1 | 1630 |Name Format | | 1631 +-+-+-+-+-+-+-+-+-+-+-+-+-+ 1632 |Short MA | 2 | 1633 |Name Length | | 1634 +-+-+-+-+-+-+-+-+-+-+-+-+-+ 1635 |Short MA | variable| 1636 |Name | | 1637 +-+-+-+-+-+-+-+-+-+-+-+-+-+ 1638 |Padding | Variable| 1639 +-+-+-+-+-+-+-+-+-+-+-+-+-+ 1641 Figure 18 MAID structure as defined in [8021Q] 1643 Maintenance Domain Name Format is set to Value: 4 1645 Maintenance Domain Name Length is set to value: 13 1647 Maintenance Domain Name is set to: NVO3BaseMode 1649 Short MA Name Format is set to value: 3 1651 Short MA Name Length is set to value: 2 1653 Shirt MA Name is set to : FFFC 1655 Padding : set of zero up to 48 octets of total length of the MAID. 1657 Please refer to [8021Q] for details. 1659 Authors' Addresses 1660 Tissa Senevirathne 1661 CISCO Systems 1662 375 East Tasman Drive. 1663 San Jose, CA 95134 1664 USA. 1666 Phone: +1 408-853-2291 1667 Email: tsenevir@gmail.com 1669 Norman Finn 1670 CISCO Systems 1671 510 McCarthy Blvd 1672 Milpitas, CA 95035 1673 USA 1675 Email: nfinn@cisco.com 1677 Samer Salam 1678 CISCO Systems 1679 595 Burrard St. Suite 2123 1680 Vancouver, BC V7X 1J1, Canada 1682 Email: ssalam@cisco.com 1684 Deepak Kumar 1685 CISCO Systems 1686 510 McCarthy Blvd, 1687 Milpitas, CA 95035, USA 1689 Phone : +1 408-853-9760 1690 Email: dekumar@cisco.com 1692 Donald Eastlake 1693 Huawei Technologies 1694 155 Beaver Street 1695 Milford, MA 01757 1697 Phone: +1-508-333-2270 1698 Email: d3e3e3@gmail.com 1700 Sam Aldrin 1701 Huawei Technologies 1702 2330 Central Express Way 1703 Santa Clara, CA 95951 1704 USA 1706 Email: aldrin.ietf@gmail.com