idnits 2.17.1 draft-jain-nvo3-overlay-oam-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 113 instances of too long lines in the document, the longest one being 9 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: Inner Ethernet Header for the Echo Request Packet MUST have the Destination Mac set to 00-00-5E-90-XX-XX (to be assigned IANA). The Source Mac should be set to Mac Address of the Originating VTEP. However, it is desired that the Inner Source Mac SHOULD not be learnt in the MAC-Table as this represent Control Packet in context of Overlay OAM. == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: Since the VxLAN Router Alert bit is set in VxLAN Header, which signifies the presence of Control Packet. The terminating VTEP SHOULD not learn the Mac address set in the Inner Mac Header of VxLAN Echo Request Packet. == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: Since the NVGRE Router Alert bit is set in NVGRE Header, which signifies the presence of Control Packet. The Terminating Overlay End Point SHOULD not learn the Mac address set in the Inner Mac Header of NVGRE Echo Request Packet. == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: The scope of End-System Ping is solely with the Cloud Provider which owns control of the Overlay End Point(s). It is expected that the Overlay End Point traps this request and checks the Presence of the End-System via its MAC Address, Route or Flow information and replies back. There SHOULD not be a case where the End-System Ping is delivered to the actual End-Point. -- The document date (February 12, 2014) is 3719 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'NTP' is mentioned on line 395, but not defined == Unused Reference: 'I-D.draft-lasserre-nvo3-framework' is defined on line 1311, but no explicit reference was found in the text == Unused Reference: 'RFC4379' is defined on line 1347, but no explicit reference was found in the text == Unused Reference: 'RFC4330' is defined on line 1356, but no explicit reference was found in the text ** Downref: Normative reference to an Informational draft: draft-lasserre-nvo3-framework (ref. 'I-D.draft-lasserre-nvo3-framework') ** Downref: Normative reference to an Informational draft: draft-mahalingam-dutt-dcops-vxlan (ref. 'I-D.draft-mahalingam-dutt-dcops-vxlan') ** Downref: Normative reference to an Informational draft: draft-sridharan-virtualization-nvgre (ref. 'I-D.draft-sridharan-virtualization-nvgre') ** Downref: Normative reference to an Informational RFC: RFC 4365 ** Obsolete normative reference: RFC 4379 (Obsoleted by RFC 8029) -- Obsolete informational reference (is this intentional?): RFC 4330 (Obsoleted by RFC 5905) Summary: 6 errors (**), 0 flaws (~~), 9 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 NVO3 P. Jain 3 Internet-Draft K. Singh 4 Intended status: Standards Track F. Balus 5 Expires: August 16, 2014 Nuage Networks 6 W. Henderickx 7 Alcatel-Lucent 8 V. Bannai 9 PayPal 10 R. Shekhar 11 A. Lohiya 12 Juniper Networks 13 February 12, 2014 15 Generic Overlay OAM and Datapath Failure Detection 16 draft-jain-nvo3-overlay-oam-01 18 Abstract 20 This proposal describes a mechanism that can be used to detect Data 21 Path Failures of various overlay technologies as VXLAN, NVGRE, 22 MPLSoGRE and MPLSoUDP and verifying/sanity of their Control and Data 23 Plane for given Overlay Segment. This document defines the following 24 for each of the above Overlay Technologies: 26 o Encapsulation of OAM Packet, such that it has same Outer and 27 Overlay Header as any End-System's data going over the same 28 Overlay Segment. 30 o The mechanism to trace the Underlay that is exercised by any 31 Overlay Segment. 33 o Procedure to verify presence of any given Tenant VM or End-System 34 within a given Overlay Segment at Overlay End-Point. 36 Even though the present proposal addresses Overlay OAM for VXLAN, 37 NVGRE, MPLSoGRE and MPLSoUDP, but the procedures described are 38 generic enough to accommodate OAM for any other Overlay Technology. 40 Status of this Memo 42 This Internet-Draft is submitted in full conformance with the 43 provisions of BCP 78 and BCP 79. 45 Internet-Drafts are working documents of the Internet Engineering 46 Task Force (IETF). Note that other groups may also distribute 47 working documents as Internet-Drafts. The list of current Internet- 48 Drafts is at http://datatracker.ietf.org/drafts/current/. 50 Internet-Drafts are draft documents valid for a maximum of six months 51 and may be updated, replaced, or obsoleted by other documents at any 52 time. It is inappropriate to use Internet-Drafts as reference 53 material or to cite them other than as "work in progress." 55 This Internet-Draft will expire on August 16, 2014. 57 Copyright Notice 59 Copyright (c) 2014 IETF Trust and the persons identified as the 60 document authors. All rights reserved. 62 This document is subject to BCP 78 and the IETF Trust's Legal 63 Provisions Relating to IETF Documents 64 (http://trustee.ietf.org/license-info) in effect on the date of 65 publication of this document. Please review these documents 66 carefully, as they describe your rights and restrictions with respect 67 to this document. Code Components extracted from this document must 68 include Simplified BSD License text as described in Section 4.e of 69 the Trust Legal Provisions and are provided without warranty as 70 described in the Simplified BSD License. 72 Table of Contents 74 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 6 76 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 8 78 3. Motivation for Overlay OAM . . . . . . . . . . . . . . . . . . 9 80 4. Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 82 5. Packet Format . . . . . . . . . . . . . . . . . . . . . . . . 11 83 5.1. Overlay OAM Encapsulation in Layer 2 Context . . . . . . . 11 84 5.2. Overlay OAM Encapsulation in Layer 3 Context . . . . . . . 11 85 5.3. Generic Overlay OAM Packet Format . . . . . . . . . . . . 11 86 5.3.1. TLV Types for various Overlay Ping Models . . . . . . 14 87 5.3.1.1. TLV for VXLAN Ping . . . . . . . . . . . . . . . . 15 88 5.3.1.2. TLV for NVGRE Ping . . . . . . . . . . . . . . . . 16 89 5.3.1.3. TLV for MPLSoGRE Ping . . . . . . . . . . . . . . 17 90 5.3.1.4. TLV for MPLSoUDP Ping . . . . . . . . . . . . . . 18 92 6. Return Codes . . . . . . . . . . . . . . . . . . . . . . . . . 19 94 7. Procedure for Overlay Segment Ping . . . . . . . . . . . . . . 20 95 7.1. Encoding of Inner Header for Echo Request in Layer 2 96 Context . . . . . . . . . . . . . . . . . . . . . . . . . 20 97 7.2. Encoding of Inner Header for Echo Request in Layer 3 98 Context . . . . . . . . . . . . . . . . . . . . . . . . . 21 99 7.3. VXLAN Procedures . . . . . . . . . . . . . . . . . . . . . 21 100 7.3.1. Sending VXLAN Echo Request . . . . . . . . . . . . . . 21 101 7.3.2. Receiving VXLAN Echo Request . . . . . . . . . . . . . 22 102 7.3.3. Sending VXLAN Echo Reply . . . . . . . . . . . . . . . 23 103 7.3.4. Receiving VXLAN Echo Reply . . . . . . . . . . . . . . 23 104 7.4. NVGRE Procedures . . . . . . . . . . . . . . . . . . . . . 23 105 7.4.1. Sending NVGRE Echo Request . . . . . . . . . . . . . . 23 106 7.4.2. Receiving NVGRE Echo Request . . . . . . . . . . . . . 24 107 7.4.3. Sending NVGRE Echo Reply . . . . . . . . . . . . . . . 25 108 7.4.4. Receiving NVGRE Echo Reply . . . . . . . . . . . . . . 25 109 7.5. MPLSoGRE Procedures . . . . . . . . . . . . . . . . . . . 25 110 7.5.1. Sending MPLSoGRE Echo Request . . . . . . . . . . . . 25 111 7.5.2. Receiving MPLSoGRE Echo Request . . . . . . . . . . . 25 112 7.5.3. Sending MPLSoGRE Echo Reply . . . . . . . . . . . . . 26 113 7.5.4. Receiving MPLSoGRE Echo Reply . . . . . . . . . . . . 26 114 7.6. MPLSoUDP Procedures . . . . . . . . . . . . . . . . . . . 26 115 7.6.1. Sending MPLSoUDP Echo Request . . . . . . . . . . . . 26 116 7.6.2. Receiving MPLSoUDP Echo Request . . . . . . . . . . . 27 117 7.6.3. Sending MPLSoUDP Echo Reply . . . . . . . . . . . . . 28 118 7.6.4. Receiving MPLSoUDP Echo Reply . . . . . . . . . . . . 28 120 8. Procedure for Trace . . . . . . . . . . . . . . . . . . . . . 29 122 9. Procedure for End-System Ping . . . . . . . . . . . . . . . . 30 123 9.1. Sub-TLV for End-System Ping . . . . . . . . . . . . . . . 30 124 9.1.1. Sub-TLV for Validating End-System MAC Address . . . . 31 125 9.1.2. Sub-TLV for Validating End-System IP Address . . . . . 32 126 9.1.3. Sub-TLV for Validating End-System MAC and IP 127 Address . . . . . . . . . . . . . . . . . . . . . . . 33 128 9.2. Sending End-System Ping Request . . . . . . . . . . . . . 34 129 9.3. Receiving End-System Ping Request . . . . . . . . . . . . 34 130 9.4. Sending End-System Ping Reply . . . . . . . . . . . . . . 35 131 9.5. Receiving End-System Ping Reply . . . . . . . . . . . . . 35 133 10. Security Considerations . . . . . . . . . . . . . . . . . . . 37 135 11. Management Considerations . . . . . . . . . . . . . . . . . . 38 137 12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 39 139 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 40 141 14. References . . . . . . . . . . . . . . . . . . . . . . . . . . 41 142 14.1. Normative References . . . . . . . . . . . . . . . . . . . 41 143 14.2. Informative References . . . . . . . . . . . . . . . . . . 42 145 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 43 146 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 147 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 148 document are to be interpreted as described in RFC2119 [RFC2119]. 150 When used in lower case, these words convey their typical use in 151 common language, and are not to be interpreted as described in 152 RFC2119 [RFC2119]. 154 1. Introduction 156 VXLAN [I-D.draft-mahalingam-dutt-dcops-vxlan], NVGRE 157 [I-D.draft-sridharan-virtualization-nvgre], MPLSoGRE [RFC4023] and 158 MPLSoUDP [I-D.draft-ietf-mpls-in-udp] are well known technologies and 159 are used as tunneling mechanism to Overlay either Layer 2 networks or 160 Layer 3 networks on top of Layer 3 Underlay networks. For all above 161 Overlay Models there are two Tunnel End Points for a given Overlay 162 Segment. One End Point is where the Overlay Originates, and other 163 where Overlay Terminates. In most cases the Tunnel End Point is 164 intended to be at the edge of the network, typically connecting an 165 access switch to an IP transport network. The access switch could be 166 a physical or a virtual switch located within the hypervisor on the 167 server which is connected to End System which is a VM. 169 This document describes a mechanism that can be used to detect Data 170 Plane failures and sanity of Overlay Control and Data Plane for a 171 given Overlay Segment, and the method to trace the Underlay path that 172 is exercised by any given Overlay Segment. 174 The document also defines procedures for validating the presence of 175 any given Tenant VM/End-System/End-System or Flow representing the 176 End-System System within a given Overlay Segment. 178 The proposal describes: 180 o The mechanism to verify Overlay Control Plane and Data Plane 181 consistency at the Overlay End Point(s), by encapsulating the OAM 182 Packet in exact the same way as that of any End System Traffic 183 that is transported over the Overlay Segment. 185 o The mechanism to trace the Underlay that is exercised by any 186 Overlay Segment. 188 o The mechanism to verify presence of any "End-System" in a given 189 Overlay Segment. 191 The proposal defines the information to check correct operation of 192 the Data Plane, as well as a mechanism to verify the Data Plane 193 against the Control Plane for a given Overlay Segment. 195 It is important consideration in this proposal to carry Echo Request 196 along same Data Path that any End System's data using the given 197 Overlay Segment takes. 199 The tenants VM(s) or End System(s) are not aware of the Overlays and 200 as such the need for the verification of the Data Path MUST solely 201 rest with the Cloud Provider. The use cases where the Tenant VM(s) 202 need to be aware of the Data Plane failures is beyond the scope of 203 this document. 205 2. Terminology 207 Terminology used in this document: 209 OAM: Operations, Administration, and Management 211 VXLAN: Virtual eXtensible Local Area Network. 213 NVGRE: Network Virtualization using GRE. 215 MPLSoGRE: Encapsulating MPLS in IP or Generic Routing Encapsulation 216 (GRE) 218 MPLSoUDP: Encapsulating MPLS in UDP. 220 Originating End Point: Overlay Segment's Head End or Starting Point 221 of Overlay Tunnel. 223 Terminating End Point: Overlay Segment's Tail End or Terminating 224 Point of Overlay Tunnel. 226 VM: Virtual Machine. 228 VNI: VXLAN Network Identifier (or VXLAN Segment ID) 230 VSID: Virtual Subnet ID. (for NVGRE) 232 NVE: Network Virtualized Edge 234 End System: Could be Tenant VM, Host, Bridge etc. - System whose data 235 is expected to go over Overlay Segment. 237 Echo Request: Throughout this document, Echo Request packet is 238 expected to be transmitted by Originator Overlay End Point and 239 destined to Overlay Terminating End Point. 241 Echo Reply: Throughout this document, Echo Reply packet is expected 242 to be transmitted by Terminating Overlay End Point and destined to 243 Overlay Originating End Point. 245 Other terminologies are as defined in 246 [I-D.draft-mahalingam-dutt-dcops-vxlan], 247 [I-D.draft-sridharan-virtualization-nvgre], [RFC4023] and 248 [I-D.draft-ietf-mpls-in-udp] 250 3. Motivation for Overlay OAM 252 When any Overlay Segment fails to deliver user traffic, there is a 253 need to provide a tool that would enable users, as Cloud Providers to 254 detect such failures, and a mechanism to isolate faults. It may also 255 be desirable to test the data path before mapping End System traffic 256 to the Overlay Segment. 258 The basic idea is to facilitate following verifications:- 260 o End-System's data that are expected to go over a particular 261 Overlay Segment actually ends up using the Data-Path represented 262 by given Overlay Segment between the two End-Points. 264 o To verify the correct value of Overlay Segment Identifier is 265 programmed at Originating and Terminating End Point(s) for a given 266 Overlay Segment. Segment Identifier will be VNI for VXLAN, VSID 267 for NVGRE, MPLS Label for MPLSoGRE and MPLSoUDP. 269 o The facilitate mechanism to trace the Underlay that is exercised 270 by any Overlay Segment. 272 o The mechanism to verify presence of any "End-System" in a given 273 Overlay Segment. 275 To facilitate verification of Overlay Segment or any End-System using 276 the Overlay, this document proposes sending of a Packet (called an 277 "Echo Request") along the same data path as other Packets belonging 278 to this Segment. Echo Request also carries information about the 279 Overlay Segment whose Data Path is to be verified. This Echo Request 280 is forwarded just like any other End System Data Packet belonging to 281 that Overlay Segment, as it contains the same Overlay Encapsulation 282 as regular End System's data. 284 On receiving Echo Request at the end of the Overlay Segment, it is 285 sent to the Control Plane of the Terminating Overlay End Point, which 286 in-turn would respond with Echo Reply. 288 To facilitate tracing of the Underlay used by any given Overlay 289 Segment, the document proposes Echo Request/Reply encapsulation in 290 "trace mode", which would allow the user or Cloud Provider to gather 291 information of the Underlay network. 293 4. Approach 295 The proposal aims at validating Data Plane and its view of Control 296 Plane for a particular Overlay Segment. To achieve this aim, the 297 draft proposes creating an Overlay OAM Packet which MUST be 298 encapsulated with the Overlay Header as that of any End-Point data 299 going over the same Overlay Segment. This would guarantee the data- 300 path for OAM Packet follows the same path as that for any End User 301 data going over the same Overlay Segment. 303 The draft outlines procedures to encode Overlay Header and Inner 304 Ethernet or IP Header based on the type of payload that Overlay is 305 expected to carry. 307 5. Packet Format 309 Generic Overlay Echo Request/Reply is a UDP Packet identified by well 310 known UDP Port XXXX. The payload carried by Overlay typically could 311 be either be Layer 2 / Ethernet Frame, or it could be Layer 3 / IP 312 Packet. 314 5.1. Overlay OAM Encapsulation in Layer 2 Context 316 If the encapsulated payload carried by Overlay is of type Ethernet, 317 then the OAM Echo Request packet would have inner Ethernet Header, 318 followed by IP and UDP Header. The payload of inner UDP would be as 319 described in below section "Generic Overlay OAM Packet Format". 321 5.2. Overlay OAM Encapsulation in Layer 3 Context 323 If the encapsulated payload carried by Overlay is of type IP, then 324 the OAM Echo Request packet would have inner IP Header, followed by 325 UDP Header. The payload of inner UDP would be as described in below 326 section "Generic Overlay OAM Packet Format". 328 5.3. Generic Overlay OAM Packet Format 330 Following is the format of UDP payload of Generic Overlay OAM Packet: 332 0 1 2 3 333 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 334 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 335 | Message Type | Reply mode | Return Code | Return Subcode| 336 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 337 | Originator Handle | 338 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 339 | Sequence Number | 340 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 341 | TimeStamp Sent (seconds) | 342 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 343 | TimeStamp Sent (microseconds) | 344 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 345 | TimeStamp Received (seconds) | 346 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 347 | TimeStamp Received (microseconds) | 348 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 349 | TLVs ... | 350 . . 351 . . 352 . . 353 | | 354 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 356 Generic Overlay OAM Packet 358 The Message Type is one of the following:- 360 Value What it means 361 ----- ------------- 362 1 Echo Request 364 2 Echo Reply 366 Reply Mode Values:- 368 Value What it means 369 ----- --------------------------------- 370 1 Do not reply 372 2 Reply via an IPv4/IPv6 UDP Packet 374 3 Reply via Overlay Segment 376 Echo Request with 1 (Do not reply) in the Reply Mode field may be 377 used for one-way connectivity tests. The receiving node may log gaps 378 in the Sequence Numbers and/or maintain delay/jitter statistics. For 379 normal operation Echo Request would have 2 (Reply via an IPv4 UDP 380 Packet) in the Reply Mode field. 382 If it is desired that the reply also comes back via Overlay Segment 383 i.e. encapsulated with the Overlay Header, then the Reply Mode filed 384 needs to be set to 3 (Reply via Overlay Segment). 386 The Originator's Handle is filled in by the Originator, and returned 387 unchanged by the receiver in the Echo Reply (if any). The value used 388 for this field can be implementation dependent, this MAY be used by 389 the Originator for matching up requests with replies. 391 The Sequence Number is assigned by the Originator of Echo Request and 392 can be (for example) used to detect missed replies. 394 The TimeStamp Sent is the time-of-day (in seconds and microseconds, 395 according to the sender's clock) in NTP format [NTP] when the VXLAN 396 Echo Request is sent. The TimeStamp Received in an Echo Reply is the 397 time-of-day (according to the receiver's clock) in NTP format that 398 the corresponding Echo Request was received. 400 TLVs (Type-Length-Value tuples) have the following format: 402 0 1 2 3 403 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 404 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 405 | Type | Length | 406 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 407 | Value | 408 . . 409 . . 410 . . 411 | | 412 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 414 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 415 | Sub-TLV Type | Length | Variable Length Value | 416 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 417 | Variable Length Value | 418 | " | 419 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 421 Types are defined below; Length is the length of the Value field in 422 octets. The Value field depends on the Type; it is zero padded to 423 align to a 4-octet boundary. There could be one or many optional 424 Sub-TLV that could be encoded under the TLV. 426 5.3.1. TLV Types for various Overlay Ping Models 428 TLV Types:- 430 Value What it means 431 ----- ------------------------------ 432 1 VXLAN Segment Ping for IPv4 434 2 VXLAN Segment Ping for IPv6 436 3 NVGRE Segment Ping for IPv4 438 4 NVGRE Segment Ping for IPv6 440 5 MPLSoGRE Segment Ping for IPv4 442 6 MPLSoGRE Segment Ping for IPv6 444 7 MPLSoUDP Segment Ping for IPv4 446 8 MPLSoUDP Segment Ping for IPv6 448 5.3.1.1. TLV for VXLAN Ping 450 0 1 2 3 451 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 452 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 453 | Type = 1(VXLAN ping IPv4)| Length | 454 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 455 | VXLAN VNI | Reserved | 456 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 457 | IPv4 Sender Address | 458 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 460 TLV if Sender Address is IPv4 462 0 1 2 3 463 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 464 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 465 | Type = 2 (VXLAN ping IPv6)| Length | 466 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 467 | VXLAN VNI | Reserved | 468 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 469 | | 470 | IPv6 Sender Address | 471 | | 472 | | 473 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 475 TLV if Sender Address is IPv6 477 5.3.1.2. TLV for NVGRE Ping 479 0 1 2 3 480 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 481 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 482 | Type = 3 (NVGRE ping IPv4)| Length | 483 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 484 | NVGRE VSID | Reserved | 485 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 486 | IPv4 Sender Address | 487 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 489 TLV if Sender Address is IPv4 491 0 1 2 3 492 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 493 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 494 | Type = 4 (NVGRE ping IPv6)| Length | 495 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 496 | NVGRE VSID | Reserved | 497 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 498 | | 499 | IPv6 Sender Address | 500 | | 501 | | 502 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 504 TLV if Sender Address is IPv6 506 5.3.1.3. TLV for MPLSoGRE Ping 508 0 1 2 3 509 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 510 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 511 | Type = 5 (MPLSoGRE ping IPv4)| Length | 512 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 513 | Route Distinguisher | 514 | (8 octets) | 515 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 516 | IPv4 Sender Address | 517 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 518 TLV if Sender Address is IPv4 520 0 1 2 3 521 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 522 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 523 | Type = 6 (MPLSoGRE ping IPv6)| Length | 524 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 525 | Route Distinguisher | 526 | (8 octets) | 527 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 528 | IPv6 Sender Address | 529 | | 530 | | 531 | | 532 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 533 TLV if Sender Address is IPv6 535 Route Distinguisher is defined as part of [RFC4365] 537 5.3.1.4. TLV for MPLSoUDP Ping 539 0 1 2 3 540 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 541 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 542 | Type = 7 (MPLSoUDP ping IPv4)| Length | 543 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 544 | Route Distinguisher | 545 | (8 octets) | 546 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 547 | Sender IPv4 Address | 548 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 549 TLV if Sender Address is IPv4 551 0 1 2 3 552 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 553 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 554 | Type = 8 (MPLSoUDP ping IPv6)| Length | 555 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 556 | Route Distinguisher | 557 | (8 octets) | 558 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 559 | Sender IPv6 Address | 560 | | 561 | | 562 | | 563 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 564 TLV if Sender Address is IPv6 566 Route Distinguisher is defined as part of [RFC4365] 568 6. Return Codes 570 Sender MUST always set the Return Code set to zero. The receiver can 571 set it to one of the values listed below when replying back to Echo- 572 Request. 574 Following are the Return Codes (Suggested):- 576 Value What it means 577 ----- ------------------------------- 578 0 No return code 580 1 Malformed Echo Request Received 582 2 Overlay Segment Not Present 584 3 Overlay Segment Not Operational 586 4 Return-Code-OK 588 7. Procedure for Overlay Segment Ping 590 Echo Request is used to test Data Plane and its view of Control Plane 591 for particular Overlay Segment. The Overlay Segment to be verified 592 is identified differently for various Overlay Technologies. For 593 VXLAN, VNI is used to identify given Overlay Segment. For NVGRE, 594 VSID is used. For MPLSoGRE and MPLSoUDP the MPLS Stack is used to 595 identify a given Overlay Segment. 597 For the Data Plane verification, the Overlay Echo Request Packet MUST 598 be encapsulated within the Overlay Header, which is same as that of 599 any End-Point data going over the same Overlay Segment. This would 600 guarantee the data-path for OAM Packet follows the same path as that 601 for any End User data going over the same Overlay Segment. 603 The payload carried by Overlay typically could be either be Layer 2 604 or Ethernet Frame, or it could be Layer 3 or IP Packet. Based on the 605 type of payload following is the way inner Header(s) of Echo Request 606 would be encoded. 608 7.1. Encoding of Inner Header for Echo Request in Layer 2 Context 610 If the encapsulated payload carried by Overlay is of type Ethernet, 611 then the OAM Echo Request packet would have inner Ethernet Header, 612 followed by IP and UDP Header. The payload of inner UDP would be as 613 described in below section "Generic Overlay OAM Packet Format". 615 Inner Ethernet Header for the Echo Request Packet MUST have the 616 Destination Mac set to 00-00-5E-90-XX-XX (to be assigned IANA). The 617 Source Mac should be set to Mac Address of the Originating VTEP. 618 However, it is desired that the Inner Source Mac SHOULD not be learnt 619 in the MAC-Table as this represent Control Packet in context of 620 Overlay OAM. 622 Inner IP header is set with the Source IP Address which is a routable 623 Address of the sender; the Destination IP Address is a (randomly 624 chosen) IPv4 Address from the range 127/8, IPv6 addresses are chosen 625 from the range 0:0:0:0:0:FFFF:127/104. The IP TTL is set to 255. 627 The inner Destination UDP port is set to xxxx (assigned by IANA for 628 Overlay OAM). 630 The "Generic Overlay OAM Packet" will now be encoded, with following 631 information. 633 The sender chooses a Originator's Handle and a Sequence Number. When 634 sending subsequent Overlay Echo Requests, the sender SHOULD increment 635 the Sequence Number by 1. 637 The TimeStamp Sent is set to the time-of-day (in seconds and 638 microseconds) that the Echo Request is sent. The TimeStamp Received 639 is set to zero. Also, the Reply Mode must be set to the desired 640 reply mode. The Return Code and Subcode are set to zero. 642 Next, the TLV is Encoded for desired Overlay Type, as per Section 643 "Types of TLVs defined for various Overlay Ping Models" 645 7.2. Encoding of Inner Header for Echo Request in Layer 3 Context 647 If the encapsulated payload carried by Overlay is of type IP, then 648 the Encoding of the Echo Request would be same as above Section 649 "Encoding of Inner Header for Echo Request in Layer 2 Context", but 650 without the presence of Inner Ethernet Header. 652 7.3. VXLAN Procedures 654 7.3.1. Sending VXLAN Echo Request 656 The Outer VxLAN header for the Echo Request packet follows the 657 encapsulation as defined in [I-D.draft-mahalingam-dutt-dcops-vxlan]. 658 The VNI is same as that of the VXLAN Segment that is being verified. 659 This would make sure that OAM Packet takes the same datapath as any 660 other End System data going over this VXLAN Segment. 662 The VXLAN Router Alert option 663 [I-D.draft-singh-nvo3-vxlan-router-alert] MUST be set in the VXLAN 664 header as shown below. 666 VXLAN Header: 667 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 668 |R|R|R|R|I|R|R|RA| Reserved | 669 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 670 | VXLAN Network Identifier (VNI) | Reserved | 671 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 673 RA: Router Alter Bit (Proposed) 675 Originating VTEP MAY set the I Bit to 0 in VXLAN Header when sending 676 OAM Frame. This would cause dropping of such VXLAN frames on any 677 Terminating VTEP that does not understand Overlay OAM framework, and 678 prevent sending those frames to End-Systems or VMs. 680 It is desired to choose the Source UDP port (in the outer header), so 681 as to exercise the same Data-Path as that of the traffic carried over 682 the VXLAN Segment and is left to the implementation. 684 The Encoding of Inner Header(s) and UDP payload of Generic Overlay 685 OAM Packet is as described in above Sub-Section i.e. "Encoding of 686 Inner Header for Echo Request in Layer 2/Layer 3 Context". 688 7.3.2. Receiving VXLAN Echo Request 690 At the Terminating Overlay End Point or VTEP, since the Overlay OAM 691 Packet is exactly same as that of End-System Packet(s). It is 692 important to send OAM packet to Control Plane and prevent it from 693 sending to the End System. The trapping and sending VXLAN Echo 694 Request to the Control Plane is triggered by one of the following 695 Packet processing exceptions: VXLAN Router Alert option, 696 [I-D.draft-singh-nvo3-vxlan-router-alert] the Inner Destination MAC 697 Address of 00-00-5E-90-XX-XX as defined in above section, and the 698 Destination IP Address in the 127/8 Address range for IPv4 Address, 699 or 0:0:0:0:0:FFFF:127/104 for IPv6 Address. 701 The Control Plane further identifies the Overlay OAM Application by 702 UDP well know destination port xxxx. 704 Since the VxLAN Router Alert bit is set in VxLAN Header, which 705 signifies the presence of Control Packet. The terminating VTEP 706 SHOULD not learn the Mac address set in the Inner Mac Header of VxLAN 707 Echo Request Packet. 709 Once the VXLAN Echo Request Packet is identified at Control Plane, it 710 is processed as follows:- 712 o General Packet sanity is verified. If the Packet is not well- 713 formed, VTEP SHOULD send VXLAN Echo Reply with the Return Code set 714 to "Malformed Echo Request received" and the Subcode to zero. The 715 header fields Originator's Handle, Sequence Number, and Timestamp 716 Sent are not examined, but are included in the VXLAN Echo Reply 717 message 719 o VNI Validation: If there is no entry for VNI, it indicates that 720 there could be a transient or permanent disconnect between Control 721 Plane and data Plane and VTEP needs to report an error with Return 722 Code of "Overlay Segment Not Present" and a Return Subcode of 723 Zero. If the mapping for VNI Exists, but the state is not 724 Operational, VTEP needs to report an error with Return Code of 725 "Overlay Segment Not Operational" If the mapping exists then send 726 VXLAN Echo Reply with a Return Code of "Return-Code-OK", and a 727 Return Subcode of Zero. The procedures for sending the Echo Reply 728 are found in subsection below section. 730 7.3.3. Sending VXLAN Echo Reply 732 If the Reply Mode is set to "Reply via an IPv4/IPv6 UDP Packet", the 733 Echo Reply is a UDP Packet. It MUST ONLY be sent in response to Echo 734 Request. The Source IP Address in the Header should be Routable 735 Address of the replier; The Destination IP Address should be IP 736 Address of the Echo Request's Originating End Point or the requester. 737 The destination UDP Port is set to XXXX (assigned by IANA for 738 identifying VXLAN OAM application). The IP TTL is set to 255. 740 The format of the Echo Reply is the same as the Echo Request. The 741 Originator Handle, the Sequence Number, and TimeStamp Sent are copied 742 from the Echo Request; the TimeStamp Received is set to the time-of- 743 day that the Echo Request is received (note that this information is 744 most useful if the time-of-day clocks on the requester and the 745 replier are synchronized). The replier MUST fill in the Return Code 746 and Subcode, as determined in the previous subsection. 748 If the Reply Mode is set to "Reply via Overlay Segment", then the 749 Replying Overlay End Point is expected to place Echo Reply packet in- 750 band in the Overlay Segment destined to the Originating Overlay End 751 Point. The detailed encapsulation for this would be covered in next 752 revision of the draft. 754 7.3.4. Receiving VXLAN Echo Reply 756 An Originating Overlay End Point should only receive Echo Reply in 757 response to an Echo Request that it sent. When the Reply Mode is 758 "Reply via an IPv4/IPv6 UDP Packet", the Echo Reply would be and IP 759 Packet/UDP Packet, and is identified by the destination UDP Port 760 XXXX. The Originating Overlay End Point should parse the Packet to 761 ensure that it is well-formed, then attempt to match up the Echo 762 Reply with an Echo Request that it had previously sent, and the 763 Originator Handle. If no match is found, then it should drop the 764 Echo Reply Packet; otherwise, it checks the Sequence Number to see if 765 it matches. 767 7.4. NVGRE Procedures 769 7.4.1. Sending NVGRE Echo Request 771 The Outer NVGRE header for the Echo Request packet follows the 772 encapsulation as defined in 773 [I-D.draft-sridharan-virtualization-nvgre]. The VSID is same as that 774 of the NVGRE Segment that is being verified. This would make sure 775 that OAM Packet takes the same datapath as any other End System data 776 going over this NVGRE Segment. 778 The NVGRE Router Alert option 779 [I-D.draft-singh-nvo3-nvgre-router-alert] MUST be set in the NVGRE 780 header as shown below. 782 GRE Header: 783 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 784 |0| |1|0| Reserved0 RA| Ver | Protocol Type 0x6558 | 785 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 786 | Virtual Subnet ID (VSID) | Reserved | 787 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 789 RA: Router Alter Bit (Proposed) 791 The Encoding of Inner Header(s) and UDP payload of Generic Overlay 792 OAM Packet is as described in above Sub-Section i.e. "Encoding of 793 Inner Header for Echo Request in Layer 2/Layer 3 Context". 795 7.4.2. Receiving NVGRE Echo Request 797 At the Terminating Overlay End Point, since the Overlay OAM Packet is 798 exactly same as that of End-System Packet(s). It is important to 799 send OAM packet to Control Plane and prevent it from sending to the 800 End System. The trapping and sending NVGRE Echo Request to the 801 Control Plane is triggered by one of the following Packet processing 802 exceptions: NVGRE Router Alert option, 803 [I-D.draft-singh-nvo3-nvgre-router-alert] the Inner Destination MAC 804 Address of 00-00-5E-90-XX-XX as defined in above section, and the 805 Destination IP Address in the 127/8 Address range for IPv4 Address, 806 or 0:0:0:0:0:FFFF:127/104 for IPv6 Address. 808 The Control Plane further identifies the Overlay OAM Application by 809 UDP well know destination port xxxx. 811 Since the NVGRE Router Alert bit is set in NVGRE Header, which 812 signifies the presence of Control Packet. The Terminating Overlay 813 End Point SHOULD not learn the Mac address set in the Inner Mac 814 Header of NVGRE Echo Request Packet. 816 Once the NVGRE Echo Request Packet is identified at Control Plane, it 817 is processed as follows:- 819 o General Packet sanity is verified. If the Packet is not well- 820 formed, NVGRE End Point SHOULD send NVGRE Echo Reply with the 821 Return Code set to "Malformed Echo Request received" and the 822 Subcode to zero. The header fields Originator's Handle, Sequence 823 Number, and Timestamp Sent are not examined, but are included in 824 the NVGRE Echo Reply message 826 o VSID Validation: If there is no entry for VSID, it indicates that 827 there could be a transient or permanent disconnect between Control 828 Plane and data Plane and NVGRE End Point needs to report an error 829 with Return Code of "Overlay Segment Not Present" and a Return 830 Subcode of Zero. If the mapping for VSID Exists, but the state is 831 not Operational, NVGRE End Point needs to report an error with 832 Return Code of "Overlay Segment Not Operational" If the mapping 833 exists then send NVGRE Echo Reply with a Return Code of "Return- 834 Code-OK", and a Return Subcode of Zero. The procedures for 835 sending the Echo Reply are found in subsection below section. 837 7.4.3. Sending NVGRE Echo Reply 839 The procedure for sending NVGRE Echo Reply are exactly same as 840 defined in above section "Sending VXLAN Echo Reply". 842 7.4.4. Receiving NVGRE Echo Reply 844 The procedure for Receiving NVGRE Echo Reply are exactly same as 845 defined in above section "Receiving VXLAN Echo Reply". 847 7.5. MPLSoGRE Procedures 849 7.5.1. Sending MPLSoGRE Echo Request 851 The Outer header of MPLSoGRE for the Echo Request packet follows the 852 encapsulation as defined in [RFC4023]. The MPLS Stack is same as 853 that of the MPLSoGRE Segment that is being verified. This would make 854 sure that OAM Packet takes the same datapath as any other End System 855 data going over this MPLSoGRE Segment. 857 However, the bottommost Label in MPLS Stack MUST be MPLS Router Alert 858 Label [RFC3032]. This would indicate the Overlay Terminating End 859 Point that the payload is a Control Packet and needs to be delivered 860 to Control Plane. 862 The Encoding of Inner Header(s) and UDP payload of Generic Overlay 863 OAM Packet is as described in above Sub-Section i.e. "Encoding of 864 Inner Header for Echo Request in Layer 2/Layer 3 Context". 866 7.5.2. Receiving MPLSoGRE Echo Request 868 At the Terminating Overlay End Point, since the Overlay OAM Packet is 869 exactly same as that of End-System Packet(s). It is important to 870 send OAM packet to Control Plane and prevent it from sending to the 871 End System. The trapping and sending MPLSoGRE Echo Request to the 872 Control Plane is triggered by one of the following Packet processing 873 exceptions: MPLS Router Alert Label, and the Destination IP Address 874 in the 127/8 Address range for IPv4 Address, or 0:0:0:0:0:FFFF:127/ 875 104 for IPv6 Address. 877 The Control Plane further identifies the Overlay OAM Application by 878 UDP well know destination port xxxx. 880 Once the MPLSoGRE Echo Request Packet is identified at Control Plane, 881 it is processed as follows:- 883 o General Packet sanity is verified. If the Packet is not well- 884 formed, MPLSoGRE End Point SHOULD send MPLSoGRE Echo Reply with 885 the Return Code set to "Malformed Echo Request received" and the 886 Subcode to zero. The header fields Originator's Handle, Sequence 887 Number, and Timestamp Sent are not examined, but are included in 888 the MPLSoGRE Echo Reply message 890 o Segment Validation: If there is no entry for service represented 891 by given Route Distinguisher for the MPLSoGRE Segment, it 892 indicates that there could be a transient or permanent disconnect 893 between Control Plane and Data Plane and MPLSoGRE End Point needs 894 to report an error with Return Code of "Overlay Segment Not 895 Present" and a Return Subcode of Zero. If the entry for service 896 represented by given Route Distinguisher for the MPLSoGRE Segment 897 is present, but is Operationally Down. The End Point needs to 898 report an error with Return Code of "Overlay Segment Not 899 Operational" If the mapping of service represented by given Route 900 Distinguisher for the MPLSoGRE Segment is present and Active, then 901 send MPLSoGRE Echo Reply with a Return Code of "Return-Code-OK". 903 7.5.3. Sending MPLSoGRE Echo Reply 905 The procedure for sending MPLSoGRE Echo Reply are exactly same as 906 defined in above section "Sending VXLAN Echo Reply". 908 7.5.4. Receiving MPLSoGRE Echo Reply 910 The procedure for Receiving MPLSoGRE Echo Reply are exactly same as 911 defined in above section "Receiving VXLAN Echo Reply". 913 7.6. MPLSoUDP Procedures 915 7.6.1. Sending MPLSoUDP Echo Request 917 The Outer header of MPLSoUDP for the Echo Request packet follows the 918 encapsulation as defined in [I-D.draft-ietf-mpls-in-udp]. The MPLS 919 Stack is same as that of the MPLSoUDP Segment that is being verified. 920 This would make sure that OAM Packet takes the same datapath as any 921 other End System data going over this MPLSoUDP Segment. 923 However, the bottommost Label in MPLS Stack MUST be MPLS Router Alert 924 Label [RFC3032]. This would indicate the Overlay Terminating End 925 Point that the payload is a Control Packet and needs to be delivered 926 to Control Plane. 928 It is desired to choose the Source UDP port (in the outer header), so 929 as to exercise the same Data-Path as that of the traffic carried over 930 the MPLSoUDP Segment and is left to the implementation. 932 The Encoding of Inner Header(s) and UDP payload of Generic Overlay 933 OAM Packet is as described in above Sub-Section i.e. "Encoding of 934 Inner Header for Echo Request in Layer 2/Layer 3 Context". 936 7.6.2. Receiving MPLSoUDP Echo Request 938 At the Terminating Overlay End Point, since the Overlay OAM Packet is 939 exactly same as that of End-System Packet(s). It is important to 940 send OAM packet to Control Plane and prevent it from sending to the 941 End System. The trapping and sending MPLSoGRE Echo Request to the 942 Control Plane is triggered by one of the following Packet processing 943 exceptions: MPLS Router Alert Label, and the Destination IP Address 944 in the 127/8 Address range for IPv4 Address, or 0:0:0:0:0:FFFF:127/ 945 104 for IPv6 Address. 947 The Control Plane further identifies the Overlay OAM Application by 948 UDP well know destination port xxxx. 950 Once the MPLSoUDP Echo Request Packet is identified at Control Plane, 951 it is processed as follows:- 953 o General Packet sanity is verified. If the Packet is not well- 954 formed, MPLSoUDP End Point SHOULD send MPLSoUDP Echo Reply with 955 the Return Code set to "Malformed Echo Request received" and the 956 Subcode to zero. The header fields Originator's Handle, Sequence 957 Number, and Timestamp Sent are not examined, but are included in 958 the MPLSoUDP Echo Reply message 960 o Segment Validation: If there is no entry for service represented 961 by given Route Distinguisher for the MPLSoUDP Segment, it 962 indicates that there could be a transient or permanent disconnect 963 between Control Plane and data Plane and MPLSoUDP End Point needs 964 to report an error with Return Code of "Overlay Segment Not 965 Present" and a Return Subcode of Zero. If the entry for service 966 represented by given Route Distinguisher for the MPLSoUDP Segment 967 is present, but is Operationally Down. The End Point needs to 968 report an error with Return Code of "Overlay Segment Not 969 Operational" If the mapping of service represented by given Route 970 Distinguisher for the MPLSoUDP Segment is present and Active, then 971 send MPLSoUDP Echo Reply with a Return Code of "Return-Code-OK". 973 7.6.3. Sending MPLSoUDP Echo Reply 975 The procedure for sending MPLSoGRE Echo Reply are exactly same as 976 defined in above section "Sending VXLAN Echo Reply". 978 7.6.4. Receiving MPLSoUDP Echo Reply 980 The procedure for Receiving MPLSoGRE Echo Reply are exactly same as 981 defined in above section "Receiving VXLAN Echo Reply". 983 8. Procedure for Trace 985 In order to be able to trace the Path that a particular flow in the 986 Overlay takes through the Underlay Network, following mechanism can 987 be used - An overlay Echo Request packet is built and sent using the 988 mechanisms described in the Section "Procedure for Overlay Segment 989 Ping" so that the overlay traceroute follows the same path as the 990 data packet for the overlay segment being traced. 992 The Echo Request packet in the traceroute mode is sent with the 993 initial TTL set to 1 in the Outer IP header and thereafter 994 incremented by 1 in each successive request. At each transit hop 995 where the TTL expires, an exception is created. Because of this 996 exception, the packet gets delivered to the Control Plane. Control 997 plane can further deliver the packet to the OAM application based on 998 the TTL exception and the specific UDP port XXXX in the incoming 999 overlay echo request packet. If the transit node has the IP 1000 reachability to the destination IP address in the outer IP header, it 1001 sends back an overlay echo reply response otherwise the Overlay Echo 1002 Request is discarded by the Overlay OAM module on the transit nodes. 1003 If the transit node does not support overlay OAM functionality, it 1004 will simply generate a regular ICMP TTL exceeded response. This 1005 could result into "false negatives". The originating Overlay node 1006 that generated the OAM echo request SHOULD try sending the echo 1007 request with TTL=n+1, n+2, ... to probe the nodes further down the 1008 path to the terminating overlay End-point. 1010 At the originating node, when the Echo Reply from the transit node 1011 corresponding to the traceroute query is received, it can correlate 1012 the incoming Echo Reply with the traceroute query by matching on the 1013 sequence numbers in the Overlay Echo Request/Reply packets. 1015 Current revision of this draft limits overlay traceroute capability 1016 to fault isolation only. A subsequent version of the draft will 1017 include mechanisms to trace all possible paths in the underlay that 1018 can be used to carry overlay tunnel traffic. 1020 9. Procedure for End-System Ping 1022 In typical Overlay deployment scenarios there is a desired to check 1023 the presence of any given Tenant VM/End-System or Flow representing 1024 the End-System System within a given Overlay Segment. This draft 1025 proposes the way to achieve it via End-System Ping. 1027 The End-System can be identified at Overlay End Point by either its 1028 IP Address, Ethernet MAC Address or combination of IP/MAC Address. 1030 In that case, it would be important to verify the End-System 1031 connectivity by procedure which goes over the Overlay Segment from 1032 Originating Overlay End-Point and verifies the presence of the End- 1033 System at the Terminating Overlay End-Point. 1035 The scope of End-System Ping is solely with the Cloud Provider which 1036 owns control of the Overlay End Point(s). It is expected that the 1037 Overlay End Point traps this request and checks the Presence of the 1038 End-System via its MAC Address, Route or Flow information and replies 1039 back. There SHOULD not be a case where the End-System Ping is 1040 delivered to the actual End-Point. 1042 9.1. Sub-TLV for End-System Ping 1044 Sub-TLV Types:- 1046 Value What it means 1047 ----- --------------------------- 1048 1 End-System MAC Sub-TLV 1050 2 End-System IPv4 Sub-TLV 1052 3 End-System IPv6 Sub-TLV 1054 4 End-System MAC/IPv4 Sub-TLV 1056 6 End-System MAC/IPv6 Sub-TLV 1058 End-System Return Code:- 1060 Value What it means 1061 ----- ---------------------- 1062 1 End-System Present 1064 2 End-System Not Present 1066 9.1.1. Sub-TLV for Validating End-System MAC Address 1068 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1069 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1070 | End-System MAC Sub-TLV (1) | Length | 1071 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1072 | MAC address #1 | 1073 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1074 | MAC address #1 | Return Code#1 | 1075 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1076 | MAC address #2 | 1077 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1078 | MAC address #2 | Return Code#2 | 1079 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1080 ~ ... ~ 1081 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1082 | MAC address #n | 1083 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1084 | MAC address #n | Return Code #n | 1085 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1087 MAC Address: MAC Address of the End-System, that user is interested 1088 to validate. 1089 Return Code: Return Code specifying status of End-System at Overlay End Point 1091 9.1.2. Sub-TLV for Validating End-System IP Address 1093 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1094 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1095 | End-System IPv4 Sub-TLV (2) | Length | 1096 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1097 | IP address #1 | 1098 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1099 | Return Code #1 | IP address #2 | 1100 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1101 | IP address #2 | Return Code #2 | 1102 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1103 ~ ... ~ 1104 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1105 | IP address #n | 1106 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1107 | Return Code #n | 1108 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1110 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1111 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1112 | End-System IPv6 Sub-TLV (3) | Length | 1113 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1114 | | 1115 | IPv6 Address #1 | 1116 | | 1117 | | 1118 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1119 | Return Code #1 | IPv6 Address #2... ~ 1120 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1121 ~ ... ~ 1122 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1123 | | 1124 | IPv6 Address #n | 1125 | | 1126 | | 1127 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1128 | Return Code #n | 1129 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1131 IP Address : IP Address of the End-System, that user is interested to 1132 validate. 1133 Return Code: Return Code specifying status of End-System at Overlay End Point 1135 9.1.3. Sub-TLV for Validating End-System MAC and IP Address 1137 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1138 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1139 | End-System IPv4/MAC Sub-TLV (4)| Length | 1140 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1141 | MAC address #1 | 1142 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1143 | MAC address #1 | IP address #1 | 1144 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1145 | IP address #1 | Return Code #1 | 1146 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1147 | MAC address #2 | 1148 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1149 | MAC address #2 | IP address #2 | 1150 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1151 | IP address #2 | Return Code #2 | 1152 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1153 ~ ... ~ 1154 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1155 | MAC address #n | 1156 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1157 | MAC address #n | IP address #n | 1158 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1159 | IP address #n | Return Code #n | 1160 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1162 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1163 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1164 | End-System IPv6/MAC Sub-TLV(5) | Length | 1165 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1166 | MAC address #1 | 1167 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1168 | MAC address #1 | | 1169 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 1170 | | 1171 + + 1172 | IPv6 address #1 | 1173 + + 1174 | | 1175 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1176 | | Return Code #1 | 1177 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1178 ~ ... ~ 1179 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1180 | MAC address #n | 1181 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1182 | MAC address #n | | 1183 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 1184 | | 1185 + 1186 | IPv6 address #1 | 1187 + + 1188 | | 1189 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1190 | | Return Code #1 | 1191 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1193 IP Address : IP Address of the End-System, that user is interested to 1194 validate. 1195 MAC Address: MAC Address of the End-System, that user is interested to 1196 validate. 1197 Return Code: Return Code specifying status of End-System at Overlay End Point 1199 9.2. Sending End-System Ping Request 1201 When it is desired to check presence of a given End-System, the Echo 1202 Request Message is prepared as described in above Section "Procedure 1203 for Overlay Segment Ping". This packet should compose of Outer 1204 Header, Overlay Header, Inner Header, Generic Overlay Header with TLV 1205 representing desired Overlay Type (VXLAN, NVGRE, MPLSoGRE or 1206 MPLSoUDP). Apart form this the packet should also have one of the 1207 Sub-TLV's as defined in above section "Sub-TLV for End-System Ping" 1208 to identify the type of End-System Ping that user is interested in. 1210 Because of the above mentioned encapsulation, it would be guaranteed 1211 that the packet follows the same Data Path as that of any End-User 1212 data going over the given Overlay Segment. 1214 User need to fill in MAC, IP or MAC/IP combination for the End- 1215 System(s) that needs to be validated at the Overlay End Point in the 1216 respective Sub-TLV for End-System Ping. 1218 9.3. Receiving End-System Ping Request 1220 On receiving the End-System Ping Request the processing to trap this 1221 Packet, and sent it to Control Plane is done by Overlay Terminating 1222 End-System as define in above Section "Procedure for Overlay Segment 1223 Ping". Once the OAM Packet reaches OAM Application, it is identified 1224 as End-System Ping Request by virtue of presence any of the Sub-TLV's 1225 as defined in Section "Sub-TLV for End-System Ping". 1227 If the Sub-TLV is of Type "End-System MAC Sub-TLV", the Overlay End 1228 Point should iterate through the list of MAC Addresses and verify the 1229 presence of individual MAC Address in its Flow Table or MAC Table for 1230 the given Overlay Segment. 1232 If the MAC Address is present, it should set the respective End- 1233 System's Return Code field in the Sub-TLV to 1 "End-System-Present". 1235 If the MAC Address is not present, it should set respective the End- 1236 System's Return Code filed in the Sub-TLV to 2 "End-System-Not- 1237 Present". 1239 If the Sub-TLV is of Type "End-System IP Sub-TLV", the Overlay End 1240 Point should iterate through the list of IP Addresses and verify the 1241 presence of individual IP Address in its Flow Table or Route Table 1242 for the given Overlay Segment. 1244 If the IP Address is present, it should set the respective End- 1245 System's Return Code field in the Sub-TLV to 1 "End-System-Present". 1247 If the IPC Address is not present, it should set respective the End- 1248 System's Return Code filed in the Sub-TLV to 2 "End-System-Not- 1249 Present". 1251 If the Sub-TLV is of Type "End-System MAC and IP Sub-TLV", the 1252 Overlay End Point should iterate through the list of MAC/IP Addresses 1253 and verify the presence of individual MAC/IP Combination in its Flow 1254 Table or MAC and IP Table for the given Overlay Segment. 1256 If the IP and MAC Address is present, it should set the respective 1257 End-System's Return Code field in the Sub-TLV to 1 "End-System- 1258 Present". 1260 If the IP and MAC Address is not present, it should set respective 1261 the End-System's Return Code filed in the Sub-TLV to 2 "End-System- 1262 Not-Present". 1264 9.4. Sending End-System Ping Reply 1266 The procedure for sending End-System Echo Reply is same as defined in 1267 above section "Sending VXLAN Echo Reply". The replier MUST fill Sub- 1268 TLV with proper Return Code for each element in the End-System Sub- 1269 TLV. 1271 9.5. Receiving End-System Ping Reply 1273 An Originating Overlay End Point should only receive Echo Reply for 1274 End-System Ping, in response to an Echo Request that it sent. By 1275 virtue of presence of End-System Sub-TLV it would identify the status 1276 of respective End-System, and report it to the user. The other part 1277 of the handling is similar to section "Receiving VXLAN Echo Reply" 1279 10. Security Considerations 1281 TBD 1283 11. Management Considerations 1285 None 1287 12. Acknowledgements 1289 This document is the outcome of many discussions among many people, 1290 including Saurabh Shrivastava, Krishna Ram Kuttuva Jeyaram and Suresh 1291 Boddapati of Nuage Networks, Jorge Rabadan of Alcatel-Lucent, Inc and 1292 Rahul Kasralikar of Juniper Networks, Inc. 1294 13. IANA Considerations 1296 Action-1: This specification reserves a IANA UDP Port Number to be 1297 used when sending the Overlay OAM Packet 1299 Action-2: This specification reserves a IANA Ethernet unicast Address 1300 for VXLAN/NVGRE Exception handling. This Address needs to be 1301 reserved from the block. "IANA Ethernet Address block - Unicast Use" 1303 14. References 1305 14.1. Normative References 1307 [I-D.draft-ietf-mpls-in-udp] 1308 Xu, Sheth, Yong, Pignataro, Yongbing , and Li, 1309 "Encapsulating MPLS in UDP", May 2013. 1311 [I-D.draft-lasserre-nvo3-framework] 1312 Lasserre, M., Balus, F., Morin, T., Bitar, N., and Y. 1313 Rekhter, "Framework for DC Network Virtualization", 1314 September 2011. 1316 [I-D.draft-mahalingam-dutt-dcops-vxlan] 1317 Mahalingam, M., Dutt, D., Agarwal, P., Kreeger, L., 1318 Sridhar, T., Bursell, M., and C. Wright, "VXLAN: A 1319 Framework for Overlaying Virtualized Layer 2 Networks 1320 over Layer 3 Networks", May 2013. 1322 [I-D.draft-singh-nvo3-nvgre-router-alert] 1323 Singh, K., Jain, P., Balus, F., and W. Henderickx, "NVGRE 1324 Router Alert Option", May 2013. 1326 [I-D.draft-singh-nvo3-vxlan-router-alert] 1327 Singh, K., Jain, P., Balus, F., and W. Henderickx, "VxLAN 1328 Router Alert Option", May 2013. 1330 [I-D.draft-sridharan-virtualization-nvgre] 1331 Sridharan, M., Duda, K., Greenberg, A., Lin, G., Pearson, 1332 M., Thaler, P., Tumuluri, C., Venkataramiah, N., and Y. 1333 Wang, "NVGRE: Network Virtualization using Generic Routing 1334 Encapsulation", February 2013. 1336 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 1337 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 1338 Encoding", RFC 3032, January 2001. 1340 [RFC4023] Worster, T., Rekhter, Y., and E. Rosen, "Encapsulating 1341 MPLS in IP or Generic Routing Encapsulation (GRE)", 1342 RFC 4023, March 2005. 1344 [RFC4365] Rosen, E., "Applicability Statement for BGP/MPLS IP 1345 Virtual Private Networks (VPNs)", RFC 4365, February 2006. 1347 [RFC4379] Kompella, K. and G. Swallow, "Detecting Multi-Protocol 1348 Label Switched (MPLS) Data Plane Failures", RFC 4379, 1349 February 2006. 1351 14.2. Informative References 1353 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1354 Requirement Levels", BCP 14, RFC 2119, March 1997. 1356 [RFC4330] Mills, D., "Simple Network Time Protocol (SNTP) Version 4 1357 for IPv4, IPv6 and OSI", RFC 4330, January 2006. 1359 Authors' Addresses 1361 Pradeep Jain 1362 Nuage Networks 1363 755 Ravendale Drive 1364 Mountain View, CA 94043 1365 USA 1367 Email: pradeep@nuagenetworks.net 1369 Kanwar Singh 1370 Nuage Networks 1371 755 Ravendale Drive 1372 Mountain View, CA 94043 1373 USA 1375 Email: kanwar@nuagenetworks.net 1377 Florin Balus 1378 Nuage Networks 1379 755 Ravendale Drive 1380 Mountain View, CA 94043 1381 USA 1383 Email: florin@nuagenetworks.net 1385 Wim Henderickx 1386 Alcatel-Lucent 1387 Copernicuslaan 50 1388 Antwerp 2018 1389 Belgium 1391 Email: wim.henderickx@alcatel-lucent.be 1393 Vinay Bannai 1394 PayPal 1395 2211 N. First St, 1396 San Jose 95131 1397 USA 1399 Email: vbannai@paypal.com 1400 Ravi Shekhar 1401 Juniper Networks 1402 1194 North Mathilda Ave. 1403 Sunnyvale, CA 94089 1404 USA 1406 Email: rshekhar@juniper.net 1408 Anil Lohiya 1409 Juniper Networks 1410 1194 North Mathilda Ave. 1411 Sunnyvale, CA 94089 1412 USA 1414 Email: alohiya@juniper.net