idnits 2.17.1
draft-tissa-nvo3-oam-fm-04.txt:
Checking boilerplate required by RFC 5378 and the IETF Trust (see
https://trustee.ietf.org/license-info):
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt:
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/checklist :
----------------------------------------------------------------------------
** There is 1 instance of too long lines in the document, the longest one
being 1 character in excess of 72.
Miscellaneous warnings:
----------------------------------------------------------------------------
== The copyright year in the IETF Trust and authors Copyright Line does not
match the current year
== Line 133 has weird spacing: '...leading to...'
== Line 927 has weird spacing: '... out of b...'
== Line 1499 has weird spacing: '... no speci...'
== The document seems to lack the recommended RFC 2119 boilerplate, even if
it appears to use RFC 2119 keywords -- however, there's a paragraph with
a matching beginning. Boilerplate error?
(The document does seem to have the reference to RFC 2119 which the
ID-Checklist requires).
-- The document date (May 5, 2017) is 2547 days in the past. Is this
intentional?
Checking references for intended status: Informational
----------------------------------------------------------------------------
== Missing Reference: 'TRILLFM' is mentioned on line 159, but not defined
== Missing Reference: 'TBDMIBD' is mentioned on line 670, but not defined
== Missing Reference: 'TBDMIB' is mentioned on line 722, but not defined
== Unused Reference: 'RFC4379' is defined on line 1550, but no explicit
reference was found in the text
== Unused Reference: 'RFC6291' is defined on line 1554, but no explicit
reference was found in the text
== Unused Reference: 'NVO3ARC' is defined on line 1563, but no explicit
reference was found in the text
== Unused Reference: 'NV03DPREQ' is defined on line 1570, but no explicit
reference was found in the text
-- Obsolete informational reference (is this intentional?): RFC 4379
(Obsoleted by RFC 8029)
Summary: 1 error (**), 0 flaws (~~), 12 warnings (==), 2 comments (--).
Run idnits with the --verbose option for more detailed information about
the items above.
--------------------------------------------------------------------------------
2 NV03 Working Group Tissa Senevirathne
3 Internet Draft Samer Salam
4 Intended Status: Informational Deepak Kumar
5 Norman Finn
7 Cisco
8 Donald Eastlake
9 Sam Aldrin
10 Huawei
11 Expires September 2017 May 5, 2017
13 NVO3 Fault Management
14 draft-tissa-nvo3-oam-fm-04.txt
16 Status of this Memo
18 This Internet-Draft is submitted in full conformance with the
19 provisions of BCP 78 and BCP 79.
21 Internet-Drafts are working documents of the Internet Engineering
22 Task Force (IETF), its areas, and its working groups. Note that
23 other groups may also distribute working documents as Internet-
24 Drafts.
26 Internet-Drafts are draft documents valid for a maximum of six months
27 and may be updated, replaced, or obsoleted by other documents at any
28 time. It is inappropriate to use Internet-Drafts as reference
29 material or to cite them other than as "work in progress."
31 The list of current Internet-Drafts can be accessed at
32 http://www.ietf.org/ietf/1id-abstracts.txt.
34 The list of Internet-Draft Shadow Directories can be accessed at
35 http://www.ietf.org/shadow.html.
37 This Internet-Draft will expire on November 6, 2017.
39 Copyright Notice
41 Copyright (c) 2017 IETF Trust and the persons identified as the
42 document authors. All rights reserved.
44 This document is subject to BCP 78 and the IETF Trust's Legal
45 Provisions Relating to IETF Documents
46 (http://trustee.ietf.org/license-info) in effect on the date of
47 publication of this document. Please review these documents
48 carefully, as they describe your rights and restrictions with respect
49 to this document. Code Components extracted from this document must
50 include Simplified BSD License text as described in Section 4.e of
51 the Trust Legal Provisions and are provided without warranty as
52 described in the Simplified BSD License.
54 Abstract
56 This document specifies Fault Management solution for network
57 virtualization overlay networks. Methods in this document follow the
58 IEEE 802.1 CFM framework and reuse OAM tools where possible.
59 Additional messages and TLVs are defined for IETF overlay OAM
60 specific applications or where extensions beyond IEEE 802.1 CFM are
61 required.
63 Table of Contents
65 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
66 2. Conventions used in this document . . . . . . . . . . . . . . . 5
67 3. NV03 OAM Layers . . . . . . . . . . . . . . . . . . . . . . . . 6
68 4. General Format of NV03 OAM frames . . . . . . . . . . . . . . . 7
69 4.1. NV03 Shim . . . . . . . . . . . . . . . . . . . . . . . . . 8
70 4.2. Identification of OAM frames . . . . . . . . . . . . . . . 8
71 4.3 OAM Channel . . . . . . . . . . . . . . . . . . . . . . . . 10
72 4.3.1 OAM Channel Functionality Summary . . . . . . . . . . . 10
73 4.3.2 Opcode Implementation Recommendation . . . . . . . . . . 10
74 5. Maintenance Associations (MA) in NVO3 . . . . . . . . . . . . . 11
75 6. MEP Addressing . . . . . . . . . . . . . . . . . . . . . . . . 12
76 6.1. Use of MIP in NV03 . . . . . . . . . . . . . . . . . . . . 15
77 7. Continuity Check Message (CCM) . . . . . . . . . . . . . . . . 16
78 8. NVO3 OAM Message Channel . . . . . . . . . . . . . . . . . . . 18
79 8.1. NVO3 OAM Message header . . . . . . . . . . . . . . . . . . 18
80 8.2. IETF Overlay OAM Opcodes . . . . . . . . . . . . . . . . . 19
81 8.3. Format of IETF Overlay OAM TLV . . . . . . . . . . . . . . 19
82 8.4. IETF Overlay OAM TLVs . . . . . . . . . . . . . . . . . . . 20
83 8.4.1. Common TLVs between 8201Q CFM and IETF Overlay OAM . . 20
84 8.4.2. IETF Overaly OAM Specific TLVs . . . . . . . . . . . . 20
85 8.4.3. OAM Application Identifier TLV . . . . . . . . . . . . 21
86 8.4.4. Out Of Band Reply Address TLV . . . . . . . . . . . . . 22
87 8.4.5. Diagnostics Label TLV . . . . . . . . . . . . . . . . . 23
88 8.4.6. Original Data Payload TLV . . . . . . . . . . . . . . . 23
89 8.4.7. Flow Identifier (flow-id) TLV . . . . . . . . . . . . . 24
90 8.4.8. Reflector Entropy TLV . . . . . . . . . . . . . . . . . 24
91 9. Loopback Message . . . . . . . . . . . . . . . . . . . . . . . 25
92 9.2. Theory of Operation . . . . . . . . . . . . . . . . . . . . 26
93 9.2.1. Actions by Originator . . . . . . . . . . . . . . . . . 26
94 9.2.2. Intermediate Devices . . . . . . . . . . . . . . . . . 26
95 9.2.3. Destination Device . . . . . . . . . . . . . . . . . . 27
96 10. Path Trace Message . . . . . . . . . . . . . . . . . . . . . . 27
97 10.1. Theory of Operation . . . . . . . . . . . . . . . . . . . 28
98 10.1.1. Action by Originator Device . . . . . . . . . . . . . 28
99 10.1.2. Intermediate Device . . . . . . . . . . . . . . . . . 29
100 10.1.3. Destination Device . . . . . . . . . . . . . . . . . . 29
101 11. Link Trace Message . . . . . . . . . . . . . . . . . . . . . . 30
102 11.1 MEP and MIP . . . . . . . . . . . . . . . . . . . . . . . . 30
103 11.2 Initiator . . . . . . . . . . . . . . . . . . . . . . . . . 30
104 11.3 Intermediate Devices . . . . . . . . . . . . . . . . . . . 31
105 11.4 Terminating Device . . . . . . . . . . . . . . . . . . . . 31
106 11.5 Output . . . . . . . . . . . . . . . . . . . . . . . . . . 31
107 12. Application of Continuity Check Message (CCM) in NVO3 . . . . 32
108 12.1. CCM Error Notification . . . . . . . . . . . . . . . . . . 32
109 12.2. Theory of Operation . . . . . . . . . . . . . . . . . . . 34
110 12.2.1. Actions by Originator Device . . . . . . . . . . . . . 34
111 12.2.2. Intermediate Devices . . . . . . . . . . . . . . . . . 34
112 12.2.3. Destination Device . . . . . . . . . . . . . . . . . . 34
113 13. Security Considerations . . . . . . . . . . . . . . . . . . . 35
114 14. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 35
115 15. References . . . . . . . . . . . . . . . . . . . . . . . . . . 35
116 15.1. Normative References . . . . . . . . . . . . . . . . . . . 35
117 15.2. Informative References . . . . . . . . . . . . . . . . . . 35
118 16. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 36
119 Appendix A. Base Mode for NVO3 OAM . . . . . . . . 36
121 1. Introduction
123 Conceptually, NVO3 architecture contains four separate layers,
124 namely, Service (customer) Layer, Overlay Layer, Transport or
125 Underlay Layer and Media Layer. Figure 1 below depicts the
126 relationship between each of these layers.
128 Fault Monitoring, Fault Verification, Fault Isolation, Loss and delay
129 measurements are integral part of NVO3 [nvo3oamReq]. These need to
130 cover both unicast and multicast traffic streams.
132 For effective fault isolation, the overlay OAM solution should
133 complement the OAM functions at adjacent layers, thereby leading to
134 nested OAM model. Nested OAM allows operators to quickly and
135 effectively troubleshoot and isolate data plane failures using a
136 common OAM framework.
138 Common OAM message format and infrastructure makes it easier to
139 accomplish nested OAM. IEEE Connectivity Fault Management (CFM)
140 [8021Q] is widely used in the Ethernet world. The same technology has
141 been extended by ITU-T [Y1731], Metro Ethernet Forum (MEF) and TRILL
142 [TRILLFM].
144 A common OAM message channel can be shared between different
145 technologies. This consistency between different OAM technologies
146 promotes nested fault monitoring and isolation between technologies
147 that share the same OAM framework.
149 This document uses the message format defined in IEEE 802.1ag
150 Connectivity Fault Management (CFM) [8021Q] as the basis for the OAM
151 messages.
153 This document presents the NVO3 OAM message structure, NVO3 frame
154 identification and NV03 OAM tools. The NVO3 OAM Management
155 Information Base (MIB) will be presented in a separate document.
157 The ITU-T Y.1731 [Y1731] standard utilizes the same messaging format
158 as [8021Q] and for OAM messages where applicable. This document takes
159 a similar stance and reuses [8021Q] and TRILL OAM [TRILLFM]. It is
160 assumed that readers are familiar with [8021Q] and [Y1731].
162 |T |--|NVE|--|R|-|B|--|B|-|R|-|NVE|-|T|
164 --- X----o-----------------------o-----X [Service Layer]
165 | O | | |
166 | A | | |
167 | M | X------o-----------o----X [NVo3 Layer]
168 | | | |
169 | M | | |
170 | I | X--o-----o--X [Transport Layer]
171 | B | | |
172 | | | |
173 --- X--o--X [Link/Circuit Layer]
175 X - Maintenance End Point (MEP)
176 o - Maintenance Intermediate Point (MIP)
178 T - Tenant System NVE - Network Virtualization Edge
179 R - Router B - Bridge
181 Figure 1 Layered OAM Architecture
183 2. Conventions used in this document
185 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL",
186 "SHALL NOT","SHOULD", "SHOULD NOT", "RECOMMENDED",
187 "MAY", and "OPTIONAL" in this
188 document are to be interpreted as described in RFC-2119 [RFC2119].
190 Acronyms used in the document include the following:
192 MP - Maintenance Point [8021Q]
194 MEP - Maintenance End Point [8021Q]
195 MIP - Maintenance Intermediate Point [8021Q]
197 MA - Maintenance Association [8021Q]
199 MD - Maintenance Domain [8021Q]
201 CCM - Continuity Check Message [8021Q]
203 LBM - Loop Back Message [8021Q]
205 PTM - Path Trace Message
207 MTV - Multi-destination Tree Verification Message
209 ECMP - Equal Cost Multipath
211 ISS - Internal Sub Layer Service [8021Q]
213 VNI - Virtual Network Instance [NVO3FRM]
215 NVE - Network Virtual Edge [NVO3FRM]
217 SAP - Service Access Point [8021Q]
219 3. NV03 OAM Layers
221 Figure 1 above depicts different layers within NVO3. Each of these
222 layers has a unique scope within the common framework. In this
223 section, we define functionality of each of these layers
225 Service Layer: Service Layer carries customer or user traffic.
226 It is originated at Tenant systems and terminates at other
227 Tenant system(s) within the same NVO3 context (VNI).
229 NVO3 Layer: NVO3 Layer carries service Layer traffic
230 encapsulated in NVO3 format. NVO3 Layer originates at an
231 NVE and terminates at another NVE.
233 Transport Layer: Transport Layer carries NVO3 Layer traffic
234 encapsulated in its data format. The transport Layer
235 can be IP, MPLS or any other protocol.
237 Link Layer: This is also known as circuit layer.
238 It carries Transport Layer traffic in a specific manner
239 from one device to another.
240 Ethernet is an example of link layer.
242 4. General Format of NV03 OAM frames
244 For accurate monitoring and/or diagnostics, OAM Messages are required
245 to follow the same data path as corresponding user packets.
246 Additionally, NVEs are required to identify NVO3 frames and act on
247 them, in addition to preventing the overlay OAM packets from leaking
248 outside of the NVO3 domain [NVO3OAMREQ]. This document defines the
249 format of the NVO3 overlay OAM messages.
251 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
252 | |
253 . Circuit Header . (variable)
254 | |
255 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
256 | |
257 + Transport Header Technology dependent
258 | |
259 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
260 | NVO3 Shim | 8 bytes
261 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
262 | |
263 . Payload fragment . 96 bytes (optional)
264 . .
265 | |
266 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
267 | OAM Ether Type |
268 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
269 | |
270 . OAM Message Channel . Variable
271 . .
272 | |
273 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
274 | Link Trailer | Variable
275 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
277 Figure 2 Format of NVO3 Overlay OAM Messages
279 Link Header: Media-dependent header. For Ethernet, this includes
280 Destination MAC, Source MAC, VLAN (optional) and EtherType fields.
282 Transport Header: Header of the Transport Layer e.g. IP , MPLS, TRILL
283 etc.
285 NVO3 Shim: This is a fixed sized field (size TBD 8 bytes [vxLAN])
286 that carries NVO3 specific information. Fields in this header will be
287 utilized to identify OAM frames and other NVO3 specific operations.
288 Please see section 4.2 below of NVO3 OAM specific operations.
290 Payload Fragment: This is an optional field. When included it has a
291 fixed size of 96 bytes. The least significant bits of the field MUST
292 be padded with zeros, up to 96 bytes, when the payload fragment is
293 less than 96 bytes. The Payload Fragment field starts with the
294 Inner.MacDA.
296 OAM Ether Type: OAM Ether Type is 16-bit EtherType that identifies
297 the OAM Message channel that follows. This document specifies using
298 the EtherType 0x8902 allocated for [8021Q] for this purpose.
299 Identifying the OAM Message Channel with a dedicated EtherType allows
300 the easy identification of the beginning of the OAM message channel
301 across multiple standards.
303 OAM Message Channel: This is a variable size section that carries OAM
304 related information. The message format defined in [8021Q] will be
305 reused.
307 Link Trailer: Media-dependent trailer. For Ethernet, this is the FCS
308 (Frame Check Sequence).
310 Note: In this draft we are proposing re-use of OAM Channel defined in
311 RFC7455 and RFC7456.
313 4.1. NV03 Shim
315 NVO3 Shim is an 8 octet vector that carries series of flags and
316 additional information [vxLAN]. Each of the flags identifies a
317 specific operation.
319 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
320 |R|R|R|R|I|R|R|R| Reserved |
321 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
322 | VXLAN Network Identifier (VNI) | Reserved |
323 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
325 Figure 3 NVO3 Shim
327 4.2. Identification of OAM frames
329 Implementations that comply with this document MUST utilize "O" flag
330 to identify NVO3 OAM frames. The "O" flag MUST NOT BE utilized for
331 forwarding decisions such as the selection of which ECMP path to use.
333 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
334 |R|R|R|R|I|R|R|O| Reserved |
335 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
336 | VXLAN Network Identifier (VNI) | Reserved |
337 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
339 Figure 4 NVO3 shim with the "O" Flag
341 O (1 bit) - Indicates this is a possible OAM frame and is subject to
342 specific handling as specified in this document. O bit is aligned
343 with Vxlan GPE.
345 Payload Fragment is optional and if hardware is capable of matching
346 OAM frame based on O bit then payload fragment handling doesn't
347 require extra bit by checking etype. OAM Frame may have 96 octets of
348 payload fragments immediately after the NVO3 shim or OAM Ethertype
349 0x8902 immediately follows the NVO3 shim after Inner DMAC and Inner
350 SMAC. This can be determined by matching 0x8902 at right position
351 after matching "O" bit, if it's not present, then look deeper after
352 96 bytes.
354 Payload Fragment is optional implementation but it's very important
355 to track actual data path in scenario described below.
357 SS1 SS2 (L3 VTEP)
358 / | \/ \
359 / | /\ \ (ECMP paths)
360 / |/ \ \
361 S1 S2 S3
362 || || || (Underlay ECMP)
363 || || ||
364 L1 L2 L3 (L2 VTEP)
366 | |
367 H1 H2
369 Figure 5 Payload Fragment use case
371 For H1 and H2 we can have bridge traffic and routed traffic on SSx.
372 For Routed traffic inner header or payload Fragement is required to
373 be looked to find the exact ECMP path.
375 All other fields carry the same meaning as defined in [vXLAN].
377 4.3 OAM Channel
379 OAM channel proposed to use is based of RFC7455 and RFC7456.
380 Advantage of re-using this channel is header is flexible to support
381 extensible functionality. It also provide backward compatibility mode
382 for hardware which were developed before OAM became standard to do
383 OAM functionality.
385 4.3.1 OAM Channel Functionality Summary
387 OAM Common Header is defined in section 8.1. MD-L allows multiple
388 level of OAM to be possible, for eg:- IP layer Overlay can be at
389 default level 3 and SFC layer OAM can be at Application Level 4.
391 Opcode provide hardware friendly and extensible extensions. Hardware
392 friendly messages are defined without TLVs.
394 Summary of Opcode(s) to be considered.
396 1. CCM - Proactive Fault Monitoring (one-way)
398 2. LBM/LBR - on-demand Fault Verification (LBR can be sent out of
399 band also and carry ICMP error code if required.)
401 3. PTM/PTR - on-demand Fault Isolation based on TTL expiry (works
402 with non OAM capable underlay, ip unnumbered underlay, and provide
403 Egress Interface to better isolate fault than traditional traceroute)
405 4. DMM/DMR - on-demand or pro-active Delay Measurement.
407 5. 1DM - On-demand or pro-active one way Delay Measurement.
409 6. LTM/LTR - on-demand Fault Isolation for scenario where all
410 underlay switches are OAM capable to provide path via hardware
411 forwarding without CPU intervention.
413 7. SLM/SLR - on-demand or pro-active Synthetic Delay Measurement.
415 8. 1SL - On-demand or pro-active one way Synthetic Delay Measurement.
417 Application Identifier TLV allow Return code and sub-code to carry
418 ICMP Errors, It allows out of band or in-band communication
419 flexibility and support OAM to be carried in multiple fragments if
420 data request is very large.
422 4.3.2 Opcode Implementation Recommendation
424 CCM - Optional LBM/LBR - Mandatory PTM/PTR - Mandatory DMM/DMR -
425 Optional (As it's performance function) 1DM - Optional LTM/LTR -
426 Optional SLM/SLR - Optional 1SL - Optional
428 5. Maintenance Associations (MA) in NVO3
430 [8021Q] defines a maintenance association as a logical relationship
431 between a group of nodes. Each Maintenance Association (MA) is
432 identified with a unique MAID of 48 bytes [8021Q]. CCM and other
433 related OAM functions operate within the scope of an MA. The
434 definition of MA is technology independent. MA is encoded in the
435 technology independent part of the OAM message Hence the MAID, as
436 defined in [8021Q], can be utilized for NVo3 OAM, without
437 modifications. This also allows us to utilize CCM and LBM messages
438 defined in [8021Q], as is.
440 In NVO3, an MA may contain two or more NVEs (MEPs). For unicast, it
441 is likely that the MA contains exactly two MEPs that are the two end-
442 points of the flow. For multicast, the MA may contain two or more
443 MEPs.
445 For NVO3, in addition to all of the standard CFM MIB [8021Q]
446 definitions, each MEP's MIB contains one or more flow definitions
447 corresponding to the set of flows that the MEP monitors. Flow entropy
448 specifies the VNI within the NVE.
450 MEPs can be created per VNI within the NVE.
452 We propose to augment the [8021Q] MIB to add NVO3 specific
453 information. Figure 5, below depicts the augmentation of the CFM MIB
454 to add the NVO3 specific Flow Entropy.
456 MA---
457 |
458 --- MEP
459 |
460 . - Remote MEP List
461 .
462 |
463 --- MEP-A
464 |
465 --- MEP-B
466 .
468 |
469 . - Flow List { Augments IEEE8021-CFM-MIB}
471 |
472 --- (Flow-1)
473 |
474 --- (Flow-2)
475 |
476 . --- (Flow-n)
477 |
478 Other MIB entries
480 Figure 6 Correlation of NVO3 augmented MIB
482 The flow list contains multiple flows. Each flow contains a VNI and
483 optional data payload. There can be more than one flow for a given
484 VNI with different data payloads e.g. unicast vs. multicast or
485 unicast with different data payloads.
487 6. MEP Addressing
489 In IEEE 802.1ag [8021Q], OAM messages address the target MEP by
490 utilizing a unique MAC address. In NVO3, MEPs are created per VNI,
491 MEPs are addressed by combination of Transport Layer address of the
492 NVE and the VNI.
494 At the MEP, OAM packets go through a hierarchy of op-code de-
495 multiplexers. The op-code de-multiplexers channel the incoming OAM
496 packets to the appropriate message processor (e.g. LBM) The reader
497 may refer to Figure 6 below for a visual depiction of these different
498 de-multiplexers.
500 1. Identify the packets that need OAM processing at the Local Device
501 as specified in Section 4.2.
503 a. Identify the MEP that is associated with the VNI.
505 2. The MEP then validates the MD-LEVEL
507 a. Redirect to MD-LEVEL De-multiplexer
509 3. MD-LEVEL de-multiplexer compares the MD-Level of the packet
510 against the MD level of the local MEPs. (Note: there can be more
511 than one MEP at the same MD-Level but belonging to different MAs)
513 a. If the packet MD-LEVEL is equal to the configured MD-LEVEL
514 of the MEP, then pass to the Opcode de-multiplexer
516 b. If the packet MD-LEVEL is less than the configured MD-LEVEL
517 of the MEP, discard the packet
519 c. If the packet MD-LEVEL is greater than the configured MD-
520 LEVEL of the MEP, then pass on to the next higher MD-LEVEL
521 de-multiplexer, if available. Otherwise, if no such higher MD-
522 LEVEL de-multiplexer exists, then forward the packet as normal
523 data.
525 4. Opcode De-multiplexer compares the opcode in the packet with the
526 supported opcodes
528 a. If Op-code is CCM, LBM, LBR, PTM, PTR, MTVM, MTVR, then pass
529 on to the correct Processor
531 b. If Op-code is Unknown, then discard.
533 |
534 .CCM LBM PTM MTV LT
535 | | | | |
536 +-+-+-+-+-+-+-+-+-+-+-+-+
537 | OP Code DE-Mux |--- Unknown
538 +-+-+-+-+-+-+-+-+-+-+-+-+
539 ^ ^ ^
540 MD==Li | | |
541 +-+-+ +-+-+ +-+-+
542 | L |-->|L2 |-.- |Ln |---- >
543 +-+-+ +-+-+ +-+-+ |
544 | ^ | | |
545 MD
| T |----------------- >| M |--- >
552 + NVO3 OAM ---- + pass through OAM ----
554 Figure 7 OAM De-Multiplexers at MEP for active SAP
556 Default MEPs are assumed to be created at NVE to handle OAM
557 functionality as per Appendix A.
559 T : Denotes Tap, that identifies OAM frames that need local
560 processing. These are the packets with OAM flag set AND OAM
561 Ether type is present after the flow entropy of the packet
563 M : Is the post processing merge, merges data and OAM messages
564 that are passed through. Additionally, the Merge component
565 ensures, as explained earlier, that OAM packets are not
566 forwarded out as native frames.
568 L : Denotes MD-Level processing. Packets with MD-Level less than
569 the Level will be dropped. Packets with equal MD-Level are
570 passed on to the opcode de-multiplexer. Others are passed on to
571 the next level MD processors or eventually to the merge point
572 (M).
574 NOTE: LBM, MTV and PT are not subject to MA de-multiplexers.
575 These packets do not have an MA encoded in the packet. Adequate
576 response can be generated to these packets, without loss of
577 functionality, by any of the MEPs.
579 6.1. Use of MIP in NV03
581 Maintenance Intermediate Points (MIP) are mainly used for fault
582 isolation. Link Trace Messages in [8021Q] utilize a well-known
583 multicast MAC address and MIPs generate responses to Link Trace
584 messages. Response to Link Trace messages or lack thereof can be
585 used for fault isolation in NVO3.
587 As explained in section 10. below, a TTL expiry approach will be
588 utilized for fault isolation and path tracing. The approach is very
589 similar to the well-known IP trace-route approach. Hence, explicit
590 addressing of MIPs is not required for the purpose of fault
591 isolation.
593 Any given NVO3 device can have multiple MIPs located within the
594 device. As such, a mechanism is required to identify which MIP
595 should respond to an incoming OAM message.
597 A similar approach to that presented above for MEPs can be used for
598 MIP processing. It is important to note that "M", the merge block of
599 a MIP, does not prevent OAM packets leaking out as native frames. On
600 edge interfaces, MEPs MUST be configured to prevent the leaking of
601 overlay OAM packets out of the NVO3 domain.
603 LT PT MTV
604 | | |
605 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
606 | OP Code De-Mux |->Unknown
607 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
608 ^ ^ ^
609 MD==Li | | |
610 +-+-+ +-+-+ +-+-+
611 | L |- >|L2 |-.- |Ln |------+
612 +-+-+ +-+-+ +-+-+ |
613 ^ |
614 | |
615 Drop | |
616 MD not --- |NVO3 OAM |
617 Present | |
618 | v
619 NVO3 Data ---- NVO3 Data -----
620 ------- >| T |------------------ >| M |--->
621 + NVO3 OAM ---- ----
623 Figure 8 OAM De-Multiplexers at MIP for active SAP
625 T: TAP processing for MIP. All packets with OAM flag set are
626 captured.
628 L : MD Level Processing, Packet with matching MD Level are "copied"
629 to the Opcode de-multiplexer and original packet is passed on to the
630 next MD level processor. Other packets are simply passed on to the
631 next MD level processor, without copying to the OP code de-
632 multiplexer.
634 M : Merge processor, merge OAM packets to be forwarded along with
635 the data flow.
637 Packets that carry Path Trace Message (PTM) or Multi-destination
638 Tree Verification (MTV) OpCodes are passed on to the respective
639 processors.
641 Packets with unknown OpCodes are counted and discarded.
643 7. Continuity Check Message (CCM)
645 CCMs are used to monitor connectivity and configuration errors.
646 [8021Q] monitors connectivity by having a MEP listening to periodic
647 CCM messages received from its remote MEP partners in the MA. An
648 [8021Q] MEP identifies cross-connect errors by comparing the MAID in
649 the received CCM message with the MEP's local MAID. The MAID [8021Q]
650 is a 48-byte field that is technology independent. Similarly, the
651 MEPID is a 2-byte field that is independent of the technology. Given
652 this generic definition of CCM fields, CCM as defined in [8021Q] can
653 be utilized in NVO3 with no changes. NVO3 specific information may
654 be carried in CCMs when encoded using IETF overlay specific TLVs or
655 sub- TLVs. This is possible since CCMs are capable of carrying
656 optional TLVs.
658 Unlike classical Ethernet environments with spanning tree, NVO3
659 supports multipath forwarding. The path taken by a packet depends on
660 the Transport header and other parts of the packet. The Maintenance
661 Association identifies the interested end-points (MEPs) of a given
662 monitored path. For unicast there are only two MEPs per MA. For
663 multicast there can be two or more MEPs in the MA. The Flow (i.e VNI
664 and other parameters) values of the monitored flows are defined
665 within the MA. CCM transmit logic will utilize these flow entropy
666 values when constructing the CCM packets. Please see below for the
667 theory of operation of CCM.
669 The CFM MIB of [8021Q] will be augmented with the definition of
670 flow- entropy. Please see [TBDMIBD] for these and other NVO3 related
671 OAM MIB definitions. The figure below depicts the correlation
672 between MA, CCM and the flow.
674 CCM implementation is Optional.
676 MA---
677 |
678 --- MEP
679 |
680 . - Remote MEP List
681 .
682 |
683 --- MEP-A
684 |
685 --- MEP-B
686 .
688 |
689 . - Flow List {Augments IEEE8021-CFM-MIB}
691 |
692 --- (Flow-1) {note we have to define
693 | destination NVE with
694 --- (Flow-2) the flow entropy discuss}
695 |
696 . ---(Flow-n)
697 |
698 . - CCM
699 |
700 --- (standard 8021ag entries)
701 |
702 --- (TTL) { Augments IEEE8021-CFM-MIB}
703 |
704 --- (Other TBD NVO3 OAM specific entries)
705 {Augmented}
706 |
707 .
708 |
709 - Other MIB entries
711 Figure 9 Augmentation of CCM MIB in NVO3
713 NOTE: Flow entropy field contain VNI and flow specific information
714 that affect the path selection and forwarding.
716 In a multi-pathing environment, a Flow - by definition - is
717 unidirectional. A question may arise as to what flow entropy should
718 be used in the response. CCMs are unidirectional and have no
719 explicit reply; as such, the issue of the response flow entropy does
720 not arise. In the transmitted CCM, each MEP reports local status
721 using the Remote Defect Indication (RDI) flag. Additionally, a MEP
722 may raise SNMP TRAPs [TBDMIB] as Alarms when a connectivity failure
723 occurs.
725 8. NVO3 OAM Message Channel
727 The NVO3 OAM Message Channel can be divided into two parts: NVO3 OAM
728 Message header and NVO3 OAM Message TLVs. Every OAM Message MUST
729 contain a single OAM message header and a set of one or more
730 specified OAM Message TLVs.
732 8.1. NVO3 OAM Message header
734 As discussed earlier, a common messaging framework between [8021Q],
735 NVO3, and other similar standards such as Y.1731 can be accomplished
736 by re-using the OAM message header defined in [8021Q].
738 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
739 |MD-L | Version | OpCode | Flags |FirstTLVOffset |
740 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
741 | |
742 | |
743 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
744 | |
745 | |
746 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
748 Figure 10 OAM Message Format
750 o MD-L: Maintenance Domain Level (3 bits). Identifies the
751 maintenance domain level. For NVO3, in general, this field is
752 set to a single value. If using Base Mode operations as defined
753 in Appendix B, this field is set to 3. However, future
754 extensions of NVO3, for example to support hierarchy, may create
755 different MD-LEVELs and MD-L field must be appropriately set in
756 those scenarios. (Please refer to [8021Q] for the definition of
757 MD-Level) o Version: Indicates the version (5 bits), as specified
758 in [8021Q]. This document does not require changing the Version
759 defined in [8021Q].
761 o Flags: Includes operational flags (1 byte). The definition of
762 flags is Opcode-specific and is covered in the applicable
763 sections.
765 o FirstTLVOffset: Defines the location of the first TLV, in
766 bytes, starting from the end of the FirstTLVOffset field (1
767 byte). (Refer to [8021Q] for the definition of the
768 FirstTLVOffset.)
770 MD-L, Version, Opcode, Flags and FirstTLVOffset fields collectively
771 are referred to as the OAM Message Header.
773 The Opcode specific information section of the OAM Message may
774 contain Session Identification number, time-stamp, etc.
776 8.2. IETF Overlay OAM Opcodes
778 The following Opcodes are defined for IETF Overlay OAM. Each of the
779 Opcodes indicates a separate type of OAM message. Details of the
780 messages are presented in the related sections.
782 IETF OAM Message Opcodes:
784 64 : Path Trace Reply 65 : Path Trace Message 66 : Multicast Tree
785 Verification Reply 67 : Multicast Tree Verification Message
787 8.3. Format of IETF Overlay OAM TLV
789 The same TLV format as defined in section 21.5.1 of [8021Q] is used
790 for IETF Overlay OAM. The following figure depicts the general
791 format of a IETF Overlay OAM TLV:
793 0 1 2
794 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
795 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
796 | Type | Length |
797 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
798 | |
799 | |
800 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
802 Figure 11 NVO3 OAM TLV
804 Type (1 octet): Specifies the Type of the TLV (see sections 8.4.
805 for TLV types).
807 Length (2 octets): Specifies the length of the 'Value' field in
808 octets. Length of the 'Value' field can be either zero or more
809 octets.
811 Value (variable): The length and the content of this field depend on
812 the type of the TLV. Please refer to applicable TLV definitions for
813 the details.
815 Semantics and usage of Type values allocated for TRILL OAM purpose
816 are defined by this document and other future related documents.
818 8.4. IETF Overlay OAM TLVs
820 Overlay related TLVs are defined in this section. Previously defined
821 [8021Q] TLVs are reused, where applicable. Block of 32 TLVs are
822 allocated for the purpose of IETF defined standards
824 8.4.1. Common TLVs between 8201Q CFM and IETF Overlay OAM
826 The following TLVs are defined in [8021Q]. We propose to re-use them
827 where applicable. The format and semantics of the TLVs are as
828 defined in [8021Q].
830 Type Name of TLV in [8021Q]
831 ---- -------------
832 0 End TLV
833 1 Sender ID TLV
834 2 Port Status TLV
835 3 Data TLV
836 4 Interface Status TLV
837 5 Reply Ingress TLV
838 6 Reply Egress TLV
839 7 LTM Egress Identifier TLV
840 8 LTR Egress Identifier TLV
841 9-30 Reserved
842 31 Organization Specific TLV
844 8.4.2. IETF Overaly OAM Specific TLVs
846 As indicated above, a block of 32 TLVs will be requested to be
847 reserved for IETF OAM purposes. Listed below is a summary of the
848 IETF Overlay OAM TLVs and their corresponding codes. Format and
849 semantics of OAM TLVs are defined in subsequent sections.
851 Type TLV Name
852 ----------- ----------------------
853 64 OAM Application Identifier
854 65 Out of Band IP Address
855 66 Original Payload
856 67 Diagnostic VLAN
857 68 scope
858 69 Previous Device address
859 70 Next Hop Device List (ECMP)
860 71 Multicast Receiver Availability
861 72 Flow Identifier
862 73 Reflector Entropy
863 74 to 95 Reserved
865 8.4.3. OAM Application Identifier TLV
867 OAM Application Identifier TLV carries Overlay OAM application
868 specific information. The Overlay OAM Application Identifier TLV
869 MUST always be present and MUST be the first TLV in the OAM
870 messages. Messages that do not include the OAM Application
871 Identifier TLV as the first TLV MUST be discarded.
873 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
874 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
875 | Type | Length | Version |
876 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
877 | Return Code |Return sub-code| Protocol |Rsvd |F|C|O|I|
878 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
880 Figure 12 OAM Application Identifier TLV
882 Type (1 octet) = TBD-TLV-64 indicate that this is the IETF Overlay
883 OAM Application Identifier TLV
885 Length (2 octets) = 6
887 IETF Overlay OAM Version (1 Octet), currently set to zero. Indicates
888 the Overlay OAM version. IETF Overlay OAM version can be different
889 than the [8021Q] version.
891 Return Code (1 Octet): Set to zero on requests. Set to an
892 appropriate value in response messages.
894 Return sub-code (1 Octet): Return sub-code is set to zero on
895 transmission of request message. Return sub-code identifies
896 categories within a specific Return code. Return sub-code MUST be
897 interpreted within a Return code.
899 Protocol: This indicates the overlay protocol on which the OAM is
900 applied. In this document we cover NVO3
902 0 : TRILL
903 1 : NVO3
905 2 - 255 : reserved
907 F (1 bit): Final flag, when set, indicates this is the last
908 response.
910 C (1 bit): Label error, if set indicates that the label (VLAN) in
911 the flow entropy is different than the label included in the
912 diagnostic TLV. This field is ignored in request messages and MUST
913 only be interpreted in response messages.
915 O (1 bit): If set, indicates, OAM out-of-band response requested.
917 I (1 bit): If set, indicates, OAM in-band response requested.
919 NOTE: When both O and I bits are set to zero, indicates that no
920 response is required (silent mode). User MAY specify both O and I or
921 one of them or none. When both O and I bits are set response is sent
922 both in-band and out-of-band.
924 8.4.4. Out Of Band Reply Address TLV
926 Out of Band Reply Address TLV specifies the address to which an
927 out of band OAM reply message MUST be sent. When O bit in the
928 Application Identifier TLV is not set, Out of Band Reply Address
929 TLV is ignored.
931 1 2 3
932 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
933 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
934 | Type | Length | Address Type |
935 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
936 | Addr Length | |
937 +-+-+-+-+-+-+-+-+ |
938 | |
939 . Reply Address .
940 | |
941 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
943 Figure 13 Out of Band IP Address TLV
945 Type (1 octet) = TBD-TLV-65
947 Length (2 octets) = Variable. Minimum length is 2.
949 Address Type (1 Octet): 0 - IPv4. 1 - IPv6. All other values
950 reserved.
952 Addr Length (1 Octet). 4 - IPv4. 16 - IPv6,
954 Reply Address (variable): Address where the reply needed to be sent.
955 Length depends on the address specification.
957 8.4.5. Diagnostics Label TLV
959 Diagnostic label specifies the VNI which the OAM messages are
960 generated. Receiving device MUST compare the VNI of the NVO3 shim to
961 that of Diagnostic TLV. Label Error Flag in the response MUST be set
962 when the two VLANs do not match.
964 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
965 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
966 | Type | Length | L-Type |
967 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
968 | Reserved | Label(VLAN) |
969 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
971 Figure 14 Diagnostic TLV
973 Type (1 octet) = TBD-TLV-66 indicates that this is the Diagnostic
974 TLV
976 Length (2 octets) = 5
978 L-Type (Label type, 1 octet)
980 0- indicate 802.1Q 12 bit VLAN.
982 2 - 24 bit ISID
984 3 - VNI 24 bits
986 Label (24 bits): Either 12 bit VLAN or 24 bit ISID or 24 bit VNI.
988 Devices should perform Label error checking when Label TLV is not
989 included in the OAM message.
991 8.4.6. Original Data Payload TLV
993 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
994 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
995 | Type | Length | |
996 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +
997 | |
998 | |
999 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1001 Figure 15 Original Data Payload TLV
1003 Type (1 Octet) = TBD-TLV-67
1005 Length (2 octets) = variable
1007 8.4.7. Flow Identifier (flow-id) TLV
1009 Flow Identifier (flow-id) uniquely identifies a specific flow. The
1010 flow-id value is unique per MEP and needs to be interpreted as such.
1012 1 2 3
1013 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
1014 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1015 | Type | Length | Reserved |
1016 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1017 | MEP-ID | flow-id |
1018 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1020 Figure 16 Flow Identifier TLV
1022 Type (1 octet) = TBD-TLV-72 Length (2 octets) = 5.
1024 Reserved (1 octet) set to 0 on transmission and ignored on
1025 reception.
1027 MEP-ID (2 octets) = MEP-ID of the originator [8021Q].
1029 Flow-id (2 octets) = uniquely identifies the flow per MEP. Different
1030 MEPs may allocate the same flow-id value. The {MEP-ID, flow-id} pair
1031 is globally unique.
1033 Inclusion of the MEP-ID in the flow-id TLV allows inclusion of MEP-
1034 ID for messages that do not contain MEP-ID in OAM header.
1035 Applications may use MEP-ID information for different types of
1036 troubleshooting.
1038 8.4.8. Reflector Entropy TLV
1040 Reflector Entropy TLV is an optional TLV. This TLV, when present,
1041 tells the responder to utilize the Reflector Entropy specified
1042 within the TLV as the flow-entropy of the response message.
1044 1 2 3
1045 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
1046 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1047 | Type | Length | Reserved |
1048 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1049 | |
1050 . Reflector Entropy .
1051 | |
1052 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1054 Figure 17 Reflector Entropy TLV
1056 Type (1 octet) =TBD-TLV-73 Reflector Entropy TLV.
1058 Length (1 octet) =97.
1060 Reserved (1 octet) = set to zero on transmission and ignored by the
1061 recipient.
1063 Reflector Entropy (96-octet) = Flow Entropy to be used by the
1064 responder. May be padded with zero if the desired flow entropy is
1065 less than 96 octets.
1067 9. Loopback Message
1069 9.1. Loopback OAM Message format
1071 1 2 3
1072 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
1073 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1074 |MD-L | Version | OpCode | Flags |FirstTLVOffset |
1075 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1076 | Loopback Transaction Identifier |
1077 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1078 | |
1079 . TLVs .
1080 | |
1081 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1083 Figure 18 Loopback OAM Message Format
1085 The above figure depicts the format of the Loopback Request and
1086 response messages as defined in [8021Q]. The Opcode for Loopback
1087 Message is set to 65 and the Opcode for the Reply Message is set to
1088 64. The Session Identification Number is a 32-bit integer that
1089 allows the requesting Device to uniquely identify the corresponding
1090 session. Responding Device, without modification, MUST echo the
1091 received "Loopback Transaction Identifier" number..
1093 9.2. Theory of Operation
1095 9.2.1. Actions by Originator
1097 Identifies the destination NVE address based on user specification
1098 or based on the specified destination, VNI and MAC
1100 Constructs the flow entropy based on user specified parameters or
1101 implementation specific default parameters.
1103 Constructs the OAM header: sets the opcode to Loopback message type
1104 (3). Assign applicable Loopback Transaction Identifier number for
1105 the request.
1107 The Overlay OAM Version TLV MUST be included and with the flags set
1108 to applicable values.
1110 Include following OAM TLVs, where applicable
1112 o Out-of-band Reply address TLV
1114 o Diagnostic Label TLV
1116 o Sender ID TLV
1118 Specify the Transport Layer parameters based on user inputs or
1119 default settings.
1121 Dispatch the OAM frame for transmission.
1123 Devices may continue to retransmit the request at periodic
1124 intervals, until a response is received or the re-transmission count
1125 expires. At each transmission Session Identification number MUST be
1126 incremented.
1128 9.2.2. Intermediate Devices
1130 Intermediate Devices forward the frame as a normal data frame and no
1131 special handling is required.
1133 9.2.3. Destination Device
1135 If the Loopback message is addressed to the local Device and
1136 satisfies the OAM identification criteria specified in section 4.2.
1137 then, the Device data plane forwards the message to the CPU for
1138 further processing.
1140 The OAM application layer further validates the received OAM frame
1141 by checking for the presence of OAM-Ethertype and the MD Level.
1142 Frames that do not contain OAM-Ethertype MUST be discarded.
1144 Construction of the OAM response:
1146 OAM application encodes the received Transport header, NVO3 shim and
1147 payload fragment (when present) in the Original payload TLV and
1148 includes it in the OAM message.
1150 Set the Return Code and Return sub code to applicable values. Update
1151 the OAM opcode to 2 (Loopback Message Reply)
1153 Optionally, if the VNI identifier value of the received differs from
1154 the value specified in the diagnostic Label, set the Label Error
1155 Flag on OAM Application Identifier TLV.
1157 Include the sender ID TLV (1)
1159 If in-band response was requested, dispatch the frame to theNVO3
1160 data plane with request-originator Transport Layer address as the
1161 destination address.
1163 If out-of-band response was requested, dispatch the frame to the IP
1164 forwarding process.
1166 10. Path Trace Message
1168 The primary use of the Path Trace Message is for fault isolation. It
1169 may also be used for plotting the path taken from a given Device to
1170 another Device.
1172 [8021Q] accomplishes the objectives of the NVO3 Path Trace Message
1173 using Link Trace Messages. Link Trace Messages utilize a well-known
1174 multicast MAC address. However, in NOV3 the transport Layer can be
1175 different technologies such as IP or MPLS etc. Hence, a definition
1176 of a new Path Trace message format is required for Overlay OAM. The
1177 Path Trace message is defined for the purpose.
1179 The Path Trace Message has the same format as Loopback Message, but
1180 utilizes a different opcode set. Opcode for Path Trace Reply Message
1181 is 64 and Request Message is 65
1183 Operation of the Path Trace message is identical to the Loopback
1184 message except that it is first transmitted with a TTL field value
1185 of 1. The sending device expects a Time Expiry Return-Code from the
1186 next hop or a successful response. If a Time Expiry Return-code is
1187 received as the response, the originator Device records the
1188 information received from intermediate node that generated the Time
1189 Expiry message and resends the message by incrementing the previous
1190 TTL value by 1. This process is continued until, a response is
1191 received from the destination Device or Path Trace process timeout
1192 occur or TTL reaches a configured maximum value.
1194 10.1. Theory of Operation
1196 10.1.1. Action by Originator Device Identifies the destination NVE
1197 address based on user specification or based on the specified
1198 destination, VNI and MAC
1200 Constructs the NVO3 shim and the flow based on user specified
1201 parameters or implementation specific default parameters.
1203 Construct the OAM header: Set the opcode to Path Trace Request
1204 message type (TBD-65). Assign an applicable Session Identification
1205 number for the request. Return-code and sub-code MUST be set to
1206 zero.
1208 The OAM Application Identifier TLV MUST be included and set the
1209 flags to applicable values.
1211 Include following OAM TLVs, where applicable
1213 o Out-of-band IP address TLV
1215 o Diagnostic Label TLV
1217 o Include the Sender ID TLV
1219 Specify the TTL of the transport header as 1 for the first request.
1221 Dispatch the OAM frame to the data plane for transmission.
1223 A Device (MEP) may continue to retransmit the request at periodic
1224 intervals, until a response is received or the re-transmission count
1225 expires. At each new re-transmission, the Session Identification
1226 number MUST be incremented. Additionally, for responses received
1227 from intermediate devices (MIP), the device address and interface
1228 information MUST be recorded.
1230 10.1.2. Intermediate Device
1232 Path Trace Messages transit through Intermediate devices
1233 transparently, unless Hop-count has expired. OAM application layer
1234 further validates the received OAM frame by examining the presence
1235 of NVO3 OAM Flag and OAM-Ethertype and by examining the MD Level.
1236 Frames that do not contain OAM-Ethertype MUST be discarded.
1238 Construction of the OAM response:
1240 OAM application encodes the received Transport header and flow
1241 entropy in the Original payload TLV and include it in the OAM
1242 message.
1244 Set the Return Code to (2) "Time Expired" and Return sub code to
1245 zero (0). Update the OAM opcode to 64 (Path Trace Message Reply).
1247 If the VNI identifier value of the received OAM message differs from
1248 the value specified in the diagnostic Label, set the Label Error
1249 Flag on OAM Application Identifier TLV.
1251 Include following TLVs
1253 Reply Ingress TLV (5)
1255 Reply Egress TLV (6)
1257 Interface Status TLV (4)
1259 Sender ID TLV (1)
1261 If Label error detected, set C flag (Label error detected) in the
1262 version.
1264 If in-band response was requested, dispatch the frame to the NVO3
1265 data plane with request-originator Transport address as the
1266 destination address.
1268 If out-of-band response was requested, dispatch the frame to the
1269 standard IP forwarding process.
1271 10.1.3. Destination Device
1272 Processing is identical to Loop back response processing in section
1273 9.2.3. with the exception that OAM Opcode is set to Path Trace Reply
1274 (64).
1276 11. Link Trace Message
1278 Link Trace Message (LTM/LTR) procedure is defined in detail in
1279 [8021Q]. In this section I am covering the summary.
1281 Link Trace Message are to be used when operator network all switches
1282 are OAM capable. In this scenario we will recommend 8021Q port based
1283 model for Link Trace Message and Reply Procedure as Bridge Brain is
1284 same as Path Trace message.
1286 SS1 SS2 (L3 VTEP)
1287 / | \/ \
1288 / | /\ \ (ECMP paths)
1289 / |/ \ \
1290 S1 S2 S3
1291 || || || (Underlay ECMP)
1292 || || ||
1293 L1 L2 L3 (L2 VTEP)
1295 | |
1296 H1 H2
1298 Figure 19 Payload Fragment use case
1300 In Figure 19, if all switches in operator network are OAM capable
1301 then hardware friendly and accurate OAM solution for Fault isolation
1302 will be Link Trace.
1304 11.1 MEP and MIP
1306 By default if network is OAM capable all switch has virtual default
1307 MEP created on the NVE and MIP created on all fabric facing
1308 interface.
1310 MEP can initiate and terminate the OAM message. MIP is stateless and
1311 passive and only Reply to OAM Message.
1313 11.2 Initiator
1315 Initiator Device generate Link Trace Message as per procedure
1316 described in [8021Q] and encapsulated as per the draft.
1318 As transit packet will hit the MIP on the egress port, a copy of
1319 packet is punted to cpu to generate Link Trace Reply (LTR) to the
1320 originating MEP and packet continue forwarding in hardware.
1322 11.3 Intermediate Devices
1324 Intermediate devices has MIP configured on all the fabric
1325 Interfaces.
1327 |-------------|
1328 . .
1329 | |
1330 MIP MIP
1331 ->--- Device --->-
1332 |
1333 |-------------|
1335 Figure 20 MIP configuration and traffic Flow
1337 In above figure traffic is entering from left side of device hitting
1338 first MIP and copy of packet is punted to cpu to generate Link Trace
1339 Reply and LTM continue forwarding in hardware.
1341 LTM hits right side MIP while exiting out of the device and copy of
1342 the packet is punted to cpu to generate Link Trace Reply and LTM
1343 continue forwarding.
1345 11.4 Terminating Device
1347 Terminating device has MIP and MEP configured, and on ingress
1348 interface packet will hit the MIP and copy of it's punted to
1349 generate LTR and packet continue forwarding in hardware.
1351 Packet then hits the terminating MEP and LTR is generated.
1353 11.5 Output
1355 If for example path was like below
1357 A Eth1 -- Eth2 B Eth3 --- Eth4 C
1359 Link trace Message is generated from A towards C.
1361 LTR will be received from
1362 Eth1 A
1363 Eth2 B
1364 Eth3 B
1365 Eth4 C
1366 MEP C
1368 In this way complete path is tracked in hardware forwarding.
1370 12. Application of Continuity Check Message (CCM) in NVO3
1372 Section 7. provides an overview of CCM Messages defined in [8021Q]
1373 and how they can be used within the NVO3 OAM. This section, presents
1374 the application and Theory of Operations of CCM within the NVO3 OAM
1375 framework. Readers are referred to [8021Q] for CCM message format
1376 and applicable TLV definitions and usages. Only the NVO3 specific
1377 aspects are explained below.
1379 In NVO3, between any two given MEPs there can be multiple potential
1380 paths. Whereas in [8021Q], there is always a single path between any
1381 two MEPs at any given time. It is important that solutions to have
1382 the ability to monitor continuity over one or more paths.
1384 CCM Messages are uni-directional, such that there is no explicit
1385 response to a received CCM message. Connectivity fault status is
1386 indicated by setting the applicable flags (e.g. RDI) of the CCM
1387 messages transmitted by an MEP.
1389 It is important that the solution presented in this document
1390 accomplishes the requirements specified in [NVO3OAMREQ] within the
1391 framework of [8021Q] in a straightforward manner and with minimum
1392 changes. Section 8 above defines multiple flows within the CCM
1393 object, each corresponding to a flow that a given MEP wishes to
1394 monitor.
1396 Receiving MEPs do not cross check whether a received CCM belongs to
1397 a specific flow from the originating MEP. Any attempt to track
1398 status of individual flows may explode the amount of state
1399 information that any given MEP has to maintain.
1401 The obvious question arises: How does the originating device know
1402 which flow or flows are at fault?
1404 This is accomplished with a combination of the RDI flag in the CCM
1405 header, flow-id TLV, and SNMP Notifications (Traps). Section 11.1
1406 below discusses the procedure.
1408 12.1. CCM Error Notification
1410 Each MEP transmits 4 CCM messages per each flow. ([8021Q] detects
1411 CCM fault when 3 consecutive CCM messages are lost). Each CCM
1412 Message has a unique sequence number and unique flow-identifier. The
1413 flow identifier is included in the OAM message via flow-id TLV.
1415 When an MEP notices a CCM timeout from a remote MEP ( MEP-A), it
1416 sets the RDI flag on the next CCM message it generates.
1417 Additionally, it logs and sends SNMP notification that contain the
1418 remote MEP Identification, flow-id and the Sequence Number of the
1419 last CCM message it received and if available, the flow-id and the
1420 Sequence Number of the first CCM message it received after the
1421 failure. Each MEP maintains a unique flow-id per each flow, hence
1422 the operator can easily identify flows that correspond to the
1423 specific flow-id.
1425 The following example illustrates the above.
1427 Assume there are two MEPs, MEP-A and MEP-B.
1429 Assume there are 3 flows between MEP-A and MEP-B.
1431 Let's assume MEP-A allocates sequence numbers as follows
1433 Flow-1 Sequence={1,2,3,4,13,14,15,16,.. } flow-id=(1)
1435 Flow-2 Sequence={5,6,7,8,17,18,19,20,.. } flow-id=(2)
1437 Flow-3 Sequence={9,10,12,11,21,22,23,24,.. } flow-id=(3)
1439 Let's Assume Flow-2 is at fault.
1441 MEP-B, receives CCM from MEP-A with sequence numbers 1,2,3,4, but
1442 did not receive 5,6,7,8. CCM timeout is set to 3 CCM intervals in
1443 [8021Q]. Hence MEP-B detects the error at the 8'th CCM message. At
1444 this time the sequence number of the last good CCM message MEP-B has
1445 received from MEP-A is 4 and flow-id of the last good CCM Message is
1446 (1). Hence MEP-B will generate a CCM error SNMP notification with
1447 MEP-A and Last good flow-id (1) and sequence number 4.
1449 When MEP-A switches to flow-3 after transmitting flow-2, MEP-B will
1450 start receiving CCM messages. In the foregoing example it will be
1451 CCM message with Sequence Numbers 9,10,11,12,21 and so on. When in
1452 receipt of a new CCM message from a specific MEP, after a CCM
1453 timeout, the NVO3 OAM will generate an SNMP Notification of CCM
1454 resume with remote MEP-ID and the first valid flow-id and the
1455 Sequence number after the CCM timeout. In the foregoing example, it
1456 is MEP-A, flow-id (3) and Sequence Number 9.
1458 The remote MEP list under the CCM MIB Object is augmented to contain
1459 "Last Sequence Number", flow-id and "CCM Timeout" variables. Last
1460 Sequence Number and flow-id are updated every time a CCM is received
1461 from a remote MEP. CCM Timeout variable is set when the CCM timeout
1462 occurs and is cleared when a CCM is received.
1464 12.2. Theory of Operation
1466 12.2.1. Actions by Originator Device
1468 Derive the VNI and entropy based on flow entropy specified in the
1469 CCM Management object.
1471 Construct the NVO3 CCM OAM header as specified in [8021Q].
1473 OAM Version TLV MUST be included as the first TLV and set the flags
1474 to applicable values.
1476 Include other TLVs specified in [8021Q]
1478 Include the following optional OAM TLVs, where applicable
1480 o Sender ID TLV
1482 Specify the TTL of the NVO3 data frame per user specification or
1483 utilize an applicable Hop count value.
1485 Dispatch the OAM frame to the NVO3 data plane for transmission.
1487 An MEP transmits a total of 4 requests, each at CCM retransmission
1488 interval. At each transmission the Session Identification number
1489 MUST be incremented by one.
1491 At the 5'th retransmission interval, flow entropy of the CCM packet
1492 is updated to the next flow entropy specified in the CCM Management
1493 Object. If current flow entropy is the last flow entropy specified,
1494 move to the first flow entropy specified and continue the process.
1496 12.2.2. Intermediate Devices
1498 Intermediate devices forward the frame as a normal data frame and
1499 no special handling is required.
1501 12.2.3. Destination Device
1503 If the CCM Message is addressed to the local MEP or multicast and
1504 satisfies OAM identification methods specified in sections 4.2.
1505 then the data plane forwards the message to the CPU for further
1506 processing.
1508 The OAM application layer further validates the received OAM frame
1509 by examining the presence of OAM-Ethertype at the end of the flow
1510 entropy. Frames that do not contain OAM-Ethertype at the end of the
1511 flow entropy MUST be discarded.
1513 Validate the MD-LEVEL and pass the packet to the Opcode de-
1514 multiplexer. The Opcode de-multiplexer delivers CCM packets to the
1515 CCM process.
1517 The CCM Process performs processing specified in [8021Q].
1519 Additionally the CCM process updates the CCM Management Object with
1520 the sequence number of the received CCM packet. Note: The last
1521 received CCM sequence number and CCM timeout are tracked per each
1522 remote MEP.
1524 If the CCM timeout is true for the sending remote MEP, then clear
1525 the CCM timeout in the CCM Management object and generate the SNMP
1526 notification as specified above.
1528 13. Security Considerations
1530 Specific security considerations related methods presented in this
1531 document are currently under investigation.
1533 14. IANA Considerations
1535 Re-use the Type and TLV from RFC7455.
1537 15. References
1539 15.1. Normative References
1541 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
1542 Requirement Levels", BCP 14, RFC 2119, March 1997.
1544 [8021Q] IEEE, "Media Access Control (MAC) Bridges and Virtual
1545 Bridged Local Area Networks", IEEE Std 802.1Q-2011, August,
1546 2011.
1548 15.2. Informative References
1550 [RFC4379] Kompella, K. et.al, "Detecting Multi-Protocol Label
1551 Switched (MPLS) Data Plane Failures", RFC 4379, February
1552 2006.
1554 [RFC6291] Andersson, L., et.al., "Guidelines for the use of the
1555 "OAM" Acronym in the IETF" RFC 6291, June 2011.
1557 [Y1731] ITU, "OAM functions and mechanisms for Ethernet based
1558 networks", ITU-T G.8013/Y.1731, July, 2011.
1560 [NVO3FRM] Lasserre, M., et.al., "Framework for DC Network
1561 Virtualization", Work in Progress, January, 2014.
1563 [NVO3ARC] Black, D., et.al., "Architecture for Overlay Networks
1564 (NVO3)", Work in Progress, December, 2013.
1566 [NVO3OAMREQ] Ashwood-Smith, P., "NVO3 Operations, Administrations
1567 and Maintenance Requirements", work in progress, January,
1568 2014.
1570 [NV03DPREQ] Bitar, N., "NVO3 Data Plane Requirements", Work in
1571 Progress, November, 2013.
1573 16. Acknowledgments
1575 This document was prepared using 2-Word-v2.0.template.dot.
1577 Appendix A. Base Mode for NVO3 OAM
1579 CFM, as defined in [8021Q], requires configuration of several
1580 parameters before the protocol can be used. These parameters include
1581 MAID, Maintenance Domain Level (MD-LEVEL) and MEPIDs. The Base Mode
1582 for NVO3 OAM defined here facilitates ease of use and provides out
1583 of the box plug-and-play capabilities.
1585 All NVO3 compliant devices that support OAM specified in this
1586 document MUST support Base Mode operation.
1588 All NVO3 compliant devices that support OAM MUST create a default MA
1589 with MAID as specified herein.
1591 MAID [8021Q] has a flexible format and includes two parts:
1592 Maintenance Domain Name and Short MA name. In the Based Mode of
1593 operation, the value of the Maintenance Domain Name must be the
1594 character string "NVO3BaseMode" (excluding the quotes "). In Base
1595 Mode operation Short MA Name format is set to 2-octet integer format
1596 (value 3 in Short MA Format field) and Short MA name set to 65532
1597 (0xFFFC).
1599 The Default MA belongs to MD-LEVEL 3.
1601 In the Base Mode of operation, each NVO3 device creates a single UP
1602 MEP associated with a virtual OAM port located within the CPU with
1603 no physical layer (NULL PHY). The MEPID associated with this MEP is
1604 the 2-octet VNI Nickname.
1606 By default, with the base mode OAM in place, all NVO3 devices
1607 operating in the Base Mode for NVO3 OAM are able to initiate LBM, PT
1608 and other OAM tools with no configuration.
1610 Implementation MAY provide default flow to be included in OAM
1611 messages. Content of the default flow is outside the scope of this
1612 document.
1614 Figure 18, below depicts encoding of MAID within the CCM messages.
1616 +-+-+-+-+-+-+-+-+-+-+-+-+-+
1617 |Field Name |Size |
1618 | | |
1619 +-+-+-+-+-+-+-+-+-+-+-+-+-+
1620 |Maintenance | 1 |
1621 |Domain Format | |
1622 +-+-+-+-+-+-+-+-+-+-+-+-+-+
1623 |Maintenance | 2 |
1624 |Domain Length | |
1625 +-+-+-+-+-+-+-+-+-+-+-+-+-+
1626 |Maintenance | variable|
1627 |Domain Name | |
1628 +-+-+-+-+-+-+-+-+-+-+-+-+-+
1629 |Short MA | 1 |
1630 |Name Format | |
1631 +-+-+-+-+-+-+-+-+-+-+-+-+-+
1632 |Short MA | 2 |
1633 |Name Length | |
1634 +-+-+-+-+-+-+-+-+-+-+-+-+-+
1635 |Short MA | variable|
1636 |Name | |
1637 +-+-+-+-+-+-+-+-+-+-+-+-+-+
1638 |Padding | Variable|
1639 +-+-+-+-+-+-+-+-+-+-+-+-+-+
1641 Figure 18 MAID structure as defined in [8021Q]
1643 Maintenance Domain Name Format is set to Value: 4
1645 Maintenance Domain Name Length is set to value: 13
1647 Maintenance Domain Name is set to: NVO3BaseMode
1649 Short MA Name Format is set to value: 3
1651 Short MA Name Length is set to value: 2
1653 Shirt MA Name is set to : FFFC
1655 Padding : set of zero up to 48 octets of total length of the MAID.
1657 Please refer to [8021Q] for details.
1659 Authors' Addresses
1660 Tissa Senevirathne
1661 CISCO Systems
1662 375 East Tasman Drive.
1663 San Jose, CA 95134
1664 USA.
1666 Phone: +1 408-853-2291
1667 Email: tsenevir@gmail.com
1669 Norman Finn
1670 CISCO Systems
1671 510 McCarthy Blvd
1672 Milpitas, CA 95035
1673 USA
1675 Email: nfinn@cisco.com
1677 Samer Salam
1678 CISCO Systems
1679 595 Burrard St. Suite 2123
1680 Vancouver, BC V7X 1J1, Canada
1682 Email: ssalam@cisco.com
1684 Deepak Kumar
1685 CISCO Systems
1686 510 McCarthy Blvd,
1687 Milpitas, CA 95035, USA
1689 Phone : +1 408-853-9760
1690 Email: dekumar@cisco.com
1692 Donald Eastlake
1693 Huawei Technologies
1694 155 Beaver Street
1695 Milford, MA 01757
1697 Phone: +1-508-333-2270
1698 Email: d3e3e3@gmail.com
1700 Sam Aldrin
1701 Huawei Technologies
1702 2330 Central Express Way
1703 Santa Clara, CA 95951
1704 USA
1706 Email: aldrin.ietf@gmail.com