idnits 2.17.1 draft-ashwood-nvo3-oam-requirements-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 19, 2015) is 3105 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-07) exists of draft-ietf-nvo3-security-requirements-05 Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 NVO3 Working Group H. Chen, Ed. 3 INTERNET-DRAFT P. Ashwood-Smith 4 Intended Status: Informational L. Xia 5 Huawei Technologies 6 R. Iyengar 7 T. Tsou 8 Huawei Technologies USA 9 A. Sajassi 10 Cisco Technologies 11 M. Boucadair 12 C. Jacquenet 13 France Telecom 14 M. Daikoku 15 KDDI corporation 16 A. Ghanwani 17 Dell 18 R. Krishnan 19 Brocade 20 Expires: April 21, 2016 October 19, 2015 22 NVO3 Operations, Administration, and Maintenance Requirements 23 draft-ashwood-nvo3-oam-requirements-04 25 Abstract 27 This document provides framework and requirements for Network 28 Virtualization Overlay (NVO3) Operations, Administration, and 29 Maintenance (OAM). 31 Status of this Memo 33 This Internet-Draft is submitted to IETF in full conformance with the 34 provisions of BCP 78 and BCP 79. 36 Internet-Drafts are working documents of the Internet Engineering 37 Task Force (IETF), its areas, and its working groups. Note that 38 other groups may also distribute working documents as 39 Internet-Drafts. 41 Internet-Drafts are draft documents valid for a maximum of six months 42 and may be updated, replaced, or obsoleted by other documents at any 43 time. It is inappropriate to use Internet-Drafts as reference 44 material or to cite them other than as "work in progress." 46 The list of current Internet-Drafts can be accessed at 47 http://www.ietf.org/1id-abstracts.html 48 The list of Internet-Draft Shadow Directories can be accessed at 49 http://www.ietf.org/shadow.html 51 Copyright and License Notice 53 Copyright (c) 2015 IETF Trust and the persons identified as the 54 document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's Legal 57 Provisions Relating to IETF Documents 58 (http://trustee.ietf.org/license-info) in effect on the date of 59 publication of this document. Please review these documents 60 carefully, as they describe your rights and restrictions with respect 61 to this document. Code Components extracted from this document must 62 include Simplified BSD License text as described in Section 4.e of 63 the Trust Legal Provisions and are provided without warranty as 64 described in the Simplified BSD License. 66 Table of Contents 68 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 69 1.1. OSI Definitions of OAM . . . . . . . . . . . . . . . . . . 4 70 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 6 71 1.3. Relationship with Other OAM Work . . . . . . . . . . . . . 6 72 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 7 73 3. NVO3 Reference Model . . . . . . . . . . . . . . . . . . . . . 7 74 4. OAM Framework for NVO3 . . . . . . . . . . . . . . . . . . . . 8 75 4.1. OAM Layering . . . . . . . . . . . . . . . . . . . . . . . 9 76 4.2. OAM Domains . . . . . . . . . . . . . . . . . . . . . . . 9 77 5. NVO3 OAM Requirements . . . . . . . . . . . . . . . . . . . . 10 78 5.1. Discovery . . . . . . . . . . . . . . . . . . . . . . . . 10 79 5.2. Connectivity Fault Management . . . . . . . . . . . . . . 10 80 5.2.1. Connectivity Fault Detection . . . . . . . . . . . . . 10 81 5.2.2. Connectivity Fault Verification . . . . . . . . . . . 11 82 5.2.3. Connectivity Fault localization . . . . . . . . . . . 11 83 5.2.4. Connectivity Fault Notification and Alarm 84 Suppression . . . . . . . . . . . . . . . . . . . . . 11 85 5.3. Connectivity Performance Management . . . . . . . . . . . 11 86 5.3.1. Frame Loss . . . . . . . . . . . . . . . . . . . . . . 11 87 5.3.2. Frame Delay . . . . . . . . . . . . . . . . . . . . . 11 88 5.3.3. Frame Delay Variation . . . . . . . . . . . . . . . . 11 89 5.3.4. Frame Throughput . . . . . . . . . . . . . . . . . . . 12 90 5.3.5. Frame Discard . . . . . . . . . . . . . . . . . . . . 12 91 5.4. Continuity Check . . . . . . . . . . . . . . . . . . . . . 12 92 5.5. Availability . . . . . . . . . . . . . . . . . . . . . . . 12 93 5.6. Data Path Forwarding . . . . . . . . . . . . . . . . . . . 12 94 5.7. Scalability . . . . . . . . . . . . . . . . . . . . . . . 13 95 5.8. Extensibility . . . . . . . . . . . . . . . . . . . . . . 13 96 5.9. Security . . . . . . . . . . . . . . . . . . . . . . . . . 13 97 5.10. Transport Independence . . . . . . . . . . . . . . . . . 14 98 5.11. Application Independence . . . . . . . . . . . . . . . . 14 99 5.12. Prioritization . . . . . . . . . . . . . . . . . . . . . 14 100 5.13. Logging and Traceability Requirements . . . . . . . . . . 14 101 5.14. Live Traffic Monitoring . . . . . . . . . . . . . . . . . 16 102 6. Items for Further Discussion . . . . . . . . . . . . . . . . . 16 103 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 18 104 8. Security Considerations . . . . . . . . . . . . . . . . . . . . 18 105 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 18 106 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 18 107 10.1 Normative References . . . . . . . . . . . . . . . . . . . 18 108 10.2 Informative References . . . . . . . . . . . . . . . . . . 18 110 1. Introduction 112 This document provides framework and requirements for Network 113 Virtualization Overlay (NVO3) Operations, Administration, and 114 Maintenance (OAM). Given that this OAM subject is far from new and 115 has been under extensive investigation by various IETF working groups 116 (and several other standards bodies) for many years, this document 117 draws from existing work, starting with [RFC6136]. As a result, 118 sections of [RFC6136] have been reused with minor changes with the 119 permission of the authors. 121 NVO3 OAM requirements are expected to be a subset of IETF/IEEE etc. 122 work done so far; however, we begin with a full set of requirements 123 and expect to prune them through several iterations of this document. 125 1.1. OSI Definitions of OAM 127 The scope of OAM for any service and/or transport/network 128 infrastructure technologies can be very broad in nature. OSI has 129 defined the following five generic functional areas commonly 130 abbreviated as "FCAPS" [NM-Standards]: 132 o Fault Management, 134 o Configuration Management, 136 o Accounting Management, 138 o Performance Management, and 140 o Security Management. 142 This document focuses on the Fault, Performance and to a limited 143 extent the Configuration Management aspects. Other functional 144 aspects of FCAPS and their relevance (or not) to NVO3 are for further 145 study. 147 Fault Management can typically be viewed in terms of the following 148 categories: 150 o Fault Detection; 152 o Fault Verification; 154 o Fault Isolation; 156 o Fault Notification and Alarm Suppression; 157 o Fault Recovery. 159 Fault detection deals with mechanism(s) that can detect both hard 160 failures such as link and device failures, and soft failures, such as 161 software failure, memory corruption, misconfiguration, etc. Fault 162 detection relies upon a set of mechanisms that first allow the 163 observation of an event, then the use of a protocol to dynamically 164 notify a network/system operator (or management system) about the 165 event occurrence, then the use of diagnostic tools to assess the 166 nature and severity of the fault. 168 After verifying that a fault has occurred along the data path, it is 169 important to be able to isolate the fault to the level of a given 170 device or link. Therefore, a fault isolation mechanism is needed in 171 Fault Management. A fault notification mechanism should be used in 172 conjunction with a fault detection mechanism to notify the devices 173 upstream and downstream to the fault detection point. The fault 174 notification mechanism should also notify NMS systems. 176 The terms "upstream" and "backward" are used here to denote the 177 direction(s) from which data traffic is flowing. The terms 178 "downstream" and "forward" denote the direction(s) to which data 179 traffic is forwarded. 181 For example, when there is a client/server relationship between two 182 layered networks (e.g., the NVO3 layer is a client of the outer IP 183 server layer, while the inner IP layer is a client of the NVO3 server 184 layer 2), fault detection at the server layer may result in the 185 following fault notifications: 187 o Sending a forward fault notification from the server layer to the 188 client layer network(s) using the fault notification format 189 appropriate to the client layer. 191 o Sending a backward fault notification to the server layer, if 192 applicable, in the reverse direction. 194 o Sending a backward fault notification to the client layer, if 195 applicable, in the reverse direction. 197 Finally, fault recovery deals with recovering from the detected 198 failure by switching to an alternate available data path (depending 199 on the nature of the fault) using alternate devices or links. In 200 fact, the controller can provision another virtual network, thus 201 automatically resolving the reported problem. 203 The controller may also directly monitor the status of virtual 204 network components such as Network Virtualization Edge elements 205 (NVEs) [RFC7365] in order to respond to their failures. In addition 206 to forward and backward fault notifications, the controller may 207 deliver notifications to a higher level orchestration component, 208 e.g., one responsible for Virtual Machine (VM) provisioning and 209 management. 211 Note, given that the IP network on which NVO3 resides is usually self 212 healing, it is expected that recovery by the NVO3 layer would not 213 normally be required, although there may be a requirement for that 214 layer to log that the problem has been detected and resolved. The 215 special cases of a static IP overlay network, or possibly of a 216 centrally controlled IP overlay network, may, however, require NVO3 217 involvement in fault recovery. 219 Performance Management deals with mechanism(s) that allow determining 220 and measuring the performance of the network/services under 221 consideration. Performance Management can be used to verify the 222 compliance to both the service-level and network-level metric 223 objectives/specifications. Performance Management typically consists 224 of measuring performance metrics, e.g., Frame Loss, Frame Delay, 225 Frame Delay Variation (aka Jitter), Frame Throughput, Frame Discard, 226 etc., across managed entities when the managed entities are in 227 available state. Performance Management is suspended across 228 unavailable managed entities. 230 1.2. Requirements Language 232 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 233 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 234 document are to be interpreted as described in [RFC2119]. 236 1.3. Relationship with Other OAM Work 238 This document leverages requirements that originate with other OAM 239 work, specifically the following: 241 o [RFC6136] provides a template and some of the high level 242 requirements and introductory wording. 244 o [IEEE802.1Q-2011] is expected to provide a subset of the 245 requirements for NVO3 both at the Tenant level and also within the 246 L3 Overlay network. 248 o [Y.1731] is expected to provide a subset of the requirements for 249 NVO3 at the Tenant level. 251 o Section 3.3.2.1 of [NVO3-DP-Reqs] lists several requirements 252 specifically concerning ECMP/LAG. 254 2. Terminology 256 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 257 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" 258 in this document are to be interpreted as described in RFC 2119 259 [RFC2119]. 261 The terminology defined in [RFC7365] and [NVO3-DP-Reqs] is used 262 throughout this document. We introduce no new terminology. 264 3. NVO3 Reference Model 266 Figure 1 below reproduces the generic NVO3 reference model as per 267 [RFC7365]. 269 +--------+ +--------+ 270 | Tenant | | Tenant | 271 | End +--+ +---| End | 272 | System | | | | System | 273 +--------+ | ............... .... | +--------+ 274 | +-+--+ +--+-+ | 275 | | NV | | NV | | 276 +--|Edge| L3 Overlay |Edge|--+ 277 +-+--+ Network +--+-+ 278 / . . \ 279 +--------+ / . +----------+ . \ +--------+ 280 | Tenant +--+ . |Controller| . +----| Tenant | 281 | End | . |(optional)| . | End | 282 | System | . +-------+--+ . | System | 283 +--------+ . . . +--------+ 284 . +----+ . . 285 .....| NV |......... 286 |Edge| 287 +----+ 288 | 289 | 290 +--+-----+ 291 | Tenant | 292 | End | 293 | System | 294 +--------+ 296 Figure 1: Generic NVO3 Reference Model 298 Figure 2 below, reproduces the Generic reference model for the NV 299 Edge (NVE) as per [NVO3-DP-Reqs]. 301 +-----------------+ 302 | Controller |(optional) 303 +--------+--------+ 304 | 305 | 306 +------- L3 Network ------+ 307 | | 308 | Tunnel Overlay | 309 +------------+---------^-+ +--------+-------------^-+ 310 | +----------+------+ | | | +------+----------+ | | 311 | | Overlay Module | | | | | Overlay Module | | | 312 | +--------+--------+ | | | +--------+--------+ | | 313 | | VNID | | | | VNID | | 314 | | OAM | | | OAM | 315 | +-------+-------+ | | | +-------+-------+ | | 316 | | VNI | | | | | VNI | | | 317 NVE1 | +-+-----------+-+ | | NVE2 | +-+-----------+-+ | | 318 | | VAPs | | | | | VAPs | | | 319 +----+-----------+-----V-+ +----+-----------+-----V-+ 320 | | | | 321 -------+-----------+--------------------+-----------+-------- 322 | | Tenant | | 323 | | Service IF | | 324 Tenant End Systems Tenant End Systems 326 Figure 2: Generic reference model for the NV Edge (NVE) 328 4. OAM Framework for NVO3 330 Figure 1 shows the generic reference model for a DC network 331 virtualization over an L3 (or L3VPN) infrastructure while Figure 2 332 showed the generic reference model for the Network Virtualization 333 (NV) Edge. As shown in both figure 1 and figure 2, the Controller 334 is an optional element that can participate to the support and the 335 operation of OAM functions. 337 L3 network(s) or L3 VPN networks (either IPv6 or IPv4, or a 338 combination thereof), provide transport for an emulated layer 2 339 created by NV Edge devices. Unicast and multicast tunneling 340 methods (de-multiplexed by Virtual Network Identifier (VNID)) are 341 used to provide connectivity between the NV Edge devices. The NV 342 Edge devices then present an emulated layer 2 network to the 343 Tenant End Systems at a Virtual Network Interface (VNI) through 344 Virtual Access Points (VAPs). The NV Edge devices map layer 2 345 unicast to layer 3 unicast point-to-point tunnels and may either 346 map layer 2 multicast to layer 3 multicast tunnels or may 347 replicate packets onto multiple layer 3 unicast tunnels. 349 4.1. OAM Layering 351 The emulated layer 2 network is provided by the NV Edge devices to 352 which the Tenant End Systems are connected. This network of NV 353 Edges can be operated by a single service provider or can span 354 across multiple administrative domains. Likewise, the L3 Overlay 355 Network can be operated by a single service provider or span 356 across multiple administrative domains. 358 While each of the layers is responsible for its own OAM, each 359 layer may consist of several different administrative domains. 360 Figure 3 shows an example. 362 TENANT |----------------------------| TENANT 364 NV Edge |----------------------| NV Edge 366 IP(VPN) |---| IP (VPN) |---| IP(VPN) 368 Figure 3: Example NVO3 OAM Layering 370 For example, at the bottom, at the L3 IP overlay network layer 371 IP(VPN) and/or Ethernet OAM mechanisms are used to probe link by 372 link, node to node etc. OAM addressing here means physical node 373 loopback or interface addresses. 375 Further up, at the NV Edge layer, NVO3 OAM messages are used to 376 probe the NV Edge to NV Edge tunnels and NV Edge entity status. 377 OAM addressing here likely means the physical node loopback 378 together with the VNI (to de-multiplex the tunnels). 380 Finally, at the Tenant layer, the IP and/or Ethernet OAM 381 mechanisms are again used but here they are operating over the 382 logical L2/L3 provided by the NV-Edge through the VAP. OAM 383 addressing at this layer deals with the logical interfaces on 384 Vswitches and Virtual Machines. 386 4.2. OAM Domains 388 Complex OAM relationships exist as a result of the hierarchical 389 layering of responsibility and of breaking up of end-to-end 390 responsibility. 392 The OAM domain above NVO3, is expected to be supported by existing 393 IP and L2 OAM methods and tools. 395 The OAM domain below NVO3, is expected to be supported by existing 396 IP/L2 and MPLS OAM methods and tools. Where this layer is actually 397 multiple domains spliced together, the existing methods to deal 398 with these boundaries are unchanged. Note however that exposing 399 LAG/ECMP detailed behavior may result in additional requirements 400 to this domain, the details of which will be specified in the 401 future versions of this draft. 403 When we refer to an OAM domain in this document, or just 'domain', 404 we therefore refer to a closed set of NV Edges or MEPs and the 405 tunnels which interconnect them. 407 Note, whether for the scenario of inter-domain or multi-layer, 408 each domain (or layer) is responsible for its own OAM, no 409 correlation of OAM function exists between each domain (or layer). 410 When an E2E connection in Tenant layer spans across multiple 411 domains and has multiple underlay layers of NV Edge layer and L3 412 IP (VPN) layer, current OAM implementation for the E2E connection 413 of Tenant layer such as Fault or Performance Management can only 414 be performed per domain and layer manually and more manual labor 415 is needed. An automatic coordination process among OAM functions 416 of each domain or layer may be useful here for improving 417 efficiency and intelligence. 419 In the case where a gateway device is use to connect two different 420 domains (whether for changing the encapsulation or other reasons), 421 it is necessary to provide mechanisms to monitor the path through 422 the gateway which involves the removal of one overlay header and 423 the creation of a new one. 425 5. NVO3 OAM Requirements 427 5.1. Discovery 429 R1) NVO3 OAM MUST allow an NV Edge device to dynamically discover 430 other NV Edge devices that share the same VNI within a given NVO3 431 domain. This may be based on a discovery mechanism used to set up 432 data path forwarding between NVEs. 434 5.2. Connectivity Fault Management 436 5.2.1. Connectivity Fault Detection 438 R2) NVO3 OAM MUST allow proactive connectivity monitoring between 439 two or more NV Edge devices that support the same VNIs within a 440 given NVO3 domain. NVO3 OAM MAY act as a protection trigger. 441 That is, automatic recovery from transmission facility failure by 442 switchover to a redundant replacement facility may be triggered by 443 notifications from NVO3 OAM. 445 R3) NVO3 OAM MAY allow monitoring/tracing of all possible paths in 446 the underlay network between a specified set of two or more NV 447 Edge devices. Using this feature, equal cost paths that traverse 448 LAG and/or ECMP may be differentiated. 450 5.2.2. Connectivity Fault Verification 452 R4) NVO3 OAM MUST allow connectivity fault verification between 453 two or more NV Edge devices that support the same VNI within a 454 given NVO3 domain. 456 5.2.3. Connectivity Fault localization 458 R5) NVO3 OAM MUST allow connectivity fault localization between 459 two or more NV Edge devices that support the same VNI within a 460 given NVO3 domain. 462 5.2.4. Connectivity Fault Notification and Alarm Suppression 464 R6) NVO3 OAM MUST support fault notification to be triggered as a 465 result of the faults occurring in the underneath network 466 infrastructure. This fault notification SHOULD be used for the 467 suppression of redundant service-level alarms. 469 5.3. Connectivity Performance Management 471 5.3.1. Frame Loss 473 R7) NVO3 OAM MUST support measurement of per VNI frame loss 474 between two NV Edge devices that support the same VNI within a 475 given NVO3 domain. 477 5.3.2. Frame Delay 479 R8) NVO3 OAM MUST support measurement of per VNI two-way frame 480 delay between two NV edge devices that support the same VNI within 481 a given NVO3 domain. 483 R9) NVO3 OAM MUST support measurement of per VNI one-way frame 484 delay between two NV Edge devices that support the same VNI within 485 a given NVO3 domain. 487 5.3.3. Frame Delay Variation 489 R10) NVO3 OAM MUST support measurement of per VNI frame delay 490 variation between two NV Edge devices that support the same VNI 491 within a given NVO3 domain. 493 5.3.4. Frame Throughput 495 R11) NVO3 OAM MAY support measurement of per VNI frame throughput 496 (in frames and bytes) between two NV Edge devices that support the 497 same VNI within a given NVO3 domain. This feature could be an 498 effective way to confirm whether or not assigned path bandwidth 499 conforms to service level agreement before providing the path 500 between two NV Edge devices. 502 5.3.5. Frame Discard 504 R12) NVO3 OAM MAY support measurement of per VNI frame discard 505 between two NV Edge devices that support the same VNI within a 506 given NVO3 domain. This feature MAY be effective to monitor bursty 507 traffic between two NV Edge devices. 509 5.4. Continuity Check 511 NVO3 OAM MUST provide functions that allow any arbitrary NV edge 512 device to perform a Continuity Check to any other NV edge device. 514 NVO3 OAM MUST provide functions that allow any arbitrary NV edge 515 device to perform a Continuity Check to any other NV edge device 516 using a path associated with a specified flow. 518 NVO3 OAM SHOULD provide functions that allow any arbitrary NV edge 519 device to perform a Continuity Check to any other NV edge device 520 over any section of any selectable least-cost path. 522 NVO3 OAM SHOULD provide the ability to perform a Continuity Check 523 on sections of any selectable path within the network. 525 5.5. Availability 527 A service may be considered unavailable if the service 528 frames/packets do not reach their intended destination (e.g., 529 connectivity is down) or the service is degraded (e.g., frame loss 530 and/or frame delay and/or delay variation threshold is exceeded). 531 Entry and exit conditions may be defined for the unavailable 532 state. Availability itself may be defined in the context of a 533 service type. Since availability measurement may be associated 534 with connectivity, frame loss, frame delay, and frame delay 535 variation measurements, no additional requirements are specified 536 currently. 538 5.6. Data Path Forwarding 539 R13) NVO3 OAM frames MUST be forwarded along the same path (i.e., 540 links (including LAG members) and nodes) as the NVO3 data frames. 542 R14) NVO3 OAM frames MAY provide a mechanism to exercise/trace all 543 data paths that result due to ECMP/LAG hops in the underlay 544 network, if these paths have been known. 546 NVO3 OAM frame MUST be possible arranged to follow the path taken 547 by a specific flow. 549 NVE MUST have the ability to identify frames that require OAM 550 processing. 552 The Controller element MAY be involved in the out-of-band OAM 553 design and deployment. Indeed, the Controller is expected to 554 maintain an up-to-date global, systemic view of all the network 555 paths and their associated status (e.g., available, idle, 556 unavailable, faulty, in maintenance, etc.) 558 5.7. Scalability 560 R15) NVO3 OAM MUST be scalable such that an NV edge device can 561 support proactive OAM for each VNI that is supported by the 562 device. 564 5.8. Extensibility 566 R16) NVO3 OAM should be extensible such that new functionality and 567 information elements related to this functionality can be 568 introduced in the future. 570 R17) NVO3 OAM MUST be defined such that devices not supporting the 571 OAM are able to forward the OAM frames in a similar fashion as the 572 regular NVO3 data frames/packets. 574 5.9. Security 576 R18) NVO3 OAM frames MUST be prevented from leaking outside their 577 NVO3 domain. 579 R19) NVO3 OAM frames from outside an NVO3 domain MUST be prevented 580 from entering the said NVO3 domain when such OAM frames belong to 581 the same level or to a lower-level OAM. (Trivially met because 582 hierarchical domains are independent technologies.) 584 R20) NVO3 OAM frames from outside an NVO3 domain MUST be 585 transported transparently inside the NVO3 domain when such OAM 586 frames belong to a higher-level NVO3 domain. (Trivially met 587 because hierarchical domains are independent technologies). 589 5.10. Transport Independence 591 Similar to transport requirement from [RFC6136], we expect NVO3 592 OAM will leverage the OAM capabilities of the transport layer 593 (e.g., IP underlay). 595 R21) NVO3 OAM MAY allow adaptation/interworking with its IP 596 underlay OAM functions. For example, this would be useful to 597 allow fault notifications from the IP layer to be sent to the NVO3 598 layer. Likewise, LAG/ECMP-originated notifications may affect the 599 NVO3 OAM decision process. 601 5.11. Application Independence 603 R22) NVO3 OAM MAY be independent of the application technologies 604 and specific application OAM capabilities. 606 5.12. Prioritization 608 R23) NVO3 OAM messages MUST be preferentially treated in NVE and 609 between NVEs, since NVO3 OAM MAY be used to trigger protection 610 switching. As noted above (R2), protection switching is the 611 automatic replacement of a failed transmission facility with a 612 working one providing equal or greater capacity, typically within 613 a few tens of milliseconds from fault detection. 615 5.13. Logging and Traceability Requirements 617 Logging is required at the Network Virtualization Authority (NVA) 618 and the Network Virtualization Edge (NVE) [and the NVO3 Gateway, 619 but the framework does not mention such a beast] in support of 620 fault management and configuration management. 622 R24) All logs MUST contain a (sufficiently accurate) timestamp to 623 allow the reporting functional instance (i.e., NVA, NVE) to 624 precisely determine the sequence of events. Clocks on different 625 functional instances SHOULD be synchronized to allow similar 626 accuracy when comparing logs from different devices. 628 R25) All logs MUST contain information that unambiguously 629 identifies the reporting functional instance 631 R26) Implementations MUST be capable of reporting the following 632 fault-related events: 634 1. Loss and resumption of connectivity 636 These reports SHOULD identify the affected VNI(s), but when the 637 loss affects a large number of VNIs simultaneously the report 638 SHOULD identify the underlying entity (e.g., route) if available. 640 2. Loss and resumption of NVE responsiveness 642 These reports will be generated by adjacent NVAs or NVEs. They 643 MUST identify the NVE concerned. 645 3. NVA or NVE change of operational state 647 These reports will be generated by the NVA or NVE concerned. They 648 MUST indicate the old and new operational states and the cause. 650 4. Loss and resumption of a VAP 652 These reports will be generated by adjacent NVAs or NVEs. They 653 MUST identify the VAP concerned. 655 R27) Implementations MUST be capable of reporting the following 656 events in support of configuration management and auditing. It 657 MUST be possible to generate the reports at both the originating 658 and executing entities. The report generated at the originating 659 entity MUST identify the executing entity and the report at the 660 executing entity MUST identify the originating entity. Both 661 reports MUST indicate the result of the transaction. 663 1. Virtual Access Point (VAP) creation or deletion 665 These reports MUST identify the VAP, the Tenant System, and the 666 port supporting the VAP. 668 2. VNI creation or deletion 670 These reports MUST identify the VNI and the VAP. 672 3. VNI renumbering 674 These reports MUST identify the VAP and the old and new VNI 675 numbers. 677 4. Reachability and forwarding information update 679 These reports MUST identify the previous and new file identifiers. 680 (Assumption: reachability and forwarding information is passed as 681 files, which are retained at the originating and executing 682 entities for a fixed period for auditing purposes.) 684 R28) As a general requirement, implementations MUST provide a 685 means whereby the operator can impose rate limits on the 686 generation of specific reports. Implementations MUST further 687 permit the operator to totally suppress reporting of specific 688 events. However, if any report types have been suppressed, non- 689 suppressible reports MUST be generated at regular intervals (e.g., 690 once an hour) indicating what report types have been suppressed. 692 5.14. Live Traffic Monitoring 694 NVO3 OAM implementations MAY provide methods to utilize live 695 traffic for troubleshooting and performance monitoring. 697 6. Items for Further Discussion 699 This section identifies a set of operational items which may be 700 elaborated further if these items fall within the scope of the 701 NVO3. 703 o VNID renumbering support 705 * Means to change the VNID assigned to a given instance MUST 706 be supported. 708 * System convergence subsequent to VNID renumbering MUST NOT 709 take longer than a few seconds, to minimize impact on the 710 tenant systems. 712 * A NVE MUST be able to map a VNID with a virtual network 713 context. 715 o VNI migration and management operations 717 * Means to delete an existing VNI MUST be supported. 719 * Means to add a new VNI MUST be supported. 721 * Means to merge several VNIs MAY be supported. 723 * Means to retrieve reporting data per VNI MUST be supported. 725 * Means to monitor the network resources per VNI MUST be 726 supported. 728 o Support of planned maintenance operations on the NVO3 729 infrastructure 730 * Graceful procedure to allow for planned maintenance 731 operation on NVE MUST be supported. This includes undoing 732 any configuration changes made for maintenance purposes 733 after completion of the maintenance. 735 o Support for communication among virtual networks 737 * For global reachability purposes, communication among 738 virtual networks MUST be supported. This can be enforced 739 using a NAT function. 741 o Activation of new network-related services to the NVO3 743 * Means to assist in activating new network services (e.g., 744 multicast) without impacting running service SHOULD be 745 supported. 747 o Inter-operator NVO3 considerations 749 * As NVO3 may be deployed over inter-operator infrastructure, 750 coordinating OAM actions in each individual domain are 751 required to ensure an end-to-end OAM. In particular, this 752 assumes existence of agreements on the measurement and 753 monitoring methods, fault detection and repair actions, 754 extending QoS classes (e.g., DSCP mapping policies), etc. 756 o An automatic coordination process among OAM functions of 757 different domains or layers which an E2E connection in Tenant 758 layer is tunneled on 760 * NVO3 OAM MAY support the automatic coordination of OAM 761 functions among different domains or layers which belong to 762 one Tenant layer E2E connection. The automatic coordination 763 means OAM function in client layer or one domain triggers 764 associated OAM functions in server layer or neighbouring 765 domain. This triggered action performs at the domain 766 boundaries, which is also the MEPs of the domain. Which OAM 767 function in client layer or one domain can trigger which OAM 768 functions in server layer or neighbouring domain depends on 769 specific condition, and can be very flexible. But the basic 770 rule is that the OAM functions performed simultaneously in 771 different domains or layers can be synthesized together to 772 get the final result. 774 * The OAM MEPs of domains MUST have the capability to know if 775 it they need to perform the above automatic coordination 776 process. This can be achieved by many ways, i.e., by 777 configuration, by checking the flag field in OAM frames. 779 * When the OAM MEPs perform the automatic coordination, a 780 specific global characteristic information MUST be carried 781 and mapped between OAM frames used in different domains or 782 layers, and be kept the same alone the whole tenant layer 783 E2E connection. The global characteristic information can 784 be the tenant network identifier (e.g., VNID), ICMP sequence 785 number, etc. It is used for identifying a set of correlated 786 OAM results obtained from these domains or layers. This set 787 of OAM results is then synthesized together to get the final 788 diagnose result. 790 * NVO3 OAM MUST support a Collection Point for collecting all 791 the OAM results and synthesizing them. It can be the SDN 792 controller, NVA, or NMS. An E2E OAM function in tenant 793 network can trigger several OAM functions in different 794 underlay networks, a Collection Point is needed to collect 795 all the OAM results from different OAM MEPs of different 796 domains or layers and synthesizes them. 798 7. IANA Considerations 799 This memo includes no request to IANA. 801 8. Security Considerations 802 Security requirements are specified in Section 5.9. For general NVO3 803 security considerations, please refer to [NVO3-Security]. 805 9. Acknowledgements 807 The authors are grateful for the contributions of David Black, Dennis 808 Qin, Erik Smith, Deepark Kumar, Dapeng Liu, and Ziye Yang to this 809 latest version. 811 10. References 813 10.1 Normative References 815 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 816 Requirement Levels", BCP 14, RFC 2119, DOI 817 10.17487/RFC2119, March 1997, . 820 10.2 Informative References 822 [IEEE802.1Q-2011] "IEEE Standard for Local and metropolitan area 823 networks - Media Access Control (MAC) Bridges and Virtual 824 Bridged Local Area Networks", 2011. 826 [NM-Standards] "ITU-T Recommendation M.3400 (02/2000) - TMN 827 Management Functions", February 2000. 829 [NVO3-DP-Reqs] Bitar, N., Lasserre, M., Balus, F., Morin, T., Jin, L. 830 and Khasnabish, B., "NVO3 Data Plane Requirements", draft- 831 ietf-nvo3-dataplane-requirements-03(work in progress), 832 April 2014. 834 [NVO3-Security] Hartman, S., Zhang, D., Wasserman, M., Qiang, Z. and 835 Zhang, M., "Security Requirements of NVO3", draft-ietf- 836 nvo3-security-requirements-05(work in progress), June 837 2015. 839 [RFC6136] Sajassi, A. and D. Mohan, "Layer 2 Virtual Private 840 Network(L2VPN) Operations, Administration, and 841 Maintenance(OAM) Requirements and Framework", March 2011. 843 [RFC7365] Lasserre, M., Balus, F., Morin, T., Bitar, N., and Y. 844 Rekhter, "Framework for DC Network Virtualization", 845 October 2014. 847 [Y.1731] "ITU-T Recommendation Y.1731 (02/08) - OAM functions and 848 mechanisms for Ethernet based networks", February 2008. 850 Authors' Addresses 852 Hao Chen 853 Huawei Technologies 854 101 Software Avenue, 855 Nanjing 210012 856 China 858 Phone: +86-25-56624440 859 EMail: philips.chenhao@huawei.com 861 Peter Ashwood-Smith 862 Huawei Technologies 863 303 Terry Fox Drive, Suite 400 864 Kanata, Ontario K2K 3J1 865 Canada 867 Phone: +1 613 595-1900 868 Email: Peter.AshwoodSmith@huawei.com 869 Liang Xia (Frank) 870 Huawei Technologies 872 Email: Frank.xialiang@huawei.com 874 Ranga Iyengar 875 Huawei Technologies USA 876 2330 Central Expy 877 Santa Clara, CA 95050 878 USA 880 Email: ranga.Iyengar@huawei.com 882 Tina Tsou 883 Huawei Technologies USA 884 2330 Central Expy 885 Santa Clara, CA 95050 886 USA 888 Email: Tina.Tsou.Zouting@huawei.com 890 Ali Sajassi 891 Cisco Technologies 892 170 West Tasman Drive 893 San Jose, CA 95134 894 USA 896 Email: sajassi@cisco.com 898 Mohamed Boucadair 899 France Telecom 900 Rennes, 35000 901 France 903 Email: mohamed.boucadair@orange.com 905 Christian Jacquenet 906 France Telecom 907 Rennes, 35000 908 France 910 Email: christian.jacquenet@orange.com 912 Masahiro Daikoku 913 KDDI corporation 914 3-10-10, Iidabashi, Chiyoda-ku 915 Tokyo 1028460 916 Japan 918 Email: ms-daikoku@kddi.com 920 Anoop Ghanwani 921 Dell 922 5450 Great America Pkwy 923 Santa Clara, CA 924 USA 926 Email: anoop@alumni.duke.edu 928 Ram Krishnan 929 Brocade 930 130 Holger Way 931 San Jose, CA 95134 932 USA 934 Email: ramk@brocade.com