idnits 2.17.1 draft-izh-ccamp-flexe-fwk-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 7, 2018) is 1990 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Obsolete informational reference (is this intentional?): RFC 7752 (Obsoleted by RFC 9552) Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Common Control and Measurment Plane I. Hussain 3 Internet-Draft R. Valiveti 4 Intended status: Informational Infinera Corp 5 Expires: May 11, 2019 Q. Wang 6 ZTE 7 L. Andersson 8 M. Chen 9 H. Zheng 10 Huawei 11 November 7, 2018 13 GMPLS Routing and Signaling Framework for Flexible Ethernet (FlexE) 14 draft-izh-ccamp-flexe-fwk-06 16 Abstract 18 This document specifies GMPLS control plane requirements, framework, 19 and architecture for FlexE technology. 21 As different from earlier Ethernet data planes FlexE allows for 22 decoupling of the Ethernet Physical layer (PHY) and Media Access 23 Control layer (MAC) rates. 25 Study Group 15 (SG15) of the ITU-T has endorsed the FlexE 26 Implementation Agreement from Optical Internetworking Forum (OIF) and 27 included it, by reference, in some of their Recommendations. 29 Status of This Memo 31 This Internet-Draft is submitted in full conformance with the 32 provisions of BCP 78 and BCP 79. 34 Internet-Drafts are working documents of the Internet Engineering 35 Task Force (IETF). Note that other groups may also distribute 36 working documents as Internet-Drafts. The list of current Internet- 37 Drafts is at https://datatracker.ietf.org/drafts/current/. 39 Internet-Drafts are draft documents valid for a maximum of six months 40 and may be updated, replaced, or obsoleted by other documents at any 41 time. It is inappropriate to use Internet-Drafts as reference 42 material or to cite them other than as "work in progress." 44 This Internet-Draft will expire on May 11, 2019. 46 Copyright Notice 48 Copyright (c) 2018 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents 53 (https://trustee.ietf.org/license-info) in effect on the date of 54 publication of this document. Please review these documents 55 carefully, as they describe your rights and restrictions with respect 56 to this document. Code Components extracted from this document must 57 include Simplified BSD License text as described in Section 4.e of 58 the Trust Legal Provisions and are provided without warranty as 59 described in the Simplified BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 64 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 4 65 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 66 3. FlexE Reference Model . . . . . . . . . . . . . . . . . . . . 5 67 4. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 6 68 5. GMPLS Controlled FlexE . . . . . . . . . . . . . . . . . . . 7 69 5.1. Types of LSPs in a FlexE capable network . . . . . . . . 8 70 5.2. Signaling Channel . . . . . . . . . . . . . . . . . . . . 8 71 5.3. MPLS LSP in the FlexE Data Plane . . . . . . . . . . . . 8 72 5.4. Configuring the data plane in FlexE capable nodes . . . . 10 73 5.4.1. Configure/Establish a FlexE Group/Link . . . . . . . 10 74 5.4.2. Configure/Establish a FlexE Client . . . . . . . . . 11 75 5.4.3. Advertise FlexE Groups and FlexE lts . . . . . . . . 11 76 5.5. Open Issues . . . . . . . . . . . . . . . . . . . . . . . 11 77 5.5.1. Multi Layer Control Plane Typ-1 (MLCP-1) . . . . . . 12 78 5.5.2. Multi Layer Control Plane Typ-2 (MLCP-2) . . . . . . 12 79 5.5.3. Multi Layer Control Plane Typ-12 (MLCP-12) . . . . . 12 80 5.5.4. Multi Layer Control Plane Typ-3 (MLCP-3) . . . . . . 13 81 6. Framework and Architecture . . . . . . . . . . . . . . . . . 13 82 6.1. FlexE Framework . . . . . . . . . . . . . . . . . . . . . 13 83 6.2. FlexE Architecture . . . . . . . . . . . . . . . . . . . 13 84 6.2.1. Architecture Components . . . . . . . . . . . . . . . 13 85 6.2.2. FlexE Layer Model . . . . . . . . . . . . . . . . . . 14 86 6.2.2.1. FlexE Group structure . . . . . . . . . . . . . . 14 87 6.2.2.2. FlexE Client mapping . . . . . . . . . . . . . . 14 88 7. Control Plane . . . . . . . . . . . . . . . . . . . . . . . . 15 89 7.1. GMPLS Routing . . . . . . . . . . . . . . . . . . . . . . 15 90 7.2. GMPLS Signaling . . . . . . . . . . . . . . . . . . . . . 16 91 7.2.1. LSP setup with pre-configured FlexE infrastructure . 17 92 7.2.2. LSP setup with partially configured FlexE 93 infrastructure . . . . . . . . . . . . . . . . . . . 17 95 7.2.3. LSP setup with non-configured FlexE infrastructure . 18 96 7.2.4. Packet Label Switching Data Plane . . . . . . . . . . 18 97 8. Operations, Administration, and Maintenance (OAM) . . . . . . 20 98 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 20 99 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 100 11. Security Considerations . . . . . . . . . . . . . . . . . . . 20 101 12. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 20 102 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 20 103 13.1. Normative References . . . . . . . . . . . . . . . . . . 20 104 13.2. Informative References . . . . . . . . . . . . . . . . . 21 105 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22 107 1. Introduction 109 Ethernet MAC rates were until recently constrained to match the rates 110 of the Ethernet PHY(s). Work within the OIF allows MAC rates to be 111 different from PHY rates. An OIF implementation agreement 112 [OIFFLEXE1] allows for complete decoupling of the MAC and PHY rates. 114 SG15 in ITU-T has endorsed the OIF FlexE data plane and parts of 115 [G.872], [G.709], [G.798] and [G.8021] depends on or are based on the 116 FlexE data plane. 118 This includes support for: 120 a. MAC rates which are greater than the rate of a single PHY; 121 multiple PHYs are bonded to achieve this 123 b. MAC rates which are less than the rate of a PHY (sub-rate) 125 c. support for channelization within a single PHY, or over a group 126 of bonded PHYs. 128 The capabilities supported by the first version of the FlexE data 129 plane are: 131 a. Support a large rate Ethernet MAC over bonded Ethernet PHYs, e.g. 132 supporting a 200G MAC over 2 bonded 100GBASE-R PHY(s) 134 b. Support a sub-rate Ethernet MAC over a single Ethernet PHY, e.g. 135 supporting a 50G MAC over a 100GBASE-R PHY 137 c. Support a collection of flexible Ethernet clients over a single 138 Ethernet PHY, e.g. supporting two MACs with the rates 25G, and 139 one with rate 50G over a single 100GBASE-R PHY 141 d. Support a sub-rate Ethernet MAC over bonded PHYs, e.g. supporting 142 a 150G Ethernet client over 2 bonded 100GBASE-R PHY(s) 144 e. Support a collection of Ethernet MAC clients over bonded Ethernet 145 PHYs, e.g. supporting a 50G, and 150G MAC over 2 bonded Ethernet 146 PHY(s) 148 Networks which support FlexE Ethernet interfaces include a basic 149 building block, this is true also when the interfaces are bonded. 150 This building block consists of two FlexE Shim functions, located at 151 opposite ends of a link, and the logical point to point links that 152 carry the Ethernet PHY signals between the two FlexE Shim Functions. 154 These logical point-to-point links may be realized in a variety of 155 ways: 157 a. direct point-to-point links with no intervening transport 158 network. 160 b. Ethernet PHY(s) may be transparently transported via an Optical 161 Transport Network (OTN), as defined by ITU-T in [G.709] and 162 [G.798]. The OTN set of client mappings has been extended to 163 support the use cases identified in the OIF FlexE implementation 164 agreement. 166 This draft considers the variants in which the two peer FlexE devices 167 are both customer-edge devices, or when one is a customer-edge and 168 the other is provider edge devices. This list of use cases will help 169 identify the Control Plane (i.e. Routing and Signaling) extensions 170 that may be required. 172 1.1. Requirements Language 174 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 175 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 176 document are to be interpreted as described in RFC 2119 [RFC2119]. 178 2. Terminology 180 a. CE (Customer Edge) - the group of functions that support the 181 termination/origination of data received from or sent to the 182 network 184 b. Ethernet PHY: an entity representing Physical Coding Sublayer 185 (PCS), Physical Media Attachment (PMA), and Physical Media 186 Dependent (PMD) layers. 188 c. FlexE Calendar: The total capacity of a FlexE Group is 189 represented as a collection of slots which have a granularity of 190 5G. The calendar for a FlexE Group composed of n 100G PHYs is 191 represented as an array of 20n slots (each representing 5G of 192 bandwidth). This calendar is partitioned into sub-calendars, 193 with 20 slots per 100G PHY. Each FlexE client is mapped into one 194 or more calendar slots (based on the bandwidth the FlexE client 195 flow will need). 197 d. FlexE Client: An Ethernet flow based on a MAC data rate that may 198 or may not correspond to any Ethernet PHY rate. 200 e. FlexE Group: A FlexE Group is composed of from 1 to n Ethernet 201 PHYs. In the first version of FlexE each PHY is identified by a 202 number in the range {1-254}. 204 f. FlexE Shim: the layer that maps or demaps the FlexE client flows 205 carried over a FlexE Group. 207 g. LMP: Link Management Protocol 209 h. LSP: Label Switched Path 211 i. OTN: Optical Transport Network 213 j. SG15: ITU-T Study Group 15 (Transport, Access and Home). 215 k. TE: Traffic Engineering 217 l. TED: Traffic Engineering Database 219 3. FlexE Reference Model 221 The figure below gives a simplified FlexE reference model. 223 ................................... 224 n x PHY . n x crunched PHYs . 225 . . 226 +----+ . +-----+ +-----+ +-----+ . +----+ 227 | CE +--------------+ PE1 +----+ P +----+ PE2 +--------+ CE | 228 +----+ . +-----+ +-----+ +-----+ . +----+ 229 . . 230 +----+ m x PHY . . +----+ 231 | CE +---------------------------------------------------+ CE | 232 +----+ . . +----+ 233 . OTN Network . 234 . . 235 .................................... 237 +----+ p x PHY +----+ 238 | CE +---------------------------------------------------+ CE | 239 +----+ +----+ 241 Figure 1: FlexE Reference Model 243 The services offered by Flexible Ethernet are essentially the same as 244 for traditional Ethernet, connection less Ethernet transport. 245 However, when the relationship between the PHY and MAC layer are 246 setup by a GMPLS control plane there is a strong connection oriented 247 aspect. 249 4. Requirements 251 This section summarizes the control plane requirements for FlexE 252 Group and FlexE Client signaling and routing. 254 Req-1 The solution SHALL support the creation of a FlexE Group, 255 consisting of one or more (i.e., in the 1 to 254 range) 100GE 256 Ethernet PHY(s). 258 There are several alternatives that can meet this 259 requirement, e.g. routing and signaling protocols, or a 260 centralized controller/management system with network access 261 to the FlexE mux/demux at each FlexE Group termination point. 263 Req-2 The solution SHOULD be able to verify that the collection of 264 Ethernet PHY(s) included in a FlexE Group have the same 265 characteristics (e.g. number of PHYs, rate of PHYs, etc.) at 266 the peer FlexE shims. 268 Req-3 The solution SHALL support the ability to delete a FlexE 269 Group. 271 Req-4 The solution SHALL support the ability to administratively 272 lock/unlock a FlexE Group. 274 Req-5 It SHALL be possible to add/remove PHY(s) to/from an 275 operational FlexE group while the group has been 276 administratively locked. 278 Req-6 The solution SHALL support the ability to advertise and 279 discover information about FlexE capable nodes, and the FlexE 280 Groups and FlexE Clients they support. 282 Req-7 The system SHALL allow the addition (or removal) of one or 283 more FlexE clients on aFlexE Group. The addition (or 284 removal) of a FlexE client flow SHALL NOT affect the services 285 for the other FlexE client signals. 287 Req-8 The system SHALL allow the FlexE client signals to flexibly 288 span the set of Ethernet PHY(s) which comprise the FlexE 289 Group. 291 Req-9 The solution SHALL support FlexE client flow resizing without 292 affecting any existing FlexE clients within the same FlexE 293 Group. 295 Req-10 The solution SHALL support establishment of MPLS LSPs that 296 requires the support of a FlexE infrastructure. 298 5. GMPLS Controlled FlexE 300 The high level goals for using a GMPLS control plane for FlexE can be 301 summarized as: 303 o Set up a FlexE Group 305 o Set up a FlexE Client 307 o Advertise FlexE Groups and FlexE Clients 309 o Set up of a higher layer LSP that requires to be run over a FlexE 310 infrastructure. 312 5.1. Types of LSPs in a FlexE capable network 314 The FlexE infrastructure may be established in three different ways 316 o The FlexE Groups and FlexE Client may be pre-configured 318 o Only the FlexE groups may be pre-configured, while the setup of 319 the FlexE client is triggered by the request to setup a MPLS LSP 321 o The setup of both FlexE Group and FlexE Client may be triggered by 322 the request to setup an MPLS LSP. 324 5.2. Signaling Channel 326 In the type of equipment for which FlexE was first specified an out 327 of band signaling channel is not commonly available. If that is the 328 case, and the GMPLS FlexE control plane will be used, the FlexE Group 329 will have to setup by e.g. a management system and a FlexE Client on 330 that FlexE Group (also configured) will have to allocated as a 331 signaling channel. 333 Further details of the setup of the FlexE Groups, FlexE Clients and 334 MPLS LSPs over a FlexE infrastructure will be found in Section 7.2. 336 5.3. MPLS LSP in the FlexE Data Plane 338 FlexE is a true link layer technology, i.e. it is not switched, this 339 means that the FlexE Groups and FlexE Clients are terminated on the 340 next-hop node, and that the switching needs to take place on a higher 341 layer. 343 The FlexE technology can be used to establish link layer connectivity 344 with high and deterministic bandwidth. However, there is no way to, 345 in a deterministic way, allocate certain traffic to a specific FlexE 346 Client. A GMPLS control plane can do this. 348 A GMPLS controlled FlexE capable node may be thought of using the 349 traditional model of a node with a separation between control and 350 data plane. 352 +------------------+ 353 | +------------+ | 354 | | GMPLS | | 355 | | Control | | 356 | | Plane | | 357 | +------------+ | 358 | ^ | 359 | | | 360 | v | 361 | +------------+ | 362 | | FlexE | | 363 | | Data | | ^ 364 | | Plane | | 365 | +------------+ | 366 +------------------+ 368 Figure 2: GMPLS controlled FlexE Node 370 The GMPLS control plane will speak extended standard GMPLS protocols 371 with its neighbours and peers. 373 Node A Node B Node Z 374 +--CP 375 +-|-----------+ |------------------+ ~ +---------+ 376 | | | | | | | 377 | | +------+ | | +--------------+ | | +-----+ | 378 LSP | +->| v | | | | ....x..... | | | | ^ | | 379 | | | . | | | | . . | | | | . | | 380 | | +--.---+ | | +--.--------.--+ | | +--.--+ | 381 FlexE | +->| o | | | | o | | o | | | | o | | 382 Client | | | o | | | | o | | o | | | | o | | 383 | | +--o---| | | +--o--+ +--o--+ | | +--o--| | 384 FlexE | +->| U | | | | U | | U | | | | U | | 385 Group | | U | | | | U | | U | | | | U | | 386 | +--U---| | | +--U--+ +--U--+ | | +--U--+ | 387 |-------U-----+ +----U--------U----+ +----U----+ 388 U U U U 389 UUUUUUUUUUUUUUUUUUUUU UUUUUUU ~ UUUUUUUU 391 Legend ... = LSP 392 ooo = FlexE Client 393 UUU = FlexE Group 395 Figure 3: GMPLS controlled network with FlexE infrastructure 397 Figure 3 describes how an MPLS LSP is mapped over a FlexE Client and 398 FlexE Group. 400 5.4. Configuring the data plane in FlexE capable nodes 402 In Figure 3 we show an LSP, a FlexE Client and a FlexE Group, the LSP 403 is there because while the FlexE Channel and Group are not switched, 404 switching in our example takes place on the LSP level. This section 405 will discuss establishment of FlexE Clients and Groups, and mapping 406 of the LSP onto a FlexE Client. 408 The establishment of a LSP over a FlexE system is very similar to how 409 this is done in any other system. Building on information gathered 410 through the routing system and using the GMPLS signaling to establish 411 the LSP. 413 5.4.1. Configure/Establish a FlexE Group/Link 415 Consider the setup of a FlexE Group between node A and B, 416 corresponding to the row of U's from node A to B in Figure 3. The 417 FlexE group is considered to consist of n PHYs, but does not have any 418 FlexE Clients defined from start. 420 When this is done by the GMPLS control plane, two conditions need to 421 be fulfilled (1) there need to be a data channel defined between node 422 A and B; and (2) a FlexE capable IGP-TE protocol needs to be running 423 in the network. 425 Node A will send an RSVP-TE message to node be with the information 426 describing the FlexE Group to be setup. This information might be 427 thought of as the "FlexE Group Label" (or part of the FlexE label). 428 It will contain at least the following information: 430 o A FlexE Group Identifier (FGid). 432 o The number of active FlexE Channels (numFC), where 0 indicates 433 that zero clients are active. 435 o Number of PHYs that the FlexE Group is composed of, for each PHY 437 * PHY identifier 439 * PHY bandwidth 441 * slot granularity/number of slots 443 * available and unavailable slots 445 When node B receives the RSVP-TE message it checks that it can setup 446 the requested FlexE Group. If the check turns positive, node send an 447 acknowledgment to node A and the FlexE Group is setup. 449 A more detailed description of how to setup a FlexE Group, will be 450 included in the draft dealing with signaling in detail. 452 5.4.2. Configure/Establish a FlexE Client 454 Consider the situation where a FlexE Group is already established (as 455 described in Section 5.4.1) and an m G FlexE Client is needed. 456 Similar to the establishment of the FlexE Group, node A will send a 457 RSV-TE message to node B. 459 This RSVP-TE message include at least the following information: 461 o FlexE Group Identifier 463 o FlexE Client Identifier 465 o from which PHYs the slots will allocated, i.e. slots might come 466 from more than one PHY. 468 o Information per PHY 470 * PHY bandwidth 472 * slot granularity 474 * available/unavailable slots 476 * allocated slots 478 A more detailed description of how to setup a FlexE Channel, will be 479 included in the draft dealing with signaling in detail. 481 5.4.3. Advertise FlexE Groups and FlexE lts 483 Once the FlexE Group and FlexE CLielts are configured they can be 484 advertised into the routing system as normal routing adjacencies, 485 including the FlexE specific TE information. 487 5.5. Open Issues 489 Note: This section is intended to be removed and the results of the 490 discussion are supposed to brought into the relevant sections of this 491 document. 492 The intention is to trigger this discussion. 494 While working on the FlexE Control Plane, questions around the 495 relationship of entities as "control plane / multi-layer control 496 plane", RSVP-TE session and the information relating to a layer 497 network. The table below summarizxes the possibilities we see. 499 +-----------+---------------------+---------------------------------+ 500 | Control | Session | Network layer info | 501 | Plane | | | 502 +-----------+---------------------+---------------------------------+ 503 | MLCP-1 | One session | Info for all network layers | 504 | MLCP-2 | Sesion for each | Each session have info for one | 505 | | network layer | network layer | 506 | MLCP-12 | More than one | info for each network layer | 507 | | sesion | included in the session | 508 | MLCP-3 | One sesion | info for a single network layer | 509 +-----------+---------------------+---------------------------------+ 511 Table 1: Multi-layer CP types 513 Sections Section 5.5.1 to Section 5.5.4 shortly describes the 514 different types of control plane identified. 516 5.5.1. Multi Layer Control Plane Typ-1 (MLCP-1) 518 A multi layer control plane type 1 (MLCP-1) has one single control 519 plane that that controls all layer networks that two nodes interact 520 over. The control plane sets up one single RSVP-TE session and all 521 layer networks are controlled over that single session. For each 522 layer network there is a set of information that the control plane 523 manages over that session. 525 5.5.2. Multi Layer Control Plane Typ-2 (MLCP-2) 527 A multi layer control plane type 2 (MLCP-2) has one single control 528 plane that that controls all layer networks that two nodes interact 529 over. The control plane sets up one RSVP-TE session for each layer 530 network and the layer networks are controlled over a dedicated 531 session. For each layer network there is a set of information that 532 the control plane manages over the dedicated session. 534 5.5.3. Multi Layer Control Plane Typ-12 (MLCP-12) 536 A multi layer control plane type 12 (MLCP-12) is a mix between MLCP-1 537 and MLCP-2, the control plane still controls all layer networks that 538 two nodes interact over. However, for some layer networks it set up 539 a RSVP-TE session the may control more than one layer network. For 540 other layer network an RSCP-TE session is used to control a single 541 layer network. For each layer network there is a set of information 542 that the control plane manages over dedicated sessions. 544 5.5.4. Multi Layer Control Plane Typ-3 (MLCP-3) 546 A multi layer control plane type 3 (MLCP-3) may be viewed as a set of 547 confederated control plsnes, where each control plane controls one 548 layer network, via a RSVP-TE session. For each layer network there 549 is a set of information that the control plane manages over the 550 dedicated session. For the case that there are more than one layer 551 network between two nodes that needs to controlled, there is one 552 dedicated control plane for each layer network. 554 6. Framework and Architecture 556 This section discusses FlexE framework and architecture. Framework 557 is taken to mean how FlexE interoperates with other parts of the data 558 communication system. Architecture is taken to mean how functional 559 groups and elements within FlexE work together to deliver the 560 expected FlexE services. Framework is taken to mean how FlexE 561 interacts with it environment. 563 6.1. FlexE Framework 565 The service offered by Flexible Ethernet is a transport service very 566 similar (or even identical) to the service offered by Ethernet. 568 There are two major additions supported by FlexE: 570 o FlexE is intended to support high bandwidth and FlexE can offer 571 granular bandwidth from 5Gbits/s and a bandwidth as high as the 572 FlexE Group allows. 574 o As FlexE groups and clients are setup as a configuration activity, 575 by a centralized controller or by a GMPLS control plane the 576 service is connection oriented. 578 6.2. FlexE Architecture 580 6.2.1. Architecture Components 582 This section discusses the different parts of FlexE signaling and 583 routing and how these parts interoperate. 585 The FlexE routing mechanism is used to provide resource available 586 information for setup of higher layer LSPs, like Ethernet PHYs' 587 information, partial-rate support information. Based on the resource 588 available information advertised by routing protocol, an end-to-end 589 FlexE connection is computed, and then the signaling protocol is used 590 to set up the end-to-end connection. 592 FlexE signaling mechanism is used to setup LSPs. 594 MPLS forwarding over a FlexE infrastructure is different from 595 forwarding over other infrastructures. When MPLS runs over a FlexE 596 infrastructure it is possible that there are more than FlexE Client 597 that meet the next-hop requirements, often it is possible to use any 598 suitable FlexE Clientfor a hop between two nodes. If the mapping 599 between a MPLS encapsulated packet and the FlexE Client, this mapping 600 need to be explicit when the LSP is set up, and the MPLS label will 601 be used to find the correct FlexE Client. 603 6.2.2. FlexE Layer Model 605 The FLexE layer model is similar Ethernet model, the Ethernet PHY 606 layer corresponds to the "FlexE Group", and the MAC layer corresponds 607 to the "FlexE Client". 609 As different from earlier Ethernet the combination of Flexe Group and 610 Client allows for a huge freedom when it comes to define the 611 bandwidth of an Ethernet connectivity. 613 6.2.2.1. FlexE Group structure 615 The FlexE Group might be supported by virtually any transport 616 network, including the Ethernet PHY. While the Ethernet PHY offers a 617 fixed bandwidth the FlexE Group has been structured into 5 Gbit/s 618 slots. This means that the FlexE Group can support FlexE clients of 619 a variety of bandwidths. 621 The first version is defined for 20 slots of 5 Git/s over a 100 Gbit/ 622 s PHY. The 100 Gbit/s PHYs can be bonded to give higher bandwidth. 624 6.2.2.2. FlexE Client mapping 626 A FlexE client is an Ethernet flow based on a MAC data rate that may 627 or may not correspond to any Ethernet PHY rate. The FlexE Shim is 628 the layer that maps or demaps the FlexE client flows carried over a 629 FlexE group. As defined in [OIFFLEXE1], MAC rates of 10, 40, and any 630 multiple of 25 Gbit/s are supported. This means that if there is a 631 100 Gbit/s FlexE Group between A and B, a FlexE client of 10, 25, 40, 632 50, 75 and 100 Gbit/s can be created. 634 However, by bonding, for example 5 PHYs of 100 Git/s to a single 635 FlexE group, FlexE clients of 500 Gbit/s can be supported. 637 7. Control Plane 639 This section discusses the procedures and extensions needed to the 640 GMPLS Control Plane to establish FlexE LSPs. 642 There are several ways to establish FlexE groups, allocate slots for 643 FlexE clients, and setup higher layer LSPs. A configuration tool, a 644 centralized controller or the GMPLS control plane can all be used. 646 To create the FlexE GMPLS control plane Groups, FlexE Clients and 647 higher layer LSPs, extensions to the following protocols may be 648 needed: 650 o "RSVP-TE: Extensions to RSVP for LSP Tunnels" (RSVP-TE) [RFC3209] 652 o "Link Management Protocol" (LMP) [RFC4204] 654 o "Path Computation Element (PCE) Communication Protocol" (PCEP) 655 [RFC5440] 657 o IS-IS Extensions for Traffic Engineering (ISIS-TE) [RFC5305] 659 o "OSPF Extensions in Support of Generalized Multi-Protocol Label 660 Switching (GMPLS)" (OSPF-TE) [RFC4203] 662 o "North-Bound Distribution of Link-State and Traffic Engineering 663 (TE) Information Using BGP" (BGP-LS) [RFC7752] 665 A FlexE control plane YANG model will also be needed. 667 Section 7.2 and Section 7.1 discusses the role of the GMPLS control 668 plane when primarily setting up LSPs. 670 When discussing the signaling and routing procedures we assume that 671 the FlexE group has been established prior to executing the 672 procedures needed to establish an LSP. Technically it is possible to 673 establish FlexE group, allocate FlexE client slots and LSP with a 674 single exchange of GMPLS signaling messages. 676 7.1. GMPLS Routing 678 To establish an LSP the Traffic Engineering (TE) information is the 679 most critical information, e.g. resource utilization on interfaces 680 and link, including the availability of slots on the FlexE groups. 681 The GPMPLS routing protocols needs to be extended to handle this 682 information. The Traffic Engineering Database (TED) will keep an 683 updated version of this information. 685 The FlexE capable nodes will be identified by IP-addresses, and the 686 routing and traffic engineering information will be flooded to all 687 nodes within the routing domain using TCP/IP. 689 When an LSP over the FlexE infrastructure is about to be setup, e.g. 690 R1 - R4 - R5 in Figure 4 the information in the TED is used verify 691 that resources are available. When it is conformed that the LSP is 692 established the TED is updated, marking the resources used for the 693 new LSP as used. Similarly when a LSP is taken down the resources 694 are marked as free. 696 7.2. GMPLS Signaling 698 As described in Section 5 the state of the FlexE infrastructure may 699 effect the actions needed to setup an LSPin a FlexE capable network. 700 The FlexE infrastructure maybe be: 702 1. fully pre-configured 704 2. partially pre-configured, i.e. the FlexE Group may be pre- 705 configured, but not the FlexE Clients 707 3. not pre-configured, i.e. the setup of FlexE Group and FlexE 708 Client will be triggered because of the request to setup an LSP. 710 Figure 4 will be used to illustrate the different cases. 712 +----+ 713 | R1 +---------------------+ 714 +----+ | 715 | 716 +----+ +--+--+ +----+ 717 | R2 +------------------+ R4 +-------------------------+ R5 | 718 +----+ +--+--+ +----+ 719 | 720 +----+ | 721 | R3 +---------------------+ PHY R1 to R4 100 Gbit(s 722 +----+ PHY R2 to R4 100 Gbit(s 723 PHY R3 to R4 100 Gbit(s 724 PHY R4 to R5 200 Gbit(s 726 Figure 4: FlexE LSP Example 728 The text in Section 7.2 is not a specification of the GMPLS signaling 729 extensions for FlexE capable network, it is a description to 730 illustrate the expected features of such a protocol. Nor do we 731 discuss failure scenarios. 733 7.2.1. LSP setup with pre-configured FlexE infrastructure 735 In this first example, referencing Figure 4, one 100 Gbit/s FlexE 736 group is configured between R1 and R4, between R2 and R4, and between 737 R3 and R4. Between R4 and R5 there is a 200 Gbit/s FlexE Group. 739 Over each 100 Gbit/s FlexE Group there are four 5 Gbit/s, two 20 740 Gbit/s and one 40 Gbit/s FlrxE Clients configured. Over the 200 Git/ 741 s FlexE Group there are eoght 5 Gbit/s, four 20 Gbit/s and tow 40 742 Gbit/s FlrxE Clients configured. 744 One of the 5 Gbit/s FlexE Clients on each FlexE Groups are used as 745 signaling channel. 747 To establish the for example a 200 Mbit/s MPLS LSP the normal GMPLS 748 request/response procedures are followed. R1 sends the request to 749 R4, R4 allocate resources on one of the FlexE Ckients, forward the 750 request to R5. R5 responds to R4 indicating the label and the FlexE 751 Client the traffic should be sent over, R4 does the same for R1. 753 The only difference between the standard signaling and what happens 754 here is that there the assigned label will be used to find the right 755 FlexE Client. 757 7.2.2. LSP setup with partially configured FlexE infrastructure 759 In the second example, also referencing Figure 4, the FlexE Groups 760 are set up in the same way as in the first example, however only one 761 5 Gbit/s FlexE Client per FlexE Group are established by 762 configuration. This FlexE Client will be used for signaling. 764 When preparing to send the request that a 5 Gbit/s MPLS LSP shall be 765 set up R1 discovers that there are no feasible FlexE Client between 766 R1 aand R4. R1 therefore sends the request to establish such a FlexE 767 Client, when receiving the request R4 allocates resources for the 768 FlexE Client on the FlexE Group. There may be different strategies 769 for allocating the bandwidth for this FlexE client. Such strategies 770 are out of scope for this document. R1 then sends the information 771 about the FlexE Client to R1, and both ends establish the FlexE 772 Client. 774 When the FlexE Client between R1 and R4 is established, R1 proceeds 775 to send the request for an MPLS LSP to R4. R4 will discover that a 776 feasible FlexE Client is missing between R4 and R5. The same 777 procedure s for setting up the FlexE Client between R1 and R4 is 778 repeated for R4 and R5. When there is a feasible FlexE Client 779 available the signaling to set up the MPLS LSP continues as normal. 781 The label allocated for the MPLS LSP will be used to find the correct 782 FlexE Client. 784 When a FlexE Clients is set up in this way they can be announced into 785 the routing system in two different ways. First, they can be made 786 generally available, i.e. it will be free to use for anyone that want 787 to set up LSPs over the FlexE Group between R1 and R4 and between R4 788 and R5. Second, the use of the FlexE Clients may be restricted to 789 the application that initially did set up the FlexE Client. 791 7.2.3. LSP setup with non-configured FlexE infrastructure 793 This example also refers to Figure 4 as different from the earlier 794 example no FlexE Group or FlexE Client configuration is done prior to 795 the first request for an MPLS LSP over the FlexE infrastructure. 797 To make the set up of LSPs in a FlexE network where no FlexE Groups 798 or FlexE Clients have been configured two conditions need to be 799 fulfilled. First an out of band signaling channel must be available. 800 Second the FlexE Capabilities must be announced in to the IGP and/or 801 centralized controller. 803 If these two conditions are fulfilled, the set up of an MPLS LSP 804 progress pretty much as in the partially configured network. The 805 difference is that the set up of both the FlexE Group and FlexE 806 Client are triggered by the request to set up an MPLS LSP. 808 As in the partially configured case FlexE Clients can be announced 809 into the routing system in two different modes, either they are 810 generally availble. It or they are reserved for the applications 811 that first established them. 813 7.2.4. Packet Label Switching Data Plane 815 This section discusses how the FlexE LSP data plane works. In 816 general it can be said that the interface offered by the FlexE Shim 817 and the FlexE client is equivalent to the interface offered by the 818 Ethernet MAC. 820 Figure 5 below illustrates the FlexE packet switching data plane 821 procedures. 823 R1 R3 R4 824 ............. ...................... ........... 825 . +-------+ . . +----------------+ . . +-----+ . 826 . | LSP | . . | LSP \ / LSP | . . | LSP | . 827 . | a | . . | a \/ b | . . | b | . 828 . +-------+ . . +----------------+ . . +-----+ . 829 . | ETH | . . | ETH | | ETH | . . | ETH | . 830 . | i/f | . . | i/f | | i/f | . . | i/f | . 831 . +-------+ . . +-----+ +-----+ . . +-----+ . 832 . | FlexE | . . |FlexE| |FlexE| . . |FlexE| . 833 . | trsp | . . |trsp | |trsp | . . |trsp | . 834 . +---+---+ . . +--+--+ +--+--+ . . +--+--+ . 835 ......|...... .....^..........|..... .....^..... 836 | | | | 837 +--------------------+ +--------------------+ 839 Figure 5: LSP over FlexE Data Plane 841 The data plane processes packets like this: 843 o The LSP encapsulating and forawrding function in node R1 receives 844 a packet that needs to be encapsulated as an MPLS packet with the 845 label "a". The label "a" is used to figure out which FlexE 846 emulated Ethernet interfaces the label encapsulated packet need to 847 be forwarded over. 849 o The Ethernet interfaces, by means of FlexE transport, forwards the 850 packet to node R3. Node R3 swaps the label "a" to label "b" and 851 uses "b" to decide over which interface to send the packet. 853 o Node R3 forwards the packet to node R, which terminates the LSP. 855 Sending MPLS encapsulated packets over a FlexE Client is similar to 856 send them over an Ethernet 802.1 interface. The critical differences 857 are: 859 o FlexE channelized sub-interfaces guarantee a deterministic 860 bandwidth for an LSP. 862 o When a application that originally establish a FlexE Client 863 reserve it for use by that application only, it is possible to 864 create uninfringeable bandwidth end-to-end for an MPLS LSP. 866 o FlexE infrastructure allows for creating very large end to end 867 bandwidth 869 8. Operations, Administration, and Maintenance (OAM) 871 To be added in a later version. 873 9. Acknowledgements 875 10. IANA Considerations 877 This memo includes no request to IANA. 879 Note to the RFC Editor: This section should be removed before 880 publishing. 882 11. Security Considerations 884 To be added in a later version. 886 12. Contributors 888 Khuzema Pithewan, Infinera Corp, kpithewan@infinera.com 890 Fatai Zhang, Huawei, zhangfatai@huawei.com 892 Jie Dong, Huawei, jie.dong@huawei.com 894 Zongpeng Du, Huawei, duzongpeng@huawei.com 896 Xian Zhang, Huawei, zhang.xian@huawei.com 898 James Huang, Huawei, james.huang@huawei.com 900 Qiwen Zhong, Huawei, zhongqiwen@huawei.com 902 Yongqing Zhu China Telecom zhuyq@gsta.com 904 Huanan Chen China Telecom chenhuanan@gsta.com 906 13. References 908 13.1. Normative References 910 [G.709] ITU, "Optical Transport Network Interfaces 911 (http://www.itu.int/rec/T-REC-G.709-201606-P/en)", July 912 2016. 914 [G.798] ITU, "Characteristics of optical transport network 915 hierarchy equipment functional blocks 916 (http://www.itu.int/rec/T-REC-G.798-201212-I/en)", 917 February 2014. 919 [G.8021] ITU, "Characteristics of Ethernet transport network 920 equipment functional blocks", November 2016. 922 [G.872] ITU, "Architecture of optical transport networks", January 923 2017. 925 [OIFFLEXE1] 926 OIF, "FLex Ethernet Implementation Agreement Version 1.0 927 (OIF-FLEXE-01.0)", March 2016. 929 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 930 Requirement Levels", BCP 14, RFC 2119, 931 DOI 10.17487/RFC2119, March 1997, 932 . 934 13.2. Informative References 936 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 937 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 938 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 939 . 941 [RFC4203] Kompella, K., Ed. and Y. Rekhter, Ed., "OSPF Extensions in 942 Support of Generalized Multi-Protocol Label Switching 943 (GMPLS)", RFC 4203, DOI 10.17487/RFC4203, October 2005, 944 . 946 [RFC4204] Lang, J., Ed., "Link Management Protocol (LMP)", RFC 4204, 947 DOI 10.17487/RFC4204, October 2005, 948 . 950 [RFC5305] Li, T. and H. Smit, "IS-IS Extensions for Traffic 951 Engineering", RFC 5305, DOI 10.17487/RFC5305, October 952 2008, . 954 [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation 955 Element (PCE) Communication Protocol (PCEP)", RFC 5440, 956 DOI 10.17487/RFC5440, March 2009, 957 . 959 [RFC7752] Gredler, H., Ed., Medved, J., Previdi, S., Farrel, A., and 960 S. Ray, "North-Bound Distribution of Link-State and 961 Traffic Engineering (TE) Information Using BGP", RFC 7752, 962 DOI 10.17487/RFC7752, March 2016, 963 . 965 Authors' Addresses 967 Iftekhar Hussain 968 Infinera Corp 969 169 Java Drive 970 Sunnyvale, CA 94089 971 USA 973 Email: IHussain@infinera.com 975 Radha Valiveti 976 Infinera Corp 977 169 Java Drive 978 Sunnyvale, CA 94089 979 USA 981 Email: rvaliveti@infinera.com 983 Qilei Wang 984 ZTE 985 Nanjing 986 CN 988 Email: wang.qilei@zte.com.cn 990 Loa Andersson 991 Huawei 992 Stockholm 993 Sweden 995 Email: loa@pi.nu 997 Mach Chen 998 Huawei 999 CN 1001 Email: mach.chen@huawei.com 1002 Haomian Zheng 1003 Huawei 1004 CN 1006 Email: zhenghaomian@huawei.com