idnits 2.17.1 draft-izh-ccamp-flexe-fwk-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 27, 2017) is 2494 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Obsolete informational reference (is this intentional?): RFC 7752 (Obsoleted by RFC 9552) Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Common Control and Measurment Plane I. Hussain 3 Internet-Draft R. Valiveti 4 Intended status: Informational Infinera Corp 5 Expires: December 29, 2017 Q. Wang 6 ZTE 7 L. Andersson 8 M. Chen 9 H. Zheng 10 Huawei 11 June 27, 2017 13 GMPLS Routing and Signaling Framework for Flexible Ethernet (FlexE) 14 draft-izh-ccamp-flexe-fwk-03 16 Abstract 18 As different from earlier Ethernet data planes FlexE allows for 19 decoupling of the Ethernet Physical layer (PHY) and Media Access 20 Control layer (MAC) rates. 22 Study Group 15 (SG15) of the ITU-T has endorsed the FlexE 23 Implementation Agreement from Optical Internetworking Forum (OIF) and 24 included it, by reference, in some of their Recomendations. 26 This document specifies the use cases of FlexE technology, GMPLS 27 control plane requirements, framework, and architecture. 29 Status of This Memo 31 This Internet-Draft is submitted in full conformance with the 32 provisions of BCP 78 and BCP 79. 34 Internet-Drafts are working documents of the Internet Engineering 35 Task Force (IETF). Note that other groups may also distribute 36 working documents as Internet-Drafts. The list of current Internet- 37 Drafts is at http://datatracker.ietf.org/drafts/current/. 39 Internet-Drafts are draft documents valid for a maximum of six months 40 and may be updated, replaced, or obsoleted by other documents at any 41 time. It is inappropriate to use Internet-Drafts as reference 42 material or to cite them other than as "work in progress." 44 This Internet-Draft will expire on December 29, 2017. 46 Copyright Notice 48 Copyright (c) 2017 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents 53 (http://trustee.ietf.org/license-info) in effect on the date of 54 publication of this document. Please review these documents 55 carefully, as they describe your rights and restrictions with respect 56 to this document. Code Components extracted from this document must 57 include Simplified BSD License text as described in Section 4.e of 58 the Trust Legal Provisions and are provided without warranty as 59 described in the Simplified BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 64 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 4 65 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 66 3. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 6 67 3.1. Back-to-Back FLexE . . . . . . . . . . . . . . . . . . . 6 68 3.2. Unaware Transport . . . . . . . . . . . . . . . . . . . . 8 69 3.3. Aware Transport . . . . . . . . . . . . . . . . . . . . . 8 70 3.4. FleE Termination in Transport . . . . . . . . . . . . . . 9 71 3.5. FlexE Client BW Resizing . . . . . . . . . . . . . . . . 10 72 4. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 11 73 5. Framework and Architecture . . . . . . . . . . . . . . . . . 13 74 5.1. FlexE Framework . . . . . . . . . . . . . . . . . . . . . 13 75 5.1.1. FlexE Reference Model . . . . . . . . . . . . . . . . 13 76 5.1.2. FlexE Services . . . . . . . . . . . . . . . . . . . 14 77 5.2. FlexE Architecture . . . . . . . . . . . . . . . . . . . 14 78 5.2.1. Architecture Components . . . . . . . . . . . . . . . 14 79 5.2.2. FlexE Layer Model . . . . . . . . . . . . . . . . . . 15 80 5.2.2.1. FlexE Group structure . . . . . . . . . . . . . . 15 81 5.2.2.2. FlexE Client mapping . . . . . . . . . . . . . . 15 82 6. Control Plane . . . . . . . . . . . . . . . . . . . . . . . . 15 83 6.1. GMPLS Routing . . . . . . . . . . . . . . . . . . . . . . 16 84 6.2. GMPLS Signaling . . . . . . . . . . . . . . . . . . . . . 17 85 6.3. FlexE Packet Label Switching Data Plane . . . . . . . . . 19 86 7. Operations, Administration, and Maintenance (OAM) . . . . . . 20 87 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 20 88 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 89 10. Security Considerations . . . . . . . . . . . . . . . . . . . 20 90 11. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 20 91 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 21 92 12.1. Normative References . . . . . . . . . . . . . . . . . . 21 93 12.2. Informative References . . . . . . . . . . . . . . . . . 21 95 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22 97 1. Introduction 99 Ethernet MAC rates were until recently constrained to match the rates 100 of the Ethernet PHY(s). Work within the OIF allows MAC rates to be 101 different from PHY rates. An OIF implementation agreement 102 [OIFFLEXE1] allows for complete decoupling of the MAC and PHY rates. 104 SG15 in ITU-T has endorsed the OIF FlexE data plane and parts of 105 [G.872], [G.709], [G.798] and [G.8021] depends on or are based on the 106 FlexE data plane. 108 This includes support for 110 a. MAC rates which are greater than the rate of a single PHY; 111 multiple PHYs are bonded to achieve this 113 b. MAC rates which are less than the rate of a PHY (sub-rate) 115 c. support of multiple FlexE CLients carried over a single PHY, or 116 over a collection of bonded PHYs. 118 The capabilities supported by the first version of the FlexE data 119 plane are: 121 a. Support a large rate Ethernet MAC over bonded Ethernet PHYs, e.g. 122 supporting a 200G MAC over 2 bonded 100GBASE-R PHY(s) 124 b. Support a sub-rate Ethernet MAC over a single Ethernet PHY, e.g. 125 supporting a 50G MAC over a 100GBASE-R PHY 127 c. Support a collection of flexible Ethernet clients over a single 128 Ethernet PHY, e.g. supporting two MACs with the rates 25G, and 129 one with rate 50G over a single 100GBASE-R PHY 131 d. Support a sub-rate Ethernet MAC over bonded PHYs, e.g. supporting 132 a 150G Ethernet client over 2 bonded 100GBASE-R PHY(s) 134 e. Support a collection of Ethernet MAC clients over bonded Ethernet 135 PHYs, e.g. supporting a 50G, and 150G MAC over 2 bonded Ethernet 136 PHY(s) 138 Networks which support FlexE Ethernet interfaces include a basic 139 building block, this is true also when the interfaces are bonded. 140 This building block consists of two FlexE Shim functions, located at 141 opposite ends of a link, and the logical point to point links that 142 carry the Ethernet PHY signals between the two FlexE Shim Functions. 144 These logical point-to-point PHY links may be realized in a variety 145 of ways: 147 a. direct point-to-point links with no intervening transport 148 network. 150 b. Ethernet PHY(s) may be transparently transported via an Optical 151 Transport Network (OTN), as defined by ITU-T in [G.709] and 152 [G.798]. The OTN set of client mappings has been extended to 153 support the use cases identified in the OIF FlexE implementation 154 agreement. 156 This document examines the use cases that arise when the logical 157 links between FlexE capable devices are 158 (a) point-to-point links without any intervening network 159 (b) realized via Optical transport networks. 161 This draft considers the variants in which the two peer FlexE devices 162 are both customer-edge devices, or when one is s customer-edge and 163 the other is provider edge devices. This list of use cases will help 164 identify the Control Plane (i.e. Routing and Signalling) extensions 165 that may be required. 167 1.1. Requirements Language 169 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 170 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 171 document are to be interpreted as described in RFC 2119 [RFC2119]. 173 2. Terminology 175 a. AC (Attachment Circuit) - the connectivity between a client/ 176 customer network and a provider network. 178 b. CE (Customer Edge) - the group of functions that support the 179 termination/origination of data received from or sent to the 180 network 182 c. Crunching: The process of compressing an Ethernet PHY signal by 183 eliminating the unavailable FlexE calendar slots at the ingress 184 to the transport network; these discarded unavailable FlexE 185 calendar slots are re-inserted (with fixed content) at the 186 transport network egress. 188 d. Ethernet PHY: an entity representing Physical Coding Sublayer 189 (PCS), Physical Media Attachment (PMA), and Physical Media 190 Dependent (PMD) layers. 192 e. FlexE Calendar: The total capacity of a FlexE Group is 193 represented as a collection of slots which have a granularity of 194 5G. The calendar for a FlexE Group composed of n 100G PHYs is 195 represented as an array of 20n slots (each representing 5G of 196 bandwidth). This calendar is partitioned into sub-calendars, 197 with 20 slots per 100G PHY. Each FlexE client is mapped into one 198 or more calendar slots (based on the bandwidth the FlexE client 199 flow will need). 201 Note this description of the FlexE Calendar is based on the first 202 version of FlexE, for future version changes in the granularity 203 and PHY rates are under study. 205 f. FlexE Client: An Ethernet flow based on a MAC data rate that may 206 or may not correspond to any Ethernet PHY rate. 208 g. FlexE Group: A FlexE Group is composed of from 1 to n Ethernet 209 PHYs. In the first version of FlexE each PHY is identified by a 210 number in the range {1-254}. 212 h. FlexE Interface: A logical interface that is composed of from 1 213 to n Ethernet interfaces. 215 i. FlexE Link: A logical link that connects two FexE interfaces 216 residing in two adjacent nodes. 218 j. FlexE Shim: the layer that maps or demaps the FlexE client flows 219 carried over a x Group. 221 k. FlexE Sub-Interface: A channelized logical sub-interface that is 222 allocated specific slots from a FlexE interface, the number of 223 slots depend on the rate of the FlexE client flow that will be 224 transmitted through this sub-interface. 226 l. FlexE Sub-Link: A logical link that connects two FlexE sub- 227 interfaces that residing in two adjacent nodes. 229 m. LMP: Link Management Protocol 231 n. LSP: Label Switched Path 233 o. OTN: Optical Transport Network 235 p. PW: Pseudowire 237 q. SG15: ITU-T Study Group 15 (Transport, Access and Home). 239 r. TE: Traffic Engineering 240 s. TED: Traffic Engineering Database 242 t. TN: Transport Network 244 3. Use Cases 246 This section describes 5 major use cases as a background to the 247 requirements in Section 4. The use cases are Back-to-Back FlexE, 248 FlexE Unware transport, FlexE Aware transport, FleE Termination in 249 Transport, and FlexE client BW Resizing. 251 FlexE aware routers and OTN equipment have a functionality (FlexE 252 Shim) that handles FlexE connectivity and termination. In the first 253 generation of FlexE the PHYs are 100 Gbit/s and are structured into 5 254 Gbit/s slots. In the simplest case a FlexE Group and a PHY are 255 identical, PHYs can also be combined to form larger FlexE Froups. 257 FlexE MACs can be built through combining one or more 5 Gbits slots. 258 The slots does not need to come from the same PHY, but need to be 259 part of the same FlexE Group 261 3.1. Back-to-Back FLexE 263 This section describes a FlexE scenario in which routers are 264 interconnected back-to-back through FlexE Groups without an 265 intermediate transport network, see Figure 1 below. 267 The scenarios describe in Section 3.1 assumes the first generation of 268 FlexE. 270 n x PHY 271 +-----+-----+ | +-----+-----+ 272 | R | F | | | F | R | 273 | o | l | | | l | o | 274 | u | e | | | e | u | 275 | t | x | | | x | t | 276 | e | E | v | E | e | 277 | r | +--------------------------+ | r | 278 | | S | | S | | 279 | A | h | | h | B | 280 | | i | | i | | 281 | | m | | m | | 282 +-----+-----+ +-----+-----+ 284 Figure 1: FlexE back-to-back Use Case 286 In this case we assume that we want to establish an x Gbit/s FlexE 287 LSP between router A and B, using y 5 Gbit/s slots from z PHYs. 289 o For the first version of FlexE, x can be 10, 40, or a multiple of 290 25 Gbit/s; 292 o y is equal to x/5; 294 o z can be any figure between 1 and n; 296 The GMPLS peers are the FlexE aware routers (routers A and B), and 297 GMPLS signaling and exchange of traffic engineering information takes 298 place between the peers. 300 To set up this FlexE LSP by an GMPLS control plane the OSPF-TE 301 [RFC4203] and ISIS-TE [RFC5305] needs to be extended to keep FlexE 302 traffic engineering information, e.g. the number of used and 303 available of 5 Git/s slots between a pair of routers. RSVP-TE needs 304 to be extended to set up right size LSP between the pair of routers. 305 The LSP creates a set of FlexE sub-interfaces on the routers and 306 concatenate them (by means of MPLS labels) to form an end-2-end path. 308 The action to establish the LSP, involves coordinating a number of 5 309 Gbit/s slots from the FlexE group to create the MAC layer and the 310 FlexE sub-interface. 312 3.2. Unaware Transport 314 In this use case the routers that originates and terminates the FlexE 315 PHYs and MACs are interconnected by an OTN network. The OTN network 316 is unaware what type of traffic is carried over the OTN network. 318 n x PHY 319 +-----+-----+ | +-----+-----+ 320 | R | F | | | F | R | 321 | o | l | | | l | o | 322 | u | e | v | e | u | 323 | t | x | ---------- | x | t | 324 | e | E | / \ | E | e | 325 | r | +----+ OTN +------+ | r | 326 | | S | \ / | S | | 327 | A | h | ---------- | h | B | 328 | | i | | i | | 329 | | m | | m | | 330 +-----+-----+ +-----+-----+ 332 Figure 2: FlexE Unaware Transport 334 This use case is from a GMPLS control plane point of view identical 335 to Figure 1. 337 The GMPLS peers are the FlexE aware routers, and GMPLS signaling and 338 exchange of traffic engineering information takes place between the 339 peers, e.g. router A and B. The OTN is FlexE unaware and is not 340 involved in the exchange of traffic engineering information and 341 signaling. 343 3.3. Aware Transport 345 In this use case the OTN edge nodes (PE) and the routers (CE) that 346 are connected to the OTN network are aware of that the connections 347 carry FlexE traffic. The Attachment Circuit (AC) carries the full 348 PHY bandwidth, while the OTN FlexE Aware PEs has a function called 349 "crunching" that removes unavailable slots. 351 ................................... 352 n x PHY . n x crunched PHYs . 353 . . 354 +----+ . +-----+ . 355 | CE +--------------+ PE1 +--------------------+ . 356 +----+ . +-----+ | . 357 . | . 358 . +--+--+ . 359 . OTN Network | P | . 360 . +--+--+ . 361 . | . 362 +----+ . +-----+ | . 363 | CE +--------------+ PE2 +--------------------+ . 364 +----+ . +-----+ . 365 .................................... 367 Figure 3: FlexE Aware Transport 369 Between PE1 and PE2 there is a mechanism ("crunching") that can 370 remove PHY slots that are not carrying traffic, this mechanism will 371 decrease the bandwidth necessary to carry by the OTN network. 373 The mapping between PHY(s) and MAC are called "calendar", the 374 calendar indicates which slots that carry traffic. 376 The active calendar is managed by the data plane, and will be changed 377 to match the intended calendar to complete the bandwidth resizing. 379 Apart from the requirements listed in Section 4 the GMPLS control 380 plane may be used to distrubute traffic engineering and control 381 information, e.g. distributing the intended calendar, when bandwidth 382 resizing is requested. 384 3.4. FleE Termination in Transport 386 The figure need to be added. 388 This use case does not generate any new requirements for a GMPLS 389 control plane as compared to Section 3.1, Section 3.2, and 390 Section 3.3. 392 3.5. FlexE Client BW Resizing 394 The table below show where FlexE resixing is supported. 396 *** 398 +------+---------+-----------+------------------------------------+ 399 | end- | use | TN | Resizing supported | 400 |points| case | Function | | 401 +------+---------+-----------+------------------------------------+ 402 |CE/CE | Sec 3.2 | FlexE | Yes, by CEs. | 403 | | | unaware | The OTN pipes are configured for | 404 | | | TN | the maximum number of calendar | 405 | | | | slots across each PHY in the FlexE | 406 | | | | group, no resizing required in the | 407 | | | | OTN Layer. | 408 +------+---------+-----------+------------------------------------+ 409 |CE/CE | Sec 3.3 | FlexE | Limited support. | 410 | | | aware | Supported at the endpoints only if | 411 | | | TN | the set of available/unavailable | 412 | | | | calendar slots is constant. Not | 413 | | | | supported otherwise, see notes at | 414 | | | | the end of Sec 3.2. | 415 +------+---------+-----------+------------------------------------+ 416 |CE/PE | Sec 3.4 | FlexE | No. | 417 | | |termination| Resizing not supported due to lack | 418 | | | within | of a general hitless resizing | 419 | | | TN | mechanism in OTN, | 420 +------+---------+-----------+------------------------------------+ 421 |CE/CE | Sec 3.1 | No TN | Yes, by CEs. | 422 | | | | The resizing of FlexE connections | 423 | | | | that transit multiple FlexE Groups | 424 | | | | (as in Figure 6) can be | 425 | | | | accomplished by coordinating the | 426 | | | | resize operations across each of | 427 | | | | the hops. | 428 +------+---------+-----------+------------------------------------+ 430 *** 432 Figure 4: FlexE Client Resizing 434 This use case does not generate any new requirements for a GMPLS 435 control plane as compared to Section 3.1, Section 3.2, and 436 Section 3.3. 438 4. Requirements 440 This section summarizes the requirements for FlexE Group and FlexE 441 client signaling and routing. The requirements are derived from the 442 usecases described in Section 3 of this document. Data plane 443 requirements (and/or solutions) (e.g. crunching of tributary slots, 444 adding unavailable tributary slots etc.) are not explicitly mentioned 445 in the following text. Given that the control plane sets up circuits 446 that transport client streams, there are no implications for the 447 control plane in matters of delay, jitter tolerance etc. The 448 requirements listed in this section will be used to identify the 449 Control Plane (i.e. Routing and Signaling) extensions that will be 450 required to support FlexE services in an OTN. 452 Req-1 The solution SHALL support the creation of a FlexE Group, 453 consisting of one or more (i.e., in the 1 to 254 range) 100GE 454 Ethernet PHY(s). 456 There are several alternatives that can meet this 457 requirement, e.g. routing and signaling protocols, or a 458 centralized controller/management system with network access 459 to the FlexE mux/demux at each FlexE Group termination point. 461 Req-2 The solution SHOULD be able to verify that the collection of 462 Ethernet PHY(s) included in a FlexE Group have the same 463 characteristics (e.g. number of PHYs, rate of PHYs, etc.) at 464 the peer FlexE shims. 466 Req-3 The solution SHALL support the ability to delete a FlexE 467 Group. 469 Req-4 The solution SHALL support the ability to administratively 470 lock/unlock a FlexE Group. 472 Req-5 It SHALL be possible to add/remove PHY(s) to/from an 473 operational FlexE group while the group has been 474 administratively locked. 476 [Note: Since the addition/removal of Ethernet PHY(s) is done 477 only when the group has been locked, this dataplane operation 478 of the FlexE Group ceases until it is placed in an unlocked 479 state.] 481 Req-6 The solution SHALL support the ability to advertise (and 482 discover) the information about FlexE capable nodes, and the 483 FlexE interfaces/sub-interfaces they are supporting. 485 Req-7 It SHALL be possible to assign the transport network 486 treatment for a FlexE Group to one the following choices: 488 (a) FlexE unaware transport 490 (b) FlexE aware transport 492 (c) FlexE termination in Transport. 494 Req-8 For the FlexE unaware case, each of the Ethernet PHY(s) in 495 the FlexE group SHALL be mapped independently to the 496 appropriately sized ODU container (as per [G.709], and 497 switched across the transport network [OIFFLEXE1]. The 498 control plane SHALL be capable of co-routing the ODU signals 499 that are transporting the member PHY(s) between the two FlexE 500 Shim functions. 502 Note: Insert applicable references to ITU, OIF spec for hard 503 skew tolerances] 505 Req-9 In the FlexE aware mode, the OTN SHALL crunch the PHY(s), and 506 map them to one or more ODUflex connections as per [G.709]. 508 When two or more ODUflex connections are used to transport 509 the collection of FlexE PHY(s) in a FlexE Group, the system 510 SHALL support the ability to constrain the routes for these 511 ODUflex connections (e.g. co-route them) so that the end-to- 512 end skew is kept to a minimum (and within the range supported 513 by the FlexE Shims). 515 Req-10 The system SHALL allow the addition (or removal) of one or 516 more FlexE clients against the FlexE Group which is being 517 terminated. The addition (or removal) of a FlexE client flow 518 SHALL NOT affect the services for the other FlexE client 519 signals. 521 Req-11 The system SHALL allow the FlexE client signals to flexibly 522 span the set of Ethernet PHY(s) which comprise the FlexE 523 Group. In other words, it SHALL be possible to distribute 524 any FlexE client flow over an arbitrary combination of 525 calendar slots (whose total capacity matches the client 526 bitrate) chosen from a subset of the PHY(s). 528 Req-12 When the FlexE Group is terminated on the Transport edge 529 node, this node SHOULD be capable of resizing one or more 530 FlexE client flow (using the "A/B" calendar signaling defined 531 by OIF) (see Figure 4). It is acceptable that this resizing 532 is not hitless, and the client signal incurs a glitch during 533 the resizing operation. 535 There is no requirement for the OTN network to support the 536 hitless resizing of the ODUFlex connection which is 537 transporting the FlexE client signal. 539 Req-13 The solution SHALL support FlexE client flow resizing without 540 affecting any existing FlexE clients within the same FlexE 541 Group. 543 Req-14 The solution SHALL support establishment of single- and 544 multihop end-2-end LSPs. 546 5. Framework and Architecture 548 This section discusses FlexE framework and archtecture. Framework is 549 taken to mean how FlexE interoperates with other parts of the data 550 communication system. Architecture is taken to mean how funtional 551 groups and elements within FlexE work together to deliver the 552 expected FlexE services. Framework is taken to mean how FlexE 553 interacts with it environment. 555 5.1. FlexE Framework 557 The service of offered by Flexible Ethernet is a transport service 558 very similar (or even identical) to the service offered by Ethernet. 560 There are two major additions supported by FlexE: 562 o FlexE is intended to support high bandwidth and FlexE can offer 563 granular bandwidth from 5Gbits/s and a bandwidth as high as the 564 FlexE Group allows. 566 o As FlexE groups and clients are set up as a configuration 567 activity, by a centralized controller or by a GMPLS control plane 568 the service is connection oriented. 570 5.1.1. FlexE Reference Model 572 The figure below gives a simplified FlexE reference model. 574 ................................... 575 n x PHY . n x crunched PHYs . 576 . . 577 +----+ . +-----+ +-----+ +-----+ . +----+ 578 | CE +--------------+ PE1 +----+ P +----+ PE2 +--------+ CE | 579 +----+ . +-----+ +-----+ +-----+ . +----+ 580 . . 581 +----+ m x PHY . . +----+ 582 | CE +---------------------------------------------------+ CE | 583 +----+ . . +----+ 584 . OTN Network . 585 . . 586 .................................... 588 +----+ p x PHY +----+ 589 | CE +---------------------------------------------------+ CE | 590 +----+ +----+ 592 Figure 5: FlexE Reference Model 594 5.1.2. FlexE Services 596 The services offered by Flexible Ethernet are essentially the same as 597 for traditional Ethernet, connection less Ethernet transport. 598 However, when the relationship between the PHY and MAC layer are set 599 up by a GMPLS control plane there is a strong connection oriented 600 aspect. 602 5.2. FlexE Architecture 604 5.2.1. Architecture Components 606 Editors Note (to be removed): this section needs some serious 607 polishing and also add the missing text. 609 This section discusses the different parts of FlexE signaling and 610 routing and how these parts interoperate. 612 The FlexE routing mechanism is used to provide resource available 613 information for set up of FlexE LSP, like Ethernet PHYs' information, 614 partial-rate support information. Based on the resource available 615 information advertised by routing protocol, an end-to-end FlexE 616 connection is computed, and then the signaling protocol is used to 617 set up the end-to-end connection. 619 FlexE signaling mechanism is used to set up a FlexE LSPs. 621 5.2.2. FlexE Layer Model 623 The FLexE layer model is similar Ethernet model, the Ethernet PHY 624 layer corresponds to the "FlexE Group", and the MAC layer corresponds 625 to the "FlexE Client". 627 As different from earlier Ethernet the combination of Flexe Group and 628 Client allows for a huge freedom when it comes to define the 629 bandwidth of an Ethernet connectivity. 631 5.2.2.1. FlexE Group structure 633 The FlexE Group might be supported by vitually any transport network, 634 including the Ethernet PHY. While the Ethernet PHY offers a fixed 635 bandwidth the FlexE Group has been structured into 5 Gbit/s slots. 636 This means that the Flexe Group can support FlexE clients of a 637 variety of bandwidths. 639 The first version is defined for 20 slots of 5 Git/s over a 100 Gbit/ 640 s PHY. The 100 Gbit/s PHYs can be bonded to give higher bandwidth. 642 5.2.2.2. FlexE Client mapping 644 A FlexE client is an Ethernet flow based on a MAC data rate that may 645 or may not correspond to any Ethernet PHY rate. The FlexE Shim is 646 the layer that maps or demaps the FlexE client flows carried over a 647 FlexE group. As defined in [OIFFLEXE1], MAC rates of 10, 40, and any 648 multiple of 25 Gbit/s are supported. This means that if there is a 649 100 Gbit/s FlexE Group between A and B, a FlexE client of 10, 25, 40, 650 50, 75 and 100 Gbit/s can be created. 652 However, by bonding, for example 5 PHYs of 100 Git/s to a single 653 FlexE group, FlexE clients of 500 Gbit/s can be supported. 655 6. Control Plane 657 This section discusses the procedures and extensions needed to the 658 GMPLS Control Plane to establish FlexE LSPs. 660 There are several ways to establish FlexE groups, allocate slots for 661 FlexE clients, and set up single-hop and multi-hop end to end FlexE 662 LSPs. A configuration tool, a centralized controller or the GMPLS 663 control plane can all be used. 665 To create the FleE GMPLS control plane extensions to the following 666 protocols may be needed: 668 o "RSVP-TE: Extensions to RSVP for LSP Tunnels" (RSVP-TE) [RFC3209] 670 o "Link Management Protocol" (LMP) [RFC4204] 672 o "Path Computation Element (PCE) Communication Protocol" (PCEP) 673 [RFC5440] 675 o IS-IS Extensions for Traffic Engineering (ISIS-TE) [RFC5305] 677 o "OSPF Extensions in Support of Generalized Multi-Protocol Label 678 Switching (GMPLS)" (OSPF-TE) [RFC4203] 680 o "North-Bound Distribution of Link-State and Traffic Engineering 681 (TE) Information Using BGP" (BGP-LS) [RFC7752] 683 A FlexE control plane YANG model will also be needed. 685 Section 6.2 and Section 6.1 discusses the role of the GMPLS control 686 plane when primarily setting up multi-hop LSPs. 688 When discussing the signaling and routing procedures and information 689 we assume that the the FlexE group has been established prior to 690 executing the procedures needed to establish a FlexE LSP. 691 Technically it is possible to establish FlexE group, allocate FlexE 692 client slots and FlexE LSP with a single exchange of GMPLS signaling 693 messages. 695 6.1. GMPLS Routing 697 To establish a FlexE LSP the Traffic Engineering (TE) information is 698 themost critical information, e.g. resource utitlisation on 699 interfaces and link, including the availability of slots on the FlexE 700 groups. The GPMPLS routing protocols needs to be extended to handle 701 this information. The Traffic Engineering Database (TED) will keep 702 an updated version of this information. 704 The FlexE capable nodes will be identified by IP-addresses, and the 705 routing and traffic engineering information will be flooded to all 706 nodes within the routing domain using TCP/IP. 708 When a FlexE LSP is about to be set up, e.g. R1 - R2 - R3 in 709 Figure 6 the information in the TED is used verify that resources are 710 available. When it is conformed that the FlexE LSP is establsihed 711 the TED is updated, marking the resources used for the new LSP as 712 used. Similarily when a LSP is taken down the resources are marked 713 as free. 715 6.2. GMPLS Signaling 717 In Figure 6 node R1 - R3 and R3 - R4, and R2 - R4 are connected by 718 100 Gbit/s FlexE groups. R2 - R4 are connected by 2 FlexE groups 719 each 100 Gbit/s. In this example we will go through the procedures 720 to set up two FlexE LSPs, the first (40 Git/s) R1 - R3 - R4, and the 721 second (80 Gbit/s) R2 - R3 - R4. 723 The slots of the FlexE group between two nodes is controlled by the 724 upstream node, while the assigment of a label for an LSP is 725 controlled by the downstream node. 727 In Figure 6 the four nodes may be interconnected by the FlexE back- 728 to-back or the Flex aware. 730 +----+ 731 | R1 +---------------------+ 732 +----+ | 733 | 734 +----+ +--+--+ +----+ 735 | R2 +------------------+ R3 +-------------------------+ R4 | 736 +----+ +--+--+ +----+ 737 | 738 +----+ | 739 | R5 +---------------------+ 740 +----+ 742 Figure 6: FlexE LSP Example 744 When an LSP is set up (e.g. R1 - R3 - R4) the following signaling 745 steps takes place: 747 1. Node R1 identify the resources needed for the LSP, in this case 748 we assume that a 40 Gbit/s LSP will be set up. 750 2. Node R1 identifies the next hop, in this case node R3. 752 3. Node R1 identifies the the slots to be used, we assume that slot 753 1, 3, 5, 7, 9, 11, 13 and 15 will be used. These slots will 754 carry a FlexE client flow beteween R1 and R3. 756 4. Node R1 informs node R3 about the intention to set up the 40 757 Gbit/s LSP and allocation of slots for the FlexE client. 759 5. When R3 receives the message from R1 it verifies that the 760 resources that R1 requests are available on the sub-link between 761 R1 and R3. If they are R3 will send a message to R4 requesting 762 a 20 Git/s FlexE LSP to be set up using for example slots 2, 4, 763 6, 8, 10 12, 14, and 16. 765 6. R4 verifies the availability of the resources, and if they are, 766 R4 will also identify that it is the termination point of the 767 intended LSP. 769 7. Being the termination point R4 will assigm a label for the FlexE 770 LSP. The label has the same format as MPLS Label specified in 771 RFC 3032 [RFC3032]. 773 8. Node R4 respoond to the message requesting the set up of the 774 LSP, by a message indicating that the requested slots are 775 accepted used and the MPLS Label that shall be used. 777 9. When node R3 gets the response from R4, it respond to R1 778 indicating that the requested slots slots are accepted and the 779 MPLS label to be used. 781 10. Once R1 gets the response from R3 the LSP is ready to carry 782 traffic. 784 When the second LSP of 80 Gbit/s is set up (R2 - R3 - R4) is set up, 785 the procedures are the same, the only difference is that between R3 786 and R4 the second LSP needs to be allocated to the second FlexE group 787 between R3 and R4, since there is not enough bandwidth available on 788 the FlexE Group where the first LSP were allocated. 790 It should also be noted that if we want to set up a third 80 Gbit/s 791 LSP R5 - R3 - R4, this set up will fail. The reason is that even 792 though the total free bandwidth between R3 and R4 is 80 Git/s, 793 neither of the existing FlexE Groups has enough bandwidth to support 794 an 80 Gbit/s LSP. Bonding of FlexE Groups that carry traffic is not 795 possible. 797 It would be a good strategy for an operator to define a 200 Gbit/s 798 FlexE group from the start if it is anticipated that thre might be 799 situations where some FlexE client flows will use slots from both 800 PHYs. 802 6.3. FlexE Packet Label Switching Data Plane 804 This section discusses how the FlexE LSP data plane works. In 805 general it can be said that the interface offered by the FlexE Shim 806 and the FlexE client is equivalent to the interface offeredd by the 807 Ethernet MAC. 809 Figure 7 below illustrates the FlexE packet switching data plane 810 procedures. 812 R1 R3 R4 813 ............. ...................... ........... 814 . +-------+ . . +----------------+ . . +-----+ . 815 . | LSP | . . | LSP \ / LSP | . . | LSP | . 816 . | a | . . | a \/ b | . . | b | . 817 . +-------+ . . +----------------+ . . +-----+ . 818 . | ETH | . . | ETH | | ETH | . . | ETH | . 819 . | i/f | . . | i/f | | i/f | . . | i/f | . 820 . +-------+ . . +-----+ +-----+ . . +-----+ . 821 . | FlexE | . . |FlexE| |FlexE| . . |FlexE| . 822 . | trsp | . . |trsp | |trsp | . . |trsp | . 823 . +---+---+ . . +--+--+ +--+--+ . . +--+--+ . 824 ......|...... .....^..........|..... .....^..... 825 | | | | 826 +--------------------+ +--------------------+ 828 Figure 7: FlexE LSP Data Plane 830 Note to reviewers: I'm not certain about the terminology for this 831 figure suggestions would be appreciated. 833 FlexE packet switching data plane processes packets like this: 835 o The LSP encapsulating and forwrding function in node R1 receives a 836 pack that needs to be encapsulated in an MPLS packet with the 837 label "a". The label "a" is used to figure out with FlexE 838 emulated Ethernet interfaces the label encapsulated packet need to 839 be forwarded over. 841 o The Ethernet interfaces, by means of FlexE transport, forwards the 842 packet to node R3. Node R3 swaps the label "a" to label "b" and 843 uses "b" to decide over which interface to send the packet. 845 o Node R3 forwards the packet to node R, which terminates the LSP. 847 Sending MPLS encapsulated packets over a FlexE sub-interface is 848 similar to send them over an Ethernet 802.1 interface. The critical 849 differences are: 851 o FlexE channelized sub-interfaces guarantee a deterministic 852 bandwidth for an LSP. 854 o FlexE allows for creating very large end to end bandwidth 856 7. Operations, Administration, and Maintenance (OAM) 858 To be added in a later version. 860 8. Acknowledgements 862 9. IANA Considerations 864 This memo includes no request to IANA. 866 Note to the RFC Editor: This section should be removed before 867 publishing. 869 10. Security Considerations 871 To be added in a later version. 873 11. Contributors 875 Khuzema Pithewan, Infinera Corp, kpithewan@infinera.com 877 Fatai Zhang, Huawei, zhangfatai@huawei.com 879 Jie Dong, Huawei, jie.dong@huawei.com 881 Zongpeng Du, Huawei, duzongpeng@huawei.com 883 Xian Zhang, Huawei, zhang.xian@huawei.com 885 James Huang, Huawei, james.huang@huawei.com 887 Qiwen Zhong, Huawei, zhongqiwen@huawei.com 889 Yongqing Zhu China Telecom zhuyq@gsta.com 891 Huanan Chen China Telecom chenhuanan@gsta.com 893 12. References 895 12.1. Normative References 897 [G.709] ITU, "Optical Transport Network Interfaces 898 (http://www.itu.int/rec/T-REC-G.709-201606-P/en)", July 899 2016. 901 [G.798] ITU, "Characteristics of optical transport network 902 hierarchy equipment functional blocks 903 (http://www.itu.int/rec/T-REC-G.798-201212-I/en)", 904 February 2014. 906 [G.8021] ITU, "Characteristics of Ethernet transport network 907 equipment functional blocks", November 2016. 909 [G.872] ITU, "Architecture of optical transport networks", January 910 2017. 912 [OIFFLEXE1] 913 OIF, "FLex Ethernet Implementation Agreement Version 1.0 914 (OIF-FLEXE-01.0)", March 2016. 916 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 917 Requirement Levels", BCP 14, RFC 2119, 918 DOI 10.17487/RFC2119, March 1997, 919 . 921 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 922 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 923 Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, 924 . 926 12.2. Informative References 928 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 929 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 930 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 931 . 933 [RFC4203] Kompella, K., Ed. and Y. Rekhter, Ed., "OSPF Extensions in 934 Support of Generalized Multi-Protocol Label Switching 935 (GMPLS)", RFC 4203, DOI 10.17487/RFC4203, October 2005, 936 . 938 [RFC4204] Lang, J., Ed., "Link Management Protocol (LMP)", RFC 4204, 939 DOI 10.17487/RFC4204, October 2005, 940 . 942 [RFC5305] Li, T. and H. Smit, "IS-IS Extensions for Traffic 943 Engineering", RFC 5305, DOI 10.17487/RFC5305, October 944 2008, . 946 [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation 947 Element (PCE) Communication Protocol (PCEP)", RFC 5440, 948 DOI 10.17487/RFC5440, March 2009, 949 . 951 [RFC7752] Gredler, H., Ed., Medved, J., Previdi, S., Farrel, A., and 952 S. Ray, "North-Bound Distribution of Link-State and 953 Traffic Engineering (TE) Information Using BGP", RFC 7752, 954 DOI 10.17487/RFC7752, March 2016, 955 . 957 Authors' Addresses 959 Iftekhar Hussain 960 Infinera Corp 961 169 Java Drive 962 Sunnyvale, CA 94089 963 USA 965 Email: IHussain@infinera.com 967 Radha Valiveti 968 Infinera Corp 969 169 Java Drive 970 Sunnyvale, CA 94089 971 USA 973 Email: rvaliveti@infinera.com 975 Qilei Wang 976 ZTE 977 Nanjing 978 CN 980 Email: wang.qilei@zte.com.cn 981 Loa Andersson 982 Huawei 983 Stockholm 984 Sweden 986 Email: loa@pi.nu 988 Mach Chen 989 Huawei 990 CN 992 Email: mach.chen@huawei.com 994 Haomian Zheng 995 Huawei 996 CN 998 Email: zhenghaomian@huawei.com