idnits 2.17.1 draft-ietf-forces-framework-00.txt: ** The Abstract section seems to be numbered -(313): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing document type: Expected "INTERNET-DRAFT" in the upper left hand corner of the first page ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document is more than 15 pages and seems to lack a Table of Contents. == There are 6 instances of lines with non-ascii characters in the document. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There are 379 instances of too long lines in the document, the longest one being 4 characters in excess of 72. ** The abstract seems to contain references ([FORCES-REQ]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 2002) is 7979 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC-2119' is mentioned on line 39, but not defined == Unused Reference: 'RFC2914' is defined on line 1080, but no explicit reference was found in the text ** Downref: Normative reference to an Historic RFC: RFC 1157 ** Downref: Normative reference to an Historic RFC: RFC 3084 Summary: 11 errors (**), 0 flaws (~~), 4 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Draft L. Yang 3 Expiration: December 2002 Intel Labs 4 File: draft-ietf-forces-framework-00.txt R. Dantu 5 Working Group: ForCES Netrake Corp. 6 T. Anderson 7 Intel Labs 8 June 2002 10 ForCES Architectural Framework 12 draft-ietf-forces-framework-00.txt 14 Status of this Memo 16 This document is an Internet-Draft and is in full conformance with 17 all provisions of Section 10 of RFC2026. Internet-Drafts are 18 working documents of the Internet Engineering Task Force (IETF), its 19 areas, and its working groups. Note that other groups may also 20 distribute working documents as Internet-Drafts. 22 Internet-Drafts are draft documents valid for a maximum of six 23 months and may be updated, replaced, or obsoleted by other documents 24 at any time. It is inappropriate to use Internet-Drafts as 25 reference material or to cite them other than as ``work in 26 progress.'' 28 The list of current Internet-Drafts can be accessed at 29 http://www.ietf.org/ietf/1id-abstracts.txt. 31 The list of Internet-Draft Shadow Directories can be accessed at 32 http://www.ietf.org/shadow.html. 34 Conventions used in this document 36 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 37 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in 38 this document are to be interpreted as described in [RFC-2119]. 40 1. Abstract 42 This document defines the architectural framework for ForCES network 43 elements (NE), and identifies the associated entities and the 44 interaction among them. This framework is intended to satisfy the 45 requirements specified in the ForCES requirements draft [FORCES- 46 REQ]. 48 2. Definitions 49 A set of terminology associated with the ForCES requirements is 50 defined in [FORCES-REQ] and we only include the ones that are most 51 relevant to this document here. 53 Forwarding Element (FE) - A logical entity that implements the 54 ForCES protocol. FEs use the underlying hardware to provide per- 55 packet processing and handling as directed by a CE via the ForCES 56 protocol. 58 Control Element (CE) - A logical entity that implements the ForCES 59 protocol and uses it to instruct one or more FEs how to process 60 packets. CEs handle functionality such as the execution of control 61 and signaling protocols. 63 ForCES Network Element (NE) - An entity composed of one or more CEs 64 and one or more FEs. To entities outside a NE, the NE represents a 65 single point of management. Similarly, a NE usually hides its 66 internal organization from external entities. 68 Pre-association Phase - The period of time during which a FE Manager 69 (see below) and a CE Manager (see below) are determining which FE 70 and CE should be part of the same network element. Any partitioning 71 of PFEs and PCEs occurs during this phase. 73 Post-association Phase - The period of time during which a FE does 74 know which CE is to control it and vice versa, including the time 75 during which the CE and FE are establishing communication with one 76 another. 78 ForCES Protocol - While there may be multiple protocols used within 79 the overall ForCES architecture, the term "ForCES protocol" refers 80 only to the ForCES post-association phase protocol (see below). 82 ForCES Post-Association Phase Protocol - The protocol used for post- 83 association phase communication between CEs and FEs. This protocol 84 does not apply to CE-to-CE communication, FE-to-FE communication, or 85 to communication between FE and CE managers. The ForCES protocol is 86 a master-slave protocol in which FEs are slaves and CEs are masters. 87 This protocol includes both the management of the communication 88 channel (e.g., connection establishment, heartbeats) and the control 89 messages themselves. This protocol could be a single protocol or 90 could consist of multiple protocols working together. 92 FE Manager - A logical entity that operates in the pre-association 93 phase and is responsible for determining to which CE(s) a FE should 94 communicate. This process is called CE discovery and may involve 95 the FE manager learning the capabilities of available CEs. A FE 96 manager may use anything from a static configuration to a pre- 97 association phase protocol (see below) to determine which CE(s) to 98 use. Being a logical entity, a FE manager might be physically 99 combined with any of the other logical entities mentioned in this 100 section. 102 CE Manager - A logical entity that operates in the pre-association 103 phase and is responsible for determining to which FE(s) a CE should 104 communicate. This process is called FE discovery and may involve 105 the CE manager learning the capabilities of available FEs. A CE 106 manager may use anything from a static configuration to a pre- 107 association phase protocol (see below) to determine which FE to use. 108 Being a logical entity, a CE manager might be physically combined 109 with any of the other logical entities mentioned in this section. 111 Pre-association Phase Protocol - A protocol between FE managers and 112 CE managers that is used to determine which CEs or FEs to use. A 113 pre-association phase protocol may include a CE and/or FE capability 114 discovery mechanism. Note that this capability discovery process is 115 wholly separate from (and does not replace) that used within the 116 ForCES protocol. However, the two capability discovery mechanisms 117 may utilize the same FE model. 119 FE Model - A model that describes the logical processing functions 120 of a FE. 122 ForCES Protocol Element - A FE or CE. 124 3. Introduction to Forwarding and Control Element Separation (ForCES) 126 An IP network element (NE) appears to external entities as a 127 monolithic piece of network equipment, e.g., a router, NAT, 128 firewall, or load balancer. Internally, however, an IP network 129 element (NE) (such as a router or switch) is composed of numerous 130 logically separated entities that cooperate to provide a given 131 functionality (such as routing or IP switching). Two types of 132 network element components exist: control element (CE) in control 133 plane and forwarding element (FE) in forwarding plane (or data 134 plane). Forwarding elements typically are ASIC, network-processor, 135 or general-purpose processor-based devices that handle data path 136 operations for each packet. Control elements are typically based on 137 general-purpose processors that provide control functionality like 138 routing and signaling protocols. 140 ForCES aims to define a framework and associated protocol(s) to 141 standardize the exchange of information between the control plane 142 and the forwarding plane. Having standard mechanisms between the CEs 143 and FEs allow these components to be physically separated. This 144 physical separation accrues several benefits to the ForCES 145 architecture. Separate components would allow component vendors to 146 specialize in one component without having to become experts in all 147 components. It also allows CEs and FEs from different component 148 vendors to interoperate with each other and hence it becomes 149 possible for system vendors to integrate together CEs and FEs from 150 different component vendors. This translates into a lot more design 151 choices and flexibility to the system vendors. Overall, ForCES will 152 enable rapid innovation in both the control and forwarding planes 153 while maintaining interoperability. Scalability is also easily 154 provided by this architecture in that additional forwarding or 155 control capacity can be added to existing network elements without 156 the need for forklift upgrades. 158 One example of such physical separation is at the blade level. 159 Figure 1 shows an example configuration of a router, with two 160 control blades and multiple router (forwarding) blades, all 161 interconnected into a switch fabric backplane. In such chassis 162 configuration, the control blades are the CEs while the router 163 blades are FEs, and the switch fabric backplane provides the 164 physical interconnect for all the blades. Routers today with this 165 kind of configuration use proprietary interface for messaging 166 between CEs and FEs. The goal of ForCES is to replace such 167 proprietary interface with a standard protocol. With a standard 168 protocol like ForCES implemented on all blades, it becomes possible 169 for control blades from vendor X and routing blades from vendor Y to 170 work seamlessly together in one chassis. 172 ------------------------- ------------------------- 173 | Control Blade A | | Control Blade B | 174 | (CE) | | (CE) | 175 ------------------------- ------------------------- 176 ^ | ^ | 177 | | | | 178 | V | V 179 --------------------------------------------------------- 180 | Switch Fabric Backplane | 181 --------------------------------------------------------- 182 ^ | ^ | ^ | 183 | | | | � | | 184 | V | V | V 185 ------------ ------------ ------------ 186 |Router | |Router | |Router | 187 |Blade #1 | |Blade #2 | |Blade #N | 188 | (FE) | | (FE) | | (FE) | 189 ------------ ------------ ------------ 190 ^ | ^ | ^ | 191 | | | | � | | 192 | V | V | V 194 Figure 1. A router configuration example with separate blades. 196 Another level of physical separation between the CEs and FEs can be 197 at the box level. In such configuration, all the CEs and FEs are 198 physically separated boxes, interconnected with some kind of high 199 speed LAN connection (like Gigabit Ethernet). These separated CEs 200 and FEs are only one hop away from each other within a local area 201 network. The CEs and FEs communicate to each other by running 202 ForCES, and the collection of these CEs and FEs together become one 203 routing unit to the external world. Figure 2 shows such an example. 205 In this example, the same physical interconnect (Ethernet) is shared 206 for both CE-to-FE and FE-to-FE communication. However, that does not 207 have to be the case. One reason to use different interconnect might 208 be that CE-to-FE interconnect does not have to be as fast as the FE- 209 to-FE interconnect, so the more expensive fast ports can be saved 210 for FE-to-FE. There may also be reliability and redundancy benefits 211 for the NE. 213 ------- ------- 214 | CE1 | | CE2 | 215 ------- ------- 216 ^ ^ 217 | | 218 V V 219 ============================================ Ethernet 220 ^ ^ � ^ 221 | | | 222 V V V 223 ------- ------- -------- 224 | FE#1| | FE#2| | FE#n | 225 ------- ------- -------- 226 ^ | ^ | ^ | 227 | | | | | | 228 | V | V | V 230 Figure 2. A router configuration example with separate boxes. 232 ------------------------------------------------- 233 | | | | | | | 234 |OSPF |RIP |BGP |CAC |LDP |� | 235 | | | | | | | 236 ------------------------------------------------- 237 | ForCES Interface | 238 ------------------------------------------------- 239 ^ ^ 240 ForCES | |data 241 control | |packets 242 messages| |(e.g., routing packets) 243 v v 244 ------------------------------------------------- 245 | ForCES Interface | 246 ------------------------------------------------- 247 | | | | | | | 248 |LPM Fwd|Meter |Shaper |NAT |Classi-|� | 249 | | | | |fier | | 250 ------------------------------------------------- 251 | FE resources | 252 ------------------------------------------------- 254 Figure 3. Examples of CE and FE functions 255 Some examples of control functions that can be implemented in the CE 256 include routing protocols like RIP, OSPF and BGP, control and 257 signaling protocols like CAC (Call Admission Control), LDP (Label 258 Distribution Protocol) for MPLS, etc. Examples of forwarding 259 functions in FE include LPM (longest prefix match) forwarder, 260 classifiers, traffic shaper, meter, NAT, etc. Figure 3 shows a 261 diagram with examples in both CE and FE. Any given NE may contain 262 one or many of these CE and FE functions in it. The diagram also 263 shows that ForCES protocol is used to transport both the control 264 messages for ForCES itself and the data packets that are 265 originated/destined from/to the control functions in CE (e.g., 266 routing packets). Section 5.2 provides more detail on this. 268 A set of requirements for control and forwarding separation is 269 identified in [FORCES-REQ]. This document describes a ForCES 270 architecture that satisfies the architectural requirements of that 271 document and defines a framework for ForCES network elements and 272 associated entities to facilitate protocol definition. 274 4. Architecture 276 This section defines the ForCES architectural framework and 277 associated logical components. This ForCES framework defines 278 components of ForCES NEs and also includes several ancillary 279 components. These components can be connected in different kinds of 280 topologies for flexible packet processing. 282 --------------------------------------- 283 | ForCES Network Element | 284 -------------- Fc | -------------- -------------- | 285 | CE Manager |---------+-| CE 1 |------| CE 2 | | 286 -------------- | | | Fr | | | 287 | | -------------- -------------- | 288 | Fl | | | Fp / | 289 | | Fp| |----------| / | 290 | | | |/ | 291 | | | | | 292 | | | Fp /|----| | 293 | | | /--------/ | | 294 -------------- Ff | -------------- -------------- | 295 | FE Manager |---------+-| FE 1 | Fi | FE 2 | | 296 -------------- | | |------| | | 297 | -------------- -------------- | 298 | | | | | | | | | | 299 ----+--+--+--+----------+--+--+--+----- 300 | | | | | | | | 301 | | | | | | | | 302 Fi/f Fi/f 304 Figure 4. ForCES Architectural Diagram 306 The diagram in Figure 4 shows the logical components of the ForCES 307 architecture and their relationships. There are two kinds of 308 components inside a ForCES network element: control element (CE) and 309 forwarding element (FE). The framework allows multiple instances of 310 CE and FE inside one NE. Each FE contains one or more physical media 311 interfaces for receiving and transmitting packets from/to the 312 external world. The aggregation of these FE interfaces becomes the 313 NE�s external interfaces. In addition to the external interfaces, 314 there must also exist some kind of interconnect within the NE so 315 that the CE and FE can communicate with each other, and one FE can 316 forward packets to another FE. The diagram also shows two entities 317 outside of the ForCES NE: CE Manager and FE Manager. These two 318 entities provide configuration to the corresponding CE or FE in the 319 pre-association phase (see Section 5.1). There is no defined role 320 for FE Manager and CE Manager in post-association phase, thus these 321 logical components are not considered part of the ForCES NE. 323 For convenience, the logical interactions between these components 324 are labeled by reference points Fp, Fc, Ff, Fr, Fl, and Fi. The FE 325 interface is labeled as Fi/f. More detail is provided in Section 4 326 and 5 for each of these reference points. It is important to 327 understand all these reference points from the architecture point of 328 view, however, the ForCES protocol is only defined for one reference 329 point (Fp). 331 The interface between two ForCES NEs is identical to the interface 332 between two conventional routers and these two NEs exchange the 333 protocol packets through the interfaces at Fi/f. ForCES NEs connect 334 to existing routers transparently. 336 4.1. Control Elements and Fr Reference Point 338 It is not necessary to define any protocols across the Fr reference 339 point to enable control and forwarding separation for simple 340 configurations like single CE and multiple FEs. However, this 341 architecture permits multiple CEs to be present in a network 342 element. In cases where an implementation uses multiple CEs, it is 343 expected the invariant that the CEs and FEs together appear as a 344 single NE MUST be maintained. 346 Multiple CEs may be used for redundancy, load sharing, distributed 347 control, or other purposes. Redundancy is the case where one or 348 more CEs are prepared to take over should an active CE fail. Load 349 sharing is the case where two or more CEs are concurrently active 350 and where any request that can be serviced by one of the CEs can 351 also be serviced by any of the other CEs. In both redundancy and 352 load sharing, the CEs involved are equivalently capable. The only 353 difference between these two cases is in terms of how many active 354 CEs there are. Distributed control is the case where two or more 355 CEs are concurrently active but where certain requests can only be 356 serviced by certain CEs. 358 When multiple CEs are employed in a ForCES NE, their internal 359 organization is considered an implementation issue that is beyond 360 the scope of the ForCES effort. CEs are wholly responsible for 361 coordinating amongst themselves via the Fr reference point to 362 provide consistency and synchronization. However, ForCES does not 363 define the implementation or protocols used between CEs, nor does it 364 define how to distribute functionality among CEs. Nevertheless, 365 ForCES will support mechanisms for CE redundancy or fail over, and 366 it is expected that vendors will provide redundancy or fail over 367 solutions within this framework. 369 4.2. Forwarding Elements and Fi reference point 371 FEs perform per-packet processing and handling as directed by CEs. 372 FEs have no initiative of their own. Instead, FEs are slaves and 373 only do as they are told. FEs may communicate with one or more CEs 374 concurrently across reference point Fp. FEs have no notion of CE 375 redundancy, load sharing, or distributed control. Instead, FEs 376 accept commands from any CE authorized to control them, and it is up 377 to the CEs to coordinate among themselves to achieve redundancy, 378 load sharing or distributed control. The idea is to keep FEs as 379 simple and dumb as possible so that FEs can focus its resource on 380 the packet processing functions. 382 ------- Fr ------- 383 | CE1 | ------| CE2 | 384 ------- ------- 385 | \ / | 386 | \ / | 387 | \ / | 388 | \/Fp | 389 | /\ | 390 | / \ | 391 | / \ | 392 ------- Fi ------- 393 | FE1 |<----->| FE2 | 394 ------- ------- 396 Figure 5. CE redundancy example. 398 For example, in Figure 5, FE1 and FE2 can be configured to accept 399 commands from both the primary CE (CE1) and the backup CE (CE2). At 400 the beginning, CE1 issues commands to FEs while CE2 silently remains 401 in sync with CE1 via CE to CE protocol over Fr reference point. When 402 CE1 fails, CE2 detects it and starts to take over. Before CE2 starts 403 issuing commands to the FEs, it might need to recheck the FEs' state 404 and instruct FEs whether or not it is ok to preserve their current 405 state. 407 Distributed control can be achieved in the similar fashion, without 408 much intelligence on the part of FEs. For example, FEs can be 409 configured to detect RSVP and BGP protocol packets, and forward RSVP 410 packets to one CE and BGP packets to another CE. Hence, FEs may need 411 to do packet filtering for forwarding packets to specific CEs. 413 This architecture permits multiple FEs to be present in a NE. 414 [FORCES-REQ] dictates that the ForCES protocol MUST be able to scale 415 to at least hundreds of FEs (see [FORCES-REQ] Section 5, requirement 416 #11). Each of these FEs may potentially have a different set of 417 packet processing functions, with different media interfaces. FEs 418 are responsible for basic maintenance of layer-2 connectivity with 419 other FEs and with external entities. Many layer-2 media include 420 sophisticated control protocols. The FORCES protocol (over the Fp 421 reference point) will be able to carry messages for such protools so 422 that, in keeping with the "dumb FE model" the CE can provide 423 appropriate intelligence and control over these media. 425 When multiple FEs are present, ForCES requires that packets MUST be 426 able to arrive at the NE by one FE and leave the NE via a different 427 FE (See [FORCES-REQ], Section 5, Requirement #3). Packets that 428 enter the NE via one FE and leave the NE via a different FE are 429 transferred between FEs across the Fi reference point. The Fi 430 reference point is a separate protocol from the Fp reference point 431 and is not currently defined by the ForCES architecture. 433 FEs could be connected in different kinds of topologies and packet 434 processing may spread across several FEs in the topology. Hence, 435 logical packet flow may be different from physical FE topology. 436 Figure 6 provides some topology examples. When it is necessary to 437 forward packets between FEs, CE needs to understand the FE topology. 438 The FE topology can be queried from FEs by CEs. If the most common 439 FE topology is full mesh among FEs, ForCES can assume it as the 440 default topology for FEs and hence no query is needed for such 441 default cases. 443 ----------------- 444 | CE | 445 ----------------- 446 ^ ^ ^ 447 / | \ 448 / v \ 449 / ----- \ 450 / +->|FE3|<-+ \ 451 v | ----- | v 452 ------- | | ------- 453 | FE1 |<-+ +->| FE2 | 454 ------- ------- 455 ^ | ^ | 456 | | | | 457 | v | v 459 (a) Full mesh among FE1, FE2 and FE3. 461 ----------- 462 | CE | 463 ----------- 464 ^ ^ ^ ^ 465 / | | \ 466 /------ | | ------\ 467 v v v v 468 ------- ------- ------- ------- 469 | FE1 |<->| FE2 |<->| FE3 |<->| FE4 | 470 ------- ------- ------- ------- 471 ^ | ^ | ^ | ^ | 472 | | | | | | | | 473 | v | v | v | v 475 (b) Multiple FEs in a daisy chain 477 ^ | 478 | v 479 ----------- 480 | FE1 |<-----------------------| 481 ----------- | 482 ^ ^ | 483 / \ | 484 | ^ / \ ^ | V 485 v | v v | v ---------- 486 --------- --------- | | 487 | FE2 | | FE3 |<------------>| CE | 488 --------- --------- | | 489 ^ ^ ^ ---------- 490 | \ / ^ ^ 491 | \ / | | 492 | v v | | 493 | ----------- | | 494 | | FE4 |<----------------------| | 495 | ----------- | 496 | | ^ | 497 | v | | 498 | | 499 |----------------------------------------| 501 (c) Multiple FEs connected by a ring 503 Figure 6. Some examples of FE topology. 505 4.3. CE Managers 507 CE managers are responsible for determining which FEs a CE should 508 control. It is legitimate for CE managers to be hard-coded with the 509 knowledge of with which FEs its CEs should communicate. A CE manager 510 may also be physically embedded into a CE and be implemented as a 511 simple keypad or other direct configuration mechanism on the CE. 512 Finally, CE managers may be physically and logically separate 513 entities that configure the CE with FE information via such 514 mechanisms as COPS-PR [RFC3084] or SNMP [RFC1157]. 516 4.4. FE Managers 518 FE managers are responsible for determining to which CE any 519 particular FE should initially communicate. Like CE managers, no 520 restrictions are placed on how a FE manager decides to which CEs its 521 FEs should communicate, nor are restrictions placed on how FE 522 managers are implemented. 524 5. Operational Phases 526 Both FEs and CEs require some configuration in place before they can 527 start information exchange and function as a coherent network 528 element. Two operational phases are identified in this framework -- 529 pre-association and post-association. 531 5.1.Pre-association Phase 533 Pre-association phase is the period of time during which a FE 534 Manager and a CE Manager are determining which FE and CE should be 535 part of the same network element. The protocols used during this 536 phase may include all or some of the message exchange over Fl, Ff 537 and Fc reference points. However, all these may be optional and none 538 of this is within the scope of ForCES protocol. 540 5.1.1. Fl Reference Point 542 FE Manager FE CE Manager CE 543 | | | | 544 | | | | 545 |(security exchange) | | 546 1|<------------------------------>| | 547 | | | | 548 |(a list of CEs and their attributes) | 549 2|<-------------------------------| | 550 | | | | 551 |(a list of FEs and their attributes) | 552 3|------------------------------->| | 553 | | | | 554 | | | | 555 |<----------------Fl------------>| | 557 Figure 7. An example of message exchange over Fl reference point 559 CE managers and FE managers may communicate across the Fl reference 560 point in the pre-association phase in order to determine which CEs 561 and FEs should communicate with each other. Communication across 562 the Fl reference point is optional in this architecture. No 563 requirements are placed on this reference point. 565 CE managers and FE managers may be operated by different entities. 566 The operator of the CE manager may not want to divulge, except to 567 specified FE managers, any characteristics of the CEs it manages. 569 Similarly, the operator of the FE manager may not want to divulge FE 570 characteristics, except to authorized entities. As such, CE 571 managers and FE managers may need to authenticate one another. 572 Subsequent communication between CE managers and FE managers may 573 require other security functions such as privacy, non-repudiation, 574 freshness, and integrity. 576 Once the necessary security functions have been performed, the CE 577 and FE managers communicate to determine which CEs and FEs should 578 communicate with each other. At the very minimum, the CE and FE 579 managers need to learn of the existence of available FEs and CEs 580 respectively. This discovery process may or may not entail one or 581 both managers learning the capabilities of the discovered ForCES 582 protocol elements. Figure 7 shows an example of possible message 583 exchange between CE manager and FE manager over Fl reference point. 585 5.1.2. Ff Reference Point 587 FE Manager FE CE Manager CE 588 | | | | 589 | | | | 590 |(security exchange) |(security exchange) 591 1|<------------>|authentication 1|<----------->|authentication 592 | | | | 593 |(FE ID, attributes) |(CE ID, attributes) 594 2|<-------------|request 2|<------------|request 595 | | | | 596 3|------------->|response 3|------------>|response 597 |(corresponding CE ID) |(corresponding FE ID) 598 | | | | 599 | | | | 600 |<-----Ff----->| |<-----Fc---->| 602 Figure 8. Examples of message exchange 603 over Ff and Fc reference points. 605 The Ff reference point is used to inform forwarding elements of the 606 association decisions made by FE managers in pre-association phase. 607 Only authorized entities may instruct a FE with respect to which CE 608 should control it. Therefore, privacy, integrity, freshness, and 609 authentication are necessary between FE manager and FEs when the FE 610 manager is remote to the FE. Once the appropriate security has been 611 established, FE manager instructs FEs across this reference point to 612 join a new NE or to disconnect from an existing NE. Figure 8 shows 613 example of message exchange over Ff reference point. 615 Note that when the FE manager function may be co-located with the FE 616 (such as by manual keypad entry of the CE IP address), in which case 617 this reference point is reduced to a built-in function. 619 5.1.3. Fc Reference Point 620 The Fc reference point is used to inform control elements of the 621 association decisions made by CE managers in pre-association phase. 622 When the CE manager is remote, only authorized entities may instruct 623 a CE to control certain FEs. Privacy, integrity, freshness and 624 authentication are also required across this reference point in such 625 a configuration. Once appropriate security has been established, 626 the CE manager instructs CEs as to which FEs they should control and 627 how they should control them. Figure 7 shows example of message 628 exchange over Fc reference point. 630 As with the FE manager and FEs, configurations are possible where 631 the CE manager and CE are co-located and no protocol is used for 632 this function. 634 5.2. Post-association Phase and Fp reference point 636 Post-association phase is the period of time during which a FE and 637 CE have been configured with information necessary to contact each 638 other and includes both communication establishment and steady-state 639 communication. The communication between CE and FE is performed 640 across the Fp ("p" meaning protocol) reference point. ForCES 641 protocol is exclusively used for all communication across the Fp 642 reference point. 644 5.2.1. Proximity and Interconnect between CEs and FEs 646 The ForCES Working Group has made a conscious decision that the 647 first version of ForCES will not be designed to support 648 configurations where the CE and FE are located arbitrarily in the 649 network. In particular, ForCES is intended for "very close" CE/FE 650 localities in IP networks, as defined by ForCES Applicability 651 Statement ([FORCES-APP]). Very Close localities consist of control 652 and forwarding elements that either are components in the same 653 physical box, or are separated at most by one local network hop. 655 CEs and FEs can be connected by a variety of interconnect 656 technologies, including Ethernet connections, backplanes, ATM (cell) 657 fabrics, etc. ForCES should be able to support each of these 658 interconnects (see [FORCES-REQ] Section 5, requirement #1). ForCES 659 will make use of an existing RFC2914 compliant L4 protocol with 660 adequate reliability, security and congestion control (e.g. TCP, 661 SCTP) for transport purposes. 663 5.2.2. Association Establishment 665 As an example, figure 9 shows some of the message exchange that need 666 to happen before the association between CE and FE is fully 667 established. Typically, FE would need to inform the CE of its own 668 capability and its topology in relation to other FEs. The capability 669 of FE is represented by FE model, described in another separate 670 document. The model would allow FE to describe what kind of packet 671 processing functions it contains, in what order these processing 672 happen, what kind of configurable parameters it allows, what 673 statistics it collects and what events it might throw, etc. Once 674 such information is available to CE, CE sends all the necessary 675 configuration to FE so that FE can start receiving and processing 676 packets. For example, CE might need to send a snapshot of the 677 current routing table to FE so that FE can start routing packets 678 correctly. Once FE starts accepting packets for processing, we say 679 the association of this FE with its CE is now established. From then 680 on, CE and FE enter steady-state communication as described in 681 5.2.2. 683 FE CE 684 | | 685 |(Hello, are you there?)| 686 1|<----------------------| 687 | | 688 |(Yes. let me join the NE please.) 689 2|---------------------->| 690 | | 691 |(Security exchange.) | 692 3|<--------------------->| 693 | | 694 |(What kind of FE are you? -- capability query) 695 4|<----------------------| 696 | | 697 |(Here is my FE functions/state: use model to describe) 698 5|---------------------->| 699 | | 700 |(How are you connected with others? -- topology query) 701 6|<----------------------| 702 | | 703 |(Here is the topology info) 704 7|---------------------->| 705 | | 706 |(Config for FE initialization, e.g. routing table) 707 8|<----------------------| 708 | | 709 |(I am ready to go. Shall I?) 710 9|---------------------->| 711 | | 712 |(Go ahead!) | 713 10|<----------------------| 714 | | 716 Figure 9. Example of message exchange between CE and FE 717 over Fp to establish NE association 719 5.2.3. Steady-state Communication 720 Once an association is established between the CE and the FE, the 721 ForCES protocol is used by the CE and the FE over Fp reference point 722 to exchange information to facilitate packet processing. 724 FE CE 725 | | 726 |(Add these new routes.)| 727 1|<----------------------| 728 | | 729 |(Successful.) | 730 2|---------------------->| 731 | | 732 | | 733 | | 734 |(Query some stats.) | 735 1|<----------------------| 736 | | 737 |(Reply with stats collected.) 738 2|---------------------->| 739 | | 740 | | 741 | | 742 |(My port is down, with port #.) 743 1|---------------------->| 744 | | 745 |(Route to this port instead...) 746 2|<----------------------| 747 | | 748 | | 750 Figure 10. Examples of message exchange between CE and FE 751 over Fp during steady-state communication 753 Based on the information acquired through CEs' control processing, 754 CEs will frequently need to manipulate the packet-forwarding 755 behaviors of their FE(s) by sending instructions to FEs. For 756 example, Figure 10 shows one such message exchange in which CE sends 757 new routes to FE so that FE can add them to its routing table. CE 758 can also query FE for statistics collected by FE and FE can also 759 notify CE of some important events, like interface up and down, etc. 760 Figure 8 also shows such examples. 762 5.2.4. Data Packets across Fp reference point 764 Control packets (such as RIP, OSPF messages) addressed to any of 765 NE's interfaces are typically redirected by the receiving FE to its 766 CE, and CE may originate packets and have its FE deliver them to 767 other NEs. Therefore, the communication across the Fp reference 768 point includes not only the control messages from CEs to FEs and the 769 status or statistics report from FEs to CEs, but also the data 770 packets that are redirected between them. Moreover, one FE may be 771 controlled by multiple CEs. In this configuration, the control 772 protocols supported by the FORCES NEs may spread across multiple 773 CEs. For example, one CE may support OSPF and another supports BGP. 774 FEs are configured to recognize these protocol packets and forward 775 them to the corresponding CE. 777 Figure 11 shows one example of how the OSPF packets originated by 778 router A are passed to router B. In this example, ForCES protocol is 779 used to transport the packets from CE to FE inside router A, and 780 then from FE to CE inside router B. In light of the fact that ForCES 781 protocol is responsible to transport both the control messages and 782 the data packets between CE and FE over Fp reference point, it is 783 possible to use either a single protocol or multiple protocols to 784 achieve that. 786 --------------------- ---------------------- 787 | | | | 788 | +--------+ | | +--------+ | 789 | |CE(OSPF)| | | |CE(OSPF)| | 790 | +--------+ | | +--------+ | 791 | | | | ^ | 792 | |Fp | | |Fp | 793 | v | | | | 794 | +--------+ | | +--------+ | 795 | | FE | | | | FE | | 796 | +--------+ | | +--------+ | 797 | | | | ^ | 798 | Router | | | Router | | 799 | A | | | B | | 800 ---------+----------- -----------+---------- 801 v ^ 802 | | 803 | | 804 ------------------->--------------- 806 Figure 11. Example to show data packet flow between two NEs. 808 5.2.5. Proxy FE 810 In the case where a physical FE cannot implement (e.g., due to the 811 lack of a general purpose CPU) the ForCES protocol directly, a proxy 812 FE can be used in the middle of Fp reference point. This allows the 813 CE communicate to the physical FE via the proxy by using ForCES, 814 while the proxy manipulates the physical FE using some intermediary 815 form of communication (e.g., a non-ForCES protocol or DMA). In such 816 an implementation, the combination of the proxy and the physical FE 817 becomes one logical FE entity. 819 5.3. Association Re-establishment 821 FEs and CEs may join and leave NEs dynamically (see [FORCES-REQ] 822 Section 5, requirements #12 and #13). When a FE or CE leaves the NE, 823 the association with the NE is broken. If the leaving party rejoins 824 a NE later, to re-establish the association, it may or may not need 825 to re-enter the pre-association phase. Loss of association can also 826 happen unexpectedly due to loss of connection between the CE and the 827 FE. Therefore, the framework allows the bi-directional transition 828 between these two phases, but the ForCES protocol is only applicable 829 for the post-association phase. However, the protocol should provide 830 mechanisms to support association re-establishment (see [FORCES-REQ] 831 Section 5, requirement #7). 833 Let's use the example in Figure 5 to see what happens when the 834 association is broken and later re-established again. Section 4.2 835 already explains what happens if CE1 fails and how CE2 can take 836 over. Note that if no CE redundancy is provided, FEs need to be told 837 at the association establishment time what to do in the case of CE 838 failure. FEs may be told to stop packet processing all together if 839 its CE fails. Or, FEs may be told to continue forwarding packets 840 even in the face of CE failure. No matter what, it needs to be part 841 of the configuration when the association is established. 843 Let's now look at the case when FE1 leaves the NE temporarily, 844 assuming CE1 is the working CE for the moment. FE1 may voluntarily 845 decides to leave the association. Or, it is more likely that FE1 846 stops functioning simply due to unexpected failure. In former case, 847 CE1 receives a "leave-association request" from FE1. In the latter, 848 CE1 detects the failure of FE1 by some other mean. In both cases, 849 CE1 would keep a note of such event for FE1 while continue 850 commanding FE2. When FE1 decides to rejoin again, or when it is back 851 up again from the failure, FE1 would need to re-discover its master 852 (CE). This can be achieved by several means. It may re-enter the 853 pre-association phase and get that information from its FE manager. 854 It may retrieve the previous CE information from its cache, if it 855 decides that the information is still valid. Or, that information 856 can be simply hard-coded or pre-configured into it. Once it 857 discovers its CE, it starts message exchange with CE to re-establish 858 the association just as outlined in Figure 9, with the possible 859 exception that it might be able to bypass the transport of the 860 complete initialization information. Suppose that FE1 still have its 861 routing table and other state information from the last association, 862 instead of sending all the information again from scratch, it can 863 choose to use more efficient mechanism to re-sync up the state with 864 its CE. For example, a checksum of the state might give a quick 865 indication of whether or not the state is in-sync with its CE. By 866 comparing its state with CE first, it sends information update only 867 if it is needed. 869 6. Applicability to RFC1812 871 [FORCES-REQ] Section 5, requirement #9 dictates that "All proposed 872 ForCES architecture MUST explain how that architecture may be 873 applied to support all of a router's functions as defined in 874 [RFC1812]." RFC1812 discusses many important requirements for IPv4 875 routers from the link layer to the application layer. This section 876 addresses the relevant requirements for implementing IPv4 routers 877 based on ForCES architecture and how ForCES satisfies these 878 requirements. 880 6.1. General Router Requirements 882 Routers have at least two or more logical interfaces. When CEs and 883 FEs are separated by ForCES within a single NE, some additional 884 interfaces are needed for intra-NE communications. Figure 12 shows 885 an example to illustrate that. This NE contains one CE and two FEs. 886 Each FE has four interfaces; two of them are used for receiving and 887 transmitting packets to the external world, while the other two are 888 for intra-NE connections. CE has two logical interfaces #9 and #10, 889 connected to interfaces #3 and #6 from FE1 and FE2, respectively. 890 Interface #4 and #5 are connected for FE1-FE2 communication. So this 891 router NE provides four external interfaces (#1, 2, 7 and 8). 893 --------------------------------- 894 | router NE | 895 | ----------- ----------- | 896 | | FE1 | | FE2 | | 897 | ----------- ----------- | 898 | 1| 2| 3| 4| 5| 6| 7| 8| | 899 | | | | | | | | | | 900 | | | | +----+ | | | | 901 | | | | | | | | 902 | | | 9| 10| | | | 903 | | | -------------- | | | 904 | | | | CE | | | | 905 | | | -------------- | | | 906 | | | | | | 907 -----+--+----------------+--+---- 908 | | | | 909 | | | | 911 Figure 12. A router NE example with four interfaces. 913 IPv4 routers must implement IP to support its packet forwarding 914 function, which is driven by its FIB (Forwarding Information Base). 915 This Internet layer forwarding (see [RFC1812] Section 5) 916 functionality naturally belongs to FEs in the ForCES architecture. 918 A router may implement transport layer protocols (like TCP and UDP) 919 that are required to support application layer protocols (see 920 [RFC1812] Section 6). One important class of application protocols 921 is routing protocols (see [RFC1812] Section 7). In ForCES 922 architecture, routing protocols are naturally implemented by CEs. 923 Routing protocols require routers communicate with each other. This 924 communication between CEs in different routers is supported in 925 ForCES by the FEs' ability to redirect data packets 926 addressed to routers (i.e., NEs) and CEs' ability to originate 927 packets and have them delivered by their FEs. This communication 928 occurs across Fp reference point inside each router and between 929 neighboring routers' interfaces, as illustrated in Figure 11. 931 6.2.Link Layer 933 Since FEs own all the external interfaces for the router, FEs need 934 to conform to all the link layer requirements in RFC1812, including 935 ARP support. Interface parameters, including MTU, IP address, etc., 936 must be configurable by CEs via ForCES. FEs must also inform CEs the 937 status change of the interfaces (like link up/down) via ForCES. 939 6.3.Internet Layer Protocols 941 Both FEs and CEs must implement IP protocol and all mandatory 942 options as RFC1812 specified. If an FE receives a packet containing 943 a completed source route, the packet has reached its final 944 destination. The option as received (the recorded route) MUST be 945 passed up to the transport layer (or to ICMP message processing) at 946 CEs. Both FEs and CEs might choose to silently discard packets 947 without sending ICMP errors, and such events should be logged. FEs 948 can report statistics for such events to CEs via ForCES. 950 When multiple FEs are involved to process packets, the appearance of 951 single NE must be strictly maintained. For example, Time-To-Live 952 (TTL) must be decremented only once within a single NE. For example, 953 it can be always decremented by the last FE with egress function. 954 Similarly, when source 956 When IP multicast is supported in routers, IGMP is implemented in 957 CEs. CEs are also required of ICMP support, while it is optional for 958 FEs to support ICMP. Such an option can be communicated to CEs as 959 part of the FE model. Therefore, FEs can always rely upon CEs to 960 send out ICMP error messages, but FEs also have the option to 961 generate ICMP error messages themselves. 963 6.4.Internet Layer Forwarding 965 IP forwarding is implemented by FEs. After routing protocol update 966 its routing tables at CEs, ForCES is used to send the new routing 967 table entries from CEs to FEs. Each FE has its own routing table and 968 uses this table to direct packets to the next hop interface. Note 969 that the routing table could be different from FE to FE because each 970 FE has a different set of next hop interfaces. 972 Upon receiving IP packets, FEs verify the IP header and process most 973 of the IP options. Some options can't be processed until the routing 974 decision has been made. Routing decision is made after examining the 975 destination IP address. If the destination address belongs to the 976 router itself, the packets are forwarded to CEs. Otherwise, FEs 977 determine the next hop IP address by looking up in its routing 978 table. FEs also determine the network interface it uses to send the 979 packets. Sometimes an FE may need to forward the packets to another 980 FE before packets can be forwarded out to the next hop. Right before 981 packets are forwarded out to the next hop, FEs decrement TTL by 1 982 and process any IP options that can not be processed before. FEs 983 perform any IP fragmentation if necessary, determine link layer 984 address (e.g., by ARP), and encapsulates the IP datagram (or each of 985 the fragments thereof) in an appropriate link layer frame and queues 986 it for output on the interface selected. 988 Other options mentioned in RFC1812 for IP forwarding may also be 989 implemented at FEs, for example, packet filtering. 991 FEs typically forward packets destined locally to CEs. FEs may also 992 forward exceptional packets (packets that FEs don't know how to 993 handle) to CEs. CEs are required to handle packets forwarded by FEs 994 for whatever different reasons. It might be necessary for ForCES to 995 attach some meta-data with the packets to indicate the reasons of 996 forwarding from FEs to CEs. Upon receiving packets with meta-data 997 from FEs, CEs can decide to either process the packets themselves, 998 or pass the packets to the upper layer protocols including routing 999 and management protocols. If CEs are to process the packets by 1000 themselves, CEs may choose to discard the packets, or modify and re- 1001 send the packets. CEs may also originate new packets and deliver 1002 them to FEs for further forwarding. 1004 Any state change during router operation must also be handled 1005 correctly according to RFC1812. For example, when an FE ceases 1006 forwarding, the entire NE may continue forwarding packets, but it 1007 needs to stop advertising routes that are affected by the failed FE. 1009 6.5.Transport Layer 1011 Transport layer is typically implemented at CEs to support higher 1012 layer application protocols like routing protocols. In practice, 1013 this means that most CEs implement both the Transmission Control 1014 Protocol (TCP) and the User Datagram Protocol (UDP). 1016 Both CEs and FEs need to implement ForCES protocol. If any layer-4 1017 transport is used to support ForCES, then both CEs and FEs need to 1018 implement the L4 transport and ForCES protocols. It is possible that 1019 all FEs inside an NE implements only one such protocol entity. 1021 6.6. Application Layer -- Routing Protocols 1023 Both interior routing protocols and exterior routing protocols are 1024 implemented on CEs. The routing packets originated by CEs are 1025 forwarded to FEs for delivery. The results of such protocols (like 1026 routing table update) are communicated to FEs via ForCES. 1028 6.7. Application Layer -- Network Management Protocol 1030 RFC1812 also dictates "Routers MUST be manageable by SNMP." (see 1031 [RFC1157] Section 8) In general, for post-association phase, most 1032 external management tasks (including SNMP) SHOULD be done through 1033 interaction with the CE in order to support the appearance of a 1034 single functional device. Therefore, it is recommended that SNMP 1035 management agent be implemented by CEs and the SNMP messages sent to 1036 FEs be redirected to their CEs. This also requires that ForCES In 1037 certain conditions (e.g. CE/FE disconnection), it may be useful to 1038 allow SNMP to be used to diagnose and repair problems. However, care 1039 should be taken when exercising such mechanisms and guidelines are 1040 provided in [FORCES-REQ], Section 5, requirement #4. 1042 7. Summary 1044 This document defines an architectural framework for ForCES. It 1045 identifies the relevant components for an ForCES network element, 1046 including (one or more) FEs, (one or more) CEs, FE manager 1047 (optional), and CE manager (optional). It also identifies the 1048 interaction among these components and discusses all the major 1049 reference points. It is important to point out that, among all the 1050 reference points, only the interface between CEs and FEs is within 1051 the scope of ForCES. ForCES alone may not be enough to support all 1052 different NE configurations. However, we believe ForCES is the most 1053 important element in realizing the physical separation and 1054 interoperability of CEs and FEs, and hence the first interface that 1055 ought to be standardized. Simple and useful configurations can still 1056 be implemented with only CE-FE interface standardized, e.g., single 1057 CE with full-meshed FEs and static configuration without the need 1058 for CE/FE managers. 1060 8. Security Considerations 1062 The security necessary across each reference point except Fp is 1063 discussed throughout the document. In general, the physical 1064 separation of two entities usually requires much stricter security 1065 measurement in place. For example, we pointed out in Section 5.1 1066 that authentication becomes necessary between CE manager and FE 1067 manager, between CE and CE manager, between FE and FE manager in 1068 some configuration. The physical separation of CE and FE also 1069 imposes serious security requirement for ForCES protocol. The 1070 security requirements for reference point Fp (i.e., ForCES protocol) 1071 are discussed in detail in [FORCES-REQ] Section 8. 1073 9. Intellectual Property Right 1075 The authors are not aware of any intellectual property right issues 1076 pertaining to this document. 1078 10. Normative References 1080 [RFC2914] S. Floyd, "Congestion Control Principles", RFC2914, 1081 September 2000. 1083 [RFC1157] J. Case, et. al., "A Simple Network Management Protocol 1084 (SNMP)", RFC1157, May 1990. 1086 [RFC3084] K. Chan, et. al., "COPS Usage for Policy Provisioning 1087 (COPS-PR)", RFC3084, March 2001. 1089 [RFC1812] F. Baker, "Requirements for IP Version 4 Routers", 1090 RFC1812, June 1995. 1092 11. Informative References 1094 [FORCES-REQ] T. Anderson, et. al., "Requirements for Separation of 1095 IP Control and Forwarding", work in progress, February 2002, . 1098 [FORCES-APP] A. Crouch, et. al., "ForCES Applicability Statement", 1099 work in progress, February 2002, . 1102 12. Acknowledgments 1104 Joel M. Halpern gave us many insightful comments and suggestions and 1105 pointed out several major issues. Many of our colleagues and people 1106 in the ForCES mailing list also provided valuable feedback, 1107 including Alan Crouch, Patrick Droz, David Putzolu, Hormuzd M. 1108 Khosravi, and Ryszard Dyrga. 1110 13. Authors' Addresses 1112 Lily L. Yang 1113 Intel Labs 1114 2111 NE 25th Avenue 1115 Hillsboro, OR 97124 USA 1116 Phone: +1 503 264 8813 1117 Email: lily.l.yang@intel.com 1119 Ram Dantu 1120 Netrake Corporation 1121 3000 Technology Drive 1122 Plano, Texas 75074 1123 Phone: +1 214 291 1111 1124 Email: ramd@netrake.com 1126 Todd A. Anderson 1127 Intel Labs 1128 2111 NE 25th Avenue 1129 Hillsboro, OR 97124 USA 1130 Phone: +1 503 712 1760 1131 Email: todd.a.anderson@intel.com 1133 1. Abstract........................................................1 1134 2. Definitions.....................................................1 1135 3. Introduction to Forwarding and Control Element Separation 1136 (ForCES)...........................................................3 1137 4. Architecture....................................................6 1138 4.1. Control Elements and Fr Reference Point....................7 1139 4.2. Forwarding Elements and Fi reference point.................8 1140 4.3. CE Managers...............................................10 1141 4.4. FE Managers...............................................11 1142 5. Operational Phases.............................................11 1143 5.1. Pre-association Phase.....................................11 1144 5.1.1. Fl Reference Point...................................11 1145 5.1.2. Ff Reference Point...................................12 1146 5.1.3. Fc Reference Point...................................12 1147 5.2. Post-association Phase and Fp reference point.............13 1148 5.2.1. Proximity and Interconnect between CEs and FEs.......13 1149 5.2.2. Association Establishment............................13 1150 5.2.3. Steady-state Communication...........................14 1151 5.2.4. Data Packets across Fp reference point...............15 1152 5.2.5. Proxy FE.............................................16 1153 5.3. Association Re-establishment..............................16 1154 6. Applicability to RFC1812.......................................17 1155 6.1. General Router Requirements...............................18 1156 6.2. Link Layer................................................19 1157 6.3. Internet Layer Protocols..................................19 1158 6.4. Internet Layer Forwarding.................................19 1159 6.5. Transport Layer...........................................20 1160 6.6. Application Layer -- Routing Protocols....................20 1161 6.7. Application Layer -- Network Management Protocol..........20 1162 7. Summary........................................................21 1163 8. Security Considerations........................................21 1164 9. Intellectual Property Right....................................21 1165 10. Normative References..........................................21 1166 11. Informative References........................................22 1167 12. Acknowledgments...............................................22 1168 13. Authors' Addresses............................................22