idnits 2.17.1 draft-ietf-forces-framework-13.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 2) being 108 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 40 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (January 2004) is 7379 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '2' is defined on line 1697, but no explicit reference was found in the text == Unused Reference: '12' is defined on line 1730, but no explicit reference was found in the text == Unused Reference: '16' is defined on line 1742, but no explicit reference was found in the text ** Downref: Normative reference to an Informational RFC: RFC 3654 (ref. '3') == Outdated reference: A later version (-13) exists of draft-ietf-idr-restart-08 -- Obsolete informational reference (is this intentional?): RFC 2246 (ref. '13') (Obsoleted by RFC 4346) -- Obsolete informational reference (is this intentional?): RFC 2401 (ref. '14') (Obsoleted by RFC 4301) -- Obsolete informational reference (is this intentional?): RFC 2409 (ref. '15') (Obsoleted by RFC 4306) Summary: 5 errors (**), 0 flaws (~~), 8 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Draft L. Yang 3 Expiration: July 2004 Intel Corp. 4 File: draft-ietf-forces-framework-13.txt R. Dantu 5 Working Group: ForCES Univ. of North Texas 6 T. Anderson 7 Intel Corp. 8 R. Gopal 9 Nokia 11 January 2004 13 Forwarding and Control Element Separation (ForCES) Framework 15 draft-ietf-forces-framework-13.txt 17 Status of this Memo 19 This document is an Internet-Draft and is in full conformance with 20 all provisions of Section 10 of RFC2026. Internet-Drafts are 21 working documents of the Internet Engineering Task Force (IETF), 22 its areas, and its working groups. Note that other groups may also 23 distribute working documents as Internet-Drafts. 25 Internet-Drafts are draft documents valid for a maximum of six 26 months and may be updated, replaced, or obsoleted by other 27 documents at any time. It is inappropriate to use Internet-Drafts 28 as reference material or to cite them other than as ``work in 29 progress.'' 31 The list of current Internet-Drafts can be accessed at 32 http://www.ietf.org/ietf/1id-abstracts.txt. 34 The list of Internet-Draft Shadow Directories can be accessed at 35 http://www.ietf.org/shadow.html. 37 Copyright Notice 39 Copyright (C) The Internet Society (2003). All Rights Reserved. 41 Abstract 43 This document defines the architectural framework for the ForCES 44 (Forwarding and Control Element Separation) network elements, and 45 identifies the associated entities and the interactions among them. 47 Table of Contents 48 1. Definitions...................................................3 49 1.1. Conventions used in this document........................3 50 1.2. Terminologies............................................3 51 2. Introduction to Forwarding and Control Element Separation 52 (ForCES).........................................................5 53 3. Architecture..................................................9 54 3.1. Control Elements and Fr Reference Point.................10 55 3.2. Forwarding Elements and Fi reference point..............11 56 3.3. CE Managers.............................................15 57 3.4. FE Managers.............................................15 58 4. Operational Phases...........................................15 59 4.1. Pre-association Phase...................................15 60 4.1.1. Fl Reference Point.................................15 61 4.1.2. Ff Reference Point.................................16 62 4.1.3. Fc Reference Point.................................17 63 4.2. Post-association Phase and Fp reference point...........17 64 4.2.1. Proximity and Interconnect between CEs and FEs.....18 65 4.2.2. Association Establishment..........................18 66 4.2.3. Steady-state Communication.........................20 67 4.2.4. Data Packets across Fp reference point.............20 68 4.2.5. Proxy FE...........................................22 69 4.3. Association Re-establishment............................22 70 4.3.1. CE graceful restart................................22 71 4.3.2. FE restart.........................................24 72 5. Applicability to RFC1812.....................................25 73 5.1. General Router Requirements.............................25 74 5.2. Link Layer..............................................26 75 5.3. Internet Layer Protocols................................27 76 5.4. Internet Layer Forwarding...............................28 77 5.5. Transport Layer.........................................29 78 5.6. Application Layer -- Routing Protocols..................29 79 5.7. Application Layer -- Network Management Protocol........29 80 6. Summary......................................................30 81 7. Acknowledgements.............................................30 82 8. Security Considerations......................................30 83 8.1. Analysis of Potential Threats Introduced by ForCES......31 84 8.1.1. "Join" or "Remove" Message Flooding on CEs.........31 85 8.1.2. Impersonation Attack...............................32 86 8.1.3. Replay Attack......................................32 87 8.1.4. Attack during Fail Over............................32 88 8.1.5. Data Integrity.....................................32 89 8.1.6. Data Confidentiality...............................33 90 8.1.7. Sharing security parameters........................33 91 8.1.8. Denial of Service Attack via External Interface....33 92 8.2. Security Recommendations for ForCES.....................34 93 8.2.1. Using TLS with ForCES..............................34 94 8.2.2. Using IPsec with ForCES............................35 95 9. Normative References.........................................37 96 10. Informative References......................................37 97 11. Authors' Addresses..........................................38 98 12. Intellectual Property Right.................................39 99 13. Full Copyright Statement....................................39 101 1. Definitions 103 1.1. Conventions used in this document 105 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 106 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in 107 this document are to be interpreted as described in RFC 2119. 109 1.2. Terminologies 111 A set of terminology associated with the ForCES requirements is 112 defined in [3] and we only include the definitions that are most 113 relevant to this document here. 115 Addressable Entity (AE) - An entity that is directly addressable 116 given some interconnect technology. For example, on IP networks, 117 it is a device to which we can communicate using an IP address; on 118 a switch fabric, it is a device to which we can communicate using a 119 switch fabric port number. 121 Physical Forwarding Element (PFE) - An AE that includes hardware 122 used to provide per-packet processing and handling. This hardware 123 may consist of (but is not limited to) network processors, ASICs 124 (Application-Specific Integrated Circuits), or general purpose 125 processors, installed on line cards, daughter boards, mezzanine 126 cards, or in stand-alone boxes. 128 PFE Partition - A logical partition of a PFE consisting of some 129 subset of each of the resources (e.g., ports, memory, forwarding 130 table entries) available on the PFE. This concept is analogous to 131 that of the resources assigned to a virtual switching element as 132 described in [8]. 134 Physical Control Element (PCE) - An AE that includes hardware used 135 to provide control functionality. This hardware typically includes 136 a general purpose processor. 138 PCE Partition - A logical partition of a PCE consisting of some 139 subset of each of the resources available on the PCE. 141 Forwarding Element (FE) - A logical entity that implements the 142 ForCES Protocol. FEs use the underlying hardware to provide per- 143 packet processing and handling as directed by a CE via the ForCES 144 Protocol. FEs may happen to be a single blade (or PFE), a 145 partition of a PFE or multiple PFEs. 147 Control Element (CE) - A logical entity that implements the ForCES 148 Protocol and uses it to instruct one or more FEs how to process 149 packets. CEs handle functionality such as the execution of control 150 and signaling protocols. CEs may consist of PCE partitions or 151 whole PCEs. 153 ForCES Network Element (NE) - An entity composed of one or more CEs 154 and one or more FEs. To entities outside an NE, the NE represents 155 a single point of management. Similarly, an NE usually hides its 156 internal organization from external entities. 158 Pre-association Phase - The period of time during which an FE 159 Manager (see below) and a CE Manager (see below) are determining 160 whether an FE and a CE should be part of the same network element. 161 It is possible for some elements of the NE to be in pre-association 162 phase while other elements are in the post-association phase. 164 Post-association Phase - The period of time during which an FE does 165 know which CE is to control it and vice versa, including the time 166 during which the CE and FE are establishing communication with one 167 another. 169 ForCES Protocol - While there may be multiple protocols used within 170 the overall ForCES architecture, the term "ForCES Protocol" refers 171 only to the ForCES post-association phase protocol (see below). 173 ForCES Post-Association Phase Protocol - The protocol used for 174 post-association phase communication between CEs and FEs. This 175 protocol does not apply to CE-to-CE communication, FE-to-FE 176 communication, nor to communication between FE and CE managers. 177 The ForCES Protocol is a master-slave protocol in which FEs are 178 slaves and CEs are masters. This protocol includes both the 179 management of the communication channel (e.g., connection 180 establishment, heartbeats) and the control messages themselves. 181 This protocol could be a single protocol or could consist of 182 multiple protocols working together, and may be unicast based or 183 multicast based. A separate protocol document will specify this 184 information. 186 FE Manager - A logical entity that operates in the pre-association 187 phase and is responsible for determining to which CE(s) an FE 188 should communicate. This process is called CE discovery and may 189 involve the FE manager learning the capabilities of available CEs. 190 An FE manager may use anything from a static configuration to a 191 pre-association phase protocol (see below) to determine which CE(s) 192 to use; however, this is currently out of scope. Being a logical 193 entity, an FE manager might be physically combined with any of the 194 other logical entities mentioned in this section. 196 CE Manager - A logical entity that operates in the pre-association 197 phase and is responsible for determining to which FE(s) a CE should 198 communicate. This process is called FE discovery and may involve 199 the CE manager learning the capabilities of available FEs. A CE 200 manager may use anything from a static configuration to a pre- 201 association phase protocol (see below) to determine which FE to 202 use, however this is currently out of scope. Being a logical 203 entity, a CE manager might be physically combined with any of the 204 other logical entities mentioned in this section. 206 Pre-association Phase Protocol - A protocol between FE managers and 207 CE managers that is used to determine which CEs or FEs to use. A 208 pre-association phase protocol may include a CE and/or FE 209 capability discovery mechanism. Note that this capability 210 discovery process is wholly separate from (and does not replace) 211 that used within the ForCES Protocol. However, the two capability 212 discovery mechanisms may utilize the same FE model. 214 FE Model - A model that describes the logical processing functions 215 of an FE. 217 ForCES Protocol Element - An FE or CE. 219 Intra-FE topology - Representation of how a single FE is realized 220 by combining possibly multiple logical functional blocks along 221 multiple data path. This is defined by the FE model. 223 FE Topology - Representation of how the multiple FEs in a single NE 224 are interconnected. Sometimes it is called inter-FE topology, to 225 be distinguished from intra-FE topology used by the FE model. 227 Inter-FE topology - see FE Topology. 229 2. Introduction to Forwarding and Control Element Separation (ForCES) 231 An IP network element (NE) appears to external entities as a 232 monolithic piece of network equipment, e.g., a router, NAT, 233 firewall, or load balancer. Internally, however, an IP network 234 element (NE) (such as a router) is composed of numerous logically 235 separated entities that cooperate to provide a given functionality 236 (such as routing). Two types of network element components exist: 237 control element (CE) in control plane and forwarding element (FE) 238 in forwarding plane (or data plane). Forwarding elements typically 239 are ASIC, network-processor, or general-purpose processor-based 240 devices that handle data path operations for each packet. Control 241 elements are typically based on general-purpose processors that 242 provide control functionality like routing and signaling protocols. 244 ForCES aims to define a framework and associated protocol(s) to 245 standardize information exchange between the control and forwarding 246 plane. Having standard mechanisms allows CEs and FEs to become 247 physically separated standard components. This physical separation 248 accrues several benefits to the ForCES architecture. Separate 249 components would allow component vendors to specialize in one 250 component without having to become experts in all components. 251 Standard protocol also allows the CEs and FEs from different 252 component vendors to interoperate with each other and hence it 253 becomes possible for system vendors to integrate together the CEs 254 and FEs from different component suppliers. This interoperability 255 translates into more design choices and flexibility for the system 256 vendors. Overall, ForCES will enable rapid innovation in both the 257 control and forwarding planes while maintaining interoperability. 258 Scalability is also easily provided by this architecture in that 259 additional forwarding or control capacity can be added to existing 260 network elements without the need for forklift upgrades. 262 ------------------------- ------------------------- 263 | Control Blade A | | Control Blade B | 264 | (CE) | | (CE) | 265 ------------------------- ------------------------- 266 ^ | ^ | 267 | | | | 268 | V | V 269 --------------------------------------------------------- 270 | Switch Fabric Backplane | 271 --------------------------------------------------------- 272 ^ | ^ | ^ | 273 | | | | . . . | | 274 | V | V | V 275 ------------ ------------ ------------ 276 |Router | |Router | |Router | 277 |Blade #1 | |Blade #2 | |Blade #N | 278 | (FE) | | (FE) | | (FE) | 279 ------------ ------------ ------------ 280 ^ | ^ | ^ | 281 | | | | . . . | | 282 | V | V | V 284 Figure 1. A router configuration example with separate blades. 286 One example of such physical separation is at the blade level. 287 Figure 1 shows such an example configuration of a router, with two 288 control blades and multiple forwarding blades, all interconnected 289 into a switch fabric backplane. In such a chassis configuration, 290 the control blades are the CEs while the router blades are FEs, and 291 the switch fabric backplane provides the physical interconnect for 292 all the blades. Control blade A may be the primary CE while 293 control blade B may be the backup CE providing redundancy. It is 294 also possible to have a redundant switch fabric for high 295 availability support. Routers today with this kind of 296 configuration use proprietary interfaces for messaging between CEs 297 and FEs. The goal of ForCES is to replace such proprietary 298 interfaces with a standard protocol. With a standard protocol like 299 ForCES implemented on all blades, it becomes possible for control 300 blades from vendor X and forwarding blades from vendor Y to work 301 seamlessly together in one chassis. 303 ------- ------- 304 | CE1 | | CE2 | 305 ------- ------- 306 ^ ^ 307 | | 308 V V 309 ============================================ Ethernet 310 ^ ^ . . . ^ 311 | | | 312 V V V 313 ------- ------- -------- 314 | FE#1| | FE#2| | FE#n | 315 ------- ------- -------- 316 ^ | ^ | ^ | 317 | | | | | | 318 | V | V | V 320 Figure 2. A router configuration example with separate boxes. 322 Another level of physical separation between the CEs and FEs can be 323 at the box level. In such configuration, all the CEs and FEs are 324 physically separated boxes, interconnected with some kind of high 325 speed LAN connection (like Gigabit Ethernet). These separated CEs 326 and FEs are only one hop away from each other within a local area 327 network. The CEs and FEs communicate to each other by running 328 ForCES, and the collection of these CEs and FEs together become one 329 routing unit to the external world. Figure 2 shows such an example. 331 In both examples shown here, the same physical interconnect is used 332 for both CE-to-FE and FE-to-FE communication. However, that does 333 not have to be the case. One reason to use different interconnects 334 is that CE-to-FE interconnect does not have to be as fast as the 335 FE-to-FE interconnect, so the more expensive fast connections can 336 be saved for FE-to-FE. The separate interconnects may also provide 337 reliability and redundancy benefits for the NE. 339 Some examples of control functions that can be implemented in the 340 CE include routing protocols like RIP, OSPF and BGP, control and 341 signaling protocols like RSVP (Resource Reservation Protocol), LDP 342 (Label Distribution Protocol) for MPLS, etc. Examples of 343 forwarding functions in the FE include LPM (longest prefix match) 344 forwarder, classifiers, traffic shaper, meter, NAT (Network Address 345 Translators), etc. Figure 3 provides example functions in both CE 346 and FE. Any given NE may contain one or many of these CE and FE 347 functions in it. The diagram also shows that ForCES Protocol is 348 used to transport both the control messages for ForCES itself and 349 the data packets that are originated/destined from/to the control 350 functions in CE (e.g., routing packets). Section 4.2.4 provides 351 more detail on this. 353 ------------------------------------------------- 354 | | | | | | | 355 |OSPF |RIP |BGP |RSVP |LDP |. . . | 356 | | | | | | | 357 ------------------------------------------------- 358 | ForCES Interface | 359 ------------------------------------------------- 360 ^ ^ 361 ForCES | |data 362 control | |packets 363 messages| |(e.g., routing packets) 364 v v 365 ------------------------------------------------- 366 | ForCES Interface | 367 ------------------------------------------------- 368 | | | | | | | 369 |LPM Fwd|Meter |Shaper |NAT |Classi-|. . . | 370 | | | | |fier | | 371 ------------------------------------------------- 372 | FE resources | 373 ------------------------------------------------- 374 Figure 3. Examples of CE and FE functions 376 A set of requirements for control and forwarding separation is 377 identified in [3]. This document describes a ForCES architecture 378 that satisfies the architectural requirements of that document and 379 defines a framework for ForCES network elements and the associated 380 entities to facilitate protocol definition. Whenever necessary, 381 this document uses many examples to illustrate the issues and/or 382 possible solutions in ForCES. These examples are intended to be 383 just examples, and should not be taken as the only or definite ways 384 of doing certain things. It is expected that separate document 385 will be produced by the ForCES working group to specify the ForCES 386 Protocol. 388 3. Architecture 390 This section defines the ForCES architectural framework and the 391 associated logical components. This ForCES framework defines 392 components of ForCES NEs including several ancillary components. 393 These components may be connected in different kinds of topologies 394 for flexible packet processing. 396 --------------------------------------- 397 | ForCES Network Element | 398 -------------- Fc | -------------- -------------- | 399 | CE Manager |---------+-| CE 1 |------| CE 2 | | 400 -------------- | | | Fr | | | 401 | | -------------- -------------- | 402 | Fl | | | Fp / | 403 | | Fp| |----------| / | 404 | | | |/ | 405 | | | | | 406 | | | Fp /|----| | 407 | | | /--------/ | | 408 -------------- Ff | -------------- -------------- | 409 | FE Manager |---------+-| FE 1 | Fi | FE 2 | | 410 -------------- | | |------| | | 411 | -------------- -------------- | 412 | | | | | | | | | | 413 ----+--+--+--+----------+--+--+--+----- 414 | | | | | | | | 415 | | | | | | | | 416 Fi/f Fi/f 418 Fp: CE-FE interface 419 Fi: FE-FE interface 420 Fr: CE-CE interface 421 Fc: Interface between the CE Manager and a CE 422 Ff: Interface between the FE Manager and an FE 423 Fl: Interface between the CE Manager and the FE Manager 424 Fi/f: FE external interface 426 Figure 4. ForCES Architectural Diagram 428 The diagram in Figure 4 shows the logical components of the ForCES 429 architecture and their relationships. There are two kinds of 430 components inside a ForCES network element: control element (CE) 431 and forwarding element (FE). The framework allows multiple 432 instances of CE and FE inside one NE. Each FE contains one or more 433 physical media interfaces for receiving and transmitting packets 434 from/to the external world. The aggregation of these FE interfaces 435 becomes the NE's external interfaces. In addition to the external 436 interfaces, there must also exist some kind of interconnect within 437 the NE so that the CE and FE can communicate with each other, and 438 one FE can forward packets to another FE. The diagram also shows 439 two entities outside of the ForCES NE: CE Manager and FE Manager. 440 These two ancillary entities provide configuration to the 441 corresponding CE or FE in the pre-association phase (see Section 442 4.1). 444 For convenience, the logical interactions between these components 445 are labeled by reference points Fp, Fc, Ff, Fr, Fl, and Fi, as 446 shown in Figure 4. The FE external interfaces are labeled as Fi/f. 447 More detail is provided in Section 3 and 4 for each of these 448 reference points. All these reference points are important in 449 understanding the ForCES architecture, however, the ForCES Protocol 450 is only defined over one reference point -- Fp. 452 The interface between two ForCES NEs is identical to the interface 453 between two conventional routers and these two NEs exchange the 454 protocol packets through the external interfaces at Fi/f. ForCES 455 NEs connect to existing routers transparently. 457 3.1. Control Elements and Fr Reference Point 459 It is not necessary to define any protocols across the Fr reference 460 point to enable control and forwarding separation for simple 461 configurations like single CE and multiple FEs. However, this 462 architecture permits multiple CEs to be present in a network 463 element. In cases where an implementation uses multiple CEs, the 464 invariant that the CEs and FEs together appear as a single NE must 465 be maintained. 467 Multiple CEs may be used for redundancy, load sharing, distributed 468 control, or other purposes. Redundancy is the case where one or 469 more CEs are prepared to take over should an active CE fail. Load 470 sharing is the case where two or more CEs are concurrently active 471 and any request that can be serviced by one of the CEs can also be 472 serviced by any of the other CEs. For both redundancy and load 473 sharing, the CEs involved are equivalently capable. The only 474 difference between these two cases is in terms of how many active 475 CEs there are. Distributed control is the case where two or more 476 CEs are concurrently active but certain requests can only be 477 serviced by certain CEs. 479 When multiple CEs are employed in a ForCES NE, their internal 480 organization is considered an implementation issue that is beyond 481 the scope of ForCES. CEs are wholly responsible for coordinating 482 amongst themselves via the Fr reference point to provide 483 consistency and synchronization. However, ForCES does not define 484 the implementation or protocols used between CEs, nor does it 485 define how to distribute functionality among CEs. Nevertheless, 486 ForCES will support mechanisms for CE redundancy or fail over, and 487 it is expected that vendors will provide redundancy or fail over 488 solutions within this framework. 490 3.2. Forwarding Elements and Fi reference point 492 An FE is a logical entity that implements the ForCES Protocol and 493 uses the underlying hardware to provide per-packet processing and 494 handling as directed by a CE. It is possible to partition one 495 physical FE into multiple logical FEs. It is also possible for one 496 FE to use multiple physical FEs. The mapping between physical 497 FE(s) and the logical FE(s) is beyond the scope of ForCES. For 498 example, a logical partition of a physical FE can be created by 499 assigning some portion of each of the resources (e.g., ports, 500 memory, forwarding table entries) available on the ForCES physical 501 FE to each of the logical FEs. Such concept of FE virtualization 502 is analogous to a virtual switching element as described in [8]. 503 If FE virtualization occurs only in the pre-association phase, it 504 has no impact on ForCES. However, if FE virtualization results in 505 resource change taken from an existing FE (already participating in 506 ForCES post-association phase), the ForCES Protocol needs to be 507 able to inform the CE of such change via asynchronous messages (see 508 [3], Section 5, requirement #6). 510 FEs perform all packet processing functions as directed by CEs. 512 FEs have no initiative of their own. Instead, FEs are slaves and 513 only do as they are told. FEs may communicate with one or more CEs 514 concurrently across reference point Fp. FEs have no notion of CE 515 redundancy, load sharing, or distributed control. Instead, FEs 516 accept commands from any CE authorized to control them, and it is 517 up to the CEs to coordinate among themselves to achieve redundancy, 518 load sharing or distributed control. The idea is to keep FEs as 519 simple and dumb as possible so that FEs can focus their resource on 520 the packet processing functions. Unless otherwise configured or 521 determined by a ForCEs Protocol exchange, each FE will process 522 authorized incoming commands directed at it as it receives them on 523 a first come first serve basis. 525 For example, in Figure 5, FE1 and FE2 can be configured to accept 526 commands from both the primary CE (CE1) and the backup CE (CE2). 527 Upon detection of CE1 failure, perhaps across the Fr or Fp 528 reference point, CE2 is configured to take over activities of CE1. 529 This is beyond the scope of ForCES and is not discussed further. 531 Distributed control can be achieved in a similar fashion, without 532 much intelligence on the part of FEs. For example, FEs can be 533 configured to detect RSVP and BGP protocol packets, and forward 534 RSVP packets to one CE and BGP packets to another CE. Hence, FEs 535 may need to do packet filtering for forwarding packets to specific 536 CEs. 538 ------- Fr ------- 539 | CE1 | ------| CE2 | 540 ------- ------- 541 | \ / | 542 | \ / | 543 | \ / | 544 | \/Fp | 545 | /\ | 546 | / \ | 547 | / \ | 548 ------- Fi ------- 549 | FE1 |<----->| FE2 | 550 ------- ------- 552 Figure 5. CE redundancy example. 554 This architecture permits multiple FEs to be present in an NE. [3] 555 dictates that the ForCES Protocol must be able to scale to at least 556 hundreds of FEs (see [3] Section 5, requirement #11). Each of 557 these FEs may potentially have a different set of packet processing 558 functions, with different media interfaces. FEs are responsible 559 for basic maintenance of layer-2 connectivity with other FEs and 560 with external entities. Many layer-2 media include sophisticated 561 control protocols. The FORCES Protocol (over the Fp reference 562 point) will be able to carry messages for such protocols so that, 563 in keeping with the dumb FE model, the CE can provide appropriate 564 intelligence and control over these media. 566 When multiple FEs are present, ForCES requires that packets must be 567 able to arrive at the NE by one FE and leave the NE via a different 568 FE (See [3], Section 5, Requirement #3). Packets that enter the NE 569 via one FE and leave the NE via a different FE are transferred 570 between FEs across the Fi reference point. Fi reference point 571 could be used by FEs to discovery their (inter-FE) topology, 572 perhaps during pre-association phase. The Fi reference point is a 573 separate protocol from the Fp reference point and is not currently 574 defined by the ForCES architecture. 576 FEs could be connected in different kinds of topologies and packet 577 processing may spread across several FEs in the topology. Hence, 578 logical packet flow may be different from physical FE topology. 579 Figure 6 provides some topology examples. When it is necessary to 580 forward packets between FEs, the CE needs to understand the FE 581 topology. The FE topology may be queried from the FEs by the CEs 582 via ForCES Protocol, but the FEs are not required to provide that 583 information to the CEs. So, the FE topology information may also be 584 gathered by other means outside of the ForCES Protocol (like inter- 585 FE topology discovery protocol). 587 ----------------- 588 | CE | 589 ----------------- 590 ^ ^ ^ 591 / | \ 592 / v \ 593 / ------- \ 594 / +->| FE3 |<-+ \ 595 / | | | | \ 596 v | ------- | v 597 ------- | | ------- 598 | FE1 |<-+ +->| FE2 | 599 | |<--------------->| | 600 ------- ------- 601 ^ | ^ | 602 | | | | 603 | v | v 605 (a) Full mesh among FE1, FE2 and FE3. 607 ----------- 608 | CE | 609 ----------- 610 ^ ^ ^ ^ 611 / | | \ 612 /------ | | ------\ 613 v v v v 614 ------- ------- ------- ------- 615 | FE1 |<->| FE2 |<->| FE3 |<->| FE4 | 616 ------- ------- ------- ------- 617 ^ | ^ | ^ | ^ | 618 | | | | | | | | 619 | v | v | v | v 621 (b) Multiple FEs in a daisy chain 623 ^ | 624 | v 625 ----------- 626 | FE1 |<-----------------------| 627 ----------- | 628 ^ ^ | 629 / \ | 630 | ^ / \ ^ | V 631 v | v v | v ---------- 632 --------- --------- | | 633 | FE2 | | FE3 |<------------>| CE | 634 --------- --------- | | 635 ^ ^ ^ ---------- 636 | \ / ^ ^ 637 | \ / | | 638 | v v | | 639 | ----------- | | 640 | | FE4 |<----------------------| | 641 | ----------- | 642 | | ^ | 643 | v | | 644 | | 645 |----------------------------------------| 647 (c) Multiple FEs connected by a ring 649 Figure 6. Some examples of FE topology. 651 3.3.CE Managers 653 CE managers are responsible for determining which FEs a CE should 654 control. It is legitimate for CE managers to be hard-coded with 655 the knowledge of with which FEs its CEs should communicate with. A 656 CE manager may also be physically embedded into a CE and be 657 implemented as a simple keypad or other direct configuration 658 mechanism on the CE. Finally, CE managers may be physically and 659 logically separate entities that configure the CE with FE 660 information via such mechanisms as COPS-PR [6] or SNMP [4]. 662 3.4. FE Managers 664 FE managers are responsible for determining with which CE any 665 particular FE should initially communicate. Like CE managers, no 666 restrictions are placed on how an FE manager decides with which CE 667 its FEs should communicate, nor are restrictions placed on how FE 668 managers are implemented. Each FE should have one and only one FE 669 manager, while different FEs may have the same or different FE 670 manager(s). Each manager can choose to exist and operate 671 independently of other manager. 673 4. Operational Phases 675 Both FEs and CEs require some configuration in place before they 676 can start information exchange and function as a coherent network 677 element. Two operational phases are identified in this framework: 678 pre-association and post-association. 680 4.1.Pre-association Phase 682 Pre-association phase is the period of time during which an FE 683 Manager and a CE Manager are determining whether an FE and a CE 684 should be part of the same network element. The protocols used 685 during this phase may include all or some of the message exchange 686 over Fl, Ff and Fc reference points. However, all these may be 687 optional and none of this is within the scope of ForCES Protocol. 689 4.1.1. Fl Reference Point 691 CE managers and FE managers may communicate across the Fl reference 692 point in the pre-association phase in order to determine whether an 693 individual CE and FE, or a set of CEs and FEs should be associated. 694 Communication across the Fl reference point is optional in this 695 architecture. No requirements are placed on this reference point. 697 CE managers and FE managers may be operated by different entities. 698 The operator of the CE manager may not want to divulge, except to 699 specified FE managers, any characteristics of the CEs it manages. 700 Similarly, the operator of the FE manager may not want to divulge 701 FE characteristics, except to authorized entities. As such, CE 702 managers and FE managers may need to authenticate one another. 703 Subsequent communication between CE managers and FE managers may 704 require other security functions such as privacy, non-repudiation, 705 freshness, and integrity. 707 FE Manager FE CE Manager CE 708 | | | | 709 | | | | 710 |(security exchange) | | 711 1|<------------------------------>| | 712 | | | | 713 |(a list of CEs and their attributes) | 714 2|<-------------------------------| | 715 | | | | 716 |(a list of FEs and their attributes) | 717 3|------------------------------->| | 718 | | | | 719 | | | | 720 |<----------------Fl------------>| | 722 Figure 7. An example of message exchange over Fl reference point 724 Once the necessary security functions have been performed, the CE 725 and FE managers communicate to determine which CEs and FEs should 726 communicate with each other. At the very minimum, the CE and FE 727 managers need to learn of the existence of available FEs and CEs 728 respectively. This discovery process may entail one or both 729 managers learning the capabilities of the discovered ForCES 730 protocol elements. Figure 7 shows an example of possible message 731 exchange between CE manager and FE manager over Fl reference point. 733 4.1.2. Ff Reference Point 735 The Ff reference point is used to inform forwarding elements of the 736 association decisions made by the FE manager in pre-association 737 phase. Only authorized entities may instruct an FE with respect to 738 which CE should control it. Therefore, privacy, integrity, 739 freshness, and authentication are necessary between the FE manager 740 and FEs when the FE manager is remote to the FE. Once the 741 appropriate security has been established, the FE manager instructs 742 the FEs across this reference point to join a new NE or to 743 disconnect from an existing NE. The FE Manager could also assign 744 unique FE identifiers to the FEs using this reference point. The 745 FE identifiers are useful in post association phase to express FE 746 topology. Figure 8 shows example of message exchange over Ff 747 reference point. 749 FE Manager FE CE Manager CE 750 | | | | 751 | | | | 752 |(security exchange) |(security exchange) 753 1|<------------>|authentication 1|<----------->|authentication 754 | | | | 755 |(FE ID, attributes) |(CE ID, attributes) 756 2|<-------------|request 2|<----------->|request 757 | | | | 758 3|------------->|response 3|------------>|response 759 |(corresponding CE ID) |(corresponding FE ID) 760 | | | | 761 | | | | 762 |<-----Ff----->| |<-----Fc---->| 764 Figure 8. Examples of message exchange 765 over Ff and Fc reference points. 767 Note that the FE manager function may be co-located with the FE 768 (such as by manual keypad entry of the CE IP address), in which 769 case this reference point is reduced to a built-in function. 771 4.1.3. Fc Reference Point 773 The Fc reference point is used to inform control elements of the 774 association decisions made by CE managers in pre-association phase. 775 When the CE manager is remote, only authorized entities may 776 instruct a CE to control certain FEs. Privacy, integrity, 777 freshness and authentication are also required across this 778 reference point in such a configuration. Once appropriate security 779 has been established, the CE manager instructs CEs as to which FEs 780 they should control and how they should control them. Figure 8 781 shows example of message exchange over Fc reference point. 783 As with the FE manager and FEs, configurations are possible where 784 the CE manager and CE are co-located and no protocol is used for 785 this function. 787 4.2. Post-association Phase and Fp reference point 788 Post-association phase is the period of time during which an FE and 789 CE have been configured with information necessary to contact each 790 other and includes both association establishment and steady-state 791 communication. The communication between CE and FE is performed 792 across the Fp ("p" meaning protocol) reference point. ForCES 793 Protocol is exclusively used for all communication across the Fp 794 reference point. 796 4.2.1. Proximity and Interconnect between CEs and FEs 798 The ForCES Working Group has made a conscious decision that the 799 first version of ForCES will be focused on "very close" CE/FE 800 localities in IP networks. Very Close localities consist of 801 control and forwarding elements that either are components in the 802 same physical box, or are separated at most by one local network 803 hop ([7]). CEs and FEs can be connected by a variety of 804 interconnect technologies, including Ethernet connections, 805 backplanes, ATM (cell) fabrics, etc. ForCES should be able to 806 support each of these interconnects (see [3] Section 5, requirement 807 #1). When the CEs and FEs are separated beyond a single L3 routing 808 hop, the ForCES Protocol will make use of an existing RFC2914 809 compliant L4 protocol with adequate reliability, security and 810 congestion control (e.g. TCP, SCTP) for transport purposes. 812 4.2.2. Association Establishment 814 FE CE 815 | | 816 |(Security exchange.) | 817 1|<--------------------->| 818 | | 819 |(Let me join the NE please.) 820 2|---------------------->| 821 | | 822 |(What kind of FE are you? -- capability query) 823 3|<----------------------| 824 | | 825 |(Here is my FE functions/state: use model to 826 describe) 827 4|---------------------->| 828 | | 829 |(Initial config for FE -- optional) 830 5|<----------------------| 831 | | 832 |(I am ready to go. Shall I?) 833 6|---------------------->| 834 | | 835 |(Go ahead!) | 836 7|<----------------------| 837 | | 839 Figure 9. Example of message exchange between CE and FE 840 over Fp to establish NE association 842 As an example, figure 9 shows some of the message exchange that may 843 happen before the association between the CE and FE is fully 844 established. Either the CE or FE can initiate the connection. 846 Security handshake is necessary to authenticate the two 847 communication endpoints to each other before any further message 848 exchange can happen. The security handshake should include mutual 849 authentication and authorization between the CE and FE, but the 850 exact details depend on the security solution chosen by ForCES 851 Protocol. Authorization can be as simple as checking against the 852 list of authorized end points provided by the FE or CE manager 853 during the pre-association phase. Both authentication and 854 authorization must be successful before the association can be 855 established. If either authentication or authorization fails, the 856 end point must not be allowed to join the NE. After the successful 857 security handshake, message authentication and confidentiality are 858 still necessary for the on-going information exchange between the 859 CE and FE, unless some form of physical security exists. Whenever 860 a packet fails authentication, it must be dropped and a 861 notification may be sent to alert the sender of the potential 862 attack. Section 8 provides more details on the security 863 considerations for ForCES. 865 After the successful security handshake, the FE needs to inform the 866 CE of its own capability and optionally its topology in relation to 867 other FEs. The capability of the FE is represented by the FE model, 868 described in a separate document. The model would allow an FE to 869 describe what kind of packet processing functions it contains, in 870 what order the processing happens, what kinds of configurable 871 parameters it allows, what statistics it collects and what events 872 it might throw, etc. Once such information is available to the CE, 873 the CE may choose to send some initial or default configuration to 874 the FE so that the FE can start receiving and processing packets 875 correctly. Such initialization may not be necessary if the FE 876 already obtains the information from its own bootstrap process. 877 Once the necessary initial information is exchanged, the process of 878 association is completed. Packet processing and forwarding at the 879 FE cannot begin until association is established. After the 880 association is established, the CE and FE enter steady-state 881 communication. 883 4.2.3. Steady-state Communication 885 Once an association is established between the CE and FE, the 886 ForCES Protocol is used by the CE and FE over Fp reference point to 887 exchange information to facilitate packet processing. 889 FE CE 890 | | 891 |(Add these new routes.)| 892 1|<----------------------| 893 | | 894 |(Successful.) | 895 2|---------------------->| 896 | | 897 | | 898 |(Query some stats.) | 899 1|<----------------------| 900 | | 901 |(Reply with stats collected.) 902 2|---------------------->| 903 | | 904 | | 905 |(My port is down, with port #.) 906 1|---------------------->| 907 | | 908 |(Here is a new forwarding table) 909 2|<----------------------| 910 | | 911 Figure 10. Examples of message exchange between CE and FE 912 over Fp during steady-state communication 914 Based on the information acquired through CEs' control processing, 915 CEs will frequently need to manipulate the packet-forwarding 916 behaviors of their FE(s) by sending instructions to FEs. For 917 example, Figure 10 shows message exchange examples in which the CE 918 sends new routes to the FE so that the FE can add them to its 919 forwarding table. The CE may query the FE for statistics collected 920 by the FE and the FE may notify the CE of important events such as 921 port failure. 923 4.2.4. Data Packets across Fp reference point 924 --------------------- ---------------------- 925 | | | | 926 | +--------+ | | +--------+ | 927 | |CE(BGP) | | | |CE(BGP) | | 928 | +--------+ | | +--------+ | 929 | | | | ^ | 930 | |Fp | | |Fp | 931 | v | | | | 932 | +--------+ | | +--------+ | 933 | | FE | | | | FE | | 934 | +--------+ | | +--------+ | 935 | | | | ^ | 936 | Router | | | Router | | 937 | A | | | B | | 938 ---------+----------- -----------+---------- 939 v ^ 940 | | 941 | | 942 ------------------->--------------- 944 Figure 11. Example to show data packet flow between two NEs. 946 Control plane protocol packets (such as RIP, OSPF messages) 947 addressed to any of NE's interfaces are typically redirected by the 948 receiving FE to its CE, and CE may originate packets and have its 949 FE deliver them to other NEs. Therefore, ForCES Protocol over Fp 950 not only transports the ForCES Protocol messages between CEs and 951 FEs, but also encapsulates the data packets from control plane 952 protocols. Moreover, one FE may be controlled by multiple CEs for 953 distributed control. In this configuration, the control protocols 954 supported by the FORCES NEs may spread across multiple CEs. For 955 example, one CE may support routing protocols like OSPF and BGP, 956 while a signaling and admission control protocol like RSVP is 957 supported in another CE. FEs are configured to recognize and 958 filter these protocol packets and forward them to the corresponding 959 CE. 961 Figure 11 shows one example of how the BGP packets originated by 962 router A are passed to router B. In this example, the ForCES 963 Protocol is used to transport the packets from the CE to the FE 964 inside router A, and then from the FE to the CE inside router B. 965 In light of the fact that the ForCES Protocol is responsible for 966 transporting both the control messages and the data packets between 967 the CE and FE over Fp reference point, it is possible to use either 968 a single protocol or multiple protocols to achieve this. 970 4.2.5. Proxy FE 972 In the case where a physical FE cannot implement (e.g., due to the 973 lack of a general purpose CPU) the ForCES Protocol directly, a 974 proxy FE can be used to terminate the Fp reference point instead of 975 the physical FE. This allows the CE communicate to the physical FE 976 via the proxy by using ForCES, while the proxy manipulates the 977 physical FE using some intermediary form of communication (e.g., a 978 non-ForCES protocol or DMA). In such an implementation, the 979 combination of the proxy and the physical FE becomes one logical FE 980 entity. It is also possible that one proxy act on behalf of 981 multiple physical FEs. 983 One needs to be aware of the security implication introduced by the 984 proxy FE. Since the physical FE is not capable of implementing 985 ForCES itself, the security mechanism of ForCES can only secure the 986 communication channel between the CE and the proxy FE, but not all 987 the way to the physical FE. It is recommended that other security 988 mechanisms (including physical security property) be employed to 989 ensure the security between the CE and the physical FE. 991 4.3. Association Re-establishment 993 FEs and CEs may join and leave NEs dynamically (see [3] Section 5, 994 requirements #12). When an FE or CE leaves the NE, the association 995 with the NE is broken. If the leaving party rejoins an NE later, 996 to re-establish the association, it may need to re-enter the pre- 997 association phase. Loss of association can also happen 998 unexpectedly due to loss of connection between the CE and the FE. 999 Therefore, the framework allows the bi-directional transition 1000 between these two phases, but the ForCES Protocol is only 1001 applicable for the post-association phase. However, the protocol 1002 should provide mechanisms to support association re-establishment. 1003 This includes the ability for CEs and FEs to determine when there 1004 is a loss of association between them, ability to restore 1005 association and efficient state (re)synchronization mechanisms (see 1006 [3] Section 5, requirement #7). Note that security association and 1007 state must be also re-established to guarantee the same level of 1008 security (including both authentication and authorization) exists 1009 before and after the association re-establishment. 1011 When an FE leaves or joins an existing NE that is already in 1012 operation, the CE needs to be aware of the impact on FE topology 1013 and deals with the change accordingly. 1015 4.3.1. CE graceful restart 1016 The failure and restart of the CE in a router can potentially cause 1017 much stress and disruption on the control plane throughout a 1018 network. Because when a CE has to restart for any reason, the 1019 router loses routing adjacencies or sessions with its routing 1020 neighbors. Neighbors who detect the lost adjacency normally re- 1021 compute new routes and then send routing updates to their own 1022 neighbors to communicate the lost adjacency. Their neighbors do 1023 the same thing to propagate throughout the network. In the 1024 meantime, the restarting router cannot receive traffic from other 1025 routers because the neighbors have stopped using the router's 1026 previously advertised routes. When the restarting router restores 1027 adjacencies, neighbors must once again re-compute new routes and 1028 send out additional routing updates. The restarting router is 1029 unable to forward packets until it has re-established routing 1030 adjacencies with neighbors, received route updates through these 1031 adjacencies, and computed new routes. Until convergence takes 1032 place throughout the network, packets may be lost in transient 1033 black holes or forwarding loops. 1035 A high availability mechanism known as the "graceful restart" has 1036 been used by the IP routing protocols (OSPF [10], BGP [11]) and 1037 MPLS label distribution protocol (LDP [9]) to help minimize the 1038 negative effects on routing throughout an entire network caused by 1039 a restarting router. Route flap on neighboring routers is avoided, 1040 and a restarting router can continue to forward packets that would 1041 otherwise be dropped. 1043 While the details differ from protocol to protocol, the general 1044 idea behind the graceful restart mechanism remains the same. With 1045 the graceful restart, a restarting router can inform its neighbors 1046 when it restarts. The neighbors may detect the lost adjacency but 1047 do not recompute new routes or send routing updates to their 1048 neighbors. The neighbors also hold on to the routes received from 1049 the restarting router before restart and assume they are still 1050 valid for a limited time. By doing so, the restarting router's FEs 1051 can also continue to receive and forward traffic from other 1052 neighbors for a limited time by using the routes they already have. 1053 The restarting router then re-establishes routing adjacencies, 1054 downloads updated routes from all its neighbors, recomputes new 1055 routes and uses them to replace the older routes it was using. It 1056 then sends these updated routes to its neighbors and signals the 1057 completion of the graceful restart process. 1059 Non-stop forwarding is a requirement for graceful restart. It is 1060 necessary so a router can continue to forward packets while it is 1061 downloading routing information and recomputing new routes. This 1062 ensures that packets will not be dropped. As one can see, one of 1063 the benefits afforded by the separation of CE and FE is exactly the 1064 ability of non-stop forwarding in the face of the CE failure and 1065 restart. The support of dynamic changes to CE/FE association in 1066 ForCES also makes it compatible with high availability mechanisms 1067 such as graceful restart. 1069 ForCES should be able to support CE graceful restart easily. When 1070 the association is established the first time, the CE must inform 1071 the FEs what to do in the case of CE failure. If graceful restart 1072 is not supported, the FEs may be told to stop packet processing all 1073 together if its CE fails. If graceful restart is supported, the 1074 FEs should be told to cache and hold on to its FE state including 1075 the forwarding tables across the restarts. A timer must be 1076 included so that the timeout causes such cached state to expire 1077 eventually. Those timers should be settable by the CE. 1079 4.3.2. FE restart 1081 In the same example in Figure 5, assuming CE1 is the working CE for 1082 the moment, what would happen if one of the FEs, say FE1, leaves 1083 the NE temporarily? FE1 may voluntarily decide to leave the 1084 association. Alternatively, FE1 may stop functioning simply due to 1085 unexpected failure. In the former case, CE1 receives a "leave- 1086 association request" from FE1. In the latter, CE1 detects the 1087 failure of FE1 by some other means. In both cases, CE1 must inform 1088 the routing protocols of such an event, most likely prompting a 1089 reachability and SPF (Shortest Path First) recalculation and 1090 associated downloading of new FIBs from CE1 to the other remaining 1091 FEs (only FE2 in this example). Such recalculation and FIB update 1092 will also be propagated from CE1 to the NE's neighbors that are 1093 affected by the connectivity of FE1. 1095 When FE1 decides to rejoin again, or when it restarts again from 1096 the failure, FE1 needs to re-discover its master (CE). This can be 1097 achieved by several means. It may re-enter the pre-association 1098 phase and get that information from its FE manager. It may 1099 retrieve the previous CE information from its cache, if it can 1100 validate the information freshness. Once it discovers its CE, it 1101 starts message exchange with the CE to re-establish the association 1102 just as outlined in Figure 9, with the possible exception that it 1103 might be able to bypass the transport of the complete initial 1104 configuration. Suppose that FE1 still has its routing table and 1105 other state information from the last association. Instead of 1106 sending all the information again from scratch, it may be able to 1107 use more efficient mechanism to re-sync up the state with its CE if 1108 such mechanism is supported by the ForCES Protocol. For example, 1109 CRC-32 of the state might give a quick indication of whether or not 1110 the state is in-sync with its CE. By comparing its state with the 1111 CE first, it sends an information update only if it is needed. 1112 ForCES Protocol may choose to implement similar optimization 1113 mechanisms, but it may also choose not to, as this is not a 1114 requirement. 1116 5. Applicability to RFC1812 1118 [3] Section 5, requirement #9 dictates "Any proposed ForCES 1119 architecture must explain how that architecture supports all of the 1120 router functions as defined in RFC1812." RFC1812 discusses many 1121 important requirements for IPv4 routers from the link layer to the 1122 application layer. This section addresses the relevant 1123 requirements in RFC1812 for implementing IPv4 routers based on 1124 ForCES architecture and explains how ForCES satisfies these 1125 requirements by providing guidelines on how to separate the 1126 functionalities required into forwarding plane and control plane. 1128 In general, the forwarding plane carries out the bulk of the per- 1129 packet processing that is required at line speed, while the control 1130 plane carries most of the computationally complex operations that 1131 are typical of the control and signaling protocols. However, it is 1132 impossible to draw a rigid line to divide the processing into CEs 1133 and FEs cleanly. Nor should the ForCES architecture limit the 1134 innovative approaches in control and forwarding plane separation. 1135 As more and more processing power is available in the FEs, some of 1136 the control functions that traditionally are performed by CEs may 1137 now be moved to FEs for better performance and scalability. Such 1138 offloaded functions may include part of ICMP or TCP processing, or 1139 part of routing protocols. Once off-loaded onto the forwarding 1140 plane, such CE functions, even though logically belonging to the 1141 control plane, now become part of the FE functions. Just like the 1142 other logical functions performed by FEs, such off-loaded functions 1143 must be expressed as part of the FE model so that the CEs can 1144 decide how to best take advantage of these off-loaded functions 1145 when present on the FEs. 1147 5.1. General Router Requirements 1149 Routers have at least two or more logical interfaces. When CEs and 1150 FEs are separated by ForCES within a single NE, some additional 1151 interfaces are needed for intra-NE communications. Figure 12 shows 1152 an example to illustrate that. This NE contains one CE and two 1153 FEs. Each FE has four interfaces; two of them are used for 1154 receiving and transmitting packets to the external world, while the 1155 other two are for intra-NE connections. CE has two logical 1156 interfaces #9 and #10, connected to interfaces #3 and #6 from FE1 1157 and FE2, respectively. Interface #4 and #5 are connected for FE1- 1158 FE2 communication. Therefore, this router NE provides four 1159 external interfaces (#1, 2, 7 and 8). 1161 --------------------------------- 1162 | router NE | 1163 | ----------- ----------- | 1164 | | FE1 | | FE2 | | 1165 | ----------- ----------- | 1166 | 1| 2| 3| 4| 5| 6| 7| 8| | 1167 | | | | | | | | | | 1168 | | | | +----+ | | | | 1169 | | | | | | | | 1170 | | | 9| 10| | | | 1171 | | | -------------- | | | 1172 | | | | CE | | | | 1173 | | | -------------- | | | 1174 | | | | | | 1175 -----+--+----------------+--+---- 1176 | | | | 1177 | | | | 1179 Figure 12. A router NE example with four interfaces. 1181 IPv4 routers must implement IP to support its packet forwarding 1182 function, which is driven by its FIB (Forwarding Information Base). 1183 This Internet layer forwarding (see RFC1812 [1] Section 5) 1184 functionality naturally belongs to FEs in the ForCES architecture. 1186 A router may implement transport layer protocols (like TCP and UDP) 1187 that are required to support application layer protocols (see 1188 RFC1812 [1] Section 6). One important class of application 1189 protocols is routing protocols (see RFC1812 [1] Section 7). In 1190 ForCES architecture, routing protocols are naturally implemented by 1191 CEs. Routing protocols require routers communicate with each 1192 other. This communication between CEs in different routers is 1193 supported in ForCES by FEs' ability to redirect data packets 1194 addressed to routers (i.e., NEs) and CEs' ability to originate 1195 packets and have them delivered by their FEs. This communication 1196 occurs across Fp reference point inside each router and between 1197 neighboring routers' external interfaces, as illustrated in Figure 1198 11. 1200 5.2.Link Layer 1201 Since FEs own all the external interfaces for the router, FEs need 1202 to conform to the link layer requirements in RFC1812. Arguably, 1203 ARP support may be implemented in either CEs or FEs. As we will 1204 see later, a number of behaviors that RFC1812 mandates fall into 1205 this category -- they may be performed by the FE and may be 1206 performed by the CE. A general guideline is needed to ensure 1207 interoperability between separated control and forwarding planes. 1208 The guideline we offer here is that CEs MUST be capable of these 1209 kind of operations while FEs MAY choose to implement them. FE 1210 model should indicate its capabilities in this regard so that CEs 1211 can decide where these functions are implemented. 1213 Interface parameters, including MTU, IP address, etc., must be 1214 configurable by CEs via ForCES. CEs must be able to determine 1215 whether a physical interface in an FE is available to send packets 1216 or not. FEs must also inform CEs the status change of the 1217 interfaces (like link up/down) via ForCES. 1219 5.3.Internet Layer Protocols 1221 Both FEs and CEs must implement IP protocol and all mandatory 1222 extensions as RFC1812 specified. CEs should implement IP options 1223 like source route and record route while FEs may choose to 1224 implement those as well. The timestamp option should be 1225 implemented by FEs to insert the timestamp most accurately. The FE 1226 must interpret the IP options that it understands and preserve the 1227 rest unchanged for use by CEs. Both FEs and CEs might choose to 1228 silently discard packets without sending ICMP errors, but such 1229 events should be logged and counted. FEs may report statistics for 1230 such events to CEs via ForCES. 1232 When multiple FEs are involved to process packets, the appearance 1233 of single NE must be strictly maintained. For example, Time-To- 1234 Live (TTL) must be decremented only once within a single NE. For 1235 example, it can be always decremented by the last FE with egress 1236 function. 1238 FEs must receive and process normally any packets with a broadcast 1239 destination address or a multicast destination address that the 1240 router has asked to receive. When IP multicast is supported in 1241 routers, IGMP is implemented in CEs. CEs are also required of ICMP 1242 support, while it is optional for FEs to support ICMP. Such an 1243 option can be communicated to CEs as part of the FE model. 1244 Therefore, FEs can always rely upon CEs to send out ICMP error 1245 messages, but FEs also have the option to generate ICMP error 1246 messages themselves. 1248 5.4.Internet Layer Forwarding 1250 IP forwarding is implemented by FEs. When the routing table is 1251 updated at CEs, ForCES is used to send the new route entries from 1252 CEs to FEs. Each FE has its own forwarding table and uses this 1253 table to direct packets to the next hop interface. 1255 Upon receiving IP packets, the FE verifies the IP header and 1256 processes most of the IP options. Some options cannot be processed 1257 until the routing decision has been made. The routing decision is 1258 made after examining the destination IP address. If the 1259 destination address belongs to the router itself, the packets are 1260 filtered and either processed locally or forwarded to CE, depending 1261 upon the instructions set-up by CE. Otherwise, the FE determines 1262 the next hop IP address by looking up in its forwarding table. The 1263 FE also determines the network interface it uses to send the 1264 packets. Sometimes an FE may need to forward the packets to 1265 another FE before packets can be forwarded out to the next hop. 1266 Right before packets are forwarded out to the next hop, the FE 1267 decrements TTL by 1 and processes any IP options that cannot be 1268 processed before. The FE performs any IP fragmentation if 1269 necessary, determines link layer address (e.g., by ARP), and 1270 encapsulates the IP datagram (or each of the fragments thereof) in 1271 an appropriate link layer frame and queues it for output on the 1272 interface selected. 1274 Other options mentioned in RFC1812 for IP forwarding may also be 1275 implemented at FEs, for example, packet filtering. 1277 FEs typically forward packets destined locally to CEs. FEs may 1278 also forward exceptional packets (packets that FEs do not know how 1279 to handle) to CEs. CEs are required to handle packets forwarded by 1280 FEs for whatever different reasons. It might be necessary for 1281 ForCES to attach some meta-data with the packets to indicate the 1282 reasons of forwarding from FEs to CEs. Upon receiving packets with 1283 meta-data from FEs, CEs can decide to either process the packets 1284 themselves, or pass the packets to the upper layer protocols 1285 including routing and management protocols. If CEs are to process 1286 the packets by themselves, CEs may choose to discard the packets, 1287 or modify and re-send the packets. CEs may also originate new 1288 packets and deliver them to FEs for further forwarding. 1290 Any state change during router operation must also be handled 1291 correctly according to RFC1812. For example, when an FE ceases 1292 forwarding, the entire NE may continue forwarding packets, but it 1293 needs to stop advertising routes that are affected by the failed 1294 FE. 1296 5.5. Transport Layer 1298 Transport layer is typically implemented at CEs to support higher 1299 layer application protocols like routing protocols. In practice, 1300 this means that most CEs implement both the Transmission Control 1301 Protocol (TCP) and the User Datagram Protocol (UDP). 1303 Both CEs and FEs need to implement ForCES Protocol. If some layer- 1304 4 transport is used to support ForCES, then both CEs and FEs need 1305 to implement the L4 transport and ForCES Protocols. 1307 5.6. Application Layer -- Routing Protocols 1309 Interior and exterior routing protocols are implemented on CEs. 1310 The routing packets originated by CEs are forwarded to FEs for 1311 delivery. The results of such protocols (like forwarding table 1312 updates) are communicated to FEs via ForCES. 1314 For performance or scalability reasons, portions of the control 1315 plane functions that need faster response may be moved from the CEs 1316 and off-loaded onto the FEs. For example in OSPF, the Hello 1317 protocol packets are generated and processed periodically. When 1318 done at CEs, the inbound Hello packets have to traverse from the 1319 external interfaces at the FEs to the CEs via the internal CE-FE 1320 channel. Similarly, the outbound Hello packets have to go from the 1321 CEs to the FEs and to the external interfaces. Frequent Hello 1322 updates place heavy processing overhead on the CEs and can 1323 overwhelm the CE-FE channel as well. Since typically there are far 1324 more FEs than CEs in a router, the off-loaded Hello packets are 1325 processed in a much more distributed and scalable fashion. By 1326 expressing such off-loaded functions in the FE model, we can ensure 1327 interoperability. However, the exact description of the off-loaded 1328 functionality corresponding to the off-loaded functions expressed 1329 in the FE model are not part of the model itself and will need to 1330 be worked out as a separate specification. 1332 5.7. Application Layer -- Network Management Protocol 1334 RFC1812 also dictates "Routers MUST be manageable by SNMP." In 1335 general, for post-association phase, most external management tasks 1336 (including SNMP) should be done through interaction with the CE in 1337 order to support the appearance of a single functional device. 1338 Therefore, it is recommended that SNMP agent be implemented by CEs 1339 and the SNMP messages received by FEs be redirected to their CEs. 1340 AgentX framework defined in RFC2741 ([5]) may be applied here such 1341 that CEs act in the role of master agent to process SNMP protocol 1342 messages while FEs act in the role of subagent to provide access to 1343 the MIB objects residing on FEs. AgentX protocol messages between 1344 the master agent (CE) and the subagent (FE) are encapsulated and 1345 transported via ForCES, just like data packets from any other 1346 application layer protocols. 1348 6. Summary 1350 This document defines an architectural framework for ForCES. It 1351 identifies the relevant components for a ForCES network element, 1352 including (one or more) FEs, (one or more) CEs, one optional FE 1353 manager, and one optional CE manager. It also identifies the 1354 interaction among these components and discusses all the major 1355 reference points. It is important to point out that, among all the 1356 reference points, only the Fp interface between CEs and FEs is 1357 within the scope of ForCES. ForCES alone may not be enough to 1358 support all desirable NE configurations. However, we believe that 1359 ForCES over Fp interface is the most important element in realizing 1360 physical separation and interoperability of CEs and FEs, and hence 1361 the first interface that ought to be standardized. Simple and 1362 useful configurations can still be implemented with only CE-FE 1363 interface being standardized, e.g., single CE with full-meshed FEs. 1365 7. Acknowledgements 1367 Joel M. Halpern gave us many insightful comments and suggestions 1368 and pointed out several major issues. T. Sridhar suggested that 1369 the AgentX protocol could be used with SNMP to manage the ForCES 1370 network elements. Susan Hares pointed out the issue of graceful 1371 restart with ForCES. Russ Housley, Avri Doria, Jamal Hadi Salim 1372 and many others in the ForCES mailing list also provided valuable 1373 feedback. 1375 8. Security Considerations 1377 The NE administrator has the freedom to determine the exact 1378 security configuration that is needed for the specific deployment. 1379 For example, ForCES may be deployed between CEs and FEs connected 1380 to each other inside a box over a backplane. In such scenario, 1381 physical security of the box ensures that most of the attacks such 1382 as man-in-the-middle, snooping, and impersonation are not possible, 1383 and hence ForCES architecture may rely on the physical security of 1384 the box to defend against these attacks and protocol mechanisms may 1385 be turned off. However, it is also shown that denial of service 1386 attack via external interface as described below in Section 8.1.8 1387 is still a potential threat even for such "all-in-one-box" 1388 deployment scenario and hence the rate limiting mechanism is still 1389 necessary. This is just one example to show that it is important 1390 to assess the security needs of the ForCES-enabled network elements 1391 under different deployment scenarios. It should be possible for 1392 the administrator to configure the level of security needed for the 1393 ForCES Protocol. 1395 In general, the physical separation of two entities usually results 1396 in a potentially insecure link between the two entities and hence 1397 much stricter security measurements are required. For example, we 1398 pointed out in Section 4.1 that authentication becomes necessary 1399 between CE manager and FE manager, between CE and CE manager, 1400 between FE and FE manager in some configurations. The physical 1401 separation of CE and FE also imposes serious security requirement 1402 for ForCES Protocol over Fp interface. This section first attempts 1403 to describe the security threats that may be introduced by the 1404 physical separation of the FEs and the CEs, and then it provides 1405 recommendation and guidelines for secure operation and management 1406 of ForCES Protocol over Fp interface based on existing standard 1407 security solutions. 1409 8.1. Analysis of Potential Threats Introduced by ForCES 1411 This section provides the threat analysis for ForCES, with a focus 1412 on Fp interface. Each threat is described in details with the 1413 effects on the ForCES Protocol entities or/and the NE as a whole, 1414 and the required functionalities that need to be in place to defend 1415 the threat. 1417 8.1.1. "Join" or "Remove" Message Flooding on CEs 1419 Threats: A malicious node could send a stream of false "join NE" 1420 or "remove from NE" requests on behalf of non-existent or 1421 unauthorized FE to legitimate CEs at a very rapid rate and thereby 1422 create unnecessary state in the CEs. 1424 Effects: If by maintaining state for non-existent or unauthorized 1425 FEs, a CE may become unavailable for other processing and hence 1426 suffer from denial of service (DoS) attack similar to the TCP SYN 1427 DoS. If multiple CEs are used, the unnecessary state information 1428 may also be conveyed to multiple CEs via Fr interface (e.g., from 1429 the active CE to the stand-by CE) and hence subject multiple CEs to 1430 DoS attack. 1432 Requirement: A CE that receives a "join" or "remove" request 1433 should not create any state information until it has authenticated 1434 the FE endpoint. 1436 8.1.2. Impersonation Attack 1438 Threats: A malicious node can impersonate a CE or FE and send out 1439 false messages. 1441 Effects: The whole NE could be compromised. 1443 Requirement: The CE or FE must authenticate the message as having 1444 come from an FE or CE on the list of the authorized ForCES elements 1445 (provided by the CE or FE Manager in the pre-association phase) 1446 before accepting and processing it. 1448 8.1.3. Replay Attack 1450 Threat: A malicious node could replay the entire message previously 1451 sent by an FE or CE entity to get around authentication. 1453 Effect: The NE could be compromised. 1455 Requirement: Replay protection mechanism needs to be part of the 1456 security solution to defend against this attack. 1458 8.1.4. Attack during Fail Over 1460 Threat: A malicious node may exploit the CE fail-over mechanism to 1461 take over the control of NE. For example, suppose two CEs, say CE-A 1462 and CE-B, are controlling several FEs. CE-A is active and CE-B is 1463 stand-by. When CE-A fails, CE-B is taking over the active CE 1464 position. The FEs already had a trusted relationship with CE-A, 1465 but the FEs may not have the same trusted relationship established 1466 with CE-B prior to the fail-over. A malicious node can take over 1467 as CE-B if such trusted relationship has not been established prior 1468 or during the fail-over. 1470 Effect: The NE may be compromised after such insecure fail-over. 1472 Requirement: The level of trust relationship between the stand-by 1473 CE and the FEs must be as strong as the one between the active CE 1474 and the FEs. The security association between the FEs and the 1475 stand-by CE may be established prior to fail-over. If not already 1476 in place, such security association must be re-established before 1477 the stand-by CE takes over. 1479 8.1.5. Data Integrity 1480 Threats: A malicious node may inject false messages to legitimate 1481 CE or FE. 1483 Effect: An FE or CE receives the fabricated packet and performs 1484 incorrect or catastrophic operation. 1486 Requirement: Protocol messages require integrity protection. 1488 8.1.6. Data Confidentiality 1490 Threat: When FE and CE are physically separated, a malicious node 1491 may eavesdrop the messages in transit. Some of the messages are 1492 critical to the functioning of the whole network, while others may 1493 contain confidential business data. Leaking of such information 1494 may result in compromise even beyond the immediate CE or FE. 1496 Effect: Sensitive information might be exposed between CE and FE. 1498 Requirement: Data confidentiality between FE and CE must be 1499 available for sensitive information. 1501 8.1.7. Sharing security parameters 1503 Threat: Consider a scenario where several FEs communicating to the 1504 same CE share the same authentication keys for the Fp interface. 1505 If any FE or the CE is compromised, all other entities are 1506 compromised. 1508 Effect: The whole NE is compromised. 1510 Recommendation: To avoid this side effect, it's better to configure 1511 different security parameters for each FE-CE communication over Fp 1512 interface. 1514 8.1.8. Denial of Service Attack via External Interface 1516 Threat: When an FE receives a packet that is destined for its CE, 1517 the FE forwards the packet over the Fp interface. Malicious node 1518 can generate huge message storm like routing protocol packets etc. 1519 through the external Fi/f interface so that the FE has to process 1520 and forward all packets to CE through Fp interface. 1522 Effect: CE encounters resource exhaustion and bandwidth starvation 1523 on Fp interface due to an overwhelming number of packets from FEs. 1525 Requirement: Some sort of rate limiting mechanism MUST to be in 1526 place at both the FE and CE. Rate Limiter SHOULD be configured at 1527 the FE for each message type that are being received through Fi/F 1528 interface. 1530 8.2. Security Recommendations for ForCES 1532 The requirements document [3] suggested that ForCES Protocol should 1533 support reliability over Fp interface, but no particular transport 1534 protocol is yet specified for ForCES. This framework document does 1535 not intend to specify the particular transport either, and so we 1536 only provide recommendations and guidelines based on the existing 1537 standard security protocols [18] that can work with the common 1538 transport candidates suitable for ForCES. 1540 We review two existing security protocol solutions, namely IPsec 1541 (IP Security) [14] or TLS (Transport Layer Security) [13]. TLS 1542 works with reliable transports such as TCP or SCTP for unicast, 1543 while IPsec can be used with any transport (UDP, TCP, SCTP) and 1544 supports both unicast and multicast. Both TLS and IPsec can be 1545 used potentially to satisfy all of the security requirements for 1546 ForCES Protocol. In addition, other approaches may be used as well 1547 but are not documented here, including using L2 security mechanisms 1548 for a given L2 interconnect technology, as long as the requirements 1549 can be satisfied. 1551 When ForCES is deployed between CEs and FEs inside a box or a 1552 physically secured room, authentication, confidentiality and 1553 integrity may be provided by the physical security of the box and 1554 so the security mechanisms may be turned off, depending on the 1555 networking topology and its administration policy. However, it is 1556 important to realize that even if the NE is in a single-box, the 1557 DoS attacks as described in Section 8.1.8 can still be launched 1558 through Fi/f interfaces. Therefore, it is important to have the 1559 corresponding counter-measurement in place even for single-box 1560 deployment. 1562 8.2.1. Using TLS with ForCES 1564 TLS [13] can be used if a reliable unicast transport such as TCP or 1565 SCTP is used for ForCES over the Fp interface. The TLS handshake 1566 protocol is used during association establishment or re- 1567 establishment phase to negotiate a TLS session between the CE and 1568 FE. Once the session is in place, the TLS record protocol is used 1569 to secure ForCES communication messages between the CE and FE. 1571 A basic outline of how TLS can be used with ForCES is described 1572 below. Steps 1) till 7) complete the security handshake as 1573 illustrated in Figure 9 while step 8) is for all the further 1574 communication between the CE and FE, including the rest of messages 1575 after the security handshake shown in Figure 9 and the steady-state 1576 communication shown in Figure 10. 1578 1) During Pre-association phase all FEs are configured with 1579 the CEs (including both the active CE and the standby CE). 1580 2) The FE establishes a TLS connection with the CE (master) 1581 and negotiates a cipher suite. 1582 3) The FE (slave) gets the CE certificate, validates the 1583 signature, checks the expiration date, checks if the 1584 certificate has been revoked. 1585 4) The CE (master) gets the FE certificate and performs the 1586 same validation as the FE in step 3). 1587 5) If any of the check fails in step 3) or step 4), endpoint 1588 must generate an error message and abort. 1589 6) After successful mutual authentication, a TLS session is 1590 established between CE and FE. 1591 7) The FE sends a "join NE" message to the CE. 1592 8) The FE and CE use TLS session for further communication. 1594 Note that there are different ways for the CE and FE to validate a 1595 received certificate. One way is to configure the FE Manager or CE 1596 Manager or other central component as CA, so that the CE or FE can 1597 query this pre-configured CA to validate that the certificate has 1598 not been revoked. Another way is to have the CE and the FE 1599 configured directly a list of valid certificates in the pre- 1600 association phase. 1602 In the case of fail-over, it is the responsibility of the active CE 1603 and the standby CE to synchronize ForCES states including the TLS 1604 states to minimize the state reestablishment during fail-over. 1605 Care must be taken to ensure that the standby CE is also 1606 authenticated in the same way as the active CE, either before or 1607 during the fail-over. 1609 8.2.2. Using IPsec with ForCES 1611 IPsec [14] can be used with any transport protocol, such as UDP, 1612 SCTP and TCP over Fp interface for ForCES. When using IPsec, we 1613 recommend using ESP in transport mode for ForCES because message 1614 confidentiality is required for ForCES. 1616 IPsec can be used with both manual and automated SA and 1617 cryptographic key management. But IPsec's replay protection 1618 mechanisms are not available if manual key management is used. 1619 Hence, automatic key management is recommended if replay protection 1620 is deemed important. Otherwise, manual key management might be 1621 sufficient for some deployment scenarios, esp. when the number of 1622 CEs and FEs is relatively small. It is recommended that the keys 1623 be changed periodically even for manual key management. 1625 IPsec can support both unicast and multicast transport. At the time 1626 this document was published, MSEC working group is actively working 1627 on standardizing protocols to provide multicast security [17]. 1628 Multicast-based solutions relying on IPsec should specify how to 1629 meet the security requirements in [3]. 1631 Unlike TLS, IPsec provides security services between the CE and FE 1632 at IP level, and so the security handshake as illustrated in Figure 1633 9 amounts to a "no-op" when manual key management is used. The 1634 following outline the steps taken for ForCES in such a case. 1636 1) During Pre-association phase all FEs are configured with 1637 the CEs (including active CE and standby CE) and SA parameters 1638 manually. 1639 2) The FE sends a "join NE" message to the CE. This message 1640 and all others that follow are afforded security service 1641 according to the manually configured IPsec SA parameters, but 1642 replay protection is not available. 1644 It is up to the administrator to decide whether to share the same 1645 key across multiple FE-CE communication, but it is recommended that 1646 different keys be used. Similarly, it is recommended that 1647 different keys be used for inbound and outbound traffic. 1649 If automatic key management is needed, IKE [15] can be used for 1650 that purpose. Other automatic key distribution techniques such as 1651 Kerberos may be used as well. The key exchange process 1652 constitutes the security handshake as illustrated in Figure 9. The 1653 following shows the steps involved in using IKE with IPsec for 1654 ForCES. Steps 1) to 6) constitute the security handshake in Figure 1655 9. 1657 1) During Pre-association phase all FEs are configured with 1658 the CEs (including active CE and standby CE), IPsec policy 1659 etc. 1660 2) The FE kicks off IKE process and tries to establish an 1661 IPsec SA with the CE (master). The FE (Slave) gets the CE 1662 certificate as part of the IKE negotiation. The FE validates 1663 signature, checks the expiration date, checks if the 1664 certificate has been revoked. 1665 3) The CE (master) gets the FE certificate and performs the 1666 same check as the FE in step 2). 1668 4) If any of the check fails in step 2) or step 3), the 1669 endpoint must generate an error message and abort. 1670 5) After successful mutual authentication, IPsec session is 1671 established between the CE and FE. 1672 6) The FE sends a "join NE" message to CE. No SADB entry is 1673 created in FE yet. 1674 7) The FE and CE use the IPsec session for further 1675 communication. 1677 FE Manager or CE Manager or other central component can be used as 1678 CA for validating CE and FE certificates during the IKE process. 1679 Alternatively, during the pre-association phase, the CE and FE can 1680 be configured directly with the required information such as 1681 certificates or passwords etc depending upon the type of 1682 authentication that administrator wants to configure. 1684 In the case of fail-over, it is the responsibility of active CE and 1685 standby CE to synchronize ForCES states and IPsec states to 1686 minimize the state reestablishment during fail-over. 1687 Alternatively, the FE needs to establish different IPsec SA during 1688 the startup operation itself with each CE. This will minimize the 1689 periodic state transfer across IPsec layer though Fr (CE-CE) 1690 Interface. 1692 9. Normative References 1694 [1] Baker, F., "Requirements for IP Version 4 Routers", RFC 1812, 1695 June 1995. 1697 [2] Floyd, S., "Congestion Control Principles", RFC 2914, September 1698 2000. 1700 [3] Khosravi, H. and Anderson, T., "Requirements for Separation of 1701 IP Control and Forwarding", RFC 3654, November 2003. 1703 10. Informative References 1705 [4] Case, J., et al., "Introduction and Applicability Statements 1706 for Internet Standard Management Framework", RFC 3410, December 1707 2002. 1709 [5] Daniele, M. et al., "Agent Extensibility (AgentX) Protocol 1710 Version 1", RFC 2741, January 2000. 1712 [6] Chan, K. et al., "COPS Usage for Policy Provisioning (COPS- 1713 PR)", RFC 3084, March 2001. 1715 [7] Crouch, A. et al., "ForCES Applicability Statement", work in 1716 progress, June 2003, . 1718 [8] Anderson, T. and J. Buerkle, "Requirements for the Dynamic 1719 Partitioning of Switching Elements", RFC 3532, May 2003. 1721 [9] Leelanivas, M. et al., "Graceful Restart Mechanism for Label 1722 Distribution Protocol", RFC 3478, February 2003. 1724 [10] Moy, J. et al., "Graceful OSPF Restart", RFC 3623, November 1725 2003. 1727 [11] Sangli, S. et al., "Graceful Restart Mechanism for BGP", work 1728 in progress, September 2003, < draft-ietf-idr-restart-08.txt>. 1730 [12] Shand, M. and L. Ginsberg, "Restart Signaling for IS-IS", work 1731 in progress, July 2003, . 1733 [13] Dierks, T. and C. Allen, "The TLS Protocol, version 1.0", RFC 1734 2246, January 1999. 1736 [14] Kent, S. and R. Atkinson, "Security Architecture for the 1737 Internet Protocol", RFC 2401, November 1998. 1739 [15] Harkins, D. and D. Carrel, "The Internet Key Exchange (IKE) ", 1740 RFC 2409, November 1998. 1742 [16] Bellovin, S., "Guidelines for Mandating the Use of Ipsec", 1743 work in progress, October 2003, . 1745 [17] Hardjono, T. and Weis, B. "The Multicast Security 1746 Architecture", work in progress, November 2003, . 1749 [18] S. Bellovin, J. Schiller, and C. Kaufman, "Security Mechanisms 1750 for the Internet", RFC 3631, December, 2003. 1752 11. Authors' Addresses 1754 L. Lily Yang 1755 Intel Corp., MS JF3-206, 1756 2111 NE 25th Avenue 1757 Hillsboro, OR 97124, USA 1758 Phone: +1 503 264 8813 1759 Email: lily.l.yang@intel.com 1761 Ram Dantu 1762 Department of Computer Science, 1763 University of North Texas, 1764 Denton, TX 76203, USA 1765 Phone: +1 940 565 2822 1766 Email: rdantu@unt.edu 1768 Todd A. Anderson 1769 Intel Corp. 1770 2111 NE 25th Avenue 1771 Hillsboro, OR 97124, USA 1772 Phone: +1 503 712 1760 1773 Email: todd.a.anderson@intel.com 1775 Ram Gopal 1776 Nokia Research Center 1777 5, Wayside Road, 1778 Burlington, MA 01803, USA 1779 Phone: +1 781 993 3685 1780 Email: ram.gopal@nokia.com 1782 12. Intellectual Property Right 1784 The IETF takes no position regarding the validity or scope of any 1785 intellectual property or other rights that might be claimed to 1786 pertain to the implementation or use of the technology described in 1787 this document or the extent to which any license under such rights 1788 might or might not be available; neither does it represent that it 1789 has made any effort to identify any such rights. Information on 1790 the IETF's procedures with respect to rights in standards-track and 1791 standards-related documentation can be found in RFC 2026. Copies 1792 of claims of rights made available for publication and any 1793 assurances of licenses to be made available, or the result of an 1794 attempt made to obtain a general license or permission for the use 1795 of such proprietary rights by implementors or users of this 1796 specification can be obtained from the IETF Secretariat. 1798 The IETF invites any interested party to bring to its attention any 1799 copyrights, patents or patent applications, or other proprietary 1800 rights which may cover technology that may be required to practice 1801 this standard. Please address the information to the IETF 1802 Executive Director. 1804 13. Full Copyright Statement 1806 Copyright (C) The Internet Society (2003). All Rights Reserved. 1808 This document and translations of it may be copied and furnished to 1809 others, and derivative works that comment on or otherwise explain 1810 it or assist in its implementation may be prepared, copied, 1811 published and distributed, in whole or in part, without restriction 1812 of any kind, provided that the above copyright notice and this 1813 paragraph are included on all such copies and derivative works. 1814 However, this document itself may not be modified in any way, such 1815 as by removing the copyright notice or references to the Internet 1816 Society or other Internet organizations, except as needed for the 1817 purpose of developing Internet standards in which case the 1818 procedures for copyrights defined in the Internet Standards process 1819 must be followed, or as required to translate it into languages 1820 other than English. 1822 The limited permissions granted above are perpetual and will not be 1823 revoked by the Internet Society or its successors or assigns. 1824 This document and the information contained herein is provided on 1825 an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET 1826 ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR 1827 IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 1828 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 1829 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE."