idnits 2.17.1 draft-ietf-forces-framework-06.txt: -(1003): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? == There are 7 instances of lines with non-ascii characters in the document. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 2003) is 7620 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: 'RFC-2119' on line 103 == Unused Reference: '12' is defined on line 1647, but no explicit reference was found in the text == Unused Reference: '16' is defined on line 1659, but no explicit reference was found in the text -- Possible downref: Non-RFC (?) normative reference: ref. '3' == Outdated reference: A later version (-13) exists of draft-ietf-idr-restart-06 -- Obsolete informational reference (is this intentional?): RFC 2246 (ref. '13') (Obsoleted by RFC 4346) -- Obsolete informational reference (is this intentional?): RFC 2401 (ref. '14') (Obsoleted by RFC 4301) -- Obsolete informational reference (is this intentional?): RFC 2409 (ref. '15') (Obsoleted by RFC 4306) Summary: 4 errors (**), 0 flaws (~~), 6 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Draft L. Yang 2 Expiration: Dec 2003 Intel Corp. 3 File: draft-ietf-forces-framework-06.txt R. Dantu 4 Working Group: ForCES Univ. of North Texas 5 T. Anderson 6 Intel Corp. 7 R. Gopal 8 Nokia 9 June 2003 11 Forwarding and Control Element Separation (ForCES) Framework 13 draft-ietf-forces-framework-06.txt 15 Status of this Memo 17 This document is an Internet-Draft and is in full conformance with 18 all provisions of Section 10 of RFC2026. Internet-Drafts are 19 working documents of the Internet Engineering Task Force (IETF), 20 its areas, and its working groups. Note that other groups may also 21 distribute working documents as Internet-Drafts. 23 Internet-Drafts are draft documents valid for a maximum of six 24 months and may be updated, replaced, or obsoleted by other 25 documents at any time. It is inappropriate to use Internet-Drafts 26 as reference material or to cite them other than as ``work in 27 progress.'' 29 The list of current Internet-Drafts can be accessed at 30 http://www.ietf.org/ietf/1id-abstracts.txt. 32 The list of Internet-Draft Shadow Directories can be accessed at 33 http://www.ietf.org/shadow.html. 35 Copyright Notice 37 Copyright (C) The Internet Society (2003). All Rights Reserved. 39 Abstract 41 This document defines the architectural framework for the ForCES 42 (Forwarding and Control Element Separation) network elements, and 43 identifies the associated entities and the interaction among them. 45 Table of Contents 46 1. Definitions...................................................3 47 2. Introduction to Forwarding and Control Element Separation 48 (ForCES).........................................................5 49 3. Architecture..................................................9 50 3.1. Control Elements and Fr Reference Point.................10 51 3.2. Forwarding Elements and Fi reference point..............11 52 3.3. CE Managers.............................................14 53 3.4. FE Managers.............................................14 54 4. Operational Phases...........................................14 55 4.1. Pre-association Phase...................................15 56 4.1.1. Fl Reference Point.................................15 57 4.1.2. Ff Reference Point.................................16 58 4.1.3. Fc Reference Point.................................17 59 4.2. Post-association Phase and Fp reference point...........17 60 4.2.1. Proximity and Interconnect between CEs and FEs.....17 61 4.2.2. Association Establishment..........................18 62 4.2.3. Steady-state Communication.........................19 63 4.2.4. Data Packets across Fp reference point.............20 64 4.2.5. Proxy FE...........................................21 65 4.3. Association Re-establishment............................21 66 4.3.1. CE graceful restart................................21 67 4.3.2. FE restart.........................................23 68 5. Applicability to RFC1812.....................................24 69 5.1. General Router Requirements.............................24 70 5.2. Link Layer..............................................25 71 5.3. Internet Layer Protocols................................26 72 5.4. Internet Layer Forwarding...............................26 73 5.5. Transport Layer.........................................27 74 5.6. Application Layer -- Routing Protocols..................28 75 5.7. Application Layer -- Network Management Protocol........28 76 6. Summary......................................................29 77 7. Security Considerations......................................29 78 7.1. Analysis of Potential Threats Introduced by ForCES......29 79 7.1.1. "Join" or "Remove" Message Flooding on CEs.........29 80 7.1.2. Impersonation Attack...............................30 81 7.1.3. Replay Attack......................................30 82 7.1.4. Attack during Fail Over............................30 83 7.1.5. Data Integrity.....................................31 84 7.1.6. Data Confidentiality...............................31 85 7.1.7. Sharing security parameters........................31 86 7.1.8. Denial of Service Attack via External Interface....32 87 7.2. Security Recommendations for ForCES.....................32 88 7.2.1. Security Configuration.............................32 89 7.2.2. Using TLS with ForCES..............................33 90 7.2.3. Using IPsec with ForCES............................34 91 8. Normative References.........................................35 92 9. Informative References.......................................36 93 10. Acknowledgements............................................36 94 11. Authors' Addresses..........................................37 95 12. Intellectual Property Right.................................37 96 13. Full Copyright Statement....................................38 98 Conventions used in this document 100 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 101 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in 102 this document are to be interpreted as described in [RFC-2119]. 104 1. Definitions 106 A set of terminology associated with the ForCES requirements is 107 defined in [3] and we only include the definitions that are most 108 relevant to this document here. 110 Addressable Entity (AE) - An entity that is directly addressable 111 given some interconnect technology. For example, on IP networks, 112 it is a device to which we can communicate using an IP address; on 113 a switch fabric, it is a device to which we can communicate using a 114 switch fabric port number. 116 Physical Forwarding Element (PFE) - An AE that includes hardware 117 used to provide per-packet processing and handling. This hardware 118 may consist of (but is not limited to) network processors, ASICs 119 (Application-Specific Integrated Circuits), or general processors, 120 installed on line cards, daughter boards, mezzanine cards, or in 121 stand-alone boxes. 123 PFE Partition - A logical partition of a PFE consisting of some 124 subset of each of the resources (e.g., ports, memory, forwarding 125 table entries) available on the PFE. This concept is analogous to 126 that of the resources assigned to a virtual switching element as 127 described in [8]. 129 Physical Control Element (PCE) - An AE that includes hardware used 130 to provide control functionality. This hardware typically includes 131 a general-purpose processor. 133 PCE Partition - A logical partition of a PCE consisting of some 134 subset of each of the resources available on the PCE. 136 Forwarding Element (FE) - A logical entity that implements the 137 ForCES protocol. FEs use the underlying hardware to provide per- 138 packet processing and handling as directed by a CE via the ForCES 139 protocol. FEs may happen to be a single blade (or PFE), a 140 partition of a PFE or multiple PFEs. 142 Control Element (CE) - A logical entity that implements the ForCES 143 protocol and uses it to instruct one or more FEs how to process 144 packets. CEs handle functionality such as the execution of control 145 and signaling protocols. CEs may consist of PCE partitions or 146 whole PCEs. 148 ForCES Network Element (NE) - An entity composed of one or more CEs 149 and one or more FEs. To entities outside an NE, the NE represents 150 a single point of management. Similarly, an NE usually hides its 151 internal organization from external entities. 153 Pre-association Phase - The period of time during which an FE 154 Manager (see below) and a CE Manager (see below) are determining 155 which FE and CE should be part of the same network element. 157 Post-association Phase - The period of time during which an FE does 158 know which CE is to control it and vice versa, including the time 159 during which the CE and FE are establishing communication with one 160 another. 162 ForCES Protocol - While there may be multiple protocols used within 163 the overall ForCES architecture, the term "ForCES protocol" refers 164 only to the ForCES post-association phase protocol (see below). 166 ForCES Post-Association Phase Protocol - The protocol used for 167 post-association phase communication between CEs and FEs. This 168 protocol does not apply to CE-to-CE communication, FE-to-FE 169 communication, nor to communication between FE and CE managers. 170 The ForCES protocol is a master-slave protocol in which FEs are 171 slaves and CEs are masters. This protocol includes both the 172 management of the communication channel (e.g., connection 173 establishment, heartbeats) and the control messages themselves. 174 This protocol could be a single protocol or could consist of 175 multiple protocols working together. 177 FE Manager - A logical entity that operates in the pre-association 178 phase and is responsible for determining to which CE(s) an FE 179 should communicate. This process is called CE discovery and may 180 involve the FE manager learning the capabilities of available CEs. 181 A FE manager may use anything from a static configuration to a pre- 182 association phase protocol (see below) to determine which CE(s) to 183 use, however this is currently out of scope. Being a logical 184 entity, an FE manager might be physically combined with any of the 185 other logical entities mentioned in this section. 187 CE Manager - A logical entity that operates in the pre-association 188 phase and is responsible for determining to which FE(s) a CE should 189 communicate. This process is called FE discovery and may involve 190 the CE manager learning the capabilities of available FEs. A CE 191 manager may use anything from a static configuration to a pre- 192 association phase protocol (see below) to determine which FE to 193 use, however this is currently out of scope. Being a logical 194 entity, a CE manager might be physically combined with any of the 195 other logical entities mentioned in this section. 197 Pre-association Phase Protocol - A protocol between FE managers and 198 CE managers that is used to determine which CEs or FEs to use. A 199 pre-association phase protocol may include a CE and/or FE 200 capability discovery mechanism. Note that this capability 201 discovery process is wholly separate from (and does not replace) 202 that used within the ForCES protocol. However, the two capability 203 discovery mechanisms may utilize the same FE model. 205 FE Model - A model that describes the logical processing functions 206 of an FE. 208 ForCES Protocol Element - A FE or CE. 210 FE Topology -- Representation of how the multiple FEs in a single 211 NE are interconnected. Sometimes it is called inter-FE topology, 212 to be distinguished from intra-FE topology (or FE block topology) 213 used by FE model. 215 Inter-FE topology - see FE Topology. 217 2. Introduction to Forwarding and Control Element Separation (ForCES) 219 An IP network element (NE) appears to external entities as a 220 monolithic piece of network equipment, e.g., a router, NAT, 221 firewall, or load balancer. Internally, however, an IP network 222 element (NE) (such as a router) is composed of numerous logically 223 separated entities that cooperate to provide a given functionality 224 (such as routing). Two types of network element components exist: 225 control element (CE) in control plane and forwarding element (FE) 226 in forwarding plane (or data plane). Forwarding elements typically 227 are ASIC, network-processor, or general-purpose processor-based 228 devices that handle data path operations for each packet. Control 229 elements are typically based on general-purpose processors that 230 provide control functionality like routing and signaling protocols. 232 ForCES aims to define a framework and associated protocol(s) to 233 standardize information exchange between the control and forwarding 234 plane. Having standard mechanisms allows CEs and FEs to become 235 physically separated standard components. This physical separation 236 accrues several benefits to the ForCES architecture. Separate 237 components would allow component vendors to specialize in one 238 component without having to become experts in all components. 239 Standard protocol also allows the CEs and FEs from different 240 component vendors to interoperate with each other and hence it 241 becomes possible for system vendors to integrate together the CEs 242 and FEs from different component suppliers. This interoperability 243 translates into a lot more design choices and flexibility for the 244 system vendors. Overall, ForCES will enable rapid innovation in 245 both the control and forwarding planes while maintaining 246 interoperability. Scalability is also easily provided by this 247 architecture in that additional forwarding or control capacity can 248 be added to existing network elements without the need for forklift 249 upgrades. 251 ------------------------- ------------------------- 252 | Control Blade A | | Control Blade B | 253 | (CE) | | (CE) | 254 ------------------------- ------------------------- 255 ^ | ^ | 256 | | | | 257 | V | V 258 --------------------------------------------------------- 259 | Switch Fabric Backplane | 260 --------------------------------------------------------- 261 ^ | ^ | ^ | 262 | | | | . . . | | 263 | V | V | V 264 ------------ ------------ ------------ 265 |Router | |Router | |Router | 266 |Blade #1 | |Blade #2 | |Blade #N | 267 | (FE) | | (FE) | | (FE) | 268 ------------ ------------ ------------ 269 ^ | ^ | ^ | 270 | | | | . . . | | 271 | V | V | V 273 Figure 1. A router configuration example with separate blades. 275 One example of such physical separation is at the blade level. 276 Figure 1 shows such an example configuration of a router, with two 277 control blades and multiple router (forwarding) blades, all 278 interconnected into a switch fabric backplane. In such a chassis 279 configuration, the control blades are the CEs while the router 280 blades are FEs, and the switch fabric backplane provides the 281 physical interconnect for all the blades. Control blade A may be 282 the primary CE while control blade B may be the backup CE providing 283 redundancy. It is also possible to have a redundant switch fabric 284 for high availability support. Routers today with this kind of 285 configuration use proprietary interfaces for messaging between CEs 286 and FEs. The goal of ForCES is to replace such proprietary 287 interfaces with a standard protocol. With a standard protocol like 288 ForCES implemented on all blades, it becomes possible for control 289 blades from vendor X and routing blades from vendor Y to work 290 seamlessly together in one chassis. 292 ------- ------- 293 | CE1 | | CE2 | 294 ------- ------- 295 ^ ^ 296 | | 297 V V 298 ============================================ Ethernet 299 ^ ^ . . . ^ 300 | | | 301 V V V 302 ------- ------- -------- 303 | FE#1| | FE#2| | FE#n | 304 ------- ------- -------- 305 ^ | ^ | ^ | 306 | | | | | | 307 | V | V | V 309 Figure 2. A router configuration example with separate boxes. 311 Another level of physical separation between the CEs and FEs can be 312 at the box level. In such configuration, all the CEs and FEs are 313 physically separated boxes, interconnected with some kind of high 314 speed LAN connection (like Gigabit Ethernet). These separated CEs 315 and FEs are only one hop away from each other within a local area 316 network. The CEs and FEs communicate to each other by running 317 ForCES, and the collection of these CEs and FEs together become one 318 routing unit to the external world. Figure 2 shows such an example. 320 In both examples shown here, the same physical interconnect is used 321 for both CE-to-FE and FE-to-FE communication. However, that does 322 not have to be the case. One reason to use different interconnects 323 is that CE-to-FE interconnect does not have to be as fast as the 324 FE-to-FE interconnect, so the more expensive fast connections can 325 be saved for FE-to-FE. The separate interconnects may also provide 326 reliability and redundancy benefits for the NE. 328 Some examples of control functions that can be implemented in the 329 CE include routing protocols like RIP, OSPF and BGP, control and 330 signaling protocols like RSVP (Resource Reservation Protocol), LDP 331 (Label Distribution Protocol) for MPLS, etc. Examples of 332 forwarding functions in the FE include LPM (longest prefix match) 333 forwarder, classifiers, traffic shaper, meter, NAT (Network Address 334 Translators), etc. Figure 3 provides example functions in both CE 335 and FE. Any given NE may contain one or many of these CE and FE 336 functions in it. The diagram also shows that ForCES protocol is 337 used to transport both the control messages for ForCES itself and 338 the data packets that are originated/destined from/to the control 339 functions in CE (e.g., routing packets). Section 4.2.4 provides 340 more detail on this. 342 ------------------------------------------------- 343 | | | | | | | 344 |OSPF |RIP |BGP |RSVP |LDP |. . . | 345 | | | | | | | 346 ------------------------------------------------- 347 | ForCES Interface | 348 ------------------------------------------------- 349 ^ ^ 350 ForCES | |data 351 control | |packets 352 messages| |(e.g., routing packets) 353 v v 354 ------------------------------------------------- 355 | ForCES Interface | 356 ------------------------------------------------- 357 | | | | | | | 358 |LPM Fwd|Meter |Shaper |NAT |Classi-|. . . | 359 | | | | |fier | | 360 ------------------------------------------------- 361 | FE resources | 362 ------------------------------------------------- 364 Figure 3. Examples of CE and FE functions 366 A set of requirements for control and forwarding separation is 367 identified in [3]. This document describes a ForCES architecture 368 that satisfies the architectural requirements of that document and 369 defines a framework for ForCES network elements and the associated 370 entities to facilitate protocol definition. Whenever necessary, 371 this document uses many examples to illustrate the issues and/or 372 possible solutions in ForCES. These examples are intended to be 373 just examples, and should not be taken as the only or definite ways 374 of doing certain things. It is expected that separate document 375 will be produced by the ForCES working group to specify the ForCES 376 protocol(s). 378 3. Architecture 380 This section defines the ForCES architectural framework and the 381 associated logical components. This ForCES framework defines 382 components of ForCES NEs including several ancillary components. 383 These components may be connected in different kinds of topologies 384 for flexible packet processing. 386 --------------------------------------- 387 | ForCES Network Element | 388 -------------- Fc | -------------- -------------- | 389 | CE Manager |---------+-| CE 1 |------| CE 2 | | 390 -------------- | | | Fr | | | 391 | | -------------- -------------- | 392 | Fl | | | Fp / | 393 | | Fp| |----------| / | 394 | | | |/ | 395 | | | | | 396 | | | Fp /|----| | 397 | | | /--------/ | | 398 -------------- Ff | -------------- -------------- | 399 | FE Manager |---------+-| FE 1 | Fi | FE 2 | | 400 -------------- | | |------| | | 401 | -------------- -------------- | 402 | | | | | | | | | | 403 ----+--+--+--+----------+--+--+--+----- 404 | | | | | | | | 405 | | | | | | | | 406 Fi/f Fi/f 408 Figure 4. ForCES Architectural Diagram 410 The diagram in Figure 4 shows the logical components of the ForCES 411 architecture and their relationships. There are two kinds of 412 components inside a ForCES network element: control element (CE) 413 and forwarding element (FE). The framework allows multiple 414 instances of CE and FE inside one NE. Each FE contains one or 415 more physical media interfaces for receiving and transmitting 416 packets from/to the external world. The aggregation of these FE 417 interfaces becomes the NE�s external interfaces. In addition to 418 the external interfaces, there must also exist some kind of 419 interconnect within the NE so that the CE and FE can communicate 420 with each other, and one FE can forward packets to another FE. 421 The diagram also shows two entities outside of the ForCES NE: CE 422 Manager and FE Manager. These two entities provide configuration 423 to the corresponding CE or FE in the pre-association phase (see 424 Section 5.1). There is no defined role for FE Manager and CE 425 Manager in post-association phase, thus these logical components 426 are not considered part of the ForCES NE. 428 For convenience, the logical interactions between these components 429 are labeled by reference points Fp, Fc, Ff, Fr, Fl, and Fi, as 430 shown in Figure 4. The FE external interfaces are labeled as Fi/f. 431 More detail is provided in Section 4 and 5 for each of these 432 reference points. All these reference points are important in 433 understanding the ForCES architecture, however, the ForCES protocol 434 is only defined over one reference point -- Fp. 436 The interface between two ForCES NEs is identical to the interface 437 between two conventional routers and these two NEs exchange the 438 protocol packets through the external interfaces at Fi/f. ForCES 439 NEs connect to existing routers transparently. 441 3.1. Control Elements and Fr Reference Point 443 It is not necessary to define any protocols across the Fr reference 444 point to enable control and forwarding separation for simple 445 configurations like single CE and multiple FEs. However, this 446 architecture permits multiple CEs to be present in a network 447 element. In cases where an implementation uses multiple CEs, the 448 invariant that the CEs and FEs together appear as a single NE must 449 be maintained. 451 Multiple CEs may be used for redundancy, load sharing, distributed 452 control, or other purposes. Redundancy is the case where one or 453 more CEs are prepared to take over should an active CE fail. Load 454 sharing is the case where two or more CEs are concurrently active 455 and any request that can be serviced by one of the CEs can also be 456 serviced by any of the other CEs. For both redundancy and load 457 sharing, the CEs involved are equivalently capable. The only 458 difference between these two cases is in terms of how many active 459 CEs there are. Distributed control is the case where two or more 460 CEs are concurrently active but certain requests can only be 461 serviced by certain CEs. 463 When multiple CEs are employed in a ForCES NE, their internal 464 organization is considered an implementation issue that is beyond 465 the scope of ForCES. CEs are wholly responsible for coordinating 466 amongst themselves via the Fr reference point to provide 467 consistency and synchronization. However, ForCES does not define 468 the implementation or protocols used between CEs, nor does it 469 define how to distribute functionality among CEs. Nevertheless, 470 ForCES will support mechanisms for CE redundancy or fail over, and 471 it is expected that vendors will provide redundancy or fail over 472 solutions within this framework. 474 3.2. Forwarding Elements and Fi reference point 476 An FE is a logical entity that implements the ForCES protocol and 477 uses the underlying hardware to provide per-packet processing and 478 handling as directed by a CE. It is possible to partition one 479 physical FE into multiple logical FEs. It is also possible for one 480 FE to use multiple physical FEs. The mapping between physical 481 FE(s) and the logical FE(s) is beyond the scope of ForCES. For 482 example, a logical partition of a physical FE can be created by 483 assigning some portion of each of the resources (e.g., ports, 484 memory, forwarding table entries) available on the physical FE to 485 each of the logical FEs. Such concept of FE virtualization is 486 analogous to a virtual switching element as described in [8]. FE 487 virtualization should occur only in the pre-association phase and 488 hence has no impact on ForCES. 490 FEs perform all packet processing functions as directed by CEs. 491 FEs have no initiative of their own. Instead, FEs are slaves and 492 only do as they are told. FEs may communicate with one or more CEs 493 concurrently across reference point Fp. FEs have no notion of CE 494 redundancy, load sharing, or distributed control. Instead, FEs 495 accept commands from any CE authorized to control them, and it is 496 up to the CEs to coordinate among themselves to achieve redundancy, 497 load sharing or distributed control. The idea is to keep FEs as 498 simple and dumb as possible so that FEs can focus their resource on 499 the packet processing functions. 501 For example, in Figure 5, FE1 and FE2 can be configured to accept 502 commands from both the primary CE (CE1) and the backup CE (CE2). 503 Upon detection of CE1 failure, perhaps across the Fr or Fp 504 reference point, CE2 is configured to take over activities of CE1. 505 This is beyond the scope of ForCES and is not discussed further. 507 Distributed control can be achieved in the similar fashion, without 508 much intelligence on the part of FEs. For example, FEs can be 509 configured to detect RSVP and BGP protocol packets, and forward 510 RSVP packets to one CE and BGP packets to another CE. Hence, FEs 511 may need to do packet filtering for forwarding packets to specific 512 CEs. 514 ------- Fr ------- 515 | CE1 | ------| CE2 | 516 ------- ------- 517 | \ / | 518 | \ / | 519 | \ / | 520 | \/Fp | 521 | /\ | 522 | / \ | 523 | / \ | 524 ------- Fi ------- 525 | FE1 |<----->| FE2 | 526 ------- ------- 528 Figure 5. CE redundancy example. 530 This architecture permits multiple FEs to be present in an NE. [3] 531 dictates that the ForCES protocol must be able to scale to at least 532 hundreds of FEs (see [3] Section 5, requirement #11). Each of 533 these FEs may potentially have a different set of packet processing 534 functions, with different media interfaces. FEs are responsible 535 for basic maintenance of layer-2 connectivity with other FEs and 536 with external entities. Many layer-2 media include sophisticated 537 control protocols. The FORCES protocol (over the Fp reference 538 point) will be able to carry messages for such protocols so that, 539 in keeping with the dumb FE model, the CE can provide appropriate 540 intelligence and control over these media. 542 When multiple FEs are present, ForCES requires that packets must be 543 able to arrive at the NE by one FE and leave the NE via a different 544 FE (See [3], Section 5, Requirement #3). Packets that enter the NE 545 via one FE and leave the NE via a different FE are transferred 546 between FEs across the Fi reference point. Fi reference point 547 could be used by FEs to discovery their (inter-FE) topology, 548 perhaps during pre-association phase. The Fi reference point is a 549 separate protocol from the Fp reference point and is not currently 550 defined by the ForCES architecture. 552 FEs could be connected in different kinds of topologies and packet 553 processing may spread across several FEs in the topology. Hence, 554 logical packet flow may be different from physical FE topology. 555 Figure 6 provides some topology examples. When it is necessary to 556 forward packets between FEs, the CE needs to understand the FE 557 topology. The FE topology can be queried from the FEs by CEs. 559 ----------------- 560 | CE | 561 ----------------- 562 ^ ^ ^ 563 / | \ 564 / v \ 565 / ------- \ 566 / +->| FE3 |<-+ \ 567 / | | | | \ 568 v | ------- | v 569 ------- | | ------- 570 | FE1 |<-+ +->| FE2 | 571 | |<--------------->| | 572 ------- ------- 573 ^ | ^ | 574 | | | | 575 | v | v 577 (a) Full mesh among FE1, FE2 and FE3. 579 ----------- 580 | CE | 581 ----------- 582 ^ ^ ^ ^ 583 / | | \ 584 /------ | | ------\ 585 v v v v 586 ------- ------- ------- ------- 587 | FE1 |<->| FE2 |<->| FE3 |<->| FE4 | 588 ------- ------- ------- ------- 589 ^ | ^ | ^ | ^ | 590 | | | | | | | | 591 | v | v | v | v 593 (b) Multiple FEs in a daisy chain 595 ^ | 596 | v 597 ----------- 598 | FE1 |<-----------------------| 599 ----------- | 600 ^ ^ | 601 / \ | 602 | ^ / \ ^ | V 603 v | v v | v ---------- 604 --------- --------- | | 605 | FE2 | | FE3 |<------------>| CE | 606 --------- --------- | | 607 ^ ^ ^ ---------- 608 | \ / ^ ^ 609 | \ / | | 610 | v v | | 611 | ----------- | | 612 | | FE4 |<----------------------| | 613 | ----------- | 614 | | ^ | 615 | v | | 616 | | 617 |----------------------------------------| 619 (c) Multiple FEs connected by a ring 621 Figure 6. Some examples of FE topology. 623 3.3.CE Managers 625 CE managers are responsible for determining which FEs a CE should 626 control. It is legitimate for CE managers to be hard-coded with 627 the knowledge of with which FEs its CEs should communicate with. A 628 CE manager may also be physically embedded into a CE and be 629 implemented as a simple keypad or other direct configuration 630 mechanism on the CE. Finally, CE managers may be physically and 631 logically separate entities that configure the CE with FE 632 information via such mechanisms as COPS-PR [6] or SNMP [4]. 634 3.4. FE Managers 636 FE managers are responsible for determining with which CE any 637 particular FE should initially communicate. Like CE managers, no 638 restrictions are placed on how an FE manager decides with which CE 639 its FEs should communicate, nor are restrictions placed on how FE 640 managers are implemented. Each FE should have one and only one FE 641 manager, while different FEs may have the same or different FE 642 manager(s). Each manager can choose to exist and operate 643 independently of other manager. 645 4. Operational Phases 647 Both FEs and CEs require some configuration in place before they 648 can start information exchange and function as a coherent network 649 element. Two operational phases are identified in this framework: 650 pre-association and post-association. 652 4.1.Pre-association Phase 654 Pre-association phase is the period of time during which an FE 655 Manager and a CE Manager are determining which FE and CE should be 656 part of the same network element. The protocols used during this 657 phase may include all or some of the message exchange over Fl, Ff 658 and Fc reference points. However, all these may be optional and 659 none of this is within the scope of ForCES protocol. 661 4.1.1. Fl Reference Point 663 CE managers and FE managers may communicate across the Fl reference 664 point in the pre-association phase in order to determine which CEs 665 and FEs should communicate with each other. Communication across 666 the Fl reference point is optional in this architecture. No 667 requirements are placed on this reference point. 669 CE managers and FE managers may be operated by different entities. 670 The operator of the CE manager may not want to divulge, except to 671 specified FE managers, any characteristics of the CEs it manages. 672 Similarly, the operator of the FE manager may not want to divulge 673 FE characteristics, except to authorized entities. As such, CE 674 managers and FE managers may need to authenticate one another. 675 Subsequent communication between CE managers and FE managers may 676 require other security functions such as privacy, non-repudiation, 677 freshness, and integrity. 679 FE Manager FE CE Manager CE 680 | | | | 681 | | | | 682 |(security exchange) | | 683 1|<------------------------------>| | 684 | | | | 685 |(a list of CEs and their attributes) | 686 2|<-------------------------------| | 687 | | | | 688 |(a list of FEs and their attributes) | 689 3|------------------------------->| | 690 | | | | 691 | | | | 692 |<----------------Fl------------>| | 694 Figure 7. An example of message exchange over Fl reference point 695 Once the necessary security functions have been performed, the CE 696 and FE managers communicate to determine which CEs and FEs should 697 communicate with each other. At the very minimum, the CE and FE 698 managers need to learn of the existence of available FEs and CEs 699 respectively. This discovery process may entail one or both 700 managers learning the capabilities of the discovered ForCES 701 protocol elements. Figure 7 shows an example of possible message 702 exchange between CE manager and FE manager over Fl reference point. 704 4.1.2. Ff Reference Point 706 The Ff reference point is used to inform forwarding elements of the 707 association decisions made by the FE manager in pre-association 708 phase. Only authorized entities may instruct an FE with respect to 709 which CE should control it. Therefore, privacy, integrity, 710 freshness, and authentication are necessary between the FE manager 711 and FEs when the FE manager is remote to the FE. Once the 712 appropriate security has been established, the FE manager instructs 713 the FEs across this reference point to join a new NE or to 714 disconnect from an existing NE. The FE Manager could also assign 715 unique FE identifiers to the FEs using this reference point. The 716 FE identifiers are useful in post association phase to express FE 717 topology. Figure 8 shows example of message exchange over Ff 718 reference point. 720 FE Manager FE CE Manager CE 721 | | | | 722 | | | | 723 |(security exchange) |(security exchange) 724 1|<------------>|authentication 1|<----------- 725 >|authentication | | | 726 | 727 |(FE ID, attributes) |(CE ID, attributes) 728 2|<-------------|request 2|<------------|request 729 | | | | 730 3|------------->|response 3|------------>|response 731 |(corresponding CE ID) |(corresponding FE ID) 732 | | | | 733 | | | | 734 |<-----Ff----->| |<-----Fc---->| 736 Figure 8. Examples of message exchange 737 over Ff and Fc reference points. 739 Note that the FE manager function may be co-located with the FE 740 (such as by manual keypad entry of the CE IP address), in which 741 case this reference point is reduced to a built-in function. 743 4.1.3. Fc Reference Point 745 The Fc reference point is used to inform control elements of the 746 association decisions made by CE managers in pre-association phase. 747 When the CE manager is remote, only authorized entities may 748 instruct a CE to control certain FEs. Privacy, integrity, 749 freshness and authentication are also required across this 750 reference point in such a configuration. Once appropriate security 751 has been established, the CE manager instructs CEs as to which FEs 752 they should control and how they should control them. Figure 8 753 shows example of message exchange over Fc reference point. 755 As with the FE manager and FEs, configurations are possible where 756 the CE manager and CE are co-located and no protocol is used for 757 this function. 759 4.2. Post-association Phase and Fp reference point 761 Post-association phase is the period of time during which an FE and 762 CE have been configured with information necessary to contact each 763 other and includes both association establishment and steady-state 764 communication. The communication between CE and FE is performed 765 across the Fp ("p" meaning protocol) reference point. ForCES 766 protocol is exclusively used for all communication across the Fp 767 reference point. 769 4.2.1. Proximity and Interconnect between CEs and FEs 771 The ForCES Working Group has made a conscious decision that the 772 first version of ForCES will not be designed to support 773 configurations where the CE and FE are located arbitrarily in the 774 network. In particular, ForCES is intended for "very close" CE/FE 775 localities in IP networks, as defined by ForCES Applicability 776 Statement ([7]). Very Close localities consist of control and 777 forwarding elements that either are components in the same physical 778 box, or are separated at most by one local network hop. 780 CEs and FEs can be connected by a variety of interconnect 781 technologies, including Ethernet connections, backplanes, ATM 782 (cell) fabrics, etc. ForCES should be able to support each of 783 these interconnects (see [3] Section 5, requirement #1). ForCES 784 will make use of an existing RFC2914 ([2]) compliant L4 protocol 785 with adequate reliability, security and congestion control (e.g. 786 TCP, SCTP) for transport purposes. 788 4.2.2. Association Establishment 790 FE CE 791 | | 792 |(Security exchange.) | 793 1|<--------------------->| 794 | | 795 |(Let me join the NE please.) 796 2|---------------------->| 797 | | 798 |(What kind of FE are you? -- capability query) 799 3|<----------------------| 800 | | 801 |(Here is my FE functions/state: use model to 802 describe) 803 4|---------------------->| 804 | | 805 |(How are you connected with other FEs?) 806 5|<----------------------| 807 | | 808 |(Here is the FE topology info) 809 6|---------------------->| 810 | | 811 |(Initial config for FE -- optional) 812 7|<----------------------| 813 | | 814 |(I am ready to go. Shall I?) 815 8|---------------------->| 816 | | 817 |(Go ahead!) | 818 9|<----------------------| 819 | | 821 Figure 9. Example of message exchange between CE and FE 822 over Fp to establish NE association 824 As an example, figure 9 shows some of the message exchange that may 825 happen before the association between the CE and FE is fully 826 established. Either the CE or FE can initiate the connection. 827 Security handshake is necessary to authenticate the two 828 communication endpoints to each other before any further message 829 exchange can happen. The exact details of the security handshake 830 depend on the security solution chosen by ForCES protocol. It is 831 most likely that either IPSec or TLS will be used. Section 9 832 provides more details on the security considerations for ForCES. 833 After the successful security handshake, the FE needs to inform the 834 CE of its own capability and its topology in relation to other FEs. 835 The capability of the FE is represented by the FE model, described 836 in a separate document. The model would allow an FE to describe 837 what kind of packet processing functions it contains, in what order 838 the processing happens, what kinds of configurable parameters it 839 allows, what statistics it collects and what events it might throw, 840 etc. Once such information is available to the CE, the CE may 841 choose to send some initial or default configuration to the FE so 842 that the FE can start receiving and processing packets correctly. 843 Such initialization may not be necessary if the FE already obtains 844 the information from its own bootstrap process. Once FE starts 845 accepting packets for processing, we say the association of this FE 846 with its CE is now established. From then on, the CE and FE enter 847 steady-state communication. 849 4.2.3. Steady-state Communication 851 Once an association is established between the CE and FE, the 852 ForCES protocol is used by the CE and FE over Fp reference point to 853 exchange information to facilitate packet processing. 855 FE CE 856 | | 857 |(Add these new routes.)| 858 1|<----------------------| 859 | | 860 |(Successful.) | 861 2|---------------------->| 862 | | 863 | | 864 |(Query some stats.) | 865 1|<----------------------| 866 | | 867 |(Reply with stats collected.) 868 2|---------------------->| 869 | | 870 | | 871 |(My port is down, with port #.) 872 1|---------------------->| 873 | | 874 |(Here is a new forwarding table) 875 2|<----------------------| 876 | | 877 Figure 10. Examples of message exchange between CE and FE 878 over Fp during steady-state communication 880 Based on the information acquired through CEs' control processing, 881 CEs will frequently need to manipulate the packet-forwarding 882 behaviors of their FE(s) by sending instructions to FEs. For 883 example, Figure 10 shows message exchange examples in which the CE 884 sends new routes to the FE so that the FE can add them to its 885 forwarding table. The CE may query the FE for statistics collected 886 by the FE and the FE may notify the CE of important events such as 887 port failure. 889 4.2.4. Data Packets across Fp reference point 891 --------------------- ---------------------- 892 | | | | 893 | +--------+ | | +--------+ | 894 | |CE(BGP) | | | |CE(BGP) | | 895 | +--------+ | | +--------+ | 896 | | | | ^ | 897 | |Fp | | |Fp | 898 | v | | | | 899 | +--------+ | | +--------+ | 900 | | FE | | | | FE | | 901 | +--------+ | | +--------+ | 902 | | | | ^ | 903 | Router | | | Router | | 904 | A | | | B | | 905 ---------+----------- -----------+---------- 906 v ^ 907 | | 908 | | 909 ------------------->--------------- 911 Figure 11. Example to show data packet flow between two NEs. 913 Control plane protocol packets (such as RIP, OSPF messages) 914 addressed to any of NE's interfaces are typically redirected by the 915 receiving FE to its CE, and CE may originate packets and have its 916 FE deliver them to other NEs. Therefore, ForCES protocol over Fp 917 not only transports the ForCES protocol messages between CEs and 918 FEs, but also encapsulates the data packets from control plane 919 protocols. Moreover, one FE may be controlled by multiple CEs for 920 distributed control. In this configuration, the control protocols 921 supported by the FORCES NEs may spread across multiple CEs. For 922 example, one CE may support routing protocols like OSPF and BGP, 923 while a signaling and admission control protocol like RSVP is 924 supported in another CE. FEs are configured to recognize and 925 filter these protocol packets and forward them to the corresponding 926 CE. 928 Figure 11 shows one example of how the BGP packets originated by 929 router A are passed to router B. In this example, the ForCES 930 protocol is used to transport the packets from the CE to the FE 931 inside router A, and then from the FE to the CE inside router B. 932 In light of the fact that the ForCES protocol is responsible for 933 transporting both the control messages and the data packets between 934 the CE and FE over Fp reference point, it is possible to use either 935 a single protocol or multiple protocols to achieve this. 937 4.2.5. Proxy FE 939 In the case where a physical FE cannot implement (e.g., due to the 940 lack of a general purpose CPU) the ForCES protocol directly, a 941 proxy FE can be used in the middle of Fp reference point. This 942 allows the CE communicate to the physical FE via the proxy by using 943 ForCES, while the proxy manipulates the physical FE using some 944 intermediary form of communication (e.g., a non-ForCES protocol or 945 DMA). In such an implementation, the combination of the proxy and 946 the physical FE becomes one logical FE entity. 948 4.3. Association Re-establishment 950 FEs and CEs may join and leave NEs dynamically (see [3] Section 5, 951 requirements #12). When an FE or CE leaves the NE, the association 952 with the NE is broken. If the leaving party rejoins an NE later, 953 to re-establish the association, it may need to re-enter the pre- 954 association phase. Loss of association can also happen 955 unexpectedly due to loss of connection between the CE and the FE. 956 Therefore, the framework allows the bi-directional transition 957 between these two phases, but the ForCES protocol is only 958 applicable for the post-association phase. However, the protocol 959 should provide mechanisms to support association re-establishment. 960 This includes the ability for CEs and FEs to determine when there 961 is a loss of association between them, ability to restore 962 association and efficient state (re)synchronization mechanisms (see 963 [3] Section 5, requirement #7). Note that security association and 964 state must be also re-established to guarantee the same level of 965 security exists before and after the association re-establishment. 967 4.3.1. CE graceful restart 969 The failure and restart of the CE in a router can potentially cause 970 much stress and disruption on the control plane throughout a 971 network. Because when a CE has to restart for any reason, the 972 router loses routing adjacencies or sessions with its routing 973 neighbors. Neighbors who detect the lost adjacency normally re- 974 compute new routes and then send routing updates to their own 975 neighbors to communicate the lost adjacency. Their neighbors do 976 the same thing to propagate throughout the network. In the 977 meantime, the restarting router cannot receive traffic from other 978 routers because the neighbors have stopped using the router�s 979 previously advertised routes. When the restarting router restores 980 adjacencies, neighbors must once again re-compute new routes and 981 send out additional routing updates. The restarting router is 982 unable to forward packets until it has re-established routing 983 adjacencies with neighbors, received route updates through these 984 adjacencies, and computed new routes. Until convergence takes 985 place throughout the network, packets may be lost in transient 986 black holes or forwarding loops. 988 A high availability mechanism known as the "graceful restart" has 989 been used by the IP routing protocols (OSPF [10], BGP [11], BGP 990 [11]) and MPLS label distribution protocol (LDP [9]) to help 991 minimize the negative effects on routing throughout an entire 992 network caused by a restarting router. Route flap on neighboring 993 routers is avoided, and a restarting router can continue to forward 994 packets that would otherwise be dropped. 996 While the details differ from protocol to protocol, the general 997 idea behind the graceful restart mechanism remains the same. With 998 the graceful restart, a restarting router can inform its neighbors 999 when it restarts. The neighbors may detect the lost adjacency but 1000 do not recompute new routes or send routing updates to their 1001 neighbors. The neighbors also hold on to the routes received from 1002 the restarting router before restart and assume they are still 1003 valid for a limited time. By doing so, the restarting router�s FEs 1004 can also continue to receive and forward traffic from other 1005 neighbors for a limited time by using the routes they already have. 1006 The restarting router then re-establishes routing adjacencies, 1007 downloads updated routes from all its neighbors, recomputes new 1008 routes and uses them to replace the older routes it was using. It 1009 then sends these updated routes to its neighbors and signals the 1010 completion of the graceful restart process. 1012 Non-stop forwarding is a requirement for graceful restart. It is 1013 necessary so a router can continue to forward packets while it is 1014 downloading routing information and recomputing new routes. This 1015 ensures that packets will not be dropped. As one can see, one of 1016 the benefits afforded by the separation of CE and FE is exactly the 1017 ability of non-stop forwarding in the face of the CE failure and 1018 restart. The support of dynamic changes to CE/FE association in 1019 ForCES also makes it compatible with high availability mechanisms 1020 such as graceful restart. 1022 ForCES should be able to support CE graceful restart easily. When 1023 the association is established the first time, the CE must inform 1024 the FEs what to do in the case of CE failure. If graceful restart 1025 is not supported, the FEs may be told to stop packet processing all 1026 together if its CE fails. If graceful restart is supported, the 1027 FEs should be told to cache and hold on to its FE state including 1028 the forwarding tables across the restarts. A timer must be 1029 included so that the timeout causes such cached state to expire 1030 eventually. Those timers should be settable by the CE. 1032 4.3.2. FE restart 1034 In the same example in Figure 5, assuming CE1 is the working CE for 1035 the moment, what would happen if one of the FEs, say FE1, leaves 1036 the NE temporarily? FE1 may voluntarily decide to leave the 1037 association. Alternatively, FE1 may stop functioning simply due to 1038 unexpected failure. In former case, CE1 receives a "leave- 1039 association request" from FE1. In the latter, CE1 detects the 1040 failure of FE1 by some other means. In both cases, CE1 must inform 1041 the routing protocols of such an event, most likely prompting a 1042 reachability and SPF (Shortest Path First) recalculation and 1043 associated downloading of new FIBs from CE1 to the other remaining 1044 FEs (only FE2 in this example). Such recalculation and FIB update 1045 will also be propagated from the CE1 to its neighbors that are 1046 affected by the connectivity of FE1. 1048 When FE1 decides to rejoin again, or when it restarts again from 1049 the failure, FE1 needs to re-discover its master (CE). This can be 1050 achieved by several means. It may re-enter the pre-association 1051 phase and get that information from its FE manager. It may 1052 retrieve the previous CE information from its cache, if it can 1053 validate the information freshness. Once it discovers its CE, it 1054 starts message exchange with CE to re-establish the association 1055 just as outlined in Figure 9, with the possible exception that it 1056 might be able to bypass the transport of the complete initial 1057 configuration. Suppose that FE1 still has its routing table and 1058 other state information from the last association, instead of 1059 sending all the information again from scratch, it may be able to 1060 use more efficient mechanism to re-sync up the state with its CE if 1061 such mechanism is supported by the ForCES protocol. For example, 1062 CRC-32 of the state might give a quick indication of whether or not 1063 the state is in-sync with its CE. By comparing its state with CE 1064 first, it sends information update only if it is needed. ForCES 1065 protocol may choose to implement similar optimization mechanisms, 1066 but it may also choose not to, as this is not a requirement. 1068 5. Applicability to RFC1812 1070 [3] Section 5, requirement #9 dictates "Any proposed ForCES 1071 architecture must explain how that architecture supports all of the 1072 router functions as defined in RFC1812." RFC1812 discusses many 1073 important requirements for IPv4 routers from the link layer to the 1074 application layer. This section addresses the relevant 1075 requirements in RFC1812 for implementing IPv4 routers based on 1076 ForCES architecture and explains how ForCES satisfies these 1077 requirements by providing guidelines on how to separate the 1078 functionalities required into forwarding plane and control plane. 1080 In general, the forwarding plane carries out the bulk of the per- 1081 packet processing that is required at line speed, while the control 1082 plane carries most of the computationally complex operations that 1083 are typical of the control and signaling protocols. However, it is 1084 impossible to draw a rigid line to divide the processing into CEs 1085 and FEs cleanly. Nor should the ForCES architecture limit the 1086 innovative approaches in control and forwarding plane separation. 1087 As more and more processing power is available in the FEs, some of 1088 the control functions that traditionally are performed by CEs may 1089 now be moved to FEs for better performance and scalability. Such 1090 offloaded functions may include part of ICMP or TCP processing, or 1091 part of routing protocols. Once off-loaded onto the forwarding 1092 plane, such CE functions, even though logically belonging to the 1093 control plane, now become part of the FE functions. Just like the 1094 other logical functions performed by FEs, such off-loaded functions 1095 must be expressed as part of the FE model so that the CEs can 1096 decide how to best take advantage of these off-loaded functions 1097 when present on the FEs. 1099 5.1. General Router Requirements 1101 Routers have at least two or more logical interfaces. When CEs and 1102 FEs are separated by ForCES within a single NE, some additional 1103 interfaces are needed for intra-NE communications. Figure 12 shows 1104 an example to illustrate that. This NE contains one CE and two 1105 FEs. Each FE has four interfaces; two of them are used for 1106 receiving and transmitting packets to the external world, while the 1107 other two are for intra-NE connections. CE has two logical 1108 interfaces #9 and #10, connected to interfaces #3 and #6 from FE1 1109 and FE2, respectively. Interface #4 and #5 are connected for FE1- 1110 FE2 communication. Therefore, this router NE provides four 1111 external interfaces (#1, 2, 7 and 8). 1113 --------------------------------- 1114 | router NE | 1115 | ----------- ----------- | 1116 | | FE1 | | FE2 | | 1117 | ----------- ----------- | 1118 | 1| 2| 3| 4| 5| 6| 7| 8| | 1119 | | | | | | | | | | 1120 | | | | +----+ | | | | 1121 | | | | | | | | 1122 | | | 9| 10| | | | 1123 | | | -------------- | | | 1124 | | | | CE | | | | 1125 | | | -------------- | | | 1126 | | | | | | 1127 -----+--+----------------+--+---- 1128 | | | | 1129 | | | | 1131 Figure 12. A router NE example with four interfaces. 1133 IPv4 routers must implement IP to support its packet forwarding 1134 function, which is driven by its FIB (Forwarding Information Base). 1135 This Internet layer forwarding (see RFC1812 [1] Section 5) 1136 functionality naturally belongs to FEs in the ForCES architecture. 1138 A router may implement transport layer protocols (like TCP and UDP) 1139 that are required to support application layer protocols (see 1140 RFC1812 [1] Section 6). One important class of application 1141 protocols is routing protocols (see RFC1812 [1] Section 7). In 1142 ForCES architecture, routing protocols are naturally implemented by 1143 CEs. Routing protocols require routers communicate with each 1144 other. This communication between CEs in different routers is 1145 supported in ForCES by FEs� ability to redirect data packets 1146 addressed to routers (i.e., NEs) and CEs� ability to originate 1147 packets and have them delivered by their FEs. This communication 1148 occurs across Fp reference point inside each router and between 1149 neighboring routers' external interfaces, as illustrated in Figure 1150 11. 1152 5.2.Link Layer 1154 Since FEs own all the external interfaces for the router, FEs need 1155 to conform to the link layer requirements in RFC1812. Arguably, 1156 ARP support may be implemented in either CEs or FEs. As we will 1157 see later, a number of behaviors that RFC1812 mandates fall into 1158 this category -- they may be performed by the FE and may be 1159 performed by the CE. A general guideline is needed to ensure 1160 interoperability between separated control and forwarding planes. 1161 The guideline we offer here is that CEs MUST be capable of these 1162 kind of operations while FEs MAY choose to implement them. FE 1163 model should indicate its capabilities in this regard so that CEs 1164 can decide where these functions are implemented. 1166 Interface parameters, including MTU, IP address, etc., must be 1167 configurable by CEs via ForCES. CEs must be able to determine 1168 whether a physical interface in an FE is available to send packets 1169 or not. FEs must also inform CEs the status change of the 1170 interfaces (like link up/down) via ForCES. 1172 5.3.Internet Layer Protocols 1174 Both FEs and CEs must implement IP protocol and all mandatory 1175 extensions as RFC1812 specified. CEs should implement IP options 1176 like source route and record route while FEs may choose to 1177 implement those as well. The timestamp option should be 1178 implemented by FEs to insert the timestamp most accurately. The FE 1179 must interpret the IP options that it understands and preserve the 1180 rest unchanged for use by CEs. Both FEs and CEs might choose to 1181 silently discard packets without sending ICMP errors, but such 1182 events should be logged and counted. FEs may report statistics for 1183 such events to CEs via ForCES. 1185 When multiple FEs are involved to process packets, the appearance 1186 of single NE must be strictly maintained. For example, Time-To- 1187 Live (TTL) must be decremented only once within a single NE. For 1188 example, it can be always decremented by the last FE with egress 1189 function. 1191 FEs must receive and process normally any packets with a broadcast 1192 destination address or a multicast destination address that the 1193 router has asked to receive. When IP multicast is supported in 1194 routers, IGMP is implemented in CEs. CEs are also required of ICMP 1195 support, while it is optional for FEs to support ICMP. Such an 1196 option can be communicated to CEs as part of the FE model. 1197 Therefore, FEs can always rely upon CEs to send out ICMP error 1198 messages, but FEs also have the option to generate ICMP error 1199 messages themselves. 1201 5.4.Internet Layer Forwarding 1202 IP forwarding is implemented by FEs. When the routing table is 1203 updated at CEs, ForCES is used to send the new route entries from 1204 CEs to FEs. Each FE has its own forwarding table and uses this 1205 table to direct packets to the next hop interface. 1207 Upon receiving IP packets, the FE verifies the IP header and 1208 processes most of the IP options. Some options cannot be processed 1209 until the routing decision has been made. The routing decision is 1210 made after examining the destination IP address. If the 1211 destination address belongs to the router itself, the packets are 1212 filtered and either processed locally or forwarded to CE, depending 1213 upon the instructions set-up by CE. Otherwise, the FE determines 1214 the next hop IP address by looking up in its forwarding table. The 1215 FE also determines the network interface it uses to send the 1216 packets. Sometimes an FE may need to forward the packets to 1217 another FE before packets can be forwarded out to the next hop. 1218 Right before packets are forwarded out to the next hop, the FE 1219 decrements TTL by 1 and processes any IP options that cannot be 1220 processed before. The FE performs any IP fragmentation if 1221 necessary, determines link layer address (e.g., by ARP), and 1222 encapsulates the IP datagram (or each of the fragments thereof) in 1223 an appropriate link layer frame and queues it for output on the 1224 interface selected. 1226 Other options mentioned in RFC1812 for IP forwarding may also be 1227 implemented at FEs, for example, packet filtering. 1229 FEs typically forward packets destined locally to CEs. FEs may 1230 also forward exceptional packets (packets that FEs do not know how 1231 to handle) to CEs. CEs are required to handle packets forwarded by 1232 FEs for whatever different reasons. It might be necessary for 1233 ForCES to attach some meta-data with the packets to indicate the 1234 reasons of forwarding from FEs to CEs. Upon receiving packets with 1235 meta-data from FEs, CEs can decide to either process the packets 1236 themselves, or pass the packets to the upper layer protocols 1237 including routing and management protocols. If CEs are to process 1238 the packets by themselves, CEs may choose to discard the packets, 1239 or modify and re-send the packets. CEs may also originate new 1240 packets and deliver them to FEs for further forwarding. 1242 Any state change during router operation must also be handled 1243 correctly according to RFC1812. For example, when an FE ceases 1244 forwarding, the entire NE may continue forwarding packets, but it 1245 needs to stop advertising routes that are affected by the failed 1246 FE. 1248 5.5. Transport Layer 1249 Transport layer is typically implemented at CEs to support higher 1250 layer application protocols like routing protocols. In practice, 1251 this means that most CEs implement both the Transmission Control 1252 Protocol (TCP) and the User Datagram Protocol (UDP). 1254 Both CEs and FEs need to implement ForCES protocol. If some layer- 1255 4 transport is used to support ForCES, then both CEs and FEs need 1256 to implement the L4 transport and ForCES protocols. 1258 5.6. Application Layer -- Routing Protocols 1260 Interior and exterior routing protocols are implemented on CEs. 1261 The routing packets originated by CEs are forwarded to FEs for 1262 delivery. The results of such protocols (like forwarding table 1263 updates) are communicated to FEs via ForCES. 1265 For performance or scalability reasons, portions of the control 1266 plane functions that need faster response may be moved from the CEs 1267 and off-loaded onto the FEs. For example in OSPF, the Hello 1268 protocol packets are generated and processed periodically. When 1269 done at CEs, the inbound Hello packets have to traverse from the 1270 external interfaces at the FEs to the CEs via the internal CE-FE 1271 channel. Similarly, the outbound Hello packets have to go from the 1272 CEs to the FEs and to the external interfaces. Frequent Hello 1273 updates place heavy processing overhead on the CEs and can 1274 overwhelm the CE-FE channel as well. Since typically there are far 1275 more FEs than CEs in a router, the off-loaded Hello packets are 1276 processed in a much more distributed and scalable fashion. By 1277 expressing such off-loaded functions in the FE model, we can ensure 1278 interoperability. 1280 5.7. Application Layer -- Network Management Protocol 1282 RFC1812 also dictates "Routers MUST be manageable by SNMP." (see 1283 [4] Section 8) In general, for post-association phase, most 1284 external management tasks (including SNMP) should be done through 1285 interaction with the CE in order to support the appearance of a 1286 single functional device. Therefore, it is recommended that SNMP 1287 management agent be implemented by CEs and the SNMP messages 1288 received by FEs be redirected to their CEs. AgentX framework 1289 defined in RFC2741 ([5]) may be applied here such that CEs act in 1290 the role of master agent to process SNMP protocol messages while 1291 FEs act in the role of subagent to provide access to the MIB 1292 objects residing on FEs. AgentX protocol messages between the 1293 master agent (CE) and the subagent (FE) are encapsulated and 1294 transported via ForCES, just like data packets from any other 1295 application layer protocols. 1297 6. Summary 1299 This document defines an architectural framework for ForCES. It 1300 identifies the relevant components for a ForCES network element, 1301 including (one or more) FEs, (one or more) CEs, one optional FE 1302 manager, and one optional CE manager. It also identifies the 1303 interaction among these components and discusses all the major 1304 reference points. It is important to point out that, among all the 1305 reference points, only the Fp interface between CEs and FEs is 1306 within the scope of ForCES. ForCES alone may not be enough to 1307 support all desirable NE configurations. However, we believe that 1308 ForCES over Fp interface is the most important element in realizing 1309 physical separation and interoperability of CEs and FEs, and hence 1310 the first interface that ought to be standardized. Simple and 1311 useful configurations can still be implemented with only CE-FE 1312 interface being standardized, e.g., single CE with full-meshed FEs. 1314 7. Security Considerations 1316 In general, the physical separation of two entities usually results 1317 in a potentially insecure link between the two entities and hence 1318 much stricter security measurements are required. For example, we 1319 pointed out in Section 4.1 that authentication becomes necessary 1320 between CE manager and FE manager, between CE and CE manager, 1321 between FE and FE manager in some configurations. The physical 1322 separation of CE and FE also imposes serious security requirement 1323 for ForCES protocol over Fp interface. This section first attempts 1324 to describe the security threats that may be introduced by the 1325 physical separation of the FEs and the CEs, and then it provides 1326 recommendation and guidelines for secure operation and management 1327 of ForCES protocol over Fp interface. 1329 7.1. Analysis of Potential Threats Introduced by ForCES 1331 This section provides the threat analysis for ForCES, with a focus 1332 on Fp interface. Each threat is described in details with the 1333 effects on the ForCES protocol entities or/and the NE as a whole, 1334 and the required functionalities that need to be in place to defend 1335 the threat. 1337 7.1.1. "Join" or "Remove" Message Flooding on CEs 1339 Threats: A malicious node could send a stream of false "join NE" 1340 or "remove from NE" requests on behalf of non-existent or 1341 unauthorized FE to legitimate CEs at a very rapid rate and thereby 1342 create unnecessary state in the CEs. 1344 Effects: If by maintaining state for non-existent or unauthorized 1345 FEs, a CE may become unavailable for other processing and hence 1346 suffer from denial of service (DoS) attack similar to the TCP SYN 1347 DoS. If multiple CEs are used, the unnecessary state information 1348 may also be conveyed to multiple CEs via Fr interface (e.g., from 1349 the active CE to the stand-by CE) and hence subject multiple CEs to 1350 DoS attack. 1352 Requirement: A CE that receives a "join" or "remove" request 1353 should not create any state information until it has authenticated 1354 the FE endpoint. 1356 7.1.2. Impersonation Attack 1358 Threats: A malicious node can impersonate a CE or FE and send out 1359 false messages. 1361 Effects: The whole NE could be compromised. 1363 Requirement: The CE or FE must authenticate the message before 1364 accepting and processing it. 1366 7.1.3. Replay Attack 1368 Threat: A malicious node could replay the entire message previously 1369 sent by an FE or CE entity to get around authentication. 1371 Effect: The NE could be compromised. 1373 Requirement: Replay protection mechanism needs to be part of the 1374 security protocol to defend this attack. 1376 7.1.4. Attack during Fail Over 1378 Threat: A malicious node may exploit the CE fail-over mechanism to 1379 take over the control of NE. For example, suppose two CEs, say CE-A 1380 and CE-B, are controlling several FEs. CE-A is active and CE-B is 1381 stand-by. When CE-A fails, CE-B is taking over the active CE 1382 position. The FEs already had a trusted relationship with CE-A, 1383 but the FEs may not have the same trusted relationship established 1384 with CE-B prior to the fail-over. A malicious node can take over as 1385 CE-B if such trusted relationship is not established during the 1386 fail-over. 1388 Effect: The NE may be compromised after such insecure fail-over. 1390 Requirement: The level of trust relationship between the stand-by 1391 CE and the FEs must be as strong as the one between the active CE 1392 and the FEs. The security association between the FEs and the 1393 stand-by CE may be established prior to fail-over. If not already 1394 in place, such security association must be re-established before 1395 the stand-by CE takes over. 1397 7.1.5. Data Integrity 1399 Threats: A malicious node may inject false messages to legitimate 1400 CE or FE. 1402 Effect: An FE or CE receives the fabricated packet and performs 1403 incorrect or catastrophic operation. 1405 Requirement: Protocol messages require integrity protection. 1407 7.1.6. Data Confidentiality 1409 Threat: When FE and CE are physically separated, a malicious node 1410 may eavesdrop the messages in transit. Some of the messages are 1411 critical to the functioning of the whole network, while others may 1412 contain confidential business data. Leaking of such information may 1413 result in compromise even beyond the immediate CE or FE. 1415 Effect: Sensitive information might be exposed between CE and FE. 1417 Requirement: Data confidentiality between FE and CE must be 1418 available for sensitive information. 1420 7.1.7. Sharing security parameters 1422 Threat: Consider a scenario where several FEs communicating to the 1423 same CE share the same authentication keys for the Fp interface. If 1424 any FE or the CE is compromised, all other entities are 1425 compromised. 1427 Effect: The whole NE is compromised. 1429 Requirement: To avoid this side effect, it�s better to configure 1430 different security parameters for each FE-CE communication over Fp 1431 interface. 1433 7.1.8. Denial of Service Attack via External Interface 1435 Threat: When an FE receives a packet that is destined for its CE, 1436 the FE forwards the packet over the Fp interface. Malicious node 1437 can generate huge message storm like routing protocol packets etc. 1438 through the external Fi/f interface so that the FE has to process 1439 and forward all packets to CE through Fp interface. 1441 Effect: CE encounters resource exhaustion and bandwidth starvation 1442 on Fp interface due to an overwhelming number of packets from FEs. 1444 Requirement: Rate limiting mechanism needs to be in place at both 1445 FE and CE. Rate Limiter can be configured at FE for each message 1446 type that are being received through Fi/F interface. 1448 7.2. Security Recommendations for ForCES 1450 The requirements document [3] suggested that ForCES protocol should 1451 support reliability over Fp interface, but no particular transport 1452 protocol is yet specified for ForCES. This framework document does 1453 not intend to specify the particular transport either, and so we 1454 only provide recommendations and guidelines based on the existing 1455 standard security protocols that can work with the common transport 1456 candidates suitable for ForCES. 1458 We highlight two existing security protocol solutions, namely IPsec 1459 (IP Security) [14] or TLS (Transport Layer Security) [13]. TLS 1460 works with reliable transports such as TCP or SCTP, while IPsec can 1461 be used with any transport (UDP, TCP, SCTP). Both TLS and IPsec can 1462 be used potentially to satisfy all of the security requirements for 1463 ForCES protocol. 1465 It is important to realize that even if the NE is in a single-box, 1466 the DoS attacks can still be launched through Fi/f interfaces. 1467 Therefore, it is still important to have counter-measurement as 1468 stated in 1.1.9 for DoS while authentication, confidentiality and 1469 integrity can be provided by the physical security of the box. 1471 7.2.1. Security Configuration 1473 The NE administrator has the freedom to determine the exact 1474 security configuration that is needed for the specific deployment. 1475 For example, ForCES may be deployed between CEs and FEs connected 1476 to each other inside a box over a backplane. In such scenario, 1477 physical security of the box ensures that most of the attacks such 1478 as man-in-the-middle, snooping, and impersonation are not possible, 1479 and hence ForCES architecture may rely on the physical security of 1480 the box to defend against these attacks and protocol mechanisms may 1481 be turned off. However, it is also shown that denial of service 1482 attack via external interface as described in Section 7.1.8 is 1483 still a potential threat even for such "all-in-one-box" deployment 1484 scenario and hence the rate limiting mechanism is still necessary. 1485 This is just one example to show that it is important to assess the 1486 security needs of the ForCES-enabled network elements under 1487 different deployment scenarios. It should be possible for the 1488 administrator to configure the level of security needed for the 1489 ForCES protocol. 1491 7.2.2. Using TLS with ForCES 1493 TLS [13] can be used if a reliable transport such as TCP or SCTP is 1494 used for ForCES over Fp interface. The TLS handshake protocol is 1495 used during association establishment or re-establishment phase to 1496 negotiate a TLS session between the CE and FE. Once the session is 1497 in place, the TLS record protocol is used to secure ForCES 1498 communication messages between the CE and FE. 1500 A basic outline of how TLS can be used with ForCES is described 1501 below. Steps 1) till 7) complete the security handshake as 1502 illustrated in Figure 9 while step 8) is for all the further 1503 communication between the CE and FE, including the rest of messages 1504 after the security handshake shown in Figure 9 and the steady-state 1505 communication shown in Figure 10. 1507 1) During Pre-association phase all FEs are configured with the CEs 1508 (including both the active CE and the standby CE). 1509 2) The FE establishes a TLS connection with the CE (master) and 1510 negotiates a cipher suite. 1511 3) The FE (slave) gets the CE certificate, validates the signature, 1512 checks the expiration date, checks if the certificate has been 1513 revoked. 1514 4) The CE (master) gets the FE certificate and performs the same 1515 validation as the FE in step 4). 1516 5) If any of the check fails in step 5 or step 6, endpoint must 1517 generate an error message and abort. 1518 6) After successful mutual authentication, a TLS session is 1519 established between CE and FE. 1520 7) The FE sends a "join NE" message to the CE. 1521 8) The FE and CE use TLS session for further communication. 1523 Note that there are different ways for the CE and FE to validate a 1524 received certificate. One way is to configure the FE Manager or CE 1525 Manager or other central component as CA, so that the CE or FE can 1526 query this pre-configured CA to validate that the certificate has 1527 not been revoked. Another way is to have the CE and the FE 1528 configured directly a list of valid certificates in the pre- 1529 association phase. 1531 In the case of fail-over, it is the responsibility of the active CE 1532 and the standby CE to synchronize ForCES states including the TLS 1533 states to minimize the state reestablishment during fail-over. Care 1534 must be taken to ensure that the standby CE is also authenticated 1535 in the same way as the active CE, either before or during the fail- 1536 over. 1538 7.2.3. Using IPsec with ForCES 1540 IPsec [14] can be used with any transport protocol, such as UDP, 1541 SCTP and TCP over Fp interface for ForCES. We recommend using ESP 1542 in transport mode for ForCES because message confidentiality is 1543 required for ForCES and the communication between the CE and FE is 1544 point-to-point. 1546 IPsec can be used with both manual and automated SA and 1547 cryptographic key management. But Ipsec�s replay protection 1548 mechanisms are not available if manual key management is used. 1549 Hence, automatic key management is recommended if replay protection 1550 is deemed important. Otherwise, manual key management might be 1551 sufficient for some deployment scenarios, esp. when the number of 1552 CEs and FEs is relatively small. It is recommended that the keys 1553 be changed periodically even for manual key management. 1555 Unlike TLS, IPsec provides security services between the CE and FE 1556 at IP level, and so the security handshake as illustrated in Figure 1557 9 amounts to a "no-op" when manual key management is used. The 1558 following outline the steps taken for ForCES in such a case. 1560 1) During Pre-association phase all FEs are configured with the CEs 1561 (including active CE and standby CE) and SA parameters manually. 1562 2) The FE sends a "join NE" message to the CE. This message and 1563 all others that follow are afforded security service according to 1564 the manually configured IPsec SA parameters, but replay protection 1565 is not available. 1567 It is up to the administrator to decide whether to share the same 1568 key across multiple FE-CE communication, but it is recommended that 1569 different keys be used. Similarly, it is recommended that 1570 different keys be used for inbound and outbound traffic. 1572 If automatic key management is needed, IKE [15] can be used for 1573 that purpose. Other automatic key distribution techniques such as 1574 Kerberos may be used as well. The key exchange process constitutes 1575 the security handshake as illustrated in Figure 9. The following 1576 shows the steps involved in using IKE with IPsec for ForCES. Steps 1577 1) to 6) constitute the security handshake in Figure 9. 1579 1) During Pre-association phase all FEs are configured with the CEs 1580 (including active CE and standby CE), IPsec policy etc. 1581 2) The FE kicks off IKE process and tries to establish an IPsec SA 1582 with the CE (master). The FE (Slave) gets the CE certificate as 1583 part of the IKE negotiation. The FE validates signature, checks the 1584 expiration date, checks if the certificate has been revoked. 1585 3) The CE (master) gets the FE certificate and performs the same 1586 check as the FE in step 3). 1587 4) If any of the check fails in step 3 or step 4, the endpoint must 1588 generate an error message and abort. 1589 5) After successful mutual authentication, IPsec session is 1590 established between the CE and FE. 1591 6) The FE sends a "join NE" message to CE. No SADB entry is created 1592 in FE yet. 1593 7) The FE and CE use the IPsec session for further communication. 1595 FE Manager or CE Manager or other central component can be used as 1596 CA for validating CE and FE certificates during the IKE process. 1597 Alternatively, during the pre-association phase, the CE and FE can 1598 be configured directly with the required information such as 1599 certificates or passwords etc depending upon the type of 1600 authentication that administrator wants to configure. 1602 In the case of fail-over, it is the responsibility of active CE and 1603 standby CE to synchronize ForCES states and IPsec states to 1604 minimize the state reestablishment during fail-over. Alternatively, 1605 the FE needs to establish different IPsec SA during the startup 1606 operation itself with each CE. This will minimize the periodic 1607 state transfer across IPsec layer though Fr (CE-CE) Interface. 1609 8. Normative References 1611 [1] Baker, F., "Requirements for IP Version 4 Routers", RFC 1812, 1612 June 1995. 1614 [2] Floyd, S., "Congestion Control Principles", RFC 2914, September 1615 2000. 1617 [3] Khosravi, H. et al., "Requirements for Separation of IP Control 1618 and Forwarding", work in progress, May 2003, . 1621 9. Informative References 1623 [4] Case, J., et al., "A Simple Network Management Protocol 1624 (SNMP)", RFC 1157, May 1990. 1626 [5] Daniele, M. et al., "Agent Extensibility (AgentX) Protocol 1627 Version 1", RFC 2741, January 2000. 1629 [6] Chan, K. et al., "COPS Usage for Policy Provisioning (COPS- 1630 PR)", RFC 3084, March 2001. 1632 [7] Crouch, A. et al., "ForCES Applicability Statement", work in 1633 progress, June 2002, . 1635 [8] Anderson, T. and J. Buerkle, "Requirements for the Dynamic 1636 Partitioning of Switching Elements", RFC 3532, May 2003. 1638 [9] Leelanivas, M. et al., "Graceful Restart Mechanism for Label 1639 Distribution Protocol", RFC 3478, February 2003. 1641 [10] Moy, J. et al., "Graceful OSPF Restart", work in progress, 1642 March 2003, . 1644 [11] Sangli, S. et al., "Graceful Restart Mechanism for BGP", work 1645 in progress, January 2003, < draft-ietf-idr-restart-06.txt>. 1647 [12] Shand, M. and L. Ginsberg, "Restart Signaling for IS-IS", work 1648 in progress, March 2003, . 1650 [13] Dierks, T. and C. Allen, "The TLS Protocol, version 1.0", RFC 1651 2246, January 1999. 1653 [14] Kent, S. and R. Atkinson, "Security Architecture for the 1654 Internet Protocol", RFC 2401, November 1998. 1656 [15] Harkins, D. and D. Carrel, "The Internet Key Exchange (IKE) ", 1657 RFC 2409, November 1998. 1659 [16] Bellovin, S., "Guidelines for Mandating the Use of Ipsec", 1660 work in progress, October 2002, . 1662 10. Acknowledgements 1663 Joel M. Halpern gave us many insightful comments and suggestions 1664 and pointed out several major issues. T. Sridhar suggested that 1665 the AgentX protocol could be used with SNMP to manage the ForCES 1666 network elements. Many of our colleagues and people in the ForCES 1667 mailing list also provided valuable feedback. 1669 11. Authors' Addresses 1671 Lily L. Yang 1672 Intel Corp., MS JF3-206, 1673 2111 NE 25th Avenue 1674 Hillsboro, OR 97124, USA 1675 Phone: +1 503 264 8813 1676 Email: lily.l.yang@intel.com 1678 Ram Dantu 1679 Department of Computer Science, 1680 University of North Texas, 1681 Denton, TX 76203, USA 1682 Phone: +1 940 565 2822 1683 Email: rdantu@unt.edu 1685 Todd A. Anderson 1686 Intel Corp. 1687 2111 NE 25th Avenue 1688 Hillsboro, OR 97124, USA 1689 Phone: +1 503 712 1760 1690 Email: todd.a.anderson@intel.com 1692 Ram Gopal 1693 Nokia Research Center 1694 5, Wayside Road, 1695 Burlington, MA 01803, USA 1696 Phone: +1 781 993 3685 1697 Email: ram.gopal@nokia.com 1699 12. Intellectual Property Right 1701 The IETF takes no position regarding the validity or scope of any 1702 intellectual property or other rights that might be claimed to 1703 pertain to the implementation or use of the technology described in 1704 this document or the extent to which any license under such rights 1705 might or might not be available; neither does it represent that it 1706 has made any effort to identify any such rights. Information on 1707 the IETF's procedures with respect to rights in standards-track and 1708 standards-related documentation can be found in RFC 2026. Copies 1709 of 1710 claims of rights made available for publication and any assurances 1711 of licenses to be made available, or the result of an attempt made 1712 to obtain a general license or permission for the use of such 1713 proprietary rights by implementors or users of this specification 1714 can be obtained from the IETF Secretariat. 1716 The IETF invites any interested party to bring to its attention any 1717 copyrights, patents or patent applications, or other proprietary 1718 rights which may cover technology that may be required to practice 1719 this standard. Please address the information to the IETF 1720 Executive Director. 1722 13. Full Copyright Statement 1724 Copyright (C) The Internet Society (2003). All Rights Reserved. 1726 This document and translations of it may be copied and furnished to 1727 others, and derivative works that comment on or otherwise explain 1728 it or assist in its implementation may be prepared, copied, 1729 published and distributed, in whole or in part, without restriction 1730 of any kind, provided that the above copyright notice and this 1731 paragraph are included on all such copies and derivative works. 1732 However, this document itself may not be modified in any way, such 1733 as by removing the copyright notice or references to the Internet 1734 Society or other Internet organizations, except as needed for the 1735 purpose of developing Internet standards in which case the 1736 procedures for copyrights defined in the Internet Standards process 1737 must be followed, or as required to translate it into languages 1738 other than English. 1740 The limited permissions granted above are perpetual and will not be 1741 revoked by the Internet Society or its successors or assigns. 1742 This document and the information contained herein is provided on 1743 an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET 1744 ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR 1745 IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 1746 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 1747 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE."