idnits 2.17.1 draft-hares-forces-vs-openflow-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([OpenFlow1-0], [RFC5812], [McKeown2008]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 1049 has weird spacing: '...hnology not ...' == Line 1094 has weird spacing: '...horized bet...' == Line 1111 has weird spacing: '...es with a re...' == Line 1125 has weird spacing: '...l ports is no...' == Line 1152 has weird spacing: '...p table entri...' == (2 more instances...) -- The document date (July 6, 2012) is 4305 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'OpenFlow1-0' is mentioned on line 59, but not defined == Missing Reference: 'Jamal-01' is mentioned on line 120, but not defined == Missing Reference: 'RFC3565' is mentioned on line 155, but not defined == Missing Reference: 'RFC 5810' is mentioned on line 156, but not defined == Missing Reference: 'RFC-5812' is mentioned on line 161, but not defined == Missing Reference: 'GENI' is mentioned on line 195, but not defined == Missing Reference: 'NVO3' is mentioned on line 197, but not defined == Missing Reference: 'SoP' is mentioned on line 198, but not defined == Missing Reference: 'CSO-Arch' is mentioned on line 199, but not defined == Missing Reference: 'F-CE-Mgr' is mentioned on line 328, but not defined == Missing Reference: 'STCP' is mentioned on line 362, but not defined == Missing Reference: 'SSL' is mentioned on line 362, but not defined == Missing Reference: 'McKeown-xx' is mentioned on line 389, but not defined == Missing Reference: 'OF-Act' is mentioned on line 393, but not defined == Missing Reference: 'OF-Inst' is mentioned on line 396, but not defined == Missing Reference: 'OF-ActSet' is mentioned on line 399, but not defined == Missing Reference: 'OF-CS' is mentioned on line 405, but not defined == Missing Reference: 'OF-LS' is mentioned on line 462, but not defined == Missing Reference: 'OF-DP' is mentioned on line 427, but not defined == Missing Reference: 'OFCMP' is mentioned on line 410, but not defined == Missing Reference: 'OF-CTLER' is mentioned on line 423, but not defined == Missing Reference: 'McKeown-2008' is mentioned on line 1270, but not defined == Missing Reference: 'OF-DS' is mentioned on line 430, but not defined == Missing Reference: 'OF-ES' is mentioned on line 436, but not defined == Missing Reference: 'OF-Feature' is mentioned on line 440, but not defined == Missing Reference: 'OF-PipeLine' is mentioned on line 468, but not defined == Missing Reference: 'OF-Port' is mentioned on line 471, but not defined == Missing Reference: 'RFC3549' is mentioned on line 512, but not defined == Missing Reference: 'Englehardt-2010' is mentioned on line 516, but not defined == Missing Reference: 'RFC365' is mentioned on line 528, but not defined == Missing Reference: '1995-2006' is mentioned on line 551, but not defined == Missing Reference: '2006-2007' is mentioned on line 554, but not defined == Missing Reference: 'XORP-2008' is mentioned on line 566, but not defined == Missing Reference: 'FORCES-LFB' is mentioned on line 651, but not defined == Missing Reference: '1995-2005' is mentioned on line 668, but not defined == Missing Reference: 'McKowen-2008' is mentioned on line 765, but not defined == Missing Reference: 'RFC2870' is mentioned on line 832, but not defined ** Obsolete undefined reference: RFC 2870 (Obsoleted by RFC 7720) == Missing Reference: 'McKeown-200' is mentioned on line 1019, but not defined == Missing Reference: 'RFC1812' is mentioned on line 1228, but not defined == Missing Reference: 'Heleplidis-Forces-LFB' is mentioned on line 1460, but not defined == Missing Reference: 'ForCES-LFB-Lib' is mentioned on line 1459, but not defined == Missing Reference: 'Switch' is mentioned on line 1568, but not defined == Unused Reference: 'LFB-Lib' is defined on line 1825, but no explicit reference was found in the text == Outdated reference: A later version (-12) exists of draft-ietf-forces-lfb-lib-06 Summary: 2 errors (**), 0 flaws (~~), 51 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Draft S.Hares 2 Intended status: Informational Huawei 3 Expires: January 6, 2013 4 July 6, 2012 6 Analysis of Comparisons between OpenFlow and ForCES 7 draft-hares-forces-vs-openflow-00.txt 9 Status of this Memo 11 This Internet-Draft is submitted in full conformance with the 12 provisions of BCP 78 and BCP 79. 14 Internet-Drafts are working documents of the Internet Engineering 15 Task Force (IETF), its areas, and its working groups. Note that 16 other groups may also distribute working documents as Internet- 17 Drafts. 19 Internet-Drafts are draft documents valid for a maximum of six 20 months and may be updated, replaced, or obsoleted by other 21 documents at any time. It is inappropriate to use Internet- 22 Drafts as reference material or to cite them other than as "work 23 in progress." 25 The list of current Internet-Drafts can be accessed at 26 http://www.ietf.org/ietf/1id-abstracts.txt 28 The list of Internet-Draft Shadow Directories can be accessed at 29 http://www.ietf.org/shadow.html 31 This Internet-Draft will expire on January 6, 2013. 33 Copyright Notice 35 Copyright (c) 2012 IETF Trust and the persons identified as the 36 document authors. All rights reserved. 38 This document is subject to BCP 78 and the IETF Trust's Legal 39 Provisions Relating to IETF Documents 40 (http://trustee.ietf.org/license-info) in effect on the date of 41 publication of this document. Please review these documents 42 carefully, as they describe your rights and restrictions with 43 respect to this document. 45 Abstract 46 While both ForCES and OpenFlow follow the basic idea of 47 separations of forwarding plane and control plane in network 48 elements, they are technically different. ForCES specification 49 contains both a modeling language [RFC5812] which allows flexible 50 definition of the Flow tables and flow logic. ForCES flow logic 51 include Logical Functional Blocks (LFBs) connected in flow logic 52 that is described in logic of direct graphs augmented by passage 53 of Metadata and grouping concepts. 55 OpenFlow's specifications contain a specific instantiation of 56 Flow tables and flow logic which has emerged from the research 57 community theories. OpenFlow's logic varies based on the 58 revision of the specification (OpenFlow-Paper [McKeown2008], 59 OpenFlow Switch Specification 1.0 [OpenFlow1-0], OpenFlow 1.1 60 [OpenFlow-1.1] Open Configuration 1.0 [OpenFlowConfig-1.0]). 62 Table of Contents 64 1. Introduction...................................................3 65 1.1. ForcES Introduction.......................................4 66 1.2. OpenFlow Introduction.....................................4 67 2. Definitions....................................................5 68 2.1. New Common Configurations.................................6 69 2.2. Forces Definitions........................................8 70 2.3. Open Flow Definitions....................................10 71 3. Comparisons between ForCES and OpenFlow.......................12 72 3.1. Difference in Historical setting.........................12 73 3.2. Difference in Goals......................................14 74 3.3. Difference in Architectural Requirements.................14 75 3.3.1. ForCES System Building Blocks.......................15 76 3.3.2. OpenFlow Building blocks............................17 77 3.3.2.1. Match Fields in OFS............................18 78 3.3.2.2. Flow Logic - Flow Table and Group tables.......18 79 3.3.3. ForCES FE types.....................................22 80 ForCES and OpenFlow FEs can operate either new switching 81 entities or integrated with existing processing as a hybrid. 82 In OFS-1.2, the Ships-in-the-Night (SIN) mode divides 83 existing ports into groups controlled by specific ports 84 (see figure x) or VLANs (figure-x).........................22 85 3.3.4. ForCES Pre-Association..............................23 86 3.3.5. Architectural requirements..........................25 87 3.3.6. ForCES versus OpenFlow - A Central Controller.......33 88 3.4. Difference in Forwarding Model...........................33 89 3.4.1. Looping.............................................34 90 3.4.2. Handling unicast and multicast......................35 91 3.5. Difference in Logical Forwarding Block Libraries.........36 92 3.6. Difference in Protocol Interface.........................36 93 3.6.1 Secure Transport..................................37 94 4. Use of ForCES and OFS in Applications.........................41 95 5. The use of ForCES or OpenFlow in S(D)N or CSO/SOP.............41 96 6. Security Considerations.......................................42 97 7. IANA Considerations...........................................42 98 8. Conclusions...................................................42 99 9. References....................................................43 100 9.1. Normative References.....................................43 101 9.2. Informative References...................................43 102 10. Acknowledgments..............................................44 104 1. Introduction 106 This document analyzes the differences between OpenFlow and 107 ForCES technically from the aspects of goals, architecture, 108 forwarding models, forwarding policy, control plane interaction, 109 configuration of nodes, and applications (firewalls, load- 110 balancers, High-availability nodes). This informational document 111 compares OpenFlow and Forces as of March, 2012 seeking to provide 112 clarity for other discussions of controller/forwarding split, 113 Software Defined Networking, Software Driven Networking, Cloud 114 Service Oriented networking (CSO), and a host of orchestrators 115 for virtualized network devices. 117 A fellow Engineer provided inspiration for this deeper comparison 118 by saying: "OpenFlow-0 is the Diff-Serv Tspec, OpenFlow-1.0 is 119 Forces--, and OpenFlow 1.1 is Forces++." Jamal Salim suggests 120 Open Flow 1.1 does not have the same functionality[Jamal-01]. 122 While this summary brings the expert listener quickly into heart 123 of the issues, this document examines: 125 - Is OpenFlow Switch 1.1.0 really "ForCES++", and "is the group 126 table safe ++ logic? What direction does Open Flow Switch 1.2 127 and 1.3 take us? 129 - Where does Open Flow's Config fit in the picture? 131 - How does this help us get to Clouds Service Oriented Networks 132 (CSO) enable by Software defined networks (SDN) or software 133 driven networks (SDN)? 135 And that, as the saying goes is the "rest of the story" of this 136 draft. Here's hoping the readers of this document will decide 137 and argue with the author to refine the next-generation of 138 hardware devices. 140 1.1. ForcES Introduction 142 ForCES (Forwarding and Control Element Separation) work in IETF 143 has defined a new environment to build network devices that split 144 the network devices into control plane and forwarding plane into 145 units. For example, a router could be considered a network 146 element (NE) with a control plane running router protocol and a 147 data plane forwarding IP traffic. 149 The drive to have ForCES NE device split arose from the desire to 150 build hardware forwarding blades out of flexible hardware 151 components. These hardware devices included Network Processors 152 and network specific ASICs. 154 The ForCES environment defines requirements [RFC3654], goals 155 [RFC3565], architecture and protocol requirements [RFC3654], a 156 controller-forwarder communication protocol [RFC 5810]. ForCES 157 also describes a policy on how to building the forwarding engine 158 out of a set of logical functional blocks (LFBs)which are 159 connected as a directed graph [RFC5812]. ForCES allows many 160 different Forwarding Engines (FE) to linked to Controller Engines 161 (CE) via the protocols. ForCES provides a modeling language [RFC- 162 5812] to describe these FE devices so that controllers can load 163 control the devices, load forwarding tables, and keep track of 164 statistics. ForCES RFCs also define how the ForCES protocol runs 165 over SCTP [RFC5811. 167 1.2. OpenFlow Introduction 169 OpenFlow[McKeown2001, p. 1]] arose out of the frustration that 170 network research projects felt at not being able to experiment 171 with new protocols on large-scale networks. Experimentation on 172 research networks did not have a large enough scale to provide a 173 reasonable test-bed for new research ideas for the Internet. Pure 174 commercial networks would not allow experimental protocols, and 175 commercial router vendors took 3-5 years to create a new protocol 176 features. The OpenFlow researchers suggested an alternative to 177 allow the research to creating a slice out of commercial network 178 to try out new ideas for network. 180 OpenFlow's initial paper grew into OpenFlow Switch Specification 181 versions 1.0 [OFS-1.0], 1.1 [OFS-1.1], 1.2[OFS-1.2], 1.3[OFS-1.3], 182 and (likely) 1.4 [OFS-1.4]. Additionally a Config and Management 183 Protocol version 1.0[OFC-1.0] and version 1.1 [OFC-1.1], and a 184 set of papers and application notes on implementations. A hybrid 185 Specification [OFHy-1.0] suggests how Open Flow may be combined 186 with existing network OpenFlow switches which mix existing 187 network devices (routers/switches) with OpenFlow controlled 188 switches in either a Ships-in-the Night (SIN) or a hybrid model 190 OpenFlow's host of specifications and ForCES flexible 191 reprogramming of the network element architecture can be 192 considered the first wave of Software-Defined Networking (SDfN) 193 where software can alter how the forwarding engine's logic 194 operates. This work arises out of the GENI research 195 [GENI][McKeown2008]. The current industry discussion regarding 196 Software-Defined Networking (SDfN) or Software-Driven Networking 197 (SDrN), Network Virtualized Overlays [NVO3], Service-Oriented 198 Protocols (SoP)[SoP] or network orchestrators of Cloud Service 199 Oriented Network (CSO)forwarding [CSO-Arch] have a great deal 200 confusions as they apply the terms to ForCES and OpenFlow. Rather 201 than carefully define each of these terms explicitly at the 202 outset, we will give brief expansions of the abbreviations and 203 return to the definitions later in the draft after examining the 204 FORCES draft. 206 This document will examine ForCES and OpenFlow goals, 207 architecture, forwarding conceptual models, Controller-forwarder 208 communication mechanisms and protocols, the policies in the 209 loading of forwarding state, configurations of nodes, and sanity 210 checking of the forwarding. 212 These basic concepts will be then examined in terms of specific 213 implementations (switch, hybrid router/switch, wireless, load- 214 balancer, firewall) as described by ForCES and OpenFlow reports. 216 Finally, the document will return to defining S[*]N, SOP, and 217 Cloud-Oriented Services (CSO). 219 2. Definitions 221 Definitions used in this document 223 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 224 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 225 "OPTIONAL" in this document are to be interpreted as described in 226 RFC-2119 [RFC2119 228 The following RFC2119 definitions used in this document are: 230 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 231 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 232 "OPTIONAL" in this document are to be interpreted as described in 233 RFC-2119 [RFC2119]. 235 ForCES definitions relevant to this documents discussion are 236 taken from [RFC3654][RFC3746][RFC5810] as noted below. The quoted 237 italized definitions come from the ForCES RFC, and the non-quoted 238 text applies ForCES RFC text to this document. 240 2.1. New Common Configurations 242 Controlling Entity (CE): is defined as an entity which remote 243 controls the forwarding engine. This Entity can be either a 244 ForCES CE or a Open Flow Controller. 246 Controlling Entity Manager (CE-MGR): This documents loosens the 247 ForCES CE-Manager definition to allow Open Flow and ForCES to be 248 compared. This document defines the CE manager as a logical 249 entity (distributed or located in physical or virtual device) 250 which controls which controllers attach to which logical 251 Forwarding Entities. The Controllers can be in the same physical 252 switch/device in the control plane or other logical software. A 253 CE-Mgr may also be within a VM hypervisor, a VM hypervisor 254 manager, or other virtual software. The CE-Mgr logical function 255 may be distributed across many CE as a defined function. This 256 definitional Allows both ForCES CE-Mgrs and Open Flow Controller 257 collaboration/management via coordinated remote configuration of 258 OF Capable Switches. 260 Controlled Router-Switch (CRSW): A Controlled Switch is a network 261 entity performing switching capability that is controlled by 262 remotely by either the ForCES protocol (FP) or the Open Flow 263 Protocol (OFP). This switch can perform IP routing, MPLS 264 switching, Trill Switching or Layer 2 Switching. 266 Forwarding Entity (FE): is defined as an entity which forwarding 267 packets or frames under the control of the CE. This entity can 268 be an ForCES FE (F-FE) or an Open Flow Capable Switch (OF-CS). An 269 Open Flow Capable switch can either be a hybrid switch or a Open- 270 Flow Only switch. 272 FE Manager (FE-MGR): The FE-Manager controls the FEs assignments. 273 This document defines FE-Manager's logical entity may be a 274 logical software process residing within local switch/device in 275 the control plane or management plane. The FE-Mgr can also within 276 a VM hypervisor, or a VM hypervisor manager, or other virtual 277 software. The FE-Mgr can be a remote service managing the 278 forwarding engine. The Open Flow Configuration [OFC-1.0] 279 Configuration Service point with its logical configuration 280 function may also have a FE-MGR function. This FE-Mgr capability 281 is an capability outside the [OFC-1.0] specification. 283 Pre-Association Phase (Pre-A): This document defines a Pre- 284 Association Phase (Pre-A) as the period during which a CE- 285 Management(Forces CE-Mgr or OF controller groups) and FE-Managers 286 (Forces FE-MGR (F-FE-MGR)or OF-CS management) determines which 287 Controlling entity (CE)controls which Forwarding Entity (FE). 289 2.2. Forces Definitions 291 Force Forwarding Element (F-FE) - "A logical entity that 292 implements the ForCES Protocol. FEs use the underlying hardware 293 to provide per-packet processing and handling as directed by a 294 CE via the ForCES Protocol." [RFC3654] ForCES forwarding FE 295 supports forwarding rules insertion. 297 ForCES Control Element (F-CE) - "A logical entity that implements 298 the ForCES Protocol and uses it to instruct one or more FEs on 299 how to process packets. CEs handle functionality such as the 300 execution of control and signaling protocols." [RFC3654] The 301 ForCES CE controller may be located within the same hardware box 302 on a different blade or across an Ethernet connection, or across 303 a L3 Link (if security used). 305 ForCES Network Element(F-NE)- "An entity composed of one or more 306 CEs and one or more FEs. To entities outside a NE, the NE 307 represents a single point of management. An NE usually hides its 308 internal organization from external entities and represents a 309 single point of management to entities outside the NE." [RFC3654] 310 The NE's single point of management can be at the IP layer, the 311 Ethernet layer, and at a virtual layer. In this document, the 312 network element is examined as being the set of network functions 313 in the hardware that collaborates to act like a switch. This 314 less strict definition allows ForCES to be compared with the Open 315 Flow work. 317 ForCES Pre-Association Phase (F-Pre-A): ForCES defines the Pre- 318 Association Phase (F-Pre-A) as "the period of time during which a 319 FE Manager(see below)and a CE Manager (see below) are determining 320 which FE is a part of the network element" [RFC3654]. 322 FE Manager(F-FE-Mgr)- ForCES (F-FE-Mgr)is "A logical entity that 323 operates in the Pre-Association Phase and is responsible for 324 determining to which CE(s) a FE should communicate. This process 325 is called CE Discovery and may involve the FE manager learning 326 capabilities of available CEs." [RFC3654] 328 CE Manager (CE-Mgr) - Forces CE-MGR[F-CE-Mgr] is "A logical 329 entity that operates in the pre-associaation phase and is 330 responsible for determining to which FE(s) a CE should 331 communicate. This process is called FE discovery and may involve 332 the CE manager learning the capabilities of available FEs. The CE 333 manager may use anything from statics configuration to a pre- 334 association phase protocol." [RFC3654] 335 ForCES Protocol (ForCES-Proto) - "While there may be multiple 336 protocols used within the overall ForCES architecture, the term 337 "ForCES protocol" refers to only the post-association phase 338 protocol." [RFC3654] The ForCES protocol operates between the "Fp 339 reference points" of the ForCES architecture (as shown in figure 340 1) [RFC5810]. "Basically, the ForCES protocol works in a master- 341 slave mode in which the FEs are slaves and the CEs are 342 masters." [RFC5810] The location and exact instantiation of the 343 CE logical entities associated with the FE logical entity is 344 flexible. The CE entities could reside on a process on a local 345 switch communicating to other process off the local switch. 347 ForCES Protocol Layer (ForCES PL) - "A layer in the ForCES 348 protocol architecture that defines the ForCES protocol messages, 349 the protocol state transfer scheme, and the ForCES protocol 350 architecture itself (including requirements of ForCES TML)" 351 [RFC5810] This layer is defined in RFC5810. 353 ForCES Protocol Transport Mapping Layer (ForCES TML) - "A layer 354 in ForCES protocol architecture that uses the capabilities of 355 existing transport protocols to specifically address protocol 356 message transportation issues, such as how the protocol 357 messages are mapped to different transport media (like TCP, IP, 358 ATM, Ethernet, etc.), and how to achieve and implement 359 reliability, multicast, ordering, etc. The ForCES TML 360 specifications are detailed in separate ForCES documents, one 361 for each TML." [RFC5810, p. x]. The ForCES TMLs focused on are 362 STCP [STCP] and SSL[SSL]. TM handles transport of messages 363 [reliable or non-reliable], "congestion control", "multicast", 364 ordering, and other things [RFC5810, p. 14]. 366 LFB (Logical Function Block) - The basic building block that is 367 operated on by the ForCES protocol. The LFB is a well-defined, 368 logically separable functional block that resides in an FE and is 369 controlled by the CE via the ForCES protocol. The LFB may reside 370 at the FE's data path and process packets or may be purely an FE 371 control or configuration entity that is operated on by the CE. 372 Note that the LFB is a functionally accurate abstraction of the 373 FE's processing capabilities, but not a hardware-accurate 374 representation of the FE implementation. 376 LFB Class and LFB Instance - LFBs are categorized by LFB classes. 377 An LFB instance represents an LFB class (or type) existence. 378 There may be multiple instances of the same LFB class (or type) 379 in an FE. An LFB class is represented by an LFB class ID, and an 380 LFB instance is represented by an LFB instance ID. As a result, 381 an LFB class ID associated with an LFB instance ID uniquely 382 specifies an LFB existence. 384 Physical Forwarding Element - The physical element that forwards 385 the packets. 387 2.3. Open Flow Definitions 389 Open Flow (OF): [McKeown-xx] defines OF as a "way for researchers 390 to run experiments in networks they use every day"[McKeown-2008, 391 p.1]. 393 Open Flow Action [OF-Act]: [OF-1.1.0] defines an OF-Act as an 394 action that may: "forward the packet to a port", or modifies the 395 packet". Actions may be specified in "Open Flow Instruction" 396 [OF-Inst] in Flow Table Entry or "action buckets in Group Table 397 Entry"[OF-1.1.0,p.4]. 399 Open Flow Action Set [OF-ActSet]: [OF-1.1] defines an OF-ActSet 400 as "a set of actions associated with the packet that are 401 accumulated while the packet is processed by each table, and are 402 executed when the packet exits the processing Pipeline."[OF-1.1.0, 403 p. 5]. 405 Open Flow Capable switch [OF-CS]: [One or more physical or 406 virtual switch device which can act as an operational context for 407 an Open Flow Logical Switch [OF-LS]. A OF-CS hosts an Open Flow 408 Data Path [OF-DP]. 410 Open Flow Configuration and Management Protocol [OFCMP]: [OFC-1.0] 411 states the [OFCMP-1.0]enables the remote configuration of Open 412 Flow Data Path (OF-DP). The OFCMP allows a controller to 413 configure the OF-DP on the Open Flow Logical Switch (OF-LS) "so 414 that the controller can communicate and contro" the OF-LS via 415 Open Flow Protocol (OFP). The OFCMP allows dynamic association of 416 resources with OF-LS in an OF-CS. [OFC-1.0] defines the protocol, 417 but not the resource allocation mechanisms. 419 Open Flow Configuration Point (OFCPT): [OFC-1.0] defines an OFCPT 420 as "a service" which sends OFCMP messages to a OF-CS with an OF- 421 LS inside. 423 Open Flow Controller [OF-CTLER]: [McKeown-2008] defines the OF- 424 CTLER as a "controller that adds and deletes Flow entries on 425 behalf of experiments" [McKeown-2008, p. 3]. 427 Open Flow Datapath [OF-DP]: [OFC-1.0] defines OF-DP as an 428 abstraction called Open Flow Logical Switch [OF-LS]. 430 Open Flow Dedicated Switches [OF-DS]: [McKeown-2008] defines OF- 431 DS as a "dumb datapath element that forwards packets between 432 ports as defined by a remote process" [McKeown-2008, p3.] The 433 Open Flow process programs the forwarding engine for this dumb 434 datapath switch. 436 Open Flow Enable Switches [OF-ES]: [McKeown-2008] defines OF-ES 437 as "commercial switches, routers or access points" enhanced by 438 adding the OF feature 440 Open Flow Feature [OF-Feature]: Open Flow[McKeown2008] and [OF- 441 1.0.0] defines the OF-Feature as adding the features of an Open 442 Flow Logical Switch [OF-LS]. These features are the Open Flow 443 "Flow Tables", "Secure Channel that connects the switch to the 444 controller", and "the Open Flow Protocol" [McKeown-2008, p. 445 3][OF-1.0.0, p.2]. 447 Open Flow's Flow Table (OF-FT): [McKeown-2008] defines a Flow 448 table in OF as having "an action with every flow table entry to 449 tell the switch how to process the flow" [McKeown-2008, p. 2] 451 Open Flow's Flow Table Entry (OF-FTE): [McKeown-2008], [OF-1.0.0], 452 [OF-1.1.1], [OF-1.1.2], and [OF-1.1.3] define the specific of an 453 single entry in a flow table. See Section x.x for a detailed 454 comparison of this entry. 456 Open Flow Group (OF-G): [OF-1.2] defines an OF group (OF-G) as a 457 list of Open Flow "action buckets" and "a means to choose one or 458 more buckets to apply on a per-packet basis" [OF-1.2, p. 5]. 460 Open Flow Group Table (OF-GT): 462 Open Flow Logical Switch[OF-LS]: OFC-1.0 defines the OF-LS as an 463 abstraction of the "open flow datapath". 465 Open Flow Packet (OF-Pkt): [OF-1.1.0] defines OF-Pkt as "ethernet 466 frame including header and payload" [OF-1.1.0, p. 4]. 468 Open Flow Pipeline [OF-PipeLine]: [OF-1.2] defines OF-Pipeline as 469 "a set of linked tables that matching, forwarding, and 471 Open Flow Port [OF-Port]: [OF-1.2] defines an Open Flow port as a 472 place where packets enter and exit the Open Flow Pipeline. 474 Open Flow Protocol (OFP): OF 1.0 defines "an open protocol to 475 program the flow table in switches and routers" in which "a 476 controller communicates with a switch"[McKeown-2008,p. 2-3]. 478 Open Flow Switch (OFS): [McKeown-2008] defines an Open Flow 479 Switch as a ethernet switch with "at least" the following three 480 functions: "(1) a Flow Table","(2) "a secure channel that 481 connects the" controller with the switch over which the open flow 482 protocol runs, and (3)an "open flow protocol" [McKeown-2008,p 483 2.][OF-1.0.0,p.2]. 485 [OF-1.1.0] defines an OFS as "one or more flow tables and a group 486 table which perform packet look-ups and forwarding", and an open 487 flow channel to an external controller"[OF-1.1.0,p.3]. The 488 external controller controls the switch via the Open Flow 489 protocol (OF-Proto)[OF-1.1.0, p.3]. [OF-1.3.0] adds that the 490 "switch communicates with the controller, and the controller 491 controls the switch"[OF-1.3.0, Section 2, paragraph 1.] 493 3. Comparisons between ForCES and OpenFlow 495 ForCES and OpenFlow are very similar in the following aspects: 497 o Both ForCES and OpenFlow are efforts to separate control plane 498 from forwarding plane; 500 o Both ForCES and OpenFlow protocols standardize information 501 exchange between the control and forwarding plane. 503 Although both ForCES and OpenFlow can be considered as the 504 solutions for forwarding and control plane separation, they are 505 different in many aspects. This section compares them in their 506 history, goals, architecture, forwarding model and protocol 507 interface. 509 3.1. Difference in Historical setting 511 ForCES work began during the 1995-2000 timeframe with the 512 development of netlink [RFC3549]. The linux netlink began its 513 linux driver history as first a "character device /dev/netlink 514 for Linux kernel 1.3.31" but was superceded by "Alexey 515 Kunzetsov's socket-based af_netlink.c in Linux v 2.1.15" 516 [Englehardt-2010]. The rtnetlink brought configuration and 517 router queries to links. The netlink socket allowed messages 518 between kernel and user space regarding routes, firewalls, and 519 monitoring. 521 The historical context of the 1995-2002 timeframe saw the initial 522 growth of the US Internet and spread into non-US sites. The 523 historical changes included changes that began the split of tight 524 binding of control plane to data plane. Forwarding plane 525 elements (ASICs, TCAMs, Network processors) and backplanes began 526 the growth of non-stop forwarding with high-availability. ForCES 527 notes that the data processors have "forwarding tables", "per- 528 flow Qos Tables and access control lists" [RFC365]. ForCES had 529 control processors were general-purpose processors that support 530 route calculations. 532 ForCES began in quasi-commercial realm of linux development where 533 linux developers used routing software with netlink to build 534 early Internet networks. Alexey's early work was deployed in 535 Russian and European networks to turn linux boxes into Routers. 536 By early 2000, this work had migrated to router boxes seeking to 537 harden routers to provide non-stop forwarding. Netlink 538 implementations were provided with many commercial OEM standards 539 for switches and routers. 541 ForCES work came out of the desire to expand the basic netlink 542 protocol into an architecture that allow quick modeling of new 543 forwarding hardware and an extensible control-plane to forwarding 544 plane communication. Early discussions in ForCES look to allow 545 coordination of multiple control planes as well as control plane 546 to forwarding plane functionality. However, the IETF decision was 547 to restrict the first versions of ForCES to architecture, CE-FE 548 communication and FE modeling. 550 OpenFlow arises out of the academic's communities realization 551 that the hardening of commercial of network infrastructure [1995- 552 2006] to support businesses, caused a "reluctance to experiment 553 with production traffic"[McKeown-2008, p. 1]. The GENI (Global 554 Environments for Network Research)[2006-2007] suggested that: 555 a)Internet's infrastructure faced "serious problems" in "security, 556 reliability, manageability, and evolvability" and "possible 557 solutions" existed in research, but there were "severe 558 experimental barriers" to test new ideas in wide-spread 559 deployments [GENI-2007, p. vii]. A new US research network would 560 allow slices of routers to be used for researcher's experiments 561 in network protocols. McKeown and colleagues work examined how 562 these experiments could be extended to run [McKeown-2008] on 563 local campuses. McKeown and colleagues examined persuading 564 commercial routers to provide an open interface for 565 experimentation or using existing open source solutions (linux, 566 XORP[XORP-2008]). Their conclusion was that "commercial solutions 567 are too closed", and "research solutions have insufficient 568 performance or fanout, and are too expensive." [McKeown-2008, p. 569 2]. 571 3.2. Difference in Goals 573 RFC3654 lists the the architectural goals of ForCES. OpenFlow 574 Switch (OFS) Specification version 1.0.0[OFS-1.0.0] and OFS 575 version 1.1.0 [OF-1.1.0] refer to [McKeown-2008] as the basis of 576 these specifications. This document's goal comparison compares 577 the four goals [McKeown-2008] sets against [RFC3654]. 579 The goal ForCES is to define the "architecture", "architectural 580 modeling" and protocols to "logically separate control and data 581 forwarding plans of an IP (IPv4, IPv6, etc.) networking device" 582 [RFC3654, p. 1]. ForCES network device (aka. network element (NE)) 583 can be a router, IP switch, firewall, switch. ForCES redefines 584 network elements to be logical relationships placed on physical 585 devices. 587 McKeown et al. state the goals of the OpenFlow approach was to 588 find "high-performance and lost-cost implementations" of new 589 network algorithms, capable of being used in "broad range of 590 research", adaptable to commercial "close platforms", and able to 591 "isolate experimental traffic from production traffic [McKeown- 592 2008, p. 2]. 594 Difference in goals underscores the original commercial focus of 595 ForCES and the experimental focus of OpenFlow. 597 3.3. Difference in Architectural Requirements 599 Architecture sets the building blocks for system, and 600 architectural requirements sets rules for interconnecting the 601 building blocks. 603 Building blocks for ForCES include the CE(s), FE(s), ForCES 604 protocol (CE to/from FE), FE-Manager, CE-Manager, and logical 605 input flows. Within the FEs there are Logical Forwarding Blocks 606 connected together in a directed graph. The flow processing 607 passes along input port, (modified)frame, metadata (which may 608 include actions). The flow stream may be output to interfaces 609 (logical or physical). 611 Building blocks for OpenFlow include Controllers (~CEs) and 612 Forwarding Units (~FEs) with OpenFlow processing. OpenFlow logic 613 is designed in terms of Flow Processing controlled by Flow Tables 614 (McKeown-2008][OFS-1.0][OFS-1.1], and Group tables [OFS-1.1]which 615 operate on the modified frame, metadata, or group of actions via 616 actions or instructions (a group of actions and forwarding 617 commands). 619 Both flow streams Flow processing may cause the flow to multiple 620 into several streams or combine multiple streams into one. 622 3.3.1. ForCES System Building Blocks 624 The building blocks within the ForCES architecture are the CEs 625 (controller elements), FEs (forwarding elements), and an 626 interconnect protocol between CE(s) and FE(s). ForCES also 627 recognized the logical functions of a FE-Manager and a CE- 628 Manager. Figure 1 shows a diagram from RFC5810 that details 629 interaction between all these components. 631 The ForCES CE controls switching, signaling, routing, and 632 management protocols. Each CE is a logical unit which may be 633 located within the same box, different boxes, or across the 634 network. ForCES architecture [RFC3746] allows CEs to control 635 forwarding in multiple FEs. 637 ForCES defines logical Forwarding Elements (FEs) that reside on a 638 variety of physical forwarding elements (PFE) such as a "single 639 blade (PFE)", partition within blade, or multiple PFEs in a 640 single box, or among multiple boxes [RFC3746, p. 2]. The ForCES 641 logical FEs could also be run within Virtual Machines (VMs) 642 within a single box or a set of boxes or a cloud. A single FE may 643 be connected to multiple CEs providing strong redundancy. FE 644 internal processing is described in terms of Logical Forwarding 645 Blocks (LFBs) connected together in a directed graph that 646 "receive, process, modify and transmit packets along with 647 metadata"[RFC5810, p. 6]. The FE model determines the LFBs, the 648 topological description, the operational characteristics, and the 649 configuration parameters of each FE. 651 The Forces Logical Forwarding Block (LFBs) Library [FORCES-LFB] 652 provides the class descriptions for Ethernet, IP Packet 653 Validation, IP Forwarding LFBs, and Redirection, MetaData, and 654 Scheduling. Forces-LFB document demonstrates how these logical 655 blocks can be placed within a machine to support IP Forwarding 656 (IPv4/IPv6)for unicast & multicast and ARP processing[Forces-LFB, 657 p. 17]. 659 ForCES architecture [RFC3746] allows CEs to control forwarding in 660 multiple FEs. ForCES also recognized the logical functions of a 661 FE-Manager and a CE-Manager. The FE manager determines the CE(s) 662 each FE should communicate with. The CE manager determines which 663 FEs each CE should communicate with. The ForCES defines the FE- 664 Manager and CE-Manager to operate in a "pre-association" phase of 665 the communication to set-up the FORCES communication path. 666 Similarities between the functions of the CE-Mangers and FE 667 managers of ForCES and modern hypervisors may come from the 668 creative interplay of early open source communities [1995-2005]. 669 Applications directly interacting with ForCES components (CEs or 670 CE-Managers or FE-Manager) could be described as interactions 671 with the CEs or CE-Managers. 673 Figure 1 shows ForCES Architectural Diagram [RFC5810]. 675 --------------------------------------- 676 | ForCES Network Element | 677 -------------- Fc | -------------- -------------- | 678 | CE Manager |---------+-| CE 1 |------| CE 2 | | 679 -------------- | | | Fr | | | 680 | | -------------- -------------- | 681 | Fl | | | Fp / | 682 | | Fp| |----------| / | 683 | | | |/ | 684 | | | | | 685 | | | Fp /|----| | 686 | | | /--------/ | | 687 -------------- Ff | -------------- -------------- | 688 | FE Manager |---------+-| FE 1 | Fi | FE 2 | | 689 -------------- | | |------| | | 690 | -------------- -------------- | 691 | | | | | | | | | | 692 ----+--+--+--+----------+--+--+--+----- 693 | | | | | | | | 694 | | | | | | | | 695 Fi/f Fi/f 697 Fp: CE-FE interface 698 Fi: FE-FE interface 699 Fr: CE-CE interface 700 Fc: Interface between the CE manager and a CE 701 Ff: Interface between the FE manager and an FE 702 Fl: Interface between the CE manager and the FE manager 703 Fi/f: FE external interface 705 Figure 1: ForCES Architectural Diagram [RFC5810] 707 3.3.2. OpenFlow Building blocks 709 OpenFlow architecture consists of set of OpenFlow Switches with a 710 Flow Table, a Secure Channel between controller and switch, and 711 an Open flow protocol. OpenFlow switches can either be only 712 controlled by OpenFlow, enabled by Open Flow [OFS 1.0] (shared), 713 or hybrid Switches (OFS 1.2). 715 ---------------- ---------------- 716 | Application1 | | Application2 | ...... 717 ---------------- ---------------- 718 | APIs | 719 ---------------------------------------- 720 CE | --------------- --------------- | 721 | | OpenFlow |------| OpenFlow | | 722 | | Controller | | Controller | | 723 | --------------- --------------- | 724 ---------------------------------------- 725 | | | 726 | OpenFlow Protocol | 727 | | | 728 NE&FE | | | NE&FE 729 -------------- | -------------- 730 | OpenFlow | | | OpenFlow | 731 | Switch | | | Switch | 732 -------------- | -------------- 733 | | | | | | | | | 734 | | | | | | | | | 735 | | | | | | | | | 736 Fi/f | NE&FE | | Fi/f 737 | -------------- | 738 | | OpenFlow | | 739 | | Switch | | 740 | -------------- | 741 | | | | | | 742 | | | | | | 743 -------- | | -------- 744 Fi/f 746 Fi/f: FE external interface 748 Figure 2: OpenFlow Architectural Diagram 749 by Using the terms NEs, FEs, CEs 751 The Flow table provides entries on how to process a flow whose 752 header fields match a pattern in the header field [OFS-1.0.0] or 753 a set of meta data generated from pipeline processing of a 754 header[OFS-1.1.0, figure 4][OFS-1.3]. 756 The matching of a packet in OFS-1.0.0 based on first exact match 757 of header and/or meta data, and secondly wild-card entries. The 758 wild-card entries contain a priority field to order the process 759 of matching. For example, a priority of "1" will be the first 760 wild card processed. 762 3.3.2.1. Match Fields in OFS 764 The match field has been expanding as the ONF specifications 765 evolve - from a 10-tuple [McKowen-2008], to 12-tuple [OFS-1.0], 766 and to a OFS-1.1.0 a 15-tuple, to 39-tuple in OFS-1.3[OFS-1.3] 767 (14 required and 25 optional). 769 The original 10-tuple includes ingress port, VLAN ID, Ethernet 770 source address, destination address and type, the IPv4 771 source/destination address, and IP protocol, TCP/UDP source & 772 destination port. The 12-tuple adds VLAN priority, and IP Tos 773 bits. The 15-tuple adds: metadata, MPLS label, MPLS traffic 774 class. The TCP/UDP source & destination port have redefined for 775 ICMP packets to have instead he the ICMP Type and ICMP code. 776 [OFS-1.3-pre]required matches include the 10-tuple plus IPv6 777 source and destination addresses and UDP source and destination 778 ports. 780 3.3.2.2. Flow Logic - Flow Table and Group tables 782 The flow table operation and data structures vary based on the 783 version of OpenFlow Switch specification. 785 OFS in versions [McKeown-2008] and [OFS-1.0.0] operate on logic 786 in flow tables which are executed in ascending order. Each Flow 787 Table ID must be greater than the current Flow table id. [OFS- 788 1.1.0][OFS-1.3] flow logic operates on flow tables and group 789 tables which allow jumps based on "Goto table" logic or 790 combinations of flows. 792 |=============| |==============| |=============| 794 |Flow table 0 |---.|Flow Table 1 | _ -. |Flow Table n | 796 |=============| |==============| |=============| 797 Figure 3: Flow Table [OFS-0.9][OFS-1.0] 799 All OFS Flow tables match on data and perform actions [OFS- 800 0.8][OFS-0.9][OFS-1.0][OFS-1.1][OFS-1.2][OFS-1.3]. Later versions 801 [OFS-1.1.0][OFS-1.2][OFS-1.3] use instructions to immediately 802 perform actions or to queue specific actions for later 803 processing. These same later versions also allow metadata to be 804 stored to be passed along to additional processing. 806 |----------------| 808 | Group table 1 | 810 |------. |action-bucket1 |----. egress-port-1 812 | | action-bucket2 ] 814 | |----------------| 816 | 818 |=============| |==============| |=============| 820 |Flow table 0 |---.|Flow Table 1 | _ -. |Flow Table n | 822 |=============| |==============| |=============| 824 Figure 3: Flow Table [OFS-1.1] 826 1) Action Specifics 828 [McKeown-2008] has three actions in the flow tables: forwarding 829 of a matched packet to a specific port or ports, sending the 830 packet to the controller, or dropping the packet. This simply 831 processing is why some engineers suggest that OFS-2008 is similar 832 to the RSVP T-Spec [RFC2870]. 834 [OFS 1.0.0] actions direct switch processing to forward packet, 835 drop packet, enqueue packet (optional), and modify-field in 836 packet (optional). Forwarding packets can be sent to all ports, 837 the controller, local switches forwarding stack, send out input 838 port, and specific ports after performing table actions & send 839 out specified port, send via normal (L2/L3/VLAN) processing, and 840 flood (via minimum spanning tree) ports. This causes some 841 engineers to consider OFS 1.0 equivalent to be Forces--. 843 [OFS-1.1.0] uses instructions within each flow entry to determine 844 how a packet and associated data is processed. These associated 845 data includes: ingress port packet came in on, the generated 846 meta-data and an action set. The action set is a set of commands 847 to execute prior to sending the packet out. The [OFS-1.1.0] 848 instructions are: "Apply-Actions", "Clear-Actions", "Write- 849 Actions", "Write-metadata", "Goto Table" [OFS-1.1.0, p. 14]. 850 Actions are either applied immediately with apply action command, 851 or stored (via an Write-Action)in action sets for later 852 processing in the action-set via a Write-Action. The existing 853 actions can be cleared with a "Clear-Action". [OFS-1.1.0] actions 854 are output packet, set queue, drop packet, process via group 855 table, push/pop tags, and set-field. [OFS-1.1.0] action sets 856 include single entries for any of the following: copy TTL inward 857 in packet, pop/push actions to packet, copy TTL outwards, 858 decrement TTL, set fields in packet, apply QoS Actions (e.g. set 859 queue), and apply group actions, and output packet. 861 2) Flow Logic Encoding 863 [OFS-1.0.0] encodes the ForCES LFB in table sequences. The LFB 864 directed graph of ForCES modeling is encoded in sequences of Flow 865 Table. OFS 1.0 specifies that the Ethernet header (similar to 866 ForCES Ethernet II) is the basic frame for all input. A fixed 867 processing ARP, IPv4, TCP/UDP, and ICMP packets are specified 868 based matches beyond the Ethernet header. The Forces LFB library 869 provides building blocks for matches beyond the Ethernet to ARP, 870 IPv4 and IPv6 packets, and Meta data. The OFS-1.0.0 does not have 871 Metadata. 873 [OFS-1.1.0] match of a flow goes against specific table 15-tuple 874 header. If the frame/packet matches, the flow table can alter the 875 packet (via immediate actions), add metadata (for later 876 handling), set an actions in action set, pass the processing to a 877 specific table (Flow Table or Group Table), or pass the 878 processing to the next table in the sequence. The information 879 passed on to the next processing is [ingress port, (post- 880 modification) frame/packet, metadata, and action set. 882 [OFS-1.1.0] allows processing between tables to carry the ingress 883 port, the packet, generated metadata, and an action set. An 884 action set is a set of commands to execute prior to sending the 885 packet out. [OFS-1.1.0] uses Flow table with instruction. The 886 instructions can be which can be "Apply-Actions", "Clear- 887 Actions", "Write-Actions", 888 "Write-metadata", "Goto Table" [OFS-1.1.0, p. 14]. Actions are 889 either applied immediately with apply action command, or stored 890 (via an Write-Action) for later processing in the action-set via 891 a Write-Action. The existing actions can be cleared with a 892 Clear-Action. 894 [OFS-1.1.0] supports two types of tables: flow tables, and group 895 tables. The group table structure has a group identifier, group 896 type, counters, and an order list action buckets. Action buckets 897 contain a set of actions with parameters. If the group table has 898 "zero" action buckets, then no processing occurs. 900 [OFS-1.1.0] group type field specifies how the action buckets 901 operate on the packet. The group can be "all", "select", 902 "indirect", and "fast-failover" [p.7]. The "all" bucket provides 903 multicast and/or broadcast support execute all buckets. The 904 packet is effectively cloned and sent out. The select, indirect, 905 and fast-failover execute one bucket. In select, the switch 906 determines which port via internal algorithm (e.g. round-robin). 907 In "indirect", the group table logic selects the bucket. The 908 "fast-failover", the first live bucket is chosen. The single- 909 bucket choses may restrict flows if the specified buckets and/or 910 output ports are down. 912 This new sequential logic is the basis of some engineers comment 913 that OpenFlow 1.1 is ForCES++. 915 ForCES people indicate that that the group table logic plus "Go 916 to" logic is simply a LFB model of a specific type. [Haleplidis- 917 2012] demonstrates on the LFB library concept can be used to 918 capture all of the OFS-1.1.0 specification. 920 3.3.3. ForCES FE types 922 ForCES and OpenFlow FEs can operate either new switching entities 923 or integrated with existing processing as a hybrid. In OFS-1.2, 924 the Ships-in-the-Night (SIN) mode divides existing ports into 925 groups controlled by specific ports (see figure x) or VLANs 926 (figure-x) 928 |=============| |==============| |=============| 930 | standard | | Open Flow | | Forces | 932 | CE | | controller | | CE | 934 |=============| |==============| |=============| 936 || || || 938 || || || (forwarding) 940 |----------------------------------------------------------- 942 | |==========| |==============| |=============| | 944 | | standard | | Open Flow | | Forces FE | | 946 | | FE board | | FE | | | | 948 | |==|==|====| |====|==|======| |====|===|====| | 950 |-----|--|--------------|--|------------------|---|---------- 952 Standard ports OFS ports ForCES ports 954 Figure x - Hybrid mode per port 955 |=============| |==============| |=============| 957 | standard | | Open Flow | | Forces | 959 | CE | | controller | | CE | 961 |=============| |==============| |=============| 963 || || || 965 || || (forwarding) || 967 |-----------------------------------------------------------| 969 | |==========| |==============| |=============| | 971 | | standard | | Open Flow | | Forces FE | | 973 | | FE board | | FE | | | | 975 | |==|==|====| |====|==|======| |====|===|====| | 977 | | | | | | | | 979 | vlan1 vlan10 vlan2 vlan15 vlan3 vlan7 | 981 | | | | | | | | | | | | | | 983 |---|-|---|-|---------|-|---|-|-------------|-|---|-|-------| 985 S OFS ports ForCES ports 987 Figure x - Hybrid mode per port 989 3.3.4. ForCES Pre-Association 991 Neither the ForCES protocol nor the OFS protocols [McKeown- 992 2008/OFS-0, OFS-1.0.0, OFS-1.1.1] specify how the CEs/controllers 993 and FEs/forwarding switch meet. 995 Do CEs go to the FEs meeting place and CEs pick up the FE that 996 delights their forwarding fancy? How do they found out where 997 eligible FEs are meeting? Do FEs have choice on which CEs select 998 them, and if so what are the criteria? 1000 The forming of intimate relationships between CEs and FEs remains 1001 to the readers of the specification as mysterious as the pre- 1002 association stages of human group dating or clique forming. 1004 ForCES specifications specifically call this phase a pre- 1005 association phase. The ForCES architecture names the entity that 1006 coordinates the forming of associations (like a Jewish match- 1007 maker) for CEs and for FEs. The CE-Manager determines which FEs 1008 each CE should talk to. The FE-Manager determines which CEs each 1009 FE should talk to. Only when the associations between each CE 1010 and its FEs, and each FE and its CEs are complete, does the 1011 system complete pass from pre-association phase to association 1012 phase. 1014 It is assumed that some protocol interactions within the logical 1015 ForCES network entity (or entities) determine how CEs will 1016 coordinate their work. However, the IETF work specifically 1017 denoted this CE-CE coordination work as a second phase of work. 1019 OpenFlow Switch specifications ([McKeown-200][OFS-0.8][OFS- 1020 0.9][OFS- 1.0] OFS-1.1] ignore the concept of this dynamic 1021 meeting processing. 1023 Either OFS specification missed this concept. Perhaps the OFS 1024 specifications assumed a static configuration as part of a boot 1025 process of the hybrid switch will set-up some basics. It is 1026 possible that the lack specification may come from the sponsors 1027 of the specification wanting proprietary pre-association 1028 interactions. If so, this provides an interesting line of 1029 demarcation between standards and OFS standard. 1031 In any case, this oddity from the OFS proponents leaves one to 1032 ask "What is the rest of the story on the pre-association phase?" 1034 3.3.5. Architectural requirements 1036 RFC3654 specifies 15 architectural requirements. Table 15 1037 provides summary of this requirements and possible OFS (McKeown- 1038 2008/OFS 0, OFS 1.0, OFS 1.1). 1040 ForCES Architectural Requirement OpenFlow Switch Architecture 1042 -------------------------------- ---------------------------- 1044 1. CE/FEs connect via variety of Controllers/switch 1045 communicate 1046 interconnect technologies. over a secure connection 1047 [RFC3654, p. 5] [McKeown-2008][OFS-1.0][OFS-1.1] 1049 FE can use different technology not specified 1050 than CE/FE topologies. 1052 2. "FEs MUST support a minimal [McKeown-2008][OFS-1.0][OFS-1.1] 1053 set of capabilities necessary specify a set of required 1054 for establishing network actions, instructions, Flow 1055 connectivity (e.g. interface actions, and an implied set of 1056 discovery, port up/down port functions(e.g.interface 1057 functions.) discovery, port up/down). 1058 Beyond this minimal set, the These OFS specifications 1059 ForCES architecture MUST also declare some set of 1060 NOT restrict the types of features optional. 1061 numbers of capabiliti8es 1062 that FEs may contain. 1063 [RF3654, p.5] 1065 3. "Packets MUST be able to arrive [OFS-1.0][OFS-1.1]specifies 1066 at the NE by one FE and leave in ofp_port the OFPP_IN_PORT 1067 the NE via different FE." flag which allows the port to 1068 [RFC3654, p.5] explicitly send it back out the 1069 input port [OFS-1.0, p. 18] 1070 [OFS-1.1, p. 26] 1072 ForCES Architectural Requirement OpenFlow Switch Architecture 1074 -------------------------------- ---------------------------- 1075 4. "A NE must support the [OFS-1.0][OFS-1.1] describes 1076 appearance of a single the devices as an individual 1077 functional device." switch, and provides 1078 (e.g. single IP TTL reduction.) the capability to reduce TTL 1079 [RFC3654,p. 5] [OFS-1.1,section 4.7, p. 12] 1081 such as push/pop of VLAN IDs 1082 and/or MPLS headers [OFS-1.1, 1083 p. 12] 1085 4b. However, external entities 4b. No pre-association logic has 1086 (e.g. FE managers and been defined. 1087 CE managers) MAY have direct 1088 Access to individual ForCES 1089 protocol elements for providing 1090 information to transition 1091 [RFC3654, p. 5] 1093 5."The architecture MUST provide Beyond a secure channel 1094 a way to prevent unauthorized between some "controller 1095 FORCES protocol elements entity and switch" 1096 from joining an NE." no prevention of unauthorized 1097 [RFC3654, p. 5] access has been encoded. 1099 6. A FE must be able to [OFS-1.0][OFS-1.1] have 1100 asynchronously inform the CE "three message types: 1101 of a failure or controller-to-switch, 1102 increase/decrease in available asynchronous, 1103 resources or capabilities symmetric [OFS-1.0, p. 10] 1104 on the FE. [OFS-1.1, p. 16]. 1105 Asynchronous messages 1107 ForCES Architectural Requirement OpenFlow Switch Architecture 1109 -------------------------------- ---------------------------- 1110 Switches send a controller (CE) 1111 Messages with a received 1112 packet 1113 (packet-in), notification of 1114 a removed flow table entry 1115 (flow-removed), changes in 1116 port 1117 status (port-status), and 1118 error 1119 conditions (error) [OFS-1.0, 1120 pp-10-12)][OFS-1.1, p. 16- 1121 17] 1123 "Thus, the FE MUST support The change in port status 1124 error monitoring and reporting" is covered,but memory status 1125 (e.g. "number of physical ports is not covered 1126 Or "memory changes"). 1127 [RFC3654,p. 5] 1129 7. "The Architecture MUST support [McKeown-2008], [OFS-1.0] 1130 mechanisms for CE redundancy [OFS-1.1]do not specifically 1131 or CE failover. provide CE Redundancy or 1132 CE failover. 1134 1. This includes the The OFS protocol supports 1135 ability for CE and FEs echo request/reply 1136 to determine when there message [OFS-1.0, p. 41] 1137 is a loss of association [OFS-1.1, p. 55-56]. 1138 between them, ability 1139 to restore the association The OFS-1.1 provides a 1140 and efficient state barrier message that 1141 (re)synchronization provides synchronization 1142 mechanisms. [OFS-1.1, p. 50]. 1144 ForCES Architectural Requirement OpenFlow Switch Architecture 1146 -------------------------------- ---------------------------- 1147 The OFS-1.1 protocol supports 1148 query by the controller of the 1149 switch features, capabilities, 1150 configuration, flow table 1151 configuration, flow table 1152 entries, group table entries, 1153 port configuration (pp. 36-42). 1154 FS-1.1 also provides 1155 Statistics on description of 1156 Switch (OFPST_DESC), 1157 Flow table status 1158 (OFPST_FLOW),aggregate flow 1159 statistics (OFPST_AGGREGATE), 1160 port statistics (OFPST_PORT), 1161 queue statistics (OFSPST_QUEUE), 1162 group statistics (OFSPST_GROUP), 1163 and experimenter extension 1164 (OFPST_EXPERIMENTER) (p.43-49). 1166 7. (CE redundancy - continued). 1167 2. This also includes the OFS-1.1 states: 1168 ability to preset the "In the case the switch loses 1169 actions an FE will take contact with the current 1170 in reaction to loss of controller, as a result of an 1171 association to its CE echo request timeout, 1172 (e.g., whether the FE TLS session timeout, or other 1173 will continue to forward disconnection, it should 1174 packets or whether it attempt to contact one or more 1175 will halt operations." backup controllers. 1176 [RFC3654, p. 6.] The ordering of the backup 1177 Controllers is not specified by 1178 the protocol." 1180 "The switch should immediately 1181 enter either "fail secure mode", 1182 or "fail standalone mode" if it 1183 loses connection to the 1184 controller, depending on the 1185 switch implementation and 1186 configuration." [OFS-1.1, p.18] 1188 In "fail secure mode", the 1189 FE behavior remains the same 1190 except FE drops "packets and 1191 messages" destined for CE. 1192 In "fail-standalone mode", 1193 FE processes all packets 1194 acts as a "legacy switch 1195 or router." [OFS-1.1, p. 18] 1197 ForCES Architectural Requirement OpenFlow Switch Architecture 1198 -------------------------------- ---------------------------- 1200 Flow entries persist over 1201 Failure in either fail secure 1202 Mode or fail standalone mode. 1204 8. "FEs must be able to redirect [McKeown-2008][OFS-1.0][OFS-1.1] 1205 control packets addressed to allow packets to forward to 1206 their interfaces to the CE. Controller for processing. 1207 The (FE) MUST also redirect 1208 other relevant packets 1209 (E.g., such as those 1210 with Router Alert Option set) 1211 to their CE. 1212 The CEs MUST be able [OFS-1.0][OFS-1.1] Flow tables 1213 to configure the packet allow packet redirection filters 1214 redirections information/filters on the FEs with action Forward 1215 on the FEs. controller (OFS-1.0, p. 6), 1216 (OFS-1.1, p. 13). 1218 The CEs MUST also be able [OFS-1.0][OFS-1.1] allow a 1219 to create packets and have CE to FE message (packet-out) 1220 its FEs deliver them. to send frames/packets out 1221 a specific port 1222 [OFS-1.0, p. 10], [OFS-1.1,p.17] 1224 9."Any proposed ForCES [OFS-1.0][OFS-1.1] do not 1225 architecture MUST explain consider this requirement. 1226 how that architecture supports Future OFS work may consider 1227 all the router functions as this set of features. 1228 defined in [RFC1812]." 1229 1. Includes: IPv4 Forwarding Options 1231 2. Should include: IPv6 forwarding options 1233 ForCES Architectural Requirement OpenFlow Switch Architecture 1235 -------------------------------- ---------------------------- 1237 10. "In a ForCES NE, the CE(s) [OFS-1.0][OFS-1.1] do not 1238 MUST be able to learn the consider this requirement. 1239 topology by which the FEs 1240 in the NE are connected." 1242 11. The ForCES NE architecture [McKeown-2008], [OFS-1.0] 1243 MUST be capable of supporting [OFS-1.1] do not consider 1244 (i.e. must scale to) at least scale of CE/FE 1245 communications. 1246 hundred of FEs and tens of 1247 thousands of ports. 1249 12. "The ForCES NE architecture [McKeown-2008], [OFS-1.0], 1250 MUST allow FEs and CEs to join [OFS-1.1] do not consider 1251 and leave NEs dynamically." issues relating to 1252 active join/leaving of 1253 CEs and FEs in communication. 1255 13. "The ForCES architecture [McKeown-2008],[OFS-1.0], 1256 MUST support multiple CEs [OFS-1.1] do not directly 1257 and FEs. However, coordination discuss how multiple CEs 1258 between CEs is out of scope will attach to FE. Or FEs 1259 of ForCES. attach to a CE. 1261 [Historical note: 1262 The restriction of CE 1263 coordination was a desired 1264 phase 2 work of the ForCES group.] 1266 ForCES Architectural Requirement OpenFlow Switch Architecture 1268 -------------------------------- ---------------------------- 1270 14. For pre-association phase [McKeown-2008], [OFS-1.0], 1271 set-up, monitoring, [OFS-1.1] do not consider 1272 configuration issues, it MAY issues relating setting up 1273 be useful to use standard the links between CEs and 1274 management mechanisms for FEs. Some "magic" occurs 1275 CEs and FEs. and the CE is talking to 1276 a particular FE. 1277 The ForCES architecture and 1278 requirements do not preclude 1279 this. 1281 [OFS-1.0][OFS-1.1] give 1282 In general, for no discussion on other 1283 post-association phase, management process (SNMP) 1284 most management tasks SHOULD outside the [OFC-1.0] 1285 be done through interactions using netconf and beep 1286 with the CE. 1287 In certain conditions 1288 (e.g., CE/FE disconnection), 1289 it may be useful to allow 1290 management tools (E.g., SNMP) 1291 to diagnose and repair problems. 1293 The following guidelines MUST 1294 be observed: 1295 1. The ability for a 1296 management tool (e.g., SNMP) 1297 to be used to read 1298 (but not change) the state 1299 of FE SHOULD NOT be precluded. 1300 2. IT MUST NOT be possible 1301 for management tools 1302 (e.g., SNMP, etc) to 1303 change the state of an 1304 FE in a manner that 1305 affects overall NE behavior 1306 without the CE being notified. 1308 3.3.6. ForCES versus OpenFlow - A Central Controller 1310 ForCES and OpenFlow seek to split the control plane and the 1311 forwarding engine. Both protocols using a secure connection can 1312 be used to interact with a central controller. ForCES has spent 1313 more time determining how CEs and FEs might find one or more 1314 central controllers. OpenFlow Specifications are just beginning 1315 to rediscover the need for this work. 1317 Both Forces an OpenFlow can provide the ability of a logically 1318 centralized controller to: 1320 o Collect the network view and make decisions according to 1321 control logics (or applications); 1323 o Interact with forwarding hardware (FE) to install forwarding 1324 policy and state, 1326 o Provide open APIs to users to add new features. 1328 ForCES has considered security issues (such as Denial of Service 1329 (DOS)) and the mechanisms for grouping CEs with an FE, or FEs 1330 with a CE, Forwarding Models, and Forwarding Libraries. 1332 OFS specifications have focused on defining one simple 1333 functionality that can be implemented in specific networks. For 1334 example, many discussions point to code deployed in Google. 1336 3.4. Difference in Forwarding Model 1338 ForCES and OFS pipeline processing of frames/packets include the 1339 basic steps of matching framing, processing frame, and 1340 outputting/dropping frame. The processing of a frame occurs in a 1341 pipeline of processes where initially processing adds the 1342 metadata and actions that subsequent processing will use to 1343 create the final packet that will be sent via output port or 1344 dropped. 1346 Key ingredients of a good pipeline process are: 1348 1) a deterministic logic based that doesn't loop, 1350 2) handles both unicast and multicast traffic, 1352 3) Flexible matching that can growth with new features, 1354 4) Metadata to allow passing of results to subsequent stages, 1356 5) Logic that allows some stages to be skipped, and 1358 6) Allows for no match (Table Miss). 1360 3.4.1. Looping 1362 In ForCES, [RFC5812] defines the FE (Forwarding Element) model 1363 based on an abstraction of Logical Functional Blocks (LFBs). In 1364 this model, each FE is composed of multiple LFBs that are 1365 interconnected in a directed graph, which is represented by the 1366 LFB topology model. The directed graph model prevents the recycle 1367 of processing in a loop. Each LFB defines a set of processing on 1368 handling frames/packets. For example, typical LFBs include 1369 IPv4/IPv6 Longest Prefix Matching, etc. XML is used to describe 1370 LFB model formally. 1372 In [OFS-0.8][OFS-0.9][OFS-1.0] the forwarding model has been 1373 static defining specific functions for early experimentation of 1374 the switch. Loops have been prevent 1376 [OFS-1.0] defines the Flow tables identified by a sequential 1377 number [0,1,2_n]. Processing loops are prevent by defining that a 1378 flow table can only transfer to a higher flow table. 1380 [OFS-1.1] provides a Group table to augment the Flow Table logic 1381 described above. If a flow matches, instructions in the flow 1382 table may direct the packet toward specific table (group table or 1383 flow table). This jumping provides skips in sequential process 1384 unlike [OFS-1.0]. In addition, the "goto" action allows skips 1385 between tables. 1387 3.4.2. Handling unicast and multicast 1389 [OFS-0.9][OFS-1.0] do not provide an easy to provide cloning for 1390 multicast. The group table in [OFS-1.1] provides the necessary 1391 cloning for multiple outputs of a single packet. 1393 A ForCES IPv4MultiLPB and IPv6MultiLPB could be defined beyond 1394 today's ForCES standards. These LFBs would use an LPM to match 1395 the multicast address, and generate a list of "L3PortID" metadata 1396 to identify a set of ports the cloned packet could be sent out. 1397 This metadata could be passed to the EthernetEncap LFB. 1399 3.4.3. Flexible matching 1401 All OFS specifications ([OFS-0.8][OFS-0.9][OFS- 1402 1.0][OFS1.1][OFS1.3-pre] seeks to match the header data against 1403 the flow table's match field. Ranging from a 10-29 possible 1404 matches in the header and metadata, the OFS provides flexible 1405 matching within the data packet (Ethernet, MPLS, IP, TCP, UDP). 1407 [Heleplidis-Forces-LFB] shows the matching capability in OFS-1.1 1408 can be implemented in ForCES LFBs. 1410 3.4.4. Metadata to allow passing of results to subsequent stages, 1412 Both ForCES and OFS ([OFS-1.1][OFS-1.3-pre]) allow metadata to be 1413 passed to subsequent spaces. 1415 3.4.5. Optionally skipping logic 1417 [OFS-1.1] provides a Group table to augment the Flow Table logic 1418 described above. If a flow matches, instructions in the flow 1419 table may direct the packet toward specific table (group table or 1420 flow table). This jumping provides skips in sequential process 1421 unlike [OFS-1.0]. The Group Table concepts provides the ability 1422 to group flows for execution, single out a single flow for 1423 additional processing, and use port liveness mechanisms for fast- 1424 failover. [OFS-1.1,p, 7]. 1426 The ForCES directed graph model can also allow the skips in 1427 processing by having multiple exits from. 1429 3.4.6. Table Miss 1431 A frame may not match any table in the forwarding pipeline. 1433 [OFS-0.9] states "if no matching entry can be found for a packet, 1434 the packet will be sent to the controller over the secure 1435 channel"[p. 8]. 1437 Experience has taught the OpenFlow community this can be 1438 problematic. 1440 [OFS-1.0.0-errta] states the following exceptions: "Sending the 1441 packet to the controller on table-miss may overload the switch or 1442 the controller, and some of those packets may have to be dropped, 1443 and this may be an issue in some deployments" [p., 3]. 1445 The OFS-1.0.0-errta suggests that the vendor extension may allow 1446 the packet to be dropped or forwarded via pipeline. However, due 1447 to many application use of table-miss to do topology discovery or 1448 watch traffic - this feature is continued. 1450 ForCES does not define a global Table-Miss, but allows the LFB 1451 model to define these issues. 1453 3.5. Difference in Logical Forwarding Block Libraries 1455 The Open-Flow group is beginning to consider flexible description 1456 of the next OFS switches using a modeling language. No modeling 1457 language has been approved as yet. 1459 The Force LFB Library [ForCES-LFB-Lib] has been defined and 1460 implemented. [Heleplidis-Forces-LFB] shows the modeling language 1461 can support OFS-1.1 definitions. 1463 3.6. Difference in Protocol Interface 1465 The OFS protocol and the ForCES protocol both use: 1467 . Secure transport protocols over which they operate (3.6.1) 1469 . Messages to establish the Controller/CE - FE connections, 1471 . Messages to Loading of forwarding logic (3.6.3) 1473 . Messages to Configuration the box (3.6.4), 1475 . Error handling messages (3.6.5), 1477 . Liveness protocols (3.6.6), and 1478 . Sending packets for processing to/from controller (3.6.8). 1480 3.6.1 Secure Transport 1482 ForCES defines two layers of protocols: ForCES Protocol Layer 1483 (ForCES PL) and ForCES Protocol Transport Mapping Layer (ForCES 1484 TML). 1486 ForCES PL defines Protocol between FEs and CEs (Fp Reference 1487 Point). ForCES Protocol Transport Mapping Layer (ForCES TML) is 1488 defined to transport the PL messages. It is expected that more 1489 than one TML will be standardized and interoperability is 1490 guaranteed as long as both endpoints support the same TML. 1491 [RFC5811] has defined a SCTP-based TML for ForCES. 1493 OpenFlow defines the protocol between controller and OpenFlow 1494 switches, i.e. OpenFlow protocol. OFS-1.1 states that the data 1495 channel is "usually encrypted using TLS, but may be run over 1496 TCP"[OFS-1.1.0,p. 16] 1498 3.6.2 Types of Messages 1500 As Table-x shows, ForCES and OFS protocol are remarkably similar. 1501 Many OFS authors indicate the influence of the ForCES protocol on 1502 the OFS work. Due to the IETF review, ForCES protocol's top 1503 level is carefully designed with orthogonal features of 1504 association setup, association teardown, config, query, events, 1505 packet redirect, and heartbeat. 1507 The OFS protocols [OFS-0.8][OFS-0.9][OFS-1.0][OFS-1.1] provide 1508 similar features, but have some overlapping functions. Similarly, 1509 the OFS protocol has 1511 o Hello (initial association), 1513 o Reading of switch Features (read/response), 1515 o config of switch and flow pipeline's via Flow tables 1516 [OFPT_FLOW_MOD), group tables [OFPT_GROUP_MOD], ports 1517 [OFPT_PORT_MOD]. 1519 o Flow Removed [OFPT_FLOW_REMOVED] - Flow entry is removed 1520 due to either an idle timer or a hard timeout. 1522 o Query of statistics (OFPT_STATS_REQUEST, 1523 OFPT_STATS_RESPONSE), 1525 o Packet-OUT- redirect of packet from controller out a port, 1527 o PACKET-IN - redirect packet inbound port to controller, 1529 o Echo request/reply - heartbeat from OFS switches. 1531 In addition, the OFS protocol has a Barrier Request/reply message 1532 that allows the controller to synchronize message processing. 1533 This general (but lose) definitional has allowed experimentation 1534 with OFS switches. 1536 Due to IETF review, the similar ForCES protocol has a clear 1537 orthogonal set of actions described in terms of execution and 1538 transaction models. The CE can set execution flags on sets on 1539 transactions (groups of functions). The execution of 1540 transactions can be: execute-all-or-none, continue-execute-on- 1541 failure, and execute-until failure. A transaction set is must me 1542 the ACDity test (atomicity, consistency, isolation, and 1543 durability)[RFC5810,section 4.3.1.2]. Transaction sets have an 1544 start, middle, and end. The transaction can also signal an abort. 1546 The notification messages for Error and Port status are uniquely 1547 specified in the OFS as a notification. In ForCES this is 1548 included with the general notification category. 1550 Table-x ForCES vs. OFS messages 1552 Message: ForCES OFS-0.9 OFS-1.0 OFS-1.1 1554 ================ ====================================== 1556 Associate ----------Hello---------- 1558 CE/FE (Req/Rsp) X X X X 1560 Association -----Feature Request/Response--- 1562 Stop (Req/Rsp) X X X X 1564 Query/ X ----Feature or STATS(Req/Rsp)--- 1566 Query Response X X X X 1568 Config [Switch] ---Config (non-flow table)----- 1570 (Req/Rsp) X X X X 1572 Config (Req/Rsp) X ------ FLOW_MOD------- 1574 X X X X 1576 Table-x ForCES vs. OFS messages 1578 Message: ForCES OFS-0.9 OFS-1.0 OFS-1.1 1580 ================ ====================================== 1582 Heartbeat(Forces) X ---Echo-Request/Response--- 1584 /Echo-Request(OFS) X X X X 1586 Redirect ---Packet_in & Packet-Out) 1588 X X X 1590 Execution flags X ----Barrier-Set-Queue 1592 X X 1594 Notification x X X 1596 3.6.3 Loading of forwarding logic 1598 ForCES and OFS both use TLVS to add, modify, and delete the flow 1599 entry. In addition, ForCES has a concept of "commit" to a set of 1600 changes to allow multiple stages of set. 1602 OFS has the concept of modifying or deleting only strictly 1603 matching flows (OFPFC_MODIFY_STRICT, OFPFC_DELETE_STRICT). This 1604 is different that the OFS default of modifying all flows with 1605 that match (with wildcard). 1607 3.6.4 Configuration 1609 ForCES defines changing configuration of the switching Forwarding 1610 pipeline within the protocol. OF-Config-1.0 has provided a 1611 protocol to use an OpenFlow Configuration Point (logical mode) 1612 that can configure one or more OFS via the OpenFlow configuration 1613 protocol. The configuration protocol runs on top of TLS or TCP. 1614 The configuration protocol sets the following information: 1616 o Failure standby mode (fail secure or fail standalone) 1617 o Encryption mode (TLS or not), 1619 o Queue configurations (min-rate, max-rate, experimenter), 1621 o Ports (speed, no-receive, no forward, no packet-in, link- 1622 down, blocked, life) and optionally (duplex-mode, copper- 1623 medium, fiber-medium auto-negotiation, pause, asymmetric- 1624 pause), 1626 o Data path id of switch. 1628 3.6.5 Error handling and sanity checking 1630 The error handling indicates errors that occur within the 1631 protocol. The Error handling includes message form and action 1632 failure. For OFS the action failure includes all interactions 1633 such as: hello failure, bad request, bad flow action, bad flow 1634 instruction, bad match, cannot modify entry in flow table, 1635 cannot modify entry in group table, cannot modify port. 1637 Due to IETF review, the ForCES errors and notifications are 1638 define to contain all cases within the protocol. The error 1639 processing contains sanity checking. 1641 3.6.6 Failure of CE/FE connection 1643 Heart beat messages in both ForCES and OFS insure "liveness" of 1644 the CE/FE connection. The ForCES heartbeats are traffic 1645 sensitive, and are only sent if no traffic has occurred. 1647 OFS predefines that switches should enter the following based on 1648 losing connections with controller: "Fail secure mode" or "fail 1649 standalone mode" [OFS-1.1.0,section 5.3]. In Fail secure mode, 1650 forwarding continues as previous with the only change that no 1651 packets can be uploaded to the processor. 1653 In fail standalone mode, the OSF switch drops into the Ethernet 1654 legacy mode [OFPP_NORMAL]. 1656 If the ForCES protocol is supporting the high-availability 1657 function, the begins the engage the high-availability statement 1658 machine. OpenFlow specifications have not yet described how High- 1659 availability will work in Open-Flow. 1661 4. Use of ForCES and OFS in Applications 1663 ForCES and OFS [OFS-1.0][OFS-1.1] have been encoding in a variety 1664 of applications. These application include: 1666 . Firewalls, 1668 . Loads balancers, 1670 . Switches 1672 . High-availability routers, 1674 . Wireless devices. 1676 . Table-x ForCES vs. OFS messages 1678 5. The use of ForCES or OpenFlow in S(D)N or CSO/SOP 1680 This section will contain a summary of the common capabilities of 1681 ForCES and OpenFLow in environments of centralized controllers, 1682 distributed controllers, and hybrid (centralized/distributed 1683 control) suggested by open flow. 1685 5.1 - Centralized controller logic 1687 ForCES and OFS have been designed for centralized controller 1688 logic. ForCES has considered the pre-association and association 1689 phase of the CE-FE relationship with all the timing issues. The 1690 execution and transaction model provide a strongly reviewed model 1691 to provide roll-forward and roll-back of transactions. The high- 1692 availability drafts for ForCES provide a clear case on how to 1693 keep high-availability of forwarding and CE processing while 1694 distributing the flows. 1696 ForCES has a clear body of work developed over years of 1697 implementation experience. 1699 OFS specifications do not deal with how controllers find FEs. 1700 However, numerous companies are developing centralized 1701 controllers. The standardization efforts for Hybrid (OFS-1.2) and 1702 the next generation OFS switch (OFS-1.3) indicate an effort to 1703 capture this growing body of wisdom. 1705 5.2 - Distributed controller logic 1706 ForCES was built to distribute the controller logic to 1707 automonomous network elements that operate either as ForCES 1708 controlled or as integrated hybrid controller. 1710 OFS has created distributed logic per switch, but considers 1711 grouping of these switches outside the OFS specifications. The 1712 Hybrid [OFS-1.2] provides use cases for Ships-in-the-Night and 1713 integrated. The Ships-in-the-Night provide per port allocation to 1714 either OFS or standard processing. The Integrated seeks to run 1715 both on a set of ports. 1717 5.3 - Hybrid controllers 1719 ForCES was built for the hybrid environment where routing and 1720 switching protocol. 1722 OFS is now entering the processing of standardizing for hybrid 1723 controllers [OFS-1.2]. 1725 6. Security Considerations 1727 No security considerations. 1729 This is an informational comparison used to inform clarify ForCES 1730 work. 1732 7. IANA Considerations 1734 No IANA considerations. 1736 8. Conclusions 1738 Both ForCES and OpenFlow follow the basic idea of separations of 1739 forwarding plane and control plane in network elements. Both are 1740 capable of operating for centralized control, distributed control, 1741 and hybrid control. 1743 [OFS-1.1] Flow Table Logic with the instructions and Group Tables 1744 is the major difference between the ForCES RFCs. As this paper 1745 has shown, the full ramifications of this difference need to be 1746 considered in terms of differences in capability of 1747 implementation. The author welcomes any additional implementation 1748 experience. 1750 [OFS-1.0][OFS-1.1] lacks a forwarding model, a standardized LFB 1751 library and the concepts of FE-CE associations (FE-Manger, CE- 1752 Manager, pre/post association phase). It appears the OpenFlow 1753 work is starting to invent the equivalent of existing ForCES work 1754 as OpenFlow work. The guide of this reinventing seems to be the 1755 Google code snippets passed to the OpenFlow Forum as examples of 1756 "running code" to provide rough consensus. 1758 9. References 1760 9.1. Normative References 1762 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1763 Requirement Levels", BCP 14, RFC 2119, March 1997. 1765 [RFC5810] Doria, A., Hadi Salim, J., Haas, R., Khosravi, H., Wang, 1766 W., Dong, L., Gopal, R., and J. Halpern, "Forwarding 1767 and Control Element Separation (ForCES) Protocol 1768 Specification", RFC 5810, March 2010. 1770 [RFC5812] Halpern, J. and J. Hadi Salim, "Forwarding and Control 1771 Element Separation (ForCES) Forwarding Element Model", 1772 RFC 5812, March 2010. 1774 [RFC5811] Hadi Salim, J. and K. Ogawa, "SCTP-Based Transport 1775 Mapping Layer (TML) for the Forwarding and Control 1776 Element Separation (ForCES) Protocol", RFC 5811, March 1777 2010 1779 9.2. Informative References 1781 [McKeown2008] 1782 McKeown, N., Anderson, T., Balakrishnan, H., et al, 1783 "Openflow: enabling innovation in campus networks", ACM 1784 SIGCOMM Computer Communication Review. 2008, 38(2):69- 1785 74. 1787 [OFS-1.0.0] 1789 OpenFlow Switch Specification - Version 1.0.0 (Wire 1790 Protocol 0x01), December 31, 2009. 1792 [OFS-1.0.1-rc3] OpenFlow Switch Errata - Version 1.0.1-r3, June 1793 12, 2012. 1795 [OFS-1.1.0] 1796 OpenFlow Switch Specification Version 1.1.0 (Wire 1797 Protocol 0x02). February 2011. 1798 (http://www.openflow.org/documents/openflow-spec- 1799 v1.1.0.pdf) 1801 [OpenFlow-1.2] Open Flow 1.2 - Hybrid Switch (Terminology, Use 1802 cases, etc), April-May notes, work-in-progress. 1804 [OFS-1.3-rc4[ OpenFlow Switch Specification 1.3 (version 1.3-rc4) 1805 (Wire Protocol 0x04) April4, 2012. [OpenFlow members 1806 only] 1808 [OFS-1.1.0] 1809 OpenFlow Switch Specification Version 1.1.0 (Wire 1810 Protocol 0x02). February 2011. 1811 (http://www.openflow.org/documents/openflow-spec- 1812 v1.1.0.pdf) 1814 [OFC-1.0] OpenFlow Configuration and Management Protocol OF- 1815 CONFIG 1.0 (January 15, 2012). 1817 [RFC3654] Khosravi, H. and T. Anderson, "Requirements for 1818 Separation of IP Control and Forwarding", RFC 3654, 1819 November 2003. 1821 [RFC3746] Yang, L., Dantu, R., Anderson, T., and R. Gopal, 1822 "Forwarding and Control Element Separation (ForCES) 1823 Framework", RFC 3746, April 2004. 1825 [LFB-Lib] Wang W., Haleplidis E., Ogawa K., Li C., J. Halpern, 1826 "ForCES Logical Function Block (LFB) Library", draft- 1827 ietf-forces-lfb-lib-06, Work in Progress. 1829 10. Acknowledgments 1831 The author would like to thank Tina Tsou, Zhiliang Wang,Jing 1832 Huang, Xingang Shi, and Xia Yin. Their earlier draft comparing 1833 the FORCES and ONF Technology inspired this draft providing an 1834 opposing viewpoint. It is said that "iron sharpens iron" so that 1835 in our debates we sharpen our understandings. 1837 The author also acknowledges Edward Crabbe's succinct comments 1838 which also inspired the in-depth comparison found in this draft. 1839 I only hope that this "wordy" draft proves worth of his pithy and 1840 concise insights. 1842 Authors' Addresses 1844 Susan Hares 1845 Huawei Technologies (USA) 1846 2330 Central Expressway 1847 Santa Clara, CA 95050 1848 USA 1850 Email: Susan Hares Susan.Hares@huawei.com