idnits 2.17.1 draft-irtf-sdnrg-layer-terminology-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 22, 2014) is 3472 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-15) exists of draft-ietf-i2rs-architecture-05 == Outdated reference: A later version (-11) exists of draft-ietf-i2rs-problem-statement-04 == Outdated reference: A later version (-17) exists of draft-ietf-i2rs-rib-info-model-03 == Outdated reference: A later version (-21) exists of draft-ietf-pce-stateful-pce-09 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 SDNRG E. Haleplidis, Ed. 3 Internet-Draft University of Patras 4 Intended status: Informational K. Pentikousis, Ed. 5 Expires: April 25, 2015 EICT 6 S. Denazis 7 University of Patras 8 J. Hadi Salim 9 Mojatatu Networks 10 D. Meyer 11 Brocade 12 O. Koufopavlou 13 University of Patras 14 October 22, 2014 16 SDN Layers and Architecture Terminology 17 draft-irtf-sdnrg-layer-terminology-04 19 Abstract 21 Software-Defined Networking (SDN) refers to a new approach for 22 network programmability, that is, the capacity to initialize, 23 control, change, and manage network behavior dynamically via open 24 interfaces. SDN emphasizes the role of software in running networks 25 through the introduction of an abstraction for the data forwarding 26 plane and, by doing so, separates it from the control plane. This 27 separation allows faster innovation cycles at both planes as 28 experience has already shown. However, there is increasing confusion 29 as to what exactly SDN is, what is the layer structure in an SDN 30 architecture and how do layers interface with each other. This 31 document, a product of the IRTF Software-Defined Networking Research 32 Group (SDNRG), addresses these questions and provides a concise 33 reference for the SDN research community based on relevant peer- 34 reviewed literature, the RFC series, and relevant documents by other 35 standards organizations. 37 Status of This Memo 39 This Internet-Draft is submitted in full conformance with the 40 provisions of BCP 78 and BCP 79. 42 Internet-Drafts are working documents of the Internet Engineering 43 Task Force (IETF). Note that other groups may also distribute 44 working documents as Internet-Drafts. The list of current Internet- 45 Drafts is at http://datatracker.ietf.org/drafts/current/. 47 Internet-Drafts are draft documents valid for a maximum of six months 48 and may be updated, replaced, or obsoleted by other documents at any 49 time. It is inappropriate to use Internet-Drafts as reference 50 material or to cite them other than as "work in progress." 52 This Internet-Draft will expire on April 25, 2015. 54 Copyright Notice 56 Copyright (c) 2014 IETF Trust and the persons identified as the 57 document authors. All rights reserved. 59 This document is subject to BCP 78 and the IETF Trust's Legal 60 Provisions Relating to IETF Documents 61 (http://trustee.ietf.org/license-info) in effect on the date of 62 publication of this document. Please review these documents 63 carefully, as they describe your rights and restrictions with respect 64 to this document. Code Components extracted from this document must 65 include Simplified BSD License text as described in Section 4.e of 66 the Trust Legal Provisions and are provided without warranty as 67 described in the Simplified BSD License. 69 Table of Contents 71 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 72 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 73 3. SDN Layers and Architecture . . . . . . . . . . . . . . . . . 6 74 3.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . 7 75 3.2. Network Devices . . . . . . . . . . . . . . . . . . . . . 11 76 3.3. Control Plane . . . . . . . . . . . . . . . . . . . . . . 12 77 3.4. Management Plane . . . . . . . . . . . . . . . . . . . . 13 78 3.5. Control and Management Plane Discussion . . . . . . . . . 14 79 3.5.1. Timescale . . . . . . . . . . . . . . . . . . . . . . 15 80 3.5.2. Persistence . . . . . . . . . . . . . . . . . . . . . 15 81 3.5.3. Locality . . . . . . . . . . . . . . . . . . . . . . 15 82 3.5.4. CAP Theorem Insights . . . . . . . . . . . . . . . . 15 83 3.6. Network Services Abstraction Layer . . . . . . . . . . . 16 84 3.7. Application Plane . . . . . . . . . . . . . . . . . . . . 17 85 4. SDN Model View . . . . . . . . . . . . . . . . . . . . . . . 18 86 4.1. ForCES . . . . . . . . . . . . . . . . . . . . . . . . . 18 87 4.2. NETCONF/YANG . . . . . . . . . . . . . . . . . . . . . . 19 88 4.3. OpenFlow . . . . . . . . . . . . . . . . . . . . . . . . 19 89 4.4. Interface to the Routing System . . . . . . . . . . . . . 20 90 4.5. SNMP . . . . . . . . . . . . . . . . . . . . . . . . . . 21 91 4.6. PCEP . . . . . . . . . . . . . . . . . . . . . . . . . . 21 92 4.7. BFD . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 93 5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 94 6. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 23 95 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 23 96 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24 97 9. Security Considerations . . . . . . . . . . . . . . . . . . . 24 98 10. Informative References . . . . . . . . . . . . . . . . . . . 24 99 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 32 101 1. Introduction 103 Software-Defined Networking (SDN) is a term of the programmable 104 networks paradigm [PNSurvey99][OF08]. In short, SDN refers to the 105 ability of software applications to program individual network 106 devices dynamically and therefore control the behavior of the network 107 as a whole [NV09]. Boucadair and Jacquenet [RFC7149] point out that 108 SDN is a set of techniques used to facilitate the design, delivery 109 and operation of network services in a deterministic, dynamic, and 110 scalable manner. 112 A key element in SDN is the introduction of an abstraction between 113 the (traditional) forwarding and control planes in order to separate 114 them and provide applications with the means necessary to 115 programmatically control the network. The goal is to leverage this 116 separation, and the associated programmability, in order to reduce 117 complexity and enable faster innovation at both planes [A4D05]. 119 The historical evolution of the programmable networks R&D area is 120 reviewed in detail in [SDNHistory][SDNSurvey], starting with efforts 121 dating back to the 1980s. As Feamster et al. [SDNHistory] document, 122 many of the ideas, concepts and concerns are applicable to the latest 123 R&D in SDN, and SDN standardization we may add, and have been under 124 extensive investigation and discussion in the research community for 125 quite some time. For example, Rooney et al. [Tempest] discuss how 126 to allow third-party access to the network without jeopardizing 127 network integrity, or how to accommodate legacy networking solutions 128 in their (then new) programmable environment. Further, the concept 129 of separating the control and forwarding planes, which is prominent 130 in SDN, has been extensively discussed even prior to 1998 131 [Tempest][P1520], in SS7 networks [ITUSS7], Ipsilon Flow Switching 132 [RFC1953][RFC2297] and ATM [ITUATM]. 134 SDN research often focuses on varying aspects of programmability, and 135 we are frequently confronted with conflicting points of view 136 regarding what exactly SDN is. For instance, we find that for 137 various reasons (e.g. work focusing on one domain and therefore not 138 necessarily applicable as-is to other domains), certain well-accepted 139 definitions do not correlate well with each other. For example, both 140 OpenFlow [OpenFlow] and NETCONF [RFC6241] have been characterized as 141 SDN interfaces, but they refer to control and management 142 respectively. 144 This motivates us to consolidate the definitions of SDN in the 145 literature and correlate them with earlier work at the IETF and the 146 research community. Of particular interest is, for example, to 147 determine which layers comprise the SDN architecture and which 148 interfaces and their corresponding attributes are best suitable to be 149 used between them. As such, the aim of this document is not to 150 standardize any particular layer or interface but rather to provide a 151 concise reference which reflects current approaches regarding the SDN 152 layers architecture. We expect that this document would be useful to 153 upcoming work in SDNRG as well as future discussions within the SDN 154 community as a whole. 156 This document addresses the work item in the SDNRG charter entitled 157 "Survey of SDN approaches and Taxonomies", fostering better 158 understanding of prominent SDN technologies in a technology-impartial 159 and business-agnostic manner but does not constitute a new IETF 160 standard. It is meant as a common base for further discussion. As 161 such, we do not make any value statements nor discuss the 162 applicability of any of the frameworks examined in this draft for any 163 particular purpose. Instead, we document their characteristics and 164 attributes and classify them, thus providing a taxonomy. This 165 document does not intend to provide an exhaustive list of SDN 166 research issues; interested readers should consider reviewing 167 [SLTSDN] and [SDNACS]. In particular, Nunes et al. [SLTSDN] 168 overview SDN-related research topics, e.g. control partitioning, 169 which is related to the CAP theorem discussed in Section 3.5.4. 171 This document has been extensively reviewed, discussed, and commented 172 by the vast majority of SDNRG members, a community which certainly 173 exceeds 100 individuals. It is the consensus of SDNRG that this 174 document should be published in the IRTF Stream RFC Series [RFC5743]. 176 The remainder of this document is organized as follows. Section 2 177 explains the terminology used in this document. Section 3 introduces 178 a high-level overview of current SDN architecture abstractions. 179 Finally, Section 4 discusses how the SDN Layer Architecture relates 180 with prominent SDN-enabling technologies. 182 2. Terminology 184 This document uses the following terms: 186 Software-Defined Networking (SDN) - A programmable networks 187 approach that supports the separation of control and forwarding 188 planes via standardized interfaces. 190 Resource - A physical or virtual component available within a 191 system. Resources can be very simple or fine-grained, e.g. a port 192 or a queue, or complex, comprised of multiple resources, e.g. a 193 network device. 195 Network Device - A device that performs one or more network 196 operations related to packet manipulation and forwarding. This 197 reference model makes no distinction whether a network device is 198 physical or virtual. A device can also be considered as a 199 container for resources and can be a resource in itself. 201 Interface - A point of interaction between two entities. When the 202 entities are placed at different locations, the interface is 203 usually implemented through a network protocol. If the entities 204 are collocated in the same physical location the interface can be 205 implemented using a software application programming interface 206 (API), inter-process communication (IPC), or a network protocol. 208 Application (App) - An application in the context of SDN is a 209 piece of software that utilizes underlying services to perform a 210 function. Application operation can be parametrized, for example 211 by passing certain arguments at call time, but it is meant to be a 212 standalone piece of software: an App does not offer any interfaces 213 to other applications or services. 215 Service - A piece of software that performs one or more functions 216 and provides one or more APIs to applications or other services of 217 the same or different layers to make use of said functions and 218 returns one or more results. Services can be combined with other 219 services, or called in a certain serialized manner, to create a 220 new service. 222 Forwarding Plane (FP) - The collection of resources across all 223 network devices responsible for forwarding traffic. 225 Operational Plane (OP) - The collection of resources responsible 226 for managing the overall operation of individual network devices. 228 Control plane (CP) - The collection of functions responsible for 229 controlling one or more network devices. CP instructs network 230 devices with respect to how to process and forward packets. The 231 control plane interacts primarily with the forwarding plane and to 232 a lesser extent with the operational plane. 234 Management plane (MP) - The collection of functions responsible 235 for monitoring, configuring and maintaining one or more network 236 devices or parts of network devices. The management plane is 237 mostly related with the operational plane and less with the 238 forwarding plane. 240 Application Plane - The collection of applications and services 241 which program network behavior. 243 Device and resource Abstraction Layer (DAL) - The device's 244 resource abstraction layer based on one or more models. If it is 245 a physical device it may be referred to as the Hardware 246 Abstraction Layer (HAL). DAL provides a uniform point of 247 reference for the device's forwarding and operational plane 248 resources. 250 Control Abstraction Layer (CAL) - The control plane's abstraction 251 layer. CAL provides access to the control plane southbound 252 interface. 254 Management Abstraction Layer (MAL) - The management plane's 255 abstraction layer. MAL provides access to the management plane 256 southbound interface. 258 Network Services Abstraction Layer (NSAL) - Provides service 259 abstractions that can be used by applications and services. 261 3. SDN Layers and Architecture 263 Figure 1 summarizes in the form of a detailed high-level schematic 264 the SDN architecture abstractions. Note that in a particular 265 implementation planes can be collocated with other planes or can be 266 physically separated, as we discuss below. 268 SDN is based on the concept of separation between a controlled entity 269 and a controller entity. The controller manipulates the controlled 270 entity via an Interface. Interfaces, when local, are mostly API 271 calls through some library or system call. However, such interfaces 272 may be extended via some protocol definition, which may use local 273 inter-process communication (IPC) or a protocol that could also act 274 remotely; the protocol may be defined as an open standard or in a 275 proprietary manner. 277 Day [PiNA] explores the use of IPC as the mainstay for the definition 278 of recursive network architectures with varying degrees of scope and 279 range of operation. RINA [RINA] outlines a recursive network 280 architecture based on IPC which capitalizes on repeating patterns and 281 structures. This document does not propose a new architecture--we 282 simply document previous work through a taxonomy. Although recursion 283 is out of scope for this work, Figure 1 illustrates a hierarchical 284 model in which layers can be stacked on top of each other and 285 employed recursively as needed. 287 o--------------------------------o 288 | | 289 | +-------------+ +----------+ | 290 | | Application | | Service | | 291 | +-------------+ +----------+ | 292 | Application Plane | 293 o---------------Y----------------o 294 | 295 *-----------------------------Y---------------------------------* 296 | Network Services Abstraction Layer (NSAL) | 297 *------Y------------------------------------------------Y-------* 298 | | 299 | Service Interface | 300 | | 301 o------Y------------------o o---------------------Y------o 302 | | Control Plane | | Management Plane | | 303 | +----Y----+ +-----+ | | +-----+ +----Y----+ | 304 | | Service | | App | | | | App | | Service | | 305 | +----Y----+ +--Y--+ | | +--Y--+ +----Y----+ | 306 | | | | | | | | 307 | *----Y-----------Y----* | | *---Y---------------Y----* | 308 | | Control Abstraction | | | | Management Abstraction | | 309 | | Layer (CAL) | | | | Layer (MAL) | | 310 | *----------Y----------* | | *----------Y-------------* | 311 | | | | | | 312 o------------|------------o o------------|---------------o 313 | | 314 | CP | MP 315 | Southbound | Southbound 316 | Interface | Interface 317 | | 318 *------------Y---------------------------------Y----------------* 319 | Device and resource Abstraction Layer (DAL) | 320 *------------Y---------------------------------Y----------------* 321 | | | | 322 | o-------Y----------o +-----+ o--------Y----------o | 323 | | Forwarding Plane | | App | | Operational Plane | | 324 | o------------------o +-----+ o-------------------o | 325 | Network Device | 326 +---------------------------------------------------------------+ 328 Figure 1: SDN Layer Architecture 330 3.1. Overview 332 This document follows a network device centric approach: Control 333 mostly refers to the device packet handling capability, while 334 management typically refer to the overall device operation aspects. 336 We view a network device as a complex resource which contains and is 337 part of multiple resources similar to [DIOPR]. Resources can be 338 simple, single components of a network device, for example a port or 339 a queue of the device, and can also be aggregated into complex 340 resources, for example a network card or a complete network device. 342 The reader should keep in mind throughout this document that we make 343 no distinction between "physical" and "virtual" resources or 344 "hardware" and "software" realizations, as we do not delve into 345 implementation or performance aspects. In other words, a resource 346 can be implemented fully in hardware, fully in software, or any 347 hybrid combination in between. Further, we do not distinguish on 348 whether a resource is implemented as an overlay or as a part/ 349 component of some other device. In general, network device software 350 can run on so-called "bare metal" or on a virtualized substrate. 351 Finally, this document does not discuss how resources are allocated, 352 orchestrated, and released. Indeed, orchestration is out of scope 353 for this document. 355 SDN spans multiple planes as illustrated in Figure 1. Starting from 356 the bottom part of the figure and moving towards the upper part, we 357 identify the following planes: 359 Forwarding Plane - Responsible for handling packets in the 360 datapath based on the instructions received from the control 361 plane. Actions of the forwarding plane include, but are not 362 limited to, forwarding, dropping and changing packets. The 363 forwarding plane is usually the termination point for control 364 plane services and applications. The forwarding plane can contain 365 forwarding resources such as classifiers. The forwarding plane is 366 also widely referred to as the "data plane" or the "data path". 368 Operational Plane - Responsible for managing the operational state 369 of the network device, e.g. whether the device is active or 370 inactive, the number of ports available, the status of each port, 371 and so on. The operational plane is usually the termination point 372 for management plane services and applications. The operational 373 plane relates to network device resources such as ports, memory, 374 and so on. We note that some participants of the IRTF SDNRG have 375 a different opinion in regards to the definition of the 376 operational plane. That is, one can argue that the operational 377 plane does not constitute a "plane" per se, but it is in practice 378 an amalgamation of functions on the forwarding plane. For others, 379 however, a "plane" allows to distinguish between different areas 380 of operations and therefore the operational plane should be 381 included as a "plane" in Figure 1. We have adopted this latter 382 view in this document. 384 Control Plane - Responsible for taking decisions on how packets 385 should be forwarded by one or more network devices and pushing 386 such decisions down to the network devices for execution. The 387 control plane usually focuses mostly on the forwarding plane and 388 less on the operational plane of the device. The control plane 389 may be interested in operational plane information which could 390 include, for instance, the current state of a particular port or 391 its capabilities. The control plane's main job is to fine-tune 392 the forwarding tables that reside in the forwarding plane, based 393 on the network topology or external service requests. 395 Management Plane - Responsible for monitoring, configuring and 396 maintaining network devices, e.g. taking decisions regarding the 397 state of a network device. The management plane usually focuses 398 mostly on the operational plane of the device and less on the 399 forwarding plane. The management plane may be used to configure 400 the forwarding plane, but it does so infrequently and through a 401 more wholesale approach than the control plane. For instance, the 402 management plane may set up all or part of the forwarding rules at 403 once, although such action would be expected to be taken 404 sparingly. 406 Application Plane - The plane where applications and services that 407 define network behavior reside. Applications that directly (or 408 primarily) support the operation of the forwarding plane (such as 409 routing processes within the control plane) are not considered 410 part of the application plane. Note that applications may be 411 implemented in a modular and distributed fashion and, therefore, 412 can often span multiple planes in Figure 1. 414 [RFC7276] has defined the data, control and management plane in terms 415 of Operations, Administration, and Maintenance (OAM). This document 416 attempts to broaden the terms defined in [RFC7276] in order to 417 reflect all aspects of an SDN architecture. 419 All planes mentioned above are connected via interfaces (as indicated 420 with "Y" in Figure 1. An interface may take multiple roles depending 421 on whether the connected planes reside on the same (physical or 422 virtual) device. If the respective planes are designed so that they 423 do not have to reside in the same device, then the interface can only 424 take the form of a protocol. If the planes are co-located on the 425 same device, then the interface could be implemented via an open/ 426 proprietary protocol, an open/proprietary software inter-process 427 communication API, or operating system kernel system calls. 429 Applications, i.e. software programs that perform specific 430 computations that consume services without providing access to other 431 applications, can be implemented natively inside a plane or can span 432 multiple planes. For instance, applications or services can span 433 both the control and management plane and, thus, be able to use both 434 the Control Plane Southbound Interface (CPSI) and Management Plane 435 Southbound Interface (MPSI), although this is only implicitly 436 illustrated in Figure 1. An example of such a case would be an 437 application that uses both [OpenFlow] and [OF-CONFIG]. 439 Services, i.e. software programs that provide APIs to other 440 applications or services, can also be natively implemented in 441 specific planes. Services that span multiple planes belong to the 442 application plane as well. 444 While not shown explicitly in Figure 1, services, applications and 445 entire planes, can be placed in a recursive manner thus providing 446 overlay semantics to the model. For example, application plane 447 services can provide through NSAL services to other applications or 448 services. Additional examples include virtual resources that are 449 realized on top of a physical resources and hierarchical control 450 plane controllers [KANDOO]. 452 Note that the focus in this document is, of course, on the north/ 453 south communication between entities in different planes. But this, 454 clearly, does not exclude entity communication within any one plane. 456 It must be noted, however, that in Figure 1 we present an abstract 457 view of the various planes, which is devoid of implementation 458 details. Many implementations in the past have opted for placing the 459 management plane on top of the control plane. This can be 460 interpreted as having the control plane acting as a service to the 461 management plane. Further, in many networks especially in Internet 462 routers and Ethernet switches, the control plane has been usually 463 implemented as tightly coupled with the network device. When taken 464 as a whole, the control plane has been distributed network-wide. On 465 the other hand, the management plane has been traditionally 466 centralized and has been responsible for managing the control plane 467 and the devices. However, with the adoption of SDN principles, this 468 distinction is no longer so clear-cut. 470 Additionally, this document considers four abstraction layers: 472 The Device and resource Abstraction Layer (DAL) abstracts the 473 device's forwarding and operational plane resources to the control 474 and management plane. Variations of DAL may abstract both planes 475 or either of the two and may abstract any plane of the device to 476 either the control or management plane. 478 The Control Abstraction Layer (CAL) abstracts the CP southbound 479 interface and the DAL from the applications and services of the 480 control plane. 482 The Management Abstraction Layer (MAL) abstracts the MP southbound 483 interface and the DAL from the applications and services of the 484 management plane. 486 The Network Services Abstraction Layer (NSAL) provides service 487 abstractions for use by applications and other services. 489 At the time of this writing, SDN-related activities have begun in 490 other SDOs. For example, at the ITU work on architectural [ITUSG13] 491 and signaling requirements and protocols [ITUSG11] has commenced, but 492 the respective study groups have yet to publish their documents with 493 the exception of [ITUY3300]. The views presented in [ITUY3300] as 494 well as [ONFArch] are well aligned with this document. 496 3.2. Network Devices 498 A Network Device is an entity that receives packets on its ports and 499 performs one or more network functions on them. For example, the 500 network device could forward a received packet, drop it, alter the 501 packet header (or payload) and forward the packet, and so on. A 502 Network Device is an aggregation of multiple resources such as ports, 503 CPU, memory and queues. Resources are either simple or can be 504 aggregated to form complex resources that can be viewed as one 505 resource. The Network Device is in itself a complex resource. 506 Examples of Network Devices include switches and routers. Additional 507 examples include network elements that may operate at a layer above 508 IP, such as firewalls, load balancers and video transcoders; or below 509 IP, such as Layer 2 switches, optical or microwave network elements. 511 Network devices can be implemented in hardware or software and can be 512 either physical or virtual. As has already been mentioned before, 513 this document makes no such distinction. Each network device has a 514 presence in a Forwarding Plane and an Operational Plane. 516 The Forwarding Plane, commonly referred to as the "data path", is 517 responsible for handling and forwarding packets. The Forwarding 518 Plane provides switching, routing, packet transformation and 519 filtering functions. Resources of the forwarding plane include but 520 are not limited to filters, meters, markers and classifiers. 522 The Operational Plane is responsible for the operational state of the 523 network device, for instance, with respect to status of network ports 524 and interfaces. Operational plane resources include, but are not 525 limited to, memory, CPU, ports, interfaces and queues. 527 The Forwarding and the Operational Planes are exposed via the Device 528 and resource Abstraction Layer (DAL), which may be expressed by one 529 or more abstraction models. Examples of Forwarding Plane abstraction 530 models are ForCES [RFC5812], OpenFlow [OpenFlow], YANG model 531 [RFC6020], and SNMP MIBs [RFC3418]. Examples of the Operational 532 Plane abstraction model include the ForCES model [RFC5812], the YANG 533 model [RFC6020], and SNMP MIBs [RFC3418]. 535 Note that applications can also reside in a network device. Examples 536 of such applications include event monitoring, and handling 537 (offloading) topology discovery or ARP [RFC0826] in the device itself 538 instead of forwarding such traffic to the control plane. 540 3.3. Control Plane 542 The control plane is usually distributed and is responsible mainly 543 for the configuration of the forwarding plane using a Control Plane 544 Southbound Interface (CPSI) with DAL as a point of reference. CP is 545 responsible for instructing FP about how to handle network packets. 547 Communication between control plane entities, colloquially referred 548 to as the "east-west" interface, is usually implemented through 549 gateway protocols such as BGP [RFC4271] or other protocols such as 550 PCEP [RFC5440]. These corresponding protocol messages are usually 551 exchanged in-band and subsequently redirected by the forwarding plane 552 to the control plane for further processing. Examples in this 553 category include [RCP], [SoftRouter] and [RouteFlow]. 555 Control Plane functionalities usually include: 557 o Topology discovery and maintenance 559 o Packet route selection and instantiation 561 o Path failover mechanisms 563 The CPSI is usually defined with the following characteristics: 565 o time-critical interface which requires low latency and sometimes 566 high bandwidth in order to perform many operations in short order 568 o oriented towards wire efficiency and device representation instead 569 of human readability 571 Examples include fast- and high-frequency of flow or table updates, 572 high throughput and robustness for packet handling and events. 574 CPSI can be implemented using a protocol, an API or even interprocess 575 communication. If the Control Plane and the Network Device are not 576 collocated, then this interface is certainly a protocol. Examples of 577 CPSIs are ForCES [RFC5810] and the Openflow protocol [OpenFlow]. 579 The Control Abstraction Layer (CAL) provides access to control 580 applications and services to various CPSIs. The Control Plane may 581 support more than one CPSIs. 583 Control applications can use CAL to control a network device without 584 providing any service to upper layers. Examples include applications 585 that perform control functions, such as OSPF, IS-IS, and BGP. 587 Control Plane service examples include a virtual private LAN service, 588 service tunnels, topology services, etc. 590 3.4. Management Plane 592 The Management Plane is usually centralized and aims to ensure that 593 the network as a whole is running optimally by communicating with the 594 network devices' Operational Plane using a Management Plane 595 Southbound Interface (MPSI) with DAL as a point of reference. 597 Management plane functionalities are typically initiated, based on an 598 overall network view, and traditionally have been human-centric. 599 However, lately algorithms are replacing most human intervention. 600 Management plane functionalities [FCAPS] typically include: 602 o Fault and Monitoring management 604 o Configuration management 606 In addition, management plane functionalities may also include 607 entities such as orchestrators, Virtual Function Managers (VNF 608 manager) and Virtualised Infrastructure Managers, as described in 609 [NFVArch]. Such entities can use management interfaces to 610 operational plane resources to request and provision resources for 611 virtual functions, as well as instruct the instantiation of virtual 612 forwarding functions on top of physical forwarding functions. The 613 possibility of a common abstraction model for both SDN and NFV is 614 explored in [SDNNFV]. Note, however, that these are only examples of 615 applications and services in the management plane and not formal 616 definitions of entities in this document. As has been noted above, 617 orchestration and therefore the definition of any associated entities 618 is out of scope for this document. 620 The MPSI, in contrast to the CPSI, is usually not a time-critical 621 interface and does not share the CPSI requirements. 623 MPSI is typically closer to human interaction than CPSI (cf. 624 [RFC3535]) and, therefore, MPSI usually has the following 625 characteristics: 627 o It is oriented more towards usability, with optimal wire 628 performance being a secondary concern 630 o Messages tend to be less frequent than in the CPSI 632 As an example of usability versus performance, we refer to the 633 consensus of the 2002 IAB Workshop [RFC3535], such as that the key 634 requirement for a network management technology is ease of use and 635 not performance. As per [RFC6632], textual configuration files 636 should be able to contain international characters. Human-readable 637 strings should utilize UTF-8, and protocol elements should be in 638 case-insensitive ASCII which require more processing capabilities to 639 parse. 641 MPSI can range from a protocol, to an API or even interprocess 642 communication. If the Management Plane is not embedded in the 643 network device, the MPSI is certainly a protocol. Examples of MPSIs 644 are ForCES [RFC5810], NETCONF [RFC6241], IPFIX [RFC7011], SYSLOG 645 [RFC5424], OVSDB [RFC7047] and SNMP [RFC3411]. 647 The Management Abstraction Layer (MAL) provides access to management 648 applications and services to various MPSIs. The Management Plane may 649 support more than one MPSI. 651 Management Applications can use MAL to manage the network device 652 without providing any service to upper layers. Examples of 653 management applications include network monitoring, fault detection 654 and recovery applications. 656 Management Plane Services provide access to other services or 657 applications above the Management Plane. 659 3.5. Control and Management Plane Discussion 661 The definition of a clear distinction between "control" and 662 "management" in the context of SDN received significant community 663 attention during the preparation of this document. We observed that 664 the role of the management plane has been earlier largely ignored or 665 specified as out-of-scope for the SDN ecosystem. In the remainder of 666 this subsection we summarize the characteristics that differentiate 667 the two planes in order to have a clear understanding of the 668 mechanics, capabilities and needs of each respective interface. 670 3.5.1. Timescale 672 A point has been raised regarding the reference timescales for the 673 control and management planes. That is, how fast is the respective 674 plane required to react, or needs to manipulate, the forwarding or 675 operational plane of the device. In general, the control plane needs 676 to send updates "often", which translates roughly to a range of 677 milliseconds; that requires high-bandwidth and low-latency links. In 678 contrast, the management plane reacts generally at longer time 679 frames, i.e. minutes, hours or even days, and thus wire-efficiency is 680 not always a critical concern. A good example of this is the case of 681 changing the configuration state of the device. 683 3.5.2. Persistence 685 Another distinction between the control and management planes relates 686 to state persistence. A state is considered ephemeral if it has a 687 very limited lifespan. A good example is determining routing, which 688 is usually associated with the control plane. On the other hand, a 689 persistent state has an extended lifespan which may range from hours 690 to days and months and is usually associated with the management 691 plane. Persistent state is also usually associated with data store 692 of the state. 694 3.5.3. Locality 696 As mentioned earlier, traditionally the control plane has been 697 executed locally on the network device and is distributed in nature 698 whilst the management plane is usually executed in a centralized 699 manner, remotely from the device. However, with the advent of SDN 700 centralizing, or "locally centralizing" the controller tends to 701 muddle the distinction of the control and management plane based on 702 locality. 704 3.5.4. CAP Theorem Insights 706 The CAP theorem views a distributed computing system as composed of 707 multiple computational resources (i.e., CPU, memory, storage) that 708 are connected via a communications network and together perform a 709 task. The theorem, or conjecture by some, identifies three 710 characteristics of distributed systems that are universally 711 desirable: 713 o Consistency, meaning that the system responds identically to a 714 query no matter which node receives the request (or does not 715 respond at all) 717 o Availability, i.e. that the system always responds to a request 718 (although the response may not be consistent or correct) 720 o Partition tolerance, namely that the system continues to function 721 even when specific nodes or the communications network fail. 723 In 2000 Eric Brewer [CAPBR] conjectured that a distributed system can 724 satisfy any two of these guarantees at the same time, but not all 725 three. This conjecture was later proven by Gilbert and Lynch [CAPGL] 726 and is now usually referred to as the CAP theorem [CAPFN]. 728 Forwarding a packet through a network correctly is a computational 729 problem. One of the major abstractions that SDN posits is that all 730 network elements are computational resources that perform the simple 731 computational task of inspecting fields in an incoming packet and 732 deciding how to forward it. Since the task of forwarding a packet 733 from network ingress to network egress is obviously carried out by a 734 large number of forwarding elements, the network of forwarding 735 devices is a distributed computational system. Hence, the CAP 736 theorem applies to forwarding of packets. 738 In the context of the CAP theorem, if one considers partition 739 tolerance of paramount importance, traditional control plane 740 operations are usually local and fast (available), while management 741 plane operations are usually centralized (consistent) and may be 742 slow. 744 The CAP theorem also provides insights into SDN architectures. For 745 example a centralized SDN controller acts as a consistent global 746 database, and specific SDN mechanisms ensure that a packet entering 747 the network is handled consistently by all SDN switches. The issue 748 of tolerance to loss of connectivity to the controller is not 749 addressed by the basic SDN model. When an SDN switch cannot reach 750 its controller, the flow will be unavailable until the connection is 751 restored. The use of multiple non-collocated SDN controllers has 752 been proposed (e.g., by configuring the SDN switch with a list of 753 controllers); this may improve partition tolerance, but at the cost 754 of loss of absolute consistency. Panda et al. [CAPFN] provide a 755 first exploration of how the CAP theorem applies to SDN. 757 3.6. Network Services Abstraction Layer 759 The Network Services Abstraction Layer (NSAL) provides access from 760 services of the control, management and application planes to other 761 services and applications. We note that the term SAL is overloaded, 762 as it is often used in several contexts ranging from system design to 763 service-oriented architectures, therefore we explicitly add "Network" 764 to the title of this layer to emphasize that this term relates to 765 Figure 1 and we map it accordingly in Section 4 to prominent SDN 766 approaches. 768 Service Interfaces can take many forms pertaining to their specific 769 requirements. Examples of service interfaces include but are not 770 limited to, RESTful APIs, open protocols such as NETCONF, inter- 771 process communication, CORBA [CORBA] interfaces, and so on. The two 772 leading approaches for service interfaces are RESTful interfaces and 773 RPC interfaces. Both follow a client-server architecture and use XML 774 or JSON to pass messages but each has some slightly different 775 characteristics. 777 RESTful interfaces, designed according to the representational state 778 transfer design paradigm [REST], have the following characteristics: 780 o Resource identification - individual resources are identified 781 using a resource identifier, for example a URI. 783 o Manipulation of resources through representations - Resources are 784 represented in a format like JSON, XML or HTML. 786 o Self-descriptive messages - Each message has enough information to 787 describe how the message is to be processed. 789 o Hypermedia as the engine of application state - a client needs no 790 prior knowledge of how to interact with a server, not through a 791 fixed interface. 793 Remote procedure calls (RPC), e.g. [RFC5531], XML-RPC and the like, 794 have the following characteristics: 796 o Individual procedures are identified using an identifier 798 o A client needs to know the procedure name and the associated 799 parameters 801 3.7. Application Plane 803 Applications and services that use services from the control and/or 804 management plane form the Application Plane. 806 Additionally, services residing in the Application Plane may provide 807 services to other services and applications that reside in the 808 application plane via the service interface. 810 Examples of applications include network topology discovery, network 811 provisioning, path reservation, etc. 813 4. SDN Model View 815 We advocate that the SDN southbound interface should encompass both 816 CSPI and MPSI. 818 SDN controllers such as [NOX] and [Beacon] are a collection of 819 control plane applications and services that implement a CPSI, [NOX] 820 and [Beacon] both use OpenFlow, and provide a northbound interface 821 for applications. The SDN northbound interface for controllers is 822 implemented in the Network Services Abstraction Layer of Figure 1. 824 The above model can be used to describe in a concise manner all 825 prominent SDN-enabling technologies, as we explain in the following 826 subsections. 828 4.1. ForCES 830 The IETF-standardized Forwarding and Control Element Separation 831 (ForCES) framework [RFC3746] consists of one model and two protocols. 832 ForCES separates the Forwarding from the Control Plane via an open 833 interface, namely the ForCES protocol [RFC5810] which operates on 834 entities of the forwarding plane that have been modeled using the 835 ForCES model [RFC5812]. 837 The ForCES model [RFC5812] is based on the fact that a network 838 element is composed of numerous logically separate entities that 839 cooperate to provide a given functionality -such as routing or IP 840 switching- and yet appear as a normal integrated network element to 841 external entities and secondly with a protocol to transport 842 information. 844 ForCES models the Forwarding Plane using Logical Functional Blocks 845 (LFBs) which when connected in a graph compose the Forwarding Element 846 (FE). LFBs are described in an XML language, based on an XML schema. 848 LFB definitions include base and custom-defined datatypes; metadata 849 definitions; input and output ports; operational parameters or 850 components; capabilities and event definitions. 852 The ForCES model can be used to define LFBs from fine- to coarse- 853 grained as needed, irrespective of whether they are physical or 854 virtual. 856 The ForCES protocol is agnostic to the model and can be used to 857 monitor, configure and control any ForCES-modeled element. The 858 protocol has very simple commands: Set, Get and Del(ete). The ForCES 859 protocol has been designed for high throughput and fast updates. 861 With respect to Figure 1, the ForCES model [RFC5812] is suitable for 862 the DAL, both for the Operational and the Forwarding Plane, using 863 LFBs. The ForCES protocol [RFC5810] has been designed and is 864 suitable for the CPSI, although it could also be utilized for the 865 MPSI. 867 4.2. NETCONF/YANG 869 The Network Configuration Protocol (NETCONF [RFC6241]), is an IETF- 870 standardized network management protocol [RFC6632]. NETCONF provides 871 mechanisms to install, manipulate, and delete the configuration of 872 network devices. 874 NETCONF protocol operations are realized as remote procedure calls 875 (RPCs). The NETCONF protocol uses an Extensible Markup Language 876 (XML) based data encoding for the configuration data as well as the 877 protocol messages. Recent studies, such as [ESNet] and [PENet], have 878 shown that NETCONF performs better than SNMP [RFC3411]. 880 Additionally, the YANG data modeling language [RFC6020] has been 881 developed for specifying NETCONF data models and protocol operations. 882 YANG is a data modeling language used to model configuration and 883 state data manipulated by NETCONF, NETCONF remote procedure calls, 884 and NETCONF notifications. 886 YANG models the hierarchical organization of data as a tree, in which 887 each node has either a value or a set of child nodes. Additionally, 888 YANG structures data models into modules and submodules allowing 889 reusability and augmentation. YANG models can describe constraints 890 to be enforced on the data. Additionally YANG has a set of base 891 datatype and allows custom defined datatypes as well. 893 YANG allows the definition of NETCONF RPCs allowing the protocol to 894 have an extensible number of commands. For RPC definition, the 895 operations names, input parameters, and output parameters are defined 896 using YANG data definition statements. 898 With respect to Figure 1, the YANG model [RFC6020] is suitable for 899 specifying DAL for the forwarding and operational plane. NETCONF 900 [RFC6241] is suitable for the MPSI. NETCONF is a management protocol 901 [RFC6241] which was not (originally) designed for fast CP updates, 902 and it might not be suitable for addressing the requirements of CPSI. 904 4.3. OpenFlow 906 OpenFlow is a framework originally developed at Stanford University, 907 and currently under active standards development [OpenFlow] through 908 the Open Networking Foundation (ONF). Initially, the goal was to 909 provide a way for researchers to run experimental protocols in a 910 production network [OFSIGC]. OpenFlow has undergone many revisions 911 and additional revisions are likely. The following description 912 reflects version 1.4 [OpenFlow]. In short, OpenFlow defines a 913 protocol through which a logically centralized controller can control 914 an OpenFlow switch. Each OpenFlow-compliant switch maintains one or 915 more flow tables which are used to perform packet lookups. Distinct 916 actions are to be taken regarding packet lookup and forwarding. A 917 group table and an OpenFlow channel to external controllers are also 918 part of the switch specification. 920 With respect to Figure 1, the Openflow switch specifications 921 [OpenFlow] define a DAL for the Forwarding Plane as well as for CPSI. 922 The OF-CONFIG protocol [OF-CONFIG] based on the YANG model [RFC6020], 923 provides a DAL for the Forwarding and Operational Plane of an 924 OpenFlow switch, and specifies NETCONF [RFC6241] as the MPSI. OF- 925 CONFIG overlaps with the OpenFlow DAL, but with NETCONF [RFC6241] as 926 the transport protocol it shares the limitations described in the 927 previous section. 929 4.4. Interface to the Routing System 931 Interface to the Routing System (I2RS) provides a standard interface 932 to the routing system for real-time or event-driven interaction 933 through a collection of protocol-based control or management 934 interfaces. Essentially, one of the main goals of I2RS, is to make 935 the routing information base (RIB) programmable thus enabling new 936 kinds of network provisioning and operation. 938 I2RS does not initially intend to create new interfaces, but rather 939 leverage or extend existing ones and define informational models for 940 the routing system. For example, the latest I2RS problem statement 941 [I-D.ietf-i2rs-problem-statement] discusses previously-defined IETF 942 protocols such as ForCES [RFC5810], NETCONF [RFC6241], and SNMP 943 [RFC3417]. Regarding the definition of informational and data 944 models, the I2RS working group has opted to use the YANG [RFC6020] 945 modelling language. 947 Currently the I2RS working group is developing an Information Model 948 [I-D.ietf-i2rs-rib-info-model] in regards to the Network Services 949 Abstraction Layer for the I2RS agent. 951 With respect to Figure 1, the I2RS architecture 952 [I-D.ietf-i2rs-architecture] encompasses the Control and Application 953 Planes and uses any CPSI and DAL that is available, whether that may 954 be ForCES [RFC5810], OpenFlow [OpenFlow] or another interface. In 955 addition, the I2RS agent is a Control Plane Service. All services or 956 applications on top of that belong to either the Control, Management 957 or the Application plane. In the I2RS documents, management access 958 to the agent may be provided by management protocols like SNMP and 959 NETCONF. The I2RS protocol may also be mapped to the Service 960 Interface as it will provide access even to other than control 961 applications. 963 4.5. SNMP 965 The Simple Network Management Protocol (SNMP) is an IETF-standardized 966 management protocol and is currently at its third revision (SNMPv3) 967 [RFC3417][RFC3412][RFC3414]. It consists of a set of standards for 968 network management, including an application layer protocol, a 969 database schema, and a set of data objects. SNMP exposes management 970 data (managed objects) in the form of variables on the managed 971 systems, which describe the system configuration. These variables 972 can then be queried and set by managing applications. 974 SNMP uses an extensible design for describing data, defined by 975 management information bases (MIBs). MIBs describe the structure of 976 the management data of a device subsystem. MIBs use a hierarchical 977 namespace containing object identifiers (OID). Each OID identifies a 978 variable that can be read or set via SNMP. MIBs use the notation 979 defined by Structure of Management Information Version 2 [RFC2578] 981 An early example of SNMP in the context of SDN is discussed in 982 [Peregrine]. 984 With respect to Figure 1, SNMP MIBs can be used to describe DAL for 985 the Forwarding and Operational Plane. Similar to YANG, SNMP MIBs are 986 able to describe DAL for the Forwarding Plane. SNMP, similar to 987 NETCONF, is suited for the MPSI. 989 4.6. PCEP 991 The Path Computation Element (PCE) [RFC4655] architecture defines an 992 entity capable of computing paths for a single service or a set of 993 services. A PCE might be a network node, network management station, 994 or dedicated computational platform that is resource-aware and has 995 the ability to consider multiple constraints for a variety of path 996 computation problems and switching technologies. The PCE 997 Communication Protocol (PCEP) [RFC5440] is an IETF protocol for 998 communication between a Path Computation Client (PCC) and a PCE, or 999 between multiple PCEs. 1001 The PCE represents a vision of networks that separates path 1002 computation for services, the signaling of end-to-end connections and 1003 actual packet forwarding. The definition of online and offline path 1004 computation is dependent on the reachability of the PCE from network 1005 and NMS nodes, and the type of optimization request which may 1006 significantly impact the optimization response time from the PCE to 1007 the PCC. 1009 The PCEP messaging mechanism facilitates the specification of 1010 computation endpoints (source and destination node addresses) and 1011 objective functions (requested algorithm and optimization criteria), 1012 and the associated constraints such as traffic parameters (e.g. 1013 requested bandwidth), the switching capability, and encoding type. 1015 With respect to Figure 1, PCE is a control plane service that 1016 provides services for control plane applications. PCEP may be used 1017 as an east-west interface between PCEs which may act as domain 1018 control entities (services and applications). The PCE working group 1019 is specifying extensions [I-D.ietf-pce-stateful-pce], which allow an 1020 active PCE to control, using PCEP, MPLS or GMPLS Label Switched Paths 1021 (LSP), thus making it applicable for the CPSI for MPLS and GMPLS 1022 switches. 1024 4.7. BFD 1026 Bidirectional Forwarding Detection (BFD) [RFC5880], is an IETF- 1027 standardized network protocol designed for detecting path failures 1028 between two forwarding elements, including physical interfaces, 1029 subinterfaces, data link(s), and to the extent possible the 1030 forwarding engines themselves, with potentially very low latency. 1031 BFD can provide low-overhead failure detection on any kind of path 1032 between systems, including direct physical links, virtual circuits, 1033 tunnels, MPLS LSPs, multihop routed paths, and unidirectional links 1034 where there exists a return path as well. It is often implemented in 1035 some component of the forwarding engine of a system, in cases where 1036 the forwarding and control engines are separated. 1038 With respect to Figure 1, a BFD agent can be implemneted as a control 1039 plane service or application that would use the CPSI towards the 1040 forwarding plane to send/receive BFD packets. However a BFD agent is 1041 usually implemented as an application on the device and use the 1042 forwarding plane to send/receive BFD packets and update the 1043 operational plane resources accordingly. Services and applications 1044 of control and management plane that monitor or has subscribed to 1045 changes of resources, learn these changes through their respective 1046 interface and will take the necessary action. 1048 5. Summary 1050 This document has been developed after a thorough and detailed 1051 analysis of related peer-reviewed literature, the RFC series, and 1052 documents produced by other relevant standards organizations. It has 1053 been reviewed publicly by the wider SDN community and we hope that it 1054 can serve as a handy tool for network researchers, engineers and 1055 practitioners in the years to come. 1057 We conclude this document with a brief summary of the SDN 1058 architecture layers terminology. In general, we consider a network 1059 element as a composition of resources. Each network element has a 1060 forwarding plane (FP), responsible for handling packets in the 1061 datapath, and an operational plane (OP), responsible for managing the 1062 operational state of the device. Resources in the network element 1063 are abstracted by the device and resource abstraction layer (DAL) to 1064 be controlled and managed by services or applications that belong to 1065 the control or management plane. The control plane (CP) is 1066 responsible for taking decisions on how packets should be forwarded. 1067 The management plane (MP) is responsible for monitoring, configuring 1068 and maintaining network devices. Service interfaces are abstracted 1069 by the network service abstraction layer (NSAL) where other more 1070 network applications or services may use them. The taxonomy 1071 introduced in this document defines distinct SDN planes, abstraction 1072 layers and interfaces, aiming to clarify SDN terminology and 1073 establish commonly accepted reference definitions across the SDN 1074 community irrespective of specific implementation choices. 1076 6. Contributors 1078 The authors would like to acknowledge (in alphabetical order) the 1079 following persons as contributors to this document. They all 1080 provided text, pointers and comments that made this document more 1081 complete: 1083 Daniel King for providing text related to PCEP. 1085 Scott Mansfield for information regarding current ITU work on SDN. 1087 Yaakov Stein for providing text related to the CAP theorem and SDO- 1088 related information. 1090 Russ White for text suggestions on the definitions on control, 1091 management and application. 1093 7. Acknowledgements 1095 The authors would like to acknowledge Salvatore Loreto and Sudhir 1096 Modali for their contributions in the initial discussion on the SDNRG 1097 mailing list as well as their draft-specific comments; they helped 1098 put this document in a better shape. 1100 Additionally we would like to thank (in alphabetical order) Shivleela 1101 Arlimatti, Roland Bless, Scott Brim, Alan Clark, Luis Miguel 1102 Contreras Murillo, Tim Copley, Linda Dunbar, Ken Gray, Deniz Gurkan, 1103 Dave Hood, Georgios Karagiannis, Bhumip Khasnabish, Sriganesh Kini, 1104 Ramki Krishnan, Dirk Kutscher, Diego Lopez, Scott Mansfield, Pedro 1105 Martinez-Julia, David E Mcdysan, Erik Nordmark, Carlos Pignataro, 1106 Robert Raszuk, Bless Roland, Francisco Javier Ros Munoz, Yaakov 1107 Stein, Dimitri Staessens, Eve Varma, Stuart Venters, Russ White and 1108 Lee Young for their critical comments and discussions at the IETF 88, 1109 IETF 89 and IETF 90 meetings and the SDNRG mailing list, which we 1110 took into consideration while revising this document. 1112 We would also like to thank (in alphabetical order) Spencer Dawkins 1113 and Eliot Lear for their IRSG reviews which further refined this 1114 document. 1116 Finally we thank Nobo Akiya for his review on the section on BFD, 1117 Julien Meuric for his review on the section of PCE, and Adrian Farrel 1118 and Benoit Claise for their IESG reviews of this document. 1120 Kostas Pentikousis is supported by [ALIEN], a research project 1121 partially funded by the European Community under the Seventh 1122 Framework Program (grant agreement no. 317880). The views expressed 1123 here are those of the author only. The European Commission is not 1124 liable for any use that may be made of the information in this 1125 document. 1127 8. IANA Considerations 1129 This memo makes no requests to IANA. 1131 9. Security Considerations 1133 This document does not propose a new network architecture or protocol 1134 and therefore does not have any impact on the security of the 1135 Internet. That said, security is paramount in networking and thus it 1136 should be given full consideration when designing a network 1137 architecture or operational deployment. Security in SDN is discussed 1138 in the literature, for example in [SDNSecurity][SDNSecServ] and 1139 [SDNSecOF]. Security considerations regarding specific interfaces, 1140 such as, for example, ForCES, I2RS, SNMP, or NETCONF are addressed in 1141 their respective documents as well as [RFC7149]. 1143 10. Informative References 1145 [A4D05] Greenberg, Albert, et al., "A clean slate 4D approach to 1146 network control and management", ACM SIGCOMM Computer 1147 Communication Review 35.5 (2005): 41-54 , 2005. 1149 [ALIEN] D. Parniewicz, R. Doriguzzi Corin, et al., "Design and 1150 Implementation of an OpenFlow Hardware Abstraction Layer", 1151 Proc. ACM SIGCOMM Workshop on Distributed Cloud Computing 1152 (DCC), Chicago, Illinois, USA, August 2014, pp. 71-76. 1153 doi> 10.1145/2627566.2627577 , 2014. 1155 [Beacon] Erickson, David., "The beacon openflow controller.", In 1156 Proceedings of the second ACM SIGCOMM workshop on Hot 1157 topics in software defined networking, pp. 13-18. ACM, 1158 2013. , 2013. 1160 [CAPBR] Eric A. Brewer, "Towards robust distributed systems.", 1161 Symposium on Principles of Distributed Computing (PODC). 1162 2000 , 2000. 1164 [CAPFN] Panda, Aurojit, Colin Scott, Ali Ghodsi, Teemu Koponen, 1165 and Scott Shenker., "CAP for Networks.", In Proceedings of 1166 the second ACM SIGCOMM workshop on Hot topics in software 1167 defined networking, pp. 91-96. ACM, 2013. , 2013. 1169 [CAPGL] Seth Gilbert, and Nancy Ann Lynch., "Brewer's conjecture 1170 and the feasibility of consistent, available, partition- 1171 tolerant web services", ACM SIGACT News 33.2 (2002): 1172 51-59. , 2002. 1174 [CORBA] Object Management Group, "Common Object Request Broker 1175 Architecture specification version 3.3", November 2012, 1176 . 1178 [DIOPR] Denazis, Spyros, Kazuho Miki, John Vicente, and Andrew 1179 Campbell., "Designing interfaces for open programmable 1180 routers.", In Active Networks, pp. 13-24. Springer Berlin 1181 Heidelberg, 1999 , 1999. 1183 [ESNet] Yu, James, and Imad Al Ajarmeh., "An empirical study of 1184 the NETCONF protocol.", In Networking and Services (ICNS), 1185 2010 Sixth International Conference on, pp. 253-258. IEEE, 1186 2010. , 2010. 1188 [FCAPS] International Telecommunication Union, "X.700: Management 1189 Framework For Open Systems Interconnection (OSI) For CCITT 1190 Applications", September 1992, 1191 . 1193 [I-D.ietf-i2rs-architecture] 1194 Atlas, A., Halpern, J., Hares, S., Ward, D., and T. 1195 Nadeau, "An Architecture for the Interface to the Routing 1196 System", draft-ietf-i2rs-architecture-05 (work in 1197 progress), July 2014. 1199 [I-D.ietf-i2rs-problem-statement] 1200 Atlas, A., Nadeau, T., and D. Ward, "Interface to the 1201 Routing System Problem Statement", draft-ietf-i2rs- 1202 problem-statement-04 (work in progress), June 2014. 1204 [I-D.ietf-i2rs-rib-info-model] 1205 Bahadur, N., Folkes, R., Kini, S., and J. Medved, "Routing 1206 Information Base Info Model", draft-ietf-i2rs-rib-info- 1207 model-03 (work in progress), May 2014. 1209 [I-D.ietf-pce-stateful-pce] 1210 Crabbe, E., Minei, I., Medved, J., and R. Varga, "PCEP 1211 Extensions for Stateful PCE", draft-ietf-pce-stateful- 1212 pce-09 (work in progress), June 2014. 1214 [ITUATM] CCITT, Geneva, Switzerland, "CCITT Recommendation 1.361, 1215 B-ISDN ATM Layer Specification", 1990. 1217 [ITUSG11] Telecommunication Standardization sector of ITU, "ITU, 1218 Study group 11", 2013, . 1221 [ITUSG13] Telecommunication Standardization sector of ITU, "ITU, 1222 Study group 13", 2013, . 1225 [ITUSS7] Telecommunication Standardization sector of ITU, "ITU, 1226 Q.700 : Introduction to CCITT Signalling System No. 7", 1227 1993. 1229 [ITUY3300] 1230 ITU-T Study Group 13, "Y.3300, Framework of software- 1231 defined networking", June 2014, . 1234 [KANDOO] Hassas Yeganeh, Soheil, and Yashar Ganjali., "Kandoo: a 1235 framework for efficient and scalable offloading of control 1236 applications.", In Proceedings of the first workshop on 1237 Hot topics in software defined networks, pp. 19-24. ACM 1238 SIGCOMM, 2012. , 2012. 1240 [NFVArch] European Telecommunication Standards Institute, "Network 1241 Functions Virtualisation (NFV): Architectural Framework; 1242 White paper, ETSI GS 9 NFV 002, 2013", December 2013, 1243 . 1246 [NOX] Gude, Natasha, Teemu Koponen, Justin Pettit, Ben Pfaff, 1247 Martin Casado, Nick McKeown, and Scott Shenker., "NOX: 1248 towards an operating system for networks.", ACM SIGCOMM 1249 Computer Communication Review 38, no. 3 (2008): 105-110. , 1250 2008. 1252 [NV09] Chowdhury, NM Mosharaf Kabir, and Raouf Boutaba, "Network 1253 virtualization: state of the art and research challenges", 1254 Communications Magazine, IEEE 47.7 (2009): 20-26 , 2009. 1256 [OF-CONFIG] 1257 Open Networking Foundation, "OpenFlow Management and 1258 Configuration Protocol 1.1.1", March 2013, 1259 . 1263 [OF08] McKeown, Nick, et al., "OpenFlow: enabling innovation in 1264 campus networks", ACM SIGCOMM Computer Communication 1265 Review 38.2 (2008): 69-74 , 2008. 1267 [OFSIGC] McKeown, Nick, Tom Anderson, Hari Balakrishnan, Guru 1268 Parulkar, Larry Peterson, Jennifer Rexford, Scott Shenker, 1269 and Jonathan Turner., "OpenFlow: enabling innovation in 1270 campus networks.", ACM SIGCOMM Computer Communication 1271 Review 38, no. 2 (2008): 69-74. , 1998. 1273 [ONFArch] Open Networking Foundation, "SDN Architecture, Issue 1", 1274 June 2014, 1275 . 1279 [OpenFlow] 1280 Open Networking Foundation, "The OpenFlow 1.4 1281 Specification.", October 2013, 1282 . 1286 [P1520] Biswas, Jit, Aurel A. Lazar, J-F. Huard, Koonseng Lim, 1287 Semir Mahjoub, L-F. Pau, Masaaki Suzuki, Soren 1288 Torstensson, Weiguo Wang, and Stephen Weinstein., "The 1289 IEEE P1520 standards initiative for programmable network 1290 interfaces.", Communications Magazine, IEEE 36, no. 10 1291 (1998): 64-70. , 1998. 1293 [PENet] Hedstrom, Brian, Akshay Watwe, and Siddharth Sakthidharan, 1294 "Protocol Efficiencies of NETCONF versus SNMP for 1295 Configuration Management Functions", PhD dissertation, 1296 Master's thesis, University of Colorado, 2011 , 2011. 1298 [PNSurvey99] 1299 Campbell, Andrew T., et al, "A survey of programmable 1300 networks", ACM SIGCOMM Computer Communication Review 29.2 1301 (1999): 7-23 , September 1992. 1303 [Peregrine] 1304 Chiueh, Tzi-cker, Cheng-Chun Tu, Yu-Cheng Wang, Pai-Wei 1305 Wang, Kai-Wen Li, and Yu-Ming Huang., "Peregrine: An All- 1306 Layer-2 Container Computer Network.", In Cloud Computing 1307 (CLOUD), 2012 IEEE 5th International Conference on, pp. 1308 686-693. IEEE, 2012 , 2012. 1310 [PiNA] John Day, "Patterns in network architecture: a return to 1311 fundamentals.", Prentice Hall (ISBN 0132252422). , 2007. 1313 [RCP] Caesar, Matthew, Donald Caldwell, Nick Feamster, Jennifer 1314 Rexford, Aman Shaikh, and Jacobus van der Merwe., "Design 1315 and implementation of a routing control platform.", In 1316 Proceedings of the 2nd conference on Symposium on 1317 Networked Systems Design & Implementation-Volume 2, pp. 1318 15-28. USENIX Association, 2005. , 2005. 1320 [REST] Fielding, Roy, "Fielding Dissertation: Chapter 5: 1321 Representational State Transfer (REST).", 2000. 1323 [RFC0826] Plummer, D., "Ethernet Address Resolution Protocol: Or 1324 converting network protocol addresses to 48.bit Ethernet 1325 address for transmission on Ethernet hardware", STD 37, 1326 RFC 826, November 1982. 1328 [RFC1953] Newman, P., Edwards, W., Hinden, R., Hoffman, E., Ching 1329 Liaw, F., Lyon, T., and G. Minshall, "Ipsilon Flow 1330 Management Protocol Specification for IPv4 Version 1.0", 1331 RFC 1953, May 1996. 1333 [RFC2297] Newman, P., Edwards, W., Hinden, R., Hoffman, E., Liaw, 1334 F., Lyon, T., and G. Minshall, "Ipsilon's General Switch 1335 Management Protocol Specification Version 2.0", RFC 2297, 1336 March 1998. 1338 [RFC2578] McCloghrie, K., Ed., Perkins, D., Ed., and J. 1339 Schoenwaelder, Ed., "Structure of Management Information 1340 Version 2 (SMIv2)", STD 58, RFC 2578, April 1999. 1342 [RFC3411] Harrington, D., Presuhn, R., and B. Wijnen, "An 1343 Architecture for Describing Simple Network Management 1344 Protocol (SNMP) Management Frameworks", STD 62, RFC 3411, 1345 December 2002. 1347 [RFC3412] Case, J., Harrington, D., Presuhn, R., and B. Wijnen, 1348 "Message Processing and Dispatching for the Simple Network 1349 Management Protocol (SNMP)", STD 62, RFC 3412, December 1350 2002. 1352 [RFC3414] Blumenthal, U. and B. Wijnen, "User-based Security Model 1353 (USM) for version 3 of the Simple Network Management 1354 Protocol (SNMPv3)", STD 62, RFC 3414, December 2002. 1356 [RFC3417] Presuhn, R., "Transport Mappings for the Simple Network 1357 Management Protocol (SNMP)", STD 62, RFC 3417, December 1358 2002. 1360 [RFC3418] Presuhn, R., "Management Information Base (MIB) for the 1361 Simple Network Management Protocol (SNMP)", STD 62, RFC 1362 3418, December 2002. 1364 [RFC3535] Schoenwaelder, J., "Overview of the 2002 IAB Network 1365 Management Workshop", RFC 3535, May 2003. 1367 [RFC3746] Yang, L., Dantu, R., Anderson, T., and R. Gopal, 1368 "Forwarding and Control Element Separation (ForCES) 1369 Framework", RFC 3746, April 2004. 1371 [RFC4271] Rekhter, Y., Li, T., and S. Hares, "A Border Gateway 1372 Protocol 4 (BGP-4)", RFC 4271, January 2006. 1374 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation 1375 Element (PCE)-Based Architecture", RFC 4655, August 2006. 1377 [RFC5424] Gerhards, R., "The Syslog Protocol", RFC 5424, March 2009. 1379 [RFC5440] Vasseur, JP. and JL. Le Roux, "Path Computation Element 1380 (PCE) Communication Protocol (PCEP)", RFC 5440, March 1381 2009. 1383 [RFC5531] Thurlow, R., "RPC: Remote Procedure Call Protocol 1384 Specification Version 2", RFC 5531, May 2009. 1386 [RFC5743] Falk, A., "Definition of an Internet Research Task Force 1387 (IRTF) Document Stream", RFC 5743, December 2009. 1389 [RFC5810] Doria, A., Hadi Salim, J., Haas, R., Khosravi, H., Wang, 1390 W., Dong, L., Gopal, R., and J. Halpern, "Forwarding and 1391 Control Element Separation (ForCES) Protocol 1392 Specification", RFC 5810, March 2010. 1394 [RFC5812] Halpern, J. and J. Hadi Salim, "Forwarding and Control 1395 Element Separation (ForCES) Forwarding Element Model", RFC 1396 5812, March 2010. 1398 [RFC5880] Katz, D. and D. Ward, "Bidirectional Forwarding Detection 1399 (BFD)", RFC 5880, June 2010. 1401 [RFC6020] Bjorklund, M., "YANG - A Data Modeling Language for the 1402 Network Configuration Protocol (NETCONF)", RFC 6020, 1403 October 2010. 1405 [RFC6241] Enns, R., Bjorklund, M., Schoenwaelder, J., and A. 1406 Bierman, "Network Configuration Protocol (NETCONF)", RFC 1407 6241, June 2011. 1409 [RFC6632] Ersue, M. and B. Claise, "An Overview of the IETF Network 1410 Management Standards", RFC 6632, June 2012. 1412 [RFC7011] Claise, B., Trammell, B., and P. Aitken, "Specification of 1413 the IP Flow Information Export (IPFIX) Protocol for the 1414 Exchange of Flow Information", STD 77, RFC 7011, September 1415 2013. 1417 [RFC7047] Pfaff, B. and B. Davie, "The Open vSwitch Database 1418 Management Protocol", RFC 7047, December 2013. 1420 [RFC7149] Boucadair, M. and C. Jacquenet, "Software-Defined 1421 Networking: A Perspective from within a Service Provider 1422 Environment", RFC 7149, March 2014. 1424 [RFC7276] Mizrahi, T., Sprecher, N., Bellagamba, E., and Y. 1425 Weingarten, "An Overview of Operations, Administration, 1426 and Maintenance (OAM) Tools", RFC 7276, June 2014. 1428 [RINA] John Day, Ibrahim Matta, and Karim Mattar., "Networking is 1429 IPC: a guiding principle to a better internet.", In 1430 Proceedings of the 2008 ACM CoNEXT Conference, p. 67. ACM, 1431 2008. , 2008. 1433 [RouteFlow] 1434 Nascimento, Marcelo R., Christian E. Rothenberg, Marcos R. 1435 Salvador, Carlos NA Correa, Sidney C. de Lucena, and 1436 Mauricio F. Magalhaes., "Virtual routers as a service: the 1437 routeflow approach leveraging software-defined networks.", 1438 In Proceedings of the 6th International Conference on 1439 Future Internet Technologies, pp. 34-37. ACM, 2011. , 1440 2011. 1442 [SDNACS] Diego Kreutz, Fernando M. V. Ramos, Paulo Verissimo, 1443 Christian Esteve Rothenberg, Siamak Azodolmolky, Steve 1444 Uhlig, "Software-Defined Networking: A Comprehensive 1445 Survey.", arXiv preprint arXiv:1406.0440 , 2014. 1447 [SDNHistory] 1448 Feamster, Nick, Jennifer Rexford, and Ellen Zegura., "The 1449 Road to SDN", ACM Queue11, no. 12 (2013): 20. , 2013. 1451 [SDNNFV] Haleplidis, Evangelos, Jamal Hadi Salim, Spyros Denazis, 1452 and Odysseas Koufopavlou., "Towards a Network Abstraction 1453 Model for SDN.", Journal of Network and Systems Management 1454 (2014): 1-19. Special Issue on Management of Software 1455 Defined Networks, Springer , 2014. 1457 [SDNSecOF] 1458 Kloti, Rowan, Vasileios Kotronis, and Paul Smith., 1459 "Openflow: A security analysis.", Proceedings Workshop on 1460 Secure Network Protocols (NPSec). IEEE (2013). , 2013, 1461 . 1463 [SDNSecServ] 1464 Sandra Scott-Hayward, Gemma O'Callaghan, and Sakir Sezer., 1465 "SDN security: A survey.", In Future Networks and Services 1466 (SDN4FNS), 2013 IEEE SDN for, pp. 1-7. IEEE, 2013. , 2013. 1468 [SDNSecurity] 1469 Diego Kreutz, Fernando Ramos, and Paulo Verissimo., 1470 "Towards secure and dependable software-defined 1471 networks.", In Proceedings of the second ACM SIGCOMM 1472 workshop on Hot topics in software defined networking, pp. 1473 55-60. ACM, 2013. , 2013. 1475 [SDNSurvey] 1476 Bruno Astuto A. Nunes, Marc Mendonca, Xuan-Nam Nguyen, 1477 Katia Obraczka, and Thierry Turletti, "A Survey of 1478 Software-Defined Networking: Past, Present, and Future of 1479 Programmable Networks", IEEE Communications Surveys and 1480 Tutorials DOI:10.1109/SURV.2014.012214.00180 , 2014. 1482 [SLTSDN] Yosr Jarraya, Taous Madi, and Mourad Debbabi, "A Survey 1483 and a Layered Taxonomy of Software-Defined Networking", To 1484 be published in Communications Surveys and Tutorials, IEEE 1485 Issue: 99 , 2014. 1487 [SoftRouter] 1488 Lakshman, T. V., T. Nandagopal, R. Ramjee, K. Sabnani, and 1489 T. Woo., "The softrouter architecture.", In Proc. ACM 1490 SIGCOMM Workshop on Hot Topics in Networking. 2004. , 1491 2004. 1493 [Tempest] Rooney, Sean, Jacobus E. van der Merwe, Simon A. Crosby, 1494 and Ian M. Leslie., "The Tempest: a framework for safe, 1495 resource assured, programmable networks.", Communications 1496 Magazine, IEEE 36, no. 10 (1998): 42-53 , 1998. 1498 Authors' Addresses 1500 Evangelos Haleplidis (editor) 1501 University of Patras 1502 Department of Electrical and Computer Engineering 1503 Patras 26500 1504 Greece 1506 Email: ehalep@ece.upatras.gr 1508 Kostas Pentikousis (editor) 1509 EICT GmbH 1510 Torgauer Strasse 12-15 1511 10829 Berlin 1512 Germany 1514 Email: k.pentikousis@eict.de 1515 Spyros Denazis 1516 University of Patras 1517 Department of Electrical and Computer Engineering 1518 Patras 26500 1519 Greece 1521 Email: sdena@upatras.gr 1523 Jamal Hadi Salim 1524 Mojatatu Networks 1525 Suite 400, 303 Moodie Dr. 1526 Ottawa, Ontario K2H 9R4 1527 Canada 1529 Email: hadi@mojatatu.com 1531 David Meyer 1532 Brocade 1534 Email: dmm@1-4-5.net 1536 Odysseas Koufopavlou 1537 University of Patras 1538 Department of Electrical and Computer Engineering 1539 Patras 26500 1540 Greece 1542 Email: odysseas@ece.upatras.gr