idnits 2.17.1 draft-finn-detnet-architecture-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 1, 2015) is 3092 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'AVnu' is defined on line 1025, but no explicit reference was found in the text == Unused Reference: 'HART' is defined on line 1034, but no explicit reference was found in the text == Unused Reference: 'I-D.finn-detnet-problem-statement' is defined on line 1047, but no explicit reference was found in the text == Unused Reference: 'IEEE802.1AS-2011' is defined on line 1074, but no explicit reference was found in the text == Unused Reference: 'IEEE802.1TSNTG' is defined on line 1114, but no explicit reference was found in the text == Unused Reference: 'IEEE802154' is defined on line 1123, but no explicit reference was found in the text == Unused Reference: 'ISA95' is defined on line 1141, but no explicit reference was found in the text == Unused Reference: 'ODVA' is defined on line 1145, but no explicit reference was found in the text == Unused Reference: 'Profinet' is defined on line 1152, but no explicit reference was found in the text == Unused Reference: 'RFC2205' is defined on line 1157, but no explicit reference was found in the text == Outdated reference: A later version (-05) exists of draft-finn-detnet-problem-statement-04 == Outdated reference: A later version (-30) exists of draft-ietf-6tisch-architecture-08 Summary: 0 errors (**), 0 flaws (~~), 13 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet N. Finn 3 Internet-Draft P. Thubert 4 Intended status: Standards Track Cisco 5 Expires: May 4, 2016 M. Johas Teener 6 Broadcom 7 November 1, 2015 9 Deterministic Networking Architecture 10 draft-finn-detnet-architecture-02 12 Abstract 14 Deterministic Networking (DetNet) provides a capability to carry 15 specified unicast or multicast data streams for real-time 16 applications with extremely low data loss rates and bounded latency. 17 Techniques used include: 1) reserving data plane resources for 18 individual (or aggregated) DetNet streams in some or all of the relay 19 systems (bridges or routers) along the path of the stream; 2) 20 providing fixed paths for DetNet streams that do not rapidly change 21 with the network topology; and 3) sequentializing, replicating, and 22 eliminating duplicate packets at various points to ensure the 23 availability of at least one path. The capabilities can be managed 24 by configuration, or by manual or automatic network management. 26 Status of This Memo 28 This Internet-Draft is submitted in full conformance with the 29 provisions of BCP 78 and BCP 79. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF). Note that other groups may also distribute 33 working documents as Internet-Drafts. The list of current Internet- 34 Drafts is at http://datatracker.ietf.org/drafts/current/. 36 Internet-Drafts are draft documents valid for a maximum of six months 37 and may be updated, replaced, or obsoleted by other documents at any 38 time. It is inappropriate to use Internet-Drafts as reference 39 material or to cite them other than as "work in progress." 41 This Internet-Draft will expire on May 4, 2016. 43 Copyright Notice 45 Copyright (c) 2015 IETF Trust and the persons identified as the 46 document authors. All rights reserved. 48 This document is subject to BCP 78 and the IETF Trust's Legal 49 Provisions Relating to IETF Documents 50 (http://trustee.ietf.org/license-info) in effect on the date of 51 publication of this document. Please review these documents 52 carefully, as they describe your rights and restrictions with respect 53 to this document. Code Components extracted from this document must 54 include Simplified BSD License text as described in Section 4.e of 55 the Trust Legal Provisions and are provided without warranty as 56 described in the Simplified BSD License. 58 Table of Contents 60 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 61 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 62 3. Providing the DetNet Quality of Service . . . . . . . . . . . 5 63 3.1. Zero Congestion Loss . . . . . . . . . . . . . . . . . . 7 64 3.2. Pinned-down paths . . . . . . . . . . . . . . . . . . . . 8 65 3.3. Seamless Redundancy . . . . . . . . . . . . . . . . . . . 8 66 4. DetNet Architecture . . . . . . . . . . . . . . . . . . . . . 9 67 4.1. The Application Plane . . . . . . . . . . . . . . . . . . 11 68 4.2. The Controller Plane . . . . . . . . . . . . . . . . . . 11 69 4.3. The Network Plane . . . . . . . . . . . . . . . . . . . . 12 70 4.4. Elements of DetNet Architecture . . . . . . . . . . . . . 13 71 4.5. DetNet streams . . . . . . . . . . . . . . . . . . . . . 14 72 4.5.1. Talker guarantees . . . . . . . . . . . . . . . . . . 14 73 4.5.2. Incomplete Networks . . . . . . . . . . . . . . . . . 16 74 4.6. Queuing, Shaping, Scheduling, and Preemption . . . . . . 16 75 4.7. Coexistence with normal traffic . . . . . . . . . . . . . 17 76 4.8. Fault Mitigation . . . . . . . . . . . . . . . . . . . . 17 77 4.9. Protocol Stack Model . . . . . . . . . . . . . . . . . . 18 78 4.10. Advertising resources, capabilities and adjacencies . . . 19 79 4.11. Provisioning model . . . . . . . . . . . . . . . . . . . 20 80 4.11.1. Centralized Path Computation and Installation . . . 20 81 4.11.2. Distributed Path Setup . . . . . . . . . . . . . . . 20 82 5. Related IETF work . . . . . . . . . . . . . . . . . . . . . . 20 83 5.1. Deterministic PHB . . . . . . . . . . . . . . . . . . . . 20 84 5.2. 6TiSCH . . . . . . . . . . . . . . . . . . . . . . . . . 21 85 6. Security Considerations . . . . . . . . . . . . . . . . . . . 21 86 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 87 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 22 88 9. Access to IEEE 802.1 documents . . . . . . . . . . . . . . . 22 89 10. Informative References . . . . . . . . . . . . . . . . . . . 22 90 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 92 1. Introduction 94 Operational Technology (OT) refers to industrial networks that are 95 typically used for monitoring systems and supporting control loops, 96 as well as movement detection systems for use in process control 97 (i.e., process manufacturing) and factory automation (i.e., discrete 98 manufacturing). Due to its different goals, OT has evolved in 99 parallel but in a manner that is radically different from IT/ICT, 100 focusing on highly secure, reliable and deterministic networks, with 101 limited scalability over a bounded area. 103 The convergence of IT and OT technologies, also called the Industrial 104 Internet, represents a major evolution for both sides. The work has 105 already started; in particular, the industrial automation space has 106 been developing a number of Ethernet-based replacements for existing 107 digital control systems, often not packet-based (fieldbus 108 technologies). 110 These replacements are meant to provide similar behavior as the 111 incumbent protocols, and their common focus is to transport a fully 112 characterized flow over a well-controlled environment (i.e., a 113 factory floor), with a bounded latency, extraordinarily low frame 114 loss, and a very narrow jitter. Examples of such protocols include 115 PROFINET, ODVA Ethernet/IP, and EtherCAT. 117 In parallel, the need for determinism in professional and home audio/ 118 video markets drove the formation of the Audio/Video Bridging (AVB) 119 standards effort of IEEE 802.1. With the explosion of demand for 120 connectivity and multimedia in transportation in general, the 121 Ethernet AVB technology has become one of the hottest topics, in 122 particular in the automotive connectivity. It is finding application 123 in all elements of the vehicle from head units, to rear seat 124 entertainment modules, to amplifiers and camera modules. While aimed 125 at less critical applications than some industrial networks, AVB 126 networks share the requirement for extremely low packet loss rates 127 and ensured finite latency and jitter. 129 Other instances of in-vehicle deterministic networks have arisen as 130 well for control networks in cars, trains and buses, as well as 131 avionics, with, for instance, the mission-critical "Avionics Full- 132 Duplex Switched Ethernet" (AFDX) that was designed as part of the 133 ARINC 664 standards. Existing automotive control networks such as 134 the LIN, CAN and FlexRay standards were not designed to cover these 135 increasing demands in terms of bandwidth and scalability that we see 136 with various kinds of Driver Assistance Systems (DAS) and new 137 multiplexing technologies based on Ethernet are now getting traction. 139 The generalization of the needs for more deterministic networks have 140 led to the IEEE 802.1 AVB Task Group becoming the Time-Sensitive 141 Networking (TSN) Task Group (TG), with a much-expanded constituency 142 from the industrial and vehicular markets. Along with this 143 expansion, the networks in consideration are becoming larger and 144 structured, requiring deterministic forwarding beyond the LAN 145 boundaries. For instance, Industrial Automation segregates the 146 network along the broad lines of the Purdue Enterprise Reference 147 Architecture (PERA), using different technologies at each level, and 148 public infrastructures such as Electricity Automation require 149 deterministic properties over the Wide Area. The realization is now 150 coming that the convergence of IT and OT networks requires Layer-3, 151 as well as Layer-2, capabilities. 153 While the initial user base has focused almost entirely on Ethernet 154 physical media and Ethernet-based bridging protocol (from several 155 Standards Development Organizations), the need for Layer-3 expressed, 156 above, must not be confined to Ethernet and Ethernet-like media, and 157 while such media must be encompassed by any useful DetNet 158 architecture, cooperation between IETF and other SDOs must not be 159 limited to IEEE or IEEE 802. Furthermore, while the work completed 160 and ongoing in other SDOs, and in IEEE 802 in particular, provide an 161 obvious starting point for a DetNet architecture, we must not assume 162 that these other SDOs' work confines the space in which the DetNet 163 architecture progresses. 165 The present architecture is the result of a collaboration of IETF 166 IEEE members, and describes an abstract model that can be applicable 167 both at Layer-2 and Layer-3, and along segments of different 168 technologies. With this new work, a path may span, for instance, 169 across a (limited) number of 802.1 bridges and then a (limited) 170 number of IP routers. In that example, the IEEE 802.1 bridges may be 171 operating at Layer-2 over Ethernet whereas the IP routers may be 172 6TiSCH nodes operating at Layer-2 and/or Layer-3 over the IEEE 173 802.15.4e MAC. 175 Many applications of interest to Deterministic Networking require the 176 ability to synchronize the clocks in end systems to a sub-microsecond 177 accuracy. Some of the queue control techniques defined in 178 Section 4.6 also require time synchronization among relay systems. 179 The means used to achieve time synchronization are not addressed in 180 this document. 182 2. Terminology 184 The following special terms are used in this document in order to 185 avoid the assumption that a given element in the architecture does or 186 does not have Internet Protocol stack, functions as a router or a 187 bridge, or otherwise plays a particular role at Layer-3 or higher: 189 bridge 190 A Customer Bridge as defined by IEEE 802.1Q 191 [IEEE802.1Q-2014]. 193 end system 194 Commonly called a "host" in IETF documents, and an "end 195 station" is IEEE 802 documents. End systems of interest to 196 this document are talkers and listeners. 198 listener 199 An end system capable of sinking a DetNet stream. 201 relay system 202 A router or a bridge. 204 reservation 205 A trail of configuration from talker to listener(s) through 206 relay systems associated with a DetNet stream, required to 207 deliver the benefits of DetNet. 209 stream 210 A DetNet stream is a sequence of packets from a single 211 talker, through some number of relay systems to one or more 212 listeners, that is limited by the talker in its maximum 213 packet size and transmission rate, and can thus be ensured 214 the DetNet Quality of Service (QoS) from the network. 216 talker 217 An end system capable of sourcing a DetNet stream. 219 3. Providing the DetNet Quality of Service 221 DetNet Quality of Service is expressed in terms of: 223 o Minimum and maximum end-to-end latency from talker to listener; 225 o Probability of loss of a packet, assuming the normal operation of 226 the relay systems and links; 228 o Probability of loss of a packet in the event of the failure of a 229 relay system or link. 231 It is a distinction of DetNet that it is concerned solely with worst- 232 case values for all of the above parameters. Average, mean, or 233 typical values are of no interest, because they do not affect the 234 ability of a real-time system to perform its tasks. For example, in 235 this document, we will often speak of assuring a DetNet flow a 236 bounded latency. In general, a trivial priority-based queuing scheme 237 will give better average latency to a flow than DetNet, but of 238 course, the worst-case latency is essentially unbounded. 240 Three techniques are employed by DetNet to achieve these QoS 241 parameters: 243 a. Zero congestion loss (Section 3.1). Network resources such as 244 link bandwidth, buffers, queues, shapers, and scheduled input/ 245 output slots are assigned in each relay system to the use of a 246 specific DetNet stream or class of streams. Given a finite 247 amount of buffer space, zero congestion loss necessarily ensures 248 a bounded end-to-end latency. Depending on the resources 249 employed, a minimum latency, and thus bounded jitter, can also be 250 achieved. 252 b. Pinned-down paths (Section 3.2). Point-to-point paths or point- 253 to-multipoint trees through the network from a talker to one or 254 more listeners can be established, and DetNet streams assigned to 255 follow a particular path or tree. 257 c. Packet replication and deletion (Section 3.3). End systems and/ 258 or relay systems can number packets sequentially, replicate them, 259 and later eliminate all but one of the replicants, at multiple 260 points in the network in order to ensure that one (or more) 261 equipment failure events still leave at least one path intact for 262 a DetNet stream. 264 These three techniques can be applied independently, giving eight 265 possible combinations, including none (no DetNet), although some 266 combinations are of wider utility than others. This separation keeps 267 the protocol stack coherent and maximizes interoperability with 268 existing and developing standards in this (IETF) and other Standards 269 Development Organizations. Some examples of typical expected 270 combinations: 272 o Pinned-down paths (a) plus packet replication (b) are exactly the 273 techniques employed by [HSR-PRP]. Pinned-down paths are achieved 274 by limiting the physical topology of the network, and the 275 sequentialization, replication, and duplicate elimination are 276 facilitated by packet tags added at the front or the end of 277 Ethernet frames. 279 o Zero congestion loss (a) alone is is offered by IEEE 802.1 Audio 280 Video bridging [IEEE802.1BA-2011]. As long as the network suffers 281 no failures, zero congestion loss can be achieved through the use 282 of a reservation protocol (MSRP), shapers in every relay system 283 (bridge), and a bit of network calculus. 285 o Using all three together gives maximum protection. 287 There are, of course, simpler methods available (and employed, today) 288 to achieve levels of latency and packet loss that are satisfactory 289 for many applications. Prioritization and over-provisioning is one 290 such technique. However, these methods generally work best in the 291 absence of any significant amount of non-critical traffic in the 292 network (if, indeed, such traffic is supported at all), or work only 293 if the critical traffic constitutes only a small portion of the 294 network's theoretical capacity, or work only if all systems are 295 functioning properly, or in the absence of actions by end systems 296 that disrupt the network's operations. 298 There are any number of methods in use, defined, or in progress for 299 accomplishing each of the above techniques. It is expected that this 300 DetNet Architecture will assist various vendors, users, and/or 301 "vertical" Standards Development Organizations (dedicated to a single 302 industry) to make selections among the available means of 303 implementing DetNet networks. 305 3.1. Zero Congestion Loss 307 The primary means by which DetNet achieves its QoS assurances is to 308 completely eliminate congestion at an output port as a cause of 309 packet loss. Given that a DetNet stream cannot be throttled, this 310 can be achieved only by the provision of sufficient buffer storage at 311 each hop through the network to ensure that no packets are dropped 312 due to a lack of buffer storage. 314 Ensuring adequate buffering requires, in turn, that the talker, and 315 every relay system along the path to the listener (or nearly every 316 relay system -- see Section 4.5.2) be careful to regulate its output 317 to not exceed the data rate for any stream, except for brief periods 318 when making up for interfering traffic. Any packet sent ahead of its 319 time potentially adds to the number of buffers required by the next 320 hop, and may thus exceed the resources allocated for a particular 321 stream. 323 The low-level mechanisms described in Section 4.6 provide the 324 necessary regulation of transmissions by an edge system or relay 325 system to ensure zero congestion loss. The reservation of the 326 bandwidth and buffers for a stream requires the provisioning 327 described in Section 4.11. 329 3.2. Pinned-down paths 331 In networks controlled by typical peer-to-peer protocols such as IEEE 332 802.1 ISIS bridged networks or IETF OSPF routed networks, a network 333 topology event in one part of the network can impact, at least 334 briefly, the delivery of data in parts of the network remote from the 335 failure or recovery event. Thus, even redundant paths through a 336 network, if controlled by the typical peer-to-peer protocols, do not 337 eliminate the chances of brief losses of contact. 339 Many real-time networks rely on physical rings or chains of two-port 340 devices, with a relatively simple ring control protocol. This 341 supports redundant paths with a minimum of wiring. As an additional 342 benefit, ring topologies can often utilize different topology 343 management protocols than those used for a mesh network, with a 344 consequent reduction in the response time to topology changes. Of 345 course, this comes at some cost in terms of increased hop count, and 346 thus latency, for the typical path. 348 In order to get the advantages of low hop count and still ensure 349 against even very brief losses of connectivity, DetNet employs 350 pinned-down paths, where the path taken by a given DetNet stream does 351 not change, at least immediately, and likely not at all, in response 352 to network topology events. When combined with seamless redundancy 353 (Section 3.3), this results in a high likelihood of continuous 354 connectivity. 356 3.3. Seamless Redundancy 358 After congestion loss has been eliminated, the most important causes 359 of packet loss are random media and/or memory faults, and equipment 360 failures. 362 Seamless redundancy involves three capabilities: 364 o Adding sequence numbers, once, to the packets of a DetNet stream. 366 o Replicating these packets and, typically, sending them along at 367 least two different paths to the listener(s). (Often, the pinned- 368 down paths of Section 3.2.) 370 o Discarding duplicated packets. 372 In the simplest case, this amounts to replicating each packet in a 373 talker that has two interfaces, and conveying them through the 374 network, along separate paths, to the similarly dual-homed listeners, 375 that discard the extras. This ensures that one path (with zero 376 congestion loss) remains, even if some relay system fails. 378 Alternatively, relay systems in the network can provide replication 379 and elimination facilities at various points in the network, so that 380 multiple failures can be accommodated. 382 This is shown in the following figure, where the two relay systems 383 each replicate (R) the DetNet stream on input, sending the stream to 384 both the other relay system and to the end system, and eliminate 385 duplicates (E) on the output interface to the right-hand end system. 386 Any one links in the network can fail, and the Detnet stream can 387 still get through. Furthermore, two links can fail, as long as they 388 are in different segments of the network. 390 > > > > > > > > relay > > > > > > > > 391 > /------------+ R system E +------------\ > 392 > / v + ^ \ > 393 end R + v | ^ + E end 394 system + v | ^ + system 395 > \ v + ^ / > 396 > \------------+ R relay E +------------/ > 397 > > > > > > > > system > > > > > > > > 399 Figure 1 401 Note that seamless redundancy does not react to and correct failures; 402 it is entirely passive. Thus, intermittent failures, mistakenly 403 created access control lists, or misrouted data is handled just the 404 same as the equipment failures that are detected handled by typical 405 routing and bridging protocols. 407 4. DetNet Architecture 409 Traffic Engineering Architecture and Signaling (TEAS) [TEAS] defines 410 traffic-engineering architectures for generic applicability across 411 packet and non-packet networks. From TEAS perspective, Traffic 412 Engineering (TE) refers to techniques that enable operators to 413 control how specific traffic flows are treated within their networks. 415 Because if its very nature of establishing pinned-down optimized 416 paths, Deterministic Networking can be seen as a new, specialized 417 branch of Traffic Engineering, and inherits its architecture with a 418 separation into planes. 420 The Deterministic Networking architecture is thus composed of three 421 planes, a (User) Application Plane, a Controller Plane, and a Network 422 Plane, which echoes that of Software-Defined Networking (SDN): Layers 423 and Architecture Terminology [RFC7426] which is represented below: 425 SDN Layers and Architecture Terminology per RFC 7426 427 o--------------------------------o 428 | | 429 | +-------------+ +----------+ | 430 | | Application | | Service | | 431 | +-------------+ +----------+ | 432 | Application Plane | 433 o---------------Y----------------o 434 | 435 *-----------------------------Y---------------------------------* 436 | Network Services Abstraction Layer (NSAL) | 437 *------Y------------------------------------------------Y-------* 438 | | 439 | Service Interface | 440 | | 441 o------Y------------------o o---------------------Y------o 442 | | Control Plane | | Management Plane | | 443 | +----Y----+ +-----+ | | +-----+ +----Y----+ | 444 | | Service | | App | | | | App | | Service | | 445 | +----Y----+ +--Y--+ | | +--Y--+ +----Y----+ | 446 | | | | | | | | 447 | *----Y-----------Y----* | | *---Y---------------Y----* | 448 | | Control Abstraction | | | | Management Abstraction | | 449 | | Layer (CAL) | | | | Layer (MAL) | | 450 | *----------Y----------* | | *----------Y-------------* | 451 | | | | | | 452 o------------|------------o o------------|---------------o 453 | | 454 | CP | MP 455 | Southbound | Southbound 456 | Interface | Interface 457 | | 458 *------------Y---------------------------------Y----------------* 459 | Device and resource Abstraction Layer (DAL) | 460 *------------Y---------------------------------Y----------------* 461 | | | | 462 | o-------Y----------o +-----+ o--------Y----------o | 463 | | Forwarding Plane | | App | | Operational Plane | | 464 | o------------------o +-----+ o-------------------o | 465 | Network Device | 466 +---------------------------------------------------------------+ 468 Figure 2 470 4.1. The Application Plane 472 Per [RFC7426], the Application Plane includes both applications and 473 services. In particular, the Application Plane incorporates the User 474 Agent, a specialized application that interacts with the end user / 475 operator and performs requests for Deterministic Networking services 476 via an abstract Stream Management Entity, (SME) which may or may not 477 be collocated with (one of) the end systems. 479 At the Application Plane, a management interface enables the 480 negotiation of streams between end systems. An abstraction of the 481 stream called a Traffic Specification (TSpec) provides the 482 representation. This abstraction is used to place a reservation over 483 the (Northbound) Service Interface and within the Application plane. 484 It is associated with an abstraction of location, such as IP 485 addresses and DNS names, to identify the end systems and eventually 486 specify intermediate relay systems. 488 4.2. The Controller Plane 490 The Controller Plane corresponds to the aggregation of the Control 491 and Management Planes in [RFC7426], though Common Control and 492 Measurement Plane (CCAMP) [CCAMP] makes an additional distinction 493 between management and measurement. When the logical separation of 494 the Control, Measurement and other Management entities is not 495 relevant, the term Controller Plane is used for simplicity to 496 represent them all, and the term controller refers to any device 497 operating in that plane, whether is it a Path Computation entity or a 498 Network Management entity (NME). The Path Computation Element (PCE) 499 [PCE] is a core element of a controller, in charge of computing 500 Deterministic paths to be applied in the Network Plane. 502 A (Northbound) Service Interface enables applications in the 503 Application Plane to communicate with the entities in the Controller 504 Plane. 506 One or more PCE(s) collaborate to implement the requests from the SME 507 as Per-Stream Per-Hop Behaviors installed in the relay systems for 508 each individual streams. The PCEs place each stream along a 509 deterministic sequence of relay systems so as to respect per-stream 510 constraints such as security and latency, and optimize the overall 511 result for metrics such as an abstract aggregated cost. The 512 deterministic sequence can typically be more complex than a direct 513 sequence and include redundancy path, with one or more packet 514 replication and elimination points. 516 4.3. The Network Plane 518 The Network Plane represents the network devices and protocols as a 519 whole, regardless of the Layer at which the network devices operate. 521 The network Plane comprises the Network Interface Cards (NIC) in the 522 end systems, which are typically IP hosts, and relay systems, which 523 are typically IP routers and switches. Network-to-Network Interfaces 524 such as used for Traffic Engineering path reservation in [RFC3209], 525 as well as User-to-Network Interfaces (UNI) such as provided by the 526 Local Management Interface (LMI) between network and end systems, are 527 all part of the Network Plane. 529 A Southbound (Network) Interface enables the entities in the 530 Controller Plane to communicate with devices in the Network Plane. 531 This interface leverages and extends TEAS to describe the physical 532 topology and resources in the Network Plane. 534 Stream Management Entity 536 End End 537 System System 539 -+-+-+-+-+-+-+ Northbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- 541 PCE PCE PCE PCE 543 -+-+-+-+-+-+-+ Southbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- 545 Relay Relay Relay Relay 546 System System System System 547 NIC NIC 548 Relay Relay Relay Relay 549 System System System System 551 Figure 3 553 The relay systems (and eventually the end systems NIC) expose their 554 capabilities and physical resources to the controller (the PCE), and 555 update the PCE with their dynamic perception of the topology, across 556 the Southbound Interface. In return, the PCE(s) set the per-stream 557 paths up, providing a Stream Characterization that is more tightly 558 coupled to the relay system Operation than a TSpec. 560 At the Network plane, relay systems exchange information regarding 561 the state of the paths, between adjacent systems and eventually with 562 the end systems, and forward packets within constraints associated to 563 each stream, or, when unable to do so, perform a last resort 564 operation such as drop or declassify. 566 This specification focuses on the Southbound interface and the 567 operation of the Network Plane. 569 4.4. Elements of DetNet Architecture 571 The DetNet architecture has a number of elements, discussed in the 572 following sections. Note that not every application requires all of 573 these elements. 575 a. A model for the definition, identification, and operation of 576 DetNet streams (Section 4.5), for use by relay systems to 577 classify and process individual packets following per-stream 578 rules. 580 b. A model for the flow of data from an end system or through a 581 relay system that can be used to predict the bounds for that 582 system's impact on the QoS of a DetNet stream, for use by the 583 Controllers to configure policing and shaping engines in Network 584 Systems over the Southbound interface. The model includes: 586 1. A model for queuing, transmission selection, shaping, 587 preemption, and timing resources that can be used by an end 588 system or relay system to control the selection of packets 589 output on an interface. These models must have sufficiently 590 well-defined characteristics, both individually and in the 591 aggregate, to give predictable results for the QoS for DetNet 592 packets (Section 4.6). 594 2. A model for identifying misbehaving DetNet streams and 595 mitigating their impact on properly functioning streams 596 (Section 4.8). 598 c. A model for the relay system to inform the controller(s) of the 599 information it needs for adequate path computations including: 601 1. Systems' individual capabilities (e.g. can do replication, 602 can do precise time). 604 2. Link capabilities and resources (e.g. bandwidth, transmission 605 delay, hardware deterministic support to the physical layer, 606 ...) 608 3. Physical resources (total and available buffers, timers, 609 queues, etc) 611 4. Network Adjacencies (neighbors) 613 d. A model for the provision of a service, by end systems or relay 614 systems, to replicate and forward a DetNet stream over redundant 615 paths. The model includes: 617 1. A model for specifying multiple stable paths (circuits) 618 across a network that can perform packet forwarding at both 619 Layer 3 and at lower layers, to which specific DetNet streams 620 can be assigned. 622 2. A model and data plane format(s) for sequencing and 623 replicating the packets of a DetNet stream, typically at or 624 near the talker, sending the replicated streams over 625 different stable paths, merging and/or re-replicating those 626 packets at other points in the network, and finally 627 eliminating the duplicates, typically at or near the 628 listener(s), in order to provide high availability 629 (Section 3.3). 631 e. The protocol stack model for an end system and/or a relay system 632 should support the above elements in a manner that maximizes the 633 applicability of existing standards and protocols to the DetNet 634 problem, and allows for the creation of new protocols only where 635 needed, thus making DetNet an add-on feature to existing 636 networks, rather than a new way to do networking. In particular 637 this protocol stack supports networks in which the path from 638 talker to listener(s) includes bridges and/or routers in any 639 order (Section 4.9). 641 f. A variety of models for the provisioning of DetNet streams can be 642 envisioned, including orchestration by a central controller or by 643 a federation of controllers, provisioning by relay systems and 644 end systems sharing peer-to-peer protocols, by off-line 645 configuration, or by a combination of these methods. The 646 provisioning models are similar to existing Layer-2 and Layer-3 647 models, in order to minimize the amount of innovation required in 648 this area (Section 4.11). 650 4.5. DetNet streams 652 4.5.1. Talker guarantees 654 DetNet streams can by synchronous or asynchronous. In synchronous 655 DetNet streams, at least the relay systems (and possibly the end 656 systems) are closely time synchronized, typically to better than 1 657 microsecond. By transmitting packets from different streams or 658 classes of streams at different times, using repeating schedules 659 synchronized among the relay systems, resources such as buffers and 660 link bandwidth can be shared over the time domain among different 661 streams. There is a tradeoff among techniques for synchronous 662 streams between the burden of fine-grained scheduling and the benefit 663 of reducing the required resources, especially buffer space. 665 In contrast, asynchronous streams are not coordinated with a fine- 666 grained schedule, so relay and end systems must assume worst-case 667 interference among streams contending for buffer resources. 668 Asynchronous DetNet streams are characterized by: 670 o A maximum packet size; 672 o An observation interval; and 674 o A maximum number of transmissions during that observation 675 interval. 677 These parameters, together with knowledge of the protocol stack used 678 (and thus the size of the various headers added to a packet), limit 679 the number of bit times per observation interval that the DetNet 680 stream can occupy the physical medium. 682 The talker promises that these limits will not be exceeded. If the 683 talker transmits less data than this limit allows, the unused 684 resources such as link bandwidth can be made available by the system 685 to non-DetNet packets. However, making those resources available to 686 DetNet packets in other streams would serve no purpose. Those other 687 streams have their own dedicated resources, on the assumption that 688 all DetNet streams can use all of their resources over a long period 689 of time. 691 Note that there is no provision in DetNet for throttling streams; the 692 assumption is that a DetNet stream, to be useful, must be delivered 693 in its entirety. That is, while any useful application is written to 694 expect a certain number of lost packets, the real-time applications 695 of interest to DetNet demand that the loss of data due to the network 696 is extraordinarily infrequent. 698 Although DetNet strives to minimize the changes required of an 699 application to allow it to shift from a special-purpose digital 700 network to an Internet Protocol network, one fundamental shift in the 701 behavior of network applications is impossible to avoid--the 702 reservation of resources before the application starts. In the first 703 place, a network cannot deliver finite latency and practically zero 704 packet loss to an arbitrarily high offered load. Secondly, achieving 705 practically zero packet loss for unthrottled (though bandwidth 706 limited) streams means that bridges and routers have to dedicate 707 buffer resources to specific streams or to classes of streams. The 708 requirements of each reservation have to be translated into the 709 parameters that control each system's queuing, shaping, and 710 scheduling functions and delivered to the hosts, bridges, and 711 routers. 713 4.5.2. Incomplete Networks 715 The presence in the network of relay systems that are not fully 716 capable of offering DetNet services complicates the ability of the 717 relay systems and/or controller to allocate resources, as extra 718 buffering, and thus extra latency, must be allocated at each point 719 that is downstream from the non-DetNet relay system for some DetNet 720 stream. 722 4.6. Queuing, Shaping, Scheduling, and Preemption 724 As described above, DetNet achieves its aims by reserving bandwidth 725 and buffer resources at every hop along the path of the stream. The 726 reservation itself is not sufficient, however. Implementors and 727 users of a number of proprietary and standard real-time networks have 728 found that standards for specific data plane techniques are required 729 to enable these assurances to be made in a multi-vendor network. The 730 fundamental reason is that latency variation in one system results in 731 the need for extra buffer space in the next-hop system(s), which in 732 turn, increases the worst-case per-hop latency. 734 Standard queuing and transmission selection algorithms allow a 735 central controller to compute the latency contribution of each relay 736 node to the end-to-end latency, to compute the amount of buffer space 737 required in each relay system for each incremental flow, and most 738 importantly, to translate from a flow specification to a set of 739 values for the managed objects that control each relay or end system. 740 The IEEE 802 has specified (and is specifying) a set of queuing, 741 shaping, and scheduling algorithms that enable each relay system 742 (bridge or router), and/or a central controller, to compute these 743 values. These algorithms include: 745 o A credit-based shaper IEEE 802.1Q Clause 34 [IEEE802.1Q-2014]. 747 o Time-gated queues governed by a rotating time schedule, 748 synchronized among all relay nodes [IEEE802.1Qbv]. 750 o Synchronized double (or triple) buffers driven by synchronized 751 time ticks. [IEEE802.1Qch]. 753 o Pre-emption of an Ethernet packet in transmission by a packet with 754 a more stringent latency requirement, followed by the resumption 755 of the preempted packet [IEEE802.1Qbu], [IEEE802.3br]. 757 While these techniques are currently embedded in Ethernet and 758 bridging standards, we can note that they are all, except perhaps for 759 packet preemption, equally applicable to other media than Ethernet, 760 and to routers as well as bridges. 762 4.7. Coexistence with normal traffic 764 A DetNet network supports the dedication of a high proportion (e.g. 765 75%) of the network bandwidth to DetNet streams. But, no matter how 766 much is dedicated for DetNet streams, it is a goal of DetNet to not 767 interfere excessively with existing QoS schemes. It is also 768 important that non-DetNet traffic not disrupt the DetNet stream, of 769 course (see Section 4.8 and Section 6). For these reasons: 771 o Bandwidth (transmission opportunities) not utilized by a DetNet 772 stream are available to non-DetNet packets (though not to other 773 DetNet streams). 775 o DetNet streams can be shaped, in order to ensure that the highest- 776 priority non-DetNet packet also is ensured a worst-case latency 777 (at any given hop). 779 o When transmission opportunities for DetNet streams are scheduled 780 in detail, then the algorithm constructing the schedule should 781 leave sufficient opportunities for non-DetNet packets to satisfy 782 the needs of the uses of the network. 784 Ideally, the net effect of the presence of DetNet streams in a 785 network on the non-DetNet packets is primarily a reduction in the 786 available bandwidth. 788 4.8. Fault Mitigation 790 One key to building robust real-time systems is to reduce the 791 infinite variety of possible failures to a number that can be 792 analyzed with reasonable confidence. DetNet aids in the process by 793 providing filters and policers to detect DetNet packets received on 794 the wrong interface, or at the wrong time, or in too great a volume, 795 and to then take actions such as discarding the offending packet, 796 shutting down the offending DetNet stream, or shutting down the 797 offending interface. 799 It is also essential that filters and service remarking be employed 800 at the network edge to prevent non-DetNet packets from being mistaken 801 for DetNet packets, and thus impinging on the resources allocated to 802 DetNet packets. 804 There exist techniques, at present and/or in various stages of 805 standardization, that can perform these fault mitigation tasks that 806 deliver a high probability that misbehaving systems will have zero 807 impact on well-behaved DetNet streams, except of course, for the 808 receiving interface(s) immediately downstream of the misbehaving 809 device. 811 4.9. Protocol Stack Model 813 [IEEE802.1CB], Annex C, offers a description of the TSN protocol 814 stack. While this standard is a work in progress, a consensus around 815 the basic architecture has formed. This stack is summarized in 816 Figure 4. 818 DetNet Protocol Stack 820 +--------------------------------+ 821 | Upper Layers | 822 +--------------------------------+ 823 | Sequence generation/recovery | 824 +--------------------------------+ 825 | Sequence encode/decode | 826 +--------------------------------+ 827 | Stream splitting/merging | 828 +--------------------------------+ 829 | Stream encode/decode | 830 +--------------------------------+ 831 | Lower layers | 832 +--------------------------------+ 834 Figure 4 836 Not all layers are required for any given application, or even for 837 any given network. The layers are, from top to bottom: 839 Sequence generation/recovery 840 Supplies the sequence number for Seamless Redundancy 841 (Section 3.3) for packets going down the stack, and discards 842 duplicate packets coming up the stack. 844 Sequence encode/decode 845 Encodes the sequence number into packets going down the 846 stack, and extracts the sequence number from packets coming 847 up the stack. This function may or may not be a null 848 transformation of the packet, and for some protocols, is not 849 explicitly present, being included in the Stream encode/ 850 decode layer, below. 852 Stream splitting/merging 853 Replicates packets going down the stack into two streams, and 854 merges streams together for packets coming up the stack, 855 based on the packet's stream identifier. Needed for Seamless 856 Redundancy (Section 3.3). 858 Stream encode/decode 859 Encapsulates packets going down the stack, based on the 860 packet's locally-significant stream identifier, in order to 861 identify to which stream the packet belongs, and extracts a 862 locally-significant stream identifier from packets coming up 863 the stack. This may be a null transformation (e.g., for 864 streams identified by IP 5-tuple) or might be an explicit 865 encapsulation (e.g., for streams identified with an MPLS 866 label). Stream identification is the basis for Seamless 867 Redundancy, for assigning per-flow resources (if any) to 868 packets and for defence against misbehaving systems 869 (Section 4.8). When streams are assigned to pinned-down 870 paths, this layer can be indistinguishable from the data 871 forwarding layer(s). 873 The reader is likely to notice that Figure 4 does not specify the 874 relationship between the DetNet layers, the IP layers, and the link 875 layers. This is intentional, because they can usefully be placed 876 different places in the stack, and even in mulitple places, depending 877 on where their peers are placed. 879 4.10. Advertising resources, capabilities and adjacencies 881 There are three classes of information that a central controller 882 needs to know that can only be obtained from the end systems and/or 883 relay systems in the network. When using a peer-to-peer control 884 plane, some of this information may be required by a system's 885 neighbors in the network. 887 o Details of the system's capabilities that are required in order to 888 accurately allocate that system's resources, as well as other 889 systems' resources. This includes, for example, which specific 890 queuing and shaping algorithms are implemented (Section 4.6), the 891 number of buffers dedicated for DetNet allocation, and the worst- 892 case forwarding delay. 894 o The dynamic state of an end or relay system's DetNet resources. 896 o The identity of the system's neighbors, and the characteristics of 897 the link(s) between the systems, including the length (in 898 nanoseconds) of the link(s). 900 4.11. Provisioning model 902 4.11.1. Centralized Path Computation and Installation 904 A centralized routing model, such as provided with a PCE (RFC 4655 905 [RFC4655]), enables global and per-stream optimizations. The model 906 is attractive but a number of issues are left to be solved. In 907 particular: 909 o whether and how the path computation can be installed by 1) an end 910 device or 2) a Network Management entity, 912 o and how the path is set up, either by installing state at each hop 913 with a direct interaction between the forwarding device and the 914 PCE, or along a path by injecting a source-routed request at one 915 end of the path. 917 4.11.2. Distributed Path Setup 919 Whether a distributed alternative without a PCE can be valuable 920 should be studied as well. Such an alternative could for instance 921 inherit from the Resource ReSerVation Protocol [RFC5127] (RSVP) 922 flows. 924 In a Layer-2 only environment, or as part of a layered approach to a 925 mixed environment, IEEE 802.1 also has work, either completed or in 926 progress. [IEEE802.1Q-2014] Clause 35 describes SRP, a peer-to-peer 927 protocol for Layer-2 roughly analogous to RSVP. Almost complete is 928 [IEEE802.1Qca], which defines how ISIS can provide multiple disjoint 929 paths or distribution trees. Also in progress is [IEEE802.1Qcc], 930 which expands the capabilities of SRP. 932 5. Related IETF work 934 5.1. Deterministic PHB 936 [I-D.svshah-tsvwg-deterministic-forwarding] defines a Differentiated 937 Services Per-Hop-Behavior (PHB) Group called Deterministic Forwarding 938 (DF). The document describes the purpose and semantics of this PHB. 939 It also describes creation and forwarding treatment of the service 940 class. The document also describes how the code-point can be mapped 941 into one of the aggregated Diffserv service classes [RFC5127]. 943 5.2. 6TiSCH 945 Industrial process control already leverages deterministic wireless 946 Low power and Lossy Networks (LLNs) to interconnect critical 947 resource-constrained devices and form wireless mesh networks, with 948 standards such as [ISA100.11a] and [WirelessHART]. 950 These standards rely on variations of the [IEEE802154e] timeSlotted 951 Channel Hopping (TSCH) [I-D.ietf-6tisch-tsch] Medium Access Control 952 (MAC), and a form of centralized Path Computation Element (PCE), to 953 deliver deterministic capabilities. 955 The TSCH MAC benefits include high reliability against interference, 956 low power consumption on characterized streams, and Traffic 957 Engineering capabilities. Typical applications are open and closed 958 control loops, as well as supervisory control streams and management. 960 The 6TiSCH Working Group focuses only on the TSCH mode of the IEEE 961 802.15.4e standard. The WG currently defines a framework for 962 managing the TSCH schedule. Future work will standardize 963 deterministic operations over so-called tracks as described in 964 [I-D.ietf-6tisch-architecture]. Tracks are an instance of a 965 deterministic path, and the DetNet work is a prerequisite to specify 966 track operations and serve process control applications. 968 [RFC5673] and [I-D.ietf-roll-rpl-industrial-applicability] section 969 2.1.3. and next discusses application-layer paradigms, such as 970 Source-sink (SS) that is a Multipeer to Multipeer (MP2MP) model that 971 is primarily used for alarms and alerts, Publish-subscribe (PS, or 972 pub/sub) that is typically used for sensor data, as well as Peer-to- 973 peer (P2P) and Peer-to-multipeer (P2MP) communications. Additional 974 considerations on Duocast and its N-cast generalization are also 975 provided for improved reliability. 977 6. Security Considerations 979 Security in the context of Deterministic Networking has an added 980 dimension; the time of delivery of a packet can be just as important 981 as the contents of the packet, itself. A man-in-the-middle attack, 982 for example, can impose, and then systematically adjust, additional 983 delays into a link, and thus disrupt or subvert a real-time 984 application without having to crack any encryption methods employed. 985 See [RFC7384] for an exploration of this issue in a related context. 987 Furthermore, in a control system where millions of dollars of 988 equipment, or even human lives, can be lost if the DetNet QoS is not 989 delivered, one must consider not only simple equipment failures, 990 where the box or wire instantly becomes perfectly silent, but bizarre 991 errors such as can be caused by software failures. Because there is 992 essential no limit to the kinds of failures that can occur, 993 protecting against realistic equipment failures is indistinguishable, 994 in most cases, from protecting against malicious behavior, whether 995 accidental or intentional. See also Section 4.8. 997 Security must cover: 999 o the protection of the signaling protocol 1001 o the authentication and authorization of the controlling systems 1003 o the identification and shaping of the streams 1005 7. IANA Considerations 1007 This document does not require an action from IANA. 1009 8. Acknowledgements 1011 The authors wish to thank Jouni Korhonen, Erik Nordmark, George 1012 Swallow, Rudy Klecka, Anca Zamfir, David Black, Thomas Watteyne, 1013 Shitanshu Shah, Craig Gunther, Rodney Cummings, Wilfried Steiner, 1014 Marcel Kiessling, Karl Weber, Ethan Grossman and Pat Thaler, for 1015 their various contribution with this work. 1017 9. Access to IEEE 802.1 documents 1019 To access password protected IEEE 802.1 drafts, see the IETF IEEE 1020 802.1 information page at https://www.ietf.org/proceedings/52/slides/ 1021 bridge-0/tsld003.htm. 1023 10. Informative References 1025 [AVnu] http://www.avnu.org/, "The AVnu Alliance tests and 1026 certifies devices for interoperability, providing a simple 1027 and reliable networking solution for AV network 1028 implementation based on the Audio Video Bridging (AVB) 1029 standards.". 1031 [CCAMP] IETF, "Common Control and Measurement Plane", 1032 . 1034 [HART] www.hartcomm.org, "Highway Addressable Remote Transducer, 1035 a group of specifications for industrial process and 1036 control devices administered by the HART Foundation". 1038 [HSR-PRP] IEC, "High availability seamless redundancy (HSR) is a 1039 further development of the PRP approach, although HSR 1040 functions primarily as a protocol for creating media 1041 redundancy while PRP, as described in the previous 1042 section, creates network redundancy. PRP and HSR are both 1043 described in the IEC 62439 3 standard.", 1044 . 1047 [I-D.finn-detnet-problem-statement] 1048 Finn, N. and P. Thubert, "Deterministic Networking Problem 1049 Statement", draft-finn-detnet-problem-statement-04 (work 1050 in progress), October 2015. 1052 [I-D.ietf-6tisch-architecture] 1053 Thubert, P., "An Architecture for IPv6 over the TSCH mode 1054 of IEEE 802.15.4", draft-ietf-6tisch-architecture-08 (work 1055 in progress), May 2015. 1057 [I-D.ietf-6tisch-tsch] 1058 Watteyne, T., Palattella, M., and L. Grieco, "Using 1059 IEEE802.15.4e TSCH in an IoT context: Overview, Problem 1060 Statement and Goals", draft-ietf-6tisch-tsch-06 (work in 1061 progress), March 2015. 1063 [I-D.ietf-roll-rpl-industrial-applicability] 1064 Phinney, T., Thubert, P., and R. Assimiti, "RPL 1065 applicability in industrial networks", draft-ietf-roll- 1066 rpl-industrial-applicability-02 (work in progress), 1067 October 2013. 1069 [I-D.svshah-tsvwg-deterministic-forwarding] 1070 Shah, S. and P. Thubert, "Deterministic Forwarding PHB", 1071 draft-svshah-tsvwg-deterministic-forwarding-04 (work in 1072 progress), August 2015. 1074 [IEEE802.1AS-2011] 1075 IEEE, "Timing and Synchronizations (IEEE 802.1AS-2011)", 1076 2011, . 1079 [IEEE802.1BA-2011] 1080 IEEE, "AVB Systems (IEEE 802.1BA-2011)", 2011, 1081 . 1084 [IEEE802.1CB] 1085 IEEE, "Seamless Redundancy (IEEE Draft P802.1CB)", 2015, 1086 . 1088 [IEEE802.1Q-2014] 1089 IEEE, "MAC Bridges and VLANs (IEEE 802.1Q-2014", 2014, 1090 . 1093 [IEEE802.1Qbu] 1094 IEEE, "Frame Preemption", 2015, 1095 . 1097 [IEEE802.1Qbv] 1098 IEEE, "Enhancements for Scheduled Traffic", 2015, 1099 . 1101 [IEEE802.1Qca] 1102 IEEE, "Path Control and Reservation", 2015, 1103 . 1105 [IEEE802.1Qcc] 1106 IEEE, "Stream Reservation Protocol (SRP) Enhancements and 1107 Performance Improvements", 2015, 1108 . 1110 [IEEE802.1Qch] 1111 IEEE, "Cyclic Queuing and Forwarding", 2011, 1112 . 1114 [IEEE802.1TSNTG] 1115 IEEE Standards Association, "IEEE 802.1 Time-Sensitive 1116 Networks Task Group", 2013, 1117 . 1119 [IEEE802.3br] 1120 IEEE, "Interspersed Express Traffic", 2015, 1121 . 1123 [IEEE802154] 1124 IEEE standard for Information Technology, "IEEE std. 1125 802.15.4, Part. 15.4: Wireless Medium Access Control (MAC) 1126 and Physical Layer (PHY) Specifications for Low-Rate 1127 Wireless Personal Area Networks", June 2011. 1129 [IEEE802154e] 1130 IEEE standard for Information Technology, "IEEE std. 1131 802.15.4e, Part. 15.4: Low-Rate Wireless Personal Area 1132 Networks (LR-WPANs) Amendment 1: MAC sublayer", April 1133 2012. 1135 [ISA100.11a] 1136 ISA/IEC, "ISA100.11a, Wireless Systems for Automation, 1137 also IEC 62734", 2011, < http://www.isa100wci.org/en- 1138 US/Documents/PDF/3405-ISA100-WirelessSystems-Future-broch- 1139 WEB-ETSI.aspx>. 1141 [ISA95] ANSI/ISA, "Enterprise-Control System Integration Part 1: 1142 Models and Terminology", 2000, . 1145 [ODVA] http://www.odva.org/, "The organization that supports 1146 network technologies built on the Common Industrial 1147 Protocol (CIP) including EtherNet/IP.". 1149 [PCE] IETF, "Path Computation Element", 1150 . 1152 [Profinet] 1153 http://us.profinet.com/technology/profinet/, "PROFINET is 1154 a standard for industrial networking in automation.", 1155 . 1157 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1158 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1159 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1160 September 1997, . 1162 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1163 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1164 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 1165 . 1167 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation 1168 Element (PCE)-Based Architecture", RFC 4655, 1169 DOI 10.17487/RFC4655, August 2006, 1170 . 1172 [RFC5127] Chan, K., Babiarz, J., and F. Baker, "Aggregation of 1173 Diffserv Service Classes", RFC 5127, DOI 10.17487/RFC5127, 1174 February 2008, . 1176 [RFC5673] Pister, K., Ed., Thubert, P., Ed., Dwars, S., and T. 1177 Phinney, "Industrial Routing Requirements in Low-Power and 1178 Lossy Networks", RFC 5673, DOI 10.17487/RFC5673, October 1179 2009, . 1181 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in 1182 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384, 1183 October 2014, . 1185 [RFC7426] Haleplidis, E., Ed., Pentikousis, K., Ed., Denazis, S., 1186 Hadi Salim, J., Meyer, D., and O. Koufopavlou, "Software- 1187 Defined Networking (SDN): Layers and Architecture 1188 Terminology", RFC 7426, DOI 10.17487/RFC7426, January 1189 2015, . 1191 [TEAS] IETF, "Traffic Engineering Architecture and Signaling", 1192 . 1194 [WirelessHART] 1195 www.hartcomm.org, "Industrial Communication Networks - 1196 Wireless Communication Network and Communication Profiles 1197 - WirelessHART - IEC 62591", 2010. 1199 Authors' Addresses 1201 Norman Finn 1202 Cisco Systems 1203 170 W Tasman Dr. 1204 San Jose, California 95134 1205 USA 1207 Phone: +1 408 526 4495 1208 Email: nfinn@cisco.com 1210 Pascal Thubert 1211 Cisco Systems 1212 Village d'Entreprises Green Side 1213 400, Avenue de Roumanille 1214 Batiment T3 1215 Biot - Sophia Antipolis 06410 1216 FRANCE 1218 Phone: +33 4 97 23 26 34 1219 Email: pthubert@cisco.com 1220 Michael Johas Teener 1221 Broadcom Corp. 1222 3151 Zanker Rd. 1223 San Jose, California 95134 1224 USA 1226 Phone: +1 831 824 4228 1227 Email: MikeJT@broadcom.com