idnits 2.17.1
draft-finn-detnet-architecture-00.txt:
Checking boilerplate required by RFC 5378 and the IETF Trust (see
https://trustee.ietf.org/license-info):
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt:
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/checklist :
----------------------------------------------------------------------------
No issues found here.
Miscellaneous warnings:
----------------------------------------------------------------------------
== The copyright year in the IETF Trust and authors Copyright Line does not
match the current year
-- The document date (March 9, 2015) is 3335 days in the past. Is this
intentional?
Checking references for intended status: Proposed Standard
----------------------------------------------------------------------------
(See RFCs 3967 and 4897 for information about using normative references
to lower-maturity documents in RFCs)
== Unused Reference: 'AVnu' is defined on line 858, but no explicit
reference was found in the text
== Unused Reference: 'HART' is defined on line 867, but no explicit
reference was found in the text
== Unused Reference: 'I-D.finn-detnet-problem-statement' is defined on line
880, but no explicit reference was found in the text
== Unused Reference: 'IEEE802.1AS-2011' is defined on line 908, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802.1Qat-2010' is defined on line 928, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802.1Qav' is defined on line 933, but no explicit
reference was found in the text
== Unused Reference: 'IEEE802.1TSNTG' is defined on line 949, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802154' is defined on line 954, but no explicit
reference was found in the text
== Unused Reference: 'ODVA' is defined on line 972, but no explicit
reference was found in the text
== Unused Reference: 'Profinet' is defined on line 979, but no explicit
reference was found in the text
== Unused Reference: 'RFC2205' is defined on line 984, but no explicit
reference was found in the text
== Outdated reference: A later version (-05) exists of
draft-finn-detnet-problem-statement-01
== Outdated reference: A later version (-30) exists of
draft-ietf-6tisch-architecture-05
== Outdated reference: A later version (-06) exists of
draft-ietf-6tisch-tsch-05
== Outdated reference: A later version (-04) exists of
draft-svshah-tsvwg-deterministic-forwarding-03
Summary: 0 errors (**), 0 flaws (~~), 16 warnings (==), 1 comment (--).
Run idnits with the --verbose option for more detailed information about
the items above.
--------------------------------------------------------------------------------
2 DetNet N. Finn
3 Internet-Draft P. Thubert
4 Intended status: Standards Track Cisco
5 Expires: September 10, 2015 March 9, 2015
7 Deterministic Networking Architecture
8 draft-finn-detnet-architecture-00
10 Abstract
12 Deterministic Networking (DetNet) provides a capability to carry
13 specified unicast or multicast data streams for real-time
14 applications with extremely low data loss rates and maximum latency.
15 Techniques used include: 1) reserving data plane resources for
16 individual (or aggregated) DetNet streams in some or all of the relay
17 systems (bridges or routers) along the path of the stream; 2)
18 providing fixed paths for DetNet streams that do not rapidly change
19 with the network topology; and 3) sequentializing, replicating, and
20 eliminating duplicate packets at various points to ensure the
21 availability of at least one path. The capabilities can be managed
22 by configuration, or by manual or automatic network management.
24 Status of This Memo
26 This Internet-Draft is submitted in full conformance with the
27 provisions of BCP 78 and BCP 79.
29 Internet-Drafts are working documents of the Internet Engineering
30 Task Force (IETF). Note that other groups may also distribute
31 working documents as Internet-Drafts. The list of current Internet-
32 Drafts is at http://datatracker.ietf.org/drafts/current/.
34 Internet-Drafts are draft documents valid for a maximum of six months
35 and may be updated, replaced, or obsoleted by other documents at any
36 time. It is inappropriate to use Internet-Drafts as reference
37 material or to cite them other than as "work in progress."
39 This Internet-Draft will expire on September 10, 2015.
41 Copyright Notice
43 Copyright (c) 2015 IETF Trust and the persons identified as the
44 document authors. All rights reserved.
46 This document is subject to BCP 78 and the IETF Trust's Legal
47 Provisions Relating to IETF Documents
48 (http://trustee.ietf.org/license-info) in effect on the date of
49 publication of this document. Please review these documents
50 carefully, as they describe your rights and restrictions with respect
51 to this document. Code Components extracted from this document must
52 include Simplified BSD License text as described in Section 4.e of
53 the Trust Legal Provisions and are provided without warranty as
54 described in the Simplified BSD License.
56 Table of Contents
58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
59 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
60 3. Providing the DetNet Quality of Service . . . . . . . . . . . 5
61 3.1. Zero Congestion Loss . . . . . . . . . . . . . . . . . . 6
62 3.2. Pinned-down paths . . . . . . . . . . . . . . . . . . . . 7
63 3.3. Seamless Redundancy . . . . . . . . . . . . . . . . . . . 7
64 4. DetNet Architecture . . . . . . . . . . . . . . . . . . . . . 8
65 4.1. The Application Plane . . . . . . . . . . . . . . . . . . 11
66 4.2. The Controller Plane . . . . . . . . . . . . . . . . . . 11
67 4.3. The Network Plane . . . . . . . . . . . . . . . . . . . . 12
68 4.4. Elements of DetNet Architecture . . . . . . . . . . . . . 13
69 4.5. DetNet streams . . . . . . . . . . . . . . . . . . . . . 14
70 4.5.1. Talker guarantees . . . . . . . . . . . . . . . . . . 14
71 4.5.2. Incomplete Networks . . . . . . . . . . . . . . . . . 15
72 4.6. Data Flow Model through Systems . . . . . . . . . . . . . 16
73 4.7. Queuing, Shaping, Scheduling, and Preemption . . . . . . 16
74 4.8. Coexistence with normal traffic . . . . . . . . . . . . . 16
75 4.9. Fault Mitigation . . . . . . . . . . . . . . . . . . . . 16
76 4.10. Protocol Stack Model . . . . . . . . . . . . . . . . . . 17
77 4.11. Advertising resources, capabilities and adjacencies . . . 17
78 4.12. Provisioning model . . . . . . . . . . . . . . . . . . . 17
79 4.12.1. Centralized Path Computation and Installation . . . 17
80 4.12.2. Distributed Path Setup . . . . . . . . . . . . . . . 17
81 5. Related IETF work . . . . . . . . . . . . . . . . . . . . . . 18
82 5.1. Deterministic PHB . . . . . . . . . . . . . . . . . . . . 18
83 5.2. 6TiSCH . . . . . . . . . . . . . . . . . . . . . . . . . 18
84 6. Security Considerations . . . . . . . . . . . . . . . . . . . 19
85 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19
86 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 19
87 9. Informative References . . . . . . . . . . . . . . . . . . . 19
88 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 23
90 1. Introduction
92 Operational Technology (OT) refers to industrial networks that are
93 typically used for monitoring systems and supporting control loops,
94 as well as movement detection systems for use in process control
95 (i.e., process manufacturing) and factory automation (i.e., discrete
96 manufacturing). Due to its different goals, OT has evolved in
97 parallel but in a manner that is radically different from IT/ICT,
98 focusing on highly secure, reliable and deterministic networks, with
99 limited scalability over a bounded area.
101 The convergence of IT and OT technologies, also called the Industrial
102 Internet, represents a major evolution for both sides. The work has
103 already started; in particular, the industrial automation space has
104 been developing a number of Ethernet-based replacements for existing
105 digital control systems, often not packet-based (fieldbus
106 technologies).
108 These replacements are meant to provide similar behavior as the
109 incumbent protocols, and their common focus is to transport a fully
110 characterized flow over a well-controlled environment (i.e., a
111 factory floor), with a bounded latency, extraordinarily low frame
112 loss, and a very narrow jitter. Examples of such protocols include
113 PROFINET, ODVA Ethernet/IP, and EtherCAT.
115 In parallel, the need for determinism in professional and home audio/
116 video markets drove the formation of the Audio/Video Bridging (AVB)
117 standards effort of IEEE 802.1. With the explosion of demand for
118 connectivity and multimedia in transportation in general, the
119 Ethernet AVB technology has become one of the hottest topics, in
120 particular in the automotive connectivity. It is finding application
121 in all elements of the vehicle from head units, to rear seat
122 entertainment modules, to amplifiers and camera modules. While aimed
123 at less critical applications than some industrial networks, AVB
124 networks share the requirement for extremely low packet loss rates
125 and ensured finite latency and jitter.
127 Other instances of in-vehicle deterministic networks have arisen as
128 well for control networks in cars, trains and buses, as well as
129 avionics, with, for instance, the mission-critical "Avionics Full-
130 Duplex Switched Ethernet" (AFDX) that was designed as part of the
131 ARINC 664 standards. Existing automotive control networks such as
132 the LIN, CAN and FlexRay standards were not designed to cover these
133 increasing demands in terms of bandwidth and scalability that we see
134 with various kinds of Driver Assistance Systems (DAS) and new
135 multiplexing technologies based on Ethernet are now getting traction.
137 The generalization of the needs for more deterministic networks have
138 led to the IEEE 802.1 AVB Task Group becoming the Time-Sensitive
139 Networking (TSN) Task Group (TG), with a much-expanded constituency
140 from the industrial and vehicular markets. Along with this
141 expansion, the networks in consideration are becoming larger and
142 structured, requiring deterministic forwarding beyond the LAN
143 boundaries. For instance, Industrial Automation segregates the
144 network along the broad lines of the Purdue Enterprise Reference
145 Architecture (PERA), using different technologies at each level, and
146 public infrastructures such as Electricity Automation require
147 deterministic properties over the Wide Area. The realization is now
148 coming that the convergence of IT and OT networks requires Layer-3,
149 as well as Layer-2, capabilities.
151 The present architecture is the result of a collaboration of the IETF
152 and the IEEE and implements an abstract model that can be applicable
153 both at Layer-2 and Layer-3, and along segments of different
154 technologie. With this new work, a path may span, for instance,
155 across a (limited) number of 802.1 bridges and then a (limited)
156 number of IP routers. In that example, the IEEE 802.1 bridges may be
157 operating at Layer-2 over Ethernet whereas the IP routers may be
158 6TiSCH nodes operating at Layer-2 and/or Layer-3 over the IEEE
159 802.15.4e MAC.
161 Many applications of interest to Deterministic Networking require the
162 ability to synchronize the clocks in end systems to a sub-microsecond
163 accuracy. Some of the queue control techniques defined in
164 Section 4.7 also require time synchronization among relay systems.
165 The means used to achieve time synchronization are not addressed in
166 this document.
168 2. Terminology
170 The follwing special terms are used in this document in order to
171 avoid the assumption that a given element in the archetecture does or
172 does not have Internet Protocol stack, functions as a router or a
173 bridge, or otherwise plays a particular role at Layer-3 or higher:
175 bridge
176 A Customer Bridge as defined by IEEE 802.1Q
177 [IEEE802.1Q-2011].
179 circuit
180 A trail of configuration from talker to listener(s) through
181 relay systems associated with a DetNet stream, required to
182 deliver the benefits of DetNet.
184 end system
185 Commonly called a "host" in IETF documents, and an "end
186 station" is IEEE 802 documents. End systems of interest to
187 this document are talkers and listeners.
189 listener
190 An end system capable of sinking a DetNet stream.
192 relay system
193 A router or a bridge.
195 stream
196 A DetNet stream is a sequence of packets from a single
197 talker, through some number of relay systems to one or more
198 listeners, that is limited by the talker in its maximum
199 packet size and transmission rate, and can thus be ensured
200 the DetNet Quality of Service (QoS) from the network.
202 talker
203 An end system capable of sourcing a DetNet stream.
205 3. Providing the DetNet Quality of Service
207 DetNet Quality of Service is expressed in terms of:
209 o Minimum and maximum end-to-end latency from talker to listener;
211 o Probability of loss of a packet, assuming the normal operation of
212 the relay systems and links;
214 o Probability of loss of a packet in the event of the failure of a
215 relay system or link.
217 It is a distinction of DetNet that it is concerned solely with worst-
218 case values for all of the above parameters. Average, mean, or
219 typical values are of no interest, because they do not affect the
220 ability of a real-time system to perform its tasks.
222 Three techniques are employed by DetNet to achieve these QoS
223 parameters:
225 a. Zero congestion loss (Section 3.1). Network resources such as
226 link bandwidth, buffers, queues, shapers, and scheduled input/
227 output slots are assigned in each relay system to the use of a
228 specific DetNet stream or group of streams. Note that, given a
229 finite amount of buffer space), zero congestion loss necessarily
230 ensures a maximum end-to-end latency. Depending on the method
231 employed, a minimum latency can also be achieved.
233 b. Pinned-down paths (Section 3.2). Point-to-point paths or point-
234 to-multipoint trees through the network from a talker to one or
235 more listeners can be established, and DetNet streams assigned to
236 follow a particular path or tree.
238 c. Packet replication and deletion (Section 3.3). End systems and/
239 or relay systems can sequence number, replicate, and eliminate
240 replicated packets at multiple points in the network in order to
241 ensure that one (or more) equipment failure events still leave at
242 least one path intact for a DetNet stream.
244 These three techniques can be applied independently, giving eight
245 possible combinations, including none (no DetNet), although some
246 combinations are of wider utility than others. This separation keeps
247 the protocol stack coherent and maximizes interoperability with
248 existing and developing standards in this (IETF) and other Standards
249 Development Organizations. Some examples of typical expected
250 combinations:
252 o Pinned-down paths (a) plus packet replication (b) are exactly the
253 techniques employed by [HSR-PRP]. Pinned-down paths are achieve
254 by limiting the physical topology of the network, and the
255 sequentialization, replication, and duplicate elimination
256 facilitated by packet tags added at the front or the end of
257 Ethernet frames.
259 o Zero congestion loss (a) alone is is offered by IEEE 802.1 Audio
260 Video bridging [IEEE802.1BA-2011]. As long as the network suffers
261 no failures, near-zero (at best, zero) congestion loss can be
262 achieved through the use of a reservation protocol (MSRP) and
263 shapers in every relay system (bridge).
265 o Using all three together gives maximum protection.
267 There are, of course, simpler methods available (and employed, today)
268 to achieve levels of latency and packet loss that are satisfactory
269 for many applications. However, these methods generally work best in
270 the absence of any significant amount of non-critical traffic in the
271 network (if, indeed, such traffic is supported at all), or work only
272 if the critical traffic constitutes only a small portion of the
273 network's theoretical capacity, or work only if all systems are
274 functioning properly, or in the absence of actions by end systems
275 that disrupt the network's operations.
277 There are any number of methods in use, defined, or in progress for
278 accomplishing each of the above techniques. It is expected that this
279 DetNet Architecture will assist various vendors, users, and/or
280 "vertical" Standards Development Organizations (dedicated to a single
281 industry) to make selections among the available means of
282 implementing DetNet networks.
284 3.1. Zero Congestion Loss
286 The primary means by which DetNet achieves its QoS assurances is to
287 completely eliminate congestion at an output port as a cause of
288 packet loss. Given that a DetNet stream cannot be throttled, this
289 can be achieved only by the provision of sufficient buffer storage at
290 each hop through the network to ensure that no packets are dropped
291 due to a lack of buffer storage.
293 Ensuring adequate buffering requires, in turn, that the talker, and
294 every relay system along the path to the listener (or nearly every
295 relay system -- see Section 4.5.2) be careful to regulate its output
296 to not exceed the data rate for any stream, except for brief perios
297 when making up for interfering traffic. Any packet sent ahead of its
298 time potentially adds to the number of buffers required by the next
299 hop, and may thus exceed the resources allocated for a particular
300 stream.
302 The low-level mechanisms described in Section 4.7 provide the
303 necessary regulation of transmissions by an edge system or relay
304 system to ensure zero congestion loss. Of course, the reservation of
305 the bandwidth and buffers for a stream requires the provisioning
306 described in Section 4.12.
308 3.2. Pinned-down paths
310 In networks controlled by typical peer-to-peer protocols such as IEEE
311 802.1 ISIS bridged networks or ETF OSPF routed networks, a network
312 topology event in one part of the network can impact, at least
313 briefly, the delivery of data in parts of the network remote from the
314 failure or recovery event. Thus, even redundant paths through a
315 network, if controlled by the typical peer-to-peer protocols, do not
316 eliminate the chances of brief losses of contact. For this reason,
317 many real-time networks rely on physical rings of two-port devices,
318 with a relatively simple ring control protocol. This both minimizes
319 recovery time and easily supports redundant paths. Of course, this
320 comes at the cost of increased hop count, and thus latency, for the
321 typical path.
323 In order to get the advantages of low hop count and still ensure
324 against even brief losses of connectivity, DetNet employs pinned-down
325 paths, where the path taken by a given DetNet stream does not change,
326 at least immediately, and likely not at all, in response to network
327 topology events. When combined with seamless redundancy
328 (Section 3.3), this results in a high likelihood of continuous
329 connectivity.
331 3.3. Seamless Redundancy
333 After congestion loss has been eliminated, the most important causes
334 of packet loss are random media and/or memory faults and equipment
335 failures.
337 Seamless redundancy involves three capabilities:
339 o Adding sequence numbers to the packets of a DetNet stream.
341 o Replicating these packets and, typically, sending them along at
342 least two different paths to the listener(s).
344 o Discarding duplicated packets.
346 In the simplest case, this amounts to replicating each packet in a
347 talker that has two interfaces, and conveying them through the
348 network, along separate paths, to the similarly dual-homed listeners,
349 that discard the extras. This ensures that one path (with zero
350 congestion loss) remains, even if some relay system fails.
352 Alternatively, relay systems in the network can provide replication
353 and elimination facilities at various points in the network, so that
354 multiple failures can be accommodated.
356 This is shown in the following figure, where the two relay systems
357 each replicate (R) the DetNet stream on input, sending the stream to
358 both the other relay system and to the end system, and eliminated
359 duplicates (E) on the output interface to the right-hand end system.
360 Any one links in the network can fail, and the Detnet stream can
361 still get through. Furthermore, two links can fail, as long as they
362 are in different segments of the network.
364 > > > > > > > > relay > > > > > > > >
365 > /------------+ R system E +------------\ >
366 > / v + ^ \ >
367 end R + v | ^ + E end
368 system + v | ^ + system
369 > \ v + ^ / >
370 > \------------+ R relay E +------------/ >
371 > > > > > > > > system > > > > > > > >
373 Figure 1
375 4. DetNet Architecture
377 Traffic Engineering Architecture and Signaling (TEAS) [TEAS] defines
378 traffic-engineering architectures for generic applicability across
379 packet and non-packet networks. From TEAS perspective, Traffic
380 Engineering (TE) refers to techniques that enable operators to
381 control how specific traffic flows are treated within their networks.
383 Because if its very nature of establishing pinned-down optimized
384 paths, Deterministic Networking can be seen as a new, specialized
385 branch of Traffic Engineering, and inherits its architecture with a
386 separation into planes.
388 The Deterministic Networking architecture is thus composed of three
389 planes, a (User) Application Plane, a Controller Plane, and a Network
390 Plane, which echoes that of Software-Defined Networking (SDN): Layers
391 and Architecture Terminology [RFC7426] which is represented below:
393 SDN Layers and Architecture Terminology per RFC 7426
395 o--------------------------------o
396 | |
397 | +-------------+ +----------+ |
398 | | Application | | Service | |
399 | +-------------+ +----------+ |
400 | Application Plane |
401 o---------------Y----------------o
402 |
403 *-----------------------------Y---------------------------------*
404 | Network Services Abstraction Layer (NSAL) |
405 *------Y------------------------------------------------Y-------*
406 | |
407 | Service Interface |
408 | |
409 o------Y------------------o o---------------------Y------o
410 | | Control Plane | | Management Plane | |
411 | +----Y----+ +-----+ | | +-----+ +----Y----+ |
412 | | Service | | App | | | | App | | Service | |
413 | +----Y----+ +--Y--+ | | +--Y--+ +----Y----+ |
414 | | | | | | | |
415 | *----Y-----------Y----* | | *---Y---------------Y----* |
416 | | Control Abstraction | | | | Management Abstraction | |
417 | | Layer (CAL) | | | | Layer (MAL) | |
418 | *----------Y----------* | | *----------Y-------------* |
419 | | | | | |
420 o------------|------------o o------------|---------------o
421 | |
422 | CP | MP
423 | Southbound | Southbound
424 | Interface | Interface
425 | |
426 *------------Y---------------------------------Y----------------*
427 | Device and resource Abstraction Layer (DAL) |
428 *------------Y---------------------------------Y----------------*
429 | | | |
430 | o-------Y----------o +-----+ o--------Y----------o |
431 | | Forwarding Plane | | App | | Operational Plane | |
432 | o------------------o +-----+ o-------------------o |
433 | Network Device |
434 +---------------------------------------------------------------+
436 Figure 2
438 4.1. The Application Plane
440 Per [RFC7426], the Application Plane includes both applications and
441 services. In particular, the Application Plane incorporates the User
442 Agent, a specialized application that interacts with the end user /
443 operator and performs requests for Deterministic Networking services
444 via an abstract Stream Management Entity, (SME) which may or may not
445 be collocated with (one of) the end systems.
447 At the Application Plane, a management interface enables the
448 negotiation of streams between end systems. An abstraction of the
449 stream called a Traffic Specification (TSpec) provides the
450 representation. This abstraction is used to place a reservation over
451 the (Northbound) Service Interface and within the Application plane.
452 It is associated with an abstraction of location, such as IP
453 addresses and DNS names, to identify the end systems and eventually
454 specify intermediate relay systems.
456 4.2. The Controller Plane
458 The Controller Plane corresponds to the aggregation of the Control
459 and Management Planes in [RFC7426], though Common Control and
460 Measurement Plane (CCAMP) [CCAMP] makes an additional distinction
461 between management and measurement. When the logical separation of
462 the Control, Measurement and other Management entities is not
463 relevant, the term Controller Plane is used for simplicity to
464 represent them all, and the term controller refers to any device
465 operating in that plane, whether is it a Path Computation entity or a
466 Network Management entity (NME). The Path Computation Element (PCE)
467 [PCE] is a core element of a controller, in charge of computing
468 Deterministic paths to be applied in the Network Plane.
470 A (Northbound) Service Interface enables applications in the
471 Application Plane to communicate with the entities in the Controller
472 Plane.
474 One or more PCE(s) collaborate to implement the requests from the SME
475 as Per-Stream Per-Hop Behaviors installed in the relay systems for
476 each individual streams. The PCEs place each stream along a
477 deterministic sequence of relay systems so as to respect per-stream
478 constraints such as security and latency, and optimize the overall
479 result for metrics such as an abstract aggregated cost. The
480 deterministic sequence can typically be more complex than a direct
481 sequence and include redundancy path, with one or more packet
482 replication and elimination points.
484 4.3. The Network Plane
486 The Network Plane represents the network devices and protocols as a
487 whole, regardless of the Layer at which the network devices operate.
489 The network Plane comprises the Network Interface Cards (NIC) in the
490 end systems, which are typically IP hosts, and relay systems, which
491 are typically IP routers and switches. Network-to-Network Interfaces
492 such as used for Traffic Engineering path reservation in [RFC3209],
493 as well as User-to-Network Interfaces (UNI) such as provided by the
494 Local Management Interface (LMI) between network and end systems, are
495 all part of the Network Plane.
497 A Southbound (Network) Interface enables the entities in the
498 Controller Plane to communicate with devices in the Network Plane.
499 This interface leverages and extends TEAS to describe the physical
500 topology and resources in the Network Plane.
502 Stream Management Entity
504 End End
505 System System
507 -+-+-+-+-+-+-+ Northbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
509 PCE PCE PCE PCE
511 -+-+-+-+-+-+-+ Southbound -+-+-+-+-+-+-+-+-+-+--+-+-+-+-+-+-
513 Relay Relay Relay Relay
514 System System System System
515 NIC NIC
516 Relay Relay Relay Relay
517 System System System System
519 Figure 3
521 The relay systems (and eventually the end systems NIC) expose their
522 capabilities and physical resources to the controller (the PCE), and
523 update the PCE with their dynamic perception of the topology, across
524 the Southbound Interface. In return, the PCE(s) set the per-stream
525 paths up, providing a Stream Characterization that is more tightly
526 coupled to the relay system Operation than a TSpec.
528 At the Network plane, relay systems exchange information regarding
529 the state of the paths, between adjacent systems and eventually with
530 the end systems, and forward packets within constraints associated to
531 each stream, or, when unable to do so, perform a last resort
532 operation such as drop or declassify.
534 This specification focuses on the Southbound interface and the
535 operation of the Network Plane.
537 4.4. Elements of DetNet Architecture
539 The DetNet architecture has a number of elements, discussed in the
540 following sections:
542 a. A model for the definition, identification, and operation of
543 DetNet streams (Section 4.5), for use by relay systems to
544 classify and process individual packets following per-stream
545 rules.
547 b. A model for the flow of data from an end system or through a
548 relay system that can be used to predict the bounds for that
549 system's impact on the QoS of a DetNet stream, without
550 significantly constraining the method of implementing that
551 system, for use by the Controllers to configure policing and
552 shaping engines in Network Systems over the Southbound interface.
553 The model includes:
555 1. A model for queuing, transmission selection, shaping,
556 preemption, and timing resources that can be used by an end
557 system or relay system to control the selection of packets
558 output on an interface. These models must have sufficiently
559 well-defined characteristics, both individually and in the
560 aggregate, to give predictable results for the QoS for DetNet
561 packets (Section 4.7).
563 2. A model for identifying misbehaving DetNet streams and
564 mitigating their impact on properly functioning streams
565 (Section 4.9).
567 c. A model for the relay system to inform the controller(s) of the
568 information it needs for adequate path computations including:
570 1. Systems' individual capabilities (e.g. can do replication,
571 can do precise time).
573 2. Link capabilities and resources (e.g. bandwidth, 0 delays,
574 hardware deterministic support to the physical layer, ...)
576 3. hysical resources (total and available buffers, timers,
577 queues, etc)
579 4. Network Adjacencies (neighbors)
581 d. A model for the provision of a service, by end systems, or relay
582 systems, to forward a DetNet stream over a simple or redundant
583 path. The model includes:
585 1. A model for an abstract relaying operation of either Routing
586 or forwarding packets of a DetNet stream to a next-hop relay
587 system, across Layer boundaries.
589 2. A model of next-hop(s) information for replicating the
590 packets of a DetNet stream, typically at or near the talker,
591 merging and/or re-replicating those packets at other points
592 in the network, and finally eliminating the duplicates,
593 typically at or near the listener(s), in order to provide
594 high availability (Section 3.3).
596 e. The protocol stack model for an end system and/or a relay system
597 should support the above elements in a manner that maximizes the
598 applicability of existing standards and protocols to the DetNet
599 problem, allows for the creation of new protocols where needed,
600 thus making DetNet an add-on feature to existing networks, rather
601 than a new way to do networking. In particular this protocol
602 stack supports networks in which the path from talker to
603 listener(s) includes bridges and/or routers in any order
604 (Section 4.10).
606 f. A variety of models for the provisioning of DetNet streams can be
607 envisioned, including orchestration by a central controller or by
608 a federation of controllers, provisioning by relay systems and
609 end systems sharing peer-to-peer protocols, by off-line
610 configuration, or by a combination of these methods. The
611 provisioning models are similar to existing Layer-2 and Layer-3
612 models, in order to minimize the amount of innovation required in
613 this area (Section 4.12).
615 4.5. DetNet streams
617 4.5.1. Talker guarantees
619 DetNet streams can by synchronous or asynchronous. The transmission
620 of packets in synchronous DetNet streams uses time synchronization
621 among the end and relay systems to control the flow of packets.
622 Asynchronous DetNet streams are characterized by:
624 o A maximum packet size;
626 o An observation interval; and
627 o A maximum number of transmissions during that observation
628 interval.
630 These parameters, together with knowledge of the protocol stack used
631 (and thus the size of the various headers added to a packet), limit
632 the number of bit times per observation interval that the DetNet
633 stream can occupy the physical medium.
635 The talker promises that these limits will not be exceeded. If the
636 talker transmits less data than this limit allows, the unused
637 resources such as link bandwidth can be made available by the system
638 to non-DetNet packets. However, making those resources available to
639 DetNet packets in other streams would serve no purpose. Those other
640 streams have their own dedicated resources, on the assumption that
641 all DetNet streams can use all of their resources over a long period
642 of time.
644 Note that there is no provision in DetNet for throttling streams; the
645 assumption is that a DetNet stream, to be useful, must be delivered
646 in its entirety. That is, while any useful application is written to
647 expect a certain number of lost packets, the real-time applications
648 of interest to DetNet demand that the loss of data due to the network
649 is extraordinarily infrequent.
651 Although DetNet strives to minimize the changes required of an
652 application to allow it to shift from a special-purpose digital
653 network to an Internet Protocol network, one fundamental shift in the
654 behavior of network applications that is impossible to avoid--the
655 reservation of resources before the application starts. In the first
656 place, a network cannot deliver finite latency and practically zero
657 packet loss to an arbitrarily high offered load. Secondly, achieving
658 practically zero packet loss for unthrottled (though bandwidth
659 limited) streams means that bridges and routers have to dedicate
660 buffer resources to specific streams or to classes of streams. The
661 requirements of each reservation have to be translated into the
662 parameters that control each system's queuing, shaping, and
663 scheduling functions and delivered to the hosts, bridges, and
664 routers.
666 4.5.2. Incomplete Networks
668 The presence in the network of relay systems that are not fully
669 capable of offering DetNet services complicates the ability of the
670 relay systems and/or controller to allocate resources, as extra
671 buffering, and thus extra latency, must be allocated at each point
672 that is downstream from the non-DetNet relay system for some DetNet
673 stream.
675 4.6. Data Flow Model through Systems
677 4.7. Queuing, Shaping, Scheduling, and Preemption
679 For this reason, the IEEE 802.1 Time-Sensitive Networking Task Group
680 has defined a set of queuing, shaping, and scheduling algorithms that
681 enable each bridge or router to compute the exact number of buffers
682 to be allocated for each stream or class of streams.
684 4.8. Coexistence with normal traffic
686 A DetNet network supports the dedication of at least 75% of the
687 network bandwidth to DetNet streams. But, no matter how much is
688 dedicated for DetNet streams, It is z goal of DetNet to not interfere
689 excessively with existing QoS schemes. It is also important that
690 non-DetNet traffic not disrupt the DetNet stream, of course (see
691 Section 4.9 and Section 6). For these reasons:
693 o Bandwidth (transmission opportunities) not utilized by a DetNet
694 stream are available to non-DetNet packets (though not to other
695 DetNet streams).
697 o DetNet streams can be shaped, in order to ensure that the highest-
698 priority non-DetNet packet also is ensured a maximum latency.
700 o When transmission opportunities for DetNet streams are scheduled
701 in detail, then the algorithm constructing the schedule should
702 leave sufficient opportunities for non-DetNet packets to satisfy
703 the needs of the uses of the network.
705 Ideally, the net effect of the presence of DetNet streams in a
706 network on the non-DetNet packets is primarily a reductoin in the
707 available bandwidth.
709 4.9. Fault Mitigation
711 One key to building robust real-time systems is to reduce the
712 infinite variety of possible failures to a number that can be
713 analyzed with reasonable confidence. DetNet aids in the process by
714 providing filters and policers to detect DetNet packets received on
715 the wrong interface, or at the wrong time, or in too great a volume,
716 and to then take actions such as disabling the offending packet,
717 shutting down the offending DetNet stream, or shutting down the
718 offending interface.
720 It is also essential that filters and service remarking be employed
721 to prevent non-DetNet packets from impinging on the resources
722 allocated to DetNet packets.
724 There exist techniques, at present and/or in various stages of
725 standardization, that can perform these fault mitigation tasks that
726 deliver a high probability that misbehaving systemd will have zero
727 impact on well-behaved DetNet streams, except of course, for the
728 receiving interface(s) immediately downstream of the misbehaving
729 device.
731 4.10. Protocol Stack Model
733 This section will be further developed. See [IEEE802.1CB], Annex C,
734 for a description of the protocol stack. This is very much a work in
735 progress, not a standard. See also [IEEE802.1Qcc].
737 4.11. Advertising resources, capabilities and adjacencies
739 4.12. Provisioning model
741 4.12.1. Centralized Path Computation and Installation
743 A centralized routing model, such as provided with a PCE (RFC 4655
744 [RFC4655]), enables global and per-stream optimizations. The model
745 is attractive but a number of issues are left to be solved. In
746 particular:
748 o whether and how the path computation can be installed by 1) an end
749 device or 2) a Network Management entity,
751 o and how the path is set up, either by installing state at each hop
752 with a direct interaction between the forwarding device and the
753 PCE, or along a path by injecting a source-routed request at one
754 end of the path.
756 4.12.2. Distributed Path Setup
758 Whether a distributed alternative without a PCE can be valuable
759 should be studied as well. Such an alternative could for instance
760 inherit from the Resource ReSerVation Protocol [RFC5127] (RSVP)
761 flows.
763 In a Layer-2 only environment, or as part of a layered approach to a
764 mixed environment, IEEE 802.1 also has work, either completed or in
765 progress. [IEEE802.1Q-2011] Clause 35 describes SRP, a peer-to-peer
766 protocol for Layer-2 roughly analogous to RSVP. Almost complete is
767 [IEEE802.1Qca], which defines how ISIS can provide multiple disjoint
768 paths or distribution trees. Also in progress is [IEEE802.1Qcc],
769 which expands the capabilities of SRP.
771 5. Related IETF work
773 5.1. Deterministic PHB
775 [I-D.svshah-tsvwg-deterministic-forwarding] defines a Differentiated
776 Services Per-Hop-Behavior (PHB) Group called Deterministic Forwarding
777 (DF). The document describes the purpose and semantics of this PHB.
778 It also describes creation and forwarding treatment of the service
779 class. The document also describes how the code-point can be mapped
780 into one of the aggregated Diffserv service classes [RFC5127].
782 5.2. 6TiSCH
784 Industrial process control already leverages deterministic wireless
785 Low power and Lossy Networks (LLNs) to interconnect critical
786 resource-constrained devices and form wireless mesh networks, with
787 standards such as [ISA100.11a] and [WirelessHART].
789 These standards rely on variations of the [IEEE802154e] timeSlotted
790 Channel Hopping (TSCH) [I-D.ietf-6tisch-tsch] Medium Access Control
791 (MAC), and a form of centralized Path Computation Element (PCE), to
792 deliver deterministic capabilities.
794 The TSCH MAC benefits include high reliability against interference,
795 low power consumption on characterized streams, and Traffic
796 Engineering capabilities. Typical applications are open and closed
797 control loops, as well as supervisory control streams and management.
799 The 6TiSCH Working Group focuses only on the TSCH mode of the IEEE
800 802.15.4e standard. The WG currently defines a framework for
801 managing the TSCH schedule. Future work will standardize
802 deterministic operations over so-called tracks as described in
803 [I-D.ietf-6tisch-architecture]. Tracks are an instance of a
804 deterministic path, and the DetNet work is a prerequisite to specify
805 track operations and serve process control applications.
807 [RFC5673] and [I-D.ietf-roll-rpl-industrial-applicability] section
808 2.1.3. and next discusses application-layer paradigms, such as
809 Source-sink (SS) that is a Multipeer to Multipeer (MP2MP) model that
810 is primarily used for alarms and alerts, Publish-subscribe (PS, or
811 pub/sub) that is typically used for sensor data, as well as Peer-to-
812 peer (P2P) and Peer-to-multipeer (P2MP) communications. Additional
813 considerations on Duocast and its N-cast generalization are also
814 provided for improved reliability.
816 6. Security Considerations
818 Security in the context of Deterministic Networking has an added
819 dimension; the time of delivery of a packet can be just as important
820 as the contents of the packet, itself. A man-in-the-middle attack,
821 for example, can impose, and then systematically adjust, additional
822 delays into a link, and thus disrupt or subvert a real-time
823 application without having to crack any encryption methods employed.
824 See [RFC7384] for an exploration of this issue in a related context.
826 Furthermore, in a control system where millions of dollars of
827 equipment, or even human lives, can be lost if the DetNet QoS is not
828 delivered, one must consider not only simple equipment failures,
829 where the box or wire instantly becomes perfectly silent, but bizarre
830 errors such as can be coused by software failures. Because there is
831 essentiall no limit to the kinds of failures that can occur,
832 protecting against realistic equipment failures is indistinguishable,
833 in most cases, from protecting against malicious behavior, whether
834 accidental or intentional. See also Section 4.9.
836 Security must cover:
838 o the protection of the signaling protocol
840 o the authentication and authorization of the controlling systems
842 o the identification and shaping of the streams
844 7. IANA Considerations
846 This document does not require an action from IANA.
848 8. Acknowledgements
850 The authors wish to thank Jouni Korhonen, Erik Nordmark, George
851 Swallow, Rudy Klecka, Anca Zamfir, David Black, Thomas Watteyne,
852 Shitanshu Shah, Craig Gunther, Rodney Cummings, Wilfried Steiner,
853 Marcel Kiessling, Karl Weber, Ethan Grossman and Pat Thaler, for
854 their various contribution with this work.
856 9. Informative References
858 [AVnu] http://www.avnu.org/, "The AVnu Alliance tests and
859 certifies devices for interoperability, providing a simple
860 and reliable networking solution for AV network
861 implementation based on the Audio Video Bridging (AVB)
862 standards.", .
864 [CCAMP] IETF, "Common Control and Measurement Plane",
865 .
867 [HART] www.hartcomm.org, "Highway Addressable Remote Transducer,
868 a group of specifications for industrial process and
869 control devices administered by the HART Foundation", .
871 [HSR-PRP] IEC, "High availability seamless redundancy (HSR) is a
872 further development of the PRP approach, although HSR
873 functions primarily as a protocol for creating media
874 redundancy while PRP, as described in the previous
875 section, creates network redundancy. PRP and HSR are both
876 described in the IEC 62439 3 standard.",
877 .
880 [I-D.finn-detnet-problem-statement]
881 Finn, N. and P. Thubert, "Deterministic Networking Problem
882 Statement", draft-finn-detnet-problem-statement-01 (work
883 in progress), October 2014.
885 [I-D.ietf-6tisch-architecture]
886 Thubert, P., Watteyne, T., Struik, R., and M. Richardson,
887 "An Architecture for IPv6 over the TSCH mode of IEEE
888 802.15.4e", draft-ietf-6tisch-architecture-05 (work in
889 progress), January 2015.
891 [I-D.ietf-6tisch-tsch]
892 Watteyne, T., Palattella, M., and L. Grieco, "Using
893 IEEE802.15.4e TSCH in an IoT context: Overview, Problem
894 Statement and Goals", draft-ietf-6tisch-tsch-05 (work in
895 progress), January 2015.
897 [I-D.ietf-roll-rpl-industrial-applicability]
898 Phinney, T., Thubert, P., and R. Assimiti, "RPL
899 applicability in industrial networks", draft-ietf-roll-
900 rpl-industrial-applicability-02 (work in progress),
901 October 2013.
903 [I-D.svshah-tsvwg-deterministic-forwarding]
904 Shah, S. and P. Thubert, "Deterministic Forwarding PHB",
905 draft-svshah-tsvwg-deterministic-forwarding-03 (work in
906 progress), March 2015.
908 [IEEE802.1AS-2011]
909 IEEE, "Timing and Synchronizations (IEEE 802.1AS-2011)",
910 2011, .
913 [IEEE802.1BA-2011]
914 IEEE, "AVB Systems (IEEE 802.1BA-2011)", 2011,
915 .
918 [IEEE802.1CB]
919 IEEE, "Seamless Redundancy (IEEE Draft P802.1CB)", 2015,
920 .
923 [IEEE802.1Q-2011]
924 IEEE, "MAC Bridges and VLANs (IEEE 802.1Q-2011", 2011,
925 .
928 [IEEE802.1Qat-2010]
929 IEEE, "Stream Reservation Protocol (IEEE 802.1Qat-2010)",
930 2010, .
933 [IEEE802.1Qav]
934 IEEE, "Forwarding and Queuing (IEEE 802.1Qav-2009)", 2009,
935 .
938 [IEEE802.1Qca]
939 IEEE, "Path Control and Reservation", 2015,
940 .
943 [IEEE802.1Qcc]
944 IEEE, "Stream Reservation Protocol (SRP) Enhancements and
945 Performance Improvements", 2015,
946 .
949 [IEEE802.1TSNTG]
950 IEEE Standards Association, "IEEE 802.1 Time-Sensitive
951 Networks Task Group", 2013,
952 .
954 [IEEE802154]
955 IEEE standard for Information Technology, "IEEE std.
956 802.15.4, Part. 15.4: Wireless Medium Access Control (MAC)
957 and Physical Layer (PHY) Specifications for Low-Rate
958 Wireless Personal Area Networks", June 2011.
960 [IEEE802154e]
961 IEEE standard for Information Technology, "IEEE std.
962 802.15.4e, Part. 15.4: Low-Rate Wireless Personal Area
963 Networks (LR-WPANs) Amendment 1: MAC sublayer", April
964 2012.
966 [ISA100.11a]
967 ISA/IEC, "ISA100.11a, Wireless Systems for Automation,
968 also IEC 62734", 2011, < http://www.isa100wci.org/en-
969 US/Documents/PDF/3405-ISA100-WirelessSystems-Future-broch-
970 WEB-ETSI.aspx>.
972 [ODVA] http://www.odva.org/, "The organization that supports
973 network technologies built on the Common Industrial
974 Protocol (CIP) including EtherNet/IP.", .
976 [PCE] IETF, "Path Computation Element",
977 .
979 [Profinet]
980 http://us.profinet.com/technology/profinet/, "PROFINET is
981 a standard for industrial networking in automation.",
982 .
984 [RFC2205] Braden, B., Zhang, L., Berson, S., Herzog, S., and S.
985 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
986 Functional Specification", RFC 2205, September 1997.
988 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
989 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
990 Tunnels", RFC 3209, December 2001.
992 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation
993 Element (PCE)-Based Architecture", RFC 4655, August 2006.
995 [RFC5127] Chan, K., Babiarz, J., and F. Baker, "Aggregation of
996 Diffserv Service Classes", RFC 5127, February 2008.
998 [RFC5673] Pister, K., Thubert, P., Dwars, S., and T. Phinney,
999 "Industrial Routing Requirements in Low-Power and Lossy
1000 Networks", RFC 5673, October 2009.
1002 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in
1003 Packet Switched Networks", RFC 7384, October 2014.
1005 [RFC7426] Haleplidis, E., Pentikousis, K., Denazis, S., Hadi Salim,
1006 J., Meyer, D., and O. Koufopavlou, "Software-Defined
1007 Networking (SDN): Layers and Architecture Terminology",
1008 RFC 7426, January 2015.
1010 [TEAS] IETF, "Traffic Engineering Architecture and Signaling",
1011 .
1013 [WirelessHART]
1014 www.hartcomm.org, "Industrial Communication Networks -
1015 Wireless Communication Network and Communication Profiles
1016 - WirelessHART - IEC 62591", 2010.
1018 Authors' Addresses
1020 Norm Finn
1021 Cisco Systems
1022 170 W Tasman Dr.
1023 San Jose, California 95134
1024 USA
1026 Phone: +1 408 526 4495
1027 Email: nfinn@cisco.com
1029 Pascal Thubert
1030 Cisco Systems
1031 Village d'Entreprises Green Side
1032 400, Avenue de Roumanille
1033 Batiment T3
1034 Biot - Sophia Antipolis 06410
1035 FRANCE
1037 Phone: +33 4 97 23 26 34
1038 Email: pthubert@cisco.com