idnits 2.17.1
draft-finn-detnet-architecture-04.txt:
Checking boilerplate required by RFC 5378 and the IETF Trust (see
https://trustee.ietf.org/license-info):
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt:
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/checklist :
----------------------------------------------------------------------------
No issues found here.
Miscellaneous warnings:
----------------------------------------------------------------------------
== The copyright year in the IETF Trust and authors Copyright Line does not
match the current year
-- The document date (March 21, 2016) is 2929 days in the past. Is this
intentional?
Checking references for intended status: Proposed Standard
----------------------------------------------------------------------------
(See RFCs 3967 and 4897 for information about using normative references
to lower-maturity documents in RFCs)
== Unused Reference: 'AVnu' is defined on line 1160, but no explicit
reference was found in the text
== Unused Reference: 'HART' is defined on line 1169, but no explicit
reference was found in the text
== Unused Reference: 'I-D.ietf-6tisch-architecture' is defined on line
1187, but no explicit reference was found in the text
== Unused Reference: 'I-D.ietf-6tisch-tsch' is defined on line 1192, but no
explicit reference was found in the text
== Unused Reference: 'I-D.ietf-roll-rpl-industrial-applicability' is
defined on line 1205, but no explicit reference was found in the text
== Unused Reference: 'I-D.svshah-tsvwg-deterministic-forwarding' is defined
on line 1211, but no explicit reference was found in the text
== Unused Reference: 'IEEE802.11-2012' is defined on line 1216, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802.1AS-2011' is defined on line 1222, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802.1TSNTG' is defined on line 1263, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802154' is defined on line 1277, but no explicit
reference was found in the text
== Unused Reference: 'IEEE802154e' is defined on line 1283, but no explicit
reference was found in the text
== Unused Reference: 'ISA95' is defined on line 1295, but no explicit
reference was found in the text
== Unused Reference: 'ODVA' is defined on line 1299, but no explicit
reference was found in the text
== Unused Reference: 'Profinet' is defined on line 1306, but no explicit
reference was found in the text
== Unused Reference: 'RFC2205' is defined on line 1311, but no explicit
reference was found in the text
== Unused Reference: 'RFC5673' is defined on line 1326, but no explicit
reference was found in the text
== Unused Reference: 'WirelessHART' is defined on line 1354, but no
explicit reference was found in the text
== Outdated reference: A later version (-30) exists of
draft-ietf-6tisch-architecture-09
== Outdated reference: A later version (-20) exists of
draft-ietf-detnet-use-cases-08
Summary: 0 errors (**), 0 flaws (~~), 20 warnings (==), 1 comment (--).
Run idnits with the --verbose option for more detailed information about
the items above.
--------------------------------------------------------------------------------
2 DetNet N. Finn
3 Internet-Draft P. Thubert
4 Intended status: Standards Track Cisco
5 Expires: September 22, 2016 M. Johas Teener
6 Broadcom
7 March 21, 2016
9 Deterministic Networking Architecture
10 draft-finn-detnet-architecture-04
12 Abstract
14 Deterministic Networking (DetNet) provides a capability to carry
15 specified unicast or multicast data flows for real-time applications
16 with extremely low data loss rates and bounded latency. Techniques
17 used include: 1) reserving data plane resources for individual (or
18 aggregated) DetNet flows in some or all of the relay systems (bridges
19 or routers) along the path of the flow; 2) providing fixed paths for
20 DetNet flows that do not rapidly change with the network topology;
21 and 3) sequentializing, replicating, and eliminating duplicate
22 packets at various points to ensure the availability of at least one
23 path. The capabilities can be managed by configuration, or by manual
24 or automatic network management.
26 Status of This Memo
28 This Internet-Draft is submitted in full conformance with the
29 provisions of BCP 78 and BCP 79.
31 Internet-Drafts are working documents of the Internet Engineering
32 Task Force (IETF). Note that other groups may also distribute
33 working documents as Internet-Drafts. The list of current Internet-
34 Drafts is at http://datatracker.ietf.org/drafts/current/.
36 Internet-Drafts are draft documents valid for a maximum of six months
37 and may be updated, replaced, or obsoleted by other documents at any
38 time. It is inappropriate to use Internet-Drafts as reference
39 material or to cite them other than as "work in progress."
41 This Internet-Draft will expire on September 22, 2016.
43 Copyright Notice
45 Copyright (c) 2016 IETF Trust and the persons identified as the
46 document authors. All rights reserved.
48 This document is subject to BCP 78 and the IETF Trust's Legal
49 Provisions Relating to IETF Documents
50 (http://trustee.ietf.org/license-info) in effect on the date of
51 publication of this document. Please review these documents
52 carefully, as they describe your rights and restrictions with respect
53 to this document. Code Components extracted from this document must
54 include Simplified BSD License text as described in Section 4.e of
55 the Trust Legal Provisions and are provided without warranty as
56 described in the Simplified BSD License.
58 Table of Contents
60 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
61 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
62 2.1. Terms used in this document . . . . . . . . . . . . . . . 4
63 2.2. IEEE 802 TSN to DetNet dictionary . . . . . . . . . . . . 5
64 3. Providing the DetNet Quality of Service . . . . . . . . . . . 5
65 3.1. Zero Congestion Loss . . . . . . . . . . . . . . . . . . 7
66 3.2. Pinned paths . . . . . . . . . . . . . . . . . . . . . . 8
67 3.3. Packet replication and deletion . . . . . . . . . . . . . 8
68 4. DetNet Architecture . . . . . . . . . . . . . . . . . . . . . 9
69 4.1. Elements of DetNet Architecture . . . . . . . . . . . . . 9
70 4.2. Traffic Engineering for DetNet . . . . . . . . . . . . . 11
71 4.2.1. The Application Plane . . . . . . . . . . . . . . . . 13
72 4.2.2. The Controller Plane . . . . . . . . . . . . . . . . 13
73 4.2.3. The Network Plane . . . . . . . . . . . . . . . . . . 14
74 4.3. DetNet flows . . . . . . . . . . . . . . . . . . . . . . 15
75 4.3.1. Source guarantees . . . . . . . . . . . . . . . . . . 15
76 4.3.2. Incomplete Networks . . . . . . . . . . . . . . . . . 16
77 4.4. Queuing, Shaping, Scheduling, and Preemption . . . . . . 16
78 4.5. Coexistence with normal traffic . . . . . . . . . . . . . 17
79 4.6. Fault Mitigation . . . . . . . . . . . . . . . . . . . . 18
80 4.7. Protocol Stack Model . . . . . . . . . . . . . . . . . . 18
81 4.8. Advertising resources, capabilities and adjacencies . . . 20
82 4.9. Provisioning model . . . . . . . . . . . . . . . . . . . 20
83 4.9.1. Centralized Path Computation and Installation . . . . 20
84 4.9.2. Distributed Path Setup . . . . . . . . . . . . . . . 21
85 4.10. Scaling to larger networks . . . . . . . . . . . . . . . 21
86 4.11. Connected islands vs. networks . . . . . . . . . . . . . 21
87 5. Compatibility with Layer-2 . . . . . . . . . . . . . . . . . 22
88 6. Open Questions . . . . . . . . . . . . . . . . . . . . . . . 22
89 6.1. Data plane shapers and schedulers . . . . . . . . . . . . 22
90 6.2. DetNet flow identification and sequencing . . . . . . . . 22
91 6.3. Flat vs. hierarchical control . . . . . . . . . . . . . . 23
92 6.4. Peer-to-peer reservation protocol . . . . . . . . . . . . 23
93 6.5. Wireless media interactions . . . . . . . . . . . . . . . 24
94 7. Security Considerations . . . . . . . . . . . . . . . . . . . 24
95 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25
96 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 25
97 10. Access to IEEE 802.1 documents . . . . . . . . . . . . . . . 25
98 11. Informative References . . . . . . . . . . . . . . . . . . . 25
99 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 29
101 1. Introduction
103 Deterministic Networking (DetNet) is a service that can be offered by
104 a network to data flows (DetNet flows) that that are limited, at
105 their source, to a maximum data rate specified by that source.
106 DetNet provides these flows extremely low packet loss rates and
107 assured maximum end-to-end delivery latency. This is accomplished by
108 dedicating network resources such as link bandwidth and buffer space
109 to DetNet flows and/or classes of DetNet flows. Unused reserved
110 resources are available to non-DetNet packets.
112 The Deterministic Networking Problem Statement
113 [I-D.finn-detnet-problem-statement] introduces Deterministic
114 Networking, and Deterministic Networking Use Cases
115 [I-D.ietf-detnet-use-cases] summarizes the need for it.
117 A goal of DetNet is a converged network in all respects. That is,
118 the presence of DetNet flows does not preclude non-DetNet flows, and
119 the benefits offered DetNet flows should not, except in extreme
120 cases, prevent existing QoS mechanisms from operating in a normal
121 fashion, subject to the bandwidth required for the DetNet flows. A
122 single source-destination pair can trade both DetNet and non-DetNet
123 flows. End systems and applications need not instantiate special
124 interfaces for DetNet flows. Networks are not restricted to certain
125 topologies; connectivity is not restricted. Any application that
126 generates a data flow that can be usefully characterized as having a
127 maximum bandwidth should be able to take advantage of DetNet, as long
128 as the necessary resources can be reserved. Reservations can be made
129 by the application itself, via network management, by an applications
130 controller, or by other means.
132 Many applications of interest to Deterministic Networking require the
133 ability to synchronize the clocks in end systems to a sub-microsecond
134 accuracy. Some of the queue control techniques defined in
135 Section 4.4 also require time synchronization among relay systems.
136 The means used to achieve time synchronization are not addressed in
137 this document.
139 The present document is an individual contribution, intended by the
140 authors for eventual adoption by the DetNet working group. As such,
141 it expresses the only the opinions of the authors.
143 2. Terminology
145 2.1. Terms used in this document
147 The following special terms are used in this document in order to
148 avoid the assumption that a given element in the architecture does or
149 does not have Internet Protocol stack, functions as a router, bridge,
150 firewall, or otherwise plays a particular role at Layer-2 or higher.
152 destination
153 An end system capable of sinking a DetNet flow.
155 DetNet domain
156 The portion of a network that is DetNet aware. It includes
157 end systems and other DetNet nodes.
159 DetNet flow
160 A DetNet flow is a sequence of packets from a single source,
161 through some number of relay systems to one or more
162 destinations, that is limited by the source in its maximum
163 packet size and transmission rate, and can thus be ensured
164 the DetNet Quality of Service (QoS) from the network.
166 DetNet node
167 A DetNet aware end system or relay system. "DetNet" may be
168 omitted in some text.
170 end system
171 Commonly called a "host" or "node" in IETF documents, and an
172 "end station" is IEEE 802 documents. End systems of interest
173 to this document are either sources or destinations of L2
174 and/or L3 DatNet streams. Note that a system that takes non-
175 DetNet aware traffic and transmits it via a DetNet flow is
176 also an end system. (For comparison, a Label Edge Router
177 (LER) would be an MPLS "end system".)
179 link
180 A connection between two DetNet nodes. It may be composed of
181 a physical link or a sub-network technology that can provide
182 appropriate traffic delivery for DetNet flows.
184 relay system
185 A router, transit node, bridge, Label Switch Router (LSR),
186 firewall, or any other system that forwards packets from one
187 interface to another.
189 reservation
190 A trail of configuration between source to destination(s)
191 through relay systems associated with a DetNet flow, required
192 to deliver the benefits of DetNet.
194 source
195 An end system capable of sourcing a DetNet flow.
197 2.2. IEEE 802 TSN to DetNet dictionary
199 This section also serves as a dictionary for translating from the
200 terms used by the IEEE 802 Time-Sensitive Networking (TSN) Task Group
201 to those of the DetNet WG.
203 listener
204 The IEEE 802 term for a destination of a DetNet flow.
206 stream
207 The IEEE 802 term for a DetNet flow.
209 talker
210 The IEEE 802 term for the source of a DetNet flow.
212 3. Providing the DetNet Quality of Service
214 DetNet Quality of Service is expressed in terms of:
216 o Minimum and maximum end-to-end latency from source to destination;
218 o Probability of loss of a packet, under various assumptions as to
219 the operational states of the relay systems and links;
221 It is a distinction of DetNet that it is concerned solely with worst-
222 case values for the end-to-end latency. Average, mean, or typical
223 values are of no interest, because they do not affect the ability of
224 a real-time system to perform its tasks. In general, a trivial
225 priority-based queuing scheme will give better average latency to a
226 data flow than DetNet, but of course, the worst-case latency is
227 essentially unbounded.
229 Three techniques are employed by DetNet to achieve these QoS
230 parameters:
232 a. Zero congestion loss (Section 3.1). Network resources such as
233 link bandwidth, buffers, queues, shapers, and scheduled input/
234 output slots are assigned in each relay system to the use of a
235 specific DetNet flow or class of DetNet flows. Given a finite
236 amount of buffer space, zero congestion loss necessarily ensures
237 a bounded end-to-end latency. Depending on the resources
238 employed, a minimum latency, and thus bounded jitter, can also be
239 achieved.
241 b. Pinned paths (Section 3.2). Point-to-point paths or point-to-
242 multipoint trees through the network from a source to one or more
243 destinations can be established, and DetNet flows assigned to
244 follow a particular path or tree.
246 c. Packet replication and deletion (Section 3.3). End systems and/
247 or relay systems can number packets sequentially, replicate them,
248 and later eliminate all but one of the replicants, at multiple
249 points in the network in order to ensure that one (or more)
250 equipment failure events still leave at least one path intact for
251 a DetNet flow. This function is a "hitless" version of, e.g.,
252 the 1+1 linear protection in [RFC6372]. That is, instead of
253 switching from one flow to the other when a failure of a flow is
254 detected, DetNet combines both flows, and performs a packet-by-
255 packet selection of which to discard, based on sequence number.
257 These techniques address both of the DetNet QoS requirements. Given
258 that relay nodes have a finite amount of buffer space, zero
259 congestion loss (Section 3.1) necessarily results in a maximum end-
260 to-end latency. It also addresses the largest contribution to packet
261 loss, which is buffer congestion. Packet replication and deletion
262 mitigates the other most important contributions to packet loss,
263 namely random media errors and equipment failure.
265 These three techniques can be applied independently, giving eight
266 possible combinations, including none (no DetNet), although some
267 combinations are of wider utility than others. This separation keeps
268 the protocol stack coherent and maximizes interoperability with
269 existing and developing standards in this (IETF) and other Standards
270 Development Organizations. Some examples of typical expected
271 combinations:
273 o Pinned paths (a) plus packet replication (b) are exactly the
274 techniques employed by [HSR-PRP]. Pinned paths are achieved by
275 limiting the physical topology of the network, and the
276 sequentialization, replication, and duplicate elimination are
277 facilitated by packet tags added at the front or the end of
278 Ethernet frames.
280 o Zero congestion loss (a) alone is is offered by IEEE 802.1 Audio
281 Video bridging [IEEE802.1BA-2011]. As long as the network suffers
282 no failures, zero congestion loss can be achieved through the use
283 of a reservation protocol (MSRP), shapers in every relay system
284 (bridge), and a bit of network calculus.
286 o Using all three together gives maximum protection.
288 There are, of course, simpler methods available (and employed, today)
289 to achieve levels of latency and packet loss that are satisfactory
290 for many applications. Prioritization and over-provisioning is one
291 such technique. However, these methods generally work best in the
292 absence of any significant amount of non-critical traffic in the
293 network (if, indeed, such traffic is supported at all), or work only
294 if the critical traffic constitutes only a small portion of the
295 network's theoretical capacity, or work only if all systems are
296 functioning properly, or in the absence of actions by end systems
297 that disrupt the network's operations.
299 There are any number of methods in use, defined, or in progress for
300 accomplishing each of the above techniques. It is expected that this
301 DetNet Architecture will assist various vendors, users, and/or
302 "vertical" Standards Development Organizations (dedicated to a single
303 industry) to make selections among the available means of
304 implementing DetNet networks.
306 3.1. Zero Congestion Loss
308 The primary means by which DetNet achieves its QoS assurances is to
309 completely eliminate congestion at an output port as a cause of
310 packet loss. Given that a DetNet flow cannot be throttled, this can
311 be achieved only by the provision of sufficient buffer storage at
312 each hop through the network to ensure that no packets are dropped
313 due to a lack of buffer storage.
315 Ensuring adequate buffering requires, in turn, that the source, and
316 every relay system along the path to the destination (or nearly every
317 relay system -- see Section 4.3.2) be careful to regulate its output
318 to not exceed the data rate for any DetNet flow, except for brief
319 periods when making up for interfering traffic. Any packet sent
320 ahead of its time potentially adds to the number of buffers required
321 by the next hop, and may thus exceed the resources allocated for a
322 particular DetNet flow.
324 The low-level mechanisms described in Section 4.4 provide the
325 necessary regulation of transmissions by an edge system or relay
326 system to ensure zero congestion loss. The reservation of the
327 bandwidth and buffers for a DetNet flow requires the provisioning
328 described in Section 4.9.
330 A DetNet node may have other resources requiring allocation and/or
331 scheduling, that might otherwise be over-subscribed and trigger
332 congestion loss.
334 3.2. Pinned paths
336 In networks controlled by typical peer-to-peer protocols such as IEEE
337 802.1 ISIS bridged networks or IETF OSPF routed networks, a network
338 topology event in one part of the network can impact, at least
339 briefly, the delivery of data in parts of the network remote from the
340 failure or recovery event. Thus, even redundant paths through a
341 network, if controlled by the typical peer-to-peer protocols, do not
342 eliminate the chances of brief losses of contact.
344 Many real-time networks rely on physical rings or chains of two-port
345 devices, with a relatively simple ring control protocol. This
346 supports redundant paths with a minimum of wiring. As an additional
347 benefit, ring topologies can often utilize different topology
348 management protocols than those used for a mesh network, with a
349 consequent reduction in the response time to topology changes. Of
350 course, this comes at some cost in terms of increased hop count, and
351 thus latency, for the typical path.
353 In order to get the advantages of low hop count and still ensure
354 against even very brief losses of connectivity, DetNet employs pinned
355 paths, where the path taken by a given DetNet flow does not change,
356 at least immediately, and likely not at all, in response to network
357 topology events. When combined with packet replication and deletion
358 (Section 3.3), this results in a high likelihood of continuous
359 connectivity. Pinned paths are commonly used in MPLS TE LSPs.
361 3.3. Packet replication and deletion
363 After congestion loss has been eliminated, the most important causes
364 of packet loss are random media and/or memory faults, and equipment
365 failures. Both causes of packet loss can be greatly reduced by
366 sending the same packets over multiple paths.
368 Packet replication and deletion, also known as seamless redundancy
369 [HSR-PRP], or 1+1 hitless protection, involves three capabilities:
371 o Providing sequencing information, once, to the packets of a DetNet
372 flow. This may be done by adding a sequence number or time stamp
373 as part of DetNet, or may be inherent in the packet, e.g. in a
374 transport protocol.
376 o Replicating these packets and, typically, sending them along at
377 least two different paths to the destination(s). (Often, the
378 pinned paths of Section 3.2.)
380 o Discarding duplicated packets.
382 In the simplest case, this amounts to replicating each packet in a
383 source that has two interfaces, and conveying them through the
384 network, along separate paths, to the similarly dual-homed
385 destinations, that discard the extras. This ensures that one path
386 (with zero congestion loss) remains, even if some relay system fails.
387 The sequence numbers can also be used for loss detection and for re-
388 ordering.
390 Alternatively, relay systems in the network can provide replication
391 and elimination facilities at various points in the network, so that
392 multiple failures can be accommodated.
394 This is shown in the following figure, where the two relay systems
395 each replicate (R) the DetNet flow on input, sending the DetNet flow
396 to both the other relay system and to the end system, and eliminate
397 duplicates (E) on the output interface to the right-hand end system.
398 Any one link in the network can fail, and the Detnet flow can still
399 get through. Furthermore, two links can fail, as long as they are in
400 different segments of the network.
402 > > > > > > > > relay > > > > > > > >
403 > /------------+ R system E +------------\ >
404 > / v + ^ \ >
405 end R + v | ^ + E end
406 system + v | ^ + system
407 > \ v + ^ / >
408 > \------------+ R relay E +------------/ >
409 > > > > > > > > system > > > > > > > >
411 Figure 1
413 Note that packet replication and deletion does not react to and
414 correct failures; it is entirely passive. Thus, intermittent
415 failures, mistakenly created access control lists, or misrouted data
416 is handled just the same as the equipment failures that are detected
417 handled by typical routing and bridging protocols.
419 4. DetNet Architecture
421 4.1. Elements of DetNet Architecture
423 The DetNet architecture has a number of elements, discussed in the
424 following sections. Note that not every application requires all of
425 these elements.
427 a. A model for the definition, identification, and operation of
428 DetNet flows (Section 4.3), for use by relay systems to classify
429 and process individual packets following per-flow rules.
431 b. A model for the flow of data out of an end system or through a
432 relay system that can be used to predict the bounds for that
433 system's impact on the QoS of a DetNet flow, for use by the
434 Controllers to configure policing and shaping engines in Network
435 Systems over the Southbound interface. The model includes:
437 1. A model for queuing, transmission selection, shaping,
438 preemption, and timing resources that can be used by an end
439 system or relay system to control the selection of packets
440 output on an interface. These models must have sufficiently
441 well-defined characteristics, both individually and in the
442 aggregate, to give predictable results for the QoS for DetNet
443 packets (Section 4.4).
445 2. A model for identifying misbehaving DetNet flows and
446 mitigating their impact on properly functioning DetNet flows
447 (Section 4.6).
449 c. A model for the relay system to inform the controller(s) of the
450 information it needs for adequate path computations (Section 4.2)
451 including:
453 1. Systems' individual capabilities (e.g. can do replication,
454 can do precise time).
456 2. Link capabilities and resources (e.g. bandwidth, transmission
457 delay, hardware deterministic support to the physical layer,
458 ...)
460 3. Physical resources (total and available buffers, timers,
461 queues, etc)
463 4. Network Adjacencies (neighbors)
465 d. A model for the provision of a service, by end systems or relay
466 systems, to replicate and forward a DetNet flow over redundant
467 paths. The model includes:
469 1. A model for specifying multiple stable paths across a network
470 that can perform packet forwarding at both Layer 3 and at
471 lower layers, to which specific DetNet flows can be assigned
472 (Section 4.2).
474 2. A model and data plane format(s) for sequencing and
475 replicating the packets of a DetNet flow, typically at or
476 near the source, sending the replicated DetNet flows over
477 different stable paths, merging and/or re-replicating those
478 packets at other points in the network, and finally
479 eliminating the duplicates, typically at or near the
480 destination(s), in order to provide high availability
481 (Section 3.3).
483 e. The protocol stack model for an end system and/or a relay system
484 should support the above elements in a manner that maximizes the
485 applicability of existing standards and protocols to the DetNet
486 problem, and allows for the creation of new protocols only where
487 needed, thus making DetNet an add-on feature to existing
488 networks, rather than a new way to do networking. In particular
489 this protocol stack supports networks in which the path from
490 source to destination(s) includes bridges and/or routers in any
491 order (Section 4.7).
493 f. A variety of models for the provisioning of DetNet flows can be
494 envisioned, including orchestration by a central controller or by
495 a federation of controllers, via control plane protocols running
496 on relay systems and end systems, by off-line configuration, or
497 by a combination of these methods. The provisioning models are
498 similar to existing Layer-2 and Layer-3 models, in order to
499 minimize the amount of innovation required in this area
500 (Section 4.9).
502 4.2. Traffic Engineering for DetNet
504 Traffic Engineering Architecture and Signaling (TEAS) [TEAS] defines
505 traffic-engineering architectures for generic applicability across
506 packet and non-packet networks. From TEAS perspective, Traffic
507 Engineering (TE) refers to techniques that enable operators to
508 control how specific traffic flows are treated within their networks.
510 Because if its very nature of establishing pinned optimized paths,
511 Deterministic Networking can be seen as a new, specialized branch of
512 Traffic Engineering, and inherits its architecture with a separation
513 into planes.
515 The Deterministic Networking architecture is thus composed of three
516 planes, a (User) Application Plane, a Controller Plane, and a Network
517 Plane, which echoes that of Software-Defined Networking (SDN): Layers
518 and Architecture Terminology [RFC7426] which is represented below:
520 SDN Layers and Architecture Terminology per RFC 7426
522 o--------------------------------o
523 | |
524 | +-------------+ +----------+ |
525 | | Application | | Service | |
526 | +-------------+ +----------+ |
527 | Application Plane |
528 o---------------Y----------------o
529 |
530 *-----------------------------Y---------------------------------*
531 | Network Services Abstraction Layer (NSAL) |
532 *------Y------------------------------------------------Y-------*
533 | |
534 | Service Interface |
535 | |
536 o------Y------------------o o---------------------Y------o
537 | | Control Plane | | Management Plane | |
538 | +----Y----+ +-----+ | | +-----+ +----Y----+ |
539 | | Service | | App | | | | App | | Service | |
540 | +----Y----+ +--Y--+ | | +--Y--+ +----Y----+ |
541 | | | | | | | |
542 | *----Y-----------Y----* | | *---Y---------------Y----* |
543 | | Control Abstraction | | | | Management Abstraction | |
544 | | Layer (CAL) | | | | Layer (MAL) | |
545 | *----------Y----------* | | *----------Y-------------* |
546 | | | | | |
547 o------------|------------o o------------|---------------o
548 | |
549 | CP | MP
550 | Southbound | Southbound
551 | Interface | Interface
552 | |
553 *------------Y---------------------------------Y----------------*
554 | Device and resource Abstraction Layer (DAL) |
555 *------------Y---------------------------------Y----------------*
556 | | | |
557 | o-------Y----------o +-----+ o--------Y----------o |
558 | | Forwarding Plane | | App | | Operational Plane | |
559 | o------------------o +-----+ o-------------------o |
560 | Network Device |
561 +---------------------------------------------------------------+
563 Figure 2
565 4.2.1. The Application Plane
567 Per [RFC7426], the Application Plane includes both applications and
568 services. In particular, the Application Plane incorporates the User
569 Agent, a specialized application that interacts with the end user /
570 operator and performs requests for Deterministic Networking services
571 via an abstract Flow Management Entity, (FME) which may or may not be
572 collocated with (one of) the end systems.
574 At the Application Plane, a management interface enables the
575 negotiation of flows between end systems. An abstraction of the flow
576 called a Traffic Specification (TSpec) provides the representation.
577 This abstraction is used to place a reservation over the (Northbound)
578 Service Interface and within the Application plane. It is associated
579 with an abstraction of location, such as IP addresses and DNS names,
580 to identify the end systems and eventually specify intermediate relay
581 systems.
583 4.2.2. The Controller Plane
585 The Controller Plane corresponds to the aggregation of the Control
586 and Management Planes in [RFC7426], though Common Control and
587 Measurement Plane (CCAMP) [CCAMP] makes an additional distinction
588 between management and measurement. When the logical separation of
589 the Control, Measurement and other Management entities is not
590 relevant, the term Controller Plane is used for simplicity to
591 represent them all, and the term controller refers to any device
592 operating in that plane, whether is it a Path Computation entity or a
593 Network Management entity (NME). The Path Computation Element (PCE)
594 [PCE] is a core element of a controller, in charge of computing
595 Deterministic paths to be applied in the Network Plane.
597 A (Northbound) Service Interface enables applications in the
598 Application Plane to communicate with the entities in the Controller
599 Plane.
601 One or more PCE(s) collaborate to implement the requests from the FME
602 as Per-fFlow Per-Hop Behaviors installed in the relay systems for
603 each individual flow. The PCEs place each flow along a deterministic
604 sequence of relay systems so as to respect per-flow constraints such
605 as security and latency, and optimize the overall result for metrics
606 such as an abstract aggregated cost. The deterministic sequence can
607 typically be more complex than a direct sequence and include
608 redundancy path, with one or more packet replication and elimination
609 points.
611 4.2.3. The Network Plane
613 The Network Plane represents the network devices and protocols as a
614 whole, regardless of the Layer at which the network devices operate.
615 It includes Forwarding Plane (data plane), Application, and
616 Operational Plane (control plane) aspects.
618 The network Plane comprises the Network Interface Cards (NIC) in the
619 end systems, which are typically IP hosts, and relay systems, which
620 are typically IP routers and switches. Network-to-Network Interfaces
621 such as used for Traffic Engineering path reservation in [RFC5921],
622 as well as User-to-Network Interfaces (UNI) such as provided by the
623 Local Management Interface (LMI) between network and end systems, are
624 both part of the Network Plane, both in the control plane and the
625 data plane.
627 A Southbound (Network) Interface enables the entities in the
628 Controller Plane to communicate with devices in the Network Plane.
629 This interface leverages and extends TEAS to describe the physical
630 topology and resources in the Network Plane.
632 Flow Management Entity
634 End End
635 System System
637 -+-+-+-+-+-+-+ Northbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
639 PCE PCE PCE PCE
641 -+-+-+-+-+-+-+ Southbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
643 Relay Relay Relay Relay
644 System System System System
645 NIC NIC
646 Relay Relay Relay Relay
647 System System System System
649 Figure 3
651 The relay systems (and eventually the end systems NIC) expose their
652 capabilities and physical resources to the controller (the PCE), and
653 update the PCE with their dynamic perception of the topology, across
654 the Southbound Interface. In return, the PCE(s) set the per-flow
655 paths up, providing a Flow Characterization that is more tightly
656 coupled to the relay system Operation than a TSpec.
658 At the Network plane, relay systems may exchange information
659 regarding the state of the paths, between adjacent systems and
660 eventually with the end systems, and forward packets within
661 constraints associated to each flow, or, when unable to do so,
662 perform a last resort operation such as drop or declassify.
664 This specification focuses on the Southbound interface and the
665 operation of the Network Plane.
667 4.3. DetNet flows
669 4.3.1. Source guarantees
671 DetNet flows can by synchronous or asynchronous. In synchronous
672 DetNet flows, at least the relay systems (and possibly the end
673 systems) are closely time synchronized, typically to better than 1
674 microsecond. By transmitting packets from different DetNet flows or
675 classes of DetNet flows at different times, using repeating schedules
676 synchronized among the relay systems, resources such as buffers and
677 link bandwidth can be shared over the time domain among different
678 DetNet flows. There is a tradeoff among techniques for synchronous
679 DetNet flows between the burden of fine-grained scheduling and the
680 benefit of reducing the required resources, especially buffer space.
682 In contrast, asynchronous DetNet flows are not coordinated with a
683 fine-grained schedule, so relay and end systems must assume worst-
684 case interference among DetNet flows contending for buffer resources.
685 Asynchronous DetNet flows are characterized by:
687 o A maximum packet size;
689 o An observation interval; and
691 o A maximum number of transmissions during that observation
692 interval.
694 These parameters, together with knowledge of the protocol stack used
695 (and thus the size of the various headers added to a packet), limit
696 the number of bit times per observation interval that the DetNet flow
697 can occupy the physical medium.
699 The source promises that these limits will not be exceeded. If the
700 source transmits less data than this limit allows, the unused
701 resources such as link bandwidth can be made available by the system
702 to non-DetNet packets. However, making those resources available to
703 DetNet packets in other DetNet flows would serve no purpose. Those
704 other DetNet flows have their own dedicated resources, on the
705 assumption that all DetNet flows can use all of their resources over
706 a long period of time.
708 Note that there is no provision in DetNet for throttling DetNet flows
709 (reducing the transmission rate via feedback); the assumption is that
710 a DetNet flow, to be useful, must be delivered in its entirety. That
711 is, while any useful application is written to expect a certain
712 number of lost packets, the real-time applications of interest to
713 DetNet demand that the loss of data due to the network is
714 extraordinarily infrequent.
716 Although DetNet strives to minimize the changes required of an
717 application to allow it to shift from a special-purpose digital
718 network to an Internet Protocol network, one fundamental shift in the
719 behavior of network applications is impossible to avoid--the
720 reservation of resources before the application starts. In the first
721 place, a network cannot deliver finite latency and practically zero
722 packet loss to an arbitrarily high offered load. Secondly, achieving
723 practically zero packet loss for unthrottled (though bandwidth
724 limited) DetNet flows means that bridges and routers have to dedicate
725 buffer resources to specific DetNet flows or to classes of DetNet
726 flows. The requirements of each reservation have to be translated
727 into the parameters that control each system's queuing, shaping, and
728 scheduling functions and delivered to the hosts, bridges, and
729 routers.
731 4.3.2. Incomplete Networks
733 The presence in the network of relay systems that are not fully
734 capable of offering DetNet services complicates the ability of the
735 relay systems and/or controller to allocate resources, as extra
736 buffering, and thus extra latency, must be allocated at points
737 downstream from the non-DetNet relay system for a DetNet flow.
739 4.4. Queuing, Shaping, Scheduling, and Preemption
741 As described above, DetNet achieves its aims by reserving bandwidth
742 and buffer resources at every hop along the path of the DetNet flow.
743 The reservation itself is not sufficient, however. Implementors and
744 users of a number of proprietary and standard real-time networks have
745 found that standards for specific data plane techniques are required
746 to enable these assurances to be made in a multi-vendor network. The
747 fundamental reason is that latency variation in one system results in
748 the need for extra buffer space in the next-hop system(s), which in
749 turn, increases the worst-case per-hop latency.
751 Standard queuing and transmission selection algorithms allow a
752 central controller to compute the latency contribution of each relay
753 node to the end-to-end latency, to compute the amount of buffer space
754 required in each relay system for each incremental DetNet flow, and
755 most importantly, to translate from a flow specification to a set of
756 values for the managed objects that control each relay or end system.
757 The IEEE 802 has specified (and is specifying) a set of queuing,
758 shaping, and scheduling algorithms that enable each relay system
759 (bridge or router), and/or a central controller, to compute these
760 values. These algorithms include:
762 o A credit-based shaper [IEEE802.1Q-2014] Clause 34.
764 o Time-gated queues governed by a rotating time schedule,
765 synchronized among all relay nodes [IEEE802.1Qbv].
767 o Synchronized double (or triple) buffers driven by synchronized
768 time ticks. [IEEE802.1Qch].
770 o Pre-emption of an Ethernet packet in transmission by a packet with
771 a more stringent latency requirement, followed by the resumption
772 of the preempted packet [IEEE802.1Qbu], [IEEE802.3br].
774 While these techniques are currently embedded in Ethernet and
775 bridging standards, we can note that they are all, except perhaps for
776 packet preemption, equally applicable to other media than Ethernet,
777 and to routers as well as bridges.
779 4.5. Coexistence with normal traffic
781 A DetNet network supports the dedication of a high proportion (e.g.
782 75%) of the network bandwidth to DetNet flows. But, no matter how
783 much is dedicated for DetNet flows, it is a goal of DetNet to coexist
784 with with existing Class of Service schemes (e.g., DiffServ). It is
785 also important that non-DetNet traffic not disrupt the DetNet flow,
786 of course (see Section 4.6 and Section 7). For these reasons:
788 o Bandwidth (transmission opportunities) not utilized by a DetNet
789 flow are available to non-DetNet packets (though not to other
790 DetNet flows).
792 o DetNet flows can be shaped or scheduled, in order to ensure that
793 the highest-priority non-DetNet packet also is ensured a worst-
794 case latency (at any given hop).
796 o When transmission opportunities for DetNet flows are scheduled in
797 detail, then the algorithm constructing the schedule should leave
798 sufficient opportunities for non-DetNet packets to satisfy the
799 needs of the users of the network. Detailed scheduling can also
800 permit the time-shared use of buffer resources by different DetNet
801 flows.
803 Ideally, the net effect of the presence of DetNet flows in a network
804 on the non-DetNet packets is primarily a reduction in the available
805 bandwidth.
807 4.6. Fault Mitigation
809 One key to building robust real-time systems is to reduce the
810 infinite variety of possible failures to a number that can be
811 analyzed with reasonable confidence. DetNet aids in the process by
812 providing filters and policers to detect DetNet packets received on
813 the wrong interface, or at the wrong time, or in too great a volume,
814 and to then take actions such as discarding the offending packet,
815 shutting down the offending DetNet flow, or shutting down the
816 offending interface.
818 It is also essential that filters and service remarking be employed
819 at the network edge to prevent non-DetNet packets from being mistaken
820 for DetNet packets, and thus impinging on the resources allocated to
821 DetNet packets.
823 There exist techniques, at present and/or in various stages of
824 standardization, that can perform these fault mitigation tasks that
825 deliver a high probability that misbehaving systems will have zero
826 impact on well-behaved DetNet flows, except of course, for the
827 receiving interface(s) immediately downstream of the misbehaving
828 device.
830 4.7. Protocol Stack Model
832 [IEEE802.1CB], Annex C, offers a description of the TSN protocol
833 stack. It may serve as the foundation for the DetNet model which
834 will be defined by the working group. While this standard is a work
835 in progress, a consensus around the basic architecture has formed.
836 This stack is summarized in Figure 4.
838 DetNet Protocol Stack
840 +--------------------------------+
841 | Upper Layers |
842 +--------------------------------+
843 | Sequence generation/recovery |
844 +--------------------------------+
845 | DetNet flow splitting/merging |
846 +--------------------------------+
847 | Sequence encode/decode |
848 +--------------------------------+
849 | DetNet flow encode/decode |
850 +--------------------------------+
851 | Lower layers |
852 +--------------------------------+
854 Figure 4
856 Not all layers are required for any given application, or even for
857 any given network. The layers are, from top to bottom:
859 Sequence generation/recovery
860 Supplies the sequence number for packet replication and
861 deletion (Section 3.3) for packets going down the stack (if
862 not already present), and discards duplicate packets coming
863 up the stack.
865 DetNet flow splitting/merging
866 Replicates packets going down the stack into two DetNet
867 flows, and merges DetNet flows together for packets coming up
868 the stack, based on the packet's DetNet flow identifier.
869 Needed for packet replication and deletion (Section 3.3).
871 Sequence encode/decode
872 Encodes the sequence number into packets going down the
873 stack, and extracts the sequence number from packets coming
874 up the stack. This function may or may not be a null
875 transformation of the packet, and for some protocols, is not
876 explicitly present, being included in the DetNet flow encode/
877 decode layer, below.
879 DetNet flow encode/decode
880 Encapsulates packets going down the stack, based on the
881 packet's locally-significant DetNet flow identifier, in order
882 to identify to which DetNet flow the packet belongs, and
883 extracts a locally-significant DetNet flow identifier from
884 packets coming up the stack. This may be a null
885 transformation (e.g., for DetNet flows identified by IP
886 5-tuple) or might be an explicit encapsulation (e.g., for
887 DetNet flows identified with an MPLS label). DetNet flow
888 identification is the basis for packet replication and
889 deletion, for assigning per-flow resources (if any) to
890 packets and for defense against misbehaving systems
891 (Section 4.6). When DetNet flows are assigned to pinned
892 paths, this layer can be indistinguishable from the data
893 forwarding layer(s).
895 The reader is likely to notice that Figure 4 does not specify the
896 relationship between the DetNet layers, the IP layers, and the link
897 layers. This is intentional, because they can usefully be placed
898 different places in the stack, and even in mulitple places, depending
899 on where their peers are placed.
901 4.8. Advertising resources, capabilities and adjacencies
903 There are three classes of information that a central controller or
904 decentralized control plane needs to know that can only be obtained
905 from the end systems and/or relay systems in the network. When using
906 a peer-to-peer control plane, some of this information may be
907 required by a system's neighbors in the network.
909 o Details of the system's capabilities that are required in order to
910 accurately allocate that system's resources, as well as other
911 systems' resources. This includes, for example, which specific
912 queuing and shaping algorithms are implemented (Section 4.4), the
913 number of buffers dedicated for DetNet allocation, and the worst-
914 case forwarding delay.
916 o The dynamic state of an end or relay system's DetNet resources.
918 o The identity of the system's neighbors, and the characteristics of
919 the link(s) between the systems, including the length (in
920 nanoseconds) of the link(s).
922 4.9. Provisioning model
924 4.9.1. Centralized Path Computation and Installation
926 A centralized routing model, such as provided with a PCE (RFC 4655
927 [RFC4655]), enables global and per-flow optimizations. (See
928 Section 4.2.) The model is attractive but a number of issues are
929 left to be solved. In particular:
931 o Whether and how the path computation can be installed by 1) an end
932 device or 2) a Network Management entity,
934 o And how the path is set up, either by installing state at each hop
935 with a direct interaction between the forwarding device and the
936 PCE, or along a path by injecting a source-routed request at one
937 end of the path.
939 4.9.2. Distributed Path Setup
941 Whether a distributed alternative without a PCE can be valuable
942 should be studied as well. Such an alternative could for instance
943 inherit from the Resource ReSerVation Protocol [RFC3209] (RSVP-TE)
944 flows.
946 In a Layer-2 only environment, or as part of a layered approach to a
947 mixed environment, IEEE 802.1 also has work, either completed or in
948 progress. [IEEE802.1Q-2014] Clause 35 describes SRP, a peer-to-peer
949 protocol for Layer-2 roughly analogous to RSVP. Almost complete is
950 [IEEE802.1Qca], which defines how ISIS can provide multiple disjoint
951 paths or distribution trees. Also in progress is [IEEE802.1Qcc],
952 which expands the capabilities of SRP.
954 The integration/interaction of the DetNet control layer an underlying
955 IEEE 802.1 sub-network control layer will need to be defined.
957 4.10. Scaling to larger networks
959 Reservations for individual DetNet flows require considerable state
960 information in each relay system, especially when adequate fault
961 mitigation (Section 4.6) is required. The DetNet data plane, in
962 order to support larger numbers of DetNet flows, must support the
963 aggregation of DetNet flows into tunnels, which themselves can be
964 viewed by the relay systems' data planes largely as individual DetNet
965 flows. Without such aggregation, the per-relay system may limit the
966 scale of DetNet networks.
968 4.11. Connected islands vs. networks
970 Given that users have deployed examples of the IEEE 802.1 TSN TG
971 standards, which provide capabilities similar to DetNet, it is
972 obvious to ask whether the IETF DetNet effort can be limited to
973 providing Layer-2 connections (VPNs) between islands of bridged TSN
974 networks. While this capability is certainly useful to some
975 applications, and must not be precluded by DetNet, tunneling alone is
976 not a sufficient goal for the DetNet WG. As shown in the
977 Deterministic Networking Use Cases draft [I-D.ietf-detnet-use-cases],
978 there are already deployments of Layer-2 TSN networks that are
979 encountering the well-known problems of over-large broadcast domains.
980 Routed solutions, and combinations routed/bridged solutions, are both
981 required.
983 5. Compatibility with Layer-2
985 Standards providing similar capabilities for bridged networks (only)
986 have been and are being generated in the IEEE 802 LAN/MAN Standards
987 Committee. The present architecture describes an abstract model that
988 can be applicable both at Layer-2 and Layer-3, and over links not
989 defined by IEEE 802. It is the intention of the authors (and
990 hopefully, as this draft progresses, of the DetNet Working Group)
991 that IETF and IEEE 802 will coordinate their work, via the
992 participation of common individuals, liaisons, and other means, to
993 maximize the compatibility of their outputs.
995 DetNet enabled systems and nodes can be interconnected by sub-
996 networks, i.e., Layer 2 technologies. These sub-networks will
997 provide DetNet compatible service for support of DetNet traffic.
998 Examples of sub-networks include 802.1TSN and a point-to-point OTN
999 link. Of course, multi-layer DetNet systems may be possible too,
1000 where one DetNet appears as a sub-network, and provides service to, a
1001 higher layer DetNet system.
1003 6. Open Questions
1005 There are a number of architectural questions that will have to be
1006 resolved before this document can be submitted for publication.
1007 Aside from the obvious fact that this present draft is subject to
1008 change, there are specific questions to which the authors wish to
1009 direct the readers' attention.
1011 6.1. Data plane shapers and schedulers
1013 A number of techniques have been defined and are being defined by
1014 IEEE 802 for queuing, shaping, and scheduling transmissions on
1015 EtherNet media, most of which are directly applicable to any other
1016 medium. Specific selections of supported techniques are required,
1017 because minimizing, and even eliminating, congestion losses depends
1018 strongly on the details of the per-hop behavior of sources and relay
1019 systems.
1021 The present authors expect that, at least, the IEEE 802 mechanisms
1022 will be supported.
1024 6.2. DetNet flow identification and sequencing
1026 The techniques to be used for DetNet flow identification must be
1027 settled. The following paragraphs provide a snapshot of the authors'
1028 opinions at the time of writing. These authors anticipate the
1029 submission of drafts in the near future on this subject.
1031 IEEE 802.1 TSN streams are identified by giving each stream (DetNet
1032 flow) a {VLAN identifier, destination MAC address} pair that is
1033 unique in the bridged network, and that the MAC address must be a
1034 multicast address. If a source is generating, for example, two
1035 unicast UDP flows to the same destination, one DetNet and one not,
1036 the DetNet flow's packets must be transformed at some point to have a
1037 multicast destination MAC address, and perhaps, a different VLAN than
1038 the non-DetNet flow's packets.
1040 A similar provision would apply to DetNet packets that are identified
1041 by MPLS labels; any bridges between the LSRs need a {VLAN identifier,
1042 destination MAC address} pair uniquely identifying the DetNet flow in
1043 the bridged network.
1045 Provision is made in current draft of [IEEE802.1CB] to make these
1046 transformations either in a Layer-2 shim in the source end system, on
1047 the output side of a router or LSR, or in a proxy function in the
1048 first-hop bridge. It remains to be seen whether this provision is
1049 adequate and/or acceptable to the IETF DetNet WG.
1051 There are also questions regarding the sequentialization of packets
1052 for use with packet replication and deletion (Section 3.3).
1053 [IEEE802.1CB] defines an EtherNet tag carrying a sequence number. If
1054 MPLS Pseudowires are used with a control word containing a sequence
1055 number, the relationship and interworking between these two formats
1056 must be defined.
1058 6.3. Flat vs. hierarchical control
1060 Boxes that are solely routers or solely bridges are rare in today's
1061 market. In a multi-tenant data center, multiple users' virtual
1062 Layer-2/Layer-3 topologies exist simultaneously, implemented on a
1063 network whose physical topology bears only accidental resemblance to
1064 the virtual topologies.
1066 While the forwarding topology (the bridges and routers) are an
1067 important consideration for a DetNet Flow Management Entity
1068 (Section 4.2.1), so is the purely physical topology. Ultimately, the
1069 model used by the management entities is based on boxes, queues, and
1070 links. The authors hope that the work of the TEAS WG will help to
1071 clarify exactly what model parameters need to be traded between the
1072 relay systems and the controller(s).
1074 6.4. Peer-to-peer reservation protocol
1076 As described in Section 4.9.2, the DetNet WG needs to decide whether
1077 to support a peer-to-peer protocol for a source and a destination to
1078 reserve resources for a DetNet stream. Assuming that enabling the
1079 involvement of the source and/or destination is desirable (see
1080 Deterministic Networking Use Cases [I-D.ietf-detnet-use-cases]), it
1081 remains to decide whether the DetNet WG will make it possible to
1082 deploy at least some DetNet capabilities in a network using only a
1083 peer-to-peer protocol, without a central controller.
1085 (Note that a UNI (see Section 4.2.3) between an end system and an
1086 edge relay system, for sources and/or listeners to request DetNet
1087 services, can be either the first hop of a per-to-peer reservation
1088 protocol, or can be deflected by the edge relay system to a central
1089 controller for resolution. Similarly, a decision by a central
1090 controller can be effected by the controller instructing the end
1091 system or edge relay system to initiate a per-to-peer protocol
1092 activity.)
1094 6.5. Wireless media interactions
1096 Deterministic Networking Use Cases [I-D.ietf-detnet-use-cases]
1097 illustrates cases where wireless media are needed in a DetNet
1098 network. Some wireless media in general use, such as IEEE 802.11
1099 [IEEE802.1Q-2014], have significantly higher packet loss rates than
1100 typical wired media, such as Ethernet [IEEE802.3-2012]. IEEE 802.11
1101 includes support for such features as MAC-layer acknowledgements and
1102 retransmissions.
1104 The techniques described in Section 3 are likely to improve the
1105 ability of a mixed wired/wireless network to offer the DetNet QoS
1106 features. The interaction of these techniques with the features of
1107 specific wireless media, although they may be significant, cannot be
1108 addressed in this document. It remains to be decided to what extent
1109 the DetNet WG will address them, and to what extent other WGs, e.g.
1110 6TiSCH, will do so.
1112 7. Security Considerations
1114 Security in the context of Deterministic Networking has an added
1115 dimension; the time of delivery of a packet can be just as important
1116 as the contents of the packet, itself. A man-in-the-middle attack,
1117 for example, can impose, and then systematically adjust, additional
1118 delays into a link, and thus disrupt or subvert a real-time
1119 application without having to crack any encryption methods employed.
1120 See [RFC7384] for an exploration of this issue in a related context.
1122 Furthermore, in a control system where millions of dollars of
1123 equipment, or even human lives, can be lost if the DetNet QoS is not
1124 delivered, one must consider not only simple equipment failures,
1125 where the box or wire instantly becomes perfectly silent, but bizarre
1126 errors such as can be caused by software failures. Because there is
1127 essential no limit to the kinds of failures that can occur,
1128 protecting against realistic equipment failures is indistinguishable,
1129 in most cases, from protecting against malicious behavior, whether
1130 accidental or intentional. See also Section 4.6.
1132 Security must cover:
1134 o the protection of the signaling protocol
1136 o the authentication and authorization of the controlling systems
1138 o the identification and shaping of the DetNet flows
1140 8. IANA Considerations
1142 This document does not require an action from IANA.
1144 9. Acknowledgements
1146 The authors wish to thank Jouni Korhonen, Erik Nordmark, George
1147 Swallow, Rudy Klecka, Anca Zamfir, David Black, Thomas Watteyne,
1148 Shitanshu Shah, Craig Gunther, Rodney Cummings, Wilfried Steiner,
1149 Marcel Kiessling, Karl Weber, Ethan Grossman, Pat Thaler, and Lou
1150 Berger for their various contribution with this work.
1152 10. Access to IEEE 802.1 documents
1154 To access password protected IEEE 802.1 drafts, see the IETF IEEE
1155 802.1 information page at https://www.ietf.org/proceedings/52/slides/
1156 bridge-0/tsld003.htm.
1158 11. Informative References
1160 [AVnu] http://www.avnu.org/, "The AVnu Alliance tests and
1161 certifies devices for interoperability, providing a simple
1162 and reliable networking solution for AV network
1163 implementation based on the Audio Video Bridging (AVB)
1164 standards.".
1166 [CCAMP] IETF, "Common Control and Measurement Plane",
1167 .
1169 [HART] www.hartcomm.org, "Highway Addressable Remote Transducer,
1170 a group of specifications for industrial process and
1171 control devices administered by the HART Foundation".
1173 [HSR-PRP] IEC, "High availability seamless redundancy (HSR) is a
1174 further development of the PRP approach, although HSR
1175 functions primarily as a protocol for creating media
1176 redundancy while PRP, as described in the previous
1177 section, creates network redundancy. PRP and HSR are both
1178 described in the IEC 62439 3 standard.",
1179 .
1182 [I-D.finn-detnet-problem-statement]
1183 Finn, N. and P. Thubert, "Deterministic Networking Problem
1184 Statement", draft-finn-detnet-problem-statement-05 (work
1185 in progress), March 2016.
1187 [I-D.ietf-6tisch-architecture]
1188 Thubert, P., "An Architecture for IPv6 over the TSCH mode
1189 of IEEE 802.15.4", draft-ietf-6tisch-architecture-09 (work
1190 in progress), November 2015.
1192 [I-D.ietf-6tisch-tsch]
1193 Watteyne, T., Palattella, M., and L. Grieco, "Using
1194 IEEE802.15.4e TSCH in an IoT context: Overview, Problem
1195 Statement and Goals", draft-ietf-6tisch-tsch-06 (work in
1196 progress), March 2015.
1198 [I-D.ietf-detnet-use-cases]
1199 Grossman, E., Gunther, C., Thubert, P., Wetterwald, P.,
1200 Raymond, J., Korhonen, J., Kaneko, Y., Das, S., Zha, Y.,
1201 Varga, B., Farkas, J., Goetz, F., and J. Schmitt,
1202 "Deterministic Networking Use Cases", draft-ietf-detnet-
1203 use-cases-08 (work in progress), March 2016.
1205 [I-D.ietf-roll-rpl-industrial-applicability]
1206 Phinney, T., Thubert, P., and R. Assimiti, "RPL
1207 applicability in industrial networks", draft-ietf-roll-
1208 rpl-industrial-applicability-02 (work in progress),
1209 October 2013.
1211 [I-D.svshah-tsvwg-deterministic-forwarding]
1212 Shah, S. and P. Thubert, "Deterministic Forwarding PHB",
1213 draft-svshah-tsvwg-deterministic-forwarding-04 (work in
1214 progress), August 2015.
1216 [IEEE802.11-2012]
1217 IEEE, "Wireless LAN Medium Access Control (MAC) and
1218 Physical Layer (PHY) Specifications", 2012,
1219 .
1222 [IEEE802.1AS-2011]
1223 IEEE, "Timing and Synchronizations (IEEE 802.1AS-2011)",
1224 2011, .
1227 [IEEE802.1BA-2011]
1228 IEEE, "AVB Systems (IEEE 802.1BA-2011)", 2011,
1229 .
1232 [IEEE802.1CB]
1233 IEEE, "Frame Replication and Elimination for Reliability
1234 (IEEE Draft P802.1CB)", 2016,
1235 .
1237 [IEEE802.1Q-2014]
1238 IEEE, "MAC Bridges and VLANs (IEEE 802.1Q-2014", 2014,
1239 .
1242 [IEEE802.1Qbu]
1243 IEEE, "Frame Preemption", 2016,
1244 .
1246 [IEEE802.1Qbv]
1247 IEEE, "Enhancements for Scheduled Traffic", 2016,
1248 .
1250 [IEEE802.1Qca]
1251 IEEE, "Path Control and Reservation", 2015,
1252 .
1254 [IEEE802.1Qcc]
1255 IEEE, "Stream Reservation Protocol (SRP) Enhancements and
1256 Performance Improvements", 2016,
1257 .
1259 [IEEE802.1Qch]
1260 IEEE, "Cyclic Queuing and Forwarding", 2016,
1261 .
1263 [IEEE802.1TSNTG]
1264 IEEE Standards Association, "IEEE 802.1 Time-Sensitive
1265 Networks Task Group", 2013,
1266 .
1268 [IEEE802.3-2012]
1269 IEEE, "IEEE Stabdard for Ethernet", 2012,
1270 .
1273 [IEEE802.3br]
1274 IEEE, "Interspersed Express Traffic", 2016,
1275 .
1277 [IEEE802154]
1278 IEEE standard for Information Technology, "IEEE std.
1279 802.15.4, Part. 15.4: Wireless Medium Access Control (MAC)
1280 and Physical Layer (PHY) Specifications for Low-Rate
1281 Wireless Personal Area Networks", June 2011.
1283 [IEEE802154e]
1284 IEEE standard for Information Technology, "IEEE std.
1285 802.15.4e, Part. 15.4: Low-Rate Wireless Personal Area
1286 Networks (LR-WPANs) Amendment 1: MAC sublayer", April
1287 2012.
1289 [ISA100.11a]
1290 ISA/IEC, "ISA100.11a, Wireless Systems for Automation,
1291 also IEC 62734", 2011, < http://www.isa100wci.org/en-
1292 US/Documents/PDF/3405-ISA100-WirelessSystems-Future-broch-
1293 WEB-ETSI.aspx>.
1295 [ISA95] ANSI/ISA, "Enterprise-Control System Integration Part 1:
1296 Models and Terminology", 2000, .
1299 [ODVA] http://www.odva.org/, "The organization that supports
1300 network technologies built on the Common Industrial
1301 Protocol (CIP) including EtherNet/IP.".
1303 [PCE] IETF, "Path Computation Element",
1304 .
1306 [Profinet]
1307 http://us.profinet.com/technology/profinet/, "PROFINET is
1308 a standard for industrial networking in automation.",
1309 .
1311 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.
1312 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
1313 Functional Specification", RFC 2205, DOI 10.17487/RFC2205,
1314 September 1997, .
1316 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
1317 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
1318 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001,
1319 .
1321 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation
1322 Element (PCE)-Based Architecture", RFC 4655,
1323 DOI 10.17487/RFC4655, August 2006,
1324 .
1326 [RFC5673] Pister, K., Ed., Thubert, P., Ed., Dwars, S., and T.
1327 Phinney, "Industrial Routing Requirements in Low-Power and
1328 Lossy Networks", RFC 5673, DOI 10.17487/RFC5673, October
1329 2009, .
1331 [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau,
1332 L., and L. Berger, "A Framework for MPLS in Transport
1333 Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010,
1334 .
1336 [RFC6372] Sprecher, N., Ed. and A. Farrel, Ed., "MPLS Transport
1337 Profile (MPLS-TP) Survivability Framework", RFC 6372,
1338 DOI 10.17487/RFC6372, September 2011,
1339 .
1341 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in
1342 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384,
1343 October 2014, .
1345 [RFC7426] Haleplidis, E., Ed., Pentikousis, K., Ed., Denazis, S.,
1346 Hadi Salim, J., Meyer, D., and O. Koufopavlou, "Software-
1347 Defined Networking (SDN): Layers and Architecture
1348 Terminology", RFC 7426, DOI 10.17487/RFC7426, January
1349 2015, .
1351 [TEAS] IETF, "Traffic Engineering Architecture and Signaling",
1352 .
1354 [WirelessHART]
1355 www.hartcomm.org, "Industrial Communication Networks -
1356 Wireless Communication Network and Communication Profiles
1357 - WirelessHART - IEC 62591", 2010.
1359 Authors' Addresses
1360 Norman Finn
1361 Cisco Systems
1362 170 W Tasman Dr.
1363 San Jose, California 95134
1364 USA
1366 Phone: +1 408 526 4495
1367 Email: nfinn@cisco.com
1369 Pascal Thubert
1370 Cisco Systems
1371 Village d'Entreprises Green Side
1372 400, Avenue de Roumanille
1373 Batiment T3
1374 Biot - Sophia Antipolis 06410
1375 FRANCE
1377 Phone: +33 4 97 23 26 34
1378 Email: pthubert@cisco.com
1380 Michael Johas Teener
1381 Broadcom Corp.
1382 3151 Zanker Rd.
1383 San Jose, California 95134
1384 USA
1386 Phone: +1 831 824 4228
1387 Email: MikeJT@broadcom.com