idnits 2.17.1
draft-ietf-detnet-architecture-05.txt:
Checking boilerplate required by RFC 5378 and the IETF Trust (see
https://trustee.ietf.org/license-info):
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt:
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/checklist :
----------------------------------------------------------------------------
No issues found here.
Miscellaneous warnings:
----------------------------------------------------------------------------
== The copyright year in the IETF Trust and authors Copyright Line does not
match the current year
== Line 639 has weird spacing: '...e stack v ...'
== Line 1141 has weird spacing: '...mediate inter...'
== Line 1144 has weird spacing: '...mediate inter...'
-- The document date (May 1, 2018) is 2180 days in the past. Is this
intentional?
Checking references for intended status: Proposed Standard
----------------------------------------------------------------------------
(See RFCs 3967 and 4897 for information about using normative references
to lower-maturity documents in RFCs)
== Missing Reference: 'Network' is mentioned on line 775, but not defined
== Unused Reference: 'AVnu' is defined on line 1623, but no explicit
reference was found in the text
== Unused Reference: 'I-D.ietf-6tisch-architecture' is defined on line
1647, but no explicit reference was found in the text
== Unused Reference: 'I-D.ietf-6tisch-tsch' is defined on line 1652, but no
explicit reference was found in the text
== Unused Reference: 'I-D.ietf-roll-rpl-industrial-applicability' is
defined on line 1667, but no explicit reference was found in the text
== Unused Reference: 'I-D.svshah-tsvwg-deterministic-forwarding' is defined
on line 1673, but no explicit reference was found in the text
== Unused Reference: 'I-D.varga-detnet-service-model' is defined on line
1678, but no explicit reference was found in the text
== Unused Reference: 'IEEE802.1AS-2011' is defined on line 1683, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802.1TSNTG' is defined on line 1728, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802.3-2015' is defined on line 1733, but no
explicit reference was found in the text
== Unused Reference: 'ISA95' is defined on line 1743, but no explicit
reference was found in the text
== Unused Reference: 'ODVA' is defined on line 1747, but no explicit
reference was found in the text
== Unused Reference: 'Profinet' is defined on line 1754, but no explicit
reference was found in the text
== Unused Reference: 'RFC5673' is defined on line 1810, but no explicit
reference was found in the text
== Outdated reference: A later version (-30) exists of
draft-ietf-6tisch-architecture-13
== Outdated reference: A later version (-09) exists of
draft-ietf-detnet-problem-statement-03
== Outdated reference: A later version (-20) exists of
draft-ietf-detnet-use-cases-15
-- Obsolete informational reference (is this intentional?): RFC 5316
(Obsoleted by RFC 9346)
Summary: 0 errors (**), 0 flaws (~~), 21 warnings (==), 2 comments (--).
Run idnits with the --verbose option for more detailed information about
the items above.
--------------------------------------------------------------------------------
2 DetNet N. Finn
3 Internet-Draft Huawei
4 Intended status: Standards Track P. Thubert
5 Expires: November 2, 2018 Cisco
6 B. Varga
7 J. Farkas
8 Ericsson
9 May 1, 2018
11 Deterministic Networking Architecture
12 draft-ietf-detnet-architecture-05
14 Abstract
16 Deterministic Networking (DetNet) provides a capability to carry
17 specified unicast or multicast data flows for real-time applications
18 with extremely low data loss rates and bounded latency. Techniques
19 used include: 1) reserving data plane resources for individual (or
20 aggregated) DetNet flows in some or all of the intermediate nodes
21 (e.g. bridges or routers) along the path of the flow; 2) providing
22 explicit routes for DetNet flows that do not rapidly change with the
23 network topology; and 3) distributing data from DetNet flow packets
24 over time and/or space to ensure delivery of each packet's data' in
25 spite of the loss of a path. The capabilities can be managed by
26 configuration, or by manual or automatic network management.
28 Status of This Memo
30 This Internet-Draft is submitted in full conformance with the
31 provisions of BCP 78 and BCP 79.
33 Internet-Drafts are working documents of the Internet Engineering
34 Task Force (IETF). Note that other groups may also distribute
35 working documents as Internet-Drafts. The list of current Internet-
36 Drafts is at https://datatracker.ietf.org/drafts/current/.
38 Internet-Drafts are draft documents valid for a maximum of six months
39 and may be updated, replaced, or obsoleted by other documents at any
40 time. It is inappropriate to use Internet-Drafts as reference
41 material or to cite them other than as "work in progress."
43 This Internet-Draft will expire on November 2, 2018.
45 Copyright Notice
47 Copyright (c) 2018 IETF Trust and the persons identified as the
48 document authors. All rights reserved.
50 This document is subject to BCP 78 and the IETF Trust's Legal
51 Provisions Relating to IETF Documents
52 (https://trustee.ietf.org/license-info) in effect on the date of
53 publication of this document. Please review these documents
54 carefully, as they describe your rights and restrictions with respect
55 to this document. Code Components extracted from this document must
56 include Simplified BSD License text as described in Section 4.e of
57 the Trust Legal Provisions and are provided without warranty as
58 described in the Simplified BSD License.
60 Table of Contents
62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
63 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
64 2.1. Terms used in this document . . . . . . . . . . . . . . . 4
65 2.2. IEEE 802 TSN to DetNet dictionary . . . . . . . . . . . . 6
66 3. Providing the DetNet Quality of Service . . . . . . . . . . . 7
67 3.1. Primary goals defining the DetNet QoS . . . . . . . . . . 7
68 3.2. Mechanisms to achieve DetNet Qos . . . . . . . . . . . . 9
69 3.2.1. Congestion protection . . . . . . . . . . . . . . . . 9
70 3.2.2. Explicit routes . . . . . . . . . . . . . . . . . . . 9
71 3.2.3. Jitter Reduction . . . . . . . . . . . . . . . . . . 10
72 3.2.4. Packet Replication and Elimination . . . . . . . . . 11
73 3.2.5. Packet encoding for service protection . . . . . . . 12
74 3.3. Secondary goals for DetNet . . . . . . . . . . . . . . . 13
75 3.3.1. Coexistence with normal traffic . . . . . . . . . . . 13
76 3.3.2. Fault Mitigation . . . . . . . . . . . . . . . . . . 13
77 4. DetNet Architecture . . . . . . . . . . . . . . . . . . . . . 14
78 4.1. DetNet stack model . . . . . . . . . . . . . . . . . . . 14
79 4.1.1. Representative Protocol Stack Model . . . . . . . . . 14
80 4.1.2. DetNet Data Plane Overview . . . . . . . . . . . . . 16
81 4.1.3. Network reference model . . . . . . . . . . . . . . . 18
82 4.2. DetNet systems . . . . . . . . . . . . . . . . . . . . . 19
83 4.2.1. End system . . . . . . . . . . . . . . . . . . . . . 19
84 4.2.2. DetNet edge, relay, and transit nodes . . . . . . . . 20
85 4.3. DetNet flows . . . . . . . . . . . . . . . . . . . . . . 21
86 4.3.1. DetNet flow types . . . . . . . . . . . . . . . . . . 21
87 4.3.2. Source guarantees . . . . . . . . . . . . . . . . . . 21
88 4.3.3. Incomplete Networks . . . . . . . . . . . . . . . . . 23
89 4.4. Traffic Engineering for DetNet . . . . . . . . . . . . . 23
90 4.4.1. The Application Plane . . . . . . . . . . . . . . . . 23
91 4.4.2. The Controller Plane . . . . . . . . . . . . . . . . 24
92 4.4.3. The Network Plane . . . . . . . . . . . . . . . . . . 24
94 4.5. Queuing, Shaping, Scheduling, and Preemption . . . . . . 25
95 4.6. Service instance . . . . . . . . . . . . . . . . . . . . 26
96 4.7. Flow identification at technology borders . . . . . . . . 27
97 4.7.1. Exporting flow identification . . . . . . . . . . . . 27
98 4.7.2. Flow attribute mapping between layers . . . . . . . . 29
99 4.7.3. Flow-ID mapping examples . . . . . . . . . . . . . . 30
100 4.8. Advertising resources, capabilities and adjacencies . . . 32
101 4.9. Provisioning model . . . . . . . . . . . . . . . . . . . 32
102 4.9.1. Centralized Path Computation and Installation . . . . 32
103 4.9.2. Distributed Path Setup . . . . . . . . . . . . . . . 32
104 4.10. Scaling to larger networks . . . . . . . . . . . . . . . 33
105 4.11. Connected islands vs. networks . . . . . . . . . . . . . 33
106 4.12. Compatibility with Layer-2 . . . . . . . . . . . . . . . 33
107 5. Security Considerations . . . . . . . . . . . . . . . . . . . 34
108 6. Privacy Considerations . . . . . . . . . . . . . . . . . . . 34
109 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 35
110 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 35
111 9. Access to IEEE 802.1 documents . . . . . . . . . . . . . . . 35
112 10. Informative References . . . . . . . . . . . . . . . . . . . 35
113 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 40
115 1. Introduction
117 Deterministic Networking (DetNet) is a service that can be offered by
118 a network to DetNet flows. DetNet provides these flows extremely low
119 packet loss rates and assured maximum end-to-end delivery latency.
120 This is accomplished by dedicating network resources such as link
121 bandwidth and buffer space to DetNet flows and/or classes of DetNet
122 flows, and by replicating packets along multiple paths. Unused
123 reserved resources are available to non-DetNet packets.
125 The Deterministic Networking Problem Statement
126 [I-D.ietf-detnet-problem-statement] introduces Deterministic
127 Networking, and Deterministic Networking Use Cases
128 [I-D.ietf-detnet-use-cases] summarizes the need for it. See
129 [I-D.dt-detnet-dp-alt] for a discussion of specific techniques that
130 can be used to identify DetNet Flows and assign them to specific
131 paths through a network.
133 A goal of DetNet is a converged network in all respects. That is,
134 the presence of DetNet flows does not preclude non-DetNet flows, and
135 the benefits offered DetNet flows should not, except in extreme
136 cases, prevent existing QoS mechanisms from operating in a normal
137 fashion, subject to the bandwidth required for the DetNet flows. A
138 single source-destination pair can trade both DetNet and non-DetNet
139 flows. End systems and applications need not instantiate special
140 interfaces for DetNet flows. Networks are not restricted to certain
141 topologies; connectivity is not restricted. Any application that
142 generates a data flow that can be usefully characterized as having a
143 maximum bandwidth should be able to take advantage of DetNet, as long
144 as the necessary resources can be reserved. Reservations can be made
145 by the application itself, via network management, by an applications
146 controller, or by other means.
148 Many applications of interest to Deterministic Networking require the
149 ability to synchronize the clocks in end systems to a sub-microsecond
150 accuracy. Some of the queue control techniques defined in
151 Section 4.5 also require time synchronization among relay and transit
152 nodes. The means used to achieve time synchronization are not
153 addressed in this document. DetNet should accommodate various
154 synchronization techniques and profiles that are defined elsewhere to
155 solve exchange time in different market segments.
157 Wired and wireless media differ greatly in a number of ways,
158 including connectivity possibilities and the reliability of packet
159 transmission. While some of the techniques described in this
160 document may be applicable to wireless media, the DetNet architecture
161 assumes the use of links with characteristics typical of wired, and
162 not wireless, media.
164 2. Terminology
166 2.1. Terms used in this document
168 The following special terms are used in this document in order to
169 avoid the assumption that a given element in the architecture does or
170 does not have Internet Protocol stack, functions as a router, bridge,
171 firewall, or otherwise plays a particular role at Layer-2 or higher.
173 App-flow
174 The native format of a DetNet flow.
176 destination
177 An end system capable of receiving a DetNet flow.
179 DetNet domain
180 The portion of a network that is DetNet aware. It includes
181 end systems and other DetNet nodes.
183 DetNet flow
184 A DetNet flow is a sequence of packets to which the DetNet
185 service is to be provided.
187 DetNet compound flow and DetNet member flow
188 A DetNet compound flow is a DetNet flow that has been
189 separated into multiple duplicate DetNet member flows, which
190 are eventually merged back into a single DetNet compound
191 flow, at the DetNet transport layer. "Compound" and "member"
192 are strictly relative to each other, not absolutes; a DetNet
193 compound flow comprising multiple DetNet member flows can, in
194 turn, be a member of a higher-order compound.
196 DetNet intermediate node
197 A DetNet relay node or transit node.
199 DetNet edge node
200 An instance of a DetNet relay node that includes either a
201 DetNet service layer proxy function for DetNet service
202 protection (e.g. the addition or removal of packet sequencing
203 information) for one or more end systems, or starts or
204 terminates congestion protection at the DetNet transport
205 layer, analogous to a Label Edge Router (LER).
207 DetNet-UNI
208 User-to-Network Interface with DetNet specific
209 functionalities. It is a packet-based reference point and
210 may provide multiple functions like encapsulation, status,
211 synchronization, etc.
213 end system
214 Commonly called a "host" or "node" in IETF documents, and an
215 "end station" is IEEE 802 documents. End systems of interest
216 to this document are either sources or destinations of DetNet
217 flows. And end system may or may not be DetNet transport
218 layer aware or DetNet service layer aware.
220 link
221 A connection between two DetNet nodes. It may be composed of
222 a physical link or a sub-network technology that can provide
223 appropriate traffic delivery for DetNet flows.
225 DetNet node
226 A DetNet aware end system, transit node, or relay node.
227 "DetNet" may be omitted in some text.
229 Detnet relay node
230 A DetNet node including a service layer function that
231 interconnects different DetNet transport layer paths to
232 provide service protection. A DetNet relay node can be a
233 bridge, a router, a firewall, or any other system that
234 participates in the DetNet service layer. It typically
235 incorporates DetNet transport layer functions as well, in
236 which case it is collocated with a transit node.
238 reservation
239 A trail of configuration between source to destination(s)
240 through transit nodes and subnets associated with a DetNet
241 flow, to provide congestion protection.
243 DetNet service layer
244 The layer at which service protection is provided, either
245 packet sequencing, replication, and elimination
246 (Section 3.2.4) or packet encoding (Section 3.2.5).
248 source
249 An end system capable of sourcing a DetNet flow.
251 DetNet transit node
252 A node operating at the DetNet transport layer, that utilizes
253 link layer and/or network layer switching across multiple
254 links and/or sub-networks to provide paths for DetNet service
255 layer functions. Optionally provides congestion protection
256 over those paths. An MPLS LSR is an example of a DetNet
257 transit node.
259 DetNet transport layer
260 The layer that optionally provides congestion protection for
261 DetNet flows over paths provided by the underlying network.
263 TSN
264 Time-Sensitive Networking, TSN is a Task Group of the IEEE
265 802.1 Working Group.
267 2.2. IEEE 802 TSN to DetNet dictionary
269 This section also serves as a dictionary for translating from the
270 terms used by the IEEE 802 Time-Sensitive Networking (TSN) Task Group
271 to those of the DetNet WG.
273 Listener
274 The IEEE 802 term for a destination of a DetNet flow.
276 relay system
277 The IEEE 802 term for a DetNet intermediate node.
279 Stream
280 The IEEE 802 term for a DetNet flow.
282 Talker
283 The IEEE 802 term for the source of a DetNet flow.
285 3. Providing the DetNet Quality of Service
287 3.1. Primary goals defining the DetNet QoS
289 The DetNet Quality of Service can be expressed in terms of:
291 o Minimum and maximum end-to-end latency from source to destination;
292 timely delivery and jitter avoidance derive from these constraints
294 o Probability of loss of a packet, under various assumptions as to
295 the operational states of the nodes and links. A derived property
296 is whether it is acceptable to deliver a duplicate packet, which
297 is an inherent risk in highly reliable and/or broadcast
298 transmissions
300 It is a distinction of DetNet that it is concerned solely with worst-
301 case values for the end-to-end latency. Average, mean, or typical
302 values are of no interest, because they do not affect the ability of
303 a real-time system to perform its tasks. In general, a trivial
304 priority-based queuing scheme will give better average latency to a
305 data flow than DetNet, but of course, the worst-case latency can be
306 essentially unbounded.
308 Three techniques are used by DetNet to provide these qualities of
309 service:
311 o Congestion protection (Section 3.2.1).
313 o Explicit routes (Section 3.2.2).
315 o Service protection (Section 3.2.4).
317 Congestion protection operates by reserving resources along the path
318 of a DetNet Flow, e.g. buffer space or link bandwidth. Congestion
319 protection greatly reduces, or even eliminates entirely, packet loss
320 due to output packet congestion within the network, but it can only
321 be supplied to a DetNet flow that is limited at the source to a
322 maximum packet size and transmission rate.
324 Congestion protection addresses both of the DetNet QoS requirements
325 (latency and packet loss). Given that DetNet nodes have a finite
326 amount of buffer space, congestion protection necessarily results in
327 a maximum end-to-end latency. It also addresses the largest
328 contribution to packet loss, which is buffer congestion.
330 After congestion, the most important contributions to packet loss are
331 typically from random media errors and equipment failures. Service
332 protection is the name for the mechanisms used by DetNet to address
333 these losses. The mechanisms employed are constrained by the
334 requirement to meet the users' latency requirements. Packet
335 replication and elimination (Section 3.2.4) and packet encoding
336 (Section 3.2.5) are described in this document to provide service
337 protection; others may be found. This mechanism distributes the
338 contents of DetNet flows over multiple paths in time and/or space, so
339 that the loss of some of the paths does need not cause the loss of
340 any packets. The paths are typically (but not necessarily) explicit
341 routes, so that they cannot suffer temporary interruptions caused by
342 the reconvergence of routing or bridging protocols.
344 These three techniques can be applied independently, giving eight
345 possible combinations, including none (no DetNet), although some
346 combinations are of wider utility than others. This separation keeps
347 the protocol stack coherent and maximizes interoperability with
348 existing and developing standards in this (IETF) and other Standards
349 Development Organizations. Some examples of typical expected
350 combinations:
352 o Explicit routes plus service protection are exactly the techniques
353 employed by [HSR-PRP]. Explicit routes are achieved by limiting
354 the physical topology of the network, and the sequentialization,
355 replication, and duplicate elimination are facilitated by packet
356 tags added at the front or the end of Ethernet frames.
358 o Congestion protection alone is is offered by IEEE 802.1 Audio
359 Video bridging [IEEE802.1BA-2011]. As long as the network suffers
360 no failures, zero congestion loss can be achieved through the use
361 of a reservation protocol (MSRP), shapers in every bridge, and a
362 bit of network calculus.
364 o Using all three together gives maximum protection.
366 There are, of course, simpler methods available (and employed, today)
367 to achieve levels of latency and packet loss that are satisfactory
368 for many applications. Prioritization and over-provisioning is one
369 such technique. However, these methods generally work best in the
370 absence of any significant amount of non-critical traffic in the
371 network (if, indeed, such traffic is supported at all), or work only
372 if the critical traffic constitutes only a small portion of the
373 network's theoretical capacity, or work only if all systems are
374 functioning properly, or in the absence of actions by end systems
375 that disrupt the network's operations.
377 There are any number of methods in use, defined, or in progress for
378 accomplishing each of the above techniques. It is expected that this
379 DetNet Architecture will assist various vendors, users, and/or
380 "vertical" Standards Development Organizations (dedicated to a single
381 industry) to make selections among the available means of
382 implementing DetNet networks.
384 3.2. Mechanisms to achieve DetNet Qos
386 3.2.1. Congestion protection
388 The primary means by which DetNet achieves its QoS assurances is to
389 reduce, or even completely eliminate, congestion at an output port as
390 a cause of packet loss. Given that a DetNet flow cannot be
391 throttled, this can be achieved only by the provision of sufficient
392 buffer storage at each hop through the network to ensure that no
393 packets are dropped due to a lack of buffer storage.
395 Ensuring adequate buffering requires, in turn, that the source, and
396 every intermediate node along the path to the destination (or nearly
397 every node -- see Section 4.3.3) be careful to regulate its output to
398 not exceed the data rate for any DetNet flow, except for brief
399 periods when making up for interfering traffic. Any packet sent
400 ahead of its time potentially adds to the number of buffers required
401 by the next hop, and may thus exceed the resources allocated for a
402 particular DetNet flow.
404 The low-level mechanisms described in Section 4.5 provide the
405 necessary regulation of transmissions by an end system or
406 intermediate node to provide congestion protection. The reservation
407 of the bandwidth and buffers for a DetNet flow requires the
408 provisioning described in Section 4.9. A DetNet node may have other
409 resources requiring allocation and/or scheduling, that might
410 otherwise be over-subscribed and trigger the rejection of a
411 reservation.
413 3.2.2. Explicit routes
415 In networks controlled by typical peer-to-peer protocols such as IEEE
416 802.1 ISIS bridged networks or IETF OSPF routed networks, a network
417 topology event in one part of the network can impact, at least
418 briefly, the delivery of data in parts of the network remote from the
419 failure or recovery event. Thus, even redundant paths through a
420 network, if controlled by the typical peer-to-peer protocols, do not
421 eliminate the chances of brief losses of contact.
423 Many real-time networks rely on physical rings or chains of two-port
424 devices, with a relatively simple ring control protocol. This
425 supports redundant paths for service protection with a minimum of
426 wiring. As an additional benefit, ring topologies can often utilize
427 different topology management protocols than those used for a mesh
428 network, with a consequent reduction in the response time to topology
429 changes. Of course, this comes at some cost in terms of increased
430 hop count, and thus latency, for the typical path.
432 In order to get the advantages of low hop count and still ensure
433 against even very brief losses of connectivity, DetNet employs
434 explicit routes, where the path taken by a given DetNet flow does not
435 change, at least immediately, and likely not at all, in response to
436 network topology events. Service protection (Section 3.2.4 or
437 Section 3.2.5) over explicit routes provides a high likelihood of
438 continuous connectivity. Explicit routes are commonly used in MPLS
439 TE LSPs.
441 3.2.3. Jitter Reduction
443 A core objective of DetNet is to enable the convergence of Non-IP
444 networks onto a common network infrastructure. This requires the
445 accurate emulation of currently deployed mission-specific networks,
446 which typically rely on point-to-point analog (e.g. 4-20mA
447 modulation) and serial-digital cables (or buses) for highly reliable,
448 synchronized and jitter-free communications. While the latency of
449 analog transmissions is basically the speed of light, legacy serial
450 links are usually slow (in the order of Kbps) compared to, say, GigE,
451 and some latency is usually acceptable. What is not acceptable is
452 the introduction of excessive jitter, which may, for instance, affect
453 the stability of control systems.
455 Applications that are designed to operate on serial links usually do
456 not provide services to recover the jitter, because jitter simply
457 does not exists there. Streams of information are expected to be
458 delivered in-order and the precise time of reception influences the
459 processes. In order to converge such existing applications, there is
460 a desire to emulate all properties of the serial cable, such as clock
461 transportation, perfect flow isolation and fixed latency. While
462 minimal jitter (in the form of specifying minimum, as well as
463 maximum, end-to-end latency) is supported by DetNet, there are
464 practical limitations on packet-based networks in this regard. In
465 general, users are encouraged to use, instead of, "do this when you
466 get the packet," a combination of:
468 o Sub-microsecond time synchronization among all source and
469 destination end systems, and
471 o Time-of-execution fields in the application packets.
473 Jitter reduction is provided by the mechanisms described in
474 Section 4.5 that also provide congestion protection.
476 3.2.4. Packet Replication and Elimination
478 After congestion loss has been eliminated, the most important causes
479 of packet loss are random media and/or memory faults, and equipment
480 failures. Both causes of packet loss can be greatly reduced by
481 spreading the data in a packet over multiple transmissions. One such
482 method for service protection is described in this section, which
483 sends the same packets over multiple paths.
485 Packet replication and elimination, also known as seamless redundancy
486 [HSR-PRP], or 1+1 hitless protection, is a function of the DetNet
487 service layer. It involves three capabilities:
489 o Providing sequencing information, once, at or near the source, to
490 the packets of a DetNet compound flow. This may be done by adding
491 a sequence number or time stamp as part of DetNet, or may be
492 inherent in the packet, e.g. in a transport protocol, or
493 associated to other physical properties such as the precise time
494 (and radio channel) of reception of the packet. Section 3.2.2.
496 o Replicating these packets into multiple DetNet member flows and,
497 typically, sending them along at least two different paths to the
498 destination(s), e.g. over the explicit routes of
500 o Eliminating duplicated packets. This may be done at any step
501 along the path to save network resources further down, in
502 particular if multiple Replication points exist. But the most
503 common case is to perform this operation at the very edge of the
504 DetNet network, preferably in or near the receiver.
506 This function is a "hitless" version of, e.g., the 1+1 linear
507 protection in [RFC6372]. That is, instead of switching from one flow
508 to the other when a failure of a flow is detected, DetNet combines
509 both flows, and performs a packet-by-packet selection of which to
510 discard, based on sequence number.
512 In the simplest case, this amounts to replicating each packet in a
513 source that has two interfaces, and conveying them through the
514 network, along separate paths, to the similarly dual-homed
515 destinations, that discard the extras. This ensures that one path
516 (with zero congestion loss) remains, even if some intermediate node
517 fails. The sequence numbers can also be used for loss detection and
518 for re-ordering.
520 Detnet relay nodes in the network can provide replication and
521 elimination facilities at various points in the network, so that
522 multiple failures can be accommodated.
524 This is shown in the following figure, where the two relay nodes each
525 replicate (R) the DetNet flow on input, sending the DetNet member
526 flows to both the other relay node and to the end system, and
527 eliminate duplicates (E) on the output interface to the right-hand
528 end system. Any one link in the network can fail, and the Detnet
529 compound flow can still get through. Furthermore, two links can
530 fail, as long as they are in different segments of the network.
532 Packet replication and elimination
534 > > > > > > > > > relay > > > > > > > >
535 > /------------+ R node E +------------\ >
536 > / v + ^ \ >
537 end R + v | ^ + E end
538 system + v | ^ + system
539 > \ v + ^ / >
540 > \------------+ R relay E +-----------/ >
541 > > > > > > > > > node > > > > > > > >
543 Figure 1
545 Packet replication and elimination does not react to and correct
546 failures; it is entirely passive. Thus, intermittent failures,
547 mistakenly created packet filters, or misrouted data is handled just
548 the same as the equipment failures that are detected handled by
549 typical routing and bridging protocols.
551 If packet replication and elimination is used over paths providing
552 congestion protection (Section 3.2.1), and member flows that take
553 different-length paths through the network are combined, a merge
554 point may require extra buffering to equalize the delays over the
555 different paths. This equalization ensures that the resultant
556 compound flow will not exceed its contracted bandwidth even after one
557 or the other of the paths is restored after a failure.
559 3.2.5. Packet encoding for service protection
561 There are methods for using multiple paths to provide service
562 protection that involve encoding the information in a packet
563 belonging to a DetNet flow into multiple transmission units,
564 combining information from multiple packets into any given
565 transmission unit. Such techniques, also known as "network coding",
566 can be used as a DetNet service protection technique.
568 3.3. Secondary goals for DetNet
570 Many applications require DetNet to provide additional services,
571 including coesistence with other QoS mechanisms Section 3.3.1 and
572 protection against misbehaving transmitters Section 3.3.2.
574 3.3.1. Coexistence with normal traffic
576 A DetNet network supports the dedication of a high proportion (e.g.
577 75%) of the network bandwidth to DetNet flows. But, no matter how
578 much is dedicated for DetNet flows, it is a goal of DetNet to coexist
579 with existing Class of Service schemes (e.g., DiffServ). It is also
580 important that non-DetNet traffic not disrupt the DetNet flow, of
581 course (see Section 3.3.2 and Section 5). For these reasons:
583 o Bandwidth (transmission opportunities) not utilized by a DetNet
584 flow are available to non-DetNet packets (though not to other
585 DetNet flows).
587 o DetNet flows can be shaped or scheduled, in order to ensure that
588 the highest-priority non-DetNet packet also is ensured a worst-
589 case latency (at any given hop).
591 o When transmission opportunities for DetNet flows are scheduled in
592 detail, then the algorithm constructing the schedule should leave
593 sufficient opportunities for non-DetNet packets to satisfy the
594 needs of the users of the network. Detailed scheduling can also
595 permit the time-shared use of buffer resources by different DetNet
596 flows.
598 Ideally, the net effect of the presence of DetNet flows in a network
599 on the non-DetNet packets is primarily a reduction in the available
600 bandwidth.
602 3.3.2. Fault Mitigation
604 One key to building robust real-time systems is to reduce the
605 infinite variety of possible failures to a number that can be
606 analyzed with reasonable confidence. DetNet aids in the process by
607 providing filters and policers to detect DetNet packets received on
608 the wrong interface, or at the wrong time, or in too great a volume,
609 and to then take actions such as discarding the offending packet,
610 shutting down the offending DetNet flow, or shutting down the
611 offending interface.
613 It is also essential that filters and service remarking be employed
614 at the network edge to prevent non-DetNet packets from being mistaken
615 for DetNet packets, and thus impinging on the resources allocated to
616 DetNet packets.
618 There exist techniques, at present and/or in various stages of
619 standardization, that can perform these fault mitigation tasks that
620 deliver a high probability that misbehaving systems will have zero
621 impact on well-behaved DetNet flows, except of course, for the
622 receiving interface(s) immediately downstream of the misbehaving
623 device. Examples of such techniques include traffic policing
624 functions (e.g. [RFC2475]) and separating flows into per-flow rate-
625 limited queues.
627 4. DetNet Architecture
629 4.1. DetNet stack model
631 4.1.1. Representative Protocol Stack Model
633 Figure 2 illustrates a conceptual DetNet data plane layering model.
634 One may compare it to that in [IEEE802.1CB], Annex C.
636 DetNet data plane protocol stack
638 | packets going | ^ packets coming ^
639 v down the stack v | up the stack |
640 +----------------------+ +-----------------------+
641 | Source | | Destination |
642 +----------------------+ +-----------------------+
643 | Service layer | | Service layer |
644 | Packet sequencing | | Duplicate elimination |
645 | Flow duplication | | Flow merging |
646 | Packet encoding | | Packet decoding |
647 +----------------------+ +-----------------------+
648 | Transport layer | | Transport layer |
649 | Congestion prot. | | Congestion prot. |
650 +----------------------+ +-----------------------+
651 | Lower layers | | Lower layers |
652 +----------------------+ +-----------------------+
653 v ^
654 \_________________________/
656 Figure 2
658 Not all layers are required for any given application, or even for
659 any given network. The layers are, from top to bottom:
661 Application
662 Shown as "source" and "destination" in the diagram.
664 OAM
665 Operations, Administration, and Maintenance leverages in-band
666 and out-of-and signaling that validates whether the service
667 is effectively obtained within QoS constraints. OAM is not
668 shown in Figure 2; it may reside in any number of the layers.
669 OAM can involve specific tagging added in the packets for
670 tracing implementation or network configuration errors;
671 traceability enables to find whether a packet is a replica,
672 which relay node performed the replication, and which segment
673 was intended for the replica.
675 Packet sequencing
676 As part of DetNet service protection, supplies the sequence
677 number for packet replication and elimination
678 (Section 3.2.4). Peers with Duplicate elimination. This
679 layer is not needed if a higher-layer transport protocol is
680 expected to perform any packet sequencing and duplicate
681 elimination required by the DetNet flow duplication.
683 Duplicate elimination
684 As part of the DetNet service layer, based on the sequenced
685 number supplied by its peer, packet sequencing, Duplicate
686 elimination discards any duplicate packets generated by
687 DetNet flow duplication. It can operate on member flows,
688 compound flows, or both. The duplication may also be
689 inferred from other information such as the precise time of
690 reception in a scheduled network. The duplicate elimination
691 layer may also perform resequencing of packets to restore
692 packet order in a flow that was disrupted by the loss of
693 packets on one or another of the multiple paths taken.
695 Flow duplication
696 As part of DetNet service protection, packets that belong to
697 a DetNet compound flow are replicated into two or more DetNet
698 member flows. This function is separate from packet
699 sequencing. Flow duplication can be an explicit duplication
700 and remarking of packets, or can be performed by, for
701 example, techniques similar to ordinary multicast
702 replication. Peers with DetNet flow merging.
704 Network flow merging
705 As part of DetNet service protection, merges DetNet member
706 flows together for packets coming up the stack belonging to a
707 specific DetNet compound flow. Peers with DetNet flow
708 duplication. DetNet flow merging, together with packet
709 sequencing, duplicate elimination, and DetNet flow
710 duplication, performs packet replication and elimination
711 (Section 3.2.4).
713 Packet encoding
714 As part of DetNet service protection, as an alternative to
715 packet sequencing and flow duplication, packet encoding
716 combines the information in multiple DetNet packets, perhaps
717 from different DetNet compound flows, and transmits that
718 information in packets on different DetNet member Flows.
719 Peers with Packet decoding.
721 Packet decoding
722 As part of DetNet service protection, as an alternative to
723 flow merging and duplicate elimination, packet decoding takes
724 packets from different DetNet member flows, and computes from
725 those packets the original DetNet packets from the compound
726 flows input to packet encoding. Peers with Packet encoding.
728 Congestion protection
729 The DetNet transport layer provides congestion protection.
730 See Section 4.5. The actual queuing and shaping mechanisms
731 are typically provided by underlying subnet layers, but since
732 these are can be closely associated with the means of
733 providing paths for DetNet flows (e.g. MPLS LSPs or {VLAN,
734 multicast destination MAC address} pairs), the path and the
735 congestion protection are conflated in this figure.
737 The packet sequencing and duplication elimination functions at the
738 source and destination ends of a DetNet compound flow may be
739 performed either in the end system or in a DetNet edge node. The
740 reader must not confuse a DetNet edge function with other kinds of
741 edge functions, e.g. an Label Edge Router, although the two functions
742 may be performed together. The DetNet edge function is concerned
743 with sequencing packets belonging to DetNet flows. The LER with
744 encapsulating/decapsulating packets for transport, and is considered
745 part of the network underlying the DetNet transport layer.
747 4.1.2. DetNet Data Plane Overview
749 A "Deterministic Network" will be composed of DetNet enabled nodes
750 i.e., End Systems, Edge Nodes, Relay Nodes and collectively deliver
751 DetNet services. DetNet enabled nodes are interconnected via Transit
752 Nodes (i.e., routers) which support DetNet, but are not DetNet
753 service aware. Transit nodes see DetNet nodes as end points. All
754 DetNet enabled nodes are connect to sub-networks, where a point-to-
755 point link is also considered as a simple sub-network. These sub-
756 networks will provide DetNet compatible service for support of DetNet
757 traffic. Examples of sub-networks include IEEE 802.1 TSN and OTN.
758 Of course, multi-layer DetNet systems may also be possible, where one
759 DetNet appears as a sub-network, and provides service to, a higher
760 layer DetNet system. A simple DetNet concept network is shown in
761 Figure 3.
763 TSN Edge Transit Relay DetNet
764 End System Node Node Node End System
766 +---------+ +.........+ +---------+
767 | Appl. |<---:Svc Proxy:-- End to End Service ---------->| Appl. |
768 +---------+ +---------+ +---------+ +---------+
769 | TSN | |TSN| |Svc|<-- DetNet flow ---: Service :-->| Service |
770 +---------+ +---+ +---+ +---------+ +---------+ +---------+
771 |Transport| |Trp| |Trp| |Transport| |Trp| |Trp| |Transport|
772 +-------.-+ +-.-+ +-.-+ +--.----.-+ +-.-+ +-.-+ +---.-----+
773 : Link : / ,-----. \ : Link : / ,-----. \
774 +........+ +-[ Sub ]-+ +........+ +-[ Sub ]-+
775 [Network] [Network]
776 `-----' `-----'
778 Figure 3: A Simple DetNet Enabled Network
780 Distinguishing the function of these two DetNet data plane layers,
781 the DetNet service layer and the DetNet transport layer, helps to
782 explore and evaluate various combinations of the data plane solutions
783 available. This separation of DetNet layers, while helpful, should
784 not be considered as formal requirement. For example, some
785 technologies may violate these strict layers and still be able to
786 deliver a DetNet service.
788 .
789 .
790 +-----------+
791 | Service | PW, RTP/(UDP), GRE
792 +-----------+
793 | Transport | (UDP)/IPv6, (UDP)/IPv4, MPLS LSPs, BIER
794 +-----------+
795 .
796 .
798 Figure 4: DetNet adaptation to data plane
800 In some networking scenarios, the end system initially provides a
801 DetNet flow encapsulation, which contains all information needed by
802 DetNet nodes (e.g., Real-time Transport Protocol (RTP) [RFC3550]
803 based DetNet flow transported over a native UDP/IP network or
804 PseudoWire). In other scenarios, the encapsulation formats might
805 differ significantly. As an example, a CPRI "application's" I/Q data
806 mapped directly to Ethernet frames may have to be transported over an
807 MPLS-based packet switched network (PSN).
809 There are many valid options to create a data plane solution for
810 DetNet traffic by selecting a technology approach for the DetNet
811 service layer and also selecting a technology approach for the DetNet
812 transport layer. There are a high number of valid combinations.
814 One of the most fundamental differences between different potential
815 data plane options is the basic addressing and headers used by DetNet
816 end systems. For example, is the basic service a Layer 2 (e.g.,
817 Ethernet) or Layer 3 (i.e., IP) service. This decision impacts how
818 DetNet end systems are addressed, and the basic forwarding logic for
819 the DetNet service layer.
821 4.1.3. Network reference model
823 The figure below shows another view of the DetNet service related
824 reference points and main components (Figure 5).
826 DetNet DetNet
827 end system end system
828 _ _
829 / \ +----DetNet-UNI (U) / \
830 /App\ | /App\
831 /-----\ | /-----\
832 | NIC | v ________ | NIC |
833 +--+--+ _____ / \ DetNet-UNI (U) --+ +--+--+
834 | / \__/ \ | |
835 | / +----+ +----+ \_____ | |
836 | / | | | | \_______ | |
837 +------U PE +----+ P +----+ \ _ v |
838 | | | | | | | ___/ \ |
839 | +--+-+ +----+ | +----+ | / \_ |
840 \ | | | | | / \ |
841 \ | +----+ +--+-+ +--+PE |-------- U------+
842 \ | | | | | | | | | \_ _/
843 \ +---+ P +----+ P +--+ +----+ | \____/
844 \___ | | | | /
845 \ +----+__ +----+ DetNet-1 DetNet-2
846 | \_____/ \___________/ |
847 | |
848 | | End-to-End-Service | | | |
849 <---------------------------------------------------------------->
850 | | DetNet-Service | | | |
851 | <--------------------------------------------------> |
852 | | | | | |
854 Figure 5: DetNet Service Reference Model (multi-domain)
856 DetNet-UNIs ("U" in Figure 5) are assumed in this document to be
857 packet-based reference points and provide connectivity over the
858 packet network. A DetNet-UNI may provide multiple functions, e.g.,
859 it may add networking technology specific encapsulation to the DetNet
860 flows if necessary; it may provide status of the availability of the
861 connection associated to a reservation; it may provide a
862 synchronization service for the end system; it may carry enough
863 signaling to place the reservation in a network without a controller,
864 or if the controller only deals with the network but not the end
865 points. Internal reference points of end systems (between the
866 application and the NIC) are more challenging from control
867 perspective and they may have extra requirements (e.g., in-order
868 delivery is expected in end system internal reference points, whereas
869 it is considered optional over the DetNet-UNI), therefore not covered
870 in this document.
872 4.2. DetNet systems
874 4.2.1. End system
876 The native data flow between the source/destination end systems is
877 referred to as application-flow (App-flow). The traffic
878 characteristics of an App-flow can be CBR (constant bit rate) or VBR
879 (variable bit rate) and can have L1 or L2 or L3 encapsulation (e.g.,
880 TDM (time-division multiplexing), Ethernet, IP). These
881 characteristics are considered as input for resource reservation and
882 might be simplified to ensure determinism during transport (e.g.,
883 making reservations for the peak rate of VBR traffic, etc.).
885 An end system may or may not be DetNet transport layer aware or
886 DetNet service layer aware. That is, an end system may or may not
887 contain DetNet specific functionality. End systems with DetNet
888 functionalities may have the same or different transport layer as the
889 connected DetNet domain. Grouping of end systems are shown in
890 Figure 6.
892 End system
893 |
894 |
895 | DetNet aware ?
896 / \
897 +------< >------+
898 NO | \ / | YES
899 | v |
900 DetNet unaware |
901 End system |
902 | Service/
903 | Transport
904 / \ aware ?
905 +--------< >-------------+
906 t-aware | \ / | s-aware
907 | v |
908 | | both |
909 | | |
910 DetNet t-aware | DetNet s-aware
911 End system | End system
912 v
913 DetNet st-aware
914 End system
916 Figure 6: Grouping of end systems
918 Note some known use cases for end systems:
920 o DetNet unaware: The classic case requiring network proxies.
922 o DetNet t-aware: An extant TSN system. It knows about some TSN
923 functions (e.g., reservation), but not about replication/
924 elimination.
926 o DetNet s-aware: An extant IEC 62439-3 system. It supplies
927 sequence numbers, but doesn't know about zero congestion loss.
929 o DetNet st-aware: A full functioning DetNet end station, it has
930 DetNet functionalities and usually the same forwarding paradigm as
931 the connected DetNet domain. It can be treated as an integral
932 part of the DetNet domain .
934 4.2.2. DetNet edge, relay, and transit nodes
936 As shown in Figure 3, DetNet edge nodes providing proxy service and
937 DetNet relay nodes providing the DetNet service layer are DetNet-
938 aware, and DetNet transit nodes need only be aware of the DetNet
939 transport layer.
941 In general, if a DetNet flow passes through one or more DetNet-
942 unaware network node between two DetNet nodes providing the DetNet
943 transport layer for that flow, there is a potential for disruption or
944 failure of the DetNet QoS. A network administrator needs to ensure
945 that the DetNet-unaware network nodes are configured to minimize the
946 chances of packet loss and delay, and provision enough exra buffer
947 space in the DetNet transit node following the DetNet-unaware network
948 nodes to absorb the induced latency variations.
950 4.3. DetNet flows
952 4.3.1. DetNet flow types
954 A DetNet flow can have different formats during while it is
955 transported between the peer end systems. Therefore, the following
956 possible types / formats of a DetNet flow are distinguished in this
957 document:
959 o App-flow: native format of a DetNet flow. It does not contain any
960 DetNet related attributes.
962 o DetNet-t-flow: specific format of a DetNet flow. Only requires
963 the congestion / latency features provided by the Detnet transport
964 layer.
966 o DetNet-s-flow: specific format of a DetNet flow. Only requires
967 the replication/elimination feature ensured by the DetNet service
968 layer.
970 o DetNet-st-flow: specific format of a DetNet flow. It requires
971 both DetNet service layer and DetNet transport layer functions
972 during forwarding.
974 4.3.2. Source guarantees
976 For the purposes of congestion protection, DetNet flows can be
977 synchronous or asynchronous. In synchronous DetNet flows, at least
978 the intermediate nodes (and possibly the end systems) are closely
979 time synchronized, typically to better than 1 microsecond. By
980 transmitting packets from different DetNet flows or classes of DetNet
981 flows at different times, using repeating schedules synchronized
982 among the intermediate nodes, resources such as buffers and link
983 bandwidth can be shared over the time domain among different DetNet
984 flows. There is a tradeoff among techniques for synchronous DetNet
985 flows between the burden of fine-grained scheduling and the benefit
986 of reducing the required resources, especially buffer space.
988 In contrast, asynchronous DetNet flows are not coordinated with a
989 fine-grained schedule, so relay and end systems must assume worst-
990 case interference among DetNet flows contending for buffer resources.
991 Asynchronous DetNet flows are characterized by:
993 o A maximum packet size;
995 o An observation interval; and
997 o A maximum number of transmissions during that observation
998 interval.
1000 These parameters, together with knowledge of the protocol stack used
1001 (and thus the size of the various headers added to a packet), limit
1002 the number of bit times per observation interval that the DetNet flow
1003 can occupy the physical medium.
1005 The source promises that these limits will not be exceeded. If the
1006 source transmits less data than this limit allows, the unused
1007 resources such as link bandwidth can be made available by the system
1008 to non-DetNet packets. However, making those resources available to
1009 DetNet packets in other DetNet flows would serve no purpose. Those
1010 other DetNet flows have their own dedicated resources, on the
1011 assumption that all DetNet flows can use all of their resources over
1012 a long period of time.
1014 There is no provision in DetNet for throttling DetNet flows (reducing
1015 the transmission rate via feedback); the assumption is that a DetNet
1016 flow, to be useful, must be delivered in its entirety. That is,
1017 while any useful application is written to expect a certain number of
1018 lost packets, the real-time applications of interest to DetNet demand
1019 that the loss of data due to the network is extraordinarily
1020 infrequent.
1022 Although DetNet strives to minimize the changes required of an
1023 application to allow it to shift from a special-purpose digital
1024 network to an Internet Protocol network, one fundamental shift in the
1025 behavior of network applications is impossible to avoid: the
1026 reservation of resources before the application starts. In the first
1027 place, a network cannot deliver finite latency and practically zero
1028 packet loss to an arbitrarily high offered load. Secondly, achieving
1029 practically zero packet loss for unthrottled (though bandwidth
1030 limited) DetNet flows means that bridges and routers have to dedicate
1031 buffer resources to specific DetNet flows or to classes of DetNet
1032 flows. The requirements of each reservation have to be translated
1033 into the parameters that control each system's queuing, shaping, and
1034 scheduling functions and delivered to the hosts, bridges, and
1035 routers.
1037 4.3.3. Incomplete Networks
1039 The presence in the network of transit nodes or subnets that are not
1040 fully capable of offering DetNet services complicates the ability of
1041 the intermediate nodes and/or controller to allocate resources, as
1042 extra buffering, and thus extra latency, must be allocated at points
1043 downstream from the non-DetNet intermediate node for a DetNet flow.
1045 4.4. Traffic Engineering for DetNet
1047 Traffic Engineering Architecture and Signaling (TEAS) [TEAS] defines
1048 traffic-engineering architectures for generic applicability across
1049 packet and non-packet networks. From TEAS perspective, Traffic
1050 Engineering (TE) refers to techniques that enable operators to
1051 control how specific traffic flows are treated within their networks.
1053 Because if its very nature of establishing explicit optimized paths,
1054 Deterministic Networking can be seen as a new, specialized branch of
1055 Traffic Engineering, and inherits its architecture with a separation
1056 into planes.
1058 The Deterministic Networking architecture is thus composed of three
1059 planes, a (User) Application Plane, a Controller Plane, and a Network
1060 Plane, which echoes that of Figure 1 of Software-Defined Networking
1061 (SDN): Layers and Architecture Terminology [RFC7426].:
1063 4.4.1. The Application Plane
1065 Per [RFC7426], the Application Plane includes both applications and
1066 services. In particular, the Application Plane incorporates the User
1067 Agent, a specialized application that interacts with the end user /
1068 operator and performs requests for Deterministic Networking services
1069 via an abstract Flow Management Entity, (FME) which may or may not be
1070 collocated with (one of) the end systems.
1072 At the Application Plane, a management interface enables the
1073 negotiation of flows between end systems. An abstraction of the flow
1074 called a Traffic Specification (TSpec) provides the representation.
1075 This abstraction is used to place a reservation over the (Northbound)
1076 Service Interface and within the Application plane. It is associated
1077 with an abstraction of location, such as IP addresses and DNS names,
1078 to identify the end systems and eventually specify intermediate
1079 nodes.
1081 4.4.2. The Controller Plane
1083 The Controller Plane corresponds to the aggregation of the Control
1084 and Management Planes in [RFC7426], though Common Control and
1085 Measurement Plane (CCAMP) [CCAMP] makes an additional distinction
1086 between management and measurement. When the logical separation of
1087 the Control, Measurement and other Management entities is not
1088 relevant, the term Controller Plane is used for simplicity to
1089 represent them all, and the term controller refers to any device
1090 operating in that plane, whether is it a Path Computation entity or a
1091 Network Management entity (NME). The Path Computation Element (PCE)
1092 [PCE] is a core element of a controller, in charge of computing
1093 Deterministic paths to be applied in the Network Plane.
1095 A (Northbound) Service Interface enables applications in the
1096 Application Plane to communicate with the entities in the Controller
1097 Plane.
1099 One or more PCE(s) collaborate to implement the requests from the FME
1100 as Per-Flow Per-Hop Behaviors installed in the intermediate nodes for
1101 each individual flow. The PCEs place each flow along a deterministic
1102 sequence of intermediate nodes so as to respect per-flow constraints
1103 such as security and latency, and optimize the overall result for
1104 metrics such as an abstract aggregated cost. The deterministic
1105 sequence can typically be more complex than a direct sequence and
1106 include redundancy path, with one or more packet replication and
1107 elimination points.
1109 4.4.3. The Network Plane
1111 The Network Plane represents the network devices and protocols as a
1112 whole, regardless of the Layer at which the network devices operate.
1113 It includes Forwarding Plane (data plane), Application, and
1114 Operational Plane (control plane) aspects.
1116 The network Plane comprises the Network Interface Cards (NIC) in the
1117 end systems, which are typically IP hosts, and intermediate nodes,
1118 which are typically IP routers and switches. Network-to-Network
1119 Interfaces such as used for Traffic Engineering path reservation in
1120 [RFC5921], as well as User-to-Network Interfaces (UNI) such as
1121 provided by the Local Management Interface (LMI) between network and
1122 end systems, are both part of the Network Plane, both in the control
1123 plane and the data plane.
1125 A Southbound (Network) Interface enables the entities in the
1126 Controller Plane to communicate with devices in the Network Plane.
1127 This interface leverages and extends TEAS to describe the physical
1128 topology and resources in the Network Plane.
1130 Flow Management Entity
1132 End End
1133 System System
1135 -+-+-+-+-+-+-+ Northbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
1137 PCE PCE PCE PCE
1139 -+-+-+-+-+-+-+ Southbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
1141 intermediate intermed. intermed. intermed.
1142 Node Node Node Node
1143 NIC NIC
1144 intermediate intermed. intermed. intermed.
1145 Node Node Node Node
1147 Figure 7
1149 The intermediate nodes (and eventually the end systems NIC) expose
1150 their capabilities and physical resources to the controller (the
1151 PCE), and update the PCE with their dynamic perception of the
1152 topology, across the Southbound Interface. In return, the PCE(s) set
1153 the per-flow paths up, providing a Flow Characterization that is more
1154 tightly coupled to the intermediate node Operation than a TSpec.
1156 At the Network plane, intermediate nodes may exchange information
1157 regarding the state of the paths, between adjacent systems and
1158 eventually with the end systems, and forward packets within
1159 constraints associated to each flow, or, when unable to do so,
1160 perform a last resort operation such as drop or declassify.
1162 This specification focuses on the Southbound interface and the
1163 operation of the Network Plane.
1165 4.5. Queuing, Shaping, Scheduling, and Preemption
1167 DetNet achieves congestion protection and bounded delivery latency by
1168 reserving bandwidth and buffer resources at every hop along the path
1169 of the DetNet flow. The reservation itself is not sufficient,
1170 however. Implementors and users of a number of proprietary and
1171 standard real-time networks have found that standards for specific
1172 data plane techniques are required to enable these assurances to be
1173 made in a multi-vendor network. The fundamental reason is that
1174 latency variation in one system results in the need for extra buffer
1175 space in the next-hop system(s), which in turn, increases the worst-
1176 case per-hop latency.
1178 Standard queuing and transmission selection algorithms allow a
1179 central controller to compute the latency contribution of each
1180 transit node to the end-to-end latency, to compute the amount of
1181 buffer space required in each transit node for each incremental
1182 DetNet flow, and most importantly, to translate from a flow
1183 specification to a set of values for the managed objects that control
1184 each relay or end system. The IEEE 802 has specified (and is
1185 specifying) a set of queuing, shaping, and scheduling algorithms that
1186 enable each transit node (bridge or router), and/or a central
1187 controller, to compute these values. These algorithms include:
1189 o A credit-based shaper [IEEE802.1Q-2014] Clause 34.
1191 o Time-gated queues governed by a rotating time schedule,
1192 synchronized among all transit nodes [IEEE802.1Qbv].
1194 o Synchronized double (or triple) buffers driven by synchronized
1195 time ticks. [IEEE802.1Qch].
1197 o Pre-emption of an Ethernet packet in transmission by a packet with
1198 a more stringent latency requirement, followed by the resumption
1199 of the preempted packet [IEEE802.1Qbu], [IEEE802.3br].
1201 While these techniques are currently embedded in Ethernet and
1202 bridging standards, we can note that they are all, except perhaps for
1203 packet preemption, equally applicable to other media than Ethernet,
1204 and to routers as well as bridges.
1206 4.6. Service instance
1208 A Service instance represents all the functions required on a node to
1209 allow the end-to-end service between the UNIs.
1211 The DetNet network reference model is shown in Figure 8 for a DetNet-
1212 Service scenario (i.e. between two DetNet-UNIs). In this figure, the
1213 end systems ("A" and "B") are connected directly to the edge nodes of
1214 the IP/MPLS network ("PE1" and "PE2"). End-systems participating
1215 DetNet communication may require connectivity before setting up an
1216 App-flow that requires the DetNet service. Such a connectivity
1217 related service instance and the one dedicated for DetNet service
1218 share the same access. Packets belonging to a DetNet flow are
1219 selected by a filter configured on the access ("F1" and "F2"). As a
1220 result, data flow specific access ("access-A + F1" and "access-B +
1221 F2") are terminated in the flow specific service instance ("SI-1" and
1222 "SI-2"). A tunnel is used to provide connectivity between the
1223 service instances.
1225 The tunnel is used to transport exclusively the packets of the DetNet
1226 flow between "SI-1" and "SI-2". The service instances are configured
1227 to implement DetNet functions and a flow specific routing or bridging
1228 function depending on what connectivity the participating end systems
1229 require (L3 or L2). The service instance and the tunnel may or may
1230 not be shared by multiple DetNet flows. Sharing the service instance
1231 by multiple DetNet flows requires properly populated forwarding
1232 tables of the service instance.
1234 access-A access-B
1235 <-----> <---------- tunnel ----------> <----->
1237 +---------+ ___ _ +---------+
1238 End system | +----+ | / \/ \_ | +----+ | End system
1239 "A" -------F1+ | | / \ | | +F2----- "B"
1240 | | +==========+ IP/MPLS +========+ | |
1241 | |SI-1| | \__ Net._/ | |SI-2| |
1242 | +----+ | \____/ | +----+ |
1243 |PE1 | | PE2|
1244 +---------+ +---------+
1246 Figure 8: DetNet network reference model
1248 The tunnel between the service instances may have some special
1249 characteristics. For example, in case of a "packet PW" based tunnel,
1250 there are differences in the usage of the packet PW for DetNet
1251 traffic compared to the network model described in [RFC6658]. In the
1252 DetNet scenario, the packet PW is used exclusively by the DetNet
1253 flow, whereas [RFC6658] states: "The packet PW appears as a single
1254 point-to-point link to the client layer. Network-layer adjacency
1255 formation and maintenance between the client equipments will follow
1256 the normal practice needed to support the required relationship in
1257 the client layer ... This packet pseudowire is used to transport all
1258 of the required layer 2 and layer 3 protocols between LSR1 and LSR2".
1260 4.7. Flow identification at technology borders
1262 4.7.1. Exporting flow identification
1264 An interesting feature of DetNet, and one that invites
1265 implementations that can be accused of "layering violations", is the
1266 need for lower layers to be aware of specific flows at higher layers,
1267 in order to provide specific queuing and shaping services for
1268 specific flows. For example:
1270 o A non-IP, strictly L2 source end system X may be sending multiple
1271 flows to the same L2 destination end system Y. Those flows may
1272 include DetNet flows with different QoS requirements, and may
1273 include non-DetNet flows.
1275 o A router may be sending any number of flows to another router.
1276 Again, those flows may include DetNet flows with different QoS
1277 requirements, and may include non-DetNet flows.
1279 o Two routers may be separated by bridges. For these bridges to
1280 perform any required per-flow queuing and shaping, they must be
1281 able to identify the individual flows.
1283 o A Label Edge Router (LERs) may have a Label Switched Path (LSP)
1284 set up for handling traffic destined for a particular IP address
1285 carrying only non-DetNet flows. If a DetNet flow to that same
1286 address is requested, a separate LSP may be needed, in order that
1287 all of the Label Switch Routers (LSRs) along the path to the
1288 destination give that flow special queuing and shaping.
1290 The need for a lower-level DetNet node to be aware of individual
1291 higher-layer flows is not unique to DetNet. But, given the endless
1292 complexity of layering and relayering over tunnels that is available
1293 to network designers, DetNet needs to provide a model for flow
1294 identification that is at least somewhat better than packet
1295 inspection. That is not to say that packet inspection to layer 4 or
1296 5 addresses will not be used, or the capability standardized; but,
1297 there are alternatives.
1299 A DetNet relay node can connect DetNet flows on different paths using
1300 different flow identification methods. For example:
1302 o A single unicast DetNet flow passing from router A through a
1303 bridged network to router B may be assigned a {VLAN, multicast
1304 destination MAC address} pair that is unique within that bridged
1305 network. The bridges can then identify the flow without accessing
1306 higher-layer headers. Of course, the receiving router must
1307 recognize and accept that multicast MAC address.
1309 o A DetNet flow passing from LSR A to LSR B may be assigned a
1310 different label than that used for other flows to the same IP
1311 destination.
1313 In any of the above cases, it is possible that an existing DetNet
1314 flow can be used as a carrier for multiple DetNet sub-flows. (Not to
1315 be confused with DetNet compound vs. member flows.) Of course, this
1316 requires that the aggregate DetNet flow be provisioned properly to
1317 carry the sub-flows.
1319 Thus, rather than packet inspection, there is the option to export
1320 higher-layer information to the lower layer. The requirement to
1321 support one or the other method for flow identification (or both) is
1322 the essential complexity that DetNet brings to existing control plane
1323 models.
1325 4.7.2. Flow attribute mapping between layers
1327 Transport of DetNet flows over multiple technology domains may
1328 require that lower layers are aware of specific flows of higher
1329 layers. Such an "exporting of flow identification" is needed each
1330 time when the forwarding paradigm is changed on the transport path
1331 (e.g., two LSRs are interconnected by a L2 bridged domain, etc.).
1332 The three main forwarding methods considered for deterministic
1333 networking are:
1335 o IP routing
1337 o MPLS label switching
1339 o Ethernet bridging
1341 add/remove add/remove
1342 Eth Flow-ID IP Flow-ID
1343 | |
1344 v v
1345 +-----------------------------------------------------------+
1346 | | | | |
1347 | Eth | MPLS | IP | Application data |
1348 | | | | |
1349 +-----------------------------------------------------------+
1350 ^
1351 |
1352 add/remove
1353 MPLS Flow-ID
1355 Figure 9: Packet with multiple Flow-IDs
1357 The additional (domain specific) Flow-ID can be
1359 o created by a domain specific function or
1361 o derived from the Flow-ID added to the App-flow,
1363 so that it must be unique inside the given domain. Note that the
1364 Flow-ID added to the App-flow is still present in the packet, but
1365 transport nodes may lack the function to recognize it; that's why the
1366 additional Flow-ID is added (pushed).
1368 4.7.3. Flow-ID mapping examples
1370 IP nodes and MPLS nodes are assumed to be configured to push such an
1371 additional (domain specific) Flow-ID when sending traffic to an
1372 Ethernet switch (as shown in the examples below).
1374 Figure 10 shows a scenario where an IP end system ("IP-A") is
1375 connected via two Ethernet switches ("ETH-n") to an IP router ("IP-
1376 1").
1378 IP domain
1379 <-----------------------------------------------
1381 +======+ +======+
1382 |L3-ID | |L3-ID |
1383 +======+ /\ +-----+ +======+
1384 / \ Forward as | |
1385 /IP-A\ per ETH-ID |IP-1 | Recognize
1386 Push ------> +-+----+ | +---+-+ <----- ETH-ID
1387 ETH-ID | +----+-----+ |
1388 | v v |
1389 | +-----+ +-----+ |
1390 +------+ | | +---------+
1391 +......+ |ETH-1+----+ETH-2| +======+
1392 .L3-ID . +-----+ +-----+ |L3-ID |
1393 +======+ +......+ +======+
1394 |ETH-ID| .L3-ID . |ETH-ID|
1395 +======+ +======+ +------+
1396 |ETH-ID|
1397 +======+
1399 Ethernet domain
1400 <---------------->
1402 Figure 10: IP nodes interconnected by an Ethernet domain
1404 End system "IP-A" uses the original App-flow specific ID ("L3-ID"),
1405 but as it is connected to an Ethernet domain it has to push an
1406 Ethernet-domain specific flow-ID ("VID + multicast MAC address",
1407 referred as "ETH-ID") before sending the packet to "ETH-1" node.
1408 Ethernet switch "ETH-1" can recognize the data flow based on the
1409 "ETH-ID" and it does forwarding toward "ETH-2". "ETH-2" switches the
1410 packet toward the IP router. "IP-1" must be configured to receive
1411 the Ethernet Flow-ID specific multicast stream, but (as it is an L3
1412 node) it decodes the data flow ID based on the "L3-ID" fields of the
1413 received packet.
1415 Figure 11 shows a scenario where MPLS domain nodes ("PE-n" and "P-m")
1416 are connected via two Ethernet switches ("ETH-n").
1418 MPLS domain
1419 <----------------------------------------------->
1421 +=======+ +=======+
1422 |MPLS-ID| |MPLS-ID|
1423 +=======+ +-----+ +-----+ +=======+ +-----+
1424 | | Forward as | | | |
1425 |PE-1 | per ETH-ID | P-2 +-----------+ PE-2|
1426 Push -----> +-+---+ | +---+-+ +-----+
1427 ETH-ID | +-----+----+ | \ Recognize
1428 | v v | +-- ETH-ID
1429 | +-----+ +-----+ |
1430 +---+ | | +----+
1431 +.......+ |ETH-1+----+ETH-2| +=======+
1432 .MPLS-ID. +-----+ +-----+ |MPLS-ID|
1433 +=======+ +=======+
1434 |ETH-ID | +.......+ |ETH-ID |
1435 +=======+ .MPLS-ID. +-------+
1436 +=======+
1437 |ETH-ID |
1438 +=======+
1439 Ethernet domain
1440 <---------------->
1442 Figure 11: MPLS nodes interconnected by an Ethernet domain
1444 "PE-1" uses the MPLS specific ID ("MPLS-ID"), but as it is connected
1445 to an Ethernet domain it has to push an Ethernet-domain specific
1446 flow-ID ("VID + multicast MAC address", referred as "ETH-ID") before
1447 sending the packet to "ETH-1". Ethernet switch "ETH-1" can recognize
1448 the data flow based on the "ETH-ID" and it does forwarding toward
1449 "ETH-2". "ETH-2" switches the packet toward the MPLS node ("P-2").
1450 "P-2" must be configured to receive the Ethernet Flow-ID specific
1451 multicast stream, but (as it is an MPLS node) it decodes the data
1452 flow ID based on the "MPLS-ID" fields of the received packet.
1454 One can appreciate from the above example that, when the means used
1455 for DetNet flow identifcation is altered or exported, the means for
1456 encoding the sequence number information must similarly be altered or
1457 exported.
1459 4.8. Advertising resources, capabilities and adjacencies
1461 There are three classes of information that a central controller or
1462 decentralized control plane needs to know that can only be obtained
1463 from the end systems and/or transit nodes in the network. When using
1464 a peer-to-peer control plane, some of this information may be
1465 required by a system's neighbors in the network.
1467 o Details of the system's capabilities that are required in order to
1468 accurately allocate that system's resources, as well as other
1469 systems' resources. This includes, for example, which specific
1470 queuing and shaping algorithms are implemented (Section 4.5), the
1471 number of buffers dedicated for DetNet allocation, and the worst-
1472 case forwarding delay.
1474 o The dynamic state of an end or transit node's DetNet resources.
1476 o The identity of the system's neighbors, and the characteristics of
1477 the link(s) between the systems, including the length (in
1478 nanoseconds) of the link(s).
1480 4.9. Provisioning model
1482 4.9.1. Centralized Path Computation and Installation
1484 A centralized routing model, such as provided with a PCE (RFC 4655
1485 [RFC4655]), enables global and per-flow optimizations. (See
1486 Section 4.4.) The model is attractive but a number of issues are
1487 left to be solved. In particular:
1489 o Whether and how the path computation can be installed by 1) an end
1490 device or 2) a Network Management entity,
1492 o And how the path is set up, either by installing state at each hop
1493 with a direct interaction between the forwarding device and the
1494 PCE, or along a path by injecting a source-routed request at one
1495 end of the path.
1497 4.9.2. Distributed Path Setup
1499 Significant work on distributed path setup can be leveraged from MPLS
1500 Traffic Engineering, in both its GMPLS and non-GMPLS forms. The
1501 protocols within scope are Resource ReSerVation Protocol [RFC3209]
1502 [RFC3473](RSVP-TE), OSPF-TE [RFC4203] [RFC5392] and ISIS-TE [RFC5307]
1503 [RFC5316]. These should be viewed as starting points as there are
1504 feature specific extensions defined that may be applicable to DetNet.
1506 In a Layer-2 only environment, or as part of a layered approach to a
1507 mixed environment, IEEE 802.1 also has work, either completed or in
1508 progress. [IEEE802.1Q-2014] Clause 35 describes SRP, a peer-to-peer
1509 protocol for Layer-2 roughly analogous to RSVP [RFC2205].
1510 [IEEE802.1Qca] defines how ISIS can provide multiple disjoint paths
1511 or distribution trees. Also in progress is [IEEE802.1Qcc], which
1512 expands the capabilities of SRP.
1514 The integration/interaction of the DetNet control layer with an
1515 underlying IEEE 802.1 sub-network control layer will need to be
1516 defined.
1518 4.10. Scaling to larger networks
1520 Reservations for individual DetNet flows require considerable state
1521 information in each transit node, especially when adequate fault
1522 mitigation (Section 3.3.2) is required. The DetNet data plane, in
1523 order to support larger numbers of DetNet flows, must support the
1524 aggregation of DetNet flows into tunnels, which themselves can be
1525 viewed by the transit nodes' data planes largely as individual DetNet
1526 flows. Without such aggregation, the per-relay system may limit the
1527 scale of DetNet networks.
1529 4.11. Connected islands vs. networks
1531 Given that users have deployed examples of the IEEE 802.1 TSN TG
1532 standards, which provide capabilities similar to DetNet, it is
1533 obvious to ask whether the IETF DetNet effort can be limited to
1534 providing Layer-2 connections (VPNs) between islands of bridged TSN
1535 networks. While this capability is certainly useful to some
1536 applications, and must not be precluded by DetNet, tunneling alone is
1537 not a sufficient goal for the DetNet WG. As shown in the
1538 Deterministic Networking Use Cases draft [I-D.ietf-detnet-use-cases],
1539 there are already deployments of Layer-2 TSN networks that are
1540 encountering the well-known problems of over-large broadcast domains.
1541 Routed solutions, and combinations routed/bridged solutions, are both
1542 required.
1544 4.12. Compatibility with Layer-2
1546 Standards providing similar capabilities for bridged networks (only)
1547 have been and are being generated in the IEEE 802 LAN/MAN Standards
1548 Committee. The present architecture describes an abstract model that
1549 can be applicable both at Layer-2 and Layer-3, and over links not
1550 defined by IEEE 802. It is the intention of the authors (and
1551 hopefully, as this draft progresses, of the DetNet Working Group)
1552 that IETF and IEEE 802 will coordinate their work, via the
1553 participation of common individuals, liaisons, and other means, to
1554 maximize the compatibility of their outputs.
1556 DetNet enabled end systems and intermediate nodes can be
1557 interconnected by sub-networks, i.e., Layer-2 technologies. These
1558 sub-networks will provide DetNet compatible service for support of
1559 DetNet traffic. Examples of sub-networks include 802.1TSN and a
1560 point-to-point OTN link. Of course, multi-layer DetNet systems may
1561 be possible too, where one DetNet appears as a sub-network, and
1562 provides service to, a higher layer DetNet system.
1564 5. Security Considerations
1566 Security in the context of Deterministic Networking has an added
1567 dimension; the time of delivery of a packet can be just as important
1568 as the contents of the packet, itself. A man-in-the-middle attack,
1569 for example, can impose, and then systematically adjust, additional
1570 delays into a link, and thus disrupt or subvert a real-time
1571 application without having to crack any encryption methods employed.
1572 See [RFC7384] for an exploration of this issue in a related context.
1574 Furthermore, in a control system where millions of dollars of
1575 equipment, or even human lives, can be lost if the DetNet QoS is not
1576 delivered, one must consider not only simple equipment failures,
1577 where the box or wire instantly becomes perfectly silent, but bizarre
1578 errors such as can be caused by software failures. Because there is
1579 essential no limit to the kinds of failures that can occur,
1580 protecting against realistic equipment failures is indistinguishable,
1581 in most cases, from protecting against malicious behavior, whether
1582 accidental or intentional. See also Section 3.3.2.
1584 Security must cover:
1586 o the protection of the signaling protocol
1588 o the authentication and authorization of the controlling systems
1590 o the identification and shaping of the DetNet flows
1592 6. Privacy Considerations
1594 DetNet is provides a Quality of Service (QoS), and as such, does not
1595 directly raise any new privacy considerations.
1597 However, the requirement for every (or almost every) node along the
1598 path of a DetNet flow to identify DetNet flows may present an
1599 additional attack surface for privacy, should the DetNet paradigm be
1600 found useful in broader environments.
1602 7. IANA Considerations
1604 This document does not require an action from IANA.
1606 8. Acknowledgements
1608 The authors wish to thank Jouni Korhonen, Erik Nordmark, George
1609 Swallow, Rudy Klecka, Anca Zamfir, David Black, Thomas Watteyne,
1610 Shitanshu Shah, Craig Gunther, Rodney Cummings, Balazs Varga,
1611 Wilfried Steiner, Marcel Kiessling, Karl Weber, Janos Farkas, Ethan
1612 Grossman, Pat Thaler, Lou Berger, and especially Michael Johas
1613 Teener, for their various contribution with this work.
1615 9. Access to IEEE 802.1 documents
1617 To access password protected IEEE 802.1 drafts, see the IETF IEEE
1618 802.1 information page at https://www.ietf.org/proceedings/52/slides/
1619 bridge-0/tsld003.htm.
1621 10. Informative References
1623 [AVnu] http://www.avnu.org/, "The AVnu Alliance tests and
1624 certifies devices for interoperability, providing a simple
1625 and reliable networking solution for AV network
1626 implementation based on the Audio Video Bridging (AVB)
1627 standards.".
1629 [CCAMP] IETF, "Common Control and Measurement Plane",
1630 .
1632 [HSR-PRP] IEC, "High availability seamless redundancy (HSR) is a
1633 further development of the PRP approach, although HSR
1634 functions primarily as a protocol for creating media
1635 redundancy while PRP, as described in the previous
1636 section, creates network redundancy. PRP and HSR are both
1637 described in the IEC 62439 3 standard.",
1638 .
1641 [I-D.dt-detnet-dp-alt]
1642 Korhonen, J., Farkas, J., Mirsky, G., Thubert, P.,
1643 Zhuangyan, Z., and L. Berger, "DetNet Data Plane Protocol
1644 and Solution Alternatives", draft-dt-detnet-dp-alt-04
1645 (work in progress), September 2016.
1647 [I-D.ietf-6tisch-architecture]
1648 Thubert, P., "An Architecture for IPv6 over the TSCH mode
1649 of IEEE 802.15.4", draft-ietf-6tisch-architecture-13 (work
1650 in progress), November 2017.
1652 [I-D.ietf-6tisch-tsch]
1653 Watteyne, T., Palattella, M., and L. Grieco, "Using
1654 IEEE802.15.4e TSCH in an IoT context: Overview, Problem
1655 Statement and Goals", draft-ietf-6tisch-tsch-06 (work in
1656 progress), March 2015.
1658 [I-D.ietf-detnet-problem-statement]
1659 Finn, N. and P. Thubert, "Deterministic Networking Problem
1660 Statement", draft-ietf-detnet-problem-statement-03 (work
1661 in progress), March 2018.
1663 [I-D.ietf-detnet-use-cases]
1664 Grossman, E., "Deterministic Networking Use Cases", draft-
1665 ietf-detnet-use-cases-15 (work in progress), April 2018.
1667 [I-D.ietf-roll-rpl-industrial-applicability]
1668 Phinney, T., Thubert, P., and R. Assimiti, "RPL
1669 applicability in industrial networks", draft-ietf-roll-
1670 rpl-industrial-applicability-02 (work in progress),
1671 October 2013.
1673 [I-D.svshah-tsvwg-deterministic-forwarding]
1674 Shah, S. and P. Thubert, "Deterministic Forwarding PHB",
1675 draft-svshah-tsvwg-deterministic-forwarding-04 (work in
1676 progress), August 2015.
1678 [I-D.varga-detnet-service-model]
1679 Varga, B. and J. Farkas, "DetNet Service Model", draft-
1680 varga-detnet-service-model-02 (work in progress), May
1681 2017.
1683 [IEEE802.1AS-2011]
1684 IEEE, "IEEE Std 802.1AS Timing and Synchronization for
1685 Time-Sensitive Applications in Bridged Local Area
1686 Networks", 2011,
1687 .
1689 [IEEE802.1BA-2011]
1690 IEEE, "IEEE Std 802.1BA Audio Video Bridging (AVB)
1691 Systems", 2011,
1692 .
1694 [IEEE802.1CB]
1695 IEEE, "IEEE Std 802.1CB Frame Replication and Elimination
1696 for Reliability", 2017,
1697 .
1699 [IEEE802.1Q-2014]
1700 IEEE, "IEEE Std 802.1Q Bridges and Bridged Networks",
1701 2014, .
1703 [IEEE802.1Qbu]
1704 IEEE, "IEEEE Std 802.1Qbu Bridges and Bridged Networks -
1705 Amendment 26: Frame Preemption", 2016,
1706 .
1708 [IEEE802.1Qbv]
1709 IEEE, "IEEEE Std 802.1Qbu Bridges and Bridged Networks -
1710 Amendment 25: Enhancements for Scheduled Traffic", 2015,
1711 .
1713 [IEEE802.1Qca]
1714 IEEE, "IEEE Std 802.1Qca Bridges and Bridged Networks -
1715 Amendment 24: Path Control and Reservation", June 2015,
1716 .
1718 [IEEE802.1Qcc]
1719 IEEE, "Stream Reservation Protocol (SRP) Enhancements and
1720 Performance Improvements (IEEE Draft P802.1Qcc)", 2017,
1721 .
1723 [IEEE802.1Qch]
1724 IEEE, "Cyclic Queuing and Forwarding (IEEE Draft
1725 P802.1Qch)", 2017,
1726 .
1728 [IEEE802.1TSNTG]
1729 IEEE Standards Association, "IEEE 802.1 Time-Sensitive
1730 Networks Task Group", 2013,
1731 .
1733 [IEEE802.3-2015]
1734 IEEE, "IEEE Std 802.3 Standard for Ethernet", 2015,
1735 .
1737 [IEEE802.3br]
1738 IEEE, "IEEE Std 802.3br Standard for Ethernet Amendment 5:
1739 Specification and Management Parameters for Interspersing
1740 Express Traffic", 2016,
1741 .
1743 [ISA95] ANSI/ISA, "Enterprise-Control System Integration Part 1:
1744 Models and Terminology", 2000,
1745 .
1747 [ODVA] http://www.odva.org/, "The organization that supports
1748 network technologies built on the Common Industrial
1749 Protocol (CIP) including EtherNet/IP.".
1751 [PCE] IETF, "Path Computation Element",
1752 .
1754 [Profinet]
1755 http://us.profinet.com/technology/profinet/, "PROFINET is
1756 a standard for industrial networking in automation.",
1757 .
1759 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.
1760 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
1761 Functional Specification", RFC 2205, DOI 10.17487/RFC2205,
1762 September 1997, .
1764 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.,
1765 and W. Weiss, "An Architecture for Differentiated
1766 Services", RFC 2475, DOI 10.17487/RFC2475, December 1998,
1767 .
1769 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
1770 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
1771 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001,
1772 .
1774 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label
1775 Switching (GMPLS) Signaling Resource ReserVation Protocol-
1776 Traffic Engineering (RSVP-TE) Extensions", RFC 3473,
1777 DOI 10.17487/RFC3473, January 2003,
1778 .
1780 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
1781 Jacobson, "RTP: A Transport Protocol for Real-Time
1782 Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550,
1783 July 2003, .
1785 [RFC4203] Kompella, K., Ed. and Y. Rekhter, Ed., "OSPF Extensions in
1786 Support of Generalized Multi-Protocol Label Switching
1787 (GMPLS)", RFC 4203, DOI 10.17487/RFC4203, October 2005,
1788 .
1790 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation
1791 Element (PCE)-Based Architecture", RFC 4655,
1792 DOI 10.17487/RFC4655, August 2006,
1793 .
1795 [RFC5307] Kompella, K., Ed. and Y. Rekhter, Ed., "IS-IS Extensions
1796 in Support of Generalized Multi-Protocol Label Switching
1797 (GMPLS)", RFC 5307, DOI 10.17487/RFC5307, October 2008,
1798 .
1800 [RFC5316] Chen, M., Zhang, R., and X. Duan, "ISIS Extensions in
1801 Support of Inter-Autonomous System (AS) MPLS and GMPLS
1802 Traffic Engineering", RFC 5316, DOI 10.17487/RFC5316,
1803 December 2008, .
1805 [RFC5392] Chen, M., Zhang, R., and X. Duan, "OSPF Extensions in
1806 Support of Inter-Autonomous System (AS) MPLS and GMPLS
1807 Traffic Engineering", RFC 5392, DOI 10.17487/RFC5392,
1808 January 2009, .
1810 [RFC5673] Pister, K., Ed., Thubert, P., Ed., Dwars, S., and T.
1811 Phinney, "Industrial Routing Requirements in Low-Power and
1812 Lossy Networks", RFC 5673, DOI 10.17487/RFC5673, October
1813 2009, .
1815 [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau,
1816 L., and L. Berger, "A Framework for MPLS in Transport
1817 Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010,
1818 .
1820 [RFC6372] Sprecher, N., Ed. and A. Farrel, Ed., "MPLS Transport
1821 Profile (MPLS-TP) Survivability Framework", RFC 6372,
1822 DOI 10.17487/RFC6372, September 2011,
1823 .
1825 [RFC6658] Bryant, S., Ed., Martini, L., Swallow, G., and A. Malis,
1826 "Packet Pseudowire Encapsulation over an MPLS PSN",
1827 RFC 6658, DOI 10.17487/RFC6658, July 2012,
1828 .
1830 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in
1831 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384,
1832 October 2014, .
1834 [RFC7426] Haleplidis, E., Ed., Pentikousis, K., Ed., Denazis, S.,
1835 Hadi Salim, J., Meyer, D., and O. Koufopavlou, "Software-
1836 Defined Networking (SDN): Layers and Architecture
1837 Terminology", RFC 7426, DOI 10.17487/RFC7426, January
1838 2015, .
1840 [TEAS] IETF, "Traffic Engineering Architecture and Signaling",
1841 .
1843 Authors' Addresses
1845 Norman Finn
1846 Huawei
1847 3755 Avocado Blvd.
1848 PMB 436
1849 La Mesa, California 91941
1850 US
1852 Phone: +1 925 980 6430
1853 Email: norman.finn@mail01.huawei.com
1855 Pascal Thubert
1856 Cisco Systems
1857 Village d'Entreprises Green Side
1858 400, Avenue de Roumanille
1859 Batiment T3
1860 Biot - Sophia Antipolis 06410
1861 FRANCE
1863 Phone: +33 4 97 23 26 34
1864 Email: pthubert@cisco.com
1866 Balazs Varga
1867 Ericsson
1868 Konyves Kalman krt. 11/B
1869 Budapest 1097
1870 Hungary
1872 Email: balazs.a.varga@ericsson.com
1873 Janos Farkas
1874 Ericsson
1875 Konyves Kalman krt. 11/B
1876 Budapest 1097
1877 Hungary
1879 Email: janos.farkas@ericsson.com