idnits 2.17.1
draft-ietf-detnet-architecture-00.txt:
Checking boilerplate required by RFC 5378 and the IETF Trust (see
https://trustee.ietf.org/license-info):
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt:
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/checklist :
----------------------------------------------------------------------------
No issues found here.
Miscellaneous warnings:
----------------------------------------------------------------------------
== The copyright year in the IETF Trust and authors Copyright Line does not
match the current year
== Line 636 has weird spacing: '...mediate inter...'
== Line 639 has weird spacing: '...mediate inter...'
== Line 836 has weird spacing: '...e stack v ...'
-- The document date (September 26, 2016) is 2761 days in the past. Is
this intentional?
Checking references for intended status: Proposed Standard
----------------------------------------------------------------------------
(See RFCs 3967 and 4897 for information about using normative references
to lower-maturity documents in RFCs)
== Unused Reference: 'AVnu' is defined on line 1233, but no explicit
reference was found in the text
== Unused Reference: 'HART' is defined on line 1242, but no explicit
reference was found in the text
== Unused Reference: 'I-D.ietf-6tisch-architecture' is defined on line
1261, but no explicit reference was found in the text
== Unused Reference: 'I-D.ietf-6tisch-tsch' is defined on line 1266, but no
explicit reference was found in the text
== Unused Reference: 'I-D.ietf-roll-rpl-industrial-applicability' is
defined on line 1284, but no explicit reference was found in the text
== Unused Reference: 'I-D.svshah-tsvwg-deterministic-forwarding' is defined
on line 1290, but no explicit reference was found in the text
== Unused Reference: 'IEEE802.11-2012' is defined on line 1295, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802.1AS-2011' is defined on line 1301, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802.1TSNTG' is defined on line 1345, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802154' is defined on line 1359, but no explicit
reference was found in the text
== Unused Reference: 'IEEE802154e' is defined on line 1365, but no explicit
reference was found in the text
== Unused Reference: 'ISA95' is defined on line 1377, but no explicit
reference was found in the text
== Unused Reference: 'ODVA' is defined on line 1381, but no explicit
reference was found in the text
== Unused Reference: 'Profinet' is defined on line 1388, but no explicit
reference was found in the text
== Unused Reference: 'RFC5673' is defined on line 1439, but no explicit
reference was found in the text
== Unused Reference: 'WirelessHART' is defined on line 1467, but no
explicit reference was found in the text
== Outdated reference: A later version (-30) exists of
draft-ietf-6tisch-architecture-10
== Outdated reference: A later version (-09) exists of
draft-ietf-detnet-problem-statement-00
== Outdated reference: A later version (-20) exists of
draft-ietf-detnet-use-cases-10
-- Obsolete informational reference (is this intentional?): RFC 5316
(Obsoleted by RFC 9346)
Summary: 0 errors (**), 0 flaws (~~), 23 warnings (==), 2 comments (--).
Run idnits with the --verbose option for more detailed information about
the items above.
--------------------------------------------------------------------------------
2 DetNet N. Finn
3 Internet-Draft Self-employed
4 Intended status: Standards Track P. Thubert
5 Expires: March 30, 2017 Cisco
6 September 26, 2016
8 Deterministic Networking Architecture
9 draft-ietf-detnet-architecture-00
11 Abstract
13 Deterministic Networking (DetNet) provides a capability to carry
14 specified unicast or multicast data flows for real-time applications
15 with extremely low data loss rates and bounded latency. Techniques
16 used include: 1) reserving data plane resources for individual (or
17 aggregated) DetNet flows in some or all of the intermediate nodes
18 (e.g. bridges or routers) along the path of the flow; 2) providing
19 explicit routes for DetNet flows that do not rapidly change with the
20 network topology; and 3) distributing data from DetNet flow packets
21 over time and/or space to ensure delivery of each packet's data' in
22 spite of the loss of a path. The capabilities can be managed by
23 configuration, or by manual or automatic network management.
25 Status of This Memo
27 This Internet-Draft is submitted in full conformance with the
28 provisions of BCP 78 and BCP 79.
30 Internet-Drafts are working documents of the Internet Engineering
31 Task Force (IETF). Note that other groups may also distribute
32 working documents as Internet-Drafts. The list of current Internet-
33 Drafts is at http://datatracker.ietf.org/drafts/current/.
35 Internet-Drafts are draft documents valid for a maximum of six months
36 and may be updated, replaced, or obsoleted by other documents at any
37 time. It is inappropriate to use Internet-Drafts as reference
38 material or to cite them other than as "work in progress."
40 This Internet-Draft will expire on March 30, 2017.
42 Copyright Notice
44 Copyright (c) 2016 IETF Trust and the persons identified as the
45 document authors. All rights reserved.
47 This document is subject to BCP 78 and the IETF Trust's Legal
48 Provisions Relating to IETF Documents
49 (http://trustee.ietf.org/license-info) in effect on the date of
50 publication of this document. Please review these documents
51 carefully, as they describe your rights and restrictions with respect
52 to this document. Code Components extracted from this document must
53 include Simplified BSD License text as described in Section 4.e of
54 the Trust Legal Provisions and are provided without warranty as
55 described in the Simplified BSD License.
57 Table of Contents
59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
60 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
61 2.1. Terms used in this document . . . . . . . . . . . . . . . 4
62 2.2. IEEE 802 TSN to DetNet dictionary . . . . . . . . . . . . 5
63 3. Providing the DetNet Quality of Service . . . . . . . . . . . 6
64 3.1. Congestion protection . . . . . . . . . . . . . . . . . . 8
65 3.2. Explicit routes . . . . . . . . . . . . . . . . . . . . . 8
66 3.3. Jitter Reduction . . . . . . . . . . . . . . . . . . . . 9
67 3.4. Packet Replication and Elimination . . . . . . . . . . . 10
68 3.5. Packet encoding for service protection . . . . . . . . . 11
69 4. DetNet Architecture . . . . . . . . . . . . . . . . . . . . . 12
70 4.1. Traffic Engineering for DetNet . . . . . . . . . . . . . 12
71 4.1.1. The Application Plane . . . . . . . . . . . . . . . . 12
72 4.1.2. The Controller Plane . . . . . . . . . . . . . . . . 13
73 4.1.3. The Network Plane . . . . . . . . . . . . . . . . . . 13
74 4.2. DetNet flows . . . . . . . . . . . . . . . . . . . . . . 14
75 4.2.1. Source guarantees . . . . . . . . . . . . . . . . . . 14
76 4.2.2. Incomplete Networks . . . . . . . . . . . . . . . . . 16
77 4.3. Queuing, Shaping, Scheduling, and Preemption . . . . . . 16
78 4.4. Coexistence with normal traffic . . . . . . . . . . . . . 17
79 4.5. Fault Mitigation . . . . . . . . . . . . . . . . . . . . 17
80 4.6. Representative Protocol Stack Model . . . . . . . . . . . 18
81 4.7. Exporting flow identification . . . . . . . . . . . . . . 20
82 4.8. Advertising resources, capabilities and adjacencies . . . 22
83 4.9. Provisioning model . . . . . . . . . . . . . . . . . . . 22
84 4.9.1. Centralized Path Computation and Installation . . . . 22
85 4.9.2. Distributed Path Setup . . . . . . . . . . . . . . . 22
86 4.10. Scaling to larger networks . . . . . . . . . . . . . . . 23
87 4.11. Connected islands vs. networks . . . . . . . . . . . . . 23
88 5. Compatibility with Layer-2 . . . . . . . . . . . . . . . . . 23
89 6. Open Questions . . . . . . . . . . . . . . . . . . . . . . . 24
90 6.1. Flat vs. hierarchical control . . . . . . . . . . . . . . 24
91 6.2. Peer-to-peer reservation protocol . . . . . . . . . . . . 24
92 6.3. Wireless media interactions . . . . . . . . . . . . . . . 25
93 7. Security Considerations . . . . . . . . . . . . . . . . . . . 25
94 8. Privacy Considerations . . . . . . . . . . . . . . . . . . . 26
95 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 26
96 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26
97 11. Access to IEEE 802.1 documents . . . . . . . . . . . . . . . 26
98 12. Informative References . . . . . . . . . . . . . . . . . . . 26
99 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 31
101 1. Introduction
103 Deterministic Networking (DetNet) is a service that can be offered by
104 a network to DetNet flows. DetNet provides these flows extremely low
105 packet loss rates and assured maximum end-to-end delivery latency.
106 This is accomplished by dedicating network resources such as link
107 bandwidth and buffer space to DetNet flows and/or classes of DetNet
108 flows, and by replicating packets along multiple paths. Unused
109 reserved resources are available to non-DetNet packets.
111 The Deterministic Networking Problem Statement
112 [I-D.ietf-detnet-problem-statement] introduces Deterministic
113 Networking, and Deterministic Networking Use Cases
114 [I-D.ietf-detnet-use-cases] summarizes the need for it. See
115 [I-D.dt-detnet-dp-alt] for a discussion of specific techniques that
116 can be used to identify DetNet Flows and assign them to specific
117 paths through a network.
119 A goal of DetNet is a converged network in all respects. That is,
120 the presence of DetNet flows does not preclude non-DetNet flows, and
121 the benefits offered DetNet flows should not, except in extreme
122 cases, prevent existing QoS mechanisms from operating in a normal
123 fashion, subject to the bandwidth required for the DetNet flows. A
124 single source-destination pair can trade both DetNet and non-DetNet
125 flows. End systems and applications need not instantiate special
126 interfaces for DetNet flows. Networks are not restricted to certain
127 topologies; connectivity is not restricted. Any application that
128 generates a data flow that can be usefully characterized as having a
129 maximum bandwidth should be able to take advantage of DetNet, as long
130 as the necessary resources can be reserved. Reservations can be made
131 by the application itself, via network management, by an applications
132 controller, or by other means.
134 Many applications of interest to Deterministic Networking require the
135 ability to synchronize the clocks in end systems to a sub-microsecond
136 accuracy. Some of the queue control techniques defined in
137 Section 4.3 also require time synchronization among relay and transit
138 nodes. The means used to achieve time synchronization are not
139 addressed in this document. DetNet should accommodate various
140 synchronization techniques and profiles that are defined elsewhere to
141 solve exchange time in different market segments.
143 The present document is an individual contribution, but it is
144 intended by the authors for adoption by the DetNet working group.
146 2. Terminology
148 2.1. Terms used in this document
150 The following special terms are used in this document in order to
151 avoid the assumption that a given element in the architecture does or
152 does not have Internet Protocol stack, functions as a router, bridge,
153 firewall, or otherwise plays a particular role at Layer-2 or higher.
155 destination
156 An end system capable of receiving a DetNet flow.
158 DetNet domain
159 The portion of a network that is DetNet aware. It includes
160 end systems and other DetNet nodes.
162 DetNet flow
163 A DetNet flow is a sequence of packets to which the DetNet
164 service is to be provided.
166 DetNet compound flow and DetNet member flow
167 A DetNet compound flow is a DetNet flow that has been
168 separated into multiple duplicate DetNet member flows, which
169 are eventually merged back into a single DetNet compound
170 flow, at the DetNet transport layer. "Compound" and "member"
171 are strictly relative to each other, not absolutes; a DetNet
172 compound flow comprising multiple DetNet member flows can, in
173 turn, be a member of a higher-order compound.
175 DetNet intermediate node
176 A DetNet relay node or transit node.
178 DetNet edge node
179 An instance of a DetNet relay node that includes either a
180 DetNet service layer proxy function for DetNet service
181 protection (e.g. the addition or removal of packet sequencing
182 information) for one or more end systems, or starts or
183 terminates congestion protection at the DetNet transport
184 layer, analogous to a Label Edge Router (LER).
186 end system
187 Commonly called a "host" or "node" in IETF documents, and an
188 "end station" is IEEE 802 documents. End systems of interest
189 to this document are either sources or destinations of DetNet
190 flows. And end system may or may not be DetNet transport
191 layer aware or DetNet service layer aware.
193 link
194 A connection between two DetNet nodes. It may be composed of
195 a physical link or a sub-network technology that can provide
196 appropriate traffic delivery for DetNet flows.
198 DetNet node
199 A DetNet aware end system, transit node, or relay node.
200 "DetNet" may be omitted in some text.
202 Detnet relay node
203 A DetNet node including a service layer function that
204 interconnects different DetNet transport layer paths to
205 provide service protection. A DetNet relay node can be a
206 bridge, a router, a firewall, or any other system that
207 participates in the DetNet service layer. It typically
208 incorporates DetNet transport layer functions as well, in
209 which case it is collocated with a transit node.
211 reservation
212 A trail of configuration between source to destination(s)
213 through transit nodes and subnets associated with a DetNet
214 flow, to provide congestion protection.
216 DetNet service layer
217 The layer at which service protection is provided, either
218 packet sequencing, replication, and elimination (Section 3.4)
219 or network coding (Section 3.5).
221 source
222 An end system capable of sourcing a DetNet flow.
224 DetNet transit node
225 A node operating at the DetNet transport layer, that utilizes
226 link layer and/or network layer switching across multiple
227 links and/or sub-networks to provide paths for DetNet service
228 layer functions. Optionally provides congestion protection
229 over those paths. An MPLS LSR is an example of a DetNet
230 transit node.
232 DetNet transport layer
233 The layer that optionally provides congestion protection for
234 DetNet flows over paths provided by the underlying network.
236 2.2. IEEE 802 TSN to DetNet dictionary
238 This section also serves as a dictionary for translating from the
239 terms used by the IEEE 802 Time-Sensitive Networking (TSN) Task Group
240 to those of the DetNet WG.
242 Listener
243 The IEEE 802 term for a destination of a DetNet flow.
245 relay system
246 The IEEE 802 term for a DetNet intermediate node.
248 Stream
249 The IEEE 802 term for a DetNet flow.
251 Talker
252 The IEEE 802 term for the source of a DetNet flow.
254 3. Providing the DetNet Quality of Service
256 The DetNet Quality of Service can be expressed in terms of:
258 o Minimum and maximum end-to-end latency from source to destination;
259 timely delivery and jitter avoidance derive from these constraints
261 o Probability of loss of a packet, under various assumptions as to
262 the operational states of the nodes and links. A derived property
263 is whether it is acceptable to deliver a duplicate packet, which
264 is an inherent risk in highly reliable and/or broadcast
265 transmissions
267 It is a distinction of DetNet that it is concerned solely with worst-
268 case values for the end-to-end latency. Average, mean, or typical
269 values are of no interest, because they do not affect the ability of
270 a real-time system to perform its tasks. In general, a trivial
271 priority-based queuing scheme will give better average latency to a
272 data flow than DetNet, but of course, the worst-case latency can be
273 essentially unbounded.
275 Three techniques are used by DetNet to provide these qualities of
276 service:
278 o Congestion protection (Section 3.1).
280 o Explicit routes (Section 3.2).
282 o Service protection.
284 Congestion protection operates by reserving resources along the path
285 of a DetNet Flow, e.g. buffer space or link bandwidth. Congestion
286 protection greatly reduces, or even eliminates entirely, packet loss
287 due to output packet congestion within the network, but it can only
288 be supplied to a DetNet flow that is limited at the source to a
289 maximum packet size and transmission rate.
291 Congestion protection addresses both of the DetNet QoS requirements
292 (latency and packet loss). Given that DetNet nodes have a finite
293 amount of buffer space, congestion protection necessarily results in
294 a maximum end-to-end latency. It also addresses the largest
295 contribution to packet loss, which is buffer congestion.
297 After congestion, the most important contributions to packet loss are
298 typically from random media errors and equipment failures. Service
299 protection is the name for the mechanisms used by DetNet to address
300 these losses. The mechanisms employed are constrained by the
301 requirement to meet the users' latency requirements. Packet
302 replication and elimination (Section 3.4) packet encoding Section 3.5
303 are described in this document to provide service protection; others
304 may be found. Both mechanisms distribute the contents of DetNet
305 flows over multiple paths in time and/or space, so that the loss of
306 some of the paths does need not cause the loss of any packets. The
307 paths are typically (but not necessarily) explicit routes, so that
308 they cannot suffer temporary interruptions caused by the
309 reconvergence of routing or bridging protocols.
311 These three techniques can be applied independently, giving eight
312 possible combinations, including none (no DetNet), although some
313 combinations are of wider utility than others. This separation keeps
314 the protocol stack coherent and maximizes interoperability with
315 existing and developing standards in this (IETF) and other Standards
316 Development Organizations. Some examples of typical expected
317 combinations:
319 o Explicit routes plus service protection are exactly the techniques
320 employed by [HSR-PRP]. Explicit routes are achieved by limiting
321 the physical topology of the network, and the sequentialization,
322 replication, and duplicate elimination are facilitated by packet
323 tags added at the front or the end of Ethernet frames.
325 o Congestion protection alone is is offered by IEEE 802.1 Audio
326 Video bridging [IEEE802.1BA-2011]. As long as the network suffers
327 no failures, zero congestion loss can be achieved through the use
328 of a reservation protocol (MSRP), shapers in every bridge, and a
329 bit of network calculus.
331 o Using all three together gives maximum protection.
333 There are, of course, simpler methods available (and employed, today)
334 to achieve levels of latency and packet loss that are satisfactory
335 for many applications. Prioritization and over-provisioning is one
336 such technique. However, these methods generally work best in the
337 absence of any significant amount of non-critical traffic in the
338 network (if, indeed, such traffic is supported at all), or work only
339 if the critical traffic constitutes only a small portion of the
340 network's theoretical capacity, or work only if all systems are
341 functioning properly, or in the absence of actions by end systems
342 that disrupt the network's operations.
344 There are any number of methods in use, defined, or in progress for
345 accomplishing each of the above techniques. It is expected that this
346 DetNet Architecture will assist various vendors, users, and/or
347 "vertical" Standards Development Organizations (dedicated to a single
348 industry) to make selections among the available means of
349 implementing DetNet networks.
351 3.1. Congestion protection
353 The primary means by which DetNet achieves its QoS assurances is to
354 reduce, or even completely eliminate, congestion at an output port as
355 a cause of packet loss. Given that a DetNet flow cannot be
356 throttled, this can be achieved only by the provision of sufficient
357 buffer storage at each hop through the network to ensure that no
358 packets are dropped due to a lack of buffer storage.
360 Ensuring adequate buffering requires, in turn, that the source, and
361 every intermediate node along the path to the destination (or nearly
362 every node -- see Section 4.2.2) be careful to regulate its output to
363 not exceed the data rate for any DetNet flow, except for brief
364 periods when making up for interfering traffic. Any packet sent
365 ahead of its time potentially adds to the number of buffers required
366 by the next hop, and may thus exceed the resources allocated for a
367 particular DetNet flow.
369 The low-level mechanisms described in Section 4.3 provide the
370 necessary regulation of transmissions by an end system or
371 intermediate node to provide congestion protection. The reservation
372 of the bandwidth and buffers for a DetNet flow requires the
373 provisioning described in Section 4.9. A DetNet node may have other
374 resources requiring allocation and/or scheduling, that might
375 otherwise be over-subscribed and trigger the rejection of a
376 reservation.
378 3.2. Explicit routes
380 In networks controlled by typical peer-to-peer protocols such as IEEE
381 802.1 ISIS bridged networks or IETF OSPF routed networks, a network
382 topology event in one part of the network can impact, at least
383 briefly, the delivery of data in parts of the network remote from the
384 failure or recovery event. Thus, even redundant paths through a
385 network, if controlled by the typical peer-to-peer protocols, do not
386 eliminate the chances of brief losses of contact.
388 Many real-time networks rely on physical rings or chains of two-port
389 devices, with a relatively simple ring control protocol. This
390 supports redundant paths for service protection with a minimum of
391 wiring. As an additional benefit, ring topologies can often utilize
392 different topology management protocols than those used for a mesh
393 network, with a consequent reduction in the response time to topology
394 changes. Of course, this comes at some cost in terms of increased
395 hop count, and thus latency, for the typical path.
397 In order to get the advantages of low hop count and still ensure
398 against even very brief losses of connectivity, DetNet employs
399 explicit routes, where the path taken by a given DetNet flow does not
400 change, at least immediately, and likely not at all, in response to
401 network topology events. Service protection (Section 3.4 or
402 Section 3.5) over explicit routes provides a high likelihood of
403 continuous connectivity. Explicit routes are commonly used in MPLS
404 TE LSPs.
406 3.3. Jitter Reduction
408 A core objective of DetNet is to enable the convergence of Non-IP
409 networks onto a common network infrastructure. This requires the
410 accurate emulation of currently deployed mission-specific networks,
411 which typically rely on point-to-point analog (e.g. 4-20mA
412 modulation) and serial-digital cables (or buses) for highly reliable,
413 synchronized and jitter-free communications. While the latency of
414 analog transmissions is basically the speed of light, legacy serial
415 links are usually slow (in the order of Kbps) compared to, say, GigE,
416 and some latency is usually acceptable. What is not acceptable is
417 the introduction of excessive jitter, which may, for instance, affect
418 the stability of control systems.
420 Applications that are designed to operate on serial links usually do
421 not provide services to recover the jitter, because jitter simply
422 does not exists there. Streams of information are expected to be
423 delivered in-order and the precise time of reception influences the
424 processes. In order to converge such existing applications, there is
425 a desire to emulate all properties of the serial cable, such as clock
426 transportation, perfect flow isolation and fixed latency. While
427 minimal jitter (in the form of specifying minimum, as well as
428 maximum, end-to-end latency) is supported by DetNet, there are
429 practical limitations on packet-based networks in this regard. In
430 general, users are encouraged to use, instead of, "do this when you
431 get the packet," a combination of:
433 o Sub-microsecond time synchronization among all source and
434 destination end systems, and
436 o Time-of-execution fields in the application packets.
438 Jitter reduction is provided by the mechanisms described in
439 Section 4.3 that also provide congestion protection.
441 3.4. Packet Replication and Elimination
443 After congestion loss has been eliminated, the most important causes
444 of packet loss are random media and/or memory faults, and equipment
445 failures. Both causes of packet loss can be greatly reduced by
446 spreading the data in a packet over multiple transmissions. One such
447 method for service protection is described in this section, which
448 sends the same packets over multiple paths. See also Section 3.5.
450 Packet replication and elimination, also known as seamless redundancy
451 [HSR-PRP], or 1+1 hitless protection, is a function of the DetNet
452 service layer. It involves three capabilities:
454 o Providing sequencing information, once, at or near the source, to
455 the packets of a DetNet compound flow. This may be done by adding
456 a sequence number or time stamp as part of DetNet, or may be
457 inherent in the packet, e.g. in a transport protocol, or
458 associated to other physical properties such as the precise time
459 (and radio channel) of reception of the packet. Section 3.2.
461 o Replicating these packets into multiple DetNet member flows and,
462 typically, sending them along at least two different paths to the
463 destination(s), e.g. over the explicit routes of
465 o Eliminating duplicated packets. This may be done at any step
466 along the path to save network resources further down, in
467 particular if multiple Replication points exist. But the most
468 common case is to perform this operation at the very edge of the
469 DetNet network, preferably in or near the receiver.
471 This function is a "hitless" version of, e.g., the 1+1 linear
472 protection in [RFC6372]. That is, instead of switching from one flow
473 to the other when a failure of a flow is detected, DetNet combines
474 both flows, and performs a packet-by-packet selection of which to
475 discard, based on sequence number.
477 In the simplest case, this amounts to replicating each packet in a
478 source that has two interfaces, and conveying them through the
479 network, along separate paths, to the similarly dual-homed
480 destinations, that discard the extras. This ensures that one path
481 (with zero congestion loss) remains, even if some intermediate node
482 fails. The sequence numbers can also be used for loss detection and
483 for re-ordering.
485 Detnet relay nodes in the network can provide replication and
486 elimination facilities at various points in the network, so that
487 multiple failures can be accommodated.
489 This is shown in the following figure, where the two relay nodes each
490 replicate (R) the DetNet flow on input, sending the DetNet member
491 flows to both the other relay node and to the end system, and
492 eliminate duplicates (E) on the output interface to the right-hand
493 end system. Any one link in the network can fail, and the Detnet
494 compound flow can still get through. Furthermore, two links can
495 fail, as long as they are in different segments of the network.
497 Packet replication and elimination
499 > > > > > > > > > relay > > > > > > > >
500 > /------------+ R node E +------------\ >
501 > / v + ^ \ >
502 end R + v | ^ + E end
503 system + v | ^ + system
504 > \ v + ^ / >
505 > \------------+ R relay E +-----------/ >
506 > > > > > > > > > node > > > > > > > >
508 Figure 1
510 Note that packet replication and elimination does not react to and
511 correct failures; it is entirely passive. Thus, intermittent
512 failures, mistakenly created packet filters, or misrouted data is
513 handled just the same as the equipment failures that are detected
514 handled by typical routing and bridging protocols.
516 If packet replication and elimination is used over paths providing
517 congestion protection (Section 3.1), and member flows that take
518 different-length paths through the network are combined, a merge
519 point may require extra buffering to equalize the delays over the
520 different paths. This equalization ensures that the resultant
521 compound flow will not exceed its contracted bandwidth even after one
522 or the other of the paths is restored after a failure.
524 3.5. Packet encoding for service protection
526 There are methods for using multiple paths to provide service
527 protection that involve encoding the information in a packet
528 belonging to a DetNet flow into multiple transmission units,
529 typically combining information from multiple packets into any given
530 transmission unit. Such techniques may be applicable for use as a
531 DetNet service protection technique, assuming that the DetNet users'
532 needs for timeliness of delivery and freedom from interference with
533 misbehaving DetNet flows can be met.
535 No specific mechanisms are defined here, at this time. This section
536 will either be enhanced or removed. Contributions are invited.
538 4. DetNet Architecture
540 4.1. Traffic Engineering for DetNet
542 Traffic Engineering Architecture and Signaling (TEAS) [TEAS] defines
543 traffic-engineering architectures for generic applicability across
544 packet and non-packet networks. From TEAS perspective, Traffic
545 Engineering (TE) refers to techniques that enable operators to
546 control how specific traffic flows are treated within their networks.
548 Because if its very nature of establishing explicit optimized paths,
549 Deterministic Networking can be seen as a new, specialized branch of
550 Traffic Engineering, and inherits its architecture with a separation
551 into planes.
553 The Deterministic Networking architecture is thus composed of three
554 planes, a (User) Application Plane, a Controller Plane, and a Network
555 Plane, which echoes that of Figure 1 of Software-Defined Networking
556 (SDN): Layers and Architecture Terminology [RFC7426].:
558 4.1.1. The Application Plane
560 Per [RFC7426], the Application Plane includes both applications and
561 services. In particular, the Application Plane incorporates the User
562 Agent, a specialized application that interacts with the end user /
563 operator and performs requests for Deterministic Networking services
564 via an abstract Flow Management Entity, (FME) which may or may not be
565 collocated with (one of) the end systems.
567 At the Application Plane, a management interface enables the
568 negotiation of flows between end systems. An abstraction of the flow
569 called a Traffic Specification (TSpec) provides the representation.
570 This abstraction is used to place a reservation over the (Northbound)
571 Service Interface and within the Application plane. It is associated
572 with an abstraction of location, such as IP addresses and DNS names,
573 to identify the end systems and eventually specify intermediate
574 nodes.
576 4.1.2. The Controller Plane
578 The Controller Plane corresponds to the aggregation of the Control
579 and Management Planes in [RFC7426], though Common Control and
580 Measurement Plane (CCAMP) [CCAMP] makes an additional distinction
581 between management and measurement. When the logical separation of
582 the Control, Measurement and other Management entities is not
583 relevant, the term Controller Plane is used for simplicity to
584 represent them all, and the term controller refers to any device
585 operating in that plane, whether is it a Path Computation entity or a
586 Network Management entity (NME). The Path Computation Element (PCE)
587 [PCE] is a core element of a controller, in charge of computing
588 Deterministic paths to be applied in the Network Plane.
590 A (Northbound) Service Interface enables applications in the
591 Application Plane to communicate with the entities in the Controller
592 Plane.
594 One or more PCE(s) collaborate to implement the requests from the FME
595 as Per-Flow Per-Hop Behaviors installed in the intermediate nodes for
596 each individual flow. The PCEs place each flow along a deterministic
597 sequence of intermediate nodes so as to respect per-flow constraints
598 such as security and latency, and optimize the overall result for
599 metrics such as an abstract aggregated cost. The deterministic
600 sequence can typically be more complex than a direct sequence and
601 include redundancy path, with one or more packet replication and
602 elimination points.
604 4.1.3. The Network Plane
606 The Network Plane represents the network devices and protocols as a
607 whole, regardless of the Layer at which the network devices operate.
608 It includes Forwarding Plane (data plane), Application, and
609 Operational Plane (control plane) aspects.
611 The network Plane comprises the Network Interface Cards (NIC) in the
612 end systems, which are typically IP hosts, and intermediate nodes,
613 which are typically IP routers and switches. Network-to-Network
614 Interfaces such as used for Traffic Engineering path reservation in
615 [RFC5921], as well as User-to-Network Interfaces (UNI) such as
616 provided by the Local Management Interface (LMI) between network and
617 end systems, are both part of the Network Plane, both in the control
618 plane and the data plane.
620 A Southbound (Network) Interface enables the entities in the
621 Controller Plane to communicate with devices in the Network Plane.
622 This interface leverages and extends TEAS to describe the physical
623 topology and resources in the Network Plane.
625 Flow Management Entity
627 End End
628 System System
630 -+-+-+-+-+-+-+ Northbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
632 PCE PCE PCE PCE
634 -+-+-+-+-+-+-+ Southbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
636 intermediate intermed. intermed. intermed.
637 Node Node Node Node
638 NIC NIC
639 intermediate intermed. intermed. intermed.
640 Node Node Node Node
642 Figure 2
644 The intermediate nodes (and eventually the end systems NIC) expose
645 their capabilities and physical resources to the controller (the
646 PCE), and update the PCE with their dynamic perception of the
647 topology, across the Southbound Interface. In return, the PCE(s) set
648 the per-flow paths up, providing a Flow Characterization that is more
649 tightly coupled to the intermediate node Operation than a TSpec.
651 At the Network plane, intermediate nodes may exchange information
652 regarding the state of the paths, between adjacent systems and
653 eventually with the end systems, and forward packets within
654 constraints associated to each flow, or, when unable to do so,
655 perform a last resort operation such as drop or declassify.
657 This specification focuses on the Southbound interface and the
658 operation of the Network Plane.
660 4.2. DetNet flows
662 4.2.1. Source guarantees
664 For the purposes of congestion protection, DetNet flows can be
665 synchronous or asynchronous. In synchronous DetNet flows, at least
666 the intermediate nodes (and possibly the end systems) are closely
667 time synchronized, typically to better than 1 microsecond. By
668 transmitting packets from different DetNet flows or classes of DetNet
669 flows at different times, using repeating schedules synchronized
670 among the intermediate nodes, resources such as buffers and link
671 bandwidth can be shared over the time domain among different DetNet
672 flows. There is a tradeoff among techniques for synchronous DetNet
673 flows between the burden of fine-grained scheduling and the benefit
674 of reducing the required resources, especially buffer space.
676 In contrast, asynchronous DetNet flows are not coordinated with a
677 fine-grained schedule, so relay and end systems must assume worst-
678 case interference among DetNet flows contending for buffer resources.
679 Asynchronous DetNet flows are characterized by:
681 o A maximum packet size;
683 o An observation interval; and
685 o A maximum number of transmissions during that observation
686 interval.
688 These parameters, together with knowledge of the protocol stack used
689 (and thus the size of the various headers added to a packet), limit
690 the number of bit times per observation interval that the DetNet flow
691 can occupy the physical medium.
693 The source promises that these limits will not be exceeded. If the
694 source transmits less data than this limit allows, the unused
695 resources such as link bandwidth can be made available by the system
696 to non-DetNet packets. However, making those resources available to
697 DetNet packets in other DetNet flows would serve no purpose. Those
698 other DetNet flows have their own dedicated resources, on the
699 assumption that all DetNet flows can use all of their resources over
700 a long period of time.
702 Note that there is no provision in DetNet for throttling DetNet flows
703 (reducing the transmission rate via feedback); the assumption is that
704 a DetNet flow, to be useful, must be delivered in its entirety. That
705 is, while any useful application is written to expect a certain
706 number of lost packets, the real-time applications of interest to
707 DetNet demand that the loss of data due to the network is
708 extraordinarily infrequent.
710 Although DetNet strives to minimize the changes required of an
711 application to allow it to shift from a special-purpose digital
712 network to an Internet Protocol network, one fundamental shift in the
713 behavior of network applications is impossible to avoid: the
714 reservation of resources before the application starts. In the first
715 place, a network cannot deliver finite latency and practically zero
716 packet loss to an arbitrarily high offered load. Secondly, achieving
717 practically zero packet loss for unthrottled (though bandwidth
718 limited) DetNet flows means that bridges and routers have to dedicate
719 buffer resources to specific DetNet flows or to classes of DetNet
720 flows. The requirements of each reservation have to be translated
721 into the parameters that control each system's queuing, shaping, and
722 scheduling functions and delivered to the hosts, bridges, and
723 routers.
725 4.2.2. Incomplete Networks
727 The presence in the network of transit nodes or subnets that are not
728 fully capable of offering DetNet services complicates the ability of
729 the intermediate nodes and/or controller to allocate resources, as
730 extra buffering, and thus extra latency, must be allocated at points
731 downstream from the non-DetNet intermediate node for a DetNet flow.
733 4.3. Queuing, Shaping, Scheduling, and Preemption
735 DetNet achieves congestion protection and bounded delivery latency by
736 reserving bandwidth and buffer resources at every hop along the path
737 of the DetNet flow. The reservation itself is not sufficient,
738 however. Implementors and users of a number of proprietary and
739 standard real-time networks have found that standards for specific
740 data plane techniques are required to enable these assurances to be
741 made in a multi-vendor network. The fundamental reason is that
742 latency variation in one system results in the need for extra buffer
743 space in the next-hop system(s), which in turn, increases the worst-
744 case per-hop latency.
746 Standard queuing and transmission selection algorithms allow a
747 central controller to compute the latency contribution of each
748 transit node to the end-to-end latency, to compute the amount of
749 buffer space required in each transit node for each incremental
750 DetNet flow, and most importantly, to translate from a flow
751 specification to a set of values for the managed objects that control
752 each relay or end system. The IEEE 802 has specified (and is
753 specifying) a set of queuing, shaping, and scheduling algorithms that
754 enable each transit node (bridge or router), and/or a central
755 controller, to compute these values. These algorithms include:
757 o A credit-based shaper [IEEE802.1Q-2014] Clause 34.
759 o Time-gated queues governed by a rotating time schedule,
760 synchronized among all transit nodes [IEEE802.1Qbv].
762 o Synchronized double (or triple) buffers driven by synchronized
763 time ticks. [IEEE802.1Qch].
765 o Pre-emption of an Ethernet packet in transmission by a packet with
766 a more stringent latency requirement, followed by the resumption
767 of the preempted packet [IEEE802.1Qbu], [IEEE802.3br].
769 While these techniques are currently embedded in Ethernet and
770 bridging standards, we can note that they are all, except perhaps for
771 packet preemption, equally applicable to other media than Ethernet,
772 and to routers as well as bridges.
774 4.4. Coexistence with normal traffic
776 A DetNet network supports the dedication of a high proportion (e.g.
777 75%) of the network bandwidth to DetNet flows. But, no matter how
778 much is dedicated for DetNet flows, it is a goal of DetNet to coexist
779 with existing Class of Service schemes (e.g., DiffServ). It is also
780 important that non-DetNet traffic not disrupt the DetNet flow, of
781 course (see Section 4.5 and Section 7). For these reasons:
783 o Bandwidth (transmission opportunities) not utilized by a DetNet
784 flow are available to non-DetNet packets (though not to other
785 DetNet flows).
787 o DetNet flows can be shaped or scheduled, in order to ensure that
788 the highest-priority non-DetNet packet also is ensured a worst-
789 case latency (at any given hop).
791 o When transmission opportunities for DetNet flows are scheduled in
792 detail, then the algorithm constructing the schedule should leave
793 sufficient opportunities for non-DetNet packets to satisfy the
794 needs of the users of the network. Detailed scheduling can also
795 permit the time-shared use of buffer resources by different DetNet
796 flows.
798 Ideally, the net effect of the presence of DetNet flows in a network
799 on the non-DetNet packets is primarily a reduction in the available
800 bandwidth.
802 4.5. Fault Mitigation
804 One key to building robust real-time systems is to reduce the
805 infinite variety of possible failures to a number that can be
806 analyzed with reasonable confidence. DetNet aids in the process by
807 providing filters and policers to detect DetNet packets received on
808 the wrong interface, or at the wrong time, or in too great a volume,
809 and to then take actions such as discarding the offending packet,
810 shutting down the offending DetNet flow, or shutting down the
811 offending interface.
813 It is also essential that filters and service remarking be employed
814 at the network edge to prevent non-DetNet packets from being mistaken
815 for DetNet packets, and thus impinging on the resources allocated to
816 DetNet packets.
818 There exist techniques, at present and/or in various stages of
819 standardization, that can perform these fault mitigation tasks that
820 deliver a high probability that misbehaving systems will have zero
821 impact on well-behaved DetNet flows, except of course, for the
822 receiving interface(s) immediately downstream of the misbehaving
823 device. Examples of such techniques include traffic policing
824 functions (e.g. [RFC2475]) and separating flows into per-flow rate-
825 limited queues.
827 4.6. Representative Protocol Stack Model
829 Figure 3 illustrates a conceptual DetNet data plane layering model.
830 One may compare it to that in [IEEE802.1CB], Annex C, a work in
831 progress.
833 DetNet data plane protocol stack
835 | packets going | ^ packets coming ^
836 v down the stack v | up the stack |
837 +----------------------+ +-----------------------+
838 | Source | | Destination |
839 +----------------------+ +-----------------------+
840 | Service layer | | Service layer |
841 | Packet sequencing | | Duplicate elimination |
842 | Flow duplication | | Flow merging |
843 | Packet encoding | | Packet decoding |
844 +----------------------+ +-----------------------+
845 | Transport layer | | Transport layer |
846 | Congestion prot. | | Congestion prot. |
847 +----------------------+ +-----------------------+
848 | Lower layers | | Lower layers |
849 +----------------------+ +-----------------------+
850 v ^
851 \_________________________/
853 Figure 3
855 Not all layers are required for any given application, or even for
856 any given network. The layers are, from top to bottom:
858 Application
859 Shown as "source" and "destination" in the diagram.
861 OAM
862 Operations, Administration, and Maintenance leverages in-band
863 and out-of-and signaling that validates whether the service
864 is effectively obtained within QoS constraints. OAM is not
865 shown in Figure 3; it may reside in any number of the layers.
867 OAM can involve specific tagging added in the packets for
868 tracing implementation or network configuration errors;
869 traceability enables to find whether a packet is a replica,
870 which relay node performed the replication, and which segment
871 was intended for the replica.
873 Packet sequencing
874 As part of DetNet service protection, supplies the sequence
875 number for packet replication and elimination (Section 3.4).
876 Peers with Duplicate elimination. This layer is not needed
877 if a higher-layer transport protocol is expected to perform
878 any packet sequencing and duplicate elimination required by
879 the DetNet flow duplication.
881 Duplicate elimination
882 As part of the DetNet service layer, based on the sequenced
883 number supplied by its peer, packet sequencing, Duplicate
884 elimination discards any duplicate packets generated by
885 DetNet flow duplication. It can operate on member flows,
886 compound flows, or both. The duplication may also be
887 inferred from other information such as the precise time of
888 reception in a scheduled network. The duplicate elimination
889 layer may also perform resequencing of packets to restore
890 packet order in a flow that was disrupted by the loss of
891 packets on one or another of the multiple paths taken.
893 Flow duplication
894 As part of DetNet service protection, replicates packets that
895 belong to a DetNet compound flow into two or more DetNet
896 member flows. Note that this function is separate from
897 packet sequencing. Flow duplication can be an explicit
898 duplication and remarking of packets, or can be performed by,
899 for example, techniques similar to ordinary multicast
900 replication. Peers with DetNet flow merging.
902 Network flow merging
903 As part of DetNet service protection, merges DetNet member
904 flows together for packets coming up the stack belonging to a
905 specific DetNet compound flow. Peers with DetNet flow
906 duplication. DetNet flow merging, together with packet
907 sequencing, duplicate elimination, and DetNet flow
908 duplication, performs packet replication and elimination
909 (Section 3.4).
911 Packet encoding
912 As part of DetNet service protection, as an alternative to
913 packet sequencing and flow duplication, packet encoding
914 combines the information in multiple DetNet packets, perhaps
915 from different DetNet compound flows, and transmits that
916 information in packets on different DetNet member Flows.
917 Peers with Packet decoding.
919 Packet decoding
920 As part of DetNet service protection, as an alternative to
921 flow merging and duplicate elimination, packet decoding takes
922 packets from different DetNet member flows, and computes from
923 those packets the original DetNet packets from the compound
924 flows input to packet encoding. Peers with Packet encoding.
926 Congestio protection
927 The DetNet transport layer provides congestion protection.
928 See Section 4.3. The actual queuing and shaping mechaniss
929 are typically provided by underlying subnet layers, but since
930 these are can be closely associated with the means of
931 providing paths for DetNet flows (e.g. MPLS LSPs or {VLAN,
932 multicast destination MAC address} pairs), the path and the
933 congestion protection are conflated in this figure.
935 Note that the packet sequencing and duplication elimination functions
936 at the source and destination ends of a DetNet compound flow may be
937 performed either in the end system or in a DetNet edge node. The
938 reader must not confuse a DetNet edge function with other kinds of
939 edge functions, e.g. an Label Edge Router, although the two functions
940 may be performed together. The DetNet edge function is concerned
941 with sequencing packets belonging to DetNet flows. The LER with
942 encapsulating/decapsulating packets for transport, and is considered
943 part of the network underlying the DetNet transport layer.
945 4.7. Exporting flow identification
947 An interesting feature of DetNet, and one that invites
948 implementations that can be accused of "layering violations", is the
949 need for lower layers to be aware of specific flows at higher layers,
950 in order to provide specific queuing and shaping services for
951 specific flows. For example:
953 o A non-IP, strictly L2 source end system X may be sending multiple
954 flows to the same L2 destination end system Y. Those flows may
955 include DetNet flows with different QoS requirements, and may
956 include non-DetNet flows.
958 o A router may be sending any number of flows to another router.
959 Again, those flows may include DetNet flows with different QoS
960 requirements, and may include non-DetNet flows.
962 o Two routers may be separated by bridges. For these bridges to
963 perform any required per-flow queuing and shaping, they must be
964 able to identify the individual flows.
966 o A Label Edge Router (LERs) may have a Label Switched Path (LSP)
967 set up for handling traffic destined for a particular IP address
968 carrying only non-DetNet flows. If a DetNet flow to that same
969 address is requested, a separate LSP may be needed, in order that
970 all of the Label Switch Routers (LSRs) along the path to the
971 destination give that flow special queuing and shaping.
973 The need for a lower-level DetNet node to be aware of individual
974 higher-layer flows is not unique to DetNet. But, given the endless
975 complexity of layering and relayering over tunnels that is available
976 to network designers, DetNet needs to provide a model for flow
977 identification that is at least somewhat better than packet
978 inspection. That is not to say that packet inspection to layer 4 or
979 5 addresses will not be used, or the capability standardized; but,
980 there are alternatives.
982 A DetNet relay node can connect DetNet flows on different paths using
983 different flow identification methods. For example:
985 o A single unicast DetNet flow passing from router A through a
986 bridged network to router B may be assigned a {VLAN, multicast
987 destination MAC address} pair that is unique within that bridged
988 network. The bridges can then identify the flow without accessing
989 higher-layer headers. Of course, the receiving router must
990 recognize and accept that multicast MAC address.
992 o A DetNet flow passing from LSR A to LSR B may be assigned a
993 different label than that used for other flows to the same IP
994 destination.
996 In any of the above cases, it is possible that an existing DetNet
997 flow can be used as a carrier for multiple DetNet sub-flows. (Not to
998 be confused with DetNet compound vs. member flows.) Of course, this
999 requires that the aggregate DetNet flow be provisioned properly to
1000 carry the sub-flows.
1002 Thus, rather than packet inspection, there is the option to export
1003 higher-layer information to the lower layer. The requirement to
1004 support one or the other method for flow identification (or both) is
1005 the essential complexity that DetNet brings to existing control plane
1006 models.
1008 4.8. Advertising resources, capabilities and adjacencies
1010 There are three classes of information that a central controller or
1011 decentralized control plane needs to know that can only be obtained
1012 from the end systems and/or transit nodes in the network. When using
1013 a peer-to-peer control plane, some of this information may be
1014 required by a system's neighbors in the network.
1016 o Details of the system's capabilities that are required in order to
1017 accurately allocate that system's resources, as well as other
1018 systems' resources. This includes, for example, which specific
1019 queuing and shaping algorithms are implemented (Section 4.3), the
1020 number of buffers dedicated for DetNet allocation, and the worst-
1021 case forwarding delay.
1023 o The dynamic state of an end or transit node's DetNet resources.
1025 o The identity of the system's neighbors, and the characteristics of
1026 the link(s) between the systems, including the length (in
1027 nanoseconds) of the link(s).
1029 4.9. Provisioning model
1031 4.9.1. Centralized Path Computation and Installation
1033 A centralized routing model, such as provided with a PCE (RFC 4655
1034 [RFC4655]), enables global and per-flow optimizations. (See
1035 Section 4.1.) The model is attractive but a number of issues are
1036 left to be solved. In particular:
1038 o Whether and how the path computation can be installed by 1) an end
1039 device or 2) a Network Management entity,
1041 o And how the path is set up, either by installing state at each hop
1042 with a direct interaction between the forwarding device and the
1043 PCE, or along a path by injecting a source-routed request at one
1044 end of the path.
1046 4.9.2. Distributed Path Setup
1048 Significant work on distributed path setup can be leveraged from MPLS
1049 Traffic Engineering, in both its GMPLS and non-GMPLS forms. The
1050 protocols within scope are Resource ReSerVation Protocol [RFC3209]
1051 [RFC3473](RSVP-TE), OSPF-TE [RFC4203] [RFC5392] and ISIS-TE [RFC5307]
1052 [RFC5316]. These should be viewed as starting points as there are
1053 feature specific extensions defined that may be applicable to DetNet.
1055 In a Layer-2 only environment, or as part of a layered approach to a
1056 mixed environment, IEEE 802.1 also has work, either completed or in
1057 progress. [IEEE802.1Q-2014] Clause 35 describes SRP, a peer-to-peer
1058 protocol for Layer-2 roughly analogous to RSVP [RFC2205].
1059 [IEEE802.1Qca] defines how ISIS can provide multiple disjoint paths
1060 or distribution trees. Also in progress is [IEEE802.1Qcc], which
1061 expands the capabilities of SRP.
1063 The integration/interaction of the DetNet control layer with an
1064 underlying IEEE 802.1 sub-network control layer will need to be
1065 defined.
1067 4.10. Scaling to larger networks
1069 Reservations for individual DetNet flows require considerable state
1070 information in each transit node, especially when adequate fault
1071 mitigation (Section 4.5) is required. The DetNet data plane, in
1072 order to support larger numbers of DetNet flows, must support the
1073 aggregation of DetNet flows into tunnels, which themselves can be
1074 viewed by the transit nodes' data planes largely as individual DetNet
1075 flows. Without such aggregation, the per-relay system may limit the
1076 scale of DetNet networks.
1078 4.11. Connected islands vs. networks
1080 Given that users have deployed examples of the IEEE 802.1 TSN TG
1081 standards, which provide capabilities similar to DetNet, it is
1082 obvious to ask whether the IETF DetNet effort can be limited to
1083 providing Layer-2 connections (VPNs) between islands of bridged TSN
1084 networks. While this capability is certainly useful to some
1085 applications, and must not be precluded by DetNet, tunneling alone is
1086 not a sufficient goal for the DetNet WG. As shown in the
1087 Deterministic Networking Use Cases draft [I-D.ietf-detnet-use-cases],
1088 there are already deployments of Layer-2 TSN networks that are
1089 encountering the well-known problems of over-large broadcast domains.
1090 Routed solutions, and combinations routed/bridged solutions, are both
1091 required.
1093 5. Compatibility with Layer-2
1095 Standards providing similar capabilities for bridged networks (only)
1096 have been and are being generated in the IEEE 802 LAN/MAN Standards
1097 Committee. The present architecture describes an abstract model that
1098 can be applicable both at Layer-2 and Layer-3, and over links not
1099 defined by IEEE 802. It is the intention of the authors (and
1100 hopefully, as this draft progresses, of the DetNet Working Group)
1101 that IETF and IEEE 802 will coordinate their work, via the
1102 participation of common individuals, liaisons, and other means, to
1103 maximize the compatibility of their outputs.
1105 DetNet enabled end systems and intermediate nodes can be
1106 interconnected by sub-networks, i.e., Layer-2 technologies. These
1107 sub-networks will provide DetNet compatible service for support of
1108 DetNet traffic. Examples of sub-networks include 802.1TSN and a
1109 point-to-point OTN link. Of course, multi-layer DetNet systems may
1110 be possible too, where one DetNet appears as a sub-network, and
1111 provides service to, a higher layer DetNet system.
1113 6. Open Questions
1115 There are a number of architectural questions that will have to be
1116 resolved before this document can be submitted for publication.
1117 Aside from the obvious fact that this present draft is subject to
1118 change, there are specific questions to which the authors wish to
1119 direct the readers' attention.
1121 6.1. Flat vs. hierarchical control
1123 Boxes that are solely routers or solely bridges are rare in today's
1124 market. In a multi-tenant data center, multiple users' virtual
1125 Layer-2/Layer-3 topologies exist simultaneously, implemented on a
1126 network whose physical topology bears only accidental resemblance to
1127 the virtual topologies.
1129 While the forwarding topology (the bridges and routers) are an
1130 important consideration for a DetNet Flow Management Entity
1131 (Section 4.1.1), so is the purely physical topology. Ultimately, the
1132 model used by the management entities is based on boxes, queues, and
1133 links. The authors hope that the work of the TEAS WG will help to
1134 clarify exactly what model parameters need to be traded between the
1135 intermediate nodes and the controller(s).
1137 6.2. Peer-to-peer reservation protocol
1139 As described in Section 4.9.2, the DetNet WG needs to decide whether
1140 to support a peer-to-peer protocol for a source and a destination to
1141 reserve resources for a DetNet stream. Assuming that enabling the
1142 involvement of the source and/or destination is desirable (see
1143 Deterministic Networking Use Cases [I-D.ietf-detnet-use-cases]), it
1144 remains to decide whether the DetNet WG will make it possible to
1145 deploy at least some DetNet capabilities in a network using only a
1146 peer-to-peer protocol, without a central controller.
1148 (Note that a UNI (see Section 4.1.3) between an end system and a
1149 DetNet edge node, for sources and/or listeners to request DetNet
1150 services, can be either the first hop of a per-to-peer reservation
1151 protocol, or can be deflected by the DetNet edge node to a central
1152 controller for resolution. Similarly, a decision by a central
1153 controller can be effected by the controller instructing the end
1154 system or DetNet edge node to initiate a per-to-peer protocol
1155 activity.)
1157 6.3. Wireless media interactions
1159 Deterministic Networking Use Cases [I-D.ietf-detnet-use-cases]
1160 illustrates cases where wireless media are needed in a DetNet
1161 network. Some wireless media in general use, such as IEEE 802.11
1162 [IEEE802.1Q-2014], have significantly higher packet loss rates than
1163 typical wired media, such as Ethernet [IEEE802.3-2012]. IEEE 802.11
1164 includes support for such features as MAC-layer acknowledgements and
1165 retransmissions.
1167 The techniques described in Section 3 are likely to improve the
1168 ability of a mixed wired/wireless network to offer the DetNet QoS
1169 features. The interaction of these techniques with the features of
1170 specific wireless media, although they may be significant, cannot be
1171 addressed in this document. It remains to be decided to what extent
1172 the DetNet WG will address them, and to what extent other WGs, e.g.
1173 6TiSCH, will do so.
1175 7. Security Considerations
1177 Security in the context of Deterministic Networking has an added
1178 dimension; the time of delivery of a packet can be just as important
1179 as the contents of the packet, itself. A man-in-the-middle attack,
1180 for example, can impose, and then systematically adjust, additional
1181 delays into a link, and thus disrupt or subvert a real-time
1182 application without having to crack any encryption methods employed.
1183 See [RFC7384] for an exploration of this issue in a related context.
1185 Furthermore, in a control system where millions of dollars of
1186 equipment, or even human lives, can be lost if the DetNet QoS is not
1187 delivered, one must consider not only simple equipment failures,
1188 where the box or wire instantly becomes perfectly silent, but bizarre
1189 errors such as can be caused by software failures. Because there is
1190 essential no limit to the kinds of failures that can occur,
1191 protecting against realistic equipment failures is indistinguishable,
1192 in most cases, from protecting against malicious behavior, whether
1193 accidental or intentional. See also Section 4.5.
1195 Security must cover:
1197 o the protection of the signaling protocol
1198 o the authentication and authorization of the controlling systems
1200 o the identification and shaping of the DetNet flows
1202 8. Privacy Considerations
1204 DetNet is provides a Quality of Service (QoS), and as such, does not
1205 directly raise any new privacy considerations.
1207 However, the requirement for every (or almost every) node along the
1208 path of a DetNet flow to identify DetNet flows may present an
1209 additional attack surface for privacy, should the DetNet paradigm be
1210 found useful in broader environments.
1212 9. IANA Considerations
1214 This document does not require an action from IANA.
1216 10. Acknowledgements
1218 The authors wish to thank Jouni Korhonen, Erik Nordmark, George
1219 Swallow, Rudy Klecka, Anca Zamfir, David Black, Thomas Watteyne,
1220 Shitanshu Shah, Craig Gunther, Rodney Cummings, Balazs Varga,
1221 Wilfried Steiner, Marcel Kiessling, Karl Weber, Janos Farkas, Ethan
1222 Grossman, Pat Thaler, Lou Berger, and especially Michael Johas
1223 Teener, for their various contribution with this work.
1225 11. Access to IEEE 802.1 documents
1227 To access password protected IEEE 802.1 drafts, see the IETF IEEE
1228 802.1 information page at https://www.ietf.org/proceedings/52/slides/
1229 bridge-0/tsld003.htm.
1231 12. Informative References
1233 [AVnu] http://www.avnu.org/, "The AVnu Alliance tests and
1234 certifies devices for interoperability, providing a simple
1235 and reliable networking solution for AV network
1236 implementation based on the Audio Video Bridging (AVB)
1237 standards.".
1239 [CCAMP] IETF, "Common Control and Measurement Plane",
1240 .
1242 [HART] www.hartcomm.org, "Highway Addressable Remote Transducer,
1243 a group of specifications for industrial process and
1244 control devices administered by the HART Foundation".
1246 [HSR-PRP] IEC, "High availability seamless redundancy (HSR) is a
1247 further development of the PRP approach, although HSR
1248 functions primarily as a protocol for creating media
1249 redundancy while PRP, as described in the previous
1250 section, creates network redundancy. PRP and HSR are both
1251 described in the IEC 62439 3 standard.",
1252 .
1255 [I-D.dt-detnet-dp-alt]
1256 Korhonen, J., Farkas, J., Mirsky, G., Thubert, P.,
1257 Zhuangyan, Z., and L. Berger, "DetNet Data Plane Protocol
1258 and Solution Alternatives", draft-dt-detnet-dp-alt-04
1259 (work in progress), September 2016.
1261 [I-D.ietf-6tisch-architecture]
1262 Thubert, P., "An Architecture for IPv6 over the TSCH mode
1263 of IEEE 802.15.4", draft-ietf-6tisch-architecture-10 (work
1264 in progress), June 2016.
1266 [I-D.ietf-6tisch-tsch]
1267 Watteyne, T., Palattella, M., and L. Grieco, "Using
1268 IEEE802.15.4e TSCH in an IoT context: Overview, Problem
1269 Statement and Goals", draft-ietf-6tisch-tsch-06 (work in
1270 progress), March 2015.
1272 [I-D.ietf-detnet-problem-statement]
1273 Finn, N. and P. Thubert, "Deterministic Networking Problem
1274 Statement", draft-ietf-detnet-problem-statement-00 (work
1275 in progress), April 2016.
1277 [I-D.ietf-detnet-use-cases]
1278 Grossman, E., Gunther, C., Thubert, P., Wetterwald, P.,
1279 Raymond, J., Korhonen, J., Kaneko, Y., Das, S., Zha, Y.,
1280 Varga, B., Farkas, J., Goetz, F., and J. Schmitt,
1281 "Deterministic Networking Use Cases", draft-ietf-detnet-
1282 use-cases-10 (work in progress), July 2016.
1284 [I-D.ietf-roll-rpl-industrial-applicability]
1285 Phinney, T., Thubert, P., and R. Assimiti, "RPL
1286 applicability in industrial networks", draft-ietf-roll-
1287 rpl-industrial-applicability-02 (work in progress),
1288 October 2013.
1290 [I-D.svshah-tsvwg-deterministic-forwarding]
1291 Shah, S. and P. Thubert, "Deterministic Forwarding PHB",
1292 draft-svshah-tsvwg-deterministic-forwarding-04 (work in
1293 progress), August 2015.
1295 [IEEE802.11-2012]
1296 IEEE, "Wireless LAN Medium Access Control (MAC) and
1297 Physical Layer (PHY) Specifications", 2012,
1298 .
1301 [IEEE802.1AS-2011]
1302 IEEE, "Timing and Synchronizations (IEEE 802.1AS-2011)",
1303 2011, .
1306 [IEEE802.1BA-2011]
1307 IEEE, "AVB Systems (IEEE 802.1BA-2011)", 2011,
1308 .
1311 [IEEE802.1CB]
1312 IEEE, "Frame Replication and Elimination for Reliability
1313 (IEEE Draft P802.1CB)", 2016,
1314 .
1316 [IEEE802.1Q-2014]
1317 IEEE, "MAC Bridges and VLANs (IEEE 802.1Q-2014", 2014,
1318 .
1321 [IEEE802.1Qbu]
1322 IEEE, "Frame Preemption", 2016,
1323 .
1325 [IEEE802.1Qbv]
1326 IEEE, "Enhancements for Scheduled Traffic", 2016,
1327 .
1329 [IEEE802.1Qca]
1330 IEEE 802.1, "IEEE 802.1Qca Bridges and Bridged Networks -
1331 Amendment 24: Path Control and Reservation", IEEE
1332 P802.1Qca/D2.1 P802.1Qca, June 2015,
1333 .
1336 [IEEE802.1Qcc]
1337 IEEE, "Stream Reservation Protocol (SRP) Enhancements and
1338 Performance Improvements", 2016,
1339 .
1341 [IEEE802.1Qch]
1342 IEEE, "Cyclic Queuing and Forwarding", 2016,
1343 .
1345 [IEEE802.1TSNTG]
1346 IEEE Standards Association, "IEEE 802.1 Time-Sensitive
1347 Networks Task Group", 2013,
1348 .
1350 [IEEE802.3-2012]
1351 IEEE, "IEEE Stabdard for Ethernet", 2012,
1352 .
1355 [IEEE802.3br]
1356 IEEE, "Interspersed Express Traffic", 2016,
1357 .
1359 [IEEE802154]
1360 IEEE standard for Information Technology, "IEEE std.
1361 802.15.4, Part. 15.4: Wireless Medium Access Control (MAC)
1362 and Physical Layer (PHY) Specifications for Low-Rate
1363 Wireless Personal Area Networks", June 2011.
1365 [IEEE802154e]
1366 IEEE standard for Information Technology, "IEEE std.
1367 802.15.4e, Part. 15.4: Low-Rate Wireless Personal Area
1368 Networks (LR-WPANs) Amendment 1: MAC sublayer", April
1369 2012.
1371 [ISA100.11a]
1372 ISA/IEC, "ISA100.11a, Wireless Systems for Automation,
1373 also IEC 62734", 2011, < http://www.isa100wci.org/en-
1374 US/Documents/PDF/3405-ISA100-WirelessSystems-Future-broch-
1375 WEB-ETSI.aspx>.
1377 [ISA95] ANSI/ISA, "Enterprise-Control System Integration Part 1:
1378 Models and Terminology", 2000, .
1381 [ODVA] http://www.odva.org/, "The organization that supports
1382 network technologies built on the Common Industrial
1383 Protocol (CIP) including EtherNet/IP.".
1385 [PCE] IETF, "Path Computation Element",
1386 .
1388 [Profinet]
1389 http://us.profinet.com/technology/profinet/, "PROFINET is
1390 a standard for industrial networking in automation.",
1391 .
1393 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.
1394 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
1395 Functional Specification", RFC 2205, DOI 10.17487/RFC2205,
1396 September 1997, .
1398 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.,
1399 and W. Weiss, "An Architecture for Differentiated
1400 Services", RFC 2475, DOI 10.17487/RFC2475, December 1998,
1401 .
1403 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
1404 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
1405 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001,
1406 .
1408 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label
1409 Switching (GMPLS) Signaling Resource ReserVation Protocol-
1410 Traffic Engineering (RSVP-TE) Extensions", RFC 3473,
1411 DOI 10.17487/RFC3473, January 2003,
1412 .
1414 [RFC4203] Kompella, K., Ed. and Y. Rekhter, Ed., "OSPF Extensions in
1415 Support of Generalized Multi-Protocol Label Switching
1416 (GMPLS)", RFC 4203, DOI 10.17487/RFC4203, October 2005,
1417 .
1419 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation
1420 Element (PCE)-Based Architecture", RFC 4655,
1421 DOI 10.17487/RFC4655, August 2006,
1422 .
1424 [RFC5307] Kompella, K., Ed. and Y. Rekhter, Ed., "IS-IS Extensions
1425 in Support of Generalized Multi-Protocol Label Switching
1426 (GMPLS)", RFC 5307, DOI 10.17487/RFC5307, October 2008,
1427 .
1429 [RFC5316] Chen, M., Zhang, R., and X. Duan, "ISIS Extensions in
1430 Support of Inter-Autonomous System (AS) MPLS and GMPLS
1431 Traffic Engineering", RFC 5316, DOI 10.17487/RFC5316,
1432 December 2008, .
1434 [RFC5392] Chen, M., Zhang, R., and X. Duan, "OSPF Extensions in
1435 Support of Inter-Autonomous System (AS) MPLS and GMPLS
1436 Traffic Engineering", RFC 5392, DOI 10.17487/RFC5392,
1437 January 2009, .
1439 [RFC5673] Pister, K., Ed., Thubert, P., Ed., Dwars, S., and T.
1440 Phinney, "Industrial Routing Requirements in Low-Power and
1441 Lossy Networks", RFC 5673, DOI 10.17487/RFC5673, October
1442 2009, .
1444 [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau,
1445 L., and L. Berger, "A Framework for MPLS in Transport
1446 Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010,
1447 .
1449 [RFC6372] Sprecher, N., Ed. and A. Farrel, Ed., "MPLS Transport
1450 Profile (MPLS-TP) Survivability Framework", RFC 6372,
1451 DOI 10.17487/RFC6372, September 2011,
1452 .
1454 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in
1455 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384,
1456 October 2014, .
1458 [RFC7426] Haleplidis, E., Ed., Pentikousis, K., Ed., Denazis, S.,
1459 Hadi Salim, J., Meyer, D., and O. Koufopavlou, "Software-
1460 Defined Networking (SDN): Layers and Architecture
1461 Terminology", RFC 7426, DOI 10.17487/RFC7426, January
1462 2015, .
1464 [TEAS] IETF, "Traffic Engineering Architecture and Signaling",
1465 .
1467 [WirelessHART]
1468 www.hartcomm.org, "Industrial Communication Networks -
1469 Wireless Communication Network and Communication Profiles
1470 - WirelessHART - IEC 62591", 2010.
1472 Authors' Addresses
1473 Norman Finn
1474 Self-employed
1475 1807 Santa Rita Rd
1476 Suite D, PMB 345
1477 Pleasanton, California 94566
1478 US
1480 Phone: +1 925 980 6430
1481 Email: nfinn@alumni.caltech.edu
1483 Pascal Thubert
1484 Cisco Systems
1485 Village d'Entreprises Green Side
1486 400, Avenue de Roumanille
1487 Batiment T3
1488 Biot - Sophia Antipolis 06410
1489 FRANCE
1491 Phone: +33 4 97 23 26 34
1492 Email: pthubert@cisco.com