idnits 2.17.1
draft-finn-detnet-architecture-07.txt:
Checking boilerplate required by RFC 5378 and the IETF Trust (see
https://trustee.ietf.org/license-info):
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt:
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/checklist :
----------------------------------------------------------------------------
No issues found here.
Miscellaneous warnings:
----------------------------------------------------------------------------
== The copyright year in the IETF Trust and authors Copyright Line does not
match the current year
== Line 637 has weird spacing: '...mediate inter...'
== Line 640 has weird spacing: '...mediate inter...'
== Line 836 has weird spacing: '...e stack v ...'
-- The document date (July 25, 2016) is 2824 days in the past. Is this
intentional?
Checking references for intended status: Proposed Standard
----------------------------------------------------------------------------
(See RFCs 3967 and 4897 for information about using normative references
to lower-maturity documents in RFCs)
== Unused Reference: 'AVnu' is defined on line 1233, but no explicit
reference was found in the text
== Unused Reference: 'HART' is defined on line 1242, but no explicit
reference was found in the text
== Unused Reference: 'I-D.ietf-6tisch-architecture' is defined on line
1261, but no explicit reference was found in the text
== Unused Reference: 'I-D.ietf-6tisch-tsch' is defined on line 1266, but no
explicit reference was found in the text
== Unused Reference: 'I-D.ietf-roll-rpl-industrial-applicability' is
defined on line 1284, but no explicit reference was found in the text
== Unused Reference: 'I-D.svshah-tsvwg-deterministic-forwarding' is defined
on line 1290, but no explicit reference was found in the text
== Unused Reference: 'IEEE802.11-2012' is defined on line 1295, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802.1AS-2011' is defined on line 1301, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802.1TSNTG' is defined on line 1345, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802154' is defined on line 1359, but no explicit
reference was found in the text
== Unused Reference: 'IEEE802154e' is defined on line 1365, but no explicit
reference was found in the text
== Unused Reference: 'ISA95' is defined on line 1377, but no explicit
reference was found in the text
== Unused Reference: 'ODVA' is defined on line 1381, but no explicit
reference was found in the text
== Unused Reference: 'Profinet' is defined on line 1388, but no explicit
reference was found in the text
== Unused Reference: 'RFC5673' is defined on line 1439, but no explicit
reference was found in the text
== Unused Reference: 'WirelessHART' is defined on line 1467, but no
explicit reference was found in the text
== Outdated reference: A later version (-04) exists of
draft-dt-detnet-dp-alt-01
== Outdated reference: A later version (-30) exists of
draft-ietf-6tisch-architecture-10
== Outdated reference: A later version (-09) exists of
draft-ietf-detnet-problem-statement-00
== Outdated reference: A later version (-20) exists of
draft-ietf-detnet-use-cases-10
-- Obsolete informational reference (is this intentional?): RFC 5316
(Obsoleted by RFC 9346)
Summary: 0 errors (**), 0 flaws (~~), 24 warnings (==), 2 comments (--).
Run idnits with the --verbose option for more detailed information about
the items above.
--------------------------------------------------------------------------------
2 DetNet N. Finn
3 Internet-Draft P. Thubert
4 Intended status: Standards Track Cisco
5 Expires: January 26, 2017 M. Johas Teener
6 Broadcom
7 July 25, 2016
9 Deterministic Networking Architecture
10 draft-finn-detnet-architecture-07
12 Abstract
14 Deterministic Networking (DetNet) provides a capability to carry
15 specified unicast or multicast data flows for real-time applications
16 with extremely low data loss rates and bounded latency. Techniques
17 used include: 1) reserving data plane resources for individual (or
18 aggregated) DetNet flows in some or all of the intermediate nodes
19 (e.g. bridges or routers) along the path of the flow; 2) providing
20 explicit routes for DetNet flows that do not rapidly change with the
21 network topology; and 3) distributing data from DetNet flow packets
22 over time and/or space to ensure delivery of each packet's data' in
23 spite of the loss of a path. The capabilities can be managed by
24 configuration, or by manual or automatic network management.
26 Status of This Memo
28 This Internet-Draft is submitted in full conformance with the
29 provisions of BCP 78 and BCP 79.
31 Internet-Drafts are working documents of the Internet Engineering
32 Task Force (IETF). Note that other groups may also distribute
33 working documents as Internet-Drafts. The list of current Internet-
34 Drafts is at http://datatracker.ietf.org/drafts/current/.
36 Internet-Drafts are draft documents valid for a maximum of six months
37 and may be updated, replaced, or obsoleted by other documents at any
38 time. It is inappropriate to use Internet-Drafts as reference
39 material or to cite them other than as "work in progress."
41 This Internet-Draft will expire on January 26, 2017.
43 Copyright Notice
45 Copyright (c) 2016 IETF Trust and the persons identified as the
46 document authors. All rights reserved.
48 This document is subject to BCP 78 and the IETF Trust's Legal
49 Provisions Relating to IETF Documents
50 (http://trustee.ietf.org/license-info) in effect on the date of
51 publication of this document. Please review these documents
52 carefully, as they describe your rights and restrictions with respect
53 to this document. Code Components extracted from this document must
54 include Simplified BSD License text as described in Section 4.e of
55 the Trust Legal Provisions and are provided without warranty as
56 described in the Simplified BSD License.
58 Table of Contents
60 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
61 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
62 2.1. Terms used in this document . . . . . . . . . . . . . . . 4
63 2.2. IEEE 802 TSN to DetNet dictionary . . . . . . . . . . . . 6
64 3. Providing the DetNet Quality of Service . . . . . . . . . . . 6
65 3.1. Congestion protection . . . . . . . . . . . . . . . . . . 8
66 3.2. Explicit routes . . . . . . . . . . . . . . . . . . . . . 9
67 3.3. Jitter Reduction . . . . . . . . . . . . . . . . . . . . 9
68 3.4. Packet Replication and Elimination . . . . . . . . . . . 10
69 3.5. Packet encoding for service protection . . . . . . . . . 12
70 4. DetNet Architecture . . . . . . . . . . . . . . . . . . . . . 12
71 4.1. Traffic Engineering for DetNet . . . . . . . . . . . . . 12
72 4.1.1. The Application Plane . . . . . . . . . . . . . . . . 12
73 4.1.2. The Controller Plane . . . . . . . . . . . . . . . . 13
74 4.1.3. The Network Plane . . . . . . . . . . . . . . . . . . 13
75 4.2. DetNet flows . . . . . . . . . . . . . . . . . . . . . . 14
76 4.2.1. Source guarantees . . . . . . . . . . . . . . . . . . 15
77 4.2.2. Incomplete Networks . . . . . . . . . . . . . . . . . 16
78 4.3. Queuing, Shaping, Scheduling, and Preemption . . . . . . 16
79 4.4. Coexistence with normal traffic . . . . . . . . . . . . . 17
80 4.5. Fault Mitigation . . . . . . . . . . . . . . . . . . . . 17
81 4.6. Representative Protocol Stack Model . . . . . . . . . . . 18
82 4.7. Exporting flow identification . . . . . . . . . . . . . . 20
83 4.8. Advertising resources, capabilities and adjacencies . . . 22
84 4.9. Provisioning model . . . . . . . . . . . . . . . . . . . 22
85 4.9.1. Centralized Path Computation and Installation . . . . 22
86 4.9.2. Distributed Path Setup . . . . . . . . . . . . . . . 23
87 4.10. Scaling to larger networks . . . . . . . . . . . . . . . 23
88 4.11. Connected islands vs. networks . . . . . . . . . . . . . 23
89 5. Compatibility with Layer-2 . . . . . . . . . . . . . . . . . 24
90 6. Open Questions . . . . . . . . . . . . . . . . . . . . . . . 24
91 6.1. Flat vs. hierarchical control . . . . . . . . . . . . . . 24
92 6.2. Peer-to-peer reservation protocol . . . . . . . . . . . . 25
93 6.3. Wireless media interactions . . . . . . . . . . . . . . . 25
94 7. Security Considerations . . . . . . . . . . . . . . . . . . . 25
95 8. Privacy Considerations . . . . . . . . . . . . . . . . . . . 26
96 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 26
97 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26
98 11. Access to IEEE 802.1 documents . . . . . . . . . . . . . . . 26
99 12. Informative References . . . . . . . . . . . . . . . . . . . 27
100 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 32
102 1. Introduction
104 Deterministic Networking (DetNet) is a service that can be offered by
105 a network to DetNet flows. DetNet provides these flows extremely low
106 packet loss rates and assured maximum end-to-end delivery latency.
107 This is accomplished by dedicating network resources such as link
108 bandwidth and buffer space to DetNet flows and/or classes of DetNet
109 flows, and by replicating packets along multiple paths. Unused
110 reserved resources are available to non-DetNet packets.
112 The Deterministic Networking Problem Statement
113 [I-D.ietf-detnet-problem-statement] introduces Deterministic
114 Networking, and Deterministic Networking Use Cases
115 [I-D.ietf-detnet-use-cases] summarizes the need for it. See
116 [I-D.dt-detnet-dp-alt] for a discussion of specific techniques that
117 can be used to identify DetNet Flows and assign them to specific
118 paths through a network.
120 A goal of DetNet is a converged network in all respects. That is,
121 the presence of DetNet flows does not preclude non-DetNet flows, and
122 the benefits offered DetNet flows should not, except in extreme
123 cases, prevent existing QoS mechanisms from operating in a normal
124 fashion, subject to the bandwidth required for the DetNet flows. A
125 single source-destination pair can trade both DetNet and non-DetNet
126 flows. End systems and applications need not instantiate special
127 interfaces for DetNet flows. Networks are not restricted to certain
128 topologies; connectivity is not restricted. Any application that
129 generates a data flow that can be usefully characterized as having a
130 maximum bandwidth should be able to take advantage of DetNet, as long
131 as the necessary resources can be reserved. Reservations can be made
132 by the application itself, via network management, by an applications
133 controller, or by other means.
135 Many applications of interest to Deterministic Networking require the
136 ability to synchronize the clocks in end systems to a sub-microsecond
137 accuracy. Some of the queue control techniques defined in
138 Section 4.3 also require time synchronization among relay and transit
139 nodes. The means used to achieve time synchronization are not
140 addressed in this document. DetNet should accommodate various
141 synchronization techniques and profiles that are defined elsewhere to
142 solve exchange time in different market segments.
144 The present document is an individual contribution, but it is
145 intended by the authors for adoption by the DetNet working group.
147 2. Terminology
149 2.1. Terms used in this document
151 The following special terms are used in this document in order to
152 avoid the assumption that a given element in the architecture does or
153 does not have Internet Protocol stack, functions as a router, bridge,
154 firewall, or otherwise plays a particular role at Layer-2 or higher.
156 destination
157 An end system capable of receiving a DetNet flow.
159 DetNet domain
160 The portion of a network that is DetNet aware. It includes
161 end systems and other DetNet nodes.
163 DetNet flow
164 A DetNet flow is a sequence of packets to which the DetNet
165 service is to be provided.
167 DetNet compound flow and DetNet member flow
168 A DetNet compound flow is a DetNet flow that has been
169 separated into multiple duplicate DetNet member flows, which
170 are eventually merged back into a single DetNet compound
171 flow, at the DetNet transport layer. "Compound" and "member"
172 are strictly relative to each other, not absolutes; a DetNet
173 compound flow comprising multiple DetNet member flows can, in
174 turn, be a member of a higher-order compound.
176 DetNet intermediate node
177 A DetNet relay node or transit node.
179 DetNet edge node
180 An instance of a DetNet relay node that includes either a
181 DetNet service layer proxy function for DetNet service
182 protection (e.g. the addition or removal of packet sequencing
183 information) for one or more end systems, or starts or
184 terminates congestion protection at the DetNet transport
185 layer, analogous to a Label Edge Router (LER).
187 end system
188 Commonly called a "host" or "node" in IETF documents, and an
189 "end station" is IEEE 802 documents. End systems of interest
190 to this document are either sources or destinations of DetNet
191 flows. And end system may or may not be DetNet transport
192 layer aware or DetNet service layer aware.
194 link
195 A connection between two DetNet nodes. It may be composed of
196 a physical link or a sub-network technology that can provide
197 appropriate traffic delivery for DetNet flows.
199 DetNet node
200 A DetNet aware end system, transit node, or relay node.
201 "DetNet" may be omitted in some text.
203 Detnet relay node
204 A DetNet node including a service layer function that
205 interconnects different DetNet transport layer paths to
206 provide service protection. A DetNet relay node can be a
207 bridge, a router, a firewall, or any other system that
208 participates in the DetNet service layer. It typically
209 incorporates DetNet transport layer functions as well, in
210 which case it is collocated with a transit node.
212 reservation
213 A trail of configuration between source to destination(s)
214 through transit nodes and subnets associated with a DetNet
215 flow, to provide congestion protection.
217 DetNet service layer
218 The layer at which service protection is provided, either
219 packet sequencing, replication, and elimination (Section 3.4)
220 or network coding (Section 3.5).
222 source
223 An end system capable of sourcing a DetNet flow.
225 DetNet transit node
226 A node operating at the DetNet transport layer, that utilizes
227 link layer and/or network layer switching across multiple
228 links and/or sub-networks to provide paths for DetNet service
229 layer functions. Optionally provides congestion protection
230 over those paths. An MPLS LSR is an example of a DetNet
231 transit node.
233 DetNet transport layer
234 The layer that optionally provides congestion protection for
235 DetNet flows over paths provided by the underlying network.
237 2.2. IEEE 802 TSN to DetNet dictionary
239 This section also serves as a dictionary for translating from the
240 terms used by the IEEE 802 Time-Sensitive Networking (TSN) Task Group
241 to those of the DetNet WG.
243 Listener
244 The IEEE 802 term for a destination of a DetNet flow.
246 relay system
247 The IEEE 802 term for a DetNet intermediate node.
249 Stream
250 The IEEE 802 term for a DetNet flow.
252 Talker
253 The IEEE 802 term for the source of a DetNet flow.
255 3. Providing the DetNet Quality of Service
257 The DetNet Quality of Service can be expressed in terms of:
259 o Minimum and maximum end-to-end latency from source to destination;
260 timely delivery and jitter avoidance derive from these constraints
262 o Probability of loss of a packet, under various assumptions as to
263 the operational states of the nodes and links. A derived property
264 is whether it is acceptable to deliver a duplicate packet, which
265 is an inherent risk in highly reliable and/or broadcast
266 transmissions
268 It is a distinction of DetNet that it is concerned solely with worst-
269 case values for the end-to-end latency. Average, mean, or typical
270 values are of no interest, because they do not affect the ability of
271 a real-time system to perform its tasks. In general, a trivial
272 priority-based queuing scheme will give better average latency to a
273 data flow than DetNet, but of course, the worst-case latency can be
274 essentially unbounded.
276 Three techniques are used by DetNet to provide these qualities of
277 service:
279 o Congestion protection (Section 3.1).
281 o Explicit routes (Section 3.2).
283 o Service protection.
285 Congestion protection operates by reserving resources along the path
286 of a DetNet Flow, e.g. buffer space or link bandwidth. Congestion
287 protection greatly reduces, or even eliminates entirely, packet loss
288 due to output packet congestion within the network, but it can only
289 be supplied to a DetNet flow that is limited at the source to a
290 maximum packet size and transmission rate.
292 Congestion protection addresses both of the DetNet QoS requirements
293 (latency and packet loss). Given that DetNet nodes have a finite
294 amount of buffer space, congestion protection necessarily results in
295 a maximum end-to-end latency. It also addresses the largest
296 contribution to packet loss, which is buffer congestion.
298 After congestion, the most important contributions to packet loss are
299 typically from random media errors and equipment failures. Service
300 protection is the name for the mechanisms used by DetNet to address
301 these losses. The mechanisms employed are constrained by the
302 requirement to meet the users' latency requirements. Packet
303 replication and elimination (Section 3.4) packet encoding Section 3.5
304 are described in this document to provide service protection; others
305 may be found. Both mechanisms distribute the contents of DetNet
306 flows over multiple paths in time and/or space, so that the loss of
307 some of the paths does need not cause the loss of any packets. The
308 paths are typically (but not necessarily) explicit routes, so that
309 they cannot suffer temporary interruptions caused by the
310 reconvergence of routing or bridging protocols.
312 These three techniques can be applied independently, giving eight
313 possible combinations, including none (no DetNet), although some
314 combinations are of wider utility than others. This separation keeps
315 the protocol stack coherent and maximizes interoperability with
316 existing and developing standards in this (IETF) and other Standards
317 Development Organizations. Some examples of typical expected
318 combinations:
320 o Explicit routes plus service protection are exactly the techniques
321 employed by [HSR-PRP]. Explicit routes are achieved by limiting
322 the physical topology of the network, and the sequentialization,
323 replication, and duplicate elimination are facilitated by packet
324 tags added at the front or the end of Ethernet frames.
326 o Congestion protection alone is is offered by IEEE 802.1 Audio
327 Video bridging [IEEE802.1BA-2011]. As long as the network suffers
328 no failures, zero congestion loss can be achieved through the use
329 of a reservation protocol (MSRP), shapers in every bridge, and a
330 bit of network calculus.
332 o Using all three together gives maximum protection.
334 There are, of course, simpler methods available (and employed, today)
335 to achieve levels of latency and packet loss that are satisfactory
336 for many applications. Prioritization and over-provisioning is one
337 such technique. However, these methods generally work best in the
338 absence of any significant amount of non-critical traffic in the
339 network (if, indeed, such traffic is supported at all), or work only
340 if the critical traffic constitutes only a small portion of the
341 network's theoretical capacity, or work only if all systems are
342 functioning properly, or in the absence of actions by end systems
343 that disrupt the network's operations.
345 There are any number of methods in use, defined, or in progress for
346 accomplishing each of the above techniques. It is expected that this
347 DetNet Architecture will assist various vendors, users, and/or
348 "vertical" Standards Development Organizations (dedicated to a single
349 industry) to make selections among the available means of
350 implementing DetNet networks.
352 3.1. Congestion protection
354 The primary means by which DetNet achieves its QoS assurances is to
355 reduce, or even completely eliminate, congestion at an output port as
356 a cause of packet loss. Given that a DetNet flow cannot be
357 throttled, this can be achieved only by the provision of sufficient
358 buffer storage at each hop through the network to ensure that no
359 packets are dropped due to a lack of buffer storage.
361 Ensuring adequate buffering requires, in turn, that the source, and
362 every intermediate node along the path to the destination (or nearly
363 every node -- see Section 4.2.2) be careful to regulate its output to
364 not exceed the data rate for any DetNet flow, except for brief
365 periods when making up for interfering traffic. Any packet sent
366 ahead of its time potentially adds to the number of buffers required
367 by the next hop, and may thus exceed the resources allocated for a
368 particular DetNet flow.
370 The low-level mechanisms described in Section 4.3 provide the
371 necessary regulation of transmissions by an end system or
372 intermediate node to provide congestion protection. The reservation
373 of the bandwidth and buffers for a DetNet flow requires the
374 provisioning described in Section 4.9. A DetNet node may have other
375 resources requiring allocation and/or scheduling, that might
376 otherwise be over-subscribed and trigger the rejection of a
377 reservation.
379 3.2. Explicit routes
381 In networks controlled by typical peer-to-peer protocols such as IEEE
382 802.1 ISIS bridged networks or IETF OSPF routed networks, a network
383 topology event in one part of the network can impact, at least
384 briefly, the delivery of data in parts of the network remote from the
385 failure or recovery event. Thus, even redundant paths through a
386 network, if controlled by the typical peer-to-peer protocols, do not
387 eliminate the chances of brief losses of contact.
389 Many real-time networks rely on physical rings or chains of two-port
390 devices, with a relatively simple ring control protocol. This
391 supports redundant paths for service protection with a minimum of
392 wiring. As an additional benefit, ring topologies can often utilize
393 different topology management protocols than those used for a mesh
394 network, with a consequent reduction in the response time to topology
395 changes. Of course, this comes at some cost in terms of increased
396 hop count, and thus latency, for the typical path.
398 In order to get the advantages of low hop count and still ensure
399 against even very brief losses of connectivity, DetNet employs
400 explicit routes, where the path taken by a given DetNet flow does not
401 change, at least immediately, and likely not at all, in response to
402 network topology events. Service protection (Section 3.4 or
403 Section 3.5) over explicit routes provides a high likelihood of
404 continuous connectivity. Explicit routes are commonly used in MPLS
405 TE LSPs.
407 3.3. Jitter Reduction
409 A core objective of DetNet is to enable the convergence of Non-IP
410 networks onto a common network infrastructure. This requires the
411 accurate emulation of currently deployed mission-specific networks,
412 which typically rely on point-to-point analog (e.g. 4-20mA
413 modulation) and serial-digital cables (or buses) for highly reliable,
414 synchronized and jitter-free communications. While the latency of
415 analog transmissions is basically the speed of light, legacy serial
416 links are usually slow (in the order of Kbps) compared to, say, GigE,
417 and some latency is usually acceptable. What is not acceptable is
418 the introduction of excessive jitter, which may, for instance, affect
419 the stability of control systems.
421 Applications that are designed to operate on serial links usually do
422 not provide services to recover the jitter, because jitter simply
423 does not exists there. Streams of information are expected to be
424 delivered in-order and the precise time of reception influences the
425 processes. In order to converge such existing applications, there is
426 a desire to emulate all properties of the serial cable, such as clock
427 transportation, perfect flow isolation and fixed latency. While
428 minimal jitter (in the form of specifying minimum, as well as
429 maximum, end-to-end latency) is supported by DetNet, there are
430 practical limitations on packet-based networks in this regard. In
431 general, users are encouraged to use, instead of, "do this when you
432 get the packet," a combination of:
434 o Sub-microsecond time synchronization among all source and
435 destination end systems, and
437 o Time-of-execution fields in the application packets.
439 Jitter reduction is provided by the mechanisms described in
440 Section 4.3 that also provide congestion protection.
442 3.4. Packet Replication and Elimination
444 After congestion loss has been eliminated, the most important causes
445 of packet loss are random media and/or memory faults, and equipment
446 failures. Both causes of packet loss can be greatly reduced by
447 spreading the data in a packet over multiple transmissions. One such
448 method for service protection is described in this section, which
449 sends the same packets over multiple paths. See also Section 3.5.
451 Packet replication and elimination, also known as seamless redundancy
452 [HSR-PRP], or 1+1 hitless protection, is a function of the DetNet
453 service layer. It involves three capabilities:
455 o Providing sequencing information, once, at or near the source, to
456 the packets of a DetNet compound flow. This may be done by adding
457 a sequence number or time stamp as part of DetNet, or may be
458 inherent in the packet, e.g. in a transport protocol, or
459 associated to other physical properties such as the precise time
460 (and radio channel) of reception of the packet. Section 3.2.
462 o Replicating these packets into multiple DetNet member flows and,
463 typically, sending them along at least two different paths to the
464 destination(s), e.g. over the explicit routes of
466 o Eliminating duplicated packets. This may be done at any step
467 along the path to save network resources further down, in
468 particular if multiple Replication points exist. But the most
469 common case is to perform this operation at the very edge of the
470 DetNet network, preferably in or near the receiver.
472 This function is a "hitless" version of, e.g., the 1+1 linear
473 protection in [RFC6372]. That is, instead of switching from one flow
474 to the other when a failure of a flow is detected, DetNet combines
475 both flows, and performs a packet-by-packet selection of which to
476 discard, based on sequence number.
478 In the simplest case, this amounts to replicating each packet in a
479 source that has two interfaces, and conveying them through the
480 network, along separate paths, to the similarly dual-homed
481 destinations, that discard the extras. This ensures that one path
482 (with zero congestion loss) remains, even if some intermediate node
483 fails. The sequence numbers can also be used for loss detection and
484 for re-ordering.
486 Detnet relay nodes in the network can provide replication and
487 elimination facilities at various points in the network, so that
488 multiple failures can be accommodated.
490 This is shown in the following figure, where the two relay nodes each
491 replicate (R) the DetNet flow on input, sending the DetNet member
492 flows to both the other relay node and to the end system, and
493 eliminate duplicates (E) on the output interface to the right-hand
494 end system. Any one link in the network can fail, and the Detnet
495 compound flow can still get through. Furthermore, two links can
496 fail, as long as they are in different segments of the network.
498 Packet replication and elimination
500 > > > > > > > > > relay > > > > > > > >
501 > /------------+ R node E +------------\ >
502 > / v + ^ \ >
503 end R + v | ^ + E end
504 system + v | ^ + system
505 > \ v + ^ / >
506 > \------------+ R relay E +-----------/ >
507 > > > > > > > > > node > > > > > > > >
509 Figure 1
511 Note that packet replication and elimination does not react to and
512 correct failures; it is entirely passive. Thus, intermittent
513 failures, mistakenly created packet filters, or misrouted data is
514 handled just the same as the equipment failures that are detected
515 handled by typical routing and bridging protocols.
517 If packet replication and elimination is used over paths providing
518 congestion protection (Section 3.1), and member flows that take
519 different-length paths through the network are combined, a merge
520 point may require extra buffering to equalize the delays over the
521 different paths. This equalization ensures that the resultant
522 compound flow will not exceed its contracted bandwidth even after one
523 or the other of the paths is restored after a failure.
525 3.5. Packet encoding for service protection
527 There are methods for using multiple paths to provide service
528 protection that involve encoding the information in a packet
529 belonging to a DetNet flow into multiple transmission units,
530 typically combining information from multiple packets into any given
531 transmission unit. Such techniques may be applicable for use as a
532 DetNet service protection technique, assuming that the DetNet users'
533 needs for timeliness of delivery and freedom from interference with
534 misbehaving DetNet flows can be met.
536 No specific mechanisms are defined here, at this time. This section
537 will either be enhanced or removed. Contributions are invited.
539 4. DetNet Architecture
541 4.1. Traffic Engineering for DetNet
543 Traffic Engineering Architecture and Signaling (TEAS) [TEAS] defines
544 traffic-engineering architectures for generic applicability across
545 packet and non-packet networks. From TEAS perspective, Traffic
546 Engineering (TE) refers to techniques that enable operators to
547 control how specific traffic flows are treated within their networks.
549 Because if its very nature of establishing explicit optimized paths,
550 Deterministic Networking can be seen as a new, specialized branch of
551 Traffic Engineering, and inherits its architecture with a separation
552 into planes.
554 The Deterministic Networking architecture is thus composed of three
555 planes, a (User) Application Plane, a Controller Plane, and a Network
556 Plane, which echoes that of Figure 1 of Software-Defined Networking
557 (SDN): Layers and Architecture Terminology [RFC7426].:
559 4.1.1. The Application Plane
561 Per [RFC7426], the Application Plane includes both applications and
562 services. In particular, the Application Plane incorporates the User
563 Agent, a specialized application that interacts with the end user /
564 operator and performs requests for Deterministic Networking services
565 via an abstract Flow Management Entity, (FME) which may or may not be
566 collocated with (one of) the end systems.
568 At the Application Plane, a management interface enables the
569 negotiation of flows between end systems. An abstraction of the flow
570 called a Traffic Specification (TSpec) provides the representation.
571 This abstraction is used to place a reservation over the (Northbound)
572 Service Interface and within the Application plane. It is associated
573 with an abstraction of location, such as IP addresses and DNS names,
574 to identify the end systems and eventually specify intermediate
575 nodes.
577 4.1.2. The Controller Plane
579 The Controller Plane corresponds to the aggregation of the Control
580 and Management Planes in [RFC7426], though Common Control and
581 Measurement Plane (CCAMP) [CCAMP] makes an additional distinction
582 between management and measurement. When the logical separation of
583 the Control, Measurement and other Management entities is not
584 relevant, the term Controller Plane is used for simplicity to
585 represent them all, and the term controller refers to any device
586 operating in that plane, whether is it a Path Computation entity or a
587 Network Management entity (NME). The Path Computation Element (PCE)
588 [PCE] is a core element of a controller, in charge of computing
589 Deterministic paths to be applied in the Network Plane.
591 A (Northbound) Service Interface enables applications in the
592 Application Plane to communicate with the entities in the Controller
593 Plane.
595 One or more PCE(s) collaborate to implement the requests from the FME
596 as Per-Flow Per-Hop Behaviors installed in the intermediate nodes for
597 each individual flow. The PCEs place each flow along a deterministic
598 sequence of intermediate nodes so as to respect per-flow constraints
599 such as security and latency, and optimize the overall result for
600 metrics such as an abstract aggregated cost. The deterministic
601 sequence can typically be more complex than a direct sequence and
602 include redundancy path, with one or more packet replication and
603 elimination points.
605 4.1.3. The Network Plane
607 The Network Plane represents the network devices and protocols as a
608 whole, regardless of the Layer at which the network devices operate.
609 It includes Forwarding Plane (data plane), Application, and
610 Operational Plane (control plane) aspects.
612 The network Plane comprises the Network Interface Cards (NIC) in the
613 end systems, which are typically IP hosts, and intermediate nodes,
614 which are typically IP routers and switches. Network-to-Network
615 Interfaces such as used for Traffic Engineering path reservation in
616 [RFC5921], as well as User-to-Network Interfaces (UNI) such as
617 provided by the Local Management Interface (LMI) between network and
618 end systems, are both part of the Network Plane, both in the control
619 plane and the data plane.
621 A Southbound (Network) Interface enables the entities in the
622 Controller Plane to communicate with devices in the Network Plane.
623 This interface leverages and extends TEAS to describe the physical
624 topology and resources in the Network Plane.
626 Flow Management Entity
628 End End
629 System System
631 -+-+-+-+-+-+-+ Northbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
633 PCE PCE PCE PCE
635 -+-+-+-+-+-+-+ Southbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
637 intermediate intermed. intermed. intermed.
638 Node Node Node Node
639 NIC NIC
640 intermediate intermed. intermed. intermed.
641 Node Node Node Node
643 Figure 2
645 The intermediate nodes (and eventually the end systems NIC) expose
646 their capabilities and physical resources to the controller (the
647 PCE), and update the PCE with their dynamic perception of the
648 topology, across the Southbound Interface. In return, the PCE(s) set
649 the per-flow paths up, providing a Flow Characterization that is more
650 tightly coupled to the intermediate node Operation than a TSpec.
652 At the Network plane, intermediate nodes may exchange information
653 regarding the state of the paths, between adjacent systems and
654 eventually with the end systems, and forward packets within
655 constraints associated to each flow, or, when unable to do so,
656 perform a last resort operation such as drop or declassify.
658 This specification focuses on the Southbound interface and the
659 operation of the Network Plane.
661 4.2. DetNet flows
662 4.2.1. Source guarantees
664 For the purposes of congestion protection, DetNet flows can be
665 synchronous or asynchronous. In synchronous DetNet flows, at least
666 the intermediate nodes (and possibly the end systems) are closely
667 time synchronized, typically to better than 1 microsecond. By
668 transmitting packets from different DetNet flows or classes of DetNet
669 flows at different times, using repeating schedules synchronized
670 among the intermediate nodes, resources such as buffers and link
671 bandwidth can be shared over the time domain among different DetNet
672 flows. There is a tradeoff among techniques for synchronous DetNet
673 flows between the burden of fine-grained scheduling and the benefit
674 of reducing the required resources, especially buffer space.
676 In contrast, asynchronous DetNet flows are not coordinated with a
677 fine-grained schedule, so relay and end systems must assume worst-
678 case interference among DetNet flows contending for buffer resources.
679 Asynchronous DetNet flows are characterized by:
681 o A maximum packet size;
683 o An observation interval; and
685 o A maximum number of transmissions during that observation
686 interval.
688 These parameters, together with knowledge of the protocol stack used
689 (and thus the size of the various headers added to a packet), limit
690 the number of bit times per observation interval that the DetNet flow
691 can occupy the physical medium.
693 The source promises that these limits will not be exceeded. If the
694 source transmits less data than this limit allows, the unused
695 resources such as link bandwidth can be made available by the system
696 to non-DetNet packets. However, making those resources available to
697 DetNet packets in other DetNet flows would serve no purpose. Those
698 other DetNet flows have their own dedicated resources, on the
699 assumption that all DetNet flows can use all of their resources over
700 a long period of time.
702 Note that there is no provision in DetNet for throttling DetNet flows
703 (reducing the transmission rate via feedback); the assumption is that
704 a DetNet flow, to be useful, must be delivered in its entirety. That
705 is, while any useful application is written to expect a certain
706 number of lost packets, the real-time applications of interest to
707 DetNet demand that the loss of data due to the network is
708 extraordinarily infrequent.
710 Although DetNet strives to minimize the changes required of an
711 application to allow it to shift from a special-purpose digital
712 network to an Internet Protocol network, one fundamental shift in the
713 behavior of network applications is impossible to avoid: the
714 reservation of resources before the application starts. In the first
715 place, a network cannot deliver finite latency and practically zero
716 packet loss to an arbitrarily high offered load. Secondly, achieving
717 practically zero packet loss for unthrottled (though bandwidth
718 limited) DetNet flows means that bridges and routers have to dedicate
719 buffer resources to specific DetNet flows or to classes of DetNet
720 flows. The requirements of each reservation have to be translated
721 into the parameters that control each system's queuing, shaping, and
722 scheduling functions and delivered to the hosts, bridges, and
723 routers.
725 4.2.2. Incomplete Networks
727 The presence in the network of transit nodes or subnets that are not
728 fully capable of offering DetNet services complicates the ability of
729 the intermediate nodes and/or controller to allocate resources, as
730 extra buffering, and thus extra latency, must be allocated at points
731 downstream from the non-DetNet intermediate node for a DetNet flow.
733 4.3. Queuing, Shaping, Scheduling, and Preemption
735 DetNet achieves congestion protection and bounded delivery latency by
736 reserving bandwidth and buffer resources at every hop along the path
737 of the DetNet flow. The reservation itself is not sufficient,
738 however. Implementors and users of a number of proprietary and
739 standard real-time networks have found that standards for specific
740 data plane techniques are required to enable these assurances to be
741 made in a multi-vendor network. The fundamental reason is that
742 latency variation in one system results in the need for extra buffer
743 space in the next-hop system(s), which in turn, increases the worst-
744 case per-hop latency.
746 Standard queuing and transmission selection algorithms allow a
747 central controller to compute the latency contribution of each
748 transit node to the end-to-end latency, to compute the amount of
749 buffer space required in each transit node for each incremental
750 DetNet flow, and most importantly, to translate from a flow
751 specification to a set of values for the managed objects that control
752 each relay or end system. The IEEE 802 has specified (and is
753 specifying) a set of queuing, shaping, and scheduling algorithms that
754 enable each transit node (bridge or router), and/or a central
755 controller, to compute these values. These algorithms include:
757 o A credit-based shaper [IEEE802.1Q-2014] Clause 34.
759 o Time-gated queues governed by a rotating time schedule,
760 synchronized among all transit nodes [IEEE802.1Qbv].
762 o Synchronized double (or triple) buffers driven by synchronized
763 time ticks. [IEEE802.1Qch].
765 o Pre-emption of an Ethernet packet in transmission by a packet with
766 a more stringent latency requirement, followed by the resumption
767 of the preempted packet [IEEE802.1Qbu], [IEEE802.3br].
769 While these techniques are currently embedded in Ethernet and
770 bridging standards, we can note that they are all, except perhaps for
771 packet preemption, equally applicable to other media than Ethernet,
772 and to routers as well as bridges.
774 4.4. Coexistence with normal traffic
776 A DetNet network supports the dedication of a high proportion (e.g.
777 75%) of the network bandwidth to DetNet flows. But, no matter how
778 much is dedicated for DetNet flows, it is a goal of DetNet to coexist
779 with existing Class of Service schemes (e.g., DiffServ). It is also
780 important that non-DetNet traffic not disrupt the DetNet flow, of
781 course (see Section 4.5 and Section 7). For these reasons:
783 o Bandwidth (transmission opportunities) not utilized by a DetNet
784 flow are available to non-DetNet packets (though not to other
785 DetNet flows).
787 o DetNet flows can be shaped or scheduled, in order to ensure that
788 the highest-priority non-DetNet packet also is ensured a worst-
789 case latency (at any given hop).
791 o When transmission opportunities for DetNet flows are scheduled in
792 detail, then the algorithm constructing the schedule should leave
793 sufficient opportunities for non-DetNet packets to satisfy the
794 needs of the users of the network. Detailed scheduling can also
795 permit the time-shared use of buffer resources by different DetNet
796 flows.
798 Ideally, the net effect of the presence of DetNet flows in a network
799 on the non-DetNet packets is primarily a reduction in the available
800 bandwidth.
802 4.5. Fault Mitigation
804 One key to building robust real-time systems is to reduce the
805 infinite variety of possible failures to a number that can be
806 analyzed with reasonable confidence. DetNet aids in the process by
807 providing filters and policers to detect DetNet packets received on
808 the wrong interface, or at the wrong time, or in too great a volume,
809 and to then take actions such as discarding the offending packet,
810 shutting down the offending DetNet flow, or shutting down the
811 offending interface.
813 It is also essential that filters and service remarking be employed
814 at the network edge to prevent non-DetNet packets from being mistaken
815 for DetNet packets, and thus impinging on the resources allocated to
816 DetNet packets.
818 There exist techniques, at present and/or in various stages of
819 standardization, that can perform these fault mitigation tasks that
820 deliver a high probability that misbehaving systems will have zero
821 impact on well-behaved DetNet flows, except of course, for the
822 receiving interface(s) immediately downstream of the misbehaving
823 device. Examples of such techniques include traffic policing
824 functions (e.g. [RFC2475]) and separating flows into per-flow rate-
825 limited queues.
827 4.6. Representative Protocol Stack Model
829 Figure 3 illustrates a conceptual DetNet data plane layering model.
830 One may compare it to that in [IEEE802.1CB], Annex C, a work in
831 progress.
833 DetNet data plane protocol stack
835 | packets going | ^ packets coming ^
836 v down the stack v | up the stack |
837 +----------------------+ +-----------------------+
838 | Source | | Destination |
839 +----------------------+ +-----------------------+
840 | Service layer | | Service layer |
841 | Packet sequencing | | Duplicate elimination |
842 | Flow duplication | | Flow merging |
843 | Packet encoding | | Packet decoding |
844 +----------------------+ +-----------------------+
845 | Transport layer | | Transport layer |
846 | Congestion prot. | | Congestion prot. |
847 +----------------------+ +-----------------------+
848 | Lower layers | | Lower layers |
849 +----------------------+ +-----------------------+
850 v ^
851 \_________________________/
853 Figure 3
855 Not all layers are required for any given application, or even for
856 any given network. The layers are, from top to bottom:
858 Application
859 Shown as "source" and "destination" in the diagram.
861 OAM
862 Operations, Administration, and Maintenance leverages in-band
863 and out-of-and signaling that validates whether the service
864 is effectively obtained within QoS constraints. OAM is not
865 shown in Figure 3; it may reside in any number of the layers.
866 OAM can involve specific tagging added in the packets for
867 tracing implementation or network configuration errors;
868 traceability enables to find whether a packet is a replica,
869 which relay node performed the replication, and which segment
870 was intended for the replica.
872 Packet sequencing
873 As part of DetNet service protection, supplies the sequence
874 number for packet replication and elimination (Section 3.4).
875 Peers with Duplicate elimination. This layer is not needed
876 if a higher-layer transport protocol is expected to perform
877 any packet sequencing and duplicate elimination required by
878 the DetNet flow duplication.
880 Duplicate elimination
881 As part of the DetNet service layer, based on the sequenced
882 number supplied by its peer, packet sequencing, Duplicate
883 elimination discards any duplicate packets generated by
884 DetNet flow duplication. It can operate on member flows,
885 compound flows, or both. The duplication may also be
886 inferred from other information such as the precise time of
887 reception in a scheduled network. The duplicate elimination
888 layer may also perform resequencing of packets to restore
889 packet order in a flow that was disrupted by the loss of
890 packets on one or another of the multiple paths taken.
892 Flow duplication
893 As part of DetNet service protection, replicates packets that
894 belong to a DetNet compound flow into two or more DetNet
895 member flows. Note that this function is separate from
896 packet sequencing. Flow duplication can be an explicit
897 duplication and remarking of packets, or can be performed by,
898 for example, techniques similar to ordinary multicast
899 replication. Peers with DetNet flow merging.
901 Network flow merging
902 As part of DetNet service protection, merges DetNet member
903 flows together for packets coming up the stack belonging to a
904 specific DetNet compound flow. Peers with DetNet flow
905 duplication. DetNet flow merging, together with packet
906 sequencing, duplicate elimination, and DetNet flow
907 duplication, performs packet replication and elimination
908 (Section 3.4).
910 Packet encoding
911 As part of DetNet service protection, as an alternative to
912 packet sequencing and flow duplication, packet encoding
913 combines the information in multiple DetNet packets, perhaps
914 from different DetNet compound flows, and transmits that
915 information in packets on different DetNet member Flows.
916 Peers with Packet decoding.
918 Packet decoding
919 As part of DetNet service protection, as an alternative to
920 flow merging and duplicate elimination, packet decoding takes
921 packets from different DetNet member flows, and computes from
922 those packets the original DetNet packets from the compound
923 flows input to packet encoding. Peers with Packet encoding.
925 Congestio protection
926 The DetNet transport layer provides congestion protection.
927 See Section 4.3. The actual queuing and shaping mechaniss
928 are typically provided by underlying subnet layers, but since
929 these are can be closely associated with the means of
930 providing paths for DetNet flows (e.g. MPLS LSPs or {VLAN,
931 multicast destination MAC address} pairs), the path and the
932 congestion protection are conflated in this figure.
934 Note that the packet sequencing and duplication elimination functions
935 at the source and destination ends of a DetNet compound flow may be
936 performed either in the end system or in a DetNet edge node. The
937 reader must not confuse a DetNet edge function with other kinds of
938 edge functions, e.g. an Label Edge Router, although the two functions
939 may be performed together. The DetNet edge function is concerned
940 with sequencing packets belonging to DetNet flows. The LER with
941 encapsulating/decapsulating packets for transport, and is considered
942 part of the network underlying the DetNet transport layer.
944 4.7. Exporting flow identification
946 An interesting feature of DetNet, and one that invites
947 implementations that can be accused of "layering violations", is the
948 need for lower layers to be aware of specific flows at higher layers,
949 in order to provide specific queuing and shaping services for
950 specific flows. For example:
952 o A non-IP, strictly L2 source end system X may be sending multiple
953 flows to the same L2 destination end system Y. Those flows may
954 include DetNet flows with different QoS requirements, and may
955 include non-DetNet flows.
957 o A router may be sending any number of flows to another router.
958 Again, those flows may include DetNet flows with different QoS
959 requirements, and may include non-DetNet flows.
961 o Two routers may be separated by bridges. For these bridges to
962 perform any required per-flow queuing and shaping, they must be
963 able to identify the individual flows.
965 o A Label Edge Router (LERs) may have a Label Switched Path (LSP)
966 set up for handling traffic destined for a particular IP address
967 carrying only non-DetNet flows. If a DetNet flow to that same
968 address is requested, a separate LSP may be needed, in order that
969 all of the Label Switch Routers (LSRs) along the path to the
970 destination give that flow special queuing and shaping.
972 The need for a lower-level DetNet node to be aware of individual
973 higher-layer flows is not unique to DetNet. But, given the endless
974 complexity of layering and relayering over tunnels that is available
975 to network designers, DetNet needs to provide a model for flow
976 identification that is at least somewhat better than packet
977 inspection. That is not to say that packet inspection to layer 4 or
978 5 addresses will not be used, or the capability standardized; but,
979 there are alternatives.
981 A DetNet relay node can connect DetNet flows on different paths using
982 different flow identification methods. For example:
984 o A single unicast DetNet flow passing from router A through a
985 bridged network to router B may be assigned a {VLAN, multicast
986 destination MAC address} pair that is unique within that bridged
987 network. The bridges can then identify the flow without accessing
988 higher-layer headers. Of course, the receiving router must
989 recognize and accept that multicast MAC address.
991 o A DetNet flow passing from LSR A to LSR B may be assigned a
992 different label than that used for other flows to the same IP
993 destination.
995 In any of the above cases, it is possible that an existing DetNet
996 flow can be used as a carrier for multiple DetNet sub-flows. (Not to
997 be confused with DetNet compound vs. member flows.) Of course, this
998 requires that the aggregate DetNet flow be provisioned properly to
999 carry the sub-flows.
1001 Thus, rather than packet inspection, there is the option to export
1002 higher-layer information to the lower layer. The requirement to
1003 support one or the other method for flow identification (or both) is
1004 the essential complexity that DetNet brings to existing control plane
1005 models.
1007 4.8. Advertising resources, capabilities and adjacencies
1009 There are three classes of information that a central controller or
1010 decentralized control plane needs to know that can only be obtained
1011 from the end systems and/or transit nodes in the network. When using
1012 a peer-to-peer control plane, some of this information may be
1013 required by a system's neighbors in the network.
1015 o Details of the system's capabilities that are required in order to
1016 accurately allocate that system's resources, as well as other
1017 systems' resources. This includes, for example, which specific
1018 queuing and shaping algorithms are implemented (Section 4.3), the
1019 number of buffers dedicated for DetNet allocation, and the worst-
1020 case forwarding delay.
1022 o The dynamic state of an end or transit node's DetNet resources.
1024 o The identity of the system's neighbors, and the characteristics of
1025 the link(s) between the systems, including the length (in
1026 nanoseconds) of the link(s).
1028 4.9. Provisioning model
1030 4.9.1. Centralized Path Computation and Installation
1032 A centralized routing model, such as provided with a PCE (RFC 4655
1033 [RFC4655]), enables global and per-flow optimizations. (See
1034 Section 4.1.) The model is attractive but a number of issues are
1035 left to be solved. In particular:
1037 o Whether and how the path computation can be installed by 1) an end
1038 device or 2) a Network Management entity,
1040 o And how the path is set up, either by installing state at each hop
1041 with a direct interaction between the forwarding device and the
1042 PCE, or along a path by injecting a source-routed request at one
1043 end of the path.
1045 4.9.2. Distributed Path Setup
1047 Significant work on distributed path setup can be leveraged from MPLS
1048 Traffic Engineering, in both its GMPLS and non-GMPLS forms. The
1049 protocols within scope are Resource ReSerVation Protocol [RFC3209]
1050 [RFC3473](RSVP-TE), OSPF-TE [RFC4203] [RFC5392] and ISIS-TE [RFC5307]
1051 [RFC5316]. These should be viewed as starting points as there are
1052 feature specific extensions defined that may be applicable to DetNet.
1054 In a Layer-2 only environment, or as part of a layered approach to a
1055 mixed environment, IEEE 802.1 also has work, either completed or in
1056 progress. [IEEE802.1Q-2014] Clause 35 describes SRP, a peer-to-peer
1057 protocol for Layer-2 roughly analogous to RSVP [RFC2205].
1058 [IEEE802.1Qca] defines how ISIS can provide multiple disjoint paths
1059 or distribution trees. Also in progress is [IEEE802.1Qcc], which
1060 expands the capabilities of SRP.
1062 The integration/interaction of the DetNet control layer with an
1063 underlying IEEE 802.1 sub-network control layer will need to be
1064 defined.
1066 4.10. Scaling to larger networks
1068 Reservations for individual DetNet flows require considerable state
1069 information in each transit node, especially when adequate fault
1070 mitigation (Section 4.5) is required. The DetNet data plane, in
1071 order to support larger numbers of DetNet flows, must support the
1072 aggregation of DetNet flows into tunnels, which themselves can be
1073 viewed by the transit nodes' data planes largely as individual DetNet
1074 flows. Without such aggregation, the per-relay system may limit the
1075 scale of DetNet networks.
1077 4.11. Connected islands vs. networks
1079 Given that users have deployed examples of the IEEE 802.1 TSN TG
1080 standards, which provide capabilities similar to DetNet, it is
1081 obvious to ask whether the IETF DetNet effort can be limited to
1082 providing Layer-2 connections (VPNs) between islands of bridged TSN
1083 networks. While this capability is certainly useful to some
1084 applications, and must not be precluded by DetNet, tunneling alone is
1085 not a sufficient goal for the DetNet WG. As shown in the
1086 Deterministic Networking Use Cases draft [I-D.ietf-detnet-use-cases],
1087 there are already deployments of Layer-2 TSN networks that are
1088 encountering the well-known problems of over-large broadcast domains.
1089 Routed solutions, and combinations routed/bridged solutions, are both
1090 required.
1092 5. Compatibility with Layer-2
1094 Standards providing similar capabilities for bridged networks (only)
1095 have been and are being generated in the IEEE 802 LAN/MAN Standards
1096 Committee. The present architecture describes an abstract model that
1097 can be applicable both at Layer-2 and Layer-3, and over links not
1098 defined by IEEE 802. It is the intention of the authors (and
1099 hopefully, as this draft progresses, of the DetNet Working Group)
1100 that IETF and IEEE 802 will coordinate their work, via the
1101 participation of common individuals, liaisons, and other means, to
1102 maximize the compatibility of their outputs.
1104 DetNet enabled end systems and intermediate nodes can be
1105 interconnected by sub-networks, i.e., Layer-2 technologies. These
1106 sub-networks will provide DetNet compatible service for support of
1107 DetNet traffic. Examples of sub-networks include 802.1TSN and a
1108 point-to-point OTN link. Of course, multi-layer DetNet systems may
1109 be possible too, where one DetNet appears as a sub-network, and
1110 provides service to, a higher layer DetNet system.
1112 6. Open Questions
1114 There are a number of architectural questions that will have to be
1115 resolved before this document can be submitted for publication.
1116 Aside from the obvious fact that this present draft is subject to
1117 change, there are specific questions to which the authors wish to
1118 direct the readers' attention.
1120 6.1. Flat vs. hierarchical control
1122 Boxes that are solely routers or solely bridges are rare in today's
1123 market. In a multi-tenant data center, multiple users' virtual
1124 Layer-2/Layer-3 topologies exist simultaneously, implemented on a
1125 network whose physical topology bears only accidental resemblance to
1126 the virtual topologies.
1128 While the forwarding topology (the bridges and routers) are an
1129 important consideration for a DetNet Flow Management Entity
1130 (Section 4.1.1), so is the purely physical topology. Ultimately, the
1131 model used by the management entities is based on boxes, queues, and
1132 links. The authors hope that the work of the TEAS WG will help to
1133 clarify exactly what model parameters need to be traded between the
1134 intermediate nodes and the controller(s).
1136 6.2. Peer-to-peer reservation protocol
1138 As described in Section 4.9.2, the DetNet WG needs to decide whether
1139 to support a peer-to-peer protocol for a source and a destination to
1140 reserve resources for a DetNet stream. Assuming that enabling the
1141 involvement of the source and/or destination is desirable (see
1142 Deterministic Networking Use Cases [I-D.ietf-detnet-use-cases]), it
1143 remains to decide whether the DetNet WG will make it possible to
1144 deploy at least some DetNet capabilities in a network using only a
1145 peer-to-peer protocol, without a central controller.
1147 (Note that a UNI (see Section 4.1.3) between an end system and a
1148 DetNet edge node, for sources and/or listeners to request DetNet
1149 services, can be either the first hop of a per-to-peer reservation
1150 protocol, or can be deflected by the DetNet edge node to a central
1151 controller for resolution. Similarly, a decision by a central
1152 controller can be effected by the controller instructing the end
1153 system or DetNet edge node to initiate a per-to-peer protocol
1154 activity.)
1156 6.3. Wireless media interactions
1158 Deterministic Networking Use Cases [I-D.ietf-detnet-use-cases]
1159 illustrates cases where wireless media are needed in a DetNet
1160 network. Some wireless media in general use, such as IEEE 802.11
1161 [IEEE802.1Q-2014], have significantly higher packet loss rates than
1162 typical wired media, such as Ethernet [IEEE802.3-2012]. IEEE 802.11
1163 includes support for such features as MAC-layer acknowledgements and
1164 retransmissions.
1166 The techniques described in Section 3 are likely to improve the
1167 ability of a mixed wired/wireless network to offer the DetNet QoS
1168 features. The interaction of these techniques with the features of
1169 specific wireless media, although they may be significant, cannot be
1170 addressed in this document. It remains to be decided to what extent
1171 the DetNet WG will address them, and to what extent other WGs, e.g.
1172 6TiSCH, will do so.
1174 7. Security Considerations
1176 Security in the context of Deterministic Networking has an added
1177 dimension; the time of delivery of a packet can be just as important
1178 as the contents of the packet, itself. A man-in-the-middle attack,
1179 for example, can impose, and then systematically adjust, additional
1180 delays into a link, and thus disrupt or subvert a real-time
1181 application without having to crack any encryption methods employed.
1182 See [RFC7384] for an exploration of this issue in a related context.
1184 Furthermore, in a control system where millions of dollars of
1185 equipment, or even human lives, can be lost if the DetNet QoS is not
1186 delivered, one must consider not only simple equipment failures,
1187 where the box or wire instantly becomes perfectly silent, but bizarre
1188 errors such as can be caused by software failures. Because there is
1189 essential no limit to the kinds of failures that can occur,
1190 protecting against realistic equipment failures is indistinguishable,
1191 in most cases, from protecting against malicious behavior, whether
1192 accidental or intentional. See also Section 4.5.
1194 Security must cover:
1196 o the protection of the signaling protocol
1198 o the authentication and authorization of the controlling systems
1200 o the identification and shaping of the DetNet flows
1202 8. Privacy Considerations
1204 DetNet is provides a Quality of Service (QoS), and as such, does not
1205 directly raise any new privacy considerations.
1207 However, the requirement for every (or almost every) node along the
1208 path of a DetNet flow to identify DetNet flows may present an
1209 additional attack surface for privacy, should the DetNet paradigm be
1210 found useful in broader environments.
1212 9. IANA Considerations
1214 This document does not require an action from IANA.
1216 10. Acknowledgements
1218 The authors wish to thank Jouni Korhonen, Erik Nordmark, George
1219 Swallow, Rudy Klecka, Anca Zamfir, David Black, Thomas Watteyne,
1220 Shitanshu Shah, Craig Gunther, Rodney Cummings, Balazs Varga,
1221 Wilfried Steiner, Marcel Kiessling, Karl Weber, Janos Farkas, Ethan
1222 Grossman, Pat Thaler, and Lou Berger for their various contribution
1223 with this work.
1225 11. Access to IEEE 802.1 documents
1227 To access password protected IEEE 802.1 drafts, see the IETF IEEE
1228 802.1 information page at https://www.ietf.org/proceedings/52/slides/
1229 bridge-0/tsld003.htm.
1231 12. Informative References
1233 [AVnu] http://www.avnu.org/, "The AVnu Alliance tests and
1234 certifies devices for interoperability, providing a simple
1235 and reliable networking solution for AV network
1236 implementation based on the Audio Video Bridging (AVB)
1237 standards.".
1239 [CCAMP] IETF, "Common Control and Measurement Plane",
1240 .
1242 [HART] www.hartcomm.org, "Highway Addressable Remote Transducer,
1243 a group of specifications for industrial process and
1244 control devices administered by the HART Foundation".
1246 [HSR-PRP] IEC, "High availability seamless redundancy (HSR) is a
1247 further development of the PRP approach, although HSR
1248 functions primarily as a protocol for creating media
1249 redundancy while PRP, as described in the previous
1250 section, creates network redundancy. PRP and HSR are both
1251 described in the IEC 62439 3 standard.",
1252 .
1255 [I-D.dt-detnet-dp-alt]
1256 Korhonen, J., Farkas, J., Mirsky, G., Thubert, P.,
1257 Zhuangyan, Z., and L. Berger, "DetNet Data Plane Protocol
1258 and Solution Alternatives", draft-dt-detnet-dp-alt-01
1259 (work in progress), July 2016.
1261 [I-D.ietf-6tisch-architecture]
1262 Thubert, P., "An Architecture for IPv6 over the TSCH mode
1263 of IEEE 802.15.4", draft-ietf-6tisch-architecture-10 (work
1264 in progress), June 2016.
1266 [I-D.ietf-6tisch-tsch]
1267 Watteyne, T., Palattella, M., and L. Grieco, "Using
1268 IEEE802.15.4e TSCH in an IoT context: Overview, Problem
1269 Statement and Goals", draft-ietf-6tisch-tsch-06 (work in
1270 progress), March 2015.
1272 [I-D.ietf-detnet-problem-statement]
1273 Finn, N. and P. Thubert, "Deterministic Networking Problem
1274 Statement", draft-ietf-detnet-problem-statement-00 (work
1275 in progress), April 2016.
1277 [I-D.ietf-detnet-use-cases]
1278 Grossman, E., Gunther, C., Thubert, P., Wetterwald, P.,
1279 Raymond, J., Korhonen, J., Kaneko, Y., Das, S., Zha, Y.,
1280 Varga, B., Farkas, J., Goetz, F., and J. Schmitt,
1281 "Deterministic Networking Use Cases", draft-ietf-detnet-
1282 use-cases-10 (work in progress), July 2016.
1284 [I-D.ietf-roll-rpl-industrial-applicability]
1285 Phinney, T., Thubert, P., and R. Assimiti, "RPL
1286 applicability in industrial networks", draft-ietf-roll-
1287 rpl-industrial-applicability-02 (work in progress),
1288 October 2013.
1290 [I-D.svshah-tsvwg-deterministic-forwarding]
1291 Shah, S. and P. Thubert, "Deterministic Forwarding PHB",
1292 draft-svshah-tsvwg-deterministic-forwarding-04 (work in
1293 progress), August 2015.
1295 [IEEE802.11-2012]
1296 IEEE, "Wireless LAN Medium Access Control (MAC) and
1297 Physical Layer (PHY) Specifications", 2012,
1298 .
1301 [IEEE802.1AS-2011]
1302 IEEE, "Timing and Synchronizations (IEEE 802.1AS-2011)",
1303 2011, .
1306 [IEEE802.1BA-2011]
1307 IEEE, "AVB Systems (IEEE 802.1BA-2011)", 2011,
1308 .
1311 [IEEE802.1CB]
1312 IEEE, "Frame Replication and Elimination for Reliability
1313 (IEEE Draft P802.1CB)", 2016,
1314 .
1316 [IEEE802.1Q-2014]
1317 IEEE, "MAC Bridges and VLANs (IEEE 802.1Q-2014", 2014,
1318 .
1321 [IEEE802.1Qbu]
1322 IEEE, "Frame Preemption", 2016,
1323 .
1325 [IEEE802.1Qbv]
1326 IEEE, "Enhancements for Scheduled Traffic", 2016,
1327 .
1329 [IEEE802.1Qca]
1330 IEEE 802.1, "IEEE 802.1Qca Bridges and Bridged Networks -
1331 Amendment 24: Path Control and Reservation", IEEE
1332 P802.1Qca/D2.1 P802.1Qca, June 2015,
1333 .
1336 [IEEE802.1Qcc]
1337 IEEE, "Stream Reservation Protocol (SRP) Enhancements and
1338 Performance Improvements", 2016,
1339 .
1341 [IEEE802.1Qch]
1342 IEEE, "Cyclic Queuing and Forwarding", 2016,
1343 .
1345 [IEEE802.1TSNTG]
1346 IEEE Standards Association, "IEEE 802.1 Time-Sensitive
1347 Networks Task Group", 2013,
1348 .
1350 [IEEE802.3-2012]
1351 IEEE, "IEEE Stabdard for Ethernet", 2012,
1352 .
1355 [IEEE802.3br]
1356 IEEE, "Interspersed Express Traffic", 2016,
1357 .
1359 [IEEE802154]
1360 IEEE standard for Information Technology, "IEEE std.
1361 802.15.4, Part. 15.4: Wireless Medium Access Control (MAC)
1362 and Physical Layer (PHY) Specifications for Low-Rate
1363 Wireless Personal Area Networks", June 2011.
1365 [IEEE802154e]
1366 IEEE standard for Information Technology, "IEEE std.
1367 802.15.4e, Part. 15.4: Low-Rate Wireless Personal Area
1368 Networks (LR-WPANs) Amendment 1: MAC sublayer", April
1369 2012.
1371 [ISA100.11a]
1372 ISA/IEC, "ISA100.11a, Wireless Systems for Automation,
1373 also IEC 62734", 2011, < http://www.isa100wci.org/en-
1374 US/Documents/PDF/3405-ISA100-WirelessSystems-Future-broch-
1375 WEB-ETSI.aspx>.
1377 [ISA95] ANSI/ISA, "Enterprise-Control System Integration Part 1:
1378 Models and Terminology", 2000, .
1381 [ODVA] http://www.odva.org/, "The organization that supports
1382 network technologies built on the Common Industrial
1383 Protocol (CIP) including EtherNet/IP.".
1385 [PCE] IETF, "Path Computation Element",
1386 .
1388 [Profinet]
1389 http://us.profinet.com/technology/profinet/, "PROFINET is
1390 a standard for industrial networking in automation.",
1391 .
1393 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.
1394 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
1395 Functional Specification", RFC 2205, DOI 10.17487/RFC2205,
1396 September 1997, .
1398 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.,
1399 and W. Weiss, "An Architecture for Differentiated
1400 Services", RFC 2475, DOI 10.17487/RFC2475, December 1998,
1401 .
1403 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
1404 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
1405 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001,
1406 .
1408 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label
1409 Switching (GMPLS) Signaling Resource ReserVation Protocol-
1410 Traffic Engineering (RSVP-TE) Extensions", RFC 3473,
1411 DOI 10.17487/RFC3473, January 2003,
1412 .
1414 [RFC4203] Kompella, K., Ed. and Y. Rekhter, Ed., "OSPF Extensions in
1415 Support of Generalized Multi-Protocol Label Switching
1416 (GMPLS)", RFC 4203, DOI 10.17487/RFC4203, October 2005,
1417 .
1419 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation
1420 Element (PCE)-Based Architecture", RFC 4655,
1421 DOI 10.17487/RFC4655, August 2006,
1422 .
1424 [RFC5307] Kompella, K., Ed. and Y. Rekhter, Ed., "IS-IS Extensions
1425 in Support of Generalized Multi-Protocol Label Switching
1426 (GMPLS)", RFC 5307, DOI 10.17487/RFC5307, October 2008,
1427 .
1429 [RFC5316] Chen, M., Zhang, R., and X. Duan, "ISIS Extensions in
1430 Support of Inter-Autonomous System (AS) MPLS and GMPLS
1431 Traffic Engineering", RFC 5316, DOI 10.17487/RFC5316,
1432 December 2008, .
1434 [RFC5392] Chen, M., Zhang, R., and X. Duan, "OSPF Extensions in
1435 Support of Inter-Autonomous System (AS) MPLS and GMPLS
1436 Traffic Engineering", RFC 5392, DOI 10.17487/RFC5392,
1437 January 2009, .
1439 [RFC5673] Pister, K., Ed., Thubert, P., Ed., Dwars, S., and T.
1440 Phinney, "Industrial Routing Requirements in Low-Power and
1441 Lossy Networks", RFC 5673, DOI 10.17487/RFC5673, October
1442 2009, .
1444 [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau,
1445 L., and L. Berger, "A Framework for MPLS in Transport
1446 Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010,
1447 .
1449 [RFC6372] Sprecher, N., Ed. and A. Farrel, Ed., "MPLS Transport
1450 Profile (MPLS-TP) Survivability Framework", RFC 6372,
1451 DOI 10.17487/RFC6372, September 2011,
1452 .
1454 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in
1455 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384,
1456 October 2014, .
1458 [RFC7426] Haleplidis, E., Ed., Pentikousis, K., Ed., Denazis, S.,
1459 Hadi Salim, J., Meyer, D., and O. Koufopavlou, "Software-
1460 Defined Networking (SDN): Layers and Architecture
1461 Terminology", RFC 7426, DOI 10.17487/RFC7426, January
1462 2015, .
1464 [TEAS] IETF, "Traffic Engineering Architecture and Signaling",
1465 .
1467 [WirelessHART]
1468 www.hartcomm.org, "Industrial Communication Networks -
1469 Wireless Communication Network and Communication Profiles
1470 - WirelessHART - IEC 62591", 2010.
1472 Authors' Addresses
1474 Norman Finn
1475 Cisco Systems
1476 170 W Tasman Dr.
1477 San Jose, California 95134
1478 USA
1480 Phone: +1 408 526 4495
1481 Email: nfinn@cisco.com
1483 Pascal Thubert
1484 Cisco Systems
1485 Village d'Entreprises Green Side
1486 400, Avenue de Roumanille
1487 Batiment T3
1488 Biot - Sophia Antipolis 06410
1489 FRANCE
1491 Phone: +33 4 97 23 26 34
1492 Email: pthubert@cisco.com
1494 Michael Johas Teener
1495 Broadcom Corp.
1496 3151 Zanker Rd.
1497 San Jose, California 95134
1498 USA
1500 Phone: +1 831 824 4228
1501 Email: MikeJT@broadcom.com