idnits 2.17.1
draft-finn-detnet-architecture-08.txt:
Checking boilerplate required by RFC 5378 and the IETF Trust (see
https://trustee.ietf.org/license-info):
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt:
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/checklist :
----------------------------------------------------------------------------
No issues found here.
Miscellaneous warnings:
----------------------------------------------------------------------------
== The copyright year in the IETF Trust and authors Copyright Line does not
match the current year
== Line 635 has weird spacing: '...mediate inter...'
== Line 638 has weird spacing: '...mediate inter...'
== Line 835 has weird spacing: '...e stack v ...'
-- The document date (August 18, 2016) is 2808 days in the past. Is this
intentional?
Checking references for intended status: Proposed Standard
----------------------------------------------------------------------------
(See RFCs 3967 and 4897 for information about using normative references
to lower-maturity documents in RFCs)
== Unused Reference: 'AVnu' is defined on line 1232, but no explicit
reference was found in the text
== Unused Reference: 'HART' is defined on line 1241, but no explicit
reference was found in the text
== Unused Reference: 'I-D.ietf-6tisch-architecture' is defined on line
1260, but no explicit reference was found in the text
== Unused Reference: 'I-D.ietf-6tisch-tsch' is defined on line 1265, but no
explicit reference was found in the text
== Unused Reference: 'I-D.ietf-roll-rpl-industrial-applicability' is
defined on line 1283, but no explicit reference was found in the text
== Unused Reference: 'I-D.svshah-tsvwg-deterministic-forwarding' is defined
on line 1289, but no explicit reference was found in the text
== Unused Reference: 'IEEE802.11-2012' is defined on line 1294, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802.1AS-2011' is defined on line 1300, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802.1TSNTG' is defined on line 1344, but no
explicit reference was found in the text
== Unused Reference: 'IEEE802154' is defined on line 1358, but no explicit
reference was found in the text
== Unused Reference: 'IEEE802154e' is defined on line 1364, but no explicit
reference was found in the text
== Unused Reference: 'ISA95' is defined on line 1376, but no explicit
reference was found in the text
== Unused Reference: 'ODVA' is defined on line 1380, but no explicit
reference was found in the text
== Unused Reference: 'Profinet' is defined on line 1387, but no explicit
reference was found in the text
== Unused Reference: 'RFC5673' is defined on line 1438, but no explicit
reference was found in the text
== Unused Reference: 'WirelessHART' is defined on line 1466, but no
explicit reference was found in the text
== Outdated reference: A later version (-04) exists of
draft-dt-detnet-dp-alt-03
== Outdated reference: A later version (-30) exists of
draft-ietf-6tisch-architecture-10
== Outdated reference: A later version (-09) exists of
draft-ietf-detnet-problem-statement-00
== Outdated reference: A later version (-20) exists of
draft-ietf-detnet-use-cases-10
-- Obsolete informational reference (is this intentional?): RFC 5316
(Obsoleted by RFC 9346)
Summary: 0 errors (**), 0 flaws (~~), 24 warnings (==), 2 comments (--).
Run idnits with the --verbose option for more detailed information about
the items above.
--------------------------------------------------------------------------------
2 DetNet N. Finn
3 Internet-Draft P. Thubert
4 Intended status: Standards Track Cisco
5 Expires: February 19, 2017 August 18, 2016
7 Deterministic Networking Architecture
8 draft-finn-detnet-architecture-08
10 Abstract
12 Deterministic Networking (DetNet) provides a capability to carry
13 specified unicast or multicast data flows for real-time applications
14 with extremely low data loss rates and bounded latency. Techniques
15 used include: 1) reserving data plane resources for individual (or
16 aggregated) DetNet flows in some or all of the intermediate nodes
17 (e.g. bridges or routers) along the path of the flow; 2) providing
18 explicit routes for DetNet flows that do not rapidly change with the
19 network topology; and 3) distributing data from DetNet flow packets
20 over time and/or space to ensure delivery of each packet's data' in
21 spite of the loss of a path. The capabilities can be managed by
22 configuration, or by manual or automatic network management.
24 Status of This Memo
26 This Internet-Draft is submitted in full conformance with the
27 provisions of BCP 78 and BCP 79.
29 Internet-Drafts are working documents of the Internet Engineering
30 Task Force (IETF). Note that other groups may also distribute
31 working documents as Internet-Drafts. The list of current Internet-
32 Drafts is at http://datatracker.ietf.org/drafts/current/.
34 Internet-Drafts are draft documents valid for a maximum of six months
35 and may be updated, replaced, or obsoleted by other documents at any
36 time. It is inappropriate to use Internet-Drafts as reference
37 material or to cite them other than as "work in progress."
39 This Internet-Draft will expire on February 19, 2017.
41 Copyright Notice
43 Copyright (c) 2016 IETF Trust and the persons identified as the
44 document authors. All rights reserved.
46 This document is subject to BCP 78 and the IETF Trust's Legal
47 Provisions Relating to IETF Documents
48 (http://trustee.ietf.org/license-info) in effect on the date of
49 publication of this document. Please review these documents
50 carefully, as they describe your rights and restrictions with respect
51 to this document. Code Components extracted from this document must
52 include Simplified BSD License text as described in Section 4.e of
53 the Trust Legal Provisions and are provided without warranty as
54 described in the Simplified BSD License.
56 Table of Contents
58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
59 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
60 2.1. Terms used in this document . . . . . . . . . . . . . . . 4
61 2.2. IEEE 802 TSN to DetNet dictionary . . . . . . . . . . . . 5
62 3. Providing the DetNet Quality of Service . . . . . . . . . . . 6
63 3.1. Congestion protection . . . . . . . . . . . . . . . . . . 8
64 3.2. Explicit routes . . . . . . . . . . . . . . . . . . . . . 8
65 3.3. Jitter Reduction . . . . . . . . . . . . . . . . . . . . 9
66 3.4. Packet Replication and Elimination . . . . . . . . . . . 10
67 3.5. Packet encoding for service protection . . . . . . . . . 11
68 4. DetNet Architecture . . . . . . . . . . . . . . . . . . . . . 12
69 4.1. Traffic Engineering for DetNet . . . . . . . . . . . . . 12
70 4.1.1. The Application Plane . . . . . . . . . . . . . . . . 12
71 4.1.2. The Controller Plane . . . . . . . . . . . . . . . . 13
72 4.1.3. The Network Plane . . . . . . . . . . . . . . . . . . 13
73 4.2. DetNet flows . . . . . . . . . . . . . . . . . . . . . . 14
74 4.2.1. Source guarantees . . . . . . . . . . . . . . . . . . 14
75 4.2.2. Incomplete Networks . . . . . . . . . . . . . . . . . 16
76 4.3. Queuing, Shaping, Scheduling, and Preemption . . . . . . 16
77 4.4. Coexistence with normal traffic . . . . . . . . . . . . . 17
78 4.5. Fault Mitigation . . . . . . . . . . . . . . . . . . . . 17
79 4.6. Representative Protocol Stack Model . . . . . . . . . . . 18
80 4.7. Exporting flow identification . . . . . . . . . . . . . . 20
81 4.8. Advertising resources, capabilities and adjacencies . . . 22
82 4.9. Provisioning model . . . . . . . . . . . . . . . . . . . 22
83 4.9.1. Centralized Path Computation and Installation . . . . 22
84 4.9.2. Distributed Path Setup . . . . . . . . . . . . . . . 22
85 4.10. Scaling to larger networks . . . . . . . . . . . . . . . 23
86 4.11. Connected islands vs. networks . . . . . . . . . . . . . 23
87 5. Compatibility with Layer-2 . . . . . . . . . . . . . . . . . 23
88 6. Open Questions . . . . . . . . . . . . . . . . . . . . . . . 24
89 6.1. Flat vs. hierarchical control . . . . . . . . . . . . . . 24
90 6.2. Peer-to-peer reservation protocol . . . . . . . . . . . . 24
91 6.3. Wireless media interactions . . . . . . . . . . . . . . . 25
92 7. Security Considerations . . . . . . . . . . . . . . . . . . . 25
93 8. Privacy Considerations . . . . . . . . . . . . . . . . . . . 26
94 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 26
95 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26
96 11. Access to IEEE 802.1 documents . . . . . . . . . . . . . . . 26
97 12. Informative References . . . . . . . . . . . . . . . . . . . 26
98 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 31
100 1. Introduction
102 Deterministic Networking (DetNet) is a service that can be offered by
103 a network to DetNet flows. DetNet provides these flows extremely low
104 packet loss rates and assured maximum end-to-end delivery latency.
105 This is accomplished by dedicating network resources such as link
106 bandwidth and buffer space to DetNet flows and/or classes of DetNet
107 flows, and by replicating packets along multiple paths. Unused
108 reserved resources are available to non-DetNet packets.
110 The Deterministic Networking Problem Statement
111 [I-D.ietf-detnet-problem-statement] introduces Deterministic
112 Networking, and Deterministic Networking Use Cases
113 [I-D.ietf-detnet-use-cases] summarizes the need for it. See
114 [I-D.dt-detnet-dp-alt] for a discussion of specific techniques that
115 can be used to identify DetNet Flows and assign them to specific
116 paths through a network.
118 A goal of DetNet is a converged network in all respects. That is,
119 the presence of DetNet flows does not preclude non-DetNet flows, and
120 the benefits offered DetNet flows should not, except in extreme
121 cases, prevent existing QoS mechanisms from operating in a normal
122 fashion, subject to the bandwidth required for the DetNet flows. A
123 single source-destination pair can trade both DetNet and non-DetNet
124 flows. End systems and applications need not instantiate special
125 interfaces for DetNet flows. Networks are not restricted to certain
126 topologies; connectivity is not restricted. Any application that
127 generates a data flow that can be usefully characterized as having a
128 maximum bandwidth should be able to take advantage of DetNet, as long
129 as the necessary resources can be reserved. Reservations can be made
130 by the application itself, via network management, by an applications
131 controller, or by other means.
133 Many applications of interest to Deterministic Networking require the
134 ability to synchronize the clocks in end systems to a sub-microsecond
135 accuracy. Some of the queue control techniques defined in
136 Section 4.3 also require time synchronization among relay and transit
137 nodes. The means used to achieve time synchronization are not
138 addressed in this document. DetNet should accommodate various
139 synchronization techniques and profiles that are defined elsewhere to
140 solve exchange time in different market segments.
142 The present document is an individual contribution, but it is
143 intended by the authors for adoption by the DetNet working group.
145 2. Terminology
147 2.1. Terms used in this document
149 The following special terms are used in this document in order to
150 avoid the assumption that a given element in the architecture does or
151 does not have Internet Protocol stack, functions as a router, bridge,
152 firewall, or otherwise plays a particular role at Layer-2 or higher.
154 destination
155 An end system capable of receiving a DetNet flow.
157 DetNet domain
158 The portion of a network that is DetNet aware. It includes
159 end systems and other DetNet nodes.
161 DetNet flow
162 A DetNet flow is a sequence of packets to which the DetNet
163 service is to be provided.
165 DetNet compound flow and DetNet member flow
166 A DetNet compound flow is a DetNet flow that has been
167 separated into multiple duplicate DetNet member flows, which
168 are eventually merged back into a single DetNet compound
169 flow, at the DetNet transport layer. "Compound" and "member"
170 are strictly relative to each other, not absolutes; a DetNet
171 compound flow comprising multiple DetNet member flows can, in
172 turn, be a member of a higher-order compound.
174 DetNet intermediate node
175 A DetNet relay node or transit node.
177 DetNet edge node
178 An instance of a DetNet relay node that includes either a
179 DetNet service layer proxy function for DetNet service
180 protection (e.g. the addition or removal of packet sequencing
181 information) for one or more end systems, or starts or
182 terminates congestion protection at the DetNet transport
183 layer, analogous to a Label Edge Router (LER).
185 end system
186 Commonly called a "host" or "node" in IETF documents, and an
187 "end station" is IEEE 802 documents. End systems of interest
188 to this document are either sources or destinations of DetNet
189 flows. And end system may or may not be DetNet transport
190 layer aware or DetNet service layer aware.
192 link
193 A connection between two DetNet nodes. It may be composed of
194 a physical link or a sub-network technology that can provide
195 appropriate traffic delivery for DetNet flows.
197 DetNet node
198 A DetNet aware end system, transit node, or relay node.
199 "DetNet" may be omitted in some text.
201 Detnet relay node
202 A DetNet node including a service layer function that
203 interconnects different DetNet transport layer paths to
204 provide service protection. A DetNet relay node can be a
205 bridge, a router, a firewall, or any other system that
206 participates in the DetNet service layer. It typically
207 incorporates DetNet transport layer functions as well, in
208 which case it is collocated with a transit node.
210 reservation
211 A trail of configuration between source to destination(s)
212 through transit nodes and subnets associated with a DetNet
213 flow, to provide congestion protection.
215 DetNet service layer
216 The layer at which service protection is provided, either
217 packet sequencing, replication, and elimination (Section 3.4)
218 or network coding (Section 3.5).
220 source
221 An end system capable of sourcing a DetNet flow.
223 DetNet transit node
224 A node operating at the DetNet transport layer, that utilizes
225 link layer and/or network layer switching across multiple
226 links and/or sub-networks to provide paths for DetNet service
227 layer functions. Optionally provides congestion protection
228 over those paths. An MPLS LSR is an example of a DetNet
229 transit node.
231 DetNet transport layer
232 The layer that optionally provides congestion protection for
233 DetNet flows over paths provided by the underlying network.
235 2.2. IEEE 802 TSN to DetNet dictionary
237 This section also serves as a dictionary for translating from the
238 terms used by the IEEE 802 Time-Sensitive Networking (TSN) Task Group
239 to those of the DetNet WG.
241 Listener
242 The IEEE 802 term for a destination of a DetNet flow.
244 relay system
245 The IEEE 802 term for a DetNet intermediate node.
247 Stream
248 The IEEE 802 term for a DetNet flow.
250 Talker
251 The IEEE 802 term for the source of a DetNet flow.
253 3. Providing the DetNet Quality of Service
255 The DetNet Quality of Service can be expressed in terms of:
257 o Minimum and maximum end-to-end latency from source to destination;
258 timely delivery and jitter avoidance derive from these constraints
260 o Probability of loss of a packet, under various assumptions as to
261 the operational states of the nodes and links. A derived property
262 is whether it is acceptable to deliver a duplicate packet, which
263 is an inherent risk in highly reliable and/or broadcast
264 transmissions
266 It is a distinction of DetNet that it is concerned solely with worst-
267 case values for the end-to-end latency. Average, mean, or typical
268 values are of no interest, because they do not affect the ability of
269 a real-time system to perform its tasks. In general, a trivial
270 priority-based queuing scheme will give better average latency to a
271 data flow than DetNet, but of course, the worst-case latency can be
272 essentially unbounded.
274 Three techniques are used by DetNet to provide these qualities of
275 service:
277 o Congestion protection (Section 3.1).
279 o Explicit routes (Section 3.2).
281 o Service protection.
283 Congestion protection operates by reserving resources along the path
284 of a DetNet Flow, e.g. buffer space or link bandwidth. Congestion
285 protection greatly reduces, or even eliminates entirely, packet loss
286 due to output packet congestion within the network, but it can only
287 be supplied to a DetNet flow that is limited at the source to a
288 maximum packet size and transmission rate.
290 Congestion protection addresses both of the DetNet QoS requirements
291 (latency and packet loss). Given that DetNet nodes have a finite
292 amount of buffer space, congestion protection necessarily results in
293 a maximum end-to-end latency. It also addresses the largest
294 contribution to packet loss, which is buffer congestion.
296 After congestion, the most important contributions to packet loss are
297 typically from random media errors and equipment failures. Service
298 protection is the name for the mechanisms used by DetNet to address
299 these losses. The mechanisms employed are constrained by the
300 requirement to meet the users' latency requirements. Packet
301 replication and elimination (Section 3.4) packet encoding Section 3.5
302 are described in this document to provide service protection; others
303 may be found. Both mechanisms distribute the contents of DetNet
304 flows over multiple paths in time and/or space, so that the loss of
305 some of the paths does need not cause the loss of any packets. The
306 paths are typically (but not necessarily) explicit routes, so that
307 they cannot suffer temporary interruptions caused by the
308 reconvergence of routing or bridging protocols.
310 These three techniques can be applied independently, giving eight
311 possible combinations, including none (no DetNet), although some
312 combinations are of wider utility than others. This separation keeps
313 the protocol stack coherent and maximizes interoperability with
314 existing and developing standards in this (IETF) and other Standards
315 Development Organizations. Some examples of typical expected
316 combinations:
318 o Explicit routes plus service protection are exactly the techniques
319 employed by [HSR-PRP]. Explicit routes are achieved by limiting
320 the physical topology of the network, and the sequentialization,
321 replication, and duplicate elimination are facilitated by packet
322 tags added at the front or the end of Ethernet frames.
324 o Congestion protection alone is is offered by IEEE 802.1 Audio
325 Video bridging [IEEE802.1BA-2011]. As long as the network suffers
326 no failures, zero congestion loss can be achieved through the use
327 of a reservation protocol (MSRP), shapers in every bridge, and a
328 bit of network calculus.
330 o Using all three together gives maximum protection.
332 There are, of course, simpler methods available (and employed, today)
333 to achieve levels of latency and packet loss that are satisfactory
334 for many applications. Prioritization and over-provisioning is one
335 such technique. However, these methods generally work best in the
336 absence of any significant amount of non-critical traffic in the
337 network (if, indeed, such traffic is supported at all), or work only
338 if the critical traffic constitutes only a small portion of the
339 network's theoretical capacity, or work only if all systems are
340 functioning properly, or in the absence of actions by end systems
341 that disrupt the network's operations.
343 There are any number of methods in use, defined, or in progress for
344 accomplishing each of the above techniques. It is expected that this
345 DetNet Architecture will assist various vendors, users, and/or
346 "vertical" Standards Development Organizations (dedicated to a single
347 industry) to make selections among the available means of
348 implementing DetNet networks.
350 3.1. Congestion protection
352 The primary means by which DetNet achieves its QoS assurances is to
353 reduce, or even completely eliminate, congestion at an output port as
354 a cause of packet loss. Given that a DetNet flow cannot be
355 throttled, this can be achieved only by the provision of sufficient
356 buffer storage at each hop through the network to ensure that no
357 packets are dropped due to a lack of buffer storage.
359 Ensuring adequate buffering requires, in turn, that the source, and
360 every intermediate node along the path to the destination (or nearly
361 every node -- see Section 4.2.2) be careful to regulate its output to
362 not exceed the data rate for any DetNet flow, except for brief
363 periods when making up for interfering traffic. Any packet sent
364 ahead of its time potentially adds to the number of buffers required
365 by the next hop, and may thus exceed the resources allocated for a
366 particular DetNet flow.
368 The low-level mechanisms described in Section 4.3 provide the
369 necessary regulation of transmissions by an end system or
370 intermediate node to provide congestion protection. The reservation
371 of the bandwidth and buffers for a DetNet flow requires the
372 provisioning described in Section 4.9. A DetNet node may have other
373 resources requiring allocation and/or scheduling, that might
374 otherwise be over-subscribed and trigger the rejection of a
375 reservation.
377 3.2. Explicit routes
379 In networks controlled by typical peer-to-peer protocols such as IEEE
380 802.1 ISIS bridged networks or IETF OSPF routed networks, a network
381 topology event in one part of the network can impact, at least
382 briefly, the delivery of data in parts of the network remote from the
383 failure or recovery event. Thus, even redundant paths through a
384 network, if controlled by the typical peer-to-peer protocols, do not
385 eliminate the chances of brief losses of contact.
387 Many real-time networks rely on physical rings or chains of two-port
388 devices, with a relatively simple ring control protocol. This
389 supports redundant paths for service protection with a minimum of
390 wiring. As an additional benefit, ring topologies can often utilize
391 different topology management protocols than those used for a mesh
392 network, with a consequent reduction in the response time to topology
393 changes. Of course, this comes at some cost in terms of increased
394 hop count, and thus latency, for the typical path.
396 In order to get the advantages of low hop count and still ensure
397 against even very brief losses of connectivity, DetNet employs
398 explicit routes, where the path taken by a given DetNet flow does not
399 change, at least immediately, and likely not at all, in response to
400 network topology events. Service protection (Section 3.4 or
401 Section 3.5) over explicit routes provides a high likelihood of
402 continuous connectivity. Explicit routes are commonly used in MPLS
403 TE LSPs.
405 3.3. Jitter Reduction
407 A core objective of DetNet is to enable the convergence of Non-IP
408 networks onto a common network infrastructure. This requires the
409 accurate emulation of currently deployed mission-specific networks,
410 which typically rely on point-to-point analog (e.g. 4-20mA
411 modulation) and serial-digital cables (or buses) for highly reliable,
412 synchronized and jitter-free communications. While the latency of
413 analog transmissions is basically the speed of light, legacy serial
414 links are usually slow (in the order of Kbps) compared to, say, GigE,
415 and some latency is usually acceptable. What is not acceptable is
416 the introduction of excessive jitter, which may, for instance, affect
417 the stability of control systems.
419 Applications that are designed to operate on serial links usually do
420 not provide services to recover the jitter, because jitter simply
421 does not exists there. Streams of information are expected to be
422 delivered in-order and the precise time of reception influences the
423 processes. In order to converge such existing applications, there is
424 a desire to emulate all properties of the serial cable, such as clock
425 transportation, perfect flow isolation and fixed latency. While
426 minimal jitter (in the form of specifying minimum, as well as
427 maximum, end-to-end latency) is supported by DetNet, there are
428 practical limitations on packet-based networks in this regard. In
429 general, users are encouraged to use, instead of, "do this when you
430 get the packet," a combination of:
432 o Sub-microsecond time synchronization among all source and
433 destination end systems, and
435 o Time-of-execution fields in the application packets.
437 Jitter reduction is provided by the mechanisms described in
438 Section 4.3 that also provide congestion protection.
440 3.4. Packet Replication and Elimination
442 After congestion loss has been eliminated, the most important causes
443 of packet loss are random media and/or memory faults, and equipment
444 failures. Both causes of packet loss can be greatly reduced by
445 spreading the data in a packet over multiple transmissions. One such
446 method for service protection is described in this section, which
447 sends the same packets over multiple paths. See also Section 3.5.
449 Packet replication and elimination, also known as seamless redundancy
450 [HSR-PRP], or 1+1 hitless protection, is a function of the DetNet
451 service layer. It involves three capabilities:
453 o Providing sequencing information, once, at or near the source, to
454 the packets of a DetNet compound flow. This may be done by adding
455 a sequence number or time stamp as part of DetNet, or may be
456 inherent in the packet, e.g. in a transport protocol, or
457 associated to other physical properties such as the precise time
458 (and radio channel) of reception of the packet. Section 3.2.
460 o Replicating these packets into multiple DetNet member flows and,
461 typically, sending them along at least two different paths to the
462 destination(s), e.g. over the explicit routes of
464 o Eliminating duplicated packets. This may be done at any step
465 along the path to save network resources further down, in
466 particular if multiple Replication points exist. But the most
467 common case is to perform this operation at the very edge of the
468 DetNet network, preferably in or near the receiver.
470 This function is a "hitless" version of, e.g., the 1+1 linear
471 protection in [RFC6372]. That is, instead of switching from one flow
472 to the other when a failure of a flow is detected, DetNet combines
473 both flows, and performs a packet-by-packet selection of which to
474 discard, based on sequence number.
476 In the simplest case, this amounts to replicating each packet in a
477 source that has two interfaces, and conveying them through the
478 network, along separate paths, to the similarly dual-homed
479 destinations, that discard the extras. This ensures that one path
480 (with zero congestion loss) remains, even if some intermediate node
481 fails. The sequence numbers can also be used for loss detection and
482 for re-ordering.
484 Detnet relay nodes in the network can provide replication and
485 elimination facilities at various points in the network, so that
486 multiple failures can be accommodated.
488 This is shown in the following figure, where the two relay nodes each
489 replicate (R) the DetNet flow on input, sending the DetNet member
490 flows to both the other relay node and to the end system, and
491 eliminate duplicates (E) on the output interface to the right-hand
492 end system. Any one link in the network can fail, and the Detnet
493 compound flow can still get through. Furthermore, two links can
494 fail, as long as they are in different segments of the network.
496 Packet replication and elimination
498 > > > > > > > > > relay > > > > > > > >
499 > /------------+ R node E +------------\ >
500 > / v + ^ \ >
501 end R + v | ^ + E end
502 system + v | ^ + system
503 > \ v + ^ / >
504 > \------------+ R relay E +-----------/ >
505 > > > > > > > > > node > > > > > > > >
507 Figure 1
509 Note that packet replication and elimination does not react to and
510 correct failures; it is entirely passive. Thus, intermittent
511 failures, mistakenly created packet filters, or misrouted data is
512 handled just the same as the equipment failures that are detected
513 handled by typical routing and bridging protocols.
515 If packet replication and elimination is used over paths providing
516 congestion protection (Section 3.1), and member flows that take
517 different-length paths through the network are combined, a merge
518 point may require extra buffering to equalize the delays over the
519 different paths. This equalization ensures that the resultant
520 compound flow will not exceed its contracted bandwidth even after one
521 or the other of the paths is restored after a failure.
523 3.5. Packet encoding for service protection
525 There are methods for using multiple paths to provide service
526 protection that involve encoding the information in a packet
527 belonging to a DetNet flow into multiple transmission units,
528 typically combining information from multiple packets into any given
529 transmission unit. Such techniques may be applicable for use as a
530 DetNet service protection technique, assuming that the DetNet users'
531 needs for timeliness of delivery and freedom from interference with
532 misbehaving DetNet flows can be met.
534 No specific mechanisms are defined here, at this time. This section
535 will either be enhanced or removed. Contributions are invited.
537 4. DetNet Architecture
539 4.1. Traffic Engineering for DetNet
541 Traffic Engineering Architecture and Signaling (TEAS) [TEAS] defines
542 traffic-engineering architectures for generic applicability across
543 packet and non-packet networks. From TEAS perspective, Traffic
544 Engineering (TE) refers to techniques that enable operators to
545 control how specific traffic flows are treated within their networks.
547 Because if its very nature of establishing explicit optimized paths,
548 Deterministic Networking can be seen as a new, specialized branch of
549 Traffic Engineering, and inherits its architecture with a separation
550 into planes.
552 The Deterministic Networking architecture is thus composed of three
553 planes, a (User) Application Plane, a Controller Plane, and a Network
554 Plane, which echoes that of Figure 1 of Software-Defined Networking
555 (SDN): Layers and Architecture Terminology [RFC7426].:
557 4.1.1. The Application Plane
559 Per [RFC7426], the Application Plane includes both applications and
560 services. In particular, the Application Plane incorporates the User
561 Agent, a specialized application that interacts with the end user /
562 operator and performs requests for Deterministic Networking services
563 via an abstract Flow Management Entity, (FME) which may or may not be
564 collocated with (one of) the end systems.
566 At the Application Plane, a management interface enables the
567 negotiation of flows between end systems. An abstraction of the flow
568 called a Traffic Specification (TSpec) provides the representation.
569 This abstraction is used to place a reservation over the (Northbound)
570 Service Interface and within the Application plane. It is associated
571 with an abstraction of location, such as IP addresses and DNS names,
572 to identify the end systems and eventually specify intermediate
573 nodes.
575 4.1.2. The Controller Plane
577 The Controller Plane corresponds to the aggregation of the Control
578 and Management Planes in [RFC7426], though Common Control and
579 Measurement Plane (CCAMP) [CCAMP] makes an additional distinction
580 between management and measurement. When the logical separation of
581 the Control, Measurement and other Management entities is not
582 relevant, the term Controller Plane is used for simplicity to
583 represent them all, and the term controller refers to any device
584 operating in that plane, whether is it a Path Computation entity or a
585 Network Management entity (NME). The Path Computation Element (PCE)
586 [PCE] is a core element of a controller, in charge of computing
587 Deterministic paths to be applied in the Network Plane.
589 A (Northbound) Service Interface enables applications in the
590 Application Plane to communicate with the entities in the Controller
591 Plane.
593 One or more PCE(s) collaborate to implement the requests from the FME
594 as Per-Flow Per-Hop Behaviors installed in the intermediate nodes for
595 each individual flow. The PCEs place each flow along a deterministic
596 sequence of intermediate nodes so as to respect per-flow constraints
597 such as security and latency, and optimize the overall result for
598 metrics such as an abstract aggregated cost. The deterministic
599 sequence can typically be more complex than a direct sequence and
600 include redundancy path, with one or more packet replication and
601 elimination points.
603 4.1.3. The Network Plane
605 The Network Plane represents the network devices and protocols as a
606 whole, regardless of the Layer at which the network devices operate.
607 It includes Forwarding Plane (data plane), Application, and
608 Operational Plane (control plane) aspects.
610 The network Plane comprises the Network Interface Cards (NIC) in the
611 end systems, which are typically IP hosts, and intermediate nodes,
612 which are typically IP routers and switches. Network-to-Network
613 Interfaces such as used for Traffic Engineering path reservation in
614 [RFC5921], as well as User-to-Network Interfaces (UNI) such as
615 provided by the Local Management Interface (LMI) between network and
616 end systems, are both part of the Network Plane, both in the control
617 plane and the data plane.
619 A Southbound (Network) Interface enables the entities in the
620 Controller Plane to communicate with devices in the Network Plane.
621 This interface leverages and extends TEAS to describe the physical
622 topology and resources in the Network Plane.
624 Flow Management Entity
626 End End
627 System System
629 -+-+-+-+-+-+-+ Northbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
631 PCE PCE PCE PCE
633 -+-+-+-+-+-+-+ Southbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
635 intermediate intermed. intermed. intermed.
636 Node Node Node Node
637 NIC NIC
638 intermediate intermed. intermed. intermed.
639 Node Node Node Node
641 Figure 2
643 The intermediate nodes (and eventually the end systems NIC) expose
644 their capabilities and physical resources to the controller (the
645 PCE), and update the PCE with their dynamic perception of the
646 topology, across the Southbound Interface. In return, the PCE(s) set
647 the per-flow paths up, providing a Flow Characterization that is more
648 tightly coupled to the intermediate node Operation than a TSpec.
650 At the Network plane, intermediate nodes may exchange information
651 regarding the state of the paths, between adjacent systems and
652 eventually with the end systems, and forward packets within
653 constraints associated to each flow, or, when unable to do so,
654 perform a last resort operation such as drop or declassify.
656 This specification focuses on the Southbound interface and the
657 operation of the Network Plane.
659 4.2. DetNet flows
661 4.2.1. Source guarantees
663 For the purposes of congestion protection, DetNet flows can be
664 synchronous or asynchronous. In synchronous DetNet flows, at least
665 the intermediate nodes (and possibly the end systems) are closely
666 time synchronized, typically to better than 1 microsecond. By
667 transmitting packets from different DetNet flows or classes of DetNet
668 flows at different times, using repeating schedules synchronized
669 among the intermediate nodes, resources such as buffers and link
670 bandwidth can be shared over the time domain among different DetNet
671 flows. There is a tradeoff among techniques for synchronous DetNet
672 flows between the burden of fine-grained scheduling and the benefit
673 of reducing the required resources, especially buffer space.
675 In contrast, asynchronous DetNet flows are not coordinated with a
676 fine-grained schedule, so relay and end systems must assume worst-
677 case interference among DetNet flows contending for buffer resources.
678 Asynchronous DetNet flows are characterized by:
680 o A maximum packet size;
682 o An observation interval; and
684 o A maximum number of transmissions during that observation
685 interval.
687 These parameters, together with knowledge of the protocol stack used
688 (and thus the size of the various headers added to a packet), limit
689 the number of bit times per observation interval that the DetNet flow
690 can occupy the physical medium.
692 The source promises that these limits will not be exceeded. If the
693 source transmits less data than this limit allows, the unused
694 resources such as link bandwidth can be made available by the system
695 to non-DetNet packets. However, making those resources available to
696 DetNet packets in other DetNet flows would serve no purpose. Those
697 other DetNet flows have their own dedicated resources, on the
698 assumption that all DetNet flows can use all of their resources over
699 a long period of time.
701 Note that there is no provision in DetNet for throttling DetNet flows
702 (reducing the transmission rate via feedback); the assumption is that
703 a DetNet flow, to be useful, must be delivered in its entirety. That
704 is, while any useful application is written to expect a certain
705 number of lost packets, the real-time applications of interest to
706 DetNet demand that the loss of data due to the network is
707 extraordinarily infrequent.
709 Although DetNet strives to minimize the changes required of an
710 application to allow it to shift from a special-purpose digital
711 network to an Internet Protocol network, one fundamental shift in the
712 behavior of network applications is impossible to avoid: the
713 reservation of resources before the application starts. In the first
714 place, a network cannot deliver finite latency and practically zero
715 packet loss to an arbitrarily high offered load. Secondly, achieving
716 practically zero packet loss for unthrottled (though bandwidth
717 limited) DetNet flows means that bridges and routers have to dedicate
718 buffer resources to specific DetNet flows or to classes of DetNet
719 flows. The requirements of each reservation have to be translated
720 into the parameters that control each system's queuing, shaping, and
721 scheduling functions and delivered to the hosts, bridges, and
722 routers.
724 4.2.2. Incomplete Networks
726 The presence in the network of transit nodes or subnets that are not
727 fully capable of offering DetNet services complicates the ability of
728 the intermediate nodes and/or controller to allocate resources, as
729 extra buffering, and thus extra latency, must be allocated at points
730 downstream from the non-DetNet intermediate node for a DetNet flow.
732 4.3. Queuing, Shaping, Scheduling, and Preemption
734 DetNet achieves congestion protection and bounded delivery latency by
735 reserving bandwidth and buffer resources at every hop along the path
736 of the DetNet flow. The reservation itself is not sufficient,
737 however. Implementors and users of a number of proprietary and
738 standard real-time networks have found that standards for specific
739 data plane techniques are required to enable these assurances to be
740 made in a multi-vendor network. The fundamental reason is that
741 latency variation in one system results in the need for extra buffer
742 space in the next-hop system(s), which in turn, increases the worst-
743 case per-hop latency.
745 Standard queuing and transmission selection algorithms allow a
746 central controller to compute the latency contribution of each
747 transit node to the end-to-end latency, to compute the amount of
748 buffer space required in each transit node for each incremental
749 DetNet flow, and most importantly, to translate from a flow
750 specification to a set of values for the managed objects that control
751 each relay or end system. The IEEE 802 has specified (and is
752 specifying) a set of queuing, shaping, and scheduling algorithms that
753 enable each transit node (bridge or router), and/or a central
754 controller, to compute these values. These algorithms include:
756 o A credit-based shaper [IEEE802.1Q-2014] Clause 34.
758 o Time-gated queues governed by a rotating time schedule,
759 synchronized among all transit nodes [IEEE802.1Qbv].
761 o Synchronized double (or triple) buffers driven by synchronized
762 time ticks. [IEEE802.1Qch].
764 o Pre-emption of an Ethernet packet in transmission by a packet with
765 a more stringent latency requirement, followed by the resumption
766 of the preempted packet [IEEE802.1Qbu], [IEEE802.3br].
768 While these techniques are currently embedded in Ethernet and
769 bridging standards, we can note that they are all, except perhaps for
770 packet preemption, equally applicable to other media than Ethernet,
771 and to routers as well as bridges.
773 4.4. Coexistence with normal traffic
775 A DetNet network supports the dedication of a high proportion (e.g.
776 75%) of the network bandwidth to DetNet flows. But, no matter how
777 much is dedicated for DetNet flows, it is a goal of DetNet to coexist
778 with existing Class of Service schemes (e.g., DiffServ). It is also
779 important that non-DetNet traffic not disrupt the DetNet flow, of
780 course (see Section 4.5 and Section 7). For these reasons:
782 o Bandwidth (transmission opportunities) not utilized by a DetNet
783 flow are available to non-DetNet packets (though not to other
784 DetNet flows).
786 o DetNet flows can be shaped or scheduled, in order to ensure that
787 the highest-priority non-DetNet packet also is ensured a worst-
788 case latency (at any given hop).
790 o When transmission opportunities for DetNet flows are scheduled in
791 detail, then the algorithm constructing the schedule should leave
792 sufficient opportunities for non-DetNet packets to satisfy the
793 needs of the users of the network. Detailed scheduling can also
794 permit the time-shared use of buffer resources by different DetNet
795 flows.
797 Ideally, the net effect of the presence of DetNet flows in a network
798 on the non-DetNet packets is primarily a reduction in the available
799 bandwidth.
801 4.5. Fault Mitigation
803 One key to building robust real-time systems is to reduce the
804 infinite variety of possible failures to a number that can be
805 analyzed with reasonable confidence. DetNet aids in the process by
806 providing filters and policers to detect DetNet packets received on
807 the wrong interface, or at the wrong time, or in too great a volume,
808 and to then take actions such as discarding the offending packet,
809 shutting down the offending DetNet flow, or shutting down the
810 offending interface.
812 It is also essential that filters and service remarking be employed
813 at the network edge to prevent non-DetNet packets from being mistaken
814 for DetNet packets, and thus impinging on the resources allocated to
815 DetNet packets.
817 There exist techniques, at present and/or in various stages of
818 standardization, that can perform these fault mitigation tasks that
819 deliver a high probability that misbehaving systems will have zero
820 impact on well-behaved DetNet flows, except of course, for the
821 receiving interface(s) immediately downstream of the misbehaving
822 device. Examples of such techniques include traffic policing
823 functions (e.g. [RFC2475]) and separating flows into per-flow rate-
824 limited queues.
826 4.6. Representative Protocol Stack Model
828 Figure 3 illustrates a conceptual DetNet data plane layering model.
829 One may compare it to that in [IEEE802.1CB], Annex C, a work in
830 progress.
832 DetNet data plane protocol stack
834 | packets going | ^ packets coming ^
835 v down the stack v | up the stack |
836 +----------------------+ +-----------------------+
837 | Source | | Destination |
838 +----------------------+ +-----------------------+
839 | Service layer | | Service layer |
840 | Packet sequencing | | Duplicate elimination |
841 | Flow duplication | | Flow merging |
842 | Packet encoding | | Packet decoding |
843 +----------------------+ +-----------------------+
844 | Transport layer | | Transport layer |
845 | Congestion prot. | | Congestion prot. |
846 +----------------------+ +-----------------------+
847 | Lower layers | | Lower layers |
848 +----------------------+ +-----------------------+
849 v ^
850 \_________________________/
852 Figure 3
854 Not all layers are required for any given application, or even for
855 any given network. The layers are, from top to bottom:
857 Application
858 Shown as "source" and "destination" in the diagram.
860 OAM
861 Operations, Administration, and Maintenance leverages in-band
862 and out-of-and signaling that validates whether the service
863 is effectively obtained within QoS constraints. OAM is not
864 shown in Figure 3; it may reside in any number of the layers.
866 OAM can involve specific tagging added in the packets for
867 tracing implementation or network configuration errors;
868 traceability enables to find whether a packet is a replica,
869 which relay node performed the replication, and which segment
870 was intended for the replica.
872 Packet sequencing
873 As part of DetNet service protection, supplies the sequence
874 number for packet replication and elimination (Section 3.4).
875 Peers with Duplicate elimination. This layer is not needed
876 if a higher-layer transport protocol is expected to perform
877 any packet sequencing and duplicate elimination required by
878 the DetNet flow duplication.
880 Duplicate elimination
881 As part of the DetNet service layer, based on the sequenced
882 number supplied by its peer, packet sequencing, Duplicate
883 elimination discards any duplicate packets generated by
884 DetNet flow duplication. It can operate on member flows,
885 compound flows, or both. The duplication may also be
886 inferred from other information such as the precise time of
887 reception in a scheduled network. The duplicate elimination
888 layer may also perform resequencing of packets to restore
889 packet order in a flow that was disrupted by the loss of
890 packets on one or another of the multiple paths taken.
892 Flow duplication
893 As part of DetNet service protection, replicates packets that
894 belong to a DetNet compound flow into two or more DetNet
895 member flows. Note that this function is separate from
896 packet sequencing. Flow duplication can be an explicit
897 duplication and remarking of packets, or can be performed by,
898 for example, techniques similar to ordinary multicast
899 replication. Peers with DetNet flow merging.
901 Network flow merging
902 As part of DetNet service protection, merges DetNet member
903 flows together for packets coming up the stack belonging to a
904 specific DetNet compound flow. Peers with DetNet flow
905 duplication. DetNet flow merging, together with packet
906 sequencing, duplicate elimination, and DetNet flow
907 duplication, performs packet replication and elimination
908 (Section 3.4).
910 Packet encoding
911 As part of DetNet service protection, as an alternative to
912 packet sequencing and flow duplication, packet encoding
913 combines the information in multiple DetNet packets, perhaps
914 from different DetNet compound flows, and transmits that
915 information in packets on different DetNet member Flows.
916 Peers with Packet decoding.
918 Packet decoding
919 As part of DetNet service protection, as an alternative to
920 flow merging and duplicate elimination, packet decoding takes
921 packets from different DetNet member flows, and computes from
922 those packets the original DetNet packets from the compound
923 flows input to packet encoding. Peers with Packet encoding.
925 Congestio protection
926 The DetNet transport layer provides congestion protection.
927 See Section 4.3. The actual queuing and shaping mechaniss
928 are typically provided by underlying subnet layers, but since
929 these are can be closely associated with the means of
930 providing paths for DetNet flows (e.g. MPLS LSPs or {VLAN,
931 multicast destination MAC address} pairs), the path and the
932 congestion protection are conflated in this figure.
934 Note that the packet sequencing and duplication elimination functions
935 at the source and destination ends of a DetNet compound flow may be
936 performed either in the end system or in a DetNet edge node. The
937 reader must not confuse a DetNet edge function with other kinds of
938 edge functions, e.g. an Label Edge Router, although the two functions
939 may be performed together. The DetNet edge function is concerned
940 with sequencing packets belonging to DetNet flows. The LER with
941 encapsulating/decapsulating packets for transport, and is considered
942 part of the network underlying the DetNet transport layer.
944 4.7. Exporting flow identification
946 An interesting feature of DetNet, and one that invites
947 implementations that can be accused of "layering violations", is the
948 need for lower layers to be aware of specific flows at higher layers,
949 in order to provide specific queuing and shaping services for
950 specific flows. For example:
952 o A non-IP, strictly L2 source end system X may be sending multiple
953 flows to the same L2 destination end system Y. Those flows may
954 include DetNet flows with different QoS requirements, and may
955 include non-DetNet flows.
957 o A router may be sending any number of flows to another router.
958 Again, those flows may include DetNet flows with different QoS
959 requirements, and may include non-DetNet flows.
961 o Two routers may be separated by bridges. For these bridges to
962 perform any required per-flow queuing and shaping, they must be
963 able to identify the individual flows.
965 o A Label Edge Router (LERs) may have a Label Switched Path (LSP)
966 set up for handling traffic destined for a particular IP address
967 carrying only non-DetNet flows. If a DetNet flow to that same
968 address is requested, a separate LSP may be needed, in order that
969 all of the Label Switch Routers (LSRs) along the path to the
970 destination give that flow special queuing and shaping.
972 The need for a lower-level DetNet node to be aware of individual
973 higher-layer flows is not unique to DetNet. But, given the endless
974 complexity of layering and relayering over tunnels that is available
975 to network designers, DetNet needs to provide a model for flow
976 identification that is at least somewhat better than packet
977 inspection. That is not to say that packet inspection to layer 4 or
978 5 addresses will not be used, or the capability standardized; but,
979 there are alternatives.
981 A DetNet relay node can connect DetNet flows on different paths using
982 different flow identification methods. For example:
984 o A single unicast DetNet flow passing from router A through a
985 bridged network to router B may be assigned a {VLAN, multicast
986 destination MAC address} pair that is unique within that bridged
987 network. The bridges can then identify the flow without accessing
988 higher-layer headers. Of course, the receiving router must
989 recognize and accept that multicast MAC address.
991 o A DetNet flow passing from LSR A to LSR B may be assigned a
992 different label than that used for other flows to the same IP
993 destination.
995 In any of the above cases, it is possible that an existing DetNet
996 flow can be used as a carrier for multiple DetNet sub-flows. (Not to
997 be confused with DetNet compound vs. member flows.) Of course, this
998 requires that the aggregate DetNet flow be provisioned properly to
999 carry the sub-flows.
1001 Thus, rather than packet inspection, there is the option to export
1002 higher-layer information to the lower layer. The requirement to
1003 support one or the other method for flow identification (or both) is
1004 the essential complexity that DetNet brings to existing control plane
1005 models.
1007 4.8. Advertising resources, capabilities and adjacencies
1009 There are three classes of information that a central controller or
1010 decentralized control plane needs to know that can only be obtained
1011 from the end systems and/or transit nodes in the network. When using
1012 a peer-to-peer control plane, some of this information may be
1013 required by a system's neighbors in the network.
1015 o Details of the system's capabilities that are required in order to
1016 accurately allocate that system's resources, as well as other
1017 systems' resources. This includes, for example, which specific
1018 queuing and shaping algorithms are implemented (Section 4.3), the
1019 number of buffers dedicated for DetNet allocation, and the worst-
1020 case forwarding delay.
1022 o The dynamic state of an end or transit node's DetNet resources.
1024 o The identity of the system's neighbors, and the characteristics of
1025 the link(s) between the systems, including the length (in
1026 nanoseconds) of the link(s).
1028 4.9. Provisioning model
1030 4.9.1. Centralized Path Computation and Installation
1032 A centralized routing model, such as provided with a PCE (RFC 4655
1033 [RFC4655]), enables global and per-flow optimizations. (See
1034 Section 4.1.) The model is attractive but a number of issues are
1035 left to be solved. In particular:
1037 o Whether and how the path computation can be installed by 1) an end
1038 device or 2) a Network Management entity,
1040 o And how the path is set up, either by installing state at each hop
1041 with a direct interaction between the forwarding device and the
1042 PCE, or along a path by injecting a source-routed request at one
1043 end of the path.
1045 4.9.2. Distributed Path Setup
1047 Significant work on distributed path setup can be leveraged from MPLS
1048 Traffic Engineering, in both its GMPLS and non-GMPLS forms. The
1049 protocols within scope are Resource ReSerVation Protocol [RFC3209]
1050 [RFC3473](RSVP-TE), OSPF-TE [RFC4203] [RFC5392] and ISIS-TE [RFC5307]
1051 [RFC5316]. These should be viewed as starting points as there are
1052 feature specific extensions defined that may be applicable to DetNet.
1054 In a Layer-2 only environment, or as part of a layered approach to a
1055 mixed environment, IEEE 802.1 also has work, either completed or in
1056 progress. [IEEE802.1Q-2014] Clause 35 describes SRP, a peer-to-peer
1057 protocol for Layer-2 roughly analogous to RSVP [RFC2205].
1058 [IEEE802.1Qca] defines how ISIS can provide multiple disjoint paths
1059 or distribution trees. Also in progress is [IEEE802.1Qcc], which
1060 expands the capabilities of SRP.
1062 The integration/interaction of the DetNet control layer with an
1063 underlying IEEE 802.1 sub-network control layer will need to be
1064 defined.
1066 4.10. Scaling to larger networks
1068 Reservations for individual DetNet flows require considerable state
1069 information in each transit node, especially when adequate fault
1070 mitigation (Section 4.5) is required. The DetNet data plane, in
1071 order to support larger numbers of DetNet flows, must support the
1072 aggregation of DetNet flows into tunnels, which themselves can be
1073 viewed by the transit nodes' data planes largely as individual DetNet
1074 flows. Without such aggregation, the per-relay system may limit the
1075 scale of DetNet networks.
1077 4.11. Connected islands vs. networks
1079 Given that users have deployed examples of the IEEE 802.1 TSN TG
1080 standards, which provide capabilities similar to DetNet, it is
1081 obvious to ask whether the IETF DetNet effort can be limited to
1082 providing Layer-2 connections (VPNs) between islands of bridged TSN
1083 networks. While this capability is certainly useful to some
1084 applications, and must not be precluded by DetNet, tunneling alone is
1085 not a sufficient goal for the DetNet WG. As shown in the
1086 Deterministic Networking Use Cases draft [I-D.ietf-detnet-use-cases],
1087 there are already deployments of Layer-2 TSN networks that are
1088 encountering the well-known problems of over-large broadcast domains.
1089 Routed solutions, and combinations routed/bridged solutions, are both
1090 required.
1092 5. Compatibility with Layer-2
1094 Standards providing similar capabilities for bridged networks (only)
1095 have been and are being generated in the IEEE 802 LAN/MAN Standards
1096 Committee. The present architecture describes an abstract model that
1097 can be applicable both at Layer-2 and Layer-3, and over links not
1098 defined by IEEE 802. It is the intention of the authors (and
1099 hopefully, as this draft progresses, of the DetNet Working Group)
1100 that IETF and IEEE 802 will coordinate their work, via the
1101 participation of common individuals, liaisons, and other means, to
1102 maximize the compatibility of their outputs.
1104 DetNet enabled end systems and intermediate nodes can be
1105 interconnected by sub-networks, i.e., Layer-2 technologies. These
1106 sub-networks will provide DetNet compatible service for support of
1107 DetNet traffic. Examples of sub-networks include 802.1TSN and a
1108 point-to-point OTN link. Of course, multi-layer DetNet systems may
1109 be possible too, where one DetNet appears as a sub-network, and
1110 provides service to, a higher layer DetNet system.
1112 6. Open Questions
1114 There are a number of architectural questions that will have to be
1115 resolved before this document can be submitted for publication.
1116 Aside from the obvious fact that this present draft is subject to
1117 change, there are specific questions to which the authors wish to
1118 direct the readers' attention.
1120 6.1. Flat vs. hierarchical control
1122 Boxes that are solely routers or solely bridges are rare in today's
1123 market. In a multi-tenant data center, multiple users' virtual
1124 Layer-2/Layer-3 topologies exist simultaneously, implemented on a
1125 network whose physical topology bears only accidental resemblance to
1126 the virtual topologies.
1128 While the forwarding topology (the bridges and routers) are an
1129 important consideration for a DetNet Flow Management Entity
1130 (Section 4.1.1), so is the purely physical topology. Ultimately, the
1131 model used by the management entities is based on boxes, queues, and
1132 links. The authors hope that the work of the TEAS WG will help to
1133 clarify exactly what model parameters need to be traded between the
1134 intermediate nodes and the controller(s).
1136 6.2. Peer-to-peer reservation protocol
1138 As described in Section 4.9.2, the DetNet WG needs to decide whether
1139 to support a peer-to-peer protocol for a source and a destination to
1140 reserve resources for a DetNet stream. Assuming that enabling the
1141 involvement of the source and/or destination is desirable (see
1142 Deterministic Networking Use Cases [I-D.ietf-detnet-use-cases]), it
1143 remains to decide whether the DetNet WG will make it possible to
1144 deploy at least some DetNet capabilities in a network using only a
1145 peer-to-peer protocol, without a central controller.
1147 (Note that a UNI (see Section 4.1.3) between an end system and a
1148 DetNet edge node, for sources and/or listeners to request DetNet
1149 services, can be either the first hop of a per-to-peer reservation
1150 protocol, or can be deflected by the DetNet edge node to a central
1151 controller for resolution. Similarly, a decision by a central
1152 controller can be effected by the controller instructing the end
1153 system or DetNet edge node to initiate a per-to-peer protocol
1154 activity.)
1156 6.3. Wireless media interactions
1158 Deterministic Networking Use Cases [I-D.ietf-detnet-use-cases]
1159 illustrates cases where wireless media are needed in a DetNet
1160 network. Some wireless media in general use, such as IEEE 802.11
1161 [IEEE802.1Q-2014], have significantly higher packet loss rates than
1162 typical wired media, such as Ethernet [IEEE802.3-2012]. IEEE 802.11
1163 includes support for such features as MAC-layer acknowledgements and
1164 retransmissions.
1166 The techniques described in Section 3 are likely to improve the
1167 ability of a mixed wired/wireless network to offer the DetNet QoS
1168 features. The interaction of these techniques with the features of
1169 specific wireless media, although they may be significant, cannot be
1170 addressed in this document. It remains to be decided to what extent
1171 the DetNet WG will address them, and to what extent other WGs, e.g.
1172 6TiSCH, will do so.
1174 7. Security Considerations
1176 Security in the context of Deterministic Networking has an added
1177 dimension; the time of delivery of a packet can be just as important
1178 as the contents of the packet, itself. A man-in-the-middle attack,
1179 for example, can impose, and then systematically adjust, additional
1180 delays into a link, and thus disrupt or subvert a real-time
1181 application without having to crack any encryption methods employed.
1182 See [RFC7384] for an exploration of this issue in a related context.
1184 Furthermore, in a control system where millions of dollars of
1185 equipment, or even human lives, can be lost if the DetNet QoS is not
1186 delivered, one must consider not only simple equipment failures,
1187 where the box or wire instantly becomes perfectly silent, but bizarre
1188 errors such as can be caused by software failures. Because there is
1189 essential no limit to the kinds of failures that can occur,
1190 protecting against realistic equipment failures is indistinguishable,
1191 in most cases, from protecting against malicious behavior, whether
1192 accidental or intentional. See also Section 4.5.
1194 Security must cover:
1196 o the protection of the signaling protocol
1197 o the authentication and authorization of the controlling systems
1199 o the identification and shaping of the DetNet flows
1201 8. Privacy Considerations
1203 DetNet is provides a Quality of Service (QoS), and as such, does not
1204 directly raise any new privacy considerations.
1206 However, the requirement for every (or almost every) node along the
1207 path of a DetNet flow to identify DetNet flows may present an
1208 additional attack surface for privacy, should the DetNet paradigm be
1209 found useful in broader environments.
1211 9. IANA Considerations
1213 This document does not require an action from IANA.
1215 10. Acknowledgements
1217 The authors wish to thank Jouni Korhonen, Erik Nordmark, George
1218 Swallow, Rudy Klecka, Anca Zamfir, David Black, Thomas Watteyne,
1219 Shitanshu Shah, Craig Gunther, Rodney Cummings, Balazs Varga,
1220 Wilfried Steiner, Marcel Kiessling, Karl Weber, Janos Farkas, Ethan
1221 Grossman, Pat Thaler, Lou Berger, and especially Michael Johas
1222 Teener, for their various contribution with this work.
1224 11. Access to IEEE 802.1 documents
1226 To access password protected IEEE 802.1 drafts, see the IETF IEEE
1227 802.1 information page at https://www.ietf.org/proceedings/52/slides/
1228 bridge-0/tsld003.htm.
1230 12. Informative References
1232 [AVnu] http://www.avnu.org/, "The AVnu Alliance tests and
1233 certifies devices for interoperability, providing a simple
1234 and reliable networking solution for AV network
1235 implementation based on the Audio Video Bridging (AVB)
1236 standards.".
1238 [CCAMP] IETF, "Common Control and Measurement Plane",
1239 .
1241 [HART] www.hartcomm.org, "Highway Addressable Remote Transducer,
1242 a group of specifications for industrial process and
1243 control devices administered by the HART Foundation".
1245 [HSR-PRP] IEC, "High availability seamless redundancy (HSR) is a
1246 further development of the PRP approach, although HSR
1247 functions primarily as a protocol for creating media
1248 redundancy while PRP, as described in the previous
1249 section, creates network redundancy. PRP and HSR are both
1250 described in the IEC 62439 3 standard.",
1251 .
1254 [I-D.dt-detnet-dp-alt]
1255 Korhonen, J., Farkas, J., Mirsky, G., Thubert, P.,
1256 Zhuangyan, Z., and L. Berger, "DetNet Data Plane Protocol
1257 and Solution Alternatives", draft-dt-detnet-dp-alt-03
1258 (work in progress), August 2016.
1260 [I-D.ietf-6tisch-architecture]
1261 Thubert, P., "An Architecture for IPv6 over the TSCH mode
1262 of IEEE 802.15.4", draft-ietf-6tisch-architecture-10 (work
1263 in progress), June 2016.
1265 [I-D.ietf-6tisch-tsch]
1266 Watteyne, T., Palattella, M., and L. Grieco, "Using
1267 IEEE802.15.4e TSCH in an IoT context: Overview, Problem
1268 Statement and Goals", draft-ietf-6tisch-tsch-06 (work in
1269 progress), March 2015.
1271 [I-D.ietf-detnet-problem-statement]
1272 Finn, N. and P. Thubert, "Deterministic Networking Problem
1273 Statement", draft-ietf-detnet-problem-statement-00 (work
1274 in progress), April 2016.
1276 [I-D.ietf-detnet-use-cases]
1277 Grossman, E., Gunther, C., Thubert, P., Wetterwald, P.,
1278 Raymond, J., Korhonen, J., Kaneko, Y., Das, S., Zha, Y.,
1279 Varga, B., Farkas, J., Goetz, F., and J. Schmitt,
1280 "Deterministic Networking Use Cases", draft-ietf-detnet-
1281 use-cases-10 (work in progress), July 2016.
1283 [I-D.ietf-roll-rpl-industrial-applicability]
1284 Phinney, T., Thubert, P., and R. Assimiti, "RPL
1285 applicability in industrial networks", draft-ietf-roll-
1286 rpl-industrial-applicability-02 (work in progress),
1287 October 2013.
1289 [I-D.svshah-tsvwg-deterministic-forwarding]
1290 Shah, S. and P. Thubert, "Deterministic Forwarding PHB",
1291 draft-svshah-tsvwg-deterministic-forwarding-04 (work in
1292 progress), August 2015.
1294 [IEEE802.11-2012]
1295 IEEE, "Wireless LAN Medium Access Control (MAC) and
1296 Physical Layer (PHY) Specifications", 2012,
1297 .
1300 [IEEE802.1AS-2011]
1301 IEEE, "Timing and Synchronizations (IEEE 802.1AS-2011)",
1302 2011, .
1305 [IEEE802.1BA-2011]
1306 IEEE, "AVB Systems (IEEE 802.1BA-2011)", 2011,
1307 .
1310 [IEEE802.1CB]
1311 IEEE, "Frame Replication and Elimination for Reliability
1312 (IEEE Draft P802.1CB)", 2016,
1313 .
1315 [IEEE802.1Q-2014]
1316 IEEE, "MAC Bridges and VLANs (IEEE 802.1Q-2014", 2014,
1317 .
1320 [IEEE802.1Qbu]
1321 IEEE, "Frame Preemption", 2016,
1322 .
1324 [IEEE802.1Qbv]
1325 IEEE, "Enhancements for Scheduled Traffic", 2016,
1326 .
1328 [IEEE802.1Qca]
1329 IEEE 802.1, "IEEE 802.1Qca Bridges and Bridged Networks -
1330 Amendment 24: Path Control and Reservation", IEEE
1331 P802.1Qca/D2.1 P802.1Qca, June 2015,
1332 .
1335 [IEEE802.1Qcc]
1336 IEEE, "Stream Reservation Protocol (SRP) Enhancements and
1337 Performance Improvements", 2016,
1338 .
1340 [IEEE802.1Qch]
1341 IEEE, "Cyclic Queuing and Forwarding", 2016,
1342 .
1344 [IEEE802.1TSNTG]
1345 IEEE Standards Association, "IEEE 802.1 Time-Sensitive
1346 Networks Task Group", 2013,
1347 .
1349 [IEEE802.3-2012]
1350 IEEE, "IEEE Stabdard for Ethernet", 2012,
1351 .
1354 [IEEE802.3br]
1355 IEEE, "Interspersed Express Traffic", 2016,
1356 .
1358 [IEEE802154]
1359 IEEE standard for Information Technology, "IEEE std.
1360 802.15.4, Part. 15.4: Wireless Medium Access Control (MAC)
1361 and Physical Layer (PHY) Specifications for Low-Rate
1362 Wireless Personal Area Networks", June 2011.
1364 [IEEE802154e]
1365 IEEE standard for Information Technology, "IEEE std.
1366 802.15.4e, Part. 15.4: Low-Rate Wireless Personal Area
1367 Networks (LR-WPANs) Amendment 1: MAC sublayer", April
1368 2012.
1370 [ISA100.11a]
1371 ISA/IEC, "ISA100.11a, Wireless Systems for Automation,
1372 also IEC 62734", 2011, < http://www.isa100wci.org/en-
1373 US/Documents/PDF/3405-ISA100-WirelessSystems-Future-broch-
1374 WEB-ETSI.aspx>.
1376 [ISA95] ANSI/ISA, "Enterprise-Control System Integration Part 1:
1377 Models and Terminology", 2000, .
1380 [ODVA] http://www.odva.org/, "The organization that supports
1381 network technologies built on the Common Industrial
1382 Protocol (CIP) including EtherNet/IP.".
1384 [PCE] IETF, "Path Computation Element",
1385 .
1387 [Profinet]
1388 http://us.profinet.com/technology/profinet/, "PROFINET is
1389 a standard for industrial networking in automation.",
1390 .
1392 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.
1393 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
1394 Functional Specification", RFC 2205, DOI 10.17487/RFC2205,
1395 September 1997, .
1397 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.,
1398 and W. Weiss, "An Architecture for Differentiated
1399 Services", RFC 2475, DOI 10.17487/RFC2475, December 1998,
1400 .
1402 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
1403 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
1404 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001,
1405 .
1407 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label
1408 Switching (GMPLS) Signaling Resource ReserVation Protocol-
1409 Traffic Engineering (RSVP-TE) Extensions", RFC 3473,
1410 DOI 10.17487/RFC3473, January 2003,
1411 .
1413 [RFC4203] Kompella, K., Ed. and Y. Rekhter, Ed., "OSPF Extensions in
1414 Support of Generalized Multi-Protocol Label Switching
1415 (GMPLS)", RFC 4203, DOI 10.17487/RFC4203, October 2005,
1416 .
1418 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation
1419 Element (PCE)-Based Architecture", RFC 4655,
1420 DOI 10.17487/RFC4655, August 2006,
1421 .
1423 [RFC5307] Kompella, K., Ed. and Y. Rekhter, Ed., "IS-IS Extensions
1424 in Support of Generalized Multi-Protocol Label Switching
1425 (GMPLS)", RFC 5307, DOI 10.17487/RFC5307, October 2008,
1426 .
1428 [RFC5316] Chen, M., Zhang, R., and X. Duan, "ISIS Extensions in
1429 Support of Inter-Autonomous System (AS) MPLS and GMPLS
1430 Traffic Engineering", RFC 5316, DOI 10.17487/RFC5316,
1431 December 2008, .
1433 [RFC5392] Chen, M., Zhang, R., and X. Duan, "OSPF Extensions in
1434 Support of Inter-Autonomous System (AS) MPLS and GMPLS
1435 Traffic Engineering", RFC 5392, DOI 10.17487/RFC5392,
1436 January 2009, .
1438 [RFC5673] Pister, K., Ed., Thubert, P., Ed., Dwars, S., and T.
1439 Phinney, "Industrial Routing Requirements in Low-Power and
1440 Lossy Networks", RFC 5673, DOI 10.17487/RFC5673, October
1441 2009, .
1443 [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau,
1444 L., and L. Berger, "A Framework for MPLS in Transport
1445 Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010,
1446 .
1448 [RFC6372] Sprecher, N., Ed. and A. Farrel, Ed., "MPLS Transport
1449 Profile (MPLS-TP) Survivability Framework", RFC 6372,
1450 DOI 10.17487/RFC6372, September 2011,
1451 .
1453 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in
1454 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384,
1455 October 2014, .
1457 [RFC7426] Haleplidis, E., Ed., Pentikousis, K., Ed., Denazis, S.,
1458 Hadi Salim, J., Meyer, D., and O. Koufopavlou, "Software-
1459 Defined Networking (SDN): Layers and Architecture
1460 Terminology", RFC 7426, DOI 10.17487/RFC7426, January
1461 2015, .
1463 [TEAS] IETF, "Traffic Engineering Architecture and Signaling",
1464 .
1466 [WirelessHART]
1467 www.hartcomm.org, "Industrial Communication Networks -
1468 Wireless Communication Network and Communication Profiles
1469 - WirelessHART - IEC 62591", 2010.
1471 Authors' Addresses
1473 Norman Finn
1474 Cisco Systems
1475 170 W Tasman Dr.
1476 San Jose, California 95134
1477 USA
1479 Phone: +1 408 526 4495
1480 Email: nfinn@cisco.com
1481 Pascal Thubert
1482 Cisco Systems
1483 Village d'Entreprises Green Side
1484 400, Avenue de Roumanille
1485 Batiment T3
1486 Biot - Sophia Antipolis 06410
1487 FRANCE
1489 Phone: +33 4 97 23 26 34
1490 Email: pthubert@cisco.com