idnits 2.17.1
draft-ietf-detnet-architecture-07.txt:
Checking boilerplate required by RFC 5378 and the IETF Trust (see
https://trustee.ietf.org/license-info):
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt:
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/checklist :
----------------------------------------------------------------------------
No issues found here.
Miscellaneous warnings:
----------------------------------------------------------------------------
== The copyright year in the IETF Trust and authors Copyright Line does not
match the current year
== Line 732 has weird spacing: '...e stack v ...'
== Line 1234 has weird spacing: '...mediate inter...'
== Line 1237 has weird spacing: '...mediate inter...'
-- The document date (August 3, 2018) is 2085 days in the past. Is this
intentional?
Checking references for intended status: Proposed Standard
----------------------------------------------------------------------------
(See RFCs 3967 and 4897 for information about using normative references
to lower-maturity documents in RFCs)
== Missing Reference: 'Network' is mentioned on line 869, but not defined
== Outdated reference: A later version (-30) exists of
draft-ietf-6tisch-architecture-14
== Outdated reference: A later version (-02) exists of
draft-ietf-detnet-dp-sol-ip-00
== Outdated reference: A later version (-02) exists of
draft-ietf-detnet-dp-sol-mpls-00
== Outdated reference: A later version (-09) exists of
draft-ietf-detnet-problem-statement-06
== Outdated reference: A later version (-20) exists of
draft-ietf-detnet-use-cases-17
Summary: 0 errors (**), 0 flaws (~~), 10 warnings (==), 1 comment (--).
Run idnits with the --verbose option for more detailed information about
the items above.
--------------------------------------------------------------------------------
2 DetNet N. Finn
3 Internet-Draft Huawei
4 Intended status: Standards Track P. Thubert
5 Expires: February 4, 2019 Cisco
6 B. Varga
7 J. Farkas
8 Ericsson
9 August 3, 2018
11 Deterministic Networking Architecture
12 draft-ietf-detnet-architecture-07
14 Abstract
16 Deterministic Networking (DetNet) provides a capability to carry
17 specified unicast or multicast data flows for real-time applications
18 with extremely low data loss rates and bounded latency. Techniques
19 used include: 1) reserving data plane resources for individual (or
20 aggregated) DetNet flows in some or all of the intermediate nodes
21 (e.g., bridges or routers) along the path of the flow; 2) providing
22 explicit routes for DetNet flows that do not immediately change with
23 the network topology; and 3) distributing data from DetNet flow
24 packets over time and/or space to ensure delivery of each packet's
25 data in spite of the loss of a path.
27 Status of This Memo
29 This Internet-Draft is submitted in full conformance with the
30 provisions of BCP 78 and BCP 79.
32 Internet-Drafts are working documents of the Internet Engineering
33 Task Force (IETF). Note that other groups may also distribute
34 working documents as Internet-Drafts. The list of current Internet-
35 Drafts is at https://datatracker.ietf.org/drafts/current/.
37 Internet-Drafts are draft documents valid for a maximum of six months
38 and may be updated, replaced, or obsoleted by other documents at any
39 time. It is inappropriate to use Internet-Drafts as reference
40 material or to cite them other than as "work in progress."
42 This Internet-Draft will expire on February 4, 2019.
44 Copyright Notice
46 Copyright (c) 2018 IETF Trust and the persons identified as the
47 document authors. All rights reserved.
49 This document is subject to BCP 78 and the IETF Trust's Legal
50 Provisions Relating to IETF Documents
51 (https://trustee.ietf.org/license-info) in effect on the date of
52 publication of this document. Please review these documents
53 carefully, as they describe your rights and restrictions with respect
54 to this document. Code Components extracted from this document must
55 include Simplified BSD License text as described in Section 4.e of
56 the Trust Legal Provisions and are provided without warranty as
57 described in the Simplified BSD License.
59 Table of Contents
61 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
62 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
63 2.1. Terms used in this document . . . . . . . . . . . . . . . 4
64 2.2. IEEE 802.1 TSN to DetNet dictionary . . . . . . . . . . . 6
65 3. Providing the DetNet Quality of Service . . . . . . . . . . . 7
66 3.1. Primary goals defining the DetNet QoS . . . . . . . . . . 7
67 3.2. Mechanisms to achieve DetNet QoS . . . . . . . . . . . . 9
68 3.2.1. Congestion protection . . . . . . . . . . . . . . . . 9
69 3.2.1.1. Eliminate congestion loss . . . . . . . . . . . . 9
70 3.2.1.2. Jitter Reduction . . . . . . . . . . . . . . . . 10
71 3.2.2. Service Protection . . . . . . . . . . . . . . . . . 11
72 3.2.2.1. In-Order Delivery . . . . . . . . . . . . . . . . 11
73 3.2.2.2. Packet Replication and Elimination . . . . . . . 11
74 3.2.2.3. Packet encoding for service protection . . . . . 13
75 3.2.3. Explicit routes . . . . . . . . . . . . . . . . . . . 13
76 3.3. Secondary goals for DetNet . . . . . . . . . . . . . . . 14
77 3.3.1. Coexistence with normal traffic . . . . . . . . . . . 14
78 3.3.2. Fault Mitigation . . . . . . . . . . . . . . . . . . 15
79 4. DetNet Architecture . . . . . . . . . . . . . . . . . . . . . 16
80 4.1. DetNet stack model . . . . . . . . . . . . . . . . . . . 16
81 4.1.1. Representative Protocol Stack Model . . . . . . . . . 16
82 4.1.2. DetNet Data Plane Overview . . . . . . . . . . . . . 18
83 4.1.3. Network reference model . . . . . . . . . . . . . . . 20
84 4.2. DetNet systems . . . . . . . . . . . . . . . . . . . . . 21
85 4.2.1. End system . . . . . . . . . . . . . . . . . . . . . 21
86 4.2.2. DetNet edge, relay, and transit nodes . . . . . . . . 22
87 4.3. DetNet flows . . . . . . . . . . . . . . . . . . . . . . 23
88 4.3.1. DetNet flow types . . . . . . . . . . . . . . . . . . 23
89 4.3.2. Source transmission behavior . . . . . . . . . . . . 23
90 4.3.3. Incomplete Networks . . . . . . . . . . . . . . . . . 25
91 4.4. Traffic Engineering for DetNet . . . . . . . . . . . . . 25
92 4.4.1. The Application Plane . . . . . . . . . . . . . . . . 25
93 4.4.2. The Controller Plane . . . . . . . . . . . . . . . . 26
94 4.4.3. The Network Plane . . . . . . . . . . . . . . . . . . 26
95 4.5. Queuing, Shaping, Scheduling, and Preemption . . . . . . 27
96 4.6. Service instance . . . . . . . . . . . . . . . . . . . . 28
97 4.7. Flow identification at technology borders . . . . . . . . 30
98 4.7.1. Exporting flow identification . . . . . . . . . . . . 30
99 4.7.2. Flow attribute mapping between layers . . . . . . . . 31
100 4.7.3. Flow-ID mapping examples . . . . . . . . . . . . . . 32
101 4.8. Advertising resources, capabilities and adjacencies . . . 34
102 4.9. Scaling to larger networks . . . . . . . . . . . . . . . 35
103 4.10. Compatibility with Layer-2 . . . . . . . . . . . . . . . 35
104 5. Security Considerations . . . . . . . . . . . . . . . . . . . 35
105 6. Privacy Considerations . . . . . . . . . . . . . . . . . . . 36
106 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 36
107 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 36
108 9. Informative References . . . . . . . . . . . . . . . . . . . 37
109 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 40
111 1. Introduction
113 Deterministic Networking (DetNet) is a service that can be offered by
114 a network to DetNet flows. DetNet provides these flows with
115 extremely low packet loss rates and assured maximum end-to-end
116 delivery latency. This is accomplished by dedicating network
117 resources such as link bandwidth and buffer space to DetNet flows
118 and/or classes of DetNet flows, and by replicating packets along
119 multiple paths. Unused reserved resources are available to non-
120 DetNet packets.
122 The Deterministic Networking Problem Statement
123 [I-D.ietf-detnet-problem-statement] introduces Deterministic
124 Networking, and Deterministic Networking Use Cases
125 [I-D.ietf-detnet-use-cases] summarizes the need for it. See
126 [I-D.ietf-detnet-dp-sol-mpls] and [I-D.ietf-detnet-dp-sol-ip] for
127 specific techniques that can be used to identify DetNet flows and
128 assign them to specific paths through a network.
130 A goal of DetNet is a converged network in all respects. That is,
131 the presence of DetNet flows does not preclude non-DetNet flows, and
132 the benefits offered DetNet flows should not, except in extreme
133 cases, prevent existing QoS mechanisms from operating in a normal
134 fashion, subject to the bandwidth required for the DetNet flows. A
135 single source-destination pair can trade both DetNet and non-DetNet
136 flows. End systems and applications need not instantiate special
137 interfaces for DetNet flows. Networks are not restricted to certain
138 topologies; connectivity is not restricted. Any application that
139 generates a data flow that can be usefully characterized as having a
140 maximum bandwidth should be able to take advantage of DetNet, as long
141 as the necessary resources can be reserved. Reservations can be made
142 by the application itself, via network management, by an
143 application's controller, or by other means, e.g., a dynamic control
144 plane (e.g., [RFC2205]).
146 Many applications that are intended to be served by Deterministic
147 Networking require the ability to synchronize the clocks in end
148 systems to a sub-microsecond accuracy. Some of the queue control
149 techniques defined in Section 4.5 also require time synchronization
150 among relay and transit nodes. The means used to achieve time
151 synchronization are not addressed in this document. DetNet can
152 accommodate various time synchronization techniques and profiles that
153 are defined elsewhere to address the needs of different market
154 segments.
156 2. Terminology
158 2.1. Terms used in this document
160 The following terms are used in the context of DetNet in this
161 document:
163 allocation
164 Resources are dedicated to support a DetNet flow. Depending
165 on an implementation, the resource may be reused by non-
166 DetNet flows when it is not used by the DetNet flow.
168 App-flow
169 The native format of a DetNet flow.
171 DetNet destination
172 An end system capable of terminating a DetNet flow.
174 DetNet domain
175 The portion of a network that is DetNet aware. It includes
176 end systems and other DetNet nodes.
178 DetNet flow
179 A DetNet flow is a sequence of packets to which the DetNet
180 service is to be provided.
182 DetNet compound flow and DetNet member flow
183 A DetNet compound flow is a DetNet flow that has been
184 separated into multiple duplicate DetNet member flows for
185 service protection at the DetNet service layer. Member flows
186 are merged back into a single DetNet compound flow such that
187 there are no duplicate packets. "Compound" and "member" are
188 strictly relative to each other, not absolutes; a DetNet
189 compound flow comprising multiple DetNet member flows can, in
190 turn, be a member of a higher-order compound.
192 DetNet intermediate node
193 A DetNet relay node or transit node.
195 DetNet edge node
196 An instance of a DetNet relay node that acts as a source and/
197 or destination at the DetNet service layer. For example, it
198 can include a DetNet service layer proxy function for DetNet
199 service protection (e.g., the addition or removal of packet
200 sequencing information) for one or more end systems, or
201 starts or terminates congestion protection at the DetNet
202 transport layer, or aggregates DetNet services into new
203 DetNet flows. It is analogous to a Label Edge Router (LER)
204 or a Provider Edge (PE) router.
206 DetNet-UNI
207 User-to-Network Interface with DetNet specific
208 functionalities. It is a packet-based reference point and
209 may provide multiple functions like encapsulation, status,
210 synchronization, etc.
212 end system
213 Commonly called a "host" in IETF documents, and an "end
214 station" is IEEE 802 documents. End systems of interest to
215 this document are either sources or destinations of DetNet
216 flows. And end system may or may not be DetNet transport
217 layer aware or DetNet service layer aware.
219 link
220 A connection between two DetNet nodes. It may be composed of
221 a physical link or a sub-network technology that can provide
222 appropriate traffic delivery for DetNet flows.
224 DetNet system
225 A DetNet aware end system, transit node, or relay node.
226 "DetNet" may be omitted in some text.
228 DetNet relay node
229 A DetNet node including a service layer function that
230 interconnects different DetNet transport layer paths to
231 provide service protection. A DetNet relay node can be a
232 bridge, a router, a firewall, or any other system that
233 participates in the DetNet service layer. It typically
234 incorporates DetNet transport layer functions as well, in
235 which case it is collocated with a transit node.
237 PEF A Packet Elimination Function (PEF) eliminates duplicate
238 copies of packets to prevent excess packets flooding the
239 network or duplicate packets being sent out of the DetNet
240 domain. PEF can be implemented by an edge node, a relay
241 node, or an end system.
243 PRF A Packet Replication Function (PRF) replicates DetNet flow
244 packets and forwards them to one or more next hops in the
245 DetNet domain. The number of packet copies sent to each next
246 hop is a DetNet flow specific parameter at the node doing the
247 replication. PRF can be implemented by an edge node, a relay
248 node, or an end system.
250 PREOF Collective name for Packet Replication, Elimination, and
251 Ordering Functions.
253 POF A Packet Ordering Function (POF) re-orders packets within a
254 DetNet flow that are received out of order. This function
255 can be implemented by an edge node, a relay node, or an end
256 system.
258 reservation
259 The set of resources allocated between a source and one or
260 more destinations through transit nodes and subnets
261 associated with a DetNet flow, to provide the provisioned
262 DetNet service.
264 DetNet service layer
265 The layer at which A DetNet service, e.g., service protection
266 is provided.
268 DetNet service proxy
269 Maps between App-flows and DetNet flows.
271 DetNet source
272 An end system capable of originating a DetNet flow.
274 DetNet transit node
275 A node operating at the DetNet transport layer, that utilizes
276 link layer and/or network layer switching across multiple
277 links and/or sub-networks to provide paths for DetNet service
278 layer functions. Typically provides congestion protection
279 over those paths. An MPLS LSR is an example of a DetNet
280 transit node.
282 DetNet transport layer
283 The layer that optionally provides congestion protection for
284 DetNet flows over paths provided by the underlying network.
286 2.2. IEEE 802.1 TSN to DetNet dictionary
288 This section also serves as a dictionary for translating from the
289 terms used by the Time-Sensitive Networking (TSN) Task Group
290 [IEEE802.1TSNTG] of the IEEE 802.1 WG to those of the DetNet WG.
292 Listener
293 The IEEE 802.1 term for a destination of a DetNet flow.
295 relay system
296 The IEEE 802.1 term for a DetNet intermediate node.
298 Stream
299 The IEEE 802.1 term for a DetNet flow.
301 Talker
302 The IEEE 802.1 term for the source of a DetNet flow.
304 3. Providing the DetNet Quality of Service
306 3.1. Primary goals defining the DetNet QoS
308 The DetNet Quality of Service can be expressed in terms of:
310 o Minimum and maximum end-to-end latency from source to destination;
311 timely delivery, and bounded jitter (packet delay variation)
312 derived from these constraints.
314 o Packet loss ratio, under various assumptions as to the operational
315 states of the nodes and links.
317 o An upper bound on out-of-order packet delivery. It is worth
318 noting that some DetNet applications are unable to tolerate any
319 out-of-order delivery.
321 It is a distinction of DetNet that it is concerned solely with worst-
322 case values for the end-to-end latency, jitter, and misordering.
323 Average, mean, or typical values are of little interest, because they
324 do not affect the ability of a real-time system to perform its tasks.
325 In general, a trivial priority-based queuing scheme will give better
326 average latency to a data flow than DetNet, but of course, the worst-
327 case latency can be essentially unbounded.
329 Three techniques are used by DetNet to provide these qualities of
330 service:
332 o Congestion protection (Section 3.2.1).
334 o Service protection (Section 3.2.2).
336 o Explicit routes (Section 3.2.3).
338 Congestion protection operates by allocating resources along the path
339 of a DetNet flow, e.g., buffer space or link bandwidth. Congestion
340 protection greatly reduces, or even eliminates entirely, packet loss
341 due to output packet congestion within the network, but it can only
342 be supplied to a DetNet flow that is limited at the source to a
343 maximum packet size and transmission rate. Note that congestion
344 protection provided via congestion detection and notification is
345 explicitly excluded from consideration in DetNet, as it serves a
346 different set of applications.
348 Congestion protection addresses two of the DetNet QoS requirements:
349 latency and packet loss. Given that DetNet nodes have a finite
350 amount of buffer space, congestion protection necessarily results in
351 a maximum end-to-end latency. It also addresses the largest
352 contribution to packet loss, which is buffer congestion.
354 After congestion, the most important contributions to packet loss are
355 typically from random media errors and equipment failures. Service
356 protection is the name for the mechanisms used by DetNet to address
357 these losses. The mechanisms employed are constrained by the
358 requirement to meet the users' latency requirements. Packet
359 replication and elimination (Section 3.2.2) and packet encoding
360 (Section 3.2.2.3) are described in this document to provide service
361 protection; others may be found. For instance, packet encoding can
362 be used to provide service protection against random media errors,
363 packet replication and elimination can be used to provide service
364 protection against equipment failures. This mechanism distributes
365 the contents of DetNet flows over multiple paths in time and/or
366 space, so that the loss of some of the paths does need not cause the
367 loss of any packets.
369 The paths are typically (but not necessarily) explicit routes, so
370 that they do not normally suffer temporary interruptions caused by
371 the convergence of routing or bridging protocols.
373 These three techniques can be applied independently, giving eight
374 possible combinations, including none (no DetNet), although some
375 combinations are of wider utility than others. This separation keeps
376 the protocol stack coherent and maximizes interoperability with
377 existing and developing standards in this (IETF) and other Standards
378 Development Organizations. Some examples of typical expected
379 combinations:
381 o Explicit routes plus service protection are exactly the techniques
382 employed by seamless redundancy mechanisms applied on a ring
383 topology as described, e.g., in [IEC62439-3-2016]. In this
384 example, explicit routes are achieved by limiting the physical
385 topology of the network to a ring. Sequentialization,
386 replication, and duplicate elimination are facilitated by packet
387 tags added at the front or the end of Ethernet frames. [RFC8227]
388 provides another example in the context of MPLS.
390 o Congestion protection alone is offered by IEEE 802.1 Audio Video
391 bridging [IEEE802.1BA]. As long as the network suffers no
392 failures, zero congestion loss can be achieved through the use of
393 a reservation protocol (MSRP [IEEE802.1Q-2018]), shapers in every
394 bridge, and proper dimensioning.
396 o Using all three together gives maximum protection.
398 There are, of course, simpler methods available (and employed, today)
399 to achieve levels of latency and packet loss that are satisfactory
400 for many applications. Prioritization and over-provisioning is one
401 such technique. However, these methods generally work best in the
402 absence of any significant amount of non-critical traffic in the
403 network (if, indeed, such traffic is supported at all), or work only
404 if the critical traffic constitutes only a small portion of the
405 network's theoretical capacity, or work only if all systems are
406 functioning properly, or in the absence of actions by end systems
407 that disrupt the network's operations.
409 There are any number of methods in use, defined, or in progress for
410 accomplishing each of the above techniques. It is expected that this
411 DetNet Architecture will assist various vendors, users, and/or
412 "vertical" Standards Development Organizations (dedicated to a single
413 industry) to make selections among the available means of
414 implementing DetNet networks.
416 3.2. Mechanisms to achieve DetNet QoS
418 3.2.1. Congestion protection
420 3.2.1.1. Eliminate congestion loss
422 The primary means by which DetNet achieves its QoS assurances is to
423 reduce, or even completely eliminate, congestion within a node as a
424 cause of packet loss. Given that a DetNet flow cannot be throttled,
425 this can be achieved only by the provision of sufficient buffer
426 storage at each hop through the network to ensure that no packets are
427 dropped due to a lack of buffer storage.
429 Ensuring adequate buffering requires, in turn, that the source, and
430 every intermediate node along the path to the destination (or nearly
431 every node, see Section 4.3.3) be careful to regulate its output to
432 not exceed the data rate for any DetNet flow, except for brief
433 periods when making up for interfering traffic. Any packet sent
434 ahead of its time potentially adds to the number of buffers required
435 by the next hop and may thus exceed the resources allocated for a
436 particular DetNet flow.
438 The low-level mechanisms described in Section 4.5 provide the
439 necessary regulation of transmissions by an end system or
440 intermediate node to provide congestion protection. The allocation
441 of the bandwidth and buffers for a DetNet flow requires provisioning
442 A DetNet node may have other resources requiring allocation and/or
443 scheduling, that might otherwise be over-subscribed and trigger the
444 rejection of a reservation.
446 3.2.1.2. Jitter Reduction
448 A core objective of DetNet is to enable the convergence of sensitive
449 non-IP networks onto a common network infrastructure. This requires
450 the accurate emulation of currently deployed mission-specific
451 networks, which for example rely on point-to-point analog (e.g.,
452 4-20mA modulation) and serial-digital cables (or buses) for highly
453 reliable, synchronized and jitter-free communications. While the
454 latency of analog transmissions is basically the speed of light,
455 legacy serial links are usually slow (in the order of Kbps) compared
456 to, say, GigE, and some latency is usually acceptable. What is not
457 acceptable is the introduction of excessive jitter, which may, for
458 instance, affect the stability of control systems.
460 Applications that are designed to operate on serial links usually do
461 not provide services to recover the jitter, because jitter simply
462 does not exist there. DetNet flows are generally expected to be
463 delivered in-order and the precise time of reception influences the
464 processes. In order to converge such existing applications, there is
465 a desire to emulate all properties of the serial cable, such as clock
466 transportation, perfect flow isolation and fixed latency. While
467 minimal jitter (in the form of specifying minimum, as well as
468 maximum, end-to-end latency) is supported by DetNet, there are
469 practical limitations on packet-based networks in this regard. In
470 general, users are encouraged to use, instead of, "do this when you
471 get the packet," a combination of:
473 o Sub-microsecond time synchronization among all source and
474 destination end systems, and
476 o Time-of-execution fields in the application packets.
478 Jitter reduction is provided by the mechanisms described in
479 Section 4.5 that also provide congestion protection.
481 3.2.2. Service Protection
483 Service protection aims to mitigate or eliminate packet loss due to
484 equipment failures, random media and/or memory faults. These types
485 of packet loss can be greatly reduced by spreading the data over
486 multiple disjoint forwarding paths. Various service protection
487 methods are described in [RFC6372], e.g., 1+1 linear protection.
488 This section describes the functional details of an additional method
489 in Section 3.2.2.2, which can be implemented as described in
490 Section 3.2.2.3 or as specified in [I-D.ietf-detnet-dp-sol-mpls] in
491 order to provide 1+n hitless protection. The appropriate service
492 protection mechanism depends on the scenario and the requirements.
494 3.2.2.1. In-Order Delivery
496 Out-of-order packet delivery can be a side effect of service
497 protection. Packets delivered out-of-order impact the amount of
498 buffering needed at the destination to properly process the received
499 data. Such packets also influence the jitter of a flow. The DetNet
500 service includes maximum allowed misordering as a constraint. Zero
501 misordering would be a valid service constraint to reflect that the
502 end system(s) of the flow cannot tolerate any out-of-order delivery.
503 DetNet Packet Ordering Functionality (POF) (Section 3.2.2.2) can be
504 used to provide in-order delivery.
506 3.2.2.2. Packet Replication and Elimination
508 This section describes a service protection method that sends copies
509 of the same packets over multiple paths.
511 The DetNet service layer includes the packet replication (PRF), the
512 packet elimination (PEF), and the packet ordering functionality (POF)
513 for use in DetNet edge, relay node, and end system packet processing.
514 Either of these functions can be enabled in a DetNet edge node, relay
515 node or end system. The collective name for all three functions is
516 PREOF. The packet replication and elimination service protection
517 method altogether involves four capabilities:
519 o Providing sequencing information to the packets of a DetNet
520 compound flow. This may be done by adding a sequence number or
521 time stamp as part of DetNet, or may be inherent in the packet,
522 e.g., in a transport protocol, or associated to other physical
523 properties such as the precise time (and radio channel) of
524 reception of the packet. This is typically done once, at or near
525 the source.
527 o The Packet Replication Function (PRF) replicates these packets
528 into multiple DetNet member flows and typically sends them along
529 multiple different paths to the destination(s), e.g., over the
530 explicit routes of Section 3.2.3. The location within a node, and
531 the mechanism used for the PRF is implementation specific.
533 o The Packet Elimination Function (PEF) eliminates duplicate packets
534 of a DetNet flow based on the sequencing information and a history
535 of received packets. The output of the PEF is always a single
536 packet. This may be done at any node along the path to save
537 network resources further downstream, in particular if multiple
538 Replication points exist. But the most common case is to perform
539 this operation at the very edge of the DetNet network, preferably
540 in or near the receiver. The location within a node, and
541 mechanism used for the PEF is implementation specific.
543 o The Packet Ordering Function (POF) uses the sequencing information
544 to re-order a DetNet flow's packets that are received out of
545 order.
547 The order in which a node applies PEF, POF, and PRF to a DetNet flow
548 is implementation specific.
550 Some service protection mechanisms rely on switching from one flow to
551 another when a failure of a flow is detected. Contrarily, packet
552 replication and elimination combines the DetNet member flows sent
553 along multiple different paths, and performs a packet-by-packet
554 selection of which to discard, e.g., based on sequencing information.
556 In the simplest case, this amounts to replicating each packet in a
557 source that has two interfaces, and conveying them through the
558 network, along separate (disjoint non-SRLG) paths, to the similarly
559 dual-homed destinations, that discard the extras. This ensures that
560 one path (with zero congestion loss) remains, even if some
561 intermediate node fails. The sequencing information can also be used
562 for loss detection and for re-ordering.
564 DetNet relay nodes in the network can provide replication and
565 elimination facilities at various points in the network, so that
566 multiple failures can be accommodated.
568 This is shown in Figure 1, where the two relay nodes each replicate
569 (R) the DetNet flow on input, sending the DetNet member flows to both
570 the other relay node and to the end system, and eliminate duplicates
571 (E) on the output interface to the right-hand end system. Any one
572 link in the network can fail, and the DetNet compound flow can still
573 get through. Furthermore, two links can fail, as long as they are in
574 different segments of the network.
576 > > > > > > > > > relay > > > > > > > >
577 > /------------+ R node E +------------\ >
578 > / v + ^ \ >
579 end R + v | ^ + E end
580 system + v | ^ + system
581 > \ v + ^ / >
582 > \------------+ R relay E +-----------/ >
583 > > > > > > > > > node > > > > > > > >
585 Figure 1: Packet replication and elimination
587 Packet replication and elimination does not react to and correct
588 failures; it is entirely passive. Thus, intermittent failures,
589 mistakenly created packet filters, or misrouted data is handled just
590 the same as the equipment failures that are handled by typical
591 routing and bridging protocols.
593 If packet replication and elimination is used over paths providing
594 congestion protection (Section 3.2.1), and member flows that take
595 different-length paths through the network are combined, a merge
596 point may require extra buffering to equalize the delays over the
597 different paths. This equalization ensures that the resultant
598 compound flow will not exceed its contracted bandwidth even after one
599 or the other of the paths is restored after a failure. The extra
600 buffering can be also used to provide in-order delivery.
602 3.2.2.3. Packet encoding for service protection
604 There are methods for using multiple paths to provide service
605 protection that involve encoding the information in a packet
606 belonging to a DetNet flow into multiple transmission units,
607 combining information from multiple packets into any given
608 transmission unit. Such techniques, also known as "network coding",
609 can be used as a DetNet service protection technique.
611 3.2.3. Explicit routes
613 In networks controlled by typical dynamic control protocols such as
614 IS-IS or OSPF, a network topology event in one part of the network
615 can impact, at least briefly, the delivery of data in parts of the
616 network remote from the failure or recovery event. Even the use of
617 redundant paths through a network defined, e.g., as defined by
618 [RFC6372] do not eliminate the chances of packet loss. Furthermore,
619 out-of-order packet delivery can be a side effect of route changes.
621 Many real-time networks rely on physical rings of two-port devices,
622 with a relatively simple ring control protocol. This supports
623 redundant paths for service protection with a minimum of wiring. As
624 an additional benefit, ring topologies can often utilize different
625 topology management protocols than those used for a mesh network,
626 with a consequent reduction in the response time to topology changes.
627 Of course, this comes at some cost in terms of increased hop count,
628 and thus latency, for the typical path.
630 In order to get the advantages of low hop count and still ensure
631 against even very brief losses of connectivity, DetNet employs
632 explicit routes, where the path taken by a given DetNet flow does not
633 change, at least immediately, and likely not at all, in response to
634 network topology events. Service protection (Section 3.2.2 or
635 Section 3.2.2.3) over explicit routes provides a high likelihood of
636 continuous connectivity. Explicit routes can be established in
637 various ways, e.g., with RSVP-TE [RFC3209], with Segment Routing (SR)
638 [RFC8402], via a Software Defined Networking approach [RFC7426], with
639 IS-IS [RFC7813], etc. Explicit routes are typically used in MPLS TE
640 LSPs.
642 Out-of-order packet delivery can be a side effect of distributing a
643 single flow over multiple paths especially when there is a change
644 from one path to another when combining the flow. This is
645 irrespective of the distribution method used, and also applies to
646 service protection over explicit routes. As described in
647 Section 3.2.2.1, out-of-order packets influence the jitter of a flow
648 and impact the amount of buffering needed to process the data;
649 therefore, DetNet service includes maximum allowed misordering as a
650 constraint. The use of explicit routes helps to provide in-order
651 delivery because there is no immediate route change with the network
652 topology, but the changes are plannable as they are between the
653 different explicit routes.
655 3.3. Secondary goals for DetNet
657 Many applications require DetNet to provide additional services,
658 including coexistence with other QoS mechanisms Section 3.3.1 and
659 protection against misbehaving transmitters Section 3.3.2.
661 3.3.1. Coexistence with normal traffic
663 A DetNet network supports the dedication of a high proportion (e.g.
664 75%) of the network bandwidth to DetNet flows. But, no matter how
665 much is dedicated for DetNet flows, it is a goal of DetNet to coexist
666 with existing Class of Service schemes (e.g., DiffServ). It is also
667 important that non-DetNet traffic not disrupt the DetNet flow, of
668 course (see Section 3.3.2 and Section 5). For these reasons:
670 o Bandwidth (transmission opportunities) not utilized by a DetNet
671 flow is available to non-DetNet packets (though not to other
672 DetNet flows).
674 o DetNet flows can be shaped or scheduled, in order to ensure that
675 the highest-priority non-DetNet packet is also ensured a worst-
676 case latency (at any given hop).
678 o When transmission opportunities for DetNet flows are scheduled in
679 detail, then the algorithm constructing the schedule should leave
680 sufficient opportunities for non-DetNet packets to satisfy the
681 needs of the users of the network. Detailed scheduling can also
682 permit the time-shared use of buffer resources by different DetNet
683 flows.
685 Ideally, the net effect of the presence of DetNet flows in a network
686 on the non-DetNet packets is primarily a reduction in the available
687 bandwidth.
689 3.3.2. Fault Mitigation
691 One key to building robust real-time systems is to reduce the
692 infinite variety of possible failures to a number that can be
693 analyzed with reasonable confidence. DetNet aids in the process by
694 allowing for filters and policers to detect DetNet packets received
695 on the wrong interface, or at the wrong time, or in too great a
696 volume, and to then take actions such as discarding the offending
697 packet, shutting down the offending DetNet flow, or shutting down the
698 offending interface.
700 It is also essential that filters and service remarking be employed
701 at the network edge to prevent non-DetNet packets from being mistaken
702 for DetNet packets, and thus impinging on the resources allocated to
703 DetNet packets.
705 There exist techniques, at present and/or in various stages of
706 standardization, that can perform these fault mitigation tasks that
707 deliver a high probability that misbehaving systems will have zero
708 impact on well-behaved DetNet flows, except of course, for the
709 receiving interface(s) immediately downstream of the misbehaving
710 device. Examples of such techniques include traffic policing
711 functions (e.g. [RFC2475]) and separating flows into per-flow rate-
712 limited queues.
714 4. DetNet Architecture
716 4.1. DetNet stack model
718 DetNet functionality (Section 3) is implemented in two adjacent
719 layers in the protocol stack: the DetNet service layer and the DetNet
720 transport layer. The DetNet service layer provides DetNet service,
721 e.g., service protection, to higher layers in the protocol stack and
722 applications. The DetNet transport layer supports DetNet service in
723 the underlying network, e.g., by providing explicit routes and
724 congestion protection to DetNet flows.
726 4.1.1. Representative Protocol Stack Model
728 Figure 2 illustrates a conceptual DetNet data plane layering model.
729 One may compare it to that in [IEEE802.1CB], Annex C.
731 | packets going | ^ packets coming ^
732 v down the stack v | up the stack |
733 +----------------------+ +-----------------------+
734 | Source | | Destination |
735 +----------------------+ +-----------------------+
736 | Service layer: | | Service layer: |
737 | Packet sequencing | | Duplicate elimination |
738 | Flow replication | | Flow merging |
739 | Packet encoding | | Packet decoding |
740 +----------------------+ +-----------------------+
741 | Transport layer: | | Transport layer: |
742 | Congestion prot. | | Congestion prot. |
743 | Explicit routes | | Explicit routes |
744 +----------------------+ +-----------------------+
745 | Lower layers | | Lower layers |
746 +----------------------+ +-----------------------+
747 v ^
748 \_________________________/
750 Figure 2: DetNet data plane protocol stack
752 Not all layers are required for any given application, or even for
753 any given network. The functionality shown in Figure 2 is:
755 Application
756 Shown as "source" and "destination" in the diagram.
758 Packet sequencing
759 As part of DetNet service protection, supplies the sequence
760 number for packet replication and elimination
761 (Section 3.2.2). Peers with Duplicate elimination. This
762 layer is not needed if a higher-layer transport protocol is
763 expected to perform any packet sequencing and duplicate
764 elimination required by the DetNet flow replication.
766 Duplicate elimination
767 As part of the DetNet service layer, based on the sequenced
768 number supplied by its peer, packet sequencing, Duplicate
769 elimination discards any duplicate packets generated by
770 DetNet flow replication. It can operate on member flows,
771 compound flows, or both. The replication may also be
772 inferred from other information such as the precise time of
773 reception in a scheduled network. The duplicate elimination
774 layer may also perform resequencing of packets to restore
775 packet order in a flow that was disrupted by the loss of
776 packets on one or another of the multiple paths taken.
778 Flow replication
779 As part of DetNet service protection, packets that belong to
780 a DetNet compound flow are replicated into two or more DetNet
781 member flows. This function is separate from packet
782 sequencing. Flow replication can be an explicit replication
783 and remarking of packets, or can be performed by, for
784 example, techniques similar to ordinary multicast
785 replication, albeit with resource allocation implications.
786 Peers with DetNet flow merging.
788 Flow merging
789 As part of DetNet service protection, merges DetNet member
790 flows together for packets coming up the stack belonging to a
791 specific DetNet compound flow. Peers with DetNet flow
792 replication. DetNet flow merging, together with packet
793 sequencing, duplicate elimination, and DetNet flow
794 replication perform packet replication and elimination
795 (Section 3.2.2).
797 Packet encoding
798 As part of DetNet service protection, as an alternative to
799 packet sequencing and flow replication, packet encoding
800 combines the information in multiple DetNet packets, perhaps
801 from different DetNet compound flows, and transmits that
802 information in packets on different DetNet member Flows.
803 Peers with Packet decoding.
805 Packet decoding
806 As part of DetNet service protection, as an alternative to
807 flow merging and duplicate elimination, packet decoding takes
808 packets from different DetNet member flows, and computes from
809 those packets the original DetNet packets from the compound
810 flows input to packet encoding. Peers with Packet encoding.
812 Congestion protection
813 The DetNet transport layer provides congestion protection.
814 See Section 4.5. The actual queuing and shaping mechanisms
815 are typically provided by underlying subnet layers, these can
816 be closely associated with the means of providing paths for
817 DetNet flows, the path and the congestion protection are
818 conflated in this figure.
820 Explicit routes
821 The DetNet transport layer provides mechanisms to ensure that
822 fixed paths are provided for DetNet flows. These explicit
823 paths avoid the impact of network convergence.
825 Operations, Administration, and Maintenance (OAM) leverages in-band
826 and out-of-band signaling that validates whether the service is
827 effectively obtained within QoS constraints. OAM is not shown in
828 Figure 2; it may reside in any number of the layers. OAM can involve
829 specific tagging added in the packets for tracing implementation or
830 network configuration errors; traceability enables to find whether a
831 packet is a replica, which relay node performed the replication, and
832 which segment was intended for the replica. Active and hybrid OAM
833 methods require additional bandwidth to perform fault management and
834 performance monitoring of the DetNet domain. OAM may, for instance,
835 generate special test probes or add OAM information into the data
836 packet.
838 The packet sequencing and replication elimination functions at the
839 source and destination ends of a DetNet compound flow may be
840 performed either in the end system or in a DetNet relay node.
842 4.1.2. DetNet Data Plane Overview
844 A "Deterministic Network" will be composed of DetNet enabled end
845 systems and nodes, i.e., edge nodes, relay nodes and collectively
846 deliver DetNet services. DetNet enabled nodes are interconnected via
847 transit nodes (e.g., LSRs) which support DetNet, but are not DetNet
848 service aware. All DetNet enabled nodes are connected to sub-
849 networks, where a point-to-point link is also considered as a simple
850 sub-network. These sub-networks will provide DetNet compatible
851 service for support of DetNet traffic. Examples of sub-networks
852 include MPLS TE, IEEE 802.1 TSN and OTN. Of course, multi-layer
853 DetNet systems may also be possible, where one DetNet appears as a
854 sub-network, and provides service to, a higher layer DetNet system.
855 A simple DetNet concept network is shown in Figure 3.
857 TSN Edge Transit Relay DetNet
858 End System Node Node Node End System
860 +---------+ +.........+ +---------+
861 | Appl. |<--:Svc Proxy:-- End to End Service ---------->| Appl. |
862 +---------+ +---------+ +---------+ +---------+
863 | TSN | |TSN| |Svc|<-- DetNet flow ---: Service :-->| Service |
864 +---------+ +---+ +---+ +---------+ +---------+ +---------+
865 |Transport| |Trp| |Trp| |Transport| |Trp| |Trp| |Transport|
866 +-------.-+ +-.-+ +-.-+ +--.----.-+ +-.-+ +-.-+ +---.-----+
867 : Link : / ,-----. \ : Link : / ,-----. \
868 +.......+ +-[ Sub ]-+ +........+ +-[ Sub ]-+
869 [Network] [Network]
870 `-----' `-----'
872 Figure 3: A Simple DetNet Enabled Network
874 Distinguishing the function of two DetNet data plane layers, the
875 DetNet service layer and the DetNet transport layer, helps to explore
876 and evaluate various combinations of the data plane solutions
877 available, some are illustrated in Figure 4. This separation of
878 DetNet layers, while helpful, should not be considered as formal
879 requirement. For example, some technologies may violate these strict
880 layers and still be able to deliver a DetNet service.
882 .
883 .
884 +-----------+
885 | Service | PW, UDP, GRE
886 +-----------+
887 | Transport | IPv6, IPv4, MPLS TE LSPs, MPLS SR
888 +-----------+
889 .
890 .
892 Figure 4: DetNet adaptation to data plane
894 In some networking scenarios, the end system initially provides a
895 DetNet flow encapsulation, which contains all information needed by
896 DetNet nodes (e.g., Real-time Transport Protocol (RTP) [RFC3550]
897 based DetNet flow transported over a native UDP/IP network or
898 PseudoWire). In other scenarios, the encapsulation formats might
899 differ significantly.
901 There are many valid options to create a data plane solution for
902 DetNet traffic by selecting a technology approach for the DetNet
903 service layer and also selecting a technology approach for the DetNet
904 transport layer. There are a high number of valid combinations.
906 One of the most fundamental differences between different potential
907 data plane options is the basic headers used by DetNet nodes. For
908 example, the basic service can be delivered based on an MPLS label or
909 an IP header. This decision impacts the basic forwarding logic for
910 the DetNet service layer. Note that in both cases, IP addresses are
911 used to address DetNet nodes. The selected DetNet transport layer
912 technology also needs to be mapped to the sub-net technology used to
913 interconnect DetNet nodes. For example, DetNet flows will need to be
914 mapped to TSN Streams.
916 4.1.3. Network reference model
918 Figure 5 shows another view of the DetNet service related reference
919 points and main components.
921 DetNet DetNet
922 end system end system
923 _ _
924 / \ +----DetNet-UNI (U) / \
925 /App\ | /App\
926 /-----\ | /-----\
927 | NIC | v ________ | NIC |
928 +--+--+ _____ / \ DetNet-UNI (U) --+ +--+--+
929 | / \__/ \ | |
930 | / +----+ +----+ \_____ | |
931 | / | | | | \_______ | |
932 +------U PE +----+ P +----+ \ _ v |
933 | | | | | | | ___/ \ |
934 | +--+-+ +----+ | +----+ | / \_ |
935 \ | | | | | / \ |
936 \ | +----+ +--+-+ +--+PE |------ U-----+
937 \ | | | | | | | | | \_ _/
938 \ +---+ P +----+ P +--+ +----+ | \____/
939 \___ | | | | /
940 \ +----+__ +----+ DetNet-1 DetNet-2
941 | \_____/ \___________/ |
942 | |
943 | | End-to-End service | | | |
944 <------------------------------------------------------------->
945 | | DetNet service | | | |
946 | <------------------------------------------------> |
947 | | | | | |
949 Figure 5: DetNet Service Reference Model (multi-domain)
951 DetNet-UNIs ("U" in Figure 5) are assumed in this document to be
952 packet-based reference points and provide connectivity over the
953 packet network. A DetNet-UNI may provide multiple functions, e.g.,
954 it may add networking technology specific encapsulation to the DetNet
955 flows if necessary; it may provide status of the availability of the
956 resources associated with a reservation; it may provide a
957 synchronization service for the end system; it may carry enough
958 signaling to place the reservation in a network without a controller,
959 or if the controller only deals with the network but not the end
960 systems. Internal reference points of end systems (between the
961 application and the NIC) are more challenging from control
962 perspective and they may have extra requirements (e.g., in-order
963 delivery is expected in end system internal reference points, whereas
964 it is considered optional over the DetNet-UNI).
966 4.2. DetNet systems
968 4.2.1. End system
970 The native data flow between the source/destination end systems is
971 referred to as application-flow (App-flow). The traffic
972 characteristics of an App-flow can be CBR (constant bit rate) or VBR
973 (variable bit rate) and can have L1 or L2 or L3 encapsulation (e.g.,
974 TDM (time-division multiplexing), Ethernet, IP). These
975 characteristics are considered as input for resource reservation and
976 might be simplified to ensure determinism during transport (e.g.,
977 making reservations for the peak rate of VBR traffic, etc.).
979 An end system may or may not be DetNet transport layer aware or
980 DetNet service layer aware. That is, an end system may or may not
981 contain DetNet specific functionality. End systems with DetNet
982 functionalities may have the same or different transport layer as the
983 connected DetNet domain. Categorization of end systems are shown in
984 Figure 6.
986 End system
987 |
988 |
989 | DetNet aware ?
990 / \
991 +------< >------+
992 NO | \ / | YES
993 | v |
994 DetNet unaware |
995 End system |
996 | Service/
997 | Transport
998 / \ aware ?
999 +--------< >-------------+
1000 t-aware | \ / | s-aware
1001 | v |
1002 | | both |
1003 | | |
1004 DetNet t-aware | DetNet s-aware
1005 End system | End system
1006 v
1007 DetNet st-aware
1008 End system
1010 Figure 6: Categorization of end systems
1012 Note some known use case examples for end systems:
1014 o DetNet unaware: The classic case requiring service proxies.
1016 o DetNet t-aware: A DetNet transport-aware system. It knows about
1017 some TSN functions (e.g., reservation), but not about service
1018 protection.
1020 o DetNet s-aware: A DetNet service-aware system. It supplies
1021 sequence numbers, but doesn't know about zero congestion loss.
1023 o DetNet st-aware: A full functioning DetNet end system, it has
1024 DetNet functionalities and usually the same forwarding paradigm as
1025 the connected DetNet domain. It can be treated as an integral
1026 part of the DetNet domain.
1028 4.2.2. DetNet edge, relay, and transit nodes
1030 As shown in Figure 3, DetNet edge nodes providing proxy service and
1031 DetNet relay nodes providing the DetNet service layer are DetNet-
1032 aware, and DetNet transit nodes need only be aware of the DetNet
1033 transport layer.
1035 In general, if a DetNet flow passes through one or more DetNet-
1036 unaware network nodes between two DetNet nodes providing the DetNet
1037 transport layer for that flow, there is a potential for disruption or
1038 failure of the DetNet QoS. A network administrator needs to ensure
1039 that the DetNet-unaware network nodes are configured to minimize the
1040 chances of packet loss and delay, and provision enough extra buffer
1041 space in the DetNet transit node following the DetNet-unaware network
1042 nodes to absorb the induced latency variations.
1044 4.3. DetNet flows
1046 4.3.1. DetNet flow types
1048 A DetNet flow can have different formats while it is transported
1049 between the peer end systems. Therefore, the following possible
1050 types / formats of a DetNet flow are distinguished in this document:
1052 o App-flow: native format of the data carried over a DetNet flow.
1053 It does not contain any DetNet related attributes.
1055 o DetNet-t-flow: specific format of a DetNet flow. Only requires
1056 the congestion / latency features provided by the DetNet transport
1057 layer.
1059 o DetNet-s-flow: specific format of a DetNet flow. Only requires
1060 the service protection feature ensured by the DetNet service
1061 layer.
1063 o DetNet-st-flow: specific format of a DetNet flow. It requires
1064 both DetNet service layer and DetNet transport layer functions
1065 during forwarding.
1067 4.3.2. Source transmission behavior
1069 For the purposes of congestion protection, DetNet flows can be
1070 synchronous or asynchronous. In synchronous DetNet flows, at least
1071 the intermediate nodes (and possibly the end systems) are closely
1072 time synchronized, typically to better than 1 microsecond. By
1073 transmitting packets from different DetNet flows or classes of DetNet
1074 flows at different times, using repeating schedules synchronized
1075 among the intermediate nodes, resources such as buffers and link
1076 bandwidth can be shared over the time domain among different DetNet
1077 flows. There is a tradeoff among techniques for synchronous DetNet
1078 flows between the burden of fine-grained scheduling and the benefit
1079 of reducing the required resources, especially buffer space.
1081 In contrast, asynchronous DetNet flows are not coordinated with a
1082 fine-grained schedule, so relay and end systems must assume worst-
1083 case interference among DetNet flows contending for buffer resources.
1084 Asynchronous DetNet flows are characterized by:
1086 o A maximum packet size;
1088 o An observation interval; and
1090 o A maximum number of transmissions during that observation
1091 interval.
1093 These parameters, together with knowledge of the protocol stack used
1094 (and thus the size of the various headers added to a packet), limit
1095 the number of bit times per observation interval that the DetNet flow
1096 can occupy the physical medium.
1098 The source is required not to exceed these limits in order to obtain
1099 DetNet service. If the source transmits less data than this limit
1100 allows, the unused resource such as link bandwidth can be made
1101 available by the system to non-DetNet packets. However, making those
1102 resources available to DetNet packets in other DetNet flows would
1103 serve no purpose. Those other DetNet flows have their own dedicated
1104 resources, on the assumption that all DetNet flows can use all of
1105 their resources over a long period of time.
1107 There is no provision in DetNet for throttling DetNet flows (reducing
1108 end-to-end transmission rate via any explicit congestion
1109 notification); the assumption is that a DetNet flow, to be useful,
1110 must be delivered in its entirety. That is, while any useful
1111 application is written to expect a certain number of lost packets,
1112 the real-time applications of interest to DetNet demand that the loss
1113 of data due to the network is an extraordinarily event.
1115 Although DetNet strives to minimize the changes required of an
1116 application to allow it to shift from a special-purpose digital
1117 network to an Internet Protocol network, one fundamental shift in the
1118 behavior of network applications is impossible to avoid: the
1119 reservation of resources before the application starts. In the first
1120 place, a network cannot deliver finite latency and practically zero
1121 packet loss to an arbitrarily high offered load. Secondly, achieving
1122 practically zero packet loss for unthrottled (though bandwidth
1123 limited) DetNet flows means that bridges and routers have to dedicate
1124 buffer resources to specific DetNet flows or to classes of DetNet
1125 flows. The requirements of each reservation have to be translated
1126 into the parameters that control each system's queuing, shaping, and
1127 scheduling functions and delivered to the hosts, bridges, and
1128 routers.
1130 4.3.3. Incomplete Networks
1132 The presence in the network of transit nodes or subnets that are not
1133 fully capable of offering DetNet services complicates the ability of
1134 the intermediate nodes and/or controller to allocate resources, as
1135 extra buffering must be allocated at points downstream from the non-
1136 DetNet intermediate node for a DetNet flow. This extra buffering may
1137 increase latency and/or jitter.
1139 4.4. Traffic Engineering for DetNet
1141 Traffic Engineering Architecture and Signaling (TEAS) [TEAS] defines
1142 traffic-engineering architectures for generic applicability across
1143 packet and non-packet networks. From a TEAS perspective, Traffic
1144 Engineering (TE) refers to techniques that enable operators to
1145 control how specific traffic flows are treated within their networks.
1147 Because if its very nature of establishing explicit optimized paths,
1148 Deterministic Networking can be seen as a new, specialized branch of
1149 Traffic Engineering, and inherits its architecture with a separation
1150 into planes.
1152 The Deterministic Networking architecture is thus composed of three
1153 planes, a (User) Application Plane, a Controller Plane, and a Network
1154 Plane, which echoes that of Figure 1 of Software-Defined Networking
1155 (SDN): Layers and Architecture Terminology [RFC7426].:
1157 4.4.1. The Application Plane
1159 Per [RFC7426], the Application Plane includes both applications and
1160 services. In particular, the Application Plane incorporates the User
1161 Agent, a specialized application that interacts with the end user /
1162 operator and performs requests for Deterministic Networking services
1163 via an abstract Flow Management Entity, (FME) which may or may not be
1164 collocated with (one of) the end systems.
1166 At the Application Plane, a management interface enables the
1167 negotiation of flows between end systems. An abstraction of the flow
1168 called a Traffic Specification (TSpec) provides the representation.
1169 This abstraction is used to place a reservation over the (Northbound)
1170 Service Interface and within the Application plane. It is associated
1171 with an abstraction of location, such as IP addresses and DNS names,
1172 to identify the end systems and eventually specify intermediate
1173 nodes.
1175 4.4.2. The Controller Plane
1177 The Controller Plane corresponds to the aggregation of the Control
1178 and Management Planes in [RFC7426], though Common Control and
1179 Measurement Plane (CCAMP) [CCAMP] makes an additional distinction
1180 between management and measurement. When the logical separation of
1181 the Control, Measurement and other Management entities is not
1182 relevant, the term Controller Plane is used for simplicity to
1183 represent them all, and the term Controller Plane Function (CPF)
1184 refers to any device operating in that plane, whether is it a Path
1185 Computation Element (PCE) [RFC4655], or a Network Management entity
1186 (NME), or a distributed control plane. The CPF is a core element of
1187 a controller, in charge of computing Deterministic paths to be
1188 applied in the Network Plane.
1190 A (Northbound) Service Interface enables applications in the
1191 Application Plane to communicate with the entities in the Controller
1192 Plane as illustrated in Figure 7.
1194 One or more CPF(s) collaborate to implement the requests from the FME
1195 as Per-Flow Per-Hop Behaviors installed in the intermediate nodes for
1196 each individual flow. The CPFs place each flow along a deterministic
1197 sequence of intermediate nodes so as to respect per-flow constraints
1198 such as security and latency, and optimize the overall result for
1199 metrics such as an abstract aggregated cost. The deterministic
1200 sequence can typically be more complex than a direct sequence and
1201 include redundancy path, with one or more packet replication and
1202 elimination points.
1204 4.4.3. The Network Plane
1206 The Network Plane represents the network devices and protocols as a
1207 whole, regardless of the Layer at which the network devices operate.
1208 It includes Forwarding Plane (data plane), Application, and
1209 Operational Plane (e.g., OAM) aspects.
1211 The network Plane comprises the Network Interface Cards (NIC) in the
1212 end systems, which are typically IP hosts, and intermediate nodes,
1213 which are typically IP routers and switches. Network-to-Network
1214 Interfaces such as used for Traffic Engineering path reservation in
1215 [RFC5921], as well as User-to-Network Interfaces (UNI) such as
1216 provided by the Local Management Interface (LMI) between network and
1217 end systems, are both part of the Network Plane, both in the control
1218 plane and the data plane.
1220 A Southbound (Network) Interface enables the entities in the
1221 Controller Plane to communicate with devices in the Network Plane as
1222 illustrated in Figure 7. This interface leverages and extends TEAS
1223 to describe the physical topology and resources in the Network Plane.
1225 End End
1226 System System
1228 -+-+-+-+-+-+-+ Northbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
1230 CPF CPF CPF CPF
1232 -+-+-+-+-+-+-+ Southbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
1234 intermediate intermed. intermed. intermed.
1235 Node Node Node Node
1236 NIC NIC
1237 intermediate intermed. intermed. intermed.
1238 Node Node Node Node
1240 Figure 7: Northbound and Southbound interfaces
1242 The intermediate nodes (and eventually the end systems NIC) expose
1243 their capabilities and physical resources to the controller (the
1244 CPF), and update the CPFs with their dynamic perception of the
1245 topology, across the Southbound Interface. In return, the CPFs set
1246 the per-flow paths up, providing a Flow Characterization that is more
1247 tightly coupled to the intermediate node Operation than a TSpec.
1249 At the Network plane, intermediate nodes may exchange information
1250 regarding the state of the paths, between adjacent systems and
1251 eventually with the end systems, and forward packets within
1252 constraints associated to each flow, or, when unable to do so,
1253 perform a last resort operation such as drop or declassify.
1255 This document focuses on the Southbound interface and the operation
1256 of the Network Plane.
1258 4.5. Queuing, Shaping, Scheduling, and Preemption
1260 DetNet achieves congestion protection and bounded delivery latency by
1261 reserving bandwidth and buffer resources at every hop along the path
1262 of the DetNet flow. The reservation itself is not sufficient,
1263 however. Implementors and users of a number of proprietary and
1264 standard real-time networks have found that standards for specific
1265 data plane techniques are required to enable these assurances to be
1266 made in a multi-vendor network. The fundamental reason is that
1267 latency variation in one system results in the need for extra buffer
1268 space in the next-hop system(s), which in turn, increases the worst-
1269 case per-hop latency.
1271 Standard queuing and transmission selection algorithms allow a
1272 central controller to compute the latency contribution of each
1273 transit node to the end-to-end latency, to compute the amount of
1274 buffer space required in each transit node for each incremental
1275 DetNet flow, and most importantly, to translate from a flow
1276 specification to a set of values for the managed objects that control
1277 each relay or end system. For example, the IEEE 802.1 WG has
1278 specified (and is specifying) a set of queuing, shaping, and
1279 scheduling algorithms that enable each transit node (bridge or
1280 router), and/or a central controller, to compute these values. These
1281 algorithms include:
1283 o A credit-based shaper [IEEE802.1Qav] (superseded by
1284 [IEEE802.1Q-2018]).
1286 o Time-gated queues governed by a rotating time schedule,
1287 synchronized among all transit nodes [IEEE802.1Qbv] (superseded by
1288 [IEEE802.1Q-2018]).
1290 o Synchronized double (or triple) buffers driven by synchronized
1291 time ticks. [IEEE802.1Qch] (superseded by [IEEE802.1Q-2018]).
1293 o Pre-emption of an Ethernet packet in transmission by a packet with
1294 a more stringent latency requirement, followed by the resumption
1295 of the preempted packet [IEEE802.1Qbu] (superseded by
1296 [IEEE802.1Q-2018]), [IEEE802.3br] (superseded by
1297 [IEEE802.3-2018]).
1299 While these techniques are currently embedded in Ethernet
1300 [IEEE802.3-2018] and bridging standards, we can note that they are
1301 all, except perhaps for packet preemption, equally applicable to
1302 other media than Ethernet, and to routers as well as bridges. Other
1303 media may have its own methods, see, e.g.,
1304 [I-D.ietf-6tisch-architecture], [RFC7554]. DetNet may include such
1305 definitions in the future, or may define how these techniques can be
1306 used by DetNet nodes.
1308 4.6. Service instance
1310 A Service instance represents all the functions required on a node to
1311 allow the end-to-end service between the UNIs.
1313 The DetNet network general reference model is shown in Figure 8 for a
1314 DetNet service scenario (i.e., between two DetNet-UNIs). In this
1315 figure, end systems ("A" and "B") are connected directly to the edge
1316 nodes of an IP/MPLS network ("PE1" and "PE2"). End systems
1317 participating in DetNet communication may require connectivity before
1318 setting up an App-flow that requires the DetNet service. Such a
1319 connectivity related service instance and the one dedicated for
1320 DetNet service share the same access. Packets belonging to a DetNet
1321 flow are selected by a filter configured on the access ("F1" and
1322 "F2"). As a result, data flow specific access ("access-A + F1" and
1323 "access-B + F2") are terminated in the flow specific service instance
1324 ("SI-1" and "SI-2"). A tunnel is used to provide connectivity
1325 between the service instances.
1327 The tunnel is used to transport exclusively the packets of the DetNet
1328 flow between "SI-1" and "SI-2". The service instances are configured
1329 to implement DetNet functions and a flow specific DetNet transport.
1330 The service instance and the tunnel may or may not be shared by
1331 multiple DetNet flows. Sharing the service instance by multiple
1332 DetNet flows requires properly populated forwarding tables of the
1333 service instance.
1335 access-A access-B
1336 <-----> <-------- tunnel ----------> <----->
1338 +---------+ ___ _ +---------+
1339 End system | +----+ | / \/ \_ | +----+ | End system
1340 "A" -------F1+ | | / \ | | +F2----- "B"
1341 | | +========+ IP/MPLS +=======+ | |
1342 | |SI-1| | \__ Net._/ | |SI-2| |
1343 | +----+ | \____/ | +----+ |
1344 |PE1 | | PE2|
1345 +---------+ +---------+
1347 Figure 8: DetNet network general reference model
1349 The tunnel between the service instances may have some special
1350 characteristics. For example, in case of a DetNet L3 service, there
1351 are differences in the usage of the PW for DetNet traffic compared to
1352 the network model described in [RFC6658]. In the DetNet scenario,
1353 the PW is likely to be used exclusively by the DetNet flow, whereas
1354 [RFC6658] states: "The packet PW appears as a single point-to-point
1355 link to the client layer. Network-layer adjacency formation and
1356 maintenance between the client equipment will follow the normal
1357 practice needed to support the required relationship in the client
1358 layer ... This packet PseudoWire is used to transport all of the
1359 required Layer-2 and Layer-3 protocols between LSR1 and LSR2".
1360 Further details are network technology specific and can be found in
1361 [I-D.ietf-detnet-dp-sol-mpls] and [I-D.ietf-detnet-dp-sol-ip].
1363 4.7. Flow identification at technology borders
1365 4.7.1. Exporting flow identification
1367 A DetNet node may need to map specific flows to lower layer flows (or
1368 Streams) in order to provide specific queuing and shaping services
1369 for specific flows. For example:
1371 o A non-IP, strictly L2 source end system X may be sending multiple
1372 flows to the same L2 destination end system Y. Those flows may
1373 include DetNet flows with different QoS requirements, and may
1374 include non-DetNet flows.
1376 o A router may be sending any number of flows to another router.
1377 Again, those flows may include DetNet flows with different QoS
1378 requirements, and may include non-DetNet flows.
1380 o Two routers may be separated by bridges. For these bridges to
1381 perform any required per-flow queuing and shaping, they must be
1382 able to identify the individual flows.
1384 o A Label Edge Router (LER) may have a Label Switched Path (LSP) set
1385 up for handling traffic destined for a particular IP address
1386 carrying only non-DetNet flows. If a DetNet flow to that same
1387 address is requested, a separate LSP may be needed, in order that
1388 all of the Label Switch Routers (LSRs) along the path to the
1389 destination give that flow special queuing and shaping.
1391 The need for a lower-layer node to be aware of individual higher-
1392 layer flows is not unique to DetNet. But, given the endless
1393 complexity of layering and relayering over tunnels that is available
1394 to network designers, DetNet needs to provide a model for flow
1395 identification that is better than packet inspection. That is not to
1396 say that packet inspection to layer 4 or 5 addresses will not be
1397 used, or the capability standardized; but, there are alternatives.
1399 A DetNet relay node can connect DetNet flows on different paths using
1400 different flow identification methods. For example:
1402 o A single unicast DetNet flow passing from router A through a
1403 bridged network to router B may be assigned a TSN Stream
1404 identifier that is unique within that bridged network. The
1405 bridges can then identify the flow without accessing higher-layer
1406 headers. Of course, the receiving router must recognize and
1407 accept that TSN Stream.
1409 o A DetNet flow passing from LSR A to LSR B may be assigned a
1410 different label than that used for other flows to the same IP
1411 destination.
1413 In any of the above cases, it is possible that an existing DetNet
1414 flow can be an aggregate carrying multiple other DetNet flows. (Not
1415 to be confused with DetNet compound vs. member flows.) Of course,
1416 this requires that the aggregate DetNet flow be provisioned properly
1417 to carry the aggregated flows.
1419 Thus, rather than packet inspection, there is the option to export
1420 higher-layer information to the lower layer. The requirement to
1421 support one or the other method for flow identification (or both) is
1422 a complexity that is part of DetNet control models.
1424 4.7.2. Flow attribute mapping between layers
1426 Transport of DetNet flows over multiple technology domains may
1427 require that lower layers are aware of specific flows of higher
1428 layers. Such an "exporting of flow identification" is needed each
1429 time when the forwarding paradigm is changed on the transport path
1430 (e.g., two LSRs are interconnected by a L2 bridged domain, etc.).
1431 The three representative forwarding methods considered for
1432 deterministic networking are:
1434 o IP routing
1436 o MPLS label switching
1438 o Ethernet bridging
1440 A packet with corresponding Flow-IDs is illustrated in Figure 9,
1441 which also indicates where each Flow-ID can be added or removed.
1443 add/remove add/remove
1444 Eth Flow-ID IP Flow-ID
1445 | |
1446 v v
1447 +-----------------------------------------------------------+
1448 | | | | |
1449 | Eth | MPLS | IP | Application data |
1450 | | | | |
1451 +-----------------------------------------------------------+
1452 ^
1453 |
1454 add/remove
1455 MPLS Flow-ID
1457 Figure 9: Packet with multiple Flow-IDs
1459 The additional (domain specific) Flow-ID can be
1461 o created by a domain specific function or
1463 o derived from the Flow-ID added to the App-flow.
1465 The Flow-ID must be unique inside a given domain. Note that the
1466 Flow-ID added to the App-flow is still present in the packet, but
1467 transport nodes may lack the function to recognize it; that's why the
1468 additional Flow-ID is added.
1470 4.7.3. Flow-ID mapping examples
1472 IP nodes and MPLS nodes are assumed to be configured to push such an
1473 additional (domain specific) Flow-ID when sending traffic to an
1474 Ethernet switch (as shown in the examples below).
1476 Figure 10 shows a scenario where an IP end system ("IP-A") is
1477 connected via two Ethernet switches ("ETH-n") to an IP router ("IP-
1478 1").
1480 IP domain
1481 <-----------------------------------------------
1483 +======+ +======+
1484 |L3-ID | |L3-ID |
1485 +======+ /\ +-----+ +======+
1486 / \ Forward as | |
1487 /IP-A\ per ETH-ID |IP-1 | Recognize
1488 Push ------> +-+----+ | +---+-+ <----- ETH-ID
1489 ETH-ID | +----+-----+ |
1490 | v v |
1491 | +-----+ +-----+ |
1492 +------+ | | +---------+
1493 +......+ |ETH-1+----+ETH-2| +======+
1494 .L3-ID . +-----+ +-----+ |L3-ID |
1495 +======+ +......+ +======+
1496 |ETH-ID| .L3-ID . |ETH-ID|
1497 +======+ +======+ +------+
1498 |ETH-ID|
1499 +======+
1501 Ethernet domain
1502 <---------------->
1504 Figure 10: IP nodes interconnected by an Ethernet domain
1506 End system "IP-A" uses the original App-flow specific ID ("L3-ID"),
1507 but as it is connected to an Ethernet domain it has to push an
1508 Ethernet-domain specific flow-ID ("VID + multicast MAC address",
1509 referred as "ETH-ID") before sending the packet to "ETH-1" node.
1510 Ethernet switch "ETH-1" can recognize the data flow based on the
1511 "ETH-ID" and it does forwarding toward "ETH-2". "ETH-2" switches the
1512 packet toward the IP router. "IP-1" must be configured to receive
1513 the Ethernet Flow-ID specific multicast flow, but (as it is an L3
1514 node) it decodes the data flow ID based on the "L3-ID" fields of the
1515 received packet.
1517 Figure 11 shows a scenario where MPLS domain nodes ("PE-n" and "P-m")
1518 are connected via two Ethernet switches ("ETH-n").
1520 MPLS domain
1521 <----------------------------------------------->
1523 +=======+ +=======+
1524 |MPLS-ID| |MPLS-ID|
1525 +=======+ +-----+ +-----+ +=======+ +-----+
1526 | | Forward as | | | |
1527 |PE-1 | per ETH-ID | P-2 +-----------+ PE-2|
1528 Push -----> +-+---+ | +---+-+ +-----+
1529 ETH-ID | +-----+----+ | \ Recognize
1530 | v v | +-- ETH-ID
1531 | +-----+ +-----+ |
1532 +---+ | | +----+
1533 +.......+ |ETH-1+----+ETH-2| +=======+
1534 .MPLS-ID. +-----+ +-----+ |MPLS-ID|
1535 +=======+ +=======+
1536 |ETH-ID | +.......+ |ETH-ID |
1537 +=======+ .MPLS-ID. +-------+
1538 +=======+
1539 |ETH-ID |
1540 +=======+
1541 Ethernet domain
1542 <---------------->
1544 Figure 11: MPLS nodes interconnected by an Ethernet domain
1546 "PE-1" uses the MPLS specific ID ("MPLS-ID"), but as it is connected
1547 to an Ethernet domain it has to push an Ethernet-domain specific
1548 flow-ID ("VID + multicast MAC address", referred as "ETH-ID") before
1549 sending the packet to "ETH-1". Ethernet switch "ETH-1" can recognize
1550 the data flow based on the "ETH-ID" and it does forwarding toward
1551 "ETH-2". "ETH-2" switches the packet toward the MPLS node ("P-2").
1552 "P-2" must be configured to receive the Ethernet Flow-ID specific
1553 multicast flow, but (as it is an MPLS node) it decodes the data flow
1554 ID based on the "MPLS-ID" fields of the received packet.
1556 One can appreciate from the above example that, when the means used
1557 for DetNet flow identification is altered or exported, the means for
1558 encoding the sequence number information must similarly be altered or
1559 exported.
1561 4.8. Advertising resources, capabilities and adjacencies
1563 There are three classes of information that a central controller or
1564 distributed control plane needs to know that can only be obtained
1565 from the end systems and/or nodes in the network. When using a peer-
1566 to-peer control plane, some of this information may be required by a
1567 system's neighbors in the network.
1569 o Details of the system's capabilities that are required in order to
1570 accurately allocate that system's resources, as well as other
1571 systems' resources. This includes, for example, which specific
1572 queuing and shaping algorithms are implemented (Section 4.5), the
1573 number of buffers dedicated for DetNet allocation, and the worst-
1574 case forwarding delay and misordering.
1576 o The dynamic state of a node's DetNet resources.
1578 o The identity of the system's neighbors, and the characteristics of
1579 the link(s) between the systems, including the length (in
1580 nanoseconds) of the link(s).
1582 4.9. Scaling to larger networks
1584 Reservations for individual DetNet flows require considerable state
1585 information in each transit node, especially when adequate fault
1586 mitigation (Section 3.3.2) is required. The DetNet data plane, in
1587 order to support larger numbers of DetNet flows, must support the
1588 aggregation of DetNet flows. Such aggregated flows can be viewed by
1589 the transit nodes' data plane largely as individual DetNet flows.
1590 Without such aggregation, the per-relay system may limit the scale of
1591 DetNet networks. Example techniques that may be used include MPLS
1592 hierarchy and IP DiffServ Code Points (DSCPs).
1594 4.10. Compatibility with Layer-2
1596 Standards providing similar capabilities for bridged networks (only)
1597 have been and are being generated in the IEEE 802 LAN/MAN Standards
1598 Committee. The present architecture describes an abstract model that
1599 can be applicable both at Layer-2 and Layer-3, and over links not
1600 defined by IEEE 802.
1602 DetNet enabled end systems and intermediate nodes can be
1603 interconnected by sub-networks, i.e., Layer-2 technologies. These
1604 sub-networks will provide DetNet compatible service for support of
1605 DetNet traffic. Examples of sub-networks include MPLS TE, 802.1 TSN,
1606 and a point-to-point OTN link. Of course, multi-layer DetNet systems
1607 may be possible too, where one DetNet appears as a sub-network, and
1608 provides service to, a higher layer DetNet system.
1610 5. Security Considerations
1612 Security in the context of Deterministic Networking has an added
1613 dimension; the time of delivery of a packet can be just as important
1614 as the contents of the packet, itself. A man-in-the-middle attack,
1615 for example, can impose, and then systematically adjust, additional
1616 delays into a link, and thus disrupt or subvert a real-time
1617 application without having to crack any encryption methods employed.
1618 See [RFC7384] for an exploration of this issue in a related context.
1620 Furthermore, in a control system where millions of dollars of
1621 equipment, or even human lives, can be lost if the DetNet QoS is not
1622 delivered, one must consider not only simple equipment failures,
1623 where the box or wire instantly becomes perfectly silent, but complex
1624 errors such as can be caused by software failures. Because there is
1625 essential no limit to the kinds of failures that can occur,
1626 protecting against realistic equipment failures is indistinguishable,
1627 in most cases, from protecting against malicious behavior, whether
1628 accidental or intentional. See also Section 3.3.2.
1630 Security must cover:
1632 o the protection of the signaling protocol
1634 o the authentication and authorization of the controlling systems
1636 o the identification and shaping of the DetNet flows
1638 6. Privacy Considerations
1640 DetNet is provides a Quality of Service (QoS), and as such, does not
1641 directly raise any new privacy considerations.
1643 However, the requirement for every (or almost every) node along the
1644 path of a DetNet flow to identify DetNet flows may present an
1645 additional attack surface for privacy, should the DetNet paradigm be
1646 found useful in broader environments.
1648 7. IANA Considerations
1650 This document does not require an action from IANA.
1652 8. Acknowledgements
1654 The authors wish to thank Lou Berger, David Black, Stewart Bryant,
1655 Rodney Cummings, Ethan Grossman, Craig Gunther, Marcel Kiessling,
1656 Rudy Klecka, Jouni Korhonen, Erik Nordmark, Shitanshu Shah, Wilfried
1657 Steiner, George Swallow, Michael Johas Teener, Pat Thaler, Thomas
1658 Watteyne, Patrick Wetterwald, Karl Weber, Anca Zamfir, for their
1659 various contribution with this work.
1661 9. Informative References
1663 [CCAMP] IETF, "Common Control and Measurement Plane Working
1664 Group",
1665 .
1667 [I-D.ietf-6tisch-architecture]
1668 Thubert, P., "An Architecture for IPv6 over the TSCH mode
1669 of IEEE 802.15.4", draft-ietf-6tisch-architecture-14 (work
1670 in progress), April 2018.
1672 [I-D.ietf-detnet-dp-sol-ip]
1673 Korhonen, J. and B. Varga, "DetNet IP Data Plane
1674 Encapsulation", draft-ietf-detnet-dp-sol-ip-00 (work in
1675 progress), July 2018.
1677 [I-D.ietf-detnet-dp-sol-mpls]
1678 Korhonen, J. and B. Varga, "DetNet MPLS Data Plane
1679 Encapsulation", draft-ietf-detnet-dp-sol-mpls-00 (work in
1680 progress), July 2018.
1682 [I-D.ietf-detnet-problem-statement]
1683 Finn, N. and P. Thubert, "Deterministic Networking Problem
1684 Statement", draft-ietf-detnet-problem-statement-06 (work
1685 in progress), July 2018.
1687 [I-D.ietf-detnet-use-cases]
1688 Grossman, E., "Deterministic Networking Use Cases", draft-
1689 ietf-detnet-use-cases-17 (work in progress), June 2018.
1691 [IEC62439-3-2016]
1692 International Electrotechnical Commission (IEC) TC 65/SC
1693 65C - Industrial networks, "IEC 62439-3:2016 Industrial
1694 communication networks - High availability automation
1695 networks - Part 3: Parallel Redundancy Protocol (PRP) and
1696 High-availability Seamless Redundancy (HSR)", 2016,
1697 .
1699 [IEEE802.1BA]
1700 IEEE Standards Association, "IEEE Std 802.1BA-2011 Audio
1701 Video Bridging (AVB) Systems", 2011,
1702 .
1704 [IEEE802.1CB]
1705 IEEE Standards Association, "IEEE Std 802.1CB-2017 Frame
1706 Replication and Elimination for Reliability", 2017,
1707 .
1709 [IEEE802.1Q-2018]
1710 IEEE Standards Association, "IEEE Std 802.1Q-2018 Bridges
1711 and Bridged Networks", 2018,
1712 .
1715 [IEEE802.1Qav]
1716 IEEE Standards Association, "IEEE Std 802.1Qav-2009
1717 Bridges and Bridged Networks - Amendment 12: Forwarding
1718 and Queuing Enhancements for Time-Sensitive Streams",
1719 2009, .
1721 [IEEE802.1Qbu]
1722 IEEE Standards Association, "IEEE Std 802.1Qbu-2016
1723 Bridges and Bridged Networks - Amendment 26: Frame
1724 Preemption", 2016,
1725 .
1727 [IEEE802.1Qbv]
1728 IEEE Standards Association, "IEEE Std 802.1Qbv-2015
1729 Bridges and Bridged Networks - Amendment 25: Enhancements
1730 for Scheduled Traffic", 2015,
1731 .
1733 [IEEE802.1Qch]
1734 IEEE Standards Association, "IEEE Std 802.1Qch-2017
1735 Bridges and Bridged Networks - Amendment 29: Cyclic
1736 Queuing and Forwarding", 2017,
1737 .
1739 [IEEE802.1TSNTG]
1740 IEEE Standards Association, "IEEE 802.1 Time-Sensitive
1741 Networking Task Group", 2013,
1742 .
1744 [IEEE802.3-2018]
1745 IEEE Standards Association, "IEEE Std 802.3-2018 Standard
1746 for Ethernet", 2018, .
1749 [IEEE802.3br]
1750 IEEE Standards Association, "IEEE Std 802.3br-2016
1751 Standard for Ethernet Amendment 5: Specification and
1752 Management Parameters for Interspersing Express Traffic",
1753 2016, .
1755 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.
1756 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
1757 Functional Specification", RFC 2205, DOI 10.17487/RFC2205,
1758 September 1997, .
1760 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.,
1761 and W. Weiss, "An Architecture for Differentiated
1762 Services", RFC 2475, DOI 10.17487/RFC2475, December 1998,
1763 .
1765 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
1766 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
1767 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001,
1768 .
1770 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
1771 Jacobson, "RTP: A Transport Protocol for Real-Time
1772 Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550,
1773 July 2003, .
1775 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation
1776 Element (PCE)-Based Architecture", RFC 4655,
1777 DOI 10.17487/RFC4655, August 2006,
1778 .
1780 [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau,
1781 L., and L. Berger, "A Framework for MPLS in Transport
1782 Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010,
1783 .
1785 [RFC6372] Sprecher, N., Ed. and A. Farrel, Ed., "MPLS Transport
1786 Profile (MPLS-TP) Survivability Framework", RFC 6372,
1787 DOI 10.17487/RFC6372, September 2011,
1788 .
1790 [RFC6658] Bryant, S., Ed., Martini, L., Swallow, G., and A. Malis,
1791 "Packet Pseudowire Encapsulation over an MPLS PSN",
1792 RFC 6658, DOI 10.17487/RFC6658, July 2012,
1793 .
1795 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in
1796 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384,
1797 October 2014, .
1799 [RFC7426] Haleplidis, E., Ed., Pentikousis, K., Ed., Denazis, S.,
1800 Hadi Salim, J., Meyer, D., and O. Koufopavlou, "Software-
1801 Defined Networking (SDN): Layers and Architecture
1802 Terminology", RFC 7426, DOI 10.17487/RFC7426, January
1803 2015, .
1805 [RFC7554] Watteyne, T., Ed., Palattella, M., and L. Grieco, "Using
1806 IEEE 802.15.4e Time-Slotted Channel Hopping (TSCH) in the
1807 Internet of Things (IoT): Problem Statement", RFC 7554,
1808 DOI 10.17487/RFC7554, May 2015,
1809 .
1811 [RFC7813] Farkas, J., Ed., Bragg, N., Unbehagen, P., Parsons, G.,
1812 Ashwood-Smith, P., and C. Bowers, "IS-IS Path Control and
1813 Reservation", RFC 7813, DOI 10.17487/RFC7813, June 2016,
1814 .
1816 [RFC8227] Cheng, W., Wang, L., Li, H., van Helvoort, H., and J.
1817 Dong, "MPLS-TP Shared-Ring Protection (MSRP) Mechanism for
1818 Ring Topology", RFC 8227, DOI 10.17487/RFC8227, August
1819 2017, .
1821 [RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L.,
1822 Decraene, B., Litkowski, S., and R. Shakir, "Segment
1823 Routing Architecture", RFC 8402, DOI 10.17487/RFC8402,
1824 July 2018, .
1826 [TEAS] IETF, "Traffic Engineering Architecture and Signaling
1827 Working Group",
1828 .
1830 Authors' Addresses
1832 Norman Finn
1833 Huawei
1834 3755 Avocado Blvd.
1835 PMB 436
1836 La Mesa, California 91941
1837 US
1839 Phone: +1 925 980 6430
1840 Email: norman.finn@mail01.huawei.com
1841 Pascal Thubert
1842 Cisco Systems
1843 Village d'Entreprises Green Side
1844 400, Avenue de Roumanille
1845 Batiment T3
1846 Biot - Sophia Antipolis 06410
1847 FRANCE
1849 Phone: +33 4 97 23 26 34
1850 Email: pthubert@cisco.com
1852 Balazs Varga
1853 Ericsson
1854 Magyar tudosok korutja 11
1855 Budapest 1117
1856 Hungary
1858 Email: balazs.a.varga@ericsson.com
1860 Janos Farkas
1861 Ericsson
1862 Magyar tudosok korutja 11
1863 Budapest 1117
1864 Hungary
1866 Email: janos.farkas@ericsson.com