idnits 2.17.1
draft-ietf-detnet-architecture-06.txt:
Checking boilerplate required by RFC 5378 and the IETF Trust (see
https://trustee.ietf.org/license-info):
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt:
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/checklist :
----------------------------------------------------------------------------
No issues found here.
Miscellaneous warnings:
----------------------------------------------------------------------------
== The copyright year in the IETF Trust and authors Copyright Line does not
match the current year
== Line 729 has weird spacing: '...e stack v ...'
== Line 1226 has weird spacing: '...mediate inter...'
== Line 1229 has weird spacing: '...mediate inter...'
-- The document date (June 28, 2018) is 2122 days in the past. Is this
intentional?
Checking references for intended status: Proposed Standard
----------------------------------------------------------------------------
(See RFCs 3967 and 4897 for information about using normative references
to lower-maturity documents in RFCs)
== Missing Reference: 'Network' is mentioned on line 862, but not defined
== Outdated reference: A later version (-30) exists of
draft-ietf-6tisch-architecture-14
== Outdated reference: A later version (-09) exists of
draft-ietf-detnet-problem-statement-05
== Outdated reference: A later version (-20) exists of
draft-ietf-detnet-use-cases-17
Summary: 0 errors (**), 0 flaws (~~), 8 warnings (==), 1 comment (--).
Run idnits with the --verbose option for more detailed information about
the items above.
--------------------------------------------------------------------------------
2 DetNet N. Finn
3 Internet-Draft Huawei
4 Intended status: Standards Track P. Thubert
5 Expires: December 30, 2018 Cisco
6 B. Varga
7 J. Farkas
8 Ericsson
9 June 28, 2018
11 Deterministic Networking Architecture
12 draft-ietf-detnet-architecture-06
14 Abstract
16 Deterministic Networking (DetNet) provides a capability to carry
17 specified unicast or multicast data flows for real-time applications
18 with extremely low data loss rates and bounded latency. Techniques
19 used include: 1) reserving data plane resources for individual (or
20 aggregated) DetNet flows in some or all of the intermediate nodes
21 (e.g., bridges or routers) along the path of the flow; 2) providing
22 explicit routes for DetNet flows that do not immediately change with
23 the network topology; and 3) distributing data from DetNet flow
24 packets over time and/or space to ensure delivery of each packet's
25 data' in spite of the loss of a path.
27 Status of This Memo
29 This Internet-Draft is submitted in full conformance with the
30 provisions of BCP 78 and BCP 79.
32 Internet-Drafts are working documents of the Internet Engineering
33 Task Force (IETF). Note that other groups may also distribute
34 working documents as Internet-Drafts. The list of current Internet-
35 Drafts is at https://datatracker.ietf.org/drafts/current/.
37 Internet-Drafts are draft documents valid for a maximum of six months
38 and may be updated, replaced, or obsoleted by other documents at any
39 time. It is inappropriate to use Internet-Drafts as reference
40 material or to cite them other than as "work in progress."
42 This Internet-Draft will expire on December 30, 2018.
44 Copyright Notice
46 Copyright (c) 2018 IETF Trust and the persons identified as the
47 document authors. All rights reserved.
49 This document is subject to BCP 78 and the IETF Trust's Legal
50 Provisions Relating to IETF Documents
51 (https://trustee.ietf.org/license-info) in effect on the date of
52 publication of this document. Please review these documents
53 carefully, as they describe your rights and restrictions with respect
54 to this document. Code Components extracted from this document must
55 include Simplified BSD License text as described in Section 4.e of
56 the Trust Legal Provisions and are provided without warranty as
57 described in the Simplified BSD License.
59 Table of Contents
61 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
62 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
63 2.1. Terms used in this document . . . . . . . . . . . . . . . 4
64 2.2. IEEE 802.1 TSN to DetNet dictionary . . . . . . . . . . . 6
65 3. Providing the DetNet Quality of Service . . . . . . . . . . . 7
66 3.1. Primary goals defining the DetNet QoS . . . . . . . . . . 7
67 3.2. Mechanisms to achieve DetNet QoS . . . . . . . . . . . . 9
68 3.2.1. Congestion protection . . . . . . . . . . . . . . . . 9
69 3.2.1.1. Eliminate congestion loss . . . . . . . . . . . . 9
70 3.2.1.2. Jitter Reduction . . . . . . . . . . . . . . . . 10
71 3.2.2. Service Protection . . . . . . . . . . . . . . . . . 11
72 3.2.2.1. In-Order Delivery . . . . . . . . . . . . . . . . 11
73 3.2.2.2. Packet Replication and Elimination . . . . . . . 11
74 3.2.2.3. Packet encoding for service protection . . . . . 13
75 3.2.3. Explicit routes . . . . . . . . . . . . . . . . . . . 13
76 3.3. Secondary goals for DetNet . . . . . . . . . . . . . . . 14
77 3.3.1. Coexistence with normal traffic . . . . . . . . . . . 14
78 3.3.2. Fault Mitigation . . . . . . . . . . . . . . . . . . 15
79 4. DetNet Architecture . . . . . . . . . . . . . . . . . . . . . 16
80 4.1. DetNet stack model . . . . . . . . . . . . . . . . . . . 16
81 4.1.1. Representative Protocol Stack Model . . . . . . . . . 16
82 4.1.2. DetNet Data Plane Overview . . . . . . . . . . . . . 18
83 4.1.3. Network reference model . . . . . . . . . . . . . . . 20
84 4.2. DetNet systems . . . . . . . . . . . . . . . . . . . . . 21
85 4.2.1. End system . . . . . . . . . . . . . . . . . . . . . 21
86 4.2.2. DetNet edge, relay, and transit nodes . . . . . . . . 22
87 4.3. DetNet flows . . . . . . . . . . . . . . . . . . . . . . 23
88 4.3.1. DetNet flow types . . . . . . . . . . . . . . . . . . 23
89 4.3.2. Source transmission behavior . . . . . . . . . . . . 23
90 4.3.3. Incomplete Networks . . . . . . . . . . . . . . . . . 25
91 4.4. Traffic Engineering for DetNet . . . . . . . . . . . . . 25
92 4.4.1. The Application Plane . . . . . . . . . . . . . . . . 25
93 4.4.2. The Controller Plane . . . . . . . . . . . . . . . . 26
94 4.4.3. The Network Plane . . . . . . . . . . . . . . . . . . 26
95 4.5. Queuing, Shaping, Scheduling, and Preemption . . . . . . 27
96 4.6. Service instance . . . . . . . . . . . . . . . . . . . . 28
97 4.7. Flow identification at technology borders . . . . . . . . 29
98 4.7.1. Exporting flow identification . . . . . . . . . . . . 29
99 4.7.2. Flow attribute mapping between layers . . . . . . . . 31
100 4.7.3. Flow-ID mapping examples . . . . . . . . . . . . . . 32
101 4.8. Advertising resources, capabilities and adjacencies . . . 34
102 4.9. Scaling to larger networks . . . . . . . . . . . . . . . 34
103 4.10. Compatibility with Layer-2 . . . . . . . . . . . . . . . 34
104 5. Security Considerations . . . . . . . . . . . . . . . . . . . 35
105 6. Privacy Considerations . . . . . . . . . . . . . . . . . . . 35
106 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 35
107 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 35
108 9. Informative References . . . . . . . . . . . . . . . . . . . 36
109 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 39
111 1. Introduction
113 Deterministic Networking (DetNet) is a service that can be offered by
114 a network to DetNet flows. DetNet provides these flows with
115 extremely low packet loss rates and assured maximum end-to-end
116 delivery latency. This is accomplished by dedicating network
117 resources such as link bandwidth and buffer space to DetNet flows
118 and/or classes of DetNet flows, and by replicating packets along
119 multiple paths. Unused reserved resources are available to non-
120 DetNet packets.
122 The Deterministic Networking Problem Statement
123 [I-D.ietf-detnet-problem-statement] introduces Deterministic
124 Networking, and Deterministic Networking Use Cases
125 [I-D.ietf-detnet-use-cases] summarizes the need for it. See
126 [I-D.ietf-detnet-dp-sol-mpls] and [I-D.ietf-detnet-dp-sol-ip] for
127 specific techniques that can be used to identify DetNet Flows and
128 assign them to specific paths through a network.
130 A goal of DetNet is a converged network in all respects. That is,
131 the presence of DetNet flows does not preclude non-DetNet flows, and
132 the benefits offered DetNet flows should not, except in extreme
133 cases, prevent existing QoS mechanisms from operating in a normal
134 fashion, subject to the bandwidth required for the DetNet flows. A
135 single source-destination pair can trade both DetNet and non-DetNet
136 flows. End systems and applications need not instantiate special
137 interfaces for DetNet flows. Networks are not restricted to certain
138 topologies; connectivity is not restricted. Any application that
139 generates a data flow that can be usefully characterized as having a
140 maximum bandwidth should be able to take advantage of DetNet, as long
141 as the necessary resources can be reserved. Reservations can be made
142 by the application itself, via network management, by an applications
143 controller, or by other means, e.g., a dynamic control plane (e.g.,
144 [RFC2205]).
146 Many applications, that are intended to be served by Deterministic
147 Networking, require the ability to synchronize the clocks in end
148 systems to a sub-microsecond accuracy. Some of the queue control
149 techniques defined in Section 4.5 also require time synchronization
150 among relay and transit nodes. The means used to achieve time
151 synchronization are not addressed in this document. DetNet should
152 accommodate various synchronization techniques and profiles that are
153 defined elsewhere to solve exchange time in different market
154 segments.
156 2. Terminology
158 2.1. Terms used in this document
160 The following terms are used in the context of DetNet in this
161 document:
163 allocation
164 Resources are dedicated to support a DetNet flow. Depending
165 on an implementation, the resource may be reused by non-
166 DetNet flows when it is not used by the DetNet flow.
168 App-flow
169 The native format of a DetNet flow.
171 DetNet destination
172 An end system capable of terminating a DetNet flow.
174 DetNet domain
175 The portion of a network that is DetNet aware. It includes
176 end systems and other DetNet nodes.
178 DetNet flow
179 A DetNet flow is a sequence of packets to which the DetNet
180 service is to be provided.
182 DetNet compound flow and DetNet member flow
183 A DetNet compound flow is a DetNet flow that has been
184 separated into multiple duplicate DetNet member flows for
185 service protection at the DetNet service layer. Member flows
186 are merged back into a single DetNet compound flow such that
187 there are no duplicate packets. "Compound" and "member" are
188 strictly relative to each other, not absolutes; a DetNet
189 compound flow comprising multiple DetNet member flows can, in
190 turn, be a member of a higher-order compound.
192 DetNet intermediate node
193 A DetNet relay node or transit node.
195 DetNet edge node
196 An instance of a DetNet relay node that acts as a source and/
197 or destination at the DetNet service layer. For example, it
198 can include a DetNet service layer proxy function for DetNet
199 service protection (e.g., the addition or removal of packet
200 sequencing information) for one or more end systems, or
201 starts or terminates congestion protection at the DetNet
202 transport layer, or aggregates DetNet services into new
203 DetNet flows. It is analogous to a Label Edge Router (LER)
204 or a Provider Edge (PE) router.
206 DetNet-UNI
207 User-to-Network Interface with DetNet specific
208 functionalities. It is a packet-based reference point and
209 may provide multiple functions like encapsulation, status,
210 synchronization, etc.
212 end system
213 Commonly called a "host" in IETF documents, and an "end
214 station" is IEEE 802 documents. End systems of interest to
215 this document are either sources or destinations of DetNet
216 flows. And end system may or may not be DetNet transport
217 layer aware or DetNet service layer aware.
219 link
220 A connection between two DetNet nodes. It may be composed of
221 a physical link or a sub-network technology that can provide
222 appropriate traffic delivery for DetNet flows.
224 DetNet system
225 A DetNet aware end system, transit node, or relay node.
226 "DetNet" may be omitted in some text.
228 DetNet relay node
229 A DetNet node including a service layer function that
230 interconnects different DetNet transport layer paths to
231 provide service protection. A DetNet relay node can be a
232 bridge, a router, a firewall, or any other system that
233 participates in the DetNet service layer. It typically
234 incorporates DetNet transport layer functions as well, in
235 which case it is collocated with a transit node.
237 PEF A Packet Elimination Function (PEF) eliminates duplicate
238 copies of packets to prevent excess packets flooding the
239 network or duplicate packets being sent out of the DetNet
240 domain. PEF can be implemented by an edge node, a relay
241 node, or an end system.
243 PRF A Packet Replication Function (PRF) replicates DetNet flow
244 packets and forwards them to one or more next hops in the
245 DetNet domain. The number of packet copies sent to each next
246 hop is a DetNet flow specific parameter at the node doing the
247 replication. PRF can be implemented by an edge node, a relay
248 node, or an end system.
250 PREOF Collective name for Packet Replication, Elimination, and
251 Ordering Functions.
253 POF A Packet Ordering Function (POF) re-orders packets within a
254 DetNet flow that are received out of order. This function
255 can be implemented by an edge node, a relay node, or an end
256 system.
258 reservation
259 The set of resources allocated between a source and one or
260 more destinations through transit nodes and subnets
261 associated with a DetNet flow, to provide the expected DetNet
262 Service.
264 DetNet service layer
265 The layer at which A DetNet Service, such as congestion or
266 service protection is provided.
268 DetNet service proxy
269 Maps between App-flows and DetNet flows.
271 DetNet source
272 An end system capable of originating a DetNet flow.
274 DetNet transit node
275 A node operating at the DetNet transport layer, that utilizes
276 link layer and/or network layer switching across multiple
277 links and/or sub-networks to provide paths for DetNet service
278 layer functions. Typically provides congestion protection
279 over those paths. An MPLS LSR is an example of a DetNet
280 transit node.
282 DetNet transport layer
283 The layer that optionally provides congestion protection for
284 DetNet flows over paths provided by the underlying network.
286 2.2. IEEE 802.1 TSN to DetNet dictionary
288 This section also serves as a dictionary for translating from the
289 terms used by the Time-Sensitive Networking (TSN) Task Group
290 [IEEE802.1TSNTG] of the IEEE 802.1 WG to those of the DetNet WG.
292 Listener
293 The IEEE 802.1 term for a destination of a DetNet flow.
295 relay system
296 The IEEE 802.1 term for a DetNet intermediate node.
298 Stream
299 The IEEE 802.1 term for a DetNet flow.
301 Talker
302 The IEEE 802.1 term for the source of a DetNet flow.
304 bridged path
305 A VLAN bridge uses the VLAN ID and the destination MAC
306 address to select the outbound port hence the path for a
307 frame.
309 3. Providing the DetNet Quality of Service
311 3.1. Primary goals defining the DetNet QoS
313 The DetNet Quality of Service can be expressed in terms of:
315 o Minimum and maximum end-to-end latency from source to destination;
316 timely delivery, and bounded jitter (packet delay variation)
317 derived from these constraints.
319 o Probability of loss of a packet, under various assumptions as to
320 the operational states of the nodes and links. If packet
321 replication is used to reduce the probability of packet loss, then
322 a related property is the probability (may be zero) of delivery of
323 a duplicate packet. Duplicate packet delivery is an inherent risk
324 in highly reliable and/or broadcast transmissions.
326 o An upper bound on out-of-order packet delivery. It is worth
327 noting that some DetNet applications are unable to tolerate any
328 out-of-order delivery.
330 It is a distinction of DetNet that it is concerned solely with worst-
331 case values for the end-to-end latency, jitter, and misordering.
332 Average, mean, or typical values are of little interest, because they
333 do not affect the ability of a real-time system to perform its tasks.
334 In general, a trivial priority-based queuing scheme will give better
335 average latency to a data flow than DetNet, but of course, the worst-
336 case latency can be essentially unbounded.
338 Three techniques are used by DetNet to provide these qualities of
339 service:
341 o Congestion protection (Section 3.2.1).
343 o Service protection (Section 3.2.2).
345 o Explicit routes (Section 3.2.3).
347 Congestion protection operates by allocating resources along the path
348 of a DetNet Flow, e.g., buffer space or link bandwidth. Congestion
349 protection greatly reduces, or even eliminates entirely, packet loss
350 due to output packet congestion within the network, but it can only
351 be supplied to a DetNet flow that is limited at the source to a
352 maximum packet size and transmission rate.
354 Congestion protection addresses two of the DetNet QoS requirements:
355 latency and packet loss. Given that DetNet nodes have a finite
356 amount of buffer space, congestion protection necessarily results in
357 a maximum end-to-end latency. It also addresses the largest
358 contribution to packet loss, which is buffer congestion.
360 After congestion, the most important contributions to packet loss are
361 typically from random media errors and equipment failures. Service
362 protection is the name for the mechanisms used by DetNet to address
363 these losses. The mechanisms employed are constrained by the
364 requirement to meet the users' latency requirements. Packet
365 replication and elimination (Section 3.2.2) and packet encoding
366 (Section 3.2.2.3) are described in this document to provide service
367 protection; others may be found. For instance, packet encoding can
368 be used to provide service protection against random media errors,
369 packet replication and elimination can be used to provide service
370 protection against equipment failures. This mechanism distributes
371 the contents of DetNet flows over multiple paths in time and/or
372 space, so that the loss of some of the paths does need not cause the
373 loss of any packets.
375 The paths are typically (but not necessarily) explicit routes, so
376 that they do not normally suffer temporary interruptions caused by
377 the convergence of routing or bridging protocols.
379 These three techniques can be applied independently, giving eight
380 possible combinations, including none (no DetNet), although some
381 combinations are of wider utility than others. This separation keeps
382 the protocol stack coherent and maximizes interoperability with
383 existing and developing standards in this (IETF) and other Standards
384 Development Organizations. Some examples of typical expected
385 combinations:
387 o Explicit routes plus service protection are exactly the techniques
388 employed by seamless redundancy mechanisms applied on a ring
389 topology as described, e.g., in [IEEE802.1CB]. In this case,
390 explicit routes are achieved by limiting the physical topology of
391 the network to a ring. Sequentialization, replication, and
392 duplicate elimination are facilitated by packet tags added at the
393 front or the end of Ethernet frames.
395 o Congestion protection alone is offered by IEEE 802.1 Audio Video
396 bridging [IEEE802.1BA]. As long as the network suffers no
397 failures, zero congestion loss can be achieved through the use of
398 a reservation protocol (MSRP [IEEE802.1Q]), shapers in every
399 bridge, and proper dimensioning.
401 o Using all three together gives maximum protection.
403 There are, of course, simpler methods available (and employed, today)
404 to achieve levels of latency and packet loss that are satisfactory
405 for many applications. Prioritization and over-provisioning is one
406 such technique. However, these methods generally work best in the
407 absence of any significant amount of non-critical traffic in the
408 network (if, indeed, such traffic is supported at all), or work only
409 if the critical traffic constitutes only a small portion of the
410 network's theoretical capacity, or work only if all systems are
411 functioning properly, or in the absence of actions by end systems
412 that disrupt the network's operations.
414 There are any number of methods in use, defined, or in progress for
415 accomplishing each of the above techniques. It is expected that this
416 DetNet Architecture will assist various vendors, users, and/or
417 "vertical" Standards Development Organizations (dedicated to a single
418 industry) to make selections among the available means of
419 implementing DetNet networks.
421 3.2. Mechanisms to achieve DetNet QoS
423 3.2.1. Congestion protection
425 3.2.1.1. Eliminate congestion loss
427 The primary means by which DetNet achieves its QoS assurances is to
428 reduce, or even completely eliminate, congestion within a node as a
429 cause of packet loss. Given that a DetNet flow cannot be throttled,
430 this can be achieved only by the provision of sufficient buffer
431 storage at each hop through the network to ensure that no packets are
432 dropped due to a lack of buffer storage.
434 Ensuring adequate buffering requires, in turn, that the source, and
435 every intermediate node along the path to the destination (or nearly
436 every node, see Section 4.3.3) be careful to regulate its output to
437 not exceed the data rate for any DetNet flow, except for brief
438 periods when making up for interfering traffic. Any packet sent
439 ahead of its time potentially adds to the number of buffers required
440 by the next hop and may thus exceed the resources allocated for a
441 particular DetNet flow.
443 The low-level mechanisms described in Section 4.5 provide the
444 necessary regulation of transmissions by an end system or
445 intermediate node to provide congestion protection. The allocation
446 of the bandwidth and buffers for a DetNet flow requires provisioning
447 A DetNet node may have other resources requiring allocation and/or
448 scheduling, that might otherwise be over-subscribed and trigger the
449 rejection of a reservation.
451 3.2.1.2. Jitter Reduction
453 A core objective of DetNet is to enable the convergence of sensitive
454 non-IP networks onto a common network infrastructure. This requires
455 the accurate emulation of currently deployed mission-specific
456 networks, which for example rely on point-to-point analog (e.g.,
457 4-20mA modulation) and serial-digital cables (or buses) for highly
458 reliable, synchronized and jitter-free communications. While the
459 latency of analog transmissions is basically the speed of light,
460 legacy serial links are usually slow (in the order of Kbps) compared
461 to, say, GigE, and some latency is usually acceptable. What is not
462 acceptable is the introduction of excessive jitter, which may, for
463 instance, affect the stability of control systems.
465 Applications that are designed to operate on serial links usually do
466 not provide services to recover the jitter, because jitter simply
467 does not exist there. DetNet flows are generally expected to be
468 delivered in-order and the precise time of reception influences the
469 processes. In order to converge such existing applications, there is
470 a desire to emulate all properties of the serial cable, such as clock
471 transportation, perfect flow isolation and fixed latency. While
472 minimal jitter (in the form of specifying minimum, as well as
473 maximum, end-to-end latency) is supported by DetNet, there are
474 practical limitations on packet-based networks in this regard. In
475 general, users are encouraged to use, instead of, "do this when you
476 get the packet," a combination of:
478 o Sub-microsecond time synchronization among all source and
479 destination end systems, and
481 o Time-of-execution fields in the application packets.
483 Jitter reduction is provided by the mechanisms described in
484 Section 4.5 that also provide congestion protection.
486 3.2.2. Service Protection
488 Service protection aims to mitigate or eliminate packet loss due to
489 equipment failures, random media and/or memory faults. These types
490 of packet loss can be greatly reduced by spreading the data over
491 multiple disjoint forwarding paths. Various service protection
492 methods are described in [RFC6372], e.g., 1+1 linear protection.
493 This section describes the functional details of an additional method
494 in Section 3.2.2.2, which can be implemented as described in
495 Section 3.2.2.3 or as specified in [I-D.ietf-detnet-dp-sol-mpls] in
496 order to provide 1+n hitless protection. The appropriate service
497 protection mechanism depends on the scenario and the requirements.
499 3.2.2.1. In-Order Delivery
501 Out-of-order packet delivery can be a side effect of service
502 protection. Packets delivered out-of-order impact the amount of
503 buffering needed at the destination to properly process the received
504 data. Such packets also influence the jitter of a flow. The DetNet
505 service includes maximum allowed misordering as a constraint. Zero
506 misordering would be a valid service constraint to reflect that the
507 end system(s) of the flow cannot tolerate any out-of-order delivery.
508 Service protection may provide a mechanism to support in-order
509 delivery.
511 3.2.2.2. Packet Replication and Elimination
513 This section describes a service protection method that sends copies
514 of the same packets over multiple paths.
516 The DetNet service layer includes the packet replication (PRF), the
517 packet elimination (PEF), and the packet ordering functionality (POF)
518 for use in DetNet edge, relay node, and end system packet processing.
519 Either of these functions can be enabled in a DetNet edge node, relay
520 node or end system. The collective name for all three functions is
521 PREOF. The packet replication and elimination service protection
522 method altogether involves four capabilities:
524 o Providing sequencing information to the packets of a DetNet
525 compound flow. This may be done by adding a sequence number or
526 time stamp as part of DetNet, or may be inherent in the packet,
527 e.g., in a transport protocol, or associated to other physical
528 properties such as the precise time (and radio channel) of
529 reception of the packet. This is typically done once, at or near
530 the source.
532 o The Packet Replication Function (PRF) replicates these packets
533 into multiple DetNet member flows and typically sends them along
534 multiple different paths to the destination(s), e.g., over the
535 explicit routes of Section 3.2.3. The location within a node, and
536 the mechanism used for the PRF is implementation specific.
538 o The Packet Elimination Function (PEF) eliminates duplicate packets
539 of a DetNet flow based on the sequencing information and a history
540 of received packets. The output of the PEF is always a single
541 packet. This may be done at any node along the path to save
542 network resources further downstream, in particular if multiple
543 Replication points exist. But the most common case is to perform
544 this operation at the very edge of the DetNet network, preferably
545 in or near the receiver. The location within a node, and
546 mechanism used for the PEF is implementation specific.
548 o The Packet Ordering Function (POF) uses the sequencing information
549 to re-order a DetNet flow's packets that are received out of
550 order.
552 The order in which a node applies PEF, POF, and PRF to a DetNet flow
553 is implementation specific.
555 Some service protection mechanisms rely on switching from one flow to
556 another when a failure of a flow is detected. Contrarily, packet
557 replication and elimination combines the DetNet member flows sent
558 along multiple different paths, and performs a packet-by-packet
559 selection of which to discard, e.g., based on sequencing information.
561 In the simplest case, this amounts to replicating each packet in a
562 source that has two interfaces, and conveying them through the
563 network, along separate (disjoint non-SRLG) paths, to the similarly
564 dual-homed destinations, that discard the extras. This ensures that
565 one path (with zero congestion loss) remains, even if some
566 intermediate node fails. The sequencing information can also be used
567 for loss detection and for re-ordering.
569 DetNet relay nodes in the network can provide replication and
570 elimination facilities at various points in the network, so that
571 multiple failures can be accommodated.
573 This is shown in Figure 1, where the two relay nodes each replicate
574 (R) the DetNet flow on input, sending the DetNet member flows to both
575 the other relay node and to the end system, and eliminate duplicates
576 (E) on the output interface to the right-hand end system. Any one
577 link in the network can fail, and the DetNet compound flow can still
578 get through. Furthermore, two links can fail, as long as they are in
579 different segments of the network.
581 > > > > > > > > > relay > > > > > > > >
582 > /------------+ R node E +------------\ >
583 > / v + ^ \ >
584 end R + v | ^ + E end
585 system + v | ^ + system
586 > \ v + ^ / >
587 > \------------+ R relay E +-----------/ >
588 > > > > > > > > > node > > > > > > > >
590 Figure 1: Packet replication and elimination
592 Packet replication and elimination does not react to and correct
593 failures; it is entirely passive. Thus, intermittent failures,
594 mistakenly created packet filters, or misrouted data is handled just
595 the same as the equipment failures that are handled by typical
596 routing and bridging protocols.
598 If packet replication and elimination is used over paths providing
599 congestion protection (Section 3.2.1), and member flows that take
600 different-length paths through the network are combined, a merge
601 point may require extra buffering to equalize the delays over the
602 different paths. This equalization ensures that the resultant
603 compound flow will not exceed its contracted bandwidth even after one
604 or the other of the paths is restored after a failure. The extra
605 buffering can be also used to provide in-order delivery.
607 3.2.2.3. Packet encoding for service protection
609 There are methods for using multiple paths to provide service
610 protection that involve encoding the information in a packet
611 belonging to a DetNet flow into multiple transmission units,
612 combining information from multiple packets into any given
613 transmission unit. Such techniques, also known as "network coding",
614 can be used as a DetNet service protection technique.
616 3.2.3. Explicit routes
618 In networks controlled by typical dynamic control protocols such as
619 IS-IS or OSPF, a network topology event in one part of the network
620 can impact, at least briefly, the delivery of data in parts of the
621 network remote from the failure or recovery event. Even the use of
622 redundant paths through a network defined, e.g., by [RFC6372] do not
623 eliminate the chances of packet loss. Furthermore, out-of-order
624 packet delivery can be a side effect of route changes.
626 Many real-time networks rely on physical rings or chains of two-port
627 devices, with a relatively simple ring control protocol. This
628 supports redundant paths for service protection with a minimum of
629 wiring. As an additional benefit, ring topologies can often utilize
630 different topology management protocols than those used for a mesh
631 network, with a consequent reduction in the response time to topology
632 changes. Of course, this comes at some cost in terms of increased
633 hop count, and thus latency, for the typical path.
635 In order to get the advantages of low hop count and still ensure
636 against even very brief losses of connectivity, DetNet employs
637 explicit routes, where the path taken by a given DetNet flow does not
638 change, at least immediately, and likely not at all, in response to
639 network topology events. Service protection (Section 3.2.2 or
640 Section 3.2.2.3) over explicit routes provides a high likelihood of
641 continuous connectivity. Explicit routes can be established various
642 ways, e.g., with RSVP-TE [RFC3209], with Segment Routing (SR)
643 [I-D.ietf-spring-segment-routing], via a Software Defined Networking
644 approach [RFC7426], with IS-IS [RFC7813], etc. Explicit routes are
645 typically used in MPLS TE LSPs.
647 Out-of-order packet delivery can be a side effect of distributing a
648 single flow over multiple paths especially when there is a change
649 from one path to another when combining the flow. This is
650 irrespective of the distribution method used, also applies to service
651 protection over explicit routes. As described in Section 3.2.2.1,
652 out-of-order packets influence the jitter of a flow and impact the
653 amount of buffering needed to process the data; therefore, DetNet
654 service includes maximum allowed misordering as a constraint. The
655 use of explicit routes helps to provide in-order delivery because
656 there is no immediate route change with the network topology, but the
657 changes are plannable as they are between the different explicit
658 routes.
660 3.3. Secondary goals for DetNet
662 Many applications require DetNet to provide additional services,
663 including coexistence with other QoS mechanisms Section 3.3.1 and
664 protection against misbehaving transmitters Section 3.3.2.
666 3.3.1. Coexistence with normal traffic
668 A DetNet network supports the dedication of a high proportion (e.g.
669 75%) of the network bandwidth to DetNet flows. But, no matter how
670 much is dedicated for DetNet flows, it is a goal of DetNet to coexist
671 with existing Class of Service schemes (e.g., DiffServ). It is also
672 important that non-DetNet traffic not disrupt the DetNet flow, of
673 course (see Section 3.3.2 and Section 5). For these reasons:
675 o Bandwidth (transmission opportunities) not utilized by a DetNet
676 flow are available to non-DetNet packets (though not to other
677 DetNet flows).
679 o DetNet flows can be shaped or scheduled, in order to ensure that
680 the highest-priority non-DetNet packet is also ensured a worst-
681 case latency (at any given hop).
683 o When transmission opportunities for DetNet flows are scheduled in
684 detail, then the algorithm constructing the schedule should leave
685 sufficient opportunities for non-DetNet packets to satisfy the
686 needs of the users of the network. Detailed scheduling can also
687 permit the time-shared use of buffer resources by different DetNet
688 flows.
690 Ideally, the net effect of the presence of DetNet flows in a network
691 on the non-DetNet packets is primarily a reduction in the available
692 bandwidth.
694 3.3.2. Fault Mitigation
696 One key to building robust real-time systems is to reduce the
697 infinite variety of possible failures to a number that can be
698 analyzed with reasonable confidence. DetNet aids in the process by
699 allowing for filters and policers to detect DetNet packets received
700 on the wrong interface, or at the wrong time, or in too great a
701 volume, and to then take actions such as discarding the offending
702 packet, shutting down the offending DetNet flow, or shutting down the
703 offending interface.
705 It is also essential that filters and service remarking be employed
706 at the network edge to prevent non-DetNet packets from being mistaken
707 for DetNet packets, and thus impinging on the resources allocated to
708 DetNet packets.
710 There exist techniques, at present and/or in various stages of
711 standardization, that can perform these fault mitigation tasks that
712 deliver a high probability that misbehaving systems will have zero
713 impact on well-behaved DetNet flows, except of course, for the
714 receiving interface(s) immediately downstream of the misbehaving
715 device. Examples of such techniques include traffic policing
716 functions (e.g. [RFC2475]) and separating flows into per-flow rate-
717 limited queues.
719 4. DetNet Architecture
721 4.1. DetNet stack model
723 4.1.1. Representative Protocol Stack Model
725 Figure 2 illustrates a conceptual DetNet data plane layering model.
726 One may compare it to that in [IEEE802.1CB], Annex C.
728 | packets going | ^ packets coming ^
729 v down the stack v | up the stack |
730 +----------------------+ +-----------------------+
731 | Source | | Destination |
732 +----------------------+ +-----------------------+
733 | Service layer: | | Service layer: |
734 | Packet sequencing | | Duplicate elimination |
735 | Flow replication | | Flow merging |
736 | Packet encoding | | Packet decoding |
737 +----------------------+ +-----------------------+
738 | Transport layer: | | Transport layer: |
739 | Congestion prot. | | Congestion prot. |
740 | Explicit routes | | Explicit routes |
741 +----------------------+ +-----------------------+
742 | Lower layers | | Lower layers |
743 +----------------------+ +-----------------------+
744 v ^
745 \_________________________/
747 Figure 2: DetNet data plane protocol stack
749 Not all layers are required for any given application, or even for
750 any given network. The functionality shown in Figure 2 is:
752 Application
753 Shown as "source" and "destination" in the diagram.
755 Packet sequencing
756 As part of DetNet service protection, supplies the sequence
757 number for packet replication and elimination
758 (Section 3.2.2). Peers with Duplicate elimination. This
759 layer is not needed if a higher-layer transport protocol is
760 expected to perform any packet sequencing and duplicate
761 elimination required by the DetNet flow replication.
763 Duplicate elimination
764 As part of the DetNet service layer, based on the sequenced
765 number supplied by its peer, packet sequencing, Duplicate
766 elimination discards any duplicate packets generated by
767 DetNet flow replication. It can operate on member flows,
768 compound flows, or both. The replication may also be
769 inferred from other information such as the precise time of
770 reception in a scheduled network. The duplicate elimination
771 layer may also perform resequencing of packets to restore
772 packet order in a flow that was disrupted by the loss of
773 packets on one or another of the multiple paths taken.
775 Flow replication
776 As part of DetNet service protection, packets that belong to
777 a DetNet compound flow are replicated into two or more DetNet
778 member flows. This function is separate from packet
779 sequencing. Flow replication can be an explicit replication
780 and remarking of packets, or can be performed by, for
781 example, techniques similar to ordinary multicast
782 replication, albeit with resource allocation implications.
783 Peers with DetNet flow merging.
785 Flow merging
786 As part of DetNet service protection, merges DetNet member
787 flows together for packets coming up the stack belonging to a
788 specific DetNet compound flow. Peers with DetNet flow
789 replication. DetNet flow merging, together with packet
790 sequencing, duplicate elimination, and DetNet flow
791 replication perform packet replication and elimination
792 (Section 3.2.2).
794 Packet encoding
795 As part of DetNet service protection, as an alternative to
796 packet sequencing and flow replication, packet encoding
797 combines the information in multiple DetNet packets, perhaps
798 from different DetNet compound flows, and transmits that
799 information in packets on different DetNet member Flows.
800 Peers with Packet decoding.
802 Packet decoding
803 As part of DetNet service protection, as an alternative to
804 flow merging and duplicate elimination, packet decoding takes
805 packets from different DetNet member flows, and computes from
806 those packets the original DetNet packets from the compound
807 flows input to packet encoding. Peers with Packet encoding.
809 Congestion protection
810 The DetNet transport layer provides congestion protection.
811 See Section 4.5. The actual queuing and shaping mechanisms
812 are typically provided by underlying subnet layers, these can
813 be closely associated with the means of providing paths for
814 DetNet flows (e.g., MPLS LSPs or bridged paths), the path and
815 the congestion protection are conflated in this figure.
817 Explicit routes
818 The DetNet transport layer provides mechanisms to ensure that
819 fixed paths are provided for DetNet flows. These explicit
820 paths avoid the impact of network convergence.
822 Operations, Administration, and Maintenance (OAM) leverages in-band
823 and out-of-band signaling that validates whether the service is
824 effectively obtained within QoS constraints. OAM is not shown in
825 Figure 2; it may reside in any number of the layers. OAM can involve
826 specific tagging added in the packets for tracing implementation or
827 network configuration errors; traceability enables to find whether a
828 packet is a replica, which relay node performed the replication, and
829 which segment was intended for the replica.
831 The packet sequencing and replication elimination functions at the
832 source and destination ends of a DetNet compound flow may be
833 performed either in the end system or in a DetNet relay node.
835 4.1.2. DetNet Data Plane Overview
837 A "Deterministic Network" will be composed of DetNet enabled end
838 systems and nodes, i.e., edge nodes, relay nodes and collectively
839 deliver DetNet services. DetNet enabled nodes are interconnected via
840 transit nodes (e.g., LSRs) which support DetNet, but are not DetNet
841 service aware. All DetNet enabled nodes are connected to sub-
842 networks, where a point-to-point link is also considered as a simple
843 sub-network. These sub-networks will provide DetNet compatible
844 service for support of DetNet traffic. Examples of sub-networks
845 include MPLS TE, IEEE 802.1 TSN and OTN. Of course, multi-layer
846 DetNet systems may also be possible, where one DetNet appears as a
847 sub-network, and provides service to, a higher layer DetNet system.
848 A simple DetNet concept network is shown in Figure 3.
850 TSN Edge Transit Relay DetNet
851 End System Node Node Node End System
853 +---------+ +.........+ +---------+
854 | Appl. |<--:Svc Proxy:-- End to End Service ---------->| Appl. |
855 +---------+ +---------+ +---------+ +---------+
856 | TSN | |TSN| |Svc|<-- DetNet flow ---: Service :-->| Service |
857 +---------+ +---+ +---+ +---------+ +---------+ +---------+
858 |Transport| |Trp| |Trp| |Transport| |Trp| |Trp| |Transport|
859 +-------.-+ +-.-+ +-.-+ +--.----.-+ +-.-+ +-.-+ +---.-----+
860 : Link : / ,-----. \ : Link : / ,-----. \
861 +.......+ +-[ Sub ]-+ +........+ +-[ Sub ]-+
862 [Network] [Network]
863 `-----' `-----'
865 Figure 3: A Simple DetNet Enabled Network
867 Distinguishing the function of two DetNet data plane layers, the
868 DetNet service layer and the DetNet transport layer, helps to explore
869 and evaluate various combinations of the data plane solutions
870 available, some are illustrated in Figure 4. This separation of
871 DetNet layers, while helpful, should not be considered as formal
872 requirement. For example, some technologies may violate these strict
873 layers and still be able to deliver a DetNet service.
875 .
876 .
877 +-----------+
878 | Service | PW, UDP, GRE
879 +-----------+
880 | Transport | IPv6, IPv4, MPLS TE LSPs, MPLS SR
881 +-----------+
882 .
883 .
885 Figure 4: DetNet adaptation to data plane
887 In some networking scenarios, the end system initially provides a
888 DetNet flow encapsulation, which contains all information needed by
889 DetNet nodes (e.g., Real-time Transport Protocol (RTP) [RFC3550]
890 based DetNet flow transported over a native UDP/IP network or
891 PseudoWire). In other scenarios, the encapsulation formats might
892 differ significantly.
894 There are many valid options to create a data plane solution for
895 DetNet traffic by selecting a technology approach for the DetNet
896 service layer and also selecting a technology approach for the DetNet
897 transport layer. There are a high number of valid combinations.
899 One of the most fundamental differences between different potential
900 data plane options is the basic headers used by DetNet nodes. For
901 example, the basic service can be delivered based on an MPLS label or
902 an IP header. This decision impacts the basic forwarding logic for
903 the DetNet service layer. Note that in both cases, IP addresses are
904 used to address DetNet nodes. The selected DetNet transport layer
905 technology also needs to be mapped to the sub-net technology used to
906 interconnect DetNet nodes. For example, DetNet flows will need to be
907 mapped to TSN Streams.
909 4.1.3. Network reference model
911 Figure 5 shows another view of the DetNet service related reference
912 points and main components.
914 DetNet DetNet
915 end system end system
916 _ _
917 / \ +----DetNet-UNI (U) / \
918 /App\ | /App\
919 /-----\ | /-----\
920 | NIC | v ________ | NIC |
921 +--+--+ _____ / \ DetNet-UNI (U) --+ +--+--+
922 | / \__/ \ | |
923 | / +----+ +----+ \_____ | |
924 | / | | | | \_______ | |
925 +------U PE +----+ P +----+ \ _ v |
926 | | | | | | | ___/ \ |
927 | +--+-+ +----+ | +----+ | / \_ |
928 \ | | | | | / \ |
929 \ | +----+ +--+-+ +--+PE |------ U-----+
930 \ | | | | | | | | | \_ _/
931 \ +---+ P +----+ P +--+ +----+ | \____/
932 \___ | | | | /
933 \ +----+__ +----+ DetNet-1 DetNet-2
934 | \_____/ \___________/ |
935 | |
936 | | End-to-End-Service | | | |
937 <------------------------------------------------------------->
938 | | DetNet-Service | | | |
939 | <------------------------------------------------> |
940 | | | | | |
942 Figure 5: DetNet Service Reference Model (multi-domain)
944 DetNet-UNIs ("U" in Figure 5) are assumed in this document to be
945 packet-based reference points and provide connectivity over the
946 packet network. A DetNet-UNI may provide multiple functions, e.g.,
947 it may add networking technology specific encapsulation to the DetNet
948 flows if necessary; it may provide status of the availability of the
949 resources associated with a reservation; it may provide a
950 synchronization service for the end system; it may carry enough
951 signaling to place the reservation in a network without a controller,
952 or if the controller only deals with the network but not the end
953 systems. Internal reference points of end systems (between the
954 application and the NIC) are more challenging from control
955 perspective and they may have extra requirements (e.g., in-order
956 delivery is expected in end system internal reference points, whereas
957 it is considered optional over the DetNet-UNI).
959 4.2. DetNet systems
961 4.2.1. End system
963 The native data flow between the source/destination end systems is
964 referred to as application-flow (App-flow). The traffic
965 characteristics of an App-flow can be CBR (constant bit rate) or VBR
966 (variable bit rate) and can have L1 or L2 or L3 encapsulation (e.g.,
967 TDM (time-division multiplexing), Ethernet, IP). These
968 characteristics are considered as input for resource reservation and
969 might be simplified to ensure determinism during transport (e.g.,
970 making reservations for the peak rate of VBR traffic, etc.).
972 An end system may or may not be DetNet transport layer aware or
973 DetNet service layer aware. That is, an end system may or may not
974 contain DetNet specific functionality. End systems with DetNet
975 functionalities may have the same or different transport layer as the
976 connected DetNet domain. Categorization of end systems are shown in
977 Figure 6.
979 End system
980 |
981 |
982 | DetNet aware ?
983 / \
984 +------< >------+
985 NO | \ / | YES
986 | v |
987 DetNet unaware |
988 End system |
989 | Service/
990 | Transport
991 / \ aware ?
992 +--------< >-------------+
993 t-aware | \ / | s-aware
994 | v |
995 | | both |
996 | | |
997 DetNet t-aware | DetNet s-aware
998 End system | End system
999 v
1000 DetNet st-aware
1001 End system
1003 Figure 6: Categorization of end systems
1005 Note some known use case examples for end systems:
1007 o DetNet unaware: The classic case requiring service proxies.
1009 o DetNet t-aware: An extant TSN system. It knows about some TSN
1010 functions (e.g., reservation), but not about service protection.
1012 o DetNet s-aware: An extant IEC 62439-3 system. It supplies
1013 sequence numbers, but doesn't know about zero congestion loss.
1015 o DetNet st-aware: A full functioning DetNet end system, it has
1016 DetNet functionalities and usually the same forwarding paradigm as
1017 the connected DetNet domain. It can be treated as an integral
1018 part of the DetNet domain.
1020 4.2.2. DetNet edge, relay, and transit nodes
1022 As shown in Figure 3, DetNet edge nodes providing proxy service and
1023 DetNet relay nodes providing the DetNet service layer are DetNet-
1024 aware, and DetNet transit nodes need only be aware of the DetNet
1025 transport layer.
1027 In general, if a DetNet flow passes through one or more DetNet-
1028 unaware network nodes between two DetNet nodes providing the DetNet
1029 transport layer for that flow, there is a potential for disruption or
1030 failure of the DetNet QoS. A network administrator needs to ensure
1031 that the DetNet-unaware network nodes are configured to minimize the
1032 chances of packet loss and delay, and provision enough extra buffer
1033 space in the DetNet transit node following the DetNet-unaware network
1034 nodes to absorb the induced latency variations.
1036 4.3. DetNet flows
1038 4.3.1. DetNet flow types
1040 A DetNet flow can have different formats while it is transported
1041 between the peer end systems. Therefore, the following possible
1042 types / formats of a DetNet flow are distinguished in this document:
1044 o App-flow: native format of the data carried over a DetNet flow.
1045 It does not contain any DetNet related attributes.
1047 o DetNet-t-flow: specific format of a DetNet flow. Only requires
1048 the congestion / latency features provided by the DetNet transport
1049 layer.
1051 o DetNet-s-flow: specific format of a DetNet flow. Only requires
1052 the service protection feature ensured by the DetNet service
1053 layer.
1055 o DetNet-st-flow: specific format of a DetNet flow. It requires
1056 both DetNet service layer and DetNet transport layer functions
1057 during forwarding.
1059 4.3.2. Source transmission behavior
1061 For the purposes of congestion protection, DetNet flows can be
1062 synchronous or asynchronous. In synchronous DetNet flows, at least
1063 the intermediate nodes (and possibly the end systems) are closely
1064 time synchronized, typically to better than 1 microsecond. By
1065 transmitting packets from different DetNet flows or classes of DetNet
1066 flows at different times, using repeating schedules synchronized
1067 among the intermediate nodes, resources such as buffers and link
1068 bandwidth can be shared over the time domain among different DetNet
1069 flows. There is a tradeoff among techniques for synchronous DetNet
1070 flows between the burden of fine-grained scheduling and the benefit
1071 of reducing the required resources, especially buffer space.
1073 In contrast, asynchronous DetNet flows are not coordinated with a
1074 fine-grained schedule, so relay and end systems must assume worst-
1075 case interference among DetNet flows contending for buffer resources.
1076 Asynchronous DetNet flows are characterized by:
1078 o A maximum packet size;
1080 o An observation interval; and
1082 o A maximum number of transmissions during that observation
1083 interval.
1085 These parameters, together with knowledge of the protocol stack used
1086 (and thus the size of the various headers added to a packet), limit
1087 the number of bit times per observation interval that the DetNet flow
1088 can occupy the physical medium.
1090 The source is required not to exceed these limits in order to obtain
1091 DetNet service. If the source transmits less data than this limit
1092 allows, the unused resource such as link bandwidth can be made
1093 available by the system to non-DetNet packets. However, making those
1094 resources available to DetNet packets in other DetNet flows would
1095 serve no purpose. Those other DetNet flows have their own dedicated
1096 resources, on the assumption that all DetNet flows can use all of
1097 their resources over a long period of time.
1099 There is no provision in DetNet for throttling DetNet flows (reducing
1100 end-to-end transmission rate via any explicit congestion
1101 notification); the assumption is that a DetNet flow, to be useful,
1102 must be delivered in its entirety. That is, while any useful
1103 application is written to expect a certain number of lost packets,
1104 the real-time applications of interest to DetNet demand that the loss
1105 of data due to the network is an extraordinarily event.
1107 Although DetNet strives to minimize the changes required of an
1108 application to allow it to shift from a special-purpose digital
1109 network to an Internet Protocol network, one fundamental shift in the
1110 behavior of network applications is impossible to avoid: the
1111 reservation of resources before the application starts. In the first
1112 place, a network cannot deliver finite latency and practically zero
1113 packet loss to an arbitrarily high offered load. Secondly, achieving
1114 practically zero packet loss for unthrottled (though bandwidth
1115 limited) DetNet flows means that bridges and routers have to dedicate
1116 buffer resources to specific DetNet flows or to classes of DetNet
1117 flows. The requirements of each reservation have to be translated
1118 into the parameters that control each system's queuing, shaping, and
1119 scheduling functions and delivered to the hosts, bridges, and
1120 routers.
1122 4.3.3. Incomplete Networks
1124 The presence in the network of transit nodes or subnets that are not
1125 fully capable of offering DetNet services complicates the ability of
1126 the intermediate nodes and/or controller to allocate resources, as
1127 extra buffering must be allocated at points downstream from the non-
1128 DetNet intermediate node for a DetNet flow. This extra buffering may
1129 increase latency and/or jitter.
1131 4.4. Traffic Engineering for DetNet
1133 Traffic Engineering Architecture and Signaling (TEAS) [TEAS] defines
1134 traffic-engineering architectures for generic applicability across
1135 packet and non-packet networks. From a TEAS perspective, Traffic
1136 Engineering (TE) refers to techniques that enable operators to
1137 control how specific traffic flows are treated within their networks.
1139 Because if its very nature of establishing explicit optimized paths,
1140 Deterministic Networking can be seen as a new, specialized branch of
1141 Traffic Engineering, and inherits its architecture with a separation
1142 into planes.
1144 The Deterministic Networking architecture is thus composed of three
1145 planes, a (User) Application Plane, a Controller Plane, and a Network
1146 Plane, which echoes that of Figure 1 of Software-Defined Networking
1147 (SDN): Layers and Architecture Terminology [RFC7426].:
1149 4.4.1. The Application Plane
1151 Per [RFC7426], the Application Plane includes both applications and
1152 services. In particular, the Application Plane incorporates the User
1153 Agent, a specialized application that interacts with the end user /
1154 operator and performs requests for Deterministic Networking services
1155 via an abstract Flow Management Entity, (FME) which may or may not be
1156 collocated with (one of) the end systems.
1158 At the Application Plane, a management interface enables the
1159 negotiation of flows between end systems. An abstraction of the flow
1160 called a Traffic Specification (TSpec) provides the representation.
1161 This abstraction is used to place a reservation over the (Northbound)
1162 Service Interface and within the Application plane. It is associated
1163 with an abstraction of location, such as IP addresses and DNS names,
1164 to identify the end systems and eventually specify intermediate
1165 nodes.
1167 4.4.2. The Controller Plane
1169 The Controller Plane corresponds to the aggregation of the Control
1170 and Management Planes in [RFC7426], though Common Control and
1171 Measurement Plane (CCAMP) [CCAMP] makes an additional distinction
1172 between management and measurement. When the logical separation of
1173 the Control, Measurement and other Management entities is not
1174 relevant, the term Controller Plane is used for simplicity to
1175 represent them all, and the term controller plane entity (CPE) refers
1176 to any device operating in that plane, whether is it a Path
1177 Computation entity, or a Network Management entity (NME)), or a
1178 distributed control plane. The Path Computation Element (PCE)
1179 [RFC4655] is a core element of a controller, in charge of computing
1180 Deterministic paths to be applied in the Network Plane.
1182 A (Northbound) Service Interface enables applications in the
1183 Application Plane to communicate with the entities in the Controller
1184 Plane as illustrated in Figure 7.
1186 One or more PCE(s) collaborate to implement the requests from the FME
1187 as Per-Flow Per-Hop Behaviors installed in the intermediate nodes for
1188 each individual flow. The PCEs place each flow along a deterministic
1189 sequence of intermediate nodes so as to respect per-flow constraints
1190 such as security and latency, and optimize the overall result for
1191 metrics such as an abstract aggregated cost. The deterministic
1192 sequence can typically be more complex than a direct sequence and
1193 include redundancy path, with one or more packet replication and
1194 elimination points.
1196 4.4.3. The Network Plane
1198 The Network Plane represents the network devices and protocols as a
1199 whole, regardless of the Layer at which the network devices operate.
1200 It includes Forwarding Plane (data plane), Application, and
1201 Operational Plane (control plane) aspects.
1203 The network Plane comprises the Network Interface Cards (NIC) in the
1204 end systems, which are typically IP hosts, and intermediate nodes,
1205 which are typically IP routers and switches. Network-to-Network
1206 Interfaces such as used for Traffic Engineering path reservation in
1207 [RFC5921], as well as User-to-Network Interfaces (UNI) such as
1208 provided by the Local Management Interface (LMI) between network and
1209 end systems, are both part of the Network Plane, both in the control
1210 plane and the data plane.
1212 A Southbound (Network) Interface enables the entities in the
1213 Controller Plane to communicate with devices in the Network Plane as
1214 illustrated in Figure 7. This interface leverages and extends TEAS
1215 to describe the physical topology and resources in the Network Plane.
1217 End End
1218 System System
1220 -+-+-+-+-+-+-+ Northbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
1222 CPE CPE CPE CPE
1224 -+-+-+-+-+-+-+ Southbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
1226 intermediate intermed. intermed. intermed.
1227 Node Node Node Node
1228 NIC NIC
1229 intermediate intermed. intermed. intermed.
1230 Node Node Node Node
1232 Figure 7: Northbound and Southbound interfaces
1234 The intermediate nodes (and eventually the end systems NIC) expose
1235 their capabilities and physical resources to the controller (the
1236 CPE), and update the CPEs with their dynamic perception of the
1237 topology, across the Southbound Interface. In return, the CPEs set
1238 the per-flow paths up, providing a Flow Characterization that is more
1239 tightly coupled to the intermediate node Operation than a TSpec.
1241 At the Network plane, intermediate nodes may exchange information
1242 regarding the state of the paths, between adjacent systems and
1243 eventually with the end systems, and forward packets within
1244 constraints associated to each flow, or, when unable to do so,
1245 perform a last resort operation such as drop or declassify.
1247 This document focuses on the Southbound interface and the operation
1248 of the Network Plane.
1250 4.5. Queuing, Shaping, Scheduling, and Preemption
1252 DetNet achieves congestion protection and bounded delivery latency by
1253 reserving bandwidth and buffer resources at every hop along the path
1254 of the DetNet flow. The reservation itself is not sufficient,
1255 however. Implementors and users of a number of proprietary and
1256 standard real-time networks have found that standards for specific
1257 data plane techniques are required to enable these assurances to be
1258 made in a multi-vendor network. The fundamental reason is that
1259 latency variation in one system results in the need for extra buffer
1260 space in the next-hop system(s), which in turn, increases the worst-
1261 case per-hop latency.
1263 Standard queuing and transmission selection algorithms allow a
1264 central controller to compute the latency contribution of each
1265 transit node to the end-to-end latency, to compute the amount of
1266 buffer space required in each transit node for each incremental
1267 DetNet flow, and most importantly, to translate from a flow
1268 specification to a set of values for the managed objects that control
1269 each relay or end system. For example, the IEEE 802.1 WG has
1270 specified (and is specifying) a set of queuing, shaping, and
1271 scheduling algorithms that enable each transit node (bridge or
1272 router), and/or a central controller, to compute these values. These
1273 algorithms include:
1275 o A credit-based shaper [IEEE802.1Q] Clause 34.
1277 o Time-gated queues governed by a rotating time schedule,
1278 synchronized among all transit nodes [IEEE802.1Qbv].
1280 o Synchronized double (or triple) buffers driven by synchronized
1281 time ticks. [IEEE802.1Qch].
1283 o Pre-emption of an Ethernet packet in transmission by a packet with
1284 a more stringent latency requirement, followed by the resumption
1285 of the preempted packet [IEEE802.1Qbu], [IEEE802.3br].
1287 While these techniques are currently embedded in Ethernet [IEEE802.3]
1288 and bridging standards, we can note that they are all, except perhaps
1289 for packet preemption, equally applicable to other media than
1290 Ethernet, and to routers as well as bridges. Other media may have
1291 its own methods, see, e.g., [I-D.ietf-6tisch-architecture],
1292 [RFC7554]. DetNet may include such definitions in the future, or may
1293 define how these techniques can be used by DetNet nodes.
1295 4.6. Service instance
1297 A Service instance represents all the functions required on a node to
1298 allow the end-to-end service between the UNIs.
1300 The DetNet network general reference model is shown in Figure 8 for a
1301 DetNet-Service scenario (i.e., between two DetNet-UNIs). In this
1302 figure, end systems ("A" and "B") are connected directly to the edge
1303 nodes of an IP/MPLS network ("PE1" and "PE2"). End systems
1304 participating in DetNet communication may require connectivity before
1305 setting up an App-flow that requires the DetNet service. Such a
1306 connectivity related service instance and the one dedicated for
1307 DetNet service share the same access. Packets belonging to a DetNet
1308 flow are selected by a filter configured on the access ("F1" and
1309 "F2"). As a result, data flow specific access ("access-A + F1" and
1310 "access-B + F2") are terminated in the flow specific service instance
1311 ("SI-1" and "SI-2"). A tunnel is used to provide connectivity
1312 between the service instances.
1314 The tunnel is used to transport exclusively the packets of the DetNet
1315 flow between "SI-1" and "SI-2". The service instances are configured
1316 to implement DetNet functions and a flow specific DetNet transport.
1317 The service instance and the tunnel may or may not be shared by
1318 multiple DetNet flows. Sharing the service instance by multiple
1319 DetNet flows requires properly populated forwarding tables of the
1320 service instance.
1322 access-A access-B
1323 <-----> <-------- tunnel ----------> <----->
1325 +---------+ ___ _ +---------+
1326 End system | +----+ | / \/ \_ | +----+ | End system
1327 "A" -------F1+ | | / \ | | +F2----- "B"
1328 | | +========+ IP/MPLS +=======+ | |
1329 | |SI-1| | \__ Net._/ | |SI-2| |
1330 | +----+ | \____/ | +----+ |
1331 |PE1 | | PE2|
1332 +---------+ +---------+
1334 Figure 8: DetNet network general reference model
1336 The tunnel between the service instances may have some special
1337 characteristics. For example, in case of a DetNet L3 service, there
1338 are differences in the usage of the PW for DetNet traffic compared to
1339 the network model described in [RFC6658]. In the DetNet scenario,
1340 the PW is likely to be used exclusively by the DetNet flow, whereas
1341 [RFC6658] states: "The packet PW appears as a single point-to-point
1342 link to the client layer. Network-layer adjacency formation and
1343 maintenance between the client equipment will follow the normal
1344 practice needed to support the required relationship in the client
1345 layer ... This packet pseudowire is used to transport all of the
1346 required Layer-2 and Layer-3 protocols between LSR1 and LSR2".
1347 Further details are network technology specific and can be found in
1348 [I-D.ietf-detnet-dp-sol-mpls] and [I-D.ietf-detnet-dp-sol-ip].
1350 4.7. Flow identification at technology borders
1352 4.7.1. Exporting flow identification
1354 A DetNet node may need to map specific flows to lower layer flows (or
1355 Streams) in order to provide specific queuing and shaping services
1356 for specific flows. For example:
1358 o A non-IP, strictly L2 source end system X may be sending multiple
1359 flows to the same L2 destination end system Y. Those flows may
1360 include DetNet flows with different QoS requirements, and may
1361 include non-DetNet flows.
1363 o A router may be sending any number of flows to another router.
1364 Again, those flows may include DetNet flows with different QoS
1365 requirements, and may include non-DetNet flows.
1367 o Two routers may be separated by bridges. For these bridges to
1368 perform any required per-flow queuing and shaping, they must be
1369 able to identify the individual flows.
1371 o A Label Edge Router (LER) may have a Label Switched Path (LSP) set
1372 up for handling traffic destined for a particular IP address
1373 carrying only non-DetNet flows. If a DetNet flow to that same
1374 address is requested, a separate LSP may be needed, in order that
1375 all of the Label Switch Routers (LSRs) along the path to the
1376 destination give that flow special queuing and shaping.
1378 The need for a lower-layer node to be aware of individual higher-
1379 layer flows is not unique to DetNet. But, given the endless
1380 complexity of layering and relayering over tunnels that is available
1381 to network designers, DetNet needs to provide a model for flow
1382 identification that is better than packet inspection. That is not to
1383 say that packet inspection to layer 4 or 5 addresses will not be
1384 used, or the capability standardized; but, there are alternatives.
1386 A DetNet relay node can connect DetNet flows on different paths using
1387 different flow identification methods. For example:
1389 o A single unicast DetNet flow passing from router A through a
1390 bridged network to router B may be assigned a TSN Stream
1391 identifier that is unique within that bridged network. The
1392 bridges can then identify the flow without accessing higher-layer
1393 headers. Of course, the receiving router must recognize and
1394 accept that TSN Stream.
1396 o A DetNet flow passing from LSR A to LSR B may be assigned a
1397 different label than that used for other flows to the same IP
1398 destination.
1400 In any of the above cases, it is possible that an existing DetNet
1401 flow can be an aggregate carrying multiple other DetNet flows. (Not
1402 to be confused with DetNet compound vs. member flows.) Of course,
1403 this requires that the aggregate DetNet flow be provisioned properly
1404 to carry the aggregated flows.
1406 Thus, rather than packet inspection, there is the option to export
1407 higher-layer information to the lower layer. The requirement to
1408 support one or the other method for flow identification (or both) is
1409 a complexity that is part of DetNet control models.
1411 4.7.2. Flow attribute mapping between layers
1413 Transport of DetNet flows over multiple technology domains may
1414 require that lower layers are aware of specific flows of higher
1415 layers. Such an "exporting of flow identification" is needed each
1416 time when the forwarding paradigm is changed on the transport path
1417 (e.g., two LSRs are interconnected by a L2 bridged domain, etc.).
1418 The three representative forwarding methods considered for
1419 deterministic networking are:
1421 o IP routing
1423 o MPLS label switching
1425 o Ethernet bridging
1427 A packet with corresponding Flow-IDs is illustrated in Figure 9.
1429 add/remove add/remove
1430 Eth Flow-ID IP Flow-ID
1431 | |
1432 v v
1433 +-----------------------------------------------------------+
1434 | | | | |
1435 | Eth | MPLS | IP | Application data |
1436 | | | | |
1437 +-----------------------------------------------------------+
1438 ^
1439 |
1440 add/remove
1441 MPLS Flow-ID
1443 Figure 9: Packet with multiple Flow-IDs
1445 The additional (domain specific) Flow-ID can be
1447 o created by a domain specific function or
1449 o derived from the Flow-ID added to the App-flow.
1451 The Flow-ID must be unique inside a given domain. Note that the
1452 Flow-ID added to the App-flow is still present in the packet, but
1453 transport nodes may lack the function to recognize it; that's why the
1454 additional Flow-ID is added.
1456 4.7.3. Flow-ID mapping examples
1458 IP nodes and MPLS nodes are assumed to be configured to push such an
1459 additional (domain specific) Flow-ID when sending traffic to an
1460 Ethernet switch (as shown in the examples below).
1462 Figure 10 shows a scenario where an IP end system ("IP-A") is
1463 connected via two Ethernet switches ("ETH-n") to an IP router ("IP-
1464 1").
1466 IP domain
1467 <-----------------------------------------------
1469 +======+ +======+
1470 |L3-ID | |L3-ID |
1471 +======+ /\ +-----+ +======+
1472 / \ Forward as | |
1473 /IP-A\ per ETH-ID |IP-1 | Recognize
1474 Push ------> +-+----+ | +---+-+ <----- ETH-ID
1475 ETH-ID | +----+-----+ |
1476 | v v |
1477 | +-----+ +-----+ |
1478 +------+ | | +---------+
1479 +......+ |ETH-1+----+ETH-2| +======+
1480 .L3-ID . +-----+ +-----+ |L3-ID |
1481 +======+ +......+ +======+
1482 |ETH-ID| .L3-ID . |ETH-ID|
1483 +======+ +======+ +------+
1484 |ETH-ID|
1485 +======+
1487 Ethernet domain
1488 <---------------->
1490 Figure 10: IP nodes interconnected by an Ethernet domain
1492 End system "IP-A" uses the original App-flow specific ID ("L3-ID"),
1493 but as it is connected to an Ethernet domain it has to push an
1494 Ethernet-domain specific flow-ID ("VID + multicast MAC address",
1495 referred as "ETH-ID") before sending the packet to "ETH-1" node.
1496 Ethernet switch "ETH-1" can recognize the data flow based on the
1497 "ETH-ID" and it does forwarding toward "ETH-2". "ETH-2" switches the
1498 packet toward the IP router. "IP-1" must be configured to receive
1499 the Ethernet Flow-ID specific multicast flow, but (as it is an L3
1500 node) it decodes the data flow ID based on the "L3-ID" fields of the
1501 received packet.
1503 Figure 11 shows a scenario where MPLS domain nodes ("PE-n" and "P-m")
1504 are connected via two Ethernet switches ("ETH-n").
1506 MPLS domain
1507 <----------------------------------------------->
1509 +=======+ +=======+
1510 |MPLS-ID| |MPLS-ID|
1511 +=======+ +-----+ +-----+ +=======+ +-----+
1512 | | Forward as | | | |
1513 |PE-1 | per ETH-ID | P-2 +-----------+ PE-2|
1514 Push -----> +-+---+ | +---+-+ +-----+
1515 ETH-ID | +-----+----+ | \ Recognize
1516 | v v | +-- ETH-ID
1517 | +-----+ +-----+ |
1518 +---+ | | +----+
1519 +.......+ |ETH-1+----+ETH-2| +=======+
1520 .MPLS-ID. +-----+ +-----+ |MPLS-ID|
1521 +=======+ +=======+
1522 |ETH-ID | +.......+ |ETH-ID |
1523 +=======+ .MPLS-ID. +-------+
1524 +=======+
1525 |ETH-ID |
1526 +=======+
1527 Ethernet domain
1528 <---------------->
1530 Figure 11: MPLS nodes interconnected by an Ethernet domain
1532 "PE-1" uses the MPLS specific ID ("MPLS-ID"), but as it is connected
1533 to an Ethernet domain it has to push an Ethernet-domain specific
1534 flow-ID ("VID + multicast MAC address", referred as "ETH-ID") before
1535 sending the packet to "ETH-1". Ethernet switch "ETH-1" can recognize
1536 the data flow based on the "ETH-ID" and it does forwarding toward
1537 "ETH-2". "ETH-2" switches the packet toward the MPLS node ("P-2").
1538 "P-2" must be configured to receive the Ethernet Flow-ID specific
1539 multicast flow, but (as it is an MPLS node) it decodes the data flow
1540 ID based on the "MPLS-ID" fields of the received packet.
1542 One can appreciate from the above example that, when the means used
1543 for DetNet flow identification is altered or exported, the means for
1544 encoding the sequence number information must similarly be altered or
1545 exported.
1547 4.8. Advertising resources, capabilities and adjacencies
1549 There are three classes of information that a central controller or
1550 distributed control plane needs to know that can only be obtained
1551 from the end systems and/or nodes in the network. When using a peer-
1552 to-peer control plane, some of this information may be required by a
1553 system's neighbors in the network.
1555 o Details of the system's capabilities that are required in order to
1556 accurately allocate that system's resources, as well as other
1557 systems' resources. This includes, for example, which specific
1558 queuing and shaping algorithms are implemented (Section 4.5), the
1559 number of buffers dedicated for DetNet allocation, and the worst-
1560 case forwarding delay and misordering.
1562 o The dynamic state of a node's DetNet resources.
1564 o The identity of the system's neighbors, and the characteristics of
1565 the link(s) between the systems, including the length (in
1566 nanoseconds) of the link(s).
1568 4.9. Scaling to larger networks
1570 Reservations for individual DetNet flows require considerable state
1571 information in each transit node, especially when adequate fault
1572 mitigation (Section 3.3.2) is required. The DetNet data plane, in
1573 order to support larger numbers of DetNet flows, must support the
1574 aggregation of DetNet flows. Such aggregated flows can be viewed by
1575 the transit nodes' data plane largely as individual DetNet flows.
1576 Without such aggregation, the per-relay system may limit the scale of
1577 DetNet networks. Example techniques that may be used include MPLS
1578 hierarchy and IP DiffServ Code Points (DSCPs).
1580 4.10. Compatibility with Layer-2
1582 Standards providing similar capabilities for bridged networks (only)
1583 have been and are being generated in the IEEE 802 LAN/MAN Standards
1584 Committee. The present architecture describes an abstract model that
1585 can be applicable both at Layer-2 and Layer-3, and over links not
1586 defined by IEEE 802.
1588 DetNet enabled end systems and intermediate nodes can be
1589 interconnected by sub-networks, i.e., Layer-2 technologies. These
1590 sub-networks will provide DetNet compatible service for support of
1591 DetNet traffic. Examples of sub-networks include MPLS TE, 802.1 TSN,
1592 and a point-to-point OTN link. Of course, multi-layer DetNet systems
1593 may be possible too, where one DetNet appears as a sub-network, and
1594 provides service to, a higher layer DetNet system.
1596 5. Security Considerations
1598 Security in the context of Deterministic Networking has an added
1599 dimension; the time of delivery of a packet can be just as important
1600 as the contents of the packet, itself. A man-in-the-middle attack,
1601 for example, can impose, and then systematically adjust, additional
1602 delays into a link, and thus disrupt or subvert a real-time
1603 application without having to crack any encryption methods employed.
1604 See [RFC7384] for an exploration of this issue in a related context.
1606 Furthermore, in a control system where millions of dollars of
1607 equipment, or even human lives, can be lost if the DetNet QoS is not
1608 delivered, one must consider not only simple equipment failures,
1609 where the box or wire instantly becomes perfectly silent, but complex
1610 errors such as can be caused by software failures. Because there is
1611 essential no limit to the kinds of failures that can occur,
1612 protecting against realistic equipment failures is indistinguishable,
1613 in most cases, from protecting against malicious behavior, whether
1614 accidental or intentional. See also Section 3.3.2.
1616 Security must cover:
1618 o the protection of the signaling protocol
1620 o the authentication and authorization of the controlling systems
1622 o the identification and shaping of the DetNet flows
1624 6. Privacy Considerations
1626 DetNet is provides a Quality of Service (QoS), and as such, does not
1627 directly raise any new privacy considerations.
1629 However, the requirement for every (or almost every) node along the
1630 path of a DetNet flow to identify DetNet flows may present an
1631 additional attack surface for privacy, should the DetNet paradigm be
1632 found useful in broader environments.
1634 7. IANA Considerations
1636 This document does not require an action from IANA.
1638 8. Acknowledgements
1640 The authors wish to thank Lou Berger, David Black, Stewart Bryant,
1641 Rodney Cummings, Ethan Grossman, Craig Gunther, Marcel Kiessling,
1642 Rudy Klecka, Jouni Korhonen, Erik Nordmark, Shitanshu Shah, Wilfried
1643 Steiner, George Swallow, Michael Johas Teener, Pat Thaler, Thomas
1644 Watteyne, Patrick Wetterwald, Karl Weber, Anca Zamfir, for their
1645 various contribution with this work.
1647 9. Informative References
1649 [CCAMP] IETF, "Common Control and Measurement Plane Working
1650 Group",
1651 .
1653 [I-D.ietf-6tisch-architecture]
1654 Thubert, P., "An Architecture for IPv6 over the TSCH mode
1655 of IEEE 802.15.4", draft-ietf-6tisch-architecture-14 (work
1656 in progress), April 2018.
1658 [I-D.ietf-detnet-dp-sol-ip]
1659 IETF, "DetNet IP Data Plane Encapsulation", July 2018,
1660 .
1663 [I-D.ietf-detnet-dp-sol-mpls]
1664 IETF, "DetNet MPLS Data Plane Encapsulation", July 2018,
1665 .
1668 [I-D.ietf-detnet-problem-statement]
1669 Finn, N. and P. Thubert, "Deterministic Networking Problem
1670 Statement", draft-ietf-detnet-problem-statement-05 (work
1671 in progress), June 2018.
1673 [I-D.ietf-detnet-use-cases]
1674 Grossman, E., "Deterministic Networking Use Cases", draft-
1675 ietf-detnet-use-cases-17 (work in progress), June 2018.
1677 [I-D.ietf-spring-segment-routing]
1678 Filsfils, C., Previdi, S., Ginsberg, L., Decraene, B.,
1679 Litkowski, S., and R. Shakir, "Segment Routing
1680 Architecture", draft-ietf-spring-segment-routing-15 (work
1681 in progress), January 2018.
1683 [IEEE802.1BA]
1684 IEEE Standards Association, "IEEE Std 802.1BA-2011 Audio
1685 Video Bridging (AVB) Systems", 2011,
1686 .
1688 [IEEE802.1CB]
1689 IEEE Standards Association, "IEEE Std 802.1CB Frame
1690 Replication and Elimination for Reliability", 2017,
1691 .
1693 [IEEE802.1Q]
1694 IEEE Standards Association, "IEEE Std 802.1Q-2018 Bridges
1695 and Bridged Networks", 2018,
1696 .
1699 [IEEE802.1Qbu]
1700 IEEE Standards Association, "IEEE Std 802.1Qbu-2016
1701 Bridges and Bridged Networks - Amendment 26: Frame
1702 Preemption", 2016,
1703 .
1705 [IEEE802.1Qbv]
1706 IEEE Standards Association, "IEEE Std 802.1Qbv-2015
1707 Bridges and Bridged Networks - Amendment 25: Enhancements
1708 for Scheduled Traffic", 2015,
1709 .
1711 [IEEE802.1Qch]
1712 IEEE Standards Association, "IEEE Std 802.1Qbv-2015
1713 Bridges and Bridged Networks - Amendment 29: Cyclic
1714 Queuing and Forwarding", 2017,
1715 .
1718 [IEEE802.1TSNTG]
1719 IEEE Standards Association, "IEEE 802.1 Time-Sensitive
1720 Networking Task Group", 2013,
1721 .
1723 [IEEE802.3]
1724 IEEE Standards Association, "IEEE Std 802.3-2015 Standard
1725 for Ethernet", 2015,
1726 .
1728 [IEEE802.3br]
1729 IEEE Standards Association, "IEEE Std 802.3br-2016
1730 Standard for Ethernet Amendment 5: Specification and
1731 Management Parameters for Interspersing Express Traffic",
1732 2016, .
1734 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.
1735 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
1736 Functional Specification", RFC 2205, DOI 10.17487/RFC2205,
1737 September 1997, .
1739 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.,
1740 and W. Weiss, "An Architecture for Differentiated
1741 Services", RFC 2475, DOI 10.17487/RFC2475, December 1998,
1742 .
1744 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
1745 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
1746 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001,
1747 .
1749 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
1750 Jacobson, "RTP: A Transport Protocol for Real-Time
1751 Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550,
1752 July 2003, .
1754 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation
1755 Element (PCE)-Based Architecture", RFC 4655,
1756 DOI 10.17487/RFC4655, August 2006,
1757 .
1759 [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau,
1760 L., and L. Berger, "A Framework for MPLS in Transport
1761 Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010,
1762 .
1764 [RFC6372] Sprecher, N., Ed. and A. Farrel, Ed., "MPLS Transport
1765 Profile (MPLS-TP) Survivability Framework", RFC 6372,
1766 DOI 10.17487/RFC6372, September 2011,
1767 .
1769 [RFC6658] Bryant, S., Ed., Martini, L., Swallow, G., and A. Malis,
1770 "Packet Pseudowire Encapsulation over an MPLS PSN",
1771 RFC 6658, DOI 10.17487/RFC6658, July 2012,
1772 .
1774 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in
1775 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384,
1776 October 2014, .
1778 [RFC7426] Haleplidis, E., Ed., Pentikousis, K., Ed., Denazis, S.,
1779 Hadi Salim, J., Meyer, D., and O. Koufopavlou, "Software-
1780 Defined Networking (SDN): Layers and Architecture
1781 Terminology", RFC 7426, DOI 10.17487/RFC7426, January
1782 2015, .
1784 [RFC7554] Watteyne, T., Ed., Palattella, M., and L. Grieco, "Using
1785 IEEE 802.15.4e Time-Slotted Channel Hopping (TSCH) in the
1786 Internet of Things (IoT): Problem Statement", RFC 7554,
1787 DOI 10.17487/RFC7554, May 2015,
1788 .
1790 [RFC7813] Farkas, J., Ed., Bragg, N., Unbehagen, P., Parsons, G.,
1791 Ashwood-Smith, P., and C. Bowers, "IS-IS Path Control and
1792 Reservation", RFC 7813, DOI 10.17487/RFC7813, June 2016,
1793 .
1795 [TEAS] IETF, "Traffic Engineering Architecture and Signaling
1796 Working Group",
1797 .
1799 Authors' Addresses
1801 Norman Finn
1802 Huawei
1803 3755 Avocado Blvd.
1804 PMB 436
1805 La Mesa, California 91941
1806 US
1808 Phone: +1 925 980 6430
1809 Email: norman.finn@mail01.huawei.com
1811 Pascal Thubert
1812 Cisco Systems
1813 Village d'Entreprises Green Side
1814 400, Avenue de Roumanille
1815 Batiment T3
1816 Biot - Sophia Antipolis 06410
1817 FRANCE
1819 Phone: +33 4 97 23 26 34
1820 Email: pthubert@cisco.com
1822 Balazs Varga
1823 Ericsson
1824 Magyar tudosok korutja 11
1825 Budapest 1117
1826 Hungary
1828 Email: balazs.a.varga@ericsson.com
1829 Janos Farkas
1830 Ericsson
1831 Magyar tudosok korutja 11
1832 Budapest 1117
1833 Hungary
1835 Email: janos.farkas@ericsson.com