idnits 2.17.1 draft-mohammadpour-detnet-bounded-delay-variation-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (10 September 2021) is 959 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-10) exists of draft-ietf-detnet-bounded-latency-07 Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet E. Mohammadpour 3 Internet-Draft J-Y. Le Boudec 4 Intended status: Informational EPFL 5 Expires: 14 March 2022 10 September 2021 7 DetNet Bounded Packet-Delay-Variation 8 draft-mohammadpour-detnet-bounded-delay-variation-00 10 Abstract 12 Some DetNet use-cases (applications) require guaranteed bounds on 13 packet delay-variation, not just on latency. This document gives a 14 methodology to derive guaranteed packet delay-variation bounds in 15 DetNet and apply it to a number of proposed mechanisms. When the 16 required packet delay-variations is very low, clock non-idealities 17 affect the bounds, even in a synchronized DetNet networks. This 18 document also gives a methodology to account for such an effect. 20 Status of This Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at https://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on 14 March 2022. 37 Copyright Notice 39 Copyright (c) 2021 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 44 license-info) in effect on the date of publication of this document. 45 Please review these documents carefully, as they describe your rights 46 and restrictions with respect to this document. Code Components 47 extracted from this document must include Simplified BSD License text 48 as described in Section 4.e of the Trust Legal Provisions and are 49 provided without warranty as described in the Simplified BSD License. 51 Table of Contents 53 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 54 2. Terminology and Definitions . . . . . . . . . . . . . . . . . 3 55 3. Clock Model . . . . . . . . . . . . . . . . . . . . . . . . . 3 56 4. Computing End-to-end Packet-Delay-Variation Bound . . . . . . 4 57 4.1. DetNet Time Model . . . . . . . . . . . . . . . . . . . . 4 58 4.2. Methodology . . . . . . . . . . . . . . . . . . . . . . . 4 59 5. Packet Scheduling Techniques . . . . . . . . . . . . . . . . 5 60 5.1. Guaranteed-Service IntServ . . . . . . . . . . . . . . . 5 61 5.2. Differentiated Services . . . . . . . . . . . . . . . . . 5 62 5.3. Credit-Based Shaper with Asynchronous Traffic Shaping . . 5 63 5.4. Cyclic Queuing and Forwarding (CQF) . . . . . . . . . . . 5 64 5.5. Dampers . . . . . . . . . . . . . . . . . . . . . . . . . 5 65 5.5.1. Damper Classification . . . . . . . . . . . . . . . . 7 66 5.5.2. Bound Computations . . . . . . . . . . . . . . . . . 8 67 5.6. Mechanism XXX . . . . . . . . . . . . . . . . . . . . . . 9 68 5.7. Mechanism XXX . . . . . . . . . . . . . . . . . . . . . . 9 69 5.8. Mechanism XXX . . . . . . . . . . . . . . . . . . . . . . 9 70 6. Example application on DetNet IP network . . . . . . . . . . 9 71 7. Security considerations . . . . . . . . . . . . . . . . . . . 10 72 8. IANA considerations . . . . . . . . . . . . . . . . . . . . . 10 73 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 74 9.1. Normative References . . . . . . . . . . . . . . . . . . 10 75 9.2. Informative References . . . . . . . . . . . . . . . . . 10 76 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 12 78 1. Introduction 80 Some applications that use DetNet networks, such as such as 81 industrial Internet of Things [ITU-Y3000] and electrical utilities 82 [RFC8578], require not just guaranteed bounds on the worst-case 83 packet delay, but also on packet delay variation (PDV), defined as 84 the difference between worst-case and best-case delays. 86 A general framework to compute latency bounds is presented in 87 [I-D.ietf-detnet-bounded-latency]. In this document, we extend this 88 framework to compute guaranteed bounds on PDV. 90 When the packet-delay-variation requirement is very low, even in a 91 time-synchronized DetNet network, clock non-idealities affect the 92 bounds, as seen, e.g., in [MohammadpourDamper]. This document gives 93 a methodology, derived from [ThomasTime], to incorporate such effects 94 in the computation of packet-delay-variation bounds within DetNet. 96 This document also applies the presented framework to compute packet- 97 delay-variation bounds on a number of packet scheduling mechanisms, 98 some of which are taken from [RFC8655] and 99 [I-D.ietf-detnet-bounded-latency], while others are specifically 100 targetting low PDV. Finally, this document gives an application of 101 the framework to compute end-to-end packet-delay-variation bounds on 102 a sample DetNet network with a combination of various packet 103 scheduling mechanisms. 105 2. Terminology and Definitions 107 This document uses the terms defined in [RFC8655]. This document 108 also uses the following terms:. 110 PDV 111 Packet Delay-Variation as in [RFC3393]. It is also called 112 "latency variation" or "jitter" in [RFC8655]. 114 3. Clock Model 116 We call H_TAI the perfect clock, i.e. the international atomic time 117 (Temps Atomique International). In practice, the local clock of a 118 system deviates from the perfect clock [ThomasTime]. In time- 119 sensitive networks, clocks can be synchronized or non-synchronized. 120 Non-synchronized clocks are independently configured and do not 121 interact with each other; this corresponds to the free-running mode 122 in Section 4.4.1 of [g810]. When clocks are synchronized, using 123 methods like Network Time Protocol (NTP), Precision Time Protocol 124 (PTP), WhiteRabbit, Global Positioning System (GPS), the occurrence 125 of an event, when measured with different clocks, is bounded by the 126 time error bound (~1us or less in PTP, WhiteRabbit, and GPS; ~100ms 127 in NTP). 129 This document follows the clock model in [ThomasTime], which applies 130 to time-sensitive networks. Consider a clock H_i that is either 131 synchronized with time error bound ω, or not synchronized (in 132 which case we set ω=+∞). Let d^H_i [resp. d^H_TAI] be a 133 delay measurement done with clock H_i [resp. in TAI], then 134 [ThomasTime]: 136 d^H_TAI - d^H_i <= min((ρ-1) * d^H_i + η, 2ω), 137 d^H_TAI - d^H_i >= - min((1- 1/ρ) * d^H_i+ η/ρ, 138 2ω), 140 where ρ is the stability bound and η the timing-jitter 141 bound of the clock H_i. Note that this set of bounds is symmetric, 142 i.e. we can exchange the roles of H_i and H_TAI in the above 143 equation. We assume that the parameters ω, ρ,η are 144 valid for all clocks in the network, i.e. we consider network-wide 145 time-error, stability and time-jitter bounds. 147 4. Computing End-to-end Packet-Delay-Variation Bound 149 Computation of end-to-end PDV bound requires a time model that 150 includes all the sources of latency within a flow path. In this 151 document we use the existing time model presented in 152 [I-D.ietf-detnet-bounded-latency]. 154 4.1. DetNet Time Model 156 Figure 1 is a breakdown of the per-hop latency experienced by a 157 packet passing through a DetNet transit node, taken from 158 [I-D.ietf-detnet-bounded-latency]. 160 DetNet transit node A DetNet transit node B 161 +-------------------------+ +------------------------+ 162 | Queuing | | Queuing | 163 | Regulator subsystem | | Regulator subsystem | 164 | +-+-+-+-+ +-+-+-+-+ | | +-+-+-+-+ +-+-+-+-+ | 165 -->+ | | | | | | | | | + +------>+ | | | | | | | | | + +---> 166 | +-+-+-+-+ +-+-+-+-+ | | +-+-+-+-+ +-+-+-+-+ | 167 | | | | 168 +-------------------------+ +------------------------+ 169 |<->|<------>|<------->|<->|<---->|<->|<------>|<------>|<->|<-- 170 2,3 4 5 6 1 2,3 4 5 6 1 2,3 171 1: Output delay 4: Processing delay 172 2: Link delay 5: Regulation delay 173 3: Frame preemption delay 6: Queuing delay 175 Figure 1: Timing model for DetNet or TSN 177 4.2. Methodology 179 Consider a DetNet flow (or an aggregate of DetNet flows). The end- 180 to-end delay-variation for the flow is defined as difference between 181 the end-to-end worst-case and best-case latencies of its packets, 182 measured in TAI, 184 e2e_PDV = e2e_worst_case_latency - e2e_best_case_latency. 186 "V" is an upper-bound on the end-to-end PDV of the flow if and only 187 if for any two packets n and m with end-to-end latencies "d_n" and 188 "d_m" 190 |d_n - d_m| <= V. 192 Then V is computed as: 194 V = e2e_latency_upper_bound - e2e_latency_lower_bound. 196 A general framework to compute end_to_end_latency_upper_bound is 197 described in [I-D.ietf-detnet-bounded-latency]. The same framework 198 can be used to compute e2e_latency_lower_bound; in Section 5, we 199 provide the bound for a set of queuing mechanisms. 201 5. Packet Scheduling Techniques 203 This section provides formulas to compute PDV bounds for a number of 204 packet scheduling mechanisms within DetNet networks. 206 5.1. Guaranteed-Service IntServ 208 TBD from [I-D.ietf-detnet-bounded-latency] and [RFC2212]. 210 5.2. Differentiated Services 212 TBD from [RFC2475] and [RFC7657]. 214 5.3. Credit-Based Shaper with Asynchronous Traffic Shaping 216 TBD from [I-D.ietf-detnet-bounded-latency] 218 5.4. Cyclic Queuing and Forwarding (CQF) 220 TBD from [IEEE8021Q] 222 5.5. Dampers 224 Dampers are proposed to reduce packet delay-variation in time- 225 sensitive networks [VermaJitter],[ZhangRCSP],[CruzScedPlus]. A 226 damper delays every DetNet packet by an amount written in a packet 227 header field, called damper header, which carries an estimate of the 228 earliness of this packet with respect to a known latency upper-bound 229 of upstream systems. This ideally leads to zero PDV; in practice, 230 there is still some small residual PDV, due to errors in acquiring 231 timestamps and in computing and implementing delays. As a positive 232 side effect, dampers create packet timings that are almost the same 233 as at the source, with small errors due to residual PDV, and thus 234 cancel most of the burstiness increase imposed by the network. The 235 residual burstiness increase that remains when dampers are used is 236 not influenced by the burstiness of cross-traffic. Thus, dampers 237 solve the burstiness cascade issue [CharnyDelay]: individual flows 238 that share a resource dedicated to a class may see their burstiness 239 increase, which may in turn increase the burstiness of other 240 downstream flows. Furthermore, dampers are stateless; hence, solving 241 the burstiness cascade in a stateless manner makes the dampers of 242 interest for DetNet networks. 244 We call jitter-compensated system (JCS) any delay element or 245 aggregate of delay elements with known latency and PDV bounds, for 246 which we want to compensate PDV by means of dampers. This is 247 typically the queuing system on the output port of a switch or router 248 used in DetNet transit nodes. It can also be a switching fabric or 249 an input port processing unit, or even a larger system. For DetNet 250 flows, a JCS should be able to time stamp packet arrivals and 251 departures using the available local times. It should also increment 252 the damper header field in every DetNet packet (if one is present) by 253 an amount equal to an estimate of the earliness of this packet with 254 respect to the known latency upper-bound δ at the JCS. If no 255 damper header is present, it inserts one, with a value equal to the 256 estimated earliness. The earliness is computed as: 258 earliness = δ - actual_delay_in_the_JCS. 260 When a DetNet flow crosses a JCS, for actual PDV removal to occur, 261 there must be a downstream damper on the path of the flow 262 [MohammadpourDamper] (e.g. if the JCS is a switch output port, the 263 next downstream damper is typically located on the output port of the 264 next downstream switch). The damper also resets the damper header, 265 so that the next downstream damper will see only the earliness 266 accumulated downstream of this damper. Designing a stand-alone 267 damper is a challenge, because such a damper may need to release a 268 large number of packets instantly or within a very short time, which 269 might not be feasible. This is why damper implementations are often 270 associated with queuing systems; then, the time at which a damper 271 releases a packet is simply the time at which the packet becomes 272 visible to the queuing system. 274 It is generally not possible, or required, to remove PDV in all 275 network elements, because time stamping and damper-header update come 276 with a cost. Therefore, it is required, for our timing analysis, to 277 consider what we call bounded-delay systems (BDSs), defined as any 278 delay element or aggregate of delay elements with known latency and 279 PDV bounds, and for which we do not compensate PDV. Constant latency 280 elements (e.g. an output link propagation delay), variable delay 281 elements with very low jitter (e.g., very high speed backbone 282 network) are examples of BDSs. 284 We classify and model existing designs of dampers in next subsections 285 and give formulas for the computation of PDV and latency bounds, 286 taken from [MohammadpourDamper]. 288 5.5.1. Damper Classification 290 An ideal damper delays a packet by exactly the amount required by the 291 damping header. Consider a packet n with damper header H_n that 292 arrives at local time Q_n to a damper. Then an ideal damper releases 293 the packet at time E_n: 295 E_n = Q_n + H_n. 297 Jitter-control Earliest-Deadline-First [VermaJitter] is an ideal 298 damper, used in combination with an Earliest-Deadline-First 299 scheduler. 301 Many other implementations of dampers use some tolerance for the 302 packet release times, due to the difficulty of implementing exact 303 timings. We call damper with tolerances Δ^L,Δ^U, a damper 304 such that the eligibility time E_n, of packet n, in local time, 305 satisfies: 307 Q_n + H_n - Δ^L >= E_n <= Q_n + H_n + Δ^U. 309 The tolerances can vary from hundreds of nanosecond to a few 310 microsecond based on implementation. RCSP [ZhangRCSP] is an instance 311 of dampers with tolerance. Since the definition of damper with 312 tolerance does not preclude packet misordering, re-sequencing and 313 head-of-line dampers avoid packet misordering. 315 Re-sequencing damper with tolerances Δ^L,Δ^U is a system 316 that behaves as the concatenation of a damper with same tolerances 317 and a re-sequencing buffer that, if needed, re-orders packets based 318 on the packet order at the entrance of the damper. The packet order 319 is with respect to a flow of interest. SCED+ [CruzScedPlus] is an 320 instance of re-sequencing dampers. 322 Head-of-line damper is introduced in [GrigorjewJCATS] and is 323 implemented as a FIFO queue. It has tolerance parameters 324 Δ^L,Δ^U as well as processing bounds φ^min,φ^max. 325 A head-of-line damper behaves as re-sequencing damper with with 326 tolerances Δ^L,Δ^U followed by a single-server FIFO queue 327 with processing bounds φ^min,φ^max. When a packet arrives, 328 its arrival time is collected and the packet is stored at the tail of 329 the queue. Only the packet at the head of the queue is examined; if 330 its eligibility time is passed, it is immediately released, otherwise 331 it is delayed and released at its eligibility time. When the head 332 packet is released, it is removed from the damper queue and the next 333 packet (if any) becomes the head of the queue and is examined. When 334 an arriving packet finds an empty queue, it is immediately examined. 335 By construction, packet ordering is preserved. 337 5.5.2. Bound Computations 339 In this subsection, we provide latency and PDV bounds for a simple 340 case (general case is available in [MohammadpourDamper]) where we 341 want to compensate the PDV imposed by the queuing delay (6) in 342 Figure 1 by the mean of dampers. Therefore the queuing subsystem is 343 a JCS with a known latency upper-bound (the latency upper-bound 344 includes the delay from first-bit-in to last-bit-in hidden in the 345 link delay (2) of Figure 1). A damper is placed before the queuing 346 subsystems (replaced the regulator in Figure 1). The delays (1), (2) 347 only the first-bit-out to first-bit-in, (3), (4) in Figure 1 are 348 assumed to be the BDSs. 350 Assume that the queuing subsystem has latency upper-bound δ and 351 PDV J, and the latency upper-bound, lower-bound and PDV bound on the 352 BDSs are π, π' and v. Also, assume that a damper with 353 tolerances Δ^L,Δ^U is placed before the queuing subsystem 354 in the DetNet transit node B. Then, the latency lower-bound, upper- 355 bound and the PDV bound from the entrance of queuing subsystem in 356 transit node A to the entrance of queuing subsytem in transit node B, 357 in TAI, are computed as follows. 359 If damper with tolerances Δ^L,Δ^U is used: 361 latency_lower-bound = δ + π' - Δ^L - ε - 362 ψ' , 364 latency_upper-bound = δ + π + Δ^U + ε + ψ 365 , 367 PDV_bound = v + 2 * ε + ψ + ψ' , 369 where 370 ψ' = min((ρ-1) * (δ + Δ^U + ε) + 2 * 371 η , 4 * ω), 373 ψ = min((1-1\ρ) * (δ - Δ^L + -ε) ) + 2 * 374 η/ρ , 4 * ω). 376 If re-sequencing damper with tolerances Δ^L,Δ^U is used and 377 all the other elements are FIFO, the same bounds as mentioned above 378 is obtained. 380 If head-of-line damper with tolerances Δ^L,Δ^U and 381 processing bounds φ^min,φ^max is used and all the other 382 elements are FIFO, the latency_upper-bound and PDV_bound are 383 increased by θ where 385 θ = ((b + r * PDV_bound) * φ^max ; if φ^max <= 1/r, 387 θ = +∞ ; if φ^max > 1/r, 389 where the DetNet flow has per-packet leaky-bucket arrival curve at 390 the entrance of queuing subsystem at DetNet transit node A, i.e., the 391 number of packets that can be emitted by the flow within any period 392 of time t is not larger than r * t + b where r is the rate of packets 393 and b is bucket size in packets. 395 When an element is not FIFO in Figure 1, comparing to dampers with 396 tolerance, using a re-sequencing damper worsens the PDV bound by J 397 and a head-of-line damper worsens the PDV bound by 2 * J; further 398 discussion is available in [MohammadpourDamper]. 400 5.6. Mechanism XXX 402 TBD 404 5.7. Mechanism XXX 406 TBD 408 5.8. Mechanism XXX 410 TBD 412 6. Example application on DetNet IP network 414 TBD 416 7. Security considerations 418 Detailed security considerations for DetNet are cataloged in 419 [RFC9055], and more general security considerations are described in 420 [RFC8655]. 422 8. IANA considerations 424 This document has no IANA actions. 426 9. References 428 9.1. Normative References 430 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 431 "Deterministic Networking Architecture", RFC 8655, 432 DOI 10.17487/RFC8655, October 2019, 433 . 435 9.2. Informative References 437 [CharnyDelay] 438 A. Charny and J.-Y. Le Boudec, "Delay Bounds in a Network 439 with Aggregate Scheduling", 2002, 440 . 443 [CruzScedPlus] 444 R. L. Cruz, "SCED+: efficient management of quality of 445 service guarantees", 1998, 446 . 448 [g810] L. Thomas and J.-Y. Le Boudec, "G.810 : Definitions and 449 terminology for synchronization networks", 450 . 452 [GrigorjewJCATS] 453 A. Grigorjew, F. Metzger, T. Hossfeld, J. Specht, F.-J. 454 Goetz, F. Chen, and J. Schmitt, "Asynchronous Traffic 455 Shaping with Jitter Control", 2020, 456 . 459 [I-D.ietf-detnet-bounded-latency] 460 N. Finn, J-Y. Le Boudec, E. Mohammadpour, J. Zhang, B. 461 Varga, and J. Farkas, "DetNet Bounded Latency", 462 . 465 [IEEE8021Q] 466 IEEE 802.1, "IEEE Std 802.1Q-2018: IEEE Standard for Local 467 and metropolitan area networks - Bridges and Bridged 468 Networks", 2018, 469 . 471 [ITU-Y3000] 472 ITU-T, "ITU-T Y.3000-series - Representative use cases and 473 key network requirements for Network 2030", 2020, 474 . 476 [MohammadpourDamper] 477 E. Mohammadpour and J.-Y. Le Boudec, "Analysis of Dampers 478 in Time-Sensitive Networks with Non-ideal Clocks", 2021, 479 . 481 [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification 482 of Guaranteed Quality of Service", RFC 2212, 483 DOI 10.17487/RFC2212, September 1997, 484 . 486 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., 487 and W. Weiss, "An Architecture for Differentiated 488 Services", RFC 2475, DOI 10.17487/RFC2475, December 1998, 489 . 491 [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 492 Metric for IP Performance Metrics (IPPM)", RFC 3393, 493 DOI 10.17487/RFC3393, November 2002, 494 . 496 [RFC7657] Black, D., Ed. and P. Jones, "Differentiated Services 497 (Diffserv) and Real-Time Communication", RFC 7657, 498 DOI 10.17487/RFC7657, November 2015, 499 . 501 [RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases", 502 RFC 8578, DOI 10.17487/RFC8578, May 2019, 503 . 505 [RFC9055] Grossman, E., Ed., Mizrahi, T., and A. Hacker, 506 "Deterministic Networking (DetNet) Security 507 Considerations", RFC 9055, DOI 10.17487/RFC9055, June 508 2021, . 510 [ThomasTime] 511 L. Thomas and J.-Y. Le Boudec, "On Time Synchronization 512 Issues in Time-Sensitive Networks with Regulators and 513 Nonideal Clocks", 2020, 514 . 516 [VermaJitter] 517 D.C. Verma, H. Zhang, and D. Ferrari, "Delay jitter 518 control for real-time communication in a packet switching 519 network", 1991, 520 . 522 [ZhangRCSP] 523 H. Zhang and D. Ferrari, "Rate-controlled static-priority 524 queueing", 1993, 525 . 527 Authors' Addresses 529 Ehsan Mohammadpour 530 EPFL 531 IC Station 14 532 CH-1015 Lausanne EPFL 533 Switzerland 535 Email: ehsan.mohammadpour@epfl.ch 537 Jean-Yves Le Boudec 538 EPFL 539 IC Station 14 540 CH-1015 Lausanne EPFL 541 Switzerland 543 Email: jean-yves.leboudec@epfl.ch