idnits 2.17.1 draft-peng-lsr-flex-algo-deterministic-routing-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: Note that the FAD Binding CQF Sub-Sub-TLV and the FAD Binding Deadline Sub-Sub-TLV MUST not appear in FAD at the same time, otherwise, the first one is selected. -- The document date (February 27, 2022) is 788 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-26) exists of draft-ietf-lsr-flex-algo-18 == Outdated reference: A later version (-17) exists of draft-ietf-lsr-ip-flexalgo-04 == Outdated reference: A later version (-13) exists of draft-ietf-rtgwg-segment-routing-ti-lfa-08 == Outdated reference: A later version (-09) exists of draft-peng-detnet-deadline-based-forwarding-00 ** Downref: Normative reference to an Informational RFC: RFC 2475 ** Downref: Normative reference to an Informational RFC: RFC 8578 Summary: 2 errors (**), 0 flaws (~~), 6 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group Shaofu. Peng 3 Internet-Draft ZTE Corporation 4 Intended status: Standards Track Tony. Li 5 Expires: August 31, 2022 Juniper Networks 6 February 27, 2022 8 IGP Flexible Algorithm with Deterministic Routing 9 draft-peng-lsr-flex-algo-deterministic-routing-02 11 Abstract 13 IGP Flex Algorithm proposes a solution that allows IGPs themselves to 14 compute constraint based paths over the network, and it also 15 specifies a way of using Segment Routing (SR) Prefix-SIDs and SRv6 16 locators, or pure IP prefix to steer packets along the constraint- 17 based paths. This document describes how to compute deterministic 18 delay paths within Flex-algo plane. 20 Status of This Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at https://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on August 31, 2022. 37 Copyright Notice 39 Copyright (c) 2022 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (https://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 55 2. Requirements Language . . . . . . . . . . . . . . . . . . . . 4 56 3. Determinisitc Links . . . . . . . . . . . . . . . . . . . . . 4 57 3.1. Deterministic Link Bound with CQF . . . . . . . . . . . . 5 58 3.2. Deterministic Link Bound with Deadline . . . . . . . . . 6 59 4. Deterministic Delay Metric Extension to ISIS . . . . . . . . 7 60 4.1. CQF Scheduding Delay Intra Node Sub-Sub-TLV . . . . . . . 8 61 4.2. Deadline Scheduding Delay Intra Node Sub-Sub-TLV . . . . 10 62 4.2.1. Another Simplified Extension . . . . . . . . . . . . 11 63 5. Deterministic Delay Metric Extension to OSPF . . . . . . . . 12 64 6. Announcement Suppression . . . . . . . . . . . . . . . . . . 12 65 7. Deterministic Routes Computation . . . . . . . . . . . . . . 12 66 7.1. Bind CQF Scheduling parameters with Flex-Algo . . . . . . 13 67 7.1.1. ISIS Advertisement of Flex-algo Binding CQF . . . . . 13 68 7.1.2. OSPF Advertisement of Flex-algo Binding CQF . . . . . 14 69 7.2. Bind Deadline Scheduling parameters with Flex-Algo . . . 14 70 7.2.1. ISIS Advertisement of Flex-algo Binding Deadline . . 14 71 7.2.2. OSPF Advertisement of Flex-algo Binding Deadline . . 15 72 7.3. CQF based Deterministic Routes Computation . . . . . . . 16 73 7.4. Deadline based Deterministic Routes Computation . . . . . 16 74 8. Routing Convergence and Redundance Considerations . . . . . . 18 75 9. Examples of Deterministic delay SPF . . . . . . . . . . . . . 20 76 9.1. CQF Based Deterministic Delay SPF Path Example . . . . . 20 77 9.2. Deadline Based Deterministic Delay SPF Path Example . . . 21 78 10. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 23 79 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24 80 11.1. ISIS Deterministic Delay Metric Sub-TLV . . . . . . . . 24 81 11.2. Sub-Sub-TLVs for ISIS Deterministic Delay Metric Sub-TLV 24 82 11.3. IGP Metric-Type Registry . . . . . . . . . . . . . . . . 24 83 11.4. ISIS Sub-Sub-TLVs for Flexible Algorithm Definition Sub- 84 TLV . . . . . . . . . . . . . . . . . . . . . . . . . . 25 85 11.5. OSPF IANA considerations . . . . . . . . . . . . . . . . 25 86 12. Security Considerations . . . . . . . . . . . . . . . . . . . 25 87 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 25 88 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 25 89 14.1. Normative References . . . . . . . . . . . . . . . . . . 25 90 14.2. Informative References . . . . . . . . . . . . . . . . . 26 91 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 27 93 1. Introduction 95 IGP Flex Algorithm [I-D.ietf-lsr-flex-algo] proposes a solution that 96 allows IGPs themselves to compute constraint based paths over the 97 network, and it also specifies a way of using Segment Routing 98 [RFC8402] Prefix-SIDs and SRv6 locators, or pure IP prefix 99 [I-D.ietf-lsr-ip-flexalgo] to steer packets along the constraint- 100 based paths. It specifies a set of extensions to ISIS, OSPFv2 and 101 OSPFv3 that enable a router to send TLVs that identify (a) 102 calculation-type, (b) specify a metric-type, and (c )describe a set 103 of constraints on the topology, that are to be used to compute the 104 best paths along the constrained topology. A given combination of 105 calculation-type, metric-type, and constraints is known as an FAD 106 (Flexible Algorithm Definition). 108 [RFC8655] describes the architecture of deterministic network and 109 defines the QoS goals of deterministic forwarding: Minimum and 110 maximum end-to-end latency from source to destination, timely 111 delivery, and bounded jitter (packet delay variation); packet loss 112 ratio under various assumptions as to the operational states of the 113 nodes and links; an upper bound on out-of-order packet delivery. In 114 order to achieve these goals, deterministic networks use resource 115 reservation, explicit routing, service protection and other means. A 116 deterministic path is typically (but not necessarily) explicit routes 117 so that it does not normally suffer temporary interruptions caused by 118 the convergence of routing or bridging protocols. 120 IGP Flex-algo has the characteristic mentioned in [RFC8655]: under a 121 single administrative control or within a closed group of 122 administrative control. IGP Flex-algo supports Min Unidirectional 123 Link Delay (defined in [RFC8570]) metric type to compute shortest 124 paths with minimum delay, however, the cumulative delay is 125 essentially the accumulation of transmission delay of all links, 126 excluding node delay. In order to make up for this gap, it is 127 necessary to enhance IGP flex-algo to compute the path with 128 deterministic delay, i.e., including deterministic node delay and 129 link transmission delay. 131 This document describes how to compute distributed shortest paths 132 with deterministic delay metric within Flex-algo plane, as the basis 133 of the whole distributed deterministic scheme. It should be noted 134 that relying on this enhancement alone does not guarantee complete 135 determinacy, it needs to be used in conjunction with other tools, 136 such as creating additional redundant deterministic delay path with 137 consistent delay metric for PREOF (Packet Replication, Elimination, 138 and Ordering Functions), smoothing the delay jitter during route 139 convergence, providing deterministic forwarding mechanism, admission 140 control, etc. 142 2. Requirements Language 144 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 145 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 146 "OPTIONAL" in this document are to be interpreted as described in BCP 147 14 [RFC2119] [RFC8174] when, and only when, they appear in all 148 capitals, as shown here. 150 3. Determinisitc Links 152 When a packet is forwarded to a link, the delay produced includes two 153 parts: the first part is the dwell delay of the packet in the node, 154 and the second part is the transmission delay of the packet on the 155 link. In packet switching networks, priority based queuing scheme is 156 generally used. It may give better average latency, but may have 157 worst case latency. [SP-LATENCY] analyzes the guaranteed latency 158 with the traditional strict priority scheme, and shows that low 159 bounded latency is achievable when high priority traffic is 160 constrained in low utilization, but deteriorates quickly with 161 increasing utilization of high priority traffics. DiffServ [RFC2475] 162 with strict priority has been widely deployed in the network, the 163 existing non-deterministic service flow may set the highest priority, 164 so it is difficult to support deterministic services based on it 165 without any modification. We call those links bound with a queue 166 mechanism that can not guarantee node delay are non-determinisitc 167 links. 169 On the contrary, those links bound with a queue mechanism that can 170 provide deterministic node delay are called deterministic links. 171 Therefore, other new scheduling mechanisms need to be introduced, and 172 their scheduling priority is higher than that of the traditional 173 strict priority queue. The typical queue mechanisms are as follows: 175 o IEEE 802.1 WG has specified IEEE802.1Qav [CBS] which uses credit- 176 based shaper mechanism to assign packets to different queues by 177 giving a credit value which is related with reserved bandwidth. 178 The credit values of different transmission queues will 179 automatically change with the packet transmission process, which 180 will ensure that the packet with lower priority will also get 181 transmission. CBS shaper is similar to Weighted Fair Queuing 182 (WFQ), and they all control the sending of packets based on 183 reserved bandwidth. The worst-case delay calculation of class A 184 of CBS is relatively simple, but other classes are complex. For 185 class A traffic, the queuing delay equals to the maximum size of 186 the interference frame (such as 2000 octes) divided by the port 187 bandwidth. 189 o IEEE 802.1 WG has specified IEEE802.1Qch [CQF] which uses cyclic 190 queuing and forwarding (CQF) mechanism and relies on time 191 synchronization. According to CQF, the maximum delay experienced 192 by a given packet is (H+1)*D, the minimum delay experienced by a 193 given packet is (H-1)*D, and the delay jitter is 2*D, where H is 194 the number of hops and D is cycle duration. Other variants based 195 on CQF can avoid relying on time synchronization, but only the 196 same cycle duration for all nodes. Basically, the packet received 197 in the current sending window (i.e., cycle) will ensure that it 198 can be sent in the next sending window, then the deterministic 199 node delay, on average, is one cycle duration, or seveval cycle 200 durations if the forwarding delay intra node (from incoming port 201 to outgoing port) can't be ignored. 203 o [I-D.peng-detnet-deadline-based-forwarding] introduced a deadline 204 based forwarding mechanism that allow packet to control its 205 expected dwell time in the node according to the planned deadline. 206 There are two policies for deadline queue to schedule packets. 207 For in-time policy, the end-to-end delay is H*(F~D), jitter is 208 H*Q, where, H is the number of hops, F is the forwarding delay 209 intra node, D is the planned deadline, and Q is the scheduling 210 delay; For on-time policy, the end-to-end delay is H*D, jitter is 211 0 (however there may be one authorization time due to the 212 granularity of queue scheduling). That is, the packet received at 213 any time will ensure that it can be sent in offset time F~D or D 214 respectively for these two policies. 216 This document mainly describes the deterministic link based on CQF or 217 Deadline algorithm. Other algorithms will be described in the 218 future. 220 3.1. Deterministic Link Bound with CQF 222 A node may configure the CQF based packet scheduling parameter 223 information for its local link, including CQF scheduling enable/ 224 disable, one or more cycle durations. Accordingly, for each cycle 225 duration, the node delay/jitter attributes of the link will be 226 obtained. The meanings of these parameters or attributes of the link 227 are as follows: 229 o CQF scheduling enable/disable: the CQF scheduling algorithm can be 230 enabled for a link, then the packets sent to that link will be 231 scheduled by the CQF scheduling algorithm. 233 o Cycle duration: the duration of the cycle of CQF, which is also 234 called cycle_size. One or more cycle_size with different lengths 235 can be configured for a link, such as 10us, 20us, 30us, and so on. 237 o Node delay/jitter: 239 * According to classical TSN CQF, for a given cycle_size, it can 240 be deduced that the minimum delay in the node of the packet is 241 0, the maximum delay in the node is 2*cycle_size, the average 242 delay in the node is one cycle_size, and the delay jitter in 243 the node is 2*cycle_size. The detailed reasons for these data 244 are as follows: if a node receives a packet at the tail end of 245 cycle i and sends that packet at the head end of cycle i+1, the 246 resulting node delay, i.e., the minimum node delay, is 0; if a 247 node receives a packet at the head end of cycle i and sends 248 that packet at the tail end of cycle i+1, the resulting node 249 delay, i.e., the maximum node delay, is 2*cycle_ size; the 250 average node delay is one cycle_size, and the node delay jitter 251 is 2*cycle_size. Each cycle_size corresponds to a different 252 set of delay/jitter attributes. 254 * However, for some variants based on TSN CQF, if the forwarding 255 delay intra node can't be ignored, e.g, wasting 2 cycle 256 duration, then the minimum node delay, the maximum node delay, 257 and the average node delay need to add 2 cycle_size 258 respectively, but the node delay jitter is still 2*cycle_size. 260 3.2. Deterministic Link Bound with Deadline 262 A node may configure the deadline based packet scheduling parameter 263 information for its local link, including deadline scheduling enable/ 264 disable, one or more deadline scheduling delays, and the scheduling 265 policy supported for each deadline scheduling delay. Accordingly, 266 for each deadline scheduling delay, the node delay/jitter attributes 267 of the link will be obtained. The meanings of these parameters or 268 attributes of the link are as follows: 270 o Deadline scheduling enable/disable: the deadline scheduling 271 algorithm can be enabled for a link, then the packet forwarded to 272 the link will be scheduled by the deadline based packet scheduling 273 algorithm. The dwell time of the packet in the node does not 274 exceed the maximum allowable dwell time D, where, D = forwarding 275 delay intra node (F) + specific deadline scheduling delay (Q). 277 o Supported deadline scheduling delay set: the set composed of one 278 or more deadline scheduling delays , assuming 279 that Q1 is the minimum and Qn is the maximum in the set. 280 Generally, the difference between two adjacent elements in the set 281 is fixed, for example, a fixed interval (I). 283 o Scheduling policy: for each scheduling delay Q, there are two 284 scheduling policies: in-time policy and on-time policy. In case 285 of in-time policy, the scheduling delay of the packet may be sent 286 to the outgoing port when it does not reach Q; In the on-time 287 policy, the packet is sent to the outgoing port only when the 288 scheduling delay of the packet is equal to Q. Therefore, for in- 289 time policy, the actual dwell time of the packet in the node is 290 within the range [F, F+Q], i.e., the minimum node delay is F, the 291 maximum node delay is F+Q, and the node delay jitter is Q; For on- 292 time policy, the actual dwell time of the packet in the node is 293 equal to F+Q, i.e., the minimum node delay is F+Q, the maximum 294 node delay is also F+Q, and the node delay jitter is 0 (however, 295 there may be one authorization time due to the granularity of 296 queue scheduling). 298 4. Deterministic Delay Metric Extension to ISIS 300 This document registers a new sub-TLV, Deterministic Delay Metric, in 301 the "Sub-TLVs for IS-IS Sub-TLVs for TLVs Advertising Neighbor 302 Information" registry, to distribute deterministic delay information. 303 The deterministic delay advertised by this sub-TLV MUST be the delay 304 from the local neighbor to the remote neighbor (i.e., the forward- 305 path latency). The format of this sub-TLV is shown in the following 306 diagram: 308 0 1 2 3 309 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 310 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 311 | Type | Length | 312 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 313 |A| RESERVED | Link transmission delay | 314 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 315 | Forwarding Delay Intra Node | 316 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 317 // sub-sub-TLVs for Scheduding Delay Intra Node // 318 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 320 Figure 1 322 where: 324 Type: TBD 326 Length: Variable. 328 A bit: This field represents the Anomalous (A) bit. The A bit is 329 set when one or more measured values of link transmission delay 330 exceed a configured maximum threshold. The A bit is cleared when 331 the measured value falls below its configured reuse threshold. If 332 the A bit is cleared, the sub-TLV represents steady-state link 333 transmission delay. 335 RESERVED: This field is reserved for future use. It MUST be set 336 to 0 when sent and MUST be ignored when received. 338 Link transmission delay: This 24-bit field carries the average 339 measured link transmission delay value (in microseconds) over a 340 configurable interval, encoded as an integer value. 341 Implementations MAY also permit the configuration of an offset 342 value (in microseconds) to be added to the measured delay value, 343 to facilitate the communication of operator-specific delay 344 constraints. When the delay value is set to the maximum value 345 16,777,215 (16.777215 seconds), then the delay is at least that 346 value and may be larger. 348 Forwarding Delay Intra Node: This 16-bit field carries the 349 forwarding delay value (in microseconds) intra node. It 350 represents the latency of packet from the incoming port (or 351 generated from control plane) to the outgoing port. If the 352 forwarding delay can be ignored, it is set to 0. 354 NOTE: for all links of a specific node, it may be possible that 355 they have the same forwarding delay, therefore the forwarding 356 delay can also be advertised by a unified node attribute. This 357 would be considered in future versions. 359 sub-sub-TLVs for Scheduding Delay Intra Node: Optional sub-sub- 360 TLVs are contained to indicate the scheduling delay that is 361 related to the specific scheduling algorithm such as CQF, 362 deadline, etc. If this field is absent, the scheduling delay is 363 unknown. Typically, a link may enable a single scheduling 364 algorithm to get deterministic scheduling delay, so that a single 365 sub-sub-TLV is included. However, it is possible for a link to 366 enable multiple different scheduling algorithms, as long as these 367 algorithms can coordinate the forwarding resources, in this case, 368 multiple sub-sub-TLVs are included. Supported Sub-sub-TLVs are 369 specified in the following sub-sections. 371 4.1. CQF Scheduding Delay Intra Node Sub-Sub-TLV 373 CQF Scheduding Delay Intra Node Sub-Sub-TLV is an optional Sub-Sub- 374 TLV of Deterministic Delay Metric Sub-TLV. At most only one CQF 375 Scheduding Delay Intra Node Sub-Sub-TLV can be included. 377 The following format is defined for the CQF Scheduding Delay Intra 378 Node Sub-Sub-TLV: 380 0 1 2 3 381 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 382 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 383 | Type | Length | Cycle_size 1 | 384 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 385 | Cycle_size 2 | ... ... | 386 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 387 | Cycle_size N | 388 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 390 Figure 2 392 where: 394 Type: TBD 396 Length: 2*N, depending on the count of the cycle_size. 398 Cycle_size: The length of cycle duration, in units of 399 microseconds. A link can support multiple cycle durations, for 400 example, 10us, 20us, 30us, etc, each for a specific service 401 requirement. 403 Only those links that enable CQF scheduling algorithm need to 404 advertise the CQF Scheduding Delay Intra Node Sub-Sub-TLV, otherwise 405 there is no need to advertise. 407 Note that the advertised cycle_size must be consistent with the CQF 408 queue scheduling mechanism actually instantiated by the link in the 409 forwarding plane. If the forwarding plane does not instantiate a CQF 410 queue scheduling supporting a certain cycle_size, which is however 411 advertised in the CQF Scheduding Delay Intra Node Sub-Sub-TLV, the 412 subsequent route computation may get wrong results. 414 For a given cycle_size, it can deduce the corresponding node delay 415 and jitter attributes, so these attributes can no longer be 416 explicitly included in the CQF Scheduding Delay Intra Node Sub-Sub- 417 TLV. 419 As mentioned earlier, if the forwarding delay intra node (assuming F) 420 is not 0, the minimum node delay, the maximum node delay, and the 421 average node delay need to take F into account respectively. F is 422 replaced by ((F/cycle_size)+1)*cycle_size for deducing. That is: 424 o If F is 0, for a given cycle_size, the minimum node delay is 0, 425 the maximum node delay is 2*cycle_size, the average node delay is 426 cycle_size, and the node delay jitter is 2*cycle_size. 428 o If F is not 0, for a given cycle_size, the minimum node delay is 429 ((F/cycle_size)+1)*cycle_size, the maximum node delay is ((F/ 430 cycle_size)+3)*cycle_size, the average node delay is ((F/ 431 cycle_size)+2)*cycle_size, and the node delay jitter is 432 2*cycle_size. 434 4.2. Deadline Scheduding Delay Intra Node Sub-Sub-TLV 436 Deadline Scheduding Delay Intra Node Sub-Sub-TLV is an optional Sub- 437 Sub-TLV of Deterministic Delay Metric Sub-TLV. At most only one 438 Deadline Scheduding Delay Intra Node Sub-Sub-TLV can be included. 440 The following format is defined for the Deadline Scheduding Delay 441 Intra Node Sub-Sub-TLV: 443 0 1 2 3 444 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 445 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 446 | Type | Length | P | Q1 | 447 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 448 | P | Q2 | P | Q3 | 449 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 450 | ... ... | P | Qn | 451 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 453 Figure 3 455 where: 457 Type: TBD 459 Length: 2*N, depending on the count of the supported deadline 460 scheduling delay. 462 Q: Indicates the scheduling delay set, , 463 supported by the link, in units of microseconds. For each 464 supported scheduling delay, the highest two bits represent the 465 scheduling policy P. The value of scheduling policy P can be: 467 0, not defined yet; 469 1, indicates that it supports the in-time policy; 471 2, indicates that it supports the on-time policy; 473 3, indicates that it supports both in-time policy and on-time 474 policy. 476 As mentioned earlier, given the scheduling delay Q and its scheduling 477 policy, combined with the forwarding delay F intra node, the 478 corresponding delay and jitter attributes in the node can be derived. 479 Therefore, these attributes can no longer be explicitly included in 480 the Deadline Scheduding Delay Intra Node Sub-Sub-TLV. 482 Note that the scheduling delay Q advertised in the Deadline 483 Scheduding Delay Intra Node Sub-Sub-TLV must be consistent with the 484 deadline queue scheduling mechanism actually instantiated by the link 485 in the forwarding plane. If the forwarding plane does not 486 instantiate the deadline queue scheduling supporting a certain 487 scheduling delay Q, which is however advertised in the Deadline 488 Scheduding Delay Intra Node Sub-Sub-TLV, the subsequent route 489 computation may get wrong results. 491 4.2.1. Another Simplified Extension 493 If the set to be advertised contains too many 494 elements, and the difference between two adjacent elements in the set 495 is a fixed interval (I), and the scheduling policy supported for all 496 elements are same, another more simplified extension, the Simplified 497 Deadline Scheduding Delay Intra Node Sub-Sub-TLV, can be defined as 498 below. 500 0 1 2 3 501 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 502 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 503 | Type | Length | Q1 | 504 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 505 | Qn | P | I | 506 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 508 Figure 4 510 where: 512 Type: TBD 514 Length: 6. 516 Q1: the minimum scheduling delay, in units of microseconds. 518 Qn: the maximum scheduling delay, in units of microseconds. 520 I: the fixed interval between any two adjacent elements in the 521 set, in units of microseconds. The highest two bits represent the 522 scheduling policy P. The value of scheduling policy P can be: 524 0, not defined yet; 526 1, indicates that it supports the in-time policy; 528 2, indicates that it supports the on-time policy; 530 3, indicates that it supports both in-time policy and on-time 531 policy. 533 5. Deterministic Delay Metric Extension to OSPF 535 To be defined in next version. 537 6. Announcement Suppression 539 The value of Deterministic Delay Metric defined in this document 540 contains node delay provided by instantiated scheduling algorithm and 541 link transmission delay provided by some measure mechanisms. For the 542 announcement of the node delay part, it is constant and depend on the 543 capability of instantiating the scheduling algorithm. However, for 544 the announcement of the link transmission delay part, a measure 545 mechanism may frequently produce different measurements. Please 546 refer to [RFC8570] section 6 for the same principle of announcement 547 suppression. 549 7. Deterministic Routes Computation 551 In order to use the deterministic link resources in the network to 552 compute a deterministic delay SPF path, corresponding Flex-algo plane 553 need to be created. To dinstinguish between traditional low latency 554 SPF path (based on metric type "Min Unidirectional Link Delay") and 555 deterministic low latency SPF path introduced in this document, new 556 metric type, i.e., Deterministic Delay Metric, will be defined and 557 used in Flexible Algorithm Definition (FAD). 559 o Metric-Type: TBD, to be used during the calculation of 560 deterministic low latency SPF path. 562 Additional FAD constraints are also necessary, to bind individual 563 item from the scheduling delay set. 565 It is possible to create multiple flex-algo instances each binding to 566 different scheduling delay for different service requirements. 568 Note that sometimes from the perspective of the end-to-end delay 569 requirements of the service flow, the node delay of the ingress PE 570 node can be ignored and regarded as 0. However, this has no 571 implication for the rules of deterministic low latency SPF path 572 calculation. During the deterministic low latency SPF path 573 calculation, we must always take the node delay into account, because 574 any node may be the intermediate node of an SPF path and its node 575 delay is taken into account. But if the node takes itself as the 576 root and does not consider its node delay when calculating the 577 deterministic low latency SPF path, the calculation results may be 578 inconsistent, resulting in the loop. 580 7.1. Bind CQF Scheduling parameters with Flex-Algo 582 The binding relationship can be configured on 583 one or more nodes participating in the same IGP Flex-algo plane, and 584 then advertised in the IGP domain. If there are multiple binding 585 relationship advertised for the same algorithm, it should choose to 586 use the binding cycle_size contained in the FAD with the highest 587 priority. 589 If a Flex-algo plane eventually uses a binding cycle_size, all links 590 participated to the Flex-algo plane must be configured with CQF 591 scheduling enabled and corresponding cycle_size, otherwise, links 592 that do not meet the conditions must be excluded from the Flex-algo 593 plane. 595 7.1.1. ISIS Advertisement of Flex-algo Binding CQF 597 The Flexible Algorithm definition can specify the binding cycle_size 598 that are used to determine the deterministic delay metric for the 599 computed path within the Flex-algo plane. 601 A new IS-IS sub-TLV is defined: the FAD Binding CQF Sub-Sub-TLV, 602 which is advertised within IS-IS Flexible Algorithm Definition Sub- 603 TLV. At most only one FAD Binding CQF Sub-Sub-TLV can be included. 605 The following format is defined for the FAD Binding CQF Sub-Sub-TLV: 607 0 1 2 3 608 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 609 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 610 | Type | Length | Binding Cycle_size | 611 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 613 Figure 5 615 where: 617 Type: TBD 619 Length: 2 620 Binding Cycle_size: Cycle_size of CQF scheduling bound by Flex- 621 algo, in units of microseconds. 623 The binding cycle_size contained in the FAD with the highest priority 624 will take effect. If the FAD with the highest priority does not 625 contain the FAD Binding CQF Sub-Sub-TLV, the traditional path 626 considering only link transmission delay will be calculated (i.e., 627 degenerating into the calculation result similar as based on Min 628 Unidirectional Link Delay metric type), otherwise, the path will 629 consider both node delay and link delay. 631 7.1.2. OSPF Advertisement of Flex-algo Binding CQF 633 To be defined in next version. 635 7.2. Bind Deadline Scheduling parameters with Flex-Algo 637 The binding relationship can be configured on one or more nodes participating in the 639 same IGP Flex-algo plane, and then advertised in the IGP domain. If 640 there are multiple binding relationship advertised for the same 641 algorithm, it should choose to use the binding scheduling delay and 642 scheduling policy contained in the FAD with the highest priority. 644 If a Flex-algo plane eventually uses a binding deadline parameter, 645 all links participated to the Flex-algo plane must be configured with 646 deadline scheduling enabled and corresponding scheduling delay and 647 scheduling policy, otherwise, links that do not meet the conditions 648 must be excluded from the Flex-algo plane. 650 7.2.1. ISIS Advertisement of Flex-algo Binding Deadline 652 The Flexible Algorithm definition can specify the binding deadline 653 scheduling delay and scheduling policy that are used to determine the 654 deterministic delay metric for the computed path within the Flex-algo 655 plane. 657 A new IS-IS sub-TLV is defined: the FAD Binding Deadline Sub-Sub-TLV, 658 which is advertised within IS-IS Flexible Algorithm Definition Sub- 659 TLV. At most only one FAD Binding Deadline Sub-Sub-TLV can be 660 included. 662 The following format is defined for the FAD Binding Deadline Sub-Sub- 663 TLV: 665 0 1 2 3 666 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 667 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 668 | Type | Length | Flags |U| P | Q // 669 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 670 // Q | 671 +-+-+-+-+-+-+-+-+ 673 Figure 6 675 where: 677 Type: TBD 679 Length: 3 681 Flags: Two flags are currently defined. 683 U-flag: 1 bit, indicating whether the value of scheduling delay 684 Q is known or unknown. 0 indicates known and 1 indicates 685 unknown. 687 P-flag: 2 bits, indicating scheduling policy. The value can 688 be: 0, not defined yet; 1, indicates the in-time policy; 2, 689 indicates the on-time policy; 3, not defined yet. 691 Q: Indicates the deadline scheduling delay Q bound by flex 692 algorithm, in units of microseconds. Note that if the U-flag is 693 1, the value of Q must be ignored and set to 0. 695 The binding deadline parameter contained in the FAD with the highest 696 priority will take effect. If the FAD with the highest priority does 697 not contain the FAD Binding Deadline Sub-Sub-TLV, the traditional 698 path considering only link transmission delay will be calculated 699 (i.e., degenerating into the calculation result similar as based on 700 Min Unidirectional Link Delay metric type), otherwise, the path will 701 consider both node delay and link delay. 703 Note that the FAD Binding CQF Sub-Sub-TLV and the FAD Binding 704 Deadline Sub-Sub-TLV MUST not appear in FAD at the same time, 705 otherwise, the first one is selected. 707 7.2.2. OSPF Advertisement of Flex-algo Binding Deadline 709 To be defined in next version. 711 7.3. CQF based Deterministic Routes Computation 713 This document use the new Metric-Type, Deterministic Delay, combined 714 with the FAD Binding CQF Sub-Sub-TLV, to compute CQF based shortest 715 path with minimum deterministic end-to-end delay, which contains 716 accumulated node delay provided by CQF and accumulated link 717 transmission delay. 719 For a Flex-algo plane that bound to a specific CQF cycle_size, the 720 delay metric of a candidate path within the Flex-algo plane equals: 722 H * node delay, where H is the number of hops, and node delay can 723 be deduced by the cycle_size and forwarding delay intra node as 724 described in Section 4.1; plus 726 Accumulated link transmission delay; 728 From the source node to the destination node, the candidate path with 729 minimum deterministic delay metric is the best one. This calculation 730 result may be different from the traditional calculation result 731 considering only link transmission delay, depending on the proportion 732 of node delay. If the number of intermediate nodes included in the 733 two candidate paths is different, the node delay may be different. 734 For example, a traditional optimal low latency path only considering 735 the link transmission delay may contain more hops, resulting in not 736 being recognized as the optimal deterministic latency path. 738 The deterministic delay jitter of a candidate path within the Flex- 739 algo plane equals: 741 node delay jitter, which is 2*cycle_size; plus 743 Accumulated link delay jitter, which is almost 0; 745 7.4. Deadline based Deterministic Routes Computation 747 This document use the new Metric-Type, Deterministic Delay, combined 748 with the FAD Binding Deadline Sub-Sub-TLV, to compute deadline based 749 shortest path with minimum deterministic end-to-end delay, which 750 contains accumulated node delay provided by deadline and accumulated 751 link transmission delay. 753 For a Flex-algo plane that bound to a specific deadline scheduling 754 parameter, the delay metric of a candidate path within the Flex-algo 755 plane equals: 757 H * node delay, where H is the number of hops, and node delay can 758 be deduced by the scheduling delay, scheduling policy and 759 forwarding delay intra node as described in Section 4.2; plus 761 Accumulated link transmission delay; 763 Assuming that the bound scheduling delay Q and scheduling policy P 764 are obtained from the FAD Binding Deadline Sub-Sub-TLV (note that if 765 the bound scheduling delay Q is an unknown value, the scheduling 766 delay Q is temporarily replaced by 0 during path compuation), the 767 node delay contributed by any intermediate node i in the candidate 768 path is: 770 o For in-time policy, the node delay is in the range of [F(i), 771 F(i)+Q], where F(i) represents the forwarding delay intra node i. 772 Because the node delay value in this case is a range, and we need 773 to get a specific value for SPF computation, thus there are 774 several options to select a specific value as node delay, i.e., 775 select F(i), or F(i)+Q, or the average of F(i) and F(i)+Q. This 776 document take F(i)+Q as the default option. 778 o For on-time policy, the node delay is equal to F(i)+Q. 780 It should be noted that the above calculation process is used to 781 select the optimal deterministic delay path from multiple candidate 782 paths. However, once the deterministic delay SPF path is obtained, 783 the deterministic delay metric of the deterministic delay SPF path 784 should reflect the actual delay. Especially: 786 o When the bound scheduling delay Q is an unknown value, the 787 deterministic delay metric of the deterministic delay SPF path is 788 an formula containing variable quantity Q. In this case, the 789 value of scheduling delay Q needs to be given through other 790 methods, such as carried in the forwarded data packet. This means 791 that the same path can provide different delays for different 792 services. 794 o For in-time policy, the min delay of the SPF path is H*F, which is 795 different with the max delay of SPF path is H*(F+Q), so that delay 796 jitter is H*Q. 798 The deterministic delay jitter of a candidate path within the Flex- 799 algo plane equals: 801 o Accumulated node delay jitter, which is H*Q for in-time policy and 802 0 for on-time policy; plus 804 o Accumulated link delay jitter, which is almost 0; 806 8. Routing Convergence and Redundance Considerations 808 As described in [I-D.ietf-lsr-flex-algo], Loop Free Alternate (LFA) 809 paths for a given Flex-Algorithm MUST be computed using the same 810 constraints as the calculation of the primary paths for that Flex- 811 Algorithm. Within the Segment Routing framework, 812 [I-D.ietf-rtgwg-segment-routing-ti-lfa] can provide TI-LFA path, as 813 the expected post-convergence paths from the point of local repair, 814 in any two connected network using a link-state IGP. However, 815 ordinary IGP convergence and FRR protection may not meet the needs of 816 deterministic services. The main reasons include: 818 o IGP convergence may cause considerable packet loss rate, even if 819 FRR switching is implemented on the basis of rapid fault 820 detection. 822 o The cumulative deterministic delay of the LFA path may be very 823 different from that of the primary path, which does not meet the 824 strict requirements for delay jitter. 826 Thus, according to Service Protection function defined in [RFC8655], 827 packets can be spreaded over multiple disjoint forwarding paths to 828 mitigate or eliminate the packet loss rate. In the context of Flex- 829 algo, an additional redundant deterministic delay path different from 830 FRR path need to be created, when if PLR enable Packet Replication 831 Function (PRF) and the destination enable Packet Elimination Function 832 (PEF). In this case, the data packets are sent along the primary 833 deterministic delay SPF path and the redundant deterministic delay 834 path at the same time, with almost the same cumulative delay. 836 The additional redundant deterministic delay path within the Flex- 837 algo plane is often a traffic engineering path that is calculated by 838 PLR based on the constraints contained in FAD and the following 839 constraints: 841 o The number of nodes intersecting the primary and redundant 842 deterministic delay paths shall be minimized; 844 o The difference between the number of hops of the primary and 845 redundant deterministic delay paths shall be minimized; 847 o The difference between the cumulative link transmission delay of 848 the primary and redundant deterministic delay paths shall be 849 minimized. 851 Unlike LFA FRR path, more scheduling parameters read from link-state 852 database can be attempted to used in the redundant deterministic 853 delay path, to obtain the delay equal or close to the primary path. 855 Take deadline based path as an example, suppose that the number of 856 hops in the primary deterministic delay SPF path is m, and the 857 intermediate nodes passing through are A1, A2, ..., Am, the 858 forwarding delay intra node for each hop is Fa, the scheduling delay 859 intra node for each hop is Qa, and the cumulative link transmission 860 delay is La, then the cumulative deterministic delay of the primary 861 deterministic delay SPF path is the following formula: 863 o Delay(primary) = m*Fa + m*Qa + La 865 Similarly, suppose that the number of hops in the redundant 866 deterministic delay path is n, and the intermediate nodes passing 867 through are B1, B2, ..., Bn, the forwarding delay intra node for each 868 hop is Fb, the scheduling delay intra node for each hop is Qb, and 869 the cumulative link transmission delay is Lb, then the cumulative 870 deterministic delay of the redundant deterministic delay path is the 871 following formula: 873 o Delay(redundant) = n*Fb + n*Qb + Lb 875 The value of Delay(primary) can be calculated based on the known 876 value of Qa that is bound to the flex-algo. Then, an appropriate Qb 877 is slelected to make Delay(redundant) equal to Delay(primary). Qb, 878 that is likely to be different from the bound value Qa, SHOULD be 879 carried in the packets sent along the redundant deterministic delay 880 path to get the expected latency. 882 If the value of Qa bound to the Flex-algo is unknown, states per 883 service should be maintained at the ingress node, to determine the 884 specific value of Qa according to SLA of the service sent along the 885 primary deterministic delay SPF path. On this basis, Qb is then 886 calculated. In this case, both Qa and Qb SHOULD be carried in the 887 packets to get the expected latency. 889 If the further packet replication function continues to be 890 implemented on an intermediate node of the network, the intermediate 891 node only needs to regard itself as the head node of the new 892 protection sub-domain, and can still adopt the above scheme. The 893 intermediate node can also get the value of Delay(primary) based on 894 the bound known Qa (or get from packets), On this basis, Qb is then 895 calculated. 897 It should be noted that both packets sent along primary deterministic 898 delay SPF path and redundant deterministic delay path within a flex- 899 algo plane MUST use SIDs or prefix related with that algortihm. 901 The FIB entries within the flex-algo plane, such as SID entries, 902 contain specific deterministic scheduling parameters to enable the 903 packet to excecute corresponding scheduling function. However, if 904 the packet also carries scheduling parameters, the one in the packet 905 must be preferred. 907 9. Examples of Deterministic delay SPF 909 As shown in Figure 7, the IGP flex-algo 128 plane contains five 910 nodes, of which each link is a bidirectional link. The figure shows 911 the transmission delay parameters of each lin, e.g, the transmission 912 delay of the link between node R1 and node R2 is 10us. 914 20us 30us 915 +----- R3 -----+ 916 | | 917 10us | | 918 R1 ----- R2 R5 919 | | 920 | 10us 20us | 921 +----- R4 -----+ 923 Figure 7 925 9.1. CQF Based Deterministic Delay SPF Path Example 927 It is assumed that the links of all nodes in the network are 928 configured with consistent CQF scheduling parameters and have 929 consistent node delay and delay jitter attributes, as follows: 931 Forwarding delay intra node = 0us 933 CQF enable/disable = ON 935 Supported cycle_size set = <10 us, 20 us> 937 Configure FAD of IGP flex-algo 128, set metric-type to Deterministic 938 Delay, and set bound CQF scheduling parameters (cycle_size = 10us). 939 Suppose that FAD is optimal after negotiation. 941 Taking node R1 as an example, it takes itself as the root to 942 calculate the deterministic delay SPT as shown in Figure 8. In the 943 figure, the sum of the node delay and the link transmssion delay is 944 marked on each link. For example, the delay of link from node R2 to 945 R3 is 10+20, where 10 is node delay and 20 is link transmission 946 delay. 948 R1 949 10+10 / 950 R2 951 10+20 / \ 10+10 952 R3 R4 953 \ 10+20 954 R5 956 Figure 8 958 Therefore, with R1 as the source node and R5 as the destination node, 959 the cumulative deterministic delay of CQF based SPF path 960 (R1-R2-R4-R5) is 70us, The cumulative deterministic delay jitter is 961 20us. 963 Assuming that node R5 advertised SID-R5 that belongs to the flex-algo 964 128 plane, the following deterministic SPF FIB entry will be created 965 on node R1. 967 KEY: SID-R5 969 Forwarding information: 971 next_hop = R2 973 interface = link(R1-R2) 975 metric_type = Deterministic Delay 977 scheduling algorithm = CQF with cycle_size 10 us 979 total_metric = 70 us 981 total_metric_variation = 20 us 983 9.2. Deadline Based Deterministic Delay SPF Path Example 985 It is assumed that the links of all nodes in the network are 986 configured with consistent deadline scheduling parameters, as 987 follows: 989 Forwarding delay intra node = 5us 991 Deadline enable/disable = ON 992 Supported scheduling delay set = <10us, 20us, 30us, 40us, 50us, 993 60us>, each item in the set support both in-time and on-time 994 policy 996 Configure FAD of IGP flex-algo 128, set metric-type to Deterministic 997 Delay, and set bound Deadline scheduling parameters (Q = 10us, with 998 in-time policy). Suppose that FAD is optimal after negotiation. 1000 Taking R1 node as an example, it takes itself as the root to 1001 calculate the deterministic delay SPT as shown in Figure 9. In the 1002 figure, the sum of the node delay and the link transmssion delay is 1003 marked on each link. Note that this document suggest to take F+Q as 1004 node delay during calculation even for in-time policy. For example, 1005 the delay of link from node R2 to R3 is 15+20, where 15 is node delay 1006 and 20 is link transmission delay. 1008 R1 1009 15+10 / 1010 R2 1011 15+20 / \ 15+10 1012 R3 R4 1013 \ 15+20 1014 R5 1016 Figure 9 1018 Therefore, with R1 as the source node and R5 as the destination node, 1019 the cumulative deterministic delay of Deadline based SPF path 1020 (R1-R2-R4-R5) is 85us, and the cumulative deterministic delay jitter 1021 is 30us. 1023 Assuming that node R5 advertised SID-R5 that belongs to the fle-algo 1024 128 plane, the following deterministic SPF FIB entry will be created 1025 on node R1. 1027 KEY: SID-R5 1029 Forwarding information: 1031 next_hop = R2 1033 interface = link(R1-R2) 1035 metric_type = Deterministic Delay 1037 scheduling algorithm = Deadline with Q=10 us with in-time 1038 policy 1039 total_metric = 85 us 1041 total_metric_variation = 30 us 1043 Similarly, if on-time policy is bound to the flex-algo, the 1044 cumulative deterministic delay of Deadline based SPF path 1045 (R1-R2-R4-R5) is 85us, but the cumulative deterministic delay jitter 1046 is 0. The deterministic SPF FIB entry created on node R1 is changed 1047 to: 1049 KEY: SID-R5 1051 Forwarding information: 1053 next_hop = R2 1055 interface = link(R1-R2) 1057 metric_type = Deterministic Delay 1059 scheduling algorithm = Deadline with Q=10 us with on-time 1060 policy 1062 total_metric = 85 us 1064 total_metric_variation = 0 1066 10. Use Cases 1068 [RFC8578] described various deterministic routing use cases from 1069 multiple industries, including: Pro Audio and Video, Electrical 1070 Utilities, Building Automation Systems, Wireless for Industrial 1071 Applications, Cellular Radio, Industrial Machine to Machine (M2M), 1072 Mining Industry, Private Blockchain, Network Slicing, etc. Among 1073 them, some industries are now transitioning to packet based 1074 infrastructure, and some industries have already linked their 1075 different subsystems through networks (intra-domain or inter-domain). 1076 These industries have put forward the requirements of delay and delay 1077 jitter with different indicators, such as BAS requires low delay 1078 (10ms ~ 100ms) and low jitter (1ms); M2M requires that the underlying 1079 network infrastructure must ensure that the maximum end-to-end 1080 message delivery time is between 100 us and 50 ms; Mining industry 1081 requires predictable time delay to realize real-time monitoring. The 1082 deterministic paths can be centralized centralized computing, or 1083 distributed computing when there is a lack of controller. 1085 The mechanism introduced in this document can get a SPF path with 1086 determinstic delay metric, but more importantly, with deterministic 1087 dealy jitter. The determinsitic delay metric of the path actually 1088 depends on the network scale. It can be large or small, but it can 1089 be guaranteed to be the smallest of all candidate paths. The 1090 determinsitic delay jitter is also bounded and may be a cumulative 1091 value related to the number of hops or a value independent of the 1092 number of hops. SPF Paths with such characteristics will benefit 1093 multiple applications as mentioned above. 1095 11. IANA Considerations 1097 11.1. ISIS Deterministic Delay Metric Sub-TLV 1099 This document registers the following Sub-TLV in the "Sub-TLVs for 1100 IS-IS Sub-TLVs for TLVs Advertising Neighbor Information" registry: 1102 +------+--------------------+----+----+----+-----+-----+-----+ 1103 | Type | Description | 22 | 23 | 25 | 141 | 222 | 223 | 1104 +======+====================+====+====+====+=====+=====+=====+ 1105 | | Deterministic Delay| | | | | | | 1106 | TBA1 | Metric | y | y | y | y | y | y | 1107 +------+--------------------+----+----+----+-----+-----+-----+ 1109 11.2. Sub-Sub-TLVs for ISIS Deterministic Delay Metric Sub-TLV 1111 This document registers the following Sub-TLV in the "Sub-TLVs for 1112 IS-IS Sub-TLVs for TLVs Advertising Neighbor Information" registry: 1114 +------+----------------------+---------------------------+ 1115 | Type | Description | Reference | 1116 +======+======================+===========================+ 1117 | | CQF Scheduding Delay | This document Section 4.1 | 1118 | TBA2 | Intra Node | | 1119 +------+----------------------+---------------------------+ 1120 | | Deadline Scheduding | This document Section 4.2 | 1121 | TBA3 | Delay Intra Node | | 1122 +------+----------------------+---------------------------+ 1124 11.3. IGP Metric-Type Registry 1126 This document registers the following values in the "IGP Metric-Type 1127 Registry" for FAD: 1129 Type: TBA4 (suggested 4) 1131 Description: Deterministic Delay Metric as defined in this 1132 document 1134 Reference: This document (Section 4) 1136 11.4. ISIS Sub-Sub-TLVs for Flexible Algorithm Definition Sub-TLV 1138 This document defines the following Sub-Sub-TLVs in the "Sub-Sub-TLVs 1139 for Flexible Algorithm Definition Sub-TLV" registry: 1141 +------+----------------------+-----------------------------+ 1142 | Type | Description | Reference | 1143 +======+======================+=============================+ 1144 | TBA5 | FAD Binding CQF | This document Section 7.1.1 | 1145 +------+----------------------+-----------------------------+ 1146 | TBA6 | FAD Binding Deadline | This document Section 7.2.1 | 1147 +------+----------------------+-----------------------------+ 1149 11.5. OSPF IANA considerations 1151 TBD. 1153 12. Security Considerations 1155 TBD. 1157 13. Acknowledgements 1159 The authors would like to acknowledge the review and inputs from 1160 Peter Psenak, Bin Tan, Quan Xiong. 1162 14. References 1164 14.1. Normative References 1166 [I-D.ietf-lsr-flex-algo] 1167 Psenak, P., Hegde, S., Filsfils, C., Talaulikar, K., and 1168 A. Gulko, "IGP Flexible Algorithm", draft-ietf-lsr-flex- 1169 algo-18 (work in progress), October 2021. 1171 [I-D.ietf-lsr-ip-flexalgo] 1172 Britto, W., Hegde, S., Kaneriya, P., Shetty, R., Bonica, 1173 R., and P. Psenak, "IGP Flexible Algorithms (Flex- 1174 Algorithm) In IP Networks", draft-ietf-lsr-ip-flexalgo-04 1175 (work in progress), December 2021. 1177 [I-D.ietf-rtgwg-segment-routing-ti-lfa] 1178 Litkowski, S., Bashandy, A., Filsfils, C., Francois, P., 1179 Decraene, B., and D. Voyer, "Topology Independent Fast 1180 Reroute using Segment Routing", draft-ietf-rtgwg-segment- 1181 routing-ti-lfa-08 (work in progress), January 2022. 1183 [I-D.peng-detnet-deadline-based-forwarding] 1184 Peng, S. and B. Tan, "Deadline Based Deterministic 1185 Forwarding", draft-peng-detnet-deadline-based- 1186 forwarding-00 (work in progress), January 2022. 1188 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1189 Requirement Levels", BCP 14, RFC 2119, 1190 DOI 10.17487/RFC2119, March 1997, 1191 . 1193 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., 1194 and W. Weiss, "An Architecture for Differentiated 1195 Services", RFC 2475, DOI 10.17487/RFC2475, December 1998, 1196 . 1198 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1199 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1200 May 2017, . 1202 [RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L., 1203 Decraene, B., Litkowski, S., and R. Shakir, "Segment 1204 Routing Architecture", RFC 8402, DOI 10.17487/RFC8402, 1205 July 2018, . 1207 [RFC8570] Ginsberg, L., Ed., Previdi, S., Ed., Giacalone, S., Ward, 1208 D., Drake, J., and Q. Wu, "IS-IS Traffic Engineering (TE) 1209 Metric Extensions", RFC 8570, DOI 10.17487/RFC8570, March 1210 2019, . 1212 [RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases", 1213 RFC 8578, DOI 10.17487/RFC8578, May 2019, 1214 . 1216 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 1217 "Deterministic Networking Architecture", RFC 8655, 1218 DOI 10.17487/RFC8655, October 2019, 1219 . 1221 14.2. Informative References 1223 [CBS] "IEEE802.1Qav", 2009, 1224 . 1226 [CQF] "IEEE802.1Qch", 2017, 1227 . 1229 [SP-LATENCY] 1230 "Guaranteed Latency with SP", 2020, 1231 . 1234 Authors' Addresses 1236 Shaofu Peng 1237 ZTE Corporation 1238 China 1240 Email: peng.shaofu@zte.com.cn 1242 Tony Li 1243 Juniper Networks 1244 United States of America 1246 Email: tony.li@tony.li