idnits 2.17.1 draft-varga-detnet-pof-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC8655]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (May 21, 2021) is 1063 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet B. Varga, Ed. 3 Internet-Draft J. Farkas 4 Intended status: Informational Ericsson 5 Expires: November 22, 2021 S. Kehrer 6 T. Heer 7 Hirschmann Automation and Control GmbH 8 May 21, 2021 10 Deterministic Networking (DetNet): Packet Ordering Function 11 draft-varga-detnet-pof-01 13 Abstract 15 Replication and Elimination functions of DetNet [RFC8655] may result 16 in out-of-order packets, which may not be acceptable for some time- 17 sensitive applications. The Packet Ordering Function (POF) algorithm 18 described herein enables to restore the correct packet order when 19 replication and elimination functions are used in DetNet networks. 21 Status of This Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at https://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on November 22, 2021. 38 Copyright Notice 40 Copyright (c) 2021 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (https://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 56 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 57 2.1. Terms Used in This Document . . . . . . . . . . . . . . . 3 58 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 3 59 2.3. Requirements Language . . . . . . . . . . . . . . . . . . 4 60 3. Requirements on POF Implementations . . . . . . . . . . . . . 4 61 4. POF Algorithms . . . . . . . . . . . . . . . . . . . . . . . 4 62 4.1. Prerequisites and Assumptions . . . . . . . . . . . . . . 4 63 4.2. POF building blocks . . . . . . . . . . . . . . . . . . . 5 64 4.3. The Basic POF Algorithm . . . . . . . . . . . . . . . . . 6 65 4.4. The Advanced POF Algorithm . . . . . . . . . . . . . . . 7 66 4.5. Further enhancements of POF algorithms . . . . . . . . . 8 67 4.6. Selecting and using the POF algorithm . . . . . . . . . . 9 68 5. Control and Management Plane Parameters for POF . . . . . . . 9 69 6. Security Considerations . . . . . . . . . . . . . . . . . . . 10 70 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 71 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 72 8.1. Normative References . . . . . . . . . . . . . . . . . . 10 73 8.2. Informative References . . . . . . . . . . . . . . . . . 10 74 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11 76 1. Introduction 78 The DetNet Working Group has defined packet replication (PRF) and 79 packet elimination (PEF) functions for achieving extremely low packet 80 loss. PRF and PEF are described in [RFC8655] and provide service 81 protection for DetNet flows. This service protection method relies 82 on copies of the same packet sent over multiple maximally disjoint 83 paths and uses sequencing information to eliminate duplicates. A 84 possible implementation of PRF and PEF functions is described in 85 [IEEE8021CB] and the related YANG model is defined in 86 [IEEEP8021CBcv]. 88 In general, use of per packet replication and elimination functions 89 may result in out-of-order delivery of packets, which may not be 90 acceptable for some deterministic applications. Correcting packet 91 order is not a trivial task, therefore details of a Packet Ordering 92 Function (POF) are specified herein. The IETF DetNet WG has defined 93 in [RFC8655] the external observable result of a POF function, i.e., 94 that packets are reordered, but without any implementation details. 96 So far in packet networks, out-of-order delivery situations were 97 handled at higher OSI layers at the end-points/hosts (e.g., in the 98 TCP stack when ackets are sent to application layer) and not within a 99 network in nodes acting at the Layer-2 or Layer-3 OSI layers. 101 Figure 1 shows a DetNet flow on which PREOF functions are applied 102 during forwarding from source to destination. 104 +------------+ 105 +---------------E----+ | | 106 +----+ | | +----R---+ | +----+ 107 |src |-------R +---+ | E-----O----+ dst| 108 +----+ | | E--------+ +----+ 109 +-----------R | 110 +-----------------+ 112 R: replication point (PRF) 113 E: elimination point (PEF) 114 O: ordering function (POF) 116 Figure 1: PREOF scenario in a DetNet network 118 Important to note, that application may react differently on out-of- 119 order delivery. A single out-of-order packet (E.g., packet order: 120 #1, #3, #2, #4, #5) may be interpreted by some applications as a 121 single error, but some other applications may treat it as a 3 errors 122 in-a-row situation. 3 errors in-a-row is a usual error threshold and 123 may cause the application to stop (e.g., to tranistion to a fail safe 124 state). 126 2. Terminology 128 2.1. Terms Used in This Document 130 This document uses the terminology established in the DetNet 131 architecture [RFC8655], and the reader is assumed to be familiar with 132 that document and its terminology. 134 2.2. Abbreviations 136 The following abbreviations are used in this document: 138 DetNet Deterministic Networking. 140 PEF Packet Elimination Function. 142 POF Packet Ordering Function. 144 PREOF Packet Replication, Elimination and Ordering Functions. 146 PRF Packet Replication Function. 148 2.3. Requirements Language 150 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 151 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 152 "OPTIONAL" in this document are to be interpreted as described in BCP 153 14 [RFC2119] [RFC8174] when, and only when, they appear in all 154 capitals, as shown here. 156 3. Requirements on POF Implementations 158 The requirements on a POF function are: 160 o to solve the out-of-order delivery problem of the Replication and 161 Elimination functions of DetNet networks. 163 o to consider the delay bound requirement of a DetNet Flow. 165 o to be simple and to require in network nodes only a minimum set of 166 states/configuration parameters and resources per DetNet Flow. 168 o to add only minimal or no delay to the forwarding process of 169 packets. 171 o not to require synchronization between PREOF nodes. 173 4. POF Algorithms 175 4.1. Prerequisites and Assumptions 177 The POF Algorithm discussed in this document makes some assumptions 178 and tradeoffs regarding the characteristics of the sequence of 179 received packets. In particular, the algorithm assumes that a Packet 180 Elimination Function (PEF) is performed on the incoming packets 181 before they are handed to the POF function. Hence, the sequence of 182 incoming packets can be out of order or incomplete but cannot contain 183 duplicate packets. However, the PREOF functions run independently 184 without any state exchange required between the PEF and the POF or 185 the PRF and the POF. Error cases in which the POF is presented 186 duplicate packets may lead to out of order delivery of duplicate 187 packets as well as to increased delays. 189 The algorithm further requires that the delay difference between two 190 replicated packets that arrive at the PRF before the POF is bounded 191 and known. Error cases that violate this condition (e.g., a packet 192 that arrives later than this bound) will result in out-of order 193 packets. 195 The algorithm also makes some tradeoffs. For simplicity, it is 196 designed in a way that allows for some out of order packets directly 197 after initialization. If this is not acceptable, Section 4.5 198 provides an alternative initialization scheme that prevents out-of- 199 order packets in the initialization phase. 201 4.2. POF building blocks 203 The method described herein provides POF for DetNet networks. The 204 configuration parameters of POF can be derived during engineering the 205 DetNet flow through the network. 207 The POF method is provided via: 209 1. Conditional buffer: for buffering the out-of-order packets of a 210 DetNet flow for a given time. 212 2. Delay calculator: buffering time considers (i) the delay 213 difference of paths used for forwarding the replicated packets 214 and (ii) the bounded delay requirement of the given DetNet flow. 216 Figure 2 shows the building blocks of a possible POF implementation. 218 +------------+ +--------------+ 219 | Delay calc | | Conditional | 220 +--| for packet >--->>---| Delay Buffer >---+ 221 | +------------+ +--------------+ | 222 | | 223 +------^--------+ | 224 ->>--| POF selector >----------------------------------+-->>---- 225 | (Flow ident.) | 226 +---------------+ 228 ->>- packet flow 230 Figure 2: POF Building Blocks 232 4.3. The Basic POF Algorithm 234 The basic POF algorithm delays all out-of-order packets until all 235 previous packet arrives or a given time (POFMaxDelay) elapses. The 236 basic POF algorithm works as follows: 238 o The sequence number of the last forwarded packet (POFLastSent) is 239 stored for each DetNet Flow. 241 o The sequence number (seq_num) of a received packet is compared to 242 that of the last forwarded one (POFLastSent). 244 o If (seq_num <= POFLastSent + 1) 246 * Then the packet is forwarded and "POFLastSent" is updated 247 (POFLastSent = seq_num). 249 * Else the received packet is buffered. 251 o A buffered packet is forwarded from the buffer when its seq_num 252 becomes equal to "POFLastSent +1," OR a predefined time 253 ("POFMaxDelay") elapses. 255 o When a packet is forwarded from the buffer "POFLastSent" is 256 updated with its seq_num (POFLastSent = seq_num). 258 Note: the difference of sequence number in consecutive packets is 259 bounded due to the history window of the Elimination function before 260 the POF. Therefore "<=" can be evaluated despite of the circular 261 sequence number space. 263 The state used by the basic POF algorithm (i.e., "POFLastSent") needs 264 initialization and maintenance. This works as follows: 266 o The next received packet must be forwarded and the POFLastSent 267 updated when the POF function was reset OR no packet was received 268 for a predefined time ("POFTakeAnyTime"). 270 o The reset of POF erases all frames/packets from the time-based 271 buffer used by POF. 273 The basic POF algorithm has two parameters to engineer: 275 o "POFMaxDelay", which cannot be smaller than the delay difference 276 of the paths used by the flow. 278 o "POFTakeAnyTime", which is calculated based on several factors, 279 for example the RECOVERY_TIMEOUT related settings of the 280 Elimination function(s) before the POF, the flow characteristics 281 (e.g., inter frame/packet time), and the delay difference of the 282 paths used by the flow. 284 Design of these parameters is out-of-scope in this document. 286 Note: multiple network failures may impact the POF function (e.g., 287 complete outage of all redundant paths). 289 The basic POF algorithm increases the delay of packets with maximum 290 "POFMaxDelay" time. Packets being in order are not delayed. This 291 basic POF method can be applied in all network scenarios where the 292 remaining delay budget of a flow at the POF point is larger than 293 "POFMaxDelay" time. 295 Figure 3 shows the delay budget relations at the POF point. 297 Path delay 298 difference 299 /-------------/ 300 <--- fast path delay ---> /--- remaining delay budget ---/ 302 |-----------------------|-------------|------------------------------| 303 0 t1 t2 T 305 <----------- slow path delay ---------> 307 /-------------------- delay budget at POF point ---------------------/ 309 Figure 3: Delay Budget Relations at the POF Point 311 4.4. The Advanced POF Algorithm 313 In network scenario where the remaining delay budget of a flow at the 314 POF point is smaller than "POFMaxDelay" time the basic method needs 315 extensions. 317 The issue is that packets on the longest path cannot be buffered in 318 order to keep delay budget of the flow. It must be noted that such a 319 packet (i.e., forwarded over the longest path) needs no buffering as 320 it is the "last chance" to deliver a packet with a given sequence 321 number. This is because all replicas already must be arrived via 322 shorter path(s). 324 The advanced POF algorithm needs two extensions of the basic POF 325 algorithm: 327 o to identify the received packet's path at the POF location and 329 o to make the value of "POFMaxDelay" for buffered packets path 330 dependent ("POFMaxDelay_i", where "i" notes the path the packet 331 has used). 333 By identifying the path of a given frame, the POF algorithm can use 334 this information to select what predefined time "POFMaxDelay_i" to 335 apply for the buffered frame/packet. So, in the advanced POF 336 algorithm "POFMaxDelay" is an array, that contains the predefined and 337 path specific buffering time for each redundant path of a flow. 338 Values in the "POFMaxDelay" array are engineered to fulfill the delay 339 budget requirement. 341 The method for identification of the packet's path at the POF 342 location depends on the network scenario. It can be implemented via 343 various techniques, for example using ingress interface information, 344 encoding the path in the packet itself (e.g., replication functions 345 can set different FlowID per egress what can be used as a PathID), or 346 in other means. Method for identification of the packet's path is 347 out of scope in this document. 349 Note: in case of using the advanced POF algorithm it might be 350 advantageous to combine PEF and POF locations in the DetNet network, 351 as it can simplify the method used for identification of the packet's 352 path at the POF location. 354 4.5. Further enhancements of POF algorithms 356 POF algorithms can be further enhanced by distinguishing the case of 357 initialization from normal operation at the price of more states and 358 more sophisticated implementation. Such enhancements could for 359 example react better after some failure scenarios (e.g., complete 360 outage of all paths of a DetNet flow) and may be dependent on the PEF 361 implementation. 363 The challenge for POF initialization is that for example after a 364 reset it is not known whether the first received packet is in-order 365 or out-of-order. The original initialization (see before) considers 366 the first packet as in-order, so out-of-order packet(s) during 367 "POFMaxTime"/"POFMaxTime_path_i" time - after the first packet was 368 received - may not be corrected. Motivation behind such an 369 initialization is POF implementation simplicity. 371 A possible enhancement of POF initialization works as follows: 373 o After a reset all received packets are buffered with their 374 predefined timer ("POFMaxTime"/"POFMaxTime_path_i"). 376 o No basic/advanced POF rules are applied until the first timer 377 expires. 379 o When the first timer expires the packet with lowest seq_num in 380 buffer is selected, forwarded, and "POFLastSent" is set with its 381 seq_num. 383 o The basic/advanced POF rules are applied for the packet(s) in the 384 buffer and the subsequently received packets. 386 4.6. Selecting and using the POF algorithm 388 The selection of the POF algorithm depends on the network scenario 389 and the remaining delay budget of a flow. Using POF and calculating 390 its parameters require proper design. Knowing the path delay 391 difference is essential for the POF algorithms described here. 392 Failure scenarios breaking the design assumptions may impact the 393 result of POF (e.g., packet received out of the expected worst-case 394 delay window - calculated based on the path delay difference - may 395 result in unwanted out-of-order delivery). 397 In DetNet scenarios there is always an Elimination function before 398 the POF (therefore duplicates are not considered by the POF). 399 Implementing them together in the same node allows POF to consider 400 PEF events/states during the re-ordering. For example, under normal 401 circumstances the difference of sequence number in consecutive 402 packets is bounded due to the history window of PEF. However, in 403 some scenarios (e.g., reset of sequence number) the difference can be 404 much larger than the history window size. 406 5. Control and Management Plane Parameters for POF 408 POF algorithms needs setting of the following parameters: 410 o Basic POF 412 * "POFMaxDelay" 414 * "POFTakeAnyTime" 416 o Advanced POF 418 * "POFMaxDelay_i" 420 * "POFTakeAnyTime" 421 * Network path identification related configuration(s) 423 Note, that in a proper design "POFTakeAnyTime" must be always larger 424 than "POFMaxDelay". 426 6. Security Considerations 428 There are no POF related security considerations for DetNet. 430 7. IANA Considerations 432 This document makes no IANA requests. 434 8. References 436 8.1. Normative References 438 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 439 Requirement Levels", BCP 14, RFC 2119, 440 DOI 10.17487/RFC2119, March 1997, 441 . 443 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 444 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 445 May 2017, . 447 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 448 "Deterministic Networking Architecture", RFC 8655, 449 DOI 10.17487/RFC8655, October 2019, 450 . 452 8.2. Informative References 454 [IEEE8021CB] 455 IEEE, "IEEE Standard for Local and metropolitan area 456 networks -- Frame Replication and Elimination for 457 Reliability", DOI 10.1109/IEEESTD.2017.8091139, October 458 2017, 459 . 461 [IEEEP8021CBcv] 462 Kehrer, S., "FRER YANG Data Model and Management 463 Information Base Module", IEEE P802.1CBcv 464 /D1.2 P802.1CBcv, March 2021, 465 . 468 Authors' Addresses 470 Balazs Varga (editor) 471 Ericsson 472 Magyar Tudosok krt. 11. 473 Budapest 1117 474 Hungary 476 Email: balazs.a.varga@ericsson.com 478 Janos Farkas 479 Ericsson 480 Magyar Tudosok krt. 11. 481 Budapest 1117 482 Hungary 484 Email: janos.farkas@ericsson.com 486 Stephan Kehrer 487 Hirschmann Automation and Control GmbH 488 Stuttgarter Strasse 45-51. 489 Neckartenzlingen 72654 490 Germany 492 Email: Stephan.Kehrer@belden.com 494 Tobias Heer 495 Hirschmann Automation and Control GmbH 496 Stuttgarter Strasse 45-51. 497 Neckartenzlingen 72654 498 Germany 500 Email: Tobias.Heer@belden.com