idnits 2.17.1 draft-pthubert-raw-architecture-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == There is 1 instance of lines with non-ascii characters in the document. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 610: '...ry hop of the forwarding paths MUST be...' RFC 2119 keyword, line 645: '... SHOULD be applied at each RPL DODAG...' Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (2 April 2020) is 1485 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'Function' is mentioned on line 448, but not defined == Outdated reference: A later version (-30) exists of draft-ietf-6tisch-architecture-28 == Outdated reference: A later version (-05) exists of draft-thubert-raw-technologies-04 == Outdated reference: A later version (-04) exists of draft-bernardos-raw-use-cases-03 == Outdated reference: A later version (-03) exists of draft-mirsky-detnet-ip-oam-02 == Outdated reference: A later version (-06) exists of draft-ietf-detnet-data-plane-framework-04 Summary: 1 error (**), 0 flaws (~~), 8 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 RAW P. Thubert, Ed. 3 Internet-Draft Cisco Systems 4 Intended status: Informational G.Z. Papadopoulos 5 Expires: 4 October 2020 IMT Atlantique 6 2 April 2020 8 Reliable and Available Wireless Architecture/Framework 9 draft-pthubert-raw-architecture-01 11 Abstract 13 Due to uncontrolled interferences, including the self-induced 14 multipath fading, deterministic networking can only be approached on 15 wireless links. The radio conditions may change -way- faster than a 16 centralized routing can adapt and reprogram, in particular when the 17 controller is distant and connectivity is slow and limited. RAW 18 separates the routing time scale at which a complex path is 19 recomputed from the forwarding time scale at which the forwarding 20 decision is taken for an individual packet. RAW operates at the 21 forwarding time scale. The RAW problem is to decide, within the 22 redundant solutions that are proposed by the routing, which will be 23 used for each individual packet to provide a DetNet service while 24 minimizing the waste of resources. 26 Status of This Memo 28 This Internet-Draft is submitted in full conformance with the 29 provisions of BCP 78 and BCP 79. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF). Note that other groups may also distribute 33 working documents as Internet-Drafts. The list of current Internet- 34 Drafts is at https://datatracker.ietf.org/drafts/current/. 36 Internet-Drafts are draft documents valid for a maximum of six months 37 and may be updated, replaced, or obsoleted by other documents at any 38 time. It is inappropriate to use Internet-Drafts as reference 39 material or to cite them other than as "work in progress." 41 This Internet-Draft will expire on 4 October 2020. 43 Copyright Notice 45 Copyright (c) 2020 IETF Trust and the persons identified as the 46 document authors. All rights reserved. 48 This document is subject to BCP 78 and the IETF Trust's Legal 49 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 50 license-info) in effect on the date of publication of this document. 51 Please review these documents carefully, as they describe your rights 52 and restrictions with respect to this document. Code Components 53 extracted from this document must include Simplified BSD License text 54 as described in Section 4.e of the Trust Legal Provisions and are 55 provided without warranty as described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 60 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 61 3. Related Work at The IETF . . . . . . . . . . . . . . . . . . 6 62 4. Use Cases and Requirements Served . . . . . . . . . . . . . . 6 63 4.1. Radio Access Protection . . . . . . . . . . . . . . . . . 7 64 4.2. Track Protection in a Wireless Mesh . . . . . . . . . . . 7 65 5. RAW Architecture Considerations . . . . . . . . . . . . . . . 8 66 5.1. Reliability and Availability . . . . . . . . . . . . . . 8 67 5.1.1. Reliability Engineering . . . . . . . . . . . . . . . 8 68 5.1.2. Reliability In Wireless Networks . . . . . . . . . . 8 69 5.2. Prerequisites . . . . . . . . . . . . . . . . . . . . . . 9 70 5.3. Routing Time Scale vs. Forwarding Time Scale . . . . . . 10 71 6. RAW Architecture Components . . . . . . . . . . . . . . . . . 11 72 6.1. Wireless Tracks . . . . . . . . . . . . . . . . . . . . . 11 73 6.2. Source-Routed vs. Distributed Forwarding Decision . . . . 12 74 6.3. PAREO Functions . . . . . . . . . . . . . . . . . . . . . 12 75 6.3.1. Packet Replication . . . . . . . . . . . . . . . . . 13 76 6.3.2. Packet Elimination . . . . . . . . . . . . . . . . . 14 77 6.3.3. Promiscuous Overhearing . . . . . . . . . . . . . . . 14 78 7. Security Considerations . . . . . . . . . . . . . . . . . . . 15 79 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15 80 9. ConTributors . . . . . . . . . . . . . . . . . . . . . . . . 15 81 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 15 82 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 15 83 11.1. Normative References . . . . . . . . . . . . . . . . . . 15 84 11.2. Informative References . . . . . . . . . . . . . . . . . 16 85 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 17 87 1. Introduction 89 Bringing determinism in a packet network means eliminating the 90 statistical effects of multiplexing that result in probabilistic 91 jitter and loss. This can be approached with a tight control of the 92 physical resources to maintain the amount of traffic within a 93 budgetted volume of data per unit of time that fits the physical 94 capabilities of the underlying technology, and the use of time-shared 95 resources (bandwidth and buffers) per circuit, and/or by shaping and/ 96 or scheduling the packets at every hop. 98 Wireless networks operate on a shared medium where uncontrolled 99 interference, including the self-induced multipath fading, adds 100 another dimension to the statistical effects that affect the 101 delivery. Scheduling transmissions can alleviate those effects by 102 leveraging diversity in the spatial, time, code, and frequency 103 domains, and provide a Reliable and Available service while 104 preserving energy and optimizing the use of the shared spectrum. 106 Deterministic Networking is an attempt to mostly eliminate packet 107 loss for a committed bandwidth with a guaranteed worst-case end-to- 108 end latency, even when co-existing with best-effort traffic in a 109 shared network. This innovation is enabled by recent developments in 110 technologies including IEEE 802.1 TSN (for Ethernet LANs) and IETF 111 DetNet (for wired IP networks). It is getting traction in various 112 industries including manufacturing, online gaming, professional A/V, 113 cellular radio and others, making possible many cost and performance 114 optimizations. 116 The "Deterministic Networking Architecture" [RFC8655] is composed of 117 three planes: the Application (User) Plane, the Controller Plane, and 118 the Network Plane. Reliable and Available Wireless (RAW) extends 119 DetNet to focus on issues that are mostly a co"ern on wireless links, 120 and inherits the architecture and the planes. A RAW Network Plane is 121 thus a Network Plane inherited by RAW from DetNet, composed of one or 122 multiple hops of homogeneous or heterogeneous technologies, e.g. a 123 Wi-Fi6 Mesh or one-hop CBRS access links federated by a 5G backhaul. 125 RAW networking aims at providing highly available and reliable end- 126 to-end performances in a network with scheduled wireless segments. 127 Uncontrolled interference and transmission obstacles may impede the 128 transmission, and techniques such as beamforming with Multi-User MIMO 129 can only alleviate some of those issues, so the term "deterministic" 130 is usually not associated with short range radios, in particular in 131 the ISM band. This uncertainty places limits to the amount of 132 traffic that can be transmitted on a link while conforming to a RAW 133 Service Level Agreement (SLA) that may vary rapidly. 135 The wireless and wired media are fundamentally different at the 136 physical level, and while the generic "Deterministic Networking 137 Problem Statement" [RFC8557] applies to both the wired and the 138 wireless media, the methods to achieve RAW must extend those used to 139 support time-sensitive networking over wires, as a RAW solution has 140 to address less consistent transmissions, energy conservation and 141 shared spectrum efficiency. 143 The development of RAW technologies has been lagging behind 144 deterministic efforts for wired systems both at the IEEE and the 145 IETF. But recent efforts at the IEEE and 3GPP indicate that wireless 146 is finally catching up at the lower layer and that it is now possible 147 for the IETF to extend DetNet for wireless segments that are capable 148 of scheduled wireless transmissions. 150 The intent for RAW is to provide DetNet elements that are specialized 151 for short range radios. From this inheritance, RAW stays agnostic to 152 the radio layer underneath though the capability to schedule 153 transmissions is assumed. How the PHY is programmed to do so, and 154 whether the radio is single-hop or meshed, are unknown at the IP 155 layer and not part of the RAW abstraction. 157 Still, in order to focus on real-worlds issues and assert the 158 feasibility of the proposed capabilities, RAW will focus on selected 159 technologies that can be scheduled at the lower layers: IEEE Std. 160 802.15.4 timeslotted channel hopping (TSCH), 3GPP 5G ultra-reliable 161 low latency communications (URLLC), IEEE 802.11ax/be where 802.11be 162 is extreme high throughput (EHT), and L-band Digital Aeronautical 163 Communications System (LDACS). See [RAW-TECHNOS] for more. 165 The establishment of a path is not in-scope for RAW. It may be the 166 product of a centralized Controller Plane as described for DetNet. 167 As opposed to wired networks, the action of installing a path over a 168 set of wireless links may be very slow relative to the speed at which 169 the radio conditions vary, and it makes sense in the wireless case to 170 provide redundant forwarding solutions along a complex path and to 171 leave it to the Network Plane to select which of those forwarding 172 solutions are to be used for a given packet based on the current 173 conditions. 175 RAW distinguishes the longer time scale at which routes are computed 176 from the the shorter forwarding time scale where per-packet decisions 177 are made. RAW operates at the forwarding time scale on one DetNet 178 flow over one path that is preestablished and installed by means 179 outside of the scope of RAW. The scope of the RAW WG comprises 180 Network plane protocol elements such as OAM and in-band control to 181 improve the RAW operation at the Service and at the forwarding sub- 182 layers, e.g., controlling whether to use packet replication, 183 Automatic Repeat reQuest (ARQ), Hybrid ARQ (HARQ) that includes 184 Forward Error Correction (FEC) and coding, with a constraint to limit 185 the use of redundancy 186 whencccccckehblncidtvdigtbfgjiugivbrkkklehrciijk it is really needed, 187 e.g., when a spike of loss is observed. This is discussed in more 188 details in Section 5.3 and the next sections. 190 2. Terminology 192 RAW defines the following terms: 194 PAREO: Packet (hybrid) ARQ, Replication, Elimination and Ordering. 195 PAREO is a superset Of DetNet's PREOF that includes radio-specific 196 techniques such as short range broadcast, MUMIMO, constructive 197 interference and overhearing, which can be leveraged separately or 198 combined to increase the reliability. 200 Flapping: In the context of RAW, a link flaps when the wireless 201 connectivity is interrupted for short transient times, typically 202 of a subsecond duration. 204 RAW reuses terminology defined for DetNet in the "Deterministic 205 Networking Architecture" [RFC8655], e.g., PREOF for Packet 206 Replication, Elimination and Ordering Functions. 208 RAW also reuses terminology defined for 6TiSCH in [6TiSCH-ARCH] such 209 as the term Track. 6TiSCH defined a Track as a complex path with 210 associated PAREO operations. 212 In the context of the RAW work, Reliability and Availability are 213 defined as follows: 215 Reliability: Reliability is a measure of the probability that an 216 item will perform its intended function for a specified interval 217 under stated conditions. For RAW, the service that is expected is 218 delivery within a bounded latency and a failure is when the packet 219 is either lost or delivered too late. RAW expresses reliability 220 in terms of Mean Time Between Failure (MTBF) and Maximum 221 Consecutive Failures (MCF). 223 Availability: Availability is a measure of the relative amount of 224 time where a path operates in stated condition, in other words 225 (uptime)/(uptime+downtime). Because a serial wireless path may 226 not be good enough to provide the required availability, and even 227 2 parallel paths may not be over a longer period of time, the RAW 228 availability implies a path that is a lot more complex than what 229 DetNet typically envisages (a Track). 231 3. Related Work at The IETF 233 RAW intersects with protocols or practices in development at the IETF 234 as follows: 236 * The Dynamic Link Exchange Protocol (DLEP) [RFC8175] from [MANET] 237 can be leveraged at each hop to derive generic radio metrics 238 (e.g., based on LQI, RSSI, queueing delays and ETX) on individual 239 hops. 241 * Operations, Administration and Maintenance (OAM) work at [DetNet] 242 such as [DetNet-IP-OAM] for the case of the IP Data Plane observes 243 the state of DetNet paths, typically MPLS and IPv6 pseudowires 244 [DetNet-DP-FW], in the direction of the traffic. RAW needs 245 feedback that flows on the reverse path and gathers instantaneous 246 values from the radio receivers at each hop to inform back the 247 source and replicating relays so they can make optimized 248 forwarding decisions. The work named ICAN may be related as well. 250 * [BFD] detect faults in the path between an ingress and an egress 251 forwarding engines, but is unaware of the complexity of a path 252 with replication, and expects bidirectionality. BFD considers 253 delivery as success whereas with RAW the bounded latency can be as 254 important as the delivery itself. 256 * [SPRING] and [BIER] define in-band signaling that influences the 257 routing when decided at the head-end on the path. There's already 258 one RAW-related draft at BIER [BIER-PREF] more may follow. RAW 259 will need new in-band signaling when the decision is distributed, 260 e.g., required chances of reliable delivery to destination within 261 latency. This signaling enables relays to tune retries and 262 replication to meet the required SLA. 264 * [CCAMP] defines protocol-independent metrics and parameters 265 (measurement attributes) for describing links and paths that are 266 required for routing and signaling in technology-specific 267 networks. RAW would be a source of requirements for CCAMP to 268 define metrics that are significant to the focus radios. 270 4. Use Cases and Requirements Served 272 [RFC8578] presents a number of wireless use cases including Wireless 273 for Industrial Applications, Pro-Audio and SmartGrid. 274 [RAW-USE-CASES] adds a number of use cases that demonstrate the need 275 for RAW capabilities for new applications such as Pro-Gaming and 276 drones. The use cases can be abstracted in two families, radio 277 access protection and Track protection in a wireless mesh. 279 4.1. Radio Access Protection 281 To maintain the committed reliability at all times, a wireless host 282 may use more than one Radio Access Network (RAN) in parallel. 284 *** ** 285 RAN 1 ----- *** ** *** 286 / * ** **** 287 +----+ / * ** **** 288 | |- * ***** 289 |Host|--zzz- RAN 2 -- * Internet ***** 290 | |- * ***** 291 +----+ $$รน * ******* 292 \ *** *** ***** 293 RAN n -------- *** ***** 295 zzz = flapping now $$$ expensive 297 Figure 1: Radio Access Protection 299 The RANs may be heterogeneous, e.g., 5G [I-D.farkas-raw-5g] and Wi-Fi 300 [RAW-TECHNOS] for high-speed communication, in which case a Layer-3 301 abstraction becomes useful to select which of the RANs are used at a 302 particular point of time, and the amount of traffic that is 303 distributed over each RAN. 305 The idea is that the rest of the path to the destination(s) is 306 protected separately (e.g., uses non-congruent paths) and/or is a lot 307 more reliable, e.g., wired. In that case, RAW observes reliability 308 of the path through each of the RANs but only operates on the first 309 hop. 311 4.2. Track Protection in a Wireless Mesh 313 A Track (more in Section 6.1) if a multihop multipath radio mesh with 314 distribute PAREO capabilities. In that case, RAW operates through 315 the mesh and makes decisions either at the Ingress or at every hop. 317 A-------B-------C-----D 318 / \ / / \ 319 Ingress ----M-------N--zzzzz--- Egress 320 \ \ / / 321 P--zzz--Q-------------R 323 zzz = flapping now 325 Figure 2: Track Protection 327 5. RAW Architecture Considerations 329 5.1. Reliability and Availability 331 5.1.1. Reliability Engineering 333 The reliability criteria of a critical system pervade through its 334 elements, and if the system comprises a data network then the data 335 network is also subject to the inherited reliability criteria. It is 336 only natural to consider the art of Reliability Engineering and apply 337 it to wireless communicaitons in the context of RAW. 339 There are 3 pillars in the art of Reliability Engineering: 341 1. Elimination of single points of failure 343 2. Reliable crossover 345 3. Detection of faults as they occur 347 5.1.2. Reliability In Wireless Networks 349 The terms Reliaility and Availability are defined for RAW in 350 Section 2. Practically speaking a number of nines is often used to 351 indicate the reliability of a data link, e.g., 5 nines indicate a 352 Packet Delivery Ratio (PDR) of 99.999%. This number is typical is a 353 wired environment where the loss is due to a random event such as a 354 solar particle that affects the transmission of a particular frame, 355 but does not affect the previous or next frame, nor frames 356 transmitted on other links. 358 For a periodic pattern such as an automation control loop, this 359 number is proportional to the Mean Time Between Failure (MTBF). If a 360 single fault can have dramatic consequences, then the MTBF is the 361 expression of the chances that an unwanted event occurs. In data 362 networks, this is rarely the case. Packet loss cannot never be fully 363 avoided and the systems are built to resist to one loss, e.g., using 364 redundancy with Retries (HARQ) or Packet Replication and Elimination 365 (PRE), or, in a typical control loop, by linear interpolation from 366 the previous measuremnents. 368 But the latter method can not resist to multiple consecutive losses, 369 and a high MTBF is desired as a guarantee that this will not happen, 370 IOW that the losses-in-a-row can be bounded. In that case, what's 371 really desired is a Maximum Consecutive Failures (MCF). If the 372 number of losses in a row passes the MCF, the control loop has to 373 abort. Engineers that build automated processes use the network 374 reliability expressed in nines or as an MTBF to provide an MCF. 376 In contrast with wires networks, errors in transmission are a 377 predominent factor for packet loss in wireless. A given hop will 378 suffer from multipath fading for multiple packets in a row till the 379 something moves that changes the reflection patterns. The wireless 380 medium itself is a Shared Risk Link Group (SRLG) for nearby users of 381 the same spectrum, as an interference may affect multiple co-channel 382 transmissions between different peers within the interference domain 383 of the interferer, possibly even when they use different 384 technologies. 386 Transmission errors are typically not independent, and there nature 387 and duration unpredictable; as long as a physical object (e.g., a 388 metallic trolley etween peers) that affects the transmission is not 389 removed, or as long as the intererer (e.g., a radar) keeps 390 transmitting, packets in a row will be affected. The key word to 391 combat losses is diversity. A single packet may be sent at different 392 times over different paths that rely on different radio frequencies 393 and different PHY technologies, e.g., narrowband ns. spread spectrum. 394 It is typically retried a nmuber of times in case of a loss, and if 395 possible the retries should again vary all possible parameters. Each 396 form of diversity combats a particular cause of loss and use of 397 diversity must be maximised to optimize the PDR. 399 5.2. Prerequisites 401 A prerequisite to the RAW work is that an end-to-end routing function 402 computes a complex sub-topology along which forwarding can happen 403 between a source and one or more destinations. For 6TiSCH, this is a 404 Track. The concept of Track is specified in the 6TiSCH Architecture 405 [6TiSCH-ARCH]. Tracks provide a high degree of redundancy and 406 diversity and enable DetNet PREOF, end-to-end network coding, and 407 possibly radio-specific abstracted techniques such as ARQ, 408 overhearing, frequency diversity, time slotting, and possibly others. 410 How the routing operation computes the Track is out of scope for RAW. 411 The scope of the RAW operation is one Track, and the goal of the RAW 412 operation is to optimize the use of the Track at the forwarding 413 timescale to maintain the expected service while optimizing the usage 414 of constrained resources such as energy and spectrum. 416 Another prerequisite is that an IP link can be established over the 417 radio with some guarantees in terms of service reliability, e.g., it 418 can be relied upon to transmit a packet within a bounded latency and 419 provides a guaranteed BER/PDR outside rare but existing transient 420 outage windows that can last from split seconds to minutes. The 421 radio layer can be programmed with abstract parameters, and can 422 return an abstract view of the state of the Link to help forwarding 423 decision (think DLEP from MANET). In the layered approach, how the 424 radio manages its PHY layer is out of control and out of scope. 425 Whether it is single hop or meshed is also unknown and out of scope. 427 5.3. Routing Time Scale vs. Forwarding Time Scale 429 With DetNet, the end-to-end routing can be centralized and can reside 430 outside the network. In wireless, and in particular in a wireless 431 mesh, the path to the controller that performs the route computation 432 and maintenance expensive in terms of critical resources such as air 433 time and energy. 435 Reaching to the routing computation can also be slow in regards to 436 the speed of events that affect the forwarding operation at the radio 437 layer. Due to the cost and latency to perform a route computation, 438 the controller plane is not expected to be sensitive/reactive to 439 transient changes. The abstraction of a link at the routing level is 440 expected to use statistical operational metrics that aggregate the 441 behavior of a link over long periods of time, and represent its 442 availability as shades of gray as opposed to either up or down. 444 +----------------+ 445 | Controller | 446 | (PCE) | 447 | [Routing ] | 448 | [Function] | 449 +----------------+ 450 ^ 451 | 452 Slow 453 | 454 _-._-._-._-._-._-. | ._-._-._-._-._-._-._-._-._-._-._-._- 455 _-._-._-._-._-._-._-. | _-._-._-._-._-._-._-._-._-._-._-._- 456 | 457 Expensive 458 .... | ....... 459 .... . | . ..... 460 .... v ... 461 .. A-------B-------C---D .. 462 ... / \ / / \ .. 463 . I ----M-------N--zzz-- E .. 464 .. \ \ / / . 465 .. P--zzz--Q----------R .. 466 .. .. 467 ....... ... 468 ............... 469 zzz = flapping now 471 Figure 3: Time Scales 473 In the case of wireless, the changes that affect the forwarding 474 decision can happen frequently and often for short durations, e.g., a 475 mobile object moves between a transmitter and a receiver, and will 476 cancel the line of sight transmission for a few seconds, or a radar 477 measures the depth of a pool and interferes on a particular channel 478 for a split second. 480 There is thus a desire to separate the long term computation of the 481 route and the short term forwarding decision. In such a model, the 482 routing operation computes a complex Track that enables multiple Non- 483 Equal Cost Multi-Path (N-ECMP) forwarding solutions, and leaves it to 484 the forwarding plane to make the per-packet decision of which of 485 these possibilities should be used. 487 In the case of wires, the concept is known in traffic engineering 488 where an alternate path can be used upon the detection of a failure 489 in the main path, e.g., using OAM in MPLS-TP or BFD over a collection 490 of SD-WAN tunnels. RAW formalizes a forwarding time scale that is an 491 order(s) of magnitude shorter than the controler plane routing time 492 scale, and separates the protocols and metrics that are used at both 493 scales. Routing can operate on long term statistics such as delivery 494 ratio over minutes to hours, but as a first approximation can ignore 495 flapping. On the other hand, the RAW forwarding decision is made at 496 packet speed, and uses information that must be pertinent at the 497 present time for the current transmission. 499 6. RAW Architecture Components 501 6.1. Wireless Tracks 503 The "6TiSCH Architecture" [6TiSCH-ARCH] introduces the concept of 504 Track a a possibly complex path with the PAREO functions operated 505 within. 507 A simple track is composed of a direct sequence of reserved hops to 508 ensure the transmission of a single packet from a source node to a 509 destination node across a multihop path. 511 A Complex Track is designed as a directed acyclic graph from a source 512 node towards a destination node to support multi-path forwarding, as 513 introduced in "6TiSCH Architecture" [6TiSCH-ARCH]. By employing PRE 514 functions [RFC8655], several paths may be computed, and these paths 515 may be more or less independent. For example, a complex Track may 516 branch off and rejoin over non-congruent paths (branches). 518 Some more details for Deterministic Network PRE techniques are 519 presented in the following Section. 521 6.2. Source-Routed vs. Distributed Forwarding Decision 523 Within a large routed topology, the route-over mesh operation builds 524 a particular complex Track with one source and one or more 525 destinations; within the Track, packets may follow different paths 526 and may be subject to RAW forwarding operations that include 527 replication, elimination, retries, overhearing and reordering. 529 The RAW forwarding decisions include the selection of points of 530 replication and elimination, how many retries can take place, and a 531 limit of validity for the packet beyond which the packet should be 532 destroyed rather than forwarded uselessly further down the Track. 534 The decision to apply the RAW techniques must be done quickly, and 535 depends on a very recent and precise knowledge of the forwarding 536 conditions within the complex Track. There is a need for an 537 observation method to provide the RAW forwarding plane with the 538 specific knowledge of the state of the Track for the type of flow of 539 interest (e.g., for a QoS level of interest). To observe the whole 540 Track in quasi real time, RAW will consider existing tools such as 541 L2-triggers, DLEP, BFD and in-band and out-of-band OAM. 543 One possible way of making the RAW forwarding decisions is to make 544 them all at the ingress and express them in-band in the packet, which 545 requires new loose or strict Hop-by-hop signaling. To control the 546 RAW forwarding operation along a Track for the individual packets, 547 RAW may leverage and extend known techniques such as DetNet tagging, 548 Segment Routing (SRv6) or BIER-TE such as done with [BIER-PREF]. 550 An alternate way is to enable each forwarding node to make the RAW 551 forwarding decisions for a packet on its own, based on its knowledge 552 of the expectation (timeliness and reliability) for that packet and a 553 recent observation of the rest of the way across the possible paths 554 within the Track. Information about the service should be placed in 555 the packet and matched with the forwarding node's capabilities and 556 policies. 558 In either case, a per-flow state is installed in all intermediate 559 nodes to recognize the flow and determine the forwarding policy to be 560 applied. 562 6.3. PAREO Functions 564 In a nutshell, PRE establishes several paths in a network to provide 565 redundancy and parallel transmissions to bound the end-to-end delay 566 to traverse the network. Optionally, promiscuous listening between 567 paths is possible, such that the nodes on one path may overhear 568 transmissions along the other path. Considering the scenario shown 569 in Figure 4, many different paths are possible for S to reach R. A 570 simple way to benefit from this topology could be to use the two 571 independent paths via nodes A, C, E and via B, D, F. But more 572 complex paths are possible by interleaving transmissions from the 573 lower level of the path to the upper level. 575 PRE may also take advantage of the shared properties of the wireless 576 medium to compensate for the potential loss that is incurred with 577 radio transmissions. For instance, when the source sends to A, B may 578 listen also and get a second chance to receive the frame without an 579 additional transmission. Note that B would not have to listen if it 580 already received that particular frame at an earlier timeslot in a 581 dedicated transmission towards B. 583 (A) (C) (E) 585 source (S) (R) (root) 587 (B) (D) (F) 589 Figure 4: A Typical Ladder Shape with Two Parallel Paths Toward 590 the Destination 592 The PRE model can be implemented in both centralized and distributed 593 scheduling approaches. In the centralized approach, a Path 594 Computation Element (PCE) scheduler calculates the routes and 595 schedules the communication among the nodes along a circuit such as a 596 Label switched path. In the distributed approach, each node selects 597 its route to the destination, typically using a source routing 598 header. In both cases, at each node in the paths, a default parent 599 and alternative parent(s) should be selected to set up complex 600 tracks. 602 In the following Subsections, all the required operations defined by 603 PRE, namely, Alternative Path Selection, Packet Replication, Packet 604 Elimination and Promiscuous Overhearing, are described. 606 6.3.1. Packet Replication 608 The objective of PRE is to provide deterministic networking 609 properties: high reliability and bounded latency. To achieve this 610 goal, determinism in every hop of the forwarding paths MUST be 611 guaranteed. By employing a Packet Replication procedure, each node 612 forwards a copy of each data packet to multiple parents: its Default 613 Parent (DP) and multiple Alternative Parents (APs). To do so, each 614 node (i.e., source and intermediate node) transmits the data packet 615 multiple times in unicast to each parent. For instance, in Figure 5, 616 the source node S is transmitting the packet to both parents, nodes A 617 and B, in two different timeslots within the same TSCH slotframe. An 618 example TSCH schedule is shown in Figure 6. Thus, the packet 619 eventually obtains parallel paths to the destination. 621 ===> (A) => (C) => (E) === 622 // \\// \\// \\ 623 source (S) //\\ //\\ (R) (root) 624 \\ // \\ // \\ // 625 ===> (B) => (D) => (F) === 627 Figure 5: Packet Replication: S transmits twice the same data 628 packet, to its DP (A) and to its AP (B). 630 Timeslot 631 +---------++------+------+------+------+------+------+------+ 632 | Channel || 0 | 1 | 2 | 3 | 4 | 5 | 6 | 633 +---------++======+======+======+======+======+======+======+ 634 | 0 || S->A | S->B | B->C | B->D | C->F | E->R | F->R | 635 +---------++------+------+------+------+------+------+------+ 636 | 1 || | A->C | A->D | C->E | D->E | D->F | | 637 +---------++------+------+------+------+------+------+------+ 639 Figure 6: Packet Replication: Sample TSCH schedule 641 6.3.2. Packet Elimination 643 The replication operation increases the traffic load in the network, 644 due to packet duplications. Thus, a Packet Elimination operation 645 SHOULD be applied at each RPL DODAG level to reduce the unnecessary 646 traffic. To this aim, once a node receives the first copy of a data 647 packet, it discards the subsequent copies. Because the first copy 648 that reaches a node is the one that matters, it is the only copy that 649 will be forwarded upward. Then, once a node performs the Packet 650 Elimination operation, it will proceed with the Packet Replication 651 operation to forward the packet toward the RPL DODAG Root. 653 6.3.3. Promiscuous Overhearing 655 Considering that the wireless medium is broadcast by nature, any 656 neighbor of a transmitter may overhear a transmission. By employing 657 the Promiscuous Overhearing operation, a DP and some AP(s) eventually 658 have more chances to receive the data packets. In Figure 7, when 659 node A is transmitting to its DP (node C), the AP (node D) and its 660 sibling (node B) may decode this data packet as well. As a result, 661 by employing corellated paths, a node may have multiple opportunities 662 to receive a given data packet. This feature not only enhances the 663 end-to-end reliability but also it reduces the end-to-end delay and 664 increases energy efficiency. 666 ===> (A) ====> (C) ====> (E) ==== 667 // ^ | \\ \\ 668 source (S) | | \\ (R) (root) 669 \\ | v \\ // 670 ===> (B) ====> (D) ====> (F) ==== 672 Figure 7: Unicast to DP with Overhearing: by employing 673 Promiscuous Overhearing, DP, AP and the sibling nodes have more 674 opportunities to receive the same data packet. 676 7. Security Considerations 678 8. IANA Considerations 680 This document has no IANA actions. 682 9. ConTributors 684 Xavi Vilajosana: Wireless Networks Research Lab, Universitat Oberta 685 de Catalunya 687 Rex Buddenberg: 689 Remous-Aris Koutsiamanis: IMT Atlantique 691 Nicolas Montavont: IMT Atlantique 693 10. Acknowledgments 695 The authors wish to thank 697 11. References 699 11.1. Normative References 701 [6TiSCH-ARCH] 702 Thubert, P., "An Architecture for IPv6 over the TSCH mode 703 of IEEE 802.15.4", Work in Progress, Internet-Draft, 704 draft-ietf-6tisch-architecture-28, 29 October 2019, 705 . 708 [RAW-TECHNOS] 709 Thubert, P., Cavalcanti, D., Vilajosana, X., and C. 710 Schmitt, "Reliable and Available Wireless Technologies", 711 Work in Progress, Internet-Draft, draft-thubert-raw- 712 technologies-04, 6 January 2020, 713 . 716 [RAW-USE-CASES] 717 Papadopoulos, G., Thubert, P., Theoleyre, F., and C. 718 Bernardos, "RAW use cases", Work in Progress, Internet- 719 Draft, draft-bernardos-raw-use-cases-03, 8 March 2020, 720 . 723 [RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases", 724 RFC 8578, DOI 10.17487/RFC8578, May 2019, 725 . 727 [RFC8175] Ratliff, S., Jury, S., Satterwhite, D., Taylor, R., and B. 728 Berry, "Dynamic Link Exchange Protocol (DLEP)", RFC 8175, 729 DOI 10.17487/RFC8175, June 2017, 730 . 732 [RFC8557] Finn, N. and P. Thubert, "Deterministic Networking Problem 733 Statement", RFC 8557, DOI 10.17487/RFC8557, May 2019, 734 . 736 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 737 "Deterministic Networking Architecture", RFC 8655, 738 DOI 10.17487/RFC8655, October 2019, 739 . 741 11.2. Informative References 743 [BIER-PREF] 744 Thubert, P., Eckert, T., Brodard, Z., and H. Jiang, "BIER- 745 TE extensions for Packet Replication and Elimination 746 Function (PREF) and OAM", Work in Progress, Internet- 747 Draft, draft-thubert-bier-replication-elimination-03, 3 748 March 2018, . 751 [DetNet-IP-OAM] 752 Mirsky, G., Chen, M., and D. Black, "Operations, 753 Administration and Maintenance (OAM) for Deterministic 754 Networks (DetNet) with IP Data Plane", Work in Progress, 755 Internet-Draft, draft-mirsky-detnet-ip-oam-02, 23 March 756 2020, . 759 [DetNet-DP-FW] 760 Varga, B., Farkas, J., Berger, L., Malis, A., and S. 761 Bryant, "DetNet Data Plane Framework", Work in Progress, 762 Internet-Draft, draft-ietf-detnet-data-plane-framework-04, 763 3 February 2020, . 766 [I-D.farkas-raw-5g] 767 Farkas, J., Dudda, T., Shapin, A., and S. Sandberg, "5G - 768 Ultra-Reliable Wireless Technology with Low Latency", Work 769 in Progress, Internet-Draft, draft-farkas-raw-5g-00, 1 770 April 2020, 771 . 773 [MANET] IETF, "Mobile Ad hoc Networking", 774 . 776 [DetNet] IETF, "Deterministic Networking", 777 . 779 [SPRING] IETF, "Source Packet Routing in Networking", 780 . 782 [BIER] IETF, "Bit Indexed Explicit Replication", 783 . 785 [BFD] IETF, "Bidirectional Forwarding Detection", 786 . 788 [CCAMP] IETF, "Common Control and Measurement Plane", 789 . 791 Authors' Addresses 793 Pascal Thubert (editor) 794 Cisco Systems, Inc 795 Building D 796 45 Allee des Ormes - BP1200 797 06254 MOUGINS - Sophia Antipolis 798 France 800 Phone: +33 497 23 26 34 801 Email: pthubert@cisco.com 802 Georgios Z. Papadopoulos 803 IMT Atlantique 804 Office B00 - 114A 805 2 Rue de la Chataigneraie 806 35510 Cesson-Sevigne - Rennes 807 France 809 Phone: +33 299 12 70 04 810 Email: georgios.papadopoulos@imt-atlantique.fr