idnits 2.17.1 draft-ietf-raw-architecture-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 825 has weird spacing: '...-- Node z-- ...' == Line 830 has weird spacing: '... Node z-- ...' -- The document date (29 November 2021) is 878 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'PCE' is mentioned on line 732, but not defined == Outdated reference: A later version (-08) exists of draft-ietf-raw-technologies-04 == Outdated reference: A later version (-11) exists of draft-ietf-raw-use-cases-03 -- Obsolete informational reference (is this intentional?): RFC 3272 (ref. 'TE') (Obsoleted by RFC 9522) == Outdated reference: A later version (-08) exists of draft-irtf-panrg-path-properties-04 == Outdated reference: A later version (-15) exists of draft-thubert-6man-ipv6-over-wireless-10 == Outdated reference: A later version (-11) exists of draft-ietf-detnet-oam-framework-05 Summary: 0 errors (**), 0 flaws (~~), 9 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 RAW P. Thubert, Ed. 3 Internet-Draft Cisco Systems 4 Intended status: Informational G.Z. Papadopoulos 5 Expires: 2 June 2022 IMT Atlantique 6 29 November 2021 8 Reliable and Available Wireless Architecture 9 draft-ietf-raw-architecture-02 11 Abstract 13 Reliable and Available Wireless (RAW) provides for high reliability 14 and availability for IP connectivity over a wireless medium. The 15 wireless medium presents significant challenges to achieve 16 deterministic properties such as low packet error rate, bounded 17 consecutive losses, and bounded latency. This document defines the 18 RAW Architecture following an OODA loop that involves OAM, PCE, PSE 19 and PAREO functions. It builds on the DetNet Architecture and 20 discusses specific challenges and technology considerations needed to 21 deliver DetNet service utilizing scheduled wireless segments and 22 other media, e.g., frequency/time-sharing physical media resources 23 with stochastic traffic. 25 Status of This Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at https://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on 2 June 2022. 42 Copyright Notice 44 Copyright (c) 2021 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 49 license-info) in effect on the date of publication of this document. 50 Please review these documents carefully, as they describe your rights 51 and restrictions with respect to this document. Code Components 52 extracted from this document must include Revised BSD License text as 53 described in Section 4.e of the Trust Legal Provisions and are 54 provided without warranty as described in the Revised BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 59 2. The RAW problem . . . . . . . . . . . . . . . . . . . . . . . 5 60 2.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 5 61 2.1.1. Acronyms . . . . . . . . . . . . . . . . . . . . . . 6 62 2.1.2. Link and Direction . . . . . . . . . . . . . . . . . 6 63 2.1.3. Path and Tracks . . . . . . . . . . . . . . . . . . . 7 64 2.1.4. Deterministic Networking . . . . . . . . . . . . . . 8 65 2.1.5. Reliability and Availability . . . . . . . . . . . . 9 66 2.1.6. OAM variations . . . . . . . . . . . . . . . . . . . 10 67 2.2. Reliability and Availability . . . . . . . . . . . . . . 11 68 2.2.1. High Availability Engineering Principles . . . . . . 11 69 2.2.2. Applying Reliability Concepts to Networking . . . . . 14 70 2.2.3. Reliability in the Context of RAW . . . . . . . . . . 15 71 2.3. Routing Time Scale vs. Forwarding Time Scale . . . . . . 16 72 3. The RAW Conceptual Model . . . . . . . . . . . . . . . . . . 18 73 4. The OODA Loop . . . . . . . . . . . . . . . . . . . . . . . . 20 74 5. Observe: The RAW OAM . . . . . . . . . . . . . . . . . . . . 21 75 6. Orient: The Path Computation Engine . . . . . . . . . . . . . 22 76 7. Decide: The Path Selection Engine . . . . . . . . . . . . . . 22 77 8. Act: The PAREO Functions . . . . . . . . . . . . . . . . . . 24 78 8.1. Packet Replication . . . . . . . . . . . . . . . . . . . 25 79 8.2. Packet Elimination . . . . . . . . . . . . . . . . . . . 26 80 8.3. Promiscuous Overhearing . . . . . . . . . . . . . . . . . 26 81 8.4. Constructive Interference . . . . . . . . . . . . . . . . 27 82 9. Security Considerations . . . . . . . . . . . . . . . . . . . 27 83 9.1. Forced Access . . . . . . . . . . . . . . . . . . . . . . 27 84 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 27 85 11. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 27 86 12. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 28 87 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 28 88 13.1. Normative References . . . . . . . . . . . . . . . . . . 28 89 13.2. Informative References . . . . . . . . . . . . . . . . . 29 90 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 31 92 1. Introduction 94 Deterministic Networking is an attempt to emulate the properties of a 95 serial link over a switched fabric, by providing a bounded latency 96 and eliminating congestion loss, even when co-existing with best- 97 effort traffic. It is getting traction in various industries 98 including professional A/V, manufacturing, online gaming, and 99 smartgrid automation, enabling cost and performance optimizations 100 (e.g., vs. loads of P2P cables). 102 Bringing determinism in a packet network means eliminating the 103 statistical effects of multiplexing that result in probabilistic 104 jitter and loss. This can be approached with a tight control of the 105 physical resources to maintain the amount of traffic within a 106 budgetted volume of data per unit of time that fits the physical 107 capabilities of the underlying network, and the use of time-shared 108 resources (bandwidth and buffers) per circuit, and/or by shaping and/ 109 or scheduling the packets at every hop. 111 This innovation was initially introduced on wired networks, with IEEE 112 802.1 Time Sensitive networking (TSN) - for Ethernet LANs - and IETF 113 DetNet. But the wired and the wireless media are fundamentally 114 different at the physical level and in the possible abstractions that 115 can be built for IPv6 [IPoWIRELESS]. Nevertheless, deterministic 116 capabilities are required in a number of wireless use cases as well 117 [RAW-USE-CASES]. With new scheduled radios such as TSCH and OFDMA 118 [RAW-TECHNOS] being developped to provide determinism over wireless 119 links at the lower layers, providing DetNet capabilities is now 120 becoming possible. 122 Wireless networks operate on a shared medium where uncontrolled 123 interference, including the self-induced multipath fading cause 124 random transmission losses. Fixed and mobile obstacles and 125 reflectors may block or alter the signal, causing transient and 126 unpredictable variations of the throughput and packet delivery ratio 127 (PDR) of a wireless link. This adds new dimensions to the 128 statistical effects that affect the quality and reliability of the 129 link. Multiple links and transmissions must be used, and the 130 challenge is to provide enough diversity and redundancy to ensure the 131 timely packet delivery while preserving energy and optimizing the use 132 of the shared spectrum. 134 Reliable and Available Wireless (RAW) takes up the challenge of 135 providing highly available and reliable end-to-end performances in a 136 network with scheduled wireless segments. To defeat those additional 137 causes of transmission delay and loss, RAW leverages deterministic 138 layer-2 capabilities while controlling the use of diversity in the 139 spatial, time, code, radio technology, and frequency domains from 140 layer-3. 142 While the generic "Deterministic Networking Problem Statement" 143 [RFC8557] applies to both the wired and the wireless media, the 144 methods to achieve RAW must extend those used to support time- 145 sensitive networking over wires, as a RAW solution has to address 146 less consistent transmissions, energy conservation and shared 147 spectrum efficiency. 149 RAW provides DetNet elements that are specialized for IPv6 flows 150 [IPv6] over deterministic short range radios [RAW-TECHNOS]. 151 Conceptually, RAW is agnostic to the radio layer underneath though 152 the capability to schedule transmissions is assumed. How the PHY is 153 programmed to do so, and whether the radio is single-hop or meshed, 154 are unknown at the IP layer and not part of the RAW abstraction. 155 Nevertheless, cross-layer optimizations may take place to ensure 156 proper link awareness (think, link quality) and packet handling 157 (think, scheduling). 159 The "Deterministic Networking Architecture" [RFC8655] is composed of 160 three planes: the Application (User) Plane, the Controller Plane, and 161 the Network Plane. The RAW Architecture extends the DetNet Network 162 Plane, to accommodate one or multiple hops of homogeneous or 163 heterogeneous wireless technologies, e.g. a Wi-Fi6 Mesh or parallel 164 CBRS access links federated by a 5G backhaul. 166 RAW and DetNet associate application that that require a particular 167 treatment to a path that was provisionned to procure that treatment. 168 This may be seen as a form of Path Aware Networking and may be 169 subject to impediments documented in [RFC9049]. 171 The establishment of a path is not in-scope for RAW. It may be the 172 product of a centralized Controller Plane as described for DetNet. 173 As opposed to wired networks, the action of installing a path over a 174 set of wireless links may be very slow relative to the speed at which 175 the radio conditions vary, and it makes sense in the wireless case to 176 provide redundant forwarding solutions along a complex path and to 177 leave it to the Network Plane to select which of those forwarding 178 solutions are to be used for a given packet based on the current 179 conditions. 181 RAW distinguishes the longer time scale at which routes are computed 182 from the the shorter forwarding time scale where per-packet decisions 183 are made. RAW operates within the Network Plane at the forwarding 184 time scale on one DetNet flow over a complex path called a Track. 185 The Track is preestablished and installed by means outside of the 186 scope of RAW; it may be strict or loose depending on whether each or 187 just a subset of the hops are observed and controlled by RAW. 189 The RAW Architecture is structured as an OODA Loop (Observe, Orient, 190 Decide, Act). It involves: 192 1. Network Plane measurement protocols for Operations, 193 Administration and Maintenance (OAM) to Observe some or all hops 194 along a Track as well as the end-to-end packet delivery 196 2. Controller plane elements to reports the links statistics to a 197 Path computation Element (PCE) in a centralized controller that 198 computes and installs the Tracks and provides meta data to Orient 199 the routing decision 201 3. A Runtime distributed Path Selection Engine (PSE) that Decides 202 which subTrack to use for the next packet(s) that are routed 203 along the Track 205 4. Packet (hybrid) ARQ, Replication, Elimination and Ordering 206 Dataplane actions that operate at the DetNet Service Layer to 207 increase the reliability of the end-to-end transmission. The RAW 208 architecture also covers in-situ signalling when the decision is 209 Acted by a node that down the Track from the PSE. 211 The overall OODA Loop optimizes the use of redundancy to achieve the 212 required reliability and availability Service Level Agreement (SLA) 213 while minimizing the use of constrained resources such as spectrum 214 and battery. 216 2. The RAW problem 218 2.1. Terminology 220 RAW reuses terminology defined for DetNet in the "Deterministic 221 Networking Architecture" [RFC8655], e.g., PREOF for Packet 222 Replication, Elimination and Ordering Functions. 224 RAW also reuses terminology defined for 6TiSCH in [6TiSCH-ARCHI] such 225 as the term Track. A Track as a complex path with associated PAREO 226 operations. The concept is abstract to the underlaying technology 227 and applies to any fully or partially wireless mesh, including, e.g., 228 a Wi-Fi mesh. RAW specifies strict and loose Tracks depending on 229 whether the path is fully controlled by RAW or traverses an opaque 230 network where RAW cannot observe and control the individual hops. 232 RAW uses the following terminology and acronyms: 234 2.1.1. Acronyms 236 2.1.1.1. ARQ 238 Automatic Repeat Request, enabling an acknowledged transmission and 239 retries. ARQ is a typical model at Layer-2 on a wireless medium. It 240 is typically avoided end-to-end on deterministic flows because it 241 introduces excessive indetermination in latency, but a limited number 242 of retries within a bounded time may be used over a wireless link and 243 yet respect end-to-end constraints. 245 2.1.1.2. OAM 247 OAM stands for Operations, Administration, and Maintenance, and 248 covers the processes, activities, tools, and standards involved with 249 operating, administering, managing and maintaining any system. This 250 document uses the terms Operations, Administration, and Maintenance, 251 in conformance with the 'Guidelines for the Use of the "OAM" Acronym 252 in the IETF' [RFC6291] and the system observed by the RAW OAM is the 253 Track. 255 2.1.1.3. OODA 257 Observe, Orient, Decide, Act. The OODA Loop is a conceptual cyclic 258 model developed by USAF Colonel John Boyd, and that is applicable in 259 multiple domains where agility can provide benefits against brute 260 force. 262 2.1.1.4. PAREO 264 Packet (hybrid) ARQ, Replication, Elimination and Ordering. PAREO is 265 a superset Of DetNet's PREOF that includes radio-specific techniques 266 such as short range broadcast, MUMIMO, constructive interference and 267 overhearing, which can be leveraged separately or combined to 268 increase the reliability. 270 2.1.2. Link and Direction 271 2.1.2.1. Flapping 273 In the context of RAW, a link flaps when the reliability of the 274 wireless connectivity drops abruptly for a short period of time, 275 typically of a subsecond to seconds duration. 277 2.1.2.2. Uplink 279 Connection from end-devices to a data communication equipment. In 280 the context of wireless, uplink refers to the connection between a 281 station (STA) and a controller (AP) or a User Equipment (UE) to a 282 Base Station (BS) such as a 3GPP 5G gNodeB (gNb). 284 2.1.2.3. Downlink 286 The reverse direction from uplink. 288 2.1.2.4. Downstream 290 Following the the direction of the flow data path along a Track. 292 2.1.2.5. Upstream 294 Against the direction of the flow data path along a Track. 296 2.1.3. Path and Tracks 298 2.1.3.1. Path 300 Quoting section 1.1.3 of [INT-ARCHI]: 302 | "At a given moment, all the IP datagrams from a particular source 303 | host to a particular destination host will typically traverse the 304 | same sequence of gateways. We use the term "path" for this 305 | sequence. Note that a path is uni-directional; it is not unusual 306 | to have different paths in the two directions between a given host 307 | pair.". 309 Section 2 of [I-D.irtf-panrg-path-properties] points to a longer, 310 more modern definition of path, which begins as follows: 312 | A sequence of adjacent path elements over which a packet can be 313 | transmitted, starting and ending with a node. A path is 314 | unidirectional. Paths are time-dependent, i.e., the sequence of 315 | path elements over which packets are sent from one node to another 316 | may change. A path is defined between two nodes. 318 It follows that the general acceptance of a path is a linear sequence 319 of nodes, as opposed to a multi-dimensional graph. In the context of 320 this document, a path is observed by following one copy of a packet 321 that is injected in a Track and possibly replicated within. 323 2.1.3.2. Track 325 A networking graph that can be followed to transport packets with 326 equivalent treatment; as opposed to the definition of a path above, a 327 Track Track is not necessarily linear. It may contain multiple paths 328 that may fork and rejoin, for instance to enable the RAW PAREO 329 operations. 331 In DetNet [RFC8655] terms, a Track has the following properties: 333 * A Track has one Ingress and one Egress nodes, which operate as 334 DetNet Edge nodes. 336 * A Track is reversible, meaning that packets can be routed against 337 the flow of data packets, e.g., to carry OAM measurements or 338 control messages back to the Ingress. 340 * The vertices of the Track are DetNet Relay nodes that operate at 341 the DetNet Service sublayer and provide the PAREO functions. 343 * The topological edges of the graph are serial sequences of DetNet 344 Transit nodes that operate at the DetNet Forwarding sublayer. 346 2.1.3.3. SubTrack 348 A Track within a Track. The RAW PSE selects a subTrack on a per- 349 packet or a per-collection of packets basis to provide the desired 350 reliability for the transported flows. 352 2.1.3.4. Segment 354 A serial path formed by a topological edge of a Track. East-West 355 Segments are oriented from Ingress (East) to Egress (West). North/ 356 South Segments can be bidirectional; to avoid loops, measures must be 357 taken to ensure that a given packet flows either Northwards or 358 Southwards along a bidirectional Segment, but never bounces back. 360 2.1.4. Deterministic Networking 362 This document reuses the terminology in section 2 of [RFC8557] and 363 section 4.1.2 of [RFC8655] for deterministic networking and 364 deterministic networks. 366 2.1.4.1. Flow 368 A collection of consecutive packets that must be placed on the same 369 Track to receive an equivalent treatment from Ingress to Egress 370 within the Track. Multiple flows may be transported along the same 371 Track. The subTrack that is selected for the flow may change over 372 time under the control of the PSE. 374 2.1.4.2. Deterministic Flow Identifier (L2) 376 A tuple identified by a stream_handle, and provided by a bridge, in 377 accordance with IEEE 802.1CB. The tuple comprises at least src MAC, 378 dst MAC, VLAN ID, and L2 priority. Continuous streams are 379 characterized by bandwidth and max packet size; scheduled streams are 380 characterized by a repeating pattern of timed transmissions. 382 2.1.4.3. Deterministic Flow Identifier (L3) 384 See section 3.3 of [DetNet-DP]. The classical IP 5-tuple that 385 identifies a flow comprises the src IP, dst IP, src port, dest port, 386 and the upper layer protocol (ULP). DetNet uses a 6-tuple where the 387 extra field is the DSCP field in the packet. The IPv6 flow label is 388 not used. for that purpose. 390 2.1.5. Reliability and Availability 392 In the context of the RAW work, Reliability and Availability are 393 defined as follows: 395 2.1.5.1. Service Level Agreement 397 In the context of RAW, an SLA (service level agreement) is a contract 398 between a provider, the network, and a client, the application flow, 399 about measurable metrics such as latency boundaries, consecutive 400 losses, and packet delivery ratio (PDR). 402 2.1.5.2. Service Level Objective 404 A service level objective (SLO) is one term in the SLO, for which 405 specific network setting and operations are implemented. For 406 instance, a dynamic tuning of the packet redundancy will address an 407 SLO of consecutive losses in a row by augmenting the chances of 408 delivery of a packet that follows a loss.). 410 2.1.5.3. Service Level Indicator 412 A service level indicator (SLI) measures the complience of an SLO to 413 the terms of the contrast. It can be for instance the statistics of 414 individual losses and losses in a row as time series.). 416 2.1.5.4. Reliability 418 Reliability is a measure of the probability that an item will perform 419 its intended function for a specified interval under stated 420 conditions (SLA). RAW expresses reliability in terms of Mean Time 421 Between Failure (MTBF) and Maximum Consecutive Failures (MCF). More 422 in [NASA].). 424 2.1.5.5. Available 426 That is exempt of unscheduled outage or derivation from the terms of 427 the SLA. A basic expectation for a RAW network is that the flow is 428 maintained in the face of any single breakage or flapping. 430 2.1.5.6. Availability 432 Availability is a measure of the relative amount of time where a RAW 433 Network operates in stated condition (SLA), expressed as 434 (uptime)/(uptime+downtime). Because a serial wireless path may not 435 be good enough to provide the required reliability, and even 2 436 parallel paths may not be over a longer period of time, the RAW 437 availability implies a journey that is a lot more complex than 438 following a serial path. 440 2.1.6. OAM variations 442 2.1.6.1. Active OAM 444 See [RFC7799]. In the context of RAW, Active OAM is used to observe 445 a particular Track, subTrack, or Segment of a Track regardless of 446 whether it is used for traffic at that time. 448 2.1.6.2. In-Band OAM 450 An active OAM packet is considered in-band for the monitored Track 451 when it traverses the same set of links and interfaces and if the OAM 452 packet receives the same QoS and PAREO treatment as the packets of 453 the data flows that are injected in the Track. 455 2.1.6.3. Out-of-Band OAM 457 Out-of-band OAM is an active OAM whose path is not topologically 458 congruent to the Track, or its test packets receive a QoS and/or 459 PAREO treatment that is different from that of the packets of the 460 data flows that are injected in the Track, or both. 462 2.1.6.4. Limited OAM 464 An active OAM packet is a Limited OAM packet when it observes the RAW 465 operation over a node, a segment, or a subTrack of the Track, though 466 not from Ingress to Egress. It is injected in the datapath and 467 extracted from the datapath around the particular function or 468 subnetwork (e.g., around a relay providing a service layer 469 replication point) that is being tested. 471 2.1.6.5. Upstream OAM 473 An upstream OAM packet is an Out-of-Band OAM packet that traverses 474 the Track from egress to ingress on the reverse direction, to capture 475 and report OAM measurements upstream. The collection may capture all 476 information along the whole Track, or it may only learn select data 477 across all, or only a particular subTrack, or Segment of a Track. 479 2.1.6.6. Residence Time 481 A residence time (RT) is defined as the time period between the 482 reception of a packet starts and the transmission of the packet 483 begins. In the context of RAW, RT is useful for a transit node, not 484 ingress or egress. 486 2.1.6.7. Additional References 488 [DetNet-OAM] provides additional terminology related to OAM in the 489 context of DetNet and by extension of RAW, whereas [RFC7799] defines 490 the Active, Passive, and Hybrid OAM methods. 492 2.2. Reliability and Availability 494 2.2.1. High Availability Engineering Principles 496 The reliability criteria of a critical system pervade through its 497 elements, and if the system comprises a data network then the data 498 network is also subject to the inherited reliability and availability 499 criteria. It is only natural to consider the art of high 500 availability engineering and apply it to wireless communications in 501 the context of RAW. 503 There are three principles [pillars] of high availability 504 engineering: 506 1. elimination of single points of failure 507 2. reliable crossover 508 3. prompt detection of failures as they occur. 510 These principles are common to all high availability systems, not 511 just ones with Internet technology at the center. Examples of both 512 non-Internet and Internet are included. 514 2.2.1.1. Elimination of Single Points of Failure 516 Physical and logical components in a system happen to fail, either as 517 the effect of wear and tear, when used beyond acceptable limits, or 518 due to a software bug. It is necessary to decouple component failure 519 from system failure to avoid the latter. This allows failed 520 components to be restored while the rest of the system continues to 521 function. 523 IP Routers leverage routing protocols to compute alternate routes in 524 case of a failure. There is a rather open-ended issue over alternate 525 routes -- for example, when links are cabled through the same 526 conduit, they form a shared risk link group (SRLG), and will share 527 the same fate if the bundle is cut. The same effect can happen with 528 virtual links that end up in a same physical transport through the 529 games of encapsulation. In a same fashion, an interferer or an 530 obstacle may affect multiple wireless transmissions at the same time, 531 even between different sets of peers. 533 Intermediate network Nodes such as routers, switches and APs, wire 534 bundles and the air medium itself can become single points of 535 failure. For High Availability, it is thus required to use 536 physically link- and Node-disjoint paths; in the wireless space, it 537 is also required to use the highest possible degree of diversity in 538 the transmissions over the air to combat the additional causes of 539 transmission loss. 541 From an economics standpoint, executing this principle properly 542 generally increases capitalization expense because of the redundant 543 equipment. In a constrained network where the waste of energy and 544 bandwidth should be minimized, an excessive use of redundant links 545 must be avoided; for RAW this means that the extra bandwidth must be 546 used wisely and with parcimony. 548 2.2.1.2. Reliable Crossover 550 Having a backup equipment has a limited value unless it can be 551 reliably switched into use within the down-time parameters. IP 552 Routers execute reliable crossover continuously because the routers 553 will use any alternate routes that are available [RFC0791]. This is 554 due to the stateless nature of IP datagrams and the dissociation of 555 the datagrams from the forwarding routes they take. The "IP Fast 556 Reroute Framework" [FRR] analyzes mechanisms for fast failure 557 detection and path repair for IP Fast-Reroute, and discusses the case 558 of multiple failures and SRLG. Examples of FRR techniques include 559 Remote Loop-Free Alternate [RLFA-FRR] and backup label-switched path 560 (LSP) tunnels for the local repair of LSP tunnels using RSVP-TE 561 [RFC4090]. 563 Deterministic flows, on the contrary, are attached to specific paths 564 where dedicated resources are reserved for each flow. This is why 565 each DetNet path must inherently provide sufficient redundancy to 566 provide the guaranteed SLA at all times. The DetNet PREOF typically 567 leverages 1+1 redundancy whereby a packet is sent twice, over non- 568 congruent paths. This avoids the gap during the fast reroute 569 operation, but doubles the traffic in the network. 571 In the case of RAW, the expectation is that multiple transient faults 572 may happen in overlapping time windows, in which case the 1+1 573 redundancy with delayed reestablishment of the second path will not 574 provide the required guarantees. The Data Plane must be configured 575 with a sufficient degree of redundancy to select an alternate 576 redundant path immediately upon a fault, without the need for a slow 577 intervention from the controller plane. 579 2.2.1.3. Prompt Notification of Failures 581 The execution of the two above principles is likely to render a 582 system where the user will rarely see a failure. But someone needs 583 to in order to direct maintenance. 585 There are many reasons for system monitoring (FCAPS for fault, 586 configuration, accounting, performance, security is a handy mental 587 checklist) but fault monitoring is sufficient reason. 589 "An Architecture for Describing Simple Network Management Protocol 590 (SNMP) Management Frameworks" [STD 62] describes how to use SNMP to 591 observe and correct long-term faults. 593 "Overview and Principles of Internet Traffic Engineering" [TE] 594 discusses the importance of measurement for network protection, and 595 provides abstract an method for network survivability with the 596 analysis of a traffic matrix as observed by SNMP, probing techniques, 597 FTP, IGP link state advertisements, and more. 599 Those measurements are needed in the context of RAW to inform the 600 controller and make the long term reactive decision to rebuild a 601 complex path. But RAW itself operates in the Network Plane at a 602 faster time scale. To act on the Data Plane, RAW needs live 603 information from the Operational Plane , e.g., using Bidirectional 604 Forwarding Detection [BFD] and its variants (bidirectional and remote 605 BFD) to protect a link, and OAM techniques to protect a path. 607 2.2.2. Applying Reliability Concepts to Networking 609 The terms Reliability and Availability are defined for use in RAW in 610 Section 2.1 and the reader is invited to read [NASA] for more details 611 on the general definition of Reliability. Practically speaking a 612 number of nines is often used to indicate the reliability of a data 613 link, e.g., 5 nines indicate a Packet Delivery Ratio (PDR) of 614 99.999%. 616 This number is typical in a wired environment where the loss is due 617 to a random event such as a solar particle that affects the 618 transmission of a particular frame, but does not affect the previous 619 or next frame, nor frames transmitted on other links. Note that the 620 QoS requirements in RAW may include a bounded latency, and a packet 621 that arrives too late is a fault and not considered as delivered. 623 For a periodic networking pattern such as an automation control loop, 624 this number is proportional to the Mean Time Between Failures (MTBF). 625 When a single fault can have dramatic consequences, the MTBF 626 expresses the chances that the unwanted fault event occurs. In data 627 networks, this is rarely the case. Packet loss cannot never be fully 628 avoided and the systems are built to resist to one loss, e.g., using 629 redundancy with Retries (HARQ) or Packet Replication and Elimination 630 (PRE), or, in a typical control loop, by linear interpolation from 631 the previous measurements. 633 But the linear interpolation method cannot resist multiple 634 consecutive losses, and a high MTBF is desired as a guarantee that 635 this will not happen, IOW that the number of losses-in-a-row can be 636 bounded. In that case, what is really desired is a Maximum 637 Consecutive Failures (MCF). If the number of losses in a row passes 638 the MCF, the control loop has to abort and the system, e.g., the 639 production line, may need to enter an emergency stop condition. 641 Engineers that build automated processes may use the network 642 reliability expressed in nines or as an MTBF as a proxy to indicate 643 an MCF, e.g., as described in section 7.4 of the "Deterministic 644 Networking Use Cases" [RFC8578]. 646 2.2.3. Reliability in the Context of RAW 648 In contrast with wired networks, errors in transmission are the 649 predominant source of packet loss in wireless networks. 651 The root cause for the loss may be of multiple origins, calling for 652 the use of different forms of diversity: 654 Multipath Fading A destructive interference by a reflection of the 655 original signal. 657 A radio signal may be received directly (line-of-sight) and/or as 658 a reflection on a physical structure (echo). The reflections take 659 a longer path and are delayed by the extra distance divided by the 660 speed of light in the medium. Depending on the frequency, the 661 echo lands with a different phase which may add up to 662 (constructive interference) or cancel the direct signal 663 (destructive interference). 665 The affected frequencies depend on the relative position of the 666 sender, the receiver, and all the reflecting objects in the 667 environment. A given hop will suffer from multipath fading for 668 multiple packets in a row till the something moves that changes 669 the reflection patterns. 671 Co-channel Interference Energy in the spectrum used for the 672 transmission confuses the receiver. 674 The wireless medium itself is a Shared Risk Link Group (SRLG) for 675 nearby users of the same spectrum, as an interference may affect 676 multiple co-channel transmissions between different peers within 677 the interference domain of the interferer, possibly even when they 678 use different technologies. 680 Obstacle in Fresnel Zone The optimal transmission happens when the 681 Fresnel Zone between the sender and the receiver is free of 682 obstacles. 684 As long as a physical object (e.g., a metallic trolley between 685 peers) that affects the transmission is not removed, the quality 686 of the link is affected. 688 In an environment that is rich of metallic structures and mobile 689 objects, a single radio link will provide a fuzzy service, meaning 690 that it cannot be trusted to transport the traffic reliably over a 691 long period of time. 693 Transmission losses are typically not independent, and their nature 694 and duration are unpredictable; as long as a physical object (e.g., a 695 metallic trolley between peers) that affects the transmission is not 696 removed, or as long as the interferer (e.g., a radar) keeps 697 transmitting, a continuous stream of packets will be affected. 699 The key technique to combat those unpredictable losses is diversity. 700 Different forms of diversity are necessary to combat different causes 701 of loss and the use of diversity must be maximized to optimize the 702 PDR. 704 A single packet may be sent at different times (time diversity) over 705 diverse paths (spatial diversity) that rely on diverse radio channels 706 (frequency diversity) and diverse PHY technologies, e.g., narrowband 707 vs. spread spectrum, or diverse codes. Using time diversity will 708 defeat short-term interferences; spatial diversity combats very local 709 causes such as multipath fading; narrowband and spread spectrum are 710 relatively innocuous to one another and can be used for diversity in 711 the presence of the other. 713 2.3. Routing Time Scale vs. Forwarding Time Scale 715 With DetNet, the Controller Plane Function that handles the routing 716 computation and maintenance (the PCE) can be centralized and can 717 reside outside the network. In a wireless mesh, the path to the PCE 718 can be expensive and slow, possibly going across the whole mesh and 719 back. Reaching to the PCE can also be slow in regards to the speed 720 of events that affect the forwarding operation at the radio layer. 722 Due to that cost and latency, the Controller Plane is not expected to 723 be sensitive/reactive to transient changes. The abstraction of a 724 link at the routing level is expected to use statistical metrics that 725 aggregate the behavior of a link over long periods of time, and 726 represent its properties as shades of gray as opposed to numerical 727 values such as a link quality indicator, or a boolean value for 728 either up or down. 730 +----------------+ 731 | Controller | 732 | [PCE] | 733 +----------------+ 734 ^ 735 | 736 Slow 737 | 738 _-._-._-._-._-._-. | ._-._-._-._-._-._-._-._-._-._-._-._- 739 _-._-._-._-._-._-._-. | _-._-._-._-._-._-._-._-._-._-._-._- 740 | 741 Expensive 742 | 743 .... | ....... 744 .... . | . ....... 745 .... v ... 746 .. A-------B-------C---D .. 747 ... / \ / / \ .. 748 . I ----M-------N--***-- E .. 749 .. \ \ / / ... 750 .. P--***--Q----------R .... 751 .. .... 752 . <----- Fast -------> .... 753 ....... .... 754 ................. 756 *** = flapping at this time 758 Figure 1: Time Scales 760 In the case of wireless, the changes that affect the forwarding 761 decision can happen frequently and often for short durations, e.g., a 762 mobile object moves between a transmitter and a receiver, and will 763 cancel the line of sight transmission for a few seconds, or a radar 764 measures the depth of a pool and interferes on a particular channel 765 for a split second. 767 There is thus a desire to separate the long term computation of the 768 route and the short term forwarding decision. In that model, the 769 routing operation computes a complex Track that enables multiple Non- 770 Equal Cost Multi-Path (N-ECMP) forwarding solutions, and leaves it to 771 the Data Plane to make the per-packet decision of which of these 772 possibilities should be used. 774 In the wired world, and more specifically in the context of Traffic 775 Engineering (TE), an alternate path can be used upon the detection of 776 a failure in the main path, e.g., using OAM in MPLS-TP or BFD over a 777 collection of SD-WAN tunnels. RAW formalizes a forwarding time scale 778 that is an order(s) of magnitude shorter than the controller plane 779 routing time scale, and separates the protocols and metrics that are 780 used at both scales. Routing can operate on long term statistics 781 such as delivery ratio over minutes to hours, but as a first 782 approximation can ignore flapping. On the other hand, the RAW 783 forwarding decision is made at the scale of the packet rate, and uses 784 information that must be pertinent at the present time for the 785 current transmission(s). 787 3. The RAW Conceptual Model 789 RAW inherits the conceptual model described in section 4 of the 790 DetNet Architecture [RFC8655]. RAW extends the DetNet service layer 791 to provide additional agility against transmission loss. 793 A RAW Network Plane may be strict or loose, depending on whether RAW 794 observes and takes actions on all hops or not. For instance, the 795 packets between two wireless entities may be relayed over a wired 796 infrastructure such as a Wi-Fi extended service set (ESS) or a 5G 797 Core; in that case, RAW observes and control the transmission over 798 the wireless first and last hops, as well as end-to-end metrics such 799 as latency, jitter, and delivery ratio. This operation is loose 800 since the structure and properties of the wired infrastructure are 801 ignored, and may be either controlled by other means such as DetNet/ 802 TSN, or neglected in the face of the wireless hops. 804 A Controller Plane Function (CPF) called the Path Computation Element 805 (PCE) [RFC4655] interacts with RAW Nodes over a Southbound API. The 806 RAW Nodes are DetNet relays that are capable of additional diversity 807 mechanisms and measurement functions related to the radio interface, 808 in particular the PAREO diversity mechanisms. 810 The PCE defines a complex Track between an Ingress End System and an 811 Egress End System, and indicates to the RAW Nodes where the PAREO 812 operations may be actionned in the Network Plane. The Track may be 813 expressed loosely to enable traversing a non-RAW subnetwork. In that 814 case, the expectation is that the non-RAW subnetwork can be neglected 815 in the RAW computation, that is, considered infinitely fast, reliable 816 and/or available in comparison with the links between RAW nodes. 818 CPF CPF CPF CPF 820 Southbound API 821 _-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._- 822 _-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._- 824 RAW --z RAW --z RAW --z RAW 825 z-- Node z-- Node z-- Node z-- Node --z 826 Ingress --z / / z-- Egress 827 End \ \ .. . End 828 Node ---z / / .. .. . z-- Node 829 z-- RAW --z RAW ( non-RAW ) -- RAW --z 830 Node z-- Node --- ( Nodes ) Node 831 ... . 832 --z wireless wired 833 z-- link --- link 835 Figure 2: RAW Nodes 837 The Link-Layer metrics are reported to the PCE in a time-aggregated, 838 e.g., statistical fashion. Example Link-Layer metrics include 839 typical Link bandwidth (the medium speed depends dynamically on the 840 PHY mode and the number of users sharing the spectrum) and average 841 and mean squared deviation of availability and reliability figures 842 such as Packet Delivery Ratio (PDR) over long periods of time. 844 Based on those metrics, the PCE installs the Track with enough 845 redundant forwarding solutions to ensure that the Network Plane can 846 reliably deliver the packets within a System Level Agreement (SLA) 847 associated to the flows that it transports. The SLA defines end-to- 848 end reliability and availability requirements, where reliability may 849 be expressed as a successful delivery in order and within a bounded 850 delay of at least one copy of a packet. 852 Depending on the use case and the SLA, the Track may comprise non-RAW 853 segments, either interleaved inside the Track, or all the way to the 854 Egress End Node (e.g., a server in the Internet). RAW observes the 855 Lower-Layer Links between RAW nodes (typically, radio links) and the 856 end-to-end Network Layer operation to decide at all times which of 857 the PAREO diversity schemes is actioned by which RAW Nodes. 859 Once a Track is established, per-segment and end-to-end reliability 860 and availability statistics are periodically reported to the PCE to 861 assure that the SLA can be met or have it recompute the Track if not. 863 4. The OODA Loop 865 The RAW Architecture is structured as an OODA Loop (Observe, Orient, 866 Decide, Act). It involves: 868 1. Network Plane measurement protocols for Operations, 869 Administration and Maintenance (OAM) to Observe some or all hops 870 along a Track as well as the end-to-end packet delivery, more in 871 Section 5; 873 2. Controller plane elements to reports the links statistics to a 874 Path computation Element (PCE) in a centralized controller that 875 computes and installs the Tracks and provides meta data to Orient 876 the routing decision, more in Section 6; 878 3. A Runtime distributed Path Selection Engine (PSE) thar Decides 879 which subTrack to use for the next packet(s) that are routed 880 along the Track, more in Section 7; 882 4. Packet (hybrid) ARQ, Replication, Elimination and Ordering 883 Dataplane actions that operate at the DetNet Service Layer to 884 increase the reliability o fthe end-to-end transmission. The RAW 885 architecture also covers in-situ signalling when the decision is 886 Acted by a node that down the Track from the PSE, more in 887 Section 8. 889 +-------> Orient (PCE) --------+ 890 | link stats, | 891 | pre-trained model | 892 | ... | 893 | | 894 | v 895 Observe (OAM) Decide (PSE) 896 ^ | 897 | | 898 | | 899 +-------- Act (PAREO) <--------+ 900 At DetNet 901 Service sublayer 903 Figure 3: The RAW OODA Loop 905 The overall OODA Loop optimizes the use of redundancy to achieve the 906 required reliability and availability Service Level Agreement (SLA) 907 while minimizing the use of constrained resources such as spectrum 908 and battery. 910 5. Observe: The RAW OAM 912 RAW In-situ OAM operation in the Network Plane may observe either a 913 full Track or subTracks that are being used at this time. Active RAW 914 OAM may be needed to observe the unused segments and evaluate the 915 desirability of a rerouting decision. Finally, the RAW Service Layer 916 Assurance may observe the individual PAREO operation of a relay node 917 to ensure that it is conforming; this might require injecting an OAM 918 packet at an upstream point inside the Track and extracting that 919 packet at another point downstream before it reaches the egress. 921 This observation feeds the RAW PSE that makes the decision on which 922 PAREO function in actioned at which RAW Node, for one a small 923 continuous series of packets. 925 ... .. 926 RAN 1 ----- ... .. ... 927 / . .. .... 928 +-------+ / . .. .... +------+ 929 |Ingress|- . ..... |Egress| 930 | End |------ RAN 2 -- . Internet ....---| End | 931 |System |- .. ..... |System| 932 +-------+ \ . ...... +------+ 933 \ ... ... ..... 934 RAN n -------- ... ..... 936 <------------------> <--------------------> 937 Observed by OAM Opaque to OAM 939 Figure 4: Observed Links in Radio Access Protection 941 In the case of a End-to-End Protection in a Wireless Mesh, the Track 942 is strict and congruent with the path so all links are observed. 943 Conversely, in the case of Radio Access Protection, the Track is 944 Loose and in that case only the first hop is observed; the rest of 945 the path is abstracted and considered infinitely reliable. 947 In the case of the Radio Access Protection, only the first hop is 948 protected; the loss of a packet that was sent over one of the 949 possible first hops is attributed to that first hop, even if a 950 particular loss effectively happens farther down the path. 952 The Links that are not observed by OAM are opaque to it, meaning that 953 the OAM information is carried across and possibly echoed as data, 954 but there is no information capture in intermediate nodes. In the 955 example above, the Internet is opaque and not controlled by RAW; 956 still the RAW OAM measures the end-to-end latency and delivery ratio 957 for packets sent via each if RAN 1, RAN 2 and RAN 3, and determines 958 whether a packet should be sent over either or a collection of those 959 access links. 961 6. Orient: The Path Computation Engine 963 RAW separates the path computation time scale at which a complex path 964 is recomputed from the path selection time scale at which the 965 forwarding decision is taken for one or a few packets (see in 966 Section 2.3). 968 The path computation is out of scope, but RAW expects that the 969 Controller plane protocol that installs the Track also provides 970 related knowledge in the form of meta data about the links, segments 971 and possible subTracks. That meta data can be a pre-digested 972 statistical model, and may include prediction of future flaps and 973 packet loss, as well as recommended actions when that happens. 975 The meta data may include: 977 * Pre-Determined subTracks to match predictable error profiles 979 * Pre-Trained models 981 * Link Quality Statistics and their projected evolution 983 The Track is installed with measurable objectives that are computed 984 by the PCE to achieve the RAW SLA. The objectives can be expressed 985 as any of maximum number of packet lost in a row, bounded latency, 986 maximal jitter, maximum nmuber of interleaved out of order packets, 987 average number of copies received at the elimination point, and 988 maximal delay between the first and the last received copy of the 989 same packet. 991 7. Decide: The Path Selection Engine 993 The RAW OODA Loop operates at the path selection time scale to 994 provide agility vs. the brute force approach of flooding the whole 995 Track. The OODA Loop controls, within the redundant solutions that 996 are proposed by the PCE, which will be used for each packet to 997 provide a Reliable and Available service while minimizing the waste 998 of constrained resources. 1000 To that effect, RAW defines the Path Selection Engine (PSE) that is 1001 the counterpart of the PCE to perform rapid local adjustments of the 1002 forwarding tables within the diversity that the PCE has selected for 1003 the Track. The PSE enables to exploit the richer forwarding 1004 capabilities with PAREO and scheduled transmissions at a faster time 1005 scale over the smaller domain that is the Track, in either a loose or 1006 a strict fashion. 1008 Compared to the PCE, the PSE operates on metrics that evolve faster, 1009 but that needs to be advertised at a fast rate but only locally, 1010 within the Track. The forwarding decision may also change rapidly, 1011 but with a scope that is also contained within the Track, with no 1012 visibility to the other Tracks and flows in the network. This is as 1013 opposed to the PCE that needs to observe the whole network, and 1014 optimize all the Tracks globally, which can only be done at a slow 1015 pace and using long-term statistical metrics, as presented in 1016 Table 1. 1018 +===============+========================+===================+ 1019 | | PCE (Not in Scope) | PSE (In Scope) | 1020 +===============+========================+===================+ 1021 | Operation | Centralized | Source-Routed or | 1022 | | | Distributed | 1023 +---------------+------------------------+-------------------+ 1024 | Communication | Slow, expensive | Fast, local | 1025 +---------------+------------------------+-------------------+ 1026 | Time Scale | hours and above | seconds and below | 1027 +---------------+------------------------+-------------------+ 1028 | Network Size | Large, many Tracks to | Small, within one | 1029 | | optimize globally | Track | 1030 +---------------+------------------------+-------------------+ 1031 | Considered | Averaged, Statistical, | Instant values / | 1032 | Metrics | Shade of grey | boolean condition | 1033 +---------------+------------------------+-------------------+ 1035 Table 1: PCE vs. PSE 1037 The PSE sits in the DetNet Service sub-Layer of Edge and Relay Nodes. 1038 On the one hand, it operates on the packet flow, learning the Track 1039 and path selection information from the packet, possibly making local 1040 decision and retagging the packet to indicate so. On the other hand, 1041 the PSE interacts with the lower layers and with its peers to obtain 1042 up-to-date information about its radio links and the quality of the 1043 overall Track, respectively, as illustrated in Figure 5. 1045 | 1046 packet | going 1047 down the | stack 1048 +==========v==========+=====================+=====================+ 1049 | (iOAM + iCTRL) | (L2 Triggers, DLEP) | (oOAM) | 1050 +==========v==========+=====================+=====================+ 1051 | Learn from Learn from | 1052 | packet tagging Maintain end-to-end | 1053 +----------v----------+ Forwarding OAM packets | 1054 | Forwarding decision < State +---------^-----------| 1055 +----------v----------+ | Enrich or | 1056 + Retag Packet | Learn abstracted > Regenerate | 1057 | and Forward | metrics about Links | OAM packets | 1058 +..........v..........+..........^..........+.........^.v.........+ 1059 | Lower layers | 1060 +..........v.....................^....................^.v.........+ 1061 frame | sent Frame | L2 Ack oOAM | | packet 1062 over | wireless In | In | | and out 1063 v | | v 1065 Figure 5: PSE 1067 8. Act: The PAREO Functions 1069 RAW may control whether and how to use packet replication and 1070 elimination (PRE), Automatic Repeat reQuest (ARQ), Hybrid ARQ (HARQ) 1071 that includes Forward Error Correction (FEC) and coding, and other 1072 wireless-specific techniques such as overhearing and constructive 1073 interferences, in order to increase the reliabiility and availability 1074 of the end-to-end transmission. 1076 Collectively, those function are called PAREO for Packet (hybrid) 1077 ARQ, Replication, Elimination and Ordering. By tuning dynamically 1078 the use of PAREO functions, RAW avoids the waste of critical 1079 resources such as spectrum and energy while providing that the 1080 guaranteed SLA, e.g., by adding redundancy only when a spike of loss 1081 is observed. 1083 In a nutshell, PAREO establishes several paths in a network to 1084 provide redundancy and parallel transmissions to bound the end-to-end 1085 delay to traverse the network. Optionally, promiscuous listening 1086 between paths is possible, such that the Nodes on one path may 1087 overhear transmissions along the other path. Considering the 1088 scenario shown in Figure 6, many different paths are possible for to 1089 traverse the network from ingress to egress. A simple way to benefit 1090 from this topology could be to use the two independent paths via 1091 Nodes A, C, E and via B, D, F. But more complex paths are possible 1092 by interleaving transmissions from the lower level of the path to the 1093 upper level. 1095 (A) -- (C) -- (E) 1096 / \ 1097 Ingress = | | | = Egress 1098 \ / 1099 (B) -- (D) -- (F) 1101 Figure 6: A Ladder Shape with Two Parallel Paths 1103 PAREO may also take advantage of the shared properties of the 1104 wireless medium to compensate for the potential loss that is incurred 1105 with radio transmissions. 1107 For instance, when the source sends to Node A, Node B may listen 1108 promiscuously and get a second chance to receive the frame without an 1109 additional transmission. Note that B would not have to listen if it 1110 already received that particular frame at an earlier timeslot in a 1111 dedicated transmission towards B. 1113 The PAREO model can be implemented in both centralized and 1114 distributed scheduling approaches. In the centralized approach, a 1115 Path Computation Element (PCE) scheduler calculates a Track and 1116 schedules the communication. In the distributed approach, the Track 1117 is computed within the network, and signaled in the packets, e.g., 1118 using BIER-TE, Segment Routing, or a Source Routing Header. 1120 8.1. Packet Replication 1122 By employing a Packet Replication procedure, a Node forwards a copy 1123 of each data packet to more than one successor. To do so, each Node 1124 (i.e., Ingress and intermediate Node) sends the data packet multiple 1125 times as separate unicast transmissions. For instance, in Figure 7, 1126 the Ingress Node is transmitting the packet to both successors, nodes 1127 A and B, at two different times. 1129 ===> (A) => (C) => (E) === 1130 // \\// \\// \\ 1131 Ingress //\\ //\\ Egress 1132 \\ // \\ // \\ // 1133 ===> (B) => (D) => (F) === 1135 Figure 7: Packet Replication 1137 An example schedule is shown in Table 2. This way, the transmission 1138 leverages with the time and spatial forms of diversity. 1140 +=========+======+======+======+======+======+======+======+ 1141 | Channel | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 1142 +=========+======+======+======+======+======+======+======+ 1143 | 0 | S->A | S->B | B->C | B->D | C->F | E->R | F->R | 1144 +---------+------+------+------+------+------+------+------+ 1145 | 1 | | A->C | A->D | C->E | D->E | D->F | | 1146 +---------+------+------+------+------+------+------+------+ 1148 Table 2: Packet Replication: Sample schedule 1150 8.2. Packet Elimination 1152 The replication operation increases the traffic load in the network, 1153 due to packet duplications. This may occur at several stages inside 1154 the Track, and to avoid an explosion of the number of copies, a 1155 Packet Elimination procedure must be applied as well. To this aim, 1156 once a Node receives the first copy of a data packet, it discards the 1157 subsequent copies. 1159 The logical functions of Replication and Elimination may be 1160 collocated in an intermediate Node, the Node first eliminating the 1161 redundant copies and then sending the packet exactly once to each of 1162 the selected successors. 1164 8.3. Promiscuous Overhearing 1166 Considering that the wireless medium is broadcast by nature, any 1167 neighbor of a transmitter may overhear a transmission. By employing 1168 the Promiscuous Overhearing operation, the next hops have additional 1169 opportunities to capture the data packets. In Figure 8, when Node A 1170 is transmitting to its DP (Node C), the AP (Node D) and its sibling 1171 (Node B) may decode this data packet as well. As a result, by 1172 employing corellated paths, a Node may have multiple opportunities to 1173 receive a given data packet. 1175 ===> (A) ====> (C) ====> (E) ==== 1176 // ^ | \\ \\ 1177 Ingress | | \\ Egress 1178 \\ | v \\ // 1179 ===> (B) ====> (D) ====> (F) ==== 1181 Figure 8: Unicast with Overhearing 1183 8.4. Constructive Interference 1185 Constructive Interference can be seen as the reverse of Promiscuous 1186 Overhearing, and refers to the case where two senders transmit the 1187 exact same signal in a fashion that the emitted symbols add up at the 1188 receiver and permit a reception that would not be possible with a 1189 single sender at the same PHY mode and the same power level. 1191 Constructive Interference was proposed on 5G, Wi-Fi7 and even tested 1192 on IEEE Std 802.14.5. The hard piece is to synchronize the senders 1193 to the point that the signals are emitted at slightly different time 1194 to offset the difference of propagation delay that corresponds to the 1195 difference of distance of the transmitters to the receiver at the 1196 speed of light to the point that the symbols are superposed long 1197 enough to be recognizable. 1199 9. Security Considerations 1201 RAW uses all forms of diversity including radio technology and 1202 physical path to increase the reliability and availability in the 1203 face of unpredictable conditions. While this is not done 1204 specifically to defeat an attacker, the amount of diversity used in 1205 RAW makes an attack harder to achieve. 1207 9.1. Forced Access 1209 RAW will typically select the cheapest collection of links that 1210 matches the requested SLA, for instance, leverage free WI-Fi vs. paid 1211 3GPP access. By defeating the cheap connectivity (e.g., PHY-layer 1212 interference) the attacker can force an End System to use the paid 1213 access and increase the cost of the transmission for the user. 1215 10. IANA Considerations 1217 This document has no IANA actions. 1219 11. Contributors 1221 The editor wishes to thank: 1223 Xavi Vilajosana: Wireless Networks Research Lab, Universitat Oberta 1224 de Catalunya 1226 Remous-Aris Koutsiamanis: IMT Atlantique 1228 Nicolas Montavont: IMT Atlantique 1230 Rex Buddenberg: Individual contributor 1232 Greg Mirsky: ZTE 1234 for their contributions to the text and ideas exposed in this 1235 document. 1237 12. Acknowledgments 1239 TBD 1241 13. References 1243 13.1. Normative References 1245 [6TiSCH-ARCHI] 1246 Thubert, P., Ed., "An Architecture for IPv6 over the Time- 1247 Slotted Channel Hopping Mode of IEEE 802.15.4 (6TiSCH)", 1248 RFC 9030, DOI 10.17487/RFC9030, May 2021, 1249 . 1251 [INT-ARCHI] 1252 Braden, R., Ed., "Requirements for Internet Hosts - 1253 Communication Layers", STD 3, RFC 1122, 1254 DOI 10.17487/RFC1122, October 1989, 1255 . 1257 [RAW-TECHNOS] 1258 Thubert, P., Cavalcanti, D., Vilajosana, X., Schmitt, C., 1259 and J. Farkas, "Reliable and Available Wireless 1260 Technologies", Work in Progress, Internet-Draft, draft- 1261 ietf-raw-technologies-04, 3 August 2021, 1262 . 1265 [RAW-USE-CASES] 1266 Papadopoulos, G. Z., Thubert, P., Theoleyre, F., and C. J. 1267 Bernardos, "RAW use cases", Work in Progress, Internet- 1268 Draft, draft-ietf-raw-use-cases-03, 20 October 2021, 1269 . 1272 [RFC4655] Farrel, A., Vasseur, J.-P., and J. Ash, "A Path 1273 Computation Element (PCE)-Based Architecture", RFC 4655, 1274 DOI 10.17487/RFC4655, August 2006, 1275 . 1277 [BFD] Katz, D. and D. Ward, "Bidirectional Forwarding Detection 1278 (BFD)", RFC 5880, DOI 10.17487/RFC5880, June 2010, 1279 . 1281 [RFC6291] Andersson, L., van Helvoort, H., Bonica, R., Romascanu, 1282 D., and S. Mansfield, "Guidelines for the Use of the "OAM" 1283 Acronym in the IETF", BCP 161, RFC 6291, 1284 DOI 10.17487/RFC6291, June 2011, 1285 . 1287 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 1288 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 1289 May 2016, . 1291 [RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases", 1292 RFC 8578, DOI 10.17487/RFC8578, May 2019, 1293 . 1295 [IPv6] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1296 (IPv6) Specification", STD 86, RFC 8200, 1297 DOI 10.17487/RFC8200, July 2017, 1298 . 1300 [RFC8557] Finn, N. and P. Thubert, "Deterministic Networking Problem 1301 Statement", RFC 8557, DOI 10.17487/RFC8557, May 2019, 1302 . 1304 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 1305 "Deterministic Networking Architecture", RFC 8655, 1306 DOI 10.17487/RFC8655, October 2019, 1307 . 1309 [RFC9049] Dawkins, S., Ed., "Path Aware Networking: Obstacles to 1310 Deployment (A Bestiary of Roads Not Taken)", RFC 9049, 1311 DOI 10.17487/RFC9049, June 2021, 1312 . 1314 13.2. Informative References 1316 [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791, 1317 DOI 10.17487/RFC0791, September 1981, 1318 . 1320 [TE] Awduche, D., Chiu, A., Elwalid, A., Widjaja, I., and X. 1321 Xiao, "Overview and Principles of Internet Traffic 1322 Engineering", RFC 3272, DOI 10.17487/RFC3272, May 2002, 1323 . 1325 [STD 62] Harrington, D., Presuhn, R., and B. Wijnen, "An 1326 Architecture for Describing Simple Network Management 1327 Protocol (SNMP) Management Frameworks", STD 62, RFC 3411, 1328 DOI 10.17487/RFC3411, December 2002, 1329 . 1331 [RFC4090] Pan, P., Ed., Swallow, G., Ed., and A. Atlas, Ed., "Fast 1332 Reroute Extensions to RSVP-TE for LSP Tunnels", RFC 4090, 1333 DOI 10.17487/RFC4090, May 2005, 1334 . 1336 [FRR] Shand, M. and S. Bryant, "IP Fast Reroute Framework", 1337 RFC 5714, DOI 10.17487/RFC5714, January 2010, 1338 . 1340 [RLFA-FRR] Bryant, S., Filsfils, C., Previdi, S., Shand, M., and N. 1341 So, "Remote Loop-Free Alternate (LFA) Fast Reroute (FRR)", 1342 RFC 7490, DOI 10.17487/RFC7490, April 2015, 1343 . 1345 [DetNet-DP] 1346 Varga, B., Ed., Farkas, J., Berger, L., Malis, A., and S. 1347 Bryant, "Deterministic Networking (DetNet) Data Plane 1348 Framework", RFC 8938, DOI 10.17487/RFC8938, November 2020, 1349 . 1351 [I-D.irtf-panrg-path-properties] 1352 Enghardt, T. and C. Kraehenbuehl, "A Vocabulary of Path 1353 Properties", Work in Progress, Internet-Draft, draft-irtf- 1354 panrg-path-properties-04, 25 October 2021, 1355 . 1358 [IPoWIRELESS] 1359 Thubert, P., "IPv6 Neighbor Discovery on Wireless 1360 Networks", Work in Progress, Internet-Draft, draft- 1361 thubert-6man-ipv6-over-wireless-10, 18 November 2021, 1362 . 1365 [DetNet-OAM] 1366 Mirsky, G., Theoleyre, F., Papadopoulos, G. Z., Bernardos, 1367 C. J., Varga, B., and J. Farkas, "Framework of Operations, 1368 Administration and Maintenance (OAM) for Deterministic 1369 Networking (DetNet)", Work in Progress, Internet-Draft, 1370 draft-ietf-detnet-oam-framework-05, 14 October 2021, 1371 . 1374 [NASA] Adams, T., "RELIABILITY: Definition & Quantitative 1375 Illustration", . 1378 Authors' Addresses 1380 Pascal Thubert (editor) 1381 Cisco Systems, Inc 1382 Building D 1383 45 Allee des Ormes - BP1200 1384 06254 MOUGINS - Sophia Antipolis 1385 France 1387 Phone: +33 497 23 26 34 1388 Email: pthubert@cisco.com 1390 Georgios Z. Papadopoulos 1391 IMT Atlantique 1392 Office B00 - 114A 1393 2 Rue de la Chataigneraie 1394 35510 Cesson-Sevigne - Rennes 1395 France 1397 Phone: +33 299 12 70 04 1398 Email: georgios.papadopoulos@imt-atlantique.fr