idnits 2.17.1 draft-pthubert-raw-architecture-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 886 has weird spacing: '...-- Node z-- ...' == Line 891 has weird spacing: '... Node z-- ...' -- The document date (15 November 2020) is 1257 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'PCE' is mentioned on line 637, but not defined == Outdated reference: A later version (-30) exists of draft-ietf-6tisch-architecture-29 -- Obsolete informational reference (is this intentional?): RFC 3272 (ref. 'TE') (Obsoleted by RFC 9522) == Outdated reference: A later version (-13) exists of draft-ietf-bier-te-arch-09 Summary: 0 errors (**), 0 flaws (~~), 6 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 RAW P. Thubert, Ed. 3 Internet-Draft Cisco Systems 4 Intended status: Informational G.Z. Papadopoulos 5 Expires: 19 May 2021 IMT Atlantique 6 R. Buddenberg 7 15 November 2020 9 Reliable and Available Wireless Architecture/Framework 10 draft-pthubert-raw-architecture-05 12 Abstract 14 Due to uncontrolled interferences, including the self-induced 15 multipath fading, deterministic networking can only be approached on 16 wireless links. The radio conditions may change -way- faster than a 17 centralized routing can adapt and reprogram, in particular when the 18 controller is distant and connectivity is slow and limited. RAW 19 separates the routing time scale at which a complex path is 20 recomputed from the forwarding time scale at which the forwarding 21 decision is taken for an individual packet. RAW operates at the 22 forwarding time scale. The RAW problem is to decide, within the 23 redundant solutions that are proposed by the routing, which will be 24 used for each individual packet to provide a DetNet service while 25 minimizing the waste of resources. 27 Status of This Memo 29 This Internet-Draft is submitted in full conformance with the 30 provisions of BCP 78 and BCP 79. 32 Internet-Drafts are working documents of the Internet Engineering 33 Task Force (IETF). Note that other groups may also distribute 34 working documents as Internet-Drafts. The list of current Internet- 35 Drafts is at https://datatracker.ietf.org/drafts/current/. 37 Internet-Drafts are draft documents valid for a maximum of six months 38 and may be updated, replaced, or obsoleted by other documents at any 39 time. It is inappropriate to use Internet-Drafts as reference 40 material or to cite them other than as "work in progress." 42 This Internet-Draft will expire on 19 May 2021. 44 Copyright Notice 46 Copyright (c) 2020 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 51 license-info) in effect on the date of publication of this document. 52 Please review these documents carefully, as they describe your rights 53 and restrictions with respect to this document. Code Components 54 extracted from this document must include Simplified BSD License text 55 as described in Section 4.e of the Trust Legal Provisions and are 56 provided without warranty as described in the Simplified BSD License. 58 Table of Contents 60 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 61 2. The RAW problem . . . . . . . . . . . . . . . . . . . . . . . 5 62 2.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 5 63 2.2. Reliability and Availability . . . . . . . . . . . . . . 6 64 2.2.1. High Availability Engineering Principles . . . . . . 6 65 2.2.2. Applying Reliability Concepts to Networking . . . . . 8 66 2.2.3. Reliability in the Context of RAW . . . . . . . . . . 9 67 2.3. Use Cases and Requirements Served . . . . . . . . . . . . 10 68 2.3.1. Radio Access Protection . . . . . . . . . . . . . . . 11 69 2.3.2. End-to-End Protection in a Wireless Mesh . . . . . . 11 70 2.4. Related Work at The IETF . . . . . . . . . . . . . . . . 12 71 3. The RAW Framework . . . . . . . . . . . . . . . . . . . . . . 13 72 3.1. Scope and Prerequisites . . . . . . . . . . . . . . . . . 13 73 3.2. Routing Time Scale vs. Forwarding Time Scale . . . . . . 14 74 3.3. Wireless Tracks . . . . . . . . . . . . . . . . . . . . . 15 75 3.4. PAREO Functions . . . . . . . . . . . . . . . . . . . . . 16 76 3.4.1. Packet Replication . . . . . . . . . . . . . . . . . 17 77 3.4.2. Packet Elimination . . . . . . . . . . . . . . . . . 18 78 3.4.3. Promiscuous Overhearing . . . . . . . . . . . . . . . 18 79 3.4.4. Constructive Interference . . . . . . . . . . . . . . 18 80 4. The RAW Architecture . . . . . . . . . . . . . . . . . . . . 19 81 4.1. The RAW Conceptual Model . . . . . . . . . . . . . . . . 19 82 4.2. The Path Selection Engine . . . . . . . . . . . . . . . . 21 83 4.3. RAW OAM . . . . . . . . . . . . . . . . . . . . . . . . . 22 84 4.4. Flow Identification vs. Path Identification . . . . . . . 23 85 4.5. Source-Routed vs. Distributed Forwarding Decision . . . . 26 86 4.6. Encapsulation and Decapsulation . . . . . . . . . . . . . 27 87 5. Security Considerations . . . . . . . . . . . . . . . . . . . 27 88 5.1. Forced Access . . . . . . . . . . . . . . . . . . . . . . 27 89 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 27 90 7. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 27 91 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 28 92 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 28 93 9.1. Normative References . . . . . . . . . . . . . . . . . . 28 94 9.2. Informative References . . . . . . . . . . . . . . . . . 29 95 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 31 97 1. Introduction 99 Bringing determinism in a packet network means eliminating the 100 statistical effects of multiplexing that result in probabilistic 101 jitter and loss. This can be approached with a tight control of the 102 physical resources to maintain the amount of traffic within a 103 budgetted volume of data per unit of time that fits the physical 104 capabilities of the underlying technology, and the use of time-shared 105 resources (bandwidth and buffers) per circuit, and/or by shaping and/ 106 or scheduling the packets at every hop. 108 Wireless networks operate on a shared medium where uncontrolled 109 interference, including the self-induced multipath fading, cause 110 random transmission losses and add new dimensions to the statistical 111 effects that affect the delivery. Scheduling can alleviate those 112 effects by enabling diverse transmissions in the spatial, time, code, 113 and frequency domains. Reliable and Available Wireless (RAW) 114 leverages scheduling and all possible forms of diversity to defeat 115 the possible causes of transmission loss while preserving energy and 116 optimizing the use of the shared spectrum. 118 Deterministic Networking is an attempt to emulate the properties of a 119 serial link over a switched fabric, by providing a bounded latency 120 and eliminating congestion loss, even when co-existing with best- 121 effort traffic. This innovation was introduced on wired networks 122 with IEEE 802.1 TSN (for Ethernet LANs) and IETF DetNet. It is 123 getting traction in various industries including professional A/V, 124 manufacturing, online gaming, and smartgrid automation, enabling cost 125 and performance optimizations (e.g., vs. loads of P2P cables). 127 The wireless and wired media are fundamentally different at the 128 physical level, and while the generic "Deterministic Networking 129 Problem Statement" [RFC8557] applies to both the wired and the 130 wireless media, the methods to achieve RAW must extend those used to 131 support time-sensitive networking over wires, as a RAW solution has 132 to address less consistent transmissions, energy conservation and 133 shared spectrum efficiency. 135 Uncontrolled interference and transmission obstacles may impede the 136 wireless transmission, causing rapid variations of the throughput and 137 packet delivery ratio (PDR) of the link. This uncertainty limits the 138 volume and/or duration of traffic that can be safely transmitted on 139 the same link while conforming to a RAW Service Level Agreement 140 (SLA). Techniques such as beamforming with Multi-User MIMO can only 141 alleviate some of those issues, and the term deterministic is usually 142 not associated with a short range radio link, in particular one 143 operated in the ISM band. 145 This increased complexity explains why the development of 146 deterministic wireless technologies has been lagging behind the 147 similar efforts for wired systems, both at the IEEE and the IETF. 148 But recent progress on scheduled radios such as TSCH and OFDMA 149 indicates that wireless is finally catching up at the lower layers. 150 Sitting at the layer above, RAW takes up the challenge of providing 151 highly available and reliable end-to-end performances in a network 152 with scheduled wireless segments. 154 RAW provides DetNet elements that are specialized for short range 155 radios. From this inheritance, RAW stays agnostic to the radio layer 156 underneath though the capability to schedule transmissions is 157 assumed. How the PHY is programmed to do so, and whether the radio 158 is single-hop or meshed, are unknown at the IP layer and not part of 159 the RAW abstraction. 161 The "Deterministic Networking Architecture" [RFC8655] is composed of 162 three planes: the Application (User) Plane, the Controller Plane, and 163 the Network Plane. The RAW Architecture extends the DetNet Network 164 Plane, to accomodate one or multiple hops of homogeneous or 165 heterogeneous wireless technologies, e.g. a Wi-Fi6 Mesh or parallel 166 CBRS access links federated by a 5G backhaul. 168 The establishment of a path is not in-scope for RAW. It may be the 169 product of a centralized Controller Plane as described for DetNet. 170 As opposed to wired networks, the action of installing a path over a 171 set of wireless links may be very slow relative to the speed at which 172 the radio conditions vary, and it makes sense in the wireless case to 173 provide redundant forwarding solutions along a complex path and to 174 leave it to the Network Plane to select which of those forwarding 175 solutions are to be used for a given packet based on the current 176 conditions. 178 RAW distinguishes the longer time scale at which routes are computed 179 from the the shorter forwarding time scale where per-packet decisions 180 are made. RAW operates within the Network Plane at the forwarding 181 time scale on one DetNet flow over a complex path called a Track. 182 The Track is preestablished and installed by means outside of the 183 scope of RAW; it may be strict or loose depending on whether each or 184 just a subset of the hops are observed and controlled by RAW. 186 The RAW Architecture covers Network Plane protocol elements such as 187 Operations, Administration and Maintenance (OAM) to observe some or 188 all hops along a Track as well as the end-to-end packet delivery, and 189 in-band control to optimize the use of redundancy to achieve the 190 required SLA with minimal use of constrained resources. 192 2. The RAW problem 194 2.1. Terminology 196 RAW reuses terminology defined for DetNet in the "Deterministic 197 Networking Architecture" [RFC8655], e.g., PREOF for Packet 198 Replication, Elimination and Ordering Functions. 200 RAW also reuses terminology defined for 6TiSCH in [6TiSCH-ARCHI] such 201 as the term Track. A Track as a complex path with associated PAREO 202 operations. The concept is abstract to the underlaying technology 203 and applies to any fully or partially wireless mesh, including, e.g., 204 a Wi-Fi mesh. RAW specifies strict and loose Tracks depending on 205 whether the path is fully controlled by RAW or traverses an opaque 206 network where RAW cannot observe and control the individual hops. 208 RAW uses the term OAM as defined in [RFC6291]. 210 RAW defines the following terms: 212 PAREO: Packet (hybrid) ARQ, Replication, Elimination and Ordering. 213 PAREO is a superset Of DetNet's PREOF that includes radio-specific 214 techniques such as short range broadcast, MUMIMO, constructive 215 interference and overhearing, which can be leveraged separately or 216 combined to increase the reliability. 218 Flapping: In the context of RAW, a link flaps when the reliability 219 of the wireless connectivity drops abruptly for a short period of 220 time, typically of a subsecond to seconds duration. 222 In the context of the RAW work, Reliability and Availability are 223 defined as follows: 225 Reliability: Reliability is a measure of the probability that an 226 item will perform its intended function for a specified interval 227 under stated conditions. For RAW, the service that is expected is 228 delivery within a bounded latency and a failure is when the packet 229 is either lost or delivered too late. RAW expresses reliability 230 in terms of Mean Time Between Failure (MTBF) and Maximum 231 Consecutive Failures (MCF). More in [NASA]. 233 Availability: Availability is a measure of the relative amount of 234 time where a path operates in stated condition, in other words 235 (uptime)/(uptime+downtime). Because a serial wireless path may 236 not be good enough to provide the required availability, and even 237 2 parallel paths may not be over a longer period of time, the RAW 238 availability implies a path that is a lot more complex than what 239 DetNet typically envisages (a Track). 241 2.2. Reliability and Availability 243 2.2.1. High Availability Engineering Principles 245 The reliability criteria of a critical system pervade through its 246 elements, and if the system comprises a data network then the data 247 network is also subject to the inherited reliability and availability 248 criteria. It is only natural to consider the art of high 249 availability engineering and apply it to wireless communicaitons in 250 the context of RAW. 252 There are three principles [pillars] of high availability 253 engineering: 255 1. elimination of single points of failure 256 2. reliable crossover 257 3. prompt detection of failures as they occur. 259 These principles are common to all high availability systems, not 260 just ones with Internet technology at the center. Examples of both 261 non-Internet and Internet are included. 263 2.2.1.1. Elimination of Single Points of Failure 265 Physical and logical components in a system happen to fail, either as 266 the effect of wear and tear, when used beyond acceptable limits, or 267 due to a software bug. It is necessary to decouple component failure 268 from system failure to avoid the latter. This allows failed 269 components to be restored while the rest of the system continues to 270 function. 272 IP Routers leverage routing protocols to compute alternate routes in 273 case of a failure. There is a rather open-ended issue over alternate 274 routes -- for example, when links are cabled through the same 275 conduit, they form a shared risk link group (SRLG), and will share 276 the same fate if the bundle is cut. The same effect can happen with 277 virtual links that end up in a same physical transport through the 278 games of encapsulation. In a same fashion, an interferer or an 279 obstacle may affect multiple wireless transmissions at the same time, 280 even between different sets of peers. 282 Intermediate network Nodes such as routers, switches and APs, wire 283 bundles and the air medium itself can become single points of 284 failure. For High Availability, it is thus required to use 285 physically link- and Node-disjoint paths; in the wireless space, it 286 is also required to use the highest possible degree of diversity in 287 the transmissions over the air to combat the additional causes of 288 transmission loss. 290 From an economics standpoint, executing this principle properly 291 generally increases capitalization expense because of the redundant 292 equipment. In a constrained network where the waste of energy and 293 bandwidth should be minimized, an excessive use of redundant links 294 must be avoided; for RAW this means that the extra bandwidth must be 295 used wisely and with parcimony. 297 2.2.1.2. Reliable Crossover 299 Having a backup equipment has a limited value unless it can be 300 reliably switched into use within the down-time parameters. IP 301 Routers execute reliable crossover continuously because the routers 302 will use any alternate routes that are available [RFC0791]. This is 303 due to the stateless nature of IP datagrams and the dissociation of 304 the datagrams from the forwarding routes they take. The "IP Fast 305 Reroute Framework" [FRR] analyzes mechanisms for fast failure 306 detection and path repair for IP Fast-Reroute, and discusses the case 307 of multiple failures and SRLG. Examples of FRR techniques include 308 Remote Loop-Free Alternate [RLFA-FRR] and backup label-switched path 309 (LSP) tunnels for the local repair of LSP tunnels using RSVP-TE 310 [RFC4090]. 312 Deterministic flows, on the contrary, are attached to specific paths 313 where dedicated resources are reserved for each flow. This is why 314 each DetNet path must inherently provide sufficient redundancy to 315 provide the guaranteed SLA at all times. The DetNet PREOF typically 316 leverages 1+1 redundancy whereby a packet is sent twice, over non- 317 congruent paths. This avoids the gap during the fast reroute 318 operation, but doubles the traffic in the network. 320 In the case of RAW, the expectation is that multiple transient faults 321 may happen in overlapping time windows, in which case the 1+1 322 redundancy with delayed reestablishment of the second path will not 323 provide the required guarantees. The Data Plane must be configured 324 with a sufficient degree of redundancy to select an alternate 325 redundant path immediately upon a fault, without the need for a slow 326 intervention from the controller plane. 328 2.2.1.3. Prompt Notification of Failures 330 The execution of the two above principles is likely to render a 331 system where the user will rarely see a failure. But someone needs 332 to in order to direct maintenance. 334 There are many reasons for system monitoring (FCAPS for fault, 335 configuration, accounting, performance, security is a handy mental 336 checklist) but fault monitoring is sufficient reason. 338 "An Architecture for Describing Simple Network Management Protocol 339 (SNMP) Management Frameworks" [STD 62] describes how to use SNMP to 340 observe and correct long-term faults. 342 "Overview and Principles of Internet Traffic Engineering" [TE] 343 discusses the importance of measurement for network protection, and 344 provides abstract an method for network survivability with the 345 analysis of a traffic matrix as observed by SNMP, probing techniques, 346 FTP, IGP link state advertisements, and more. 348 Those measurements are needed in the context of RAW to inform the 349 controller and make the long term reactive decision to rebuild a 350 complex path. But RAW itself operates in the Network Plane at a 351 faster time scale. To act on the Data Plane, RAW needs live 352 information from the Operational Plane , e.g., using Bidirectional 353 Forwarding Detection [BFD] and its variants (bidirectional and remote 354 BFD) to protect a link, and OAM techniques to protect a path. 356 2.2.2. Applying Reliability Concepts to Networking 358 The terms Reliability and Availability are defined for use in RAW in 359 Section 2.1 and the reader is invited to read [NASA] for more details 360 on the general definition of Reliability. Practically speaking a 361 number of nines is often used to indicate the reliability of a data 362 link, e.g., 5 nines indicate a Packet Delivery Ratio (PDR) of 363 99.999%. 365 This number is typical in a wired environment where the loss is due 366 to a random event such as a solar particle that affects the 367 transmission of a particular frame, but does not affect the previous 368 or next frame, nor frames transmitted on other links. Note that the 369 QoS requirements in RAW may include a bounded latency, and a packet 370 that arrives too late is a fault and not considered as delivered. 372 For a periodic networking pattern such as an automation control loop, 373 this number is proportional to the Mean Time Between Failures (MTBF). 374 When a single fault can have dramatic consequences, the MTBF 375 expresses the chances that the unwanted fault event occurs. In data 376 networks, this is rarely the case. Packet loss cannot never be fully 377 avoided and the systems are built to resist to one loss, e.g., using 378 redundancy with Retries (HARQ) or Packet Replication and Elimination 379 (PRE), or, in a typical control loop, by linear interpolation from 380 the previous measuremnents. 382 But the linear interpolation method cannot resist multiple 383 consecutive losses, and a high MTBF is desired as a guarantee that 384 this will not happen, IOW that the number of losses-in-a-row can be 385 bounded. In that case, what is really desired is a Maximum 386 Consecutive Failures (MCF). If the number of losses in a row passes 387 the MCF, the control loop has to abort and the system, e.g., the 388 production line, may need to enter an emergency stop condition. 390 Engineers that build automated processes may use the network 391 reliability expressed in nines or as an MTBF as a proxy to indicate 392 an MCF, e.g., as described in section 7.4 of the "Deterministic 393 Networking Use Cases" [RFC8578]. 395 2.2.3. Reliability in the Context of RAW 397 In contrast with wired networks, errors in transmission are the 398 predominant source of packet loss in wireless networks. 400 The root cause for the loss may be of multiple origins, calling for 401 the use of different forms of diversity: 403 Multipath Fading: A destructive interference by a reflection of the 404 original signal. 406 A radio signal may be received directly (line-of-sight) and/or as 407 a reflection on a physical structure (echo). The reflections take 408 a longer path and are delayed by the extra distance divided by the 409 speed of light in the medium. Depending on the frequency, the 410 echo lands with a different phase which may add up to 411 (constructive interference) or cancel the direct signal 412 (destructive interference). 414 The affected frequencies depend on the relative position of the 415 sender, the receiver, and all the reflecting objects in the 416 environment. A given hop will suffer from multipath fading for 417 multiple packets in a row till the something moves that changes 418 the reflection patterns. 420 Co-channel Interference: Energy in the spectrum used for the 421 transmission confuses the receiver. 423 The wireless medium itself is a Shared Risk Link Group (SRLG) for 424 nearby users of the same spectrum, as an interference may affect 425 multiple co-channel transmissions between different peers within 426 the interference domain of the interferer, possibly even when they 427 use different technologies. 429 Obstacle in Fresnel Zone: The optimal transmission happens when the 430 Fresnel Zone between the sender and the receiver is free of 431 obstacles. 433 As long as a physical object (e.g., a metallic trolley between 434 peers) that affects the transmission is not removed, the quality 435 of the link is affected. 437 In an environment that is rich of metallic structures and mobile 438 objects, a single radio link will provide a fuzzy service, meaning 439 that it cannot be trusted to transport the traffic reliably over a 440 long period of time. 442 Transmission losses are typically not independent, and their nature 443 and duration are unpredictable; as long as a physical object (e.g., a 444 metallic trolley between peers) that affects the transmission is not 445 removed, or as long as the interferer (e.g., a radar) keeps 446 transmitting, a continuous stream of packets will be affected. 448 The key technique to combat those unpredictable losses is diversity. 449 Different forms of diversity are necessary to combat different causes 450 of loss and the use of diversity must be maximised to optimize the 451 PDR. 453 A single packet may be sent at different times (time diversity) over 454 diverse paths (spatial diversity) that rely on diverse radio channels 455 (frequency diversity) and diverse PHY technologies, e.g., narrowband 456 vs. spread spectrum, or diverse codes. Using time diversity will 457 defeat short-term interferences; spatial diversity combats very local 458 causes such as multipath fading; narrowband and spread spectrum are 459 relatively innocuous to one another and can be used for diversity in 460 the presence of the other. 462 2.3. Use Cases and Requirements Served 464 In order to focus on real-worlds issues and assert the feasibility of 465 the proposed capabilities, RAW focuses on selected technologies that 466 can be scheduled at the lower layers: IEEE Std. 802.15.4 timeslotted 467 channel hopping (TSCH), 3GPP 5G ultra-reliable low latency 468 communications (URLLC), IEEE 802.11ax/be where 802.11be is extreme 469 high throughput (EHT), and L-band Digital Aeronautical Communications 470 System (LDACS). See [RAW-TECHNOS] for more. 472 "Deterministic Networking Use Cases" [RFC8578] presents a number of 473 wireless use cases including Wireless, such as application to 474 Industrial Applications, Pro-Audio, and SmartGrid Automation. 475 [RAW-USE-CASES] adds a number of use cases that demonstrate the need 476 for RAW capabilities for new applications such as Pro-Gaming and 477 drones. The use cases can be abstracted in two families, Loose 478 Protection, e.g., protecting the first hop in Radio Access Protection 479 and Strict Protection, e.g., providing End-to-End Protection in a 480 wireless mesh. 482 2.3.1. Radio Access Protection 484 To maintain the required SLA at all times, a wireless Host may use 485 more than one Radio Access Network (RAN) in parallel. 487 ... .. 488 RAN 1 ----- ... .. ... 489 / . .. .... 490 +--------+ / . .... +-----------+ 491 |Wireless|- . ..... | Service | 492 | Device |-***-- RAN 2 -- . Internet ....---| / | 493 |(STA/UE)|- .. ..... |Application| 494 +--------+ $$$ . ....... +-----------+ 495 \ ... ... ..... 496 RAN n -------- ... ..... 498 *** = flapping at this time $$$ expensive 500 Figure 1: Radio Access Protection 502 The RANs may be heterogeneous, e.g., 3GPP 5G [RAW-5G] and Wi-Fi 503 [RAW-TECHNOS] for high-speed communication, in which case a Layer-3 504 abstraction becomes useful to select which of the RANs are used at a 505 particular point of time, and the amount of traffic that is 506 distributed over each RAN. 508 The idea is that the rest of the path to the destination(s) is 509 protected separately (e.g., uses non-congruent paths, leverages 510 DetNet / TSN, etc...) and is a lot more reliable, e.g., wired. In 511 that case, RAW observes the reliability of the end-to-end operation 512 through each of the RANs but only observes and controls the wireless 513 operation the first hop. 515 A variation of that use case has a pair of wireless Hosts connected 516 over a wired core / backbone network. In that case, RAW observes and 517 controls the Ingress and Egress RANs, while neglecting the hops in 518 the core. The resulting loose Track may be instanciated, e.g., using 519 tunneling or loose source routing between the RANs. 521 2.3.2. End-to-End Protection in a Wireless Mesh 523 In radio technologies that support mesh networking (e.g., Wi-Fi and 524 TSCH), a Track is a complex path with distributed PAREO capabilities. 525 In that case, RAW operates through the multipath and makes decisions 526 either at the Ingress or at every hop (more in Section 3.3). 528 A-------B-------C-----D 529 / \ / / \ 530 Ingress ----M-------N--zzzzz--- Egress 531 \ \ / / 532 P--zzz--Q-------------R 534 zzz = flapping now 536 Figure 2: End-to-End Protection 538 The Protection may be imposed by the source based on end-to-end OAM, 539 or performed hop-by-hop, in which case the OAM must enables the 540 intermediate Nodes to estimate the quality of the rest of the 541 feasible paths in the remainder of the Track to the destination. 543 2.4. Related Work at The IETF 545 RAW intersects with protocols or practices in development at the IETF 546 as follows: 548 * The Dynamic Link Exchange Protocol (DLEP) [RFC8175] from [MANET] 549 can be leveraged at each hop to derive generic radio metrics 550 (e.g., based on LQI, RSSI, queueing delays and ETX) on individual 551 hops. 553 * OAM work at [detnet] such as [DetNet-IP-OAM] for the case of the 554 IP Data Plane observes the state of DetNet paths, typically MPLS 555 and IPv6 pseudowires [DetNet-DP-FW], in the direction of the 556 traffic. RAW needs feedback that flows on the reverse path and 557 gathers instantaneous values from the radio receivers at each hop 558 to inform back the source and replicating relays so they can make 559 optimized forwarding decisions. The work named ICAN may be 560 related as well. 562 * [BFD] detect faults in the path between an Ingress and an Egress 563 forwarding engines, but is unaware of the complexity of a path 564 with replication, and expects bidirectionality. BFD considers 565 delivery as success whereas with RAW the bounded latency can be as 566 important as the delivery itself. 568 * [SPRING] and [BIER] define in-band signaling that influences the 569 routing when decided at the head-end on the path. There's already 570 one RAW-related draft at BIER [BIER-PREF] more may follow. RAW 571 will need new in-band signaling when the decision is distributed, 572 e.g., required chances of reliable delivery to destination within 573 latency. This signaling enables relays to tune retries and 574 replication to meet the required SLA. 576 * [CCAMP] defines protocol-independent metrics and parameters 577 (measurement attributes) for describing links and paths that are 578 required for routing and signaling in technology-specific 579 networks. RAW would be a source of requirements for CCAMP to 580 define metrics that are significant to the focus radios. 582 3. The RAW Framework 584 3.1. Scope and Prerequisites 586 A prerequisite to the RAW operation is that an end-to-end routing 587 function computes a complex sub-topology along which forwarding can 588 happen between a source and one or more destinations. The concept of 589 Track is specified in the 6TiSCH Architecture [6TiSCH-ARCHI] to 590 represent that complex sub-topology. Tracks provide a high degree of 591 redundancy and diversity and enable the DetNet PREOF, network coding, 592 and possibly RAW specific techniques such as PAREO, leveraging 593 frequency diversity, time diversity, and possibly other forms of 594 diversity as well. 596 How the routing operation (e.g., PCE) in the Controller Plane 597 computes the Track is out of scope for RAW. The scope of the RAW 598 operation is one Track, and the goal of the RAW operation is to 599 optimize the use of the Track at the forwarding timescale to maintain 600 the expected SLA while optimizing the usage of constrained resources 601 such as energy and spectrum. 603 Another prerequisite is that an IP link can be established over the 604 radio with some guarantees in terms of service reliability, e.g., it 605 can be relied upon to transmit a packet within a bounded latency and 606 provides a guaranteed BER/PDR outside rare but existing transient 607 outage windows that can last from split seconds to minutes. The 608 radio layer can be programmed with abstract parameters, and can 609 return an abstract view of the state of the Link to help the Network 610 Layer forwarding decision (think DLEP from MANET). 612 How the radio interface manages its lower layers is out of control 613 and out of scope for RAW. In the same fashion, the non-RAW portion 614 along a loose Track is by definition out of control and out of scope 615 for RAW. Whether it is a single hop or a mesh is also unknown and 616 out of scope. 618 3.2. Routing Time Scale vs. Forwarding Time Scale 620 With DetNet, the Controller Plane Function that handles the routing 621 computation and maintenance (the PCE) can be centralized and can 622 reside outside the network. In a wireless mesh, the path to the PCE 623 can be expensive and slow, possibly going across the whole mesh and 624 back. Reaching to the PCE can also be slow in regards to the speed 625 of events that affect the forwarding operation at the radio layer. 627 Due to that cost and latency, the Controller Plane is not expected to 628 be sensitive/reactive to transient changes. The abstraction of a 629 link at the routing level is expected to use statistical metrics that 630 aggregate the behavior of a link over long periods of time, and 631 represent its properties as shades of gray as opposed to numerical 632 values such as a link quality indicator, or a boolean value for 633 either up or down. 635 +----------------+ 636 | Controller | 637 | [PCE] | 638 +----------------+ 639 ^ 640 | 641 Slow 642 | 643 _-._-._-._-._-._-. | ._-._-._-._-._-._-._-._-._-._-._-._- 644 _-._-._-._-._-._-._-. | _-._-._-._-._-._-._-._-._-._-._-._- 645 | 646 Expensive 647 | 648 .... | ....... 649 .... . | . ....... 650 .... v ... 651 .. A-------B-------C---D .. 652 ... / \ / / \ .. 653 . I ----M-------N--***-- E .. 654 .. \ \ / / ... 655 .. P--***--Q----------R .... 656 .. .... 657 . <----- Fast -------> .... 658 ....... .... 659 ................. 661 *** = flapping at this time 663 Figure 3: Time Scales 665 In the case of wireless, the changes that affect the forwarding 666 decision can happen frequently and often for short durations, e.g., a 667 mobile object moves between a transmitter and a receiver, and will 668 cancel the line of sight transmission for a few seconds, or a radar 669 measures the depth of a pool and interferes on a particular channel 670 for a split second. 672 There is thus a desire to separate the long term computation of the 673 route and the short term forwarding decision. In that model, the 674 routing operation computes a complex Track that enables multiple Non- 675 Equal Cost Multi-Path (N-ECMP) forwarding solutions, and leaves it to 676 the Data Plane to make the per-packet decision of which of these 677 possibilities should be used. 679 In the wired world, and more specifically in the context of Traffic 680 Engineering (TE), an alternate path can be used upon the detection of 681 a failure in the main path, e.g., using OAM in MPLS-TP or BFD over a 682 collection of SD-WAN tunnels. RAW formalizes a forwarding time scale 683 that is an order(s) of magnitude shorter than the controller plane 684 routing time scale, and separates the protocols and metrics that are 685 used at both scales. Routing can operate on long term statistics 686 such as delivery ratio over minutes to hours, but as a first 687 approximation can ignore flapping. On the other hand, the RAW 688 forwarding decision is made at the scale of the packet rate, and uses 689 information that must be pertinent at the present time for the 690 current transmission(s). 692 3.3. Wireless Tracks 694 The "6TiSCH Architecture" [6TiSCH-ARCHI] introduces the concept of 695 Track. RAW extends the concept to any wireless mesh technology, 696 including, e.g., Wi-Fi. A simple Track is composed of a direct 697 sequence of reserved hops to ensure the transmission of a single 698 packet from a source Node to a destination Node across a multihop 699 path. 701 A Complex Track provides multiple non-equal cost multipath (NECM) 702 forwarding solutions. The Complex Track enables to support multi- 703 path redundant forwarding by employing PRE functions [RFC8655] and 704 the ingress and within the Track. For example, a Complex Track may 705 branch off and rejoin over non-congruent segments. 707 In the context of RAW, some links or segments in the Track may be 708 reversible, meaning that they can be used in either direction. In 709 that case, an indication in the packet signals the direction of the 710 reversible links or segments that the packet traverses and thus 711 places a constraint that prevents loops from occuring. An indidual 712 packet follows a destination-oriented directed acyclic graph (DODAG) 713 towards a destination Node inside the Complex Track. 715 3.4. PAREO Functions 717 RAW may control whether and how to use packet replication and 718 elimination (PRE), Automatic Repeat reQuest (ARQ), Hybrid ARQ (HARQ) 719 that includes Forward Error Correction (FEC) and coding, and other 720 wireless-specific techniques such as overhearing and constructive 721 interferences, in order to increase the reliabiility and availability 722 of the end-to-end transmission. 724 Collectively, those function are called PAREO for Packet (hybrid) 725 ARQ, Replication, Elimination and Ordering. By tuning dynamically 726 the use of PAREO functions, RAW avoids the waste of critical 727 resources such as spectrum and energy while providing that the 728 guaranteed SLA, e.g., by adding redundancy only when a spike of loss 729 is observed. 731 In a nutshell, PAREO establishes several paths in a network to 732 provide redundancy and parallel transmissions to bound the end-to-end 733 delay to traverse the network. Optionally, promiscuous listening 734 between paths is possible, such that the Nodes on one path may 735 overhear transmissions along the other path. Considering the 736 scenario shown in Figure 4, many different paths are possible for S 737 to reach R. A simple way to benefit from this topology could be to 738 use the two independent paths via Nodes A, C, E and via B, D, F. But 739 more complex paths are possible by interleaving transmissions from 740 the lower level of the path to the upper level. 742 (A) -- (C) -- (E) 743 / \ 744 Ingress | | | Egress 745 \ / 746 (B) -- (D) -- (F) 748 Figure 4: A Ladder Shape with Two Parallel Paths 750 PAREO may also take advantage of the shared properties of the 751 wireless medium to compensate for the potential loss that is incurred 752 with radio transmissions. 754 For instance, when the source sends to Node A, Node B may listen 755 promiscuously and get a second chance to receive the frame without an 756 additional transmission. Note that B would not have to listen if it 757 already received that particular frame at an earlier timeslot in a 758 dedicated transmission towards B. 760 The PAREO model can be implemented in both centralized and 761 distributed scheduling approaches. In the centralized approach, a 762 Path Computation Element (PCE) scheduler calculates a Track and 763 schedules the communication. In the distributed approach, the Track 764 is computed within the network, and signaled in the packets, e.g., 765 using BIER-TE, Segment Routing, or a Source Routing Header. 767 3.4.1. Packet Replication 769 By employing a Packet Replication procedure, a Node forwards a copy 770 of each data packet to more than one successor. To do so, each Node 771 (i.e., Ingress and intermediate Node) sends the data packet multiple 772 times as separate unicast transmissions. For instance, in Figure 5, 773 the Ingress Node is transmitting the packet to both successors, nodes 774 A and B, at two different times. 776 ===> (A) => (C) => (E) === 777 // \\// \\// \\ 778 Ingress //\\ //\\ Egress 779 \\ // \\ // \\ // 780 ===> (B) => (D) => (F) === 782 Figure 5: Packet Replication 784 An example schedule is shown in Table 1. This way, the transmission 785 leverages with the time and spatial forms of diversity. 787 +=========+======+======+======+======+======+======+======+ 788 | Channel | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 789 +=========+======+======+======+======+======+======+======+ 790 | 0 | S->A | S->B | B->C | B->D | C->F | E->R | F->R | 791 +---------+------+------+------+------+------+------+------+ 792 | 1 | | A->C | A->D | C->E | D->E | D->F | | 793 +---------+------+------+------+------+------+------+------+ 795 Table 1: Packet Replication: Sample schedule 797 3.4.2. Packet Elimination 799 The replication operation increases the traffic load in the network, 800 due to packet duplications. This may occur at several stages inside 801 the Track, and to avoid an explosion of the number of copies, a 802 Packet Elimination procedure must be applied as well. To this aim, 803 once a Node receives the first copy of a data packet, it discards the 804 subsequent copies. 806 The logical functions of Replication and Elimination may be 807 collocated in an intermediate Node, the Node first eliminating the 808 redundant copies and then sending the packet exactly once to each of 809 the selected successors. 811 3.4.3. Promiscuous Overhearing 813 Considering that the wireless medium is broadcast by nature, any 814 neighbor of a transmitter may overhear a transmission. By employing 815 the Promiscuous Overhearing operation, the next hops have additional 816 opportunities to capture the data packets. In Figure 6, when Node A 817 is transmitting to its DP (Node C), the AP (Node D) and its sibling 818 (Node B) may decode this data packet as well. As a result, by 819 employing corellated paths, a Node may have multiple opportunities to 820 receive a given data packet. 822 ===> (A) ====> (C) ====> (E) ==== 823 // ^ | \\ \\ 824 Ingress | | \\ Egress 825 \\ | v \\ // 826 ===> (B) ====> (D) ====> (F) ==== 828 Figure 6: Unicast with Overhearing 830 3.4.4. Constructive Interference 832 Constructive Interference can be seen as the reverse of Promiscuous 833 Overhearing, and refers to the case where two senders transmit the 834 exact same signal in a fashion that the emitted symbols add up at the 835 receiver and permit a reception that would not be possible with a 836 single sender at the same PHY mode and the same power level. 838 Constructive Interference was proposed on 5G, Wi-Fi7 and even tested 839 on IEEE Std 802.14.5. The hard piece is to synchronize the senders 840 to the point that the signals are emitted at slightly different time 841 to offset the difference of propagation delay that corresponds to the 842 difference of distance of the transmitters to the receiver at the 843 speed of light to the point that the symbols are superposed long 844 enough to be recognizable. 846 4. The RAW Architecture 848 4.1. The RAW Conceptual Model 850 RAW inherits the conceptual model described in section 4 of the 851 DetNet Architecture [RFC8655]. RAW extends the DetNet service layer 852 to provide additional agility against transmission loss. 854 A RAW Network Plane may be strict or loose, depending on whether RAW 855 observes and takes actions on all hops or not. For instance, the 856 packets between two wireless entities may be relayed over a wired 857 infrastructure such as a Wi-Fi extended service set (ESS) or a 5G 858 Core; in that case, RAW observes and control the transmission over 859 the wireless first and last hops, as well as end-to-end metrics such 860 as latency, jitter, and delivery ratio. This operation is loose 861 since the structure and properties of the wired infrastructure are 862 ignored, and may be either controlled by other means such as DetNet/ 863 TSN, or neglected in the face of the wireless hops. 865 A Controller Plane Function (CPF) called the Path Computation Element 866 (PCE) [RFC4655] interacts with RAW Nodes over a Southbound API. The 867 RAW Nodes are DetNet relays that are capable of additional diversity 868 mechanisms and measurement functions related to the radio interface, 869 in particular the PAREO diversity mechanisms. 871 The PCE defines a complex Track between an Ingress End System and an 872 Egress End System, and indicates to the RAW Nodes where the PAREO 873 operations may be actionned in the Network Plane. The Track may be 874 expressed loosely to enable traversing a non-RAW subnetwork. In that 875 case, the expectation is that the non-RAW subnetwork can be neglected 876 in the RAW computation, that is, considered infinitely fast, reliable 877 and/or available in comparison with the links between RAW nodes. 879 CPF CPF CPF CPF 881 Southbound API 882 _-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._- 883 _-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._- 885 RAW --z RAW --z RAW --z RAW 886 z-- Node z-- Node z-- Node z-- Node --z 887 Ingress --z / / z-- Egress 888 End \ \ .. . End 889 Node ---z / / .. .. . z-- Node 890 z-- RAW --z RAW ( non-RAW ) -- RAW --z 891 Node z-- Node --- ( Nodes ) Node 892 ... . 893 --z wireless wired 894 z-- link --- link 896 Figure 7: RAW Nodes 898 The Link-Layer metrics are reported to the PCE in a time-aggregated, 899 e.g., statistical fashion. Example Link-Layer metrics include 900 typical Link bandwidth (the medium speed depends dynamically on the 901 PHY mode and the number of users sharing the spectrum) and average 902 and mean squared deviation of availability and reliability figures 903 such as Packet Delivery Ratio (PDR) over long periods of time. 905 Based on those metrics, the PCE installs the Track with enough 906 redundant forwarding solutions to ensure that the Network Plane can 907 reliably deliver the packets within a System Level Agreement (SLA) 908 associated to the flows that it transports. The SLA defines end-to- 909 end reliability and availability requirements, where reliability may 910 be expressed as a successful delivery in order and within a bounded 911 delay of at least one copy of a packet. 913 Depending on the use case and the SLA, the Track may comprise non-RAW 914 segments, either interleaved inside the Track, or all the way to the 915 Egress End Node (e.g., a server in the Internet). RAW observes the 916 Lower-Layer Links between RAW nodes (typically, radio links) and the 917 end-to-end Network Layer operation to decide at all times which of 918 the PAREO diversity schemes is actioned by which RAW Nodes. 920 Once a Track is established, per-segment and end-to-end reliability 921 and availability statistics are periodically reported to the PCE to 922 assure that the SLA can be met or have it recompute the Track if not. 924 4.2. The Path Selection Engine 926 RAW separates the path computation time scale at which a complex path 927 is recomputed from the path selection time scale at which the 928 forwarding decision is taken for one or a few packets (more in 929 Section 3.2). RAW operates at the path selection time scale. The 930 RAW problem is to decide, within the redundant solutions that are 931 proposed by the PCE, which will be used for each packet to provide a 932 Reliable and Available service while minimizing the waste of 933 constrained resources. 935 To that effect, RAW defines the Path Selection Engine (PSE) that is 936 the counter-part of the PCE to perform rapid local adjustments of the 937 forwarding tables within the diversity that the PCE has selected for 938 the Track. The PSE enables to exploit the richer forwarding 939 capabilities with PAREO and scheduled transmissions at a faster time 940 scale over the smaller domain that is the Track, in either a loose or 941 a strict fashion. 943 Compared to the PCE, the PSE operates on metrics that evolve faster, 944 but that needs to be advertised at a fast rate but only locally, 945 within the Track. The forwarding decision may also change rapidly, 946 but wiht a scope that is also contained within the Track, with no 947 visibility to the other Tracks and flows in the network. This is as 948 opposed to the PCE that needs to observe the whole network, and 949 optimize all the Tracks globally, which can only be done at a slow 950 pace and using long-term statistical metrics, as presented in 951 Table 2. 953 +===============+========================+===================+ 954 | | PCE (Not in Scope) | PSE (In Scope) | 955 +===============+========================+===================+ 956 | Operation | Centralized | Source-Routed or | 957 | | | Distributed | 958 +---------------+------------------------+-------------------+ 959 | Communication | Slow, expensive | Fast, local | 960 +---------------+------------------------+-------------------+ 961 | Time Scale | hours and above | seconds and below | 962 +---------------+------------------------+-------------------+ 963 | Network Size | Large, many Tracks to | Small, within one | 964 | | optimize globally | Track | 965 +---------------+------------------------+-------------------+ 966 | Considered | Averaged, Statistical, | Instant values / | 967 | Metrics | Shade of grey | boolean condition | 968 +---------------+------------------------+-------------------+ 970 Table 2: PCE vs. PSE 972 The PSE sits in the DetNet Service sub-Layer of Edge and Relay Nodes. 973 On the one hand, it operates on the packet flow, learning the Track 974 and path selection information from the packet, possibly making local 975 decision and retagging the packet to indicate so. On the other hand, 976 the PSE interacts with the lower layers and with its peers to obtain 977 up-to-date information about its radio links and the quality of the 978 overall Track, respectively, as illustrated in Figure 8. 980 | 981 packet | going 982 down the | stack 983 +==========v==========+=====================+=====================+ 984 | (iOAM + iCTRL) | (L2 Triggers, DLEP) | (oOAM) | 985 +==========v==========+=====================+=====================+ 986 | Learn from Learn from | 987 | packet tagging Maintain end-to-end | 988 +----------v----------+ Forwarding OAM packets | 989 | Forwarding decision < State +---------^-----------| 990 +----------v----------+ | Enrich or | 991 + Retag Packet | Learn abstracted > Regenerate | 992 | and Forward | metrics about Links | OAM packets | 993 +..........v..........+..........^..........+.........^.v.........+ 994 | Lower layers | 995 +..........v.....................^....................^.v.........+ 996 frame | sent Frame | L2 Ack oOAM | | packet 997 over | wireless In | In | | and out 998 v | | v 1000 Figure 8: PSE 1002 4.3. RAW OAM 1004 The RAW OAM operation in the Network Plane observes either a full 1005 Track or subTracks that are being used at this time. This 1006 observeation feeds the RAW PSE that makes the decision on which PAREO 1007 function in actioned at which RAW Node, for one a small continuous 1008 series of packets. 1010 ... .. 1011 RAN 1 ----- ... .. ... 1012 / . .. .... 1013 +-------+ / . .. .... +------+ 1014 |Ingress|- . ..... |Egress| 1015 | End |------ RAN 2 -- . Internet ....---| End | 1016 |System |- .. ..... |System| 1017 +-------+ \ . ...... +------+ 1018 \ ... ... ..... 1019 RAN n -------- ... ..... 1021 <------------------> <--------------------> 1022 Observed by OAM Opaque to OAM 1024 Figure 9: Observed Links in Radio Access Protection 1026 In the case of a End-to-End Protection in a Wireless Mesh, the Track 1027 is strict and congruent with the path so all links are observed. 1028 Conversely, in the case of Radio Access Protection, the Track is 1029 Loose and in that case only the first hop is observed; the rest of 1030 the path is abstracted and considered infinitely reliable. 1032 In the case of the Radio Access Protection, only the first hop is 1033 protected; the loss of a packet that was sent over one of the 1034 possible first hops is attributed to that first hop, even if a 1035 particular loss effectively happens farther down the path. 1037 The Links that are not observed by OAM are opaque to it, meaning that 1038 the OAM information is carried across and possibly echoed as data, 1039 but there is no information capture in intermediate nodes. In the 1040 example above, the Internet is opaque and not controlled by RAW; 1041 still the RAW OAM measures the end-to-end latency and delivery ratio 1042 for packets sent via each if RAN 1, RAN 2 and RAN 3, and determines 1043 whether a packet should be sent over either or a collection of those 1044 access links. 1046 4.4. Flow Identification vs. Path Identification 1048 Section 4.7 of the DetNet Architecture [RFC8655] ties the app-flow 1049 identification which is an appliation layer concept with the network 1050 path identification that depends on the networking technology by 1051 "exporting of flow identification", e.g., to a MPLS label. 1053 With RAW, this exporting operation is injective but not bijective. 1054 e.g., a flow is fully placed within one RAW Track, but not all 1055 packets along that Track are necessarily part of the same flow. For 1056 instance, out-of-band OAM packets must circulate in the exact same 1057 fashion as the flows that they observe. It results that the flow 1058 identification that maps to to app-flow at the network layer must be 1059 separate from the path identification that is used to forward a 1060 packet. 1062 Section 3.4 of the DetNet data-plane framework [DetNet-DP-FW] 1063 indicates that for a DetNet IP Data Plane, a flow is identified by an 1064 IPv6 6-tuple. With RAW, that 6-tuple is not what indicates the 1065 Track, in other words, the flow ID is not the Track ID. 1067 For instance, the 6TiSCH Architecture [6TiSCH-ARCHI] uses a 1068 combination of the address of the Egress End System and an instance 1069 identifier in a Hop-by-hop option to indicate a Track. This way, if 1070 a packet "escapes" the Track, it will reach the Track Egress point 1071 through normal routing and be treated at the service layer through, 1072 say, elimination and reordering. 1074 The RAW service includes forwarding over a subset of the Links that 1075 form the Track (a subTrack). Packets from the same or a different 1076 flow that are routed through the same Track will not necessarily 1077 traverse the same Links. The PSE selects a subTrack for a packet 1078 based on the links that are preferred and those that should be 1079 avoided at this time. 1081 Each packet is forwarded within the subTrack that provides the best 1082 adequation with the SLA of the flow and the energy and bandwidth 1083 constraints of the network. 1085 Flow 1 (6-tuple) ----+ 1086 | 1087 Flow 2 (6-tuple) ---+ | 1088 | | 1089 OAM -----------+ | | 1090 | | | 1091 | | | 1092 | | | | | 1093 | v v v | 1094 | | 1095 +---------+---------+ 1096 | 1097 | 1098 Track i (Egress IP Address, instanceId) 1099 | 1100 | 1101 | 1102 +---------+-----+--....-------+ 1103 | | | 1104 | | | 1105 subTrack 1 subTrack 2 subTrack n 1106 | | | 1107 | | | 1108 V V V 1109 +-----------------------------------+ 1110 | | 1111 | Destination | 1112 | | 1113 +-----------------------------------+ 1115 Figure 10: Flow Injection 1117 With 6TiSCH, packets are tagged with the same (destination address, 1118 instance ID) will experience the same RAW service regardless of the 1119 IPv6 6-tuple that indicates the flow. The forwarding does not depend 1120 on whether the packets transport application flows or OAM. In the 1121 generic case, the Track or the subTrack can be signaled in the packet 1122 through other means, e.g., encoded in the suffix of the destination 1123 address as a Segment Routing Service Instruction [SR-ARCHI], or 1124 leveraging Bit Index Explicit Replication [BIER] Traffic Engineering 1125 [BIER-TE]. 1127 4.5. Source-Routed vs. Distributed Forwarding Decision 1129 Within a large routed topology, the route-over mesh operation builds 1130 a particular complex Track with one source and one or more 1131 destinations; within the Track, packets may follow different paths 1132 and may be subject to RAW forwarding operations that include 1133 replication, elimination, retries, overhearing and reordering. 1135 The RAW forwarding decisions include the selection of points of 1136 replication and elimination, how many retries can take place, and a 1137 limit of validity for the packet beyond which the packet should be 1138 destroyed rather than forwarded uselessly further down the Track. 1140 The decision to apply the RAW techniques must be done quickly, and 1141 depends on a very recent and precise knowledge of the forwarding 1142 conditions within the complex Track. There is a need for an 1143 observation method to provide the RAW Data Plane with the specific 1144 knowledge of the state of the Track for the type of flow of interest 1145 (e.g., for a QoS level of interest). To observe the whole Track in 1146 quasi real time, RAW considers existing tools such as L2-triggers, 1147 DLEP, BFD and leverages in-band and out-of-band OAM to capture and 1148 report that information to the PSE. 1150 One possible way of making the RAW forwarding decisions within a 1151 Track is to position a unique PSE at the Ingress and express its 1152 decision in-band in the packet, which requires the explicit signaling 1153 of the subTrack within the Track. In that case, the RAW forwarding 1154 operation along the Track is encoded by the source, e.g., by 1155 indicating the subTrack in the Segment Routing (SRv6) Service 1156 Instruction, or by leveraging BIER-TE such as done with [BIER-PREF]. 1158 The alternate way is to operate the PSE in each forwarding Node, 1159 which makes the RAW forwarding decisions for a packet on its own, 1160 based on its knowledge of the expectation (timeliness and 1161 reliability) for that packet and a recent observation of the rest of 1162 the way across the possible paths based on OAM. Information about 1163 the desired service should be placed in the packet and matched with 1164 the forwarding Node's capabilities and policies. 1166 In either case, a per-track/subTrack state is installed in all the 1167 intermediate Nodes to recognize the packets that are following a 1168 Track and determine the forwarding operation to be applied. 1170 4.6. Encapsulation and Decapsulation 1172 In the generic case where the Track Ingress Node is not the source of 1173 the Packet, the Ingress Node needs to encapsulate IP-in-IP to ensure 1174 that the Destination IP Address is that of the Egress Node and that 1175 the necessary Headers (Routing Header, Segment Routing Header and/or 1176 Hop-By-Hop Header) can be added to the packet to signal the Track or 1177 the subTrack, conforming [IPv6] that discourages the insertion of a 1178 Header on the fly. 1180 In the specific case where the Ingress Node is the source of the 1181 packet, the encapsulation can be avoided, provided that the source 1182 adds the necessary headers and that the destination is set to the 1183 Egress Node. Forwarding to a final destination beyond the Egress 1184 Node is possible, e.g., with a Segment Routing Header that signals 1185 the rest of the way. In that case a Hop-by-Hop Header is not 1186 recommmended since its validity is within the Track only. 1188 5. Security Considerations 1190 RAW uses all forms of diversity including radio technology and 1191 physical path to increase the reliability and availability in the 1192 face of unpredictable conditions. While this is not done 1193 specifically to defeat an attacker, the amount of diversity used in 1194 RAW makes an attack harder to achieve. 1196 5.1. Forced Access 1198 RAW will typically select the cheapest collection of links that 1199 matches the requested SLA, for instance, leverage free WI-Fi vs. paid 1200 3GPP access. By defeating the cheap connectivity (e.g., PHY-layer 1201 interference) the attacker can force an End System to use the paid 1202 access and increase the cost of the transmission for the user. 1204 6. IANA Considerations 1206 This document has no IANA actions. 1208 7. Contributors 1210 Xavi Vilajosana: Wireless Networks Research Lab, Universitat Oberta 1211 de Catalunya 1213 Remous-Aris Koutsiamanis: IMT Atlantique 1215 Nicolas Montavont: IMT Atlantique 1217 8. Acknowledgments 1219 TBD 1221 9. References 1223 9.1. Normative References 1225 [6TiSCH-ARCHI] 1226 Thubert, P., "An Architecture for IPv6 over the TSCH mode 1227 of IEEE 802.15.4", Work in Progress, Internet-Draft, 1228 draft-ietf-6tisch-architecture-29, 27 August 2020, 1229 . 1232 [RAW-TECHNOS] 1233 Thubert, P., Cavalcanti, D., Vilajosana, X., Schmitt, C., 1234 and J. Farkas, "Reliable and Available Wireless 1235 Technologies", Work in Progress, Internet-Draft, draft- 1236 thubert-raw-technologies-05, 18 May 2020, 1237 . 1240 [RAW-USE-CASES] 1241 Papadopoulos, G., Thubert, P., Theoleyre, F., and C. 1242 Bernardos, "RAW use cases", Work in Progress, Internet- 1243 Draft, draft-bernardos-raw-use-cases-04, 13 July 2020, 1244 . 1247 [RFC4655] Farrel, A., Vasseur, J.-P., and J. Ash, "A Path 1248 Computation Element (PCE)-Based Architecture", RFC 4655, 1249 DOI 10.17487/RFC4655, August 2006, 1250 . 1252 [BFD] Katz, D. and D. Ward, "Bidirectional Forwarding Detection 1253 (BFD)", RFC 5880, DOI 10.17487/RFC5880, June 2010, 1254 . 1256 [RFC6291] Andersson, L., van Helvoort, H., Bonica, R., Romascanu, 1257 D., and S. Mansfield, "Guidelines for the Use of the "OAM" 1258 Acronym in the IETF", BCP 161, RFC 6291, 1259 DOI 10.17487/RFC6291, June 2011, 1260 . 1262 [RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases", 1263 RFC 8578, DOI 10.17487/RFC8578, May 2019, 1264 . 1266 [IPv6] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1267 (IPv6) Specification", STD 86, RFC 8200, 1268 DOI 10.17487/RFC8200, July 2017, 1269 . 1271 [SR-ARCHI] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L., 1272 Decraene, B., Litkowski, S., and R. Shakir, "Segment 1273 Routing Architecture", RFC 8402, DOI 10.17487/RFC8402, 1274 July 2018, . 1276 [BIER] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., 1277 Przygienda, T., and S. Aldrin, "Multicast Using Bit Index 1278 Explicit Replication (BIER)", RFC 8279, 1279 DOI 10.17487/RFC8279, November 2017, 1280 . 1282 [RFC8175] Ratliff, S., Jury, S., Satterwhite, D., Taylor, R., and B. 1283 Berry, "Dynamic Link Exchange Protocol (DLEP)", RFC 8175, 1284 DOI 10.17487/RFC8175, June 2017, 1285 . 1287 [RFC8557] Finn, N. and P. Thubert, "Deterministic Networking Problem 1288 Statement", RFC 8557, DOI 10.17487/RFC8557, May 2019, 1289 . 1291 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 1292 "Deterministic Networking Architecture", RFC 8655, 1293 DOI 10.17487/RFC8655, October 2019, 1294 . 1296 9.2. Informative References 1298 [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791, 1299 DOI 10.17487/RFC0791, September 1981, 1300 . 1302 [TE] Awduche, D., Chiu, A., Elwalid, A., Widjaja, I., and X. 1303 Xiao, "Overview and Principles of Internet Traffic 1304 Engineering", RFC 3272, DOI 10.17487/RFC3272, May 2002, 1305 . 1307 [STD 62] Harrington, D., Presuhn, R., and B. Wijnen, "An 1308 Architecture for Describing Simple Network Management 1309 Protocol (SNMP) Management Frameworks", STD 62, RFC 3411, 1310 DOI 10.17487/RFC3411, December 2002, 1311 . 1313 [RFC4090] Pan, P., Ed., Swallow, G., Ed., and A. Atlas, Ed., "Fast 1314 Reroute Extensions to RSVP-TE for LSP Tunnels", RFC 4090, 1315 DOI 10.17487/RFC4090, May 2005, 1316 . 1318 [FRR] Shand, M. and S. Bryant, "IP Fast Reroute Framework", 1319 RFC 5714, DOI 10.17487/RFC5714, January 2010, 1320 . 1322 [RLFA-FRR] Bryant, S., Filsfils, C., Previdi, S., Shand, M., and N. 1323 So, "Remote Loop-Free Alternate (LFA) Fast Reroute (FRR)", 1324 RFC 7490, DOI 10.17487/RFC7490, April 2015, 1325 . 1327 [BIER-PREF] 1328 Thubert, P., Eckert, T., Brodard, Z., and H. Jiang, "BIER- 1329 TE extensions for Packet Replication and Elimination 1330 Function (PREF) and OAM", Work in Progress, Internet- 1331 Draft, draft-thubert-bier-replication-elimination-03, 3 1332 March 2018, . 1335 [DetNet-IP-OAM] 1336 Mirsky, G., Chen, M., and D. Black, "Operations, 1337 Administration and Maintenance (OAM) for Deterministic 1338 Networks (DetNet) with IP Data Plane", Work in Progress, 1339 Internet-Draft, draft-mirsky-detnet-ip-oam-03, 7 August 1340 2020, . 1343 [DetNet-DP-FW] 1344 Varga, B., Farkas, J., Berger, L., Malis, A., and S. 1345 Bryant, "DetNet Data Plane Framework", Work in Progress, 1346 Internet-Draft, draft-ietf-detnet-data-plane-framework-06, 1347 6 May 2020, . 1350 [RAW-5G] Farkas, J., Dudda, T., Shapin, A., and S. Sandberg, "5G - 1351 Ultra-Reliable Wireless Technology with Low Latency", Work 1352 in Progress, Internet-Draft, draft-farkas-raw-5g-00, 1 1353 April 2020, 1354 . 1356 [BIER-TE] Eckert, T., Cauchie, G., and M. Menth, "Tree Engineering 1357 for Bit Index Explicit Replication (BIER-TE)", Work in 1358 Progress, Internet-Draft, draft-ietf-bier-te-arch-09, 30 1359 October 2020, 1360 . 1362 [NASA] Adams, T., "RELIABILITY: Definition & Quantitative 1363 Illustration", . 1366 [MANET] IETF, "Mobile Ad hoc Networking", 1367 . 1369 [detnet] IETF, "Deterministic Networking", 1370 . 1372 [SPRING] IETF, "Source Packet Routing in Networking", 1373 . 1375 [BIER] IETF, "Bit Indexed Explicit Replication", 1376 . 1378 [BFD] IETF, "Bidirectional Forwarding Detection", 1379 . 1381 [CCAMP] IETF, "Common Control and Measurement Plane", 1382 . 1384 Authors' Addresses 1386 Pascal Thubert (editor) 1387 Cisco Systems, Inc 1388 Building D 1389 45 Allee des Ormes - BP1200 1390 06254 MOUGINS - Sophia Antipolis 1391 France 1393 Phone: +33 497 23 26 34 1394 Email: pthubert@cisco.com 1396 Georgios Z. Papadopoulos 1397 IMT Atlantique 1398 Office B00 - 114A 1399 2 Rue de la Chataigneraie 1400 35510 Cesson-Sevigne - Rennes 1401 France 1403 Phone: +33 299 12 70 04 1404 Email: georgios.papadopoulos@imt-atlantique.fr 1405 Rex Buddenberg 1406 CA 1407 United States of America 1409 Email: buddenbergr@gmail.com