idnits 2.17.1 draft-pthubert-raw-architecture-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 888 has weird spacing: '...-- Node z-- ...' == Line 893 has weird spacing: '... Node z-- ...' -- The document date (1 June 2021) is 1059 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'PCE' is mentioned on line 639, but not defined == Outdated reference: A later version (-08) exists of draft-ietf-raw-technologies-01 == Outdated reference: A later version (-11) exists of draft-ietf-raw-use-cases-01 -- Obsolete informational reference (is this intentional?): RFC 3272 (ref. 'TE') (Obsoleted by RFC 9522) == Outdated reference: A later version (-13) exists of draft-ietf-detnet-ip-oam-02 == Outdated reference: A later version (-13) exists of draft-ietf-bier-te-arch-09 == Outdated reference: A later version (-15) exists of draft-thubert-6man-ipv6-over-wireless-09 Summary: 0 errors (**), 0 flaws (~~), 9 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 RAW P. Thubert, Ed. 3 Internet-Draft Cisco Systems 4 Intended status: Informational G.Z. Papadopoulos 5 Expires: 3 December 2021 IMT Atlantique 6 L. Berger 7 LabN Consulting, L.L.C. 8 R. Buddenberg 9 1 June 2021 11 Reliable and Available Wireless Architecture/Framework 12 draft-pthubert-raw-architecture-06 14 Abstract 16 Reliable and Available Wireless (RAW) provides for high reliability 17 and availability for IP connectivity over a wireless medium. The 18 wireless medium presents significant challenges to achieve 19 deterministic properties such as low packet error rate, bounded 20 consecutive losses, and bounded latency. This document defines the 21 RAW Architecture. It builds on the DetNet Architecture and discusses 22 specific challenges and technology considerations needed to deliver 23 DetNet service utilizing scheduled wireless segments and other media, 24 e.g., frequency/time-sharing physical media resources with stochastic 25 traffic. 27 Status of This Memo 29 This Internet-Draft is submitted in full conformance with the 30 provisions of BCP 78 and BCP 79. 32 Internet-Drafts are working documents of the Internet Engineering 33 Task Force (IETF). Note that other groups may also distribute 34 working documents as Internet-Drafts. The list of current Internet- 35 Drafts is at https://datatracker.ietf.org/drafts/current/. 37 Internet-Drafts are draft documents valid for a maximum of six months 38 and may be updated, replaced, or obsoleted by other documents at any 39 time. It is inappropriate to use Internet-Drafts as reference 40 material or to cite them other than as "work in progress." 42 This Internet-Draft will expire on 3 December 2021. 44 Copyright Notice 46 Copyright (c) 2021 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 51 license-info) in effect on the date of publication of this document. 52 Please review these documents carefully, as they describe your rights 53 and restrictions with respect to this document. Code Components 54 extracted from this document must include Simplified BSD License text 55 as described in Section 4.e of the Trust Legal Provisions and are 56 provided without warranty as described in the Simplified BSD License. 58 Table of Contents 60 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 61 2. The RAW problem . . . . . . . . . . . . . . . . . . . . . . . 5 62 2.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 5 63 2.2. Reliability and Availability . . . . . . . . . . . . . . 6 64 2.2.1. High Availability Engineering Principles . . . . . . 6 65 2.2.2. Applying Reliability Concepts to Networking . . . . . 8 66 2.2.3. Reliability in the Context of RAW . . . . . . . . . . 9 67 2.3. Use Cases and Requirements Served . . . . . . . . . . . . 10 68 2.3.1. Radio Access Protection . . . . . . . . . . . . . . . 11 69 2.3.2. End-to-End Protection in a Wireless Mesh . . . . . . 12 70 2.4. Related Work at The IETF . . . . . . . . . . . . . . . . 12 71 3. The RAW Framework . . . . . . . . . . . . . . . . . . . . . . 13 72 3.1. Scope and Prerequisites . . . . . . . . . . . . . . . . . 13 73 3.2. Routing Time Scale vs. Forwarding Time Scale . . . . . . 14 74 3.3. Wireless Tracks . . . . . . . . . . . . . . . . . . . . . 16 75 3.4. PAREO Functions . . . . . . . . . . . . . . . . . . . . . 16 76 3.4.1. Packet Replication . . . . . . . . . . . . . . . . . 17 77 3.4.2. Packet Elimination . . . . . . . . . . . . . . . . . 18 78 3.4.3. Promiscuous Overhearing . . . . . . . . . . . . . . . 18 79 3.4.4. Constructive Interference . . . . . . . . . . . . . . 19 80 4. The RAW Architecture . . . . . . . . . . . . . . . . . . . . 19 81 4.1. The RAW Conceptual Model . . . . . . . . . . . . . . . . 19 82 4.2. The Path Selection Engine . . . . . . . . . . . . . . . . 21 83 4.3. RAW OAM . . . . . . . . . . . . . . . . . . . . . . . . . 23 84 4.4. Flow Identification vs. Path Identification . . . . . . . 24 85 4.5. Source-Routed vs. Distributed Forwarding Decision . . . . 26 86 4.6. Encapsulation and Decapsulation . . . . . . . . . . . . . 27 87 5. Security Considerations . . . . . . . . . . . . . . . . . . . 27 88 5.1. Forced Access . . . . . . . . . . . . . . . . . . . . . . 27 89 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 27 90 7. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 27 91 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 28 92 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 28 93 9.1. Normative References . . . . . . . . . . . . . . . . . . 28 94 9.2. Informative References . . . . . . . . . . . . . . . . . 29 95 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 31 97 1. Introduction 99 Deterministic Networking is an attempt to emulate the properties of a 100 serial link over a switched fabric, by providing a bounded latency 101 and eliminating congestion loss, even when co-existing with best- 102 effort traffic. It is getting traction in various industries 103 including professional A/V, manufacturing, online gaming, and 104 smartgrid automation, enabling cost and performance optimizations 105 (e.g., vs. loads of P2P cables). 107 Bringing determinism in a packet network means eliminating the 108 statistical effects of multiplexing that result in probabilistic 109 jitter and loss. This can be approached with a tight control of the 110 physical resources to maintain the amount of traffic within a 111 budgetted volume of data per unit of time that fits the physical 112 capabilities of the underlying network, and the use of time-shared 113 resources (bandwidth and buffers) per circuit, and/or by shaping and/ 114 or scheduling the packets at every hop. 116 This innovation was initially introduced on wired networks, with IEEE 117 802.1 Time Sensitive networking (TSN) - for Ethernet LANs - and IETF 118 DetNet. But the wired and the wireless media are fundamentally 119 different at the physical level and in the possible abstractions that 120 can be built for IP [IPoWIRELESS]. Wireless networks operate on a 121 shared medium where uncontrolled interference, including the self- 122 induced multipath fading, cause random transmission losses and add 123 new dimensions to the statistical effects that affect reachability 124 and packet delivery. 126 To defeat those additional causes of transmission delay and loss, 127 Reliable and Available Wireless (RAW) leverages scheduled 128 transmissions with redundancy and diversity in the spatial, time, 129 code, and frequency domains. The challenge is to provide enough 130 diversity and redundancy to ensure the timely packet delivery while 131 preserving energy and optimizing the use of the shared spectrum. 133 While the generic "Deterministic Networking Problem Statement" 134 [RFC8557] applies to both the wired and the wireless media, the 135 methods to achieve RAW must extend those used to support time- 136 sensitive networking over wires, as a RAW solution has to address 137 less consistent transmissions, energy conservation and shared 138 spectrum efficiency. 140 Uncontrolled interference and transmission obstacles may impede the 141 wireless transmission, causing rapid variations of the throughput and 142 packet delivery ratio (PDR) of the link. This uncertainty limits the 143 volume and/or duration of traffic that can be safely transmitted on 144 the same link while conforming to a RAW Service Level Agreement 145 (SLA). 147 This increased complexity explains why the development of 148 deterministic wireless technologies has been lagging behind the 149 similar efforts for wired systems, both at the IEEE and the IETF. 150 But recent progress on scheduled radios such as TSCH and OFDMA 151 indicates that wireless is finally catching up at the lower layers. 152 Sitting at the layer above, RAW takes up the challenge of providing 153 highly available and reliable end-to-end performances in a network 154 with scheduled wireless segments. 156 RAW provides DetNet elements that are specialized for short range 157 radios. From this inheritance, RAW stays agnostic to the radio layer 158 underneath though the capability to schedule transmissions is 159 assumed. How the PHY is programmed to do so, and whether the radio 160 is single-hop or meshed, are unknown at the IP layer and not part of 161 the RAW abstraction. 163 The "Deterministic Networking Architecture" [RFC8655] is composed of 164 three planes: the Application (User) Plane, the Controller Plane, and 165 the Network Plane. The RAW Architecture extends the DetNet Network 166 Plane, to accomodate one or multiple hops of homogeneous or 167 heterogeneous wireless technologies, e.g. a Wi-Fi6 Mesh or parallel 168 CBRS access links federated by a 5G backhaul. 170 The establishment of a path is not in-scope for RAW. It may be the 171 product of a centralized Controller Plane as described for DetNet. 172 As opposed to wired networks, the action of installing a path over a 173 set of wireless links may be very slow relative to the speed at which 174 the radio conditions vary, and it makes sense in the wireless case to 175 provide redundant forwarding solutions along a complex path and to 176 leave it to the Network Plane to select which of those forwarding 177 solutions are to be used for a given packet based on the current 178 conditions. 180 RAW distinguishes the longer time scale at which routes are computed 181 from the the shorter forwarding time scale where per-packet decisions 182 are made. RAW operates within the Network Plane at the forwarding 183 time scale on one DetNet flow over a complex path called a Track. 184 The Track is preestablished and installed by means outside of the 185 scope of RAW; it may be strict or loose depending on whether each or 186 just a subset of the hops are observed and controlled by RAW. 188 The RAW Architecture covers Network Plane protocol elements such as 189 Operations, Administration and Maintenance (OAM) to observe some or 190 all hops along a Track as well as the end-to-end packet delivery, and 191 in-band control to optimize the use of redundancy to achieve the 192 required SLA with minimal use of constrained resources. 194 2. The RAW problem 196 2.1. Terminology 198 RAW reuses terminology defined for DetNet in the "Deterministic 199 Networking Architecture" [RFC8655], e.g., PREOF for Packet 200 Replication, Elimination and Ordering Functions. 202 RAW also reuses terminology defined for 6TiSCH in [6TiSCH-ARCHI] such 203 as the term Track. A Track as a complex path with associated PAREO 204 operations. The concept is abstract to the underlaying technology 205 and applies to any fully or partially wireless mesh, including, e.g., 206 a Wi-Fi mesh. RAW specifies strict and loose Tracks depending on 207 whether the path is fully controlled by RAW or traverses an opaque 208 network where RAW cannot observe and control the individual hops. 210 RAW uses the term OAM as defined in [RFC6291]. 212 RAW defines the following terms: 214 PAREO: Packet (hybrid) ARQ, Replication, Elimination and Ordering. 215 PAREO is a superset Of DetNet's PREOF that includes radio-specific 216 techniques such as short range broadcast, MUMIMO, constructive 217 interference and overhearing, which can be leveraged separately or 218 combined to increase the reliability. 220 Flapping: In the context of RAW, a link flaps when the reliability 221 of the wireless connectivity drops abruptly for a short period of 222 time, typically of a subsecond to seconds duration. 224 In the context of the RAW work, Reliability and Availability are 225 defined as follows: 227 Reliability: Reliability is a measure of the probability that an 228 item will perform its intended function for a specified interval 229 under stated conditions. For RAW, the service that is expected is 230 delivery within a bounded latency and a failure is when the packet 231 is either lost or delivered too late. RAW expresses reliability 232 in terms of Mean Time Between Failure (MTBF) and Maximum 233 Consecutive Failures (MCF). More in [NASA]. 235 Availability: Availability is a measure of the relative amount of 236 time where a path operates in stated condition, in other words 237 (uptime)/(uptime+downtime). Because a serial wireless path may 238 not be good enough to provide the required availability, and even 239 2 parallel paths may not be over a longer period of time, the RAW 240 availability implies a path that is a lot more complex than what 241 DetNet typically envisages (a Track). 243 2.2. Reliability and Availability 245 2.2.1. High Availability Engineering Principles 247 The reliability criteria of a critical system pervade through its 248 elements, and if the system comprises a data network then the data 249 network is also subject to the inherited reliability and availability 250 criteria. It is only natural to consider the art of high 251 availability engineering and apply it to wireless communicaitons in 252 the context of RAW. 254 There are three principles [pillars] of high availability 255 engineering: 257 1. elimination of single points of failure 258 2. reliable crossover 259 3. prompt detection of failures as they occur. 261 These principles are common to all high availability systems, not 262 just ones with Internet technology at the center. Examples of both 263 non-Internet and Internet are included. 265 2.2.1.1. Elimination of Single Points of Failure 267 Physical and logical components in a system happen to fail, either as 268 the effect of wear and tear, when used beyond acceptable limits, or 269 due to a software bug. It is necessary to decouple component failure 270 from system failure to avoid the latter. This allows failed 271 components to be restored while the rest of the system continues to 272 function. 274 IP Routers leverage routing protocols to compute alternate routes in 275 case of a failure. There is a rather open-ended issue over alternate 276 routes -- for example, when links are cabled through the same 277 conduit, they form a shared risk link group (SRLG), and will share 278 the same fate if the bundle is cut. The same effect can happen with 279 virtual links that end up in a same physical transport through the 280 games of encapsulation. In a same fashion, an interferer or an 281 obstacle may affect multiple wireless transmissions at the same time, 282 even between different sets of peers. 284 Intermediate network Nodes such as routers, switches and APs, wire 285 bundles and the air medium itself can become single points of 286 failure. For High Availability, it is thus required to use 287 physically link- and Node-disjoint paths; in the wireless space, it 288 is also required to use the highest possible degree of diversity in 289 the transmissions over the air to combat the additional causes of 290 transmission loss. 292 From an economics standpoint, executing this principle properly 293 generally increases capitalization expense because of the redundant 294 equipment. In a constrained network where the waste of energy and 295 bandwidth should be minimized, an excessive use of redundant links 296 must be avoided; for RAW this means that the extra bandwidth must be 297 used wisely and with parcimony. 299 2.2.1.2. Reliable Crossover 301 Having a backup equipment has a limited value unless it can be 302 reliably switched into use within the down-time parameters. IP 303 Routers execute reliable crossover continuously because the routers 304 will use any alternate routes that are available [RFC0791]. This is 305 due to the stateless nature of IP datagrams and the dissociation of 306 the datagrams from the forwarding routes they take. The "IP Fast 307 Reroute Framework" [FRR] analyzes mechanisms for fast failure 308 detection and path repair for IP Fast-Reroute, and discusses the case 309 of multiple failures and SRLG. Examples of FRR techniques include 310 Remote Loop-Free Alternate [RLFA-FRR] and backup label-switched path 311 (LSP) tunnels for the local repair of LSP tunnels using RSVP-TE 312 [RFC4090]. 314 Deterministic flows, on the contrary, are attached to specific paths 315 where dedicated resources are reserved for each flow. This is why 316 each DetNet path must inherently provide sufficient redundancy to 317 provide the guaranteed SLA at all times. The DetNet PREOF typically 318 leverages 1+1 redundancy whereby a packet is sent twice, over non- 319 congruent paths. This avoids the gap during the fast reroute 320 operation, but doubles the traffic in the network. 322 In the case of RAW, the expectation is that multiple transient faults 323 may happen in overlapping time windows, in which case the 1+1 324 redundancy with delayed reestablishment of the second path will not 325 provide the required guarantees. The Data Plane must be configured 326 with a sufficient degree of redundancy to select an alternate 327 redundant path immediately upon a fault, without the need for a slow 328 intervention from the controller plane. 330 2.2.1.3. Prompt Notification of Failures 332 The execution of the two above principles is likely to render a 333 system where the user will rarely see a failure. But someone needs 334 to in order to direct maintenance. 336 There are many reasons for system monitoring (FCAPS for fault, 337 configuration, accounting, performance, security is a handy mental 338 checklist) but fault monitoring is sufficient reason. 340 "An Architecture for Describing Simple Network Management Protocol 341 (SNMP) Management Frameworks" [STD 62] describes how to use SNMP to 342 observe and correct long-term faults. 344 "Overview and Principles of Internet Traffic Engineering" [TE] 345 discusses the importance of measurement for network protection, and 346 provides abstract an method for network survivability with the 347 analysis of a traffic matrix as observed by SNMP, probing techniques, 348 FTP, IGP link state advertisements, and more. 350 Those measurements are needed in the context of RAW to inform the 351 controller and make the long term reactive decision to rebuild a 352 complex path. But RAW itself operates in the Network Plane at a 353 faster time scale. To act on the Data Plane, RAW needs live 354 information from the Operational Plane , e.g., using Bidirectional 355 Forwarding Detection [BFD] and its variants (bidirectional and remote 356 BFD) to protect a link, and OAM techniques to protect a path. 358 2.2.2. Applying Reliability Concepts to Networking 360 The terms Reliability and Availability are defined for use in RAW in 361 Section 2.1 and the reader is invited to read [NASA] for more details 362 on the general definition of Reliability. Practically speaking a 363 number of nines is often used to indicate the reliability of a data 364 link, e.g., 5 nines indicate a Packet Delivery Ratio (PDR) of 365 99.999%. 367 This number is typical in a wired environment where the loss is due 368 to a random event such as a solar particle that affects the 369 transmission of a particular frame, but does not affect the previous 370 or next frame, nor frames transmitted on other links. Note that the 371 QoS requirements in RAW may include a bounded latency, and a packet 372 that arrives too late is a fault and not considered as delivered. 374 For a periodic networking pattern such as an automation control loop, 375 this number is proportional to the Mean Time Between Failures (MTBF). 376 When a single fault can have dramatic consequences, the MTBF 377 expresses the chances that the unwanted fault event occurs. In data 378 networks, this is rarely the case. Packet loss cannot never be fully 379 avoided and the systems are built to resist to one loss, e.g., using 380 redundancy with Retries (HARQ) or Packet Replication and Elimination 381 (PRE), or, in a typical control loop, by linear interpolation from 382 the previous measuremnents. 384 But the linear interpolation method cannot resist multiple 385 consecutive losses, and a high MTBF is desired as a guarantee that 386 this will not happen, IOW that the number of losses-in-a-row can be 387 bounded. In that case, what is really desired is a Maximum 388 Consecutive Failures (MCF). If the number of losses in a row passes 389 the MCF, the control loop has to abort and the system, e.g., the 390 production line, may need to enter an emergency stop condition. 392 Engineers that build automated processes may use the network 393 reliability expressed in nines or as an MTBF as a proxy to indicate 394 an MCF, e.g., as described in section 7.4 of the "Deterministic 395 Networking Use Cases" [RFC8578]. 397 2.2.3. Reliability in the Context of RAW 399 In contrast with wired networks, errors in transmission are the 400 predominant source of packet loss in wireless networks. 402 The root cause for the loss may be of multiple origins, calling for 403 the use of different forms of diversity: 405 Multipath Fading: A destructive interference by a reflection of the 406 original signal. 408 A radio signal may be received directly (line-of-sight) and/or as 409 a reflection on a physical structure (echo). The reflections take 410 a longer path and are delayed by the extra distance divided by the 411 speed of light in the medium. Depending on the frequency, the 412 echo lands with a different phase which may add up to 413 (constructive interference) or cancel the direct signal 414 (destructive interference). 416 The affected frequencies depend on the relative position of the 417 sender, the receiver, and all the reflecting objects in the 418 environment. A given hop will suffer from multipath fading for 419 multiple packets in a row till the something moves that changes 420 the reflection patterns. 422 Co-channel Interference: Energy in the spectrum used for the 423 transmission confuses the receiver. 425 The wireless medium itself is a Shared Risk Link Group (SRLG) for 426 nearby users of the same spectrum, as an interference may affect 427 multiple co-channel transmissions between different peers within 428 the interference domain of the interferer, possibly even when they 429 use different technologies. 431 Obstacle in Fresnel Zone: The optimal transmission happens when the 432 Fresnel Zone between the sender and the receiver is free of 433 obstacles. 435 As long as a physical object (e.g., a metallic trolley between 436 peers) that affects the transmission is not removed, the quality 437 of the link is affected. 439 In an environment that is rich of metallic structures and mobile 440 objects, a single radio link will provide a fuzzy service, meaning 441 that it cannot be trusted to transport the traffic reliably over a 442 long period of time. 444 Transmission losses are typically not independent, and their nature 445 and duration are unpredictable; as long as a physical object (e.g., a 446 metallic trolley between peers) that affects the transmission is not 447 removed, or as long as the interferer (e.g., a radar) keeps 448 transmitting, a continuous stream of packets will be affected. 450 The key technique to combat those unpredictable losses is diversity. 451 Different forms of diversity are necessary to combat different causes 452 of loss and the use of diversity must be maximised to optimize the 453 PDR. 455 A single packet may be sent at different times (time diversity) over 456 diverse paths (spatial diversity) that rely on diverse radio channels 457 (frequency diversity) and diverse PHY technologies, e.g., narrowband 458 vs. spread spectrum, or diverse codes. Using time diversity will 459 defeat short-term interferences; spatial diversity combats very local 460 causes such as multipath fading; narrowband and spread spectrum are 461 relatively innocuous to one another and can be used for diversity in 462 the presence of the other. 464 2.3. Use Cases and Requirements Served 466 In order to focus on real-worlds issues and assert the feasibility of 467 the proposed capabilities, RAW focuses on selected technologies that 468 can be scheduled at the lower layers: IEEE Std. 802.15.4 timeslotted 469 channel hopping (TSCH), 3GPP 5G ultra-reliable low latency 470 communications (URLLC), IEEE 802.11ax/be where 802.11be is extreme 471 high throughput (EHT), and L-band Digital Aeronautical Communications 472 System (LDACS). See [RAW-TECHNOS] for more. 474 "Deterministic Networking Use Cases" [RFC8578] presents a number of 475 wireless use cases including Wireless, such as application to 476 Industrial Applications, Pro-Audio, and SmartGrid Automation. 477 [RAW-USE-CASES] adds a number of use cases that demonstrate the need 478 for RAW capabilities for new applications such as Pro-Gaming and 479 drones. The use cases can be abstracted in two families, Loose 480 Protection, e.g., protecting the first hop in Radio Access Protection 481 and Strict Protection, e.g., providing End-to-End Protection in a 482 wireless mesh. 484 2.3.1. Radio Access Protection 486 To maintain the required SLA at all times, a wireless Host may use 487 more than one Radio Access Network (RAN) in parallel. 489 ... .. 490 RAN 1 ----- ... .. ... 491 / . .. .... 492 +--------+ / . .... +-----------+ 493 |Wireless|- . ..... | Service | 494 | Device |-***-- RAN 2 -- . Internet ....---| / | 495 |(STA/UE)|- .. ..... |Application| 496 +--------+ $$$ . ....... +-----------+ 497 \ ... ... ..... 498 RAN n -------- ... ..... 500 *** = flapping at this time $$$ expensive 502 Figure 1: Radio Access Protection 504 The RANs may be heterogeneous, e.g., 3GPP 5G [RAW-5G] and Wi-Fi 505 [RAW-TECHNOS] for high-speed communication, in which case a Layer-3 506 abstraction becomes useful to select which of the RANs are used at a 507 particular point of time, and the amount of traffic that is 508 distributed over each RAN. 510 The idea is that the rest of the path to the destination(s) is 511 protected separately (e.g., uses non-congruent paths, leverages 512 DetNet / TSN, etc...) and is a lot more reliable, e.g., wired. In 513 that case, RAW observes the reliability of the end-to-end operation 514 through each of the RANs but only observes and controls the wireless 515 operation the first hop. 517 A variation of that use case has a pair of wireless Hosts connected 518 over a wired core / backbone network. In that case, RAW observes and 519 controls the Ingress and Egress RANs, while neglecting the hops in 520 the core. The resulting loose Track may be instanciated, e.g., using 521 tunneling or loose source routing between the RANs. 523 2.3.2. End-to-End Protection in a Wireless Mesh 525 In radio technologies that support mesh networking (e.g., Wi-Fi and 526 TSCH), a Track is a complex path with distributed PAREO capabilities. 527 In that case, RAW operates through the multipath and makes decisions 528 either at the Ingress or at every hop (more in Section 3.3). 530 A-------B-------C-----D 531 / \ / / \ 532 Ingress ----M-------N--zzzzz--- Egress 533 \ \ / / 534 P--zzz--Q-------------R 536 zzz = flapping now 538 Figure 2: End-to-End Protection 540 The Protection may be imposed by the source based on end-to-end OAM, 541 or performed hop-by-hop, in which case the OAM must enables the 542 intermediate Nodes to estimate the quality of the rest of the 543 feasible paths in the remainder of the Track to the destination. 545 2.4. Related Work at The IETF 547 RAW intersects with protocols or practices in development at the IETF 548 as follows: 550 * The Dynamic Link Exchange Protocol (DLEP) [RFC8175] from [MANET] 551 can be leveraged at each hop to derive generic radio metrics 552 (e.g., based on LQI, RSSI, queueing delays and ETX) on individual 553 hops. 555 * OAM work at [detnet] such as [DetNet-IP-OAM] for the case of the 556 IP Data Plane observes the state of DetNet paths, typically MPLS 557 and IPv6 pseudowires [DetNet-DP], in the direction of the traffic. 558 RAW needs feedback that flows on the reverse path and gathers 559 instantaneous values from the radio receivers at each hop to 560 inform back the source and replicating relays so they can make 561 optimized forwarding decisions. The work named ICAN may be 562 related as well. 564 * [BFD] detect faults in the path between an Ingress and an Egress 565 forwarding engines, but is unaware of the complexity of a path 566 with replication, and expects bidirectionality. BFD considers 567 delivery as success whereas with RAW the bounded latency can be as 568 important as the delivery itself. 570 * [SPRING] and [BIER] define in-band signaling that influences the 571 routing when decided at the head-end on the path. There's already 572 one RAW-related draft at BIER [BIER-PREF] more may follow. RAW 573 will need new in-band signaling when the decision is distributed, 574 e.g., required chances of reliable delivery to destination within 575 latency. This signaling enables relays to tune retries and 576 replication to meet the required SLA. 578 * [CCAMP] defines protocol-independent metrics and parameters 579 (measurement attributes) for describing links and paths that are 580 required for routing and signaling in technology-specific 581 networks. RAW would be a source of requirements for CCAMP to 582 define metrics that are significant to the focus radios. 584 3. The RAW Framework 586 3.1. Scope and Prerequisites 588 A prerequisite to the RAW operation is that an end-to-end routing 589 function computes a complex sub-topology along which forwarding can 590 happen between a source and one or more destinations. The concept of 591 Track is specified in the 6TiSCH Architecture [6TiSCH-ARCHI] to 592 represent that complex sub-topology. Tracks provide a high degree of 593 redundancy and diversity and enable the DetNet PREOF, network coding, 594 and possibly RAW specific techniques such as PAREO, leveraging 595 frequency diversity, time diversity, and possibly other forms of 596 diversity as well. 598 How the routing operation (e.g., PCE) in the Controller Plane 599 computes the Track is out of scope for RAW. The scope of the RAW 600 operation is one Track, and the goal of the RAW operation is to 601 optimize the use of the Track at the forwarding timescale to maintain 602 the expected SLA while optimizing the usage of constrained resources 603 such as energy and spectrum. 605 Another prerequisite is that an IP link can be established over the 606 radio with some guarantees in terms of service reliability, e.g., it 607 can be relied upon to transmit a packet within a bounded latency and 608 provides a guaranteed BER/PDR outside rare but existing transient 609 outage windows that can last from split seconds to minutes. The 610 radio layer can be programmed with abstract parameters, and can 611 return an abstract view of the state of the Link to help the Network 612 Layer forwarding decision (think DLEP from MANET). 614 How the radio interface manages its lower layers is out of control 615 and out of scope for RAW. In the same fashion, the non-RAW portion 616 along a loose Track is by definition out of control and out of scope 617 for RAW. Whether it is a single hop or a mesh is also unknown and 618 out of scope. 620 3.2. Routing Time Scale vs. Forwarding Time Scale 622 With DetNet, the Controller Plane Function that handles the routing 623 computation and maintenance (the PCE) can be centralized and can 624 reside outside the network. In a wireless mesh, the path to the PCE 625 can be expensive and slow, possibly going across the whole mesh and 626 back. Reaching to the PCE can also be slow in regards to the speed 627 of events that affect the forwarding operation at the radio layer. 629 Due to that cost and latency, the Controller Plane is not expected to 630 be sensitive/reactive to transient changes. The abstraction of a 631 link at the routing level is expected to use statistical metrics that 632 aggregate the behavior of a link over long periods of time, and 633 represent its properties as shades of gray as opposed to numerical 634 values such as a link quality indicator, or a boolean value for 635 either up or down. 637 +----------------+ 638 | Controller | 639 | [PCE] | 640 +----------------+ 641 ^ 642 | 643 Slow 644 | 645 _-._-._-._-._-._-. | ._-._-._-._-._-._-._-._-._-._-._-._- 646 _-._-._-._-._-._-._-. | _-._-._-._-._-._-._-._-._-._-._-._- 647 | 648 Expensive 649 | 650 .... | ....... 651 .... . | . ....... 652 .... v ... 653 .. A-------B-------C---D .. 654 ... / \ / / \ .. 655 . I ----M-------N--***-- E .. 656 .. \ \ / / ... 657 .. P--***--Q----------R .... 658 .. .... 659 . <----- Fast -------> .... 660 ....... .... 661 ................. 663 *** = flapping at this time 665 Figure 3: Time Scales 667 In the case of wireless, the changes that affect the forwarding 668 decision can happen frequently and often for short durations, e.g., a 669 mobile object moves between a transmitter and a receiver, and will 670 cancel the line of sight transmission for a few seconds, or a radar 671 measures the depth of a pool and interferes on a particular channel 672 for a split second. 674 There is thus a desire to separate the long term computation of the 675 route and the short term forwarding decision. In that model, the 676 routing operation computes a complex Track that enables multiple Non- 677 Equal Cost Multi-Path (N-ECMP) forwarding solutions, and leaves it to 678 the Data Plane to make the per-packet decision of which of these 679 possibilities should be used. 681 In the wired world, and more specifically in the context of Traffic 682 Engineering (TE), an alternate path can be used upon the detection of 683 a failure in the main path, e.g., using OAM in MPLS-TP or BFD over a 684 collection of SD-WAN tunnels. RAW formalizes a forwarding time scale 685 that is an order(s) of magnitude shorter than the controller plane 686 routing time scale, and separates the protocols and metrics that are 687 used at both scales. Routing can operate on long term statistics 688 such as delivery ratio over minutes to hours, but as a first 689 approximation can ignore flapping. On the other hand, the RAW 690 forwarding decision is made at the scale of the packet rate, and uses 691 information that must be pertinent at the present time for the 692 current transmission(s). 694 3.3. Wireless Tracks 696 The "6TiSCH Architecture" [6TiSCH-ARCHI] introduces the concept of 697 Track. RAW extends the concept to any wireless mesh technology, 698 including, e.g., Wi-Fi. A simple Track is composed of a direct 699 sequence of reserved hops to ensure the transmission of a single 700 packet from a source Node to a destination Node across a multihop 701 path. 703 A Complex Track provides multiple non-equal cost multipath (NECM) 704 forwarding solutions. The Complex Track enables to support multi- 705 path redundant forwarding by employing PRE functions [RFC8655] and 706 the ingress and within the Track. For example, a Complex Track may 707 branch off and rejoin over non-congruent segments. 709 In the context of RAW, some links or segments in the Track may be 710 reversible, meaning that they can be used in either direction. In 711 that case, an indication in the packet signals the direction of the 712 reversible links or segments that the packet traverses and thus 713 places a constraint that prevents loops from occuring. An indidual 714 packet follows a destination-oriented directed acyclic graph (DODAG) 715 towards a destination Node inside the Complex Track. 717 3.4. PAREO Functions 719 RAW may control whether and how to use packet replication and 720 elimination (PRE), Automatic Repeat reQuest (ARQ), Hybrid ARQ (HARQ) 721 that includes Forward Error Correction (FEC) and coding, and other 722 wireless-specific techniques such as overhearing and constructive 723 interferences, in order to increase the reliabiility and availability 724 of the end-to-end transmission. 726 Collectively, those function are called PAREO for Packet (hybrid) 727 ARQ, Replication, Elimination and Ordering. By tuning dynamically 728 the use of PAREO functions, RAW avoids the waste of critical 729 resources such as spectrum and energy while providing that the 730 guaranteed SLA, e.g., by adding redundancy only when a spike of loss 731 is observed. 733 In a nutshell, PAREO establishes several paths in a network to 734 provide redundancy and parallel transmissions to bound the end-to-end 735 delay to traverse the network. Optionally, promiscuous listening 736 between paths is possible, such that the Nodes on one path may 737 overhear transmissions along the other path. Considering the 738 scenario shown in Figure 4, many different paths are possible for S 739 to reach R. A simple way to benefit from this topology could be to 740 use the two independent paths via Nodes A, C, E and via B, D, F. But 741 more complex paths are possible by interleaving transmissions from 742 the lower level of the path to the upper level. 744 (A) -- (C) -- (E) 745 / \ 746 Ingress | | | Egress 747 \ / 748 (B) -- (D) -- (F) 750 Figure 4: A Ladder Shape with Two Parallel Paths 752 PAREO may also take advantage of the shared properties of the 753 wireless medium to compensate for the potential loss that is incurred 754 with radio transmissions. 756 For instance, when the source sends to Node A, Node B may listen 757 promiscuously and get a second chance to receive the frame without an 758 additional transmission. Note that B would not have to listen if it 759 already received that particular frame at an earlier timeslot in a 760 dedicated transmission towards B. 762 The PAREO model can be implemented in both centralized and 763 distributed scheduling approaches. In the centralized approach, a 764 Path Computation Element (PCE) scheduler calculates a Track and 765 schedules the communication. In the distributed approach, the Track 766 is computed within the network, and signaled in the packets, e.g., 767 using BIER-TE, Segment Routing, or a Source Routing Header. 769 3.4.1. Packet Replication 771 By employing a Packet Replication procedure, a Node forwards a copy 772 of each data packet to more than one successor. To do so, each Node 773 (i.e., Ingress and intermediate Node) sends the data packet multiple 774 times as separate unicast transmissions. For instance, in Figure 5, 775 the Ingress Node is transmitting the packet to both successors, nodes 776 A and B, at two different times. 778 ===> (A) => (C) => (E) === 779 // \\// \\// \\ 780 Ingress //\\ //\\ Egress 781 \\ // \\ // \\ // 782 ===> (B) => (D) => (F) === 784 Figure 5: Packet Replication 786 An example schedule is shown in Table 1. This way, the transmission 787 leverages with the time and spatial forms of diversity. 789 +=========+======+======+======+======+======+======+======+ 790 | Channel | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 791 +=========+======+======+======+======+======+======+======+ 792 | 0 | S->A | S->B | B->C | B->D | C->F | E->R | F->R | 793 +---------+------+------+------+------+------+------+------+ 794 | 1 | | A->C | A->D | C->E | D->E | D->F | | 795 +---------+------+------+------+------+------+------+------+ 797 Table 1: Packet Replication: Sample schedule 799 3.4.2. Packet Elimination 801 The replication operation increases the traffic load in the network, 802 due to packet duplications. This may occur at several stages inside 803 the Track, and to avoid an explosion of the number of copies, a 804 Packet Elimination procedure must be applied as well. To this aim, 805 once a Node receives the first copy of a data packet, it discards the 806 subsequent copies. 808 The logical functions of Replication and Elimination may be 809 collocated in an intermediate Node, the Node first eliminating the 810 redundant copies and then sending the packet exactly once to each of 811 the selected successors. 813 3.4.3. Promiscuous Overhearing 815 Considering that the wireless medium is broadcast by nature, any 816 neighbor of a transmitter may overhear a transmission. By employing 817 the Promiscuous Overhearing operation, the next hops have additional 818 opportunities to capture the data packets. In Figure 6, when Node A 819 is transmitting to its DP (Node C), the AP (Node D) and its sibling 820 (Node B) may decode this data packet as well. As a result, by 821 employing corellated paths, a Node may have multiple opportunities to 822 receive a given data packet. 824 ===> (A) ====> (C) ====> (E) ==== 825 // ^ | \\ \\ 826 Ingress | | \\ Egress 827 \\ | v \\ // 828 ===> (B) ====> (D) ====> (F) ==== 830 Figure 6: Unicast with Overhearing 832 3.4.4. Constructive Interference 834 Constructive Interference can be seen as the reverse of Promiscuous 835 Overhearing, and refers to the case where two senders transmit the 836 exact same signal in a fashion that the emitted symbols add up at the 837 receiver and permit a reception that would not be possible with a 838 single sender at the same PHY mode and the same power level. 840 Constructive Interference was proposed on 5G, Wi-Fi7 and even tested 841 on IEEE Std 802.14.5. The hard piece is to synchronize the senders 842 to the point that the signals are emitted at slightly different time 843 to offset the difference of propagation delay that corresponds to the 844 difference of distance of the transmitters to the receiver at the 845 speed of light to the point that the symbols are superposed long 846 enough to be recognizable. 848 4. The RAW Architecture 850 4.1. The RAW Conceptual Model 852 RAW inherits the conceptual model described in section 4 of the 853 DetNet Architecture [RFC8655]. RAW extends the DetNet service layer 854 to provide additional agility against transmission loss. 856 A RAW Network Plane may be strict or loose, depending on whether RAW 857 observes and takes actions on all hops or not. For instance, the 858 packets between two wireless entities may be relayed over a wired 859 infrastructure such as a Wi-Fi extended service set (ESS) or a 5G 860 Core; in that case, RAW observes and control the transmission over 861 the wireless first and last hops, as well as end-to-end metrics such 862 as latency, jitter, and delivery ratio. This operation is loose 863 since the structure and properties of the wired infrastructure are 864 ignored, and may be either controlled by other means such as DetNet/ 865 TSN, or neglected in the face of the wireless hops. 867 A Controller Plane Function (CPF) called the Path Computation Element 868 (PCE) [RFC4655] interacts with RAW Nodes over a Southbound API. The 869 RAW Nodes are DetNet relays that are capable of additional diversity 870 mechanisms and measurement functions related to the radio interface, 871 in particular the PAREO diversity mechanisms. 873 The PCE defines a complex Track between an Ingress End System and an 874 Egress End System, and indicates to the RAW Nodes where the PAREO 875 operations may be actionned in the Network Plane. The Track may be 876 expressed loosely to enable traversing a non-RAW subnetwork. In that 877 case, the expectation is that the non-RAW subnetwork can be neglected 878 in the RAW computation, that is, considered infinitely fast, reliable 879 and/or available in comparison with the links between RAW nodes. 881 CPF CPF CPF CPF 883 Southbound API 884 _-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._- 885 _-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._-._- 887 RAW --z RAW --z RAW --z RAW 888 z-- Node z-- Node z-- Node z-- Node --z 889 Ingress --z / / z-- Egress 890 End \ \ .. . End 891 Node ---z / / .. .. . z-- Node 892 z-- RAW --z RAW ( non-RAW ) -- RAW --z 893 Node z-- Node --- ( Nodes ) Node 894 ... . 895 --z wireless wired 896 z-- link --- link 898 Figure 7: RAW Nodes 900 The Link-Layer metrics are reported to the PCE in a time-aggregated, 901 e.g., statistical fashion. Example Link-Layer metrics include 902 typical Link bandwidth (the medium speed depends dynamically on the 903 PHY mode and the number of users sharing the spectrum) and average 904 and mean squared deviation of availability and reliability figures 905 such as Packet Delivery Ratio (PDR) over long periods of time. 907 Based on those metrics, the PCE installs the Track with enough 908 redundant forwarding solutions to ensure that the Network Plane can 909 reliably deliver the packets within a System Level Agreement (SLA) 910 associated to the flows that it transports. The SLA defines end-to- 911 end reliability and availability requirements, where reliability may 912 be expressed as a successful delivery in order and within a bounded 913 delay of at least one copy of a packet. 915 Depending on the use case and the SLA, the Track may comprise non-RAW 916 segments, either interleaved inside the Track, or all the way to the 917 Egress End Node (e.g., a server in the Internet). RAW observes the 918 Lower-Layer Links between RAW nodes (typically, radio links) and the 919 end-to-end Network Layer operation to decide at all times which of 920 the PAREO diversity schemes is actioned by which RAW Nodes. 922 Once a Track is established, per-segment and end-to-end reliability 923 and availability statistics are periodically reported to the PCE to 924 assure that the SLA can be met or have it recompute the Track if not. 926 4.2. The Path Selection Engine 928 RAW separates the path computation time scale at which a complex path 929 is recomputed from the path selection time scale at which the 930 forwarding decision is taken for one or a few packets (more in 931 Section 3.2). RAW operates at the path selection time scale. The 932 RAW problem is to decide, within the redundant solutions that are 933 proposed by the PCE, which will be used for each packet to provide a 934 Reliable and Available service while minimizing the waste of 935 constrained resources. 937 To that effect, RAW defines the Path Selection Engine (PSE) that is 938 the counter-part of the PCE to perform rapid local adjustments of the 939 forwarding tables within the diversity that the PCE has selected for 940 the Track. The PSE enables to exploit the richer forwarding 941 capabilities with PAREO and scheduled transmissions at a faster time 942 scale over the smaller domain that is the Track, in either a loose or 943 a strict fashion. 945 Compared to the PCE, the PSE operates on metrics that evolve faster, 946 but that needs to be advertised at a fast rate but only locally, 947 within the Track. The forwarding decision may also change rapidly, 948 but wiht a scope that is also contained within the Track, with no 949 visibility to the other Tracks and flows in the network. This is as 950 opposed to the PCE that needs to observe the whole network, and 951 optimize all the Tracks globally, which can only be done at a slow 952 pace and using long-term statistical metrics, as presented in 953 Table 2. 955 +===============+========================+===================+ 956 | | PCE (Not in Scope) | PSE (In Scope) | 957 +===============+========================+===================+ 958 | Operation | Centralized | Source-Routed or | 959 | | | Distributed | 960 +---------------+------------------------+-------------------+ 961 | Communication | Slow, expensive | Fast, local | 962 +---------------+------------------------+-------------------+ 963 | Time Scale | hours and above | seconds and below | 964 +---------------+------------------------+-------------------+ 965 | Network Size | Large, many Tracks to | Small, within one | 966 | | optimize globally | Track | 967 +---------------+------------------------+-------------------+ 968 | Considered | Averaged, Statistical, | Instant values / | 969 | Metrics | Shade of grey | boolean condition | 970 +---------------+------------------------+-------------------+ 972 Table 2: PCE vs. PSE 974 The PSE sits in the DetNet Service sub-Layer of Edge and Relay Nodes. 975 On the one hand, it operates on the packet flow, learning the Track 976 and path selection information from the packet, possibly making local 977 decision and retagging the packet to indicate so. On the other hand, 978 the PSE interacts with the lower layers and with its peers to obtain 979 up-to-date information about its radio links and the quality of the 980 overall Track, respectively, as illustrated in Figure 8. 982 | 983 packet | going 984 down the | stack 985 +==========v==========+=====================+=====================+ 986 | (iOAM + iCTRL) | (L2 Triggers, DLEP) | (oOAM) | 987 +==========v==========+=====================+=====================+ 988 | Learn from Learn from | 989 | packet tagging Maintain end-to-end | 990 +----------v----------+ Forwarding OAM packets | 991 | Forwarding decision < State +---------^-----------| 992 +----------v----------+ | Enrich or | 993 + Retag Packet | Learn abstracted > Regenerate | 994 | and Forward | metrics about Links | OAM packets | 995 +..........v..........+..........^..........+.........^.v.........+ 996 | Lower layers | 997 +..........v.....................^....................^.v.........+ 998 frame | sent Frame | L2 Ack oOAM | | packet 999 over | wireless In | In | | and out 1000 v | | v 1002 Figure 8: PSE 1004 4.3. RAW OAM 1006 The RAW OAM operation in the Network Plane observes either a full 1007 Track or subTracks that are being used at this time. This 1008 observeation feeds the RAW PSE that makes the decision on which PAREO 1009 function in actioned at which RAW Node, for one a small continuous 1010 series of packets. 1012 ... .. 1013 RAN 1 ----- ... .. ... 1014 / . .. .... 1015 +-------+ / . .. .... +------+ 1016 |Ingress|- . ..... |Egress| 1017 | End |------ RAN 2 -- . Internet ....---| End | 1018 |System |- .. ..... |System| 1019 +-------+ \ . ...... +------+ 1020 \ ... ... ..... 1021 RAN n -------- ... ..... 1023 <------------------> <--------------------> 1024 Observed by OAM Opaque to OAM 1026 Figure 9: Observed Links in Radio Access Protection 1028 In the case of a End-to-End Protection in a Wireless Mesh, the Track 1029 is strict and congruent with the path so all links are observed. 1030 Conversely, in the case of Radio Access Protection, the Track is 1031 Loose and in that case only the first hop is observed; the rest of 1032 the path is abstracted and considered infinitely reliable. 1034 In the case of the Radio Access Protection, only the first hop is 1035 protected; the loss of a packet that was sent over one of the 1036 possible first hops is attributed to that first hop, even if a 1037 particular loss effectively happens farther down the path. 1039 The Links that are not observed by OAM are opaque to it, meaning that 1040 the OAM information is carried across and possibly echoed as data, 1041 but there is no information capture in intermediate nodes. In the 1042 example above, the Internet is opaque and not controlled by RAW; 1043 still the RAW OAM measures the end-to-end latency and delivery ratio 1044 for packets sent via each if RAN 1, RAN 2 and RAN 3, and determines 1045 whether a packet should be sent over either or a collection of those 1046 access links. 1048 4.4. Flow Identification vs. Path Identification 1050 Section 4.7 of the DetNet Architecture [RFC8655] ties the app-flow 1051 identification which is an appliation layer concept with the network 1052 path identification that depends on the networking technology by 1053 "exporting of flow identification", e.g., to a MPLS label. 1055 With RAW, this exporting operation is injective but not bijective. 1056 e.g., a flow is fully placed within one RAW Track, but not all 1057 packets along that Track are necessarily part of the same flow. For 1058 instance, out-of-band OAM packets must circulate in the exact same 1059 fashion as the flows that they observe. It results that the flow 1060 identification that maps to to app-flow at the network layer must be 1061 separate from the path identification that is used to forward a 1062 packet. 1064 Section 3.4 of the DetNet data-plane framework [DetNet-DP] indicates 1065 that for a DetNet IP Data Plane, a flow is identified by an IPv6 1066 6-tuple. With RAW, that 6-tuple is not what indicates the Track, in 1067 other words, the flow ID is not the Track ID. 1069 For instance, the 6TiSCH Architecture [6TiSCH-ARCHI] uses a 1070 combination of the address of the Egress End System and an instance 1071 identifier in a Hop-by-hop option to indicate a Track. This way, if 1072 a packet "escapes" the Track, it will reach the Track Egress point 1073 through normal routing and be treated at the service layer through, 1074 say, elimination and reordering. 1076 The RAW service includes forwarding over a subset of the Links that 1077 form the Track (a subTrack). Packets from the same or a different 1078 flow that are routed through the same Track will not necessarily 1079 traverse the same Links. The PSE selects a subTrack for a packet 1080 based on the links that are preferred and those that should be 1081 avoided at this time. 1083 Each packet is forwarded within the subTrack that provides the best 1084 adequation with the SLA of the flow and the energy and bandwidth 1085 constraints of the network. 1087 Flow 1 (6-tuple) ----+ 1088 | 1089 Flow 2 (6-tuple) ---+ | 1090 | | 1091 OAM -----------+ | | 1092 | | | 1093 | | | 1094 | | | | | 1095 | v v v | 1096 | | 1097 +---------+---------+ 1098 | 1099 | 1100 Track i (Egress IP Address, instanceId) 1101 | 1102 | 1103 | 1104 +---------+-----+--....-------+ 1105 | | | 1106 | | | 1107 subTrack 1 subTrack 2 subTrack n 1108 | | | 1109 | | | 1110 V V V 1111 +-----------------------------------+ 1112 | | 1113 | Destination | 1114 | | 1115 +-----------------------------------+ 1117 Figure 10: Flow Injection 1119 With 6TiSCH, packets are tagged with the same (destination address, 1120 instance ID) will experience the same RAW service regardless of the 1121 IPv6 6-tuple that indicates the flow. The forwarding does not depend 1122 on whether the packets transport application flows or OAM. In the 1123 generic case, the Track or the subTrack can be signaled in the packet 1124 through other means, e.g., encoded in the suffix of the destination 1125 address as a Segment Routing Service Instruction [SR-ARCHI], or 1126 leveraging Bit Index Explicit Replication [BIER] Traffic Engineering 1127 [BIER-TE]. 1129 4.5. Source-Routed vs. Distributed Forwarding Decision 1131 Within a large routed topology, the route-over mesh operation builds 1132 a particular complex Track with one source and one or more 1133 destinations; within the Track, packets may follow different paths 1134 and may be subject to RAW forwarding operations that include 1135 replication, elimination, retries, overhearing and reordering. 1137 The RAW forwarding decisions include the selection of points of 1138 replication and elimination, how many retries can take place, and a 1139 limit of validity for the packet beyond which the packet should be 1140 destroyed rather than forwarded uselessly further down the Track. 1142 The decision to apply the RAW techniques must be done quickly, and 1143 depends on a very recent and precise knowledge of the forwarding 1144 conditions within the complex Track. There is a need for an 1145 observation method to provide the RAW Data Plane with the specific 1146 knowledge of the state of the Track for the type of flow of interest 1147 (e.g., for a QoS level of interest). To observe the whole Track in 1148 quasi real time, RAW considers existing tools such as L2-triggers, 1149 DLEP, BFD and leverages in-band and out-of-band OAM to capture and 1150 report that information to the PSE. 1152 One possible way of making the RAW forwarding decisions within a 1153 Track is to position a unique PSE at the Ingress and express its 1154 decision in-band in the packet, which requires the explicit signaling 1155 of the subTrack within the Track. In that case, the RAW forwarding 1156 operation along the Track is encoded by the source, e.g., by 1157 indicating the subTrack in the Segment Routing (SRv6) Service 1158 Instruction, or by leveraging BIER-TE such as done with [BIER-PREF]. 1160 The alternate way is to operate the PSE in each forwarding Node, 1161 which makes the RAW forwarding decisions for a packet on its own, 1162 based on its knowledge of the expectation (timeliness and 1163 reliability) for that packet and a recent observation of the rest of 1164 the way across the possible paths based on OAM. Information about 1165 the desired service should be placed in the packet and matched with 1166 the forwarding Node's capabilities and policies. 1168 In either case, a per-track/subTrack state is installed in all the 1169 intermediate Nodes to recognize the packets that are following a 1170 Track and determine the forwarding operation to be applied. 1172 4.6. Encapsulation and Decapsulation 1174 In the generic case where the Track Ingress Node is not the source of 1175 the Packet, the Ingress Node needs to encapsulate IP-in-IP to ensure 1176 that the Destination IP Address is that of the Egress Node and that 1177 the necessary Headers (Routing Header, Segment Routing Header and/or 1178 Hop-By-Hop Header) can be added to the packet to signal the Track or 1179 the subTrack, conforming [IPv6] that discourages the insertion of a 1180 Header on the fly. 1182 In the specific case where the Ingress Node is the source of the 1183 packet, the encapsulation can be avoided, provided that the source 1184 adds the necessary headers and that the destination is set to the 1185 Egress Node. Forwarding to a final destination beyond the Egress 1186 Node is possible, e.g., with a Segment Routing Header that signals 1187 the rest of the way. In that case a Hop-by-Hop Header is not 1188 recommmended since its validity is within the Track only. 1190 5. Security Considerations 1192 RAW uses all forms of diversity including radio technology and 1193 physical path to increase the reliability and availability in the 1194 face of unpredictable conditions. While this is not done 1195 specifically to defeat an attacker, the amount of diversity used in 1196 RAW makes an attack harder to achieve. 1198 5.1. Forced Access 1200 RAW will typically select the cheapest collection of links that 1201 matches the requested SLA, for instance, leverage free WI-Fi vs. paid 1202 3GPP access. By defeating the cheap connectivity (e.g., PHY-layer 1203 interference) the attacker can force an End System to use the paid 1204 access and increase the cost of the transmission for the user. 1206 6. IANA Considerations 1208 This document has no IANA actions. 1210 7. Contributors 1212 Xavi Vilajosana: Wireless Networks Research Lab, Universitat Oberta 1213 de Catalunya 1215 Remous-Aris Koutsiamanis: IMT Atlantique 1217 Nicolas Montavont: IMT Atlantique 1219 8. Acknowledgments 1221 TBD 1223 9. References 1225 9.1. Normative References 1227 [6TiSCH-ARCHI] 1228 Thubert, P., Ed., "An Architecture for IPv6 over the Time- 1229 Slotted Channel Hopping Mode of IEEE 802.15.4 (6TiSCH)", 1230 RFC 9030, DOI 10.17487/RFC9030, May 2021, 1231 . 1233 [RAW-TECHNOS] 1234 Thubert, P., Cavalcanti, D., Vilajosana, X., Schmitt, C., 1235 and J. Farkas, "Reliable and Available Wireless 1236 Technologies", Work in Progress, Internet-Draft, draft- 1237 ietf-raw-technologies-01, 19 February 2021, 1238 . 1241 [RAW-USE-CASES] 1242 Papadopoulos, G. Z., Thubert, P., Theoleyre, F., and C. J. 1243 Bernardos, "RAW use cases", Work in Progress, Internet- 1244 Draft, draft-ietf-raw-use-cases-01, 21 February 2021, 1245 . 1247 [RFC4655] Farrel, A., Vasseur, J.-P., and J. Ash, "A Path 1248 Computation Element (PCE)-Based Architecture", RFC 4655, 1249 DOI 10.17487/RFC4655, August 2006, 1250 . 1252 [BFD] Katz, D. and D. Ward, "Bidirectional Forwarding Detection 1253 (BFD)", RFC 5880, DOI 10.17487/RFC5880, June 2010, 1254 . 1256 [RFC6291] Andersson, L., van Helvoort, H., Bonica, R., Romascanu, 1257 D., and S. Mansfield, "Guidelines for the Use of the "OAM" 1258 Acronym in the IETF", BCP 161, RFC 6291, 1259 DOI 10.17487/RFC6291, June 2011, 1260 . 1262 [RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases", 1263 RFC 8578, DOI 10.17487/RFC8578, May 2019, 1264 . 1266 [IPv6] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1267 (IPv6) Specification", STD 86, RFC 8200, 1268 DOI 10.17487/RFC8200, July 2017, 1269 . 1271 [SR-ARCHI] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L., 1272 Decraene, B., Litkowski, S., and R. Shakir, "Segment 1273 Routing Architecture", RFC 8402, DOI 10.17487/RFC8402, 1274 July 2018, . 1276 [BIER] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., 1277 Przygienda, T., and S. Aldrin, "Multicast Using Bit Index 1278 Explicit Replication (BIER)", RFC 8279, 1279 DOI 10.17487/RFC8279, November 2017, 1280 . 1282 [RFC8175] Ratliff, S., Jury, S., Satterwhite, D., Taylor, R., and B. 1283 Berry, "Dynamic Link Exchange Protocol (DLEP)", RFC 8175, 1284 DOI 10.17487/RFC8175, June 2017, 1285 . 1287 [RFC8557] Finn, N. and P. Thubert, "Deterministic Networking Problem 1288 Statement", RFC 8557, DOI 10.17487/RFC8557, May 2019, 1289 . 1291 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 1292 "Deterministic Networking Architecture", RFC 8655, 1293 DOI 10.17487/RFC8655, October 2019, 1294 . 1296 9.2. Informative References 1298 [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791, 1299 DOI 10.17487/RFC0791, September 1981, 1300 . 1302 [TE] Awduche, D., Chiu, A., Elwalid, A., Widjaja, I., and X. 1303 Xiao, "Overview and Principles of Internet Traffic 1304 Engineering", RFC 3272, DOI 10.17487/RFC3272, May 2002, 1305 . 1307 [STD 62] Harrington, D., Presuhn, R., and B. Wijnen, "An 1308 Architecture for Describing Simple Network Management 1309 Protocol (SNMP) Management Frameworks", STD 62, RFC 3411, 1310 DOI 10.17487/RFC3411, December 2002, 1311 . 1313 [RFC4090] Pan, P., Ed., Swallow, G., Ed., and A. Atlas, Ed., "Fast 1314 Reroute Extensions to RSVP-TE for LSP Tunnels", RFC 4090, 1315 DOI 10.17487/RFC4090, May 2005, 1316 . 1318 [FRR] Shand, M. and S. Bryant, "IP Fast Reroute Framework", 1319 RFC 5714, DOI 10.17487/RFC5714, January 2010, 1320 . 1322 [RLFA-FRR] Bryant, S., Filsfils, C., Previdi, S., Shand, M., and N. 1323 So, "Remote Loop-Free Alternate (LFA) Fast Reroute (FRR)", 1324 RFC 7490, DOI 10.17487/RFC7490, April 2015, 1325 . 1327 [DetNet-DP] 1328 Varga, B., Ed., Farkas, J., Berger, L., Malis, A., and S. 1329 Bryant, "Deterministic Networking (DetNet) Data Plane 1330 Framework", RFC 8938, DOI 10.17487/RFC8938, November 2020, 1331 . 1333 [BIER-PREF] 1334 Thubert, P., Eckert, T., Brodard, Z., and H. Jiang, "BIER- 1335 TE extensions for Packet Replication and Elimination 1336 Function (PREF) and OAM", Work in Progress, Internet- 1337 Draft, draft-thubert-bier-replication-elimination-03, 3 1338 March 2018, . 1341 [DetNet-IP-OAM] 1342 Mirsky, G., Chen, M., and D. Black, "Operations, 1343 Administration and Maintenance (OAM) for Deterministic 1344 Networks (DetNet) with IP Data Plane", Work in Progress, 1345 Internet-Draft, draft-ietf-detnet-ip-oam-02, 30 March 1346 2021, 1347 . 1349 [RAW-5G] Farkas, J., Dudda, T., Shapin, A., and S. Sandberg, "5G - 1350 Ultra-Reliable Wireless Technology with Low Latency", Work 1351 in Progress, Internet-Draft, draft-farkas-raw-5g-00, 1 1352 April 2020, 1353 . 1355 [BIER-TE] Eckert, T., Cauchie, G., and M. Menth, "Tree Engineering 1356 for Bit Index Explicit Replication (BIER-TE)", Work in 1357 Progress, Internet-Draft, draft-ietf-bier-te-arch-09, 30 1358 October 2020, 1359 . 1361 [IPoWIRELESS] 1362 Thubert, P., "IPv6 Neighbor Discovery on Wireless 1363 Networks", Work in Progress, Internet-Draft, draft- 1364 thubert-6man-ipv6-over-wireless-09, 17 May 2021, 1365 . 1368 [NASA] Adams, T., "RELIABILITY: Definition & Quantitative 1369 Illustration", . 1372 [MANET] IETF, "Mobile Ad hoc Networking", 1373 . 1375 [detnet] IETF, "Deterministic Networking", 1376 . 1378 [SPRING] IETF, "Source Packet Routing in Networking", 1379 . 1381 [BIER] IETF, "Bit Indexed Explicit Replication", 1382 . 1384 [BFD] IETF, "Bidirectional Forwarding Detection", 1385 . 1387 [CCAMP] IETF, "Common Control and Measurement Plane", 1388 . 1390 Authors' Addresses 1392 Pascal Thubert (editor) 1393 Cisco Systems, Inc 1394 Building D 1395 45 Allee des Ormes - BP1200 1396 06254 MOUGINS - Sophia Antipolis 1397 France 1399 Phone: +33 497 23 26 34 1400 Email: pthubert@cisco.com 1401 Georgios Z. Papadopoulos 1402 IMT Atlantique 1403 Office B00 - 114A 1404 2 Rue de la Chataigneraie 1405 35510 Cesson-Sevigne - Rennes 1406 France 1408 Phone: +33 299 12 70 04 1409 Email: georgios.papadopoulos@imt-atlantique.fr 1411 Lou Berger 1412 LabN Consulting, L.L.C. 1413 United States of America 1415 Email: lberger@labn.net 1417 Rex Buddenberg 1418 CA 1419 United States of America 1421 Email: buddenbergr@gmail.com