idnits 2.17.1 draft-pthubert-raw-architecture-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == There is 1 instance of lines with non-ascii characters in the document. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 702: '...ry hop of the forwarding paths MUST be...' RFC 2119 keyword, line 736: '... SHOULD be applied at each RPL DODAG...' Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 827 has weird spacing: '...-- Node z-- ...' == Line 832 has weird spacing: '... Node z-- ...' -- The document date (18 May 2020) is 1438 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'Function' is mentioned on line 601, but not defined == Outdated reference: A later version (-30) exists of draft-ietf-6tisch-architecture-28 == Outdated reference: A later version (-05) exists of draft-thubert-raw-technologies-04 == Outdated reference: A later version (-04) exists of draft-bernardos-raw-use-cases-03 -- Obsolete informational reference (is this intentional?): RFC 3272 (ref. 'TE') (Obsoleted by RFC 9522) == Outdated reference: A later version (-03) exists of draft-mirsky-detnet-ip-oam-02 Summary: 1 error (**), 0 flaws (~~), 9 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 RAW P. Thubert, Ed. 3 Internet-Draft Cisco Systems 4 Intended status: Informational G.Z. Papadopoulos 5 Expires: 19 November 2020 IMT Atlantique 6 R. Buddenberg 7 18 May 2020 9 Reliable and Available Wireless Architecture/Framework 10 draft-pthubert-raw-architecture-02 12 Abstract 14 Due to uncontrolled interferences, including the self-induced 15 multipath fading, deterministic networking can only be approached on 16 wireless links. The radio conditions may change -way- faster than a 17 centralized routing can adapt and reprogram, in particular when the 18 controller is distant and connectivity is slow and limited. RAW 19 separates the routing time scale at which a complex path is 20 recomputed from the forwarding time scale at which the forwarding 21 decision is taken for an individual packet. RAW operates at the 22 forwarding time scale. The RAW problem is to decide, within the 23 redundant solutions that are proposed by the routing, which will be 24 used for each individual packet to provide a DetNet service while 25 minimizing the waste of resources. 27 Status of This Memo 29 This Internet-Draft is submitted in full conformance with the 30 provisions of BCP 78 and BCP 79. 32 Internet-Drafts are working documents of the Internet Engineering 33 Task Force (IETF). Note that other groups may also distribute 34 working documents as Internet-Drafts. The list of current Internet- 35 Drafts is at https://datatracker.ietf.org/drafts/current/. 37 Internet-Drafts are draft documents valid for a maximum of six months 38 and may be updated, replaced, or obsoleted by other documents at any 39 time. It is inappropriate to use Internet-Drafts as reference 40 material or to cite them other than as "work in progress." 42 This Internet-Draft will expire on 19 November 2020. 44 Copyright Notice 46 Copyright (c) 2020 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 51 license-info) in effect on the date of publication of this document. 52 Please review these documents carefully, as they describe your rights 53 and restrictions with respect to this document. Code Components 54 extracted from this document must include Simplified BSD License text 55 as described in Section 4.e of the Trust Legal Provisions and are 56 provided without warranty as described in the Simplified BSD License. 58 Table of Contents 60 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 61 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 62 3. Related Work at The IETF . . . . . . . . . . . . . . . . . . 6 63 4. Use Cases and Requirements Served . . . . . . . . . . . . . . 6 64 4.1. Radio Access Protection . . . . . . . . . . . . . . . . . 7 65 4.2. End-to-End Protection in a Wireless Mesh . . . . . . . . 7 66 5. RAW Considerations . . . . . . . . . . . . . . . . . . . . . 8 67 5.1. Reliability and Availability . . . . . . . . . . . . . . 8 68 5.1.1. High Availability Engineering Principles . . . . . . 8 69 5.1.2. Applying Reliability Concepts to Networking . . . . . 10 70 5.1.3. Reliability in the Context of RAW . . . . . . . . . . 11 71 5.2. RAW Prerequisites . . . . . . . . . . . . . . . . . . . . 12 72 5.3. Routing Time Scale vs. Forwarding Time Scale . . . . . . 13 73 6. RAW Architecture Elements . . . . . . . . . . . . . . . . . . 14 74 6.1. PAREO Functions . . . . . . . . . . . . . . . . . . . . . 14 75 6.1.1. Packet Replication . . . . . . . . . . . . . . . . . 15 76 6.1.2. Packet Elimination . . . . . . . . . . . . . . . . . 16 77 6.1.3. Promiscuous Overhearing . . . . . . . . . . . . . . . 16 78 6.1.4. Constructive Interference . . . . . . . . . . . . . . 17 79 6.2. Wireless Tracks . . . . . . . . . . . . . . . . . . . . . 17 80 7. RAW Architecture . . . . . . . . . . . . . . . . . . . . . . 17 81 7.1. PCE vs. PSE . . . . . . . . . . . . . . . . . . . . . . . 19 82 7.2. RAW OAM . . . . . . . . . . . . . . . . . . . . . . . . . 20 83 7.3. Source-Routed vs. Distributed Forwarding Decision . . . . 20 84 7.4. Flow Identification . . . . . . . . . . . . . . . . . . . 21 85 8. Security Considerations . . . . . . . . . . . . . . . . . . . 22 86 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 87 10. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 22 88 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 23 89 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 90 12.1. Normative References . . . . . . . . . . . . . . . . . . 23 91 12.2. Informative References . . . . . . . . . . . . . . . . . 24 92 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 94 1. Introduction 96 Bringing determinism in a packet network means eliminating the 97 statistical effects of multiplexing that result in probabilistic 98 jitter and loss. This can be approached with a tight control of the 99 physical resources to maintain the amount of traffic within a 100 budgetted volume of data per unit of time that fits the physical 101 capabilities of the underlying technology, and the use of time-shared 102 resources (bandwidth and buffers) per circuit, and/or by shaping and/ 103 or scheduling the packets at every hop. 105 Wireless networks operate on a shared medium where uncontrolled 106 interference, including the self-induced multipath fading, adds 107 another dimension to the statistical effects that affect the 108 delivery. Scheduling transmissions can alleviate those effects by 109 leveraging diversity in the spatial, time, code, and frequency 110 domains, and provide a Reliable and Available service while 111 preserving energy and optimizing the use of the shared spectrum. 113 Deterministic Networking is an attempt to mostly eliminate packet 114 loss for a committed bandwidth with a guaranteed worst-case end-to- 115 end latency, even when co-existing with best-effort traffic in a 116 shared network. This innovation is enabled by recent developments in 117 technologies including IEEE 802.1 TSN (for Ethernet LANs) and IETF 118 DetNet (for wired IP networks). It is getting traction in various 119 industries including manufacturing, online gaming, professional A/V, 120 cellular radio and others, making possible many cost and performance 121 optimizations. 123 The "Deterministic Networking Architecture" [RFC8655] is composed of 124 three planes: the Application (User) Plane, the Controller Plane, and 125 the Network Plane. Reliable and Available Wireless (RAW) extends RAW 126 to focus on issues that are mostly a co"ern on wireless links, and 127 inherits the architecture and the planes. A RAW Network Plane is 128 thus a Network Plane inherited by RAW from DetNet, composed of one or 129 multiple hops of homogeneous or heterogeneous technologies, e.g. a 130 Wi-Fi6 Mesh or one-hop CBRS access links federated by a 5G backhaul. 132 RAW networking aims at providing highly available and reliable end- 133 to-end performances in a network with scheduled wireless segments. 134 Uncontrolled interference and transmission obstacles may impede the 135 transmission, and techniques such as beamforming with Multi-User MIMO 136 can only alleviate some of those issues, so the term "deterministic" 137 is usually not associated with short range radios, in particular in 138 the ISM band. This uncertainty places limits to the amount of 139 traffic that can be transmitted on a link while conforming to a RAW 140 Service Level Agreement (SLA) that may vary rapidly. 142 The wireless and wired media are fundamentally different at the 143 physical level, and while the generic "Deterministic Networking 144 Problem Statement" [RFC8557] applies to both the wired and the 145 wireless media, the methods to achieve RAW must extend those used to 146 support time-sensitive networking over wires, as a RAW solution has 147 to address less consistent transmissions, energy conservation and 148 shared spectrum efficiency. 150 The development of RAW technologies has been lagging behind 151 deterministic efforts for wired systems both at the IEEE and the 152 IETF. But recent efforts at the IEEE and 3GPP indicate that wireless 153 is finally catching up at the lower layer and that it is now possible 154 for the IETF to extend DetNet for wireless segments that are capable 155 of scheduled wireless transmissions. 157 The intent for RAW is to provide DetNet elements that are specialized 158 for short range radios. From this inheritance, RAW stays agnostic to 159 the radio layer underneath though the capability to schedule 160 transmissions is assumed. How the PHY is programmed to do so, and 161 whether the radio is single-hop or meshed, are unknown at the IP 162 layer and not part of the RAW abstraction. 164 Still, in order to focus on real-worlds issues and assert the 165 feasibility of the proposed capabilities, RAW will focus on selected 166 technologies that can be scheduled at the lower layers: IEEE Std. 167 802.15.4 timeslotted channel hopping (TSCH), 3GPP 5G ultra-reliable 168 low latency communications (URLLC), IEEE 802.11ax/be where 802.11be 169 is extreme high throughput (EHT), and L-band Digital Aeronautical 170 Communications System (LDACS). See [RAW-TECHNOS] for more. 172 The establishment of a path is not in-scope for RAW. It may be the 173 product of a centralized Controller Plane as described for DetNet. 174 As opposed to wired networks, the action of installing a path over a 175 set of wireless links may be very slow relative to the speed at which 176 the radio conditions vary, and it makes sense in the wireless case to 177 provide redundant forwarding solutions along a complex path and to 178 leave it to the Network Plane to select which of those forwarding 179 solutions are to be used for a given packet based on the current 180 conditions. 182 RAW distinguishes the longer time scale at which routes are computed 183 from the the shorter forwarding time scale where per-packet decisions 184 are made. RAW operates at the forwarding time scale on one DetNet 185 flow over one path that is preestablished and installed by means 186 outside of the scope of RAW. The scope of the RAW WG comprises 187 Network plane protocol elements such as Operations, Administration 188 and Maintenance (OAM) and in-band control to improve the RAW 189 operation at the Service and at the forwarding sub-layers. RAW 190 controls whether to use packet replication, Automatic Repeat reQuest 191 (ARQ), Hybrid ARQ (HARQ) that includes Forward Error Correction (FEC) 192 and coding, with a constraint to limit the use of redundancy as is 193 really needed, e.g., when a spike of loss is observed. This is 194 discussed in more details in Section 5.3 and the next sections. 196 2. Terminology 198 RAW reuses terminology defined for DetNet in the "Deterministic 199 Networking Architecture" [RFC8655], e.g., PREOF for Packet 200 Replication, Elimination and Ordering Functions. 202 RAW also reuses terminology defined for 6TiSCH in [6TiSCH-ARCH] such 203 as the term Track. 6TiSCH defined a Track as a complex path with 204 associated PAREO operations. 206 RAW uses the term OAM as defined in [RFC6291]. 208 RAW defines the following terms: 210 PAREO: Packet (hybrid) ARQ, Replication, Elimination and Ordering. 211 PAREO is a superset Of DetNet's PREOF that includes radio-specific 212 techniques such as short range broadcast, MUMIMO, constructive 213 interference and overhearing, which can be leveraged separately or 214 combined to increase the reliability. 216 Flapping: In the context of RAW, a link flaps when the wireless 217 connectivity is interrupted for short transient times, typically 218 of a subsecond duration. 220 In the context of the RAW work, Reliability and Availability are 221 defined as follows: 223 Reliability: Reliability is a measure of the probability that an 224 item will perform its intended function for a specified interval 225 under stated conditions. For RAW, the service that is expected is 226 delivery within a bounded latency and a failure is when the packet 227 is either lost or delivered too late. RAW expresses reliability 228 in terms of Mean Time Between Failure (MTBF) and Maximum 229 Consecutive Failures (MCF). More in [NASA]. 231 Availability: Availability is a measure of the relative amount of 232 time where a path operates in stated condition, in other words 233 (uptime)/(uptime+downtime). Because a serial wireless path may 234 not be good enough to provide the required availability, and even 235 2 parallel paths may not be over a longer period of time, the RAW 236 availability implies a path that is a lot more complex than what 237 DetNet typically envisages (a Track). 239 3. Related Work at The IETF 241 RAW intersects with protocols or practices in development at the IETF 242 as follows: 244 * The Dynamic Link Exchange Protocol (DLEP) [RFC8175] from [MANET] 245 can be leveraged at each hop to derive generic radio metrics 246 (e.g., based on LQI, RSSI, queueing delays and ETX) on individual 247 hops. 249 * OAM work at [detnet] such as [DetNet-IP-OAM] for the case of the 250 IP Data Plane observes the state of DetNet paths, typically MPLS 251 and IPv6 pseudowires [DetNet-DP-FW], in the direction of the 252 traffic. RAW needs feedback that flows on the reverse path and 253 gathers instantaneous values from the radio receivers at each hop 254 to inform back the source and replicating relays so they can make 255 optimized forwarding decisions. The work named ICAN may be 256 related as well. 258 * [BFD] detect faults in the path between an ingress and an egress 259 forwarding engines, but is unaware of the complexity of a path 260 with replication, and expects bidirectionality. BFD considers 261 delivery as success whereas with RAW the bounded latency can be as 262 important as the delivery itself. 264 * [SPRING] and [BIER] define in-band signaling that influences the 265 routing when decided at the head-end on the path. There's already 266 one RAW-related draft at BIER [BIER-PREF] more may follow. RAW 267 will need new in-band signaling when the decision is distributed, 268 e.g., required chances of reliable delivery to destination within 269 latency. This signaling enables relays to tune retries and 270 replication to meet the required SLA. 272 * [CCAMP] defines protocol-independent metrics and parameters 273 (measurement attributes) for describing links and paths that are 274 required for routing and signaling in technology-specific 275 networks. RAW would be a source of requirements for CCAMP to 276 define metrics that are significant to the focus radios. 278 4. Use Cases and Requirements Served 280 [RFC8578] presents a number of wireless use cases including Wireless 281 for Industrial Applications, Pro-Audio and SmartGrid. 282 [RAW-USE-CASES] adds a number of use cases that demonstrate the need 283 for RAW capabilities for new applications such as Pro-Gaming and 284 drones. The use cases can be abstracted in two families, Loose 285 Tracks, e.g., for first op Radio Access Protection and Strict Tracks, 286 e.g., for End-to-End Protection in a wireless mesh. 288 4.1. Radio Access Protection 290 To maintain the committed reliability at all times, a wireless host 291 may use more than one Radio Access Network (RAN) in parallel. 293 *** ** 294 RAN 1 ----- *** ** *** 295 / * ** **** 296 +----+ / * ** **** 297 | |- * ***** 298 |Host|--zzz- RAN 2 -- * Internet ***** 299 | |- * ***** 300 +----+ $$รน * ******* 301 \ *** *** ***** 302 RAN n -------- *** ***** 304 zzz = flapping now $$$ expensive 306 Figure 1: Radio Access Protection 308 The RANs may be heterogeneous, e.g., 5G [I-D.farkas-raw-5g] and Wi-Fi 309 [RAW-TECHNOS] for high-speed communication, in which case a Layer-3 310 abstraction becomes useful to select which of the RANs are used at a 311 particular point of time, and the amount of traffic that is 312 distributed over each RAN. 314 The idea is that the rest of the path to the destination(s) is 315 protected separately (e.g., uses non-congruent paths) and/or is a lot 316 more reliable, e.g., wired. In that case, RAW observes reliability 317 of the path through each of the RANs but only operates on the first 318 hop. 320 4.2. End-to-End Protection in a Wireless Mesh 322 In radio technologies that support mesh networking (e.g., Wi-Fi and 323 TSCH), a Track is a complex path with distributed PAREO capabilities. 324 In that case, RAW operates through the multipath and makes decisions 325 either at the Ingress or at every hop (more in Section 6.2). 327 A-------B-------C-----D 328 / \ / / \ 329 Ingress ----M-------N--zzzzz--- Egress 330 \ \ / / 331 P--zzz--Q-------------R 333 zzz = flapping now 335 Figure 2: End-to-End Protection 337 The Protection may be imposed by the source based on end-to-end OAM, 338 or performed hop-by-hop, in which case the OAM must enables the 339 intermediate Nodes to estimate the quality of the rest of the 340 feasible paths in the sub-Track to the destination. 342 5. RAW Considerations 344 5.1. Reliability and Availability 346 5.1.1. High Availability Engineering Principles 348 The reliability criteria of a critical system pervade through its 349 elements, and if the system comprises a data network then the data 350 network is also subject to the inherited reliability and availability 351 criteria. It is only natural to consider the art of high 352 availability engineering and apply it to wireless communicaitons in 353 the context of RAW. 355 There are three principles [pillars] of high availability 356 engineering: 358 1. elimination of single points of failure 359 2. reliable crossover 360 3. prompt detection of failures as they occur. 362 These principles are common to all high availability systems, not 363 just ones with Internet technology at the center. Examples of both 364 non-Internet and Internet are included. 366 5.1.1.1. Elimination of Single Points of Failure 368 Physical and logical components in a system happen to fail, either as 369 the effect of wear and tear, when used beyond acceptable limits, or 370 due to a software bug. It is necessary to decouple component failure 371 from system failure to avoid the latter. This allows failed 372 components to be restored while the rest of the system continues to 373 function. 375 A non-Internet example is a standby generator available to power the 376 system on failure of grid power. An Internet example is more than 377 one communication several non-congruent link/path between Nodes in a 378 routable network. 380 There is a rather open-ended issue over alternate routes -- for 381 example, when links are cabled through the same conduit, they form a 382 shared risk link group (SRLG), and will share the same fate if the 383 bundle is cut. Just how distributed the infrastructure is a matter 384 of discussion; there is no single right answer. It should be noted 385 that intermediate Nodes such as routers, switches, and the air medium 386 itself can become single points of failure; this must be avoided, 387 using link- and Node-disjoint paths, and, for RAW, a high degree of 388 diversity in the transmissions over the air. 390 From an economics standpoint, executing this principle properly 391 generally increases capitalization expense because of the redundant 392 equipment. In a constrained network where the waste of energy and 393 bandwidth should be minimized, an excessive use of redundant links 394 must be avoided; for RAW this means that the extra bandwidth must 395 only be used as a replacement of that lost due to a failure. 397 5.1.1.2. Reliable Crossover 399 Having a backup equipment has a limited value unless it can be 400 reliably switched into use within the down-time parameters. 402 Using the backup generator example: one that does not automatically 403 sense grid power failure, start itself, and place itself on line does 404 not represent reliable crossover. 406 Routers and IGPs execute reliable crossover continuously because the 407 routers will use any alternate routes that are available [RFC0791]. 408 This is due to the stateless nature of IP datagrams and the 409 dissociation of the datagrams from the forwarding routes they take. 410 The "IP Fast Reroute Framework" [FRR] analyzes mechanisms for fast 411 failure detection and path repair for IP Fast-Reroute, and discusses 412 the case of multiple failures and SRLG. Examples of FRR techniques 413 include Remote Loop-Free Alternate [RLFA-FRR] and backup label- 414 switched path (LSP) tunnels for the local repair of LSP tunnels using 415 RSVP-TE [RFC4090]. 417 The DetNet PREOF leverages 1+1 redundancy whereby a packet is sent 418 twice, over non-congruent paths. This avoids the gap during the fast 419 reroute operation, but doubles the traffic in the network. In the 420 case of RAW, the expectation is that multiple transient faults may 421 happen in overlapping time windows, in which case the 1+1 redundancy 422 with delayed reestablishment of the second path will not provide the 423 required guarantees. The Data Plane must be configured with a 424 sufficient degree of redundancy to select an alternate redundat path 425 immediately upon a fault, without the need for a slow intervention 426 from the controller plane. 428 5.1.1.3. Prompt Notification of Failures 430 The execution of the two above principles is likely to render a 431 system where the user will rarely see a failure. But someone needs 432 to in order to direct maintenance. 434 There are many reasons for system monitoring (FCAPS for fault, 435 configuration, accounting, performance, security is a handy mental 436 checklist) but fault monitoring is sufficient reason [STD 62] 437 describes how to use SNMP to observe and correct long-term faults. 438 "Overview and Principles of Internet Traffic Engineering" [TE] 439 discusses the importance of measurement for network protection, and 440 provides abstract an method for network survivability with the 441 analysis of a traffic matrix as observed by SNMP, probing techniques, 442 FTP, IGP link state advertisements, and more. 444 Using the art of SNMP, the above described backup generator would 445 include an SNMP agent that can report the status of the generator 446 (get messages) on demand, and report changes in status (e.g. startup, 447 amount of fuel in the tank) (trap messages). 449 Those measurements are needed in the context of RAW to inform the 450 controller and make the long term reactive decision to rebuild a 451 complex path. But RAW itself operates in the Network Plane at a 452 faster time scale. To act on the Data Plane, RAW needs live 453 information from the Operational Plane , e.g., using Bidirectional 454 Forwarding Detection [BFD] and its variants (bidirectional and remote 455 BFD) to protect a link, and OAM techniques to protect a path. 457 5.1.2. Applying Reliability Concepts to Networking 459 The terms Reliaility and Availability are defined for use in RAW in 460 Section 2 and the reader is invited to read [NASA] for more details 461 on the general definition of Reliability. Practically speaking a 462 number of nines is often used to indicate the reliability of a data 463 link, e.g., 5 nines indicate a Packet Delivery Ratio (PDR) of 464 99.999%. 466 This number is typical in a wired environment where the loss is due 467 to a random event such as a solar particle that affects the 468 transmission of a particular frame, but does not affect the previous 469 or next frame, nor frames transmitted on other links. Note that the 470 QoS requirements in RAW may include a bounded latency, and a packet 471 that arrives too late is a fault and not considered as delivered. 473 For a periodic pattern such as an automation control loop, this 474 number is proportional to the Mean Time Between Failures (MTBF). If 475 a single fault can have dramatic consequences, then the MTBF is the 476 expression of the chances that an unwanted event occurs. In data 477 networks, this is rarely the case. Packet loss cannot never be fully 478 avoided and the systems are built to resist to one loss, e.g., using 479 redundancy with Retries (HARQ) or Packet Replication and Elimination 480 (PRE), or, in a typical control loop, by linear interpolation from 481 the previous measuremnents. 483 But the linear interpolation method can not resist to multiple 484 consecutive losses, and a high MTBF is desired as a guarantee that 485 this will not happen, IOW that the losses-in-a-row can be bounded. 486 In that case, what's really desired is a Maximum Consecutive Failures 487 (MCF). If the number of losses in a row passes the MCF, the control 488 loop has to abort. Engineers that build automated processes may use 489 the network reliability expressed in nines or as an MTBF to provide 490 an MCF, e.g., as described in section 7.4 of [RFC8578]. 492 5.1.3. Reliability in the Context of RAW 494 In contrast with wired networks, errors in transmission are the 495 predominent source of packet loss in wireless networks. The root 496 cause may be of multiple origins: 498 Multipath Fading: A destructive interference by a reflection of the 499 original signal. 501 A radio signal may be received directly (line-of-sight) and/or as 502 a reflection on a physical structure (echo). The reflections take 503 a longer path and are delayed by the extra distance divided by the 504 speed of light in the medium. Depending on the frequency, the 505 echo lands with a different phase which may add up to 506 (constructive interference) or destroy the signal (destructive 507 interference). 509 The affected frequencies depend on the relative position of the 510 sender, the receiver, and all the reflecting objects in the 511 environment. A given hop will suffer from multipath fading for 512 multiple packets in a row till the something moves that changes 513 the reflection patterns. 515 Co-channel Interference: Energy in the spectrum used for the 516 transmission confuses the receiver. 518 The wireless medium itself is a Shared Risk Link Group (SRLG) for 519 nearby users of the same spectrum, as an interference may affect 520 multiple co-channel transmissions between different peers within 521 the interference domain of the interferer, possibly even when they 522 use different technologies. 524 Obstacle in Fresnel Zone: The optimal transmission happens when the 525 Fresnel Zone between the sender and the receiver is free of 526 obstacles. 528 As long as a physical object (e.g., a metallic trolley between 529 peers) that affects the transmission is not removed, the quality 530 of the link is affected. 532 In an environment that is rich of metallic structures and mobile 533 objects, a single radio link will provide a fuzzy service, meaning 534 that it cannot be trusted to transport the traffic reliably over a 535 long period of time. 537 Transmission errors are typically not independent, and their nature 538 and duration are unpredictable; as long as a physical object (e.g., a 539 metallic trolley between peers) that affects the transmission is not 540 removed, or as long as the interferer (e.g., a radar) keeps 541 transmitting, a continuous stream of packets will be affected. 543 The key word to combat losses is diversity. A single packet may be 544 sent at different times over different paths that rely on different 545 radio frequencies and different PHY technologies, e.g., narrowband 546 vs. spread spectrum. It is typically retried a number of times in 547 case of a loss, and if possible the retries should again vary all 548 possible parameters. Each form of diversity combats a particular 549 cause of loss and use of diversity must be maximised to optimize the 550 PDR. 552 5.2. RAW Prerequisites 554 A prerequisite to the RAW work is that an end-to-end routing function 555 computes a complex sub-topology along which forwarding can happen 556 between a source and one or more destinations. For 6TiSCH, this is a 557 Track. The concept of Track is specified in the 6TiSCH Architecture 558 [6TiSCH-ARCH]. Tracks provide a high degree of redundancy and 559 diversity and enable RAW PREOF, end-to-end network coding, and 560 possibly radio-specific abstracted techniques such as ARQ, 561 overhearing, frequency diversity, time slotting, and possibly others. 563 How the routing operation computes the Track is out of scope for RAW. 564 The scope of the RAW operation is one Track, and the goal of the RAW 565 operation is to optimize the use of the Track at the forwarding 566 timescale to maintain the expected service while optimizing the usage 567 of constrained resources such as energy and spectrum. 569 Another prerequisite is that an IP link can be established over the 570 radio with some guarantees in terms of service reliability, e.g., it 571 can be relied upon to transmit a packet within a bounded latency and 572 provides a guaranteed BER/PDR outside rare but existing transient 573 outage windows that can last from split seconds to minutes. The 574 radio layer can be programmed with abstract parameters, and can 575 return an abstract view of the state of the Link to help forwarding 576 decision (think DLEP from MANET). In the layered approach, how the 577 radio manages its PHY layer is out of control and out of scope. 578 Whether it is single hop or meshed is also unknown and out of scope. 580 5.3. Routing Time Scale vs. Forwarding Time Scale 582 With DetNet, the end-to-end routing can be centralized and can reside 583 outside the network. In wireless, and in particular in a wireless 584 mesh, the path to the controller that performs the route computation 585 and maintenance expensive in terms of critical resources such as air 586 time and energy. 588 Reaching to the routing computation can also be slow in regards to 589 the speed of events that affect the forwarding operation at the radio 590 layer. Due to the cost and latency to perform a route computation, 591 the controller plane is not expected to be sensitive/reactive to 592 transient changes. The abstraction of a link at the routing level is 593 expected to use statistical operational metrics that aggregate the 594 behavior of a link over long periods of time, and represent its 595 availability as shades of gray as opposed to either up or down. 597 +----------------+ 598 | Controller | 599 | (PCE) | 600 | [Routing ] | 601 | [Function] | 602 +----------------+ 603 ^ 604 | 605 Slow 606 | 607 _-._-._-._-._-._-. | ._-._-._-._-._-._-._-._-._-._-._-._- 608 _-._-._-._-._-._-._-. | _-._-._-._-._-._-._-._-._-._-._-._- 609 | 610 Expensive 611 .... | ....... 612 .... . | . ..... 613 .... v ... 614 .. A-------B-------C---D .. 615 ... / \ / / \ .. 616 . I ----M-------N--zzz-- E .. 617 .. \ \ / / . 618 .. P--zzz--Q----------R .. 619 .. .. 620 ....... ... 621 ............... 622 zzz = flapping now 624 Figure 3: Time Scales 626 In the case of wireless, the changes that affect the forwarding 627 decision can happen frequently and often for short durations, e.g., a 628 mobile object moves between a transmitter and a receiver, and will 629 cancel the line of sight transmission for a few seconds, or a radar 630 measures the depth of a pool and interferes on a particular channel 631 for a split second. 633 There is thus a desire to separate the long term computation of the 634 route and the short term forwarding decision. In such a model, the 635 routing operation computes a complex Track that enables multiple Non- 636 Equal Cost Multi-Path (N-ECMP) forwarding solutions, and leaves it to 637 the Data Plane to make the per-packet decision of which of these 638 possibilities should be used. 640 In the case of wires, the concept is known in traffic engineering 641 where an alternate path can be used upon the detection of a failure 642 in the main path, e.g., using OAM in MPLS-TP or BFD over a collection 643 of SD-WAN tunnels. RAW formalizes a forwarding time scale that is an 644 order(s) of magnitude shorter than the controler plane routing time 645 scale, and separates the protocols and metrics that are used at both 646 scales. Routing can operate on long term statistics such as delivery 647 ratio over minutes to hours, but as a first approximation can ignore 648 flapping. On the other hand, the RAW forwarding decision is made at 649 packet speed, and uses information that must be pertinent at the 650 present time for the current transmission. 652 6. RAW Architecture Elements 654 6.1. PAREO Functions 656 In a nutshell, PRE establishes several paths in a network to provide 657 redundancy and parallel transmissions to bound the end-to-end delay 658 to traverse the network. Optionally, promiscuous listening between 659 paths is possible, such that the Nodes on one path may overhear 660 transmissions along the other path. Considering the scenario shown 661 in Figure 4, many different paths are possible for S to reach R. A 662 simple way to benefit from this topology could be to use the two 663 independent paths via Nodes A, C, E and via B, D, F. But more 664 complex paths are possible by interleaving transmissions from the 665 lower level of the path to the upper level. 667 PRE may also take advantage of the shared properties of the wireless 668 medium to compensate for the potential loss that is incurred with 669 radio transmissions. For instance, when the source sends to A, B may 670 listen also and get a second chance to receive the frame without an 671 additional transmission. Note that B would not have to listen if it 672 already received that particular frame at an earlier timeslot in a 673 dedicated transmission towards B. 675 (A) (C) (E) 677 source (S) (R) (root) 679 (B) (D) (F) 681 Figure 4: A Typical Ladder Shape with Two Parallel Paths Toward 682 the Destination 684 The PRE model can be implemented in both centralized and distributed 685 scheduling approaches. In the centralized approach, a Path 686 Computation Element (PCE) scheduler calculates the routes and 687 schedules the communication among the Nodes along a circuit such as a 688 Label switched path. In the distributed approach, each Node selects 689 its route to the destination, typically using a source routing 690 header. In both cases, at each Node in the paths, a default parent 691 and alternative parent(s) should be selected to set up complex 692 tracks. 694 In the following Subsections, all the required operations defined by 695 PRE, namely, Alternative Path Selection, Packet Replication, Packet 696 Elimination and Promiscuous Overhearing, are described. 698 6.1.1. Packet Replication 700 The objective of PRE is to provide deterministic networking 701 properties: high reliability and bounded latency. To achieve this 702 goal, determinism in every hop of the forwarding paths MUST be 703 guaranteed. By employing a Packet Replication procedure, each Node 704 forwards a copy of each data packet to multiple parents: its Default 705 Parent (DP) and multiple Alternative Parents (APs). To do so, each 706 Node (i.e., source and intermediate Node) transmits the data packet 707 multiple times in unicast to each parent. For instance, in Figure 5, 708 the source Node S is transmitting the packet to both parents, Nodes A 709 and B, at two different times. An example schedule is shown in 710 Table 1. Thus, the packet can use non-congruent paths to the 711 destination. 713 ===> (A) => (C) => (E) === 714 // \\// \\// \\ 715 source (S) //\\ //\\ (R) (root) 716 \\ // \\ // \\ // 717 ===> (B) => (D) => (F) === 719 Figure 5: Packet Replication: S transmits twice the same data 720 packet, to its DP (A) and to its AP (B). 722 +---------+------+------+------+------+------+------+------+ 723 | Channel | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 724 +=========+======+======+======+======+======+======+======+ 725 | 0 | S->A | S->B | B->C | B->D | C->F | E->R | F->R | 726 +---------+------+------+------+------+------+------+------+ 727 | 1 | | A->C | A->D | C->E | D->E | D->F | | 728 +---------+------+------+------+------+------+------+------+ 730 Table 1: Packet Replication: Sample schedule 732 6.1.2. Packet Elimination 734 The replication operation increases the traffic load in the network, 735 due to packet duplications. Thus, a Packet Elimination operation 736 SHOULD be applied at each RPL DODAG level to reduce the unnecessary 737 traffic. To this aim, once a Node receives the first copy of a data 738 packet, it discards the subsequent copies. Because the first copy 739 that reaches a Node is the one that matters, it is the only copy that 740 will be forwarded upward. Then, once a Node performs the Packet 741 Elimination operation, it will proceed with the Packet Replication 742 operation to forward the packet toward the RPL DODAG Root. 744 6.1.3. Promiscuous Overhearing 746 Considering that the wireless medium is broadcast by nature, any 747 neighbor of a transmitter may overhear a transmission. By employing 748 the Promiscuous Overhearing operation, a DP and some AP(s) eventually 749 have more chances to receive the data packets. In Figure 6, when 750 Node A is transmitting to its DP (Node C), the AP (Node D) and its 751 sibling (Node B) may decode this data packet as well. As a result, 752 by employing corellated paths, a Node may have multiple opportunities 753 to receive a given data packet. This feature not only enhances the 754 end-to-end reliability but also it reduces the end-to-end delay and 755 increases energy efficiency. 757 ===> (A) ====> (C) ====> (E) ==== 758 // ^ | \\ \\ 759 source (S) | | \\ (R) (root) 760 \\ | v \\ // 761 ===> (B) ====> (D) ====> (F) ==== 763 Figure 6: Unicast to DP with Overhearing: by employing 764 Promiscuous Overhearing, DP, AP and the sibling Nodes have more 765 opportunities to receive the same data packet. 767 6.1.4. Constructive Interference 769 Constructive Interference can be seen as the reverse of Promiscuous 770 Overhearing, and refers to the case where two senders transmit the 771 exact same signal in a fashion that the emitted symbols add up at the 772 receiver and permit a reception that would not be possible with a 773 single sender at the same PHY mode and the same power level. 775 Constructive Interference was proposed on 5G, Wi-Fi7 and even tested 776 on IEEE 802.14.5. The hard piece is to synchronize the senders to 777 the point that the signals are emitted at slightly different time to 778 offset the difference of propagation delay that corresponds to the 779 difference of distance of the transmitters to the receiver at the 780 speed of light to the point that the symbols are superposed long 781 enough to be recognizable. 783 6.2. Wireless Tracks 785 The "6TiSCH Architecture" [6TiSCH-ARCH] introduces the concept of 786 Track a a possibly complex path with the PAREO functions operated 787 within. 789 A simple track is composed of a direct sequence of reserved hops to 790 ensure the transmission of a single packet from a source Node to a 791 destination Node across a multihop path. 793 A Complex Track is designed as a directed acyclic graph from a source 794 Node towards a destination Node to support multi-path forwarding, as 795 introduced in "6TiSCH Architecture" [6TiSCH-ARCH]. By employing PRE 796 functions [RFC8655], several paths may be computed, and these paths 797 may be more or less independent. For example, a complex Track may 798 branch off and rejoin over non-congruent paths (branches). 800 Some more details for Deterministic Network PRE techniques are 801 presented in the following Section. 803 7. RAW Architecture 805 RAW inherits the conceptual model described in section 4 of the 806 DetNet Architecture [RFC8655]. 808 A Controller Plane Function (CPF) called the Path Computation 809 Element(PCE) [RFC4655] interacts with RAW Nodes over a Southbound 810 API. The RAW Nodes are DetNet relays that are capable of additional 811 diversity mechanisms and measurement functions related to the radio 812 interface, in particular the PAREO redundancy mechanisms. 814 The PCE defines a complex path between an Ingress End System and an 815 Egress End System, and indicates to the RAW Nodes where the PAREO 816 operations may be actioned in the Network Plane. The path may be 817 loosely expressed in order to traverse a non-RAW subnetwork. In that 818 case, the expectation is that the non-RAW subnetwork can be neglected 819 in the RAW computation, that is, considered infinitely fast, reliable 820 and/or available in comparison with the links between RAW nodes. 822 CPF CPF CPF CPF 824 -+-+-+-+-+-+-+ Southbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- 826 RAW --z RAW --z RAW --z RAW 827 z-- Node z-- Node z-- Node z-- Node --z 828 Ingress --z / / / z-- Egress 829 End Z Z Z End 830 Node ---z / / / z-- Node 831 z-- RAW --z RAW ( non-RAW ) --- RAW ---z 832 Node z-- Node --- ( Nodes ) Node 834 --z radio wired 835 z-- link --- link 837 Figure 7: RAW Nodes 839 The Link-Layer metrics are reported to the PCE in a time-aggregated, 840 e.g., statistical fashion. Example Link-Layer metrics include 841 typical Link bandwidth (the medium speed depends dynamically on the 842 PHY mode and the number of users sharing the spectrum) and average 843 availability and reliability figures. 845 Based on those metrics, the PCE installs a complex path with enough 846 redundant forwarding solutions to ensure that the Network Plane can 847 reliably deliver the packets within a System Level Agreement (SLA) 848 associated to the flow. The SLA defines end-to-end reliability and 849 availability figures, where reliability may be expressed a successful 850 delivery within a bounded delay. One a path is established, end-to- 851 end subpath and overall reliability and availability metrics are also 852 reported to the PCE to assure that the SLA is continuously served and 853 recompute the path if not. 855 Depending on the SLA, the path or a leg of the path may include non- 856 RAW Nodes, either interleaved inside the path, or more typically till 857 the Egress End Node. RAW observes the Lower-Layer Links between RAW 858 nodes (typically, radio links) and the end-to-end Network Layer 859 subpath to decide at all times which of the PAREO redundancy is 860 actioned by which RAW Nodes. 862 7.1. PCE vs. PSE 864 Section 5.3 shows that the time scale at which RAW needs to operate 865 is not that of the Controller Plane that needs to deal with a 866 possibly large whole network and make global optimization across 867 multiple flows that may contend for limited resources. 869 RAW separates the path computation time scale at which a complex path 870 is recomputed from the path selection time scale at which the 871 forwarding decision is taken for one or a few packets. RAW operates 872 at the path selection time scale. The RAW problem is to decide, 873 within the redundant solutions that are proposed by the PCE, which 874 will be used for each packet to provide a Reliable and Available 875 service while minimizing the waste of resources. 877 To that effect, RAW defines the Path Selection Engine (PSE) that is 878 the counter-part of the PCE to perform rapid local adjustments of the 879 forwarding tables to avoid excessive use of the resource diversity 880 that the PCE selects. The PSE enables to exploit the richer 881 forwarding capabilities with PAREO and scheduled transmissions at a 882 faster time scale over the smaller domain that is the Track, either 883 Loose or Strict. 885 +---------------+------------------------+-------------------+ 886 | | PCE (Not in Scope) | PSE (In Scope) | 887 +===============+========================+===================+ 888 | Operation | Centralized | Source-Routed or | 889 | | | Distributed | 890 +---------------+------------------------+-------------------+ 891 | Communication | Slow, expensive | Fast, local | 892 +---------------+------------------------+-------------------+ 893 | Time Scale | Long (hours, days) | Short (seconds, | 894 | | | sub-second) | 895 +---------------+------------------------+-------------------+ 896 | Network Size | Large, many Tracks to | Small, within one | 897 | | optimize globally | Track | 898 +---------------+------------------------+-------------------+ 899 | Considered | Averaged, Statistical, | Instant values / | 900 | Metrics | Shade of grey | boolean condition | 901 +---------------+------------------------+-------------------+ 903 Table 2: PCE vs. PSE 905 7.2. RAW OAM 907 The RAW OAM operation in the Network Plane observes a subset of the 908 links along that redundant path and the RAW PSE makes the decision on 909 which PAREO function in actioned at which RAW Node, for a packet or a 910 small collection of packets. 912 In the case of a End-to-End Protection in a Wireless Mesh, the Track 913 is strict and congruent with the path so all links are observed. 914 Conversely, in the case of Radio Access Protection, the Track is 915 Loose and in that case only the first hop is observed; the rest of 916 the path is abstracted and considered infinitely reliable, meaning 917 that the loss of a packet that was sent over one of the possible 918 first hops is attributed to that first hop, even what a particular 919 loss effectively happens farther down the path. 921 *** ** 922 RAN 1 ----- *** ** *** 923 / * ** **** 924 +-------+ / * ** **** +------+ 925 |Ingress|- * ***** |Egress| 926 | End |------ RAN 2 -- * Internet ****---| End | 927 |System |- * ***** |System| 928 +-------+ \ * ******* +------+ 929 \ *** *** ***** 930 RAN n -------- *** ***** 932 <------------------> <--------------------> 933 Observed by OAM Opaque to OAM 935 Figure 8: Observed Links in Radio Access Protection 937 The Links that are not observed by OAM are opaque to it, meaning that 938 the OAM information is carried and possibly echoed as data. In the 939 example above, the Internet is opaque and not controlled by RAW, but 940 RAW measures the end-to-end latency and delivery ratio for packets 941 sent over each if RAN 1, RAN 2 and RAN 3, and determines whether a 942 packet should be sent over either or a collection of those access 943 links. 945 7.3. Source-Routed vs. Distributed Forwarding Decision 947 Within a large routed topology, the route-over mesh operation builds 948 a particular complex Track with one source and one or more 949 destinations; within the Track, packets may follow different paths 950 and may be subject to RAW forwarding operations that include 951 replication, elimination, retries, overhearing and reordering. 953 The RAW forwarding decisions include the selection of points of 954 replication and elimination, how many retries can take place, and a 955 limit of validity for the packet beyond which the packet should be 956 destroyed rather than forwarded uselessly further down the Track. 958 The decision to apply the RAW techniques must be done quickly, and 959 depends on a very recent and precise knowledge of the forwarding 960 conditions within the complex Track. There is a need for an 961 observation method to provide the RAW Data Plane with the specific 962 knowledge of the state of the Track for the type of flow of interest 963 (e.g., for a QoS level of interest). To observe the whole Track in 964 quasi real time, RAW will consider existing tools such as 965 L2-triggers, DLEP, BFD and in-band and out-of-band OAM. 967 One possible way of making the RAW forwarding decisions is to make 968 them all at the ingress and express them in-band in the packet, which 969 requires new loose or strict Hop-by-hop signaling. To control the 970 RAW forwarding operation along a Track for the individual packets, 971 RAW may leverage and extend known techniques such as DetNet tagging, 972 Segment Routing (SRv6) or BIER-TE such as done with [BIER-PREF]. 974 An alternate way is to enable each forwarding Node to make the RAW 975 forwarding decisions for a packet on its own, based on its knowledge 976 of the expectation (timeliness and reliability) for that packet and a 977 recent observation of the rest of the way across the possible paths 978 within the Track. Information about the service should be placed in 979 the packet and matched with the forwarding Node's capabilities and 980 policies. 982 In either case, a per-flow state is installed in all intermediate 983 Nodes to recognize the flow and determine the forwarding policy to be 984 applied. 986 7.4. Flow Identification 988 Section 4.7 of the DetNet Architecture [RFC8655] ties the app-flow 989 identification which is an appliation layer concept with the network 990 path identification that depends on the networking technology by 991 "exporting of flow identification", e.g., to a MPLS label. 993 With RAW, this exporting operation is injective but not bijective. 994 e.g., a flow is fully placed within one RAW Track, but not all 995 packets along that Track are necessarily part of the same flow. For 996 instance, out-of-band OAM packets must circulate in the exact same 997 fashion as the flows that they observe. It results that the flow 998 identification that maps to to app-flow at the network layer must be 999 separate from the path identification that is used to forward a 1000 packet. 1002 Flow 1 (6-tuple) ----+ 1003 | 1004 Flow 2 (6-tuple) ---+ | 1005 | | 1006 OAM -----------+ | | 1007 | | | 1008 | | | 1009 | | | | | 1010 | v v v | 1011 | | 1012 +---------+---------+ 1013 | 1014 | 1015 +------------> Track 1 1016 (IP address, instanceId) 1018 Figure 9: Flow Injection 1020 Section 3.4 of the DetNet data-plane framework [DetNet-DP-FW] 1021 indicates that for a DetNet IP Data Plane, a flow is identified by an 1022 IPv6 6-tuple. With RAW, that 6-tuple is not what indicates the 1023 Track, in other words, the flow ID is not the Track ID. 1025 For instance, the 6TiSCH Architecture [6TiSCH-ARCH] uses a 1026 combination of the address of the Ingress End System and an instance 1027 identifier in a Hop-by-hop option to indicate a Track. Packets that 1028 are tagged with the same (address, instance ID) tuple will experience 1029 the same forwarding behavior regardless of the IPv6 6-tuple, and 1030 regardless of whether they transport application flows or OAM. 1032 8. Security Considerations 1034 9. IANA Considerations 1036 This document has no IANA actions. 1038 10. Contributors 1040 Xavi Vilajosana: Wireless Networks Research Lab, Universitat Oberta 1041 de Catalunya 1043 Rex Buddenberg: 1045 Remous-Aris Koutsiamanis: IMT Atlantique 1047 Nicolas Montavont: IMT Atlantique 1049 11. Acknowledgments 1051 TBD 1053 12. References 1055 12.1. Normative References 1057 [6TiSCH-ARCH] 1058 Thubert, P., "An Architecture for IPv6 over the TSCH mode 1059 of IEEE 802.15.4", Work in Progress, Internet-Draft, 1060 draft-ietf-6tisch-architecture-28, 29 October 2019, 1061 . 1064 [RAW-TECHNOS] 1065 Thubert, P., Cavalcanti, D., Vilajosana, X., and C. 1066 Schmitt, "Reliable and Available Wireless Technologies", 1067 Work in Progress, Internet-Draft, draft-thubert-raw- 1068 technologies-04, 6 January 2020, 1069 . 1072 [RAW-USE-CASES] 1073 Papadopoulos, G., Thubert, P., Theoleyre, F., and C. 1074 Bernardos, "RAW use cases", Work in Progress, Internet- 1075 Draft, draft-bernardos-raw-use-cases-03, 8 March 2020, 1076 . 1079 [RFC4655] Farrel, A., Vasseur, J.-P., and J. Ash, "A Path 1080 Computation Element (PCE)-Based Architecture", RFC 4655, 1081 DOI 10.17487/RFC4655, August 2006, 1082 . 1084 [BFD] Katz, D. and D. Ward, "Bidirectional Forwarding Detection 1085 (BFD)", RFC 5880, DOI 10.17487/RFC5880, June 2010, 1086 . 1088 [RFC6291] Andersson, L., van Helvoort, H., Bonica, R., Romascanu, 1089 D., and S. Mansfield, "Guidelines for the Use of the "OAM" 1090 Acronym in the IETF", BCP 161, RFC 6291, 1091 DOI 10.17487/RFC6291, June 2011, 1092 . 1094 [RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases", 1095 RFC 8578, DOI 10.17487/RFC8578, May 2019, 1096 . 1098 [RFC8175] Ratliff, S., Jury, S., Satterwhite, D., Taylor, R., and B. 1099 Berry, "Dynamic Link Exchange Protocol (DLEP)", RFC 8175, 1100 DOI 10.17487/RFC8175, June 2017, 1101 . 1103 [RFC8557] Finn, N. and P. Thubert, "Deterministic Networking Problem 1104 Statement", RFC 8557, DOI 10.17487/RFC8557, May 2019, 1105 . 1107 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 1108 "Deterministic Networking Architecture", RFC 8655, 1109 DOI 10.17487/RFC8655, October 2019, 1110 . 1112 12.2. Informative References 1114 [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791, 1115 DOI 10.17487/RFC0791, September 1981, 1116 . 1118 [TE] Awduche, D., Chiu, A., Elwalid, A., Widjaja, I., and X. 1119 Xiao, "Overview and Principles of Internet Traffic 1120 Engineering", RFC 3272, DOI 10.17487/RFC3272, May 2002, 1121 . 1123 [STD 62] Harrington, D., Presuhn, R., and B. Wijnen, "An 1124 Architecture for Describing Simple Network Management 1125 Protocol (SNMP) Management Frameworks", STD 62, RFC 3411, 1126 DOI 10.17487/RFC3411, December 2002, 1127 . 1129 [RFC4090] Pan, P., Ed., Swallow, G., Ed., and A. Atlas, Ed., "Fast 1130 Reroute Extensions to RSVP-TE for LSP Tunnels", RFC 4090, 1131 DOI 10.17487/RFC4090, May 2005, 1132 . 1134 [FRR] Shand, M. and S. Bryant, "IP Fast Reroute Framework", 1135 RFC 5714, DOI 10.17487/RFC5714, January 2010, 1136 . 1138 [RLFA-FRR] Bryant, S., Filsfils, C., Previdi, S., Shand, M., and N. 1139 So, "Remote Loop-Free Alternate (LFA) Fast Reroute (FRR)", 1140 RFC 7490, DOI 10.17487/RFC7490, April 2015, 1141 . 1143 [BIER-PREF] 1144 Thubert, P., Eckert, T., Brodard, Z., and H. Jiang, "BIER- 1145 TE extensions for Packet Replication and Elimination 1146 Function (PREF) and OAM", Work in Progress, Internet- 1147 Draft, draft-thubert-bier-replication-elimination-03, 3 1148 March 2018, . 1151 [DetNet-IP-OAM] 1152 Mirsky, G., Chen, M., and D. Black, "Operations, 1153 Administration and Maintenance (OAM) for Deterministic 1154 Networks (DetNet) with IP Data Plane", Work in Progress, 1155 Internet-Draft, draft-mirsky-detnet-ip-oam-02, 23 March 1156 2020, . 1159 [DetNet-DP-FW] 1160 Varga, B., Farkas, J., Berger, L., Malis, A., and S. 1161 Bryant, "DetNet Data Plane Framework", Work in Progress, 1162 Internet-Draft, draft-ietf-detnet-data-plane-framework-06, 1163 6 May 2020, . 1166 [I-D.farkas-raw-5g] 1167 Farkas, J., Dudda, T., Shapin, A., and S. Sandberg, "5G - 1168 Ultra-Reliable Wireless Technology with Low Latency", Work 1169 in Progress, Internet-Draft, draft-farkas-raw-5g-00, 1 1170 April 2020, 1171 . 1173 [NASA] Adams, T., "RELIABILITY: Definition & Quantitative 1174 Illustration", . 1177 [MANET] IETF, "Mobile Ad hoc Networking", 1178 . 1180 [detnet] IETF, "Deterministic Networking", 1181 . 1183 [SPRING] IETF, "Source Packet Routing in Networking", 1184 . 1186 [BIER] IETF, "Bit Indexed Explicit Replication", 1187 . 1189 [BFD] IETF, "Bidirectional Forwarding Detection", 1190 . 1192 [CCAMP] IETF, "Common Control and Measurement Plane", 1193 . 1195 Authors' Addresses 1197 Pascal Thubert (editor) 1198 Cisco Systems, Inc 1199 Building D 1200 45 Allee des Ormes - BP1200 1201 06254 MOUGINS - Sophia Antipolis 1202 France 1204 Phone: +33 497 23 26 34 1205 Email: pthubert@cisco.com 1207 Georgios Z. Papadopoulos 1208 IMT Atlantique 1209 Office B00 - 114A 1210 2 Rue de la Chataigneraie 1211 35510 Cesson-Sevigne - Rennes 1212 France 1214 Phone: +33 299 12 70 04 1215 Email: georgios.papadopoulos@imt-atlantique.fr 1217 Rex Buddenberg 1218 CA 1219 United States of America 1221 Email: buddenbergr@gmail.com