idnits 2.17.1 draft-feeney-t2trg-inter-network-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (May 25, 2018) is 2164 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'TCG316' is defined on line 887, but no explicit reference was found in the text == Outdated reference: A later version (-16) exists of draft-irtf-t2trg-iot-seccons-15 Summary: 1 error (**), 0 flaws (~~), 3 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group L. Feeney 3 Internet-Draft Uppsala University 4 Intended status: Informational V. Fodor 5 Expires: November 26, 2018 KTH 6 May 25, 2018 8 Inter-network Coexistence in the Internet of Things 9 draft-feeney-t2trg-inter-network-03 11 Abstract 13 The breadth of IoT applications implies that there will be many 14 diverse, administratively independent networks operating in the same 15 physical location. In many cases, these networks will use unlicensed 16 spectrum, due to its low cost and ease of deployment. However, this 17 spectrum is becoming increasingly crowded. IoT networks will 18 therefore be subject to wireless interference, both from similar 19 networks and from networks that use the wireless channel in very 20 different ways. 22 High-density, heterogeneous wireless environments present formidable 23 challenges for network coexistence. The PHY and MAC layers are 24 primarily responsible for managing how radios use the channel. But 25 higher layer protocols are also a key factor in inter-network 26 interaction. To date, there have been few performance studies of 27 coexistence in future IoT operating environments, particularly with 28 respect to protocol behavior and network-scale interactions. 30 This document describes key challenges for coexistence and highlights 31 some recent research results that demonstrate the impact of protocol 32 level interactions on network performance. It identifies both 33 concrete and speculative opportunities for the IRTF T2TRG community. 34 The former include developing and documenting best practices for 35 performance evaluation and contributing IoT-related protocols being 36 developed within IETF. The latter include speculative research into 37 the design of high-layer protocols that allow networks to actively 38 coordinate their access to the shared channel. 40 Status of This Memo 42 This Internet-Draft is submitted in full conformance with the 43 provisions of BCP 78 and BCP 79. 45 Internet-Drafts are working documents of the Internet Engineering 46 Task Force (IETF). Note that other groups may also distribute 47 working documents as Internet-Drafts. The list of current Internet- 48 Drafts is at https://datatracker.ietf.org/drafts/current/. 50 Internet-Drafts are draft documents valid for a maximum of six months 51 and may be updated, replaced, or obsoleted by other documents at any 52 time. It is inappropriate to use Internet-Drafts as reference 53 material or to cite them other than as "work in progress." 55 This Internet-Draft will expire on November 26, 2018. 57 Copyright Notice 59 Copyright (c) 2018 IETF Trust and the persons identified as the 60 document authors. All rights reserved. 62 This document is subject to BCP 78 and the IETF Trust's Legal 63 Provisions Relating to IETF Documents 64 (https://trustee.ietf.org/license-info) in effect on the date of 65 publication of this document. Please review these documents 66 carefully, as they describe your rights and restrictions with respect 67 to this document. Code Components extracted from this document must 68 include Simplified BSD License text as described in Section 4.e of 69 the Trust Legal Provisions and are provided without warranty as 70 described in the Simplified BSD License. 72 Table of Contents 74 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 75 2. IoT interaction challenges . . . . . . . . . . . . . . . . . 5 76 2.1. Scale . . . . . . . . . . . . . . . . . . . . . . . . . . 5 77 2.2. Independence . . . . . . . . . . . . . . . . . . . . . . 5 78 2.3. Resource limitations . . . . . . . . . . . . . . . . . . 5 79 2.4. Diversity . . . . . . . . . . . . . . . . . . . . . . . . 6 80 2.4.1. Radio and PHY . . . . . . . . . . . . . . . . . . . . 6 81 2.4.2. Network structures . . . . . . . . . . . . . . . . . 7 82 2.4.3. Protocols . . . . . . . . . . . . . . . . . . . . . . 7 83 3. Interaction behaviors . . . . . . . . . . . . . . . . . . . . 9 84 3.1. WiFi . . . . . . . . . . . . . . . . . . . . . . . . . . 10 85 3.2. IEEE 802.15.4 . . . . . . . . . . . . . . . . . . . . . . 10 86 3.3. Recent results in IoT networks . . . . . . . . . . . . . 10 87 3.4. Higher layer protocols . . . . . . . . . . . . . . . . . 11 88 4. Network coexistence in the IRTF/IETF context . . . . . . . . 11 89 4.1. Performance evaluation and protocol design . . . . . . . 12 90 4.2. Adaptive mitigation strategies . . . . . . . . . . . . . 13 91 4.3. Active mitigation strategies . . . . . . . . . . . . . . 14 92 4.4. Role of Spectrum Regulation . . . . . . . . . . . . . . . 15 93 5. Security Considerations . . . . . . . . . . . . . . . . . . . 16 94 6. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 17 95 7. Informative References . . . . . . . . . . . . . . . . . . . 18 96 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 20 97 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 20 99 1. Introduction 101 An IoT application is a set of wireless devices that act together to 102 perform some sensing and control function. Most applications also 103 have some connectivity to external resources, such as a mobile app or 104 cloud-based service. In general, each application is deployed 105 independently of any other applications that may be operating in the 106 area and is a physically and administratively separate network. 108 An enormous range of IoT applications are expected to become 109 pervasive in daily life. Networks will be installed in public 110 spaces, businesses, and residences by a wide range of individual, 111 commercial, and government actors. As a result, there will be many 112 diverse, administratively independent networks operating in the same 113 physical location. For example, a future home environment may 114 include IoT applications for security, heating and cooling, elder 115 care, air quality monitoring, personal health and fitness, smart home 116 appliances, structural monitoring, lighting, utilities, and 117 entertainment. 119 Many of these networks will use unlicensed spectrum due to low cost 120 and simplicity of deployment for both the user and developer. In 121 unlicensed spectrum, there is no authority that has a management 122 relationship with (or even knows about) all of the potentially 123 interfering networks that can be present in some location. This 124 means that there is no entity that can coordinate networks' use of 125 the shared wireless channel. Networks will therefore experience 126 interference caused by transmissions from devices belonging to other 127 networks. 129 The PHY and MAC layers have primary responsibility for ensuring that 130 devices share the channel efficiently, while spectrum regulations 131 limit devices' output power and overall channel utilization. But the 132 MAC protocol can only explicitly coordinate devices within a single 133 network. It provides only limited protection from other networks, 134 some of which may have very different transmission footprints over 135 time, spectrum or physical space. 137 Network coexistence is mainly evaluated in terms of PHY layer and 138 radio hardware resilience to interference. This is generally based 139 on analytic modeling of the probability of successful packet 140 reception for varying SNIR conditions or on carefully controlled 141 measurements of interacting RF waveforms. (See e.g. [NIST] for a 142 discussion of relevant issues and [SKH11] for an example of such an 143 analysis for IEEE 802.15.4g.) 145 Analytic modeling of network interactions at the MAC layer is much 146 harder, because each network adapts its transmission parameters and 147 timing in response to the others. The presence of interfering 148 networks is usually modeled as increasing the intensity of some 149 statistical process representing noise or loss. Testbed tools, such 150 as JamLab, have also been used to generate controlled interference. 151 Other studies are based on simulation or testbed measurements of 152 simple scenarios. The research literature contains a number of such 153 studies, especially for IEEE 802.11 and IEEE 802.15.4. (See [SURVEY] 154 and [SURVEY2] for an overview.) 156 In practice, currently deployed networks mostly rely on low IoT 157 traffic loads and careful channel selection to achieve adequate 158 performance. This may not be sustainable as rapid growth in IoT (and 159 mobile data offloading) lead to increasing pressure on unlicensed 160 spectrum. There are very few studies that evaluate complex, 161 heterogeneous IoT interference scenarios, particularly with regard to 162 protocol behavior and network-scale interactions. But as recent work 163 [WETZ17] demonstrates, real world instances of IoT interference do 164 occur and require considerable effort to diagnose. 166 This document explores key challenges for network coexistence in 167 future IoT environments and highlights some recent research results 168 ([FF16], [F3G15], [YTB17]). These suggest that protocol level 169 interactions can significantly affect network performance, even in 170 simple scenarios where the channel is not heavily loaded. Higher 171 layer protocols will need to be aware of the potential impact of 172 inter-network interference and avoid contributing to adverse 173 interactions. 175 The community does not yet have a solid understanding of the 176 reliability and effectiveness of IoT protocols in the presence of 177 inter-network interference. In part, this is because the tools and 178 techniques for performance evaluation of network coexistence 179 scenarios are still immature. This document considers some of the 180 challenges and requirements for both simulation and testbed 181 approaches. 183 We also identify both concrete and speculative areas where T2TRG is 184 well-positioned to contribute: The former includes the development of 185 best practices for performance evaluation and informing the ongoing 186 development of IoT-related protocols being developed within the IETF. 187 The latter includes speculative research into the development of 188 protocols that allow independent networks to actively coordinate 189 their use of the shared wireless channel. 191 2. IoT interaction challenges 193 Widespread deployment of diverse IoT applications presents four main 194 challenges: 1) scale 2) lack of a trust relationship between 195 independently deployed networks 3) resource limitations, especially 196 battery capacity 4) diversity of application requirements and channel 197 utilization behavior. 199 2.1. Scale 201 As IoT becomes pervasive, there will be many independent networks 202 operating in any given location. Devices will experience high levels 203 of both homogeneous and heterogeneous radio interference. 205 Since networks use different kinds of radios and have different 206 wireless coverage areas, their topologies will overlap with each 207 other in complex ways. Interference will therefore involve not just 208 individual wireless links, but also larger regions in the network and 209 protocols operating at network scale. 211 Interaction scenarios will also be highly dynamic, with mobility and 212 user activity leading to frequent changes in the set of interfering 213 devices. 215 2.2. Independence 217 In unlicensed spectrum, there is no obvious basis for an 218 administrative relationship between networks. Networks with 219 overlapping wireless coverage may well have been deployed by at 220 different times by unrelated actors. Nor is there any common 221 authority that has an administrative relationship with all of the 222 potentially interfering networks in any given location. 224 This means that there is no external entity that networks can trust 225 to coordinate access to the shared channel. Devices within any one 226 network will be able to authenticate themselves to each other and 227 their own administrator (usually a non-expert user). But there is no 228 way for them to authenticate themselves to each other - an IoT 229 network may not have any meaningful external identity. Even if two 230 networks can exchange this information, there is no obvious way for 231 each to determine whether the other will participate appropriately 232 with respect to some coexistence mechanism. 234 2.3. Resource limitations 236 IoT networks are severely resource constrained in many respects, 237 including channel capacity, energy, hardware capabilities and cost. 239 The large number of devices sharing the wireless channel naturally 240 limits the capacity available to each device. In addition, many 241 devices use low bit-rate radios, which further reduces the 242 communication capacity. (Note, however, that adverse interactions 243 between networks can occur even in cases where the channel is only 244 lightly used.) 246 For energy-harvesting and battery-powered devices, maximizing 247 lifetime is essential. Protocol design is dominated by the need to 248 minimize device activity and especially by the need to keep the 249 energy-hungry radio turned off as much as possible, while still 250 maintaining necessary connectivity. Even sensing the channel 251 conditions is an extremely expensive operation. This limits 252 networks' ability to observe and adapt to the behavior of their 253 neighbors. 255 Finally, for many IoT applications, devices must be low cost and 256 easily deployed and managed by non-expert users. They often have 257 very limited memory and CPU resources. These factors constrain the 258 design space and limit the complexity of proposed solutions. 260 2.4. Diversity 262 Even networks that use the same radio hardware and protocols will 263 interfere with each other. But the diversity of IoT radios, 264 protocols and applications creates additional challenges. Even 265 characterizing the space of possible interactions may be challenging: 266 Protocols can be anything from freely available, to consortia-driven 267 standards such as ZigBee or WirelessHART, to completely proprietary. 269 This diversity is driven by the diversity of IoT applications. 270 Applications will differ significantly in their devices' 271 communication range and the overall network coverage area. They will 272 vary in the number of devices and traffic load. They will have 273 different requirements for latency and reliability. And they will 274 use different energy sources and have different requirements on 275 energy efficiency and lifetime. To meet these requirements, 276 applications will use a wide variety of radios, protocols and network 277 structures. 279 2.4.1. Radio and PHY 281 Different radio technologies divide the spectrum into channels 282 differently: In the 2.4GHz unlicensed band, for example, IEEE 802.11 283 has up to 14 overlapping channels, while IEEE 802.15.4 and 284 BluetoothLE have 16 and 40 non-overlapping ones. Radios also use a 285 variety of modulation techniques at the PHY layer to define how data 286 is encoded on the channel as RF energy. 288 This means that there are many different ways that RF energy is 289 distributed over time and spectrum. As a result, it may not be 290 possible for channel sensing mechanisms to reliably detect the 291 presence of potentially interfering transmissions or identify the 292 source of interference and packet loss. 294 Each PHY layer makes different tradeoffs between transmit power, 295 communication range, bit-rate, bandwidth, energy consumption and 296 resilience. In sub-GHz spectrum, for example, IEEE 802.15.4g/Wi-SUN 297 (smart utility network) provides 50-200 kbps bit-rates with ranges of 298 > 1000m. Very low power EnOcean devices provide similar bit-rates, 299 but ranges of < 100m. By contrast, LoRa provides bit-rates of at 300 most a few kbps, but can obtain 10km of range. Differences in bit- 301 rate and frame size mean that packet transmit times can range from < 302 10 ms to > 200 ms. Radios operating in 2.4GHz, such as IEEE 303 802.15.4, IEEE 802.11 and Bluetooth, show similar diversity. 305 2.4.2. Network structures 307 Along with different kinds of radios, different kinds of network 308 structures can be used to meet application requirements for density 309 and coverage area. The most common structure is the star topology, 310 where all devices communicate directly with a controller. Networks 311 can also cover larger areas or achieve higher reliability by using 312 multi-hop forwarding over various topologies, such as directed 313 acyclic graphs, cluster trees, and meshes. 315 These structures affect how transmissions within a network are 316 correlated with each other in time and space, such as forwarding a 317 frame across a mesh. It can also affect interactions between 318 networks, particularly networks whose radios have very different 319 coverage areas. For example, a long-range device belonging to one 320 network may be located in the midst of a mesh of short-range devices 321 belonging to another network. 323 2.4.3. Protocols 325 The MAC layer defines how senders coordinate their transmissions 326 within a network. Like the PHY layer, different MAC layers create 327 different distributions of RF energy in time and (for channel hopping 328 protocols) spectrum. 330 CSMA-based (channel sensing and backoff) protocols can provide some 331 protection from external transmissions, since they defer to any 332 ongoing transmission that they detect. Conflicts due to hidden 333 terminals can occur even within a single network, but differences 334 between radio technologies and network structures may exacerbate the 335 problem. In addition, MAC timing parameters, such as backoff times, 336 are generally proportional to bit-rate and frame transmit times. 337 Timing incompatibilities between interfering senders can reduce the 338 effectiveness of backoff and retransmissions in heterogeneous 339 environments. 341 TDMA-based (transmission schedule) protocols can be more efficient in 342 their use of the channel and energy than CSMA protocols. But because 343 the networks define their slot structure and transmission schedules 344 independently, they may allocate transmission slots that conflict 345 with each other. Since senders rely on their assigned schedule, such 346 conflicts can be costly. 348 Minimizing energy consumption is often the absolute priority for IoT 349 design. It is necessary to keep radio turned off as much as 350 possible, while still ensuring connectivity. As with MAC protocols 351 (with which they are sometimes integrated), there are a variety of 352 approaches. With synchronous methods, devices wake up according to a 353 schedule that ensures that senders and receivers are awake at the 354 same time (as in IEEE 802.15.4 beacon-enabled PANs or TSCH). 355 Asynchronous methods allow devices to coordinate their wake up 356 schedules on-demand (as in ContikiMAC). 358 Coordinating the duty cycles of a sender and receiver imposes strict 359 timing constraints on radio operations. As with the PHY and MAC, 360 each power save protocol creates its own distribution of RF energy 361 over time. Depending on application requirements and tradeoffs for 362 latency and battery lifetime, duty cycles could be on timescale of < 363 1s to > 1000s. 365 Many IoT networks use IP(v6), but there is also considerable 366 diversity in higher layer protocols, both open and proprietary. 367 Routing protocols make different tradeoffs between latency, 368 reliability, energy efficiency and overhead, depending on the 369 application requirements. The operation of the routing protocol also 370 affects the distribution of RF energy in physical space, as frames 371 are forwarded toward a root or across a mesh. The routing protocol 372 may also react to the presence of interference by attempting to re- 373 route its traffic. 375 Higher layer protocols largely abstract away from the behavior of 376 individual wireless links. They use a variety of mechanisms to 377 maintain communication performance under conditions of loss and 378 delay, including retransmissions, multi-path communication, and 379 application-specific adaptations. 381 Finally, the variety of transport, transfer and application protocols 382 used in IoT networks reflects the diversity of use cases: The RESTful 383 model is central for IoT applications based on web services 385 [I-D.keranen-t2trg-rest-iot]. Wireless sensing applications often 386 use in-network data processing and aggregation to reduce their 387 communication load. Industrial IoT applications emphasize low 388 latency and reliability. Wide-area IoT/SUN networks collect small 389 amounts of data from a very large number of devices. As a result, 390 applications may have very different priorities with respect to 391 packet loss, delay, and energy consumption. 393 3. Interaction behaviors 395 All elements of network functionality - MAC, power saving, topology 396 and routing, congestion control, data transfer, application - 397 contribute patterns of channel utilization over time, frequency, and 398 physical space. At the same time, protocols adapt their behavior in 399 response to channel conditions; relying on channel sensing and frame 400 errors at low layers and on loss and delay at higher layers. Inter- 401 network interaction therefore occurs on multiple time- and spatial- 402 scales and involves all layers of the protocol stack. 404 Motivating scenarios include: 406 o How will sub-GHz LPWAN networks such as LoRa and SigFox, whose 407 base stations cover wide areas, interact with multiple shorter- 408 range networks using IEEE 802.15.4g/WiSUN, Z-Wave, or EnOcean 409 radios? 411 o What happens if two or more independent networks using 412 6LoWPAN+RPL+CoAP are operating in the same room? Or two 6TiSCH 413 networks, each using a different scheduling function? What if an 414 a beacon-enabled PAN interacts with a ZigBee- or ContikiMAC- or 415 Thread-based network? What if people wearing BluetoothLE-based 416 body-area networks are also moving around in the area? Especially 417 in a WiFi heavy environment, the value of channel hopping for 418 interference mitigation may be limited. 420 o More generally, can networks using protocols optimized for 421 different metrics (e.g. latency vs battery lifetime) operate 422 effectively in the same location? 424 To date, there have been very few studies that examine network 425 performance under realistic - dense, heterogeneous, dynamic - 426 interference scenarios. Some existing observations and results are 427 noted here. 429 3.1. WiFi 431 Interference between WiFi networks is a long-standing problem, 432 particularly in dense residential and urban areas, where there are 433 many independently deployed networks and large amounts of traffic. 435 To some extent, this has been mitigated by expansion into 5GHz 436 unlicensed spectrum and by major improvements in WiFi, including 437 higher bit-rates and directional transmission (beamforming). The 438 WiFi environment also has some properties that are helpful for 439 coexistence: WiFi networks are largely homogeneous, consisting of an 440 AP and associated devices that communicate directly with their AP. 441 WiFi also uses a CSMA-based MAC, which means that senders inherently 442 defer to any ongoing WiFi transmission, regardless of its source. 443 And the dominant application is media streaming, which is supported 444 by adaptive mechanisms everywhere from the server to the user 445 application. 447 However, WiFi performance may come under increasing pressure, due not 448 only to the increasing number of IoT networks, but also to the 449 forthcoming deployment of LTE traffic into 5GHz unlicensed spectrum. 451 3.2. IEEE 802.15.4 453 A common scenario in 2.4GHz spectrum will involve high-power, high- 454 traffic WiFi networks impacting networks based on low power, low bit- 455 rate radios, such as IEEE 802.15.4. 457 Practical existing solutions are mostly based on IEEE 802.15.4 458 devices identifying and using the least interfered channels, either 459 statically or by channel hopping. But in areas where there is a lot 460 of WiFi traffic, there may be very few such channels. WiFi 461 conventionally uses non-overlapping WiFi channels 1, 6, and 11, 462 leaving just three minimally interfered IEEE 802.15.4 channels. As a 463 result, low power IoT networks operating in these areas may be 464 crowded into a small number of "good" channels. These may come under 465 increasing pressure as IoT deployment increases. 467 3.3. Recent results in IoT networks 469 Recent research suggests that protocol level interactions can lead to 470 severe performance degradation, even when the channel is not heavily 471 loaded. While these studies focus on various IEEE 802.15.4 MAC 472 layers, the results suggest broader implications for protocol design. 474 [F3G15] and [FF16] show that IEEE 802.15.4 beacon-enabled PANs can 475 experience episodes of severe disruption due to protocol level 476 interactions. This includes behaviors such as short-term 477 oscillations in throughput and extended periods of disconnectivity - 478 even when the channel itself is only lightly loaded. Similarly, 479 [YTB17] shows that interfering IEEE 802.15.4 6TiSCH-based networks 480 experience packet loss and so-called blackout periods, as well as 481 increased energy consumption. 483 These behaviors appear to be due to a combination of timing 484 rigidities in the MAC protocol, periodicity in the radio duty cycle, 485 and clock drift between networks. Battery constraints force devices 486 to spend most of their time with their radios turned off. Senders 487 and receivers therefore need some way to coordinate their radio wake 488 up times so that they can exchange packets. These mechanisms often 489 depend heavily on careful timing of radio operations, instead of (or 490 in addition to) explicit control traffic. This timing dependence can 491 make networks more sensitive to disruption than might be expected 492 from just considering overall channel utilization and collision 493 probabilities. Periodicity can exacerbate these effects. In 494 addition, clock drift results in networks' synchronizing and 495 desynchronizing with each other. This can result in interaction 496 effects at timescales on the orders of minutes or even days. 497 Generalizing these observations suggests that it will be necessary to 498 reconsider tradeoffs between energy consumption and resilience. 500 3.4. Higher layer protocols 502 To date, there have been few studies that address the performance of 503 high layer protocols, such as routing or data transfer, in network 504 coexistence. Certainly, extended outages at the link layer will 505 affect their operation and there is a risk that higher layer 506 protocols' reaction will exacerbate the impact of interference. 507 Conversely, it is possible that higher layer protocols may act to 508 mitigate the impact of interference, e.g. through congestion 509 avoidance. 511 4. Network coexistence in the IRTF/IETF context 513 The research literature contains a variety of proposals for improving 514 protocol performance in the presence of interference (see [SURVEY], 515 [SURVEY2] for an overview). In many cases, they assume rather 516 narrowly defined interaction scenarios and none seem to have been 517 deployed in practice. 519 Network coexistence in realistic IoT environments remains an open 520 issue, particularly with respect to protocol and network-scale 521 interactions. T2TRG is well-positioned to contribute to addressing 522 it by: 524 o Developing and advocating best practices for performance 525 evaluation, focusing on realistic future wireless environments. 527 o Contributing to the ongoing development of IoT-related IETF 528 protocols, so that they are as resilient as possible to inter- 529 network interference. 531 o Supporting speculative research into the possibility of higher 532 layer protocols for active coordination between networks sharing 533 unlicensed spectrum. 535 4.1. Performance evaluation and protocol design 537 Performance evaluation of IoT protocols should take into account 538 their behavior in the presence of many diverse, administratively 539 independent networks operating in the same spectrum. To date, there 540 have been few studies that fully reflect this aspect of the future 541 IoT operating environment. This suggests that the community does not 542 yet have a complete understanding of effectiveness and reliability of 543 IoT protocols. 545 Given the community's limited experience with such evaluation, it is 546 unsurprising that there are not yet clear principles for designing 547 experiments that can provide meaningful results. Experiments must 548 reflect a realistic interference environment and capture behaviors 549 caused by interactions within the protocol stack, within a network, 550 and between networks - while still being both manageable and 551 informative for the the user. Best practices for designing such 552 experiments have not been established and existing simulation and 553 testbed tools have significant limitations. 555 Protocol-oriented network simulators (e.g. ns-2/3, OMNeT++, OPNET) 556 enable performance evaluation at scale: It is straightforward to 557 simulate an extremely large number of scenarios behavior over a long 558 period. Simulation also provides complete control and visibility 559 into the operation of the simulated system. However, these 560 advantages come at the cost of reduced fidelity, especially for 561 wireless propagation and reception. Modeling of interference between 562 different kinds of radios is particularly lacking. 564 By contrast, testbeds provide ground-truth about network performance 565 in a specific scenario. There are a number of open WSN/IoT testbeds 566 (e.g. [FINTEROP], [FITIOT]) that provide access to various 567 collections of hardware. However, the community has had little 568 experience using them for evaluating coexistence scenarios. 570 There are three main challenges: One is the logistics of deploying 571 long-running experiments involving multiple applications and many 572 devices. Another practical challenge is instrumenting and collecting 573 data from the entire protocol stack and correlating the results 574 across networks, especially with resource-constrained devices. This 575 functionality is essential for obtaining data that allows users to 576 reason about the observed performance. Finally, there is a deeper 577 challenge in defining experiments that allow the user to 578 systematically explore the space of possible interactions, despite 579 the complexity and variability of the inter-network interference 580 environment. 582 In this context, T2TRG can contribute to the development of and 583 advocacy for best practices for performance evaluation. The results 584 of such studies can inform ongoing protocol development. This 585 includes protocols being developed in the IETF 6lo, 6TiSCH 586 (especially 6top), LPWAN, LWIG, ROLL and CoRe Working Groups. (It 587 is, of course, also necessary to take into account interactions with 588 protocols from other open and proprietary sources.) 590 4.2. Adaptive mitigation strategies 592 Network coexistence is likely to rely heavily on improving resilience 593 to interference in the MAC layer, which is ultimately responsible for 594 determining when a sender transmits. 596 But a MAC protocol cannot explicitly coordinate with devices in other 597 networks; it may not even be able to identify what kinds of networks 598 are sharing the channel, much less exchange (authenticated) control 599 traffic. The MAC layer must instead adapt to the presence of other 600 networks based on channel sensing and frame loss. This is a 601 significant challenge in complex interference environments, 602 especially for battery-powered devices, which must avoid the high 603 energy cost of listening to the channel as much as possible. While 604 the MAC layer and power saving protocols are themselves largely 605 outside IETF scope, these topics are relevant to the work of IETF 606 WG's such as 6lo, 6TiSCH, LPWAN and LWIG. 608 Like the MAC layer, higher layer protocols also adapt their behavior, 609 using packet loss and delay. But complex interactions such as those 610 described above can lead to disruptions that are difficult for higher 611 layer protocols to predict or adapt to in an effective way. It is 612 therefore important to ensure that protocol behaviors, such as route 613 selection, congestion control or keep-alive mechanisms, contribute to 614 (or at least do not hurt) resilience to inter-network interference. 615 These topics are particularly relevant to IETF protocols such as RPL 616 and CoAP. 618 4.3. Active mitigation strategies 620 More speculatively, there may be opportunities for higher layer 621 protocols to actively participate in interference mitigation, by 622 sharing information about their operation and even by explicit 623 coordination between networks. 625 When two networks use the same PHY layer, it is possible for frames 626 transmitted by devices in one network to be successfully received by 627 devices in other networks. These frames are usually discarded 628 immediately, since they fail a MAC layer authentication check. But 629 if they are not discarded (and are not encrypted), the networks can 630 observe each others' control traffic or even explicitly exchange 631 information. Such a mechanism could allow them to announce their 632 expected channel utilization patterns, for example. MAC layer or 633 even IPv6 frames could be used for this purpose. 635 Alternatively, many IoT applications have some administrative 636 component that is connected to the Internet infrastructure, such as 637 mobile app-based user interface or cloud-based data collection. Even 638 limited connectivity opens possibilities for making use of a rich 639 array of resources. For example, this may be a way to provide access 640 to additional computing power or to allow networks make use of 641 external services with which they have an administrative 642 relationship. This might enable a coordination mechanism based on 643 negotiation via some trusted cloud-based service. 645 The inspiration here is from several different approaches: Cognitive 646 radio solutions where secondary users obtain information about 647 activity of primary users from trusted sources; Citizens Broadband 648 Radio Service (CBRS) and its spectrum allocation service; and 649 research into distributed coordination services e.g. [SEMCK14]. 650 However, all of these approaches rely on either strict spectrum 651 regulation or a strong assumption of compatibility and cooperative 652 behavior among networks. 654 Even more speculatively, a secure distributed ledger could be used to 655 allow networks to announce themselves in a location, to provide 656 information about their channel utilization, and to obtain 657 information about co-located networks. Such a ledger could further 658 act as a reputation management system or as a resource broker. This 659 is potentially related to distributed infrastructure work in the IRTF 660 DINRG. 662 However, these are very much an open research area and there are 663 substantial challenges in developing such mechanisms: 665 1) There is an enormous diversity of radios, channel access methods 666 and utilization patterns that might need to be described. It is not 667 clear what information should be signaled or what actions a receiver 668 should take in response. 670 2) Battery lifetime, channel capacity, and device CPU and memory 671 resources continue to be significant limitations. In particular, the 672 radio duty cycle is highly constrained, limiting both sensing and 673 communication. 675 3) Any cooperative mechanism must operate effectively in the absence 676 of any administrative or trust relationship between networks. 677 Alternatively, there must be some way to establish an appropriate 678 level of trust. This presents a significant challenge to the 679 practical implementation of cooperative mechanisms proposed in the 680 literature. (See Security Considerations below.) 682 4) The privacy implications of networks sharing information about 683 their activity must be carefully considered. (See Security 684 Considerations below.) 686 Despite the challenges, this topic seems particularly amenable to 687 standards and interoperability-oriented approaches enabled by IRTF 688 T2TRG. There may be synergy with IRTF T2TRG work in IoT semantic 689 interoperability: Can IoT networks describe not only the 'things' 690 they connect, but also themselves? In addition, the IRTF DIN 691 research group is active in the area of secure distributed Internet 692 infrastructure. 694 4.4. Role of Spectrum Regulation 696 Network coexistence is ultimately a problem of spectrum regulation. 697 Regulation of unlicensed spectrum has historically focused on output 698 power and overall spectrum utilization. For example, in 868 MHz 699 spectrum, LoRa relies on transmit duty cycle (DC) limits (which range 700 from 0.1% to 1%, depending on sub-band) to ensure efficient channel 701 utilization. 703 In some cases, listen-before-talk (LBT) has been mandated for 704 unlicensed bands, including (optionally) 868MHz. This results in a 705 more complex regulatory structures, due to the need to specify 706 detection thresholds, listening intervals, and backoff behaviors. 707 The regulations specify minimum requirements, rather than a mechanism 708 that is common to all networks. This can lead to networks with 709 different backoff behaviors sharing a channel. Issues of 710 compatibility and fairness between various LBT strategies are an 711 active topic of study, notably with regard to WiFi and LTE 712 coexistence in 5GHz spectrum (e.g. [KYK16]). 714 The IETF community has a strong interest in ensuring that spectrum 715 regulation not only enables efficient use of unlicensed spectrum for 716 IoT applications, but also avoids overly prescriptive mandates that 717 constrain diversity and innovation. 719 5. Security Considerations 721 An overview of security challenges in IoT environments is given in 722 [I-D.irtf-t2trg-iot-seccons]. The current document focuses on 723 coexistence between independently administrated networks operating in 724 the same location. The biggest security challenge for managing 725 network interactions is that such networks do not necessarily have 726 any basis for a trust relationship. 728 Regulations concerning unlicensed spectrum only control radio 729 behaviors such as transmit power and overall channel utilization. 730 Regulations do not mandate the use of any specific protocol. It is 731 therefore not possible to externally enforce that networks 732 participate in some specific coexistence protocol (as long as they 733 otherwise comply with regulations). 735 Most wireless protocols adapt their behavior to channel conditions to 736 some extent, such as contention backoff, channel blacklisting, or re- 737 routing. But the more a network changes its behavior in response to 738 small amounts of information from an untrusted source, the more 739 leverage an attacker has to disrupt it. Similarly, the more 740 information about its future behavior a network provides to an 741 untrusted destination, the easier it is for an attacker to disrupt 742 it. The risk is further exacerbated in energy-constrained networks, 743 because a device may be forced to spend energy unnecessarily. In 744 addition, the high energy cost of listening to the channel makes it 745 expensive to build trust by observing the behavior of other networks. 747 Any proposed solution will therefore need to be resilient to the 748 possibility of incompatible, oblivious, selfish, or even hostile 749 networks when designing a coexistence mechanism. This is especially 750 true for methods in which two networks actively coordinate their use 751 of the shared channel. At a minimum, participating in information 752 exchange should not substantially increase vulnerability to 753 disruption in the case of a malicious (or merely incompatible) actor. 755 In addition, networks that try to be friendly toward each other may 756 disclose substantial information about their operation. There are 757 privacy issues associated with IoT networks making such information 758 visible, because of their close coupling with human activity. 759 Particularly for health-related applications, even being able to 760 identify the type of application or its level of activity may reveal 761 sensitive data. Ideally, it should be possible for a network to both 762 obfuscate its communication patterns (if needed) and act 763 cooperatively. 765 One maxim that may be useful in designing the set of information that 766 a network discloses as a matter of course with the intention of 767 facilitating coexistence is that the information disclosed should not 768 provide more insight than that information an attacker might have 769 gained by simply observing the network for a while. But note that 770 simply disclosing that information in an accessible way still changes 771 the economy of surveillance -- the objective is that it also changes 772 the economy of coexistence, and these effects need to be carefully 773 weighed against each other. 775 6. Conclusion 777 The future IoT operating environment will contain many diverse, 778 administratively independent networks sharing unlicensed spectrum. 779 Ensuring network coexistence is essential for avoiding the "tragedy 780 of the commons" and enabling practical deployment of IoT solutions. 782 The community currently lacks a good understanding of the impact of 783 inter-network interactions, particularly with regard to protocol 784 behavior and network-scale interactions. However, recent results for 785 both IEEE 802.15.4 PANs and 6TiSCH + RPL networks suggest that inter- 786 network interactions can lead to episodes of significant disruption, 787 even when the channel itself is not overloaded. More research is 788 needed into both the causes of adverse interactions and ways to 789 mitigate them, particularly with regard to the role of higher layer 790 protocols. 792 Network coexistence is and will continue to be largely driven by 793 spectrum regulation and the PHY and MAC layers. However, this issue 794 are also relevant to the work of IETF Working Groups, such as 6lo, 795 6TiSCH, LPWAN, ROLL, CoRE, and LWIG. We identify three areas where 796 T2TRG can play a significant role: 798 o Performance evaluation should reflect that the IoT wireless 799 environment will contain diverse interfering networks. Tools and 800 techniques for investigating inter-network interaction are still 801 immature. The community could benefit substantially from the 802 development and documentation of best practices in this area. 804 o The results of such performance evaluation can assist IETF Working 805 Groups in improving the resilience of IoT-related protocols. 807 o There may also be a role for novel network coexistence mechanisms 808 based on information sharing or explicit coordination between 809 networks. This is a speculative research topic that seems 810 particularly amenable to standards and interoperability oriented 811 approaches. However, there are substantial challenges. 813 7. Informative References 815 [F3G15] Feeney, L., Frey, M., Fodor, V., and M. Gunes, "Modes of 816 inter-network interaction in beacon-enabled IEEE 802.15.4 817 networks", 2015 14th Annual Mediterranean Ad Hoc 818 Networking Workshop (MED-HOC-NET), 819 DOI 10.1109/medhocnet.2015.7173294, June 2015. 821 [FF16] Feeney, L. and V. Fodor, "Reliability in co-located 822 802.15.4 personal area networks", Proceedings of the 6th 823 ACM International Workshop on Pervasive Wireless 824 Healthcare - MobiHealth '16, DOI 10.1145/2944921.2944923, 825 2016. 827 [FINTEROP] 828 Kim, E. and S. Ziegler, "Towards an open framework of 829 online interoperability and performance tests for the 830 Internet of Things", 2017 Global Internet of Things 831 Summit (GIoTS), DOI 10.1109/giots.2017.8016248, June 2017. 833 [FITIOT] Adjih, C., Baccelli, E., Fleury, E., Harter, G., Mitton, 834 N., Noel, T., Pissard-Gibollet, R., Saint-Marcel, F., 835 Schreiner, G., Vandaele, J., and T. Watteyne, "FIT IoT- 836 LAB: A large scale open experimental IoT testbed", 2015 837 IEEE 2nd World Forum on Internet of Things (WF-IoT), DOI 838 10.1109/wf-iot.2015.7389098, December 2015. 840 [I-D.irtf-t2trg-iot-seccons] 841 Garcia-Morchon, O., Kumar, S., and M. Sethi, "State-of- 842 the-Art and Challenges for the Internet of Things 843 Security", draft-irtf-t2trg-iot-seccons-15 (work in 844 progress), May 2018. 846 [I-D.keranen-t2trg-rest-iot] 847 Keranen, A., Kovatsch, M., and K. Hartke, "RESTful Design 848 for Internet of Things Systems", draft-keranen-t2trg-rest- 849 iot-05 (work in progress), September 2017. 851 [KYK16] Kim, C., Yang, C., and C. Kang, "Adaptive Listen-Before- 852 Talk (LBT) scheme for LTE and Wi-Fi systems coexisting in 853 unlicensed band", 2016 13th IEEE Annual Consumer 854 Communications & Networking Conference (CCNC), 855 DOI 10.1109/ccnc.2016.7444845, January 2016. 857 [NIST] Koepke, G., Young, W., Ladbury, J., and J. Coder, 858 "Interference and Coexistence of Wireless Systems in 859 Critical Infrastructure", National Institute of Standards 860 and Technology report, DOI 10.6028/nist.tn.1885, July 861 2015. 863 [SEMCK14] Skjegstad, M., Ellingsater, B., Maseng, T., Crowcroft, J., 864 and O. Kure, "Large-scale distributed Internet-based 865 discovery mechanism for dynamic spectrum allocation", 2014 866 IEEE International Symposium on Dynamic Spectrum Access 867 Networks (DYSPAN), DOI 10.1109/dyspan.2014.6817824, April 868 2014. 870 [SKH11] Sum, C., Kojima, F., and H. Harada, "Coexistence of 871 homogeneous and heterogeneous systems for IEEE 802.15.4g 872 smart utility networks", 2011 IEEE International Symposium 873 on Dynamic Spectrum Access Networks (DySPAN), 874 DOI 10.1109/dyspan.2011.5936241, May 2011. 876 [SURVEY] Han, Y., Ekici, E., Kremo, H., and O. Altintas, "Spectrum 877 sharing methods for the coexistence of multiple RF 878 systems: A survey", Ad Hoc Networks Vol. 53, pp. 53-78, 879 DOI 10.1016/j.adhoc.2016.09.009, December 2016. 881 [SURVEY2] Baccour, N., Puccinelli, D., Voigt, T., Koubaa, A., Noda, 882 C., Fotouhi, H., Alves, M., Youssef, H., Zuniga, M., 883 Boano, C., and K. Roemer, "External Radio Interference", 884 SpringerBriefs in Electrical and Computer Engineering pp. 885 21-63, DOI 10.1007/978-3-319-00774-8_2, 2013. 887 [TCG316] Tinnirello, I., Croce, D., Galioto, N., Garlisi, D., and 888 F. Giuliano, "Cross-Technology WiFi/ZigBee Communications: 889 Dealing With Channel Insertions and Deletions", IEEE 890 Communications Letters Vol. 20, pp. 2300-2303, 891 DOI 10.1109/lcomm.2016.2603978, November 2016. 893 [WETZ17] Wetzker, U., Splitt, I., Zimmerling, M., Boano, C., and K. 894 Romer, "Troubleshooting Wireless Coexistence Problems in 895 the Industrial Internet of Things", 2016 IEEE Intl 896 Conference on Computational Science and Engineering (CSE) 897 and IEEE Intl Conference on Embedded and Ubiquitous 898 Computing (EUC) and 15th Intl Symposium on Distributed 899 Computing and Applications for Business 900 Engineering (DCABES), DOI 10.1109/cse-euc-dcabes.2016.167, 901 August 2016. 903 [YTB17] Ben Yaala, S., Theoleyre, F., and R. Bouallegue, 904 "Cooperative resynchronization to improve the reliability 905 of colocated IEEE 802.15.4 -TSCH networks in dense 906 deployments", Ad Hoc Networks Vol. 64, pp. 112-126, 907 DOI 10.1016/j.adhoc.2017.07.002, September 2017. 909 Acknowledgements 911 The authors would like to thank Michael Frey, Charalampos Orfanidis, 912 Martin Jacobsson, and Per Gunningberg for their valuable 913 collaboration in simulation and measurement studies of inter-network 914 interference. We would also like to thank Carsten Bormann for his 915 support and encouragement in preparing this document, particularly 916 the discussion of security considerations. David Oran's detailed 917 comments on the text are also much appreciated. 919 Authors' Addresses 921 Laura Marie Feeney 922 Uppsala University 923 Box 337 924 Uppsala SE-751 05 925 Sweden 927 Email: lmfeeney@it.uu.se 929 Viktoria Fodor 930 KTH 931 Osquldas vaeg 10 932 Stockholm SE-100 44 933 Sweden 935 Email: vjfodor@kth.se