idnits 2.17.1 draft-hong-t2trg-iot-edge-computing-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (13 July 2020) is 1375 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-10) exists of draft-bernardos-sfc-fog-ran-07 == Outdated reference: A later version (-21) exists of draft-ietf-core-oscore-groupcomm-09 == Outdated reference: A later version (-05) exists of draft-mcbride-edge-data-discovery-overview-03 == Outdated reference: A later version (-04) exists of draft-sarathchandra-coin-appcentres-02 Summary: 1 error (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Hong 3 Internet-Draft Y-G. Hong 4 Intended status: Informational ETRI 5 Expires: 14 January 2021 X. de Foy 6 InterDigital Communications, LLC 7 M. Kovatsch 8 Huawei Technologies Duesseldorf GmbH 9 E. Schooler 10 Intel 11 D. Kutscher 12 University of Applied Sciences Emden/Leer 13 13 July 2020 15 IoT Edge Challenges and Functions 16 draft-hong-t2trg-iot-edge-computing-05 18 Abstract 20 Many IoT applications have requirements that cannot be met by the 21 traditional Cloud (aka cloud computing). These include time 22 sensitivity, data volume, uplink cost, operation in the face of 23 intermittent services, privacy and security. As a result, the IoT is 24 driving the Internet toward Edge computing. This document outlines 25 the requirements of the emerging IoT Edge and its challenges. It 26 presents a general model, and major components of the IoT Edge, with 27 the goal to provide a common base for future discussions in T2TRG and 28 other IRTF and IETF groups. 30 Status of This Memo 32 This Internet-Draft is submitted in full conformance with the 33 provisions of BCP 78 and BCP 79. 35 Internet-Drafts are working documents of the Internet Engineering 36 Task Force (IETF). Note that other groups may also distribute 37 working documents as Internet-Drafts. The list of current Internet- 38 Drafts is at https://datatracker.ietf.org/drafts/current/. 40 Internet-Drafts are draft documents valid for a maximum of six months 41 and may be updated, replaced, or obsoleted by other documents at any 42 time. It is inappropriate to use Internet-Drafts as reference 43 material or to cite them other than as "work in progress." 45 This Internet-Draft will expire on 14 January 2021. 47 Copyright Notice 49 Copyright (c) 2020 IETF Trust and the persons identified as the 50 document authors. All rights reserved. 52 This document is subject to BCP 78 and the IETF Trust's Legal 53 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 54 license-info) in effect on the date of publication of this document. 55 Please review these documents carefully, as they describe your rights 56 and restrictions with respect to this document. Code Components 57 extracted from this document must include Simplified BSD License text 58 as described in Section 4.e of the Trust Legal Provisions and are 59 provided without warranty as described in the Simplified BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 64 2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 3 65 2.1. Internet of Things (IoT) . . . . . . . . . . . . . . . . 3 66 2.2. Cloud Computing . . . . . . . . . . . . . . . . . . . . . 4 67 2.3. Edge Computing . . . . . . . . . . . . . . . . . . . . . 4 68 2.4. Example of IoT Edge Computing Use Cases . . . . . . . . . 6 69 2.4.1. Smart Construction . . . . . . . . . . . . . . . . . 6 70 2.4.2. Smart Grid . . . . . . . . . . . . . . . . . . . . . 6 71 2.4.3. Smart Water System . . . . . . . . . . . . . . . . . 7 72 3. IoT Challenges Leading Towards Edge Computing . . . . . . . . 7 73 3.1. Time Sensitivity . . . . . . . . . . . . . . . . . . . . 7 74 3.2. Uplink Cost . . . . . . . . . . . . . . . . . . . . . . . 8 75 3.3. Resilience to Intermittent Services . . . . . . . . . . . 8 76 3.4. Privacy and Security . . . . . . . . . . . . . . . . . . 8 77 4. IoT Edge Computing Functions . . . . . . . . . . . . . . . . 9 78 4.1. Overview of IoT Edge Computing Today . . . . . . . . . . 9 79 4.2. General Model . . . . . . . . . . . . . . . . . . . . . . 10 80 4.3. OAM Components . . . . . . . . . . . . . . . . . . . . . 14 81 4.3.1. Virtualization Management . . . . . . . . . . . . . . 14 82 4.3.2. Resource Discovery and Authentication . . . . . . . . 15 83 4.3.3. Edge Organization and Federation . . . . . . . . . . 15 84 4.4. Functional Components . . . . . . . . . . . . . . . . . . 16 85 4.4.1. External APIs . . . . . . . . . . . . . . . . . . . . 16 86 4.4.2. Communication Brokering . . . . . . . . . . . . . . . 16 87 4.4.3. In-Network Computation . . . . . . . . . . . . . . . 17 88 4.4.4. Edge Caching . . . . . . . . . . . . . . . . . . . . 18 89 4.4.5. Other Services . . . . . . . . . . . . . . . . . . . 19 90 4.5. Application Components . . . . . . . . . . . . . . . . . 19 91 4.5.1. IoT End Devices Management . . . . . . . . . . . . . 19 92 4.5.2. Data Management . . . . . . . . . . . . . . . . . . . 19 93 4.6. Simulation and Emulation Environments . . . . . . . . . . 20 94 5. Security Considerations . . . . . . . . . . . . . . . . . . . 20 95 6. Acknowledgment . . . . . . . . . . . . . . . . . . . . . . . 21 96 7. Informative References . . . . . . . . . . . . . . . . . . . 21 97 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 99 1. Introduction 101 Currently, many IoT services leverage the Cloud, since it can provide 102 virtually unlimited storage and processing power. The reliance of 103 IoT on back-end cloud computing brings additional advantages such as 104 flexibility and efficiency. Today's IoT systems are fairly static 105 with respect to integrating and supporting computation. It's not 106 that there is no computation, but systems are often limited to static 107 configurations (edge gateways, cloud services). 109 However, IoT devices are creating vast amounts of data at the network 110 edge. To meet IoT use case requirements, that data increasingly is 111 being stored, processed, analyzed, and acted upon close to the data 112 producers. These requirements include time sensitivity, data volume, 113 uplink cost, resiliency in the face of intermittent connectivity, 114 privacy, and security, which cannot be addressed by today's 115 centralized cloud computing. These requirements suggest a more 116 flexible way to distribute computing (and storage) and to integrate 117 it in the edge-cloud continuum. We will refer to this integration of 118 edge computing and IoT as "IoT edge computing". Our draft describes 119 background, uses cases, challenges, and presents system models and 120 functional components. 122 2. Background 124 2.1. Internet of Things (IoT) 126 Since the term "Internet of Things" (IoT) was coined by Kevin Ashton 127 in 1999 working on Radio-Frequency Identification (RFID) technology 128 [Ashton], the concept of IoT has evolved. It now reflects a vision 129 of connecting the physical world to the virtual world of computers 130 using (wireless) networks over which Things can send and receive 131 information without human intervention. Recently, the term has 132 become more literal by actually connecting Things to the Internet and 133 converging on Internet and Web technology. 135 Things are usually embedded systems of various kinds, such as home 136 appliances, mobile equipment, wearable devices, etc. Things are 137 widely distributed, but typically have limited storage and processing 138 power, which raise concerns regarding reliability, performance, 139 energy consumption, security, and privacy [Lin]. This limited 140 storage and processing power leads to complementing IoT with cloud 141 computing. 143 2.2. Cloud Computing 145 Cloud computing has been defined in [NIST]: "cloud computing is a 146 model for enabling ubiquitous, convenient, on-demand network access 147 to a shared pool of configurable computing resources (e.g., networks, 148 servers, storage, applications, and services) that can be rapidly 149 provisioned and released with minimal management effort or service 150 provider interaction". Cloud computing has become a predominant 151 technology that offers virtually unlimited capacity in terms of 152 storage and processing power, at low cost. This offering enabled the 153 realization of a new computing model, in which virtualized resources 154 can be leased in an on-demand fashion, being provided as general 155 utilities. Companies like Amazon, Google, Facebook, etc. widely 156 adopted this paradigm for delivering services over the Internet, 157 gaining both economical and technical benefits [Botta]. 159 Today, an unprecedented volume and variety of data is generated by 160 Things and applications deployed in edge networks consume this data. 161 Some of these applications may need very short response times, some 162 may access personal data, while others may generate vast amounts of 163 data. Today's cloud-based service models are not suitable for these 164 applications, which can instead leverage edge computing. 166 2.3. Edge Computing 168 Edge computing, in some settings also referred to as fog computing, 169 is a new paradigm in which substantial computing and storage 170 resources are placed at the edge of the Internet, that is, in close 171 proximity to mobile devices, sensors, actuators, or machines. Edge 172 computing happens near data sources [Mahadev], or closer 173 (topologically, physically, in term of latency, etc.) to where 174 decisions or interactions with the physical world are happening. It 175 processes both downstream data, e.g. originated from cloud services, 176 and upstream data, e.g. originated from end devices or network 177 elements. The term fog computing usually represents the notion of a 178 multi-tiered edge computing, that is, several layers of compute 179 infrastructure between the end devices and cloud services. 181 An edge device is any computing or networking resource residing 182 between data sources and cloud-based datacenters. In edge computing, 183 end devices not only consume data, but also produce data. And at the 184 network edge, devices not only request services and information from 185 the Cloud, but also handle computing tasks including processing, 186 storage, caching, and load balancing on data sent to and from the 187 Cloud [Shi]. This does not preclude end devices from hosting 188 computation themselves when possible, independently or as part of a 189 distributed edge computing platform (this is also referred to as Mist 190 Computing). 192 Several standards defining organization and industry forums have 193 provided definitions of edge and fog computing: 195 * ISO defines edge computing as a "form of distributed computing in 196 which significant processing and data storage takes place on nodes 197 which are at the edge of the network" [ISO_TR]. 199 * ETSI defines multi-access edge computing as a "system which 200 provides an IT service environment and cloud-computing 201 capabilities at the edge of an access network which contains one 202 or more type of access technology, and in close proximity to its 203 users" [ETSI_MEC_01]. 205 * The Industrial Internet Consortium (formerly OpenFog) defines fog 206 computing as "a horizontal, system-level architecture that 207 distributes computing, storage, control and networking functions 208 closer to the users along a cloud-to-thing continuum" [OpenFog]. 210 Based on these definitions, we can summarize a general philosophy of 211 edge computing as to distribute the required functions close to users 212 and data, while the difference to classic local systems is the usage 213 of management and orchestration features adopted from cloud 214 computing. 216 Actors from various industries approach edge computing using 217 different terms and reference models, although in practice these 218 approaches are not incompatible and may integrate with each other: 220 * The telecommunication industry tends to use a model where edge 221 computing services are deployed over NFV infrastructure at 222 aggregation points, or in proximity to the user equipment (e.g., 223 gNodeBs) [ETSI_MEC_03]. 225 * Enterprise and campus solutions often interpret edge computing as 226 an "edge cloud", that is, a smaller data center directly connected 227 to the local network (often referred to as "on-premise"). 229 * The automation industry defines the edge as the connection point 230 between IT from OT (Operational Technology). Hence, here edge 231 computing sometimes refers to applying IT solutions to OT problems 232 such as analytics, more flexible user interfaces, or simply having 233 more compute power than an automation controller. 235 2.4. Example of IoT Edge Computing Use Cases 237 IoT edge computing can be used in home, industry, grid, healthcare, 238 city, transportation, agriculture, and/or education scenarios. We 239 discuss here only a few examples of such use cases, to point out 240 differentiating requirements. 242 2.4.1. Smart Construction 244 In traditional construction domain, heavy equipment and machinery 245 pose risks to humans and property. Thus, there have been many 246 attempts to deploy technology to protect lives and property in 247 construction sites. For example, measurements of noise, vibration, 248 and gas can be recorded and reported to an inspector. Today, data 249 produced by such measurements is collected by a local gateway and 250 transferred to a remote cloud server. This incurs transmission 251 costs, e.g., over a LTE connection, and storage costs, e.g., when 252 using Amazon Web Services. When an inspector needs to investigate an 253 incident, he checks the information stored on the cloud server. 255 To determine the exact cause of an incident, sensor data including 256 audio and video are transferred to a remote server. In this case, 257 audio and video data volume is typically very large and the cost of 258 transmission can be an issue. By leveraging IoT edge computing, 259 sensor data can be processed and analyzed on a gateway located within 260 or near a construction site. And with the help of statistical 261 analysis or machine learning technologies, we can predict future 262 incidents in advance and trigger an on-site alarm. Furthermore, 263 predicting the time of an incident can help reducing significantly 264 the volume and cost of transmitted data, by transmitting video at 265 high resolution during critical periods, while otherwise using a 266 lower resolution. 268 2.4.2. Smart Grid 270 In future smart city scenarios, the Smart Grid will be critical in 271 ensuring highly available/efficient energy control in city-wide 272 electricity management. Edge computing is expected to play a 273 significant role in those systems to improve transmission efficiency 274 of electricity; to react and restore power after a disturbance; to 275 reduce operation costs and reuse renewable energy effectively, since 276 these operations involve local decision making. In addition, edge 277 computing can help monitoring power generation and power demand, and 278 making local electrical energy storage decisions in the smart grid 279 system. 281 2.4.3. Smart Water System 283 The water system is one of the most important aspects of a city. 284 Effective use of water, and cost-effective and environment-friendly 285 water treatment are critical aspects of this system. Edge computing 286 can help with monitoring water consumption and transport, and with 287 predicting future water usage level. Examples of application 288 include: water harvesting, ground water monitoring, locally analyzing 289 collected information related to water control and management to 290 limit water losses. 292 3. IoT Challenges Leading Towards Edge Computing 294 This section describes challenges met by IoT, that are motivating the 295 adoption of edge computing for IoT. Those are distinct from research 296 challenges applicable to IoT edge computing, some of which will be 297 mentioned in Section 4.3. 299 IoT technology is used with more and more demanding applications,e.g. 300 in industrial, automotive or healthcare domains, leading to new 301 challenges. For example, industrial machines such as laser cutters 302 already produce over 1 terabyte per hour, and similar amounts can be 303 generated in autonomous cars [NVIDIA]. 90% of IoT data is expected to 304 be stored, processed, analyzed, and acted upon close to the source 305 [Kelly], as cloud computing models alone cannot address the new 306 challenges [Chiang]. 308 Below we discuss IoT use case requirements that are moving cloud 309 capabilities to be more proximate and more distributed and 310 disaggregated. 312 3.1. Time Sensitivity 314 Many industrial control systems, such as manufacturing systems, smart 315 grids, oil and gas systems, etc., often require stringent end-to-end 316 latency between the sensor and control node. While some IoT 317 applications may require latency below a few tens of milliseconds 318 [Weiner], industrial robots and motion control systems have use cases 319 for cycle times in the order of microseconds [_60802]. In some cases 320 speed-of-light limitations may simply prevent a solution based on 321 remote cloud, however it is not the only challenge relative to time 322 sensitivity. An important aspect for real-time communications is not 323 only the latency, but also guarantees for jitter. This means control 324 packets need to arrive with as little variation as possible and 325 within a strict deadline. Given the best-effort characteristics of 326 the Internet this challenge is virtually impossible to address, 327 without using end-to-end guarantees for individual message delivery 328 and continuous data flows. 330 3.2. Uplink Cost 332 Many IoT deployments are not challenged by a constrained network 333 bandwidth to the Cloud. The fifth generation mobile networks (5G) 334 and Wi-Fi 6 both theoretically top out at 10 gigabits per second 335 (i.e., 4.5 terabyte per hour), which enables high-bandwidth uplinks. 336 However, the resulting cost for high-bandwidth connectivity to upload 337 all data to the Cloud is unjustifiable and impractical for most IoT 338 applications. In some settings, e.g. in aeronautical communication, 339 higher communication costs reduce the amount of data that can be 340 practically uploaded even further. 342 3.3. Resilience to Intermittent Services 344 Many IoT devices such as sensors, data collectors, actuators, 345 controllers, etc. have very limited hardware resources and cannot 346 rely solely on their limited resources to meet all their computing 347 and/or storage needs. They require reliable, uninterrupted or 348 resilient services to augment their capabilities in order to fulfill 349 their application tasks. This is hard and partly impossible to 350 achieve with cloud services for systems such as vehicles, drones, or 351 oil rigs that have intermittent network connectivity. The dual is 352 also true, a cloud back-end might want to have a reading of the 353 device even if it's currently asleep. 355 3.4. Privacy and Security 357 When IoT services are deployed at home, personal information can be 358 learned from detected usage data. For example, one can extract 359 information about employment, family status, age, and income by 360 analyzing smart meter data [ENERGY]. Policy makers started to 361 provide frameworks that limit the usage of personal data and put 362 strict requirements on data controllers and processors. However, 363 data stored indefinitely in the Cloud also increases the risk of data 364 leakage, for instance, through attacks on rich targets. 366 Industrial systems are often argued to not have privacy implications, 367 as no personal data is gathered. Yet data from such systems is often 368 highly sensitive, as one might be able to infer trade secrets such as 369 the setup of production lines. Hence, the owner of these systems are 370 generally reluctant to upload IoT data to the Cloud. 372 Furthermore, passive observers can perform traffic analysis on the 373 device-to-cloud path. Hiding traffic patterns associated with sensor 374 networks can therefore be another requirement for edge computing. 376 4. IoT Edge Computing Functions 378 In this section we first look at the current state of IoT edge 379 computing Section 4.1, and then define a general system model 380 Section 4.2. This provides context for IoT edge computing functions, 381 which are listed in Section 4.3. 383 4.1. Overview of IoT Edge Computing Today 385 This section provides an overview of today's IoT edge computing 386 field, based on a limited review of standards, research, open-source 387 and proprietary products in 388 [I-D-defoy-t2trg-iot-edge-computing-background]. 390 IoT gateways, both open-source (such as EdgeX Foundry or Home Edge) 391 and proprietary (such as Amazon Greengrass, Microsoft Azure IoT Edge, 392 Google Cloud IoT Core, and gateways from Bosh, Siemens), represent a 393 common class of IoT edge computing products, where the gateway is 394 providing a local service on customer premises, and is remotely 395 managed through a cloud service. IoT communication protocols are 396 typically used between IoT devices and the gateway, including CoAP, 397 MQTT and many specialized IoT protocols (such as OPC UA and DDS in 398 the Industrial IoT space), while the gateway communicates with the 399 distant cloud typically using HTTPS. Virtualization platforms enable 400 the deployment of virtual edge computing functions (as VMs, 401 application containers, etc.), including IoT gateway software, on 402 servers in the mobile network infrastructure (at base station and 403 concentration points), in edge datacenters (in central offices) or 404 regional datacenters located near central offices. End devices are 405 envisioned to become computing devices in forward looking projects, 406 but are not commonly used as such today. 408 Physical or virtual IoT gateways can host application programs, which 409 are typically built using an SDK to access local services through a 410 programmatic API. Edge cloud system operators host their customers' 411 applications VMs or containers on servers located in or near access 412 networks, which can implement local edge services. For example, 413 mobile networks can provide edge services for radio network 414 information, location and bandwidth management. 416 Life cycle management of services and applications on physical IoT 417 gateways is often cloud-based. Edge cloud management platforms and 418 products (such as StarlingX, Akraino Edge Stack, Mobile EdgeX) adapt 419 cloud management technologies (e.g., Kubernetes) to the edge cloud, 420 i.e., to smaller, distributed computing devices running outside a 421 controlled data center. Services and application life-cycle is 422 typically using a NFV-like management and orchestration model. 424 The platform typically includes services to advertise or consume APIs 425 (e.g., Mp1 interface in ETSI MEC supports service discovery and 426 communication), and enables communicating with local and remote 427 endpoints (e.g., message routing function in IoT gateways). The 428 service platform is typically extensible by edge applications, since 429 they can advertise an API that other edge applications can consume. 430 IoT communication services include protocols translation, analytics 431 and transcoding. Communication between edge computing devices is 432 enabled in tiered deployments or distributed deployments. 434 An edge cloud platform may enable pass-through without storage or 435 local storage (e.g., on IoT gateways). Some edge cloud platforms use 436 a distributed form of storage such as an ICN network (e.g., NFN nodes 437 can store data in NDN) or a distributed storage platform (e.g., 438 Ceph). External storage, e.g., on databases in distant or local IT 439 cloud, is typically used for filtered data deemed worthy of long term 440 storage, although in some case it may be for all data, for example 441 when required for regulatory reasons. 443 Stateful computing is supported on platforms hosting native programs, 444 VMs or containers. Stateless computing is supported on platforms 445 providing a "serverless computing" service (a.k.a. function-as- 446 a-service), or on systems based on named function networking. 448 In many IoT use cases, a typical network usage pattern is high volume 449 uplink with some form of traffic reduction enabled by processing over 450 edge computing devices. Alternatives to traffic reduction include 451 deferred transmission (to off-peak hours or using physical shipping). 452 Downlink traffic includes application control and software updates. 453 Other, downlink-heavy traffic patterns are not excluded but are more 454 often associated with non-IoT usage (e.g., video CDNs). 456 4.2. General Model 458 Edge computing is expected to play an important role in deploying new 459 IoT services integrated with Big Data and AI, enabled by flexible in- 460 network computing platforms. Although there are lots of approaches 461 to edge computing, we attempt to lay out a general model and list 462 associated logical functions in this section. In practice, this 463 model can map to different architectures, such as: 465 * A single IoT gateway, or a hierarchy of IoT gateways, typically 466 connected to the cloud (e.g., to extend the traditionally cloud- 467 based management of IoT devices and data to the edge). A common 468 role of an IoT Gateway is to provide access to an heterogeneous 469 set of IoT devices/sensors; handle IoT data; and deliver IoT data 470 to its final destination in a cloud network. Whereas an IoT 471 gateway needs interactions with cloud like as conventional cloud 472 computing, it can also operate independently. 474 * A set of distributed computing nodes, e.g., embedded in switches, 475 routers, edge cloud servers or mobile devices. Some IoT end 476 devices can have enough computing capabilities to participate in 477 such distributed systems due to advances in hardware technology. 478 In this model, edge computing nodes can collaborate with each 479 other to share their resources. 481 +---------------------+ 482 | Remote network | +---------------+ 483 |(e.g., cloud network)| | Service | 484 +-----------+---------+ | Operator | 485 | +------+--------+ 486 | | 487 +--------------+-------------------+-----------+ 488 | Edge Computing Domain | 489 | | 490 | One or more Computing Nodes | 491 | (IoT gateway, end devices, switches, | 492 | routers, mini/micro-datacenters, etc.) | 493 | | 494 | OAM Components | 495 | - Virtualization Management | 496 | - Resource Discovery and Authentication | 497 | - Edge Organization and Federation | 498 | - ... | 499 | | 500 | Functional Components | 501 | - External APIs | 502 | - Communication Brokering | 503 | - In-Network Computation | 504 | - Edge Caching | 505 | - Other Services | 506 | - ... | 507 | | 508 | Application Components | 509 | - IoT End Devices Management | 510 | - Data Management | 511 | - ... | 512 | | 513 +------+--------------+-------- - - - -+- - - -+ 514 | | | | | 515 | | +-----+--+ 516 +----+---+ +-----+--+ | |compute | | 517 | End | | End | ... |node/end| 518 |Device 1| |Device 2| ...| |device n| | 519 +--------+ +--------+ +--------+ 520 + - - - - - - - -+ 522 Figure 1: Model of IoT Edge Computing 524 In the above model, the edge computing domain is interconnected with 525 IoT end devices (southbound connectivity) and possibly with a remote/ 526 cloud network (northbound connectivity), and with a service 527 operator's system. Edge computing nodes provide multiple logical 528 functions, or components, which may not all be present in a given 529 system. They may be implemented in a centralized or distributed 530 fashion, in the edge network, or through some interworking between 531 the edge network and a remote cloud network. 533 +----------------------------------------------+ 534 | Edge Computing Domain | 535 | | 536 | +--------+ +--------+ +--------+ | 537 | |Compute | |Compute | |Compute | | 538 | |node/End| |node/End| .... |node/End| | 539 | |device 1| |device 2| .... |device m| | 540 | +----+---+ +----+---+ +----+---+ | 541 | | | | | 542 | +---+-------------+-----------------+--+ | 543 | | IoT Edge Gateway | | 544 | +-----------+-------------------+------+ | 545 | | | | 546 +--------------+-------------------+-----------+ 547 | | 548 +-----------+---------+ +------+-------+ 549 | Remote network | | Service | 550 |(e.g., cloud network)| | Operator(s) | 551 +-----------+---------+ +------+-------+ 552 | | 553 +--------------+-------------------+-----------+ 554 | | | | 555 | +-----------+-------------------+------+ | 556 | | IoT Edge Gateway | | 557 | +---+-------------+-----------------+--+ | 558 | | | | | 559 | +----+---+ +----+---+ +----+---+ | 560 | |Compute | |Compute | |Compute | | 561 | |node/End| |node/End| .... |node/End| | 562 | |device 1| |device 2| .... |device n| | 563 | +--------+ +--------+ +--------+ | 564 | | 565 | Edge Computing Domain | 566 +----------------------------------------------+ 568 Figure 2: Example: Machine Learning over a Distributed IoT Edge 569 Computing System 571 In the above example of system, the edge computing domain is composed 572 of IoT edge gateways and IoT end devices which are also used as 573 computing nodes. Edge computing domains are connected with a remote/ 574 cloud network, and with their respective service operator's system. 575 IoT end devices/computing nodes provide logical functions, as part of 576 a distributed machine learning application. The processing 577 capabilities in IoT end devices being limited, they require the 578 support of other nodes: the training process for AI services is 579 executed at IoT edge gateways or cloud networks and the prediction 580 (inference) service is executed in the IoT end devices. 582 We now attempt to enumerate major edge computing domain components. 583 They are here loosely organized into OAM, functional and application 584 components, with the understanding that the distinction between these 585 classes may not always be clear, depending on actual system 586 architectures. Some representative research challenges are 587 associated with those functions. We used input from co-authors, IRTF 588 attendees and some comprehensive reviews of the field ([Yousefpour], 589 [Zhang2], [Khan]). 591 4.3. OAM Components 593 Edge computing OAM goes beyond the network-related OAM functions 594 listed in [RFC6291]. Besides infrastructure (network, storage and 595 computing resources), edge computing systems can also include 596 computing environments (for VMs, software containers, functions), IoT 597 end devices, data and code. 599 Operation related functions include performance monitoring for 600 service level agreement measurement; fault management and 601 provisioning for links, nodes, compute and storage resources, 602 platforms and services. Administration covers network/compute/ 603 storage resources, platforms and services discovery, configuration 604 and planning. Management covers monitoring and diagnostics of 605 failures, as well as means to minimize their occurrence and take 606 corrective actions. This may include software updates management, 607 high service availability through redundancy and multipath 608 communication. Centralized (e.g., SDN) and decentralized management 609 systems can be used. 611 We further detail a few OAM components. 613 4.3.1. Virtualization Management 615 Some IoT edge computing systems make use of virtualized (compute, 616 storage and networking) resources, which need to be allocated and 617 configured. This function is covered to a large extent by ETSI NFV 618 and MEC standards activities. Projects such as [LFEDGE-EVE] further 619 cover virtualization and its management into distributed edge 620 computing settings. 622 Related challenges include: 624 * Minimizing virtual function instantiation time and resource usage 625 * Integration of edge computing with virtualized radio networks (Fog 626 RAN) [I-D.bernardos-sfc-fog-ran] and with 5G access networks 628 * Handling of multi-tenancy with regards to limited resources at the 629 network edge 631 4.3.2. Resource Discovery and Authentication 633 Discovery and authentication may target platforms, infrastructure 634 resources, such as compute, network and storage, but also other 635 resources such as IoT end devices, sensors, data, code units, 636 services, applications or users interacting with the system. Broker- 637 based solutions can be used, e.g. using an IoT gateway as broker to 638 discover IoT resources. Today, centralized gateway-based systems 639 rely, for device authentication, on the installation of a secret on 640 IoT end devices and on computing devices (e.g., a device certificate 641 stored in a hardware security module). 643 Related challenges include: 645 * Discovery, authentication and trust establishment between end 646 devices, compute nodes and platforms, with regards to concerns 647 such as mobility, heterogeneity, scale, multiple trust domains, 648 constrained devices, anonymity and traceability 650 * Intermittent connectivity to the Internet, preventing relying on a 651 third-party authority [Echeverria] 653 * Resiliency to failures [Harchol], denial of service attacks, 654 easier physical access for attackers 656 4.3.3. Edge Organization and Federation 658 In a distributed system context, once edge devices have discovered 659 and authenticated each other, they can be organized, or self- 660 organize, into hierarchies or clusters. Organization may range from 661 centralized to peer-to-peer. Such groups can also form federations 662 with other edge or remote clouds. 664 Related challenges include: 666 * Sharing resources in multi-vendor/operator scenarios, with a goal 667 to optimize criteria such as profit [Anglano], resource usage, 668 latency or energy consumption 670 * Support for scaling, and enabling fault-tolerance or self-healing 671 [Jeong]. Besides using hierarchical organization to cope with 672 scaling, another available and possibly complementary mechanism is 673 multicast ([RFC7390] [I-D.ietf-core-oscore-groupcomm]) 675 * Capacity planning, placement of infrastructure nodes to minimize 676 delay [Fan], cost, energy, etc. 678 * Incentives for participation, e.g. in peer-to-peer federation 679 schemes 681 4.4. Functional Components 683 4.4.1. External APIs 685 An IoT edge cloud may provide a northbound data plane or management 686 plane interface to a remote network, e.g., a cloud, home or 687 enterprise network. This interface does not exist in standalone 688 (local-only) scenarios. To support such an interface when it exists, 689 an edge computing component needs to expose an API, deal with 690 authentication and authorization, support secure communication. 692 An IoT edge cloud may provide an API or interface to local or mobile 693 users, for example to provide access to services and applications, or 694 to manage data published by local/mobile devices. 696 Related challenges include: 698 * Defining edge computing abstractions suitable for users and cloud 699 systems to interact with edge computing systems. In one example, 700 this interaction can be based on the PaaS model [Yangui] 702 4.4.2. Communication Brokering 704 A typical function of IoT edge computing is to facilitate 705 communication with IoT end devices: for example, enable clients to 706 register as recipients for data from devices, as well as forwarding/ 707 routing of traffic to or from IoT end devices, enabling various data 708 discovery and redistribution patterns, e.g., north-south with clouds, 709 east-west with other edge devices 710 [I-D.mcbride-edge-data-discovery-overview]. Another related aspect 711 is dispatching of alerts and notifications to interested consumers 712 both inside and outside of the edge computing domain. Protocol 713 translation, analytics and transcoding may also be performed when 714 necessary. 716 Communication brokering may be centralized in some systems, e.g., 717 using a hub-and-spoke message broker, or distributed like with 718 message buses, possibly in a layered bus approach. Distributed 719 systems may leverage direct communication between end devices, over 720 device-to-device links. A broker can ensure communication 721 reliability, traceability, and in some cases transaction management. 723 Related challenges include: 725 * Enabling secure and resilient communication between IoT end 726 devices and remote cloud, e.g. through multipath support 728 4.4.3. In-Network Computation 730 A core function of IoT edge computing is to enable computation 731 offloading, i.e., to perform computation on an edge node on behalf of 732 a device or user, but also to orchestrate computation (in a 733 centralized or distributed manner) and manage applications lifecycle. 734 Support for in-network computation may vary in term of capability, 735 e.g., computing nodes can host virtual machines, software containers, 736 software actors or unikernels able run stateful or stateless code, or 737 a rule engine providing an API to register actions in response to 738 conditions such as IoT device ID, sensor values to check, thresholds, 739 etc. 741 QoS can be provided in some systems through the combination of 742 network QoS (e.g., traffic engineering or wireless resource 743 scheduling) and compute/storage resource allocations. For example in 744 some systems a bandwidth manager service can be exposed to enable 745 allocation of bandwidth to/from an edge computing application 746 instance. 748 Related challenges include: 750 * (Computation placement) Selecting, in a centralized or 751 distributed/peer-to-peer manner, an appropriate compute device 752 based on available resources, location of data input and data 753 sinks, compute node properties, etc., and with varying goals 754 including for example end-to-end latency, privacy, high 755 availability, energy conservation, network efficiency (e.g. using 756 load balancing techniques to avoid congestion) 758 * Onboarding code on a platform or compute device, and invoking 759 remote code execution, possibly as part of a distributed 760 programming model and with respect to similar concerns of latency, 761 privacy, etc. These operations should deal with heterogeneous 762 compute nodes [Schafer], and may in some cases also support end 763 devices as compute nodes 765 * Adapting Quality of Results (QoR) for applications where a perfect 766 result is not necessary [Li] 768 * Assisted or automatic partitioning of code 769 [I-D.sarathchandra-coin-appcentres] 771 * Supporting computation across trust domains, e.g. verifying 772 computation results 774 * Relocating an instance from one compute node to another, while 775 maintaining a given service level. 777 * Session continuity when communicating with end devices that are 778 mobile, possibly at high speed (e.g. in vehicular scenarios) 780 * Defining, managing and verifying SLAs for edge computing systems. 781 Pricing is a related challenge 783 4.4.4. Edge Caching 785 A purpose of local caching may be to enable local data processing 786 (e.g., pre-processing or analysis), or to enable delayed virtual or 787 physical shipping. A responsibility of the edge caching component is 788 to manage data persistence, e.g., to schedule removal of data when it 789 is no longer needed. Another aspect of this component may be to 790 authenticate and encrypt data. It can for example take the form of a 791 distributed storage system. 793 Related challenges include 795 * (Cache and data placement) Using cache positioning and data 796 placement strategies to minimize data retrieval delay [Liu], 797 energy consumption. Caches may be positioned in the access 798 network infrastructure or may be on end devices using device-to- 799 device communication 801 * Maintaining data consistency, freshness and privacy in systems 802 that are distributed, constrained and dynamic (e.g. due to end 803 devices and computing nodes churn or mobility). For example, age 804 of information [Yates], a performance metric that captures the 805 timeliness of information from a sender (e.g. an IoT device), can 806 be exposed to networks to enable tradeoffs in this problem space 808 4.4.5. Other Services 810 Data generated by IoT devices and associated information obtained 811 from the access network may be used to provide high level services 812 such as end device location, radio network information and bandwidth 813 management. 815 4.5. Application Components 817 IoT edge computing can host applications such as the ones mentioned 818 in Section 2.4. While describing components of individual 819 applications is out of our scope, some of those applications share 820 similar functions, such as IoT end device management, data 821 management, described below. 823 4.5.1. IoT End Devices Management 825 IoT end device management includes managing information about the IoT 826 devices, including their sensors, how to communicate with them, etc. 827 Edge computing addresses the scalability challenges from the massive 828 number of IoT end devices by separating the scalability domain into 829 edge/local networks and remote network. 831 Challenges listed in Section 4.3.2 may be applicable to IoT end 832 devices management as well. 834 4.5.2. Data Management 836 Data storage and processing at the edge is a major aspect of IoT edge 837 computing, directly addressing high level IoT challenges listed in 838 Section 3. Data analysis such as performed in AI/ML tasks performed 839 at the edge may benefit from specialized hardware support on 840 computing nodes. 842 Related challenges include: 844 * Addressing concerns on resource usage, security and privacy when 845 sharing, discovering or managing data. For example by presenting 846 data in views composed of an aggregation of related data [Zhang], 847 protecting data communication between authenticated peers 848 [Basudan], classifying data (e.g., in terms of privacy, 849 importance, validity, etc.), compressing data 851 * Data driven programming models [Renart], e.g. event-based, 852 including handling of naming and data abstractions 854 * Addressing concerns such as limited resources, privacy, dynamic 855 and heterogeneous environment, to deploy machine learning at the 856 edge. For example, making machine learning more lightweight and 857 distributed, supporting shorter training time and simplified 858 models, and supporting models that can be compressed for efficient 859 communication [Murshed] 861 * While edge computing can support IoT services independently of 862 cloud computing, it can also be connected to cloud computing. 863 Thus, the relationship of IoT edge computing to cloud computing, 864 with regard to data management, is another potential challenge 865 [ISO_TR] 867 4.6. Simulation and Emulation Environments 869 IoT Edge Computing brings new challenges to simulation and emulation 870 tools used by researchers and developers. A varied set of 871 applications, network and computing technologies can coexist in a 872 distributed system, which make modelling difficult. Scale, mobility 873 and resource management are additional challenges [SimulatingFog]. 875 Tools include simulators, where simplified application logic runs on 876 top of a fog network model, and emulators, where actual applications 877 can be deployed, typically in software containers, over a cloud 878 infrastructure (e.g. Docker, Kubernetes) itself running over a 879 network emulating edge network conditions such as variable delays, 880 throughput and mobility events. To gain in scale, emulated and 881 simulated systems can be used together in hybrid federation-based 882 approaches [PseudoDynamicTesting], while to gain in realism physical 883 devices can be interconnected with emulated systems. Examples of 884 related work and platforms include the publicly accessible MEC 885 sandbox work recently initiated in ETSI [ETSI_Sandbox], and open 886 source simulators and emulators ([AdvantEDGE] emulator and tools 887 cited in [SimulatingFog]). 889 5. Security Considerations 891 As discussed in Section 4.3.2, authentication and trust (between 892 computing nodes, management nodes, end devices) can be challenging as 893 scale, mobility and heterogeneity increase. The sometimes 894 disconnected nature of edge resources can prevent relying on a third- 895 party authority. Distributed edge computing is exposed to issues 896 with reliability and denial of service attacks. Personal or 897 proprietary IoT data leakage is also a major threat, especially due 898 to the distributed nature of the systems (Section 4.5.2). 900 However, edge computing also brings solutions in the security space: 901 maintaining privacy by computing sensitive data closer to data 902 generators is a major use case for IoT edge computing. An edge cloud 903 can be used to take actions based on sensitive data, or anonymizing, 904 aggregating or compressing data prior to transmitting to a remote 905 cloud server. Edge computing communication brokering functions can 906 also be used to secure communication between edge and cloud networks. 908 6. Acknowledgment 910 The authors would like to thank Joo-Sang Youn, Akbar Rahman, Michel 911 Roy, Robert Gazda, Rute Sofia, Thomas Fossati and Chonggang Wang for 912 their valuable comments and suggestions on this document. 914 7. Informative References 916 [AdvantEDGE] 917 "Mobile Edge Emulation Platform", Source Code Repository , 918 2020, . 920 [Anglano] Anglano, C., Canonico, M., Castagno, P., Guazzone, M., and 921 M. Sereno, "A game-theoretic approach to coalition 922 formation in fog provider federations", IEEE Third 923 International Conference on Fog and Mobile Edge Computing 924 (FMEC), pages 123-130 , 2018. 926 [Ashton] Ashton, K., "That Internet of Things thing", RFID J. vol. 927 22, no. 7, pp. 97-114 , 2009. 929 [Basudan] Basudan, S., Lin, X., and K. Sankaranarayanan, "A privacy- 930 preserving vehicular crowdsensing-based road surface 931 condition monitoring system using fog computing", IEEE 932 Internet of Things Journal, 4(3):772-782 , 2017. 934 [Botta] Botta, A., Donato, W., Persico, V., and A. Pescape, 935 "Integration of Cloud Computing and Internet of Things: A 936 survey", Future Gener. Comput. Syst., vol. 56, pp. 937 684-700 , 2016. 939 [Chiang] Chiang, M. and T. Zhang, "Fog and IoT: An overview of 940 research opportunities", IEEE Internet Things J., vol. 3, 941 no. 6, pp. 854-864 , 2016. 943 [Echeverria] 944 Echeverria, S., Klinedinst, D., Williams, K., and G. A 945 Lewis, "Establishing trusted identities in disconnected 946 edge environments", IEEE/ACM Symposium Edge Computing 947 (SEC), pages 51-63. , 2016. 949 [ENERGY] Beckel, C., Sadamori, L., Staake, T., and S. Santini, 950 "Revealing Household Characteristics from Smart Meter 951 Data", Energy, vol. 78, pp. 397-410 , 2014. 953 [ETSI_MEC_01] 954 ETSI, ., "Multi-access Edge Computing (MEC); Terminology", 955 ETSI GS 001 , 2019, . 958 [ETSI_MEC_03] 959 ETSI, ., "Mobile Edge Computing (MEC); Framework and 960 Reference Architecture", ETSI GS 003 , 2019, 961 . 964 [ETSI_Sandbox] 965 "Multi-access Edge Computing (MEC) MEC Sandbox Work Item", 966 Portal , 2020, 967 . 970 [Fan] Fan, Q. and N. Ansari, "Cost aware cloudlet placement for 971 big data processing at the edge", IEEE International 972 Conference on Communications (ICC), pages 1-6 , 2017. 974 [Harchol] Harchol, Y., Mushtaq, A., McCauley, J., Panda, A., and S. 975 Shenker, "Cessna: Resilient edge-computing", Workshop on 976 Mobile Edge Communications, pages 1-6. ACM , 2018. 978 [I-D-defoy-t2trg-iot-edge-computing-background] 979 de Foy, X., Hong, J., Hong, Y., Kovatsch, M., Schooler, 980 E., and D. Kutscher, "Machine learning at the network 981 edge: A survey", draft-defoy-t2trg-iot-edge-computing- 982 background-00 , 2020, . 986 [I-D.bernardos-sfc-fog-ran] 987 Bernardos, C., Rahman, A., and A. Mourad, "Service 988 Function Chaining Use Cases in Fog RAN", Work in Progress, 989 Internet-Draft, draft-bernardos-sfc-fog-ran-07, 11 March 990 2020, . 993 [I-D.ietf-core-oscore-groupcomm] 994 Tiloca, M., Selander, G., Palombini, F., and J. Park, 995 "Group OSCORE - Secure Group Communication for CoAP", Work 996 in Progress, Internet-Draft, draft-ietf-core-oscore- 997 groupcomm-09, 23 June 2020, . 1000 [I-D.mcbride-edge-data-discovery-overview] 1001 McBride, M., Kutscher, D., Schooler, E., and C. Bernardos, 1002 "Edge Data Discovery for COIN", Work in Progress, 1003 Internet-Draft, draft-mcbride-edge-data-discovery- 1004 overview-03, 29 January 2020, . 1008 [I-D.sarathchandra-coin-appcentres] 1009 Sarathchandra, C., Trossen, D., and M. Boniface, "In- 1010 Network Computing for App-Centric Micro-Services", Work in 1011 Progress, Internet-Draft, draft-sarathchandra-coin- 1012 appcentres-02, 28 February 2020, . 1016 [ISO_TR] "Information Technology - Cloud Computing - Edge Computing 1017 Landscape", ISO/IEC TR 23188 , 2018. 1019 [Jeong] Jeong, T., Chung, J., Hong, J.W., and S. Ha, "Towards a 1020 distributed computing framework for fog", IEEE Fog World 1021 Congress (FWC), pages 1-6 , 2017. 1023 [Kelly] Kelly, R., "Internet of Things Data to Top 1.6 Zettabytes 1024 by 2022", 2015, 1025 . 1029 [Khan] Khan, L.U., Yaqoob, I., Tran, N.H., Kazmi, S.M.A., Dang, 1030 T.N., and C.S. Hong, "Edge Computing Enabled Smart Cities: 1031 A Comprehensive Survey", arXiv:1909.08747 , 2019. 1033 [LFEDGE-EVE] 1034 Linux Foundation, ., "Project Edge Virtualization Engine 1035 (EVE)", Portal , 2020, 1036 . 1038 [Li] Li, Y., Chen, Y., Lan, T., and G. Venkataramani, "Mobiqor: 1039 Pushing the envelope of mobile edge computing via quality- 1040 of-result optimization", IEEE 37th International 1041 Conference on Distributed Computing Systems (ICDCS), pages 1042 1261-1270 , 2017. 1044 [Lin] Lin, J., Yu, W., Zhang, N., Yang, X., Zhang, H., and W. 1045 Zhao, "A survey on Internet of Things: Architecture, 1046 enabling technologies, security and privacy, and 1047 applications", IEEE Internet of Things J., vol. 4, no. 5, 1048 pp. 1125-1142 , 2017. 1050 [Liu] Liu, J., Bai, B., Zhang, J., and K.B. Letaief, "Cache 1051 placement in fog-rans: From centralized to distributed 1052 algorithms", IEEE Transactions on Wireless Communications, 1053 16(11):7039-7051 , 2017. 1055 [Mahadev] Satyanarayanan, M., "The Emergence of Edge Computing", 1056 Computer, vol. 50, no. 1, pp. 30-39 , 2017. 1058 [Murshed] Murshed, M., Murphy, C., Hou, D., Khan, N., 1059 Ananthanarayanan, G., and F. Hussain, "Machine learning at 1060 the network edge: A survey", arXiv:1908.00080 , 2019. 1062 [NIST] Mell, P. and T. Grance, "The NIST definition of Cloud 1063 Computing", Natl. Inst. Stand. Technol, vol. 53, no. 6, p. 1064 50 , 2009. 1066 [NVIDIA] Grzywaczewski, A., "Training AI for Self-Driving Vehicles: 1067 the Challenge of Scale", NVIDIA Developer Blog , 2017, 1068 . 1071 [OpenFog] "OpenFog Reference Architecture for Fog Computing", 1072 OpenFog Consortium , 2017. 1074 [PseudoDynamicTesting] 1075 Ficco, M., Esposito, C., Xiang, Y., and F. Palmieri, 1076 "Pseudo-Dynamic Testing of Realistic Edge-Fog Cloud 1077 Ecosystems", IEEE Communications Magazine, Nov. 2017 , 1078 2017. 1080 [Renart] Renart, E.G., Diaz-Montes, J., and M. Parashar, "Data- 1081 driven stream processing at the edge", IEEE 1st 1082 International Conference on Fog and Edge Computing 1083 (ICFEC), pages 31-40 , 2017. 1085 [RFC6291] Andersson, L., van Helvoort, H., Bonica, R., Romascanu, 1086 D., and S. Mansfield, "Guidelines for the Use of the "OAM" 1087 Acronym in the IETF", BCP 161, RFC 6291, 1088 DOI 10.17487/RFC6291, June 2011, 1089 . 1091 [RFC7390] Rahman, A., Ed. and E. Dijk, Ed., "Group Communication for 1092 the Constrained Application Protocol (CoAP)", RFC 7390, 1093 DOI 10.17487/RFC7390, October 2014, 1094 . 1096 [Schafer] Schafer, D., Edinger, J., VanSyckel, S., Paluska, J.M., 1097 and C. Becker, "Tasklets: Overcoming Heterogeneity in 1098 Distributed Computing Systems", IEEE 36th International 1099 Conference on Distributed Computing Systems Workshops 1100 (ICDCSW), Nara, pp. 156-161 , 2016. 1102 [Shi] Shi, W., Cao, J., Zhang, Q., Li, Y., and L. Xu, "Edge 1103 computing: vision and challenges", IEEE Internet of Things 1104 J., vol. 3, no. 5, pp. 637-646 , 2016. 1106 [SimulatingFog] 1107 Svorobej, S. and . al, "Simulating Fog and Edge Computing 1108 Scenarios: An Overview and Research Challenges", MPDI 1109 Future Internet 2019 , 2019. 1111 [Weiner] Weiner, M., Jorgovanovic, M., Sahai, A., and B. Nikolie, 1112 "Design of a low-latency, high-reliability wireless 1113 communication system for control applications", IEEE Int. 1114 Conf. Commun. (ICC), Sydney, NSW, Australia, pp. 1115 3829-3835 , 2014. 1117 [Yangui] Yangui, S., Ravindran, P., Bibani, O., H Glitho, R., Ben 1118 Hadj-Alouane, N., Morrow, M.J., and P.A. Polakos, "A 1119 platform as-a-service for hybrid cloud/fog environments", 1120 IEEE International Symposium on Local and Metropolitan 1121 Area Networks (LANMAN), pages 1-7 , 2016. 1123 [Yates] Yates, R.D. and S.K. Kaul, "The Age of Information: Real- 1124 Time Status Updating by Multiple Sources", IEEE 1125 Transactions on Information Theory, vol. 65, no. 3, pp. 1126 1807-1827 , 2019. 1128 [Yousefpour] 1129 Yousefpour, A., Fung, C., Nguyen, T., Kadiyala, K., 1130 Jalali, F., Niakanlahiji, A., Kong, J., and J.P. Jue, "All 1131 one needs to know about fog computing and related edge 1132 computing paradigms: A complete survey", Journal of 1133 Systems Architecture, vol. 98, pp. 289-330 , 2019. 1135 [Zhang] Zhang, Q., Zhang, X., Zhang, Q., Shi, W., and H. Zhong, 1136 "Firework: Big data sharing and processing in 1137 collaborative edge environment", Fourth IEEE Workshop on 1138 Hot Topics in Web Systems and Technologies (HotWeb), pages 1139 20-25 , 2016. 1141 [Zhang2] Zhang, J., Chen, B., Zhao, Y., Cheng, X., and F. Hu, "Data 1142 Security and Privacy-Preserving in Edge Computing 1143 Paradigm: Survey and Open Issues", IEEE Access, vol. 6, 1144 pp. 18209-18237 , 2018. 1146 [_60802] IEC/IEEE, ., "Use Cases IEC/IEEE 60802 V1.3", IEC/IEEE 1147 60802 , 2018, . 1150 Authors' Addresses 1152 Jungha Hong 1153 ETRI 1154 218 Gajeong-ro, Yuseung-Gu 1155 Daejeon 1157 Email: jhong@etri.re.kr 1159 Yong-Geun Hong 1160 ETRI 1161 218 Gajeong-ro, Yuseung-Gu 1162 Daejeon 1164 Email: yghong@etri.re.kr 1166 Xavier de Foy 1167 InterDigital Communications, LLC 1168 1000 Sherbrooke West 1169 Montreal H3A 3G4 1170 Canada 1172 Email: xavier.defoy@interdigital.com 1173 Matthias Kovatsch 1174 Huawei Technologies Duesseldorf GmbH 1175 Riesstr. 25 C // 3.OG 1176 80992 Munich 1177 Germany 1179 Email: ietf@kovatsch.net 1181 Eve Schooler 1182 Intel 1183 2200 Mission College Blvd. 1184 Santa Clara, CA, 95054-1537 1185 United States of America 1187 Email: eve.m.schooler@intel.com 1189 Dirk Kutscher 1190 University of Applied Sciences Emden/Leer 1191 Constantiaplatz 4 1192 26723 Emden 1193 Germany 1195 Email: ietf@dkutscher.net