idnits 2.17.1 draft-hong-t2trg-iot-edge-computing-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (25 May 2020) is 1431 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-10) exists of draft-bernardos-sfc-fog-ran-07 == Outdated reference: A later version (-21) exists of draft-ietf-core-oscore-groupcomm-08 == Outdated reference: A later version (-05) exists of draft-mcbride-edge-data-discovery-overview-03 == Outdated reference: A later version (-04) exists of draft-sarathchandra-coin-appcentres-02 Summary: 1 error (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Hong 3 Internet-Draft Y-G. Hong 4 Intended status: Informational ETRI 5 Expires: 26 November 2020 X. de Foy 6 InterDigital Communications, LLC 7 M. Kovatsch 8 Huawei Technologies Duesseldorf GmbH 9 E. Schooler 10 Intel 11 D. Kutscher 12 University of Applied Sciences Emden/Leer 13 25 May 2020 15 IoT Edge Challenges and Functions 16 draft-hong-t2trg-iot-edge-computing-04 18 Abstract 20 Many IoT applications have requirements that cannot be met by the 21 traditional Cloud (aka Cloud computing). These include time 22 sensitivity, data volume, uplink cost, operation in the face of 23 intermittent services, privacy and security. As a result, the IoT is 24 driving the Internet toward Edge computing. This document outlines 25 the requirements of the emerging IoT Edge and its challenges. It 26 presents a general model, and major components of the IoT Edge, with 27 the goal to provide a common base for future discussions in T2TRG and 28 other IETF WGs and RGs. 30 Status of This Memo 32 This Internet-Draft is submitted in full conformance with the 33 provisions of BCP 78 and BCP 79. 35 Internet-Drafts are working documents of the Internet Engineering 36 Task Force (IETF). Note that other groups may also distribute 37 working documents as Internet-Drafts. The list of current Internet- 38 Drafts is at https://datatracker.ietf.org/drafts/current/. 40 Internet-Drafts are draft documents valid for a maximum of six months 41 and may be updated, replaced, or obsoleted by other documents at any 42 time. It is inappropriate to use Internet-Drafts as reference 43 material or to cite them other than as "work in progress." 45 This Internet-Draft will expire on 26 November 2020. 47 Copyright Notice 49 Copyright (c) 2020 IETF Trust and the persons identified as the 50 document authors. All rights reserved. 52 This document is subject to BCP 78 and the IETF Trust's Legal 53 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 54 license-info) in effect on the date of publication of this document. 55 Please review these documents carefully, as they describe your rights 56 and restrictions with respect to this document. Code Components 57 extracted from this document must include Simplified BSD License text 58 as described in Section 4.e of the Trust Legal Provisions and are 59 provided without warranty as described in the Simplified BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 64 2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 3 65 2.1. Internet of Things (IoT) . . . . . . . . . . . . . . . . 3 66 2.2. Cloud Computing . . . . . . . . . . . . . . . . . . . . . 4 67 2.3. Edge Computing . . . . . . . . . . . . . . . . . . . . . 4 68 2.4. Example of IoT Edge Computing Use Cases . . . . . . . . . 6 69 2.4.1. Smart Construction . . . . . . . . . . . . . . . . . 6 70 2.4.2. Smart Grid . . . . . . . . . . . . . . . . . . . . . 6 71 2.4.3. Smart Water System . . . . . . . . . . . . . . . . . 7 72 3. IoT Challenges Leading Towards Edge Computing . . . . . . . . 7 73 3.1. Time Sensitivity . . . . . . . . . . . . . . . . . . . . 7 74 3.2. Uplink Cost . . . . . . . . . . . . . . . . . . . . . . . 8 75 3.3. Resilience to Intermittent Services . . . . . . . . . . . 8 76 3.4. Privacy and Security . . . . . . . . . . . . . . . . . . 8 77 4. IoT Edge Computing Functions . . . . . . . . . . . . . . . . 9 78 4.1. Overview of IoT Edge Computing Today . . . . . . . . . . 9 79 4.2. General Model . . . . . . . . . . . . . . . . . . . . . . 10 80 4.3. OAM Components . . . . . . . . . . . . . . . . . . . . . 14 81 4.3.1. Virtualization Management . . . . . . . . . . . . . . 14 82 4.3.2. Resources Discovery and Authentication . . . . . . . 15 83 4.3.3. Edge Organization and Federation . . . . . . . . . . 15 84 4.4. Functional Components . . . . . . . . . . . . . . . . . . 16 85 4.4.1. External APIs . . . . . . . . . . . . . . . . . . . . 16 86 4.4.2. Communication Brokering . . . . . . . . . . . . . . . 16 87 4.4.3. In-Network Computation . . . . . . . . . . . . . . . 17 88 4.4.4. Edge Caching . . . . . . . . . . . . . . . . . . . . 18 89 4.4.5. Other Services . . . . . . . . . . . . . . . . . . . 19 90 4.5. Application Components . . . . . . . . . . . . . . . . . 19 91 4.5.1. IoT End Devices Management . . . . . . . . . . . . . 19 92 4.5.2. Data Management . . . . . . . . . . . . . . . . . . . 19 93 4.6. Simulation and Emulation Environments . . . . . . . . . . 20 94 5. Security Considerations . . . . . . . . . . . . . . . . . . . 20 95 6. Acknowledgment . . . . . . . . . . . . . . . . . . . . . . . 21 96 7. Informative References . . . . . . . . . . . . . . . . . . . 21 97 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 99 1. Introduction 101 Currently, many IoT services leverage the Cloud, since it can provide 102 virtually unlimited storage and processing power. The reliance of 103 IoT on back-end cloud computing brings additional advantages such as 104 flexibility and efficiency. Today's IoT systems are fairly static 105 with respect to integrating and supporting computation. It's not 106 that there is no computation, but systems are often limited to static 107 configurations (edge gateways, the Cloud). 109 However, IoT devices are creating vast amounts of data at the network 110 edge. To meet IoT use case requirements, that data increasingly is 111 being stored, processed, analyzed, and acted upon close to the data 112 producers. These requirements include time sensitivity, data volume, 113 uplink cost, resiliency in the face of intermittent connectivity, 114 privacy, and security, which cannot be addressed by today's 115 centralized cloud computing. These requirements suggest a more 116 flexible way to distribute computing (and storage) and to integrate 117 it in the edge-cloud continuum. We will refer to this integration of 118 edge computing and IoT as "IoT edge computing". Our draft describes 119 background, uses cases, challenges, and presents system models and 120 functional components. 122 2. Background 124 2.1. Internet of Things (IoT) 126 Since the term "Internet of Things" (IoT) was coined by Kevin Ashton 127 in 1999 working on Radio-Frequency Identification (RFID) technology 128 [Ashton], the concept of IoT has evolved. It now reflects a vision 129 of connecting the physical world to the virtual world of computers 130 using (wireless) networks over which Things can send and receive 131 information without human intervention. Recently, the term has 132 become more literal by actually connecting Things to the Internet and 133 converging on Internet and Web technology. 135 Things are usually embedded systems of various kinds, such as home 136 appliances, mobile equipment, wearable devices, etc. Things are 137 widely distributed, but typically have limited storage and processing 138 power, which raise concerns regarding reliability, performance, 139 energy consumption, security, and privacy [Lin]. This limited 140 storage and processing power leads to complementing IoT with cloud 141 computing. 143 2.2. Cloud Computing 145 Cloud computing has been defined in [NIST]: "cloud computing is a 146 model for enabling ubiquitous, convenient, on-demand network access 147 to a shared pool of configurable computing resources (e.g., networks, 148 servers, storage, applications, and services) that can be rapidly 149 provisioned and released with minimal management effort or service 150 provider interaction". Cloud computing has become a predominant 151 technology that offers virtually unlimited capacity in terms of 152 storage and processing power, at low cost. This offering enabled the 153 realization of a new computing model, in which virtualized resources 154 can be leased in an on-demand fashion, being provided as general 155 utilities. Companies like Amazon, Google, Facebook, etc. widely 156 adopted this paradigm for delivering services over the Internet, 157 gaining both economical and technical benefits [Botta]. 159 Today, an unprecedented volume and variety of data is generated by 160 Things and applications deployed in edge networks consume this data. 161 Some of these applications may need very short response times, some 162 may access personal data, while others may generate vast amounts of 163 data. Today's cloud-based service models are not suitable for these 164 applications, which can instead leverage edge computing. 166 2.3. Edge Computing 168 Edge computing, in some settings also referred to as fog computing, 169 is a new paradigm in which substantial computing and storage 170 resources are placed at the edge of the Internet, that is, in close 171 proximity to mobile devices, sensors, actuators, or machines. Edge 172 computing happens near data sources [Mahadev], or closer 173 (topologically, physically, in term of latency, etc.) to where 174 decisions or interactions with the physical world are happening. It 175 works on both downstream data on behalf of cloud services and 176 upstream data on behalf of IoT services. The term fog computing 177 usually represents the notion of a multi-tiered edge computing, that 178 is, several layers of compute infrastructure between the end devices 179 and the cloud. 181 An edge device is any computing or networking resource residing 182 between data sources and cloud-based datacenters. In edge computing, 183 end devices not only consume data, but also produce data. And at the 184 network edge, devices not only request services and information from 185 the cloud, but also handle computing tasks including processing, 186 storage, caching, and load balancing on data sent to and from the 187 cloud [Shi]. This does not preclude end devices from hosting 188 computation themselves when possible, independently or as part of a 189 distributed edge computing platform (this is also referred to as Mist 190 Computing). 192 Several standards defining organization and industry forums have 193 provided definitions of edge and fog computing: 195 * ISO defines edge computing as a "form of distributed computing in 196 which significant processing and data storage takes place on nodes 197 which are at the edge of the network" [ISO_TR]. 199 * ETSI defines multi-access edge computing as a "system which 200 provides an IT service environment and cloud-computing 201 capabilities at the edge of an access network which contains one 202 or more type of access technology, and in close proximity to its 203 users" [ETSI_MEC_01]. 205 * The Industrial Internet Consortium (formerly OpenFog) defines fog 206 computing as "a horizontal, system-level architecture that 207 distributes computing, storage, control and networking functions 208 closer to the users along a cloud-to-thing continuum" [OpenFog]. 210 Based on these definitions, we can summarize a general philosophy of 211 edge computing as to distribute the required functions close to users 212 and data, while the difference to classic local systems is the usage 213 of management and orchestration features adopted from cloud 214 computing. 216 Actors from various industries approach edge computing using 217 different terms and reference models, although in practice these 218 approaches are not incompatible and may integrate with each other: 220 * The telecommunication industry tends to use a model where edge 221 computing services are deployed over NFV infrastructure at 222 aggregation points, or in proximity to the user equipment (e.g., 223 gNodeBs) [ETSI_MEC_03]. 225 * Enterprise and campus solutions often interpret edge computing as 226 an "edge cloud", that is, a smaller data center directly connected 227 to the local network (often referred to as "on-premise"). 229 * The automation industry defines the edge as the connection point 230 between IT from OT (Operational Technology). Hence, here edge 231 computing sometimes refers to applying IT solutions to OT problems 232 such as analytics, more flexible user interfaces, or simply having 233 more compute power than an automation controller. 235 2.4. Example of IoT Edge Computing Use Cases 237 IoT edge computing can be used in home, industry, grid, healthcare, 238 city, transportation, agriculture, and/or education scenarios. We 239 discuss here only a few examples of such use cases, to point out 240 differentiating requirements. 242 2.4.1. Smart Construction 244 In traditional construction domain, heavy equipment and machinery 245 pose risks to humans and property. Thus, there have been many 246 attempts to deploy technology to protect lives and property in 247 construction sites. For example, measurements of noise, vibration, 248 and gas can be recorded and reported to an inspector. Today, data 249 produced by such measurements is collected by a local gateway and 250 transferred to a remote cloud server. This incurs transmission 251 costs, e.g., over a LTE connection, and storage costs, e.g., when 252 using Amazon Web Services. When an inspector needs to investigate an 253 incident, he checks the information stored on the cloud server. 255 To determine the exact cause of an incident, sensor data including 256 audio and video are transferred to a remote server. In this case, 257 audio and video data volume is typically very large and the cost of 258 transmission can be an issue. By leveraging IoT edge computing, 259 sensor data can be processed and analyzed on a gateway located within 260 or near a construction site. And with the help of statistical 261 analysis or machine learning technologies, we can predict future 262 incidents in advance and trigger an on-site alarm. Furthermore, 263 predicting the time of an incident can help reducing significantly 264 the volume and cost of transmitted data, by transmitting video at 265 high resolution during critical periods, while otherwise using a 266 lower resolution. 268 2.4.2. Smart Grid 270 In future smart city scenarios, the Smart Grid will be critical in 271 ensuring highly available/efficient energy control in city-wide 272 electricity management. Edge computing is expected to play a 273 significant role in those systems to improve transmission efficiency 274 of electricity; to react and restore power after a disturbance; to 275 reduce operation costs and reuse renewable energy effectively, since 276 these operations involve local decision making. In addition, edge 277 computing can help monitoring power generation and power demand, and 278 making local electrical energy storage decisions in the smart grid 279 system. 281 2.4.3. Smart Water System 283 The water system is one of the most important aspects of a city. 284 Effective use of water, and cost-effective and environment-friendly 285 water treatment are critical aspects of this system. Edge computing 286 can help with monitoring water consumption and transport, and with 287 predicting future water usage level. Examples of application 288 include: water harvesting, ground water monitoring, locally analyzing 289 collected information related to water control and management to 290 limit water losses. 292 3. IoT Challenges Leading Towards Edge Computing 294 This section describes challenges met by IoT, that are motivating the 295 adoption of edge computing for IoT. Those are distinct from research 296 challenges applicable to IoT edge computing, some of which will be 297 mentioned in Section 4.3. 299 IoT technology is used with more and more demanding applications,e.g. 300 in industrial, automotive or healthcare domains, leading to new 301 challenges. For example, industrial machines such as laser cutters 302 already produce over 1 terabyte per hour, and similar amounts can be 303 generated in autonomous cars [NVIDIA]. 90% of IoT data is expected to 304 be stored, processed, analyzed, and acted upon close to the source 305 [Kelly], as cloud computing models alone cannot address the new 306 challenges [Chiang]. 308 Below we discuss IoT use case requirements that are moving cloud 309 capabilities to be more proximate and more distributed and 310 disaggregated. 312 3.1. Time Sensitivity 314 Many industrial control systems, such as manufacturing systems, smart 315 grids, oil and gas systems, etc., often require stringent end-to-end 316 latency between the sensor and control node. While some IoT 317 applications may require latency below a few tens of milliseconds 318 [Weiner], industrial robots and motion control systems have use cases 319 for cycle times in the order of microseconds [_60802]. An important 320 aspect for real-time communications is not only the latency, but also 321 guarantees for jitter. This means control packets need to arrive 322 with as little variation as possible and within a strict deadline. 323 Given the best-effort characteristics of the Internet, or in some 324 cases given speed-of-light limitations, this challenge is virtually 325 impossible to address, without comprehending end-to-end guarantees 326 for individual message delivery and continuous data flows. 328 3.2. Uplink Cost 330 Many IoT deployments are not challenged by a constrained network 331 bandwidth to the cloud. The fifth generation mobile networks (5G) 332 and Wi-Fi 6 both theoretically top out at 10 gigabits per second 333 (i.e., 4.5 terabyte per hour), which enables high-bandwidth uplinks. 334 However, the resulting cost for high-bandwidth connectivity to upload 335 all data to the cloud is unjustifiable and impractical for most IoT 336 applications. In some settings, e.g. in aeronautical communication, 337 higher communication costs reduce the amount of data that can be 338 practically uploaded even further. 340 3.3. Resilience to Intermittent Services 342 Many IoT devices such as sensors, data collectors, actuators, 343 controllers, etc. have very limited hardware resources and cannot 344 rely solely on their limited resources to meet all their computing 345 and/or storage needs. They require reliable, uninterrupted or 346 resilient services to augment their capabilities in order to fulfill 347 their application tasks. This is hard and partly impossible to 348 achieve with cloud services for systems such as vehicles, drones, or 349 oil rigs that have intermittent network connectivity. The dual is 350 also true, a cloud back-end might want to have a reading of the 351 device even if it's currently asleep. 353 3.4. Privacy and Security 355 When IoT services are deployed at home, personal information can be 356 learned from detected usage data. For example, one can extract 357 information about employment, family status, age, and income by 358 analyzing smart meter data [ENERGY]. Policy makers started to 359 provide frameworks that limit the usage of personal data and put 360 strict requirements on data controllers and processors. However, 361 data stored indefinitely in the cloud also increases the risk of data 362 leakage, for instance, through attacks on rich targets. 364 Industrial systems are often argued to not have privacy implications, 365 as no personal data is gathered. Yet data from such systems is often 366 highly sensitive, as one might be able to infer trade secrets such as 367 the setup of production lines. Hence, the owner of these systems are 368 generally reluctant to upload IoT data to the cloud. 370 Furthermore, passive observers can perform traffic analysis on the 371 device-to-cloud path. Hiding traffic patterns associated with sensor 372 networks can therefore be another requirement for edge computing. 374 4. IoT Edge Computing Functions 376 In this section we first look at the current state of IoT edge 377 computing Section 4.1, and then define a general system model 378 Section 4.2. This provides context for IoT edge computing functions, 379 which are listed in Section 4.3. 381 4.1. Overview of IoT Edge Computing Today 383 This section provides an overview of today's IoT edge computing 384 field, based on a limited review of standards, research, open-source 385 and proprietary products in 386 [I-D-defoy-t2trg-iot-edge-computing-background]. 388 IoT gateways, both open-source (such as EdgeX Foundry or Home Edge) 389 and proprietary (such as Amazon Greengrass, Microsoft Azure IoT Edge, 390 Google Cloud IoT Core, and gateways from Bosh, Siemens), represent a 391 common class of IoT edge computing products, where the gateway is 392 providing a local service on customer premises, and is remotely 393 managed through a cloud service. IoT communication protocols are 394 typically used between IoT devices and the gateway, including CoAP, 395 MQTT and many specialized IoT protocols (such as OPC UA and DDS in 396 the Industrial IoT space), while the gateway communicates with the 397 distant cloud typically using HTTPS. Virtualization platforms enable 398 the deployment of virtual edge computing functions (as VMs, 399 application containers, etc.), including IoT gateway software, on 400 servers in the mobile network infrastructure (at base station and 401 concentration points), in edge datacenters (in central offices) or 402 regional datacenters located near central offices. End devices are 403 envisioned to become computing devices in forward looking projects, 404 but are not commonly used as such today. 406 Physical or virtual IoT gateways can host application programs, which 407 are typically built using an SDK to access local services through a 408 programmatic API. Edge cloud system operators host their customers' 409 applications VMs or containers on servers located in or near access 410 networks, which can implement local edge services. For example, 411 mobile networks can provide edge services for radio network 412 information, location and bandwidth management. 414 Life cycle management of services and applications on physical IoT 415 gateways is often cloud-based. Edge cloud management platforms and 416 products (such as StarlingX, Akraino Edge Stack, Mobile EdgeX) adapt 417 cloud management technologies (e.g., Kubernetes) to the edge cloud, 418 i.e., to smaller, distributed computing devices running outside a 419 controlled data center. Services and application life-cycle is 420 typically using a NFV-like management and orchestration model. 422 The platform typically includes services to advertise or consume APIs 423 (e.g., Mp1 interface in ETSI MEC supports service discovery and 424 communication), and enables communicating with local and remote 425 endpoints (e.g., message routing function in IoT gateways). The 426 service platform is typically extensible by edge applications, since 427 they can advertise an API that other edge applications can consume. 428 IoT communication services include protocols translation, analytics 429 and transcoding. Communication between edge computing devices is 430 enabled in tiered deployments or distributed deployments. 432 An edge cloud platform may enable pass-through without storage or 433 local storage (e.g., on IoT gateways). Some edge cloud platforms use 434 a distributed form of storage such as an ICN network (e.g., NFN nodes 435 can store data in NDN) or a distributed storage platform (e.g., 436 Ceph). External storage, e.g., on databases in distant or local IT 437 cloud, is typically used for filtered data deemed worthy of long term 438 storage, although in some case it may be for all data, for example 439 when required for regulatory reasons. 441 Stateful computing is supported on platforms hosting native programs, 442 VMs or containers. Stateless computing is supported on platforms 443 providing a "serverless computing" service (a.k.a. function-as- 444 a-service), or on systems based on named function networking. 446 In many IoT use cases, a typical network usage pattern is high volume 447 uplink with some form of traffic reduction enabled by processing over 448 edge computing devices. Alternatives to traffic reduction include 449 deferred transmission (to off-peak hours or using physical shipping). 450 Downlink traffic includes application control and software updates. 451 Other, downlink-heavy traffic patterns are not excluded but are more 452 often associated with non-IoT usage (e.g., video CDNs). 454 4.2. General Model 456 Edge computing is expected to play an important role in deploying new 457 IoT services integrated with Big Data and AI, enabled by flexible in- 458 network computing platforms. Although there are lots of approaches 459 to edge computing, we attempt to lay out a general model and list 460 associated logical functions in this section. In practice, this 461 model can map to different architectures, such as: 463 * A single IoT gateway, or a hierarchy of IoT gateways, typically 464 connected to the cloud (e.g., to extend the traditionally cloud- 465 based management of IoT devices and data to the edge). A common 466 role of an IoT Gateway is to provide access to an heterogeneous 467 set of IoT devices/sensors; handle IoT data; and deliver IoT data 468 to its final destination in a cloud network. Whereas an IoT 469 gateway needs interactions with cloud like as conventional cloud 470 computing, it can also operate independently. 472 * A set of distributed computing nodes, e.g., embedded in switches, 473 routers, edge cloud servers or mobile devices. Some IoT end 474 devices can have enough computing capabilities to participate in 475 such distributed systems due to advances in hardware technology. 476 In this model, edge computing nodes can collaborate with each 477 other to share their resources. 479 +---------------------+ 480 | Remote network | +---------------+ 481 |(e.g., cloud network)| | Service | 482 +-----------+---------+ | Operator | 483 | +------+--------+ 484 | | 485 +--------------+-------------------+-----------+ 486 | Edge Computing Domain | 487 | | 488 | One or more Computing Nodes | 489 | (IoT gateway, end devices, switches, | 490 | routers, mini/micro-datacenters, etc.) | 491 | | 492 | OAM Components | 493 | - Virtualization Management | 494 | - Resources Discovery and Authentication | 495 | - Edge Organization and Federation | 496 | - ... | 497 | | 498 | Functional Components | 499 | - External APIs | 500 | - Communication Brokering | 501 | - In-Network Computation | 502 | - Edge Caching | 503 | - Other Services | 504 | - ... | 505 | | 506 | Application Components | 507 | - IoT End Devices Management | 508 | - Data Management | 509 | - ... | 510 | | 511 +------+--------------+-------- - - - -+- - - -+ 512 | | | | | 513 | | +-----+--+ 514 +----+---+ +-----+--+ | |compute | | 515 | End | | End | ... |node/end| 516 |Device 1| |Device 2| ...| |device n| | 517 +--------+ +--------+ +--------+ 518 + - - - - - - - -+ 520 Figure 1: Model of IoT Edge Computing 522 In the above model, the edge computing domain is interconnected with 523 IoT end devices (southbound connectivity) and possibly with a remote/ 524 cloud network (northbound connectivity), and with a service 525 operator's system. Edge computing nodes provide multiple logical 526 functions, or components, which may not all be present in a given 527 system. They may be implemented in a centralized or distributed 528 fashion, in the edge network, or through some interworking between 529 the edge network and a remote cloud network. 531 +----------------------------------------------+ 532 | Edge Computing Domain | 533 | | 534 | +--------+ +--------+ +--------+ | 535 | |Compute | |Compute | |Compute | | 536 | |node/End| |node/End| .... |node/End| | 537 | |device 1| |device 2| .... |device m| | 538 | +----+---+ +----+---+ +----+---+ | 539 | | | | | 540 | +---+-------------+-----------------+--+ | 541 | | IoT Edge Gateway | | 542 | +-----------+-------------------+------+ | 543 | | | | 544 +--------------+-------------------+-----------+ 545 | | 546 +-----------+---------+ +------+-------+ 547 | Remote network | | Service | 548 |(e.g., cloud network)| | Operator(s) | 549 +-----------+---------+ +------+-------+ 550 | | 551 +--------------+-------------------+-----------+ 552 | | | | 553 | +-----------+-------------------+------+ | 554 | | IoT Edge Gateway | | 555 | +---+-------------+-----------------+--+ | 556 | | | | | 557 | +----+---+ +----+---+ +----+---+ | 558 | |Compute | |Compute | |Compute | | 559 | |node/End| |node/End| .... |node/End| | 560 | |device 1| |device 2| .... |device n| | 561 | +--------+ +--------+ +--------+ | 562 | | 563 | Edge Computing Domain | 564 +----------------------------------------------+ 566 Figure 2: Example: Machine Learning over a Distributed IoT Edge 567 Computing System 569 In the above example of system, the edge computing domain is composed 570 of IoT edge gateways and IoT end devices which are also used as 571 computing nodes. Edge computing domains are connected with a remote/ 572 cloud network, and with their respective service operator's system. 573 IoT end devices/computing nodes provide logical functions, as part of 574 a distributed machine learning application. The processing 575 capabilities in IoT end devices being limited, they require the 576 support of other nodes: the training process for AI services is 577 executed at IoT edge gateways or cloud networks and the prediction 578 (inference) service is executed in the IoT end devices. 580 We now attempt to enumerate major edge computing domain components. 581 They are here loosely organized into OAM, functional and application 582 components, with the understanding that the distinction between these 583 classes may not always be clear, depending on actual system 584 architectures. Some representative research challenges are 585 associated with those functions. We used input from co-authors, IRTF 586 attendees and some comprehensive reviews of the field ([Yousefpour], 587 [Zhang2], [Khan]). 589 4.3. OAM Components 591 Edge computing OAM goes beyond the network-related OAM functions 592 listed in [RFC6291]. Besides infrastructure (network, storage and 593 computing resources), edge computing systems can also include 594 computing environments (for VMs, software containers, functions), IoT 595 end devices, data and code. 597 Operation related functions include performance monitoring for 598 service level agreement measurement; fault management and 599 provisioning for links, nodes, compute and storage resources, 600 platforms and services. Administration covers network/compute/ 601 storage resources, platforms and services discovery, configuration 602 and planning. Management covers monitoring and diagnostics of 603 failures, as well as means to minimize their occurrence and take 604 corrective actions. This may include software updates management, 605 high service availability through redundancy and multipath 606 communication. Centralized (e.g., SDN) and decentralized management 607 systems can be used. 609 We further detail a few OAM components. 611 4.3.1. Virtualization Management 613 Some IoT edge computing systems make use of virtualized (compute, 614 storage and networking) resources, which need to be allocated and 615 configured. This function is covered to a large extent by ETSI NFV 616 and MEC standards activities. Projects such as [LFEDGE-EVE] further 617 cover virtualization and its management into distributed edge 618 computing settings. 620 Related challenges include: 622 * Minimizing virtual function instantiation time and resource usage 623 * Integration of edge computing with virtualized radio networks (Fog 624 RAN) [I-D.bernardos-sfc-fog-ran] and with 5G access networks 626 * Handling of multi-tenancy with regards to limited resources at the 627 network edge 629 4.3.2. Resources Discovery and Authentication 631 Discovery and authentication may target platforms, infrastructure 632 resources, such as compute, network and storage, but also other 633 resources such as IoT end devices, sensors, data, code units, 634 services, applications or users interacting with the system. Broker- 635 based solutions can be used, e.g. using an IoT gateway as broker to 636 discover IoT resources. Today, centralized gateway-based systems 637 rely, for device authentication, on the installation of a secret on 638 IoT end devices and on computing devices (e.g., a device certificate 639 stored in a hardware security module). 641 Related challenges include: 643 * Discovery, authentication and trust establishment between end 644 devices, compute nodes and platforms, with regards to concerns 645 such as mobility, heterogeneity, scale, multiple trust domains, 646 constrained devices, anonymity and traceability 648 * Intermittent connectivity to the Internet, preventing relying on a 649 third-party authority [Echeverria] 651 * Resiliency to failures [Harchol], denial of service attacks, 652 easier physical access for attackers 654 4.3.3. Edge Organization and Federation 656 In a distributed system context, once edge devices have discovered 657 and authenticated each other, they can be organized, or self- 658 organize, into hierarchies or clusters. Organization may range from 659 centralized to peer-to-peer. Such groups can also form federations 660 with other edge or remote clouds. 662 Related challenges include: 664 * Sharing resources in multi-vendor/operator scenarios, with a goal 665 to optimize criteria such as profit [Anglano], resource usage, 666 latency or energy consumption 668 * Support for scaling, and enabling fault-tolerance or self-healing 669 [Jeong]. Besides using hierarchical organization to cope with 670 scaling, another available and possibly complementary mechanism is 671 multicast ([RFC7390] [I-D.ietf-core-oscore-groupcomm]) 673 * Capacity planning, placement of infrastructure nodes to minimize 674 delay [Fan], cost, energy, etc. 676 * Incentives for participation, e.g. in peer-to-peer federation 677 schemes 679 4.4. Functional Components 681 4.4.1. External APIs 683 An IoT edge cloud may provide a northbound data plane or management 684 plane interface to a remote network, e.g., a cloud, home or 685 enterprise network. This interface does not exist in standalone 686 (local-only) scenarios. To support such an interface when it exists, 687 an edge computing component needs to expose an API, deal with 688 authentication and authorization, support secure communication. 690 An IoT edge cloud may provide an API or interface to local or mobile 691 users, for example to provide access to services and applications, or 692 to manage data published by local/mobile devices. 694 Related challenges include: 696 * Defining edge computing abstractions suitable for users and cloud 697 systems to interact with edge computing systems. In one example, 698 this interaction can be based on the PaaS model [Yangui] 700 4.4.2. Communication Brokering 702 A typical function of IoT edge computing is to facilitate 703 communication with IoT end devices: for example, enable clients to 704 register as recipients for data from devices, as well as forwarding/ 705 routing of traffic to or from IoT end devices, enabling various data 706 discovery and redistribution patterns, e.g., north-south with clouds, 707 east-west with other edge devices 708 [I-D.mcbride-edge-data-discovery-overview]. Another related aspect 709 is dispatching of alerts and notifications to interested consumers 710 both inside and outside of the edge computing domain. Protocol 711 translation, analytics and transcoding may also be performed when 712 necessary. 714 Communication brokering may be centralized in some systems, e.g., 715 using a hub-and-spoke message broker, or distributed like with 716 message buses, possibly in a layered bus approach. Distributed 717 systems may leverage direct communication between end devices, over 718 device-to-device links. A broker can ensure communication 719 reliability, traceability, and in some cases transaction management. 721 Related challenges include: 723 * Enabling secure and resilient communication between IoT end 724 devices and remote cloud, e.g. through multipath support 726 4.4.3. In-Network Computation 728 A core function of IoT edge computing is to enable computation 729 offloading, i.e., to perform computation on an edge node on behalf of 730 a device or user, but also to orchestrate computation (in a 731 centralized or distributed manner) and manage applications lifecycle. 732 Support for in-network computation may vary in term of capability, 733 e.g., computing nodes can host virtual machines, software containers, 734 software actors or unikernels able run stateful or stateless code, or 735 a rule engine providing an API to register actions in response to 736 conditions such as IoT device ID, sensor values to check, thresholds, 737 etc. 739 QoS can be provided in some systems through the combination of 740 network QoS (e.g., traffic engineering or wireless resource 741 scheduling) and compute/storage resource allocations. For example in 742 some systems a bandwidth manager service can be exposed to enable 743 allocation of bandwidth to/from an edge computing application 744 instance. 746 Related challenges include: 748 * (Computation placement) Selecting, in a centralized or 749 distributed/peer-to-peer manner, an appropriate compute device 750 based on available resources, location of data input and data 751 sinks, compute node properties, etc., and with varying goals 752 including for example end-to-end latency, privacy, high 753 availability, energy conservation, network efficiency (e.g. using 754 load balancing techniques to avoid congestion) 756 * Onboarding code on a platform or compute device, and invoking 757 remote code execution, possibly as part of a distributed 758 programming model and with respect to similar concerns of latency, 759 privacy, etc. These operations should deal with heterogeneous 760 compute nodes [Schafer], and may in some cases also support end 761 devices as compute nodes 763 * Adapting Quality of Results (QoR) for applications where a perfect 764 result is not necessary [Li] 766 * Assisted or automatic partitioning of code 767 [I-D.sarathchandra-coin-appcentres] 769 * Supporting computation across trust domains, e.g. verifying 770 computation results 772 * Relocating an instance from one compute node to another, while 773 maintaining a given service level. 775 * Session continuity when communicating with end devices that are 776 mobile, possibly at high speed (e.g. in vehicular scenarios) 778 * Defining, managing and verifying SLAs for edge computing systems. 779 Pricing is a related challenge 781 4.4.4. Edge Caching 783 A purpose of local caching may be to enable local data processing 784 (e.g., pre-processing or analysis), or to enable delayed virtual or 785 physical shipping. A responsibility of the edge caching component is 786 to manage data persistence, e.g., to schedule removal of data when it 787 is no longer needed. Another aspect of this component may be to 788 authenticate and encrypt data. It can for example take the form of a 789 distributed storage system. 791 Related challenges include 793 * (Cache and data placement) Using cache positioning and data 794 placement strategies to minimize data retrieval delay [Liu], 795 energy consumption. Caches may be positioned in the access 796 network infrastructure or may be on end devices using device-to- 797 device communication 799 * Maintaining data consistency, freshness and privacy in systems 800 that are distributed, constrained and dynamic (e.g. due to end 801 devices and computing nodes churn or mobility). For example, age 802 of information [Yates], a performance metric that captures the 803 timeliness of information from a sender (e.g. an IoT device), can 804 be exposed to networks to enable tradeoffs in this problem space 806 4.4.5. Other Services 808 Data generated by IoT devices and associated information obtained 809 from the access network may be used to provide high level services 810 such as end device location, radio network information and bandwidth 811 management. 813 4.5. Application Components 815 IoT edge computing can host applications such as the ones mentioned 816 in Section 2.4. While describing components of individual 817 applications is out of our scope, some of those applications share 818 similar functions, such as IoT end device management, data 819 management, described below. 821 4.5.1. IoT End Devices Management 823 IoT end device management includes managing information about the IoT 824 devices, including their sensors, how to communicate with them, etc. 825 Edge computing addresses the scalability challenges from the massive 826 number of IoT end devices by separating the scalability domain into 827 edge/local networks and remote network. 829 Challenges listed in Section 4.3.2 may be applicable to IoT end 830 devices management as well. 832 4.5.2. Data Management 834 Data storage and processing at the edge is a major aspect of IoT edge 835 computing, directly addressing high level IoT challenges listed in 836 Section 3. Data analysis such as performed in AI/ML tasks performed 837 at the edge may benefit from specialized hardware support on 838 computing nodes. 840 Related challenges include: 842 * Addressing concerns on resource usage, security and privacy when 843 sharing, discovering or managing data. For example by presenting 844 data in views composed of an aggregation of related data [Zhang], 845 protecting data communication between authenticated peers 846 [Basudan], classifying data (e.g., in terms of privacy, 847 importance, validity, etc.), compressing data 849 * Data driven programming models [Renart], e.g. event-based, 850 including handling of naming and data abstractions 852 * Addressing concerns such as limited resources, privacy, dynamic 853 and heterogeneous environment, to deploy machine learning at the 854 edge. For example, making machine learning more lightweight and 855 distributed, supporting shorter training time and simplified 856 models, and supporting models that can be compressed for efficient 857 communication [Murshed] 859 * While edge computing can support IoT services independently of 860 cloud computing, it can also be connected to cloud computing. 861 Thus, the relationship of IoT edge computing to cloud computing, 862 with regard to data management, is another potential challenge 863 [ISO_TR] 865 4.6. Simulation and Emulation Environments 867 IoT Edge Computing brings new challenges to simulation and emulation 868 tools used by researchers and developers. A varied set of 869 applications, network and computing technologies can coexist in a 870 distributed system, which make modelling difficult. Scale, mobility 871 and resource management are additional challenges [SimulatingFog]. 873 Tools include simulators, where simplified application logic runs on 874 top of a fog network model, and emulators, where actual applications 875 can be deployed, typically in software containers, over a cloud 876 infrastructure (e.g. Docker, Kubernetes) itself running over a 877 network emulating edge network conditions such as variable delays, 878 throughput and mobility events. To gain in scale, emulated and 879 simulated systems can be used together in hybrid federation-based 880 approaches [PseudoDynamicTesting], while to gain in realism physical 881 devices can be interconnected with emulated systems. Examples of 882 related work and platforms include the publicly accessible MEC 883 sandbox work recently initiated in ETSI [ETSI_Sandbox], and open 884 source simulators and emulators ([AdvantEDGE] emulator and tools 885 cited in [SimulatingFog]). 887 5. Security Considerations 889 As discussed in Section 4.3.2, authentication and trust (between 890 computing nodes, management nodes, end devices) can be challenging as 891 scale, mobility and heterogeneity increase. The sometimes 892 disconnected nature of edge resources can prevent relying on a third- 893 party authority. Distributed edge computing is exposed to issues 894 with reliability and denial of service attacks. Personal or 895 proprietary IoT data leakage is also a major threat, especially due 896 to the distributed nature of the systems (Section 4.5.2). 898 However, edge computing also brings solutions in the security space: 899 maintaining privacy by computing sensitive data closer to data 900 generators is a major use case for IoT edge computing. An edge cloud 901 can be used to take actions based on sensitive data, or anonymizing, 902 aggregating or compressing data prior to transmitting to a remote 903 cloud server. Edge computing communication brokering functions can 904 also be used to secure communication between edge and cloud networks. 906 6. Acknowledgment 908 The authors would like to thank Joo-Sang Youn, Akbar Rahman, Michel 909 Roy, Robert Gazda, Rute Sofia, Thomas Fossati and Chonggang Wang for 910 their valuable comments and suggestions on this document. 912 7. Informative References 914 [AdvantEDGE] 915 "Mobile Edge Emulation Platform", Source Code Repository , 916 2020, . 918 [Anglano] Anglano, C., Canonico, M., Castagno, P., Guazzone, M., and 919 M. Sereno, "A game-theoretic approach to coalition 920 formation in fog provider federations", IEEE Third 921 International Conference on Fog and Mobile Edge Computing 922 (FMEC), pages 123-130 , 2018. 924 [Ashton] Ashton, K., "That Internet of Things thing", RFID J. vol. 925 22, no. 7, pp. 97-114 , 2009. 927 [Basudan] Basudan, S., Lin, X., and K. Sankaranarayanan, "A privacy- 928 preserving vehicular crowdsensing-based road surface 929 condition monitoring system using fog computing", IEEE 930 Internet of Things Journal, 4(3):772-782 , 2017. 932 [Botta] Botta, A., Donato, W., Persico, V., and A. Pescape, 933 "Integration of Cloud Computing and Internet of Things: A 934 survey", Future Gener. Comput. Syst., vol. 56, pp. 935 684-700 , 2016. 937 [Chiang] Chiang, M. and T. Zhang, "Fog and IoT: An overview of 938 research opportunities", IEEE Internet Things J., vol. 3, 939 no. 6, pp. 854-864 , 2016. 941 [Echeverria] 942 Echeverria, S., Klinedinst, D., Williams, K., and G. A 943 Lewis, "Establishing trusted identities in disconnected 944 edge environments", IEEE/ACM Symposium Edge Computing 945 (SEC), pages 51-63. , 2016. 947 [ENERGY] Beckel, C., Sadamori, L., Staake, T., and S. Santini, 948 "Revealing Household Characteristics from Smart Meter 949 Data", Energy, vol. 78, pp. 397-410 , 2014. 951 [ETSI_MEC_01] 952 ETSI, ., "Multi-access Edge Computing (MEC); Terminology", 953 ETSI GS 001 , 2019, . 956 [ETSI_MEC_03] 957 ETSI, ., "Mobile Edge Computing (MEC); Framework and 958 Reference Architecture", ETSI GS 003 , 2019, 959 . 962 [ETSI_Sandbox] 963 "Multi-access Edge Computing (MEC) MEC Sandbox Work Item", 964 Portal , 2020, 965 . 968 [Fan] Fan, Q. and N. Ansari, "Cost aware cloudlet placement for 969 big data processing at the edge", IEEE International 970 Conference on Communications (ICC), pages 1-6 , 2017. 972 [Harchol] Harchol, Y., Mushtaq, A., McCauley, J., Panda, A., and S. 973 Shenker, "Cessna: Resilient edge-computing", Workshop on 974 Mobile Edge Communications, pages 1-6. ACM , 2018. 976 [I-D-defoy-t2trg-iot-edge-computing-background] 977 de Foy, X., Hong, J., Hong, Y., Kovatsch, M., Schooler, 978 E., and D. Kutscher, "Machine learning at the network 979 edge: A survey", draft-defoy-t2trg-iot-edge-computing- 980 background-00 , 2020, . 984 [I-D.bernardos-sfc-fog-ran] 985 Bernardos, C., Rahman, A., and A. Mourad, "Service 986 Function Chaining Use Cases in Fog RAN", Work in Progress, 987 Internet-Draft, draft-bernardos-sfc-fog-ran-07, 11 March 988 2020, . 991 [I-D.ietf-core-oscore-groupcomm] 992 Tiloca, M., Selander, G., Palombini, F., and J. Park, 993 "Group OSCORE - Secure Group Communication for CoAP", Work 994 in Progress, Internet-Draft, draft-ietf-core-oscore- 995 groupcomm-08, 6 April 2020, . 998 [I-D.mcbride-edge-data-discovery-overview] 999 McBride, M., Kutscher, D., Schooler, E., and C. Bernardos, 1000 "Edge Data Discovery for COIN", Work in Progress, 1001 Internet-Draft, draft-mcbride-edge-data-discovery- 1002 overview-03, 29 January 2020, . 1006 [I-D.sarathchandra-coin-appcentres] 1007 Sarathchandra, C., Trossen, D., and M. Boniface, "In- 1008 Network Computing for App-Centric Micro-Services", Work in 1009 Progress, Internet-Draft, draft-sarathchandra-coin- 1010 appcentres-02, 28 February 2020, . 1014 [ISO_TR] "Information Technology - Cloud Computing - Edge Computing 1015 Landscape", ISO/IEC TR 23188 , 2018. 1017 [Jeong] Jeong, T., Chung, J., Hong, J.W., and S. Ha, "Towards a 1018 distributed computing framework for fog", IEEE Fog World 1019 Congress (FWC), pages 1-6 , 2017. 1021 [Kelly] Kelly, R., "Internet of Things Data to Top 1.6 Zettabytes 1022 by 2022", 2015, 1023 . 1027 [Khan] Khan, L.U., Yaqoob, I., Tran, N.H., Kazmi, S.M.A., Dang, 1028 T.N., and C.S. Hong, "Edge Computing Enabled Smart Cities: 1029 A Comprehensive Survey", arXiv:1909.08747 , 2019. 1031 [LFEDGE-EVE] 1032 Linux Foundation, ., "Project Edge Virtualization Engine 1033 (EVE)", Portal , 2020, 1034 . 1036 [Li] Li, Y., Chen, Y., Lan, T., and G. Venkataramani, "Mobiqor: 1037 Pushing the envelope of mobile edge computing via quality- 1038 of-result optimization", IEEE 37th International 1039 Conference on Distributed Computing Systems (ICDCS), pages 1040 1261-1270 , 2017. 1042 [Lin] Lin, J., Yu, W., Zhang, N., Yang, X., Zhang, H., and W. 1043 Zhao, "A survey on Internet of Things: Architecture, 1044 enabling technologies, security and privacy, and 1045 applications", IEEE Internet of Things J., vol. 4, no. 5, 1046 pp. 1125-1142 , 2017. 1048 [Liu] Liu, J., Bai, B., Zhang, J., and K.B. Letaief, "Cache 1049 placement in fog-rans: From centralized to distributed 1050 algorithms", IEEE Transactions on Wireless Communications, 1051 16(11):7039-7051 , 2017. 1053 [Mahadev] Satyanarayanan, M., "The Emergence of Edge Computing", 1054 Computer, vol. 50, no. 1, pp. 30-39 , 2017. 1056 [Murshed] Murshed, M., Murphy, C., Hou, D., Khan, N., 1057 Ananthanarayanan, G., and F. Hussain, "Machine learning at 1058 the network edge: A survey", arXiv:1908.00080 , 2019. 1060 [NIST] Mell, P. and T. Grance, "The NIST definition of Cloud 1061 Computing", Natl. Inst. Stand. Technol, vol. 53, no. 6, p. 1062 50 , 2009. 1064 [NVIDIA] Grzywaczewski, A., "Training AI for Self-Driving Vehicles: 1065 the Challenge of Scale", NVIDIA Developer Blog , 2017, 1066 . 1069 [OpenFog] "OpenFog Reference Architecture for Fog Computing", 1070 OpenFog Consortium , 2017. 1072 [PseudoDynamicTesting] 1073 Ficco, M., Esposito, C., Xiang, Y., and F. Palmieri, 1074 "Pseudo-Dynamic Testing of Realistic Edge-Fog Cloud 1075 Ecosystems", IEEE Communications Magazine, Nov. 2017 , 1076 2017. 1078 [Renart] Renart, E.G., Diaz-Montes, J., and M. Parashar, "Data- 1079 driven stream processing at the edge", IEEE 1st 1080 International Conference on Fog and Edge Computing 1081 (ICFEC), pages 31-40 , 2017. 1083 [RFC6291] Andersson, L., van Helvoort, H., Bonica, R., Romascanu, 1084 D., and S. Mansfield, "Guidelines for the Use of the "OAM" 1085 Acronym in the IETF", BCP 161, RFC 6291, 1086 DOI 10.17487/RFC6291, June 2011, 1087 . 1089 [RFC7390] Rahman, A., Ed. and E. Dijk, Ed., "Group Communication for 1090 the Constrained Application Protocol (CoAP)", RFC 7390, 1091 DOI 10.17487/RFC7390, October 2014, 1092 . 1094 [Schafer] Schafer, D., Edinger, J., VanSyckel, S., Paluska, J.M., 1095 and C. Becker, "Tasklets: Overcoming Heterogeneity in 1096 Distributed Computing Systems", IEEE 36th International 1097 Conference on Distributed Computing Systems Workshops 1098 (ICDCSW), Nara, pp. 156-161 , 2016. 1100 [Shi] Shi, W., Cao, J., Zhang, Q., Li, Y., and L. Xu, "Edge 1101 computing: vision and challenges", IEEE Internet of Things 1102 J., vol. 3, no. 5, pp. 637-646 , 2016. 1104 [SimulatingFog] 1105 Svorobej, S. and . al, "Simulating Fog and Edge Computing 1106 Scenarios: An Overview and Research Challenges", MPDI 1107 Future Internet 2019 , 2019. 1109 [Weiner] Weiner, M., Jorgovanovic, M., Sahai, A., and B. Nikolie, 1110 "Design of a low-latency, high-reliability wireless 1111 communication system for control applications", IEEE Int. 1112 Conf. Commun. (ICC), Sydney, NSW, Australia, pp. 1113 3829-3835 , 2014. 1115 [Yangui] Yangui, S., Ravindran, P., Bibani, O., H Glitho, R., Ben 1116 Hadj-Alouane, N., Morrow, M.J., and P.A. Polakos, "A 1117 platform as-a-service for hybrid cloud/fog environments", 1118 IEEE International Symposium on Local and Metropolitan 1119 Area Networks (LANMAN), pages 1-7 , 2016. 1121 [Yates] Yates, R.D. and S.K. Kaul, "The Age of Information: Real- 1122 Time Status Updating by Multiple Sources", IEEE 1123 Transactions on Information Theory, vol. 65, no. 3, pp. 1124 1807-1827 , 2019. 1126 [Yousefpour] 1127 Yousefpour, A., Fung, C., Nguyen, T., Kadiyala, K., 1128 Jalali, F., Niakanlahiji, A., Kong, J., and J.P. Jue, "All 1129 one needs to know about fog computing and related edge 1130 computing paradigms: A complete survey", Journal of 1131 Systems Architecture, vol. 98, pp. 289-330 , 2019. 1133 [Zhang] Zhang, Q., Zhang, X., Zhang, Q., Shi, W., and H. Zhong, 1134 "Firework: Big data sharing and processing in 1135 collaborative edge environment", Fourth IEEE Workshop on 1136 Hot Topics in Web Systems and Technologies (HotWeb), pages 1137 20-25 , 2016. 1139 [Zhang2] Zhang, J., Chen, B., Zhao, Y., Cheng, X., and F. Hu, "Data 1140 Security and Privacy-Preserving in Edge Computing 1141 Paradigm: Survey and Open Issues", IEEE Access, vol. 6, 1142 pp. 18209-18237 , 2018. 1144 [_60802] IEC/IEEE, ., "Use Cases IEC/IEEE 60802 V1.3", IEC/IEEE 1145 60802 , 2018, . 1148 Authors' Addresses 1150 Jungha Hong 1151 ETRI 1152 218 Gajeong-ro, Yuseung-Gu 1153 Daejeon 1155 Email: jhong@etri.re.kr 1157 Yong-Geun Hong 1158 ETRI 1159 218 Gajeong-ro, Yuseung-Gu 1160 Daejeon 1162 Email: yghong@etri.re.kr 1164 Xavier de Foy 1165 InterDigital Communications, LLC 1166 1000 Sherbrooke West 1167 Montreal H3A 3G4 1168 Canada 1170 Email: xavier.defoy@interdigital.com 1171 Matthias Kovatsch 1172 Huawei Technologies Duesseldorf GmbH 1173 Riesstr. 25 C // 3.OG 1174 80992 Munich 1175 Germany 1177 Email: ietf@kovatsch.net 1179 Eve Schooler 1180 Intel 1181 2200 Mission College Blvd. 1182 Santa Clara, CA, 95054-1537 1183 United States of America 1185 Email: eve.m.schooler@intel.com 1187 Dirk Kutscher 1188 University of Applied Sciences Emden/Leer 1189 Constantiaplatz 4 1190 26723 Emden 1191 Germany 1193 Email: ietf@dkutscher.net