idnits 2.17.1 draft-hong-t2trg-iot-edge-computing-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 125 has weird spacing: '... IoT on back-...' == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (17 January 2020) is 1561 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 1 error (**), 0 flaws (~~), 3 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Hong 3 Internet-Draft Y-G. Hong 4 Intended status: Informational ETRI 5 Expires: 20 July 2020 X. de Foy 6 InterDigital Communications, LLC 7 M. Kovatsch 8 Huawei Technologies Duesseldorf GmbH 9 E. Schooler 10 Intel 11 D. Kutscher 12 University of Applied Sciences Emden/Leer 13 17 January 2020 15 IoT Edge Challenges and Functions 16 draft-hong-t2trg-iot-edge-computing-02 18 Abstract 20 Many IoT applications have requirements that cannot be met by 21 the traditional Cloud (aka Cloud computing). These include time 22 sensitivity, data volume, uplink cost, operation in the face of 23 intermittent services, privacy and security. As a result, the IoT is 24 driving the Internet toward Edge computing. This document outlines 25 the requirements of the emerging IoT Edge and its challenges. It 26 presents a snapshot of the state-of-the-art, a general model, and 27 major components of the IoT Edge, with the goal to provide a common 28 base for future discussions in T2TRG and other IETF WGs and RGs. 30 Status of This Memo 32 This Internet-Draft is submitted in full conformance with the 33 provisions of BCP 78 and BCP 79. 35 Internet-Drafts are working documents of the Internet Engineering 36 Task Force (IETF). Note that other groups may also distribute 37 working documents as Internet-Drafts. The list of current Internet- 38 Drafts is at https://datatracker.ietf.org/drafts/current/. 40 Internet-Drafts are draft documents valid for a maximum of six months 41 and may be updated, replaced, or obsoleted by other documents at any 42 time. It is inappropriate to use Internet-Drafts as reference 43 material or to cite them other than as "work in progress." 45 This Internet-Draft will expire on 20 July 2020. 47 Copyright Notice 49 Copyright (c) 2020 IETF Trust and the persons identified as the 50 document authors. All rights reserved. 52 This document is subject to BCP 78 and the IETF Trust's Legal 53 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 54 license-info) in effect on the date of publication of this document. 55 Please review these documents carefully, as they describe your rights 56 and restrictions with respect to this document. Code Components 57 extracted from this document must include Simplified BSD License text 58 as described in Section 4.e of the Trust Legal Provisions and are 59 provided without warranty as described in the Simplified BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 64 2. Conventions and Terminology . . . . . . . . . . . . . . . . . 4 65 3. Background . . . . . . . . . . . . . . . . . . . . . . . . . 4 66 3.1. Internet of Things (IoT) . . . . . . . . . . . . . . . . 4 67 3.2. Cloud Computing . . . . . . . . . . . . . . . . . . . . . 4 68 3.3. Edge Computing . . . . . . . . . . . . . . . . . . . . . 5 69 3.4. Example of IoT Edge Computing Use Cases . . . . . . . . . 6 70 3.4.1. Smart Construction . . . . . . . . . . . . . . . . . 6 71 3.4.2. Smart Grid . . . . . . . . . . . . . . . . . . . . . 7 72 3.4.3. Smart Water System . . . . . . . . . . . . . . . . . 7 73 3.5. Common Aspects of Current IoT Edge Computing Service 74 Platforms . . . . . . . . . . . . . . . . . . . . . . . . 8 75 4. Challenges for IoT and Impacts of Edge Computing . . . . . . 9 76 4.1. Time Sensitivity . . . . . . . . . . . . . . . . . . . . 9 77 4.2. Uplink Cost . . . . . . . . . . . . . . . . . . . . . . . 10 78 4.3. Resilience to Intermittent Services . . . . . . . . . . . 10 79 4.4. Privacy and Security . . . . . . . . . . . . . . . . . . 10 80 5. IoT Edge Computing Functions . . . . . . . . . . . . . . . . 11 81 5.1. OAM Components . . . . . . . . . . . . . . . . . . . . . 13 82 5.1.1. Resources Discovery . . . . . . . . . . . . . . . . . 13 83 5.1.2. Virtualization Management . . . . . . . . . . . . . . 13 84 5.1.3. Authentication and Authorization . . . . . . . . . . 13 85 5.1.4. Edge Organization and Federation . . . . . . . . . . 14 86 5.2. Functional Components . . . . . . . . . . . . . . . . . . 14 87 5.2.1. External APIs . . . . . . . . . . . . . . . . . . . . 14 88 5.2.2. Communication Brokering . . . . . . . . . . . . . . . 14 89 5.2.3. In-Network Computation . . . . . . . . . . . . . . . 15 90 5.2.4. Edge Caching . . . . . . . . . . . . . . . . . . . . 15 91 5.2.5. Other Services . . . . . . . . . . . . . . . . . . . 15 92 5.3. Application Components . . . . . . . . . . . . . . . . . 15 93 5.3.1. IoT End Devices Management . . . . . . . . . . . . . 15 94 5.3.2. Data Management . . . . . . . . . . . . . . . . . . . 16 96 5.4. Simulation and Emulation Environments . . . . . . . . . . 16 97 6. Security Considerations . . . . . . . . . . . . . . . . . . . 16 98 7. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 16 99 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 17 100 8.1. Normative References . . . . . . . . . . . . . . . . . . 17 101 8.2. Informative References . . . . . . . . . . . . . . . . . 17 102 Appendix A. Overview of the IoT Edge Computing . . . . . . . . . 20 103 A.1. Open Source Projects . . . . . . . . . . . . . . . . . . 20 104 A.1.1. Gateway/CPE Platforms . . . . . . . . . . . . . . . . 20 105 A.1.2. Edge Cloud Management Platforms . . . . . . . . . . . 22 106 A.1.3. Related Projects . . . . . . . . . . . . . . . . . . 22 107 A.2. Products . . . . . . . . . . . . . . . . . . . . . . . . 23 108 A.2.1. IoT Gateways . . . . . . . . . . . . . . . . . . . . 23 109 A.2.2. Edge Cloud Platforms . . . . . . . . . . . . . . . . 23 110 A.3. Standards Initiatives . . . . . . . . . . . . . . . . . . 24 111 A.3.1. ETSI Multi-access Edge Computing . . . . . . . . . . 24 112 A.3.2. Edge Computing Support in 3GPP . . . . . . . . . . . 25 113 A.3.3. OpenFog and Industrial Internet Consortium . . . . . 25 114 A.3.4. Related Standards . . . . . . . . . . . . . . . . . . 26 115 A.4. Research Projects . . . . . . . . . . . . . . . . . . . . 26 116 A.4.1. Named Function Networking . . . . . . . . . . . . . . 26 117 A.4.2. 5G-CORAL . . . . . . . . . . . . . . . . . . . . . . 27 118 A.4.3. FLAME . . . . . . . . . . . . . . . . . . . . . . . . 27 119 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 28 121 1. Introduction 123 Currently, many IoT services leverage the Cloud, since it can provide 124 virtually unlimited storage and processing power. The reliance of 125 IoT on back-end cloud computing brings additional advantages such as 126 flexibility and efficiency. Today's IoT systems are fairly static 127 with respect to integrating and supporting computation. It's not 128 that there is no computation, but systems are often limited to static 129 configurations (edge gateways, the Cloud). 131 However, IoT devices are creating vast amounts of data at the network 132 edge. To meet IoT use case requirements, that data increasingly is 133 being stored, processed, analyzed, and acted upon close to the data 134 producers. These requirements include time sensitivity, data volume, 135 uplink cost, resiliency in the face of intermittent connectivity, 136 privacy, and security, which cannot be addressed by today's 137 centralized cloud computing. These requirements suggest a more 138 flexible way to distribute computing (and storage) and to integrate 139 it in the edge-cloud continuum. We will refer to this integration of 140 edge computing and IoT as "IoT edge computing". Our draft describes 141 uses cases, challenges, a proposed system model and derived 142 functional components. 144 2. Conventions and Terminology 146 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 147 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 148 document are to be interpreted as described in [RFC2119]. 150 3. Background 152 3.1. Internet of Things (IoT) 154 Since the term "Internet of Things" (IoT) was coined by Kevin Ashton 155 in 1999 working on Radio-Frequency Identification (RFID) technology 156 at the Auto-ID Center of the Massachusetts Institute of Technology 157 (MIT) [Ashton], the concept of IoT had been broadened to reflect the 158 vision of connecting the physical world to the virtual world of 159 computers using (wireless) sensor networks with any kind of 160 technology with which the Things can send and receive information 161 without human intervention. Recently, the term has become more 162 literal by actually connecting Things to the Internet and converging 163 on Internet and also Web technology. Things are usually embedded 164 systems of various kinds, such as home appliances, mobile equipment, 165 wearable devices, etc. Things are widely distributed, but typically 166 have limited storage and processing power, which raise concerns 167 regarding reliability, performance, energy consumption, security, and 168 privacy [Lin]. 170 3.2. Cloud Computing 172 Cloud computing has been defined in [NIST]: "cloud computing is a 173 model for enabling ubiquitous, convenient, on-demand network access 174 to a shared pool of configurable computing resources (e.g., networks, 175 servers, storage, applications, and services) that can be rapidly 176 provisioned and released with minimal management effort or service 177 provider interaction". Cloud computing has been a predominant 178 technology which has virtually unlimited capacity in terms of storage 179 and processing power. The availability of virtually unlimited 180 storage and processing capabilities at low cost enabled the 181 realization of a new computing model, in which virtualized resources 182 can be leased in an on-demand fashion, being provided as general 183 utilities. Companies like Amazon, Google, Facebook, etc. widely 184 adopted this paradigm for delivering services over the Internet, 185 gaining both economical and technical benefits [Botta]. 187 Now with IoT, we will reach the era of post-clouds where 188 unprecedented volume and variety of data will be generated by things 189 at edge/local networks and many applications will be deployed on the 190 edge networks to consume these IoT data. Some of the applications 191 may need very short response times, some may contain personal data, 192 and others may generate vast amounts of data. Today's cloud-based 193 service models are not suitable for these applications. 195 It is predicted that by 2019, 45% of the data created in IoT will be 196 stored, processed, analyzed and acted close to, or at the edge of the 197 network and about 50 billion devices will connect to the Internet by 198 2020 [Evans]. So, moving all data from edge/local networks to the 199 cloud data center may not be an efficient way anymore to process vast 200 amounts of data. 202 In cloud computing, users traditionally only consumed IoT data 203 through cloud services. Now, however, users are also producing IoT 204 data with their mobile devices. This change requires even more 205 functionality at edge/local networks [Shi], to support mobile edge 206 computing considerations. 208 3.3. Edge Computing 210 Edge computing, under certain aspects also referred to as fog 211 computing, is a new paradigm in which substantial computing and 212 storage resources are placed at the edge of the Internet, that is,in 213 close proximity to mobile devices, sensors, actuators, or machines, 214 so that computing happens near data sources [Mahadev], or closer to 215 where decisions or interactions with the physical world are 216 happening. It works on both downstream data on behalf of cloud 217 services and upstream data on behalf of IoT services. 219 An edge device is any computing or networking resource residing 220 between data sources and cloud-based datacenters. In edge computing, 221 end devices not only consume data, but also produce data. And at the 222 network edge, devices not only request services and information from 223 the cloud, but also handle computing tasks including processing, 224 storage, caching, and load balancing on data sent to and from the 225 cloud [Shi]. 227 The definition of edge computing from ISO is a "form of distributed 228 computing in which significant processing and data storage takes 229 place on nodes which are at the edge of the network" [ISO_TR]. 230 ETSI's definition of multi-access edge computing is a "system which 231 provides an IT service environment and cloud-computing capabilities 232 at the edge of an access network which contains one or more type of 233 access technology, and in close proximity to its users" 234 [ETSI_MEC_01]. 236 The similar concept of fog computing from the Industrial Internet 237 Consortium (formerly OpenFog) is "a horizontal, system-level 238 architecture that distributes computing, storage, control and 239 networking functions closer to the users along a cloud-to-thing 240 continuum" Appendix A.3.3. The term fog computing usually represents 241 the notion of a multi-tiered edge computing, that is, several layers 242 of compute infrastructure between the end devices and the cloud. 244 Based on these definitions, we can summarize a general philosophy of 245 edge computing as to distribute the required functions close to users 246 and data, while the difference to classic local systems is the usage 247 of management and orchestration features adopted from cloud 248 computing. 250 Actors from various industries approach edge computing using 251 different terms and reference models,although in practice these 252 approaches are not incompatible and may integrate with each other: 254 * The telecommunication industry tends to use a model where edge 255 computing services are deployed over NFV infrastructure at 256 aggregation points, or in proximity to the user equipment (e.g., 257 gNodeBs) Appendix A.3.1. 259 * Enterprise and campus solutions often interpret edge computing as 260 an "edge cloud", that is, a smaller data center directly connected 261 to the local network (often referred to as "on-premise"). 263 * The automation industry defines the edge as the connection point 264 between IT from OT (Operational Technology). Hence, here edge 265 computing sometimes referres to applying IT solutions to OT 266 problems such as analytics, more flexible user interfaces, or 267 simply having more compute power than an automation controller. 269 It is clear that the combination of these models leads to a multi- 270 tier edge computing solution as mentioned above. 272 3.4. Example of IoT Edge Computing Use Cases 274 IoT edge computing can be used in home, industry, grid, healthcare, 275 city, transportation, agriculture, and/or education scenarios. We 276 discuss here only a few examples of such use cases, to point out 277 differentiating requirements. 279 3.4.1. Smart Construction 281 In traditional construction domain, heavy equipment and machinery 282 pose risks to humans and property. Thus, there have been many 283 attempts to deploy technology to protect lives and property in 284 construction sites. For example, measurements of noise, vibration, 285 and gas can be recorded on a remote server and reported to an 286 inspector. Today, data produced by such measurements is collected by 287 a local gateway and transferred to a remote server. This incurs 288 transmission costs, e.g., over a LTE connection, and storage costs, 289 e.g., when using Amazon Web Services. When an inspector needs to 290 investigate an incident, he checks the information stored on a 291 server. 293 If we leverage IoT edge computing, sensor data can be processed and 294 analyzed on a gateway located within or near a construction site. 295 And with the help of statistical analysis or machine learning 296 technologies, we can predict future incidents in advance and this 297 prediction can trigger an on-site alarm and a notification to an 298 inspector. 300 To determine the exact cause of an incident, sensor data including 301 audio and video are transferred to a remote server. In this case, 302 audio and video data volume is typically very large and the cost of 303 transmission can be an issue. Edge computing can be leveraged to 304 predict the time of an incident, which can reduce the volume of 305 transmitted data; while during a normal time period audio and video 306 data may be transmitted with a low resolution, during an emergency, 307 this transmission may use a high resolution. This adjustment can 308 reduce transmission cost significantly. 310 3.4.2. Smart Grid 312 In future smart city scenarios, the Smart Grid will be critical in 313 ensuring highly available/efficient energy control in city-wide 314 electricity management. Edge computing is expected to play a 315 significant role in those systems to improve transmission efficiency 316 of electricity; to react and restore power after a disturbance; to 317 reduce operation costs and reuse renewable energy effectively, since 318 these operations involve local decision making. In addition, edge 319 computing can help monitoring power generation and power demand, and 320 making electrical energy storage decisions in the smart grid system. 322 3.4.3. Smart Water System 324 The water system is one of the most important aspects of a city. 325 Effective use of water, and cost-effective and environment-friendly 326 water treatment are critical aspects of this system. They can be 327 facilitated by edge computing in smart water systems, to help monitor 328 water consumption, transportation and prediction of future water use. 329 For example, water harvesting and ground water monitoring will be 330 supported through edge computing. Edge computing will also enable 331 locally analyzing collected information related to water control and 332 management, and limit water losses. 334 3.5. Common Aspects of Current IoT Edge Computing Service Platforms 336 This section provides an overview of today's IoT edge computing 337 field, based on a limited review of standards, research, open-source 338 and proprietary products in Appendix A. 340 IoT gateways (Appendix A.2.1, Appendix A.1.1) represent a common 341 class of IoT edge computing products, where the gateway is providing 342 a local service on customer premises, and is remotely managed through 343 a cloud service. IoT communication protocols are typically used 344 between IoT devices and the gateway, including CoAP, MQTT and many 345 specialized IoT protocols (such as OPC UA and DDS in the Industrial 346 IoT space), while the gateway communicates with the distant cloud 347 typically using HTTPS. Virtualization platforms enable the 348 deployment of virtual edge computing functions (as VMs, application 349 containers, etc.), including IoT gateway software, on servers in the 350 mobile network infrastructure (at base station and concentration 351 points), in edge datacenters (in central offices) or regional 352 datacenters located near central offices. End devices are envisioned 353 to become computing devices in forward looking projects, but are not 354 commonly used as such today. 356 Physical or virtual IoT gateways can host application programs, which 357 are typically built using an SDK to access local services through a 358 programmatic API. Edge cloud system operators host their customers' 359 applications VMs or containers on servers located in or near access 360 networks, which can implement local edge services. For example, 361 mobile networks can provide edge services for radio network 362 information, location and bandwidth management. 364 Life cycle management of services and applications on physical IoT 365 gateways is often cloud-based. Edge cloud management platforms and 366 products (Appendix A.1.2, Appendix A.2.2) adapt cloud management 367 technologies (e.g., Kubernetes) to the edge cloud, i.e., to smaller, 368 distributed computing devices running outside a controlled data 369 center. Services and application life-cycle is typically using a 370 NFV-like management and orchestration model. 372 The platform typically includes services to advertise or consume APIs 373 (e.g., Mp1 interface in ETSI MEC supports service discovery and 374 communication), and enables communicating with local and remote 375 endpoints (e.g., message routing function in IoT gateways). The 376 service platform is typically extensible by edge applications, since 377 they can advertise an API that other edge applications can consume. 378 IoT communication services include protocols translation, analytics 379 and transcoding. Communication between edge computing devices is 380 enabled in tiered deployments or distributed deployments. 382 An edge cloud platform may enable pass-through without storage or 383 local storage (e.g., on IoT gateways). Some edge cloud platforms use 384 a distributed form of storage such as an ICN network (e.g., NFN nodes 385 can store data in NDN, Appendix A.4.1) or a distributed storage 386 platform (e.g., such as Ceph, Appendix A.1.2). External storage, 387 e.g., on databases in distant or local IT cloud, is typically used 388 for filtered data deemed worthy of long term storage, or in some 389 cases for all data, for example when required for regulatory reasons. 391 Stateful computing is supported on platforms hosting native programs, 392 VMs or containers. Stateless computing is supported on platforms 393 providing a "serverless computing" service (a.k.a. function-as- 394 a-service), or on systems based on named function networking. 396 In many IoT use cases, a typical network usage pattern is high volume 397 uplink with some form of traffic reduction enabled by processing over 398 edge computing devices. Alternatives to traffic reduction include 399 deferred transmission (to off-peak hours or using physical shipping). 400 Downlink traffic includes application control and software updates. 401 Other, downlink-heavy traffic patterns are not excluded but are more 402 often associated with non-IoT usage (e.g., video CDNs). 404 4. Challenges for IoT and Impacts of Edge Computing 406 As the IoT is maturing, systems are converging, deployments are 407 growing, and IoT technology is used with more and more demanding 408 applications such as industrial, automotive, or healthcare. This 409 leads to new challenges for the network infrastructure. In 410 particular, the amount of data created at the edge is expected to be 411 vast. Industrial machines such as laser cutters already produce over 412 1 terabyte per hour, the same applies for autonomous cars [NVIDIA]. 413 90% of IoT data is expected to be stored, processed, analyzed, and 414 acted upon close to the source [Kelly], as cloud computing models 415 alone cannot address the new challenges [Chiang]. Below we discuss 416 IoT use case requirements that are moving cloud capabilities to be 417 more proximate and more distributed and disaggregated. 419 4.1. Time Sensitivity 421 Many industrial control systems, such as manufacturing systems, smart 422 grids, oil and gas systems, etc., often require stringent end-to-end 423 latency between the sensor and control node. While some IoT 424 applications may require latency below a few tens of milliseconds 425 [Weiner], industrial robots and motion control systems have use cases 426 for cycle times in the order of microseconds [_60802]. An important 427 aspect for real-time communications is not only the latency, but also 428 guarantees for jitter. This means control packets need to arrive 429 with as little variation as possible with a strict deadline. Given 430 the best-effort characteristics of the Internet, this challenge is 431 virtually impossible to address, without comprehending end-to-end 432 guarantees for individual message delivery and continuous data flows. 434 4.2. Uplink Cost 436 Many IoT deployments are not challenged by a constrained network 437 bandwidth to the cloud. The fifth generation mobile networks (5G) 438 and Wi-Fi 6 both theoretically top out at 10 gigabits per second 439 (i.e., 4.5 terabyte per hour), which enables high-bandwidth uplinks. 440 However, the resulting cost for high-bandwidth connectivity to upload 441 all data to the cloud is unjustifiable and impractical for most IoT 442 applications. In some settings, e.g. in aeronautical communication, 443 higher communication costs reduce the amount of data that can be 444 practically uploaded even further. 446 4.3. Resilience to Intermittent Services 448 Many IoT devices such as sensors, data collectors, actuators, 449 controllers, etc. have very limited hardware resources and cannot 450 rely solely on their limited resources to meet all their computing 451 and/or storage needs. They require reliable, uninterrupted or 452 resilient services to augment their capabilities in order to fulfill 453 their application tasks. This is hard and partly impossible to 454 achieve with cloud services for systems such as vehicles, drones, or 455 oil rigs that have intermittent network connectivity. 457 4.4. Privacy and Security 459 When IoT services are deployed at home, personal information can be 460 learned from detected usage data. For example, one can extract 461 information about employment, family status, age, and income by 462 analyzing smart meter data [ENERGY]. Policy makers started to 463 provide frameworks that limit the usage of personal data and put 464 strict requirements on data controllers and processors. However, 465 data stored indefinitely in the cloud also increases the risk of data 466 leakage, for instance, through attacks on rich targets. 468 Industrial systems are often argued to not have privacy implications, 469 as no personal data is gathered. Yet data from such systems is often 470 highly sensitive, as one might be able to infer trade secrets such as 471 the setup of production lines. Hence, the owner of these systems are 472 generally reluctant to upload IoT data to the cloud. 474 5. IoT Edge Computing Functions 476 Edge computing is expected to play an important role in deploying new 477 IoT services integrated with Big Data and AI, enabled by flexible in- 478 network computing platforms. Although there are lots of approaches 479 to edge computing, we attempt to lay out a general model and list 480 associated logical functions in this section. In practice, this 481 model can map to different architectures, such as: 483 * A single IoT gateway, or a hierarchy of IoT gateways, typically 484 connected to the cloud (e.g., to extend the traditionally cloud- 485 based management of IoT devices and data to the edge). A common 486 role of an IoT Gateway is to provide access to an heterogeneous 487 set of IoT devices/sensors; handle IoT data; and deliver IoT data 488 to its final destination in a cloud network. Whereas an IoT 489 gateway needs interactions with cloud like as conventional cloud 490 computing, it can also operate independently. 492 * A set of distributed computing nodes, e.g., embedded in switches, 493 routers, edge cloud servers or mobile devices. In the future, 494 some IoT end devices may have enough computing capabilities to 495 participate in such distributed systems. In this model, edge 496 computing nodes can collaborate with each other to share their 497 resources. 499 +---------------------+ 500 | Remote network | +---------------+ 501 |(e.g., cloud network)| | Service | 502 +-----------+---------+ | Operator | 503 | +------+--------+ 504 | | 505 +--------------+-------------------+-----------+ 506 | Edge Computing Domain | 507 | | 508 | OAM Components | 509 | - Resources Discovery | 510 | - Virtualization Management | 511 | - Authentication | 512 | - Edge Organization and Federation | 513 | - IoT End Devices Management | 514 | - ... | 515 | | 516 | Functional Components | 517 | - External APIs | 518 | - Communication Brokering | 519 | - In-Network Computation | 520 | - Edge Caching | 521 | - Data Analysis | 522 | - Other Services | 523 | - ... | 524 | | 525 +------+--------------+---------------+--------+ 526 | | | 527 | | | 528 +----+---+ +-----+--+ +-----+--+ 529 | End | | End | .... | End | 530 |Device 1| |Device 2| .... |Device n| 531 +--------+ +--------+ +--------+ 533 Figure 1: Model of IoT Edge Computing 535 In this general model, the edge computing domain is interconnected 536 with IoT end devices (southbound connectivity) and possibly with a 537 remote/cloud network (northbound connectivity), and with a service 538 operator's system. Edge computing nodes provide multiple logical 539 functions, or components, which may not all be present in a given 540 system. They may be implemented in a centralized or distributed 541 fashion, in the edge network, or through some interworking between 542 the edge network and a remote cloud network. 544 We now attempt to enumerate major edge computing domain components. 545 They are here loosely organized into OAM, functional and application 546 components, with the understanding that the distinction between these 547 classes may not always be clear, depending on actual system 548 architectures. 550 5.1. OAM Components 552 Edge computing OAM goes beyond the network-related OAM functions 553 listed in [RFC6291]. Besides infrastructure (network, storage and 554 computing resources), edge computing systems can also include 555 computing environments (for VMs, software containers, functions), IoT 556 end devices, data and code. 558 Operation related functions include performance monitoring for 559 service level agreement measurement; fault management and 560 provisioning for links, nodes, compute and storage resources, 561 platforms and services. Administration covers network/compute/ 562 storage resources, platforms and services discovery, configuration 563 and planning. Management covers monitoring and diagnostics of 564 failures, as well as means to minimize their occurrence and take 565 corrective actions. This may include software updates management, 566 high service availability through redundancy and multipath 567 communication. 569 We further detail a few OAM components. 571 5.1.1. Resources Discovery 573 This component is about finding infrastructure resources, such as 574 compute, network and storage, but also other resources such as IoT 575 end devices, sensors, data, code, or services. 577 5.1.2. Virtualization Management 579 Some IoT edge computing systems make use of virtualized (compute, 580 storage and networking) resources, which need to be allocated and 581 configured. 583 5.1.3. Authentication and Authorization 585 This can cover authenticating platforms, end devices, data, code 586 units and applications or users interacting with the system. Today, 587 centralized gateway-based systems rely for device authentication on 588 the installation of a secret on IoT end devices and on computing 589 devices (e.g., a device certificate stored in a hardware security 590 module). 592 5.1.4. Edge Organization and Federation 594 In a distributed system context, once edge devices have discovered 595 and authenticated each other, they can be organized, or self- 596 organize, into hierarchies or clusters. Such groups can also form 597 federations with other edge or remote clouds. 599 5.2. Functional Components 601 5.2.1. External APIs 603 An IoT edge cloud may provide a northbound data plane or management 604 plane interface to a remote network, e.g., a cloud, home or 605 enterprise network. This interface does not exist in standalone 606 (local-only) scenarios. To support such an interface when it exists, 607 an edge computing component needs to expose an API, deal with 608 authentication and authorization, support secure communication. 610 An IoT edge cloud may provide an API or interface to local users 611 (e.g., to facilitate local management), or to mobile users (e.g., to 612 provide access to services and applications, or to manage data 613 published by the mobile device). 615 5.2.2. Communication Brokering 617 A typical function of IoT edge computing is to facilitate 618 communication with IoT end devices: for example, enable clients to 619 register as recipients for data from devices, as well as forwarding/ 620 routing of traffic to or from IoT end devices, enabling various data 621 discovery and redistribution patterns, e.g., north-south with clouds, 622 east-west with other edge devices [DATA-DISCOVERY]. Another aspect 623 of a communication component is dispatching of alerts and 624 notifications to interested consumers both inside and outside of the 625 edge computing domain. Protocol translation, analytics and 626 transcoding may also be performed when necessary. 628 Communication brokering may be centralized in some systems, e.g., 629 using a hub-and-spoke message broker, or distributed like with 630 message buses, possibly in a layered bus approach. Distributed 631 systems may leverage direct communication between end devices and 632 communication devices, such as device-to-device links. Brokers 633 functions can include ensuring communication reliability, 634 traceability, and in some cases transaction management. 636 QoS can be provided in some systems through the combination of 637 network QoS (e.g., traffic engineering or wireless resource 638 scheduling) and compute/storage resource allocations. In some 639 systems a bandwidth manager service can be exposed to enable 640 allocation of bandwidth to/from an edge computing application 641 instance. 643 5.2.3. In-Network Computation 645 A core function of IoT edge computing is to enable computation 646 offloading, i.e., to perform computation on an edge node on behalf of 647 a device or user. The support for in-network computation may vary in 648 term of capability, e.g., computing nodes can host a virtual machine 649 able run stateful or stateless code, or a rule engine providing an 650 API to register actions in response to conditions such as IoT device 651 ID, sensor values to check, thresholds, etc. Computation offloading 652 includes orchestration or application lifecycle related aspects, such 653 as: selecting an appropriate compute device based on available 654 resources, compute node properties, etc., and with varying goals 655 including for example load balancing and energy conservation; 656 onboarding code on a platform or compute device; assisted or 657 automatic partitioning of code; invoking remote code execution; 658 relocating an instance from one compute node to another. 660 5.2.4. Edge Caching 662 A purpose of local caching may be to enable local data processing 663 (e.g., pre-processing or analysis), or to enable delayed virtual or 664 physical shipping. A responsibility of the edge caching component is 665 to manage data persistence, e.g., to schedule removal of data when it 666 is no longer needed. Another aspect of this component may be to 667 authenticate and encrypt data. It can for example take the form of a 668 distributed storage system, and deal with related issues, e.g., 669 reaching and maintaining data consistency; enabling efficient access 670 to data, for example using some form of sharding. 672 5.2.5. Other Services 674 Data generated by IoT devices and associated information obtained 675 from the access network may be used to provide high level services 676 such as end device location, radio network information and bandwidth 677 management. 679 5.3. Application Components 681 5.3.1. IoT End Devices Management 683 IoT end device management includes managing information about the IoT 684 devices, including their sensors, how to communicate with them, etc. 685 Edge computing addresses the scalability challenges from the massive 686 number of IoT end devices and IoT data value by separating the 687 scalability domain into edge/local networks and remote network. 689 5.3.2. Data Management 691 With regard to the high level challenges listed in Section 4, data 692 storage and processing at the edge is a major aspect of IoT edge 693 computing. Data may therefore need to be classified (e.g., in terms 694 of privacy, importance, validity, etc.). Data analysis such as 695 performed in AI/ML tasks performed at the edge may benefit from 696 specialized hardware support on computing nodes. IoT edge computing 697 will face challenges in term of, for example, programmability, 698 naming, data abstraction, data service management and data discovery 699 (discussed in communication brokering). Furthermore, while edge 700 computing can support IoT services independently of cloud computing, 701 it can also be connected to cloud computing. Thus, the relationship 702 of IoT edge computing to cloud computing, with regard to data 703 management, is another potential challenge [ISO_TR]. 705 5.4. Simulation and Emulation Environments 707 IoT Edge Computing brings new challenges to simulation and emulation 708 tools used by researchers and developers. A varied set of 709 applications, network and computing technologies can coexist in a 710 distributed system, which make modelling difficult. Scale, mobility 711 and resource management are additional challenges [SimulatingFog]. 713 Tools include simulators, where simplified application logic runs on 714 top of a fog network model, and emulators, where actual applications 715 can be deployed, typically in software containers, over a cloud 716 infrastructure (e.g. Docker, Kubernetes) itself running over a 717 network emulating edge network conditions such as variable delays, 718 throughput and mobility events. To gain in scale, emulated and 719 simulated systems can be used together in hybrid federation-based 720 approaches [PseudoDynamicTesting], while to gain in realism physical 721 devices can be interconnected with emulated systems. Examples of 722 related work and platforms include the publicly accessible MEC 723 sandbox work recently initiated in ETSI [ETSI_Sandbox], and open 724 source simulators and emulators ([AdvantEDGE] emulator and tools 725 cited in [SimulatingFog]). 727 6. Security Considerations 729 T.B.D. 731 7. Acknowledgement 733 The authors would like to thank Joo-Sang Youn, Akbar Rahman, Michel 734 Roy and Robert Gazda for their valuable comments and suggestions on 735 this document. 737 8. References 739 8.1. Normative References 741 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 742 Requirement Levels", BCP 14, RFC 2119, 743 DOI 10.17487/RFC2119, March 1997, 744 . 746 8.2. Informative References 748 [AdvantEDGE] 749 "Mobile Edge Emulation Platform", Source Code Repository , 750 January 2020, 751 . 753 [Ashton] Ashton, K., "That Internet of Things thing", RFID J. vol. 754 22, no. 7, pp. 97-114 , 2009. 756 [Botta] Botta, A., Donato, W., Persico, V., and A. Pescape, 757 "Integration of Cloud Computing and Internet of Things: A 758 survey", Future Gener. Comput. Syst., vol. 56, pp. 759 684-700 , 2016. 761 [Chiang] Chiang, M. and T. Zhang, "Fog and IoT: An overview of 762 research opportunities", IEEE Internet Things J., vol. 3, 763 no. 6, pp. 854-864 , 2016. 765 [DATA-DISCOVERY] 766 McBride, M., Kutscher, D., Schooler, E., and CJ. 767 Bernardos, "Edge Data Discovery for COIN", Work in 768 Progress, Internet-Draft, draft-mcbride-edge-data- 769 discovery-overview, January 2020, 770 . 773 [ENERGY] Beckel, C., Sadamori, L., Staake, T., and S. Santini, 774 "Revealing Household Characteristics from Smart Meter 775 Data", Energy, vol. 78, pp. 397-410 , 2014. 777 [ETSI_MEC_01] 778 ETSI, ., "Multi-access Edge Computing (MEC); Terminology", 779 ETSI GS 001 , 2019, . 782 [ETSI_MEC_02] 783 ETSI, ., "Multi-access Edge Computing (MEC); Phase 2: Use 784 Cases and Requirements", ETSI GS 002 , 2016, 785 . 788 [ETSI_MEC_03] 789 ETSI, ., "Mobile Edge Computing (MEC); Framework and 790 Reference Architecture", ETSI GS 003 , 2019, 791 . 794 [ETSI_MEC_WP_28] 795 ETSI, ., "MEC in 5G networks", White Paper , 2018, 796 . 799 [ETSI_Sandbox] 800 "Multi-access Edge Computing (MEC) MEC Sandbox Work Item", 801 Portal , January 2020, 802 . 805 [Evans] Evans, D., "The Internet of Things: How the next evolution 806 of the Internet is changing everything", CISCO White 807 Paper , 2011. 809 [FLAME] Horizon 2020 Programme, ., "FLAME Project", Portal , 2019, 810 . 812 [IEEE-1934] 813 IEEE, ., "FOG - Fog Computing and Networking Architecture 814 Framework", Portal , 2019, 815 . 817 [ISO_TR] "Information Technology - Cloud Computing - Edge Computing 818 Landscape", ISO/IEC TR 23188 , 2018. 820 [Kelly] Kelly, R., "Internet of Things Data to Top 1.6 Zettabytes 821 by 2022", 2016, 822 . 826 [Lin] Lin, J., Yu, W., Zhang, N., Yang, X., Zhang, H., and W. 827 Zhao, "A survey on Internet of Things: Architecture, 828 enabling technologies, security and privacy, and 829 applications", IEEE Internet of Things J., vol. 4, no. 5, 830 pp. 1125-1142 , 2017. 832 [Linux_Foundation_Edge] 833 Linux Foundation, ., "Linux Foundation Edge", Portal , 834 2019, . 836 [Mahadev] Satyanarayanan, M., "The Emergence of Edge Computing", 837 Computer, vol. 50, no. 1, pp. 30-39 , 2017. 839 [NIST] Mell, P. and T. Grance, "The NIST definition of Cloud 840 Computing", Natl. Inst. Stand. Technol, vol. 53, no. 6, p. 841 50 , 2009. 843 [NVIDIA] Grzywaczewski, A., "Training AI for Self-Driving Vehicles: 844 the Challenge of Scale", NVIDIA Developer Blog , 2017, 845 . 848 [OpenEdgeComputing] 849 "Open Edge Computing", Portal , 2019, 850 . 852 [OpenFog] "OpenFog Reference Architecture for Fog Computing", 853 OpenFog Consortium , 2017. 855 [POINT] Horizon 2020 Programme, ., "IP Over ICN - the better IP 856 (POINT) Project", Portal , 2019, 857 . 859 [PseudoDynamicTesting] 860 Ficco, M., Esposito, C., Xiang, Y., and F. Palmieri, 861 "Revealing Household Characteristics from Smart Meter 862 Data", IEEE Communications Magazine, Nov. 2017 , 2017. 864 [RFC6291] Andersson, L., van Helvoort, H., Bonica, R., Romascanu, 865 D., and S. Mansfield, "Guidelines for the Use of the "OAM" 866 Acronym in the IETF", BCP 161, RFC 6291, 867 DOI 10.17487/RFC6291, June 2011, 868 . 870 [Shi] Shi, W., Cao, J., Zhang, Q., Li, Y., and L. Xu, "Edge 871 computing: vision and challenges", IEEE Internet of Things 872 J., vol. 3, no. 5, pp. 637-646 , 2016. 874 [Sifalakis] 875 Sifalakis, M., Kohler, B., Scherb, C., and C. Tschudin, 876 "An Information Centric Network for Computing the 877 Distribution of Computations", Proceedings of the 1st 878 International Conference on Information-centric networking 879 (INC) , 2014. 881 [SimulatingFog] 882 Svorobej, S. and . al, "Simulating Fog and Edge Computing 883 Scenarios: An Overview and Research Challenges", MPDI 884 Future Internet 2019 , 2019. 886 [StarlingX] 887 OpenStack Foundation, ., "StarlingX", Portal , 2019, 888 . 890 [Weiner] Weiner, M., Jorgovanovic, M., Sahai, A., and B. Nikolie, 891 "Design of a low-latency, high-reliability wireless 892 communication system for control applications", IEEE Int. 893 Conf. Commun. (ICC), Sydney, NSW, Australia, pp. 894 3829-3835 , 2014. 896 [_3GPP.23.501] 897 3GPP, ., "System Architecture for the 5G System", 3GPP TS 898 23.501 , 2019, 899 . 901 [_5G-CORAL] 902 Horizon 2020 Programme, ., "5G Convergent Virtualised 903 Radio Access Network Living at the Edge (5G-CORAL) 904 Project", Portal , 2019, . 906 [_60802] IEC/IEEE, ., "Use Cases IEC/IEEE 60802 V1.3", IEC/IEEE 907 60802 , 2018, . 910 Appendix A. Overview of the IoT Edge Computing 912 This list of initiatives, projects and products aim to provide an 913 overview of the IoT edge computing. 915 Our goal is to be representative rather than exhaustive. 917 Please help us complete this overview by communicating with us about 918 entries we have missed. 920 A.1. Open Source Projects 922 A.1.1. Gateway/CPE Platforms 924 EdgeX Foundry, Home Edge, Edge Virtualization Engine are Linux 925 Foundation projects ([Linux_Foundation_Edge]) aiming to provide a 926 platform for edge computing devices. 928 Such an open source platform can, for example, host proprietary 929 programs currently run on IoT gateway products (Appendix A.2). 931 EdgeX Foundry develops an edge computing framework running on the IoT 932 gateway. 934 Home Edge develops an edge computing framework especially dedicated 935 to home computing devices, controlling home appliances, sensors, 936 etc., and enabling AI applications, especially distributed and 937 parallel machine learning. 939 The Edge Virtualization Engine (EVE) project develops a 940 virtualization platform (for VMs and containers) designed to run 941 outside of the datacenter, in an edge network; EVE is deployed on 942 bare-metal hardware. 944 Computing devices: Hardware support for EdgeX and EVE is similar: 945 they support x86 and ARM-based computing devices; A typical target 946 can be a Linux Raspberry Pi with 1GB RAM, 64bit CPU, 32GB storage. 948 Service platform: EdgeX uses a micro-service architecture. Micro- 949 services on the gateway are connected together, and to outside 950 applications, through REST, or messaging technologies such as 951 MQTT, AMQP and 0MQ. The gateway can communicate with external 952 backend applications or other gateways (north-south in tiered 953 deployments or east-west in more distributed deployments). 954 Gateway-device communication can use a wide range of IoT 955 protocols. "Export services" enable on-gateway and off-gateway 956 clients to register as recipient for data from devices. Core 957 services are microservices that deal with persisting data from 958 devices or alternatively "streaming" device data through, without 959 persistence (core data service); managing information about the 960 IoT devices, including their sensors, how to communicate with 961 them, etc. (metadata service); and actual communication with IoT 962 devices, on behalf of other on-gateway or off-gateway services 963 (command service). A rule engine provides an API to register 964 actions in response to conditions typically including an IoT 965 device ID, sensor values to check, thresholds, etc. The 966 scheduling micro service deals with organizing the removal of data 967 persisted on the gateway. Alerts and notifications microservice 968 can be used to dispatch alert/notifications from internal or 969 external sources to interested consumers including backend 970 servers, or human operators through email or SMS. 972 Edge cloud applications: Target applications for EdgeX include 973 Industrial IoT (e.g., IoT sensor data and actuator control mixed 974 with augmented reality application for technicians). Home Edge 975 focuses on smart home use cases, including using AI lifestyle and 976 safety applications. 978 A.1.2. Edge Cloud Management Platforms 980 This set of open-source projects setup and manage clouds of 981 individual edge computing devices. 983 StarlingX ([StarlingX]) extends OpenStack to provide virtualization 984 platform management for edge clouds, which are distributed (in the 985 range of 100 compute devices), secure and highly available. 987 Akraino Edge Stack, another project from the Linux Fundation Edge 988 [Linux_Foundation_Edge], has a wider scope of developing a management 989 platform adapted for the edge (e.g., covering 1000 plus locations), 990 aiming for zero-touch provisioning, and zero-touch lifecycle 991 management. 993 Computing devices: Compute devices are typically Linux-based 994 application servers or more constrained devices. 996 Service platform: StarlingX adds new management services to 997 OpenStack by leveraging building blocks such as Ceph for 998 distributed storage, Kubernetes for orchestration. The new 999 services are for management of configuration (enabling auto- 1000 discovery and configuration), faults, hosts (enabling host failure 1001 detection and auto-recovery), services (providing high 1002 availability through service redundancy and multi-path 1003 communication) and software (enabling updates). 1005 Edge cloud applications: An edge computing platform may support a 1006 wide range of use cases. E.g., autonomous vehicles, industrial 1007 automation and robotics, cloud RAN, metering and monitoring, 1008 mobile HD video, content delivery, healthcare imaging and 1009 diagnostics, caching and surveillance, augmented/virtual reality, 1010 small cell services for high density locations (stadiums), 1011 universal CPE applications, retail. 1013 A.1.3. Related Projects 1015 Open Edge Computing ([OpenEdgeComputing]) is an initiative from 1016 universities, manufacturers, infrastructure providers and operators, 1017 enabling efficiently offloading cloudlets (VMs) to the edge. 1018 Computing devices are typically powerful, well-connected servers 1019 located in mobile networks (e.g., collocated with base stations or 1020 aggregation sites). The service platform is built on top of 1021 OpenStack++, an extension of OpenStack to support cloudlets. This 1022 project is mentioned here as a related project because of its edge 1023 computing focus, and potential for some IoT use cases. Nevertheless, 1024 its primary use cases are typically non-IoT related, such as 1025 offloading processing-intensive applications from a mobile device to 1026 the edge. 1028 A.2. Products 1030 A.2.1. IoT Gateways 1032 Multiple products are marketed as IoT gateways (Amazon Greengrass, 1033 Microsoft Azure IoT Edge, Google Cloud IoT Core, and gateway 1034 solutions from Bosh and Siemens). They are typically composed of a 1035 software frameworks that can run on a wide range of IoT gateway 1036 hardware devices to provide local support for cloud services, as well 1037 as some other local IoT gateway features such as relaying 1038 communication and caching content. Remote cloud is both used for 1039 management of the IoT gateways, and for hosting customer application 1040 components. Some IoT gateway products (Amazon Snowball) have a 1041 primary purpose of storing edge data on premises, to enable 1042 physically moving this data into the cloud without incurring digital 1043 data transfer cost. 1045 Computing devices: Typical computing devices run Linux, Windows or a 1046 Real-Time OS over an ARM or x86 architecture. The level of 1047 service support on the computing device can range from low-level 1048 packages giving maximum control to embedded developers, to high- 1049 level SDKs. Typical requirements can start at 1GHz and 128MB RAM, 1050 e.g., ranging from Raspberry Pi to a server-level appliance. 1052 Service platform: IoT gateways can provide a range of service 1053 including: running stateless functions; routing messages between 1054 connected IoT devices (using a wide range of IoT protocols); 1055 caching data; enabling some form of synchronization between IoT 1056 devices; authenticating and encrypting device data. Association 1057 between IoT devices and gateway based can require a device 1058 certificate. 1060 Edge cloud applications: Pre-processing of IoT data for later 1061 processing in the cloud is a major driver. Use cases include 1062 industrial automation, farming, etc. 1064 A.2.2. Edge Cloud Platforms 1066 Services such as MobileEdgeX provide a platform for application 1067 developers to deploy software (e.g., as software containers) on edge 1068 networks. 1070 Computing devices: Bare metal and virtual servers provided by mobile 1071 network operators are used as computing devices. 1073 Service platform: The service platform provides end device location 1074 service, using GPS data obtained from platform software deployed 1075 in end devices, correlated with location information obtained from 1076 the mobile network. The service platform manages the deployment 1077 of application instances (containers) on servers close to end 1078 devices, using a declarative specification of optimal location 1079 from the application provider. 1081 Edge cloud applications: Use cases include autonomous mobility, 1082 asset management, AI-based systems (e.g., quality inspection, 1083 assistance systems, safety and security cameras) and privacy- 1084 preserving video processing. There are also non-IoT use cases 1085 such as augmented reality and gaming. 1087 A.3. Standards Initiatives 1089 A.3.1. ETSI Multi-access Edge Computing 1091 The ETSI MEC industry standardization group develops specifications 1092 that enable efficient and seamless integration of applications from 1093 vendors, service providers, and 3rd parties across multi-vendor MEC 1094 platforms ([ETSI_MEC_03]). 1096 Basic principles followed include: leveraging NFV infrastructure; 1097 being compliant with 3GPP systems; focusing on orchestration, MEC 1098 services, applications and platforms. 1100 Phase 1 (2015-2016) focused on basic platform services. Phase 2 1101 (2017-2019) focuses on: supporting non-3GPP radio access 1102 technologies, especially WiFi; supporting a distributed, multi- 1103 operator and multi-vendor architecture; supporting non-VM based 1104 virtualization such as containers and PaaS. 1106 Computing devices: Computing devices are typically application 1107 servers, attached to an eNodeB or at a higher level of aggregation 1108 point, and provide service to end users. 1110 Service platform: The mobile edge platform offers an environment 1111 where the mobile edge applications can discover, advertise, 1112 consume and offer mobile edge services. The platform can provide 1113 certain native services such as radio network information, 1114 location, bandwidth management etc. The platform manager is 1115 responsible for managing the life cycle of applications including 1116 informing the mobile edge orchestrator of relevant application 1117 related events, managing the application rules and requirements 1118 including service authorizations, traffic rules, DNS 1119 configuration. 1121 Edge cloud applications: Some of the use cases for MEC 1122 ([ETSI_MEC_02]) are IoT-related, including: security and safety 1123 (face recognition and monitoring), sensor data monitoring, active 1124 device location (e.g., crowd management), low latency vehicle-to- 1125 infrastructure and vehicle-to-vehicle (V2X, e.g., hazard 1126 warnings), video production and delivery, camera as a service. 1128 A.3.2. Edge Computing Support in 3GPP 1130 The 3GPP standards organization included edge computing support in 5G 1131 [_3GPP.23.501]. Integration of MEC and 5G systems has been studied 1132 in ETSI as well [ETSI_MEC_WP_28]. 1134 Computing devices: From 3GPP standpoint, a mobile device may access 1135 any computing device located in a local data network, i.e., 1136 traffic is steered towards the local data network where the 1137 computing device is located. 1139 Service platform: An external party may influence steering, QoS and 1140 charging of traffic towards the computing device. Session and 1141 service continuity can ensure that edge service is maintained when 1142 a client device moves. The network supports multiple-anchor 1143 connections, which makes it possible to connect a client device to 1144 both a local and a remote data network. The client device can be 1145 made aware of the availability of a local area data network, based 1146 on its location. 1148 Edge cloud applications: Edge cloud applications in 3GPP can help 1149 support the major use cases envisioned for 5G, including massive 1150 IoT and V2X. 1152 A.3.3. OpenFog and Industrial Internet Consortium 1154 The OpenFog Consortium (now merged with the Industrial Internet 1155 Consortium) aims to standardize industrial IoT, fog, and edge 1156 computing. It produced a reference architecture for the fog 1157 ([OpenFog]), which has been published as IEEE standard P1934 in 2018. 1158 This work continues within the Industrial Internet Consortium. 1160 Computing devices: Fog nodes include computational, networking, 1161 storage and acceleration elements. This includes nodes collocated 1162 with sensors and actuators, roadside or mobile nodes involved in 1163 V2X connectivity. Fog nodes should be programmable and may 1164 support multi-tenancy. Fog computing devices must employ a 1165 hardware-based immutable root of trust, i.e., a trusted hardware 1166 component which receives control at power-on. 1168 Service platform: The service platform is structured around 1169 "pillars" including: security end-to-end, scalability by adding 1170 internal components or adding more fog nodes,openness in term of 1171 discovery of/by other nodes and networks, autonomy from 1172 centralized clouds (for discovery, orchestration and management, 1173 security and operation) and hierarchical organization of fog 1174 nodes. 1176 Edge cloud applications: Major use cases include smart cars and 1177 traffic control, visual security and surveillance, smart cities. 1179 A.3.4. Related Standards 1181 The IEEE Fog Computing and Networking Architecture Framework Working 1182 Group [IEEE-1934] published the OpenFog architecture as an IEEE 1183 document, and plan to do further work on taxonomy, architecture 1184 framework, and compliance guidelines. 1186 A.4. Research Projects 1188 A.4.1. Named Function Networking 1190 Named Function Networking ([Sifalakis]) is a research project that 1191 aims to extend ICN concepts (especially named data networking) to 1192 have the network orchestrate computation. Interests are sent for a 1193 combination of function and argument names, instead of using the 1194 content name in NDN. 1196 Computing devices: NFN-capable switches are collocated with 1197 computing devices. 1199 Service platform: NFN enables accessing static data and dynamic 1200 computation results in one data-oriented framework, thus 1201 benefiting from usual ICN features such as data authenticity and 1202 caching, as well as enabling the network to perform various 1203 optimizations, e.g., moving data, code or both closer to 1204 requesters. NFN also enables secure access to individual elements 1205 within Named Data Objects, e.g., for filtering or aggregation. 1207 Edge cloud applications: Use cases include some form of MapReduce 1208 operations and service chaining. NDN, on which NFN is based, has 1209 been studied in the context of IoT, where it can provide local 1210 trust management and rendezvous service. 1212 A.4.2. 5G-CORAL 1214 The 5G-CORAL project ([_5G-CORAL]) aims to enable convergence of 1215 access across multiple radio access technologies using fog computing, 1216 using for this purpose an edge and fog computing system (EFS). 1218 Computing devices: Computing devices used in 5G-CORAL include cloud 1219 and central data center servers, edge data center servers, and 1220 fixed or mobile "fog computing devices", which can be computing 1221 devices located in vehicles or factories, e.g., IoT gateways, 1222 mobile phones, cyber-physical devices, etc. 1224 Service platform: 5G-CORAL architecture is based on an integrated 1225 virtualized edge and fog computing system (EFS), that aims to be 1226 flexible, scalable and interoperable with other domains including 1227 transport (fronthaul, backhaul), core and clouds. An 1228 Orchestration and Control System (OCS) enables automatic discovery 1229 of heterogeneous, multiple-owner resources, and federate them into 1230 a unified hosting environment. OCS monitors resource usage to 1231 guarantee service levels. Finally, OCS also includes 1232 orchestration and life cycle functions, including live migration 1233 and scaling. Applications (user and third-party) both inside and 1234 outside the EFS subscribe to EFS services through APIs, with 1235 emphasis on IoT and cyber-physical functionalities. 1237 Edge cloud applications: EFS-hosted services include analytics 1238 obtained from IoT gateways (e.g., LORA or eNodeB gateways), 1239 context information services from RATs, transport (fronthaul and 1240 backhaul) and core networks. EFS-hosted functions include network 1241 performance acceleration functions, virtualized C-RAN functions 1242 for access nodes and possible end user devices. 1244 A.4.3. FLAME 1246 The FLAME project ([FLAME]) aims to improve performance of 1247 interactive media systems while keeping infrastructure costs low. 1249 It builds over virtualization technologies such as XOS, OpenStack and 1250 ONOS/ODL to offer a programmable media service platform. 1252 FLAME leverages IP-over-ICN technology developed through earlier 1253 projects including POINT ([POINT]). 1255 Computing devices: The FLAME platform provides a service layer on 1256 top of an infrastructure platform, which can include cloud servers 1257 as well as computing devices collocated with WiFi access points. 1259 Service platform: The FLAME platform can be seen as an "edge + 1260 cloud" computing platform with a use case focus on media 1261 dissemination, although the basic platform can be suitable for 1262 micro-services in general. The computing platform is comprised 1263 of: computing devices, an infrastructure platform (XOS, OpenStack, 1264 ONOS/ODL), NFV-MANO components (orchestrator, virtual 1265 infrastructure manager) and FLAME platform core services (PCE, 1266 network access point, surrogate manager). 1268 Edge cloud applications: IoT use cases include public safety, such 1269 as supporting body-worn camera for police and social workers. As 1270 opposed to other multi-media applications that are also envisioned 1271 (pre-processing, user reporting, curation...), where a typical 1272 goal is to curate content early at the edge, to reduce expected 1273 high data volume, public safety use cases are typically about 1274 implementing triggers at the edge: everything needs to be kept 1275 anyway, to be available in case of an audit. Content is stored 1276 offline during off peak-hours delivery. For privacy and data 1277 volume concerns, triggers for, e.g., alerting police, cannot be 1278 performed in the cloud and should be performed as close to the 1279 data source as possible. 1281 Authors' Addresses 1283 Jungha Hong 1284 ETRI 1285 218 Gajeong-ro, Yuseung-Gu 1286 Daejeon 1288 Email: jhong@etri.re.kr 1290 Yong-Geun Hong 1291 ETRI 1292 218 Gajeong-ro, Yuseung-Gu 1293 Daejeon 1295 Email: yghong@etri.re.kr 1297 Xavier de Foy 1298 InterDigital Communications, LLC 1299 1000 Sherbrooke West 1300 Montreal H3A 3G4 1301 Canada 1303 Email: xavier.defoy@interdigital.com 1304 Matthias Kovatsch 1305 Huawei Technologies Duesseldorf GmbH 1306 Riesstr. 25 C // 3.OG 1307 80992 Munich 1308 Germany 1310 Email: ietf@kovatsch.net 1312 Eve Schooler 1313 Intel 1314 2200 Mission College Blvd. 1315 Santa Clara, CA, 95054-1537 1316 United States of America 1318 Email: eve.m.schooler@intel.com 1320 Dirk Kutscher 1321 University of Applied Sciences Emden/Leer 1322 Constantiaplatz 4 1323 26723 Emden 1324 Germany 1326 Email: ietf@dkutscher.net