idnits 2.17.1 draft-choi-icnrg-aiot-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 1) being 60 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** There are 325 instances of too long lines in the document, the longest one being 5 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 627 has weird spacing: '... of unsup...' -- The document date (March 11, 2019) is 1872 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'Kriegal' is mentioned on line 537, but not defined == Unused Reference: 'Kriegel' is defined on line 750, but no explicit reference was found in the text Summary: 2 errors (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 ICN Research Group J.K.Choi 3 Internet-Draft N.K.Kim 4 Intended status: Informational J.S.Han 5 Expires: September 12, 2019 M.K.Kim 6 KAIST 7 G.M.Lee 8 Liverpool John Moores University 9 March 11, 2019 11 Requirements and Challenges for User-level Service Managements of IoT 12 Network by utilizing Artificial Intelligence 14 draft-choi-icnrg-aiot-00 16 Abstract 18 This document describes the requirements and challenges to employ 19 artificial intelligence (AI) into the constraint Internet of Things 20 (IoT) service environment for embedding intelligence and increasing 21 efficiency. 23 The IoT service environment includes heterogeneous and multiple IoT 24 devices and systems that work together in a cooperative and 25 intelligent way to manage homes, buildings, and complex autonomous 26 systems. Therefore, it is becoming very essential to integrate IoT 27 and AI technologies to increase the synergy between them. However, 28 there are several limitations to achieve AI enabled IoT as the 29 availability of IoT devices is not always high, and IoT networks 30 cannot guarantee a certain level of performance in real-time 31 applications due to resource constraints. 33 This document intends to present a right direction to empower AI in 34 IoT for learning and analyzing the usage behaviors of IoT 35 devices/systems and human behaviors based on previous records and 36 experiences. With AI enabled IoT, the IoT service environment can be 37 intelligently managed in order to compensate for the unexpected 38 performance degradation often caused by abnormal situations. 40 Status of This Memo 42 This Internet-Draft is submitted in full conformance with the 43 provisions of BCP 78 and BCP 79. 45 Internet-Drafts are working documents of the Internet Engineering 46 Task Force (IETF). Note that other groups may also distribute 47 working documents as Internet-Drafts. The list of current Internet- 48 Drafts is at http://datatracker.ietf.org/drafts/current/. 50 Internet-Drafts are draft documents valid for a maximum of six months 51 and may be updated, replaced, or obsoleted by other documents at any 52 time. It is inappropriate to use Internet-Drafts as reference 53 material or to cite them other than as "work in progress." 54 This Internet-Draft will expire on September 12, 2019 56 Copyright Notice 58 Copyright (c) 2019 IETF Trust and the persons identified as the 59 document authors. All rights reserved. 61 This document is subject to BCP 78 and the IETF Trust's Legal 62 Provisions Relating to IETF Documents 63 (http://trustee.ietf.org/license-info) in effect on the date of 64 publication of this document. Please review these documents 65 carefully, as they describe your rights and restrictions with respect 66 to this document. Code Components extracted from this document must 67 include Simplified BSD License text as described in Section 4.e of 68 the Trust Legal Provisions and are provided without warranty as 69 described in the Simplified BSD License. 71 Table of Contents 73 1. Introduction ................................................ 3 74 2. Challenging Issues of IoT network 75 ............................ 6 76 2.1. Untrusted and incorrect IoT devices ..................... 6 77 2.2. Traffic burstiness of IoT network ....................... 6 78 2.3. Management overheads of heterogeneous IoT sensors 79 ........ 7 80 3. Overview of AI/ML-based IoT services ......................... 9 81 4. Requirements for AI/ML-based IoT services ................... 11 82 4.1. Requirements for AI/ML-based IoT data collection and delivery 84 ........................................................... 11 85 4.2. Requirements for intelligent and context-aware IoT services12 86 5. State of arts of the artificial intelligence/machine learning 87 technologies for IoT services 88 .................................. 12 89 5.1. Machine learning and artificial intelligence technologies 90 review ..................................................... 12 91 5.1.1. Supervised learning for IoT ....................... 12 92 5.1.2. Unsupervised learning for IoT ..................... 13 93 5.1.3. Reinforcement learning for IoT .................... 14 94 5.1.4. Neural Network based algorithms for IoT ........... 15 95 5.2. Technologies for lightweight and real-time intelligence 96 . 17 97 6. IANA Considerations ........................................ 18 98 7. Acknowledgements ........................................... 18 99 8. Contributors ............................................... 18 100 9. Informative References 101 ...................................... 19 103 1. Introduction 105 The document explains the effects of applying artificial 106 intelligence/machine learning (AI/ML) algorithms in the Internet of 107 Thing (IoT) service environments. 109 IoT applications will be deployed in heterogeneous and different 110 areas such as the energy, transportation, automation and 111 manufacturing industries as well as the information and communication 112 technology (ICT) industry. Many IoT sensors and devices can connect 113 to an IoT service environment where IoT objects cannot interoperate 114 with each other and can interact with different applications. The IoT 115 service may not run in a single administrative domain. If market 116 demand exists, the cross-domain service scenarios for IoT 117 applications could be widely deployed. Future IoT applications occur 118 at multiple domains of heterogeneity with various time scales. 120 The IoT service requirements for common architectures and public APIs 121 poses some challenges to the underlying service environment and 122 networking technologies. Some IoT applications require significant 123 security and privacy as well as significant resource and time 124 constraints. These mission-critical applications can be separated 125 from many common IoT applications that current technology may not 126 provide. It means that IoT service requirements are difficult to 127 classify common requirements and functional requirements depending on 128 IoT service scenario. 130 Recently, artificial intelligence technologies can help the context- 131 aware IoT service scenarios apply rule-based knowledge accumulation. 132 The IoT service assumes that many sensing devices are connected to 133 single or multiple IoT network domains. Each sensor sends small 134 packets to the IoT servers periodically or non-periodically. 136 Detection data contains periodic status information that monitors 137 whether the system is in a normal state or not. In some cases, alert 138 information is included for quick processing. Most IoT applications 139 can operate in two modes. One is a simple monitoring mode and the 140 other is an abnormal mode for rapid processing. In a simple 141 monitoring phase, the IoT device periodically sends sensing data to 142 the server. If the measured data is outside the normal range, the IoT 143 service can change the operating mode to an abnormal phase and 144 activate future probes. Alarm conditions should be promptly notified 145 to responsible persons. For mission-critical applications, reliable 146 communication with robust QoS requirements in terms of error and 147 latency is required. 149 Periodic data accumulation from IoT devices is cumbersome. Under 150 normal conditions, the IoT data is simply accumulated without further 151 action. In an unusual situation, incoming IoT data can cause an 152 urgent action to notify the administrator of the problem. Streaming 153 data traffic from thousands of IoT devices is annoying to store in 154 the database because it is not easy to extract unidentified or future 155 incidents. Only a significant portion of the incoming data stream can 156 be stored in a real-time database that is time-sensitive and capable 157 of rapid query processing. A combination of different IoT detection 158 data, including location, time, and status, allows you to sort and 159 categorize a portion of streaming data when an additional inspection 160 is required, and perform real-time processing. One of the missions of 161 the IoT database is to be able to extract preliminary symptoms of 162 unexpected accidents from a large amount of streaming data. 164 If some transmitted data is important to invoke the corresponding 165 action, there are some questions about whether the incoming data is 166 correct. If the incoming data contains accurate and time-critical 167 events, appropriate real-time control and management can be performed. 168 However, if the incoming data is inaccurate or intentionally 169 corrupted, additional accidents may occur. In these cases, incoming 170 data can trigger to initiate additional inspections to protect 171 against future unacceptable situations. But, if time-critical data is 172 missed due to errors in the sensing devices and the delivery protocol, 173 there is no reason to configure IoT networks and devices at a high 174 cost. 176 It is not easy to analyze data collected through IoT devices 177 installed to monitor complex IoT service environments. If the sensor 178 malfunctions, the data of the sensor cannot be trusted. Additional 179 investigation should be done if abnormal status from specific sensors 180 is collected. The data of the redundant sensor installed in the same 181 area should be received or combined with other sensor information 182 adjacent to the sensor to determine the abnormal state. 184 For sensors installed in a specific area, sensing records will remain 185 for a certain period of time. IoT service operators can look at the 186 operational history of the sensor for a period of time to determine 187 what problems were encountered when data was collected. When an 188 abnormal situation occurs, IoT sensor should investigate whether it 189 noticed normal operations and notified the IoT service operator. If 190 the abnormal situation is not properly detected, the operator should 191 analyze whether it was caused by malfunction of the IoT sensor or 192 other reasons. 194 In the IoT service environment, it is possible to analyze the 195 situation accurately by applying recent artificial intelligence and 196 machine learning technologies. If there is an operational record of 197 the past, it is possible to determine when an abnormal situation 198 arises. Most problems are likely to be repeated, so if the past 199 learning experience is accumulated, the anomaly of IoT services can 200 be easily and immediately identified. In addition, when information 201 gathered from various sensors is synthesized, it is possible to 202 accurately determine whether abnormal situations have occurred. 204 Various types of IoT sensors are installed with certain purposes. It 205 expects that all the IoT sensors intend to monitor the occurrence of 206 special abnormal situations in advance. Therefore, it should be set 207 in advance what actions are required when a specific anomaly occurs. 208 The appropriate work is performed on the abnormal situation according 209 to the procedure, predefined by the human. By using artificial 210 intelligence and machine learning algorithms, the appropriate actions 211 are taken when an abnormal situation is detected from various IoT 212 sensors. 214 2. Challenging Issues of IoT network 216 This section describes the challenging issues of data sensing, 217 collection, transfer, and intelligent decision from untrusted data 218 quality and unexpected situations of IoT service environments. 220 2.1. Untrusted and incorrect IoT devices 222 IoT traffic is similar to traditional Internet traffic with small 223 packet sizes. Mobile IoT traffic can cause some errors and delays 224 because wireless links are unstable and signal strength may be 225 degraded with device mobility. If the signal strength of the IoT 226 device with a power limit is not so strong, the reception quality of 227 the IoT server may not be sufficient to obtain the measurement data. 229 For mission-critical applications, such as smart-grid and factory- 230 automation, expensive IoT sensors with self-rechargeable batteries 231 and redundant hardware logic may be required. However, unexpected 232 abnormal situations may occur due to sensor malfunctions. There are 233 trade-offs between implementation cost and efficiency for cost- 234 effective IoT services. When smart-grid and factory-automation 235 applications are equipped with IoT devices, the acceptable quality 236 from IoT solutions can be required. Sometimes, expensive and 237 duplicated IoT solutions may be needed. 239 2.2. Traffic burstiness of IoT network 241 IoT traffic includes two types of traffic characteristic: periodic 242 with small packet sizes and bursty with high bandwidth. Under normal 243 conditions, the IoT traffic periodically transmits status information 244 with a small bandwidth, several kilobits/sec. However, in an abnormal 245 state, IoT devices need a high bandwidth, up to several tens of 246 megabits/sec, in order to identify actual events and investigate 247 accurate status information. In addition, traffic volume can 248 explosively increase in response to emergencies. For example, in the 249 case of smart-grid application, the bandwidth of several kilobits/sec 250 is usually used, and when an urgent situation occurs, a broadband 251 channel is required up to several tens of megabits/sec. 253 The other traffic can be integrated at an IoT network to increase 254 bandwidth efficiency. If an emergency situation occurs in the IoT 255 service, IoT traffic volumes suddenly increase, in which case network 256 processing capacity may be not sufficient. If the IoT service is 257 integrated with voice and video applications, the problem can become 258 more complex. As time goes by, traffic congestion and bottlenecks are 259 frequent in some areas. In addition, if an existing service policy 260 changes (for example, prioritizing certain traffic or suddenly 261 changing the route), other unexpected problems may be encountered. 262 Various congestion control and load balancing algorithms with the 263 help of artificial intelligence can be applied to handle time-varying 264 traffic on a network. 266 Until now, much research has been done on traffic variability in an 267 integrated network service environment. All networks have their own 268 traffic characteristics, depending on geographical area, number of 269 subscribers, subscribers' preferences, and types of applications used. 270 In the case of IoT traffic, the normal bandwidth is very small. If 271 the IoT traffic volume increases abruptly in an abnormal situation, 272 the network may suffer unacceptable delay and loss. If emergency 273 situations detected by IoT networks occur in a smart grid or 274 intelligent transportation system, the processing power of the IoT 275 network alone cannot solve the problem and the help of existing 276 network resources is inevitable. 278 2.3. Management overheads of heterogeneous IoT sensors 280 Traffic management in an integrated network environment is not easy. 281 In order to operate the network steadily, a network operator has its 282 own know-hows and experiences. If there are plenty of network 283 resources, it is easy to set up a bypass route even if network 284 failure or congestion occurs in a specific area. For operating 285 network steadily, network resources may be designed to be over- 286 provisioned in order to cope with various possible outages. A network 287 operator predicts the amount of traffic generated by the 288 corresponding equipment and grasps to what extent a transmission 289 bandwidth is required. If traffic fluctuation is very severe, the 290 network operator can allocate network resources in advance. In case 291 of frequent failures or severe traffic fluctuation, some network 292 resources are separated in order not to affect normal traffic. 294 More than a billion IoT devices are expected to connect to 295 smartphones, tablets, wearables, and vehicles. Therefore, IoT 296 services are targeted at mobile applications. In particular, 297 intelligent transportation systems need the help of IoT technology to 298 provide traffic monitoring and prevent public or private traffic 299 accidents. IoT technology can play an important role in reducing 300 traffic congestion, saving people's travel time and costs, and 301 providing a pleasant journey. 303 The IoT service has troublesome administrative problems to configure 304 an IoT network which consists of IoT servers, gateways, and many 305 sensing devices. The small-sized but large-numbered IoT devices may 306 incur administrative overhead since all the IoT devices should be 307 initialized and the bootstrapping information of IoT resources should 308 be loaded into the IoT service environments. Whenever some IoT 309 devices are newly added and some devices have to be removed, the 310 dynamic reconfiguration of IoT resources is essential. In addition, 311 the IoT device's preinstalled software should be regularly inspected 312 and upgraded according to its version. Frequent upgrades and changes 313 to some IoT devices may require autonomic management and 314 bootstrapping techniques. 316 Network management generally assumes that all network resources 317 operate reliably with acceptable quality. In most failure situations, 318 the network operator decides to switch to a redundant backup device 319 or bypass the failed communication path. If some IoT devices are not 320 stable, duplicate IoT devices can be installed for the same purpose. 321 If IoT resources are not duplicated, various mechanisms are needed to 322 reduce the damage. Therefore, it is necessary to prioritize the 323 management tasks to be performed first when an abnormality occurs in 324 the IoT service environment. However, managing duplicate networks can 325 cause another problem. If two IoT devices are running at the same 326 time, the recipient can get redundant information. If two or more 327 unusual situations occur at the same time, it is difficult to solve 328 the problem since tasks for urgent processing should be distinguished 329 from tasks that can be performed over time. 331 In addition, the operations manager's mistakes or misunderstanding of 332 problem situations can lead to other unexpected complications. 333 Therefore, artificial intelligence technologies can help what kind of 334 network management work is required when an unexpected complicated 335 situation occurs even though a procedure for an abnormal situation is 336 already prepared. 338 3. Overview of AI/ML-based IoT services 340 In this section, successful applications of artificial intelligence 341 in IoT domains are provided. The common property of IoT applications 342 and services is that they require fast analytics rather than later 343 analytics with piled data. Recently, neural-network-based artificial 344 intelligence technologies are widely used across many IoT 345 applications. 347 Simple IoT applications include dynamic contexts that share common 348 features among social relations at the same administration domain. 349 IoT devices in the same domain can provide their service contexts to 350 the IoT server. When a dynamic change occurs in an IoT service 351 context, the IoT device needs real-time processing to activate urgent 352 events, alert notifications, update, and reconnect contexts. The IoT 353 service must support real-time interactions between the IoT device 354 and the system in the same domain. The IoT service contexts must be 355 shared between physical objects and social members in the same domain 356 as well. 358 Artificial intelligence technologies have been shown promising in 359 many areas, including IoT. For example, contextual information for a 360 car-sharing business must interact with customers, car owners, and 361 car sharing providers. All entities in the value chain of a car 362 sharing business must share the corresponding situation to pick up, 363 board, and return shared cars. Communication networks and interactive 364 information, including registration and payment, can be shared 365 tightly among the entities. Home IoT service environment can be 366 equipped with sensors for theft detection, door lock, temperature, 367 fire detection, gas detection, short circuit, air condition to name a 368 few. Office IoT service environments, including buildings such as 369 shopping centers and bus/airport terminals, have their own sensors, 370 including alarm sensors. When an alarm signal is detected by the 371 sensor, the physical position and occurrence time of the sensor is 372 determined in advance. All signals from various sensors are analyzed 373 comprehensively to make the right decision. If some sensors 374 frequently malfunction, the situation can be grasped more accurately 375 by analyzing the information of the adjacent sensor. In particular, 376 when installing multiple sensors in a particular building (e.g., 377 surveillance camera, location monitoring, temperature, etc.), a much 378 wider range of sensors can be used when utilizing artificial 379 intelligence and machine learning technologies. 381 (Smart home) Smart home concept span over multiple IoT applications, 382 health, energy, entertainment, education, etc. It involves voice 383 recognition, natural language processing, image-based object 384 recognition, appliance management, and many more artificial 385 intelligence technologies integrated with IoT. Smart connected- 386 devices monitor the house to provide better control over home 387 supplies and expenses. The energy consumption and efficiency of home 388 appliances are monitored and analyzed with deep learning based 389 technologies, such as artificial neural network, long-short-term- 390 memory, etc. 392 (Smart city) Smart city, as well, contains multiple IoT domains, 393 transportation, infrastructure, energy, agriculture, etc. Since 394 heterogeneous data from different domains are gathered in smart 395 cities, various artificial intelligence approaches are studied in 396 smart-city application. Public transportation behaviors and crowd 397 movements patterns are important issues, and they are often dealt 398 with neural network based methods, long-short-term-memory and 399 convolutional neural network. 401 (Smart energy) As two-way communication energy infrastructure is 402 deployed, smart grid has become a big IoT application, which requires 403 intelligent data processing. The traditional energy providers are 404 highly interested in recognizing local energy consumption patterns 405 and forecasting the needs in order to make appropriate decisions on 406 real-time. Moreover, the energy consumers, as well, want analyzed 407 information on their own energy consumption behaviors. Recently, many 408 works on energy consumption prediction, energy flexibility analysis, 409 etc. are actively ongoing. Most works are based on the latest deep 410 learning technologies, such as multi-layered-perceptron, recurrent 411 neural network, long-short-term-memory, autoencoder, etc. 413 (Smart transportation) The intelligent transportation system is 414 another source of big data in IoT domains. Many use cases, such as 415 traffic flow and congestion prediction, traffic sign recognition, 416 vehicle intrusion detection, etc., have been studied. Moreover, a lot 417 of advanced artificial intelligence technologies are required in 418 autonomous and smart vehicles, which require many intelligent sub- 419 tasks, such as pedestrian's detection, obstacle avoidance, etc. 421 (Smart healthcare) IoT and artificial intelligence are integrated 422 into the healthcare and wellbeing domain as well. By analyzing food 423 images with convolutional neural network on mobile devices, dietary 424 intakes can be measured. With voice signal captured from sensor 425 devices, voice pathologies can be detected. Moreover, recurrent 426 neural network and long-short-term-memory technologies are actively 427 being studied for early diagnosis and prediction of diseases with 428 time series medical data. 430 (Smart agriculture) To manage a vast area of land, IoT and artificial 431 intelligence technologies are recently used in agriculture domains. 432 Deep neural network and convolutional neural network are utilized for 433 crop detection or classification and disease recognition in the 434 plants. Moreover, for automatic farming with autonomous machine 435 operation, obstacle avoidance, fruit location, and many more sub- 436 tasks are handled with advanced artificial intelligence technologies. 438 4. Requirements for AI/ML-based IoT services 440 (to be included) 442 4.1. Requirements for AI/ML-based IoT data collection and delivery 444 (to be included) 446 4.2. Requirements for intelligent and context-aware IoT services 448 (to be included) 450 5. State of arts of the artificial intelligence/machine learning 451 technologies for IoT services 453 In this section, well-known machine learning and artificial 454 intelligence technologies applicable to IoT applications are reviewed. 456 5.1. Machine learning and artificial intelligence technologies review 458 The classical machine learning models can be divided into three types, 459 supervised, unsupervised, and reinforcement learnings. Therefore, in 460 this subsection, machine learning and artificial intelligence 461 technology reviews are done in four different categories: supervised, 462 unsupervised, reinforcement, and neural-network-based. 464 5.1.1. Supervised learning for IoT 466 Supervised learning is a task-based type of machine learning, which 467 approximates function describing the relationship and causality 468 between input and output data. Therefore, the input data needs to be 469 clearly defined with proper output data since supervised learning 470 models learn explicitly from direct feedback. 472 (K-Nearest Neighbor) Given a new data point in K-Nearest Neighbor 473 (KNN) classifier, it is classified according to its K number of the 474 closest data points in the training set. To find the K nearest 475 neighbors of the new data point, it needs to use a distance metric 476 which can affect classifier performance, such as Euclidean, 477 Mahalanobis or Hamming. One limitation of KNN in applying for IoT 478 network is that it is unscalable to large datasets because it 479 requires the entire training dataset to classify a newly incoming 480 data. However, KNN required less processing power capability compared 481 to other complex learning methods. 483 (Naive Bayes) Given a new data point in Naive Bayes classifiers, 484 it is classified based on Bayes' theorem with the "naive" assumption 485 of independence between the features. Since Naive Bayes classifiers 486 don't need a large number of data points to be trained, they can deal 487 with high-dimensional data points. Therefore, they are fast and 488 highly scalable. However, since its "naive" assumptions are somewhat 489 strong, a certain level of prior knowledge on the dataset is required. 491 (Support Vector Machine) Support Vector Machine (SVM) is a binary and 492 non-probabilistic classifier which finds the hyperplane maximizing 493 the margin between the classes of the training dataset. SVM has been 494 the most pervasive machine learning technology until the study on 495 neural network technologies are advanced recently. However, SVM still 496 has advantages over neural network based and probabilistic approaches 497 in terms of memory usage and capability to deal with high-dimensional 498 data. In this manner, SVM can be used for IoT applications with 499 severe data storage constraint. 501 (Regression) Regression is a method for approximating the 502 relationships of the dependent variable, which is being estimated, 503 with the independent variables, which are used for the estimation. 504 Therefore, this method is widely used for forecasting and inferring 505 causal relationships between input data and output data in time- 506 sensitive IoT application. 508 (Random Forests) In random forests, instead of training a single 509 decision tree, a group of trees is trained. Each tree is trained on a 510 subset of the training set using a randomly chosen subset of M input 511 variables. Random forests considering various tree structures have 512 very high accuracy, so it can be utilized in the accuracy-critical 513 IoT applications. 515 5.1.2. Unsupervised learning for IoT 517 Unsupervised learning is a data-driven type of machine learning which 518 finds hidden structure in unlabeled dataset without feedback during 519 the learning process. Unlike supervised learning, unsupervised 520 learning focuses on discovering patterns in the data distributions 521 and gaining insights from them. 523 (K-means clustering) K-means clustering aims to assign observations 524 into K number of clusters in which each observation belongs to the 525 cluster having the most similarities. The measure of similarity is 526 the distance between K cluster centers and each observation. K-means 527 is a very fast and highly scalable clustering algorithm, so it can be 528 used for IoT applications with real-time processing requirements such 529 as smart transportation. 531 (Density-based spatial clustering of applications with noise) 532 Density-Based approach to Spatial Clustering of Applications with 533 Noise (DBSCAN) is a method that clusters dataset based on the density 534 of its data samples. In this model, dense regions which include data 535 samples with many close neighbors are considered as clusters, and 536 data samples in low-density regions are classified as outliers 537 [Kriegal]. Since this method is robust to outliers, DBSCAN is 538 efficient data clustering method for IoT network environments with 539 untrusted big datasets in practice. 541 5.1.3. Reinforcement learning for IoT 543 Reinforcement learning is a reactive type of machine learning that 544 learn a series of actions in a given set of possible states, actions, 545 and rewards or penalties. It can be seen as the exploring decision- 546 making process and choosing the action series with the most reward or 547 the least penalty which can be cost, priority, time to name a few. 548 Reinforcement learning can be helpful for selecting action of IoT 549 device by providing a guideline. 551 (Q-learning) Q-Learning is a model-free, off-policy reinforcement 552 learning algorithm based on the well-known Bellman Equation. The goal 553 is to learn an action-selection policy maximizing the Q-value, which 554 tells an agent what action to take. It can be used for IoT device to 555 determine which action it should take according to conditions. 557 (State-Action-Reward-State-Action) Though State-Action-Reward-State- 558 Action (SARSA) is a much similar algorithm to Q-learning, the main 559 difference is that it is an on-policy algorithm in which agent 560 interacts with the environment and updates the policy based on 561 actions taken. It means that the Q-value is updated by an action 562 performed by the current policy instead of the greed policy that 563 maximizes Q-value. In this perspective, it is relevant when an action 564 of one IoT device will greatly influence the condition of the 565 environment. 567 (Deep Q Network) Deep Q network (DQN) is developed to solve the 568 exploration problem for unseen states. In the case of Q-learning, the 569 agent is not capable of estimating value for unseen states. To handle 570 this generality problem, DQN leverages neural network technology. As 571 a variation of the classic Q-Learning algorithm, DQN utilizes a deep 572 convolutional neural net architecture for Q-function approximation. 573 In real environments not all possible states and conditions are not 574 able to be observed. Therefore, DQN is more relevant than Q-learning 575 or SARSA in real applications such as IoT. Since DQN could be used 576 within only discrete action space, it can be utilized for traffic 577 routing in the IoT network. 579 (Deep Deterministic Policy Gradient) DQN has solved generality and 580 exploration problem of the unseen or rare states. Deep Deterministic 581 Policy Gradient (DDPG) takes DQN into the continuous action domain. 582 DDPG is a deterministic policy gradient based actor-critic, model- 583 free algorithm. The actor decides the best action for each state and 584 critic is used to evaluate the policy, the chosen action set. In IoT 585 applications, DDPG can be utilized for the tasks that require 586 controlled in continuous action spaces, such as energy-efficient 587 temperature control, computation offloading, network traffic 588 scheduling, etc. 590 5.1.4. Neural Network based algorithms for IoT 592 (Recurrent Neural Network) Recurrent Neural Network (RNN) is a 593 discriminative type of supervised learning model that takes serial or 594 time-series input data. RNN is specifically developed to address 595 issue of time dependency of sequential time-series input data. It 596 processes sequences of data through internal memory, and it is useful 597 in IoT applications with time-dependent data, such as identifying 598 time-dependent patterns of sensor data, estimating consumption 599 behavior over time, etc. 601 (Long Short Term Memory) As an extension of RNN, Long Short Term 602 Memory (LSTM) is a discriminative type of supervised learning model 603 that is specialized for serial or time-series input data as well 604 [Hochreiter]. The main difference of LSTM from RNN is that it 605 utilizes the concept of gates. It actively controls forget gates to 606 prevent the long term time dependency from waning. Therefore, 607 compared to RNN, it is more suitable for data with long time 608 relationship and IoT applications requiring analysis on the long lag 609 of dependency, such as activity recognition, disaster prediction, to 610 name a few [Chung]. 612 (Convolutional Neural Network) Convolutional neural network (CNN) is 613 a discriminative type of supervised learning model. It is developed 614 specifically for processing 2-dimensional image data by considering 615 local connectivity, but now generally used for multidimensional data 616 such as multi channel sound signals, IoT sensor values, etc. As in 617 CNN neurons are connected only to a small subset of the input and 618 share weight parameters, CNN is much more sparse compared to fully 619 connected network. However, it needs a large training dataset, 620 especially for visual tasks. In CNN, a new activation function for 621 neural network, Rectified Linear Unit (ReLU), was proposed, which 622 accelerates training time without affecting the generalization of the 623 network [Krizhevsky]. In IoT domains, it is often used for detection 624 tasks that require some visual analysis. 626 (Variational Autoencoder) Autoencoder (AE) is a generative type 627 of unsupervised learning model. AE is trained to generate output to 628 reconstruct input data, thus it has the same number of input and 629 output units. It is suitable for feature extraction and 630 dimensionality reduction. Because of its behavior to reconstructing 631 the input data at the output layer, it is often used for machinery 632 fault diagnosis in IoT applications. The most popular type of AE, 633 Variational Autoencoder (VAE) is a generative type of semi-supervised 634 learning model. Its assumptions on the structure of the data are weak 635 enough for real applications and its training process through 636 backpropagation is fast [Doersch]. Therefore, VAE is suitable in IoT 637 applications where data tends to be diverse and scarce. 639 (Generative Adversarial Network) Generative Adversarial Network (GAN) 640 is a hybrid type of semi-supervised learning model which contain two 641 neural networks, namely the generative and discriminative networks 642 [Goodfellow]. The generator is trained to learn the data distribution 643 from a training dataset in order to generate new data which can 644 deceive the latter network, so-called the discriminator. Then, the 645 discriminator learns to discriminate the generated data from the real 646 data. In IoT applications, GAN can be used in situations when 647 something needs to be generated from the available data, such as 648 localization, way-finding, and data type conversion. 650 5.2. Technologies for lightweight and real-time intelligence 652 As the era of IoT has come, some sort of light-weight intelligence is 653 needed to support smart objects. Prior to the era of IoT, most of the 654 works on learning did not consider resource-constrained environments. 655 Especially, deep learning models require many resources such as 656 processing power, memory, stable power source, etc. However, it has 657 been recently shown that the parameters of the deep learning models 658 contain redundant information, so that some parts of them can be 659 delicately removed to reduce complexity without much degradation of 660 performance [Ba], [Denil]. In this section, the technologies to 661 achieve real-time and serverless learning in IoT environments are 662 introduced. 664 (network compression) Network compression is a method to convert a 665 dense network into a sparse one. With this technology the network can 666 be reduced in its size and complexity. By pruning irrelevant parts or 667 sharing redundant parameters, the storage and computational 668 requirements can be decreased [Han]. After pruning, the performance 669 of the network is examined and the pruning process is repeated until 670 the performance reaches the minimum requirements for the specific 671 applications and use cases. As many parameters are removed or shared, 672 the memory required is reduced, as well as computational burden and 673 energy. Especially as most energy in neural network is used to access 674 memory, the consumed energy dramatically drops. Although its main 675 limitation is that there is not a general solution to compress all 676 kinds of network, but it rather depends on the characteristics of 677 each network. However, network compression is still the most 678 widespread method to make deep learning technologies to be 679 lightweight and IoT-friendly. 681 (approximate computing) Approximate computing is an approach to 682 support deep learning in smart devices [Venkataramani], [Moons]. It 683 is based on the facts that the results of deep learning do not need 684 to be exact in many IoT applications but still valid if the results 685 are in an acceptable range. By integrating approximate computing into 686 deep learning, not only the execution time but also the energy 687 consumption are reduced [Mohammadi]. Based on the optimal trade-off 688 between accuracy and run-time or energy consumption, the network can 689 be adjustably approximated. The network approximate technology can be 690 well-used in such situations when the response time is more important 691 than sophisticatedly analyzed results. Although it is a technology to 692 facilitate real-time and lightweight intelligence, the process of 693 training models and converting it to approximate network require some 694 amount of resource. Therefore, the approximated model can be deployed 695 on smart devices but the learning and approximation processes still 696 need to take places on resource rich platforms. 698 6. IANA Considerations 700 This document requests no action by IANA. 702 7. Acknowledgements 704 8. Contributors 705 9. Informative References 707 [Hochreiter] S. Hochreiter and J. Schmidhuber, "Long short-term 708 memory," Neural Comput., vol. 9, no. 8, pp. 1735-1780, Nov. 1997. 710 [Chung] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, "Empirical 711 evaluation of gated recurrent neural networks on sequence modeling," 712 arXiv preprint arXiv:1412.3555v1 [cs.NE], 2014. 714 [Krizhevsky] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet 715 classification with deep convolutional neural networks," in Proc. Adv. 716 Neural Inf. Process. Syst., 2012, pp. 1097-1105. 718 [Doersch] C. Doersch, "Tutorial on variational autoencoders," arXiv 719 preprint arXiv:1606.05908v2 [stat.ML], 2016. 721 [Goodfellow]. I. Goodfellow et al., "Generative adversarial nets," in 722 Proc. Adv. Neural Inf. Process. Syst., 2014, pp. 2672-2680. 724 [Ba] J. Ba and R. Caruana, "Do deep nets really need to be deep?" in 725 Proc. Adv. Neural Inf. Process. Syst., Montreal, QC, Canada, 2014, pp. 726 2654-2662. 728 [Denil] M. Denil, B. Shakibi, L. Dinh, N. de Freitas, and M. Ranzato, 729 "Predicting parameters in deep learning," in Proc. Adv. Neural Inf. 730 Process. Syst., 2013, pp. 2148-2156. 732 [Han] S. Han, J. Pool, J. Tran, and W. Dally, "Learning both weights 733 and connections for efficient neural network," in Proc. Adv. Neural 734 Inf. Process. Syst., Montreal, QC, Canada, 2015, pp. 1135-1143. 736 [Venkataramani] S. Venkataramani, A. Ranjan, K. Roy, and A. 737 Raghunathan, "AxNN: Energy-efficient neuromorphic systems using 738 approximate computing," in Proc. Int. Symp. Low Power Electron. 739 Design, ACM, 2014, pp. 27-32. 741 [Moons] B. Moons, B. De Brabandere, L. Van Gool, and M. Verhelst, 742 "Energy- efficient ConvNets through approximate computing," in Proc. 743 IEEE Winter Conf. Appl. Comput. Vis. (WACV), Lake Placid, NY, USA: 744 IEEE, 2016, pp. 1-8. 746 [Mohammadi] Mohammadi, Mehdi, et al. "Deep learning for IoT big data 747 and streaming analytics: A survey," IEEE Communications Surveys & 748 Tutorials, 2018, pp. 2923-2960. 750 [Kriegel] Kriegel, HansPeter, et al. "Densitybased clustering," 751 Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 752 2011, pp. 231-240. 754 Authors' Addresses 756 Jun Kyun Choi (editor) 757 Korea Advanced Institute of Science and Technology (KAIST) 758 193 Munji Ro, Yuseong-gu, Daejeon 759 Korea 761 Email: jkchoi59@kaist.ac.kr 763 Na Kyoung Kim 764 Korea Advanced Institute of Science and Technology (KAIST) 765 193 Munji Ro, Yuseong-gu, Daejeon 766 Korea 768 Email: nkim71@kaist.ac.kr 770 Jae Seob Han 771 Korea Advanced Institute of Science and Technology (KAIST) 772 193 Munji Ro, Yuseong-gu, Daejeon 773 Korea 775 Email: j89449@kaist.ac.kr 777 Min Kyung Kim 778 Korea Advanced Institute of Science and Technology (KAIST) 779 193 Munji Ro, Yuseong-gu, Daejeon 780 Korea 782 Email: mkkim1778@kaist.ac.kr 784 Gyu Myoung Lee 785 Liverpool John Moores University 786 Barkhill Rd, Merseyside, Liverpool L17 6BD 787 United Kingdom 789 Email: G.M.Lee@ljmu.ac.uk