idnits 2.17.1 draft-wang-detnet-joint-scheduling-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 11, 2017) is 2357 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'I-D.finn-detnet-architecture' is defined on line 493, but no explicit reference was found in the text Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 DetNet H. Wang 2 Internet Draft P. Wang 3 Intended status: Standards Track H. Yang 4 Expires: May 15, 2018 Chongqing University of 5 Posts and Telecommunications 6 November 11, 2017 8 Joint Real-Time Scheduling Methods for Deterministic Industrial 9 Field/Backhaul Networks 10 draft-wang-detnet-joint-scheduling-02 12 Abstract 14 In industrial field/backhaul networks, joint real-time scheduling 15 method is important to make end-to-end flows meet their deadline. 16 This document proposes four joint scheduling methods, and these 17 methods have considered four scenarios: time-slotted industrial 18 backhaul network, regarding industrial backhaul network as a black 19 box system, ignoring delay of industrial backhaul and establishing 20 latency model of industrial backhaul network. 22 Status of this Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF), its areas, and its working groups. Note that 29 other groups may also distribute working documents as Internet- 30 Drafts. 32 Internet-Drafts are draft documents valid for a maximum of six 33 months and may be updated, replaced, or obsoleted by other documents 34 at any time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 The list of current Internet-Drafts can be accessed at 38 http://www.ietf.org/ietf/1id-abstracts.txt 40 The list of Internet-Draft Shadow Directories can be accessed at 41 http://www.ietf.org/shadow.html 43 This Internet-Draft will expire on May 15, 2018. 45 Copyright Notice 47 Copyright (c) 2017 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (http://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with 55 respect to this document. Code Components extracted from this 56 document must include Simplified BSD License text as described in 57 Section 4.e of the Trust Legal Provisions and are provided without 58 warranty as described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction ................................................. 2 63 2. Deterministic Industrial Field/Backhaul Network Requirement .. 4 64 3. Deterministic Industrial Field/Backhaul Network Joint Scheduling 65 Key Technology ............................................... 5 66 3.1. End-to-end Network Data Stream .......................... 5 67 3.2. Network Communication Resource .......................... 5 68 3.3. Network Time Slot Scheduling ............................ 6 69 4. Joint Real-Time Scheduling Methods for Deterministic Industrial 70 Field-Backhaul Network ....................................... 6 71 4.1. Time-Slotted Industrial Backhaul Networks ............... 6 72 4.2. Consider Industrial Backhaul Network as a Black Box .... 10 73 4.3. Ignore the Delay of Industrial Backhaul Network ........ 11 74 4.4. Build Delay Model of Industrial Backhaul Network ....... 11 75 5. Security Considerations ..................................... 11 76 6. IANA Considerations ......................................... 11 77 7. References .................................................. 12 78 7.1. Normative References ................................... 12 79 7.2. Informative References ................................. 12 80 Authors' Addresses ............................................. 13 82 1. Introduction 84 Industrial field network is a network that can be deployed in 85 process control industry to monitor industrial field equipment for 86 achieving the target of control and management. Industrial field 87 network can improve production efficiency, reduce human intervention 88 and decrease cost. It is significant for industrial modernization. 90 Industrial field bus and industrial Ethernet are two kinds of common 91 solutions of industrial automation, while they are both a wired 92 network. With the development of industrial wireless technology, 93 Wireless Sensor Networks (WSN), which is a typical industrial 94 wireless network, has been applied to industrial network gradually. 95 WSN can free traditional field devices from the limits of abundant 96 cables, and it is easy and flexible to deploy in industrial 97 environment. Wireless network can be applied to building automation, 98 process automation, and industrial automation. Currently, There are 99 three major industrial wireless networks international standards: 100 ISA100.11a[IEC62734], WirelessHART[IEC62591], WIA-PA[IEC62601]. 102 Industrial backhaul network is a transition network, which combines 103 industrial field network with high-level network to achieve the goal 104 of interconnection. It mainly solves the problem that makes the 105 sensor or control data from industrial field network transmit to 106 high-level network. Generally, industrial field network is limited 107 to a specific region, such as a plant. Through industrial backhaul 108 network, data of industrial field network can be transferred to 109 internet or other industrial field networks. Industrial backhaul 110 network is a medium-sized network, which can cover from a few 111 kilometers to tens of kilometers. The major technology of industrial 112 wireless backhaul network consists of Wi-Fi, WiMAX and LTE. 114 In order to adapt to the burgeoning industry 4.0, which aims to 115 elevate the level of manufacturing, industrial field network should 116 not be confined to a plant network only. Therefore, it is necessary 117 to introduce the technology of industrial backhaul network to break 118 the restrictions of interconnection between different networks, and 119 construct a hybrid industrial network. Figure 1 indicates a typical 120 network architecture of the hybrid industrial network. It is a type 121 of architecture of industrial deterministic network that was 122 illustrated with use cases in the drafts proposed by DetNet 123 Workgroup of IETF of [I-D.bas-usecase-detnet] and [I-D.finn-detnet- 124 architecture]. 126 +-----------------------------------+ 127 | | 128 | | 129 | Backhaul network | 130 | | 131 | | 132 +-----------------------------------+ 133 / \ 134 / \ 135 +-------------------------------+ +-------------------------------+ 136 | | | | 137 | | | | 138 | Field network | | Field network | 139 | | | | 140 | | | | 141 +-------------------------------+ +-------------------------------+ 143 Figure 1. Typical industrial field/backhaul network 145 In the hybrid network architecture, field network may be an 146 ISA100.11a, which is an industrial WSN protocol. In Figure 1, a node 147 deployed in a plant can communicate with a node deployed in another 148 plant through backhaul network. 150 2. Deterministic Industrial Field/Backhaul Network Requirement 152 The draft of [I-D.finn-detnet-problem-statement], proposed by DetNet 153 Workgroup of IETF, has described the requirements of deterministic 154 network and deterministic scheduling partially. Because industrial 155 field network directly faces the monitoring of industrial process, 156 there is a difference between industrial field network and general 157 network. Industrial field network has high demands on the 158 deterministic delay bounds. In a field network, the delay of data 159 transmission will affect productivity, and even generate industrial 160 accidents when happening high packet loss ratio and transmission 161 latency. For example, real-time measure and control of liquid level 162 is required to avoid overfilling of oil tanks, because overflow may 163 lead to serious economic loss and environmental threats. Therefore, 164 it requires a deterministic joint scheduling method to guarantee the 165 deterministic transmission of data stream in the new network 166 architecture. 168 3. Deterministic Industrial Field/Backhaul Network Joint Scheduling Key 169 Technology 171 3.1. End-to-end Network Data Stream 173 In industrial field/backhaul network, end-to-end data stream 174 indicates a complete transmission path that a source node of field 175 network transfers to destination node located in another field 176 network through an industrial backhaul network. 178 Industrial field/backhaul network data stream has following features: 180 o Period. Every data stream generates data with periodicity. 182 o Deterministic. Every data stream has a deadline, and scheduling 183 methods should ensure each data stream arrives at destination 184 node before its deadline. 186 o Sequential. A path of an end-to-end data stream contains some 187 transmission links. In the process of scheduling, it must be 188 scheduled in the order of sequence of links on the path. 190 o Priority. End-to-end data stream has a priority. When data 191 streams with different priorities occur collisions, the data 192 streams with lower priority should be delayed by higher priority 193 data streams. 195 3.2. Network Communication Resource 197 In deterministic industrial field/backhaul network architecture, 198 network communication resources include time slot, channel and link. 199 If backhaul network adopts Software Defined Network (SDN) 200 architecture, then the SDN controller can schedule the bandwidth and 201 cache of switch. Therefore, bandwidth and cache resources can be 202 included in schedulable communication resources. 204 o Time slot. Time slot is the basic transmission unit in the 205 network communications based on Time Division Multiple Access 206 (TDMA). In the entire network, the length of time slots is fixed 207 and stays the same. Only one sending packet and its corresponding 208 ACK can be accommodated in one time slot. 210 o Channel. In order to increase network throughput, industrial 211 field network provides a number of channels with different 212 frequencies. 214 o Link. Link refers to a direct packet transmission between two 215 nodes that located in a communication radius of each other. A 216 data stream comprises many links. 218 3.3. Network Time Slot Scheduling 220 In TDMA-based industrial field network, time is divided into time 221 slots with the same length. In the time-slot scheduling process, it 222 will cause link collision when a node wants to transmit and receive 223 simultaneously, and it will cause channel collision when the same 224 channel is used within a certain range. As shown in Figure 2, the 225 time-slot scheduling process should avoid such collisions. 227 +---+ +---+ +---+ +---+ +---+ +---+ +---+ 228 | A |-->| B |-->| C | | A |-->| B | | C |-->| D | 229 +---+ +---+ +---+ +---+ +---+ +---+ +---+ 231 +---------+------------+ +---------+------------+ 232 |Time slot| Time slot 0| |Time slot| Time slot 0| 233 +---------+------------+ +---------+------------+ 234 |Channel 0| A->B | |Channel 0| A->B | 235 +---------+------------+ | | C->D | 236 |Channel 1| B->C | +---------+------------+ 237 +---------+------------+ 238 Figure 2. Link Collision & Channel Collision 240 4. Joint Real-Time Scheduling Methods for Deterministic Industrial 241 Field-Backhaul Network 243 Joint real-time scheduling methods for industrial field/backhaul 244 networks intend to solve the deterministic problem of industrial 245 field/backhaul networks. Due to the investigative architecture 246 includes backhaul network, the deterministic scheduling algorithm 247 needs to collaborate with backhaul network to conduct joint 248 scheduling to ensure data deterministic transmission. The proposed 249 joint scheduling methods are described as follows. 251 4.1. Time-Slotted Industrial Backhaul Networks 253 In order to ensure determinism, industrial field networks utilize 254 TDMA to make the network time-slotted. If the industrial backhaul 255 network can also be time-slotted, then the deterministic scheduling 256 algorithm can jointly schedule with small modification. Industrial 257 backhaul network can be a variety of network standards such as WIFI, 258 WiMAX, and LTE. WiMAX and LTE are high cost and poor feasibility, 259 thus we assume the IEEE 802.11 as backhaul network. Wi-Fi network 260 has various operating modes, such as peer-to-peer mode, point to 261 multi-point networking mode and the relay network mode. Here we 262 consider the hierarchical network architecture in a way of point to 263 multi-point networking mode, as shown in Figure 3. 265 +----------------------------------------+ 266 | | 267 | +--------+ | 268 | +-------| Head AP|-------+ | 269 | | +--------+ | | 270 | | | | 271 | +--------+ +--------+ | 272 +---+---| AP1 | | AP2 |---+---+ 273 | | +--------+ +--------+ | | 274 | +----------------------------------------+ | 275 | | 276 +---------------------------------++----------------------------------+ 277 |ISA100.11a field wireless network||ISA100.11a field wireless network | 278 +---------------------------------++----------------------------------+ 280 Figure 3. Industrial Backhaul Network consisting of WIFI 282 Although IEEE 802.11 supports multiple channels, but AP is not able 283 to perform channel hopping between transmission timeslots, which 284 means that the AP cannot use a channel in the current time slot and 285 use another channel the next time slot. We assume that AP1 and AP2 286 in Figure 3 can transmit packets simultaneously as long as their 287 transmission tasks do not contain the same AP, i.e. head AP. For 288 example, when a data stream of field network is transmitting packets 289 to AP1 in a time slot, AP2 is able to receive packets from head AP, 290 or send packets to field network in the same time slot. Therefore, 291 the backhaul network framework with wireless APs can be considered 292 as a single-channel linear network, which is shown in Figure 4. 294 +---------+ +--------+ +--------+ +--------+ +---------+ 295 | Gateway |--> | AP |--> | AP |--> | AP |--> | Gateway | 296 +---------+ +--------+ +--------+ +--------+ +---------+ 298 Figure 4. A single-channel linear network 300 Therefore, the data stream in industrial field/backhaul network can 301 be seen as equivalent to the data stream in field network, except 302 that each data stream needs to flow through the WIFI. The scheduling 303 process is illustrated as follows: 305 1. Abstract end-to-end data stream in the entire network, and 306 allocate a priority for each stream. 308 2. Establish the delay model of network data stream. If collisions 309 happened between different priority data stream, the low priority 310 data stream will be delayed by high priority, so a model can be 311 built under the worst circumstances that the low-priority data 312 streams impacted by all higher priority data streams. 314 3. Estimate the network schedulability. A data stream is schedulable 315 when the minimum time for the data stream to complete its once 316 transmission task plus the worst delay time caused by higher 317 priority data streams is less than or equal to its deadline, In 318 the current priority allocation scheme, if each data stream is 319 schedulable, the network can be considered as schedulable. If the 320 data stream cannot be scheduled, then try to change the priority 321 allocation scheme and estimate again until a corresponding scheme 322 is found or return no schedulable results. 324 4. Allocate time slot and channel for each data stream. Traverse 325 data streams according to their priorities, and each data stream 326 should allocate link that is about to be released in a time slot. 327 According to the rule that low priority data streams should give 328 way to high priority data streams, the channels can be utilized 329 if it is not unoccupied. However, if collisions happened between 330 data streams of different priority, then the lower priority data 331 stream should be placed in the next time slot until there are no 332 unallocated higher priority data streams. Repeat these steps 333 until the whole network scheduling is completed. 335 The scheduling process is described in Figure 5: 337 +----------+ 338 | Begin | 339 +----------+ 340 | 341 | 342 +---------------------------+ 343 | Initial the priority of | 344 | each data stream | 345 +---------------------------+ 346 |<--------------------------------------+ 347 | | 348 +--------------------+ +------------------------------+ 349 / Traverse every data \ no | If the data stream cannot be | 350 / stream and estimate the\--------->| scheduled, then change the | 351 \ schedulablity according/ | priority allocation scheme | 352 \ to delay model / | and estimate again | 353 +--------------------+ +------------------------------+ 354 | 355 |yes 356 +-----------------------------------+ 357 | Traverse data streams according to| 358 | their priority, each data stream | 359 | should allocate the next link that| 360 | is about to be released in each | 361 | time slot to the greatest extent | 362 +-----------------------------------+ 363 | 364 | 365 +-----------------------------------+ 366 | The spare channels can be utilized| 367 | if there is no collision. If | 368 | collisions happened, then the | 369 | lower priority data stream should | 370 | be placed in the next time slot | 371 +-----------------------------------+ 372 | 373 | 374 +-------+ 375 | End | 376 +-------+ 377 Figure 5. Scheduling of times-slotted industrial backhaul network 379 Further, if the backhaul network can support TDMA mechanism like the 380 industrial field network completely, the deterministic scheduling 381 methods in field network can be applied in industrial field/backhaul 382 networks. 384 For backhaul network using wired technology, time-sensitive network 385 based on Ethernet is preferred for industrial scenarios. Time- 386 sensitive network can provide dedicated slots for scheduled traffic, 387 so above scheduling method can be used in this kind of backhaul 388 network to guarantee the deterministic performance for data flows 389 across field and backhaul networks. 391 4.2. Consider Industrial Backhaul Network as a Black Box 393 In order to solve the deterministic problem of industrial network, 394 backhaul network can be regarded as a black box so that we can only 395 consider its delay impacts and ignore its internal details. 397 When the packet passes through the industrial backhaul network, we 398 can give it a timestamp at the application layer and read it after 399 the transmission is ready to leave the backhaul network. Delay 400 caused in backhaul network can be calculated, and a fitting curve of 401 delay can be obtained by collecting large amount of data. It has 402 been verified experimentally that the delay is concentrated in a 403 numerical range despite its randomness. Therefore, we can estimate 404 the approximate delay time caused by industrial backhaul network. 406 A main scheduling path can be configured according to the average 407 delay of the backhaul network. Some redundant paths should be pre- 408 configured in case the delay of the main path is too high. The 409 scheduling process of industrial field/backhaul network can be 410 divided into three sections, as shown in Figure 6: 412 +--------------------+ +-----------------+ +------------------------+ 413 | Scheduling of | | Delay of | | Scheduling of | 414 |source field subnet |->| backhaul network|->|destination field subnet| 415 | (deterministic) | |(indeterministic)| |( deterministic dynamic)| 416 +--------------------+ +-----------------+ +------------------------+ 417 Period 1 Period 2 Period 3 419 Figure 6. Three periods of scheduling 421 In source field subnet we can apply the deterministic scheduling 422 algorithm of field network to get the time spent by each data stream 423 before entering the source subnet. Then the data stream enters the 424 backhaul network, which will cause indeterministic delay in a 425 numerical range. When the data stream leaves the backhaul network, 426 the timestamp should be parsed. If the deadline is missed, it 427 indicates that the packet has gone through poor network and needs to 428 be retransmitted. If there is free time after leaving the backhaul 429 network, scheduling path can be dynamically selected at downward 430 gateway to get the schedulability of the end-to-end data stream. 432 4.3. Ignore the Delay of Industrial Backhaul Network 434 Since the field network is slow-speed (250 KB/s), while industrial 435 backhaul network is high-speed, if the industrial backhaul networks 436 adopt IEEE 802.11 protocol, gigabit wireless routers supporting IEEE 437 802.11 ac can make the delay of industrial backhaul network quite 438 low. As a result, the joint deterministic scheduling of the entire 439 network only needs to consider the field networks. 441 4.4. Build Delay Model of Industrial Backhaul Network 443 If industrial backhaul network is constructed with IEEE 802.11, the 444 network access delay test model in IEEE 802.11 Distributed 445 Coordination Function (DCF) mode can be established by using Markov 446 chain or queue theory. While the model in IEEE 802.11 Point 447 Coordination Function (PCF) mode can be established based on queue 448 theory. 450 Therefore, the field network needs to build a delay model, while 451 backhaul network follows another delay model, then the total 452 transmission scheduling delay will have certain regularity. The 453 total transmission delay will meet delay requirements with specified 454 probability by scheduling, in other words, the unsuccessful 455 scheduling is acceptable. 457 5. Security Considerations 459 6. IANA Considerations 461 This memo includes no request to IANA. 463 7. References 465 7.1. Normative References 467 7.2. Informative References 469 [IEC62734] 470 ISA/IEC, "ISA100.11a, Wireless Systems for Automation, 471 also IEC 62734", 2011, . 475 [IEC62591] 476 IEC, "Industrial Communication Networks - 477 Wireless Communication Network and Communication Profiles 478 - WirelessHART - IEC 62591", 2010, 479 482 [IEC62601] 483 IEC, "Industrial networks - Wireless communication network 484 and communication profiles - WIA-PA - IEC 62601", 2015, < 485 https://webstore.iec.ch/preview/info_iec62601%7Bed2.0%7Db 486 .pdf> 488 [I-D.finn-detnet-problem-statement] 489 Finn, N. and P. Thubert, "Deterministic Networking Problem 490 Statement", draft-finn-detnet-problem-statement-05 (work in 491 progress), March 2016. 493 [I-D.finn-detnet-architecture] 494 Finn, N., Thubert, P., and M. Teener, "Deterministic 495 Networking Architecture", draft-finn-detnet-architecture-08 496 (work in progress), August 2016. 498 [I-D.bas-usecase-detnet] 499 Kaneko, Y., Toshiba and Das, S, "Building Automation Use 500 Cases and Requirements for Deterministic Networking", draft- 501 bas-usecase-detnet-00 (work in progress), October 2015. 503 Authors' Addresses 505 Heng Wang 506 Chongqing University of Posts and Telecommunications 507 2 Chongwen Road 508 Chongqing, 400065 509 China 511 Phone: (86)-23-6248-7845 512 Email: wangheng@cqupt.edu.cn 514 Ping Wang 515 Chongqing University of Posts and Telecommunications 516 2 Chongwen Road 517 Chongqing, 400065 518 China 520 Phone: (86)-23-6246-1061 521 Email: wangping@cqupt.edu.cn 523 Hang Yang 524 Chongqing University of Posts and Telecommunications 525 2 Chongwen Road 526 Chongqing, 400065 527 China 529 Phone: (86)-23-6246-1061 530 Email: 18716322620@163.com