idnits 2.17.1 draft-du-detnet-layer3-low-latency-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (October 25, 2021) is 914 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-10) exists of draft-ietf-detnet-bounded-latency-07 Summary: 0 errors (**), 0 flaws (~~), 3 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group Z. Du 3 Internet-Draft P. Liu 4 Intended status: Informational China Mobile 5 Expires: April 28, 2022 October 25, 2021 7 Micro-burst Decreasing in Layer3 Network for Low-Latency Traffic 8 draft-du-detnet-layer3-low-latency-04 10 Abstract 12 It is complex to support deterministic forwarding in a large scale 13 network because there is too much dynamic traffic in the network and 14 the data model becomes hard to predict after aggregation in the 15 intermediate nodes. This document introduces the problem of micro- 16 bursts in layer3 network, and analyses the method to decrease the 17 micro-bursts in layer3 network for low-latency traffic. 19 Requirements Language 21 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 22 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 23 document are to be interpreted as described in RFC 2119 [RFC2119]. 25 Status of This Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at https://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on April 28, 2022. 42 Copyright Notice 44 Copyright (c) 2021 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (https://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 60 2. Gaps for Large-scale Layer 3 Deterministic Network . . . . . 3 61 3. Micro-burst Problem in IP Forwarding . . . . . . . . . . . . 3 62 4. Analysis of the Method to Decrease Micro-bursts . . . . . . . 5 63 5. An Example of Method to Decrease Micro-bursts . . . . . . . . 5 64 5.1. Working Flow of the Method . . . . . . . . . . . . . . . 6 65 5.2. Process of Edge Node . . . . . . . . . . . . . . . . . . 6 66 5.3. Process of Forwarding Node . . . . . . . . . . . . . . . 6 67 5.4. Analysis of the Proposed Method . . . . . . . . . . . . . 7 68 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8 69 7. Security Considerations . . . . . . . . . . . . . . . . . . . 8 70 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 8 71 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 8 72 9.1. Normative References . . . . . . . . . . . . . . . . . . 8 73 9.2. Informative References . . . . . . . . . . . . . . . . . 8 74 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 9 76 1. Introduction 78 The DetNet architecture [RFC8655] is supposed to work in campus-wide 79 networks and private WANs, including the large-scale ISP network 80 scenario, such as the 5G bearing network, as mentioned in [RFC8578]. 81 It is essential for the large-scale ISP network to be able to provide 82 the low-latency service. The low-latency requirement exists in both 83 L2 and L3 networks, and in both small and large networks. 85 However, as talked in [I-D.qiang-detnet-large-scale-detnet], 86 deploying deterministic services in a large-scale network brings a 87 lot of new challenges. A novel method called LDN (Large-scale 88 Deterministic Network) is introduced in 89 [I-D.qiang-detnet-large-scale-detnet] and 90 [I-D.dang-queuing-with-multiple-cyclic-buffers], which explore the 91 deterministic forwarding over a large-scale network. 93 This document also explores the deterministic service in the large- 94 scale layer 3 network, and analyses the method based on micro-burst 95 decreasing, which can benefit the forwarding of low-latency traffic 96 in the large-scale network. 98 2. Gaps for Large-scale Layer 3 Deterministic Network 100 In this document, the large-scale network means that there are many 101 dynamic flows in the network, but it is hard to do per-flow shaping 102 on the intermediate nodes because they have high pressure on 103 forwarding on the data plane. 105 According to [RFC8655], DetNet operates at the IP layer and delivers 106 service over lower-layer technologies such as MPLS and IEEE 802.1 107 Time-Sensitive Networking [TSN]. However, the TSN mechanisms are 108 designed for L2 network originally, and cannot be directly used in 109 the large-scale layer 3 network because of various reasons. Some of 110 them are described as below. 112 Some TSN mechanisms need synchronization of the network equipments, 113 which is easier in a small network, but hard in a large network. It 114 brings in some complex maintenance jobs across a long distance that 115 are not needed before. 117 Some TSN mechanisms need a per-flow state in the forwarding plane, 118 which is un-scalable. Aggregation methods need to be considered. 120 Some TSN mechanisms need a constant and forecastable traffic 121 characteristics, which is more complicated in a large network which 122 includes much more flows joining in or leaving randomly and the 123 traffic characteristics are more dynamic. 125 The main aspects of the problems are the simplicity and the 126 scalability. The former can ensure that the mechanism is easy to 127 deploy, and the second can ensure that the mechanism is able to bear 128 a large number of deterministic services. 130 3. Micro-burst Problem in IP Forwarding 132 The current IP forwarding mechanism is considered to be a good 133 example fulfilling the requirements of simplicity and scalability. 134 However, the traditional IP network is based on statistical 135 multiplexing, and can only provide Best Effort service, short of SLA 136 guaranteed mechanisms. 138 When we rethink the problem in the current IP forwarding mechanism, 139 we can find that in the current IP network, a long delay in queuing, 140 or some packet losses due to burst are acceptable; however, it may be 141 unacceptable in the deterministic forwarding. Therefore, they have 142 different design principles in the low layer. 144 The current forwarding mechanism in an IP router, which is based on 145 statistical multiplexing, can not provide the deterministic service 146 because of various reasons. Even be given a high priority, a 147 critical packet can experience a long congestion delay or be lost in 148 a relatively light-loaded network, which is caused by micro-bursts in 149 the network. 151 Micro-burst is a special case of network congestion, which typically 152 lasts a short period, at the granularity of millisecond. In a micro- 153 burst, a lot of data are received on the interface suddenly, and the 154 temporary bandwidth requirement would be tens of or hundreds of the 155 average bandwidth requirement, or even exceed the interface 156 bandwidth. 158 In most cases, the buffer on the equipment can handle the micro- 159 bursts. However, in some corner cases, micro-bursts bring in a long 160 delay (for example, at the granularity of millisecond) or even packet 161 loss. 163 The following paragraphs introduce the causes of the micro-burst. 165 Firstly, IP traffic has an instinct of burstiness no matter in the 166 macro or micro aspect, i.e., it does not have a constant traffic 167 model even after aggregations. 169 Secondly, IP network has a flexible topology, where the incoming 170 traffic may exceed the bandwidth of the outgoing interface. For 171 example, an interface with a large bandwidth may need to send traffic 172 to an interface with a smaller bandwidth, or multiple flows from 173 several incoming interfaces may need to occupy the same outgoing 174 interface. 176 Thirdly, the IP node has been designed to send traffic as quickly as 177 possible, and it is not aware whether the downstream node's buffer 178 can handle the traffic. For example, Figure 1 below shows the 179 problem of the current IP scheduling mechanism. Before the 180 scheduling in an IP network, the packets are well paced, but after 181 the scheduling, the packets will be gathered even the total traffic 182 rate is unchanged. When an IP outgoing interface receives multiple 183 critical flows from several incoming interfaces, the situation 184 becomes worse. However, an IP router will try to send them as soon 185 as possible, so occasionally, in some later hops, micro-bursts will 186 emerge. 188 _ _ _ _ _ _ _ _ _ _ _ 189 | | | | | | | | | | | | | | | | | | | | | | 190 --------------------------------------------------------------------- 191 Before scheduling in an IP network 193 _ _ _ _ _ _ _ _ _ _ _ 194 | || || || || || | | || || || || | 195 --------------------------------------------------------------------- 196 After scheduling in an IP network 198 Figure 1: Change of the traffic characteristics in an IP network 200 4. Analysis of the Method to Decrease Micro-bursts 202 This document analyses the method to support the low latency traffic 203 bearing in an IP network, such as the 5G bearing network, by avoiding 204 micro-bursts in the network as much as possible. The principle in 205 this method is to forward critical and BE traffic separately, and do 206 not distinguish different critical flows on the forwarding plane on 207 the intermediate nodes. 209 As talked before, the target method should be scalable and easy to 210 deploy. As the intermediate nodes have high pressure on forwarding 211 packets, the target method should not bring in too much complex 212 process on the data plane. Several requirements are listed as 213 follows. 215 The first is that the DetNet traffic should support aggregation. The 216 intermediate nodes should not do per-flow process on the date plane. 218 The second is that separation process of the control plane and data 219 plane on the intermediate nodes. The status of the aggregated DetNet 220 traffic on the control plane may change frequently in the large-scale 221 network. We should not assume that the control plane on an 222 intermediate node can interact with the data plane frequently, for 223 example, to change a shaper parameter frequently. On the data plane, 224 some self-decision process should be supported. 226 5. An Example of Method to Decrease Micro-bursts 228 In this section, we describes an example of method fulfilling the 229 requirements mentioned in the last section. It needs the cooperation 230 of the edge nodes and the forwarding/intermediate nodes in an IP 231 network. 233 5.1. Working Flow of the Method 235 Generally, the method contains two steps: 237 Step1: per flow schedule on the edge node. The purpose is to make 238 sure that each critical traffic has a constant traffic model. 240 Step2: per interface schedule on the intermediate node. Traffic are 241 aggregated to ensure the scalability, and the pacing also makes sure 242 that they do not gather. The purpose is to make the critical traffic 243 be forwarded as the shape when outgoing the edge, not as quickly as 244 possible. We assume that the sending rate of the buffer for the 245 critical traffic is the same as the receiving rate (how to achieve 246 this is out of scope of this document). If all work well, the buffer 247 will be maintained with a proper depth. 249 Other requirements include an RSVP-TE liked mechanism with a good 250 scalability, which should be used to make sure the bandwidth is not 251 exceeded on the interface. 253 5.2. Process of Edge Node 255 The edge node of the IP network can recognize each critical flows 256 just as in the TSN network, and then give them individually a good 257 shaping. In fact, in TSN mechanisms, no micro-burst will emerge for 258 critical traffic, and each TSN mechanism is proved to be effective 259 under certain conditions. 261 This document suggests the edge node to shape the critical traffic by 262 using the CBS method in [IEEE802.1Qav], or the shaping methods in 263 [IEEE802.1Qcr]. Generally, the shaping methods can generate a paced 264 traffic for each critical flow. 266 The parameters of the shaper, such as the sending rate, can be 267 configured for each flow by some means. 269 5.3. Process of Forwarding Node 271 For the forwarding node, it is uneasy to recognize each critical flow 272 because of the high pressure of forwarding a large amount of packets. 273 It is suggested that no per-flow state is maintained on the 274 forwarding node. It is to say that, on the forwarding node, the 275 critical flows should be aggregated and handled together. 277 This document suggests that the forwarding node can deploy a specific 278 queue on each outgoing interface. The queue will buffer all critical 279 traffic that need to go out through that interface, and will pace 280 them by using methods mentioned in the last section. 282 A shaping method in TSN is used here instead of the original 283 forwarding method in an IP router, which can make the critical 284 traffic be forwarded orderly instead of as soon as possible. 285 Therefore, micro-bursts can be decreased in the network. 287 If all the forwarding nodes can do their jobs properly, i.e., they 288 can well pace the critical traffic, no or rare micro-bursts for the 289 critical traffic would take place. In this way, the critical traffic 290 will have a relatively low latency in the IP network with less 291 uncertainty of micro-bursts. 293 As no per-flow state is maintained on the forwarding node, the 294 sending rate of the shaper is hard to decide. As said in the last 295 session, the sending rate is suggested to be adjusted referring to 296 the incoming rate of the queue. The purpose is to maintain a proper 297 buffer depth for the queue. 299 Although it is claimed that the proposed method is simpler than the 300 TSN mechanisms, forwarding/intermediate nodes also need to be 301 updated. The detailed realization of the method on the intermediate 302 nodes is out of scope of this document. 304 5.4. Analysis of the Proposed Method 306 The method proposed does not need synchronization, just as the 307 asynchronous mechanisms studied in [IEEE802.1Qcr]. Furthermore, the 308 method has a larger aggregation granularity, which can fulfill the 309 requirements of simplicity and scalability as much as possible. 310 However, in theory, it has a larger uncertainty on the forwarding 311 than the zero congestion loss target in the TSN mechanisms. 313 We compare three mechanisms in the following paragraphs. The first 314 is the priority based light-load mechanism, i.e., the traditional 315 method. The second is the TSN mechanism, such as CQF. The third is 316 the proposed mechanism. 318 In the first mechanism, we only give a high priority to the critical 319 traffic, and thus the scalability of the deterministic system is 320 good. However, the uncertainty on the forwarding plane perhaps can 321 not fulfill the requirements in the industry network where SLA 322 requirements are very essential. Perhaps, it is only able to work 323 well when a small amount of critical traffic exist in the network. 325 If we use the scheduling method in the TSN, such as CQF. Its 326 uncertainty is very low, but its scalability is not very good as said 327 in Section 2. It should be noted that in a large deterministic 328 system, the ISP normally will not guarantee the user 100 percent 329 reliability, instead of which it perhaps is a value very close to. 331 The proposed method has a better scalability than the TSN mechanisms, 332 and a better reliability than the priority based method. If we 333 assume that different services need different deterministic levels, 334 this method may be helpful for the service that does not need a very 335 high deterministic level. For example, the method can be used in the 336 consumption Internet, in which the deterministic service needs a 337 relatively lower deterministic level than the industry Internet. 339 6. IANA Considerations 341 This document has no IANA actions. 343 7. Security Considerations 345 Detailed security considerations can refer to 346 [I-D.ietf-detnet-bounded-latency] and [I-D.ietf-detnet-security]. 348 8. Acknowledgements 350 Thanks for the valuable comments from Janos Farkas, Lou Berger, and 351 David Black. 353 9. References 355 9.1. Normative References 357 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 358 Requirement Levels", BCP 14, RFC 2119, 359 DOI 10.17487/RFC2119, March 1997, 360 . 362 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 363 "Deterministic Networking Architecture", RFC 8655, 364 DOI 10.17487/RFC8655, October 2019, 365 . 367 9.2. Informative References 369 [I-D.dang-queuing-with-multiple-cyclic-buffers] 370 Liu, B. and J. Dang, "A Queuing Mechanism with Multiple 371 Cyclic Buffers", draft-dang-queuing-with-multiple-cyclic- 372 buffers-00 (work in progress), February 2021. 374 [I-D.ietf-detnet-bounded-latency] 375 Finn, N., Boudec, J. L., Mohammadpour, E., Zhang, J., 376 Varga, B., and J. Farkas, "DetNet Bounded Latency", draft- 377 ietf-detnet-bounded-latency-07 (work in progress), 378 September 2021. 380 [I-D.ietf-detnet-security] 381 Grossman, E., Mizrahi, T., and A. J. Hacker, 382 "Deterministic Networking (DetNet) Security 383 Considerations", draft-ietf-detnet-security-16 (work in 384 progress), March 2021. 386 [I-D.qiang-detnet-large-scale-detnet] 387 Qiang, L., Geng, X., Liu, B., Eckert, T., Geng, L., and G. 388 Li, "Large-Scale Deterministic IP Network", draft-qiang- 389 detnet-large-scale-detnet-05 (work in progress), September 390 2019. 392 [IEEE802.1Qav] 393 IEEE 802.1, "IEEE 802.1Qav-2009 - IEEE Standard for Local 394 and metropolitan area networks-- Virtual Bridged Local 395 Area Networks Amendment 12: Forwarding and Queuing 396 Enhancements for Time-Sensitive Streams", 2009, 397 . 399 [IEEE802.1Qcr] 400 IEEE 802.1, "IEEE 802.1Qcr-2020 - IEEE Standard for Local 401 and Metropolitan Area Networks--Bridges and Bridged 402 Networks - Amendment 34: Asynchronous Traffic Shaping", 403 2020, 404 . 406 [RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases", 407 RFC 8578, DOI 10.17487/RFC8578, May 2019, 408 . 410 [TSN] IEEE 802.1, "Time-Sensitive Networking (TSN) Task Group", 411 2012, . 413 Authors' Addresses 415 Zongpeng Du 416 China Mobile 417 No.32 XuanWuMen West Street 418 Beijing 100053 419 China 421 Email: duzongpeng@foxmail.com 422 Peng Liu 423 China Mobile 424 No.32 XuanWuMen West Street 425 Beijing 100053 426 China 428 Email: liupengyjy@chinamobile.com