idnits 2.17.1 draft-ietf-tsvwg-tunnel-congestion-feedback-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 18 has weird spacing: '...A basic model...' == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document date (July 8, 2016) is 2849 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'RFC793' is mentioned on line 91, but not defined ** Obsolete undefined reference: RFC 793 (Obsoleted by RFC 9293) == Missing Reference: 'ConEx' is mentioned on line 105, but not defined == Unused Reference: 'RFC3168' is defined on line 640, but no explicit reference was found in the text == Unused Reference: 'RFC6040' is defined on line 658, but no explicit reference was found in the text == Unused Reference: 'CONEX' is defined on line 673, but no explicit reference was found in the text ** Obsolete normative reference: RFC 4960 (Obsoleted by RFC 9260) == Outdated reference: A later version (-15) exists of draft-ietf-tsvwg-circuit-breaker-01 Summary: 2 errors (**), 0 flaws (~~), 9 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force X. Wei 3 INTERNET-DRAFT Huawei Technologies 4 Intended Status: Informational L.Zhu 5 Expires: January 9, 2017 Huawei Technologies 6 L.Deng 7 China Mobile 8 July 8, 2016 10 Tunnel Congestion Feedback 11 draft-ietf-tsvwg-tunnel-congestion-feedback-02 13 Abstract 15 This document describes a mechanism to calculate congestion of a 16 tunnel segment based on RFC6040 recommendations, and a feedback 17 protocol by which to send the measured congestion of the tunnel from 18 egress to ingress . A basic model for measuring tunnel congestion 19 and feedback is described, and a protocol for carrying the feedback 20 data is outlined. 22 Status of this Memo 24 This Internet-Draft is submitted to IETF in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF), its areas, and its working groups. Note that 29 other groups may also distribute working documents as 30 Internet-Drafts. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 The list of current Internet-Drafts can be accessed at 38 http://www.ietf.org/1id-abstracts.html 40 The list of Internet-Draft Shadow Directories can be accessed at 41 http://www.ietf.org/shadow.html 43 Copyright and License Notice 45 Copyright (c) 2016 IETF Trust and the persons identified as the 46 document authors. All rights reserved. 48 This document is subject to BCP 78 and the IETF Trust's Legal 49 Provisions Relating to IETF Documents 50 (http://trustee.ietf.org/license-info) in effect on the date of 51 publication of this document. Please review these documents 52 carefully, as they describe your rights and restrictions with respect 53 to this document. Code Components extracted from this document must 54 include Simplified BSD License text as described in Section 4.e of 55 the Trust Legal Provisions and are provided without warranty as 56 described in the Simplified BSD License. 58 Table of Contents 60 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 61 2. Conventions And Terminologies . . . . . . . . . . . . . . . . . 3 62 3. Congestion Information Feedback Models . . . . . . . . . . . . 4 63 3.1 Direct Model . . . . . . . . . . . . . . . . . . . . . . . . 4 64 3.2 Centralized Model . . . . . . . . . . . . . . . . . . . . . 4 65 4. Congestion Level Measurement . . . . . . . . . . . . . . . . . 5 66 5. Congestion Information Delivery . . . . . . . . . . . . . . . . 8 67 5.1 IPFIX Extentions . . . . . . . . . . . . . . . . . . . . . . 9 68 5.1.1 ce-cePacketTotalCount . . . . . . . . . . . . . . . . . 9 69 5.1.2 ect0-nectPacketTotalCount . . . . . . . . . . . . . . . 9 70 5.1.3 ect1-nectPacketTotalCount . . . . . . . . . . . . . . . 10 71 5.1.4 ce-nectPacketTotalCount . . . . . . . . . . . . . . . . 10 72 5.1.5 ce-ect0PacketTotalCount . . . . . . . . . . . . . . . . 10 73 5.1.6 ce-ect1PacketTotalCount . . . . . . . . . . . . . . . . 11 74 5.1.7 ect0-ect0PacketTotalCount . . . . . . . . . . . . . . . 11 75 5.1.8 ect1-ect1PacketTotalCount . . . . . . . . . . . . . . . 11 76 6. Congestion Management . . . . . . . . . . . . . . . . . . . . . 12 77 7. Security . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 78 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 12 79 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 14 80 9.1 Normative References . . . . . . . . . . . . . . . . . . . 14 81 9.2 Informative References . . . . . . . . . . . . . . . . . . 15 82 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 15 83 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 16 85 1. Introduction 87 In IP network, persistent congestion (or named congestion collapse) 88 lowers transport throughput, leading to waste of network resource. 89 Appropriate congestion control mechanisms are therefore critical to 90 prevent the network from falling into the persistent congestion 91 state. Currently, transport protocols such as TCP[RFC793], 92 SCTP[RFC4960], DCCP[RFC4340], have their built-in congestion control 93 mechanisms, and even for certain single transport protocol like TCP 94 there can be a couple of different congestion control mechanisms to 95 choose from. All these congestion control mechanisms are implemented 96 on host side, and there are reasons that only host side congestion 97 control is not sufficient for the whole network to keep away from 98 persistent congestion. For example, (1) some protocol's congestion 99 control scheme may have internal design flaws; (2) improper software 100 implementation of protocol; (3) some transport protocols do not even 101 provide congestion control at all. 103 In order to have a better control on network congestion status, it's 104 necessary for the network side to do certain kind of traffic control. 105 For example, ConEx [ConEx] provides a method for network operator to 106 learn about traffic's congestion contribution information, and then 107 congestion management action can be taken based on this information. 109 Tunnels are widely deployed in various networks including public 110 Internet, datacenter network, and enterprise network etc. A tunnel 111 consists of ingress, an egress and a set of interior routers. For the 112 tunnel scenario, a tunnel-based mechanism which is different from 113 ConEx is introduced for network traffic control to keep the network 114 from persistent congestion. Here, tunnel ingress will implement 115 congestion management function to control the traffic entering the 116 tunnel. 118 In order to perform congestion management at ingress, the ingress 119 must first obtain the inner tunnel congestion level information. Yet 120 the ingress cannot use the locally visible traffic rates, because it 121 would require additional knowledge of downstream capacity and 122 topology, as well as cross traffic that does not pass through this 123 ingress. 125 This document provides a mechanism of feeding back inner tunnel 126 congestion level to the ingress. Using this mechanism the egress can 127 feed the tunnel congestion level information it collects back to the 128 ingress. After receiving this information the ingress will be able to 129 perform congestion management according to network management policy. 131 2. Conventions And Terminologies 132 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 133 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 134 document are to be interpreted as described in RFC 2119 [RFC2119] 136 DP: Decision Point, an logical entity that make congestion management 137 decision based on the received congestion feedback information. 139 AP: Action Point, an logical entity that implements congestion 140 management action according to the decision made by Decision Point. 142 3. Congestion Information Feedback Models 144 According to specific network deployment, there are two kinds of 145 feedback model: direct model and centralized model. 147 3.1 Direct Model 148 Feedback 149 +-----------------------------------+ 150 | | 151 | | 152 | V 153 +--------------+ +-------------+ 154 | +--------+ | | +---------+ | 155 | |Exporter| | | |Collector| | 156 | +---|----+ | | +---|-----+ | 157 | +--|--+ | | +|-+ | 158 | |Meter| | | |DP| | 159 | +-----+ | | +--+ | 160 | | | +--+ | 161 | | | |AP| | 162 | | | +--+ | 163 |Egress | | Ingress | 164 +--------------+ +-------------+ 166 (a) Direct Feedback Model. 168 Direct model means egress feeds information directly to ingress. The 169 egress consists of Meter function and Exporter function, the Meter 170 function collects network congestion level information, and convey 171 the information to Exporter which feeds back the information to the 172 Collector function locating at ingress, after that congestion 173 management Decision Point (DP) function on ingress will make 174 congestion management decision based on the information from 175 Collector. The ingress here will act as both the decision point that 176 decides how to do congestion management and the action point that 177 implements congestion management decision. 179 3.2 Centralized Model 180 +-------------------+ 181 |+---------+ +--+ | 182 feedback ||Collector|---|DP| | 183 +---->|+---------+ +--+ |######### 184 | | | # 185 | | Controller | # 186 | +-------------------+ # 187 | # 188 | # 189 +--------------+ +------V------+ 190 | +--------+ | | | 191 | |Exporter| | | | 192 | +---|----+ | | | 193 | +--|--+ | | | 194 | |Meter| | | | 195 | +-----+ | | | 196 | | | +--+ | 197 | | | |AP| | 198 | | | +--+ | 199 |Egress | | Ingress | 200 +--------------+ +-------------+ 202 (b) Centralized Feedback Model 204 In the centralized model, the ingress only takes the role of action 205 point, and it implements traffic control decision from another entity 206 named "controller". Here, after Exporter function on egress has 207 collected network congestion level information, it feeds back the 208 information to the collector of a controller instead of the ingress. 209 Then the controller makes congestion management decision and sends 210 the decision to the ingress to implement. 212 4. Congestion Level Measurement 214 This section describes how to measure congestion level in a tunnel. 216 There could be different approaches of packet loss detection for 217 different tunneling protocol scenarios. For instance, if there is a 218 sequence field in the tunneling protocol header, it will be easy for 219 egress to detect packet loss through the gaps in sequence number 220 space. Another approach is to compare the number of packets entering 221 ingress and the number of packets arriving at egress over the same 222 span of packets. This document will focus on the latter one which is 223 a more general approach. 225 If the routers support Explicit Congestion Notification (ECN), after 226 router's queue length is over a predefined threshold, the routers 227 will marks the ECN-capable packets as Congestion Experienced (CE) or 228 drop not-ECT packets with the probability proportional to queue 229 length; if the queue overflows all packets will be dropped. If the 230 routers do not support ECN, after router's queue length is over a 231 predefined threshold, the routers will drop both the ECN-capable 232 packets and the not-ECT packets with the probability proportional to 233 the queue length. It's assumed all routers in the tunnel support ECN. 235 Faked ECN-capable transport (ECT) is used at ingress to defer packet 236 loss to egress. The basic idea of faked ECT is that, when 237 encapsulating packets, ingress first marks tunnel outer header 238 according to RFC6040, and then remarks outer header of Not-ECT packet 239 as ECT, there will be three kinds of combination of outer header ECN 240 field and inner header ECN field: CE|CE, ECT|N-ECT, ECT|ECT (in the 241 form of outer ECN| inner ECN). 243 In case all interior routers support ECN, the network congestion 244 level could be indicated through the ratio of CE-marked packet and 245 the ratio of packet drop, the relationship between these two kinds of 246 indicator is complementary. If the congestion level in tunnel is not 247 high enough, the packets would be marked as CE instead of being 248 dropped, and then it is easy to calculate congestion level according 249 to the ratio of CE-marked packets. If the congestion level is so high 250 that ECT packet will be dropped, then the packet loss ratio could be 251 calculated by comparing total packets entering ingress and total 252 packets arriving at egress over the same span of packets, if packet 253 loss is detected, it could be assumed that severe congestion has 254 occurred in the tunnel. Because loss is only ever a sign of serious 255 congestion, so it doesn't need to measure loss ratio accurately. 257 The basic procedure of congestion level measurement is as follows: 259 +-------+ +------+ 260 |Ingress| |Egress| 261 +-------+ +------+ 262 | | 263 +----------------+ | 264 |cumulative count| | 265 +----------------+ | 266 | | 267 | | 268 |------------------------>| 269 | | 270 |<------------------------| 271 | | 272 | | 274 (a) Direct model feedback procedure 276 +----------+ +-------+ +------+ 277 |Controller| |Ingress| |Egress| 278 +----------+ +-------+ +------+ 279 | | | 280 | +----------------+ | 281 | |cumulative count| | 282 | +----------------+ | 283 | | | 284 | | | 285 | |------------------------>| 286 | | | 287 | | 288 | | 289 | | 290 | | 291 |<---------------------------------------| 292 | | 293 | | 294 | | 296 (b) Centralized model feedback procedure 298 Ingress encapsulates packets and marks outer header according to 299 faked ECT as described above. Ingress cumulatively counts packets for 300 three types of ECN combination (CE|CE, ECT|N-ECT, ECT|ECT) and then 301 the ingress regularly sends cumulative packet counts message of each 302 type of ECN combination to the egress. When each message arrives, the 303 egress cumulatively counts packets coming from the ingress and adds 304 its own packet counts of each type of ECN combination (CE|CE, ECT|N- 305 ECT, CE|N-ECT, CE|ECT, ECT|ECT) to the message and either returns the 306 whole message to the ingress, or to a central controller. 308 The counting of packets can be at the granularity of the all traffic 309 from the ingress to the egress to learn about the overall congestion 310 status of the path between the ingress and the egress. The counting 311 can also be at the granularity of individual customer's traffic or a 312 specific set of flows to learn about their congestion contribution. 314 5. Congestion Information Delivery 316 As described above, the tunnel ingress needs to convey message of 317 cumulative packet counts of each type of ECN combination to tunnel 318 egress, and the tunnel egress also needs to feed the message of 319 cumulative packet counts of each type of ECN combination to the 320 ingress or central collector. This section describes how the messages 321 could be conveyed. 323 The message can travel along the same path with network data traffic, 324 referred as in band signal; or go through a different path from 325 network data traffic, referred as out of band signal. Because out of 326 band scheme needs additional separate path which might limit its 327 actual deployment, the in band scheme will be discussed here. 329 Because the message is transmitted in band, so the message packet may 330 get lost in case of network congestion. To cope with the situation 331 that the message packet gets lost, the packet counts values are sent 332 as cumulative counters. Then if a message is lost the next message 333 will recover the missing information. 335 IPFIX [RFC7011] is selected as information feedback protocol. IPFIX 336 is preferred to use SCTP as transport. SCTP allows partially reliable 337 delivery [RFC3758], which ensures the feedback message will not be 338 blocked in case of packet loss due to network congestion. 340 Ingress can do congestion management at different granularity which 341 means both the overall aggregated inner tunnel congestion level and 342 congestion level contributed by certain traffic(s) could be measured 343 for different congestion management purpose. For example, if the 344 ingress only wants to limit congestion volume caused by certain 345 traffic(s),e.g UDP-based traffic, then congestion volume for the 346 traffic will be fed back; or if the ingress do overall congestion 347 management, the aggregated congestion volume will be fed back. 349 When sending message from ingress to egress, the ingress acts as 350 IPFIX exporter and egress acts as IPFIX collector; When feedback 351 congestion level information from egress to ingress or to controller, 352 the the egress acts as IPFIX exporter and ingress or controller acts 353 as IPFIX collector. 355 The combination of congestion level measurement and congestion 356 information delivery procedure should be as following: 358 # The ingress determines template record to be used. The template 359 record can be preconfigured or determined at runtime, the content of 360 template record will be determined according to the granularity of 361 congestion management, if the ingress wants to limit congestion 362 volume contributed by specific traffic flow then the elements such as 363 source IP address, destination IP address, flow id and CE-marked 364 packet volume of the flow etc will be included in the template 365 record. 367 # Meter on ingress measures traffic volume according to template 368 record chosen and then the measurement records are sent to egress in 369 band. 371 # Meter on egress measures congestion level information according to 372 template record, the template record can be preconfigured or use the 373 template record from ingress, the content of template record should 374 be the same as template record of ingress. 376 # Exporter of egress sends measurement record together with the 377 measurement record of ingress to Controller or back to the ingress. 379 5.1 IPFIX Extentions 381 This sub-section defines a list of new IPFIX Information Elements 382 according to RFC7013 [RFC7013]. 384 5.1.1 ce-cePacketTotalCount 386 Description: The total number of incoming packets with CE|CE ECN 387 marking combination for this Flow at the Observation Point since the 388 Metering Process (re-)initialization for this Observation Point. 390 Abstract Data Type: unsigned64 392 Data Type Semantics: totalCounter 394 ElementId: TBD1 396 Statues: current 398 Units: packets 400 5.1.2 ect0-nectPacketTotalCount 401 Description: The total number of incoming packets with ECT(0)|N-ECT 402 ECN marking combination for this Flow at the Observation Point since 403 the Metering Process (re-)initialization for this Observation Point. 405 Abstract Data Type: unsigned64 407 Data Type Semantics: totalCounter 409 ElementId: TBD2 411 Statues: current 413 Units: packets 415 5.1.3 ect1-nectPacketTotalCount 417 Description: The total number of incoming packets with ECT(1)|N-ECT 418 ECN marking combination for this Flow at the Observation Point since 419 the Metering Process (re-)initialization for this Observation Point. 421 Abstract Data Type: unsigned64 423 Data Type Semantics: totalCounter 425 ElementId: TBD3 427 Statues: current 429 Units: packets 431 5.1.4 ce-nectPacketTotalCount 433 Description: The total number of incoming packets with CE|N-ECT ECN 434 marking combination for this Flow at the Observation Point since the 435 Metering Process (re-)initialization for this Observation Point. 437 Abstract Data Type: unsigned64 439 Data Type Semantics: totalCounter 441 ElementId: TBD4 443 Statues: current 445 Units: packets 447 5.1.5 ce-ect0PacketTotalCount 448 Description: The total number of incoming packets with CE|ECT(0) ECN 449 marking combination for this Flow at the Observation Point since the 450 Metering Process (re-)initialization for this Observation Point. 452 Abstract Data Type: unsigned64 454 Data Type Semantics: totalCounter 456 ElementId: TBD5 458 Statues: current 460 Units: packets 462 5.1.6 ce-ect1PacketTotalCount 464 Description: The total number of incoming packets with CE|ECT(1) ECN 465 marking combination for this Flow at the Observation Point since the 466 Metering Process (re-)initialization for this Observation Point. 468 Abstract Data Type: unsigned64 470 Data Type Semantics: totalCounter 472 ElementId: TBD6 474 Statues: current 476 Units: packets 477 5.1.7 ect0-ect0PacketTotalCount 479 Description: The total number of incoming packets with ECT(0)|ECT(0) 480 ECN marking combination for this Flow at the Observation Point since 481 the Metering Process (re-)initialization for this Observation Point. 483 Abstract Data Type: unsigned64 485 Data Type Semantics: totalCounter 487 ElementId: TBD7 489 Statues: current 491 Units: packets 493 5.1.8 ect1-ect1PacketTotalCount 495 Description: The total number of incoming packets with ECT(1)|ECT(1) 496 ECN marking combination for this Flow at the Observation Point since 497 the Metering Process (re-)initialization for this Observation Point. 499 Abstract Data Type: unsigned64 501 Data Type Semantics: totalCounter 503 ElementId: TBD8 505 Statues: current 507 Units: packets 509 6. Congestion Management 511 After tunnel ingress (or controller) receives congestion level 512 information, then congestion management actions could be taken based 513 on the information, e.g. if the congestion level is higher than a 514 predefined threshold, then action could be taken to reduce the 515 congestion level. 517 The design of network side congestion management SHOULD take host 518 side e2e congestion control mechanism into consideration, which means 519 the congestion management needs to avoid the impacts on e2e 520 congestion control. For instance, congestion management action must 521 be delayed by more than a worst-case global RTT, otherwise tunnel 522 traffic management will not give normal e2e congestion control enough 523 time to do its job, and the system could go unstable. 525 The detailed description of congestion management is out of scope of 526 this document, as examples, congestion management such as circuit 527 breaker [CB] and congestion policing [CP] could be applied. Circuit 528 breaker is an automatic mechanism to estimate congestion, and to 529 terminate flow(s) when persistent congestion is detected to prevent 530 network congestion collapse; Congestion policing is used in data 531 center to limit the amount of congestion any tenant can cause 532 according to the congestion information in the tunnels. 534 7. Security 536 This document describes the tunnel congestion calculation and 537 feedback. For feeding back congestion, security mechanisms of IPFIX 538 are expected to be sufficient. No additional security concerns are 539 expected. 541 8. IANA Considerations 543 This document defines a set of new IPFIX Information Elements 544 (IE),which need to be registered at IANA IPFIX Information Element 545 Registry. 547 ElementID: TBD1 548 Name:ce-cePacketTotalCount 549 Data Type: unsigned64 550 Data Type Semantics: totalCounter 551 Status: current 552 Description:The total number of incoming packets with CE|CE ECN 553 marking combination for this Flow at the Observation Point since the 554 Metering Process (re-)initialization for this Observation Point. 555 Units: packets 557 ElementID: TBD2 558 Name:ect0-nectPacketTotalCount 559 Data Type: unsigned64 560 Data Type Semantics: totalCounter 561 Status: current 562 Description:The total number of incoming packets with ECT(0)|N-ECT 563 ECN marking combination for this Flow at the Observation Point since 564 the Metering Process (re-)initialization for this Observation Point. 565 Units: packets 567 ElementID: TBD3 568 Name: ect1-nectPacketTotalCount 569 Data Type: unsigned64 570 Data Type Semantics: totalCounter 571 Status: current 572 Description:The total number of incoming packets with ECT(1)|N-ECT 573 ECN marking combination for this Flow at the Observation Point since 574 the Metering Process (re-)initialization for this Observation Point. 575 Units: packets 577 ElementID: TBD4 578 Name:ce-nectPacketTotalCount 579 Data Type: unsigned64 580 Data Type Semantics: totalCounter 581 Status: current 582 Description:The total number of incoming packets with CE|N-ECT ECN 583 marking combination for this Flow at the Observation Point since the 584 Metering Process (re-)initialization for this Observation Point. 585 Units: packets 587 ElementID: TBD5 588 Name:ce-ect0PacketTotalCount 589 Data Type: unsigned64 590 Data Type Semantics: totalCounter 591 Status: current 592 Description:The total number of incoming packets with CE|ECT(0) ECN 593 marking combination for this Flow at the Observation Point since the 594 Metering Process (re-)initialization for this Observation Point. 595 Units: packets 597 ElementID: TBD6 598 Name:ce-ect1PacketTotalCount 599 Data Type: unsigned64 600 Data Type Semantics: totalCounter 601 Status: current 602 Description:The total number of incoming packets with CE|ECT(1) ECN 603 marking combination for this Flow at the Observation Point since the 604 Metering Process (re-)initialization for this Observation Point. 605 Units: packets 607 ElementID: TBD7 608 Name:ect0-ect0PacketTotalCount 609 Data Type: unsigned64 610 Data Type Semantics: totalCounter 611 Status: current 612 Description:The total number of incoming packets with ECT(0)|ECT(0) 613 ECN marking combination for this Flow at the Observation Point since 614 the Metering Process (re-)initialization for this Observation Point. 615 Units: packets 617 ElementID: TBD8 618 Name:ect1-ect1PacketTotalCount 619 Data Type: unsigned64 620 Data Type Semantics: totalCounter 621 Status: current 622 Description:The total number of incoming packets with 623 ECT(1)|ECT(1)ECN marking combination for this Flow at the Observation 624 Point since the Metering Process (re-)initialization for this 625 Observation Point. 626 Units: packets 628 [TO BE REMOVED: This registration should take place at the following 629 location: http://www.iana.org/assignments/ipfix/ipfix.xhtml#ipfix- 630 information-elements] 632 9. References 634 9.1 Normative References 636 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 637 Requirement Levels", BCP 14, RFC 2119, March 1997, 638 . 640 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 641 of Explicit Congestion Notification (ECN) to IP", 642 RFC 3168, September 2001, . 645 [RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P. 646 Conrad, "Stream Control Transmission Protocol (SCTP) 647 Partial Reliability Extension", RFC 3758, May 2004, 648 . 650 [RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram 651 Congestion Control Protocol (DCCP)", RFC 4340, March 2006, 652 . 654 [RFC4960] Stewart, R., Ed., "Stream Control Transmission Protocol", 655 RFC 4960, September 2007, . 658 [RFC6040] Briscoe, B., "Tunnelling of Explicit Congestion 659 Notification", RFC 6040, November 2010, . 662 [RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, 663 "Specification of the IP Flow Information Export (IPFIX) 664 Protocol for the Exchange of Flow Information", STD 77, 665 RFC 7011, September 2013, . 668 [RFC7013] Trammell, B. and B. Claise, "Guidelines for Authors and 669 Reviewers of IP Flow Information Export (IPFIX) 670 Information Elements", BCP 184, RFC 7013, September 2013, 671 . 673 [CONEX] Matt Mathis, Bob Briscoe. "Congestion Exposure (ConEx) 674 Concepts, Abstract Mechanism and Requirements", RFC7713, 675 December 2015 677 9.2 Informative References 679 [CB] G. Fairhurst. "Network Transport Circuit Breakers", draft-ietf- 680 tsvwg-circuit-breaker-01, April 02, 2015 682 [CP] Bob Briscoe, Murari Sridharan. "Network Performance Isolation 683 in Data Centres using Congestion Policing", draft-briscoe- 684 conex-data-centre-02, February 14, 2014 686 10. Acknowledgements 687 Thanks Bob Briscoe for his insightful suggestions on the basic 688 mechanisms of congestion information collection and many other useful 689 comments. Thanks David Black for his useful technical suggestions. 690 Also, thanks Anthony Chan and John Kaippallimalil for their careful 691 reviews. 693 Authors' Addresses 695 Xinpeng Wei 696 Beiqing Rd. Z-park No.156, Haidian District, 697 Beijing, 100095, P. R. China 698 E-mail: weixinpeng@huawei.com 700 Zhu Lei 701 Beiqing Rd. Z-park No.156, Haidian District, 702 Beijing, 100095, P. R. China 703 E-mail:lei.zhu@huawei.com 705 Lingli Deng 706 Beijing, 100095, P. R. China 707 E-mail: denglingli@gmail.com