idnits 2.17.1 draft-ietf-tsvwg-circuit-breaker-11.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (December 22, 2015) is 3048 days in the past. Is this intentional? Checking references for intended status: Best Current Practice ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC1112' is defined on line 1090, but no explicit reference was found in the text -- Obsolete informational reference (is this intentional?): RFC 2309 (Obsoleted by RFC 7567) Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TSVWG Working Group G. Fairhurst 3 Internet-Draft University of Aberdeen 4 Intended status: Best Current Practice December 22, 2015 5 Expires: June 24, 2016 7 Network Transport Circuit Breakers 8 draft-ietf-tsvwg-circuit-breaker-11 10 Abstract 12 This document explains what is meant by the term "network transport 13 Circuit Breaker" (CB). It describes the need for circuit breakers 14 for network tunnels and applications when using non-congestion- 15 controlled traffic, and explains where circuit breakers are, and are 16 not, needed. It also defines requirements for building a circuit 17 breaker and the expected outcomes of using a circuit breaker within 18 the Internet. 20 Status of This Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at http://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on June 24, 2016. 37 Copyright Notice 39 Copyright (c) 2015 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (http://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 55 1.1. Types of Circuit Breaker . . . . . . . . . . . . . . . . 5 56 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 57 3. Design of a Circuit-Breaker (What makes a good circuit 58 breaker?) . . . . . . . . . . . . . . . . . . . . . . . . . . 6 59 3.1. Functional Components . . . . . . . . . . . . . . . . . . 6 60 4. Requirements for a Network Transport Circuit Breaker . . . . 9 61 5. Other network topologies . . . . . . . . . . . . . . . . . . 13 62 5.1. Use with a multicast control/routing protocol . . . . . . 13 63 5.2. Use with control protocols supporting pre-provisioned 64 capacity . . . . . . . . . . . . . . . . . . . . . . . . 14 65 5.3. Unidirectional Circuit Breakers over Controlled Paths . . 15 66 6. Examples of Circuit Breakers . . . . . . . . . . . . . . . . 15 67 6.1. A Fast-Trip Circuit Breaker . . . . . . . . . . . . . . . 15 68 6.1.1. A Fast-Trip Circuit Breaker for RTP . . . . . . . . . 16 69 6.2. A Slow-trip Circuit Breaker . . . . . . . . . . . . . . . 16 70 6.3. A Managed Circuit Breaker . . . . . . . . . . . . . . . . 17 71 6.3.1. A Managed Circuit Breaker for SAToP Pseudo-Wires . . 17 72 6.3.2. A Managed Circuit Breaker for Pseudowires (PWs) . . . 18 73 7. Examples where circuit breakers may not be needed. . . . . . 18 74 7.1. CBs over pre-provisioned Capacity . . . . . . . . . . . . 19 75 7.2. CBs with tunnels carrying Congestion-Controlled Traffic . 19 76 7.3. CBs with Uni-directional Traffic and no Control Path . . 20 77 8. Security Considerations . . . . . . . . . . . . . . . . . . . 20 78 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 79 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 21 80 11. Revision Notes . . . . . . . . . . . . . . . . . . . . . . . 21 81 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 82 12.1. Normative References . . . . . . . . . . . . . . . . . . 23 83 12.2. Informative References . . . . . . . . . . . . . . . . . 23 84 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 25 86 1. Introduction 88 The term "Circuit Breaker" originates in electricity supply, and has 89 nothing to do with network circuits or virtual circuits. In 90 electricity supply, a Circuit Breaker is intended as a protection 91 mechanism of last resort. Under normal circumstances, a Circuit 92 Breaker ought not to be triggered; it is designed to protect the 93 supply network and attached equipment when there is overload. Just 94 as people do not expect the electrical circuit-breaker (or fuse) in 95 their home to be triggered, except when there is a wiring fault or a 96 problem with an electrical appliance. 98 In networking, the Circuit Breaker (CB) principle can be used as a 99 protection mechanism of last resort to avoid persistent excessive 100 congestion impacting other flows that share network capacity. 101 Persistent congestion was a feature of the early Internet of the 102 1980s. This resulted in excess traffic starving other connection 103 from access to the Internet. It was countered by the requirement to 104 use congestion control (CC) by the Transmission Control Protocol 105 (TCP) [Jacobsen88]. These mechanisms operate in Internet hosts to 106 cause TCP connections to "back off" during congestion. The addition 107 of a congestion control to TCP (currently documented in [RFC5681] 108 ensured the stability of the Internet, because it was able to detect 109 congestion and promptly react. This worked well while TCP was by far 110 the dominant traffic in the Internet, and most TCP flows were long- 111 lived (ensuring that they could detect and respond to congestion 112 before the flows terminated). This is no longer the case, and non- 113 congestion-controlled traffic, including many applications of the 114 User Datagram Protocol (UDP) can form a significant proportion of the 115 total traffic traversing a link. The current Internet therefore 116 requires that non-congestion-controlled traffic needs to be 117 considered to avoid persistent excessive congestion. 119 A network transport Circuit Breaker is an automatic mechanism that is 120 used to continuously monitor a flow or aggregate set of flows. The 121 mechanism seeks to detect when the flow(s) experience persistent 122 excessive congestion and when this is detected to terminate (or 123 significantly reduce the rate of) the flow(s). This is a safety 124 measure to prevent starvation of network resources denying other 125 flows from access to the Internet, such measures are essential for an 126 Internet that is heterogeneous and for traffic that is hard to 127 predict in advance. Avoiding persistent excessive prevention is 128 important to reduce the potential for "Congestion Collapse" 129 [RFC2914]. 131 There are important differences between a transport circuit-breaker 132 and a congestion control method. Specifically, congestion control 133 (as implemented in TCP, SCTP, and DCCP) operates on the timescale on 134 the order of a packet round-trip-time (RTT), the time from sender to 135 destination and return. Congestion control methods are able to react 136 to a single packet loss/marking and continuously adapt to reduce the 137 transmission rate for each loss or congestion event. The goal is 138 usually to limit the maximum transmission rate to a rate that 139 reflects a fair use of the available capacity across a network path. 140 These methods typically operate on individual traffic flows (e.g., a 141 5-tuple). 143 In contrast, Circuit Breakers are recommended for non-congestion- 144 controlled Internet flows and for traffic aggregates, e.g., traffic 145 sent using a network tunnel. They operate on timescales much longer 146 than the packet RTT, and trigger under situations of abnormal 147 excessive congestion. People have been implementing what this draft 148 characterizes as circuit breakers on an ad hoc basis to protect 149 Internet traffic, this draft therefore provides guidance on how to 150 deploy and use these mechanisms. Later sections provide examples of 151 cases where circuit-breakers may or may not be desirable. 153 A Circuit Breaker needs to measure (meter) the traffic to determine 154 if the network is experiencing congestion and needs to be designed to 155 trigger robustly when there is persistent excessive congestion. 157 A Circuit Breaker trigger will often utilize a series of successive 158 sample measurements metered at an ingress point and an egress point 159 (either of which could be a transport endpoint). The trigger needs 160 to operate on a timescale much longer than the path round trip time 161 (e.g., seconds to possibly many tens of seconds). This longer period 162 is needed to provide sufficient time for transports (or applications) 163 to adjust their rate following congestion, and for the network load 164 to stabilize after any adjustment. This is to ensure that a Circuit 165 Breaker does not accidentally trigger following a single (or even 166 successive) congestion events (congestion events are what triggers 167 congestion control, and are to be regarded as normal on a network 168 link operating near its capacity). Once triggered, a control 169 function needs to remove traffic from the network, either by 170 disabling the flow or by significantly reducing the level of traffic. 171 This reaction provides the required protection to prevent persistent 172 excessive congestion being experienced by other flows that share the 173 congested part of the network path. 175 Section 4 defines requirements for building a Circuit Breaker. 177 The operational conditions that cause a Circuit Breaker to trigger 178 should be regarded as abnormal. Examples of situations that could 179 trigger a Circuit Breaker include: 181 o anomalous traffic that exceeds the provisioned capacity (or whose 182 traffic characteristics exceed the threshold configured for the 183 Circuit Breaker); 185 o traffic generated by an application at a time when the provisioned 186 network capacity is being utilised for other purposes; 188 o routing changes that cause additional traffic to start using the 189 path monitored by the Circuit Breaker; 191 o misconfiguration of a service/network device where the capacity 192 available is insufficient to support the current traffic 193 aggregate; 195 o misconfiguration of an admission controller or traffic policer 196 that allows more traffic than expected across the path monitored 197 by the Circuit Breaker. 199 In many cases the reason for triggering a Circuit Breaker will not be 200 evident to the source of the traffic (user, application, endpoint, 201 etc). In contrast, an application that uses congestion control will 202 generate elastic traffic that may be expected to regulate the load it 203 introduces under congestion. This will therefore often be a 204 preferred solution for applications that can respond to congestion 205 signals or that can use a congestion-controlled transport. 207 A Circuit Breaker can be used to limit traffic from applications that 208 are unable, or choose not, to use congestion control, or where the 209 congestion control properties of their traffic cannot be relied upon 210 (e.g., traffic carried over a network tunnel). In such 211 circumstances, it is all but impossible for the Circuit Breaker to 212 signal back to the impacted applications, and it may further be the 213 case that applications may have some difficulty determining that a 214 Circuit Breaker has in fact been tripped, and where in the network 215 this happened. Application developers are advised to avoid these 216 circumstances, where possible, by deploying appropriate congestion 217 control mechanisms. 219 1.1. Types of Circuit Breaker 221 There are various forms of network transport circuit breaker. These 222 are differentiated mainly on the timescale over which they are 223 triggered, but also in the intended protection they offer: 225 o Fast-Trip Circuit Breakers: The relatively short timescale used by 226 this form of circuit breaker is intended to provide protection for 227 network traffic from a single flow or related group of flows. 229 o Slow-Trip Circuit Breakers: This circuit breaker utilizes a longer 230 timescale and is designed to protect network traffic from 231 congestion by traffic aggregates. 233 o Managed Circuit Breakers: Utilize the operations and management 234 functions that might be present in a managed service to implement 235 a circuit breaker. 237 Examples of each type of circuit breaker are provided in section 4. 239 2. Terminology 241 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 242 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 243 document are to be interpreted as described in [RFC2119]. 245 3. Design of a Circuit-Breaker (What makes a good circuit breaker?) 247 Although circuit breakers have been talked about in the IETF for many 248 years, there has not yet been guidance on the cases where circuit 249 breakers are needed or upon the design of circuit breaker mechanisms. 250 This document seeks to offer advice on these two topics. 252 Circuit Breakers are RECOMMENDED for IETF protocols and tunnels that 253 carry non-congestion-controlled Internet flows and for traffic 254 aggregates. This includes traffic sent using a network tunnel. 255 Designers of other protocols and tunnel encapsulations also ought to 256 consider the use of these techniques as a last resort to protect 257 traffic that shares the network path being used. 259 This document defines the requirements for design of a Circuit 260 Breaker and provides examples of how a Circuit Breaker can be 261 constructed. The specifications of individual protocols and tunnel 262 encapsulations need to detail the protocol mechanisms needed to 263 implement a Circuit Breaker. 265 Section 3.1 describes the functional components of a circuit breaker 266 and section 3.2 defines requirements for implementing a Circuit 267 Breaker. 269 3.1. Functional Components 271 The basic design of a transport circuit breaker involves 272 communication between an ingress point (a sender) and an egress point 273 (a receiver) of a network flow or set of flows. A simple picture of 274 Circuit Breaker operation is provided in figure 1. This shows a set 275 of routers (each labelled R) connecting a set of endpoints. 277 A Circuit Breaker is used to control traffic passing through a subset 278 of these routers, acting between the ingress and a egress point 279 network devices. The path between the ingress and egress could be 280 provided by a tunnel or other network-layer technique. One expected 281 use would be at the ingress and egress of a service, where all 282 traffic being considered terminates beyond the egress point, and 283 hence the ingress and egress carry the same set of flows. 285 +--------+ +--------+ 286 |Endpoint| |Endpoint| 287 +--+-----+ >>> circuit breaker traffic >>> +--+-----+ 288 | | 289 | +-+ +-+ +---------+ +-+ +-+ +-+ +--------+ +-+ +-+ | 290 +-+R+--+R+->+ Ingress +--+R+--+R+--+R+--+ Egress |--+R+--+R+-+ 291 +++ +-+ +------+--+ +-+ +-+ +-+ +-----+--+ +++ +-+ 292 | ^ | | | 293 | | +--+------+ +------+--+ | 294 | | | Ingress | | Egress | | 295 | | | Meter | | Meter | | 296 | | +----+----+ +----+----+ | 297 | | | | | 298 +-+ | | +----+----+ | | +-+ 299 |R+--+ | | Measure +<----------------+ +--+R| 300 +++ | +----+----+ Reported +++ 301 | | | Egress | 302 | | +----+----+ Measurement | 303 +--+-----+ | | Trigger + +--+-----+ 304 |Endpoint| | +----+----+ |Endpoint| 305 +--------+ | | +--------+ 306 +---<---+ 307 Reaction 309 Figure 1: A CB controlling the part of the end-to-end path between an 310 ingress point and an egress point. (Note: In some cases, the trigger 311 and measure functions could alternatively be located at other 312 locations (e.g., at a network operations centre.) 314 In the context of a Circuit Breaker, the ingress and egress functions 315 could be implemented in different places. For example, they could be 316 located in network devices at a tunnel ingress and at the tunnel 317 egress. In some cases, they could be located at one or both network 318 endpoints (see figure 2), implemented as components within a 319 transport protocol. 321 +----------+ +----------+ 322 | Ingress | +-+ +-+ +-+ | Egress | 323 | Endpoint +->+R+--+R+--+R+--+ Endpoint | 324 +--+----+--+ +-+ +-+ +-+ +----+-----+ 325 ^ | | 326 | +--+------+ +----+----+ 327 | | Ingress | | Egress | 328 | | Meter | | Meter | 329 | +----+----+ +----+----+ 330 | | | 331 | +--- +----+ | 332 | | Measure +<-----------------+ 333 | +----+----+ Reported 334 | | Egress 335 | +----+----+ Measurement 336 | | Trigger | 337 | +----+----+ 338 | | 339 +---<--+ 340 Reaction 342 Figure 2: An endpoint CB implemented at the sender (ingress) and 343 receiver (egress). 345 The set of components needed to implement a Circuit Breaker are: 347 1. An ingress meter (at the sender or tunnel ingress) records the 348 number of packets/bytes sent in each measurement interval. This 349 measures the offered network load for a flow or set of flows. 350 For example, the measurement interval could be many seconds (or 351 every few tens of seconds or a series of successive shorter 352 measurements that are combined by the Circuit Breaker Measurement 353 function). 355 2. An egress meter (at the receiver or tunnel egress) records the 356 number/bytes received in each measurement interval. This 357 measures the supported load for the flow or set of flows, and 358 could utilize other signals to detect the effect of congestion 359 (e.g., loss/marking experienced over the path). The measurements 360 at the egress could be synchronised (including an offset for the 361 time of flight of the data, or referencing the measurements to a 362 particular packet) to ensure any counters refer to the same span 363 of packets. 365 3. The measured values at the ingress and egress are communicated to 366 the Circuit Breaker Measurement function. This could use several 367 methods including: Sending return measurement packets from a 368 receiver to a trigger function at the sender; An implementation 369 using Operations, Administration and Management (OAM); or be 370 sending another in-band signalling datagram to the trigger 371 function. This could also be implemented purely as a control 372 plane function, e.g., using a software-defined network 373 controller. 375 4. The measurement function combines the ingress and egress 376 measurements to assess the present level of network congestion. 377 (For example, the loss rate for each measurement interval could 378 be deduced from calculating the difference between ingress and 379 egress counter values.) Note the method does not require high 380 accuracy for the period of the measurement interval (or therefore 381 the measured value, since isolated and/or infrequent loss events 382 need to be disregarded.) 384 5. A trigger function determines whether the measurements indicate 385 persistent excessive congestion. This function defines an 386 appropriate threshold for determining that there is persistent 387 excessive congestion between the ingress and egress. This 388 preferably considers a rate or ratio, rather than an absolute 389 value (e.g., more than 10% loss, but other methods could also be 390 based on the rate of transmission as well as the loss rate). The 391 transport Circuit Breaker is triggered when the threshold is 392 exceeded in multiple measurement intervals (e.g., 3 successive 393 measurements). Designs need to be robust so that single or 394 spurious events do not trigger a reaction. 396 6. A reaction that is applied at the Ingress when the Circuit 397 Breaker is triggered. This seeks to automatically remove the 398 traffic causing persistent excessive congestion. 400 7. A feedback mechanism that triggers when either the receive or 401 ingress and egress measurements are not available, since this 402 also could indicate a loss of control packets (also a symptom of 403 heavy congestion or inability to control the load). 405 4. Requirements for a Network Transport Circuit Breaker 407 The requirements for implementing a Circuit Breaker are: 409 o There needs to be a communication path used for control messages 410 from the ingress meter and the egress meter to the point of 411 measurement. The Circuit Breaker MUST trigger if there is a 412 failure of the communication path used for the control messages. 413 That is, the feedback indicating a congested period needs to be 414 designed so that the Circuit Breaker is triggered when it fails to 415 receive measurement reports that indicate an absence of 416 congestion, rather than relying on the successful transmission of 417 a "congested" signal back to the sender. (The feedback signal 418 could itself be lost under congestion). 420 o A Circuit Breaker is REQUIRED to define a measurement period over 421 which the Circuit Breaker Measurement function measures the level 422 of congestion or loss. This method does not have to detect 423 individual packet loss, but MUST have a way to know that packets 424 have been lost/marked from the traffic flow. 426 o An egress meter can also count Explicit Congestion Notification 427 (ECN) [RFC3168] congestion marks as a part of measurement of 428 congestion, but in this case, loss MUST also be measured to 429 provide a complete view of the level of congestion. For tunnels, 430 [ID-ietf-tsvwg-tunnel-congestion-feedback] describes a way to 431 measure both loss and ECN-marking; these measurements could be 432 used on a relatively short timescale to drive a congestion control 433 response and/or aggregated over a longer timescale with a higher 434 trigger threshold to drive a Circuit Breaker. Subsequent bullet 435 items in this section discuss the necessity of using a longer 436 timescale and a higher trigger threshold. 438 o The measurement period used by a Circuit Breaker Measurement 439 function MUST be longer than the time that current Congestion 440 Control algorithms need to reduce their rate following detection 441 of congestion. This is important because end-to-end Congestion 442 Control algorithms require at least one RTT to notify and adjust 443 the traffic to experienced congestion, and congestion bottlenecks 444 can share traffic with a diverse range of RTTs. The measurement 445 period is therefore expected to be significantly longer than the 446 RTT experienced by the Circuit Breaker itself. 448 o If necessary, MAY combine successive individual meter samples from 449 the ingress and egress to ensure observation of an average over a 450 sufficiently long interval. (Note when meter samples need to be 451 combined, the combination needs to reflect the sum of the 452 individual sample counts divided by the total time/volume over 453 which the samples were measured. Individual samples over 454 different intervals can not be directly combined to generate an 455 average value.) 457 o A Circuit Breaker is REQUIRED to define a threshold to determine 458 whether the measured congestion is considered excessive. 460 o A Circuit Breaker is REQUIRED to define the triggering interval, 461 defining the period over which the trigger uses the collected 462 measurements. Circuit Breakers need to trigger over a 463 sufficiently long period to avoid additionally penalizing flows 464 with a long path RTT (e.g., many path RTTs). 466 o A Circuit Breaker MUST be robust to multiple congestion events. 467 This usually will define a number of measured persistent 468 congestion events per triggering period. For example, a Circuit 469 Breaker MAY combine the results of several measurement periods to 470 determine if the Circuit Breaker is triggered. (e.g., triggered 471 when persistent excessive congestion is detected in 3 of the 472 measurements within the triggering interval). 474 o A Circuit Breaker SHOULD be constructed so that it does not 475 trigger under light or intermittent congestion. 477 o The default response to a trigger SHOULD disable all traffic that 478 contributed to congestion. 480 o Once triggered, the Circuit Breaker MUST react decisively by 481 disabling or significantly reducing traffic at the source (e.g., 482 ingress). 484 o The reaction needs to be much more severe than that of a 485 Congestion Control algorithm (such as TCP's congestion control 486 [RFC5681] or TCP-Friendly Rate Control, TFRC [RFC5348]), because 487 the Circuit Breaker reacts to more persistent congestion and 488 operates over longer timescales (i.e., the overload condition will 489 have persisted for a longer time before the Circuit Breaker is 490 triggered). 492 o A reaction that results in a reduction SHOULD result in reducing 493 the traffic by at least an order of magnitude. A response that 494 achieves the reduction by terminating flows, rather than randomly 495 dropping packets, will often be more desirable to users of the 496 service. A Circuit Breaker that reduces the rate of a flow, MUST 497 continue to monitor the level of congestion and MUST further react 498 to reduce the rate if the Circuit Breaker is again triggered. 500 o The reaction to a triggered Circuit Breaker MUST continue for a 501 period that is at least the triggering interval. Operator 502 intervention will usually be required to restore a flow. If an 503 automated response is needed to reset the trigger, then this needs 504 to not be immediate. The design of an automated reset mechanism 505 needs to be sufficiently conservative that it does not adversely 506 interact with other mechanisms (including other Circuit Breaker 507 algorithms that control traffic over a common path). It SHOULD 508 NOT perform an automated reset when there is evidence of continued 509 congestion. 511 o When a Circuit Breaker is triggered, it SHOULD be regarded as an 512 abnormal network event. As such, this event SHOULD be logged. 513 The measurements that lead to triggering of the Circuit Breaker 514 SHOULD also be logged. 516 o A Circuit Breaker requires control communication between endpoints 517 and/or network devices. The source and integrity of control 518 messages (measurements and triggers) MUST be protected from off- 519 path attacks (Section 8). When there is a risk of on-path attack, 520 a cryptographic authentication mechanism for all control/ 521 measurement messages is RECOMMENDED (Section 8). 523 o The circuit breaker MUST be designed to be robust to packet loss 524 that can also be experienced during congestion/overload. This 525 does not imply that it is desirable to provide reliable delivery 526 (e.g., over TCP), since this can incur additional delay in 527 responding to congestion. Appropriate mechanisms could be to 528 duplicate control messages to provide increased robustness to 529 loss, or/and to regard a lack of control traffic as an indication 530 that excessive congestion may be being experienced 531 [ID-ietf-tsvwg-RFC5405.bis]. 533 o The control communication may be in-band or out-of-band. In-band 534 communication is RECOMMENDED when either design would be possible. 535 If this traffic is sent over a shared path, it is RECOMMENDED that 536 this control traffic is prioritized to reduce the probability of 537 loss under congestion. Control traffic also needs to be 538 considered when provisioning a network that uses a circuit 539 breaker. 541 in-Band: An in-band control method SHOULD assume that loss of 542 control messages is an indication of potential congestion on 543 the path, and repeated loss ought to cause the Circuit Breaker 544 to be triggered. This design has the advantage that it 545 provides fate-sharing of the traffic flow(s) and the control 546 communications. 548 Out-of-Band: An out-of-band control method SHOULD NOT trigger 549 Circuit Breaker reaction when there is loss of control messages 550 (e.g., a loss of measurements). This avoids failure 551 amplification/propagation when the measurement and data paths 552 fail independently. A failure of an out-of-band communication 553 path SHOULD be regarded as abnormal network event and be 554 handled as appropriate for the network, e.g., this event SHOULD 555 be logged, and additional network operator action might be 556 appropriate, depending on the network and the traffic involved. 558 5. Other network topologies 560 A Circuit Breaker can be deployed in networks with topologies 561 different to that presented in figure 2. This section describes 562 examples of such usage, and possible places where functions may be 563 implemented. 565 5.1. Use with a multicast control/routing protocol 567 +----------+ +--------+ +----------+ 568 | Ingress | +-+ +-+ +-+ | Egress | | Egress | 569 | Endpoint +->+R+--+R+--+R+--+ Router |--+ Endpoint +->+ 570 +----+-----+ +-+ +-+ +-+ +---+--+-+ +----+-----+ | 571 ^ ^ ^ ^ | ^ | | 572 | | | | | | | | 573 +----+----+ + - - - < - - - - + | +----+----+ | Reported 574 | Ingress | multicast Prune | | Egress | | Ingress 575 | Meter | | | Meter | | Measurement 576 +---------+ | +----+----+ | 577 | | | 578 | +----+----+ | 579 | | Measure +<--+ 580 | +----+----+ 581 | | 582 | +----+----+ 583 multicast | | Trigger | 584 Leave | +----+----+ 585 Message | | 586 +----<----+ 588 Figure 3: An example of a multicast CB controlling the end-to-end 589 path between an ingress endpoint and an egress endpoint. 591 Figure 3 shows one example of how a multicast circuit breaker could 592 be implemented at a pair of multicast endpoints (e.g., to implement a 593 Fast-Trip Circuit Breaker, Section 6.1). The ingress endpoint (the 594 sender that sources the multicast traffic) meters the ingress load, 595 generating an ingress measurement (e.g., recording timestamped packet 596 counts), and sends this measurement to the multicast group together 597 with the traffic it has measured. 599 Routers along a multicast path forward the multicast traffic 600 (including the ingress measurement) to all active endpoint receivers. 601 Each last hop (egress) router forwards the traffic to one or more 602 egress endpoint(s). 604 In this figure, each endpoint includes a meter that performs a local 605 egress load measurement. An endpoint also extracts the received 606 ingress measurement from the traffic, and compares the ingress and 607 egress measurements to determine if the Circuit Breaker ought to be 608 triggered. This measurement has to be robust to loss (see previous 609 section). If the Circuit Breaker is triggered, it generates a 610 multicast leave message for the egress (e.g., an IGMP or MLD message 611 sent to the last hop router), which causes the upstream router to 612 cease forwarding traffic to the egress endpoint. 614 Any multicast router that has no active receivers for a particular 615 multicast group will prune traffic for that group, sending a prune 616 message to its upstream router. This starts the process of releasing 617 the capacity used by the traffic and is a standard multicast routing 618 function (e.g., using the PIM-SM routing protocol). Each egress 619 operates autonomously, and the circuit breaker "reaction" is executed 620 by the multicast control plane (e.g., by the PIM multicast routing 621 protocol), requiring no explicit signalling by the circuit breaker 622 along the communication path used for the control messages. Note: 623 there is no direct communication with the Ingress, and hence a 624 triggered Circuit Breaker only controls traffic downstream of the 625 first hop router. It does not stop traffic flowing from the sender 626 to the first hop router; this is however the common practice for 627 multicast deployment. 629 The method could also be used with a multicast tunnel or subnetwork 630 (e.g., Section 6.2, Section 6.3), where a meter at the ingress 631 generates additional control messages to carry the measurement data 632 towards the egress where the egress metering is implemented. 634 5.2. Use with control protocols supporting pre-provisioned capacity 636 Some paths are provisioned using a control protocol, e.g., flows 637 provisioned using the Multi-Protocol Label Switching (MPLS) services, 638 path provisioned using the Resource reservation protocol (RSVP), 639 networks utilizing Software Defined Network (SDN) functions, or 640 admission-controlled Differentiated Services. 642 Figure 1 shows one expected use case, where in this usage a separate 643 device could be used to perform the measurement and trigger 644 functions. The reaction generated by the trigger could take the form 645 of a network control message sent to the ingress and/or other network 646 elements causing these elements to react to the Circuit Breaker. 647 Examples of this type of use are provided in section Section 6.3. 649 5.3. Unidirectional Circuit Breakers over Controlled Paths 651 A Circuit Breaker can be used to control uni-directional UDP traffic, 652 providing that there is a communication path that can be used for 653 control messages to connect the functional components at the Ingress 654 and Egress. This communication path for the control messages can 655 exist in networks for which the traffic flow is purely 656 unidirectional. For example, a multicast stream that sends packets 657 across an Internet path and can use multicast routing to prune flows 658 to shed network load. Some other types of subnetwork also utilize 659 control protocols that can be used to control traffic flows. 661 6. Examples of Circuit Breakers 663 There are multiple types of Circuit Breaker that could be defined for 664 use in different deployment cases. This section provides examples of 665 different types of circuit breaker: 667 6.1. A Fast-Trip Circuit Breaker 669 [RFC2309] discusses the dangers of congestion-unresponsive flows and 670 states that "all UDP-based streaming applications should incorporate 671 effective congestion avoidance mechanisms". All applications ought 672 to use a full-featured transport (TCP, SCTP, DCCP), and if not, an 673 application (e.g., those using UDP and its UDP-Lite variant) needs to 674 provide appropriate congestion avoidance. Guidance for applications 675 that do not use congestion-controlled transports is provided in 676 [ID-ietf-tsvwg-RFC5405.bis]. Such mechanisms can be designed to 677 react on much shorter timescales than a circuit breaker, that only 678 observes a traffic envelope. Congestion control methods can also 679 interact with an application to more effectively control its sending 680 rate. 682 A fast-trip circuit breaker is the most responsive form of Circuit 683 Breaker. It has a response time that is only slightly larger than 684 that of the traffic that it controls. It is suited to traffic with 685 well-understood characteristics (and could include one or more 686 trigger functions specifically tailored the type of traffic for which 687 it is designed). It is not suited to arbitrary network traffic and 688 may be unsuitable for traffic aggregates, since it could prematurely 689 trigger (e.g., when multiple congestion-controlled flows lead to 690 short-term overload). 692 Although the mechanisms can be implemented in a RTP-aware network 693 devices, these mechanisms are also suitable for implementation in 694 endpoints (e.g., as a part of the tranport system), where they can 695 also compliment end-to-end congestion control methods. A shorter 696 response time enables these mechanisms to triggers before other forms 697 of circuit breaker (e.g., circuit breakers operating on traffic 698 aggregates at a point along the network path). 700 6.1.1. A Fast-Trip Circuit Breaker for RTP 702 A set of fast-trip Circuit Breaker methods have been specified for 703 use together by a Real-time Transport Protocol (RTP) flow using the 704 RTP/AVP Profile [RTP-CB]. It is expected that, in the absence of 705 severe congestion, all RTP applications running on best-effort IP 706 networks will be able to run without triggering these circuit 707 breakers. A fast-trip RTP Circuit Breaker is therefore implemented 708 as a fail-safe that when triggered will terminate RTP traffic. 710 The sending endpoint monitors reception of in-band RTP Control 711 Protocol (RTCP) reception report blocks, as contained in SR or RR 712 packets, that convey reception quality feedback information. This is 713 used to measure (congestion) loss, possibly in combination with ECN 714 [RFC6679]. 716 The Circuit Breaker action (shutdown of the flow) is triggered when 717 any of the following trigger conditions are true: 719 1. An RTP Circuit Breaker triggers on reported lack of progress. 721 2. An RTP Circuit Breaker triggers when no receiver reports messages 722 are received. 724 3. An RTP Circuit Breaker uses a TFRC-style check and sets a hard 725 upper limit to the long-term RTP throughput (over many RTTs). 727 4. An RTP Circuit Breaker includes the notion of Media Usability. 728 This circuit breaker is triggered when the quality of the 729 transported media falls below some required minimum acceptable 730 quality. 732 6.2. A Slow-trip Circuit Breaker 734 A slow-trip Circuit Breaker could be implemented in an endpoint or 735 network device. This type of Circuit Breaker is much slower at 736 responding to congestion than a fast-trip Circuit Breaker and is 737 expected to be more common. 739 One example where a slow-trip Circuit Breaker is needed is where 740 flows or traffic-aggregates use a tunnel or encapsulation and the 741 flows within the tunnel do not all support TCP-style congestion 742 control (e.g., TCP, SCTP, TFRC), see [ID-ietf-tsvwg-RFC5405.bis] 743 section 3.1.3. A use case is where tunnels are deployed in the 744 general Internet (rather than "controlled environments" within an 745 Internet service provider or enterprise network), especially when the 746 tunnel could need to cross a customer access router. 748 6.3. A Managed Circuit Breaker 750 A managed Circuit Breaker is implemented in the signalling protocol 751 or management plane that relates to the traffic aggregate being 752 controlled. This type of circuit breaker is typically applicable 753 when the deployment is within a "controlled environment". 755 A Circuit Breaker requires more than the ability to determine that a 756 network path is forwarding data, or to measure the rate of a path - 757 which are often normal network operational functions. There is an 758 additional need to determine a metric for congestion on the path and 759 to trigger a reaction when a threshold is crossed that indicates 760 persistent excessive congestion. 762 The control messages can use either in-band or out-of-band 763 communications. 765 6.3.1. A Managed Circuit Breaker for SAToP Pseudo-Wires 767 [RFC4553], SAToP Pseudo-Wires (PWE3), section 8 describes an example 768 of a managed circuit breaker for isochronous flows. 770 If such flows were to run over a pre-provisioned (e.g., Multi- 771 Protocol Label Switching, MPLS) infrastructure, then it could be 772 expected that the Pseudowire (PW) would not experience congestion, 773 because a flow is not expected to either increase (or decrease) their 774 rate. If instead Pseudo-Wire traffic is multiplexed with other 775 traffic over the general Internet, it could experience congestion. 776 [RFC4553] states: "If SAToP PWs run over a PSN providing best-effort 777 service, they SHOULD monitor packet loss in order to detect "severe 778 congestion". The currently recommended measurement period is 1 779 second, and the trigger operates when there are more than three 780 measured Severely Errored Seconds (SES) within a period. If such a 781 condition is detected, a SAToP PW ought to shut down bidirectionally 782 for some period of time...". 784 The concept was that when the packet loss ratio (congestion) level 785 increased above a threshold, the PW was by default disabled. This 786 use case considered fixed-rate transmission, where the PW had no 787 reasonable way to shed load. 789 The trigger needs to be set at the rate that the PW was likely to 790 experience a serious problem, possibly making the service non- 791 compliant. At this point, triggering the Circuit Breaker would 792 remove the traffic preventing undue impact on congestion-responsive 793 traffic (e.g., TCP). Part of the rationale, was that high loss 794 ratios typically indicated that something was "broken" and ought to 795 have already resulted in operator intervention, and therefore need to 796 trigger this intervention. 798 An operator-based response provides opportunity for other action to 799 restore the service quality, e.g., by shedding other loads or 800 assigning additional capacity, or to consciously avoid reacting to 801 the trigger while engineering a solution to the problem. This could 802 require the trigger to be sent to a third location (e.g., a network 803 operations centre, NOC) responsible for operation of the tunnel 804 ingress, rather than the tunnel ingress itself. 806 6.3.2. A Managed Circuit Breaker for Pseudowires (PWs) 808 Pseudowires (PWs) [RFC3985] have become a common mechanism for 809 tunneling traffic, and may compete for network resources both with 810 other PWs and with non-PW traffic, such as TCP/IP flows. 812 [ID-ietf-pals-congcons] discusses congestion conditions that can 813 arise when PWs compete with elastic (i.e., congestion responsive) 814 network traffic (e.g, TCP traffic). Elastic PWs carrying IP traffic 815 (see [RFC4488]) do not raise major concerns because all of the 816 traffic involved responds, reducing the transmission rate when 817 network congestion is detected. 819 In contrast, inelastic PWs (e.g., a fixed bandwidth Time Division 820 Multiplex, TDM) [RFC4553] [RFC5086] [RFC5087]) have the potential to 821 harm congestion responsive traffic or to contribute to excessive 822 congestion because inelastic PWs do not adjust their transmission 823 rate in response to congestion. [ID-ietf-pals-congcons] analyses TDM 824 PWs, with an initial conclusion that a TDM PW operating with a degree 825 of loss that may result in congestion-related problems is also 826 operating with a degree of loss that results in an unacceptable TDM 827 service. For that reason, the draft suggests that a managed circuit 828 breaker that shuts down a PW when it persistently fails to deliver 829 acceptable TDM service is a useful means for addressing these 830 congestion concerns. 832 7. Examples where circuit breakers may not be needed. 834 A Circuit Breaker is not required for a single congestion-controlled 835 flow using TCP, SCTP, TFRC, etc. In these cases, the congestion 836 control methods are already designed to prevent persistent excessive 837 congestion. 839 7.1. CBs over pre-provisioned Capacity 841 One common question is whether a Circuit Breaker is needed when a 842 tunnel is deployed in a private network with pre-provisioned 843 capacity. 845 In this case, compliant traffic that does not exceed the provisioned 846 capacity ought not to result in persistent congestion. A Circuit 847 Breaker will hence only be triggered when there is non-compliant 848 traffic. It could be argued that this event ought never to happen - 849 but it could also be argued that the Circuit Breaker equally ought 850 never to be triggered. If a Circuit Breaker were to be implemented, 851 it will provide an appropriate response if persistent congestion 852 occurs in an operational network. 854 Implementing a Circuit Breaker will not reduce the performance of the 855 flows, but in the event that persistent excessive congestion occurs 856 it protects network traffic that shares network capacity with these 857 flows. A Circuit Breaker also could be used to protect other sharing 858 network traffic from a failure that causes the Circuit Breaker 859 traffic to be routed over a non-pre-provisioned path. 861 7.2. CBs with tunnels carrying Congestion-Controlled Traffic 863 IP-based traffic is generally assumed to be congestion-controlled, 864 i.e., it is assumed that the transport protocols generating IP-based 865 traffic at the sender already employ mechanisms that are sufficient 866 to address congestion on the path [ID-ietf-tsvwg-RFC5405.bis]. A 867 question therefore arises when people deploy a tunnel that is thought 868 to only carry an aggregate of TCP traffic (or traffic using some 869 other congestion control method): Is there advantage in this case in 870 using a Circuit Breaker? 872 For sure, traffic in a such a tunnel will respond to congestion. 873 However, the answer to the question is not always obvious, because 874 the overall traffic formed by an aggregate of flows that implement a 875 congestion control mechanism does not necessarily prevent persistent 876 congestion. For instance, most congestion control mechanisms require 877 long-lived flows to react to reduce the rate of a flow, an aggregate 878 of many short flows could result in many terminating before they 879 experience congestion. It is also often impossible for a tunnel 880 service provider to know that the tunnel only contains congestion- 881 controlled traffic (e.g., Inspecting packet headers could not be 882 possible). The important thing to note is that if the aggregate of 883 the traffic does not result in persistent excessive congestion 884 (impacting other flows), then the Circuit Breaker will not trigger. 885 This is the expected case in this context - so implementing a Circuit 886 Breaker will not reduce performance of the tunnel, but in the event 887 that persistent excessive congestion occurs this protects other 888 network traffic that shares capacity with the tunnel traffic. 890 7.3. CBs with Uni-directional Traffic and no Control Path 892 A one-way forwarding path could have no associated communication path 893 for sending control messages, and therefore cannot be controlled 894 using an automated process. This service could be provided using a 895 path that has dedicated capacity and does not share this capacity 896 with other elastic Internet flows (i.e., flows that vary their rate). 898 A way to mitigate the impact on other flows when capacity could be 899 shared is to manage the traffic envelope by using ingress policing. 901 Supporting this type of traffic in the general Internet requires 902 operator monitoring to detect and respond to persistent excessive 903 congestion. 905 8. Security Considerations 907 All Circuit Breaker mechanisms rely upon coordination between the 908 ingress and egress meters and communication with the trigger 909 function. This is usually achieved by passing network control 910 information (or protocol messages) across the network. Timely 911 operation of a circuit breaker depends on the choice of measurement 912 period. If the receiver has an interval that is overly long, then 913 the responsiveness of the circuit breaker decreases. This impacts 914 the ability of the circuit breaker to detect and react to congestion. 916 A Circuit Breaker could potentially be exploited by an attacker to 917 mount a Denial of Service (DoS) attack against the traffic being 918 measured. Mechanisms therefore need to be implemented to prevent 919 attacks on the network control information that would result in DoS. 920 The source and integrity of control information (measurements and 921 triggers) MUST be protected from off-path attacks. Without 922 protection, it could be trivial for an attacker to inject fake or 923 modified control/measurement messages (e.g., indicating high packet 924 loss rates) causing a Circuit Breaker to trigger and to therefore 925 mount a DoS attack that disrupts a flow. 927 Simple protection can be provided by using a randomized source port, 928 or equivalent field in the packet header (such as the RTP SSRC value 929 and the RTP sequence number) expected not to be known to an off-path 930 attacker. Stronger protection can be achieved using a secure 931 authentication protocol. This attack is relatively easy for an on- 932 path attacker when the messages are neither encrypted nor 933 authenticated. When there is a risk of on-path attack, a 934 cryptographic authentication mechanism for all control/measurement 935 messages is RECOMMENDED to mitigate this concern. There is a design 936 trade-off between the cost of introducing cryptographic security for 937 control messages and the desire to protect control communication. 938 For some deployment scenarios the value of additional protection from 939 DoS attack will therefore lead to a requirement to authenticate all 940 control messages. 942 Transmission of network control messages consumes network capacity. 943 This control traffic needs to be considered in the design of a 944 Circuit Breaker and could potentially add to network congestion. If 945 this traffic is sent over a shared path, it is RECOMMENDED that this 946 control traffic is prioritized to reduce the probability of loss 947 under congestion. Control traffic also needs to be considered when 948 provisioning a network that uses a circuit breaker. 950 The circuit breaker MUST be designed to be robust to packet loss that 951 can also be experienced during congestion/overload. Loss of control 952 messages could be a side-effect of a congested network, but also 953 could arise from other causes Section 4. 955 The security implications depend on the design of the mechanisms, the 956 type of traffic being controlled and the intended deployment 957 scenario. Each design of a Circuit Breaker MUST therefore evaluate 958 whether the particular circuit breaker mechanism has new security 959 implications. 961 9. IANA Considerations 963 This document makes no request from IANA. 965 10. Acknowledgments 967 There are many people who have discussed and described the issues 968 that have motivated this draft. Contributions and comments included: 969 Lars Eggert, Colin Perkins, David Black, Matt Mathis, Andrew 970 McGregor, Bob Briscoe and Eliot Lear. This work was part-funded by 971 the European Community under its Seventh Framework Programme through 972 the Reducing Internet Transport Latency (RITE) project (ICT-317700). 974 11. Revision Notes 976 XXX RFC-Editor: Please remove this section prior to publication XXX 978 Draft 00 980 This was the first revision. Help and comments are greatly 981 appreciated. 983 Draft 01 985 Contained clarifications and changes in response to received 986 comments, plus addition of diagram and definitions. Comments are 987 welcome. 989 WG Draft 00 991 Approved as a WG work item on 28th Aug 2014. 993 WG Draft 01 995 Incorporates feedback after Dallas IETF TSVWG meeting. This version 996 is thought ready for WGLC comments. Definitions of abbreviations. 998 WG Draft 02 1000 Minor fixes for typos. Rewritten security considerations section. 1002 WG Draft 03 1004 Updates following WGLC comments (see TSV mailing list). Comments 1005 from C Perkins; D Black and off-list feedback. 1007 A clear recommendation of intended scope. 1009 Changes include: Improvement of language on timescales and minimum 1010 measurement period; clearer articulation of endpoint and multicast 1011 examples - with new diagrams; separation of the controlled network 1012 case; updated text on position of trigger function; corrections to 1013 RTP-CB text; clarification of loss v ECN metrics; checks against 1014 submission checklist 9use of keywords, added meters to diagrams). 1016 WG Draft 04 1018 Added section on PW CB for TDM - a newly adopted draft (D. Black). 1020 WG Draft 05 1022 Added clarifications requested during AD review. 1024 WG Draft 06 1026 Fixed some remaining typos. 1028 Update following detailed review by Bob Briscoe, and comments by D. 1029 Black. 1031 WG Draft 07 1033 Additional update following review by Bob Briscoe. 1035 WG Draft 08 1037 Updated text on the response to lack of meter measurements with 1038 managed circuit breakers. Additional comments from Eliot Lear (APPs 1039 area). 1041 WG Draft 09 1043 Updated text on applications from Eliot Lear. Additional feedback 1044 from Bob Briscoe. 1046 WG Draft 10 1048 Updated text following comments by D Black including a rewritten ECN 1049 requirements bullet with of a reference to a tunnel measurement 1050 method in [ID-ietf-tsvwg-tunnel-congestion-feedback]. 1052 WG Draft 11 1054 Minor corrections after second WGLC. 1056 12. References 1058 12.1. Normative References 1060 [ID-ietf-tsvwg-RFC5405.bis] 1061 Eggert, L., Fairhurst, G., and G. Shepherd, "UDP Usage 1062 Guidelines (Work-in-Progress)", 2015. 1064 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1065 Requirement Levels", BCP 14, RFC 2119, 1066 DOI 10.17487/RFC2119, March 1997, 1067 . 1069 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1070 of Explicit Congestion Notification (ECN) to IP", 1071 RFC 3168, DOI 10.17487/RFC3168, September 2001, 1072 . 1074 12.2. Informative References 1076 [ID-ietf-pals-congcons] 1077 Stein, YJ., Black, D., and B. Briscoe, "Pseudowire 1078 Congestion Considerations (Work-in-Progress)", 2015. 1080 [ID-ietf-tsvwg-tunnel-congestion-feedback] 1081 Wei, X., Zhu, L., and L. Dend, "Tunnel Congestion Feedback 1082 (Work-in-Progress)", 2015. 1084 [Jacobsen88] 1085 European Telecommunication Standards, Institute (ETSI), 1086 "Congestion Avoidance and Control", SIGCOMM Symposium 1087 proceedings on Communications architectures and 1088 protocols", August 1998. 1090 [RFC1112] Deering, S., "Host extensions for IP multicasting", STD 5, 1091 RFC 1112, DOI 10.17487/RFC1112, August 1989, 1092 . 1094 [RFC2309] Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering, 1095 S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G., 1096 Partridge, C., Peterson, L., Ramakrishnan, K., Shenker, 1097 S., Wroclawski, J., and L. Zhang, "Recommendations on 1098 Queue Management and Congestion Avoidance in the 1099 Internet", RFC 2309, DOI 10.17487/RFC2309, April 1998, 1100 . 1102 [RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, 1103 RFC 2914, DOI 10.17487/RFC2914, September 2000, 1104 . 1106 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 1107 Edge-to-Edge (PWE3) Architecture", RFC 3985, 1108 DOI 10.17487/RFC3985, March 2005, 1109 . 1111 [RFC4488] Levin, O., "Suppression of Session Initiation Protocol 1112 (SIP) REFER Method Implicit Subscription", RFC 4488, 1113 DOI 10.17487/RFC4488, May 2006, 1114 . 1116 [RFC4553] Vainshtein, A., Ed. and YJ. Stein, Ed., "Structure- 1117 Agnostic Time Division Multiplexing (TDM) over Packet 1118 (SAToP)", RFC 4553, DOI 10.17487/RFC4553, June 2006, 1119 . 1121 [RFC5086] Vainshtein, A., Ed., Sasson, I., Metz, E., Frost, T., and 1122 P. Pate, "Structure-Aware Time Division Multiplexed (TDM) 1123 Circuit Emulation Service over Packet Switched Network 1124 (CESoPSN)", RFC 5086, DOI 10.17487/RFC5086, December 2007, 1125 . 1127 [RFC5087] Stein, Y(J)., Shashoua, R., Insler, R., and M. Anavi, 1128 "Time Division Multiplexing over IP (TDMoIP)", RFC 5087, 1129 DOI 10.17487/RFC5087, December 2007, 1130 . 1132 [RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP 1133 Friendly Rate Control (TFRC): Protocol Specification", 1134 RFC 5348, DOI 10.17487/RFC5348, September 2008, 1135 . 1137 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 1138 Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, 1139 . 1141 [RFC6679] Westerlund, M., Johansson, I., Perkins, C., O'Hanlon, P., 1142 and K. Carlberg, "Explicit Congestion Notification (ECN) 1143 for RTP over UDP", RFC 6679, DOI 10.17487/RFC6679, August 1144 2012, . 1146 [RTP-CB] Perkins, C. and V. Singh, "Multimedia Congestion Control: 1147 Circuit Breakers for Unicast RTP Sessions", February 2014. 1149 Author's Address 1151 Godred Fairhurst 1152 University of Aberdeen 1153 School of Engineering 1154 Fraser Noble Building 1155 Aberdeen, Scotland AB24 3UE 1156 UK 1158 Email: gorry@erg.abdn.ac.uk 1159 URI: http://www.erg.abdn.ac.uk