idnits 2.17.1 draft-ietf-tsvwg-circuit-breaker-10.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 26, 2015) is 3066 days in the past. Is this intentional? Checking references for intended status: Best Current Practice ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC1112' is defined on line 1082, but no explicit reference was found in the text -- Obsolete informational reference (is this intentional?): RFC 2309 (Obsoleted by RFC 7567) Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TSVWG Working Group G. Fairhurst 3 Internet-Draft University of Aberdeen 4 Intended status: Best Current Practice November 26, 2015 5 Expires: May 29, 2016 7 Network Transport Circuit Breakers 8 draft-ietf-tsvwg-circuit-breaker-10 10 Abstract 12 This document explains what is meant by the term "network transport 13 Circuit Breaker" (CB). It describes the need for circuit breakers 14 for network tunnels and applications when using non-congestion- 15 controlled traffic, and explains where circuit breakers are, and are 16 not, needed. It also defines requirements for building a circuit 17 breaker and the expected outcomes of using a circuit breaker within 18 the Internet. 20 Status of This Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at http://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on May 29, 2016. 37 Copyright Notice 39 Copyright (c) 2015 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (http://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 55 1.1. Types of Circuit Breaker . . . . . . . . . . . . . . . . 5 56 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 57 3. Design of a Circuit-Breaker (What makes a good circuit 58 breaker?) . . . . . . . . . . . . . . . . . . . . . . . . . . 6 59 3.1. Functional Components . . . . . . . . . . . . . . . . . . 6 60 4. Requirements for a Network Transport Circuit Breaker . . . . 9 61 5. Other network topologies . . . . . . . . . . . . . . . . . . 13 62 5.1. Use with a multicast control/routing protocol . . . . . . 13 63 5.2. Use with control protocols supporting pre-provisioned 64 capacity . . . . . . . . . . . . . . . . . . . . . . . . 14 65 5.3. Unidirectional Circuit Breakers over Controlled Paths . . 15 66 6. Examples of Circuit Breakers . . . . . . . . . . . . . . . . 15 67 6.1. A Fast-Trip Circuit Breaker . . . . . . . . . . . . . . . 15 68 6.1.1. A Fast-Trip Circuit Breaker for RTP . . . . . . . . . 16 69 6.2. A Slow-trip Circuit Breaker . . . . . . . . . . . . . . . 16 70 6.3. A Managed Circuit Breaker . . . . . . . . . . . . . . . . 17 71 6.3.1. A Managed Circuit Breaker for SAToP Pseudo-Wires . . 17 72 6.3.2. A Managed Circuit Breaker for Pseudowires (PWs) . . . 18 73 7. Examples where circuit breakers may not be needed. . . . . . 18 74 7.1. CBs over pre-provisioned Capacity . . . . . . . . . . . . 19 75 7.2. CBs with tunnels carrying Congestion-Controlled Traffic . 19 76 7.3. CBs with Uni-directional Traffic and no Control Path . . 20 77 8. Security Considerations . . . . . . . . . . . . . . . . . . . 20 78 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 79 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 21 80 11. Revision Notes . . . . . . . . . . . . . . . . . . . . . . . 21 81 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 82 12.1. Normative References . . . . . . . . . . . . . . . . . . 23 83 12.2. Informative References . . . . . . . . . . . . . . . . . 23 84 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 25 86 1. Introduction 88 The term "Circuit Breaker" originates in electricity supply, and has 89 nothing to do with network circuits or virtual circuits. In 90 electricity supply, a Circuit Breaker is intended as a protection 91 mechanism of last resort. Under normal circumstances, a Circuit 92 Breaker ought not to be triggered; it is designed to protect the 93 supply network and attached equipment when there is overload. Just 94 as people do not expect the electrical circuit-breaker (or fuse) in 95 their home to be triggered, except when there is a wiring fault or a 96 problem with an electrical appliance. 98 In networking, the Circuit Breaker (CB) principle can be used as a 99 protection mechanism of last resort to avoid persistent congestion 100 impacting other flows that share network capacity. Persistent 101 congestion was a feature of the early Internet of the 1980s. This 102 resulted in excess traffic starving other connection from access to 103 the Internet. It was countered by the requirement to use congestion 104 control (CC) by the Transmission Control Protocol (TCP) [Jacobsen88]. 105 These mechanisms operate in Internet hosts to cause TCP connections 106 to "back off" during congestion. The addition of a congestion 107 control to TCP (currently documented in [RFC5681] ensured the 108 stability of the Internet, because it was able to detect congestion 109 and promptly react. This worked well while TCP was by far the 110 dominant traffic in the Internet, and most TCP flows were long-lived 111 (ensuring that they could detect and respond to congestion before the 112 flows terminated). This is no longer the case, and non-congestion- 113 controlled traffic, including many applications of the User Datagram 114 Protocol (UDP) can form a significant proportion of the total traffic 115 traversing a link. The current Internet therefore requires that non- 116 congestion-controlled traffic needs to be considered to avoid 117 persistent congestion. 119 A network transport Circuit Breaker is an automatic mechanism that is 120 used to continuously monitor a flow or aggregate set of flows. The 121 mechanism seeks to detect when the flow(s) experience persistent 122 congestion and when this is detected to terminate (or significantly 123 reduce the rate of) the flow(s). This is a safety measure to prevent 124 starvation of network resources denying other flows from access to 125 the Internet, such measures are essential for an Internet that is 126 heterogeneous and for traffic that is hard to predict in advance. 127 Avoiding persistent prevention is important to reduce the potential 128 for "Congestion Collapse" [RFC2914]. 130 There are important differences between a transport circuit-breaker 131 and a congestion control method. Specifically, congestion control 132 (as implemented in TCP, SCTP, and DCCP) operates on the timescale on 133 the order of a packet round-trip-time (RTT), the time from sender to 134 destination and return. Congestion control methods are able to react 135 to a single packet loss/marking and continuously adapt to reduce the 136 transmission rate for each loss or congestion event. The goal is 137 usually to limit the maximum transmission rate to a rate that 138 reflects a fair use of the available capacity across a network path. 139 These methods typically operate on individual traffic flows (e.g., a 140 5-tuple). 142 In contrast, Circuit Breakers are recommended for non-congestion- 143 controlled Internet flows and for traffic aggregates, e.g., traffic 144 sent using a network tunnel. They operate on timescales much longer 145 than the packet RTT, and trigger under situations of abnormal 146 excessive congestion. People have been implementing what this draft 147 characterizes as circuit breakers on an ad hoc basis to protect 148 Internet traffic, this draft therefore provides guidance on how to 149 deploy and use these mechanisms. Later sections provide examples of 150 cases where circuit-breakers may or may not be desirable. 152 A Circuit Breaker needs to measure (meter) the traffic to determine 153 if the network is experiencing congestion and needs to be designed to 154 trigger robustly when there is persistent congestion. 156 A Circuit Breaker trigger will often utilize a series of successive 157 sample measurements metered at an ingress point and an egress point 158 (either of which could be a transport endpoint). The trigger needs 159 to operate on a timescale much longer than the path round trip time 160 (e.g., seconds to possibly many tens of seconds). This longer period 161 is needed to provide sufficient time for transports (or applications) 162 to adjust their rate following congestion, and for the network load 163 to stabilize after any adjustment. This is to ensure that a Circuit 164 Breaker does not accidentally trigger following a single (or even 165 successive) congestion events (congestion events are what triggers 166 congestion control, and are to be regarded as normal on a network 167 link operating near its capacity). Once triggered, a control 168 function needs to remove traffic from the network, either by 169 disabling the flow or by significantly reducing the level of traffic. 170 This reaction provides the required protection to prevent persistent 171 congestion being experienced by other flows that share the congested 172 part of the network path. 174 Section 4 defines requirements for building a Circuit Breaker. 176 The operational conditions that cause a Circuit Breaker to trigger 177 should be regarded as abnormal. Examples of situations that could 178 trigger a Circuit Breaker include: 180 o anomalous traffic that exceeds the provisioned capacity (or whose 181 traffic characteristics exceed the threshold configured for the 182 Circuit Breaker); 184 o traffic generated by an application at a time when the provisioned 185 network capacity is being utilised for other purposes; 187 o routing changes that cause additional traffic to start using the 188 path monitored by the Circuit Breaker; 190 o misconfiguration of a service/network device where the capacity 191 available is insufficient to support the current traffic 192 aggregate; 194 o misconfiguration of an admission controller or traffic policer 195 that allows more traffic than expected across the path monitored 196 by the Circuit Breaker. 198 In many cases the reason for triggering a Circuit Breaker will not be 199 evident to the source of the traffic (user, application, endpoint, 200 etc). In contrast, an application that uses congestion control will 201 generate elastic traffic that may be expected to regulate the load it 202 introduces under congestion. This will therefore often be a 203 preferred solution for applications that can respond to congestion 204 signals or that can use a congestion-controlled transport. 206 A Circuit Breaker can be used to limit traffic from applications that 207 are unable, or choose not, to use congestion control, or where the 208 congestion control properties of their traffic cannot be relied upon 209 (e.g., traffic carried over a network tunnel). In such 210 circumstances, it is all but impossible for the Circuit Breaker to 211 signal back to the impacted applications, and it may further be the 212 case that applications may have some difficulty determining that a 213 Circuit Breaker has in fact been tripped, and where in the network 214 this happened. Application developers are advised to avoid these 215 circumstances, where possible, by deploying appropriate congestion 216 control mechanisms. 218 1.1. Types of Circuit Breaker 220 There are various forms of network transport circuit breaker. These 221 are differentiated mainly on the timescale over which they are 222 triggered, but also in the intended protection they offer: 224 o Fast-Trip Circuit Breakers: The relatively short timescale used by 225 this form of circuit breaker is intended to provide protection for 226 network traffic from a single flow or related group of flows. 228 o Slow-Trip Circuit Breakers: This circuit breaker utilizes a longer 229 timescale and is designed to protect network traffic from 230 congestion by traffic aggregates. 232 o Managed Circuit Breakers: Utilize the operations and management 233 functions that might be present in a managed service to implement 234 a circuit breaker. 236 Examples of each type of circuit breaker are provided in section 4. 238 2. Terminology 240 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 241 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 242 document are to be interpreted as described in [RFC2119]. 244 3. Design of a Circuit-Breaker (What makes a good circuit breaker?) 246 Although circuit breakers have been talked about in the IETF for many 247 years, there has not yet been guidance on the cases where circuit 248 breakers are needed or upon the design of circuit breaker mechanisms. 249 This document seeks to offer advice on these two topics. 251 Circuit Breakers are RECOMMENDED for IETF protocols and tunnels that 252 carry non-congestion-controlled Internet flows and for traffic 253 aggregates. This includes traffic sent using a network tunnel. 254 Designers of other protocols and tunnel encapsulations also ought to 255 consider the use of these techniques as a last resort to protect 256 traffic that shares the network path being used. 258 This document defines the requirements for design of a Circuit 259 Breaker and provides examples of how a Circuit Breaker can be 260 constructed. The specifications of individual protocols and tunnel 261 encapsulations need to detail the protocol mechanisms needed to 262 implement a Circuit Breaker. 264 Section 3.1 describes the functional components of a circuit breaker 265 and section 3.2 defines requirements for implementing a Circuit 266 Breaker. 268 3.1. Functional Components 270 The basic design of a transport circuit breaker involves 271 communication between an ingress point (a sender) and an egress point 272 (a receiver) of a network flow or set of flows. A simple picture of 273 Circuit Breaker operation is provided in figure 1. This shows a set 274 of routers (each labelled R) connecting a set of endpoints. 276 A Circuit Breaker is used to control traffic passing through a subset 277 of these routers, acting between the ingress and a egress point 278 network devices. The path between the ingress and egress could be 279 provided by a tunnel or other network-layer technique. One expected 280 use would be at the ingress and egress of a service, where all 281 traffic being considered terminates beyond the egress point, and 282 hence the ingress and egress carry the same set of flows. 284 +--------+ +--------+ 285 |Endpoint| |Endpoint| 286 +--+-----+ >>> circuit breaker traffic >>> +--+-----+ 287 | | 288 | +-+ +-+ +---------+ +-+ +-+ +-+ +--------+ +-+ +-+ | 289 +-+R+--+R+->+ Ingress +--+R+--+R+--+R+--+ Egress |--+R+--+R+-+ 290 +++ +-+ +------+--+ +-+ +-+ +-+ +-----+--+ +++ +-+ 291 | ^ | | | 292 | | +--+------+ +------+--+ | 293 | | | Ingress | | Egress | | 294 | | | Meter | | Meter | | 295 | | +----+----+ +----+----+ | 296 | | | | | 297 +-+ | | +----+----+ | | +-+ 298 |R+--+ | | Measure +<----------------+ +--+R| 299 +++ | +----+----+ Reported +++ 300 | | | Egress | 301 | | +----+----+ Measurement | 302 +--+-----+ | | Trigger + +--+-----+ 303 |Endpoint| | +----+----+ |Endpoint| 304 +--------+ | | +--------+ 305 +---<---+ 306 Reaction 308 Figure 1: A CB controlling the part of the end-to-end path between an 309 ingress point and an egress point. (Note: In some cases, the trigger 310 and measure functions could alternatively be located at other 311 locations (e.g., at a network operations centre.) 313 In the context of a Circuit Breaker, the ingress and egress functions 314 could be implemented in different places. For example, they could be 315 located in network devices at a tunnel ingress and at the tunnel 316 egress. In some cases, they could be located at one or both network 317 endpoints (see figure 2), implemented as components within a 318 transport protocol. 320 +----------+ +----------+ 321 | Ingress | +-+ +-+ +-+ | Egress | 322 | Endpoint +->+R+--+R+--+R+--+ Endpoint | 323 +--+----+--+ +-+ +-+ +-+ +----+-----+ 324 ^ | | 325 | +--+------+ +----+----+ 326 | | Ingress | | Egress | 327 | | Meter | | Meter | 328 | +----+----+ +----+----+ 329 | | | 330 | +--- +----+ | 331 | | Measure +<-----------------+ 332 | +----+----+ Reported 333 | | Egress 334 | +----+----+ Measurement 335 | | Trigger | 336 | +----+----+ 337 | | 338 +---<--+ 339 Reaction 341 Figure 2: An endpoint CB implemented at the sender (ingress) and 342 receiver (egress). 344 The set of components needed to implement a Circuit Breaker are: 346 1. An ingress meter (at the sender or tunnel ingress) records the 347 number of packets/bytes sent in each measurement interval. This 348 measures the offered network load for a flow or set of flows. 349 For example, the measurement interval could be many seconds (or 350 every few tens of seconds or a series of successive shorter 351 measurements that are combined by the Circuit Breaker Measurement 352 function). 354 2. An egress meter (at the receiver or tunnel egress) records the 355 number/bytes received in each measurement interval. This 356 measures the supported load for the flow or set of flows, and 357 could utilize other signals to detect the effect of congestion 358 (e.g., loss/marking experienced over the path). The measurements 359 at the egress could be synchronised (including an offset for the 360 time of flight of the data, or referencing the measurements to a 361 particular packet) to ensure any counters refer to the same span 362 of packets. 364 3. The measured values at the ingress and egress are communicated to 365 the Circuit Breaker Measurement function. This could use several 366 methods including: Sending return measurement packets from a 367 receiver to a trigger function at the sender; An implementation 368 using Operations, Administration and Management (OAM); or be 369 sending another in-band signalling datagram to the trigger 370 function. This could also be implemented purely as a control 371 plane function, e.g., using a software-defined network 372 controller. 374 4. The measurement function combines the ingress and egress 375 measurements to assess the present level of network congestion. 376 (For example, the loss rate for each measurement interval could 377 be deduced from calculating the difference between ingress and 378 egress counter values.) Note the method does not require high 379 accuracy for the period of the measurement interval (or therefore 380 the measured value, since isolated and/or infrequent loss events 381 need to be disregarded.) 383 5. A trigger function determines whether the measurements indicate 384 persistent congestion. This function defines an appropriate 385 threshold for determining that there is persistent congestion 386 between the ingress and egress. This preferably considers a rate 387 or ratio, rather than an absolute value (e.g., more than 10% 388 loss, but other methods could also be based on the rate of 389 transmission as well as the loss rate). The transport Circuit 390 Breaker is triggered when the threshold is exceeded in multiple 391 measurement intervals (e.g., 3 successive measurements). Designs 392 need to be robust so that single or spurious events do not 393 trigger a reaction. 395 6. A reaction that is applied at the Ingress when the Circuit 396 Breaker is triggered. This seeks to automatically remove the 397 traffic causing persistent congestion. 399 7. A feedback mechanism that triggers when either the receive or 400 ingress and egress measurements are not available, since this 401 also could indicate a loss of control packets (also a symptom of 402 heavy congestion or inability to control the load). 404 4. Requirements for a Network Transport Circuit Breaker 406 The requirements for implementing a Circuit Breaker are: 408 o There needs to be a communication path used for control messages 409 from the ingress meter and the egress meter to the point of 410 measurement. The Circuit Breaker MUST trigger if there is a 411 failure of the communication path used for the control messages. 412 That is, the feedback indicating a congested period needs to be 413 designed so that the Circuit Breaker is triggered when it fails to 414 receive measurement reports that indicate an absence of 415 congestion, rather than relying on the successful transmission of 416 a "congested" signal back to the sender. (The feedback signal 417 could itself be lost under congestion). 419 o A Circuit Breaker is REQUIRED to define a measurement period over 420 which the Circuit Breaker Measurement function measures the level 421 of congestion or loss. This method does not have to detect 422 individual packet loss, but MUST have a way to know that packets 423 have been lost/marked from the traffic flow. 425 o An egress meter can also count Explicit Congestion Notification 426 (ECN) [RFC3168] congestion marks as a part of measurement of 427 congestion, but in this case, loss MUST also be measured to 428 provide a complete view of the level of congestion. For tunnels, 429 [ID-ietf-tsvwg-tunnel-congestion-feedback] describes a way to 430 measure both loss and ECN-marking; these measurements could be 431 used on a relatively short timescale to drive a congestion control 432 response and/or aggregated over a longer timescale with a higher 433 trigger threshold to drive a Circuit Breaker. Subsequent bullet 434 items in this section discuss the necessity of using a longer 435 timescale and a higher trigger threshold. 437 o The measurement period used by a Circuit Breaker Measurement 438 function MUST be longer than the time that current Congestion 439 Control algorithms need to reduce their rate following detection 440 of congestion. This is important because end-to-end Congestion 441 Control algorithms require at least one RTT to notify and adjust 442 the traffic to experienced congestion, and congestion bottlenecks 443 can share traffic with a diverse range of RTTs. The measurement 444 period is therefore expected to be significantly longer than the 445 RTT experienced by the Circuit Breaker itself. 447 o If necessary, MAY combine successive individual meter samples from 448 the ingress and egress to ensure observation of an average over a 449 sufficiently long interval. (Note when meter samples need to be 450 combined, the combination needs to reflect the sum of the 451 individual sample counts divided by the total time/volume over 452 which the samples were measured. Individual samples over 453 different intervals can not be directly combined to generate an 454 average value.) 456 o A Circuit Breaker is REQUIRED to define a threshold to determine 457 whether the measured congestion is considered excessive. 459 o A Circuit Breaker is REQUIRED to define the triggering interval, 460 defining the period over which the trigger uses the collected 461 measurements. Circuit Breakers need to trigger over a 462 sufficiently long period to avoid additionally penalizing flows 463 with a long path RTT (e.g., many path RTTs). 465 o A Circuit Breaker MUST be robust to multiple congestion events. 466 This usually will define a number of measured persistent 467 congestion events per triggering period. For example, a Circuit 468 Breaker MAY combine the results of several measurement periods to 469 determine if the Circuit Breaker is triggered. (e.g., triggered 470 when persistent congestion is detected in 3 of the measurements 471 within the triggering interval). 473 o A Circuit Breaker SHOULD be constructed so that it does not 474 trigger under light or intermittent congestion. 476 o The default response to a trigger SHOULD disable all traffic that 477 contributed to congestion. 479 o Once triggered, the Circuit Breaker MUST react decisively by 480 disabling or significantly reducing traffic at the source (e.g., 481 ingress). 483 o The reaction needs to be much more severe than that of a 484 Congestion Control algorithm (such as TCP's congestion control 485 [RFC5681] or TCP-Friendly Rate Control, TFRC [RFC5348]), because 486 the Circuit Breaker reacts to more persistent congestion and 487 operates over longer timescales (i.e., the overload condition will 488 have persisted for a longer time before the Circuit Breaker is 489 triggered). 491 o A reaction that results in a reduction SHOULD result in reducing 492 the traffic by at least an order of magnitude. A response that 493 achieves the reduction by terminating flows, rather than randomly 494 dropping packets, will often be more desirable to users of the 495 service. A Circuit Breaker that reduces the rate of a flow, MUST 496 continue to monitor the level of congestion and MUST further react 497 to reduce the rate if the Circuit Breaker is again triggered. 499 o The reaction to a triggered Circuit Breaker MUST continue for a 500 period that is at least the triggering interval. Operator 501 intervention will usually be required to restore a flow. If an 502 automated response is needed to reset the trigger, then this needs 503 to not be immediate. The design of an automated reset mechanism 504 needs to be sufficiently conservative that it does not adversely 505 interact with other mechanisms (including other Circuit Breaker 506 algorithms that control traffic over a common path). It SHOULD 507 NOT perform an automated reset when there is evidence of continued 508 congestion. 510 o When a Circuit Breaker is triggered, it SHOULD be regarded as an 511 abnormal network event. As such, this event SHOULD be logged. 512 The measurements that lead to triggering of the Circuit Breaker 513 SHOULD also be logged. 515 o A Circuit Breaker requires control communication between endpoints 516 and/or network devices. The source and integrity of control 517 messages (measurements and triggers) MUST be protected from off- 518 path attacks (Section 8). When there is a risk of on-path attack, 519 a cryptographic authentication mechanism for all control/ 520 measurement messages is RECOMMENDED (Section 8). 522 o The circuit breaker MUST be designed to be robust to packet loss 523 that can also be experienced during congestion/overload. This 524 does not imply that it is desirable to provide reliable delivery 525 (e.g., over TCP), since this can incur additional delay in 526 responding to congestion. Appropriate mechanisms could be to 527 duplicate control messages to provide increased robustness to 528 loss, or/and to regard a lack of control traffic as an indication 529 that excessive congestion may be being experienced 530 [ID-ietf-tsvwg-RFC5405.bis]. 532 o The control communication may be in-band or out-of-band. In-band 533 communication is RECOMMENDED when either design would be possible. 534 If this traffic is sent over a shared path, it is RECOMMENDED that 535 this control traffic is prioritized to reduce the probability of 536 loss under congestion. Control traffic also needs to be 537 considered when provisioning a network that uses a circuit 538 breaker. 540 in-Band: An in-band control method SHOULD assume that loss of 541 control messages is an indication of potential congestion on 542 the path, and repeated loss ought to cause the Circuit Breaker 543 to be triggered. This design has the advantage that it 544 provides fate-sharing of the traffic flow(s) and the control 545 communications. 547 Out-of-Band: An out-of-band control method SHOULD NOT trigger 548 Circuit Breaker reaction when there is loss of control messages 549 (e.g., a loss of measurements). This avoids failure 550 amplification/propagation when the measurement and data paths 551 fail independently. A failure of an out-of-band communication 552 path SHOULD be regarded as abnormal network event and be 553 handled as appropriate for the network, e.g., this event SHOULD 554 be logged, and additional network operator action might be 555 appropriate, depending on the network and the traffic involved. 557 5. Other network topologies 559 A Circuit Breaker can be deployed in networks with topologies 560 different to that presented in figure 2. This section describes 561 examples of such usage, and possible places where functions may be 562 implemented. 564 5.1. Use with a multicast control/routing protocol 566 +----------+ +--------+ +----------+ 567 | Ingress | +-+ +-+ +-+ | Egress | | Egress | 568 | Endpoint +->+R+--+R+--+R+--+ Router |--+ Endpoint +->+ 569 +----+-----+ +-+ +-+ +-+ +---+--+-+ +----+-----+ | 570 ^ ^ ^ ^ | ^ | | 571 | | | | | | | | 572 +----+----+ + - - - < - - - - + | +----+----+ | Reported 573 | Ingress | multicast Prune | | Egress | | Ingress 574 | Meter | | | Meter | | Measurement 575 +---------+ | +----+----+ | 576 | | | 577 | +----+----+ | 578 | | Measure +<--+ 579 | +----+----+ 580 | | 581 | +----+----+ 582 multicast | | Trigger | 583 Leave | +----+----+ 584 Message | | 585 +----<----+ 587 Figure 3: An example of a multicast CB controlling the end-to-end 588 path between an ingress endpoint and an egress endpoint. 590 Figure 3 shows one example of how a multicast circuit breaker could 591 be implemented at a pair of multicast endpoints (e.g., to implement a 592 Fast-Trip Circuit Breaker, Section 6.1). The ingress endpoint (the 593 sender that sources the multicast traffic) meters the ingress load, 594 generating an ingress measurement (e.g., recording timestamped packet 595 counts), and sends this measurement to the multicast group together 596 with the traffic it has measured. 598 Routers along a multicast path forward the multicast traffic 599 (including the ingress measurement) to all active endpoint receivers. 600 Each last hop (egress) router forwards the traffic to one or more 601 egress endpoint(s). 603 In this figure, each endpoint includes a meter that performs a local 604 egress load measurement. An endpoint also extracts the received 605 ingress measurement from the traffic, and compares the ingress and 606 egress measurements to determine if the Circuit Breaker ought to be 607 triggered. This measurement has to be robust to loss (see previous 608 section). If the Circuit Breaker is triggered, it generates a 609 multicast leave message for the egress (e.g., an IGMP or MLD message 610 sent to the last hop router), which causes the upstream router to 611 cease forwarding traffic to the egress endpoint. 613 Any multicast router that has no active receivers for a particular 614 multicast group will prune traffic for that group, sending a prune 615 message to its upstream router. This starts the process of releasing 616 the capacity used by the traffic and is a standard multicast routing 617 function (e.g., using the PIM-SM routing protocol). Each egress 618 operates autonomously, and the circuit breaker "reaction" is executed 619 by the multicast control plane (e.g., by the PIM multicast routing 620 protocol), requiring no explicit signalling by the circuit breaker 621 along the communication path used for the control messages. Note: 622 there is no direct communication with the Ingress, and hence a 623 triggered Circuit Breaker only controls traffic downstream of the 624 first hop router. It does not stop traffic flowing from the sender 625 to the first hop router; this is however the common practice for 626 multicast deployment. 628 The method could also be used with a multicast tunnel or subnetwork 629 (e.g., Section 6.2, Section 6.3), where a meter at the ingress 630 generates additional control messages to carry the measurement data 631 towards the egress where the egress metering is implemented. 633 5.2. Use with control protocols supporting pre-provisioned capacity 635 Some paths are provisioned using a control protocol, e.g., flows 636 provisioned using the Multi-Protocol Label Switching (MPLS) services, 637 path provisioned using the Resource reservation protocol (RSVP), 638 networks utilizing Software Defined Network (SDN) functions, or 639 admission-controlled Differentiated Services. 641 Figure 1 shows one expected use case, where in this usage a separate 642 device could be used to perform the measurement and trigger 643 functions. The reaction generated by the trigger could take the form 644 of a network control message sent to the ingress and/or other network 645 elements causing these elements to react to the Circuit Breaker. 646 Examples of this type of use are provided in section Section 6.3. 648 5.3. Unidirectional Circuit Breakers over Controlled Paths 650 A Circuit Breaker can be used to control uni-directional UDP traffic, 651 providing that there is a communication path that can be used for 652 control messages to connect the functional components at the Ingress 653 and Egress. This communication path for the control messages can 654 exist in networks for which the traffic flow is purely 655 unidirectional. For example, a multicast stream that sends packets 656 across an Internet path and can use multicast routing to prune flows 657 to shed network load. Some other types of subnetwork also utilize 658 control protocols that can be used to control traffic flows. 660 6. Examples of Circuit Breakers 662 There are multiple types of Circuit Breaker that could be defined for 663 use in different deployment cases. This section provides examples of 664 different types of circuit breaker: 666 6.1. A Fast-Trip Circuit Breaker 668 [RFC2309] discusses the dangers of congestion-unresponsive flows and 669 states that "all UDP-based streaming applications should incorporate 670 effective congestion avoidance mechanisms". All applications ought 671 to use a full-featured transport (TCP, SCTP, DCCP), and if not, an 672 application (e.g., those using UDP and its UDP-Lite variant) needs to 673 provide appropriate congestion avoidance. Guidance for applications 674 that do not use congestion-controlled transports is provided in 675 [ID-ietf-tsvwg-RFC5405.bis]. Such mechanisms can be designed to 676 react on much shorter timescales than a circuit breaker, that only 677 observes a traffic envelope. Congestion control methods can also 678 interact with an application to more effectively control its sending 679 rate. 681 A fast-trip circuit breaker is the most responsive form of Circuit 682 Breaker. It has a response time that is only slightly larger than 683 that of the traffic that it controls. It is suited to traffic with 684 well-understood characteristics (and could include one or more 685 trigger functions specifically tailored the type of traffic for which 686 it is designed). It is not suited to arbitrary network traffic and 687 may be unsuitable for traffic aggregates, since it could prematurely 688 trigger (e.g., when multiple congestion-controlled flows lead to 689 short-term overload). 691 Although the mechanisms can be implemented in a RTP-aware network 692 devices, these mechanisms are also suitable for implementation in 693 endpoints (e.g., as a part of the tranport system), where they can 694 also compliment end-to-end congestion control methods. A shorter 695 response time enables these mechanisms to triggers before other forms 696 of circuit breaker (e.g., circuit breakers operating on traffic 697 aggregates at a point along the network path). 699 6.1.1. A Fast-Trip Circuit Breaker for RTP 701 A set of fast-trip Circuit Breaker methods have been specified for 702 use together by a Real-time Transport Protocol (RTP) flow using the 703 RTP/AVP Profile [RTP-CB]. It is expected that, in the absence of 704 severe congestion, all RTP applications running on best-effort IP 705 networks will be able to run without triggering these circuit 706 breakers. A fast-trip RTP Circuit Breaker is therefore implemented 707 as a fail-safe that when triggered will terminate RTP traffic. 709 The sending endpoint monitors reception of in-band RTP Control 710 Protocol (RTCP) reception report blocks, as contained in SR or RR 711 packets, that convey reception quality feedback information. This is 712 used to measure (congestion) loss, possibly in combination with ECN 713 [RFC6679]. 715 The Circuit Breaker action (shutdown of the flow) is triggered when 716 any of the following trigger conditions are true: 718 1. An RTP Circuit Breaker triggers on reported lack of progress. 720 2. An RTP Circuit Breaker triggers when no receiver reports messages 721 are received. 723 3. An RTP Circuit Breaker uses a TFRC-style check and sets a hard 724 upper limit to the long-term RTP throughput (over many RTTs). 726 4. An RTP Circuit Breaker includes the notion of Media Usability. 727 This circuit breaker is triggered when the quality of the 728 transported media falls below some required minimum acceptable 729 quality. 731 6.2. A Slow-trip Circuit Breaker 733 A slow-trip Circuit Breaker could be implemented in an endpoint or 734 network device. This type of Circuit Breaker is much slower at 735 responding to congestion than a fast-trip Circuit Breaker and is 736 expected to be more common. 738 One example where a slow-trip Circuit Breaker is needed is where 739 flows or traffic-aggregates use a tunnel or encapsulation and the 740 flows within the tunnel do not all support TCP-style congestion 741 control (e.g., TCP, SCTP, TFRC), see [ID-ietf-tsvwg-RFC5405.bis] 742 section 3.1.3. A use case is where tunnels are deployed in the 743 general Internet (rather than "controlled environments" within an 744 Internet service provider or enterprise network), especially when the 745 tunnel could need to cross a customer access router. 747 6.3. A Managed Circuit Breaker 749 A managed Circuit Breaker is implemented in the signalling protocol 750 or management plane that relates to the traffic aggregate being 751 controlled. This type of circuit breaker is typically applicable 752 when the deployment is within a "controlled environment". 754 A Circuit Breaker requires more than the ability to determine that a 755 network path is forwarding data, or to measure the rate of a path - 756 which are often normal network operational functions. There is an 757 additional need to determine a metric for congestion on the path and 758 to trigger a reaction when a threshold is crossed that indicates 759 persistent congestion. 761 The control messages can use either in-band or out-of-band 762 communications. 764 6.3.1. A Managed Circuit Breaker for SAToP Pseudo-Wires 766 [RFC4553], SAToP Pseudo-Wires (PWE3), section 8 describes an example 767 of a managed circuit breaker for isochronous flows. 769 If such flows were to run over a pre-provisioned (e.g., Multi- 770 Protocol Label Switching, MPLS) infrastructure, then it could be 771 expected that the Pseudowire (PW) would not experience congestion, 772 because a flow is not expected to either increase (or decrease) their 773 rate. If instead Pseudo-Wire traffic is multiplexed with other 774 traffic over the general Internet, it could experience congestion. 775 [RFC4553] states: "If SAToP PWs run over a PSN providing best-effort 776 service, they SHOULD monitor packet loss in order to detect "severe 777 congestion". The currently recommended measurement period is 1 778 second, and the trigger operates when there are more than three 779 measured Severely Errored Seconds (SES) within a period. If such a 780 condition is detected, a SAToP PW ought to shut down bidirectionally 781 for some period of time...". 783 The concept was that when the packet loss ratio (congestion) level 784 increased above a threshold, the PW was by default disabled. This 785 use case considered fixed-rate transmission, where the PW had no 786 reasonable way to shed load. 788 The trigger needs to be set at the rate that the PW was likely to 789 experience a serious problem, possibly making the service non- 790 compliant. At this point, triggering the Circuit Breaker would 791 remove the traffic preventing undue impact on congestion-responsive 792 traffic (e.g., TCP). Part of the rationale, was that high loss 793 ratios typically indicated that something was "broken" and ought to 794 have already resulted in operator intervention, and therefore need to 795 trigger this intervention. 797 An operator-based response provides opportunity for other action to 798 restore the service quality, e.g., by shedding other loads or 799 assigning additional capacity, or to consciously avoid reacting to 800 the trigger while engineering a solution to the problem. This could 801 require the trigger to be sent to a third location (e.g., a network 802 operations centre, NOC) responsible for operation of the tunnel 803 ingress, rather than the tunnel ingress itself. 805 6.3.2. A Managed Circuit Breaker for Pseudowires (PWs) 807 Pseudowires (PWs) [RFC3985] have become a common mechanism for 808 tunneling traffic, and may compete for network resources both with 809 other PWs and with non-PW traffic, such as TCP/IP flows. 811 [ID-ietf-pals-congcons] discusses congestion conditions that can 812 arise when PWs compete with elastic (i.e., congestion responsive) 813 network traffic (e.g, TCP traffic). Elastic PWs carrying IP traffic 814 (see [RFC4488]) do not raise major concerns because all of the 815 traffic involved responds, reducing the transmission rate when 816 network congestion is detected. 818 In contrast, inelastic PWs (e.g., a fixed bandwidth Time Division 819 Multiplex, TDM) [RFC4553] [RFC5086] [RFC5087]) have the potential to 820 harm congestion responsive traffic or to contribute to excessive 821 congestion because inelastic PWs do not adjust their transmission 822 rate in response to congestion. [ID-ietf-pals-congcons] analyses TDM 823 PWs, with an initial conclusion that a TDM PW operating with a degree 824 of loss that may result in congestion-related problems is also 825 operating with a degree of loss that results in an unacceptable TDM 826 service. For that reason, the draft suggests that a managed circuit 827 breaker that shuts down a PW when it persistently fails to deliver 828 acceptable TDM service is a useful means for addressing these 829 congestion concerns. 831 7. Examples where circuit breakers may not be needed. 833 A Circuit Breaker is not required for a single congestion-controlled 834 flow using TCP, SCTP, TFRC, etc. In these cases, the congestion 835 control methods are already designed to prevent persistent 836 congestion. 838 7.1. CBs over pre-provisioned Capacity 840 One common question is whether a Circuit Breaker is needed when a 841 tunnel is deployed in a private network with pre-provisioned 842 capacity. 844 In this case, compliant traffic that does not exceed the provisioned 845 capacity ought not to result in persistent congestion. A Circuit 846 Breaker will hence only be triggered when there is non-compliant 847 traffic. It could be argued that this event ought never to happen - 848 but it could also be argued that the Circuit Breaker equally ought 849 never to be triggered. If a Circuit Breaker were to be implemented, 850 it will provide an appropriate response if persistent congestion 851 occurs in an operational network. 853 Implementing a Circuit Breaker will not reduce the performance of the 854 flows, but in the event that persistent congestion occurs it protects 855 network traffic that shares network capacity with these flows. A 856 Circuit Breaker also could be used to protect other sharing network 857 traffic from a failure that causes the Circuit Breaker traffic to be 858 routed over a non-pre-provisioned path. 860 7.2. CBs with tunnels carrying Congestion-Controlled Traffic 862 IP-based traffic is generally assumed to be congestion-controlled, 863 i.e., it is assumed that the transport protocols generating IP-based 864 traffic at the sender already employ mechanisms that are sufficient 865 to address congestion on the path [ID-ietf-tsvwg-RFC5405.bis]. A 866 question therefore arises when people deploy a tunnel that is thought 867 to only carry an aggregate of TCP traffic (or traffic using some 868 other congestion control method): Is there advantage in this case in 869 using a Circuit Breaker? 871 For sure, traffic in a such a tunnel will respond to congestion. 872 However, the answer to the question is not always obvious, because 873 the overall traffic formed by an aggregate of flows that implement a 874 congestion control mechanism does not necessarily prevent persistent 875 congestion. For instance, most congestion control mechanisms require 876 long-lived flows to react to reduce the rate of a flow, an aggregate 877 of many short flows could result in many terminating before they 878 experience congestion. It is also often impossible for a tunnel 879 service provider to know that the tunnel only contains congestion- 880 controlled traffic (e.g., Inspecting packet headers could not be 881 possible). The important thing to note is that if the aggregate of 882 the traffic does not result in persistent congestion (impacting other 883 flows), then the Circuit Breaker will not trigger. This is the 884 expected case in this context - so implementing a Circuit Breaker 885 will not reduce performance of the tunnel, but in the event that 886 persistent congestion occurs this protects other network traffic that 887 shares capacity with the tunnel traffic. 889 7.3. CBs with Uni-directional Traffic and no Control Path 891 A one-way forwarding path could have no associated communication path 892 for sending control messages, and therefore cannot be controlled 893 using an automated process. This service could be provided using a 894 path that has dedicated capacity and does not share this capacity 895 with other elastic Internet flows (i.e., flows that vary their rate). 897 A way to mitigate the impact on other flows when capacity could be 898 shared is to manage the traffic envelope by using ingress policing. 900 Supporting this type of traffic in the general Internet requires 901 operator monitoring to detect and respond to persistent congestion. 903 8. Security Considerations 905 All Circuit Breaker mechanisms rely upon coordination between the 906 ingress and egress meters and communication with the trigger 907 function. This is usually achieved by passing network control 908 information (or protocol messages) across the network. Timely 909 operation of a circuit breaker depends on the choice of measurement 910 period. If the receiver has an interval that is overly long, then 911 the responsiveness of the circuit breaker decreases. This impacts 912 the ability of the circuit breaker to detect and react to congestion. 914 A Circuit Breaker could potentially be exploited by an attacker to 915 mount a Denial of Service (DoS) attack against the traffic being 916 measured. Mechanisms therefore need to be implemented to prevent 917 attacks on the network control information that would result in DoS. 918 The source and integrity of control information (measurements and 919 triggers) MUST be protected from off-path attacks. Without 920 protection, it could be trivial for an attacker to inject fake or 921 modified control/measurement messages (e.g., indicating high packet 922 loss rates) causing a Circuit Breaker to trigger and to therefore 923 mount a DoS attack that disrupts a flow. 925 Simple protection can be provided by using a randomized source port, 926 or equivalent field in the packet header (such as the RTP SSRC value 927 and the RTP sequence number) expected not to be known to an off-path 928 attacker. Stronger protection can be achieved using a secure 929 authentication protocol. This attack is relatively easy for an on- 930 path attacker when the messages are neither encrypted nor 931 authenticated. When there is a risk of on-path attack, a 932 cryptographic authentication mechanism for all control/measurement 933 messages is RECOMMENDED to mitigate this concern. There is a design 934 trade-off between the cost of introducing cryptographic security for 935 control messages and the desire to protect control communication. 936 For some deployment scenarios the value of additional protection from 937 DoS attack will therefore lead to a requirement to authenticate all 938 control messages. 940 Transmission of network control messages consumes network capacity. 941 This control traffic needs to be considered in the design of a 942 Circuit Breaker and could potentially add to network congestion. If 943 this traffic is sent over a shared path, it is RECOMMENDED that this 944 control traffic is prioritized to reduce the probability of loss 945 under congestion. Control traffic also needs to be considered when 946 provisioning a network that uses a circuit breaker. 948 The circuit breaker MUST be designed to be robust to packet loss that 949 can also be experienced during congestion/overload. Loss of control 950 messages could be a side-effect of a congested network, but also 951 could arise from other causes Section 4. 953 The security implications depend on the design of the mechanisms, the 954 type of traffic being controlled and the intended deployment 955 scenario. Each design of a Circuit Breaker MUST therefore evaluate 956 whether the particular circuit breaker mechanism has new security 957 implications. 959 9. IANA Considerations 961 This document makes no request from IANA. 963 10. Acknowledgments 965 There are many people who have discussed and described the issues 966 that have motivated this draft. Contributions and comments included: 967 Lars Eggert, Colin Perkins, David Black, Matt Mathis, Andrew 968 McGregor, Bob Briscoe and Eliot Lear. This work was part-funded by 969 the European Community under its Seventh Framework Programme through 970 the Reducing Internet Transport Latency (RITE) project (ICT-317700). 972 11. Revision Notes 974 XXX RFC-Editor: Please remove this section prior to publication XXX 976 Draft 00 978 This was the first revision. Help and comments are greatly 979 appreciated. 981 Draft 01 982 Contained clarifications and changes in response to received 983 comments, plus addition of diagram and definitions. Comments are 984 welcome. 986 WG Draft 00 988 Approved as a WG work item on 28th Aug 2014. 990 WG Draft 01 992 Incorporates feedback after Dallas IETF TSVWG meeting. This version 993 is thought ready for WGLC comments. Definitions of abbreviations. 995 WG Draft 02 997 Minor fixes for typos. Rewritten security considerations section. 999 WG Draft 03 1001 Updates following WGLC comments (see TSV mailing list). Comments 1002 from C Perkins; D Black and off-list feedback. 1004 A clear recommendation of intended scope. 1006 Changes include: Improvement of language on timescales and minimum 1007 measurement period; clearer articulation of endpoint and multicast 1008 examples - with new diagrams; separation of the controlled network 1009 case; updated text on position of trigger function; corrections to 1010 RTP-CB text; clarification of loss v ECN metrics; checks against 1011 submission checklist 9use of keywords, added meters to diagrams). 1013 WG Draft 04 1015 Added section on PW CB for TDM - a newly adopted draft (D. Black). 1017 WG Draft 05 1019 Added clarifications requested during AD review. 1021 WG Draft 06 1023 Fixed some remaining typos. 1025 Update following detailed review by Bob Briscoe, and comments by D. 1026 Black. 1028 WG Draft 07 1029 Additional update following review by Bob Briscoe. 1031 WG Draft 08 1033 Updated text on the response to lack of meter measurements with 1034 managed circuit breakers. Additional comments from Eliot Lear (APPs 1035 area). 1037 WG Draft 09 1039 Updated text on applications from Eliot Lear. Additional feedback 1040 from Bob Briscoe. 1042 WG Draft 10 1044 Updated text following comments by D Black including a rewritten ECN 1045 requirements bullet with of a reference to a tunnel measurement 1046 method in [ID-ietf-tsvwg-tunnel-congestion-feedback]. 1048 12. References 1050 12.1. Normative References 1052 [ID-ietf-tsvwg-RFC5405.bis] 1053 Eggert, L., Fairhurst, G., and G. Shepherd, "UDP Usage 1054 Guidelines (Work-in-Progress)", 2015. 1056 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1057 Requirement Levels", BCP 14, RFC 2119, 1058 DOI 10.17487/RFC2119, March 1997, 1059 . 1061 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1062 of Explicit Congestion Notification (ECN) to IP", 1063 RFC 3168, DOI 10.17487/RFC3168, September 2001, 1064 . 1066 12.2. Informative References 1068 [ID-ietf-pals-congcons] 1069 Stein, YJ., Black, D., and B. Briscoe, "Pseudowire 1070 Congestion Considerations (Work-in-Progress)", 2015. 1072 [ID-ietf-tsvwg-tunnel-congestion-feedback] 1073 Wei, X., Zhu, L., and L. Dend, "Tunnel Congestion Feedback 1074 (Work-in-Progress)", 2015. 1076 [Jacobsen88] 1077 European Telecommunication Standards, Institute (ETSI), 1078 "Congestion Avoidance and Control", SIGCOMM Symposium 1079 proceedings on Communications architectures and 1080 protocols", August 1998. 1082 [RFC1112] Deering, S., "Host extensions for IP multicasting", STD 5, 1083 RFC 1112, DOI 10.17487/RFC1112, August 1989, 1084 . 1086 [RFC2309] Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering, 1087 S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G., 1088 Partridge, C., Peterson, L., Ramakrishnan, K., Shenker, 1089 S., Wroclawski, J., and L. Zhang, "Recommendations on 1090 Queue Management and Congestion Avoidance in the 1091 Internet", RFC 2309, DOI 10.17487/RFC2309, April 1998, 1092 . 1094 [RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, 1095 RFC 2914, DOI 10.17487/RFC2914, September 2000, 1096 . 1098 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 1099 Edge-to-Edge (PWE3) Architecture", RFC 3985, 1100 DOI 10.17487/RFC3985, March 2005, 1101 . 1103 [RFC4488] Levin, O., "Suppression of Session Initiation Protocol 1104 (SIP) REFER Method Implicit Subscription", RFC 4488, 1105 DOI 10.17487/RFC4488, May 2006, 1106 . 1108 [RFC4553] Vainshtein, A., Ed. and YJ. Stein, Ed., "Structure- 1109 Agnostic Time Division Multiplexing (TDM) over Packet 1110 (SAToP)", RFC 4553, DOI 10.17487/RFC4553, June 2006, 1111 . 1113 [RFC5086] Vainshtein, A., Ed., Sasson, I., Metz, E., Frost, T., and 1114 P. Pate, "Structure-Aware Time Division Multiplexed (TDM) 1115 Circuit Emulation Service over Packet Switched Network 1116 (CESoPSN)", RFC 5086, DOI 10.17487/RFC5086, December 2007, 1117 . 1119 [RFC5087] Stein, Y(J)., Shashoua, R., Insler, R., and M. Anavi, 1120 "Time Division Multiplexing over IP (TDMoIP)", RFC 5087, 1121 DOI 10.17487/RFC5087, December 2007, 1122 . 1124 [RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP 1125 Friendly Rate Control (TFRC): Protocol Specification", 1126 RFC 5348, DOI 10.17487/RFC5348, September 2008, 1127 . 1129 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 1130 Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, 1131 . 1133 [RFC6679] Westerlund, M., Johansson, I., Perkins, C., O'Hanlon, P., 1134 and K. Carlberg, "Explicit Congestion Notification (ECN) 1135 for RTP over UDP", RFC 6679, DOI 10.17487/RFC6679, August 1136 2012, . 1138 [RTP-CB] Perkins, C. and V. Singh, "Multimedia Congestion Control: 1139 Circuit Breakers for Unicast RTP Sessions", February 2014. 1141 Author's Address 1143 Godred Fairhurst 1144 University of Aberdeen 1145 School of Engineering 1146 Fraser Noble Building 1147 Aberdeen, Scotland AB24 3UE 1148 UK 1150 Email: gorry@erg.abdn.ac.uk 1151 URI: http://www.erg.abdn.ac.uk