idnits 2.17.1 draft-ietf-tsvwg-circuit-breaker-13.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (February 17, 2016) is 2984 days in the past. Is this intentional? Checking references for intended status: Best Current Practice ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC1112' is defined on line 1121, but no explicit reference was found in the text -- Obsolete informational reference (is this intentional?): RFC 2309 (Obsoleted by RFC 7567) -- Obsolete informational reference (is this intentional?): RFC 4601 (Obsoleted by RFC 7761) Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TSVWG Working Group G. Fairhurst 3 Internet-Draft University of Aberdeen 4 Intended status: Best Current Practice February 17, 2016 5 Expires: August 20, 2016 7 Network Transport Circuit Breakers 8 draft-ietf-tsvwg-circuit-breaker-13 10 Abstract 12 This document explains what is meant by the term "network transport 13 Circuit Breaker" (CB). It describes the need for circuit breakers 14 for network tunnels and applications when using non-congestion- 15 controlled traffic, and explains where circuit breakers are, and are 16 not, needed. It also defines requirements for building a circuit 17 breaker and the expected outcomes of using a circuit breaker within 18 the Internet. 20 Status of This Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at http://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on August 20, 2016. 37 Copyright Notice 39 Copyright (c) 2016 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (http://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 55 1.1. Types of Circuit Breaker . . . . . . . . . . . . . . . . 5 56 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 57 3. Design of a Circuit-Breaker (What makes a good circuit 58 breaker?) . . . . . . . . . . . . . . . . . . . . . . . . . . 6 59 3.1. Functional Components . . . . . . . . . . . . . . . . . . 6 60 3.2. Other network topologies . . . . . . . . . . . . . . . . 9 61 3.2.1. Use with a multicast control/routing protocol . . . . 10 62 3.2.2. Use with control protocols supporting pre-provisioned 63 capacity . . . . . . . . . . . . . . . . . . . . . . 11 64 3.2.3. Unidirectional Circuit Breakers over Controlled Paths 11 65 4. Requirements for a Network Transport Circuit Breaker . . . . 12 66 5. Examples of Circuit Breakers . . . . . . . . . . . . . . . . 15 67 5.1. A Fast-Trip Circuit Breaker . . . . . . . . . . . . . . . 15 68 5.1.1. A Fast-Trip Circuit Breaker for RTP . . . . . . . . . 16 69 5.2. A Slow-trip Circuit Breaker . . . . . . . . . . . . . . . 16 70 5.3. A Managed Circuit Breaker . . . . . . . . . . . . . . . . 17 71 5.3.1. A Managed Circuit Breaker for SAToP Pseudo-Wires . . 17 72 5.3.2. A Managed Circuit Breaker for Pseudowires (PWs) . . . 18 73 6. Examples where circuit breakers may not be needed. . . . . . 19 74 6.1. CBs over pre-provisioned Capacity . . . . . . . . . . . . 19 75 6.2. CBs with tunnels carrying Congestion-Controlled Traffic . 19 76 6.3. CBs with Uni-directional Traffic and no Control Path . . 20 77 7. Security Considerations . . . . . . . . . . . . . . . . . . . 20 78 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 79 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 22 80 10. Revision Notes . . . . . . . . . . . . . . . . . . . . . . . 22 81 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 24 82 11.1. Normative References . . . . . . . . . . . . . . . . . . 24 83 11.2. Informative References . . . . . . . . . . . . . . . . . 24 84 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 26 86 1. Introduction 88 The term "Circuit Breaker" originates in electricity supply, and has 89 nothing to do with network circuits or virtual circuits. In 90 electricity supply, a Circuit Breaker is intended as a protection 91 mechanism of last resort. Under normal circumstances, a Circuit 92 Breaker ought not to be triggered; it is designed to protect the 93 supply network and attached equipment when there is overload. People 94 do not expect an electrical circuit-breaker (or fuse) in their home 95 to be triggered, except when there is a wiring fault or a problem 96 with an electrical appliance. 98 In networking, the Circuit Breaker (CB) principle can be used as a 99 protection mechanism of last resort to avoid persistent excessive 100 congestion impacting other flows that share network capacity. 101 Persistent congestion was a feature of the early Internet of the 102 1980s. This resulted in excess traffic starving other connections 103 from access to the Internet. It was countered by the requirement to 104 use congestion control (CC) in the Transmission Control Protocol 105 (TCP) [Jacobsen88]. These mechanisms operate in Internet hosts to 106 cause TCP connections to "back off" during congestion. The addition 107 of a congestion control to TCP (currently documented in [RFC5681] 108 ensured the stability of the Internet, because it was able to detect 109 congestion and promptly react. This worked well while TCP was by far 110 the dominant traffic in the Internet, and most TCP flows were long- 111 lived (ensuring that they could detect and respond to congestion 112 before the flows terminated). This is no longer the case, and non- 113 congestion-controlled traffic, including many applications using the 114 User Datagram Protocol (UDP), can form a significant proportion of 115 the total traffic traversing a link. The current Internet therefore 116 requires that non-congestion-controlled traffic needs to be 117 considered to avoid persistent excessive congestion. 119 A network transport Circuit Breaker is an automatic mechanism that is 120 used to continuously monitor a flow or aggregate set of flows. The 121 mechanism seeks to detect when the flow(s) experience persistent 122 excessive congestion and when this is detected to terminate (or 123 significantly reduce the rate of) the flow(s). This is a safety 124 measure to prevent starvation of network resources denying other 125 flows from access to the Internet. Such measures are essential for 126 an Internet that is heterogeneous and for traffic that is hard to 127 predict in advance. Avoiding persistent excessive prevention is 128 important to reduce the potential for "Congestion Collapse" 129 [RFC2914]. 131 There are important differences between a transport circuit-breaker 132 and a congestion control method. Specifically, congestion control 133 (as implemented in TCP, SCTP, and DCCP) operates on a timescale on 134 the order of a packet round-trip-time (RTT), the time from sender to 135 destination and return. Congestion control methods are able to react 136 to a single packet loss/marking and continuously adapt to reduce the 137 transmission rate for each loss or congestion event. The goal is 138 usually to limit the maximum transmission rate to a rate that 139 reflects a fair use of the available capacity across a network path. 140 These methods typically operate on individual traffic flows (e.g., a 141 5-tuple that includes the IP addresses, protocol, and ports). 143 In contrast, Circuit Breakers are recommended for non-congestion- 144 controlled Internet flows and for traffic aggregates, e.g., traffic 145 sent using a network tunnel. They operate on timescales much longer 146 than the packet RTT, and trigger under situations of abnormal 147 excessive congestion. People have been implementing what this 148 document characterizes as circuit breakers on an ad hoc basis to 149 protect Internet traffic. This document therefore provides guidance 150 on how to deploy and use these mechanisms. Later sections provide 151 examples of cases where circuit-breakers may or may not be desirable. 153 Other mechanisms may also be available to network operators to detect 154 excessive congestion (e.g., an observation of excessive utilisation 155 for a port on a network device). Utilising such information, 156 operational mechanisms could react to reduce network load over a 157 shorter timescale than those of a network transport Circuit Breaker. 158 The role of the Circuit Breaker over such paths remains as a method 159 of last resort. Because it acts over a longer timescale, the Circuit 160 Breaker should trigger when other reactions did not succeed in 161 reducing persistent excessive congestion. 163 A Circuit Breaker needs to measure (meter) the traffic to determine 164 if the network is experiencing congestion and needs to be designed to 165 trigger robustly when there is persistent excessive congestion. 167 A Circuit Breaker trigger will often utilize a series of successive 168 sample measurements metered at an ingress point and an egress point 169 (either of which could be a transport endpoint). The trigger needs 170 to operate on a timescale much longer than the path round trip time 171 (e.g., seconds to possibly many tens of seconds). This longer period 172 is needed to provide sufficient time for transport congetsion control 173 (or applications) to adjust their rate following congestion, and for 174 the network load to stabilize after any adjustment. This is to 175 ensure that a Circuit Breaker does not accidentally trigger following 176 a single (or even successive) congestion events (congestion events 177 trigger transport congestion control, and are to be regarded as 178 normal on a network link operating near capacity). Once triggered, 179 the Circuit Breaker needs to provide a control function (called the 180 "reaction"). This removes traffic from the network, either by 181 disabling the flow or by significantly reducing the level of traffic. 182 This reaction provides the required protection to prevent persistent 183 excessive congestion being experienced by other flows that share the 184 congested part of the network path. 186 Section 4 defines requirements for building a Circuit Breaker. 188 The operational conditions that cause a Circuit Breaker to trigger 189 should be regarded as abnormal. Examples of situations that could 190 trigger a Circuit Breaker include: 192 o anomalous traffic that exceeds the provisioned capacity (or whose 193 traffic characteristics exceed the threshold configured for the 194 Circuit Breaker); 196 o traffic generated by an application at a time when the provisioned 197 network capacity is being utilised for other purposes; 199 o routing changes that cause additional traffic to start using the 200 path monitored by the Circuit Breaker; 202 o misconfiguration of a service/network device where the capacity 203 available is insufficient to support the current traffic 204 aggregate; 206 o misconfiguration of an admission controller or traffic policer 207 that allows more traffic than expected across the path monitored 208 by the Circuit Breaker. 210 In many cases, the reason for triggering a Circuit Breaker will not 211 be evident to the source of the traffic (user, application, endpoint, 212 etc). In contrast, an application that uses congestion control will 213 generate elastic traffic. This may be expected to regulate the 214 network load under congestion and will therefore often be a preferred 215 solution for applications that can respond to congestion signals or 216 that can use a congestion-controlled transport. 218 A Circuit Breaker can be used to limit traffic from applications that 219 are unable, or choose not, to use congestion control, or where the 220 congestion control properties of their traffic cannot be relied upon 221 (e.g., traffic carried over a network tunnel). In such 222 circumstances, it is all but impossible for the Circuit Breaker to 223 signal back to the impacted applications. In some cases applications 224 may have difficulty in determining that a Circuit Breaker has 225 triggered, and where in the network this happened. Application 226 developers are advised to avoid these circumstances, where possible, 227 by deploying appropriate congestion control mechanisms. 229 1.1. Types of Circuit Breaker 231 There are various forms of network transport circuit breaker. These 232 are differentiated mainly on the timescale over which they are 233 triggered, but also in the intended protection they offer: 235 o Fast-Trip Circuit Breakers: The relatively short timescale used by 236 this form of circuit breaker is intended to provide protection for 237 network traffic from a single flow or related group of flows. 239 o Slow-Trip Circuit Breakers: This circuit breaker utilizes a longer 240 timescale and is designed to protect network traffic from 241 congestion by traffic aggregates. 243 o Managed Circuit Breakers: Utilize the operations and management 244 functions that might be present in a managed service to implement 245 a circuit breaker. 247 Examples of each type of circuit breaker are provided in section 4. 249 2. Terminology 251 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 252 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 253 document are to be interpreted as described in [RFC2119]. 255 3. Design of a Circuit-Breaker (What makes a good circuit breaker?) 257 Although circuit breakers have been talked about in the IETF for many 258 years, there has not yet been guidance on the cases where circuit 259 breakers are needed or upon the design of circuit breaker mechanisms. 260 This document seeks to offer advice on these two topics. 262 Circuit Breakers are RECOMMENDED for IETF protocols and tunnels that 263 carry non-congestion-controlled Internet flows and for traffic 264 aggregates. This includes traffic sent using a network tunnel. 265 Designers of other protocols and tunnel encapsulations also ought to 266 consider the use of these techniques as a last resort to protect 267 traffic that shares the network path being used. 269 This document defines the requirements for design of a Circuit 270 Breaker and provides examples of how a Circuit Breaker can be 271 constructed. The specifications of individual protocols and tunnel 272 encapsulations need to detail the protocol mechanisms needed to 273 implement a Circuit Breaker. 275 Section 3.1 describes the functional components of a circuit breaker 276 and section 3.2 defines requirements for implementing a Circuit 277 Breaker. 279 3.1. Functional Components 281 The basic design of a transport circuit breaker involves 282 communication between an ingress point (a sender) and an egress point 283 (a receiver) of a network flow or set of flows. A simple picture of 284 Circuit Breaker operation is provided in figure 1. This shows a set 285 of routers (each labelled R) connecting a set of endpoints. 287 A Circuit Breaker is used to control traffic passing through a subset 288 of these routers, acting between the ingress and a egress point 289 network devices. The path between the ingress and egress could be 290 provided by a tunnel or other network-layer technique. One expected 291 use would be at the ingress and egress of a service, where all 292 traffic being considered terminates beyond the egress point, and 293 hence the ingress and egress carry the same set of flows. 295 +--------+ +--------+ 296 |Endpoint| |Endpoint| 297 +--+-----+ >>> circuit breaker traffic >>> +--+-----+ 298 | | 299 | +-+ +-+ +---------+ +-+ +-+ +-+ +--------+ +-+ +-+ | 300 +-+R+--+R+->+ Ingress +--+R+--+R+--+R+--+ Egress |--+R+--+R+-+ 301 +++ +-+ +------+--+ +-+ +-+ +-+ +-----+--+ +++ +-+ 302 | ^ | | | 303 | | +--+------+ +------+--+ | 304 | | | Ingress | | Egress | | 305 | | | Meter | | Meter | | 306 | | +----+----+ +----+----+ | 307 | | | | | 308 +-+ | | +----+----+ | | +-+ 309 |R+--+ | | Measure +<----------------+ +--+R| 310 +++ | +----+----+ Reported +++ 311 | | | Egress | 312 | | +----+----+ Measurement | 313 +--+-----+ | | Trigger + +--+-----+ 314 |Endpoint| | +----+----+ |Endpoint| 315 +--------+ | | +--------+ 316 +---<---+ 317 Reaction 319 Figure 1: A CB controlling the part of the end-to-end path between an 320 ingress point and an egress point. (Note: In some cases, the trigger 321 and measure functions could alternatively be located at other 322 locations (e.g., at a network operations centre.) 324 In the context of a Circuit Breaker, the ingress and egress functions 325 could be implemented in different places. For example, they could be 326 located in network devices at a tunnel ingress and at the tunnel 327 egress. In some cases, they could be located at one or both network 328 endpoints (see figure 2), implemented as components within a 329 transport protocol. 331 +----------+ +----------+ 332 | Ingress | +-+ +-+ +-+ | Egress | 333 | Endpoint +->+R+--+R+--+R+--+ Endpoint | 334 +--+----+--+ +-+ +-+ +-+ +----+-----+ 335 ^ | | 336 | +--+------+ +----+----+ 337 | | Ingress | | Egress | 338 | | Meter | | Meter | 339 | +----+----+ +----+----+ 340 | | | 341 | +--- +----+ | 342 | | Measure +<-----------------+ 343 | +----+----+ Reported 344 | | Egress 345 | +----+----+ Measurement 346 | | Trigger | 347 | +----+----+ 348 | | 349 +---<--+ 350 Reaction 352 Figure 2: An endpoint CB implemented at the sender (ingress) and 353 receiver (egress). 355 The set of components needed to implement a Circuit Breaker are: 357 1. An ingress meter (at the sender or tunnel ingress) that records 358 the number of packets/bytes sent in each measurement interval. 359 This measures the offered network load for a flow or set of 360 flows. For example, the measurement interval could be many 361 seconds (or every few tens of seconds or a series of successive 362 shorter measurements that are combined by the Circuit Breaker 363 Measurement function). 365 2. An egress meter (at the receiver or tunnel egress) that records 366 the number/bytes received in each measurement interval. This 367 measures the supported load for the flow or set of flows, and 368 could utilize other signals to detect the effect of congestion 369 (e.g., loss/congestion marking [RFC3168] experienced over the 370 path). The measurements at the egress could be synchronised 371 (including an offset for the time of flight of the data, or 372 referencing the measurements to a particular packet) to ensure 373 any counters refer to the same span of packets. 375 3. A method that communicates the measured values at the ingress and 376 egress to the Circuit Breaker Measurement function. This could 377 use several methods including: Sending return measurement packets 378 from a receiver to a trigger function at the sender; an 379 implementation using Operations, Administration and Management 380 (OAM); or sending an in-band signalling datagram to the trigger 381 function. This could also be implemented purely as a control 382 plane function, e.g., using a software-defined network 383 controller. 385 4. A measurement function that combines the ingress and egress 386 measurements to assess the present level of network congestion. 387 (For example, the loss rate for each measurement interval could 388 be deduced from calculating the difference between ingress and 389 egress counter values.) Note the method does not require high 390 accuracy for the period of the measurement interval (or therefore 391 the measured value, since isolated and/or infrequent loss events 392 need to be disregarded.) 394 5. A trigger function that determines whether the measurements 395 indicate persistent excessive congestion. This function defines 396 an appropriate threshold for determining that there is persistent 397 excessive congestion between the ingress and egress. This 398 preferably considers a rate or ratio, rather than an absolute 399 value (e.g., more than 10% loss, but other methods could also be 400 based on the rate of transmission as well as the loss rate). The 401 transport Circuit Breaker is triggered when the threshold is 402 exceeded in multiple measurement intervals (e.g., 3 successive 403 measurements). Designs need to be robust so that single or 404 spurious events do not trigger a reaction. 406 6. A reaction that is applied at the Ingress when the Circuit 407 Breaker is triggered. This seeks to automatically remove the 408 traffic causing persistent excessive congestion. 410 7. A feedback mechanism that triggers when either the receive or 411 ingress and egress measurements are not available, since this 412 also could indicate a loss of control packets (also a symptom of 413 heavy congestion or inability to control the load). 415 3.2. Other network topologies 417 A Circuit Breaker can be deployed in networks with topologies 418 different to that presented in figures 1 and 2. This section 419 describes examples of such usage, and possible places where functions 420 may be implemented. 422 3.2.1. Use with a multicast control/routing protocol 424 +----------+ +--------+ +----------+ 425 | Ingress | +-+ +-+ +-+ | Egress | | Egress | 426 | Endpoint +->+R+--+R+--+R+--+ Router |--+ Endpoint +->+ 427 +----+-----+ +-+ +-+ +-+ +---+--+-+ +----+-----+ | 428 ^ ^ ^ ^ | ^ | | 429 | | | | | | | | 430 +----+----+ + - - - < - - - - + | +----+----+ | Reported 431 | Ingress | multicast Prune | | Egress | | Ingress 432 | Meter | | | Meter | | Measurement 433 +---------+ | +----+----+ | 434 | | | 435 | +----+----+ | 436 | | Measure +<--+ 437 | +----+----+ 438 | | 439 | +----+----+ 440 multicast | | Trigger | 441 Leave | +----+----+ 442 Message | | 443 +----<----+ 445 Figure 3: An example of a multicast CB controlling the end-to-end 446 path between an ingress endpoint and an egress endpoint. 448 Figure 3 shows one example of how a multicast Circuit Breaker could 449 be implemented at a pair of multicast endpoints (e.g., to implement a 450 Fast-Trip Circuit Breaker, Section 5.1). The ingress endpoint (the 451 sender that sources the multicast traffic) meters the ingress load, 452 generating an ingress measurement (e.g., recording timestamped packet 453 counts), and sends this measurement to the multicast group together 454 with the traffic it has measured. 456 Routers along a multicast path forward the multicast traffic 457 (including the ingress measurement) to all active endpoint receivers. 458 Each last hop (egress) router forwards the traffic to one or more 459 egress endpoint(s). 461 In this figure, each endpoint includes a meter that performs a local 462 egress load measurement. An endpoint also extracts the received 463 ingress measurement from the traffic, and compares the ingress and 464 egress measurements to determine if the Circuit Breaker ought to be 465 triggered. This measurement has to be robust to loss (see previous 466 section). If the Circuit Breaker is triggered, it generates a 467 multicast leave message for the egress (e.g., an IGMP or MLD message 468 sent to the last hop router), which causes the upstream router to 469 cease forwarding traffic to the egress endpoint. 471 Any multicast router that has no active receivers for a particular 472 multicast group will prune traffic for that group, sending a prune 473 message to its upstream router. This starts the process of releasing 474 the capacity used by the traffic and is a standard multicast routing 475 function (e.g., using Protocol Independent Multicast Sparse Mode 476 (PIM-SM) routing protocol [RFC4601]). Each egress operates 477 autonomously, and the Circuit Breaker "reaction" is executed by the 478 multicast control plane (e.g., by PIM) requiring no explicit 479 signalling by the Circuit Breaker along the communication path used 480 for the control messages. Note: there is no direct communication 481 with the Ingress, and hence a triggered Circuit Breaker only controls 482 traffic downstream of the first hop multicast router. It does not 483 stop traffic flowing from the sender to the first hop router; this is 484 common practice for multicast deployment. 486 The method could also be used with a multicast tunnel or subnetwork 487 (e.g., Section 5.2, Section 5.3), where a meter at the ingress 488 generates additional control messages to carry the measurement data 489 towards the egress where the egress metering is implemented. 491 3.2.2. Use with control protocols supporting pre-provisioned capacity 493 Some paths are provisioned using a control protocol, e.g., flows 494 provisioned using the Multi-Protocol Label Switching (MPLS) services, 495 paths provisioned using the resource reservation protocol (RSVP), 496 networks utilizing Software Defined Network (SDN) functions, or 497 admission-controlled Differentiated Services. Figure 1 shows one 498 expected use case, where in this usage a separate device could be 499 used to perform the measurement and trigger functions. The reaction 500 generated by the trigger could take the form of a network control 501 message sent to the ingress and/or other network elements causing 502 these elements to react to the Circuit Breaker. Examples of this 503 type of use are provided in section Section 5.3. 505 3.2.3. Unidirectional Circuit Breakers over Controlled Paths 507 A Circuit Breaker can be used to control uni-directional UDP traffic, 508 providing that there is a communication path that can be used for 509 control messages to connect the functional components at the Ingress 510 and Egress. This communication path for the control messages can 511 exist in networks for which the traffic flow is purely 512 unidirectional. For example, a multicast stream that sends packets 513 across an Internet path and can use multicast routing to prune flows 514 to shed network load. Some other types of subnetwork also utilize 515 control protocols that can be used to control traffic flows. 517 4. Requirements for a Network Transport Circuit Breaker 519 The requirements for implementing a Circuit Breaker are: 521 1. There needs to be a communication path used for control messages 522 from the ingress meter and the egress meter to the point of 523 measurement. The CB MUST trigger if there is a failure of the 524 communication path used for the control messages. That is, the 525 feedback indicating a congested period needs to be designed so 526 that the CB is triggered when it fails to receive measurement 527 reports that indicate an absence of congestion, rather than 528 relying on the successful transmission of a "congested" signal 529 back to the sender. (The feedback signal could itself be lost 530 under congestion). 532 2. A CB is REQUIRED to define a measurement period over which the 533 CB Measurement function measures the level of congestion or 534 loss. This method does not have to detect individual packet 535 loss, but MUST have a way to know that packets have been lost/ 536 marked from the traffic flow. 538 3. An egress meter can also count Explicit Congestion Notification 539 (ECN) [RFC3168] congestion marks as a part of measurement of 540 congestion, but in this case, loss MUST also be measured to 541 provide a complete view of the level of congestion. For 542 tunnels, [ID-ietf-tsvwg-tunnel-congestion-feedback] describes a 543 way to measure both loss and ECN-marking; these measurements 544 could be used on a relatively short timescale to drive a 545 congestion control response and/or aggregated over a longer 546 timescale with a higher trigger threshold to drive a CB. 547 Subsequent bullet items in this section discuss the necessity of 548 using a longer timescale and a higher trigger threshold. 550 4. The measurement period used by a CB Measurement function MUST be 551 longer than the time that current Congestion Control algorithms 552 need to reduce their rate following detection of congestion. 553 This is important because end-to-end Congestion Control 554 algorithms require at least one RTT to notify and adjust the 555 traffic when congestion is experienced, and congestion 556 bottlenecks can share traffic with a diverse range of RTTs. The 557 measurement period is therefore expected to be significantly 558 longer than the RTT experienced by the CB itself. 560 5. If necessary, a CB MAY combine successive individual meter 561 samples from the ingress and egress to ensure observation of an 562 average measurement over a sufficiently long interval. (Note 563 when meter samples need to be combined, the combination needs to 564 reflect the sum of the individual sample counts divided by the 565 total time/volume over which the samples were measured. 566 Individual samples over different intervals can not be directly 567 combined to generate an average value.) 569 6. A CB is REQUIRED to define a threshold to determine whether the 570 measured congestion is considered excessive. 572 7. A CB is REQUIRED to define the triggering interval, defining the 573 period over which the trigger uses the collected measurements. 574 CBs need to trigger over a sufficiently long period to avoid 575 additionally penalizing flows with a long path RTT (e.g., many 576 path RTTs). 578 8. A CB MUST be robust to multiple congestion events. This usually 579 will define a number of measured persistent congestion events 580 per triggering period. For example, a CB MAY combine the 581 results of several measurement periods to determine if the CB is 582 triggered (e.g., it is triggered when persistent excessive 583 congestion is detected in 3 of the measurements within the 584 triggering interval). 586 9. A CB MUST be constructed so that it does not trigger under light 587 or intermittent congestion. 589 10. The default response to a trigger SHOULD disable all traffic 590 that contributed to congestion. 592 11. Once triggered, the CB MUST react decisively by disabling or 593 significantly reducing traffic at the source (e.g., ingress). 595 12. The reaction MUST be much more severe than that of a Congestion 596 Control algorithm (such as TCP's congestion control [RFC5681] or 597 TCP-Friendly Rate Control, TFRC [RFC5348]), because the CB 598 reacts to more persistent congestion and operates over longer 599 timescales (i.e., the overload condition will have persisted for 600 a longer time before the CB is triggered). 602 13. A reaction that results in a reduction SHOULD result in reducing 603 the traffic by at least an order of magnitude. A response that 604 achieves the reduction by terminating flows, rather than 605 randomly dropping packets, will often be more desirable to users 606 of the service. A CB that reduces the rate of a flow, MUST 607 continue to monitor the level of congestion and MUST further 608 react to reduce the rate if the CB is again triggered. 610 14. The reaction to a triggered CB MUST continue for a period that 611 is at least the triggering interval. Operator intervention will 612 usually be required to restore a flow. If an automated response 613 is needed to reset the trigger, then this needs to not be 614 immediate. The design of an automated reset mechanism needs to 615 be sufficiently conservative that it does not adversely interact 616 with other mechanisms (including other CB algorithms that 617 control traffic over a common path). It SHOULD NOT perform an 618 automated reset when there is evidence of continued congestion. 620 15. When a CB is triggered, it SHOULD be regarded as an abnormal 621 network event. As such, this event SHOULD be logged. The 622 measurements that lead to triggering of the CB SHOULD also be 623 logged. 625 16. A CB requires control communication between endpoints and/or 626 network devices, this results in security requirements 627 (Section 7). The authenticity of the source and integrity of 628 the control messages (measurements and triggers) MUST be 629 protected from off-path attacks. When there is a risk of on- 630 path attack, a cryptographic authentication mechanism for all 631 control/measurement messages is RECOMMENDED. 633 17. The Circuit Breaker MUST be designed to be robust to packet loss 634 that can also be experienced during congestion/overload. This 635 does not imply that it is desirable to provide reliable delivery 636 (e.g., over TCP), since this can incur additional delay in 637 responding to congestion. Appropriate mechanisms could be to 638 duplicate control messages to provide increased robustness to 639 loss, or/and to regard a lack of control traffic as an 640 indication that excessive congestion may be being experienced 641 [ID-ietf-tsvwg-RFC5405.bis]. 643 18. The control communication may be in-band or out-of-band. The 644 use of in-band communication is RECOMMENDED when either design 645 would be possible. If this traffic is sent over a shared path, 646 it is RECOMMENDED that this control traffic is prioritized to 647 reduce the probability of loss under congestion. Control 648 traffic also needs to be considered when provisioning a network 649 that uses a Circuit Breaker. 651 in-Band: An in-band control method SHOULD assume that loss of 652 control messages is an indication of potential congestion on 653 the path, and repeated loss ought to cause the CB to be 654 triggered. This design has the advantage that it provides 655 fate-sharing of the traffic flow(s) and the control 656 communications. This fate-sharing property is weaker when 657 some or all of the measured traffic is sent using a path that 658 differs from the path taken by the control traffic (e.g., 659 where traffic follows a different path due to use of equal- 660 cost multipath routing, traffic engineering, or tunnels for 661 specific types of traffic). 663 Out-of-Band: An out-of-band control method SHOULD NOT trigger 664 CB reaction when there is loss of control messages (e.g., a 665 loss of measurements). This avoids failure amplification/ 666 propagation when the measurement and data paths fail 667 independently. A failure of an out-of-band communication 668 path SHOULD be regarded as abnormal network event and be 669 handled as appropriate for the network, e.g., this event 670 SHOULD be logged, and additional network operator action 671 might be appropriate, depending on the network and the 672 traffic involved. 674 5. Examples of Circuit Breakers 676 There are multiple types of Circuit Breaker that could be defined for 677 use in different deployment cases. This section provides examples of 678 different types of Circuit Breaker: 680 5.1. A Fast-Trip Circuit Breaker 682 [RFC2309] discusses the dangers of congestion-unresponsive flows and 683 states that "all UDP-based streaming applications should incorporate 684 effective congestion avoidance mechanisms". Applications that do not 685 use a full-featured transport (TCP, SCTP, DCCP), (e.g., those using 686 UDP and its UDP-Lite variant) need to provide appropriate congestion 687 avoidance. Guidance for applications that do not use congestion- 688 controlled transports is provided in [ID-ietf-tsvwg-RFC5405.bis]. 689 Such mechanisms can be designed to react on much shorter timescales 690 than a Circuit Breaker, that only observes a traffic envelope. 691 Congestion control methods can also interact with an application to 692 more effectively control its sending rate. 694 A fast-trip Circuit Breaker is the most responsive form of Circuit 695 Breaker. It has a response time that is only slightly larger than 696 that of the traffic that it controls. It is suited to traffic with 697 well-understood characteristics (and could include one or more 698 trigger functions specifically tailored the type of traffic for which 699 it is designed). It is not suited to arbitrary network traffic and 700 may be unsuitable for traffic aggregates, since it could prematurely 701 trigger (e.g., when the combined traffic from multiple congestion- 702 controlled flows leads to short-term overload). 704 Although the mechanisms can be implemented in RTP-aware network 705 devices, these mechanisms are also suitable for implementation in 706 endpoints (e.g., as a part of the transport system) where they can 707 also compliment end-to-end congestion control methods. A shorter 708 response time enables these mechanisms to triggers before other forms 709 of Circuit Breaker (e.g., Circuit Breakers operating on traffic 710 aggregates at a point along the network path). 712 5.1.1. A Fast-Trip Circuit Breaker for RTP 714 A set of fast-trip Circuit Breaker methods have been specified for 715 use together by a Real-time Transport Protocol (RTP) flow using the 716 RTP/AVP Profile [RTP-CB]. It is expected that, in the absence of 717 severe congestion, all RTP applications running on best-effort IP 718 networks will be able to run without triggering these Circuit 719 Breakers. A fast-trip RTP Circuit Breaker is therefore implemented 720 as a fail-safe that when triggered will terminate RTP traffic. 722 The sending endpoint monitors reception of in-band RTP Control 723 Protocol (RTCP) reception report blocks, as contained in SR or RR 724 packets, that convey reception quality feedback information. This is 725 used to measure (congestion) loss, possibly in combination with ECN 726 [RFC6679]. 728 The Circuit Breaker action (shutdown of the flow) is triggered when 729 any of the following trigger conditions are true: 731 1. An RTP Circuit Breaker triggers on reported lack of progress. 733 2. An RTP Circuit Breaker triggers when no receiver reports messages 734 are received. 736 3. An RTP Circuit Breaker triggers when the the long-term RTP 737 throughput (over many RTTs) ecxceeds a hard upper limit 738 determined by a method that esembles TCP-Friendly Rate Control 739 (TFRC). 741 4. An RTP Circuit Breaker includes the notion of Media Usability. 742 This Circuit Breaker is triggered when the quality of the 743 transported media falls below some required minimum acceptable 744 quality. 746 5.2. A Slow-trip Circuit Breaker 748 A slow-trip Circuit Breaker could be implemented in an endpoint or 749 network device. This type of Circuit Breaker is much slower at 750 responding to congestion than a fast-trip Circuit Breaker and is 751 expected to be more common. 753 One example where a slow-trip Circuit Breaker is needed is where 754 flows or traffic-aggregates use a tunnel or encapsulation and the 755 flows within the tunnel do not all support TCP-style congestion 756 control (e.g., TCP, SCTP, TFRC), see [ID-ietf-tsvwg-RFC5405.bis] 757 section 3.1.3. A use case is where tunnels are deployed in the 758 general Internet (rather than "controlled environments" within an 759 Internet service provider or enterprise network), especially when the 760 tunnel could need to cross a customer access router. 762 5.3. A Managed Circuit Breaker 764 A managed Circuit Breaker is implemented in the signalling protocol 765 or management plane that relates to the traffic aggregate being 766 controlled. This type of Circuit Breaker is typically applicable 767 when the deployment is within a "controlled environment". 769 A Circuit Breaker requires more than the ability to determine that a 770 network path is forwarding data, or to measure the rate of a path - 771 which are often normal network operational functions. There is an 772 additional need to determine a metric for congestion on the path and 773 to trigger a reaction when a threshold is crossed that indicates 774 persistent excessive congestion. 776 The control messages can use either in-band or out-of-band 777 communications. 779 5.3.1. A Managed Circuit Breaker for SAToP Pseudo-Wires 781 [RFC4553], SAToP Pseudo-Wires (PWE3), section 8 describes an example 782 of a managed Circuit Breaker for isochronous flows. 784 If such flows were to run over a pre-provisioned (e.g., Multi- 785 Protocol Label Switching, MPLS) infrastructure, then it could be 786 expected that the Pseudowire (PW) would not experience congestion, 787 because a flow is not expected to either increase (or decrease) their 788 rate. If, instead, PW traffic is multiplexed with other traffic over 789 the general Internet, it could experience congestion. [RFC4553] 790 states: "If SAToP PWs run over a PSN providing best-effort service, 791 they SHOULD monitor packet loss in order to detect "severe 792 congestion". The currently recommended measurement period is 1 793 second, and the trigger operates when there are more than three 794 measured Severely Errored Seconds (SES) within a period. If such a 795 condition is detected, a SAToP PW ought to shut down bidirectionally 796 for some period of time...". 798 The concept was that when the packet loss ratio (congestion) level 799 increased above a threshold, the PW was by default disabled. This 800 use case considered fixed-rate transmission, where the PW had no 801 reasonable way to shed load. 803 The trigger needs to be set at the rate that the PW was likely to 804 experience a serious problem, possibly making the service non- 805 compliant. At this point, triggering the Circuit Breaker would 806 remove the traffic preventing undue impact on congestion-responsive 807 traffic (e.g., TCP). Part of the rationale, was that high loss 808 ratios typically indicated that something was "broken" and ought to 809 have already resulted in operator intervention, and therefore need to 810 trigger this intervention. 812 An operator-based response to triggering of a Circuit Breaker 813 provides an opportunity for other action to restore the service 814 quality, e.g., by shedding other loads or assigning additional 815 capacity, or to consciously avoid reacting to the trigger while 816 engineering a solution to the problem. This could require the 817 trigger function to send a control message to a third location (e.g., 818 a network operations centre, NOC) that is responsible for operation 819 of the tunnel ingress, rather than the tunnel ingress itself. 821 5.3.2. A Managed Circuit Breaker for Pseudowires (PWs) 823 Pseudowires (PWs) [RFC3985] have become a common mechanism for 824 tunneling traffic, and may compete for network resources both with 825 other PWs and with non-PW traffic, such as TCP/IP flows. 827 [ID-ietf-pals-congcons] discusses congestion conditions that can 828 arise when PWs compete with elastic (i.e., congestion responsive) 829 network traffic (e.g, TCP traffic). Elastic PWs carrying IP traffic 830 (see [RFC4488]) do not raise major concerns because all of the 831 traffic involved responds, reducing the transmission rate when 832 network congestion is detected. 834 In contrast, inelastic PWs (e.g., a fixed bandwidth Time Division 835 Multiplex, TDM) [RFC4553] [RFC5086] [RFC5087]) have the potential to 836 harm congestion responsive traffic or to contribute to excessive 837 congestion because inelastic PWs do not adjust their transmission 838 rate in response to congestion. [ID-ietf-pals-congcons] analyses TDM 839 PWs, with an initial conclusion that a TDM PW operating with a degree 840 of loss that may result in congestion-related problems is also 841 operating with a degree of loss that results in an unacceptable TDM 842 service. For that reason, the document suggests that a managed 843 Circuit Breaker that shuts down a PW when it persistently fails to 844 deliver acceptable TDM service is a useful means for addressing these 845 congestion concerns. 847 6. Examples where circuit breakers may not be needed. 849 A Circuit Breaker is not required for a single congestion-controlled 850 flow using TCP, SCTP, TFRC, etc. In these cases, the congestion 851 control methods are already designed to prevent persistent excessive 852 congestion. 854 6.1. CBs over pre-provisioned Capacity 856 One common question is whether a Circuit Breaker is needed when a 857 tunnel is deployed in a private network with pre-provisioned 858 capacity. 860 In this case, compliant traffic that does not exceed the provisioned 861 capacity ought not to result in persistent congestion. A Circuit 862 Breaker will hence only be triggered when there is non-compliant 863 traffic. It could be argued that this event ought never to happen - 864 but it could also be argued that the Circuit Breaker equally ought 865 never to be triggered. If a Circuit Breaker were to be implemented, 866 it will provide an appropriate response if persistent congestion 867 occurs in an operational network. 869 Implementing a Circuit Breaker will not reduce the performance of the 870 flows, but in the event that persistent excessive congestion occurs 871 it protects network traffic that shares network capacity with these 872 flows. A Circuit Breaker also protects network traffic using a non- 873 pre-provisioned path from a failure caused by additional netwokr load 874 resulting when Circuit Breaker traffic is routed over this path. 876 6.2. CBs with tunnels carrying Congestion-Controlled Traffic 878 IP-based traffic is generally assumed to be congestion-controlled, 879 i.e., it is assumed that the transport protocols generating IP-based 880 traffic at the sender already employ mechanisms that are sufficient 881 to address congestion on the path. A question therefore arises when 882 people deploy a tunnel that is thought to only carry an aggregate of 883 TCP traffic (or traffic using some other congestion control method): 884 Is there advantage in this case in using a Circuit Breaker? 886 TCP (and SCTP) traffic in a tunnel is expected to reduce the 887 transmission rate when network congestion is detected. Other 888 transports (e.g, using UDP) can employ mechanisms that are sufficient 889 to address congestion on the path [ID-ietf-tsvwg-RFC5405.bis]. Even 890 when all transports sharing a tunnel individually reduce their 891 transmission rate when network congestion is detected, the answer to 892 the question is not always obvious. The overall traffic formed by an 893 aggregate of flows that implement a congestion control mechanism does 894 not necessarily prevent persistent congestion. For instance, most 895 congestion control mechanisms require long-lived flows to react to 896 reduce the rate of a flow. An aggregate of many short flows could 897 result in many terminating before they experience congestion. It is 898 also often impossible for a tunnel service provider to know that the 899 tunnel only contains congestion-controlled traffic (e.g., Inspecting 900 packet headers could not be possible). Some IP-based applications 901 may be implement adequate mechanisms to address congestion. The 902 important thing to note is that if the aggregate of the traffic does 903 not result in persistent excessive congestion (impacting other 904 flows), then the Circuit Breaker will not trigger. This is the 905 expected case in this context - so implementing a Circuit Breaker 906 will not reduce performance of the tunnel, but in the event that 907 persistent excessive congestion occurs the Circuit Breaker protects 908 other network traffic that shares capacity with the tunnel traffic. 910 6.3. CBs with Uni-directional Traffic and no Control Path 912 A one-way forwarding path could have no associated communication path 913 for sending control messages, and therefore cannot be controlled 914 using a Circuit Breaker (compare with Section 3.2.3). 916 A one-way service could be provided using a path that has dedicated 917 capacity and does not share this capacity with other elastic Internet 918 flows (i.e., flows that vary their rate). One way to mitigate the 919 impact on other flows when capacity is shared is to manage the 920 traffic envelope by using ingress policing. Supporting this type of 921 traffic in the general Internet requires operator monitoring to 922 detect and respond to persistent excessive congestion. 924 7. Security Considerations 926 All Circuit Breaker mechanisms rely upon coordination between the 927 ingress and egress meters and communication with the trigger 928 function. This is usually achieved by passing network control 929 information (or protocol messages) across the network. Timely 930 operation of a Circuit Breaker depends on the choice of measurement 931 period. If the receiver has an interval that is overly long, then 932 the responsiveness of the Circuit Breaker decreases. This impacts 933 the ability of the Circuit Breaker to detect and react to congestion. 934 If the interval is too short the Circuit Breaker could trigger 935 prematurely resulting in insufficent time for other mechanisms to 936 act, potentially resulting in uneccessary disruption to the service. 938 A Circuit Breaker could potentially be exploited by an attacker to 939 mount a Denial of Service (DoS) attack against the traffic being 940 measured. Mechanisms therefore need to be implemented to prevent 941 attacks on the network control information that would result in DoS. 942 The authenticity of the source and integrity of the control messages 943 (measurements and triggers) MUST be protected from off-path attacks. 944 Without protection, it could be trivial for an attacker to inject 945 fake or modified control/measurement messages (e.g., indicating high 946 packet loss rates) causing a Circuit Breaker to trigger and to 947 therefore mount a DoS attack that disrupts a flow. 949 Simple protection can be provided by using a randomized source port, 950 or equivalent field in the packet header (such as the RTP SSRC value 951 and the RTP sequence number) expected not to be known to an off-path 952 attacker. Stronger protection can be achieved using a secure 953 authentication protocol. This attack is relatively easy for an on- 954 path attacker when the messages are neither encrypted nor 955 authenticated. When there is a risk of on-path attack, a 956 cryptographic authentication mechanism for all control/measurement 957 messages is RECOMMENDED to mitigate this concern. There is a design 958 trade-off between the cost of introducing cryptographic security for 959 control messages and the desire to protect control communication. 960 For some deployment scenarios the value of additional protection from 961 DoS attack will therefore lead to a requirement to authenticate all 962 control messages. 964 Transmission of network control messages consumes network capacity. 965 This control traffic needs to be considered in the design of a 966 Circuit Breaker and could potentially add to network congestion. If 967 this traffic is sent over a shared path, it is RECOMMENDED that this 968 control traffic is prioritized to reduce the probability of loss 969 under congestion. Control traffic also needs to be considered when 970 provisioning a network that uses a Circuit Breaker. 972 The Circuit Breaker MUST be designed to be robust to packet loss that 973 can also be experienced during congestion/overload. Loss of control 974 messages could be a side-effect of a congested network, but also 975 could arise from other causes Section 4. 977 The security implications depend on the design of the mechanisms, the 978 type of traffic being controlled and the intended deployment 979 scenario. Each design of a Circuit Breaker MUST therefore evaluate 980 whether the particular Circuit Breaker mechanism has new security 981 implications. 983 8. IANA Considerations 985 This document makes no request from IANA. 987 9. Acknowledgments 989 There are many people who have discussed and described the issues 990 that have motivated this document. Contributions and comments 991 included: Lars Eggert, Colin Perkins, David Black, Matt Mathis, 992 Andrew McGregor, Bob Briscoe and Eliot Lear. This work was part- 993 funded by the European Community under its Seventh Framework 994 Programme through the Reducing Internet Transport Latency (RITE) 995 project (ICT-317700). 997 10. Revision Notes 999 XXX RFC-Editor: Please remove this section prior to publication XXX 1001 Draft 00 1003 This was the first revision. Help and comments are greatly 1004 appreciated. 1006 Draft 01 1008 Contained clarifications and changes in response to received 1009 comments, plus addition of diagram and definitions. Comments are 1010 welcome. 1012 WG Draft 00 1014 Approved as a WG work item on 28th Aug 2014. 1016 WG Draft 01 1018 Incorporates feedback after Dallas IETF TSVWG meeting. This version 1019 is thought ready for WGLC comments. Definitions of abbreviations. 1021 WG Draft 02 1023 Minor fixes for typos. Rewritten security considerations section. 1025 WG Draft 03 1027 Updates following WGLC comments (see TSV mailing list). Comments 1028 from C Perkins; D Black and off-list feedback. 1030 A clear recommendation of intended scope. 1032 Changes include: Improvement of language on timescales and minimum 1033 measurement period; clearer articulation of endpoint and multicast 1034 examples - with new diagrams; separation of the controlled network 1035 case; updated text on position of trigger function; corrections to 1036 RTP-CB text; clarification of loss v ECN metrics; checks against 1037 submission checklist 9use of keywords, added meters to diagrams). 1039 WG Draft 04 1041 Added section on PW CB for TDM - a newly adopted draft (D. Black). 1043 WG Draft 05 1045 Added clarifications requested during AD review. 1047 WG Draft 06 1049 Fixed some remaining typos. 1051 Update following detailed review by Bob Briscoe, and comments by D. 1052 Black. 1054 WG Draft 07 1056 Additional update following review by Bob Briscoe. 1058 WG Draft 08 1060 Updated text on the response to lack of meter measurements with 1061 managed circuit breakers. Additional comments from Eliot Lear (APPs 1062 area). 1064 WG Draft 09 1066 Updated text on applications from Eliot Lear. Additional feedback 1067 from Bob Briscoe. 1069 WG Draft 10 1071 Updated text following comments by D Black including a rewritten ECN 1072 requirements bullet with of a reference to a tunnel measurement 1073 method in [ID-ietf-tsvwg-tunnel-congestion-feedback]. 1075 WG Draft 11 1077 Minor corrections after second WGLC. 1079 WG Draft 12 1081 Update following Gen-ART, RTG, and OPS review comments. 1083 WG Draft 13 1085 Fixed a typo. 1087 11. References 1089 11.1. Normative References 1091 [ID-ietf-tsvwg-RFC5405.bis] 1092 Eggert, L., Fairhurst, G., and G. Shepherd, "UDP Usage 1093 Guidelines (Work-in-Progress)", 2015. 1095 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1096 Requirement Levels", BCP 14, RFC 2119, 1097 DOI 10.17487/RFC2119, March 1997, 1098 . 1100 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1101 of Explicit Congestion Notification (ECN) to IP", 1102 RFC 3168, DOI 10.17487/RFC3168, September 2001, 1103 . 1105 11.2. Informative References 1107 [ID-ietf-pals-congcons] 1108 Stein, YJ., Black, D., and B. Briscoe, "Pseudowire 1109 Congestion Considerations (Work-in-Progress)", 2015. 1111 [ID-ietf-tsvwg-tunnel-congestion-feedback] 1112 Wei, X., Zhu, L., and L. Dend, "Tunnel Congestion Feedback 1113 (Work-in-Progress)", 2015. 1115 [Jacobsen88] 1116 European Telecommunication Standards, Institute (ETSI), 1117 "Congestion Avoidance and Control", SIGCOMM Symposium 1118 proceedings on Communications architectures and 1119 protocols", August 1998. 1121 [RFC1112] Deering, S., "Host extensions for IP multicasting", STD 5, 1122 RFC 1112, DOI 10.17487/RFC1112, August 1989, 1123 . 1125 [RFC2309] Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering, 1126 S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G., 1127 Partridge, C., Peterson, L., Ramakrishnan, K., Shenker, 1128 S., Wroclawski, J., and L. Zhang, "Recommendations on 1129 Queue Management and Congestion Avoidance in the 1130 Internet", RFC 2309, DOI 10.17487/RFC2309, April 1998, 1131 . 1133 [RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, 1134 RFC 2914, DOI 10.17487/RFC2914, September 2000, 1135 . 1137 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 1138 Edge-to-Edge (PWE3) Architecture", RFC 3985, 1139 DOI 10.17487/RFC3985, March 2005, 1140 . 1142 [RFC4488] Levin, O., "Suppression of Session Initiation Protocol 1143 (SIP) REFER Method Implicit Subscription", RFC 4488, 1144 DOI 10.17487/RFC4488, May 2006, 1145 . 1147 [RFC4553] Vainshtein, A., Ed. and YJ. Stein, Ed., "Structure- 1148 Agnostic Time Division Multiplexing (TDM) over Packet 1149 (SAToP)", RFC 4553, DOI 10.17487/RFC4553, June 2006, 1150 . 1152 [RFC4601] Fenner, B., Handley, M., Holbrook, H., and I. Kouvelas, 1153 "Protocol Independent Multicast - Sparse Mode (PIM-SM): 1154 Protocol Specification (Revised)", RFC 4601, 1155 DOI 10.17487/RFC4601, August 2006, 1156 . 1158 [RFC5086] Vainshtein, A., Ed., Sasson, I., Metz, E., Frost, T., and 1159 P. Pate, "Structure-Aware Time Division Multiplexed (TDM) 1160 Circuit Emulation Service over Packet Switched Network 1161 (CESoPSN)", RFC 5086, DOI 10.17487/RFC5086, December 2007, 1162 . 1164 [RFC5087] Stein, Y(J)., Shashoua, R., Insler, R., and M. Anavi, 1165 "Time Division Multiplexing over IP (TDMoIP)", RFC 5087, 1166 DOI 10.17487/RFC5087, December 2007, 1167 . 1169 [RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP 1170 Friendly Rate Control (TFRC): Protocol Specification", 1171 RFC 5348, DOI 10.17487/RFC5348, September 2008, 1172 . 1174 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 1175 Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, 1176 . 1178 [RFC6679] Westerlund, M., Johansson, I., Perkins, C., O'Hanlon, P., 1179 and K. Carlberg, "Explicit Congestion Notification (ECN) 1180 for RTP over UDP", RFC 6679, DOI 10.17487/RFC6679, August 1181 2012, . 1183 [RTP-CB] Perkins, C. and V. Singh, "Multimedia Congestion Control: 1184 Circuit Breakers for Unicast RTP Sessions", February 2014. 1186 Author's Address 1188 Godred Fairhurst 1189 University of Aberdeen 1190 School of Engineering 1191 Fraser Noble Building 1192 Aberdeen, Scotland AB24 3UE 1193 UK 1195 Email: gorry@erg.abdn.ac.uk 1196 URI: http://www.erg.abdn.ac.uk