idnits 2.17.1 draft-ietf-pcn-architecture-11.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** The document seems to lack a License Notice according IETF Trust Provisions of 28 Dec 2009, Section 6.b.i or Provisions of 12 Sep 2009 Section 6.b -- however, there's a paragraph with a matching beginning. Boilerplate error? (You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Feb 2009 rather than one of the newer Notices. See https://trustee.ietf.org/license-info/.) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document seems to contain a disclaimer for pre-RFC5378 work, and may have content which was first submitted before 10 November 2008. The disclaimer is necessary when there are original authors that you have been unable to contact, or if some do not wish to grant the BCP78 rights to the IETF Trust. If you are able to get all authors (current and original) to grant those rights, you can and should remove the disclaimer; otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (April 7, 2009) is 5470 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Congestion and Pre-Congestion Philip. Eardley (Editor) 3 Notification Working Group BT 4 Internet-Draft April 7, 2009 5 Intended status: Informational 6 Expires: October 9, 2009 8 Pre-Congestion Notification (PCN) Architecture 9 draft-ietf-pcn-architecture-11 11 Status of this Memo 13 This Internet-Draft is submitted to IETF in full conformance with the 14 provisions of BCP 78 and BCP 79. This document may contain material 15 from IETF Documents or IETF Contributions published or made publicly 16 available before November 10, 2008. The person(s) controlling the 17 copyright in some of this material may not have granted the IETF 18 Trust the right to allow modifications of such material outside the 19 IETF Standards Process. Without obtaining an adequate license from 20 the person(s) controlling the copyright in such materials, this 21 document may not be modified outside the IETF Standards Process, and 22 derivative works of it may not be created outside the IETF Standards 23 Process, except to format it for publication as an RFC or to 24 translate it into languages other than English. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF), its areas, and its working groups. Note that 28 other groups may also distribute working documents as Internet- 29 Drafts. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 The list of current Internet-Drafts can be accessed at 37 http://www.ietf.org/ietf/1id-abstracts.txt. 39 The list of Internet-Draft Shadow Directories can be accessed at 40 http://www.ietf.org/shadow.html. 42 This Internet-Draft will expire on October 9, 2009. 44 Copyright Notice 46 Copyright (c) 2009 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents in effect on the date of 51 publication of this document (http://trustee.ietf.org/license-info). 52 Please review these documents carefully, as they describe your rights 53 and restrictions with respect to this document. 55 Abstract 57 This document describes a general architecture for flow admission and 58 termination based on pre-congestion information in order to protect 59 the quality of service of established inelastic flows within a single 60 Diffserv domain. 62 Status 64 Table of Contents 66 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 67 1.1. Overview of PCN . . . . . . . . . . . . . . . . . . . . . 4 68 1.2. Example use case for PCN . . . . . . . . . . . . . . . . 4 69 1.3. Applicability of PCN . . . . . . . . . . . . . . . . . . 8 70 1.4. Documents about PCN . . . . . . . . . . . . . . . . . . . 9 71 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 10 72 3. High-level functional architecture . . . . . . . . . . . . . . 12 73 3.1. Flow admission . . . . . . . . . . . . . . . . . . . . . 14 74 3.2. Flow termination . . . . . . . . . . . . . . . . . . . . 15 75 3.3. Flow admission and/or flow termination when there are 76 only two PCN encoding states . . . . . . . . . . . . . . 16 77 3.4. Information transport . . . . . . . . . . . . . . . . . . 17 78 3.5. PCN-traffic . . . . . . . . . . . . . . . . . . . . . . . 17 79 3.6. Backwards compatibility . . . . . . . . . . . . . . . . . 18 80 4. Detailed Functional architecture . . . . . . . . . . . . . . . 19 81 4.1. PCN-interior-node functions . . . . . . . . . . . . . . . 20 82 4.2. PCN-ingress-node functions . . . . . . . . . . . . . . . 21 83 4.3. PCN-egress-node functions . . . . . . . . . . . . . . . . 22 84 4.4. Admission control functions . . . . . . . . . . . . . . . 22 85 4.5. Flow termination functions . . . . . . . . . . . . . . . 23 86 4.6. Addressing . . . . . . . . . . . . . . . . . . . . . . . 24 87 4.7. Tunnelling . . . . . . . . . . . . . . . . . . . . . . . 24 88 4.8. Fault handling . . . . . . . . . . . . . . . . . . . . . 26 89 5. Operations and Management . . . . . . . . . . . . . . . . . . 26 90 5.1. Configuration Operations and Management . . . . . . . . . 27 91 5.1.1. System options . . . . . . . . . . . . . . . . . . . . 27 92 5.1.2. Parameters . . . . . . . . . . . . . . . . . . . . . . 28 93 5.2. Performance & Provisioning Operations and Management . . 30 94 5.3. Accounting Operations and Management . . . . . . . . . . 31 95 5.4. Fault Operations and Management . . . . . . . . . . . . . 31 96 5.5. Security Operations and Management . . . . . . . . . . . 32 97 6. Applicability of PCN . . . . . . . . . . . . . . . . . . . . . 33 98 6.1. Benefits . . . . . . . . . . . . . . . . . . . . . . . . 33 99 6.2. Deployment scenarios . . . . . . . . . . . . . . . . . . 35 100 6.3. Assumptions and constraints on scope . . . . . . . . . . 36 101 6.3.1. Assumption 1: Trust and support of PCN - 102 controlled environment . . . . . . . . . . . . . . . . 37 103 6.3.2. Assumption 2: Real-time applications . . . . . . . . . 37 104 6.3.3. Assumption 3: Many flows and additional load . . . . . 38 105 6.3.4. Assumption 4: Emergency use out of scope . . . . . . . 38 106 6.4. Challenges . . . . . . . . . . . . . . . . . . . . . . . 39 107 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 41 108 8. Security considerations . . . . . . . . . . . . . . . . . . . 41 109 9. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 42 110 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 42 111 11. Comments Solicited (to be removed by RFC Editor) . . . . . . . 43 112 12. Changes (to be removed by RFC Editor) . . . . . . . . . . . . 43 113 12.1. Changes from -10 to -11 . . . . . . . . . . . . . . . . . 43 114 12.2. Changes from -09 to -10 . . . . . . . . . . . . . . . . . 44 115 12.3. Changes from -08 to -09 . . . . . . . . . . . . . . . . . 44 116 12.4. Changes from -07 to -08 . . . . . . . . . . . . . . . . . 44 117 12.5. Changes from -06 to -07 . . . . . . . . . . . . . . . . . 45 118 12.6. Changes from -05 to -06 . . . . . . . . . . . . . . . . . 45 119 12.7. Changes from -04 to -05 . . . . . . . . . . . . . . . . . 46 120 12.8. Changes from -03 to -04 . . . . . . . . . . . . . . . . . 46 121 12.9. Changes from -02 to -03 . . . . . . . . . . . . . . . . . 47 122 12.10. Changes from -01 to -02 . . . . . . . . . . . . . . . . . 48 123 12.11. Changes from -00 to -01 . . . . . . . . . . . . . . . . . 49 124 13. References . . . . . . . . . . . . . . . . . . . . . . . . . . 51 125 13.1. Normative References . . . . . . . . . . . . . . . . . . 51 126 13.2. Informative References . . . . . . . . . . . . . . . . . 51 127 Appendix A. Possible future work items . . . . . . . . . . . . . 55 128 A.1. Probing . . . . . . . . . . . . . . . . . . . . . . . . . 57 129 A.1.1. Introduction . . . . . . . . . . . . . . . . . . . . . 57 130 A.1.2. Probing functions . . . . . . . . . . . . . . . . . . 58 131 A.1.3. Discussion of rationale for probing, its downsides 132 and open issues . . . . . . . . . . . . . . . . . . . 58 133 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 61 135 1. Introduction 137 1.1. Overview of PCN 139 The objective of Pre-Congestion Notification (PCN) is to protect the 140 quality of service (QoS) of inelastic flows within a Diffserv domain, 141 in a simple, scalable and robust fashion. Two mechanisms are used: 142 admission control, to decide whether to admit or block a new flow 143 request, and (in abnormal circumstances) flow termination to decide 144 whether to terminate some of the existing flows. To achieve this, 145 the overall rate of PCN traffic is metered on every link in the 146 domain, and PCN packets are appropriately marked when certain 147 configured rates are exceeded. These configured rates are below the 148 rate of the link thus providing notification to boundary nodes about 149 overloads before any congestion occurs (hence "pre-congestion 150 notification"). The level of marking allows boundary nodes to make 151 decisions about whether to admit or terminate. 153 Within a PCN-domain, PCN-traffic is forwarded in a prioritised 154 Diffserv traffic class. Every link in the PCN-domain is configured 155 with two rates (PCN-threshold-rate and PCN-excess-rate). If the 156 overall rate of PCN-traffic on a link exceeds a configured rate, then 157 a PCN-interior-node marks PCN-packets appropriately. The PCN-egress- 158 nodes use this information to make admission control and flow 159 termination decisions. Flow admission control determines whether a 160 new flow can be admitted without any impact, in normal circumstances, 161 on the QoS of existing PCN-flows. However, in abnormal 162 circumstances, for instance a disaster affecting multiple nodes and 163 causing traffic re-routes, then the QoS on existing PCN-flows may 164 degrade even though care was exercised when admitting those flows. 165 The flow termination mechanism removes sufficient traffic in order to 166 protect the QoS of the remaining PCN-flows. All PCN-boundary-nodes 167 and PCN-interior-nodes are PCN-enabled and are trusted for correct 168 PCN operation. PCN-ingress-nodes police arriving packets to check 169 that they are part of an admitted PCN-flow that keeps within its 170 agreed flowspec, and hence they maintain per flow state. PCN- 171 interior-nodes meter all PCN traffic, and hence do not need to 172 maintain any per flow state. Decisions about flow admission and 173 termination are made for a particular pair of PCN-boundary-nodes, and 174 hence PCN-egress-nodes must be able to identify which PCN-ingress- 175 node sent each PCN-packet. 177 1.2. Example use case for PCN 179 This section outlines an end-to-end QoS scenario that uses the PCN 180 mechanisms within one domain. The parts outside the PCN-domain are 181 out of scope for PCN, but are included to help clarify how PCN could 182 be used. Note that the section is only an example - in particular 183 there are other possibilities (see later) for how the PCN-boundary- 184 nodes perform admission control and flow termination. 186 As a fundamental building block, each link of the PCN-domain operates 187 a [PCN08-2] (Figure 1): 189 o Threshold meter and marker, which marks all PCN-packets if the PCN 190 traffic rate is greater than a first configured rate, the PCN- 191 threshold-rate. The admission control mechanism limits the PCN- 192 traffic on each link to *roughly* its PCN-threshold-rate. 194 o Excess traffic meter and marker, which marks a proportion of PCN- 195 packets, such that the amount marked equals the traffic rate in 196 excess of a second configured rate, the PCN-excess-rate. The flow 197 termination mechanism limits the PCN-traffic on each link to 198 *roughly* its PCN-excess-rate. 200 Overall the aim is to give an "early warning" of potential congestion 201 before there is any significant build-up of PCN-packets in the queue 202 on the link; we term this "pre-congestion notification" by analogy 203 with ECN (Explicit Congestion Notification, [RFC3168]). Note that 204 the link only meters the bulk PCN-traffic (and not per flow). 206 == Metering & == 207 ==Marking behaviour== ==PCN mechanisms== 208 ^ 209 Rate of ^ 210 PCN-traffic on | 211 bottleneck link | 212 | 213 | Some pkts Terminate some 214 | excess-traffic-marked admitted flows 215 | & & 216 | Rest of pkts Block new flows 217 | threshold-marked 218 | 219 PCN-excess-rate -|------------------------------------------------ 220 (=PCN-supportable-rate)| 221 | All pkts Block new flows 222 | threshold-marked 223 | 224 PCN-threshold-rate -|------------------------------------------------ 225 (=PCN-admissible-rate)| 226 | No pkts Admit new flows 227 | PCN-marked 228 | 230 Figure 1: Example of how the PCN admission control and flow 231 termination mechanisms operate as the rate of PCN-traffic increases. 233 The two forms of PCN-marking are indicated by setting of the ECN and 234 DSCP (Differentiated Services Codepoint [RFC2474]) fields to known 235 values, which are configured for the domain. Thus the PCN-egress- 236 nodes can monitor the PCN-markings in order to measure the severity 237 of pre-congestion. In addition, the PCN-ingress-nodes need to set 238 the ECN and DSCP fields to that configured for an unmarked PCN- 239 packet, and the PCN-egress-nodes need to revert to values appropriate 240 outside the PCN-domain. 242 For admission control, we assume end-to-end RSVP signalling (Resource 243 Reservation Protocol) [RFC2205]) in this example. The PCN-domain is 244 a single RSVP hop. The PCN-domain operates Diffserv, and we assume 245 that PCN-traffic is scheduled with the expedited forwarding (EF) per- 246 hop behaviour, [RFC3246]. Hence the overall solution is in line with 247 the "IntServ over Diffserv" framework defined in [RFC2998], as shown 248 in Figure 2. 250 ___ ___ _______________________________________ ____ ___ 251 | | | | | PCN- PCN- PCN- | | | | | 252 | | | | |ingress interior egress| | | | | 253 | | | | | -node -nodes -node | | | | | 254 | | | | |-------+ +-------+ +-------+ +------| | | | | 255 | | | | | | | PCN | | PCN | | | | | | | 256 | |..| |..|Ingress|..|meter &|..|meter &|..|Egress|..| |..| | 257 | |..| |..|Policer|..|marker |..|marker |..|Meter |..| |..| | 258 | | | | |-------+ +-------+ +-------+ +------| | | | | 259 | | | | | \ / | | | | | 260 | | | | | \ / | | | | | 261 | | | | | \ PCN-feedback-information / | | | | | 262 | | | | | \ (for admission control) / | | | | | 263 | | | | | --<-----<----<----<-----<-- | | | | | 264 | | | | | PCN-feedback-information | | | | | 265 | | | | | (for flow termination) | | | | | 266 |___| |___| |_______________________________________| |____| |___| 268 Sx Access PCN-domain Access Rx 269 End Network Network End 270 Host Host 271 <---- signalling across PCN-domain---> 272 (for admission control & flow termination) 274 <-------------------end-to-end QoS signalling protocol---------------> 276 Figure 2: Example of possible overall QoS architecture 278 A source wanting to start a new QoS flow sends an RSVP PATH message. 279 Normal hop-by-hop IntServ [RFC1633] is used outside the PCN-domain 280 (we assume successfully). The PATH message travels across the PCN- 281 domain; the PCN-egress-node reads the PHOP object to discover the 282 specific PCN-ingress-node for this flow. The RESV message travels 283 back from the receiver, and triggers the PCN-egress-node to check 284 what fraction of the PCN-traffic, from the relevant PCN-ingress-node, 285 is currently being threshold-marked. It adds an object with this 286 information onto the RESV message, and hence the PCN-ingress-node 287 learns about the level of pre-congestion on the path. If this level 288 is below some threshold, then the PCN-ingress-node admits the new 289 flow into the PCN-domain. The RSVP message triggers the PCN-ingress- 290 node to install two normal IntServ items: five-tuple information, so 291 that it can subsequently identify data packets that are part of a 292 previously admitted PCN-flow; and a traffic profile, so that it can 293 police the flow to within its contract. Similarly, the RSVP message 294 triggers the PCN-egress-node to install five-tuple and PHOP 295 information, so that it can identify packets as part of a flow from a 296 specific PCN-ingress-node. 298 The flow termination mechanism may happen when some abnormal 299 circumstances causes a link to become so pre-congested that it 300 excess-traffic-marks (and perhaps also drops) PCN-packets. In this 301 example, when a PCN-egress-node observes such a packet it then, with 302 some probability, terminates this PCN-flow; the probability is 303 configured low enough to avoid over-termination and high enough to 304 ensure rapid termination of enough flows. It also informs the 305 relevant PCN-ingress-node, so it can block any further traffic on the 306 terminated flow. 308 1.3. Applicability of PCN 310 Compared with alternative QoS mechanisms, PCN has certain advantages 311 and disadvantages that will make it appropriate in particular 312 scenarios. For example, compared with hop-by-hop IntServ [RFC1633], 313 PCN only requires per flow state at the PCN-ingress-nodes. Compared 314 with the Diffserv architecture [RFC2475], an operator needs to be 315 less accurate and/or conservative in its prediction of the traffic 316 matrix. The Diffserv architecture's traffic conditioning agreements 317 are static and coarse; they are defined at subscription time, and 318 they are used (for instance) to limit the total traffic at each 319 ingress of the domain regardless of the egress for the traffic. On 320 the other hand, PCN firstly uses admission control based on 321 measurements of the current conditions between the specific pair of 322 PCN-boundary-nodes, and secondly, in case of a disaster, PCN protects 323 the QoS of most flows by terminating a few selected ones. 325 PCN's admission control is a measurement-based mechanism. Hence it 326 assumes that the present is a reasonable prediction of the future: 327 the network conditions are measured at the time of a new flow 328 request, but the actual network performance must be acceptable during 329 the call some time later. Hence PCN is unsuitable in several 330 circumstances: 332 o If the source adapts its bit rate dependent on the level of pre- 333 congestion, because then the aggregate traffic might become 334 unstable. The assumption in this document is that PCN-packets 335 come from real time applications generating inelastic traffic, 336 such as the Controlled Load Service, [RFC2211]. 338 o If a potential bottleneck link has capacity for only a few flows, 339 because then a new flow can move a link directly from no pre- 340 congestion to being so overloaded that it has to drop packets. 341 The assumption in this document is that this isn't a problem. 343 o If there is the danger of a "flash crowd" in which many admission 344 requests arrive within the reaction time of PCN's admission 345 mechanism, because then they all might get admitted and so 346 overload the network. The assumption in this document is that, if 347 it is necessary, then flash crowds are limited in some fashion 348 beyond the scope of this document, for instance by rate limiting 349 QoS requests. 351 The applicability of PCN is discussed further in Section 6. 353 1.4. Documents about PCN 355 The purpose of this document is to describe a general architecture 356 for flow admission and termination based on (pre-) congestion 357 information in order to protect the quality of service of flows 358 within a Diffserv domain. This document describes the PCN 359 architecture at a high level (Section 3) and in more detail (Section 360 4). It also defines some terminology, and considerations about 361 operations and management, and security. Section 6 considers the 362 applicability of PCN in more detail, covering its benefits, 363 deployment scenarios, assumptions and potential challenges. The 364 Appendix covers some potential future work items. 366 Aspects of PCN are also documented elsewhere: 368 o Metering and marking: [PCN08-2] standardises threshold metering 369 and marking, and excess traffic metering and marking. A PCN- 370 packet may be marked, depending on the metering results. 372 o Encoding: the "baseline" encoding is described in [PCN08-1], which 373 standardises two PCN encoding states (PCN-marked and not PCN- 374 marked), whilst (experimental) extensions to the baseline encoding 375 can provide three encoding states (threshold-marked, excess- 376 traffic-marked, not PCN-marked, or perhaps further encoding states 377 as suggested in [Westberg08]). Section 3.6 considers backwards 378 compatability of PCN encoding with ECN. 380 o PCN-boundary-node behaviour: how the PCN-boundary-nodes convert 381 the PCN-markings into decisions about flow admission and flow 382 termination, as described in Informational documents. The concept 383 is that the standardised metering and marking by PCN-nodes allows 384 several possible PCN-boundary-node behaviours. A number of 385 possibilities are outlined in this document; detailed descriptions 386 and comparisons are in [Charny07-1] and [Menth08-3]. 388 o Signalling between PCN-boundary-nodes: Signalling is needed to 389 transport PCN-feedback-information between the PCN-boundary-nodes 390 (in the example above, this is the fraction of traffic, between 391 the pair of PCN-boundary-nodes, that is PCN-marked). The exact 392 details vary for different PCN-boundary-node behaviours, and so 393 should be described in those documents. It may require an 394 extension to the signalling protocol - standardisation is out of 395 scope of the PCN WG. 397 o The interface by which the PCN-boundary-nodes learn identification 398 information about the admitted flows: the exact requirements vary 399 for different PCN-boundary-node behaviours and for different 400 signalling protocols, and so should be described in those 401 documents. They will be similar to those described in the example 402 above - a PCN-ingress-node needs to be able to identify that a 403 packet is part of a previously admitted flow (typically from its 404 five-tuple) and each PCN-boundary-node needs to be able to 405 identify the other PCN-boundary-node for the flow. 407 2. Terminology 409 o PCN-domain: a PCN-capable domain; a contiguous set of PCN-enabled 410 nodes that perform Diffserv scheduling [RFC2474]; the complete set 411 of PCN-nodes that in principle can, through PCN-marking packets, 412 influence decisions about flow admission and termination for the 413 PCN-domain; the PCN-domain includes the PCN-egress-nodes, which 414 measure these PCN-marks, and the PCN-ingress-nodes. 416 o PCN-boundary-node: a PCN-node that connects one PCN-domain to a 417 node either in another PCN-domain or in a non PCN-domain. 419 o PCN-interior-node: a node in a PCN-domain that is not a PCN- 420 boundary-node. 422 o PCN-node: a PCN-boundary-node or a PCN-interior-node 424 o PCN-egress-node: a PCN-boundary-node in its role in handling 425 traffic as it leaves a PCN-domain. 427 o PCN-ingress-node: a PCN-boundary-node in its role in handling 428 traffic as it enters a PCN-domain. 430 o PCN-traffic, PCN-packets, PCN-BA: a PCN-domain carries traffic of 431 different Diffserv behaviour aggregates (BAs) [RFC2474]. The 432 PCN-BA uses the PCN mechanisms to carry PCN-traffic and the 433 corresponding packets are PCN-packets. The same network will 434 carry traffic of other Diffserv BAs. The PCN-BA is distinguished 435 by a combination of the Diffserv codepoint (DSCP) and ECN fields. 437 o PCN-flow: the unit of PCN-traffic that the PCN-boundary-node 438 admits (or terminates); the unit could be a single microflow (as 439 defined in [RFC2474]) or some identifiable collection of 440 microflows. 442 o Pre-congestion: a condition of a link within a PCN-domain such 443 that the PCN-node performs PCN-marking, in order to provide an 444 "early warning" of potential congestion before there is any 445 significant build-up of PCN-packets in the real queue. (Hence, by 446 analogy with ECN we call our mechanism Pre-Congestion 447 Notification.) 449 o PCN-marking: the process of setting the header in a PCN-packet 450 based on defined rules, in reaction to pre-congestion; either 451 threshold-marking or excess-traffic-marking. 453 o PCN-threshold-rate: a reference rate configured for each link in 454 the PCN-domain, which is lower than the PCN-excess-rate. It is 455 used by a metering behaviour that determines whether a packet 456 should be PCN-marked with a first encoding, "threshold-marked". 458 o Threshold-metering: a metering behaviour that, if the PCN-traffic 459 exceeds the PCN-threshold-rate, indicates that all PCN-traffic is 460 to be threshold-marked. 462 o Threshold-marking: the setting of the header in a PCN-packet to a 463 specific encoding, based on indications from the threshold-meter. 465 o PCN-excess-rate: a reference rate configured for each link in the 466 PCN-domain, which is higher than the PCN-threshold-rate. It is 467 used by a metering behaviour that determines whether a packet 468 should be PCN-marked with a second encoding, "excess-traffic- 469 marked". 471 o Excess-traffic-metering: a metering behaviour that, if the PCN- 472 traffic exceeds the PCN-excess-rate, indicates that the amount of 473 PCN-traffic to be PCN-marked is equal to the amount in excess of 474 the PCN-excess-rate. 476 o Excess-traffic-marking: the setting of the header in a PCN-packet 477 to a specific encoding, based on indications from the excess- 478 traffic-meter. 480 o PCN-colouring: the process of setting the header in a PCN-packet 481 by a PCN-boundary-node; performed by a PCN-ingress-node so that 482 PCN-nodes can easily identify PCN-packets; performed by a PCN- 483 egress-node so that the header is appropriate for nodes beyond the 484 PCN-domain. 486 o Ingress-egress-aggregate: The collection of PCN-packets from all 487 PCN-flows that travel in one direction between a specific pair of 488 PCN-boundary-nodes. 490 o PCN-feedback-information: information signalled by a PCN-egress- 491 node to a PCN-ingress-node (or a central control node), which is 492 needed for the flow admission and flow termination mechanisms. 494 o PCN-admissible-rate: the rate of PCN-traffic on a link up to which 495 PCN admission control should accept new PCN-flows. 497 o PCN-supportable-rate: the rate of PCN-traffic on a link down to 498 which PCN flow termination should, if necessary, terminate already 499 admitted PCN-flows. 501 3. High-level functional architecture 503 The high-level approach is to split functionality between: 505 o PCN-interior-nodes 'inside' the PCN-domain, which monitor their 506 own state of pre-congestion and mark PCN-packets as appropriate. 507 They are not flow-aware, nor aware of ingress-egress-aggregates. 508 The functionality is also done by PCN-ingress-nodes for their 509 outgoing interfaces (ie those 'inside' the PCN-domain). 511 o PCN-boundary-nodes at the edge of the PCN-domain, which control 512 admission of new PCN-flows and termination of existing PCN-flows, 513 based on information from PCN-interior-nodes. This information is 514 in the form of the PCN-marked data packets (which are intercepted 515 by the PCN-egress-nodes) and not signalling messages. Generally 516 PCN-ingress-nodes are flow-aware. 518 The aim of this split is to keep the bulk of the network simple, 519 scalable and robust, whilst confining policy, application-level and 520 security interactions to the edge of the PCN-domain. For example the 521 lack of flow awareness means that the PCN-interior-nodes don't care 522 about the flow information associated with PCN-packets, nor do the 523 PCN-boundary-nodes care about which PCN-interior-nodes its ingress- 524 egress-aggregates traverse. 526 In order to generate information about the current state of the PCN- 527 domain, each PCN-node PCN-marks packets if it is "pre-congested". 528 Exactly when a PCN-node decides if it is "pre-congested" (the 529 algorithm) and exactly how packets are "PCN-marked" (the encoding) 530 will be defined in separate standards-track documents, but at a high 531 level it is as follows: 533 o the algorithms: a PCN-node meters the amount of PCN-traffic on 534 each one of its outgoing (or incoming) links. The measurement is 535 made as an aggregate of all PCN-packets, and not per flow. There 536 are two algorithms, one for threshold-metering and one for excess- 537 traffic-metering. The meters trigger PCN-marking as necessary. 539 o the encoding(s): a PCN-node PCN-marks a PCN-packet by modifying a 540 combination of the DSCP and ECN fields. In the "baseline" 541 encoding [PCN08-1], the ECN field is set to 11 and the DSCP is not 542 altered. Extension encodings may be defined that, at most, use a 543 second DSCP (eg as in [Moncaster08]) and/or set the ECN field to 544 values other than 11 (eg as in [Menth08-2]). 546 In a PCN-domain the operator may have two or three encoding states 547 available. The baseline encoding provides two encoding states (not 548 PCN-marked, PCN-marked), whilst extended encodings can provide three 549 encoding states (not PCN-marked, threshold-marked, excess-traffic- 550 marked). 552 An operator may choose to deploy either admission control or flow 553 termination or both. Although designed to work together, they are 554 independent mechanisms, and the use of one does not require or 555 prevent the use of the other. Three encoding states naturally allows 556 both flow admission and flow termination. If there are only two 557 encoding states, then there are several options - see Section 3.3. 559 The PCN-boundary-nodes monitor the PCN-marked packets in order to 560 extract information about the current state of the PCN-domain. Based 561 on this monitoring, a distributed decision is made about whether to 562 admit a prospective new flow or whether to terminate existing 563 flow(s). Sections 4.4 and 4.5 mention various possibilities for how 564 the functionality could be distributed. 566 PCN-metering and PCN-marking needs to be configured on all 567 (potentially pre-congested) links in the PCN-domain to ensure that 568 the PCN mechanisms protect all links. The actual functionality can 569 be configured on the outgoing or incoming interfaces of PCN-nodes - 570 or one algorithm could be configured on the outgoing interface and 571 the other on the incoming interface. The important point is that a 572 consistent choice is made across the PCN-domain to ensure that the 573 PCN mechanisms protect all links. See [PCN08-2] for further 574 discussion. 576 The objective of threshold-marking, as triggerd by the threshold- 577 metering algorithm, is to threshold-mark all PCN-packets whenever the 578 rate of PCN-packets is greater than some configured rate, the PCN- 579 threshold-rate. The objective of excess-traffic-metering, as 580 triggered by the excess-traffic-marking algorithm, is to excess- 581 traffic-mark PCN-packets at a rate equal to the difference between 582 the bit rate of PCN-packets and some configured rate, the PCN-excess- 583 rate. Note that this description reflects the overall intent of the 584 algorithms rather than their instantaneous behaviour, since the rate 585 measured at a particular moment depends on the detailed algorithm, 586 its implementation, and the traffic's variance as well as its rate 587 (eg marking may well continue after a recent overload even after the 588 instantaneous rate has dropped). The algorithms are specified in 589 [PCN08-2]. 591 Admission and termination approaches are detailed and compared in 592 [Charny07-1] and [Menth08-3]. The discussion below is just a brief 593 summary. Sections 3.1 and 3.2 assume there are three encoding states 594 available, whilst Section 3.3 assumes there are two encoding states 595 available. 597 From the perspective of the outside world, a PCN-domain essentially 598 looks like a Diffserv domain, but without the Diffserv architecture's 599 traffic conditioning agreements. PCN-traffic is either transported 600 across it transparently or policed at the PCN-ingress-node (ie 601 dropped or carried at a lower QoS). One difference is that PCN- 602 traffic has better QoS guarantees than normal Diffserv traffic, 603 because the PCN mechanisms better protect the QoS of admitted flows. 604 Another difference may occur in the rare circumstance when there is a 605 failure: on the one hand some PCN-flows may get terminated, but on 606 the other hand other flows will get their QoS restored. Non PCN- 607 traffic is treated transparently, ie the PCN-domain is a normal 608 Diffserv domain. 610 3.1. Flow admission 612 The objective of PCN's flow admission control mechanism is to limit 613 the PCN-traffic on each link in the PCN-domain to *roughly* its PCN- 614 admissible-rate, by admitting or blocking prospective new flows, in 615 order to protect the QoS of existing PCN-flows. With three encoding 616 states available, the PCN-threshold-rate is configured by the 617 operator as equal to the PCN-admissible-rate on each link. It is set 618 lower than the traffic rate at which the link becomes congested and 619 the node drops packets. 621 Exactly how the admission control decision is made will be defined 622 separately in informational documents. This document describes two 623 approaches (others might be possible): 625 o the PCN-egress-node measures (possibly as a moving average) the 626 fraction of the PCN-traffic that is threshold-marked. The 627 fraction is measured for a specific ingress-egress-aggregate. If 628 the fraction is below a threshold value then the new flow is 629 admitted, and if the fraction is above the threshold value then it 630 is blocked. The fraction could be measured as an EWMA 631 (exponentially weighted moving average), which has sometimes been 632 called the "congestion level estimate". 634 o the PCN-egress-node monitors PCN-traffic and if it receives one 635 (or several) threshold-marked packets, then the new flow is 636 blocked, otherwise it is admitted. One possibility may be to 637 react to the marking state of an initial flow set-up packet (eg 638 RSVP PATH). Another is that after one (or several) threshold- 639 marks then all flows are blocked until after a specific period of 640 no congestion. 642 Note that the admission control decision is made for a particular 643 pair of PCN-boundary-nodes. So it is quite possible for a new flow 644 to be admitted between one pair of PCN-boundary-nodes, whilst at the 645 same time another admission request is blocked between a different 646 pair of PCN-boundary-nodes. 648 3.2. Flow termination 650 The objective of PCN's flow termination mechanism is to limit the 651 PCN-traffic on each link to *roughly* its PCN-supportable-rate, by 652 terminating some existing PCN-flows, in order to protect the QoS of 653 the remaining PCN-flows. With three encoding states available, the 654 PCN-excess-rate is configured by the operator as equal to the PCN- 655 supportable-rate on each link. It may be set lower than the traffic 656 rate at which the link becomes congested and the node drops packets. 658 Exactly how the flow termination decision is made will be defined 659 separately in informational documents. This document describes 660 several approaches (others might be possible): 662 o In one approach the PCN-egress-node measures the rate of PCN- 663 traffic that is not excess-traffic-marked, which is the amount of 664 PCN-traffic that can actually be supported, and communicates this 665 to the PCN-ingress-node. Also the PCN-ingress-node measures the 666 rate of PCN-traffic that is destined for this specific PCN-egress- 667 node. The difference represents the excess amount that should be 668 terminated. 670 o Another approach instead measures the rate of excess-traffic- 671 marked traffic and terminates this amount of traffic. This 672 terminates less traffic than the previous bullet if some nodes are 673 dropping PCN-traffic. 675 o Another approach monitors PCN-packets and terminates some of the 676 PCN-flows that have an excess-traffic-marked packet. (If all such 677 flows were terminated, far too much traffic would be terminated, 678 so a random selection needs to be made from those with an excess- 679 traffic-marked packet, [Menth08-1].) 681 Since flow termination is designed for "abnormal" circumstances, it 682 is quite likely that some PCN-nodes are congested and hence packets 683 are being dropped and/or significantly queued. The flow termination 684 mechanism must accommodate this. 686 Note also that the termination control decision is made for a 687 particular pair of PCN-boundary-nodes. So it is quite possible for 688 PCN-flows to be terminated between one pair of PCN-boundary-nodes, 689 whilst at the same time none are terminated between a different pair 690 of PCN-boundary-nodes. 692 3.3. Flow admission and/or flow termination when there are only two PCN 693 encoding states 695 If a PCN-domain has only two encoding states available (PCN-marked 696 and not PCN-marked), ie it is using the baseline encoding [PCN08-1], 697 then an operator has three options (others might be possible): 699 o admission control only: PCN-marking means threshold-marking, ie 700 only the threshold-metering algorithm triggers PCN-marking. Only 701 PCN admission control is available. 703 o flow termination only: PCN-marking means excess-traffic-marking, 704 ie only the excess-traffic-metering algorithm triggers PCN- 705 marking. Only PCN termination control is available. 707 o both admission control and flow termination: only the excess- 708 traffic-metering algorithm triggers PCN-marking, however the 709 configured rate (PCN-excess-rate) is set equal to the PCN- 710 admissible-rate, as shown in Figure 3. [Charny07-2] describes how 711 both admission control and flow termination can be triggered in 712 this case and also gives some of the pros and cons of this 713 approach. The main downside is that admission control is less 714 accurate. 716 == Metering & == 717 ==Marking behaviour== ==PCN mechanisms== 718 ^ 719 Rate of ^ 720 PCN-traffic on | 721 bottleneck link | Terminate some 722 | admitted flows 723 | & 724 | Block new flows 725 | 726 | Some pkts 727 U*PCN-excess-rate -| excess-traffic-marked ----------------- 728 (=PCN-supportable-rate)| 729 | Block new flows 730 | 731 | 732 PCN-excess-rate -|------------------------------------------------ 733 (=PCN-admissible-rate)| 734 | No pkts Admit new flows 735 | PCN-marked 736 | 738 Figure 3: Schematic of how the PCN admission control and flow 739 termination mechanisms operate as the rate of PCN-traffic increases, 740 for a PCN-domain with two encoding states and using the approach of 741 [Charny07-2]. Note: U is a global parameter for all links in the 742 PCN-domain. 744 3.4. Information transport 746 The transport of pre-congestion information from a PCN-node to a PCN- 747 egress-node is through PCN-markings in data packet headers, ie "in- 748 band": no signalling protocol messaging is needed. Signalling is 749 needed to transport PCN-feedback-information, for example to convey 750 the fraction of PCN-marked traffic from a PCN-egress-node to the 751 relevant PCN-ingress-node. Exactly what information needs to be 752 transported will be described in future documents about possible 753 boundary mechanisms. The signalling could be done by an extension of 754 RSVP or NSIS, for instance; [Lefaucheur06] describes the extensions 755 needed for RSVP. 757 3.5. PCN-traffic 759 The following are some high-level points about how PCN works: 761 o There needs to be a way for a PCN-node to distinguish PCN-traffic 762 from other traffic. This is through a combination of the DSCP 763 field and/or ECN field. 765 o It is not advised to have non PCN-traffic that competes for the 766 same capacity as PCN-traffic but, if there is such traffic, there 767 needs to be a mechanism to limit it. "Capacity" means the 768 forwarding bandwidth on a link; "competes" means that non PCN- 769 packets will delay PCN-packets in the queue for the link. Hence 770 more non PCN-traffic results in poorer QoS for PCN. Further, the 771 unpredictable amount of non PCN-traffic makes the PCN mechanisms 772 less accurate and so reduces PCN's ability to protect the QoS of 773 admitted PCN-flows 775 o Two examples of such non PCN-traffic (ie that competes for the 776 same capacity as PCN-traffic) are: 778 1. traffic that is priority scheduled over PCN (perhaps a particular 779 application or an operator's control messages). 781 2. traffic that is scheduled at the same priority as PCN (for 782 example if the Voice-Admit codepoint is used for PCN-traffic 783 [PCN08-1] and there is non-PCN voice-admit traffic in the PCN- 784 domain). 786 o If there is such non PCN-traffic (ie that competes for the same 787 capacity as PCN-traffic), then PCN's mechanisms should take 788 account of it, in order to improve the accuracy of the decision 789 about whether to admit (or terminate) a PCN-flow. For example, 790 one mechanism is that such non PCN-traffic contributes to the PCN 791 meters (ie is metered by the threshold-marking and excess-traffic- 792 marking algorithms). 794 o There will be non PCN-traffic that doesn't compete for the same 795 capacity as PCN-traffic, because it is forwarded at lower 796 priority. Hence it shouldn't contribute to the PCN meters. 797 Examples are best effort and assured forwarding traffic. However, 798 a PCN-node should dedicate some capacity to lower priority traffic 799 so that it isn't starved. 801 o The document assumes that the PCN mechanisms are applied to a 802 single behaviour aggregate in the PCN-domain. However, it would 803 also be possible to apply them independently to more than one 804 behaviour aggregate, which are distinguished by DSCP. 806 3.6. Backwards compatibility 808 PCN specifies semantics for the ECN field that differ from the 809 default semantics of [RFC3168]. A particular PCN encoding scheme 810 needs to describe how it meets the guidelines of BCP 124 [RFC4774] 811 for specifying alternative semantics for the ECN field. In summary 812 the approach is to: 814 o use a DSCP to allow PCN-nodes to distinguish PCN-traffic that uses 815 the alternative ECN semantics; 817 o define these semantics for use within a controlled region, the 818 PCN-domain; 820 o take appropriate action if ECN capable, non-PCN traffic arrives at 821 a PCN-ingress-node with the DSCP used by PCN. 823 For the baseline encoding [PCN08-1], the 'appropriate action' is to 824 block ECN-capable traffic that uses the same DSCP as PCN from 825 entering the PCN-domain directly. Blocking means it is dropped or 826 downgraded to a lower priority behaviour aggregate, or alternatively 827 such traffic may be tunnelled through the PCN-domain. The reason 828 that 'appropriate action' is needed is that the PCN-egress-node 829 clears the ECN field to 00. 831 Extended encoding schemes may need to take different 'appropriate 832 action'. 834 4. Detailed Functional architecture 836 This section is intended to provide a systematic summary of the new 837 functional architecture in the PCN-domain. First it describes 838 functions needed at the three specific types of PCN-node; these are 839 data plane functions and are in addition to their normal router 840 functions. Then it describes further functionality needed for both 841 flow admission control and flow termination; these are signalling and 842 decision-making functions, and there are various possibilities for 843 where the functions are physically located. The section is split 844 into: 846 1. functions needed at PCN-interior-nodes 848 2. functions needed at PCN-ingress-nodes 850 3. functions needed at PCN-egress-nodes 852 4. other functions needed for flow admission control 854 5. other functions needed for flow termination control 856 Note: Probing is covered in the Appendix. 858 The section then discusses some other detailed topics: 860 1. addressing 862 2. tunnelling 864 3. fault handling 866 4.1. PCN-interior-node functions 868 Each link of the PCN-domain is configured with the following 869 functionality: 871 o Behaviour aggregate classification - determine whether an incoming 872 packet is a PCN-packet or not. 874 o PCN-meter - measure the 'amount of PCN-traffic'. The measurement 875 is made on the overall PCN-traffic, and not per flow. Algorithms 876 determine whether to indicate to the PCN-marking functionality 877 that packets should be PCN-marked. 879 o PCN-mark - as triggered by indications from the PCN-meter 880 functionality, if necessary PCN-mark packets wth the appropiate 881 encoding. 883 o Drop - if the queue overflows then naturally packets are dropped. 884 In addition, the link may be configured with a maximum rate for 885 PCN-traffic (below the physical link rate), above which PCN- 886 packets are dropped. 888 The functions are defined in [PCN08-2] and the baseline encoding in 889 [PCN08-1] (extended encodings are to be defined in other documents). 891 +---------+ Result 892 +->|Threshold|-------+ 893 | | Meter | | 894 | +---------+ V 895 +----------+ +- - - - -+ | +------+ 896 | BA | | | | | | Marked 897 Packet =>|Classifier|==>| Dropper |==?===============>|Marker|==> Packet 898 Stream | | | | | | | Stream 899 +----------+ +- - - - -+ | +------+ 900 | +---------+ ^ 901 | | Excess | | 902 +->| Traffic |-------+ 903 | Meter | Result 904 +---------+ 906 Figure 4: Schematic of PCN-interior-node functionality 908 4.2. PCN-ingress-node functions 910 Each ingress link of the PCN-domain is configured with the following 911 functionality: 913 o Packet classification - determine whether an incoming packet is 914 part of a previously admitted flow, by using a filter spec (eg 915 DSCP, source and destination addresses, port numbers, and 916 protocol). 918 o Police - police, by dropping, any packets received with a DSCP 919 indicating PCN transport that do not belong to an admitted flow. 920 (A prospective PCN-flow that is rejected could be blocked or 921 admitted into a lower priority behaviour aggregate.) Similarly, 922 police packets that are part of a previously admitted flow, to 923 check that the flow keeps to the agreed rate or flowspec (eg 924 [RFC1633] for a microflow and its NSIS equivalent). 926 o PCN-colour - set the DSCP and ECN fields appropriately for the 927 PCN-domain, for example as in [PCN08-1]. 929 o Meter - some approaches to flow termination require the PCN- 930 ingress-node to measure the (aggregate) rate of PCN-traffic 931 towards a particular PCN-egress-node. 933 The first two are policing functions, needed to make sure that PCN- 934 packets admitted into the PCN-domain belong to a flow that has been 935 admitted and to ensure that the flow keeps to the flowspec agreed (eg 936 doesn't exceed an agreed maximum rate and is inelastic traffic). 937 Installing the filter spec will typically be done by the signalling 938 protocol, as will re-installing the filter, for example after a re- 939 route that changes the PCN-ingress-node (see [Briscoe06] for an 940 example using RSVP). PCN-colouring allows the rest of the PCN-domain 941 to recognise PCN-packets. 943 4.3. PCN-egress-node functions 945 Each egress link of the PCN-domain is configured with the following 946 functionality: 948 o Packet classify - determine which PCN-ingress-node a PCN-packet 949 has come from. 951 o Meter - "measure PCN-traffic" or "monitor PCN-marks". 953 o PCN-colour - for PCN-packets, set the DSCP and ECN fields to the 954 appropriate values for use outside the PCN-domain. 956 The metering functionality of course depends on whether it is 957 targeted at admission control or flow termination. Alternatives 958 involve the PCN-egress-node "measuring" as an aggregate (ie not per 959 flow) all PCN-packets from a particular PCN-ingress-node, or 960 "monitoring" the PCN-traffic and reacting to one (or several) PCN- 961 marked packets. For PCN-colouring, [PCN08-1] specifies that the PCN- 962 egress-node re-sets the ECN field to 00; other encodings may define 963 different behaviour. 965 4.4. Admission control functions 967 As well as the functions covered above, other specific admission 968 control functions need to be performed (others might be possible): 970 o Make decision about admission - based on the output of the PCN- 971 egress-node's meter function. In the case where it "measures PCN- 972 traffic", the measured traffic on the ingress-egress-aggregate is 973 compared with some reference level. In the case where it 974 "monitors PCN-marks", then the decision is based on whether one 975 (or several) packets is (are) PCN-marked or not (eg the RSVP PATH 976 message). In either case, the admission decision also takes 977 account of policy and application layer requirements [RFC2753]. 979 o Communicate decision about admission - signal the decision to the 980 node making the admission control request (which may be outside 981 the PCN-domain), and to the policer (PCN-ingress-node function) 982 for enforcement of the decision. 984 There are various possibilities for how the functionality could be 985 distributed (we assume the operator would configure which is used): 987 o The decision is made at the PCN-egress-node and the decision 988 (admit or block) is signalled to the PCN-ingress-node. 990 o The decision is recommended by the PCN-egress-node (admit or 991 block) but the decision is definitively made by the PCN-ingress- 992 node. The rationale is that the PCN-egress-node naturally has the 993 necessary information about the amount of PCN-marks on the 994 ingress-egress-aggregate, but the PCN-ingress-node is the policy 995 enforcement point [RFC2753], which polices incoming traffic to 996 ensure it is part of an admitted PCN-flow. 998 o The decision is made at the PCN-ingress-node, which requires that 999 the PCN-egress-node signals PCN-feedback-information to the PCN- 1000 ingress-node. For example, it could signal the current fraction 1001 of PCN-traffic that is PCN-marked. 1003 o The decision is made at a centralised node (see Appendix). 1005 Note: Admission control functionality is not performed by normal PCN- 1006 interior-nodes. 1008 4.5. Flow termination functions 1010 As well as the functions covered above, other specific termination 1011 control functions need to be performed (others might be possible): 1013 o PCN-meter at PCN-egress-node - similarly to flow admission, there 1014 are two types of possibilities: to "measure PCN-traffic" on the 1015 ingress-egress-aggregate, and to "monitor PCN-marks" and react to 1016 one (or several) PCN-marks. 1018 o (if required) PCN-meter at PCN-ingress-node - make "measurements 1019 of PCN-traffic" being sent towards a particular PCN-egress-node; 1020 again, this is done for the ingress-egress-aggregate and not per 1021 flow. 1023 o (if required) Communicate PCN-feedback-information to the node 1024 that makes the flow termination decision. For example, as in 1025 [Briscoe06], communicate the PCN-egress-node's measurements to the 1026 PCN-ingress-node. 1028 o Make decision about flow termination - use the information from 1029 the PCN-meter(s) to decide which PCN-flow or PCN-flows to 1030 terminate. The decision takes account of policy and application 1031 layer requirements [RFC2753]. 1033 o Communicate decision about flow termination - signal the decision 1034 to the node that is able to terminate the flow (which may be 1035 outside the PCN-domain), and to the policer (PCN-ingress-node 1036 function) for enforcement of the decision. 1038 There are various possibilities for how the functionality could be 1039 distributed, similar to those discussed above in the Admission 1040 control section. 1042 Note: Flow termination functionality is not performed by normal PCN- 1043 interior-nodes. 1045 4.6. Addressing 1047 PCN-nodes may need to know the address of other PCN-nodes. Note: in 1048 all cases PCN-interior-nodes don't need to know the address of any 1049 other PCN-nodes (except as normal their next hop neighbours, for 1050 routing purposes). 1052 The PCN-egress-node needs to know the address of the PCN-ingress-node 1053 associated with a flow, at a minimum so that the PCN-ingress-node can 1054 be informed to enforce the admission decision (and any flow 1055 termination decision) through policing. There are various 1056 possibilities for how the PCN-egress-node can do this, ie associate 1057 the received packet to the correct ingress-egress-aggregate. It is 1058 not the intention of this document to mandate a particular mechanism. 1060 o The addressing information can be gathered from signalling. For 1061 example, regular processing of an RSVP PATH message, as the PCN- 1062 ingress-node is the previous RSVP hop (PHOP) ([Lefaucheur06]). Or 1063 the PCN-ingress-node could signal its address to the PCN-egress- 1064 node. 1066 o Always tunnel PCN-traffic across the PCN-domain. Then the PCN- 1067 ingress-node's address is simply the source address of the outer 1068 packet header. The PCN-ingress-node needs to learn the address of 1069 the PCN-egress-node, either by manual configuration or by one of 1070 the automated tunnel endpoint discovery mechanisms (such as 1071 signalling or probing over the data route, interrogating routing 1072 or using a centralised broker). 1074 4.7. Tunnelling 1076 Tunnels may originate and/or terminate within a PCN-domain (eg IP 1077 over IP, IP over MPLS). It is important that the PCN-marking of any 1078 packet can potentially influence PCN's flow admission control and 1079 termination - it shouldn't matter whether the packet happens to be 1080 tunnelled at the PCN-node that PCN-marks the packet, or indeed 1081 whether it's decapsulated or encapsulated by a subsequent PCN-node. 1082 This suggests that the "uniform conceptual model" described in 1084 [RFC2983] should be re-applied in the PCN context. In line with this 1085 and the approach of [RFC4303] and [Briscoe08-2], the following rule 1086 is applied if encapsulation is done within the PCN-domain: 1088 o any PCN-marking is copied into the outer header 1090 Note: A tunnel will not provide this behaviour if it complies with 1091 [RFC3168] tunnelling in either mode, but it will if it complies with 1092 [RFC4301] IPSec tunnelling. 1094 Similarly, in line with the "uniform conceptual model" of [RFC2983], 1095 the "full-functionality option" of [RFC3168], and [RFC4301], the 1096 following rule is applied if decapsulation is done within the PCN- 1097 domain: 1099 o if the outer header's marking state is more severe then it is 1100 copied onto the inner header. 1102 Note: the order of increasing severity is: not PCN-marked; threshold- 1103 marked; excess-traffic-marked. 1105 An operator may wish to tunnel PCN-traffic from PCN-ingress-nodes to 1106 PCN-egress-nodes. The PCN-marks shouldn't be visible outside the 1107 PCN-domain, which can be achieved by the PCN-egress-node doing the 1108 PCN-colouring function (Section 4.3) after all the other (PCN and 1109 tunnelling) functions. The potential reasons for doing such 1110 tunnelling are: the PCN-egress-node then automatically knows the 1111 address of the relevant PCN-ingress-node for a flow; even if ECMP is 1112 running, all PCN-packets on a particular ingress-egress-aggregate 1113 follow the same path. (ECMP: Equal Cost Multi-Path, Section 6.4.) 1114 But it also has drawbacks, for example the additional overhead in 1115 terms of bandwidth and processing, and the cost of setting up a mesh 1116 of tunnels between PCN-boundary-nodes (there is an N^2 scaling 1117 issue). 1119 Potential issues arise for a "partially PCN-capable tunnel", ie where 1120 only one tunnel endpoint is in the PCN domain: 1122 1. The tunnel originates outside a PCN-domain and ends inside it. 1123 If the packet arrives at the tunnel ingress with the same 1124 encoding as used within the PCN-domain to indicate PCN-marking, 1125 then this could lead the PCN-egress-node to falsely measure pre- 1126 congestion. 1128 2. The tunnel originates inside a PCN-domain and ends outside it. 1129 If the packet arrives at the tunnel ingress already PCN-marked, 1130 then it will still have the same encoding when it's decapsulated 1131 which could potentially confuse nodes beyond the tunnel egress. 1133 In line with the solution for partially capable Diffserv tunnels in 1134 [RFC2983], the following rules are applied: 1136 o For case (1), the tunnel egress node clears any PCN-marking on the 1137 inner header. This rule is applied before the 'copy on 1138 decapsulation' rule above. 1140 o For case (2), the tunnel ingress node clears any PCN-marking on 1141 the inner header. This rule is applied after the 'copy on 1142 encapsulation' rule above. 1144 Note that the above implies that one has to know, or determine, the 1145 characteristics of the other end of the tunnel as part of 1146 establishing it. 1148 Tunnelling constraints were a major factor in the choice of the 1149 baseline encoding. As explained in [PCN08-1], with current 1150 tunnelling endpoints only the 11 codepoint of the ECN field survives 1151 decapsulation, and hence the baseline encoding only uses the 11 1152 codepoint to indicate PCN-marking. Extended encoding schemes need to 1153 explain their interactions with (or assumptions about) tunnelling. A 1154 lengthy discussion of all the issues associated with layered 1155 encapsulation of congestion notification (for ECN as well as PCN) is 1156 in [Briscoe08-2]. 1158 4.8. Fault handling 1160 If a PCN-interior-node (or one of its links) fails, then lower layer 1161 protection mechanisms or the regular IP routing protocol will 1162 eventually re-route around it. If the new route can carry all the 1163 admitted traffic, flows will gracefully continue. If instead this 1164 causes early warning of pre-congestion on the new route, then 1165 admission control based on pre-congestion notification will ensure 1166 new flows will not be admitted until enough existing flows have 1167 departed. Re-routing may result in heavy (pre-)congestion, when the 1168 flow termination mechanism will kick in. 1170 If a PCN-boundary-node fails then we would like the regular QoS 1171 signalling protocol to be responsible for taking appropriate action. 1172 As an example [Briscoe08-2] considers what happens if RSVP is the QoS 1173 signalling protocol. 1175 5. Operations and Management 1177 This Section considers operations and management issues, under the 1178 FCAPS headings: the Operations and Management of Faults, 1179 Configuration, Accounting, Performance and Security. Provisioning is 1180 discussed with performance. 1182 5.1. Configuration Operations and Management 1184 Threshold-metering and -marking and excess-traffic-metering and 1185 -marking are standardised in [PCN08-2]. However, more diversity in 1186 PCN-boundary-node behaviours is expected, in order to interface with 1187 diverse industry architectures. It may be possible to have different 1188 PCN-boundary-node behaviours for different ingress-egress-aggregates 1189 within the same PCN-domain. 1191 PCN metering behaviour is enabled on either the egress or the ingress 1192 interfaces of PCN-nodes. A consistent choice must be made across the 1193 PCN-domain to ensure that the PCN mechanisms protect all links. 1195 PCN configuration control variables fall into the following 1196 categories: 1198 o system options (enabling or disabling behaviours) 1200 o parameters (setting levels, addresses etc) 1202 One possibility is that all configurable variables sit within an SNMP 1203 management framework [RFC3411], being structured within a defined 1204 management information base (MIB) on each node, and being remotely 1205 readable and settable via a suitably secure management protocol 1206 (SNMPv3). 1208 Some configuration options and parameters have to be set once to 1209 'globally' control the whole PCN-domain. Where possible, these are 1210 identified below. This may affect operational complexity and the 1211 chances of interoperability problems between equipment from different 1212 vendors. 1214 It may be possible for an operator to configure some PCN-interior- 1215 nodes so that they don't run the PCN mechanisms, if it knows that 1216 these links will never become (pre-)congested. 1218 5.1.1. System options 1220 On PCN-interior-nodes there will be very few system options: 1222 o Whether two PCN-markings (threshold-marked and excess-traffic- 1223 marked) are enabled or only one. Typically all nodes throughout a 1224 PCN-domain will be configured the same in this respect. However, 1225 exceptions could be made. For example, if most PCN-nodes used 1226 both markings, but some legacy hardware was incapable of running 1227 two algorithms, an operator might be willing to configure these 1228 legacy nodes solely for excess-traffic-marking to enable flow 1229 termination as a back-stop. It would be sensible to place such 1230 nodes where they could be provisioned with a greater leeway over 1231 expected traffic levels. 1233 o In the case where only one PCN-marking is enabled, all nodes must 1234 be configured to generate PCN-marks from the same meter (ie either 1235 the threshold meter or the excess traffic meter). 1237 PCN-boundary-nodes (ingress and egress) will have more system 1238 options: 1240 o Which of admission and flow termination are enabled. If any PCN- 1241 interior-node is configured to generate a marking, all PCN- 1242 boundary-nodes must be able to interpret that marking (which 1243 includes understanding, in a PCN-domain that uses only one type of 1244 PCN-marking, whether they are generated by PCN-interior-nodes' 1245 threshold meters or the excess traffic meters). Therefore all 1246 PCN-boundary-nodes must be configured the same in this respect. 1248 o Where flow admission and termination decisions are made: at PCN- 1249 ingress-nodes or at PCN-egress-nodes (or at a centralised node, 1250 see Appendix). Theoretically, this configuration choice could be 1251 negotiated for each pair of PCN-boundary-nodes, but we cannot 1252 imagine why such complexity would be required, except perhaps in 1253 future inter-domain scenarios. 1255 o How PCN-markings are translated into admission control and flow 1256 termination decisions (see Section 3.1 and Section 3.2). 1258 PCN-egress-nodes will have further system options: 1260 o How the mapping should be established between each packet and its 1261 aggregate, eg by MPLS label, by IP packet filter spec; and how to 1262 take account of ECMP. 1264 o If an equipment vendor provides a choice, there may be options to 1265 select which smoothing algorithm to use for measurements. 1267 5.1.2. Parameters 1269 Like any Diffserv domain, every node within a PCN-domain will need to 1270 be configured with the DSCP(s) used to identify PCN-packets. On each 1271 interior link the main configuration parameters are the PCN- 1272 threshold-rate and PCN-excess-rate. A larger PCN-threshold-rate 1273 enables more PCN-traffic to be admitted on a link, hence improving 1274 capacity utilisation. A PCN-excess-rate set further above the PCN- 1275 threshold-rate allows greater increases in traffic (whether due to 1276 natural fluctuations or some unexpected event) before any flows are 1277 terminated, ie minimises the chances of unnecessarily triggering the 1278 termination mechanism. For instance, an operator may want to design 1279 their network so that it can cope with a failure of any single PCN- 1280 node without terminating any flows. 1282 Setting these rates on first deployment of PCN will be very similar 1283 to the traditional process for sizing an admission controlled 1284 network, depending on: the operator's requirements for minimising 1285 flow blocking (grade of service), the expected PCN traffic load on 1286 each link and its statistical characteristics (the traffic matrix), 1287 contingency for re-routing the PCN traffic matrix in the event of 1288 single or multiple failures, and the expected load from other classes 1289 relative to link capacities [Menth07]. But once a domain is in 1290 operation, a PCN design goal is to be able to determine growth in 1291 these configured rates much more simply, by monitoring PCN-marking 1292 rates from actual rather than expected traffic (see Section 5.2 on 1293 Performance & Provisioning). 1295 Operators may also wish to configure a rate greater than the PCN- 1296 excess-rate that is the absolute maximum rate that a link allows for 1297 PCN-traffic. This may simply be the physical link rate, but some 1298 operators may wish to configure a logical limit to prevent starvation 1299 of other traffic classes during any brief period after PCN-traffic 1300 exceeds the PCN-excess-rate but before flow termination brings it 1301 back below this rate. 1303 Threshold-metering requires a threshold token bucket depth to be 1304 configured, excess-traffic-metering needs a value for the MTU 1305 (maximum size of a PCN-packet on the link) and both require setting a 1306 maximum size of their token buckets. It will be preferable for there 1307 to be rules to set defaults for these parameters, but then allow 1308 operators to change them, for instance if average traffic 1309 characteristics change over time. 1311 The PCN-egress-node may allow configuration of the following: 1313 o how it smooths metering of PCN-markings (eg EWMA parameters) 1315 Whichever node makes admission and flow termination decisions will 1316 contain algorithms for converting PCN-marking levels into admission 1317 or flow termination decisions. These will also require configurable 1318 parameters, for instance: 1320 o an admission control algorithm that is based on the fraction of 1321 marked packets will at least require a marking threshold setting 1322 above which it denies admission to new flows; 1324 o flow termination algorithms will probably require a parameter to 1325 delay termination of any flows until it is more certain that an 1326 anomalous event is not transient; 1328 o a parameter to control the trade-off between how quickly excess 1329 flows are terminated, and over-termination. 1331 One particular approach, [Charny07-2] would require a global 1332 parameter to be defined on all PCN-nodes, but only needs one PCN 1333 marking rate to be configured on each link. The global parameter is 1334 a scaling factor between admission and termination (the PCN-traffic 1335 rate on a link up to which flows are admitted vs the rate above which 1336 flows are terminated). [Charny07-2] discusses in full the impact of 1337 this particular approach on the operation of PCN. 1339 5.2. Performance & Provisioning Operations and Management 1341 Monitoring of performance factors measurable from *outside* the PCN 1342 domain will be no different with PCN than with any other packet-based 1343 flow admission control system, both at the flow level (blocking 1344 probability, etc) and the packet level (jitter [RFC3393], [Y.1541], 1345 loss rate [RFC4656], mean opinion score [P.800], etc). The 1346 difference is that PCN is intentionally designed to indicate 1347 *internally* which exact resource(s) are the cause of performance 1348 problems and by how much. 1350 Even better, PCN indicates which resources will probably cause 1351 problems if they are not upgraded soon. This can be achieved by the 1352 management system monitoring the total amount (in bytes) of PCN- 1353 marking generated by each queue over a period. Given possible long 1354 provisioning lead times, pre-congestion volume is the best metric to 1355 reveal whether sufficient persistent demand has occurred to warrant 1356 an upgrade. Because, even before utilisation becomes problematic, 1357 the statistical variability of traffic will cause occasional bursts 1358 of pre-congestion. This 'early warning system' decouples the process 1359 of adding customers from the provisioning process. This should cut 1360 the time to add a customer when compared against admission control 1361 provided over native Diffserv [RFC2998], because it saves having to 1362 verify the capacity planning process before adding each customer. 1364 Alternatively, before triggering an upgrade, the long term pre- 1365 congestion volume on each link can be used to balance traffic load 1366 across the PCN-domain by adjusting the link weights of the routing 1367 system. When an upgrade to a link's configured PCN-rates is 1368 required, it may also be necessary to upgrade the physical capacity 1369 available to other classes. But usually there will be sufficient 1370 physical capacity for the upgrade to go ahead as a simple 1371 configuration change. Alternatively, [Songhurst06] describes an 1372 adaptive rather than preconfigured system, where the configured PCN- 1373 threshold-rate is replaced with a high and low water mark and the 1374 marking algorithm automatically optimises how physical capacity is 1375 shared using the relative loads from PCN and other traffic classes. 1377 All the above processes require just three extra counters associated 1378 with each PCN queue: threshold-markings, excess-traffic-markings and 1379 drop. Every time a PCN packet is marked or dropped its size in bytes 1380 should be added to the appropriate counter. Then the management 1381 system can read the counters at any time and subtract a previous 1382 reading to establish the incremental volume of each type of 1383 (pre-)congestion. Readings should be taken frequently, so that 1384 anomalous events (eg re-routes) can be distinguished from regular 1385 fluctuating demand if required. 1387 5.3. Accounting Operations and Management 1389 Accounting is only done at trust boundaries so it is out of scope of 1390 this document, which is confined to intra-domain issues. Use of PCN 1391 internal to a domain makes no difference to the flow signalling 1392 events crossing trust boundaries outside the PCN-domain, which are 1393 typically used for accounting. 1395 5.4. Fault Operations and Management 1397 Fault Operations and Management is about preventing faults, telling 1398 the management system (or manual operator) that the system has 1399 recovered (or not) from a failure, and about maintaining information 1400 to aid fault diagnosis. 1402 Admission blocking and particularly flow termination mechanisms 1403 should rarely be needed in practice. It would be unfortunate if they 1404 didn't work after an option had been accidentally disabled. 1405 Therefore it will be necessary to regularly test that the live system 1406 works as intended (devising a meaningful test is left as an exercise 1407 for the operator). 1409 Section 4 describes how the PCN architecture has been designed to 1410 ensure admitted flows continue gracefully after recovering 1411 automatically from link or node failures. The need to record and 1412 monitor re-routing events affecting signalling is unchanged by the 1413 addition of PCN to a Diffserv domain. Similarly, re-routing events 1414 within the PCN-domain will be recorded and monitored just as they 1415 would be without PCN. 1417 PCN-marking does make it possible to record 'near-misses'. For 1418 instance, at the PCN-egress-node a 'reporting threshold' could be set 1419 to monitor how often - and for how long - the system comes close to 1420 triggering flow blocking without actually doing so. Similarly, 1421 bursts of flow termination marking could be recorded even if they are 1422 not sufficiently sustained to trigger flow termination. Such 1423 statistics could be correlated with per-queue counts of marking 1424 volume (Section 5.2) to upgrade resources in danger of causing 1425 service degradation, or to trigger manual tracing of intermittent 1426 incipient errors that would otherwise have gone unnoticed. 1428 Finally, of course, many faults are caused by failings in the 1429 management process ('human error'): a wrongly configured address in a 1430 node, a wrong address given in a signalling protocol, a wrongly 1431 configured parameter in a queueing algorithm, a node set into a 1432 different mode from other nodes, and so on. Generally, a clean 1433 design with few configurable options ensures this class of faults can 1434 be traced more easily and prevented more often. Sound management 1435 practice at run-time also helps. For instance: a management system 1436 should be used that constrains configuration changes within system 1437 rules (eg preventing an option setting inconsistent with other 1438 nodes); configuration options should also be recorded in an offline 1439 database; and regular automatic consistency checks between live 1440 systems and the database should be performed. PCN adds nothing 1441 specific to this class of problems. 1443 5.5. Security Operations and Management 1445 Security Operations and Management is about using secure operational 1446 practices as well as being able to track security breaches or near- 1447 misses at run-time. PCN adds few specifics to the general good 1448 practice required in this field [RFC4778], other than those below. 1449 The correct functions of the system should be monitored (Section 5.2) 1450 in multiple independent ways and correlated to detect possible 1451 security breaches. Persistent (pre-)congestion marking should raise 1452 an alarm (both on the node doing the marking and on the PCN-egress- 1453 node metering it). Similarly, persistently poor external QoS metrics 1454 (such as jitter or mean opinion score) should raise an alarm. The 1455 following are examples of symptoms that may be the result of innocent 1456 faults, rather than attacks, but until diagnosed they should be 1457 logged and trigger a security alarm: 1459 o Anomalous patterns of non-conforming incoming signals and packets 1460 rejected at the PCN-ingress-nodes (eg packets already marked PCN- 1461 capable, or traffic persistently starving token bucket policers). 1463 o PCN-capable packets arriving at a PCN-egress-node with no 1464 associated state for mapping them to a valid ingress-egress- 1465 aggregate. 1467 o A PCN-ingress-node receiving feedback signals about the pre- 1468 congestion level on a non-existent aggregate, or that are 1469 inconsistent with other signals (eg unexpected sequence numbers, 1470 inconsistent addressing, conflicting reports of the pre-congestion 1471 level, etc). 1473 o Pre-congestion marking arriving at a PCN-egress-node with 1474 (pre-)congestion markings focused on particular flows, rather than 1475 randomly distributed throughout the aggregate. 1477 6. Applicability of PCN 1479 6.1. Benefits 1481 The key benefits of the PCN mechanisms are that they are simple, 1482 scalable, and robust because: 1484 o Per flow state is only required at the PCN-ingress-nodes 1485 ("stateless core"). This is required for policing purposes (to 1486 prevent non-admitted PCN traffic from entering the PCN-domain) and 1487 so on. It is not generally required that other network entities 1488 are aware of individual flows (although they may be in particular 1489 deployment scenarios). 1491 o Admission control is resilient: with PCN QoS is decoupled from the 1492 routing system. Hence in general admitted flows can survive 1493 capacity, routing or topology changes without additional 1494 signalling. The PCN-admissible-rate on each link can be chosen 1495 small enough that admitted traffic can still be carried after a 1496 rerouting in most failure cases [Menth07]. This is an important 1497 feature as QoS violations in core networks due to link failures 1498 are more likely than QoS violations due to increased traffic 1499 volume [Iyer03]. 1501 o The PCN-metering behaviours only operate on the overall PCN- 1502 traffic on the link, not per flow. 1504 o The information of these measurements is signalled to the PCN- 1505 egress-nodes by the PCN-marks in the packet headers, ie [Style] 1506 "in-band". No additional signalling protocol is required for 1507 transporting the PCN-marks. Therefore no secure binding is 1508 required between data packets and separate congestion messages. 1510 o The PCN-egress-nodes make separate measurements, operating on the 1511 aggregate PCN-traffic from each PCN-ingress-node, ie not per flow. 1512 Similarly, signalling by the PCN-egress-node of PCN-feedback- 1513 information (which is used for flow admission and termination 1514 decisions) is at the granularity of the ingress-egress-aggregate. 1515 An alternative approach is that the PCN-egress-nodes monitor the 1516 PCN-traffic and signal PCN-feedback-information (which is used for 1517 flow admission and termination decisions) at the granularity of 1518 one (or a few) PCN-marks. 1520 o The admitted PCN-load is controlled dynamically. Therefore it 1521 adapts as the traffic matrix changes, and also if the network 1522 topology changes (eg after a link failure). Hence an operator can 1523 be less conservative when deploying network capacity, and less 1524 accurate in their prediction of the PCN-traffic matrix. 1526 o The termination mechanism complements admission control. It 1527 allows the network to recover from sudden unexpected surges of 1528 PCN-traffic on some links, thus restoring QoS to the remaining 1529 flows. Such scenarios are expected to be rare but not impossible. 1530 They can be caused by large network failures that redirect lots of 1531 admitted PCN-traffic to other links, or by malfunction of the 1532 measurement-based admission control in the presence of admitted 1533 flows that send for a while with an atypically low rate and then 1534 increase their rates in a correlated way. 1536 o Flow termination can also enable an operator to be less 1537 conservative when deploying network capacity. It is an 1538 alternative to running links at low utilisation in order to 1539 protect against link or node failures. This is especially the 1540 case with SRLGs (shared risk link groups, which are links that 1541 share a resource, such as a fibre, whose failure affects all those 1542 links [RFC4216]). Fully protecting traffic against a single SRLG 1543 failure requires low utilisation (~10%) of the link bandwidth on 1544 some links before failure [Charny08]. 1546 o The PCN-supportable-rate may be set below the maximum rate that 1547 PCN-traffic can be transmitted on a link, in order to trigger 1548 termination of some PCN-flows before loss (or excessive delay) of 1549 PCN-packets occurs, or to keep the maximum PCN-load on a link 1550 below a level configured by the operator. 1552 o Provisioning of the network is decoupled from the process of 1553 adding new customers. By contrast, with the Diffserv architecture 1554 [RFC2475] operators rely on subscription-time Service Level 1555 Agreements, which statically define the parameters of the traffic 1556 that will be accepted from a customer, and so the operator has to 1557 verify provision is sufficient each time a new customer is added 1558 to check that the Service Level Agreement can be fulfilled. A 1559 PCN-domain doesn't need such traffic conditioning. 1561 6.2. Deployment scenarios 1563 Operators of networks will want to use the PCN mechanisms in various 1564 arrangements, for instance depending on how they are performing 1565 admission control outside the PCN-domain (users after all are 1566 concerned about QoS end-to-end), what their particular goals and 1567 assumptions are, how many PCN encoding states are available, and so 1568 on. 1570 A PCN-domain may have three encoding states (or pedantically, an 1571 operator may choose to use up three encoding states for PCN): not 1572 PCN-marked, threshold-marked, excess-traffic-marked. Then both PCN 1573 admission control and flow termination can be supported. As 1574 illustrated in Figure 1, admission control accepts new flows until 1575 the PCN-traffic rate on the bottleneck link rises above the PCN- 1576 threshold-rate, whilst if necessary the flow termination mechanism 1577 terminates flows down to the PCN-excess-rate on the bottleneck link. 1579 On the other hand, a PCN-domain may have two encoding states (as in 1580 [PCN08-1]) (or pedantically, an operator may choose to use up two 1581 encoding states for PCN): not PCN-marked, PCN-marked. Then there are 1582 three possibilities, as discussed in the following paragraphs (see 1583 also Section 3.3). 1585 First, an operator could just use PCN's admission control, solving 1586 heavy congestion (caused by re-routing) by 'just waiting' - as 1587 sessions end, PCN-traffic naturally reduces, and meanwhile the 1588 admission control mechanism will prevent admission of new flows that 1589 use the affected links. So the PCN-domain will naturally return to 1590 normal operation, but with reduced capacity. The drawback of this 1591 approach would be that, until sufficient sessions have ended to 1592 relieve the congestion, all PCN-flows as well as lower priority 1593 services will be adversely affected. 1595 Second, an operator could just rely for admission control on 1596 statically provisioned capacity per PCN-ingress-node (regardless of 1597 the PCN-egress-node of a flow), as is typical in the hose model of 1598 the Diffserv architecture [RFC2475]. Such traffic conditioning 1599 agreements can lead to focused overload: many flows happen to focus 1600 on a particular link and then all flows through the congested link 1601 fail catastrophically. PCN's flow termination mechanism could then 1602 be used to counteract such a problem. 1604 Third, both admission control and flow termination can be triggered 1605 from the single type of PCN-marking; the main downside is that 1606 admission control is less accurate [Charny07-2]. This possibility is 1607 illustrated in Figure 3. 1609 Within the PCN-domain there is some flexibility about how the 1610 decision making functionality is distributed. These possibilities 1611 are outlined in Section 4.4 and also discussed elsewhere, such as in 1612 [Menth08-3]. 1614 The flow admission and termination decisions need to be enforced 1615 through per flow policing by the PCN-ingress-nodes. If there are 1616 several PCN-domains on the end-to-end path, then each needs to police 1617 at its PCN-ingress-nodes. One exception is if the operator runs both 1618 the access network (not a PCN-domain) and the core network (a PCN- 1619 domain); per flow policing could be devolved to the access network 1620 and not done at the PCN-ingress-node. Note: to aid readability, the 1621 rest of this draft assumes that policing is done by the PCN-ingress- 1622 nodes. 1624 PCN admission control has to fit with the overall approach to 1625 admission control. For instance [Briscoe06] describes the case where 1626 RSVP signalling runs end-to-end. The PCN-domain is a single RSVP 1627 hop, ie only the PCN-boundary-nodes process RSVP messages, with RSVP 1628 messages processed on each hop outside the PCN-domain, as in IntServ 1629 over Diffserv [RFC2998]. It would also be possible for the RSVP 1630 signalling to be originated and/or terminated by proxies, with 1631 application-layer signalling between the end user and the proxy (eg 1632 SIP signalling with a home hub). A similar example would use NSIS 1633 signalling instead of RSVP. (NSIS: Next Steps in Signalling, 1634 [RFC3726].) 1636 It is possible that a user wants its inelastic traffic to use the PCN 1637 mechanisms but also react to ECN marking outside the PCN-domain 1638 [Sarker08]. Two possible ways to do this are to tunnel all PCN- 1639 packets across the PCN-domain, so that the ECN marks are carried 1640 transparently across the PCN-domain, or to use an encoding like 1641 [Moncaster08]. Tunnelling is discussed further in Section 4.7. 1643 Some further possible deployment models are outlined in the Appendix. 1645 6.3. Assumptions and constraints on scope 1647 The scope is restricted by the following assumptions: 1649 1. these components are deployed in a single Diffserv domain, within 1650 which all PCN-nodes are PCN-enabled and are trusted for truthful 1651 PCN-marking and transport 1653 2. all flows handled by these mechanisms are inelastic and 1654 constrained to a known peak rate through policing or shaping 1656 3. the number of PCN-flows across any potential bottleneck link is 1657 sufficiently large that stateless, statistical mechanisms can be 1658 effective. To put it another way, the aggregate bit rate of PCN- 1659 traffic across any potential bottleneck link needs to be 1660 sufficiently large relative to the maximum additional bit rate 1661 added by one flow. This is the basic assumption of measurement- 1662 based admission control. 1664 4. PCN-flows may have different precedence, but the applicability of 1665 the PCN mechanisms for emergency use (911, GETS, WPS, MLPP, etc.) 1666 is out of scope. 1668 6.3.1. Assumption 1: Trust and support of PCN - controlled environment 1670 It is assumed that the PCN-domain is a controlled environment, ie all 1671 the nodes in a PCN-domain run PCN and are trusted. There are several 1672 reasons this assumption: 1674 o The PCN-domain has to be encircled by a ring of PCN-boundary- 1675 nodes, otherwise traffic could enter a PCN-BA without being 1676 subject to admission control, which would potentially degrade the 1677 QoS of existing PCN-flows. 1679 o Similarly, a PCN-boundary-node has to trust that all the PCN-nodes 1680 mark PCN-traffic consistently. A node not performing PCN-marking 1681 wouldn't be able to alert when it suffered pre-congestion, which 1682 potentially would lead to too many PCN-flows being admitted (or 1683 too few being terminated). Worse, a rogue node could perform 1684 various attacks, as discussed in the Security Considerations 1685 section. 1687 One way of assuring the above two points is that the entire PCN- 1688 domain is run by a single operator. Another possibility is that 1689 there are several operators that trust each other in their handling 1690 of PCN-traffic. 1692 Note: All PCN-nodes need to be trustworthy. However if it is known 1693 that an interface cannot become pre-congested then it is not strictly 1694 necessary for it to be capable of PCN-marking. But this must be 1695 known even in unusual circumstances, eg after the failure of some 1696 links. 1698 6.3.2. Assumption 2: Real-time applications 1700 It is assumed that any variation of source bit rate is independent of 1701 the level of pre-congestion. We assume that PCN-packets come from 1702 real time applications generating inelastic traffic, ie sending 1703 packets at the rate the codec produces them, regardless of the 1704 availability of capacity [RFC4594]. For example, voice and video 1705 requiring low delay, jitter and packet loss, the Controlled Load 1706 Service, [RFC2211], and the Telephony service class, [RFC4594]. This 1707 assumption is to help focus the effort where it looks like PCN would 1708 be most useful, ie the sorts of applications where per flow QoS is a 1709 known requirement. In other words we focus on PCN providing a 1710 benefit to inelastic traffic (PCN may or may not provide a benefit to 1711 other types of traffic). 1713 As a consequence, it is assumed that PCN-metering and PCN-marking is 1714 being applied to traffic scheduled with the expedited forwarding per- 1715 hop behaviour, [RFC3246], or a per-hop behaviour with similar 1716 characteristics. 1718 6.3.3. Assumption 3: Many flows and additional load 1720 It is assumed that there are many PCN-flows on any bottleneck link in 1721 the PCN-domain (or, to put it another way, the aggregate bit rate of 1722 PCN-traffic across any potential bottleneck link is sufficiently 1723 large relative to the maximum additional bit rate added by one PCN- 1724 flow). Measurement-based admission control assumes that the present 1725 is a reasonable prediction of the future: the network conditions are 1726 measured at the time of a new flow request, however the actual 1727 network performance must be acceptable during the call some time 1728 later. One issue is that if there are only a few variable rate 1729 flows, then the aggregate traffic level may vary a lot, perhaps 1730 enough to cause some packets to get dropped. If there are many flows 1731 then the aggregate traffic level should be statistically smoothed. 1732 How many flows is enough depends on a number of factors such as the 1733 variation in each flow's rate, the total rate of PCN-traffic, and the 1734 size of the "safety margin" between the traffic level at which we 1735 start admission-marking and at which packets are dropped or 1736 significantly delayed. 1738 No explicit assumptions are made about how many PCN-flows are in each 1739 ingress-egress-aggregate. Performance evaluation work may clarify 1740 whether it is necessary to make any additional assumption on 1741 aggregation at the ingress-egress-aggregate level. 1743 6.3.4. Assumption 4: Emergency use out of scope 1745 PCN-flows may have different precedence, but the applicability of the 1746 PCN mechanisms for emergency use (911, GETS, WPS, MLPP, etc) is out 1747 of scope of this document. 1749 6.4. Challenges 1751 Prior work on PCN and similar mechanisms has thrown up a number of 1752 considerations about PCN's design goals (things PCN should be good 1753 at) and some issues that have been hard to solve in a fully 1754 satisfactory manner. Taken as a whole it represents a list of trade- 1755 offs (it is unlikely that they can all be 100% achieved) and perhaps 1756 as evaluation criteria to help an operator (or the IETF) decide 1757 between options. 1759 The following are open issues. They are mainly taken from 1760 [Briscoe06], which also describes some possible solutions. Note that 1761 some may be considered unimportant in general or in specific 1762 deployment scenarios or by some operators. 1764 NOTE: Potential solutions are out of scope for this document. 1766 o ECMP (Equal Cost Multi-Path) Routing: The level of pre-congestion 1767 is measured on a specific ingress-egress-aggregate. However, if 1768 the PCN-domain runs ECMP, then traffic on this ingress-egress- 1769 aggregate may follow several different paths - some of the paths 1770 could be pre-congested whilst others are not. There are three 1771 potential problems: 1773 1. over-admission: a new flow is admitted (because the pre- 1774 congestion level measured by the PCN-egress-node is 1775 sufficiently diluted by unmarked packets from non-congested 1776 paths that a new flow is admitted), but its packets travel 1777 through a pre-congested PCN-node. 1779 2. under-admission: a new flow is blocked (because the pre- 1780 congestion level measured by the PCN-egress-node is 1781 sufficiently increased by PCN-marked packets from pre- 1782 congested paths that a new flow is blocked), but its packets 1783 travel along an uncongested path. 1785 3. ineffective termination: a flow is terminated, but its path 1786 doesn't travel through the (pre-)congested router(s). Since 1787 flow termination is a 'last resort', which protects the 1788 network should over-admission occur, this problem is probably 1789 more important to solve than the other two. 1791 o ECMP and signalling: It is possible that, in a PCN-domain running 1792 ECMP, the signalling packets (eg RSVP, NSIS) follow a different 1793 path than the data packets, which could matter if the signalling 1794 packets are used as probes. Whether this is an issue depends on 1795 which fields the ECMP algorithm uses; if the ECMP algorithm is 1796 restricted to the source and destination IP addresses, then it 1797 will not be an issue. ECMP and signalling interactions are a 1798 specific instance of a general issue for non-traditional routing 1799 combined with resource management along a path [Hancock02]. 1801 o Tunnelling: There are scenarios where tunnelling makes it 1802 difficult to determine the path in the PCN-domain. The problem, 1803 its impact, and the potential solutions are similar to those for 1804 ECMP. 1806 o Scenarios with only one tunnel endpoint in the PCN domain may make 1807 it harder for the PCN-egress-node to gather from the signalling 1808 messages (eg RSVP, NSIS) the identity of the PCN-ingress-node. 1810 o Bi-Directional Sessions: Many applications have bi-directional 1811 sessions - hence there are two microflows that should be admitted 1812 (or terminated) as a pair - for instance a bi-directional voice 1813 call only makes sense if microflows in both directions are 1814 admitted. However, the PCN mechanisms concern admission and 1815 termination of a single flow, and coordination of the decision for 1816 both flows is a matter for the signalling protocol and out of 1817 scope of PCN. One possible example would use SIP pre-conditions. 1818 However, there are others. 1820 o Global Coordination: PCN makes its admission decision based on 1821 PCN-markings on a particular ingress-egress-aggregate. Decisions 1822 about flows through a different ingress-egress-aggregate are made 1823 independently. However, one can imagine network topologies and 1824 traffic matrices where, from a global perspective, it would be 1825 better to make a coordinated decision across all the ingress- 1826 egress-aggregates for the whole PCN-domain. For example, to block 1827 (or even terminate) flows on one ingress-egress-aggregate so that 1828 more important flows through a different ingress-egress-aggregate 1829 could be admitted. The problem may well be relatively 1830 insignificant. 1832 o Aggregate Traffic Characteristics: Even when the number of flows 1833 is stable, the traffic level through the PCN-domain will vary 1834 because the sources vary their traffic rates. PCN works best when 1835 there is not too much variability in the total traffic level at a 1836 PCN-node's interface (ie in the aggregate traffic from all 1837 sources). Too much variation means that a node may (at one 1838 moment) not be doing any PCN-marking and then (at another moment) 1839 drop packets because it is overloaded. This makes it hard to tune 1840 the admission control scheme to stop admitting new flows at the 1841 right time. Therefore the problem is more likely with fewer, 1842 burstier flows. 1844 o Flash crowds and Speed of Reaction: PCN is a measurement-based 1845 mechanism and so there is an inherent delay between packet marking 1846 by PCN-interior-nodes and any admission control reaction at PCN- 1847 boundary-nodes. For example, potentially if a big burst of 1848 admission requests occurs in a very short space of time (eg 1849 prompted by a televote), they could all get admitted before enough 1850 PCN-marks are seen to block new flows. In other words, any 1851 additional load offered within the reaction time of the mechanism 1852 must not move the PCN-domain directly from a no congestion state 1853 to overload. This 'vulnerability period' may have an impact at 1854 the signalling level, for instance QoS requests should be rate 1855 limited to bound the number of requests able to arrive within the 1856 vulnerability period. 1858 o Silent at start: after a successful admission request the source 1859 may wait some time before sending data (eg waiting for the called 1860 party to answer). Then the risk is that, in some circumstances, 1861 PCN's measurements underestimate what the pre-congestion level 1862 will be when the source does start sending data. 1864 7. IANA Considerations 1866 This memo includes no request to IANA. 1868 8. Security considerations 1870 Security considerations essentially come from the Trust Assumption 1871 (Section 6.3.1), ie that all PCN-nodes are PCN-enabled and are 1872 trusted for truthful PCN-metering and PCN-marking. PCN splits 1873 functionality between PCN-interior-nodes and PCN-boundary-nodes, and 1874 the security considerations are somewhat different for each, mainly 1875 because PCN-boundary-nodes are flow-aware and PCN-interior-nodes are 1876 not. 1878 o Because the PCN-boundary-nodes are flow-aware, they are trusted to 1879 use that awareness correctly. The degree of trust required 1880 depends on the kinds of decisions they have to make and the kinds 1881 of information they need to make them. There is nothing specific 1882 to PCN. 1884 o The PCN-ingress-nodes police packets to ensure a PCN-flow sticks 1885 within its agreed limit, and to ensure that only PCN-flows that 1886 have been admitted contribute PCN-traffic into the PCN-domain. 1887 The policer must drop (or perhaps downgrade to a different DSCP) 1888 any PCN-packets received that are outside this remit. This is 1889 similar to the existing IntServ behaviour. Between them the PCN- 1890 boundary-nodes must encircle the PCN-domain, otherwise PCN-packets 1891 could enter the PCN-domain without being subject to admission 1892 control, which would potentially destroy the QoS of existing 1893 flows. 1895 o PCN-interior-nodes are not flow-aware. This prevents some 1896 security attacks where an attacker targets specific flows in the 1897 data plane - for instance for DoS or eavesdropping. 1899 o The PCN-boundary-nodes rely on correct PCN-marking by the PCN- 1900 interior-nodes. For instance a rogue PCN-interior-node could PCN- 1901 mark all packets so that no flows were admitted. Another 1902 possibility is that it doesn't PCN-mark any packets, even when it 1903 is pre-congested. More subtly, the rogue PCN-interior-node could 1904 perform these attacks selectively on particular flows, or it could 1905 PCN-mark the correct fraction overall, but carefully choose which 1906 flows it marked. 1908 o The PCN-boundary-nodes should be able to deal with DoS attacks and 1909 state exhaustion attacks based on fast changes in per flow 1910 signalling. 1912 o The signalling between the PCN-boundary-nodes must be protected 1913 from attacks. For example the recipient needs to validate that 1914 the message is indeed from the node that claims to have sent it. 1915 Possible measures include digest authentication and protection 1916 against replay and man-in-the-middle attacks. For the specific 1917 protocol RSVP, hop-by-hop authentication is in [RFC2747], and 1918 [Behringer07] may also be useful. 1920 Operational security advice is given in Section 5.5. 1922 9. Conclusions 1924 The document describes a general architecture for flow admission and 1925 termination based on pre-congestion information in order to protect 1926 the quality of service of established inelastic flows within a single 1927 Diffserv domain. The main topic is the functional architecture. It 1928 also mentions other topics like the assumptions and open issues. 1930 10. Acknowledgements 1932 This document is a revised version of an earlier individual draft 1933 authored by: P. Eardley, J. Babiarz, K. Chan, A. Charny, R. Geib, G. 1934 Karagiannis, M. Menth, T. Tsou. They are therefore contributors to 1935 this document. 1937 Thanks to those who have made comments on this document: Lachlan 1938 Andrew, Joe Babiarz, Fred Baker, David Black, Steven Blake, Ron 1939 Bonica, Scott Bradner, Bob Briscoe, Ross Callon, Jason Canon, Ken 1940 Carlberg, Anna Charny, Joachim Charzinski, Andras Csaszar, Francis 1941 Dupont, Lars Eggert, Pasi Eronen, Adrian Farrel, Ruediger Geib, Wei 1942 Gengyu, Robert Hancock, Fortune Huang, Christian Hublet, Cullen 1943 Jennings, Ingemar Johansson, Georgios Karagiannis, Hein Mekkes, 1944 Michael Menth, Toby Moncaster, Dimitri Papadimitriou, Dan Romascanu, 1945 Daisuke Satoh, Ben Strulo, Tom Taylor, Hannes Tschofenig, Tina Tsou, 1946 David Ward, Lars Westberg, Magnus Westerlund, Delei Yu. Thanks to 1947 Bob Briscoe who extensively revised the Operations and Management 1948 section. 1950 This document is the result of discussions in the PCN WG and 1951 forerunner activity in the TSVWG. A number of previous drafts were 1952 presented to TSVWG; their authors were: B, Briscoe, P. Eardley, D. 1953 Songhurst, F. Le Faucheur, A. Charny, J. Babiarz, K. Chan, S. Dudley, 1954 G. Karagiannis, A. Bader, L. Westberg, J. Zhang, V. Liatsos, X-G. 1955 Liu, A. Bhargava. 1957 11. Comments Solicited (to be removed by RFC Editor) 1959 Comments and questions are encouraged and very welcome. They can be 1960 addressed to the IETF PCN working group mailing list . 1962 12. Changes (to be removed by RFC Editor) 1964 12.1. Changes from -10 to -11 1966 Changes to deal with IESG comments from routing area review: 1968 o Small clarifications to Introduction 1970 o the term "marking" now only used to refer only to setting the 1971 codepoint (not as a shorthand for 'metering and setting the 1972 codepoint') 1974 o Added Figure 4 (Schematic of PCN-interior-node functionality) 1975 (from [PCN08-2] 1977 o Appendix A brought back into the main body. 1979 o Other minor clarifications 1981 12.2. Changes from -09 to -10 1983 Changes to deal with IESG comments: 1985 o New introduction to provide gentler introduction for the PCN 1986 novice: quick summary of PCN's applicability; quick example of how 1987 it all hangs together in one end-to-end qos scenario; quick 1988 summary of PCN "documentation" 1990 o OAM changed to Operations and Management 1992 o Processed some of the minor suggestions in the Gen-ART Review by 1993 Francis Dupont 1995 o Two wording tweaks in Sections 3.2 & 3.4 (as agreed on mailing 1996 list) 1998 o Updated boilerplate. this draft may include material pre- Nov 10 1999 2008 blah. 2001 12.3. Changes from -08 to -09 2003 Small changes to deal with WG Chair comments: 2005 o tweak language in various places to make it more RFC-like and less 2006 that of a scholarly work, for instance from "we propose" to "this 2007 document describes" 2009 o tweak language in various places to make it a stand alone 2010 architecture document rather than a discussion of the PCN WG. Now 2011 only mentions WG at start of Annex. 2013 o References: IDs are no longer referenced to by the draft name 2015 o References: removed some of less important references to IDs 2017 12.4. Changes from -07 to -08 2019 Small changes from second WG last call: 2021 o Section 2: added definition for PCN-admissible-rate and PCN- 2022 supportable-rate. Small changes to use these terms as follows: 2023 Section 3, bullets 2 & 9; S6.1 para 1; S6.2 para1; S6.3 bullet 3; 2024 added to Figs 1 & 2. 2026 o added the phrase "(others might be possible") before the list of 2027 approaches in Section 6.3, 7.4 & 7.5. 2029 o added references to RFC2753 (A framework for policy-based 2030 admission control) in S7.4 & S7.5. 2032 o throughout, updated references now that marking behaviour & 2033 baseline encoding are WG drafts. 2035 o a few typos corrected 2037 12.5. Changes from -06 to -07 2039 References re-formatted to pass ID nits. No other changes. 2041 12.6. Changes from -05 to -06 2043 Minor clarifications throughout, the least insignificant are as 2044 follows: 2046 o Section 1: added to the list of encoding states in an 'extended' 2047 scheme: "or perhaps further encoding states as suggested in 2048 draft-westberg-pcn-load-control" 2050 o Section 2: added definition for PCN-colouring (to clarify that the 2051 term is used consistently differently from 'PCN-marking') 2053 o Section 6.1 and 6.2: added "(others might be possible)" before the 2054 list of high level approaches for making flow admission 2055 (termination) decisions. 2057 o Section 6.2: corrected a significant typo in 2nd bullet (more -> 2058 less) 2060 o Section 6.3: corrected a couple of significant typos in Figure 2 2062 o Section 6.5 (PCN-traffic) re-written for clarity. Non PCN-traffic 2063 contributing to PCN meters is now given as an example (there may 2064 be cases where don't need to meter it). 2066 o Section 7.7: added to the text about encapsulation being done 2067 within the PCN-domain: "Note: A tunnel will not provide this 2068 behaviour if it complies with [RFC3168] tunnelling in either mode, 2069 but it will if it complies with [RFC4301] IPSec tunnelling." 2071 o Section 7.7: added mention of [RFC4301] to the text about 2072 decapsulation being done within the PCN-domain. 2074 o Section 8: deleted the text about design goals, since this is 2075 already covered adequately earlier eg in S3. 2077 o Section 11: replaced the last sentence of bullet 1 by "There is 2078 nothing specific to PCN." 2080 o Appendix: added to open issues: possibility of automatically and 2081 periodically probing. 2083 o References: Split out Normative references (RFC2474 & RFC3246). 2085 12.7. Changes from -04 to -05 2087 Minor nits removed as follows: 2089 o Further minor changes to reflect that baseline encoding is 2090 consensus, standards track document, whilst there can be 2091 (experimental track) encoding extensions 2093 o Traffic conditioning updated to reflect discussions in Dublin, 2094 mainly that PCN-interior-nodes don't police PCN-traffic (so 2095 deleted bullet in S7.1) and that it is not advised to have non 2096 PCN-traffic that shares the same capacity (on a link) as PCN- 2097 traffic (so added bullet in S6.5) 2099 o Probing moved into Appendix A and deleted the 'third viewpoint' 2100 (admission control based on the marking of a single packet like an 2101 RSVP PATH message) - since this isn't really probing, and in any 2102 case is already mentioned in S6.1. 2104 o Minor changes to S9 Operations and management - mainly to reflect 2105 that consensus on marking behaviour has simplified things so eg 2106 there are fewer parameters to configure. 2108 o A few terminology-related errors expunged, and two pictures added 2109 to help. 2111 o Re-phrased the claim about the natural decision point in S7.4 2113 o Clarified that extended encoding schemes need to explain their 2114 interactions with (or assumptions about) tunnelling (S7.7) and how 2115 they meet the guidelines of BCP124 (S6.6) 2117 o Corrected the third bullet in S6.2 (to reflect consensus about 2118 PCN-marking) 2120 12.8. Changes from -03 to -04 2122 o Minor changes throughout to reflect the consensus call about PCN- 2123 marking (as reflected in [PCN08-2]). 2125 o Minor changes throughout to reflect the current decisions about 2126 encoding (as reflected in [PCN08-1] and [Moncaster08]). 2128 o Introduction: re-structured to create new sections on Benefits, 2129 Deployment scenarios and Assumptions. 2131 o Introduction: Added pointers to other PCN documents. 2133 o Terminology: changed PCN-lower-rate to PCN-threshold-rate and PCN- 2134 upper-rate to PCN-excess-rate; excess-rate-marking to excess- 2135 traffic-marking. 2137 o Benefits: added bullet about SRLGs. 2139 o Deployment scenarios: new section combining material from various 2140 places within the document. 2142 o S6 (high level functional architecture): re-structured and edited 2143 to improve clarity, and reflect the latest PCN-marking and 2144 encoding drafts. 2146 o S6.4: added claim that the most natural place to make an admission 2147 decision is a PCN-egress-node. 2149 o S6.5: updated the bullet about non-PCN-traffic that uses the same 2150 DSCP as PCN-traffic. 2152 o S6.6: added a section about backwards compatibility with respect 2153 to [RFC4774]. 2155 o Appendix A: added bullet about end-to-end PCN. 2157 o Probing: moved to Appendix B. 2159 o Other minor clarifications, typos etc. 2161 12.9. Changes from -02 to -03 2163 o Abstract: Clarified by removing the term 'aggregated'. Follow-up 2164 clarifications later in draft: S1: expanded PCN-egress-nodes 2165 bullet to mention case where the PCN-feedback-information is about 2166 one (or a few) PCN-marks, rather than aggregated information; S3 2167 clarified PCN-meter; S5 minor changes; conclusion. 2169 o S1: added a paragraph about how the PCN-domain looks to the 2170 outside world (essentially it looks like a Diffserv domain). 2172 o S2: tweaked the PCN-traffic terminology bullet: changed PCN 2173 traffic classes to PCN behaviour aggregates, to be more in line 2174 with traditional Diffserv jargon (-> follow-up changes later in 2175 draft); included a definition of PCN-flows (and corrected a couple 2176 of 'PCN microflows' to 'PCN-flows' later in draft) 2178 o S3.5: added possibility of downgrading to best effort, where PCN- 2179 packets arrive at PCN-ingress-node already ECN marked (CE or ECN 2180 nonce) 2182 o S4: added note about whether talk about PCN operating on an 2183 interface or on a link. In S8.1 (OAM) mentioned that PCN 2184 functionality needs to be configured consistently on either the 2185 ingress or the egress interface of PCN-nodes in a PCN-domain. 2187 o S5.2: clarified that signalling protocol installs flow filter spec 2188 at PCN-ingress-node (& updates after possible re-route) 2190 o S5.6: addressing: clarified 2192 o S5.7: added tunnelling issue of N^2 scaling if you set up a mesh 2193 of tunnels between PCN-boundary-nodes 2195 o S7.3: Clarified the "third viewpoint" of probing (always probe). 2197 o S8.1: clarified that SNMP is only an example; added note that an 2198 operator may be able to not run PCN on some PCN-interior-nodes, if 2199 it knows that these links will never become (pre-)congested; added 2200 note that it may be possible to have different PCN-boundary-node 2201 behaviours for different ingress-egress-aggregates within the same 2202 PCN-domain. 2204 o Appendix: Created an Appendix about "Possible work items beyond 2205 the scope of the current PCN WG Charter". Material moved from 2206 near start of S3 and elsewhere throughout draft. Moved text about 2207 centralised decision node to Appendix. 2209 o Other minor clarifications. 2211 12.10. Changes from -01 to -02 2213 o S1: Benefits: provisioning bullet extended to stress that PCN does 2214 not use RFC2475-style traffic conditioning. 2216 o S1: Deployment models: mentioned, as variant of PCN-domain 2217 extending to end nodes, that may extend to LAN edge switch. 2219 o S3.1: Trust Assumption: added note about not needing PCN-marking 2220 capability if known that an interface cannot become pre-congested. 2222 o S4: now divided into sub-sections 2224 o S4.1: Admission control: added second proposed method for how to 2225 decide to block new flows (PCN-egress-node receives one (or 2226 several) PCN-marked packets). 2228 o S5: Probing sub-section removed. Material now in new S7. 2230 o S5.6: Addressing: clarified how PCN-ingress-node can discover 2231 address of PCN-egress-node 2233 o S5.6: Addressing: centralised node case, added that PCN-ingress- 2234 node may need to know address of PCN-egress-node 2236 o S5.8: Tunnelling: added case of "partially PCN-capable tunnel" and 2237 degraded bullet on this in S6 (Open Issues) 2239 o S7: Probing: new section. Much more comprehensive than old S5.5. 2241 o S8: Operations and Management: substantially revised. 2243 o other minor changes not affecting semantics 2245 12.11. Changes from -00 to -01 2247 In addition to clarifications and nit squashing, the main changes 2248 are: 2250 o S1: Benefits: added one about provisioning (and contrast with 2251 Diffserv SLAs) 2253 o S1: Benefits: clarified that the objective is also to stop PCN- 2254 packets being significantly delayed (previously only mentioned not 2255 dropping packets) 2257 o S1: Deployment models: added one where policing is done at ingress 2258 of access network and not at ingress of PCN-domain (assume trust 2259 between networks) 2261 o S1: Deployment models: corrected MPLS-TE to MPLS 2263 o S2: Terminology: adjusted definition of PCN-domain 2265 o S3.5: Other assumptions: corrected, so that two assumptions (PCN- 2266 nodes not performing ECN and PCN-ingress-node discarding arriving 2267 CE packet) only apply if the PCN WG decides to encode PCN-marking 2268 in the ECN-field. 2270 o S4 & S5: changed PCN-marking algorithm to marking behaviour 2272 o S4: clarified that PCN-interior-node functionality applies for 2273 each outgoing interface, and added clarification: "The 2274 functionality is also done by PCN-ingress-nodes for their outgoing 2275 interfaces (ie those 'inside' the PCN-domain)." 2277 o S4 (near end): altered to say that a PCN-node "should" dedicate 2278 some capacity to lower priority traffic so that it isn't starved 2279 (was "may") 2281 o S5: clarified to say that PCN functionality is done on an 2282 'interface' (rather than on a 'link') 2284 o S5.2: deleted erroneous mention of service level agreement 2286 o S5.5: Probing: re-written, especially to distinguish probing to 2287 test the ingress-egress-aggregate from probing to test a 2288 particular ECMP path. 2290 o S5.7: Addressing: added mention of probing; added that in the case 2291 where traffic is always tunnelled across the PCN-domain, add a 2292 note that he PCN-ingress-node needs to know the address of the 2293 PCN-egress-node. 2295 o S5.8: Tunnelling: re-written, especially to provide a clearer 2296 description of copying on tunnel entry/exit, by adding explanation 2297 (keeping tunnel encaps/decaps and PCN-marking orthogonal), 2298 deleting one bullet ("if the inner header's marking state is more 2299 sever then it is preserved" - shouldn't happen), and better 2300 referencing of other IETF documents. 2302 o S6: Open issues: stressed that "NOTE: Potential solutions are out 2303 of scope for this document" and edited a couple of sentences that 2304 were close to solution space. 2306 o S6: Open issues: added one about scenarios with only one tunnel 2307 endpoint in the PCN domain . 2309 o S6: Open issues: ECMP: added under-admission as another potential 2310 risk 2312 o S6: Open issues: added one about "Silent at start" 2313 o S10: Conclusions: a small conclusions section added 2315 13. References 2317 13.1. Normative References 2319 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 2320 "Definition of the Differentiated Services Field (DS 2321 Field) in the IPv4 and IPv6 Headers", RFC 2474, 2322 December 1998. 2324 [RFC3246] Davie, B., Charny, A., Bennet, J., Benson, K., Le Boudec, 2325 J., Courtney, W., Davari, S., Firoiu, V., and D. 2326 Stiliadis, "An Expedited Forwarding PHB (Per-Hop 2327 Behavior)", RFC 3246, March 2002. 2329 13.2. Informative References 2331 [RFC1633] Braden, B., Clark, D., and S. Shenker, "Integrated 2332 Services in the Internet Architecture: an Overview", 2333 RFC 1633, June 1994. 2335 [RFC2205] Braden, B., Zhang, L., Berson, S., Herzog, S., and S. 2336 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 2337 Functional Specification", RFC 2205, September 1997. 2339 [RFC2211] Wroclawski, J., "Specification of the Controlled-Load 2340 Network Element Service", RFC 2211, September 1997. 2342 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., 2343 and W. Weiss, "An Architecture for Differentiated 2344 Services", RFC 2475, December 1998. 2346 [RFC2747] Baker, F., Lindell, B., and M. Talwar, "RSVP Cryptographic 2347 Authentication", RFC 2747, January 2000. 2349 [RFC2753] Yavatkar, R., Pendarakis, D., and R. Guerin, "A Framework 2350 for Policy-based Admission Control", RFC 2753, 2351 January 2000. 2353 [RFC2983] Black, D., "Differentiated Services and Tunnels", 2354 RFC 2983, October 2000. 2356 [RFC2998] Bernet, Y., Ford, P., Yavatkar, R., Baker, F., Zhang, L., 2357 Speer, M., Braden, R., Davie, B., Wroclawski, J., and E. 2358 Felstaine, "A Framework for Integrated Services Operation 2359 over Diffserv Networks", RFC 2998, November 2000. 2361 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 2362 of Explicit Congestion Notification (ECN) to IP", 2363 RFC 3168, September 2001. 2365 [RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, 2366 P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi- 2367 Protocol Label Switching (MPLS) Support of Differentiated 2368 Services", RFC 3270, May 2002. 2370 [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 2371 Metric for IP Performance Metrics (IPPM)", RFC 3393, 2372 November 2002. 2374 [RFC3411] Harrington, D., Presuhn, R., and B. Wijnen, "An 2375 Architecture for Describing Simple Network Management 2376 Protocol (SNMP) Management Frameworks", STD 62, RFC 3411, 2377 December 2002. 2379 [RFC3726] Brunner, M., "Requirements for Signaling Protocols", 2380 RFC 3726, April 2004. 2382 [RFC4216] Zhang, R. and J. Vasseur, "MPLS Inter-Autonomous System 2383 (AS) Traffic Engineering (TE) Requirements", RFC 4216, 2384 November 2005. 2386 [RFC4301] Kent, S. and K. Seo, "Security Architecture for the 2387 Internet Protocol", RFC 4301, December 2005. 2389 [RFC4303] Kent, S., "IP Encapsulating Security Payload (ESP)", 2390 RFC 4303, December 2005. 2392 [RFC4594] Babiarz, J., Chan, K., and F. Baker, "Configuration 2393 Guidelines for DiffServ Service Classes", RFC 4594, 2394 August 2006. 2396 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 2397 Zekauskas, "A One-way Active Measurement Protocol 2398 (OWAMP)", RFC 4656, September 2006. 2400 [RFC4774] Floyd, S., "Specifying Alternate Semantics for the 2401 Explicit Congestion Notification (ECN) Field", BCP 124, 2402 RFC 4774, November 2006. 2404 [RFC4778] Kaeo, M., "Operational Security Current Practices in 2405 Internet Service Provider Environments", RFC 4778, 2406 January 2007. 2408 [RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion 2409 Marking in MPLS", RFC 5129, January 2008. 2411 [RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching 2412 (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic 2413 Class" Field", RFC 5462, February 2009. 2415 [P.800] "Methods for subjective determination of transmission 2416 quality", ITU-T Recommendation P.800, August 1996. 2418 [Y.1541] "Network Performance Objectives for IP-based Services", 2419 ITU-T Recommendation Y.1541, February 2006. 2421 [PCN08-1] "Baseline Encoding and Transport of Pre-Congestion 2422 Information (work in progress)", Oct 2008. 2424 [PCN08-2] "Metering and marking behaviour of PCN-nodes (work in 2425 progress)", Oct 2008. 2427 [PWE3-08] "Pseudowire Congestion Control Framework (work in 2428 progress)", May 2008. 2430 [Babiarz06] 2431 "SIP Controlled Admission and Preemption (work in 2432 progress)", Oct 2006. 2434 [Behringer07] 2435 "Applicability of Keying Methods for RSVP Security (work 2436 in progress)", Nov 2007. 2438 [Briscoe06] 2439 "An edge-to-edge Deployment Model for Pre-Congestion 2440 Notification: Admission Control over a Diffserv Region 2441 (work in progress)", October 2006. 2443 [Briscoe08-1] 2444 "Emulating Border Flow Policing using Re-PCN on Bulk Data 2445 (work in progress)", Sept 2008. 2447 [Briscoe08-2] 2448 "Tunnelling of Congestion Notification (work in 2449 progress)", July 2008. 2451 [Charny07-1] 2452 "Comparison of Proposed PCN Approaches (work in 2453 progress)", November 2007. 2455 [Charny07-2] 2456 "Pre-Congestion Notification Using Single Marking for 2457 Admission and Termination (work in progress)", 2458 November 2007. 2460 [Charny07-3] 2461 "Email to PCN WG mailing list", November 2007, . 2464 [Charny08] 2465 "Email to PCN WG mailing list", March 2008, . 2468 [Eardley07] 2469 "Email to PCN WG mailing list", October 2007, . 2472 [Hancock02] 2473 "Slide 14 of 'NSIS: An Outline Framework for QoS 2474 Signalling'", May 2002, . 2477 [Iyer03] "An approach to alleviate link overload as observed on an 2478 IP backbone", IEEE INFOCOM , 2003, 2479 . 2481 [Lefaucheur06] 2482 "RSVP Extensions for Admission Control over Diffserv using 2483 Pre-congestion Notification (PCN) (work in progress)", 2484 June 2006. 2486 [Menth07] "PCN-Based Resilient Network Admission Control: The Impact 2487 of a Single Bit"", Technical Report , 2007, . 2491 [Menth08-1] 2492 "Edge-Assisted Marked Flow Termination (work in 2493 progress)", February 2008. 2495 [Menth08-2] 2496 "PCN Encoding for Packet-Specific Dual Marking (PSDM) 2497 (work in progress)", July 2008. 2499 [Menth08-3] 2500 "PCN-Based Admission Control and Flow Termination", 2008, 2501 . 2504 [Moncaster08] 2505 "A three state extended PCN encoding scheme (work in 2506 progress)", June 2008. 2508 [Sarker08] 2509 "Usecases and Benefits of end to end ECN support in PCN 2510 Domains (work in progress)", November 2008. 2512 [Songhurst06] 2513 "Guaranteed QoS Synthesis for Admission Control with 2514 Shared Capacity", BT Technical Report TR-CXR9-2006-001, 2515 Feburary 2006, . 2518 [Style] "Guardian Style", Note: This document uses the 2519 abbreviations 'ie' and 'eg' (not 'i.e.' and 'e.g.'), as in 2520 many style guides, eg, 2007, 2521 . 2523 [Tsou08] "Applicability Statement for the Use of Pre-Congestion 2524 Notification in a Resource-Controlled Network (work in 2525 progress)", November 2008. 2527 [Westberg08] 2528 "LC-PCN: The Load Control PCN Solution (work in 2529 progress)", November 2008. 2531 Appendix A. Possible future work items 2533 This section mentions some topics that are outside the PCN WG's 2534 current charter, but which have been mentioned as areas of interest. 2535 They might be work items for: the PCN WG after a future re- 2536 chartering; some other IETF WG; another standards body; an operator- 2537 specific usage that is not standardised. 2539 NOTE: it should be crystal clear that this section discusses 2540 possibilities only. 2542 The first set of possibilities relate to the restrictions described 2543 in Section 6.3: 2545 o a single PCN-domain encompasses several autonomous systems that do 2546 not trust each other, perhaps by using a mechanism like re-PCN, 2547 [Briscoe08-1]. 2549 o not all the nodes run PCN. For example, the PCN-domain is a 2550 multi-site enterprise network. The sites are connected by a VPN 2551 tunnel; although PCN doesn't operate inside the tunnel, the PCN 2552 mechanisms still work properly because of the good QoS on the 2553 virtual link (the tunnel). Another example is that PCN is 2554 deployed on the general Internet (ie widely but not universally 2555 deployed). 2557 o applying the PCN mechanisms to other types of traffic, ie beyond 2558 inelastic traffic. For instance, applying the PCN mechanisms to 2559 traffic scheduled with the Assured Forwarding per-hop behaviour. 2560 One example could be flow-rate adaptation by elastic applications 2561 that adapt according to the pre-congestion information. 2563 o the aggregation assumption doesn't hold, because the link capacity 2564 is too low. Measurement-based admission control is less accurate, 2565 with a greater risk of over-admission for instance. 2567 o the applicability of PCN mechanisms for emergency use (911, GETS, 2568 WPS, MLPP, etc.) 2570 Other possibilities include: 2572 o Probing. This is discussed in Section A.1 below. 2574 o The PCN-domain extends to the end users. The scenario is 2575 described in [Babiarz06]. The end users need to be trusted to do 2576 their own policing. If there is sufficient traffic, then the 2577 aggregation assumption may hold. A variant is that the PCN-domain 2578 extends out as far as the LAN edge switch. 2580 o indicating pre-congestion through signalling messages rather than 2581 in-band (in the form of PCN-marked packets) 2583 o the decision-making functionality is at a centralised node rather 2584 than at the PCN-boundary-nodes. This requires that the PCN- 2585 egress-node signals PCN-feedback-information to the centralised 2586 node, and that the centralised node signals to the PCN-ingress- 2587 node the decision about admission (or termination). It may need 2588 the centralised node and the PCN-boundary-nodes to be configured 2589 with each other's addresses. The centralised case is described 2590 further in [Tsou08]. 2592 o Signalling extensions for specific protocols (eg RSVP, NSIS). For 2593 example: the details of how the signalling protocol installs the 2594 flowspec at the PCN-ingress-node for an admitted PCN-flow; and how 2595 the signalling protocol carries the PCN-feedback-information. 2596 Perhaps also for other functions such as: coping with failure of a 2597 PCN-boundary-node ([Briscoe06] considers what happens if RSVP is 2598 the QoS signalling protocol); establishing a tunnel across the 2599 PCN-domain if it is necessary to carry ECN marks transparently. 2601 o Policing by the PCN-ingress-node may not be needed if the PCN- 2602 domain can trust that the upstream network has already policed the 2603 traffic on its behalf. 2605 o PCN for Pseudowire: PCN may be used as a congestion avoidance 2606 mechanism for edge to edge pseudowire emulations [PWE3-08]. 2608 o PCN for MPLS: [RFC3270] defines how to support the Diffserv 2609 architecture in MPLS networks (Multi-protocol label switching). 2610 [RFC5129] describes how to add PCN for admission control of 2611 microflows into a set of MPLS aggregates. PCN-marking is done in 2612 MPLS's EXP field (which [RFC5462] re-names the Class of Service 2613 (CoS) field). 2615 o PCN for Ethernet: Similarly, it may be possible to extend PCN into 2616 Ethernet networks, where PCN-marking is done in the Ethernet 2617 header. NOTE: Specific consideration of this extension is outside 2618 the IETF's remit. 2620 A.1. Probing 2622 A.1.1. Introduction 2624 Probing is a potential mechanism to assist admission control. 2626 PCN's admission control, as described so far, is essentially a 2627 reactive mechanism where the PCN-egress-node monitors the pre- 2628 congestion level for traffic from each PCN-ingress-node; if the level 2629 rises then it blocks new flows on that ingress-egress-aggregate. 2630 However, it's possible that an ingress-egress-aggregate carries no 2631 traffic, and so the PCN-egress-node can't make an admission decision 2632 using the usual method described earlier. 2634 One approach is to be "optimistic" and simply admit the new flow. 2635 However it's possible to envisage a scenario where the traffic levels 2636 on other ingress-egress-aggregates are already so high that they're 2637 blocking new PCN-flows, and admitting a new flow onto this 'empty' 2638 ingress-egress-aggregate adds extra traffic onto a link that is 2639 already pre-congested - which may 'tip the balance' so that PCN's 2640 flow termination mechanism is activated or some packets are dropped. 2641 This risk could be lessened by configuring on each link sufficient 2642 'safety margin' above the PCN-threshold-rate. 2644 An alternative approach is to make PCN a more proactive mechanism. 2645 The PCN-ingress-node explicitly determines, before admitting the 2646 prospective new flow, whether the ingress-egress-aggregate can 2647 support it. This can be seen as a "pessimistic" approach, in 2648 contrast to the "optimism" of the approach above. It involves 2649 probing: a PCN-ingress-node generates and sends probe packets in 2650 order to test the pre-congestion level that the flow would 2651 experience. 2653 One possibility is that a probe packet is just a dummy data packet, 2654 generated by the PCN-ingress-node and addressed to the PCN-egress- 2655 node. 2657 A.1.2. Probing functions 2659 The probing functions are: 2661 o Make decision that probing is needed. As described above, this is 2662 when the ingress-egress-aggregate (or the ECMP path - Section 6.4) 2663 carries no PCN-traffic. An alternative is always to probe, ie 2664 probe before admitting every PCN-flow. 2666 o (if required) Communicate the request that probing is needed - the 2667 PCN-egress-node signals to the PCN-ingress-node that probing is 2668 needed 2670 o (if required) Generate probe traffic - the PCN-ingress-node 2671 generates the probe traffic. The appropriate number (or rate) of 2672 probe packets will depend on the PCN-metering algorithm; for 2673 example an excess-traffic-metering algorithm triggers fewer PCN- 2674 marks than a threshold-metering algorithm, and so will need more 2675 probe packets. 2677 o Forward probe packets - as far as PCN-interior-nodes are 2678 concerned, probe packets are handled the same as (ordinary data) 2679 PCN-packets, in terms of routing, scheduling and PCN-marking. 2681 o Consume probe packets - the PCN-egress-node consumes probe packets 2682 to ensure that they don't travel beyond the PCN-domain. 2684 A.1.3. Discussion of rationale for probing, its downsides and open 2685 issues 2687 It is an unresolved question whether probing is really needed, but 2688 two viewpoints have been put forward as to why it is useful. The 2689 first is perhaps the most obvious: there is no PCN-traffic on the 2690 ingress-egress-aggregate. The second assumes that multipath routing 2691 ECMP is running in the PCN-domain. We now consider each in turn. 2693 The first viewpoint assumes the following: 2695 o There is no PCN-traffic on the ingress-egress-aggregate (so a 2696 normal admission decision cannot be made). 2698 o Simply admitting the new flow has a significant risk of leading to 2699 overload: packets dropped or flows terminated. 2701 On the former bullet, [Eardley07] suggests that, during the future 2702 busy hour of a national network with about 100 PCN-boundary-nodes, 2703 there are likely to be significant numbers of aggregates with very 2704 few flows under nearly all circumstances. 2706 The latter bullet could occur if new flows start on many of the empty 2707 ingress-egress-aggregates, which together overload a link in the PCN- 2708 domain. To be a problem this would probably have to happen in a 2709 short time period (flash crowd) because, after the reaction time of 2710 the system, other (non-empty) ingress-egress-aggregates that pass 2711 through the link will measure pre-congestion and so block new flows. 2712 Also, flows naturally end anyway. 2714 The downsides of probing for this viewpoint are: 2716 o Probing adds delay to the admission control process. 2718 o Sufficient probing traffic has to be generated to test the pre- 2719 congestion level of the ingress-egress-aggregate. But the probing 2720 traffic itself may cause pre-congestion, causing other PCN-flows 2721 to be blocked or even terminated - and in the flash crowd scenario 2722 there will be probing on many ingress-egress-aggregates. 2724 The second viewpoint applies in the case where there is multipath 2725 routing (ECMP) in the PCN-domain. Note that ECMP is often used on 2726 core networks. There are two possibilities: 2728 (1) If admission control is based on measurements of the ingress- 2729 egress-aggregate, then the viewpoint that probing is useful assumes: 2731 o there's a significant chance that the traffic is unevenly balanced 2732 across the ECMP paths, and hence there's a significant risk of 2733 admitting a flow that should be blocked (because it follows an 2734 ECMP path that is pre-congested) or blocking a flow that should be 2735 admitted. 2737 o Note: [Charny07-3] suggests unbalanced traffic is quite possible, 2738 even with quite a large number of flows on a PCN-link (eg 1000) 2739 when Assumption 3 (aggregation) is likely to be satisfied. 2741 (2) If admission control is based on measurements of pre-congestion 2742 on specific ECMP paths, then the viewpoint that probing is useful 2743 assumes: 2745 o There is no PCN-traffic on the ECMP path on which to base an 2746 admission decision. 2748 o Simply admitting the new flow has a significant risk of leading to 2749 overload. 2751 o The PCN-egress-node can match a packet to an ECMP path. 2753 o Note: This is similar to the first viewpoint and so similarly 2754 could occur in a flash crowd if a new flow starts more-or-less 2755 simultaneously on many of the empty ECMP paths. Because there are 2756 several (sometimes many) ECMP paths between each pair of PCN- 2757 boundary-nodes, it's presumably more likely that an ECMP path is 2758 'empty' than an ingress-egress-aggregate is. To constrain the 2759 number of ECMP paths, a few tunnels could be set-up between each 2760 pair of PCN-boundary-nodes. Tunnelling also solves the issue in 2761 the bullet immediately above (which is otherwise hard because an 2762 ECMP routing decision is made independently on each node). 2764 The downsides of probing for this viewpoint are: 2766 o Probing adds delay to the admission control process. 2768 o Sufficient probing traffic has to be generated to test the pre- 2769 congestion level of the ECMP path. But there's the risk that the 2770 probing traffic itself may cause pre-congestion, causing other 2771 PCN-flows to be blocked or even terminated. 2773 o The PCN-egress-node needs to consume the probe packets to ensure 2774 they don't travel beyond the PCN-domain, since they might confuse 2775 the destination end node. This is non-trivial, since probe 2776 packets are addressed to the destination end node, in order to 2777 test the relevant ECMP path (ie they are not addressed to the PCN- 2778 egress-node, unlike the first viewpoint above). 2780 The open issues associated with this viewpoint include: 2782 o What rate and pattern of probe packets does the PCN-ingress-node 2783 need to generate, so that there's enough traffic to make the 2784 admission decision? 2786 o What difficulty does the delay (whilst probing is done), and 2787 possible packet drops, cause applications? 2789 o Can the delay be alleviated by automatically and periodically 2790 probing on the ingress-egress-aggregate? Or does this add too 2791 much overhead? 2793 o Are there other ways of dealing with the flash crowd scenario? 2794 For instance, by limiting the rate at which new flows are 2795 admitted; or perhaps by a PCN-egress-node blocking new flows on 2796 its empty ingress-egress-aggregates when its non-empty ones are 2797 pre-congested. 2799 o (Second viewpoint only) How does the PCN-egress-node disambiguate 2800 probe packets from data packets (so it can consume the former)? 2801 The PCN-egress-node must match the characteristic setting of 2802 particular bits in the probe packet's header or body - but these 2803 bits must not be used by any PCN-interior-node's ECMP algorithm. 2804 In the general case this isn't possible, but it should be possible 2805 for a typical ECMP algorithm (which examines: the source and 2806 destination IP addresses and port numbers, the protocol ID, and 2807 the DSCP). 2809 Author's Address 2811 Philip Eardley 2812 BT 2813 B54/77, Sirius House Adastral Park Martlesham Heath 2814 Ipswich, Suffolk IP5 3RE 2815 United Kingdom 2817 Email: philip.eardley@bt.com