idnits 2.17.1 draft-ietf-diffserv-phb-ef-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 1) being 441 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 4 instances of too long lines in the document, the longest one being 3 characters in excess of 72. ** There are 43 instances of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Downref: Normative reference to an Informational RFC: RFC 2475 -- Possible downref: Non-RFC (?) normative reference: ref. '2BIT' -- Possible downref: Non-RFC (?) normative reference: ref. 'CBQ' ** Downref: Normative reference to an Informational RFC: RFC 2415 -- Possible downref: Non-RFC (?) normative reference: ref. 'LCN' Summary: 8 errors (**), 0 flaws (~~), 3 warnings (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Engineering Task Force Van Jacobson 2 Differentiated Services Working Group Kathleen Nichols 3 Internet Draft Cisco Systems 4 Expires August, 1999 Kedarnath Poduri 5 Bay Networks 6 February, 1999 8 An Expedited Forwarding PHB 9 11 Status of this Memo 13 This document is an Internet-Draft and is in full conformance 14 with all provisions of Section 10 of RFC2026. 16 This document is a product of the IETF Differentiated Services 17 Working Group. Comments are solicited and should be directed 18 to the working group mailing list. 20 Internet-Drafts are working documents of the Internet Engineering 21 Task Force (IETF), its areas, and its working groups. Note that 22 other groups may also distribute working documents as 23 Internet-Drafts. 25 Internet-Drafts are draft documents valid for a maximum of six 26 months and may be updated, replaced, or obsoleted by other 27 documents at any time. It is inappropriate to use Internet- 28 Drafts as reference material or to cite them other than as 29 "work in progress." 31 The list of current Internet-Drafts can be accessed at 32 http://www.ietf.org/ietf/1id-abstracts.txt 34 The list of Internet-Draft Shadow Directories can be accessed at 35 http://www.ietf.org/shadow.html. 37 Distribution of this memo is unlimited. 39 Abstract 41 The definition of PHBs (per-hop forwarding behaviors) is a 42 critical part of the work of the Diffserv Working Group. This 43 document describes a PHB called Expedited Forwarding. We show the 44 generality of this PHB by noting that it can be produced by more 45 than one mechanism and give an example of its use to produce at 46 least one service, a Virtual Leased Line. A recommended 47 codepoint for this PHB is given. 49 A pdf version of this document is available at 50 ftp://ftp.ee.lbl.gov/papers/ef_phb.pdf 52 1. Introduction 54 Network nodes that implement the differentiated services 55 enhancements to IP use a codepoint in the IP header to select a 56 per-hop behavior (PHB) as the specific forwarding treatment for 57 that packet [RFC2474, RFC2475]. This draft describes a 58 particular PHB called expedited forwarding (EF). The EF PHB can 59 be used to build a low loss, low latency, low jitter, assured 60 bandwidth, end-to-end service through DS domains. Such a service 61 appears to the endpoints like a point-to-point connection or a 62 ?virtual leased line?. This service has also been described as 63 Premium service [2BIT]. 65 Loss, latency and jitter are all due to the queues traffic 66 experiences while transiting the network. Therefore providing 67 low loss, latency and jitter for some traffic aggregate means 68 ensuring that the aggregate sees no (or very small) queues. 69 Queues arise when (short-term) traffic arrival rate exceeds 70 departure rate at some node. Thus a service that ensures no 71 queues for some aggregate is equivalent to bounding rates such 72 that, at every transit node, the aggregate's maximum arrival rate 73 is less than that aggregate's minimum departure rate. 75 Creating such a service has two parts: 77 1) Configuring nodes so that the aggregate has a well-defined 78 minimum departure rate. (?Well-defined? means independent of the 79 dynamic state of the node. In particular, independent of the 80 intensity of other traffic at the node.) 82 2) Conditioning the aggregate (via policing and shaping) so that its 83 arrival rate at any node is always less than that node's 84 configured minimum departure rate. 86 The EF PHB provides the first part of the service. The network 87 boundary traffic conditioners described in [RFC2475] provide the 88 second part. 90 The EF PHB is not a mandatory part of the Differentiated Services 91 architecture, i.e., a node is not required to implement the EF 92 PHB in order to be considered DS-compliant. However, when a DS- 93 compliant node claims to implement the EF PHB, the implementation 94 must conform to the specification given in this document. 96 The next sections describe the EF PHB in detail and give examples 97 of how it might be implemented. The keywords ?MUST?, ?MUST NOT?, 98 ?REQUIRED?, ?SHOULD?, ?SHOULD NOT?, and ?MAY? that appear in this 99 document are to be interpreted as described in [Bradner97]. 101 2. Description of EF per-hop behavior 103 The EF PHB is defined as a forwarding treatment for a particular 104 diffserv aggregate where the departure rate of the aggregate's 105 packets from any diffserv node must equal or exceed a 106 configurable rate. The EF traffic SHOULD receive this rate 107 independent of the intensity of any other traffic attempting to 108 transit the node. It SHOULD average at least the configured rate 109 when measured over any time interval equal to or longer than the 110 time it takes to send an output link MTU sized packet at the 111 configured rate. (Behavior at time scales shorter than a packet 112 time at the configured rate is deliberately not specified.) The 113 configured minimum rate MUST be settable by a network 114 administrator (using whatever mechanism the node supports for 115 non-volatile configuration). 117 If the EF PHB is implemented by a mechanism that allows unlimited 118 preemption of other traffic (e.g., a priority queue), the 119 implementation MUST include some means to limit the damage EF 120 traffic could inflict on other traffic (e.g., a token bucket rate 121 limiter). Traffic that exceeds this limit MUST be discarded. This 122 maximum EF rate, and burst size if appropriate, MUST be settable 123 by a network administrator (using whatever mechanism the node 124 supports for non-volatile configuration). The minimum and maximum 125 rates may be the same and configured by a single parameter. 127 The Appendix describes how this PHB can be used to construct end- 128 to-end services. 130 2.2 Example Mechanisms to Implement the EF PHB 132 Several types of queue scheduling mechanisms may be employed to 133 deliver the forwarding behavior described in section 2.1 and thus 134 implement the EF PHB. A simple priority queue will give the 135 appropriate behavior as long as there is no higher priority queue 136 that could preempt the EF for more than a packet time at the 137 configured rate. (This could be accomplished by having a rate 138 policer such as a token bucket associated with each priority 139 queue to bound how much the queue can starve other traffic.) 141 It's also possible to use a single queue in a group of queues 142 serviced by a weighted round robin scheduler where the share of 143 the output bandwidth assigned to the EF queue is equal to the 144 configured rate. This could be implemented, for example, using 145 one PHB of a Class Selector Compliant set of PHBs [RFC2474]. 147 Another possible implementation is a CBQ [CBQ] scheduler that 148 gives the EF queue priority up to the configured rate. 150 All of these mechanisms have the basic properties required for 151 the EF PHB though different choices result in different ancillary 152 behavior such as jitter seen by individual microflows. See 153 Appendix A.3 for simulations that quantify some of these differences. 155 2.3 Recommended codepoint for this PHB 157 Codepoint 101110 is recommended for the EF PHB. 159 2.4 Mutability 161 Packets marked for EF PHB MAY be remarked at a DS domain boundary 162 only to other codepoints that satisfy the EF PHB. Packets marked 163 for EF PHBs SHOULD NOT be demoted or promoted to another PHB by a 164 DS domain. 166 2.5 Tunneling 168 When EF packets are tunneled, the tunneling packets must be 169 marked as EF. 171 2.6 Interaction with other PHBs 173 Other PHBs and PHB groups may be deployed in the same DS node or 174 domain with the EF PHB as long as the requirement of section 2.1 175 is met. 177 3. Security Considerations 179 To protect itself against denial of service attacks, the edge of 180 a DS domain MUST strictly police all EF marked packets to a rate 181 negotiated with the adjacent upstream domain. (This rate must be 182 <= the EF PHB configured rate.) Packets in excess of the 183 negotiated rate MUST be dropped. If two adjacent domains have 184 not negotiated an EF rate, the downstream domain MUST use 0 as 185 the rate (i.e., drop all EF marked packets). 187 Since the end-to-end premium service constructed from the EF PHB 188 requires that the upstream domain police and shape EF marked 189 traffic to meet the rate negotiated with the downstream domain, 190 the downstream domain's policer should never have to drop 191 packets. Thus these drops SHOULD be noted (e.g., via SNMP traps) 192 as possible security violations or serious misconfiguration. 193 Similarly, since the aggregate EF traffic rate is constrained at 194 every interior node, the EF queue should never overflow so if it 195 does the drops SHOULD be noted as possible attacks or serious 196 misconfiguration. 198 4. References 200 [Bradner97] S. Bradner, ?Key words for use in RFCs to Indicate 201 Requirement Levels?, Internet RFC 2119, March 1997. 203 [RFC2474] K. Nichols, S. Blake, F. Baker, and D. Black, 204 ?Definition of the Differentiated Services Field (DS Field) in 205 the IPv4 and IPv6 Headers?, Internet RFC 2474, December 1998. 207 [RFC2475] D. Black, S. Blake, M. Carlson, E. Davies, Z. Wang, and 208 W. Weiss, ?An Architecture for Differentiated Services?, 209 Internet RFC 2475, December 1998. 211 [2BIT] K. Nichols, V. Jacobson, and L. Zhang, ?A Two-bit 212 Differentiated Services Architecture for the Internet?, Internet 213 Draft , November 1997, 214 ftp://ftp.ee.lbl.gov/papers/dsarch.pdf 216 [CBQ] S. Floyd and V. Jacobson, ?Link-sharing and Resource 217 Management Models for Packet Networks?, IEEE/ACM Transactions on 218 Networking, Vol. 3 no. 4, pp. 365-386, August 1995. 220 [RFC2415] K. Poduri and K. Nichols, ?Simulation Studies of 221 Increased Initial TCP Window Size?, Internet RFC 2415, 222 September 1998. 224 [LCN] K. Nichols, ?Improving Network Simulation with 225 Feedback?, Proceedings of LCN '98, October, 1998 227 5. Authors' Addresses 229 Van Jacobson 230 Cisco Systems, Inc 231 170 W. Tasman Drive 232 San Jose, CA 95134-1706 233 van@cisco.com 235 Kathleen Nichols 236 Cisco Systems, Inc 237 170 W. Tasman Drive 238 San Jose, CA 95134-1706 239 kmn@cisco.com 241 Kedarnath Poduri 242 Bay Networks, Inc. 243 4401 Great America Parkway 244 Santa Clara, CA 95052-8185 245 kpoduri@baynetworks.com 247 Appendix A: Example use of and experiences with the EF PHB 249 A.1 Virtual Leased Line Service 251 A VLL Service, also known as Premium service [2BIT], is 252 quantified by a peak bandwidth. 254 A.2 Experiences with its use in ESNET 256 A prototype of the VLL service has been deployed on DOE's ESNet 257 backbone. This uses weighted-round-robin queuing features of 258 Cisco 75xx series routers to implement the EF PHB. The early 259 tests have been very successful and work is in progress to make 260 the service available on a routine production basis (see 261 ftp://ftp.ee.lbl.gov/talks/vj-doeqos.pdf and 262 ftp://ftp.ee.lbl.gov/talks/vj-i2qos-may98.pdf for details). 264 A.3 Simulation Results 266 A.3.1 Jitter variation 268 In section 2.2, we pointed out that a number of mechanisms might 269 be used to implement the EF PHB. The simplest of these is a 270 priority queue (PQ) where the arrival rate of the queue is 271 strictly less than its service rate. As jitter comes from the 272 queuing delay along the path, a feature of this implementation is 273 that EF-marked microflows will see very little jitter at their 274 subscribed rate since packets spend little time in queues. The EF 275 PHB does not have an explicit jitter requirement but it is clear 276 from the definition that the expected jitter in a packet stream 277 that uses a service based on the EF PHB will be less with PQ than 278 with best-effort delivery. We used simulation to explore how 279 weighted round-robin (WRR) compares to PQ in jitter. We chose 280 these two since they?re the best and worst cases, respectively, 281 for jitter and we wanted to supply rough guidelines for EF 282 implementers choosing to use WRR or similar mechanisms. 284 Our simulation model is implemented in a modified ns-2 described 285 in [RFC2415] and [LCN]. We used the CBQ modules included with ns- 286 2 as a basis to implement priority queuing and WRR. Our topology 287 has six hops with decreasing bandwidth in the direction of a 288 single 1.5 Mbps bottleneck link (see figure 6). Sources produce 289 EF-marked packets at an average bit rate equal to their 290 subscribed packet rate. Packets are produced with a variation of 291 +-10% from the interpacket spacing at the subscribed packet rate. 292 The individual source rates were picked aggregate to 30% of the 293 bottleneck link or 450 Kbps. A mixture of FTPs and HTTPs is then 294 used to fill the link. Individual EF packet sources produce 295 either all 160 byte packets or all 1500 byte packets. Though we 296 present the statistics of flows with one size of packet, all of 297 the experiments used a mixture of short and long packet EF 298 sources so the EF queues had a mix of both packet lengths. 300 We defined jitter as the absolute value of the difference between 301 the arrival times of two adjacent packets minus their departure 302 times, |(aj-dj) - (ai-di)|. For the target flow of each 303 experiment, we record the median and 90th percentile values of 304 jitter (expressed as % of the subscribed EF rate) in a table. The 305 pdf version of this document contains graphs of the jitter 306 percentiles. 308 Our experiments compared the jitter of WRR and PQ implementations 309 of the EF PHB. We assessed the effect of different choices of WRR 310 queue weight and number of queues on jitter. For WRR, we define 311 the service-to-arrival rate ratio as the service rate of the EF 312 queue (or the queue?s minimum share of the output link) times the 313 output link bandwidth divided by the peak arrival rate of EF- 314 marked packets at the queue. Results will not be stable if the 315 WRR weight is chosen to exactly balance arrival and departure 316 rates thus we used a minimum service-to-arrival ratio of 1.03. In 317 our simulations this means that the EF queue gets at least 31% of 318 the output links. In WRR simulations we kept the link full with 319 other traffic as described above, splitting the non-EF-marked 320 traffic among the non-EF queues. (It should be clear from the 321 experiment description that we are attempting to induce worst- 322 case jitter and do not expect these settings or traffic to 323 represent a ?normal? operating point.) 325 Our first set of experiments uses the minimal service-to-arrival 326 ratio of 1.06 and we vary the number of individual microflows 327 composing the EF aggregate from 2 to 36. We compare these to a PQ 328 implementation with 24 flows. First, we examine a microflow at a 329 subscribed rate of 56 Kbps sending 1500 byte packets, then one at 330 the same rate but sending 160 byte packets. Table 1 shows the 331 50th and 90th percentile jitter in percent of a packet time at the 332 subscribed rate. Figure 1 plots the 1500 byte flows and figure 2 333 the 160 byte flows. Note that a packet-time for a 1500 byte 334 packet at 56 Kbps is 214 ms, for a 160 byte packet 23 ms. The 335 jitter for the large packets rarely exceeds half a subscribed 336 rate packet-time, though most jitters for the small packets are 337 at least one subscribed rate packet-time. Keep in mind that the 338 EF aggregate is a mixture of small and large packets in all cases 339 so short packets can wait for long packets in the EF queue. PQ 340 gives a very low jitter. 342 Table 1: Variation in jitter with number of EF flows: 343 Service/arrival ratio of 1.06 and subscription rate of 56 Kbps 344 (all values given as % of subscribed rate) 346 1500 byte pack. 160 byte packet 347 # EF flows 50th % 90th % 50th % 90th % 348 PQ (24) 1 5 17 43 349 2 11 47 96 513 350 4 12 35 100 278 351 8 10 25 96 126 352 24 18 47 96 143 354 Next we look at the effects of increasing the service-to-arrival 355 ratio. This means that EF packets should remain enqueued for less 356 time though the bandwidth available to the other queues remains 357 the same. In this set of experiments the number of flows in the 358 EF aggregate was fixed at eight and the total number of queues at 359 five (four non-EF queues). Table 2 shows the results for 1500 and 360 160 byte flows. Figures 3 plots the 1500 byte results and figure 361 4 the 160 byte results. Performance gains leveled off at service- 362 to-arrival ratios of 1.5. Note that the higher service-to-arrival 363 ratios do not give the same performance as PQ, but now 90% of 364 packets experience less than a subscribed packet-time of jitter 365 even for the small packets. 367 Table 2: Variation in Jitter of EF flows: service/arrival ratio varies, 368 8 flow aggregate, 56 Kbps subscribed rate 370 WRR 1500 byte pack. 160 byte packet 371 Ser/Arr 50th % 90th % 50th % 90th % 372 PQ 1 3 17 43 373 1.03 14 27 100 178 374 1.30 7 21 65 113 375 1.50 5 13 57 104 376 1.70 5 13 57 100 377 2.00 5 13 57 104 378 3.00 5 13 57 100 380 Increasing the number of queues at the output interfaces can lead 381 to more variability in the service time for EF packets so we 382 carried out an experiment varying the number of queues at each 383 output port. We fixed the number of flows in the aggregate to 384 eight and used the minimal 1.03 service-to-arrival ratio. Results 385 are shown in figure 5 and table 3. Figure 5 includes PQ with 8 386 flows as a baseline. 388 Table 3: Variation in Jitter with Number of Queues at Output Interface: 389 Service-to-arrival ratio is 1.03, 8 flow aggregate 391 # EF 1500 byte packet 392 flows 50th % 90th % 393 PQ (8) 1 3 394 2 7 21 395 4 7 21 396 6 8 22 397 8 10 23 399 It appears that most jitter for WRR is low and can be reduced by 400 a proper choice of the EF queue's WRR share of the output link 401 with respect to its subscribed rate. As noted, WRR is a worst 402 case while PQ is the best case. Other possibilities include WFQ 403 or CBQ with a fixed rate limit for the EF queue but giving it 404 priority over other queues. We expect the latter to have 405 performance nearly identical with PQ though future simulations 406 are needed to verify this. We have not yet systematically 407 explored effects of hop count, EF allocations other than 30% of 408 the link bandwidth, or more complex topologies. The information 409 in this section is not part of the EF PHB definition but provided 410 simply as background to guide implementers. 412 A.3.2 VLL service 414 We used simulation to see how well a VLL service built from the 415 EF PHB behaved, that is, does it look like a `leased line' at the 416 subscribed rate. In the simulations of the last section, none of 417 the EF packets were dropped in the network and the target rate 418 was always achieved for those CBR sources. However, we wanted to 419 see if VLL really looks like a `wire' to a TCP using it. So we 420 simulated long-lived FTPs using a VLL service. Table 4 gives the 421 percentage of each link allocated to EF traffic (bandwidths are 422 lower on the links with fewer EF microflows), the subscribed VLL 423 rate, the average rate for the same type of sender-receiver pair 424 connected by a full duplex dedicated link at the subscribed rate 425 and the average of the VLL flows for each simulation (all sender- 426 receiver pairs had the same value). Losses only occur when the 427 input shaping buffer overflows but not in the network. The target 428 rate is not achieved due to the well-known TCP behavior. 430 Table 4: Performance of FTPs using a VLL service 432 % link Average delivered rate (Kbps) 433 to EF Subscribed Dedicated VLL 434 20 100 90 90 435 40 150 143 143 436 60 225 213 215