idnits 2.17.1 draft-ietf-diffserv-phb-ef-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-25) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 1) being 386 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 5 instances of too long lines in the document, the longest one being 3 characters in excess of 72. ** There are 8 instances of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 192 has weird spacing: '...tecture for D...' == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'HEADER' -- Possible downref: Non-RFC (?) normative reference: ref. 'ARCH' -- Possible downref: Non-RFC (?) normative reference: ref. '2BIT' -- Possible downref: Non-RFC (?) normative reference: ref. 'CBQ' -- Possible downref: Non-RFC (?) normative reference: ref. 'IW' -- Possible downref: Non-RFC (?) normative reference: ref. 'LCN' Summary: 10 errors (**), 0 flaws (~~), 4 warnings (==), 8 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Engineering Task Force Van Jacobson 2 Differentiated Services Working Group Kathleen Nichols 3 Internet Draft Cisco Systems 4 Expires May, 1999 Kedarnath Poduri 5 Bay Networks 6 November, 1998 8 An Expedited Forwarding PHB 9 11 Status of this Memo 13 This document is a submission to the IETF Differentiated Services 14 (DiffServ) Working Group. Comments are solicited and should be 15 addressed to the working group mailing list or to the editor. 17 This document is an Internet-Draft. Internet Drafts are working 18 documents of the Internet Engineering Task Force (IETF), its areas, and 19 its working Groups. Note that other groups may also distribute working 20 documents as Internet Drafts. 22 Internet-Drafts draft documents are valid for a maximum of six months 23 and may be updated, replaced, or obsolete by other documents at any 24 time. It is inappropriate to use Internet-Drafts as reference material 25 or to cite them other than as "work in progress." 27 To view the entire list of current Internet-Drafts, please check the 28 "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow 29 Directories on ftp.is.co.za (Africa), ftp.nordu.net (Northern Europe), 30 ftp.nis.garr.it (Southern Europe), munnari.oz.au (Pacific Rim), 31 ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast). 33 Distribution of this memo is unlimited. 35 Abstract 37 The definition of PHBs (per-hop forwarding behaviors) is a critical part 38 of the work of the Diffserv Working Group. This document describes a 39 PHB called Expedited Forwarding. We show the generality of this PHB by 40 noting that it can be produced by more than one mechanism and give an 41 example of its use to produce at least one service, a Virtual Leased 42 Line. A recommended codepoint for this PHB is given. 44 A pdf version of this document is available at 45 ftp://ftp.ee.lbl.gov/papers/ef_phb.pdf 47 1. Introduction 49 Network nodes that implement the differentiated services enhancements to 50 IP use a codepoint in the IP header to select a per-hop behavior (PHB) 51 as the specific forwarding treatment for that packet [HEADER, ARCH]. 52 This draft describes a particular PHB called expedited forwarding (EF). 53 The EF PHB can be used to build a low loss, low latency, low jitter, 54 assured bandwidth, end-to-end service through DS domains. Such a 55 service appears to the endpoints like a point-to-point connection or a 56 "virtual leased line". This service has also been described as Premium 57 service [2BIT]. 59 Loss, latency and jitter are all due to the queues traffic experiences 60 while transiting the network. Therefore providing low loss, latency and 61 jitter for some traffic aggregate means ensuring that the aggregate sees 62 no (or very small) queues. Queues arise when (short-term) traffic 63 arrival rate exceeds departure rate at some node. Thus a service that 64 ensures no queues for some aggregate is equivalent to bounding rates 65 such that, at every transit node, the aggregate's max arrival rate is 66 less than that aggregate's min departure rate. 68 Creating such a service has two parts: 70 1) Configuring nodes so that the aggregate has a well-defined minimum 71 departure rate. (`Well-defined' means independent of the dynamic state 72 of the node. In particular, independent of the intensity of other 73 traffic at the node.) 75 2) Conditioning the aggregate (via policing and shaping) so that it's 76 arrival rate at any node is always less than that node's configured 77 minimum departure rate. The EF PHB provides the first part of the 78 service. The network boundary traffic conditioners described in [ARCH] 79 provide the second part. 81 The EF PHB is not a mandatory part of the Differentiated Services 82 architecture. I.e., a node is not required to implement the EF PHB in 83 order to be considered DS-compliant. However, when a DS-compliant node 84 claims to implement the EF PHB, the implementation must conform to the 85 specification given in this document. 87 The next sections describe the EF PHB in detail and give examples of how 88 it might be implemented. The keywords "MUST", "MUST NOT", "REQUIRED", 89 "SHOULD", "SHOULD NOT", and "MAY" that appear in this document are to be 90 interpreted as described in [Bradner97]. 92 2. Description of EF per-hop behavior 94 The EF PHB is defined as a forwarding treatment for a particular 95 diffserv aggregate where the departure rate of the aggregate's packets 96 from any diffserv node must equal or exceed a configurable rate. The EF 97 traffic should receive this rate independent of the intensity of any 98 other traffic attempting to transit the node. It should average at 99 least the configured rate when measured over any time interval equal to 100 or longer than a packet time at the configured rate. (Behavior at time 101 scales shorter than a packet time at the configured rate is deliberately 102 not specified.) The configured minimum rate must be settable by a 103 network administrator (using whatever mechanism the node supports for 104 non-volatile configuration). 106 If the EF PHB is implemented by a mechanism that allows unlimited 107 preemption of other traffic (e.g., a priority queue), the implementation 108 must include some means to limit the damage EF traffic could inflict on 109 other traffic (e.g., a token bucket rate limiter). This maximum EF rate 110 must be settable by a network administrator (using whatever mechanism 111 the node supports for non-volatile configuration). The minimum and 112 maximum rates can be the same and configured by a single parameter. 114 The Appendix describes how this PHB can be used to construct end-to-end 115 services. 117 2.2 Example Mechanisms to Implement the EF PHB 119 Several types of queue scheduling mechanisms may be employed to deliver 120 the forwarding behavior described in section 2.1 and thus implement the 121 EF PHB. A simple priority queue will give the appropriate behavior as 122 long as there is no higher priority queue that could preempt the EF for 123 more than a packet time at the configured rate. (This could be 124 accomplished by having a rate policer such as a token bucket associated 125 with each priority queue to bound how much the queue can starve other 126 traffic.) 128 It's also possible to use a single queue in a group of queues serviced 129 by a weighted round robin scheduler where the share of the output 130 bandwidth assigned to the EF queue is equal to the configured rate. 131 This could be implemented, for example, using one PHB of a Class 132 Selector Compliant set of PHBs [HEADER]. 134 Another possible implementation is a CBQ [CBQ] scheduler that gives the 135 EF queue priority up to the configured rate. 137 All of these mechanisms give the basic properties required for the EF 138 PHB though different choices result in differences in auxiliary behavior 139 such as jitter seen by individual microflows. See Appendix A.3 for 140 simulations that quantify some of these differences. 142 2.3 Recommended codepoint for this PHB 144 Codepoint 101110 is recommended for the EF PHB. 146 2.4 Mutability 148 Packets marked for EF PHB may be remarked at a DS domain boundary to 149 other codepoints that satisfy the EF PHB only. Packets marked for EF 150 PHBs SHOULD NOT be demoted or promoted to another PHB by a DS domain. 152 2.5 Tunneling 154 When EF packets are tunneled, the tunneling packets must be marked as 155 EF. 157 2.6 Interaction with other PHBs 159 Other PHBs and PHB groups may be deployed in the same DS node or domain 160 with the EF PHB as long as the requirement of section 2.1 is met. 162 3. Security Considerations 164 To protect itself against denial of service attacks, the edge of a DS 165 domain MUST strictly police all EF marked packets to a rate negotiated 166 with the adjacent upstream domain. (This rate must be <= the EF PHB 167 configured rate.) Packets in excess of the negotiated rate MUST be 168 dropped. If two adjacent domains have not negotiated an EF rate, the 169 downstream domain MUST use 0 as the rate (i.e., drop all EF marked 170 packets). 172 Since the end-to-end premium service constructed from the EF PHB 173 requires that the upstream domain police and shape EF marked traffic to 174 meet the rate negotiated with the downstream domain, the downstream 175 domain's policer should never have to drop packets. Thus these drops 176 should be noted (e.g., via SNMP traps) as possible security violations 177 or serious misconfiguration. Similarly, since the aggregate EF traffic 178 rate is constrained at every interior node, the EF queue should never 179 overflow so if it does the drops should be noted as possible attacks or 180 serious misconfiguration. 182 4. References 184 [Bradner97] S. Bradner, "Key words for use in RFCs to Indicate Requirement 185 Levels", Internet RFC 2119, March 1997. 187 [HEADER] K. Nichols, S. Blake, F. Baker, and D. Black, "Definition of the 188 Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers", 189 , August 1998. 191 [ARCH] D. Black, S. Blake, M. Carlson, E. Davies, Z. Wang, and W. Weiss, 192 "An Architecture for Differentiated Services", Internet Draft 193 , August 1998. 195 [2BIT] K. Nichols, V. Jacobson, and L. Zhang, "A Two-bit Differentiated 196 Services Architecture for the Internet", Internet Draft 197 , November 1997, 198 ftp://ftp.ee.lbl.gov/papers/dsarch.pdf 200 [CBQ] S. Floyd and V. Jacobson, "Link-sharing and Resource Management 201 Models for Packet Networks", IEEE/ACM Transactions on Networking, 202 Vol. 3 no. 4, pp. 365-386, August 1995. 204 [IW] K. Poduri and K. Nichols, "Simulation Studies of Increased Initial 205 TCP Window Size", , August 1998. 207 [LCN] K. Nichols, "Improving Network Simulation with Feedback", Proceedings 208 of LCN '98, October, 1998. 210 5. Authors' Addresses 212 Van Jacobson 213 Cisco Systems, Inc 214 170 W. Tasman Drive 215 San Jose, CA 95134-1706 216 van@cisco.com 218 Kathleen Nichols 219 Cisco Systems, Inc 220 170 W. Tasman Drive 221 San Jose, CA 95134-1706 222 kmn@cisco.com 224 Kedarnath Poduri 225 Bay Networks, Inc. 226 4401 Great America Parkway 227 Santa Clara, CA 95052-8185 228 kpoduri@baynetworks.com 230 Appendix A: Example use of and experiences with the EF PHB 232 A.1 Virtual Leased Line Service 234 A VLL Service, also known as Premium service [2BIT], is quantified by a 235 peak bandwidth. 237 A.2 Experiences with its use in ESNET 239 A prototype of the VLL service has been deployed on DOE's ESNet 240 backbone. This uses weighted-round-robin queuing features of Cisco 75xx 241 series routers to implement the EF PHB. The early tests have been very 242 successful and work is in progress to make the service available on a 243 routine production basis (see ftp://ftp.ee.lbl.gov/talks/vj-doeqos.pdf 244 and ftp://ftp.ee.lbl.gov/talks/vj-i2qos-may98.pdf for details). 246 A.3 Simulation Results 248 A.3.1 Jitter variation 250 In section 2.2, we pointed out that a number of mechanisms might be used 251 to implement the EF PHB. The simplest is a priority queue (PQ) where 252 the arrival rate of the queue is strictly less than its service rate. 253 As jitter comes from the queuing delay along the path, a feature of this 254 implementation is that EF-marked microflows will see very little jitter 255 at their subscribed rate if all DS nodes along the path use this 256 implementation since packets spend little time in queues. The EF PHB 257 does not have an explicit jitter requirement, but it is clear from the 258 definition that the expected jitter in packets that use a service based 259 on the EF PHB will be less than for best-effort style packet delivery. 260 A PQ implementation for the EF PHB should give the smallest jitter, but 261 we used simulation to explore how other implementations, particularly 262 weighted round-robin (WRR), compare in jitter. PQ and WRR seemed to be 263 the best and worst cases, respectively, for jitter and we wanted to 264 supply some rough guidelines for implementers choosing to use WRR. 266 Our simulation model is implemented in ns-2 as described in [IW] and 267 [LCN]. We've made some further modifications to ns-2, using the CBQ 268 modules included with ns-2 as a basis to implement priority queuing and 269 WRR. Our topology has six hops with decreasing bandwidth in the 270 direction of a single 1.5 Mbps bottleneck link. Sources produce 271 EF-marked packets at an average bit rate equal to their subscribed 272 packet rate. Packets are produced with a variation of +/-10% from the 273 interpacket spacing at the subscribed packet rate. The individual 274 source rates were picked to add up to 30% of the bottleneck link or 450 275 Kbps. A mixture of other kinds of traffic, FTPs and HTTPs, is used to 276 fill the link. EF packet sources produce either all 160 byte packets or 277 all 1500 byte packets. Though we focus on the statistics of flows with 278 one size of packet, all of the experiments used a mixture of short 279 packet EF sources and long packet EF sources so the EF queues had a mix 280 of both packet lengths. 282 We used as the jitter definition the absolute value of the difference 283 between the arrival times of two adjacent packets minus their departure 284 times, |(aj-ai) - (dj-di)|. For the target flow of each experiment, we 285 record the median, 90th and 95th percentile values of jitter (in 286 milliseconds) in a table. The pdf version of this document contains 287 graphs of the jitter percentiles. 289 We explored the jitter behavior for WRR implementations of the EF PHB. 290 We wanted to see how different the jitter behavior is from that of PQ 291 and to examine the effects of different choices of WRR queue weight and 292 number of queues on jitter. To this end, we define the 293 service-to-arrival rate ratio as the WRR rate of an EF queue (or the 294 queue's minimum share of the output link) times the output link 295 bandwidth divided by the peak arrival rate of EF-marked packets at a 296 queue. If the WRR weight is chosen to exactly balance arrival and 297 departure rates, results will not be stable. Thus the minimum ratio of 298 service rate to arrival rate used here is 1.03 which, in our 299 simulations, means that the EF queue gets a weight of 31% of the output 300 links. In our WRR simulations, we kept the link full with other traffic 301 as described above, splitting the non-EF-marked traffic among the non-EF 302 queues. 304 Our first set of experiments uses the minimal service-to-arrival ratio 305 of 1.06 and we vary the number of individual microflows composing the EF 306 aggregate from 2 to 36. We compare these WRR implementations to a PQ 307 implementation with 24 flows. First, we examine a microflow at a 308 subscribed rate of 56 Kbps sending 1500 byte packets, then one at the 309 same rate but sending 160 byte packets. Table 1 shows the 50th, 90th 310 and 95th percentile jitter in milliseconds. Figure 1 plots the 1500 311 byte flows and figure 2 the 160 byte flows. Note that a packet-time for 312 a 1500 byte packet at 56 Kbps is 214 ms, for a 160 byte packet 23 ms. 313 Thus the jitter for the large packets rarely exceeds half a subscribed 314 rate packet-time, though most jitters for the small packets are at least 315 one subscribed rate packet-time. Keep in mind that the EF aggregate is 316 composed of mixtures of small and large packets in both cases, so the 317 short packets can still queue behind long packets in the EF queue. 318 Also, the service-to-arrival ratio used here is the minimum possible to 319 implement the EF PHB. PQ gives a very low jitter. 321 (see pdf form of document for all tables) 323 Next we look at the effects of increasing the service-to-arrival ratio 324 to see if it reduces jitter. This means that EF packets are expected to 325 remain enqueued for less time, though the amount of bandwidth available 326 for all other queues remains the same. In this set of experiments, the 327 number of flows composing the aggregate was fixed at eight and the total 328 number of queues at five (four non-EF queues). Table 2 shows the 329 results, first for a 1500 byte flow, then for a 160 byte flow. Figures 330 3 plots the 1500 byte results and figure 4 the 160 byte results. Table 331 2 gives the 95th percentile values of jitter for the same. Performance 332 gains leveled off at service-to-arrival ratios of 1.5. Note that the 333 higher service-to-arrival ratios still do not give the same performance 334 as PQ, but now 90% of packets experience less than a subscribed 335 packet-time of jitter, even for the small packets. We believe that 336 implementers should use service-to-arrival ratios of at least 1.5 and 337 further study may be desired to determine the efficacy of higher ratios. 339 Increasing the number of queues at the output interfaces can lead to 340 more variability in the service time for EF packets so we carried out an 341 experiment varying the number of queues at each output port. We fixed 342 the number of flows in the aggregate to eight and used a 1.03 343 service-to-arrival ratio. Results are shown in figure 5 and table 3. 344 Figure 5 includes PQ with 8 flows as a baseline. 346 It appears that most jitter for WRR is low and can be reduced by a 347 proper choice of the EF queue's WRR share of the output link with 348 respect to its subscribed rate. As noted, WRR is probably a "worst 349 case" while PQ is the best case. Other possibilities include WFQ or CBQ 350 with a fixed rate limit for the EF queue, but giving it priority over 351 other queues. We expect the latter to have performance nearly identical 352 with PQ, though future simulations can verify this. We have not yet 353 systematically explored effects of hop count, EF allocations of more or 354 less than 30% of the link bandwidth, or more complex topologies. Note 355 that this information is simply to guide implementers. 357 A.3.2 VLL service 359 We used simulation to see how well a VLL service built from the EF PHB 360 behaved, that is, does it look like a "leased line" at the subscribed 361 rate. In the simulations of the last section, none of the EF packets 362 were dropped in the network and the target rate was always achieved for 363 those CBR sources. However, we wanted to see if VLL really looks like a 364 "wire" to a TCP using it. So we simulated long-lived FTPs using a VLL 365 service. Table 4 gives the percentage of each link allocated to EF 366 traffic (bandwidths are lower on the links with fewer EF microflows), 367 the subscribed VLL rate, the average rate for the same type of 368 sender-receiver pair connected by a full duplex dedicated link at the 369 subscribed rate and the average of the VLL flows for each simulation 370 (all sender-receiver pairs had the same value). Losses only occur when 371 the input shaping buffer overflows but not in the network. The target 372 rate is not achieved due to the well-known TCP behavior.