idnits 2.17.1 draft-ietf-diffserv-phb-ef-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-25) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 8) being 61 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 9 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 63 instances of too long lines in the document, the longest one being 6 characters in excess of 72. ** There are 38 instances of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 304 has weird spacing: '...er with numbe...' == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'CBQ' is defined on line 187, but no explicit reference was found in the text -- Possible downref: Non-RFC (?) normative reference: ref. 'HEADER' -- Possible downref: Non-RFC (?) normative reference: ref. 'ARCH' -- Possible downref: Non-RFC (?) normative reference: ref. '2BIT' -- Possible downref: Non-RFC (?) normative reference: ref. 'CBQ' -- Possible downref: Non-RFC (?) normative reference: ref. 'IW' -- Possible downref: Non-RFC (?) normative reference: ref. 'LCN' Summary: 10 errors (**), 0 flaws (~~), 6 warnings (==), 8 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Engineering Task Force Van Jacobson 2 Differentiated Services Working Group LBNL 3 Internet Draft Kathleen Nichols 4 Expires February, 1999 Kedarnath Poduri 5 Bay Networks 6 August, 1998 8 An Expedited Forwarding PHB 9 11 Status of this Memo 13 This document is a submission to the IETF Differentiated Services (DiffServ) 14 Working Group. Comments are solicited and should be addressed to the working 15 group mailing list or to the editor. 17 This document is an Internet-Draft. Internet Drafts are working documents 18 of the Internet Engineering Task Force (IETF), its areas, and its working 19 Groups. Note that other groups may also distribute working documents as 20 Internet Drafts. 22 Internet-Drafts draft documents are valid for a maximum of six months and 23 may be updated, replaced, or obsolete by other documents at any time. It is 24 inappropriate to use Internet-Drafts as reference material or to cite them 25 other than as "work in progress." 27 To learn the current status of any Internet-Draft, please check the 28 "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow 29 Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au 30 (Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West 31 Coast). 33 Distribution of this memo is unlimited. 35 Abstract 37 The definition of PHBs (per-hop forwarding behaviors) is a critical part of 38 the work of the Diffserv Working Group. This document describes a PHB 39 called Expedited Forwarding. We show the generality of this PHB by noting 40 that it can be produced by more than one mechanism and give an example of 41 its use to produce at least one service, a Virtual Leased Line. A 42 recommended codepoint for this PHB is given. 44 A pdf version of this document is available at 45 ftp://ftp.ee.lbl.gov/papers/ef_phb.pdf 46 1. Introduction 48 Network nodes that implement the differentiated services enhancements to 49 IP use a codepoint in the IP header to select a per-hop behavior (PHB) 50 as the specific forwarding treatment for that packet [HEADER, ARCH]. 51 This draft describes a particular PHB called expedited forwarding (EF). 52 The EF PHB can be used to build a low loss, low latency, low jitter, 53 assured bandwidth, end-to-end service through DS domains. Such a 54 service appears to the endpoints like a point-to-point connection or a 55 "virtual leased line". This service has also been described as Premium 56 service [2BIT]. 58 Loss, latency and jitter are all due to the queues traffic experiences 59 while transiting the network. Therefore providing low loss, latency and 60 jitter for some traffic aggregate means ensuring that the aggregate sees 61 no (or very small) queues. Queues arise when (short term) traffic 62 arrival rate exceeds departure rate at some node. Thus a service that 63 ensures no queues for some aggregate is equivalent to bounding rates 64 such that, at every transit node, the aggregate's max arrival rate is 65 less than that aggregate's min departure rate. 67 Creating such a service has two parts: 69 1) configuring nodes so that the aggregate has a well-defined 70 minimum departure rate. (`Well-defined' means independent 71 of the dynamic state of the node. In particular, independent 72 of the intensity of other traffic at the node.) 74 2) conditioning the aggregate (via policing and shaping) so that 75 it's arrival rate at any node is always less than that node's 76 configured minimum departure rate. 78 The EF PHB provides the first part of the service. The network 79 boundary traffic conditioners described in [ARCH] provide the 80 second part. 82 The next sections describe the EF PHB in detail and give examples of how 83 it might be implemented. The keywords "MUST", "MUST NOT", "REQUIRED", 84 "SHOULD", "SHOULD NOT", and "MAY" that appear in this document are to be 85 interpreted as described in [Bradner97]. 87 2. Description of EF per-hop behavior 89 2.1 Description 91 The EF PHB is defined as a forwarding treatment for a particular 92 diffserv aggregate where the departure rate of the aggregate's packets 93 from any diffserv node must equal or exceed a configurable rate. The EF 94 traffic should receive this rate independent of the intensity of any 95 other traffic attempting to transit the node. It should average at 96 least the configured rate when measured over any time interval equal to 97 or longer than a packet time at the configured rate. (Behavior at time 98 scales shorter than a packet time at the configured rate is deliberately 99 not specified.) The configured minimum rate must be settable by a 100 network administrator (using whatever mechanism the node supports for 101 non-volatile configuration). 103 The Appendix describes how this PHB can be used to construct end-to-end 104 services. 106 2.2 Example Mechanisms to Implement the EF PHB 108 Several types of queue scheduling mechanisms may be employed to deliver the 109 forwarding behavior described in section 2.1 and thus implement the EF PHB. 110 A simple priority queue will give the appropriate behavior as long as there 111 is no higher priority queue the could preempt the EF for more than a 112 packet time at the configured rate. (This could be accomplished by 113 having a rate policer such as a token bucket associated with each priority 114 queue to bound how much the queue can starve other traffic.) 116 It's also possible to use a single queue in a group of queues 117 serviced by a weighted round robin scheduler where the share of 118 the output bandwidth assigned to the EF queue is equal to the 119 configured rate. This could be implemented, for example, using 120 one PHB of a Class Selector Compliant set of PHBs [HEADER]. 122 Another possible implementation is a CBQ scheduler that gives the 123 EF queue priority up to the configured rate. 125 All of these mechanisms give the basic properties required for the 126 EF PHB though different choices result in differences in auxiliary 127 behavior such as jitter seen by individual microflows. See Appendix 128 A.3 for simulations that quantify some of these differences. 130 2.3 Recommended codepoint for this PHB 132 Codepoint 101100 is recommended for the EF PHB. 134 2.4 Mutability 136 Packets marked for EF PHB may be remarked at a DS domain boundary to 137 other codepoints that satisfy the EF PHB only. Packets marked for EF 138 PHBs SHOULD NOT be demoted or promoted to another PHB by a DS domain. 140 2.5 Tunneling 142 When EF packets are tunneled, the tunneling packets must be marked as EF. 144 2.6 Interaction with other PHBs 146 Other PHBs and PHB groups may be deployed in the same DS node or domain 147 with the EF PHB as long as the requirement of section 2.1 is met. 149 3. Security Considerations 151 To protect itself against denial of service attacks, the edge of a DS 152 domain MUST strictly police all EF marked packets to a rate negotiated 153 with the adjacent upstream domain. (This rate must be <= the EF PHB 154 configured rate.) Packets in excess of the negotiated rate MUST be 155 dropped. If two adjacent domains have not negotiated an EF rate, the 156 downstream domain MUST use 0 as the rate (i.e., drop all EF marked packets). 158 Since the end-to-end premium service constructed from the EF PHB requires 159 that the upstream domain police and shape EF marked traffic to meet the 160 rate negotiated with the downstream domain, the downstream domain's 161 policer should never have to drop packets. Thus these drops should 162 be noted (e.g., via SNMP traps) as possible security violations or 163 serious misconfiguration. Similarly, since the aggregate EF traffic 164 rate is constrained at every interior node, the EF queue should never 165 overflow so if it does the drops should be noted as possible attacks 166 or serious misconfiguration. 168 4. References 170 [Bradner97] S. Bradner, "Key words for use in RFCs to Indicate 171 Requirement Levels", Internet RFC 2119, March 1997. 173 [HEADER] K. Nichols, S. Blake, F. Baker, and D. Black, "Definition of 174 the Differentiated Services Field (DS Field) in the IPv4 and 175 IPv6 Headers", , August 1998. 177 [ARCH] D. Black, S. Blake, M. Carlson, E. Davies, Z. Wang, and 178 W. Weiss, "An Architecture for Differentiated Services", 179 Internet Draft , 180 August 1998. 182 [2BIT] K. Nichols, V. Jacobson, and L. Zhang, "A Two-bit 183 Differentiated Services Architecture for the Internet", 184 Internet Draft , 185 November 1997, ftp://ftp.ee.lbl.gov/papers/dsarch.pdf. 187 [CBQ] S. Floyd and V. Jacobson, "Link-sharing and Resource 188 Management Models for Packet Networks", IEEE/ACM 189 Transactions on Networking, Vol. 3 no. 4, pp. 365-386, 190 August 1995. 192 [IW] K. Poduri and K. Nichols, "Simulation Studies of Increased 193 Initial TCP Window Size", , 194 August, 1998. 196 [LCN] K. Nichols, "Improving Network Simulation with Feedback", to 197 appear in proceedings of LCN '98, October, 1998 199 5. Authors' Addresses 201 Van Jacobson 202 Lawrence Berkeley National Laboratory 203 M/S 50B-2239 204 One Cyclotron Road 205 Berkeley, CA 94720 206 van@ee.lbl.gov 208 Kathleen Nichols 209 Bay Networks, Inc. 210 4401 Great America Parkway 211 Santa Clara, CA 95052-8185 212 knichols@baynetworks.com 213 +1 408-495-3252 215 Kedarnath Poduri 216 Bay Networks, Inc. 217 Bay Networks, Inc. 218 4401 Great America Parkway 219 Santa Clara, CA 95052-8185 220 kpoduri@baynetworks.com 221 Appendix. Example use of and experiences with the EF PHB 223 A.1 Virtual Leased Line Service 225 A VLL Service, also known as Premium service [2BIT], is quantified by a 226 peak bandwidth. 228 A.2 Experiences with its use in ESNET 230 A prototype of the VLL service has been deployed on DOE's ESNet 231 backbone. This uses weighted-round-robin queuing features of cisco 7xxx 232 series routers to implement the EF PHB. The early tests have been very 233 successful (details are available in ftp://ftp.ee.lbl.gov/talks/vj-doeqos.pdf 234 and ftp://ftp.ee.lbl.gov/talks/vj-i2qos-may98.pdf) and work is in progress 235 to make the service available on a routine production basis. 237 A.3 Simulation Results 239 In section 2.2, we pointed out that a number of mechanisms may be used to 240 implement the EF PHB. The simplest is a priority queue where the arrival 241 rate of the queue is strictly less than its departure rate. As jitter comes 242 from the queuing delay along the path, a feature of this implementation is 243 that EF-marked microflows will see very little jitter at their subscribed 244 rate if all DS nodes along the path use this implementation since packets 245 spend little time in queues. This low-jitter behavior is not a requirement 246 of the EF PHB, but we want to explore how other implementations, in this 247 case WRR, compare in jitter. We've compared PQ and WRR because these seemed 248 to be the best and worst cases, respectively, for jitter. 250 Our basic simulation model is implemented in ns-2 as described in [IW] and 251 [LCN]. We've made some further modifications to ns-2, using the CBQ modules 252 included with ns-2 as a basis to implement priority queuing and WRR. 254 We experimented with a six-hop topology with decreasing bandwidth in the 255 direction of a single 1.5 Mbps bottleneck link. For our EF-marked packets, 256 we set up sources to produce packets at a constant bit rate with a 257 variation of +/-10% of the subscribed packet rate. The individual source 258 rates were picked to add up to 30% of the bottleneck link or 450 Kbps. A 259 mixture of other kinds of traffic, FTPs and HTTPs, is used to fill the 260 link. We report jitter as the added delay normalized by the time to send a 261 packet at the subscribed peak rate. The pdf version of this document 262 contains graphs of percentile vs jitter and we include text tables that 263 report the 95th percentile from each of the scenarios. We used different 264 packet sizes for the EF-marked packets in our simulations, but always used 265 the same packet size for all EF-marked packets in any particular 266 simulation. We report percentile of packets seeing less than a particular 267 normalized packet size in jitter. 269 We will consider the implementation of the EF PHB with a priority queue 270 (PQ) as a kind of baseline or "ideal" case. To summarize the results we've 271 seen for PQ jitter, jitter is most strongly dependent on packet size. For 272 1500 byte packets, all jitter is less than 0.5 packet times. For 160 byte 273 packets, 95% of packet jitter is less than 3.5 packet times with most packets 274 having less than one packet's worth of jitter. The PQ results will be shown 275 with the WRR results below. 277 Next we explored the jitter behavior for WRR implementations of the EF PHB. 278 What we wanted to explore was how different the jitter behavior is from 279 that of PQ implementations. Major features that can affect jitter are packet 280 size, number of queues for the WRR scheduler, and the amount by which the 281 guaranteed minimum service rate of the EF queue exceeds the peak arrival 282 rate to the EF queue. We have not yet systematically explored effects of 283 hop count, EF allocations of more or less than 30% of the link bandwidth, 284 or more complex topologies. However, this information is simply to guide 285 those who are interested in a low jitter implementation and is not required 286 for implementing the EF PHB with WRR. 288 In our WRR simulations, we kept the link full with other traffic as 289 described above, splitting the non-EF-marked traffic among the non-EF 290 queues. If the WRR weight is chosen to exactly balance arrival and 291 departure rates, our results will not be stable except for the simplest 292 cases, so we always overallocate by a minimum of 1% of the output link 293 bandwidth or, in this case, 3% of the peak arrival rate of EF-marked 294 packets. We recommend at least this overallocation to implementors. In 295 figure 1 and table 1, we show results from varying the number of 296 individual microflows composing the EF aggregate of 450 Kbps. In this case 297 all EF packets are 1500 bytes and the EF queue gets a weight of 31% of the 298 output links. The leftmost curve shows the results for a PQ with 24 flows. 299 Note that the maximum jitter of 3.2 packets occurs only for 36 flows, but 300 the 95th percentile of all scenarios is less than 1 packet of jitter. 301 Figure 2 and table 1 shows the results when a packet size of 160 bytes 302 is used. 304 Table 1: Variation in jitter with number of EF flows 306 -------------------------------------------------------------- 307 Num of EF flows Jitter Jitter 308 (95th percentile (95th percentile 309 1500 Byte Packets) 160 Byte Packets) 310 -------------------------------------------------------------- 312 PQ (24) 0.07 0.5 314 2 0.6 6.6 316 4 0.4 3.9 318 8 0.3 1.8 320 24 0.6 2.1 322 -------------------------------------------------------------- 324 Next we look at the effects of overallocating the link share, that is 325 giving a minimum service rate that exceeds the peak arrival rate by various 326 amounts. (Of course, with WRR, that bandwidth is still available for other 327 packets.) We fixed the number of flows at eight and the total number of 328 queues at five (four non-EF queues). In figure 3 we report results for 1500 329 byte EF packets and in figure 4 we show 160 byte packets. Table 2 gives the 330 95th percentile values of jitter for the same. Overallocation by up to 100% 331 still does not give the same performance as PQ, but note that most packets 332 experience small jitter. In fact, overallocation does not appear to have much 333 improvement associated with it. 335 Table 2: Variation in Jitter with Overallocation of BW to EF queues. 337 -------------------------------------------------------------- 338 % of Over- Jitter Jitter 339 Allocation (95th percentile (95th percentile 340 1500 Byte Packets) 160 Byte Packets) 341 -------------------------------------------------------------- 343 PQ 0.05 0.5 345 3 0.3 2.2 347 30 0.2 1.4 349 50 0.15 1.2 351 70 0.15 1.2 353 100 0.15 1.2 355 --------------------------------------------------------------- 356 We know that increasing the number of queues at the output interfaces can 357 lead to more variability in the service time for EF packets. We set the 358 number of flows to eight and used a 31% weight for the 30% EF allocation 359 and varied the number of queues at each output interface. Results are shown 360 in figure 5 and table 3. Note that most packets experience little jitter. 361 PQ with 8 flows is included as a baseline. 363 Table 3: Variation in Jitter with Number of Queues at Output Interface 365 --------------------------------------- 366 Num of Queues Jitter 367 (95th percentile 368 1500 Byte Packets) 369 --------------------------------------- 371 PQ 0.05 373 2 0.2 375 4 0.3 377 6 0.3 379 8 0.35 381 --------------------------------------- 383 We intend to perform further studies and vary other parameters, but at 384 present it appears that most packet jitter for WRR is low, but by 385 overallocating the EF queue's WRR share of the output link with respect to its 386 subscribed rate packet jitter can be reduced if desired. 388 As noted, WRR is probably a "worst case" while PQ is the best case. 389 Other possibilities include WFQ or CBQ with a fixed rate limit for the EF 390 queue, but giving it priority over other queues. We expect the latter to 391 have performance nearly identical with PQ, though future simulations can 392 verify this.