idnits 2.17.1 draft-iab-qos-02.txt: ** The Abstract section seems to be numbered Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document is more than 15 pages and seems to lack a Table of Contents. == The page length should not exceed 58 lines per page, but there was 18 longer pages, the longest (page 2) being 59 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 584 has weird spacing: '...e field as th...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (August 2000) is 8648 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 6 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Architecture Board G.Huston 2 Internet Draft Telstra 3 Document: draft-iab-qos-02.txt August 2000 4 Category: Informational 6 Next Steps for the IP QoS Architecture 8 Status of this Memo 10 This document is an Internet-Draft and is in full conformance with 11 all provisions of Section 10 of RFC2026 [1]. 13 Internet-Drafts are working documents of the Internet Engineering 14 Task Force (IETF), its areas, and its working groups. Note that 15 other groups may also distribute working documents as Internet- 16 Drafts. Internet-Drafts are draft documents valid for a maximum of 17 six months and may be updated, replaced, or obsoleted by other 18 documents at any time. It is inappropriate to use Internet- Drafts 19 as reference material or to cite them other than as "work in 20 progress." 22 The list of current Internet-Drafts can be accessed at 23 http://www.ietf.org/ietf/1id-abstracts.txt 25 The list of Internet-Draft Shadow Directories can be accessed at 26 http://www.ietf.org/shadow.html. 28 1. Abstract 30 While there has been significant progress in the definition of QoS 31 architectures for internet networks, there are a number of aspects 32 of QoS that appear to need further elaboration as they relate to 33 translating a set of tools into a coherent platform for end-to-end 34 service delivery. This document highlights the outstanding 35 architectural issues relating to the deployment and use of QoS 36 mechanisms within internet networks, noting those areas where 37 further standards work may assist with the deployment of QoS 38 internets. 40 This document is the outcome of a collaborative exercise on the part 41 of the Internet Architecture Board. 43 2. Introduction 45 The default service offering associated with the Internet is 46 characterized as a best-effort variable service response. Within 47 this service profile the network makes no attempt to actively 48 differentiate its service response between the traffic streams 49 generated by concurrent users of the network. As the load generated 50 by the active traffic flows within the network varies, the network's 51 best effort service response will also vary. 53 Huston Informational - Expires December 2000 1 54 The objective of various Internet Quality of Service (QoS) efforts 55 is to augment this base service with a number of selectable service 56 responses. These service responses may be distinguished from the 57 best-effort service by some form of superior service level, or they 58 may be distinguished by providing a predictable service response 59 which is unaffected by external conditions such as the number of 60 concurrent traffic flows, or their generated traffic load. 62 Any network service response is an outcome of the resources 63 available to service a load, and the level of the load itself. To 64 offer such distinguished services there is not only a requirement to 65 provide a differentiated service response within the network, there 66 is also a requirement to control the service-qualified load admitted 67 into the network, so that the resources allocated by the network to 68 support a particular service response are capable of providing that 69 response for the imposed load. This combination of admission control 70 agents and service management elements can be summarized as "rules 71 plus behaviors". To use the terminology of the Differentiated 72 Service architecture [4,] this admission control function is 73 undertaken by a traffic conditioner (an entity which performs 74 traffic conditioning functions and which may contain meters, 75 markers, droppers, and shapers), where the actions of the 76 conditioner are governed by explicit or implicit admission control 77 agents. 79 As a general observation of QoS architectures, the service load 80 control aspect of QoS is perhaps the most troubling component of the 81 architecture. While there are a wide array of well understood 82 service response mechanisms that are available to IP networks, 83 matching a set of such mechanisms within a controlled environment to 84 respond to a set of service loads to achieve a completely consistent 85 service response remains an area of weakness within existing IP QoS 86 architectures. The control elements span a number of generic 87 requirements, including end-to-end application signaling, end-to- 88 network service signaling and resource management signaling to allow 89 policy-based control of network resources. This control may also 90 span a particular scope, and use 'edge to edge' signaling, intended 91 to support particular service responses within a defined network 92 scope. 94 One way of implementing this control of imposed load to match the 95 level of available resources is through an application-driven 96 process of service level negotiation (also known as application 97 signaled QoS). Here, the application first signals its service 98 requirements to the network, and the network responds to this 99 request. The application will proceed if the network has indicated 100 that it is able to carry the additional load at the requested 101 service level. If the network indicates that it cannot accommodate 102 the service requirements the application may proceed in any case, on 103 the basis that the network will service the application's data on a 104 best effort basis. This negotiation between the application and the 106 Huston Informational - Expires December 2000 2 107 network can take the form of explicit negotiation and commitment, 108 where there is a single negotiation phase, followed by a commitment 109 to the service level on the part of the network. This application- 110 signaled approach can be used within the Integrated Services 111 architecture, where the application frames its service request 112 within the resource reservation protocol (RSVP), and then passes 113 this request into the network. The network can either respond 114 positively in terms of its agreement to commit to this service 115 profile, or it can reject the request. If the network commits to the 116 request with a resource reservation, the application can then pass 117 traffic into the network with the expectation that as long as the 118 traffic remains within the traffic load profile that was originally 119 associated with the request, the network will meet the requested 120 service levels. There is no requirement for the application to 121 periodically reconfirm the service reservation itself, as the 122 interaction between RSVP and the network constantly refreshes the 123 reservation while it remains active. The reservation remains in 124 force until the application explicitly requests termination of the 125 reservation, or the network signals to the application that it is 126 unable to continue with a service commitment to the reservation [3]. 127 There are variations to this model, including an aggregation model 128 where a proxy agent can fold a number of application-signaled 129 reservations into a common aggregate reservation along a common sub- 130 path, and a matching deaggregator can reestablish the collection of 131 individual resource reservations upon leaving the aggregate region 132 [5]. The essential feature of this Integrated Services model is the 133 "all or nothing" nature of the model. Either the network commits to 134 the reservation, in which case the requestor does not have to 135 subsequently monitor the network's level of response to the service, 136 or the network indicates that it cannot meet the resource 137 reservation. 139 An alternative approach to load control is to decouple the network 140 load control function from the application. This is the basis of the 141 Differentiated Services architecture. Here, a network implements a 142 load control function as part of the function of admission of 143 traffic into the network, admitting no more traffic within each 144 service category as there are assumed to be resources in the network 145 to deliver the intended service response. Necessarily there is some 146 element of imprecision in this function given that traffic may take 147 an arbitrary path through the network. In terms of the interaction 148 between the network and the application, this takes the form of a 149 service request without prior negotiation, where the application 150 requests a particular service response by simply marking each packet 151 with a code to indicate the desired service. Architecturally, this 152 approach decouples the end systems and the network, allowing a 153 network to implement an active admission function in order to 154 moderate the workload that is placed upon the network's resources 155 without specific reference to individual resource requests from end 156 systems. While this decoupling of control allows a network's 157 operator greater ability to manage its resources and a greater 158 ability to ensure the integrity of its services, there is a greater 159 potential level of imprecision in attempting to match applications' 160 service requirements to the network's service capabilities. 162 Huston Informational - Expires December 2000 3 163 3. State and Stateless QoS 165 These two approaches to load control can be characterized as state- 166 based and stateless approaches respectively. 168 The architecture of the Integrated Services model equates the 169 cumulative sum of honored service requests to the current reserved 170 resource levels of the network. In order for a resource reservation 171 to be honored by the network, the network must maintain some form of 172 remembered state to describe the resources that have been reserved, 173 and the network path over which the reserved service will operate. 174 This is to ensure integrity of the reservation. In addition, each 175 active network element within the network path must maintain a local 176 state that allows incoming IP packets to be correctly classified 177 into a reservation class. This classification allows the packet to 178 be placed into a packet flow context that is associated with an 179 appropriate service response consistent with the original end-to-end 180 service reservation. This local state also extends to the function 181 of metering packets for conformance on a flow-by-flow basis, and the 182 additional overheads associated with maintenance of the state of 183 each of these meters. 185 In the second approach, that of a Differentiated Services model, the 186 packet is marked with a code to trigger the appropriate service 187 response from the network elements that handles the packet, so that 188 there is no strict requirement to install a per-reservation state on 189 these network elements. Also, the end application or the service 190 requestor is not required to provide the network with advance notice 191 relating to the destination of the traffic, nor any indication of 192 the intended traffic profile or the associated service profile. In 193 the absence of such information any form of per-application or per- 194 path resource reservation is not feasible. In this model there is no 195 maintained per-flow state within the network. 197 The state-based Integrated Services architectural model admits the 198 potential to support greater level of accuracy, and a finer level of 199 granularity on the part of the network to respond to service 200 requests. Each individual application's service request can be used 201 to generate a reservation state within the network that is intended 202 to prevent the resources associated with the reservation to be 203 reassigned or otherwise preempted to service other reservations or 204 to service best effort traffic loads. The state-based model is 205 intended to be exclusionary, where other traffic is displaced in 206 order to meet the reservation's service targets. 208 As noted in RFC2208 [2], there are several areas of concern about 209 the deployment of this form of service architecture. With regard to 210 concerns of per-flow service scalability, the resource requirements 211 (computational processing and memory consumption) for running per- 212 flow resource reservations on routers increase in direct proportion 213 to the number of separate reservations that need to be accommodated. 214 By the same token, router forwarding performance may be impacted 216 Huston Informational - Expires December 2000 4 217 adversely by the packet-classification and scheduling mechanisms 218 intended to provide differentiated services for these resource- 219 reserved flows. This service architecture also poses some challenges 220 to the queuing mechanisms, where there is the requirement to 221 allocate absolute levels of egress bandwidth to individual flows, 222 while still supporting an unmanaged low priority best effort traffic 223 class. 225 The stateless approach to service management is more approximate in 226 the nature of its outcomes. Here there is no explicit negotiation 227 between the application's signaling of the service request and the 228 network's capability to deliver a particular service response. If 229 the network is incapable of meeting the service request, then the 230 request simply will not be honored. In such a situation there is no 231 requirement for the network to inform the application that the 232 request cannot be honored, and it is left to the application to 233 determine if the service has not been delivered. The major attribute 234 of this approach is that it can possess excellent scaling properties 235 from the perspective of the network. If the network is capable of 236 supporting a limited number of discrete service responses, and the 237 routers uses per-packet marking to trigger the service response, 238 then the processor and memory requirements in each router do not 239 increase in proportion to the level of traffic passed through the 240 router. Of course this approach does introduce some degree of 241 compromise in that the service response is more approximate as seen 242 by the end client, and scaling the number of clients and 243 applications in such an environment may not necessarily result in a 244 highly accurate service response to every client's application. 246 It is not intended to describe these service architectures in 247 further detail within this document. The reader is referred to 248 RFC1633 [3] for an overview of the Integrated Services Architecture 249 (IntServ) and RFC2475 [4] for an overview of the Differentiated 250 Services architecture (DiffServ). 252 These two approaches are the endpoints of what can be seen as a 253 continuum of control models, where the fine-grained precision of the 254 per application invocation reservation model can be aggregated into 255 larger, more general and potentially more approximate aggregate 256 reservation states, and the end-to-end element-by-element 257 reservation control can be progressively approximated by treating a 258 collection of subnetworks or an entire transit network as an 259 aggregate service element. There are a number of work in progress 260 efforts which are directed towards these aggregated control models, 261 including aggregation of RSVP [5], the RSVP DCLASS Object [6] to 262 allow Differentiated Services Code Points (DSCPs) to be carried in 263 RSVP message objects, and operation of Integrated Services over 264 Differentiated Services networks [7]. 266 4. Next Steps for QoS Architectures 268 Both the Integrated Services architecture and the Differentiated 269 Services architecture have some critical elements in terms of their 270 current definition which appear to be acting as deterrents to 272 Huston Informational - Expires December 2000 5 273 widespread deployment. Some of these issues will probably be 274 addressed within the efforts to introduce aggregated control and 275 response models into these QoS architectures, while others may 276 require further refinement through standards-related activities. 278 4.1 QoS-Enabled Applications 280 One of the basic areas of uncertainty with QoS architectures is 281 whether QoS is a per-application service, whether QoS is a 282 transport-layer option, or both. Per-application services have 283 obvious implications of extending the QoS architecture into some 284 form of Application Protocol Interface (API), so that applications 285 could negotiate a QoS response from the network and alter their 286 behavior according to the outcome of the response. Examples of this 287 approach include GQOS [8], and RAPI [9]. As a transport layer 288 option, it could be envisaged that any application could have its 289 traffic carried by some form of QoS-enabled network services by 290 changing the host configuration, or by changing the configuration at 291 some other network control point, without making any explicit 292 changes to the application itself. The strength of the transport 293 layer approach is that there is no requirement to substantially 294 alter application behavior, as the application is itself unaware of 295 the administratively assigned QoS. The weakness of this approach is 296 that the application is unable to communicate what may be useful 297 information to the network or to the policy systems that are 298 managing the network's service responses. In the absence of such 299 information the network may provide a service response that is far 300 superior than the application's true requirements, or far inferior 301 than what is required for the application to function correctly. An 302 additional weakness of a transport level approach refers to those 303 class of applications that can adapt their traffic profile to meet 304 the available resources within the network. As a transport level 305 mechanism, such network availability information as may be available 306 to the transport level is not passed back to the application. 308 In the case of the Integrated Services architecture, this transport 309 layer approach does not appear to be an available option, as the 310 application does require some alteration to function correctly in 311 this environment. The application must be able to provide to the 312 service reservation module a profile of its anticipated traffic, or 313 in other words the application must be able to predict its traffic 314 load. In addition, the application must be able to share the 315 reservation state with the network, so that if the network state 316 fails, the application can be informed of the failure. The more 317 general observation is that a network can only formulate an accurate 318 response to an application's requirements if the application is 319 willing to offer precise statement of its traffic profile, and is 320 willing to be policed in order to have its traffic fit within this 321 profile. 323 In the case of the Differentiated Services architecture there is no 324 explicit provision for the application to communicate with the 325 network regarding service levels. This does allow the use of a 327 Huston Informational - Expires December 2000 6 328 transport level option within the end system that does not require 329 explicit alteration of the application to mark its generated traffic 330 with one of the available Differentiated Services service profiles. 331 However, whether the application is aware of such service profiles 332 or not, there is no level of service assurance to the application in 333 such a model. If the Differentiated Services boundary traffic 334 conditioners enter a load shedding state, the application is not 335 signaled of this condition, and is not explicitly aware that the 336 requested service response is not being provided by the network. If 337 the network itself changes state and is unable to meet the 338 cumulative traffic loads admitted by the ingress traffic 339 conditioners, neither the ingress traffic conditioners, nor the 340 client applications, are informed of this failure to maintain the 341 associated service quality. While there is no explicit need to alter 342 application behavior in this architecture, as the basic DiffServ 343 mechanism is one that is managed within the network itself, the 344 consequence is that an application may not be aware whether a 345 particular service state is being delivered to the application. 347 There is potential in using an explicit signaling model, such as 348 used by IntServ, but carrying a signal which allows the network to 349 manage the application's traffic within an aggregated service class 350 [6]. Here the application does not pass a complete picture of its 351 intended service profile to the network, but instead is providing 352 some level of additional information to the network to assist in 353 managing its resources, both in terms of the generic service class 354 that the network can associate with the application's traffic, and 355 the intended path of the traffic through the network. 357 An additional factor for QoS enabled applications is that of 358 receiver capability negotiation. There is no value in the sender 359 establishing a QoS-enabled path across a network to the receiver if 360 the receiver is incapable of absorbing the consequent data flow. 361 This implies that QoS enabled applications also require some form of 362 end-to-end capability negotiation, possibly through a generic 363 protocol to allow the sender to match its QoS requirements to the 364 minimum of the flow resources that can be provided by the network 365 and the flow resources that can be processed by the receiver. In the 366 case of the Integrated services architecture the application end-to- 367 end interaction can be integrated into the RSVP negotiation. In the 368 case of the Differentiated Services architecture there is no clear 369 path of integrating such receiver control into the signaling model 370 of the architecture as it stands. 372 If high quality services are to be provided, where `high quality' is 373 implied as being `high precision with a fine level of granularity', 374 then the implication is that all parts of the network that may be 375 involved with servicing the request either have to be over- 376 provisioned such that no load state can compromise the service 377 quality, or the network element must undertake explicit allocation 378 of resources to each flow that is associated with each service 379 request. 381 For end-to-end service delivery it does appear that QoS 383 Huston Informational - Expires December 2000 7 384 architectures will need to extend to the level of the application 385 requesting the service profile. It appears that further refinement 386 of the QoS architecture is required to integrate DiffServ network 387 services into an end-to-end service delivery model, as noted in [7]. 389 4.2 The Service Environment 391 The outcome of the considerations of these two approaches to QoS 392 architecture within the network is that there appears to be no 393 single comprehensive service environment that possesses both service 394 accuracy and scaling properties. 396 The maintained reservation state of the Integrated Services 397 architecture and the end-to-end signaling function of RSVP are part 398 of a service management architecture, but it is not cost effective, 399 or even feasible, to operate a per-application reservation and 400 classification state across the high speed core of a network [2]. 402 While the aggregated behavior state of the Differentiated Services 403 architecture does offer excellent scaling properties, the lack of 404 end-to-end signaling facilities makes such an approach one that 405 cannot operate in isolation within any environment. The 406 Differentiated Services architecture can be characterized as a 407 boundary-centric operational model. With this boundary-centric 408 architecture, the signaling of resource availability from the 409 interior of the network to the boundary traffic conditioners is not 410 defined, nor is the signaling from the traffic conditioners to the 411 application that is resident on the end system. This has been noted 412 as an additional work item in the IntServ operations over DiffServ 413 work, concerning `definition of mechanisms to efficiently and 414 dynamically provision resources in a DiffServ network region. This 415 might include protocols by which an _oracle_ (_) conveys information 416 about resource availability within a DiffServ region to border 417 routers._ [7] 419 What appears to be required within the Differentiated Services 420 service model is both resource availability signaling from the core 421 of the network to the DiffServ boundary and some form of signaling 422 from the boundary to the client application. 424 4.3 QoS Discovery 426 There is no robust mechanism for network path discovery with 427 specific service performance attributes. The assumption within both 428 IntServ and DiffServ architectures is that the best effort routing 429 path is used, where the path is either capable of sustaining the 430 service load, or not. 432 Assuming that the deployment of service differentiating 433 infrastructure will be piecemeal, even if only in the initial stages 434 of a QoS rollout, such an assumption may be unwarranted. If this is 435 the case, then how can a host application determine if there is a 436 distinguished service path to the destination? No existing 438 Huston Informational - Expires December 2000 8 439 mechanisms exist within either of these architectures to query the 440 network for the potential to support a specific service profile. 441 Such a query would need to examine a number of candidate paths, 442 rather than simply examining the lowest metric routing path, so that 443 this discovery function is likely to be associated with some form of 444 QoS routing functionality. 446 From this perspective, there is still further refinement that may be 447 required in the model of service discovery and the associated task 448 of resource reservation. 450 4.4 QoS Routing and Resource Management 452 To date QoS routing has been developed at some distance from the 453 task of development of QoS architectures. The implicit assumption 454 within the current QoS architectural models is that the routing best 455 effort path will be used for both best effort traffic and 456 distinguished service traffic. 458 There is no explicit architectural option to allow the network 459 service path to be aligned along other than the single best routing 460 metric path, so that available network resources can be efficiently 461 applied to meet service requests. Considerations of maximizing 462 network efficiency would imply that some form of path selection is 463 necessary within a QoS architecture, allowing the set of service 464 requirements to be optimally supported within the network's 465 aggregate resource capability. 467 In addition to path selection, SPF-based interior routing protocols 468 allow for the flooding of link metric information across all network 469 elements. This mechanism appears to be a productive direction to 470 provide the control-level signaling between the interior of the 471 network and the network admission elements, allowing the admission 472 systems to admit traffic based on current resource availability 473 rather than on necessarily conservative statically defined admission 474 criteria. 476 There is a more fundamental issue here concerning resource 477 management and traffic engineering. The approach of single path 478 selection with static load characteristics does not match a 479 networked environment which contains a richer mesh of connectivity 480 and dynamic load characteristics. In order to make efficient use of 481 a rich connectivity mesh, it is necessary to be able to direct 482 traffic with a common ingress and egress point across a set of 483 available network paths, spreading the load across a broader 484 collection of network links. At its basic form this is essentially a 485 traffic engineering problem. To support this function it is 486 necessary to calculate per-path dynamic load metrics, and allow the 487 network's ingress system the ability to distribute incoming traffic 488 across these paths in accordance with some model of desired traffic 489 balance. To apply this approach to a QoS architecture would imply 490 that each path has some form of vector of quality attributes, and 491 incoming traffic is balanced across a subset of available paths 492 where the quality attribute of the traffic is matched with the 494 Huston Informational - Expires December 2000 9 495 quality vector of each available path. This augmentation to the 496 semantics of the traffic engineering is matched by a corresponding 497 shift in the calculation and interpretation of the path's quality 498 vector. In this approach what needs to be measured is not the path's 499 resource availability level (or idle proportion), but the path's 500 potential to carry additional traffic at a certain level of quality. 501 This potential metric is one that allows existing lower priority 502 traffic to be displaced to alternative paths. The path's quality 503 metric can be interpreted as a metric describing the displacement 504 capability of the path, rather than a resource availability metric. 506 This area of active network resource management, coupled with 507 dynamic network resource discovery, and the associated control level 508 signaling to network admission systems appears to be a topic for 509 further research at this point in time. 511 4.5 TCP and QoS 513 A congestion-managed rate-adaptive traffic flow (such as used by 514 TCP) uses the feedback from the ACK packet stream to time subsequent 515 data transmissions. The resultant traffic flow rate is an outcome of 516 the service quality provided to both the forward data packets and 517 the reverse ACK packets. If the ACK stream is treated by the network 518 with a different service profile to the outgoing data packets, it 519 remains an open question as to what extent will the data forwarding 520 service be compromised in terms of achievable throughput. High rates 521 of jitter on the ACK stream can cause ACK compression, that in turn 522 will cause high burst rates on the subsequent data send. Such bursts 523 will stress the service capacity of the network and will compromise 524 TCP throughput rates. 526 One way to address this is to use some form of symmetric service, 527 where the ACK packets are handled using the same service class as 528 the forward data packets. If symmetric service profiles are 529 important for TCP sessions, how can this be structured in a fashion 530 that does not incorrectly account for service usage? In other words, 531 how can both directions of a TCP flow be accurately accounted to one 532 party? 534 Additionally, there is the interaction between the routing system 535 and the two TCP data flows. The Internet routing architecture does 536 not intrinsically preserve TCP flow symmetry, and the network path 537 taken by the forward packets of a TCP session may not exactly 538 correspond to the path used by the reverse packet flow. 540 TCP also exposes an additional performance constraint in the manner 541 of the traffic conditioning elements in a QoS-enabled network. 542 Traffic conditioners within QoS architectures are typically 543 specified using a rate enforcement mechanism of token buckets. Token 544 bucket traffic conditioners behave in a manner that is analogous to 545 a First In First Out queue. Such traffic conditioning systems impose 546 tail drop behavior on TCP streams. This tail drop behavior can 547 produce TCP timeout retransmission, unduly penalizing the average 549 Huston Informational - Expires December 2000 10 550 TCP goodput rate to a level that may be well below the level 551 specified by the token bucket traffic conditioner. Token buckets can 552 be considered as TCP-hostile network elements. 554 The larger issue exposed in this consideration is that provision of 555 some form of assured service to congestion-managed traffic flows 556 requires traffic conditioning elements that operate using weighted 557 RED-like control behaviors within the network, with less 558 deterministic traffic patterns as an outcome. A requirement to 559 manage TCP burst behavior through token bucket control mechanisms is 560 most appropriately managed in the sender's TCP stack. 562 There are a number of open areas in this topic that would benefit 563 from further research. The nature of the interaction between the 564 end-to-end TCP control system and a collection of service 565 differentiation mechanisms with a network is has a large number of 566 variables. The issues concern the time constants of the control 567 systems, the amplitude of feedback loops, and the extent to which 568 each control system assumes an operating model of other active 569 control systems that are applied to the same traffic flow, and the 570 mode of convergence to a stable operational state for each control 571 system. 573 4.6 Per-Flow States and Per-Packet classifiers 575 Both the IntServ and DiffServ architectures use packet classifiers 576 as an intrinsic part of their architecture. These classifiers can be 577 considered as coarse or fine level classifiers. Fine-grained 578 classifiers can be considered as classifiers that attempt to isolate 579 elements of traffic from an invocation of an application (a `micro- 580 flow') and use a number of fields in the IP packet header to assist 581 in this, typically including the source and destination IP addresses 582 and source and source and destination port addresses. Coarse-grained 583 classifiers attempt to isolate traffic that belongs to an aggregated 584 service state, and typically use the DiffServ code field as the 585 classifying field. In the case of DiffServ there is the potential to 586 use fine-grained classifiers as part of the network ingress element, 587 and coarse-gained classifiers within the interior of the network. 589 Within flow-sensitive IntServ deployments, every active network 590 element that undertakes active service discrimination is requirement 591 to operate fine-grained packet classifiers. The granularity of the 592 classifiers can be relaxed with the specification of aggregate 593 classifiers [5], but at the expense of the precision and accuracy of 594 the service response. 596 Within the IntServ architecture the fine-grained classifiers are 597 defined to the level of granularity of an individual traffic flow, 598 using the packet's 5-tuple of (source address, destination address, 599 source port, destination port, protocol) as the means to identify an 600 individual traffic flow. The DiffServ Multi-Field (MF) classifiers 601 are also able to use this 5-tuple to map individual traffic flows 602 into supported behavior aggregates. 604 Huston Informational - Expires December 2000 11 605 The use of IPSEC, NAT and various forms of IP tunnels result in a 606 occlusion of the flow identification within the IP packet header, 607 combining individual flows into a larger aggregate state that may be 608 too coarse for the network's service policies. The issue with such 609 mechanisms is that they may occur within the network path in a 610 fashion that is not visible to the end application, compromising the 611 ability for the application to determine whether the requested 612 service profile is being delivered by the network. In the case of 613 IPSEC there is a proposal to carry the IPSEC Security Parameter 614 Index (SPI) in the RSVP object [10], as a surrogate for the port 615 addresses. In the case of NAT and various forms of IP tunnels, there 616 appears to be no coherent way to preserve fine-grained 617 classification characteristics across NAT devices, or across tunnel 618 encapsulation. 620 IP packet fragmentation also affects the ability of the network to 621 identify individual flows, as the trailing fragments of the IP 622 packet will not include the TCP or UDP port address information. 623 This admits the possibility of trailing fragments of a packet within 624 a distinguished service class being classified into the base best 625 effort service category, and delaying the ultimate delivery of the 626 IP packet to the destination until the trailing best effort 627 delivered fragments have arrived. 629 The observation made here is that QoS services do have a number of 630 caveats that should be placed on both the application and the 631 network. Applications should perform path MTU discovery in order to 632 avoid packet fragmentation. Deployment of various forms of payload 633 encryption, header address translation and header encapsulation 634 should be undertaken with due attention to their potential impacts 635 on service delivery packet classifiers. 637 4.7 The Service Set 639 The underlying question posed here is how many distinguished service 640 responses are adequate to provide a functionally adequate range of 641 service responses? 643 The Differentiated Services architecture does not make any limiting 644 restrictions on the number of potential services that a network 645 operator can offer. The network operator may be limited to a choice 646 of up to 64 discrete services in terms of the 6 bit service code 647 point in the IP header but as the mapping from service to code point 648 can be defined by each network operator, there can be any number of 649 potential services. 651 As always, there is such a thing as too much of a good thing, and a 652 large number of potential services leads to a set of issues around 653 end-to-end service coherency when spanning multiple network domains. 654 A small set of distinguished services can be supported across a 655 large set of service providers by equipment vendors and by 656 application designers alike. An ill-defined large set of potential 657 services often serves little productive purpose. This does point to 658 a potential refinement of the QoS architecture to define a small 660 Huston Informational - Expires December 2000 12 661 core set of service profiles as _well-known_ service profiles, and 662 place all other profiles within a _private use_ category. 664 4.8 Measuring Service Delivery 666 There is a strong requirement within any QoS architecture for 667 network management approaches that provide a coherent view of the 668 operating state of the network. This differs from a conventional 669 element-by-element management view of the network in that the desire 670 here is to be able to provide a view of the available resources 671 along a particular path within a network, and map this view to an 672 admission control function which can determine whether to admit a 673 service differentiated flow along the nominated network path. 675 As well as managing the admission systems through resource 676 availability measurement, there is a requirement to be able to 677 measure the operating parameters of the delivered service. Such 678 measurement methodologies are required in order to answer the 679 question of how the network operator provides objective measurements 680 to substantiate the claim that the delivered service quality 681 conformed to the service specifications. Equally, there is a 682 requirement for a measurement methodology to allow the client to 683 measure the delivered service quality so that any additional expense 684 that may be associated with the use of premium services can be 685 justified in terms of superior application performance. 687 Such measurement methodologies appear to fall within the realm of 688 additional refinement to the QoS architecture. 690 4.9 QoS Accounting 692 It is reasonable to anticipate that such forms of premium service 693 and customized service will attract an increment on the service 694 tariff. The provision of a distinguished service is undertaken with 695 some level of additional network resources to support the service, 696 and the tariff premium should reflect this altered resource 697 allocation. Not only does such an incremental tariff shift the added 698 cost burden to those clients who are requesting a disproportionate 699 level of resources, but it provides a means to control the level of 700 demand for premium service levels. 702 If there are to be incremental tariffs on the use of premium 703 services, then some accounting of the use of the premium service 704 would appear to be necessary relating use of the service to a 705 particular client. So far there is no definition of such an 706 accounting model nor a definition as to how to gather the data to 707 support the resource accounting function. 709 The impact of this QoS service model may be quite profound to the 710 models of Internet service provision. The commonly adopted model in 711 both the public internet and within enterprise networks is that of a 712 model of access, where the clients service tariff is based on the 713 characteristics of access to the services, rather than that of the 714 actual use of the service. The introduction of QoS services creates 716 Huston Informational - Expires December 2000 13 717 a strong impetus to move to usage-based tariffs, where the tariff is 718 based on the level of use of the network's resources. This, in turn, 719 generates a requirement to meter resource use, which is a form of 720 usage accounting. This topic was been previously studied within the 721 IETF under the topic of _Internet Accounting_ [11], and further 722 refinement of the concepts used in this model, as they apply to QoS 723 accounting may prove to be a productive initial step in formulating 724 a standards-based model for QoS accounting. 726 4.10 QoS Deployment Diversity 728 It is extremely improbable that any single form of service 729 differentiation technology will be rolled out across the Internet 730 and across all enterprise networks. 732 Some networks will deploy some form of service differentiation 733 technology while others will not. Some of these service platforms 734 will interoperate seamlessly and other less so. To expect all 735 applications, host systems, network routers, network policies, and 736 inter-provider arrangements to coalesce into a single homogenous 737 service environment that can support a broad range of service 738 responses is an somewhat unlikely outcome given the diverse nature 739 of the available technologies and industry business models. It is 740 more likely that we will see a number of small scale deployment of 741 service differentiation mechanisms and some efforts to bridge these 742 environments together in some way. 744 In this heterogeneous service environment the task of service 745 capability discovery is as critical as being able to invoke service 746 responses and measure the service outcomes. QoS architectures will 747 need to include protocol capabilities in supporting service 748 discovery mechanisms. 750 In addition, such a heterogeneous deployment environment will create 751 further scaling pressure on the operational network as now there is 752 an additional dimension to the size of the network. Each potential 753 path to each host is potentially qualified by the service 754 capabilities of the path. While one path may be considered as a 755 candidate best effort path, another path may offer a more precise 756 match between the desired service attributes and the capabilities of 757 the path to sustain the service. Inter-domain policy also impacts 758 upon this path choice, where inter-domain transit agreements may 759 specifically limit the types and total level of quality requests 760 than may be supported between the domains. Much of the brunt of such 761 scaling pressures will be seen in the inter-domain and intra-domain 762 routing domain where there are pressures to increase the number of 763 attributes of a routing entry, and also to use the routing protocol 764 in some form of service signaling role. 766 4.11 QoS Inter-Domain signaling 768 QoS Path selection is both an intra-domain (interior) and an inter- 769 domain (exterior) issue. Within the inter-domain space, the current 771 Huston Informational - Expires December 2000 14 772 routing technologies allow each domain to connect to a number of 773 other domains, and to express its policies with respect to received 774 traffic in terms of inter-domain route object attributes. 775 Additionally, each domain may express its policies with respect to 776 sending traffic through the use of boundary route object filters, 777 allowing a domain to express its preference for selecting one 778 domain's advertised routes over another. The inter-domain routing 779 space is a state of dynamic equilibrium between these various route 780 policies. 782 The introduction of differentiated services adds a further dimension 783 to this policy space. For example, while a providers may execute an 784 interconnection agreement with one party to exchange best effort 785 traffic, it may execute another agreement with a second party to 786 exchange service qualified traffic. The outcome of this form of 787 interconnection is that the service provider will require external 788 route advertisements to be qualified by the accepted service 789 profiles. Generalizing from this scenario, it is reasonable to 790 suggest that we will require the qualification of routing 791 advertisements with some form of service quality attributes. This 792 implies that we will require some form of quality vector-based 793 forwarding function, at least in the inter-domain space, and some 794 associated routing protocol can pass a quality of service vector in 795 an operationally stable fashion. 797 The implication of this requirement is that the number of objects 798 being managed by routing systems must expand dramatically, as the 799 size and number of objects managed within the routing domain 800 increases, and the calculation of a dynamic equilibrium of import 801 and export policies between interconnected providers will also be 802 subject to the same level of scaling pressure. 804 This has implications within the inter-domain forwarding space as 805 well, as the forwarding decision in such a services differentiated 806 environment is then qualified by some form of service quality 807 vector. This is required in order to pass exterior traffic to the 808 appropriate exterior interconnection gateway. 810 4.12 QoS Deployment Logistics 812 How does the widespread deployment of service-aware networks 813 commence? Which gets built first - host applications or network 814 infrastructure? 816 No network operator will make the significant investment in 817 deployment and support of distinguished service infrastructure 818 unless there is a set of clients and applications available to make 819 immediate use of such facilities. Clients will not make the 820 investment in enhanced services unless they see performance gains in 821 applications that are designed to take advantage of such enhanced 822 services. No application designer will attempt to integrate service 823 quality features into the application unless there is a model of 824 operation supported by widespread deployment that makes the 825 additional investment in application complexity worthwhile and 827 Huston Informational - Expires December 2000 15 828 clients who are willing to purchase such applications. With all 829 parts of the deployment scenario waiting for the others to move, 830 widespread deployment of distinguished services may require some 831 other external impetus. 833 Further aspects of this deployment picture lie in the issues of 834 network provisioning and the associated task of traffic engineering. 835 Engineering a network to meet the demands of best effort flows 836 follows a well understood pattern of matching network points of user 837 concentrations to content delivery network points with best effort 838 paths. Integrating QoS-mediated traffic engineering into the 839 provisioning model suggests a provisioning requirement that also 840 requires input from a QoS demand model. 842 5. The objective of the QoS architecture 844 What is the precise nature of the problem that QoS is attempting to 845 solve? Perhaps this is one of the more fundamental questions 846 underlying the QoS effort, and the diversity of potential responses 847 is a pointer to the breadth of scope of the QoS effort. 849 All of the following responses form a part of the QoS intention: 851 - To control the network service response such that the response 852 to a specific service element is consistent and predictable. 854 - To control the network service response such that a service 855 element is provided with a level of response equal to or above a 856 guaranteed minimum. 858 - To allow a service element to establish in advance the service 859 response that can or will be obtained from the network. 861 - To control the contention for network resources such that a 862 service element is provided with a superior level of network 863 resource. 865 - To control the contention for network resources such that a 866 service element does not obtain an unfair allocation of 867 resources (to some definition of 'fairness'). 869 - To allow for efficient total utilization of network resources 870 while servicing a spectrum of directed network service outcomes. 872 Broadly speaking, the first three responses can be regarded as 873 'application-centric', and the latter as 'network-centric'. It is 874 critical to bear in mind that none of these responses can be 875 addressed in isolation within any effective QoS architecture. Within 876 the end-to-end architectural model of the Internet, applications 877 make minimal demands on the underlying IP network. In the case of 878 TCP, the protocol uses an end-to-end control signal approach to 879 dynamically adjust to the prevailing network state. QoS 880 architectures add a somewhat different constraint, in that the 882 Huston Informational - Expires December 2000 16 883 network is placed in an active role within the task of resource 884 allocation and service delivery, rather than being a passive object 885 that requires end systems to adapt. 887 6. Towards an end-to-end QoS architecture 889 The challenge facing the QoS architecture lies in addressing the 890 weaknesses noted above, and in integrating the various elements of 891 the architecture into a cohesive whole that is capable of sustaining 892 end-to-end service models across a wide diversity of internet 893 platforms. It should be noted that such an effort may not 894 necessarily result in a single resultant architecture, and that it 895 is possible to see a number of end-to-end approaches based on 896 different combinations of the existing components. 898 One approach is to attempt to combine both architectures into an 899 end-to-end model, using IntServ as the architecture which allows 900 applications to interact with the network, and DiffServ as the 901 architecture to manage admission the network's resources [7]. In 902 this approach, the basic tension that needs to be resolved lies in 903 difference between the per-application view of the IntServ 904 architecture and the network boundary-centric view of the DiffServ 905 architecture. 907 One building block for such an end-to-end service architecture is a 908 service signaling protocol. The RSVP signaling protocol can address 909 the needs of applications that require a per-service end-to-end 910 service signaling environment. The abstracted model of RSVP is that 911 of a discovery signaling protocol that allows an application to use 912 a single transaction to communicate its service requirements to both 913 the network and the remote party, and through the response 914 mechanism, to allow these network elements to commit to the service 915 requirements. The barriers to deployment for this model lie in an 916 element-by element approach to service commitment, implying that 917 each network element must undertake some level of signaling and 918 processing as dictated by this imposed state. For high precision 919 services this implies per-flow signaling and per-flow processing to 920 support this service model. This fine-grained high precision 921 approach to service management is seen as imposing an unacceptable 922 level of overhead on the central core elements of large carrier 923 networks. 925 The DiffServ approach uses a model of abstraction which attempts to 926 create an external view of a compound network as a single 927 subnetwork. From this external perspective the network can be 928 perceived as two boundary service points, ingress and egress. The 929 advantage of this approach is that there exists the potential to 930 eliminate the requirement for per-flow state and per-flow processing 931 on the interior elements of such a network, and instead provide 932 aggregate service responses. 934 One approach is for applications to use RSVP to request that their 935 flows be admitted into the network. If a request is accepted, it 937 Huston Informational - Expires December 2000 17 938 would imply that there is a committed resource reservation within 939 the IntServ-capable components of the network, and that the service 940 requirements have been mapped into a compatible aggregate service 941 class within the DiffServ-capable network [7]. The DiffServ core 942 must be capable of carrying the RSVP messages across the DiffServ 943 network, so that further resource reservation is possible within the 944 IntServ network upon egress from the DiffServ environment. The 945 approach calls for the DiffServ network to use per-flow multi-field 946 (MF) classifier, where the MF classification is based on the RSVP- 947 signaled flow specification. The service specification of the RSVP- 948 signaled resource reservation is mapped into a compatible aggregate 949 DiffServ behavior aggregate and the MF classifier marks packets 950 according to the selected behavior. Alternatively the boundary of 951 the IntServ and DiffServ networks can use the IntServ egress to mark 952 the flow packets with the appropriate DSCP, allowing the DiffServ 953 ingress element to use the BA classifier, and dispense with the per- 954 flow MF classifier. 956 A high precision end-to-end QoS model requires that any admission 957 failure within the DiffServ network be communicated to the end 958 application, presumably via RSVP. This allows the application to 959 take some form of corrective action, either by modifying it's 960 service requirements or terminating the application. If the service 961 agreement between the DiffServ network is statically provisioned, 962 then this static information can be loaded into the IntServ boundary 963 systems, and IntServ can manage the allocation of available DiffServ 964 behavior aggregate resources. If the service agreement is 965 dynamically variable, some form of signaling is required between the 966 two networks to pass this resource availability information back 967 into the RSVP signaling environment. 969 7. Conclusions 971 None of these observations are intended to be any reason to condemn 972 the QoS architectures as completely impractical, nor are they 973 intended to provide any reason to believe that the efforts of 974 deploying QoS architectures will not come to fruition. 976 What this document is intended to illustrate is that there are still 977 a number of activities that are essential precursors to widespread 978 deployment and use of such QoS networks, and that there is a need to 979 fill in the missing sections with something substantial in terms of 980 adoption of additional refinements to the existing QoS model. 982 The architectural direction that appears to offer the most promising 983 outcome for QoS is not one of universal adoption of a single 984 architecture, but instead use a tailored approach where aggregated 985 service elements are used in the core of a network where scalability 986 is a major design objective and use per-flow service elements at the 987 edge of the network where accuracy of the service response is a 988 sustainable outcome. 990 Architecturally, this points to no single QoS architecture, but 991 rather to a set of QoS mechanisms and a number of ways these 993 Huston Informational - Expires December 2000 18 994 mechanisms can be configured to interoperate in a stable and 995 consistent fashion. 997 8. Security Considerations 999 The Internet is not an architecture that includes a strict 1000 implementation of fairness of access to the common transmission and 1001 switching resource. The introduction of any form of fairness, and, 1002 in the case of QoS, weighted fairness, implies a requirement for 1003 transparency in the implementation of the fairness contract between 1004 the network provider and the network's users. This requires some 1005 form of resource accounting and auditing, which, in turn, requires 1006 the use of authentication and access control. The balancing factor 1007 is that a shared resource should not overtly expose the level of 1008 resource usage of any one user to any other, so that some level of 1009 secrecy is required in this environment 1011 The QoS environment also exposes the potential of theft of resources 1012 through the unauthorized admission of traffic with an associated 1013 service profile. QoS signaling protocols which are intended to 1014 undertake resource management and admission control require the use 1015 of identity authentication and integrity protection in order to 1016 mitigate this potential for theft of resources. 1018 Both forms of QoS architecture require the internal elements of the 1019 network to be able to undertake classification of traffic based on 1020 some form of identification that is carried in the packet header in 1021 the clear. Classifications systems that use multi-field specifiers, 1022 or per-flow specifiers rely on the carriage of end-to-end packet 1023 header fields being carried in the clear. This has conflicting 1024 requirements for security architectures that attempt to mask such 1025 end-to-end identifiers within an encrypted payload. 1027 QoS architectures can be considered as a means of exerting control 1028 over network resource allocation. In the event of a rapid change in 1029 resource availability (e.g. disaster) it is an undesirable outcome 1030 if the remaining resources are completely allocated to a single 1031 class of service to the exclusion of all other classes. Such an 1032 outcome constitutes a denial of service, where the traffic control 1033 system (routing) selects paths that are incapable of carrying any 1034 traffic of a particular service class. 1036 9. References 1038 [1] Bradner, S., "The Internet Standards Process- Revision 3", 1039 BCP9, RFC2026, October 1996. 1041 [2] Mankin, A., Baker, F., Braden, R., O'Dell, M., Romanow, A., 1042 Weinrib, A., Zhang, L., "Resource ReSerVation Protocol (RSVP) 1043 Version 1 Applicability Statement", RFC2208, September 1997. 1045 [3] Braden. R., Clark, D., Shenker, S., "Integrated Services in the 1046 Internet Architecture: an Overview", RFC1633, June 1994. 1048 Huston Informational - Expires December 2000 19 1050 [4] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., Weiss, 1051 W., "An Architecture for Differentiated Services", RFC2475, December 1052 1998. 1054 [5] Baker, F., Iturralde, C., Le Faucher, F., Davie, B., 1055 "Aggregation of RSVP for IPv4 and IPv6 Reservations", work in 1056 progress, March 2000. 1058 [6] Bernet, Y., "Format of the RSVP DCLASS Object", work in 1059 progress, October 1999. 1061 [7] Bernet, Y., Yavatkar, R., Ford, P., Baker, F., Zhang, L., 1062 Speer, M., Baden, R., Davie, B., Wroclawski, J., Felstaine, E., "A 1063 Framework for Integrated Services Operation Over DiffServ Networks", 1064 work in progress, May 2000. 1066 [8] "Quality of Service Technical Overview", Microsoft Technical 1067 Library, Microsoft Corporation, September 1999. 1069 [9] "Resource Reservation Protocol API (RAPI)", Open Group 1070 Technical Standard, C809 ISBN 1-85912-226-4, The Open Group, 1071 December 1998. 1073 [10] Berger, L., O'Malley, T., "RSVP Extensions for IPSEC Data 1074 Flows", RFC 2007, September 1997. 1076 [11] Mills, C., Hirsh, D, Ruth, G., "Internet Accounting: 1077 Background", RFC1272, November 1991. 1079 10. Acknowledgments 1081 Valuable contributions to this draft came from Yoram Bernet, Brian 1082 Carpenter, Jon Crowcroft, Tony Hain and Henning Schulzrinne. 1084 11. Author's Addresses 1086 Geoff Huston 1087 Telstra 1088 5/490 Northbourne Ave 1089 Dickson ACT 2602 1090 AUSTRALIA 1091 Email: gih@telstra.net