idnits 2.17.1 draft-ietf-wrec-known-prob-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 2 instances of too long lines in the document, the longest one being 1 character in excess of 72. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 317: '...300, 301 or 410] MUST NOT be returned ...' RFC 2119 keyword, line 751: '... caches MAY store partial respons...' Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == Line 513 has weird spacing: '... Users may s...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (November 13, 2000) is 8566 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '7' is defined on line 808, but no explicit reference was found in the text == Unused Reference: '8' is defined on line 812, but no explicit reference was found in the text == Unused Reference: '10' is defined on line 817, but no explicit reference was found in the text == Unused Reference: '11' is defined on line 819, but no explicit reference was found in the text ** Downref: Normative reference to an Informational RFC: RFC 2525 (ref. '1') ** Downref: Normative reference to an Informational draft: draft-ietf-wrec-taxonomy (ref. '2') ** Obsolete normative reference: RFC 2616 (ref. '3') (Obsoleted by RFC 7230, RFC 7231, RFC 7232, RFC 7233, RFC 7234, RFC 7235) == Outdated reference: A later version (-10) exists of draft-mogul-http-delta-07 ** Downref: Normative reference to an Informational RFC: RFC 2186 (ref. '5') -- Possible downref: Non-RFC (?) normative reference: ref. '6' -- Possible downref: Non-RFC (?) normative reference: ref. '7' -- Possible downref: Non-RFC (?) normative reference: ref. '8' -- Possible downref: Non-RFC (?) normative reference: ref. '9' -- Possible downref: Non-RFC (?) normative reference: ref. '10' -- Possible downref: Non-RFC (?) normative reference: ref. '11' -- Possible downref: Non-RFC (?) normative reference: ref. '12' -- Possible downref: Non-RFC (?) normative reference: ref. '13' -- Possible downref: Non-RFC (?) normative reference: ref. '14' -- Possible downref: Non-RFC (?) normative reference: ref. '15' Summary: 9 errors (**), 0 flaws (~~), 8 warnings (==), 12 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group I. Cooper 3 Internet-Draft Equinix 4 Expires: May 14, 2001 J. Dilley 5 Akamai 6 November 13, 2000 8 Known HTTP Proxy/Caching Problems 9 draft-ietf-wrec-known-prob-03.txt 11 Status of this Memo 13 This document is an Internet-Draft and is in full conformance with 14 all provisions of Section 10 of RFC2026. 16 Internet-Drafts are working documents of the Internet Engineering 17 Task Force (IETF), its areas, and its working groups. Note that 18 other groups may also distribute working documents as 19 Internet-Drafts. 21 Internet-Drafts are draft documents valid for a maximum of six 22 months and may be updated, replaced, or obsoleted by other documents 23 at any time. It is inappropriate to use Internet-Drafts as reference 24 material or to cite them other than as "work in progress." 26 The list of current Internet-Drafts can be accessed at 27 http://www.ietf.org/ietf/1id-abstracts.txt. 29 The list of Internet-Draft Shadow Directories can be accessed at 30 http://www.ietf.org/shadow.html. 32 This Internet-Draft will expire on May 14, 2001. 34 Copyright Notice 36 Copyright (C) The Internet Society (2000). All Rights Reserved. 38 Abstract 40 This memo catalogs a number of known problems with World Wide Web 41 (caching) proxies and cache servers. The goal of the document is to 42 provide a discussion of the problems and proposed workarounds, and 43 ultimately to improve conditions by illustrating problems. The 44 construction of this document is a joint effort of the Web caching 45 community. 47 Table of Contents 49 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 50 1.1 Problem Template . . . . . . . . . . . . . . . . . . . . . . 3 51 2. Known Problems . . . . . . . . . . . . . . . . . . . . . . . 6 52 2.1 Known Specification Problems . . . . . . . . . . . . . . . . 6 53 2.1.1 Vary header is underspecified and/or misleading . . . . . . 6 54 2.2 Known Architectural Problems . . . . . . . . . . . . . . . . 10 55 2.2.1 Interception proxies break client cache directives . . . . . 10 56 2.2.2 Interception proxies prevent introduction of new HTTP 57 methods . . . . . . . . . . . . . . . . . . . . . . . . . . 11 58 2.2.3 Interception proxies break IP address-based authentication . 12 59 2.2.4 Caching proxy peer selection in heterogeneous networks . . . 12 60 2.2.5 ICP Performance . . . . . . . . . . . . . . . . . . . . . . 14 61 2.2.6 Caching proxy meshes can break HTTP serialization of content 15 62 2.3 Known Implementation Problems . . . . . . . . . . . . . . . 16 63 2.3.1 User agent/proxy failover . . . . . . . . . . . . . . . . . 16 64 2.3.2 Some servers send bad Content-Length headers for files that 65 contain CR . . . . . . . . . . . . . . . . . . . . . . . . . 16 66 3. Security Considerations . . . . . . . . . . . . . . . . . . 18 67 References . . . . . . . . . . . . . . . . . . . . . . . . . 19 68 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . 19 69 A. Archived Known Problems . . . . . . . . . . . . . . . . . . 21 70 A.1 Architectural . . . . . . . . . . . . . . . . . . . . . . . 21 71 A.1.1 Cannot specify multiple URIs for replicated resources . . . 21 72 A.1.2 Replica distance is unknown . . . . . . . . . . . . . . . . 21 73 A.1.3 Proxy resource location . . . . . . . . . . . . . . . . . . 22 74 A.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . 23 75 A.2.1 Use of Cache-Control headers . . . . . . . . . . . . . . . . 23 76 A.2.2 Lack of HTTP/1.1 compliance for caching proxies . . . . . . 24 77 A.2.3 ETag support . . . . . . . . . . . . . . . . . . . . . . . . 25 78 A.2.4 Servers and content should be optimized for caching . . . . 26 79 A.3 Administration . . . . . . . . . . . . . . . . . . . . . . . 26 80 A.3.1 Lack of fine-grained, standardized hierarchy controls . . . 26 81 A.3.2 Proxy/Server exhaustive log format standard for analysis . . 27 82 A.3.3 Trace log timestamps . . . . . . . . . . . . . . . . . . . . 28 83 A.3.4 Exchange format for log summaries . . . . . . . . . . . . . 29 84 Full Copyright Statement . . . . . . . . . . . . . . . . . . 31 86 1. Introduction 88 This memo discusses problems with proxies - which act as 89 application-level intermediaries for Web requests - and more 90 specifically with caching proxies, which retain copies of previously 91 requested resources in the hope of improving overall quality of 92 service by serving the content locally. Commonly used terminology in 93 this memo can be found in the "Internet Web Replication and Caching 94 Taxonomy[2]. 96 No individual or organization has complete knowledge of the known 97 problems in Web caching, and the editors are grateful to the 98 contributors to this document. 100 1.1 Problem Template 102 A common problem template is used within the following sections. We 103 gratefully acknowledge RFC2525[1] which helped define an initial 104 format for this known problems list. The template format is 105 summarized in the following table and described in more detail below. 107 Name: short, descriptive name of the problem (3-5 words) 108 Classification: classifies the problem: performance, security, etc 109 Description: describes the problem succinctly 110 Significance: magnitude of problem, environments where it exists 111 Implications: the impact of the problem on systems and networks 112 See Also: a reference to a related known problem 113 Indications: states how to detect the presence of this problem 114 Solution(s): describe the solution(s) to this problem, if any 115 Workaround: practical workaround for the problem 116 References: information about the problem or solution 117 Contact: contact name and email address for this section 119 Name 120 A short, descriptive, name (3-5 words) name associated with the 121 problem. 123 Classification 124 Problems are grouped into categories of similar problems for ease 125 of reading of this memo. Choose the category that best describes 126 the problem. The suggested categories include three general 127 categories and several more specific categories. 129 * Architecture: the fundamental design is incomplete, or 130 incorrect 132 * Specification: the spec is ambiguous, incomplete, or incorrect. 134 * Implementation: the implementation of the spec is incorrect. 136 * Performance: perceived page response at the client is 137 excessive; network bandwidth consumption is excessive; demand 138 on origin or proxy servers exceed reasonable bounds. 140 * Administration: care and feeding of caches is, or causes, a 141 problem. 143 * Security: privacy, integrity, or authentication concerns. This 144 is the first draft of this memo. The classification structure 145 is in revision. In the published drafts of the memo the 146 classification structure should be fixed but may be revised 147 from time to time. 149 Description 150 A definition of the problem, succinct but including necessary 151 background information. 153 Significance (High, Medium, Low) 154 May include a brief summary of the environments for which the 155 problem is significant. 157 Implications 158 Why the problem is viewed as a problem. What inappropriate 159 behavior results from it? This section should substantiate the 160 magnitude of any problem indicated with High significance. 162 See Also 163 Optional. List of other known problems that are related to this 164 one. 166 Indications 167 How to detect the presence of the problem. This may include 168 references to one or more substantiating documents that 169 demonstrate the problem. This should include the network 170 configuration that led to the problem such that it can be 171 reproduced. Problems that are not reproduceable will not appear 172 in this memo. 174 Solution(s) 175 Solutions that permanently fix the problem, if such are known. 176 For example, what version of the software does not exhibit the 177 problem? Indicate if the solution is accepted by the community, 178 one of several solutions pending agreement, or open possibly with 179 experimental solutions. 181 Workaround 182 Practical workaround if no solution is available or usable. The 183 workaround should have sufficient detail for someone experiencing 184 the problem to get around it. 186 References 187 References to related information in technical publications or on 188 the web. Where can someone interested in learning more go to find 189 out more about this problem, its solution, or workarounds? 191 Contact 192 Contact name and email address of the person who supplied the 193 information for this section. If you would prefer to remain 194 anonymous the editor's name will appear here instead, but we 195 believe in credit where credit is due. 197 2. Known Problems 199 The remaining sections of this document present the currently 200 documented known problems. The problems are ordered by 201 classification and significance. Issues with protocol specification 202 or architecture are first, followed by implementation issues. Issues 203 of high significance are first, followed by lower significance. 205 Some of the problems initially identified in the drafts of this 206 document have been moved to Appendix A since they discuss issues 207 whose resolution primarily involves education rather than protocol 208 work. 210 A full list of the problems is available in the table of contents. 212 2.1 Known Specification Problems 214 2.1.1 Vary header is underspecified and/or misleading 216 Name 217 The "Vary" header is underspecified and/or misleading 219 Classification 220 Specification 222 Description 223 The Vary header in HTTP/1.1[3] was designed to allow a caching 224 proxy to safely cache responses even if it did not entirely 225 understand the server's choice of variants. As RFC2616 says: 227 "The Vary header field can be used to express the 228 parameters the server uses to select a representation 229 that is subject to server-driven negotiation." 231 One might expect that this mechanism is useful in general for 232 extensions that change the response message based on some aspects 233 of the request. However, that is not true. 235 We realized during the design of the HTTP delta encoding 236 specification[4] that an HTTP/1.1 caching proxy that does not 237 understand delta encoding might cache a delta-encoded response 238 and then later deliver it to a non-delta-capable client, unless 239 the extension included some mechanism to prevent this. 240 Initially, we thought that Vary would suffice, but the following 241 scenario proves this wrong. (See [4]for background on the new 242 headers.) 244 Suppose client A sends this request via caching proxy P: 246 GET http://example.com/foo.html HTTP/1.1 247 Host: example.com 248 If-None-Match: "abc" 249 A-IM: vcdiff 251 and the origin server returns, via P, this response: 253 HTTP/1.1 226 IM Used 254 Etag: "def" 255 Date: Wed, 19 Apr 2000 18:46:13 GMT 256 IM: vcdiff 257 Cache-Control: max-age-60 258 Vary: A-IM, If-None-Match 260 the body of which is a delta-encoded response (it encodes the 261 difference between the Etag "abc" instance of foo.html, and the 262 "def" instance). Assume that P stores this response in its 263 cache, and that P does not understand the vcdiff encoding. Later, 264 client B, also ignorant of delta-encoding, sends this request via 265 P: 267 GET http://example.com/foo.html HTTP/1.1 268 Host: example.com 270 What can P do now? According to the specification for the Vary 271 header in RFC2616, 273 "The Vary field value indicates the set of request-header 274 fields that fully determines, while the response is fresh, 275 whether a cache is permitted to use the response to reply to 276 a subsequent request without revalidation." 278 Implicitly, however, the cache would be allowed to use the stored 279 response in response to client B WITH "revalidation." This is the 280 potential bug. 282 An obvious implementation of the caching proxy would send this 283 request to test whether its cache entry is fresh (i.e., to 284 revalidate the entry): 286 GET /foo.html HTTP/1.1 287 Host: example.com 288 If-None-Match: "def" 290 That is, the proxy simply forwards the new request, after doing 291 the usual transformation on the URL and tacking on the "obvious" 292 If-None-Match header. 294 If the origin server's Etag for the current instance is still 295 "def", it would naturally respond: 297 HTTP/1.1 304 Not Modified 298 Etag: "def" 299 Date: Wed, 19 Apr 2000 18:46:14 GMT 301 thus telling the caching proxy P that it can use its stored 302 response. But this cache response actually involves a 303 delta-encoding that would not be sensible to client B, signaled 304 by a header field that would be ignored by B, and so the client 305 displays garbage. 307 The problem here is that the original request (from client A) 308 generated a response that is not sensible to client B, not merely 309 one that is not "the appropriate representation" (as the result 310 of server-driven negotiation). 312 One might argue that the proxy P shouldn't be storing status-226 313 responses in the first place. True in theory, perhaps, but 314 unfortunately RFC2616, section 13.4, says: 316 "A response received with any [status code other than 200, 317 203, 206, 300, 301 or 410] MUST NOT be returned in a reply 318 to a subsequent request unless there are cache-control 319 directives or another header(s) that explicitly allow it. 320 For example, these include the following: an Expires header 321 (section 14.21); a "max-age", "s-maxage", "must- 322 revalidate", "proxy-revalidate", "public" or "private" 323 cache-control directive (section 14.9)." 325 In other words, the specification does allow caching of responses 326 with yet-to-be-defined status codes if the response carries a 327 plausible Cache-Control directive. So unless we want to ban 328 servers implementing this kind of extension from using these 329 Cache-Control directives at all, the Vary header just won't work. 331 Significance 332 Medium 334 Implications 335 Certain plausible extensions to the HTTP/1.1 protocol might not 336 interoperate correctly with older HTTP/1.1 caches, if the 337 extensions depend on an interpretation of Vary that is not the 338 same as is used by the cache implementer. 340 This would have the effect either of causing hard-to-debug cache 341 transparency failures, or of discouraging the deployment of such 342 extensions, or of encouraging the implementers of such extensions 343 to disable caching entirely. 345 Indications 346 The problem is visible when hand-simulating plausible message 347 exchanges, especially when using the proposed delta encoding 348 extension. It probably has not been visible in practice yet. 350 Solution(s) 352 1. Section 13.4 of the HTTP/1.1 specification should probably be 353 changed to prohibit caching of responses with status codes 354 that the cache doesn't understand, whether or not they 355 include Expires headers and the like. (It might require some 356 care to define what "understands" means, leaving room for 357 future extensions with new status codes.) The behavior in 358 this case needs to be defined as equivalent to 359 "Cache-Control: no-store" rather than "no-cache", since the 360 latter allows revalidation(!). 362 Possibly the specification of Vary (in 14.44) should require 363 that it be treated as "Cache-Control: no-store" whenever the 364 status code is unknown - that should solve the problem in the 365 scenario given here. 367 2. Designers of HTTP/1.1 extensions should consider using 368 mechanisms other than Vary to prevent false caching. 370 It is not clear whether the Vary mechanism is widely 371 implemented in caches; if not, this favors solution #1. 373 Workaround 374 A cache could treat the presence of a Vary header in a response 375 as an implicit "Cache-control: no-store", except for "known" 376 status codes, even though this is not required by RFC2616. This 377 would avoid any transparency failures. "Known status codes" for 378 basic HTTP/1.1 caches probably includes: 200, 203, 206, 300, 301, 379 410 (although this list should be re-evaluated in light of the 380 problem discussed here). 382 References 383 See draft-mogul-http-delta-07.txt (or its successor) for the 384 specification of the delta encoding extension, as well as for an 385 example of the use of a Cache-Control extension instead of "Vary." 387 Contact 388 Jeff Mogul 389 (who accepts much of the blame for getting this part of RFC2616 390 wrong in the first place) 392 2.2 Known Architectural Problems 394 2.2.1 Interception proxies break client cache directives 396 Name 397 Interception proxies break client cache directives 399 Classification 400 Architecture 402 Description 403 HTTP[3] is designed for the user agent to be aware if it is 404 connected to an origin server or to a proxy. User agents 405 believing they are transacting with an origin server but which 406 are really in a connection with an interception proxy may fail to 407 send critical cache-control information they would have otherwise 408 included in their request. 410 Significance 411 High 413 Implications 414 Clients may receive data that is not synchronized with the origin 415 even when they request an end to end refresh because of the lack 416 of inclusion of either a "Cache-control: no-cache" or 417 "must-revalidate" header. These headers have no impact on origin 418 server behavior so may not be included by the browser if it 419 believes it is connected to that resource. Other related data 420 implications are possible as well. For instance, data security 421 may be compromised by the lack of inclusion of "private" or 422 "no-store" clauses of the Cache-control header under similar 423 conditions. 425 Indications 426 Easily detected by placing fresh (un-expired) content on a proxy 427 while changing the authoritative copy and requesting an end to 428 end reload of the data through a proxy in both interception and 429 explicit modes. 431 Solution(s) 432 Eliminate the need for interception proxies and IP spoofing which 433 will return correct context awareness to the client. 435 Workaround 436 Include relevant cache-control: directives in every request at 437 the cost of increased bandwidth and CPU requirements. 439 Contact 440 Patrick McManus 442 2.2.2 Interception proxies prevent introduction of new HTTP methods 444 Name 445 Interception proxies prevent introduction of new HTTP methods 447 Classification 448 Architecture 450 Description 451 A proxy that receives a request with a method unknown to it is 452 required to generate an HTTP 501 Error as a response. HTTP 453 methods are designed to be extensible so there may be 454 applications deployed with initial support just for the user 455 agent and origin server. An interception proxy that hijacks 456 requests which include new methods destined for servers that have 457 implemented those methods creates a de-facto firewall where none 458 may be intended. 460 Significance 461 Medium within interception proxy environments. 463 Implications 464 Renders new compliant applications useless unless modifications 465 are made to proxy software. Because new methods are not required 466 to be globally standardized it is impossible to keep up to date 467 in the general case. 469 Solution(s) 470 Eliminate the need for interception proxies. A client receiving a 471 501 in a traditional HTTP environment may either choose to repeat 472 the request to the origin server directly, or perhaps be 473 configured to use a different cache. 475 Workaround 476 Level 5 switches (sometimes called Level 7 or application layer 477 switches) can be used to keep HTTP traffic with unknown methods 478 out of the proxy. However, these devices have heavy buffering 479 responsibilities, still require TCP sequence number spoofing, and 480 do not interact well with persistent connections. 482 The HTTP/1.1 specification allows a proxy to switch over to 483 tunnel mode when it receives a request with a method or HTTP 484 version it does not understand how to handle. 486 Contact 487 Patrick McManus 488 Henrik Nordstrom (HTTP/1.1 clarification) 490 2.2.3 Interception proxies break IP address-based authentication 492 Name 493 Interception proxies break IP address-based authentication 495 Classification 496 Architecture 498 Description 499 Some web servers are not open for public access, but restrict 500 themselves to accept only certain IP address ranges for security 501 reasons. Using interception proxies at the ISP level, for 502 example, will alter the source (client) IP addresses to that of 503 the proxy itself. This will break such authentication mechanisms 504 and prohibit the otherwise allowed clients access to the servers. 506 Significance 507 Medium 509 Implications 510 This creates end user confusion and frustration. 512 Indications 513 Users may start to see refused connections to servers after 514 interception proxies are deployed. 516 Solution(s) 517 Use user-based authentication instead of (IP) address-based 518 authentication. 520 Workaround 521 By using IP filters at the intercepting device (L4 switch) and 522 bypass all requests to such servers concerned. 524 Contact 525 Keith K. Chau 527 2.2.4 Caching proxy peer selection in heterogeneous networks 529 Name 530 Caching proxy peer selection in heterogeneous networks 532 Classification 533 Architecture 535 Description 536 ICP[5] based caching proxy peer selection in networks with large 537 variance in latency and bandwidth between peers can lead to 538 non-optimal peer selection. For example take Proxy C with two 539 siblings, Sib1 and Sib2, and the following network topology 540 (summarized). 542 * Cache C's link to Sib1, 2 Mbit/sec with 300 msec latency 544 * Cache C's link to Sib2, 64 Kbit/sec with 10 msec latency. 546 ICP[5] does not work well in this context. If a user submits a 547 request to Proxy C for page P that results in a miss. C will send 548 an ICP request to Sib1 and Sib2. Assume both siblings have the 549 requested object P. The ICP-HIT reply will always come from Sib2 550 before Sib1. However, for large objects it is clear that the 551 retrieval will be faster from Sib1 rather than Sib2. 553 In fact, the problem is more complex because Sib1 and Sib2 can't 554 have a 100% hit ratio. With a hit rate of 10%, it is more 555 efficient to use Sib1 with resources larger than 48K. The best 556 choice depends on at least the hit rate and link characteristics; 557 maybe other parameters as well. 559 Significance 560 Medium 562 Implications 563 By selecting the first peer to respond, peer selection algorithms 564 are not optimizing retrieval latency to end users. Furthermore 565 they are causing more work for the high-latency peer since it 566 must respond to such requests but will never be chosen to serve 567 content if the lower latency peer has a copy. 569 Indications 570 Inherent in design of ICP v1, ICP v2, and any cache mesh protocol 571 that selects peer based upon first response. 573 This problem is not exhibited by cache digest or other protocols 574 which (attempt to) maintain knowledge of peer contents and only 575 hit peers that are believed to have a copy of the requested page. 577 Solution(s) 578 This problem is architectural with the peer selection protocol. 580 Workaround 581 Cache mesh design when using such a protocol should be done in 582 such a way that there is not a high latency variance among peers. 583 In the example presented in the above description the high 584 latency high bandwidth peer could be used as a parent, but should 585 not be used as a sibling. 587 Contact 588 Ivan Lovric 589 John Dilley 591 2.2.5 ICP Performance 593 Name 594 ICP performance 596 Classification 597 Architecture(ICP), Performance 599 Description 600 ICP[5] exhibits O(n^2) scaling properties, where n is the number 601 of peer proxies participating in the protocol. This can lead ICP 602 traffic to dominate HTTP traffic within a network. 604 Significance 605 Medium 607 Implications 608 If a proxy has many ICP peers the bandwidth demand of ICP can be 609 excessive. System managers must carefully regulate ICP peering. 610 ICP also leads proxies to become homogeneous in what they serve. 611 This means if your proxy does not have a document it is unlikely 612 your peers will have it either. Therefore, ICP traffic requests 613 are largely unable to locate a local copy of an object (see 614 [8] 615 . 617 Indications 618 Inherent in design of ICP v1, ICP v2. 620 Solution(s) 621 This problem is architectural - protocol redesign or replacement 622 are required to solve it if ICP is to continue to be used. 624 Workaround 625 Implementation workarounds exist, for example to turn off use of 626 ICP, to carefully regulate peering, or to use another mechanism 627 if available, such as cache digests. A cache digest protocol 628 shares a summary of cache contents using a Bloom Filter 629 technique. This allows a cache to estimate whether a peer has a 630 document. Filters are updated regularly but are not always 631 up-to-date so cannot help when a spike in popularity occurs. They 632 also increase traffic but not as much as ICP. 634 Proxy clustering protocols organize proxies into a mesh provide 635 another alternative solution. There is ongoing research on this 636 topic. 638 Contact 639 John Dilley 641 2.2.6 Caching proxy meshes can break HTTP serialization of content 643 Name 644 Caching proxy meshes can break HTTP serialization of content 646 Classification 647 Architecture (HTTP protocol) 649 Description 650 A caching proxy mesh where a request may travel different paths, 651 depending on the state of the mesh and associated caches, can 652 break HTTP content serialization, possibly causing the end user 653 to receive older content than seen on an earlier request, where 654 the request traversed another path in the mesh. 656 Significance 657 Medium 659 Implications 660 Can cause end user confusion. May in some situations (sibling 661 cache hit, object has changed state from cacheable to 662 uncacheable) be close to impossible to get the caches properly 663 updated with the new content. 665 Indications 666 Older content is unexpectedly returned from a caching proxy mesh 667 after some time. 669 Solutions(s) 670 Work with caching proxy vendors and researchers to find a 671 suitable protocol for maintaining proxy relations and object 672 state in a mesh. 674 Workaround 675 When designing a hierarchy/mesh, make sure that for each 676 end-user/URL combination there is only one single path in the 677 mesh during normal operation. 679 Contact 680 Henrik Nordstrom 682 2.3 Known Implementation Problems 684 2.3.1 User agent/proxy failover 686 Name 687 User agent/proxy failover 689 Classification 690 Implementation 692 Description 693 Failover between proxies at the user agent (using a proxy.pac[6] 694 file) is erratic and no standard behavior is defined. 695 Additionally, behavior is hard-coded into the browser, so that 696 proxy administrators cannot use failover at the user agent 697 effectively. 699 Significance 700 Medium 702 Implications 703 Architects are forced to implement failover at the proxy itself, 704 when it may be more appropriate and economical to do it within 705 the user agent. 707 Indications 708 If a browser detects that its primary proxy is down, it will wait 709 n minutes before trying the next one it is configured to use. It 710 will then wait y minutes before asking the user if they'd like to 711 try the original proxy again. This is very confusing for end 712 users. 714 Solution(s) 715 Work with browser vendors to establish standard extensions to 716 JavaScript proxy.pac libraries that will allow configuration of 717 these timeouts. 719 Workaround 720 User education; redundancy at the proxy level. 722 Contact 723 Mark Nottingham 725 2.3.2 Some servers send bad Content-Length headers for files that 726 contain CR 728 Name 729 Some servers send bad Content-Length headers for files that 730 contain CR 732 Classification 733 Implementation 735 Description 736 Certain web servers send a Content-length value that is larger 737 than number of bytes in the HTTP message body. This happens when 738 the server strips off CR characters from text files with lines 739 terminated with CRLF as the file is written to the client. The 740 server probably uses the stat() system call to get the file size 741 for the Content-Length header. Servers that exhibit this behavior 742 include the GN Web server[9] (version 2.14 at least). 744 Significance 745 Low. Surveys indicate only a small number of sites run faulty 746 servers. 748 Implications 749 In this case, an HTTP client (e.g. user agent or proxy) may 750 believe it received a partial response. HTTP/1.1[3] advises that 751 caches MAY store partial responses. 753 Indications 754 Count the number of bytes in the message body and comparing it to 755 the Content-length value. If they differ the server exhibits this 756 problem. 758 Solutions 759 Upgrade or replace the buggy server. 761 Workaround 762 Some browsers and proxies use one TCP connection per object and 763 ignore the Content-Length. The document end of file is identified 764 by the close of the TCP socket. 766 Contact 767 Duane Wessels 769 3. Security Considerations 771 This memo does not raise security considerations in itself. See the 772 individual submissions for details of security concerns and issues. 774 References 776 [1] Paxson, V., Allman, M., Dawson, S., Fenner, W., Griner, J., 777 Heavens, I., Lahey, K., Semke, J. and B. Volz, "Known TCP 778 Implementation Problems", RFC 2525, March 1999, 779 . 781 [2] Cooper, I., Melve, I. and G. Tomlinson, "Internet Web 782 Replication and Caching Taxonomy", 783 draft-ietf-wrec-taxonomy-05.txt (work in progress), July 2000, 784 . 787 [3] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., 788 Leach, P. and T. Berners-Lee, "Hypertext Transfer Protocol -- 789 HTTP/1.1", RFC 2616, June 1999, 790 . 792 [4] Mogul, J., Krishnamurthy, B., Douglis, F., Feldmann, A., 793 Goland, Y., van Hoff, A. and D. Hellerstein, "HTTP Delta in 794 HTTP", draft-mogul-http-delta-07.txt (work in progress), 795 October 2000, 796 . 799 [5] Wessels, D. and K. Claffy, "Internet Cache Protocol (ICP), 800 Version 2", RFC 2186, September 1997, 801 . 803 [6] Netscape, Inc., "Navigator Proxy Auto-Config File Format", 804 March 1996, 805 . 808 [7] Davison, B., "Web Traffic Logs: An Imperfect Resource for 809 Evaluation", July 1999, 810 . 812 [8] 815 [9] 817 [10] 819 [11] 821 [12] 823 [13] 825 [14] 827 [15] 829 Authors' Addresses 831 Ian Cooper 832 Equinix 834 EMail: icooper@equinix.com 836 John Dilley 837 Akamai Technologies, Inc. 838 1400 Fashion Island Blvd 839 Suite 703 840 San Mateo, CA 94404 841 USA 843 Phone: +1 650 627-5244 844 EMail: jad@akamai.com 846 Appendix A. Archived Known Problems 848 The following sub-sections are an archive of problems identified in 849 the initial production of this memo. These are typically problems 850 requiring further work/research or user education. They are 851 included here for reference purposes only. 853 A.1 Architectural 855 A.1.1 Cannot specify multiple URIs for replicated resources 857 Name 858 Cannot specify multiple URIs for replicated resources 860 Classification 861 Architecture 863 Description 864 There is no way to specify that multiple URIs may be used for a 865 single resource, one for each replica of the resource. Similarly, 866 there is no way to say that some set of proxies (each identified 867 by a URI) may be used to resolve a URI. 869 Significance 870 Medium 872 Implications 873 Forces users to understand the replication model and mechanism. 874 Makes it difficult to create a replication framework without 875 protocol support for replication and naming. 877 Indications 878 Inherent in HTTP 1.0, HTTP 1.1. 880 Solution(s) 881 Architectural - protocol design is necessary. 883 Workaround 884 Replication mechanisms force users to locate a replica or mirror 885 site for replicated content. 887 Contact 888 Daniel LaLiberte 890 A.1.2 Replica distance is unknown 892 Name 893 Replica distance is unknown 895 Classification 896 Architecture 898 Description 899 There is no recommended way to find out which of several servers 900 or proxies is closer either to the requesting client or to 901 another machine, either geographically or in the network topology. 903 Significance 904 Medium 906 Implications 907 Clients must guess which replica is closer to them when 908 requesting a copy of a document that may be served from multiple 909 locations. Users must know the set of servers that can serve a 910 particular object. This in general is hard to determine and 911 maintain. Users must understand network topology in order to 912 choose the closest copy. Note that the closest copy is not always 913 the one that will result in quickest service. A nearby but 914 heavily loaded server may be slower than a more distant but 915 lightly loaded server. 917 Indications 918 Inherent in HTTP 1.0, HTTP 1.1. 920 Solution(s) 921 Architectural - protocol work is necessary. This is a specific 922 instance of a general problem in widely distributed systems. A 923 general solution is unlikely, however a specific solution in the 924 web context is possible. 926 Workaround 927 Servers can (many do) provide location hints in a replica 928 selection web page. Users choose one based upon their location. 929 Users can learn which replica server gives them best performance. 930 Note that the closest replica geographically is not necessarily 931 the closest in terms of network topology. Expecting users to 932 understand network topology is unreasonable. 934 Contact 935 Daniel LaLiberte 937 A.1.3 Proxy resource location 939 Name 940 Proxy resource location 942 Classification 943 Architecture 945 Description 946 There is no way for a client or server (including another proxy) 947 to inform a proxy of an alternate address (perhaps including the 948 proxy to use to reach that address) to use to fetch a resource. 949 If the client does not trust where the redirected resource came 950 from, it may need to validate it or validate where it came from. 952 Significance 953 Medium 955 Implications 956 Proxies have no systematic way to locate resources within other 957 proxies or origin servers. This makes it more difficult to share 958 information among proxies. Information sharing would improve 959 global efficiency. 961 Indications 962 Inherent in HTTP 1.0, HTTP 1.1. 964 Solution(s) 965 Architectural - protocol design is necessary. 967 Workaround 968 Certain proxies share location hints in the form of summary 969 digests of their contents (e.g., Squid). Certain proxy protocols 970 enable a proxy query another for its contents (e.g., ICP). (See 971 however "ICP Performance" issue (Section 2.2.5).) 973 Contact 974 Daniel LaLiberte 976 A.2 Implementation 978 A.2.1 Use of Cache-Control headers 980 Name 981 Use of Cache-control headers 983 Classification 984 Implementation 986 Description 987 Many (if not most) implementations incorrectly interpret 988 Cache-control response headers. 990 Significance 991 High 993 Implications 994 Cache-control headers will be spurned by end users if there are 995 conflicting or non-standard implementations. 997 Indications 998 - 1000 Solution(s) 1001 Work with vendors and others to assure proper application 1003 Workaround 1004 None 1006 Contact 1007 Mark Nottingham 1009 A.2.2 Lack of HTTP/1.1 compliance for caching proxies 1011 Name 1012 Lack of HTTP/1.1 compliance for caching proxies 1014 Classification 1015 Implementation 1017 Description 1018 Although performance benchmarking of caches is starting to be 1019 explored, protocol compliance is just as important. 1021 Significance 1022 High 1024 Implications 1025 Caching proxy vendors implement their interpretation of the 1026 specification; because the specification is very large, sometimes 1027 vague and ambiguous, this can lead to inconsistent behavior 1028 between caching proxies. 1030 Caching proxies need to comply to the specification (or the 1031 specification needs to change). 1033 Indications 1034 There is no currently known compliance test being used. 1036 There is work underway to quantify how closely servers comply 1037 with the current specification. A joint technical report between 1038 AT&T (#990803-05-TM, available at 1039 [10] 1040 ) and HP Labs (to be published) describes the compliance testing. 1041 This report examines how well each of a set of top 1042 traffic-producing sites support certain HTTP/1.1 features. 1044 The Measurement Factory (formerly IRCache) is working to develop 1045 protocol compliance testing software. Running such a conformance 1046 test suite against proxy cache products would measure compliance 1047 and ultimately would help assure they comply to the specification. 1049 Solution(s) 1050 Testing should commence and be reported in an open industry 1051 forum. Proxy implementations should conform to the specification. 1053 Workaround 1054 There is no workaround for non-compliance. 1056 Contact 1057 Mark Nottingham 1058 Duane Wessels 1060 A.2.3 ETag support 1062 Name 1063 ETag support 1065 Classification 1066 Implementation 1068 Description 1069 Available caching proxies appear not to support ETag (strong) 1070 validation. 1072 Significance 1073 Medium 1075 Implications 1076 Last-Modified/If-Modified-Since validation is inappropriate for 1077 many requirements, both because of its weakness and its use of 1078 dates. Lack of a usable, strong coherency protocol leads 1079 developers and end users not to trust caches. 1081 Indications 1082 - 1084 Solution(s) 1085 Work with vendors to implement ETags; work for better validation 1086 protocols. 1088 Workaround 1089 Use Last-Modified/If-Modified-Since validation. 1091 Contact 1092 Mark Nottingham 1094 A.2.4 Servers and content should be optimized for caching 1096 Name 1097 Servers and content should be optimized for caching 1099 Classification 1100 Implementation (Performance) 1102 Description 1103 Many web servers and much web content could be implemented to be 1104 more conducive to caching, reducing bandwidth demand and page 1105 load delay. 1107 Significance 1108 Medium 1110 Implications 1111 By making poor use of caches, origin servers encourage longer 1112 load times, greater load on caching proxies, and increased 1113 network demand. 1115 Indications 1116 The problem is most apparent for pages that have low or zero 1117 expires time, yet do not change. 1119 Solution(s) 1120 - 1122 Workaround 1123 For example, servers could start using unique object identifiers 1124 for write-only content: if an object changes it gets a new name, 1125 otherwise it is considered to be immutable and therefore have an 1126 infinite expire age. Certain hosting providers do this already. 1128 Contact 1129 Peter Danzig 1131 A.3 Administration 1133 A.3.1 Lack of fine-grained, standardized hierarchy controls 1135 Name 1136 Lack of fine-grained, standardized hierarchy controls 1138 Classification 1139 Administration 1141 Description 1142 There is no standard for instructing a proxy as to how it should 1143 resolve what parent to fetch a given object from. Because of 1144 this, implementations vary greatly, and it can be difficult to 1145 make them interoperate correctly in a complex environment. 1147 Significance 1148 Medium 1150 Implications 1151 Complications in deployment of caches in a complex network 1152 (especially corporate networks) 1154 Indications 1155 Inability of some proxies to be configured to direct traffic 1156 based on domain name, reverse lookup IP address, raw IP address, 1157 in normal operation and in failover mode. Inability in some 1158 proxies to set a preferred parent / backup parent configuration. 1160 Solution(s) 1161 ? 1163 Workaround 1164 Work with vendors to establish an acceptable configuration within 1165 the limits of their product; standardize on one product. 1167 Contact 1168 Mark Nottingham 1170 A.3.2 Proxy/Server exhaustive log format standard for analysis 1172 Name 1173 Proxy/Server exhaustive log format standard for analysis 1175 Classification 1176 Administration 1178 Description 1179 Most proxy or origin server logs used for characterization or 1180 evaluation do not provide sufficient detail to determine 1181 cacheability of responses. 1183 Significance 1184 Low (for operationality; high significance for research efforts) 1186 Implications 1187 Characterizations and simulations are based on non-representative 1188 workloads. 1190 See Also 1191 W3C Web Characterization Activity 1192 [11] since they are also concerned 1193 with collecting high quality logs and building characterizations 1194 from them. 1196 Indications 1197 - 1199 Solution(s) 1200 To properly clean and to accurately determine cacheability of 1201 responses, a complete log is required (including all request 1202 headers as well as all response headers such as "User-agent" [for 1203 removal of spiders] and "Expires", "max-age", "Set-cookie", 1204 "no-cache", etc.) 1206 Workaround 1207 - 1209 References 1210 See "Web Traffic Logs: An Imperfect Resource for Evaluation" in 1211 INET99 [7] 1212 for some discussion of this. 1214 Contact 1215 Brian D. Davison 1216 Terence Kelly 1218 A.3.3 Trace log timestamps 1220 Name 1221 Trace log timestamps 1223 Classification 1224 Administration 1226 Description 1227 Some proxies/servers log requests without sufficient timing 1228 detail. Millisecond resolution is often too small to preserve 1229 request ordering and either the servers should record request 1230 reception time in addition to completion time, or elapsed time 1231 plus either one. 1233 Significance 1234 Low (for operationality; medium significance for research efforts) 1236 Implications 1237 Characterization and simulation fidelity is improved with 1238 accurate timing and ordering information. Since logs are 1239 generally written in order of request completion, these logs 1240 cannot be re-played without knowing request generation times and 1241 reordering accordingly. 1243 See Also 1244 - 1246 Indications 1247 Timestamps can be identical for multiple entries (when only 1248 millisecond resolution is used). Request orderings can be jumbled 1249 when clients open additional connections for embedded objects 1250 while still receiving the container object. 1252 Solution(s) 1253 Since request completion time is common (e.g. Squid), recommend 1254 continuing to use it (with microsecond resolution if possible) 1255 plus recording elapsed time since request reception. 1257 Workaround 1258 - 1260 References 1261 See "Web Traffic Logs: An Imperfect Resource for Evaluation" in 1262 INET99 [7] 1263 for some discussion of this. 1265 Contact 1266 Brian D. Davison 1268 A.3.4 Exchange format for log summaries 1270 Name 1271 Exchange format for log summaries 1273 Classification 1274 Administration/Analysis? 1276 Description 1277 Although we have (more or less) a standard log file format for 1278 proxies (plain vanilla Common Logfile and Squid), there isn't a 1279 commonly accepted format for summaries of those log files. 1280 Summaries could be generated by the cache itself, or by 1281 post-processing existing log file formats such as Squid's. 1283 Significance 1284 High, since it means that each log file summarizing/analysis tool 1285 is essentially reinventing the wheel (un-necessary repetition of 1286 code), and the cost of processing a large number of large log 1287 files through a variety of analysis tools is (again for no good 1288 reason) excessive. 1290 Implications 1291 In order to perform a meaningful analysis (e.g. to measure 1292 performance in relation to loading/configuration over time) the 1293 access logs from multiple busy caches, it's often necessary to 1294 run first one tool then another, each against the entire log file 1295 (or a significantly large subset of the log). With log files 1296 running into hundreds of MB even after compression (for a cache 1297 dealing with millions of transactions per day) this is a 1298 non-trivial task. 1300 See Also 1301 IP packet/header sniffing - it may be that individual 1302 transactions are at a level of granularity which simply isn't 1303 sensible to be attempting on extremely busy caches. There may 1304 also be legal implications in some countries, e.g. if this 1305 analysis identifies individuals. 1307 Indications 1308 Disks/memory full(!) Stats (using multiple programs) take too 1309 long to run. Stats crunching must be distributed out to multiple 1310 machines because of its high computational cost. 1312 Solution(s) 1313 Have the proxy produce a standardized summary of its activity 1314 either automatically or via an external (e.g. third party) tool, 1315 in a commonly agreed format. The format could be something like 1316 XML or the Extended Common Logfile, but the format and contents 1317 are subjects for discussion. Ideally this approach would permit 1318 individual cache server products to supply subsets of the 1319 possible summary info, since it may not be feasible for all 1320 servers to provide all of the information which people would like 1321 to see. 1323 Workaround 1324 Devise a private summary format for your own personal use - but 1325 this complicates or even precludes the exchange of summary info 1326 with other interested parties. 1328 References 1329 See the web pages for the commonly used cache stats analysis 1330 programs, e.g. Calamaris[12], squidtimes[13], squidclients[14], 1331 etc.[15] 1333 Contact 1334 Martin Hamilton 1336 Full Copyright Statement 1338 Copyright (C) The Internet Society (2000). All Rights Reserved. 1340 This document and translations of it may be copied and furnished to 1341 others, and derivative works that comment on or otherwise explain it 1342 or assist in its implementation may be prepared, copied, published 1343 and distributed, in whole or in part, without restriction of any 1344 kind, provided that the above copyright notice and this paragraph 1345 are included on all such copies and derivative works. However, this 1346 document itself may not be modified in any way, such as by removing 1347 the copyright notice or references to the Internet Society or other 1348 Internet organizations, except as needed for the purpose of 1349 developing Internet standards in which case the procedures for 1350 copyrights defined in the Internet Standards process must be 1351 followed, or as required to translate it into languages other than 1352 English. 1354 The limited permissions granted above are perpetual and will not be 1355 revoked by the Internet Society or its successors or assigns. 1357 This document and the information contained herein is provided on an 1358 "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING 1359 TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING 1360 BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION 1361 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF 1362 MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 1364 Acknowledgement 1366 Funding for the RFC editor function is currently provided by the 1367 Internet Society.