Network Working Group I. Cooper Internet-Draft Mirror Image Expires: January 11, 2001 J. Dilley Akamai July 13, 2000 Known HTTP Proxy/Caching Problems draft-ietf-wrec-known-prob-02.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on January 11, 2001. Copyright Notice Copyright (C) The Internet Society (2000). All Rights Reserved. Abstract This memo catalogs a number of known problems with World Wide Web (caching) proxies and cache servers. The goal of the document is to provide a discussion of the problems and proposed workarounds, and ultimately to improve conditions by illustrating problems. The construction of this document is a joint effort of the web caching community. It is being done under the auspices of the IETF Web Replication and Caching working group. We gratefully acknowledge RFC2525[1], which helped define the initial format for this known problems list. Cooper & Dilley Expires January 11, 2001 [Page 1] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 Problem Template . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Document Template . . . . . . . . . . . . . . . . . . . . . 5 2. Known Problems . . . . . . . . . . . . . . . . . . . . . . . 6 2.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . 6 2.1.1 Interception proxies break client cache directives . . . . . 6 2.1.2 Interception proxies prevent introduction of new HTTP methods . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1.3 Cannot specify multiple URIs for replicated resources . . . 8 2.1.4 Replica distance is unknown . . . . . . . . . . . . . . . . 8 2.1.5 Proxy resource location . . . . . . . . . . . . . . . . . . 9 2.1.6 Caching proxy peer selection in heterogeneous networks . . . 10 2.1.7 ICP performance . . . . . . . . . . . . . . . . . . . . . . 11 2.1.8 Caching proxy meshes can break HTTP serialization of content 12 2.1.9 Interception proxies break IP address-based authentication . 13 2.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . 14 2.2.1 Use of Cache-Control headers . . . . . . . . . . . . . . . . 14 2.2.2 Lack of HTTP/1.1 compliance for caching proxies . . . . . . 15 2.2.3 ETag support . . . . . . . . . . . . . . . . . . . . . . . . 16 2.2.4 User agent/proxy failover . . . . . . . . . . . . . . . . . 17 2.2.5 Servers and content should be optimized for caching . . . . 17 2.2.6 Some servers send bad Content-Length header files that contain CR . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.3 Administration . . . . . . . . . . . . . . . . . . . . . . . 19 2.3.1 Lack of fine-grained, standardized hierarchy controls . . . 19 2.3.2 Proxy/Server exhaustive log format standard for analysis . . 20 2.3.3 Trace log timestamps . . . . . . . . . . . . . . . . . . . . 21 2.3.4 Exchange format for log summaries . . . . . . . . . . . . . 22 3. Security Considerations . . . . . . . . . . . . . . . . . . 24 References . . . . . . . . . . . . . . . . . . . . . . . . . 25 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . 25 A. Text Template . . . . . . . . . . . . . . . . . . . . . . . 26 B. RFC2629 XML Template . . . . . . . . . . . . . . . . . . . . 27 Full Copyright Statement . . . . . . . . . . . . . . . . . . 28 Cooper & Dilley Expires January 11, 2001 [Page 2] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 1. Introduction This memo discusses problems with both proxies, which act as application-level intermediaries for Web requests, and caching proxies which additionally hold copies of previously requested resources in the hope of improving overall quality of service by serving the content more locally. Commonly used terminology in this memo can be found in the companion document "Internet Web Replication and Caching Taxonomy"[2]. No individual or organization has complete knowledge of the known problems in web caching. If you know of a problem that is not documented on this list you are encouraged to send it to the WREC mailing list, for discussion or to the memo's editors, & , for review and inclusion in the list. 1.1 Problem Template Each problem is defined in a common format, summarized in the following table and described below. Name: short, descriptive name of the problem (3-5 words) Classification: classifies the problem: performance, security, etc Description: describes the problem succinctly Significance: magnitude of problem, environments where it exists Implications: the impact of the problem on systems and networks See Also: a reference to a related known problem Indications: states how to detect the presence of this problem Solution(s): describe the solution(s) to this problem, if any Workaround: practical workaround for the problem References: information about the problem or solution Contact: contact name and email address for this section Name A short, descriptive name (3-5 words) name associated with the problem. Classification Problems are grouped into categories of similar problems for ease of reading of this memo. Choose the category that best describes the problem. The suggested categories include three general categories and several more specific categories. * Architecture: the fundamental design is incomplete, or incorrect * Specification: the spec is ambiguous, incomplete, or incorrect. Cooper & Dilley Expires January 11, 2001 [Page 3] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 * Implementation: the implementation of the spec is incorrect. * Performance: perceived page response at the client is excessive; network bandwidth consumption is excessive; demand on origin or proxy servers exceed reasonable bounds. * Administration: care and feeding of caches is or causes a problem. * Security: privacy, integrity, or authentication concerns. This is the first draft of this memo. The classification structure is in revision. In the published drafts of the memo the classification structure should be fixed but may be revised from time to time. Description A definition of the problem, succinct but including necessary background information. Significance (High, Medium, Low) May include a brief summary of the environments for which the problem is significant. Implications Why the problem is viewed as a problem. What inappropriate behavior results from it? This section should substantiate the magnitude of any problem indicated with High significance. See Also Optional. List of other known problems that are related to this one. Indications How to detect the presence of the problem. This may include references to one or more substantiating documents that demonstrate the problem. This should include the network configuration that led to the problem such that it can be reproduced. Problems that are not reproduceable will not appear in this memo. Solution(s) Solutions that permanently fix the problem, if such are known. For example, what version of the software does not exhibit the problem? Indicate if the solution is accepted by the community, one of several solutions pending agreement, or open possibly with experimental solutions. Workaround Practical workaround if no solution is available or usable. The Cooper & Dilley Expires January 11, 2001 [Page 4] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 workaround should have sufficient detail for someone experiencing the problem to get around it. References References to related information in technical publications or on the web. Where can someone interested in learning more go to find out more about this problem, its solution, or workarounds? Contact Contact name and email address of the person who supplied the information for this section. If you would prefer to remain anonymous the editor's name will appear here instead, but we believe in credit where credit is due. 1.2 Document Template Text and RFC2629 style XML templates for the submission of known problems can be found in Appendix A and Appendix B. Cooper & Dilley Expires January 11, 2001 [Page 5] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 2. Known Problems The remaining sections present the currently documented known problems. The problems are ordered by classification and significance. Issues with protocol specification or architecture are first, followed by implementation issues. Issues of high significance are first, followed by lower significance. A full list of the problems is available in the table of contents. 2.1 Architecture 2.1.1 Interception proxies break client cache directives Name Interception proxies break client cache directives Classification Architecture Description HTTP[3] is designed for the user agent to be aware if it is connected to an origin server or to a proxy. User agents believing they are transacting with an origin server but which are really in a connection with an interception proxy may fail to send critical cache-control information they would have otherwise included in their request. Significance High Implications Clients may receive data that is not synchronized with the origin even when they request an end to end refresh because of the lack of inclusion of either a cache-control: no-cache or must-revalidate header. These headers have no impact on origin server behavior so may not be included by the browser if it believes it is connected to that resource. Other related data implications are possible as well. For instance data security may be compromised by the lack of inclusion of private or no-store clauses of the cache-control header under similar conditions. Indications Easily detected by placing fresh (un-expired) content on a proxy while changing the authoritative copy and requesting an end to end reload of the data through a proxy in both interception and explicit modes. Solution(s) Eliminate the need for interception proxies and IP spoofing which Cooper & Dilley Expires January 11, 2001 [Page 6] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 will return correct context awareness to the client. Workaround Include relevant cache-control: directives in every request at the cost of increased bandwidth and CPU requirements. Contact Patrick McManus 2.1.2 Interception proxies prevent introduction of new HTTP methods Name Interception proxies prevent introduction of new HTTP methods Classification Architecture Description A proxy that receives a request with a method unknown to it is required to generate an HTTP 501 Error as a response. HTTP methods are designed to be extensible so there may be applications deployed with initial support just for the user agent and origin server. An interception proxy that hijacks requests which include new methods destined for servers that have implemented those methods creates a de-facto firewall where none may be intended. Significance Medium within interception proxy environments. Implications Renders new compliant applications useless unless modifications are made to proxy software. Because new methods are not required to be globally standardized it is impossible to keep up to date in the general case. Solution(s) Eliminate the need for interception proxies. A client receiving a 501 in a traditional HTTP environment may either choose to repeat the request to the origin server directly, or perhaps be configured to use a different cache. Workaround Level 5 switches (sometimes called Level 7 or application layer switches) can be used to keep HTTP traffic with unknown methods out of the proxy. However, these devices have heavy buffering responsibilities, still require TCP sequence number spoofing, and do not interact well with persistent connections. Cooper & Dilley Expires January 11, 2001 [Page 7] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 The HTTP/1.1 specification allows a proxy to switch over to tunnel mode when it receives a request with a method or HTTP version it does not understand how to handle. Contact Patrick McManus Henrik Nordstrom (HTTP/1.1 clarification) 2.1.3 Cannot specify multiple URIs for replicated resources Name Cannot specify multiple URIs for replicated resources Classification Architecture Description There is no way to specify that multiple URIs may be used for a single resource, one for each replica of the resource. Similarly, there is no way to say that some set of proxies (each identified by a URI) may be used to resolve a URI. Significance Medium Implications Forces users to understand the replication model and mechanism. Makes it difficult to create a replication framework without protocol support for replication and naming. Indications Inherent in HTTP 1.0, HTTP 1.1. Solution(s) Architectural - protocol design is necessary. Workaround Replication mechanisms force users to locate a replica or mirror site for replicated content. Contact Daniel LaLiberte 2.1.4 Replica distance is unknown Name Replica distance is unknown Classification Cooper & Dilley Expires January 11, 2001 [Page 8] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 Architecture Description There is no recommended way to find out which of several servers or proxies is closer either to the requesting client or to another machine, either geographically or in the network topology. Significance Medium Implications Clients must guess which replica is closer to them when requesting a copy of a document that may be served from multiple locations. Users must know the set of servers that can serve a particular object. This in general is hard to determine and maintain. Users must understand network topology in order to choose the closest copy. Note that the closest copy is not always the one that will result in quickest service. A nearby but heavily loaded server may be slower than a more distant but lightly loaded server. Indications Inherent in HTTP 1.0, HTTP 1.1. Solution(s) Architectural - protocol work is necessary. This is a specific instance of a general problem in widely distributed systems. A general solution is unlikely, however a specific solution in the web context is possible. Workaround Servers can (many do) provide location hints in a replica selection web page. Users choose one based upon their location. Users can learn which replica server gives them best performance. Note that the closest replica geographically is not necessarily the closest in terms of network topology. Expecting users to understand network topology is unreasonable. Contact Daniel LaLiberte 2.1.5 Proxy resource location Name Proxy resource location Classification Architecture Cooper & Dilley Expires January 11, 2001 [Page 9] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 Description There is no way to tell a proxy that it may request a resource from an alternate location; the receiver may need to check the authenticity of the resource if redirected. Significance Medium Implications Proxies have no systematic way to locate resources within other proxies or origin servers. This makes it more difficult to share information among proxies. Information sharing would improve global efficiency. Indications Inherent in HTTP 1.0, HTTP 1.1. Solution(s) Architectural - protocol design is necessary. Workaround Certain proxies share location hints in the form of summary digests of their contents (e.g., Squid). Certain proxy protocols enable a proxy query another for its contents (e.g., ICP). (See however "ICP Performance" issue.) Contact Daniel LaLiberte 2.1.6 Caching proxy peer selection in heterogeneous networks Name Caching proxy peer selection in heterogeneous networks Classification Architecture Description Caching proxy peer selection in networks with large variance in latency and bandwidth between peers can lead to non-optimal peer selection. For example take Proxy C with two siblings, Sib1 and Sib2, and the following network topology (summarized). * Cache C's link to Sib1, 2 Mbit/sec with 300 msec latency * Cache C's link to Sib2, 64 Kbit/sec with 10 msec latency. ICP does not work well in this context. If a user submits a Cooper & Dilley Expires January 11, 2001 [Page 10] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 request to Proxy C for page P that results in a miss. C will send an ICP request to Sib1 and Sib2. Assume both siblings have the requested object P. The ICP-HIT reply will always come from Sib2 before Sib1. However, for large objects it is clear that the retrieval will be faster from Sib1 rather than Sib2. In fact, the problem is more complex because Sib1 and Sib2 can't have a 100% hit ratio. With a hit rate of 10%, it is more efficient to use Sib1 with resources larger than 48K. The best choice depends on at least the hit rate and link characteristics; maybe other parameters as well. Significance Medium Implications By selecting the first peer to respond, peer selection algorithms are not optimizing retrieval latency to end users. Furthermore they are causing more work for the high-latency peer since it must respond to such requests but will never be chosen to serve content if the lower latency peer has a copy. Indications Inherent in design of ICP v1, ICP v2, and any proxy mesh protocol that selects peer based upon first response. This problem is not exhibited by cache digest or other protocols which (attempt to) maintain knowledge of peer contents and only hit peers that are believed to have a copy of the requested page. Solution(s) This problem is architectural with the peer selection protocol. Workaround Proxy mesh design when using such a protocol should be done in such a way that there is not a high latency variance among peers. In the example presented in the above description the high latency high bandwidth peer could be used as a parent, but should not be used as a sibling. Contact Ivan LOVRIC John Dilley 2.1.7 ICP performance Name ICP performance Cooper & Dilley Expires January 11, 2001 [Page 11] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 Classification Architecture(ICP), Performance Description The ICP protocol exhibits O(n^2) scaling properties, where n is the number of peer proxies participating in the protocol. This can lead ICP traffic to dominate HTTP traffic within a network. Significance Medium Implications If a proxy has many ICP peers the bandwidth demand of ICP can be excessive. System managers must carefully regulate ICP peering. ICP also leads proxies to become heterogeneous in what they serve. This means if your proxy does not have a document it is unlikely your peers will have it either. Therefore, ICP traffic requests are largely unable to locate a local copy of an object [credit to Ingrid Melve's 3WCW talk for this]. Indications Inherent in design of ICP v1, ICP v2. Solution(s) This problem is architectural - protocol redesign or replacement are required to solve it if ICP is to continue to be used. Workaround Implementation workarounds exist, for example to turn off use of ICP, to carefully regulate peering, or to use another mechanism if available, such as cache digests. A cache digest protocol shares a summary of cache contents using a Bloom Filter technique. This allows a cache to estimate whether a peer has a document. Filters are updated regularly but are not always up-to-date so cannot help when a spike in popularity occurs. They also increase traffic but not as much as ICP. Proxy clustering protocols organize proxies into a mesh provide another alternative solution. There is ongoing research on this topic. Contact John Dilley 2.1.8 Caching proxy meshes can break HTTP serialization of content Name Caching proxy meshes can break HTTP serialization of content Cooper & Dilley Expires January 11, 2001 [Page 12] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 Classification Architecture (HTTP protocol) Description A caching proxy mesh where a request may travel different paths depending on the state of the mesh and associated caches can break HTTP content serialization, possibly causing the end user to receive older content than seen on an earlier request where the request traveled another path in the mesh. Significance Medium Implications Can cause end user confusion. May in some situations (sibling cache hit, object has changed state from cacheable to uncacheable) be close to impossible to get the caches properly updated with the new content. Indications Older content is unexpectedly returned from a caching proxy mesh after some time. Solutions(s) Work with caching proxy vendors and researchers to find a suitable protocol for maintaining proxy relations and object state in a mesh. Workaround When designing a hierarchy/mesh, make sure that for each end-user/URL combination there is only one single path in the mesh during normal operation. Contact Henrik Nordstrom 2.1.9 Interception proxies break IP address-based authentication Name Interception proxies break IP address-based authentication Classification Architecture Description Some web servers are not open for public access, but restrict themselves to accept only certain IP address ranges for security reasons. Using interception proxies at the ISP level, for example, will alter the source (client) IP addresses to that of Cooper & Dilley Expires January 11, 2001 [Page 13] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 the proxy itself. This will break such authentication mechanisms and prohibit the otherwise allowed clients access to the servers. Significance Medium Implications This creates end user confusion and frustration. Indications Users may start to see refused connections to servers after interception proxies are deployed. Solution(s) Use user-based authentication instead of (IP) address-based authentication. Workaround By using IP filters at the intercepting device (L4 switch) and bypass all requests to such servers concerned. Contact Keith K. Chau 2.2 Implementation 2.2.1 Use of Cache-Control headers Name Use of Cache-Control headers Classification Implementation Description Many (if not most) implementations incorrectly interpret Cache-Control response headers. Significance High Implications CC headers will be spurned by end users if there are conflicting or non-standard implementations. Indications Check: Squid, NetCache, Cache Engine, HTTP State Management draft for use of CC: no-cache and must-revalidate against HTTP/1.1rev6. Cooper & Dilley Expires January 11, 2001 [Page 14] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 Solution(s) Work with vendors and others to assure proper application Workaround None Contact Mark Nottingham 2.2.2 Lack of HTTP/1.1 compliance for caching proxies Name Lack of HTTP/1.1 compliance for caching proxies Classification Implementation Description Although performance benchmarking of caches is starting to be explored, protocol compliance is just as important. Significance High Implications Caching proxy vendors implement their interpretation of the specification; because the specification is very large, sometimes vague and ambiguous, this can lead to inconsistent behavior between caching proxies. Caching proxies need to comply to the specification (or the specification needs to change). Indications There is no currently known compliance test being used. There is work underway to quantify how closely servers comply with the current specification. A joint technical report between AT&T (#990803-05-TM, available at and HP Labs (to be published) describes the compliance testing. This report examines how well each of a set of top traffic-producing sites support certain HTTP/1.1 features. The IRCache group is working to develop protocol compliance testing software. Running such a conformance test suite against proxy cache products would measure compliance and ultimately would help assure they comply to the specification. Cooper & Dilley Expires January 11, 2001 [Page 15] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 Solution(s) Testing should commence and be reported in an open industry forum. Proxy implementations should conform to the specification. Workaround There is no workaround for non-compliance. Contact Mark Nottingham IRCache: Duane Wessels Glenn Chisholm 2.2.3 ETag support Name ETag support Classification Implementation Description No currently released caching proxy implements ETag (strong) validation. Significance Medium Implications LM/IMS validation is inappropriate for many requirements, both because of its weakness and its use of dates. Lack of a usable, strong coherency protocol leads developers and end users not to trust caches. Indications - Solution(s) Work with vendors to implement ETags; work for better validation protocols. Workaround use LM/IMS validation. Contact Mark Nottingham Cooper & Dilley Expires January 11, 2001 [Page 16] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 2.2.4 User agent/proxy failover Name User agent/proxy failover Classification Implementation Description Failover between proxies at the user agent (using a proxy.pac file) is erratic and no standard behavior is defined. Additionally, behavior is hard-coded into the browser, so that proxy administrators cannot use failover at the user agent effectively. Significance Medium Implications Architects are forced to implement failover at the proxy itself, when it may be more appropriate and economical to do it within the user agent. Indications If a browser detects that its primary proxy is down, it will wait n minutes before trying the next one it is configured to use. It will then wait y minutes before asking the user if they'd like to try the original proxy again. This is very confusing for end users. Solution(s) Work with browser vendors to establish standard extensions to JavaScript proxy.pac libraries that will allow configuration of these timeouts. Workaround User education; redundancy at the proxy level. Contact Mark Nottingham 2.2.5 Servers and content should be optimized for caching Name Servers and content should be optimized for caching Classification Implementation (Performance) Cooper & Dilley Expires January 11, 2001 [Page 17] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 Description Many web servers and much web content could be implemented to be more conducive to caching, reducing bandwidth demand and page load delay. Significance Medium Implications By making poor use of caches, origin servers encourage longer load times, greater load on cache servers, and increased network demand. Indications The problem is most apparent for pages that have low or zero expires time, yet do not change. Solution(s) ... Workaround For example servers could start using unique object identifiers for write-only content: if an object changes it gets a new name, otherwise is is considered to be immutable and therefore have an infinite expire age. Certain hosting providers do this already. Contact Peter Danzig 2.2.6 Some servers send bad Content-Length header files that contain CR Name Some servers send bad Content-Length header files that contain CR. Classification Implementation Description Certain web servers send a Content-length value that is larger than number of bytes in the HTTP message body. This happens when the server strips off CR characters from text files with lines terminated with CRLF as the file is written to the client. The server probably uses the stat() system call to get the file size for the Content-Length header. Servers that exhibit this behavior include the GN Web server (version 2.14 at least) (http://gopher.unicom.com/gn-info/). Significance Low. Surveys indicate only a small number of sites run faulty Cooper & Dilley Expires January 11, 2001 [Page 18] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 servers. Implications In this case, an HTTP client (e.g. user agent or proxy) may believe it received a partial response. HTTP/1.1 (RFC 2616) advises that caches MAY store partial responses. Indications Count the number of bytes in the message body and comparing it to the Content-length value. If they differ the server exhibits this problem. Solutions Upgrade or replace the buggy server. Workaround Some browsers and proxies use one TCP connection per object and ignore the Content-Length. The document end of file is identified by the close of the TCP socket. Contact Duane Wessels 2.3 Administration 2.3.1 Lack of fine-grained, standardized hierarchy controls Name Lack of fine-grained, standardized hierarchy controls Classification Administration Description There is no standard for instructing a proxy as to how it should resolve what parent to fetch a given object from. Because of this, implementations vary greatly, and it can be difficult to make them interoperate correctly in a complex environment. Significance Medium Implications Complications in deployment of caches in a complex network (esp. corporate networks) Indications Inability of some proxies to be configured to direct traffic based on domain name, reverse lookup IP address, raw IP address, Cooper & Dilley Expires January 11, 2001 [Page 19] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 in normal operation and in failover mode. Inability in some proxies to set a preferred parent / backup parent configuration. Solution(s) ? Workaround Work with vendors to establish an acceptable configuration within the limits of their product; standardize on one product. Contact Mark Nottingham 2.3.2 Proxy/Server exhaustive log format standard for analysis Name Proxy/Server exhaustive log format standard for analysis Classification Administration Description Most proxy or origin server logs used for characterization or evaluation do not provide sufficient detail to determine cacheability of responses. Significance Low (for operationality; high significance for research efforts) Implications Characterizations and simulations are based on non-representative workloads. See Also W3C Web Characterization Activity (http://www.w3.org/WCA/) since they are are also concerned with collecting high quality logs and building characterizations from them. Indications - Solution(s) To properly clean and to accurately determine cacheability of responses, a complete log is required (including all request headers as well as all response headers such as User-agent [for removal of spiders] and Expires, max-age, set-cookie, no-cache, etc.) Workaround Cooper & Dilley Expires January 11, 2001 [Page 20] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 - References See "Web Traffic Logs: An Imperfect Resource for Evaluation" in INET99 for some discussion of this. Contact Brian D. Davison Terence Kelly 2.3.3 Trace log timestamps Name Trace log timestamps Classification Administration Description Some proxies/servers log requests without sufficient timing detail. Millisecond resolution is often too small to preserve request ordering and either the servers should record request reception time in addition to completion time, or elapsed time plus either one. Significance Low (for operationality; medium significance for research efforts) Implications Characterization and simulation fidelity is improved with accurate timing and ordering information. Since logs are generally written in order of request completion, these logs cannot be re-played without knowing request generation times and reordering accordingly. See Also - Indications Timestamps can be identical for multiple entries (when only millisecond resolution is used). Request orderings can be jumbled when clients open additional connections for embedded objects while still receiving the container object. Solution(s) Since request completion time is common (e.g. Squid), recommend continuing to use it (with microsecond resolution if possible) plus recording elapsed time since request reception. Cooper & Dilley Expires January 11, 2001 [Page 21] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 Workaround - References See "Web Traffic Logs: An Imperfect Resource for Evaluation" in INET99 for some discussion of this. Contact Brian D. Davison 2.3.4 Exchange format for log summaries Name Exchange format for log summaries Classification Administration/Analysis? Description Although we have (more or less) a standard log file format for proxies (plain vanilla Common Logfile and Squid), there isn't a commonly accepted format for summaries of those log files. Summaries could be generated by the cache itself, or by post-processing existing log file formats such as Squid's. Significance High, since it means that each log file summarizing/analysis tool is essentially reinventing the wheel (un-necessary repetition of code), and the cost of processing a large number of large log files through a variety of analysis tools is (again for no good reason) excessive. Implications In order to perform a meaningful analysis (e.g. to measure performance in relation to loading/configuration over time) the access logs from multiple busy caches, it's often necessary to run first one tool then another, each against the entire log file (or a significantly large subset of the log). With log files running into hundreds of MB even after compression (for a cache dealing with millions of transactions per day) this is a non-trivial task. See Also IP packet/header sniffing - it may be that individual transactions are at a level of granularity which simply isn't sensible to be attempting on extremely busy caches. There may also be legal implications in some countries, e.g. if this analysis identifies individuals. Cooper & Dilley Expires January 11, 2001 [Page 22] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 Indications Disks/memory full(!) Stats (using multiple programs) take too long to run. Stats crunching must be distributed out to multiple machines because of its high computational cost. Solution(s) Have the proxy produce a standardized summary of its activity either automatically or via an external (e.g. third party) tool, in a commonly agreed format. The format could be something like XML or the Extended Common Logfile, but the format and contents are subjects for discussion. Ideally this approach would permit individual cache server products to supply subsets of the possible summary info, since it may not be feasible for all servers to provide all of the information which people would like to see. Workaround Devise a private summary format for your own personal use - but this complicates or even precludes the exchange of summary info with other interested parties. References See the web pages for the commonly used cache stats analysis programs, e.g. Calamaris, squidtimes, squidclients, ... Contact Martin Hamilton Cooper & Dilley Expires January 11, 2001 [Page 23] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 3. Security Considerations See individual submissions within this memo. Cooper & Dilley Expires January 11, 2001 [Page 24] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 References [1] Paxson, V., Allman, M., Dawson, S., Fenner, W., Griner, J., Heavens, I., Lahey, K., Semke, J. and B. Volz, "Known TCP Implementation Problems", RFC 2525, March 1999, . [2] Cooper, I., Melve, I. and G. Tomlinson, "Internet Web Replication and Caching Taxonomy", draft-ietf-wrec-taxonomy-05.txt (work in progress), July 2000, . [3] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P. and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999, . Authors' Addresses Ian Cooper Mirror Image Internet, Inc. 49 Dragon Court Woburn, MA 01801 USA Phone: +1 781 376 1109 EMail: ian.cooper@mirror-image.com John Dilley Akamai Technologies, Inc. 1400 Fashion Island Blvd Suite 703 San Mateo, CA 94404 USA Phone: +1 650 627-5244 EMail: jad@akamai.com Cooper & Dilley Expires January 11, 2001 [Page 25] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 Appendix A. Text Template Name A short, descriptive name (3-5 words) name associated with the problem. Classification Architecture, Specification, Implementation, Performance, Administration, or Security Description A definition of the problem, succinct but including necessary background information. Significance High, Medium, or Low Implications Why the problem is viewed as a problem See Also Optional. List of other known problems that are related to this one. Indications How to detect the presence of the problem. Solution(s) Solutions that permanently fix the problem, if such are known. Workaround Practical workaround if no solution is available or usable. The workaround should have sufficient detail for someone experiencing the problem to get around it. References References to related information in technical publications or on the web. Contact Contact name and of the person who supplied the information for this section. If you would prefer to remain anonymous the editor's name will appear here instead, but we believe in credit where credit is due. Cooper & Dilley Expires January 11, 2001 [Page 26] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 Appendix B. RFC2629 XML Template
A short, descriptive name (3-5 words) name associated with the problem. Architecture, Specification, Implementation, Performance, Administration, or Security A definition of the problem, succinct but including necessary background information. High, Medium, or Low Why the problem is viewed as a problem Optional. List of other known problems that are related to this one. How to detect the presence of the problem. Solutions that permanently fix the problem, if such are known. Practical workaround if no solution is available or usable. The workaround should have sufficient detail for someone experiencing the problem to get around it. References to related information in technical publications or on the web. Contact name and of the person who supplied the information for this section. If you would prefer to remain anonymous the editor's name will appear here instead, but we believe in credit where credit is due.
Cooper & Dilley Expires January 11, 2001 [Page 27] Internet-Draft Known HTTP Proxy/Caching Problems July 2000 Full Copyright Statement Copyright (C) The Internet Society (2000). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC editor function is currently provided by the Internet Society. Cooper & Dilley Expires January 11, 2001 [Page 28]