idnits 2.17.1 draft-rousskov-icap-trailers-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Introduction section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) -- The draft header indicates that this document updates RFC3507, but the abstract doesn't seem to mention this, which it should. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year (Using the creation date from RFC3507, updated by this document, for RFC5378 checks: 2001-10-11) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (October 10, 2016) is 2756 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) -- Duplicate reference: RFC3507, mentioned in 'Errata', was also mentioned in 'RFC3507'. Summary: 3 errors (**), 0 flaws (~~), 1 warning (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Rousskov 3 Internet-Draft The Measurement Factory 4 Updates: 3507 (if approved) October 10, 2016 5 Intended status: Informational 6 Expires: April 13, 2017 8 ICAP Trailers 9 draft-rousskov-icap-trailers-01 11 Abstract 13 This document defines an ICAP trailer feature which allows ICAP 14 agents to reliably send message metadata after the message body. The 15 ICAP trailer is independent from the HTTP trailer that might also be 16 encapsulated in an ICAP message. ICAP changes defined here are 17 backward compatible and address a long-standing ICAP specification 18 errata entry. 20 Status of This Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at http://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on April 13, 2017. 37 Copyright Notice 39 Copyright (c) 2016 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (http://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 Table of Contents 54 1. Motivation and Design Choices . . . . . . . . . . . . . . . . 2 55 2. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 4 56 3. Overall Operation . . . . . . . . . . . . . . . . . . . . . . 4 57 4. Notations . . . . . . . . . . . . . . . . . . . . . . . . . . 5 58 5. Extended Use of the Allow Header Field . . . . . . . . . . . 5 59 6. Message Syntax . . . . . . . . . . . . . . . . . . . . . . . 6 60 7. Trailer Field Syntax . . . . . . . . . . . . . . . . . . . . 6 61 8. Client Requirements . . . . . . . . . . . . . . . . . . . . . 7 62 9. Server Requirements . . . . . . . . . . . . . . . . . . . . . 8 63 10. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 9 64 11. Security Considerations . . . . . . . . . . . . . . . . . . . 11 65 12. Normative References . . . . . . . . . . . . . . . . . . . . 12 66 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 12 68 1. Motivation and Design Choices 70 ICAP [RFC3507] specification says that the Trailer header field is 71 "defined in ICAP the same as [...] in HTTP". Unfortunately, that 72 phrase alone is not enough for trailer-related interoperability in 73 the ICAP context because of the following conflicting 74 interpretations, requirements, and needs: 76 o Both ICAP and HTTP message headers might contain a Trailer field. 78 o HTTP messages might contain HTTP trailers (that ICAP servers could 79 be interested in receiving or even sending). An HTTP trailer can 80 be present with or without an HTTP Trailer header field. 82 o ICAP agents need to distinguish an HTTP trailer from an ICAP 83 trailer. 85 o HTTP uses Chunked Transfer Coding [RFC7230] to transmit trailers. 86 The chunked coding is applied to the entire HTTP message body. 87 This choice places an HTTP trailer inside an HTTP message body. 89 o It is possible to interpret the ICAP specification as placing the 90 ICAP trailer either inside the HTTP message body or inside the 91 ICAP message body. 93 o ICAP does not chunk-encode ICAP message bodies. Instead, ICAP 94 message bodies contain a combination of zero, one, or two HTTP 95 headers possibly followed by a chunked-encoded HTTP message body. 97 o Chunked coding does not support multiple trailers: Chunked HTTP 98 messages always contain exactly one (but possibly empty) trailer 99 part. 101 o HTTP effectively restricts trailers usage to messages with bodies, 102 presumably because, without a body, the information in the trailer 103 can usually be placed in the message header. In ICAP context, it 104 is not obvious whether trailers ought to be restricted to messages 105 with HTTP bodies (embedded in ICAP bodies) or to messages with 106 ICAP bodies (that might only contain HTTP headers and no HTTP 107 body). 109 These problems led to a ban on ICAP trailers [Errata]. 111 Several designs were considered for introducing proper ICAP trailers 112 support: 114 1. Extend chunked coding to support multiple trailers (one for HTTP 115 and one for ICAP). This option was rejected because many ICAP 116 agents use existing HTTP-focused libraries to parse embedded HTTP 117 bodies. As anecdotal evidence related to the ICAP-only "ieof" 118 chunk extension support shows, it would be difficult to extend 119 those libraries to handle a complicated ICAP-only extension. 120 Also, this design would make it difficult to send an ICAP trailer 121 when processing large HTTP messages without bodies. 123 2. Embed ICAP trailer fields inside the chunked HTTP message body 124 trailer, using an ICAP-specific field name prefix (e.g., 125 "ICAP-"). This option was rejected because it would either allow 126 malicious HTTP messages to inject ICAP trailers or require ICAP 127 clients to hide conflicting HTTP trailer fields from the ICAP 128 server. This design also badly violates layering boundaries by 129 mixing HTTP- and ICAP-level information in the same protocol 130 structure. 132 3. Extend Encapsulated header with a "trailer" token. This option 133 was rejected because the Encapsulated header describes embedded 134 HTTP message parts and an ICAP trailer is not a part of any HTTP 135 message. In other words, ICAP trailers do not get encapsulated. 137 4. Clarify the ICAP Trailer semantics (and transfer mechanism) 138 without introducing any new trailer support-negotiation 139 mechanism. This option was rejected because trailers affect 140 message framing and many existing ICAP agent implementations 141 cannot parse any form of trailers. 143 5. Add a new trailer support-negotiation mechanism (e.g., "Allow: 144 trailers") and a new trailer presence-signaling mechanism (e.g., 145 Trailer2) while leaving the poorly defined Trailer header 146 semantics as is. This option was narrowly rejected because a new 147 trailer support-negotiation mechanism alone was deemed sufficient 148 to resolve conflicts between this specification and any 149 reasonable existing implementation of the poorly defined Trailer 150 semantics. 152 6. Add a new trailer support-negotiation mechanism and only clarify 153 Trailer header semantics (and transfer mechanism) upon successful 154 negotiation, while reusing the well-known Trailer header field 155 name as the trailer presence-signaling mechanism (requiring 156 successful support negotiation). This specification documents 157 this design. 159 2. Use Cases 161 Trailers allow an ICAP agent to transmit metadata after the message 162 body. Such delayed transmission is useful when the same information 163 was not available at the start of the message transmission. For 164 example: 166 o A client uses an ICAP trailer to relay the current HTTP/1.1 167 connection or HTTP/2 stream status after transmitting a large HTTP 168 message. A server uses that information to optimize message 169 analysis (e.g., skip or abort analysis of HTTP requests sent by 170 already disconnected HTTP clients). 172 o A server uses an ICAP trailer to relay audit information about 173 viruses present at the end of a large HTTP message. 175 o A server uses an ICAP trailer to relay prefetching information 176 about HTML parts referenced from a large HTTP message. 178 3. Overall Operation 180 This section informally describes the overall feature operation. 181 This description is deliberately imprecise and cannot be used to 182 build compliant implementations. The following sections contain 183 actual protocol requirements. 185 To announce feature support, ICAP agents exchange "Allow: trailers" 186 settings during an OPTIONS transaction. To send a trailer at the end 187 of a particular REQMOD or RESPMOD transaction, the agent first sends 188 both "Allow: trailers" and "Trailer" header fields. The Trailer 189 field lists header field names expected in the message trailer 190 section. After sending the entire ICAP message body, the agent sends 191 the trailer section (a.k.a. "trailer"). The trailer section is 192 syntactically equivalent to the ICAP message header section. The 193 trailer section does not have to contain any of the promised fields 194 and might even have no fields at all. 196 A trailer makes sense only in an ICAP message with a body. However, 197 a trailer could be sent if the ICAP message body encapsulates just 198 HTTP headers. 200 If a trailer is sent, its bytes are always the last bytes sent during 201 the entire ICAP transaction. Thus, a client never sends a trailer at 202 the end of Preview unless it sent the "ieof" chunk extension as well. 203 Similarly, a server never a sends trailer with a 100 (Continue) 204 control message. 206 4. Notations 208 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 209 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 210 document are to be interpreted as described in [RFC2119]. 211 Conformance criteria and error handling considerations are defined in 212 Section 2.5 of [RFC7230]. 214 This specification uses the Augmented Backus-Naur Form (ABNF) 215 notation of [RFC5234] with a list extension defined in Section 7 of 216 [RFC7230]. All syntax rules not explicitly defined in this 217 specification (e.g., header-field and CRLF) are defined in (or 218 included by reference from) [RFC7230]. 220 The "Allow/X" notation is defined in Section 5. 222 5. Extended Use of the Allow Header Field 224 The use of the Allow header field defined in sections 4.6 and 4.10.2 225 of [RFC3507] is extended while preserving its original syntax and 226 general semantics (i.e., a comma-separated list of tokens, each 227 identifying an ICAP feature supported by the sending agent): 229 Allow = 1#token 231 [RFC3507] defines Allow usage in OPTIONS requests and responses. 232 This specification extends Allow usage to other messages and adds 233 "trailers" to the list of possible Allow value tokens. 235 An agent MUST treat multiple Allow header fields as one Allow field 236 with a comma-separated list of individual field value tokens, 237 concatenated in the order of their appearance in the ICAP header. An 238 ICAP agent MUST ignore Allow header tokens it does not understand. 239 This document does not specify the significance of Allow tokens order 240 and impact of repeated tokens. 242 An agent MAY send an Allow header in any message. Such header 243 contains a list of ICAP features supported by the sending agent. For 244 example, a client could send an "Allow: trailers" in a REQMOD request 245 and receive an "Allow: 204, trailers, 206" header field in response. 246 The exact meaning of each Allow token in a context of its message is 247 defined by the corresponding feature specification. By sending 248 Allow, the agent indicates compliance with each listed feature's 249 specification but does not necessarily commit to offer or use any of 250 the listed features during future transactions. 252 To keep this specification succinct, we introduce "Allow/X" notation 253 to mean an ICAP Allow header field which value contains an "X" token, 254 possibly among other tokens. For example, "Allow/trailers" stands 255 for an Allow header field with a "trailers" token and possibly other 256 tokens. This notation covers multi-field Allow headers as well 257 because they are equivalent to a combined single-field Allow header. 259 The same approach to extending Allow header usage was successfully 260 applied to the ICAP 206 extension (XXX: reference our expired ICAP 261 icap-ext-partial-content draft). This specification is compatible 262 with the ICAP 206 extension. 264 6. Message Syntax 266 An ICAP message with a trailer is a concatenation of a regular ICAP 267 message and an trailer section. The trailer section syntax is 268 identical to the ICAP header syntax: 270 ICAP-message-without-trailer = 271 ICAP-message-with-trailer = ICAP-message-without-trailer 272 trailer-section 274 trailer-section = *( trailer-field CRLF ) 275 CRLF 276 trailer-field = header-field 278 Note that any trailer, even a trailer without fields, ends with CRLF. 279 That terminating sequence is essential for proper message framing on 280 persistent ICAP connections. 282 A sender MUST NOT generate a trailer section that contains a field 283 necessary for message framing (e.g., Encapsulated, Preview, and 284 Trailer), routing (e.g., Host), or authentication. 286 7. Trailer Field Syntax 288 The ICAP Trailer header value syntax is identical to the HTTP Trailer 289 syntax [RFC7230]: 291 Trailer = 1#field-name 293 A Trailer header field sender SHOULD enumerate the names of all 294 expected trailer fields. This a priori knowledge of trailer fields 295 might help the recipient with trailer processing (e.g., certain 296 message annotation actions could be delayed in anticipation of the 297 trailer section). However, enumerating all expected trailer fields 298 can be impractical or even impossible in some environments. A 299 Trailer sender MAY send a trailer section with a set of field names 300 that differs from the set of field names listed in the Trailer header 301 field. 303 This specification does not place any restriction on the order of 304 field names in the Trailer header field. Senders SHOULD NOT generate 305 duplicate names for the Trailer header field. 307 8. Client Requirements 309 A client compliant with this specification SHOULD send Allow/trailers 310 in each OPTIONS request. A non-authenticating server cannot be 311 expected to mark an Allow/trailers-sending client specially, but this 312 support announcement requirement is meant to minimize 313 interoperability problems associated with servers sending Allow/ 314 trailers in OPTIONS responses. Some servers that do not support 315 trailers might not be able to ignore Allow/trailers in OPTIONS 316 requests; therefore, a client SHOULD offer a configuration option or 317 other means of disabling sending Allow/trailers in OPTIONS requests. 319 An ICAP service sending Allow/trailers in OPTIONS response is called 320 a trailers-supporting service. That service designation, maintained 321 by the client, starts upon receiving the OPTIONS service response 322 carrying Allow/trailers and lasts until OPTIONS expiration or a new 323 OPTIONS response from that service. 325 A client compliant with this specification SHOULD send Allow/trailers 326 in each request to a trailers-supporting service. Doing so allows 327 the service to respond with a trailer (and is also necessary for 328 sending a client trailer, as detailed further below). 330 A client receiving both a Trailer header field and Allow/trailers in 331 the response MUST expect a trailer section in that response. In all 332 other cases, a client MUST use the usual trailer-free ICAP response 333 syntax. A client receiving a Trailer header field without Allow/ 334 trailers in a response MAY treat the response as syntactically 335 malformed and, regardless of this response treatment, MUST NOT reuse 336 the connection for any other messages (including pending pipelined 337 requests, if any). 339 A client MAY send a trailer in any request that satisfies all of 340 these conditions: 342 1. the request is sent to a trailers-supporting service; 344 2. the request has a body. 346 A client MUST NOT send a trailer in any other request. 348 To send a trailer, the client MUST send Allow/trailers and a Trailer 349 header field in the same request. A client MUST NOT send a 350 combination of those two header fields without sending a trailer. 352 9. Server Requirements 354 A server compliant with this specification SHOULD send Allow/trailers 355 in each successful response to an OPTIONS request carrying Allow/ 356 trailers. Although ICAP/1.0 [RFC3507] allows a list of features in 357 the Allow header, some ICAP clients might not be able to handle an 358 Allow header other than "Allow: 204"; therefore, a server SHOULD NOT 359 send Allow/trailers in a response to a request without Allow/ 360 trailers. 362 A server compliant with this specification MAY send Allow/trailers in 363 a response without a trailer to a request with Allow/trailers. The 364 client receiving a no-trailer REQMOD or RESPMOD response with Allow/ 365 trailers ought to ignore Allow/trailers. Nevertheless, the server is 366 allowed to respond with Allow/trailers in this context because doing 367 so might simplify server implementation and configuration. 369 A server receiving both a Trailer header field and Allow/trailers in 370 the request MUST expect a trailer in that request. In all other 371 cases, a server MUST use the usual trailer-free ICAP request syntax. 372 A server receiving a Trailer header field without Allow/trailers in a 373 request MAY treat this request as syntactically malformed and, 374 regardless of the request treatment, MUST NOT reuse the connection 375 for any future requests. A previously received pipelined request is 376 not a "future request", even if the server has not finished 377 responding to it yet. 379 A server MAY send a trailer in any response that satisfies all of 380 these conditions: 382 1. the response is for a request containing Allow/trailers; 384 2. the response has a body. 386 A server MUST NOT send a trailer in any other response. 388 To send a trailer, the server MUST send Allow/trailers and a Trailer 389 header field in the same response. A server MUST NOT send a 390 combination of those two header fields without sending a trailer. 392 10. Examples 394 The following examples illustrate trailer exchanges between ICAP 395 agents compliant with this specification. To clarify message and 396 message part boundaries, all CRLF sequences after major message parts 397 are shown as "\r\n" lines, while boundaries between ICAP requests and 398 responses are signified by empty lines. CRLF sequences at the end of 399 other lines are implied. Unimportant low-level details such as 400 irrelevant HTTP and ICAP header fields or Encapsulated offsets are 401 shown as "...". 403 Figure 1 shows an OPTIONS request and response when both the client 404 and the server are compliant with this specification. The client 405 also supports the ICAP 206 extension, but the server does not. 407 OPTIONS icap://example.net/sample-service ICAP/1.0 408 ... 409 Allow: 204, trailers, 206 410 \r\n 412 ICAP/1.0 200 OK 413 ... 414 Allow: 204 415 Allow: trailers 416 \r\n 418 Figure 1: OPTIONS handshake 420 The OPTIONS response in Figure 1 contains two Allow headers to 421 illustrate one of several possible implementations. A compliant 422 server can also send a single Allow header with a list of values, 423 just like the client does in the above example. 425 Figure 2 is a RESPMOD request with an ICAP trailer that the client 426 can send after receiving an OPTIONS response with Allow/trailers 427 shown in Figure 1. Note that the Trailer field value ("TBD") does 428 not match the actual field names in the trailer. 430 RESPMOD icap://example.net/sample-service ICAP/1.0 431 ... 432 Allow: 204, trailers 433 Trailer: TBD 434 Encapsulated: req-hdr=0, res-hdr=..., res-body=... 435 \r\n 436 GET /origin-resource HTTP/1.1 437 ... 438 \r\n 439 HTTP/1.1 200 OK 440 ... 441 Content-Length: 24 442 \r\n 443 18 444 Origin server sent this. 445 0 446 \r\n 447 X-Client-Log-Lineno: 15612570 448 X-Client-Status: disconnected (at 1470262108) 449 \r\n 451 Figure 2: Request Trailer 453 Figure 3 is a RESPMOD response with both HTTP and ICAP trailers 454 present. The ICAP trailer contains X-Threat-Found and Connection 455 fields. The ICAP trailer Connection field overwrites the Connection 456 field in the ICAP header. To respond with this ICAP trailer, the 457 server ought to receive Allow/trailers in the corresponding ICAP 458 request (e.g., like the request shown in in Figure 2). 460 ICAP/1.0 200 OK 461 ... 462 Connection: keep-alive 463 Encapsulated: res-hdr=0, res-body=... 464 Allow: trailers 465 Trailer: X-ICAP-Log-Lineno, X-Threat-Found 466 \r\n 467 HTTP/1.1 200 OK 468 ... 469 Content-Length: 22 470 Trailer: X-Content-Checksum 471 \r\n 472 16 473 Origin server sent \0. 474 0 475 X-Content-Checksum: sha1-short=183caa016 476 \r\n 477 X-Threat-Found: Type=0; Resolution=1; Threat=backslash 478 Connection: close 479 \r\n 481 Figure 3: Two Trailers 483 TODO: Figure out how to keep figure descriptions and figures on the 484 same page without inserting empty space into HTML rendering of the 485 draft. 487 11. Security Considerations 489 Proper trailer support reduces old ICAP [RFC3507] security concerns 490 because implementations unaware of trailer complexities are arguably 491 more likely to misbehave when receiving HTTP or ICAP trailers. 493 Sending request headers with multiple Allow tokens could crash poor- 494 quality ICAP servers unaware of this specification. Trailer support 495 negotiation rules partially mitigate that risk by restricting unaware 496 implementations exposure; such implementations are exposed only 497 during OPTIONS exchanges. Since OPTIONS transaction has to precede 498 any HTTP message processing, and since virtually all ICAP client- 499 server relationships are stable, most poor-quality implementations 500 would be detected early and reliably. 502 A naive implementation of trailer interpretation logic might update 503 an already "frozen" or "committed" (at header parsing time) state of 504 the ICAP transaction or connection, resulting in crashes and other 505 problems. For example, such an implementation could panic after 506 discovering a Connection trailer field value that contradicts the 507 Connection header field value that has already been received and 508 processed at the beginning of the same transaction. 510 12. Normative References 512 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 513 Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/ 514 RFC2119, March 1997, 515 . 517 [RFC3507] Elson, J. and A. Cerpa, "Internet Content Adaptation 518 Protocol (ICAP)", RFC 3507, DOI 10.17487/RFC3507, April 519 2003, . 521 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 522 Specifications: ABNF", STD 68, RFC 5234, DOI 10.17487/ 523 RFC5234, January 2008, 524 . 526 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 527 Protocol (HTTP/1.1): Message Syntax and Routing", RFC 528 7230, DOI 10.17487/RFC7230, June 2014, 529 . 531 [Errata] Various Authors, , "RFC 3507 Errata", 532 . 534 Author's Address 536 Alex Rousskov 537 The Measurement Factory 539 Email: rousskov@measurement-factory.com 540 URI: http://www.measurement-factory.com/