idnits 2.17.1 draft-pratt-httpbis-rand-access-live-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (November 13, 2016) is 2721 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC7230' is defined on line 350, but no explicit reference was found in the text == Unused Reference: 'RANGE-UNIT-REGISTRY' is defined on line 362, but no explicit reference was found in the text == Unused Reference: 'RFC4234' is defined on line 367, but no explicit reference was found in the text ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) ** Obsolete normative reference: RFC 7233 (Obsoleted by RFC 9110) -- Obsolete informational reference (is this intentional?): RFC 4234 (Obsoleted by RFC 5234) Summary: 3 errors (**), 0 flaws (~~), 5 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Hypertext Transfer Protocol C. Pratt 3 Internet-Draft CableLabs 4 Intended status: Informational B. Stark 5 Expires: May 17, 2017 AT&T 6 D. Thakore 7 CableLabs 8 November 13, 2016 10 HTTP Random Access and Live Content 11 draft-pratt-httpbis-rand-access-live-00 13 Abstract 15 To accommodate byte range requests for content that has data appended 16 over time, this document defines semantics that allow a HTTP client 17 and server to perform byte-range GET and HEAD requests that start at 18 an arbitrary byte offset within the representation and ends at an 19 indeterminate offset. 21 Requirements Language 23 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 24 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 25 document are to be interpreted as described in RFC 2119 [RFC2119]. 27 Status of This Memo 29 This Internet-Draft is submitted in full conformance with the 30 provisions of BCP 78 and BCP 79. 32 Internet-Drafts are working documents of the Internet Engineering 33 Task Force (IETF). Note that other groups may also distribute 34 working documents as Internet-Drafts. The list of current Internet- 35 Drafts is at http://datatracker.ietf.org/drafts/current/. 37 Internet-Drafts are draft documents valid for a maximum of six months 38 and may be updated, replaced, or obsoleted by other documents at any 39 time. It is inappropriate to use Internet-Drafts as reference 40 material or to cite them other than as "work in progress." 42 This Internet-Draft will expire on May 17, 2017. 44 Copyright Notice 46 Copyright (c) 2016 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents 51 (http://trustee.ietf.org/license-info) in effect on the date of 52 publication of this document. Please review these documents 53 carefully, as they describe your rights and restrictions with respect 54 to this document. Code Components extracted from this document must 55 include Simplified BSD License text as described in Section 4.e of 56 the Trust Legal Provisions and are provided without warranty as 57 described in the Simplified BSD License. 59 Table of Contents 61 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 62 2. Performing Range requests on Random-Access Aggregating 63 ("live") Content . . . . . . . . . . . . . . . . . . . . . . 3 64 2.1. Establishing the Randomly Accessible Byte Range . . . . . 3 65 2.2. Byte-Range Requests Beyond the Randomly Accessible Byte 66 Range . . . . . . . . . . . . . . . . . . . . . . . . . . 4 67 2.3. Byte-Range Responses Beyond the Randomly Accessible Byte 68 Range . . . . . . . . . . . . . . . . . . . . . . . . . . 5 69 3. Other Applications of Random-Access Aggregating Content . . . 6 70 3.1. Requests Starting at the Aggregation ("Live") Point . . . 6 71 3.2. Shift Buffer Representations . . . . . . . . . . . . . . 6 72 4. Security Considerations . . . . . . . . . . . . . . . . . . . 7 73 5. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 8 74 6. References . . . . . . . . . . . . . . . . . . . . . . . . . 8 75 6.1. Normative References . . . . . . . . . . . . . . . . . . 8 76 6.2. Informative References . . . . . . . . . . . . . . . . . 8 77 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 8 79 1. Introduction 81 Some Hypertext Transfer Protocol (HTTP) Clients use byte-range 82 requests (Range requests using the "bytes" Range Unit) to transfer 83 select portions of large representations. And in some cases large 84 representations require content to be continuously or periodically 85 appended - such as representations consisting of live audio or video 86 sources, blockchain databases, and log files. Clients cannot access 87 the appended/live content using a Range request with the bytes range 88 unit using the currently defined byte-range semantics accepting 89 performance or behavior sacrifices which are not acceptable for many 90 applications. 92 For instance, HTTP Clients have the ability to access appended 93 content by simply transferring the entire accessible portion of the 94 representation from the beginning and continuing to read the appended 95 content as it's made available. Obviously, this is highly 96 inefficient for cases where the representation is large and only a 97 portion of the randomly accessible content is needed by the Client. 98 And when bandwidth is limited, the client may never "catch up" with 99 the appending content. 101 Clients can also attempt to access appended content by sending 102 periodic bytes Range requests using the last-known end byte position. 103 Performing periodic bytes Range requests in this fashion (polling) 104 introduces latency since the Client will necessarily be somewhat 105 behind the aggregated content - mimicking the behavior (and latency) 106 of segmented content representations such as HLS or MPEG-DASH. And 107 performing these Range requests at higher frequency incurs more 108 processing overhead and HTTP traffic as the periodic requests will 109 often return no content - since content is usually aggregated in 110 groups of bytes (e.g. a video frame, audio sample, block, or log 111 entry). 113 To accommodate byte-range requests on large representations which 114 have data appended over time efficiently and with low latency, this 115 recommendation defines semantics whereby the HTTP Client performs 116 byte-range requests using a combination of open-ended byte-range HEAD 117 requests and GET requests using "Large Value" last-byte-pos values. 119 2. Performing Range requests on Random-Access Aggregating ("live") 120 Content 122 There are two critical operations for accessing randomly accessing 123 live/aggregating representations: 125 Establishing the randomly accessible range of the representation, 126 and 128 Performing range requests that continue beyond the randomly 129 accessible range. 131 2.1. Establishing the Randomly Accessible Byte Range 133 Establishing if a representation is continuously aggregating ("live") 134 and determining the randomly accessible byte range can be performed 135 using the existing definition for an open-ended byte-range request. 136 Specifically, [RFC7233] defines: 138 byte-range-spec = first-byte-pos "-" [ last-byte-pos ] 140 which allows a Client to send a request with a first-byte-pos and 141 leave last-byte-pos absent. A Server that receives a satisfiable 142 byte-range request (with first-byte-pos smaller than the current 143 representation length) must respond with a 206 status code (Partial 144 Content) with a Content-Range header indicating the satisfiable byte 145 range(s). For example, a Client-issued HEAD request against a 146 continuously aggregating representation hosted on a Server could 147 contain a byte-range header of the form: 149 Range: bytes=0- 151 could return 153 Content-Range: bytes 0-1234567/* 155 from the Server indicating that (1) the complete representation 156 length is unknown (via the "*" in place of the complete-length field) 157 and (2) that only bytes 0-1234567 were accessable at the time the 158 request was processed. The Client can infer from this response that 159 bytes 0-1234567 of the representation can be requested and returned 160 in a timely fashion (the bytes are immediately available). 162 2.2. Byte-Range Requests Beyond the Randomly Accessible Byte Range 164 Once a Client has determined that a representation has an 165 indeterminate length and established the byte range that can be 166 accessed, it may want to perform a request that starts within the 167 randomly accessible content range and ends at an indefinite "live" 168 point - a point where the byte-range GET request is fulfilled on- 169 demand as the content is aggregated. 171 For example, for a large video asset, a client may wish to start a 172 content transfer from the video "key" frame immediately before the 173 point of aggregation and continue the content transfer indefinitely 174 as content is aggregated - in order to support low-latency startup of 175 a live video stream. 177 Unlike a byte-range Range request, a byte-range Content-Range 178 response header cannot be "open ended", per [RFC7233]: 180 byte-content-range = bytes-unit SP ( byte-range-resp / 181 unsatisfied-range ) 183 byte-range-resp = byte-range "/" ( complete-length / "*" ) 185 byte-range = first-byte-pos "-" last-byte-pos 187 unsatisfied-range = "*/" complete-length 189 complete-length = 1*DIGIT 191 last-byte-pos is required in byte-range. So in order to preserve 192 interoperability with existing HTTP clients, servers, proxies, and 193 caches, this document proposes a mechanism for a Client to indicate 194 support for handling an indeterminate-length byte-range response, and 195 a mechanism for a Server to indicate if/when it's providing a 196 indeterminate-length response. 198 A Client can indicate support for indeterminate-length byte-ranges by 199 providing a Very Large Value for the last-byte-pos in the byte-range 200 request. For example, a Client can perform a byte-range GET request 201 of the form: 203 Range: bytes=1230000-999999999999 205 where the last-byte-pos in the Request is much larger than the last- 206 byte-pos returned in response to an open-ended byte-range request. 208 2.3. Byte-Range Responses Beyond the Randomly Accessible Byte Range 210 A Server can indicate that it's supplying an continuously aggregating 211 ("live") response by supplying the Client request's last-byte-pos in 212 the Content-Range response header. 214 For example: 216 Range: bytes=1230000-999999999999 218 could return 220 Content-Range: bytes 1230000-999999999999/* 222 from the Server to indicate that the response will start at byte 223 1230000 and continues indefinitely to include all aggregated content, 224 as it becomes available. 226 A Server that doesn't support or supply a continuously aggregating 227 ("live") response can supply a value other than the Client request's 228 last-byte-pos in the Content-Range response header. 230 For example: 232 Range: bytes=1230000-999999999999 234 could return 236 Content-Range: bytes 1230000-1234567/* 238 from the Server to indicate that the response will start at byte 239 1230000 and end at byte 1234567 and will not include any aggregated 240 content. This is the response expected from a typically-configured 241 HTTP Server - one that doesn't support byte-range requests on 242 aggregated content. 244 A Client that doesn't receive a response indicating it is 245 continuously aggregating must use other means to access aggregated 246 content (e.g. periodic byte-range polling). 248 A Server that returns a continuously aggregating ("live") response 249 should return data using chunked transfer coding and not provide a 250 Content-Length header. A 0-length chunk indicates that aggregation 251 of the transferring resource is permanently discontinued. 253 3. Other Applications of Random-Access Aggregating Content 255 3.1. Requests Starting at the Aggregation ("Live") Point 257 If a Client would like to start the content transfer at the 258 Aggregation ("live") point without including any randomly accessible 259 portion of the representation, then it should supply the last-byte- 260 pos from the most-recently received byte-range-spec and a Very Large 261 Value for the last-byte-pos in the byte-range request. 263 For example a HEAD request containing: 265 Range: bytes=0- 267 could return 269 Content-Range: bytes 0-1234567/* 271 and a GET request containing 273 Range: bytes=1234567-999999999999 275 could return 277 Content-Range: bytes 1234567-999999999999/* 279 with the response body starting with continuously aggregating 280 ("live") data and continuing indefinitely. 282 3.2. Shift Buffer Representations 284 Some representations lend themselves to front-end content deletion in 285 addition to aggregation. While still supporting random access, 286 representations of this type have a portion at the beginning ("0" 287 end) of the randomly accessible region become inaccessible over time. 289 Examples of this kind of representation would be a audio-video time- 290 shift buffer or a rolling log file. 292 For example a HEAD request containing: 294 Range: bytes=0- 296 could return 298 Content-Range: bytes 1000000-1234567/* 300 indicating that the first 1000000 bytes were not accessible at the 301 time the HEAD request was processed. Subsequent HEAD requests could 302 return: 304 Content-Range: bytes 1000000-1234567/* 306 Content-Range: bytes 1010000-1244567/* 308 Content-Range: bytes 1020000-1254567/* 310 Note though that the difference between the first-byte-pos and last- 311 byte-pos need not be constant. 313 The Client could then follow-up with a GET request containing 315 Range: bytes=1020000-999999999999 317 with the Server returning 319 Content-Range: bytes 1020000-999999999999/* 321 with the response body returning bytes 1020000-1254567 immediately 322 and aggregated ("live") data being returned as the content is 323 aggregated. 325 4. Security Considerations 327 One potential issue with this recommendation is related to the use of 328 very-large last-byte-pos values. Some Client and Server 329 implementations may not be prepared to deal with byte position values 330 of 2^^63 and beyond. So in applications where there's no expectation 331 that the representation will ever exceed 2^^63, a value smaller than 332 this value should be used as the Very Large last-byte-pos in a byte- 333 seek request or content-range response. 335 5. Acknowledgements 337 Mark Nottingham, Patrick McManus, Julian Reschke, Remy Lebeau, Rodger 338 Combs, Thorsten Lohmar, Martin Thompson, Adrien de Croy, K. Morgan, 339 Roy T. Fielding, Jeremy Poulter 341 6. References 343 6.1. Normative References 345 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 346 Requirement Levels", BCP 14, RFC 2119, 347 DOI 10.17487/RFC2119, March 1997, 348 . 350 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 351 Protocol (HTTP/1.1): Message Syntax and Routing", 352 RFC 7230, DOI 10.17487/RFC7230, June 2014, 353 . 355 [RFC7233] Fielding, R., Ed., Lafon, Y., Ed., and J. Reschke, Ed., 356 "Hypertext Transfer Protocol (HTTP/1.1): Range Requests", 357 RFC 7233, DOI 10.17487/RFC7233, June 2014, 358 . 360 6.2. Informative References 362 [RANGE-UNIT-REGISTRY] 363 IANA, "Hypertext Transfer Protocol (HTTP) Parameters", 364 2016, . 367 [RFC4234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 368 Specifications: ABNF", RFC 4234, DOI 10.17487/RFC4234, 369 October 2005, . 371 Authors' Addresses 373 Craig Pratt 374 CableLabs 375 Portland, OR 97229 376 US 378 Email: craig@ecaspia.com 379 Barbara Stark 380 AT&T 381 Atlanta, GA 382 US 384 Email: barbara.stark@att.com 386 Darshak Thakore 387 CableLabs 388 858 Coal Creek Circle 389 Louisville, CO 80027 391 Email: d.thakore@cablelabs.com