idnits 2.17.1 draft-ietf-httpbis-rand-access-live-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (March 7, 2017) is 2600 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Experimental ---------------------------------------------------------------------------- == Unused Reference: 'RFC7230' is defined on line 355, but no explicit reference was found in the text == Unused Reference: 'RANGE-UNIT-REGISTRY' is defined on line 367, but no explicit reference was found in the text == Unused Reference: 'RFC4234' is defined on line 372, but no explicit reference was found in the text ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) ** Obsolete normative reference: RFC 7233 (Obsoleted by RFC 9110) -- Obsolete informational reference (is this intentional?): RFC 4234 (Obsoleted by RFC 5234) Summary: 3 errors (**), 0 flaws (~~), 5 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 HTTP Working Group C. Pratt 3 Internet-Draft CableLabs 4 Intended status: Experimental B. Stark 5 Expires: September 8, 2017 AT&T 6 D. Thakore 7 CableLabs 8 March 7, 2017 10 HTTP Random Access and Live Content 11 draft-ietf-httpbis-rand-access-live-00 13 Abstract 15 To accommodate byte range requests for content that has data appended 16 over time, this document defines semantics that allow a HTTP client 17 and server to perform byte-range GET and HEAD requests that start at 18 an arbitrary byte offset within the representation and ends at an 19 indeterminate offset. 21 Editorial Note (To be removed by RFC Editor before publication) 23 Discussion of this draft takes place on the HTTPBIS working group 24 mailing list (ietf-http-wg@w3.org), which is archived at 25 . 27 Working Group information can be found at ; 28 source code and issues list for this draft can be found at 29 . 31 Status of This Memo 33 This Internet-Draft is submitted in full conformance with the 34 provisions of BCP 78 and BCP 79. 36 Internet-Drafts are working documents of the Internet Engineering 37 Task Force (IETF). Note that other groups may also distribute 38 working documents as Internet-Drafts. The list of current Internet- 39 Drafts is at http://datatracker.ietf.org/drafts/current/. 41 Internet-Drafts are draft documents valid for a maximum of six months 42 and may be updated, replaced, or obsoleted by other documents at any 43 time. It is inappropriate to use Internet-Drafts as reference 44 material or to cite them other than as "work in progress." 46 This Internet-Draft will expire on September 8, 2017. 48 Copyright Notice 50 Copyright (c) 2017 IETF Trust and the persons identified as the 51 document authors. All rights reserved. 53 This document is subject to BCP 78 and the IETF Trust's Legal 54 Provisions Relating to IETF Documents 55 (http://trustee.ietf.org/license-info) in effect on the date of 56 publication of this document. Please review these documents 57 carefully, as they describe your rights and restrictions with respect 58 to this document. Code Components extracted from this document must 59 include Simplified BSD License text as described in Section 4.e of 60 the Trust Legal Provisions and are provided without warranty as 61 described in the Simplified BSD License. 63 Table of Contents 65 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 66 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 67 2. Performing Range requests on Random-Access Aggregating 68 ("live") Content . . . . . . . . . . . . . . . . . . . . . . 3 69 2.1. Establishing the Randomly Accessible Byte Range . . . . . 4 70 2.2. Byte-Range Requests Beyond the Randomly Accessible Byte 71 Range . . . . . . . . . . . . . . . . . . . . . . . . . . 4 72 2.3. Byte-Range Responses Beyond the Randomly Accessible Byte 73 Range . . . . . . . . . . . . . . . . . . . . . . . . . . 5 74 3. Other Applications of Random-Access Aggregating Content . . . 6 75 3.1. Requests Starting at the Aggregation ("Live") Point . . . 6 76 3.2. Shift Buffer Representations . . . . . . . . . . . . . . 7 77 4. Security Considerations . . . . . . . . . . . . . . . . . . . 8 78 5. References . . . . . . . . . . . . . . . . . . . . . . . . . 8 79 5.1. Normative References . . . . . . . . . . . . . . . . . . 8 80 5.2. Informative References . . . . . . . . . . . . . . . . . 8 81 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 8 82 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 9 84 1. Introduction 86 Some Hypertext Transfer Protocol (HTTP) Clients use byte-range 87 requests (Range requests using the "bytes" Range Unit) to transfer 88 select portions of large representations. And in some cases large 89 representations require content to be continuously or periodically 90 appended - such as representations consisting of live audio or video 91 sources, blockchain databases, and log files. Clients cannot access 92 the appended/live content using a Range request with the bytes range 93 unit using the currently defined byte-range semantics without 94 accepting performance or behavior sacrifices which are not acceptable 95 for many applications. 97 For instance, HTTP Clients have the ability to access appended 98 content by simply transferring the entire accessible portion of the 99 representation from the beginning and continuing to read the appended 100 content as it's made available. Obviously, this is highly 101 inefficient for cases where the representation is large and only a 102 portion of the randomly accessible content is needed by the Client. 103 And when bandwidth is limited, the client may never "catch up" with 104 the appending content. 106 Clients can also attempt to access appended content by sending 107 periodic bytes Range requests using the last-known end byte position 108 (polling). Performing low-frequency periodic bytes Range requests in 109 this fashion (polling) introduces latency since the Client will 110 necessarily be somewhat behind the aggregated content - mimicking the 111 behavior (and latency) of segmented content representations such as 112 HLS or MPEG-DASH. And performing these Range requests at higher 113 frequency incurs more processing overhead and HTTP traffic as the 114 periodic requests will often return no content - since content is 115 usually aggregated in groups of bytes (e.g. a video frame, audio 116 sample, block, or log entry). 118 To accommodate byte-range requests on large representations which 119 have data appended over time efficiently and with low latency, this 120 recommendation defines semantics whereby the HTTP Client performs 121 byte-range requests using a combination of open-ended byte-range HEAD 122 requests and GET requests using "Large Value" last-byte-pos values. 124 1.1. Requirements Language 126 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 127 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 128 document are to be interpreted as described in RFC 2119 [RFC2119]. 130 2. Performing Range requests on Random-Access Aggregating ("live") 131 Content 133 There are two critical operations for accessing randomly accessing 134 live/aggregating representations: 136 Establishing the randomly accessible range of the representation, 137 and 139 Performing range requests that continue beyond the randomly 140 accessible range. 142 2.1. Establishing the Randomly Accessible Byte Range 144 Establishing if a representation is continuously aggregating ("live") 145 and determining the randomly accessible byte range can both be 146 determined using the existing definition for an open-ended byte-range 147 request. Specifically, [RFC7233] defines a byte-range request of the 148 form: 150 byte-range-spec = first-byte-pos "-" [ last-byte-pos ] 152 which allows a Client to send a request with a first-byte-pos and 153 leave last-byte-pos absent. A Server that receives a satisfiable 154 byte-range request (with first-byte-pos smaller than the current 155 representation length) must respond with a 206 status code (Partial 156 Content) with a Content-Range header indicating the currently 157 satisfiable byte range. For example, a Client-issued HEAD request 158 performed against a continuously aggregating representation hosted on 159 a Server could contain a byte-range header of the form: 161 Range: bytes=0- 163 could return 165 Content-Range: bytes 0-1234567/* 167 from the Server indicating that (1) the complete representation 168 length is unknown (via the "*" in place of the complete-length field) 169 and (2) that only bytes 0-1234567 were accessable at the time the 170 request was processed. The Client can infer from this response that 171 bytes 0-1234567 of the representation can be requested and returned 172 in a timely fashion (the bytes are immediately available). 174 2.2. Byte-Range Requests Beyond the Randomly Accessible Byte Range 176 Once a Client has determined that a representation has an 177 indeterminate length and established the byte range that can be 178 accessed, it may want to perform a request that starts within the 179 randomly accessible content range and ends at an indefinite "live" 180 point - a point where the byte-range GET request is fulfilled on- 181 demand as the content is aggregated. 183 For example, for a large video asset, a client may wish to start a 184 content transfer from the video "key" frame immediately before the 185 point of aggregation and continue the content transfer indefinitely 186 as content is aggregated - in order to support low-latency startup of 187 a live video stream. 189 Unlike a byte-range Range request, a byte-range Content-Range 190 response header cannot be "open ended", per [RFC7233]: 192 byte-content-range = bytes-unit SP ( byte-range-resp / 193 unsatisfied-range ) 195 byte-range-resp = byte-range "/" ( complete-length / "*" ) 197 byte-range = first-byte-pos "-" last-byte-pos 199 unsatisfied-range = "*/" complete-length 201 complete-length = 1*DIGIT 203 last-byte-pos is required in byte-range. So in order to preserve 204 interoperability with existing HTTP clients, servers, proxies, and 205 caches, this document proposes a mechanism for a Client to indicate 206 support for handling an indeterminate-length byte-range response, and 207 a mechanism for a Server to indicate if/when it's providing a 208 indeterminate-length response. 210 A Client can indicate support for indeterminate-length byte-ranges by 211 providing a Very Large Value for the last-byte-pos in the byte-range 212 request. For example, a Client can perform a byte-range GET request 213 of the form: 215 Range: bytes=1230000-999999999999 217 where the last-byte-pos in the Request is much larger than the last- 218 byte-pos returned in response to an open-ended byte-range request. 220 2.3. Byte-Range Responses Beyond the Randomly Accessible Byte Range 222 A Server may indicate that it is supplying an continuously 223 aggregating ("live") response by supplying the Client request's last- 224 byte-pos in the Content-Range response header. 226 For example: 228 Range: bytes=1230000-999999999999 230 could return 232 Content-Range: bytes 1230000-999999999999/* 234 from the Server to indicate that the response will start at byte 235 1230000 and continues indefinitely to include all aggregated content, 236 as it becomes available. 238 A Server that doesn't support or supply a continuously aggregating 239 ("live") response should supply the currently satisfiable byte range, 240 as it would with an open-ended byte request. 242 For example: 244 Range: bytes=1230000-999999999999 246 could return 248 Content-Range: bytes 1230000-1234567/* 250 from the Server to indicate that the response will start at byte 251 1230000 and end at byte 1234567 and will not include any aggregated 252 content. This is the response expected from a typically-configured 253 HTTP Server - one that doesn't support byte-range requests on 254 aggregated content. 256 A Client that doesn't receive a response indicating it is 257 continuously aggregating must use other means to access aggregated 258 content (e.g. periodic byte-range polling). 260 A Server that returns a continuously aggregating ("live") response 261 should return data using chunked transfer coding and not provide a 262 Content-Length header. A 0-length chunk indicates that aggregation 263 of the transferring resource is permanently discontinued. 265 3. Other Applications of Random-Access Aggregating Content 267 3.1. Requests Starting at the Aggregation ("Live") Point 269 If a Client would like to start the content transfer at the 270 Aggregation ("live") point without including any randomly accessible 271 portion of the representation, then it should supply the last-byte- 272 pos from the most-recently received byte-range-spec and a Very Large 273 Value for the last-byte-pos in the byte-range request. 275 For example a HEAD request containing: 277 Range: bytes=0- 279 could return 281 Content-Range: bytes 0-1234567/* 283 and a GET request containing 285 Range: bytes=1234567-999999999999 287 could return 289 Content-Range: bytes 1234567-999999999999/* 291 with the response body starting with continuously aggregating 292 ("live") data and continuing indefinitely. 294 3.2. Shift Buffer Representations 296 Some representations lend themselves to front-end content deletion in 297 addition to aggregation. While still supporting random access, 298 representations of this type have a portion at the beginning ("0" 299 end) of the randomly accessible region become inaccessible over time. 300 Examples of this kind of representation would be a audio-video time- 301 shift buffer or a rolling log file. 303 For example a HEAD request containing: 305 Range: bytes=0- 307 could return 309 Content-Range: bytes 1000000-1234567/* 311 indicating that the first 1000000 bytes were not accessible at the 312 time the HEAD request was processed. Subsequent HEAD requests could 313 return: 315 Content-Range: bytes 1000000-1234567/* 317 Content-Range: bytes 1010000-1244567/* 319 Content-Range: bytes 1020000-1254567/* 321 Note though that the difference between the first-byte-pos and last- 322 byte-pos need not be constant. 324 The Client could then follow-up with a GET request containing 326 Range: bytes=1020000-999999999999 328 with the Server returning 330 Content-Range: bytes 1020000-999999999999/* 332 with the response body returning bytes 1020000-1254567 immediately 333 and aggregated ("live") data being returned as the content is 334 aggregated. 336 4. Security Considerations 338 One potential issue with this recommendation is related to the use of 339 very-large last-byte-pos values. Some Client and Server 340 implementations may not be prepared to deal with byte position values 341 of 2^^63 and beyond. So in applications where there's no expectation 342 that the representation will ever exceed 2^^63, a value smaller than 343 this value should be used as the Very Large last-byte-pos in a byte- 344 seek request or content-range response. 346 5. References 348 5.1. Normative References 350 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 351 Requirement Levels", BCP 14, RFC 2119, 352 DOI 10.17487/RFC2119, March 1997, 353 . 355 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 356 Protocol (HTTP/1.1): Message Syntax and Routing", 357 RFC 7230, DOI 10.17487/RFC7230, June 2014, 358 . 360 [RFC7233] Fielding, R., Ed., Lafon, Y., Ed., and J. Reschke, Ed., 361 "Hypertext Transfer Protocol (HTTP/1.1): Range Requests", 362 RFC 7233, DOI 10.17487/RFC7233, June 2014, 363 . 365 5.2. Informative References 367 [RANGE-UNIT-REGISTRY] 368 IANA, "Hypertext Transfer Protocol (HTTP) Parameters", 369 2016, . 372 [RFC4234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 373 Specifications: ABNF", RFC 4234, DOI 10.17487/RFC4234, 374 October 2005, . 376 Appendix A. Acknowledgements 378 Mark Nottingham, Patrick McManus, Julian Reschke, Remy Lebeau, Rodger 379 Combs, Thorsten Lohmar, Martin Thompson, Adrien de Croy, K. Morgan, 380 Roy T. Fielding, Jeremy Poulter. 382 Authors' Addresses 384 Craig Pratt 385 CableLabs 386 858 Coal Creek Circle 387 Louisville, CO 80027 389 Email: pratt@acm.org 391 Barbara Stark 392 AT&T 393 Atlanta, GA 394 US 396 Email: barbara.stark@att.com 398 Darshak Thakore 399 CableLabs 400 858 Coal Creek Circle 401 Louisville, CO 80027 403 Email: d.thakore@cablelabs.com