idnits 2.17.1 draft-finkelman-httpbis-has-redirecting-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 4, 2018) is 2001 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group O. Finkelman 3 Internet-Draft Qwilt 4 Intended status: Informational A. Begen 5 Expires: May 8, 2019 Networked Media 6 November 4, 2018 8 HTTP Redirects in HTTP Adaptive Streaming 9 draft-finkelman-httpbis-has-redirecting-00 11 Abstract 13 This document motivates the need for clarifications of client 14 behavior in cases of HTTP redirect to a content delivery network 15 (CDN) when the redirected object contains relative references. This 16 document focuses on HTTP Adaptive Streaming (HAS) use cases, but it 17 might be possible to generalize to other use cases. The goal of this 18 document is to present the current status of things and to explore 19 potential solutions. 21 Status of This Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at https://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on May 8, 2019. 38 Copyright Notice 40 Copyright (c) 2018 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (https://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 56 2. Glossary of Terms . . . . . . . . . . . . . . . . . . . . . . 3 57 3. Current State of Things . . . . . . . . . . . . . . . . . . . 3 58 4. Objectives . . . . . . . . . . . . . . . . . . . . . . . . . 4 59 5. Potential Solutions . . . . . . . . . . . . . . . . . . . . . 4 60 6. Examples and Interpretation of the Current RFCs . . . . . . . 5 61 6.1. Example 1: Absolute URI to Manifest . . . . . . . . . . . 6 62 6.2. Example 2: Relative Reference in an Encapsulating Entity 7 63 6.3. Example 3: Relative Reference with Redirect . . . . . . . 8 64 7. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 8 65 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 66 9. Security Considerations . . . . . . . . . . . . . . . . . . . 9 67 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 9 68 11. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 9 69 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 9 70 12.1. Normative References . . . . . . . . . . . . . . . . . . 9 71 12.2. Informative References . . . . . . . . . . . . . . . . . 10 72 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 10 74 1. Introduction 76 HTTP redirect, commonly using the "302 Found" response code, is 77 widely used in Content Delivery Networks (CDN) for HTTP Adaptive 78 Streaming (HAS), specifically for HTTP Live Streaming (HLS) [RFC8216] 79 and Dynamic Adaptive Streaming over HTTP (DASH) [MPEG-DASH]. Using 80 HTTP redirect rather than delegation by CNAME is a more powerful and 81 flexible solution. 83 At a high level, the required behavior from a HAS client is that 84 after getting an HTTP redirect to a playlist address that takes the 85 client to a CDN cache, if the playlist contains relative references 86 to the media segments, the client should be 'sticky' and request the 87 subsequent media segments directly from the cache without the need 88 for repeated redirects, which naturally would add undesirable 89 latency. Unfortunately, in some cases, common misinterpretations of 90 the RFC lead to wrong behavior of clients, while in other cases, this 91 required behavior contradicts the HTTP RFC. Thus, we are facing 92 issues with streaming clients that behave differently under different 93 use cases of HTTP redirects when relative references are in use 94 within the playlist. 96 The Streaming Video Alliance (SVA) and DASH Industry Forum (DASH-IF) 97 are attempting to resolve these issues and create clear guidelines 98 describing how video players, browsers and streaming applications 99 should behave in such HTTP redirect cases. 101 A discussion of this use case can also be found in Section 2.2.1 of 102 [RFC6983]. 104 The following sequence explains this step by step: 106 1. The client gets a URI to the playlist file from the origin 107 website (content provider). This URI may point directly at a CDN 108 address or be CNAMEd to a CDN node. 110 2. The client requests the playlist file from the CDN. 112 3. The CDN responds with an HTTP redirect, or a series of redirects 113 that eventually takes the client to the final cache. 115 4. The client requests the playlist from this final cache. 117 5. The client parses the relative URIs to media segments from the 118 playlist. 120 6. The client uses the final cache address to construct the absolute 121 URIs for the media segments that point to the final cache. 123 7. The client requests the media segments from the final cache. 125 The redirection should still be temporary in the sense that if a 126 client does a refresh, or repeatedly fails to get the playlist or the 127 media files from the cache address, it can go back to the original 128 playlist URI and request it again. 130 2. Glossary of Terms 132 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 133 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 134 "OPTIONAL" in this document are to be interpreted as described in BCP 135 14 [RFC2119] [RFC8174] when, and only when, they appear in all 136 capitals, as shown here. 138 3. Current State of Things 140 Currently, there are inconsistencies in the behaviors of HAS clients 141 when constructing the URI of a relative referenced object, read from 142 another resource, which was redirected. In some cases the client 143 constructs these URI relative to the redirected address, while in 144 other cases the client constructs it relative to the original URI of 145 the playlist before the redirection. 147 The result of the latter behavior is that the client returns to the 148 original host for each and every request, and then gets redirected 149 again to the cache address. This has a significant negative impact 150 on the the user experience (due to increased risk of slower media 151 segment downloads). Further, it causes redundant work in the 152 redirecting CDN host. The reasons for this are, in some cases, 153 misinterpretation of the standard and, in other cases, contradictions 154 between this requirement and the existing standard. 156 4. Objectives 158 Our objectives are to document a clear understanding when a client 159 SHOULD compute the absolute URI for relative reference taken from a 160 playlist using the actual retrieval URI of the playlist, and have a 161 specification that leads to a consistent behavior that meets the HAS 162 requirement, regardless of how the retrieval URI of the playlist was 163 resolved. 165 5. Potential Solutions 167 The below are high-level initial thoughts in order to get the 168 feedback from the HTTPbis working group. As we see it, the 169 alternatives could be one of the following, depending on the 170 understanding of the current state of things: 172 1. Have a single consistent interpretation of the existing RFCs 173 showing in which cases the behavior should be as required above. 175 2. Draft an extension to Section 5 of [RFC3986] to handle the 176 required cases. 178 3. Create a new HTTP 3xx response code that will be specified to 179 serve the required use cases. 181 4. Handle it outside the HTTP standard, i.e., in the SVA and DASH- 182 IF. 184 A solution should take under consideration the more advanced use 185 cases of: 187 1. A chain of redirects to get to the final cache. 189 2. Redirect back: What happens if there is a need to redirect the 190 client back to the original URI? How does the client behave 191 after such redirect of a specific media object? Is there any 192 difference between this case and another forward redirection? 194 3. An absolute URI appearing in the middle of a playlist, between 195 relative references (for example, for an advertisement segment) 196 should not break the sequence of things, it should be treated as 197 an absolute URI and the subsequent segments that are relative 198 references should be requested from the cache again. 200 4. Failover: If the cache does not answer, the client should go back 201 and request the playlist from the original URI, and not request 202 the relative reference object directly as it does not know how to 203 construct an alternative URI for it. 205 5. The client MAY, on its own consideration, re-request the playlist 206 again from the original URI, and may be redirected again to the 207 same cache, or other cache. In that sense the redirect is 208 temporary. 210 6. The solution should handle both the use case in which the 211 encapsulating entity was redirected and it includes relative 212 references to the media segment, as in HLS (Section 5.1.3 of 213 [RFC3986]), and the use case that encapsulating entity includes 214 the base URL embedded in it, as in DASH (Section 5.1.1 of 215 [RFC3986]), which may lead to the understanding that DASH 216 services should avoid using an absolute URI as base URL in the 217 Media Presentation Description (MPD), if they wish HTTP redirect 218 to CDN to work properly. 220 6. Examples and Interpretation of the Current RFCs 222 This is our interpretation of the current state of things using HLS 223 manifest examples, in each example the manifest URI is resolved 224 differently and we observe the implications on the resolution of the 225 relative references contained within the manifest file. 227 In the first example the master manifest URI is given as an absolute 228 URI, while in the second example the master manifest is given itself 229 as a relative reference from the encapsulation that provided it, both 230 lead to the required behavior. 232 In the third example, similar to the second, the manifest is given as 233 a relative reference but the request for the manifest is redirected. 234 This leads to the undesired behavior of the client going back to the 235 original location for every request, requiring redirect for every 236 media chunk. 238 6.1. Example 1: Absolute URI to Manifest 240 Let us take a HLS example and assume the player is passed an absolute 241 URI for the HLS manifest (e.g., a user has entered the manifest 242 address directly into the player): 244 https://video.cdn.example.com/vod/movie/master.m3u8 246 GET /vod/movie/master.m3u8 248 Host: video.cdn.example.com 250 HTTP/1.1 302 Found 252 Location: https://edge1.dcdn.example.net/vod/movie/master.m3u8 254 The client then requests the HLS manifest from the edge cache. 256 GET /vod/movie/master.m3u8 258 Host: edge1.dcdn.example.net 260 HTTP/1.1 200 OK 262 The client reads the per-bitrate manifest reference from the master 263 manifest. 265 #EXT-X-STREAM-INF:BANDWIDTH=2121091,RESOLUTION=640x360,CODECS="mp4 266 a.40.2,avc1.4d001f" 268 1800K/1800_complete.m3u8 270 The player needs to resolve 1800K/1800_complete.m3u8. 272 Following [RFC3986]: The m3u8 does not have a base URI embedded 273 within it, so, Section 5.1.1 does not apply. The relative reference 274 (and therefore the representation it references) is enclosed in an 275 encapsulating entity, so, Section 5.1.2 applies. Section 5.1.2 says 276 'Thus, the default base URI of a representation is the base URI of 277 the entity in which the representation is encapsulated.' 279 So what is the base URI of the encapsulating entity (/vod/movie/ 280 master.m3u8)? 282 1. If we follow [RFC3986] again: The master.m3u8 does not have a 283 base URI embedded within it, so, Section 5.1.1 does not apply. 285 2. The master.m3u8 is not encapsulated in any other entity, so, 286 5.1.2 does not apply. A URI was used to retrieve the m3u8, so, 287 Section 5.1.3 applies. 289 3. Section 5.1.3 says 'if a URI was used to retrieve the 290 representation, that URI shall be considered the base URI. Note 291 that if the retrieval was the result of a redirected request, the 292 last URI used (i.e., the URI that resulted in the actual 293 retrieval of the representation) is the base URI.' 295 4. As the retrieval was the result of a redirected request, the 296 second sentence applies and the last URI is 297 https://edge1.dcdn.example.net/vod/movie/master.m3u8 and so that 298 is the base URI for the manifest (the encapsulating entity). 300 5. Therefore the base URI for the relative URI 301 1800K/1800_complete.m3u8 is 302 https://edge1.dcdn.example.net/vod/movie/master.m3u8 and the 303 player should request https://edge1.dcdn.example.net/vod/ 304 movie/1800K/1800_complete.m3u8 306 6.2. Example 2: Relative Reference in an Encapsulating Entity 308 Let us take the same example, but instead of the player being passed 309 an absolute URI, let us assume the URI is relative and embedded 310 within another entity, i.e., a top-level HTML page 311 https://video.cdn.example.com/index.html. In that case /vod/movie/ 312 master.m3u8 (the encapsulating entity of 1800K/1800_complete.m3u8) is 313 itself encapsulated (in /index.html) and Section 5.1.2 does apply 314 this time, and the base URI of /vod/movie/master.m3u8 depends on the 315 base URI of /index.html 317 The base URI of /index.html (which is not encapsulated in another 318 entity) follows Section 5.1.3, and depends on how /index.html was 319 retrieved. 321 1. If /index.html was retrieved without redirection, the base URI is 322 https://video.cdn.example.com/index.html and the base URI for the 323 relative URI 1800K/1800_complete.m3u8 is 324 https://video.cdn.example.com/vod/movie/master.m3u8 and the 325 player should request https://video.cdn.example.com/vod/ 326 movie/1800K/1800_complete.m3u8 328 2. If /index.html was retrieved via a redirect (e.g., to 329 https://edge1.dcdn.example.net/index.html), the base URI is 330 https://edge1.dcdn.example.net/index.html, and the base URI for 331 the relative URI 1800K/1800_complete.m3u8 is 332 https://edge1.dcdn.example.net/vod/movie/master.m3u8 and the 333 player should request https://edge1.dcdn.example.net/vod/ 334 movie/1800K/1800_complete.m3u8 336 6.3. Example 3: Relative Reference with Redirect 338 In this example, as in Example 2 above, the manifest URI is relative 339 and embedded within another entity within a top-level HTML page 340 https://video.cdn.example.com/index.html. The base URI of 341 /index.html is an absolute URI and the request for /index.html is not 342 redirected, but, in this case, the request for /vod/movie/master.m3u8 343 is redirected. 345 What would be resulting URI for 1800K/1800_complete.m3u8 in this case 346 ? 348 1. According to Section 5.1.2 of [RFC3986] 1800K/1800_complete.m3u8 349 will get its base URI from the base URI of the encapsulating 350 entity which is /vod/movie/master.m3u8. 352 2. As /vod/movie/master.m3u8 is itself encapsulated within 353 /index.html then according to Section 5.1.2 its base URI is the 354 taken from the base URI of /index.html. Since this base URI is 355 https://video.cdn.example.com/index.html, the base URI of the 356 master.m3u8 is resolved to 357 https://video.cdn.example.com/vod/movie/master.m3u8, though it 358 was redirected and its actual retrieval URI was 359 https://edge1.dcdn.example.net/vod/movie/master.m3u8 361 3. The 1800K/1800_complete.m3u8, taking its base URI from the the 362 encapsulating master.m3u8, is resolved to 363 https://video.cdn.example.com/vod/movie/1800K/1800_complete.m3u8 365 One can see that in this case, as Section 5.1.2 does not define what 366 happens if the encapsulating entity was redirected, the 367 straightforward interpretation leads to the undesired result of the 368 1800_complete.m3u8 being requested from the original URI rather than 369 the redirection URI. 371 7. Conclusion 373 The examples above show that the behavior under playlist redirect 374 depends on how the playlist URI was resolved, and whether the 375 playlist URI was an absolute URI or a relative one. It also depends 376 on how exactly Section 5.1.2 of [RFC3986] is interpreted in the 377 absence of clear directions for the case that the encapsulating 378 entity itself was a relative reference which was redirected. 380 As explained above, we would like to have a single clear 381 specification defining a consistent client behavior that is 'sticky' 382 after a playlist redirect, regardless of how the playlist URI was 383 resolved. We, therefore, hope that the members of HTTPbis working 384 group could share their views regarding the preferred approach in 385 order to resolve this issue. 387 8. IANA Considerations 389 There are no IANA considerations. 391 9. Security Considerations 393 TBC. 395 10. Acknowledgements 397 The authors would like to thank the members of the Open Caching 398 working group of the Streaming Video Alliance (SVA) for their 399 feedback and especially Ben Niven-Jenkins and Kevin J. Ma for their 400 help in understanding the current state of the standards for the 401 provided examples. 403 11. Contributors 405 TBC. 407 12. References 409 12.1. Normative References 411 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 412 Requirement Levels", BCP 14, RFC 2119, 413 DOI 10.17487/RFC2119, March 1997, 414 . 416 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 417 Resource Identifier (URI): Generic Syntax", STD 66, 418 RFC 3986, DOI 10.17487/RFC3986, January 2005, 419 . 421 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 422 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 423 May 2017, . 425 12.2. Informative References 427 [MPEG-DASH] 428 ISO/IEC 23009-1:2014, "Dynamic Adaptive Streaming over 429 HTTP-- Part 1: Media Presentation Description and Segment 430 Formats", 2014. 432 [RFC6983] van Brandenburg, R., van Deventer, O., Le Faucheur, F., 433 and K. Leung, "Models for HTTP-Adaptive-Streaming-Aware 434 Content Distribution Network Interconnection (CDNI)", 435 RFC 6983, DOI 10.17487/RFC6983, July 2013, 436 . 438 [RFC8216] Pantos, R., Ed. and W. May, "HTTP Live Streaming", 439 RFC 8216, DOI 10.17487/RFC8216, August 2017, 440 . 442 Authors' Addresses 444 Ori Finkelman 445 Qwilt 446 6, Ha'harash 447 Hod HaSharon 4524079 448 Israel 450 Email: orif@qwilt.com 452 Ali Begen 453 Networked Media 454 Konya 455 Turkey 457 Email: ali.begen@networked.media