idnits 2.17.1 draft-brandenburg-cdni-has-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (April 12, 2013) is 4029 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-17) exists of draft-ietf-cdni-requirements-03 == Outdated reference: A later version (-14) exists of draft-ietf-cdni-framework-03 Summary: 0 errors (**), 0 flaws (~~), 3 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group R. van Brandenburg 3 Internet-Draft O. van Deventer 4 Intended status: Informational TNO 5 Expires: October 14, 2013 F. Le Faucheur 6 K. Leung 7 Cisco Systems 8 April 12, 2013 10 Models for adaptive-streaming-aware CDN Interconnection 11 draft-brandenburg-cdni-has-05 13 Abstract 15 This documents presents thoughts on the potential impact of 16 supporting HTTP Adaptive Streaming technologies in CDN 17 Interconnection (CDNI) scenarios. The intent is to present the 18 authors' analysis of the CDNI-HAS problem space and discuss different 19 options put forward both by the authors (and by others during 20 informal discussions) on how to deal with HAS in the context of CDNI. 21 THis document has been used as input information during the WG 22 process for making its decision regarding support for HAS. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at http://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on October 14, 2013. 41 Copyright Notice 43 Copyright (c) 2013 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (http://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 59 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 60 2. HTTP Adaptive Streaming aspects relevant to CDNI . . . . . . 5 61 2.1. Segmentation versus Fragmentation . . . . . . . . . . . . 5 62 2.2. Addressing chunks . . . . . . . . . . . . . . . . . . . . 6 63 2.2.1. Relative URLs . . . . . . . . . . . . . . . . . . . . 7 64 2.2.2. Absolute URLs with Redirection . . . . . . . . . . . 8 65 2.2.3. Absolute URL without Redirection . . . . . . . . . . 9 66 2.3. Live vs. VoD . . . . . . . . . . . . . . . . . . . . . . 10 67 2.4. Stream splicing . . . . . . . . . . . . . . . . . . . . . 11 68 3. Possible HAS Optimizations . . . . . . . . . . . . . . . . . 11 69 3.1. File Management and Content Collections . . . . . . . . . 12 70 3.1.1. General Remarks . . . . . . . . . . . . . . . . . . . 12 71 3.1.2. Candidate approaches . . . . . . . . . . . . . . . . 12 72 3.1.2.1. Option 1.1: No HAS awareness . . . . . . . . . . 12 73 3.1.2.2. Option 1.2: Allow single file storage of 74 fragmented content . . . . . . . . . . . . . . . 13 75 3.1.2.3. Option 1.3: Access correlation hint . . . . . . . 14 76 3.1.3. Recommendation . . . . . . . . . . . . . . . . . . . 14 77 3.2. Content Acquisition of Content Collections . . . . . . . 14 78 3.2.1. General Remarks . . . . . . . . . . . . . . . . . . . 14 79 3.2.2. Candidate Approaches . . . . . . . . . . . . . . . . 15 80 3.2.2.1. Option 2.1: No HAS awareness . . . . . . . . . . 15 81 3.2.2.2. Option 2.2: Allow single file acquisition of 82 fragmented content . . . . . . . . . . . . . . . 16 83 3.2.3. Recommendation . . . . . . . . . . . . . . . . . . . 16 84 3.3. Request Routing of HAS content . . . . . . . . . . . . . 17 85 3.3.1. General remarks . . . . . . . . . . . . . . . . . . . 17 86 3.3.2. Candidate approaches . . . . . . . . . . . . . . . . 17 87 3.3.2.1. Option 3.1: No HAS awareness . . . . . . . . . . 17 88 3.3.2.2. Option 3.2: Manifest File rewriting by uCDN . . . 19 89 3.3.2.3. Option 3.3: Two-step Manifest File rewriting . . 20 90 3.3.3. Recommendation . . . . . . . . . . . . . . . . . . . 22 91 3.4. Logging . . . . . . . . . . . . . . . . . . . . . . . . . 22 92 3.4.1. General remarks . . . . . . . . . . . . . . . . . . . 22 93 3.4.2. Candidate Approaches . . . . . . . . . . . . . . . . 23 94 3.4.2.1. Option 4.1: "Do-Nothing" Approach . . . . . . . . 23 95 3.4.2.2. Option 4.2: "CDNI Metadata Content Collection ID" 96 Approach . . . . . . . . . . . . . . . . . . . . 25 98 3.4.2.3. Option 4.3: "CDNI Logging Interface Compression" 99 Approach . . . . . . . . . . . . . . . . . . . . 26 100 3.4.2.4. Option 4.4: "Full HAS awareness/per-Session-Logs" 101 Approach . . . . . . . . . . . . . . . . . . . . 27 102 3.4.3. Recommendation . . . . . . . . . . . . . . . . . . . 28 103 3.5. URL Signing . . . . . . . . . . . . . . . . . . . . . . . 30 104 3.5.1. HAS Implications . . . . . . . . . . . . . . . . . . 30 105 3.5.2. CDNI Considerations . . . . . . . . . . . . . . . . . 31 106 3.5.3. Option 5.1: Do Nothing . . . . . . . . . . . . . . . 32 107 3.5.4. Option 5.2: Flexible URL Signing by CSP . . . . . . . 32 108 3.5.5. Option 5.3: Flexible URL Signing by Upstream CDN . . 35 109 3.5.6. Option 5.4: Authorization Group ID and HTTP 110 Cookie . . . . . . . . . . . . . . . . . . . . . . . 35 111 3.5.7. Option 5.5: HAS-awareness with HTTP Cookie in CDN . . 36 112 3.5.8. Option 5.6: HAS-awareness with Manifest in CDN . . . 38 113 3.5.9. Recommendation . . . . . . . . . . . . . . . . . . . 38 114 3.6. Content Purge . . . . . . . . . . . . . . . . . . . . . . 39 115 3.6.1. Option 6.1: No HAS awareness . . . . . . . . . . . . 40 116 3.6.2. Option 6.2: Purge Identifiers . . . . . . . . . . . . 40 117 3.6.3. Recommendation . . . . . . . . . . . . . . . . . . . 41 118 3.7. Other issues . . . . . . . . . . . . . . . . . . . . . . 41 119 4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 41 120 5. Security Considerations . . . . . . . . . . . . . . . . . . . 41 121 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 42 122 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 42 123 7.1. Normative References . . . . . . . . . . . . . . . . . . 42 124 7.2. Informative References . . . . . . . . . . . . . . . . . 42 125 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 42 127 1. Introduction 129 [RFC6707] defines the problem space for CDN Interconnection (CDNI) 130 and the associated CDNI interfaces. This includes support, through 131 interconnected CDNs, of content delivery to endusers using HTTP 132 progressive download and HTTP Adaptive Streaming (HAS). 134 HTTP Adaptive Streaming is an umbrella term for various HTTP-based 135 streaming technologies that allow a client to adaptively switch 136 between multiple bitrates depending on current network conditions. A 137 defining aspect of HAS is that, since it is based on HTTP, it is a 138 pull-based mechanism, with a client actively requesting content 139 segments, instead of the content being pushed to the client by a 140 server. Due to this pull-based nature, media servers delivering 141 content using HAS often show different characteristics when compared 142 with media servers delivering content using traditional streaming 143 methods such as RTP/RTSP, RTMP and MMS. 145 This document presents a discussion on what the impact of these 146 different characteristics is to the CDNI interfaces and what HAS- 147 specific optimizations may be required or may be desirable. The 148 scope of this document is to present the authors' analysis of the 149 CDNI-HAS problem space and discuss different options put forward both 150 by the authors (and by others during informal discussions) on how to 151 deal with HAS in the context of CDNI. The documents concludes by 152 presenting the authors' recommendations on how the CDNI WG should 153 deal with HAS in its initial charter, with a focus on 'making it 154 work' instead of including 'nice-to-have' optimizations that might 155 delay the development of the CDNI WG deliverables identified in its 156 initial charter. 158 It should be noted that the document is not a WG document, but has 159 been used as input information during the WG process for making its 160 decision regarding support for HAS. We expect the analysis presented 161 in the document will also be useful in the future if and when the WG 162 re-charters and wants to re-assess the level of HAS optimizations to 163 be supported in CDNI scenarios. 165 1.1. Terminology 167 This document uses the terminology defined in [RFC6707] and 168 [I-D.ietf-cdni-framework]. 170 For convenience, the definition of HAS-related terms are restated 171 here: 173 Content Item: A uniquely addressable content element in a CDN. A 174 content item is defined by the fact that it has its own Content 175 Metadata associated with it. It is the object of a request routing 176 operation in a CDN. An example of a Content Item is a video file/ 177 stream, an audio file/stream or an image file. 179 Chunk: a fixed length element that is the result of a segmentation or 180 fragmentation operation and that is independently addressable. 182 Fragment: A specific form of chunk (see Section 2.1). A fragment is 183 stored as part of a larger file that includes all chunks that are 184 part of the Chunk Collection. 186 Segment: A specific form of chunk (see Section 2.1). A segment is 187 stored as a single file from a file system perspective. 189 Original Content: Non-chunked content that is the basis for a 190 segmentation of fragmentation operation. Based on Original Content, 191 multiple alternative representations (using different encoding 192 methods, supporting different resolutions and/or targeting different 193 bitrates) may be derived, each of which may be fragmented or 194 segmented. 196 Chunk Collection: The set of all chunks that are the result of a 197 single segmentation or fragmentation operation being performed on a 198 single representation of the Original Content. A Chunk Collection is 199 described in a Manifest File. 201 Content Collection: The set of all Chunk Collections that are derived 202 from the same Original Content. A Content Collection may consist of 203 multiple Chunk Collections, each corresponding to a single 204 representation of the Original Content. A Content Collection may be 205 described by one or more Manifest Files. 207 Manifest File: A Manifest File, also referred to as Media 208 Presentation Description (MPD) file, is a file that list the way the 209 content has been chunked (possibly for multiple encodings) and where 210 the various chunks are located (in the case of segments) or how they 211 can be addressed (in the case of fragments). 213 2. HTTP Adaptive Streaming aspects relevant to CDNI 215 In the last couple of years, a wide variety of HAS-like protocols 216 have emerged. Among them are proprietary solutions such as Apple's 217 HTTP Live Streaming (HLS), Microsoft's HTTP Smooth Streaming (HSS) 218 and Adobe's HTTP Dynamic Streaming (HDS), and various standardized 219 solutions such as 3GPP Adaptive HTTP Streaming (AHS) and MPEG Dynamic 220 Adaptive Streaming over HTTP (DASH). While all of these technologies 221 share a common set of features, each has its own defining elements. 222 This chapter will look at some of the common characteristics and some 223 of the differences between these technologies and how those might be 224 relevant to CDNI. In particular, Section 2.1 will describe the 225 various methods to store HAS content and Section 2.2 will list three 226 methods that are used to address HAS content in a CDN. After these 227 generic HAS aspects are discussed, two special situations that need 228 to be taken into account when discussing HAS are addressed: 229 Section 2.3 discusses the differences between Live and VoD content, 230 while Section 2.4 discusses the scenario where multiple streams are 231 combined in a single Manifest File (e.g. for ad insertion purposes). 233 2.1. Segmentation versus Fragmentation 235 All HAS implementations are based around a concept referred to as 236 chunking: the concept of having a server split content up in numerous 237 fixed duration chunks, which are independently decodable. By 238 sequentially requesting and receiving chunks, a client can recreate 239 and play out the content. An advantage of this mechanism is that it 240 allows a client to seamlessly switch between different encodings of 241 the same Original Content at chunk boundaries. Before requesting a 242 particular chunk, a client can choose between multiple alternative 243 encodings of the same chunk, irrespective of the encoding of the 244 chunks it has requested earlier. 246 While every HAS implementation uses some form of chunking, not all 247 implementations store the resulting chunks in the same way. In 248 general, there are two distinct methods of performing chunking and 249 storing the results: segmentation and fragmentation. 251 - With segmentation, which is for example mandatory in all versions 252 of Apple's HLS prior to version 7, the chunks, in this case also 253 referred to as segments, are stored completely independent from 254 each other, with each segment being stored as a separate file from 255 a file system perspective. This means that each segment has its 256 own unique URL with which it can be retrieved. 258 - With fragmentation (or virtual segmentation), which is for example 259 used in Microsoft's Smooth Streaming, all chunks, or fragments, 260 belonging to the same Chunk Collection are stored together, as 261 part of a single file. While there are a number of container 262 formats which allow for storing this type of chunked content, 263 Fragmented MP4 is most commonly used. With fragmentation, a 264 specific chunk is addressable by subfixing the common file URL 265 with an identifier uniquely identifying the chunk that one is 266 interested in, either by timestamp, by byterange, or in some other 267 way. 269 While one can argue about the merits of each of these two different 270 methods of handling chunks, both have their advantages and drawbacks 271 in a CDN environment. For example, fragmentation is often regarded 272 as a method that introduces less overhead, both from a storage and 273 processing perspective. Segmentation on the other hand, is regarded 274 as being more flexible and easier to cache. In practice, current HAS 275 implementations increasingly support both methods. 277 2.2. Addressing chunks 279 In order for a client to request chunks, either in the form of 280 segments or in the form of fragments, it needs to know how the 281 content has been chunked and where to find the chunks. For this 282 purpose, most HAS protocols use a concept that is often referred to 283 as a Manifest File (also known as Media Presentation Description, or 284 MPD); i.e. a file that lists the way the content has been chunked 285 and where the various chunks are located (in the case of segments) or 286 how they can be addressed (in the case of fragments). A Manifest 287 File, or set of Manifest Files, may also identify the different 288 representations, and thus Chunk Collections, available for a content. 290 In general, a HAS client will first request and receive a Manifest 291 File, and then, after parsing the information in the Manifest File, 292 proceed with sequentially requesting the chunks listed in the 293 Manifest File. Each HAS implementation has its own Manifest File 294 format and even within a particular format there are different 295 methods available to specify the location of a chunk. 297 Of course managing the location of files is a core aspect of every 298 CDN, and each CDN will have its own method of doing so. Some CDNs 299 may be purely cache-based, with no higher-level knowledge of where 300 each file resides at each instant in time. Other CDNs may have 301 dedicated management nodes which, at each instant in time, do know at 302 which servers each file resides. The CDNI interfaces designed in the 303 CDNI WG will probably need to be agnostic to these kinds of CDN- 304 internal architecture decisions. In the case of HAS there is a 305 strict relationship between the location of the content in the CDN 306 (in this case chunks) and the content itself (the locations specified 307 in the Manifest File). It is therefore useful to have an 308 understanding of the different methods in use in CDNs today for 309 specifying chunk locations in Manifest Files. The different methods 310 for doing so are described in sections 2.2.1 to 2.2.3. 312 Although these sections are especially relevant for segmented 313 content, due to its inherent distributed nature, the discussed 314 methods are also applicable to fragmented content. Furthermore, it 315 should be noted that the methods detailed below for specifying 316 locations of content items in Manifest Files do not only relate to 317 temporally segmented content (e.g. segments and fragments), but are 318 also relevant in situations where content is made available in 319 multiple representations (e.g., in different qualities, encoding 320 methods, resolutions and/or bitrates). In this case the content 321 consists of multiple chunk collections, which may be described by 322 either a single Manifest File or multiple interrelated Manifest 323 Files. In the latter case, there may be a high-level Manifest File 324 describing the various available bitrates, with URLs pointing to 325 separate Manifest Files describing the details of each specific 326 bitrate. For specifying the locations of the other Manifest Files, 327 the same methods apply that are used for specifying chunk locations. 329 One final note relates to the delivery of the Manifest Files 330 themselves. While in most situations the delivery of both the 331 Manifest File and the chunks are handled by the CDN, there are 332 scenarios imaginable in which the Manifest File is delivered by e.g. 333 the Content Provider, and the Manifest File is therefore not visible 334 to the CDN. 336 2.2.1. Relative URLs 337 One method for specifying chunk locations in a Manifest File is 338 through the use of relative URLs. A relative URL is a URL that does 339 not include the HOST part of a URL but only includes (part of) the 340 PATH part of a URL. In practice, a relative URL is used by the 341 client as being relative to the location where the Manifest File has 342 been acquired from. In these cases a relative URL will take the form 343 of a string that has to be appended to the location of the Manifest 344 File to get the location of a specific chunk. This means that in the 345 case a Manifest File with relative URLs is used, all chunks will be 346 delivered by the same surrogate that delivered the Manifest File. A 347 relative URL will therefore not include a hostname. 349 For example, in the case a Manifest File has been requested (and 350 received) from: 352 http://surrogate.server.cdn.example.com/content_1/manifest.xml 354 , a relative URL pointing to a specific segment referenced in the 355 Manifest File might be: 357 segments/segment1_1.ts 359 Which means that the client should take the location of the Manifest 360 File and append the relative URL. In this case, the segment would 361 then be requested from http://surrogate.server.cdn.example.com/ 362 content_1/segments/segment1_1.ts 364 The downside of using relative URLs is that it forces a CDN to 365 deliver all segments belonging to a given content item with the same 366 surrogate that delivered the Manifest File for that content item, 367 which results in limited flexibility. Another drawback is that 368 Relative URLs do not allow for fallback URLs; should the surrogate 369 that delivered the Manifest File break down, the client is no longer 370 able to request chunks. The advantage of relative URLs is that it is 371 very easy to transfer content between different surrogates and even 372 CDNs. 374 2.2.2. Absolute URLs with Redirection 376 Another method for specifying locations of chunks (or other Manifest 377 Files) in a Manifest File is through the use of an absolute URL. An 378 absolute URL contains a fully formed URL (i.e. the client does not 379 have to calculate the URL as in the case of the relative URL but can 380 use the URL from the Manifest File directly). 382 In the context of Manifest Files, there are two types of absolute 383 URLs imaginable: Absolute URLs with Redirection and Absolute URLs 384 without Redirection. The two methods differ in whether the URL 385 points to a request routing node which will redirect the client to a 386 surrogate (Absolute URL with Redirection) or point directly to a 387 surrogate hosting the requested content (Absolute URL without 388 Redirection). 390 In the case of Absolute URLs with Redirection, a request for a chunk 391 is handled by the request routing system of a CDN just as if it were 392 a standalone (non-HAS) content request, which might include looking 393 up the surrogate (and/or CDN) best suited for delivering the 394 requested chunk to the particular user and sending an HTTP redirect 395 to the user with the URL pointing to the requested chunk on the 396 specified surrogate (and/or CDN), or a DNS response pointing to the 397 specific surrogate. 399 An example of an Absolute URL with Redirection might look as follows: 401 http://requestrouting.cdn.example.com/ 402 content_request?content=content_1&segment=segment1_1.ts 404 As can be seen from this example URL, the URL includes a pointer to a 405 general CDN request routing function and includes some arguments 406 identifying the requested segment. 408 The advantage of using Absolute URLs with Redirection is that it 409 allows for maximum flexibility (since chunks can be distributed 410 across surrogates and CDN in any imaginable way) without having to 411 modify the Manifest File every time one or more chunks are moved (as 412 is the case when Absolute URLs without Redirection are used). The 413 downside of this method is that it can adds significant load to a CDN 414 request routing system, since it has to perform a redirect every time 415 a client requests a new chunk. 417 2.2.3. Absolute URL without Redirection 419 In the case of the Absolute URL without Redirection, the URL points 420 directly to the specific chunk on the actual surrogate that will 421 deliver the requested chunk to the client. In other words, there 422 will be no HTTP redirection operation taking place between the client 423 requesting the chunk and the chunk being delivered to the client by 424 the surrogate. 426 An example of an Absolute URLs without Redirection is the following: 428 http://surrogate.cdn.example.com/content_1/segments/segment1_1.ts 430 As can be seen from this example URL, the URL includes both the 431 identifier of the requested segment (in this case segment1_1.ts), as 432 well as the server that is expected to deliver the segment (in this 433 case surrogate.cdn.example.com). With this, the client has enough 434 information to directly request the specific segment from the 435 specified surrogate. 437 The advantage of using Absolute URLs without Redirection is that it 438 allows more flexibility compared to using Relative URLs (since 439 segments do not necessarily have to be delivered by the same server) 440 while not requiring per-segment redirection (which would add 441 significant load to the node doing the redirection). The drawback of 442 Absolute URLs without Redirection is that it requires a modification 443 of the Manifest File every time content is moved to a different 444 location (either within a CDN or across CDNs). 446 2.3. Live vs. VoD 448 Though the formats and addresses of Manifest Files and chunk files do 449 not typically differ significantly between live and Video-on-Demand 450 (VoD) content, the time at which the Manifest Files and chunk files 451 become available does differ significantly. For live content, chunk 452 files and their corresponding Manifest Files are created and 453 delivered in real-time. This poses a number of potential issues for 454 HAS optimization: 456 - With live content, chunk files are made available in real-time. 457 This limits the applicability of bundling for content acquisition 458 purposes. Prepositioning may still be employed, however, any 459 significant latency in the prepositioning may diminish the value 460 of prepositioning if a client requests the chunk prior to 461 prepositioning, or if the prepositioning request is serviced after 462 the chunk playout time has passed. 464 - In the case of live content, Manifest Files must be updated for 465 each chunk and therefore must be retrieved by the client prior to 466 each chunk request. Any Manifest-File based optimization schemes 467 must therefore be prepared to optimize on a per-segment request 468 basis. Manifest Files may also be polled multiple times prior to 469 the actual availability of the next chunk. 471 - Since live Manifest Files are updated as each new chunk becomes 472 available, the cacheability of Manifest Files is limited. Though 473 timestamping and reasonable TTLs can improve delivery performance, 474 timely replication and delivery of updated Manifest Files is 475 critical to ensuring uninterrupted playback. 477 - Manifest Files are typically updated after the corresponding chunk 478 is available for delivery, to prevent premature requests for 479 chunks which are not yet available. HAS optimization approaches 480 which employ dynamic Manifest File generation must be synchronized 481 with chunk creation to prevent playback errors. 483 2.4. Stream splicing 485 Stream splicing is used to create media mashups, combining content 486 from multiple sources. A common example in which content resides 487 outside the CDNs is with advertisement insertion, for both VoD and 488 live streams. Manifest Files which contain Absolute URLs with 489 redirection may contain chunk or nested Manifest File URLs which 490 point to content not delivered via any of the interconnected CDNs. 492 Furthermore, client and downstream proxy devices may depend on non- 493 URL information provided in the Manifest File (e.g., comments or 494 custom tags) for performing stream splicing. This often occurs 495 outside the scope of the interconnected CDNs. HAS optimization 496 schemes which employ dynamic Manifest File generation or rewriting 497 must be cognizant of chunk URLs, nested Manifest File URLs, and other 498 metadata which should not be modified or removed. Improper 499 modification of these URLs or other metadata may cause playback 500 interruptions, and in the case of unplayed advertisements, may result 501 in loss of revenue for content providers. 503 3. Possible HAS Optimizations 505 In the previous chapter, some of the unique properties of HAS have 506 been discussed. Furthermore, some of the CDN-specific design 507 decisions with regards to addressing chunks have been detailed. In 508 this chapter, the impact of supporting HAS in CDN Interconnection 509 scenarios will be discussed. 511 There are a number of topics, or problem areas, that are of 512 particular interest when considering the combination of HAS and CDNI. 513 For each of these problem areas it holds that there are a number of 514 different ways in which the CDNI Interfaces can deal with them. In 515 general it can be said that each problem area can either be solved in 516 a way that minimizes the amount of HAS-specific changes to the CDNI 517 Interfaces or in way that maximizes the flexibility and efficiency 518 with which the CDNI Interfaces can deliver HAS content. The goal for 519 the CDNI WG should probably be to try to find the middle ground 520 between these two extremes and try to come up with solutions that 521 optimize the balance between efficiency and additional complexity. 523 In order to allow the WG to make this decision, this chapter will 524 briefly describe each of the following problem areas together with a 525 number of different options for dealing with them. Section 3.1 will 526 discuss the problem of how to deal with file management of groups of 527 files, or Content Collections. Section 3.2 will deal with a related 528 topic: how to do content acquisition of Content Collections between 529 the uCDN and dCDN. After that, Section 3.3 describes the various 530 options for the request routing of HAS content, particularly related 531 to Manifest Files. Section 3.4 talks about a number of possible 532 optimizations for the logging of HAS content, while Section 3.5 533 discusses the options regarding URL signing. Section 3.6 finally, 534 describes different scenarios for dealing with the removal of HAS 535 content from CDNs. 537 3.1. File Management and Content Collections 539 3.1.1. General Remarks 541 One of the unique properties of HAS content is that it does not 542 consist of a single file or stream but of multiple interrelated files 543 (segment, fragments and/or Manifest Files). In this document this 544 group of files is also referred to as a Content Collection. Another 545 important aspect is the difference between segments and fragments 546 (see Section 2.1). 548 Irrespective of whether segments or fragments are used, different 549 CDNs might handle Content Collections differently from a file 550 management perspective. For example, some CDNs might handle all 551 files belonging to a Content Collection as individual files, which 552 are stored independently from each other. An advantage of this 553 approach is that makes it easy to cache individual chunks. Other 554 CDNs might store all fragments belonging to a Content Collection in a 555 bundle, as if they were a single file (e.g. by using a fragmented 556 MP4 container). The advantage of this approach is that it reduces 557 file management overhead. 559 This section will look at the various ways with which the CDNI 560 interfaces might deal with these differences in handling Content 561 Collections from a file management perspective. The different 562 options can be distinguished based on the level of HAS-awareness they 563 require on the part of the different CDNs and the CDNI interfaces. 565 3.1.2. Candidate approaches 567 3.1.2.1. Option 1.1: No HAS awareness 569 This first option assumes no HAS awareness in both the involved CDNs 570 and the CDNI Interfaces. This means that the uCDN uses individual 571 files and the dCDN is not explicitly made aware of the relationship 572 between chunks and it doesn't know which files are part of the same 573 Content Collection. In practice this scenario would mean that the 574 file management method used by the uCDN is simply imposed on the dCDN 575 as well. 577 This scenario also means that it is not possible for the dCDN to use 578 any form of file bundling, such as the single-file mechanism which 579 can be to store fragmented content as a single file (see 580 Section 2.1). The one exception to this rule is the situation where 581 the content is fragmented and the Manifest Files on the uCDN contains 582 byte range requests, in which case the dCDN might be able to acquire 583 fragmented content as a single file (see Section 3.2.2.2). 585 Effect on CDNI interfaces: 587 o None 589 Advantages/Drawbacks: 591 + No HAS awareness necessary in CDNs, no changes to CDNI Interfaces 592 necessary 594 - The dCDN is forced to store chunks as individual files. 596 3.1.2.2. Option 1.2: Allow single file storage of fragmented content 598 In some cases, the dCDN might prefer to store fragmented content as a 599 single file on its surrogates to reduce file management overhead. In 600 order to do so, it needs to be able to either acquire the content as 601 a single file (see Section 3.2.2.2), or merge the different chunks 602 together and place them in the same container (e.g. fragmented MP4). 603 The downside of this is that in order to do so, the dCDN needs to be 604 fully HAS aware. 606 Effect on CDNI interfaces: 608 o CDNI Metadata Interface: Add fields for indicating the particular 609 type of HAS (e.g. MPEG DASH or HLS) that is used and whether 610 segments or fragments are used 612 o CDNI Metadata Interface: Add field for indicating the name and 613 type of the Manifest File(s) 615 Advantages/Drawbacks: 617 + Allows dCDN to store fragmented content as a single file, reducing 618 file management overhead 620 - Complex operation, requiring dCDN to be fully HAS aware 622 3.1.2.3. Option 1.3: Access correlation hint 624 An intermediary approach between the two extremes detailed in the 625 previous two sections is one that uses a 'Access Correlation Hint'. 626 This hint, which is added to the CDNI Metadata of all chunks of a 627 particular Content Collection, indicates that those files are likely 628 to be requested in a short time window from each other. This 629 information can help a dCDN to implement local file storage 630 optimizations for VoD items (e.g. by bundling all files with the 631 same Access Correlation Hint value in a single bundle/file), thereby 632 reducing the number of files it has to manage while not requiring any 633 HAS awareness. 635 Effect on CDNI interfaces: 637 o CDNI Metadata Interface: Add field for indicating Access 638 Correlation Hint 640 Advantages/Drawbacks: 642 + Allows dCDN to perform file management optimization 644 + Does not require any HAS awareness 646 + Very small impact on CDNI Interfaces 648 - Expected benefit compared with Option 1.1 is small 650 3.1.3. Recommendation 652 Based on the listed pros and cons, the authors recommend the WG to go 653 for Option 1.1, the 'Do Nothing'-approach. The likely benefits from 654 going for Option 1.3 are not believed to be significant enough to 655 warrant changing the CDNI Metadata Interface. Although Option 1.2 656 would bring definite benefits for HAS aware dCDNs, going for this 657 options would require significant CDNI extensions that would impact 658 the WG's milestones. The authors therefore don't recommend to 659 include it in the current work but mark it as a possible candidate 660 for rechartering once the initial CDNI solution is completed. 662 3.2. Content Acquisition of Content Collections 664 3.2.1. General Remarks 666 In the previous section the relationship between file management and 667 HAS in a CDNI scenario has been discussed. This section will discuss 668 a related topic, which is content acquisition between two CDNs. 670 With regards to content acquisition, it is important to note the 671 difference between CDNs that do Dynamic Acquisition of content and 672 CDNs that perform Content Pre-positioning. In the case of dynamic 673 acquisition, a CDN only requests a particular content item when a 674 cache-miss occurs. In the case of pre-positioning, a CDN pro- 675 actively places content items on the nodes on which it expects 676 traffic for that particular content item. For each of these types of 677 CDNs, there might be a benefit in being HAS aware. For example, in 678 the case of dynamic acquisition, being HAS aware means that after a 679 cache miss for a giving chunk occurs, that node might not only 680 acquire the requested chunk, but might also acquire some related 681 chunks that are expected to be requested in the near future. In the 682 case of pre-positioning, similar benefits can be had. 684 3.2.2. Candidate Approaches 686 3.2.2.1. Option 2.1: No HAS awareness 688 This first option assumes no HAS awareness in both the involved CDNs 689 and the CDNI Interfaces. Just as with Option 1.1 discussed in the 690 previous section with regards to file management, having no HAS 691 awareness means that the dCDN is not aware of the relationship 692 between chunks. In the case of content acquisition, this means that 693 each and every file belonging to a Content Collection will have to be 694 individually acquired from the uCDN by the dCDN. The exception to 695 the rule is in cases with fragmented content where the uCDN uses 696 Manifest Files which contain byte range requests. In this case the 697 dCDN can simply omit the byte range identifier and acquire the 698 complete file. 700 The advantage of this approach is that it is highly flexible. If a 701 client only requests a small portion of the chunks belonging to a 702 particular Content Collection, the dCDN only has to acquire those 703 chunks from the uCDN, saving both bandwidth and storage capacity. 705 The downside of acquiring content on a per-chunk basis is that it 706 creates more transaction overhead between the dCDN and uCDN compared 707 to a method in which entire Content Collections can be acquired as 708 part of one transaction. 710 Effect on CDNI interfaces: 712 o None 714 Advantages/Drawbacks: 716 + Per-chunk content acquisition allows for high level of flexibility 717 between dCDN and uCDN 719 - Per-chunk content acquisition creates more transaction overhead 720 between dCDN and uCDN 722 3.2.2.2. Option 2.2: Allow single file acquisition of fragmented 723 content 725 As discussed in Section 3.2.2.1, there is one (fairly rare) case 726 where fragmented content can be acquired as a single file without any 727 HAS awareness and that is when fragmented content is used and where 728 the Manifest File specifies byte range request. This section 729 discusses how to perform single file acquisition in the other (very 730 common) cases. To do so, the dCDN would have to have full-HAS 731 awareness (at least to the extent of being able to map between single 732 file and individual chunks to serve). 734 Effect on CDNI interfaces: 736 o CDNI Metadata Interface: Add fields for indicating the particular 737 type of HAS (e.g. MPEG DASH or HLS) that is used and whether 738 segments or fragments are used 740 o CDNI Metadata Interface: Add field for indicating the name and 741 type of the Manifest File(s) 743 Advantages/Drawbacks: 745 + Allows for more efficient content acquisition in all HAS-specific 746 supported forms 748 - Requires full HAS awareness on part of dCDN 750 - Requires significant CDNI Metadata Interface extensions 752 3.2.3. Recommendation 754 Based on the listed pros and cons, the authors recommend the WG to go 755 for Option 2.1 since it is sufficient to 'make HAS work'. While 756 Option 2.2 would bring benefits to the acquisition of large Content 757 Collections, it would require significant CDNI extensions which would 758 impact the WG's milestones. Option 2.2 might be a candidate to 759 include in possible rechartering once the initial CDNI solution is 760 completed. 762 3.3. Request Routing of HAS content 764 3.3.1. General remarks 766 In this section the effect HAS content has on request routing will be 767 identified. Of particular interest in this case are the different 768 types of Manifest Files that might be used. In Section 2.2, three 769 different methods for identifying and addressing chunks from within a 770 Manifest File were described: Relative URLs, Absolute URLs without 771 Redirection and Absolute URLs with Redirection. Of course not every 772 current CDN will use and/or support all three methods. Some CDNs may 773 only use one of the three methods, while others may support two or 774 all three. 776 An important factor in deciding which chunk addressing method is used 777 is the Content Provider. Some Content Providers may have a strong 778 preference for a particular method and deliver the Manifest Files to 779 the CDN in a particular way. Depending on the CDN and the agreement 780 it has with the Content Provider, a CDN may either host the Manifest 781 Files as they were created by the Content Provider, or modify the 782 Manifest File to adapt it to its particular architecture (e.g. by 783 changing relative URLs to Absolute URLs which point to the CDN 784 Request Routing function). 786 3.3.2. Candidate approaches 788 3.3.2.1. Option 3.1: No HAS awareness 790 This first option assumes no HAS awareness in both the involved CDNs 791 and the CDNI Interfaces. This scenario also assumes that neither the 792 dCDN nor the uCDN have the ability to actively manipulate Manifest 793 Files. As was also discussed with regards to file management and 794 content acquisition, having no HAS awareness means that each file 795 constituting a Content Collections is handled on an individual basis, 796 with the dCDN unaware of any relationship between files. 798 The only chunk addressing method that works without question in this 799 case is Absolute URLs with Redirection. In other words, the Content 800 Provider that ingested the content into the uCDN created a Manifest 801 File with each chunk location pointing to the Request Routing 802 function of the uCDN. Alternatively, the Content Provider may have 803 ingested the Manifest File containing relative URLs and the uCDN 804 ingestion function has translated these to Absolute URLs pointing to 805 the Request Routing function. 807 In this Absolute URL with Redirection case, the uCDN can simply have 808 the Manifest File be delivered by the dCDN as if it were a regular 809 file. Once the client parses the Manifest File, it will request any 810 subsequent chunks from the uCDN Request Routing function. That 811 function can then decide to outsource the delivery of that chunk to 812 the dCDN. Depending on whether HTTP-based (recursive or iterative) 813 or DNS-based request routing is used, the uCDN Request Routing 814 function will then either directly or indirectly redirect the client 815 to the Request Routing function of the dCDN (assuming it does not 816 have the necessary information to redirect the client directly to a 817 surrogate in the dCDN). 819 The drawback of this method is that it creates a large amount of 820 request routing overhead for both the uCDN and dCDN. For each chunk 821 the full inter-CDN Request Routing process is invoked (which can 822 result in two HTTP redirections in the case of iterative redirection, 823 or result in one HTTP redirection plus one CDNI Request Routing/ 824 Redirection Interface request/response). Even in the case where DNS- 825 based redirection is used, there might be significant overhead 826 involved since both the dCDN and uCDN Request Routing function might 827 have to perform database lookups and query each other. While with 828 DNS this overhead might be reduced by using DNS' inherent caching 829 mechanism, this will have significant impact on the accuracy of the 830 redirect. 832 With no HAS awareness, Relative URLs might or might not work 833 depending on the type of Relative URL that is used. When a uCDN 834 delegates the delivery of a Manifest File containing Relative URLs to 835 a dCDN, the client goes directly to the dCDN surrogate from which it 836 has received the Manifest File for every subsequent chunk. As long 837 as the Relative URL is not path-absolute (see [RFC3986]), this 838 approach will work fine. 840 Since using Absolute URLs without Redirection inherently require a 841 HAS aware CDN, they also cannot be used in this case. The reason for 842 this is that with Absolute URLs without Redirection, the URLs in the 843 Manifest File will point directly to a surrogate in the uCDN. Since 844 this scenario assumes no HAS awareness on the part of the dCDN or 845 uCDN, it is impossible for either of these CDNs to rewrite the 846 Manifest File and thus allow the client to either go to a surrogate 847 in the dCDN or to a request routing function. 849 Effect on CDNI interfaces: 851 o None 853 Advantages/Drawbacks: 855 + Supports Absolute URLs with Redirection 857 + Supports Relative URLs 858 + Does not require HAS awareness and/or changes to the CDNI 859 Interfaces 861 - Not possible to use Absolute URLs without Redirection 863 - Creates significant signaling overhead in case Absolute URLs with 864 Redirection are used (inter-CDN request redirection for each 865 chunk) 867 3.3.2.2. Option 3.2: Manifest File rewriting by uCDN 869 While Option 3.1 does allow for Absolute URLs with Redirection to be 870 used, it does so in a way that creates a high-level of request 871 routing overhead for both the dCDN and the uCDN. This option 872 presents a solution to significantly reduce this overhead. 874 In this scenario, the uCDN is able to rewrite the Manifest File (or 875 generate a new one) to be able to remove itself from the request 876 routing chain for chunks being referenced in the Manifest File. As 877 described in Section 3.3.2.1, in the case of no HAS awareness the 878 client will go to the uCDN request routing function for each chunk 879 request. This request routing function can then redirect the client 880 to the dCDN request routing function. By rewriting the Manifest File 881 (or generating a new one), the uCDN is able to remove this first 882 step, and have the Manifest File point directly to the dCDN request 883 routing function. 885 A key advantage of this solution is that it does not directly have an 886 impact on the CDNI Interfaces and is therefore transparent to these 887 interfaces. It is a CDN-internal function that a uCDN can perform 888 autonomously by using information configured for regular CDNI 889 operation or that is received from the dCDN as part of the regular 890 communication using the CDNI Request Routing/Redirection Interface. 892 More specifically, in order for the uCDN to rewrite the Manifest 893 File, the minimum information needed is the location of the dCDN 894 request routing function (or alternatively the location of the dCDN 895 delivering surrogate). This information can be available from 896 configuration or can be derived from the regular CDNI Request Routing 897 /Redirection Interface. For example, the uCDN may ask the dCDN for 898 the location of its request routing node (through the CDNI Request 899 Routing/Redirection Interface) every time a request for a Manifest 900 File is received and processed by the uCDN request routing function. 901 The uCDN would then modify the Manifest File and deliver the Manifest 902 File to the client. One advantage of this method is that it 903 maximizes efficiency and flexibility by allowing the dCDN to 904 optionally respond with the locations of its surrogates instead of 905 the location of its request routing function (and effectively turning 906 the URLs into Absolute URLs without Redirection). There are many 907 variations around this approach, such as where the modification of 908 the Manifest File in only performed once (or once per period of time) 909 by the uCDN request routing function, when the first client for that 910 particular Content Collection (and redirected to that particular 911 dCDN) sends a Manifest File request. The advantage of such a 912 variation is that the uCDN only has to modify the Manifest File once 913 (or once per time period). The drawback of this variation is that 914 the dCDN is no longer in a position to influence the request routing 915 decision across individual content requests. 917 It should be noted that there are a number of things to take into 918 account when changing a Manifest File (see for example Section 2.3 919 and Section 2.4 on live HAS content and ad insertion). Furthermore, 920 some Content Providers might have issues with a CDN changing Manifest 921 Files. However, in this option the Manifest File manipulation is 922 only being performed by the uCDN, which can be expected to be aware 923 of these limitations if it wants to perform Manifest File 924 manipulation since it is in its own best interest that its customer's 925 content gets delivered in the proper way and since there is a direct 926 commercial and technical relationship between the uCDN (the 927 Authoritative CDN in this scenario) and its customer (the Content 928 Provider). Should the Content Provider want to limit Manifest File 929 manipulation, it can simply arrange this with the uCDN bilaterally. 931 Effect on CDNI interfaces: 933 o None 935 Advantages/Drawbacks: 937 + Possible to significantly decrease signaling overhead when using 938 Absolute URLs. 940 + (Optional) Possible to have uCDN rewrite the Manifest File with 941 locations of surrogates in dCDN (turning Absolute URLs with 942 Redirection in Absolute URLs without Redirection) 944 + No changes to CDNI Interfaces 946 + Does not require HAS awareness in dCDN 948 - Requires high level of HAS awareness in uCDN (for modifying 949 Manifest Files) 951 3.3.2.3. Option 3.3: Two-step Manifest File rewriting 952 One of the possibilities with Option 3.2 is allowing the dCDN to 953 provide the locations of a specific surrogate to the uCDN, so that 954 the uCDN can fit the Manifest File with Absolute URLs without 955 Redirection and the client can request chunks directly from a dCDN 956 surrogate. However, some dCDNs might not be willing to provide this 957 information to the uCDN. In that case they can only provide the uCDN 958 with the location of their request routing function and thereby 959 preventing use of Absolute URLs without Redirection. 961 One method for solving this limitation is allowing two-step Manifest 962 File manipulation. In the first step the uCDN would perform its own 963 modification, and place the locations of the dCDN request routing 964 function in the Manifest File. Then, once a request for the Manifest 965 File comes in at the dCDN request routing function, it would perform 966 a second modification in which it replaces the URLs in the Manifest 967 Files with the URLs of its surrogates. This way the dCDN can still 968 profit from having limited request routing traffic, while not having 969 to share sensitive surrogate information with the uCDN. 971 The downside of this approach is that it not only assumes HAS 972 awareness in the dCDN but it also requires some HAS-specific 973 additions to the CDNI Metadata Interface. In order for the dCDN to 974 be able to change the Manifest File, it has to have some information 975 about the structure of the content. Specifically, it needs to have 976 information about which chunks make up the Content Collection. 978 Effect on CDNI interfaces (apart from those already listed under 979 Option 3.2): 981 o CDNI Metadata Interface: Add necessary fields for conveying HAS 982 specific information (e.g. the files that make up the Content 983 Collection) to the dCDN. 985 o CDNI Metadata Interface: Allow dCDN to modify Manifest File 987 Advantages/Drawbacks (apart from those already listed under Option 988 3.2): 990 + Allows dCDN to use Absolute URLs without Redirection without 991 having to convey sensitive information to the uCDN 993 - Requires high level of HAS awareness in dCDN (for modifying 994 Manifest Files) 996 - Requires adding HAS-specific and Manifest File manipulation 997 specific information to the CDNI Metadata Interface 999 3.3.3. Recommendation 1001 Based on the listed pros and cons, the authors recommend to go for 1002 Option 3.1, with Option 3.2 as an optional feature that may be 1003 supported as a CDN-internal behavior by an uCDN. While Option 3.1 1004 allows for HAS content to be delivered using the CDNI interfaces, it 1005 does so with some limitations regarding supported Manifest Files and, 1006 in some cases, with large signaling overhead. Option 3.2 can solve 1007 most of these limitations and presents a significant reduction of the 1008 request routing overhead. Since Option 3.2 does not require any 1009 changes to the CDNI interfaces but only changes the way the uCDN uses 1010 the existing interfaces, supporting it is not expected to result in a 1011 significant delay of the WG's milestones. The authors recommend the 1012 WG to not include Option 3.3, since it raises some questions of 1013 potential brittleness and including it would result in a significant 1014 delay of the WG's milestones. 1016 3.4. Logging 1018 3.4.1. General remarks 1020 As stated in [RFC6707], "the CDNI Logging interface enables details 1021 of logs or events to be exchanged between interconnected CDNs". 1023 As discussed in [I-D.draft-bertrand-cdni-logging], the CDNI logging 1024 information can be used for multiple purposes including maintenance/ 1025 debugging by uCDN, accounting (e.g. in view of billing or 1026 settlement), reporting and management of end-user experience (e.g. 1027 to the CSP), analytics (e.g. by the CSP) and control of content 1028 distribution policy enforcement (e.g. by the CSP). 1030 The key consideration for HAS with respect to logging is the 1031 potential increase of the number of Log records by two to three 1032 orders of magnitude, as compared to regular HTTP delivery of a video, 1033 since, by default, log records would typically be generated on a per- 1034 chunk-delivery basis instead of per-content-item-delivery basis. 1035 This impacts the scale of every processing step in the Logging 1036 Process (see [I-D.draft-bertrand-cdni-logging]), including: 1038 a. Logging information generation and storing on CDN elements 1039 (Surrogate, Request Routers, ...) 1041 b. Logging information aggregation within a CDN 1043 c. Logging information manipulation (including information 1044 protection, filtering, update and rectification) 1046 d. (Where needed) Logging information CDNI reformatting (e.g. 1047 reformatting from CDN-specific format to the CDNI Logging 1048 Interface format for export by dCDN to uCDN) 1050 e. Logging exchange via CDNI Logging Interface 1052 f. (Where needed) Logging re-reformatting (e.g. reformatting from 1053 CDNI Logging Interface format into log-consuming specific 1054 application) 1056 g. Logging consumption/processing (e.g. feed logs into uCDN 1057 accounting application, feed logs into uCDN reporting system to 1058 provide per CSP views, feed logs into debugging tools) 1060 Note that there may be multiple instances of step [f] and [g] running 1061 in parallel. 1063 While the CDNI Logging Interface is only used to perform step [e], we 1064 note that its format directly affects step [d] and [f] and that its 1065 format also indirectly affects the other steps (for example if the 1066 CDNI Logging Interface requires per-chunk log records, step [a], [b] 1067 and [d] cannot operate on a per-HAS-session basis and they also need 1068 to operate on a per-chunk basis). 1070 This section discusses the main candidate approaches identified for 1071 CDNI in terms of dealing with HAS with respect to Logging. 1073 3.4.2. Candidate Approaches 1075 3.4.2.1. Option 4.1: "Do-Nothing" Approach 1077 In this approach nothing is done specifically for HAS so that each 1078 HAS-chunk delivery is considered, for CDNI Logging, as a standalone 1079 content delivery. In particular, a separate log record for each HAS- 1080 chunk delivery is included in the CDNI Logging Interface in step [e] 1081 (as defined in Section 3.4.1). This approach requires that step [a], 1082 [b], [c], [d] and [e] also be performed on a per-chunk basis. This 1083 approach allows [g] to be performed either on a per-chunk basis 1084 (assuming step [f] maintains per-chunk records) or on a more 1085 "summarized" manner such as per-HAS-Session basis (assuming step [f] 1086 summarizes per-chunk records into per-HAS-session records). 1088 Effect on CDNI interfaces: 1090 o None 1092 Advantages/Drawbacks: 1094 + No information loss (i.e. all details of each individual chunk 1095 delivery are preserved). While this full level of detail may not 1096 be needed for some Log consuming applications (e.g. billing), 1097 this full level of detail is likely valuable (possibly required) 1098 for some Log consuming applications (e.g. debugging) 1100 + Easier integration (at least in the short term) into existing 1101 Logging tools since those are all capable of handling per-chunk 1102 records 1104 + No extension needed on CDNI interfaces 1106 - High volume of logging information to be handled (storing & 1107 processing) at every step of the Logging process from [a] to [g] 1108 (while summarization in step [f] is conceivable, it may be 1109 difficult to achieve in practice without any hints for correlation 1110 in the log records). 1112 An interesting question is whether a dCDN could use the CDNI Logging 1113 interface specified for the "Do-Nothing" approach to report 1114 summarized "per-session" log information in the case where the dCDN 1115 performs such summarization. The high level idea would be that, when 1116 a dCDN performs HAS log summarization for its own purposes anyways, 1117 this dCDN could include, in the CDNI Logging interface, one (or a 1118 few) log entry for a HAS session (instead of one entry per HAS-chunk) 1119 that summarizes the deliveries of many/all HAS-chunk for a session. 1120 However, the authors feel that, when considering the details of this, 1121 this is not achievable without explicit agreement between the uCDN 1122 and dCDN about how to perform/interpret such summarization. For 1123 example, when a HAS session switches between representations, the 1124 uCDN and dCDN would have to agree on things such as: 1126 o whether the session will be represented by a single log entry 1127 (which therefore cannot convey the distribution across 1128 representations) or multiple log entries such as one entry per 1129 contiguous period at a given representation (which therefore would 1130 be generally very difficult to correlate back into a single 1131 session) 1133 o what would the single URI included in the log entry correspond to? 1134 the Manifest/top-level-playlist/next-level-playlist,... 1136 The authors feel that since explicit agreement is needed between uCDN 1137 and dCDN on how to perform/interpret the summarization, then, to this 1138 can only work if specified as part of the CDNI Logging interface and 1139 then effectively boils down to Option 4.4 defined below ("Full HAS 1140 awareness/per-Session-Logs" Approach). 1142 We note that support by CDNI of a mechanism (independent of HAS) 1143 allowing the customization of the fields to be reported in log 1144 entries by the dCDN to the uCDN would have a mitigation effect on the 1145 HAS logging scaling concerns because it ensures that only the 1146 necessary subset of fields are actually stored, reported and 1147 processed. 1149 3.4.2.2. Option 4.2: "CDNI Metadata Content Collection ID" Approach 1151 In this approach, a "Content Collection IDentifier (CCID)" field is 1152 distributed through the CDNI Metadata Interface and the same CCID 1153 value is associated through the CDNI Metadata interface with every 1154 chunk of the same Content Collection. The CCID value needs to be 1155 such that it allows, in combination with the content URI, to uniquely 1156 identify a Content Collection. When distributed, and CCID logging is 1157 requested from the dCDN, the dCDN Surrogates are to store the CCID 1158 value in the corresponding log entries. The objective of this field 1159 is to facilitate optional summarization of per-chunk records at step 1160 [f] into something along the lines of per-HAS-session logs, at least 1161 for the Log consuming applications that do not require per-chunk 1162 detailed information (for example billing). 1164 We note that, if the downstream CDN happens to have sufficient HAS 1165 awareness to be able to generate a "Session IDentifier (Session-ID)", 1166 optionally including such Session-ID (in addition to the CCID) in the 1167 per-chunk log record would further facilitate optional summarization 1168 performed at step [f]. The Session-ID value to be included in a log 1169 record by the delivering CDN is such that 1171 o different per-chunk log records with the same Session-ID value 1172 must correspond to the same user session (i.e delivery of same 1173 content to same enduser at a given point in time). 1175 o log records for different chunks of the same user session (i.e 1176 delivery of same content to same enduser at a given point in time) 1177 should be provided with the same session-ID value. While 1178 undesirable, there may be situations where the delivering CDN uses 1179 more than one session-ID value for different per-chunk log records 1180 of a given session, for example in scenarios of fail-over or load- 1181 balancing across multiple Surrogates and where the delivering CDN 1182 does not implement mechanisms to synchronize session-IDs across 1183 Surrogates. 1185 Effect on CDNI interfaces: 1187 o CDNI Metadata interface: One additional metadata field (CCID) in 1188 CDNI Metadata Interface. We note that a similar Content 1189 Collection ID is discussed for handling of other aspects of HAS 1190 and observe that further thought is needed to determine whether 1191 such CCID should be shared for multiple purposes or should be 1192 independent. 1194 o CDNI Logging interface: Two additional fields (CCID and Session- 1195 ID) in CDNI Logging records. 1197 Advantages/Drawbacks: 1199 + No information loss (i.e. all details of each individual chunk 1200 delivery are preserved). While this full level of detail may not 1201 be needed for some Log consuming applications (e.g. billing), 1202 this full level of detail is likely valuable (possibly required) 1203 for some Log consuming applications (e.g. debugging) 1205 + Easier integration (at least in the short term) into existing 1206 Logging tools since those are all capable of handling per-chunk 1207 records 1209 + Very minor extension to CDNI interfaces needed 1211 + Facilitated summarization of records related to a HAS session in 1212 step [f] and therefore ability to operate on lower volume of 1213 logging information in step [g] by log consuming applications that 1214 do not need per-chunk record details (e.g. billing) or that need 1215 per-session information (e.g. analytics) 1217 - High volume of logging information to be handled (storing & 1218 processing) at every step of the Logging process from [a] to [f]. 1220 3.4.2.3. Option 4.3: "CDNI Logging Interface Compression" Approach 1222 In this approach, a loss-less compression technique is applied to the 1223 sets of Logging records (e.g. Logging files) for transfer on the 1224 IETF CDNI Logging Interface. The objective of this approach is to 1225 reduce the volume of information to be stored and transferred in step 1226 [e]. 1228 Effect on CDNI interfaces: 1230 o One additional compression mechanism to be included in the CDNI 1231 Logging Interface 1233 Advantages/Drawbacks: 1235 + No information loss (i.e. all details of each individual chunk 1236 delivery are preserved). While this full level of detail may not 1237 be needed for some Log consuming applications (e.g. billing), 1238 this full level of detail is likely valuable (possibly required) 1239 for some Log consuming applications (e.g. debugging) 1241 + Easier integration (at least in the short term) into existing 1242 Logging tools since those are all capable of handling per-chunk 1243 records 1245 + Small extension to CDNI interfaces needed 1247 + Reduced volume of logging information in step [e] 1249 + Compression likely to be also applicable to logs for non-HAS 1250 content 1252 - High volume of logging information to be handled (storing & 1253 processing) at every step of the Logging process from [a] to [g], 1254 except [e]. 1256 3.4.2.4. Option 4.4: "Full HAS awareness/per-Session-Logs" Approach 1258 In this approach, HAS-awareness is assumed across the CDNs 1259 interconnected via CDNI and the necessary information to describe the 1260 HAS relationship across all chunks of the same Content Collection is 1261 distributed through the CDNI Metadata Interface. In this approach, 1262 the dCDN Surrogates leverage the HAS information distributed through 1263 the CDNI metadata and their HAS-awareness to generate summarized 1264 logging information in the very first place (or alternatively, if 1265 per-chunk-logs are generated, to accurately correlate and summarize 1266 per-chunk-logs into per-session logs) for exchange over the CDNI 1267 Logging interface. The objective of that approach is to operate on 1268 lower volume of logging information as early as possible in the 1269 successive steps of the Logging process. 1271 Effect on CDNI interfaces: 1273 o CDNI Metadata Interface: significant extension to convey HAS 1274 relationship across chunks of a Content Collection. Note that 1275 this extension requires specific support for every HAS-protocol to 1276 be supported over the CDNI mesh 1278 o CDNI Logging Interface: extension to specify summarized per- 1279 session logs 1281 Advantages/Drawbacks: 1283 + Lower volume of logging information to be handled (storing & 1284 processing) at every step of the Logging process from [a] to [g] 1286 + Accurate generation of summarized logs because of HAS awareness in 1287 dCDN (for example, where the Surrogate is also serving the 1288 Manifest File(s) for a content collection, the Surrogate may be 1289 able to extract definitive information about the relationship 1290 between all chunks) 1292 - Very significant extensions to CDNI interfaces needed including 1293 per HAS-protocol specific support 1295 - Very significant additional requirement for HAS awareness on dCDN 1296 and for this HAS-awareness to be consistent with the defined CDNI 1297 Logging summarization 1299 - Some information loss (i.e. all details of each individual chunk 1300 delivery are not preserved). The actual information loss depends 1301 on the summarization approach selected (typically the lower the 1302 information loss, the lower the summarization gain) so the right 1303 sweet-spot would have to be selected. While full level of detail 1304 may not be needed for some Log consuming applications (e.g. 1305 billing), the full level of detail is likely valuable (possibly 1306 required) for some Log consuming applications (e.g. debugging) 1308 - Less easy integration (at least in the short term) into existing 1309 Logging tools since those are all capable of handling per-chunk 1310 records and may not be capable of handling CDNI summarized records 1312 - Challenges in defining behavior (and achieving summarization gain) 1313 in the presence of load-balancing of a given HAS-session across 1314 multiple Surrogates (in same or different dCDN) 1316 3.4.3. Recommendation 1318 Because of its benefits (in particular simplicity, universal support 1319 by CDNs and support by all log-consuming applications), the authors 1320 recommend that the per-chunk logging of Option 4.1 be supported by 1321 the CDNI Logging interface as a "High Priority" (as defined in 1322 [I-D.draft-ietf-cdni-requirements]) and be a mandatory capability of 1323 CDNs implementing CDNI. 1325 Because of its very low complexity and its benefits in facilitating 1326 some useful scenarios (e.g. per-session analytics), we recommend 1327 that the CCID mechanisms and Session-ID mechanism of Option 4.2 be 1328 supported by the CDNI Metadata interface and the CDNI Logging 1329 interface as a "Medium Priority" (as defined in 1330 [I-D.draft-ietf-cdni-requirements]) and be an optional capability of 1331 CDNs implementing CDNI. 1333 The authors also recommend that: 1335 (i) the ability for the uCDN to request that the CCID and Session-ID 1336 field be included in log entries provided by the dCDN be supported 1337 by the relevant CDNI interfaces, and 1339 (ii) the ability for the dCDN to include the CCID field and Session- 1340 ID in CDNI log entries (when the dCDN is capable of doing so) and 1341 indicate so inside the CDNI Logging interface (in line with the 1342 "customizable" log format expected to be defined independently of 1343 HAS), 1345 be supported as a "Medium Priority" (as defined in 1346 [I-D.draft-ietf-cdni-requirements]) and be an optional capability of 1347 CDNs implementing CDNI. 1349 When performing dCDN selection, an uCDN may want to take into account 1350 whether a given dCDN is capable of reporting the CCID and Session-ID. 1351 Thus, the authors recommend that the ability for a dCDN to advertise 1352 its support of the optional CCID and Session-ID capability be 1353 supported by the CDNI request Routing /Footprint and Capabilities 1354 Advertisment Interface as a "Medium Priority" (as defined in 1355 [I-D.draft-ietf-cdni-requirements]). 1357 The authors also recommend that a generic mechanism (independent of 1358 HAS) be supported allowing the customization of the fields to be 1359 reported in logs by CDNs over the CDNI Logging Interface because of 1360 the reduction of the logging information volume exchanged across CDNs 1361 by removing the information that is not of interest to the other CDN. 1363 Because it can be achieved with very little complexity and it 1364 provides some clear storage/communication compression benefits, the 1365 authors recommend that, in line with the concept of Option 4.3, some 1366 existing very common compression techniques (e.g. gzip) be supported 1367 by the CDNI Logging interface as a "Medium Priority" (as defined in 1368 [I-D.draft-ietf-cdni-requirements]) and be an optional capability of 1369 CDNs implementing CDNI. 1371 Because of its complexity, the time it would take to understand the 1372 trade-offs of candidate summarization approaches and the time it 1373 would take to specify the corresponding support in the CDNI Logging 1374 interface, the authors recommend that the log summarization discussed 1375 in option 4.4 not be supported by the CDNI Logging interface at this 1376 stage and be kept as a candidate topic of great interest for a 1377 rechartering of the CDNI WG once the first set of deliverables is 1378 produced. When doing so, we suggest to investigate the notion of 1379 complementing the "push-style" CDNI logging interface supporting 1380 summarization by an on-demand pull-type of interface allowing an uCDN 1381 to request the subset of the detailed logging information that it may 1382 need but is lost in the summarized pushed information. 1384 The authors note that while a CDN only needs to adhere to the CDNI 1385 Logging interface on its external interfaces and can perform logging 1386 in a different format within the CDN, any possible CDNI Logging 1387 approach effectively places some constraints on the dCDN logging 1388 format. For example, to support the "Do-Nothing" Approach, a CDN 1389 need to perform and retain per chunk logs. As another example, to 1390 support the "Full HAS awareness/per-Session-Logs" Approach, the dCDN 1391 cannot operate on logging format that summarize "more than" or "in an 1392 incompatible way with" the summarization specified for CDNI Logging. 1393 However, the authors feel such constraints are (i) inevitable, (ii) 1394 outweighed by the benefits of a standardized logging interface and 1395 (iii) acceptable because in case of incompatible summarization, all/ 1396 most CDNs are capable of reverting to per-chunk logging as per the 1397 Do-Nothing Approach that we recommend as the base mandatory approach. 1399 3.5. URL Signing 1401 URL Signing is an authorization method for content delivery. This is 1402 based on embedding the HTTP URL with information that can be 1403 validated to ensure the request has legitimate access to the content. 1404 There are two parts: 1) parameters that convey authorization 1405 restrictions (e.g. source IP address and time period) and/or 1406 protected URL portion, and 2) message digest that confirms the 1407 integrity of the URL and authenticates the URL creator. The 1408 authorization parameters can be anything agreed upon between the 1409 entity that creates the URL and the entity that validates the URL. A 1410 key is used to generate the message digest (i.e. sign the URL) and 1411 validate the message digest. The two functions may or may not use 1412 the same key. 1414 There are two types of keys used for URL Signing: asymmetric keys and 1415 symmetric key. Asymmetric keys always have a key pair made up of a 1416 public key and private key. The private key and public key are used 1417 for signing and validating the URL, respectively. A symmetric key is 1418 the same key that is used for both functions. Regardless of the type 1419 of key, the entity that validates the URL has to obtain the key. 1420 Distribution for the symmetric key requires security to prevent 1421 others from taking it. Public key can be distributed freely while 1422 private key is kept by the URL signer. The method for key 1423 distribution is out of scope. 1425 URL Signing operates in the following way. A signed URL is provided 1426 by the content owner (i.e. URL signer) to the user during website 1427 navigation. When the user selects the URL, the HTTP request is sent 1428 to the CDN which validates that URL before delivering the content. 1430 3.5.1. HAS Implications 1431 The authorization lifetime for URL Signing is affected by HAS. The 1432 expiration time in the authorization parameters of URL Signing limits 1433 the period that the content referenced by the URL can be accessed. 1434 This works for URL that directly access the media content. But for 1435 HAS content, the Manifest File contains another layer of URL that 1436 reference the chunks. The chunk URL that is embedded in the content 1437 may be requested at an indeterminate amount of time later. The time 1438 period between access to the Manifest File and chunk retrieval may 1439 vary significantly. The type of content (i.e. Live or VoD) impacts 1440 the time variance as well. HAS content has this property that needs 1441 to be addressed for URL Signing. 1443 3.5.2. CDNI Considerations 1445 For CDNI, the two types of request routing are DNS-based and HTTP- 1446 based. The use of symmetric vs. asymmetric key for URL Signing has 1447 implications on the trust model between CSP and CDNs and the key 1448 distribution method that can be used. 1450 DNS-based request routing does not change the URL. In the case of 1451 symmetric key, the CSP and the Authoritative CDN have a business 1452 relationship that allows them to share a key (or multiple keys) for 1453 URL Signing. When the user request a content from the Authoritative 1454 CDN, the URL is signed by the CSP. The Authoritative CDN (as a 1455 Upstream CDN) redirects the request to a Downstream CDN via DNS. 1456 There may be more than one level of redirection to reach the 1457 Delivering CDN. The user would obtain the IP address from DNS and 1458 send the HTTP request to the Delivering CDN, which needs to validate 1459 the URL. This requires the key to be distributed from Authoritative 1460 CDN to the Delivering CDN. This may be problematic when the key is 1461 exposed to the Delivering CDN that does not have relationship with 1462 the CSP. The combination of DNS-based request routing and symmetric 1463 key function is a generic issue for URL Signing and not specific to 1464 HAS content. In the case of asymmetric keys, CSP signs URL with its 1465 private key. The Delivering CDN validates the URL with the 1466 associated public key. 1468 HTTP request routing changes the URL during redirection procedure. 1469 In the case of symmetric key, CSP signs the original URL with the 1470 same key used by the Authoritative CDN to validate the URL. The 1471 Authoritative CDN (as a Upstream CDN) redirects the request to the 1472 Downstream CDN. The new URL is signed by the Upstream CDN with the 1473 same key used by the Downstream CDN to validate that URL. The key 1474 used by the Upstream CDN to validate the original URL is expect to be 1475 different than the key used to sign the new URL. In the case of 1476 asymmetric keys, CSP signs the original URL with its private key. 1477 Authoritative CDN validates that URL with the CSP's public key. The 1478 Authoritative CDN redirects the request to the Downstream CDN. The 1479 new URL is signed by the Upstream CDN with its private key. The 1480 Downstream CDN validates that URL with the Upstream CDN's public key. 1481 There may be more than one level of redirection to reach the 1482 Delivering CDN. The URL Signing operation described previously 1483 applies at each level between the Upstream CDN and Downstream CDN for 1484 both the symmetric key and asymmetric keys cases. 1486 URL Signing requires support in most of the CDNI Interfaces. The 1487 CDNI Metadata interface should specify the content that is subject to 1488 URL signing and provide information to perform the function. The 1489 Downstream CDN should inform the Upstream CDN that it supports URL 1490 Signing in the asynchronous capabilities information advertisement as 1491 part of the Request Routing interface. This allows the CDN selection 1492 function in request routing to choose the Downstream CDN with URL 1493 signing capability when the CDNI metadata of the content requires 1494 this authorization method. The Logging interface provides 1495 information on the authorization method (e.g. URL Signing) and 1496 related authorization parameters used for content delivery. Having 1497 the information in the URL is not sufficient to know that the 1498 surrogate enforced the authorization. URL Signing has no impact on 1499 the Control interface. 1501 3.5.3. Option 5.1: Do Nothing 1503 "Do Nothing" approach means that CSP can only perform URL Signing for 1504 the top level Manifest File. The top level Manifest File contains 1505 chunk URLs or lower level Manifest File URLs, which are not modified 1506 (i.e. no URL Signing for the embedded URLs). In essence, the lower 1507 level Manifest Files and chunks are delivered without content access 1508 authorization. 1510 Effect on CDNI interfaces: 1512 o None 1514 Advantages/Drawbacks: 1516 + Top level Manifest File access is protected 1518 + Upstream CDN and Downstream CDN do not need to be aware of HAS 1519 content 1521 - Lower level Manifest Files and chunks are not protected, making 1522 this approach unqualified for content access authorization 1524 3.5.4. Option 5.2: Flexible URL Signing by CSP 1525 In addition to URL Signing for the top level Manifest File, CSP 1526 performs flexible URL Signing for the lower level Manifest Files and 1527 chunks. For each HAS session, the top level Manifest File contains 1528 signed chunk URLs or signed lower level Manifest File URLs for the 1529 specific session. The lower level Manifest File contains session- 1530 based signed chunk URLs. CSP generates the Manifest Files 1531 dynamically for the session. The chunk (segment/fragment) is 1532 delivered with content access authorization using flexible URL 1533 Signing which protects the invariant portion of the URL. Segment URL 1534 (e.g. HLS) is individually signed for the invariant URL portion 1535 (Relative URL) or the entire URL (Absolute URL without Redirection) 1536 in the Manifest File. Fragment URL (e.g. Smooth Streaming) is 1537 signed for the invariant portion of the template URL in the Manifest 1538 File. More details are provided later in this section. The URL 1539 Signing expiration time for the chunk needs to be long enough to play 1540 the video. There are implications of signing the URLs in the 1541 Manifest File. For Live content, the Manifest Files are requested at 1542 a high frequency. For VoD content, the Manifest File may be quite 1543 large. URL Signing can add more computational load and delivery 1544 latency in high volume cases. 1546 For HAS content, the Manifest File contains the Relative Locator, 1547 Absolute Locator without Redirection, or Absolute Locator with 1548 Redirection for specifying the chunk location. Signing the chunk URL 1549 requires CSP to know the portion of the URL that remains when the 1550 content is requested from the Delivery CDN surrogate. 1552 For Absolute URL without Redirection, the CSP knows that the chunk 1553 URL which is explicitly linked with the delivery CDN surrogate and 1554 can sign the URL based on that information. Since the entire URL is 1555 set and does not change, the surrogate can validate the URL. The CSP 1556 and the Delivery CDN are expected to have a business relationship in 1557 this case. So either symmetric key or asymmetric keys can be used 1558 for URL Signing. 1560 For Relative URL, the URL of the Manifest File provides the root 1561 location. The method of request routing affects the URL used to 1562 ultimately request the chunk from the Delivery CDN surrogate. For 1563 DNS, the original URL does not change. This allows CSP to sign the 1564 chunk URL based on the Manifest File URL and the Relative URL. For 1565 HTTP, the URL changes during redirection. In this case, CSP does not 1566 know the redirected URL that will be used to request the Manifest 1567 File. This uncertainty makes it impossible to accurately sign the 1568 chunk URLs in the Manifest File. Basically, URL Signing using this 1569 reference method, "as is" for entire URL protection, is not 1570 supported. However, instead of signing the entire URL, the CSP signs 1571 the Relative URL (i.e. invariant portion of the URL) and conveys the 1572 protected portion in the authorization parameters embedded in the 1573 chunk URL. This approach works the same way as Absolute URL without 1574 Redirection, except the HOST part and (part of) the PATH part of the 1575 URL are not signed and validated. The security level should remain 1576 the same as content access authorization ensures that the user that 1577 requested the content has the credentials. This scheme does not seem 1578 to compromise the authorization model since the resource is still 1579 protected by the authorization parameters and message digest. 1580 Perhaps, further evaluation on security would be helpful. 1582 For Absolute URL with Redirection, the method of request routing 1583 affects the URL used to ultimately request the chunk from the 1584 Delivery CDN surrogate. This case has the same conditions as the 1585 Relative URL. The difference is that the URL is for the chunk 1586 instead of the Manifest File. For DNS, the chunk URL does not change 1587 and can be signed by the CSP. For HTTP, the URL used to deliver the 1588 chunk is unknown to the CSP. In this case, CSP cannot sign the URL 1589 and this method of reference for the chunk is not supported. 1591 Effect on CDNI interfaces: 1593 o Requires the ability to exclude the variant portion of URL in the 1594 signing process (NOTE: Issue is specific to URL Signing support 1595 for HAS content and not CDNI?) 1597 Advantages/Drawbacks: 1599 + Manifest File and chunks are protected 1601 + Upstream CDN and Downstream CDN do not need to be aware of HAS 1602 content 1604 + DNS-based request routing with asymmetric keys and HTTP-based 1605 request routing for Relative URL and Absolute URL without 1606 Redirection works 1608 - CSP has to generate Manifest Files with session-based signed URLs 1609 and becomes involved in content access authorization for every HAS 1610 session 1612 - Manifest Files are not cacheable 1614 - DNS-based request routing with symmetric key may be problematic 1615 due to need for transitive trust between CSP and Delivery CDN 1617 - HTTP-based request routing for Absolute URL with Redirection does 1618 not work because the URL used Delivery CDN surrogate is unknown to 1619 the CSP 1621 3.5.5. Option 5.3: Flexible URL Signing by Upstream CDN 1623 This is similar to the previous section, with the exception that the 1624 Upstream CDN performs flexible URL for the lower level Manifest Files 1625 and chunks. URL Signing for the top level Manifest File is still 1626 provided by the CSP. 1628 Effect on CDNI interfaces: 1630 o Requires the ability to exclude the variant portion of URL in the 1631 signing process (NOTE: Issue is specific to URL Signing support 1632 for HAS content and not CDNI?) 1634 Advantages/Drawbacks: 1636 + Manifest File and chunks are protected 1638 + CSP does not need to be involved in content access authorization 1639 for every HAS session 1641 + Downstream CDN does not need to be aware of HAS content 1643 + DNS-based request routing with asymmetric keys and HTTP-based 1644 request routing for Relative URL and Absolute URL without 1645 Redirection works 1647 - Upstream CDN has to generate Manifest Files with session-based 1648 signed URLs and becomes involved in content access authorization 1649 for every HAS session 1651 - Manifest Files are not cacheable 1653 - Manifest File needs to be distributed through the uCDN 1655 - DNS-based request routing with symmetric key may be problematic 1656 due to need for transitive trust between uCDN and non-adjacent 1657 Delivery CDN 1659 - HTTP-based request routing for Absolute URL with Redirection does 1660 not work because the URL used Delivery CDN surrogate is unknown to 1661 the uCDN 1663 3.5.6. Option 5.4: Authorization Group ID and HTTP Cookie 1665 Based on the Authorization Group ID metadata, CDN validates the URL 1666 Signing or validates the HTTP cookie for request of content in the 1667 group. CSP performs URL Signing for the top level Manifest File. 1668 The top level Manifest File contains lower level Manifest File URLs 1669 or chunk URLs. The lower level Manifest Files and chunks are 1670 delivered with content access authorization using HTTP cookie that 1671 contains session state associated with authorization of the top level 1672 Manifest File. The Group ID Metadata is used to associate the 1673 related content (i.e. Manifest Files and chunks). It also specifies 1674 content (e.g. regexp method) that needs to be validated by either 1675 URL Signing or HTTP cookie. Note that the creator of the metadata is 1676 HAS-aware. Duration of the chunk access may be included in the URL 1677 Signing of the top level Manifest File and set in the cookie. 1678 Alternatively, the access control duration could be provided by the 1679 CDNI Metadata interface. 1681 Effect on CDNI interfaces: 1683 o CDNI Metadata Interface - Authorization Group ID metadata 1684 identifies the content that is subject to validation of URL 1685 Signing or validation of HTTP cookie associated with the URL 1686 Signing 1688 o CDNI Logging Interface - Report the authorization method used to 1689 validate the request for content delivery 1691 Advantages/Drawbacks: 1693 + Manifest File and chunks are protected 1695 + CDN does not need to be aware of HAS content 1697 + CSP does not need to change the Manifest Files 1699 - Authorization Group ID metadata is required (i.e. CDNI Metadata 1700 Interface enhancement) 1702 - Requires the use of HTTP cookie which may not be acceptable in 1703 some environments (e.g. where some targeted User-Agents do not 1704 support HTTP Cookie) 1706 - Manifest File has to be delivered by surrogate 1708 3.5.7. Option 5.5: HAS-awareness with HTTP Cookie in CDN 1710 CDN is aware of HAS content and uses URL Signing and HTTP cookie for 1711 content access authorization. URL Signing is fundamentally about 1712 authorizing access to a Content Item or its specific Content 1713 Collections (representations) for a specific user during a time 1714 period with possibly some other criteria. A chunk is an instance of 1715 the sets of chunks referenced by the Manifest File for the Content 1716 Item or its specific Content Collections. This relationship means 1717 that once the Downstream CDN has authorized the Manifest File, it can 1718 assume that the associated chunks are implicitly authorized. The new 1719 function for the CDN is to link the Manifest File with the chunks for 1720 the HTTP session. This can be accomplished by using an HTTP cookie 1721 for the HAS session. 1723 After validating the URL and detecting that the requested content is 1724 a top level Manifest File, the delivery CDN surrogate sets a HTTP 1725 cookie with a signed session token for the HTTP session. When a 1726 request for a lower level Manifest File or chunk arrives, the 1727 surrogate confirms that the HTTP cookie value contains the correct 1728 session token. If so, the lower level Manifest File or chunk is 1729 delivered due to transitive authorization property. Duration of the 1730 chunk access may be included in the URL Signing of the top level 1731 Manifest File and set in the cookie. The details of the operation 1732 are left to be determined later. 1734 Effect on CDNI interfaces: 1736 o CDNI Metadata Interface - New metadata identifies the content that 1737 is subject to validation of URL Signing and information in the 1738 cookie for the type of HAS content 1740 o Request Routing interface - Downstream CDN should inform the 1741 Upstream CDN that it supports URL Signing for known HAS content 1742 types in the asynchronous capabilities information advertisement. 1743 This allows the CDN selection function in request routing to 1744 choose the appropriate Downstream CDN when the CDNI metadata 1745 identifies the content 1747 o CDNI Logging Interface - Report the authorization method used to 1748 validate the request for content delivery 1750 Advantages/Drawbacks: 1752 + Manifest File and chunks are protected 1754 + CSP does not need to change the Manifest Files 1756 - Requires full HAS awareness on part of Upstream CDN and Downstream 1757 CDN 1759 - Requires CDNI Interfaces extensions 1761 - Requires the use of HTTP cookie which may not be acceptable in 1762 some environments (e.g. where some targeted User-Agents do not 1763 support HTTP Cookie) 1765 - Manifest File has to be delivered by surrogate 1767 3.5.8. Option 5.6: HAS-awareness with Manifest in CDN 1769 CDN is aware of HAS content and uses URL Signing for content access 1770 authorization of Manifest File and chunk. CDN generates or rewrites 1771 the Manifest Files and learns about the chunks based on the Manifest 1772 File. The embedded URLs in the Manifest File are signed by the CDN. 1773 Duration of the chunk access may be included in the URL Signing. The 1774 details of the operation are left to be determined later. Since this 1775 approach is based on signing the URLs in the Manifest File, the 1776 implications for Live and VoD content mentioned in Section 3.5.4 1777 apply. 1779 Effect on CDNI interfaces: 1781 o CDNI Metadata Interface - New metadata identifies the content that 1782 is subject to validation of URL Signing and information in the 1783 cookie for the type of HAS content 1785 o Request Routing interface - Downstream CDN should inform the 1786 Upstream CDN that it supports URL Signing for known HAS content 1787 types in the asynchronous capabilities information advertisement. 1788 This allows the CDN selection function in request routing to 1789 choose the appropriate Downstream CDN when the CDNI metadata 1790 identifies the content 1792 o CDNI Logging Interface - Report the authorization method used to 1793 validate the request for content delivery 1795 Advantages/Drawbacks: 1797 + Manifest File and chunks are protected 1799 + CSP does not need to change the Manifest Files 1801 - Requires full HAS awareness on part of Upstream CDN and Downstream 1802 CDN 1804 - Requires CDNI Interfaces extensions 1806 - Requires CDN to generate or rewrite the Manifest File 1808 - Manifest File has to be delivered by surrogate 1810 3.5.9. Recommendation 1811 The authors consider that Option 5.1 (Do Nothing) is not suitable for 1812 access control of HAS content. 1814 Where the HTTP Cookie mechanism is supported by the targeted User- 1815 Agents and the security requirements can be addressed through proper 1816 use of HTTP Cookies, the authors recommend use of Option 5.4 1817 (Authorization Group ID with HTTP Cookie) and therefore be supported 1818 by the CDNI solution. This method does not require manifest file 1819 manipulation which may be a significant deployment obstacle. 1820 Otherwise, the authors recommend that Option 5.2 (Flexible URL 1821 Signing by the CSP) or Option 5.3 (Flexible URI Signing by the 1822 Upstream CDN) be used and therefore that flexible URI be supported by 1823 the CDNI solution. Option 5.2 and Option 5.3 protect all the 1824 content, does not require Downstream CDN to be aware of HAS, does not 1825 impact CDNI interfaces, supports all different types of devices, and 1826 supports the common cases of request routing for HAS content (i.e. 1827 DNS-based request routing with asymmetric keys and HTTP-based request 1828 routing for Relative URL). 1830 HAS-awareness in CDN (Option 5.5 and Option 5.6) have some advantages 1831 that should be considered for future support (e.g. CDN that is aware 1832 of HAS content can manage the content more efficiently at a broader 1833 context. Content distribution, storage, delivery, deletion, access 1834 authorization, etc. can all benefit.). Including HAS-awareness as 1835 part of the current CDNI charter, however, would almost certainly 1836 delay the CDNI WG's milestones, and the authors therefore do not 1837 recommend it right now. 1839 3.6. Content Purge 1841 At some point in time, a uCDN might want to remove content from a 1842 dCDN. With regular content, this process can be relatively 1843 straightforward; a uCDN will typically send the request for content 1844 removal to the dCDN including a reference to the content which it 1845 wants to remove (e.g. in the form of a URL). Due to the fact that 1846 HAS content consists of large groups of files however, things might 1847 be more complex. Section 3.1 describes a number of different 1848 scenarios for doing file management on these groups of files, while 1849 Section 3.2 list the options for performing Content Acquisition on 1850 these Content Collections. This section will present the options for 1851 requesting a Content Purge for the removal of a Content Collection 1852 from a dCDN. 1854 3.6.1. Option 6.1: No HAS awareness 1856 The most straightforward way to signal content purge requests is to 1857 just send a single purge request for every file that makes up the 1858 Content Collection. While this method is very simple and does not 1859 require HAS awareness, it obviously creates a signaling overhead 1860 between the uCDN and dCDN since a reference is to be provided for 1861 each content chunk to be purged. 1863 Effect on CDNI interfaces: 1865 o None 1867 Advantages/Drawbacks (apart from those listed under Option 3.3): 1869 + Does not require changes to the CDNI Interfaces or HAS awareness 1871 - Requires individual purge request for every file making up a 1872 Content Collection (or, alternatively, requires the ability to 1873 convey references to all the chunks making up a Content Collection 1874 inside a purge request) which creates signaling overhead 1876 3.6.2. Option 6.2: Purge Identifiers 1878 There exists a potentially more efficient method for performing 1879 content removal of large numbers of files simultaneously. By 1880 including a "Purge IDentifier (Purge-ID)" in the metadata of a 1881 particular file, it is possible to virtually group together different 1882 files making up a Content Collection. A Purge-ID can take the form 1883 of an arbitrary number or string which is communicated as part of the 1884 CDNI Metadata Interface and which is the same for all files making up 1885 a particular Content Item, and different across different Content 1886 Items. If a uCDN wants to request the dCDN to remove a Content 1887 Collection, it can send a purge request containing this Purge-ID. 1888 The dCDN can then remove all files that share the corresponding 1889 Purge-ID. 1891 The advantage of this method is that it is relatively simple to use 1892 by both the dCDN and uCDN and requiring only limited additions to the 1893 CDNI Metadata Interface and CDNI Control Interface. 1895 The Purge-ID is similar to the Content Collection ID discussed in 1896 Section 3.4.2.2 for handling HAS Logging and we note that further 1897 thought is needed to determine whether the CCID and Purge-ID should 1898 be collapsed into a single element or remain separate elements. 1900 Effect on CDNI interfaces: 1902 o CDNI Metadata Interface: Add metadata field for indicating Purge- 1903 ID 1905 o CDNI Control Interface: Add functionality to convey a Purge-ID in 1906 purge requests 1908 Advantages/Drawbacks: 1910 + Allows for efficient purging of content from a dCDN 1912 + Does not require HAS awareness on part of dCDN 1914 3.6.3. Recommendation 1916 Based on the listed pros and cons, the authors recommend the WG to 1917 have mandatory support Option 1.1, the 'Do Nothing'-approach. In 1918 addition, because of its very low complexity and its benefit in 1919 facilitating low-overhead purge of large numbers of content items 1920 simultaneously, the authors recommend that the Purge IDdentifier of 1921 Option 6.2 be supported as an optional feature by the CDNI Metadata 1922 interface and the CDNI Control interface. 1924 3.7. Other issues 1926 This section includes some HAS-specific issues that came up during 1927 the discussion of this draft and which do not fall under any of the 1928 categories discussed in the previous sections. 1930 - As described in Section 2.2, a Manifest File might either be 1931 delivered by a CDN or by the CSP, thereby being invisible to the 1932 CDN delivering the chunks. Obviously, the decision on whether the 1933 CDN or CSP delivers the Manifest File is made between the uCDN and 1934 CSP, and the dCDN has no choice in the matter. However, some 1935 dCDNs might only want to offer their services in the cases where 1936 they have access to the Manifest File (e.g. because their 1937 internal architecture is based around the knowledge inside the 1938 Manifest File). For these cases, it might be useful to include a 1939 field in the CDNI Capability Advertisement to allow dCDNs to 1940 advertise the fact that they require access to the Manifest File. 1942 4. IANA Considerations 1944 This document makes no request of IANA. 1946 5. Security Considerations 1947 This document does not discuss security issues around HTTP or HAS 1948 delivery. Those are expected to be discussed in the CDNI WG 1949 documents including [I-D.ietf-cdni-framework]. 1951 6. Acknowledgements 1953 The authors would like to thank Kevin Ma, Stef van der Ziel, Bhaskar 1954 Bhupalam, Mahesh Viveganandhan, Larry Peterson, Ben Niven-Jenkins and 1955 Matt Caulfield for their valuable contributions to this document. 1957 7. References 1959 7.1. Normative References 1961 [RFC6707] Niven-Jenkins, B., Le Faucheur, F., and N. Bitar, "Content 1962 Distribution Network Interconnection (CDNI) Problem 1963 Statement", RFC 6707, September 2012. 1965 7.2. Informative References 1967 [I-D.draft-bertrand-cdni-logging] 1968 Bertrand, G., Ed. and E. Stephan, "CDNI Logging 1969 Interface", . 1971 [I-D.draft-ietf-cdni-requirements] 1972 Leung, K. and Y. Lee, "Content Distribution Network 1973 Interconnection (CDNI) Requirements, draft-ietf-cdni- 1974 requirements-03", June 2012. 1976 [I-D.ietf-cdni-framework] 1977 Peterson, L. and B. Davie, "Framework for CDN 1978 Interconnection", draft-ietf-cdni-framework-03 (work in 1979 progress), February 2013. 1981 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 1982 Resource Identifier (URI): Generic Syntax, RFC3986", 1983 January 2005. 1985 Authors' Addresses 1987 Ray van Brandenburg 1988 TNO 1989 Brassersplein 2 1990 Delft 2612CT 1991 the Netherlands 1993 Phone: +31-88-866-7000 1994 Email: ray.vanbrandenburg@tno.nl 1995 Oskar van Deventer 1996 TNO 1997 Brassersplein 2 1998 Delft 2612CT 1999 the Netherlands 2001 Phone: +31-88-866-7000 2002 Email: oskar.vandeventer@tno.nl 2004 Francois Le Faucheur 2005 Cisco Systems 2006 Greenside, 400 Avenue de Roumanille 2007 Sophia Antipolis 06410 2008 France 2010 Phone: +33 4 97 23 26 19 2011 Email: flefauch@cisco.com 2013 Kent Leung 2014 Cisco Systems 2015 170 West Tasman Drive 2016 San Jose, CA 95134 2017 USA 2019 Phone: +1 408-526-5030 2020 Email: kleung@cisco.com