idnits 2.17.1 draft-brandenburg-cdni-has-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 12, 2012) is 4306 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'I-D.ietf-cdni-use-cases' is defined on line 1952, but no explicit reference was found in the text == Unused Reference: 'I-D.draft-lefaucheur-cdni-logging-delivery' is defined on line 1969, but no explicit reference was found in the text == Outdated reference: A later version (-08) exists of draft-ietf-cdni-problem-statement-03 == Outdated reference: A later version (-10) exists of draft-ietf-cdni-use-cases-03 == Outdated reference: A later version (-17) exists of draft-ietf-cdni-requirements-03 Summary: 0 errors (**), 0 flaws (~~), 6 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 CDNI R. van Brandenburg 3 Internet-Draft O. van Deventer 4 Intended status: Informational TNO 5 Expires: January 13, 2013 F. Le Faucheur 6 K. Leung 7 Cisco Systems 8 July 12, 2012 10 Models for adaptive-streaming-aware CDN Interconnection 11 draft-brandenburg-cdni-has-03 13 Abstract 15 This documents presents thoughts on the potential impact of 16 supporting HTTP Adaptive Streaming technologies in CDN 17 Interconnection scenarios. Our intent is to spur discussion on how 18 the different CDNI interfaces could, and should, deal with content 19 delivered using adaptive streaming technologies and to facilitate 20 working group decisions. 22 Status of this Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at http://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on January 13, 2013. 39 Copyright Notice 41 Copyright (c) 2012 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (http://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 57 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 58 2. HTTP Adaptive Streaming aspects relevant to CDNI . . . . . . . 5 59 2.1. Segmentation versus Fragmentation . . . . . . . . . . . . 5 60 2.2. Addressing chunks . . . . . . . . . . . . . . . . . . . . 6 61 2.2.1. Relative URLs . . . . . . . . . . . . . . . . . . . . 8 62 2.2.2. Absolute URLs with Redirection . . . . . . . . . . . . 8 63 2.2.3. Absolute URL without Redirection . . . . . . . . . . . 9 64 2.3. Live vs. VoD . . . . . . . . . . . . . . . . . . . . . . . 10 65 2.4. Stream splicing . . . . . . . . . . . . . . . . . . . . . 11 66 3. Possible HAS Optimizations . . . . . . . . . . . . . . . . . . 11 67 3.1. File Management and Content Collections . . . . . . . . . 12 68 3.1.1. General Remarks . . . . . . . . . . . . . . . . . . . 12 69 3.1.2. Candidate approaches . . . . . . . . . . . . . . . . . 12 70 3.1.2.1. Option 1.1: No HAS awareness . . . . . . . . . . . 12 71 3.1.2.2. Option 1.2: Allow single file storage of 72 fragmented content . . . . . . . . . . . . . . . . 13 73 3.1.2.3. Option 1.3: Access correlation hint . . . . . . . 14 74 3.1.3. Recommendation . . . . . . . . . . . . . . . . . . . . 14 75 3.2. Content Acquisition of Content Collections . . . . . . . . 15 76 3.2.1. General Remarks . . . . . . . . . . . . . . . . . . . 15 77 3.2.2. Candidate Approaches . . . . . . . . . . . . . . . . . 15 78 3.2.2.1. Option 2.1: No HAS awareness . . . . . . . . . . . 15 79 3.2.2.2. Option 2.2: Allow single file acquisition of 80 fragmented content . . . . . . . . . . . . . . . . 16 81 3.2.3. Recommendation . . . . . . . . . . . . . . . . . . . . 16 82 3.3. Request Routing of HAS content . . . . . . . . . . . . . . 17 83 3.3.1. General remarks . . . . . . . . . . . . . . . . . . . 17 84 3.3.2. Candidate approaches . . . . . . . . . . . . . . . . . 17 85 3.3.2.1. Option 3.1: No HAS awareness . . . . . . . . . . . 17 86 3.3.2.2. Option 3.2: Manifest File rewriting by uCDN . . . 19 87 3.3.2.3. Option 3.3: Two-step Manifest File rewriting . . . 20 88 3.3.3. Recommendation . . . . . . . . . . . . . . . . . . . . 21 89 3.4. Logging . . . . . . . . . . . . . . . . . . . . . . . . . 22 90 3.4.1. General remarks . . . . . . . . . . . . . . . . . . . 22 91 3.4.2. Candidate Approaches . . . . . . . . . . . . . . . . . 23 92 3.4.2.1. Option 4.1: "Do-Nothing" Approach . . . . . . . . 23 93 3.4.2.2. Option 4.2: "CDNI Metadata Content Collection 94 ID" Approach . . . . . . . . . . . . . . . . . . . 25 95 3.4.2.3. Option 4.3: "CDNI Logging Interface 96 Compression" Approach . . . . . . . . . . . . . . 26 97 3.4.2.4. Option 4.4: "Full HAS 98 awareness/per-Session-Logs" Approach . . . . . . . 27 99 3.4.3. Recommendation . . . . . . . . . . . . . . . . . . . . 29 100 3.5. URL Signing . . . . . . . . . . . . . . . . . . . . . . . 30 101 3.5.1. HAS Implications . . . . . . . . . . . . . . . . . . . 31 102 3.5.2. CDNI Considerations . . . . . . . . . . . . . . . . . 31 103 3.5.3. Option 5.1: Do Nothing . . . . . . . . . . . . . . . . 33 104 3.5.4. Option 5.2.1: Flexible URL Signing by CSP . . . . . . 33 105 3.5.5. Option 5.2.2: Flexible URL Signing by Upstream CDN . . 35 106 3.5.6. Option 5.3: Authorization Group ID and HTTP Cookie . . 36 107 3.5.7. Option 5.4.1: HAS-awareness with HTTP Cookie in CDN . 37 108 3.5.8. Option 5.4.2: HAS-awareness with Manifest in CDN . . . 38 109 3.5.9. URL Signing Options Analysis . . . . . . . . . . . . . 39 110 3.6. Content Purge . . . . . . . . . . . . . . . . . . . . . . 40 111 3.6.1. Option 6.1: No HAS awareness . . . . . . . . . . . . . 40 112 3.6.2. Option 6.2: Purge Identifiers . . . . . . . . . . . . 40 113 3.7. Other issues . . . . . . . . . . . . . . . . . . . . . . . 41 114 4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 41 115 5. Security Considerations . . . . . . . . . . . . . . . . . . . 42 116 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 42 117 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 42 118 7.1. Normative References . . . . . . . . . . . . . . . . . . . 42 119 7.2. Informative References . . . . . . . . . . . . . . . . . . 42 120 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 43 122 1. Introduction 124 HTTP Adaptive Streaming (HAS) is an umbrella term for various HTTP- 125 based streaming technologies that allow a client to adaptively switch 126 between multiple bitrates depending on current network conditions. A 127 defining aspect of HAS is that, since it is based on HTTP, it is a 128 pull-based mechanism, with a client actively requesting content 129 segments, instead of the content being pushed to the client by a 130 server. Due to this pull-based nature, media servers delivering 131 content using HAS often show different characteristics when compared 132 with media servers delivering content using traditional streaming 133 methods such as RTP/RTSP, RTMP and MMS. This document presents a 134 discussion on what the impact of these different characteristics is 135 to the CDNI interfaces and what HAS-specific optimizations may be 136 required or may be desirable. The scope of this document in its 137 current form is explicitly not to propose any specific solution, but 138 merely to present the available options so that the WG can make an 139 informed decision on which way to go. 141 1.1. Terminology 143 This document uses the terminology defined in 144 [I-D.ietf-cdni-problem-statement]. 146 In addition, the following terms are used throughout this document: 148 Content Item: A uniquely addressable content element in a CDN. A 149 content item is defined by the fact that it has its own Content 150 Metadata associated with it. It is the object of a request routing 151 operation in a CDN. An example of a Content Item is a video file/ 152 stream, an audio file/stream or an image file. 154 Chunk: a fixed length element that is the result of a segmentation or 155 fragmentation operation and that is independently addressable. 157 Fragment: A specific form of chunk (see Section 2.1). A fragment is 158 stored as part of a larger file that includes all chunks that are 159 part of the Chunk Collection. 161 Segment: A specific form of chunk (see Section 2.1). A segment is 162 stored as a single file from a file system perspective. 164 Original Content: Not-chunked content that is the basis for a 165 segmentation of fragmentation operation. Based on Original Content, 166 multiple alternative representations (using different encoding 167 methods, supporting different resolutions and/or targeting different 168 bitrates) may be derived, each of which may be fragmented or 169 segmented. 171 Chunk Collection: The set of all chunks that are the result of a 172 single segmentation or fragmentation operation being performed on a 173 single representation of the Original Content. A Chunk Collection is 174 described in a Manifest File. 176 Content Collection: The set of all Chunk Collections that are derived 177 from the same Original Content. A Content Collection may consist of 178 multiple Chunk Collections, each corresponding to a single 179 representation of the Original Content. A Content Collection may be 180 described by one or more Manifest Files. 182 Manifest File: A Manifest File, also referred to as Media 183 Presentation Description (MPD) file, is a file that list the way the 184 content has been chunked (possibly for multiple encodings) and where 185 the various chunks are located (in the case of segments) or how they 186 can be addressed (in the case of fragments). 188 2. HTTP Adaptive Streaming aspects relevant to CDNI 190 In the last couple of years, a wide variety of HAS-like protocols 191 have emerged. Among them are proprietary solutions such as Apple's 192 HTTP Live Streaming (HLS), Microsoft's Smooth Streaming (HSS) and 193 Adobe's HTTP Dynamic Streaming (HDS), and various standardized 194 solutions such as 3GPP Adaptive HTTP Streaming (AHS) and MPEG Dynamic 195 Adaptive Streaming over HTTP (DASH). While all of these technologies 196 share a common set of features, each has its own defining elements. 197 This chapter will look at some of the common characteristics and some 198 of the differences between these technologies and how those might be 199 relevant to CDNI. In particular, Section 2.1 will describe the 200 various methods to store HAS content and Section 2.2 will list three 201 methods that are used to address HAS content in a CDN. After these 202 generic HAS aspects are discussed, two special situations that need 203 to be taken into account when discussing HAS are addressed: 204 Section 2.3 discusses the differences between Live and VoD content, 205 while Section 2.4 discusses the scenario where multiple streams are 206 combined in a single manifest files (e.g. for ad insertion purposes). 208 2.1. Segmentation versus Fragmentation 210 All HAS implementations are based around a concept referred to as 211 chunking: the concept of having a server split content up in numerous 212 fixed duration chunks, which are independently decodable. By 213 sequentially requesting and receiving chunks, a client can recreate 214 and play out the content. An advantage of this mechanism is that it 215 allows a client to seamlessly switch between different encodings of 216 the same Original Content at chunk boundaries. Before requesting a 217 particular chunk, a client can choose between multiple alternative 218 encodings of the same chunk, irrespective of the encoding of the 219 chunks it has requested earlier. 221 While every HAS implementation uses some form of chunking, not all 222 implementations store the resulting chunks in the same way. In 223 general, there are two distinct methods of performing chunking and 224 storing the results: segmentation and fragmentation. 226 - With segmentation, which is for example mandatory in all versions 227 of Apple's HLS prior to version 7, the chunks, in this case also 228 referred to as segments, are stored completely independent from 229 each other, with each segment being stored as a separate file from 230 a file system perspective. This means that each segment has its 231 own unique URL with which it can be retrieved. 233 - With fragmentation (or virtual segmentation), which is for example 234 used in Microsoft's Smooth Streaming, all chunks, or fragments, 235 belonging to the same Chunk Collection are stored together, as 236 part of a single file. While there are a number of container 237 formats which allow for storing this type chunked content, 238 Fragmented MP4 is most commonly used. With fragmentation, a 239 specific chunk is addressable by subfixing the common file URL 240 with an identifier uniquely identifying the chunk one is 241 interested in, either by timestamp, by byterange, or in some other 242 way. 244 While one can argue about the merits of each of these two different 245 methods of handling chunks, both have their advantages and drawbacks 246 in a CDN environment. For example, fragmentation is often regarded 247 as a method that introduces less overhead, both from a storage and 248 processing perspective. Segmentation on the other hand, is regarded 249 as being more flexible and easier to cache. In practice, current HAS 250 implementations increasingly support both methods. 252 2.2. Addressing chunks 254 In order for a client to request chunks, either in the form of 255 segments or in the form of fragments, it needs to know how the 256 content has been chunked and where to find the chunks. For this 257 purpose, most HAS protocols use a concept that is often referred to 258 as a Manifest File (also known as Media Presentation Description, or 259 MPD); i.e. a file that lists the way the content has been chunked and 260 where the various chunks are located (in the case of segments) or how 261 they can be addressed (in the case of fragments). A Manifest File, 262 or set of Manifest Files, may also identify the different encodings, 263 and thus Chunk Collections, the content is available in. 265 In general, a HAS client will first request and receive a Manifest 266 File, and then, after parsing the information in the Manifest File, 267 proceed with sequentially requesting the chunks listed in the 268 Manifest File. Each HAS implementation has its own Manifest File 269 format and even within a particular format there are different 270 methods available to specify the location of a chunk. 272 Of course managing the location of files is a core aspect of every 273 CDN, and each CDN will have its own method of doing so. Some CDNs 274 may be purely cache-based, with no higher-level knowledge of where 275 each file resides at each instant in time. Other CDNs may have 276 dedicated management nodes which, at each instant in time, do know at 277 which servers each file resides. The CDNI interfaces designed in the 278 CDNI WG will probably need to be agnostic to these kinds of CDN- 279 internal architecture decisions. In the case of HAS there is a 280 strict relationship between the location of the content in the CDN 281 (in this case chunks) and the content itself (the locations specified 282 in the Manifest File). It is therefore useful to have an 283 understanding of the different methods in use in CDNs today for 284 specifying chunk locations in Manifest Files. The different methods 285 for doing so are described in sections 2.2.1 to 2.2.3. 287 Although these sections are especially relevant for segmented 288 content, due to its inherent distributed nature, the discussed 289 methods are also applicable to fragmented content. Furthermore, it 290 should be noted that the methods detailed below for specifying 291 locations of content items in Manifest Files do not only relate to 292 temporally segmented content (e.g. segments and fragments), but are 293 also relevant in situations where content is made available in 294 multiple representations (e.g., in different qualities, encoding 295 methods, resolutions and/or bitrates). In this case the content 296 consists of multiple chunk collections, which may be described by 297 either a single Manifest File or multiple interrelated manifest 298 files. In the latter case, there may be a high-level Manifest File 299 describing the various available bitrates, with URLs pointing to 300 separate Manifest Files describing the details of each specific 301 bitrate. For specifying the locations of the other Manifest Files, 302 the same methods apply that are used for specifying chunk locations. 304 One final note relates to the delivery of the manifest files 305 themselves. While in most situations the delivery of both the 306 manifest file and the chunks are handled by the CDN, there are 307 scenarios imaginable in which the manifest file is delivered by e.g. 308 the Content Provider, and the manifest is therefore not visible to 309 the CDN. 311 2.2.1. Relative URLs 313 One method for specifying chunk locations in a Manifest File is 314 through the use of relative URLs. A relative URL is a URL that does 315 not include the HOST part of a URL but only includes (part of) the 316 PATH part of a URL. In practice, a relative URL is used by the 317 client as being relative to the location where the Manifest File has 318 been acquired from. In these cases a relative URL will take the form 319 of a string that has to be appended to the location of the Manifest 320 File to get the location of a specific chunk. This means that in the 321 case a manifest with relative URLs is used, all chunks will be 322 delivered by the same surrogate that delivered the Manifest File. A 323 relative URL will therefore not include a hostname. 325 For example, in the case a Manifest File has been requested (and 326 received) from: 328 http://surrogate.server.cdn.example.com/content_1/manifest.xml 330 , a relative URL pointing to a specific segment referenced in the 331 manifest might be: 333 segments/segment1_1.ts 335 Which means that the client should take the location of the manifest 336 file and append the relative URL. In this case, the segment would 337 then be requested from http://surrogate.server.cdn.example.com/ 338 content_1/segments/segment1_1.ts 340 The downside of using relative URLs is that it forces a CDN to 341 deliver all segments belonging to a given content item with the same 342 surrogate that delivered the Manifest File for that content item, 343 which results in limited flexibility. Another drawback is that 344 Relative URLs do not allow for fallback URLs; should the surrogate 345 that delivered the manifest file break down, the client is no longer 346 able to request chunks. The advantage of relative URLs is that it is 347 very easy to transfer content between different surrogates and even 348 CDNs. 350 2.2.2. Absolute URLs with Redirection 352 Another method for specifying locations of chunks (or other manifest 353 files) in a Manifest File is through the use of an absolute URL. An 354 absolute URL contains a fully formed URL (i.e. the client does not 355 have to calculate the URL as in the case of the relative URL but can 356 use the URL from the manifest directly). 358 In the context of Manifest Files, there are two types of absolute 359 URLs imaginable: Absolute URLs with Redirection and Absolute URLs 360 without Redirection. The two methods differ in whether the URL 361 points to a request routing node which will redirect the client to a 362 surrogate (Absolute URL with Redirection) or point directly to a 363 surrogate hosting the requested content (Absolute URL without 364 Redirection). 366 In the case of Absolute URLs with Redirection, a request for a chunk 367 is handled by the request routing system of a CDN just as if it were 368 a standalone (non-HAS) content request, which might include looking 369 up the surrogate (and/or CDN) best suited for delivering the 370 requested chunk to the particular user and sending an HTTP redirect 371 to the user with the URL pointing to the requested chunk on the 372 specified surrogate (and/or CDN), or a DNS response pointing to the 373 specific surrogate. 375 An example of an Absolute URL with Redirection might look as follows: 377 http://requestrouting.cdn.example.com/ 378 content_request?content=content_1&segment=segment1_1.ts 380 As can be seen from this example URL, the URL includes a pointer to a 381 general CDN request routing function and includes some arguments 382 identifying the requested segment. 384 The advantage of using Absolute URLs with Redirection is that it 385 allows for maximum flexibility (since chunks can be distributed 386 across surrogates and CDN in any imaginable way) without having to 387 modify the Manifest File every time one or more chunks are moved (as 388 is the case when Absolute URLs without Redirection are used). The 389 downside of this method is that it can adds significant load to a CDN 390 request routing system, since it has to perform a redirect every time 391 a client requests a new chunk. 393 2.2.3. Absolute URL without Redirection 395 In the case of the Absolute URL without Redirection, the URL points 396 directly to the specific chunk on the actual surrogate that will 397 deliver the requested chunk to the client. In other words, there 398 will be no HTTP redirection operation taking place between the client 399 requesting the chunk and the chunk being delivered to the client by 400 the surrogate. 402 An example of an Absolute URLs without Redirection is the following: 404 http://surrogate.cdn.example.com/content_1/segments/segment1_1.ts 406 As can be seen from this example URL, the URL includes both the 407 identifier of the requested segment (in this case segment1_1.ts), as 408 well as the server that is expected to deliver the segment (in this 409 case surrogate.cdn.example.com). With this, the client has enough 410 information to directly request the specific segment from the 411 specified surrogate. 413 The advantage of using Absolute URLs without Redirection is that it 414 allows more flexibility compared to using Relative URLs (since 415 segments do not necessarily have to be delivered by the same server) 416 while not requiring per-segment redirection (which would add 417 significant load to the node doing the redirection). The drawback of 418 Absolute URLs without Redirection is that it requires a modification 419 of the Manifest File every time content is moved to a different 420 location (either within a CDN or across CDNs). 422 2.3. Live vs. VoD 424 Though the formats and addresses of manifest and chunk files do not 425 typically differ significantly between live and Video-on-Demand (VoD) 426 content, the time at which the manifests and chunk files become 427 available does differ significantly. For live content, chunk files 428 and their corresponding manifest files are created and delivered in 429 real-time. This poses a number of potential issues for HAS 430 optimization: 432 - With live content, chunk files are made available in real-time. 433 This limits the applicability of bundling for content acquisition 434 purposes. Prepositioning may still be employed, however, any 435 significant latency in the prepositioning may diminish the value 436 of prepositioning if a client requests the chunk prior to 437 prepositioning, or if the prepositioning request is serviced after 438 the chunk playout time has passed. 440 - In the case of live content, manifest files must be updated for 441 each chunk and therefore must be retrieved by the client prior to 442 each chunk request. Any manifest-based optimization schemes must 443 therefore be prepared to optimize on a per-segment request basis. 444 Manifest files may also be polled multiple times prior to the 445 actual availability of the next chunk. 447 - Since live manifest files are updated as each new chunk becomes 448 available, the cacheability of manifest files is limited. Though 449 timestamping and reasonable TTLs can improve delivery performance, 450 timely replication and delivery of updated manifest files is 451 critical to ensuring uninterrupted playback. 453 - Manifest files are typically updated after the corresponding chunk 454 is available for delivery, to prevent premature requests for 455 chunks which are not yet available. HAS optimization approaches 456 which employ dynamic manifest generation must be synchronized with 457 chunk creation to prevent playback errors. 459 2.4. Stream splicing 461 Stream splicing is used to create media mashups, combining content 462 from multiple sources. A common example in which content resides 463 outside the CDNs is with advertisement insertion, for both VoD and 464 live streams. Manifest files which contain Absolute URLs with 465 redirection may contain chunk or nested manifest URLs which point to 466 content not delivered via any of the interconnected CDNs. 468 Furthermore, client and downstream proxy devices may depend on non- 469 URL information provided in the manifest (e.g., comments or custom 470 tags) for performing stream splicing. This often occurs outside the 471 scope of the interconnected CDNs. HAS optimization schemes which 472 employ dynamic manifest generation or rewriting must be cognizant of 473 chunk URLs, nested manifest URLs, and other metadata which should not 474 be modified or removed. Improper modification of these URLs or other 475 metadata may cause playback interruptions, and in the case of 476 unplayed advertisements, may result in loss of revenue for content 477 providers. 479 3. Possible HAS Optimizations 481 In the previous chapter, some of the unique properties of HAS have 482 been discussed. Furthermore, some of the CDN-specific design 483 decisions with regards to addressing chunks have been detailed. In 484 this chapter, the impact of supporting HAS in CDN Interconnection 485 scenarios will be discussed. 487 There are a number of topics, or problem areas, that are of 488 particular interest when considering the combination of HAS and CDNI. 489 For each of these problem areas it holds that there are a number of 490 different ways in which the CDNI Interfaces can deal with them. In 491 general it can be said that each problem area can either be solved in 492 a way that minimizes the amount of HAS-specific changes to the CDNI 493 Interfaces or in way that maximizes the flexibility and efficiency 494 with which the CDNI Interfaces can deliver HAS content. The goal for 495 the CDNI WG should probably be to try to find the middle ground 496 between these two extremes and try to come up with solutions that 497 optimize the balance between efficiency and additional complexity. 499 In order to allow the WG to make this decision, this chapter will 500 briefly describe each of the following problem areas together with a 501 number of different options for dealing with them. Section 3.1 will 502 discuss the problem of how to deal with file management of groups of 503 files, or Content Collections. Section 3.2 will deal with a related 504 topic: how to do content acquisition of Content Collections between 505 the uCDN and dCDN. After that, Section 3.3 describes the various 506 options for the request routing of HAS content, particularly related 507 to Manifest Files. Section 3.4 talks about a number of possible 508 optimizations for the logging of HAS content, while Section 3.5 509 discusses the options regarding URL signing. Section 3.6 finally, 510 describes different scenarios for dealing with the removal of HAS 511 content from CDNs. 513 3.1. File Management and Content Collections 515 3.1.1. General Remarks 517 One of the unique properties of HAS content is that it does not 518 consist of a single file or stream but of multiple interrelated files 519 (segment, fragments and/or Manifest Files). In this document this 520 group of files is also referred to as a Content Collection. Another 521 important aspect is the difference between segments and fragments 522 (see Section 2.1). 524 Irrespective of whether segments or fragments are used, different 525 CDNs might handle Content Collections differently from a file 526 management perspective. For example, some CDNs might handle all 527 files belonging to a Content Collection as individual files, which 528 are stored independently from each other. An advantage of this 529 approach is that makes it easy to cache individual chunks. Other 530 CDNs might store all fragments belonging to a Content Collection in a 531 bundle, as if they were a single file (e.g. by using a fragmented MP4 532 container). The advantage of this approach is that it reduces file 533 management overhead. 535 This section will look at the various ways with which the CDNI 536 interfaces might deal with these differences in handling Content 537 Collections from a file management perspective. The different 538 options can be distinguished based on the level of HAS-awareness they 539 require on the part of the different CDNs and the CDNI interfaces. 541 3.1.2. Candidate approaches 543 3.1.2.1. Option 1.1: No HAS awareness 545 This first option assumes no HAS awareness in both the involved CDNs 546 and the CDNI Interfaces. This means that the uCDN uses individual 547 files and the dCDN is not explicitely made aware of the relationship 548 between chunks and it doesn't know which files are part of the same 549 Content Collection. In practice this scenario would mean that the 550 file management method used by the uCDN is simply imposed on the dCDN 551 as well. 553 This scenario also means that it is not possible for the dCDN to use 554 any form of file bundling, such as the single-file mechanism which 555 can be to store fragmented content as a single file (see 556 Section 2.1). The one exception to this rule is the situation where 557 the content is fragmented and the Manifest Files on the uCDN contains 558 byte range requests, in which case the dCDN might be able to acquire 559 fragmented content as a single file (see Section 3.2.2.2). 561 Effect on CDN Interfaces: 563 o None 565 Advantages/Drawbacks: 567 + No HAS awareness necessary in CDNs, no changes to CDNI Interfaces 568 necessary 570 - The dCDN is forced to store chunks as individual files. 572 3.1.2.2. Option 1.2: Allow single file storage of fragmented content 574 In some cases, the dCDN might prefer to store fragmented content as a 575 single file on its surrogates to reduce file management overhead. In 576 order to do so, it needs to be able to either acquire the content as 577 a single file (see Section 3.2.2.2), or merge the different chunks 578 together and place them in the same container (e.g. fragmented MP4). 579 The downside of this is that in order to do so, the dCDN needs to be 580 fully HAS aware. 582 Effect on CDN Interfaces: 584 o CDNI Metadata Interface: Add fields for indicating the particular 585 type of HAS (e.g. MPEG DASH or HLS) that is used and whether 586 segments or fragments are used 588 o CDNI Metadata Interface: Add field for indicating the name and 589 type of the manifest file(s) 591 Advantages/Drawbacks: 593 + Allows dCDN to store fragmented content as a single file, reducing 594 file management overhead 596 - Complex operation, requiring dCDN to be fully HAS aware 598 3.1.2.3. Option 1.3: Access correlation hint 600 An intermediary approach between the two extremes detailed in the 601 previous two sections is one that uses a 'Access Correlation Hint'. 602 This hint, which is added to the CDNI Metadata of all chunks of a 603 particular Content Collection, indicates that those files are likely 604 to be requested in a short time window from each other. This 605 information can help a dCDN to implement local file storage 606 optimizations for VoD items (e.g. by bundling all files with the same 607 Access Correlation Hint value in a single bundle/file), thereby 608 reducing the number of files it has to manage while not requiring any 609 HAS awareness. 611 Effect on CDN Interfaces: 613 o CDNI Metadata Interface: Add field for indicating Access 614 Correlation Hint 616 Advantages/Drawbacks: 618 + Allows dCDN to perform file management optimization 620 + Does not require any HAS awareness 622 + Very small impact on CDNI Interfaces 624 - Expected benefit compared with Option 1.1 is small 626 3.1.3. Recommendation 628 Based on the listed pros and cons, the authors recommend the WG to go 629 for Option 1.1, the 'Do Nothing'-approach. The likely benefits from 630 going for Option 1.3 are not believed to be significant enough to 631 warrant changing the CDNI Metadata Interface. Although Option 1.2 632 would bring definite benefits for HAS aware dCDNs, going for this 633 options would require significant CDNI extensions that would impact 634 the WG's milestones. The authors therefore don't recommend to 635 include it in the current work but mark it as a possible candidate 636 for rechartering once the initial CDNI solution is completed. 638 3.2. Content Acquisition of Content Collections 640 3.2.1. General Remarks 642 In the previous section the relationship between file management and 643 HAS in a CDNI scenario has been discussed. This section will discuss 644 a related topic, which is content acquisition between two CDNs. 646 With regards to content acquisition, it is important to note the 647 difference between CDNs that do Dynamic Acquisition of content and 648 CDNs that perform Content Pre-positioning. In the case of dynamic 649 acquisition, a CDN only requests a particular content item when a 650 cache-miss occurs. In the case of pre-positioning, a CDN pro- 651 actively places content items on the nodes on which it expects 652 traffic for that particular content item. For each of these types of 653 CDNs, there might be a benefit in being HAS aware. For example, in 654 the case of dynamic acquisition, being HAS aware means that after a 655 cache miss for a giving chunk occurs, that node might not only 656 acquire the requested chunk, but might also acquire some related 657 chunks that are expected to be requested in the near future. In the 658 case of pre-positioning, similar benefits can be had. 660 3.2.2. Candidate Approaches 662 3.2.2.1. Option 2.1: No HAS awareness 664 This first option assumes no HAS awareness in both the involved CDNs 665 and the CDNI Interfaces. Just as with Option 1.1 discussed in the 666 previous section with regards to file management, having no HAS 667 awareness means that the dCDN is not aware of the relationship 668 between chunks. In the case of content acquisition, this means that 669 each and every file belonging to a Content Collection will have to be 670 individually acquired from the uCDN by the dCDN. The exception to 671 the rule is in cases with fragmented content where the uCDN uses 672 Manifest Files which contain byte range requests. In this case the 673 dCDN can simply omit the byte range identifier and acquire the 674 complete file. 676 The advantage of this approach is that it is highly flexible. If a 677 client only requests a small portion of the chunks belonging to a 678 particular Content Collection, the dCDN only has to acquire those 679 chunks from the uCDN, saving both bandwidth and storage capacity. 681 The downside of acquiring content on a per-chunk basis is that it 682 creates more transaction overhead between the dCDN and uCDN compared 683 to a method in which entire Content Collections can be acquired as 684 part of one transaction. 686 Effect on CDN Interfaces: 688 o None 690 Advantages/Drawbacks: 692 + Per-chunk content acquisition allows for high level of flexibility 693 between dCDN and uCDN 695 - Per-chunk content acquisition creates more transaction overhead 696 between dCDN and uCDN 698 3.2.2.2. Option 2.2: Allow single file acquisition of fragmented 699 content 701 As discussed in Section 3.2.2.1, there is one (fairly rare) in cases 702 where fragmented content can be acquired as a single file without any 703 HAS awareness and that is when fragmented content is used and where a 704 Manifest File includes byte range request. This section discusses 705 how to perform single file acquisition in the other (very common) 706 cases. To do so, the dCDN would have to have full-HAS awareness (at 707 least to the extent of being able to map between single file and 708 individual chunks to serve). 710 Effect on CDN Interfaces: 712 o CDNI Metadata Interface: Add fields for indicating the particular 713 type of HAS (e.g. MPEG DASH or HLS) that is used and whether 714 segments or fragments are used 716 o CDNI Metadata Interface: Add field for indicating the name and 717 type of the manifest file(s) 719 Advantages/Drawbacks: 721 + Allows for more efficient content acquisition in all HAS-specific 722 supported forms 724 - Requires full HAS awareness on part of dCDN 726 - Requires significant CDNI Metadata Interface extensions 728 3.2.3. Recommendation 730 Based on the listed pros and cons, the authors recommend the WG to go 731 for Option 2.1 since it is sufficient to 'make HAS work'. While 732 Option 2.2 would bring benefits to the acquisition of large Content 733 Collections, it would require significant CDNI extensions which would 734 impact the WG's milestones. Option 2.2 might be a candidate to 735 include in possible rechartering once the initial CDNI solution is 736 completed. 738 3.3. Request Routing of HAS content 740 3.3.1. General remarks 742 In this section the effect HAS content has on request routing will be 743 identified. Of particular interest in this case are the different 744 types of Manifest Files that might be used. In Section 2.2, three 745 different methods for identifying and addressing chunks from within a 746 Manifest File were described: Relative URLs, Absolute URLs without 747 Redirection and Absolute URLs with Redirection. Of course not every 748 current CDN will use and/or support all three methods. Some CDNs may 749 only use one of the three methods, while others may support two or 750 all three. 752 An important factor in deciding which chunk addressing method is used 753 is the Content Provider. Some Content Providers may have a strong 754 preference for a particular method and deliver the Manifest Files to 755 the CDN in a particular way. Depending on the CDN and the agreement 756 it has with the Content Provider, a CDN may either host the Manifest 757 Files as they were created by the Content Provider, or modify the 758 Manifest File to adapt it to its particular architecture (e.g. by 759 changing relative URLs to Absolute URLs which point to the CDN 760 Request Routing function). 762 3.3.2. Candidate approaches 764 3.3.2.1. Option 3.1: No HAS awareness 766 This first option assumes no HAS awareness in both the involved CDNs 767 and the CDNI Interfaces. This scenario also assumes that neither the 768 dCDN nor the uCDN have the ability to actively manipulate Manifest 769 Files. As was also discussed with regards to file management and 770 content acquisition, having no HAS awareness means that each file 771 constituting a Content Collections is handled on an individual basis, 772 with the dCDN unaware of any relationship between files. 774 The only chunk addressing method that works without question in this 775 case is Absolute URLs with Redirection. In other words, the Content 776 Provider that ingested the content into the uCDN created a Manifest 777 File with each chunk location pointing to the Request Routing 778 function of the uCDN. Alternatively, the Content Provider may have 779 ingested the Manifest File containing relative URLs and the uCDN 780 ingestion function has translated these to Absolute URLs pointing to 781 the Request Routing function. 783 In this Absolute URL with Redirection case, the uCDN can simply have 784 the Manifest File be delivered by the dCDN as if it were a regular 785 file. Once the client parses the Manifest File, it will request any 786 subsequent chunks from the uCDN Request Routing function. That 787 function can then decide to outsource the delivery of that chunk to 788 the dCDN. Depending on whether HTTP-based (recursive or iterative) 789 or DNS-based request routing is used, the uCDN Request Routing 790 function will then either directly or indirectly redirect the client 791 to the Request Routing function of the dCDN (assuming it does not 792 have the necessary information to redirect the client directly to a 793 surrogate in the dCDN). 795 The drawback of this method is that it creates a large amount of 796 request routing overhead for both the uCDN and dCDN. For each chunk 797 the full inter-CDN Request Routing process is invoked (which can 798 result in two HTTP redirections in the case of iterative redirection, 799 or result in one HTTP redirection plus one CDNI Request Routing/ 800 Redirection Interface request/response). Even in the case where DNS- 801 based redirection is used, there might be significant overhead 802 involved since both the dCDN and uCDN Request Routing function might 803 have to perform database lookups and query each other. While with 804 DNS this overhead might be reduced by using DNS' inherent caching 805 mechanism, this will have significant impact on the accuracy of the 806 redirect. 808 With no HAS awareness, Relative URLs might or might not work 809 depending on the type of Relative URL that is used. When a uCDN 810 delegates the delivery of a Manifest File containing Relative URLs to 811 a dCDN, the client goes directly to the dCDN surrogate from which it 812 has received the Manifest File for every subsequent chunk. As long 813 as the Relative URL is not path-absolute (see [RFC3986]), this 814 approach will work fine. 816 Since using Absolute URLs without Redirection inherently require a 817 HAS aware CDN, they also cannot be used in this case. The reason for 818 this is that with Absolute URLs without Redirection, the URLs in the 819 Manifest File will point directly to a surrogate in the uCDN. Since 820 this scenario assumes no HAS awareness on the part of the dCDN or 821 uCDN, it is impossible for either of these CDNs to rewrite the 822 Manifest File and thus allow the client to either go to a surrogate 823 in the dCDN or to a request routing function. 825 Effect on CDN Interfaces: 827 o None 829 Advantages/Drawbacks: 831 + Supports Absolute URLs with Redirection 833 + Supports Relative URLs 835 + Does not require HAS awareness and/or changes to the CDNI 836 Interfaces 838 - Not possible to use Absolute URLs without Redirection 840 - Creates significant signaling overhead in case Absolute URLs with 841 Redirection are used (inter-CDN request redirection for each 842 chunk) 844 3.3.2.2. Option 3.2: Manifest File rewriting by uCDN 846 While Option 3.1 does allow for Absolute URLs with Redirection to be 847 used, it does so in a way that creates a high-level of request 848 routing overhead for both the dCDN and the uCDN. This option 849 presents a solution to significantly reduce this overhead. 851 In this scenario, the uCDN is able to rewrite the Manifest File (or 852 generate a new one) to be able to remove itself from the request 853 routing chain for chunks being referenced in the Manifest File. As 854 described in Section 3.3.2.1, in the case of no HAS awareness the 855 client will go to the uCDN request routing function for each chunk 856 request. This request routing function can then redirect the client 857 to the dCDN request routing function. By rewriting the Manifest File 858 (or generating a new one), the uCDN is able to remove this first 859 step, and have the Manifest File point directly to the dCDN request 860 routing function. 862 The main advantage of this solution is that it does not direcly have 863 an impact on the CDNI Interfaces and is therefore relatively 864 transparent to these interfaces. It is a function that a uCDN can 865 perform independently by using the information that it receives from 866 the dCDN as part of the regular CDNI Request Routing Interface 867 communication. 869 More specifically, in order for the uCDN to rewrite the manifest 870 file, it only needs the location of the dCDN request routing function 871 (or even better: the location of the dCDN surrogate). Obtaining this 872 information is part of a the regular CDNI Request Routing Interface 873 and can be done in of of two ways. The first way would be to have 874 the uCDN ask the dCDN for the location of its request routing node 875 (through the CDNI Request Routing/Redirection Interface) every time a 876 request for a Manifest File comes in at the uCDN request routing 877 function. The uCDN would then modify the manifest file and deliver 878 the manifest file to the client. A second way to do it would be for 879 the modification of the manifest file to only happen once, when the 880 first client for that particular Content Collection (and redirected 881 to that particular dCDN) sends a Manifest File request. The 882 advantage of the first method is that it maximizes effiency and 883 flexibility by allowing the dCDN to respond with the locations of its 884 surrogates instead of the location of its request routing function 885 (and effectively turning the URLs into Absolute URLs without 886 Redirection). The advantage of the second method is that the uCDN 887 only has to modify the Manifest File once. 889 It should be noted that there are a number of things to take into 890 account when changing a manifest file (see for example Section 2.3 891 and Section 2.4 on live HAS content and ad insertion). Furthermore, 892 some Content Providers might have issues with a CDN changing manifest 893 files. However, in this option the manifest manipulation is only 894 being performed by the uCDN, which can be expected to be aware of 895 these limitations if it wants to perform manifest manipulation since 896 it is in its own best interest that it's customer's content gets 897 delivered in the proper way. Should the Content Provider want to 898 limit manifest manipulation, it can simply arrange this with the uCDN 899 bilaterally. 901 Effect on CDN Interfaces: 903 o None 905 Advantages/Drawbacks: 907 + Possible to significantly decrease signalling overhead when using 908 Absolute URLs. 910 + (Optional) Possible to have uCDN rewrite the manifest with 911 locations of surrogates in dCDN (turning Absolute URLs with 912 Redirection in Absolute URLs without Redirection) 914 + No changes to CDNI Interfaces 916 + Does not require HAS awareness in dCDN 918 - Requires high level of HAS awareness in uCDN (for modifying 919 manifest files) 921 3.3.2.3. Option 3.3: Two-step Manifest File rewriting 923 One of the possibilities with Option 3.3 is allowing the dCDN to 924 provide the locations of a specific surrogate to the uCDN, so that 925 the uCDN can fit the Manifest File with Absolute URLs without 926 Redirection and the client can request chunks directly from a dCDN 927 surrogate. However, some dCDNs might not be willing to provide this 928 information to the uCDN. In that case they can only provide the uCDN 929 with the location of their request routing function and thereby not 930 be able to use Absolute URLs without Redirection. 932 One method for solving this limitation is allowing two-step Manifest 933 File manipulation. In the first step the uCDN would perform its own 934 modification, and place the locations of the dCDN request routing 935 function in the Manifest File. Then, once a request for the Manifest 936 File comes in at the dCDN request routing function, it would perform 937 a second modification in which it replaces the URLs in the Manifest 938 Files with the URLs of its surrogates. This way the dCDN can still 939 profit from having minimal request routing traffic, while not having 940 to share sensitive surrogate information with the uCDN. 942 The downside of this approach is that it not only assumes HAS 943 awareness in the dCDN but that it also requires some HAS-specific 944 additions to the CDNI Metadata Interface. In order for the dCDN to 945 be able to change the Manifest File, it has to have some information 946 about the structure of the content. Specifically, it needs to have 947 information about which chunks make up the Content Collection. 949 Effect on CDN Interfaces (apart from those listed under Option 3.3): 951 o CDNI Metadata Interface: Add necessary fields for conveying HAS 952 specific information (e.g. the files that make up the Content 953 Collection) to the dCDN. 955 o dCDN: Allow for modification of manifest file 957 Advantages/Drawbacks (apart from those listed under Option 3.3): 959 + Allows dCDN to use Absolute URLs without Redirection without 960 having to convey sensitive information to the uCDN 962 - Requires high level of HAS awareness in dCDN (for modifying 963 manifest files) 965 - Requires adding HAS-specific information to the CDNI Metadata 966 Interface 968 3.3.3. Recommendation 970 Based on the listed pros and cons, the authors recommend to go for 971 Option 3.1, with Option 3.2 as an optional feature for uCDN that 972 support this. While Option 3.1 allows for HAS content to be 973 delivered using the CDNI interfaces, it does so with some limitations 974 regarding supported manifest files and with large signalling 975 overhead. Option 3.2 can solve most of these limitations and 976 presents a significant reduction of the request routing overhead. 977 Since Option 3.2 does not require any changes to the CDNI interfaces 978 but only changes the way the uCDN uses the existing interfaces, 979 supporting it is not expected to result in a significant delay of the 980 WG's milestones. The authors recommend the WG to not include Option 981 3.3, since it raises some questions of potential brittleness and 982 including it would result in a significant delay of the WG's 983 milestones. 985 3.4. Logging 987 3.4.1. General remarks 989 As stated in [I-D.ietf-cdni-problem-statement], "the CDNI Logging 990 interface enables details of logs or events to be exchanged between 991 interconnected CDNs". 993 As discussed in [I-D.draft-bertrand-cdni-logging], the CDNI logging 994 information can be used for multiple purposes including maintenance/ 995 debugging by uCDN, accounting (e.g. in view of billing or 996 settlement), reporting and management of end-user experience (e.g. to 997 the CSP), analytics (e.g. by the CSP) and control of content 998 distribution policy enforcement (e.g. by the CSP). 1000 The key consideration for HAS with respect to logging is the 1001 potential increase of the number of Log records by two to three 1002 orders of magnitude, as compared to regular HTTP delivery of a video, 1003 since, by default, log records would typically be generated on a per- 1004 chunk-delivery basis instead of per-content-item-delivery basis. 1005 This impacts the scale of every processing step in the Logging 1006 Process (see Section 8 of [I-D.draft-bertrand-cdni-logging]), 1007 including: 1009 a. Logging Generation and Storing of logs on CDN elements 1010 (Surrogate, Request Routers,..) 1012 b. Logging Aggregation within a CDN 1014 c. Logging Manipulation (including Logging Protection, Logging 1015 Filtering , Logging Update and Rectification) 1017 d. (Where needed) Logging CDNI Reformatting (e.g. reformatting from 1018 CDN-specific format to the CDNI Logging Interface format for 1019 export by dCDN to uCDN) 1021 e. Logging exchange via CDNI Logging Interface 1022 f. (Where needed) Logging Re-Reformatting (e.g. reformatting from 1023 CDNI Logging Interface format into log-consuming specific 1024 application) 1026 g. Logging consumption/processing (e.g. feed logs into uCDN 1027 accounting application, feed logs into uCDN reporting system to 1028 provide per CSP views, feed logs into debugging tool to debug) 1030 Note that there may be multiple instances of step [f] and [g] running 1031 in parallel. 1033 While the CDNI Logging Interface is only used to perform step [e], we 1034 note that its format directly affects step [d] and [f] and that its 1035 format also indirectly affects the other steps (for example if the 1036 CDNI Logging Interface requires per-chunk log records, step [a], [b] 1037 and [d] cannot operate on a per-HAS-session basis and also need to 1038 operate on a per-chunk basis). 1040 3.4.2. Candidate Approaches 1042 The following sub-sections discusses the main candidate approaches 1043 identified so far for CDNI in terms of dealing with HAS with respect 1044 to Logging. 1046 3.4.2.1. Option 4.1: "Do-Nothing" Approach 1048 In this approach nothing is done specifically for HAS so that each 1049 HAS-chunk delivery is considered, for CDNI Logging, as a standalone 1050 content delivery. In particular, a separate log record for each HAS- 1051 chunk delivery is included in the CDNI Logging Interface in step [e]. 1052 This approach requires that step [a], [b], [c], [d] and [e] also be 1053 performed on a per-chunk basis. This approach allows [g] to be 1054 performed either on a per-chunk basis (assuming step [f] maintains 1055 per-chunk records) or on a more "summarized" manner such as per-HAS- 1056 Session basis (assuming step [f] summarizes per-chunk records into 1057 per-HAS-session records). 1059 Effect on CDN Interfaces: 1061 o None 1063 Effect on uCDN and dCDN: 1065 o None 1067 Advantages/Drawbacks: 1069 + No information loss (i.e. all details of each individual chunk 1070 delivery are preserved). While this full level of detail may not 1071 be needed for some Log consuming applications (e.g. billing), this 1072 full level of detail is likely valuable (possibly required) for 1073 some Log consuming applications (e.g. debugging) 1075 + Easier integration (at least in the short term) into existing 1076 Logging tools since those are all capable of handling per-chunk 1077 records 1079 + No extension needed on CDNI interfaces 1081 - High volume of logging information to be handled (storing & 1082 processing) at every step of the Logging process from [a] to [g] 1083 (while summarization in step [f] is conceivable, it may be 1084 difficult to achieve in practice without any hints for correlation 1085 in the log records). While the high volume of logging information 1086 is a potential concern, we are seeking expert input on whether it 1087 is a real practical issue, and if yes, then in what timeframe/ 1088 assumptions. 1090 An interesting question is whether a dCDN could use the CDNI Logging 1091 interface specified for the "Do-Nothing" approach to report 1092 summarized "per-session" log information in the case where the dCDN 1093 performs such summarization. The high level idea would be that, when 1094 a dCDN performs HAS log summarization for its own purposes anyways, 1095 this dCDN could include, in the CDNI Logging interface, one (or a 1096 few) log entry for a HAS session (instead of one entry per HAS-chunk) 1097 that summarizes the deliveries of many/all HAS-chunk for a session. 1098 However, the authors feel that, when considering the details of this, 1099 this is not achievable without explicit agreement between the uCDN 1100 and dCDN about how to perform/interpret such summarization. For 1101 example, when a HAS session switches between representations, the 1102 uCDN and dCDN would have to agree on things such as: 1104 o whether the session will be represented by a single log entry 1105 (which therefore cannot convey the distribution across 1106 representations) or multiple log entries such as one entry per 1107 contiguous period at a given representation (which therefore would 1108 be generally very difficult to correlate back into a single 1109 session) 1111 o what would the single URI included in the log entry correspond to? 1112 the manifest/top-level-playlist/next-level-playlist,... 1114 The authors feel that if explicit agreement is needed between uCDN 1115 and dCDN on how to perform/interpret the summarization is required, 1116 then this should be specified as part of the CDNI Logging interface 1117 and then effectively boils down to Option 4.5 defined below ("Full 1118 HAS awareness" and "per-Session-Logs" Approach). 1120 We note that support by CDNI of a mechanism (independent of HAS) 1121 allowing the customization of the fields to be reported in log 1122 entries by the dCDN to the uCDN would have a minor mitigation effect 1123 on the HAS logging scaling concerns because it ensures that only the 1124 necessary subset of fields are actually stored, reported and 1125 processed. 1127 3.4.2.2. Option 4.2: "CDNI Metadata Content Collection ID" Approach 1129 In this approach, a "Content Collection ID" (CCID) field is 1130 distributed through the CDNI Metadata Interface and the same CCID 1131 value is associated through the CDNI Metadata interface with every 1132 chunk of the same Content Collection. The CCID value needs to be 1133 such that it allows, in combination with the content URI, to uniquely 1134 identify a Content Collection. When distributed, and CCID logging is 1135 requested from the dCDN, the dCDN Surrogates are to store the CCID 1136 value in the corresponding log entries. The objective of this field 1137 is to facilitate optional summarization of per-chunk records at step 1138 [f] into something along the lines of per-HAS-session logs, at least 1139 for the Log consuming applications that do not require per-chunk 1140 detailed information (for example billing). 1142 We note that, if the downstream CDN happens to have sufficient HAS 1143 awareness to be able to generate a session identifier (Session-ID), 1144 optionally including such Session-ID (in addition to the CCID) in the 1145 per-chunk log record would further facilitate optional summarization 1146 performed at step [f]. The Session-ID value to be included in a log 1147 record by the delivering CDN is such that 1149 o different per-chunk log records with the same Session-ID value 1150 must correspond to the same user session (.i.e delivery of same 1151 content to same enduser at a given point in time). 1153 o log records for different chunks of the same user session (.i.e 1154 delivery of same content to same enduser at a given point in time) 1155 should be provided with the same session-ID value. While 1156 undesirable, there may be situations where the delivering CDN uses 1157 more than one session-ID value for different per-chunk log records 1158 of a given session, for example in scenarios of fail-over or load- 1159 balancing across multiple Surrogates and where the delivering CDN 1160 does not implement mechanism to synchronize session-IDs across 1161 Surrogates. 1163 Effect on CDN Interfaces: 1165 o CDNI Metadata interface: One additional metadata field (CCID) in 1166 CDNI Metadata Interface. We note that a similar Content 1167 Collection ID is discussed for handling of other aspects of HAS 1168 and observe that further thought is needed to determine whether 1169 such CCID should be shared for multiple purposes or should be 1170 independent. 1172 o CDNI Logging interface: Two additional fields (CCID and 1173 Session-ID) in CDNI Logging records. 1175 Effect on uCDN and dCDN: 1177 o None 1179 Advantages/Drawbacks: 1181 + No information loss (i.e. all details of each individual chunk 1182 delivery are preserved). While this full level of detail may not 1183 be needed for some Log consuming applications (e.g. billing), this 1184 full level of detail is likely valuable (possibly required) for 1185 some Log consuming applications (e.g. debugging) 1187 + Easier integration (at least in the short term) into existing 1188 Logging tools since those are all capable of handling per-chunk 1189 records 1191 + Very minor extension to CDNI interfaces needed 1193 + Facilitated summarization of records related to a HAS session in 1194 step [f] and therefore ability to operate on lower volume of 1195 logging information in step [g] by log consuming applications that 1196 do not need per-chunk record details (e.g. billing) or that need 1197 per-session information (e.g. analytics) 1199 - High volume of logging information to be handled (storing & 1200 processing) at every step of the Logging process from [a] to [f]. 1201 While the high volume of logging information is a potential 1202 concern, we are seeking input on whether it is a real practical 1203 issue, and if yes in what timeframe/assumptions 1205 3.4.2.3. Option 4.3: "CDNI Logging Interface Compression" Approach 1207 In this approach, a loss-less compression technique is applied to the 1208 sets of Logging records (e.g. Logging files) for transfer on the 1209 IETF CDNI Logging Interface. The objective of this approach is to 1210 reduce the volume of information to be stored and transferred in step 1211 [e]. 1213 Effect on CDN Interfaces: 1215 o One additional compression mechanism to be included in the CDNI 1216 Logging Interface 1218 Effect on uCDN and dCDN: 1220 o None 1222 Advantages/Drawbacks: 1224 + No information loss (i.e. all details of each individual chunk 1225 delivery are preserved). While this full level of detail may not 1226 be needed for some Log consuming applications (e.g. billing), this 1227 full level of detail is likely valuable (possibly required) for 1228 some Log consuming applications (e.g. debugging) 1230 + Easier integration (at least in the short term) into existing 1231 Logging tools since those are all capable of handling per-chunk 1232 records 1234 + Small extension to CDNI interfaces needed 1236 + Reduced volume of logging information in step [e] 1238 + Compression likely to be also applicable to logs for non-HAS 1239 content 1241 - High volume of logging information to be handled (storing & 1242 processing) at every step of the Logging process from [a] to [g], 1243 except [e]. While the high volume of logging information is a 1244 potential concern, we are seeking expert input on whether it is a 1245 real practical issue, and if yes, then in what timeframe/ 1246 assumptions 1248 Input is sought on expected compression gains achievable in practice 1249 over sets of logs containing per-chunk records. 1251 3.4.2.4. Option 4.4: "Full HAS awareness/per-Session-Logs" Approach 1253 In this approach, HAS-awareness is assumed across the CDNs 1254 interconnected via CDNI and the necessary information to describe the 1255 HAS relationship across all chunks of the same Content Collection is 1256 distributed through the CDNI Metadata Interface. In this approach, 1257 the dCDN Surrogates leverage the HAS information distributed through 1258 the CDNI metadata and their HAS-awareness to generate summarized 1259 logging information in the very first place. The objective of that 1260 approach is to operate on lower volume of logging information right 1261 from the very first step of the Logging process. 1263 The summarized HAS logs generated by the Surrogates in this approach 1264 are similar to those discussed in the section " "CDNI Metadata 1265 Content Collection ID With dCDN Summarization" Approach" and the same 1266 trade-offs between information loss and summarization gain apply. 1268 Effect on CDN Interfaces: 1270 o One significant extension of the CDNI Metadata Interface to convey 1271 HAS relationship across chunks of a Content Collection. Note that 1272 this extension requires specific support for every HAS-protocol to 1273 be supported over the CDNI mesh 1275 Effect on uCDN and dCDN: 1277 o Full HAS-awareness by dCDN Surrogates 1279 Advantages/Drawbacks: 1281 + Lower volume of logging information to be handled (storing & 1282 processing) at every step of the Logging process from [a] to [g] 1284 + Accurate generation of summarized logs because of HAS awareness on 1285 Surrogate (for example, where the Surrogate is also serving the 1286 manifest file(s) for a content collection, the Surrogate may be 1287 able to extract definitive information about the relationship 1288 between all chunks) 1290 - Very significant extensions to CDNI interfaces needed including 1291 per HAS-protocol specific support 1293 - Very significant additional requirement for HAS awareness on dCDN 1294 and for this HAS-awareness to be consistent with the defined CDNI 1295 Logging summarization 1297 - Some information loss (i.e. all details of each individual chunk 1298 delivery are not preserved). The actual information loss depends 1299 on the summarization approach selected (typically the lower the 1300 information loss, the lower the summarization gain) so the right 1301 sweet-spot would had ego be selected. While full level of detail 1302 may not be needed for some Log consuming applications (e.g. 1303 billing), the full level of detail is likely valuable (possibly 1304 required) for some Log consuming applications (e.g. debugging) 1306 - Less easy integration (at least in the short term) into existing 1307 Logging tools since those are all capable of handling per-chunk 1308 records and may not be capable of handling CDNI summarized records 1310 - Challenges in defining behavior (and achieving summarization gain) 1311 in the presence of load-balancing of a given HAS-session across 1312 multiple Surrogates (in same or different dCDN) 1314 Input is sought on expected compression gains achievable in practice 1315 over sets of logs containing per-chunk records. 1317 3.4.3. Recommendation 1319 Because of its benefits (in particular simplicity, universal support 1320 by CDNs and support by all log-consuming applications), we recommend 1321 that the per-chunk logging of Option 4.1 be supported by the CDNI 1322 Logging interface as a "High Priority" (as defined in 1323 [I-D.draft-ietf-cdni-requirements]) and be a mandatory capability of 1324 CDNs implementing CDNI. 1326 Because of its very low complexity and its benefit in facilitating 1327 some useful scenarios (e.g. per-session analytics), we recommend that 1328 the CCID mechanisms and Session-ID mechanism of Option 4.2 be 1329 supported by the CDNI Metadata interface and the CDNI Logging 1330 interface as a "Medium Priority" (as defined in 1331 [I-D.draft-ietf-cdni-requirements]) and be an optional capability of 1332 CDNs implementing CDNI. 1334 We recommend that: 1336 (i) the ability for the uCDN to request that the CCID and Session-ID 1337 field be included in log entries provided by the dCDN be supported 1338 by the relevant CDNI interfaces (tentatively the CDNI Metadata 1339 interface) and 1341 (ii) the ability for the dCDN to include the CCID field and 1342 SEssion-ID in CDNI log entries (when the dCDN is capable of doing 1343 so) and indicate so inside the CDNI Logging interface (in line 1344 with the "customizable" log format expected to be defined 1345 independently of HAS), 1347 be supported as a "Medium Priority" (as defined in 1348 [I-D.draft-ietf-cdni-requirements]) and be an optional capability of 1349 CDNs implementing CDNI. 1351 When performing dCDN selection, an uCDN may want to take into account 1352 whether a given dCDN is capable of reporting the CCID and session-ID. 1353 Thus, we recommend that the ability for a dCDN to advertise its 1354 support of the optional CCID and SessionID capability be supported by 1355 the CDNI request Routing /Footprint and Capabilities Advertisment 1356 Interface as a "Medium Priority" (as defined in 1357 [I-D.draft-ietf-cdni-requirements]). 1359 Because it can be achieved with very little complexity and it 1360 provides some clear storage/communication compression benefits, we 1361 recommend that, in line with the concept of Option 4.3, some existing 1362 very common compression techniques (e.g. gzip) be supported by the 1363 CDNI Logging interface as a "Medium Priority" (as defined in 1364 [I-D.draft-ietf-cdni-requirements]) and be an optional capability of 1365 CDNs implementing CDNI. 1367 Because of its complexity, the time it would take to understand the 1368 trade-offs of candidate summarization approaches and the time it 1369 would take to specify the corresponding support in the CDNI Logging 1370 interface, we recommend that the log summarization discussed in 1371 option 4.4 not be supported by the CDNI Logging interface at this 1372 stage and be kept as candidate topic of great interest for a 1373 rechartering of the CDNI WG once the first set of deliverables is 1374 produced. When doing so, we suggest to investigate the notion of 1375 complementing the "push-style" CDNI logging interface supporting 1376 summarization by an on-demand pull-type of interface allowing an uCDN 1377 to request the subset of the detailed logging information that it may 1378 need but is lost in the summarized pushed information. 1380 We note that while a CDN only needs to adhere to the CDNI Logging 1381 interface on its external interfaces and can perform logging in a 1382 different format within the CDN, any possible CDNI Logging approach 1383 effectively places some constraints on the dCDN logging format. For 1384 example, to support the "Do-Nothing" Approach, a CDN need to perform 1385 and retain per chunk logs. As another example, to support the "Full 1386 HAS awareness/per-Session-Logs" Approach, the dCDN cannot operate on 1387 logging format that summarize "more than" or "in an incompatible way 1388 with" the summarization specified for CDNI Logging. However, the 1389 authors feel such constraints are (i) inevitable, (ii) outweighed by 1390 the benefits of a standardized logging interface and (iii) acceptable 1391 because in case of incompatibel summarization, all/most CDNs are 1392 capable of reverting to per-chunk logging as per the Do-Nothing 1393 Approach that we recommend be used as the base minimum approach. 1395 3.5. URL Signing 1397 URL Signing is an authorization method for content delivery. This is 1398 based on embedding the HTTP URL with information that can be 1399 validated to ensure the request has legitimate access to the content. 1400 There are two parts: 1) parameters that convey authorization 1401 restrictions (e.g. source IP address and time period) and/or 1402 protected URL portion, and 2) message digest that confirms the 1403 integrity of the URL and authenticates the URL creator. The 1404 authorization parameters can be anything agreed upon between the 1405 entity that creates the URL and the entity that validates the URL. A 1406 key is used to generate the message digest (i.e. sign the URL) and 1407 validate the message digest. The two functions may or may not use 1408 the same key. 1410 There are two types of keys used for URL Signing: asymmetric keys and 1411 symmetric key. Asymmetric keys always have a key pair made up of a 1412 public key and private key. The private key and public key are used 1413 for signing and validating the URL, respectively. A symmetric key is 1414 the same key that is used for both functions. Regardless of the type 1415 of key, the entity that validates the URL has to obtain the key. 1416 Distribution for the symmetric key requires security to prevent 1417 others from taking it. Public key can be distributed freely while 1418 private key is kept by the URL signer. The method for key 1419 distribution is out of scope. 1421 URL Signing operates in the following way. A signed URL is provided 1422 by the content owner (i.e. URL signer) to the user during website 1423 navigation. When the user selects the URL, the HTTP request is sent 1424 to the CDN which validates that URL before delivering the content. 1426 3.5.1. HAS Implications 1428 The authorization lifetime for URL Signing is affected by HAS. The 1429 expiration time in the authorization parameters of URL Signing limits 1430 the period that the content referenced by the URL can be accessed. 1431 This works for URL that directly access the media content. But for 1432 HAS content, the manifest file contains another layer of URL that 1433 reference the chunks. The chunk URL that's embedded in the content 1434 may be requested at an indeterminate amount of time later. The time 1435 period between access to the manifest and chunk retrieval may vary 1436 significantly. The type of content (i.e. Live or VoD) impacts the 1437 time variance as well. HAS content has this property that needs to 1438 be addressed for URL Signing. 1440 3.5.2. CDNI Considerations 1442 For CDNI, the two types of request routing are DNS-based and HTTP- 1443 based. The use of symmetric vs. asymmetric key for URL Signing has 1444 implications on the trust model between CSP and CDNs and the key 1445 distribution method that can be used. 1447 DNS-based request routing does not change the URL. In the case of 1448 symmetric key, the CSP and the Authoritative CDN have a business 1449 relationship that allows them to share a key (or multiple keys) for 1450 URL Signing. When the user request a content from the Authoritative 1451 CDN, the URL is signed by the CSP. The Authorititative CDN (as a 1452 Upstream CDN) redirects the request to a Downstream CDN via DNS. 1453 There may be more than one level of redirection to reach the 1454 Delivering CDN. The user would obtain the IP address from DNS and 1455 send the HTTP request to the Delivering CDN, which needs to validate 1456 the URL. This requires the key to be distributed from Authoritative 1457 CDN to the Delivering CDN. This may be problematic when the key is 1458 exposed to the Delivering CDN that does not have relationship with 1459 the CSP. The combination of DNS-based request routing and symmetric 1460 key function is a generic issue for URL Signing and not specific to 1461 HAS content. In the case of asymmetric keys, CSP signs URL with its 1462 private key. The Delivering CDN validates the URL with the 1463 associated public key. 1465 HTTP request routing changes the URL during redirection procedure. 1466 In the case of symmetric key, CSP signs the original URL with the 1467 same key used by the Authoritative CDN to validate the URL. The 1468 Authoritative CDN (as a Upstream CDN) redirects the request to the 1469 Downstream CDN. The new URL is signed by the Upstream CDN with the 1470 same key used by the Downstream CDN to validate that URL. The key 1471 used by the Upstream CDN to validate the original URL is expect to be 1472 different than the key used to sign the new URL. In the case of 1473 asymmetric keys, CSP signs the original URL with its private key. 1474 Authoritative CDN validates that URL with the CSP's public key. The 1475 Authoritative CDN redirects the request to the Downstream CDN. The 1476 new URL is signed by the Upstream CDN with its private key. The 1477 Downstream CDN validates that URL with the Upstream CDN's public key. 1478 There may be more than one level of redirection to reach the 1479 Delivering CDN. The URL Signing operation described previously 1480 applies at each level between the Upstream CDN and Downstream CDN for 1481 both the symmetric key and asymmetric keys cases. 1483 URL Signing requires support in most of the CDNI Interfaces. The 1484 CDNI Metadata interface should specify the content that is subject to 1485 URL signing and provide information to perform the function. The 1486 Downstream CDN should inform the Upstream CDN that it supports URL 1487 Signing in the asynchronous capabilities information advertisement as 1488 part of the Request Routing interface. This allows the CDN selection 1489 function in request routing to choose the Downstream CDN with URL 1490 signing capability when the CDNI metadata of the content requires 1491 this authorization method. The Logging interface provides 1492 information on the authorization method (e.g. URL Signing) and 1493 related authorization parameters used for content delivery. Having 1494 the information in the URL is not sufficient to know that the 1495 surrogate enforced the authorization. URL Signing has no impact on 1496 the Control interface. 1498 3.5.3. Option 5.1: Do Nothing 1500 "Do Nothing" approach means that CSP can only perform URL Signing for 1501 the top level manifest file. The top level manifest file contains 1502 chunk URLs or lower level manifest file URLs, which are not modified 1503 (i.e. no URL Signing for the embedded URLs). In essence, the lower 1504 level manifest files and chunks are delivered without content access 1505 authorization. 1507 Effect on CDN Interfaces: 1509 o None 1511 Advantages/Drawbacks: 1513 + Top level manifest file access is protected 1515 + Upstream CDN and Downstream CDN do not need to be aware of HAS 1516 content 1518 - Lower level manifest files and chunks are not protected, making 1519 this approach unqualified for content access authorization 1521 3.5.4. Option 5.2.1: Flexible URL Signing by CSP 1523 In addition to URL Signing for the top level manifest file, CSP 1524 performs flexible URL Signing for the lower level manifest files and 1525 chunks. For each HAS session, the top level manifest file contains 1526 signed chunk URLs or signed lower level manifest file URLs for the 1527 specific session. The lower level manifest file contains session- 1528 based signed chunk URLs. CSP generates the manifest files 1529 dynamically for the session. The chunk (segment/fragment) is 1530 delivered with content access authorization using flexible URL 1531 Signing which protects the invariant portion of the URL. Segment URL 1532 (e.g. HLS) is individually signed for the invariant URL portion 1533 (Relative URL) or the entire URL (Absolute URL without Redirection) 1534 in the manifest file. Fragment URL (e.g. Smooth Streaming) is 1535 signed for the invariant portion of the template URL in the manifest 1536 file. More details are provided later in this section. The URL 1537 Signing expiration time for the chunk needs to be long enough to play 1538 the video. There are implications of signing the URLs in the 1539 manifest file. For Live content, the manifest files are requested at 1540 a high frequency. For VoD content, the manifest file may be quite 1541 large. URL Signing can add more computational load and delivery 1542 latency in high volume cases. 1544 For HAS content, the Manifest File contains the Relative Locator, 1545 Absolute Locator without Redirection, or Absolute Locator with 1546 Redirection for specifying the chunk location. Signing the chunk URL 1547 requires CSP to know the portion of the URL that remains when the 1548 content is requested from the Delivery CDN surrogate. 1550 For Absolute URL without Redirection, the CSP knows that the chunk 1551 URL which is explicitly linked with the delivery CDN surrogate and 1552 can sign the URL based on that information. Since the entire URL is 1553 set and does not change, the surrogate can validate the URL. The CSP 1554 and the Delivery CDN are expected to have a business relationship in 1555 this case. So either symmetric key or asymmetric keys can be used 1556 for URL Signing. 1558 For Relative URL, the URL of the Manifest File provides the root 1559 location. The method of request routing affects the URL used to 1560 ultimately request the chunk from the Delivery CDN surrogate. For 1561 DNS, the original URL does not change. This allows CSP to sign the 1562 chunk URL based on the Manifest File URL and the Relative URL. For 1563 HTTP, the URL changes during redirection. In this case, CSP does not 1564 know the redirected URL that will be used to request the Manifest 1565 File. This uncertainty makes it impossible to accurately sign the 1566 chunk URLs in the Manifest File. Basically, URL Signing using this 1567 reference method, "as is" for entire URL protection, is not 1568 supported. However, instead of signing the entire URL, the CSP signs 1569 the Relative URL (i.e. invariant portion of the URL) and conveys the 1570 protected portion in the authorization parameters embedded in the 1571 chunk URL. This approach works the same way as Absolute URL without 1572 Redirection, except the HOST part and (part of) the PATH part of the 1573 URL are not signed and validated. The security level should remain 1574 the same as content access authorization ensures that the user that 1575 requested the content has the credentials. This scheme does not seem 1576 to compromise the authorization model since the resource is still 1577 protected by the authorization parameters and message digest. 1578 Perhaps, further evaluation on security would be helpful. 1580 For Absolute URL with Redirection, the method of request routing 1581 affects the URL used to ultimately request the chunk from the 1582 Delivery CDN surrogate. This case has the same conditions as the 1583 Relative URL. The difference is that the URL is for the chunk 1584 instead of the Manifest File. For DNS, the chunk URL does not change 1585 and can be signed by the CSP. For HTTP, the URL used to deliver the 1586 chunk is unknown to the CSP. In this case, CSP cannot sign the URL 1587 and this method of reference for the chunk is not supported. 1589 Effect on CDN Interfaces: 1591 o Requires the ability to exclude the variant portion of URL in the 1592 signing process (NOTE: Issue is specific to URL Signing support 1593 for HAS content and not CDNI?) 1595 Advantages/Drawbacks: 1597 + Manifest file and chunks are protected 1599 + Upstream CDN and Downstream CDN do not need to be aware of HAS 1600 content 1602 + DNS-based request routing with asymmetric keys and HTTP-based 1603 request routing for Relative URL and Absolute URL without 1604 Redirection works 1606 - CSP has to generate manifest files with session-based signed URLs 1607 and becomes involved in content access authorization for every HAS 1608 session 1610 - Manifest files are not cacheable 1612 - DNS-based request routing with symmetric key may be problematic 1613 due to need for transitive trust between CSP and Delivery CDN 1615 - HTTP-based request routing for Absolute URL with Redirection does 1616 not work because the URL used Delivery CDN surrogate is unknown to 1617 the CSP 1619 3.5.5. Option 5.2.2: Flexible URL Signing by Upstream CDN 1621 This is similar to the previous section, with the exception that the 1622 Upstream CDN performs flexible URL for the lower level manifest files 1623 and chunks. URL Signing for the top level manifest file is still 1624 provided by the CSP. 1626 Effect on CDN Interfaces: 1628 o Requires the ability to exclude the variant portion of URL in the 1629 signing process (NOTE: Issue is specific to URL Signing support 1630 for HAS content and not CDNI?) 1632 Advantages/Drawbacks: 1634 + Manifest file and chunks are protected 1636 + CSP does not need to be involved in content access authorization 1637 for every HAS session 1639 + Downstream CDN does not need to be aware of HAS content 1640 + DNS-based request routing with asymmetric keys and HTTP-based 1641 request routing for Relative URL and Absolute URL without 1642 Redirection works 1644 - Upstream CDN has to generate manifest files with session-based 1645 signed URLs and becomes involved in content access authorization 1646 for every HAS session 1648 - Manifest files are not cacheable 1650 - Manifest file needs to be distributed through the uCDN 1652 - DNS-based request routing with symmetric key may be problematic 1653 due to need for transitive trust between uCDN and non-adjacent 1654 Delivery CDN 1656 - HTTP-based request routing for Absolute URL with Redirection does 1657 not work because the URL used Delivery CDN surrogate is unknown to 1658 the uCDN 1660 3.5.6. Option 5.3: Authorization Group ID and HTTP Cookie 1662 Based on the Authorization Group ID metadata, CDN validates the URL 1663 Signing or validates the HTTP cookie for request of content in the 1664 group. CSP performs URL Signing for the top level manifest file. 1665 The top level manifest file contains lower level manifest file URLs 1666 or chunk URLs. The lower level manifest files and chunks are 1667 delivered with content access authorization using HTTP cookie that 1668 contains session state associated with authorization of the top level 1669 manifest file. The Group ID Metadata is used to associate the 1670 related content (i.e. manifest files and chunks). It also specifies 1671 content (e.g. regexp method) that needs to be validated by either URL 1672 Signing or HTTP cookie. Note that the creator of the metadata is 1673 HAS-aware. Duration of the chunk access may be included in the URL 1674 Signing of the top level manifest file and set in the cookie. 1675 Duration may be provided by the CDNI Metadata interface instead. 1677 Effect on CDN Interfaces: 1679 o CDNI Metadata Interface - Authorization Group ID metadata 1680 identifies the content that is subject to validation of URL 1681 Signing or validation of HTTP cookie associated with the URL 1682 Signing 1684 o CDNI Logging Interface - Report the authorization method used to 1685 validate the request for content delivery 1687 Advantages/Drawbacks: 1689 + Manifest file and chunks are protected 1691 + CDN does not need to be aware of HAS content 1693 + CSP does not need to change the manifest files 1695 - Authorization Group ID metadata is required (i.e. CDNI Metadata 1696 Interface enhancement) 1698 - Requires the use of HTTP cookie which may be considered to be not 1699 desirable or even feasible 1701 - Manifest file has to be delivered by surrogate 1703 3.5.7. Option 5.4.1: HAS-awareness with HTTP Cookie in CDN 1705 CDN is aware of HAS content and uses URL Signing and HTTP cookie for 1706 content access authorization. URL Signing is fundamentally about 1707 authorizing access to a Content Item or its specific Content 1708 Collections (representations) for a specific user during a time 1709 period with possibly some other criteria. A chunk is an instance of 1710 the sets of chunks referenced by the Manifest File for the Content 1711 Item or its specific Content Collections. This relationship means 1712 that once the Downstream CDN has authorized the Manifest File, it can 1713 assume that the associated chunks are implicitly authorized. The new 1714 function for the CDN is to link the Manifest File with the chunks for 1715 the HTTP session. This can be accomplished by using an HTTP cookie 1716 for the HAS session. 1718 After validating the URL and detecting that the requested content is 1719 a top level Manifest File, the delivery CDN surrogate sets a HTTP 1720 cookie with a signed session token for the HTTP session. When a 1721 request for a lower level manifest file or chunk arrives, the 1722 surrogate confirms that the HTTP cookie value contains the correct 1723 session token. If so, the lower level manifest file or chunk is 1724 delivered due to transitive authorization property. Duration of the 1725 chunk access may be included in the URL Signing of the top level 1726 manifest file and set in the cookie. The details of the operation 1727 are left to be determined later. 1729 Effect on CDN Interfaces: 1731 o CDNI Metadata Interface - New metadata identifies the content that 1732 is subject to validation of URL Signing and information in the 1733 cookie for the type of HAS content 1735 o Request Routing interface - Downstream CDN should inform the 1736 Upstream CDN that it supports URL Signing for known HAS content 1737 types in the asynchronous capabilities information advertisement. 1738 This allows the CDN selection function in request routing to 1739 choose the appropriate Downstream CDN when the CDNI metadata 1740 identifies the content 1742 o CDNI Logging Interface - Report the authorization method used to 1743 validate the request for content delivery 1745 Advantages/Drawbacks: 1747 + Manifest file and chunks are protected 1749 + CSP does not need to change the manifest files 1751 - Requires full HAS awareness on part of Upstream CDN and Downstream 1752 CDN 1754 - Requires CDNI Interfaces extensions 1756 - Requires the use of HTTP cookie which may be considered to be not 1757 desirable or even feasible 1759 - Manifest file has to be delivered by surrogate 1761 3.5.8. Option 5.4.2: HAS-awareness with Manifest in CDN 1763 CDN is aware of HAS content and uses URL Signing for content access 1764 authorization of manifest file and chunk. CDN generates or rewrites 1765 the manifest files and learns about the chunks based on the manifest 1766 file. The embedded URLs in the manifest file are signed by the CDN. 1767 Duration of the chunk access may be included in the URL Signing. The 1768 details of the operation are left to be determined later. Since this 1769 approach is based on signing the URLs in the manifest file, the 1770 implications for Live and VoD content mentioned in Section 3.5.4 1771 apply. 1773 Effect on CDN Interfaces: 1775 o CDNI Metadata Interface - New metadata identifies the content that 1776 is subject to validation of URL Signing and information in the 1777 cookie for the type of HAS content 1779 o Request Routing interface - Downstream CDN should inform the 1780 Upstream CDN that it supports URL Signing for known HAS content 1781 types in the asynchronous capabilities information advertisement. 1782 This allows the CDN selection function in request routing to 1783 choose the appropriate Downstream CDN when the CDNI metadata 1784 identifies the content 1786 o CDNI Logging Interface - Report the authorization method used to 1787 validate the request for content delivery 1789 Advantages/Drawbacks: 1791 + Manifest file and chunks are protected 1793 + CSP does not need to change the manifest files 1795 - Requires full HAS awareness on part of Upstream CDN and Downstream 1796 CDN 1798 - Requires CDNI Interfaces extensions 1800 - Requires CDN to generate or rewrite the manifest file 1802 - Manifest file has to be delivered by surrogate 1804 3.5.9. URL Signing Options Analysis 1806 Summary: 1808 "Do nothing" (#1) approach requires no change to CSP or CDN but is 1809 not acceptable because of the lack of protection for the content. 1810 "Flexible URL Signing" (#2) approach requires flexible URL Signing 1811 support in CDN and depends on CSP (#2A) or Upstream CDN (#2B) to be 1812 involved in every HAS session. "Authorization Group ID and HTTP 1813 Cookie" (#3) approach requires new CDNI metadata to associate the URL 1814 Signing with HTTP cookie to validate a request for content in the 1815 logical group. "HAS-awareness in CDN" (#4) requires CDN to be aware 1816 of HAS content and to support HAS with the use of HTTP cookie (#4A) 1817 or manifest manipulation (#4B). 1819 Recommendations: 1821 Flexible URL Signing by CSP (#2A) or Upstream CDN (#2B) is 1822 recommended because the approach protects all the content, does not 1823 require Downstream CDN to be aware of HAS, does not impact CDNI 1824 interfaces, supports all different types of devices, and supports the 1825 common cases of request routing for HAS content (i.e. DNS-based 1826 request routing with asymmetric keys and HTTP-based request routing 1827 for Relative URL). The requirement for Upstream CDN to manipulate 1828 the manifest file is not considered to be a significant obstacle for 1829 deployment as long as the Downstream CDN remains unaware of HAS. 1831 (FUTURE) HAS-awareness in CDN (#4) has some advantages that should be 1832 considered for future support (e.g. CDN that is aware of HAS content 1833 can manage the content more efficiently at a broader context. 1835 Content distribution, storage, delivery, deletion, access 1836 authorization, etc. can all benefit.) 1838 3.6. Content Purge 1840 At some point in time, a uCDN might want to remove content from a 1841 dCDN. With regular content, this process can be relatively 1842 straightforward; a uCDN will typically send the request for content 1843 removal to the dCDN including a reference to the content which it 1844 wants to remove (e.g. in the form of a URL). Due to the fact that 1845 HAS content consists of large groups of files however, things might 1846 be more complex. Section 3.1 describes a number of different 1847 scenarios for doing file management on these groups of files, while 1848 Section 3.2 list the options for performing Content Acquisition on 1849 these Content Collections. This section will present the options for 1850 requesting a Content Purge for the removal of a Content Collection 1851 from a dCDN. 1853 3.6.1. Option 6.1: No HAS awareness 1855 The most straightforward way to signal content purge requests is to 1856 just send a single purge request for every file that makes up the 1857 Content Collection. While this method is very simple and does not 1858 require HAS awareness, it obviously creates a large signalling 1859 overhead between the uCDN and dCDN. 1861 Effect on CDN Interfaces: 1863 o None 1865 Advantages/Drawbacks (apart from those listed under Option 3.3): 1867 + Does not require changes to the CDNI Interfaces or HAS awareness 1869 - Requires individual purge request for every file making up a 1870 Content Collection which creates large signalling overhead 1872 3.6.2. Option 6.2: Purge Identifiers 1874 There exists a potentially more efficient method for performing 1875 content removal of large numbers of files simultaneously. By 1876 including purge identifiers in the metadata of a particular file, it 1877 is possible to virtually group together different files making up a 1878 Content Collection. A purge identifier can take the form of a random 1879 number which is communicated as part of the CDNI Metadata Interface 1880 and which is the same for all files making up a particular Content 1881 Item. If a uCDN wants to request the dCDN to remove a Content 1882 Collection, it can send a purge request containing this purge 1883 identifier. The dCDN can then remove all files that contain the 1884 shared identifier. 1886 The advantage of this method is that it is relatively simple to use 1887 by both the dCDN and uCDN and requiring only limited additions to the 1888 CDNI Metadata Interface and CDNI Control Interface. 1890 [Editor's Note: Could the Purge Identifier introduced in this section 1891 be related to the Content Collection Identifier introduced in 1892 Section 3.4.2.2? Chould they be the same identifier?] 1894 Effect on CDN Interfaces: 1896 o CDNI Metadata Interface: Add metadata field for indicating Purge 1897 Identifier 1899 o CDNI Control Interface: Add functionality to be able to send 1900 content purge requests containing Purge Identifiers 1902 Advantages/Drawbacks: 1904 + Allows for efficient purging of content from a dCDN 1906 + Does not require HAS awareness on part of dCDN 1908 3.7. Other issues 1910 This section includes some HAS-specific issues that came up during 1911 the discussion of this draft and which do not fall under any of the 1912 categories discussed in the previous sections. 1914 - As described in Section 2.2, a manifest file might either be 1915 delivered by a CDN or by the CSP, thereby being invisible to the 1916 CDN delivering the chunks. Obviously, the decision on whether the 1917 CDN or CSP delivers the manifest is made between the uCDN and CSP, 1918 and the dCDN has no choice in the matter. However, some dCDNs 1919 might only want to offer their services in the cases where they 1920 have access to the manifest file (e.g. because their internal 1921 architecture is based around the knowledge inside the manifest 1922 file). For these cases, it might be useful to include a field in 1923 the CDNI Capability Advertisement to allow dCDNs to advertise the 1924 fact that they require access to the manifest file. 1926 4. IANA Considerations 1928 This document makes no request of IANA. 1930 Note to RFC Editor: this section may be removed on publication as an 1931 RFC. 1933 5. Security Considerations 1935 TBD. 1937 6. Acknowledgements 1939 The authors would like to thank Kevin J. Ma, Stef van der Ziel, and 1940 Bhaskar Bhupalam for their valuable contributions to this draft. 1942 7. References 1944 7.1. Normative References 1946 [I-D.ietf-cdni-problem-statement] 1947 Niven-Jenkins, B., Le Faucheur, F., and N. Bitar, "Content 1948 Distribution Network Interconnection (CDNI) Problem 1949 Statement, draft-ietf-cdni-problem-statement-03", 1950 January 2012. 1952 [I-D.ietf-cdni-use-cases] 1953 Bertrand, G., Ed., Stephan, E., Watson, G., Burbridge, T., 1954 Eardley, P., and K. Ma, "Use Cases for Content Delivery 1955 Network Interconnection, draft-ietf-cdni-use-cases-03", 1956 January 2012. 1958 7.2. Informative References 1960 [I-D.draft-bertrand-cdni-logging] 1961 Bertrand, G., Ed. and E. Stephan, "CDNI Logging 1962 Interface". 1964 [I-D.draft-ietf-cdni-requirements] 1965 Leung, K. and Y. Lee, "Content Distribution Network 1966 Interconnection (CDNI) Requirements, 1967 draft-ietf-cdni-requirements-03", June 2012. 1969 [I-D.draft-lefaucheur-cdni-logging-delivery] 1970 Le Faucheur, F., Viveganandhan, M., and K. Leung, "CDNI 1971 Logging Formats for HTTP and HTTP Adaptive Streaming 1972 Deliveries". 1974 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 1975 Resource Identifier (URI): Generic Syntax, RFC3986", 1976 January 2005. 1978 Authors' Addresses 1980 Ray van Brandenburg 1981 TNO 1982 Brassersplein 2 1983 Delft 2612CT 1984 the Netherlands 1986 Phone: +31-88-866-7000 1987 Email: ray.vanbrandenburg@tno.nl 1989 Oskar van Deventer 1990 TNO 1991 Brassersplein 2 1992 Delft 2612CT 1993 the Netherlands 1995 Phone: +31-88-866-7000 1996 Email: oskar.vandeventer@tno.nl 1998 Francois Le Faucheur 1999 Cisco Systems 2000 Greenside, 400 Avenue de Roumanille 2001 Sophia Antipolis 06410 2002 France 2004 Phone: +33 4 97 23 26 19 2005 Email: flefauch@cisco.com 2007 Kent Leung 2008 Cisco Systems 2009 170 West Tasman Drive 2010 San Jose, CA 95134 2011 USA 2013 Phone: +1 408-526-5030 2014 Email: kleung@cisco.com