idnits 2.17.1 draft-brandenburg-cdni-has-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 27, 2012) is 4313 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'I-D.ietf-cdni-use-cases' is defined on line 1863, but no explicit reference was found in the text == Outdated reference: A later version (-08) exists of draft-ietf-cdni-problem-statement-03 == Outdated reference: A later version (-10) exists of draft-ietf-cdni-use-cases-03 Summary: 0 errors (**), 0 flaws (~~), 4 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 CDNI R. van Brandenburg 3 Internet-Draft O. van Deventer 4 Intended status: Informational TNO 5 Expires: December 29, 2012 F. Le Faucheur 6 K. Leung 7 Cisco Systems 8 June 27, 2012 10 Models for adaptive-streaming-aware CDN Interconnection 11 draft-brandenburg-cdni-has-02 13 Abstract 15 This documents presents thoughts on the potential impact of 16 supporting HTTP Adaptive Streaming technologies in CDN 17 Interconnection scenarios. Our intent is to spur discussion on how 18 the different CDNI interfaces could, and should, deal with content 19 delivered using adaptive streaming technologies and to facilitate 20 working group decisions. 22 Status of this Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at http://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on December 29, 2012. 39 Copyright Notice 41 Copyright (c) 2012 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (http://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 57 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 58 2. HTTP Adaptive Streaming aspects relevant to CDNI . . . . . . . 5 59 2.1. Segmentation versus Fragmentation . . . . . . . . . . . . 5 60 2.2. Addressing chunks . . . . . . . . . . . . . . . . . . . . 6 61 2.2.1. Relative URLs . . . . . . . . . . . . . . . . . . . . 8 62 2.2.2. Absolute URLs with Redirection . . . . . . . . . . . . 8 63 2.2.3. Absolute URL without Redirection . . . . . . . . . . . 9 64 2.3. Live vs. VoD . . . . . . . . . . . . . . . . . . . . . . . 10 65 2.4. Stream splicing . . . . . . . . . . . . . . . . . . . . . 11 66 3. Possible HAS Optimizations . . . . . . . . . . . . . . . . . . 11 67 3.1. File Management and Content Collections . . . . . . . . . 12 68 3.1.1. General Remarks . . . . . . . . . . . . . . . . . . . 12 69 3.1.2. Candidate approaches . . . . . . . . . . . . . . . . . 12 70 3.1.2.1. Option 1.1: No HAS awareness . . . . . . . . . . . 12 71 3.1.2.2. Option 1.2: Allow single file storage of 72 fragmented content . . . . . . . . . . . . . . . . 13 73 3.1.2.3. Option 1.3: Access correlation hint . . . . . . . 14 74 3.1.3. Recommendation . . . . . . . . . . . . . . . . . . . . 14 75 3.2. Content Acquisition of Content Collections . . . . . . . . 15 76 3.2.1. General Remarks . . . . . . . . . . . . . . . . . . . 15 77 3.2.2. Candidate Approaches . . . . . . . . . . . . . . . . . 15 78 3.2.2.1. Option 2.1: No HAS awareness . . . . . . . . . . . 15 79 3.2.2.2. Option 2.2: Allow single file acquisition of 80 fragmented content . . . . . . . . . . . . . . . . 16 81 3.2.3. Recommendation . . . . . . . . . . . . . . . . . . . . 16 82 3.3. Request Routing of HAS content . . . . . . . . . . . . . . 17 83 3.3.1. General remarks . . . . . . . . . . . . . . . . . . . 17 84 3.3.2. Candidate approaches . . . . . . . . . . . . . . . . . 17 85 3.3.2.1. Option 3.1: No HAS awareness . . . . . . . . . . . 17 86 3.3.2.2. Option 3.2: Manifest File rewriting by uCDN . . . 19 87 3.3.2.3. Option 3.3: Two-step Manifest File rewriting . . . 21 88 3.3.3. Recommendation . . . . . . . . . . . . . . . . . . . . 22 89 3.4. Logging . . . . . . . . . . . . . . . . . . . . . . . . . 22 90 3.4.1. General remarks . . . . . . . . . . . . . . . . . . . 22 91 3.4.2. Candidate Approaches . . . . . . . . . . . . . . . . . 23 92 3.4.2.1. Option 4.1: "Do-Nothing" Approach . . . . . . . . 23 93 3.4.2.2. Option 4.2: "CDNI Metadata Content Collection 94 ID" Approach . . . . . . . . . . . . . . . . . . . 25 95 3.4.2.3. Option 4.3: "CDNI Metadata Content Collection 96 ID With dCDN Summarization" Approach . . . . . . . 26 97 3.4.2.4. Option 4.4: "CDNI Logging Interface 98 Compression" Approach . . . . . . . . . . . . . . 27 99 3.4.2.5. Option 4.5: "Full HAS 100 awareness/per-Session-Logs" Approach . . . . . . . 28 101 3.4.3. Recommendation . . . . . . . . . . . . . . . . . . . . 30 102 3.5. URL Signing . . . . . . . . . . . . . . . . . . . . . . . 31 103 3.5.1. URL Signing in CDNI . . . . . . . . . . . . . . . . . 32 104 3.5.2. Option 5.1: Do Nothing about HAS Content . . . . . . . 33 105 3.5.3. Option 5.2: Flexible URL Signing for HAS Content . . . 34 106 3.5.4. Option 5.3: Authorization Group ID for HAS Content . . 36 107 3.5.5. Option 5.4: Handle HAS Content in CDN . . . . . . . . 36 108 3.5.6. URL Signing Options Analysis . . . . . . . . . . . . . 38 109 3.6. Content Purge . . . . . . . . . . . . . . . . . . . . . . 38 110 3.6.1. Option 6.1: No HAS awareness . . . . . . . . . . . . . 39 111 3.6.2. Option 6.2: Purge Identifiers . . . . . . . . . . . . 39 112 4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 40 113 5. Security Considerations . . . . . . . . . . . . . . . . . . . 40 114 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 40 115 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 40 116 7.1. Normative References . . . . . . . . . . . . . . . . . . . 40 117 7.2. Informative References . . . . . . . . . . . . . . . . . . 41 118 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 41 120 1. Introduction 122 HTTP Adaptive Streaming (HAS) is an umbrella term for various HTTP- 123 based streaming technologies that allow a client to adaptively switch 124 between multiple bitrates depending on current network conditions. A 125 defining aspect of HAS is that, since it is based on HTTP, it is a 126 pull-based mechanism, with a client actively requesting content 127 segments, instead of the content being pushed to the client by a 128 server. Due to this pull-based nature, media servers delivering 129 content using HAS often show different characteristics when compared 130 with media servers delivering content using traditional streaming 131 methods such as RTP/RTSP, RTMP and MMS. This document presents a 132 discussion on what the impact of these different characteristics is 133 to the CDNI interfaces and what HAS-specific optimizations may be 134 required or may be desirable. The scope of this document in its 135 current form is explicitly not to propose any specific solution, but 136 merely to present the available options so that the WG can make an 137 informed decision on which way to go. 139 1.1. Terminology 141 This document uses the terminology defined in 142 [I-D.ietf-cdni-problem-statement]. 144 In addition, the following terms are used throughout this document: 146 Content Item: A uniquely addressable content element in a CDN. A 147 content item is defined by the fact that it has its own Content 148 Metadata associated with it. It is the object of a request routing 149 operation in a CDN. An example of a Content Item is a video file/ 150 stream, an audio file/stream or an image file. 152 Chunk: a fixed length element that is the result of a segmentation or 153 fragmentation operation and that is independently addressable. 155 Fragment: A specific form of chunk (see Section 2.1). A fragment is 156 stored as part of a larger file that includes all chunks that are 157 part of the Chunk Collection. 159 Segment: A specific form of chunk (see Section 2.1). A segment is 160 stored as a single file from a file system perspective. 162 Original Content: Not-chunked content that is the basis for a 163 segmentation of fragmentation operation. Based on Original Content, 164 multiple alternative representations (using different encoding 165 methods, supporting different resolutions and/or targeting different 166 bitrates) may be derived, each of which may be fragmented or 167 segmented. 169 Chunk Collection: The set of all chunks that are the result of a 170 single segmentation or fragmentation operation being performed on a 171 single representation of the Original Content. A Chunk Collection is 172 described in a Manifest File. 174 Content Collection: The set of all Chunk Collections that are derived 175 from the same Original Content. A Content Collection may consist of 176 multiple Chunk Collections, each corresponding to a single 177 representation of the Original Content. A Content Collection may be 178 described by one or more Manifest Files. 180 Manifest File: A Manifest File, also referred to as Media 181 Presentation Description (MPD) file, is a file that list the way the 182 content has been chunked (possibly for multiple encodings) and where 183 the various chunks are located (in the case of segments) or how they 184 can be addressed (in the case of fragments). 186 2. HTTP Adaptive Streaming aspects relevant to CDNI 188 In the last couple of years, a wide variety of HAS-like protocols 189 have emerged. Among them are proprietary solutions such as Apple's 190 HTTP Live Streaming (HLS), Microsoft's Smooth Streaming (HSS) and 191 Adobe's HTTP Dynamic Streaming (HDS), and various standardized 192 solutions such as 3GPP Adaptive HTTP Streaming (AHS) and MPEG Dynamic 193 Adaptive Streaming over HTTP (DASH). While all of these technologies 194 share a common set of features, each has its own defining elements. 195 This chapter will look at some of the common characteristics and some 196 of the differences between these technologies and how those might be 197 relevant to CDNI. In particular, Section 2.1 will describe the 198 various methods to store HAS content and Section 2.2 will list three 199 methods that are used to address HAS content in a CDN. After these 200 generic HAS aspects are discussed, two special situations that need 201 to be taken into account when discussing HAS are addressed: 202 Section 2.3 discusses the differences between Live and VoD content, 203 while Section 2.4 discusses the scenario where multiple streams are 204 combined in a single manifest files (e.g. for ad insertion purposes). 206 2.1. Segmentation versus Fragmentation 208 All HAS implementations are based around a concept referred to as 209 chunking: the concept of having a server split content up in numerous 210 fixed duration chunks, which are independently decodable. By 211 sequentially requesting and receiving chunks, a client can recreate 212 and play out the content. An advantage of this mechanism is that it 213 allows a client to seamlessly switch between different encodings of 214 the same Original Content at chunk boundaries. Before requesting a 215 particular chunk, a client can choose between multiple alternative 216 encodings of the same chunk, irrespective of the encoding of the 217 chunks it has requested earlier. 219 While every HAS implementation uses some form of chunking, not all 220 implementations store the resulting chunks in the same way. In 221 general, there are two distinct methods of performing chunking and 222 storing the results: segmentation and fragmentation. 224 - With segmentation, which is for example mandatory in all versions 225 of Apple's HLS prior to version 7, the chunks, in this case also 226 referred to as segments, are stored completely independent from 227 each other, with each segment being stored as a separate file from 228 a file system perspective. This means that each segment has its 229 own unique URL with which it can be retrieved. 231 - With fragmentation (or virtual segmentation), which is for example 232 used in Microsoft's Smooth Streaming, all chunks, or fragments, 233 belonging to the same Chunk Collection are stored together, as 234 part of a single file. While there are a number of container 235 formats which allow for storing this type chunked content, 236 Fragmented MP4 is most commonly used. With fragmentation, a 237 specific chunk is addressable by subfixing the common file URL 238 with an identifier uniquely identifying the chunk one is 239 interested in, either by timestamp, by byterange, or in some other 240 way. 242 While one can argue about the merits of each of these two different 243 methods of handling chunks, both have their advantages and drawbacks 244 in a CDN environment. For example, fragmentation is often regarded 245 as a method that introduces less overhead, both from a storage and 246 processing perspective. Segmentation on the other hand, is regarded 247 as being more flexible and easier to cache. In practice, current HAS 248 implementations increasingly support both methods. 250 2.2. Addressing chunks 252 In order for a client to request chunks, either in the form of 253 segments or in the form of fragments, it needs to know how the 254 content has been chunked and where to find the chunks. For this 255 purpose, most HAS protocols use a concept that is often referred to 256 as a Manifest File (also known as Media Presentation Description, or 257 MPD); i.e. a file that lists the way the content has been chunked and 258 where the various chunks are located (in the case of segments) or how 259 they can be addressed (in the case of fragments). A Manifest File, 260 or set of Manifest Files, may also identify the different encodings, 261 and thus Chunk Collections, the content is available in. 263 In general, a HAS client will first request and receive a Manifest 264 File, and then, after parsing the information in the Manifest File, 265 proceed with sequentially requesting the chunks listed in the 266 Manifest File. Each HAS implementation has its own Manifest File 267 format and even within a particular format there are different 268 methods available to specify the location of a chunk. 270 Of course managing the location of files is a core aspect of every 271 CDN, and each CDN will have its own method of doing so. Some CDNs 272 may be purely cache-based, with no higher-level knowledge of where 273 each file resides at each instant in time. Other CDNs may have 274 dedicated management nodes which, at each instant in time, do know at 275 which servers each file resides. The CDNI interfaces designed in the 276 CDNI WG will probably need to be agnostic to these kinds of CDN- 277 internal architecture decisions. In the case of HAS there is a 278 strict relationship between the location of the content in the CDN 279 (in this case chunks) and the content itself (the locations specified 280 in the Manifest File). It is therefore useful to have an 281 understanding of the different methods in use in CDNs today for 282 specifying chunk locations in Manifest Files. The different methods 283 for doing so are described in sections 2.2.1 to 2.2.3. 285 Although these sections are especially relevant for segmented 286 content, due to its inherent distributed nature, the discussed 287 methods are also applicable to fragmented content. Furthermore, it 288 should be noted that the methods detailed below for specifying 289 locations of content items in Manifest Files do not only relate to 290 temporally segmented content (e.g. segments and fragments), but are 291 also relevant in situations where content is made available in 292 multiple representations (e.g., in different qualities, encoding 293 methods, resolutions and/or bitrates). In this case the content 294 consists of multiple chunk collections, which may be described by 295 either a single Manifest File or multiple interrelated manifest 296 files. In the latter case, there may be a high-level Manifest File 297 describing the various available bitrates, with URLs pointing to 298 separate Manifest Files describing the details of each specific 299 bitrate. For specifying the locations of the other Manifest Files, 300 the same methods apply that are used for specifying chunk locations. 302 One final note relates to the delivery of the manifest files 303 themselves. While in most situations the delivery of both the 304 manifest file and the chunks are handled by the CDN, there are 305 scenarios imaginable in which the manifest file is delivered by e.g. 306 the Content Provider, and the manifest is therefore not visible to 307 the CDN. 309 2.2.1. Relative URLs 311 One method for specifying chunk locations in a Manifest File is 312 through the use of relative URLs. A relative URL is a URL that does 313 not include the HOST part of a URL but only includes (part of) the 314 PATH part of a URL. In practice, a relative URL is used by the 315 client as being relative to the location where the Manifest File has 316 been acquired from. In these cases a relative URL will take the form 317 of a string that has to be appended to the location of the Manifest 318 File to get the location of a specific chunk. This means that in the 319 case a manifest with relative URLs is used, all chunks will be 320 delivered by the same surrogate that delivered the Manifest File. A 321 relative URL will therefore not include a hostname. 323 For example, in the case a Manifest File has been requested (and 324 received) from: 326 http://surrogate.server.cdn.example.com/content_1/manifest.xml 328 , a relative URL pointing to a specific segment referenced in the 329 manifest might be: 331 segments/segment1_1.ts 333 Which means that the client should take the location of the manifest 334 file and append the relative URL. In this case, the segment would 335 then be requested from http://surrogate.server.cdn.example.com/ 336 content_1/segments/segment1_1.ts 338 The downside of using relative URLs is that it forces a CDN to 339 deliver all segments belonging to a given content item with the same 340 surrogate that delivered the Manifest File for that content item, 341 which results in limited flexibility. Another drawback is that 342 Relative URLs do not allow for fallback URLs; should the surrogate 343 that delivered the manifest file break down, the client is no longer 344 able to request chunks. The advantage of relative URLs is that it is 345 very easy to transfer content between different surrogates and even 346 CDNs. 348 2.2.2. Absolute URLs with Redirection 350 Another method for specifying locations of chunks (or other manifest 351 files) in a Manifest File is through the use of an absolute URL. An 352 absolute URL contains a fully formed URL (i.e. the client does not 353 have to calculate the URL as in the case of the relative URL but can 354 use the URL from the manifest directly). 356 In the context of Manifest Files, there are two types of absolute 357 URLs imaginable: Absolute URLs with Redirection and Absolute URLs 358 without Redirection. The two methods differ in whether the URL 359 points to a request routing node which will redirect the client to a 360 surrogate (Absolute URL with Redirection) or point directly to a 361 surrogate hosting the requested content (Absolute URL without 362 Redirection). 364 In the case of Absolute URLs with Redirection, a request for a chunk 365 is handled by the request routing system of a CDN just as if it were 366 a standalone (non-HAS) content request, which might include looking 367 up the surrogate (and/or CDN) best suited for delivering the 368 requested chunk to the particular user and sending an HTTP redirect 369 to the user with the URL pointing to the requested chunk on the 370 specified surrogate (and/or CDN), or a DNS response pointing to the 371 specific surrogate. 373 An example of an Absolute URL with Redirection might look as follows: 375 http://requestrouting.cdn.example.com/ 376 content_request?content=content_1&segment=segment1_1.ts 378 As can be seen from this example URL, the URL includes a pointer to a 379 general CDN request routing function and includes some arguments 380 identifying the requested segment. 382 The advantage of using Absolute URLs with Redirection is that it 383 allows for maximum flexibility (since chunks can be distributed 384 across surrogates and CDN in any imaginable way) without having to 385 modify the Manifest File every time one or more chunks are moved (as 386 is the case when Absolute URLs without Redirection are used). The 387 downside of this method is that it can adds significant load to a CDN 388 request routing system, since it has to perform a redirect every time 389 a client requests a new chunk. 391 2.2.3. Absolute URL without Redirection 393 In the case of the Absolute URL without Redirection, the URL points 394 directly to the specific chunk on the actual surrogate that will 395 deliver the requested chunk to the client. In other words, there 396 will be no HTTP redirection operation taking place between the client 397 requesting the chunk and the chunk being delivered to the client by 398 the surrogate. 400 An example of an Absolute URLs without Redirection is the following: 402 http://surrogate.cdn.example.com/content_1/segments/segment1_1.ts 404 As can be seen from this example URL, the URL includes both the 405 identifier of the requested segment (in this case segment1_1.ts), as 406 well as the server that is expected to deliver the segment (in this 407 case surrogate.cdn.example.com). With this, the client has enough 408 information to directly request the specific segment from the 409 specified surrogate. 411 The advantage of using Absolute URLs without Redirection is that it 412 allows more flexibility compared to using Relative URLs (since 413 segments do not necessarily have to be delivered by the same server) 414 while not requiring per-segment redirection (which would add 415 significant load to the node doing the redirection). The drawback of 416 Absolute URLs without Redirection is that it requires a modification 417 of the Manifest File every time content is moved to a different 418 location (either within a CDN or across CDNs). 420 2.3. Live vs. VoD 422 Though the formats and addresses of manifest and chunk files do not 423 typically differ significantly between live and Video-on-Demand (VoD) 424 content, the time at which the manifests and chunk files become 425 available does differ significantly. For live content, chunk files 426 and their corresponding manifest files are created and delivered in 427 real-time. This poses a number of potential issues for HAS 428 optimization: 430 - With live content, chunk files are made available in real-time. 431 This limits the applicability of bundling for content acquisition 432 purposes. Prepositioning may still be employed, however, any 433 significant latency in the prepositioning may diminish the value 434 of prepositioning if a client requests the chunk prior to 435 prepositioning, or if the prepositioning request is serviced after 436 the chunk playout time has passed. 438 - In the case of live content, manifest files must be updated for 439 each chunk and therefore must be retrieved by the client prior to 440 each chunk request. Any manifest-based optimization schemes must 441 therefore be prepared to optimize on a per-segment request basis. 442 Manifest files may also be polled multiple times prior to the 443 actual availability of the next chunk. 445 - Since live manifest files are updated as each new chunk becomes 446 available, the cacheability of manifest files is limited. Though 447 timestamping and reasonable TTLs can improve delivery performance, 448 timely replication and delivery of updated manifest files is 449 critical to ensuring uninterrupted playback. 451 - Manifest files are typically updated after the corresponding chunk 452 is available for delivery, to prevent premature requests for 453 chunks which are not yet available. HAS optimization approaches 454 which employ dynamic manifest generation must be synchronized with 455 chunk creation to prevent playback errors. 457 2.4. Stream splicing 459 Stream splicing is used to create media mashups, combining content 460 from multiple sources. A common example in which content resides 461 outside the CDNs is with advertisement insertion, for both VoD and 462 live streams. Manifest files which contain Absolute URLs with 463 redirection may contain chunk or nested manifest URLs which point to 464 content not delivered via any of the interconnected CDNs. 466 Furthermore, client and downstream proxy devices may depend on non- 467 URL information provided in the manifest (e.g., comments or custom 468 tags) for performing stream splicing. This often occurs outside the 469 scope of the interconnected CDNs. HAS optimization schemes which 470 employ dynamic manifest generation or rewriting must be cognizant of 471 chunk URLs, nested manifest URLs, and other metadata which should not 472 be modified or removed. Improper modification of these URLs or other 473 metadata may cause playback interruptions, and in the case of 474 unplayed advertisements, may result in loss of revenue for content 475 providers. 477 3. Possible HAS Optimizations 479 In the previous chapter, some of the unique properties of HAS have 480 been discussed. Furthermore, some of the CDN-specific design 481 decisions with regards to addressing chunks have been detailed. In 482 this chapter, the impact of supporting HAS in CDN Interconnection 483 scenarios will be discussed. 485 There are a number of topics, or problem areas, that are of 486 particular interest when considering the combination of HAS and CDNI. 487 For each of these problem areas it holds that there are a number of 488 different ways in which the CDNI Interfaces can deal with them. In 489 general it can be said that each problem area can either be solved in 490 a way that minimizes the amount of HAS-specific changes to the CDNI 491 Interfaces or in way that maximizes the flexibility and efficiency 492 with which the CDNI Interfaces can deliver HAS content. The goal for 493 the CDNI WG should probably be to try to find the middle ground 494 between these two extremes and try to come up with solutions that 495 optimize the balance between efficiency and additional complexity. 497 In order to allow the WG to make this decision, this chapter will 498 briefly describe each of the following problem areas together with a 499 number of different options for dealing with them. Section 3.1 will 500 discuss the problem of how to deal with file management of groups of 501 files, or Content Collections. Section 3.2 will deal with a related 502 topic: how to do content acquisition of Content Collections between 503 the uCDN and dCDN. After that, Section 3.3 describes the various 504 options for the request routing of HAS content, particularly related 505 to Manifest Files. Section 3.4 talks about a number of possible 506 optimizations for the logging of HAS content, while Section 3.5 507 discusses the options regarding URL signing. Section 3.6 finally, 508 describes different scenarios for dealing with the removal of HAS 509 content from CDNs. 511 3.1. File Management and Content Collections 513 3.1.1. General Remarks 515 One of the unique properties of HAS content is that it does not 516 consist of a single file or stream but of multiple interrelated files 517 (segment, fragments and/or Manifest Files). In this document this 518 group of files is also referred to as a Content Collection. Another 519 important aspect is the difference between segments and fragments 520 (see Section 2.1). 522 Irrespective of whether segments or fragments are used, different 523 CDNs might handle Content Collections differently from a file 524 management perspective. For example, some CDNs might handle all 525 files belonging to a Content Collection as individual files, which 526 are stored independently from each other. An advantage of this 527 approach is that makes it easy to cache individual chunks. Other 528 CDNs might store all fragments belonging to a Content Collection in a 529 bundle, as if they were a single file (e.g. by using a fragmented MP4 530 container). The advantage of this approach is that it reduces file 531 management overhead. 533 This section will look at the various ways with which the CDNI 534 interfaces might deal with these differences in handling Content 535 Collections from a file management perspective. The different 536 options can be distinguished based on the level of HAS-awareness they 537 require on the part of the different CDNs and the CDNI interfaces. 539 3.1.2. Candidate approaches 541 3.1.2.1. Option 1.1: No HAS awareness 543 This first option assumes no HAS awareness in both the involved CDNs 544 and the CDNI Interfaces. This means that the uCDN uses individual 545 files and the dCDN is not explicitely made aware of the relationship 546 between chunks and it doesn't know which files are part of the same 547 Content Collection. In practice this scenario would mean that the 548 file management method used by the uCDN is simply imposed on the dCDN 549 as well. 551 This scenario also means that it is not possible for the dCDN to use 552 any form of file bundling, such as the single-file mechanism which 553 can be to store fragmented content as a single file (see 554 Section 2.1). The one exception to this rule is the situation where 555 the content is fragmented and the Manifest Files on the uCDN contains 556 byte range requests, in which case the dCDN might be able to acquire 557 fragmented content as a single file (see Section 3.2.2.2). 559 Effect on CDN Interfaces: 561 o None 563 Advantages/Drawbacks: 565 + No HAS awareness necessary in CDNs, no changes to CDNI Interfaces 566 necessary 568 - The dCDN is forced to store chunks as individual files. 570 3.1.2.2. Option 1.2: Allow single file storage of fragmented content 572 In some cases, the dCDN might prefer to store fragmented content as a 573 single file on its surrogates to reduce file management overhead. In 574 order to do so, it needs to be able to either acquire the content as 575 a single file (see Section 3.2.2.2), or merge the different chunks 576 together and place them in the same container (e.g. fragmented MP4). 577 The downside of this is that in order to do so, the dCDN needs to be 578 fully HAS aware. 580 Effect on CDN Interfaces: 582 o CDNI Metadata Interface: Add fields for indicating the particular 583 type of HAS (e.g. MPEG DASH or HLS) that is used and whether 584 segments or fragments are used 586 o CDNI Metadata Interface: Add field for indicating the name and 587 type of the manifest file(s) 589 Advantages/Drawbacks: 591 + Allows dCDN to store fragmented content as a single file, reducing 592 file management overhead 594 - Complex operation, requiring dCDN to be fully HAS aware 596 3.1.2.3. Option 1.3: Access correlation hint 598 An intermediary approach between the two extremes detailed in the 599 previous two sections is one that uses a 'Access Correlation Hint'. 600 This hint, which is added to the CDNI Metadata of all chunks of a 601 particular Content Collection, indicates that those files are likely 602 to be requested in a short time window from each other. This 603 information can help a dCDN to implement local file storage 604 optimizations for VoD items (e.g. by bundling all files with the same 605 Access Correlation Hint value in a single bundle/file), thereby 606 reducing the number of files it has to manage while not requiring any 607 HAS awareness. 609 Effect on CDN Interfaces: 611 o CDNI Metadata Interface: Add field for indicating Access 612 Correlation Hint 614 Advantages/Drawbacks: 616 + Allows dCDN to perform file management optimization 618 + Does not require any HAS awareness 620 + Very small impact on CDNI Interfaces 622 - Expected benefit compared with Option 1.1 is small 624 3.1.3. Recommendation 626 Based on the listed pros and cons, the authors recommend the WG to go 627 for Option 1.1, the 'Do Nothing'-approach. The likely benefits from 628 going for Option 1.3 are not believed to be significant enough to 629 warrant changing the CDNI Metadata Interface. Although Option 1.2 630 would bring definite benefits for HAS aware dCDNs, going for this 631 options would require significant CDNI extensions that would impact 632 the WG's milestones. The authors therefore don't recommend to 633 include it in the current work but mark it as a possible candidate 634 for rechartering once the initial CDNI solution is completed. 636 3.2. Content Acquisition of Content Collections 638 3.2.1. General Remarks 640 In the previous section the relationship between file management and 641 HAS in a CDNI scenario has been discussed. This section will discuss 642 a related topic, which is content acquisition between two CDNs. 644 With regards to content acquisition, it is important to note the 645 difference between CDNs that do Dynamic Acquisition of content and 646 CDNs that perform Content Pre-positioning. In the case of dynamic 647 acquisition, a CDN only requests a particular content item when a 648 cache-miss occurs. In the case of pre-positioning, a CDN pro- 649 actively places content items on the nodes on which it expects 650 traffic for that particular content item. For each of these types of 651 CDNs, there might be a benefit in being HAS aware. For example, in 652 the case of dynamic acquisition, being HAS aware means that after a 653 cache miss for a giving chunk occurs, that node might not only 654 acquire the requested chunk, but might also acquire some related 655 chunks that are expected to be requested in the near future. In the 656 case of pre-positioning, similar benefits can be had. 658 3.2.2. Candidate Approaches 660 3.2.2.1. Option 2.1: No HAS awareness 662 This first option assumes no HAS awareness in both the involved CDNs 663 and the CDNI Interfaces. Just as with Option 1.1 discussed in the 664 previous section with regards to file management, having no HAS 665 awareness means that the dCDN is not aware of the relationship 666 between chunks. In the case of content acquisition, this means that 667 each and every file belonging to a Content Collection will have to be 668 individually acquired from the uCDN by the dCDN. The exception to 669 the rule is in cases with fragmented content where the uCDN uses 670 Manifest Files which contain byte range requests. In this case the 671 dCDN can simply omit the byte range identifier and acquire the 672 complete file. 674 The advantage of this approach is that it is highly flexible. If a 675 client only requests a small portion of the chunks belonging to a 676 particular Content Collection, the dCDN only has to acquire those 677 chunks from the uCDN, saving both bandwidth and storage capacity. 679 The downside of acquiring content on a per-chunk basis is that it 680 creates more transaction overhead between the dCDN and uCDN compared 681 to a method in which entire Content Collections can be acquired as 682 part of one transaction. 684 Effect on CDN Interfaces: 686 o None 688 Advantages/Drawbacks: 690 + Per-chunk content acquisition allows for high level of flexibility 691 between dCDN and uCDN 693 - Per-chunk content acquisition creates more transaction overhead 694 between dCDN and uCDN 696 3.2.2.2. Option 2.2: Allow single file acquisition of fragmented 697 content 699 As discussed in Section 3.2.2.1, there is one (fairly rare) in cases 700 where fragmented content can be acquired as a single file without any 701 HAS awareness and that is when fragmented content is used and where a 702 Manifest File includes byte range request. This section discusses 703 how to perform single file acquisition in the other (very common) 704 cases. To do so, the dCDN would have to have full-HAS awareness (at 705 least to the extent of being able to map between single file and 706 individual chunks to serve). 708 Effect on CDN Interfaces: 710 o CDNI Metadata Interface: Add fields for indicating the particular 711 type of HAS (e.g. MPEG DASH or HLS) that is used and whether 712 segments or fragments are used 714 o CDNI Metadata Interface: Add field for indicating the name and 715 type of the manifest file(s) 717 Advantages/Drawbacks: 719 + Allows for more efficient content acquisition in all HAS-specific 720 supported forms 722 - Requires full HAS awareness on part of dCDN 724 - Requires significant CDNI Metadata Interface extensions 726 3.2.3. Recommendation 728 Based on the listed pros and cons, the authors recommend the WG to go 729 for Option 2.1 since it is sufficient to 'make HAS work'. While 730 Option 2.2 would bring benefits to the acquisition of large Content 731 Collections, it would require significant CDNI extensions which would 732 impact the WG's milestones. Option 2.2 might be a candidate to 733 include in possible rechartering once the initial CDNI solution is 734 completed. 736 3.3. Request Routing of HAS content 738 3.3.1. General remarks 740 In this section the effect HAS content has on request routing will be 741 identified. Of particular interest in this case are the different 742 types of Manifest Files that might be used. In Section 2.2, three 743 different methods for identifying and addressing chunks from within a 744 Manifest File were described: Relative URLs, Absolute URLs without 745 Redirection and Absolute URLs with Redirection. Of course not every 746 current CDN will use and/or support all three methods. Some CDNs may 747 only use one of the three methods, while others may support two or 748 all three. 750 An important factor in deciding which chunk addressing method is used 751 is the Content Provider. Some Content Providers may have a strong 752 preference for a particular method and deliver the Manifest Files to 753 the CDN in a particular way. Depending on the CDN and the agreement 754 it has with the Content Provider, a CDN may either host the Manifest 755 Files as they were created by the Content Provider, or modify the 756 Manifest File to adapt it to its particular architecture (e.g. by 757 changing relative URLs to Absolute URLs which point to the CDN 758 Request Routing function). 760 3.3.2. Candidate approaches 762 3.3.2.1. Option 3.1: No HAS awareness 764 This first option assumes no HAS awareness in both the involved CDNs 765 and the CDNI Interfaces. This scenario also assumes that neither the 766 dCDN nor the uCDN have the ability to actively manipulate Manifest 767 Files. As was also discussed with regards to file management and 768 content acquisition, having no HAS awareness means that each file 769 constituting a Content Collections is handled on an individual basis, 770 with the dCDN unaware of any relationship between files. 772 The only chunk addressing method that works without question in this 773 case is Absolute URLs with Redirection. In other words, the Content 774 Provider that ingested the content into the uCDN created a Manifest 775 File with each chunk location pointing to the Request Routing 776 function of the uCDN. Alternatively, the Content Provider may have 777 ingested the Manifest File containing relative URLs and the uCDN 778 ingestion function has translated these to Absolute URLs pointing to 779 the Request Routing function. 781 In this Absolute URL with Redirection case, the uCDN can simply have 782 the Manifest File be delivered by the dCDN as if it were a regular 783 file. Once the client parses the Manifest File, it will request any 784 subsequent chunks from the uCDN Request Routing function. That 785 function can then decide to outsource the delivery of that chunk to 786 the dCDN. Depending on whether HTTP-based (recursive or iterative) 787 or DNS-based request routing is used, the uCDN Request Routing 788 function will then either directly or indirectly redirect the client 789 to the Request Routing function of the dCDN (assuming it does not 790 have the necessary information to redirect the client directly to a 791 surrogate in the dCDN). 793 The drawback of this method is that it creates a large amount of 794 request routing overhead for both the uCDN and dCDN. For each chunk 795 the full inter-CDN Request Routing process is invoked (which can 796 result in two HTTP redirections in the case of iterative redirection, 797 or result in one HTTP redirection plus one CDNI Request Routing/ 798 Redirection Interface request/response). Even in the case where DNS- 799 based redirection is used, there might be significant overhead 800 involved since both the dCDN and uCDN Request Routing function might 801 have to perform database lookups and query each other. While with 802 DNS this overhead might be reduced by using DNS' inherent caching 803 mechanism, this will have significant impact on the accuracy of the 804 redirect. 806 With no HAS awareness, Relative URLs might or might not work 807 depending on the HAS client implementation that is used. When a uCDN 808 delegates the delivery of a Manifest File containing Relative URLs to 809 a dCDN, the client goes directly to the dCDN surrogate from which it 810 has received the Manifest File for every subsequent chunk. The 811 problem that might arise here is that the dCDN surrogate might not 812 have enough information to relate the chunk request back to a 813 particular uCDN. WIthout this information, it is impossible for the 814 dCDN surrogate to deliver the chunk and properly log this request as 815 being delivered on behalf of the uCDN. 817 Since using Absolute URLs without Redirection inherently require a 818 HAS aware CDN, they also cannot be used in this case. The reason for 819 this is that with Absolute URLs without Redirection, the URLs in the 820 Manifest File will point directly to a surrogate in the uCDN. Since 821 this scenario assumes no HAS awareness on the part of the dCDN or 822 uCDN, it is impossible for either of these CDNs to rewrite the 823 Manifest File and thus allow the client to either go to a surrogate 824 in the dCDN or to a request routing function. 826 Effect on CDN Interfaces: 828 o None 830 Advantages/Drawbacks: 832 + Supports Absolute URLs with Redirection 834 + Supports Relative URLs 836 + Does not require HAS awareness and/or changes to the CDNI 837 Interfaces 839 - Not possible to use Absolute URLs without Redirection 841 - Brittleness when Relative URLs are used. Makes assumptions on 842 client-side implementation of the HAS client or on structure of 843 PATH or QUERY 845 - Creates significant signaling overhead in case Absolute URLs with 846 Redirection are used (inter-CDN request redirection for each 847 chunk) 849 3.3.2.2. Option 3.2: Manifest File rewriting by uCDN 851 While Option 3.1 does allow for Absolute URLs with Redirection to be 852 used, it does so in a way that creates a high-level of request 853 routing overhead for both the dCDN and the uCDN. This option 854 presents a solution to significantly reduce this overhead. 856 In this scenario, the uCDN is able to modify the Manifest File to be 857 able to remove itself from the request routing chain for chunks being 858 referenced in the Manifest File. As described in Section 3.3.2.1, in 859 the case of no HAS awareness the client will go to the uCDN request 860 routing function for each chunk request. This request routing 861 function can then redirect the client to the dCDN request routing 862 function. By rewriting the Manifest File, the uCDN is able to remove 863 this first step, and have the Manifest File point directly to the 864 dCDN request routing function. 866 The main advantage of this solution is that it does not direcly have 867 an impact on the CDNI Interfaces and is therefore relatively 868 transparent to these interfaces. It is a function that a uCDN can 869 perform independently by using the information that it receives from 870 the dCDN as part of the regular CDNI Request Routing Interface 871 communication. 873 More specifically, in order for the uCDN to rewrite the manifest 874 file, it only needs the location of the dCDN request routing function 875 (or even better: the location of the dCDN surrogate). Obtaining this 876 information is part of a the regular CDNI Request Routing Interface 877 and can be done in of of two ways. The first way would be to have 878 the uCDN ask the dCDN for the location of its request routing node 879 (through the CDNI Request Routing/Redirection Interface) every time a 880 request for a Manifest File comes in at the uCDN request routing 881 function. The uCDN would then modify the manifest file and deliver 882 the manifest file to the client. A second way to do it would be for 883 the modification of the manifest file to only happen once, when the 884 first client for that particular Content Collection (and redirected 885 to that particular dCDN) sends a Manifest File request. The 886 advantage of the first method is that it maximizes effiency and 887 flexibility by allowing the dCDN to respond with the locations of its 888 surrogates instead of the location of its request routing function 889 (and effectively turning the URLs into Absolute URLs without 890 Redirection). The advantage of the second method is that the uCDN 891 only has to modify the Manifest File once. 893 It should be noted that there are a number of things to take into 894 account when changing a manifest file (see for example Section 2.3 895 and Section 2.4 on live HAS content and ad insertion). Furthermore, 896 some Content Providers might have issues with a CDN changing manifest 897 files. However, in this option the manifest manipulation is only 898 being performed by the uCDN, which can be expected to be aware of 899 these limitations if it wants to perform manifest manipulation since 900 it is in its own best interest that it's customer's content gets 901 delivered in the proper way. Should the Content Provider want to 902 limit manifest manipulation, it can simply arrange this with the uCDN 903 bilaterally. 905 Effect on CDN Interfaces: 907 o None 909 Advantages/Drawbacks: 911 + Possible to significantly decrease signalling overhead when using 912 Absolute URLs. 914 + (Optional) Possible to have uCDN modify manifest with locations of 915 surrogates in dCDN (turning Absolute URLs with Redirection in 916 Absolute URLs without Redirection) 918 + No changes to CDNI Interfaces 920 + Does not require HAS awareness in dCDN 921 - Requires high level of HAS awareness in uCDN (for modifying 922 manifest files) 924 3.3.2.3. Option 3.3: Two-step Manifest File rewriting 926 One of the possibilities with Option 3.3 is allowing the dCDN to 927 provide the locations of a specific surrogate to the uCDN, so that 928 the uCDN can fit the Manifest File with Absolute URLs without 929 Redirection and the client can request chunks directly from a dCDN 930 surrogate. However, some dCDNs might not be willing to provide this 931 information to the uCDN. In that case they can only provide the uCDN 932 with the location of their request routing function and thereby not 933 be able to use Absolute URLs without Redirection. 935 One method for solving this limitation is allowing two-step Manifest 936 File manipulation. In the first step the uCDN would perform its own 937 modification, and place the locations of the dCDN request routing 938 function in the Manifest File. Then, once a request for the Manifest 939 File comes in at the dCDN request routing function, it would perform 940 a second modification in which it replaces the URLs in the Manifest 941 Files with the URLs of its surrogates. This way the dCDN can still 942 profit from having minimal request routing traffic, while not having 943 to share sensitive surrogate information with the uCDN. 945 The downside of this approach is that it not only assumes HAS 946 awareness in the dCDN but that it also requires some HAS-specific 947 additions to the CDNI Metadata Interface. In order for the dCDN to 948 be able to change the Manifest File, it has to have some information 949 about the structure of the content. Specifically, it needs to have 950 information about which chunks make up the Content Collection. 952 Effect on CDN Interfaces (apart from those listed under Option 3.3): 954 o CDNI Metadata Interface: Add necessary fields for conveying HAS 955 specific information (e.g. the files that make up the Content 956 Collection) to the dCDN. 958 o dCDN: Allow for modification of manifest file 960 Advantages/Drawbacks (apart from those listed under Option 3.3): 962 + Allows dCDN to use Absolute URLs without Redirection without 963 having to convey sensitive information to the uCDN 965 - Requires high level of HAS awareness in dCDN (for modifying 966 manifest files) 968 - Requires adding HAS-specific information to the CDNI Metadata 969 Interface 971 3.3.3. Recommendation 973 Based on the listed pros and cons, the authors recommend to go for 974 Option 3.1, with Option 3.2 as an optional feature for uCDN that 975 support this. While Option 3.1 allows for HAS content to be 976 delivered using the CDNI interfaces, it does so with some limitations 977 regarding supported manifest files and with large signalling 978 overhead. Option 3.2 can solve most of these limitations and 979 presents a significant reduction of the request routing overhead. 980 Since Option 3.2 does not require any changes to the CDNI interfaces 981 but only changes the way the uCDN uses the existing interfaces, 982 supporting it is not expected to result in a significant delay of the 983 WG's milestones. The authors recommend the WG to not include Option 984 3.3, since it raises some questions of potential brittleness and 985 including it would result in a significant delay of the WG's 986 milestones. 988 3.4. Logging 990 3.4.1. General remarks 992 As stated in [I-D.ietf-cdni-problem-statement], "the CDNI Logging 993 interface enables details of logs or events to be exchanged between 994 interconnected CDNs". 996 As discussed in [I-D.draft-bertrand-cdni-logging], the CDNI logging 997 information can be used for multiple purposes including maintenance/ 998 debugging by uCDN, accounting (e.g. in view of billing or 999 settlement), reporting and management of end-user experience (e.g. to 1000 the CSP), analytics (e.g. by the CSP) and control of content 1001 distribution policy enforcement (e.g. by the CSP). 1003 The key consideration for HAS with respect to logging is the 1004 potential increase of the number of Log records by two to three 1005 orders of magnitude, as compared to regular HTTP delivery of a video, 1006 since, by default, log records would typically be generated on a per- 1007 chunk-delivery basis instead of per-content-item-delivery basis. 1008 This impacts the scale of every processing step in the Logging 1009 Process (see Section 8 of [I-D.draft-bertrand-cdni-logging]), 1010 including: 1012 a. Logging Generation and Storing of logs on CDN elements 1013 (Surrogate, Request Routers,..) 1015 b. Logging Aggregation within a CDN 1017 c. Logging Manipulation (including Logging Protection, Logging 1018 Filtering , Logging Update and Rectification) 1020 d. (Where needed) Logging CDNI Reformatting (e.g. reformatting from 1021 CDN-specific format to the CDNI Logging Interface format for 1022 export by dCDN to uCDN) 1024 e. Logging exchange via CDNI Logging Interface 1026 f. (Where needed) Logging Re-Reformatting (e.g. reformatting from 1027 CDNI Logging Interface format into log-consuming specific 1028 application) 1030 g. Logging consumption/processing (e.g. feed logs into uCDN 1031 accounting application, feed logs into uCDN reporting system to 1032 provide per CSP views, feed logs into debugging tool to debug) 1034 Note that there may be multiple instances of step [f] and [g] running 1035 in parallel. 1037 While the CDNI Logging Interface is only used to perform step [e], we 1038 note that its format directly affects step [d] and [f] and that its 1039 format also indirectly affects the other steps (for example if the 1040 CDNI Logging Interface requires per-chunk log records, step [a], [b] 1041 and [d] cannot operate on a per-HAS-session basis and also need to 1042 operate on a per-chunk basis). 1044 3.4.2. Candidate Approaches 1046 The following sub-sections discusses the main candidate approaches 1047 identified so far for CDNI in terms of dealing with HAS with respect 1048 to Logging. 1050 3.4.2.1. Option 4.1: "Do-Nothing" Approach 1052 In this approach nothing is done specifically for HAS so that each 1053 HAS-chunk delivery is considered, for CDNI Logging, as a standalone 1054 content delivery. In particular, a separate log record for each HAS- 1055 chunk delivery is included in the CDNI Logging Interface in step [e]. 1056 This approach requires that step [a], [b], [c], [d] and [e] also be 1057 performed on a per-chunk basis. This approach allows [g] to be 1058 performed either on a per-chunk basis (assuming step [f] maintains 1059 per-chunk records) or on a more "summarized" manner such as per-HAS- 1060 Session basis (assuming step [f] summarizes per-chunk records into 1061 per-HAS-session records). 1063 Effect on CDN Interfaces: 1065 o None 1067 Effect on uCDN and dCDN: 1069 o None 1071 Advantages/Drawbacks: 1073 + No information loss (i.e. all details of each individual chunk 1074 delivery are preserved). While this full level of detail may not 1075 be needed for some Log consuming applications (e.g. billing), this 1076 full level of detail is likely valuable (possibly required) for 1077 some Log consuming applications (e.g. debugging) 1079 + Easier integration (at least in the short term) into existing 1080 Logging tools since those are all capable of handling per-chunk 1081 records 1083 + No extension needed on CDNI interfaces 1085 - High volume of logging information to be handled (storing & 1086 processing) at every step of the Logging process from [a] to [g] 1087 (while summarization in step [f] is conceivable, it may be 1088 difficult to achieve in practice without any hints for correlation 1089 in the log records). While the high volume of logging information 1090 is a potential concern, we are seeking expert input on whether it 1091 is a real practical issue, and if yes, then in what timeframe/ 1092 assumptions. 1094 An interesting question is whether a dCDN could use the CDNI Logging 1095 interface specified for the "Do-Nothing" approach to report 1096 summarized "per-session" log information in the case where the dCDN 1097 performs such summarization. The high level idea would be that, when 1098 a dCDN performs HAS log summarization for its own purposes anyways, 1099 this dCDN could include, in the CDNI Logging interface, one (or a 1100 few) log entry for a HAS session (instead of one entry per HAS-chunk) 1101 that summarizes the deliveries of many/all HAS-chunk for a session. 1102 However, the authors feel that, when considering the details of this, 1103 this is not achievable without explicit agreement between the uCDN 1104 and dCDN about how to perform/interpret such summarization. For 1105 example, when a HAS session switches between representations, the 1106 uCDN and dCDN would have to agree on things such as: 1108 o whether the session will be represented by a single log entry 1109 (which therefore cannot convey the distribution across 1110 representations) or multiple log entries such as one entry per 1111 contiguous period at a given representation (which therefore would 1112 be generally very difficult to correlate back into a single 1113 session) 1115 o what would the single URI included in the log entry correspond to? 1116 the manifest/top-level-playlist/next-level-playlist,... 1118 The authors feel that if explicit agreement is needed between uCDN 1119 and dCDN on how to perform/interpret the summarization is required, 1120 then this should be specified as part of the CDNI Logging interface 1121 and then effectively boils down to Option 4.5 defined below ("Full 1122 HAS awareness" and "per-Session-Logs" Approach). 1124 We note that support by CDNI of a mechanism (independent of HAS) 1125 allowing the customization of the fields to be reported in log 1126 entries by the dCDN to the uCDN would have a minor mitigation effect 1127 on the HAS logging scaling concerns because it ensures that only the 1128 necessary subset of fields are actually stored, reported and 1129 processed. 1131 3.4.2.2. Option 4.2: "CDNI Metadata Content Collection ID" Approach 1133 In this approach, a "Content Collection ID" (CCID) field is 1134 distributed through the CDNI Metadata Interface and the same CCID 1135 value is associated through the CDNI Metadata interface with every 1136 chunk of the same Content Collection. The CCID value needs to be 1137 such that it allows, in combination with the content URI, to uniquely 1138 identify a Content Collection. When distributed, and CCID logging is 1139 requested from the dCDN, the dCDN Surrogates are to store the CCID 1140 value in the corresponding log entries. The objective of this field 1141 is to facilitate optional summarization of per-chunk records at step 1142 [f] into something along the lines of per-HAS-session logs, at least 1143 for the Log consuming applications that do not require per-chunk 1144 detailed information (for example billing). 1146 [Editor's Note: would there be value in adding a little more info in 1147 the metadata such as which HAS-scheme is used?] 1149 Effect on CDN Interfaces: 1151 o One additional metadata field (CCID) in CDNI Metadata Interface. 1152 We note that a similar Content Collection ID is discussed for 1153 handling of other aspects of HAS and observe that further thought 1154 is needed to determine whether such CCID should be shared for 1155 multiple purposes or should be independent. 1157 Effect on uCDN and dCDN: 1159 o None 1161 Advantages/Drawbacks: 1163 + No information loss (i.e. all details of each individual chunk 1164 delivery are preserved). While this full level of detail may not 1165 be needed for some Log consuming applications (e.g. billing), this 1166 full level of detail is likely valuable (possibly required) for 1167 some Log consuming applications (e.g. debugging) 1169 + Easier integration (at least in the short term) into existing 1170 Logging tools since those are all capable of handling per-chunk 1171 records 1173 + Very minor extension to CDNI interfaces needed 1175 + Facilitated summarization of records related to a HAS session in 1176 step [f] and therefore ability to operate on lower volume of 1177 logging information in step [g] by log consuming applications that 1178 do not need per-chunk record details (e.g. billing) or that need 1179 per-session information (e.g. analytics) 1181 - High volume of logging information to be handled (storing & 1182 processing) at every step of the Logging process from [a] to [f]. 1183 While the high volume of logging information is a potential 1184 concern, we are seeking input on whether it is a real practical 1185 issue, and if yes in what timeframe/assumptions 1187 3.4.2.3. Option 4.3: "CDNI Metadata Content Collection ID With dCDN 1188 Summarization" Approach 1190 In this approach, the same "Content Collection ID" (CCID) field as 1191 discussed in Option 4.2 is distributed through the CDNI Metadata 1192 Interface, and the same CCID value is associated with every chunk of 1193 the same Content Collection. In this approach, a summarization of 1194 per-chunk records is performed at step [d] (or in earlier steps 1195 within dCDN) taking advantage of the CCID, so that a reduced volume 1196 of logging information is to be handled in steps [e] to [g] of the 1197 logging process (and is optionally also possible in steps [a] to 1198 [c]). The objective of this approach is to reduce the volume of 1199 logging information early in the Logging process. 1201 Regarding the summarization performed at step [d] (or in earlier 1202 steps), there is a continuum in terms of trade-off between level of 1203 summarization of per-chunk records and information loss. For 1204 example, it appears possible to perform a summarization that results 1205 is significant gains with limited information loss, perhaps using 1206 summarized logs along the lines of the Event-Based Logging format 1207 discussed in section 3.2.2 of 1208 [I-D.draft-lefaucheur-cdni-logging-delivery]. Alternatively, it may 1209 be possible to perform a summarization that results in very 1210 significant gains with significant information loss, perhaps using 1211 summarized logs along the lines of the Summary-Based Logging format 1212 discussed in section 3.2.3 of 1213 [I-D.draft-lefaucheur-cdni-logging-delivery]. 1215 Effect on CDN Interfaces: 1217 o One additional metadata field (CCID) in CDNI Metadata Interface 1219 o Summarized logging information in CDNI Logging Information 1221 Effect on uCDN and dCDN: 1223 o None 1225 Advantages/Drawbacks: 1227 + Lower volume of logging information to be handled (storing & 1228 processing) at every step of the Logging process from [e] to [g], 1229 and optionally from [a] to [d] also 1231 + Small extensions to CDNI interfaces needed 1233 - Some information loss (i.e. all details of each individual chunk 1234 delivery are not preserved). The actual information loss depends 1235 on the summarization approach selected (typically the lower the 1236 information loss, the lower the summarization gain) so the right 1237 sweet-spot would had ego be selected. While full level of detail 1238 may not be needed for some Log consuming applications (e.g. 1239 billing), the full level of detail is likely valuable (possibly 1240 required) for some Log consuming applications (e.g. debugging) 1242 - Less easy integration (at least in the short term) into existing 1243 Logging tools since those are all capable of handling per-chunk 1244 records and may not be capable of handling CDNI summarized records 1246 3.4.2.4. Option 4.4: "CDNI Logging Interface Compression" Approach 1248 In this approach, a loss-less compression technique is applied to the 1249 sets of Logging records (e.g. Logging files) for transfer on the 1250 IETF CDNI Logging Interface. The objective of this approach is to 1251 reduce the volume of information to be stored and transferred in step 1252 [e]. 1254 Effect on CDN Interfaces: 1256 o One additional compression mechanism to be included in the CDNI 1257 Logging Interface 1259 Effect on uCDN and dCDN: 1261 o None 1263 Advantages/Drawbacks: 1265 + No information loss (i.e. all details of each individual chunk 1266 delivery are preserved). While this full level of detail may not 1267 be needed for some Log consuming applications (e.g. billing), this 1268 full level of detail is likely valuable (possibly required) for 1269 some Log consuming applications (e.g. debugging) 1271 + Easier integration (at least in the short term) into existing 1272 Logging tools since those are all capable of handling per-chunk 1273 records 1275 + Small extension to CDNI interfaces needed 1277 + Reduced volume of logging information in step [e] 1279 + Compression likely to be also applicable to logs for non-HAS 1280 content 1282 - High volume of logging information to be handled (storing & 1283 processing) at every step of the Logging process from [a] to [g], 1284 except [e]. While the high volume of logging information is a 1285 potential concern, we are seeking expert input on whether it is a 1286 real practical issue, and if yes, then in what timeframe/ 1287 assumptions 1289 Input is sought on expected compression gains achievable in practice 1290 over sets of logs containing per-chunk records. 1292 3.4.2.5. Option 4.5: "Full HAS awareness/per-Session-Logs" Approach 1294 In this approach, HAS-awareness is assumed across the CDNs 1295 interconnected via CDNI and the necessary information to describe the 1296 HAS relationship across all chunks of the same Content Collection is 1297 distributed through the CDNI Metadata Interface. In this approach, 1298 the dCDN Surrogates leverage the HAS information distributed through 1299 the CDNI metadata and their HAS-awareness to generate summarized 1300 logging information in the very first place. The objective of that 1301 approach is to operate on lower volume of logging information right 1302 from the very first step of the Logging process. 1304 The summarized HAS logs generated by the Surrogates in this approach 1305 are similar to those discussed in the section " "CDNI Metadata 1306 Content Collection ID With dCDN Summarization" Approach" and the same 1307 trade-offs between information loss and summarization gain apply. 1309 Effect on CDN Interfaces: 1311 o One significant extension of the CDNI Metadata Interface to convey 1312 HAS relationship across chunks of a Content Collection. Note that 1313 this extension requires specific support for every HAS-protocol to 1314 be supported over the CDNI mesh 1316 Effect on uCDN and dCDN: 1318 o Full HAS-awareness by dCDN Surrogates 1320 Advantages/Drawbacks: 1322 + Lower volume of logging information to be handled (storing & 1323 processing) at every step of the Logging process from [a] to [g] 1325 + Accurate generation of summarized logs because of HAS awareness on 1326 Surrogate (for example, where the Surrogate is also serving the 1327 manifest file(s) for a content collection, the Surrogate may be 1328 able to extract definitive information about the relationship 1329 between all chunks) 1331 - Very significant extensions to CDNI interfaces needed including 1332 per HAS-protocol specific support 1334 - Very significant additional requirement for HAS awareness on dCDN 1335 and for this HAS-awareness to be consistent with the defined CDNI 1336 Logging summarization 1338 - Some information loss (i.e. all details of each individual chunk 1339 delivery are not preserved). The actual information loss depends 1340 on the summarization approach selected (typically the lower the 1341 information loss, the lower the summarization gain) so the right 1342 sweet-spot would had ego be selected. While full level of detail 1343 may not be needed for some Log consuming applications (e.g. 1344 billing), the full level of detail is likely valuable (possibly 1345 required) for some Log consuming applications (e.g. debugging) 1347 - Less easy integration (at least in the short term) into existing 1348 Logging tools since those are all capable of handling per-chunk 1349 records and may not be capable of handling CDNI summarized records 1351 - Challenges in defining behavior (and achieving summarization gain) 1352 in the presence of load-balancing of a given HAS-session across 1353 multiple Surrogates (in same or different dCDN) 1355 Input is sought on expected compression gains achievable in practice 1356 over sets of logs containing per-chunk records. 1358 3.4.3. Recommendation 1360 Because of its benefits (in particular simplicity, universal support 1361 by CDNs and support by all log-consuming applications), we recommend 1362 that the per-chunk logging of Option 4.1 be supported by the CDNI 1363 Logging interface as a "High Priority" (as defined in 1364 ) and be a mandatory capability of CDNs 1365 implementing CDNI. 1367 Because of its very low complexity and its benefit in facilitating 1368 some useful scenarios (e.g. per-session analytics), we recommend that 1369 the CCID mechanisms of Option 4.2 be supported by the CDNI Metadata 1370 interface and the CDNI Logging interface as a "Medium Priority" (as 1371 defined in ) and be an optional 1372 capability of CDNs implementing CDNI. 1374 We recommend that: 1376 (i) the ability for the uCDN to request that the CCID field be 1377 included in log entries provided by the dCDN be supported by the 1378 relevant CDNI interfaces (tentatively the CDNI Metadata interface) 1379 and 1381 (ii) the ability for the dCDN to include the CCID field in CDNI log 1382 entries (when the dCDN is capable of doing so) and indicate so 1383 inside the CDNI Logging interface (in line with the "customizable" 1384 log format expected to be defined independently of HAS), 1386 be supported as a "Medium Priority" (as defined in 1387 ) and be an optional capability of CDNs 1388 implementing CDNI. 1390 Because it can be achieved with very little complexity and it 1391 provides some clear storage/communication compression benefits, we 1392 recommend that some existing very common compression techniques (e.g. 1393 gzip) be supported by the CDNI Logging interface as a "Medium 1394 Priority" (as defined in ) and be an 1395 optional capability of CDNs implementing CDNI. 1397 Because of its complexity, the time it would take to understand the 1398 trade-offs of candidate summarization approaches and the time it 1399 would take to specify the corresponding support in the CDNI Logging 1400 interface, we recommend that the log summarization discussed in 1401 option 4.3 and option 4.5 not be supported by the CDNI Logging 1402 interface at this stage and be kept as candidate topic of great 1403 interest for a rechartering of the CDNI WG once the first set of 1404 deliverables is produced. When doing so, we suggest to investigate 1405 the notion of complementing the "push-style" CDNI logging interface 1406 supporting summarization by an on-demand pull-type of interface 1407 allowing an uCDN to request the subset of the detailed logging 1408 information that it may need but is lost in the summarized pushed 1409 information. 1411 We note that while a CDN only needs to adhere to the CDNI Logging 1412 interface on its external interfaces and can perform logging in a 1413 different format within the CDN, any possible CDNI Logging approach 1414 effectively places some constraints on the dCDN logging format. For 1415 example, to support the "Do-Nothing" Approach, a CDN need to perform 1416 and retain per chunk logs. As another example, to support the "Full 1417 HAS awareness/per-Session-Logs" Approach, the dCDN cannot operate on 1418 logging format that summarize "more than" or "in an incompatible way 1419 with" the summarization specified for CDNI Logging. However, the 1420 authors feel such constraints are (i) inevitable, (ii) outweighed by 1421 the benefits of a standardized logging interface and (iii) acceptable 1422 because in case of incompatibel summarization, all/most CDNs are 1423 capable of reverting to per-chunk logging as per the Do-Nothing 1424 Approach that we recommend be used as the base minimum approach. 1426 3.5. URL Signing 1428 URL Signing is an authorization method for content delivery. This is 1429 based on embedding the HTTP URL with information that can be 1430 validated to ensure the request has legitimate access to the content. 1431 There are two parts: 1) parameters that convey authorization 1432 restrictions (e.g. source IP address and time period) and/or 1433 protected URL portion, and 2) message digest that confirms the 1434 integrity of the URL and authenticates the URL creator. The 1435 authorization parameters can be anything agreed upon between the 1436 entity that creates the URL and the entity that validates the URL. A 1437 key is used to generate the message digest (i.e. sign the URL) and 1438 validate the message digest. The two functions may or may not use 1439 the same key. 1441 There are two types of keys used for URL Signing: asymmetric keys and 1442 symmetric key. Asymmetric keys always have a key pair made up of a 1443 public key and private key. The private key and public key are used 1444 for signing and validating the URL, respectively. A symmetric key is 1445 the same key that is used for both functions. Regardless of the type 1446 of key, the entity that validates the URL has to obtain the key. 1448 Distribution for the symmetric key requires security to prevent 1449 others from taking it. Public key can be distributed freely while 1450 private key is kept by the URL signer. The method for key 1451 distribution is out of scope. 1453 URL Signing operates in the following way. A signed URL is provided 1454 by the content owner (i.e. URL signer) to the user during website 1455 navigation. When the user selects the URL, the HTTP request is sent 1456 to the CDN which validates that the URL before delivering the 1457 content. 1459 3.5.1. URL Signing in CDNI 1461 For CDNI, the two types of request routing are DNS-based and HTTP- 1462 based. The use of symmetric vs. asymmetric key for URL Signing has 1463 implications on the trust model between CSP and CDNs and the key 1464 distribution method that can be used. 1466 DNS-based request routing does not change the URL. In the case of 1467 symmetric key, the CSP and the Authoritative CDN have a business 1468 relationship that allows them to share a key (or multiple keys) for 1469 URL Signing. When the user request a content from the Authoritative 1470 CDN, the URL is signed by the CSP. The Authorititative CDN (as a 1471 Upstream CDN) redirects the request to a Downstream CDN via DNS. 1472 There may be more than one level of redirection to reach the 1473 Delivering CDN. The user would obtain the IP address from DNS and 1474 send the HTTP request to the Delivering CDN, which needs to validate 1475 the URL. This requires the key to be distributed from Authoritative 1476 CDN to the Delivering CDN. This may be problematic when the key is 1477 exposed to the Delivering CDN that does not have relationship with 1478 the CSP. The combination of DNS-based request routing and symmetric 1479 key function is a generic issue for URL Signing and not specific to 1480 HAS content. In the case of asymmetric keys, CSP signs URL with its 1481 private key. The Delivering CDN validates the URL with the 1482 associated public key. 1484 HTTP request routing changes the URL during redirection procedure. 1485 In the case of symmetric key, CSP signs the original URL with the 1486 same key used by the Authoritative CDN to validate the URL. The 1487 Authoritative CDN (as a Upstream CDN) redirects the request to the 1488 Downstream CDN. The new URL is signed by the Upstream CDN with the 1489 same key used by the Downstream CDN to validate that URL. The key 1490 used by the Upstream CDN to validate the original URL is expect to be 1491 different than the key used to sign the new URL. In the case of 1492 asymmetric keys, CSP signs the original URL with its private key. 1493 Authoritative CDN validates that URL with the CSP's public key. The 1494 Authoritative CDN redirects the request to the Downstream CDN. The 1495 new URL is signed by the Upstream CDN with its private key. The 1496 Downstream CDN validates that URL with the Upstream CDN's public key. 1497 There may be more than one level of redirection to reach the 1498 Delivering CDN. The URL Signing operation described previously 1499 applies at each level between the Upstream CDN and Downstream CDN for 1500 both the symmetric key and asymmetric keys cases. 1502 URL Signing requires support in most of the CDNI Interfaces. The 1503 CDNI Metadata interface should specify the content that is subject to 1504 URL signing and provide information to perform the function. The 1505 Downstream CDN should inform the Upstream CDN that it supports URL 1506 Signing in the asynchronous capabilities information advertisement as 1507 part of the Request Routing interface. This allows the CDN selection 1508 function in request routing to choose the Downstream CDN with URL 1509 signing capability when the CDNI metadata of the content requires 1510 this authorization method. The Logging interface provides 1511 information on the authorization method (e.g. URL Signing) and 1512 related authorization parameters used for content delivery. URL 1513 Signing has no impact on the Control interface. 1515 3.5.2. Option 5.1: Do Nothing about HAS Content 1517 "Do Nothing" approach means that CSP can only perform URL Signing for 1518 the top level manifest file. The top level manifest file contains 1519 chunk URLs or lower level manifest file URLs, which are not modified 1520 (i.e. no URL Signing for the embedded URLs). In essence, the lower 1521 level manifest files and chunks are delivered without content access 1522 authorization. 1524 Effect on CDN Interfaces: 1526 o None 1528 Advantages/Drawbacks: 1530 + Top level manifest file access is protected 1532 + Does not require HAS awareness and/or changes to the CDNI 1533 Interfaces 1535 + CSP does not need to change the manifest files (embedded URLs 1536 remains the same) 1538 - Lower level manifest files and chunks are not protected, making 1539 this approach unqualified for content access authorization 1541 3.5.3. Option 5.2: Flexible URL Signing for HAS Content 1543 In addition to URL Signing for the top level manifest file, CSP 1544 performs flexible URL Signing for the lower level manifest files and 1545 chunks. For each HAS session, the top level manifest file contains 1546 signed chunk URLs or signed lower level manifest file URLs for the 1547 specific session. The lower level manifest file contains session- 1548 based signed chunk URLs. CSP generates the manifest files 1549 dynamically for the session. The chunk (segment/fragment) is 1550 delivered with content access authorization using flexible URL 1551 Signing which protects the invariant portion of the URL. Segment URL 1552 (e.g. HLS) is individually signed for the invariant URL portion 1553 (Relative URL) or the entire URL (Absolute URL without Redirection) 1554 in the manifest file. Fragment URL (e.g. Smooth Streaming) is 1555 signed for the invariant portion of the template URL in the manifest 1556 file. More details are provided later in this section. The URL 1557 Signing expiration time for the chunk needs to be long enough to play 1558 the video. 1560 For HAS content, the Manifest File contains the Relative Locator, 1561 Absolute Locator without Redirection, or Absolute Locator with 1562 Redirection for specifying the chunk location. Signing the chunk URL 1563 requires CSP to know the portion of the URL that remains when the 1564 content is requested from the Delivery CDN surrogate. 1566 For Absolute URL without Redirection, the CSP knows that the chunk 1567 URL which is explicitly linked with the delivery CDN surrogate and 1568 can sign the URL based on that information. Since the entire URL is 1569 set and does not change, the surrogate can validate the URL. The CSP 1570 and the Delivery CDN are expected to have a business relationship in 1571 this case. So either symmetric key or asymmetric keys can be used 1572 for URL Signing. 1574 For Relative URL, the URL of the Manifest File provides the root 1575 location. The method of request routing affects the URL used to 1576 ultimately request the chunk from the Delivery CDN surrogate. For 1577 DNS, the original URL does not change. This allows CSP to sign the 1578 chunk URL based on the Manifest File URL and the Relative URL. For 1579 HTTP, the URL changes during redirection. In this case, CSP does not 1580 know the redirected URL that will be used to request the Manifest 1581 File. This uncertainty makes it impossible to accurately sign the 1582 chunk URLs in the Manifest File. Basically, URL Signing using this 1583 reference method, “as is” for entire URL protection, is 1584 not supported. However, instead of signing the entire URL, the CSP 1585 signs the Relative URL (i.e. invariant portion of the URL) and 1586 conveys the protected portion in the authorization parameters 1587 embedded in the chunk URL. This approach works the same way as 1588 Absolute URL without Redirection, except the HOST part and (part of) 1589 the PATH part of the URL are not signed and validated. The security 1590 level should remain the same as content access authorization ensures 1591 that the user that requested the content has the credentials. This 1592 scheme does not seem to compromise the authorization model since the 1593 resource is still protected by the authorization parameters and 1594 message digest. Perhaps, further evaluation on security would be 1595 helpful. 1597 For Absolute URL with Redirection, the method of request routing 1598 affects the URL used to ultimately request the chunk from the 1599 Delivery CDN surrogate. This case has the same conditions as the 1600 Relative URL. The difference is that the URL is for the chunk 1601 instead of the Manifest File. For DNS, the chunk URL does not change 1602 and can be signed by the CSP. For HTTP, the URL used to deliver the 1603 chunk is unknown to the CSP. In this case, CSP cannot sign the URL 1604 and this method of reference for the chunk is not supported. 1606 Effect on CDN Interfaces: 1608 o Requires the ability to exclude the variant portion of URL in the 1609 signing process (NOTE: Issue is specific to URL Signing support 1610 for HAS content and not CDNI?) 1612 Advantages/Drawbacks: 1614 + Manifest file and chunks are protected 1616 + CDN does not need to be aware of HAS content 1618 + DNS-based request routing with asymmetric keys and HTTP-based 1619 request routing for Relative URL and Absolute URL without 1620 Redirection works 1622 - CSP has to generate manifest files with session-based signed URLs 1623 and becomes involved in content access authorization for every HAS 1624 session 1626 - Manifest files are not cacheable 1628 - DNS-based request routing with symmetric key may be problematic 1629 due to need for transitive trust between CSP and Delivery CDN 1631 - HTTP-based request routing for Absolute URL with Redirection does 1632 not work because the URL used Delivery CDN surrogate is unknown to 1633 the CSP 1635 3.5.4. Option 5.3: Authorization Group ID for HAS Content 1637 Based on the Authorization Group ID metadata, CDN validates the URL 1638 Signing or validates the HTTP cookie for request of content in the 1639 group. CSP performs URL Signing for the top level manifest file. 1640 The top level manifest file contains lower level manifest file URLs 1641 or chunk URLs. The lower level manifest files and chunks are 1642 delivered with content access control with content access control 1643 using HTTP cookie that contains session state associated with 1644 authorization of the top level manifest file. The Group ID Metadata 1645 is used to associate the related content (i.e. manifest files and 1646 chunks). It also specifies content (e.g. regexp method) that needs 1647 to be validated by either URL Signing or HTTP cookie. Duration of 1648 the chunk access may be included in the URL Signing of the top level 1649 manifest file and set in the cookie. Duration may be in the Metadata 1650 instead. 1652 Effect on CDN Interfaces: 1654 o CDNI Metadata Interface - Authorization Group ID metadata 1655 identifies the content that is subject to validation of URL 1656 Signing or validation of HTTP cookie associated with the URL 1657 Signing 1659 o CDNI Logging Interface - Report the authorization method used to 1660 validate the request for content delivery 1662 Advantages/Drawbacks: 1664 + Manifest file and chunks are protected 1666 + CDN does not need to be aware of HAS content 1668 + CSP does not need to change the manifest files 1670 - Authorization Group ID metadata is required (i.e. CDNI Metadata 1671 Interface enhancement) 1673 - Using HTTP cookie requires enabling this function and the cookie 1674 security implications apply 1676 - Logic needed for handling surrogate switchover 1678 3.5.5. Option 5.4: Handle HAS Content in CDN 1680 CDN is aware of HAS content and uses URL Signing and HTTP cookie for 1681 content access authorization. URL Signing is fundamentally about 1682 authorizing access to a Content Item or its specific Content 1683 Collections (representations) for a specific user during a time 1684 period with possibly some other criteria. A chunk is an instance of 1685 the sets of chunks referenced by the Manifest File for the Content 1686 Item or its specific Content Collections. This relationship means 1687 that once the Downstream CDN has authorized the Manifest File, it can 1688 assume that the associated chunks are implicitly authorized. The new 1689 function for the CDN is to link the Manifest File with the chunks for 1690 the HTTP session. This can be accomplished by using authorization 1691 token or session based encryption. This section covers the former 1692 and next section covers the latter. 1694 After validating the URL and detecting that the requested content is 1695 a Manifest File, the delivery CDN surrogate creates a state and sets 1696 a HTTP cookie with authorization token for the HTTP session. When a 1697 request for a chunk arrives, the surrogate confirms that the HTTP 1698 cookie value contains the correct authorization token. If so, the 1699 chunk is delivered due to transitive authorization property. 1700 Duration of the chunk access may be included in the URL Signing of 1701 the top level manifest file and set in the cookie 1703 Effect on CDN Interfaces: 1705 o CDNI Metadata Interface - New metadata identifies the content that 1706 is subject to validation of URL Signing and information in the 1707 cookie for the type of HAS content 1709 o Request Routing interface - Downstream CDN should inform the 1710 Upstream CDN that it supports URL Signing for known HAS content 1711 types in the asynchronous capabilities information advertisement. 1712 This allows the CDN selection function in request routing to 1713 choose the appropriate Downstream CDN when the CDNI metadata 1714 identifies the content 1716 o CDNI Logging Interface - Report the authorization method used to 1717 validate the request for content delivery 1719 Advantages/Drawbacks: 1721 + Manifest file and chunks are protected 1723 + CSP does not need to change the manifest files 1725 - CDN needs to be aware of HAS content 1727 - Use of HTTP cookie (for HAS session state) requires enabling this 1728 function and the cookie security implications apply 1730 - Logic needed for handling surrogate switchover 1732 3.5.6. URL Signing Options Analysis 1734 List 1735 includes "top level manifest file protection", "lower level manifest 1736 file and chunk protection", "No manifest file change per HAS session 1737 by CSP", "No HTTP cookie used", "No authorization group ID metadata", 1738 "No HAS awareness", "authorization time window", "live streaming 1739 support", "surrogate never received top level manifest file", 1740 "manifest file is hosted separately from chunk server". 1742 Summary: 1744 "Do nothing about HAS content" (#1) approach requires no change to 1745 CSP or CDN but is undesirable because of the lack of protection for 1746 the content. "Flexible URL Signing for HAS content" (#2) approach 1747 requires flexible URL Signing support in CDN and depends on CSP to be 1748 involved in every HAS session. "Authorization Group ID for HAS 1749 content" (#3) approach requires new CDNI metadata to associate the 1750 URL Signing with HTTP cookie to validate a request for content in the 1751 logical group. "Handle HAS content in CDN" (#4) requires CDN to be 1752 aware of HAS content and impacts multiple CDNI Interfaces. 1754 Recommendations: 1756 Debatable between #2 vs. #3. Is the requirement for CSP to generate 1757 manifest files for every HAS session a practical approach? Is non- 1758 cacheable manifest file a major issue or minor nuisance? What are 1759 the other key factors to decide between the two choices? 1761 (FUTURE) Option #4 has some advantages that should be considered for 1762 future support (e.g. CDN that is aware of HAS content can manage the 1763 content more efficiently at a broader context. Content distribution, 1764 storage, delivery, deletion, access authorization, etc. can all 1765 benefit.) 1767 3.6. Content Purge 1769 At some point in time, a uCDN might want to remove content from a 1770 dCDN. With regular content, this process can be relatively 1771 straightforward; a uCDN will typically send the request for content 1772 removal to the dCDN including a reference to the content which it 1773 wants to remove (e.g. in the form of a URL). Due to the fact that 1774 HAS content consists of large groups of files however, things might 1775 be more complex. Section 3.1 describes a number of different 1776 scenarios for doing file management on these groups of files, while 1777 Section 3.2 list the options for performing Content Acquisition on 1778 these Content Collections. This section will present the options for 1779 requesting a Content Purge for the removal of a Content Collection 1780 from a dCDN. 1782 3.6.1. Option 6.1: No HAS awareness 1784 The most straightforward way to signal content purge requests is to 1785 just send a single purge request for every file that makes up the 1786 Content Collection. While this method is very simple and does not 1787 require HAS awareness, it obviously creates a large signalling 1788 overhead between the uCDN and dCDN. 1790 Effect on CDN Interfaces: 1792 o None 1794 Advantages/Drawbacks (apart from those listed under Option 3.3): 1796 + Does not require changes to the CDNI Interfaces or HAS awareness 1798 - Requires individual purge request for every file making up a 1799 Content Collection which creates large signalling overhead 1801 3.6.2. Option 6.2: Purge Identifiers 1803 There exists a potentially more efficient method for performing 1804 content removal of large numbers of files simultaneously. By 1805 including purge identifiers in the metadata of a particular file, it 1806 is possible to virtually group together different files making up a 1807 Content Collection. A purge identifier can take the form of a random 1808 number which is communicated as part of the CDNI Metadata Interface 1809 and which is the same for all files making up a particular Content 1810 Item. If a uCDN wants to request the dCDN to remove a Content 1811 Collection, it can send a purge request containing this purge 1812 identifier. The dCDN can then remove all files that contain the 1813 shared identifier. 1815 The advantage of this method is that it is relatively simple to use 1816 by both the dCDN and uCDN and requiring only limited additions to the 1817 CDNI Metadata Interface and CDNI Control Interface. 1819 [Editor's Note: Could the Purge Identifier introduced in this section 1820 be related to the Content Collection Identifier introduced in 1821 Section 3.4.2.2? Chould they be the same identifier?] 1823 Effect on CDN Interfaces: 1825 o CDNI Metadata Interface: Add metadata field for indicating Purge 1826 Identifier 1828 o CDNI Control Interface: Add functionality to be able to send 1829 content purge requests containing Purge Identifiers 1831 Advantages/Drawbacks: 1833 + Allows for efficient purging of content from a dCDN 1835 + Does not require HAS awareness on part of dCDN 1837 4. IANA Considerations 1839 This document makes no request of IANA. 1841 Note to RFC Editor: this section may be removed on publication as an 1842 RFC. 1844 5. Security Considerations 1846 TBD. 1848 6. Acknowledgements 1850 The authors would like to thank Kevin J. Ma for his valuable 1851 contributions to this draft. 1853 7. References 1855 7.1. Normative References 1857 [I-D.ietf-cdni-problem-statement] 1858 Niven-Jenkins, B., Le Faucheur, F., and N. Bitar, "Content 1859 Distribution Network Interconnection (CDNI) Problem 1860 Statement, draft-ietf-cdni-problem-statement-03", 1861 January 2012. 1863 [I-D.ietf-cdni-use-cases] 1864 Bertrand, G., Ed., Stephan, E., Watson, G., Burbridge, T., 1865 Eardley, P., and K. Ma, "Use Cases for Content Delivery 1866 Network Interconnection, draft-ietf-cdni-use-cases-03", 1867 January 2012. 1869 7.2. Informative References 1871 [I-D.draft-bertrand-cdni-logging] 1872 Bertrand, G., Ed. and E. Stephan, "CDNI Logging 1873 Interface". 1875 [I-D.draft-lefaucheur-cdni-logging-delivery] 1876 Le Faucheur, F., Viveganandhan, M., and K. Leung, "CDNI 1877 Logging Formats for HTTP and HTTP Adaptive Streaming 1878 Deliveries". 1880 Authors' Addresses 1882 Ray van Brandenburg 1883 TNO 1884 Brassersplein 2 1885 Delft 2612CT 1886 the Netherlands 1888 Phone: +31-88-866-7000 1889 Email: ray.vanbrandenburg@tno.nl 1891 Oskar van Deventer 1892 TNO 1893 Brassersplein 2 1894 Delft 2612CT 1895 the Netherlands 1897 Phone: +31-88-866-7000 1898 Email: oskar.vandeventer@tno.nl 1900 Francois Le Faucheur 1901 Cisco Systems 1902 Greenside, 400 Avenue de Roumanille 1903 Sophia Antipolis 06410 1904 France 1906 Phone: +33 4 97 23 26 19 1907 Email: flefauch@cisco.com 1908 Kent Leung 1909 Cisco Systems 1910 170 West Tasman Drive 1911 San Jose, CA 95134 1912 USA 1914 Phone: +1 408-526-5030 1915 Email: kleung@cisco.com