idnits 2.17.1 draft-ietf-cdni-logging-12.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 6 instances of too long lines in the document, the longest one being 11 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 4, 2014) is 3584 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 2616 (Obsoleted by RFC 7230, RFC 7231, RFC 7232, RFC 7233, RFC 7234, RFC 7235) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) == Outdated reference: A later version (-21) exists of draft-ietf-cdni-metadata-07 == Outdated reference: A later version (-17) exists of draft-ietf-httpbis-http2-13 -- Obsolete informational reference (is this intentional?): RFC 2818 (Obsoleted by RFC 9110) Summary: 3 errors (**), 0 flaws (~~), 3 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force F. Le Faucheur, Ed. 3 Internet-Draft Cisco Systems 4 Intended status: Standards Track G. Bertrand, Ed. 5 Expires: January 5, 2015 I. Oprescu, Ed. 6 Orange 7 R. Peterkofsky 8 Skytide, Inc. 9 July 4, 2014 11 CDNI Logging Interface 12 draft-ietf-cdni-logging-12 14 Abstract 16 This memo specifies the Logging interface between a downstream CDN 17 (dCDN) and an upstream CDN (uCDN) that are interconnected as per the 18 CDN Interconnection (CDNI) framework. First, it describes a 19 reference model for CDNI logging. Then, it specifies the CDNI 20 Logging File format and the actual protocol for exchange of CDNI 21 Logging Files. 23 Status of This Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at http://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on January 5, 2015. 40 Copyright Notice 42 Copyright (c) 2014 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (http://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 58 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 59 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 4 60 2. CDNI Logging Reference Model . . . . . . . . . . . . . . . . 5 61 2.1. CDNI Logging interactions . . . . . . . . . . . . . . . . 5 62 2.2. Overall Logging Chain . . . . . . . . . . . . . . . . . . 8 63 2.2.1. Logging Generation and During-Generation Aggregation 9 64 2.2.2. Logging Collection . . . . . . . . . . . . . . . . . 10 65 2.2.3. Logging Filtering . . . . . . . . . . . . . . . . . . 10 66 2.2.4. Logging Rectification and Post-Generation Aggregation 11 67 2.2.5. Log-Consuming Applications . . . . . . . . . . . . . 12 68 2.2.5.1. Maintenance/Debugging . . . . . . . . . . . . . . 12 69 2.2.5.2. Accounting . . . . . . . . . . . . . . . . . . . 12 70 2.2.5.3. Analytics and Reporting . . . . . . . . . . . . . 12 71 2.2.5.4. Security . . . . . . . . . . . . . . . . . . . . 13 72 2.2.5.5. Legal Logging Duties . . . . . . . . . . . . . . 13 73 2.2.5.6. Notions common to multiple Log Consuming 74 Applications . . . . . . . . . . . . . . . . . . 13 75 3. CDNI Logging File . . . . . . . . . . . . . . . . . . . . . . 15 76 3.1. Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 15 77 3.2. CDNI Logging File Structure . . . . . . . . . . . . . . . 16 78 3.3. CDNI Logging File Directives . . . . . . . . . . . . . . 19 79 3.4. CDNI Logging Records . . . . . . . . . . . . . . . . . . 23 80 3.4.1. HTTP Request Logging Record . . . . . . . . . . . . . 23 81 3.5. CDNI Logging File Example . . . . . . . . . . . . . . . . 32 82 3.6. Cascaded CDNI Logging Files Example . . . . . . . . . . . 34 83 4. Protocol for Exchange of CDNI Logging File After Full 84 Collection . . . . . . . . . . . . . . . . . . . . . . . . . 37 85 4.1. CDNI Logging Feed . . . . . . . . . . . . . . . . . . . . 38 86 4.1.1. Atom Formatting . . . . . . . . . . . . . . . . . . . 38 87 4.1.2. Updates to Log Files and the Feed . . . . . . . . . . 38 88 4.1.3. Redundant Feeds . . . . . . . . . . . . . . . . . . . 39 89 4.1.4. Example CDNI Logging Feed . . . . . . . . . . . . . . 39 90 4.2. CDNI Logging File Pull . . . . . . . . . . . . . . . . . 41 91 5. Protocol for Exchange of CDNI Logging File During Collection 42 92 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 43 93 6.1. CDNI Logging Directive Names Registry . . . . . . . . . . 43 94 6.2. CDNI Logging Record-Types Registry . . . . . . . . . . . 43 95 6.3. CDNI Logging Field Names Registry . . . . . . . . . . . . 44 96 6.4. CDNI Logging MIME Media Type . . . . . . . . . . . . . . 46 98 7. Security Considerations . . . . . . . . . . . . . . . . . . . 46 99 7.1. Authentication, Confidentiality, Integrity Protection . . 46 100 7.2. Denial of Service . . . . . . . . . . . . . . . . . . . . 47 101 7.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 47 102 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 48 103 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 48 104 9.1. Normative References . . . . . . . . . . . . . . . . . . 48 105 9.2. Informative References . . . . . . . . . . . . . . . . . 49 106 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 50 108 1. Introduction 110 This memo specifies the CDNI Logging interface between a downstream 111 CDN (dCDN) and an upstream CDN (uCDN). First, it describes a 112 reference model for CDNI logging. Then, it specifies the CDNI 113 Logging File format and the actual protocol for exchange of CDNI 114 Logging Files. 116 The reader should be familiar with the following documents: 118 o CDNI problem statement [RFC6707] and framework 119 [I-D.ietf-cdni-framework] identify a Logging interface, 121 o Section 8 of [I-D.ietf-cdni-requirements] specifies a set of 122 requirements for Logging, 124 o [RFC6770] outlines real world use-cases for interconnecting CDNs. 125 These use cases require the exchange of Logging information 126 between the dCDN and the uCDN. 128 As stated in [RFC6707], "the CDNI Logging interface enables details 129 of logs or events to be exchanged between interconnected CDNs". 131 The present document describes: 133 o The CDNI Logging reference model (Section 2), 135 o The CDNI Logging File format (Section 3), 137 o The CDNI Logging File Exchange protocol (Section 4). 139 1.1. Terminology 141 In this document, the first letter of each CDNI-specific term is 142 capitalized. We adopt the terminology described in [RFC6707] and 143 [I-D.ietf-cdni-framework], and extend it with the additional terms 144 defined below. 146 Intra-CDN Logging information: logging information generated and 147 collected within a CDN. The format of the Intra-CDN Logging 148 information may be different to the format of the CDNI Logging 149 information. 151 CDNI Logging information: logging information exchanged across CDNs 152 using the CDNI Logging Interface. 154 Logging information: logging information generated and collected 155 within a CDN or obtained from another CDN using the CDNI Logging 156 Interface. 158 CDNI Logging Field: an atomic element of information that can be 159 included in a CDNI Logging Record. The time an event/task started, 160 the IP address of an End User to whom content was delivered, and the 161 URI of the content delivered are examples of CDNI Logging Fields. 163 CDNI Logging Record: an information record providing information 164 about a specific event. This comprises a collection of CDNI Logging 165 Fields. 167 CDNI Logging File: a file containing CDNI Logging Records, as well as 168 additional information facilitating the processing of the CDNI 169 Logging Records. 171 CDN Reporting: the process of providing the relevant information that 172 will be used to create a formatted content delivery report provided 173 to the CSP in deferred time. Such information typically includes 174 aggregated data that can cover a large period of time (e.g., from 175 hours to several months). Uses of Reporting include the collection 176 of charging data related to CDN services and the computation of Key 177 Performance Indicators (KPIs). 179 CDN Monitoring: the process of providing or displaying content 180 delivery information in a timely fashion with respect to the 181 corresponding deliveries. Monitoring typically includes visibility 182 of the deliveries in progress for service operation purposes. It 183 presents a view of the global health of the services as well as 184 information on usage and performance, for network services 185 supervision and operation management. In particular, monitoring data 186 can be used to generate alarms. 188 1.2. Requirements Language 190 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 191 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 192 document are to be interpreted as described in RFC 2119 [RFC2119]. 194 2. CDNI Logging Reference Model 196 2.1. CDNI Logging interactions 198 The CDNI logging reference model between a given uCDN and a given 199 dCDN involves the following interactions: 201 o customization by the uCDN of the CDNI Logging information to be 202 provided by the dCDN to the uCDN (e.g., control of which CDNI 203 Logging Fields are to be communicated to the uCDN for a given task 204 performed by the dCDN or control of which types of events are to 205 be logged). The dCDN takes into account this CDNI Logging 206 customization information to determine what Logging information to 207 provide to the uCDN, but it may, or may not, take into account 208 this CDNI Logging customization information to influence what CDN 209 logging information is to be generated and collected within the 210 dCDN (e.g., even if the uCDN requests a restricted subset of the 211 logging information, the dCDN may elect to generate a broader set 212 of logging information). The mechanism to support the 213 customization by the uCDN of CDNI Logging information is outside 214 the scope of this document and left for further study. Until such 215 a mechanism is available, the uCDN and dCDN are expected to agree 216 off-line on what exact set of CDNI Logging information is to be 217 provided by the dCDN to the uCDN, and to rely on management plane 218 actions to configure the CDNI Logging functions in the dCDN to 219 generate this information set and in the uCDN to expect this 220 information set. 222 o generation and collection by the dCDN of the intra-CDN Logging 223 information related to the completion of any task performed by the 224 dCDN on behalf of the uCDN (e.g., delivery of the content to an 225 End User) or related to events happening in the dCDN that are 226 relevant to the uCDN (e.g., failures or unavailability in dCDN). 227 This takes place within the dCDN and does not directly involve 228 CDNI interfaces. 230 o communication by the dCDN to the uCDN of the Logging information 231 collected by the dCDN relevant to the uCDN. This is supported by 232 the CDNI Logging interface and in the scope of the present 233 document. For example, the uCDN may use this Logging information 234 to charge the CSP, to perform analytics and monitoring for 235 operational reasons, to provide analytics and monitoring views on 236 its content delivery to the CSP or to perform trouble-shooting. 237 This document exclusively specifies non-real-time exchange of 238 Logging information. Closer to real-time exchange of Logging 239 information (say sub-minute or sub-second) is outside the scope of 240 the present document and left for further study. This document 241 exclusively specifies exchange of Logging information related to 242 content delivery. Exchange of Logging information related to 243 operational events (e.g., dCDN request routing function 244 unavailable, content acquisition failure by dCDN) for audit or 245 operational reactive adjustments by uCDN is outside the scope of 246 the present document and left for further study. 248 o customization by the dCDN of the CDNI Logging information to be 249 provided by the uCDN on behalf of the dCDN. The mechanism to 250 support the customization by the dCDN of CDNI Logging information 251 is outside the scope of this document and left for further study. 253 o generation and collection by the uCDN of Intra-CDN Logging 254 information related to the completion of any task performed by the 255 uCDN on behalf of the dCDN (e.g., serving of content by uCDN to 256 dCDN for acquisition purposes by dCDN) or related to events 257 happening in the uCDN that are relevant to the dCDN. This takes 258 place within the uCDN and does not directly involve CDNI 259 interfaces. 261 o communication by the uCDN to the dCDN of the Logging information 262 collected by the uCDN relevant to the dCDN. For example, the dCDN 263 might potentially benefit from this information for security 264 auditing or content acquisition troubleshooting. This is outside 265 the scope of this document and left for further study. 267 Figure 1 provides an example of CDNI Logging interactions (focusing 268 only on the interactions that are in the scope of this document) in a 269 particular scenario where four CDNs are involved in the delivery of 270 content from a given CSP: the uCDN has a CDNI interconnection with 271 dCDN-1 and dCDN-2. In turn, dCDN2 has a CDNI interconnection with 272 dCDN3. In this example, uCDN, dCDN-1, dCDN-2 and dCDN-3 all 273 participate in the delivery of content for the CSP. In this example, 274 the CDNI Logging interface enables the uCDN to obtain Logging 275 information from all the dCDNs involved in the delivery. In the 276 example, the uCDN uses the Logging information: 278 o to analyze the performance of the delivery performed by the dCDNs 279 and to adjust its operations after the fact (e.g., request 280 routing) as appropriate, 282 o to provide (non-real-time) reporting and monitoring information to 283 the CSP. 285 For instance, the uCDN merges Logging information, extracts relevant 286 KPIs, and presents a formatted report to the CSP, in addition to a 287 bill for the content delivered by uCDN itself or by its dCDNs on the 288 CSP's behalf. The uCDN may also provide Logging information as raw 289 log files to the CSP, so that the CSP can use its own logging 290 analysis tools. 292 +-----+ 293 | CSP | 294 +-----+ 295 ^ Reporting and monitoring data 296 * Billing 297 ,--*--. 298 Logging ,-' `-. 299 Data =>( uCDN )<= Logging 300 // `-. _,-' \\ Data 301 || `-'-'-' || 302 ,-----. ,-----. 303 ,-' `-. ,-' `-. 304 ( dCDN-1 ) ( dCDN-2 )<== Logging 305 `-. ,-' `-. _,-' \\ Data 306 `--'--' `--'-' || 307 ,-----. 308 ,' `-. 309 ( dCDN-3 ) 310 `. ,-' 311 `--'--' 313 ===> CDNI Logging Interface 314 ***> outside the scope of CDNI 316 Figure 1: Interactions in CDNI Logging Reference Model 318 A dCDN (e.g., dCDN-2) integrates the relevant Logging information 319 obtained from its dCDNs (i.e., dCDN-3) in the Logging information 320 that it provides to the uCDN, so that the uCDN ultimately obtains all 321 Logging information relevant to a CSP for which it acts as the 322 authoritative CDN. Such aggregation is further discussed in 323 Section 3.6. 325 Note that the format of Logging information that a CDN provides over 326 the CDNI interface might be different from the one that the CDN uses 327 internally. In this case, the CDN needs to reformat the Logging 328 information before it provides this information to the other CDN over 329 the CDNI Logging interface. Similarly, a CDN might reformat the 330 Logging information that it receives over the CDNI Logging interface 331 before injecting it into its log-consuming applications or before 332 providing some of this Logging information to the CSP. Such 333 reformatting operations introduce latency in the logging distribution 334 chain and introduce a processing burden. Therefore, there are 335 benefits in specifying CDNI Logging formats that are suitable for use 336 inside CDNs and also are close to the intra-CDN Logging formats 337 commonly used in CDNs today. 339 2.2. Overall Logging Chain 341 This section discusses the overall logging chain within and across 342 CDNs to clarify how CDN Logging information is expected to fit in 343 this overall chain. Figure 2 illustrates the overall logging chain 344 within the dCDN, across CDNs using the CDNI Logging interface and 345 within the uCDN. Note that the logging chain illustrated in the 346 Figure is obviously only an example and varies depending on the 347 specific environments. For example, there may be more or fewer 348 instantiations of each entity (e.g., there may be 4 Log consuming 349 applications in a given CDN). As another example, there may be one 350 instance of Rectification process per Log Consuming Application 351 instead of a shared one. 353 Log Consuming Log Consuming 354 App App 355 ^ ^ 356 | | 357 Rectification---------- 358 ^ 359 | 360 Filtering 361 ^ 362 | 363 Collection 364 ^ ^ 365 | | 366 | Generation 367 | 368 | uCDN 369 CDNI Logging ------------------------------------------------------- 370 exchange dCDN 371 ^ 372 | Log Consuming Log Consuming 373 | App App 374 | ^ ^ 375 | | | 376 Rectification Rectification--------- 377 ^ ^ 378 | | 379 Filtering 380 ^ 381 | 382 Collection 383 ^ ^ 384 | | 385 Generation Generation 387 Figure 2: CDNI Logging in the overall Logging Chain 389 The following subsections describe each of the processes potentially 390 involved in the logging chain of Figure 2. 392 2.2.1. Logging Generation and During-Generation Aggregation 394 CDNs typically generate Logging information for all significant task 395 completions, events, and failures. Logging information is typically 396 generated by many devices in the CDN including the surrogates, the 397 request routing system, and the control system. 399 The amount of Logging information generated can be huge. Therefore, 400 during contract negotiations, interconnected CDNs often agree on a 401 retention duration for Logging information, and/or potentially on a 402 maximum volume of Logging information that the dCDN ought to keep. 403 If this volume is exceeded, the dCDN is expected to alert the uCDN 404 but may not keep more Logging information for the considered time 405 period. In addition, CDNs may aggregate Logging information and 406 transmit only summaries for some categories of operations instead of 407 the full Logging information. Note that such aggregation leads to an 408 information loss, which may be problematic for some usages of the 409 Logging information (e.g., debugging). 411 [RFC6983] discusses logging for HTTP Adaptive Streaming (HAS). In 412 accordance with the recommendations articulated there, it is expected 413 that a surrogate will generate separate Logging information for 414 delivery of each chunk of HAS content. This ensures that separate 415 Logging information can then be provided to interconnected CDNs over 416 the CDNI Logging interface. Still in line with the recommendations 417 of [RFC6983], the Logging information for per-chunck delivery may 418 include some information (a Content Collection IDentifier and a 419 Session IDentifier) intended to facilitate subsequent post-generation 420 aggregation of per-chunk logs into per-session logs. Note that a CDN 421 may also elect to generate aggregate per-session logs when performing 422 HAS delivery, but this needs to be in addition to, and not instead 423 of, the per-chunk delivery logs. We note that aggregate per-session 424 logs for HAS delivery are for further study and outside the scope of 425 this document. 427 2.2.2. Logging Collection 429 This is the process that continuously collects Logging information 430 generated by the log-generating entities within a CDN. 432 In a CDNI environment, in addition to collecting Logging information 433 from log-generating entities within the local CDN, the Collection 434 process also collects Logging information provided by another CDN, or 435 other CDNs, through the CDNI Logging interface. This is illustrated 436 in Figure 2 where we see that the Collection process of the uCDN 437 collects Logging information from log-generating entities within the 438 uCDN as well as Logging information coming from the dCDNs through the 439 CDNI Logging interface. 441 2.2.3. Logging Filtering 443 A CDN may be required to only present different subsets of the whole 444 Logging information collected to various log-consuming applications. 445 This is achieved by the Filtering process. 447 In particular, the Filtering process can also filter the right subset 448 of Logging information that needs to be provided to a given 449 interconnected CDN. For example, the filtering process in the dCDN 450 can be used to ensure that only the Logging information related to 451 tasks performed on behalf of a given uCDN are made available to that 452 uCDN (thereby filtering out all the Logging information related to 453 deliveries by the dCDN of content for its own CSPs). Similarly, the 454 Filtering process may filter or partially mask some fields, for 455 example, to protect End Users' privacy when communicating CDNI 456 Logging information to another CDN. Filtering of Logging information 457 prior to communication of this information to other CDNs via the CDNI 458 Logging interface requires that the downstream CDN can recognize the 459 subset of Logging information that relate to each interconnected CDN. 461 The CDN will also filter some internal scope information such as 462 information related to its internal alarms (security, failures, load, 463 etc). 465 In some use cases described in [RFC6770], the interconnected CDNs do 466 not want to disclose details on their internal topology. The 467 filtering process can then also filter confidential data on the 468 dCDNs' topology (number of servers, location, etc.). In particular, 469 information about the requests served by each Surrogate may be 470 confidential. Therefore, the Logging information needs to be 471 protected so that data such as Surrogates' hostnames are not 472 disclosed to the uCDN. In the "Inter-Affiliates Interconnection" use 473 case, this information may be disclosed to the uCDN because both the 474 dCDN and the uCDN are operated by entities of the same group. 476 2.2.4. Logging Rectification and Post-Generation Aggregation 478 If Logging information is generated periodically, it is important 479 that the sessions that start in one Logging period and end in another 480 are correctly reported. If they are reported in the starting period, 481 then the Logging information of this period will be available only 482 after the end of the session, which delays the Logging information 483 generation. A simple approach is to provide the complete Logging 484 Record for a session in the Logging Period of the session end. 486 A Logging rectification/update mechanism could be useful to reach a 487 good trade-off between the Logging information generation delay and 488 the Logging information accuracy. 490 In the presence of HAS, some log-consuming applications can benefit 491 from aggregate per-session logs. For example, for analytics, per- 492 session logs allow display of session-related trends which are much 493 more meaningful for some types of analysis than chunk-related trends. 494 In the case where aggregate logs have been generated directly by the 495 log-generating entities, those can be used by the applications. In 496 the case where aggregate logs have not been generated, the 497 Rectification process can be extended with a Post-Generation 498 Aggregation process that generates per-session logs from the per- 499 chunk logs, possibly leveraging the information included in the per- 500 chunk logs for that purpose (Content Collection IDentifier and a 501 Session IDentifier). However, in accordance with [RFC6983], this 502 document does not define exchange of such aggregate logs on the CDNI 503 Logging interface. We note that this is for further study and 504 outside the scope of this document. 506 2.2.5. Log-Consuming Applications 508 2.2.5.1. Maintenance/Debugging 510 Logging information is useful to permit the detection (and limit the 511 risk) of content delivery failures. In particular, Logging 512 information facilitates the detection of configuration issues. 514 To detect faults, Logging information needs to report success and 515 failure of CDN delivery operations. The uCDN can summarize such 516 information into KPIs. For instance, Logging information needs to 517 allow the computation of the number of times, during a given time 518 period, that content delivery related to a specific service succeeds/ 519 fails. 521 Logging information enables the CDN providers to identify and 522 troubleshoot performance degradations. In particular, Logging 523 information enables tracking of traffic data (e.g., the amount of 524 traffic that has been forwarded by a dCDN on behalf of an uCDN over a 525 given period of time), which is particularly useful for CDN and 526 network planning operations. 528 2.2.5.2. Accounting 530 Logging information is essential for accounting, to permit inter-CDN 531 billing and CSP billing by uCDNs. For instance, Logging information 532 provided by dCDNs enables the uCDN to compute the total amount of 533 traffic delivered by every dCDN for a particular Content Provider, as 534 well as, the associated bandwidth usage (e.g., peak, 95th 535 percentile), and the maximum number of simultaneous sessions over a 536 given period of time. 538 2.2.5.3. Analytics and Reporting 540 The goal of analytics is to gather any relevant information to track 541 audience, analyze user behavior, and monitor the performance and 542 quality of content delivery. For instance, Logging information 543 enables the CDN providers to report on content consumption (e.g., 544 delivered sessions per content) in a specific geographic area. 546 The goal of reporting is to gather any relevant information to 547 monitor the performance and quality of content delivery and allow 548 detection of delivery issues. For instance, reporting could track 549 the average delivery throughput experienced by End Users in a given 550 region for a specific CSP or content set over a period of time. 552 2.2.5.4. Security 554 The goal of security is to prevent and monitor unauthorized access, 555 misuse, modification, and denial of access of a service. A set of 556 information is logged for security purposes. In particular, a record 557 of access to content is usually collected to permit the CSP to detect 558 infringements of content delivery policies and other abnormal End 559 User behaviors. 561 2.2.5.5. Legal Logging Duties 563 Depending on the country considered, the CDNs may have to retain 564 specific Logging information during a legal retention period, to 565 comply with judicial requisitions. 567 2.2.5.6. Notions common to multiple Log Consuming Applications 569 2.2.5.6.1. Logging Information Views 571 Within a given log-consuming application, different views may be 572 provided to different users depending on privacy, business, and 573 scalability constraints. 575 For example, an analytics tool run by the uCDN can provide one view 576 to an uCDN operator that exploits all the Logging information 577 available to the uCDN, while the tool may provide a different view to 578 each CSP exploiting only the Logging information related to the 579 content of the given CSP. 581 As another example, maintenance and debugging tools may provide 582 different views to different CDN operators, based on their 583 operational role. 585 2.2.5.6.2. Key Performance Indicators (KPIs) 587 This section presents, for explanatory purposes, a non-exhaustive 588 list of Key Performance Indicators (KPIs) that can be extracted/ 589 produced from logs. 591 Multiple log-consuming applications, such as analytics, monitoring, 592 and maintenance applications, often compute and track such KPIs. 594 In a CDNI environment, depending on the situation, these KPIs may be 595 computed by the uCDN or by the dCDN. But it is usually the uCDN that 596 computes KPIs, because the uCDN and dCDN may have different 597 definitions of the KPIs and the computation of some KPIs requires a 598 vision of all the deliveries performed by the uCDN and all its dCDNs. 600 Here is a list of important examples of KPIs: 602 o Number of delivery requests received from End Users in a given 603 region for each piece of content, during a given period of time 604 (e.g., hour/day/week/month) 606 o Percentage of delivery successes/failures among the aforementioned 607 requests 609 o Number of failures listed by failure type (e.g., HTTP error code) 610 for requests received from End Users in a given region and for 611 each piece of content, during a given period of time (e.g., 612 hour/day/week/month) 614 o Number and cause of premature delivery termination for End Users 615 in a given region and for each piece of content, during a given 616 period of time (e.g., hour/day/week/month) 618 o Maximum and mean number of simultaneous sessions established by 619 End Users in a given region, for a given Content Provider, and 620 during a given period of time (e.g., hour/day/week/month) 622 o Volume of traffic delivered for sessions established by End Users 623 in a given region, for a given Content Provider, and during a 624 given period of time (e.g., hour/day/week/month) 626 o Maximum, mean, and minimum delivery throughput for sessions 627 established by End Users in a given region, for a given Content 628 Provider, and during a given period of time (e.g., hour/day/week/ 629 month) 631 o Cache-hit and byte-hit ratios for requests received from End Users 632 in a given region for each piece of content, during a given period 633 of time (e.g., hour/day/week/month) 635 o Top 10 most popularly requested contents (during a given day/week/ 636 month) 638 o Terminal type (mobile, PC, STB, if this information can be 639 acquired from the browser type header, for example). 641 Additional KPIs can be computed from other sources of information 642 than the Logging information, for instance, data collected by a 643 content portal or by specific client-side application programming 644 interfaces. Such KPIs are out of scope for the present document. 646 The KPIs used depend strongly on the considered log-consuming 647 application -- the CDN operator may be interested in different 648 metrics than the CSP is. In particular, CDN operators are often 649 interested in delivery and acquisition performance KPIs, information 650 related to Surrogates' performance, caching information to evaluate 651 the cache-hit ratio, information about the delivered file size to 652 compute the volume of content delivered during peak hour, etc. 654 Some of the KPIs, for instance those providing an instantaneous 655 vision of the active sessions for a given CSP's content, are useful 656 essentially if they are provided in a timely manner. By contrast, 657 some other KPIs, such as those averaged on a long period of time, can 658 be provided in non-real-time. 660 3. CDNI Logging File 662 3.1. Rules 664 This specification uses the Augmented Backus-Naur Form (ABNF) 665 notation and core rules of [RFC5234]. In particular, the present 666 document uses the following rules from [RFC5234]: 668 CR = %x0D ; carriage return 670 ALPHA = %x41-5A / %x61-7A ; A-Z / a-z 672 DIGIT = %x30-39 ; 0-9 674 DQUOTE = %x22 ; " (Double Quote) 676 CRLF = CR LF ; Internet standard newline 678 HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F" 680 HTAB = %x09 ; horizontal tab 682 LF = %x0A ; linefeed 684 OCTET = %x00-FF ; 8 bits of data 686 The present document also uses the following rules from [RFC3986]: 688 host = as specified in section 3.2.2 of [RFC3986]. 690 IPv4address = as specified in section 3.2.2 of [RFC3986]. 692 IPv6address = as specified in section 3.2.2 of [RFC3986]. 694 The present document also defines the following additional rules: 696 ADDRESS = IPv4address / IPv6address 698 ALPHANUM = ALPHA / DIGIT 700 DATE = 4DIGIT "-" 2DIGIT "-" 2DIGIT 702 Dates are recorded in the format YYYY-MM-DD where YYYY, MM and 703 DD stand for the numeric year, month and day respectively. All 704 dates are specified in Universal Time Coordinated (UTC). 706 DEC = 1*DIGIT ["." *DIGIT] 708 NAMEFORMAT = ALPHANUM *(ALPHANUM / "_" / "-") 710 QSTRING = DQUOTE *NDQUOTE DQUOTE ; where 712 NDQUOTE = / 2DQUOTE ; whereby a 713 DQUOTE is conveyed inside a QSTRING unambiguously by repeating 714 it. 716 NHTABSTRING = *NHTAB ; where 718 NHTAB = 720 TIME = 2DIGIT ":" 2DIGIT ":" 2DIGIT ["." *DIGIT] 722 Times are recorded in the form HH:MM:SS or HH:MM:SS.S where HH 723 is the hour in 24 hour format, MM is minutes and SS is seconds. 724 All times are specified in Universal Time Coordinated (UTC). 726 3.2. CDNI Logging File Structure 728 As defined in Section 1.1: a CDNI Logging Field is as an atomic 729 logging information element, a CDNI Logging Record is a collection of 730 CDNI Logging Fields containing all logging information corresponding 731 to a single logging event, and a CDNI Logging File contains a 732 collection of CDNI Logging Records. This structure is illustrated in 733 Figure 3. The use of a file structure for transfer of CDNI Logging 734 information is selected since this is the most common practise today 735 for exchange of logging information within and across CDNs. 737 +----------------------------------------------------------+ 738 |CDNI Logging File | 739 | | 740 | #Directive 1 | 741 | #Directive 2 | 742 | ... | 743 | #Directive P | 744 | | 745 | +------------------------------------------------------+ | 746 | |CDNI Logging Record 1 | | 747 | | +-------------+ +-------------+ +-------------+ | | 748 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 749 | | | Field 1 | | Field 2 | | Field N | | | 750 | | +-------------+ +-------------+ +-------------+ | | 751 | +------------------------------------------------------+ | 752 | | 753 | +------------------------------------------------------+ | 754 | |CDNI Logging Record 2 | | 755 | | +-------------+ +-------------+ +-------------+ | | 756 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 757 | | | Field 1 | | Field 2 | | Field N | | | 758 | | +-------------+ +-------------+ +-------------+ | | 759 | +------------------------------------------------------+ | 760 | | 761 | ... | 762 | | 763 | #Directive P+1 | 764 | | 765 | ... | 766 | | 767 | +------------------------------------------------------+ | 768 | |CDNI Logging Record M | | 769 | | +-------------+ +-------------+ +-------------+ | | 770 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 771 | | | Field 1 | | Field 2 | | Field N | | | 772 | | +-------------+ +-------------+ +-------------+ | | 773 | +------------------------------------------------------+ | 774 | | 775 | | 776 | #Directive P+Q | 777 +----------------------------------------------------------+ 779 Figure 3: Structure of Logging Files 781 The CDNI Logging File format is inspired from the W3C Extended Log 782 File Format [ELF]. However, it is fully specified by the present 783 document. Where the present document differs from the W3C Extended 784 Log File Format, an implementation of the CDNI Logging interface MUST 785 comply with the present document. 787 Using a format that resembles the W3C Extended Log File Format is 788 intended to keep CDNI logging format close to the intra-CDN Logging 789 information format commonly used in CDNs today, thereby minimizing 790 systematic translation at CDN/CDNI boundary. 792 A CDNI Logging File MUST contain a sequence of lines containing US- 793 ASCII characters [CHAR_SET] terminated by CRLF. 795 Each line of a CDNI Logging File MUST contain either a directive or a 796 CDNI Logging Record. 798 Directives record information about the CDNI Logging process itself. 799 Lines containing directives MUST begin with the "#" character. 800 Directives are specified in Section 3.3. 802 Logging Records provide actual details of the logged event. Logging 803 Records are specified in Section 3.4. 805 The CDNI File structure is defined by the following rules: 807 DIRLINE = "#" directive CRLF 809 DIRGROUP = 1*DIRLINE 811 RECLINE = CRLF 813 RECGROUP = *RECLINE 815 = 1* 817 3.3. CDNI Logging File Directives 819 The CDNI Logging File directives are defined by the following rules: 821 directive = DIRNAME ":" HTAB DIRVAL 823 DIRNAME = 826 DIRVAL = 830 An implementation of the CDNI Logging interface MUST support all of 831 the following directives, listed below by their directive name: 833 o Version: 835 * format: "CDNI" "/" 1*DIGIT "." 1*DIGIT 837 * directive value: indicates the version of the CDNI Logging File 838 format. The value MUST be "CDNI/1.0" for the version specified 839 in the present document. 841 * occurrence: there MUST be one and only one instance of this 842 directive per CDNI Logging File. It MUST be the first line of 843 the CDNI Logging File. 845 o UUID: 847 * format: NHTABSTRING 849 * directive value: this a Universally Unique IDentifier (UUID) 850 from the UUID Uniform Resource Name (URN) namespace specified 851 in [RFC4122]) for the CDNI Logging File. 853 * occurrence: there MUST be one and only one instance of this 854 directive per CDNI Logging File. 856 o Claimed-Origin: 858 * format: host 860 * directive value: this contains the claimed identification of 861 the entity transmitting the CDNI Logging File (e.g., the host 862 in a dCDN supporting the CDNI Logging interface) or the entity 863 responsible for transmitting the CDNI Logging File (e.g., the 864 dCDN). 866 * occurrence: there MUST be zero or exactly one instance of this 867 directive per CDNI Logging File. This directive MAY be 868 included by the dCDN. It MUST NOT be included or modified by 869 the uCDN. 871 o Verified-Origin: 873 * format: host 875 * directive value: this contains the identification, as 876 established by the entity receiving the CDNI Logging File, of 877 the entity transmitting the CDNI Logging File (e.g., the host 878 in a dCDN supporting the CDNI Logging interface) or the entity 879 responsible for transmitting the CDNI Logging File (e.g., the 880 dCDN). 882 * occurrence: there MUST be zero or exactly one instance of this 883 directive per CDNI Logging File. This directive MAY be added 884 by the uCDN (e.g., before storing the CDNI Logging File). It 885 MUST NOT be included by the dCDN. The mechanisms used by the 886 uCDN to establish and validate the entity responsible for the 887 CDNI Logging File is outside the scope of the present document. 888 We observe that, in particular, this may be achieved through 889 authentication mechanisms that are part of the transport layer 890 of the CDNI Logging File pull mechanism (Section 4.2). 892 o Record-Type: 894 * format: NAMEFORMAT 896 * directive value: indicates the type of the CDNI Logging Records 897 that follow this directive, until another Record-Type directive 898 (or the end of the CDNI Logging File). This can be any CDNI 899 Logging Record type registered in the CDNI Logging Record-types 900 registry (Section 6.2). For example this may be 901 "cdni_http_request_v1" as specified in Section 3.4.1. 903 * occurrence: there MUST be at least one instance of this 904 directive per CDNI Logging File. The first instance of this 905 directive MUST precede a Fields directive and MUST precede all 906 CDNI Logging Records. 908 o Fields: 910 * format: FIENAME * ; where FIENAME can take any 911 CDNI Logging field name registered in the CDNI Logging Field 912 Names registry (Section 6.3). 914 * directive value: this lists the names of all the fields for 915 which a value is to appear in the CDNI Logging Records that 916 follow the instance of this directive (until another instance 917 of this directive). The names of the fields, as well as their 918 occurrences, MUST comply with the corresponding rules specified 919 in the document referenced in the CDNI Logging Record-types 920 registry (Section 6.2) for the corresponding CDNI Logging 921 Record-Type. 923 * occurrence: there MUST be at least one instance of this 924 directive per Record-Type directive. The first instance of 925 this directive for a given Record-Type MUST appear before any 926 CDNI Logging Record for this Record-Type. One situation where 927 more than one instance of the Fields directive can appear 928 within a given CDNI Logging File, is when there is a change, in 929 the middle of a fairly large logging period, in the agreement 930 between the uCDN and the dCDN about the set of Fields that are 931 to be exchanged. The multiple occurences allow records with 932 the old set of fields and records with the new set of fields to 933 be carried inside the same Logging File. 935 o Integrity-Hash: 937 * format: 32HEXDIG 939 * directive value: This directive permits the detection of a 940 corrupted CDNI Logging File. This can be useful, for instance, 941 if a problem occurs on the filesystem of the dCDN Logging 942 system and leads to a truncation of a logging file. The valid 943 Integrity-Hash value is included in this directive by the 944 entity that transmits the CDNI Logging File. It is computed by 945 applying the MD5 ([RFC1321]) cryptographic hash function on the 946 CDNI Logging File, including all the directives and logging 947 records, up to the Integrity-Hash directive itself, excluding 948 the Integrity-Hash directive itself. The Integrity-Hash value 949 is represented as a US-ASCII encoded hexadecimal number, 32 950 digits long (representing a 128 bit hash value). The entity 951 receiving the CDNI Logging File also computes in a similar way 952 the MD5 hash on the received CDNI Logging File and compares 953 this hash to the value of the Integrity-Hash directive. If the 954 two values are equal, then the received CDNI Logging File MUST 955 be considered non-corrupted. If the two values are different, 956 the received CDNI Logging File MUST be considered corrupted. 957 The behavior of the entity that received a corrupted CDNI 958 Logging File is outside the scope of this specification; we 959 note that the entity MAY attempt to pull again the same CDNI 960 Logging File from the transmitting entity. If the entity 961 receiving a non-corrupted CDNI Logging File adds a Verified- 962 Origin directive, it MUST then recompute and update the 963 Integrity-Hash directive so it also protects the added 964 Verified-Origin directive. 966 * occurrence: there MUST be zero or exactly one instance of this 967 directive. There SHOULD be exactly one instance of this 968 directive. One situation where that directive could be omitted 969 is where integrity protection is already provided via another 970 mechanism (for example if an integrity hash is associated to 971 the CDNI Logging File out of band through the CDNI Logging Feed 972 ( Section 4.1) leveraging ATOM extensions such as those 973 proposed in [I-D.snell-atompub-link-extensions]. When present, 974 the Integrity-Hash field MUST be the last line of the CDNI 975 Logging File. 977 3.4. CDNI Logging Records 979 A CDNI Logging Record consists of a sequence of CDNI Logging Fields 980 relating to that single CDNI Logging Record. 982 CDNI Logging Fields MUST be separated by the "horizontal tabulation 983 (HTAB)" character. 985 To facilitate readability, a prefix scheme is used for CDNI Logging 986 field names in a similar way to the one used in W3C Extended Log File 987 Format [ELF]. The semantics of the prefix in the present document 988 is: 990 o c: refers to the User Agent that issues the request (corresponds 991 to the "client" of W3C Extended Log Format) 993 o d: refers to the dCDN (relative to a given CDN acting as a uCDN) 995 o s: refers to the dCDN Surrogate that serves the request 996 (corresponds to the "server" of W3C Extended Log Format) 998 o u: refers to the uCDN (relative to a given CDN acting as a dCDN) 1000 o cs: refers to communication from the User Agent towards the dCDN 1001 Surrogate 1003 o sc: refers to communication from the dCDN Surrogate towards the 1004 User Agent 1006 An implementation of the CDNI Logging interface as per the present 1007 specification MUST support the CDNI HTTP Request Logging Record as 1008 specified in Section 3.4.1. 1010 A CDNI Logging Record is defined by the following rules: 1012 FIEVAL = 1014 = FIEVAL * ; where FIEVAL 1015 contains the CDNI Logging field value corresponding to the CDNI 1016 Logging field names (FIENAME) listed is the last Fields directive 1017 preceding the present CDNI Logging Record. 1019 3.4.1. HTTP Request Logging Record 1021 This section defines the CDNI Logging Record of Record-Type 1022 "cdni_http_request_v1". It is applicable to content delivery 1023 performed by the dCDN using HTTP/1.0([RFC1945]), HTTP/1.1([RFC2616]) 1024 or HTTPS ([RFC2818]). We observe that, in the case of HTTPS 1025 delivery, there may be value in logging additional information 1026 specific to the operation of HTTP over TLS and we note that this is 1027 outside the scope of the present document and may be addressed in a 1028 future document defining another CDNI Logging Record or another 1029 version of the HTTP Request Logging Record. 1031 The "cdni_http_request_v1" Record-Type is also expected to be 1032 applicable to HTTP/2.0 [I-D.ietf-httpbis-http2] (which is still under 1033 development at the time of writing the present document) since a 1034 fundamental design tenet of HTTP/2.0 is to preserve the HTTP/1.1 1035 semantics. We observe that, in the case of HTTP/2.0 delivery, there 1036 may be value in logging additional information specific to the 1037 additional functionality of HTTP/2.0 (e.g. information related to 1038 connection identification, to stream identification, to stream 1039 priority and to flow control). We note that such additional 1040 information is outside the scope of the present document and may be 1041 addressed in a future document defining another CDNI Logging Record 1042 or another version of the HTTP Request Logging Record. 1044 The "cdni_http_request_v1" Record-Type contains the following CDNI 1045 Logging Fields, listed by their field name: 1047 o date: 1049 * format: DATE 1051 * field value: the date at which the processing of request 1052 completed on the Surrogate. 1054 * occurrence: there MUST be one and only one instance of this 1055 field. 1057 o time: 1059 * format: TIME 1061 * field value: the time at which the processing of request 1062 completed on the Surrogate. 1064 * occurrence: there MUST be one and only one instance of this 1065 field. 1067 o time-taken: 1069 * format: DEC 1071 * field value: decimal value of the duration, in seconds, between 1072 the start of the processing of the request and the completion 1073 of the request processing (e.g., completion of delivery) by the 1074 Surrogate. 1076 * occurrence: there MUST be one and only one instance of this 1077 field. 1079 o c-ip: 1081 * format: ADDRESS 1083 * field value: the source IPv4 or IPv6 address (i.e., the 1084 "client" address) in the request received by the Surrogate. 1086 * occurrence: there MUST be one and only one instance of this 1087 field. 1089 o c-ip-anonymizing: 1091 * format: 1*DIGIT 1093 * field value: the number of rightmost bits of the address in the 1094 c-ip field that are zeroed-out in order to anonymize the 1095 logging record. The mechanism by which the two ends of the 1096 CDNI Logging interface agree on whether anonymization is to be 1097 supported and the number of bits that need to be zeroed-out for 1098 this purpose are outside the scope of the present document. 1100 * occurrence: there MUST be zero or exactly one instance of this 1101 field. 1103 o c-port: 1105 * format: 1*DIGIT 1107 * field value: the source TCP port (i.e., the "client" port) in 1108 the request received by the Surrogate. 1110 * occurrence: there MUST be zero or exactly one instance of this 1111 field. 1113 o s-ip: 1115 * format: ADDRESS 1117 * field value: the IPv4 or IPv6 address of the Surrogate that 1118 served the request (i.e., the "server" address). 1120 * occurrence: there MUST be zero or exactly one instance of this 1121 field. 1123 o s-hostname: 1125 * format: host 1127 * field value: the hostname of the Surrogate that served the 1128 request (i.e., the "server" hostname). 1130 * occurrence: there MUST be zero or exactly one instance of this 1131 field. 1133 o s-port: 1135 * format: 1*DIGIT 1137 * field value: the destination TCP port (i.e., the "server" port) 1138 in the request received by the Surrogate. 1140 * occurrence: there MUST be zero or exactly one instance of this 1141 field. 1143 o cs-method: 1145 * format: NHTABSTRING 1147 * field value: this is the method of the request received by the 1148 Surrogate. In the case of HTTP delivery, this is the HTTP 1149 method in the request. 1151 * occurrence: There MUST be one and only one instance of this 1152 field. 1154 o cs-uri: 1156 * format: NHTABSTRING 1158 * field value: this is the complete URL of the request received 1159 by the Surrogate. It is exactly in the format of a http_URL 1160 specified in [RFC2616]) or, when the request was a HTTPS 1161 request ([RFC2818]), it is in the format of a http_URL but with 1162 the scheme part set to "https" instead of "http". 1164 * occurrence: there MUST be zero or exactly one instance of this 1165 field. 1167 o u-uri: 1169 * format: NHTABSTRING 1171 * field value: this is a complete URL, derived from the complete 1172 URI of the request received by the Surrogate (i.e., the cs-uri) 1173 but transformed by the entity generating or transmitting the 1174 CDNI Logging Record, in a way that is agreed upon between the 1175 two ends of the CDNI Logging interface, so the transformed URI 1176 is meaningful to the uCDN. For example, the two ends of the 1177 CDNI Logging interface could agree that the u-uri is 1178 constructed from the cs-uri by removing the part of the 1179 hostname that exposes which individual Surrogate actually 1180 performed the delivery. The details of modification performed 1181 to generate the u-uri, as well as the mechanism to agree on 1182 these modifications between the two sides of the CDNI Logging 1183 interface are outside the scope of the present document. 1185 * occurrence: there MUST be one and only one instance of this 1186 field. 1188 o protocol: 1190 * format: NHTABSTRING 1192 * field value: this is value of the HTTP-Version field as 1193 specified in [RFC2616] of the Request-Line of the request 1194 received by the Surrogate (e.g., "HTTP/1.1"). 1196 * occurrence: there MUST be one and only one instance of this 1197 field. 1199 o sc-status: 1201 * format: 3DIGIT 1203 * field value: this is the Status-Code in the response from the 1204 Surrogate. tIn the case of HTTP delivery, this is the HTTP 1205 Status-Code in the HTTP response. 1207 * occurrence: There MUST be one and only one instance of this 1208 field. 1210 o sc-total-bytes: 1212 * format: 1*DIGIT 1214 * field value: this is the total number of bytes of the response 1215 sent by the Surrogate in response to the request. In the case 1216 of HTTP delivery, this includes the bytes of the Status-Line, 1217 the bytes of the HTTP headers and the bytes of the message- 1218 body. 1220 * occurrence: There MUST be one and only one instance of this 1221 field. 1223 o sc-entity-bytes: 1225 * format: 1*DIGIT 1227 * field value: this is the number of bytes of the message-body in 1228 the HTTP response sent by the Surrogate in response to the 1229 request. This does not include the bytes of the Status-Line or 1230 the bytes of the HTTP headers. 1232 * occurrence: there MUST be zero or exactly one instance of this 1233 field. 1235 o cs(): 1237 * format: QSTRING 1239 * field value: the value of the HTTP header (identified by the 1240 in the CDNI Logging field name) as it 1241 appears in the request processed by the Surrogate, but 1242 prepended by a DQUOTE and appended by a DQUOTE. For example, 1243 when the CDNI Logging field name (FIENAME) listed in the 1244 preceding Fields directive is cs(User-Agent), this CDNI Logging 1245 field value contains the value of the User-Agent HTTP header as 1246 received by the Surrogate in the request it processed, but 1247 prepended by a DQUOTE and appended by a DQUOTE. If the HTTP 1248 header as it appeared in the request processed by the Surrogate 1249 contains one or more DQUOTE, each DQUOTE MUST be escaped by an 1250 additional DQUOTE. For example, if the HTTP header contains 1251 My_Header"value", then the field value of the cs() is "My_Header""value""". 1254 * occurrence: there MAY be zero, one or any number of instance of 1255 this field. 1257 o sc(): 1259 * format: QSTRING 1261 * field value: the value of the HTTP header (identified by the 1262 in the CDNI Logging field name) as it 1263 appears in the response issued by the Surrogate to serve the 1264 request, but prepended by a DQUOTE and appended by a DQUOTE. 1266 If the HTTP header as it appeared in the request processed by 1267 the Surrogate contains one or more DQUOTE, each DQUOTE MUST be 1268 escaped by an additional DQUOTE. For example, if the HTTP 1269 header contains My_Header"value", then the field value of the 1270 cs() is "My_Header""value""". 1272 * occurrence: there MAY be zero, one or any number of instances 1273 of this field. For a given , there MUST be 1274 zero or exactly one instance of this field. 1276 o s-ccid: 1278 * format: QSTRING 1280 * field value: this contains the value of the Content Collection 1281 IDentifier (CCID) associated by the uCDN to the content served 1282 by the Surrogate via the CDNI Metadata interface 1283 ([I-D.ietf-cdni-metadata]), prepended by a DQUOTE and appended 1284 by a DQUOTE. If the CCID conveyed in the CDNI Metadata 1285 interface contains one or more DQUOTE, each DQUOTE MUST be 1286 escaped by an additional DQUOTE. For example, if the CCID 1287 conveyed in the CDNI Metadata interface is My_CCIDD"value", 1288 then the field value of the s-ccid is "My_CCID""value""". 1290 * occurrence: there MUST be zero or exactly one instance of this 1291 field. For a given , there MUST be zero or 1292 exactly one instance of this field. 1294 o s-sid: 1296 * format: QSTRING 1298 * field value: this contains the value of a Session IDentifier 1299 (SID) generated by the dCDN for a specific HTTP session, 1300 prepended by a DQUOTE and appended by a DQUOTE. In particular, 1301 for HTTP Adaptive Streaming (HAS) session, the Session 1302 IDentifier value is included in the Logging record for every 1303 content chunk delivery of that session in view of facilitating 1304 the later correlation of all the per content chunk log records 1305 of a given HAS session. See section 3.4.2.2. of [RFC6983] for 1306 more discussion on the concept of Session IDentifier in the 1307 context of HAS. If the SID conveyed contains one or more 1308 DQUOTE, each DQUOTE MUST be escaped by an additional DQUOTE. 1309 For example, if the SID is My_SID"value", then the field value 1310 of the s-sid is "My_SID""value""". 1312 * occurrence: there MUST be zero or exactly one instance of this 1313 field. 1315 o s-cached: 1317 * format: 1DIGIT 1319 * field value: this characterises whether the Surrogate served 1320 the request using content already stored on its local cache or 1321 not. The allowed values are "0" (for miss) and "1" (for hit). 1322 "1" MUST be used when the Surrogate did serve the request using 1323 exclusively content already stored on its local cache. "0" MUST 1324 be used otherwise (including cases where the Surrogate served 1325 the request using some, but not all, content already stored on 1326 its local cache). Note that a "0" only means a cache miss in 1327 the Surrogate and does not provide any information on whether 1328 the content was already stored, or not, in another device of 1329 the dCDN, i.e., whether this was a "dCDN hit" or "dCDN miss". 1331 * occurrence: there MUST be zero or exactly one instance of this 1332 field. 1334 The "Fields" directive corresponding to a HTTP Request Logging Record 1335 MUST contain all the fields names whose occurrence is specified above 1336 as "There MUST be one and only one instance of this field". The 1337 corresponding fields value MUST be present in every HTTP Request 1338 Logging Record. 1340 The "Fields" directive corresponding to a HTTP Request Logging Record 1341 MAY list all the fields value whose occurrence is specified above as 1342 "there MUST be zero or exactly one instance of this field" or "there 1343 MAY be zero, one or any number of instances of this field". The set 1344 of such field names actually listed in the "Fields" directive is 1345 selected by the CDN generating the CDNI Logging File based on 1346 agreements between the interconnected CDNs established through 1347 mechanisms outside the scope of this specification (e.g., contractual 1348 agreements). When such a field name is not listed in the "Fields" 1349 directive, the corresponding field value MUST NOT be included in the 1350 Logging Record. When such a field name is listed in the "Fields" 1351 directive, the corresponding field value MUST be included in the 1352 Logging Record; if the value for the field is not available, this 1353 MUST be conveyed via a dash character ("-"). 1355 The fields names listed in the "Fields" directive MAY be listed in 1356 the order in which they are listed in Section 3.4.1 or MAY be listed 1357 in any other order. 1359 A dCDN-side implementation of the CDNI Logging interface MUST 1360 implement all the following Logging Fields in a CDNI Logging Record 1361 of Record-Type "cdni_http_request_v1", and MUST support the ability 1362 to include valid values for each of them: 1364 o date 1366 o time 1368 o time-taken 1370 o c-ip 1372 o c-port 1374 o s-ip 1376 o s-hostname 1378 o s-port 1380 o cs-method 1382 o cs-uri 1384 o u-uri 1386 o protocol 1388 o sc-status 1390 o sc-total-bytes 1392 o sc-entity-bytes 1394 o cs() 1396 o sc() 1398 o s-cached 1400 A dCDN-side implementation of the CDNI Logging interface MAY support 1401 the following Logging Fields in a CDNI Logging Record of Record-Type 1402 "cdni_http_request_v1": 1404 o c-ip-anonymizing 1406 o s-ccid 1408 o s-sid 1409 If a dCDN-side implementation of the CDNI Logging interface supports 1410 these Fields, it MUST support the ability to include valid values for 1411 them. 1413 An uCDN-side implementation of the CDNI Logging interface MUST be 1414 able to accept CDNI Logging Files with CDNI Logging Records of 1415 Record-Type "cdni_http_request_v1" containing any CDNI Logging Field 1416 defined in Section 3.4.1 as long as the CDNI Logging Record and the 1417 CDNI Logging File are compliant with the present document. 1419 3.5. CDNI Logging File Example 1421 Let us consider the upstream CDN and the downstream CDN labelled uCDN 1422 and dCDN-1 in Figure 1. When dCDN-1 acts as a downstream CDN for 1423 uCDN and performs content delivery on behalf of uCDN, dCDN-1 will 1424 include the CDNI Logging Records corresponding to the content 1425 deliveries performed on behalf of uCDN in the CDNI Logging Files for 1426 uCDN. An example CDNI Logging File communicated by dCDN-1 to uCDN is 1427 shown below in Figure 4. 1429 #Version:CDNI/1.0 1431 #UUID:urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6 1433 #Claimed-Origin:cdni-logging-entity.dcdn-1.example.com 1435 #Record-Type:cdni_http_request_v1 1437 #Fields:datetimetime-takenc-ip 1438 cs-methodu-uriprotocolsc-status 1439 sc-total-bytescs(User-Agent)cs(Referer) 1440 s-cached 1442 2013-05-1700:38:06.8259.05810.5.7.1GET 1443 http://cdni-ucdn.dcdn-1.example.com/video/movie100.mp4 1444 HTTP/1.12006729891"Mozilla/5.0 1445 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1446 Gecko) Chrome/5.0.375.127 Safari/533.4" 1447 "host1.example.com"1 1449 2013-05-1700:39:09.14515.3210.5.10.5GET 1450 http://cdni-ucdn.dcdn-1.example.com/video/movie118.mp4 1451 HTTP/1.120015799210"Mozilla/5.0 1452 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1453 Gecko) Chrome/5.0.375.127 Safari/533.4" 1454 "host1.example.com"1 1456 2013-05-1700:42:53.43752.87910.5.10.5GET 1457 http://cdni-ucdn.dcdn-1.example.com/video/picture11.mp4 1458 HTTP/1.020097234724"Mozilla/5.0 1459 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1460 Gecko) Chrome/5.0.375.127 Safari/533.4" 1461 "host5.example.com"0 1463 #Integrity-Hash:fe113dfce8fec91323a4fc02261af26e 1465 [Editor's Note: compute the correct Integrity-Hash value once example 1466 is frozen] 1468 Figure 4: CDNI Logging File Example 1470 If uCDN establishes by some means (e.g. via TLS authentication when 1471 pulling the CDNI Logging File) the identity of the entity from which 1472 it pulled the CDNI Logging File, uCDN can add to the CDNI Logging a 1473 Verified-Origin directive as illustrated below: 1475 #Verified-Origin:cdni-logging-entity.dcdn-1.example.com 1476 As illustrated in Figure 2, uCDN will then ingest the corresponding 1477 CDNI Logging Records into its Collection process, alongside the 1478 Logging Records generated locally by the uCDN itself. This allows 1479 uCDN to aggregate Logging Records for deliveries performed by itself 1480 (through Records generated locally) as well as for deliveries 1481 performed by its downstream CDN(s). This aggregate information can 1482 then be used (after Filtering and Rectification, as illustrated in 1483 Figure 2) by Log Consuming Applications that take into account 1484 deliveries performed by uCDN as well as by all of its downstream 1485 CDNs. 1487 We observe that the time between 1489 1. when a delivery is completed in dCDN and 1491 2. when the corresponding Logging Record is ingested by the 1492 Collection process in uCDN 1494 depends on a number of parameters such as the Logging Period agreed 1495 to by uCDN and dCDN, how much time uCDN waits before pulling the CDNI 1496 Logging File once it is advertised in the CDNI Logging Feed, and the 1497 time to complete the pull of the CDNI Logging File. Therefore, if we 1498 consider the set of Logging Records aggregated by the Collection 1499 process in uCDN in a given time interval, there could be a permanent 1500 significant timing difference between the CDNI Logging Records 1501 received from the dCDN and the Logging Records generated locally. 1502 For example, in a given time interval, the Collection process in uCDN 1503 may be aggregating Logging Records generated locally by uCDN for 1504 deliveries performed in the last hour and CDNI Logging Records 1505 generated in the dCDN for deliveries in the hour before last. 1507 3.6. Cascaded CDNI Logging Files Example 1509 Let us consider the cascaded CDN scenario of uCDN, dCDN-2 and dCDN-3 1510 as depicted in Figure 1. After completion of a delivery by dCDN-3 on 1511 behalf of dCDN-2, dCDN-3 will include a corresponding Logging Record 1512 in a CDNI Logging File that will be pulled by dCDN-2 and that is 1513 illustrated below in Figure 5. In practice, a CDNI Logging File is 1514 likely to contain a very high number of CDNI Logging Records. 1515 However, for readability, the example in Figure 5 contains a single 1516 CDNI Logging Record. 1518 #Version:CDNI/1.0 1520 #UUID:urn:uuid:65718ef-0123-9876-adce4321bcde 1522 #Claimed-Origin:cdni-logging-entity.dcdn-3.example.com 1524 #Record-Type:cdni_http_request_v1 1526 #Fields:datetimetime-takenc-ip 1527 cs-methodu-uriprotocolsc-status 1528 sc-total-bytescs(User-Agent)cs(Referer) 1529 s-cached 1531 2013-05-1700:39:09.11914.0710.5.10.9GET 1532 http://cdni-dcdn-2.dcdn-3.example.com/video/movie118.mp4 1533 HTTP/1.120015799210"Mozilla/5.0 1534 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1535 Gecko) Chrome/5.0.375.127 Safari /533.4" 1536 "host1.example.com"1 1538 #Integrity-Hash:fe113dfce8fec91323a4fc02261af26e 1540 [Editor's Note: compute the correct Integrity-Hash value once example 1541 is frozen] 1543 Figure 5: Cascaded CDNI Logging File Example (dCDN-3 to dCDN-2) 1545 If dCDN-2 establishes by some means (e.g. via TLS authentication when 1546 pulling the CDNI Logging File) the identity of the entity from which 1547 it pulled the CDNI Logging File, dCDN-2 can add to the CDNI Logging a 1548 Verified-Origin directive as illustrated below: 1550 #Verified-Origin:cdni-logging-entity.dcdn-3.example.com 1552 dCDN-2 (behaving as an upstream CDN from the viewpoint of dCDN-3) 1553 will then ingest the CDNI Logging Record for the considered dCDN-3 1554 delivery into its Collection process (as illustrated in Figure 2). 1555 This Logging Record may be aggregated with Logging Records generated 1556 locally by dCDN-2 for deliveries performed by dCDN-2 itself. Say, 1557 for illustration, that the content delivery performed by dCDN-3 on 1558 behalf of dCDN-2 had actually been redirected to dCDN-2 by uCDN, and 1559 say that another content delivery has just been redirected by uCDN to 1560 dCDN-2 and that dCDN-2 elected to perform the corresponding delivery 1561 itself. Then after Filtering and Rectification (as illustrated in 1562 Figure 2), dCDN-2 will include the two Logging Records corresponding 1563 respectively to the delivery performed by dCDN-3 and the delivery 1564 performed by dCDN-2, in the next CDNI Logging File that will be 1565 communicated to uCDN. An example of such CDNI Logging File is 1566 illustrated below in Figure 6. 1568 #Version:CDNI/1.0 1570 #UUID:urn:uuid:1234567-8fedc-abab-0987654321ff 1572 #Claimed-Origin:cdni-logging-entity.dcdn-2.example.com 1574 #Record-Type:cdni_http_request_v1 1576 #Fields:datetimetime-takenc-ip 1577 cs-methodu-uriprotocolsc-status 1578 sc-total-bytescs(User-Agent)cs(Referer) 1579 s-cached 1581 2013-05-1700:39:09.11914.0710.5.10.9GET 1582 http://cdni-ucdn.dcdn-2.example.com/video/movie118.mp4 1583 HTTP/1.120015799210"Mozilla/5.0 1584 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1585 Gecko) Chrome/5.0.375.127 Safari /533.4" 1586 "host1.example.com"1 1588 2013-05-1701:42:53.43752.87910.5.10.12GET 1589 http://cdni-ucdn.dcdn-2.example.com/video/picture11.mp4 1590 HTTP/1.020097234724"Mozilla/5.0 1591 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1592 Gecko) Chrome/5.0.375.127 Safari /533.4" 1593 "host5.example.com"0 1595 #Integrity-Hash:fe113dfce8fec91323a4fc02261af26e 1597 [Editor's Note: compute the correct Integrity-Hash value once example 1598 is frozen] 1600 Figure 6: Cascaded CDNI Logging File Example (dCDN-2 to uCDN) 1602 If uCDN establishes by some means (e.g. via TLS authentication when 1603 pulling the CDNI Logging File) the identity of the entity from which 1604 it pulled the CDNI Logging File, uCDN can add to the CDNI Logging a 1605 Verified-Origin directive as illustrated below: 1607 #Verified-Origin:cdni-logging-entity.dcdn-2.example.com 1609 In the example of Figure 6, we observe that: 1611 o the first Logging Record corresponds to the Logging Record 1612 communicated earlier to dCDN-2 by dCDN-3, which corresponds to a 1613 delivery redirected by uCDN to dCDN-2 and then redirected by 1614 dCDN-2 to dCDN-3. The fields values in this Logging Record are 1615 copied from the corresponding CDNI Logging REcord communicated to 1616 dCDN2 by dCDN-3, with the exception of the u-uri that now reflects 1617 the URI convention between uCDN and dCDN-2 and that presents the 1618 delivery to uCDN as if it was performed by dCDN-2 itself. This 1619 reflects the fact that dCDN-2 had taken the full responsibility of 1620 the corresponding delivery (even if in this case, dCDN-2 elected 1621 to redirect the delivery to dCDN-3 so it is actually performed by 1622 dCDN-3 on behalf of dCDN-2). 1624 o the second Logging Record corresponds to a delivery redirected by 1625 uCDN to dCDN-2 and performed by dCDN-2 itself. The time of the 1626 delivery in this Logging Record may be significantly more recent 1627 than the first Logging Record since it was generated locally while 1628 the first Logging Record was generated by dCDN-3 and had to be 1629 advertised , and then pulled and then ingested into the dCDN-2 1630 Collection process, before being aggregated with the second 1631 Logging Record. 1633 4. Protocol for Exchange of CDNI Logging File After Full Collection 1635 This section specifies a protocol for the exchange of CDNI Logging 1636 Files as specified in Section 3 after the CDNI Logging File is fully 1637 collected by the dCDN. 1639 This protocol comprises: 1641 o a CDNI Logging feed, allowing the dCDN to notify the uCDN about 1642 the CDNI Logging Files that can be retrieved by that uCDN from the 1643 dCDN, as well as all the information necessary for retrieving each 1644 of these CDNI Logging Files. The CDNI Logging feed is specified 1645 in Section 4.1. 1647 o a CDNI Logging File pull mechanism, allowing the uCDN to obtain 1648 from the dCDN a given CDNI Logging File at the uCDN's convenience. 1649 The CDNI Logging File pull mechanisms is specified in Section 4.2. 1651 An implementation of the CDNI Logging interface on the dCDN side (the 1652 entity generating the CDNI Logging file) MUST support the server side 1653 of the CDNI Logging feed (as specified in Section 4.1) and the server 1654 side of the CDNI Logging pull mechanism (as specified in 1655 Section 4.2). 1657 An implementation of the CDNI Logging interface on the uCDN side (the 1658 entity consuming the CDNI Logging file) MUST support the client side 1659 of the CDNI Logging feed (as specified in Section 4.1) and the client 1660 side of the CDNI Logging pull mechanism (as specified in 1661 Section 4.2). 1663 4.1. CDNI Logging Feed 1665 The server-side implementation of the CDNI Logging feed MUST produce 1666 an Atom feed [RFC4287]. This feed is used to advertise log files 1667 that are available for the client-side to retrieve using the CDNI 1668 Logging pull mechanism. 1670 4.1.1. Atom Formatting 1672 A CDNI Logging feed MUST be structured as an Archived feed, as 1673 defined in [RFC5005], and MUST be formatted in Atom [RFC4287]. This 1674 means it consists of a subscription document that is regularly 1675 updated as new CDNI Logging Files become available, and information 1676 about older CDNI Logging files is moved into archive documents. Once 1677 created, archive documents are never modified. 1679 Each CDNI Logging File listed in an Atom feed MUST be described in an 1680 atom:entry container element. 1682 The atom:entry MUST contain an atom:content element whose "src" 1683 attribute is a link to the CDNI Logging File and whose "type" 1684 attribute is the MIME Media Type indicating that the entry is a CDNI 1685 Logging File. We define this MIME Media Type as "application/ 1686 cdni.LoggingFile" (See Section 6.4). 1688 For compatibility with some Atom feed readers the atom:entry MAY also 1689 contain an atom:link entry whose "href" attribute is a link to the 1690 CDNI Logging File and whose "type" attribute is the MIME Media Type 1691 indicating that the entry is a CDNI Logging File using the 1692 "application/cdni.LoggingFile" MIME Media Type (See Section 6.4). 1694 The URI used in the atom:id of the atom:entry MUST contain the UUID 1695 of the CDNI Logging File. 1697 The atom:updated in the atom:entry MUST indicate the time at which 1698 the CDNI Logging File was last updated. 1700 4.1.2. Updates to Log Files and the Feed 1702 CDNI Logging Files MUST NOT be modified by the dCDN once published in 1703 the CDNI Logging feed. 1705 The frequency with which the subscription feed is updated, the period 1706 of time covered by each CDNI Logging File or each archive document, 1707 and timeliness of publishing of CDNI Logging Files are outside the 1708 scope of the present document and are expected to be agreed upon by 1709 uCDN and dCDN via other means (e.g., human agreement). 1711 The server-side implementation MUST be able to set, and SHOULD set, 1712 HTTP cache control headers on the subscription feed to indicate the 1713 frequency at which the client-side is to poll for updates. 1715 The client-side MAY use HTTP cache control headers (set by the 1716 server-side) on the subscription feed to determine the frequency at 1717 which to poll for updates. The client-side MAY instead, or in 1718 addition, use other information to determine when to poll for updates 1719 (e.g., a polling frequency that may have been negotiated between the 1720 uCDN and dCDN by mechanisms outside the scope of the present document 1721 and that is to override the indications provided in the HTTP cache 1722 control headers). 1724 The potential retention limits (e.g., sliding time window) within 1725 which the dCDN is to retain and be ready to serve an archive document 1726 is outside the scope of the present document and is expected to be 1727 agreed upon by uCDN and dCDN via other means (e.g., human agreement). 1728 The server-side implementation MUST retain, and be ready to serve, 1729 any archive document within the agreed retention limits. Outside 1730 these agreed limits, the server-side implementation MAY indicate its 1731 inability to serve (e.g., with HTTP status code 404) an archive 1732 document or MAY refuse to serve it (e.g., with HTTP status code 403 1733 or 410). 1735 4.1.3. Redundant Feeds 1737 The server-side implementation MAY present more than one CDNI Logging 1738 feed for redundancy. Each CDNI Logging File MAY be published in more 1739 than one feed. 1741 A client-side implementation MAY support such redundant CDNI Logging 1742 feeds. If it supports redundant CDNI Logging feed, the client-side 1743 can use the UUID of the CDNI Logging File, presented in the atom:id 1744 element of the Atom feed, to avoid unnecessarily pulling and storing 1745 a given CDNI Logging File more than once. 1747 4.1.4. Example CDNI Logging Feed 1749 Figure 7 illustrates an example of the subscription document of a 1750 CDNI Logging feed. 1752 1753 > 1755 CDNI Logging Feed 1756 2013-03-23T14:46:11Z 1757 urn:uuid:663ae677-40fb-e99a-049d-c5642916b8ce 1758 1760 1762 1764 CDNI Log Feed 1765 Generator 1766 dcdn.example 1767 1768 CDNI Logging File for uCDN at 1769 2013-03-23 14:15:00 1770 urn:uuid:12345678-1234-abcd-00aa-01234567abcd 1771 2013-03-23T14:15:00Z 1772 1775 CDNI Logging File for uCDN at 1776 2013-03-23 14:15:00 1777 1778 1779 CDNI Logging File for uCDN at 1780 2013-03-23 14:30:00 1781 urn:uuid:87654321-4321-dcba-aa00-dcba7654321 1782 2013-03-23T14:30:00Z 1783 1786 CDNI Logging File for uCDN at 1787 2013-03-23 14:30:00 1788 1789 ... 1790 1791 ... 1792 1793 1795 Figure 7: Example subscription document of a CDNI Logging Feed 1797 4.2. CDNI Logging File Pull 1799 A client-side implementation of the CDNI Logging interface MAY pull, 1800 at its convenience, a CDNI Logging File that is published by the 1801 server-side in the CDNI Logging Feed (in the subscription document or 1802 an archive document). To do so, the client-side: 1804 o MUST implement HTTP/1.1 [RFC2616], MAY also support other HTTP 1805 versions (e.g., HTTP/2.0 [I-D.ietf-httpbis-http2]) and MAY 1806 negotiate which HTTP version is actually used. This allows 1807 operators and implementers to choose to use later versions of HTTP 1808 to take advantage of new features, while still ensuring 1809 interoperability with systems that only support HTTP/1.1. 1811 o MUST use the URI that was associated to the CDNI Logging File 1812 (within the "src" attribute of the corresponding atom:content 1813 element) in the CDNI Logging Feed; 1815 o MUST support exchange of CDNI Logging Files with no content 1816 encoding applied to the representation; 1818 o MUST support exchange of CDNI Logging Files with "gzip" content 1819 encoding (as defined in [RFC2616]) applied to the representation. 1821 Note that a client-side implementation of the CDNI Logging interface 1822 MAY pull a CDNI Logging File that it has already pulled. 1824 The server-side implementation MUST respond to valid pull request by 1825 a client-side implementation for a CDNI Logging File published by the 1826 server-side in the CDNI Logging Feed (in the subscription document or 1827 an archive document). The server-side implementation: 1829 o MUST implement HTTP/1.1 to handle the client-side request and MAY 1830 also support other HTTP versions (e.g., HTTP/2.0); 1832 o MUST include the CDNI Logging File identified by the request URI 1833 inside the body of the HTTP response; 1835 o MUST support exchange of CDNI Logging Files with no content 1836 encoding applied to the representation; 1838 o MUST support exchange of CDNI Logging Files with "gzip" content 1839 encoding (as defined in [RFC2616]) applied to the representation. 1841 Content negotiation approaches defined in [RFC2616] (e.g., using 1842 Accept-Encoding request-header field or Content-Encoding entity- 1843 header field) MAY be used by the client-side and server-side 1844 implementations to establish the content-coding to be used for a 1845 particular exchange of a CDNI Logging File. 1847 Applying compression content encoding (such as "gzip") is expected to 1848 mitigate the impact of exchanging the large volumes of logging 1849 information expected across CDNs. This is expected to be 1850 particularly useful in the presence of HTTP Adaptive Streaming (HAS) 1851 which, as per the present version of the document, will result in a 1852 separate CDNI Log Record for each HAS segment delivery in the CDNI 1853 Logging File. 1855 The potential retention limits (e.g., sliding time window, maximum 1856 aggregate file storage quotas) within which the dCDN is to retain and 1857 be ready to serve a CDNI Logging File previously advertised in the 1858 CDNI Logging Feed is outside the scope of the present document and is 1859 expected to be agreed upon by uCDN and dCDN via other means (e.g., 1860 human agreement). The server-side implementation MUST retain, and be 1861 ready to serve, any CDNI Logging File within the agreed retention 1862 limits. Outside these agreed limits, the server-side implementation 1863 MAY indicate its inability to serve (e.g., with HTTP status code 404) 1864 a CDNI Logging File or MAY refuse to serve it (e.g., with HTTP status 1865 code 403 or 410). 1867 5. Protocol for Exchange of CDNI Logging File During Collection 1869 We note that, in addition to the CDNI Logging File exchange protocol 1870 specified in Section 4, implementations of the CDNI Logging interface 1871 MAY also support other mechanisms to exchange CDNI Logging Files. In 1872 particular, such mechanisms might allow the exchange of the CDNI 1873 Logging File to start before the file is fully collected. This can 1874 allow CDNI Logging Records to be communicated by the dCDN to the uCDN 1875 as they are gathered by the dCDN without having to wait until all the 1876 CDNI Logging Records of the same logging period are collected in the 1877 corresponding CDNI Logging File. This approach is commonly referred 1878 to as "tailing" of the file. 1880 Such an approach could be used, for example, to exchange logging 1881 information with a significantly reduced time-lag (e.g., sub-minute 1882 or sub-second) between when the event occurred in the dCDN and when 1883 the corresponding CDNI Logging Record is made available to the uCDN. 1884 This can satisfy log-consuming applications requiring extremely fresh 1885 logging information such as near-real-time content delivery 1886 monitoring. Such mechanisms are for further study and outside the 1887 scope of this document. 1889 6. IANA Considerations 1891 6.1. CDNI Logging Directive Names Registry 1893 The IANA is requested to create a new registry, CDNI Logging 1894 Directive Names. 1896 The initial contents of the CDNI Logging File Directives registry 1897 comprise the names of the directives specified in Section 3.3 of the 1898 present document, and are as follows: 1900 +------------------------------+-----------+ 1901 | Directive Name | Reference | 1902 +------------------------------+-----------+ 1903 | Version | RFC xxxx | 1904 | UUID | RFC xxxx | 1905 | Claimed-Origin | RFC xxxx | 1906 | Verified-Origin | RFC xxxx | 1907 | Record-Type | RFC xxxx | 1908 | Fields | RFC xxxx | 1909 | Integrity-Hash | RFC xxxx | 1910 +------------------------------+-----------+ 1912 Figure 8 1914 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 1915 the present document] 1917 Within the registry, names are to be allocated by IANA according to 1918 the "Specification Required" policy specified in [RFC5226]. 1919 Directive names are to be allocated by IANA with a format of 1920 NAMEFORMAT (see Section 3.1). 1922 Each specification that defines a new CDNI Logging directive needs to 1923 contain a description for the new directive with the same set of 1924 information as provided in Section 3.3 (i.e., format, directive value 1925 and occurrence). 1927 6.2. CDNI Logging Record-Types Registry 1929 The IANA is requested to create a new registry, CDNI Logging Record- 1930 Types. 1932 The initial contents of the CDNI Logging Record-Types registry 1933 comprise the names of the CDNI Logging Record types specified in 1934 Section 3.4 of the present document, and are as follows: 1936 +------------------------------+-----------+----------------------------------+ 1937 | Record-Types | Reference | Description | 1938 +------------------------------+-----------+----------------------------------+ 1939 | cdni_http_request_v1 | RFC xxxx | CDNI Logging Record version 1 | 1940 | | | for content delivery using HTTP | 1941 +------------------------------+-----------+----------------------------------+ 1943 Figure 9 1945 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 1946 the present document] 1948 Within the registry, Record-Types are to be allocated by IANA 1949 according to the "Specification Required" policy specified in 1950 [RFC5226]. Record-Types are to be allocated by IANA with a format of 1951 NAMEFORMAT (see Section 3.1). Record-Types corresponding to 1952 specifications produced by the IETF CDNI Working Group are to be 1953 allocated a name starting with "cdni_". All other Record-Types are 1954 to be allocated a name that does not start with "cdni". 1956 Each specification that defines a new Record-Type needs to contain a 1957 description for the new Record-Type with the same set of information 1958 as provided in Section 3.4.1. This includes: 1960 o a list of all the CDNI Logging Fields that can appear in a CDNI 1961 Logging Record of the new Record-Type 1963 o for all these Fields: a specification of the occurrence for each 1964 Field in the new Record-Type 1966 o for every newly defined Field, i.e., for every Field that results 1967 in a registration in the CDNI Logging Field Names Registry 1968 (Section 6.3): a specification of the field name, format and field 1969 value. 1971 6.3. CDNI Logging Field Names Registry 1973 The IANA is requested to create a new registry, CDNI Logging Field 1974 Names. 1976 The above registry is intended to be shared across the currently 1977 defined Record-Type (i.e., cdni_http_request_v1) as well as potential 1978 other CDNI Logging Record-Types that may be defined in separate 1979 specifications. When a Field from this registry is used by another 1980 CDNI Logging Record-Type, it is to be used with the exact semantics 1981 and format specified in the document that registered this field and 1982 that is identified in the Reference column of the registry. If 1983 another CDNI Logging Record-Type requires a Field with semantics that 1984 are not strictly identical, or a format that is not strictly 1985 identical then this new Field is to be registered in the registry 1986 with a different Field name. When a Field from this registry is used 1987 by another CDNI Logging Record-Type, it can be used with different 1988 occurence rules. 1990 The initial contents of the CDNI Logging Fields Names registry 1991 comprise the names of the CDNI Logging fields specified in 1992 Section 3.4 of the present document, and are as follows: 1994 +---------------------------------------------+-----------+ 1995 | Field Name | Reference | 1996 +---------------------------------------------+-----------+ 1997 | date | RFC xxxx | 1998 | time | RFC xxxx | 1999 | time-taken | RFC xxxx | 2000 | c-ip | RFC xxxx | 2001 | c-ip-anonymizing | RFC xxxx | 2002 | c-port | RFC xxxx | 2003 | s-ip | RFC xxxx | 2004 | s-hostname | RFC xxxx | 2005 | s-port | RFC xxxx | 2006 | cs-method | RFC xxxx | 2007 | cs-uri | RFC xxxx | 2008 | u-uri | RFC xxxx | 2009 | protocol | RFC xxxx | 2010 | sc-status | RFC xxxx | 2011 | sc-total-bytes | RFC xxxx | 2012 | sc-entity-bytes | RFC xxxx | 2013 | cs() | RFC xxxx | 2014 | sc() | RFC xxxx | 2015 | s-ccid | RFC xxxx | 2016 | s-sid | RFC xxxx | 2017 | s-cached | RFC xxxx | 2018 +---------------------------------------------+-----------+ 2020 Figure 10 2022 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2023 the present document] 2025 Within the registry, names are to be allocated by IANA according to 2026 the "Specification Required" policy specified in [RFC5226]. Field 2027 names are to be allocated by IANA with a format of NHTABSTRING (see 2028 Section 3.1). 2030 6.4. CDNI Logging MIME Media Type 2032 The IANA is requested to allocate the "application/cdni.LoggingFile" 2033 MIME Media Type (whose use is specified in Section 4.1.1 of the 2034 present document) in the MIME Media Types registry. 2036 7. Security Considerations 2038 7.1. Authentication, Confidentiality, Integrity Protection 2040 An implementation of the CDNI Logging interface MUST support TLS 2041 transport of the CDNI Logging feed (Section 4.1) and of the CDNI 2042 Logging File pull (Section 4.2) as per [RFC2818]. 2044 The use of TLS for transport of the CDNI Logging feed and CDNI 2045 Logging File pull allows: 2047 o the dCDN and uCDN to authenticate each other (to ensure they are 2048 transmitting/receiving CDNI Logging File from an authenticated 2049 CDN) 2051 o the CDNI Logging information to be transmitted with 2052 confidentiality 2054 o the integrity of the CDNI Logging information to be protected 2055 during the exchange 2057 In an environment where any such protection is required, TLS SHOULD 2058 be used (including authentication of the remote end) by the server- 2059 side and the client-side of the CDNI Logging feed and the CDNI 2060 Logging File pull mechanism unless alternate methods are used for 2061 ensuring the confidentiality of the information in the logging files 2062 (such as setting up an IPsec tunnel between the two CDNs or using a 2063 physically secured internal network between two CDNs that are owned 2064 by the same corporate entity). 2066 An implementation of the CDNI Logging interface MUST support the 2067 TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 cipher suite ( [RFC5288]). An 2068 implementation of the CDNI Logging interface SHOULD prefer cipher 2069 suites which support perfect forward secrecy over cipher suites that 2070 don't. 2072 The Integrity-Hash directive inside the CDNI Logging File provides 2073 additional integrity protection, this time targeting potential 2074 corruption of the CDNI logging information during the CDNI Logging 2075 File generation, storage or exchange. This mechanism does not itself 2076 allow restoration of the corrupted CDNI Logging information, but it 2077 allows detection of such corruption and therefore triggering of 2078 appropriate corrective actions (e.g., discard of corrupted 2079 information, attempt to re-obtain the CDNI Logging information). 2080 Note that the Integrity-Hash does not protect against tampering by a 2081 third party, since such a third party could have recomputed and 2082 updated the Integrity-Hash after tampering. Protection against third 2083 party tampering can be achieved as discussed above through the use of 2084 TLS. 2086 7.2. Denial of Service 2088 This document does not define specific mechanism to protect against 2089 Denial of Service (DoS) attacks on the Logging Interface. However, 2090 the CDNI Logging feed and CDNI Logging pull endpoints can be 2091 protected against DoS attacks through the use of TLS transport and/or 2092 via mechanisms outside the scope of the CDNI Logging interface such 2093 as firewalling or use of Virtual Private Networks (VPNs). 2095 Protection of dCDN Surrogates against spoofed delivery requests is 2096 outside the scope of the CDNI Logging interface. 2098 7.3. Privacy 2100 CDNs have the opportunity to collect detailed information about the 2101 downloads performed by End Users. The provision of this information 2102 to another CDN introduces potential End Users privacy protection 2103 concerns. 2105 The use of TLS for transport of the CDNI Logging feed and CDNI 2106 Logging pull as discussed in Section 7.1 protects the confidentiality 2107 of logged information by preventing any other party than the 2108 authorised uCDN to gain access to the logging information. 2110 We observe that when CDNI interconnection is realised as per 2111 [I-D.ietf-cdni-framework], the uCDN handles the initial End User 2112 requests (before it is redirected to the dCDN) so, regardless of 2113 which information is, or is not, communicated to the uCDN through the 2114 CDNI Logging interface, the uCDN has visibility on significant 2115 information such as the IP address of the End User request and the 2116 URL of the request. 2118 Nonetheless, if the dCDN and uCDN agree that anonymization is 2119 required to avoid making some detailed information available to the 2120 uCDN (such as how many bytes of the content have been watched by an 2121 End User and/or at what time) or is required to meet some legal 2122 obligations, then the uCDN and dCDN can agree to exchange anonymized 2123 End Users IP address in CDNI Logging Files and the c-ip-anonymization 2124 field can be used to convey the number of bits that have been 2125 anonymized so that the meaningful information can still be easily 2126 extracted from the anonymized addressses (e.g., for geolocation aware 2127 analytics). 2129 We note that anonymization of End Users IP address does not fully 2130 protect against deriving potentially sensitive information about 2131 traffic patterns; in general, increasing the number of bits that are 2132 anonymized can mitigate the risks of deriving such sensitive traffic 2133 pattern information. 2135 We also note that independently of IP addresses, the query string 2136 portion of the URL that may be conveyed inside the cs-uri and u-uri 2137 fields of CDNI Logging Files, or the HTTP cookies( [RFC6265]) that 2138 may be conveyed inside the cs() field of CDNI 2139 Logging Fields, may contain personnal information or information that 2140 can be exploited to derive personal information. Where this is a 2141 concern, the CDNI Logging interface specification allows the dCDN to 2142 not include the cs-uri and to include a u-uri that removes (or hides) 2143 the sensitive part of the query string and allows the dCDN to not 2144 include the cs() fields corresponding to HTTP 2145 headers associated with cookies. 2147 8. Acknowledgments 2149 This document borrows from the W3C Extended Log Format [ELF]. 2151 Rob Murray significantly contributed into the text of Section 4.1. 2153 The authors thank Ben Niven-Jenkins, Kevin Ma, David Mandelberg and 2154 Ray van Brandenburg for their ongoing input. 2156 Finally, we also thank Sebastien Cubaud, Pawel Grochocki, Christian 2157 Jacquenet, Yannick Le Louedec, Anne Marrec , Emile Stephan, Fabio 2158 Costa, Sara Oueslati, Yvan Massot, Renaud Edel, Joel Favier and the 2159 contributors of the EU FP7 OCEAN project for their input in the early 2160 versions of this document. 2162 9. References 2164 9.1. Normative References 2166 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2167 Requirement Levels", BCP 14, RFC 2119, March 1997. 2169 [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., 2170 Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext 2171 Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. 2173 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 2174 Resource Identifier (URI): Generic Syntax", STD 66, RFC 2175 3986, January 2005. 2177 [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally 2178 Unique IDentifier (UUID) URN Namespace", RFC 4122, July 2179 2005. 2181 [RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom 2182 Syndication Format", RFC 4287, December 2005. 2184 [RFC5005] Nottingham, M., "Feed Paging and Archiving", RFC 5005, 2185 September 2007. 2187 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 2188 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 2189 May 2008. 2191 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 2192 Specifications: ABNF", STD 68, RFC 5234, January 2008. 2194 [RFC5288] Salowey, J., Choudhury, A., and D. McGrew, "AES Galois 2195 Counter Mode (GCM) Cipher Suites for TLS", RFC 5288, 2196 August 2008. 2198 9.2. Informative References 2200 [CHAR_SET] 2201 "IANA Character Sets registry", 2202 . 2205 [ELF] Phillip M. Hallam-Baker, and Brian Behlendorf, "Extended 2206 Log File Format, W3C (work in progress), WD-logfile- 2207 960323", . 2209 [I-D.ietf-cdni-framework] 2210 Peterson, L., Davie, B., and R. Brandenburg, "Framework 2211 for CDN Interconnection", draft-ietf-cdni-framework-14 2212 (work in progress), June 2014. 2214 [I-D.ietf-cdni-metadata] 2215 Niven-Jenkins, B., Murray, R., Caulfield, M., Leung, K., 2216 and K. Ma, "CDN Interconnection Metadata", draft-ietf- 2217 cdni-metadata-07 (work in progress), July 2014. 2219 [I-D.ietf-cdni-requirements] 2220 Leung, K. and Y. Lee, "Content Distribution Network 2221 Interconnection (CDNI) Requirements", draft-ietf-cdni- 2222 requirements-17 (work in progress), January 2014. 2224 [I-D.ietf-httpbis-http2] 2225 Belshe, M., Peon, R., and M. Thomson, "Hypertext Transfer 2226 Protocol version 2", draft-ietf-httpbis-http2-13 (work in 2227 progress), June 2014. 2229 [I-D.snell-atompub-link-extensions] 2230 Snell, J., "Atom Link Extensions", draft-snell-atompub- 2231 link-extensions-09 (work in progress), June 2012. 2233 [RFC1321] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321, 2234 April 1992. 2236 [RFC1945] Berners-Lee, T., Fielding, R., and H. Nielsen, "Hypertext 2237 Transfer Protocol -- HTTP/1.0", RFC 1945, May 1996. 2239 [RFC2818] Rescorla, E., "HTTP Over TLS", RFC 2818, May 2000. 2241 [RFC6265] Barth, A., "HTTP State Management Mechanism", RFC 6265, 2242 April 2011. 2244 [RFC6707] Niven-Jenkins, B., Le Faucheur, F., and N. Bitar, "Content 2245 Distribution Network Interconnection (CDNI) Problem 2246 Statement", RFC 6707, September 2012. 2248 [RFC6770] Bertrand, G., Stephan, E., Burbridge, T., Eardley, P., Ma, 2249 K., and G. Watson, "Use Cases for Content Delivery Network 2250 Interconnection", RFC 6770, November 2012. 2252 [RFC6983] van Brandenburg, R., van Deventer, O., Le Faucheur, F., 2253 and K. Leung, "Models for HTTP-Adaptive-Streaming-Aware 2254 Content Distribution Network Interconnection (CDNI)", RFC 2255 6983, July 2013. 2257 Authors' Addresses 2258 Francois Le Faucheur (editor) 2259 Cisco Systems 2260 E.Space Park - Batiment D 2261 6254 Allee des Ormes - BP 1200 2262 Mougins cedex 06254 2263 FR 2265 Phone: +33 4 97 23 26 19 2266 Email: flefauch@cisco.com 2268 Gilles Bertrand (editor) 2269 Orange 2270 38-40 rue du General Leclerc 2271 Issy les Moulineaux 92130 2272 FR 2274 Phone: +33 1 45 29 89 46 2275 Email: gilles.bertrand@orange.com 2277 Iuniana Oprescu (editor) 2278 Orange 2279 38-40 rue du General Leclerc 2280 Issy les Moulineaux 92130 2281 FR 2283 Phone: +33 6 89 06 92 72 2284 Email: iuniana.oprescu@orange.com 2286 Roy Peterkofsky 2287 Skytide, Inc. 2288 One Kaiser Plaza, Suite 785 2289 Oakland CA 94612 2290 USA 2292 Phone: +01 510 250 4284 2293 Email: roy@skytide.com