idnits 2.17.1 draft-ietf-cdni-logging-26.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (May 25, 2016) is 2890 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFCthis' is mentioned on line 2376, but not defined -- Possible downref: Non-RFC (?) normative reference: ref. 'AES' -- Possible downref: Non-RFC (?) normative reference: ref. 'GCM' ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) ** Obsolete normative reference: RFC 7231 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7232 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7233 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7234 (Obsoleted by RFC 9111) ** Obsolete normative reference: RFC 7235 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7525 (Obsoleted by RFC 9325) ** Obsolete normative reference: RFC 7540 (Obsoleted by RFC 9113) -- Possible downref: Non-RFC (?) normative reference: ref. 'SHA-3' == Outdated reference: A later version (-21) exists of draft-ietf-cdni-metadata-17 -- Obsolete informational reference (is this intentional?): RFC 2818 (Obsoleted by RFC 9110) Summary: 9 errors (**), 0 flaws (~~), 3 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force F. Le Faucheur, Ed. 3 Internet-Draft 4 Intended status: Standards Track G. Bertrand, Ed. 5 Expires: November 26, 2016 Orange 6 I. Oprescu, Ed. 8 R. Peterkofsky 9 Google Inc. 10 May 25, 2016 12 CDNI Logging Interface 13 draft-ietf-cdni-logging-26 15 Abstract 17 This memo specifies the Logging interface between a downstream CDN 18 (dCDN) and an upstream CDN (uCDN) that are interconnected as per the 19 CDN Interconnection (CDNI) framework. First, it describes a 20 reference model for CDNI logging. Then, it specifies the CDNI 21 Logging File format and the actual protocol for exchange of CDNI 22 Logging Files. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at http://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on November 26, 2016. 41 Copyright Notice 43 Copyright (c) 2016 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (http://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 59 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 60 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 5 61 2. CDNI Logging Reference Model . . . . . . . . . . . . . . . . 5 62 2.1. CDNI Logging interactions . . . . . . . . . . . . . . . . 5 63 2.2. Overall Logging Chain . . . . . . . . . . . . . . . . . . 8 64 2.2.1. Logging Generation and During-Generation Aggregation 9 65 2.2.2. Logging Collection . . . . . . . . . . . . . . . . . 10 66 2.2.3. Logging Filtering . . . . . . . . . . . . . . . . . . 10 67 2.2.4. Logging Rectification and Post-Generation Aggregation 11 68 2.2.5. Log-Consuming Applications . . . . . . . . . . . . . 12 69 2.2.5.1. Maintenance/Debugging . . . . . . . . . . . . . . 12 70 2.2.5.2. Accounting . . . . . . . . . . . . . . . . . . . 13 71 2.2.5.3. Analytics and Reporting . . . . . . . . . . . . . 13 72 2.2.5.4. Content Protection . . . . . . . . . . . . . . . 13 73 2.2.5.5. Notions common to multiple Log Consuming 74 Applications . . . . . . . . . . . . . . . . . . 14 75 3. CDNI Logging File . . . . . . . . . . . . . . . . . . . . . . 16 76 3.1. Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 16 77 3.2. CDNI Logging File Structure . . . . . . . . . . . . . . . 17 78 3.3. CDNI Logging Directives . . . . . . . . . . . . . . . . . 20 79 3.4. CDNI Logging Records . . . . . . . . . . . . . . . . . . 24 80 3.4.1. HTTP Request Logging Record . . . . . . . . . . . . . 25 81 3.5. CDNI Logging File Extension . . . . . . . . . . . . . . . 36 82 3.6. CDNI Logging File Examples . . . . . . . . . . . . . . . 36 83 3.7. Cascaded CDNI Logging Files Example . . . . . . . . . . . 39 84 4. Protocol for Exchange of CDNI Logging File After Full 85 Collection . . . . . . . . . . . . . . . . . . . . . . . . . 42 86 4.1. CDNI Logging Feed . . . . . . . . . . . . . . . . . . . . 43 87 4.1.1. Atom Formatting . . . . . . . . . . . . . . . . . . . 43 88 4.1.2. Updates to Log Files and the Feed . . . . . . . . . . 43 89 4.1.3. Redundant Feeds . . . . . . . . . . . . . . . . . . . 44 90 4.1.4. Example CDNI Logging Feed . . . . . . . . . . . . . . 44 91 4.2. CDNI Logging File Pull . . . . . . . . . . . . . . . . . 46 92 5. Protocol for Exchange of CDNI Logging File During Collection 47 93 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 48 94 6.1. CDNI Logging Directive Names Registry . . . . . . . . . . 48 95 6.2. CDNI Logging File version Registry . . . . . . . . . . . 49 96 6.3. CDNI Logging record-types Registry . . . . . . . . . . . 49 97 6.4. CDNI Logging Field Names Registry . . . . . . . . . . . . 50 98 6.5. CDNI Logging MIME Media Type . . . . . . . . . . . . . . 51 99 7. Security Considerations . . . . . . . . . . . . . . . . . . . 52 100 7.1. Authentication, Authorization, Confidentiality, Integrity 101 Protection . . . . . . . . . . . . . . . . . . . . . . . 52 102 7.2. Denial of Service . . . . . . . . . . . . . . . . . . . . 53 103 7.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 53 104 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 54 105 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 55 106 9.1. Normative References . . . . . . . . . . . . . . . . . . 55 107 9.2. Informative References . . . . . . . . . . . . . . . . . 57 108 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 58 110 1. Introduction 112 This memo specifies the CDNI Logging interface between a downstream 113 CDN (dCDN) and an upstream CDN (uCDN). First, it describes a 114 reference model for CDNI logging. Then, it specifies the CDNI 115 Logging File format and the actual protocol for exchange of CDNI 116 Logging Files. 118 The reader should be familiar with the following documents: 120 o CDNI problem statement [RFC6707] and framework [RFC7336] identify 121 a Logging interface, 123 o Section 8 of [RFC7337] specifies a set of requirements for 124 Logging, 126 o [RFC6770] outlines real world use-cases for interconnecting CDNs. 127 These use cases require the exchange of Logging information 128 between the dCDN and the uCDN. 130 As stated in [RFC6707], "the CDNI Logging interface enables details 131 of logs or events to be exchanged between interconnected CDNs". 133 The present document describes: 135 o The CDNI Logging reference model (Section 2), 137 o The CDNI Logging File format (Section 3), 139 o The CDNI Logging File Exchange protocol (Section 4). 141 1.1. Terminology 143 In this document, the first letter of each CDNI-specific term is 144 capitalized. We adopt the terminology described in [RFC6707] and 145 [RFC7336], and extend it with the additional terms defined below. 147 Intra-CDN Logging information: logging information generated and 148 collected within a CDN. The format of the Intra-CDN Logging 149 information may be different to the format of the CDNI Logging 150 information. 152 CDNI Logging information: logging information exchanged across CDNs 153 using the CDNI Logging Interface. 155 Logging information: logging information generated and collected 156 within a CDN or obtained from another CDN using the CDNI Logging 157 Interface. 159 CDNI Logging Field: an atomic element of information that can be 160 included in a CDNI Logging Record. The time an event/task started, 161 the IP address of an End User to whom content was delivered, and the 162 Uniform Resource Identifier (URI) of the content delivered, are 163 examples of CDNI Logging fields. 165 CDNI Logging Record: an information record providing information 166 about a specific event. This comprises a collection of CDNI Logging 167 fields. 169 CDNI Logging File: a file containing CDNI Logging Records, as well as 170 additional information facilitating the processing of the CDNI 171 Logging Records. 173 CDN Reporting: the process of providing the relevant information that 174 will be used to create a formatted content delivery report provided 175 to the CSP in deferred time. Such information typically includes 176 aggregated data that can cover a large period of time (e.g., from 177 hours to several months). Uses of Reporting include the collection 178 of charging data related to CDN services and the computation of Key 179 Performance Indicators (KPIs). 181 CDN Monitoring: the process of providing or displaying content 182 delivery information in a timely fashion with respect to the 183 corresponding deliveries. Monitoring typically includes visibility 184 of the deliveries in progress for service operation purposes. It 185 presents a view of the global health of the services as well as 186 information on usage and performance, for network services 187 supervision and operation management. In particular, monitoring data 188 can be used to generate alarms. 190 1.2. Requirements Language 192 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 193 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 194 "OPTIONAL" in this document are to be interpreted as described in RFC 195 2119 [RFC2119]. 197 2. CDNI Logging Reference Model 199 2.1. CDNI Logging interactions 201 The CDNI logging reference model between a given uCDN and a given 202 dCDN involves the following interactions: 204 o customization by the uCDN of the CDNI Logging information to be 205 provided by the dCDN to the uCDN (e.g., control of which CDNI 206 Logging fields are to be communicated to the uCDN for a given task 207 performed by the dCDN or control of which types of events are to 208 be logged). The dCDN takes into account this CDNI Logging 209 customization information to determine what Logging information to 210 provide to the uCDN, but it may, or may not, take into account 211 this CDNI Logging customization information to influence what CDN 212 logging information is to be generated and collected within the 213 dCDN (e.g., even if the uCDN requests a restricted subset of the 214 logging information, the dCDN may elect to generate a broader set 215 of logging information). The mechanism to support the 216 customization by the uCDN of CDNI Logging information is outside 217 the scope of this document and left for further study. Until such 218 a mechanism is available, the uCDN and dCDN are expected to agree 219 off-line on what exact set of CDNI Logging information is to be 220 provided by the dCDN to the uCDN, and to rely on management plane 221 actions to configure the CDNI Logging functions in the dCDN to 222 generate this information set and in the uCDN to expect this 223 information set. 225 o generation and collection by the dCDN of the intra-CDN Logging 226 information related to the completion of any task performed by the 227 dCDN on behalf of the uCDN (e.g., delivery of the content to an 228 End User) or related to events happening in the dCDN that are 229 relevant to the uCDN (e.g., failures or unavailability in dCDN). 230 This takes place within the dCDN and does not directly involve 231 CDNI interfaces. 233 o communication by the dCDN to the uCDN of the Logging information 234 collected by the dCDN relevant to the uCDN. This is supported by 235 the CDNI Logging interface and in the scope of the present 236 document. For example, the uCDN may use this Logging information 237 to charge the CSP, to perform analytics and monitoring for 238 operational reasons, to provide analytics and monitoring views on 239 its content delivery to the CSP or to perform trouble-shooting. 240 This document exclusively specifies non-real-time exchange of 241 Logging information. Closer to real-time exchange of Logging 242 information (say sub-minute or sub-second) is outside the scope of 243 the present document and left for further study. This document 244 exclusively specifies exchange of Logging information related to 245 content delivery. Exchange of Logging information related to 246 operational events (e.g., dCDN request routing function 247 unavailable, content acquisition failure by dCDN) for audit or 248 operational reactive adjustments by uCDN is outside the scope of 249 the present document and left for further study. 251 o customization by the dCDN of the CDNI Logging information to be 252 provided by the uCDN on behalf of the dCDN. The mechanism to 253 support the customization by the dCDN of CDNI Logging information 254 is outside the scope of this document and left for further study. 256 o generation and collection by the uCDN of Intra-CDN Logging 257 information related to the completion of any task performed by the 258 uCDN on behalf of the dCDN (e.g., serving of content by uCDN to 259 dCDN for acquisition purposes by dCDN) or related to events 260 happening in the uCDN that are relevant to the dCDN. This takes 261 place within the uCDN and does not directly involve CDNI 262 interfaces. 264 o communication by the uCDN to the dCDN of the Logging information 265 collected by the uCDN relevant to the dCDN. For example, the dCDN 266 might potentially benefit from this information for security 267 auditing or content acquisition troubleshooting. This is outside 268 the scope of this document and left for further study. 270 Figure 1 provides an example of CDNI Logging interactions (focusing 271 only on the interactions that are in the scope of this document) in a 272 particular scenario where four CDNs are involved in the delivery of 273 content from a given CSP: the uCDN has a CDNI interconnection with 274 dCDN-1 and dCDN-2. In turn, dCDN-2 has a CDNI interconnection with 275 dCDN-3, where dCDN-2 is acting as an upstream CDN relative to dCDN-3. 276 In this example, uCDN, dCDN-1, dCDN-2 and dCDN-3 all participate in 277 the delivery of content for the CSP. In this example, the CDNI 278 Logging interface enables the uCDN to obtain Logging information from 279 all the dCDNs involved in the delivery. In the example, the uCDN 280 uses the Logging information: 282 o to analyze the performance of the delivery performed by the dCDNs 283 and to adjust its operations after the fact (e.g., request 284 routing) as appropriate, 286 o to provide (non-real-time) reporting and monitoring information to 287 the CSP. 289 For instance, the uCDN merges Logging information, extracts relevant 290 KPIs, and presents a formatted report to the CSP, in addition to a 291 bill for the content delivered by uCDN itself or by its dCDNs on the 292 CSP's behalf. The uCDN may also provide Logging information as raw 293 log files to the CSP, so that the CSP can use its own logging 294 analysis tools. 296 +-----+ 297 | CSP | 298 +-----+ 299 ^ Reporting and monitoring data 300 * Billing 301 ,--*--. 302 Logging ,-' `-. 303 Data =>( uCDN )<= Logging 304 // `-. _,-' \\ Data 305 || `-'-'-' || 306 ,-----. ,-----. 307 ,-' `-. ,-' `-. 308 ( dCDN-1 ) ( dCDN-2 )<== Logging 309 `-. ,-' `-. _,-' \\ Data 310 `--'--' `--'-' || 311 ,-----. 312 ,' `-. 313 ( dCDN-3 ) 314 `. ,-' 315 `--'--' 317 ===> CDNI Logging Interface 318 ***> outside the scope of CDNI 320 Figure 1: Interactions in CDNI Logging Reference Model 322 A downstream CDN relative to uCDN (e.g., dCDN-2) integrates the 323 relevant Logging information obtained from its own downstream CDNs 324 (i.e., dCDN-3) in the Logging information that it provides to the 325 uCDN, so that the uCDN ultimately obtains all Logging information 326 relevant to a CSP for which it acts as the authoritative CDN. Such 327 aggregation is further discussed in Section 3.7. 329 Note that the format of Logging information that a CDN provides over 330 the CDNI interface might be different from the one that the CDN uses 331 internally. In this case, the CDN needs to reformat the Logging 332 information before it provides this information to the other CDN over 333 the CDNI Logging interface. Similarly, a CDN might reformat the 334 Logging information that it receives over the CDNI Logging interface 335 before injecting it into its log-consuming applications or before 336 providing some of this Logging information to the CSP. Such 337 reformatting operations introduce latency in the logging distribution 338 chain and introduce a processing burden. Therefore, there are 339 benefits in specifying CDNI Logging formats that are suitable for use 340 inside CDNs and also are close to the intra-CDN Logging formats 341 commonly used in CDNs today. 343 2.2. Overall Logging Chain 345 This section discusses the overall logging chain within and across 346 CDNs to clarify how CDN Logging information is expected to fit in 347 this overall chain. Figure 2 illustrates the overall logging chain 348 within the dCDN, across CDNs using the CDNI Logging interface and 349 within the uCDN. Note that the logging chain illustrated in the 350 Figure is obviously only an example and varies depending on the 351 specific environments. For example, there may be more or fewer 352 instantiations of each entity (e.g., there may be 4 Log consuming 353 applications in a given CDN). As another example, there may be one 354 instance of Rectification process per Log Consuming Application 355 instead of a shared one. 357 Log Consuming Log Consuming 358 App App 359 ^ ^ 360 | | 361 Rectification---------- 362 ^ 363 | 364 Filtering 365 ^ 366 | 367 Collection 368 ^ ^ 369 | | 370 | Generation 371 | 372 | uCDN 373 CDNI Logging --------------------------------------------------- 374 exchange dCDN 375 ^ 376 | Log Consuming Log Consuming 377 | App App 378 | ^ ^ 379 | | | 380 Rectification Rectification--------- 381 ^ ^ 382 | | 383 Filtering 384 ^ 385 | 386 Collection 387 ^ ^ 388 | | 389 Generation Generation 391 Figure 2: CDNI Logging in the overall Logging Chain 393 The following subsections describe each of the processes potentially 394 involved in the logging chain of Figure 2. 396 2.2.1. Logging Generation and During-Generation Aggregation 398 CDNs typically generate Logging information for all significant task 399 completions, events, and failures. Logging information is typically 400 generated by many devices in the CDN including the surrogates, the 401 request routing system, and the control system. 403 The amount of Logging information generated can be huge. Therefore, 404 during contract negotiations, interconnected CDNs often agree on a 405 retention duration for Logging information, and/or potentially on a 406 maximum volume of Logging information that the dCDN ought to keep. 407 If this volume is exceeded, the dCDN is expected to alert the uCDN 408 but may not keep more Logging information for the considered time 409 period. In addition, CDNs may aggregate Logging information and 410 transmit only summaries for some categories of operations instead of 411 the full Logging information. Note that such aggregation leads to an 412 information loss, which may be problematic for some usages of the 413 Logging information (e.g., debugging). 415 [RFC6983] discusses logging for HTTP Adaptive Streaming (HAS). In 416 accordance with the recommendations articulated there, it is expected 417 that a surrogate will generate separate Logging information for 418 delivery of each chunk of HAS content. This ensures that separate 419 Logging information can then be provided to interconnected CDNs over 420 the CDNI Logging interface. Still in line with the recommendations 421 of [RFC6983], the Logging information for per-chunk delivery may 422 include some information (a Content Collection IDentifier and a 423 Session IDentifier) intended to facilitate subsequent post-generation 424 aggregation of per-chunk logs into per-session logs. Note that a CDN 425 may also elect to generate aggregate per-session logs when performing 426 HAS delivery, but this needs to be in addition to, and not instead 427 of, the per-chunk delivery logs. We note that aggregate per-session 428 logs for HAS delivery are for further study and outside the scope of 429 this document. 431 2.2.2. Logging Collection 433 This is the process that continuously collects Logging information 434 generated by the log-generating entities within a CDN. 436 In a CDNI environment, in addition to collecting Logging information 437 from log-generating entities within the local CDN, the Collection 438 process also collects Logging information provided by another CDN, or 439 other CDNs, through the CDNI Logging interface. This is illustrated 440 in Figure 2 where we see that the Collection process of the uCDN 441 collects Logging information from log-generating entities within the 442 uCDN as well as Logging information coming from the dCDNs through the 443 CDNI Logging interface. 445 2.2.3. Logging Filtering 447 A CDN may be required to only present different subsets of the whole 448 Logging information collected to various log-consuming applications. 449 This is achieved by the Filtering process. 451 In particular, the Filtering process can also filter the right subset 452 of Logging information that needs to be provided to a given 453 interconnected CDN. For example, the filtering process in the dCDN 454 can be used to ensure that only the Logging information related to 455 tasks performed on behalf of a given uCDN are made available to that 456 uCDN (thereby filtering out all the Logging information related to 457 deliveries by the dCDN of content for its own CSPs). Similarly, the 458 Filtering process may filter or partially mask some fields, for 459 example, to protect End Users' privacy when communicating CDNI 460 Logging information to another CDN. Filtering of Logging information 461 prior to communication of this information to other CDNs via the CDNI 462 Logging interface requires that the downstream CDN can recognize the 463 subset of Logging information that relate to each interconnected CDN. 465 The CDN will also filter some internal scope information such as 466 information related to its internal alarms (security, failures, load, 467 etc). 469 In some use cases described in [RFC6770], the interconnected CDNs do 470 not want to disclose details on their internal topology. The 471 filtering process can then also filter confidential data on the 472 dCDNs' topology (number of servers, location, etc.). In particular, 473 information about the requests served by each Surrogate may be 474 confidential. Therefore, the Logging information needs to be 475 protected so that data such as Surrogates' hostnames are not 476 disclosed to the uCDN. In the "Inter-Affiliates Interconnection" use 477 case, this information may be disclosed to the uCDN because both the 478 dCDN and the uCDN are operated by entities of the same group. 480 2.2.4. Logging Rectification and Post-Generation Aggregation 482 If Logging information is generated periodically, it is important 483 that the sessions that start in one Logging period and end in another 484 are correctly reported. If they are reported in the starting period, 485 then the Logging information of this period will be available only 486 after the end of the session, which delays the Logging information 487 generation. A simple approach is to provide the complete Logging 488 Record for a session in the Logging Period of the session end. 490 A Logging rectification/update mechanism could be useful to reach a 491 good trade-off between the Logging information generation delay and 492 the Logging information accuracy. 494 In the presence of HAS, some log-consuming applications can benefit 495 from aggregate per-session logs. For example, for analytics, per- 496 session logs allow display of session-related trends which are much 497 more meaningful for some types of analysis than chunk-related trends. 498 In the case where aggregate logs have been generated directly by the 499 log-generating entities, those can be used by the applications. In 500 the case where aggregate logs have not been generated, the 501 Rectification process can be extended with a Post-Generation 502 Aggregation process that generates per-session logs from the per- 503 chunk logs, possibly leveraging the information included in the per- 504 chunk logs for that purpose (Content Collection IDentifier and a 505 Session IDentifier). However, in accordance with [RFC6983], this 506 document does not define exchange of such aggregate logs on the CDNI 507 Logging interface. We note that this is for further study and 508 outside the scope of this document. 510 2.2.5. Log-Consuming Applications 512 2.2.5.1. Maintenance/Debugging 514 Logging information is useful to permit the detection (and limit the 515 risk) of content delivery failures. In particular, Logging 516 information facilitates the detection of configuration issues. 518 To detect faults, Logging information needs to report success and 519 failure of CDN delivery operations. The uCDN can summarize such 520 information into KPIs. For instance, Logging information needs to 521 allow the computation of the number of times, during a given time 522 period, that content delivery related to a specific service succeeds/ 523 fails. 525 Logging information enables the CDN providers to identify and 526 troubleshoot performance degradations. In particular, Logging 527 information enables tracking of traffic data (e.g., the amount of 528 traffic that has been forwarded by a dCDN on behalf of an uCDN over a 529 given period of time), which is particularly useful for CDN and 530 network planning operations. 532 Some of these maintenance and debugging applications only require 533 aggregate logging information highly compatible with use of 534 anonymization of IP addresses (as supported by the present document 535 and specified in the definition of the c-groupid field under 536 Section 3.4.1). However, in some situations, it may be useful, where 537 compatible with privacy protection, to access some CDNI Logging 538 Records containing full non-anonymized IP addresses. This is allowed 539 in the definition of the c-groupid (under Section 3.4.1), with very 540 significant privacy protection limitations that are discussed in the 541 definition of the c-groupid field. For example, this may be useful 542 for detailed fault tracking of a particular end user content delivery 543 issue. Where there is a hard requirement by uCDN or CSP to associate 544 a given enduser to individual CDNI Logging Records (e.g., to allow 545 a-posteriori analysis of individual delivery for example in 546 situations of performance-based penalties), instead of using 547 aggregates containing a single client as discussed in the c-groupid 548 field definition, an alternate approach is to ensure that a client 549 identifier is embedded in the request fields that can be logged in a 550 CDNI Logging Record (for example by including the client identifier 551 in the URI query string or in a HTTP Header). That latter approach 552 offers two strong benefits: first, the aggregate inside the c-groupid 553 can contain more than one client, thereby ensuring stronger privacy 554 protection; second, it allows a reliable identification of the client 555 while IP address does not in many situations (e.g., behind NAT, where 556 dynamic IP addresses are used and reused,...). However, care SHOULD 557 be taken that the client identifiers exposed in other fields of the 558 CDNI Records cannot themselves be linked back to actual users. 560 2.2.5.2. Accounting 562 Logging information is essential for accounting, to permit inter-CDN 563 billing and CSP billing by uCDNs. For instance, Logging information 564 provided by dCDNs enables the uCDN to compute the total amount of 565 traffic delivered by every dCDN for a particular Content Provider, as 566 well as, the associated bandwidth usage (e.g., peak, 95th 567 percentile), and the maximum number of simultaneous sessions over a 568 given period of time. 570 2.2.5.3. Analytics and Reporting 572 The goals of analytics include gathering any relevant information in 573 order to be able to develop statistics on content download, analyze 574 user behavior, and monitor the performance and quality of content 575 delivery. For instance, Logging information enables the CDN 576 providers to report on content consumption (e.g., delivered sessions 577 per content) in a specific geographic area. 579 The goal of reporting is to gather any relevant information to 580 monitor the performance and quality of content delivery and allow 581 detection of delivery issues. For instance, reporting could track 582 the average delivery throughput experienced by End Users in a given 583 region for a specific CSP or content set over a period of time. 585 2.2.5.4. Content Protection 587 The goal of content protection is to prevent and monitor unauthorized 588 access, misuse, modification, and denial of access to a content. A 589 set of information is logged in a CDN for security purposes. In 590 particular, a record of access to content is usually collected to 591 permit the CSP to detect infringements of content delivery policies 592 and other abnormal End User behaviors. 594 2.2.5.5. Notions common to multiple Log Consuming Applications 596 2.2.5.5.1. Logging Information Views 598 Within a given log-consuming application, different views may be 599 provided to different users depending on privacy, business, and 600 scalability constraints. 602 For example, an analytics tool run by the uCDN can provide one view 603 to an uCDN operator that exploits all the Logging information 604 available to the uCDN, while the tool may provide a different view to 605 each CSP exploiting only the Logging information related to the 606 content of the given CSP. 608 As another example, maintenance and debugging tools may provide 609 different views to different CDN operators, based on their 610 operational role. 612 2.2.5.5.2. Key Performance Indicators (KPIs) 614 This section presents, for explanatory purposes, a non-exhaustive 615 list of Key Performance Indicators (KPIs) that can be extracted/ 616 produced from logs. 618 Multiple log-consuming applications, such as analytics, monitoring, 619 and maintenance applications, often compute and track such KPIs. 621 In a CDNI environment, depending on the situation, these KPIs may be 622 computed by the uCDN or by the dCDN. But it is usually the uCDN that 623 computes KPIs, because the uCDN and dCDN may have different 624 definitions of the KPIs and the computation of some KPIs requires a 625 vision of all the deliveries performed by the uCDN and all its dCDNs. 627 Here is a list of important examples of KPIs: 629 o Number of delivery requests received from End Users in a given 630 region for each piece of content, during a given period of time 631 (e.g., hour/day/week/month) 633 o Percentage of delivery successes/failures among the aforementioned 634 requests 636 o Number of failures listed by failure type (e.g., HTTP error code) 637 for requests received from End Users in a given region and for 638 each piece of content, during a given period of time (e.g., 639 hour/day/week/month) 641 o Number and cause of premature delivery termination for End Users 642 in a given region and for each piece of content, during a given 643 period of time (e.g., hour/day/week/month) 645 o Maximum and mean number of simultaneous sessions established by 646 End Users in a given region, for a given Content Provider, and 647 during a given period of time (e.g., hour/day/week/month) 649 o Volume of traffic delivered for sessions established by End Users 650 in a given region, for a given Content Provider, and during a 651 given period of time (e.g., hour/day/week/month) 653 o Maximum, mean, and minimum delivery throughput for sessions 654 established by End Users in a given region, for a given Content 655 Provider, and during a given period of time (e.g., hour/day/week/ 656 month) 658 o Cache-hit and byte-hit ratios for requests received from End Users 659 in a given region for each piece of content, during a given period 660 of time (e.g., hour/day/week/month) 662 o Top 10 most popularly requested contents (during a given day/week/ 663 month) 665 o Terminal type (mobile, PC, STB, if this information can be 666 acquired from the browser type inferred from the User Agent 667 string, for example). 669 Additional KPIs can be computed from other sources of information 670 than the Logging information, for instance, data collected by a 671 content portal or by specific client-side application programming 672 interfaces. Such KPIs are out of scope for the present document. 674 The KPIs used depend strongly on the considered log-consuming 675 application -- the CDN operator may be interested in different 676 metrics than the CSP is. In particular, CDN operators are often 677 interested in delivery and acquisition performance KPIs, information 678 related to Surrogates' performance, caching information to evaluate 679 the cache-hit ratio, information about the delivered file size to 680 compute the volume of content delivered during peak hour, etc. 682 Some of the KPIs, for instance those providing an instantaneous 683 vision of the active sessions for a given CSP's content, are useful 684 essentially if they are provided in a timely manner. By contrast, 685 some other KPIs, such as those averaged on a long period of time, can 686 be provided in non-real-time. 688 3. CDNI Logging File 690 3.1. Rules 692 This specification uses the Augmented Backus-Naur Form (ABNF) 693 notation and core rules of [RFC5234]. In particular, the present 694 document uses the following rules from [RFC5234]: 696 CR = %x0D ; carriage return 698 ALPHA = %x41-5A / %x61-7A ; A-Z / a-z 700 DIGIT = %x30-39 ; 0-9 702 DQUOTE = %x22 ; " (Double Quote) 704 CRLF = CR LF ; Internet standard newline 706 HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F" 708 HTAB = %x09 ; horizontal tab 710 LF = %x0A ; linefeed 712 VCHAR = %x21-7E ; visible (printing) characters 714 OCTET = %x00-FF ; 8 bits of data 716 The present document also uses the following rules from [RFC3986]: 718 host = as specified in section 3.2.2 of [RFC3986]. 720 IPv4address = as specified in section 3.2.2 of [RFC3986]. 722 IPv6address = as specified in section 3.2.2 of [RFC3986]. 724 partial-time = as specified in [RFC3339]. 726 The present document also defines the following additional rules: 728 ADDRESS = IPv4address / IPv6address 730 ALPHANUM = ALPHA / DIGIT 732 DATE = 4DIGIT "-" 2DIGIT "-" 2DIGIT 734 ; Dates are encoded as "full-date" specified in [RFC3339]. 736 DEC = 1*DIGIT ["." 1*DIGIT] 738 NAMEFORMAT = ALPHANUM *(ALPHANUM / "_" / "-") 740 QSTRING = DQUOTE *(NDQUOTE / PCT-ENCODED) DQUOTE 742 NDQUOTE = %x20-21 / %x23-24 / %x26-7E / UTF8-2 / UTF8-3 / UTF8-4 744 ; whereby a DQUOTE is conveyed inside a QSTRING unambiguously 745 by escaping it with PCT-ENCODED. 747 PCT-ENCODED = "%" HEXDIG HEXDIG 749 ; percent encoding is used for escaping octets that might be 750 possible in HTTP headers such as bare CR, bare LF, CR LF, HTAB, 751 SP or null. These octets are rendered with percent encoding in 752 ABNF as specified by [RFC3986] in order to avoid considering 753 them as separators for the logging records. 755 NHTABSTRING = 1*(SP / VCHAR) 757 TIME = partial-time 759 USER-COMMENT = * (SP / VCHAR / UTF8-2 / UTF8-3 / UTF8-4) 761 3.2. CDNI Logging File Structure 763 As defined in Section 1.1: a CDNI Logging Field is as an atomic 764 logging information element, a CDNI Logging Record is a collection of 765 CDNI Logging fields containing all logging information corresponding 766 to a single logging event, and a CDNI Logging File contains a 767 collection of CDNI Logging Records. This structure is illustrated in 768 Figure 3. The use of a file structure for transfer of CDNI Logging 769 information is selected since this is the most common practise today 770 for exchange of logging information within and across CDNs. 772 +----------------------------------------------------------+ 773 |CDNI Logging File | 774 | | 775 | #Directive 1 | 776 | #Directive 2 | 777 | ... | 778 | #Directive P | 779 | | 780 | +------------------------------------------------------+ | 781 | |CDNI Logging Record 1 | | 782 | | +-------------+ +-------------+ +-------------+ | | 783 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 784 | | | Field 1 | | Field 2 | | Field N | | | 785 | | +-------------+ +-------------+ +-------------+ | | 786 | +------------------------------------------------------+ | 787 | | 788 | +------------------------------------------------------+ | 789 | |CDNI Logging Record 2 | | 790 | | +-------------+ +-------------+ +-------------+ | | 791 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 792 | | | Field 1 | | Field 2 | | Field N | | | 793 | | +-------------+ +-------------+ +-------------+ | | 794 | +------------------------------------------------------+ | 795 | | 796 | ... | 797 | | 798 | #Directive P+1 | 799 | | 800 | ... | 801 | | 802 | +------------------------------------------------------+ | 803 | |CDNI Logging Record M | | 804 | | +-------------+ +-------------+ +-------------+ | | 805 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 806 | | | Field 1 | | Field 2 | | Field N | | | 807 | | +-------------+ +-------------+ +-------------+ | | 808 | +------------------------------------------------------+ | 809 | | 810 | | 811 | #Directive P+Q | 812 +----------------------------------------------------------+ 814 Figure 3: Structure of Logging Files 816 The CDNI Logging File format is inspired from the W3C Extended Log 817 File Format [ELF]. However, it is fully specified by the present 818 document. Where the present document differs from the W3C Extended 819 Log File Format, an implementation of the CDNI Logging interface MUST 820 comply with the present document. The W3C Extended Log File Format 821 was used as a starting point, reused where possible and expanded when 822 necessary. 824 Using a format that resembles the W3C Extended Log File Format is 825 intended to keep CDNI logging format close to the intra-CDN Logging 826 information format commonly used in CDNs today, thereby minimizing 827 systematic translation at CDN/CDNI boundary. 829 A CDNI Logging File MUST contain a sequence of lines containing US- 830 ASCII characters [CHAR_SET] terminated by CRLF. Each line of a CDNI 831 Logging File MUST contain either a directive or a CDNI Logging 832 Record. 834 Directives record information about the CDNI Logging process itself. 835 Lines containing directives MUST begin with the "#" character. 836 Directives are specified in Section 3.3. 838 Logging Records provide actual details of the logged event. Logging 839 Records are specified in Section 3.4. 841 The CDNI Logging File has a specific structure. It always starts 842 with a directive line and the first directive it contains MUST be the 843 version. 845 The directive lines form together a group that contains at least one 846 directive line. Each directives group is followed by a group of 847 logging records. The records group contains zero or more actual 848 logging record lines about the event being logged. A record line 849 consists of the values corresponding to all or a subset of the 850 possible Logging fields defined within the scope of the record-type 851 directive. These values MUST appear in the order defined by the 852 fields directive. 854 Note that future extensions MUST be compliant with the previous 855 description. The following examples depict the structure of a 856 CDNILOGFILE as defined currently by the record-type 857 "cdni_http_request_v1." 859 DIRLINE = "#" directive CRLF 861 DIRGROUP = 1*DIRLINE 863 RECLINE = 866 RECGROUP = *RECLINE 868 CDNILOGFILE = 1*(DIRGROUP RECGROUP) 870 All directive names and field names defined in the logging file are 871 case-insensitive as per the basic ABNF([RFC5234]). 873 3.3. CDNI Logging Directives 875 A CDNI Logging directive line contains the directive name followed by 876 ":" HTAB and the directive value. 878 Directive names MUST be of the format NAMEFORMAT. All directive 879 names MUST be registered in the CDNI Logging Directives Names 880 registry. Unknown directives MUST be ignored. Directive values can 881 have various formats. All possible directive values for the record- 882 type "cdni_http_request_v1" are further detailed in this section. 884 The following example shows the structure of a directive and 885 enumerates strictly the directive values presently defined in the 886 version "CDNI/1.0" of the CDNI Logging File. 888 directive = DIRNAME ":" HTAB DIRVAL 890 DIRNAME = NAMEFORMAT 892 DIRVAL = NHTABSTRING / QSTRING / host / USER-COMMENT / FIENAME * 893 (HTAB FIENAME) / 64HEXDIG 895 An implementation of the CDNI Logging interface MUST support all of 896 the following directives, listed below by their directive name: 898 o version: 900 * format: NHTABSTRING 902 * directive value: indicates the version of the CDNI Logging File 903 format. The entity transmitting a CDNI Logging File as per the 904 present document MUST set the value to "CDNI/1.0". In the 905 future, other versions of CDNI Logging File might be specified; 906 those would use a value different to "CDNI/1.0" allowing the 907 entity receiving the CDNI Logging File to identify the 908 corresponding version. 910 * occurrence: there MUST be one and only one instance of this 911 directive per CDNI Logging File. It MUST be the first line of 912 the CDNI Logging File. 914 * example: "version: HTAB CDNI/1.0". 916 o UUID: 918 * format: NHTABSTRING 920 * directive value: this a Uniform Resource Name (URN) from the 921 Universally Unique IDentifier (UUID) URN namespace specified in 922 [RFC4122]). The UUID contained in the URN uniquely identifies 923 the CDNI Logging File. 925 * occurrence: there MUST be one and only one instance of this 926 directive per CDNI Logging File. 928 * example: "UUID: HTAB NHTABSTRING". 930 o claimed-origin: 932 * format: host 934 * directive value: this contains the claimed identification of 935 the entity transmitting the CDNI Logging File (e.g., the host 936 in a dCDN supporting the CDNI Logging interface) or the entity 937 responsible for transmitting the CDNI Logging File (e.g., the 938 dCDN). 940 * occurrence: there MUST be zero or exactly one instance of this 941 directive per CDNI Logging File. This directive MAY be 942 included by the dCDN. It MUST NOT be included or modified by 943 the uCDN. 945 * example: "claimed-origin: HTAB host". 947 o established-origin: 949 * format: host 951 * directive value: this contains the identification, as 952 established by the entity receiving the CDNI Logging File, of 953 the entity transmitting the CDNI Logging File (e.g., the host 954 in a dCDN supporting the CDNI Logging interface) or the entity 955 responsible for transmitting the CDNI Logging File (e.g., the 956 dCDN). 958 * occurrence: there MUST be zero or exactly one instance of this 959 directive per CDNI Logging File. This directive MAY be added 960 by the uCDN (e.g., before storing the CDNI Logging File). It 961 MUST NOT be included by the dCDN. The mechanisms used by the 962 uCDN to establish and validate the entity responsible for the 963 CDNI Logging File is outside the scope of the present document. 964 We observe that, in particular, this may be achieved through 965 authentication mechanisms that are part of the transport layer 966 of the CDNI Logging File pull mechanism (Section 4.2). 968 * ABNF example: "established-origin: HTAB host". 970 o remark: 972 * format: USER-COMMENT 974 * directive value: this contains comment information. Data 975 contained in this field is to be ignored by analysis tools. 977 * occurrence: there MAY be zero, one or any number of instance of 978 this directive per CDNI Logging File. 980 * example: "remark: HTAB USER-COMMENT". 982 o record-type: 984 * format: NAMEFORMAT 986 * directive value: indicates the type of the CDNI Logging Records 987 that follow this directive, until another record-type directive 988 (or the end of the CDNI Logging File). This can be any CDNI 989 Logging Record type registered in the CDNI Logging Record-types 990 registry (Section 6.3). For example this may be 991 "cdni_http_request_v1" as specified in Section 3.4.1. 993 * occurrence: there MUST be at least one instance of this 994 directive per CDNI Logging File. The first instance of this 995 directive MUST precede a fields directive and MUST precede all 996 CDNI Logging Records. 998 * example: "record-type: HTAB cdni_http_request_v1". 1000 o fields: 1002 * format: FIENAME *(HTAB FIENAME) ; where FIENAME can take any 1003 CDNI Logging field name registered in the CDNI Logging Field 1004 Names registry (Section 6.4) that is valid for the record type 1005 specified in the record-type directive. 1007 * directive value: this lists the names of all the fields for 1008 which a value is to appear in the CDNI Logging Records that 1009 follow the instance of this directive (until another instance 1010 of this directive). The names of the fields, as well as their 1011 occurrences, MUST comply with the corresponding rules specified 1012 in the document referenced in the CDNI Logging Record-types 1013 registry (Section 6.3) for the corresponding CDNI Logging 1014 record-type. 1016 * occurrence: there MUST be at least one instance of this 1017 directive per record-type directive. The first instance of 1018 this directive for a given record-type MUST appear before any 1019 CDNI Logging Record for this record-type. One situation where 1020 more than one instance of the fields directive can appear 1021 within a given CDNI Logging File, is when there is a change, in 1022 the middle of a fairly large logging period, in the agreement 1023 between the uCDN and the dCDN about the set of fields that are 1024 to be exchanged. The multiple occurrences allow records with 1025 the old set of fields and records with the new set of fields to 1026 be carried inside the same Logging File. 1028 * example: "fields: HTAB FIENAME * (HTAB FIENAME)". 1030 o SHA256-hash: 1032 * format: 64HEXDIG 1034 * directive value: This directive permits the detection of a 1035 corrupted CDNI Logging File. This can be useful, for instance, 1036 if a problem occurs on the filesystem of the dCDN Logging 1037 system and leads to a truncation of a logging file. The valid 1038 SHA256-hash value is included in this directive by the entity 1039 that transmits the CDNI Logging File. It MUST be computed by 1040 applying the SHA-256 ([RFC6234]) cryptographic hash function on 1041 the CDNI Logging File, including all the directives and logging 1042 records, up to the SHA256-hash directive itself, excluding the 1043 SHA256-hash directive itself. The SHA256-hash value MUST be 1044 represented as a US-ASCII encoded hexadecimal number, 64 digits 1045 long (representing a 256 bit hash value). The entity receiving 1046 the CDNI Logging File also computes in a similar way the 1047 SHA-256 hash on the received CDNI Logging File and compares 1048 this hash to the value of the SHA256-hash directive. If the 1049 two values are equal, then the received CDNI Logging File is to 1050 be considered non-corrupted. If the two values are different, 1051 the received CDNI Logging File is to be considered corrupted. 1052 The behavior of the entity that received a corrupted CDNI 1053 Logging File is outside the scope of this specification; we 1054 note that the entity MAY attempt to pull again the same CDNI 1055 Logging File from the transmitting entity. If the entity 1056 receiving a non-corrupted CDNI Logging File adds an 1057 established-origin directive, it MUST then recompute and update 1058 the SHA256-hash directive so it also protects the added 1059 established-origin directive. 1061 * occurrence: there MUST be zero or exactly one instance of this 1062 directive. There SHOULD be exactly one instance of this 1063 directive. One situation where that directive could be omitted 1064 is where integrity protection is already provided via another 1065 mechanism (for example if an integrity hash is associated to 1066 the CDNI Logging File out-of-band through the CDNI Logging Feed 1067 ( Section 4.1) leveraging ATOM extensions such as those 1068 proposed in [I-D.snell-atompub-link-extensions]. When present, 1069 the SHA256-hash field MUST be the last line of the CDNI Logging 1070 File. 1072 * example: "SHA256-hash: HTAB 64HEXDIG". 1074 An uCDN-side implementation of the CDNI Logging interface MUST reject 1075 a CDNI Logging File that does not comply with the occurrences 1076 specified above for each and every directive. For example, an uCDN- 1077 side implementation of the CDNI Logging interface receiving a CDNI 1078 Logging file with zero occurrence of the version directive, or with 1079 two occurrences of the SHA256-hash, MUST reject this CDNI Logging 1080 File. 1082 An entity receiving a CDNI Logging File with a value set to 1083 "CDNI/1.0" MUST process the CDNI Logging File as per the present 1084 document. An entity receiving a CDNI Logging File with a value set 1085 to a different value MUST process the CDNI Logging File as per the 1086 specification referenced in the CDNI Logging File version registry 1087 (see Section 6.1) if the implementation supports this specification 1088 and MUST reject the CDNI Logging File otherwise. 1090 3.4. CDNI Logging Records 1092 A CDNI Logging Record consists of a sequence of CDNI Logging fields 1093 relating to that single CDNI Logging Record. 1095 CDNI Logging fields MUST be separated by the "horizontal tabulation 1096 (HTAB)" character. 1098 To facilitate readability, a prefix scheme is used for CDNI Logging 1099 field names in a similar way to the one used in W3C Extended Log File 1100 Format [ELF]. The semantics of the prefix in the present document 1101 is: 1103 o "c-" refers to the User Agent that issues the request (corresponds 1104 to the "client" of W3C Extended Log Format) 1106 o "d-" refers to the dCDN (relative to a given CDN acting as an 1107 uCDN) 1109 o "s-" refers to the dCDN Surrogate that serves the request 1110 (corresponds to the "server" of W3C Extended Log Format) 1112 o "u-" refers to the uCDN (relative to a given CDN acting as a dCDN) 1114 o "cs-" refers to communication from the User Agent towards the dCDN 1115 Surrogate 1117 o "sc-" refers to communication from the dCDN Surrogate towards the 1118 User Agent 1120 An implementation of the CDNI Logging interface as per the present 1121 specification MUST support the CDNI HTTP Request Logging Record as 1122 specified in Section 3.4.1. 1124 A CDNI Logging Record contains the corresponding values for the 1125 fields that are enumerated in the last fields directive before the 1126 current log line. Note that the order in which the field values 1127 appear is dictated by the order of the fields names in the fields 1128 directive. There SHOULD be no dependency between the various fields 1129 values. 1131 3.4.1. HTTP Request Logging Record 1133 This section defines the CDNI Logging Record of record-type 1134 "cdni_http_request_v1". It is applicable to content delivery 1135 performed by the dCDN using HTTP/1.0([RFC1945]), 1136 HTTP/1.1([RFC7230],[RFC7231], [RFC7232], [RFC7233], [RFC7234], 1137 [RFC7235]) or HTTPS ([RFC2818], [RFC7230]). We observe that, in the 1138 case of HTTPS delivery, there may be value in logging additional 1139 information specific to the operation of HTTP over TLS and we note 1140 that this is outside the scope of the present document and may be 1141 addressed in a future document defining another CDNI Logging Record 1142 or another version of the HTTP Request Logging Record. 1144 The "cdni_http_request_v1" record-type is also expected to be 1145 applicable to HTTP/2 [RFC7540] since a fundamental design tenet of 1146 HTTP/2 is to preserve the HTTP/1.1 semantics. We observe that, in 1147 the case of HTTP/2 delivery, there may be value in logging additional 1148 information specific to the additional functionality of HTTP/2 (e.g., 1149 information related to connection identification, to stream 1150 identification, to stream priority and to flow control). We note 1151 that such additional information is outside the scope of the present 1152 document and may be addressed in a future document defining another 1153 CDNI Logging Record or another version of the HTTP Request Logging 1154 Record. 1156 The "cdni_http_request_v1" record-type contains the following CDNI 1157 Logging fields, listed by their field name: 1159 o date: 1161 * format: DATE 1163 * field value: the date at which the processing of request 1164 completed on the Surrogate. 1166 * occurrence: there MUST be one and only one instance of this 1167 field. 1169 o time: 1171 * format: TIME 1173 * field value: the time, which MUST be expressed in Coordinated 1174 Universal Time (UTC), at which the processing of request 1175 completed on the Surrogate. 1177 * occurrence: there MUST be one and only one instance of this 1178 field. 1180 o time-taken: 1182 * format: DEC 1184 * field value: decimal value of the duration, in seconds, between 1185 the start of the processing of the request and the completion 1186 of the request processing (e.g., completion of delivery) by the 1187 Surrogate. 1189 * occurrence: there MUST be one and only one instance of this 1190 field. 1192 o c-groupid: 1194 * format: NHTABSTRING 1196 * field value: an opaque identifier for an aggregate set of 1197 clients, derived from the client IPv4 or IPv6 address in the 1198 request received by the Surrogate and/or other network-level 1199 identifying information. The c-groupid serves to group clients 1200 into aggregates. Example aggregates include civil geolocation 1201 information (the country, second-level administrative division, 1202 or postal code from which the client is presumed to make the 1203 request based on a geolocation database lookup) or network 1204 topological information (e.g., the BGP AS number announcing the 1205 prefix containing the address). The c-groupid MAY be 1206 structured e.g., US/TN/MEM/38138. Agreement between the dCDN 1207 and the uCDN on a mapping between IPv4 and IPv6 addresses and 1208 aggregates is presumed to occur out-of-band. The aggregation 1209 mapping SHOULD be chosen such that each aggregate contains more 1210 than one client. 1212 + When the aggregate is chosen so that it contains a single 1213 client (e.g., to allow more detailed analytics, or to allow 1214 a-posteriori analysis of individual delivery for example in 1215 situations of performance-based penalties) the c-groupid MAY 1216 be structured where some elements identify aggregates and 1217 one element identifies the client, e.g., US/TN/ 1218 MEM/38138/43a5bdd6-95c4-4d62-be65-7410df0021e2. In that 1219 case: 1221 - the element identifying the client SHOULD be 1222 algorithmically generated (from the client IPv4 or IPv6 1223 address in the request received by the Surrogate and/or 1224 other network-level identifying information) in a way 1225 that SHOULD NOT be linkable back to the global addressing 1226 context and that SHOULD vary over time (to offer 1227 protection against long term attacks). 1229 - It is RECOMMENDED that the mapping varies at least once 1230 every 24 hours. 1232 - The algorithmic mapping and variation over time MAY allow 1233 the uCDN (with the knowledge of the algorithm and time 1234 variation and associated attributes and keys) to 1235 reconstruct the actual client IPv4 or IPv6 address and/or 1236 other network-level identifying information when required 1237 (e.g., to allow a-posteriori analysis of individual 1238 delivery for example in situations of performance-based 1239 penalties). However, these enduser addresses SHOULD only 1240 be reconstructed on-demand and the CDNI Logging File 1241 SHOULD only be stored with the anonymised c-groupid 1242 value. 1244 - Allowing reconstruction of client address information 1245 carries with it grave risks to end-user privacy. Since 1246 the c-groupid is in this case equivalent in 1247 identification power to a client IP address, its use may 1248 be restricted by regulation or law as personally 1249 identifiable information. For this reason, such use is 1250 NOT RECOMMENDED. 1252 - One method for mapping that MAY be be supported by 1253 implementations relies on a symmetric key that is known 1254 only to the uCDN and dCDN and HMAC-based Extract-and- 1255 Expand Key Derivation Function (HKDF) key derivation 1256 ([RFC5869]), as will be used in TLS 1.3 1257 ([I-D.ietf-tls-rfc5246-bis]). When that method is used: 1259 o The uCDN and dCDN need to agree on the "salt" and 1260 "input keying material", as described in Section 2.2 1261 of [RFC5869] and the initial "info" parameter (which 1262 could be something like the business names of the two 1263 organizations in UTF-8, concatenated), as described in 1264 Section 2.3 of [RFC5869]. The hash SHOULD be either 1265 SHA-2 or SHA-3 [SHA-3] and the encryption algorithm 1266 SHOULD be 128-bit AES [AES] in Galois Counter Mode 1267 (GCM) [GCM] (AES-GCM) or better. The PRK SHOULD be 1268 chosen by both parties contributing alternate random 1269 bytes until sufficient length exists. After the 1270 initial setup, client-information can be encrypted 1271 using the key generated by the "expand" step of 1272 Section 2.3 of [RFC5869]. The encrypted value SHOULD 1273 be hex encoded or base64 encoded (as specified in 1274 section 4 of [RFC4648]). At the agreed-upon 1275 expiration time, a new key SHOULD be generated and 1276 used. New keys SHOULD be indicated by prefixing the 1277 key with a special character such as exclamation 1278 point. In this way, shorter lifetimes can be used as 1279 needed. 1281 * occurrence: there MUST be one and only one instance of this 1282 field. 1284 o s-ip: 1286 * format: ADDRESS 1288 * field value: the IPv4 or IPv6 address of the Surrogate that 1289 served the request (i.e., the "server" address). 1291 * occurrence: there MUST be zero or exactly one instance of this 1292 field. 1294 o s-hostname: 1296 * format: host 1297 * field value: the hostname of the Surrogate that served the 1298 request (i.e., the "server" hostname). 1300 * occurrence: there MUST be zero or exactly one instance of this 1301 field. 1303 o s-port: 1305 * format: 1*DIGIT 1307 * field value: the destination TCP port (i.e., the "server" port) 1308 in the request received by the Surrogate. 1310 * occurrence: there MUST be zero or exactly one instance of this 1311 field. 1313 o cs-method: 1315 * format: NHTABSTRING 1317 * field value: this is the method of the request received by the 1318 Surrogate. In the case of HTTP delivery, this is the HTTP 1319 method in the request. 1321 * occurrence: There MUST be one and only one instance of this 1322 field. 1324 o cs-uri: 1326 * format: NHTABSTRING 1328 * field value: this is the "effective request URI" of the request 1329 received by the Surrogate as specified in [RFC7230]. It 1330 complies with the "http" URI scheme or the "https" URI scheme 1331 as specified in [RFC7230]). Note that cs-uri can be privacy 1332 sensitive. In that case, and where appropriate, u-uri could be 1333 used instead of cs-uri. 1335 * occurrence: there MUST be zero or exactly one instance of this 1336 field. 1338 o u-uri: 1340 * format: NHTABSTRING 1342 * field value: this is a complete URI, derived from the 1343 "effective request URI" ([RFC7230]) of the request received by 1344 the Surrogate (i.e., the cs-uri) but transformed by the entity 1345 generating or transmitting the CDNI Logging Record, in a way 1346 that is agreed upon between the two ends of the CDNI Logging 1347 interface, so the transformed URI is meaningful to the uCDN. 1348 For example, the two ends of the CDNI Logging interface could 1349 agree that the u-uri is constructed from the cs-uri by removing 1350 the part of the hostname that exposes which individual 1351 Surrogate actually performed the delivery. The details of 1352 modification performed to generate the u-uri, as well as the 1353 mechanism to agree on these modifications between the two sides 1354 of the CDNI Logging interface are outside the scope of the 1355 present document. 1357 * occurrence: there MUST be one and only one instance of this 1358 field. 1360 o protocol: 1362 * format: NHTABSTRING 1364 * field value: this is value of the HTTP-Version field as 1365 specified in [RFC7230] of the Request-Line of the request 1366 received by the Surrogate (e.g., "HTTP/1.1"). 1368 * occurrence: there MUST be one and only one instance of this 1369 field. 1371 o sc-status: 1373 * format: 3DIGIT 1375 * field value: this is the Status-Code in the response from the 1376 Surrogate. In the case of HTTP delivery, this is the HTTP 1377 Status-Code in the HTTP response. 1379 * occurrence: There MUST be one and only one instance of this 1380 field. 1382 o sc-total-bytes: 1384 * format: 1*DIGIT 1386 * field value: this is the total number of bytes of the response 1387 sent by the Surrogate in response to the request. In the case 1388 of HTTP delivery, this includes the bytes of the Status-Line, 1389 the bytes of the HTTP headers and the bytes of the message- 1390 body. 1392 * occurrence: There MUST be one and only one instance of this 1393 field. 1395 o sc-entity-bytes: 1397 * format: 1*DIGIT 1399 * field value: this is the number of bytes of the message-body in 1400 the HTTP response sent by the Surrogate in response to the 1401 request. This does not include the bytes of the Status-Line or 1402 the bytes of the HTTP headers. 1404 * occurrence: there MUST be zero or exactly one instance of this 1405 field. 1407 o cs(insert_HTTP_header_name_here): 1409 * format: QSTRING 1411 * field value: the value of the HTTP header (identified by the 1412 insert_HTTP_header_name_here in the CDNI Logging field name) as 1413 it appears in the request processed by the Surrogate, but 1414 prepended by a DQUOTE and appended by a DQUOTE. For example, 1415 when the CDNI Logging field name (FIENAME) listed in the 1416 preceding fields directive is cs(User-Agent), this CDNI Logging 1417 field value contains the value of the User-Agent HTTP header as 1418 received by the Surrogate in the request it processed, but 1419 prepended by a DQUOTE and appended by a DQUOTE. If the HTTP 1420 header as it appeared in the request processed by the Surrogate 1421 contains one or more DQUOTE, each DQUOTE MUST be escaped with 1422 percent encoding. For example, if the HTTP header contains 1423 My_Header"value", then the field value of the 1424 cs(insert_HTTP_header_name_here) is "My_Header%x22value%x22". 1425 The entity transmitting the CDNI Logging File MUST ensure that 1426 the respective insert_HTTP_header_name_here of the 1427 cs(insert_HTTP_header_name_here) listed in the fields directive 1428 comply with HTTP specifications. In particular, this field 1429 name does not include any HTAB, since this would prevent proper 1430 parsing of the fields directive by the entity receiving the 1431 CDNI Logging File. 1433 * occurrence: there MAY be zero, one or any number of instance of 1434 this field. 1436 o sc(insert_HTTP_header_name_here): 1438 * format: QSTRING 1439 * field value: the value of the HTTP header (identified by the 1440 insert_HTTP_header_name_here in the CDNI Logging field name) as 1441 it appears in the response issued by the Surrogate to serve the 1442 request, but prepended by a DQUOTE and appended by a DQUOTE. 1443 If the HTTP header as it appeared in the request processed by 1444 the Surrogate contains one or more DQUOTE, each DQUOTE MUST be 1445 escaped with percent encoding. For example, if the HTTP header 1446 contains My_Header"value", then the field value of the 1447 sc(insert_HTTP_header_name_here) is "My_Header%x22value%x22". 1448 The entity transmitting the CDNI Logging File MUST ensure that 1449 the respective insert_HTTP_header_name_here of the 1450 cs(insert_HTTP_header_name_here) listed in the fields directive 1451 comply with HTTP specifications. In particular, this field 1452 name does not include any HTAB, since this would prevent proper 1453 parsing of the fields directive by the entity receiving the 1454 CDNI Logging File. 1456 * occurrence: there MAY be zero, one or any number of instances 1457 of this field. For a given insert_HTTP_header_name_here, there 1458 MUST be zero or exactly one instance of this field. 1460 o s-ccid: 1462 * format: QSTRING 1464 * field value: this contains the value of the Content Collection 1465 IDentifier (CCID) associated by the uCDN to the content served 1466 by the Surrogate via the CDNI Metadata interface 1467 ([I-D.ietf-cdni-metadata]), prepended by a DQUOTE and appended 1468 by a DQUOTE. If the CCID conveyed in the CDNI Metadata 1469 interface contains one or more DQUOTE, each DQUOTE MUST be 1470 escaped with percent encoding. For example, if the CCID 1471 conveyed in the CDNI Metadata interface is My_CCIDD"value", 1472 then the field value of the s-ccid is "My_CCID%x22value%X22". 1474 * occurrence: there MUST be zero or exactly one instance of this 1475 field. For a given insert_HTTP_header_name_here, there MUST be 1476 zero or exactly one instance of this field. 1478 o s-sid: 1480 * format: QSTRING 1482 * field value: this contains the value of a Session IDentifier 1483 (SID) generated by the dCDN for a specific HTTP session, 1484 prepended by a DQUOTE and appended by a DQUOTE. In particular, 1485 for HTTP Adaptive Streaming (HAS) session, the Session 1486 IDentifier value is included in the Logging record for every 1487 content chunk delivery of that session in view of facilitating 1488 the later correlation of all the per content chunk log records 1489 of a given HAS session. See section 3.4.2.2. of [RFC6983] for 1490 more discussion on the concept of Session IDentifier in the 1491 context of HAS. If the SID conveyed contains one or more 1492 DQUOTE, each DQUOTE MUST be escaped with percent encoding. For 1493 example, if the SID is My_SID"value", then the field value of 1494 the s-sid is "My_SID%x22value%x22". 1496 * occurrence: there MUST be zero or exactly one instance of this 1497 field. 1499 o s-cached: 1501 * format: 1DIGIT 1503 * field value: this characterises whether the Surrogate served 1504 the request using content already stored on its local cache or 1505 not. The allowed values are "0" (for miss) and "1" (for hit). 1506 "1" MUST be used when the Surrogate did serve the request using 1507 exclusively content already stored on its local cache. "0" MUST 1508 be used otherwise (including cases where the Surrogate served 1509 the request using some, but not all, content already stored on 1510 its local cache). Note that a "0" only means a cache miss in 1511 the Surrogate and does not provide any information on whether 1512 the content was already stored, or not, in another device of 1513 the dCDN, i.e., whether this was a "dCDN hit" or "dCDN miss". 1515 * occurrence: there MUST be zero or exactly one instance of this 1516 field. 1518 The "fields" directive corresponding to a HTTP Request Logging Record 1519 MUST contain all the fields names whose occurrence is specified above 1520 as "There MUST be one and only one instance of this field". The 1521 corresponding fields value MUST be present in every HTTP Request 1522 Logging Record. 1524 The "fields" directive corresponding to a HTTP Request Logging Record 1525 MAY list all the fields value whose occurrence is specified above as 1526 "there MUST be zero or exactly one instance of this field" or "there 1527 MAY be zero, one or any number of instances of this field". The set 1528 of such field names actually listed in the "fields" directive is 1529 selected by the CDN generating the CDNI Logging File based on 1530 agreements between the interconnected CDNs established through 1531 mechanisms outside the scope of this specification (e.g., contractual 1532 agreements). When such a field name is not listed in the "fields" 1533 directive, the corresponding field value MUST NOT be included in the 1534 Logging Record. When such a field name is listed in the "fields" 1535 directive, the corresponding field value MUST be included in the 1536 Logging Record; if the value for the field is not available, this 1537 MUST be conveyed via a dash character ("-"). 1539 The fields names listed in the "fields" directive MAY be listed in 1540 the order in which they are listed in Section 3.4.1 or MAY be listed 1541 in any other order. 1543 Logging some specific fields from HTTP requests and responses can 1544 introduce serious security and privacy risks. For example, cookies 1545 will often contain (months) long lived token values that can be used 1546 to log into a service as the relevant user. Similar values may be 1547 included in other header fields or within URLs or elsewhere in HTTP 1548 requests and responses. Centralising such values in a CDNI Logging 1549 File can therefore represent a significant increase in risk both for 1550 the user and the web service provider, but also for the CDNs 1551 involved. Implementations ought therefore to attempt to lower the 1552 probability of such bad outcomes e.g. by only allowing a configured 1553 set of headers to be added to CDNI Logging Records, or by not 1554 supporting wildcard selection of HTTP request/response fields to add. 1555 Such mechanisms can reduce the probability that security (or privacy) 1556 sensitive values are centralised in CDNI Logging Files. Also, when 1557 agreeing on which HTTP request/response fields are to be provided in 1558 CDNI Logging Files, the uCDN and dCDN administrators ought to 1559 consider these risks. Furthermore, CDNs making use of c-groupid to 1560 identify an aggregate of clients rather than individual clients ought 1561 to realize that by logging certain header fields they may create the 1562 possibility to re-identify individual clients. In these cases 1563 heeding the above advice, or not logging header fields at all, is 1564 particularly important if the goal is to provide logs that do not 1565 identify individual clients." 1567 A dCDN-side implementation of the CDNI Logging interface MUST 1568 implement all the following Logging fields in a CDNI Logging Record 1569 of record-type "cdni_http_request_v1", and MUST support the ability 1570 to include valid values for each of them: 1572 o date 1574 o time 1576 o time-taken 1578 o c-groupid 1580 o s-ip 1582 o s-hostname 1583 o s-port 1585 o cs-method 1587 o cs-uri 1589 o u-uri 1591 o protocol 1593 o sc-status 1595 o sc-total-bytes 1597 o sc-entity-bytes 1599 o cs(insert_HTTP_header_name_here) 1601 o sc(insert_HTTP_header_name_here) 1603 o s-cached 1605 A dCDN-side implementation of the CDNI Logging interface MAY support 1606 the following Logging fields in a CDNI Logging Record of record-type 1607 "cdni_http_request_v1": 1609 o s-ccid 1611 o s-sid 1613 If a dCDN-side implementation of the CDNI Logging interface supports 1614 these fields, it MUST support the ability to include valid values for 1615 them. 1617 An uCDN-side implementation of the CDNI Logging interface MUST be 1618 able to accept CDNI Logging Files with CDNI Logging Records of 1619 record-type "cdni_http_request_v1" containing any CDNI Logging Field 1620 defined in Section 3.4.1 as long as the CDNI Logging Record and the 1621 CDNI Logging File are compliant with the present document. 1623 In case an uCDN-side implementation of the CDNI Logging interface 1624 receives a CDNI Logging File with HTTP Request Logging Records that 1625 do not contain field values for exactly the set of field names 1626 actually listed in the preceding "fields" directive, the 1627 implementation MUST reject those HTTP Request Logging Records, and 1628 MUST accept the other HTTP Request Logging Records. 1630 To ensure that the logging file is correct, the text MUST be 1631 sanitized before being logged. Null, bare CR, bare LF and HTAB have 1632 to be removed by escaping them through percent encoding to avoid 1633 confusion with the logging record separators. 1635 3.5. CDNI Logging File Extension 1637 The CDNI Logging File contains blocks of directives and blocks of 1638 corresponding records. The supported set of directives is defined 1639 relative to the CDNI Logging File Format version. The complete set 1640 of directives for version "CDNI/1.0" are defined in Section 3.3. The 1641 directive list is not expected to require much extension, but when it 1642 does, the new directive MUST be defined and registered in the "CDNI 1643 Logging Directive Names" registry, as described in Figure 9, and a 1644 new version MUST be defined and registered in the "CDNI Logging File 1645 version" registry, as described in Section 6.2. For example, adding 1646 a new CDNI Logging Directive, e.g., "foo", to the set of directives 1647 defined for "CDNI/1.0" in Section 3.3, would require registering both 1648 the new CDNI Logging Directive "foo" and a new CDNI Logging File 1649 version, e.g., "CDNI/2.0", which includes all of the existing CDNI 1650 Logging Directives of "CDNI/1.0" plus "foo". 1652 It is expected that as new logging requirements arise, the list of 1653 fields to log will change and expand. When adding new fields, the 1654 new fields MUST be defined and registered in the "CDNI Logging Field 1655 Names" registry, as described in Section 6.4, and a new record-type 1656 MUST be defined and registered in the "CDNI Logging record-types" 1657 registry, as described in Section 6.3. For example, adding a new 1658 CDNI Logging Field, e.g., "c-bar", to the set of fields defined for 1659 "cdni_http_request_v1" in Section 3.4.1, would require registering 1660 both the new CDNI Logging Field "c-bar" and a new CDNI record-type, 1661 e.g., "cdni_http_request_v2", which includes all of the existing CDNI 1662 Logging Fields of "cdni_http_request_v1" plus "c-bar". 1664 3.6. CDNI Logging File Examples 1666 Let us consider the upstream CDN and the downstream CDN labelled uCDN 1667 and dCDN-1 in Figure 1. When dCDN-1 acts as a downstream CDN for 1668 uCDN and performs content delivery on behalf of uCDN, dCDN-1 will 1669 include the CDNI Logging Records corresponding to the content 1670 deliveries performed on behalf of uCDN in the CDNI Logging Files for 1671 uCDN. An example CDNI Logging File communicated by dCDN-1 to uCDN is 1672 shown below in Figure 4. 1674 #version:cdni/1.0 1676 #UUID:urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6 1678 #claimed-origin:cdni-logging-entity.dcdn-1.example.com 1680 #record-type:cdni_http_request_v1 1682 #fields:datetimetime-takenc-groupid 1683 cs-methodu-uriprotocol 1684 sc-statussc-total-bytescs(User-Agent) 1685 cs(Referer)s-cached 1687 2013-05-1700:38:06.8259.058US/TN/MEM/38138 1688 GET 1689 http://cdni-ucdn.dcdn-1.example.com/video/movie100.mp4 1690 HTTP/1.12006729891"Mozilla/5.0 1691 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1692 Gecko) Chrome/5.0.375.127 Safari/533.4" 1693 "host1.example.com"1 1695 2013-05-1700:39:09.14515.32FR/PACA/NCE/06100 1696 GET 1697 http://cdni-ucdn.dcdn-1.example.com/video/movie118.mp4 1698 HTTP/1.120015799210"Mozilla/5.0 1699 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1700 Gecko) Chrome/5.0.375.127 Safari/533.4" 1701 "host1.example.com"1 1703 2013-05-1700:42:53.43752.879US/TN/MEM/38138 1704 GET 1705 http://cdni-ucdn.dcdn-1.example.com/video/picture11.mp4 1706 HTTP/1.020097234724"Mozilla/5.0 1707 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1708 Gecko) Chrome/5.0.375.127 Safari/533.4" 1709 "host5.example.com"0 1711 #SHA256-hash: 64-hexadecimal-digit hash value 1713 Figure 4: CDNI Logging File Example 1715 If uCDN establishes by some means (e.g., via TLS authentication when 1716 pulling the CDNI Logging File) the identity of the entity from which 1717 it pulled the CDNI Logging File, uCDN can add to the CDNI Logging an 1718 established-origin directive as illustrated below: 1720 #established-origin:cdni-logging-entity.dcdn- 1721 1.example.com 1722 As illustrated in Figure 2, uCDN will then ingest the corresponding 1723 CDNI Logging Records into its Collection process, alongside the 1724 Logging Records generated locally by the uCDN itself. This allows 1725 uCDN to aggregate Logging Records for deliveries performed by itself 1726 (through Records generated locally) as well as for deliveries 1727 performed by its downstream CDN(s). This aggregate information can 1728 then be used (after Filtering and Rectification, as illustrated in 1729 Figure 2) by Log Consuming Applications that take into account 1730 deliveries performed by uCDN as well as by all of its downstream 1731 CDNs. 1733 We observe that the time between 1735 1. when a delivery is completed in dCDN and 1737 2. when the corresponding Logging Record is ingested by the 1738 Collection process in uCDN 1740 depends on a number of parameters such as the Logging Period agreed 1741 to by uCDN and dCDN, how much time uCDN waits before pulling the CDNI 1742 Logging File once it is advertised in the CDNI Logging Feed, and the 1743 time to complete the pull of the CDNI Logging File. Therefore, if we 1744 consider the set of Logging Records aggregated by the Collection 1745 process in uCDN in a given time interval, there could be a permanent 1746 significant timing difference between the CDNI Logging Records 1747 received from the dCDN and the Logging Records generated locally. 1748 For example, in a given time interval, the Collection process in uCDN 1749 may be aggregating Logging Records generated locally by uCDN for 1750 deliveries performed in the last hour and CDNI Logging Records 1751 generated in the dCDN for deliveries in the hour before last. 1753 Say, that for some reason (for example a Surrogate bug), dCDN-1 could 1754 not collect the total number of bytes of the responses sent by the 1755 Surrogate (in other words, the value for sc-total-bytes is not 1756 available). Then the corresponding CDNI Logging records would 1757 contain a dash character ("-") in lieu of the value for the sc-total- 1758 bytes field (as specified in Section 3.4.1). In that case, the CDNI 1759 Logging File that would be communicated by dCDN-1 to uCDN is shown 1760 below in Figure 5. 1762 #version:cdni/1.0 1764 #UUID:urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6 1766 #claimed-origin:cdni-logging-entity.dcdn-1.example.com 1768 #record-type:cdni_http_request_v1 1770 #fields:datetimetime-takenc-groupid 1771 cs-methodu-uriprotocol 1772 sc-statussc-total-bytescs(User-Agent) 1773 cs(Referer)s-cached 1775 2013-05-1700:38:06.8259.058US/TN/MEM/38138 1776 GET 1777 http://cdni-ucdn.dcdn-1.example.com/video/movie100.mp4 1778 HTTP/1.1200-"Mozilla/5.0 1779 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1780 Gecko) Chrome/5.0.375.127 Safari/533.4" 1781 "host1.example.com"1 1783 2013-05-1700:39:09.14515.32FR/PACA/NCE/06100 1784 GET 1785 http://cdni-ucdn.dcdn-1.example.com/video/movie118.mp4 1786 HTTP/1.1200-"Mozilla/5.0 1787 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1788 Gecko) Chrome/5.0.375.127 Safari/533.4" 1789 "host1.example.com"1 1791 2013-05-1700:42:53.43752.879US/TN/MEM/38138 1792 GET 1793 http://cdni-ucdn.dcdn-1.example.com/video/picture11.mp4 1794 HTTP/1.0200-"Mozilla/5.0 1795 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1796 Gecko) Chrome/5.0.375.127 Safari/533.4" 1797 "host5.example.com"0 1799 #SHA256-hash: 64-hexadecimal-digit hash value 1801 Figure 5: CDNI Logging File Example With A Missing Field Value 1803 3.7. Cascaded CDNI Logging Files Example 1805 Let us consider the cascaded CDN scenario of uCDN, dCDN-2 and dCDN-3 1806 as depicted in Figure 1. After completion of a delivery by dCDN-3 on 1807 behalf of dCDN-2, dCDN-3 will include a corresponding Logging Record 1808 in a CDNI Logging File that will be pulled by dCDN-2 and that is 1809 illustrated below in Figure 6. In practice, a CDNI Logging File is 1810 likely to contain a very high number of CDNI Logging Records. 1811 However, for readability, the example in Figure 6 contains a single 1812 CDNI Logging Record. 1814 #version:CDNI/1.0 1816 #UUID:urn:uuid:65718ef-0123-9876-adce4321bcde 1818 #claimed-origin:cdni-logging-entity.dcdn-3.example.com 1820 #record-type:cdni_http_request_v1 1822 #fields:datetimetime-takenc-groupid 1823 cs-methodu-uriprotocol 1824 sc-statussc-total-bytescs(User-Agent) 1825 cs(Referer)s-cached 1827 2013-05-1700:39:09.11914.07US/CA/SFO/94114 1828 GET 1829 http://cdni-dcdn-2.dcdn-3.example.com/video/movie118.mp4 1830 HTTP/1.120015799210"Mozilla/5.0 1831 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1832 Gecko) Chrome/5.0.375.127 Safari /533.4" 1833 "host1.example.com"1 1835 #SHA256-hash: 64-hexadecimal-digit hash value 1837 Figure 6: Cascaded CDNI Logging File Example (dCDN-3 to dCDN-2) 1839 If dCDN-2 establishes by some means (e.g., via TLS authentication 1840 when pulling the CDNI Logging File) the identity of the entity from 1841 which it pulled the CDNI Logging File, dCDN-2 can add to the CDNI 1842 Logging an established-origin directive as illustrated below: 1844 #established-origin:cdni-logging-entity.dcdn- 1845 3.example.com 1847 dCDN-2 (behaving as an upstream CDN from the viewpoint of dCDN-3) 1848 will then ingest the CDNI Logging Record for the considered dCDN-3 1849 delivery into its Collection process (as illustrated in Figure 2). 1850 This Logging Record may be aggregated with Logging Records generated 1851 locally by dCDN-2 for deliveries performed by dCDN-2 itself. Say, 1852 for illustration, that the content delivery performed by dCDN-3 on 1853 behalf of dCDN-2 had actually been redirected to dCDN-2 by uCDN, and 1854 say that another content delivery has just been redirected by uCDN to 1855 dCDN-2 and that dCDN-2 elected to perform the corresponding delivery 1856 itself. Then after Filtering and Rectification (as illustrated in 1857 Figure 2), dCDN-2 will include the two Logging Records corresponding 1858 respectively to the delivery performed by dCDN-3 and the delivery 1859 performed by dCDN-2, in the next CDNI Logging File that will be 1860 communicated to uCDN. An example of such CDNI Logging File is 1861 illustrated below in Figure 7. 1863 #version:CDNI/1.0 1865 #UUID:urn:uuid:1234567-8fedc-abab-0987654321ff 1867 #claimed-origin:cdni-logging-entity.dcdn-2.example.com 1869 #record-type:cdni_http_request_v1 1871 #fields:datetimetime-takenc-groupid 1872 cs-methodu-uriprotocol 1873 sc-statussc-total-bytescs(User-Agent) 1874 cs(Referer)s-cached 1876 2013-05-1700:39:09.11914.07US/CA/SFO/94114 1877 GET 1878 http://cdni-ucdn.dcdn-2.example.com/video/movie118.mp4 1879 HTTP/1.120015799210"Mozilla/5.0 1880 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1881 Gecko) Chrome/5.0.375.127 Safari /533.4" 1882 "host1.example.com"1 1884 2013-05-1701:42:53.43752.879FR/IDF/PAR/75001 1885 GET 1886 http://cdni-ucdn.dcdn-2.example.com/video/picture11.mp4 1887 HTTP/1.020097234724"Mozilla/5.0 1888 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1889 Gecko) Chrome/5.0.375.127 Safari /533.4" 1890 "host5.example.com"0 1892 #SHA256-hash: 64-hexadecimal-digit hash value 1894 Figure 7: Cascaded CDNI Logging File Example (dCDN-2 to uCDN) 1896 If uCDN establishes by some means (e.g., via TLS authentication when 1897 pulling the CDNI Logging File) the identity of the entity from which 1898 it pulled the CDNI Logging File, uCDN can add to the CDNI Logging an 1899 established-origin directive as illustrated below: 1901 #established-origin:cdni-logging-entity.dcdn- 1902 2.example.com 1904 In the example of Figure 7, we observe that: 1906 o the first Logging Record corresponds to the Logging Record 1907 communicated earlier to dCDN-2 by dCDN-3, which corresponds to a 1908 delivery redirected by uCDN to dCDN-2 and then redirected by 1909 dCDN-2 to dCDN-3. The fields values in this Logging Record are 1910 copied from the corresponding CDNI Logging REcord communicated to 1911 dCDN2 by dCDN-3, with the exception of the u-uri that now reflects 1912 the URI convention between uCDN and dCDN-2 and that presents the 1913 delivery to uCDN as if it was performed by dCDN-2 itself. This 1914 reflects the fact that dCDN-2 had taken the full responsibility of 1915 the corresponding delivery (even if in this case, dCDN-2 elected 1916 to redirect the delivery to dCDN-3 so it is actually performed by 1917 dCDN-3 on behalf of dCDN-2). 1919 o the second Logging Record corresponds to a delivery redirected by 1920 uCDN to dCDN-2 and performed by dCDN-2 itself. The time of the 1921 delivery in this Logging Record may be significantly more recent 1922 than the first Logging Record since it was generated locally while 1923 the first Logging Record was generated by dCDN-3 and had to be 1924 advertised , and then pulled and then ingested into the dCDN-2 1925 Collection process, before being aggregated with the second 1926 Logging Record. 1928 4. Protocol for Exchange of CDNI Logging File After Full Collection 1930 This section specifies a protocol for the exchange of CDNI Logging 1931 Files as specified in Section 3 after the CDNI Logging File is fully 1932 collected by the dCDN. 1934 This protocol comprises: 1936 o a CDNI Logging feed, allowing the dCDN to notify the uCDN about 1937 the CDNI Logging Files that can be retrieved by that uCDN from the 1938 dCDN, as well as all the information necessary for retrieving each 1939 of these CDNI Logging Files. The CDNI Logging feed is specified 1940 in Section 4.1. 1942 o a CDNI Logging File pull mechanism, allowing the uCDN to obtain 1943 from the dCDN a given CDNI Logging File at the uCDN's convenience. 1944 The CDNI Logging File pull mechanisms is specified in Section 4.2. 1946 An implementation of the CDNI Logging interface on the dCDN side (the 1947 entity generating the CDNI Logging file) MUST support the server side 1948 of the CDNI Logging feed (as specified in Section 4.1) and the server 1949 side of the CDNI Logging pull mechanism (as specified in 1950 Section 4.2). 1952 An implementation of the CDNI Logging interface on the uCDN side (the 1953 entity consuming the CDNI Logging file) MUST support the client side 1954 of the CDNI Logging feed (as specified in Section 4.1) and the client 1955 side of the CDNI Logging pull mechanism (as specified in 1956 Section 4.2). 1958 4.1. CDNI Logging Feed 1960 The server-side implementation of the CDNI Logging feed MUST produce 1961 an Atom feed [RFC4287]. This feed is used to advertise log files 1962 that are available for the client-side to retrieve using the CDNI 1963 Logging pull mechanism. 1965 4.1.1. Atom Formatting 1967 A CDNI Logging feed MUST be structured as an Archived feed, as 1968 defined in [RFC5005], and MUST be formatted in Atom [RFC4287]. This 1969 means it consists of a subscription document that is regularly 1970 updated as new CDNI Logging Files become available, and information 1971 about older CDNI Logging files is moved into archive documents. Once 1972 created, archive documents are never modified. 1974 Each CDNI Logging File listed in an Atom feed MUST be described in an 1975 atom:entry container element. 1977 The atom:entry MUST contain an atom:content element whose "src" 1978 attribute is a link to the CDNI Logging File and whose "type" 1979 attribute is the MIME Media Type indicating that the entry is a CDNI 1980 logging file. This MIME Media Type is defined as "application/cdni" 1981 (See [RFC7736]) with the Payload Type (ptype) parameter set to 1982 "logging-file". 1984 For compatibility with some Atom feed readers the atom:entry MAY also 1985 contain an atom:link entry whose "href" attribute is a link to the 1986 CDNI Logging File and whose "type" attribute is the MIME Media Type 1987 indicating that the entry is a CDNI Logging File using the 1988 "application/cdni" MIME Media Type with the Payload Type (ptype) 1989 parameter set to "logging-file"(See [RFC7736]). 1991 The URI used in the atom:id of the atom:entry MUST contain the UUID 1992 of the CDNI Logging File. 1994 The atom:updated in the atom:entry MUST indicate the time at which 1995 the CDNI Logging File was last updated. 1997 4.1.2. Updates to Log Files and the Feed 1999 CDNI Logging Files MUST NOT be modified by the dCDN once published in 2000 the CDNI Logging feed. 2002 The frequency with which the subscription feed is updated, the period 2003 of time covered by each CDNI Logging File or each archive document, 2004 and timeliness of publishing of CDNI Logging Files are outside the 2005 scope of the present document and are expected to be agreed upon by 2006 uCDN and dCDN via other means (e.g., human agreement). 2008 The server-side implementation MUST be able to set, and SHOULD set, 2009 HTTP cache control headers on the subscription feed to indicate the 2010 frequency at which the client-side is to poll for updates. 2012 The client-side MAY use HTTP cache control headers (set by the 2013 server-side) on the subscription feed to determine the frequency at 2014 which to poll for updates. The client-side MAY instead, or in 2015 addition, use other information to determine when to poll for updates 2016 (e.g., a polling frequency that may have been negotiated between the 2017 uCDN and dCDN by mechanisms outside the scope of the present document 2018 and that is to override the indications provided in the HTTP cache 2019 control headers). 2021 The potential retention limits (e.g., sliding time window) within 2022 which the dCDN is to retain and be ready to serve an archive document 2023 is outside the scope of the present document and is expected to be 2024 agreed upon by uCDN and dCDN via other means (e.g., human agreement). 2025 The server-side implementation MUST retain, and be ready to serve, 2026 any archive document within the agreed retention limits. Outside 2027 these agreed limits, the server-side implementation MAY indicate its 2028 inability to serve (e.g., with HTTP status code 404) an archive 2029 document or MAY refuse to serve it (e.g., with HTTP status code 403 2030 or 410). 2032 4.1.3. Redundant Feeds 2034 The server-side implementation MAY present more than one CDNI Logging 2035 feed for redundancy. Each CDNI Logging File MAY be published in more 2036 than one feed. 2038 A client-side implementation MAY support such redundant CDNI Logging 2039 feeds. If it supports redundant CDNI Logging feed, the client-side 2040 can use the UUID of the CDNI Logging File, presented in the atom:id 2041 element of the Atom feed, to avoid unnecessarily pulling and storing 2042 a given CDNI Logging File more than once. 2044 4.1.4. Example CDNI Logging Feed 2046 Figure 8 illustrates an example of the subscription document of a 2047 CDNI Logging feed. 2049 2050 2051 CDNI Logging Feed 2052 2013-03-23T14:46:11Z 2053 urn:uuid:663ae677-40fb-e99a-049d-c5642916b8ce 2054 2056 2058 2060 CDNI Log Feed 2061 Generator 2062 dcdn.example 2063 2064 CDNI Logging File for uCDN at 2065 2013-03-23 14:15:00 2066 urn:uuid:12345678-1234-abcd-00aa-01234567abcd 2067 2013-03-23T14:15:00Z 2068 2072 CDNI Logging File for uCDN at 2073 2013-03-23 14:15:00 2074 2075 2076 CDNI Logging File for uCDN at 2077 2013-03-23 14:30:00 2078 urn:uuid:87654321-4321-dcba-aa00-dcba7654321 2079 2013-03-23T14:30:00Z 2080 2084 CDNI Logging File for uCDN at 2085 2013-03-23 14:30:00 2086 2087 ... 2088 2089 ... 2090 2091 2093 Figure 8: Example subscription document of a CDNI Logging Feed 2095 4.2. CDNI Logging File Pull 2097 A client-side implementation of the CDNI Logging interface MAY pull, 2098 at its convenience, a CDNI Logging File that is published by the 2099 server-side in the CDNI Logging Feed (in the subscription document or 2100 an archive document). To do so, the client-side: 2102 o MUST implement HTTP/1.1 ([RFC7230],[RFC7231], [RFC7232], 2103 [RFC7233], [RFC7234], [RFC7235]), MAY also support other HTTP 2104 versions (e.g., HTTP/2 [RFC7540]) and MAY negotiate which HTTP 2105 version is actually used. This allows operators and implementers 2106 to choose to use later versions of HTTP to take advantage of new 2107 features, while still ensuring interoperability with systems that 2108 only support HTTP/1.1. 2110 o MUST use the URI that was associated to the CDNI Logging File 2111 (within the "src" attribute of the corresponding atom:content 2112 element) in the CDNI Logging Feed; 2114 o MUST support exchange of CDNI Logging Files with no content 2115 encoding applied to the representation; 2117 o MUST support exchange of CDNI Logging Files with "gzip" content 2118 encoding (as defined in [RFC7230]) applied to the representation. 2120 Note that a client-side implementation of the CDNI Logging interface 2121 MAY pull a CDNI Logging File that it has already pulled. 2123 The server-side implementation MUST respond to valid pull request by 2124 a client-side implementation for a CDNI Logging File published by the 2125 server-side in the CDNI Logging Feed (in the subscription document or 2126 an archive document). The server-side implementation: 2128 o MUST implement HTTP/1.1 to handle the client-side request and MAY 2129 also support other HTTP versions (e.g., HTTP/2); 2131 o MUST include the CDNI Logging File identified by the request URI 2132 inside the body of the HTTP response; 2134 o MUST support exchange of CDNI Logging Files with no content 2135 encoding applied to the representation; 2137 o MUST support exchange of CDNI Logging Files with "gzip" content 2138 encoding (as defined in [RFC7231]) applied to the representation. 2140 Content negotiation approaches defined in [RFC7231] (e.g., using 2141 Accept-Encoding request-header field or Content-Encoding entity- 2142 header field) MAY be used by the client-side and server-side 2143 implementations to establish the content-coding to be used for a 2144 particular exchange of a CDNI Logging File. 2146 Applying compression content encoding (such as "gzip") is expected to 2147 mitigate the impact of exchanging the large volumes of logging 2148 information expected across CDNs. This is expected to be 2149 particularly useful in the presence of HTTP Adaptive Streaming (HAS) 2150 which, as per the present version of the document, will result in a 2151 separate CDNI Log Record for each HAS segment delivery in the CDNI 2152 Logging File. 2154 The potential retention limits (e.g., sliding time window, maximum 2155 aggregate file storage quotas) within which the dCDN is to retain and 2156 be ready to serve a CDNI Logging File previously advertised in the 2157 CDNI Logging Feed is outside the scope of the present document and is 2158 expected to be agreed upon by uCDN and dCDN via other means (e.g., 2159 human agreement). The server-side implementation MUST retain, and be 2160 ready to serve, any CDNI Logging File within the agreed retention 2161 limits. Outside these agreed limits, the server-side implementation 2162 MAY indicate its inability to serve (e.g., with HTTP status code 404) 2163 a CDNI Logging File or MAY refuse to serve it (e.g., with HTTP status 2164 code 403 or 410). 2166 5. Protocol for Exchange of CDNI Logging File During Collection 2168 We note that, in addition to the CDNI Logging File exchange protocol 2169 specified in Section 4, implementations of the CDNI Logging interface 2170 may also support other mechanisms to exchange CDNI Logging Files. In 2171 particular, such mechanisms might allow the exchange of the CDNI 2172 Logging File to start before the file is fully collected. This can 2173 allow CDNI Logging Records to be communicated by the dCDN to the uCDN 2174 as they are gathered by the dCDN without having to wait until all the 2175 CDNI Logging Records of the same logging period are collected in the 2176 corresponding CDNI Logging File. This approach is commonly referred 2177 to as "tailing" of the file. 2179 Such an approach could be used, for example, to exchange logging 2180 information with a significantly reduced time-lag (e.g., sub-minute 2181 or sub-second) between when the event occurred in the dCDN and when 2182 the corresponding CDNI Logging Record is made available to the uCDN. 2183 This can satisfy log-consuming applications requiring extremely fresh 2184 logging information such as near-real-time content delivery 2185 monitoring. Such mechanisms are for further study and outside the 2186 scope of this document. 2188 6. IANA Considerations 2190 When IANA allocates new extensions to CDNI Logging Directive Names 2191 Registry, CDNI Logging File version Registry, CDNI Logging record- 2192 type Registry or CDNI Logging fields Registry, IANA MUST take into 2193 account that the directive names are case-insensitive as per the 2194 basic ABNF([RFC5234]). 2196 6.1. CDNI Logging Directive Names Registry 2198 The IANA is requested to create a new "CDNI Logging Directive Names" 2199 subregistry under the "Content Delivery Networks Interconnection 2200 (CDNI) Parameters" registry. 2202 The initial contents of the CDNI Logging Directives registry comprise 2203 the names of the directives specified in Section 3.3 of the present 2204 document, and are as follows: 2206 +------------------------------+-----------+ 2207 | Directive Name | Reference | 2208 +------------------------------+-----------+ 2209 | version | RFC xxxx | 2210 | UUID | RFC xxxx | 2211 | claimed-origin | RFC xxxx | 2212 | established-origin | RFC xxxx | 2213 | remark | RFC xxxx | 2214 | record-type | RFC xxxx | 2215 | fields | RFC xxxx | 2216 | SHA256-hash | RFC xxxx | 2217 +------------------------------+-----------+ 2219 Figure 9 2221 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2222 the present document] 2224 Within the registry, names are to be allocated by IANA according to 2225 the "Specification Required" policy specified in [RFC5226]. 2226 Directive names are to be allocated by IANA with a format of 2227 NAMEFORMAT (see Section 3.1). All directive names defined in the 2228 logging file are case-insensitive as per the basic ABNF([RFC5234]). 2230 Each specification that defines a new CDNI Logging directive needs to 2231 contain a description for the new directive with the same set of 2232 information as provided in Section 3.3 (i.e., format, directive value 2233 and occurrence). 2235 6.2. CDNI Logging File version Registry 2237 The IANA is requested to create a new "CDNI Logging File version" 2238 subregistry under the "Content Delivery Networks Interconnection 2239 (CDNI) Parameters" registry. 2241 The initial contents of the CDNI Logging Logging File version 2242 registry comprise the value "CDNI/1.0" specified in Section 3.3 of 2243 the present document, and are as follows: 2245 +-----------------+-----------+----------------------------------+ 2246 | version | Reference | Description | 2247 +-----------------+-----------+----------------------------------+ 2248 | cdni/1.0 | RFC xxxx | CDNI Logging File version 1.0 | 2249 | | | as specified in RFC xxxx | 2250 +-----------------+-----------+----------------------------------+ 2252 Figure 10 2254 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2255 the present document] 2257 Within the registry, version values are to be allocated by IANA 2258 according to the "Specification Required" policy specified in 2259 [RFC5226]. Version values are to be allocated by IANA with a format 2260 of NAMEFORMAT (see Section 3.1). All version values defined in the 2261 logging file are case-insensitive as per the basic ABNF([RFC5234]). 2263 6.3. CDNI Logging record-types Registry 2265 The IANA is requested to create a new "CDNI Logging record-types" 2266 subregistry under the "Content Delivery Networks Interconnection 2267 (CDNI) Parameters" registry. 2269 The initial contents of the CDNI Logging record-types registry 2270 comprise the names of the CDNI Logging Record types specified in 2271 Section 3.4 of the present document, and are as follows: 2273 +----------------------+-----------+---------------------------------+ 2274 | record-types | Reference | Description | 2275 +----------------------+-----------+---------------------------------+ 2276 | cdni_http_request_v1 | RFC xxxx | CDNI Logging Record version 1 | 2277 | | | for content delivery using HTTP | 2278 +----------------------+-----------+---------------------------------+ 2280 Figure 11 2282 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2283 the present document] 2285 Within the registry, record-types are to be allocated by IANA 2286 according to the "Specification Required" policy specified in 2287 [RFC5226]. Record-types are to be allocated by IANA with a format of 2288 NAMEFORMAT (see Section 3.1). All record-types defined in the 2289 logging file are case-insensitive as per the basic ABNF([RFC5234]). 2291 Each specification that defines a new record-type needs to contain a 2292 description for the new record-type with the same set of information 2293 as provided in Section 3.4.1. This includes: 2295 o a list of all the CDNI Logging fields that can appear in a CDNI 2296 Logging Record of the new record-type 2298 o for all these fields: a specification of the occurrence for each 2299 Field in the new record-type 2301 o for every newly defined Field, i.e., for every Field that results 2302 in a registration in the CDNI Logging Field Names Registry 2303 (Section 6.4): a specification of the field name, format and field 2304 value. 2306 6.4. CDNI Logging Field Names Registry 2308 The IANA is requested to create a new "CDNI Logging Field Names" 2309 subregistry under the "Content Delivery Networks Interconnection 2310 (CDNI) Parameters" registry. 2312 This registry is intended to be shared across the currently defined 2313 record-type (i.e., cdni_http_request_v1) as well as potential other 2314 CDNI Logging record-types that may be defined in separate 2315 specifications. When a Field from this registry is used by another 2316 CDNI Logging record-type, it is to be used with the exact semantics 2317 and format specified in the document that registered this field and 2318 that is identified in the Reference column of the registry. If 2319 another CDNI Logging record-type requires a Field with semantics that 2320 are not strictly identical, or a format that is not strictly 2321 identical then this new Field is to be registered in the registry 2322 with a different Field name. When a Field from this registry is used 2323 by another CDNI Logging record-type, it can be used with different 2324 occurrence rules. 2326 The initial contents of the CDNI Logging fields Names registry 2327 comprise the names of the CDNI Logging fields specified in 2328 Section 3.4 of the present document, and are as follows: 2330 +------------------------------------------+-----------+ 2331 | Field Name | Reference | 2332 +------------------------------------------+-----------+ 2333 | date | RFC xxxx | 2334 | time | RFC xxxx | 2335 | time-taken | RFC xxxx | 2336 | c-groupid | RFC xxxx | 2337 | s-ip | RFC xxxx | 2338 | s-hostname | RFC xxxx | 2339 | s-port | RFC xxxx | 2340 | cs-method | RFC xxxx | 2341 | cs-uri | RFC xxxx | 2342 | u-uri | RFC xxxx | 2343 | protocol | RFC xxxx | 2344 | sc-status | RFC xxxx | 2345 | sc-total-bytes | RFC xxxx | 2346 | sc-entity-bytes | RFC xxxx | 2347 | cs(insert_HTTP_header_name_here) | RFC xxxx | 2348 | sc(insert_HTTP_header_name_here) | RFC xxxx | 2349 | s-ccid | RFC xxxx | 2350 | s-sid | RFC xxxx | 2351 | s-cached | RFC xxxx | 2352 +------------------------------------------+-----------+ 2354 Figure 12 2356 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2357 the present document] 2359 Within the registry, names are to be allocated by IANA according to 2360 the "Specification Required" policy specified in [RFC5226]. Field 2361 names are to be allocated by IANA with a format of NHTABSTRING (see 2362 Section 3.1). All field names defined in the logging file are case- 2363 insensitive as per the basic ABNF([RFC5234]). 2365 6.5. CDNI Logging MIME Media Type 2367 The IANA is requested to register the following new Payload Type in 2368 the CDNI Payload Type registry for use with the application/cdni MIME 2369 media type. 2371 [RFC Editor Note: Please replace the references to [RFCthis] below 2372 with this document's RFC number before publication.] 2373 +----------------------+---------------+ 2374 | Payload Type | Specification | 2375 +----------------------+---------------+ 2376 | logging-file | [RFCthis] | 2377 +----------------------+---------------+ 2379 Figure 13: MIME Media Type payload 2381 The purpose of the logging-file payload type is to distinguish 2382 between CDNI Logging Files and other CDNI messages. 2384 Interface: LI 2386 Encoding: see Section 3.2, Section 3.3 and Section 3.4 2388 7. Security Considerations 2390 7.1. Authentication, Authorization, Confidentiality, Integrity 2391 Protection 2393 An implementation of the CDNI Logging interface MUST support TLS 2394 transport of the CDNI Logging feed (Section 4.1) and of the CDNI 2395 Logging File pull (Section 4.2) as per [RFC2818] and [RFC7230]. 2397 The use of TLS for transport of the CDNI Logging feed and CDNI 2398 Logging File pull allows: 2400 o the dCDN and uCDN to authenticate each other 2402 and, once they have mutually authenticated each other, it allows: 2404 o the dCDN and uCDN to authorize each other (to ensure they are 2405 transmitting/receiving CDNI Logging File to/from an authorized 2406 CDN) 2408 o the CDNI Logging information to be transmitted with 2409 confidentiality 2411 o the integrity of the CDNI Logging information to be protected 2412 during the exchange. 2414 In an environment where any such protection is required, mutually 2415 authenticated encrypted transport MUST be used to ensure 2416 confidentiality of the logging information, and to do so, TLS MUST be 2417 used (including authentication of the remote end) by the server-side 2418 and the client-side of the CDNI Logging feed, as well as the server- 2419 side and the client-side of the CDNI Logging File pull mechanism. 2421 When TLS is used, the general TLS usage guidance in [RFC7525] MUST be 2422 followed. 2424 The SHA256-hash directive inside the CDNI Logging File provides 2425 additional integrity protection, this time targeting potential 2426 corruption of the CDNI logging information during the CDNI Logging 2427 File generation, storage or exchange. This mechanism does not itself 2428 allow restoration of the corrupted CDNI Logging information, but it 2429 allows detection of such corruption and therefore triggering of 2430 appropriate corrective actions (e.g., discard of corrupted 2431 information, attempt to re-obtain the CDNI Logging information). 2432 Note that the SHA256-hash does not protect against tampering by a 2433 third party, since such a third party could have recomputed and 2434 updated the SHA256-hash after tampering. Protection against third 2435 party tampering when the CDNI Logging File is communicated over the 2436 CDN Logging Interface can be achieved as discussed above through the 2437 use of TLS. 2439 7.2. Denial of Service 2441 This document does not define specific mechanism to protect against 2442 Denial of Service (DoS) attacks on the Logging Interface. However, 2443 the CDNI Logging feed and CDNI Logging pull endpoints are typically 2444 to be accessed only by a very small number of valid remote endpoints 2445 and therefore can be easily protected against DoS attacks through the 2446 usual conventional DOS protection mechanisms such as firewalling or 2447 use of Virtual Private Networks (VPNs). 2449 Protection of dCDN Surrogates against spoofed delivery requests is 2450 outside the scope of the CDNI Logging interface. 2452 7.3. Privacy 2454 CDNs have the opportunity to collect detailed information about the 2455 downloads performed by End Users. A dCDN is expected to collect such 2456 information into CDNI Logging Files, which are then communicated to 2457 an uCDN. 2459 Having detailed CDNI logging information known by the dCDN in itself 2460 does not represent a particular privacy concern since the dCDN is 2461 obviously fully aware of all information logged since it generated 2462 the information in the first place. Making detailed CDNI logging 2463 information known to the uCDN does not represent a particular privacy 2464 concern because the uCDN is already exposed at request redirection 2465 time to most of the information that shows up as CDNI logging 2466 information (e.g., enduser IP@, URL, HTTP headers - at least when 2467 HTTP redirection is used between uCDN and dCDN). Transporting 2468 detailed CDNI logging information over the HTTP based CDNI Logging 2469 Interface does not represent a particular privacy concern because it 2470 is protected by usual IETF privacy-protection mechanism (e.g.,TLS). 2472 However, one privacy concern arises from the fact that large volumes 2473 of detailed information about content delivery to users, potentially 2474 traceable back to indvidual users, may be collected in CDNI Logging 2475 files. These CDNI Logging files represent high-value targets, likely 2476 concentrated in a fairly centralised system (although the CDNI 2477 Logging architecture does not mandate a particular level of 2478 centralisation/distribution) and at risk of potential data 2479 exfiltration. Note that the means of such data exfiltration are 2480 beyond the scope of the CDNI Logging interface itself (e.g., 2481 corrupted employee, corrupted logging storage system,...). This 2482 privacy concern calls for some protection. 2484 The collection of large volumes of such information into CDNI Logging 2485 Files introduces potential End Users privacy protection concerns. 2486 Mechanisms to address these concerns are discussed in the definition 2487 of the c-groupid field specified in Section 3.4.1. 2489 The use of mutually authenticated TLS to establish a secure session 2490 for the transport of the CDNI Logging feed and CDNI Logging pull as 2491 discussed in Section 7.1 provides confidentiality while the logging 2492 information is in transit and prevents any party other than the 2493 authorised uCDN to gain access to the logging information. 2495 We also note that the query string portion of the URL that may be 2496 conveyed inside the cs-uri and u-uri fields of CDNI Logging Files, or 2497 the HTTP cookies( [RFC6265]) that may be conveyed as part of the 2498 cs() field of CDNI Logging files, may contain 2499 personnal information or information that can be exploited to derive 2500 personal information. Where this is a concern, the CDNI Logging 2501 interface specification allows the dCDN to not include the cs-uri and 2502 to include a u-uri that removes (or hides) the sensitive part of the 2503 query string and allows the dCDN to not include the cs() fields corresponding to HTTP headers associated with cookies. 2506 8. Acknowledgments 2508 This document borrows from the W3C Extended Log Format [ELF]. 2510 Rob Murray significantly contributed into the text of Section 4.1. 2512 The authors thank Ben Niven-Jenkins, Kevin Ma, David Mandelberg and 2513 Ray van Brandenburg for their ongoing input. 2515 Brian Trammel and Rich Salz made significant contributions into 2516 making this interface privacy-friendly. 2518 Finally, we also thank Sebastien Cubaud, Pawel Grochocki, Christian 2519 Jacquenet, Yannick Le Louedec, Anne Marrec, Emile Stephan, Fabio 2520 Costa, Sara Oueslati, Yvan Massot, Renaud Edel, Joel Favier and the 2521 contributors of the EU FP7 OCEAN project for their input in the early 2522 versions of this document. 2524 9. References 2526 9.1. Normative References 2528 [AES] NIST, "Advanced Encryption Standard (AES)", August 2015, 2529 . 2531 [GCM] NIST, "Recommendation for Block Cipher Modes of Operation: 2532 Galois/Counter Mode (GCM) and GMAC", November 2007, . 2535 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2536 Requirement Levels", BCP 14, RFC 2119, 2537 DOI 10.17487/RFC2119, March 1997, 2538 . 2540 [RFC3339] Klyne, G. and C. Newman, "Date and Time on the Internet: 2541 Timestamps", RFC 3339, DOI 10.17487/RFC3339, July 2002, 2542 . 2544 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 2545 Resource Identifier (URI): Generic Syntax", STD 66, 2546 RFC 3986, DOI 10.17487/RFC3986, January 2005, 2547 . 2549 [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally 2550 Unique IDentifier (UUID) URN Namespace", RFC 4122, 2551 DOI 10.17487/RFC4122, July 2005, 2552 . 2554 [RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom 2555 Syndication Format", RFC 4287, DOI 10.17487/RFC4287, 2556 December 2005, . 2558 [RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data 2559 Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006, 2560 . 2562 [RFC5005] Nottingham, M., "Feed Paging and Archiving", RFC 5005, 2563 DOI 10.17487/RFC5005, September 2007, 2564 . 2566 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 2567 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 2568 DOI 10.17487/RFC5226, May 2008, 2569 . 2571 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 2572 Specifications: ABNF", STD 68, RFC 5234, 2573 DOI 10.17487/RFC5234, January 2008, 2574 . 2576 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2577 Protocol (HTTP/1.1): Message Syntax and Routing", 2578 RFC 7230, DOI 10.17487/RFC7230, June 2014, 2579 . 2581 [RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2582 Protocol (HTTP/1.1): Semantics and Content", RFC 7231, 2583 DOI 10.17487/RFC7231, June 2014, 2584 . 2586 [RFC7232] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2587 Protocol (HTTP/1.1): Conditional Requests", RFC 7232, 2588 DOI 10.17487/RFC7232, June 2014, 2589 . 2591 [RFC7233] Fielding, R., Ed., Lafon, Y., Ed., and J. Reschke, Ed., 2592 "Hypertext Transfer Protocol (HTTP/1.1): Range Requests", 2593 RFC 7233, DOI 10.17487/RFC7233, June 2014, 2594 . 2596 [RFC7234] Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, 2597 Ed., "Hypertext Transfer Protocol (HTTP/1.1): Caching", 2598 RFC 7234, DOI 10.17487/RFC7234, June 2014, 2599 . 2601 [RFC7235] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2602 Protocol (HTTP/1.1): Authentication", RFC 7235, 2603 DOI 10.17487/RFC7235, June 2014, 2604 . 2606 [RFC7525] Sheffer, Y., Holz, R., and P. Saint-Andre, 2607 "Recommendations for Secure Use of Transport Layer 2608 Security (TLS) and Datagram Transport Layer Security 2609 (DTLS)", BCP 195, RFC 7525, DOI 10.17487/RFC7525, May 2610 2015, . 2612 [RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext 2613 Transfer Protocol Version 2 (HTTP/2)", RFC 7540, 2614 DOI 10.17487/RFC7540, May 2015, 2615 . 2617 [SHA-3] NIST, "SHA-3 STANDARD: PERMUTATION-BASED HASH AND 2618 EXTENDABLE OUTPUT FUNCTIONS", November 2001, 2619 . 2621 9.2. Informative References 2623 [CHAR_SET] 2624 "IANA Character Sets registry", 2625 . 2628 [ELF] Phillip M. Hallam-Baker, and Brian Behlendorf, "Extended 2629 Log File Format, W3C (work in progress), WD-logfile- 2630 960323", . 2632 [I-D.ietf-cdni-metadata] 2633 Niven-Jenkins, B., Murray, R., Caulfield, M., and K. Ma, 2634 "CDN Interconnection Metadata", draft-ietf-cdni- 2635 metadata-17 (work in progress), May 2016. 2637 [I-D.ietf-tls-rfc5246-bis] 2638 Dierks, T. and E. Rescorla, "The Transport Layer Security 2639 (TLS) Protocol Version 1.3", draft-ietf-tls-rfc5246-bis-00 2640 (work in progress), April 2014. 2642 [I-D.snell-atompub-link-extensions] 2643 Snell, J., "Atom Link Extensions", draft-snell-atompub- 2644 link-extensions-09 (work in progress), June 2012. 2646 [RFC1945] Berners-Lee, T., Fielding, R., and H. Frystyk, "Hypertext 2647 Transfer Protocol -- HTTP/1.0", RFC 1945, 2648 DOI 10.17487/RFC1945, May 1996, 2649 . 2651 [RFC2818] Rescorla, E., "HTTP Over TLS", RFC 2818, 2652 DOI 10.17487/RFC2818, May 2000, 2653 . 2655 [RFC5869] Krawczyk, H. and P. Eronen, "HMAC-based Extract-and-Expand 2656 Key Derivation Function (HKDF)", RFC 5869, 2657 DOI 10.17487/RFC5869, May 2010, 2658 . 2660 [RFC6234] Eastlake 3rd, D. and T. Hansen, "US Secure Hash Algorithms 2661 (SHA and SHA-based HMAC and HKDF)", RFC 6234, 2662 DOI 10.17487/RFC6234, May 2011, 2663 . 2665 [RFC6265] Barth, A., "HTTP State Management Mechanism", RFC 6265, 2666 DOI 10.17487/RFC6265, April 2011, 2667 . 2669 [RFC6707] Niven-Jenkins, B., Le Faucheur, F., and N. Bitar, "Content 2670 Distribution Network Interconnection (CDNI) Problem 2671 Statement", RFC 6707, DOI 10.17487/RFC6707, September 2672 2012, . 2674 [RFC6770] Bertrand, G., Ed., Stephan, E., Burbridge, T., Eardley, 2675 P., Ma, K., and G. Watson, "Use Cases for Content Delivery 2676 Network Interconnection", RFC 6770, DOI 10.17487/RFC6770, 2677 November 2012, . 2679 [RFC6983] van Brandenburg, R., van Deventer, O., Le Faucheur, F., 2680 and K. Leung, "Models for HTTP-Adaptive-Streaming-Aware 2681 Content Distribution Network Interconnection (CDNI)", 2682 RFC 6983, DOI 10.17487/RFC6983, July 2013, 2683 . 2685 [RFC7336] Peterson, L., Davie, B., and R. van Brandenburg, Ed., 2686 "Framework for Content Distribution Network 2687 Interconnection (CDNI)", RFC 7336, DOI 10.17487/RFC7336, 2688 August 2014, . 2690 [RFC7337] Leung, K., Ed. and Y. Lee, Ed., "Content Distribution 2691 Network Interconnection (CDNI) Requirements", RFC 7337, 2692 DOI 10.17487/RFC7337, August 2014, 2693 . 2695 [RFC7736] Ma, K., "Content Delivery Network Interconnection (CDNI) 2696 Media Type Registration", RFC 7736, DOI 10.17487/RFC7736, 2697 December 2015, . 2699 Authors' Addresses 2701 Francois Le Faucheur (editor) 2703 Phone: +33 6 19 98 50 90 2704 Email: flefauch@gmail.com 2705 Gilles Bertrand (editor) 2706 Orange 2707 38-40 rue du General Leclerc 2708 Issy les Moulineaux 92130 2709 FR 2711 Phone: +33 1 45 29 89 46 2712 Email: gilles.bertrand@orange.com 2714 Iuniana Oprescu (editor) 2715 FR 2717 Email: iuniana.oprescu@gmail.com 2719 Roy Peterkofsky 2720 Google Inc. 2721 345 Spear St, 4th Floor 2722 San Francisco CA 94105 2723 USA 2725 Email: peterkofsky@google.com