idnits 2.17.1 draft-ietf-cdni-logging-27.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 8, 2016) is 2850 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFCthis' is mentioned on line 2375, but not defined -- Possible downref: Non-RFC (?) normative reference: ref. 'AES' -- Possible downref: Non-RFC (?) normative reference: ref. 'GCM' ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) ** Obsolete normative reference: RFC 7231 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7232 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7233 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7234 (Obsoleted by RFC 9111) ** Obsolete normative reference: RFC 7235 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7525 (Obsoleted by RFC 9325) ** Obsolete normative reference: RFC 7540 (Obsoleted by RFC 9113) -- Possible downref: Non-RFC (?) normative reference: ref. 'SHA-3' == Outdated reference: A later version (-21) exists of draft-ietf-cdni-metadata-17 -- Obsolete informational reference (is this intentional?): RFC 2818 (Obsoleted by RFC 9110) Summary: 9 errors (**), 0 flaws (~~), 3 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force F. Le Faucheur, Ed. 3 Internet-Draft 4 Intended status: Standards Track G. Bertrand, Ed. 5 Expires: December 10, 2016 6 I. Oprescu, Ed. 8 R. Peterkofsky 9 Google Inc. 10 June 8, 2016 12 CDNI Logging Interface 13 draft-ietf-cdni-logging-27 15 Abstract 17 This memo specifies the Logging interface between a downstream CDN 18 (dCDN) and an upstream CDN (uCDN) that are interconnected as per the 19 CDN Interconnection (CDNI) framework. First, it describes a 20 reference model for CDNI logging. Then, it specifies the CDNI 21 Logging File format and the actual protocol for exchange of CDNI 22 Logging Files. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at http://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on December 10, 2016. 41 Copyright Notice 43 Copyright (c) 2016 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (http://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 59 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 60 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 5 61 2. CDNI Logging Reference Model . . . . . . . . . . . . . . . . 5 62 2.1. CDNI Logging interactions . . . . . . . . . . . . . . . . 5 63 2.2. Overall Logging Chain . . . . . . . . . . . . . . . . . . 8 64 2.2.1. Logging Generation and During-Generation Aggregation 9 65 2.2.2. Logging Collection . . . . . . . . . . . . . . . . . 10 66 2.2.3. Logging Filtering . . . . . . . . . . . . . . . . . . 10 67 2.2.4. Logging Rectification and Post-Generation Aggregation 11 68 2.2.5. Log-Consuming Applications . . . . . . . . . . . . . 12 69 2.2.5.1. Maintenance/Debugging . . . . . . . . . . . . . . 12 70 2.2.5.2. Accounting . . . . . . . . . . . . . . . . . . . 13 71 2.2.5.3. Analytics and Reporting . . . . . . . . . . . . . 13 72 2.2.5.4. Content Protection . . . . . . . . . . . . . . . 13 73 2.2.5.5. Notions common to multiple Log Consuming 74 Applications . . . . . . . . . . . . . . . . . . 14 75 3. CDNI Logging File . . . . . . . . . . . . . . . . . . . . . . 16 76 3.1. Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 16 77 3.2. CDNI Logging File Structure . . . . . . . . . . . . . . . 17 78 3.3. CDNI Logging Directives . . . . . . . . . . . . . . . . . 20 79 3.4. CDNI Logging Records . . . . . . . . . . . . . . . . . . 24 80 3.4.1. HTTP Request Logging Record . . . . . . . . . . . . . 25 81 3.5. CDNI Logging File Extension . . . . . . . . . . . . . . . 36 82 3.6. CDNI Logging File Examples . . . . . . . . . . . . . . . 36 83 3.7. Cascaded CDNI Logging Files Example . . . . . . . . . . . 39 84 4. Protocol for Exchange of CDNI Logging File After Full 85 Collection . . . . . . . . . . . . . . . . . . . . . . . . . 42 86 4.1. CDNI Logging Feed . . . . . . . . . . . . . . . . . . . . 43 87 4.1.1. Atom Formatting . . . . . . . . . . . . . . . . . . . 43 88 4.1.2. Updates to Log Files and the Feed . . . . . . . . . . 43 89 4.1.3. Redundant Feeds . . . . . . . . . . . . . . . . . . . 44 90 4.1.4. Example CDNI Logging Feed . . . . . . . . . . . . . . 44 91 4.2. CDNI Logging File Pull . . . . . . . . . . . . . . . . . 46 92 5. Protocol for Exchange of CDNI Logging File During Collection 47 93 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 48 94 6.1. CDNI Logging Directive Names Registry . . . . . . . . . . 48 95 6.2. CDNI Logging File version Registry . . . . . . . . . . . 48 96 6.3. CDNI Logging record-types Registry . . . . . . . . . . . 49 97 6.4. CDNI Logging Field Names Registry . . . . . . . . . . . . 50 98 6.5. CDNI Logging MIME Media Type . . . . . . . . . . . . . . 51 99 7. Security Considerations . . . . . . . . . . . . . . . . . . . 52 100 7.1. Authentication, Authorization, Confidentiality, Integrity 101 Protection . . . . . . . . . . . . . . . . . . . . . . . 52 102 7.2. Denial of Service . . . . . . . . . . . . . . . . . . . . 53 103 7.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 53 104 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 55 105 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 55 106 9.1. Normative References . . . . . . . . . . . . . . . . . . 55 107 9.2. Informative References . . . . . . . . . . . . . . . . . 57 108 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 59 110 1. Introduction 112 This memo specifies the CDNI Logging interface between a downstream 113 CDN (dCDN) and an upstream CDN (uCDN). First, it describes a 114 reference model for CDNI logging. Then, it specifies the CDNI 115 Logging File format and the actual protocol for exchange of CDNI 116 Logging Files. 118 The reader should be familiar with the following documents: 120 o CDNI problem statement [RFC6707] and framework [RFC7336] identify 121 a Logging interface, 123 o Section 8 of [RFC7337] specifies a set of requirements for 124 Logging, 126 o [RFC6770] outlines real world use-cases for interconnecting CDNs. 127 These use cases require the exchange of Logging information 128 between the dCDN and the uCDN. 130 As stated in [RFC6707], "the CDNI Logging interface enables details 131 of logs or events to be exchanged between interconnected CDNs". 133 The present document describes: 135 o The CDNI Logging reference model (Section 2), 137 o The CDNI Logging File format (Section 3), 139 o The CDNI Logging File Exchange protocol (Section 4). 141 1.1. Terminology 143 In this document, the first letter of each CDNI-specific term is 144 capitalized. We adopt the terminology described in [RFC6707] and 145 [RFC7336], and extend it with the additional terms defined below. 147 Intra-CDN Logging information: logging information generated and 148 collected within a CDN. The format of the Intra-CDN Logging 149 information may be different to the format of the CDNI Logging 150 information. 152 CDNI Logging information: logging information exchanged across CDNs 153 using the CDNI Logging Interface. 155 Logging information: logging information generated and collected 156 within a CDN or obtained from another CDN using the CDNI Logging 157 Interface. 159 CDNI Logging Field: an atomic element of information that can be 160 included in a CDNI Logging Record. The time an event/task started, 161 the IP address of an End User to whom content was delivered, and the 162 Uniform Resource Identifier (URI) of the content delivered, are 163 examples of CDNI Logging fields. 165 CDNI Logging Record: an information record providing information 166 about a specific event. This comprises a collection of CDNI Logging 167 fields. 169 CDNI Logging File: a file containing CDNI Logging Records, as well as 170 additional information facilitating the processing of the CDNI 171 Logging Records. 173 CDN Reporting: the process of providing the relevant information that 174 will be used to create a formatted content delivery report provided 175 to the CSP in deferred time. Such information typically includes 176 aggregated data that can cover a large period of time (e.g., from 177 hours to several months). Uses of Reporting include the collection 178 of charging data related to CDN services and the computation of Key 179 Performance Indicators (KPIs). 181 CDN Monitoring: the process of providing or displaying content 182 delivery information in a timely fashion with respect to the 183 corresponding deliveries. Monitoring typically includes visibility 184 of the deliveries in progress for service operation purposes. It 185 presents a view of the global health of the services as well as 186 information on usage and performance, for network services 187 supervision and operation management. In particular, monitoring data 188 can be used to generate alarms. 190 1.2. Requirements Language 192 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 193 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 194 "OPTIONAL" in this document are to be interpreted as described in RFC 195 2119 [RFC2119]. 197 2. CDNI Logging Reference Model 199 2.1. CDNI Logging interactions 201 The CDNI logging reference model between a given uCDN and a given 202 dCDN involves the following interactions: 204 o customization by the uCDN of the CDNI Logging information to be 205 provided by the dCDN to the uCDN (e.g., control of which CDNI 206 Logging fields are to be communicated to the uCDN for a given task 207 performed by the dCDN or control of which types of events are to 208 be logged). The dCDN takes into account this CDNI Logging 209 customization information to determine what Logging information to 210 provide to the uCDN, but it may, or may not, take into account 211 this CDNI Logging customization information to influence what CDN 212 logging information is to be generated and collected within the 213 dCDN (e.g., even if the uCDN requests a restricted subset of the 214 logging information, the dCDN may elect to generate a broader set 215 of logging information). The mechanism to support the 216 customization by the uCDN of CDNI Logging information is outside 217 the scope of this document and left for further study. Until such 218 a mechanism is available, the uCDN and dCDN are expected to agree 219 off-line on what exact set of CDNI Logging information is to be 220 provided by the dCDN to the uCDN, and to rely on management plane 221 actions to configure the CDNI Logging functions in the dCDN to 222 generate this information set and in the uCDN to expect this 223 information set. 225 o generation and collection by the dCDN of the intra-CDN Logging 226 information related to the completion of any task performed by the 227 dCDN on behalf of the uCDN (e.g., delivery of the content to an 228 End User) or related to events happening in the dCDN that are 229 relevant to the uCDN (e.g., failures or unavailability in dCDN). 230 This takes place within the dCDN and does not directly involve 231 CDNI interfaces. 233 o communication by the dCDN to the uCDN of the Logging information 234 collected by the dCDN relevant to the uCDN. This is supported by 235 the CDNI Logging interface and in the scope of the present 236 document. For example, the uCDN may use this Logging information 237 to charge the CSP, to perform analytics and monitoring for 238 operational reasons, to provide analytics and monitoring views on 239 its content delivery to the CSP or to perform trouble-shooting. 240 This document exclusively specifies non-real-time exchange of 241 Logging information. Closer to real-time exchange of Logging 242 information (say sub-minute or sub-second) is outside the scope of 243 the present document and left for further study. This document 244 exclusively specifies exchange of Logging information related to 245 content delivery. Exchange of Logging information related to 246 operational events (e.g., dCDN request routing function 247 unavailable, content acquisition failure by dCDN) for audit or 248 operational reactive adjustments by uCDN is outside the scope of 249 the present document and left for further study. 251 o customization by the dCDN of the CDNI Logging information to be 252 provided by the uCDN on behalf of the dCDN. The mechanism to 253 support the customization by the dCDN of CDNI Logging information 254 is outside the scope of this document and left for further study. 256 o generation and collection by the uCDN of Intra-CDN Logging 257 information related to the completion of any task performed by the 258 uCDN on behalf of the dCDN (e.g., serving of content by uCDN to 259 dCDN for acquisition purposes by dCDN) or related to events 260 happening in the uCDN that are relevant to the dCDN. This takes 261 place within the uCDN and does not directly involve CDNI 262 interfaces. 264 o communication by the uCDN to the dCDN of the Logging information 265 collected by the uCDN relevant to the dCDN. For example, the dCDN 266 might potentially benefit from this information for security 267 auditing or content acquisition troubleshooting. This is outside 268 the scope of this document and left for further study. 270 Figure 1 provides an example of CDNI Logging interactions (focusing 271 only on the interactions that are in the scope of this document) in a 272 particular scenario where four CDNs are involved in the delivery of 273 content from a given CSP: the uCDN has a CDNI interconnection with 274 dCDN-1 and dCDN-2. In turn, dCDN-2 has a CDNI interconnection with 275 dCDN-3, where dCDN-2 is acting as an upstream CDN relative to dCDN-3. 276 In this example, uCDN, dCDN-1, dCDN-2 and dCDN-3 all participate in 277 the delivery of content for the CSP. In this example, the CDNI 278 Logging interface enables the uCDN to obtain Logging information from 279 all the dCDNs involved in the delivery. In the example, the uCDN 280 uses the Logging information: 282 o to analyze the performance of the delivery performed by the dCDNs 283 and to adjust its operations after the fact (e.g., request 284 routing) as appropriate, 286 o to provide (non-real-time) reporting and monitoring information to 287 the CSP. 289 For instance, the uCDN merges Logging information, extracts relevant 290 KPIs, and presents a formatted report to the CSP, in addition to a 291 bill for the content delivered by uCDN itself or by its dCDNs on the 292 CSP's behalf. The uCDN may also provide Logging information as raw 293 log files to the CSP, so that the CSP can use its own logging 294 analysis tools. 296 +-----+ 297 | CSP | 298 +-----+ 299 ^ Reporting and monitoring data 300 * Billing 301 ,--*--. 302 Logging ,-' `-. 303 Data =>( uCDN )<= Logging 304 // `-. _,-' \\ Data 305 || `-'-'-' || 306 ,-----. ,-----. 307 ,-' `-. ,-' `-. 308 ( dCDN-1 ) ( dCDN-2 )<== Logging 309 `-. ,-' `-. _,-' \\ Data 310 `--'--' `--'-' || 311 ,-----. 312 ,' `-. 313 ( dCDN-3 ) 314 `. ,-' 315 `--'--' 317 ===> CDNI Logging Interface 318 ***> outside the scope of CDNI 320 Figure 1: Interactions in CDNI Logging Reference Model 322 A downstream CDN relative to uCDN (e.g., dCDN-2) integrates the 323 relevant Logging information obtained from its own downstream CDNs 324 (i.e., dCDN-3) in the Logging information that it provides to the 325 uCDN, so that the uCDN ultimately obtains all Logging information 326 relevant to a CSP for which it acts as the authoritative CDN. Such 327 aggregation is further discussed in Section 3.7. 329 Note that the format of Logging information that a CDN provides over 330 the CDNI interface might be different from the one that the CDN uses 331 internally. In this case, the CDN needs to reformat the Logging 332 information before it provides this information to the other CDN over 333 the CDNI Logging interface. Similarly, a CDN might reformat the 334 Logging information that it receives over the CDNI Logging interface 335 before injecting it into its log-consuming applications or before 336 providing some of this Logging information to the CSP. Such 337 reformatting operations introduce latency in the logging distribution 338 chain and introduce a processing burden. Therefore, there are 339 benefits in specifying CDNI Logging formats that are suitable for use 340 inside CDNs and also are close to the intra-CDN Logging formats 341 commonly used in CDNs today. 343 2.2. Overall Logging Chain 345 This section discusses the overall logging chain within and across 346 CDNs to clarify how CDN Logging information is expected to fit in 347 this overall chain. Figure 2 illustrates the overall logging chain 348 within the dCDN, across CDNs using the CDNI Logging interface and 349 within the uCDN. Note that the logging chain illustrated in the 350 Figure is obviously only an example and varies depending on the 351 specific environments. For example, there may be more or fewer 352 instantiations of each entity (e.g., there may be 4 Log consuming 353 applications in a given CDN). As another example, there may be one 354 instance of Rectification process per Log Consuming Application 355 instead of a shared one. 357 Log Consuming Log Consuming 358 App App 359 ^ ^ 360 | | 361 Rectification---------- 362 ^ 363 | 364 Filtering 365 ^ 366 | 367 Collection 368 ^ ^ 369 | | 370 | Generation 371 | 372 | uCDN 373 CDNI Logging --------------------------------------------------- 374 exchange dCDN 375 ^ 376 | Log Consuming Log Consuming 377 | App App 378 | ^ ^ 379 | | | 380 Rectification Rectification--------- 381 ^ ^ 382 | | 383 Filtering 384 ^ 385 | 386 Collection 387 ^ ^ 388 | | 389 Generation Generation 391 Figure 2: CDNI Logging in the overall Logging Chain 393 The following subsections describe each of the processes potentially 394 involved in the logging chain of Figure 2. 396 2.2.1. Logging Generation and During-Generation Aggregation 398 CDNs typically generate Logging information for all significant task 399 completions, events, and failures. Logging information is typically 400 generated by many devices in the CDN including the surrogates, the 401 request routing system, and the control system. 403 The amount of Logging information generated can be huge. Therefore, 404 during contract negotiations, interconnected CDNs often agree on a 405 retention duration for Logging information, and/or potentially on a 406 maximum volume of Logging information that the dCDN ought to keep. 407 If this volume is exceeded, the dCDN is expected to alert the uCDN 408 but may not keep more Logging information for the considered time 409 period. In addition, CDNs may aggregate Logging information and 410 transmit only summaries for some categories of operations instead of 411 the full Logging information. Note that such aggregation leads to an 412 information loss, which may be problematic for some usages of the 413 Logging information (e.g., debugging). 415 [RFC6983] discusses logging for HTTP Adaptive Streaming (HAS). In 416 accordance with the recommendations articulated there, it is expected 417 that a surrogate will generate separate Logging information for 418 delivery of each chunk of HAS content. This ensures that separate 419 Logging information can then be provided to interconnected CDNs over 420 the CDNI Logging interface. Still in line with the recommendations 421 of [RFC6983], the Logging information for per-chunk delivery may 422 include some information (a Content Collection IDentifier and a 423 Session IDentifier) intended to facilitate subsequent post-generation 424 aggregation of per-chunk logs into per-session logs. Note that a CDN 425 may also elect to generate aggregate per-session logs when performing 426 HAS delivery, but this needs to be in addition to, and not instead 427 of, the per-chunk delivery logs. We note that aggregate per-session 428 logs for HAS delivery are for further study and outside the scope of 429 this document. 431 2.2.2. Logging Collection 433 This is the process that continuously collects Logging information 434 generated by the log-generating entities within a CDN. 436 In a CDNI environment, in addition to collecting Logging information 437 from log-generating entities within the local CDN, the Collection 438 process also collects Logging information provided by another CDN, or 439 other CDNs, through the CDNI Logging interface. This is illustrated 440 in Figure 2 where we see that the Collection process of the uCDN 441 collects Logging information from log-generating entities within the 442 uCDN as well as Logging information coming from the dCDNs through the 443 CDNI Logging interface. 445 2.2.3. Logging Filtering 447 A CDN may be required to only present different subsets of the whole 448 Logging information collected to various log-consuming applications. 449 This is achieved by the Filtering process. 451 In particular, the Filtering process can also filter the right subset 452 of Logging information that needs to be provided to a given 453 interconnected CDN. For example, the filtering process in the dCDN 454 can be used to ensure that only the Logging information related to 455 tasks performed on behalf of a given uCDN are made available to that 456 uCDN (thereby filtering out all the Logging information related to 457 deliveries by the dCDN of content for its own CSPs). Similarly, the 458 Filtering process may filter or partially mask some fields, for 459 example, to protect End Users' privacy when communicating CDNI 460 Logging information to another CDN. Filtering of Logging information 461 prior to communication of this information to other CDNs via the CDNI 462 Logging interface requires that the downstream CDN can recognize the 463 subset of Logging information that relate to each interconnected CDN. 465 The CDN will also filter some internal scope information such as 466 information related to its internal alarms (security, failures, load, 467 etc). 469 In some use cases described in [RFC6770], the interconnected CDNs do 470 not want to disclose details on their internal topology. The 471 filtering process can then also filter confidential data on the 472 dCDNs' topology (number of servers, location, etc.). In particular, 473 information about the requests served by each Surrogate may be 474 confidential. Therefore, the Logging information needs to be 475 protected so that data such as Surrogates' hostnames are not 476 disclosed to the uCDN. In the "Inter-Affiliates Interconnection" use 477 case, this information may be disclosed to the uCDN because both the 478 dCDN and the uCDN are operated by entities of the same group. 480 2.2.4. Logging Rectification and Post-Generation Aggregation 482 If Logging information is generated periodically, it is important 483 that the sessions that start in one Logging period and end in another 484 are correctly reported. If they are reported in the starting period, 485 then the Logging information of this period will be available only 486 after the end of the session, which delays the Logging information 487 generation. A simple approach is to provide the complete Logging 488 Record for a session in the Logging Period of the session end. 490 A Logging rectification/update mechanism could be useful to reach a 491 good trade-off between the Logging information generation delay and 492 the Logging information accuracy. 494 In the presence of HAS, some log-consuming applications can benefit 495 from aggregate per-session logs. For example, for analytics, per- 496 session logs allow display of session-related trends which are much 497 more meaningful for some types of analysis than chunk-related trends. 498 In the case where aggregate logs have been generated directly by the 499 log-generating entities, those can be used by the applications. In 500 the case where aggregate logs have not been generated, the 501 Rectification process can be extended with a Post-Generation 502 Aggregation process that generates per-session logs from the per- 503 chunk logs, possibly leveraging the information included in the per- 504 chunk logs for that purpose (Content Collection IDentifier and a 505 Session IDentifier). However, in accordance with [RFC6983], this 506 document does not define exchange of such aggregate logs on the CDNI 507 Logging interface. We note that this is for further study and 508 outside the scope of this document. 510 2.2.5. Log-Consuming Applications 512 2.2.5.1. Maintenance/Debugging 514 Logging information is useful to permit the detection (and limit the 515 risk) of content delivery failures. In particular, Logging 516 information facilitates the detection of configuration issues. 518 To detect faults, Logging information needs to report success and 519 failure of CDN delivery operations. The uCDN can summarize such 520 information into KPIs. For instance, Logging information needs to 521 allow the computation of the number of times, during a given time 522 period, that content delivery related to a specific service succeeds/ 523 fails. 525 Logging information enables the CDN providers to identify and 526 troubleshoot performance degradations. In particular, Logging 527 information enables tracking of traffic data (e.g., the amount of 528 traffic that has been forwarded by a dCDN on behalf of an uCDN over a 529 given period of time), which is particularly useful for CDN and 530 network planning operations. 532 Some of these maintenance and debugging applications only require 533 aggregate logging information highly compatible with use of 534 anonymization of IP addresses (as supported by the present document 535 and specified in the definition of the c-groupid field under 536 Section 3.4.1). However, in some situations, it may be useful, where 537 compatible with privacy protection, to access some CDNI Logging 538 Records containing full non-anonymized IP addresses. This is allowed 539 in the definition of the c-groupid (under Section 3.4.1), with very 540 significant privacy protection limitations that are discussed in the 541 definition of the c-groupid field. For example, this may be useful 542 for detailed fault tracking of a particular end user content delivery 543 issue. Where there is a hard requirement by uCDN or CSP to associate 544 a given enduser to individual CDNI Logging Records (e.g., to allow 545 a-posteriori analysis of individual delivery for example in 546 situations of performance-based penalties), instead of using 547 aggregates containing a single client as discussed in the c-groupid 548 field definition, an alternate approach is to ensure that a client 549 identifier is embedded in the request fields that can be logged in a 550 CDNI Logging Record (for example by including the client identifier 551 in the URI query string or in a HTTP Header). That latter approach 552 offers two strong benefits: first, the aggregate inside the c-groupid 553 can contain more than one client, thereby ensuring stronger privacy 554 protection; second, it allows a reliable identification of the client 555 while IP address does not in many situations (e.g., behind NAT, where 556 dynamic IP addresses are used and reused,...). However, care SHOULD 557 be taken that the client identifiers exposed in other fields of the 558 CDNI Records cannot themselves be linked back to actual users. 560 2.2.5.2. Accounting 562 Logging information is essential for accounting, to permit inter-CDN 563 billing and CSP billing by uCDNs. For instance, Logging information 564 provided by dCDNs enables the uCDN to compute the total amount of 565 traffic delivered by every dCDN for a particular Content Provider, as 566 well as, the associated bandwidth usage (e.g., peak, 95th 567 percentile), and the maximum number of simultaneous sessions over a 568 given period of time. 570 2.2.5.3. Analytics and Reporting 572 The goals of analytics include gathering any relevant information in 573 order to be able to develop statistics on content download, analyze 574 user behavior, and monitor the performance and quality of content 575 delivery. For instance, Logging information enables the CDN 576 providers to report on content consumption (e.g., delivered sessions 577 per content) in a specific geographic area. 579 The goal of reporting is to gather any relevant information to 580 monitor the performance and quality of content delivery and allow 581 detection of delivery issues. For instance, reporting could track 582 the average delivery throughput experienced by End Users in a given 583 region for a specific CSP or content set over a period of time. 585 2.2.5.4. Content Protection 587 The goal of content protection is to prevent and monitor unauthorized 588 access, misuse, modification, and denial of access to a content. A 589 set of information is logged in a CDN for security purposes. In 590 particular, a record of access to content is usually collected to 591 permit the CSP to detect infringements of content delivery policies 592 and other abnormal End User behaviors. 594 2.2.5.5. Notions common to multiple Log Consuming Applications 596 2.2.5.5.1. Logging Information Views 598 Within a given log-consuming application, different views may be 599 provided to different users depending on privacy, business, and 600 scalability constraints. 602 For example, an analytics tool run by the uCDN can provide one view 603 to an uCDN operator that exploits all the Logging information 604 available to the uCDN, while the tool may provide a different view to 605 each CSP exploiting only the Logging information related to the 606 content of the given CSP. 608 As another example, maintenance and debugging tools may provide 609 different views to different CDN operators, based on their 610 operational role. 612 2.2.5.5.2. Key Performance Indicators (KPIs) 614 This section presents, for explanatory purposes, a non-exhaustive 615 list of Key Performance Indicators (KPIs) that can be extracted/ 616 produced from logs. 618 Multiple log-consuming applications, such as analytics, monitoring, 619 and maintenance applications, often compute and track such KPIs. 621 In a CDNI environment, depending on the situation, these KPIs may be 622 computed by the uCDN or by the dCDN. But it is usually the uCDN that 623 computes KPIs, because the uCDN and dCDN may have different 624 definitions of the KPIs and the computation of some KPIs requires a 625 vision of all the deliveries performed by the uCDN and all its dCDNs. 627 Here is a list of important examples of KPIs: 629 o Number of delivery requests received from End Users in a given 630 region for each piece of content, during a given period of time 631 (e.g., hour/day/week/month) 633 o Percentage of delivery successes/failures among the aforementioned 634 requests 636 o Number of failures listed by failure type (e.g., HTTP error code) 637 for requests received from End Users in a given region and for 638 each piece of content, during a given period of time (e.g., 639 hour/day/week/month) 641 o Number and cause of premature delivery termination for End Users 642 in a given region and for each piece of content, during a given 643 period of time (e.g., hour/day/week/month) 645 o Maximum and mean number of simultaneous sessions established by 646 End Users in a given region, for a given Content Provider, and 647 during a given period of time (e.g., hour/day/week/month) 649 o Volume of traffic delivered for sessions established by End Users 650 in a given region, for a given Content Provider, and during a 651 given period of time (e.g., hour/day/week/month) 653 o Maximum, mean, and minimum delivery throughput for sessions 654 established by End Users in a given region, for a given Content 655 Provider, and during a given period of time (e.g., hour/day/week/ 656 month) 658 o Cache-hit and byte-hit ratios for requests received from End Users 659 in a given region for each piece of content, during a given period 660 of time (e.g., hour/day/week/month) 662 o Top 10 most popularly requested contents (during a given day/week/ 663 month) 665 o Terminal type (mobile, PC, STB, if this information can be 666 acquired from the browser type inferred from the User Agent 667 string, for example). 669 Additional KPIs can be computed from other sources of information 670 than the Logging information, for instance, data collected by a 671 content portal or by specific client-side application programming 672 interfaces. Such KPIs are out of scope for the present document. 674 The KPIs used depend strongly on the considered log-consuming 675 application -- the CDN operator may be interested in different 676 metrics than the CSP is. In particular, CDN operators are often 677 interested in delivery and acquisition performance KPIs, information 678 related to Surrogates' performance, caching information to evaluate 679 the cache-hit ratio, information about the delivered file size to 680 compute the volume of content delivered during peak hour, etc. 682 Some of the KPIs, for instance those providing an instantaneous 683 vision of the active sessions for a given CSP's content, are useful 684 essentially if they are provided in a timely manner. By contrast, 685 some other KPIs, such as those averaged on a long period of time, can 686 be provided in non-real-time. 688 3. CDNI Logging File 690 3.1. Rules 692 This specification uses the Augmented Backus-Naur Form (ABNF) 693 notation and core rules of [RFC5234]. In particular, the present 694 document uses the following rules from [RFC5234]: 696 CR = %x0D ; carriage return 698 ALPHA = %x41-5A / %x61-7A ; A-Z / a-z 700 DIGIT = %x30-39 ; 0-9 702 DQUOTE = %x22 ; " (Double Quote) 704 CRLF = CR LF ; Internet standard newline 706 HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F" 708 HTAB = %x09 ; horizontal tab 710 LF = %x0A ; linefeed 712 VCHAR = %x21-7E ; visible (printing) characters 714 OCTET = %x00-FF ; 8 bits of data 716 The present document also uses the following rules from [RFC3986]: 718 host = as specified in section 3.2.2 of [RFC3986]. 720 IPv4address = as specified in section 3.2.2 of [RFC3986]. 722 IPv6address = as specified in section 3.2.2 of [RFC3986]. 724 partial-time = as specified in [RFC3339]. 726 The present document also defines the following additional rules: 728 ADDRESS = IPv4address / IPv6address 730 ALPHANUM = ALPHA / DIGIT 732 DATE = 4DIGIT "-" 2DIGIT "-" 2DIGIT 734 ; Dates are encoded as "full-date" specified in [RFC3339]. 736 DEC = 1*DIGIT ["." 1*DIGIT] 738 NAMEFORMAT = ALPHANUM *(ALPHANUM / "_" / "-") 740 QSTRING = DQUOTE *(NDQUOTE / PCT-ENCODED) DQUOTE 742 NDQUOTE = %x20-21 / %x23-24 / %x26-7E / UTF8-2 / UTF8-3 / UTF8-4 744 ; whereby a DQUOTE is conveyed inside a QSTRING unambiguously 745 by escaping it with PCT-ENCODED. 747 PCT-ENCODED = "%" HEXDIG HEXDIG 749 ; percent encoding is used for escaping octets that might be 750 possible in HTTP headers such as bare CR, bare LF, CR LF, HTAB, 751 SP or null. These octets are rendered with percent encoding in 752 ABNF as specified by [RFC3986] in order to avoid considering 753 them as separators for the logging records. 755 NHTABSTRING = 1*(SP / VCHAR) 757 TIME = partial-time 759 USER-COMMENT = * (SP / VCHAR / UTF8-2 / UTF8-3 / UTF8-4) 761 3.2. CDNI Logging File Structure 763 As defined in Section 1.1: a CDNI Logging Field is as an atomic 764 logging information element, a CDNI Logging Record is a collection of 765 CDNI Logging fields containing all logging information corresponding 766 to a single logging event, and a CDNI Logging File contains a 767 collection of CDNI Logging Records. This structure is illustrated in 768 Figure 3. The use of a file structure for transfer of CDNI Logging 769 information is selected since this is the most common practise today 770 for exchange of logging information within and across CDNs. 772 +----------------------------------------------------------+ 773 |CDNI Logging File | 774 | | 775 | #Directive 1 | 776 | #Directive 2 | 777 | ... | 778 | #Directive P | 779 | | 780 | +------------------------------------------------------+ | 781 | |CDNI Logging Record 1 | | 782 | | +-------------+ +-------------+ +-------------+ | | 783 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 784 | | | Field 1 | | Field 2 | | Field N | | | 785 | | +-------------+ +-------------+ +-------------+ | | 786 | +------------------------------------------------------+ | 787 | | 788 | +------------------------------------------------------+ | 789 | |CDNI Logging Record 2 | | 790 | | +-------------+ +-------------+ +-------------+ | | 791 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 792 | | | Field 1 | | Field 2 | | Field N | | | 793 | | +-------------+ +-------------+ +-------------+ | | 794 | +------------------------------------------------------+ | 795 | | 796 | ... | 797 | | 798 | #Directive P+1 | 799 | | 800 | ... | 801 | | 802 | +------------------------------------------------------+ | 803 | |CDNI Logging Record M | | 804 | | +-------------+ +-------------+ +-------------+ | | 805 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 806 | | | Field 1 | | Field 2 | | Field N | | | 807 | | +-------------+ +-------------+ +-------------+ | | 808 | +------------------------------------------------------+ | 809 | | 810 | | 811 | #Directive P+Q | 812 +----------------------------------------------------------+ 814 Figure 3: Structure of Logging Files 816 The CDNI Logging File format is inspired from the W3C Extended Log 817 File Format [ELF]. However, it is fully specified by the present 818 document. Where the present document differs from the W3C Extended 819 Log File Format, an implementation of the CDNI Logging interface MUST 820 comply with the present document. The W3C Extended Log File Format 821 was used as a starting point, reused where possible and expanded when 822 necessary. 824 Using a format that resembles the W3C Extended Log File Format is 825 intended to keep CDNI logging format close to the intra-CDN Logging 826 information format commonly used in CDNs today, thereby minimizing 827 systematic translation at CDN/CDNI boundary. 829 A CDNI Logging File MUST contain a sequence of lines containing US- 830 ASCII characters [CHAR_SET] terminated by CRLF. Each line of a CDNI 831 Logging File MUST contain either a directive or a CDNI Logging 832 Record. 834 Directives record information about the CDNI Logging process itself. 835 Lines containing directives MUST begin with the "#" character. 836 Directives are specified in Section 3.3. 838 Logging Records provide actual details of the logged event. Logging 839 Records are specified in Section 3.4. 841 The CDNI Logging File has a specific structure. It always starts 842 with a directive line and the first directive it contains MUST be the 843 version. 845 The directive lines form together a group that contains at least one 846 directive line. Each directives group is followed by a group of 847 logging records. The records group contains zero or more actual 848 logging record lines about the event being logged. A record line 849 consists of the values corresponding to all or a subset of the 850 possible Logging fields defined within the scope of the record-type 851 directive. These values MUST appear in the order defined by the 852 fields directive. 854 Note that future extensions MUST be compliant with the previous 855 description. The following examples depict the structure of a 856 CDNILOGFILE as defined currently by the record-type 857 "cdni_http_request_v1." 859 DIRLINE = "#" directive CRLF 861 DIRGROUP = 1*DIRLINE 863 RECLINE = 866 RECGROUP = *RECLINE 868 CDNILOGFILE = 1*(DIRGROUP RECGROUP) 870 3.3. CDNI Logging Directives 872 A CDNI Logging directive line contains the directive name followed by 873 ":" HTAB and the directive value. 875 Directive names MUST be of the format NAMEFORMAT. All directive 876 names MUST be registered in the CDNI Logging Directives Names 877 registry. Directive names are case-insensitive as per the basic 878 ABNF([RFC5234]). Unknown directives MUST be ignored. Directive 879 values can have various formats. All possible directive values for 880 the record-type "cdni_http_request_v1" are further detailed in this 881 section. 883 The following example shows the structure of a directive and 884 enumerates strictly the directive values presently defined in the 885 version "cdni/1.0" of the CDNI Logging File. 887 directive = DIRNAME ":" HTAB DIRVAL 889 DIRNAME = NAMEFORMAT 891 DIRVAL = NHTABSTRING / QSTRING / host / USER-COMMENT / FIENAME * 892 (HTAB FIENAME) / 64HEXDIG 894 An implementation of the CDNI Logging interface MUST support all of 895 the following directives, listed below by their directive name: 897 o version: 899 * format: NHTABSTRING 901 * directive value: indicates the version of the CDNI Logging File 902 format. The entity transmitting a CDNI Logging File as per the 903 present document MUST set the value to "cdni/1.0". In the 904 future, other versions of CDNI Logging File might be specified; 905 those would use a value different to "cdni/1.0" allowing the 906 entity receiving the CDNI Logging File to identify the 907 corresponding version. CDNI Logging File versions are case- 908 insensitive as per the basic ABNF([RFC5234]). 910 * occurrence: there MUST be one and only one instance of this 911 directive per CDNI Logging File. It MUST be the first line of 912 the CDNI Logging File. 914 * example: "version: HTAB cdni/1.0". 916 o UUID: 918 * format: NHTABSTRING 920 * directive value: this a Uniform Resource Name (URN) from the 921 Universally Unique IDentifier (UUID) URN namespace specified in 922 [RFC4122]). The UUID contained in the URN uniquely identifies 923 the CDNI Logging File. 925 * occurrence: there MUST be one and only one instance of this 926 directive per CDNI Logging File. 928 * example: "UUID: HTAB NHTABSTRING". 930 o claimed-origin: 932 * format: host 934 * directive value: this contains the claimed identification of 935 the entity transmitting the CDNI Logging File (e.g., the host 936 in a dCDN supporting the CDNI Logging interface) or the entity 937 responsible for transmitting the CDNI Logging File (e.g., the 938 dCDN). 940 * occurrence: there MUST be zero or exactly one instance of this 941 directive per CDNI Logging File. This directive MAY be 942 included by the dCDN. It MUST NOT be included or modified by 943 the uCDN. 945 * example: "claimed-origin: HTAB host". 947 o established-origin: 949 * format: host 951 * directive value: this contains the identification, as 952 established by the entity receiving the CDNI Logging File, of 953 the entity transmitting the CDNI Logging File (e.g., the host 954 in a dCDN supporting the CDNI Logging interface) or the entity 955 responsible for transmitting the CDNI Logging File (e.g., the 956 dCDN). 958 * occurrence: there MUST be zero or exactly one instance of this 959 directive per CDNI Logging File. This directive MAY be added 960 by the uCDN (e.g., before storing the CDNI Logging File). It 961 MUST NOT be included by the dCDN. The mechanisms used by the 962 uCDN to establish and validate the entity responsible for the 963 CDNI Logging File is outside the scope of the present document. 964 We observe that, in particular, this may be achieved through 965 authentication mechanisms that are part of the transport layer 966 of the CDNI Logging File pull mechanism (Section 4.2). 968 * ABNF example: "established-origin: HTAB host". 970 o remark: 972 * format: USER-COMMENT 974 * directive value: this contains comment information. Data 975 contained in this field is to be ignored by analysis tools. 977 * occurrence: there MAY be zero, one or any number of instance of 978 this directive per CDNI Logging File. 980 * example: "remark: HTAB USER-COMMENT". 982 o record-type: 984 * format: NAMEFORMAT 986 * directive value: indicates the type of the CDNI Logging Records 987 that follow this directive, until another record-type directive 988 (or the end of the CDNI Logging File). This can be any CDNI 989 Logging Record type registered in the CDNI Logging Record-types 990 registry (Section 6.3). For example this may be 991 "cdni_http_request_v1" as specified in Section 3.4.1. CDNI 992 Logging record-types are case-insensitive as per the basic 993 ABNF([RFC5234]). 995 * occurrence: there MUST be at least one instance of this 996 directive per CDNI Logging File. The first instance of this 997 directive MUST precede a fields directive and MUST precede all 998 CDNI Logging Records. 1000 * example: "record-type: HTAB cdni_http_request_v1". 1002 o fields: 1004 * format: FIENAME *(HTAB FIENAME) ; where FIENAME can take any 1005 CDNI Logging field name registered in the CDNI Logging Field 1006 Names registry (Section 6.4) that is valid for the record type 1007 specified in the record-type directive. 1009 * directive value: this lists the names of all the fields for 1010 which a value is to appear in the CDNI Logging Records that 1011 follow the instance of this directive (until another instance 1012 of this directive). The names of the fields, as well as their 1013 occurrences, MUST comply with the corresponding rules specified 1014 in the document referenced in the CDNI Logging Record-types 1015 registry (Section 6.3) for the corresponding CDNI Logging 1016 record-type. 1018 * occurrence: there MUST be at least one instance of this 1019 directive per record-type directive. The first instance of 1020 this directive for a given record-type MUST appear before any 1021 CDNI Logging Record for this record-type. One situation where 1022 more than one instance of the fields directive can appear 1023 within a given CDNI Logging File, is when there is a change, in 1024 the middle of a fairly large logging period, in the agreement 1025 between the uCDN and the dCDN about the set of fields that are 1026 to be exchanged. The multiple occurrences allow records with 1027 the old set of fields and records with the new set of fields to 1028 be carried inside the same Logging File. 1030 * example: "fields: HTAB FIENAME * (HTAB FIENAME)". 1032 o SHA256-hash: 1034 * format: 64HEXDIG 1036 * directive value: This directive permits the detection of a 1037 corrupted CDNI Logging File. This can be useful, for instance, 1038 if a problem occurs on the filesystem of the dCDN Logging 1039 system and leads to a truncation of a logging file. The valid 1040 SHA256-hash value is included in this directive by the entity 1041 that transmits the CDNI Logging File. It MUST be computed by 1042 applying the SHA-256 ([RFC6234]) cryptographic hash function on 1043 the CDNI Logging File, including all the directives and logging 1044 records, up to the SHA256-hash directive itself, excluding the 1045 SHA256-hash directive itself. The SHA256-hash value MUST be 1046 represented as a US-ASCII encoded hexadecimal number, 64 digits 1047 long (representing a 256 bit hash value). The entity receiving 1048 the CDNI Logging File also computes in a similar way the 1049 SHA-256 hash on the received CDNI Logging File and compares 1050 this hash to the value of the SHA256-hash directive. If the 1051 two values are equal, then the received CDNI Logging File is to 1052 be considered non-corrupted. If the two values are different, 1053 the received CDNI Logging File is to be considered corrupted. 1054 The behavior of the entity that received a corrupted CDNI 1055 Logging File is outside the scope of this specification; we 1056 note that the entity MAY attempt to pull again the same CDNI 1057 Logging File from the transmitting entity. If the entity 1058 receiving a non-corrupted CDNI Logging File adds an 1059 established-origin directive, it MUST then recompute and update 1060 the SHA256-hash directive so it also protects the added 1061 established-origin directive. 1063 * occurrence: there MUST be zero or exactly one instance of this 1064 directive. There SHOULD be exactly one instance of this 1065 directive. One situation where that directive could be omitted 1066 is where integrity protection is already provided via another 1067 mechanism (for example if an integrity hash is associated to 1068 the CDNI Logging File out-of-band through the CDNI Logging Feed 1069 ( Section 4.1) leveraging ATOM extensions such as those 1070 proposed in [I-D.snell-atompub-link-extensions]. When present, 1071 the SHA256-hash field MUST be the last line of the CDNI Logging 1072 File. 1074 * example: "SHA256-hash: HTAB 64HEXDIG". 1076 An uCDN-side implementation of the CDNI Logging interface MUST ignore 1077 a CDNI Logging File that does not comply with the occurrences 1078 specified above for each and every directive. For example, an uCDN- 1079 side implementation of the CDNI Logging interface receiving a CDNI 1080 Logging file with zero occurrence of the version directive, or with 1081 two occurrences of the SHA256-hash, MUST ignore this CDNI Logging 1082 File. 1084 An entity receiving a CDNI Logging File with a value set to 1085 "cdni/1.0" MUST process the CDNI Logging File as per the present 1086 document. An entity receiving a CDNI Logging File with a value set 1087 to a different value MUST process the CDNI Logging File as per the 1088 specification referenced in the CDNI Logging File version registry 1089 (see Section 6.1) if the implementation supports this specification 1090 and MUST ignore the CDNI Logging File otherwise. 1092 3.4. CDNI Logging Records 1094 A CDNI Logging Record consists of a sequence of CDNI Logging fields 1095 relating to that single CDNI Logging Record. 1097 CDNI Logging fields MUST be separated by the "horizontal tabulation 1098 (HTAB)" character. 1100 To facilitate readability, a prefix scheme is used for CDNI Logging 1101 field names in a similar way to the one used in W3C Extended Log File 1102 Format [ELF]. The semantics of the prefix in the present document 1103 is: 1105 o "c-" refers to the User Agent that issues the request (corresponds 1106 to the "client" of W3C Extended Log Format) 1108 o "d-" refers to the dCDN (relative to a given CDN acting as an 1109 uCDN) 1111 o "s-" refers to the dCDN Surrogate that serves the request 1112 (corresponds to the "server" of W3C Extended Log Format) 1114 o "u-" refers to the uCDN (relative to a given CDN acting as a dCDN) 1116 o "cs-" refers to communication from the User Agent towards the dCDN 1117 Surrogate 1119 o "sc-" refers to communication from the dCDN Surrogate towards the 1120 User Agent 1122 An implementation of the CDNI Logging interface as per the present 1123 specification MUST support the CDNI HTTP Request Logging Record as 1124 specified in Section 3.4.1. 1126 A CDNI Logging Record contains the corresponding values for the 1127 fields that are enumerated in the last fields directive before the 1128 current log line. Note that the order in which the field values 1129 appear is dictated by the order of the fields names in the fields 1130 directive. There SHOULD be no dependency between the various fields 1131 values. 1133 3.4.1. HTTP Request Logging Record 1135 This section defines the CDNI Logging Record of record-type 1136 "cdni_http_request_v1". It is applicable to content delivery 1137 performed by the dCDN using HTTP/1.0([RFC1945]), 1138 HTTP/1.1([RFC7230],[RFC7231], [RFC7232], [RFC7233], [RFC7234], 1139 [RFC7235]) or HTTPS ([RFC2818], [RFC7230]). We observe that, in the 1140 case of HTTPS delivery, there may be value in logging additional 1141 information specific to the operation of HTTP over TLS and we note 1142 that this is outside the scope of the present document and may be 1143 addressed in a future document defining another CDNI Logging Record 1144 or another version of the HTTP Request Logging Record. 1146 The "cdni_http_request_v1" record-type is also expected to be 1147 applicable to HTTP/2 [RFC7540] since a fundamental design tenet of 1148 HTTP/2 is to preserve the HTTP/1.1 semantics. We observe that, in 1149 the case of HTTP/2 delivery, there may be value in logging additional 1150 information specific to the additional functionality of HTTP/2 (e.g., 1151 information related to connection identification, to stream 1152 identification, to stream priority and to flow control). We note 1153 that such additional information is outside the scope of the present 1154 document and may be addressed in a future document defining another 1155 CDNI Logging Record or another version of the HTTP Request Logging 1156 Record. 1158 The "cdni_http_request_v1" record-type contains the following CDNI 1159 Logging fields, listed by their field name: 1161 o date: 1163 * format: DATE 1165 * field value: the date at which the processing of request 1166 completed on the Surrogate. 1168 * occurrence: there MUST be one and only one instance of this 1169 field. 1171 o time: 1173 * format: TIME 1175 * field value: the time, which MUST be expressed in Coordinated 1176 Universal Time (UTC), at which the processing of request 1177 completed on the Surrogate. 1179 * occurrence: there MUST be one and only one instance of this 1180 field. 1182 o time-taken: 1184 * format: DEC 1186 * field value: decimal value of the duration, in seconds, between 1187 the start of the processing of the request and the completion 1188 of the request processing (e.g., completion of delivery) by the 1189 Surrogate. 1191 * occurrence: there MUST be one and only one instance of this 1192 field. 1194 o c-groupid: 1196 * format: NHTABSTRING 1198 * field value: an opaque identifier for an aggregate set of 1199 clients, derived from the client IPv4 or IPv6 address in the 1200 request received by the Surrogate and/or other network-level 1201 identifying information. The c-groupid serves to group clients 1202 into aggregates. Example aggregates include civil geolocation 1203 information (the country, second-level administrative division, 1204 or postal code from which the client is presumed to make the 1205 request based on a geolocation database lookup) or network 1206 topological information (e.g., the BGP AS number announcing the 1207 prefix containing the address). The c-groupid MAY be 1208 structured e.g., US/TN/MEM/38138. Agreement between the dCDN 1209 and the uCDN on a mapping between IPv4 and IPv6 addresses and 1210 aggregates is presumed to occur out-of-band. The aggregation 1211 mapping SHOULD be chosen such that each aggregate contains more 1212 than one client. 1214 + When the aggregate is chosen so that it contains a single 1215 client (e.g., to allow more detailed analytics, or to allow 1216 a-posteriori analysis of individual delivery for example in 1217 situations of performance-based penalties) the c-groupid MAY 1218 be structured where some elements identify aggregates and 1219 one element identifies the client, e.g., US/TN/ 1220 MEM/38138/43a5bdd6-95c4-4d62-be65-7410df0021e2. In the case 1221 where the aggregate is chosen so that it contains a single 1222 client: 1224 - the element identifying the client SHOULD be 1225 algorithmically generated (from the client IPv4 or IPv6 1226 address in the request received by the Surrogate and/or 1227 other network-level identifying information) in a way 1228 that SHOULD NOT be linkable back to the global addressing 1229 context and that SHOULD vary over time (to offer 1230 protection against long term attacks). 1232 - It is RECOMMENDED that the mapping varies at least once 1233 every 24 hours. 1235 - The algorithmic mapping and variation over time can, in 1236 some cases, allow the uCDN (with the knowledge of the 1237 algorithm and time variation and associated attributes 1238 and keys) to reconstruct the actual client IPv4 or IPv6 1239 address and/or other network-level identifying 1240 information when required (e.g., to allow a-posteriori 1241 analysis of individual delivery for example in situations 1242 of performance-based penalties). However, these enduser 1243 addresses SHOULD only be reconstructed on-demand and the 1244 CDNI Logging File SHOULD only be stored with the 1245 anonymised c-groupid value. 1247 - Allowing reconstruction of client address information 1248 carries with it grave risks to end-user privacy. Since 1249 the c-groupid is in this case equivalent in 1250 identification power to a client IP address, its use may 1251 be restricted by regulation or law as personally 1252 identifiable information. For this reason, such use is 1253 NOT RECOMMENDED. 1255 - One method for mapping that MAY be be supported by 1256 implementations relies on a symmetric key that is known 1257 only to the uCDN and dCDN and HMAC-based Extract-and- 1258 Expand Key Derivation Function (HKDF) key derivation 1259 ([RFC5869]), as will be used in TLS 1.3 1260 ([I-D.ietf-tls-rfc5246-bis]). When that method is used: 1262 o The uCDN and dCDN need to agree on the "salt" and 1263 "input keying material", as described in Section 2.2 1264 of [RFC5869] and the initial "info" parameter (which 1265 could be something like the business names of the two 1266 organizations in UTF-8, concatenated), as described in 1267 Section 2.3 of [RFC5869]. The hash SHOULD be either 1268 SHA-2 or SHA-3 [SHA-3] and the encryption algorithm 1269 SHOULD be 128-bit AES [AES] in Galois Counter Mode 1270 (GCM) [GCM] (AES-GCM) or better. The PRK SHOULD be 1271 chosen by both parties contributing alternate random 1272 bytes until sufficient length exists. After the 1273 initial setup, client-information can be encrypted 1274 using the key generated by the "expand" step of 1275 Section 2.3 of [RFC5869]. The encrypted value SHOULD 1276 be hex encoded or base64 encoded (as specified in 1277 section 4 of [RFC4648]). At the agreed-upon 1278 expiration time, a new key SHOULD be generated and 1279 used. New keys SHOULD be indicated by prefixing the 1280 key with a special character such as exclamation 1281 point. In this way, shorter lifetimes can be used as 1282 needed. 1284 * occurrence: there MUST be one and only one instance of this 1285 field. 1287 o s-ip: 1289 * format: ADDRESS 1291 * field value: the IPv4 or IPv6 address of the Surrogate that 1292 served the request (i.e., the "server" address). 1294 * occurrence: there MUST be zero or exactly one instance of this 1295 field. 1297 o s-hostname: 1299 * format: host 1301 * field value: the hostname of the Surrogate that served the 1302 request (i.e., the "server" hostname). 1304 * occurrence: there MUST be zero or exactly one instance of this 1305 field. 1307 o s-port: 1309 * format: 1*DIGIT 1311 * field value: the destination TCP port (i.e., the "server" port) 1312 in the request received by the Surrogate. 1314 * occurrence: there MUST be zero or exactly one instance of this 1315 field. 1317 o cs-method: 1319 * format: NHTABSTRING 1321 * field value: this is the method of the request received by the 1322 Surrogate. In the case of HTTP delivery, this is the HTTP 1323 method in the request. 1325 * occurrence: There MUST be one and only one instance of this 1326 field. 1328 o cs-uri: 1330 * format: NHTABSTRING 1332 * field value: this is the "effective request URI" of the request 1333 received by the Surrogate as specified in [RFC7230]. It 1334 complies with the "http" URI scheme or the "https" URI scheme 1335 as specified in [RFC7230]). Note that cs-uri can be privacy 1336 sensitive. In that case, and where appropriate, u-uri could be 1337 used instead of cs-uri. 1339 * occurrence: there MUST be zero or exactly one instance of this 1340 field. 1342 o u-uri: 1344 * format: NHTABSTRING 1345 * field value: this is a complete URI, derived from the 1346 "effective request URI" ([RFC7230]) of the request received by 1347 the Surrogate (i.e., the cs-uri) but transformed by the entity 1348 generating or transmitting the CDNI Logging Record, in a way 1349 that is agreed upon between the two ends of the CDNI Logging 1350 interface, so the transformed URI is meaningful to the uCDN. 1351 For example, the two ends of the CDNI Logging interface could 1352 agree that the u-uri is constructed from the cs-uri by removing 1353 the part of the hostname that exposes which individual 1354 Surrogate actually performed the delivery. The details of 1355 modification performed to generate the u-uri, as well as the 1356 mechanism to agree on these modifications between the two sides 1357 of the CDNI Logging interface are outside the scope of the 1358 present document. 1360 * occurrence: there MUST be one and only one instance of this 1361 field. 1363 o protocol: 1365 * format: NHTABSTRING 1367 * field value: this is value of the HTTP-Version field as 1368 specified in [RFC7230] of the Request-Line of the request 1369 received by the Surrogate (e.g., "HTTP/1.1"). 1371 * occurrence: there MUST be one and only one instance of this 1372 field. 1374 o sc-status: 1376 * format: 3DIGIT 1378 * field value: this is the Status-Code in the response from the 1379 Surrogate. In the case of HTTP delivery, this is the HTTP 1380 Status-Code in the HTTP response. 1382 * occurrence: There MUST be one and only one instance of this 1383 field. 1385 o sc-total-bytes: 1387 * format: 1*DIGIT 1389 * field value: this is the total number of bytes of the response 1390 sent by the Surrogate in response to the request. In the case 1391 of HTTP delivery, this includes the bytes of the Status-Line, 1392 the bytes of the HTTP headers and the bytes of the message- 1393 body. 1395 * occurrence: There MUST be one and only one instance of this 1396 field. 1398 o sc-entity-bytes: 1400 * format: 1*DIGIT 1402 * field value: this is the number of bytes of the message-body in 1403 the HTTP response sent by the Surrogate in response to the 1404 request. This does not include the bytes of the Status-Line or 1405 the bytes of the HTTP headers. 1407 * occurrence: there MUST be zero or exactly one instance of this 1408 field. 1410 o cs(insert_HTTP_header_name_here): 1412 * format: QSTRING 1414 * field value: the value of the HTTP header (identified by the 1415 insert_HTTP_header_name_here in the CDNI Logging field name) as 1416 it appears in the request processed by the Surrogate, but 1417 prepended by a DQUOTE and appended by a DQUOTE. For example, 1418 when the CDNI Logging field name (FIENAME) listed in the 1419 preceding fields directive is cs(User-Agent), this CDNI Logging 1420 field value contains the value of the User-Agent HTTP header as 1421 received by the Surrogate in the request it processed, but 1422 prepended by a DQUOTE and appended by a DQUOTE. If the HTTP 1423 header as it appeared in the request processed by the Surrogate 1424 contains one or more DQUOTE, each DQUOTE MUST be escaped with 1425 percent encoding. For example, if the HTTP header contains 1426 My_Header"value", then the field value of the 1427 cs(insert_HTTP_header_name_here) is "My_Header%x22value%x22". 1428 The entity transmitting the CDNI Logging File MUST ensure that 1429 the respective insert_HTTP_header_name_here of the 1430 cs(insert_HTTP_header_name_here) listed in the fields directive 1431 comply with HTTP specifications. In particular, this field 1432 name does not include any HTAB, since this would prevent proper 1433 parsing of the fields directive by the entity receiving the 1434 CDNI Logging File. 1436 * occurrence: there MAY be zero, one or any number of instance of 1437 this field. 1439 o sc(insert_HTTP_header_name_here): 1441 * format: QSTRING 1443 * field value: the value of the HTTP header (identified by the 1444 insert_HTTP_header_name_here in the CDNI Logging field name) as 1445 it appears in the response issued by the Surrogate to serve the 1446 request, but prepended by a DQUOTE and appended by a DQUOTE. 1447 If the HTTP header as it appeared in the request processed by 1448 the Surrogate contains one or more DQUOTE, each DQUOTE MUST be 1449 escaped with percent encoding. For example, if the HTTP header 1450 contains My_Header"value", then the field value of the 1451 sc(insert_HTTP_header_name_here) is "My_Header%x22value%x22". 1452 The entity transmitting the CDNI Logging File MUST ensure that 1453 the respective insert_HTTP_header_name_here of the 1454 cs(insert_HTTP_header_name_here) listed in the fields directive 1455 comply with HTTP specifications. In particular, this field 1456 name does not include any HTAB, since this would prevent proper 1457 parsing of the fields directive by the entity receiving the 1458 CDNI Logging File. 1460 * occurrence: there MAY be zero, one or any number of instances 1461 of this field. For a given insert_HTTP_header_name_here, there 1462 MUST be zero or exactly one instance of this field. 1464 o s-ccid: 1466 * format: QSTRING 1468 * field value: this contains the value of the Content Collection 1469 IDentifier (CCID) associated by the uCDN to the content served 1470 by the Surrogate via the CDNI Metadata interface 1471 ([I-D.ietf-cdni-metadata]), prepended by a DQUOTE and appended 1472 by a DQUOTE. If the CCID conveyed in the CDNI Metadata 1473 interface contains one or more DQUOTE, each DQUOTE MUST be 1474 escaped with percent encoding. For example, if the CCID 1475 conveyed in the CDNI Metadata interface is My_CCIDD"value", 1476 then the field value of the s-ccid is "My_CCID%x22value%X22". 1478 * occurrence: there MUST be zero or exactly one instance of this 1479 field. For a given insert_HTTP_header_name_here, there MUST be 1480 zero or exactly one instance of this field. 1482 o s-sid: 1484 * format: QSTRING 1486 * field value: this contains the value of a Session IDentifier 1487 (SID) generated by the dCDN for a specific HTTP session, 1488 prepended by a DQUOTE and appended by a DQUOTE. In particular, 1489 for HTTP Adaptive Streaming (HAS) session, the Session 1490 IDentifier value is included in the Logging record for every 1491 content chunk delivery of that session in view of facilitating 1492 the later correlation of all the per content chunk log records 1493 of a given HAS session. See section 3.4.2.2. of [RFC6983] for 1494 more discussion on the concept of Session IDentifier in the 1495 context of HAS. If the SID conveyed contains one or more 1496 DQUOTE, each DQUOTE MUST be escaped with percent encoding. For 1497 example, if the SID is My_SID"value", then the field value of 1498 the s-sid is "My_SID%x22value%x22". 1500 * occurrence: there MUST be zero or exactly one instance of this 1501 field. 1503 o s-cached: 1505 * format: 1DIGIT 1507 * field value: this characterises whether the Surrogate served 1508 the request using content already stored on its local cache or 1509 not. The allowed values are "0" (for miss) and "1" (for hit). 1510 "1" MUST be used when the Surrogate did serve the request using 1511 exclusively content already stored on its local cache. "0" MUST 1512 be used otherwise (including cases where the Surrogate served 1513 the request using some, but not all, content already stored on 1514 its local cache). Note that a "0" only means a cache miss in 1515 the Surrogate and does not provide any information on whether 1516 the content was already stored, or not, in another device of 1517 the dCDN, i.e., whether this was a "dCDN hit" or "dCDN miss". 1519 * occurrence: there MUST be zero or exactly one instance of this 1520 field. 1522 CDNI Logging field names are case-insensitive as per the basic 1523 ABNF([RFC5234]). The "fields" directive corresponding to a HTTP 1524 Request Logging Record MUST contain all the fields names whose 1525 occurrence is specified above as "There MUST be one and only one 1526 instance of this field". The corresponding fields value MUST be 1527 present in every HTTP Request Logging Record. 1529 The "fields" directive corresponding to a HTTP Request Logging Record 1530 MAY list all the fields value whose occurrence is specified above as 1531 "there MUST be zero or exactly one instance of this field" or "there 1532 MAY be zero, one or any number of instances of this field". The set 1533 of such field names actually listed in the "fields" directive is 1534 selected by the CDN generating the CDNI Logging File based on 1535 agreements between the interconnected CDNs established through 1536 mechanisms outside the scope of this specification (e.g., contractual 1537 agreements). When such a field name is not listed in the "fields" 1538 directive, the corresponding field value MUST NOT be included in the 1539 Logging Record. When such a field name is listed in the "fields" 1540 directive, the corresponding field value MUST be included in the 1541 Logging Record; if the value for the field is not available, this 1542 MUST be conveyed via a dash character ("-"). 1544 The fields names listed in the "fields" directive MAY be listed in 1545 the order in which they are listed in Section 3.4.1 or MAY be listed 1546 in any other order. 1548 Logging some specific fields from HTTP requests and responses can 1549 introduce serious security and privacy risks. For example, cookies 1550 will often contain (months) long lived token values that can be used 1551 to log into a service as the relevant user. Similar values may be 1552 included in other header fields or within URLs or elsewhere in HTTP 1553 requests and responses. Centralising such values in a CDNI Logging 1554 File can therefore represent a significant increase in risk both for 1555 the user and the web service provider, but also for the CDNs 1556 involved. Implementations ought therefore to attempt to lower the 1557 probability of such bad outcomes e.g. by only allowing a configured 1558 set of headers to be added to CDNI Logging Records, or by not 1559 supporting wildcard selection of HTTP request/response fields to add. 1560 Such mechanisms can reduce the probability that security (or privacy) 1561 sensitive values are centralised in CDNI Logging Files. Also, when 1562 agreeing on which HTTP request/response fields are to be provided in 1563 CDNI Logging Files, the uCDN and dCDN administrators ought to 1564 consider these risks. Furthermore, CDNs making use of c-groupid to 1565 identify an aggregate of clients rather than individual clients ought 1566 to realize that by logging certain header fields they may create the 1567 possibility to re-identify individual clients. In these cases 1568 heeding the above advice, or not logging header fields at all, is 1569 particularly important if the goal is to provide logs that do not 1570 identify individual clients." 1572 A dCDN-side implementation of the CDNI Logging interface MUST 1573 implement all the following Logging fields in a CDNI Logging Record 1574 of record-type "cdni_http_request_v1", and MUST support the ability 1575 to include valid values for each of them: 1577 o date 1579 o time 1581 o time-taken 1583 o c-groupid 1584 o s-ip 1586 o s-hostname 1588 o s-port 1590 o cs-method 1592 o cs-uri 1594 o u-uri 1596 o protocol 1598 o sc-status 1600 o sc-total-bytes 1602 o sc-entity-bytes 1604 o cs(insert_HTTP_header_name_here) 1606 o sc(insert_HTTP_header_name_here) 1608 o s-cached 1610 A dCDN-side implementation of the CDNI Logging interface MAY support 1611 the following Logging fields in a CDNI Logging Record of record-type 1612 "cdni_http_request_v1": 1614 o s-ccid 1616 o s-sid 1618 If a dCDN-side implementation of the CDNI Logging interface supports 1619 these fields, it MUST support the ability to include valid values for 1620 them. 1622 An uCDN-side implementation of the CDNI Logging interface MUST be 1623 able to accept CDNI Logging Files with CDNI Logging Records of 1624 record-type "cdni_http_request_v1" containing any CDNI Logging Field 1625 defined in Section 3.4.1 as long as the CDNI Logging Record and the 1626 CDNI Logging File are compliant with the present document. 1628 In case an uCDN-side implementation of the CDNI Logging interface 1629 receives a CDNI Logging File with HTTP Request Logging Records that 1630 do not contain field values for exactly the set of field names 1631 actually listed in the preceding "fields" directive, the 1632 implementation MUST ignore those HTTP Request Logging Records, and 1633 MUST accept the other HTTP Request Logging Records. 1635 To ensure that the logging file is correct, the text MUST be 1636 sanitized before being logged. Null, bare CR, bare LF and HTAB have 1637 to be removed by escaping them through percent encoding to avoid 1638 confusion with the logging record separators. 1640 3.5. CDNI Logging File Extension 1642 The CDNI Logging File contains blocks of directives and blocks of 1643 corresponding records. The supported set of directives is defined 1644 relative to the CDNI Logging File Format version. The complete set 1645 of directives for version "cdni/1.0" are defined in Section 3.3. The 1646 directive list is not expected to require much extension, but when it 1647 does, the new directive MUST be defined and registered in the "CDNI 1648 Logging Directive Names" registry, as described in Figure 9, and a 1649 new version MUST be defined and registered in the "CDNI Logging File 1650 version" registry, as described in Section 6.2. For example, adding 1651 a new CDNI Logging Directive, e.g., "foo", to the set of directives 1652 defined for "cdni/1.0" in Section 3.3, would require registering both 1653 the new CDNI Logging Directive "foo" and a new CDNI Logging File 1654 version, e.g., "CDNI/2.0", which includes all of the existing CDNI 1655 Logging Directives of "cdni/1.0" plus "foo". 1657 It is expected that as new logging requirements arise, the list of 1658 fields to log will change and expand. When adding new fields, the 1659 new fields MUST be defined and registered in the "CDNI Logging Field 1660 Names" registry, as described in Section 6.4, and a new record-type 1661 MUST be defined and registered in the "CDNI Logging record-types" 1662 registry, as described in Section 6.3. For example, adding a new 1663 CDNI Logging Field, e.g., "c-bar", to the set of fields defined for 1664 "cdni_http_request_v1" in Section 3.4.1, would require registering 1665 both the new CDNI Logging Field "c-bar" and a new CDNI record-type, 1666 e.g., "cdni_http_request_v2", which includes all of the existing CDNI 1667 Logging Fields of "cdni_http_request_v1" plus "c-bar". 1669 3.6. CDNI Logging File Examples 1671 Let us consider the upstream CDN and the downstream CDN labelled uCDN 1672 and dCDN-1 in Figure 1. When dCDN-1 acts as a downstream CDN for 1673 uCDN and performs content delivery on behalf of uCDN, dCDN-1 will 1674 include the CDNI Logging Records corresponding to the content 1675 deliveries performed on behalf of uCDN in the CDNI Logging Files for 1676 uCDN. An example CDNI Logging File communicated by dCDN-1 to uCDN is 1677 shown below in Figure 4. 1679 #version:cdni/1.0 1681 #UUID:urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6 1683 #claimed-origin:cdni-logging-entity.dcdn-1.example.com 1685 #record-type:cdni_http_request_v1 1687 #fields:datetimetime-takenc-groupid 1688 cs-methodu-uriprotocol 1689 sc-statussc-total-bytescs(User-Agent) 1690 cs(Referer)s-cached 1692 2013-05-1700:38:06.8259.058US/TN/MEM/38138 1693 GET 1694 http://cdni-ucdn.dcdn-1.example.com/video/movie100.mp4 1695 HTTP/1.12006729891"Mozilla/5.0 1696 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1697 Gecko) Chrome/5.0.375.127 Safari/533.4" 1698 "host1.example.com"1 1700 2013-05-1700:39:09.14515.32FR/PACA/NCE/06100 1701 GET 1702 http://cdni-ucdn.dcdn-1.example.com/video/movie118.mp4 1703 HTTP/1.120015799210"Mozilla/5.0 1704 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1705 Gecko) Chrome/5.0.375.127 Safari/533.4" 1706 "host1.example.com"1 1708 2013-05-1700:42:53.43752.879US/TN/MEM/38138 1709 GET 1710 http://cdni-ucdn.dcdn-1.example.com/video/picture11.mp4 1711 HTTP/1.020097234724"Mozilla/5.0 1712 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1713 Gecko) Chrome/5.0.375.127 Safari/533.4" 1714 "host5.example.com"0 1716 #SHA256-hash: 64-hexadecimal-digit hash value 1718 Figure 4: CDNI Logging File Example 1720 If uCDN establishes by some means (e.g., via TLS authentication when 1721 pulling the CDNI Logging File) the identity of the entity from which 1722 it pulled the CDNI Logging File, uCDN can add to the CDNI Logging an 1723 established-origin directive as illustrated below: 1725 #established-origin:cdni-logging-entity.dcdn- 1726 1.example.com 1727 As illustrated in Figure 2, uCDN will then ingest the corresponding 1728 CDNI Logging Records into its Collection process, alongside the 1729 Logging Records generated locally by the uCDN itself. This allows 1730 uCDN to aggregate Logging Records for deliveries performed by itself 1731 (through Records generated locally) as well as for deliveries 1732 performed by its downstream CDN(s). This aggregate information can 1733 then be used (after Filtering and Rectification, as illustrated in 1734 Figure 2) by Log Consuming Applications that take into account 1735 deliveries performed by uCDN as well as by all of its downstream 1736 CDNs. 1738 We observe that the time between 1740 1. when a delivery is completed in dCDN and 1742 2. when the corresponding Logging Record is ingested by the 1743 Collection process in uCDN 1745 depends on a number of parameters such as the Logging Period agreed 1746 to by uCDN and dCDN, how much time uCDN waits before pulling the CDNI 1747 Logging File once it is advertised in the CDNI Logging Feed, and the 1748 time to complete the pull of the CDNI Logging File. Therefore, if we 1749 consider the set of Logging Records aggregated by the Collection 1750 process in uCDN in a given time interval, there could be a permanent 1751 significant timing difference between the CDNI Logging Records 1752 received from the dCDN and the Logging Records generated locally. 1753 For example, in a given time interval, the Collection process in uCDN 1754 may be aggregating Logging Records generated locally by uCDN for 1755 deliveries performed in the last hour and CDNI Logging Records 1756 generated in the dCDN for deliveries in the hour before last. 1758 Say, that for some reason (for example a Surrogate bug), dCDN-1 could 1759 not collect the total number of bytes of the responses sent by the 1760 Surrogate (in other words, the value for sc-total-bytes is not 1761 available). Then the corresponding CDNI Logging records would 1762 contain a dash character ("-") in lieu of the value for the sc-total- 1763 bytes field (as specified in Section 3.4.1). In that case, the CDNI 1764 Logging File that would be communicated by dCDN-1 to uCDN is shown 1765 below in Figure 5. 1767 #version:cdni/1.0 1769 #UUID:urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6 1771 #claimed-origin:cdni-logging-entity.dcdn-1.example.com 1773 #record-type:cdni_http_request_v1 1775 #fields:datetimetime-takenc-groupid 1776 cs-methodu-uriprotocol 1777 sc-statussc-total-bytescs(User-Agent) 1778 cs(Referer)s-cached 1780 2013-05-1700:38:06.8259.058US/TN/MEM/38138 1781 GET 1782 http://cdni-ucdn.dcdn-1.example.com/video/movie100.mp4 1783 HTTP/1.1200-"Mozilla/5.0 1784 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1785 Gecko) Chrome/5.0.375.127 Safari/533.4" 1786 "host1.example.com"1 1788 2013-05-1700:39:09.14515.32FR/PACA/NCE/06100 1789 GET 1790 http://cdni-ucdn.dcdn-1.example.com/video/movie118.mp4 1791 HTTP/1.1200-"Mozilla/5.0 1792 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1793 Gecko) Chrome/5.0.375.127 Safari/533.4" 1794 "host1.example.com"1 1796 2013-05-1700:42:53.43752.879US/TN/MEM/38138 1797 GET 1798 http://cdni-ucdn.dcdn-1.example.com/video/picture11.mp4 1799 HTTP/1.0200-"Mozilla/5.0 1800 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1801 Gecko) Chrome/5.0.375.127 Safari/533.4" 1802 "host5.example.com"0 1804 #SHA256-hash: 64-hexadecimal-digit hash value 1806 Figure 5: CDNI Logging File Example With A Missing Field Value 1808 3.7. Cascaded CDNI Logging Files Example 1810 Let us consider the cascaded CDN scenario of uCDN, dCDN-2 and dCDN-3 1811 as depicted in Figure 1. After completion of a delivery by dCDN-3 on 1812 behalf of dCDN-2, dCDN-3 will include a corresponding Logging Record 1813 in a CDNI Logging File that will be pulled by dCDN-2 and that is 1814 illustrated below in Figure 6. In practice, a CDNI Logging File is 1815 likely to contain a very high number of CDNI Logging Records. 1816 However, for readability, the example in Figure 6 contains a single 1817 CDNI Logging Record. 1819 #version:cdni/1.0 1821 #UUID:urn:uuid:65718ef-0123-9876-adce4321bcde 1823 #claimed-origin:cdni-logging-entity.dcdn-3.example.com 1825 #record-type:cdni_http_request_v1 1827 #fields:datetimetime-takenc-groupid 1828 cs-methodu-uriprotocol 1829 sc-statussc-total-bytescs(User-Agent) 1830 cs(Referer)s-cached 1832 2013-05-1700:39:09.11914.07US/CA/SFO/94114 1833 GET 1834 http://cdni-dcdn-2.dcdn-3.example.com/video/movie118.mp4 1835 HTTP/1.120015799210"Mozilla/5.0 1836 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1837 Gecko) Chrome/5.0.375.127 Safari /533.4" 1838 "host1.example.com"1 1840 #SHA256-hash: 64-hexadecimal-digit hash value 1842 Figure 6: Cascaded CDNI Logging File Example (dCDN-3 to dCDN-2) 1844 If dCDN-2 establishes by some means (e.g., via TLS authentication 1845 when pulling the CDNI Logging File) the identity of the entity from 1846 which it pulled the CDNI Logging File, dCDN-2 can add to the CDNI 1847 Logging an established-origin directive as illustrated below: 1849 #established-origin:cdni-logging-entity.dcdn- 1850 3.example.com 1852 dCDN-2 (behaving as an upstream CDN from the viewpoint of dCDN-3) 1853 will then ingest the CDNI Logging Record for the considered dCDN-3 1854 delivery into its Collection process (as illustrated in Figure 2). 1855 This Logging Record may be aggregated with Logging Records generated 1856 locally by dCDN-2 for deliveries performed by dCDN-2 itself. Say, 1857 for illustration, that the content delivery performed by dCDN-3 on 1858 behalf of dCDN-2 had actually been redirected to dCDN-2 by uCDN, and 1859 say that another content delivery has just been redirected by uCDN to 1860 dCDN-2 and that dCDN-2 elected to perform the corresponding delivery 1861 itself. Then after Filtering and Rectification (as illustrated in 1862 Figure 2), dCDN-2 will include the two Logging Records corresponding 1863 respectively to the delivery performed by dCDN-3 and the delivery 1864 performed by dCDN-2, in the next CDNI Logging File that will be 1865 communicated to uCDN. An example of such CDNI Logging File is 1866 illustrated below in Figure 7. 1868 #version:cdni/1.0 1870 #UUID:urn:uuid:1234567-8fedc-abab-0987654321ff 1872 #claimed-origin:cdni-logging-entity.dcdn-2.example.com 1874 #record-type:cdni_http_request_v1 1876 #fields:datetimetime-takenc-groupid 1877 cs-methodu-uriprotocol 1878 sc-statussc-total-bytescs(User-Agent) 1879 cs(Referer)s-cached 1881 2013-05-1700:39:09.11914.07US/CA/SFO/94114 1882 GET 1883 http://cdni-ucdn.dcdn-2.example.com/video/movie118.mp4 1884 HTTP/1.120015799210"Mozilla/5.0 1885 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1886 Gecko) Chrome/5.0.375.127 Safari /533.4" 1887 "host1.example.com"1 1889 2013-05-1701:42:53.43752.879FR/IDF/PAR/75001 1890 GET 1891 http://cdni-ucdn.dcdn-2.example.com/video/picture11.mp4 1892 HTTP/1.020097234724"Mozilla/5.0 1893 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1894 Gecko) Chrome/5.0.375.127 Safari /533.4" 1895 "host5.example.com"0 1897 #SHA256-hash: 64-hexadecimal-digit hash value 1899 Figure 7: Cascaded CDNI Logging File Example (dCDN-2 to uCDN) 1901 If uCDN establishes by some means (e.g., via TLS authentication when 1902 pulling the CDNI Logging File) the identity of the entity from which 1903 it pulled the CDNI Logging File, uCDN can add to the CDNI Logging an 1904 established-origin directive as illustrated below: 1906 #established-origin:cdni-logging-entity.dcdn- 1907 2.example.com 1909 In the example of Figure 7, we observe that: 1911 o the first Logging Record corresponds to the Logging Record 1912 communicated earlier to dCDN-2 by dCDN-3, which corresponds to a 1913 delivery redirected by uCDN to dCDN-2 and then redirected by 1914 dCDN-2 to dCDN-3. The fields values in this Logging Record are 1915 copied from the corresponding CDNI Logging REcord communicated to 1916 dCDN2 by dCDN-3, with the exception of the u-uri that now reflects 1917 the URI convention between uCDN and dCDN-2 and that presents the 1918 delivery to uCDN as if it was performed by dCDN-2 itself. This 1919 reflects the fact that dCDN-2 had taken the full responsibility of 1920 the corresponding delivery (even if in this case, dCDN-2 elected 1921 to redirect the delivery to dCDN-3 so it is actually performed by 1922 dCDN-3 on behalf of dCDN-2). 1924 o the second Logging Record corresponds to a delivery redirected by 1925 uCDN to dCDN-2 and performed by dCDN-2 itself. The time of the 1926 delivery in this Logging Record may be significantly more recent 1927 than the first Logging Record since it was generated locally while 1928 the first Logging Record was generated by dCDN-3 and had to be 1929 advertised , and then pulled and then ingested into the dCDN-2 1930 Collection process, before being aggregated with the second 1931 Logging Record. 1933 4. Protocol for Exchange of CDNI Logging File After Full Collection 1935 This section specifies a protocol for the exchange of CDNI Logging 1936 Files as specified in Section 3 after the CDNI Logging File is fully 1937 collected by the dCDN. 1939 This protocol comprises: 1941 o a CDNI Logging feed, allowing the dCDN to notify the uCDN about 1942 the CDNI Logging Files that can be retrieved by that uCDN from the 1943 dCDN, as well as all the information necessary for retrieving each 1944 of these CDNI Logging Files. The CDNI Logging feed is specified 1945 in Section 4.1. 1947 o a CDNI Logging File pull mechanism, allowing the uCDN to obtain 1948 from the dCDN a given CDNI Logging File at the uCDN's convenience. 1949 The CDNI Logging File pull mechanisms is specified in Section 4.2. 1951 An implementation of the CDNI Logging interface on the dCDN side (the 1952 entity generating the CDNI Logging file) MUST support the server side 1953 of the CDNI Logging feed (as specified in Section 4.1) and the server 1954 side of the CDNI Logging pull mechanism (as specified in 1955 Section 4.2). 1957 An implementation of the CDNI Logging interface on the uCDN side (the 1958 entity consuming the CDNI Logging file) MUST support the client side 1959 of the CDNI Logging feed (as specified in Section 4.1) and the client 1960 side of the CDNI Logging pull mechanism (as specified in 1961 Section 4.2). 1963 4.1. CDNI Logging Feed 1965 The server-side implementation of the CDNI Logging feed MUST produce 1966 an Atom feed [RFC4287]. This feed is used to advertise log files 1967 that are available for the client-side to retrieve using the CDNI 1968 Logging pull mechanism. 1970 4.1.1. Atom Formatting 1972 A CDNI Logging feed MUST be structured as an Archived feed, as 1973 defined in [RFC5005], and MUST be formatted in Atom [RFC4287]. This 1974 means it consists of a subscription document that is regularly 1975 updated as new CDNI Logging Files become available, and information 1976 about older CDNI Logging files is moved into archive documents. Once 1977 created, archive documents are never modified. 1979 Each CDNI Logging File listed in an Atom feed MUST be described in an 1980 atom:entry container element. 1982 The atom:entry MUST contain an atom:content element whose "src" 1983 attribute is a link to the CDNI Logging File and whose "type" 1984 attribute is the MIME Media Type indicating that the entry is a CDNI 1985 logging file. This MIME Media Type is defined as "application/cdni" 1986 (See [RFC7736]) with the Payload Type (ptype) parameter set to 1987 "logging-file". 1989 For compatibility with some Atom feed readers the atom:entry MAY also 1990 contain an atom:link entry whose "href" attribute is a link to the 1991 CDNI Logging File and whose "type" attribute is the MIME Media Type 1992 indicating that the entry is a CDNI Logging File using the 1993 "application/cdni" MIME Media Type with the Payload Type (ptype) 1994 parameter set to "logging-file"(See [RFC7736]). 1996 The URI used in the atom:id of the atom:entry MUST contain the UUID 1997 of the CDNI Logging File. 1999 The atom:updated in the atom:entry MUST indicate the time at which 2000 the CDNI Logging File was last updated. 2002 4.1.2. Updates to Log Files and the Feed 2004 CDNI Logging Files MUST NOT be modified by the dCDN once published in 2005 the CDNI Logging feed. 2007 The frequency with which the subscription feed is updated, the period 2008 of time covered by each CDNI Logging File or each archive document, 2009 and timeliness of publishing of CDNI Logging Files are outside the 2010 scope of the present document and are expected to be agreed upon by 2011 uCDN and dCDN via other means (e.g., human agreement). 2013 The server-side implementation MUST be able to set, and SHOULD set, 2014 HTTP cache control headers on the subscription feed to indicate the 2015 frequency at which the client-side is to poll for updates. 2017 The client-side MAY use HTTP cache control headers (set by the 2018 server-side) on the subscription feed to determine the frequency at 2019 which to poll for updates. The client-side MAY instead, or in 2020 addition, use other information to determine when to poll for updates 2021 (e.g., a polling frequency that may have been negotiated between the 2022 uCDN and dCDN by mechanisms outside the scope of the present document 2023 and that is to override the indications provided in the HTTP cache 2024 control headers). 2026 The potential retention limits (e.g., sliding time window) within 2027 which the dCDN is to retain and be ready to serve an archive document 2028 is outside the scope of the present document and is expected to be 2029 agreed upon by uCDN and dCDN via other means (e.g., human agreement). 2030 The server-side implementation MUST retain, and be ready to serve, 2031 any archive document within the agreed retention limits. Outside 2032 these agreed limits, the server-side implementation MAY indicate its 2033 inability to serve (e.g., with HTTP status code 404) an archive 2034 document or MAY refuse to serve it (e.g., with HTTP status code 403 2035 or 410). 2037 4.1.3. Redundant Feeds 2039 The server-side implementation MAY present more than one CDNI Logging 2040 feed for redundancy. Each CDNI Logging File MAY be published in more 2041 than one feed. 2043 A client-side implementation MAY support such redundant CDNI Logging 2044 feeds. If it supports redundant CDNI Logging feed, the client-side 2045 can use the UUID of the CDNI Logging File, presented in the atom:id 2046 element of the Atom feed, to avoid unnecessarily pulling and storing 2047 a given CDNI Logging File more than once. 2049 4.1.4. Example CDNI Logging Feed 2051 Figure 8 illustrates an example of the subscription document of a 2052 CDNI Logging feed. 2054 2055 2056 CDNI Logging Feed 2057 2013-03-23T14:46:11Z 2058 urn:uuid:663ae677-40fb-e99a-049d-c5642916b8ce 2059 2061 2063 2065 CDNI Log Feed 2066 Generator 2067 dcdn.example 2068 2069 CDNI Logging File for uCDN at 2070 2013-03-23 14:15:00 2071 urn:uuid:12345678-1234-abcd-00aa-01234567abcd 2072 2013-03-23T14:15:00Z 2073 2077 CDNI Logging File for uCDN at 2078 2013-03-23 14:15:00 2079 2080 2081 CDNI Logging File for uCDN at 2082 2013-03-23 14:30:00 2083 urn:uuid:87654321-4321-dcba-aa00-dcba7654321 2084 2013-03-23T14:30:00Z 2085 2089 CDNI Logging File for uCDN at 2090 2013-03-23 14:30:00 2091 2092 ... 2093 2094 ... 2095 2096 2098 Figure 8: Example subscription document of a CDNI Logging Feed 2100 4.2. CDNI Logging File Pull 2102 A client-side implementation of the CDNI Logging interface MAY pull, 2103 at its convenience, a CDNI Logging File that is published by the 2104 server-side in the CDNI Logging Feed (in the subscription document or 2105 an archive document). To do so, the client-side: 2107 o MUST implement HTTP/1.1 ([RFC7230],[RFC7231], [RFC7232], 2108 [RFC7233], [RFC7234], [RFC7235]), MAY also support other HTTP 2109 versions (e.g., HTTP/2 [RFC7540]) and MAY negotiate which HTTP 2110 version is actually used. This allows operators and implementers 2111 to choose to use later versions of HTTP to take advantage of new 2112 features, while still ensuring interoperability with systems that 2113 only support HTTP/1.1. 2115 o MUST use the URI that was associated to the CDNI Logging File 2116 (within the "src" attribute of the corresponding atom:content 2117 element) in the CDNI Logging Feed; 2119 o MUST support exchange of CDNI Logging Files with no content 2120 encoding applied to the representation; 2122 o MUST support exchange of CDNI Logging Files with "gzip" content 2123 encoding (as defined in [RFC7230]) applied to the representation. 2125 Note that a client-side implementation of the CDNI Logging interface 2126 MAY pull a CDNI Logging File that it has already pulled. 2128 The server-side implementation MUST respond to valid pull request by 2129 a client-side implementation for a CDNI Logging File published by the 2130 server-side in the CDNI Logging Feed (in the subscription document or 2131 an archive document). The server-side implementation: 2133 o MUST implement HTTP/1.1 to handle the client-side request and MAY 2134 also support other HTTP versions (e.g., HTTP/2); 2136 o MUST include the CDNI Logging File identified by the request URI 2137 inside the body of the HTTP response; 2139 o MUST support exchange of CDNI Logging Files with no content 2140 encoding applied to the representation; 2142 o MUST support exchange of CDNI Logging Files with "gzip" content 2143 encoding (as defined in [RFC7231]) applied to the representation. 2145 Content negotiation approaches defined in [RFC7231] (e.g., using 2146 Accept-Encoding request-header field or Content-Encoding entity- 2147 header field) MAY be used by the client-side and server-side 2148 implementations to establish the content-coding to be used for a 2149 particular exchange of a CDNI Logging File. 2151 Applying compression content encoding (such as "gzip") is expected to 2152 mitigate the impact of exchanging the large volumes of logging 2153 information expected across CDNs. This is expected to be 2154 particularly useful in the presence of HTTP Adaptive Streaming (HAS) 2155 which, as per the present version of the document, will result in a 2156 separate CDNI Log Record for each HAS segment delivery in the CDNI 2157 Logging File. 2159 The potential retention limits (e.g., sliding time window, maximum 2160 aggregate file storage quotas) within which the dCDN is to retain and 2161 be ready to serve a CDNI Logging File previously advertised in the 2162 CDNI Logging Feed is outside the scope of the present document and is 2163 expected to be agreed upon by uCDN and dCDN via other means (e.g., 2164 human agreement). The server-side implementation MUST retain, and be 2165 ready to serve, any CDNI Logging File within the agreed retention 2166 limits. Outside these agreed limits, the server-side implementation 2167 MAY indicate its inability to serve (e.g., with HTTP status code 404) 2168 a CDNI Logging File or MAY refuse to serve it (e.g., with HTTP status 2169 code 403 or 410). 2171 5. Protocol for Exchange of CDNI Logging File During Collection 2173 We note that, in addition to the CDNI Logging File exchange protocol 2174 specified in Section 4, implementations of the CDNI Logging interface 2175 may also support other mechanisms to exchange CDNI Logging Files. In 2176 particular, such mechanisms might allow the exchange of the CDNI 2177 Logging File to start before the file is fully collected. This can 2178 allow CDNI Logging Records to be communicated by the dCDN to the uCDN 2179 as they are gathered by the dCDN without having to wait until all the 2180 CDNI Logging Records of the same logging period are collected in the 2181 corresponding CDNI Logging File. This approach is commonly referred 2182 to as "tailing" of the file. 2184 Such an approach could be used, for example, to exchange logging 2185 information with a significantly reduced time-lag (e.g., sub-minute 2186 or sub-second) between when the event occurred in the dCDN and when 2187 the corresponding CDNI Logging Record is made available to the uCDN. 2188 This can satisfy log-consuming applications requiring extremely fresh 2189 logging information such as near-real-time content delivery 2190 monitoring. Such mechanisms are for further study and outside the 2191 scope of this document. 2193 6. IANA Considerations 2195 6.1. CDNI Logging Directive Names Registry 2197 The IANA is requested to create a new "CDNI Logging Directive Names" 2198 subregistry under the "Content Delivery Networks Interconnection 2199 (CDNI) Parameters" registry. 2201 The initial contents of the CDNI Logging Directives registry comprise 2202 the names of the directives specified in Section 3.3 of the present 2203 document, and are as follows: 2205 +------------------------------+-----------+ 2206 | Directive Name | Reference | 2207 +------------------------------+-----------+ 2208 | version | RFC xxxx | 2209 | UUID | RFC xxxx | 2210 | claimed-origin | RFC xxxx | 2211 | established-origin | RFC xxxx | 2212 | remark | RFC xxxx | 2213 | record-type | RFC xxxx | 2214 | fields | RFC xxxx | 2215 | SHA256-hash | RFC xxxx | 2216 +------------------------------+-----------+ 2218 Figure 9 2220 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2221 the present document] 2223 Within the registry, names are to be allocated by IANA according to 2224 the "Specification Required" policy specified in [RFC5226]. 2225 Directive names are to be allocated by IANA with a format of 2226 NAMEFORMAT (see Section 3.1). All directive names defined in the 2227 logging file are case-insensitive as per the basic ABNF([RFC5234]). 2229 Each specification that defines a new CDNI Logging directive needs to 2230 contain a description for the new directive with the same set of 2231 information as provided in Section 3.3 (i.e., format, directive value 2232 and occurrence). 2234 6.2. CDNI Logging File version Registry 2236 The IANA is requested to create a new "CDNI Logging File version" 2237 subregistry under the "Content Delivery Networks Interconnection 2238 (CDNI) Parameters" registry. 2240 The initial contents of the CDNI Logging Logging File version 2241 registry comprise the value "cdni/1.0" specified in Section 3.3 of 2242 the present document, and are as follows: 2244 +-----------------+-----------+----------------------------------+ 2245 | version | Reference | Description | 2246 +-----------------+-----------+----------------------------------+ 2247 | cdni/1.0 | RFC xxxx | CDNI Logging File version 1.0 | 2248 | | | as specified in RFC xxxx | 2249 +-----------------+-----------+----------------------------------+ 2251 Figure 10 2253 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2254 the present document] 2256 Within the registry, version values are to be allocated by IANA 2257 according to the "Specification Required" policy specified in 2258 [RFC5226]. Version values are to be allocated by IANA with a format 2259 of NAMEFORMAT (see Section 3.1). All version values defined in the 2260 logging file are case-insensitive as per the basic ABNF([RFC5234]). 2262 6.3. CDNI Logging record-types Registry 2264 The IANA is requested to create a new "CDNI Logging record-types" 2265 subregistry under the "Content Delivery Networks Interconnection 2266 (CDNI) Parameters" registry. 2268 The initial contents of the CDNI Logging record-types registry 2269 comprise the names of the CDNI Logging Record types specified in 2270 Section 3.4 of the present document, and are as follows: 2272 +----------------------+-----------+---------------------------------+ 2273 | record-types | Reference | Description | 2274 +----------------------+-----------+---------------------------------+ 2275 | cdni_http_request_v1 | RFC xxxx | CDNI Logging Record version 1 | 2276 | | | for content delivery using HTTP | 2277 +----------------------+-----------+---------------------------------+ 2279 Figure 11 2281 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2282 the present document] 2284 Within the registry, record-types are to be allocated by IANA 2285 according to the "Specification Required" policy specified in 2286 [RFC5226]. Record-types are to be allocated by IANA with a format of 2287 NAMEFORMAT (see Section 3.1). All record-types defined in the 2288 logging file are case-insensitive as per the basic ABNF([RFC5234]). 2290 Each specification that defines a new record-type needs to contain a 2291 description for the new record-type with the same set of information 2292 as provided in Section 3.4.1. This includes: 2294 o a list of all the CDNI Logging fields that can appear in a CDNI 2295 Logging Record of the new record-type 2297 o for all these fields: a specification of the occurrence for each 2298 Field in the new record-type 2300 o for every newly defined Field, i.e., for every Field that results 2301 in a registration in the CDNI Logging Field Names Registry 2302 (Section 6.4): a specification of the field name, format and field 2303 value. 2305 6.4. CDNI Logging Field Names Registry 2307 The IANA is requested to create a new "CDNI Logging Field Names" 2308 subregistry under the "Content Delivery Networks Interconnection 2309 (CDNI) Parameters" registry. 2311 This registry is intended to be shared across the currently defined 2312 record-type (i.e., cdni_http_request_v1) as well as potential other 2313 CDNI Logging record-types that may be defined in separate 2314 specifications. When a Field from this registry is used by another 2315 CDNI Logging record-type, it is to be used with the exact semantics 2316 and format specified in the document that registered this field and 2317 that is identified in the Reference column of the registry. If 2318 another CDNI Logging record-type requires a Field with semantics that 2319 are not strictly identical, or a format that is not strictly 2320 identical then this new Field is to be registered in the registry 2321 with a different Field name. When a Field from this registry is used 2322 by another CDNI Logging record-type, it can be used with different 2323 occurrence rules. 2325 The initial contents of the CDNI Logging fields Names registry 2326 comprise the names of the CDNI Logging fields specified in 2327 Section 3.4 of the present document, and are as follows: 2329 +------------------------------------------+-----------+ 2330 | Field Name | Reference | 2331 +------------------------------------------+-----------+ 2332 | date | RFC xxxx | 2333 | time | RFC xxxx | 2334 | time-taken | RFC xxxx | 2335 | c-groupid | RFC xxxx | 2336 | s-ip | RFC xxxx | 2337 | s-hostname | RFC xxxx | 2338 | s-port | RFC xxxx | 2339 | cs-method | RFC xxxx | 2340 | cs-uri | RFC xxxx | 2341 | u-uri | RFC xxxx | 2342 | protocol | RFC xxxx | 2343 | sc-status | RFC xxxx | 2344 | sc-total-bytes | RFC xxxx | 2345 | sc-entity-bytes | RFC xxxx | 2346 | cs(insert_HTTP_header_name_here) | RFC xxxx | 2347 | sc(insert_HTTP_header_name_here) | RFC xxxx | 2348 | s-ccid | RFC xxxx | 2349 | s-sid | RFC xxxx | 2350 | s-cached | RFC xxxx | 2351 +------------------------------------------+-----------+ 2353 Figure 12 2355 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2356 the present document] 2358 Within the registry, names are to be allocated by IANA according to 2359 the "Specification Required" policy specified in [RFC5226]. Field 2360 names are to be allocated by IANA with a format of NHTABSTRING (see 2361 Section 3.1). All field names defined in the logging file are case- 2362 insensitive as per the basic ABNF([RFC5234]). 2364 6.5. CDNI Logging MIME Media Type 2366 The IANA is requested to register the following new Payload Type in 2367 the CDNI Payload Type registry for use with the application/cdni MIME 2368 media type. 2370 [RFC Editor Note: Please replace the references to [RFCthis] below 2371 with this document's RFC number before publication.] 2372 +----------------------+---------------+ 2373 | Payload Type | Specification | 2374 +----------------------+---------------+ 2375 | logging-file | [RFCthis] | 2376 +----------------------+---------------+ 2378 Figure 13: MIME Media Type payload 2380 The purpose of the logging-file payload type is to distinguish 2381 between CDNI Logging Files and other CDNI messages. 2383 Interface: LI 2385 Encoding: see Section 3.2, Section 3.3 and Section 3.4 2387 7. Security Considerations 2389 7.1. Authentication, Authorization, Confidentiality, Integrity 2390 Protection 2392 An implementation of the CDNI Logging interface MUST support TLS 2393 transport of the CDNI Logging feed (Section 4.1) and of the CDNI 2394 Logging File pull (Section 4.2) as per [RFC2818] and [RFC7230]. 2396 TLS MUST be used by the server-side and the client-side of the CDNI 2397 Logging feed, as well as the server-side and the client-side of the 2398 CDNI Logging File pull mechanism, including authentication of the 2399 remote end, unless alternate methods are used for ensuring the 2400 security of the information exchanged over the LI interface (such as 2401 setting up an IPsec tunnel between the two CDNs or using a physically 2402 secured internal network between two CDNs that are owned by the same 2403 corporate entity). 2405 The use of TLS for transport of the CDNI Logging feed and CDNI 2406 Logging File pull allows: 2408 o the dCDN and uCDN to authenticate each other using TLS client auth 2409 and TLS server auth. 2411 and, once they have mutually authenticated each other, it allows: 2413 o the dCDN and uCDN to authorize each other (to ensure they are 2414 transmitting/receiving CDNI Logging File to/from an authorized 2415 CDN) 2417 o the CDNI Logging information to be transmitted with 2418 confidentiality 2420 o the integrity of the CDNI Logging information to be protected 2421 during the exchange. 2423 When TLS is used, the general TLS usage guidance in [RFC7525] MUST be 2424 followed. 2426 The SHA256-hash directive inside the CDNI Logging File provides 2427 additional integrity protection, this time targeting potential 2428 corruption of the CDNI logging information during the CDNI Logging 2429 File generation, storage or exchange. This mechanism does not itself 2430 allow restoration of the corrupted CDNI Logging information, but it 2431 allows detection of such corruption and therefore triggering of 2432 appropriate corrective actions (e.g., discard of corrupted 2433 information, attempt to re-obtain the CDNI Logging information). 2434 Note that the SHA256-hash does not protect against tampering by a 2435 third party, since such a third party could have recomputed and 2436 updated the SHA256-hash after tampering. Protection against third 2437 party tampering, when the CDNI Logging File is communicated over the 2438 CDN Logging Interface, can be achieved as discussed above through the 2439 use of TLS. 2441 7.2. Denial of Service 2443 This document does not define specific mechanism to protect against 2444 Denial of Service (DoS) attacks on the Logging Interface. However, 2445 the CDNI Logging feed and CDNI Logging pull endpoints are typically 2446 to be accessed only by a very small number of valid remote endpoints 2447 and therefore can be easily protected against DoS attacks through the 2448 usual conventional DOS protection mechanisms such as firewalling or 2449 use of Virtual Private Networks (VPNs). 2451 Protection of dCDN Surrogates against spoofed delivery requests is 2452 outside the scope of the CDNI Logging interface. 2454 7.3. Privacy 2456 CDNs have the opportunity to collect detailed information about the 2457 downloads performed by End Users. A dCDN is expected to collect such 2458 information into CDNI Logging Files, which are then communicated to 2459 an uCDN. 2461 Having detailed CDNI logging information known by the dCDN in itself 2462 does not represent a particular privacy concern since the dCDN is 2463 obviously fully aware of all information logged since it generated 2464 the information in the first place. 2466 Transporting detailed CDNI logging information over the HTTP based 2467 CDNI Logging Interface does not represent a particular privacy 2468 concern because it is protected by usual IETF privacy-protection 2469 mechanism (e.g.,TLS). 2471 When HTTP redirection is used between the uCDN and the dCDN, making 2472 detailed CDNI logging information known to the uCDN does not 2473 represent a particular privacy concern because the uCDN is already 2474 exposed at request redirection time to most of the information that 2475 shows up as CDNI logging information (e.g., enduser IP@, URL, HTTP 2476 headers). When DNS redirection is used between the uCDN and the 2477 dCDN, there are cases where there is no privacy concern in making 2478 detailed CDNI logging information known to the uCDN; this may be the 2479 case, for example, where (1) it is considered that because the uCDN 2480 has the authority (with respect to the CSP) and control on how the 2481 requests are delivered (including whether it is served by the uCDN 2482 itself or by a dCDN), the uCDN is entitled to access all detailed 2483 information related to the corresponding deliveries, and (2) there is 2484 no legal reasons to restrict access by the uCDN to all these detailed 2485 information. Conversely, still when DNS redirection is used between 2486 the uCDN and the dCDN, there are cases where there may be some 2487 privacy concern in making detailed CDNI logging information known to 2488 the uCDN; this may be the case, for example, because the uCDN is in a 2489 different jurisdiction to the dCDN resulting is some legal reasons to 2490 restrict access by the uCDN to all the detailed information related 2491 to the deliveries. In this latter case, the privacy concern can be 2492 taken into account when the uCDN and dCDN agree about which fields 2493 are to be conveyed inside the CDNI Logging Files and which privacy 2494 protection mechanism is to be used as discussed in the definition of 2495 the c-groupid field specified in Section 3.4.1. 2497 Another privacy concern arises from the fact that large volumes of 2498 detailed information about content delivery to users, potentially 2499 traceable back to indvidual users, may be collected in CDNI Logging 2500 files. These CDNI Logging files represent high-value targets, likely 2501 concentrated in a fairly centralised system (although the CDNI 2502 Logging architecture does not mandate a particular level of 2503 centralisation/distribution) and at risk of potential data 2504 exfiltration. Note that the means of such data exfiltration are 2505 beyond the scope of the CDNI Logging interface itself (e.g., 2506 corrupted employee, corrupted logging storage system,...). This 2507 privacy concern calls for some protection. 2509 The collection of large volumes of such information into CDNI Logging 2510 Files introduces potential End Users privacy protection concerns. 2511 Mechanisms to address these concerns are discussed in the definition 2512 of the c-groupid field specified in Section 3.4.1. 2514 The use of mutually authenticated TLS to establish a secure session 2515 for the transport of the CDNI Logging feed and CDNI Logging pull as 2516 discussed in Section 7.1 provides confidentiality while the logging 2517 information is in transit and prevents any party other than the 2518 authorised uCDN to gain access to the logging information. 2520 We also note that the query string portion of the URL that may be 2521 conveyed inside the cs-uri and u-uri fields of CDNI Logging Files, or 2522 the HTTP cookies( [RFC6265]) that may be conveyed as part of the 2523 cs() field of CDNI Logging files, may contain 2524 personnal information or information that can be exploited to derive 2525 personal information. Where this is a concern, the CDNI Logging 2526 interface specification allows the dCDN to not include the cs-uri and 2527 to include a u-uri that removes (or hides) the sensitive part of the 2528 query string and allows the dCDN to not include the cs() fields corresponding to HTTP headers associated with cookies. 2531 8. Acknowledgments 2533 This document borrows from the W3C Extended Log Format [ELF]. 2535 Rob Murray significantly contributed into the text of Section 4.1. 2537 The authors thank Ben Niven-Jenkins, Kevin Ma, David Mandelberg and 2538 Ray van Brandenburg for their ongoing input. 2540 Brian Trammel and Rich Salz made significant contributions into 2541 making this interface privacy-friendly. 2543 Finally, we also thank Sebastien Cubaud, Pawel Grochocki, Christian 2544 Jacquenet, Yannick Le Louedec, Anne Marrec, Emile Stephan, Fabio 2545 Costa, Sara Oueslati, Yvan Massot, Renaud Edel, Joel Favier and the 2546 contributors of the EU FP7 OCEAN project for their input in the early 2547 versions of this document. 2549 9. References 2551 9.1. Normative References 2553 [AES] NIST, "Advanced Encryption Standard (AES)", August 2015, 2554 . 2556 [GCM] NIST, "Recommendation for Block Cipher Modes of Operation: 2557 Galois/Counter Mode (GCM) and GMAC", November 2007, . 2560 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2561 Requirement Levels", BCP 14, RFC 2119, 2562 DOI 10.17487/RFC2119, March 1997, 2563 . 2565 [RFC3339] Klyne, G. and C. Newman, "Date and Time on the Internet: 2566 Timestamps", RFC 3339, DOI 10.17487/RFC3339, July 2002, 2567 . 2569 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 2570 Resource Identifier (URI): Generic Syntax", STD 66, 2571 RFC 3986, DOI 10.17487/RFC3986, January 2005, 2572 . 2574 [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally 2575 Unique IDentifier (UUID) URN Namespace", RFC 4122, 2576 DOI 10.17487/RFC4122, July 2005, 2577 . 2579 [RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom 2580 Syndication Format", RFC 4287, DOI 10.17487/RFC4287, 2581 December 2005, . 2583 [RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data 2584 Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006, 2585 . 2587 [RFC5005] Nottingham, M., "Feed Paging and Archiving", RFC 5005, 2588 DOI 10.17487/RFC5005, September 2007, 2589 . 2591 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 2592 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 2593 DOI 10.17487/RFC5226, May 2008, 2594 . 2596 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 2597 Specifications: ABNF", STD 68, RFC 5234, 2598 DOI 10.17487/RFC5234, January 2008, 2599 . 2601 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2602 Protocol (HTTP/1.1): Message Syntax and Routing", 2603 RFC 7230, DOI 10.17487/RFC7230, June 2014, 2604 . 2606 [RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2607 Protocol (HTTP/1.1): Semantics and Content", RFC 7231, 2608 DOI 10.17487/RFC7231, June 2014, 2609 . 2611 [RFC7232] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2612 Protocol (HTTP/1.1): Conditional Requests", RFC 7232, 2613 DOI 10.17487/RFC7232, June 2014, 2614 . 2616 [RFC7233] Fielding, R., Ed., Lafon, Y., Ed., and J. Reschke, Ed., 2617 "Hypertext Transfer Protocol (HTTP/1.1): Range Requests", 2618 RFC 7233, DOI 10.17487/RFC7233, June 2014, 2619 . 2621 [RFC7234] Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, 2622 Ed., "Hypertext Transfer Protocol (HTTP/1.1): Caching", 2623 RFC 7234, DOI 10.17487/RFC7234, June 2014, 2624 . 2626 [RFC7235] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2627 Protocol (HTTP/1.1): Authentication", RFC 7235, 2628 DOI 10.17487/RFC7235, June 2014, 2629 . 2631 [RFC7525] Sheffer, Y., Holz, R., and P. Saint-Andre, 2632 "Recommendations for Secure Use of Transport Layer 2633 Security (TLS) and Datagram Transport Layer Security 2634 (DTLS)", BCP 195, RFC 7525, DOI 10.17487/RFC7525, May 2635 2015, . 2637 [RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext 2638 Transfer Protocol Version 2 (HTTP/2)", RFC 7540, 2639 DOI 10.17487/RFC7540, May 2015, 2640 . 2642 [SHA-3] NIST, "SHA-3 STANDARD: PERMUTATION-BASED HASH AND 2643 EXTENDABLE OUTPUT FUNCTIONS", November 2001, 2644 . 2646 9.2. Informative References 2648 [CHAR_SET] 2649 "IANA Character Sets registry", 2650 . 2653 [ELF] Phillip M. Hallam-Baker, and Brian Behlendorf, "Extended 2654 Log File Format, W3C (work in progress), WD-logfile- 2655 960323", . 2657 [I-D.ietf-cdni-metadata] 2658 Niven-Jenkins, B., Murray, R., Caulfield, M., and K. Ma, 2659 "CDN Interconnection Metadata", draft-ietf-cdni- 2660 metadata-17 (work in progress), May 2016. 2662 [I-D.ietf-tls-rfc5246-bis] 2663 Dierks, T. and E. Rescorla, "The Transport Layer Security 2664 (TLS) Protocol Version 1.3", draft-ietf-tls-rfc5246-bis-00 2665 (work in progress), April 2014. 2667 [I-D.snell-atompub-link-extensions] 2668 Snell, J., "Atom Link Extensions", draft-snell-atompub- 2669 link-extensions-09 (work in progress), June 2012. 2671 [RFC1945] Berners-Lee, T., Fielding, R., and H. Frystyk, "Hypertext 2672 Transfer Protocol -- HTTP/1.0", RFC 1945, 2673 DOI 10.17487/RFC1945, May 1996, 2674 . 2676 [RFC2818] Rescorla, E., "HTTP Over TLS", RFC 2818, 2677 DOI 10.17487/RFC2818, May 2000, 2678 . 2680 [RFC5869] Krawczyk, H. and P. Eronen, "HMAC-based Extract-and-Expand 2681 Key Derivation Function (HKDF)", RFC 5869, 2682 DOI 10.17487/RFC5869, May 2010, 2683 . 2685 [RFC6234] Eastlake 3rd, D. and T. Hansen, "US Secure Hash Algorithms 2686 (SHA and SHA-based HMAC and HKDF)", RFC 6234, 2687 DOI 10.17487/RFC6234, May 2011, 2688 . 2690 [RFC6265] Barth, A., "HTTP State Management Mechanism", RFC 6265, 2691 DOI 10.17487/RFC6265, April 2011, 2692 . 2694 [RFC6707] Niven-Jenkins, B., Le Faucheur, F., and N. Bitar, "Content 2695 Distribution Network Interconnection (CDNI) Problem 2696 Statement", RFC 6707, DOI 10.17487/RFC6707, September 2697 2012, . 2699 [RFC6770] Bertrand, G., Ed., Stephan, E., Burbridge, T., Eardley, 2700 P., Ma, K., and G. Watson, "Use Cases for Content Delivery 2701 Network Interconnection", RFC 6770, DOI 10.17487/RFC6770, 2702 November 2012, . 2704 [RFC6983] van Brandenburg, R., van Deventer, O., Le Faucheur, F., 2705 and K. Leung, "Models for HTTP-Adaptive-Streaming-Aware 2706 Content Distribution Network Interconnection (CDNI)", 2707 RFC 6983, DOI 10.17487/RFC6983, July 2013, 2708 . 2710 [RFC7336] Peterson, L., Davie, B., and R. van Brandenburg, Ed., 2711 "Framework for Content Distribution Network 2712 Interconnection (CDNI)", RFC 7336, DOI 10.17487/RFC7336, 2713 August 2014, . 2715 [RFC7337] Leung, K., Ed. and Y. Lee, Ed., "Content Distribution 2716 Network Interconnection (CDNI) Requirements", RFC 7337, 2717 DOI 10.17487/RFC7337, August 2014, 2718 . 2720 [RFC7736] Ma, K., "Content Delivery Network Interconnection (CDNI) 2721 Media Type Registration", RFC 7736, DOI 10.17487/RFC7736, 2722 December 2015, . 2724 Authors' Addresses 2726 Francois Le Faucheur (editor) 2727 FR 2729 Phone: +33 6 19 98 50 90 2730 Email: flefauch@gmail.com 2732 Gilles Bertrand (editor) 2734 Phone: +41 76 675 91 44 2735 Email: gilbertrand@gmail.com 2737 Iuniana Oprescu (editor) 2738 FR 2740 Email: iuniana.oprescu@gmail.com 2742 Roy Peterkofsky 2743 Google Inc. 2744 345 Spear St, 4th Floor 2745 San Francisco CA 94105 2746 USA 2748 Email: peterkofsky@google.com