idnits 2.17.1 draft-ietf-cdni-logging-23.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 18, 2016) is 2961 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFCthis' is mentioned on line 2316, but not defined ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) ** Obsolete normative reference: RFC 7231 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7232 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7233 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7234 (Obsoleted by RFC 9111) ** Obsolete normative reference: RFC 7235 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7525 (Obsoleted by RFC 9325) ** Obsolete normative reference: RFC 7540 (Obsoleted by RFC 9113) == Outdated reference: A later version (-21) exists of draft-ietf-cdni-metadata-12 -- Obsolete informational reference (is this intentional?): RFC 2818 (Obsoleted by RFC 9110) Summary: 9 errors (**), 0 flaws (~~), 3 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force F. Le Faucheur, Ed. 3 Internet-Draft Cisco Systems 4 Intended status: Standards Track G. Bertrand, Ed. 5 Expires: September 19, 2016 Orange 6 I. Oprescu, Ed. 8 R. Peterkofsky 9 Google Inc. 10 March 18, 2016 12 CDNI Logging Interface 13 draft-ietf-cdni-logging-23 15 Abstract 17 This memo specifies the Logging interface between a downstream CDN 18 (dCDN) and an upstream CDN (uCDN) that are interconnected as per the 19 CDN Interconnection (CDNI) framework. First, it describes a 20 reference model for CDNI logging. Then, it specifies the CDNI 21 Logging File format and the actual protocol for exchange of CDNI 22 Logging Files. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at http://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on September 19, 2016. 41 Copyright Notice 43 Copyright (c) 2016 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (http://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 59 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 60 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 5 61 2. CDNI Logging Reference Model . . . . . . . . . . . . . . . . 5 62 2.1. CDNI Logging interactions . . . . . . . . . . . . . . . . 5 63 2.2. Overall Logging Chain . . . . . . . . . . . . . . . . . . 8 64 2.2.1. Logging Generation and During-Generation Aggregation 9 65 2.2.2. Logging Collection . . . . . . . . . . . . . . . . . 10 66 2.2.3. Logging Filtering . . . . . . . . . . . . . . . . . . 10 67 2.2.4. Logging Rectification and Post-Generation Aggregation 11 68 2.2.5. Log-Consuming Applications . . . . . . . . . . . . . 12 69 2.2.5.1. Maintenance/Debugging . . . . . . . . . . . . . . 12 70 2.2.5.2. Accounting . . . . . . . . . . . . . . . . . . . 13 71 2.2.5.3. Analytics and Reporting . . . . . . . . . . . . . 13 72 2.2.5.4. Content Protection . . . . . . . . . . . . . . . 13 73 2.2.5.5. Notions common to multiple Log Consuming 74 Applications . . . . . . . . . . . . . . . . . . 14 75 3. CDNI Logging File . . . . . . . . . . . . . . . . . . . . . . 16 76 3.1. Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 16 77 3.2. CDNI Logging File Structure . . . . . . . . . . . . . . . 17 78 3.3. CDNI Logging Directives . . . . . . . . . . . . . . . . . 20 79 3.4. CDNI Logging Records . . . . . . . . . . . . . . . . . . 24 80 3.4.1. HTTP Request Logging Record . . . . . . . . . . . . . 25 81 3.5. CDNI Logging File Extension . . . . . . . . . . . . . . . 35 82 3.6. CDNI Logging File Example . . . . . . . . . . . . . . . . 36 83 3.7. Cascaded CDNI Logging Files Example . . . . . . . . . . . 38 84 4. Protocol for Exchange of CDNI Logging File After Full 85 Collection . . . . . . . . . . . . . . . . . . . . . . . . . 41 86 4.1. CDNI Logging Feed . . . . . . . . . . . . . . . . . . . . 42 87 4.1.1. Atom Formatting . . . . . . . . . . . . . . . . . . . 42 88 4.1.2. Updates to Log Files and the Feed . . . . . . . . . . 42 89 4.1.3. Redundant Feeds . . . . . . . . . . . . . . . . . . . 43 90 4.1.4. Example CDNI Logging Feed . . . . . . . . . . . . . . 43 91 4.2. CDNI Logging File Pull . . . . . . . . . . . . . . . . . 45 92 5. Protocol for Exchange of CDNI Logging File During Collection 46 93 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 47 94 6.1. CDNI Logging Directive Names Registry . . . . . . . . . . 47 95 6.2. CDNI Logging File version Registry . . . . . . . . . . . 48 96 6.3. CDNI Logging record-types Registry . . . . . . . . . . . 48 97 6.4. CDNI Logging Field Names Registry . . . . . . . . . . . . 49 98 6.5. CDNI Logging MIME Media Type . . . . . . . . . . . . . . 50 99 7. Security Considerations . . . . . . . . . . . . . . . . . . . 51 100 7.1. Authentication, Authorization, Confidentiality, Integrity 101 Protection . . . . . . . . . . . . . . . . . . . . . . . 51 102 7.2. Denial of Service . . . . . . . . . . . . . . . . . . . . 52 103 7.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 52 104 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 53 105 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 54 106 9.1. Normative References . . . . . . . . . . . . . . . . . . 54 107 9.2. Informative References . . . . . . . . . . . . . . . . . 55 108 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 57 110 1. Introduction 112 This memo specifies the CDNI Logging interface between a downstream 113 CDN (dCDN) and an upstream CDN (uCDN). First, it describes a 114 reference model for CDNI logging. Then, it specifies the CDNI 115 Logging File format and the actual protocol for exchange of CDNI 116 Logging Files. 118 The reader should be familiar with the following documents: 120 o CDNI problem statement [RFC6707] and framework [RFC7336] identify 121 a Logging interface, 123 o Section 8 of [RFC7337] specifies a set of requirements for 124 Logging, 126 o [RFC6770] outlines real world use-cases for interconnecting CDNs. 127 These use cases require the exchange of Logging information 128 between the dCDN and the uCDN. 130 As stated in [RFC6707], "the CDNI Logging interface enables details 131 of logs or events to be exchanged between interconnected CDNs". 133 The present document describes: 135 o The CDNI Logging reference model (Section 2), 137 o The CDNI Logging File format (Section 3), 139 o The CDNI Logging File Exchange protocol (Section 4). 141 1.1. Terminology 143 In this document, the first letter of each CDNI-specific term is 144 capitalized. We adopt the terminology described in [RFC6707] and 145 [RFC7336], and extend it with the additional terms defined below. 147 Intra-CDN Logging information: logging information generated and 148 collected within a CDN. The format of the Intra-CDN Logging 149 information may be different to the format of the CDNI Logging 150 information. 152 CDNI Logging information: logging information exchanged across CDNs 153 using the CDNI Logging Interface. 155 Logging information: logging information generated and collected 156 within a CDN or obtained from another CDN using the CDNI Logging 157 Interface. 159 CDNI Logging Field: an atomic element of information that can be 160 included in a CDNI Logging Record. The time an event/task started, 161 the IP address of an End User to whom content was delivered, and the 162 Uniform Resource Identifier (URI) of the content delivered, are 163 examples of CDNI Logging fields. 165 CDNI Logging Record: an information record providing information 166 about a specific event. This comprises a collection of CDNI Logging 167 fields. 169 CDNI Logging File: a file containing CDNI Logging Records, as well as 170 additional information facilitating the processing of the CDNI 171 Logging Records. 173 CDN Reporting: the process of providing the relevant information that 174 will be used to create a formatted content delivery report provided 175 to the CSP in deferred time. Such information typically includes 176 aggregated data that can cover a large period of time (e.g., from 177 hours to several months). Uses of Reporting include the collection 178 of charging data related to CDN services and the computation of Key 179 Performance Indicators (KPIs). 181 CDN Monitoring: the process of providing or displaying content 182 delivery information in a timely fashion with respect to the 183 corresponding deliveries. Monitoring typically includes visibility 184 of the deliveries in progress for service operation purposes. It 185 presents a view of the global health of the services as well as 186 information on usage and performance, for network services 187 supervision and operation management. In particular, monitoring data 188 can be used to generate alarms. 190 1.2. Requirements Language 192 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 193 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 194 "OPTIONAL" in this document are to be interpreted as described in RFC 195 2119 [RFC2119]. 197 2. CDNI Logging Reference Model 199 2.1. CDNI Logging interactions 201 The CDNI logging reference model between a given uCDN and a given 202 dCDN involves the following interactions: 204 o customization by the uCDN of the CDNI Logging information to be 205 provided by the dCDN to the uCDN (e.g., control of which CDNI 206 Logging fields are to be communicated to the uCDN for a given task 207 performed by the dCDN or control of which types of events are to 208 be logged). The dCDN takes into account this CDNI Logging 209 customization information to determine what Logging information to 210 provide to the uCDN, but it may, or may not, take into account 211 this CDNI Logging customization information to influence what CDN 212 logging information is to be generated and collected within the 213 dCDN (e.g., even if the uCDN requests a restricted subset of the 214 logging information, the dCDN may elect to generate a broader set 215 of logging information). The mechanism to support the 216 customization by the uCDN of CDNI Logging information is outside 217 the scope of this document and left for further study. Until such 218 a mechanism is available, the uCDN and dCDN are expected to agree 219 off-line on what exact set of CDNI Logging information is to be 220 provided by the dCDN to the uCDN, and to rely on management plane 221 actions to configure the CDNI Logging functions in the dCDN to 222 generate this information set and in the uCDN to expect this 223 information set. 225 o generation and collection by the dCDN of the intra-CDN Logging 226 information related to the completion of any task performed by the 227 dCDN on behalf of the uCDN (e.g., delivery of the content to an 228 End User) or related to events happening in the dCDN that are 229 relevant to the uCDN (e.g., failures or unavailability in dCDN). 230 This takes place within the dCDN and does not directly involve 231 CDNI interfaces. 233 o communication by the dCDN to the uCDN of the Logging information 234 collected by the dCDN relevant to the uCDN. This is supported by 235 the CDNI Logging interface and in the scope of the present 236 document. For example, the uCDN may use this Logging information 237 to charge the CSP, to perform analytics and monitoring for 238 operational reasons, to provide analytics and monitoring views on 239 its content delivery to the CSP or to perform trouble-shooting. 240 This document exclusively specifies non-real-time exchange of 241 Logging information. Closer to real-time exchange of Logging 242 information (say sub-minute or sub-second) is outside the scope of 243 the present document and left for further study. This document 244 exclusively specifies exchange of Logging information related to 245 content delivery. Exchange of Logging information related to 246 operational events (e.g., dCDN request routing function 247 unavailable, content acquisition failure by dCDN) for audit or 248 operational reactive adjustments by uCDN is outside the scope of 249 the present document and left for further study. 251 o customization by the dCDN of the CDNI Logging information to be 252 provided by the uCDN on behalf of the dCDN. The mechanism to 253 support the customization by the dCDN of CDNI Logging information 254 is outside the scope of this document and left for further study. 256 o generation and collection by the uCDN of Intra-CDN Logging 257 information related to the completion of any task performed by the 258 uCDN on behalf of the dCDN (e.g., serving of content by uCDN to 259 dCDN for acquisition purposes by dCDN) or related to events 260 happening in the uCDN that are relevant to the dCDN. This takes 261 place within the uCDN and does not directly involve CDNI 262 interfaces. 264 o communication by the uCDN to the dCDN of the Logging information 265 collected by the uCDN relevant to the dCDN. For example, the dCDN 266 might potentially benefit from this information for security 267 auditing or content acquisition troubleshooting. This is outside 268 the scope of this document and left for further study. 270 Figure 1 provides an example of CDNI Logging interactions (focusing 271 only on the interactions that are in the scope of this document) in a 272 particular scenario where four CDNs are involved in the delivery of 273 content from a given CSP: the uCDN has a CDNI interconnection with 274 dCDN-1 and dCDN-2. In turn, dCDN-2 has a CDNI interconnection with 275 dCDN-3, where dCDN-2 is acting as an upstream CDN relative to dCDN-3. 276 In this example, uCDN, dCDN-1, dCDN-2 and dCDN-3 all participate in 277 the delivery of content for the CSP. In this example, the CDNI 278 Logging interface enables the uCDN to obtain Logging information from 279 all the dCDNs involved in the delivery. In the example, the uCDN 280 uses the Logging information: 282 o to analyze the performance of the delivery performed by the dCDNs 283 and to adjust its operations after the fact (e.g., request 284 routing) as appropriate, 286 o to provide (non-real-time) reporting and monitoring information to 287 the CSP. 289 For instance, the uCDN merges Logging information, extracts relevant 290 KPIs, and presents a formatted report to the CSP, in addition to a 291 bill for the content delivered by uCDN itself or by its dCDNs on the 292 CSP's behalf. The uCDN may also provide Logging information as raw 293 log files to the CSP, so that the CSP can use its own logging 294 analysis tools. 296 +-----+ 297 | CSP | 298 +-----+ 299 ^ Reporting and monitoring data 300 * Billing 301 ,--*--. 302 Logging ,-' `-. 303 Data =>( uCDN )<= Logging 304 // `-. _,-' \\ Data 305 || `-'-'-' || 306 ,-----. ,-----. 307 ,-' `-. ,-' `-. 308 ( dCDN-1 ) ( dCDN-2 )<== Logging 309 `-. ,-' `-. _,-' \\ Data 310 `--'--' `--'-' || 311 ,-----. 312 ,' `-. 313 ( dCDN-3 ) 314 `. ,-' 315 `--'--' 317 ===> CDNI Logging Interface 318 ***> outside the scope of CDNI 320 Figure 1: Interactions in CDNI Logging Reference Model 322 A downstream CDN relative to uCDN (e.g., dCDN-2) integrates the 323 relevant Logging information obtained from its own downstream CDNs 324 (i.e., dCDN-3) in the Logging information that it provides to the 325 uCDN, so that the uCDN ultimately obtains all Logging information 326 relevant to a CSP for which it acts as the authoritative CDN. Such 327 aggregation is further discussed in Section 3.7. 329 Note that the format of Logging information that a CDN provides over 330 the CDNI interface might be different from the one that the CDN uses 331 internally. In this case, the CDN needs to reformat the Logging 332 information before it provides this information to the other CDN over 333 the CDNI Logging interface. Similarly, a CDN might reformat the 334 Logging information that it receives over the CDNI Logging interface 335 before injecting it into its log-consuming applications or before 336 providing some of this Logging information to the CSP. Such 337 reformatting operations introduce latency in the logging distribution 338 chain and introduce a processing burden. Therefore, there are 339 benefits in specifying CDNI Logging formats that are suitable for use 340 inside CDNs and also are close to the intra-CDN Logging formats 341 commonly used in CDNs today. 343 2.2. Overall Logging Chain 345 This section discusses the overall logging chain within and across 346 CDNs to clarify how CDN Logging information is expected to fit in 347 this overall chain. Figure 2 illustrates the overall logging chain 348 within the dCDN, across CDNs using the CDNI Logging interface and 349 within the uCDN. Note that the logging chain illustrated in the 350 Figure is obviously only an example and varies depending on the 351 specific environments. For example, there may be more or fewer 352 instantiations of each entity (e.g., there may be 4 Log consuming 353 applications in a given CDN). As another example, there may be one 354 instance of Rectification process per Log Consuming Application 355 instead of a shared one. 357 Log Consuming Log Consuming 358 App App 359 ^ ^ 360 | | 361 Rectification---------- 362 ^ 363 | 364 Filtering 365 ^ 366 | 367 Collection 368 ^ ^ 369 | | 370 | Generation 371 | 372 | uCDN 373 CDNI Logging --------------------------------------------------- 374 exchange dCDN 375 ^ 376 | Log Consuming Log Consuming 377 | App App 378 | ^ ^ 379 | | | 380 Rectification Rectification--------- 381 ^ ^ 382 | | 383 Filtering 384 ^ 385 | 386 Collection 387 ^ ^ 388 | | 389 Generation Generation 391 Figure 2: CDNI Logging in the overall Logging Chain 393 The following subsections describe each of the processes potentially 394 involved in the logging chain of Figure 2. 396 2.2.1. Logging Generation and During-Generation Aggregation 398 CDNs typically generate Logging information for all significant task 399 completions, events, and failures. Logging information is typically 400 generated by many devices in the CDN including the surrogates, the 401 request routing system, and the control system. 403 The amount of Logging information generated can be huge. Therefore, 404 during contract negotiations, interconnected CDNs often agree on a 405 retention duration for Logging information, and/or potentially on a 406 maximum volume of Logging information that the dCDN ought to keep. 407 If this volume is exceeded, the dCDN is expected to alert the uCDN 408 but may not keep more Logging information for the considered time 409 period. In addition, CDNs may aggregate Logging information and 410 transmit only summaries for some categories of operations instead of 411 the full Logging information. Note that such aggregation leads to an 412 information loss, which may be problematic for some usages of the 413 Logging information (e.g., debugging). 415 [RFC6983] discusses logging for HTTP Adaptive Streaming (HAS). In 416 accordance with the recommendations articulated there, it is expected 417 that a surrogate will generate separate Logging information for 418 delivery of each chunk of HAS content. This ensures that separate 419 Logging information can then be provided to interconnected CDNs over 420 the CDNI Logging interface. Still in line with the recommendations 421 of [RFC6983], the Logging information for per-chunk delivery may 422 include some information (a Content Collection IDentifier and a 423 Session IDentifier) intended to facilitate subsequent post-generation 424 aggregation of per-chunk logs into per-session logs. Note that a CDN 425 may also elect to generate aggregate per-session logs when performing 426 HAS delivery, but this needs to be in addition to, and not instead 427 of, the per-chunk delivery logs. We note that aggregate per-session 428 logs for HAS delivery are for further study and outside the scope of 429 this document. 431 2.2.2. Logging Collection 433 This is the process that continuously collects Logging information 434 generated by the log-generating entities within a CDN. 436 In a CDNI environment, in addition to collecting Logging information 437 from log-generating entities within the local CDN, the Collection 438 process also collects Logging information provided by another CDN, or 439 other CDNs, through the CDNI Logging interface. This is illustrated 440 in Figure 2 where we see that the Collection process of the uCDN 441 collects Logging information from log-generating entities within the 442 uCDN as well as Logging information coming from the dCDNs through the 443 CDNI Logging interface. 445 2.2.3. Logging Filtering 447 A CDN may be required to only present different subsets of the whole 448 Logging information collected to various log-consuming applications. 449 This is achieved by the Filtering process. 451 In particular, the Filtering process can also filter the right subset 452 of Logging information that needs to be provided to a given 453 interconnected CDN. For example, the filtering process in the dCDN 454 can be used to ensure that only the Logging information related to 455 tasks performed on behalf of a given uCDN are made available to that 456 uCDN (thereby filtering out all the Logging information related to 457 deliveries by the dCDN of content for its own CSPs). Similarly, the 458 Filtering process may filter or partially mask some fields, for 459 example, to protect End Users' privacy when communicating CDNI 460 Logging information to another CDN. Filtering of Logging information 461 prior to communication of this information to other CDNs via the CDNI 462 Logging interface requires that the downstream CDN can recognize the 463 subset of Logging information that relate to each interconnected CDN. 465 The CDN will also filter some internal scope information such as 466 information related to its internal alarms (security, failures, load, 467 etc). 469 In some use cases described in [RFC6770], the interconnected CDNs do 470 not want to disclose details on their internal topology. The 471 filtering process can then also filter confidential data on the 472 dCDNs' topology (number of servers, location, etc.). In particular, 473 information about the requests served by each Surrogate may be 474 confidential. Therefore, the Logging information needs to be 475 protected so that data such as Surrogates' hostnames are not 476 disclosed to the uCDN. In the "Inter-Affiliates Interconnection" use 477 case, this information may be disclosed to the uCDN because both the 478 dCDN and the uCDN are operated by entities of the same group. 480 2.2.4. Logging Rectification and Post-Generation Aggregation 482 If Logging information is generated periodically, it is important 483 that the sessions that start in one Logging period and end in another 484 are correctly reported. If they are reported in the starting period, 485 then the Logging information of this period will be available only 486 after the end of the session, which delays the Logging information 487 generation. A simple approach is to provide the complete Logging 488 Record for a session in the Logging Period of the session end. 490 A Logging rectification/update mechanism could be useful to reach a 491 good trade-off between the Logging information generation delay and 492 the Logging information accuracy. 494 In the presence of HAS, some log-consuming applications can benefit 495 from aggregate per-session logs. For example, for analytics, per- 496 session logs allow display of session-related trends which are much 497 more meaningful for some types of analysis than chunk-related trends. 498 In the case where aggregate logs have been generated directly by the 499 log-generating entities, those can be used by the applications. In 500 the case where aggregate logs have not been generated, the 501 Rectification process can be extended with a Post-Generation 502 Aggregation process that generates per-session logs from the per- 503 chunk logs, possibly leveraging the information included in the per- 504 chunk logs for that purpose (Content Collection IDentifier and a 505 Session IDentifier). However, in accordance with [RFC6983], this 506 document does not define exchange of such aggregate logs on the CDNI 507 Logging interface. We note that this is for further study and 508 outside the scope of this document. 510 2.2.5. Log-Consuming Applications 512 2.2.5.1. Maintenance/Debugging 514 Logging information is useful to permit the detection (and limit the 515 risk) of content delivery failures. In particular, Logging 516 information facilitates the detection of configuration issues. 518 To detect faults, Logging information needs to report success and 519 failure of CDN delivery operations. The uCDN can summarize such 520 information into KPIs. For instance, Logging information needs to 521 allow the computation of the number of times, during a given time 522 period, that content delivery related to a specific service succeeds/ 523 fails. 525 Logging information enables the CDN providers to identify and 526 troubleshoot performance degradations. In particular, Logging 527 information enables tracking of traffic data (e.g., the amount of 528 traffic that has been forwarded by a dCDN on behalf of an uCDN over a 529 given period of time), which is particularly useful for CDN and 530 network planning operations. 532 Some of these maintenance and debugging applications only require 533 aggregate logging information highly compatible with use of 534 anonymization of IP addresses (as supported by the present document 535 and specified in the definition of the c-groupid field under 536 Section 3.4.1). However, in some situations, it may be useful, where 537 compatible with privacy protection, to access some CDNI Logging 538 Records containing full non-anonymized IP addresses. This is allowed 539 in the definition of the c-groupid (under Section 3.4.1), with very 540 significant privacy protection limitations that are discussed in the 541 definition of the c-groupid field. For example, this may be useful 542 for detailed fault tracking of a particular end user content delivery 543 issue. Where there is a hard requirement by uCDN or CSP to associate 544 a given enduser to individual CDNI Logging Records (e.g., to allow 545 a-posteriori analysis of individual delivery for example in 546 situations of performance-based penalties), instead of using 547 aggregates containing a single client as discussed in the c-groupid 548 field definition, an alternate approach is to ensure that a client 549 identifier is embedded in the request fields that can be logged in a 550 CDNI Logging Record (for example by including the client identifier 551 in the URI query string or in a HTTP Header). That latter approach 552 offers two strong benefits: first, the aggregate inside the c-groupid 553 can contain more than one client, thereby ensuring stronger privacy 554 protection; second, it allows a reliable identification of the client 555 while IP address does not in many situations (e.g., behind NAT, where 556 dynamic IP addresses are used and reused,...). However, care SHOULD 557 be taken that the client identifiers exposed in other fields of the 558 CDNI Records cannot themselves be linked back to actual users. 560 2.2.5.2. Accounting 562 Logging information is essential for accounting, to permit inter-CDN 563 billing and CSP billing by uCDNs. For instance, Logging information 564 provided by dCDNs enables the uCDN to compute the total amount of 565 traffic delivered by every dCDN for a particular Content Provider, as 566 well as, the associated bandwidth usage (e.g., peak, 95th 567 percentile), and the maximum number of simultaneous sessions over a 568 given period of time. 570 2.2.5.3. Analytics and Reporting 572 The goals of analytics include gathering any relevant information in 573 order to be able to develop statistics on content download, analyze 574 user behavior, and monitor the performance and quality of content 575 delivery. For instance, Logging information enables the CDN 576 providers to report on content consumption (e.g., delivered sessions 577 per content) in a specific geographic area. 579 The goal of reporting is to gather any relevant information to 580 monitor the performance and quality of content delivery and allow 581 detection of delivery issues. For instance, reporting could track 582 the average delivery throughput experienced by End Users in a given 583 region for a specific CSP or content set over a period of time. 585 2.2.5.4. Content Protection 587 The goal of content protection is to prevent and monitor unauthorized 588 access, misuse, modification, and denial of access to a content. A 589 set of information is logged in a CDN for security purposes. In 590 particular, a record of access to content is usually collected to 591 permit the CSP to detect infringements of content delivery policies 592 and other abnormal End User behaviors. 594 2.2.5.5. Notions common to multiple Log Consuming Applications 596 2.2.5.5.1. Logging Information Views 598 Within a given log-consuming application, different views may be 599 provided to different users depending on privacy, business, and 600 scalability constraints. 602 For example, an analytics tool run by the uCDN can provide one view 603 to an uCDN operator that exploits all the Logging information 604 available to the uCDN, while the tool may provide a different view to 605 each CSP exploiting only the Logging information related to the 606 content of the given CSP. 608 As another example, maintenance and debugging tools may provide 609 different views to different CDN operators, based on their 610 operational role. 612 2.2.5.5.2. Key Performance Indicators (KPIs) 614 This section presents, for explanatory purposes, a non-exhaustive 615 list of Key Performance Indicators (KPIs) that can be extracted/ 616 produced from logs. 618 Multiple log-consuming applications, such as analytics, monitoring, 619 and maintenance applications, often compute and track such KPIs. 621 In a CDNI environment, depending on the situation, these KPIs may be 622 computed by the uCDN or by the dCDN. But it is usually the uCDN that 623 computes KPIs, because the uCDN and dCDN may have different 624 definitions of the KPIs and the computation of some KPIs requires a 625 vision of all the deliveries performed by the uCDN and all its dCDNs. 627 Here is a list of important examples of KPIs: 629 o Number of delivery requests received from End Users in a given 630 region for each piece of content, during a given period of time 631 (e.g., hour/day/week/month) 633 o Percentage of delivery successes/failures among the aforementioned 634 requests 636 o Number of failures listed by failure type (e.g., HTTP error code) 637 for requests received from End Users in a given region and for 638 each piece of content, during a given period of time (e.g., 639 hour/day/week/month) 641 o Number and cause of premature delivery termination for End Users 642 in a given region and for each piece of content, during a given 643 period of time (e.g., hour/day/week/month) 645 o Maximum and mean number of simultaneous sessions established by 646 End Users in a given region, for a given Content Provider, and 647 during a given period of time (e.g., hour/day/week/month) 649 o Volume of traffic delivered for sessions established by End Users 650 in a given region, for a given Content Provider, and during a 651 given period of time (e.g., hour/day/week/month) 653 o Maximum, mean, and minimum delivery throughput for sessions 654 established by End Users in a given region, for a given Content 655 Provider, and during a given period of time (e.g., hour/day/week/ 656 month) 658 o Cache-hit and byte-hit ratios for requests received from End Users 659 in a given region for each piece of content, during a given period 660 of time (e.g., hour/day/week/month) 662 o Top 10 most popularly requested contents (during a given day/week/ 663 month) 665 o Terminal type (mobile, PC, STB, if this information can be 666 acquired from the browser type inferred from the User Agent 667 string, for example). 669 Additional KPIs can be computed from other sources of information 670 than the Logging information, for instance, data collected by a 671 content portal or by specific client-side application programming 672 interfaces. Such KPIs are out of scope for the present document. 674 The KPIs used depend strongly on the considered log-consuming 675 application -- the CDN operator may be interested in different 676 metrics than the CSP is. In particular, CDN operators are often 677 interested in delivery and acquisition performance KPIs, information 678 related to Surrogates' performance, caching information to evaluate 679 the cache-hit ratio, information about the delivered file size to 680 compute the volume of content delivered during peak hour, etc. 682 Some of the KPIs, for instance those providing an instantaneous 683 vision of the active sessions for a given CSP's content, are useful 684 essentially if they are provided in a timely manner. By contrast, 685 some other KPIs, such as those averaged on a long period of time, can 686 be provided in non-real-time. 688 3. CDNI Logging File 690 3.1. Rules 692 This specification uses the Augmented Backus-Naur Form (ABNF) 693 notation and core rules of [RFC5234]. In particular, the present 694 document uses the following rules from [RFC5234]: 696 CR = %x0D ; carriage return 698 ALPHA = %x41-5A / %x61-7A ; A-Z / a-z 700 DIGIT = %x30-39 ; 0-9 702 DQUOTE = %x22 ; " (Double Quote) 704 CRLF = CR LF ; Internet standard newline 706 HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F" 708 HTAB = %x09 ; horizontal tab 710 LF = %x0A ; linefeed 712 VCHAR = %x21-7E ; visible (printing) characters 714 OCTET = %x00-FF ; 8 bits of data 716 The present document also uses the following rules from [RFC3986]: 718 host = as specified in section 3.2.2 of [RFC3986]. 720 IPv4address = as specified in section 3.2.2 of [RFC3986]. 722 IPv6address = as specified in section 3.2.2 of [RFC3986]. 724 partial-time = as specified in [RFC3339]. 726 The present document also defines the following additional rules: 728 ADDRESS = IPv4address / IPv6address 730 ALPHANUM = ALPHA / DIGIT 732 DATE = 4DIGIT "-" 2DIGIT "-" 2DIGIT 734 ; Dates are encoded as "full-date" specified in [RFC3339]. 736 DEC = 1*DIGIT ["." 1*DIGIT] 738 NAMEFORMAT = ALPHANUM *(ALPHANUM / "_" / "-") 740 QSTRING = DQUOTE *(NDQUOTE / PCT-ENCODED) DQUOTE 742 NDQUOTE = %x20-21 / %x23-24 / %x26-7E / UTF8-2 / UTF8-3 / UTF8-4 744 ; whereby a DQUOTE is conveyed inside a QSTRING unambiguously 745 by escaping it with PCT-ENCODED. 747 PCT-ENCODED = "%" HEXDIG HEXDIG 749 ; percent encoding is used for escaping octets that might be 750 possible in HTTP headers such as bare CR, bare LF, CR LF, HTAB, 751 SP or null. These octets are rendered with percent encoding in 752 ABNF as specified by [RFC3986] in order to avoid considering 753 them as separators for the logging records. 755 NHTABSTRING = 1*(SP / VCHAR) 757 TIME = partial-time 759 USER-COMMENT = * (SP / VCHAR / UTF8-2 / UTF8-3 / UTF8-4) 761 3.2. CDNI Logging File Structure 763 As defined in Section 1.1: a CDNI Logging Field is as an atomic 764 logging information element, a CDNI Logging Record is a collection of 765 CDNI Logging fields containing all logging information corresponding 766 to a single logging event, and a CDNI Logging File contains a 767 collection of CDNI Logging Records. This structure is illustrated in 768 Figure 3. The use of a file structure for transfer of CDNI Logging 769 information is selected since this is the most common practise today 770 for exchange of logging information within and across CDNs. 772 +----------------------------------------------------------+ 773 |CDNI Logging File | 774 | | 775 | #Directive 1 | 776 | #Directive 2 | 777 | ... | 778 | #Directive P | 779 | | 780 | +------------------------------------------------------+ | 781 | |CDNI Logging Record 1 | | 782 | | +-------------+ +-------------+ +-------------+ | | 783 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 784 | | | Field 1 | | Field 2 | | Field N | | | 785 | | +-------------+ +-------------+ +-------------+ | | 786 | +------------------------------------------------------+ | 787 | | 788 | +------------------------------------------------------+ | 789 | |CDNI Logging Record 2 | | 790 | | +-------------+ +-------------+ +-------------+ | | 791 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 792 | | | Field 1 | | Field 2 | | Field N | | | 793 | | +-------------+ +-------------+ +-------------+ | | 794 | +------------------------------------------------------+ | 795 | | 796 | ... | 797 | | 798 | #Directive P+1 | 799 | | 800 | ... | 801 | | 802 | +------------------------------------------------------+ | 803 | |CDNI Logging Record M | | 804 | | +-------------+ +-------------+ +-------------+ | | 805 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 806 | | | Field 1 | | Field 2 | | Field N | | | 807 | | +-------------+ +-------------+ +-------------+ | | 808 | +------------------------------------------------------+ | 809 | | 810 | | 811 | #Directive P+Q | 812 +----------------------------------------------------------+ 814 Figure 3: Structure of Logging Files 816 The CDNI Logging File format is inspired from the W3C Extended Log 817 File Format [ELF]. However, it is fully specified by the present 818 document. Where the present document differs from the W3C Extended 819 Log File Format, an implementation of the CDNI Logging interface MUST 820 comply with the present document. The W3C Extended Log File Format 821 was used as a starting point, reused where possible and expanded when 822 necessary. 824 Using a format that resembles the W3C Extended Log File Format is 825 intended to keep CDNI logging format close to the intra-CDN Logging 826 information format commonly used in CDNs today, thereby minimizing 827 systematic translation at CDN/CDNI boundary. 829 A CDNI Logging File MUST contain a sequence of lines containing US- 830 ASCII characters [CHAR_SET] terminated by CRLF. Each line of a CDNI 831 Logging File MUST contain either a directive or a CDNI Logging 832 Record. 834 Directives record information about the CDNI Logging process itself. 835 Lines containing directives MUST begin with the "#" character. 836 Directives are specified in Section 3.3. 838 Logging Records provide actual details of the logged event. Logging 839 Records are specified in Section 3.4. 841 The CDNI Logging File has a specific structure. It always starts 842 with a directive line and the first directive it contains MUST be the 843 version. 845 The directive lines form together a group that contains at least one 846 directive line. Each directives group is followed by a group of 847 logging records. The records group contains zero or more actual 848 logging record lines about the event being logged. A record line 849 consists of the values corresponding to all or a subset of the 850 possible Logging fields defined within the scope of the record-type 851 directive. These values MUST appear in the order defined by the 852 fields directive. 854 Note that future extensions MUST be compliant with the previous 855 description. The following examples depict the structure of a 856 CDNILOGFILE as defined currently by the record-type 857 "cdni_http_request_v1." 859 DIRLINE = "#" directive CRLF 861 DIRGROUP = 1*DIRLINE 863 RECLINE = 866 RECGROUP = *RECLINE 868 CDNILOGFILE = 1*(DIRGROUP RECGROUP) 870 All directive names and field names defined in the logging file are 871 case-insensitive as per the basic ABNF([RFC5234]). 873 3.3. CDNI Logging Directives 875 A CDNI Logging directive line contains the directive name followed by 876 ":" HTAB and the directive value. 878 Directive names MUST be of the format NAMEFORMAT. All directive 879 names MUST be registered in the CDNI Logging Directives Names 880 registry. Unknown directives MUST be ignored. Directive values can 881 have various formats. All possible directive values for the record- 882 type "cdni_http_request_v1" are further detailed in this section. 884 The following example shows the structure of a directive and 885 enumerates strictly the directive values presently defined in the 886 version "CDNI/1.0" of the CDNI Logging File. 888 directive = DIRNAME ":" HTAB DIRVAL 890 DIRNAME = NAMEFORMAT 892 DIRVAL = NHTABSTRING / QSTRING / host / USER-COMMENT / FIENAME * 893 (HTAB FIENAME) / 64HEXDIG 895 An implementation of the CDNI Logging interface MUST support all of 896 the following directives, listed below by their directive name: 898 o version: 900 * format: NHTABSTRING 902 * directive value: indicates the version of the CDNI Logging File 903 format. The entity transmitting a CDNI Logging File as per the 904 present document MUST set the value to "CDNI/1.0". In the 905 future, other versions of CDNI Logging File might be specified; 906 those would use a value different to "CDNI/1.0" allowing the 907 entity receiving the CDNI Logging File to identify the 908 corresponding version. 910 * occurrence: there MUST be one and only one instance of this 911 directive per CDNI Logging File. It MUST be the first line of 912 the CDNI Logging File. 914 * example: "version: HTAB CDNI/1.0". 916 o UUID: 918 * format: NHTABSTRING 920 * directive value: this a Uniform Resource Name (URN) from the 921 Universally Unique IDentifier (UUID) URN namespace specified in 922 [RFC4122]). The UUID contained in the URN uniquely identifies 923 the CDNI Logging File. 925 * occurrence: there MUST be one and only one instance of this 926 directive per CDNI Logging File. 928 * example: "UUID: HTAB NHTABSTRING". 930 o claimed-origin: 932 * format: host 934 * directive value: this contains the claimed identification of 935 the entity transmitting the CDNI Logging File (e.g., the host 936 in a dCDN supporting the CDNI Logging interface) or the entity 937 responsible for transmitting the CDNI Logging File (e.g., the 938 dCDN). 940 * occurrence: there MUST be zero or exactly one instance of this 941 directive per CDNI Logging File. This directive MAY be 942 included by the dCDN. It MUST NOT be included or modified by 943 the uCDN. 945 * example: "claimed-origin: HTAB host". 947 o established-origin: 949 * format: host 951 * directive value: this contains the identification, as 952 established by the entity receiving the CDNI Logging File, of 953 the entity transmitting the CDNI Logging File (e.g., the host 954 in a dCDN supporting the CDNI Logging interface) or the entity 955 responsible for transmitting the CDNI Logging File (e.g., the 956 dCDN). 958 * occurrence: there MUST be zero or exactly one instance of this 959 directive per CDNI Logging File. This directive MAY be added 960 by the uCDN (e.g., before storing the CDNI Logging File). It 961 MUST NOT be included by the dCDN. The mechanisms used by the 962 uCDN to establish and validate the entity responsible for the 963 CDNI Logging File is outside the scope of the present document. 964 We observe that, in particular, this may be achieved through 965 authentication mechanisms that are part of the transport layer 966 of the CDNI Logging File pull mechanism (Section 4.2). 968 * ABNF example: "established-origin: HTAB host". 970 o remark: 972 * format: USER-COMMENT 974 * directive value: this contains comment information. Data 975 contained in this field is to be ignored by analysis tools. 977 * occurrence: there MAY be zero, one or any number of instance of 978 this directive per CDNI Logging File. 980 * example: "remark: HTAB USER-COMMENT". 982 o record-type: 984 * format: NAMEFORMAT 986 * directive value: indicates the type of the CDNI Logging Records 987 that follow this directive, until another record-type directive 988 (or the end of the CDNI Logging File). This can be any CDNI 989 Logging Record type registered in the CDNI Logging Record-types 990 registry (Section 6.3). For example this may be 991 "cdni_http_request_v1" as specified in Section 3.4.1. 993 * occurrence: there MUST be at least one instance of this 994 directive per CDNI Logging File. The first instance of this 995 directive MUST precede a fields directive and MUST precede all 996 CDNI Logging Records. 998 * example: "record-type: HTAB cdni_http_request_v1". 1000 o fields: 1002 * format: FIENAME *(HTAB FIENAME) ; where FIENAME can take any 1003 CDNI Logging field name registered in the CDNI Logging Field 1004 Names registry (Section 6.4) that is valid for the record type 1005 specified in the record-type directive. 1007 * directive value: this lists the names of all the fields for 1008 which a value is to appear in the CDNI Logging Records that 1009 follow the instance of this directive (until another instance 1010 of this directive). The names of the fields, as well as their 1011 occurrences, MUST comply with the corresponding rules specified 1012 in the document referenced in the CDNI Logging Record-types 1013 registry (Section 6.3) for the corresponding CDNI Logging 1014 record-type. 1016 * occurrence: there MUST be at least one instance of this 1017 directive per record-type directive. The first instance of 1018 this directive for a given record-type MUST appear before any 1019 CDNI Logging Record for this record-type. One situation where 1020 more than one instance of the fields directive can appear 1021 within a given CDNI Logging File, is when there is a change, in 1022 the middle of a fairly large logging period, in the agreement 1023 between the uCDN and the dCDN about the set of fields that are 1024 to be exchanged. The multiple occurrences allow records with 1025 the old set of fields and records with the new set of fields to 1026 be carried inside the same Logging File. 1028 * example: "fields: HTAB FIENAME * (HTAB FIENAME)". 1030 o SHA256-hash: 1032 * format: 64HEXDIG 1034 * directive value: This directive permits the detection of a 1035 corrupted CDNI Logging File. This can be useful, for instance, 1036 if a problem occurs on the filesystem of the dCDN Logging 1037 system and leads to a truncation of a logging file. The valid 1038 SHA256-hash value is included in this directive by the entity 1039 that transmits the CDNI Logging File. It MUST be computed by 1040 applying the SHA-256 ([RFC6234]) cryptographic hash function on 1041 the CDNI Logging File, including all the directives and logging 1042 records, up to the SHA256-hash directive itself, excluding the 1043 SHA256-hash directive itself. The SHA256-hash value MUST be 1044 represented as a US-ASCII encoded hexadecimal number, 64 digits 1045 long (representing a 256 bit hash value). The entity receiving 1046 the CDNI Logging File also computes in a similar way the 1047 SHA-256 hash on the received CDNI Logging File and compares 1048 this hash to the value of the SHA256-hash directive. If the 1049 two values are equal, then the received CDNI Logging File is to 1050 be considered non-corrupted. If the two values are different, 1051 the received CDNI Logging File is to be considered corrupted. 1052 The behavior of the entity that received a corrupted CDNI 1053 Logging File is outside the scope of this specification; we 1054 note that the entity MAY attempt to pull again the same CDNI 1055 Logging File from the transmitting entity. If the entity 1056 receiving a non-corrupted CDNI Logging File adds an 1057 established-origin directive, it MUST then recompute and update 1058 the SHA256-hash directive so it also protects the added 1059 established-origin directive. 1061 * occurrence: there MUST be zero or exactly one instance of this 1062 directive. There SHOULD be exactly one instance of this 1063 directive. One situation where that directive could be omitted 1064 is where integrity protection is already provided via another 1065 mechanism (for example if an integrity hash is associated to 1066 the CDNI Logging File out-of-band through the CDNI Logging Feed 1067 ( Section 4.1) leveraging ATOM extensions such as those 1068 proposed in [I-D.snell-atompub-link-extensions]. When present, 1069 the SHA256-hash field MUST be the last line of the CDNI Logging 1070 File. 1072 * example: "SHA256-hash: HTAB 64HEXDIG". 1074 An uCDN-side implementation of the CDNI Logging interface MUST reject 1075 a CDNI Logging File that does not comply with the occurrences 1076 specified above for each and every directive. For example, an uCDN- 1077 side implementation of the CDNI Logging interface receiving a CDNI 1078 Logging file with zero occurrence of the version directive, or with 1079 two occurrences of the SHA256-hash, MUST reject this CDNI Logging 1080 File. 1082 An entity receiving a CDNI Logging File with a value set to 1083 "CDNI/1.0" MUST process the CDNI Logging File as per the present 1084 document. An entity receiving a CDNI Logging File with a value set 1085 to a different value MUST process the CDNI Logging File as per the 1086 specification referenced in the CDNI Logging File version registry 1087 (see Section 6.1) if the implementation supports this specification 1088 and MUST reject the CDNI Logging File otherwise. 1090 3.4. CDNI Logging Records 1092 A CDNI Logging Record consists of a sequence of CDNI Logging fields 1093 relating to that single CDNI Logging Record. 1095 CDNI Logging fields MUST be separated by the "horizontal tabulation 1096 (HTAB)" character. 1098 To facilitate readability, a prefix scheme is used for CDNI Logging 1099 field names in a similar way to the one used in W3C Extended Log File 1100 Format [ELF]. The semantics of the prefix in the present document 1101 is: 1103 o "c-" refers to the User Agent that issues the request (corresponds 1104 to the "client" of W3C Extended Log Format) 1106 o "d-" refers to the dCDN (relative to a given CDN acting as an 1107 uCDN) 1109 o "s-" refers to the dCDN Surrogate that serves the request 1110 (corresponds to the "server" of W3C Extended Log Format) 1112 o "u-" refers to the uCDN (relative to a given CDN acting as a dCDN) 1114 o "cs-" refers to communication from the User Agent towards the dCDN 1115 Surrogate 1117 o "sc-" refers to communication from the dCDN Surrogate towards the 1118 User Agent 1120 An implementation of the CDNI Logging interface as per the present 1121 specification MUST support the CDNI HTTP Request Logging Record as 1122 specified in Section 3.4.1. 1124 A CDNI Logging Record contains the corresponding values for the 1125 fields that are enumerated in the last fields directive before the 1126 current log line. Note that the order in which the field values 1127 appear is dictated by the order of the fields names in the fields 1128 directive. There SHOULD be no dependency between the various fields 1129 values. 1131 3.4.1. HTTP Request Logging Record 1133 This section defines the CDNI Logging Record of record-type 1134 "cdni_http_request_v1". It is applicable to content delivery 1135 performed by the dCDN using HTTP/1.0([RFC1945]), 1136 HTTP/1.1([RFC7230],[RFC7231], [RFC7232], [RFC7233], [RFC7234], 1137 [RFC7235]) or HTTPS ([RFC2818], [RFC7230]). We observe that, in the 1138 case of HTTPS delivery, there may be value in logging additional 1139 information specific to the operation of HTTP over TLS and we note 1140 that this is outside the scope of the present document and may be 1141 addressed in a future document defining another CDNI Logging Record 1142 or another version of the HTTP Request Logging Record. 1144 The "cdni_http_request_v1" record-type is also expected to be 1145 applicable to HTTP/2 [RFC7540] since a fundamental design tenet of 1146 HTTP/2 is to preserve the HTTP/1.1 semantics. We observe that, in 1147 the case of HTTP/2 delivery, there may be value in logging additional 1148 information specific to the additional functionality of HTTP/2 (e.g., 1149 information related to connection identification, to stream 1150 identification, to stream priority and to flow control). We note 1151 that such additional information is outside the scope of the present 1152 document and may be addressed in a future document defining another 1153 CDNI Logging Record or another version of the HTTP Request Logging 1154 Record. 1156 The "cdni_http_request_v1" record-type contains the following CDNI 1157 Logging fields, listed by their field name: 1159 o date: 1161 * format: DATE 1163 * field value: the date at which the processing of request 1164 completed on the Surrogate. 1166 * occurrence: there MUST be one and only one instance of this 1167 field. 1169 o time: 1171 * format: TIME 1173 * field value: the time at which the processing of request 1174 completed on the Surrogate. 1176 * occurrence: there MUST be one and only one instance of this 1177 field. 1179 o time-taken: 1181 * format: DEC 1183 * field value: decimal value of the duration, in seconds, between 1184 the start of the processing of the request and the completion 1185 of the request processing (e.g., completion of delivery) by the 1186 Surrogate. 1188 * occurrence: there MUST be one and only one instance of this 1189 field. 1191 o c-groupid: 1193 * format: NHTABSTRING 1195 * field value: an opaque identifier for an aggregate set of 1196 clients, derived from the client IPv4 or IPv6 address in the 1197 request received by the Surrogate and/or other network-level 1198 identifying information. The c-groupid serves to group clients 1199 into aggregates. Example aggregates include civil geolocation 1200 information (the country, second-level administrative division, 1201 or postal code from which the client is presumed to make the 1202 request based on a geolocation database lookup) or network 1203 topological information (e.g., the BGP AS number announcing the 1204 prefix containing the address). The c-groupid MAY be 1205 structured e.g., US/TN/MEM/38138. Agreement between the dCDN 1206 and the uCDN on a mapping between IPv4 and IPv6 addresses and 1207 aggregates is presumed to occur out-of-band. The aggregation 1208 mapping SHOULD be chosen such that each aggregate contains more 1209 than one client. 1211 + When the aggregate is chosen so that it contains a single 1212 client (e.g., to allow more detailed analytics, or to allow 1213 a-posteriori analysis of individual delivery for example in 1214 situations of performance-based penalties) the c-groupid MAY 1215 be structured where some elements identify aggregates and 1216 one element identifies the client, e.g., US/TN/ 1217 MEM/38138/43a5bdd6-95c4-4d62-be65-7410df0021e2. In that 1218 case: 1220 - the element identifying the client SHOULD be 1221 algorithmically generated (from the client IPv4 or IPv6 1222 address in the request received by the Surrogate and/or 1223 other network-level identifying information) in a way 1224 that SHOULD NOT be linkable back to the global addressing 1225 context and that SHOULD vary over time (to offer 1226 protection against long term attacks). 1228 - It is RECOMMENDED that the mapping varies at least once 1229 every 24 hours. 1231 - The algorithmic mapping and variation over time MAY allow 1232 the uCDN (with the knowledge of the algorithm and time 1233 variation and associated attributes and keys) to 1234 reconstruct the actual client IPv4 or IPv6 address and/or 1235 other network-level identifying information when required 1236 (e.g., to allow a-posteriori analysis of individual 1237 delivery for example in situations of performance-based 1238 penalties). However, these enduser addresses SHOULD only 1239 be reconstructed on-demand and the CDNI Logging File 1240 SHOULD only be stored with the anonymised c-groupid 1241 value. 1243 - Allowing reconstruction of client address information 1244 carries with it grave risks to end-user privacy. Since 1245 the c-groupid is in this case equivalent in 1246 identification power to a client IP address, its use may 1247 be restricted by regulation or law as personally 1248 identifiable information. For this reason, such use is 1249 NOT RECOMMENDED. 1251 - One method for mapping that MAY be be supported by 1252 implementations relies on a symmetric key that is known 1253 only to the uCDN and dCDN and HMAC-based Extract-and- 1254 Expand Key Derivation Function (HKDF) key derivation 1255 ([RFC5869]), as will be used in TLS 1.3 1256 ([I-D.ietf-tls-rfc5246-bis]). When that method is used: 1258 o The uCDN and dCDN need to agree on the "salt" and 1259 "input keying material", as described in Section 2.2 1260 of [RFC5869] and the initial "info" parameter (which 1261 could be something like the business names of the two 1262 organizations in UTF-8, concatenated), as described in 1263 Section 2.3 of [RFC5869]. The hash SHOULD be either 1264 SHA-2 or SHA-3 and the encryption algorithm SHOULD be 1265 128-bit AES-GCM or better. The PRK SHOULD be chosen 1266 by both parties contributing alternate random bytes 1267 until sufficient length exists. After the initial 1268 setup, client-information can be encrypted using the 1269 key generated by the "expand" step of Section 2.3 of 1270 [RFC5869]. The encrypted value SHOULD be hex or 1271 base64 encoded. At the agreed-upon expiration time, a 1272 new key SHOULD be generated and used. New keys SHOULD 1273 be indicated by prefixing the key with a special 1274 character such as exclamation point. In this way, 1275 shorter lifetimes can be used as needed. 1277 * occurrence: there MUST be one and only one instance of this 1278 field. 1280 o s-ip: 1282 * format: ADDRESS 1284 * field value: the IPv4 or IPv6 address of the Surrogate that 1285 served the request (i.e., the "server" address). 1287 * occurrence: there MUST be zero or exactly one instance of this 1288 field. 1290 o s-hostname: 1292 * format: host 1294 * field value: the hostname of the Surrogate that served the 1295 request (i.e., the "server" hostname). 1297 * occurrence: there MUST be zero or exactly one instance of this 1298 field. 1300 o s-port: 1302 * format: 1*DIGIT 1304 * field value: the destination TCP port (i.e., the "server" port) 1305 in the request received by the Surrogate. 1307 * occurrence: there MUST be zero or exactly one instance of this 1308 field. 1310 o cs-method: 1312 * format: NHTABSTRING 1314 * field value: this is the method of the request received by the 1315 Surrogate. In the case of HTTP delivery, this is the HTTP 1316 method in the request. 1318 * occurrence: There MUST be one and only one instance of this 1319 field. 1321 o cs-uri: 1323 * format: NHTABSTRING 1325 * field value: this is the "effective request URI" of the request 1326 received by the Surrogate as specified in [RFC7230]. It 1327 complies with the "http" URI scheme or the "https" URI scheme 1328 as specified in [RFC7230]). Note that cs-uri can be privacy 1329 sensitive. In that case, and where appropriate, u-uri could be 1330 used instead of cs-uri. 1332 * occurrence: there MUST be zero or exactly one instance of this 1333 field. 1335 o u-uri: 1337 * format: NHTABSTRING 1339 * field value: this is a complete URI, derived from the 1340 "effective request URI" ([RFC7230]) of the request received by 1341 the Surrogate (i.e., the cs-uri) but transformed by the entity 1342 generating or transmitting the CDNI Logging Record, in a way 1343 that is agreed upon between the two ends of the CDNI Logging 1344 interface, so the transformed URI is meaningful to the uCDN. 1346 For example, the two ends of the CDNI Logging interface could 1347 agree that the u-uri is constructed from the cs-uri by removing 1348 the part of the hostname that exposes which individual 1349 Surrogate actually performed the delivery. The details of 1350 modification performed to generate the u-uri, as well as the 1351 mechanism to agree on these modifications between the two sides 1352 of the CDNI Logging interface are outside the scope of the 1353 present document. 1355 * occurrence: there MUST be one and only one instance of this 1356 field. 1358 o protocol: 1360 * format: NHTABSTRING 1362 * field value: this is value of the HTTP-Version field as 1363 specified in [RFC7230] of the Request-Line of the request 1364 received by the Surrogate (e.g., "HTTP/1.1"). 1366 * occurrence: there MUST be one and only one instance of this 1367 field. 1369 o sc-status: 1371 * format: 3DIGIT 1373 * field value: this is the Status-Code in the response from the 1374 Surrogate. In the case of HTTP delivery, this is the HTTP 1375 Status-Code in the HTTP response. 1377 * occurrence: There MUST be one and only one instance of this 1378 field. 1380 o sc-total-bytes: 1382 * format: 1*DIGIT 1384 * field value: this is the total number of bytes of the response 1385 sent by the Surrogate in response to the request. In the case 1386 of HTTP delivery, this includes the bytes of the Status-Line, 1387 the bytes of the HTTP headers and the bytes of the message- 1388 body. 1390 * occurrence: There MUST be one and only one instance of this 1391 field. 1393 o sc-entity-bytes: 1395 * format: 1*DIGIT 1397 * field value: this is the number of bytes of the message-body in 1398 the HTTP response sent by the Surrogate in response to the 1399 request. This does not include the bytes of the Status-Line or 1400 the bytes of the HTTP headers. 1402 * occurrence: there MUST be zero or exactly one instance of this 1403 field. 1405 o cs(insert_HTTP_header_name_here): 1407 * format: QSTRING 1409 * field value: the value of the HTTP header (identified by the 1410 insert_HTTP_header_name_here in the CDNI Logging field name) as 1411 it appears in the request processed by the Surrogate, but 1412 prepended by a DQUOTE and appended by a DQUOTE. For example, 1413 when the CDNI Logging field name (FIENAME) listed in the 1414 preceding fields directive is cs(User-Agent), this CDNI Logging 1415 field value contains the value of the User-Agent HTTP header as 1416 received by the Surrogate in the request it processed, but 1417 prepended by a DQUOTE and appended by a DQUOTE. If the HTTP 1418 header as it appeared in the request processed by the Surrogate 1419 contains one or more DQUOTE, each DQUOTE MUST be escaped with 1420 percent encoding. For example, if the HTTP header contains 1421 My_Header"value", then the field value of the 1422 cs(insert_HTTP_header_name_here) is "My_Header%x22value%x22". 1423 The entity transmitting the CDNI Logging File MUST ensure that 1424 the respective insert_HTTP_header_name_here of the 1425 cs(insert_HTTP_header_name_here) listed in the fields directive 1426 comply with HTTP specifications. In particular, this field 1427 name does not include any HTAB, since this would prevent proper 1428 parsing of the fields directive by the entity receiving the 1429 CDNI Logging File. 1431 * occurrence: there MAY be zero, one or any number of instance of 1432 this field. 1434 o sc(insert_HTTP_header_name_here): 1436 * format: QSTRING 1438 * field value: the value of the HTTP header (identified by the 1439 insert_HTTP_header_name_here in the CDNI Logging field name) as 1440 it appears in the response issued by the Surrogate to serve the 1441 request, but prepended by a DQUOTE and appended by a DQUOTE. 1442 If the HTTP header as it appeared in the request processed by 1443 the Surrogate contains one or more DQUOTE, each DQUOTE MUST be 1444 escaped with percent encoding. For example, if the HTTP header 1445 contains My_Header"value", then the field value of the 1446 sc(insert_HTTP_header_name_here) is "My_Header%x22value%x22". 1447 The entity transmitting the CDNI Logging File MUST ensure that 1448 the respective insert_HTTP_header_name_here of the 1449 cs(insert_HTTP_header_name_here) listed in the fields directive 1450 comply with HTTP specifications. In particular, this field 1451 name does not include any HTAB, since this would prevent proper 1452 parsing of the fields directive by the entity receiving the 1453 CDNI Logging File. 1455 * occurrence: there MAY be zero, one or any number of instances 1456 of this field. For a given insert_HTTP_header_name_here, there 1457 MUST be zero or exactly one instance of this field. 1459 o s-ccid: 1461 * format: QSTRING 1463 * field value: this contains the value of the Content Collection 1464 IDentifier (CCID) associated by the uCDN to the content served 1465 by the Surrogate via the CDNI Metadata interface 1466 ([I-D.ietf-cdni-metadata]), prepended by a DQUOTE and appended 1467 by a DQUOTE. If the CCID conveyed in the CDNI Metadata 1468 interface contains one or more DQUOTE, each DQUOTE MUST be 1469 escaped with percent encoding. For example, if the CCID 1470 conveyed in the CDNI Metadata interface is My_CCIDD"value", 1471 then the field value of the s-ccid is "My_CCID%x22value%X22". 1473 * occurrence: there MUST be zero or exactly one instance of this 1474 field. For a given insert_HTTP_header_name_here, there MUST be 1475 zero or exactly one instance of this field. 1477 o s-sid: 1479 * format: QSTRING 1481 * field value: this contains the value of a Session IDentifier 1482 (SID) generated by the dCDN for a specific HTTP session, 1483 prepended by a DQUOTE and appended by a DQUOTE. In particular, 1484 for HTTP Adaptive Streaming (HAS) session, the Session 1485 IDentifier value is included in the Logging record for every 1486 content chunk delivery of that session in view of facilitating 1487 the later correlation of all the per content chunk log records 1488 of a given HAS session. See section 3.4.2.2. of [RFC6983] for 1489 more discussion on the concept of Session IDentifier in the 1490 context of HAS. If the SID conveyed contains one or more 1491 DQUOTE, each DQUOTE MUST be escaped with percent encoding. For 1492 example, if the SID is My_SID"value", then the field value of 1493 the s-sid is "My_SID%x22value%x22". 1495 * occurrence: there MUST be zero or exactly one instance of this 1496 field. 1498 o s-cached: 1500 * format: 1DIGIT 1502 * field value: this characterises whether the Surrogate served 1503 the request using content already stored on its local cache or 1504 not. The allowed values are "0" (for miss) and "1" (for hit). 1505 "1" MUST be used when the Surrogate did serve the request using 1506 exclusively content already stored on its local cache. "0" MUST 1507 be used otherwise (including cases where the Surrogate served 1508 the request using some, but not all, content already stored on 1509 its local cache). Note that a "0" only means a cache miss in 1510 the Surrogate and does not provide any information on whether 1511 the content was already stored, or not, in another device of 1512 the dCDN, i.e., whether this was a "dCDN hit" or "dCDN miss". 1514 * occurrence: there MUST be zero or exactly one instance of this 1515 field. 1517 The "fields" directive corresponding to a HTTP Request Logging Record 1518 MUST contain all the fields names whose occurrence is specified above 1519 as "There MUST be one and only one instance of this field". The 1520 corresponding fields value MUST be present in every HTTP Request 1521 Logging Record. 1523 The "fields" directive corresponding to a HTTP Request Logging Record 1524 MAY list all the fields value whose occurrence is specified above as 1525 "there MUST be zero or exactly one instance of this field" or "there 1526 MAY be zero, one or any number of instances of this field". The set 1527 of such field names actually listed in the "fields" directive is 1528 selected by the CDN generating the CDNI Logging File based on 1529 agreements between the interconnected CDNs established through 1530 mechanisms outside the scope of this specification (e.g., contractual 1531 agreements). When such a field name is not listed in the "fields" 1532 directive, the corresponding field value MUST NOT be included in the 1533 Logging Record. When such a field name is listed in the "fields" 1534 directive, the corresponding field value MUST be included in the 1535 Logging Record; if the value for the field is not available, this 1536 MUST be conveyed via a dash character ("-"). 1538 The fields names listed in the "fields" directive MAY be listed in 1539 the order in which they are listed in Section 3.4.1 or MAY be listed 1540 in any other order. 1542 Logging some specific fields from HTTP requests and responses can 1543 introduce serious security and privacy risks. For example, cookies 1544 will often contain (months) long lived token values that can be used 1545 to log into a service as the relevant user. Similar values may be 1546 included in other header fields or within URLs or elsewhere in HTTP 1547 requests and responses. Centralising such values in a CDNI Logging 1548 File can therefore represent a significant increase in risk both for 1549 the user and the web service provider, but also for the CDNs 1550 involved. Implementations ought therefore to attempt to lower the 1551 probability of such bad outcomes e.g. by only allowing a configured 1552 set of headers to be added to CDNI Logging Records, or by not 1553 supporting wildcard selection of HTTP request/response fields to add. 1554 Such mechanisms can reduce the probability that security (or privacy) 1555 sensitive values are centralised in CDNI Logging Files. Also, when 1556 agreeing on which HTTP request/response fields are to be provided in 1557 CDNI Logging Files, the uCDN and dCDN administrators ought to 1558 consider these risks. 1560 A dCDN-side implementation of the CDNI Logging interface MUST 1561 implement all the following Logging fields in a CDNI Logging Record 1562 of record-type "cdni_http_request_v1", and MUST support the ability 1563 to include valid values for each of them: 1565 o date 1567 o time 1569 o time-taken 1571 o c-groupid 1573 o s-ip 1575 o s-hostname 1577 o s-port 1579 o cs-method 1581 o cs-uri 1583 o u-uri 1585 o protocol 1586 o sc-status 1588 o sc-total-bytes 1590 o sc-entity-bytes 1592 o cs(insert_HTTP_header_name_here) 1594 o sc(insert_HTTP_header_name_here) 1596 o s-cached 1598 A dCDN-side implementation of the CDNI Logging interface MAY support 1599 the following Logging fields in a CDNI Logging Record of record-type 1600 "cdni_http_request_v1": 1602 o s-ccid 1604 o s-sid 1606 If a dCDN-side implementation of the CDNI Logging interface supports 1607 these fields, it MUST support the ability to include valid values for 1608 them. 1610 An uCDN-side implementation of the CDNI Logging interface MUST be 1611 able to accept CDNI Logging Files with CDNI Logging Records of 1612 record-type "cdni_http_request_v1" containing any CDNI Logging Field 1613 defined in Section 3.4.1 as long as the CDNI Logging Record and the 1614 CDNI Logging File are compliant with the present document. 1616 In case an uCDN-side implementation of the CDNI Logging interface 1617 receives a CDNI Logging File with HTTP Request Logging Records that 1618 do not contain field values for exactly the set of field names 1619 actually listed in the preceding "fields" directive, the 1620 implementation MUST reject those HTTP Request Logging Records, and 1621 MUST accept the other HTTP Request Logging Records. 1623 To ensure that the logging file is correct, the text MUST be 1624 sanitized before being logged. Null, bare CR, bare LF and HTAB have 1625 to be removed by escaping them through percent encoding to avoid 1626 confusion with the logging record separators. 1628 3.5. CDNI Logging File Extension 1630 The CDNI Logging File contains blocks of directives and blocks of 1631 corresponding records. The supported set of directives is defined 1632 relative to the CDNI Logging File Format version. The complete set 1633 of directives for version "CDNI/1.0" are defined in Section 3.3. The 1634 directive list is not expected to require much extension, but when it 1635 does, the new directive MUST be defined and registered in the "CDNI 1636 Logging Directive Names" registry, as described in Figure 8, and a 1637 new version MUST be defined and registered in the "CDNI Logging File 1638 version" registry, as described in Section 6.2. For example, adding 1639 a new CDNI Logging Directive, e.g., "foo", to the set of directives 1640 defined for "CDNI/1.0" in Section 3.3, would require registering both 1641 the new CDNI Logging Directive "foo" and a new CDNI Logging File 1642 version, e.g., "CDNI/2.0", which includes all of the existing CDNI 1643 Logging Directives of "CDNI/1.0" plus "foo". 1645 It is expected that as new logging requirements arise, the list of 1646 fields to log will change and expand. When adding new fields, the 1647 new fields MUST be defined and registered in the "CDNI Logging Field 1648 Names" registry, as described in Section 6.4, and a new record-type 1649 MUST be defined and registered in the "CDNI Logging record-types" 1650 registry, as described in Section 6.3. For example, adding a new 1651 CDNI Logging Field, e.g., "c-bar", to the set of fields defined for 1652 "cdni_http_request_v1" in Section 3.4.1, would require registering 1653 both the new CDNI Logging Field "c-bar" and a new CDNI record-type, 1654 e.g., "cdni_http_request_v2", which includes all of the existing CDNI 1655 Logging Fields of "cdni_http_request_v1" plus "c-bar". 1657 3.6. CDNI Logging File Example 1659 Let us consider the upstream CDN and the downstream CDN labelled uCDN 1660 and dCDN-1 in Figure 1. When dCDN-1 acts as a downstream CDN for 1661 uCDN and performs content delivery on behalf of uCDN, dCDN-1 will 1662 include the CDNI Logging Records corresponding to the content 1663 deliveries performed on behalf of uCDN in the CDNI Logging Files for 1664 uCDN. An example CDNI Logging File communicated by dCDN-1 to uCDN is 1665 shown below in Figure 4. 1667 #version:cdni/1.0 1669 #UUID:urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6 1671 #claimed-origin:cdni-logging-entity.dcdn-1.example.com 1673 #record-type:cdni_http_request_v1 1675 #fields:datetimetime-takenc-groupid 1676 cs-methodu-uriprotocol 1677 sc-statussc-total-bytescs(User-Agent) 1678 cs(Referer)s-cached 1680 2013-05-1700:38:06.8259.058US/TN/MEM/38138 1681 GET 1682 http://cdni-ucdn.dcdn-1.example.com/video/movie100.mp4 1683 HTTP/1.12006729891"Mozilla/5.0 1684 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1685 Gecko) Chrome/5.0.375.127 Safari/533.4" 1686 "host1.example.com"1 1688 2013-05-1700:39:09.14515.32FR/PACA/NCE/06100 1689 GET 1690 http://cdni-ucdn.dcdn-1.example.com/video/movie118.mp4 1691 HTTP/1.120015799210"Mozilla/5.0 1692 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1693 Gecko) Chrome/5.0.375.127 Safari/533.4" 1694 "host1.example.com"1 1696 2013-05-1700:42:53.43752.879US/TN/MEM/38138 1697 GET 1698 http://cdni-ucdn.dcdn-1.example.com/video/picture11.mp4 1699 HTTP/1.020097234724"Mozilla/5.0 1700 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1701 Gecko) Chrome/5.0.375.127 Safari/533.4" 1702 "host5.example.com"0 1704 #SHA256-hash: 64-hexadecimal-digit hash value 1706 Figure 4: CDNI Logging File Example 1708 If uCDN establishes by some means (e.g., via TLS authentication when 1709 pulling the CDNI Logging File) the identity of the entity from which 1710 it pulled the CDNI Logging File, uCDN can add to the CDNI Logging an 1711 established-origin directive as illustrated below: 1713 #established-origin:cdni-logging-entity.dcdn- 1714 1.example.com 1715 As illustrated in Figure 2, uCDN will then ingest the corresponding 1716 CDNI Logging Records into its Collection process, alongside the 1717 Logging Records generated locally by the uCDN itself. This allows 1718 uCDN to aggregate Logging Records for deliveries performed by itself 1719 (through Records generated locally) as well as for deliveries 1720 performed by its downstream CDN(s). This aggregate information can 1721 then be used (after Filtering and Rectification, as illustrated in 1722 Figure 2) by Log Consuming Applications that take into account 1723 deliveries performed by uCDN as well as by all of its downstream 1724 CDNs. 1726 We observe that the time between 1728 1. when a delivery is completed in dCDN and 1730 2. when the corresponding Logging Record is ingested by the 1731 Collection process in uCDN 1733 depends on a number of parameters such as the Logging Period agreed 1734 to by uCDN and dCDN, how much time uCDN waits before pulling the CDNI 1735 Logging File once it is advertised in the CDNI Logging Feed, and the 1736 time to complete the pull of the CDNI Logging File. Therefore, if we 1737 consider the set of Logging Records aggregated by the Collection 1738 process in uCDN in a given time interval, there could be a permanent 1739 significant timing difference between the CDNI Logging Records 1740 received from the dCDN and the Logging Records generated locally. 1741 For example, in a given time interval, the Collection process in uCDN 1742 may be aggregating Logging Records generated locally by uCDN for 1743 deliveries performed in the last hour and CDNI Logging Records 1744 generated in the dCDN for deliveries in the hour before last. 1746 3.7. Cascaded CDNI Logging Files Example 1748 Let us consider the cascaded CDN scenario of uCDN, dCDN-2 and dCDN-3 1749 as depicted in Figure 1. After completion of a delivery by dCDN-3 on 1750 behalf of dCDN-2, dCDN-3 will include a corresponding Logging Record 1751 in a CDNI Logging File that will be pulled by dCDN-2 and that is 1752 illustrated below in Figure 5. In practice, a CDNI Logging File is 1753 likely to contain a very high number of CDNI Logging Records. 1754 However, for readability, the example in Figure 5 contains a single 1755 CDNI Logging Record. 1757 #version:CDNI/1.0 1759 #UUID:urn:uuid:65718ef-0123-9876-adce4321bcde 1761 #claimed-origin:cdni-logging-entity.dcdn-3.example.com 1763 #record-type:cdni_http_request_v1 1765 #fields:datetimetime-takenc-groupid 1766 cs-methodu-uriprotocol 1767 sc-statussc-total-bytescs(User-Agent) 1768 cs(Referer)s-cached 1770 2013-05-1700:39:09.11914.07US/CA/SFO/94114 1771 GET 1772 http://cdni-dcdn-2.dcdn-3.example.com/video/movie118.mp4 1773 HTTP/1.120015799210"Mozilla/5.0 1774 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1775 Gecko) Chrome/5.0.375.127 Safari /533.4" 1776 "host1.example.com"1 1778 #SHA256-hash: 64-hexadecimal-digit hash value 1780 Figure 5: Cascaded CDNI Logging File Example (dCDN-3 to dCDN-2) 1782 If dCDN-2 establishes by some means (e.g., via TLS authentication 1783 when pulling the CDNI Logging File) the identity of the entity from 1784 which it pulled the CDNI Logging File, dCDN-2 can add to the CDNI 1785 Logging an established-origin directive as illustrated below: 1787 #established-origin:cdni-logging-entity.dcdn- 1788 3.example.com 1790 dCDN-2 (behaving as an upstream CDN from the viewpoint of dCDN-3) 1791 will then ingest the CDNI Logging Record for the considered dCDN-3 1792 delivery into its Collection process (as illustrated in Figure 2). 1793 This Logging Record may be aggregated with Logging Records generated 1794 locally by dCDN-2 for deliveries performed by dCDN-2 itself. Say, 1795 for illustration, that the content delivery performed by dCDN-3 on 1796 behalf of dCDN-2 had actually been redirected to dCDN-2 by uCDN, and 1797 say that another content delivery has just been redirected by uCDN to 1798 dCDN-2 and that dCDN-2 elected to perform the corresponding delivery 1799 itself. Then after Filtering and Rectification (as illustrated in 1800 Figure 2), dCDN-2 will include the two Logging Records corresponding 1801 respectively to the delivery performed by dCDN-3 and the delivery 1802 performed by dCDN-2, in the next CDNI Logging File that will be 1803 communicated to uCDN. An example of such CDNI Logging File is 1804 illustrated below in Figure 6. 1806 #version:CDNI/1.0 1808 #UUID:urn:uuid:1234567-8fedc-abab-0987654321ff 1810 #claimed-origin:cdni-logging-entity.dcdn-2.example.com 1812 #record-type:cdni_http_request_v1 1814 #fields:datetimetime-takenc-groupid 1815 cs-methodu-uriprotocol 1816 sc-statussc-total-bytescs(User-Agent) 1817 cs(Referer)s-cached 1819 2013-05-1700:39:09.11914.07US/CA/SFO/94114 1820 GET 1821 http://cdni-ucdn.dcdn-2.example.com/video/movie118.mp4 1822 HTTP/1.120015799210"Mozilla/5.0 1823 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1824 Gecko) Chrome/5.0.375.127 Safari /533.4" 1825 "host1.example.com"1 1827 2013-05-1701:42:53.43752.879FR/IDF/PAR/75001 1828 GET 1829 http://cdni-ucdn.dcdn-2.example.com/video/picture11.mp4 1830 HTTP/1.020097234724"Mozilla/5.0 1831 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1832 Gecko) Chrome/5.0.375.127 Safari /533.4" 1833 "host5.example.com"0 1835 #SHA256-hash: 64-hexadecimal-digit hash value 1837 Figure 6: Cascaded CDNI Logging File Example (dCDN-2 to uCDN) 1839 If uCDN establishes by some means (e.g., via TLS authentication when 1840 pulling the CDNI Logging File) the identity of the entity from which 1841 it pulled the CDNI Logging File, uCDN can add to the CDNI Logging an 1842 established-origin directive as illustrated below: 1844 #established-origin:cdni-logging-entity.dcdn- 1845 2.example.com 1847 In the example of Figure 6, we observe that: 1849 o the first Logging Record corresponds to the Logging Record 1850 communicated earlier to dCDN-2 by dCDN-3, which corresponds to a 1851 delivery redirected by uCDN to dCDN-2 and then redirected by 1852 dCDN-2 to dCDN-3. The fields values in this Logging Record are 1853 copied from the corresponding CDNI Logging REcord communicated to 1854 dCDN2 by dCDN-3, with the exception of the u-uri that now reflects 1855 the URI convention between uCDN and dCDN-2 and that presents the 1856 delivery to uCDN as if it was performed by dCDN-2 itself. This 1857 reflects the fact that dCDN-2 had taken the full responsibility of 1858 the corresponding delivery (even if in this case, dCDN-2 elected 1859 to redirect the delivery to dCDN-3 so it is actually performed by 1860 dCDN-3 on behalf of dCDN-2). 1862 o the second Logging Record corresponds to a delivery redirected by 1863 uCDN to dCDN-2 and performed by dCDN-2 itself. The time of the 1864 delivery in this Logging Record may be significantly more recent 1865 than the first Logging Record since it was generated locally while 1866 the first Logging Record was generated by dCDN-3 and had to be 1867 advertised , and then pulled and then ingested into the dCDN-2 1868 Collection process, before being aggregated with the second 1869 Logging Record. 1871 4. Protocol for Exchange of CDNI Logging File After Full Collection 1873 This section specifies a protocol for the exchange of CDNI Logging 1874 Files as specified in Section 3 after the CDNI Logging File is fully 1875 collected by the dCDN. 1877 This protocol comprises: 1879 o a CDNI Logging feed, allowing the dCDN to notify the uCDN about 1880 the CDNI Logging Files that can be retrieved by that uCDN from the 1881 dCDN, as well as all the information necessary for retrieving each 1882 of these CDNI Logging Files. The CDNI Logging feed is specified 1883 in Section 4.1. 1885 o a CDNI Logging File pull mechanism, allowing the uCDN to obtain 1886 from the dCDN a given CDNI Logging File at the uCDN's convenience. 1887 The CDNI Logging File pull mechanisms is specified in Section 4.2. 1889 An implementation of the CDNI Logging interface on the dCDN side (the 1890 entity generating the CDNI Logging file) MUST support the server side 1891 of the CDNI Logging feed (as specified in Section 4.1) and the server 1892 side of the CDNI Logging pull mechanism (as specified in 1893 Section 4.2). 1895 An implementation of the CDNI Logging interface on the uCDN side (the 1896 entity consuming the CDNI Logging file) MUST support the client side 1897 of the CDNI Logging feed (as specified in Section 4.1) and the client 1898 side of the CDNI Logging pull mechanism (as specified in 1899 Section 4.2). 1901 4.1. CDNI Logging Feed 1903 The server-side implementation of the CDNI Logging feed MUST produce 1904 an Atom feed [RFC4287]. This feed is used to advertise log files 1905 that are available for the client-side to retrieve using the CDNI 1906 Logging pull mechanism. 1908 4.1.1. Atom Formatting 1910 A CDNI Logging feed MUST be structured as an Archived feed, as 1911 defined in [RFC5005], and MUST be formatted in Atom [RFC4287]. This 1912 means it consists of a subscription document that is regularly 1913 updated as new CDNI Logging Files become available, and information 1914 about older CDNI Logging files is moved into archive documents. Once 1915 created, archive documents are never modified. 1917 Each CDNI Logging File listed in an Atom feed MUST be described in an 1918 atom:entry container element. 1920 The atom:entry MUST contain an atom:content element whose "src" 1921 attribute is a link to the CDNI Logging File and whose "type" 1922 attribute is the MIME Media Type indicating that the entry is a CDNI 1923 logging file. This MIME Media Type is defined as "application/cdni" 1924 (See [RFC7736]) with the Payload Type (ptype) parameter set to 1925 "logging-file". 1927 For compatibility with some Atom feed readers the atom:entry MAY also 1928 contain an atom:link entry whose "href" attribute is a link to the 1929 CDNI Logging File and whose "type" attribute is the MIME Media Type 1930 indicating that the entry is a CDNI Logging File using the 1931 "application/cdni" MIME Media Type with the Payload Type (ptype) 1932 parameter set to "logging-file"(See [RFC7736]). 1934 The URI used in the atom:id of the atom:entry MUST contain the UUID 1935 of the CDNI Logging File. 1937 The atom:updated in the atom:entry MUST indicate the time at which 1938 the CDNI Logging File was last updated. 1940 4.1.2. Updates to Log Files and the Feed 1942 CDNI Logging Files MUST NOT be modified by the dCDN once published in 1943 the CDNI Logging feed. 1945 The frequency with which the subscription feed is updated, the period 1946 of time covered by each CDNI Logging File or each archive document, 1947 and timeliness of publishing of CDNI Logging Files are outside the 1948 scope of the present document and are expected to be agreed upon by 1949 uCDN and dCDN via other means (e.g., human agreement). 1951 The server-side implementation MUST be able to set, and SHOULD set, 1952 HTTP cache control headers on the subscription feed to indicate the 1953 frequency at which the client-side is to poll for updates. 1955 The client-side MAY use HTTP cache control headers (set by the 1956 server-side) on the subscription feed to determine the frequency at 1957 which to poll for updates. The client-side MAY instead, or in 1958 addition, use other information to determine when to poll for updates 1959 (e.g., a polling frequency that may have been negotiated between the 1960 uCDN and dCDN by mechanisms outside the scope of the present document 1961 and that is to override the indications provided in the HTTP cache 1962 control headers). 1964 The potential retention limits (e.g., sliding time window) within 1965 which the dCDN is to retain and be ready to serve an archive document 1966 is outside the scope of the present document and is expected to be 1967 agreed upon by uCDN and dCDN via other means (e.g., human agreement). 1968 The server-side implementation MUST retain, and be ready to serve, 1969 any archive document within the agreed retention limits. Outside 1970 these agreed limits, the server-side implementation MAY indicate its 1971 inability to serve (e.g., with HTTP status code 404) an archive 1972 document or MAY refuse to serve it (e.g., with HTTP status code 403 1973 or 410). 1975 4.1.3. Redundant Feeds 1977 The server-side implementation MAY present more than one CDNI Logging 1978 feed for redundancy. Each CDNI Logging File MAY be published in more 1979 than one feed. 1981 A client-side implementation MAY support such redundant CDNI Logging 1982 feeds. If it supports redundant CDNI Logging feed, the client-side 1983 can use the UUID of the CDNI Logging File, presented in the atom:id 1984 element of the Atom feed, to avoid unnecessarily pulling and storing 1985 a given CDNI Logging File more than once. 1987 4.1.4. Example CDNI Logging Feed 1989 Figure 7 illustrates an example of the subscription document of a 1990 CDNI Logging feed. 1992 1993 1994 CDNI Logging Feed 1995 2013-03-23T14:46:11Z 1996 urn:uuid:663ae677-40fb-e99a-049d-c5642916b8ce 1997 1999 2001 2003 CDNI Log Feed 2004 Generator 2005 dcdn.example 2006 2007 CDNI Logging File for uCDN at 2008 2013-03-23 14:15:00 2009 urn:uuid:12345678-1234-abcd-00aa-01234567abcd 2010 2013-03-23T14:15:00Z 2011 2015 CDNI Logging File for uCDN at 2016 2013-03-23 14:15:00 2017 2018 2019 CDNI Logging File for uCDN at 2020 2013-03-23 14:30:00 2021 urn:uuid:87654321-4321-dcba-aa00-dcba7654321 2022 2013-03-23T14:30:00Z 2023 2027 CDNI Logging File for uCDN at 2028 2013-03-23 14:30:00 2029 2030 ... 2031 2032 ... 2033 2034 2036 Figure 7: Example subscription document of a CDNI Logging Feed 2038 4.2. CDNI Logging File Pull 2040 A client-side implementation of the CDNI Logging interface MAY pull, 2041 at its convenience, a CDNI Logging File that is published by the 2042 server-side in the CDNI Logging Feed (in the subscription document or 2043 an archive document). To do so, the client-side: 2045 o MUST implement HTTP/1.1 ([RFC7230],[RFC7231], [RFC7232], 2046 [RFC7233], [RFC7234], [RFC7235]), MAY also support other HTTP 2047 versions (e.g., HTTP/2 [RFC7540]) and MAY negotiate which HTTP 2048 version is actually used. This allows operators and implementers 2049 to choose to use later versions of HTTP to take advantage of new 2050 features, while still ensuring interoperability with systems that 2051 only support HTTP/1.1. 2053 o MUST use the URI that was associated to the CDNI Logging File 2054 (within the "src" attribute of the corresponding atom:content 2055 element) in the CDNI Logging Feed; 2057 o MUST support exchange of CDNI Logging Files with no content 2058 encoding applied to the representation; 2060 o MUST support exchange of CDNI Logging Files with "gzip" content 2061 encoding (as defined in [RFC7230]) applied to the representation. 2063 Note that a client-side implementation of the CDNI Logging interface 2064 MAY pull a CDNI Logging File that it has already pulled. 2066 The server-side implementation MUST respond to valid pull request by 2067 a client-side implementation for a CDNI Logging File published by the 2068 server-side in the CDNI Logging Feed (in the subscription document or 2069 an archive document). The server-side implementation: 2071 o MUST implement HTTP/1.1 to handle the client-side request and MAY 2072 also support other HTTP versions (e.g., HTTP/2); 2074 o MUST include the CDNI Logging File identified by the request URI 2075 inside the body of the HTTP response; 2077 o MUST support exchange of CDNI Logging Files with no content 2078 encoding applied to the representation; 2080 o MUST support exchange of CDNI Logging Files with "gzip" content 2081 encoding (as defined in [RFC7231]) applied to the representation. 2083 Content negotiation approaches defined in [RFC7231] (e.g., using 2084 Accept-Encoding request-header field or Content-Encoding entity- 2085 header field) MAY be used by the client-side and server-side 2086 implementations to establish the content-coding to be used for a 2087 particular exchange of a CDNI Logging File. 2089 Applying compression content encoding (such as "gzip") is expected to 2090 mitigate the impact of exchanging the large volumes of logging 2091 information expected across CDNs. This is expected to be 2092 particularly useful in the presence of HTTP Adaptive Streaming (HAS) 2093 which, as per the present version of the document, will result in a 2094 separate CDNI Log Record for each HAS segment delivery in the CDNI 2095 Logging File. 2097 The potential retention limits (e.g., sliding time window, maximum 2098 aggregate file storage quotas) within which the dCDN is to retain and 2099 be ready to serve a CDNI Logging File previously advertised in the 2100 CDNI Logging Feed is outside the scope of the present document and is 2101 expected to be agreed upon by uCDN and dCDN via other means (e.g., 2102 human agreement). The server-side implementation MUST retain, and be 2103 ready to serve, any CDNI Logging File within the agreed retention 2104 limits. Outside these agreed limits, the server-side implementation 2105 MAY indicate its inability to serve (e.g., with HTTP status code 404) 2106 a CDNI Logging File or MAY refuse to serve it (e.g., with HTTP status 2107 code 403 or 410). 2109 5. Protocol for Exchange of CDNI Logging File During Collection 2111 We note that, in addition to the CDNI Logging File exchange protocol 2112 specified in Section 4, implementations of the CDNI Logging interface 2113 may also support other mechanisms to exchange CDNI Logging Files. In 2114 particular, such mechanisms might allow the exchange of the CDNI 2115 Logging File to start before the file is fully collected. This can 2116 allow CDNI Logging Records to be communicated by the dCDN to the uCDN 2117 as they are gathered by the dCDN without having to wait until all the 2118 CDNI Logging Records of the same logging period are collected in the 2119 corresponding CDNI Logging File. This approach is commonly referred 2120 to as "tailing" of the file. 2122 Such an approach could be used, for example, to exchange logging 2123 information with a significantly reduced time-lag (e.g., sub-minute 2124 or sub-second) between when the event occurred in the dCDN and when 2125 the corresponding CDNI Logging Record is made available to the uCDN. 2126 This can satisfy log-consuming applications requiring extremely fresh 2127 logging information such as near-real-time content delivery 2128 monitoring. Such mechanisms are for further study and outside the 2129 scope of this document. 2131 6. IANA Considerations 2133 When IANA allocates new extensions to CDNI Logging Directive Names 2134 Registry, CDNI Logging File version Registry, CDNI Logging record- 2135 type Registry or CDNI Logging fields Registry, IANA MUST take into 2136 account that the directive names are case-insensitive as per the 2137 basic ABNF([RFC5234]). 2139 6.1. CDNI Logging Directive Names Registry 2141 The IANA is requested to create a new "CDNI Logging Directive Names" 2142 subregistry under the "Content Delivery Networks Interconnection 2143 (CDNI) Parameters" registry. 2145 The initial contents of the CDNI Logging Directives registry comprise 2146 the names of the directives specified in Section 3.3 of the present 2147 document, and are as follows: 2149 +------------------------------+-----------+ 2150 | Directive Name | Reference | 2151 +------------------------------+-----------+ 2152 | version | RFC xxxx | 2153 | UUID | RFC xxxx | 2154 | claimed-origin | RFC xxxx | 2155 | established-origin | RFC xxxx | 2156 | remark | RFC xxxx | 2157 | record-type | RFC xxxx | 2158 | fields | RFC xxxx | 2159 | SHA256-hash | RFC xxxx | 2160 +------------------------------+-----------+ 2162 Figure 8 2164 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2165 the present document] 2167 Within the registry, names are to be allocated by IANA according to 2168 the "Specification Required" policy specified in [RFC5226]. 2169 Directive names are to be allocated by IANA with a format of 2170 NAMEFORMAT (see Section 3.1). All directive names and field names 2171 defined in the logging file are case-insensitive as per the basic 2172 ABNF([RFC5234]). 2174 Each specification that defines a new CDNI Logging directive needs to 2175 contain a description for the new directive with the same set of 2176 information as provided in Section 3.3 (i.e., format, directive value 2177 and occurrence). 2179 6.2. CDNI Logging File version Registry 2181 The IANA is requested to create a new "CDNI Logging File version" 2182 subregistry under the "Content Delivery Networks Interconnection 2183 (CDNI) Parameters" registry. 2185 The initial contents of the CDNI Logging Logging File version 2186 registry comprise the value "CDNI/1.0" specified in Section 3.3 of 2187 the present document, and are as follows: 2189 +-----------------+-----------+----------------------------------+ 2190 | version | Reference | Description | 2191 +-----------------+-----------+----------------------------------+ 2192 | cdni/1.0 | RFC xxxx | CDNI Logging File version 1.0 | 2193 | | | as specified in RFC xxxx | 2194 +-----------------+-----------+----------------------------------+ 2196 Figure 9 2198 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2199 the present document] 2201 Within the registry, version values are to be allocated by IANA 2202 according to the "Specification Required" policy specified in 2203 [RFC5226]. Version values are to be allocated by IANA with a format 2204 of NAMEFORMAT (see Section 3.1). 2206 6.3. CDNI Logging record-types Registry 2208 The IANA is requested to create a new "CDNI Logging record-types" 2209 subregistry under the "Content Delivery Networks Interconnection 2210 (CDNI) Parameters" registry. 2212 The initial contents of the CDNI Logging record-types registry 2213 comprise the names of the CDNI Logging Record types specified in 2214 Section 3.4 of the present document, and are as follows: 2216 +----------------------+-----------+---------------------------------+ 2217 | record-types | Reference | Description | 2218 +----------------------+-----------+---------------------------------+ 2219 | cdni_http_request_v1 | RFC xxxx | CDNI Logging Record version 1 | 2220 | | | for content delivery using HTTP | 2221 +----------------------+-----------+---------------------------------+ 2223 Figure 10 2225 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2226 the present document] 2227 Within the registry, record-types are to be allocated by IANA 2228 according to the "Specification Required" policy specified in 2229 [RFC5226]. record-types are to be allocated by IANA with a format of 2230 NAMEFORMAT (see Section 3.1). 2232 Each specification that defines a new record-type needs to contain a 2233 description for the new record-type with the same set of information 2234 as provided in Section 3.4.1. This includes: 2236 o a list of all the CDNI Logging fields that can appear in a CDNI 2237 Logging Record of the new record-type 2239 o for all these fields: a specification of the occurrence for each 2240 Field in the new record-type 2242 o for every newly defined Field, i.e., for every Field that results 2243 in a registration in the CDNI Logging Field Names Registry 2244 (Section 6.4): a specification of the field name, format and field 2245 value. 2247 6.4. CDNI Logging Field Names Registry 2249 The IANA is requested to create a new "CDNI Logging Field Names" 2250 subregistry under the "Content Delivery Networks Interconnection 2251 (CDNI) Parameters" registry. 2253 This registry is intended to be shared across the currently defined 2254 record-type (i.e., cdni_http_request_v1) as well as potential other 2255 CDNI Logging record-types that may be defined in separate 2256 specifications. When a Field from this registry is used by another 2257 CDNI Logging record-type, it is to be used with the exact semantics 2258 and format specified in the document that registered this field and 2259 that is identified in the Reference column of the registry. If 2260 another CDNI Logging record-type requires a Field with semantics that 2261 are not strictly identical, or a format that is not strictly 2262 identical then this new Field is to be registered in the registry 2263 with a different Field name. When a Field from this registry is used 2264 by another CDNI Logging record-type, it can be used with different 2265 occurrence rules. 2267 The initial contents of the CDNI Logging fields Names registry 2268 comprise the names of the CDNI Logging fields specified in 2269 Section 3.4 of the present document, and are as follows: 2271 +------------------------------------------+-----------+ 2272 | Field Name | Reference | 2273 +------------------------------------------+-----------+ 2274 | date | RFC xxxx | 2275 | time | RFC xxxx | 2276 | time-taken | RFC xxxx | 2277 | c-groupid | RFC xxxx | 2278 | s-ip | RFC xxxx | 2279 | s-hostname | RFC xxxx | 2280 | s-port | RFC xxxx | 2281 | cs-method | RFC xxxx | 2282 | cs-uri | RFC xxxx | 2283 | u-uri | RFC xxxx | 2284 | protocol | RFC xxxx | 2285 | sc-status | RFC xxxx | 2286 | sc-total-bytes | RFC xxxx | 2287 | sc-entity-bytes | RFC xxxx | 2288 | cs(insert_HTTP_header_name_here) | RFC xxxx | 2289 | sc(insert_HTTP_header_name_here) | RFC xxxx | 2290 | s-ccid | RFC xxxx | 2291 | s-sid | RFC xxxx | 2292 | s-cached | RFC xxxx | 2293 +------------------------------------------+-----------+ 2295 Figure 11 2297 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2298 the present document] 2300 Within the registry, names are to be allocated by IANA according to 2301 the "Specification Required" policy specified in [RFC5226]. Field 2302 names are to be allocated by IANA with a format of NHTABSTRING (see 2303 Section 3.1). 2305 6.5. CDNI Logging MIME Media Type 2307 The IANA is requested to register the following new Payload Type in 2308 the CDNI Payload Type registry for use with the application/cdni MIME 2309 media type. 2311 [RFC Editor Note: Please replace the references to [RFCthis] below 2312 with this document's RFC number before publication.] 2313 +----------------------+---------------+ 2314 | Payload Type | Specification | 2315 +----------------------+---------------+ 2316 | logging-file | [RFCthis] | 2317 +----------------------+---------------+ 2319 Figure 12: MIME Media Type payload 2321 The purpose of the logging-file payload type is to distinguish 2322 between CDNI Logging Files and other CDNI messages. 2324 Interface: LI 2326 Encoding: see Section 3.2, Section 3.3 and Section 3.4 2328 7. Security Considerations 2330 7.1. Authentication, Authorization, Confidentiality, Integrity 2331 Protection 2333 An implementation of the CDNI Logging interface MUST support TLS 2334 transport of the CDNI Logging feed (Section 4.1) and of the CDNI 2335 Logging File pull (Section 4.2) as per [RFC2818] and [RFC7230]. 2337 The use of TLS for transport of the CDNI Logging feed and CDNI 2338 Logging File pull allows: 2340 o the dCDN and uCDN to authenticate each other 2342 and, once they have mutually authenticated each other, it allows: 2344 o the dCDN and uCDN to authorize each other (to ensure they are 2345 transmitting/receiving CDNI Logging File to/from an authorized 2346 CDN) 2348 o the CDNI Logging information to be transmitted with 2349 confidentiality 2351 o the integrity of the CDNI Logging information to be protected 2352 during the exchange. 2354 In an environment where any such protection is required, mutually 2355 authenticated encrypted transport MUST be used to ensure 2356 confidentiality of the logging information. To that end, TLS MUST be 2357 used (including authentication of the remote end) by the server-side 2358 and the client-side of the CDNI Logging feed, as well as the server- 2359 side and the client-side of the CDNI Logging File pull mechanism. 2361 When TLS is used, the general TLS usage guidance in [RFC7525] MUST be 2362 followed. 2364 The SHA256-hash directive inside the CDNI Logging File provides 2365 additional integrity protection, this time targeting potential 2366 corruption of the CDNI logging information during the CDNI Logging 2367 File generation, storage or exchange. This mechanism does not itself 2368 allow restoration of the corrupted CDNI Logging information, but it 2369 allows detection of such corruption and therefore triggering of 2370 appropriate corrective actions (e.g., discard of corrupted 2371 information, attempt to re-obtain the CDNI Logging information). 2372 Note that the SHA256-hash does not protect against tampering by a 2373 third party, since such a third party could have recomputed and 2374 updated the SHA256-hash after tampering. Protection against third 2375 party tampering can be achieved as discussed above through the use of 2376 TLS. 2378 7.2. Denial of Service 2380 This document does not define specific mechanism to protect against 2381 Denial of Service (DoS) attacks on the Logging Interface. However, 2382 the CDNI Logging feed and CDNI Logging pull endpoints are typically 2383 to be accessed only by a very small number of valid remote endpoints 2384 and therefore can be easily protected against DoS attacks through the 2385 usual conventional DOS protection mechanisms such as firewalling or 2386 use of Virtual Private Networks (VPNs). 2388 Protection of dCDN Surrogates against spoofed delivery requests is 2389 outside the scope of the CDNI Logging interface. 2391 7.3. Privacy 2393 CDNs have the opportunity to collect detailed information about the 2394 downloads performed by End Users. A dCDN is expected to collect such 2395 information into CDNI Logging Files, which are then communicated to 2396 an uCDN. 2398 Having detailed CDNI logging information known by the dCDN in itself 2399 does not represent a particular privacy concern since the dCDN is 2400 obviously fully aware of all information logged since it generated 2401 the information in the first place. Making detailed CDNI logging 2402 information known to the uCDN does not represent a particular privacy 2403 concern because the uCDN is already exposed at request redirection 2404 time to most of the information that shows up as CDNI logging 2405 information (e.g., enduser IP@, URL, HTTP headers - at least when 2406 HTTP redirection is used between uCDN and dCDN). Transporting 2407 detailed CDNI logging information over the HTTP based CDNI Logging 2408 Interface does not represent a particular privacy concern because it 2409 is protected by usual IETF privacy-protection mechanism (e.g.,TLS). 2411 However, one privacy concern arises from the fact that large volumes 2412 of detailed information about content delivery to users, potentially 2413 traceable back to indvidual users, may be collected in CDNI Logging 2414 files. These CDNI Logging files represent high-value targets, likely 2415 concentrated in a fairly centralised system (although the CDNI 2416 Logging architecture does not mandate a particular level of 2417 centralisation/distribution) and at risk of potential data 2418 exfiltration. Note that the means of such data exfiltration are 2419 beyond the scope of the CDNI Logging interface itself (e.g., 2420 corrupted employee, corrupted logging storage system,...). This 2421 privacy concern calls for some protection. 2423 The collection of large volumes of such information into CDNI Logging 2424 Files introduces potential End Users privacy protection concerns. 2425 Mechanisms to address these concerns are discussed in the definition 2426 of the c-groupid field specified in Section 3.4.1. 2428 The use of mutually authenticated TLS to establish a secure session 2429 for the transport of the CDNI Logging feed and CDNI Logging pull as 2430 discussed in Section 7.1 provides confidentiality while the logging 2431 information is in transit and prevents any party other than the 2432 authorised uCDN to gain access to the logging information. 2434 We also note that the query string portion of the URL that may be 2435 conveyed inside the cs-uri and u-uri fields of CDNI Logging Files, or 2436 the HTTP cookies( [RFC6265]) that may be conveyed as part of the 2437 cs() field of CDNI Logging files, may contain 2438 personnal information or information that can be exploited to derive 2439 personal information. Where this is a concern, the CDNI Logging 2440 interface specification allows the dCDN to not include the cs-uri and 2441 to include a u-uri that removes (or hides) the sensitive part of the 2442 query string and allows the dCDN to not include the cs() fields corresponding to HTTP headers associated with cookies. 2445 8. Acknowledgments 2447 This document borrows from the W3C Extended Log Format [ELF]. 2449 Rob Murray significantly contributed into the text of Section 4.1. 2451 The authors thank Ben Niven-Jenkins, Kevin Ma, David Mandelberg and 2452 Ray van Brandenburg for their ongoing input. 2454 Brian Trammel and Rich Salz made significant contributions into 2455 making this interface privacy-friendly. 2457 Finally, we also thank Sebastien Cubaud, Pawel Grochocki, Christian 2458 Jacquenet, Yannick Le Louedec, Anne Marrec, Emile Stephan, Fabio 2459 Costa, Sara Oueslati, Yvan Massot, Renaud Edel, Joel Favier and the 2460 contributors of the EU FP7 OCEAN project for their input in the early 2461 versions of this document. 2463 9. References 2465 9.1. Normative References 2467 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2468 Requirement Levels", BCP 14, RFC 2119, 2469 DOI 10.17487/RFC2119, March 1997, 2470 . 2472 [RFC3339] Klyne, G. and C. Newman, "Date and Time on the Internet: 2473 Timestamps", RFC 3339, DOI 10.17487/RFC3339, July 2002, 2474 . 2476 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 2477 Resource Identifier (URI): Generic Syntax", STD 66, 2478 RFC 3986, DOI 10.17487/RFC3986, January 2005, 2479 . 2481 [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally 2482 Unique IDentifier (UUID) URN Namespace", RFC 4122, 2483 DOI 10.17487/RFC4122, July 2005, 2484 . 2486 [RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom 2487 Syndication Format", RFC 4287, DOI 10.17487/RFC4287, 2488 December 2005, . 2490 [RFC5005] Nottingham, M., "Feed Paging and Archiving", RFC 5005, 2491 DOI 10.17487/RFC5005, September 2007, 2492 . 2494 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 2495 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 2496 DOI 10.17487/RFC5226, May 2008, 2497 . 2499 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 2500 Specifications: ABNF", STD 68, RFC 5234, 2501 DOI 10.17487/RFC5234, January 2008, 2502 . 2504 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2505 Protocol (HTTP/1.1): Message Syntax and Routing", 2506 RFC 7230, DOI 10.17487/RFC7230, June 2014, 2507 . 2509 [RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2510 Protocol (HTTP/1.1): Semantics and Content", RFC 7231, 2511 DOI 10.17487/RFC7231, June 2014, 2512 . 2514 [RFC7232] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2515 Protocol (HTTP/1.1): Conditional Requests", RFC 7232, 2516 DOI 10.17487/RFC7232, June 2014, 2517 . 2519 [RFC7233] Fielding, R., Ed., Lafon, Y., Ed., and J. Reschke, Ed., 2520 "Hypertext Transfer Protocol (HTTP/1.1): Range Requests", 2521 RFC 7233, DOI 10.17487/RFC7233, June 2014, 2522 . 2524 [RFC7234] Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, 2525 Ed., "Hypertext Transfer Protocol (HTTP/1.1): Caching", 2526 RFC 7234, DOI 10.17487/RFC7234, June 2014, 2527 . 2529 [RFC7235] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2530 Protocol (HTTP/1.1): Authentication", RFC 7235, 2531 DOI 10.17487/RFC7235, June 2014, 2532 . 2534 [RFC7525] Sheffer, Y., Holz, R., and P. Saint-Andre, 2535 "Recommendations for Secure Use of Transport Layer 2536 Security (TLS) and Datagram Transport Layer Security 2537 (DTLS)", BCP 195, RFC 7525, DOI 10.17487/RFC7525, May 2538 2015, . 2540 [RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext 2541 Transfer Protocol Version 2 (HTTP/2)", RFC 7540, 2542 DOI 10.17487/RFC7540, May 2015, 2543 . 2545 9.2. Informative References 2547 [CHAR_SET] 2548 "IANA Character Sets registry", 2549 . 2552 [ELF] Phillip M. Hallam-Baker, and Brian Behlendorf, "Extended 2553 Log File Format, W3C (work in progress), WD-logfile- 2554 960323", . 2556 [I-D.ietf-cdni-metadata] 2557 Niven-Jenkins, B., Murray, R., Caulfield, M., and K. Ma, 2558 "CDN Interconnection Metadata", draft-ietf-cdni- 2559 metadata-12 (work in progress), October 2015. 2561 [I-D.ietf-tls-rfc5246-bis] 2562 Dierks, T. and E. Rescorla, "The Transport Layer Security 2563 (TLS) Protocol Version 1.3", draft-ietf-tls-rfc5246-bis-00 2564 (work in progress), April 2014. 2566 [I-D.snell-atompub-link-extensions] 2567 Snell, J., "Atom Link Extensions", draft-snell-atompub- 2568 link-extensions-09 (work in progress), June 2012. 2570 [RFC1945] Berners-Lee, T., Fielding, R., and H. Frystyk, "Hypertext 2571 Transfer Protocol -- HTTP/1.0", RFC 1945, 2572 DOI 10.17487/RFC1945, May 1996, 2573 . 2575 [RFC2818] Rescorla, E., "HTTP Over TLS", RFC 2818, 2576 DOI 10.17487/RFC2818, May 2000, 2577 . 2579 [RFC5869] Krawczyk, H. and P. Eronen, "HMAC-based Extract-and-Expand 2580 Key Derivation Function (HKDF)", RFC 5869, 2581 DOI 10.17487/RFC5869, May 2010, 2582 . 2584 [RFC6234] Eastlake 3rd, D. and T. Hansen, "US Secure Hash Algorithms 2585 (SHA and SHA-based HMAC and HKDF)", RFC 6234, 2586 DOI 10.17487/RFC6234, May 2011, 2587 . 2589 [RFC6265] Barth, A., "HTTP State Management Mechanism", RFC 6265, 2590 DOI 10.17487/RFC6265, April 2011, 2591 . 2593 [RFC6707] Niven-Jenkins, B., Le Faucheur, F., and N. Bitar, "Content 2594 Distribution Network Interconnection (CDNI) Problem 2595 Statement", RFC 6707, DOI 10.17487/RFC6707, September 2596 2012, . 2598 [RFC6770] Bertrand, G., Ed., Stephan, E., Burbridge, T., Eardley, 2599 P., Ma, K., and G. Watson, "Use Cases for Content Delivery 2600 Network Interconnection", RFC 6770, DOI 10.17487/RFC6770, 2601 November 2012, . 2603 [RFC6983] van Brandenburg, R., van Deventer, O., Le Faucheur, F., 2604 and K. Leung, "Models for HTTP-Adaptive-Streaming-Aware 2605 Content Distribution Network Interconnection (CDNI)", 2606 RFC 6983, DOI 10.17487/RFC6983, July 2013, 2607 . 2609 [RFC7336] Peterson, L., Davie, B., and R. van Brandenburg, Ed., 2610 "Framework for Content Distribution Network 2611 Interconnection (CDNI)", RFC 7336, DOI 10.17487/RFC7336, 2612 August 2014, . 2614 [RFC7337] Leung, K., Ed. and Y. Lee, Ed., "Content Distribution 2615 Network Interconnection (CDNI) Requirements", RFC 7337, 2616 DOI 10.17487/RFC7337, August 2014, 2617 . 2619 [RFC7736] Ma, K., "Content Delivery Network Interconnection (CDNI) 2620 Media Type Registration", RFC 7736, DOI 10.17487/RFC7736, 2621 December 2015, . 2623 Authors' Addresses 2625 Francois Le Faucheur (editor) 2626 Cisco Systems 2627 E.Space Park - Batiment D 2628 6254 Allee des Ormes - BP 1200 2629 Mougins cedex 06254 2630 FR 2632 Phone: +33 4 97 23 26 19 2633 Email: flefauch@cisco.com 2635 Gilles Bertrand (editor) 2636 Orange 2637 38-40 rue du General Leclerc 2638 Issy les Moulineaux 92130 2639 FR 2641 Phone: +33 1 45 29 89 46 2642 Email: gilles.bertrand@orange.com 2643 Iuniana Oprescu (editor) 2644 FR 2646 Email: iuniana.oprescu@gmail.com 2648 Roy Peterkofsky 2649 Google Inc. 2650 345 Spear St, 4th Floor 2651 San Francisco CA 94105 2652 USA 2654 Email: peterkofsky@google.com