idnits 2.17.1 draft-ietf-cdni-logging-24.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (April 1, 2016) is 2939 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFCthis' is mentioned on line 2323, but not defined ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) ** Obsolete normative reference: RFC 7231 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7232 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7233 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7234 (Obsoleted by RFC 9111) ** Obsolete normative reference: RFC 7235 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7525 (Obsoleted by RFC 9325) ** Obsolete normative reference: RFC 7540 (Obsoleted by RFC 9113) == Outdated reference: A later version (-21) exists of draft-ietf-cdni-metadata-13 -- Obsolete informational reference (is this intentional?): RFC 2818 (Obsoleted by RFC 9110) Summary: 9 errors (**), 0 flaws (~~), 3 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force F. Le Faucheur, Ed. 3 Internet-Draft Cisco Systems 4 Intended status: Standards Track G. Bertrand, Ed. 5 Expires: October 3, 2016 Orange 6 I. Oprescu, Ed. 8 R. Peterkofsky 9 Google Inc. 10 April 1, 2016 12 CDNI Logging Interface 13 draft-ietf-cdni-logging-24 15 Abstract 17 This memo specifies the Logging interface between a downstream CDN 18 (dCDN) and an upstream CDN (uCDN) that are interconnected as per the 19 CDN Interconnection (CDNI) framework. First, it describes a 20 reference model for CDNI logging. Then, it specifies the CDNI 21 Logging File format and the actual protocol for exchange of CDNI 22 Logging Files. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at http://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on October 3, 2016. 41 Copyright Notice 43 Copyright (c) 2016 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (http://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 59 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 60 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 5 61 2. CDNI Logging Reference Model . . . . . . . . . . . . . . . . 5 62 2.1. CDNI Logging interactions . . . . . . . . . . . . . . . . 5 63 2.2. Overall Logging Chain . . . . . . . . . . . . . . . . . . 8 64 2.2.1. Logging Generation and During-Generation Aggregation 9 65 2.2.2. Logging Collection . . . . . . . . . . . . . . . . . 10 66 2.2.3. Logging Filtering . . . . . . . . . . . . . . . . . . 10 67 2.2.4. Logging Rectification and Post-Generation Aggregation 11 68 2.2.5. Log-Consuming Applications . . . . . . . . . . . . . 12 69 2.2.5.1. Maintenance/Debugging . . . . . . . . . . . . . . 12 70 2.2.5.2. Accounting . . . . . . . . . . . . . . . . . . . 13 71 2.2.5.3. Analytics and Reporting . . . . . . . . . . . . . 13 72 2.2.5.4. Content Protection . . . . . . . . . . . . . . . 13 73 2.2.5.5. Notions common to multiple Log Consuming 74 Applications . . . . . . . . . . . . . . . . . . 14 75 3. CDNI Logging File . . . . . . . . . . . . . . . . . . . . . . 16 76 3.1. Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 16 77 3.2. CDNI Logging File Structure . . . . . . . . . . . . . . . 17 78 3.3. CDNI Logging Directives . . . . . . . . . . . . . . . . . 20 79 3.4. CDNI Logging Records . . . . . . . . . . . . . . . . . . 24 80 3.4.1. HTTP Request Logging Record . . . . . . . . . . . . . 25 81 3.5. CDNI Logging File Extension . . . . . . . . . . . . . . . 36 82 3.6. CDNI Logging File Example . . . . . . . . . . . . . . . . 36 83 3.7. Cascaded CDNI Logging Files Example . . . . . . . . . . . 38 84 4. Protocol for Exchange of CDNI Logging File After Full 85 Collection . . . . . . . . . . . . . . . . . . . . . . . . . 41 86 4.1. CDNI Logging Feed . . . . . . . . . . . . . . . . . . . . 42 87 4.1.1. Atom Formatting . . . . . . . . . . . . . . . . . . . 42 88 4.1.2. Updates to Log Files and the Feed . . . . . . . . . . 42 89 4.1.3. Redundant Feeds . . . . . . . . . . . . . . . . . . . 43 90 4.1.4. Example CDNI Logging Feed . . . . . . . . . . . . . . 43 91 4.2. CDNI Logging File Pull . . . . . . . . . . . . . . . . . 45 92 5. Protocol for Exchange of CDNI Logging File During Collection 46 93 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 47 94 6.1. CDNI Logging Directive Names Registry . . . . . . . . . . 47 95 6.2. CDNI Logging File version Registry . . . . . . . . . . . 48 96 6.3. CDNI Logging record-types Registry . . . . . . . . . . . 48 97 6.4. CDNI Logging Field Names Registry . . . . . . . . . . . . 49 98 6.5. CDNI Logging MIME Media Type . . . . . . . . . . . . . . 50 99 7. Security Considerations . . . . . . . . . . . . . . . . . . . 51 100 7.1. Authentication, Authorization, Confidentiality, Integrity 101 Protection . . . . . . . . . . . . . . . . . . . . . . . 51 102 7.2. Denial of Service . . . . . . . . . . . . . . . . . . . . 52 103 7.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 52 104 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 53 105 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 54 106 9.1. Normative References . . . . . . . . . . . . . . . . . . 54 107 9.2. Informative References . . . . . . . . . . . . . . . . . 55 108 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 57 110 1. Introduction 112 This memo specifies the CDNI Logging interface between a downstream 113 CDN (dCDN) and an upstream CDN (uCDN). First, it describes a 114 reference model for CDNI logging. Then, it specifies the CDNI 115 Logging File format and the actual protocol for exchange of CDNI 116 Logging Files. 118 The reader should be familiar with the following documents: 120 o CDNI problem statement [RFC6707] and framework [RFC7336] identify 121 a Logging interface, 123 o Section 8 of [RFC7337] specifies a set of requirements for 124 Logging, 126 o [RFC6770] outlines real world use-cases for interconnecting CDNs. 127 These use cases require the exchange of Logging information 128 between the dCDN and the uCDN. 130 As stated in [RFC6707], "the CDNI Logging interface enables details 131 of logs or events to be exchanged between interconnected CDNs". 133 The present document describes: 135 o The CDNI Logging reference model (Section 2), 137 o The CDNI Logging File format (Section 3), 139 o The CDNI Logging File Exchange protocol (Section 4). 141 1.1. Terminology 143 In this document, the first letter of each CDNI-specific term is 144 capitalized. We adopt the terminology described in [RFC6707] and 145 [RFC7336], and extend it with the additional terms defined below. 147 Intra-CDN Logging information: logging information generated and 148 collected within a CDN. The format of the Intra-CDN Logging 149 information may be different to the format of the CDNI Logging 150 information. 152 CDNI Logging information: logging information exchanged across CDNs 153 using the CDNI Logging Interface. 155 Logging information: logging information generated and collected 156 within a CDN or obtained from another CDN using the CDNI Logging 157 Interface. 159 CDNI Logging Field: an atomic element of information that can be 160 included in a CDNI Logging Record. The time an event/task started, 161 the IP address of an End User to whom content was delivered, and the 162 Uniform Resource Identifier (URI) of the content delivered, are 163 examples of CDNI Logging fields. 165 CDNI Logging Record: an information record providing information 166 about a specific event. This comprises a collection of CDNI Logging 167 fields. 169 CDNI Logging File: a file containing CDNI Logging Records, as well as 170 additional information facilitating the processing of the CDNI 171 Logging Records. 173 CDN Reporting: the process of providing the relevant information that 174 will be used to create a formatted content delivery report provided 175 to the CSP in deferred time. Such information typically includes 176 aggregated data that can cover a large period of time (e.g., from 177 hours to several months). Uses of Reporting include the collection 178 of charging data related to CDN services and the computation of Key 179 Performance Indicators (KPIs). 181 CDN Monitoring: the process of providing or displaying content 182 delivery information in a timely fashion with respect to the 183 corresponding deliveries. Monitoring typically includes visibility 184 of the deliveries in progress for service operation purposes. It 185 presents a view of the global health of the services as well as 186 information on usage and performance, for network services 187 supervision and operation management. In particular, monitoring data 188 can be used to generate alarms. 190 1.2. Requirements Language 192 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 193 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 194 "OPTIONAL" in this document are to be interpreted as described in RFC 195 2119 [RFC2119]. 197 2. CDNI Logging Reference Model 199 2.1. CDNI Logging interactions 201 The CDNI logging reference model between a given uCDN and a given 202 dCDN involves the following interactions: 204 o customization by the uCDN of the CDNI Logging information to be 205 provided by the dCDN to the uCDN (e.g., control of which CDNI 206 Logging fields are to be communicated to the uCDN for a given task 207 performed by the dCDN or control of which types of events are to 208 be logged). The dCDN takes into account this CDNI Logging 209 customization information to determine what Logging information to 210 provide to the uCDN, but it may, or may not, take into account 211 this CDNI Logging customization information to influence what CDN 212 logging information is to be generated and collected within the 213 dCDN (e.g., even if the uCDN requests a restricted subset of the 214 logging information, the dCDN may elect to generate a broader set 215 of logging information). The mechanism to support the 216 customization by the uCDN of CDNI Logging information is outside 217 the scope of this document and left for further study. Until such 218 a mechanism is available, the uCDN and dCDN are expected to agree 219 off-line on what exact set of CDNI Logging information is to be 220 provided by the dCDN to the uCDN, and to rely on management plane 221 actions to configure the CDNI Logging functions in the dCDN to 222 generate this information set and in the uCDN to expect this 223 information set. 225 o generation and collection by the dCDN of the intra-CDN Logging 226 information related to the completion of any task performed by the 227 dCDN on behalf of the uCDN (e.g., delivery of the content to an 228 End User) or related to events happening in the dCDN that are 229 relevant to the uCDN (e.g., failures or unavailability in dCDN). 230 This takes place within the dCDN and does not directly involve 231 CDNI interfaces. 233 o communication by the dCDN to the uCDN of the Logging information 234 collected by the dCDN relevant to the uCDN. This is supported by 235 the CDNI Logging interface and in the scope of the present 236 document. For example, the uCDN may use this Logging information 237 to charge the CSP, to perform analytics and monitoring for 238 operational reasons, to provide analytics and monitoring views on 239 its content delivery to the CSP or to perform trouble-shooting. 240 This document exclusively specifies non-real-time exchange of 241 Logging information. Closer to real-time exchange of Logging 242 information (say sub-minute or sub-second) is outside the scope of 243 the present document and left for further study. This document 244 exclusively specifies exchange of Logging information related to 245 content delivery. Exchange of Logging information related to 246 operational events (e.g., dCDN request routing function 247 unavailable, content acquisition failure by dCDN) for audit or 248 operational reactive adjustments by uCDN is outside the scope of 249 the present document and left for further study. 251 o customization by the dCDN of the CDNI Logging information to be 252 provided by the uCDN on behalf of the dCDN. The mechanism to 253 support the customization by the dCDN of CDNI Logging information 254 is outside the scope of this document and left for further study. 256 o generation and collection by the uCDN of Intra-CDN Logging 257 information related to the completion of any task performed by the 258 uCDN on behalf of the dCDN (e.g., serving of content by uCDN to 259 dCDN for acquisition purposes by dCDN) or related to events 260 happening in the uCDN that are relevant to the dCDN. This takes 261 place within the uCDN and does not directly involve CDNI 262 interfaces. 264 o communication by the uCDN to the dCDN of the Logging information 265 collected by the uCDN relevant to the dCDN. For example, the dCDN 266 might potentially benefit from this information for security 267 auditing or content acquisition troubleshooting. This is outside 268 the scope of this document and left for further study. 270 Figure 1 provides an example of CDNI Logging interactions (focusing 271 only on the interactions that are in the scope of this document) in a 272 particular scenario where four CDNs are involved in the delivery of 273 content from a given CSP: the uCDN has a CDNI interconnection with 274 dCDN-1 and dCDN-2. In turn, dCDN-2 has a CDNI interconnection with 275 dCDN-3, where dCDN-2 is acting as an upstream CDN relative to dCDN-3. 276 In this example, uCDN, dCDN-1, dCDN-2 and dCDN-3 all participate in 277 the delivery of content for the CSP. In this example, the CDNI 278 Logging interface enables the uCDN to obtain Logging information from 279 all the dCDNs involved in the delivery. In the example, the uCDN 280 uses the Logging information: 282 o to analyze the performance of the delivery performed by the dCDNs 283 and to adjust its operations after the fact (e.g., request 284 routing) as appropriate, 286 o to provide (non-real-time) reporting and monitoring information to 287 the CSP. 289 For instance, the uCDN merges Logging information, extracts relevant 290 KPIs, and presents a formatted report to the CSP, in addition to a 291 bill for the content delivered by uCDN itself or by its dCDNs on the 292 CSP's behalf. The uCDN may also provide Logging information as raw 293 log files to the CSP, so that the CSP can use its own logging 294 analysis tools. 296 +-----+ 297 | CSP | 298 +-----+ 299 ^ Reporting and monitoring data 300 * Billing 301 ,--*--. 302 Logging ,-' `-. 303 Data =>( uCDN )<= Logging 304 // `-. _,-' \\ Data 305 || `-'-'-' || 306 ,-----. ,-----. 307 ,-' `-. ,-' `-. 308 ( dCDN-1 ) ( dCDN-2 )<== Logging 309 `-. ,-' `-. _,-' \\ Data 310 `--'--' `--'-' || 311 ,-----. 312 ,' `-. 313 ( dCDN-3 ) 314 `. ,-' 315 `--'--' 317 ===> CDNI Logging Interface 318 ***> outside the scope of CDNI 320 Figure 1: Interactions in CDNI Logging Reference Model 322 A downstream CDN relative to uCDN (e.g., dCDN-2) integrates the 323 relevant Logging information obtained from its own downstream CDNs 324 (i.e., dCDN-3) in the Logging information that it provides to the 325 uCDN, so that the uCDN ultimately obtains all Logging information 326 relevant to a CSP for which it acts as the authoritative CDN. Such 327 aggregation is further discussed in Section 3.7. 329 Note that the format of Logging information that a CDN provides over 330 the CDNI interface might be different from the one that the CDN uses 331 internally. In this case, the CDN needs to reformat the Logging 332 information before it provides this information to the other CDN over 333 the CDNI Logging interface. Similarly, a CDN might reformat the 334 Logging information that it receives over the CDNI Logging interface 335 before injecting it into its log-consuming applications or before 336 providing some of this Logging information to the CSP. Such 337 reformatting operations introduce latency in the logging distribution 338 chain and introduce a processing burden. Therefore, there are 339 benefits in specifying CDNI Logging formats that are suitable for use 340 inside CDNs and also are close to the intra-CDN Logging formats 341 commonly used in CDNs today. 343 2.2. Overall Logging Chain 345 This section discusses the overall logging chain within and across 346 CDNs to clarify how CDN Logging information is expected to fit in 347 this overall chain. Figure 2 illustrates the overall logging chain 348 within the dCDN, across CDNs using the CDNI Logging interface and 349 within the uCDN. Note that the logging chain illustrated in the 350 Figure is obviously only an example and varies depending on the 351 specific environments. For example, there may be more or fewer 352 instantiations of each entity (e.g., there may be 4 Log consuming 353 applications in a given CDN). As another example, there may be one 354 instance of Rectification process per Log Consuming Application 355 instead of a shared one. 357 Log Consuming Log Consuming 358 App App 359 ^ ^ 360 | | 361 Rectification---------- 362 ^ 363 | 364 Filtering 365 ^ 366 | 367 Collection 368 ^ ^ 369 | | 370 | Generation 371 | 372 | uCDN 373 CDNI Logging --------------------------------------------------- 374 exchange dCDN 375 ^ 376 | Log Consuming Log Consuming 377 | App App 378 | ^ ^ 379 | | | 380 Rectification Rectification--------- 381 ^ ^ 382 | | 383 Filtering 384 ^ 385 | 386 Collection 387 ^ ^ 388 | | 389 Generation Generation 391 Figure 2: CDNI Logging in the overall Logging Chain 393 The following subsections describe each of the processes potentially 394 involved in the logging chain of Figure 2. 396 2.2.1. Logging Generation and During-Generation Aggregation 398 CDNs typically generate Logging information for all significant task 399 completions, events, and failures. Logging information is typically 400 generated by many devices in the CDN including the surrogates, the 401 request routing system, and the control system. 403 The amount of Logging information generated can be huge. Therefore, 404 during contract negotiations, interconnected CDNs often agree on a 405 retention duration for Logging information, and/or potentially on a 406 maximum volume of Logging information that the dCDN ought to keep. 407 If this volume is exceeded, the dCDN is expected to alert the uCDN 408 but may not keep more Logging information for the considered time 409 period. In addition, CDNs may aggregate Logging information and 410 transmit only summaries for some categories of operations instead of 411 the full Logging information. Note that such aggregation leads to an 412 information loss, which may be problematic for some usages of the 413 Logging information (e.g., debugging). 415 [RFC6983] discusses logging for HTTP Adaptive Streaming (HAS). In 416 accordance with the recommendations articulated there, it is expected 417 that a surrogate will generate separate Logging information for 418 delivery of each chunk of HAS content. This ensures that separate 419 Logging information can then be provided to interconnected CDNs over 420 the CDNI Logging interface. Still in line with the recommendations 421 of [RFC6983], the Logging information for per-chunk delivery may 422 include some information (a Content Collection IDentifier and a 423 Session IDentifier) intended to facilitate subsequent post-generation 424 aggregation of per-chunk logs into per-session logs. Note that a CDN 425 may also elect to generate aggregate per-session logs when performing 426 HAS delivery, but this needs to be in addition to, and not instead 427 of, the per-chunk delivery logs. We note that aggregate per-session 428 logs for HAS delivery are for further study and outside the scope of 429 this document. 431 2.2.2. Logging Collection 433 This is the process that continuously collects Logging information 434 generated by the log-generating entities within a CDN. 436 In a CDNI environment, in addition to collecting Logging information 437 from log-generating entities within the local CDN, the Collection 438 process also collects Logging information provided by another CDN, or 439 other CDNs, through the CDNI Logging interface. This is illustrated 440 in Figure 2 where we see that the Collection process of the uCDN 441 collects Logging information from log-generating entities within the 442 uCDN as well as Logging information coming from the dCDNs through the 443 CDNI Logging interface. 445 2.2.3. Logging Filtering 447 A CDN may be required to only present different subsets of the whole 448 Logging information collected to various log-consuming applications. 449 This is achieved by the Filtering process. 451 In particular, the Filtering process can also filter the right subset 452 of Logging information that needs to be provided to a given 453 interconnected CDN. For example, the filtering process in the dCDN 454 can be used to ensure that only the Logging information related to 455 tasks performed on behalf of a given uCDN are made available to that 456 uCDN (thereby filtering out all the Logging information related to 457 deliveries by the dCDN of content for its own CSPs). Similarly, the 458 Filtering process may filter or partially mask some fields, for 459 example, to protect End Users' privacy when communicating CDNI 460 Logging information to another CDN. Filtering of Logging information 461 prior to communication of this information to other CDNs via the CDNI 462 Logging interface requires that the downstream CDN can recognize the 463 subset of Logging information that relate to each interconnected CDN. 465 The CDN will also filter some internal scope information such as 466 information related to its internal alarms (security, failures, load, 467 etc). 469 In some use cases described in [RFC6770], the interconnected CDNs do 470 not want to disclose details on their internal topology. The 471 filtering process can then also filter confidential data on the 472 dCDNs' topology (number of servers, location, etc.). In particular, 473 information about the requests served by each Surrogate may be 474 confidential. Therefore, the Logging information needs to be 475 protected so that data such as Surrogates' hostnames are not 476 disclosed to the uCDN. In the "Inter-Affiliates Interconnection" use 477 case, this information may be disclosed to the uCDN because both the 478 dCDN and the uCDN are operated by entities of the same group. 480 2.2.4. Logging Rectification and Post-Generation Aggregation 482 If Logging information is generated periodically, it is important 483 that the sessions that start in one Logging period and end in another 484 are correctly reported. If they are reported in the starting period, 485 then the Logging information of this period will be available only 486 after the end of the session, which delays the Logging information 487 generation. A simple approach is to provide the complete Logging 488 Record for a session in the Logging Period of the session end. 490 A Logging rectification/update mechanism could be useful to reach a 491 good trade-off between the Logging information generation delay and 492 the Logging information accuracy. 494 In the presence of HAS, some log-consuming applications can benefit 495 from aggregate per-session logs. For example, for analytics, per- 496 session logs allow display of session-related trends which are much 497 more meaningful for some types of analysis than chunk-related trends. 498 In the case where aggregate logs have been generated directly by the 499 log-generating entities, those can be used by the applications. In 500 the case where aggregate logs have not been generated, the 501 Rectification process can be extended with a Post-Generation 502 Aggregation process that generates per-session logs from the per- 503 chunk logs, possibly leveraging the information included in the per- 504 chunk logs for that purpose (Content Collection IDentifier and a 505 Session IDentifier). However, in accordance with [RFC6983], this 506 document does not define exchange of such aggregate logs on the CDNI 507 Logging interface. We note that this is for further study and 508 outside the scope of this document. 510 2.2.5. Log-Consuming Applications 512 2.2.5.1. Maintenance/Debugging 514 Logging information is useful to permit the detection (and limit the 515 risk) of content delivery failures. In particular, Logging 516 information facilitates the detection of configuration issues. 518 To detect faults, Logging information needs to report success and 519 failure of CDN delivery operations. The uCDN can summarize such 520 information into KPIs. For instance, Logging information needs to 521 allow the computation of the number of times, during a given time 522 period, that content delivery related to a specific service succeeds/ 523 fails. 525 Logging information enables the CDN providers to identify and 526 troubleshoot performance degradations. In particular, Logging 527 information enables tracking of traffic data (e.g., the amount of 528 traffic that has been forwarded by a dCDN on behalf of an uCDN over a 529 given period of time), which is particularly useful for CDN and 530 network planning operations. 532 Some of these maintenance and debugging applications only require 533 aggregate logging information highly compatible with use of 534 anonymization of IP addresses (as supported by the present document 535 and specified in the definition of the c-groupid field under 536 Section 3.4.1). However, in some situations, it may be useful, where 537 compatible with privacy protection, to access some CDNI Logging 538 Records containing full non-anonymized IP addresses. This is allowed 539 in the definition of the c-groupid (under Section 3.4.1), with very 540 significant privacy protection limitations that are discussed in the 541 definition of the c-groupid field. For example, this may be useful 542 for detailed fault tracking of a particular end user content delivery 543 issue. Where there is a hard requirement by uCDN or CSP to associate 544 a given enduser to individual CDNI Logging Records (e.g., to allow 545 a-posteriori analysis of individual delivery for example in 546 situations of performance-based penalties), instead of using 547 aggregates containing a single client as discussed in the c-groupid 548 field definition, an alternate approach is to ensure that a client 549 identifier is embedded in the request fields that can be logged in a 550 CDNI Logging Record (for example by including the client identifier 551 in the URI query string or in a HTTP Header). That latter approach 552 offers two strong benefits: first, the aggregate inside the c-groupid 553 can contain more than one client, thereby ensuring stronger privacy 554 protection; second, it allows a reliable identification of the client 555 while IP address does not in many situations (e.g., behind NAT, where 556 dynamic IP addresses are used and reused,...). However, care SHOULD 557 be taken that the client identifiers exposed in other fields of the 558 CDNI Records cannot themselves be linked back to actual users. 560 2.2.5.2. Accounting 562 Logging information is essential for accounting, to permit inter-CDN 563 billing and CSP billing by uCDNs. For instance, Logging information 564 provided by dCDNs enables the uCDN to compute the total amount of 565 traffic delivered by every dCDN for a particular Content Provider, as 566 well as, the associated bandwidth usage (e.g., peak, 95th 567 percentile), and the maximum number of simultaneous sessions over a 568 given period of time. 570 2.2.5.3. Analytics and Reporting 572 The goals of analytics include gathering any relevant information in 573 order to be able to develop statistics on content download, analyze 574 user behavior, and monitor the performance and quality of content 575 delivery. For instance, Logging information enables the CDN 576 providers to report on content consumption (e.g., delivered sessions 577 per content) in a specific geographic area. 579 The goal of reporting is to gather any relevant information to 580 monitor the performance and quality of content delivery and allow 581 detection of delivery issues. For instance, reporting could track 582 the average delivery throughput experienced by End Users in a given 583 region for a specific CSP or content set over a period of time. 585 2.2.5.4. Content Protection 587 The goal of content protection is to prevent and monitor unauthorized 588 access, misuse, modification, and denial of access to a content. A 589 set of information is logged in a CDN for security purposes. In 590 particular, a record of access to content is usually collected to 591 permit the CSP to detect infringements of content delivery policies 592 and other abnormal End User behaviors. 594 2.2.5.5. Notions common to multiple Log Consuming Applications 596 2.2.5.5.1. Logging Information Views 598 Within a given log-consuming application, different views may be 599 provided to different users depending on privacy, business, and 600 scalability constraints. 602 For example, an analytics tool run by the uCDN can provide one view 603 to an uCDN operator that exploits all the Logging information 604 available to the uCDN, while the tool may provide a different view to 605 each CSP exploiting only the Logging information related to the 606 content of the given CSP. 608 As another example, maintenance and debugging tools may provide 609 different views to different CDN operators, based on their 610 operational role. 612 2.2.5.5.2. Key Performance Indicators (KPIs) 614 This section presents, for explanatory purposes, a non-exhaustive 615 list of Key Performance Indicators (KPIs) that can be extracted/ 616 produced from logs. 618 Multiple log-consuming applications, such as analytics, monitoring, 619 and maintenance applications, often compute and track such KPIs. 621 In a CDNI environment, depending on the situation, these KPIs may be 622 computed by the uCDN or by the dCDN. But it is usually the uCDN that 623 computes KPIs, because the uCDN and dCDN may have different 624 definitions of the KPIs and the computation of some KPIs requires a 625 vision of all the deliveries performed by the uCDN and all its dCDNs. 627 Here is a list of important examples of KPIs: 629 o Number of delivery requests received from End Users in a given 630 region for each piece of content, during a given period of time 631 (e.g., hour/day/week/month) 633 o Percentage of delivery successes/failures among the aforementioned 634 requests 636 o Number of failures listed by failure type (e.g., HTTP error code) 637 for requests received from End Users in a given region and for 638 each piece of content, during a given period of time (e.g., 639 hour/day/week/month) 641 o Number and cause of premature delivery termination for End Users 642 in a given region and for each piece of content, during a given 643 period of time (e.g., hour/day/week/month) 645 o Maximum and mean number of simultaneous sessions established by 646 End Users in a given region, for a given Content Provider, and 647 during a given period of time (e.g., hour/day/week/month) 649 o Volume of traffic delivered for sessions established by End Users 650 in a given region, for a given Content Provider, and during a 651 given period of time (e.g., hour/day/week/month) 653 o Maximum, mean, and minimum delivery throughput for sessions 654 established by End Users in a given region, for a given Content 655 Provider, and during a given period of time (e.g., hour/day/week/ 656 month) 658 o Cache-hit and byte-hit ratios for requests received from End Users 659 in a given region for each piece of content, during a given period 660 of time (e.g., hour/day/week/month) 662 o Top 10 most popularly requested contents (during a given day/week/ 663 month) 665 o Terminal type (mobile, PC, STB, if this information can be 666 acquired from the browser type inferred from the User Agent 667 string, for example). 669 Additional KPIs can be computed from other sources of information 670 than the Logging information, for instance, data collected by a 671 content portal or by specific client-side application programming 672 interfaces. Such KPIs are out of scope for the present document. 674 The KPIs used depend strongly on the considered log-consuming 675 application -- the CDN operator may be interested in different 676 metrics than the CSP is. In particular, CDN operators are often 677 interested in delivery and acquisition performance KPIs, information 678 related to Surrogates' performance, caching information to evaluate 679 the cache-hit ratio, information about the delivered file size to 680 compute the volume of content delivered during peak hour, etc. 682 Some of the KPIs, for instance those providing an instantaneous 683 vision of the active sessions for a given CSP's content, are useful 684 essentially if they are provided in a timely manner. By contrast, 685 some other KPIs, such as those averaged on a long period of time, can 686 be provided in non-real-time. 688 3. CDNI Logging File 690 3.1. Rules 692 This specification uses the Augmented Backus-Naur Form (ABNF) 693 notation and core rules of [RFC5234]. In particular, the present 694 document uses the following rules from [RFC5234]: 696 CR = %x0D ; carriage return 698 ALPHA = %x41-5A / %x61-7A ; A-Z / a-z 700 DIGIT = %x30-39 ; 0-9 702 DQUOTE = %x22 ; " (Double Quote) 704 CRLF = CR LF ; Internet standard newline 706 HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F" 708 HTAB = %x09 ; horizontal tab 710 LF = %x0A ; linefeed 712 VCHAR = %x21-7E ; visible (printing) characters 714 OCTET = %x00-FF ; 8 bits of data 716 The present document also uses the following rules from [RFC3986]: 718 host = as specified in section 3.2.2 of [RFC3986]. 720 IPv4address = as specified in section 3.2.2 of [RFC3986]. 722 IPv6address = as specified in section 3.2.2 of [RFC3986]. 724 partial-time = as specified in [RFC3339]. 726 The present document also defines the following additional rules: 728 ADDRESS = IPv4address / IPv6address 730 ALPHANUM = ALPHA / DIGIT 732 DATE = 4DIGIT "-" 2DIGIT "-" 2DIGIT 734 ; Dates are encoded as "full-date" specified in [RFC3339]. 736 DEC = 1*DIGIT ["." 1*DIGIT] 738 NAMEFORMAT = ALPHANUM *(ALPHANUM / "_" / "-") 740 QSTRING = DQUOTE *(NDQUOTE / PCT-ENCODED) DQUOTE 742 NDQUOTE = %x20-21 / %x23-24 / %x26-7E / UTF8-2 / UTF8-3 / UTF8-4 744 ; whereby a DQUOTE is conveyed inside a QSTRING unambiguously 745 by escaping it with PCT-ENCODED. 747 PCT-ENCODED = "%" HEXDIG HEXDIG 749 ; percent encoding is used for escaping octets that might be 750 possible in HTTP headers such as bare CR, bare LF, CR LF, HTAB, 751 SP or null. These octets are rendered with percent encoding in 752 ABNF as specified by [RFC3986] in order to avoid considering 753 them as separators for the logging records. 755 NHTABSTRING = 1*(SP / VCHAR) 757 TIME = partial-time 759 USER-COMMENT = * (SP / VCHAR / UTF8-2 / UTF8-3 / UTF8-4) 761 3.2. CDNI Logging File Structure 763 As defined in Section 1.1: a CDNI Logging Field is as an atomic 764 logging information element, a CDNI Logging Record is a collection of 765 CDNI Logging fields containing all logging information corresponding 766 to a single logging event, and a CDNI Logging File contains a 767 collection of CDNI Logging Records. This structure is illustrated in 768 Figure 3. The use of a file structure for transfer of CDNI Logging 769 information is selected since this is the most common practise today 770 for exchange of logging information within and across CDNs. 772 +----------------------------------------------------------+ 773 |CDNI Logging File | 774 | | 775 | #Directive 1 | 776 | #Directive 2 | 777 | ... | 778 | #Directive P | 779 | | 780 | +------------------------------------------------------+ | 781 | |CDNI Logging Record 1 | | 782 | | +-------------+ +-------------+ +-------------+ | | 783 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 784 | | | Field 1 | | Field 2 | | Field N | | | 785 | | +-------------+ +-------------+ +-------------+ | | 786 | +------------------------------------------------------+ | 787 | | 788 | +------------------------------------------------------+ | 789 | |CDNI Logging Record 2 | | 790 | | +-------------+ +-------------+ +-------------+ | | 791 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 792 | | | Field 1 | | Field 2 | | Field N | | | 793 | | +-------------+ +-------------+ +-------------+ | | 794 | +------------------------------------------------------+ | 795 | | 796 | ... | 797 | | 798 | #Directive P+1 | 799 | | 800 | ... | 801 | | 802 | +------------------------------------------------------+ | 803 | |CDNI Logging Record M | | 804 | | +-------------+ +-------------+ +-------------+ | | 805 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 806 | | | Field 1 | | Field 2 | | Field N | | | 807 | | +-------------+ +-------------+ +-------------+ | | 808 | +------------------------------------------------------+ | 809 | | 810 | | 811 | #Directive P+Q | 812 +----------------------------------------------------------+ 814 Figure 3: Structure of Logging Files 816 The CDNI Logging File format is inspired from the W3C Extended Log 817 File Format [ELF]. However, it is fully specified by the present 818 document. Where the present document differs from the W3C Extended 819 Log File Format, an implementation of the CDNI Logging interface MUST 820 comply with the present document. The W3C Extended Log File Format 821 was used as a starting point, reused where possible and expanded when 822 necessary. 824 Using a format that resembles the W3C Extended Log File Format is 825 intended to keep CDNI logging format close to the intra-CDN Logging 826 information format commonly used in CDNs today, thereby minimizing 827 systematic translation at CDN/CDNI boundary. 829 A CDNI Logging File MUST contain a sequence of lines containing US- 830 ASCII characters [CHAR_SET] terminated by CRLF. Each line of a CDNI 831 Logging File MUST contain either a directive or a CDNI Logging 832 Record. 834 Directives record information about the CDNI Logging process itself. 835 Lines containing directives MUST begin with the "#" character. 836 Directives are specified in Section 3.3. 838 Logging Records provide actual details of the logged event. Logging 839 Records are specified in Section 3.4. 841 The CDNI Logging File has a specific structure. It always starts 842 with a directive line and the first directive it contains MUST be the 843 version. 845 The directive lines form together a group that contains at least one 846 directive line. Each directives group is followed by a group of 847 logging records. The records group contains zero or more actual 848 logging record lines about the event being logged. A record line 849 consists of the values corresponding to all or a subset of the 850 possible Logging fields defined within the scope of the record-type 851 directive. These values MUST appear in the order defined by the 852 fields directive. 854 Note that future extensions MUST be compliant with the previous 855 description. The following examples depict the structure of a 856 CDNILOGFILE as defined currently by the record-type 857 "cdni_http_request_v1." 859 DIRLINE = "#" directive CRLF 861 DIRGROUP = 1*DIRLINE 863 RECLINE = 866 RECGROUP = *RECLINE 868 CDNILOGFILE = 1*(DIRGROUP RECGROUP) 870 All directive names and field names defined in the logging file are 871 case-insensitive as per the basic ABNF([RFC5234]). 873 3.3. CDNI Logging Directives 875 A CDNI Logging directive line contains the directive name followed by 876 ":" HTAB and the directive value. 878 Directive names MUST be of the format NAMEFORMAT. All directive 879 names MUST be registered in the CDNI Logging Directives Names 880 registry. Unknown directives MUST be ignored. Directive values can 881 have various formats. All possible directive values for the record- 882 type "cdni_http_request_v1" are further detailed in this section. 884 The following example shows the structure of a directive and 885 enumerates strictly the directive values presently defined in the 886 version "CDNI/1.0" of the CDNI Logging File. 888 directive = DIRNAME ":" HTAB DIRVAL 890 DIRNAME = NAMEFORMAT 892 DIRVAL = NHTABSTRING / QSTRING / host / USER-COMMENT / FIENAME * 893 (HTAB FIENAME) / 64HEXDIG 895 An implementation of the CDNI Logging interface MUST support all of 896 the following directives, listed below by their directive name: 898 o version: 900 * format: NHTABSTRING 902 * directive value: indicates the version of the CDNI Logging File 903 format. The entity transmitting a CDNI Logging File as per the 904 present document MUST set the value to "CDNI/1.0". In the 905 future, other versions of CDNI Logging File might be specified; 906 those would use a value different to "CDNI/1.0" allowing the 907 entity receiving the CDNI Logging File to identify the 908 corresponding version. 910 * occurrence: there MUST be one and only one instance of this 911 directive per CDNI Logging File. It MUST be the first line of 912 the CDNI Logging File. 914 * example: "version: HTAB CDNI/1.0". 916 o UUID: 918 * format: NHTABSTRING 920 * directive value: this a Uniform Resource Name (URN) from the 921 Universally Unique IDentifier (UUID) URN namespace specified in 922 [RFC4122]). The UUID contained in the URN uniquely identifies 923 the CDNI Logging File. 925 * occurrence: there MUST be one and only one instance of this 926 directive per CDNI Logging File. 928 * example: "UUID: HTAB NHTABSTRING". 930 o claimed-origin: 932 * format: host 934 * directive value: this contains the claimed identification of 935 the entity transmitting the CDNI Logging File (e.g., the host 936 in a dCDN supporting the CDNI Logging interface) or the entity 937 responsible for transmitting the CDNI Logging File (e.g., the 938 dCDN). 940 * occurrence: there MUST be zero or exactly one instance of this 941 directive per CDNI Logging File. This directive MAY be 942 included by the dCDN. It MUST NOT be included or modified by 943 the uCDN. 945 * example: "claimed-origin: HTAB host". 947 o established-origin: 949 * format: host 951 * directive value: this contains the identification, as 952 established by the entity receiving the CDNI Logging File, of 953 the entity transmitting the CDNI Logging File (e.g., the host 954 in a dCDN supporting the CDNI Logging interface) or the entity 955 responsible for transmitting the CDNI Logging File (e.g., the 956 dCDN). 958 * occurrence: there MUST be zero or exactly one instance of this 959 directive per CDNI Logging File. This directive MAY be added 960 by the uCDN (e.g., before storing the CDNI Logging File). It 961 MUST NOT be included by the dCDN. The mechanisms used by the 962 uCDN to establish and validate the entity responsible for the 963 CDNI Logging File is outside the scope of the present document. 964 We observe that, in particular, this may be achieved through 965 authentication mechanisms that are part of the transport layer 966 of the CDNI Logging File pull mechanism (Section 4.2). 968 * ABNF example: "established-origin: HTAB host". 970 o remark: 972 * format: USER-COMMENT 974 * directive value: this contains comment information. Data 975 contained in this field is to be ignored by analysis tools. 977 * occurrence: there MAY be zero, one or any number of instance of 978 this directive per CDNI Logging File. 980 * example: "remark: HTAB USER-COMMENT". 982 o record-type: 984 * format: NAMEFORMAT 986 * directive value: indicates the type of the CDNI Logging Records 987 that follow this directive, until another record-type directive 988 (or the end of the CDNI Logging File). This can be any CDNI 989 Logging Record type registered in the CDNI Logging Record-types 990 registry (Section 6.3). For example this may be 991 "cdni_http_request_v1" as specified in Section 3.4.1. 993 * occurrence: there MUST be at least one instance of this 994 directive per CDNI Logging File. The first instance of this 995 directive MUST precede a fields directive and MUST precede all 996 CDNI Logging Records. 998 * example: "record-type: HTAB cdni_http_request_v1". 1000 o fields: 1002 * format: FIENAME *(HTAB FIENAME) ; where FIENAME can take any 1003 CDNI Logging field name registered in the CDNI Logging Field 1004 Names registry (Section 6.4) that is valid for the record type 1005 specified in the record-type directive. 1007 * directive value: this lists the names of all the fields for 1008 which a value is to appear in the CDNI Logging Records that 1009 follow the instance of this directive (until another instance 1010 of this directive). The names of the fields, as well as their 1011 occurrences, MUST comply with the corresponding rules specified 1012 in the document referenced in the CDNI Logging Record-types 1013 registry (Section 6.3) for the corresponding CDNI Logging 1014 record-type. 1016 * occurrence: there MUST be at least one instance of this 1017 directive per record-type directive. The first instance of 1018 this directive for a given record-type MUST appear before any 1019 CDNI Logging Record for this record-type. One situation where 1020 more than one instance of the fields directive can appear 1021 within a given CDNI Logging File, is when there is a change, in 1022 the middle of a fairly large logging period, in the agreement 1023 between the uCDN and the dCDN about the set of fields that are 1024 to be exchanged. The multiple occurrences allow records with 1025 the old set of fields and records with the new set of fields to 1026 be carried inside the same Logging File. 1028 * example: "fields: HTAB FIENAME * (HTAB FIENAME)". 1030 o SHA256-hash: 1032 * format: 64HEXDIG 1034 * directive value: This directive permits the detection of a 1035 corrupted CDNI Logging File. This can be useful, for instance, 1036 if a problem occurs on the filesystem of the dCDN Logging 1037 system and leads to a truncation of a logging file. The valid 1038 SHA256-hash value is included in this directive by the entity 1039 that transmits the CDNI Logging File. It MUST be computed by 1040 applying the SHA-256 ([RFC6234]) cryptographic hash function on 1041 the CDNI Logging File, including all the directives and logging 1042 records, up to the SHA256-hash directive itself, excluding the 1043 SHA256-hash directive itself. The SHA256-hash value MUST be 1044 represented as a US-ASCII encoded hexadecimal number, 64 digits 1045 long (representing a 256 bit hash value). The entity receiving 1046 the CDNI Logging File also computes in a similar way the 1047 SHA-256 hash on the received CDNI Logging File and compares 1048 this hash to the value of the SHA256-hash directive. If the 1049 two values are equal, then the received CDNI Logging File is to 1050 be considered non-corrupted. If the two values are different, 1051 the received CDNI Logging File is to be considered corrupted. 1052 The behavior of the entity that received a corrupted CDNI 1053 Logging File is outside the scope of this specification; we 1054 note that the entity MAY attempt to pull again the same CDNI 1055 Logging File from the transmitting entity. If the entity 1056 receiving a non-corrupted CDNI Logging File adds an 1057 established-origin directive, it MUST then recompute and update 1058 the SHA256-hash directive so it also protects the added 1059 established-origin directive. 1061 * occurrence: there MUST be zero or exactly one instance of this 1062 directive. There SHOULD be exactly one instance of this 1063 directive. One situation where that directive could be omitted 1064 is where integrity protection is already provided via another 1065 mechanism (for example if an integrity hash is associated to 1066 the CDNI Logging File out-of-band through the CDNI Logging Feed 1067 ( Section 4.1) leveraging ATOM extensions such as those 1068 proposed in [I-D.snell-atompub-link-extensions]. When present, 1069 the SHA256-hash field MUST be the last line of the CDNI Logging 1070 File. 1072 * example: "SHA256-hash: HTAB 64HEXDIG". 1074 An uCDN-side implementation of the CDNI Logging interface MUST reject 1075 a CDNI Logging File that does not comply with the occurrences 1076 specified above for each and every directive. For example, an uCDN- 1077 side implementation of the CDNI Logging interface receiving a CDNI 1078 Logging file with zero occurrence of the version directive, or with 1079 two occurrences of the SHA256-hash, MUST reject this CDNI Logging 1080 File. 1082 An entity receiving a CDNI Logging File with a value set to 1083 "CDNI/1.0" MUST process the CDNI Logging File as per the present 1084 document. An entity receiving a CDNI Logging File with a value set 1085 to a different value MUST process the CDNI Logging File as per the 1086 specification referenced in the CDNI Logging File version registry 1087 (see Section 6.1) if the implementation supports this specification 1088 and MUST reject the CDNI Logging File otherwise. 1090 3.4. CDNI Logging Records 1092 A CDNI Logging Record consists of a sequence of CDNI Logging fields 1093 relating to that single CDNI Logging Record. 1095 CDNI Logging fields MUST be separated by the "horizontal tabulation 1096 (HTAB)" character. 1098 To facilitate readability, a prefix scheme is used for CDNI Logging 1099 field names in a similar way to the one used in W3C Extended Log File 1100 Format [ELF]. The semantics of the prefix in the present document 1101 is: 1103 o "c-" refers to the User Agent that issues the request (corresponds 1104 to the "client" of W3C Extended Log Format) 1106 o "d-" refers to the dCDN (relative to a given CDN acting as an 1107 uCDN) 1109 o "s-" refers to the dCDN Surrogate that serves the request 1110 (corresponds to the "server" of W3C Extended Log Format) 1112 o "u-" refers to the uCDN (relative to a given CDN acting as a dCDN) 1114 o "cs-" refers to communication from the User Agent towards the dCDN 1115 Surrogate 1117 o "sc-" refers to communication from the dCDN Surrogate towards the 1118 User Agent 1120 An implementation of the CDNI Logging interface as per the present 1121 specification MUST support the CDNI HTTP Request Logging Record as 1122 specified in Section 3.4.1. 1124 A CDNI Logging Record contains the corresponding values for the 1125 fields that are enumerated in the last fields directive before the 1126 current log line. Note that the order in which the field values 1127 appear is dictated by the order of the fields names in the fields 1128 directive. There SHOULD be no dependency between the various fields 1129 values. 1131 3.4.1. HTTP Request Logging Record 1133 This section defines the CDNI Logging Record of record-type 1134 "cdni_http_request_v1". It is applicable to content delivery 1135 performed by the dCDN using HTTP/1.0([RFC1945]), 1136 HTTP/1.1([RFC7230],[RFC7231], [RFC7232], [RFC7233], [RFC7234], 1137 [RFC7235]) or HTTPS ([RFC2818], [RFC7230]). We observe that, in the 1138 case of HTTPS delivery, there may be value in logging additional 1139 information specific to the operation of HTTP over TLS and we note 1140 that this is outside the scope of the present document and may be 1141 addressed in a future document defining another CDNI Logging Record 1142 or another version of the HTTP Request Logging Record. 1144 The "cdni_http_request_v1" record-type is also expected to be 1145 applicable to HTTP/2 [RFC7540] since a fundamental design tenet of 1146 HTTP/2 is to preserve the HTTP/1.1 semantics. We observe that, in 1147 the case of HTTP/2 delivery, there may be value in logging additional 1148 information specific to the additional functionality of HTTP/2 (e.g., 1149 information related to connection identification, to stream 1150 identification, to stream priority and to flow control). We note 1151 that such additional information is outside the scope of the present 1152 document and may be addressed in a future document defining another 1153 CDNI Logging Record or another version of the HTTP Request Logging 1154 Record. 1156 The "cdni_http_request_v1" record-type contains the following CDNI 1157 Logging fields, listed by their field name: 1159 o date: 1161 * format: DATE 1163 * field value: the date at which the processing of request 1164 completed on the Surrogate. 1166 * occurrence: there MUST be one and only one instance of this 1167 field. 1169 o time: 1171 * format: TIME 1173 * field value: the time, expressed in Coordinated Universal Time 1174 (UTC), at which the processing of request completed on the 1175 Surrogate. 1177 * occurrence: there MUST be one and only one instance of this 1178 field. 1180 o time-taken: 1182 * format: DEC 1184 * field value: decimal value of the duration, in seconds, between 1185 the start of the processing of the request and the completion 1186 of the request processing (e.g., completion of delivery) by the 1187 Surrogate. 1189 * occurrence: there MUST be one and only one instance of this 1190 field. 1192 o c-groupid: 1194 * format: NHTABSTRING 1196 * field value: an opaque identifier for an aggregate set of 1197 clients, derived from the client IPv4 or IPv6 address in the 1198 request received by the Surrogate and/or other network-level 1199 identifying information. The c-groupid serves to group clients 1200 into aggregates. Example aggregates include civil geolocation 1201 information (the country, second-level administrative division, 1202 or postal code from which the client is presumed to make the 1203 request based on a geolocation database lookup) or network 1204 topological information (e.g., the BGP AS number announcing the 1205 prefix containing the address). The c-groupid MAY be 1206 structured e.g., US/TN/MEM/38138. Agreement between the dCDN 1207 and the uCDN on a mapping between IPv4 and IPv6 addresses and 1208 aggregates is presumed to occur out-of-band. The aggregation 1209 mapping SHOULD be chosen such that each aggregate contains more 1210 than one client. 1212 + When the aggregate is chosen so that it contains a single 1213 client (e.g., to allow more detailed analytics, or to allow 1214 a-posteriori analysis of individual delivery for example in 1215 situations of performance-based penalties) the c-groupid MAY 1216 be structured where some elements identify aggregates and 1217 one element identifies the client, e.g., US/TN/ 1218 MEM/38138/43a5bdd6-95c4-4d62-be65-7410df0021e2. In that 1219 case: 1221 - the element identifying the client SHOULD be 1222 algorithmically generated (from the client IPv4 or IPv6 1223 address in the request received by the Surrogate and/or 1224 other network-level identifying information) in a way 1225 that SHOULD NOT be linkable back to the global addressing 1226 context and that SHOULD vary over time (to offer 1227 protection against long term attacks). 1229 - It is RECOMMENDED that the mapping varies at least once 1230 every 24 hours. 1232 - The algorithmic mapping and variation over time MAY allow 1233 the uCDN (with the knowledge of the algorithm and time 1234 variation and associated attributes and keys) to 1235 reconstruct the actual client IPv4 or IPv6 address and/or 1236 other network-level identifying information when required 1237 (e.g., to allow a-posteriori analysis of individual 1238 delivery for example in situations of performance-based 1239 penalties). However, these enduser addresses SHOULD only 1240 be reconstructed on-demand and the CDNI Logging File 1241 SHOULD only be stored with the anonymised c-groupid 1242 value. 1244 - Allowing reconstruction of client address information 1245 carries with it grave risks to end-user privacy. Since 1246 the c-groupid is in this case equivalent in 1247 identification power to a client IP address, its use may 1248 be restricted by regulation or law as personally 1249 identifiable information. For this reason, such use is 1250 NOT RECOMMENDED. 1252 - One method for mapping that MAY be be supported by 1253 implementations relies on a symmetric key that is known 1254 only to the uCDN and dCDN and HMAC-based Extract-and- 1255 Expand Key Derivation Function (HKDF) key derivation 1256 ([RFC5869]), as will be used in TLS 1.3 1257 ([I-D.ietf-tls-rfc5246-bis]). When that method is used: 1259 o The uCDN and dCDN need to agree on the "salt" and 1260 "input keying material", as described in Section 2.2 1261 of [RFC5869] and the initial "info" parameter (which 1262 could be something like the business names of the two 1263 organizations in UTF-8, concatenated), as described in 1264 Section 2.3 of [RFC5869]. The hash SHOULD be either 1265 SHA-2 or SHA-3 and the encryption algorithm SHOULD be 1266 128-bit AES-GCM or better. The PRK SHOULD be chosen 1267 by both parties contributing alternate random bytes 1268 until sufficient length exists. After the initial 1269 setup, client-information can be encrypted using the 1270 key generated by the "expand" step of Section 2.3 of 1271 [RFC5869]. The encrypted value SHOULD be hex or 1272 base64 encoded. At the agreed-upon expiration time, a 1273 new key SHOULD be generated and used. New keys SHOULD 1274 be indicated by prefixing the key with a special 1275 character such as exclamation point. In this way, 1276 shorter lifetimes can be used as needed. 1278 * occurrence: there MUST be one and only one instance of this 1279 field. 1281 o s-ip: 1283 * format: ADDRESS 1285 * field value: the IPv4 or IPv6 address of the Surrogate that 1286 served the request (i.e., the "server" address). 1288 * occurrence: there MUST be zero or exactly one instance of this 1289 field. 1291 o s-hostname: 1293 * format: host 1295 * field value: the hostname of the Surrogate that served the 1296 request (i.e., the "server" hostname). 1298 * occurrence: there MUST be zero or exactly one instance of this 1299 field. 1301 o s-port: 1303 * format: 1*DIGIT 1305 * field value: the destination TCP port (i.e., the "server" port) 1306 in the request received by the Surrogate. 1308 * occurrence: there MUST be zero or exactly one instance of this 1309 field. 1311 o cs-method: 1313 * format: NHTABSTRING 1315 * field value: this is the method of the request received by the 1316 Surrogate. In the case of HTTP delivery, this is the HTTP 1317 method in the request. 1319 * occurrence: There MUST be one and only one instance of this 1320 field. 1322 o cs-uri: 1324 * format: NHTABSTRING 1326 * field value: this is the "effective request URI" of the request 1327 received by the Surrogate as specified in [RFC7230]. It 1328 complies with the "http" URI scheme or the "https" URI scheme 1329 as specified in [RFC7230]). Note that cs-uri can be privacy 1330 sensitive. In that case, and where appropriate, u-uri could be 1331 used instead of cs-uri. 1333 * occurrence: there MUST be zero or exactly one instance of this 1334 field. 1336 o u-uri: 1338 * format: NHTABSTRING 1340 * field value: this is a complete URI, derived from the 1341 "effective request URI" ([RFC7230]) of the request received by 1342 the Surrogate (i.e., the cs-uri) but transformed by the entity 1343 generating or transmitting the CDNI Logging Record, in a way 1344 that is agreed upon between the two ends of the CDNI Logging 1345 interface, so the transformed URI is meaningful to the uCDN. 1347 For example, the two ends of the CDNI Logging interface could 1348 agree that the u-uri is constructed from the cs-uri by removing 1349 the part of the hostname that exposes which individual 1350 Surrogate actually performed the delivery. The details of 1351 modification performed to generate the u-uri, as well as the 1352 mechanism to agree on these modifications between the two sides 1353 of the CDNI Logging interface are outside the scope of the 1354 present document. 1356 * occurrence: there MUST be one and only one instance of this 1357 field. 1359 o protocol: 1361 * format: NHTABSTRING 1363 * field value: this is value of the HTTP-Version field as 1364 specified in [RFC7230] of the Request-Line of the request 1365 received by the Surrogate (e.g., "HTTP/1.1"). 1367 * occurrence: there MUST be one and only one instance of this 1368 field. 1370 o sc-status: 1372 * format: 3DIGIT 1374 * field value: this is the Status-Code in the response from the 1375 Surrogate. In the case of HTTP delivery, this is the HTTP 1376 Status-Code in the HTTP response. 1378 * occurrence: There MUST be one and only one instance of this 1379 field. 1381 o sc-total-bytes: 1383 * format: 1*DIGIT 1385 * field value: this is the total number of bytes of the response 1386 sent by the Surrogate in response to the request. In the case 1387 of HTTP delivery, this includes the bytes of the Status-Line, 1388 the bytes of the HTTP headers and the bytes of the message- 1389 body. 1391 * occurrence: There MUST be one and only one instance of this 1392 field. 1394 o sc-entity-bytes: 1396 * format: 1*DIGIT 1398 * field value: this is the number of bytes of the message-body in 1399 the HTTP response sent by the Surrogate in response to the 1400 request. This does not include the bytes of the Status-Line or 1401 the bytes of the HTTP headers. 1403 * occurrence: there MUST be zero or exactly one instance of this 1404 field. 1406 o cs(insert_HTTP_header_name_here): 1408 * format: QSTRING 1410 * field value: the value of the HTTP header (identified by the 1411 insert_HTTP_header_name_here in the CDNI Logging field name) as 1412 it appears in the request processed by the Surrogate, but 1413 prepended by a DQUOTE and appended by a DQUOTE. For example, 1414 when the CDNI Logging field name (FIENAME) listed in the 1415 preceding fields directive is cs(User-Agent), this CDNI Logging 1416 field value contains the value of the User-Agent HTTP header as 1417 received by the Surrogate in the request it processed, but 1418 prepended by a DQUOTE and appended by a DQUOTE. If the HTTP 1419 header as it appeared in the request processed by the Surrogate 1420 contains one or more DQUOTE, each DQUOTE MUST be escaped with 1421 percent encoding. For example, if the HTTP header contains 1422 My_Header"value", then the field value of the 1423 cs(insert_HTTP_header_name_here) is "My_Header%x22value%x22". 1424 The entity transmitting the CDNI Logging File MUST ensure that 1425 the respective insert_HTTP_header_name_here of the 1426 cs(insert_HTTP_header_name_here) listed in the fields directive 1427 comply with HTTP specifications. In particular, this field 1428 name does not include any HTAB, since this would prevent proper 1429 parsing of the fields directive by the entity receiving the 1430 CDNI Logging File. 1432 * occurrence: there MAY be zero, one or any number of instance of 1433 this field. 1435 o sc(insert_HTTP_header_name_here): 1437 * format: QSTRING 1439 * field value: the value of the HTTP header (identified by the 1440 insert_HTTP_header_name_here in the CDNI Logging field name) as 1441 it appears in the response issued by the Surrogate to serve the 1442 request, but prepended by a DQUOTE and appended by a DQUOTE. 1443 If the HTTP header as it appeared in the request processed by 1444 the Surrogate contains one or more DQUOTE, each DQUOTE MUST be 1445 escaped with percent encoding. For example, if the HTTP header 1446 contains My_Header"value", then the field value of the 1447 sc(insert_HTTP_header_name_here) is "My_Header%x22value%x22". 1448 The entity transmitting the CDNI Logging File MUST ensure that 1449 the respective insert_HTTP_header_name_here of the 1450 cs(insert_HTTP_header_name_here) listed in the fields directive 1451 comply with HTTP specifications. In particular, this field 1452 name does not include any HTAB, since this would prevent proper 1453 parsing of the fields directive by the entity receiving the 1454 CDNI Logging File. 1456 * occurrence: there MAY be zero, one or any number of instances 1457 of this field. For a given insert_HTTP_header_name_here, there 1458 MUST be zero or exactly one instance of this field. 1460 o s-ccid: 1462 * format: QSTRING 1464 * field value: this contains the value of the Content Collection 1465 IDentifier (CCID) associated by the uCDN to the content served 1466 by the Surrogate via the CDNI Metadata interface 1467 ([I-D.ietf-cdni-metadata]), prepended by a DQUOTE and appended 1468 by a DQUOTE. If the CCID conveyed in the CDNI Metadata 1469 interface contains one or more DQUOTE, each DQUOTE MUST be 1470 escaped with percent encoding. For example, if the CCID 1471 conveyed in the CDNI Metadata interface is My_CCIDD"value", 1472 then the field value of the s-ccid is "My_CCID%x22value%X22". 1474 * occurrence: there MUST be zero or exactly one instance of this 1475 field. For a given insert_HTTP_header_name_here, there MUST be 1476 zero or exactly one instance of this field. 1478 o s-sid: 1480 * format: QSTRING 1482 * field value: this contains the value of a Session IDentifier 1483 (SID) generated by the dCDN for a specific HTTP session, 1484 prepended by a DQUOTE and appended by a DQUOTE. In particular, 1485 for HTTP Adaptive Streaming (HAS) session, the Session 1486 IDentifier value is included in the Logging record for every 1487 content chunk delivery of that session in view of facilitating 1488 the later correlation of all the per content chunk log records 1489 of a given HAS session. See section 3.4.2.2. of [RFC6983] for 1490 more discussion on the concept of Session IDentifier in the 1491 context of HAS. If the SID conveyed contains one or more 1492 DQUOTE, each DQUOTE MUST be escaped with percent encoding. For 1493 example, if the SID is My_SID"value", then the field value of 1494 the s-sid is "My_SID%x22value%x22". 1496 * occurrence: there MUST be zero or exactly one instance of this 1497 field. 1499 o s-cached: 1501 * format: 1DIGIT 1503 * field value: this characterises whether the Surrogate served 1504 the request using content already stored on its local cache or 1505 not. The allowed values are "0" (for miss) and "1" (for hit). 1506 "1" MUST be used when the Surrogate did serve the request using 1507 exclusively content already stored on its local cache. "0" MUST 1508 be used otherwise (including cases where the Surrogate served 1509 the request using some, but not all, content already stored on 1510 its local cache). Note that a "0" only means a cache miss in 1511 the Surrogate and does not provide any information on whether 1512 the content was already stored, or not, in another device of 1513 the dCDN, i.e., whether this was a "dCDN hit" or "dCDN miss". 1515 * occurrence: there MUST be zero or exactly one instance of this 1516 field. 1518 The "fields" directive corresponding to a HTTP Request Logging Record 1519 MUST contain all the fields names whose occurrence is specified above 1520 as "There MUST be one and only one instance of this field". The 1521 corresponding fields value MUST be present in every HTTP Request 1522 Logging Record. 1524 The "fields" directive corresponding to a HTTP Request Logging Record 1525 MAY list all the fields value whose occurrence is specified above as 1526 "there MUST be zero or exactly one instance of this field" or "there 1527 MAY be zero, one or any number of instances of this field". The set 1528 of such field names actually listed in the "fields" directive is 1529 selected by the CDN generating the CDNI Logging File based on 1530 agreements between the interconnected CDNs established through 1531 mechanisms outside the scope of this specification (e.g., contractual 1532 agreements). When such a field name is not listed in the "fields" 1533 directive, the corresponding field value MUST NOT be included in the 1534 Logging Record. When such a field name is listed in the "fields" 1535 directive, the corresponding field value MUST be included in the 1536 Logging Record; if the value for the field is not available, this 1537 MUST be conveyed via a dash character ("-"). 1539 The fields names listed in the "fields" directive MAY be listed in 1540 the order in which they are listed in Section 3.4.1 or MAY be listed 1541 in any other order. 1543 Logging some specific fields from HTTP requests and responses can 1544 introduce serious security and privacy risks. For example, cookies 1545 will often contain (months) long lived token values that can be used 1546 to log into a service as the relevant user. Similar values may be 1547 included in other header fields or within URLs or elsewhere in HTTP 1548 requests and responses. Centralising such values in a CDNI Logging 1549 File can therefore represent a significant increase in risk both for 1550 the user and the web service provider, but also for the CDNs 1551 involved. Implementations ought therefore to attempt to lower the 1552 probability of such bad outcomes e.g. by only allowing a configured 1553 set of headers to be added to CDNI Logging Records, or by not 1554 supporting wildcard selection of HTTP request/response fields to add. 1555 Such mechanisms can reduce the probability that security (or privacy) 1556 sensitive values are centralised in CDNI Logging Files. Also, when 1557 agreeing on which HTTP request/response fields are to be provided in 1558 CDNI Logging Files, the uCDN and dCDN administrators ought to 1559 consider these risks. Furthermore, CDNs making use of c-groupid to 1560 identify an aggregate of clients rather than individual clients ought 1561 to realize that by logging certain header fields they may create the 1562 possibility to re-identify individual clients. In these cases 1563 heeding the above advice, or not logging header fields at all, is 1564 particularly important if the goal is to provide logs that do not 1565 identify individual clients." 1567 A dCDN-side implementation of the CDNI Logging interface MUST 1568 implement all the following Logging fields in a CDNI Logging Record 1569 of record-type "cdni_http_request_v1", and MUST support the ability 1570 to include valid values for each of them: 1572 o date 1574 o time 1576 o time-taken 1578 o c-groupid 1580 o s-ip 1582 o s-hostname 1584 o s-port 1586 o cs-method 1587 o cs-uri 1589 o u-uri 1591 o protocol 1593 o sc-status 1595 o sc-total-bytes 1597 o sc-entity-bytes 1599 o cs(insert_HTTP_header_name_here) 1601 o sc(insert_HTTP_header_name_here) 1603 o s-cached 1605 A dCDN-side implementation of the CDNI Logging interface MAY support 1606 the following Logging fields in a CDNI Logging Record of record-type 1607 "cdni_http_request_v1": 1609 o s-ccid 1611 o s-sid 1613 If a dCDN-side implementation of the CDNI Logging interface supports 1614 these fields, it MUST support the ability to include valid values for 1615 them. 1617 An uCDN-side implementation of the CDNI Logging interface MUST be 1618 able to accept CDNI Logging Files with CDNI Logging Records of 1619 record-type "cdni_http_request_v1" containing any CDNI Logging Field 1620 defined in Section 3.4.1 as long as the CDNI Logging Record and the 1621 CDNI Logging File are compliant with the present document. 1623 In case an uCDN-side implementation of the CDNI Logging interface 1624 receives a CDNI Logging File with HTTP Request Logging Records that 1625 do not contain field values for exactly the set of field names 1626 actually listed in the preceding "fields" directive, the 1627 implementation MUST reject those HTTP Request Logging Records, and 1628 MUST accept the other HTTP Request Logging Records. 1630 To ensure that the logging file is correct, the text MUST be 1631 sanitized before being logged. Null, bare CR, bare LF and HTAB have 1632 to be removed by escaping them through percent encoding to avoid 1633 confusion with the logging record separators. 1635 3.5. CDNI Logging File Extension 1637 The CDNI Logging File contains blocks of directives and blocks of 1638 corresponding records. The supported set of directives is defined 1639 relative to the CDNI Logging File Format version. The complete set 1640 of directives for version "CDNI/1.0" are defined in Section 3.3. The 1641 directive list is not expected to require much extension, but when it 1642 does, the new directive MUST be defined and registered in the "CDNI 1643 Logging Directive Names" registry, as described in Figure 8, and a 1644 new version MUST be defined and registered in the "CDNI Logging File 1645 version" registry, as described in Section 6.2. For example, adding 1646 a new CDNI Logging Directive, e.g., "foo", to the set of directives 1647 defined for "CDNI/1.0" in Section 3.3, would require registering both 1648 the new CDNI Logging Directive "foo" and a new CDNI Logging File 1649 version, e.g., "CDNI/2.0", which includes all of the existing CDNI 1650 Logging Directives of "CDNI/1.0" plus "foo". 1652 It is expected that as new logging requirements arise, the list of 1653 fields to log will change and expand. When adding new fields, the 1654 new fields MUST be defined and registered in the "CDNI Logging Field 1655 Names" registry, as described in Section 6.4, and a new record-type 1656 MUST be defined and registered in the "CDNI Logging record-types" 1657 registry, as described in Section 6.3. For example, adding a new 1658 CDNI Logging Field, e.g., "c-bar", to the set of fields defined for 1659 "cdni_http_request_v1" in Section 3.4.1, would require registering 1660 both the new CDNI Logging Field "c-bar" and a new CDNI record-type, 1661 e.g., "cdni_http_request_v2", which includes all of the existing CDNI 1662 Logging Fields of "cdni_http_request_v1" plus "c-bar". 1664 3.6. CDNI Logging File Example 1666 Let us consider the upstream CDN and the downstream CDN labelled uCDN 1667 and dCDN-1 in Figure 1. When dCDN-1 acts as a downstream CDN for 1668 uCDN and performs content delivery on behalf of uCDN, dCDN-1 will 1669 include the CDNI Logging Records corresponding to the content 1670 deliveries performed on behalf of uCDN in the CDNI Logging Files for 1671 uCDN. An example CDNI Logging File communicated by dCDN-1 to uCDN is 1672 shown below in Figure 4. 1674 #version:cdni/1.0 1676 #UUID:urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6 1678 #claimed-origin:cdni-logging-entity.dcdn-1.example.com 1680 #record-type:cdni_http_request_v1 1682 #fields:datetimetime-takenc-groupid 1683 cs-methodu-uriprotocol 1684 sc-statussc-total-bytescs(User-Agent) 1685 cs(Referer)s-cached 1687 2013-05-1700:38:06.8259.058US/TN/MEM/38138 1688 GET 1689 http://cdni-ucdn.dcdn-1.example.com/video/movie100.mp4 1690 HTTP/1.12006729891"Mozilla/5.0 1691 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1692 Gecko) Chrome/5.0.375.127 Safari/533.4" 1693 "host1.example.com"1 1695 2013-05-1700:39:09.14515.32FR/PACA/NCE/06100 1696 GET 1697 http://cdni-ucdn.dcdn-1.example.com/video/movie118.mp4 1698 HTTP/1.120015799210"Mozilla/5.0 1699 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1700 Gecko) Chrome/5.0.375.127 Safari/533.4" 1701 "host1.example.com"1 1703 2013-05-1700:42:53.43752.879US/TN/MEM/38138 1704 GET 1705 http://cdni-ucdn.dcdn-1.example.com/video/picture11.mp4 1706 HTTP/1.020097234724"Mozilla/5.0 1707 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1708 Gecko) Chrome/5.0.375.127 Safari/533.4" 1709 "host5.example.com"0 1711 #SHA256-hash: 64-hexadecimal-digit hash value 1713 Figure 4: CDNI Logging File Example 1715 If uCDN establishes by some means (e.g., via TLS authentication when 1716 pulling the CDNI Logging File) the identity of the entity from which 1717 it pulled the CDNI Logging File, uCDN can add to the CDNI Logging an 1718 established-origin directive as illustrated below: 1720 #established-origin:cdni-logging-entity.dcdn- 1721 1.example.com 1722 As illustrated in Figure 2, uCDN will then ingest the corresponding 1723 CDNI Logging Records into its Collection process, alongside the 1724 Logging Records generated locally by the uCDN itself. This allows 1725 uCDN to aggregate Logging Records for deliveries performed by itself 1726 (through Records generated locally) as well as for deliveries 1727 performed by its downstream CDN(s). This aggregate information can 1728 then be used (after Filtering and Rectification, as illustrated in 1729 Figure 2) by Log Consuming Applications that take into account 1730 deliveries performed by uCDN as well as by all of its downstream 1731 CDNs. 1733 We observe that the time between 1735 1. when a delivery is completed in dCDN and 1737 2. when the corresponding Logging Record is ingested by the 1738 Collection process in uCDN 1740 depends on a number of parameters such as the Logging Period agreed 1741 to by uCDN and dCDN, how much time uCDN waits before pulling the CDNI 1742 Logging File once it is advertised in the CDNI Logging Feed, and the 1743 time to complete the pull of the CDNI Logging File. Therefore, if we 1744 consider the set of Logging Records aggregated by the Collection 1745 process in uCDN in a given time interval, there could be a permanent 1746 significant timing difference between the CDNI Logging Records 1747 received from the dCDN and the Logging Records generated locally. 1748 For example, in a given time interval, the Collection process in uCDN 1749 may be aggregating Logging Records generated locally by uCDN for 1750 deliveries performed in the last hour and CDNI Logging Records 1751 generated in the dCDN for deliveries in the hour before last. 1753 3.7. Cascaded CDNI Logging Files Example 1755 Let us consider the cascaded CDN scenario of uCDN, dCDN-2 and dCDN-3 1756 as depicted in Figure 1. After completion of a delivery by dCDN-3 on 1757 behalf of dCDN-2, dCDN-3 will include a corresponding Logging Record 1758 in a CDNI Logging File that will be pulled by dCDN-2 and that is 1759 illustrated below in Figure 5. In practice, a CDNI Logging File is 1760 likely to contain a very high number of CDNI Logging Records. 1761 However, for readability, the example in Figure 5 contains a single 1762 CDNI Logging Record. 1764 #version:CDNI/1.0 1766 #UUID:urn:uuid:65718ef-0123-9876-adce4321bcde 1768 #claimed-origin:cdni-logging-entity.dcdn-3.example.com 1770 #record-type:cdni_http_request_v1 1772 #fields:datetimetime-takenc-groupid 1773 cs-methodu-uriprotocol 1774 sc-statussc-total-bytescs(User-Agent) 1775 cs(Referer)s-cached 1777 2013-05-1700:39:09.11914.07US/CA/SFO/94114 1778 GET 1779 http://cdni-dcdn-2.dcdn-3.example.com/video/movie118.mp4 1780 HTTP/1.120015799210"Mozilla/5.0 1781 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1782 Gecko) Chrome/5.0.375.127 Safari /533.4" 1783 "host1.example.com"1 1785 #SHA256-hash: 64-hexadecimal-digit hash value 1787 Figure 5: Cascaded CDNI Logging File Example (dCDN-3 to dCDN-2) 1789 If dCDN-2 establishes by some means (e.g., via TLS authentication 1790 when pulling the CDNI Logging File) the identity of the entity from 1791 which it pulled the CDNI Logging File, dCDN-2 can add to the CDNI 1792 Logging an established-origin directive as illustrated below: 1794 #established-origin:cdni-logging-entity.dcdn- 1795 3.example.com 1797 dCDN-2 (behaving as an upstream CDN from the viewpoint of dCDN-3) 1798 will then ingest the CDNI Logging Record for the considered dCDN-3 1799 delivery into its Collection process (as illustrated in Figure 2). 1800 This Logging Record may be aggregated with Logging Records generated 1801 locally by dCDN-2 for deliveries performed by dCDN-2 itself. Say, 1802 for illustration, that the content delivery performed by dCDN-3 on 1803 behalf of dCDN-2 had actually been redirected to dCDN-2 by uCDN, and 1804 say that another content delivery has just been redirected by uCDN to 1805 dCDN-2 and that dCDN-2 elected to perform the corresponding delivery 1806 itself. Then after Filtering and Rectification (as illustrated in 1807 Figure 2), dCDN-2 will include the two Logging Records corresponding 1808 respectively to the delivery performed by dCDN-3 and the delivery 1809 performed by dCDN-2, in the next CDNI Logging File that will be 1810 communicated to uCDN. An example of such CDNI Logging File is 1811 illustrated below in Figure 6. 1813 #version:CDNI/1.0 1815 #UUID:urn:uuid:1234567-8fedc-abab-0987654321ff 1817 #claimed-origin:cdni-logging-entity.dcdn-2.example.com 1819 #record-type:cdni_http_request_v1 1821 #fields:datetimetime-takenc-groupid 1822 cs-methodu-uriprotocol 1823 sc-statussc-total-bytescs(User-Agent) 1824 cs(Referer)s-cached 1826 2013-05-1700:39:09.11914.07US/CA/SFO/94114 1827 GET 1828 http://cdni-ucdn.dcdn-2.example.com/video/movie118.mp4 1829 HTTP/1.120015799210"Mozilla/5.0 1830 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1831 Gecko) Chrome/5.0.375.127 Safari /533.4" 1832 "host1.example.com"1 1834 2013-05-1701:42:53.43752.879FR/IDF/PAR/75001 1835 GET 1836 http://cdni-ucdn.dcdn-2.example.com/video/picture11.mp4 1837 HTTP/1.020097234724"Mozilla/5.0 1838 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1839 Gecko) Chrome/5.0.375.127 Safari /533.4" 1840 "host5.example.com"0 1842 #SHA256-hash: 64-hexadecimal-digit hash value 1844 Figure 6: Cascaded CDNI Logging File Example (dCDN-2 to uCDN) 1846 If uCDN establishes by some means (e.g., via TLS authentication when 1847 pulling the CDNI Logging File) the identity of the entity from which 1848 it pulled the CDNI Logging File, uCDN can add to the CDNI Logging an 1849 established-origin directive as illustrated below: 1851 #established-origin:cdni-logging-entity.dcdn- 1852 2.example.com 1854 In the example of Figure 6, we observe that: 1856 o the first Logging Record corresponds to the Logging Record 1857 communicated earlier to dCDN-2 by dCDN-3, which corresponds to a 1858 delivery redirected by uCDN to dCDN-2 and then redirected by 1859 dCDN-2 to dCDN-3. The fields values in this Logging Record are 1860 copied from the corresponding CDNI Logging REcord communicated to 1861 dCDN2 by dCDN-3, with the exception of the u-uri that now reflects 1862 the URI convention between uCDN and dCDN-2 and that presents the 1863 delivery to uCDN as if it was performed by dCDN-2 itself. This 1864 reflects the fact that dCDN-2 had taken the full responsibility of 1865 the corresponding delivery (even if in this case, dCDN-2 elected 1866 to redirect the delivery to dCDN-3 so it is actually performed by 1867 dCDN-3 on behalf of dCDN-2). 1869 o the second Logging Record corresponds to a delivery redirected by 1870 uCDN to dCDN-2 and performed by dCDN-2 itself. The time of the 1871 delivery in this Logging Record may be significantly more recent 1872 than the first Logging Record since it was generated locally while 1873 the first Logging Record was generated by dCDN-3 and had to be 1874 advertised , and then pulled and then ingested into the dCDN-2 1875 Collection process, before being aggregated with the second 1876 Logging Record. 1878 4. Protocol for Exchange of CDNI Logging File After Full Collection 1880 This section specifies a protocol for the exchange of CDNI Logging 1881 Files as specified in Section 3 after the CDNI Logging File is fully 1882 collected by the dCDN. 1884 This protocol comprises: 1886 o a CDNI Logging feed, allowing the dCDN to notify the uCDN about 1887 the CDNI Logging Files that can be retrieved by that uCDN from the 1888 dCDN, as well as all the information necessary for retrieving each 1889 of these CDNI Logging Files. The CDNI Logging feed is specified 1890 in Section 4.1. 1892 o a CDNI Logging File pull mechanism, allowing the uCDN to obtain 1893 from the dCDN a given CDNI Logging File at the uCDN's convenience. 1894 The CDNI Logging File pull mechanisms is specified in Section 4.2. 1896 An implementation of the CDNI Logging interface on the dCDN side (the 1897 entity generating the CDNI Logging file) MUST support the server side 1898 of the CDNI Logging feed (as specified in Section 4.1) and the server 1899 side of the CDNI Logging pull mechanism (as specified in 1900 Section 4.2). 1902 An implementation of the CDNI Logging interface on the uCDN side (the 1903 entity consuming the CDNI Logging file) MUST support the client side 1904 of the CDNI Logging feed (as specified in Section 4.1) and the client 1905 side of the CDNI Logging pull mechanism (as specified in 1906 Section 4.2). 1908 4.1. CDNI Logging Feed 1910 The server-side implementation of the CDNI Logging feed MUST produce 1911 an Atom feed [RFC4287]. This feed is used to advertise log files 1912 that are available for the client-side to retrieve using the CDNI 1913 Logging pull mechanism. 1915 4.1.1. Atom Formatting 1917 A CDNI Logging feed MUST be structured as an Archived feed, as 1918 defined in [RFC5005], and MUST be formatted in Atom [RFC4287]. This 1919 means it consists of a subscription document that is regularly 1920 updated as new CDNI Logging Files become available, and information 1921 about older CDNI Logging files is moved into archive documents. Once 1922 created, archive documents are never modified. 1924 Each CDNI Logging File listed in an Atom feed MUST be described in an 1925 atom:entry container element. 1927 The atom:entry MUST contain an atom:content element whose "src" 1928 attribute is a link to the CDNI Logging File and whose "type" 1929 attribute is the MIME Media Type indicating that the entry is a CDNI 1930 logging file. This MIME Media Type is defined as "application/cdni" 1931 (See [RFC7736]) with the Payload Type (ptype) parameter set to 1932 "logging-file". 1934 For compatibility with some Atom feed readers the atom:entry MAY also 1935 contain an atom:link entry whose "href" attribute is a link to the 1936 CDNI Logging File and whose "type" attribute is the MIME Media Type 1937 indicating that the entry is a CDNI Logging File using the 1938 "application/cdni" MIME Media Type with the Payload Type (ptype) 1939 parameter set to "logging-file"(See [RFC7736]). 1941 The URI used in the atom:id of the atom:entry MUST contain the UUID 1942 of the CDNI Logging File. 1944 The atom:updated in the atom:entry MUST indicate the time at which 1945 the CDNI Logging File was last updated. 1947 4.1.2. Updates to Log Files and the Feed 1949 CDNI Logging Files MUST NOT be modified by the dCDN once published in 1950 the CDNI Logging feed. 1952 The frequency with which the subscription feed is updated, the period 1953 of time covered by each CDNI Logging File or each archive document, 1954 and timeliness of publishing of CDNI Logging Files are outside the 1955 scope of the present document and are expected to be agreed upon by 1956 uCDN and dCDN via other means (e.g., human agreement). 1958 The server-side implementation MUST be able to set, and SHOULD set, 1959 HTTP cache control headers on the subscription feed to indicate the 1960 frequency at which the client-side is to poll for updates. 1962 The client-side MAY use HTTP cache control headers (set by the 1963 server-side) on the subscription feed to determine the frequency at 1964 which to poll for updates. The client-side MAY instead, or in 1965 addition, use other information to determine when to poll for updates 1966 (e.g., a polling frequency that may have been negotiated between the 1967 uCDN and dCDN by mechanisms outside the scope of the present document 1968 and that is to override the indications provided in the HTTP cache 1969 control headers). 1971 The potential retention limits (e.g., sliding time window) within 1972 which the dCDN is to retain and be ready to serve an archive document 1973 is outside the scope of the present document and is expected to be 1974 agreed upon by uCDN and dCDN via other means (e.g., human agreement). 1975 The server-side implementation MUST retain, and be ready to serve, 1976 any archive document within the agreed retention limits. Outside 1977 these agreed limits, the server-side implementation MAY indicate its 1978 inability to serve (e.g., with HTTP status code 404) an archive 1979 document or MAY refuse to serve it (e.g., with HTTP status code 403 1980 or 410). 1982 4.1.3. Redundant Feeds 1984 The server-side implementation MAY present more than one CDNI Logging 1985 feed for redundancy. Each CDNI Logging File MAY be published in more 1986 than one feed. 1988 A client-side implementation MAY support such redundant CDNI Logging 1989 feeds. If it supports redundant CDNI Logging feed, the client-side 1990 can use the UUID of the CDNI Logging File, presented in the atom:id 1991 element of the Atom feed, to avoid unnecessarily pulling and storing 1992 a given CDNI Logging File more than once. 1994 4.1.4. Example CDNI Logging Feed 1996 Figure 7 illustrates an example of the subscription document of a 1997 CDNI Logging feed. 1999 2000 2001 CDNI Logging Feed 2002 2013-03-23T14:46:11Z 2003 urn:uuid:663ae677-40fb-e99a-049d-c5642916b8ce 2004 2006 2008 2010 CDNI Log Feed 2011 Generator 2012 dcdn.example 2013 2014 CDNI Logging File for uCDN at 2015 2013-03-23 14:15:00 2016 urn:uuid:12345678-1234-abcd-00aa-01234567abcd 2017 2013-03-23T14:15:00Z 2018 2022 CDNI Logging File for uCDN at 2023 2013-03-23 14:15:00 2024 2025 2026 CDNI Logging File for uCDN at 2027 2013-03-23 14:30:00 2028 urn:uuid:87654321-4321-dcba-aa00-dcba7654321 2029 2013-03-23T14:30:00Z 2030 2034 CDNI Logging File for uCDN at 2035 2013-03-23 14:30:00 2036 2037 ... 2038 2039 ... 2040 2041 2043 Figure 7: Example subscription document of a CDNI Logging Feed 2045 4.2. CDNI Logging File Pull 2047 A client-side implementation of the CDNI Logging interface MAY pull, 2048 at its convenience, a CDNI Logging File that is published by the 2049 server-side in the CDNI Logging Feed (in the subscription document or 2050 an archive document). To do so, the client-side: 2052 o MUST implement HTTP/1.1 ([RFC7230],[RFC7231], [RFC7232], 2053 [RFC7233], [RFC7234], [RFC7235]), MAY also support other HTTP 2054 versions (e.g., HTTP/2 [RFC7540]) and MAY negotiate which HTTP 2055 version is actually used. This allows operators and implementers 2056 to choose to use later versions of HTTP to take advantage of new 2057 features, while still ensuring interoperability with systems that 2058 only support HTTP/1.1. 2060 o MUST use the URI that was associated to the CDNI Logging File 2061 (within the "src" attribute of the corresponding atom:content 2062 element) in the CDNI Logging Feed; 2064 o MUST support exchange of CDNI Logging Files with no content 2065 encoding applied to the representation; 2067 o MUST support exchange of CDNI Logging Files with "gzip" content 2068 encoding (as defined in [RFC7230]) applied to the representation. 2070 Note that a client-side implementation of the CDNI Logging interface 2071 MAY pull a CDNI Logging File that it has already pulled. 2073 The server-side implementation MUST respond to valid pull request by 2074 a client-side implementation for a CDNI Logging File published by the 2075 server-side in the CDNI Logging Feed (in the subscription document or 2076 an archive document). The server-side implementation: 2078 o MUST implement HTTP/1.1 to handle the client-side request and MAY 2079 also support other HTTP versions (e.g., HTTP/2); 2081 o MUST include the CDNI Logging File identified by the request URI 2082 inside the body of the HTTP response; 2084 o MUST support exchange of CDNI Logging Files with no content 2085 encoding applied to the representation; 2087 o MUST support exchange of CDNI Logging Files with "gzip" content 2088 encoding (as defined in [RFC7231]) applied to the representation. 2090 Content negotiation approaches defined in [RFC7231] (e.g., using 2091 Accept-Encoding request-header field or Content-Encoding entity- 2092 header field) MAY be used by the client-side and server-side 2093 implementations to establish the content-coding to be used for a 2094 particular exchange of a CDNI Logging File. 2096 Applying compression content encoding (such as "gzip") is expected to 2097 mitigate the impact of exchanging the large volumes of logging 2098 information expected across CDNs. This is expected to be 2099 particularly useful in the presence of HTTP Adaptive Streaming (HAS) 2100 which, as per the present version of the document, will result in a 2101 separate CDNI Log Record for each HAS segment delivery in the CDNI 2102 Logging File. 2104 The potential retention limits (e.g., sliding time window, maximum 2105 aggregate file storage quotas) within which the dCDN is to retain and 2106 be ready to serve a CDNI Logging File previously advertised in the 2107 CDNI Logging Feed is outside the scope of the present document and is 2108 expected to be agreed upon by uCDN and dCDN via other means (e.g., 2109 human agreement). The server-side implementation MUST retain, and be 2110 ready to serve, any CDNI Logging File within the agreed retention 2111 limits. Outside these agreed limits, the server-side implementation 2112 MAY indicate its inability to serve (e.g., with HTTP status code 404) 2113 a CDNI Logging File or MAY refuse to serve it (e.g., with HTTP status 2114 code 403 or 410). 2116 5. Protocol for Exchange of CDNI Logging File During Collection 2118 We note that, in addition to the CDNI Logging File exchange protocol 2119 specified in Section 4, implementations of the CDNI Logging interface 2120 may also support other mechanisms to exchange CDNI Logging Files. In 2121 particular, such mechanisms might allow the exchange of the CDNI 2122 Logging File to start before the file is fully collected. This can 2123 allow CDNI Logging Records to be communicated by the dCDN to the uCDN 2124 as they are gathered by the dCDN without having to wait until all the 2125 CDNI Logging Records of the same logging period are collected in the 2126 corresponding CDNI Logging File. This approach is commonly referred 2127 to as "tailing" of the file. 2129 Such an approach could be used, for example, to exchange logging 2130 information with a significantly reduced time-lag (e.g., sub-minute 2131 or sub-second) between when the event occurred in the dCDN and when 2132 the corresponding CDNI Logging Record is made available to the uCDN. 2133 This can satisfy log-consuming applications requiring extremely fresh 2134 logging information such as near-real-time content delivery 2135 monitoring. Such mechanisms are for further study and outside the 2136 scope of this document. 2138 6. IANA Considerations 2140 When IANA allocates new extensions to CDNI Logging Directive Names 2141 Registry, CDNI Logging File version Registry, CDNI Logging record- 2142 type Registry or CDNI Logging fields Registry, IANA MUST take into 2143 account that the directive names are case-insensitive as per the 2144 basic ABNF([RFC5234]). 2146 6.1. CDNI Logging Directive Names Registry 2148 The IANA is requested to create a new "CDNI Logging Directive Names" 2149 subregistry under the "Content Delivery Networks Interconnection 2150 (CDNI) Parameters" registry. 2152 The initial contents of the CDNI Logging Directives registry comprise 2153 the names of the directives specified in Section 3.3 of the present 2154 document, and are as follows: 2156 +------------------------------+-----------+ 2157 | Directive Name | Reference | 2158 +------------------------------+-----------+ 2159 | version | RFC xxxx | 2160 | UUID | RFC xxxx | 2161 | claimed-origin | RFC xxxx | 2162 | established-origin | RFC xxxx | 2163 | remark | RFC xxxx | 2164 | record-type | RFC xxxx | 2165 | fields | RFC xxxx | 2166 | SHA256-hash | RFC xxxx | 2167 +------------------------------+-----------+ 2169 Figure 8 2171 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2172 the present document] 2174 Within the registry, names are to be allocated by IANA according to 2175 the "Specification Required" policy specified in [RFC5226]. 2176 Directive names are to be allocated by IANA with a format of 2177 NAMEFORMAT (see Section 3.1). All directive names and field names 2178 defined in the logging file are case-insensitive as per the basic 2179 ABNF([RFC5234]). 2181 Each specification that defines a new CDNI Logging directive needs to 2182 contain a description for the new directive with the same set of 2183 information as provided in Section 3.3 (i.e., format, directive value 2184 and occurrence). 2186 6.2. CDNI Logging File version Registry 2188 The IANA is requested to create a new "CDNI Logging File version" 2189 subregistry under the "Content Delivery Networks Interconnection 2190 (CDNI) Parameters" registry. 2192 The initial contents of the CDNI Logging Logging File version 2193 registry comprise the value "CDNI/1.0" specified in Section 3.3 of 2194 the present document, and are as follows: 2196 +-----------------+-----------+----------------------------------+ 2197 | version | Reference | Description | 2198 +-----------------+-----------+----------------------------------+ 2199 | cdni/1.0 | RFC xxxx | CDNI Logging File version 1.0 | 2200 | | | as specified in RFC xxxx | 2201 +-----------------+-----------+----------------------------------+ 2203 Figure 9 2205 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2206 the present document] 2208 Within the registry, version values are to be allocated by IANA 2209 according to the "Specification Required" policy specified in 2210 [RFC5226]. Version values are to be allocated by IANA with a format 2211 of NAMEFORMAT (see Section 3.1). 2213 6.3. CDNI Logging record-types Registry 2215 The IANA is requested to create a new "CDNI Logging record-types" 2216 subregistry under the "Content Delivery Networks Interconnection 2217 (CDNI) Parameters" registry. 2219 The initial contents of the CDNI Logging record-types registry 2220 comprise the names of the CDNI Logging Record types specified in 2221 Section 3.4 of the present document, and are as follows: 2223 +----------------------+-----------+---------------------------------+ 2224 | record-types | Reference | Description | 2225 +----------------------+-----------+---------------------------------+ 2226 | cdni_http_request_v1 | RFC xxxx | CDNI Logging Record version 1 | 2227 | | | for content delivery using HTTP | 2228 +----------------------+-----------+---------------------------------+ 2230 Figure 10 2232 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2233 the present document] 2234 Within the registry, record-types are to be allocated by IANA 2235 according to the "Specification Required" policy specified in 2236 [RFC5226]. record-types are to be allocated by IANA with a format of 2237 NAMEFORMAT (see Section 3.1). 2239 Each specification that defines a new record-type needs to contain a 2240 description for the new record-type with the same set of information 2241 as provided in Section 3.4.1. This includes: 2243 o a list of all the CDNI Logging fields that can appear in a CDNI 2244 Logging Record of the new record-type 2246 o for all these fields: a specification of the occurrence for each 2247 Field in the new record-type 2249 o for every newly defined Field, i.e., for every Field that results 2250 in a registration in the CDNI Logging Field Names Registry 2251 (Section 6.4): a specification of the field name, format and field 2252 value. 2254 6.4. CDNI Logging Field Names Registry 2256 The IANA is requested to create a new "CDNI Logging Field Names" 2257 subregistry under the "Content Delivery Networks Interconnection 2258 (CDNI) Parameters" registry. 2260 This registry is intended to be shared across the currently defined 2261 record-type (i.e., cdni_http_request_v1) as well as potential other 2262 CDNI Logging record-types that may be defined in separate 2263 specifications. When a Field from this registry is used by another 2264 CDNI Logging record-type, it is to be used with the exact semantics 2265 and format specified in the document that registered this field and 2266 that is identified in the Reference column of the registry. If 2267 another CDNI Logging record-type requires a Field with semantics that 2268 are not strictly identical, or a format that is not strictly 2269 identical then this new Field is to be registered in the registry 2270 with a different Field name. When a Field from this registry is used 2271 by another CDNI Logging record-type, it can be used with different 2272 occurrence rules. 2274 The initial contents of the CDNI Logging fields Names registry 2275 comprise the names of the CDNI Logging fields specified in 2276 Section 3.4 of the present document, and are as follows: 2278 +------------------------------------------+-----------+ 2279 | Field Name | Reference | 2280 +------------------------------------------+-----------+ 2281 | date | RFC xxxx | 2282 | time | RFC xxxx | 2283 | time-taken | RFC xxxx | 2284 | c-groupid | RFC xxxx | 2285 | s-ip | RFC xxxx | 2286 | s-hostname | RFC xxxx | 2287 | s-port | RFC xxxx | 2288 | cs-method | RFC xxxx | 2289 | cs-uri | RFC xxxx | 2290 | u-uri | RFC xxxx | 2291 | protocol | RFC xxxx | 2292 | sc-status | RFC xxxx | 2293 | sc-total-bytes | RFC xxxx | 2294 | sc-entity-bytes | RFC xxxx | 2295 | cs(insert_HTTP_header_name_here) | RFC xxxx | 2296 | sc(insert_HTTP_header_name_here) | RFC xxxx | 2297 | s-ccid | RFC xxxx | 2298 | s-sid | RFC xxxx | 2299 | s-cached | RFC xxxx | 2300 +------------------------------------------+-----------+ 2302 Figure 11 2304 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2305 the present document] 2307 Within the registry, names are to be allocated by IANA according to 2308 the "Specification Required" policy specified in [RFC5226]. Field 2309 names are to be allocated by IANA with a format of NHTABSTRING (see 2310 Section 3.1). 2312 6.5. CDNI Logging MIME Media Type 2314 The IANA is requested to register the following new Payload Type in 2315 the CDNI Payload Type registry for use with the application/cdni MIME 2316 media type. 2318 [RFC Editor Note: Please replace the references to [RFCthis] below 2319 with this document's RFC number before publication.] 2320 +----------------------+---------------+ 2321 | Payload Type | Specification | 2322 +----------------------+---------------+ 2323 | logging-file | [RFCthis] | 2324 +----------------------+---------------+ 2326 Figure 12: MIME Media Type payload 2328 The purpose of the logging-file payload type is to distinguish 2329 between CDNI Logging Files and other CDNI messages. 2331 Interface: LI 2333 Encoding: see Section 3.2, Section 3.3 and Section 3.4 2335 7. Security Considerations 2337 7.1. Authentication, Authorization, Confidentiality, Integrity 2338 Protection 2340 An implementation of the CDNI Logging interface MUST support TLS 2341 transport of the CDNI Logging feed (Section 4.1) and of the CDNI 2342 Logging File pull (Section 4.2) as per [RFC2818] and [RFC7230]. 2344 The use of TLS for transport of the CDNI Logging feed and CDNI 2345 Logging File pull allows: 2347 o the dCDN and uCDN to authenticate each other 2349 and, once they have mutually authenticated each other, it allows: 2351 o the dCDN and uCDN to authorize each other (to ensure they are 2352 transmitting/receiving CDNI Logging File to/from an authorized 2353 CDN) 2355 o the CDNI Logging information to be transmitted with 2356 confidentiality 2358 o the integrity of the CDNI Logging information to be protected 2359 during the exchange. 2361 In an environment where any such protection is required, mutually 2362 authenticated encrypted transport MUST be used to ensure 2363 confidentiality of the logging information, and to do so, TLS MUST be 2364 used (including authentication of the remote end) by the server-side 2365 and the client-side of the CDNI Logging feed, as well as the server- 2366 side and the client-side of the CDNI Logging File pull mechanism. 2368 When TLS is used, the general TLS usage guidance in [RFC7525] MUST be 2369 followed. 2371 The SHA256-hash directive inside the CDNI Logging File provides 2372 additional integrity protection, this time targeting potential 2373 corruption of the CDNI logging information during the CDNI Logging 2374 File generation, storage or exchange. This mechanism does not itself 2375 allow restoration of the corrupted CDNI Logging information, but it 2376 allows detection of such corruption and therefore triggering of 2377 appropriate corrective actions (e.g., discard of corrupted 2378 information, attempt to re-obtain the CDNI Logging information). 2379 Note that the SHA256-hash does not protect against tampering by a 2380 third party, since such a third party could have recomputed and 2381 updated the SHA256-hash after tampering. Protection against third 2382 party tampering can be achieved as discussed above through the use of 2383 TLS. 2385 7.2. Denial of Service 2387 This document does not define specific mechanism to protect against 2388 Denial of Service (DoS) attacks on the Logging Interface. However, 2389 the CDNI Logging feed and CDNI Logging pull endpoints are typically 2390 to be accessed only by a very small number of valid remote endpoints 2391 and therefore can be easily protected against DoS attacks through the 2392 usual conventional DOS protection mechanisms such as firewalling or 2393 use of Virtual Private Networks (VPNs). 2395 Protection of dCDN Surrogates against spoofed delivery requests is 2396 outside the scope of the CDNI Logging interface. 2398 7.3. Privacy 2400 CDNs have the opportunity to collect detailed information about the 2401 downloads performed by End Users. A dCDN is expected to collect such 2402 information into CDNI Logging Files, which are then communicated to 2403 an uCDN. 2405 Having detailed CDNI logging information known by the dCDN in itself 2406 does not represent a particular privacy concern since the dCDN is 2407 obviously fully aware of all information logged since it generated 2408 the information in the first place. Making detailed CDNI logging 2409 information known to the uCDN does not represent a particular privacy 2410 concern because the uCDN is already exposed at request redirection 2411 time to most of the information that shows up as CDNI logging 2412 information (e.g., enduser IP@, URL, HTTP headers - at least when 2413 HTTP redirection is used between uCDN and dCDN). Transporting 2414 detailed CDNI logging information over the HTTP based CDNI Logging 2415 Interface does not represent a particular privacy concern because it 2416 is protected by usual IETF privacy-protection mechanism (e.g.,TLS). 2418 However, one privacy concern arises from the fact that large volumes 2419 of detailed information about content delivery to users, potentially 2420 traceable back to indvidual users, may be collected in CDNI Logging 2421 files. These CDNI Logging files represent high-value targets, likely 2422 concentrated in a fairly centralised system (although the CDNI 2423 Logging architecture does not mandate a particular level of 2424 centralisation/distribution) and at risk of potential data 2425 exfiltration. Note that the means of such data exfiltration are 2426 beyond the scope of the CDNI Logging interface itself (e.g., 2427 corrupted employee, corrupted logging storage system,...). This 2428 privacy concern calls for some protection. 2430 The collection of large volumes of such information into CDNI Logging 2431 Files introduces potential End Users privacy protection concerns. 2432 Mechanisms to address these concerns are discussed in the definition 2433 of the c-groupid field specified in Section 3.4.1. 2435 The use of mutually authenticated TLS to establish a secure session 2436 for the transport of the CDNI Logging feed and CDNI Logging pull as 2437 discussed in Section 7.1 provides confidentiality while the logging 2438 information is in transit and prevents any party other than the 2439 authorised uCDN to gain access to the logging information. 2441 We also note that the query string portion of the URL that may be 2442 conveyed inside the cs-uri and u-uri fields of CDNI Logging Files, or 2443 the HTTP cookies( [RFC6265]) that may be conveyed as part of the 2444 cs() field of CDNI Logging files, may contain 2445 personnal information or information that can be exploited to derive 2446 personal information. Where this is a concern, the CDNI Logging 2447 interface specification allows the dCDN to not include the cs-uri and 2448 to include a u-uri that removes (or hides) the sensitive part of the 2449 query string and allows the dCDN to not include the cs() fields corresponding to HTTP headers associated with cookies. 2452 8. Acknowledgments 2454 This document borrows from the W3C Extended Log Format [ELF]. 2456 Rob Murray significantly contributed into the text of Section 4.1. 2458 The authors thank Ben Niven-Jenkins, Kevin Ma, David Mandelberg and 2459 Ray van Brandenburg for their ongoing input. 2461 Brian Trammel and Rich Salz made significant contributions into 2462 making this interface privacy-friendly. 2464 Finally, we also thank Sebastien Cubaud, Pawel Grochocki, Christian 2465 Jacquenet, Yannick Le Louedec, Anne Marrec, Emile Stephan, Fabio 2466 Costa, Sara Oueslati, Yvan Massot, Renaud Edel, Joel Favier and the 2467 contributors of the EU FP7 OCEAN project for their input in the early 2468 versions of this document. 2470 9. References 2472 9.1. Normative References 2474 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2475 Requirement Levels", BCP 14, RFC 2119, 2476 DOI 10.17487/RFC2119, March 1997, 2477 . 2479 [RFC3339] Klyne, G. and C. Newman, "Date and Time on the Internet: 2480 Timestamps", RFC 3339, DOI 10.17487/RFC3339, July 2002, 2481 . 2483 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 2484 Resource Identifier (URI): Generic Syntax", STD 66, 2485 RFC 3986, DOI 10.17487/RFC3986, January 2005, 2486 . 2488 [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally 2489 Unique IDentifier (UUID) URN Namespace", RFC 4122, 2490 DOI 10.17487/RFC4122, July 2005, 2491 . 2493 [RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom 2494 Syndication Format", RFC 4287, DOI 10.17487/RFC4287, 2495 December 2005, . 2497 [RFC5005] Nottingham, M., "Feed Paging and Archiving", RFC 5005, 2498 DOI 10.17487/RFC5005, September 2007, 2499 . 2501 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 2502 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 2503 DOI 10.17487/RFC5226, May 2008, 2504 . 2506 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 2507 Specifications: ABNF", STD 68, RFC 5234, 2508 DOI 10.17487/RFC5234, January 2008, 2509 . 2511 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2512 Protocol (HTTP/1.1): Message Syntax and Routing", 2513 RFC 7230, DOI 10.17487/RFC7230, June 2014, 2514 . 2516 [RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2517 Protocol (HTTP/1.1): Semantics and Content", RFC 7231, 2518 DOI 10.17487/RFC7231, June 2014, 2519 . 2521 [RFC7232] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2522 Protocol (HTTP/1.1): Conditional Requests", RFC 7232, 2523 DOI 10.17487/RFC7232, June 2014, 2524 . 2526 [RFC7233] Fielding, R., Ed., Lafon, Y., Ed., and J. Reschke, Ed., 2527 "Hypertext Transfer Protocol (HTTP/1.1): Range Requests", 2528 RFC 7233, DOI 10.17487/RFC7233, June 2014, 2529 . 2531 [RFC7234] Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, 2532 Ed., "Hypertext Transfer Protocol (HTTP/1.1): Caching", 2533 RFC 7234, DOI 10.17487/RFC7234, June 2014, 2534 . 2536 [RFC7235] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 2537 Protocol (HTTP/1.1): Authentication", RFC 7235, 2538 DOI 10.17487/RFC7235, June 2014, 2539 . 2541 [RFC7525] Sheffer, Y., Holz, R., and P. Saint-Andre, 2542 "Recommendations for Secure Use of Transport Layer 2543 Security (TLS) and Datagram Transport Layer Security 2544 (DTLS)", BCP 195, RFC 7525, DOI 10.17487/RFC7525, May 2545 2015, . 2547 [RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext 2548 Transfer Protocol Version 2 (HTTP/2)", RFC 7540, 2549 DOI 10.17487/RFC7540, May 2015, 2550 . 2552 9.2. Informative References 2554 [CHAR_SET] 2555 "IANA Character Sets registry", 2556 . 2559 [ELF] Phillip M. Hallam-Baker, and Brian Behlendorf, "Extended 2560 Log File Format, W3C (work in progress), WD-logfile- 2561 960323", . 2563 [I-D.ietf-cdni-metadata] 2564 Niven-Jenkins, B., Murray, R., Caulfield, M., and K. Ma, 2565 "CDN Interconnection Metadata", draft-ietf-cdni- 2566 metadata-13 (work in progress), March 2016. 2568 [I-D.ietf-tls-rfc5246-bis] 2569 Dierks, T. and E. Rescorla, "The Transport Layer Security 2570 (TLS) Protocol Version 1.3", draft-ietf-tls-rfc5246-bis-00 2571 (work in progress), April 2014. 2573 [I-D.snell-atompub-link-extensions] 2574 Snell, J., "Atom Link Extensions", draft-snell-atompub- 2575 link-extensions-09 (work in progress), June 2012. 2577 [RFC1945] Berners-Lee, T., Fielding, R., and H. Frystyk, "Hypertext 2578 Transfer Protocol -- HTTP/1.0", RFC 1945, 2579 DOI 10.17487/RFC1945, May 1996, 2580 . 2582 [RFC2818] Rescorla, E., "HTTP Over TLS", RFC 2818, 2583 DOI 10.17487/RFC2818, May 2000, 2584 . 2586 [RFC5869] Krawczyk, H. and P. Eronen, "HMAC-based Extract-and-Expand 2587 Key Derivation Function (HKDF)", RFC 5869, 2588 DOI 10.17487/RFC5869, May 2010, 2589 . 2591 [RFC6234] Eastlake 3rd, D. and T. Hansen, "US Secure Hash Algorithms 2592 (SHA and SHA-based HMAC and HKDF)", RFC 6234, 2593 DOI 10.17487/RFC6234, May 2011, 2594 . 2596 [RFC6265] Barth, A., "HTTP State Management Mechanism", RFC 6265, 2597 DOI 10.17487/RFC6265, April 2011, 2598 . 2600 [RFC6707] Niven-Jenkins, B., Le Faucheur, F., and N. Bitar, "Content 2601 Distribution Network Interconnection (CDNI) Problem 2602 Statement", RFC 6707, DOI 10.17487/RFC6707, September 2603 2012, . 2605 [RFC6770] Bertrand, G., Ed., Stephan, E., Burbridge, T., Eardley, 2606 P., Ma, K., and G. Watson, "Use Cases for Content Delivery 2607 Network Interconnection", RFC 6770, DOI 10.17487/RFC6770, 2608 November 2012, . 2610 [RFC6983] van Brandenburg, R., van Deventer, O., Le Faucheur, F., 2611 and K. Leung, "Models for HTTP-Adaptive-Streaming-Aware 2612 Content Distribution Network Interconnection (CDNI)", 2613 RFC 6983, DOI 10.17487/RFC6983, July 2013, 2614 . 2616 [RFC7336] Peterson, L., Davie, B., and R. van Brandenburg, Ed., 2617 "Framework for Content Distribution Network 2618 Interconnection (CDNI)", RFC 7336, DOI 10.17487/RFC7336, 2619 August 2014, . 2621 [RFC7337] Leung, K., Ed. and Y. Lee, Ed., "Content Distribution 2622 Network Interconnection (CDNI) Requirements", RFC 7337, 2623 DOI 10.17487/RFC7337, August 2014, 2624 . 2626 [RFC7736] Ma, K., "Content Delivery Network Interconnection (CDNI) 2627 Media Type Registration", RFC 7736, DOI 10.17487/RFC7736, 2628 December 2015, . 2630 Authors' Addresses 2632 Francois Le Faucheur (editor) 2633 Cisco Systems 2634 E.Space Park - Batiment D 2635 6254 Allee des Ormes - BP 1200 2636 Mougins cedex 06254 2637 FR 2639 Phone: +33 4 97 23 26 19 2640 Email: flefauch@cisco.com 2642 Gilles Bertrand (editor) 2643 Orange 2644 38-40 rue du General Leclerc 2645 Issy les Moulineaux 92130 2646 FR 2648 Phone: +33 1 45 29 89 46 2649 Email: gilles.bertrand@orange.com 2650 Iuniana Oprescu (editor) 2651 FR 2653 Email: iuniana.oprescu@gmail.com 2655 Roy Peterkofsky 2656 Google Inc. 2657 345 Spear St, 4th Floor 2658 San Francisco CA 94105 2659 USA 2661 Email: peterkofsky@google.com