idnits 2.17.1 draft-ietf-cdni-logging-14.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 25, 2014) is 3464 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) ** Obsolete normative reference: RFC 7231 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7232 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7233 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7234 (Obsoleted by RFC 9111) ** Obsolete normative reference: RFC 7235 (Obsoleted by RFC 9110) == Outdated reference: A later version (-21) exists of draft-ietf-cdni-metadata-07 == Outdated reference: A later version (-17) exists of draft-ietf-httpbis-http2-14 -- Obsolete informational reference (is this intentional?): RFC 2818 (Obsoleted by RFC 9110) Summary: 7 errors (**), 0 flaws (~~), 3 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force F. Le Faucheur, Ed. 3 Internet-Draft Cisco Systems 4 Intended status: Standards Track G. Bertrand, Ed. 5 Expires: April 28, 2015 I. Oprescu, Ed. 6 Orange 7 R. Peterkofsky 8 Skytide, Inc. 9 October 25, 2014 11 CDNI Logging Interface 12 draft-ietf-cdni-logging-14 14 Abstract 16 This memo specifies the Logging interface between a downstream CDN 17 (dCDN) and an upstream CDN (uCDN) that are interconnected as per the 18 CDN Interconnection (CDNI) framework. First, it describes a 19 reference model for CDNI logging. Then, it specifies the CDNI 20 Logging File format and the actual protocol for exchange of CDNI 21 Logging Files. 23 Status of This Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at http://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on April 28, 2015. 40 Copyright Notice 42 Copyright (c) 2014 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (http://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 58 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 59 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 4 60 2. CDNI Logging Reference Model . . . . . . . . . . . . . . . . 5 61 2.1. CDNI Logging interactions . . . . . . . . . . . . . . . . 5 62 2.2. Overall Logging Chain . . . . . . . . . . . . . . . . . . 8 63 2.2.1. Logging Generation and During-Generation Aggregation 9 64 2.2.2. Logging Collection . . . . . . . . . . . . . . . . . 10 65 2.2.3. Logging Filtering . . . . . . . . . . . . . . . . . . 10 66 2.2.4. Logging Rectification and Post-Generation Aggregation 11 67 2.2.5. Log-Consuming Applications . . . . . . . . . . . . . 12 68 2.2.5.1. Maintenance/Debugging . . . . . . . . . . . . . . 12 69 2.2.5.2. Accounting . . . . . . . . . . . . . . . . . . . 12 70 2.2.5.3. Analytics and Reporting . . . . . . . . . . . . . 12 71 2.2.5.4. Security . . . . . . . . . . . . . . . . . . . . 13 72 2.2.5.5. Legal Logging Duties . . . . . . . . . . . . . . 13 73 2.2.5.6. Notions common to multiple Log Consuming 74 Applications . . . . . . . . . . . . . . . . . . 13 75 3. CDNI Logging File . . . . . . . . . . . . . . . . . . . . . . 15 76 3.1. Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 15 77 3.2. CDNI Logging File Structure . . . . . . . . . . . . . . . 16 78 3.3. CDNI Logging File Directives . . . . . . . . . . . . . . 19 79 3.4. CDNI Logging Records . . . . . . . . . . . . . . . . . . 23 80 3.4.1. HTTP Request Logging Record . . . . . . . . . . . . . 23 81 3.5. CDNI Logging File Example . . . . . . . . . . . . . . . . 32 82 3.6. Cascaded CDNI Logging Files Example . . . . . . . . . . . 34 83 4. Protocol for Exchange of CDNI Logging File After Full 84 Collection . . . . . . . . . . . . . . . . . . . . . . . . . 37 85 4.1. CDNI Logging Feed . . . . . . . . . . . . . . . . . . . . 38 86 4.1.1. Atom Formatting . . . . . . . . . . . . . . . . . . . 38 87 4.1.2. Updates to Log Files and the Feed . . . . . . . . . . 38 88 4.1.3. Redundant Feeds . . . . . . . . . . . . . . . . . . . 39 89 4.1.4. Example CDNI Logging Feed . . . . . . . . . . . . . . 39 90 4.2. CDNI Logging File Pull . . . . . . . . . . . . . . . . . 41 91 5. Protocol for Exchange of CDNI Logging File During Collection 42 92 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 43 93 6.1. CDNI Logging Directive Names Registry . . . . . . . . . . 43 94 6.2. CDNI Logging Record-Types Registry . . . . . . . . . . . 43 95 6.3. CDNI Logging Field Names Registry . . . . . . . . . . . . 44 96 6.4. CDNI Logging MIME Media Type . . . . . . . . . . . . . . 46 98 7. Security Considerations . . . . . . . . . . . . . . . . . . . 46 99 7.1. Authentication, Confidentiality, Integrity Protection . . 46 100 7.2. Denial of Service . . . . . . . . . . . . . . . . . . . . 47 101 7.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 47 102 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 48 103 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 48 104 9.1. Normative References . . . . . . . . . . . . . . . . . . 48 105 9.2. Informative References . . . . . . . . . . . . . . . . . 49 106 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 51 108 1. Introduction 110 This memo specifies the CDNI Logging interface between a downstream 111 CDN (dCDN) and an upstream CDN (uCDN). First, it describes a 112 reference model for CDNI logging. Then, it specifies the CDNI 113 Logging File format and the actual protocol for exchange of CDNI 114 Logging Files. 116 The reader should be familiar with the following documents: 118 o CDNI problem statement [RFC6707] and framework [RFC7336] identify 119 a Logging interface, 121 o Section 8 of [RFC7337] specifies a set of requirements for 122 Logging, 124 o [RFC6770] outlines real world use-cases for interconnecting CDNs. 125 These use cases require the exchange of Logging information 126 between the dCDN and the uCDN. 128 As stated in [RFC6707], "the CDNI Logging interface enables details 129 of logs or events to be exchanged between interconnected CDNs". 131 The present document describes: 133 o The CDNI Logging reference model (Section 2), 135 o The CDNI Logging File format (Section 3), 137 o The CDNI Logging File Exchange protocol (Section 4). 139 1.1. Terminology 141 In this document, the first letter of each CDNI-specific term is 142 capitalized. We adopt the terminology described in [RFC6707] and 143 [RFC7336], and extend it with the additional terms defined below. 145 Intra-CDN Logging information: logging information generated and 146 collected within a CDN. The format of the Intra-CDN Logging 147 information may be different to the format of the CDNI Logging 148 information. 150 CDNI Logging information: logging information exchanged across CDNs 151 using the CDNI Logging Interface. 153 Logging information: logging information generated and collected 154 within a CDN or obtained from another CDN using the CDNI Logging 155 Interface. 157 CDNI Logging Field: an atomic element of information that can be 158 included in a CDNI Logging Record. The time an event/task started, 159 the IP address of an End User to whom content was delivered, and the 160 Uniform Resource Identifier (URI) of the content delivered, are 161 examples of CDNI Logging Fields. 163 CDNI Logging Record: an information record providing information 164 about a specific event. This comprises a collection of CDNI Logging 165 Fields. 167 CDNI Logging File: a file containing CDNI Logging Records, as well as 168 additional information facilitating the processing of the CDNI 169 Logging Records. 171 CDN Reporting: the process of providing the relevant information that 172 will be used to create a formatted content delivery report provided 173 to the CSP in deferred time. Such information typically includes 174 aggregated data that can cover a large period of time (e.g., from 175 hours to several months). Uses of Reporting include the collection 176 of charging data related to CDN services and the computation of Key 177 Performance Indicators (KPIs). 179 CDN Monitoring: the process of providing or displaying content 180 delivery information in a timely fashion with respect to the 181 corresponding deliveries. Monitoring typically includes visibility 182 of the deliveries in progress for service operation purposes. It 183 presents a view of the global health of the services as well as 184 information on usage and performance, for network services 185 supervision and operation management. In particular, monitoring data 186 can be used to generate alarms. 188 1.2. Requirements Language 190 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 191 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 192 document are to be interpreted as described in RFC 2119 [RFC2119]. 194 2. CDNI Logging Reference Model 196 2.1. CDNI Logging interactions 198 The CDNI logging reference model between a given uCDN and a given 199 dCDN involves the following interactions: 201 o customization by the uCDN of the CDNI Logging information to be 202 provided by the dCDN to the uCDN (e.g., control of which CDNI 203 Logging Fields are to be communicated to the uCDN for a given task 204 performed by the dCDN or control of which types of events are to 205 be logged). The dCDN takes into account this CDNI Logging 206 customization information to determine what Logging information to 207 provide to the uCDN, but it may, or may not, take into account 208 this CDNI Logging customization information to influence what CDN 209 logging information is to be generated and collected within the 210 dCDN (e.g., even if the uCDN requests a restricted subset of the 211 logging information, the dCDN may elect to generate a broader set 212 of logging information). The mechanism to support the 213 customization by the uCDN of CDNI Logging information is outside 214 the scope of this document and left for further study. Until such 215 a mechanism is available, the uCDN and dCDN are expected to agree 216 off-line on what exact set of CDNI Logging information is to be 217 provided by the dCDN to the uCDN, and to rely on management plane 218 actions to configure the CDNI Logging functions in the dCDN to 219 generate this information set and in the uCDN to expect this 220 information set. 222 o generation and collection by the dCDN of the intra-CDN Logging 223 information related to the completion of any task performed by the 224 dCDN on behalf of the uCDN (e.g., delivery of the content to an 225 End User) or related to events happening in the dCDN that are 226 relevant to the uCDN (e.g., failures or unavailability in dCDN). 227 This takes place within the dCDN and does not directly involve 228 CDNI interfaces. 230 o communication by the dCDN to the uCDN of the Logging information 231 collected by the dCDN relevant to the uCDN. This is supported by 232 the CDNI Logging interface and in the scope of the present 233 document. For example, the uCDN may use this Logging information 234 to charge the CSP, to perform analytics and monitoring for 235 operational reasons, to provide analytics and monitoring views on 236 its content delivery to the CSP or to perform trouble-shooting. 237 This document exclusively specifies non-real-time exchange of 238 Logging information. Closer to real-time exchange of Logging 239 information (say sub-minute or sub-second) is outside the scope of 240 the present document and left for further study. This document 241 exclusively specifies exchange of Logging information related to 242 content delivery. Exchange of Logging information related to 243 operational events (e.g., dCDN request routing function 244 unavailable, content acquisition failure by dCDN) for audit or 245 operational reactive adjustments by uCDN is outside the scope of 246 the present document and left for further study. 248 o customization by the dCDN of the CDNI Logging information to be 249 provided by the uCDN on behalf of the dCDN. The mechanism to 250 support the customization by the dCDN of CDNI Logging information 251 is outside the scope of this document and left for further study. 253 o generation and collection by the uCDN of Intra-CDN Logging 254 information related to the completion of any task performed by the 255 uCDN on behalf of the dCDN (e.g., serving of content by uCDN to 256 dCDN for acquisition purposes by dCDN) or related to events 257 happening in the uCDN that are relevant to the dCDN. This takes 258 place within the uCDN and does not directly involve CDNI 259 interfaces. 261 o communication by the uCDN to the dCDN of the Logging information 262 collected by the uCDN relevant to the dCDN. For example, the dCDN 263 might potentially benefit from this information for security 264 auditing or content acquisition troubleshooting. This is outside 265 the scope of this document and left for further study. 267 Figure 1 provides an example of CDNI Logging interactions (focusing 268 only on the interactions that are in the scope of this document) in a 269 particular scenario where four CDNs are involved in the delivery of 270 content from a given CSP: the uCDN has a CDNI interconnection with 271 dCDN-1 and dCDN-2. In turn, dCDN2 has a CDNI interconnection with 272 dCDN3. In this example, uCDN, dCDN-1, dCDN-2 and dCDN-3 all 273 participate in the delivery of content for the CSP. In this example, 274 the CDNI Logging interface enables the uCDN to obtain Logging 275 information from all the dCDNs involved in the delivery. In the 276 example, the uCDN uses the Logging information: 278 o to analyze the performance of the delivery performed by the dCDNs 279 and to adjust its operations after the fact (e.g., request 280 routing) as appropriate, 282 o to provide (non-real-time) reporting and monitoring information to 283 the CSP. 285 For instance, the uCDN merges Logging information, extracts relevant 286 KPIs, and presents a formatted report to the CSP, in addition to a 287 bill for the content delivered by uCDN itself or by its dCDNs on the 288 CSP's behalf. The uCDN may also provide Logging information as raw 289 log files to the CSP, so that the CSP can use its own logging 290 analysis tools. 292 +-----+ 293 | CSP | 294 +-----+ 295 ^ Reporting and monitoring data 296 * Billing 297 ,--*--. 298 Logging ,-' `-. 299 Data =>( uCDN )<= Logging 300 // `-. _,-' \\ Data 301 || `-'-'-' || 302 ,-----. ,-----. 303 ,-' `-. ,-' `-. 304 ( dCDN-1 ) ( dCDN-2 )<== Logging 305 `-. ,-' `-. _,-' \\ Data 306 `--'--' `--'-' || 307 ,-----. 308 ,' `-. 309 ( dCDN-3 ) 310 `. ,-' 311 `--'--' 313 ===> CDNI Logging Interface 314 ***> outside the scope of CDNI 316 Figure 1: Interactions in CDNI Logging Reference Model 318 A dCDN (e.g., dCDN-2) integrates the relevant Logging information 319 obtained from its dCDNs (i.e., dCDN-3) in the Logging information 320 that it provides to the uCDN, so that the uCDN ultimately obtains all 321 Logging information relevant to a CSP for which it acts as the 322 authoritative CDN. Such aggregation is further discussed in 323 Section 3.6. 325 Note that the format of Logging information that a CDN provides over 326 the CDNI interface might be different from the one that the CDN uses 327 internally. In this case, the CDN needs to reformat the Logging 328 information before it provides this information to the other CDN over 329 the CDNI Logging interface. Similarly, a CDN might reformat the 330 Logging information that it receives over the CDNI Logging interface 331 before injecting it into its log-consuming applications or before 332 providing some of this Logging information to the CSP. Such 333 reformatting operations introduce latency in the logging distribution 334 chain and introduce a processing burden. Therefore, there are 335 benefits in specifying CDNI Logging formats that are suitable for use 336 inside CDNs and also are close to the intra-CDN Logging formats 337 commonly used in CDNs today. 339 2.2. Overall Logging Chain 341 This section discusses the overall logging chain within and across 342 CDNs to clarify how CDN Logging information is expected to fit in 343 this overall chain. Figure 2 illustrates the overall logging chain 344 within the dCDN, across CDNs using the CDNI Logging interface and 345 within the uCDN. Note that the logging chain illustrated in the 346 Figure is obviously only an example and varies depending on the 347 specific environments. For example, there may be more or fewer 348 instantiations of each entity (e.g., there may be 4 Log consuming 349 applications in a given CDN). As another example, there may be one 350 instance of Rectification process per Log Consuming Application 351 instead of a shared one. 353 Log Consuming Log Consuming 354 App App 355 ^ ^ 356 | | 357 Rectification---------- 358 ^ 359 | 360 Filtering 361 ^ 362 | 363 Collection 364 ^ ^ 365 | | 366 | Generation 367 | 368 | uCDN 369 CDNI Logging --------------------------------------------------- 370 exchange dCDN 371 ^ 372 | Log Consuming Log Consuming 373 | App App 374 | ^ ^ 375 | | | 376 Rectification Rectification--------- 377 ^ ^ 378 | | 379 Filtering 380 ^ 381 | 382 Collection 383 ^ ^ 384 | | 385 Generation Generation 387 Figure 2: CDNI Logging in the overall Logging Chain 389 The following subsections describe each of the processes potentially 390 involved in the logging chain of Figure 2. 392 2.2.1. Logging Generation and During-Generation Aggregation 394 CDNs typically generate Logging information for all significant task 395 completions, events, and failures. Logging information is typically 396 generated by many devices in the CDN including the surrogates, the 397 request routing system, and the control system. 399 The amount of Logging information generated can be huge. Therefore, 400 during contract negotiations, interconnected CDNs often agree on a 401 retention duration for Logging information, and/or potentially on a 402 maximum volume of Logging information that the dCDN ought to keep. 403 If this volume is exceeded, the dCDN is expected to alert the uCDN 404 but may not keep more Logging information for the considered time 405 period. In addition, CDNs may aggregate Logging information and 406 transmit only summaries for some categories of operations instead of 407 the full Logging information. Note that such aggregation leads to an 408 information loss, which may be problematic for some usages of the 409 Logging information (e.g., debugging). 411 [RFC6983] discusses logging for HTTP Adaptive Streaming (HAS). In 412 accordance with the recommendations articulated there, it is expected 413 that a surrogate will generate separate Logging information for 414 delivery of each chunk of HAS content. This ensures that separate 415 Logging information can then be provided to interconnected CDNs over 416 the CDNI Logging interface. Still in line with the recommendations 417 of [RFC6983], the Logging information for per-chunck delivery may 418 include some information (a Content Collection IDentifier and a 419 Session IDentifier) intended to facilitate subsequent post-generation 420 aggregation of per-chunk logs into per-session logs. Note that a CDN 421 may also elect to generate aggregate per-session logs when performing 422 HAS delivery, but this needs to be in addition to, and not instead 423 of, the per-chunk delivery logs. We note that aggregate per-session 424 logs for HAS delivery are for further study and outside the scope of 425 this document. 427 2.2.2. Logging Collection 429 This is the process that continuously collects Logging information 430 generated by the log-generating entities within a CDN. 432 In a CDNI environment, in addition to collecting Logging information 433 from log-generating entities within the local CDN, the Collection 434 process also collects Logging information provided by another CDN, or 435 other CDNs, through the CDNI Logging interface. This is illustrated 436 in Figure 2 where we see that the Collection process of the uCDN 437 collects Logging information from log-generating entities within the 438 uCDN as well as Logging information coming from the dCDNs through the 439 CDNI Logging interface. 441 2.2.3. Logging Filtering 443 A CDN may be required to only present different subsets of the whole 444 Logging information collected to various log-consuming applications. 445 This is achieved by the Filtering process. 447 In particular, the Filtering process can also filter the right subset 448 of Logging information that needs to be provided to a given 449 interconnected CDN. For example, the filtering process in the dCDN 450 can be used to ensure that only the Logging information related to 451 tasks performed on behalf of a given uCDN are made available to that 452 uCDN (thereby filtering out all the Logging information related to 453 deliveries by the dCDN of content for its own CSPs). Similarly, the 454 Filtering process may filter or partially mask some fields, for 455 example, to protect End Users' privacy when communicating CDNI 456 Logging information to another CDN. Filtering of Logging information 457 prior to communication of this information to other CDNs via the CDNI 458 Logging interface requires that the downstream CDN can recognize the 459 subset of Logging information that relate to each interconnected CDN. 461 The CDN will also filter some internal scope information such as 462 information related to its internal alarms (security, failures, load, 463 etc). 465 In some use cases described in [RFC6770], the interconnected CDNs do 466 not want to disclose details on their internal topology. The 467 filtering process can then also filter confidential data on the 468 dCDNs' topology (number of servers, location, etc.). In particular, 469 information about the requests served by each Surrogate may be 470 confidential. Therefore, the Logging information needs to be 471 protected so that data such as Surrogates' hostnames are not 472 disclosed to the uCDN. In the "Inter-Affiliates Interconnection" use 473 case, this information may be disclosed to the uCDN because both the 474 dCDN and the uCDN are operated by entities of the same group. 476 2.2.4. Logging Rectification and Post-Generation Aggregation 478 If Logging information is generated periodically, it is important 479 that the sessions that start in one Logging period and end in another 480 are correctly reported. If they are reported in the starting period, 481 then the Logging information of this period will be available only 482 after the end of the session, which delays the Logging information 483 generation. A simple approach is to provide the complete Logging 484 Record for a session in the Logging Period of the session end. 486 A Logging rectification/update mechanism could be useful to reach a 487 good trade-off between the Logging information generation delay and 488 the Logging information accuracy. 490 In the presence of HAS, some log-consuming applications can benefit 491 from aggregate per-session logs. For example, for analytics, per- 492 session logs allow display of session-related trends which are much 493 more meaningful for some types of analysis than chunk-related trends. 494 In the case where aggregate logs have been generated directly by the 495 log-generating entities, those can be used by the applications. In 496 the case where aggregate logs have not been generated, the 497 Rectification process can be extended with a Post-Generation 498 Aggregation process that generates per-session logs from the per- 499 chunk logs, possibly leveraging the information included in the per- 500 chunk logs for that purpose (Content Collection IDentifier and a 501 Session IDentifier). However, in accordance with [RFC6983], this 502 document does not define exchange of such aggregate logs on the CDNI 503 Logging interface. We note that this is for further study and 504 outside the scope of this document. 506 2.2.5. Log-Consuming Applications 508 2.2.5.1. Maintenance/Debugging 510 Logging information is useful to permit the detection (and limit the 511 risk) of content delivery failures. In particular, Logging 512 information facilitates the detection of configuration issues. 514 To detect faults, Logging information needs to report success and 515 failure of CDN delivery operations. The uCDN can summarize such 516 information into KPIs. For instance, Logging information needs to 517 allow the computation of the number of times, during a given time 518 period, that content delivery related to a specific service succeeds/ 519 fails. 521 Logging information enables the CDN providers to identify and 522 troubleshoot performance degradations. In particular, Logging 523 information enables tracking of traffic data (e.g., the amount of 524 traffic that has been forwarded by a dCDN on behalf of an uCDN over a 525 given period of time), which is particularly useful for CDN and 526 network planning operations. 528 2.2.5.2. Accounting 530 Logging information is essential for accounting, to permit inter-CDN 531 billing and CSP billing by uCDNs. For instance, Logging information 532 provided by dCDNs enables the uCDN to compute the total amount of 533 traffic delivered by every dCDN for a particular Content Provider, as 534 well as, the associated bandwidth usage (e.g., peak, 95th 535 percentile), and the maximum number of simultaneous sessions over a 536 given period of time. 538 2.2.5.3. Analytics and Reporting 540 The goal of analytics is to gather any relevant information to track 541 audience, analyze user behavior, and monitor the performance and 542 quality of content delivery. For instance, Logging information 543 enables the CDN providers to report on content consumption (e.g., 544 delivered sessions per content) in a specific geographic area. 546 The goal of reporting is to gather any relevant information to 547 monitor the performance and quality of content delivery and allow 548 detection of delivery issues. For instance, reporting could track 549 the average delivery throughput experienced by End Users in a given 550 region for a specific CSP or content set over a period of time. 552 2.2.5.4. Security 554 The goal of security is to prevent and monitor unauthorized access, 555 misuse, modification, and denial of access of a service. A set of 556 information is logged for security purposes. In particular, a record 557 of access to content is usually collected to permit the CSP to detect 558 infringements of content delivery policies and other abnormal End 559 User behaviors. 561 2.2.5.5. Legal Logging Duties 563 Depending on the country considered, the CDNs may have to retain 564 specific Logging information during a legal retention period, to 565 comply with judicial requisitions. 567 2.2.5.6. Notions common to multiple Log Consuming Applications 569 2.2.5.6.1. Logging Information Views 571 Within a given log-consuming application, different views may be 572 provided to different users depending on privacy, business, and 573 scalability constraints. 575 For example, an analytics tool run by the uCDN can provide one view 576 to an uCDN operator that exploits all the Logging information 577 available to the uCDN, while the tool may provide a different view to 578 each CSP exploiting only the Logging information related to the 579 content of the given CSP. 581 As another example, maintenance and debugging tools may provide 582 different views to different CDN operators, based on their 583 operational role. 585 2.2.5.6.2. Key Performance Indicators (KPIs) 587 This section presents, for explanatory purposes, a non-exhaustive 588 list of Key Performance Indicators (KPIs) that can be extracted/ 589 produced from logs. 591 Multiple log-consuming applications, such as analytics, monitoring, 592 and maintenance applications, often compute and track such KPIs. 594 In a CDNI environment, depending on the situation, these KPIs may be 595 computed by the uCDN or by the dCDN. But it is usually the uCDN that 596 computes KPIs, because the uCDN and dCDN may have different 597 definitions of the KPIs and the computation of some KPIs requires a 598 vision of all the deliveries performed by the uCDN and all its dCDNs. 600 Here is a list of important examples of KPIs: 602 o Number of delivery requests received from End Users in a given 603 region for each piece of content, during a given period of time 604 (e.g., hour/day/week/month) 606 o Percentage of delivery successes/failures among the aforementioned 607 requests 609 o Number of failures listed by failure type (e.g., HTTP error code) 610 for requests received from End Users in a given region and for 611 each piece of content, during a given period of time (e.g., 612 hour/day/week/month) 614 o Number and cause of premature delivery termination for End Users 615 in a given region and for each piece of content, during a given 616 period of time (e.g., hour/day/week/month) 618 o Maximum and mean number of simultaneous sessions established by 619 End Users in a given region, for a given Content Provider, and 620 during a given period of time (e.g., hour/day/week/month) 622 o Volume of traffic delivered for sessions established by End Users 623 in a given region, for a given Content Provider, and during a 624 given period of time (e.g., hour/day/week/month) 626 o Maximum, mean, and minimum delivery throughput for sessions 627 established by End Users in a given region, for a given Content 628 Provider, and during a given period of time (e.g., hour/day/week/ 629 month) 631 o Cache-hit and byte-hit ratios for requests received from End Users 632 in a given region for each piece of content, during a given period 633 of time (e.g., hour/day/week/month) 635 o Top 10 most popularly requested contents (during a given day/week/ 636 month) 638 o Terminal type (mobile, PC, STB, if this information can be 639 acquired from the browser type header, for example). 641 Additional KPIs can be computed from other sources of information 642 than the Logging information, for instance, data collected by a 643 content portal or by specific client-side application programming 644 interfaces. Such KPIs are out of scope for the present document. 646 The KPIs used depend strongly on the considered log-consuming 647 application -- the CDN operator may be interested in different 648 metrics than the CSP is. In particular, CDN operators are often 649 interested in delivery and acquisition performance KPIs, information 650 related to Surrogates' performance, caching information to evaluate 651 the cache-hit ratio, information about the delivered file size to 652 compute the volume of content delivered during peak hour, etc. 654 Some of the KPIs, for instance those providing an instantaneous 655 vision of the active sessions for a given CSP's content, are useful 656 essentially if they are provided in a timely manner. By contrast, 657 some other KPIs, such as those averaged on a long period of time, can 658 be provided in non-real-time. 660 3. CDNI Logging File 662 3.1. Rules 664 This specification uses the Augmented Backus-Naur Form (ABNF) 665 notation and core rules of [RFC5234]. In particular, the present 666 document uses the following rules from [RFC5234]: 668 CR = %x0D ; carriage return 670 ALPHA = %x41-5A / %x61-7A ; A-Z / a-z 672 DIGIT = %x30-39 ; 0-9 674 DQUOTE = %x22 ; " (Double Quote) 676 CRLF = CR LF ; Internet standard newline 678 HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F" 680 HTAB = %x09 ; horizontal tab 682 LF = %x0A ; linefeed 684 OCTET = %x00-FF ; 8 bits of data 686 The present document also uses the following rules from [RFC3986]: 688 host = as specified in section 3.2.2 of [RFC3986]. 690 IPv4address = as specified in section 3.2.2 of [RFC3986]. 692 IPv6address = as specified in section 3.2.2 of [RFC3986]. 694 The present document also defines the following additional rules: 696 ADDRESS = IPv4address / IPv6address 698 ALPHANUM = ALPHA / DIGIT 700 DATE = 4DIGIT "-" 2DIGIT "-" 2DIGIT 702 Dates are recorded in the format YYYY-MM-DD where YYYY, MM and 703 DD stand for the numeric year, month and day respectively. All 704 dates are specified in Universal Time Coordinated (UTC). 706 DEC = 1*DIGIT ["." *DIGIT] 708 NAMEFORMAT = ALPHANUM *(ALPHANUM / "_" / "-") 710 QSTRING = DQUOTE *NDQUOTE DQUOTE ; where 712 NDQUOTE = / 2DQUOTE ; whereby a 713 DQUOTE is conveyed inside a QSTRING unambiguously by repeating 714 it. 716 NHTABSTRING = *NHTAB ; where 718 NHTAB = 720 TIME = 2DIGIT ":" 2DIGIT ":" 2DIGIT ["." *DIGIT] 722 Times are recorded in the form HH:MM:SS or HH:MM:SS.S where HH 723 is the hour in 24 hour format, MM is minutes and SS is seconds. 724 All times are specified in Universal Time Coordinated (UTC). 726 3.2. CDNI Logging File Structure 728 As defined in Section 1.1: a CDNI Logging Field is as an atomic 729 logging information element, a CDNI Logging Record is a collection of 730 CDNI Logging Fields containing all logging information corresponding 731 to a single logging event, and a CDNI Logging File contains a 732 collection of CDNI Logging Records. This structure is illustrated in 733 Figure 3. The use of a file structure for transfer of CDNI Logging 734 information is selected since this is the most common practise today 735 for exchange of logging information within and across CDNs. 737 +----------------------------------------------------------+ 738 |CDNI Logging File | 739 | | 740 | #Directive 1 | 741 | #Directive 2 | 742 | ... | 743 | #Directive P | 744 | | 745 | +------------------------------------------------------+ | 746 | |CDNI Logging Record 1 | | 747 | | +-------------+ +-------------+ +-------------+ | | 748 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 749 | | | Field 1 | | Field 2 | | Field N | | | 750 | | +-------------+ +-------------+ +-------------+ | | 751 | +------------------------------------------------------+ | 752 | | 753 | +------------------------------------------------------+ | 754 | |CDNI Logging Record 2 | | 755 | | +-------------+ +-------------+ +-------------+ | | 756 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 757 | | | Field 1 | | Field 2 | | Field N | | | 758 | | +-------------+ +-------------+ +-------------+ | | 759 | +------------------------------------------------------+ | 760 | | 761 | ... | 762 | | 763 | #Directive P+1 | 764 | | 765 | ... | 766 | | 767 | +------------------------------------------------------+ | 768 | |CDNI Logging Record M | | 769 | | +-------------+ +-------------+ +-------------+ | | 770 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 771 | | | Field 1 | | Field 2 | | Field N | | | 772 | | +-------------+ +-------------+ +-------------+ | | 773 | +------------------------------------------------------+ | 774 | | 775 | | 776 | #Directive P+Q | 777 +----------------------------------------------------------+ 779 Figure 3: Structure of Logging Files 781 The CDNI Logging File format is inspired from the W3C Extended Log 782 File Format [ELF]. However, it is fully specified by the present 783 document. Where the present document differs from the W3C Extended 784 Log File Format, an implementation of the CDNI Logging interface MUST 785 comply with the present document. 787 Using a format that resembles the W3C Extended Log File Format is 788 intended to keep CDNI logging format close to the intra-CDN Logging 789 information format commonly used in CDNs today, thereby minimizing 790 systematic translation at CDN/CDNI boundary. 792 A CDNI Logging File MUST contain a sequence of lines containing US- 793 ASCII characters [CHAR_SET] terminated by CRLF. 795 Each line of a CDNI Logging File MUST contain either a directive or a 796 CDNI Logging Record. 798 Directives record information about the CDNI Logging process itself. 799 Lines containing directives MUST begin with the "#" character. 800 Directives are specified in Section 3.3. 802 Logging Records provide actual details of the logged event. Logging 803 Records are specified in Section 3.4. 805 The CDNI File structure is defined by the following rules: 807 DIRLINE = "#" directive CRLF 809 DIRGROUP = 1*DIRLINE 811 RECLINE = CRLF 813 RECGROUP = *RECLINE 815 = 1* 817 3.3. CDNI Logging File Directives 819 The CDNI Logging File directives are defined by the following rules: 821 directive = DIRNAME ":" HTAB DIRVAL 823 DIRNAME = 826 DIRVAL = 830 An implementation of the CDNI Logging interface MUST support all of 831 the following directives, listed below by their directive name: 833 o Version: 835 * format: "CDNI" "/" 1*DIGIT "." 1*DIGIT 837 * directive value: indicates the version of the CDNI Logging File 838 format. The value MUST be "CDNI/1.0" for the version specified 839 in the present document. 841 * occurrence: there MUST be one and only one instance of this 842 directive per CDNI Logging File. It MUST be the first line of 843 the CDNI Logging File. 845 o UUID: 847 * format: NHTABSTRING 849 * directive value: this a Universally Unique IDentifier (UUID) 850 from the UUID Uniform Resource Name (URN) namespace specified 851 in [RFC4122]) for the CDNI Logging File. 853 * occurrence: there MUST be one and only one instance of this 854 directive per CDNI Logging File. 856 o Claimed-Origin: 858 * format: host 860 * directive value: this contains the claimed identification of 861 the entity transmitting the CDNI Logging File (e.g., the host 862 in a dCDN supporting the CDNI Logging interface) or the entity 863 responsible for transmitting the CDNI Logging File (e.g., the 864 dCDN). 866 * occurrence: there MUST be zero or exactly one instance of this 867 directive per CDNI Logging File. This directive MAY be 868 included by the dCDN. It MUST NOT be included or modified by 869 the uCDN. 871 o Established-Origin: 873 * format: host 875 * directive value: this contains the identification, as 876 established by the entity receiving the CDNI Logging File, of 877 the entity transmitting the CDNI Logging File (e.g., the host 878 in a dCDN supporting the CDNI Logging interface) or the entity 879 responsible for transmitting the CDNI Logging File (e.g., the 880 dCDN). 882 * occurrence: there MUST be zero or exactly one instance of this 883 directive per CDNI Logging File. This directive MAY be added 884 by the uCDN (e.g., before storing the CDNI Logging File). It 885 MUST NOT be included by the dCDN. The mechanisms used by the 886 uCDN to establish and validate the entity responsible for the 887 CDNI Logging File is outside the scope of the present document. 888 We observe that, in particular, this may be achieved through 889 authentication mechanisms that are part of the transport layer 890 of the CDNI Logging File pull mechanism (Section 4.2). 892 o Record-Type: 894 * format: NAMEFORMAT 896 * directive value: indicates the type of the CDNI Logging Records 897 that follow this directive, until another Record-Type directive 898 (or the end of the CDNI Logging File). This can be any CDNI 899 Logging Record type registered in the CDNI Logging Record-types 900 registry (Section 6.2). For example this may be 901 "cdni_http_request_v1" as specified in Section 3.4.1. 903 * occurrence: there MUST be at least one instance of this 904 directive per CDNI Logging File. The first instance of this 905 directive MUST precede a Fields directive and MUST precede all 906 CDNI Logging Records. 908 o Fields: 910 * format: FIENAME * ; where FIENAME can take any 911 CDNI Logging field name registered in the CDNI Logging Field 912 Names registry (Section 6.3). 914 * directive value: this lists the names of all the fields for 915 which a value is to appear in the CDNI Logging Records that 916 follow the instance of this directive (until another instance 917 of this directive). The names of the fields, as well as their 918 occurrences, MUST comply with the corresponding rules specified 919 in the document referenced in the CDNI Logging Record-types 920 registry (Section 6.2) for the corresponding CDNI Logging 921 Record-Type. 923 * occurrence: there MUST be at least one instance of this 924 directive per Record-Type directive. The first instance of 925 this directive for a given Record-Type MUST appear before any 926 CDNI Logging Record for this Record-Type. One situation where 927 more than one instance of the Fields directive can appear 928 within a given CDNI Logging File, is when there is a change, in 929 the middle of a fairly large logging period, in the agreement 930 between the uCDN and the dCDN about the set of Fields that are 931 to be exchanged. The multiple occurences allow records with 932 the old set of fields and records with the new set of fields to 933 be carried inside the same Logging File. 935 o Integrity-Hash: 937 * format: 32HEXDIG 939 * directive value: This directive permits the detection of a 940 corrupted CDNI Logging File. This can be useful, for instance, 941 if a problem occurs on the filesystem of the dCDN Logging 942 system and leads to a truncation of a logging file. The valid 943 Integrity-Hash value is included in this directive by the 944 entity that transmits the CDNI Logging File. It is computed by 945 applying the MD5 ([RFC1321]) cryptographic hash function on the 946 CDNI Logging File, including all the directives and logging 947 records, up to the Integrity-Hash directive itself, excluding 948 the Integrity-Hash directive itself. The Integrity-Hash value 949 is represented as a US-ASCII encoded hexadecimal number, 32 950 digits long (representing a 128 bit hash value). The entity 951 receiving the CDNI Logging File also computes in a similar way 952 the MD5 hash on the received CDNI Logging File and compares 953 this hash to the value of the Integrity-Hash directive. If the 954 two values are equal, then the received CDNI Logging File MUST 955 be considered non-corrupted. If the two values are different, 956 the received CDNI Logging File MUST be considered corrupted. 957 The behavior of the entity that received a corrupted CDNI 958 Logging File is outside the scope of this specification; we 959 note that the entity MAY attempt to pull again the same CDNI 960 Logging File from the transmitting entity. If the entity 961 receiving a non-corrupted CDNI Logging File adds an 962 Established-Origin directive, it MUST then recompute and update 963 the Integrity-Hash directive so it also protects the added 964 Established-Origin directive. 966 * occurrence: there MUST be zero or exactly one instance of this 967 directive. There SHOULD be exactly one instance of this 968 directive. One situation where that directive could be omitted 969 is where integrity protection is already provided via another 970 mechanism (for example if an integrity hash is associated to 971 the CDNI Logging File out of band through the CDNI Logging Feed 972 ( Section 4.1) leveraging ATOM extensions such as those 973 proposed in [I-D.snell-atompub-link-extensions]. When present, 974 the Integrity-Hash field MUST be the last line of the CDNI 975 Logging File. 977 3.4. CDNI Logging Records 979 A CDNI Logging Record consists of a sequence of CDNI Logging Fields 980 relating to that single CDNI Logging Record. 982 CDNI Logging Fields MUST be separated by the "horizontal tabulation 983 (HTAB)" character. 985 To facilitate readability, a prefix scheme is used for CDNI Logging 986 field names in a similar way to the one used in W3C Extended Log File 987 Format [ELF]. The semantics of the prefix in the present document 988 is: 990 o c: refers to the User Agent that issues the request (corresponds 991 to the "client" of W3C Extended Log Format) 993 o d: refers to the dCDN (relative to a given CDN acting as a uCDN) 995 o s: refers to the dCDN Surrogate that serves the request 996 (corresponds to the "server" of W3C Extended Log Format) 998 o u: refers to the uCDN (relative to a given CDN acting as a dCDN) 1000 o cs: refers to communication from the User Agent towards the dCDN 1001 Surrogate 1003 o sc: refers to communication from the dCDN Surrogate towards the 1004 User Agent 1006 An implementation of the CDNI Logging interface as per the present 1007 specification MUST support the CDNI HTTP Request Logging Record as 1008 specified in Section 3.4.1. 1010 A CDNI Logging Record is defined by the following rules: 1012 FIEVAL = 1014 = FIEVAL * ; where FIEVAL 1015 contains the CDNI Logging field value corresponding to the CDNI 1016 Logging field names (FIENAME) listed is the last Fields directive 1017 preceding the present CDNI Logging Record. 1019 3.4.1. HTTP Request Logging Record 1021 This section defines the CDNI Logging Record of Record-Type 1022 "cdni_http_request_v1". It is applicable to content delivery 1023 performed by the dCDN using HTTP/1.0([RFC1945]), 1024 HTTP/1.1([RFC7230],[RFC7231], [RFC7232], [RFC7233], [RFC7234], 1026 [RFC7235]) or HTTPS ([RFC2818], [RFC7230]). We observe that, in the 1027 case of HTTPS delivery, there may be value in logging additional 1028 information specific to the operation of HTTP over TLS and we note 1029 that this is outside the scope of the present document and may be 1030 addressed in a future document defining another CDNI Logging Record 1031 or another version of the HTTP Request Logging Record. 1033 The "cdni_http_request_v1" Record-Type is also expected to be 1034 applicable to HTTP/2.0 [I-D.ietf-httpbis-http2] (which is still under 1035 development at the time of writing the present document) since a 1036 fundamental design tenet of HTTP/2.0 is to preserve the HTTP/1.1 1037 semantics. We observe that, in the case of HTTP/2.0 delivery, there 1038 may be value in logging additional information specific to the 1039 additional functionality of HTTP/2.0 (e.g. information related to 1040 connection identification, to stream identification, to stream 1041 priority and to flow control). We note that such additional 1042 information is outside the scope of the present document and may be 1043 addressed in a future document defining another CDNI Logging Record 1044 or another version of the HTTP Request Logging Record. 1046 The "cdni_http_request_v1" Record-Type contains the following CDNI 1047 Logging Fields, listed by their field name: 1049 o date: 1051 * format: DATE 1053 * field value: the date at which the processing of request 1054 completed on the Surrogate. 1056 * occurrence: there MUST be one and only one instance of this 1057 field. 1059 o time: 1061 * format: TIME 1063 * field value: the time at which the processing of request 1064 completed on the Surrogate. 1066 * occurrence: there MUST be one and only one instance of this 1067 field. 1069 o time-taken: 1071 * format: DEC 1072 * field value: decimal value of the duration, in seconds, between 1073 the start of the processing of the request and the completion 1074 of the request processing (e.g., completion of delivery) by the 1075 Surrogate. 1077 * occurrence: there MUST be one and only one instance of this 1078 field. 1080 o c-ip: 1082 * format: ADDRESS 1084 * field value: the source IPv4 or IPv6 address (i.e., the 1085 "client" address) in the request received by the Surrogate. 1087 * occurrence: there MUST be one and only one instance of this 1088 field. 1090 o c-ip-anonymizing: 1092 * format: 1*DIGIT 1094 * field value: the number of rightmost bits of the address in the 1095 c-ip field that are zeroed-out in order to anonymize the 1096 logging record. The mechanism by which the two ends of the 1097 CDNI Logging interface agree on whether anonymization is to be 1098 supported and the number of bits that need to be zeroed-out for 1099 this purpose are outside the scope of the present document. 1101 * occurrence: there MUST be zero or exactly one instance of this 1102 field. 1104 o c-port: 1106 * format: 1*DIGIT 1108 * field value: the source TCP port (i.e., the "client" port) in 1109 the request received by the Surrogate. 1111 * occurrence: there MUST be zero or exactly one instance of this 1112 field. 1114 o s-ip: 1116 * format: ADDRESS 1118 * field value: the IPv4 or IPv6 address of the Surrogate that 1119 served the request (i.e., the "server" address). 1121 * occurrence: there MUST be zero or exactly one instance of this 1122 field. 1124 o s-hostname: 1126 * format: host 1128 * field value: the hostname of the Surrogate that served the 1129 request (i.e., the "server" hostname). 1131 * occurrence: there MUST be zero or exactly one instance of this 1132 field. 1134 o s-port: 1136 * format: 1*DIGIT 1138 * field value: the destination TCP port (i.e., the "server" port) 1139 in the request received by the Surrogate. 1141 * occurrence: there MUST be zero or exactly one instance of this 1142 field. 1144 o cs-method: 1146 * format: NHTABSTRING 1148 * field value: this is the method of the request received by the 1149 Surrogate. In the case of HTTP delivery, this is the HTTP 1150 method in the request. 1152 * occurrence: There MUST be one and only one instance of this 1153 field. 1155 o cs-uri: 1157 * format: NHTABSTRING 1159 * field value: this is the "effective request URI" of the request 1160 received by the Surrogate as specified in [RFC7230]. It 1161 complies with the "http" URI scheme or the "https" URI scheme 1162 as specified in [RFC7230]). 1164 * occurrence: there MUST be zero or exactly one instance of this 1165 field. 1167 o u-uri: 1169 * format: NHTABSTRING 1171 * field value: this is a complete URI, derived from the 1172 "effective request URI" ([RFC7230]) of the request received by 1173 the Surrogate (i.e., the cs-uri) but transformed by the entity 1174 generating or transmitting the CDNI Logging Record, in a way 1175 that is agreed upon between the two ends of the CDNI Logging 1176 interface, so the transformed URI is meaningful to the uCDN. 1177 For example, the two ends of the CDNI Logging interface could 1178 agree that the u-uri is constructed from the cs-uri by removing 1179 the part of the hostname that exposes which individual 1180 Surrogate actually performed the delivery. The details of 1181 modification performed to generate the u-uri, as well as the 1182 mechanism to agree on these modifications between the two sides 1183 of the CDNI Logging interface are outside the scope of the 1184 present document. 1186 * occurrence: there MUST be one and only one instance of this 1187 field. 1189 o protocol: 1191 * format: NHTABSTRING 1193 * field value: this is value of the HTTP-Version field as 1194 specified in [RFC7230] of the Request-Line of the request 1195 received by the Surrogate (e.g., "HTTP/1.1"). 1197 * occurrence: there MUST be one and only one instance of this 1198 field. 1200 o sc-status: 1202 * format: 3DIGIT 1204 * field value: this is the Status-Code in the response from the 1205 Surrogate. In the case of HTTP delivery, this is the HTTP 1206 Status-Code in the HTTP response. 1208 * occurrence: There MUST be one and only one instance of this 1209 field. 1211 o sc-total-bytes: 1213 * format: 1*DIGIT 1215 * field value: this is the total number of bytes of the response 1216 sent by the Surrogate in response to the request. In the case 1217 of HTTP delivery, this includes the bytes of the Status-Line, 1218 the bytes of the HTTP headers and the bytes of the message- 1219 body. 1221 * occurrence: There MUST be one and only one instance of this 1222 field. 1224 o sc-entity-bytes: 1226 * format: 1*DIGIT 1228 * field value: this is the number of bytes of the message-body in 1229 the HTTP response sent by the Surrogate in response to the 1230 request. This does not include the bytes of the Status-Line or 1231 the bytes of the HTTP headers. 1233 * occurrence: there MUST be zero or exactly one instance of this 1234 field. 1236 o cs(): 1238 * format: QSTRING 1240 * field value: the value of the HTTP header (identified by the 1241 in the CDNI Logging field name) as it 1242 appears in the request processed by the Surrogate, but 1243 prepended by a DQUOTE and appended by a DQUOTE. For example, 1244 when the CDNI Logging field name (FIENAME) listed in the 1245 preceding Fields directive is cs(User-Agent), this CDNI Logging 1246 field value contains the value of the User-Agent HTTP header as 1247 received by the Surrogate in the request it processed, but 1248 prepended by a DQUOTE and appended by a DQUOTE. If the HTTP 1249 header as it appeared in the request processed by the Surrogate 1250 contains one or more DQUOTE, each DQUOTE MUST be escaped by an 1251 additional DQUOTE. For example, if the HTTP header contains 1252 My_Header"value", then the field value of the cs() is "My_Header""value""". 1255 * occurrence: there MAY be zero, one or any number of instance of 1256 this field. 1258 o sc(): 1260 * format: QSTRING 1262 * field value: the value of the HTTP header (identified by the 1263 in the CDNI Logging field name) as it 1264 appears in the response issued by the Surrogate to serve the 1265 request, but prepended by a DQUOTE and appended by a DQUOTE. 1266 If the HTTP header as it appeared in the request processed by 1267 the Surrogate contains one or more DQUOTE, each DQUOTE MUST be 1268 escaped by an additional DQUOTE. For example, if the HTTP 1269 header contains My_Header"value", then the field value of the 1270 cs() is "My_Header""value""". 1272 * occurrence: there MAY be zero, one or any number of instances 1273 of this field. For a given , there MUST be 1274 zero or exactly one instance of this field. 1276 o s-ccid: 1278 * format: QSTRING 1280 * field value: this contains the value of the Content Collection 1281 IDentifier (CCID) associated by the uCDN to the content served 1282 by the Surrogate via the CDNI Metadata interface 1283 ([I-D.ietf-cdni-metadata]), prepended by a DQUOTE and appended 1284 by a DQUOTE. If the CCID conveyed in the CDNI Metadata 1285 interface contains one or more DQUOTE, each DQUOTE MUST be 1286 escaped by an additional DQUOTE. For example, if the CCID 1287 conveyed in the CDNI Metadata interface is My_CCIDD"value", 1288 then the field value of the s-ccid is "My_CCID""value""". 1290 * occurrence: there MUST be zero or exactly one instance of this 1291 field. For a given , there MUST be zero or 1292 exactly one instance of this field. 1294 o s-sid: 1296 * format: QSTRING 1298 * field value: this contains the value of a Session IDentifier 1299 (SID) generated by the dCDN for a specific HTTP session, 1300 prepended by a DQUOTE and appended by a DQUOTE. In particular, 1301 for HTTP Adaptive Streaming (HAS) session, the Session 1302 IDentifier value is included in the Logging record for every 1303 content chunk delivery of that session in view of facilitating 1304 the later correlation of all the per content chunk log records 1305 of a given HAS session. See section 3.4.2.2. of [RFC6983] for 1306 more discussion on the concept of Session IDentifier in the 1307 context of HAS. If the SID conveyed contains one or more 1308 DQUOTE, each DQUOTE MUST be escaped by an additional DQUOTE. 1309 For example, if the SID is My_SID"value", then the field value 1310 of the s-sid is "My_SID""value""". 1312 * occurrence: there MUST be zero or exactly one instance of this 1313 field. 1315 o s-cached: 1317 * format: 1DIGIT 1319 * field value: this characterises whether the Surrogate served 1320 the request using content already stored on its local cache or 1321 not. The allowed values are "0" (for miss) and "1" (for hit). 1322 "1" MUST be used when the Surrogate did serve the request using 1323 exclusively content already stored on its local cache. "0" MUST 1324 be used otherwise (including cases where the Surrogate served 1325 the request using some, but not all, content already stored on 1326 its local cache). Note that a "0" only means a cache miss in 1327 the Surrogate and does not provide any information on whether 1328 the content was already stored, or not, in another device of 1329 the dCDN, i.e., whether this was a "dCDN hit" or "dCDN miss". 1331 * occurrence: there MUST be zero or exactly one instance of this 1332 field. 1334 The "Fields" directive corresponding to a HTTP Request Logging Record 1335 MUST contain all the fields names whose occurrence is specified above 1336 as "There MUST be one and only one instance of this field". The 1337 corresponding fields value MUST be present in every HTTP Request 1338 Logging Record. 1340 The "Fields" directive corresponding to a HTTP Request Logging Record 1341 MAY list all the fields value whose occurrence is specified above as 1342 "there MUST be zero or exactly one instance of this field" or "there 1343 MAY be zero, one or any number of instances of this field". The set 1344 of such field names actually listed in the "Fields" directive is 1345 selected by the CDN generating the CDNI Logging File based on 1346 agreements between the interconnected CDNs established through 1347 mechanisms outside the scope of this specification (e.g., contractual 1348 agreements). When such a field name is not listed in the "Fields" 1349 directive, the corresponding field value MUST NOT be included in the 1350 Logging Record. When such a field name is listed in the "Fields" 1351 directive, the corresponding field value MUST be included in the 1352 Logging Record; if the value for the field is not available, this 1353 MUST be conveyed via a dash character ("-"). 1355 The fields names listed in the "Fields" directive MAY be listed in 1356 the order in which they are listed in Section 3.4.1 or MAY be listed 1357 in any other order. 1359 A dCDN-side implementation of the CDNI Logging interface MUST 1360 implement all the following Logging Fields in a CDNI Logging Record 1361 of Record-Type "cdni_http_request_v1", and MUST support the ability 1362 to include valid values for each of them: 1364 o date 1366 o time 1368 o time-taken 1370 o c-ip 1372 o c-port 1374 o s-ip 1376 o s-hostname 1378 o s-port 1380 o cs-method 1382 o cs-uri 1384 o u-uri 1386 o protocol 1388 o sc-status 1390 o sc-total-bytes 1392 o sc-entity-bytes 1394 o cs() 1396 o sc() 1398 o s-cached 1400 A dCDN-side implementation of the CDNI Logging interface MAY support 1401 the following Logging Fields in a CDNI Logging Record of Record-Type 1402 "cdni_http_request_v1": 1404 o c-ip-anonymizing 1406 o s-ccid 1407 o s-sid 1409 If a dCDN-side implementation of the CDNI Logging interface supports 1410 these Fields, it MUST support the ability to include valid values for 1411 them. 1413 An uCDN-side implementation of the CDNI Logging interface MUST be 1414 able to accept CDNI Logging Files with CDNI Logging Records of 1415 Record-Type "cdni_http_request_v1" containing any CDNI Logging Field 1416 defined in Section 3.4.1 as long as the CDNI Logging Record and the 1417 CDNI Logging File are compliant with the present document. 1419 3.5. CDNI Logging File Example 1421 Let us consider the upstream CDN and the downstream CDN labelled uCDN 1422 and dCDN-1 in Figure 1. When dCDN-1 acts as a downstream CDN for 1423 uCDN and performs content delivery on behalf of uCDN, dCDN-1 will 1424 include the CDNI Logging Records corresponding to the content 1425 deliveries performed on behalf of uCDN in the CDNI Logging Files for 1426 uCDN. An example CDNI Logging File communicated by dCDN-1 to uCDN is 1427 shown below in Figure 4. 1429 #Version:CDNI/1.0 1431 #UUID:urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6 1433 #Claimed-Origin:cdni-logging-entity.dcdn-1.example.com 1435 #Record-Type:cdni_http_request_v1 1437 #Fields:datetimetime-takenc-ip 1438 cs-methodu-uriprotocolsc-status 1439 sc-total-bytescs(User-Agent)cs(Referer) 1440 s-cached 1442 2013-05-1700:38:06.8259.05810.5.7.1GET 1443 http://cdni-ucdn.dcdn-1.example.com/video/movie100.mp4 1444 HTTP/1.12006729891"Mozilla/5.0 1445 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1446 Gecko) Chrome/5.0.375.127 Safari/533.4" 1447 "host1.example.com"1 1449 2013-05-1700:39:09.14515.3210.5.10.5GET 1450 http://cdni-ucdn.dcdn-1.example.com/video/movie118.mp4 1451 HTTP/1.120015799210"Mozilla/5.0 1452 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1453 Gecko) Chrome/5.0.375.127 Safari/533.4" 1454 "host1.example.com"1 1456 2013-05-1700:42:53.43752.87910.5.10.5GET 1457 http://cdni-ucdn.dcdn-1.example.com/video/picture11.mp4 1458 HTTP/1.020097234724"Mozilla/5.0 1459 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1460 Gecko) Chrome/5.0.375.127 Safari/533.4" 1461 "host5.example.com"0 1463 #Integrity-Hash:fe113dfce8fec91323a4fc02261af26e 1465 [Editor's Note: compute the correct Integrity-Hash value once example 1466 is frozen] 1468 Figure 4: CDNI Logging File Example 1470 If uCDN establishes by some means (e.g. via TLS authentication when 1471 pulling the CDNI Logging File) the identity of the entity from which 1472 it pulled the CDNI Logging File, uCDN can add to the CDNI Logging an 1473 Established-Origin directive as illustrated below: 1475 #Established-Origin:cdni-logging-entity.dcdn- 1476 1.example.com 1478 As illustrated in Figure 2, uCDN will then ingest the corresponding 1479 CDNI Logging Records into its Collection process, alongside the 1480 Logging Records generated locally by the uCDN itself. This allows 1481 uCDN to aggregate Logging Records for deliveries performed by itself 1482 (through Records generated locally) as well as for deliveries 1483 performed by its downstream CDN(s). This aggregate information can 1484 then be used (after Filtering and Rectification, as illustrated in 1485 Figure 2) by Log Consuming Applications that take into account 1486 deliveries performed by uCDN as well as by all of its downstream 1487 CDNs. 1489 We observe that the time between 1491 1. when a delivery is completed in dCDN and 1493 2. when the corresponding Logging Record is ingested by the 1494 Collection process in uCDN 1496 depends on a number of parameters such as the Logging Period agreed 1497 to by uCDN and dCDN, how much time uCDN waits before pulling the CDNI 1498 Logging File once it is advertised in the CDNI Logging Feed, and the 1499 time to complete the pull of the CDNI Logging File. Therefore, if we 1500 consider the set of Logging Records aggregated by the Collection 1501 process in uCDN in a given time interval, there could be a permanent 1502 significant timing difference between the CDNI Logging Records 1503 received from the dCDN and the Logging Records generated locally. 1504 For example, in a given time interval, the Collection process in uCDN 1505 may be aggregating Logging Records generated locally by uCDN for 1506 deliveries performed in the last hour and CDNI Logging Records 1507 generated in the dCDN for deliveries in the hour before last. 1509 3.6. Cascaded CDNI Logging Files Example 1511 Let us consider the cascaded CDN scenario of uCDN, dCDN-2 and dCDN-3 1512 as depicted in Figure 1. After completion of a delivery by dCDN-3 on 1513 behalf of dCDN-2, dCDN-3 will include a corresponding Logging Record 1514 in a CDNI Logging File that will be pulled by dCDN-2 and that is 1515 illustrated below in Figure 5. In practice, a CDNI Logging File is 1516 likely to contain a very high number of CDNI Logging Records. 1517 However, for readability, the example in Figure 5 contains a single 1518 CDNI Logging Record. 1520 #Version:CDNI/1.0 1522 #UUID:urn:uuid:65718ef-0123-9876-adce4321bcde 1524 #Claimed-Origin:cdni-logging-entity.dcdn-3.example.com 1526 #Record-Type:cdni_http_request_v1 1528 #Fields:datetimetime-takenc-ip 1529 cs-methodu-uriprotocolsc-status 1530 sc-total-bytescs(User-Agent)cs(Referer) 1531 s-cached 1533 2013-05-1700:39:09.11914.0710.5.10.9GET 1534 http://cdni-dcdn-2.dcdn-3.example.com/video/movie118.mp4 1535 HTTP/1.120015799210"Mozilla/5.0 1536 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1537 Gecko) Chrome/5.0.375.127 Safari /533.4" 1538 "host1.example.com"1 1540 #Integrity-Hash:fe113dfce8fec91323a4fc02261af26e 1542 [Editor's Note: compute the correct Integrity-Hash value once example 1543 is frozen] 1545 Figure 5: Cascaded CDNI Logging File Example (dCDN-3 to dCDN-2) 1547 If dCDN-2 establishes by some means (e.g. via TLS authentication when 1548 pulling the CDNI Logging File) the identity of the entity from which 1549 it pulled the CDNI Logging File, dCDN-2 can add to the CDNI Logging 1550 an Established-Origin directive as illustrated below: 1552 #Established-Origin:cdni-logging-entity.dcdn- 1553 3.example.com 1555 dCDN-2 (behaving as an upstream CDN from the viewpoint of dCDN-3) 1556 will then ingest the CDNI Logging Record for the considered dCDN-3 1557 delivery into its Collection process (as illustrated in Figure 2). 1558 This Logging Record may be aggregated with Logging Records generated 1559 locally by dCDN-2 for deliveries performed by dCDN-2 itself. Say, 1560 for illustration, that the content delivery performed by dCDN-3 on 1561 behalf of dCDN-2 had actually been redirected to dCDN-2 by uCDN, and 1562 say that another content delivery has just been redirected by uCDN to 1563 dCDN-2 and that dCDN-2 elected to perform the corresponding delivery 1564 itself. Then after Filtering and Rectification (as illustrated in 1565 Figure 2), dCDN-2 will include the two Logging Records corresponding 1566 respectively to the delivery performed by dCDN-3 and the delivery 1567 performed by dCDN-2, in the next CDNI Logging File that will be 1568 communicated to uCDN. An example of such CDNI Logging File is 1569 illustrated below in Figure 6. 1571 #Version:CDNI/1.0 1573 #UUID:urn:uuid:1234567-8fedc-abab-0987654321ff 1575 #Claimed-Origin:cdni-logging-entity.dcdn-2.example.com 1577 #Record-Type:cdni_http_request_v1 1579 #Fields:datetimetime-takenc-ip 1580 cs-methodu-uriprotocolsc-status 1581 sc-total-bytescs(User-Agent)cs(Referer) 1582 s-cached 1584 2013-05-1700:39:09.11914.0710.5.10.9GET 1585 http://cdni-ucdn.dcdn-2.example.com/video/movie118.mp4 1586 HTTP/1.120015799210"Mozilla/5.0 1587 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1588 Gecko) Chrome/5.0.375.127 Safari /533.4" 1589 "host1.example.com"1 1591 2013-05-1701:42:53.43752.87910.5.10.12GET 1592 http://cdni-ucdn.dcdn-2.example.com/video/picture11.mp4 1593 HTTP/1.020097234724"Mozilla/5.0 1594 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1595 Gecko) Chrome/5.0.375.127 Safari /533.4" 1596 "host5.example.com"0 1598 #Integrity-Hash:fe113dfce8fec91323a4fc02261af26e 1600 [Editor's Note: compute the correct Integrity-Hash value once example 1601 is frozen] 1603 Figure 6: Cascaded CDNI Logging File Example (dCDN-2 to uCDN) 1605 If uCDN establishes by some means (e.g. via TLS authentication when 1606 pulling the CDNI Logging File) the identity of the entity from which 1607 it pulled the CDNI Logging File, uCDN can add to the CDNI Logging an 1608 Established-Origin directive as illustrated below: 1610 #Established-Origin:cdni-logging-entity.dcdn- 1611 2.example.com 1613 In the example of Figure 6, we observe that: 1615 o the first Logging Record corresponds to the Logging Record 1616 communicated earlier to dCDN-2 by dCDN-3, which corresponds to a 1617 delivery redirected by uCDN to dCDN-2 and then redirected by 1618 dCDN-2 to dCDN-3. The fields values in this Logging Record are 1619 copied from the corresponding CDNI Logging REcord communicated to 1620 dCDN2 by dCDN-3, with the exception of the u-uri that now reflects 1621 the URI convention between uCDN and dCDN-2 and that presents the 1622 delivery to uCDN as if it was performed by dCDN-2 itself. This 1623 reflects the fact that dCDN-2 had taken the full responsibility of 1624 the corresponding delivery (even if in this case, dCDN-2 elected 1625 to redirect the delivery to dCDN-3 so it is actually performed by 1626 dCDN-3 on behalf of dCDN-2). 1628 o the second Logging Record corresponds to a delivery redirected by 1629 uCDN to dCDN-2 and performed by dCDN-2 itself. The time of the 1630 delivery in this Logging Record may be significantly more recent 1631 than the first Logging Record since it was generated locally while 1632 the first Logging Record was generated by dCDN-3 and had to be 1633 advertised , and then pulled and then ingested into the dCDN-2 1634 Collection process, before being aggregated with the second 1635 Logging Record. 1637 4. Protocol for Exchange of CDNI Logging File After Full Collection 1639 This section specifies a protocol for the exchange of CDNI Logging 1640 Files as specified in Section 3 after the CDNI Logging File is fully 1641 collected by the dCDN. 1643 This protocol comprises: 1645 o a CDNI Logging feed, allowing the dCDN to notify the uCDN about 1646 the CDNI Logging Files that can be retrieved by that uCDN from the 1647 dCDN, as well as all the information necessary for retrieving each 1648 of these CDNI Logging Files. The CDNI Logging feed is specified 1649 in Section 4.1. 1651 o a CDNI Logging File pull mechanism, allowing the uCDN to obtain 1652 from the dCDN a given CDNI Logging File at the uCDN's convenience. 1653 The CDNI Logging File pull mechanisms is specified in Section 4.2. 1655 An implementation of the CDNI Logging interface on the dCDN side (the 1656 entity generating the CDNI Logging file) MUST support the server side 1657 of the CDNI Logging feed (as specified in Section 4.1) and the server 1658 side of the CDNI Logging pull mechanism (as specified in 1659 Section 4.2). 1661 An implementation of the CDNI Logging interface on the uCDN side (the 1662 entity consuming the CDNI Logging file) MUST support the client side 1663 of the CDNI Logging feed (as specified in Section 4.1) and the client 1664 side of the CDNI Logging pull mechanism (as specified in 1665 Section 4.2). 1667 4.1. CDNI Logging Feed 1669 The server-side implementation of the CDNI Logging feed MUST produce 1670 an Atom feed [RFC4287]. This feed is used to advertise log files 1671 that are available for the client-side to retrieve using the CDNI 1672 Logging pull mechanism. 1674 4.1.1. Atom Formatting 1676 A CDNI Logging feed MUST be structured as an Archived feed, as 1677 defined in [RFC5005], and MUST be formatted in Atom [RFC4287]. This 1678 means it consists of a subscription document that is regularly 1679 updated as new CDNI Logging Files become available, and information 1680 about older CDNI Logging files is moved into archive documents. Once 1681 created, archive documents are never modified. 1683 Each CDNI Logging File listed in an Atom feed MUST be described in an 1684 atom:entry container element. 1686 The atom:entry MUST contain an atom:content element whose "src" 1687 attribute is a link to the CDNI Logging File and whose "type" 1688 attribute is the MIME Media Type indicating that the entry is a CDNI 1689 Logging File. We define this MIME Media Type as "application/ 1690 cdni.LoggingFile" (See Section 6.4). 1692 For compatibility with some Atom feed readers the atom:entry MAY also 1693 contain an atom:link entry whose "href" attribute is a link to the 1694 CDNI Logging File and whose "type" attribute is the MIME Media Type 1695 indicating that the entry is a CDNI Logging File using the 1696 "application/cdni.LoggingFile" MIME Media Type (See Section 6.4). 1698 The URI used in the atom:id of the atom:entry MUST contain the UUID 1699 of the CDNI Logging File. 1701 The atom:updated in the atom:entry MUST indicate the time at which 1702 the CDNI Logging File was last updated. 1704 4.1.2. Updates to Log Files and the Feed 1706 CDNI Logging Files MUST NOT be modified by the dCDN once published in 1707 the CDNI Logging feed. 1709 The frequency with which the subscription feed is updated, the period 1710 of time covered by each CDNI Logging File or each archive document, 1711 and timeliness of publishing of CDNI Logging Files are outside the 1712 scope of the present document and are expected to be agreed upon by 1713 uCDN and dCDN via other means (e.g., human agreement). 1715 The server-side implementation MUST be able to set, and SHOULD set, 1716 HTTP cache control headers on the subscription feed to indicate the 1717 frequency at which the client-side is to poll for updates. 1719 The client-side MAY use HTTP cache control headers (set by the 1720 server-side) on the subscription feed to determine the frequency at 1721 which to poll for updates. The client-side MAY instead, or in 1722 addition, use other information to determine when to poll for updates 1723 (e.g., a polling frequency that may have been negotiated between the 1724 uCDN and dCDN by mechanisms outside the scope of the present document 1725 and that is to override the indications provided in the HTTP cache 1726 control headers). 1728 The potential retention limits (e.g., sliding time window) within 1729 which the dCDN is to retain and be ready to serve an archive document 1730 is outside the scope of the present document and is expected to be 1731 agreed upon by uCDN and dCDN via other means (e.g., human agreement). 1732 The server-side implementation MUST retain, and be ready to serve, 1733 any archive document within the agreed retention limits. Outside 1734 these agreed limits, the server-side implementation MAY indicate its 1735 inability to serve (e.g., with HTTP status code 404) an archive 1736 document or MAY refuse to serve it (e.g., with HTTP status code 403 1737 or 410). 1739 4.1.3. Redundant Feeds 1741 The server-side implementation MAY present more than one CDNI Logging 1742 feed for redundancy. Each CDNI Logging File MAY be published in more 1743 than one feed. 1745 A client-side implementation MAY support such redundant CDNI Logging 1746 feeds. If it supports redundant CDNI Logging feed, the client-side 1747 can use the UUID of the CDNI Logging File, presented in the atom:id 1748 element of the Atom feed, to avoid unnecessarily pulling and storing 1749 a given CDNI Logging File more than once. 1751 4.1.4. Example CDNI Logging Feed 1753 Figure 7 illustrates an example of the subscription document of a 1754 CDNI Logging feed. 1756 1757 > 1759 CDNI Logging Feed 1760 2013-03-23T14:46:11Z 1761 urn:uuid:663ae677-40fb-e99a-049d-c5642916b8ce 1762 1764 1766 1768 CDNI Log Feed 1769 Generator 1770 dcdn.example 1771 1772 CDNI Logging File for uCDN at 1773 2013-03-23 14:15:00 1774 urn:uuid:12345678-1234-abcd-00aa-01234567abcd 1775 2013-03-23T14:15:00Z 1776 1779 CDNI Logging File for uCDN at 1780 2013-03-23 14:15:00 1781 1782 1783 CDNI Logging File for uCDN at 1784 2013-03-23 14:30:00 1785 urn:uuid:87654321-4321-dcba-aa00-dcba7654321 1786 2013-03-23T14:30:00Z 1787 1790 CDNI Logging File for uCDN at 1791 2013-03-23 14:30:00 1792 1793 ... 1794 1795 ... 1796 1797 1799 Figure 7: Example subscription document of a CDNI Logging Feed 1801 4.2. CDNI Logging File Pull 1803 A client-side implementation of the CDNI Logging interface MAY pull, 1804 at its convenience, a CDNI Logging File that is published by the 1805 server-side in the CDNI Logging Feed (in the subscription document or 1806 an archive document). To do so, the client-side: 1808 o MUST implement HTTP/1.1 ([RFC7230],[RFC7231], [RFC7232], 1809 [RFC7233], [RFC7234], [RFC7235]), MAY also support other HTTP 1810 versions (e.g., HTTP/2.0 [I-D.ietf-httpbis-http2]) and MAY 1811 negotiate which HTTP version is actually used. This allows 1812 operators and implementers to choose to use later versions of HTTP 1813 to take advantage of new features, while still ensuring 1814 interoperability with systems that only support HTTP/1.1. 1816 o MUST use the URI that was associated to the CDNI Logging File 1817 (within the "src" attribute of the corresponding atom:content 1818 element) in the CDNI Logging Feed; 1820 o MUST support exchange of CDNI Logging Files with no content 1821 encoding applied to the representation; 1823 o MUST support exchange of CDNI Logging Files with "gzip" content 1824 encoding (as defined in [RFC7230]) applied to the representation. 1826 Note that a client-side implementation of the CDNI Logging interface 1827 MAY pull a CDNI Logging File that it has already pulled. 1829 The server-side implementation MUST respond to valid pull request by 1830 a client-side implementation for a CDNI Logging File published by the 1831 server-side in the CDNI Logging Feed (in the subscription document or 1832 an archive document). The server-side implementation: 1834 o MUST implement HTTP/1.1 to handle the client-side request and MAY 1835 also support other HTTP versions (e.g., HTTP/2.0); 1837 o MUST include the CDNI Logging File identified by the request URI 1838 inside the body of the HTTP response; 1840 o MUST support exchange of CDNI Logging Files with no content 1841 encoding applied to the representation; 1843 o MUST support exchange of CDNI Logging Files with "gzip" content 1844 encoding (as defined in [RFC7231]) applied to the representation. 1846 Content negotiation approaches defined in [RFC7231] (e.g., using 1847 Accept-Encoding request-header field or Content-Encoding entity- 1848 header field) MAY be used by the client-side and server-side 1849 implementations to establish the content-coding to be used for a 1850 particular exchange of a CDNI Logging File. 1852 Applying compression content encoding (such as "gzip") is expected to 1853 mitigate the impact of exchanging the large volumes of logging 1854 information expected across CDNs. This is expected to be 1855 particularly useful in the presence of HTTP Adaptive Streaming (HAS) 1856 which, as per the present version of the document, will result in a 1857 separate CDNI Log Record for each HAS segment delivery in the CDNI 1858 Logging File. 1860 The potential retention limits (e.g., sliding time window, maximum 1861 aggregate file storage quotas) within which the dCDN is to retain and 1862 be ready to serve a CDNI Logging File previously advertised in the 1863 CDNI Logging Feed is outside the scope of the present document and is 1864 expected to be agreed upon by uCDN and dCDN via other means (e.g., 1865 human agreement). The server-side implementation MUST retain, and be 1866 ready to serve, any CDNI Logging File within the agreed retention 1867 limits. Outside these agreed limits, the server-side implementation 1868 MAY indicate its inability to serve (e.g., with HTTP status code 404) 1869 a CDNI Logging File or MAY refuse to serve it (e.g., with HTTP status 1870 code 403 or 410). 1872 5. Protocol for Exchange of CDNI Logging File During Collection 1874 We note that, in addition to the CDNI Logging File exchange protocol 1875 specified in Section 4, implementations of the CDNI Logging interface 1876 MAY also support other mechanisms to exchange CDNI Logging Files. In 1877 particular, such mechanisms might allow the exchange of the CDNI 1878 Logging File to start before the file is fully collected. This can 1879 allow CDNI Logging Records to be communicated by the dCDN to the uCDN 1880 as they are gathered by the dCDN without having to wait until all the 1881 CDNI Logging Records of the same logging period are collected in the 1882 corresponding CDNI Logging File. This approach is commonly referred 1883 to as "tailing" of the file. 1885 Such an approach could be used, for example, to exchange logging 1886 information with a significantly reduced time-lag (e.g., sub-minute 1887 or sub-second) between when the event occurred in the dCDN and when 1888 the corresponding CDNI Logging Record is made available to the uCDN. 1889 This can satisfy log-consuming applications requiring extremely fresh 1890 logging information such as near-real-time content delivery 1891 monitoring. Such mechanisms are for further study and outside the 1892 scope of this document. 1894 6. IANA Considerations 1896 6.1. CDNI Logging Directive Names Registry 1898 The IANA is requested to create a new registry, CDNI Logging 1899 Directive Names. 1901 The initial contents of the CDNI Logging File Directives registry 1902 comprise the names of the directives specified in Section 3.3 of the 1903 present document, and are as follows: 1905 +------------------------------+-----------+ 1906 | Directive Name | Reference | 1907 +------------------------------+-----------+ 1908 | Version | RFC xxxx | 1909 | UUID | RFC xxxx | 1910 | Claimed-Origin | RFC xxxx | 1911 | Established-Origin | RFC xxxx | 1912 | Record-Type | RFC xxxx | 1913 | Fields | RFC xxxx | 1914 | Integrity-Hash | RFC xxxx | 1915 +------------------------------+-----------+ 1917 Figure 8 1919 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 1920 the present document] 1922 Within the registry, names are to be allocated by IANA according to 1923 the "Specification Required" policy specified in [RFC5226]. 1924 Directive names are to be allocated by IANA with a format of 1925 NAMEFORMAT (see Section 3.1). 1927 Each specification that defines a new CDNI Logging directive needs to 1928 contain a description for the new directive with the same set of 1929 information as provided in Section 3.3 (i.e., format, directive value 1930 and occurrence). 1932 6.2. CDNI Logging Record-Types Registry 1934 The IANA is requested to create a new registry, CDNI Logging Record- 1935 Types. 1937 The initial contents of the CDNI Logging Record-Types registry 1938 comprise the names of the CDNI Logging Record types specified in 1939 Section 3.4 of the present document, and are as follows: 1941 +----------------------+-----------+----------------------------------+ 1942 | Record-Types | Reference | Description | 1943 +----------------------+-----------+----------------------------------+ 1944 | cdni_http_request_v1 | RFC xxxx | CDNI Logging Record version 1 | 1945 | | | for content delivery using HTTP | 1946 +----------------------+-----------+----------------------------------+ 1948 Figure 9 1950 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 1951 the present document] 1953 Within the registry, Record-Types are to be allocated by IANA 1954 according to the "Specification Required" policy specified in 1955 [RFC5226]. Record-Types are to be allocated by IANA with a format of 1956 NAMEFORMAT (see Section 3.1). Record-Types corresponding to 1957 specifications produced by the IETF CDNI Working Group are to be 1958 allocated a name starting with "cdni_". All other Record-Types are 1959 to be allocated a name that does not start with "cdni". 1961 Each specification that defines a new Record-Type needs to contain a 1962 description for the new Record-Type with the same set of information 1963 as provided in Section 3.4.1. This includes: 1965 o a list of all the CDNI Logging Fields that can appear in a CDNI 1966 Logging Record of the new Record-Type 1968 o for all these Fields: a specification of the occurrence for each 1969 Field in the new Record-Type 1971 o for every newly defined Field, i.e., for every Field that results 1972 in a registration in the CDNI Logging Field Names Registry 1973 (Section 6.3): a specification of the field name, format and field 1974 value. 1976 6.3. CDNI Logging Field Names Registry 1978 The IANA is requested to create a new registry, CDNI Logging Field 1979 Names. 1981 This registry is intended to be shared across the currently defined 1982 Record-Type (i.e., cdni_http_request_v1) as well as potential other 1983 CDNI Logging Record-Types that may be defined in separate 1984 specifications. When a Field from this registry is used by another 1985 CDNI Logging Record-Type, it is to be used with the exact semantics 1986 and format specified in the document that registered this field and 1987 that is identified in the Reference column of the registry. If 1988 another CDNI Logging Record-Type requires a Field with semantics that 1989 are not strictly identical, or a format that is not strictly 1990 identical then this new Field is to be registered in the registry 1991 with a different Field name. When a Field from this registry is used 1992 by another CDNI Logging Record-Type, it can be used with different 1993 occurence rules. 1995 The initial contents of the CDNI Logging Fields Names registry 1996 comprise the names of the CDNI Logging fields specified in 1997 Section 3.4 of the present document, and are as follows: 1999 +------------------------------------------+-----------+ 2000 | Field Name | Reference | 2001 +------------------------------------------+-----------+ 2002 | date | RFC xxxx | 2003 | time | RFC xxxx | 2004 | time-taken | RFC xxxx | 2005 | c-ip | RFC xxxx | 2006 | c-ip-anonymizing | RFC xxxx | 2007 | c-port | RFC xxxx | 2008 | s-ip | RFC xxxx | 2009 | s-hostname | RFC xxxx | 2010 | s-port | RFC xxxx | 2011 | cs-method | RFC xxxx | 2012 | cs-uri | RFC xxxx | 2013 | u-uri | RFC xxxx | 2014 | protocol | RFC xxxx | 2015 | sc-status | RFC xxxx | 2016 | sc-total-bytes | RFC xxxx | 2017 | sc-entity-bytes | RFC xxxx | 2018 | cs() | RFC xxxx | 2019 | sc() | RFC xxxx | 2020 | s-ccid | RFC xxxx | 2021 | s-sid | RFC xxxx | 2022 | s-cached | RFC xxxx | 2023 +------------------------------------------+-----------+ 2025 Figure 10 2027 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2028 the present document] 2030 Within the registry, names are to be allocated by IANA according to 2031 the "Specification Required" policy specified in [RFC5226]. Field 2032 names are to be allocated by IANA with a format of NHTABSTRING (see 2033 Section 3.1). 2035 6.4. CDNI Logging MIME Media Type 2037 The IANA is requested to allocate the "application/cdni.LoggingFile" 2038 MIME Media Type (whose use is specified in Section 4.1.1 of the 2039 present document) in the MIME Media Types registry. 2041 7. Security Considerations 2043 7.1. Authentication, Confidentiality, Integrity Protection 2045 An implementation of the CDNI Logging interface MUST support TLS 2046 transport of the CDNI Logging feed (Section 4.1) and of the CDNI 2047 Logging File pull (Section 4.2) as per [RFC2818] and [RFC7230]. 2049 The use of TLS for transport of the CDNI Logging feed and CDNI 2050 Logging File pull allows: 2052 o the dCDN and uCDN to authenticate each other (to ensure they are 2053 transmitting/receiving CDNI Logging File from an authenticated 2054 CDN) 2056 o the CDNI Logging information to be transmitted with 2057 confidentiality 2059 o the integrity of the CDNI Logging information to be protected 2060 during the exchange 2062 In an environment where any such protection is required, TLS SHOULD 2063 be used (including authentication of the remote end) by the server- 2064 side and the client-side of the CDNI Logging feed and the CDNI 2065 Logging File pull mechanism unless alternate methods are used for 2066 ensuring the confidentiality of the information in the logging files 2067 (such as setting up an IPsec tunnel between the two CDNs or using a 2068 physically secured internal network between two CDNs that are owned 2069 by the same corporate entity). 2071 An implementation of the CDNI Logging interface MUST support the 2072 TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 cipher suite ( [RFC5288]). An 2073 implementation of the CDNI Logging interface SHOULD prefer cipher 2074 suites which support perfect forward secrecy over cipher suites that 2075 don't. 2077 The Integrity-Hash directive inside the CDNI Logging File provides 2078 additional integrity protection, this time targeting potential 2079 corruption of the CDNI logging information during the CDNI Logging 2080 File generation, storage or exchange. This mechanism does not itself 2081 allow restoration of the corrupted CDNI Logging information, but it 2082 allows detection of such corruption and therefore triggering of 2083 appropriate corrective actions (e.g., discard of corrupted 2084 information, attempt to re-obtain the CDNI Logging information). 2085 Note that the Integrity-Hash does not protect against tampering by a 2086 third party, since such a third party could have recomputed and 2087 updated the Integrity-Hash after tampering. Protection against third 2088 party tampering can be achieved as discussed above through the use of 2089 TLS. 2091 7.2. Denial of Service 2093 This document does not define specific mechanism to protect against 2094 Denial of Service (DoS) attacks on the Logging Interface. However, 2095 the CDNI Logging feed and CDNI Logging pull endpoints can be 2096 protected against DoS attacks through the use of TLS transport and/or 2097 via mechanisms outside the scope of the CDNI Logging interface such 2098 as firewalling or use of Virtual Private Networks (VPNs). 2100 Protection of dCDN Surrogates against spoofed delivery requests is 2101 outside the scope of the CDNI Logging interface. 2103 7.3. Privacy 2105 CDNs have the opportunity to collect detailed information about the 2106 downloads performed by End Users. The provision of this information 2107 to another CDN introduces potential End Users privacy protection 2108 concerns. 2110 The use of TLS for transport of the CDNI Logging feed and CDNI 2111 Logging pull as discussed in Section 7.1 protects the confidentiality 2112 of logged information by preventing any other party than the 2113 authorised uCDN to gain access to the logging information. 2115 We observe that when CDNI interconnection is realised as per 2116 [RFC7336], the uCDN handles the initial End User requests (before it 2117 is redirected to the dCDN) so, regardless of which information is, or 2118 is not, communicated to the uCDN through the CDNI Logging interface, 2119 the uCDN has visibility on significant information such as the IP 2120 address of the End User request and the URL of the request. 2122 Nonetheless, if the dCDN and uCDN agree that anonymization is 2123 required to avoid making some detailed information available to the 2124 uCDN (such as how many bytes of the content have been watched by an 2125 End User and/or at what time) or is required to meet some legal 2126 obligations, then the uCDN and dCDN can agree to exchange anonymized 2127 End Users IP address in CDNI Logging Files and the c-ip-anonymization 2128 field can be used to convey the number of bits that have been 2129 anonymized so that the meaningful information can still be easily 2130 extracted from the anonymized addressses (e.g., for geolocation aware 2131 analytics). 2133 We note that anonymization of End Users IP address does not fully 2134 protect against deriving potentially sensitive information about 2135 traffic patterns; in general, increasing the number of bits that are 2136 anonymized can mitigate the risks of deriving such sensitive traffic 2137 pattern information. 2139 We also note that independently of IP addresses, the query string 2140 portion of the URL that may be conveyed inside the cs-uri and u-uri 2141 fields of CDNI Logging Files, or the HTTP cookies( [RFC6265]) that 2142 may be conveyed inside the cs() field of CDNI 2143 Logging Fields, may contain personnal information or information that 2144 can be exploited to derive personal information. Where this is a 2145 concern, the CDNI Logging interface specification allows the dCDN to 2146 not include the cs-uri and to include a u-uri that removes (or hides) 2147 the sensitive part of the query string and allows the dCDN to not 2148 include the cs() fields corresponding to HTTP 2149 headers associated with cookies. 2151 8. Acknowledgments 2153 This document borrows from the W3C Extended Log Format [ELF]. 2155 Rob Murray significantly contributed into the text of Section 4.1. 2157 The authors thank Ben Niven-Jenkins, Kevin Ma, David Mandelberg and 2158 Ray van Brandenburg for their ongoing input. 2160 Finally, we also thank Sebastien Cubaud, Pawel Grochocki, Christian 2161 Jacquenet, Yannick Le Louedec, Anne Marrec , Emile Stephan, Fabio 2162 Costa, Sara Oueslati, Yvan Massot, Renaud Edel, Joel Favier and the 2163 contributors of the EU FP7 OCEAN project for their input in the early 2164 versions of this document. 2166 9. References 2168 9.1. Normative References 2170 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2171 Requirement Levels", BCP 14, RFC 2119, March 1997. 2173 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 2174 Resource Identifier (URI): Generic Syntax", STD 66, RFC 2175 3986, January 2005. 2177 [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally 2178 Unique IDentifier (UUID) URN Namespace", RFC 4122, July 2179 2005. 2181 [RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom 2182 Syndication Format", RFC 4287, December 2005. 2184 [RFC5005] Nottingham, M., "Feed Paging and Archiving", RFC 5005, 2185 September 2007. 2187 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 2188 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 2189 May 2008. 2191 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 2192 Specifications: ABNF", STD 68, RFC 5234, January 2008. 2194 [RFC5288] Salowey, J., Choudhury, A., and D. McGrew, "AES Galois 2195 Counter Mode (GCM) Cipher Suites for TLS", RFC 5288, 2196 August 2008. 2198 [RFC7230] Fielding, R. and J. Reschke, "Hypertext Transfer Protocol 2199 (HTTP/1.1): Message Syntax and Routing", RFC 7230, June 2200 2014. 2202 [RFC7231] Fielding, R. and J. Reschke, "Hypertext Transfer Protocol 2203 (HTTP/1.1): Semantics and Content", RFC 7231, June 2014. 2205 [RFC7232] Fielding, R. and J. Reschke, "Hypertext Transfer Protocol 2206 (HTTP/1.1): Conditional Requests", RFC 7232, June 2014. 2208 [RFC7233] Fielding, R., Lafon, Y., and J. Reschke, "Hypertext 2209 Transfer Protocol (HTTP/1.1): Range Requests", RFC 7233, 2210 June 2014. 2212 [RFC7234] Fielding, R., Nottingham, M., and J. Reschke, "Hypertext 2213 Transfer Protocol (HTTP/1.1): Caching", RFC 7234, June 2214 2014. 2216 [RFC7235] Fielding, R. and J. Reschke, "Hypertext Transfer Protocol 2217 (HTTP/1.1): Authentication", RFC 7235, June 2014. 2219 9.2. Informative References 2221 [CHAR_SET] 2222 "IANA Character Sets registry", 2223 . 2226 [ELF] Phillip M. Hallam-Baker, and Brian Behlendorf, "Extended 2227 Log File Format, W3C (work in progress), WD-logfile- 2228 960323", . 2230 [I-D.ietf-cdni-metadata] 2231 Niven-Jenkins, B., Murray, R., Caulfield, M., Leung, K., 2232 and K. Ma, "CDN Interconnection Metadata", draft-ietf- 2233 cdni-metadata-07 (work in progress), July 2014. 2235 [I-D.ietf-httpbis-http2] 2236 Belshe, M., Peon, R., and M. Thomson, "Hypertext Transfer 2237 Protocol version 2", draft-ietf-httpbis-http2-14 (work in 2238 progress), July 2014. 2240 [I-D.snell-atompub-link-extensions] 2241 Snell, J., "Atom Link Extensions", draft-snell-atompub- 2242 link-extensions-09 (work in progress), June 2012. 2244 [RFC1321] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321, 2245 April 1992. 2247 [RFC1945] Berners-Lee, T., Fielding, R., and H. Nielsen, "Hypertext 2248 Transfer Protocol -- HTTP/1.0", RFC 1945, May 1996. 2250 [RFC2818] Rescorla, E., "HTTP Over TLS", RFC 2818, May 2000. 2252 [RFC6265] Barth, A., "HTTP State Management Mechanism", RFC 6265, 2253 April 2011. 2255 [RFC6707] Niven-Jenkins, B., Le Faucheur, F., and N. Bitar, "Content 2256 Distribution Network Interconnection (CDNI) Problem 2257 Statement", RFC 6707, September 2012. 2259 [RFC6770] Bertrand, G., Stephan, E., Burbridge, T., Eardley, P., Ma, 2260 K., and G. Watson, "Use Cases for Content Delivery Network 2261 Interconnection", RFC 6770, November 2012. 2263 [RFC6983] van Brandenburg, R., van Deventer, O., Le Faucheur, F., 2264 and K. Leung, "Models for HTTP-Adaptive-Streaming-Aware 2265 Content Distribution Network Interconnection (CDNI)", RFC 2266 6983, July 2013. 2268 [RFC7336] Peterson, L., Davie, B., and R. van Brandenburg, 2269 "Framework for Content Distribution Network 2270 Interconnection (CDNI)", RFC 7336, August 2014. 2272 [RFC7337] Leung, K. and Y. Lee, "Content Distribution Network 2273 Interconnection (CDNI) Requirements", RFC 7337, August 2274 2014. 2276 Authors' Addresses 2278 Francois Le Faucheur (editor) 2279 Cisco Systems 2280 E.Space Park - Batiment D 2281 6254 Allee des Ormes - BP 1200 2282 Mougins cedex 06254 2283 FR 2285 Phone: +33 4 97 23 26 19 2286 Email: flefauch@cisco.com 2288 Gilles Bertrand (editor) 2289 Orange 2290 38-40 rue du General Leclerc 2291 Issy les Moulineaux 92130 2292 FR 2294 Phone: +33 1 45 29 89 46 2295 Email: gilles.bertrand@orange.com 2297 Iuniana Oprescu (editor) 2298 Orange 2299 38-40 rue du General Leclerc 2300 Issy les Moulineaux 92130 2301 FR 2303 Phone: +33 6 89 06 92 72 2304 Email: iuniana.oprescu@orange.com 2306 Roy Peterkofsky 2307 Skytide, Inc. 2308 One Kaiser Plaza, Suite 785 2309 Oakland CA 94612 2310 USA 2312 Phone: +01 510 250 4284 2313 Email: roy@skytide.com