idnits 2.17.1 draft-ietf-cdni-logging-15.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (February 4, 2015) is 3368 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) ** Obsolete normative reference: RFC 7230 (Obsoleted by RFC 9110, RFC 9112) ** Obsolete normative reference: RFC 7231 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7232 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7233 (Obsoleted by RFC 9110) ** Obsolete normative reference: RFC 7234 (Obsoleted by RFC 9111) ** Obsolete normative reference: RFC 7235 (Obsoleted by RFC 9110) == Outdated reference: A later version (-21) exists of draft-ietf-cdni-metadata-08 == Outdated reference: A later version (-17) exists of draft-ietf-httpbis-http2-16 -- Obsolete informational reference (is this intentional?): RFC 2818 (Obsoleted by RFC 9110) Summary: 7 errors (**), 0 flaws (~~), 3 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force F. Le Faucheur, Ed. 3 Internet-Draft Cisco Systems 4 Intended status: Standards Track G. Bertrand, Ed. 5 Expires: August 8, 2015 I. Oprescu, Ed. 6 Orange 7 R. Peterkofsky 8 Skytide, Inc. 9 February 4, 2015 11 CDNI Logging Interface 12 draft-ietf-cdni-logging-15 14 Abstract 16 This memo specifies the Logging interface between a downstream CDN 17 (dCDN) and an upstream CDN (uCDN) that are interconnected as per the 18 CDN Interconnection (CDNI) framework. First, it describes a 19 reference model for CDNI logging. Then, it specifies the CDNI 20 Logging File format and the actual protocol for exchange of CDNI 21 Logging Files. 23 Status of This Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at http://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on August 8, 2015. 40 Copyright Notice 42 Copyright (c) 2015 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (http://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 58 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 59 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 4 60 2. CDNI Logging Reference Model . . . . . . . . . . . . . . . . 5 61 2.1. CDNI Logging interactions . . . . . . . . . . . . . . . . 5 62 2.2. Overall Logging Chain . . . . . . . . . . . . . . . . . . 8 63 2.2.1. Logging Generation and During-Generation Aggregation 9 64 2.2.2. Logging Collection . . . . . . . . . . . . . . . . . 10 65 2.2.3. Logging Filtering . . . . . . . . . . . . . . . . . . 10 66 2.2.4. Logging Rectification and Post-Generation Aggregation 11 67 2.2.5. Log-Consuming Applications . . . . . . . . . . . . . 12 68 2.2.5.1. Maintenance/Debugging . . . . . . . . . . . . . . 12 69 2.2.5.2. Accounting . . . . . . . . . . . . . . . . . . . 12 70 2.2.5.3. Analytics and Reporting . . . . . . . . . . . . . 12 71 2.2.5.4. Security . . . . . . . . . . . . . . . . . . . . 13 72 2.2.5.5. Legal Logging Duties . . . . . . . . . . . . . . 13 73 2.2.5.6. Notions common to multiple Log Consuming 74 Applications . . . . . . . . . . . . . . . . . . 13 75 3. CDNI Logging File . . . . . . . . . . . . . . . . . . . . . . 15 76 3.1. Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 15 77 3.2. CDNI Logging File Structure . . . . . . . . . . . . . . . 16 78 3.3. CDNI Logging File Directives . . . . . . . . . . . . . . 19 79 3.4. CDNI Logging Records . . . . . . . . . . . . . . . . . . 23 80 3.4.1. HTTP Request Logging Record . . . . . . . . . . . . . 23 81 3.5. CDNI Logging File Example . . . . . . . . . . . . . . . . 32 82 3.6. Cascaded CDNI Logging Files Example . . . . . . . . . . . 34 83 4. Protocol for Exchange of CDNI Logging File After Full 84 Collection . . . . . . . . . . . . . . . . . . . . . . . . . 37 85 4.1. CDNI Logging Feed . . . . . . . . . . . . . . . . . . . . 38 86 4.1.1. Atom Formatting . . . . . . . . . . . . . . . . . . . 38 87 4.1.2. Updates to Log Files and the Feed . . . . . . . . . . 38 88 4.1.3. Redundant Feeds . . . . . . . . . . . . . . . . . . . 39 89 4.1.4. Example CDNI Logging Feed . . . . . . . . . . . . . . 39 90 4.2. CDNI Logging File Pull . . . . . . . . . . . . . . . . . 41 91 5. Protocol for Exchange of CDNI Logging File During Collection 42 92 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 43 93 6.1. CDNI Logging Directive Names Registry . . . . . . . . . . 43 94 6.2. CDNI Logging Record-Types Registry . . . . . . . . . . . 43 95 6.3. CDNI Logging Field Names Registry . . . . . . . . . . . . 44 96 6.4. CDNI Logging MIME Media Type . . . . . . . . . . . . . . 46 98 7. Security Considerations . . . . . . . . . . . . . . . . . . . 46 99 7.1. Authentication, Confidentiality, Integrity Protection . . 46 100 7.2. Denial of Service . . . . . . . . . . . . . . . . . . . . 47 101 7.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 47 102 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 48 103 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 48 104 9.1. Normative References . . . . . . . . . . . . . . . . . . 48 105 9.2. Informative References . . . . . . . . . . . . . . . . . 49 106 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 51 108 1. Introduction 110 This memo specifies the CDNI Logging interface between a downstream 111 CDN (dCDN) and an upstream CDN (uCDN). First, it describes a 112 reference model for CDNI logging. Then, it specifies the CDNI 113 Logging File format and the actual protocol for exchange of CDNI 114 Logging Files. 116 The reader should be familiar with the following documents: 118 o CDNI problem statement [RFC6707] and framework [RFC7336] identify 119 a Logging interface, 121 o Section 8 of [RFC7337] specifies a set of requirements for 122 Logging, 124 o [RFC6770] outlines real world use-cases for interconnecting CDNs. 125 These use cases require the exchange of Logging information 126 between the dCDN and the uCDN. 128 As stated in [RFC6707], "the CDNI Logging interface enables details 129 of logs or events to be exchanged between interconnected CDNs". 131 The present document describes: 133 o The CDNI Logging reference model (Section 2), 135 o The CDNI Logging File format (Section 3), 137 o The CDNI Logging File Exchange protocol (Section 4). 139 1.1. Terminology 141 In this document, the first letter of each CDNI-specific term is 142 capitalized. We adopt the terminology described in [RFC6707] and 143 [RFC7336], and extend it with the additional terms defined below. 145 Intra-CDN Logging information: logging information generated and 146 collected within a CDN. The format of the Intra-CDN Logging 147 information may be different to the format of the CDNI Logging 148 information. 150 CDNI Logging information: logging information exchanged across CDNs 151 using the CDNI Logging Interface. 153 Logging information: logging information generated and collected 154 within a CDN or obtained from another CDN using the CDNI Logging 155 Interface. 157 CDNI Logging Field: an atomic element of information that can be 158 included in a CDNI Logging Record. The time an event/task started, 159 the IP address of an End User to whom content was delivered, and the 160 Uniform Resource Identifier (URI) of the content delivered, are 161 examples of CDNI Logging Fields. 163 CDNI Logging Record: an information record providing information 164 about a specific event. This comprises a collection of CDNI Logging 165 Fields. 167 CDNI Logging File: a file containing CDNI Logging Records, as well as 168 additional information facilitating the processing of the CDNI 169 Logging Records. 171 CDN Reporting: the process of providing the relevant information that 172 will be used to create a formatted content delivery report provided 173 to the CSP in deferred time. Such information typically includes 174 aggregated data that can cover a large period of time (e.g., from 175 hours to several months). Uses of Reporting include the collection 176 of charging data related to CDN services and the computation of Key 177 Performance Indicators (KPIs). 179 CDN Monitoring: the process of providing or displaying content 180 delivery information in a timely fashion with respect to the 181 corresponding deliveries. Monitoring typically includes visibility 182 of the deliveries in progress for service operation purposes. It 183 presents a view of the global health of the services as well as 184 information on usage and performance, for network services 185 supervision and operation management. In particular, monitoring data 186 can be used to generate alarms. 188 1.2. Requirements Language 190 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 191 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 192 document are to be interpreted as described in RFC 2119 [RFC2119]. 194 2. CDNI Logging Reference Model 196 2.1. CDNI Logging interactions 198 The CDNI logging reference model between a given uCDN and a given 199 dCDN involves the following interactions: 201 o customization by the uCDN of the CDNI Logging information to be 202 provided by the dCDN to the uCDN (e.g., control of which CDNI 203 Logging Fields are to be communicated to the uCDN for a given task 204 performed by the dCDN or control of which types of events are to 205 be logged). The dCDN takes into account this CDNI Logging 206 customization information to determine what Logging information to 207 provide to the uCDN, but it may, or may not, take into account 208 this CDNI Logging customization information to influence what CDN 209 logging information is to be generated and collected within the 210 dCDN (e.g., even if the uCDN requests a restricted subset of the 211 logging information, the dCDN may elect to generate a broader set 212 of logging information). The mechanism to support the 213 customization by the uCDN of CDNI Logging information is outside 214 the scope of this document and left for further study. Until such 215 a mechanism is available, the uCDN and dCDN are expected to agree 216 off-line on what exact set of CDNI Logging information is to be 217 provided by the dCDN to the uCDN, and to rely on management plane 218 actions to configure the CDNI Logging functions in the dCDN to 219 generate this information set and in the uCDN to expect this 220 information set. 222 o generation and collection by the dCDN of the intra-CDN Logging 223 information related to the completion of any task performed by the 224 dCDN on behalf of the uCDN (e.g., delivery of the content to an 225 End User) or related to events happening in the dCDN that are 226 relevant to the uCDN (e.g., failures or unavailability in dCDN). 227 This takes place within the dCDN and does not directly involve 228 CDNI interfaces. 230 o communication by the dCDN to the uCDN of the Logging information 231 collected by the dCDN relevant to the uCDN. This is supported by 232 the CDNI Logging interface and in the scope of the present 233 document. For example, the uCDN may use this Logging information 234 to charge the CSP, to perform analytics and monitoring for 235 operational reasons, to provide analytics and monitoring views on 236 its content delivery to the CSP or to perform trouble-shooting. 237 This document exclusively specifies non-real-time exchange of 238 Logging information. Closer to real-time exchange of Logging 239 information (say sub-minute or sub-second) is outside the scope of 240 the present document and left for further study. This document 241 exclusively specifies exchange of Logging information related to 242 content delivery. Exchange of Logging information related to 243 operational events (e.g., dCDN request routing function 244 unavailable, content acquisition failure by dCDN) for audit or 245 operational reactive adjustments by uCDN is outside the scope of 246 the present document and left for further study. 248 o customization by the dCDN of the CDNI Logging information to be 249 provided by the uCDN on behalf of the dCDN. The mechanism to 250 support the customization by the dCDN of CDNI Logging information 251 is outside the scope of this document and left for further study. 253 o generation and collection by the uCDN of Intra-CDN Logging 254 information related to the completion of any task performed by the 255 uCDN on behalf of the dCDN (e.g., serving of content by uCDN to 256 dCDN for acquisition purposes by dCDN) or related to events 257 happening in the uCDN that are relevant to the dCDN. This takes 258 place within the uCDN and does not directly involve CDNI 259 interfaces. 261 o communication by the uCDN to the dCDN of the Logging information 262 collected by the uCDN relevant to the dCDN. For example, the dCDN 263 might potentially benefit from this information for security 264 auditing or content acquisition troubleshooting. This is outside 265 the scope of this document and left for further study. 267 Figure 1 provides an example of CDNI Logging interactions (focusing 268 only on the interactions that are in the scope of this document) in a 269 particular scenario where four CDNs are involved in the delivery of 270 content from a given CSP: the uCDN has a CDNI interconnection with 271 dCDN-1 and dCDN-2. In turn, dCDN2 has a CDNI interconnection with 272 dCDN3. In this example, uCDN, dCDN-1, dCDN-2 and dCDN-3 all 273 participate in the delivery of content for the CSP. In this example, 274 the CDNI Logging interface enables the uCDN to obtain Logging 275 information from all the dCDNs involved in the delivery. In the 276 example, the uCDN uses the Logging information: 278 o to analyze the performance of the delivery performed by the dCDNs 279 and to adjust its operations after the fact (e.g., request 280 routing) as appropriate, 282 o to provide (non-real-time) reporting and monitoring information to 283 the CSP. 285 For instance, the uCDN merges Logging information, extracts relevant 286 KPIs, and presents a formatted report to the CSP, in addition to a 287 bill for the content delivered by uCDN itself or by its dCDNs on the 288 CSP's behalf. The uCDN may also provide Logging information as raw 289 log files to the CSP, so that the CSP can use its own logging 290 analysis tools. 292 +-----+ 293 | CSP | 294 +-----+ 295 ^ Reporting and monitoring data 296 * Billing 297 ,--*--. 298 Logging ,-' `-. 299 Data =>( uCDN )<= Logging 300 // `-. _,-' \\ Data 301 || `-'-'-' || 302 ,-----. ,-----. 303 ,-' `-. ,-' `-. 304 ( dCDN-1 ) ( dCDN-2 )<== Logging 305 `-. ,-' `-. _,-' \\ Data 306 `--'--' `--'-' || 307 ,-----. 308 ,' `-. 309 ( dCDN-3 ) 310 `. ,-' 311 `--'--' 313 ===> CDNI Logging Interface 314 ***> outside the scope of CDNI 316 Figure 1: Interactions in CDNI Logging Reference Model 318 A dCDN (e.g., dCDN-2) integrates the relevant Logging information 319 obtained from its dCDNs (i.e., dCDN-3) in the Logging information 320 that it provides to the uCDN, so that the uCDN ultimately obtains all 321 Logging information relevant to a CSP for which it acts as the 322 authoritative CDN. Such aggregation is further discussed in 323 Section 3.6. 325 Note that the format of Logging information that a CDN provides over 326 the CDNI interface might be different from the one that the CDN uses 327 internally. In this case, the CDN needs to reformat the Logging 328 information before it provides this information to the other CDN over 329 the CDNI Logging interface. Similarly, a CDN might reformat the 330 Logging information that it receives over the CDNI Logging interface 331 before injecting it into its log-consuming applications or before 332 providing some of this Logging information to the CSP. Such 333 reformatting operations introduce latency in the logging distribution 334 chain and introduce a processing burden. Therefore, there are 335 benefits in specifying CDNI Logging formats that are suitable for use 336 inside CDNs and also are close to the intra-CDN Logging formats 337 commonly used in CDNs today. 339 2.2. Overall Logging Chain 341 This section discusses the overall logging chain within and across 342 CDNs to clarify how CDN Logging information is expected to fit in 343 this overall chain. Figure 2 illustrates the overall logging chain 344 within the dCDN, across CDNs using the CDNI Logging interface and 345 within the uCDN. Note that the logging chain illustrated in the 346 Figure is obviously only an example and varies depending on the 347 specific environments. For example, there may be more or fewer 348 instantiations of each entity (e.g., there may be 4 Log consuming 349 applications in a given CDN). As another example, there may be one 350 instance of Rectification process per Log Consuming Application 351 instead of a shared one. 353 Log Consuming Log Consuming 354 App App 355 ^ ^ 356 | | 357 Rectification---------- 358 ^ 359 | 360 Filtering 361 ^ 362 | 363 Collection 364 ^ ^ 365 | | 366 | Generation 367 | 368 | uCDN 369 CDNI Logging --------------------------------------------------- 370 exchange dCDN 371 ^ 372 | Log Consuming Log Consuming 373 | App App 374 | ^ ^ 375 | | | 376 Rectification Rectification--------- 377 ^ ^ 378 | | 379 Filtering 380 ^ 381 | 382 Collection 383 ^ ^ 384 | | 385 Generation Generation 387 Figure 2: CDNI Logging in the overall Logging Chain 389 The following subsections describe each of the processes potentially 390 involved in the logging chain of Figure 2. 392 2.2.1. Logging Generation and During-Generation Aggregation 394 CDNs typically generate Logging information for all significant task 395 completions, events, and failures. Logging information is typically 396 generated by many devices in the CDN including the surrogates, the 397 request routing system, and the control system. 399 The amount of Logging information generated can be huge. Therefore, 400 during contract negotiations, interconnected CDNs often agree on a 401 retention duration for Logging information, and/or potentially on a 402 maximum volume of Logging information that the dCDN ought to keep. 403 If this volume is exceeded, the dCDN is expected to alert the uCDN 404 but may not keep more Logging information for the considered time 405 period. In addition, CDNs may aggregate Logging information and 406 transmit only summaries for some categories of operations instead of 407 the full Logging information. Note that such aggregation leads to an 408 information loss, which may be problematic for some usages of the 409 Logging information (e.g., debugging). 411 [RFC6983] discusses logging for HTTP Adaptive Streaming (HAS). In 412 accordance with the recommendations articulated there, it is expected 413 that a surrogate will generate separate Logging information for 414 delivery of each chunk of HAS content. This ensures that separate 415 Logging information can then be provided to interconnected CDNs over 416 the CDNI Logging interface. Still in line with the recommendations 417 of [RFC6983], the Logging information for per-chunck delivery may 418 include some information (a Content Collection IDentifier and a 419 Session IDentifier) intended to facilitate subsequent post-generation 420 aggregation of per-chunk logs into per-session logs. Note that a CDN 421 may also elect to generate aggregate per-session logs when performing 422 HAS delivery, but this needs to be in addition to, and not instead 423 of, the per-chunk delivery logs. We note that aggregate per-session 424 logs for HAS delivery are for further study and outside the scope of 425 this document. 427 2.2.2. Logging Collection 429 This is the process that continuously collects Logging information 430 generated by the log-generating entities within a CDN. 432 In a CDNI environment, in addition to collecting Logging information 433 from log-generating entities within the local CDN, the Collection 434 process also collects Logging information provided by another CDN, or 435 other CDNs, through the CDNI Logging interface. This is illustrated 436 in Figure 2 where we see that the Collection process of the uCDN 437 collects Logging information from log-generating entities within the 438 uCDN as well as Logging information coming from the dCDNs through the 439 CDNI Logging interface. 441 2.2.3. Logging Filtering 443 A CDN may be required to only present different subsets of the whole 444 Logging information collected to various log-consuming applications. 445 This is achieved by the Filtering process. 447 In particular, the Filtering process can also filter the right subset 448 of Logging information that needs to be provided to a given 449 interconnected CDN. For example, the filtering process in the dCDN 450 can be used to ensure that only the Logging information related to 451 tasks performed on behalf of a given uCDN are made available to that 452 uCDN (thereby filtering out all the Logging information related to 453 deliveries by the dCDN of content for its own CSPs). Similarly, the 454 Filtering process may filter or partially mask some fields, for 455 example, to protect End Users' privacy when communicating CDNI 456 Logging information to another CDN. Filtering of Logging information 457 prior to communication of this information to other CDNs via the CDNI 458 Logging interface requires that the downstream CDN can recognize the 459 subset of Logging information that relate to each interconnected CDN. 461 The CDN will also filter some internal scope information such as 462 information related to its internal alarms (security, failures, load, 463 etc). 465 In some use cases described in [RFC6770], the interconnected CDNs do 466 not want to disclose details on their internal topology. The 467 filtering process can then also filter confidential data on the 468 dCDNs' topology (number of servers, location, etc.). In particular, 469 information about the requests served by each Surrogate may be 470 confidential. Therefore, the Logging information needs to be 471 protected so that data such as Surrogates' hostnames are not 472 disclosed to the uCDN. In the "Inter-Affiliates Interconnection" use 473 case, this information may be disclosed to the uCDN because both the 474 dCDN and the uCDN are operated by entities of the same group. 476 2.2.4. Logging Rectification and Post-Generation Aggregation 478 If Logging information is generated periodically, it is important 479 that the sessions that start in one Logging period and end in another 480 are correctly reported. If they are reported in the starting period, 481 then the Logging information of this period will be available only 482 after the end of the session, which delays the Logging information 483 generation. A simple approach is to provide the complete Logging 484 Record for a session in the Logging Period of the session end. 486 A Logging rectification/update mechanism could be useful to reach a 487 good trade-off between the Logging information generation delay and 488 the Logging information accuracy. 490 In the presence of HAS, some log-consuming applications can benefit 491 from aggregate per-session logs. For example, for analytics, per- 492 session logs allow display of session-related trends which are much 493 more meaningful for some types of analysis than chunk-related trends. 494 In the case where aggregate logs have been generated directly by the 495 log-generating entities, those can be used by the applications. In 496 the case where aggregate logs have not been generated, the 497 Rectification process can be extended with a Post-Generation 498 Aggregation process that generates per-session logs from the per- 499 chunk logs, possibly leveraging the information included in the per- 500 chunk logs for that purpose (Content Collection IDentifier and a 501 Session IDentifier). However, in accordance with [RFC6983], this 502 document does not define exchange of such aggregate logs on the CDNI 503 Logging interface. We note that this is for further study and 504 outside the scope of this document. 506 2.2.5. Log-Consuming Applications 508 2.2.5.1. Maintenance/Debugging 510 Logging information is useful to permit the detection (and limit the 511 risk) of content delivery failures. In particular, Logging 512 information facilitates the detection of configuration issues. 514 To detect faults, Logging information needs to report success and 515 failure of CDN delivery operations. The uCDN can summarize such 516 information into KPIs. For instance, Logging information needs to 517 allow the computation of the number of times, during a given time 518 period, that content delivery related to a specific service succeeds/ 519 fails. 521 Logging information enables the CDN providers to identify and 522 troubleshoot performance degradations. In particular, Logging 523 information enables tracking of traffic data (e.g., the amount of 524 traffic that has been forwarded by a dCDN on behalf of an uCDN over a 525 given period of time), which is particularly useful for CDN and 526 network planning operations. 528 2.2.5.2. Accounting 530 Logging information is essential for accounting, to permit inter-CDN 531 billing and CSP billing by uCDNs. For instance, Logging information 532 provided by dCDNs enables the uCDN to compute the total amount of 533 traffic delivered by every dCDN for a particular Content Provider, as 534 well as, the associated bandwidth usage (e.g., peak, 95th 535 percentile), and the maximum number of simultaneous sessions over a 536 given period of time. 538 2.2.5.3. Analytics and Reporting 540 The goal of analytics is to gather any relevant information to track 541 audience, analyze user behavior, and monitor the performance and 542 quality of content delivery. For instance, Logging information 543 enables the CDN providers to report on content consumption (e.g., 544 delivered sessions per content) in a specific geographic area. 546 The goal of reporting is to gather any relevant information to 547 monitor the performance and quality of content delivery and allow 548 detection of delivery issues. For instance, reporting could track 549 the average delivery throughput experienced by End Users in a given 550 region for a specific CSP or content set over a period of time. 552 2.2.5.4. Security 554 The goal of security is to prevent and monitor unauthorized access, 555 misuse, modification, and denial of access of a service. A set of 556 information is logged for security purposes. In particular, a record 557 of access to content is usually collected to permit the CSP to detect 558 infringements of content delivery policies and other abnormal End 559 User behaviors. 561 2.2.5.5. Legal Logging Duties 563 Depending on the country considered, the CDNs may have to retain 564 specific Logging information during a legal retention period, to 565 comply with judicial requisitions. 567 2.2.5.6. Notions common to multiple Log Consuming Applications 569 2.2.5.6.1. Logging Information Views 571 Within a given log-consuming application, different views may be 572 provided to different users depending on privacy, business, and 573 scalability constraints. 575 For example, an analytics tool run by the uCDN can provide one view 576 to an uCDN operator that exploits all the Logging information 577 available to the uCDN, while the tool may provide a different view to 578 each CSP exploiting only the Logging information related to the 579 content of the given CSP. 581 As another example, maintenance and debugging tools may provide 582 different views to different CDN operators, based on their 583 operational role. 585 2.2.5.6.2. Key Performance Indicators (KPIs) 587 This section presents, for explanatory purposes, a non-exhaustive 588 list of Key Performance Indicators (KPIs) that can be extracted/ 589 produced from logs. 591 Multiple log-consuming applications, such as analytics, monitoring, 592 and maintenance applications, often compute and track such KPIs. 594 In a CDNI environment, depending on the situation, these KPIs may be 595 computed by the uCDN or by the dCDN. But it is usually the uCDN that 596 computes KPIs, because the uCDN and dCDN may have different 597 definitions of the KPIs and the computation of some KPIs requires a 598 vision of all the deliveries performed by the uCDN and all its dCDNs. 600 Here is a list of important examples of KPIs: 602 o Number of delivery requests received from End Users in a given 603 region for each piece of content, during a given period of time 604 (e.g., hour/day/week/month) 606 o Percentage of delivery successes/failures among the aforementioned 607 requests 609 o Number of failures listed by failure type (e.g., HTTP error code) 610 for requests received from End Users in a given region and for 611 each piece of content, during a given period of time (e.g., 612 hour/day/week/month) 614 o Number and cause of premature delivery termination for End Users 615 in a given region and for each piece of content, during a given 616 period of time (e.g., hour/day/week/month) 618 o Maximum and mean number of simultaneous sessions established by 619 End Users in a given region, for a given Content Provider, and 620 during a given period of time (e.g., hour/day/week/month) 622 o Volume of traffic delivered for sessions established by End Users 623 in a given region, for a given Content Provider, and during a 624 given period of time (e.g., hour/day/week/month) 626 o Maximum, mean, and minimum delivery throughput for sessions 627 established by End Users in a given region, for a given Content 628 Provider, and during a given period of time (e.g., hour/day/week/ 629 month) 631 o Cache-hit and byte-hit ratios for requests received from End Users 632 in a given region for each piece of content, during a given period 633 of time (e.g., hour/day/week/month) 635 o Top 10 most popularly requested contents (during a given day/week/ 636 month) 638 o Terminal type (mobile, PC, STB, if this information can be 639 acquired from the browser type header, for example). 641 Additional KPIs can be computed from other sources of information 642 than the Logging information, for instance, data collected by a 643 content portal or by specific client-side application programming 644 interfaces. Such KPIs are out of scope for the present document. 646 The KPIs used depend strongly on the considered log-consuming 647 application -- the CDN operator may be interested in different 648 metrics than the CSP is. In particular, CDN operators are often 649 interested in delivery and acquisition performance KPIs, information 650 related to Surrogates' performance, caching information to evaluate 651 the cache-hit ratio, information about the delivered file size to 652 compute the volume of content delivered during peak hour, etc. 654 Some of the KPIs, for instance those providing an instantaneous 655 vision of the active sessions for a given CSP's content, are useful 656 essentially if they are provided in a timely manner. By contrast, 657 some other KPIs, such as those averaged on a long period of time, can 658 be provided in non-real-time. 660 3. CDNI Logging File 662 3.1. Rules 664 This specification uses the Augmented Backus-Naur Form (ABNF) 665 notation and core rules of [RFC5234]. In particular, the present 666 document uses the following rules from [RFC5234]: 668 CR = %x0D ; carriage return 670 ALPHA = %x41-5A / %x61-7A ; A-Z / a-z 672 DIGIT = %x30-39 ; 0-9 674 DQUOTE = %x22 ; " (Double Quote) 676 CRLF = CR LF ; Internet standard newline 678 HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F" 680 HTAB = %x09 ; horizontal tab 682 LF = %x0A ; linefeed 684 OCTET = %x00-FF ; 8 bits of data 686 The present document also uses the following rules from [RFC3986]: 688 host = as specified in section 3.2.2 of [RFC3986]. 690 IPv4address = as specified in section 3.2.2 of [RFC3986]. 692 IPv6address = as specified in section 3.2.2 of [RFC3986]. 694 The present document also defines the following additional rules: 696 ADDRESS = IPv4address / IPv6address 698 ALPHANUM = ALPHA / DIGIT 700 DATE = 4DIGIT "-" 2DIGIT "-" 2DIGIT 702 Dates are recorded in the format YYYY-MM-DD where YYYY, MM and 703 DD stand for the numeric year, month and day respectively. All 704 dates are specified in Universal Time Coordinated (UTC). 706 DEC = 1*DIGIT ["." *DIGIT] 708 NAMEFORMAT = ALPHANUM *(ALPHANUM / "_" / "-") 710 QSTRING = DQUOTE *NDQUOTE DQUOTE ; where 712 NDQUOTE = / 2DQUOTE ; whereby a 713 DQUOTE is conveyed inside a QSTRING unambiguously by repeating 714 it. 716 NHTABSTRING = *NHTAB ; where 718 NHTAB = 720 TIME = 2DIGIT ":" 2DIGIT ":" 2DIGIT ["." *DIGIT] 722 Times are recorded in the form HH:MM:SS or HH:MM:SS.S where HH 723 is the hour in 24 hour format, MM is minutes and SS is seconds. 724 All times are specified in Universal Time Coordinated (UTC). 726 3.2. CDNI Logging File Structure 728 As defined in Section 1.1: a CDNI Logging Field is as an atomic 729 logging information element, a CDNI Logging Record is a collection of 730 CDNI Logging Fields containing all logging information corresponding 731 to a single logging event, and a CDNI Logging File contains a 732 collection of CDNI Logging Records. This structure is illustrated in 733 Figure 3. The use of a file structure for transfer of CDNI Logging 734 information is selected since this is the most common practise today 735 for exchange of logging information within and across CDNs. 737 +----------------------------------------------------------+ 738 |CDNI Logging File | 739 | | 740 | #Directive 1 | 741 | #Directive 2 | 742 | ... | 743 | #Directive P | 744 | | 745 | +------------------------------------------------------+ | 746 | |CDNI Logging Record 1 | | 747 | | +-------------+ +-------------+ +-------------+ | | 748 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 749 | | | Field 1 | | Field 2 | | Field N | | | 750 | | +-------------+ +-------------+ +-------------+ | | 751 | +------------------------------------------------------+ | 752 | | 753 | +------------------------------------------------------+ | 754 | |CDNI Logging Record 2 | | 755 | | +-------------+ +-------------+ +-------------+ | | 756 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 757 | | | Field 1 | | Field 2 | | Field N | | | 758 | | +-------------+ +-------------+ +-------------+ | | 759 | +------------------------------------------------------+ | 760 | | 761 | ... | 762 | | 763 | #Directive P+1 | 764 | | 765 | ... | 766 | | 767 | +------------------------------------------------------+ | 768 | |CDNI Logging Record M | | 769 | | +-------------+ +-------------+ +-------------+ | | 770 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 771 | | | Field 1 | | Field 2 | | Field N | | | 772 | | +-------------+ +-------------+ +-------------+ | | 773 | +------------------------------------------------------+ | 774 | | 775 | | 776 | #Directive P+Q | 777 +----------------------------------------------------------+ 779 Figure 3: Structure of Logging Files 781 The CDNI Logging File format is inspired from the W3C Extended Log 782 File Format [ELF]. However, it is fully specified by the present 783 document. Where the present document differs from the W3C Extended 784 Log File Format, an implementation of the CDNI Logging interface MUST 785 comply with the present document. 787 Using a format that resembles the W3C Extended Log File Format is 788 intended to keep CDNI logging format close to the intra-CDN Logging 789 information format commonly used in CDNs today, thereby minimizing 790 systematic translation at CDN/CDNI boundary. 792 A CDNI Logging File MUST contain a sequence of lines containing US- 793 ASCII characters [CHAR_SET] terminated by CRLF. 795 Each line of a CDNI Logging File MUST contain either a directive or a 796 CDNI Logging Record. 798 Directives record information about the CDNI Logging process itself. 799 Lines containing directives MUST begin with the "#" character. 800 Directives are specified in Section 3.3. 802 Logging Records provide actual details of the logged event. Logging 803 Records are specified in Section 3.4. 805 The CDNI File structure is defined by the following rules: 807 DIRLINE = "#" directive CRLF 809 DIRGROUP = 1*DIRLINE 811 RECLINE = CRLF 813 RECGROUP = *RECLINE 815 = 1* 817 3.3. CDNI Logging File Directives 819 The CDNI Logging File directives are defined by the following rules: 821 directive = DIRNAME ":" HTAB DIRVAL 823 DIRNAME = 826 DIRVAL = 830 An implementation of the CDNI Logging interface MUST support all of 831 the following directives, listed below by their directive name: 833 o Version: 835 * format: "CDNI" "/" 1*DIGIT "." 1*DIGIT 837 * directive value: indicates the version of the CDNI Logging File 838 format. The value MUST be "CDNI/1.0" for the version specified 839 in the present document. 841 * occurrence: there MUST be one and only one instance of this 842 directive per CDNI Logging File. It MUST be the first line of 843 the CDNI Logging File. 845 o UUID: 847 * format: NHTABSTRING 849 * directive value: this a Universally Unique IDentifier (UUID) 850 from the UUID Uniform Resource Name (URN) namespace specified 851 in [RFC4122]) for the CDNI Logging File. 853 * occurrence: there MUST be one and only one instance of this 854 directive per CDNI Logging File. 856 o Claimed-Origin: 858 * format: host 860 * directive value: this contains the claimed identification of 861 the entity transmitting the CDNI Logging File (e.g., the host 862 in a dCDN supporting the CDNI Logging interface) or the entity 863 responsible for transmitting the CDNI Logging File (e.g., the 864 dCDN). 866 * occurrence: there MUST be zero or exactly one instance of this 867 directive per CDNI Logging File. This directive MAY be 868 included by the dCDN. It MUST NOT be included or modified by 869 the uCDN. 871 o Established-Origin: 873 * format: host 875 * directive value: this contains the identification, as 876 established by the entity receiving the CDNI Logging File, of 877 the entity transmitting the CDNI Logging File (e.g., the host 878 in a dCDN supporting the CDNI Logging interface) or the entity 879 responsible for transmitting the CDNI Logging File (e.g., the 880 dCDN). 882 * occurrence: there MUST be zero or exactly one instance of this 883 directive per CDNI Logging File. This directive MAY be added 884 by the uCDN (e.g., before storing the CDNI Logging File). It 885 MUST NOT be included by the dCDN. The mechanisms used by the 886 uCDN to establish and validate the entity responsible for the 887 CDNI Logging File is outside the scope of the present document. 888 We observe that, in particular, this may be achieved through 889 authentication mechanisms that are part of the transport layer 890 of the CDNI Logging File pull mechanism (Section 4.2). 892 o Record-Type: 894 * format: NAMEFORMAT 896 * directive value: indicates the type of the CDNI Logging Records 897 that follow this directive, until another Record-Type directive 898 (or the end of the CDNI Logging File). This can be any CDNI 899 Logging Record type registered in the CDNI Logging Record-types 900 registry (Section 6.2). For example this may be 901 "cdni_http_request_v1" as specified in Section 3.4.1. 903 * occurrence: there MUST be at least one instance of this 904 directive per CDNI Logging File. The first instance of this 905 directive MUST precede a Fields directive and MUST precede all 906 CDNI Logging Records. 908 o Fields: 910 * format: FIENAME * ; where FIENAME can take any 911 CDNI Logging field name registered in the CDNI Logging Field 912 Names registry (Section 6.3). 914 * directive value: this lists the names of all the fields for 915 which a value is to appear in the CDNI Logging Records that 916 follow the instance of this directive (until another instance 917 of this directive). The names of the fields, as well as their 918 occurrences, MUST comply with the corresponding rules specified 919 in the document referenced in the CDNI Logging Record-types 920 registry (Section 6.2) for the corresponding CDNI Logging 921 Record-Type. 923 * occurrence: there MUST be at least one instance of this 924 directive per Record-Type directive. The first instance of 925 this directive for a given Record-Type MUST appear before any 926 CDNI Logging Record for this Record-Type. One situation where 927 more than one instance of the Fields directive can appear 928 within a given CDNI Logging File, is when there is a change, in 929 the middle of a fairly large logging period, in the agreement 930 between the uCDN and the dCDN about the set of Fields that are 931 to be exchanged. The multiple occurences allow records with 932 the old set of fields and records with the new set of fields to 933 be carried inside the same Logging File. 935 o Integrity-Hash: 937 * format: 32HEXDIG 939 * directive value: This directive permits the detection of a 940 corrupted CDNI Logging File. This can be useful, for instance, 941 if a problem occurs on the filesystem of the dCDN Logging 942 system and leads to a truncation of a logging file. The valid 943 Integrity-Hash value is included in this directive by the 944 entity that transmits the CDNI Logging File. It is computed by 945 applying the MD5 ([RFC1321]) cryptographic hash function on the 946 CDNI Logging File, including all the directives and logging 947 records, up to the Integrity-Hash directive itself, excluding 948 the Integrity-Hash directive itself. The Integrity-Hash value 949 is represented as a US-ASCII encoded hexadecimal number, 32 950 digits long (representing a 128 bit hash value). The entity 951 receiving the CDNI Logging File also computes in a similar way 952 the MD5 hash on the received CDNI Logging File and compares 953 this hash to the value of the Integrity-Hash directive. If the 954 two values are equal, then the received CDNI Logging File MUST 955 be considered non-corrupted. If the two values are different, 956 the received CDNI Logging File MUST be considered corrupted. 957 The behavior of the entity that received a corrupted CDNI 958 Logging File is outside the scope of this specification; we 959 note that the entity MAY attempt to pull again the same CDNI 960 Logging File from the transmitting entity. If the entity 961 receiving a non-corrupted CDNI Logging File adds an 962 Established-Origin directive, it MUST then recompute and update 963 the Integrity-Hash directive so it also protects the added 964 Established-Origin directive. 966 * occurrence: there MUST be zero or exactly one instance of this 967 directive. There SHOULD be exactly one instance of this 968 directive. One situation where that directive could be omitted 969 is where integrity protection is already provided via another 970 mechanism (for example if an integrity hash is associated to 971 the CDNI Logging File out of band through the CDNI Logging Feed 972 ( Section 4.1) leveraging ATOM extensions such as those 973 proposed in [I-D.snell-atompub-link-extensions]. When present, 974 the Integrity-Hash field MUST be the last line of the CDNI 975 Logging File. 977 3.4. CDNI Logging Records 979 A CDNI Logging Record consists of a sequence of CDNI Logging Fields 980 relating to that single CDNI Logging Record. 982 CDNI Logging Fields MUST be separated by the "horizontal tabulation 983 (HTAB)" character. 985 To facilitate readability, a prefix scheme is used for CDNI Logging 986 field names in a similar way to the one used in W3C Extended Log File 987 Format [ELF]. The semantics of the prefix in the present document 988 is: 990 o c: refers to the User Agent that issues the request (corresponds 991 to the "client" of W3C Extended Log Format) 993 o d: refers to the dCDN (relative to a given CDN acting as a uCDN) 995 o s: refers to the dCDN Surrogate that serves the request 996 (corresponds to the "server" of W3C Extended Log Format) 998 o u: refers to the uCDN (relative to a given CDN acting as a dCDN) 1000 o cs: refers to communication from the User Agent towards the dCDN 1001 Surrogate 1003 o sc: refers to communication from the dCDN Surrogate towards the 1004 User Agent 1006 An implementation of the CDNI Logging interface as per the present 1007 specification MUST support the CDNI HTTP Request Logging Record as 1008 specified in Section 3.4.1. 1010 A CDNI Logging Record is defined by the following rules: 1012 FIEVAL = 1014 = FIEVAL * ; where FIEVAL 1015 contains the CDNI Logging field value corresponding to the CDNI 1016 Logging field names (FIENAME) listed is the last Fields directive 1017 preceding the present CDNI Logging Record. 1019 3.4.1. HTTP Request Logging Record 1021 This section defines the CDNI Logging Record of Record-Type 1022 "cdni_http_request_v1". It is applicable to content delivery 1023 performed by the dCDN using HTTP/1.0([RFC1945]), 1024 HTTP/1.1([RFC7230],[RFC7231], [RFC7232], [RFC7233], [RFC7234], 1026 [RFC7235]) or HTTPS ([RFC2818], [RFC7230]). We observe that, in the 1027 case of HTTPS delivery, there may be value in logging additional 1028 information specific to the operation of HTTP over TLS and we note 1029 that this is outside the scope of the present document and may be 1030 addressed in a future document defining another CDNI Logging Record 1031 or another version of the HTTP Request Logging Record. 1033 The "cdni_http_request_v1" Record-Type is also expected to be 1034 applicable to HTTP/2.0 [I-D.ietf-httpbis-http2] (which is still under 1035 development at the time of writing the present document) since a 1036 fundamental design tenet of HTTP/2.0 is to preserve the HTTP/1.1 1037 semantics. We observe that, in the case of HTTP/2.0 delivery, there 1038 may be value in logging additional information specific to the 1039 additional functionality of HTTP/2.0 (e.g. information related to 1040 connection identification, to stream identification, to stream 1041 priority and to flow control). We note that such additional 1042 information is outside the scope of the present document and may be 1043 addressed in a future document defining another CDNI Logging Record 1044 or another version of the HTTP Request Logging Record. 1046 The "cdni_http_request_v1" Record-Type contains the following CDNI 1047 Logging Fields, listed by their field name: 1049 o date: 1051 * format: DATE 1053 * field value: the date at which the processing of request 1054 completed on the Surrogate. 1056 * occurrence: there MUST be one and only one instance of this 1057 field. 1059 o time: 1061 * format: TIME 1063 * field value: the time at which the processing of request 1064 completed on the Surrogate. 1066 * occurrence: there MUST be one and only one instance of this 1067 field. 1069 o time-taken: 1071 * format: DEC 1072 * field value: decimal value of the duration, in seconds, between 1073 the start of the processing of the request and the completion 1074 of the request processing (e.g., completion of delivery) by the 1075 Surrogate. 1077 * occurrence: there MUST be one and only one instance of this 1078 field. 1080 o c-ip: 1082 * format: ADDRESS 1084 * field value: the source IPv4 or IPv6 address (i.e., the 1085 "client" address) in the request received by the Surrogate. 1087 * occurrence: there MUST be one and only one instance of this 1088 field. 1090 o c-ip-anonymizing: 1092 * format: 1*DIGIT 1094 * field value: the number of rightmost bits of the address in the 1095 c-ip field that are zeroed-out in order to anonymize the 1096 logging record. The mechanism by which the two ends of the 1097 CDNI Logging interface agree on whether anonymization is to be 1098 supported and the number of bits that need to be zeroed-out for 1099 this purpose are outside the scope of the present document. 1101 * occurrence: there MUST be zero or exactly one instance of this 1102 field. 1104 o c-port: 1106 * format: 1*DIGIT 1108 * field value: the source TCP port (i.e., the "client" port) in 1109 the request received by the Surrogate. 1111 * occurrence: there MUST be zero or exactly one instance of this 1112 field. 1114 o s-ip: 1116 * format: ADDRESS 1118 * field value: the IPv4 or IPv6 address of the Surrogate that 1119 served the request (i.e., the "server" address). 1121 * occurrence: there MUST be zero or exactly one instance of this 1122 field. 1124 o s-hostname: 1126 * format: host 1128 * field value: the hostname of the Surrogate that served the 1129 request (i.e., the "server" hostname). 1131 * occurrence: there MUST be zero or exactly one instance of this 1132 field. 1134 o s-port: 1136 * format: 1*DIGIT 1138 * field value: the destination TCP port (i.e., the "server" port) 1139 in the request received by the Surrogate. 1141 * occurrence: there MUST be zero or exactly one instance of this 1142 field. 1144 o cs-method: 1146 * format: NHTABSTRING 1148 * field value: this is the method of the request received by the 1149 Surrogate. In the case of HTTP delivery, this is the HTTP 1150 method in the request. 1152 * occurrence: There MUST be one and only one instance of this 1153 field. 1155 o cs-uri: 1157 * format: NHTABSTRING 1159 * field value: this is the "effective request URI" of the request 1160 received by the Surrogate as specified in [RFC7230]. It 1161 complies with the "http" URI scheme or the "https" URI scheme 1162 as specified in [RFC7230]). 1164 * occurrence: there MUST be zero or exactly one instance of this 1165 field. 1167 o u-uri: 1169 * format: NHTABSTRING 1171 * field value: this is a complete URI, derived from the 1172 "effective request URI" ([RFC7230]) of the request received by 1173 the Surrogate (i.e., the cs-uri) but transformed by the entity 1174 generating or transmitting the CDNI Logging Record, in a way 1175 that is agreed upon between the two ends of the CDNI Logging 1176 interface, so the transformed URI is meaningful to the uCDN. 1177 For example, the two ends of the CDNI Logging interface could 1178 agree that the u-uri is constructed from the cs-uri by removing 1179 the part of the hostname that exposes which individual 1180 Surrogate actually performed the delivery. The details of 1181 modification performed to generate the u-uri, as well as the 1182 mechanism to agree on these modifications between the two sides 1183 of the CDNI Logging interface are outside the scope of the 1184 present document. 1186 * occurrence: there MUST be one and only one instance of this 1187 field. 1189 o protocol: 1191 * format: NHTABSTRING 1193 * field value: this is value of the HTTP-Version field as 1194 specified in [RFC7230] of the Request-Line of the request 1195 received by the Surrogate (e.g., "HTTP/1.1"). 1197 * occurrence: there MUST be one and only one instance of this 1198 field. 1200 o sc-status: 1202 * format: 3DIGIT 1204 * field value: this is the Status-Code in the response from the 1205 Surrogate. In the case of HTTP delivery, this is the HTTP 1206 Status-Code in the HTTP response. 1208 * occurrence: There MUST be one and only one instance of this 1209 field. 1211 o sc-total-bytes: 1213 * format: 1*DIGIT 1215 * field value: this is the total number of bytes of the response 1216 sent by the Surrogate in response to the request. In the case 1217 of HTTP delivery, this includes the bytes of the Status-Line, 1218 the bytes of the HTTP headers and the bytes of the message- 1219 body. 1221 * occurrence: There MUST be one and only one instance of this 1222 field. 1224 o sc-entity-bytes: 1226 * format: 1*DIGIT 1228 * field value: this is the number of bytes of the message-body in 1229 the HTTP response sent by the Surrogate in response to the 1230 request. This does not include the bytes of the Status-Line or 1231 the bytes of the HTTP headers. 1233 * occurrence: there MUST be zero or exactly one instance of this 1234 field. 1236 o cs(): 1238 * format: QSTRING 1240 * field value: the value of the HTTP header (identified by the 1241 in the CDNI Logging field name) as it 1242 appears in the request processed by the Surrogate, but 1243 prepended by a DQUOTE and appended by a DQUOTE. For example, 1244 when the CDNI Logging field name (FIENAME) listed in the 1245 preceding Fields directive is cs(User-Agent), this CDNI Logging 1246 field value contains the value of the User-Agent HTTP header as 1247 received by the Surrogate in the request it processed, but 1248 prepended by a DQUOTE and appended by a DQUOTE. If the HTTP 1249 header as it appeared in the request processed by the Surrogate 1250 contains one or more DQUOTE, each DQUOTE MUST be escaped by an 1251 additional DQUOTE. For example, if the HTTP header contains 1252 My_Header"value", then the field value of the cs() is "My_Header""value""". 1255 * occurrence: there MAY be zero, one or any number of instance of 1256 this field. 1258 o sc(): 1260 * format: QSTRING 1262 * field value: the value of the HTTP header (identified by the 1263 in the CDNI Logging field name) as it 1264 appears in the response issued by the Surrogate to serve the 1265 request, but prepended by a DQUOTE and appended by a DQUOTE. 1266 If the HTTP header as it appeared in the request processed by 1267 the Surrogate contains one or more DQUOTE, each DQUOTE MUST be 1268 escaped by an additional DQUOTE. For example, if the HTTP 1269 header contains My_Header"value", then the field value of the 1270 cs() is "My_Header""value""". 1272 * occurrence: there MAY be zero, one or any number of instances 1273 of this field. For a given , there MUST be 1274 zero or exactly one instance of this field. 1276 o s-ccid: 1278 * format: QSTRING 1280 * field value: this contains the value of the Content Collection 1281 IDentifier (CCID) associated by the uCDN to the content served 1282 by the Surrogate via the CDNI Metadata interface 1283 ([I-D.ietf-cdni-metadata]), prepended by a DQUOTE and appended 1284 by a DQUOTE. If the CCID conveyed in the CDNI Metadata 1285 interface contains one or more DQUOTE, each DQUOTE MUST be 1286 escaped by an additional DQUOTE. For example, if the CCID 1287 conveyed in the CDNI Metadata interface is My_CCIDD"value", 1288 then the field value of the s-ccid is "My_CCID""value""". 1290 * occurrence: there MUST be zero or exactly one instance of this 1291 field. For a given , there MUST be zero or 1292 exactly one instance of this field. 1294 o s-sid: 1296 * format: QSTRING 1298 * field value: this contains the value of a Session IDentifier 1299 (SID) generated by the dCDN for a specific HTTP session, 1300 prepended by a DQUOTE and appended by a DQUOTE. In particular, 1301 for HTTP Adaptive Streaming (HAS) session, the Session 1302 IDentifier value is included in the Logging record for every 1303 content chunk delivery of that session in view of facilitating 1304 the later correlation of all the per content chunk log records 1305 of a given HAS session. See section 3.4.2.2. of [RFC6983] for 1306 more discussion on the concept of Session IDentifier in the 1307 context of HAS. If the SID conveyed contains one or more 1308 DQUOTE, each DQUOTE MUST be escaped by an additional DQUOTE. 1309 For example, if the SID is My_SID"value", then the field value 1310 of the s-sid is "My_SID""value""". 1312 * occurrence: there MUST be zero or exactly one instance of this 1313 field. 1315 o s-cached: 1317 * format: 1DIGIT 1319 * field value: this characterises whether the Surrogate served 1320 the request using content already stored on its local cache or 1321 not. The allowed values are "0" (for miss) and "1" (for hit). 1322 "1" MUST be used when the Surrogate did serve the request using 1323 exclusively content already stored on its local cache. "0" MUST 1324 be used otherwise (including cases where the Surrogate served 1325 the request using some, but not all, content already stored on 1326 its local cache). Note that a "0" only means a cache miss in 1327 the Surrogate and does not provide any information on whether 1328 the content was already stored, or not, in another device of 1329 the dCDN, i.e., whether this was a "dCDN hit" or "dCDN miss". 1331 * occurrence: there MUST be zero or exactly one instance of this 1332 field. 1334 The "Fields" directive corresponding to a HTTP Request Logging Record 1335 MUST contain all the fields names whose occurrence is specified above 1336 as "There MUST be one and only one instance of this field". The 1337 corresponding fields value MUST be present in every HTTP Request 1338 Logging Record. 1340 The "Fields" directive corresponding to a HTTP Request Logging Record 1341 MAY list all the fields value whose occurrence is specified above as 1342 "there MUST be zero or exactly one instance of this field" or "there 1343 MAY be zero, one or any number of instances of this field". The set 1344 of such field names actually listed in the "Fields" directive is 1345 selected by the CDN generating the CDNI Logging File based on 1346 agreements between the interconnected CDNs established through 1347 mechanisms outside the scope of this specification (e.g., contractual 1348 agreements). When such a field name is not listed in the "Fields" 1349 directive, the corresponding field value MUST NOT be included in the 1350 Logging Record. When such a field name is listed in the "Fields" 1351 directive, the corresponding field value MUST be included in the 1352 Logging Record; if the value for the field is not available, this 1353 MUST be conveyed via a dash character ("-"). 1355 The fields names listed in the "Fields" directive MAY be listed in 1356 the order in which they are listed in Section 3.4.1 or MAY be listed 1357 in any other order. 1359 A dCDN-side implementation of the CDNI Logging interface MUST 1360 implement all the following Logging Fields in a CDNI Logging Record 1361 of Record-Type "cdni_http_request_v1", and MUST support the ability 1362 to include valid values for each of them: 1364 o date 1366 o time 1368 o time-taken 1370 o c-ip 1372 o c-port 1374 o s-ip 1376 o s-hostname 1378 o s-port 1380 o cs-method 1382 o cs-uri 1384 o u-uri 1386 o protocol 1388 o sc-status 1390 o sc-total-bytes 1392 o sc-entity-bytes 1394 o cs() 1396 o sc() 1398 o s-cached 1400 A dCDN-side implementation of the CDNI Logging interface MAY support 1401 the following Logging Fields in a CDNI Logging Record of Record-Type 1402 "cdni_http_request_v1": 1404 o c-ip-anonymizing 1406 o s-ccid 1407 o s-sid 1409 If a dCDN-side implementation of the CDNI Logging interface supports 1410 these Fields, it MUST support the ability to include valid values for 1411 them. 1413 An uCDN-side implementation of the CDNI Logging interface MUST be 1414 able to accept CDNI Logging Files with CDNI Logging Records of 1415 Record-Type "cdni_http_request_v1" containing any CDNI Logging Field 1416 defined in Section 3.4.1 as long as the CDNI Logging Record and the 1417 CDNI Logging File are compliant with the present document. 1419 3.5. CDNI Logging File Example 1421 Let us consider the upstream CDN and the downstream CDN labelled uCDN 1422 and dCDN-1 in Figure 1. When dCDN-1 acts as a downstream CDN for 1423 uCDN and performs content delivery on behalf of uCDN, dCDN-1 will 1424 include the CDNI Logging Records corresponding to the content 1425 deliveries performed on behalf of uCDN in the CDNI Logging Files for 1426 uCDN. An example CDNI Logging File communicated by dCDN-1 to uCDN is 1427 shown below in Figure 4. 1429 #Version:CDNI/1.0 1431 #UUID:urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6 1433 #Claimed-Origin:cdni-logging-entity.dcdn-1.example.com 1435 #Record-Type:cdni_http_request_v1 1437 #Fields:datetimetime-takenc-ip 1438 cs-methodu-uriprotocolsc-status 1439 sc-total-bytescs(User-Agent)cs(Referer) 1440 s-cached 1442 2013-05-1700:38:06.8259.05810.5.7.1GET 1443 http://cdni-ucdn.dcdn-1.example.com/video/movie100.mp4 1444 HTTP/1.12006729891"Mozilla/5.0 1445 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1446 Gecko) Chrome/5.0.375.127 Safari/533.4" 1447 "host1.example.com"1 1449 2013-05-1700:39:09.14515.3210.5.10.5GET 1450 http://cdni-ucdn.dcdn-1.example.com/video/movie118.mp4 1451 HTTP/1.120015799210"Mozilla/5.0 1452 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1453 Gecko) Chrome/5.0.375.127 Safari/533.4" 1454 "host1.example.com"1 1456 2013-05-1700:42:53.43752.87910.5.10.5GET 1457 http://cdni-ucdn.dcdn-1.example.com/video/picture11.mp4 1458 HTTP/1.020097234724"Mozilla/5.0 1459 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1460 Gecko) Chrome/5.0.375.127 Safari/533.4" 1461 "host5.example.com"0 1463 #Integrity-Hash:...32-hexadecimal-digit hash value... 1465 Figure 4: CDNI Logging File Example 1467 If uCDN establishes by some means (e.g. via TLS authentication when 1468 pulling the CDNI Logging File) the identity of the entity from which 1469 it pulled the CDNI Logging File, uCDN can add to the CDNI Logging an 1470 Established-Origin directive as illustrated below: 1472 #Established-Origin:cdni-logging-entity.dcdn- 1473 1.example.com 1475 As illustrated in Figure 2, uCDN will then ingest the corresponding 1476 CDNI Logging Records into its Collection process, alongside the 1477 Logging Records generated locally by the uCDN itself. This allows 1478 uCDN to aggregate Logging Records for deliveries performed by itself 1479 (through Records generated locally) as well as for deliveries 1480 performed by its downstream CDN(s). This aggregate information can 1481 then be used (after Filtering and Rectification, as illustrated in 1482 Figure 2) by Log Consuming Applications that take into account 1483 deliveries performed by uCDN as well as by all of its downstream 1484 CDNs. 1486 We observe that the time between 1488 1. when a delivery is completed in dCDN and 1490 2. when the corresponding Logging Record is ingested by the 1491 Collection process in uCDN 1493 depends on a number of parameters such as the Logging Period agreed 1494 to by uCDN and dCDN, how much time uCDN waits before pulling the CDNI 1495 Logging File once it is advertised in the CDNI Logging Feed, and the 1496 time to complete the pull of the CDNI Logging File. Therefore, if we 1497 consider the set of Logging Records aggregated by the Collection 1498 process in uCDN in a given time interval, there could be a permanent 1499 significant timing difference between the CDNI Logging Records 1500 received from the dCDN and the Logging Records generated locally. 1501 For example, in a given time interval, the Collection process in uCDN 1502 may be aggregating Logging Records generated locally by uCDN for 1503 deliveries performed in the last hour and CDNI Logging Records 1504 generated in the dCDN for deliveries in the hour before last. 1506 3.6. Cascaded CDNI Logging Files Example 1508 Let us consider the cascaded CDN scenario of uCDN, dCDN-2 and dCDN-3 1509 as depicted in Figure 1. After completion of a delivery by dCDN-3 on 1510 behalf of dCDN-2, dCDN-3 will include a corresponding Logging Record 1511 in a CDNI Logging File that will be pulled by dCDN-2 and that is 1512 illustrated below in Figure 5. In practice, a CDNI Logging File is 1513 likely to contain a very high number of CDNI Logging Records. 1514 However, for readability, the example in Figure 5 contains a single 1515 CDNI Logging Record. 1517 #Version:CDNI/1.0 1519 #UUID:urn:uuid:65718ef-0123-9876-adce4321bcde 1521 #Claimed-Origin:cdni-logging-entity.dcdn-3.example.com 1523 #Record-Type:cdni_http_request_v1 1525 #Fields:datetimetime-takenc-ip 1526 cs-methodu-uriprotocolsc-status 1527 sc-total-bytescs(User-Agent)cs(Referer) 1528 s-cached 1530 2013-05-1700:39:09.11914.0710.5.10.9GET 1531 http://cdni-dcdn-2.dcdn-3.example.com/video/movie118.mp4 1532 HTTP/1.120015799210"Mozilla/5.0 1533 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1534 Gecko) Chrome/5.0.375.127 Safari /533.4" 1535 "host1.example.com"1 1537 #Integrity-Hash:...32-hexadecimal-digit hash value... 1539 Figure 5: Cascaded CDNI Logging File Example (dCDN-3 to dCDN-2) 1541 If dCDN-2 establishes by some means (e.g. via TLS authentication when 1542 pulling the CDNI Logging File) the identity of the entity from which 1543 it pulled the CDNI Logging File, dCDN-2 can add to the CDNI Logging 1544 an Established-Origin directive as illustrated below: 1546 #Established-Origin:cdni-logging-entity.dcdn- 1547 3.example.com 1549 dCDN-2 (behaving as an upstream CDN from the viewpoint of dCDN-3) 1550 will then ingest the CDNI Logging Record for the considered dCDN-3 1551 delivery into its Collection process (as illustrated in Figure 2). 1552 This Logging Record may be aggregated with Logging Records generated 1553 locally by dCDN-2 for deliveries performed by dCDN-2 itself. Say, 1554 for illustration, that the content delivery performed by dCDN-3 on 1555 behalf of dCDN-2 had actually been redirected to dCDN-2 by uCDN, and 1556 say that another content delivery has just been redirected by uCDN to 1557 dCDN-2 and that dCDN-2 elected to perform the corresponding delivery 1558 itself. Then after Filtering and Rectification (as illustrated in 1559 Figure 2), dCDN-2 will include the two Logging Records corresponding 1560 respectively to the delivery performed by dCDN-3 and the delivery 1561 performed by dCDN-2, in the next CDNI Logging File that will be 1562 communicated to uCDN. An example of such CDNI Logging File is 1563 illustrated below in Figure 6. 1565 #Version:CDNI/1.0 1567 #UUID:urn:uuid:1234567-8fedc-abab-0987654321ff 1569 #Claimed-Origin:cdni-logging-entity.dcdn-2.example.com 1571 #Record-Type:cdni_http_request_v1 1573 #Fields:datetimetime-takenc-ip 1574 cs-methodu-uriprotocolsc-status 1575 sc-total-bytescs(User-Agent)cs(Referer) 1576 s-cached 1578 2013-05-1700:39:09.11914.0710.5.10.9GET 1579 http://cdni-ucdn.dcdn-2.example.com/video/movie118.mp4 1580 HTTP/1.120015799210"Mozilla/5.0 1581 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1582 Gecko) Chrome/5.0.375.127 Safari /533.4" 1583 "host1.example.com"1 1585 2013-05-1701:42:53.43752.87910.5.10.12GET 1586 http://cdni-ucdn.dcdn-2.example.com/video/picture11.mp4 1587 HTTP/1.020097234724"Mozilla/5.0 1588 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like 1589 Gecko) Chrome/5.0.375.127 Safari /533.4" 1590 "host5.example.com"0 1592 #Integrity-Hash:...32-hexadecimal-digit hash value... 1594 Figure 6: Cascaded CDNI Logging File Example (dCDN-2 to uCDN) 1596 If uCDN establishes by some means (e.g. via TLS authentication when 1597 pulling the CDNI Logging File) the identity of the entity from which 1598 it pulled the CDNI Logging File, uCDN can add to the CDNI Logging an 1599 Established-Origin directive as illustrated below: 1601 #Established-Origin:cdni-logging-entity.dcdn- 1602 2.example.com 1604 In the example of Figure 6, we observe that: 1606 o the first Logging Record corresponds to the Logging Record 1607 communicated earlier to dCDN-2 by dCDN-3, which corresponds to a 1608 delivery redirected by uCDN to dCDN-2 and then redirected by 1609 dCDN-2 to dCDN-3. The fields values in this Logging Record are 1610 copied from the corresponding CDNI Logging REcord communicated to 1611 dCDN2 by dCDN-3, with the exception of the u-uri that now reflects 1612 the URI convention between uCDN and dCDN-2 and that presents the 1613 delivery to uCDN as if it was performed by dCDN-2 itself. This 1614 reflects the fact that dCDN-2 had taken the full responsibility of 1615 the corresponding delivery (even if in this case, dCDN-2 elected 1616 to redirect the delivery to dCDN-3 so it is actually performed by 1617 dCDN-3 on behalf of dCDN-2). 1619 o the second Logging Record corresponds to a delivery redirected by 1620 uCDN to dCDN-2 and performed by dCDN-2 itself. The time of the 1621 delivery in this Logging Record may be significantly more recent 1622 than the first Logging Record since it was generated locally while 1623 the first Logging Record was generated by dCDN-3 and had to be 1624 advertised , and then pulled and then ingested into the dCDN-2 1625 Collection process, before being aggregated with the second 1626 Logging Record. 1628 4. Protocol for Exchange of CDNI Logging File After Full Collection 1630 This section specifies a protocol for the exchange of CDNI Logging 1631 Files as specified in Section 3 after the CDNI Logging File is fully 1632 collected by the dCDN. 1634 This protocol comprises: 1636 o a CDNI Logging feed, allowing the dCDN to notify the uCDN about 1637 the CDNI Logging Files that can be retrieved by that uCDN from the 1638 dCDN, as well as all the information necessary for retrieving each 1639 of these CDNI Logging Files. The CDNI Logging feed is specified 1640 in Section 4.1. 1642 o a CDNI Logging File pull mechanism, allowing the uCDN to obtain 1643 from the dCDN a given CDNI Logging File at the uCDN's convenience. 1644 The CDNI Logging File pull mechanisms is specified in Section 4.2. 1646 An implementation of the CDNI Logging interface on the dCDN side (the 1647 entity generating the CDNI Logging file) MUST support the server side 1648 of the CDNI Logging feed (as specified in Section 4.1) and the server 1649 side of the CDNI Logging pull mechanism (as specified in 1650 Section 4.2). 1652 An implementation of the CDNI Logging interface on the uCDN side (the 1653 entity consuming the CDNI Logging file) MUST support the client side 1654 of the CDNI Logging feed (as specified in Section 4.1) and the client 1655 side of the CDNI Logging pull mechanism (as specified in 1656 Section 4.2). 1658 4.1. CDNI Logging Feed 1660 The server-side implementation of the CDNI Logging feed MUST produce 1661 an Atom feed [RFC4287]. This feed is used to advertise log files 1662 that are available for the client-side to retrieve using the CDNI 1663 Logging pull mechanism. 1665 4.1.1. Atom Formatting 1667 A CDNI Logging feed MUST be structured as an Archived feed, as 1668 defined in [RFC5005], and MUST be formatted in Atom [RFC4287]. This 1669 means it consists of a subscription document that is regularly 1670 updated as new CDNI Logging Files become available, and information 1671 about older CDNI Logging files is moved into archive documents. Once 1672 created, archive documents are never modified. 1674 Each CDNI Logging File listed in an Atom feed MUST be described in an 1675 atom:entry container element. 1677 The atom:entry MUST contain an atom:content element whose "src" 1678 attribute is a link to the CDNI Logging File and whose "type" 1679 attribute is the MIME Media Type indicating that the entry is a CDNI 1680 Logging File. We define this MIME Media Type as "application/ 1681 cdni.LoggingFile" (See Section 6.4). 1683 For compatibility with some Atom feed readers the atom:entry MAY also 1684 contain an atom:link entry whose "href" attribute is a link to the 1685 CDNI Logging File and whose "type" attribute is the MIME Media Type 1686 indicating that the entry is a CDNI Logging File using the 1687 "application/cdni.LoggingFile" MIME Media Type (See Section 6.4). 1689 The URI used in the atom:id of the atom:entry MUST contain the UUID 1690 of the CDNI Logging File. 1692 The atom:updated in the atom:entry MUST indicate the time at which 1693 the CDNI Logging File was last updated. 1695 4.1.2. Updates to Log Files and the Feed 1697 CDNI Logging Files MUST NOT be modified by the dCDN once published in 1698 the CDNI Logging feed. 1700 The frequency with which the subscription feed is updated, the period 1701 of time covered by each CDNI Logging File or each archive document, 1702 and timeliness of publishing of CDNI Logging Files are outside the 1703 scope of the present document and are expected to be agreed upon by 1704 uCDN and dCDN via other means (e.g., human agreement). 1706 The server-side implementation MUST be able to set, and SHOULD set, 1707 HTTP cache control headers on the subscription feed to indicate the 1708 frequency at which the client-side is to poll for updates. 1710 The client-side MAY use HTTP cache control headers (set by the 1711 server-side) on the subscription feed to determine the frequency at 1712 which to poll for updates. The client-side MAY instead, or in 1713 addition, use other information to determine when to poll for updates 1714 (e.g., a polling frequency that may have been negotiated between the 1715 uCDN and dCDN by mechanisms outside the scope of the present document 1716 and that is to override the indications provided in the HTTP cache 1717 control headers). 1719 The potential retention limits (e.g., sliding time window) within 1720 which the dCDN is to retain and be ready to serve an archive document 1721 is outside the scope of the present document and is expected to be 1722 agreed upon by uCDN and dCDN via other means (e.g., human agreement). 1723 The server-side implementation MUST retain, and be ready to serve, 1724 any archive document within the agreed retention limits. Outside 1725 these agreed limits, the server-side implementation MAY indicate its 1726 inability to serve (e.g., with HTTP status code 404) an archive 1727 document or MAY refuse to serve it (e.g., with HTTP status code 403 1728 or 410). 1730 4.1.3. Redundant Feeds 1732 The server-side implementation MAY present more than one CDNI Logging 1733 feed for redundancy. Each CDNI Logging File MAY be published in more 1734 than one feed. 1736 A client-side implementation MAY support such redundant CDNI Logging 1737 feeds. If it supports redundant CDNI Logging feed, the client-side 1738 can use the UUID of the CDNI Logging File, presented in the atom:id 1739 element of the Atom feed, to avoid unnecessarily pulling and storing 1740 a given CDNI Logging File more than once. 1742 4.1.4. Example CDNI Logging Feed 1744 Figure 7 illustrates an example of the subscription document of a 1745 CDNI Logging feed. 1747 1748 1749 CDNI Logging Feed 1750 2013-03-23T14:46:11Z 1751 urn:uuid:663ae677-40fb-e99a-049d-c5642916b8ce 1752 1754 1756 1758 CDNI Log Feed 1759 Generator 1760 dcdn.example 1761 1762 CDNI Logging File for uCDN at 1763 2013-03-23 14:15:00 1764 urn:uuid:12345678-1234-abcd-00aa-01234567abcd 1765 2013-03-23T14:15:00Z 1766 1769 CDNI Logging File for uCDN at 1770 2013-03-23 14:15:00 1771 1772 1773 CDNI Logging File for uCDN at 1774 2013-03-23 14:30:00 1775 urn:uuid:87654321-4321-dcba-aa00-dcba7654321 1776 2013-03-23T14:30:00Z 1777 1780 CDNI Logging File for uCDN at 1781 2013-03-23 14:30:00 1782 1783 ... 1784 1785 ... 1786 1787 1789 Figure 7: Example subscription document of a CDNI Logging Feed 1791 4.2. CDNI Logging File Pull 1793 A client-side implementation of the CDNI Logging interface MAY pull, 1794 at its convenience, a CDNI Logging File that is published by the 1795 server-side in the CDNI Logging Feed (in the subscription document or 1796 an archive document). To do so, the client-side: 1798 o MUST implement HTTP/1.1 ([RFC7230],[RFC7231], [RFC7232], 1799 [RFC7233], [RFC7234], [RFC7235]), MAY also support other HTTP 1800 versions (e.g., HTTP/2.0 [I-D.ietf-httpbis-http2]) and MAY 1801 negotiate which HTTP version is actually used. This allows 1802 operators and implementers to choose to use later versions of HTTP 1803 to take advantage of new features, while still ensuring 1804 interoperability with systems that only support HTTP/1.1. 1806 o MUST use the URI that was associated to the CDNI Logging File 1807 (within the "src" attribute of the corresponding atom:content 1808 element) in the CDNI Logging Feed; 1810 o MUST support exchange of CDNI Logging Files with no content 1811 encoding applied to the representation; 1813 o MUST support exchange of CDNI Logging Files with "gzip" content 1814 encoding (as defined in [RFC7230]) applied to the representation. 1816 Note that a client-side implementation of the CDNI Logging interface 1817 MAY pull a CDNI Logging File that it has already pulled. 1819 The server-side implementation MUST respond to valid pull request by 1820 a client-side implementation for a CDNI Logging File published by the 1821 server-side in the CDNI Logging Feed (in the subscription document or 1822 an archive document). The server-side implementation: 1824 o MUST implement HTTP/1.1 to handle the client-side request and MAY 1825 also support other HTTP versions (e.g., HTTP/2.0); 1827 o MUST include the CDNI Logging File identified by the request URI 1828 inside the body of the HTTP response; 1830 o MUST support exchange of CDNI Logging Files with no content 1831 encoding applied to the representation; 1833 o MUST support exchange of CDNI Logging Files with "gzip" content 1834 encoding (as defined in [RFC7231]) applied to the representation. 1836 Content negotiation approaches defined in [RFC7231] (e.g., using 1837 Accept-Encoding request-header field or Content-Encoding entity- 1838 header field) MAY be used by the client-side and server-side 1839 implementations to establish the content-coding to be used for a 1840 particular exchange of a CDNI Logging File. 1842 Applying compression content encoding (such as "gzip") is expected to 1843 mitigate the impact of exchanging the large volumes of logging 1844 information expected across CDNs. This is expected to be 1845 particularly useful in the presence of HTTP Adaptive Streaming (HAS) 1846 which, as per the present version of the document, will result in a 1847 separate CDNI Log Record for each HAS segment delivery in the CDNI 1848 Logging File. 1850 The potential retention limits (e.g., sliding time window, maximum 1851 aggregate file storage quotas) within which the dCDN is to retain and 1852 be ready to serve a CDNI Logging File previously advertised in the 1853 CDNI Logging Feed is outside the scope of the present document and is 1854 expected to be agreed upon by uCDN and dCDN via other means (e.g., 1855 human agreement). The server-side implementation MUST retain, and be 1856 ready to serve, any CDNI Logging File within the agreed retention 1857 limits. Outside these agreed limits, the server-side implementation 1858 MAY indicate its inability to serve (e.g., with HTTP status code 404) 1859 a CDNI Logging File or MAY refuse to serve it (e.g., with HTTP status 1860 code 403 or 410). 1862 5. Protocol for Exchange of CDNI Logging File During Collection 1864 We note that, in addition to the CDNI Logging File exchange protocol 1865 specified in Section 4, implementations of the CDNI Logging interface 1866 MAY also support other mechanisms to exchange CDNI Logging Files. In 1867 particular, such mechanisms might allow the exchange of the CDNI 1868 Logging File to start before the file is fully collected. This can 1869 allow CDNI Logging Records to be communicated by the dCDN to the uCDN 1870 as they are gathered by the dCDN without having to wait until all the 1871 CDNI Logging Records of the same logging period are collected in the 1872 corresponding CDNI Logging File. This approach is commonly referred 1873 to as "tailing" of the file. 1875 Such an approach could be used, for example, to exchange logging 1876 information with a significantly reduced time-lag (e.g., sub-minute 1877 or sub-second) between when the event occurred in the dCDN and when 1878 the corresponding CDNI Logging Record is made available to the uCDN. 1879 This can satisfy log-consuming applications requiring extremely fresh 1880 logging information such as near-real-time content delivery 1881 monitoring. Such mechanisms are for further study and outside the 1882 scope of this document. 1884 6. IANA Considerations 1886 6.1. CDNI Logging Directive Names Registry 1888 The IANA is requested to create a new registry, CDNI Logging 1889 Directive Names. 1891 The initial contents of the CDNI Logging File Directives registry 1892 comprise the names of the directives specified in Section 3.3 of the 1893 present document, and are as follows: 1895 +------------------------------+-----------+ 1896 | Directive Name | Reference | 1897 +------------------------------+-----------+ 1898 | Version | RFC xxxx | 1899 | UUID | RFC xxxx | 1900 | Claimed-Origin | RFC xxxx | 1901 | Established-Origin | RFC xxxx | 1902 | Record-Type | RFC xxxx | 1903 | Fields | RFC xxxx | 1904 | Integrity-Hash | RFC xxxx | 1905 +------------------------------+-----------+ 1907 Figure 8 1909 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 1910 the present document] 1912 Within the registry, names are to be allocated by IANA according to 1913 the "Specification Required" policy specified in [RFC5226]. 1914 Directive names are to be allocated by IANA with a format of 1915 NAMEFORMAT (see Section 3.1). 1917 Each specification that defines a new CDNI Logging directive needs to 1918 contain a description for the new directive with the same set of 1919 information as provided in Section 3.3 (i.e., format, directive value 1920 and occurrence). 1922 6.2. CDNI Logging Record-Types Registry 1924 The IANA is requested to create a new registry, CDNI Logging Record- 1925 Types. 1927 The initial contents of the CDNI Logging Record-Types registry 1928 comprise the names of the CDNI Logging Record types specified in 1929 Section 3.4 of the present document, and are as follows: 1931 +----------------------+-----------+----------------------------------+ 1932 | Record-Types | Reference | Description | 1933 +----------------------+-----------+----------------------------------+ 1934 | cdni_http_request_v1 | RFC xxxx | CDNI Logging Record version 1 | 1935 | | | for content delivery using HTTP | 1936 +----------------------+-----------+----------------------------------+ 1938 Figure 9 1940 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 1941 the present document] 1943 Within the registry, Record-Types are to be allocated by IANA 1944 according to the "Specification Required" policy specified in 1945 [RFC5226]. Record-Types are to be allocated by IANA with a format of 1946 NAMEFORMAT (see Section 3.1). Record-Types corresponding to 1947 specifications produced by the IETF CDNI Working Group are to be 1948 allocated a name starting with "cdni_". All other Record-Types are 1949 to be allocated a name that does not start with "cdni". 1951 Each specification that defines a new Record-Type needs to contain a 1952 description for the new Record-Type with the same set of information 1953 as provided in Section 3.4.1. This includes: 1955 o a list of all the CDNI Logging Fields that can appear in a CDNI 1956 Logging Record of the new Record-Type 1958 o for all these Fields: a specification of the occurrence for each 1959 Field in the new Record-Type 1961 o for every newly defined Field, i.e., for every Field that results 1962 in a registration in the CDNI Logging Field Names Registry 1963 (Section 6.3): a specification of the field name, format and field 1964 value. 1966 6.3. CDNI Logging Field Names Registry 1968 The IANA is requested to create a new registry, CDNI Logging Field 1969 Names. 1971 This registry is intended to be shared across the currently defined 1972 Record-Type (i.e., cdni_http_request_v1) as well as potential other 1973 CDNI Logging Record-Types that may be defined in separate 1974 specifications. When a Field from this registry is used by another 1975 CDNI Logging Record-Type, it is to be used with the exact semantics 1976 and format specified in the document that registered this field and 1977 that is identified in the Reference column of the registry. If 1978 another CDNI Logging Record-Type requires a Field with semantics that 1979 are not strictly identical, or a format that is not strictly 1980 identical then this new Field is to be registered in the registry 1981 with a different Field name. When a Field from this registry is used 1982 by another CDNI Logging Record-Type, it can be used with different 1983 occurence rules. 1985 The initial contents of the CDNI Logging Fields Names registry 1986 comprise the names of the CDNI Logging fields specified in 1987 Section 3.4 of the present document, and are as follows: 1989 +------------------------------------------+-----------+ 1990 | Field Name | Reference | 1991 +------------------------------------------+-----------+ 1992 | date | RFC xxxx | 1993 | time | RFC xxxx | 1994 | time-taken | RFC xxxx | 1995 | c-ip | RFC xxxx | 1996 | c-ip-anonymizing | RFC xxxx | 1997 | c-port | RFC xxxx | 1998 | s-ip | RFC xxxx | 1999 | s-hostname | RFC xxxx | 2000 | s-port | RFC xxxx | 2001 | cs-method | RFC xxxx | 2002 | cs-uri | RFC xxxx | 2003 | u-uri | RFC xxxx | 2004 | protocol | RFC xxxx | 2005 | sc-status | RFC xxxx | 2006 | sc-total-bytes | RFC xxxx | 2007 | sc-entity-bytes | RFC xxxx | 2008 | cs() | RFC xxxx | 2009 | sc() | RFC xxxx | 2010 | s-ccid | RFC xxxx | 2011 | s-sid | RFC xxxx | 2012 | s-cached | RFC xxxx | 2013 +------------------------------------------+-----------+ 2015 Figure 10 2017 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 2018 the present document] 2020 Within the registry, names are to be allocated by IANA according to 2021 the "Specification Required" policy specified in [RFC5226]. Field 2022 names are to be allocated by IANA with a format of NHTABSTRING (see 2023 Section 3.1). 2025 6.4. CDNI Logging MIME Media Type 2027 The IANA is requested to allocate the "application/cdni.LoggingFile" 2028 MIME Media Type (whose use is specified in Section 4.1.1 of the 2029 present document) in the MIME Media Types registry. 2031 7. Security Considerations 2033 7.1. Authentication, Confidentiality, Integrity Protection 2035 An implementation of the CDNI Logging interface MUST support TLS 2036 transport of the CDNI Logging feed (Section 4.1) and of the CDNI 2037 Logging File pull (Section 4.2) as per [RFC2818] and [RFC7230]. 2039 The use of TLS for transport of the CDNI Logging feed and CDNI 2040 Logging File pull allows: 2042 o the dCDN and uCDN to authenticate each other (to ensure they are 2043 transmitting/receiving CDNI Logging File from an authenticated 2044 CDN) 2046 o the CDNI Logging information to be transmitted with 2047 confidentiality 2049 o the integrity of the CDNI Logging information to be protected 2050 during the exchange 2052 In an environment where any such protection is required, TLS SHOULD 2053 be used (including authentication of the remote end) by the server- 2054 side and the client-side of the CDNI Logging feed and the CDNI 2055 Logging File pull mechanism unless alternate methods are used for 2056 ensuring the confidentiality of the information in the logging files 2057 (such as setting up an IPsec tunnel between the two CDNs or using a 2058 physically secured internal network between two CDNs that are owned 2059 by the same corporate entity). 2061 An implementation of the CDNI Logging interface MUST support the 2062 TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 cipher suite ( [RFC5288]). An 2063 implementation of the CDNI Logging interface SHOULD prefer cipher 2064 suites which support perfect forward secrecy over cipher suites that 2065 don't. 2067 The Integrity-Hash directive inside the CDNI Logging File provides 2068 additional integrity protection, this time targeting potential 2069 corruption of the CDNI logging information during the CDNI Logging 2070 File generation, storage or exchange. This mechanism does not itself 2071 allow restoration of the corrupted CDNI Logging information, but it 2072 allows detection of such corruption and therefore triggering of 2073 appropriate corrective actions (e.g., discard of corrupted 2074 information, attempt to re-obtain the CDNI Logging information). 2075 Note that the Integrity-Hash does not protect against tampering by a 2076 third party, since such a third party could have recomputed and 2077 updated the Integrity-Hash after tampering. Protection against third 2078 party tampering can be achieved as discussed above through the use of 2079 TLS. 2081 7.2. Denial of Service 2083 This document does not define specific mechanism to protect against 2084 Denial of Service (DoS) attacks on the Logging Interface. However, 2085 the CDNI Logging feed and CDNI Logging pull endpoints can be 2086 protected against DoS attacks through the use of TLS transport and/or 2087 via mechanisms outside the scope of the CDNI Logging interface such 2088 as firewalling or use of Virtual Private Networks (VPNs). 2090 Protection of dCDN Surrogates against spoofed delivery requests is 2091 outside the scope of the CDNI Logging interface. 2093 7.3. Privacy 2095 CDNs have the opportunity to collect detailed information about the 2096 downloads performed by End Users. The provision of this information 2097 to another CDN introduces potential End Users privacy protection 2098 concerns. 2100 The use of TLS for transport of the CDNI Logging feed and CDNI 2101 Logging pull as discussed in Section 7.1 protects the confidentiality 2102 of logged information by preventing any other party than the 2103 authorised uCDN to gain access to the logging information. 2105 We observe that when CDNI interconnection is realised as per 2106 [RFC7336], the uCDN handles the initial End User requests (before it 2107 is redirected to the dCDN) so, regardless of which information is, or 2108 is not, communicated to the uCDN through the CDNI Logging interface, 2109 the uCDN has visibility on significant information such as the IP 2110 address of the End User request and the URL of the request. 2112 Nonetheless, if the dCDN and uCDN agree that anonymization is 2113 required to avoid making some detailed information available to the 2114 uCDN (such as how many bytes of the content have been watched by an 2115 End User and/or at what time) or is required to meet some legal 2116 obligations, then the uCDN and dCDN can agree to exchange anonymized 2117 End Users IP address in CDNI Logging Files and the c-ip-anonymization 2118 field can be used to convey the number of bits that have been 2119 anonymized so that the meaningful information can still be easily 2120 extracted from the anonymized addressses (e.g., for geolocation aware 2121 analytics). 2123 We note that anonymization of End Users IP address does not fully 2124 protect against deriving potentially sensitive information about 2125 traffic patterns; in general, increasing the number of bits that are 2126 anonymized can mitigate the risks of deriving such sensitive traffic 2127 pattern information. 2129 We also note that independently of IP addresses, the query string 2130 portion of the URL that may be conveyed inside the cs-uri and u-uri 2131 fields of CDNI Logging Files, or the HTTP cookies( [RFC6265]) that 2132 may be conveyed inside the cs() field of CDNI 2133 Logging Fields, may contain personnal information or information that 2134 can be exploited to derive personal information. Where this is a 2135 concern, the CDNI Logging interface specification allows the dCDN to 2136 not include the cs-uri and to include a u-uri that removes (or hides) 2137 the sensitive part of the query string and allows the dCDN to not 2138 include the cs() fields corresponding to HTTP 2139 headers associated with cookies. 2141 8. Acknowledgments 2143 This document borrows from the W3C Extended Log Format [ELF]. 2145 Rob Murray significantly contributed into the text of Section 4.1. 2147 The authors thank Ben Niven-Jenkins, Kevin Ma, David Mandelberg and 2148 Ray van Brandenburg for their ongoing input. 2150 Finally, we also thank Sebastien Cubaud, Pawel Grochocki, Christian 2151 Jacquenet, Yannick Le Louedec, Anne Marrec , Emile Stephan, Fabio 2152 Costa, Sara Oueslati, Yvan Massot, Renaud Edel, Joel Favier and the 2153 contributors of the EU FP7 OCEAN project for their input in the early 2154 versions of this document. 2156 9. References 2158 9.1. Normative References 2160 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2161 Requirement Levels", BCP 14, RFC 2119, March 1997. 2163 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 2164 Resource Identifier (URI): Generic Syntax", STD 66, RFC 2165 3986, January 2005. 2167 [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally 2168 Unique IDentifier (UUID) URN Namespace", RFC 4122, July 2169 2005. 2171 [RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom 2172 Syndication Format", RFC 4287, December 2005. 2174 [RFC5005] Nottingham, M., "Feed Paging and Archiving", RFC 5005, 2175 September 2007. 2177 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 2178 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 2179 May 2008. 2181 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 2182 Specifications: ABNF", STD 68, RFC 5234, January 2008. 2184 [RFC5288] Salowey, J., Choudhury, A., and D. McGrew, "AES Galois 2185 Counter Mode (GCM) Cipher Suites for TLS", RFC 5288, 2186 August 2008. 2188 [RFC7230] Fielding, R. and J. Reschke, "Hypertext Transfer Protocol 2189 (HTTP/1.1): Message Syntax and Routing", RFC 7230, June 2190 2014. 2192 [RFC7231] Fielding, R. and J. Reschke, "Hypertext Transfer Protocol 2193 (HTTP/1.1): Semantics and Content", RFC 7231, June 2014. 2195 [RFC7232] Fielding, R. and J. Reschke, "Hypertext Transfer Protocol 2196 (HTTP/1.1): Conditional Requests", RFC 7232, June 2014. 2198 [RFC7233] Fielding, R., Lafon, Y., and J. Reschke, "Hypertext 2199 Transfer Protocol (HTTP/1.1): Range Requests", RFC 7233, 2200 June 2014. 2202 [RFC7234] Fielding, R., Nottingham, M., and J. Reschke, "Hypertext 2203 Transfer Protocol (HTTP/1.1): Caching", RFC 7234, June 2204 2014. 2206 [RFC7235] Fielding, R. and J. Reschke, "Hypertext Transfer Protocol 2207 (HTTP/1.1): Authentication", RFC 7235, June 2014. 2209 9.2. Informative References 2211 [CHAR_SET] 2212 "IANA Character Sets registry", 2213 . 2216 [ELF] Phillip M. Hallam-Baker, and Brian Behlendorf, "Extended 2217 Log File Format, W3C (work in progress), WD-logfile- 2218 960323", . 2220 [I-D.ietf-cdni-metadata] 2221 Niven-Jenkins, B., Murray, R., Caulfield, M., and K. Ma, 2222 "CDN Interconnection Metadata", draft-ietf-cdni- 2223 metadata-08 (work in progress), October 2014. 2225 [I-D.ietf-httpbis-http2] 2226 Belshe, M., Peon, R., and M. Thomson, "Hypertext Transfer 2227 Protocol version 2", draft-ietf-httpbis-http2-16 (work in 2228 progress), November 2014. 2230 [I-D.snell-atompub-link-extensions] 2231 Snell, J., "Atom Link Extensions", draft-snell-atompub- 2232 link-extensions-09 (work in progress), June 2012. 2234 [RFC1321] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321, 2235 April 1992. 2237 [RFC1945] Berners-Lee, T., Fielding, R., and H. Nielsen, "Hypertext 2238 Transfer Protocol -- HTTP/1.0", RFC 1945, May 1996. 2240 [RFC2818] Rescorla, E., "HTTP Over TLS", RFC 2818, May 2000. 2242 [RFC6265] Barth, A., "HTTP State Management Mechanism", RFC 6265, 2243 April 2011. 2245 [RFC6707] Niven-Jenkins, B., Le Faucheur, F., and N. Bitar, "Content 2246 Distribution Network Interconnection (CDNI) Problem 2247 Statement", RFC 6707, September 2012. 2249 [RFC6770] Bertrand, G., Stephan, E., Burbridge, T., Eardley, P., Ma, 2250 K., and G. Watson, "Use Cases for Content Delivery Network 2251 Interconnection", RFC 6770, November 2012. 2253 [RFC6983] van Brandenburg, R., van Deventer, O., Le Faucheur, F., 2254 and K. Leung, "Models for HTTP-Adaptive-Streaming-Aware 2255 Content Distribution Network Interconnection (CDNI)", RFC 2256 6983, July 2013. 2258 [RFC7336] Peterson, L., Davie, B., and R. van Brandenburg, 2259 "Framework for Content Distribution Network 2260 Interconnection (CDNI)", RFC 7336, August 2014. 2262 [RFC7337] Leung, K. and Y. Lee, "Content Distribution Network 2263 Interconnection (CDNI) Requirements", RFC 7337, August 2264 2014. 2266 Authors' Addresses 2268 Francois Le Faucheur (editor) 2269 Cisco Systems 2270 E.Space Park - Batiment D 2271 6254 Allee des Ormes - BP 1200 2272 Mougins cedex 06254 2273 FR 2275 Phone: +33 4 97 23 26 19 2276 Email: flefauch@cisco.com 2278 Gilles Bertrand (editor) 2279 Orange 2280 38-40 rue du General Leclerc 2281 Issy les Moulineaux 92130 2282 FR 2284 Phone: +33 1 45 29 89 46 2285 Email: gilles.bertrand@orange.com 2287 Iuniana Oprescu (editor) 2288 Orange 2289 38-40 rue du General Leclerc 2290 Issy les Moulineaux 92130 2291 FR 2293 Phone: +33 6 89 06 92 72 2294 Email: iuniana.oprescu@orange.com 2296 Roy Peterkofsky 2297 Skytide, Inc. 2298 One Kaiser Plaza, Suite 785 2299 Oakland CA 94612 2300 USA 2302 Phone: +01 510 250 4284 2303 Email: roy@skytide.com