idnits 2.17.1 draft-ietf-cdni-logging-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 12, 2013) is 3934 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'MED' is mentioned on line 1556, but not defined == Missing Reference: 'GEN-5' is mentioned on line 1874, but not defined == Missing Reference: 'GEN-6' is mentioned on line 1874, but not defined == Missing Reference: 'GEN-7' is mentioned on line 1874, but not defined == Missing Reference: 'GEN-8' is mentioned on line 1874, but not defined == Missing Reference: 'GEN-9' is mentioned on line 1874, but not defined == Missing Reference: 'GEN-12' is mentioned on line 1874, but not defined == Missing Reference: 'GEN-1' is mentioned on line 1876, but not defined == Missing Reference: 'GEN-2' is mentioned on line 1877, but not defined == Missing Reference: 'GEN-3' is mentioned on line 1878, but not defined == Missing Reference: 'GEN-4' is mentioned on line 1879, but not defined == Missing Reference: 'GEN-10' is mentioned on line 1880, but not defined == Missing Reference: 'GEN-11' is mentioned on line 1882, but not defined == Missing Reference: 'LI-1' is mentioned on line 1894, but not defined == Missing Reference: 'LI-2' is mentioned on line 1898, but not defined == Missing Reference: 'LI-3' is mentioned on line 1904, but not defined == Missing Reference: 'LI-4' is mentioned on line 1907, but not defined == Missing Reference: 'LI-5' is mentioned on line 1914, but not defined == Missing Reference: 'LI-6' is mentioned on line 1917, but not defined == Missing Reference: 'LI-7' is mentioned on line 1920, but not defined == Missing Reference: 'LI-8' is mentioned on line 1925, but not defined == Missing Reference: 'LI-9' is mentioned on line 1931, but not defined == Missing Reference: 'LI-10' is mentioned on line 1936, but not defined == Missing Reference: 'LI-11' is mentioned on line 1936, but not defined == Missing Reference: 'LI-12' is mentioned on line 1936, but not defined == Missing Reference: 'LI-13' is mentioned on line 1941, but not defined == Missing Reference: 'LI-14' is mentioned on line 1944, but not defined == Missing Reference: 'LI-15' is mentioned on line 1944, but not defined == Missing Reference: 'LI-16' is mentioned on line 1947, but not defined == Missing Reference: 'LI-17' is mentioned on line 1951, but not defined == Missing Reference: 'SEC-3' is mentioned on line 1955, but not defined == Missing Reference: 'SEC-5' is mentioned on line 1955, but not defined ** Obsolete normative reference: RFC 2616 (Obsoleted by RFC 7230, RFC 7231, RFC 7232, RFC 7233, RFC 7234, RFC 7235) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) == Outdated reference: A later version (-14) exists of draft-ietf-cdni-framework-03 == Outdated reference: A later version (-21) exists of draft-ietf-cdni-metadata-01 == Outdated reference: A later version (-17) exists of draft-ietf-cdni-requirements-09 -- Obsolete informational reference (is this intentional?): RFC 2818 (Obsoleted by RFC 9110) Summary: 2 errors (**), 0 flaws (~~), 36 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force F. Le Faucheur, Ed. 3 Internet-Draft Cisco Systems 4 Intended status: Standards Track G. Bertrand, Ed. 5 Expires: January 13, 2014 I. Oprescu, Ed. 6 Orange 7 R. Peterkofsky 8 Skytide, Inc. 9 July 12, 2013 11 CDNI Logging Interface 12 draft-ietf-cdni-logging-05 14 Abstract 16 This memo specifies the Logging interface between a downstream CDN 17 (dCDN) and an upstream CDN (uCDN) that are interconnected as per the 18 CDN Interconnection (CDNI) framework. First, it describes a 19 reference model for CDNI logging. Then, it specifies the CDNI 20 Logging File format and the actual protocol for exchange of CDNI 21 Logging Files. 23 Status of This Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at http://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on January 13, 2014. 40 Copyright Notice 42 Copyright (c) 2013 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (http://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 58 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 59 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 5 60 2. CDNI Logging Reference Model . . . . . . . . . . . . . . . . 5 61 2.1. CDNI Logging interactions . . . . . . . . . . . . . . . . 5 62 2.2. Overall Logging Chain . . . . . . . . . . . . . . . . . . 8 63 2.2.1. Logging Generation and During-Generation Aggregation 9 64 2.2.2. Logging Collection . . . . . . . . . . . . . . . . . 10 65 2.2.3. Logging Filtering . . . . . . . . . . . . . . . . . . 10 66 2.2.4. Logging Rectification and Post-Generation Aggregation 11 67 2.2.5. Log-Consuming Applications . . . . . . . . . . . . . 11 68 2.2.5.1. Maintenance/Debugging . . . . . . . . . . . . . . 11 69 2.2.5.2. Accounting . . . . . . . . . . . . . . . . . . . 12 70 2.2.5.3. Analytics and Reporting . . . . . . . . . . . . . 12 71 2.2.5.4. Security . . . . . . . . . . . . . . . . . . . . 12 72 2.2.5.5. Legal Logging Duties . . . . . . . . . . . . . . 13 73 2.2.5.6. Notions common to multiple Log Consuming 74 Applications . . . . . . . . . . . . . . . . . . 13 75 3. CDNI Logging File . . . . . . . . . . . . . . . . . . . . . . 15 76 3.1. Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 15 77 3.2. CDNI Logging File Structure . . . . . . . . . . . . . . . 16 78 3.3. CDNI Logging File Directives . . . . . . . . . . . . . . 18 79 3.4. CDNI Logging Records . . . . . . . . . . . . . . . . . . 21 80 3.4.1. HTTP Request Logging Record . . . . . . . . . . . . . 22 81 3.5. CDNI Logging File Example . . . . . . . . . . . . . . . . 28 82 4. CDNI Logging File Exchange Protocol . . . . . . . . . . . . . 29 83 4.1. CDNI Logging Feed . . . . . . . . . . . . . . . . . . . . 30 84 4.1.1. Atom Formatting . . . . . . . . . . . . . . . . . . . 30 85 4.1.2. Updates to Log Files and the Feed . . . . . . . . . . 31 86 4.1.3. Redundant Feeds . . . . . . . . . . . . . . . . . . . 31 87 4.1.4. Example CDNI Logging Feed . . . . . . . . . . . . . . 31 88 4.2. CDNI Logging File Pull . . . . . . . . . . . . . . . . . 32 89 5. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 33 90 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34 91 6.1. CDNI Logging Directive Names Registry . . . . . . . . . . 34 92 6.2. CDNI Logging Record-Types Registry . . . . . . . . . . . 35 93 6.3. CDNI Logging Field Names Registry . . . . . . . . . . . . 35 94 6.4. CDNI Logging MIME Media Type . . . . . . . . . . . . . . 36 95 7. Security Considerations . . . . . . . . . . . . . . . . . . . 36 96 7.1. Authentication, Confidentiality, Integrity Protection . . 37 97 7.2. Non Repudiation . . . . . . . . . . . . . . . . . . . . . 37 98 7.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 37 99 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 38 100 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 38 101 9.1. Normative References . . . . . . . . . . . . . . . . . . 38 102 9.2. Informative References . . . . . . . . . . . . . . . . . 39 103 Appendix A. Requirements . . . . . . . . . . . . . . . . . . . . 40 104 A.1. Compliance with cdni-requirements . . . . . . . . . . . . 40 105 A.1.1. General requirements . . . . . . . . . . . . . . . . 40 106 A.1.2. Logging Interface requirements . . . . . . . . . . . 40 107 A.1.3. Security requirements . . . . . . . . . . . . . . . . 42 108 A.2. Considerations on CDNI Logging Applicability . . . . . . 42 109 A.2.1. Timeliness . . . . . . . . . . . . . . . . . . . . . 42 110 A.2.2. Reliability . . . . . . . . . . . . . . . . . . . . . 42 111 A.2.3. Security . . . . . . . . . . . . . . . . . . . . . . 43 112 A.2.4. Scalability . . . . . . . . . . . . . . . . . . . . . 43 113 A.2.5. Consistency between CDNI Logging and CDN Logging . . 43 114 A.2.6. Dispatching/Filtering . . . . . . . . . . . . . . . . 43 115 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 44 117 1. Introduction 119 This memo specifies the CDNI Logging interface between a downstream 120 CDN (dCDN) and an upstream CDN (uCDN). First, it describes a 121 reference model for CDNI logging. Then, it specifies the CDNI 122 Logging File format and the actual protocol for exchange of CDNI 123 Logging Files. 125 The reader should be familiar with the following documents: 127 o CDNI problem statement [RFC6707] and framework 128 [I-D.ietf-cdni-framework] identify a Logging interface, 130 o Section 8 of [I-D.ietf-cdni-requirements] specifies a set of 131 requirements for Logging, 133 o [RFC6770] outlines real world use-cases for interconnecting CDNs. 134 These use cases require the exchange of Logging information 135 between the dCDN and the uCDN. 137 As stated in [RFC6707], "the CDNI Logging interface enables details 138 of logs or events to be exchanged between interconnected CDNs". 140 The present document describes: 142 o The CDNI Logging reference model (Section 2), 144 o The CDNI Logging File format (Section 3), 145 o The CDNI Logging File Exchange protocol (Section 4). 147 1.1. Terminology 149 In this document, the first letter of each CDNI-specific term is 150 capitalized. We adopt the terminology described in [RFC6707] and 151 [I-D.ietf-cdni-framework], and extend it with the additional terms 152 defined below. 154 For clarity, we use the word "Log" only for referring to internal CDN 155 logs and we use the word "Logging" for any inter-CDN information 156 exchange and processing operations related to CDNI Logging interface. 157 Log and Logging formats may be different. 159 CDN Logging information: logging information generated and collected 160 within a CDN 162 CDNI Logging information: logging information exchanged across CDNs 163 using the CDNI Logging Interface 165 Logging information: logging information generated and collected 166 within a CDN or obtained from another CDN using the CDNI Logging 167 Interface 169 CDNI Logging Field: an atomic element of information that can be 170 included in a CDNI Logging Record. The time an event/task started, 171 the IP address of an End user to whom content was delivered, and the 172 URI of the content delivered are examples of CDNI Logging Fields. 174 CDNI Logging Record: an information record providing information 175 about a specific event. This comprises a collection of CDNI Logging 176 Fields. 178 CDNI Logging File: a file containing CDNI Logging Records, as well as 179 additional information facilitating the processing of the CDNI 180 Logging Records. 182 CDN Reporting: the process of providing the relevant information that 183 will be used to create a formatted content delivery report provided 184 to the CSP in deferred time. Such information typically includes 185 aggregated data that can cover a large period of time (e.g., from 186 hours to several months). Uses of Reporting include the collection 187 of charging data related to CDN services and the computation of Key 188 Performance Indicators (KPIs). 190 CDN Monitoring: the process of providing content delivery information 191 in real-time. Monitoring typically includes data in real time to 192 provide visibility of the deliveries in progress, for service 193 operation purposes. It presents a view of the global health of the 194 services as well as information on usage and performance, for network 195 services supervision and operation management. In particular, 196 monitoring data can be used to generate alarms. 198 1.2. Requirements Language 200 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 201 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 202 document are to be interpreted as described in RFC 2119 [RFC2119]. 204 2. CDNI Logging Reference Model 206 2.1. CDNI Logging interactions 208 The CDNI logging reference model between a given uCDN and a given 209 dCDN involves the following interactions: 211 o customization by the uCDN of the CDNI logging information to be 212 provided by the dCDN to the uCDN (e.g. control of which logging 213 fields are to be communicated to the uCDN for a given task 214 performed by the dCDN, control of which types of events are to be 215 logged). The dCDN takes into account this CDNI logging 216 customization information to determine what logging information to 217 provide to the uCDN, but it may, or may not, take into account 218 this CDNI logging customization information to influence what CDN 219 logging information is to be generated and collected within the 220 dCDN (e.g. even if the uCDN requests a restricted subset of the 221 logging information, the dCDN may elect to generate a broader set 222 of logging information). The mechanism to support the 223 customisation by the uCDN of CDNI Logging information is outside 224 the scope of this document and left for further study. We note 225 that the CDNI Control interface or the CDNI Metadata interface 226 appear as candidate interfaces on which to potentially build such 227 a customisation mechanism in the future. Before such a mechanism 228 is available, the uCDN and dCDN are expected to agree off-line on 229 what CDNI logging information is to be provide by dCDN to UCDN and 230 rely on management plane actions to configure the CDNI Logging 231 functions to generate (respectively, expect) in dCDN 232 (respectively, in uCDN). 234 o generation and collection by the dCDN of logging information 235 related to the completion of any task performed by the dCDN on 236 behalf of the uCDN (e.g., delivery of the content to an end user) 237 or related to events happening in the dCDN that are relevant to 238 the uCDN (e.g., failures or unavailability in dCDN). This takes 239 place within the dCDN and does not directly involve CDNI 240 interfaces. 242 o communication by the dCDN to the uCDN of the logging information 243 collected by the dCDN relevant to the uCDN. This is supported by 244 the CDNI Logging interface and in the scope of the present 245 document. For example, the uCDN may use this logging information 246 to charge the CSP, to perform analytics and monitoring for 247 operational reasons, to provide analytics and monitoring views on 248 its content delivery to the CSP or to perform trouble-shooting. 250 o customization by the dCDN of the logging to be performed by the 251 uCDN on behalf of the dCDN. The mechanism to support the 252 customisation by the dCDN of CDNI Logging information is outside 253 the scope of this document and left for further study. 255 o generation and collection by the uCDN of logging information 256 related to the completion of any task performed by the uCDN on 257 behalf of the dCDN (e.g., serving of content by uCDN to dCDN for 258 acquisition purposes by dCDN) or related to events happening in 259 the uCDN that are relevant to the dCDN. This takes place within 260 the uCDN and does not directly involve CDNI interfaces. 262 o communication by the uCDN to the dCDN of the logging information 263 collected by the uCDN relevant to the dCDN. For example, the dCDN 264 might potentially benefit form this information for security 265 auditing or content acquisition troubleshooting. This is outside 266 the scope of this document and left for further study. 268 Figure 1 provides an example of CDNI Logging interactions (focusing 269 only on the interactions that are in the scope of this document) in a 270 particular scenario where 4 CDNs are involved in the delivery of 271 content from a given CSP: the uCDN has a CDNI interconnection with 272 dCDN-1 and dCDN-2. In turn, dCDN2 has a CDNI interconnection with 273 dCDN3. In this example, uCDN, dCDN-1, dCDN-2 and dCDN-3 all 274 participate in the delivery of content for the CSP. In this example, 275 the CDNI Logging interface enables the uCDN to obtain logging 276 information from all the dCDNs involved in the delivery. In the 277 example, uCDN uses the Logging data: 279 o to analyze the performance of the delivery operated by the dCDNs 280 and to adjust its operations (e.g., request routing) as 281 appropriate, 283 o to provide reporting (non real-time) and monitoring (real-time) 284 information to CSP. 286 For instance, uCDN merges Logging data, extracts relevant KPIs, and 287 presents a formatted report to the CSP, in addition to a bill for the 288 content delivered by uCDN itself or by its dCDNs on his behalf. uCDN 289 may also provide Logging data as raw log files to the CSP, so that 290 the CSP can use its own logging analysis tools. 292 +-----+ 293 | CSP | 294 +-----+ 295 ^ Reporting and monitoring data 296 * Billing 297 ,--*--. 298 Logging ,-' `-. 299 Data =>( uCDN )<= Logging 300 // `-. _,-' \\ Data 301 || `-'-'-' || 302 ,-----. ,-----. 303 ,-' `-. ,-' `-. 304 ( dCDN-1 ) ( dCDN-2 )<== Logging 305 `-. ,-' `-. _,-' \\ Data 306 `--'--' `--'-' || 307 ,-----. 308 ,' `-. 309 ( dCDN-3 ) 310 `. ,-' 311 `--'--' 313 ===> CDNI Logging Interface 314 ***> outside the scope of CDNI 316 Figure 1: Interactions in CDNI Logging Reference Model 318 A dCDN (e.g., dCDN-2) integrates the relevant logging information 319 obtained from its dCDNs (e.g., dCDN-3) in the logging information 320 that it provides to the uCDN, so that the uCDN ultimately obtains all 321 logging information relevant to a CSP for which it acts as the 322 authoritative CDN. 324 Note that the format of Logging information that a CDN provides over 325 the CDNI interface might be different from the one that the CDN uses 326 internally. In this case, the CDN needs to reformat the Logging 327 information before it provides this information to the other CDN over 328 the CDNI Logging interface. Similarly, a CDN might reformat the 329 Logging data that it receives over the CDNI Logging interface before 330 injecting it into its log-consuming applications or before providing 331 some of this logging information to the CSP. Such reformatting 332 operations introduce latency in the logging distribution chain and 333 introduce a processing burden. Therefore, there are benefits in 334 specifying CDNI Logging format that are suitable for use inside CDNs 335 and also are close to the CDN Log formats commonly used in CDNs 336 today. 338 2.2. Overall Logging Chain 340 This section discusses the overall logging chain within and across 341 CDNs to clarify how CDN Logging information is expected to fit in 342 this overall chain. Figure 2 illustrates the overall logging chain 343 within the dCDN, across CDNs using the CDNI Logging interface and 344 within the uCDN. Note that the logging chain illustrated in the 345 Figure is obviously only indicative and varies depending on the 346 specific environments. For example, there may be more or less 347 instantiations of each entity (i.e., there may be 4 Log consuming 348 applications in a given CDN). As another example, there may be one 349 instance of Rectification process per Log Consuming Application 350 instead of a shared one. 352 Log Consuming Log Consuming 353 App App 354 /\ /\ 355 | | 356 Rectification-------- 357 /\ 358 | 359 Filtering 360 /\ 361 | 362 Collection uCDN 363 /\ /\ 364 | | 365 | Generation 366 | 367 CDNI Logging --------------------------------------------- 368 exchange 369 /\ Log Consuming Log Consuming 370 | App App 371 | /\ /\ 372 | | | 373 Rectification Rectification--------- 374 /\ /\ 375 | | 376 Filtering 377 /\ 378 | 379 Collection dCDN 380 /\ /\ 381 | | 382 Generation Generation 384 Figure 2: CDNI Logging in the overall Logging Chain 386 The following subsections describe each of the processes potentially 387 involved in the logging chain of Figure 2. 389 2.2.1. Logging Generation and During-Generation Aggregation 391 CDNs typically generate logging information for all significant task 392 completions, events, and failures. Logs are typically generated by 393 many devices in the CDN including the surrogates, the request routing 394 system, and the control system. 396 The amount of Logging information generated can be huge. Therefore, 397 during contract negotiations, interconnected CDNs often agree on a 398 Logging retention duration, and optionally, on a maximum size of the 399 Logging data that the dCDN must keep. If this size is exceeded, the 400 dCDN must alert the uCDN but may not keep more Logs for the 401 considered time period. In addition, CDNs may aggregate logs and 402 transmit only summaries for some categories of operations instead of 403 the full Logging data. Note that such aggregation leads to an 404 information loss, which may be problematic for some usages of Logging 405 (e.g., debugging). 407 [I-D.brandenburg-cdni-has] discusses logging for HTTP Adaptive 408 Streaming (HAS). In accordance with the recommendations articulated 409 there, it is expected that a surrogate will generate separate logging 410 information for delivery of each chunk of HAS content. This ensures 411 that separate logging information can then be provided to 412 interconnected CDNs over the CDNI Logging interface. Still in line 413 with the recommendations of [I-D.brandenburg-cdni-has], the logging 414 information for per-chunck delivery may include some information (a 415 Content Collection IDentifier and a Session IDentifier) intended to 416 facilitate subsequent post-generation aggregation of per-chunk logs 417 into per-session logs. Note that a CDN may also elect to generate 418 aggregate per-session logs when performing HAS delivery, but this 419 needs to be in addition to, and not instead of, the per-chunk 420 delivery logs. We note that this may be revisited in future versions 421 of this document. 423 Note that in the case of non real-time logging, the trigger of the 424 transmission or generation of the logging file appears to be a 425 synchronous process from a protocol standpoint. The implementation 426 algorithm can choose to enforce a maximum size for the logging file 427 beyond which the transmission is automatically triggered (and thus 428 allow for an asynchronous transmission process). 430 2.2.2. Logging Collection 432 This is the process that continuously collects logs generated by the 433 log-generating entities within a CDN. 435 In a CDNI environment, in addition to collecting logging information 436 from log-generating entities within the local CDN, the Collection 437 process also collects logging information provided by another CDN, or 438 other CDNs, through the CDNI Logging interface. This is illustrated 439 in Figure 2 where we see that the Collection process of the uCDN 440 collects logging information from log-generating entities within the 441 uCDN as well as logging information coming through CDNI Logging 442 exchange with the dCDN through the CDNI Logging interface. 444 2.2.3. Logging Filtering 446 A CDN may require to only present different subset of the whole 447 logging information collected to various log-consuming applications. 448 This is achieved by the Filtering process. 450 In particular, the Filtering process can also filter the right subset 451 of information that needs to be provided to a given interconnected 452 CDN. For example, the filtering process in the dCDN can be used to 453 ensure that only the logging information related to tasks performed 454 on behalf of a given uCDN are made available to that uCDN (thereby 455 filtering all the logging information related to deliveries by the 456 dCDN of content for its own CSPs). Similarly, the Filtering process 457 may filter or partially mask some fields, for example, to protect End 458 Users' privacy when communicating CDNI Logging information to another 459 CDN. Filtering of logging information prior to communication of this 460 information to other CDNs via the CDNI Logging interface requires 461 that the downstream CDN can recognize the set of log records that 462 relate to each interconnected CDN. 464 The CDN will also filter some internal scope information such as 465 information related to its internal alarms (security, failures, load, 466 etc). 468 In some use cases described in [RFC6770], the interconnected CDNs do 469 not want to disclose details on their internal topology. The 470 filtering process can then also filter confidential data on the 471 dCDNs' topology (number of servers, location, etc.). In particular, 472 information about the requests served by every Surrogate may be 473 confidential. Therefore, the Logging information must be protected 474 so that data such as Surrogates' hostnames is not disclosed to the 475 uCDN. In the "Inter-Affiliates Interconnection" use case, this 476 information may be disclosed to the uCDN because both the dCDN and 477 the uCDN are operated by entities of the same group. 479 2.2.4. Logging Rectification and Post-Generation Aggregation 481 If Logging is generated periodically, it is important that the 482 sessions that start in one Logging period and end in another are 483 correctly reported. If they are reported in the starting period, 484 then the Logging of this period will be available only after the end 485 of the session, which delays the Logging generation. 487 A Logging rectification/update mechanism could be useful to reach a 488 good trade-off between the Logging generation delay and the Logging 489 accuracy. Depending on the selected Logging protocol(s), such 490 mechanism may be invaluable for real time Logging, which must be 491 provided rapidly and cannot wait for the end of operations in 492 progress. 494 In the presence of HAS, some log-consuming applications can benefit 495 from aggregate per-session logs. For example, for analytics, per- 496 session logs allow display of session-related trends which are much 497 more meaningful for some types of analysis than chunk-related trends. 498 In the case where the log-generating entities have generated during- 499 generation aggregate logs, those can be used by the applications. In 500 the case where aggregate logs have not been generated, the 501 Rectification process can be extended with a Post-Generation 502 Aggregation process that generates per-session logs from the per- 503 chunk logs, possibly leveraging the information included in the per- 504 chunk logs for that purpose (Content Collection IDentifier and a 505 Session IDentifier). However, in accordance with 506 [I-D.brandenburg-cdni-has], this document does not define exchange of 507 such aggregate logs on the CDNI Logging interface. We note that this 508 may be revisited in future versions of this document. 510 2.2.5. Log-Consuming Applications 512 2.2.5.1. Maintenance/Debugging 514 Logging is useful to permit the detection (and limit the risk) of 515 content delivery failures. In particular, Logging facilitates the 516 resolution of configuration issues. 518 To detect faults, Logging must enable the reporting of any CDN 519 operation success and failure, such as request redirection, content 520 acquisition, etc. The uCDN can summarize such information into KPIs. 521 For instance, Logging format should allow the computation of the 522 number of times during a given epoch that content delivery related to 523 a specific service succeeds/fails. 525 Logging enables the CDN providers to identify and troubleshoot 526 performance degradations. In particular, Logging enables the 527 communication of traffic data (e.g., the amount of traffic that has 528 been forwarded by a dCDN on behalf of an uCDN over a given period of 529 time), which is particularly useful for CDN and network planning 530 operations. 532 2.2.5.2. Accounting 534 Logging is essential for accounting, to permit inter-CDN billing and 535 CSP billing by uCDNs. For instance, Logging information provided by 536 dCDNs enables the uCDN to compute the total amount of traffic 537 delivered by every dCDN for a particular Content Provider, as well 538 as, the associated bandwidth usage (e.g., peak, 95th percentile), and 539 the maximum number of simultaneous sessions over a given period of 540 time. 542 2.2.5.3. Analytics and Reporting 544 The goal of analytics is to gather any relevant information to track 545 audience, analyze user behavior, and monitor the performance and 546 quality of content delivery. For instance, Logging enables the CDN 547 providers to report on content consumption (e.g., delivered sessions 548 per content) in a specific geographic area. 550 The goal of reporting is to gather any relevant information to 551 monitor the performance and quality of content delivery and allow 552 detection of delivery issues. For instance, reporting could track 553 the average delivery throughput experienced by End-Users in a given 554 region for a specific CSP or content set over a period of time. 556 2.2.5.4. Security 558 The goal of security is to prevent and monitor unauthorized access, 559 misuse, modification, and denial of access of a service. A set of 560 information is logged for security purposes. In particular, a record 561 of access to content is usually collected to permit the CSP to detect 562 infringements of content delivery policies and other abnormal End 563 User behaviors. 565 2.2.5.5. Legal Logging Duties 567 Depending on the country considered, the CDNs may have to retain 568 specific Logging information during a legal retention period, to 569 comply with judicial requisitions. 571 2.2.5.6. Notions common to multiple Log Consuming Applications 573 2.2.5.6.1. Logging Information Views 575 Within a given log-consuming application, different views may be 576 provided to different users depending on privacy, business, and 577 scalability constraints. 579 For example, an analytics tool run by the uCDN can provide one view 580 to an uCDN operator that exploits all the logging information 581 available to the uCDN, while the tool may provide a different view to 582 each CSP exploiting only the logging information related to the 583 content of the given CSP. 585 As another example, maintenance and debugging tools may provide 586 different views to different CDN operators, based on their 587 operational role. 589 2.2.5.6.2. Key Performance Indicators (KPIs) 591 This section presents, for explanatory purposes, a non-exhaustive 592 list of Key Performance Indicators (KPIs) that can be extracted/ 593 produced from logs. 595 Multiple log-consuming applications, such as analytics, monitoring, 596 and maintenance applications, often compute and track such KPIs. 598 In a CDNI environment, depending on the situation, these KPIs may be 599 computed by the uCDN or by the dCDN. But it is usually the uCDN that 600 computes KPIs, because uCDN and dCDN may have different definitions 601 of the KPIs and the computation of some KPIs requires a vision of all 602 the deliveries performed by the uCDN and all its dCDNs. 604 Here is a list of important examples of KPIs: 606 o Number of delivery requests received from End-Users in a given 607 region for each piece of content, during a given period of time 608 (e.g., hour/day/week/month) 610 o Percentage of delivery successes/failures among the aforementioned 611 requests 613 o Number of failures listed by failure type (e.g., HTTP error code) 614 for requests received from End Users in a given region and for 615 each piece of content, during a given period of time (e.g., hour/ 616 day/week/month) 618 o Number and cause of premature delivery termination for End Users 619 in a given region and for each piece of content, during a given 620 period of time (e.g., hour/day/week/month) 622 o Maximum and mean number of simultaneous sessions established by 623 End Users in a given region, for a given Content Provider, and 624 during a given period of time (e.g., hour/day/week/month) 626 o Volume of traffic delivered for sessions established by End Users 627 in a given region, for a given Content Provider, and during a 628 given period of time (e.g., hour/day/week/month) 630 o Maximum, mean, and minimum delivery throughput for sessions 631 established by End Users in a given region, for a given Content 632 Provider, and during a given period of time (e.g., hour/day/week/ 633 month) 635 o Cache-hit and byte-hit ratios for requests received from End Users 636 in a given region for each piece of content, during a given period 637 of time (e.g., hour/day/week/month) 639 o Top 10 of the most popularly requested content (during a given day 640 /week/month), 642 o Terminal type (mobile, PC, STB, if this information can be 643 acquired from the browser type header, for example). 645 Additional KPIs can be computed from other sources of information 646 than the Logging, for instance, data collected by a content portal or 647 by specific client-side application programming interfaces. Such 648 KPIs are out of scope for the present memo. 650 The KPIs used depend strongly on the considered log-consuming 651 application -- the CDN operator may be interested in different 652 metrics than the CSP is. In particular, CDN operators are often 653 interested in delivery and acquisition performance KPIs, information 654 related to Surrogates' performance, caching information to evaluate 655 the cache-hit ratio, information about the delivered file size to 656 compute the volume of content delivered during peak hour, etc. 658 Some of the KPIs, for instance those providing an instantaneous 659 vision of the active sessions for a given CSP's content, are useful 660 essentially if they are provided in real-time. By contrast, some 661 other KPIs, such as the one averaged on a long period of time, can be 662 provided in non-real time. 664 3. CDNI Logging File 666 3.1. Rules 668 This specification uses the Augmented Backus-Naur Form (ABNF) 669 notation and core rules of [RFC5234]. In particular, the present 670 document uses the following rules from [RFC5234]: 672 CR = %x0D ; carriage return 674 DIGIT = %x30-39 ; 0-9 676 DQUOTE = %x22 ; " (Double Quote) 678 CRLF = CR LF ; Internet standard newline 680 HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F" 682 HTAB = %x09 ; horizontal tab 684 LF = %x0A ; linefeed 686 OCTET = %x00-FF ; 8 bits of data 688 The present document also uses the following rules from [RFC3986]: 690 host = as specified in section 3.2.2 of [RFC3986]. 692 IPv4address = as specified in section 3.2.2 of [RFC3986]. 694 IPv6address = as specified in section 3.2.2 of [RFC3986]. 696 The present document also defines the folowing additional rules: 698 ADDRESS = IPv4address / IPv6address 700 DATE = 4DIGIT "-" 2DIGIT "-" 2DIGIT 702 Dates are recorded in the format YYYY-MM-DD where YYYY, MM and 703 DD stand for the numeric year, month and day respectively. All 704 dates are specified in Universal Time Coordinated (UTC). 706 DEC = 1*DIGIT ["." *DIGIT] 708 QSTRING = DQUOTE *NDQUOTE DQUOTE ; where 709 NDQUOTE = / 2DQUOTE ; whereby a 710 DQUOTE is conveyed inside a QSTRING unambiguously by repeating 711 it. 713 NHTABSTRING = *NHTAB ; where 715 NHTAB = 717 TIME = 2DIGIT ":" 2DIGIT ":" 2DIGIT ["." *DIGIT] 719 Times are recorded in the form HH:MM:SS or HH:MM:SS.S where HH 720 is the hour in 24 hour format, MM is minutes and SS is seconds. 721 All times are specified in Universal Time Coordinated (UTC). 723 3.2. CDNI Logging File Structure 725 As defined in Section 1.1 a CDNI logging field is as an atomic 726 logging information element and a CDNI Logging Record is a collection 727 of CDNI Logging Fields containing all logging information 728 corresponding to a single logging event. This document defines a 729 third level of structure, the CDNI Logging File, that is a collection 730 of CDNI Logging Records. This structure is illustrated in Figure 3. 731 The use of a file structure for transfer of CDNI Logging information 732 is selected since this is the most common practise today for exchange 733 of logging information within and across CDNs. 735 +----------------------------------------------------------+ 736 |CDNI Logging File | 737 | | 738 | #Directive 1 | 739 | #Directive 2 | 740 | ... | 741 | #Directive P | 742 | | 743 | +------------------------------------------------------+ | 744 | |CDNI Logging Record 1 | | 745 | | +-------------+ +-------------+ +-------------+ | | 746 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 747 | | | Field 1 | | Field 2 | | Field N | | | 748 | | +-------------+ +-------------+ +-------------+ | | 749 | +------------------------------------------------------+ | 750 | | 751 | +------------------------------------------------------+ | 752 | |CDNI Logging Record 2 | | 753 | | +-------------+ +-------------+ +-------------+ | | 754 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 755 | | | Field 1 | | Field 2 | | Field N | | | 756 | | +-------------+ +-------------+ +-------------+ | | 757 | +------------------------------------------------------+ | 758 | | 759 | ... | 760 | | 761 | #Directive P+1 | 762 | | 763 | ... | 764 | | 765 | +------------------------------------------------------+ | 766 | |CDNI Logging Record M | | 767 | | +-------------+ +-------------+ +-------------+ | | 768 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 769 | | | Field 1 | | Field 2 | | Field N | | | 770 | | +-------------+ +-------------+ +-------------+ | | 771 | +------------------------------------------------------+ | 772 | | 773 | | 774 | #Directive P+Q | 775 +----------------------------------------------------------+ 777 Figure 3: Structure of Logging Files 779 The CDNI Logging File format is inspired from the W3C Extended Log 780 File Format [ELF]. However, it is fully specified by the present 781 document. Where the present document differs from the W3C Extended 782 Log File Format, an implementation of CDNI Logging MUST comply with 783 the present document. 785 A CDNI Logging File MUST contain a sequence of lines containing US- 786 ASCII characters [CHAR_SET] terminated by CRLF. 788 Each line of a CDNI Logging File MUST contain either a directive or a 789 CDNI Logging Record. 791 Directives record information about the CDNI Logging process itself. 792 Lines containing directives MUST begin with the "#" character. 793 Directives are specified in Section 3.3. 795 Logging Records provide actual details of the logged event. Logging 796 Records are specified in Section 3.4. 798 The CDNI File structure is defined by the following rules: 800 DIRLINE = "#" directive CRLF 802 DIRGROUP = 1*DIRLINE 803 RECLINE = CRLF 805 RECGROUP = *RECLINE 807 = 1* 809 3.3. CDNI Logging File Directives 811 The CDNI Logging File directives are defined by the following rules: 813 directive = DIRNAME ":" HTAB DIRVAL 815 DIRNAME = any CDNI Logging Directive name registered in the CDNI 816 Logging Directive Names registry (Section 6.1). 818 DIRVAL = 821 An implementation of the CDNI Logging interface MUST support the 822 following directives, listed below by their directive name: 824 o Version: 826 * format: "CDNI" "/" 1*DIGIT "." 1*DIGIT 828 * directive value: indicates the version of the CDNI Logging File 829 format. The value MUST be "CDNI/1.0" for the version specified 830 in the present document. 832 * occurrence: there MUST be one and only one instance of this 833 directive. It MUST be the first line of the CDNI Logging file. 835 o UUID: 837 * format: NHTABSTRING 839 * directive value: this a Universally Unique IDentifier (UUID) 840 from the UUID Uniform Resource Name (URN) namespace specified 841 in [RFC4122]) for the CDNI Logging File . 843 * occurrence: there MUST be one and only one instance of this 844 directive. 846 o Claimed-Origin: 848 * format: host 849 * directive value: this contains the claimed identification of 850 the entity transmitting the CDNI Logging File (e.g. the host in 851 a dCDN supporting the CDNI Logging interface) or the entity 852 responsible for transmitting the CDNI Logging File (e.g. the 853 dCDN). 855 * occurrence: there MUST be zero or one instance of this 856 directive. This directive MAY be included by the dCDN. It 857 MUST NOT be included or modified by the uCDN. 859 o Verified-Origin: 861 * format: host 863 * directive value: this contains the identification, as 864 established by the entity receiving the CDNI Logging file, of 865 the entity transmitting the CDNI Logging File (e.g. the host in 866 a dCDN supporting the CDNI Logging interface) or the entity 867 responsible for transmitting the CDNI Logging File (e.g. the 868 dCDN). 870 * occurrence: there MUST be zero or one instance of this 871 directive. This directive MAY be added by the uCDN. It MUST 872 NOT be included by the dCDN. The mechanisms used by the uCDN 873 to establih and validate the entity respondible for the CDNI 874 Logging File is outside the scope of the present document. We 875 observe that, in particular, this may be achieved through 876 authentication mechanisms that are part of the CDNI Logging 877 File pull mechanism (Section 4.2). 879 o Record-Type: 881 * format: NHTABSTRING 883 * directive value: indicates the type of the CDNI Logging Records 884 that follow this directive, until another Record-Type directive 885 (or the end of the CDNI Logging File). This can be any CDNI 886 Logging Record type registered in the CDNI Logging Record-types 887 registry (Section 6.2). "cdni_http_request_v1" MUST be 888 indicated as the Record-Type directive value for CDNI Logging 889 records corresponding to HTTP request (e.g. a HTTP delivery 890 request) as specified in Section 3.4.1. 892 * occurrence: there MUST be at least one instance of this 893 directive. The first instance of this directive MUST precede a 894 Fields directive and precede any CDNI Logging Record. 896 o Fields: 898 * format: FIENAME * ; where FIENAME can take any 899 CDNI Logging field name registered in the CDNI Logging Field 900 Names registry (Section 6.3). 902 * directive value: this lists the names of all the fields for 903 which a value is to appear in the CDNI Logging Records that are 904 after this directive. The names of the fields, as well as 905 their possible occurrences, are specified for each type of CDNI 906 Logging Records in Section 3.4. 908 * occurrence: there MUST be at least one instance of this 909 directive per Record-Type directive. The first instance of 910 this directive for a given Record-Type MUST precede any CDNI 911 Logging Record for this Record-Type. 913 o Integrity-Hash: 915 * format: 32HEXDIG 917 * directive value: This directive permits the detection of a 918 corrupted CDNI Logging File. This can be useful, for instance, 919 if a problem occurs on the filesystem of the dCDN Logging 920 system and leads to a truncation of a logging file. The 921 Integrity-Hash value is computed, and included in this 922 directive by the entity that transmits the CDNI Logging File. 923 It is computed by applying the MD5 ([RFC1321]) cryptographic 924 hash function on the CDNI Logging File, including all the 925 directives and logging records, up to the Intergrity-Hash 926 directive itself, excluding the Integrity-Hash directive itself 927 and, when present, also excluding the Non-Repudiation-Hash 928 directive. The Integrity-Hash value is represented as a US- 929 ASCII encoded hexadecimal number, 32 digits long (representing 930 a 128 bit hash value). The entity receiving the CDNI Logging 931 File also computes in a similar way the MD5 hash on the 932 received CDNI Logging File and compares this hash to the value 933 of the Integrity-Hash directive. If the two values are equal, 934 then the received CDNI Logging File MUST be considered non- 935 corrupted. If the two values are different, the received CDNI 936 Logging File MUST be considered corrupted. The behavior of the 937 entity that received a corrupted CDNI Logging File is outside 938 the scope of this specification; we note that the entity MAY 939 attempt to pull again the same CDNI Logging file from the 940 transmitting entity. If the entity receiving the CDNI Logging 941 File adds a Verified-Origin directive, it MUST recompute and 942 update the Integrity-Hash directive so it also protects the 943 added Verified-Origin directive. 945 * occurrence: there MUST be zero or one instance of this 946 directive. There SHOULD be one instance of this directive. 947 One situation where that directive could be omitted is where 948 integrity protection is already provided via another mechanism 949 (for example if an integrity hash is associated to the CDNI 950 Logging file out of band through the CDNI Logging Logging Feed 951 Section 4.1 leveraging ATOM extensions such as those proposed 952 in [I-D.snell-atompub-link-extensions]. When present, this 953 field MUST be the last line of the CDNI Logging File when the 954 Non-Repudiation-Hash is absent, and MUST be the one before last 955 line of the CDNI Logging File when the Non-Repudiation-Hash is 956 present. 958 3.4. CDNI Logging Records 960 A CDNI Logging Record consists of a sequence of CDNI Logging Fields 961 relating to that single CDNI Logging Record. 963 CDNI Logging Fields MUST be separated by the "horizontal tabulation 964 (HTAB)" character. 966 To facilitate readability, a prefix scheme is used for CDNI Logging 967 field names in a similar way to the one used in W3C Extended Log File 968 Format [ELF] . The semantics of the prefix in the present document 969 is: 971 o c: refers to the User Agent that issues the request (corresponds 972 to the "client" of W3C Extended Log Format) 974 o d: refers to the dCDN (relative to a given CDN acting as a uCDN) 976 o s: refers to the dCDN Surrogate that serves the request 977 (corresponds to the "server" of W3C Extended Log Format) 979 o u: refers to the uCDN (relative to a given CDN acting as a dCDN) 981 o cs: refers to communication from the User-Agent towards the dCDN 982 Surrogate 984 o sc: refers to communication from the dCDN Surrogate towards the 985 User-Agent 987 An implementation of the CDNI Logging interface as per the present 988 specification MUST support the CDNI HTTP Delivery Records as 989 specified in Section 3.4.1. [Editor's Note": other types of CDNI 990 Logging records will be listed here if we specify other types for 991 this version eg Request Routing]. 993 A CDNI Logging Record is defined by the following rules: 995 FIEVAL = 997 = FIEVAL * ; where FIEVAL 998 contains the CDNI Logging field values corresponding to the CDNI 999 Logging field names (FIENAME) listed is the last Fields directive 1000 predecing the present CDNI Logging Record. 1002 3.4.1. HTTP Request Logging Record 1004 The HTTP Request Logging Record is a CDNI Logging Record of Record- 1005 Type "cdni_http_request_v1". It contains the following CDNI Logging 1006 Fields, listed by their field name: 1008 o date: 1010 * format: DATE 1012 * field value: the date at which the processing of request 1013 completed on the Surrogate. 1015 * occurrence: there MUST be one and only one instance of this 1016 field. 1018 o time: 1020 * format: TIME 1022 * field value: the time at which the processing of request 1023 completed on the Surrogate. 1025 * occurrence: there MUST be one and only one instance of this 1026 field. 1028 o time-taken: 1030 * format: DEC 1032 * field value: decimal value of the duration, in seconds, between 1033 the start of the processing of the request and the completion 1034 of the request processing (e.g. completion of delivery) by the 1035 Surrogate. 1037 * occurrence: there MUST be one and only one instance of this 1038 field. 1040 o c-ip: 1042 * format: ADDRESS 1044 * field value: the source IPv4 or IPv6 address (i.e. the "client" 1045 address) in the request received by the Surrogate. 1047 * occurrence: there MUST be one and only one instance of this 1048 field. 1050 o c-ip-anonimizing: 1052 * format: 1*DIGIT 1054 * field value: the number of rightmost bits of the address in the 1055 c-ip field that are zeroed-out in order to anonymize the 1056 logging record. The mechanism by which the two ends of the 1057 CDNI Logging interface agree on whether anonimization is to be 1058 supported and the number of bits that need to be zeroed-out for 1059 this purpose are outside the scope of the present document. 1061 * occurrence: there MUST be zero or one instance of this field. 1063 o c-port: 1065 * format: 1*DIGIT 1067 * field value: the source TCP port (i.e. the "client" port) in 1068 the request received by the Surrogate. 1070 * occurrence: there MUST be zero or exactly one instance of this 1071 field. 1073 o s-ip: 1075 * format: ADDRESS 1077 * field value: the IPv4 or IPv6 address of the Surrogate that 1078 served the request (i.e. the "server" address). 1080 * occurrence: there MUST be zero or exactly one instance of this 1081 field. 1083 o s-hostname: 1085 * format: host 1087 * field value: the hostname of the Surrogate that served the 1088 request (i.e. the "server" hostname). 1090 * occurrence: there MUST be zero or exactly one instance of this 1091 field. 1093 o s-port: 1095 * format: 1*DIGIT 1097 * field value: the destination TCP port (i.e. the "server" port) 1098 in the request received by the Surrogate. 1100 * occurrence: there MUST be zero or exactly one instance of this 1101 field. 1103 o cs-method: 1105 * format: NHTABSTRING 1107 * field value: this is the HTTP method of the HTTP request 1108 received by the Surrogate. 1110 * occurrence: There MUST be one and only one instance of this 1111 field. 1113 o cs-uri: 1115 * format: NHTABSTRING 1117 * field value: this is the complete URL of the request received 1118 by the Surrogate. It is exactly in the format of a http_URL 1119 specified in [RFC2616]) or, when the request was a HTTPS 1120 request ([RFC2818]), it is in the format of a http_URL but with 1121 the scheme part set to "https" instead of "http". 1123 * occurrence: there MUST be zero or exactly one instance of this 1124 field. 1126 o u-uri: 1128 * format: NHTABSTRING 1130 * field value: this is a complete URL, derived from the complete 1131 URI of the request received by the Surrogate (i.e. the cs-uri) 1132 but transformed by the entity generating or transmitting the 1133 CDNI Logging Record, in a way that is agreed upon between the 1134 two ends of the CDNI Logging interface, so the transformed URI 1135 is meaningful to the uCDN. For example, the two ends of the 1136 CDNI Logging interface could agree that the u-uri is 1137 constructed from the cs-uri by removing the part of the 1138 hostname that exposes which individual Surrogate actually 1139 performed the delivery. The details of modification performed 1140 to generate the u-uri, as well as the mechanism to agree on 1141 these modifications between the two sides of the CDNI Logging 1142 interface are outside the scope of the present document. 1144 * occurrence: there MUST be one and only one instance of this 1145 field. 1147 o protocol: 1149 * format: NHTABSTRING 1151 * field value: this is value of the HTTP-Version field as 1152 specified in [RFC2616] of the Request-Line of the request 1153 received by the Surrogate (e.g. "HTTP/1.1"). 1155 * occurrence: there MUST be one and only one instance of this 1156 field. 1158 o sc-status: 1160 * format: 3DIGIT 1162 * field value: this is the HTTP Status-Code in the HTTP response 1163 from the Surrogate. 1165 * occurrence: There MUST be one and only one instance of this 1166 field. 1168 o sc-total-bytes: 1170 * format: 1*DIGIT 1172 * field value: this is the total number of bytes of the HTTP 1173 response sent by the Surrogate in response to the request. 1174 This includes the bytes of the Status-Line (including HTTP 1175 headers) and of the message-body. 1177 * occurrence: There MUST be one and only one instance of this 1178 field. 1180 o sc-entity-bytes: 1182 * format: 1*DIGIT 1184 * field value: this is the number of bytes of the message-body in 1185 the HTTP response sent by the Surrogate in response to the 1186 request. This does not include the bytes of the Status-Line 1187 (and therefore does not include the bytes of the HTTP headers). 1189 * occurrence: there MUST be zero or exactly one instance of this 1190 field. 1192 o cs(): 1194 * format: QSTRING 1196 * field value: the value of the HTTP header (identified by the 1197 in the CDNI Logging field name) as it 1198 appears in the request processed by the Surrogate. For 1199 example, when the CDNI Logging field name (FIENAME) listed in 1200 the prededing Fields directive is "cs(User-Agent"), this CDNI 1201 Logging field value contains the value of the User-Agent HTTP 1202 header as received by the Surrogate in the request it 1203 processed. 1205 * occurrence: there MUST be zero, one or any number of instance 1206 of this field. 1208 o sc(): 1210 * format: QSTRING 1212 * field value: the value of the HTTP header (identified by the 1213 in the CDNI Logging field name) as it 1214 appears in the response issued by the Surrogate to serve the 1215 request. 1217 * occurrence: there MUST be zero, one or any number of instance 1218 of this field. 1220 o s-ccid: 1222 * format: QSTRING 1224 * field value: this contains the value of the Content Collection 1225 IDentifier associated by the uCDN to the content served by the 1226 Surrogate via the CDNI Metadata interface 1227 ([I-D.ietf-cdni-metadata]). 1229 * occurrence: there MUST be zero or exactly one instance of this 1230 field. 1232 o s-sid: 1234 * format: QSTRING 1236 * field value: this contains the value of a Session IDentifier 1237 generated by the dCDN for a specific HTTP Adaptive Streaming 1238 (HAS) session and whose value is included in the Logging record 1239 for every content chunk delivery of that session in view of 1240 facilitating the later correlation of all the per content chunk 1241 log records of a given HAS session. See section 3.4.2.2. of 1242 [I-D.brandenburg-cdni-has] for more discussion on the concept 1243 of Session IDentifier. 1245 * occurrence: there MUST be zero or exactly one instance of this 1246 field. 1248 o s-cached: 1250 * format: 1DIGIT 1252 * field value: this characterises whether the Surrogate could 1253 serve the request using content already stored on its local 1254 cache. The allowed values are "0" (for miss) and "1" for hit). 1255 "1" MUST be used when the Surrogate could serve the request 1256 using exclusively content already stored on its local cache. 1257 "0" MUST be used otherwise (including cases where the Surrogate 1258 served the request using some, but not all, content already 1259 stored on its local cache). Note that a "0" only means a cache 1260 miss in the Surrogate and does not provide any information on 1261 whether the content was already stored, or not, in another 1262 device of the dCDN i.e. whether this was a "dCDN hit" or "dCDN 1263 miss". 1265 * occurrence: there MUST be zero or exactly one instance of this 1266 field. 1268 o s-uri-signing: 1270 * format: 1DIGIT 1272 * field value: this characterises the uri signing validation 1273 performed by the Surrogate on the request. The allowed values 1274 are: 1276 * 1278 + "0" : no uri signature validation performed 1280 + "1" : uri signature validation performed and validated 1281 + "2" : uri signature validation performed and rejected 1283 * occurrence: there MUST be zero or exactly one instance of this 1284 field. 1286 The "Fields" directive corresponding to a HTTP Request Logging Record 1287 MUST list all the fields name whose occurrence is specified above as 1288 "There MUST be one and only one instance of this field". The 1289 corresponding fields value MUST be present in every HTTP Request 1290 Logging Record. 1292 The "Fields" directive corresponding to a HTTP Request Logging Record 1293 MAY list all the fields value whose occurrence is specified above as 1294 "there MUST be zero or exactly one instance of this field" or "there 1295 MUST be zero, one or any number of instance of this field". The set 1296 of such fields name actually listed in the "Fields" directive is 1297 selected by the implementation generating the CDNI Logging File based 1298 on agreements between the interconnected CDNs established through 1299 mechanisms outside the scope of this specification (e.g. contractual 1300 agreements) . When such a field name is not listed in the "Fields" 1301 directive, the corresponding field value MUST NOT be included in the 1302 Logging Record. When such a field name is listed in the "Fields" 1303 directive, the corresponding field value MUST be included in the 1304 Logging Record; in that case, if the value for the field is not 1305 available, this MUST be conveyed via a dash character ("-"). 1307 The fields name listed in the "Fields" directive MAY be listed in the 1308 order in which they are listed in Section 3.4.1 or MAY be listed in 1309 any other order. 1311 3.5. CDNI Logging File Example 1313 #Version:CDNI/1.0 1315 #UUID:"urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6" 1317 #Claimed-Origin:cdni-logging-entity.dcdn.example.com 1319 #Record-Type:cdni_http_request_v1 1321 #Fields:datetimetime-takenc-ipcs- 1322 methodu-uriprotocolsc-statussc-total- 1323 bytescs(User-Agent)cs(Referer)s-cached 1324 2013-05-1700:38:06.8259.05810.5.7.1GETh 1325 ttp://cdni-ucdn.dcdn.example.com/video/movie100.mp4HTTP/ 1326 1.12006729891"Mozilla/5.0 (Windows; U; Windows NT 1327 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 1328 Safari /533.4""host1.example.com"1 1330 2013-05-1700:39:09.14515.3210.5.10.5GET 1331 http://cdni-ucdn.dcdn.example.com/video/movie118.mp4HTTP/ 1332 1.120015799210"Mozilla/5.0 (Windows; U; Windows NT 1333 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 1334 Safari /533.4""host1.example.com"1 1336 2013-05-1700:42:53.43752.87910.5.10.5GEThttp://cdni-ucdn.dcdn.example.com/video/picture11.mp4HTTP/ 1338 1.020097234724"Mozilla/5.0 (Windows; U; Windows NT 1339 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 1340 Safari /533.4""host5.example.com"0 1342 #Integrity-Hash: 9e107d9d372bb6826bd81d3542a419d6 [Editor's Note: 1343 include the correct MD5-hash value for the actual example] 1345 4. CDNI Logging File Exchange Protocol 1347 This document specifies a protocol for the exchange of CDNI Logging 1348 Files as specified in Section 3. 1350 This protocol comprises: 1352 o a CDNI Logging feed, allowing the dCDN to notify the uCDN about 1353 the CDNI Logging files that can be retrieved by that uCDN from the 1354 dCDN, as well as all the information necessary for retrieving each 1355 of these CDNI Logging File. The CDNI Logging feed is specified in 1356 Section 4.1. 1358 o a CDNI Logging File pull mechanism, allowing the uCDN to obtain 1359 from the dCDN a given CDNI Logging File at the uCDN convenience. 1360 The CDNI Logging File pull mechanisms is specified in Section 4.2. 1362 An implementation of the CDNI Logging interface as per the present 1363 document generating CDNI Logging file (i.e. on the dCDN side) MUST 1364 support the server side of the CDNI Logging feed and the server side 1365 of the CDNI Logging pull mechanism. 1367 An implementation of the CDNI Logging interface as per the present 1368 document consuming CDNI Logging file (i.e. on the uCDN side) MUST 1369 support the client side of the CDNI Logging feed and the client side 1370 of the CDNI Logging pull mechanism. 1372 We note that implementations of the CDNI Logging interface MAY also 1373 support other mechanisms to exchange CDNI Logging Files, for example 1374 in view of exchanging logging information with minimum time-lag (e.g. 1375 sub-minute or sub-second) between when the event occurred in the dCDN 1376 and when the corresponding Logging Record is made available to the 1377 uCDN (e.g. for log-consuming applications requiring extremely fresh 1378 logging information such as near-real-time content delivery 1379 monitoring). Such mechanism might be defined in future version of 1380 the present document. 1382 4.1. CDNI Logging Feed 1384 The server-side implementation of the CDNI Logging feed produces an 1385 Atom feed [RFC4287]. This feed is used to advertise log files that 1386 are available for the client-side to retrieve using the CDNI Logging 1387 pull mechanism. 1389 4.1.1. Atom Formatting 1391 A CDNI Logging feed MUST be structured as an Archived feed, as 1392 defined in [RFC5005], and MUST be formatted in Atom [RFC4287]. This 1393 means it consists of a subscription document that is regularly 1394 updated as new CDNI logging files become available, and information 1395 about older CDNI Logging files is moved into archive documents. Once 1396 created, archive documents are never modified. 1398 Each CDNI Logging file listed in an Atom feed MUST be described in an 1399 atom:entry container element. 1401 The atom:entry MUST contain an atom:content element whose "src" 1402 attribute is a link to the CDNI Logging file and whose "type" 1403 attribute is the MIME Media Type indicating that the entry if a CDNI 1404 Logging File. We define this MIME Media Type as "application/ 1405 cdni.LoggingFile" (See Section 6.4). 1407 For compatibility with some Atom feed readers the atom:entry MAY also 1408 contain an atom:link entry whose "href" attribute is a link to the 1409 CDNI Logging file and whose "type" attribute is the MIME Media Type 1410 indicating that the entry if a CDNI Logging File. We define this 1411 MIME Media Type as "application/cdni.LoggingFile" (See Section 6.4). 1413 The IRI used in the atom:id of the atom:entry MUST contain the UUID 1414 of the CDNI Logging file. 1416 The atom:updated in the atom:entry MUST indicate the time at which 1417 the CDNI Logging file was last updated. 1419 4.1.2. Updates to Log Files and the Feed 1421 CDNI Logging files MUST NOT be modified by the dCDN once published in 1422 the CDNI Logging feed. 1424 The frequency with which the subscription feed is updated, the period 1425 of time covered by each CDNI Logging file or each archive document, 1426 and timeliness of publishing of CDNI Logging files is outside the 1427 scope of the present document and is expected to be agreed upon by 1428 uCDN and dCDN via other means (e.g. human agreement). 1430 The server-side implementation SHOULD use HTTP cache control headers 1431 on the subscription feed to indicate the frequency at which the 1432 client-side is to poll for updates. 1434 4.1.3. Redundant Feeds 1436 The server-side implementation MAY present more than one CDNI Logging 1437 feed and for redundancy, CDNI Logging files MAY be published in more 1438 than one feed. 1440 A client-side implementation MAY support such redundant CDNI Logging 1441 feeds. If it supports redundant CDNI Logging feed, the client-side 1442 SHOULD use the UUID of the CDNI Logging file, presented in the 1443 atom:id element of the Atom feed, to avoid uncessarily pulling each 1444 CDNI Logging file more than once. 1446 4.1.4. Example CDNI Logging Feed 1448 Figure 4 illustrates an example of the subscription document of a 1449 CDNI Logging feed. 1451 1452 > 1454 CDNI Logging Feed 1455 2013-03-23T16:21:11Z 1456 urn:uuid:663ae677-40fb-e99a-049d-c5642916b8ce 1457 1459 1461 1463 CDNI Log Feed 1464 Generator 1465 dcdn.example 1466 1467 CDNI Logging File for uCDN at 1468 2013-03-23 14:55:00 1469 urn:uuid:12345678-1234-abcd-00aa-01234567abcd 1470 2013-03-23T14:55:00Z 1471 1474 CDNI Logging File for uCDN at 1475 2013-03-23 14:55:00 1476 1477 1478 CDNI Logging File for uCDN at 1479 2013-03-23 15:55:00 1480 urn:uuid:87654321-4321-dcba-aa00-dcba7654321 1481 2013-03-23T15:55:00Z 1482 1485 CDNI Logging File for uCDN at 1486 2013-03-23 15:55:00 1487 1488 1489 ... 1490 1491 ... 1492 ? 1494 Figure 4: Example subscription document of a CDNI Logging Feed 1496 4.2. CDNI Logging File Pull 1498 A client-side implementation of the CDNI Logging interface pulls, at 1499 its convenience, any CDNI Logging File that is published by the 1500 server-side in the CDNI Logging Feed. To do so, the client-side: 1502 o MUST use HTTP v1.1 1504 o SHOULD use TLS (i.e. use what is loosely referred to as "HTTPS") 1505 whenever protection of the CDNI LOgging information is required 1506 (see Section 7.1) 1508 o MUST use the URI associated to the CDNI Logging File (in the "src" 1509 attribute of the corresponding atom:content element) in the CDNI 1510 Logging Feed 1512 o SHOULD indicate the compression schemes it supports 1513 Note that a client-side implementation of the CDNI Logging interface 1514 MAY pull a CDNI Logging File that it has already pulled, as long as 1515 the file is still published by the server-side in the subscription 1516 document of CDNI Logging Feed. 1518 [Editor's note: if a given Logging file is moved away from 1519 subscription document to an archive document, do we agree it may no 1520 longer be accessible to uCDN?] 1522 The server-side implementation MUST respond to any valid pull request 1523 by a client-side implementation for a CDNI Logging File published by 1524 the server-side in the subscription document of the CDNI Logging 1525 Feed. The server-side implementation: 1527 o MUST handle the client-side request as per HTTP v1.1 1529 o MUST include the CDNI Logging File identified by the request URI 1530 inside the body of the HTTP response 1532 o MUST support the gzip and deflate compression schemes 1534 o MAY support other compression schemes 1536 o when the client-side request indicates client-supported 1537 compression schemes, the server-side SHOULD use a compression 1538 scheme that it supports and is supported by the client-side 1540 5. Open Issues 1542 o Compression: When we say the server MUST support gzip & 1543 deflate we probably need to think through whether we mean content- 1544 encoding, transfer-encoding or both. The semantics get a little 1545 confusing so we probably just need to think them through to ensure 1546 we allow a server to store compressed logs as transmit them 1547 compressed. 1549 o Handling of Event logs and notifications: There are two aspects of 1550 that question: 1552 * non-real-time exchange of event logs from dCDN to uCDN for 1553 audit purposes. This could be added to current spec presumably 1554 in the form of additional Record-Types and without requiring a 1555 significant change to the current CDNI Logging file exchange 1556 approach. It is proposed that this be handled as a [MED] 1557 requirement. e.g. try first specify what events and what 1558 information needs to be exchanged; and depending on progress, 1559 decide to include in initial logging spec or not 1561 * real-time exchange of event notification from dCDN to uCDN for 1562 immediate operational action (eg on notification by dCDN that 1563 dCDN request routing is down, uCDN stops redirecting to this 1564 dCDN). This would presumably require definition/extension of 1565 another CDNI interface or significant change/extension to the 1566 current CDNI logging spec. It is proposed that this be kept 1567 out of the scope of the current cdni-logging spec . 1569 Another question is what set of events should be logged/notified. 1570 The first type of events relates to "service-level" events i.e. 1571 high level events that affect the service that the dCDN is 1572 providing to the uCDN (e.g.dCDN request routing is down, dCDN is 1573 overloaded). There is general agreements that it is desirable to 1574 be able to log/notify such service-level events. The second type 1575 of events is "atomic-level" events i.e. low level events that may 1576 be useful to identify or track a component issue or a delivery 1577 issue. logging/notifying about such events may be useful in some 1578 situations (eg uCDN and dCDN have a particular relationship 1579 allowing them to share detailed operational information) and may 1580 not be useful in some situations (because the dCDN does not want 1581 to expose details of its CDN operation). Ideal approach is to 1582 define both types of events and have the first type as MUST and 1583 the second type as MAY. Fall back approach would be to only 1584 define the first type initially. 1586 o Add precise definition of what must be supported by transmitting 1587 implementation and what must be implemented by receiving 1588 application (regardless of what may actually be used in a given 1589 deployment). For example, it may be reasonable to mandate that a 1590 receiving implementation be able to receive all the directives 1591 specified in the doc and all fields. 1593 o Clean-up MUST/SHOULD/MAY to clarify (where needed) implementations 1594 requirements (ie what any implementation must be capable of doing) 1595 vs deployment requirements (ie what must be used in a given 1596 environment). 1598 6. IANA Considerations 1600 6.1. CDNI Logging Directive Names Registry 1602 The IANA is requested to create a new registry, CDNI Logging 1603 Directive Names. 1605 The initial contents of the CDNI Logging File Directives registry 1606 comprise the names of the directives specified in Section 3.3 of the 1607 present document, and are as follows: 1609 +------------------------------+-----------+ 1610 + Directive Name + Reference | 1611 +------------------------------+-----------+ 1612 + Version + RFC xxxx | 1613 + UUID + RFC xxxx | 1614 + Claimed-Origin + RFC xxxx | 1615 + Verified-Origin + RFC xxxx | 1616 + Record-Type + RFC xxxx | 1617 + Fields + RFC xxxx | 1618 + Integrity-Hash + RFC xxxx | 1619 +------------------------------+-----------+ 1621 Figure 5 1623 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 1624 the present document] 1626 Within the registry, names are to be allocated by IANA according to 1627 the "Specification Required" policy specified in [RFC5226]. 1629 6.2. CDNI Logging Record-Types Registry 1631 The IANA is requested to create a new registry, CDNI Logging Record- 1632 Types. 1634 The initial contents of the CDNI Logging Record-Types registry 1635 comprise the names of the CDNI Logging Record types specified in 1636 Section 3.4 of the present document, and are as follows: 1638 +------------------------------+-----------+ 1639 + Record-Types + Reference | 1640 +------------------------------+-----------+ 1641 + cdni_http_request_v1 + RFC xxxx | 1642 +------------------------------+-----------+ 1644 Figure 6 1646 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 1647 the present document] 1649 Within the registry, Record-Types are to be allocated by IANA 1650 according to the "Specification Required" policy specified in 1651 [RFC5226]. 1653 6.3. CDNI Logging Field Names Registry 1655 The IANA is requested to create a new registry, CDNI Logging Field 1656 Names. 1658 The initial contents of the CDNI Logging Fields Names registry 1659 comprise the names of the CDNI Logging fields specified in 1660 Section 3.4 of the present document, and are as follows: 1662 +---------------------------------------------+-----------+ 1663 + Field Name + Reference | 1664 +---------------------------------------------+-----------+ 1665 + date + RFC xxxx | 1666 + time + RFC xxxx | 1667 + time-taken + RFC xxxx | 1668 + c-ip + RFC xxxx | 1669 + c-ip-anonimizing + RFC xxxx | 1670 + c-port + RFC xxxx | 1671 + s-ip + RFC xxxx | 1672 + s-hostname + RFC xxxx | 1673 + s-port + RFC xxxx | 1674 + cs- method + RFC xxxx | 1675 + cs-uri + RFC xxxx | 1676 + u-uri + RFC xxxx | 1677 + protocol + RFC xxxx | 1678 + sc-status + RFC xxxx | 1679 + sc- total-bytes + RFC xxxx | 1680 + sc-entity-bytes + RFC xxxx | 1681 + cs() + RFC xxxx | 1682 + sc() + RFC xxxx | 1683 + s-ccid + RFC xxxx | 1684 + s-sid + RFC xxxx | 1685 + s-cached + RFC xxxx | 1686 + s-uri-signing + RFC xxxx | 1687 +---------------------------------------------+-----------+ 1689 Figure 7 1691 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 1692 the present document] 1694 Within the registry, names are to be allocated by IANA according to 1695 the "Specification Required" policy specified in [RFC5226]. 1697 6.4. CDNI Logging MIME Media Type 1699 The IANA is requested to allocate the "application/cdni.LoggingFile" 1700 MIME Media Type (whose use is specified in Section 4.1.1 of the 1701 present document) in the MIME Media Types registry. 1703 7. Security Considerations 1704 7.1. Authentication, Confidentiality, Integrity Protection 1706 The use of TLS for transport of the CDNI Logging feed mechanism 1707 (Section 4.1) and CDNI Logging File pull mechanism (Section 4.2) 1708 allows: 1710 o the dCDN and uCDN to authenticate each other (to ensure they are 1711 transmitting/receiving CDNI Logging File from an authenticated 1712 CDN) 1714 o the CDNI Logging information to be transmitted with 1715 confidentiality 1717 o the integrity of the CDNI Logging information to be protected 1718 during the exchange 1720 In an environment where any such protection is required, TLS SHOULD 1721 be used for transport of the CDNI Logging feed and the CDNI Logging 1722 File pull. 1724 A CDNI Logging implementation MUST support TLS transport of the CDNI 1725 Logging feed and the CDNI Logging File pull. 1727 The Integrity-Hash directive inside the CDNI Logging File provides 1728 additional integrity protection, this time targeting potential 1729 corruption of the CDNI logging information during the CDNI Logging 1730 File generation. This mechanism does not allow restoration of the 1731 corrupted CDNI Logging information, but it allows detection of such 1732 corruption and therefore triggering of appropraite correcting actions 1733 (e.g. discard of corrupted information, attempt to re-obtain the CDNI 1734 Logging information). 1736 7.2. Non Repudiation 1738 The Non-Repudiation-Signature directive in the CDNI Logging File 1739 allows support of non-repudiation of the CDNI Logging File by the 1740 dCDN. The optional Non-Repudiation-Hash can be used on the CDNI 1741 Logging interface where needed. 1743 7.3. Privacy 1745 CDNs have the opportunity to collect detailed information about the 1746 downloads performed by End-Users. The provision of this information 1747 to another CDN introduces potential End-Users privacy protection 1748 concerns. We observe that when CDNI interconnection is realised as 1749 per [I-D.ietf-cdni-framework], the uCDN handles the initial End-User 1750 requests (before it is redirected to the dCDN) so, regardless of 1751 which information is, or is not, communicated to the uCDN through the 1752 CDNI Logging interface, the uCDN has visibility on significant 1753 information such as the IP address of the End-User request and the 1754 URL of the request. Nonetheless, if the dCDN and uCDN agree that 1755 anonymization is required to avoid making some detailed information 1756 available to the uCDN (such as how much bytes of the content has been 1757 watched by an enduser and/or at what time) or is required to meet 1758 some legal obligations, then the uCDN and dCDN can agree to exchange 1759 anonymized End-User IP addresses in CDNI Logging files and the c-ip- 1760 anonymization field can be used to convey the number of bits that 1761 have been anonymized so that the meaningful information can still be 1762 easily extracted from the anonymized addressses (e.g. for geolocation 1763 aware analytics). 1765 8. Acknowledgments 1767 This document borrows from the W3C Extended Log Format [ELF]. 1769 Rob Murray significantly contributed into the text of Section 4.1 . 1771 The authors would like to thank Sebastien Cubaud, Pawel Grochocki, 1772 Christian Jacquenet, Yannick Le Louedec, Anne Marrec and Emile 1773 Stephan for their contributions on early versions of this document. 1774 The authors would like also to thank Fabio Costa, Sara Oueslati, Yvan 1775 Massot, Renaud Edel, and Joel Favier for their input and comments. 1776 Finally, they thank the contributors of the EU FP7 OCEAN project for 1777 valuable inputs. 1779 9. References 1781 9.1. Normative References 1783 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1784 Requirement Levels", BCP 14, RFC 2119, March 1997. 1786 [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., 1787 Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext 1788 Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. 1790 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 1791 Resource Identifier (URI): Generic Syntax", STD 66, RFC 1792 3986, January 2005. 1794 [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally 1795 Unique IDentifier (UUID) URN Namespace", RFC 4122, July 1796 2005. 1798 [RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom 1799 Syndication Format", RFC 4287, December 2005. 1801 [RFC5005] Nottingham, M., "Feed Paging and Archiving", RFC 5005, 1802 September 2007. 1804 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 1805 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 1806 May 2008. 1808 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 1809 Specifications: ABNF", STD 68, RFC 5234, January 2008. 1811 9.2. Informative References 1813 [CHAR_SET] 1814 , "IANA Character Sets registry", , . 1817 [ELF] Phillip M. Hallam-Baker, . and . Brian Behlendorf, 1818 "Extended Log File Format, W3C (work in progress), WD- 1819 logfile-960323", , . 1821 [I-D.brandenburg-cdni-has] 1822 Brandenburg, R., Deventer, O., Faucheur, F., and K. Leung, 1823 "Models for adaptive-streaming-aware CDN Interconnection", 1824 draft-brandenburg-cdni-has-05 (work in progress), April 1825 2013. 1827 [I-D.ietf-cdni-framework] 1828 Peterson, L. and B. Davie, "Framework for CDN 1829 Interconnection", draft-ietf-cdni-framework-03 (work in 1830 progress), February 2013. 1832 [I-D.ietf-cdni-metadata] 1833 Niven-Jenkins, B., Murray, R., Watson, G., Caulfield, M., 1834 Leung, K., and K. Ma, "CDN Interconnect Metadata", draft- 1835 ietf-cdni-metadata-01 (work in progress), February 2013. 1837 [I-D.ietf-cdni-requirements] 1838 Leung, K. and Y. Lee, "Content Distribution Network 1839 Interconnection (CDNI) Requirements", draft-ietf-cdni- 1840 requirements-09 (work in progress), July 2013. 1842 [I-D.snell-atompub-link-extensions] 1843 Snell, J., "Atom Link Extensions", draft-snell-atompub- 1844 link-extensions-09 (work in progress), June 2012. 1846 [RFC1321] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321, 1847 April 1992. 1849 [RFC2818] Rescorla, E., "HTTP Over TLS", RFC 2818, May 2000. 1851 [RFC6707] Niven-Jenkins, B., Le Faucheur, F., and N. Bitar, "Content 1852 Distribution Network Interconnection (CDNI) Problem 1853 Statement", RFC 6707, September 2012. 1855 [RFC6770] Bertrand, G., Stephan, E., Burbridge, T., Eardley, P., Ma, 1856 K., and G. Watson, "Use Cases for Content Delivery Network 1857 Interconnection", RFC 6770, November 2012. 1859 Appendix A. Requirements 1861 A.1. Compliance with cdni-requirements 1863 This section discusses compliance of the present specification 1864 against all the relevant requirements of 1865 [I-D.ietf-cdni-requirements]. 1867 [Editor's Note: we may want to re-structure this into a table that 1868 would more clearly show compliance level] 1870 A.1.1. General requirements 1872 Some of the general CDNI requirements defined in 1873 [I-D.ietf-cdni-requirements] are not applicable to the CDNI Logging 1874 Interface [GEN-5, GEN-6, GEN-7, GEN-8, GEN-9, GEN-12]. 1876 The Logging Interface does not define any new protocols [GEN-1], does 1877 not require any change or upgrade on the user agent [GEN-2] or on the 1878 Content Service Provider side [GEN-3]. Also, no intra-CDN 1879 information is necessary [GEN-4] and the CDNI Logging Interface 1880 supports any interconnection topology [GEN-10]. However, The CDNI 1881 Logging Interface does not define a specific loop avoidance mechanism 1882 [GEN-11], but the exchange of logs is usually done in a point to 1883 point manner between two well identified entities situated 1884 respectively in the uCDN and the dCDN. 1886 The CDNI Loggin Interface supports specific logging for the HTTP 1887 Adaptive Streaming content. [I-D.brandenburg-cdni-has] offers more 1888 details about particular logging fields required for HTTP Adaptive 1889 Streaming. 1891 A.1.2. Logging Interface requirements 1893 Reliable transfer is achieved by the transport protocol: the logging 1894 information is transmitted over HTTP running over TCP [LI-1]. 1896 The CDNI Logging Interface supports logs for all content deliveries 1897 both complete and incomplete performed by the dCDN on behalf of the 1898 uCDN [LI-2]. 1900 The CDNI Logging Interface does not impose any restrictions related 1901 to the transmission of logs generated by intermediary CDNs. The dCDN 1902 formats internally all the final logging files, including those 1903 received from intermediary CDNs and the files locally generated. The 1904 dCDN then sends all required logging files to the uCDN [LI-3]. 1906 The ATOM feed allows the uCDN to trigger the download of logging 1907 files whenever needed [LI-4]. 1909 The uCDN can pull logging files from the dCDN whenever a new file is 1910 available. The timing constraints for the generation of the logging 1911 files are to be defined offline, since the CDNI Logging Interface 1912 does not include a negotiation mechanism for the frequency of logging 1913 file generation. Note that the current version of this document 1914 refers strictly to non real-time logging [LI-5]. 1916 Section Section 3.4 describes the CDNI Logging Records and the 1917 possible fields that can be included in a record [LI-6]. 1919 As a transport mechanism, the CDNI Logging Interface uses the ATOM 1920 protocol over HTTP (or HTTPS) [LI-7]. 1922 A CDN can query another CDN for relevant current logging files by 1923 using the ATOM feed that allows to check for newly published content. 1924 Note that the current version of this document refers strictly to non 1925 real-time logging [LI-8]. 1927 The current version of the document does not specify any mechanisms 1928 for producing aggregate / summarized logs, but the exchanged logging 1929 files provide all information that is necessary to the uCDN in order 1930 to obtain aggregated logs. Future versions might include such 1931 mechanisms [LI-9]. 1933 No logging of performance data or consumed resources for the dCDN 1934 itself or any other cascaded CDN is included in the current version 1935 of the document. Future versions of this document might define such 1936 information [LI-10, LI-11, LI-12]. 1938 The current version of the document specifies the logging information 1939 related strictly to the delivery process. Logging files for any 1940 other functionalities (e.g., content purge, request routing events 1941 etc.) might be taken into account in future versions [LI-13]. 1943 Extensibility, the logging and exchange of proprietary information 1944 fields are detailed in Section 6 IANA Considerations [LI-14, LI-15]. 1946 The ATOM protocol allows the dCDN to publish the list of available 1947 resources (i.e. logging files) [LI-16]. 1949 Section 3.4 provides details about the fields of the HTTP Adaptive 1950 Streaming specific logging records, including the Content Collection 1951 Identifier (s-ccid) and Session Identifier (s-sid) [LI-17]. 1953 A.1.3. Security requirements 1955 [SEC-3, SEC-5] are not applicable to the CDNI Loggin Interface, all 1956 remaining security requirements are addressed as discussed in 1957 Section 7. 1959 A.2. Considerations on CDNI Logging Applicability 1961 This section discusses a number of considerations related to the 1962 applicability of the CDNI Logging interface as specified in the 1963 present document. 1965 [Editor's note: How do we incorporate this info into the I-D: in 1966 appendix? in main body? does it remain after publication or is 1967 temporary?] 1969 A.2.1. Timeliness 1971 Some applications consuming CDNI Logging information, such as 1972 accounting or trend analytics, only require logging information to be 1973 available with a timeliness of the order of a day or the hour. This 1974 document focuses on addressing this requirement. 1976 Some applications consuming CDNI Logging information, such as real- 1977 time analytics, require logging information to be available in real- 1978 time (i.e. of the order of a second after the corresponding event). 1979 This document leaves this requirement out of scope. 1981 A.2.2. Reliability 1983 CDNI logging information must be transmitted reliably. The transport 1984 protocol should contain an anti-replay mechanism. 1986 A.2.3. Security 1988 CDNI logging information exchange must allow authentication, 1989 integrity protection, and confidentiality protection. Also, a non- 1990 repudiation mechanism is mandatory, the transport protocol should 1991 support it. 1993 A.2.4. Scalability 1995 CDNI logging information exchange must support large scale 1996 information exchange, particularly so in the presence of HTTP 1997 Adaptive Streaming. 1999 For example, if we consider a client pulling HTTP Progressive 2000 Download content with an average duration of 10 minutes, this 2001 represents 1/600 CDNI delivery Logging Records per second. If we 2002 assume the dCDN is simultaneously serving 100,000 such clients on 2003 behalf of the uCDN, the dCDN will be generating 167 Logging Records 2004 per second to be communicated to the uCDN over the CDNI Logging 2005 interface. Or equivalently, if we assume an average delivery rate of 2006 2Mb/s, the dCDN generates 0.83 CDNI Logging Records per second for 2007 every Gb/s of streaming on behalf of the uCDN. 2009 For example, if we consider a client pulling HAS content and 2010 receiving a video chunk every 2 seconds, a separate audio chunck 2011 every 2 seconds and a refreshed manifest every 10 seconds, this 2012 represents 1.1 delivery Logging Record per second. If we assume the 2013 dCDN is simultaneously serving 100,000 such clients on behalf of the 2014 uCDN, the dCDN will be generating 110,000 Logging Records per second 2015 to be communicated to the uCDN over the CDNI Logging interface. Or 2016 equivalently, if we assume an average delivery rate of 2Mb/s, the 2017 dCDN generates 550 CDNI Logging Records per second for every Gb/s of 2018 streaming on behalf of the uCDN. 2020 A.2.5. Consistency between CDNI Logging and CDN Logging 2022 There are benefits in using a CDNI logging format as close as 2023 possible to intra-CDN logging format commonly used in CDNs today in 2024 order to minimize systematic translation at CDN/CDNI boundary. 2026 A.2.6. Dispatching/Filtering 2028 When a CDN is acting as a dCDN for multiple uCDNs, the dCDN needs to 2029 dispatch each CDNI Logging Record to the uCDN that redirected the 2030 corresponding request. The CDNI Logging format need to allow, and 2031 possibly facilitate, such a dispatching. 2033 Authors' Addresses 2035 Francois Le Faucheur (editor) 2036 Cisco Systems 2037 E.Space Park - Batiment D 2038 6254 Allee des Ormes - BP 1200 2039 Mougins cedex 06254 2040 FR 2042 Phone: +33 4 97 23 26 19 2043 Email: flefauch@cisco.com 2045 Gilles Bertrand (editor) 2046 Orange 2047 38-40 rue du General Leclerc 2048 Issy les Moulineaux 92130 2049 FR 2051 Phone: +33 1 45 29 89 46 2052 Email: gilles.bertrand@orange.com 2054 Iuniana Oprescu (editor) 2055 Orange 2056 38-40 rue du General Leclerc 2057 Issy les Moulineaux 92130 2058 FR 2060 Phone: +33 6 89 06 92 72 2061 Email: iuniana.oprescu@orange.com 2063 Roy Peterkofsky 2064 Skytide, Inc. 2065 One Kaiser Plaza, Suite 785 2066 Oakland CA 94612 2067 USA 2069 Phone: +01 510 250 4284 2070 Email: roy@skytide.com