idnits 2.17.1 draft-ietf-cdni-logging-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 18, 2013) is 3837 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 2616 (Obsoleted by RFC 7230, RFC 7231, RFC 7232, RFC 7233, RFC 7234, RFC 7235) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) == Outdated reference: A later version (-14) exists of draft-ietf-cdni-framework-05 == Outdated reference: A later version (-21) exists of draft-ietf-cdni-metadata-02 == Outdated reference: A later version (-17) exists of draft-ietf-cdni-requirements-10 -- Obsolete informational reference (is this intentional?): RFC 2818 (Obsoleted by RFC 9110) Summary: 2 errors (**), 0 flaws (~~), 4 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force F. Le Faucheur, Ed. 3 Internet-Draft Cisco Systems 4 Intended status: Standards Track G. Bertrand, Ed. 5 Expires: April 21, 2014 I. Oprescu, Ed. 6 Orange 7 R. Peterkofsky 8 Skytide, Inc. 9 October 18, 2013 11 CDNI Logging Interface 12 draft-ietf-cdni-logging-08 14 Abstract 16 This memo specifies the Logging interface between a downstream CDN 17 (dCDN) and an upstream CDN (uCDN) that are interconnected as per the 18 CDN Interconnection (CDNI) framework. First, it describes a 19 reference model for CDNI logging. Then, it specifies the CDNI 20 Logging File format and the actual protocol for exchange of CDNI 21 Logging Files. 23 Status of This Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at http://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on April 21, 2014. 40 Copyright Notice 42 Copyright (c) 2013 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (http://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 58 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 59 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 4 60 2. CDNI Logging Reference Model . . . . . . . . . . . . . . . . 5 61 2.1. CDNI Logging interactions . . . . . . . . . . . . . . . . 5 62 2.2. Overall Logging Chain . . . . . . . . . . . . . . . . . . 8 63 2.2.1. Logging Generation and During-Generation Aggregation 9 64 2.2.2. Logging Collection . . . . . . . . . . . . . . . . . 10 65 2.2.3. Logging Filtering . . . . . . . . . . . . . . . . . . 10 66 2.2.4. Logging Rectification and Post-Generation Aggregation 11 67 2.2.5. Log-Consuming Applications . . . . . . . . . . . . . 11 68 2.2.5.1. Maintenance/Debugging . . . . . . . . . . . . . . 11 69 2.2.5.2. Accounting . . . . . . . . . . . . . . . . . . . 12 70 2.2.5.3. Analytics and Reporting . . . . . . . . . . . . . 12 71 2.2.5.4. Security . . . . . . . . . . . . . . . . . . . . 12 72 2.2.5.5. Legal Logging Duties . . . . . . . . . . . . . . 12 73 2.2.5.6. Notions common to multiple Log Consuming 74 Applications . . . . . . . . . . . . . . . . . . 13 75 3. CDNI Logging File . . . . . . . . . . . . . . . . . . . . . . 15 76 3.1. Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 15 77 3.2. CDNI Logging File Structure . . . . . . . . . . . . . . . 16 78 3.3. CDNI Logging File Directives . . . . . . . . . . . . . . 18 79 3.4. CDNI Logging Records . . . . . . . . . . . . . . . . . . 21 80 3.4.1. HTTP Request Logging Record . . . . . . . . . . . . . 22 81 3.5. CDNI Logging File Example . . . . . . . . . . . . . . . . 29 82 4. CDNI Logging File Exchange Protocol . . . . . . . . . . . . . 30 83 4.1. CDNI Logging Feed . . . . . . . . . . . . . . . . . . . . 30 84 4.1.1. Atom Formatting . . . . . . . . . . . . . . . . . . . 31 85 4.1.2. Updates to Log Files and the Feed . . . . . . . . . . 31 86 4.1.3. Redundant Feeds . . . . . . . . . . . . . . . . . . . 32 87 4.1.4. Example CDNI Logging Feed . . . . . . . . . . . . . . 32 88 4.2. CDNI Logging File Pull . . . . . . . . . . . . . . . . . 33 89 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 35 90 5.1. CDNI Logging Directive Names Registry . . . . . . . . . . 35 91 5.2. CDNI Logging Record-Types Registry . . . . . . . . . . . 35 92 5.3. CDNI Logging Field Names Registry . . . . . . . . . . . . 36 93 5.4. CDNI Logging MIME Media Type . . . . . . . . . . . . . . 37 94 6. Security Considerations . . . . . . . . . . . . . . . . . . . 37 95 6.1. Authentication, Confidentiality, Integrity Protection . . 37 96 6.2. Denial of Service . . . . . . . . . . . . . . . . . . . . 38 97 6.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 38 98 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 38 99 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 39 100 8.1. Normative References . . . . . . . . . . . . . . . . . . 39 101 8.2. Informative References . . . . . . . . . . . . . . . . . 39 102 Appendix A. Compliance with CDNI Requirements . . . . . . . . . 40 103 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 45 105 1. Introduction 107 This memo specifies the CDNI Logging interface between a downstream 108 CDN (dCDN) and an upstream CDN (uCDN). First, it describes a 109 reference model for CDNI logging. Then, it specifies the CDNI 110 Logging File format and the actual protocol for exchange of CDNI 111 Logging Files. 113 The reader should be familiar with the following documents: 115 o CDNI problem statement [RFC6707] and framework 116 [I-D.ietf-cdni-framework] identify a Logging interface, 118 o Section 8 of [I-D.ietf-cdni-requirements] specifies a set of 119 requirements for Logging, 121 o [RFC6770] outlines real world use-cases for interconnecting CDNs. 122 These use cases require the exchange of Logging information 123 between the dCDN and the uCDN. 125 As stated in [RFC6707], "the CDNI Logging interface enables details 126 of logs or events to be exchanged between interconnected CDNs". 128 The present document describes: 130 o The CDNI Logging reference model (Section 2), 132 o The CDNI Logging File format (Section 3), 134 o The CDNI Logging File Exchange protocol (Section 4). 136 1.1. Terminology 138 In this document, the first letter of each CDNI-specific term is 139 capitalized. We adopt the terminology described in [RFC6707] and 140 [I-D.ietf-cdni-framework], and extend it with the additional terms 141 defined below. 143 For clarity, we use the word "Log" only for referring to internal CDN 144 logs and we use the word "Logging" for any inter-CDN information 145 exchange and processing operations related to CDNI Logging interface. 146 Log and Logging formats may be different. 148 CDN Logging information: logging information generated and collected 149 within a CDN 151 CDNI Logging information: logging information exchanged across CDNs 152 using the CDNI Logging Interface 154 Logging information: logging information generated and collected 155 within a CDN or obtained from another CDN using the CDNI Logging 156 Interface 158 CDNI Logging Field: an atomic element of information that can be 159 included in a CDNI Logging Record. The time an event/task started, 160 the IP address of an End user to whom content was delivered, and the 161 URI of the content delivered are examples of CDNI Logging Fields. 163 CDNI Logging Record: an information record providing information 164 about a specific event. This comprises a collection of CDNI Logging 165 Fields. 167 CDNI Logging File: a file containing CDNI Logging Records, as well as 168 additional information facilitating the processing of the CDNI 169 Logging Records. 171 CDN Reporting: the process of providing the relevant information that 172 will be used to create a formatted content delivery report provided 173 to the CSP in deferred time. Such information typically includes 174 aggregated data that can cover a large period of time (e.g., from 175 hours to several months). Uses of Reporting include the collection 176 of charging data related to CDN services and the computation of Key 177 Performance Indicators (KPIs). 179 CDN Monitoring: the process of providing content delivery information 180 in real-time. Monitoring typically includes data in real time to 181 provide visibility of the deliveries in progress, for service 182 operation purposes. It presents a view of the global health of the 183 services as well as information on usage and performance, for network 184 services supervision and operation management. In particular, 185 monitoring data can be used to generate alarms. 187 1.2. Requirements Language 189 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 190 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 191 document are to be interpreted as described in RFC 2119 [RFC2119]. 193 2. CDNI Logging Reference Model 195 2.1. CDNI Logging interactions 197 The CDNI logging reference model between a given uCDN and a given 198 dCDN involves the following interactions: 200 o customization by the uCDN of the CDNI logging information to be 201 provided by the dCDN to the uCDN (e.g. control of which logging 202 fields are to be communicated to the uCDN for a given task 203 performed by the dCDN, control of which types of events are to be 204 logged). The dCDN takes into account this CDNI logging 205 customization information to determine what logging information to 206 provide to the uCDN, but it may, or may not, take into account 207 this CDNI logging customization information to influence what CDN 208 logging information is to be generated and collected within the 209 dCDN (e.g. even if the uCDN requests a restricted subset of the 210 logging information, the dCDN may elect to generate a broader set 211 of logging information). The mechanism to support the 212 customisation by the uCDN of CDNI Logging information is outside 213 the scope of this document and left for further study. We note 214 that the CDNI Control interface or the CDNI Metadata interface 215 appear as candidate interfaces on which to potentially build such 216 a customisation mechanism in the future. Before such a mechanism 217 is available, the uCDN and dCDN are expected to agree off-line on 218 what CDNI logging information is to be provide by dCDN to UCDN and 219 rely on management plane actions to configure the CDNI Logging 220 functions to generate (respectively, expect) in dCDN 221 (respectively, in uCDN). 223 o generation and collection by the dCDN of logging information 224 related to the completion of any task performed by the dCDN on 225 behalf of the uCDN (e.g., delivery of the content to an end user) 226 or related to events happening in the dCDN that are relevant to 227 the uCDN (e.g., failures or unavailability in dCDN). This takes 228 place within the dCDN and does not directly involve CDNI 229 interfaces. 231 o communication by the dCDN to the uCDN of the logging information 232 collected by the dCDN relevant to the uCDN. This is supported by 233 the CDNI Logging interface and in the scope of the present 234 document. For example, the uCDN may use this logging information 235 to charge the CSP, to perform analytics and monitoring for 236 operational reasons, to provide analytics and monitoring views on 237 its content delivery to the CSP or to perform trouble-shooting. 238 This document exclusively specifies non-real-time exchange of 239 logging information. Closer to real-time exchange of logging 240 information (say sub-minute or sub-second) is outside the scope of 241 the present document and left for further study. This document 242 exclusively specifies exchange of logging information related to 243 content delivery. Exchange of logging information related to 244 operational events (e.g. dCDN request routing function 245 unavailable, content acquisition failure by dCDN) for audit or 246 operational reactive adjustments by uCDN is outside the scope of 247 the present document and left for further study. 249 o customization by the dCDN of the logging to be performed by the 250 uCDN on behalf of the dCDN. The mechanism to support the 251 customisation by the dCDN of CDNI Logging information is outside 252 the scope of this document and left for further study. 254 o generation and collection by the uCDN of logging information 255 related to the completion of any task performed by the uCDN on 256 behalf of the dCDN (e.g., serving of content by uCDN to dCDN for 257 acquisition purposes by dCDN) or related to events happening in 258 the uCDN that are relevant to the dCDN. This takes place within 259 the uCDN and does not directly involve CDNI interfaces. 261 o communication by the uCDN to the dCDN of the logging information 262 collected by the uCDN relevant to the dCDN. For example, the dCDN 263 might potentially benefit from this information for security 264 auditing or content acquisition troubleshooting. This is outside 265 the scope of this document and left for further study. 267 Figure 1 provides an example of CDNI Logging interactions (focusing 268 only on the interactions that are in the scope of this document) in a 269 particular scenario where 4 CDNs are involved in the delivery of 270 content from a given CSP: the uCDN has a CDNI interconnection with 271 dCDN-1 and dCDN-2. In turn, dCDN2 has a CDNI interconnection with 272 dCDN3. In this example, uCDN, dCDN-1, dCDN-2 and dCDN-3 all 273 participate in the delivery of content for the CSP. In this example, 274 the CDNI Logging interface enables the uCDN to obtain logging 275 information from all the dCDNs involved in the delivery. In the 276 example, uCDN uses the Logging data: 278 o to analyze the performance of the delivery operated by the dCDNs 279 and to adjust its operations after the fact (e.g., request 280 routing) as appropriate, 282 o to provide (non real-time) reporting and monitoring information to 283 CSP. 285 For instance, uCDN merges Logging data, extracts relevant KPIs, and 286 presents a formatted report to the CSP, in addition to a bill for the 287 content delivered by uCDN itself or by its dCDNs on his behalf. uCDN 288 may also provide Logging data as raw log files to the CSP, so that 289 the CSP can use its own logging analysis tools. 291 +-----+ 292 | CSP | 293 +-----+ 294 ^ Reporting and monitoring data 295 * Billing 296 ,--*--. 297 Logging ,-' `-. 298 Data =>( uCDN )<= Logging 299 // `-. _,-' \\ Data 300 || `-'-'-' || 301 ,-----. ,-----. 302 ,-' `-. ,-' `-. 303 ( dCDN-1 ) ( dCDN-2 )<== Logging 304 `-. ,-' `-. _,-' \\ Data 305 `--'--' `--'-' || 306 ,-----. 307 ,' `-. 308 ( dCDN-3 ) 309 `. ,-' 310 `--'--' 312 ===> CDNI Logging Interface 313 ***> outside the scope of CDNI 315 Figure 1: Interactions in CDNI Logging Reference Model 317 A dCDN (e.g., dCDN-2) integrates the relevant logging information 318 obtained from its dCDNs (e.g., dCDN-3) in the logging information 319 that it provides to the uCDN, so that the uCDN ultimately obtains all 320 logging information relevant to a CSP for which it acts as the 321 authoritative CDN. 323 Note that the format of Logging information that a CDN provides over 324 the CDNI interface might be different from the one that the CDN uses 325 internally. In this case, the CDN needs to reformat the Logging 326 information before it provides this information to the other CDN over 327 the CDNI Logging interface. Similarly, a CDN might reformat the 328 Logging data that it receives over the CDNI Logging interface before 329 injecting it into its log-consuming applications or before providing 330 some of this logging information to the CSP. Such reformatting 331 operations introduce latency in the logging distribution chain and 332 introduce a processing burden. Therefore, there are benefits in 333 specifying CDNI Logging format that are suitable for use inside CDNs 334 and also are close to the CDN Log formats commonly used in CDNs 335 today. 337 2.2. Overall Logging Chain 339 This section discusses the overall logging chain within and across 340 CDNs to clarify how CDN Logging information is expected to fit in 341 this overall chain. Figure 2 illustrates the overall logging chain 342 within the dCDN, across CDNs using the CDNI Logging interface and 343 within the uCDN. Note that the logging chain illustrated in the 344 Figure is obviously only an example and varies depending on the 345 specific environments. For example, there may be more or less 346 instantiations of each entity (i.e., there may be 4 Log consuming 347 applications in a given CDN). As another example, there may be one 348 instance of Rectification process per Log Consuming Application 349 instead of a shared one. 351 Log Consuming Log Consuming 352 App App 353 /\ /\ 354 | | 355 Rectification-------- 356 /\ 357 | 358 Filtering 359 /\ 360 | 361 Collection uCDN 362 /\ /\ 363 | | 364 | Generation 365 | 366 CDNI Logging --------------------------------------------- 367 exchange 368 /\ Log Consuming Log Consuming 369 | App App 370 | /\ /\ 371 | | | 372 Rectification Rectification--------- 373 /\ /\ 374 | | 375 Filtering 376 /\ 377 | 378 Collection dCDN 379 /\ /\ 380 | | 381 Generation Generation 383 Figure 2: CDNI Logging in the overall Logging Chain 385 The following subsections describe each of the processes potentially 386 involved in the logging chain of Figure 2. 388 2.2.1. Logging Generation and During-Generation Aggregation 390 CDNs typically generate logging information for all significant task 391 completions, events, and failures. Logs are typically generated by 392 many devices in the CDN including the surrogates, the request routing 393 system, and the control system. 395 The amount of Logging information generated can be huge. Therefore, 396 during contract negotiations, interconnected CDNs often agree on a 397 Logging retention duration, and optionally, on a maximum size of the 398 Logging data that the dCDN must keep. If this size is exceeded, the 399 dCDN must alert the uCDN but may not keep more Logs for the 400 considered time period. In addition, CDNs may aggregate logs and 401 transmit only summaries for some categories of operations instead of 402 the full Logging data. Note that such aggregation leads to an 403 information loss, which may be problematic for some usages of Logging 404 (e.g., debugging). 406 [RFC6983] discusses logging for HTTP Adaptive Streaming (HAS). In 407 accordance with the recommendations articulated there, it is expected 408 that a surrogate will generate separate logging information for 409 delivery of each chunk of HAS content. This ensures that separate 410 logging information can then be provided to interconnected CDNs over 411 the CDNI Logging interface. Still in line with the recommendations 412 of [RFC6983], the logging information for per-chunck delivery may 413 include some information (a Content Collection IDentifier and a 414 Session IDentifier) intended to facilitate subsequent post-generation 415 aggregation of per-chunk logs into per-session logs. Note that a CDN 416 may also elect to generate aggregate per-session logs when performing 417 HAS delivery, but this needs to be in addition to, and not instead 418 of, the per-chunk delivery logs. We note that this may be revisited 419 in future versions of this document. 421 Note that in the case of non real-time logging, the trigger of the 422 transmission or generation of the logging file appears to be a 423 synchronous process from a protocol standpoint. The implementation 424 algorithm can choose to enforce a maximum size for the logging file 425 beyond which the transmission is automatically triggered (and thus 426 allow for an asynchronous transmission process). 428 2.2.2. Logging Collection 430 This is the process that continuously collects logs generated by the 431 log-generating entities within a CDN. 433 In a CDNI environment, in addition to collecting logging information 434 from log-generating entities within the local CDN, the Collection 435 process also collects logging information provided by another CDN, or 436 other CDNs, through the CDNI Logging interface. This is illustrated 437 in Figure 2 where we see that the Collection process of the uCDN 438 collects logging information from log-generating entities within the 439 uCDN as well as logging information coming through CDNI Logging 440 exchange with the dCDN through the CDNI Logging interface. 442 2.2.3. Logging Filtering 444 A CDN may require to only present different subset of the whole 445 logging information collected to various log-consuming applications. 446 This is achieved by the Filtering process. 448 In particular, the Filtering process can also filter the right subset 449 of information that needs to be provided to a given interconnected 450 CDN. For example, the filtering process in the dCDN can be used to 451 ensure that only the logging information related to tasks performed 452 on behalf of a given uCDN are made available to that uCDN (thereby 453 filtering all the logging information related to deliveries by the 454 dCDN of content for its own CSPs). Similarly, the Filtering process 455 may filter or partially mask some fields, for example, to protect End 456 Users' privacy when communicating CDNI Logging information to another 457 CDN. Filtering of logging information prior to communication of this 458 information to other CDNs via the CDNI Logging interface requires 459 that the downstream CDN can recognize the set of log records that 460 relate to each interconnected CDN. 462 The CDN will also filter some internal scope information such as 463 information related to its internal alarms (security, failures, load, 464 etc). 466 In some use cases described in [RFC6770], the interconnected CDNs do 467 not want to disclose details on their internal topology. The 468 filtering process can then also filter confidential data on the 469 dCDNs' topology (number of servers, location, etc.). In particular, 470 information about the requests served by every Surrogate may be 471 confidential. Therefore, the Logging information must be protected 472 so that data such as Surrogates' hostnames is not disclosed to the 473 uCDN. In the "Inter-Affiliates Interconnection" use case, this 474 information may be disclosed to the uCDN because both the dCDN and 475 the uCDN are operated by entities of the same group. 477 2.2.4. Logging Rectification and Post-Generation Aggregation 479 If Logging is generated periodically, it is important that the 480 sessions that start in one Logging period and end in another are 481 correctly reported. If they are reported in the starting period, 482 then the Logging of this period will be available only after the end 483 of the session, which delays the Logging generation. 485 A Logging rectification/update mechanism could be useful to reach a 486 good trade-off between the Logging generation delay and the Logging 487 accuracy. Depending on the selected Logging protocol(s), such 488 mechanism may be invaluable for real time Logging, which must be 489 provided rapidly and cannot wait for the end of operations in 490 progress. 492 In the presence of HAS, some log-consuming applications can benefit 493 from aggregate per-session logs. For example, for analytics, per- 494 session logs allow display of session-related trends which are much 495 more meaningful for some types of analysis than chunk-related trends. 496 In the case where the log-generating entities have generated during- 497 generation aggregate logs, those can be used by the applications. In 498 the case where aggregate logs have not been generated, the 499 Rectification process can be extended with a Post-Generation 500 Aggregation process that generates per-session logs from the per- 501 chunk logs, possibly leveraging the information included in the per- 502 chunk logs for that purpose (Content Collection IDentifier and a 503 Session IDentifier). However, in accordance with [RFC6983], this 504 document does not define exchange of such aggregate logs on the CDNI 505 Logging interface. We note that this may be revisited in future 506 versions of this document. 508 2.2.5. Log-Consuming Applications 510 2.2.5.1. Maintenance/Debugging 512 Logging is useful to permit the detection (and limit the risk) of 513 content delivery failures. In particular, Logging facilitates the 514 detection of configuration issues. 516 To detect faults, Logging must enable the reporting of any CDN 517 delivery operation success and failure. The uCDN can summarize such 518 information into KPIs. For instance, Logging needs to allow the 519 computation of the number of times, during a given time period, that 520 content delivery related to a specific service succeeds/fails. 522 Logging enables the CDN providers to identify and troubleshoot 523 performance degradations. In particular, Logging enables the 524 communication of traffic data (e.g., the amount of traffic that has 525 been forwarded by a dCDN on behalf of an uCDN over a given period of 526 time), which is particularly useful for CDN and network planning 527 operations. 529 2.2.5.2. Accounting 531 Logging is essential for accounting, to permit inter-CDN billing and 532 CSP billing by uCDNs. For instance, Logging information provided by 533 dCDNs enables the uCDN to compute the total amount of traffic 534 delivered by every dCDN for a particular Content Provider, as well 535 as, the associated bandwidth usage (e.g., peak, 95th percentile), and 536 the maximum number of simultaneous sessions over a given period of 537 time. 539 2.2.5.3. Analytics and Reporting 541 The goal of analytics is to gather any relevant information to track 542 audience, analyze user behavior, and monitor the performance and 543 quality of content delivery. For instance, Logging enables the CDN 544 providers to report on content consumption (e.g., delivered sessions 545 per content) in a specific geographic area. 547 The goal of reporting is to gather any relevant information to 548 monitor the performance and quality of content delivery and allow 549 detection of delivery issues. For instance, reporting could track 550 the average delivery throughput experienced by End-Users in a given 551 region for a specific CSP or content set over a period of time. 553 2.2.5.4. Security 555 The goal of security is to prevent and monitor unauthorized access, 556 misuse, modification, and denial of access of a service. A set of 557 information is logged for security purposes. In particular, a record 558 of access to content is usually collected to permit the CSP to detect 559 infringements of content delivery policies and other abnormal End 560 User behaviors. 562 2.2.5.5. Legal Logging Duties 563 Depending on the country considered, the CDNs may have to retain 564 specific Logging information during a legal retention period, to 565 comply with judicial requisitions. 567 2.2.5.6. Notions common to multiple Log Consuming Applications 569 2.2.5.6.1. Logging Information Views 571 Within a given log-consuming application, different views may be 572 provided to different users depending on privacy, business, and 573 scalability constraints. 575 For example, an analytics tool run by the uCDN can provide one view 576 to an uCDN operator that exploits all the logging information 577 available to the uCDN, while the tool may provide a different view to 578 each CSP exploiting only the logging information related to the 579 content of the given CSP. 581 As another example, maintenance and debugging tools may provide 582 different views to different CDN operators, based on their 583 operational role. 585 2.2.5.6.2. Key Performance Indicators (KPIs) 587 This section presents, for explanatory purposes, a non-exhaustive 588 list of Key Performance Indicators (KPIs) that can be extracted/ 589 produced from logs. 591 Multiple log-consuming applications, such as analytics, monitoring, 592 and maintenance applications, often compute and track such KPIs. 594 In a CDNI environment, depending on the situation, these KPIs may be 595 computed by the uCDN or by the dCDN. But it is usually the uCDN that 596 computes KPIs, because uCDN and dCDN may have different definitions 597 of the KPIs and the computation of some KPIs requires a vision of all 598 the deliveries performed by the uCDN and all its dCDNs. 600 Here is a list of important examples of KPIs: 602 o Number of delivery requests received from End-Users in a given 603 region for each piece of content, during a given period of time 604 (e.g., hour/day/week/month) 606 o Percentage of delivery successes/failures among the aforementioned 607 requests 609 o Number of failures listed by failure type (e.g., HTTP error code) 610 for requests received from End Users in a given region and for 611 each piece of content, during a given period of time (e.g., hour/ 612 day/week/month) 614 o Number and cause of premature delivery termination for End Users 615 in a given region and for each piece of content, during a given 616 period of time (e.g., hour/day/week/month) 618 o Maximum and mean number of simultaneous sessions established by 619 End Users in a given region, for a given Content Provider, and 620 during a given period of time (e.g., hour/day/week/month) 622 o Volume of traffic delivered for sessions established by End Users 623 in a given region, for a given Content Provider, and during a 624 given period of time (e.g., hour/day/week/month) 626 o Maximum, mean, and minimum delivery throughput for sessions 627 established by End Users in a given region, for a given Content 628 Provider, and during a given period of time (e.g., hour/day/week/ 629 month) 631 o Cache-hit and byte-hit ratios for requests received from End Users 632 in a given region for each piece of content, during a given period 633 of time (e.g., hour/day/week/month) 635 o Top 10 of the most popularly requested content (during a given day 636 /week/month), 638 o Terminal type (mobile, PC, STB, if this information can be 639 acquired from the browser type header, for example). 641 Additional KPIs can be computed from other sources of information 642 than the Logging, for instance, data collected by a content portal or 643 by specific client-side application programming interfaces. Such 644 KPIs are out of scope for the present memo. 646 The KPIs used depend strongly on the considered log-consuming 647 application -- the CDN operator may be interested in different 648 metrics than the CSP is. In particular, CDN operators are often 649 interested in delivery and acquisition performance KPIs, information 650 related to Surrogates' performance, caching information to evaluate 651 the cache-hit ratio, information about the delivered file size to 652 compute the volume of content delivered during peak hour, etc. 654 Some of the KPIs, for instance those providing an instantaneous 655 vision of the active sessions for a given CSP's content, are useful 656 essentially if they are provided in real-time. By contrast, some 657 other KPIs, such as the one averaged on a long period of time, can be 658 provided in non-real time. 660 3. CDNI Logging File 662 3.1. Rules 664 This specification uses the Augmented Backus-Naur Form (ABNF) 665 notation and core rules of [RFC5234]. In particular, the present 666 document uses the following rules from [RFC5234]: 668 CR = %x0D ; carriage return 670 DIGIT = %x30-39 ; 0-9 672 DQUOTE = %x22 ; " (Double Quote) 674 CRLF = CR LF ; Internet standard newline 676 HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F" 678 HTAB = %x09 ; horizontal tab 680 LF = %x0A ; linefeed 682 OCTET = %x00-FF ; 8 bits of data 684 The present document also uses the following rules from [RFC3986]: 686 host = as specified in section 3.2.2 of [RFC3986]. 688 IPv4address = as specified in section 3.2.2 of [RFC3986]. 690 IPv6address = as specified in section 3.2.2 of [RFC3986]. 692 The present document also defines the folowing additional rules: 694 ADDRESS = IPv4address / IPv6address 696 DATE = 4DIGIT "-" 2DIGIT "-" 2DIGIT 698 Dates are recorded in the format YYYY-MM-DD where YYYY, MM and 699 DD stand for the numeric year, month and day respectively. All 700 dates are specified in Universal Time Coordinated (UTC). 702 DEC = 1*DIGIT ["." *DIGIT] 704 QSTRING = DQUOTE *NDQUOTE DQUOTE ; where 705 NDQUOTE = / 2DQUOTE ; whereby a 706 DQUOTE is conveyed inside a QSTRING unambiguously by repeating 707 it. 709 NHTABSTRING = *NHTAB ; where 711 NHTAB = 713 TIME = 2DIGIT ":" 2DIGIT ":" 2DIGIT ["." *DIGIT] 715 Times are recorded in the form HH:MM:SS or HH:MM:SS.S where HH 716 is the hour in 24 hour format, MM is minutes and SS is seconds. 717 All times are specified in Universal Time Coordinated (UTC). 719 3.2. CDNI Logging File Structure 721 As defined in Section 1.1 a CDNI logging field is as an atomic 722 logging information element and a CDNI Logging Record is a collection 723 of CDNI Logging Fields containing all logging information 724 corresponding to a single logging event. This document defines a 725 third level of structure, the CDNI Logging File, that is a collection 726 of CDNI Logging Records. This structure is illustrated in Figure 3. 727 The use of a file structure for transfer of CDNI Logging information 728 is selected since this is the most common practise today for exchange 729 of logging information within and across CDNs. 731 +----------------------------------------------------------+ 732 |CDNI Logging File | 733 | | 734 | #Directive 1 | 735 | #Directive 2 | 736 | ... | 737 | #Directive P | 738 | | 739 | +------------------------------------------------------+ | 740 | |CDNI Logging Record 1 | | 741 | | +-------------+ +-------------+ +-------------+ | | 742 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 743 | | | Field 1 | | Field 2 | | Field N | | | 744 | | +-------------+ +-------------+ +-------------+ | | 745 | +------------------------------------------------------+ | 746 | | 747 | +------------------------------------------------------+ | 748 | |CDNI Logging Record 2 | | 749 | | +-------------+ +-------------+ +-------------+ | | 750 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 751 | | | Field 1 | | Field 2 | | Field N | | | 752 | | +-------------+ +-------------+ +-------------+ | | 753 | +------------------------------------------------------+ | 754 | | 755 | ... | 756 | | 757 | #Directive P+1 | 758 | | 759 | ... | 760 | | 761 | +------------------------------------------------------+ | 762 | |CDNI Logging Record M | | 763 | | +-------------+ +-------------+ +-------------+ | | 764 | | |CDNI Logging | |CDNI Logging | ... |CDNI Logging | | | 765 | | | Field 1 | | Field 2 | | Field N | | | 766 | | +-------------+ +-------------+ +-------------+ | | 767 | +------------------------------------------------------+ | 768 | | 769 | | 770 | #Directive P+Q | 771 +----------------------------------------------------------+ 773 Figure 3: Structure of Logging Files 775 The CDNI Logging File format is inspired from the W3C Extended Log 776 File Format [ELF]. However, it is fully specified by the present 777 document. Where the present document differs from the W3C Extended 778 Log File Format, an implementation of CDNI Logging MUST comply with 779 the present document. 781 Using a format that resembles the W3C Extended Log File Format is 782 intended to keep CDNI logging format close to intra-CDN logging 783 format commonly used in CDNs today, thereby minimizing systematic 784 translation at CDN/CDNI boundary. 786 A CDNI Logging File MUST contain a sequence of lines containing US- 787 ASCII characters [CHAR_SET] terminated by CRLF. 789 Each line of a CDNI Logging File MUST contain either a directive or a 790 CDNI Logging Record. 792 Directives record information about the CDNI Logging process itself. 793 Lines containing directives MUST begin with the "#" character. 794 Directives are specified in Section 3.3. 796 Logging Records provide actual details of the logged event. Logging 797 Records are specified in Section 3.4. 799 The CDNI File structure is defined by the following rules: 801 DIRLINE = "#" directive CRLF 803 DIRGROUP = 1*DIRLINE 805 RECLINE = CRLF 807 RECGROUP = *RECLINE 809 = 1* 811 3.3. CDNI Logging File Directives 813 The CDNI Logging File directives are defined by the following rules: 815 directive = DIRNAME ":" HTAB DIRVAL 817 DIRNAME = any CDNI Logging Directive name registered in the CDNI 818 Logging Directive Names registry (Section 5.1). 820 DIRVAL = 823 An implementation of the CDNI Logging interface MUST support all of 824 the following directives, listed below by their directive name: 826 o Version: 828 * format: "CDNI" "/" 1*DIGIT "." 1*DIGIT 830 * directive value: indicates the version of the CDNI Logging File 831 format. The value MUST be "CDNI/1.0" for the version specified 832 in the present document. 834 * occurrence: there MUST be one and only one instance of this 835 directive per CDNI Logging File. It MUST be the first line of 836 the CDNI Logging file. 838 o UUID: 840 * format: NHTABSTRING 842 * directive value: this a Universally Unique IDentifier (UUID) 843 from the UUID Uniform Resource Name (URN) namespace specified 844 in [RFC4122]) for the CDNI Logging File . 846 * occurrence: there MUST be one and only one instance of this 847 directive per CDNI Logging File. 849 o Claimed-Origin: 851 * format: host 853 * directive value: this contains the claimed identification of 854 the entity transmitting the CDNI Logging File (e.g. the host in 855 a dCDN supporting the CDNI Logging interface) or the entity 856 responsible for transmitting the CDNI Logging File (e.g. the 857 dCDN). 859 * occurrence: there MUST be zero or one instance of this 860 directive per CDNI Logging File. This directive MAY be 861 included by the dCDN. It MUST NOT be included or modified by 862 the uCDN. 864 o Verified-Origin: 866 * format: host 868 * directive value: this contains the identification, as 869 established by the entity receiving the CDNI Logging file, of 870 the entity transmitting the CDNI Logging File (e.g. the host in 871 a dCDN supporting the CDNI Logging interface) or the entity 872 responsible for transmitting the CDNI Logging File (e.g. the 873 dCDN). 875 * occurrence: there MUST be zero or one instance of this 876 directive per CDNI Logging File. This directive MAY be added 877 by the uCDN (e.g. before storing the CDNI Logging File). It 878 MUST NOT be included by the dCDN. The mechanisms used by the 879 uCDN to establish and validate the entity responsible for the 880 CDNI Logging File is outside the scope of the present document. 881 We observe that, in particular, this may be achieved through 882 authentication mechanisms that are part of the CDNI Logging 883 File pull mechanism (Section 4.2). 885 o Record-Type: 887 * format: NHTABSTRING 889 * directive value: indicates the type of the CDNI Logging Records 890 that follow this directive, until another Record-Type directive 891 (or the end of the CDNI Logging File). This can be any CDNI 892 Logging Record type registered in the CDNI Logging Record-types 893 registry (Section 5.2). "cdni_http_request_v1" MUST be 894 indicated as the Record-Type directive value for CDNI Logging 895 records corresponding to HTTP request (e.g. a HTTP delivery 896 request) as specified in Section 3.4.1. 898 * occurrence: there MUST be at least one instance of this 899 directive per CDNI Logging File. The first instance of this 900 directive MUST precede a Fields directive and precede any CDNI 901 Logging Record. 903 o Fields: 905 * format: FIENAME * ; where FIENAME can take any 906 CDNI Logging field name registered in the CDNI Logging Field 907 Names registry (Section 5.3). 909 * directive value: this lists the names of all the fields for 910 which a value is to appear in the CDNI Logging Records that 911 follow the instance of this directive (until another instance 912 of this directive). The names of the fields, as well as their 913 possible occurrences, are specified for each type of CDNI 914 Logging Records in Section 3.4. 916 * occurrence: there MUST be at least one instance of this 917 directive per Record-Type directive. The first instance of 918 this directive for a given Record-Type MUST appear before any 919 CDNI Logging Record for this Record-Type. 921 o Integrity-Hash: 923 * format: 32HEXDIG 925 * directive value: This directive permits the detection of a 926 corrupted CDNI Logging File. This can be useful, for instance, 927 if a problem occurs on the filesystem of the dCDN Logging 928 system and leads to a truncation of a logging file. The valid 929 Integrity-Hash value is included in this directive by the 930 entity that transmits the CDNI Logging File. It is computed by 931 applying the MD5 ([RFC1321]) cryptographic hash function on the 932 CDNI Logging File, including all the directives and logging 933 records, up to the Intergrity-Hash directive itself, excluding 934 the Integrity-Hash directive itself. The Integrity-Hash value 935 is represented as a US-ASCII encoded hexadecimal number, 32 936 digits long (representing a 128 bit hash value). The entity 937 receiving the CDNI Logging File also computes in a similar way 938 the MD5 hash on the received CDNI Logging File and compares 939 this hash to the value of the Integrity-Hash directive. If the 940 two values are equal, then the received CDNI Logging File MUST 941 be considered non-corrupted. If the two values are different, 942 the received CDNI Logging File MUST be considered corrupted. 943 The behavior of the entity that received a corrupted CDNI 944 Logging File is outside the scope of this specification; we 945 note that the entity MAY attempt to pull again the same CDNI 946 Logging file from the transmitting entity. If the entity 947 receiving the CDNI Logging File adds a Verified-Origin 948 directive, it MUST recompute and update the Integrity-Hash 949 directive so it also protects the added Verified-Origin 950 directive. 952 * occurrence: there MUST be zero or one instance of this 953 directive. There SHOULD be one instance of this directive. 954 One situation where that directive could be omitted is where 955 integrity protection is already provided via another mechanism 956 (for example if an integrity hash is associated to the CDNI 957 Logging file out of band through the CDNI Logging Logging Feed 958 Section 4.1 leveraging ATOM extensions such as those proposed 959 in [I-D.snell-atompub-link-extensions]. When present, this 960 field MUST be the last line of the CDNI Logging File. 962 3.4. CDNI Logging Records 964 A CDNI Logging Record consists of a sequence of CDNI Logging Fields 965 relating to that single CDNI Logging Record. 967 CDNI Logging Fields MUST be separated by the "horizontal tabulation 968 (HTAB)" character. 970 To facilitate readability, a prefix scheme is used for CDNI Logging 971 field names in a similar way to the one used in W3C Extended Log File 972 Format [ELF] . The semantics of the prefix in the present document 973 is: 975 o c: refers to the User Agent that issues the request (corresponds 976 to the "client" of W3C Extended Log Format) 978 o d: refers to the dCDN (relative to a given CDN acting as a uCDN) 980 o s: refers to the dCDN Surrogate that serves the request 981 (corresponds to the "server" of W3C Extended Log Format) 983 o u: refers to the uCDN (relative to a given CDN acting as a dCDN) 985 o cs: refers to communication from the User-Agent towards the dCDN 986 Surrogate 988 o sc: refers to communication from the dCDN Surrogate towards the 989 User-Agent 991 An implementation of the CDNI Logging interface as per the present 992 specification MUST support the CDNI HTTP Delivery Records as 993 specified in Section 3.4.1. 995 A CDNI Logging Record is defined by the following rules: 997 FIEVAL = 999 = FIEVAL * ; where FIEVAL 1000 contains the CDNI Logging field values corresponding to the CDNI 1001 Logging field names (FIENAME) listed is the last Fields directive 1002 predecing the present CDNI Logging Record. 1004 3.4.1. HTTP Request Logging Record 1006 The HTTP Request Logging Record is a CDNI Logging Record of Record- 1007 Type "cdni_http_request_v1". It contains the following CDNI Logging 1008 Fields, listed by their field name: 1010 o date: 1012 * format: DATE 1014 * field value: the date at which the processing of request 1015 completed on the Surrogate. 1017 * occurrence: there MUST be one and only one instance of this 1018 field. 1020 o time: 1022 * format: TIME 1024 * field value: the time at which the processing of request 1025 completed on the Surrogate. 1027 * occurrence: there MUST be one and only one instance of this 1028 field. 1030 o time-taken: 1032 * format: DEC 1034 * field value: decimal value of the duration, in seconds, between 1035 the start of the processing of the request and the completion 1036 of the request processing (e.g. completion of delivery) by the 1037 Surrogate. 1039 * occurrence: there MUST be one and only one instance of this 1040 field. 1042 o c-ip: 1044 * format: ADDRESS 1046 * field value: the source IPv4 or IPv6 address (i.e. the "client" 1047 address) in the request received by the Surrogate. 1049 * occurrence: there MUST be one and only one instance of this 1050 field. 1052 o c-ip-anonimizing: 1054 * format: 1*DIGIT 1056 * field value: the number of rightmost bits of the address in the 1057 c-ip field that are zeroed-out in order to anonymize the 1058 logging record. The mechanism by which the two ends of the 1059 CDNI Logging interface agree on whether anonimization is to be 1060 supported and the number of bits that need to be zeroed-out for 1061 this purpose are outside the scope of the present document. 1063 * occurrence: there MUST be zero or one instance of this field. 1065 o c-port: 1067 * format: 1*DIGIT 1069 * field value: the source TCP port (i.e. the "client" port) in 1070 the request received by the Surrogate. 1072 * occurrence: there MUST be zero or exactly one instance of this 1073 field. 1075 o s-ip: 1077 * format: ADDRESS 1079 * field value: the IPv4 or IPv6 address of the Surrogate that 1080 served the request (i.e. the "server" address). 1082 * occurrence: there MUST be zero or exactly one instance of this 1083 field. 1085 o s-hostname: 1087 * format: host 1089 * field value: the hostname of the Surrogate that served the 1090 request (i.e. the "server" hostname). 1092 * occurrence: there MUST be zero or exactly one instance of this 1093 field. 1095 o s-port: 1097 * format: 1*DIGIT 1099 * field value: the destination TCP port (i.e. the "server" port) 1100 in the request received by the Surrogate. 1102 * occurrence: there MUST be zero or exactly one instance of this 1103 field. 1105 o cs-method: 1107 * format: NHTABSTRING 1109 * field value: this is the HTTP method of the HTTP request 1110 received by the Surrogate. 1112 * occurrence: There MUST be one and only one instance of this 1113 field. 1115 o cs-uri: 1117 * format: NHTABSTRING 1119 * field value: this is the complete URL of the request received 1120 by the Surrogate. It is exactly in the format of a http_URL 1121 specified in [RFC2616]) or, when the request was a HTTPS 1122 request ([RFC2818]), it is in the format of a http_URL but with 1123 the scheme part set to "https" instead of "http". 1125 * occurrence: there MUST be zero or exactly one instance of this 1126 field. 1128 o u-uri: 1130 * format: NHTABSTRING 1132 * field value: this is a complete URL, derived from the complete 1133 URI of the request received by the Surrogate (i.e. the cs-uri) 1134 but transformed by the entity generating or transmitting the 1135 CDNI Logging Record, in a way that is agreed upon between the 1136 two ends of the CDNI Logging interface, so the transformed URI 1137 is meaningful to the uCDN. For example, the two ends of the 1138 CDNI Logging interface could agree that the u-uri is 1139 constructed from the cs-uri by removing the part of the 1140 hostname that exposes which individual Surrogate actually 1141 performed the delivery. The details of modification performed 1142 to generate the u-uri, as well as the mechanism to agree on 1143 these modifications between the two sides of the CDNI Logging 1144 interface are outside the scope of the present document. 1146 * occurrence: there MUST be one and only one instance of this 1147 field. 1149 o protocol: 1151 * format: NHTABSTRING 1153 * field value: this is value of the HTTP-Version field as 1154 specified in [RFC2616] of the Request-Line of the request 1155 received by the Surrogate (e.g. "HTTP/1.1"). 1157 * occurrence: there MUST be one and only one instance of this 1158 field. 1160 o sc-status: 1162 * format: 3DIGIT 1164 * field value: this is the HTTP Status-Code in the HTTP response 1165 from the Surrogate. 1167 * occurrence: There MUST be one and only one instance of this 1168 field. 1170 o sc-total-bytes: 1172 * format: 1*DIGIT 1174 * field value: this is the total number of bytes of the HTTP 1175 response sent by the Surrogate in response to the request. 1176 This includes the bytes of the Status-Line (including HTTP 1177 headers) and of the message-body. 1179 * occurrence: There MUST be one and only one instance of this 1180 field. 1182 o sc-entity-bytes: 1184 * format: 1*DIGIT 1186 * field value: this is the number of bytes of the message-body in 1187 the HTTP response sent by the Surrogate in response to the 1188 request. This does not include the bytes of the Status-Line 1189 (and therefore does not include the bytes of the HTTP headers). 1191 * occurrence: there MUST be zero or exactly one instance of this 1192 field. 1194 o cs(): 1196 * format: QSTRING 1198 * field value: the value of the HTTP header (identified by the 1199 in the CDNI Logging field name) as it 1200 appears in the request processed by the Surrogate. For 1201 example, when the CDNI Logging field name (FIENAME) listed in 1202 the prededing Fields directive is "cs(User-Agent"), this CDNI 1203 Logging field value contains the value of the User-Agent HTTP 1204 header as received by the Surrogate in the request it 1205 processed. 1207 * occurrence: there MUST be zero, one or any number of instance 1208 of this field. 1210 o sc(): 1212 * format: QSTRING 1214 * field value: the value of the HTTP header (identified by the 1215 in the CDNI Logging field name) as it 1216 appears in the response issued by the Surrogate to serve the 1217 request. 1219 * occurrence: there MUST be zero, one or any number of instance 1220 of this field. 1222 o s-ccid: 1224 * format: QSTRING 1226 * field value: this contains the value of the Content Collection 1227 IDentifier associated by the uCDN to the content served by the 1228 Surrogate via the CDNI Metadata interface 1229 ([I-D.ietf-cdni-metadata]). 1231 * occurrence: there MUST be zero or exactly one instance of this 1232 field. 1234 o s-sid: 1236 * format: QSTRING 1238 * field value: this contains the value of a Session IDentifier 1239 generated by the dCDN for a specific HTTP Adaptive Streaming 1240 (HAS) session and whose value is included in the Logging record 1241 for every content chunk delivery of that session in view of 1242 facilitating the later correlation of all the per content chunk 1243 log records of a given HAS session. See section 3.4.2.2. of 1244 [RFC6983] for more discussion on the concept of Session 1245 IDentifier. 1247 * occurrence: there MUST be zero or exactly one instance of this 1248 field. 1250 o s-cached: 1252 * format: 1DIGIT 1254 * field value: this characterises whether the Surrogate served 1255 the request using content already stored on its local cache or 1256 not. The allowed values are "0" (for miss) and "1" (for hit). 1257 "1" MUST be used when the Surrogate did serve the request using 1258 exclusively content already stored on its local cache. "0" MUST 1259 be used otherwise (including cases where the Surrogate served 1260 the request using some, but not all, content already stored on 1261 its local cache). Note that a "0" only means a cache miss in 1262 the Surrogate and does not provide any information on whether 1263 the content was already stored, or not, in another device of 1264 the dCDN i.e. whether this was a "dCDN hit" or "dCDN miss". 1266 * occurrence: there MUST be zero or exactly one instance of this 1267 field. 1269 The "Fields" directive corresponding to a HTTP Request Logging Record 1270 MUST list all the fields name whose occurrence is specified above as 1271 "There MUST be one and only one instance of this field". The 1272 corresponding fields value MUST be present in every HTTP Request 1273 Logging Record. 1275 The "Fields" directive corresponding to a HTTP Request Logging Record 1276 MAY list all the fields value whose occurrence is specified above as 1277 "there MUST be zero or exactly one instance of this field" or "there 1278 MUST be zero, one or any number of instance of this field". The set 1279 of such fields name actually listed in the "Fields" directive is 1280 selected by the implementation generating the CDNI Logging File based 1281 on agreements between the interconnected CDNs established through 1282 mechanisms outside the scope of this specification (e.g. contractual 1283 agreements). When such a field name is not listed in the "Fields" 1284 directive, the corresponding field value MUST NOT be included in the 1285 Logging Record. When such a field name is listed in the "Fields" 1286 directive, the corresponding field value MUST be included in the 1287 Logging Record; in that case, if the value for the field is not 1288 available, this MUST be conveyed via a dash character ("-"). 1290 The fields name listed in the "Fields" directive MAY be listed in the 1291 order in which they are listed in Section 3.4.1 or MAY be listed in 1292 any other order. 1294 A dCDN-side implementation of the CDNI Logging interface MUST support 1295 the ability to include valid values for the following Logging Fields 1296 in a CDNI Logging Record of Record-Type "cdni_http_request_v1": 1298 o date 1300 o time 1302 o time-taken 1304 o c-ip 1306 o c-port 1308 o s-ip 1310 o s-hostname 1312 o s-port 1314 o cs- method 1316 o cs-uri 1318 o u-uri 1320 o protocol 1322 o sc-status 1324 o sc- total-bytes 1326 o sc-entity-bytes 1328 o cs() 1330 o sc() 1331 o s-cached 1333 A dCDN-side implementation of the CDNI Logging interface MAY support 1334 the ability to include valid values for the following Logging Fields 1335 in a CDNI Logging Record of Record-Type "cdni_http_request_v1": 1337 o c-ip-anonimizing 1339 o s-ccid 1341 o s-sid 1343 An uCDN-side implementation of the CDNI Logging interface MUST be 1344 able to accept CDNI Logging Files with CDNI Logging Records of 1345 Record-Type "cdni_http_request_v1" containing any CDNI Logging Field 1346 defined in Section 3.4.1 as long as the CDNI Logging Record and the 1347 CDNI Logging File are compliant with the present document. 1349 3.5. CDNI Logging File Example 1351 #Version:CDNI/1.0 1353 #UUID:"urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6" 1355 #Claimed-Origin:cdni-logging-entity.dcdn.example.com 1357 #Record-Type:cdni_http_request_v1 1359 #Fields:datetimetime-takenc-ipcs- 1360 methodu-uriprotocolsc-statussc-total- 1361 bytescs(User-Agent)cs(Referer)s-cached 1363 2013-05-1700:38:06.8259.05810.5.7.1GETh 1364 ttp://cdni-ucdn.dcdn.example.com/video/movie100.mp4HTTP/ 1365 1.12006729891"Mozilla/5.0 (Windows; U; Windows NT 1366 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 1367 Safari /533.4""host1.example.com"1 1369 2013-05-1700:39:09.14515.3210.5.10.5GET 1370 http://cdni-ucdn.dcdn.example.com/video/movie118.mp4HTTP/ 1371 1.120015799210"Mozilla/5.0 (Windows; U; Windows NT 1372 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 1373 Safari /533.4""host1.example.com"1 1374 2013-05-1700:42:53.43752.87910.5.10.5GEThttp://cdni-ucdn.dcdn.example.com/video/picture11.mp4HTTP/ 1376 1.020097234724"Mozilla/5.0 (Windows; U; Windows NT 1377 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 1378 Safari /533.4""host5.example.com"0 1380 #Integrity-Hash:fe113dfce8fec91323a4fc02261af26e 1382 4. CDNI Logging File Exchange Protocol 1384 This document specifies a protocol for the exchange of CDNI Logging 1385 Files as specified in Section 3. 1387 This protocol comprises: 1389 o a CDNI Logging feed, allowing the dCDN to notify the uCDN about 1390 the CDNI Logging files that can be retrieved by that uCDN from the 1391 dCDN, as well as all the information necessary for retrieving each 1392 of these CDNI Logging File. The CDNI Logging feed is specified in 1393 Section 4.1. 1395 o a CDNI Logging File pull mechanism, allowing the uCDN to obtain 1396 from the dCDN a given CDNI Logging File at the uCDN convenience. 1397 The CDNI Logging File pull mechanisms is specified in Section 4.2. 1399 An implementation of the CDNI Logging interface as per the present 1400 document generating CDNI Logging file (i.e. on the dCDN side) MUST 1401 support the server side of the CDNI Logging feed and the server side 1402 of the CDNI Logging pull mechanism. 1404 An implementation of the CDNI Logging interface as per the present 1405 document consuming CDNI Logging file (i.e. on the uCDN side) MUST 1406 support the client side of the CDNI Logging feed and the client side 1407 of the CDNI Logging pull mechanism. 1409 We note that implementations of the CDNI Logging interface MAY also 1410 support other mechanisms to exchange CDNI Logging Files, for example 1411 in view of exchanging logging information with minimum time-lag (e.g. 1412 sub-minute or sub-second) between when the event occurred in the dCDN 1413 and when the corresponding Logging Record is made available to the 1414 uCDN (e.g. for log-consuming applications requiring extremely fresh 1415 logging information such as near-real-time content delivery 1416 monitoring). Such mechanisms are outside the scope of the present 1417 document but might be defined in future version of this document . 1419 4.1. CDNI Logging Feed 1420 The server-side implementation of the CDNI Logging feed MUST produce 1421 an Atom feed [RFC4287]. This feed is used to advertise log files 1422 that are available for the client-side to retrieve using the CDNI 1423 Logging pull mechanism. 1425 4.1.1. Atom Formatting 1427 A CDNI Logging feed MUST be structured as an Archived feed, as 1428 defined in [RFC5005], and MUST be formatted in Atom [RFC4287]. This 1429 means it consists of a subscription document that is regularly 1430 updated as new CDNI logging files become available, and information 1431 about older CDNI Logging files is moved into archive documents. Once 1432 created, archive documents are never modified. 1434 Each CDNI Logging file listed in an Atom feed MUST be described in an 1435 atom:entry container element. 1437 The atom:entry MUST contain an atom:content element whose "src" 1438 attribute is a link to the CDNI Logging file and whose "type" 1439 attribute is the MIME Media Type indicating that the entry is a CDNI 1440 Logging File. We define this MIME Media Type as "application/ 1441 cdni.LoggingFile" (See Section 5.4). 1443 For compatibility with some Atom feed readers the atom:entry MAY also 1444 contain an atom:link entry whose "href" attribute is a link to the 1445 CDNI Logging file and whose "type" attribute is the MIME Media Type 1446 indicating that the entry is a CDNI Logging File using the 1447 "application/cdni.LoggingFile" MIME Media Type (See Section 5.4). 1449 The IRI used in the atom:id of the atom:entry MUST contain the UUID 1450 of the CDNI Logging file. 1452 The atom:updated in the atom:entry MUST indicate the time at which 1453 the CDNI Logging file was last updated. 1455 4.1.2. Updates to Log Files and the Feed 1457 CDNI Logging files MUST NOT be modified by the dCDN once published in 1458 the CDNI Logging feed. 1460 The frequency with which the subscription feed is updated, the period 1461 of time covered by each CDNI Logging file or each archive document, 1462 and timeliness of publishing of CDNI Logging files are outside the 1463 scope of the present document and are expected to be agreed upon by 1464 uCDN and dCDN via other means (e.g. human agreement). 1466 The server-side implementation SHOULD use HTTP cache control headers 1467 on the subscription feed to indicate the frequency at which the 1468 client-side is to poll for updates. 1470 The potential retention limits (e.g. sliding time window) within 1471 which the dCDN is to retain and be ready to serve an archive document 1472 is outside the scope of the present document and is expected to be 1473 agreed upon by uCDN and dCDN via other means (e.g. human agreement). 1474 The server-side implementation MUST retain, and be ready to serve, 1475 any archive document within the agreed retention limits. Outside 1476 these agreed limits, the server-side implementation MAY be unable to 1477 serve (e.g., with HTTP status code 404) an archive document or MAY 1478 refuse to serve it (e.g., with HTTP status code 403 or 410). 1480 4.1.3. Redundant Feeds 1482 The server-side implementation MAY present more than one CDNI Logging 1483 feed and for redundancy, CDNI Logging files MAY be published in more 1484 than one feed. 1486 A client-side implementation MAY support such redundant CDNI Logging 1487 feeds. If it supports redundant CDNI Logging feed, the client-side 1488 SHOULD use the UUID of the CDNI Logging file, presented in the 1489 atom:id element of the Atom feed, to avoid unnecessarily pulling and 1490 storing each CDNI Logging file more than once. 1492 4.1.4. Example CDNI Logging Feed 1494 Figure 4 illustrates an example of the subscription document of a 1495 CDNI Logging feed. 1497 1498 > 1500 CDNI Logging Feed 1501 2013-03-23T14:46:11Z 1502 urn:uuid:663ae677-40fb-e99a-049d-c5642916b8ce 1503 1505 1507 1509 CDNI Log Feed 1510 Generator 1511 dcdn.example 1512 1513 CDNI Logging File for uCDN at 1514 2013-03-23 14:15:00 1515 urn:uuid:12345678-1234-abcd-00aa-01234567abcd 1516 2013-03-23T14:15:00Z 1517 1520 CDNI Logging File for uCDN at 1521 2013-03-23 14:15:00 1522 1523 1524 CDNI Logging File for uCDN at 1525 2013-03-23 14:30:00 1526 urn:uuid:87654321-4321-dcba-aa00-dcba7654321 1527 2013-03-23T14:30:00Z 1528 1531 CDNI Logging File for uCDN at 1532 2013-03-23 15:30:00 1533 1534 ... 1535 1536 ... 1537 1538 1540 Figure 4: Example subscription document of a CDNI Logging Feed 1542 4.2. CDNI Logging File Pull 1544 A client-side implementation of the CDNI Logging interface MAY pull, 1545 at its convenience, a CDNI Logging File that is published by the 1546 server-side in the CDNI Logging Feed (in the subscription document or 1547 an archive document). To do so, the client-side: 1549 o MUST use HTTP v1.1 ( [RFC2616]); 1551 o SHOULD use TLS (i.e. use what is loosely referred to as "HTTPS") 1552 as per [RFC2818] whenever protection of the CDNI Logging 1553 information is required (see Section 6.1); 1555 o MUST use the URI that was associated to the CDNI Logging File 1556 (within the "src" attribute of the corresponding atom:content 1557 element) in the CDNI Logging Feed 1559 o MUST support exchange of CDNI Logging Files with no content 1560 encoding applied to the representation; 1562 o SHOULD support exchange of CDNI Logging Files with "gzip" content 1563 encoding (as defined in [RFC2616]) applied to the representation. 1565 Note that a client-side implementation of the CDNI Logging interface 1566 MAY pull a CDNI Logging File that it has already pulled. 1568 The server-side implementation MUST respond to valid pull request by 1569 a client-side implementation for a CDNI Logging File published by the 1570 server-side in the CDNI Logging Feed (in the subscription document or 1571 an archive document). The server-side implementation: 1573 o MUST handle the client-side request as per HTTP v1.1; 1575 o MUST include the CDNI Logging File identified by the request URI 1576 inside the body of the HTTP response; 1578 o MUST support exchange of CDNI Logging Files with no content 1579 encoding applied to the representation; 1581 o SHOULD support exchange of CDNI Logging Files with "gzip" content 1582 encoding (as defined in [RFC2616]) applied to the representation. 1584 Content negotiation approaches defined in [RFC2616] (e.g. using 1585 Accept-Encoding request-header field or Content-Encoding entity- 1586 header field) MAY be used by the client-side and server-side 1587 implementations to establish the content-coding to be used for a 1588 particular exchange of a CDNI Logging File. 1590 Applying compression content encoding (such as "gzip") is expected to 1591 mitigate the impact of exchanging the large volumes of logging 1592 information expected across CDNs. This is expected to be 1593 particularly useful in the presence of HTTP Adaptive Streaming (HAS) 1594 which, as per the present version of the document, will result in a 1595 separate CDNI Log Record for each HAS segment delivery in the CDNI 1596 Logging File. 1598 The potential retention limits (e.g. sliding time window, maximum 1599 aggregate file storage quotas) within which the dCDN is to retain and 1600 be ready to serve a CDNI Logging File previously advertised in the 1601 CDNI Logging Feed is outside the scope of the present document and is 1602 expected to be agreed upon by uCDN and dCDN via other means (e.g. 1603 human agreement). The server-side implementation MUST retain, and be 1604 ready to serve, any CDNI Logging File within the agreed retention 1605 limits. Outside these agreed limits, the server-side implementation 1606 MAY be unable to serve (e.g., with HTTP status code 404) a CDNI 1607 Logging File or MAY refuse to serve it (e.g., with HTTP status code 1608 403 or 410). 1610 5. IANA Considerations 1612 5.1. CDNI Logging Directive Names Registry 1614 The IANA is requested to create a new registry, CDNI Logging 1615 Directive Names. 1617 The initial contents of the CDNI Logging File Directives registry 1618 comprise the names of the directives specified in Section 3.3 of the 1619 present document, and are as follows: 1621 +------------------------------+-----------+ 1622 | Directive Name + Reference | 1623 +------------------------------+-----------+ 1624 | Version + RFC xxxx | 1625 | UUID + RFC xxxx | 1626 | Claimed-Origin + RFC xxxx | 1627 | Verified-Origin + RFC xxxx | 1628 | Record-Type + RFC xxxx | 1629 | Fields + RFC xxxx | 1630 | Integrity-Hash + RFC xxxx | 1631 +------------------------------+-----------+ 1633 Figure 5 1635 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 1636 the present document] 1638 Within the registry, names are to be allocated by IANA according to 1639 the "Specification Required" policy specified in [RFC5226]. 1641 5.2. CDNI Logging Record-Types Registry 1643 The IANA is requested to create a new registry, CDNI Logging Record- 1644 Types. 1646 The initial contents of the CDNI Logging Record-Types registry 1647 comprise the names of the CDNI Logging Record types specified in 1648 Section 3.4 of the present document, and are as follows: 1650 +------------------------------+-----------+ 1651 | Record-Types + Reference | 1652 +------------------------------+-----------+ 1653 | cdni_http_request_v1 + RFC xxxx | 1654 +------------------------------+-----------+ 1656 Figure 6 1658 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 1659 the present document] 1661 Within the registry, Record-Types are to be allocated by IANA 1662 according to the "Specification Required" policy specified in 1663 [RFC5226]. 1665 5.3. CDNI Logging Field Names Registry 1667 The IANA is requested to create a new registry, CDNI Logging Field 1668 Names. 1670 The initial contents of the CDNI Logging Fields Names registry 1671 comprise the names of the CDNI Logging fields specified in 1672 Section 3.4 of the present document, and are as follows: 1674 +---------------------------------------------+-----------+ 1675 | Field Name + Reference | 1676 +---------------------------------------------+-----------+ 1677 | date + RFC xxxx | 1678 | time + RFC xxxx | 1679 | time-taken + RFC xxxx | 1680 | c-ip + RFC xxxx | 1681 | c-ip-anonimizing + RFC xxxx | 1682 | c-port + RFC xxxx | 1683 | s-ip + RFC xxxx | 1684 | s-hostname + RFC xxxx | 1685 | s-port + RFC xxxx | 1686 | cs- method + RFC xxxx | 1687 | cs-uri + RFC xxxx | 1688 | u-uri + RFC xxxx | 1689 | protocol + RFC xxxx | 1690 | sc-status + RFC xxxx | 1691 | sc- total-bytes + RFC xxxx | 1692 | sc-entity-bytes + RFC xxxx | 1693 | cs() + RFC xxxx | 1694 | sc() + RFC xxxx | 1695 | s-ccid + RFC xxxx | 1696 | s-sid + RFC xxxx | 1697 | s-cached + RFC xxxx | 1698 +---------------------------------------------+-----------+ 1700 Figure 7 1702 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of 1703 the present document] 1704 Within the registry, names are to be allocated by IANA according to 1705 the "Specification Required" policy specified in [RFC5226]. 1707 5.4. CDNI Logging MIME Media Type 1709 The IANA is requested to allocate the "application/cdni.LoggingFile" 1710 MIME Media Type (whose use is specified in Section 4.1.1 of the 1711 present document) in the MIME Media Types registry. 1713 6. Security Considerations 1715 6.1. Authentication, Confidentiality, Integrity Protection 1717 The use of TLS as per [RFC2818] for transport of the CDNI Logging 1718 feed mechanism (Section 4.1) and CDNI Logging File pull mechanism 1719 (Section 4.2) allows: 1721 o the dCDN and uCDN to authenticate each other (to ensure they are 1722 transmitting/receiving CDNI Logging File from an authenticated 1723 CDN) 1725 o the CDNI Logging information to be transmitted with 1726 confidentiality 1728 o the integrity of the CDNI Logging information to be protected 1729 during the exchange 1731 In an environment where any such protection is required, TLS SHOULD 1732 be used for transport of the CDNI Logging feed and the CDNI Logging 1733 File pull. 1735 A CDNI Logging implementation MUST support TLS transport of the CDNI 1736 Logging feed and the CDNI Logging File pull. 1738 The Integrity-Hash directive inside the CDNI Logging File provides 1739 additional integrity protection, this time targeting potential 1740 corruption of the CDNI logging information during the CDNI Logging 1741 File generation. This mechanism does not allow restoration of the 1742 corrupted CDNI Logging information, but it allows detection of such 1743 corruption and therefore triggering of appropraite correcting actions 1744 (e.g. discard of corrupted information, attempt to re-obtain the CDNI 1745 Logging information). 1747 6.2. Denial of Service 1749 This document does not define specific mechanism to protect against 1750 Denial of Service (DoS) attacks on the Logging Interface. However, 1751 the CDNI Logging feed and CDNI Logging pull endpoints can be 1752 protected against DoS attacks through the use of TLS transport and/or 1753 via mechanisms outside the scope of the CDNI Logging interface such 1754 as firewalling or use of Virtual Private Networks (VPNs). 1756 Protection of dCDN Surrogates against spoofed delivery requests is 1757 outside the scope of the CDNI Logging interface. 1759 6.3. Privacy 1761 CDNs have the opportunity to collect detailed information about the 1762 downloads performed by End-Users. The provision of this information 1763 to another CDN introduces potential End-Users privacy protection 1764 concerns. We observe that when CDNI interconnection is realised as 1765 per [I-D.ietf-cdni-framework], the uCDN handles the initial End-User 1766 requests (before it is redirected to the dCDN) so, regardless of 1767 which information is, or is not, communicated to the uCDN through the 1768 CDNI Logging interface, the uCDN has visibility on significant 1769 information such as the IP address of the End-User request and the 1770 URL of the request. Nonetheless, if the dCDN and uCDN agree that 1771 anonymization is required to avoid making some detailed information 1772 available to the uCDN (such as how much bytes of the content has been 1773 watched by an enduser and/or at what time) or is required to meet 1774 some legal obligations, then the uCDN and dCDN can agree to exchange 1775 anonymized End-User IP addresses in CDNI Logging files and the c-ip- 1776 anonymization field can be used to convey the number of bits that 1777 have been anonymized so that the meaningful information can still be 1778 easily extracted from the anonymized addressses (e.g. for geolocation 1779 aware analytics). 1781 7. Acknowledgments 1783 This document borrows from the W3C Extended Log Format [ELF]. 1785 Rob Murray significantly contributed into the text of Section 4.1 . 1787 The authors thank Ben Niven-Jenkins, Kevin Ma, David Mandelberg and 1788 Ray van Brandenburg for their ongoing input. 1790 Finally, we also thank Sebastien Cubaud, Pawel Grochocki, Christian 1791 Jacquenet, Yannick Le Louedec, Anne Marrec , Emile Stephan, Fabio 1792 Costa, Sara Oueslati, Yvan Massot, Renaud Edel, Joel Favier and the 1793 contributors of the EU FP7 OCEAN project for their input in the early 1794 versions of this document. 1796 8. References 1798 8.1. Normative References 1800 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1801 Requirement Levels", BCP 14, RFC 2119, March 1997. 1803 [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., 1804 Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext 1805 Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. 1807 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 1808 Resource Identifier (URI): Generic Syntax", STD 66, RFC 1809 3986, January 2005. 1811 [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally 1812 Unique IDentifier (UUID) URN Namespace", RFC 4122, July 1813 2005. 1815 [RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom 1816 Syndication Format", RFC 4287, December 2005. 1818 [RFC5005] Nottingham, M., "Feed Paging and Archiving", RFC 5005, 1819 September 2007. 1821 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 1822 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 1823 May 2008. 1825 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 1826 Specifications: ABNF", STD 68, RFC 5234, January 2008. 1828 8.2. Informative References 1830 [CHAR_SET] 1831 , "IANA Character Sets registry", , . 1834 [ELF] Phillip M. Hallam-Baker, . and . Brian Behlendorf, 1835 "Extended Log File Format, W3C (work in progress), WD- 1836 logfile-960323", , . 1838 [I-D.ietf-cdni-framework] 1839 Peterson, L. and B. Davie, "Framework for CDN 1840 Interconnection", draft-ietf-cdni-framework-05 (work in 1841 progress), September 2013. 1843 [I-D.ietf-cdni-metadata] 1844 Niven-Jenkins, B., Murray, R., Watson, G., Caulfield, M., 1845 Leung, K., and K. Ma, "CDN Interconnect Metadata", draft- 1846 ietf-cdni-metadata-02 (work in progress), July 2013. 1848 [I-D.ietf-cdni-requirements] 1849 Leung, K. and Y. Lee, "Content Distribution Network 1850 Interconnection (CDNI) Requirements", draft-ietf-cdni- 1851 requirements-10 (work in progress), September 2013. 1853 [I-D.snell-atompub-link-extensions] 1854 Snell, J., "Atom Link Extensions", draft-snell-atompub- 1855 link-extensions-09 (work in progress), June 2012. 1857 [RFC1321] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321, 1858 April 1992. 1860 [RFC2818] Rescorla, E., "HTTP Over TLS", RFC 2818, May 2000. 1862 [RFC6707] Niven-Jenkins, B., Le Faucheur, F., and N. Bitar, "Content 1863 Distribution Network Interconnection (CDNI) Problem 1864 Statement", RFC 6707, September 2012. 1866 [RFC6770] Bertrand, G., Stephan, E., Burbridge, T., Eardley, P., Ma, 1867 K., and G. Watson, "Use Cases for Content Delivery Network 1868 Interconnection", RFC 6770, November 2012. 1870 [RFC6983] van Brandenburg, R., van Deventer, O., Le Faucheur, F., 1871 and K. Leung, "Models for HTTP-Adaptive-Streaming-Aware 1872 Content Distribution Network Interconnection (CDNI)", RFC 1873 6983, July 2013. 1875 Appendix A. Compliance with CDNI Requirements 1877 [Editor's Note: This appendix is intended to help the WG understand 1878 compliance of the CDNI Logging interface against the requirements 1879 defined in the CDNI requirements document, in oder to establish 1880 readiness for of the document publication. This appendix is expected 1881 to be removed for bepublication]. 1883 [Editor's Note: this appendix may need a small update if ietf-cdni- 1884 requirements introduces an additional requirement for Privacy/ 1885 Anonimization as recently discussed on the list, and if LI14 & LI-15 1886 are modified] 1888 The three tables below review compliance against, respectively, the 1889 Generic CDNI requirements, the CDNI Logging interface requirements 1890 and the CDNI security requirements of [I-D.ietf-cdni-requirements]. 1891 The first two columns of the tables indicate the requirement number, 1892 and the requirement priority as defined in 1893 [I-D.ietf-cdni-requirements]. The third column of the table 1894 indicates the level of compliance of the CDNI Logging interface 1895 specified in the present document against the given requirement, and 1896 the fourth column provides additional comment and explanation on how 1897 or why the compliance is achieved or not achieved. 1899 +-------+-------+-----------+---------------------------------------+ 1900 | Re- | Prior-| Compli- | Comment | 1901 | quire-| ity | ance | | 1902 | ment | | | | 1903 +-------+-------+-----------+---------------------------------------+ 1904 | GEN-1 | MED | Full | Leverages existing protocols incl | 1905 | | | | including HTTP, TLS and ATOM | 1906 +-------+-------+-----------+---------------------------------------+ 1907 | GEN-2 | HIGH | Full | Does not require any change or upgrade| 1908 | | | | to the user agent | 1909 +-------+-------+-----------+---------------------------------------+ 1910 | GEN-3 | HIGH | Full | Does not require any change or upgrade| 1911 | | | | to the Content Service Provider | 1912 +-------+-------+-----------+---------------------------------------+ 1913 | GEN-4 | HIGH | Full | Does not depend on intra-CDN info | 1914 +-------+-------+-----------+---------------------------------------+ 1915 | GEN-5 | HIGH | Full | Supports logging of HTTP delivery | 1916 +-------+-------+-----------+---------------------------------------+ 1917 | GEN-6 | HIGH | N/A | | 1918 +-------+-------+-----------+---------------------------------------+ 1919 | GEN-7 | LOW | Not | Only supports logging for HTTP | 1920 | | | Compliant | delivery, but easily extensible to | 1921 | | | | add support for other delivery protos | 1922 +-------+-------+-----------+---------------------------------------+ 1923 | GEN-8 | LOW | N/A | | 1924 +-------+-------+-----------+---------------------------------------+ 1925 | GEN-9 | MED | Full | Supports logging across cascaded CDNs | 1926 +-------+-------+-----------+---------------------------------------+ 1927 | GEN-10| MED | Full | Supports any toplogy of interconnected| 1928 | | | | CDNs | 1929 +-------+-------+-----------+---------------------------------------+ 1930 | GEN-11| HIGH | Partial | No explicit mechanism for loop | 1931 | | | | avoidance is defined; the exchange of | 1932 | | | | logs is usually done in a point to | 1933 | | | | point manner between two well identi- | 1934 | | | | fied entities situated in the uCDN and| 1935 | | | | dCDN. Loop avoidance is expected to be| 1936 | | | | handled by implementations based on | 1937 | | | | inferring the CDN path from the URI | 1938 | | | | structure in the HTTP redirection case| 1939 | | | | and/or administrative information | 1940 | | | | (topology restrictions in case of DNS | 1941 | | | | redirection method also handled admi- | 1942 | | | | nistratively) | 1943 +-------+-------+-----------+---------------------------------------+ 1944 | GEN-12| HIGH | N/A | | 1945 +-------+-------+-----------+---------------------------------------+ 1946 | GEN-13| HIGH | Full | Supports Logging for HTTP Adaptive | 1947 | | | | Streaming (HSAS) content, with one | 1948 | | | | Logging Record per HAS segment. | 1949 | | | | Supports a few optional logging fields| 1950 | | | | specific to HAS. Does not support | 1951 | | | | summarized Logging Records for HAS, | 1952 | | | | but extensible to add that. | 1953 +-------+-------+-----------+---------------------------------------+ 1955 Figure 8: Compliance to Generic CDNI Requirements 1957 +-------+-------+-----------+---------------------------------------+ 1958 | Re- | Prior-| Compli- | Comment | 1959 | quire-| ity | ance | | 1960 | ment | | | | 1961 +-------+-------+-----------+---------------------------------------+ 1962 | LI-1 | HIGH | Full | Reliable transfer is achieved by the | 1963 | | | | transport protocol: the logging data | 1964 | | | | is transmitted over HTTP over TCP. | 1965 | | | | Also, supports optional redundancy of | 1966 | | | | the Logging feed. | 1967 +-------+-------+-----------+---------------------------------------+ 1968 | LI-2 | HIGH | Full | Supports | 1969 | | | | logs for all content deliveries both | 1970 | | | | complete and incomplete performed by | 1971 | | | | the dCDN on behalf of the uCDN | 1972 +-------+-------+-----------+---------------------------------------+ 1973 | LI-3 | MED | Full | The CDNI Logging Interface does not | 1974 | | | | impose any restrictions related to the| 1975 | | | | transmission of logs generated by | 1976 | | | | intermediary CDNs; the dCDN formats | 1977 | | | | internally all the final logging files| 1978 | | | | including those received from interme-| 1979 | | | | diary CDNs and the locally generated | 1980 +-------+-------+-----------+---------------------------------------+ 1981 | LI-4 | HIGH | Full | The ATOM feed allows the uCDN to trig-| 1982 | | | | ger the download of logging files | 1983 | | | | whenever needed | 1984 +-------+-------+-----------+---------------------------------------+ 1985 | LI-5 | MED | Partial | The uCDN can pull logging files from | 1986 | | | | the dCDN whenever a new file is | 1987 | | | | available. The timing constraints for | 1988 | | | | the generation of the logging files | 1989 | | | | are to be defined offline, and can be | 1990 | | | | defined to an arbitrary period. This | 1991 | | | | is expected to be compatible with | 1992 | | | | applications that have low timing | 1993 | | | | constraints (e.g. 24 hours) such as | 1994 | | | | billing. This is expected to be | 1995 | | | | compatible with applications that | 1996 | | | | have high timing constraints (e.g. 5 | 1997 | | | | minutes) such as monitoring or | 1998 | | | | analytics. This is not expected to be | 1999 | | | | compatible with applications that have| 2000 | | | | very high timing constraints (e.g. | 2001 | | | | a few seconds or below) | 2002 +-------+-------+-----------+---------------------------------------+ 2003 | LI-6 | HIGH | Full | Section 3.4 describes the CDNI Logging| 2004 | | | | Records and the possible fields that | 2005 | | | | can be included in a record. | 2006 | | | | Supports a single type of CDNI event | 2007 | | | | i.e. HTTP delivery | 2008 +-------+-------+-----------+---------------------------------------+ 2009 | LI-7 | HIGH | Full | Defines an ATOM based feed and HTTP | 2010 | | | | or HTTPS transport | 2011 +-------+-------+-----------+---------------------------------------+ 2012 | LI-8 | MED | Partial | Allows as uCDN to pull current CDNI | 2013 | | | | Logging files to access current | 2014 | | | | Logging records. Does not allow uCDN | 2015 | | | | to request Log Records before next | 2016 | | | | Logging file is made available. | 2017 +-------+-------+-----------+---------------------------------------+ 2018 | LI-9 | LOW | Not | The current version of the document | 2019 | | | Compliant | does not specify any mechanisms for | 2020 | | | | producing aggregate / summarized logs,| 2021 | | | | but exchanged logging files provide | 2022 | | | | all the information that is necessary | 2023 | | | | to the uCDN in order to produce aggre-| 2024 | | | | gated logs. Extensible to add such | 2025 | | | | mechanisms in the future | 2026 +-------+-------+-----------+---------------------------------------+ 2027 | LI-10 | LOW | Not | Future versions might define such a | 2028 | | | compliant | mechanism for logging performance | 2029 | | | | data. Allows uCDN to derive some perf | 2030 | | | | indicators from delivery Records | 2031 +-------+-------+-----------+---------------------------------------+ 2032 | LI-11 | MED | Not | Future versions might define such a | 2033 | | | compliant | mechanism for logging data about | 2034 | | | | resources consumed by the dCDN | 2035 +-------+-------+-----------+---------------------------------------+ 2036 | LI-12 | MED | Not | Future versions might define such a | 2037 | | | compliant | mechanism for logging data about | 2038 | | | | resources consumed by cascaded CDNs | 2039 +-------+-------+-----------+---------------------------------------+ 2040 | LI-13 | HIGH | Not | Not supported by CDNI Logging | 2041 | | | compliant | interface. However, it is expected | 2042 | | | | that the CDNI Control interface will | 2043 | | | | allow tracing of delete request | 2044 | | | | results (e.g. success, failure). | 2045 +-------+-------+-----------+---------------------------------------+ 2046 | LI-14 | HIGH | Full | Details about extensibility mechanisms| 2047 | | | | in Section 6. | 2048 +-------+-------+-----------+---------------------------------------+ 2049 | LI-15 | HIGH | Full | Details about proprietary fields in | 2050 | | | | Section 6. | 2051 +-------+-------+-----------+---------------------------------------+ 2052 | LI-16 | HIGH | Full | The CDNI Logging feed indicates which | 2053 | | | | Logging file is (or was) available | 2054 +-------+-------+-----------+---------------------------------------+ 2055 | LI-17 | MED | Full | Content Collection ID and Session ID | 2056 | | | | are supported for logging records re- | 2057 | | | | lated to HTTP Adaptive Streaming | 2058 +-------+-------+-----------+---------------------------------------+ 2060 Figure 9: Compliance to CDNI Logging interface Requirements 2062 +-------+-------+-----------+---------------------------------------+ 2063 | Re- | Prior-| Compli- | Comment | 2064 | quire-| ity | ance | | 2065 | ment | | | | 2066 +-------+-------+-----------+---------------------------------------+ 2067 | SEC-1 | HIGH | Full | TLS can be used for transport of any | 2068 | | | | CDNI logging related information which| 2069 | | | | provides authentication, confidentia- | 2070 | | | | lity, integrity protection as well as | 2071 | | | | protection agasint spoofing and replay| 2072 +-------+-------+-----------+---------------------------------------+ 2073 | SEC-2 | HIGH | Partial | No specific mechanism against Denial | 2074 | | | | of Service attacks is defined on the | 2075 | | | | Logging Interface. Spoofed requests | 2076 | | | | can be avoided by using TLS. | 2077 | | | | Protection against spoofed delivery | 2078 | | | | requests are outside the scope of CDNI| 2079 | | | | Logging. | 2080 +-------+-------+-----------+---------------------------------------+ 2081 | SEC-3 | MED | N/A | Establishing CDN path with non- | 2082 | | | | repudiation is outside the scope of | 2083 | | | | CDNI Logging. Does not prevent use of | 2084 | | | | such mechanism (e.g. including info | 2085 | | | | in content URI). | 2086 +-------+-------+-----------+---------------------------------------+ 2087 | SEC-4 | MED | Not | A non-repudiation mechanism for CDNI | 2088 | | | compliant | logging might be defined in a separate| 2089 | | | | document | 2090 +-------+-------+-----------+---------------------------------------+ 2091 | SEC-5 | LOW | N/A | | 2092 +-------+-------+-----------+---------------------------------------+ 2094 Figure 10: Compliance to CDNI Security Requirements 2096 Authors' Addresses 2098 Francois Le Faucheur (editor) 2099 Cisco Systems 2100 E.Space Park - Batiment D 2101 6254 Allee des Ormes - BP 1200 2102 Mougins cedex 06254 2103 FR 2105 Phone: +33 4 97 23 26 19 2106 Email: flefauch@cisco.com 2108 Gilles Bertrand (editor) 2109 Orange 2110 38-40 rue du General Leclerc 2111 Issy les Moulineaux 92130 2112 FR 2114 Phone: +33 1 45 29 89 46 2115 Email: gilles.bertrand@orange.com 2117 Iuniana Oprescu (editor) 2118 Orange 2119 38-40 rue du General Leclerc 2120 Issy les Moulineaux 92130 2121 FR 2123 Phone: +33 6 89 06 92 72 2124 Email: iuniana.oprescu@orange.com 2125 Roy Peterkofsky 2126 Skytide, Inc. 2127 One Kaiser Plaza, Suite 785 2128 Oakland CA 94612 2129 USA 2131 Phone: +01 510 250 4284 2132 Email: roy@skytide.com