idnits 2.17.1 draft-bertrand-cdni-logging-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 1 instance of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to contain a disclaimer for pre-RFC5378 work, but was first submitted on or after 10 November 2008. The disclaimer is usually necessary only for documents that revise or obsolete older RFCs, and that take significant amounts of text from those RFCs. If you can contact all authors of the source material and they are willing to grant the BCP78 rights to the IETF Trust, you can and should remove the disclaimer. Otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (October 22, 2012) is 4175 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'MED' is mentioned on line 1799, but not defined == Unused Reference: 'RFC2119' is defined on line 1519, but no explicit reference was found in the text == Unused Reference: 'I-D.bertrand-cdni-experiments' is defined on line 1534, but no explicit reference was found in the text -- No information found for draft-brandenburg-cdni-has - is the name correct? == Outdated reference: A later version (-14) exists of draft-ietf-cdni-framework-01 == Outdated reference: A later version (-17) exists of draft-ietf-cdni-requirements-03 Summary: 0 errors (**), 0 flaws (~~), 8 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force G. Bertrand, Ed. 3 Internet-Draft E. Stephan 4 Intended status: Informational France Telecom - Orange 5 Expires: April 25, 2013 R. Peterkofsky 6 Skytide, Inc. 7 F. Le Faucheur 8 Cisco Systems 9 P. Grochocki 10 Orange Polska 11 October 22, 2012 13 CDNI Logging Interface 14 draft-bertrand-cdni-logging-02 16 Abstract 18 This memo specifies the Logging interface between a downstream CDN 19 (dCDN) and an upstream CDN (uCDN) that are interconnected as per the 20 CDN Interconnection (CDNI) framework. First, it describes a 21 reference model for CDNI logging. Then, it specifies the actual 22 protocol for CDNI logging information exchange covering the 23 information elements as well as the transport of those. 25 Status of this Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at http://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on April 25, 2013. 42 Copyright Notice 44 Copyright (c) 2012 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 This document may contain material from IETF Documents or IETF 58 Contributions published or made publicly available before November 59 10, 2008. The person(s) controlling the copyright in some of this 60 material may not have granted the IETF Trust the right to allow 61 modifications of such material outside the IETF Standards Process. 62 Without obtaining an adequate license from the person(s) controlling 63 the copyright in such materials, this document may not be modified 64 outside the IETF Standards Process, and derivative works of it may 65 not be created outside the IETF Standards Process, except to format 66 it for publication as an RFC or to translate it into languages other 67 than English. 69 Table of Contents 71 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 72 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 6 73 1.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 8 74 2. CDNI Logging Reference Model . . . . . . . . . . . . . . . . . 8 75 2.1. CDNI Logging interactions . . . . . . . . . . . . . . . . 8 76 2.2. Overall Logging Chain . . . . . . . . . . . . . . . . . . 13 77 2.2.1. Logging Generation and During-Generation 78 Aggregation . . . . . . . . . . . . . . . . . . . . . 15 79 2.2.2. Logging Collection . . . . . . . . . . . . . . . . . . 15 80 2.2.3. Logging Filtering . . . . . . . . . . . . . . . . . . 16 81 2.2.4. Logging Rectification and Post-Generation 82 Aggregation . . . . . . . . . . . . . . . . . . . . . 16 83 2.2.5. Log-Consuming Applications . . . . . . . . . . . . . . 17 84 2.2.5.1. Maintenance/Debugging . . . . . . . . . . . . . . 17 85 2.2.5.2. Accounting . . . . . . . . . . . . . . . . . . . . 17 86 2.2.5.3. Analytics and Reporting . . . . . . . . . . . . . 18 87 2.2.5.4. Security . . . . . . . . . . . . . . . . . . . . . 18 88 2.2.5.5. Legal Logging Duties . . . . . . . . . . . . . . . 18 89 2.2.5.6. Notions common to multiple Log Consuming 90 Applications . . . . . . . . . . . . . . . . . . . 18 91 3. CDNI Logging Information Structure and Transport . . . . . . . 20 92 4. CDNI Logging Fields . . . . . . . . . . . . . . . . . . . . . 22 93 4.1. Generic Fields . . . . . . . . . . . . . . . . . . . . . . 22 94 4.1.1. Semantics of Generic CDNI Logging Fields . . . . . . . 22 95 4.1.2. Syntax of Generic CDNI Logging Fields . . . . . . . . 24 96 4.2. Logging Fields for Content Delivery . . . . . . . . . . . 25 97 4.2.1. Semantics for Delivery CDNI Logging Fields . . . . . . 25 98 4.2.2. Syntax for Delivery CDNI Logging Fields . . . . . . . 26 99 4.3. Logging Fields for Content Acquisition . . . . . . . . . . 26 100 4.3.1. Semantics for Acquisition CDNI Logging Fields . . . . 27 101 4.3.2. Syntax for Acquisition CDNI Logging Fields . . . . . . 27 102 4.4. Logging Fields for Control . . . . . . . . . . . . . . . . 27 103 4.5. Logging Fields for Other Operations . . . . . . . . . . . 27 104 5. CDNI Logging Records . . . . . . . . . . . . . . . . . . . . . 28 105 5.1. Content Delivery . . . . . . . . . . . . . . . . . . . . . 28 106 5.2. Content Acquisition . . . . . . . . . . . . . . . . . . . 29 107 5.2.1. Logging Records Provided by dCDN to uCDN . . . . . . . 29 108 5.2.2. Logging Records Provided by uCDN to dCDN . . . . . . . 29 109 5.3. Content Invalidation and Purging . . . . . . . . . . . . . 30 110 5.4. Logging Extensibility . . . . . . . . . . . . . . . . . . 30 111 6. CDNI Logging File Format . . . . . . . . . . . . . . . . . . . 30 112 6.1. Logging Files . . . . . . . . . . . . . . . . . . . . . . 31 113 6.2. File Format . . . . . . . . . . . . . . . . . . . . . . . 31 114 6.2.1. Headers . . . . . . . . . . . . . . . . . . . . . . . 31 115 6.2.2. Body (Logging Records) Format . . . . . . . . . . . . 32 116 6.2.3. Footer Format . . . . . . . . . . . . . . . . . . . . 33 118 7. CDNI Logging File Transport Protocol . . . . . . . . . . . . . 33 119 8. Logging Control . . . . . . . . . . . . . . . . . . . . . . . 33 120 9. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 34 121 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 35 122 11. Security Considerations . . . . . . . . . . . . . . . . . . . 35 123 11.1. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 35 124 11.2. Non Repudiation . . . . . . . . . . . . . . . . . . . . . 35 125 12. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 35 126 13. References . . . . . . . . . . . . . . . . . . . . . . . . . . 35 127 13.1. Normative References . . . . . . . . . . . . . . . . . . . 35 128 13.2. Informative References . . . . . . . . . . . . . . . . . . 36 129 Appendix A. Examples Log Format . . . . . . . . . . . . . . . . . 37 130 A.1. W3C Common Log File (CLF) Format . . . . . . . . . . . . . 37 131 A.2. W3C Extended Log File (ELF) Format . . . . . . . . . . . . 38 132 A.3. National Center for Supercomputing Applications (NCSA) 133 Common Log Format . . . . . . . . . . . . . . . . . . . . 39 134 A.4. NCSA Combined Log Format . . . . . . . . . . . . . . . . . 39 135 A.5. NCSA Separate Log Format . . . . . . . . . . . . . . . . . 39 136 A.6. Squid 2.0 Native Log Format for Access Logs . . . . . . . 40 137 Appendix B. Requirements . . . . . . . . . . . . . . . . . . . . 41 138 B.1. Additional Requirements . . . . . . . . . . . . . . . . . 41 139 B.2. Compliancy with Requirements draft . . . . . . . . . . . . 42 140 Appendix C. CDNI WG's position on candidate protocols for 141 Logging Transport . . . . . . . . . . . . . . . . . . 42 142 C.1. CDNI WG's position on Syslog . . . . . . . . . . . . . . . 42 143 C.2. CDNI WG's position on SNMP . . . . . . . . . . . . . . . . 42 144 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 42 146 1. Introduction 148 This memo specifies the Logging interface between a downstream CDN 149 (dCDN) and an upstream CDN (uCDN). First, it describes a reference 150 model for CDNI logging. Then, it specifies the actual protocol for 151 CDNI logging information exchange covering the information elements 152 as well as the transport of those. 154 The reader should be familiar with the work of the CDNI WG: 156 o CDNI problem statement [RFC6707] and framework 157 [I-D.ietf-cdni-framework] identify a Logging interface, 159 o Section 7 of [I-D.ietf-cdni-requirements] specifies a set of 160 requirements for Logging, 162 o [I-D.ietf-cdni-use-cases] outlines real world use-cases for 163 interconnecting CDNs. These use cases require the exchange of 164 Logging information between the dCDN and the uCDN. 166 As stated in [RFC6707], "the CDNI Logging interface enables details 167 of logs or events to be exchanged between interconnected CDNs". 169 The present document describes: 171 o The CDNI Logging reference model (Section 2), 173 o The CDNI Logging information structure and Transport (Section 3), 175 o The CDNI Logging Fields (Section 4), 177 o The CDNI Logging Records (Section 5), 179 o The CDNI Logging File format (Section 6), 181 o The CDNI Logging File Transport Protocol (Section 7), 183 o and, finally, the description of the CDNI Logging Control that is 184 to be supported by the CDNI Control Interface Section 8. 186 In the Appendices, the document provides: 188 o A list of identified requirements (Appendix B.1), which should be 189 considered for inclusion in [I-D.ietf-cdni-requirements], 191 1.1. Terminology 193 In this document, the first letter of each CDNI-specific term is 194 capitalized. We adopt the terminology described in [RFC6707] and 195 [I-D.ietf-cdni-framework], and extend it with the additional terms 196 defined below. 198 For clarity, we use the word "Log" only for referring to internal CDN 199 logs and we use the word "Logging" for any inter-CDN information 200 exchange and processing operations related to the CDNI Logging 201 interface. Log and Logging formats may be different. 203 Log: CDN internal information collection and processing operations. 205 Logging: Inter-CDN information exchange and processing operations. 207 CDNI Logging Field: an atomic element of information that can be 208 included in a CDNI Logging Record. The time an event/task started, 209 the IP address of an End user to whom content was delivered, and the 210 URI of the content delivered are examples of CDNI logging fields. 212 CDNI Logging Record: an information record providing information 213 about a specific event. This comprises a collection of CDNI Logging 214 Fields. 216 Separator Character: a specific character used to enable the parsing 217 of Logging Records. This character separates the Logging Fields that 218 compose a Logging Record. 220 Logging File: a file containing Logging Records and additional 221 information for easing the processing of the Logging Records. 223 CDN Reporting: the process of providing the relevant information that 224 will be used to create a formatted content delivery report provided 225 to the CSP in deferred time. Such information typically includes 226 aggregated data that can cover a large period of time (e.g., from 227 hours to several months). Uses of Reporting include the collection 228 of charging data related to CDN services and the computation of Key 229 Performance Indicators (KPIs). 231 CDN Monitoring: the process of providing content delivery information 232 in real-time. Monitoring typically includes data in real time to 233 provide visibility of the deliveries in progress, for service 234 operation purposes. It presents a view of the global health of the 235 services as well as information on usage and performance, for network 236 services supervision and operation management. In particular, 237 monitoring data can be used to generate alarms. 239 End-User experience management: study of Logging data using 240 statistical analysis to discover, understand, and predict user 241 behavior patterns. 243 Class-of-requests: A Class-of-requests identifies a set of content 244 Requests, related to a specific CSP, received from clients in a given 245 footprint and sharing common properties. These properties include: 247 o Any header, URL parameter, query parameter of an HTTP (or RTMP) 248 content request 250 o Any header, or sub-domain of the FQDN of a DNS lookup request 252 Examples: 254 o Class-of-Requests = all the requests that include the HTTP header 255 "User-Agent: Mozilla/5.0" related to CSP 256 "http://*.cdn.example.com" from AS3215 258 o Class-of-Requests = all the DNS requests from anywhere and related 259 to CSP "cdn*.example.com" 261 Delivery Service: A Delivery Service is defined by a set of Class-of- 262 Requests and a list of parameters that apply to all these Class-of- 263 Requests (logging format, delivery quality/capabilities 264 requirements...) 266 Service Agreement: A service agreement is defined by a uCDN 267 identifier, a dCDN identifier, a set of Delivery Services and a list 268 of parameters that apply to the Service Agreement. 270 Once a Service Agreement is agreed between the administrative 271 entities managing the CDNs to be interconnected, the upstream CDN and 272 the downstream CDN of the CDNI interconnection must be configured 273 according to this agreed Service Agreement. For instance, a given 274 uCDN (uCDN1) may request a given dCDN (dCDN1) to configure one 275 Delivery Service for handling requests for HTTP Adaptive streaming 276 videos delegated by uCDN1 and related to a specific CSP (CSP1) and 277 another one for handling requests for static pictures delegated by 278 uCDN1 and related to CSP1. These Delivery services would belong to 279 the Service Agreement between uCDN1 and dCDN1 for CSP1. In this 280 simple example, uCDN1 may request dCDN1 to include Delivery Service 281 information in its CDNI Logging, to help uCDN1 to provide relevant 282 reports to CSP1. 284 1.2. Abbreviations 286 o API: Application Programming Interface 288 o CCID: Content Collection Identifier 290 o CDN: Content Delivery Network 292 o CDNP: Content Delivery Network Provider 294 o CoDR: Content Delivery Record 296 o CSP: Content Service Provider 298 o DASH: Dynamic Adaptive Streaming over HTTP 300 o dCDN: downstream CDN 302 o FTP: File Transfer Protocol 304 o HAS: HTTP Adaptive Streaming 306 o KPI: Key Performance Indicator 308 o PVR: Personal Video Recorder 310 o SID: Session Identifier 312 o SFTP: SSH File Transfer Protocol 314 o SNMP: Simple Network Management Protocol 316 o uCDN: upstream CDN 318 2. CDNI Logging Reference Model 320 2.1. CDNI Logging interactions 322 The CDNI logging reference model between a given uCDN and a given 323 dCDN involves the following interactions: 325 o control by the uCDN of the logging to be performed by the dCDN 326 (e.g. control of which logging fields are to be communicated to 327 the uCDN for a given task performed by the dCDN, control of which 328 types of events are to be logged). This is supported by the CDNI 329 Control interface. 331 o generation and collection by the dCDN of logging information 332 related to the completion of any task performed by the dCDN on 333 behalf of the uCDN (e.g. delivery of the content to an end user) 334 or related to events happening in the dCDN that are relevant to 335 the uCDN (e.g. failures or unavailability in dCDN). This takes 336 place within the dCDN and does not directly involve CDNI 337 interfaces. 339 o communication by the dCDN to the uCDN of the logging information 340 collected by the dCDN relevant to the uCDN. This is supported by 341 the CDNI Logging interface. For example, the uCDN may use this 342 logging information to charge the CSP, to perform analytics and 343 mornitoring for operational reasons, to provide analytics and 344 monitoring views on its content delivery to the CSP, or to perform 345 troubleshooting. 347 o control by the dCDN of the logging to be performed by the uCDN on 348 behalf of the dCDN. This is supported by the CDNI Control 349 interface. 351 o generation and collection by the uCDN of logging information 352 related to the completion of any task performed by the uCDN on 353 behalf of the dCDN (e.g. serving of content by uCDN to dCDN for 354 acquisition purposes by dCDN) or related to events happening in 355 the uCDN that are relevant to the dCDN. This takes place within 356 the uCDN and does not directly involve CDNI interfaces. 358 o communication by the uCDN to the dCDN of the logging information 359 collected by the uCDN relevant to the dCDN. This is supported by 360 the CDNI Logging interface. For example, the dCDN may use this 361 logging information for security auditing or content acquisition 362 troubleshooting. 364 Figure 1 provides an example of CDNI Logging interactions in a 365 particular scenario where 4 CDNs are involved in the delivery of 366 content from a given CSP: the uCDN has a CDNI interconnection with 367 dCDN1 and dCDN2. In turn, dCDN2 has a CDNI interconnection with 368 dCDN3. uCDN, dCDN1, dCDN2 and dCDN3 deliver content for the CSP. In 369 this example, the CDNI Logging interface enables the uCDN to obtain 370 logging information from all the dCDNs involved in the delivery. In 371 the example, uCDN uses the Logging data: 373 o to analyze the performance of the delivery operated by the dCDNs 374 and to adjust its operations (e.g., request routing) as 375 appropriate 377 o to provide reporting (non-real time) and monitoring (real time) 378 information to CSP. 380 For instance, uCDN merges Logging data, extracts relevant KPIs, and 381 presents a formatted report to CSP, in addition to a bill for the 382 content delivered by uCDN itself or its dCDNs on his behalf. uCDN may 383 also provide Logging data as raw log files to CSP, so that CSP can 384 use its own Logging analysis tools. 386 +-----+ 387 | CSP | 388 +-----+ 389 ^ Reporting and monitoring data 390 * Billing 391 ,--*--. 392 Logging ,-' `-. 393 Data =>( uCDN )<= Logging 394 // `-. _,-' \\ Data 395 || `-'-'-' || 396 ,--v--. ^ ^ ,--v--. 397 ,-' `-. + + ,-' `-. 398 ( dCDN-1 )<+++ +++>( dCDN-2 )<== Logging 399 `-. ,-' Logging `-. _,-' \\ Data 400 `--'--' Control `--'-' || 401 ^ ,--v--. 402 Logging + ,' `-. 403 Control++++>( dCDN-3 ) 404 `. ,-' 405 `--'--' 407 <====> CDNI Logging Interface 408 <++++> CDNI Control Interface 409 ***> outside the scope of CDNI 411 Figure 1: Interactions in CDNI Logging Reference Model 413 A dCDN (e.g. dCDN-2) integrates the relevant logging data obtained 414 from its dCDNs (e.g. dCDN-3) in the logging data that it provides to 415 the uCDN, so that the uCDN ultimately obtains all logging information 416 relevant to a CSP for which it acts as the authoritative CDN. 418 Note that the format of Logging data that a CDN provides over the 419 CDNI interface might be different from the one that the CDN uses 420 internally. In this case, the CDN needs to reformat the Logging data 421 before it provides this data to the other CDN over the CDNI Logging 422 interface. Similarly, a CDN might reformat the Logging data that it 423 receives over the CDNI Logging interafce before injecting it into its 424 log-consuming applications or before providing some of this logging 425 information to the CSP. Such reformatting operations introduce 426 latency in the logging distribution chain and introduce a processing 427 burden. Therefore, there are benefits in specifying CDNI Logging 428 format that are as close as possible from the CDN Log formats 429 commonly used in CDNs today. 431 Figure 2 maps the CDNI Logging interactions discussed above onto the 432 CDNI Reference Model defined in [RFC6707]. 434 -------- 435 / \ 436 | CSP | 437 \ / 438 -------- 439 * 440 * Reporting, Monitoring, 441 * Billing /\ 442 * / \ 443 ---------------------- |CDNI| ---------------------- 444 / Upstream CDN \ | | / Downstream CDN \ 445 | +-------------+ | Control Interface| +-------------+ | 446 | + + | (Logging Control)| | | | 447 |******* Control |<++++++|++++|++++++>| Control *******| 448 |* +------*----*-+ | | | | +-*----*------+ *| 449 |* * * | | | | * * *| 450 |* +------*------+ | Logging Interface| +------*------+ *| 451 |* + + | (Logging Data ) | | | *| 452 |* ***** Logging |<======|====|========>| Logging ***** *| 453 |* * +-*-----------+ | | | | +-----------*-+ * *| 454 |* * * * | | | | * * * *| 455 .....*...+-*---------*-+ | | | | +-*---------*-+...*.*... 456 . |* * *** Req-Routing | | | | | | Req-Routing *** * *| . 457 . |* * * +-------------+.| | | | +-------------+ * * *| . 458 . |* * * . | | | * * *| . 459 . |* * * +-------------+ |. | | | +-------------+ * * *| . 460 . |* * * | Distribution| | . | | | | Distribution| * * *| . 461 . |* * * | | | . \ / | | | * * *| . 462 . |* * * |+---------+ | | . \/ | | +---------+| * * *| . 463 . |* * ***| +---------+| | ....Request......+---------+ |*** * *| . 464 . |* *****+-|Surrogate|************************|Surrogate|-+***** *| . 465 . |******* +---------+| | Acquisition | |+----------+ *******| . 466 . | +-------------+ | | +-------*-----+ | . 467 . \ / \ * / . 468 . ---------------------- ---------*------------ . 469 . * . 470 . * Delivery . 471 . * . 472 . +--*---+ . 473 ...............Request.............................| User |..Request.. 474 | Agent| 475 +------+ 477 <====> CDNI Logging Interface 478 <++++> CDNI Control Interface 479 **** interfaces outside the scope of CDNI 480 .... interfaces outside the scope of CDNI 481 Figure 2: Mapping of CDNI Logging interactions on the CDNI Reference 482 Model 484 As illustrated in Figure 2, the Logging Control (including signaling 485 of which logging fields are to be communicated across CDNs for a 486 given task) occurs over the Control Interface level. The rationale 487 for using the Control interface for Logging Control (instead of for 488 instance using the Metadata interface) includes: 490 o the Logging Control interactions typically define fairly static 491 information for initializing and controlling the Logging 492 interface, which matches the role of the Control Interface as 493 described in [I-D.ietf-cdni-framework] and [RFC6707]. 495 o the Logging Control information (specifying the Logging 496 information format and scope is primarily intended to be consumed 497 by the (typically fairly centralized) logical entity responsible 498 for collecting intra-CDN logs, processing, filtering those and 499 then exporting the relevant subset of logs/fields to the other 500 CDNs. 502 o the surrogates within a given CDN are typically not expected to 503 need to be aware of the specific set of fields or set of events 504 that have been requested by various interconnected CDNs. Rather 505 the surrogates are likely to perform some generic logging for all 506 services regardless of the peculiarities of every CDNI agreement. 507 Processing (e.g. filtering, format adaptation) of the generic 508 logging information generated by the Surrogates is expected to 509 take place to ensure that each interconnected CDN receives the 510 specific set of fields and logs it has requested through Logging 511 Control. Therefore there is no need to ensure that the Logging 512 control information be easily distributable through the CDNs right 513 down to surrogates. 515 o the Control interface is expected to support the capability to 516 apply control at the granularity of content sets (e.g. for content 517 Purge) which is required for Logging Control since it is expected 518 that a CDN may require different sets of logging fields and events 519 for different sets of content (e.g. because it only needs to 520 perform coarse billing for a given CSP while it needs to provide 521 detailed analytics for another CSP). 523 2.2. Overall Logging Chain 525 This section discusses the overall logging chain within and across 526 CDNs to clarify how CDN Logging information is expected to fit in 527 this overall chain. Figure 3 illustrates the overall logging chain 528 within the dCDN, across CDNs using the CDNI Logging interface and 529 within the uCDN. For readability, the Figure only considers logging 530 information flowing from the dCDN to the uCDN. Note that the logging 531 chain illustrated in the Figure is obviously only indicative and 532 varies in specific environments. For example, there may be more or 533 less instantiations of each entity (ie there may be 4 Log consuming 534 applications in a given CDN. As another example, there may be one 535 instance of Rectification process per Log Consuming Application 536 instead of a shared one. 538 Log Consuming Log Consuming 539 App App 540 /\ /\ 541 | | 542 Rectification-------- 543 /\ 544 | 545 Filtering 546 /\ 547 | 548 Collection uCDN 549 /\ /\ 550 | | 551 | Generation 552 | 553 CDNI Logging --------------------------------------------- 554 exchange 555 /\ Log Consuming Log Consuming 556 | App App 557 | /\ /\ 558 | | | 559 Rectification Rectification--------- 560 /\ /\ 561 | | 562 Filtering 563 /\ 564 | 565 Collection dCDN 566 /\ /\ 567 | | 568 Generation Generation 570 Figure 3: CDNI Logging in the overall Logging Chain 572 The following subsections describe each of the processes potentially 573 involved in the logging chain of Figure 3. 575 2.2.1. Logging Generation and During-Generation Aggregation 577 CDNs typically generate logging information for all significant task 578 completions, events, and failures. Logs are typicallly generated by 579 many devices in the CDN including the surrogates, the request routing 580 system, and the control system. 582 The amoung of Logging information generated can be huge. Therefore, 583 during contract negotiations, interconnected CDNs often agree on a 584 Logging retention duration, and optionally, on a maximum size of the 585 Logging data that the dCDN must keep. If this size is exceeded, the 586 dCDN must alert the uCDN but may not keep more Logs for the 587 considered time period. In addition, CDNs may aggregate logs and 588 transmit only summaries for some categories of operations instead of 589 the full Logging data. Note that such aggregation leads to an 590 information loss, which may be problematic for some usages of Logging 591 (e.g., debugging). 593 [I-D.brandenburg-cdni-has] discusses logging for HTTP Adaptive 594 Streaming (HAS). In accordance with the recommendations articulated 595 there, it is expected that a surrogate will generate separate logging 596 information for delivery of each chunk of HAS content. This ensures 597 that separate logging information can then be provided to 598 interconnected CDNs over the CDNI Logging interface. Still in line 599 with the recommendations of [I-D.brandenburg-cdni-has], the logging 600 information for per-chunck delivery may include some information (a 601 Content Collection IDentifier and a Session IDentifier as discussed 602 in Section 4.1.1) intended to facilitate subsequent post-generation 603 aggregation of per-chunk logs into per-session logs. Note that a CDN 604 may also elect to generate aggregate per-session logs when performing 605 HAS delivery, but this needs to be in addition to, and not instead 606 of, the per-chunk delivery logs. We note that this may be revisited 607 in future versions of this document. 609 2.2.2. Logging Collection 611 This is the process that continuously collects logs generated by the 612 log-generating entities within a CDN. 614 In a CDNI environment, in addition to collecting logging information 615 from log-generating entities within the local CDN, the Collection 616 process also collects logging information provided by another CDN, or 617 other CDNs, through the CDNI Logging interface. This is illustrated 618 in Figure 3 where we see that the Collecton process of the uCDN 619 collects logging information from log-generating entities within the 620 uCDN as well as logging information coming through CDNI Logging 621 exchange with the dCDN through the CDNI Logging interface. 623 2.2.3. Logging Filtering 625 A CDN may require to only present different subset of the whole 626 logging information collected to various log-consuming applications. 627 This is achieved by the Filtering process. 629 In particular, the Filtering process can also filter the right subset 630 of information that needs to be provided to a given interconnected 631 CDN. For example, the filtering process in the dCDN can be used to 632 ensure that only the logging information related to tasks performed 633 on behalf of a given uCDN are made available to that uCDN (thereby 634 filtering all the logging information related to deliveries by the 635 dCDN of content for its own CSPs). Similarly, the Filtering process 636 may filter or partially mask some fields, for example, to protect End 637 Users' privacy when communicating CDNI Logging information to another 638 CDN. Filtering of logging information prior to communication of this 639 information to other CDNs via the CDNI Logging interface requires 640 that the downstream CDN can recognize the set of log records that 641 relate to each interconnected CDN. 643 The CDN will also filter some internal scope information such as 644 information related to its internal alarms (security, failures, load, 645 etc). 647 In some use cases described in [I-D.ietf-cdni-use-cases], the 648 interconnected CDNs do not want to disclose details on their internal 649 topology. The filering process can then also filter confidential 650 data on the dCDNs' topology (number of servers, location, etc.). In 651 particular, information about the requests served by every Surrogate 652 may be confidential. Therefore, the Logging information must be 653 protected so that data such as Surrogates' hostnames is not disclosed 654 to the uCDN. In the "Inter-Affiliates Interconnection" use case, 655 this information may be disclosed to the uCDN because both the dCDN 656 and the uCDN are operated by entities of the same group. 658 2.2.4. Logging Rectification and Post-Generation Aggregation 660 If Logging is generated periodically, it is important that the 661 sessions that start in one Logging period and end in another are 662 correctly reported. If they are reported in the starting period, 663 then the Logging of this period will be available only after the end 664 of the session, which delays the Logging generation. 666 A Logging rectification/update mechanism could be useful to reach a 667 good trade-off between the Logging generation delay and the Logging 668 accuracy. Depending on the selected Logging protocol(s), such 669 mechanism may be invaluable for real time Logging, which must be 670 provided rapidly and cannot wait for the end of operations in 671 progress. 673 In the presence of HAS, some log-consuming applications can benefit 674 from aggregate per-session logs. For example, for analytics, per- 675 session logs allow display of session-related trends which are much 676 more meaningful for some types of analysis than chunk-related trends. 677 In the case where the log-generating entities have generated during- 678 generation aggregate logs, those can be used by the applications. In 679 the case where aggregate logs have not been generated, the 680 Rectification process can be extended with a Post-Generation 681 Aggregation process that generates per-session logs from the per- 682 chunk logs, possibly leveraging the information included in the per- 683 chunk logs for that purpose (Content Collection IDentifier and a 684 Session IDentifier). However, in accordance with 685 [I-D.brandenburg-cdni-has], this document does not define exchange of 686 such aggregate logs on the CDNI Logging interface. We note that this 687 may be revisited in future versions of this document. 689 2.2.5. Log-Consuming Applications 691 2.2.5.1. Maintenance/Debugging 693 Logging is useful to permit the detection (and limit the risk) of 694 content delivery failures. In particular, Logging facilitates the 695 resolution of configuration issues. 697 To detect faults, Logging must enable the reporting of any CDN 698 operation success and failure, such as request redirection, content 699 acquisition, etc. The uCDN can summarize such information into KPIs. 700 For instance, Logging format should allow the computation of the 701 number of times during a given epoch that content delivery related to 702 a specific service succeeds/fails. 704 Logging enables the CDN providers to identify and troubleshoot 705 performance degradations. In particular, Logging enables the 706 communication of traffic data (e.g., the amount of traffic that has 707 been forwarded by a dCDN on behalf of an uCDN over a given period of 708 time), which is particularly useful for CDN and network planning 709 operations. 711 2.2.5.2. Accounting 713 Logging is essential for accounting, to permit inter-CDN billing and 714 CSP billing by uCDNs. For instance, Logging enables the uCDN to 715 check the total amount of traffic delivered by every dCDN and for 716 every Delivery Service, as well as, the associated bandwidth usage 717 (e.g., peak, 95th percentile), and the maximum number of simultaneous 718 sessions over a given period of time. 720 2.2.5.3. Analytics and Reporting 722 The goal of analytics is to gather any relevant information to track 723 audience, analyze user behavior, and monitor the performance and 724 quality of content delivery. For instance, Logging enables the CDN 725 providers to report on content consumption (e.g., delivered sessions 726 per content) in a specific geographic area. 728 The goal of reporting is to gather any relevant information to 729 monitor the performance and quality of content delivery and allow 730 detection of delivery issues. For instance, reporting could track 731 the average delivery throughput experienced by End Users in a given 732 region for a specific CSP or content set over a period of time. 734 2.2.5.4. Security 736 The goal of security is to prevent and monitor unauthorized access, 737 misuse, modification, and denial of access of a service. A set of 738 information is logged for security purposes. In particular, a record 739 of access to content is usually collected to permit the CSP to detect 740 infringements of content delivery policies and other abnormal End 741 User behaviors. 743 2.2.5.5. Legal Logging Duties 745 Depending on the country considered, the CDNs may have to retain 746 specific Logging information during a legal retention period, to 747 comply with judicial requirements. 749 2.2.5.6. Notions common to multiple Log Consuming Applications 751 2.2.5.6.1. Logging Information Views 753 Within a given log-consuming application, different views may be 754 provided to differnet users depending on privacy, business, and 755 scalability constraints. 757 For example, an analytics tool run by the uCDN can provide one view 758 to an uCDN operator that exploits all the logging information 759 available to the uCDN, while the tool may provide a different view to 760 each CSP exploiting only the logging information related to the 761 content of the given CSP. 763 As another example, maintenance and debugging tools may provide 764 different views to different CDN operators, based on their 765 operational role. 767 2.2.5.6.2. Key Performance Indicators (KPIs) 769 This section presents, for explanatory purposes, a non-exhaustive 770 list of Key Performance Indicators (KPIs) that can be extracted/ 771 produced from logs. 773 Multiple log-consuming applications, such as analytics, monitoring, 774 and maintenance applications, often compute and track such KPIs. 776 In a CDNI environment, depending on teh situation, these KPIs may be 777 computed by the uCDN or by the dCDN. But it is usually the uCDN that 778 computes KPIs, because uCDN and dCDN may have different definitions 779 of the KPIs and the computation of some KPIs requires a vision of all 780 the deliveries performed by the uCDN and all its dCDNs. 782 Here is a list of important examples of KPIs: 784 o Number of delivery requests received from End Users in a given 785 region for each piece of content, during a given period of time 786 (e.g., hour/day/week/month) 788 o Percentage of delivery successes/failures among the aforementioned 789 requests 791 o Number of failures listed by failure type (e.g., HTTP error code) 792 for requests received from End Users in a given region and for 793 each piece of content, during a given period of time (e.g., hour/ 794 day/week/month) 796 o Number and cause of premature delivery termination for End Users 797 in a given region and for each piece of content, during a given 798 period of time (e.g., hour/day/week/month) 800 o Maximum and mean number of simultaneous sessions established by 801 End Users in a given region, for a given Delivery Service, and 802 during a given period of time (e.g., hour/day/week/month) 804 o Volume of traffic delivered for sessions established by End Users 805 in a given region, for a given Delivery Service, and during a 806 given period of time (e.g., hour/day/week/month) 808 o Maximum, mean, and minimum delivery throughput for sessions 809 established by End Users in a given region, for a given Delivery 810 Service, and during a given period of time (e.g., hour/day/week/ 811 month) 813 o Cache-hit and byte-hit ratios for requests received from End Users 814 in a given region for each piece of content, during a given period 815 of time (e.g., hour/day/week/month) 817 o Top 10 of the most popularly requested content (during a given 818 day/week/month), 820 o Terminal type (mobile, PC, STB, if this information can be 821 acquired from the browser type header, for example). 823 Additional KPIs can be computed from other sources of information 824 than the Logging -- for instance, data collected by a content portal 825 or by specific client-side APIs. Such KPIs are out of scope for the 826 present memo. 828 The KPIs used depend strongly on the considered log-consuming 829 application -- the CDN operator may be interested in different 830 metrics than the CSP is. In particular, CDN operators are often 831 interested in delivery and acquisition performance KPIs, information 832 related to Surrogates' performance, caching information to evaluate 833 the cache-hit ratio, information about the delivered file size to 834 compute the volume of content delivered during peak hour, etc. 836 Some of the KPIs, for instance those providing an instantaneous 837 vision of the active sessions for a given CSP's content, are useful 838 especially if they are provided in real time. By contrast, some 839 other KPIs, such as those averaged over a long period of time, can be 840 provided in non-real time. 842 3. CDNI Logging Information Structure and Transport 844 As defined in Section 1.1 a CDNI logging field is as an atomic 845 logging information element and a CDNI Logging Record is a collection 846 of CDNI Logging Fields containing all logging information 847 corresponding to a single logging event. 849 This document defines non-real time transport of CDNI Logging 850 information over the CDNI interface. For such non-real time 851 transport, this document defines a third level of structure, the CDNI 852 Logging File, that is a collection of CDNI Logging Records. This 853 structure is described in Figure 4. This document then specifies how 854 to transport such CDNI Files across interconnected CDNs. We observe 855 that this approach can be tuned in a real deployment to achieve near- 856 real time exchange of CDNI Logging information, e.g. by increasing 857 the frequency of logging file creation and distribution throughout 858 the Logging chain, but it is not expected that this approach can 859 support real time transport (e.g. sub-second) of CDNI logging 860 information. 862 +------------------------------------------------------+ 863 |CDNI Logging File | 864 | | 865 | +--------------------------------------------------+ | 866 | |CDNI Logging Record | | 867 | | +-------------+ +-------------+ +-------------+ | | 868 | | |CDNI Logging | |CDNI Logging | |CDNI Logging | | | 869 | | | Field | | Field | | Field | | | 870 | | +-------------+ +-------------+ +-------------+ | | 871 | +--------------------------------------------------+ | 872 | | 873 | +--------------------------------------------------+ | 874 | |CDNI Logging Record | | 875 | | +-------------+ +-------------+ +-------------+ | | 876 | | |CDNI Logging | |CDNI Logging | |CDNI Logging | | | 877 | | | Field | | Field | | Field | | | 878 | | +-------------+ +-------------+ +-------------+ | | 879 | +--------------------------------------------------+ | 880 | | 881 | +--------------------------------------------------+ | 882 | |CDNI Logging Record | | 883 | | +-------------+ +-------------+ +-------------+ | | 884 | | |CDNI Logging | |CDNI Logging | |CDNI Logging | | | 885 | | | Field | | Field | | Field | | | 886 | | +-------------+ +-------------+ +-------------+ | | 887 | +--------------------------------------------------+ | 888 +------------------------------------------------------+ 890 Figure 4: Structure of Logging Files 892 It is expected that future version of this document will also specify 893 real time transport of CDNI Logging information over the CDNI 894 interface. We note that this might involve direct transport of CDNI 895 Logging Records without prior grouping into a file structure to avoid 896 the latency associated with creating and transporting such a file 897 structure throughout the logging chain. 899 The semantics and encoding of the CDNI Logging fields are specified 900 in Section 4. The semantics and encoding of CDNI Records are 901 specified in Section 5. The CDNI Logging File format is specified in 902 Section 6. The protocol for transport of CDNI Logging File is 903 specified in Section 7. 905 4. CDNI Logging Fields 907 Existing CDNs Logging functions collect and consolidate logs 908 performed by their Surrogates. Surrogates usually store the logs 909 using a format derived from Web servers' and caching proxies' log 910 standards such as W3C, NCSA [ELF] [CLF], or Squid format [squid]. In 911 practice, these formats are adapted to cope with CDN specifics. 912 Appendix A presents examples of commonly used log formats. 914 4.1. Generic Fields 916 This section specifies a set of generic CDNI Logging Fields that are 917 expected to be found in multiple types of CDNI Logging records. 919 4.1.1. Semantics of Generic CDNI Logging Fields 921 The semantics of the generic CDNI Logging Fields are specified in 922 Table 1. 924 +------------+------------------------------------------------------+ 925 | Name | Description | 926 +------------+------------------------------------------------------+ 927 | Start-time | A start date and time associated with a logged | 928 | | event; for instance, the time at which a Surrogate | 929 | | received a content delivery request or the time at | 930 | | which an origin server received a content | 931 | | acquisition request. | 932 | End-time | An end date and time associated with a logged event. | 933 | | For instance, the time at which a Surrogate | 934 | | completed the handling of a content delivery request | 935 | | (e.g., end of delivery or error). | 936 | Duration | The duration of an operation in milliseconds. For | 937 | | instance, this field could be used to provide the | 938 | | time it took the Surrogate to send the requested | 939 | | file to the End-User or the time it took the | 940 | | Surrogate to acquire the file on a cache-miss event. | 941 | | In the case where Start-time, End-time, and Duration | 942 | | appear in a Logging Record, the Duration is to be | 943 | | interpreted as a total activity time related to the | 944 | | logged operation. | 945 | Client-IP | The IP address of the User Agent that issued the | 946 | | logged request or of a proxy, for instance | 947 | | "203.0.113.1". | 948 | Client-por | The source port of the logged request (e.g., 9542) | 949 | t | | 950 | Destinatio | The IP address of the host that received the logged | 951 | n-IP | request (e.g., 192.0.2.2). | 952 | Destinatio | The destination port of the logged request (e.g., | 953 | n-port | 80). | 954 | Operation | The kind of operation that is logged; for instance, | 955 | | Acquisition, Delivery, or Purging. | 956 | URI_full | The full requested URL (e.g., | 957 | | "http://node1.peer-a.op-b.net/cdn.csp.com/movies/pot | 958 | | ter.avi?param=11&user=toto"). When HTTP request | 959 | | redirection is used, this URI includes the Surrogat | 960 | | eFQDN. If the association of requests to Surrogates | 961 | | is confidential, the dCDN can present only URI_part | 962 | | to uCDN. | 963 | URI_part | The requested URL path (e.g., | 964 | | /cdn.csp.com/movies/potter.avi?param=11&user=toto if | 965 | | the full request URL was | 966 | | "http://node1.peer-a.op-b.net/cdn.csp.com/movies/pot | 967 | | ter.avi?param=11&user=toto"). The URI without | 968 | | host-name typically includes the "CDN domain" | 969 | | (ex.cdn.csp.com) - cf. [I-D.ietf-cdni-framework]: i | 970 | | tenables the identification of the CSP service agree | 971 | | dbetween the CSP and the CDNP operating the uCDN. | 972 | Protocol | The protocol and protocol version of the message | 973 | | that triggered the Logging entry (e.g., HTTP/1.1). | 974 | Request-me | The protocol method of the request message that | 975 | thod | triggered the Logging entry. | 976 | Status | The protocol method of the reply message related to | 977 | | the Logging entry | 978 | Bytes-Sent | The number of bytes at application-layer | 979 | | protocol-level (e.g., HTTP) of the reply message | 980 | | related to the Logging entry. It includes the size | 981 | | of the response headers. | 982 | Headers-Se | The number of bytes corresponding to response | 983 | nt | headers at application-layer protocol-level (e.g., | 984 | | HTTP) of the reply message related to the Logging | 985 | | entry. | 986 | Bytes-rece | The number of bytes (headers + body) of the message | 987 | ived | that triggered the Logging entry. | 988 | Referrer | The value of the Referrer header in an HTTP request. | 989 | User-Agent | The value of the User Agent header in an HTTP | 990 | | request. | 991 | Cookie | The value of the Cookie header in an HTTP request. | 992 | Byte-Range | [Ed. note: to be defined] | 993 | Cache-cont | The value of the cache-control header in an HTTP | 994 | rol | answer. This header is particularly important for | 995 | | content acquisition logs. | 996 | Record-dig | A digest of the Logging Record; it enables detecting | 997 | est | corrupted Logging Records. | 998 | CCID | A Content Collection IDentifier (CCID) eases the | 999 | | correlation of several Logging Records related to a | 1000 | | Content Collection (e.g., a movie split in chunks). | 1001 | SID | A Session Identifier (SID) eases the correlation | 1002 | | (and aggregation) of several Logging Records related | 1003 | | to a session. The SID is especially relevant for | 1004 | | summarizing HAS Logging information | 1005 | | [I-D.brandenburg-cdni-has]. | 1006 +------------+------------------------------------------------------+ 1008 Table 1: Semantics of Generic CDNI Logging Fields 1010 NB: we define three fields related to the timing of logged 1011 operations: Start-time, End-time, and Duration. Start-time is 1012 typically useful for human readers (e.g., while debugging), however, 1013 some servers log the operation's End-time which corresponds to the 1014 time of log record generation. In absence of Logging summarization, 1015 only two of these three fields are required to obtain relevant timing 1016 information on the operation. However, when some kind of Logging 1017 aggregation/summarization is used, it can be advantageous to keep the 1018 three fields: for instance, in the case of HAS, keeping the three 1019 fields permits computing an average delivery bitrate from a single 1020 Logging Record aggregating information on the delivery of multiple 1021 consecutive video chunks. 1023 Multiple header fields, in addition to the ones explicitly listed in 1024 the table could be reproduced in the Logging records. 1026 Note that uCDN may want to filter Logging data by user (and not by IP 1027 address) to provide more relevant information to the CSP. In such 1028 case, a user may be identified as a combination of several pieces of 1029 information such as the client IP and User Agent or through the SID. 1031 The URI_full provides information on the Surrogate that provided the 1032 content. This information can be relevant, for instance, for the 1033 Inter-Affiliates use case described in [I-D.ietf-cdni-use-cases]. 1034 However, in some cases it may be considered as confidential and the 1035 dCDN may provide URI_part instead. 1037 4.1.2. Syntax of Generic CDNI Logging Fields 1039 Table 2 illustrates the definition of the information elements. It 1040 provides examples using Apache log format strings [apache] when they 1041 exist. 1043 [Ed Note, this should be replaced with actual selected format for 1044 CDNI] 1046 [Ed. note: specify for all Logging Fields the type (e.g., varchar, 1047 int, float, ...) and the maximum size (e.g., varchar(200))] 1049 +----------+-------------------+------------------------------------+ 1050 | Name | String | Example | 1051 +----------+-------------------+------------------------------------+ 1052 | Time | %t | [10/Oct/2000:13:55:36-0700] | 1053 | Duration | %D | - | 1054 | Client-I | %a | 203.0.113.45 | 1055 | P | | | 1056 | Operatio | - | - | 1057 | n | | | 1058 | URI_full | %U | - | 1059 | Protocol | %H | HTTP/1.0 | 1060 | Request | %m | GET | 1061 | method | | | 1062 | Status | %>s | 200 | 1063 | Bytes | %O | 2326 | 1064 | Sent | | | 1065 | Bytes | %I | 432 | 1066 | received | | | 1067 | Header | \"%{Referrer}i\" | "http://www.example.com/start.html | 1068 | | \"%{User-agent}i\ | ""Mozilla/4.08 [en] (Win98; I | 1069 | | " | ;Nav)" | 1070 +----------+-------------------+------------------------------------+ 1072 Table 2: Examples using Apache format 1074 4.2. Logging Fields for Content Delivery 1076 Beyond the Logging Fields described in previous section, this section 1077 defines additional Logging Fields that are specifically related to 1078 Content Delivery operations. Note that the uCDN may not transfer the 1079 information provided in some of these fields to the CSP, depending on 1080 the CSP's interest in the information and on the information's 1081 confidentiality level. 1083 4.2.1. Semantics for Delivery CDNI Logging Fields 1085 The semantics of the generic CDNI Logging Fileds are specified in 1086 Table 3. 1088 +-------------------+-----------------------------------------------+ 1089 | Name | Definition | 1090 +-------------------+-----------------------------------------------+ 1091 | uCDN-ID | An element authenticating the operator of the | 1092 | | uCDN as the authority having delegated the | 1093 | | request to the dCDN. | 1094 | Delivering-CDN-ID | An identifier (e.g., an aggregation of an IP | 1095 | | address and a FQDN) of the Delivering CDN. | 1096 | | The Delivering-CDN-ID might be considered as | 1097 | | confidential by the dCDN. In such case, the | 1098 | | dCDN could either not provide this field to | 1099 | | the uCDN or overwrite the Delivering-CDN-ID | 1100 | | with its on identifier. | 1101 | Cache-bytes | The number of body bytes served from caches. | 1102 | | This quantity permits the computation of the | 1103 | | byte hit ratio. | 1104 | Action | The Action describes how a given request was | 1105 | | treated locally: through which transport | 1106 | | protocol, with or without content | 1107 | | revalidation, with a cache hit or cache miss, | 1108 | | with fresh or stale content, and (if | 1109 | | relevant) with which error. Example with | 1110 | | Squid format [squid]: "TCP_REFRESH_FAIL_HIT" | 1111 | | means that an expired copy of an object | 1112 | | requested through TCP was in the cache. | 1113 | | Squid attempted to make an If-Modified-Since | 1114 | | request, but it failed. The old (stale) | 1115 | | object was delivered to the client. | 1116 +-------------------+-----------------------------------------------+ 1118 Table 3: Semantics of the Delivery CDNI Logging Fields 1120 [Ed. note: Other information that could be logged include operations 1121 related to the authorization of the requests, URL rewriting rules 1122 enforced, the X-FORWARDED-FOR non standard HTTP header...] 1124 4.2.2. Syntax for Delivery CDNI Logging Fields 1126 [Ed Note: To be added] 1128 4.3. Logging Fields for Content Acquisition 1130 This section specifies Logging fields that are specific to Content 1131 Acquisition operations. 1133 4.3.1. Semantics for Acquisition CDNI Logging Fields 1135 Table 4 specifies the semantics of the Acquisition specific CDNI 1136 Logging Fields. 1138 +--------------------+----------------------------------------------+ 1139 | Name | Definition | 1140 +--------------------+----------------------------------------------+ 1141 | dCDN identifier | An element authenticating the operator of | 1142 | | the dCDN as the authority requesting the | 1143 | | content to the uCDN | 1144 | Caching_date | Date at which the delivered content was | 1145 | | stored in cache | 1146 | Validity_headers | A copy of all headers related to content | 1147 | | validity: no-cache, ETag, Vary, | 1148 | | last-modified... | 1149 | Lookup_duration | Duration of the DNS resolution for resolving | 1150 | | the FQDN of (uCDN's or CSP's) origin server. | 1151 | Delay_to_first_bit | Duration of the operations from the sending | 1152 | | of the content acquisition request to the | 1153 | | reception of the first bit of the requested | 1154 | | content. | 1155 | Delay_to_last_bit | Duration of the operations from the sending | 1156 | | of the content acquisition request to the | 1157 | | reception of the last bit of the requested | 1158 | | content. | 1159 +--------------------+----------------------------------------------+ 1161 Table 4: Semantics of the Acquisition CDNI Logging Fields 1163 These information elements may be used in Content Acquisition Logging 1164 provided by dCDN to uCDN and, potentially, in Content Acquisition 1165 Logging provided by uCDN to dCDN. 1167 4.3.2. Syntax for Acquisition CDNI Logging Fields 1169 [Ed Note: To be added] 1171 4.4. Logging Fields for Control 1173 [Ed. note: LOGS RELATED TO KEY EXCHANGES FOR INSTANCE, SECTION TO BE 1174 WRITTEN AFTER THE CONTROL INTERFACE IS MORE CLEARLY DEFINED] 1176 4.5. Logging Fields for Other Operations 1178 Logging can be used for debugging. Therefore, all kinds of CDN 1179 operations might be logged, depending on the agreement between the 1180 dCDN and the uCDN. In particular, operations related to Request 1181 Routing and Metadata can be logged. 1183 5. CDNI Logging Records 1185 [Ed. note: we need to specify the encoding of the file, the 1186 separation character, etc...] 1188 This section defines a set of central events that a dCDN should 1189 register and publish through the Logging interface. 1191 We classify the logged events depending on the CDN operation to which 1192 they relate: Content Delivery, Content Acquisition, Content 1193 Invalidation/Purging, etc. 1195 5.1. Content Delivery 1197 Some CSPs pay a lot of attention to the protection of their content 1198 (e.g., premium video CSPs). To fulfill the needs of these CSPs, a 1199 CDN shall log all the details of the content delivery authorizations. 1200 This means that a dCDN must be able to provide Logging detailing the 1201 content delivery/content acquisition authorizations and denials as 1202 well as information on why the request is authorized/denied. 1204 CSPs and CDN service providers pay a lot of attention to errors 1205 related to content delivery. It is therefore of upmost importance 1206 that the dCDN provides detailed error information in the Logging 1207 data. This information should typically be available even when 1208 Logging is aggregated. 1210 The content delivery events triggering the generation of a Logging 1211 Record include: 1213 o Reception of a content request, 1215 The generated Logging Record typically embeds information about: 1217 o Denial of delivery (error or unauthorized request, e.g., HTTP 401) 1218 for a request, 1220 o Beginning of delivery (authorization) of a requested content, 1222 o End of an authorized delivery (success), 1224 o End of an authorized delivery (failure during the delivery, e.g., 1225 HTTP 403). 1227 5.2. Content Acquisition 1229 5.2.1. Logging Records Provided by dCDN to uCDN 1231 When the uCDN requires the dCDN to provide Logging for acquisition 1232 related events, the events triggering the generation of a Logging 1233 Record include: 1235 o Emission of a content acquisition request (first try or retry) for 1236 a cache hit or a cache miss with content revalidation 1238 The generated Logging Record typically embeds information about: 1240 o Reception of a reply indicating denial of delivery (error or 1241 unauthorized request) for a content acquisition request, 1243 o End of an authorized acquisition (success), 1245 o End of an authorized acquisition (failure) 1247 Note that a dCDN may acquire content only from the uCDN. It this 1248 case, the uCDN can log the dCDN's content acquisition operations 1249 itself, and thus, the uCDN may not require the dCDN to log 1250 acquisition related events. However, comparing the dCDN and uCDN 1251 logs is often useful for debugging and for security auditing. 1253 5.2.2. Logging Records Provided by uCDN to dCDN 1255 When the dCDN requires the uCDN to provide Logging for acquisition 1256 related events, the events triggering the generation of a Logging 1257 Record include: 1259 o Reception of a content acquisition request for the considered 1260 Delivery Service for a cache hit or a cache miss with content 1261 revalidation 1263 The generated Logging Record typically embeds information about: 1265 o Emission of a reply indicating denial of delivery (error or 1266 unauthorized request) for a content acquisition request, 1268 o End of an authorized acquisition (success), 1270 o End of an authorized acquisition (failure). 1272 5.3. Content Invalidation and Purging 1274 When the uCDN requests a dCDN to log invalidation/purging events 1275 (e.g., for security), the events triggering the generation of a 1276 Logging Record include: 1278 o Reception of a content invalidation/purging request 1280 The generated Logging Record typically embeds information about: 1282 o Denial of the invalidation/purging request (error or unauthorized 1283 request, with details about the causes of the error), 1285 o Beginning of invalidation/purging (authorization) for a given 1286 content purging request, 1288 o End of an authorized invalidation/purging (success), 1290 o End of an authorized invalidation/purging (failure). 1292 5.4. Logging Extensibility 1294 Future usages might introduce the need for additional Logging fields. 1295 In addition, some use-cases such as an Inter-Affiliate 1296 Interconnection [I-D.ietf-cdni-use-cases], might take advantage of 1297 extended Logging exchanges. Therefore, it is important to permit 1298 CDNs to use additional Logging fields besides the standard ones, if 1299 they want. For instance, an "Account-name" identifying the contract 1300 enforced by the dCDN for a given request could be provided in 1301 extended fields. 1303 The required Logging Records may depend on the considered services. 1304 For instance, static file delivery (e.g., pictures) typically does 1305 not include any delivery restrictions. By contrast, video delivery 1306 typically implies strong content delivery restrictions, as explained 1307 in [I-D.ietf-cdni-use-cases], and Logging could include information 1308 about the enforcement of these restrictions. Therefore, to ease the 1309 support of varied services as well as of future services, the Logging 1310 interface should support optional Logging Records. 1312 6. CDNI Logging File Format 1314 Interconnected CDNs may support various Logging formats. However, 1315 they must support at least the default Logging File format described 1316 here. 1318 6.1. Logging Files 1320 [Ed. Note: How many files (one per type of Delivery Service (e.g., 1321 HTTP, WMP) and per type of Event (e.g., Errors, Delivery, 1322 Acquisition,...?)and what would be inside... These aspects needs to 1323 be detailed...] 1325 6.2. File Format 1327 The Logging file format should be independent from the selected 1328 transport protocol, to guarantee a flexible choice of transport 1329 protocols. [Ed. note: for the real time Logging exchanges, this 1330 might be hard] 1332 All Logging Records in a Logging File must share the same format 1333 (same set of Logging Fields, in the same order, with the same 1334 semantics, separated by the same Separator Character), to ease the 1335 parsing of the Logging data by the CDN that receives the Logging 1336 File. The CDN that provides the Logging data is responsible for 1337 guaranteeing the consistency of the Logging records' formats, 1338 typically via its log filtering and aggregation processes (see 1339 Section 2.2.3). 1341 6.2.1. Headers 1343 Logging files must include a header with the information described in 1344 Figure 5. 1346 +----------------+-------------------+------------------------------+ 1347 | Field | Description | Examples | 1348 +----------------+-------------------+------------------------------+ 1349 | Format | Identification of | standard_cdni_errors_http_v1 | 1350 | | CDNI Log format. | | 1351 | Fields | A description of | | 1352 | | the record format | | 1353 | | (list of fields). | | 1354 | Log-ID | Identifier | abcdef1234 | 1355 | | for the CDNI Log | | 1356 | | file (facilitates | | 1357 | | detection of | | 1358 | | duplicate Logs | | 1359 | | and tracking in | | 1360 | | case of | | 1361 | | aggregation). | | 1362 | Log-Timestamp | Time, in | [20/Feb/2012:00:29.510+0200] | 1363 | | milliseconds, the | | 1364 | | CDNI Log was | | 1365 | | generated. | | 1366 | Log-Origin | Identifier of the | cdn1.cdni.example.com | 1367 | | authority (e.g., | | 1368 | | dCDN or uCDN) | | 1369 | | providing the Log-| | 1370 | | -ging | | 1371 +----------------+-------------------+------------------------------+ 1373 Figure 5: Logging Headers 1375 All time-related Logging Fields and data in the Logging File headers/ 1376 footers must provide a time zone and be at least at millisecond (ms) 1377 accuracy. The accuracy must be consistent to permit the computation 1378 of KPIs involving operations realized on several CDNs. 1380 [Ed. note: would it make sense to add a kind of "example Logging 1381 Record" in the Logging file and associated semantic (e.g. in a 1382 structure data format) ?] 1384 6.2.2. Body (Logging Records) Format 1386 [Ed. note: the W3C extended log format is a good base candidate to 1387 look at.] 1389 [Ed. note: Records used for real time information and non-real time 1390 information could use different formats. In this version, we do not 1391 yet tackle the problem of real time logging exchanges] 1393 6.2.3. Footer Format 1395 Logging files must include a footer with the information described in 1396 Figure 6. 1398 +---------+----------------------------------------------+----------+ 1399 | Field | Description | Examples | 1400 +---------+----------------------------------------------+----------+ 1401 | Log | Digest of the complete Log (facilitates | | 1402 | Digest | detection of Log corruption) | | 1403 +---------+----------------------------------------------+----------+ 1405 Figure 6: Logging footers 1407 This digest field permits the detection of corrupted Logging files. 1408 This can be useful, for instance, if a problem occurs on the 1409 filesystem of the dCDN Logging system and leads to a truncation of a 1410 logging file. Additional mechanisms to avoid corrupted Logging files 1411 are expected to be provided by the Logging transport protocol, cf. 1412 Section 7. 1414 7. CDNI Logging File Transport Protocol 1416 As presented in [RFC6707], several protocols already exist that could 1417 potentially be used to exchange CDNI Logging between interconnected 1418 CDNs. 1420 The offline exchange of non real-time Logging could rely on several 1421 protocols. In particular, the dCDN could publish the Logging on a 1422 server where the uCDN would retrieve them using a secure protocol 1423 (yet to be identified). 1425 [Ed. note: Propose protocol, e.g. SSH File Transfer Protocol (SFTP) 1426 [I-D.ietf-secsh-filexfer]. and add call flow] 1428 [Ed note: include options for lossless compression] 1430 8. Logging Control 1432 The CDNI Control interface is responsible for correctly configuring 1433 the Logging interface between interconnected CDNs, for every Delivery 1434 Service and according to the Logging configuration agreed during 1435 business negotiations. 1437 This section will identify the parameters that the CDNI Control 1438 interface should manage on uCDN and dCDN for activating, updating, or 1439 removing a CDNI Logging configuration for a given Delivery Service. 1441 [Ed. Note: uCDN shall be able to select the type of events that a 1442 dCDN should include in the Logging that the latter provides to the 1443 uCDN. This will be discussed during business negotiations and the 1444 Control must enforce the agreed configuration. The use of multiple 1445 levels of Logging granularity such as Syslog's "severity levels" 1446 (Emergency, Alert, Critical, ..., Debug) [RFC5424] may help in 1447 providing the most relevant amount of information depending on the 1448 intended Logging usage, as specified during the Logging format 1449 negotiation.] 1451 [Ed. note: the specification all Logging Fields' maximum size (e.g., 1452 varchar(200)) might be constrained in some CDNs so need to exchange 1453 that information during the configuration] 1455 9. Open Issues 1457 The main remaining tasks on this ID are the following: 1459 o Detail the Logging Fields' syntax 1461 o Recommend a Logging File Transport Protocol and detail the call- 1462 flows 1464 o Detail mechanisms for Real-Time Logging 1466 [Ed. Note: The format for Time is still to be agreed on. RFC 5322 1467 (Section 3.3) format could be used or ISO 8601 formatted date and 1468 time in UTC (same format as proposed in 1469 [draft-caulfield-cdni-metadata-core-00]). Also see RFC5424 Section 1470 6.2.3.] 1472 [Ed. Note:When to log the end of a session when the End-User pauses 1473 a video display?] 1475 [Ed. note: (comment from Kevin) how are errors handled ? If the 1476 client gets handed a bunch of 403s and 404s, but still gets the 1477 content eventually, without triggering an event, are those still 1478 logged? For Bytes-Sent, if there were aborted requests, do those get 1479 counted as well? Not all client behavior can be correlated with the 1480 simplified log] 1482 10. IANA Considerations 1484 This memo includes no request to IANA. 1486 11. Security Considerations 1488 11.1. Privacy 1490 CDNs have the opportunity to collect detailed information about the 1491 downloads performed by End-Users. The provision of this information 1492 to another CDN introduces End-Users privacy protection concerns. 1494 11.2. Non Repudiation 1496 Logging provides the raw material for charging. It permits the dCDN 1497 to bill the uCDN for the content deliveries that the dCDN makes on 1498 behalf of the uCDN. It also permits the uCDN to bill the CSP for the 1499 content Delivery Service. Therefore, non-repudiation of Logging data 1500 is essential. 1502 12. Acknowledgments 1504 The authors would like to thank Sebastien Cubaud, Anne Marrec, 1505 Yannick Le Louedec, and Christian Jacquenet for detailed feedback on 1506 early versions of this document and for their input on existing Log 1507 formats. 1509 The authors would like also to thank Fabio Costa, Yvan Massot, Renaud 1510 Edel, and Joel Favier for their input and comments. 1512 Finally, they thank the contributors of the EU FP7 OCEAN project for 1513 valuable inputs. 1515 13. References 1517 13.1. Normative References 1519 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1520 Requirement Levels", BCP 14, RFC 2119, March 1997. 1522 [RFC5424] Gerhards, R., "The Syslog Protocol", RFC 5424, March 2009. 1524 13.2. Informative References 1526 [CLF] A. Luotonen, "The Common Log-file Format, W3C (work in 1527 progress)", 1995, . 1530 [ELF] Phillip M. Hallam-Baker and Brian Behlendorf, "Extended 1531 Log File Format, W3C (work in progress), WD-logfile- 1532 960323", . 1534 [I-D.bertrand-cdni-experiments] 1535 Faucheur, F. and L. Peterson, "Content Distribution 1536 Network Interconnection (CDNI) Experiments", 1537 draft-bertrand-cdni-experiments-02 (work in progress), 1538 February 2012. 1540 [I-D.brandenburg-cdni-has] 1541 Brandenburg, R., Deventer, O., Faucheur, F., and K. Leung, 1542 "Models for adaptive-streaming-aware CDN Interconnection", 1543 draft-brandenburg-cdni-has-03 (work in progress), 1544 July 2012. 1546 [I-D.ietf-cdni-framework] 1547 Peterson, L. and B. Davie, "Framework for CDN 1548 Interconnection", draft-ietf-cdni-framework-01 (work in 1549 progress), July 2012. 1551 [I-D.ietf-cdni-requirements] 1552 Leung, K. and Y. Lee, "Content Distribution Network 1553 Interconnection (CDNI) Requirements", 1554 draft-ietf-cdni-requirements-03 (work in progress), 1555 June 2012. 1557 [I-D.ietf-cdni-use-cases] 1558 Bertrand, G., Emile, S., Burbridge, T., Eardley, P., Ma, 1559 K., and G. Watson, "Use Cases for Content Delivery Network 1560 Interconnection", draft-ietf-cdni-use-cases-10 (work in 1561 progress), August 2012. 1563 [I-D.ietf-secsh-filexfer] 1564 Galbraith, J. and O. Saarenmaa, "SSH File Transfer 1565 Protocol", draft-ietf-secsh-filexfer-13 (work in 1566 progress), July 2006. 1568 [RFC6707] Niven-Jenkins, B., Le Faucheur, F., and N. Bitar, "Content 1569 Distribution Network Interconnection (CDNI) Problem 1570 Statement", RFC 6707, September 2012. 1572 [apache] "Apache 2.2 log files documentation", Feb. 2012, 1573 . 1575 [squid] "Squid Log-Format documentation", Feb. 2012, 1576 . 1578 Appendix A. Examples Log Format 1580 This section provides example of log formats implemented in existing 1581 CDNs, web servers, and caching proxies. 1583 Web servers (e.g., Apache) maintain at least one log file for logging 1584 accesses to content (the Access Log). They can typically be 1585 configured to log errors in a separate log file (the Error Log). The 1586 log formats can be specified in the server's configuration files. 1587 However, webmasters often use standard log formats to ease the log 1588 processing with available log analysis tools. 1590 A.1. W3C Common Log File (CLF) Format 1592 The Common Log File (CLF) format defined by the World Wide Web 1593 Consortium (W3C) working group is compatible with many log analysis 1594 tools and is supported by the main web servers (e.g., Apache) Access 1595 Logs. 1597 According to [CLF], the common log-file format is as follows: 1598 remotehost rfc931 authuser [date] "request" status bytes. 1600 Example (from [apache]): 127.0.0.1 - frank [10/Oct/2000:13:55:36 1601 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326 1603 The fields are defined as follows [CLF]: 1605 +------------+------------------------------------------------------+ 1606 | Element | Definition | 1607 +------------+------------------------------------------------------+ 1608 | remotehost | Remote hostname (or IP number if DNS hostname is not | 1609 | | available, or if DNSLookup is Off. | 1610 | rfc931 | The remote logname of the user. | 1611 | authuser | The username that the user employed to authenticate | 1612 | | himself. | 1613 | [date] | Date and time of the request. | 1614 | "request" | An exact copy of the request line that came from the | 1615 | | client. | 1616 | status | The status code of the HTTP reply returned to the | 1617 | | client. | 1618 | bytes | The content-length of the document transferred. | 1619 +------------+------------------------------------------------------+ 1621 Table 5: Information elements in CLF format 1623 A.2. W3C Extended Log File (ELF) Format 1625 The Extended Log File (ELF) format defined by W3C extends the CLF 1626 with new fields. This format is supported by Microsoft IIS 4.0 and 1627 5.0. 1629 The supported fields are listed below [ELF]. 1631 +------------+---------------------------------------------------+ 1632 | Element | Definition | 1633 +------------+---------------------------------------------------+ 1634 | date | Date at which transaction completed | 1635 | time | Time at which transaction completed | 1636 | time-taken | Time taken for transaction to complete in seconds | 1637 | bytes | bytes transferred | 1638 | cached | Records whether a cache hit occurred | 1639 | ip | IP address and port | 1640 | dns | DNS name | 1641 | status | Status code | 1642 | comment | Comment returned with status code | 1643 | method | Method | 1644 | uri | URI | 1645 | uri-stem | Stem portion alone of URI (omitting query) | 1646 | uri-query | Query portion alone of URI | 1647 +------------+---------------------------------------------------+ 1649 Table 6: Information elements in ELF format 1651 Some fields start with a prefix (e.g., "c-", "s-"), which explains 1652 which host (client/server/proxy) the field refers to. 1654 o Prefix Description 1656 o c- Client 1658 o s- Server 1660 o r- Remote 1662 o cs- Client to Server. 1664 o sc- Server to Client. 1666 o sr- Server to Remote Server (used by proxies) 1668 o rs- Remote Server to Server (used by proxies) 1670 Example: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs- 1671 username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status 1672 time-taken 1674 2011-11-23 15:22:01 x.x.x.x GET /file 80 y.y.y.y Mozilla/ 1675 5.0+(Windows;+U;+Windows+NT+6.1;+en-US;+rv:1.9.1.6)+Gecko/ 1676 20091201+Firefox/3.5.6+GTB6 200 0 0 2137 1678 A.3. National Center for Supercomputing Applications (NCSA) Common Log 1679 Format 1681 This format for Access Logs offers the following fields: 1683 o host rfc931 date:time "request" statuscode bytes 1685 o x.x.x.x userfoo [10/Jan/2010:21:15:05 +0500] "GET /index.html 1686 HTTP/1.0" 200 1043 1688 A.4. NCSA Combined Log Format 1690 The NCSA Combined log format is an extension of the NCSA Common log 1691 format with three (optional) additional fields: the referral field, 1692 the user_agent field, and the cookie field. 1694 o host rfc931 username date:time request statuscode bytes referrer 1695 user_agent cookie 1697 o Example: x.x.x.x - userfoo [21/Jan/2012:12:13:56 +0500] "GET 1698 /index.html HTTP/1.0" 200 1043 "http://www.example.com/" "Mozilla/ 1699 4.05 [en] (WinNT; I)" "USERID=CustomerA;IMPID=01234" 1701 A.5. NCSA Separate Log Format 1703 The NCSA Separate log format refers to a log format in which the 1704 information gathered is separated into three separate files. This 1705 way, every entry in the Access Log (in the NCSA Common log format) is 1706 complemented with an entry in a Referral log and another one in an 1707 Agent log. These three records can be correlated easily thanks to 1708 the date:time value. The format of the Referral log is as follows: 1710 o date:time referrer 1712 o Example: [21/Jan/2012:12:13:56 +0500] 1713 "http://www.example.com/index.html" 1715 The format of the Agent log is as follows: 1717 o date:time agent 1719 o [21/Jan/2012:12:13:56 +0500] "Microsoft Internet Explorer - 5.0" 1721 A.6. Squid 2.0 Native Log Format for Access Logs 1723 Squid [squid] is a popular piece of open-source software for 1724 transforming a Linux host into a caching proxy. Variations of Squid 1725 log format are supported by some CDNs. 1727 Squid common access log format is as follow: time elapsed remotehost 1728 code/status bytes method URL rfc931 peerstatus/peerhost type. 1730 Squid also supports a more detailed native access log format: 1731 Timestamp Elapsed Client Action/Code Size Method URI Ident Hierarchy/ 1732 From Content 1734 According to Squid 2.0 documentation [squid], these fields are 1735 defined as follows: 1737 +-----------+-------------------------------------------------------+ 1738 | Element | Definition | 1739 +-----------+-------------------------------------------------------+ 1740 | time | Unix timestamp as UTC seconds with a millisecond | 1741 | | resolution. | 1742 | duration | The elapsed time in milliseconds the transaction | 1743 | | busied the cache. | 1744 | client | The client IP address. | 1745 | address | | 1746 | bytes | The size is the amount of data delivered to the | 1747 | | client, including headers. | 1748 | request | The request method to obtain an object. | 1749 | method | | 1750 | URL | The requested URL. | 1751 | rfc931 | may contain the ident lookups for the requesting | 1752 | | client (turned off by default) | 1753 | hierarchy | The hierarchy information provides information on how | 1754 | code | the request was handled (forwarding it to another | 1755 | | cache, or requesting the content to the Origin | 1756 | | Server). | 1757 | type | The content type of the object as seen in the HTTP | 1758 | | reply header. | 1759 +-----------+-------------------------------------------------------+ 1761 Table 7: Information elements in Squid format 1763 Squid also uses a "store log", which covers the objects currently 1764 kept on disk or removed ones, for debugging purposes typically. 1766 Appendix B. Requirements 1768 B.1. Additional Requirements 1770 Section 7 of [I-D.ietf-cdni-requirements], already specifies a set of 1771 requirements for Logging (LOG-1 to LOG-16). Some security 1772 requirements also affect Logging (e.g., SEC-4). 1774 This section is a placeholder for requirements identified in the work 1775 on logging, before they are proposed to the requirements draft 1776 authors. 1778 Logging data is sensitive as it provides the raw material for 1779 producing bills etc. Therefore, the protocol delivering the Logging 1780 data must be reliable to avoid information loss. In addition, the 1781 protocol must scale to support the transport of large amounts of 1782 Logging data. 1784 CDNs need to trust Logging information, thus, they want to know: 1786 o who issued the Logging (authentication), and 1788 o if the Logging has been modified by a third party (integrity). 1790 Logging also contains confidential data, and therefore, it should be 1791 protected from eavesdropping. 1793 All these needs translate into security requirements on both the 1794 Logging data format and on the Logging protocol. 1796 Finally, this protocol must comply with the requirements identified 1797 in [I-D.ietf-cdni-requirements]. 1799 [Ed. note: cf. requirements draft: "SEC-4 [MED] The CDNI solution 1800 should be able to ensure that the Downstream CDN cannot spoof a 1801 transaction log attempting to appear as if it corresponds to a 1802 request redirected by a given Upstream CDN when that request has not 1803 been redirected by this Upstream CDN. This ensures non-repudiation 1804 by the Upstream CDN of transaction logs generated by the Downstream 1805 CDN for deliveries performed by the Downstream CDN on behalf of the 1806 Upstream CDN."] 1808 B.2. Compliancy with Requirements draft 1810 This section checks that all the identified requirements in the 1811 Requirements draft are fulfilled by this document. 1813 [Ed. node: to be written later] 1815 Appendix C. CDNI WG's position on candidate protocols for Logging 1816 Transport 1818 This section will be expanded later with the position of the WG 1819 considering the alternative candidate protocols for Logging in CDNI. 1821 [Ed. Note: in a later version, this memo will include an analysis of 1822 candidate protocols, based upon a set of (basic) requirements, such 1823 as reliable transport mode, preservation of the integrity of the 1824 information conveyed by the protocol, etc.] 1826 C.1. CDNI WG's position on Syslog 1828 [Ed. node: to be written later] 1830 [Ed. note: add a few sentences to clarify why not directly use 1831 syslog... Operational reasons... ] 1833 C.2. CDNI WG's position on SNMP 1835 As explained in [RFC6707], "SNMP traps pose scalability concerns and 1836 SNMP does not support guaranteed delivery of Traps and therefore 1837 could result in log records being lost and the consequent CoDRs and 1838 billing records for that content delivery not being produced as well 1839 as that content delivery being invisible to any analytics platforms." 1841 Authors' Addresses 1843 Gilles Bertrand (editor) 1844 France Telecom - Orange 1845 38-40 rue du General Leclerc 1846 Issy les Moulineaux, 92130 1847 FR 1849 Phone: +33 1 45 29 89 46 1850 Email: gilles.bertrand@orange.com 1851 Stephan Emile 1852 France Telecom - Orange 1853 2 avenue Pierre Marzin 1854 Lannion F-22307 1855 France 1857 Email: emile.stephan@orange.com 1859 Roy Peterkofsky 1860 Skytide, Inc. 1861 One Kaiser Plaza, Suite 785 1862 Oakland CA 94612 1863 USA 1865 Phone: +01 510 250 4284 1866 Email: roy@skytide.com 1868 Francois Le Faucheur 1869 Cisco Systems 1870 Greenside, 400 Avenue de Roumanille 1871 Sophia Antipolis 06410 1872 FR 1874 Phone: +33 4 97 23 26 19 1875 Email: flefauch@cisco.com 1877 Pawel Grochocki 1878 Orange Polska 1879 ul. Obrzezna 7 1880 Warsaw 02-691 1881 Poland 1883 Email: pawel.grochocki@orange.com