idnits 2.17.1 draft-lefaucheur-cdni-logging-delivery-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 13, 2012) is 4303 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'HIGH' is mentioned on line 384, but not defined == Missing Reference: 'MID' is mentioned on line 412, but not defined == Outdated reference: A later version (-02) exists of draft-bertrand-cdni-logging-00 == Outdated reference: A later version (-05) exists of draft-brandenburg-cdni-has-03 == Outdated reference: A later version (-14) exists of draft-ietf-cdni-framework-00 == Outdated reference: A later version (-17) exists of draft-ietf-cdni-requirements-03 == Outdated reference: A later version (-10) exists of draft-ietf-cdni-use-cases-09 Summary: 0 errors (**), 0 flaws (~~), 8 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group F. Le Faucheur 3 Internet-Draft M. Viveganandhan 4 Intended status: Informational K. Leung 5 Expires: January 14, 2013 Cisco 6 July 13, 2012 8 CDNI Logging Formats for HTTP and HTTP Adaptive Streaming Deliveries 9 draft-lefaucheur-cdni-logging-delivery-01 11 Abstract 13 The interconnection of Content Distribution Networks (CDNs) is 14 motivated by several use cases. CDN Interconnection can be achieved 15 through four CDNI interfaces, one of which is the CDNI Logging 16 interface. This document discusses CDNI logging formats for content 17 deliveries performed using HTTP or HTTP adaptive streaming. 19 Requirements Language 21 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 22 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 23 document are to be interpreted as described in RFC 2119 [RFC2119]. 25 Status of this Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at http://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on January 14, 2013. 42 Copyright Notice 44 Copyright (c) 2012 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 60 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4 61 2. CDNI Logging for regular HTTP Delivery . . . . . . . . . . . . 4 62 2.1. Regular HTTP Log Triggers . . . . . . . . . . . . . . . . 4 63 2.2. Regular HTTP Log Fields . . . . . . . . . . . . . . . . . 4 64 3. CDNI Logging for HTTP Adaptive Streaming . . . . . . . . . . . 6 65 3.1. HAS Chunk-Based Log Triggers . . . . . . . . . . . . . . . 6 66 3.2. HAS Chunk-Based Log fields . . . . . . . . . . . . . . . . 6 67 4. Performance Monitoring . . . . . . . . . . . . . . . . . . . . 7 68 5. CDNI Log Encoding and Transport . . . . . . . . . . . . . . . 7 69 5.1. CDNI Log Header Information . . . . . . . . . . . . . . . 8 70 5.2. CDNI per-Log-Record Information . . . . . . . . . . . . . 8 71 5.3. CDNI Log Footer Information . . . . . . . . . . . . . . . 9 72 5.4. CDNI Customized Log Format . . . . . . . . . . . . . . . . 9 73 6. Impact on CDNI Metadata Requirements . . . . . . . . . . . . . 9 74 7. Impact on CDNI Footprint and Capabilities Advertisement . . . 10 75 8. Generation of CDNI Logs . . . . . . . . . . . . . . . . . . . 10 76 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 77 10. Security Considerations . . . . . . . . . . . . . . . . . . . 11 78 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 11 79 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11 80 12.1. Normative References . . . . . . . . . . . . . . . . . . . 11 81 12.2. Informative References . . . . . . . . . . . . . . . . . . 11 82 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12 84 1. Introduction 86 The interconnection of Content Distribution Networks (CDNs) is 87 motivated by several use cases, such as those described in 88 [I-D.ietf-cdni-use-cases]. The overall problem space for CDN 89 Interconnection (CDNI) is described in 90 [I-D.ietf-cdni-problem-statement]. The CDN Interconnection framework 91 is defined in [I-D.ietf-cdni-framework] and the requirements for the 92 CDN Interconnection solution are specified in 93 [I-D.ietf-cdni-requirements]. 95 One of the CDNI interfaces defined in these documents is the CDNI 96 Logging interface whose description is quoted here: 98 "o CDNI Logging interface: This interface allows the Logging system 99 in interconnected CDNs to communicate the relevant activity logs in 100 order to allow log consuming applications to operate in a multi-CDN 101 environments. For example, an upstream CDN may collect delivery logs 102 from a downstream CDN in order to perform consolidated charging of 103 the CSP or for settlement purposes across CDNs. Similarly, an 104 upstream CDN may collect delivery logs from a downstream CDN in order 105 to provide consolidated reporting and monitoring to the CSP." 107 The logging interface is discussed in details in 108 [I-D.bertrand-cdni-logging]. The present document identifies a 109 number of additional considerations regarding logging for content 110 delivery performed using HTTP. 112 [I-D.brandenburg-cdni-has] discusses the interactions of HTTP 113 Adaptive Streaming with CDNI Logging and provides recommendations on 114 how to specifically perform CDNI Logging for delievry performed using 115 HTTP Adaptive Streaming. The present document discusses in more 116 details how these recommendations woudl impact the CDNI Logging 117 interface. 119 Section 2 discusses the aspect of CDNI Logging that are specific to 120 content delivery performed via regular HTTP. Section 3 discusses the 121 aspect of CDNI Logging that are specific to content delivery 122 performed via HTTP adaptive streaming. Section 5 discusses the 123 aspect of CDNI Logging that are generic to all the CDNI Logs for 124 delivery. 126 CDNI Logging for other events than content delivery (e.g. failures, 127 request routing, service monitoring,...) are not discussed in the 128 present document. 130 1.1. Terminology 132 This document uses all terminology defined in 133 [I-D.brandenburg-cdni-has]. 135 2. CDNI Logging for regular HTTP Delivery 137 This section discusses the triggers on which a CDNI log is generated 138 for delivery of content using regular HTTP delivery, as well as the 139 fields contained in the corresponding log. 141 2.1. Regular HTTP Log Triggers 143 A CDNI Log is generated for a regular HTTP delivery on the following 144 triggers: 146 +-----------------+-------------------------------------------------+ 147 | Event | Description | 148 +-----------------+-------------------------------------------------+ 149 | content Request | Reception and processing of a request for a | 150 | | content | 151 +-----------------+-------------------------------------------------+ 153 2.2. Regular HTTP Log Fields 155 A CDNI Log for regular HTTP delivery contains the following fields 156 (or a subset therefof as discused in Section 5.4): 158 +------------------+------------+-----------------------------------+ 159 | Field | Descriptio | Examples | 160 | | n | | 161 +------------------+------------+-----------------------------------+ 162 | Current-Time | Time, in | [20/Feb/2012:00:29.510+0200] | 163 | | millisecon | | 164 | | ds, at | | 165 | | which the | | 166 | | request | | 167 | | was | | 168 | | received. | | 169 | Time-to-Serve | Time, in | 952195 | 170 | | microsecon | | 171 | | ds, taken | | 172 | | to | | 173 | | complete | | 174 | | the | | 175 | | request. | | 176 | Client-IP | IP address | 203.0.113.2 | 177 | | of the | | 178 | | requesting | | 179 | | client. | | 180 | Action | Squid | TCP_HIT, TCP_MISS, ... | 181 | | action | | 182 | | describing | | 183 | | how the | | 184 | | request | | 185 | | was | | 186 | | treated | | 187 | | locally | | 188 | | descriptio | | 189 | | n(e.g., | | 190 | | cache | | 191 | | hit/miss) | | 192 | | . | | 193 | Status-Returned | HTTP | 200, 404, ... | 194 | | response | | 195 | | code. | | 196 | Bytes-Transferre | Bytes sent | 23567992 | 197 | d | to the | | 198 | | client, | | 199 | | including | | 200 | | the | | 201 | | headers. | | 202 | Method | HTTP | GET | 203 | | request | | 204 | | method. | | 205 | URI | URI of | http://cache3.cdn1.com/movie/ice/ | 206 | | requested | icemovie.mpg | 207 | | content. | | 208 | Content-Type | MIME-Type | video/mpeg | 209 | | from the | | 210 | | reply | | 211 | | header. | | 212 | User-Agent (and | content of | Mozilla/5.0 (X11; Linux x86_64; | 213 | possibly some | the | rv:12.0) Gecko/20100101 | 214 | other HTTP | User-Agent | Firefox/12.0 | 215 | headers) | HTTP | | 216 | | Header | | 217 | URI-Signing-Vali | Flag | 0/1 | 218 | dation | indicating | | 219 | | whether | | 220 | | URI | | 221 | | signature | | 222 | | validation | | 223 | | was | | 224 | | perfomed | | 225 +------------------+------------+-----------------------------------+ 227 3. CDNI Logging for HTTP Adaptive Streaming 229 This section discusses the CDNI logs for delivery of content using 230 HTTP adaptive streaming. In line with the recommendations of 231 [I-D.brandenburg-cdni-has], this document proposes that a log record 232 be generated for delivery of each chunk or manifest file. 234 3.1. HAS Chunk-Based Log Triggers 236 A chunk-based Log record is generated for HAS delivery on the 237 following triggers: 239 +--------------------+----------------------------------------------+ 240 | Event | Description | 241 +--------------------+----------------------------------------------+ 242 | Chunk/Manifest | Reception of a request for a Segment (or | 243 | Request | Manifest File) | 244 +--------------------+----------------------------------------------+ 246 3.2. HAS Chunk-Based Log fields 248 A chunk-based Log record for HTTP adaptive streaming may contain all 249 the same fields as a CDNI Log for a regular HTTP request, as well as 250 the following additional fields: 252 +------------+-------------------+----------------------------------+ 253 | Field | Description | Examples | 254 +------------+-------------------+----------------------------------+ 255 | Content | Identifier for | format and scope of unicity are | 256 | Collection | Content | TBD | 257 | ID | Collection | | 258 | Session-id | a string | 6141F5795BE774691D234A0465B9667A | 259 | | generated by the | | 260 | | delivering CDN | | 261 | | and unique (to | | 262 | | the delivering | | 263 | | CDN) to identify | | 264 | | the Session. (*) | | 265 +------------+-------------------+----------------------------------+ 267 (*) The Session-ID value to be included in a log record by the 268 delivering CDN is such that: 270 o different per-chunk log records with the same Session-ID value 271 must correspond to the same user session (.i.e delivery of same 272 content to same enduser at a given point in time) 274 o log records for different chunks of the same user session (.i.e 275 delivery of same content to same enduser at a given point in time) 276 should be provided with the same session-ID value. While 277 undesirable, there may be situations where the delivering CDN uses 278 more than one session-ID value for different per-chunk log records 279 of a given session, for example in scenarios of fail-over or load- 280 balancing across multiple Surrogates and where the delivering CDN 281 does not implement mechanism to synchronize session-IDs across 282 Surrogates. 284 4. Performance Monitoring 286 The CDNI Log fields listed in the previous sections allow monitoring 287 of essential delivery performance indicators across the CDN Mesh. 288 For example, for a regular HTTP delivery, these fields allow tracking 289 of the time taken to serve the requested content, tracking of content 290 delivery failures, tracking of partial deliveries and tracking of 291 cache hit ratios. As another example, for HTTP adaptive streaming, 292 these fields allow tracking of the presentation (and its fluctuation 293 over time) served to the End-User as well as the End-User random 294 content access (e.g. Play/Stop/Seek). 296 Subsequent versions of this document may discuss potential additional 297 log fields for enhanced performance monitoring. 299 5. CDNI Log Encoding and Transport 301 Details for CDNI Log encoding and transport will be specified in 302 subsequent versions. WE observe that this is expected to allow 303 optional use of common compression techniques (e.g. gzip). However, 304 Section 5.1 identifies the information that is to be included in the 305 header of CDNI Logs, Section 5.2 identifies information that is to be 306 attached to every CDNI Log record and Section 5.3 identifies the 307 information that is to be included in the footer of CDNI Logs. 308 Finally, Section 5.4 discusses the notion of customized Logging. 310 5.1. CDNI Log Header Information 312 The header of CDNI Logs contains the following fields: 314 +----------------+-------------------+------------------------------+ 315 | Field | Description | Examples | 316 +----------------+-------------------+------------------------------+ 317 | Format-Version | Version of the | v1.0 | 318 | | CDNI Log format. | | 319 | Log-Field-List | The list of the | time cs-method cs-uri | 320 | | fields provided | | 321 | | in the log | | 322 | | records | | 323 | Log-ID | Unique identifier | | 324 | | for the CDNI Log | | 325 | | (facilitates | | 326 | | detection of | | 327 | | duplicate Logs | | 328 | | and tracking in | | 329 | | case of | | 330 | | aggregation). | | 331 | Log-Timestamp | Time, in | [20/Feb/2012:00:29.510+0200] | 332 | | milliseconds, the | | 333 | | CDNI Log was | | 334 | | generated. | | 335 +----------------+-------------------+------------------------------+ 337 5.2. CDNI per-Log-Record Information 339 In addition to the log fields discussed in previous sections, each 340 CDNI log record contains the following fields: 342 +----------+---------------------------------------------+----------+ 343 | Field | Description | Examples | 344 +----------+---------------------------------------------+----------+ 345 | Log | Digest of the Log records (facilitates | | 346 | Record | recovery of uncorrupted Log records inside | | 347 | Digest | a corrupted CDNI Log) | | 348 +----------+---------------------------------------------+----------+ 350 5.3. CDNI Log Footer Information 352 The footer of CDNI Logs contains the following fields: 354 +---------+----------------------------------------------+----------+ 355 | Field | Description | Examples | 356 +---------+----------------------------------------------+----------+ 357 | Log | Digest of the complete Log (facilitates | | 358 | Digest | detection of Log corruption) | | 359 +---------+----------------------------------------------+----------+ 361 5.4. CDNI Customized Log Format 363 This document proposes that customized logs be supported by CDNI in 364 the following manner: 366 o the uCDN uses the CDNI Metadata interface to indicate to the dCDN 367 which subset of the CDNI logging fields are to be provided in a 368 log record for corresponding to a request for a given content 370 o the dCDN provides, via the CDNI Logging interface, log records 371 containing the subset of CDNI logging fields requested by the 372 uCDN. 374 o The dCDN explicitly lists in the CDNI Log Header the fields 375 actually provided (as discussed in Section 5.1). 377 6. Impact on CDNI Metadata Requirements 379 We request that the following requirements be added in section 6 of 380 [I-D.ietf-cdni-requirements]: 382 " 384 META-X [HIGH] The CDNI Metadata Distribution interface shall support 385 an OPTIONAL mechanism allowing the Upstream CDN to indicate to the 386 Downstream CDN which CDNI Log fields are to be provided for all, for 387 specific sets of, or for specific content items delivered using HTTP. 388 A CDNI implementation that does not support this optional CDNI 389 Metadata Distribution Interface mechanism MUST ignore this log format 390 indication and generate CDNI logging format for adaptive streaming 391 using the default set of CDNI Logging fields. [Editor's note: A 392 default set of logging fields need to be defined] 394 META-X [MID] The CDNI Metadata Distribution interface shall allow the 395 uCDN to signal to the dCDN the Content Collection ID value for all, 396 for specific sets of, or for specific content items delivered using 397 HTTP. Whenever the dCDN is instructed by the uCDN (using the 398 customized logging mechanism described in Section 5.4) to report the 399 Content Collection ID field in the log records, the dCDN is to use 400 the value provided through the CDNI Metadata interface for the 401 corresponding content. 403 " 405 7. Impact on CDNI Footprint and Capabilities Advertisement 407 We request that the following requirement be added in section 5 of 408 [I-D.ietf-cdni-requirements]: 410 " 412 REQ-X [MID] The CDNI Request Routing/Footprint and Advertisement 413 Interface shall support advertisement of the following capabilities: 415 o support for customized CDNI Logging 417 o support of Content Collection ID logging 419 o support for Session-ID logging 421 " 423 8. Generation of CDNI Logs 425 Like other CDNI interfaces, the CDNI Logging interface specifies 426 operations across CDNs and not inside a CDN. Therefore, the log 427 formats specified in this document apply to CDNI logging information 428 exchanged across CDNs and does not constrain the process for 429 generating such inter-CDN logs within a given CDN. The format of the 430 logs generated by a given CDN Surrogate is beyond the scope of the 431 present document. We observe that a given CDN could elect to have 432 its Surrogates natively generate logs in the same format as the one 433 to be used for exchange with another CDN, or that the CDN could elect 434 to have its Surrogates generate logs in any other format (as long as 435 they include the necessary information) and have these logs then 436 reformatted prior to exchange with another CDN. 438 9. IANA Considerations 440 [This will be specified in subsequent versions]. 442 10. Security Considerations 444 CDNI Logs exchanged over the CDNI Logging interface can be consumed 445 by very sensitive applications including inter-CDN accounting and 446 billing. The associated security concerns are discussed in 447 [I-D.ietf-cdni-framework]. At this stage, we observe that the CDNI 448 Logging interface can leverage the existing security mechanisms 449 supported by the underlying transport protocol that will be selected 450 for transport of CDNI Logs (e.g. to support authentication of the 451 entities exchanging CDNI Logs through the CDNI Logging interface, to 452 support privacy and protection against spoofing). This will be 453 further discussed in subsequent versions of this document. 455 11. Acknowledgements 457 The authors want to thank Gene Halbrooks for his input into this 458 document. 460 12. References 462 12.1. Normative References 464 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 465 Requirement Levels", BCP 14, RFC 2119, March 1997. 467 12.2. Informative References 469 [I-D.bertrand-cdni-logging] 470 Bertrand, G. and S. Emile, "CDNI Logging Interface", 471 draft-bertrand-cdni-logging-00 (work in progress), 472 February 2012. 474 [I-D.brandenburg-cdni-has] 475 Brandenburg, R., Deventer, O., Faucheur, F., and K. Leung, 476 "Models for adaptive-streaming-aware CDN Interconnection", 477 draft-brandenburg-cdni-has-03 (work in progress), 478 July 2012. 480 [I-D.ietf-cdni-framework] 481 Peterson, L. and B. Davie, "Framework for CDN 482 Interconnection", draft-ietf-cdni-framework-00 (work in 483 progress), April 2012. 485 [I-D.ietf-cdni-problem-statement] 486 Niven-Jenkins, B., Faucheur, F., and N. Bitar, "Content 487 Distribution Network Interconnection (CDNI) Problem 488 Statement", draft-ietf-cdni-problem-statement-08 (work in 489 progress), June 2012. 491 [I-D.ietf-cdni-requirements] 492 Leung, K. and Y. Lee, "Content Distribution Network 493 Interconnection (CDNI) Requirements", 494 draft-ietf-cdni-requirements-03 (work in progress), 495 June 2012. 497 [I-D.ietf-cdni-use-cases] 498 Bertrand, G., Emile, S., Burbridge, T., Eardley, P., Ma, 499 K., and G. Watson, "Use Cases for Content Delivery Network 500 Interconnection", draft-ietf-cdni-use-cases-09 (work in 501 progress), July 2012. 503 Authors' Addresses 505 Francois Le Faucheur 506 Cisco Systems 507 Greenside, 400 Avenue de Roumanille 508 Sophia Antipolis 06410 509 France 511 Phone: +33 4 97 23 26 19 512 Email: flefauch@cisco.com 514 Mahesh Viveganandhan 515 Cisco Systems 516 375 East Tasman Drive 517 San Jose 95134 518 USA 520 Email: mvittal@cisco.com 522 Kent Leung 523 Cisco Systems 524 3625 Cisco Way 525 San Jose 95134 526 USA 528 Phone: +1 408 526 5030 529 Email: kleung@cisco.com