idnits 2.17.1 draft-ietf-decade-arch-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Sep 2009 rather than the newer Notice from 28 Dec 2009. (See https://trustee.ietf.org/license-info/) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 23 instances of too long lines in the document, the longest one being 3 characters in excess of 72. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 749: '...reaming session, MAY employ several DE...' RFC 2119 keyword, line 787: '... A server MUST accept download reque...' RFC 2119 keyword, line 790: '...nates the objects MUST generate DECADE...' RFC 2119 keyword, line 794: '... a DECADE server MUST not change the n...' RFC 2119 keyword, line 796: '... A DECADE server MAY verify the integr...' (4 more instances...) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: The application that originates the objects MUST generate DECADE object names according to the naming specification in Section 4.4. The naming scheme provides that the name is unique. DECADE clients (as parts of application entities) upload a named object to a server, and a DECADE server MUST not change the name. It MUST be possible for downloading clients, to access the object using the original name. A DECADE server MAY verify the integrity and other security properties of uploaded objects. -- The document date (March 7, 2011) is 4792 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-06) exists of draft-ietf-decade-problem-statement-02 == Outdated reference: A later version (-06) exists of draft-ietf-decade-survey-04 Summary: 3 errors (**), 0 flaws (~~), 4 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DECADE R. Alimi 3 Internet-Draft Google 4 Intended status: Standards Track Y. Yang 5 Expires: September 8, 2011 Yale University 6 A. Rahman 7 InterDigital Communications, LLC 8 D. Kutscher 9 NEC 10 H. Liu 11 Yale University 12 March 7, 2011 14 DECADE Architecture 15 draft-ietf-decade-arch-00 17 Abstract 19 Peer-to-peer (P2P) applications have become widely used on the 20 Internet today and make up a large portion of the traffic in many 21 networks. One technique to improve the network efficiency of P2P 22 applications is to introduce storage capabilities within the 23 networks. The DECADE Working Group has been formed with the goal of 24 developing an architecture to provide this capability. This document 25 presents an architecture, discusses the underlying principles, and 26 identifies core components and protocols supporting the architecture. 28 Status of this Memo 30 This Internet-Draft is submitted to IETF in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF), its areas, and its working groups. Note that 35 other groups may also distribute working documents as Internet- 36 Drafts. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 The list of current Internet-Drafts can be accessed at 44 http://www.ietf.org/ietf/1id-abstracts.txt. 46 The list of Internet-Draft Shadow Directories can be accessed at 47 http://www.ietf.org/shadow.html. 49 This Internet-Draft will expire on September 8, 2011. 51 Copyright Notice 53 Copyright (c) 2011 IETF Trust and the persons identified as the 54 document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's Legal 57 Provisions Relating to IETF Documents 58 (http://trustee.ietf.org/license-info) in effect on the date of 59 publication of this document. Please review these documents 60 carefully, as they describe your rights and restrictions with respect 61 to this document. Code Components extracted from this document must 62 include Simplified BSD License text as described in Section 4.e of 63 the Trust Legal Provisions and are provided without warranty as 64 described in the BSD License. 66 Table of Contents 68 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 69 2. Entities . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 70 2.1. DECADE Storage Servers . . . . . . . . . . . . . . . . . . 6 71 2.2. DECADE Storage Provider . . . . . . . . . . . . . . . . . 6 72 2.3. DECADE Content Providers . . . . . . . . . . . . . . . . . 6 73 2.4. DECADE Content Consumers . . . . . . . . . . . . . . . . . 6 74 2.5. Content Distribution Application . . . . . . . . . . . . . 6 75 2.6. Application End-Point . . . . . . . . . . . . . . . . . . 7 76 3. Architectural Principles . . . . . . . . . . . . . . . . . . . 7 77 3.1. Decoupled Control and Data Planes . . . . . . . . . . . . 7 78 3.2. Immutable Data Objects . . . . . . . . . . . . . . . . . . 8 79 3.3. Data Object Identifiers . . . . . . . . . . . . . . . . . 9 80 3.4. Explicit Control . . . . . . . . . . . . . . . . . . . . . 9 81 3.5. Resource and Data Access Control through User 82 Delegation . . . . . . . . . . . . . . . . . . . . . . . . 10 83 3.5.1. Resource Allocation . . . . . . . . . . . . . . . . . 10 84 3.5.2. User Delegations . . . . . . . . . . . . . . . . . . . 10 85 4. System Components . . . . . . . . . . . . . . . . . . . . . . 11 86 4.1. Content Distribution Application . . . . . . . . . . . . . 13 87 4.1.1. Data Sequencing and Naming . . . . . . . . . . . . . . 13 88 4.1.2. Native Protocols . . . . . . . . . . . . . . . . . . . 13 89 4.1.3. DECADE Client . . . . . . . . . . . . . . . . . . . . 14 90 4.2. DECADE Server . . . . . . . . . . . . . . . . . . . . . . 14 91 4.2.1. Access Control . . . . . . . . . . . . . . . . . . . . 14 92 4.2.2. Resource Scheduling . . . . . . . . . . . . . . . . . 15 93 4.2.3. Data Store . . . . . . . . . . . . . . . . . . . . . . 15 94 4.3. Protocols . . . . . . . . . . . . . . . . . . . . . . . . 15 95 4.3.1. DECADE Resource Protocol . . . . . . . . . . . . . . . 15 96 4.3.2. Standard Data Transports . . . . . . . . . . . . . . . 16 97 4.4. DECADE Data Sequencing and Naming . . . . . . . . . . . . 16 98 4.5. In-Network Storage Components Mapped to DECADE 99 Architecture . . . . . . . . . . . . . . . . . . . . . . . 17 100 4.5.1. Data Access Interface . . . . . . . . . . . . . . . . 17 101 4.5.2. Data Management Operations . . . . . . . . . . . . . . 17 102 4.5.3. Data Search Capability . . . . . . . . . . . . . . . . 17 103 4.5.4. Access Control Authorization . . . . . . . . . . . . . 17 104 4.5.5. Resource Control Interface . . . . . . . . . . . . . . 17 105 4.5.6. Discovery Mechanism . . . . . . . . . . . . . . . . . 18 106 4.5.7. Storage Mode . . . . . . . . . . . . . . . . . . . . . 18 107 5. DECADE Protocols . . . . . . . . . . . . . . . . . . . . . . . 18 108 5.1. DECADE Resource Protocol (DRP) . . . . . . . . . . . . . . 18 109 5.2. Standard Data Transport (SDT) . . . . . . . . . . . . . . 19 110 5.2.1. Writing/Uploading Objects . . . . . . . . . . . . . . 19 111 5.2.2. Downloading Objects . . . . . . . . . . . . . . . . . 20 112 6. Server-to-Server Protocols . . . . . . . . . . . . . . . . . . 21 113 6.1. Operational Overview . . . . . . . . . . . . . . . . . . . 21 114 6.2. Potential Optimizations . . . . . . . . . . . . . . . . . 22 115 6.2.1. Pipelining to Avoid Store-and-Forward Delays . . . . . 22 116 7. Security Considerations . . . . . . . . . . . . . . . . . . . 23 117 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 118 9. Informative References . . . . . . . . . . . . . . . . . . . . 23 119 Appendix A. Appendix: Evaluation of Candidate Existing 120 Protocols for DECADE DRP and SDT . . . . . . . . . . 23 121 A.1. HTTP . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 122 A.1.1. HTTP Support for DECADE Resource Protocol 123 Primitives . . . . . . . . . . . . . . . . . . . . . . 24 124 A.1.2. HTTP Support for DECADE Standard Transport 125 Protocol Primitives . . . . . . . . . . . . . . . . . 24 126 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 25 128 1. Introduction 130 Peer-to-peer (P2P) applications have become widely used on the 131 Internet today to distribute contents, and they contribute a large 132 portion of the traffic in many networks. The DECADE Working Group 133 has been formed with the goal of developing an architecture to 134 introduce in-network storage to be used by such applications, to 135 achieve more efficient content distribution. Specifically, in many 136 subscriber networks, it is typically more expensive to upgrade 137 network equipment in the "last-mile", because it can involve 138 replacing equipment and upgrading wiring at individual homes, 139 businesses, and devices such as DSLAMs and CMTSs. On the other hand, 140 it can be cheaper to upgrade the core infrastructure, which involves 141 fewer components that are shared by many subscribers. See 142 [I-D.ietf-decade-problem-statement] for a more complete discussion of 143 the problem domain and general discussions of the capabilities to be 144 provided by DECADE. 146 This document presents a potential architecture of providing in- 147 network storage that can be integrated into content distribution 148 applications. The primary focus is P2P-based content distribution, 149 but the architecture may be useful to other applications with similar 150 characteristics and requirements. In particular, content 151 distribution applications that may split data into smaller pieces for 152 distribution may be able to utilize DECADE. 154 The design philosophy of the DECADE architecture is to provide only 155 the core functionalities that are needed for applications to make use 156 of in-network storage. With such core functionalities, the protocol 157 may be simpler and easier to support by storage providers. If more 158 complex functionalities are needed by a certain application or class 159 of applications, it may be layered on top of the DECADE protocol. 161 The DECADE protocol will leverage existing transport and application 162 layer protocols and will be designed to work with a small set of 163 alternative IETF protocols. 165 This document proceeds in two steps. First, it details the core 166 architectural principles that can guide the DECADE design. Next, 167 given these core principles, this document presents the core 168 components of the DECADE architecture and identifies usage of 169 existing protocols and where there is a need for new protocol 170 development. 172 This document will be updated to track the progress of the DECADE 173 survey [I-D.ietf-decade-survey] and requirements [I-D.gu-decade-reqs] 174 drafts. 176 2. Entities 178 2.1. DECADE Storage Servers 180 DECADE storage servers are operated by DECADE storage providers and 181 provide the DECADE functionality as specified in this document, 182 including mechanisms to store, retrieve and manage data. A storage 183 provider may typically operate multiple storage servers. 185 2.2. DECADE Storage Provider 187 A DECADE in-network storage provider deploys and/or manages DECADE 188 servers within a network. A storage provider may also own or manage 189 the network in which the DECADE servers are deployed. 191 A DECADE storage provider, possibly in cooperation with one or more 192 network providers, determines deployment locations for DECADE servers 193 and determines the available resources for each. 195 2.3. DECADE Content Providers 197 DECADE content providers access DECADE storage servers (by way of a 198 DECADE client) to upload and manage data. A content provider can 199 access one or more storage servers. A content provider may be a 200 single process or a distributed application (e.g., in a P2P 201 scenario). 203 2.4. DECADE Content Consumers 205 DECADE content consumers access storage servers (by way of a DECADE 206 client) to download data that has previously been stored by a content 207 provider. A content consumer can access one or more storage servers. 208 A content consumer may be a single process or a distributed 209 application (e.g., in a P2P scenario). An instance of a distributed 210 application, such as a P2P application, may both provide content to 211 and consume content from DECADE storage servers. 213 2.5. Content Distribution Application 215 A content distribution application is a distributed application 216 designed for dissemination of possibly-large data to multiple 217 consumers. Content Distribution Applications typically divide 218 content into smaller immutable blocks for dissemination. 220 The term Application Developer refers to the developer of a 221 particular Content Distribution Application. 223 2.6. Application End-Point 225 An Application End-Point is an instance of a Content Distribution 226 Application that makes use of DECADE server(s). A particular 227 Application End-Point may be a DECADE Content Provider, a DECADE 228 Content Consumer, or both. 230 An Application End-Point need not be an active member of a "swarm" to 231 interact with the DECADE storage system. That is, an End-Point may 232 interact with the DECADE storage servers as an offline activity. 234 3. Architectural Principles 236 We identify the following key principles. 238 3.1. Decoupled Control and Data Planes 240 The DECADE infrastructure is intended to support multiple content 241 distribution applications. A complete content distribution 242 application implements a set of control functions including content 243 search, indexing and collection, access control, ad insertion, 244 replication, request routing, and QoS scheduling. Different content 245 distribution applications can have unique considerations designing 246 the control and signaling functions. For example, a major 247 competitive advantage of many successful P2P systems is their 248 substantial expertise in achieving highly efficient utilization of 249 peer and infrastructural resources. For instance, many live P2P 250 systems have their specific algorithms in constructing topologies to 251 achieve low-latency, high-bandwidth streaming. They continue to 252 fine-tune such algorithms. In other words, in-network storage should 253 export basic mechanisms and allow as much flexibility as possible to 254 the control planes to implement specific policies. This conforms to 255 the end-to-end systems principle and allows innovation and 256 satisfaction of specific business goals. 258 Specifically, in the DECADE architecture, the control plane focuses 259 on the application-specific, complex, and/or processing intensive 260 functions while the data plane provides storage and data transport 261 functions. 263 o Control plane: Signals details of where the data is to be 264 downloaded from. The control signals may also include the time, 265 quality of service, and receivers of the download. It also 266 provides higher layer meta-data management functions such as 267 defining the sequence of data blocks forming a higher layer 268 content object. These are behaviors designed and implemented by 269 the Application. By Application, we mean the broad sense that 270 includes other control plane protocols. 272 o Data plane: Stores and transfers data as instructed by the 273 Application's Control Plane. 275 Decoupling control plane and data plane is not new. For example, 276 OpenFlow is an implementation of this principle for Internet routing, 277 where the computation of the forwarding table and the application of 278 the forwarding table are separated. Google File System applies the 279 principle to file system design, by utilizing the Master to handle 280 the meta-data management, and the Chunk Servers to handle the data 281 plane functions (i.e., read and write of chunks of data). NFS4 also 282 implements this principle. 284 Note that applications may have different Data Plane implementations 285 in order to support particular requirements (e.g., low latency). In 286 order to provide interoperability, the DECADE architecture does not 287 intend to enable arbitrary data transport protocols. However, the 288 architecture may allow for more-than-one data transport protocols to 289 be used. 291 Also note that although an application's existing control plane 292 functions remain implemented within the application, the particular 293 implementation may need to be adjusted to support DECADE. 295 3.2. Immutable Data Objects 297 A property of bulk contents to be broadly distributed is that they 298 typically are immutable -- once a piece of content is generated, it 299 is typically not modified. It is not common that bulk contents such 300 as video frames and images need to be modified after distribution. 302 Many content distribution applications divide content objects into 303 blocks for two reasons: (1) multipath: different blocks may be 304 fetched from different content sources in parallel, and (2) faster 305 recovery and verification: individual blocks may be recovered and 306 verified. Typically, applications use a block size larger than a 307 single packet in order to reduce control overhead. 309 Common applications whose content matches this model include P2P 310 streaming (live and video-on-demand) and P2P file-sharing content. 311 However, other additional types of applications may match this model. 313 DECADE adopts a design in which immutable data objects may be stored 314 at a storage server. Applications may consider existing blocks as 315 DECADE data objects, or they may adjust block sizes before storing in 316 a DECADE server. 318 Focusing on immutable data blocks in the data plane can substantially 319 simplify the data plane design, since consistency requirements can be 320 relaxed. It also allows effective reuse of data blocks and de- 321 duplication of redundant data. 323 Depending on its specific requirements, an application may store data 324 in DECADE servers such that each data object is completely self- 325 contained (e.g., a complete, independently decodable video segment). 326 An application may also divide data into chunks that require 327 application level assembly. The DECADE architecture and protocols 328 are agnostic to the nature of the data objects and do not specify a 329 fixed size for them. 331 Note that immutable content may still be deleted. Also note that 332 immutable data blocks do not imply that contents cannot be modified. 333 For example, a meta-data management function of the control plane may 334 associate a name with a sequence of immutable blocks. If one of the 335 blocks is modified, the meta-data management function changes the 336 mapping of the name to a new sequence of immutable blocks. 338 3.3. Data Object Identifiers 340 Objects that are stored in a DECADE storage server can be accessed by 341 DECADE content consumers by a resource identifier that has been 342 assigned within a certain application context. 344 Because a DECADE content consumer can access more than one storage 345 server within a single application context, a data object that is 346 replicated across different storage servers managed by a DECADE 347 storage provider, can be accessed by a single identifier. 349 Note that since data objects are immutable, it is possible to support 350 persistent identifiers for data objects. 352 3.4. Explicit Control 354 To support the functions of an application's control plane, 355 applications must be able to know and control which data is stored at 356 particular locations. Thus, in contrast with content caches, 357 applications are given explicit control over the placement (selection 358 of a DECADE server), deletion (or expiration policy), and access 359 control for stored data. 361 Consider deletion/expiration policy as a simple example. An 362 application may require that a DECADE server store content for a 363 relatively short period of time (e.g., for live-streaming data). 364 Another application may need to store content for a longer duration 365 (e.g., for video-on-demand). 367 3.5. Resource and Data Access Control through User Delegation 369 DECADE provides a shared infrastructure to be used by multiple 370 tenants of multiple content distribution applications. Thus, it 371 needs to provide both resource and data access control. 373 3.5.1. Resource Allocation 375 There are two primary interacting entities in the DECADE 376 architecture. First, Storage Providers control where DECADE storage 377 servers are provisioned and their total available resources. Second, 378 Applications control data transfers amongst available DECADE servers 379 and between DECADE servers and end-points. A form of isolation is 380 required to enable concurrently-running Applications to each 381 explicitly manage their own content and share of resources at the 382 available servers. 384 The Storage Provider delegates the management of the resources at a 385 DECADE server to one or more applications. Applications are able to 386 explicitly and independently manage their own shares of resources. 388 3.5.2. User Delegations 390 Storage providers have the ability to explicitly manage the entities 391 allowed to utilize the resources at a DECADE server. This capability 392 is needed for reasons such as capacity-planning and legal 393 considerations in certain deployment scenarios. 395 To provide a scalable way to manage applications granted resources at 396 a DECADE server, we consider an architecture that adds a layer of 397 indirection. Instead of granting resources to an application, the 398 DECADE server grants a share of the resources to a user. The user 399 may in turn share the granted resources amongst multiple 400 applications. The share of resources granted by a storage provider 401 is called a User Delegation. 403 A User Delegation may be granted to an end-user (e.g., an ISP 404 subscriber), a Content Provider, or an Application Provider. A 405 particular instance of an application may make use of the storage 406 resources: 408 o granted to the end-user (with the end-user's permission), 410 o granted to the Content Provider (with the Content Provider's 411 permission), and/or 413 o granted to the Application Provider. 415 4. System Components 417 The primary focus of the current version of this document is on the 418 architectural principles. The detailed system components will be 419 discussed in the next document revision. 421 This section presents an overview of the components in the DECADE 422 architecture. 424 .--------------------------------------------------------------. 425 | Application End-Point | 426 | .------------. .-----------------. .----------. | 427 | | App-Layer | ... | App Data Object | | App Data | | 428 | | Algorithms | | Sequencing | | Naming | | 429 | `------------' `-----------------' `----------' | 430 | | 431 | .----------------------------------------------------------. | 432 | | DECADE Client | | 433 | | | | 434 | | .-------------------------. .--------------------------. | | 435 | | | Resource Controller | | Data Controller | | | 436 | | | .--------. .----------. | | .------------. .-------. | | | 437 Native | | | | Data | | Resource | | | | Data | | Data | | | | 438 App | | | | Access | | Sharing | | | | Scheduling | | Index | | | | 439 Protocol(s)| | | | Policy | | Policy | | | | | | | | | | 440 .--> | | | '--------' `----------' | | `------------' `-------' | | | 441 | | | `-------------------------' `--------------------------' | | 442 | | | | ^ | | 443 | | `------------ | ----------------- | -----------------------' | 444 | `-------------- | ----------------- | -------------------------' 445 | | | 446 v | DECADE | Standard 447 .-------------. | Resource | Data 448 | Application | | Protocol (DRP) | Transport (SDT) 449 | End-Point | | | 450 `-------------' | | Content Distribution 451 ^ ^ | | Application 452 = | ===== | ============== | ================= | ========================== 453 | | | | DECADE Server(s) 454 | | | | 455 | | .- | ----------------- | ----------------------. 456 | | | | v | 457 | | | | .----------------. | 458 | | | |----> | Access Control | <--------. | 459 | DRP | SDT | | `----------------' | | 460 | | | | ^ | | 461 | | | | v | | 462 | | | | .---------------------. | | 463 | | | `-> | Resource Scheduling | <------| | 464 v v DRP | `---------------------' | | 465 .------------. <------> | ^ | | 466 | DECADE | | v .------------. | 467 | Server | SDT | .-----------------. | User | | 468 `------------' <------> | | Data Store | | Delegation | | 469 | `-----------------' | Management | | 470 | DECADE Server `------------' | 471 `----------------------------------------------' 473 Figure 1: DECADE Architecture Components 475 A component diagram of the DECADE architecture is displayed in 476 Figure 1. The diagram illustrates the major components of a Content 477 Distribution Application related to DECADE, as well as the functional 478 components of a DECADE Server. 480 To keep the scope narrow, we only discuss the primary components 481 related to protocol development. Particular deployments may require 482 additional components (e.g., monitoring and accounting at a DECADE 483 server), but they are intentionally omitted from the current version 484 of this document. 486 4.1. Content Distribution Application 488 Content Distribution Applications have many functional components. 489 For example, many P2P applications have components to manage overlay 490 topology management, piece selection, etc. In supporting DECADE, it 491 may be advantageous to consider DECADE within some of these 492 components. However, in this architecture document, we focus on the 493 components directly employed to support DECADE. 495 4.1.1. Data Sequencing and Naming 497 DECADE is primarily designed to support applications that can divide 498 distributed contents into immutable data objects. To accomplish 499 this, applications include a component responsible for re-assembling 500 data objects and also creating the individual data objects. We call 501 this component Application Data Sequencing. The specific 502 implementation is entirely decided by the application. 504 In assembling or producing the data objects, an important 505 consideration is the naming of these objects. We call the component 506 responsible for assigning and interpreting application-layer names 507 the Application Data Naming component. The specific implementation 508 is entirely decided by the application. 510 4.1.2. Native Protocols 512 Applications may still use existing protocols. In particular, an 513 application may reuse existing protocols primarily for control/ 514 signaling. However, an application may still retain its existing 515 data transport protocols, in addition to DECADE as the data transport 516 protocol. This can be important for applications that are designed 517 to be highly robust (e.g., if DECADE servers are unavailable). 519 4.1.3. DECADE Client 521 An application may be modified to support DECADE. We call the layer 522 providing the DECADE support to an application the DECADE Client. It 523 is important to note that a DECADE Client need not be embedded into 524 an application. It could be implemented alone, or could be 525 integrated in other entities such as network devices themselves. 527 4.1.3.1. Resource Controller 529 Applications may have different Resource sharing policies and Data 530 access policies to control their resource and data in DECADE servers. 531 These policies can be existing policies of applications (e.g., tit- 532 for-tat) or custom policies adapted for DECADE. The specific 533 implementation is decided by the application. 535 4.1.3.2. Data Controller 537 DECADE is designed to decouple the control and the data transport of 538 applications. Data transport between applications and DECADE servers 539 uses standard data transport protocols. It may need to schedule the 540 data being transferred according to network conditions, available 541 DECADE Servers, and/or available DECADE Server resources. An index 542 indicates data available at remote DECADE servers. The index (or a 543 subset of it) may be advertised to other Application End-Points. 545 4.2. DECADE Server 547 DECADE server is an important functional component of DECADE. It 548 stores data from Application End-Points, and provides control and 549 access of those data to Application End-Points. Note that a DECADE 550 server is not necessarily a single physical machine, it could also be 551 implemented as a cluster of machines. 553 4.2.1. Access Control 555 An Application End-Point can access its own data or other Application 556 End-Point's data (provided sufficient authorization) in DECADE 557 servers. Application End-Points may also authorize other End-Points 558 to store data. If an access is authorized by an Application End- 559 Point, the DECADE Server will provide access. 561 Note that even if a request is authorized, it may still fail to 562 complete due to insufficient resources by either the requesting 563 Application End-Point or the providing Application End-Point. 565 4.2.2. Resource Scheduling 567 Applications may apply their existing resource sharing policies or 568 use a custom policy for DECADE. DECADE servers perform resource 569 scheduling according to the resource sharing policies indicated by 570 Application End-Points as well as configured User Delegations. 572 Access control and resource control are separated in DECADE server. 573 It is possible that an Application End-Point provides only access to 574 its data without any resources. In order to access this data, 575 another Application End-Point may use the granted access along with 576 its own available resources to store or retrieve data from a DECADE 577 Server. 579 4.2.3. Data Store 581 Data from applications may be stored into disks. Data can be deleted 582 from disks either explicitly or automatically (e.g., after a TTL). 583 It may be possible to perform optimizations in certain cases, such as 584 avoiding writing temporary data (e.g., live streaming) to a disk. 586 4.3. Protocols 588 The DECADE Architecture uses two protocols. First, the DECADE 589 Resource Protocol is responsible for communicating access control and 590 resource scheduling policies to the DECADE Server. Second, standard 591 data transport protocols (e.g., WebDAV or NFS) are used to transfer 592 data objects to and from a DECADE Server. The DECADE architecture 593 will specify a small number of Standard Data Transport instances. 595 Decoupling the protocols in this way allows DECADE to both directly 596 utilize existing standard data transports and to evolve 597 independently. 599 It is also important to note that the two protocols do not need to be 600 separate on the wire. For example, the DECADE Resource Protocol 601 messages may be piggybacked within the extension fields provided by 602 certain data transport protocols. However, this document considers 603 them as two separate, logical functional components for clarity. 605 4.3.1. DECADE Resource Protocol 607 The DECADE Resource Protocol is responsible for communicating both 608 access control and resource sharing policies to DECADE Servers used 609 for data transport. 611 The DECADE architecture specification will provide exactly one DECADE 612 Resource Protocol. 614 4.3.2. Standard Data Transports 616 Existing data transport protocols are used to read and write data 617 from a DECADE Server. Protocols under consideration are WebDAV and 618 NFS. 620 4.4. DECADE Data Sequencing and Naming 622 We have discussed above that an Application may have its own behavior 623 for both sequencing and naming data objects. In order to provide a 624 simple and generic interface, the DECADE Server is only responsible 625 for storing and retrieving individual data objects. 627 DECADE names are not necessarily correlated with the naming or 628 sequencing used by the Application using a DECADE client. The DECADE 629 client is expected to maintain a mapping from its own naming to the 630 DECADE naming. Furthermore, the DECADE naming scheme implies no 631 sequencing or grouping of objects, even if this is done at the 632 application layer. 634 Multiple applications may make use of a DECADE infrastructure, and 635 each Application may employ its own naming scheme. To remain 636 independent of particular applications, DECADE uses a simple, common 637 naming scheme that supports unique naming of individual data objects. 638 This is achieved by deriving object names from hashes of the object 639 content. This scheme is made possible by the fact that DECADE data 640 objects are immutable. Details of the naming scheme (complete 641 syntax, hash algorithms etc.) will be defined in a future version of 642 this document. 644 By naming data objects directly as the content hash, DECADE names 645 satisfy important objectives: 647 o Simple integrity verification 649 o Unique names (with high probability) 651 o Application independent, without a new IANA-maintained registry 653 A particular advantage of using the content hash is that it is 654 straightforward for as server or client to validate a data object 655 before storing or transmitting it. While these capabilities could be 656 achieved by supplying the content hash in both read and write 657 requests as metadata, using the content hash as the name satisfies 658 the objectives and is straightforward. 660 Another advantage of this scheme is that a DECADE client knows the 661 name of a data object before it is completely stored at the DECADE 662 server. This allows for particular optimizations, such as 663 advertising data object while the data object is being stored, 664 removing store-and-forward delays. For example, a DECADE client A 665 may simultaneously begin storing an object to a DECADE server, and 666 advertise that the object is available to DECADE client B. If it is 667 supported by the DECADE server, client B may begin downloading the 668 object before A is finished storing the object. 670 4.5. In-Network Storage Components Mapped to DECADE Architecture 672 In this section we evaluate how the basic components of an in-network 673 storage system identified in Section 3 of [I-D.ietf-decade-survey] 674 map into the DECADE architecture. 676 It is important to note that complex and/or application-specific 677 behavior is delegated to applications instead of tuning the storage 678 system wherever possible. 680 4.5.1. Data Access Interface 682 Users can read and write objects of arbitrary size through the DECADE 683 Client's Data Controller, making use of a standard data transport. 685 4.5.2. Data Management Operations 687 Users can move or delete previously stored objects via the DECADE 688 Client's Data Controller, making use of a standard data transport. 690 4.5.3. Data Search Capability 692 Users can enumerate or search contents of DECADE servers to find 693 objects matching desired criteria through services provided by the 694 Content Distribution Application (e.g., buffer-map exchanges, a DHT, 695 or peer-exchange). In doing so, End-Points may consult their local 696 data index in the DECADE Client's Data Controller. 698 4.5.4. Access Control Authorization 700 All methods of access control are supported: public-unrestricted, 701 public-restricted and private. Access Control Policies are generated 702 by a Content Distribution Application and provided to the DECADE 703 Client's Resource Controller. The DECADE Server is responsible for 704 implementing the access control checks. 706 4.5.5. Resource Control Interface 708 Users can manage the resources (e.g. bandwidth) on the DECADE server 709 that can be used by other Application End-Points. Resource Sharing 710 Policies are generated by a Content Distribution Application and 711 provided to the DECADE Client's Resource Controller. The DECADE 712 Server is responsible for implementing the resource sharing policies. 714 4.5.6. Discovery Mechanism 716 This is outside the scope of the DECADE architecture. However, it is 717 expected that DNS or some other well known protocol will be used for 718 the users to discover the DECADE servers. 720 4.5.7. Storage Mode 722 DECADE Servers provide an object-based storage mode. Immutable data 723 objects may be stored at a DECADE server. Applications may consider 724 existing blocks as DECADE data objects, or they may adjust block 725 sizes before storing in a DECADE server. 727 5. DECADE Protocols 729 This section specifies the DECADE Resource Protocol (DRP) and the 730 Standard Data Transport (SDT) in terms of abstract protocol 731 interactions that are intended to mapped to specific protocols such 732 as HTTP. It is possible that a single specific protocol provides 733 both, DRP and SDT functionality. 735 The DRP is the protocol used by a DECADE client to configure the 736 resources and authorization used to satisfy requests (reading, 737 writing, and management operations concerning DECADE objects) at a 738 DECADE server. The SDT is used to send the operations to the DECADE 739 server. Necessary DRP metadata is supplied using mechanisms in the 740 SDT that are provided for extensibility (e.g., additional request 741 parameters). A detailed architectural description of the DRP and SDT 742 is still a work in progress. Important aspects, including details of 743 authorization, will be added in a future version of this document. 745 5.1. DECADE Resource Protocol (DRP) 747 DRP provides configuration of access control and resource sharing 748 policies on DECADE servers. A content distribution application, 749 e.g., a live P2P streaming session, MAY employ several DECADE 750 servers, for instance, servers in different operator domains, and DRP 751 allows one instance of such an application, e.g., an application 752 endpoint, to configure access control and resource sharing policies 753 on a set of servers. 755 On a single DECADE server, the following resources have to be 756 managed: 758 communication resources: DECADE servers may limited communication 759 resources in terms of bandwidth (upload/download) but also in 760 terms of number of connected clients (connections) at a time. 762 storage resources: DECADE servers may limited storage resources. 764 Note: this list of resources will be extended in a future version of 765 this document. 767 5.2. Standard Data Transport (SDT) 769 A DECADE server provide a data access interface, and SDT is used to 770 write objects to a server and to read (download) objects from a 771 server. Semantically, SDT is a client-server protocol, i.e., the 772 DECADE server always responds to client requests. 774 5.2.1. Writing/Uploading Objects 776 For writing objects, a client uploads an object to a DECADE server. 777 The object on the server will be named (associated to an identifier), 778 and this name can be used to access (download) the object later, 779 e.g., the client can pass the name as a reference to other client 780 that can then refer to the object. 782 DECADE objects can be self-contained objects such as multimedia 783 resources, files etc., but also chunks, such as chunks of a P2P 784 distribution protocol that can be part of a containing object or a 785 stream. 787 A server MUST accept download requests for an object that is still 788 being uploaded. 790 The application that originates the objects MUST generate DECADE 791 object names according to the naming specification in Section 4.4. 792 The naming scheme provides that the name is unique. DECADE clients 793 (as parts of application entities) upload a named object to a server, 794 and a DECADE server MUST not change the name. It MUST be possible 795 for downloading clients, to access the object using the original 796 name. A DECADE server MAY verify the integrity and other security 797 properties of uploaded objects. 799 In the following we provide an abstract specification of the upload 800 operation that we name 'PUT METHOD'. See Appendix A.1 for an example 801 how this could be mapped to HTTP. 803 Method PUT: 805 Parameters: 807 NAME: The naming of the object according to Section 4.4 809 OBJECT: The object itself. The protocol MUST provide transparent 810 binary object transport. 812 Description: The PUT method is used by a DECADE client to upload an 813 object with an associated name 'NAME' to a DECADE server. 815 RESPONSES: The DECADE server MUST respond with one the following 816 response messages: 818 OK: The object has been uploaded successfully and has replaced an 819 existing object with the same name. 821 CREATED: The object has been uploaded successfully and is now 822 available under the specified name. 824 ERRORs: possible error codes later will be specified in a later 825 version of this document 827 5.2.2. Downloading Objects 829 A DECADE client can request named objects from a DECADE server. In 830 the following, we provide an abstract specification of the download 831 operation that we name 'GET METHOD'. See Section 4.4 for an example 832 how this could be mapped to HTTP. 834 Method GET: 836 Parameters: 838 NAME: The naming of the object according to Section 4.4. 840 Description: The GET method is used by a DECADE client to download 841 an object with an associated name 'NAME' from a DECADE server. 843 RESPONSES: The DECADE server MUST respond with one the following 844 response messages: 846 OK: The request has succeeded, and an entity corresponding to the 847 requested resource is sent in the response. 849 ERRORs: 851 NOTFOUND: The DECADE server has not found anything matching 852 the request object name. 854 Other Errors: TBD in a future version of this document 856 6. Server-to-Server Protocols 858 An important feature of DECADE is the capability for one DECADE 859 server to directly download data objects from another DECADE server. 860 This capability allows Applications to directly replicate data 861 objects between servers without requiring end-hosts to use uplink 862 capacity to upload data objects to a different DECADE server. 863 Similar to other operations in DRP and SDT, replicating data objects 864 between DECADE servers is an explicit operation. 866 To support this functionality, DECADE re-uses the already-specified 867 protocols to support operations directly between servers. DECADE 868 servers are not assumed to trust each other nor are configured to do 869 so. All data operations are performed on behalf of DECADE clients 870 via explicit instruction, so additional capabilities are needed in 871 the DECADE client-server protocols DECADE clients must be able to 872 indicate to a DECADE server the following additional parameters: 874 o which remote DECADE server(s) to access; 876 o the operation to be performed (PUT or GET); and 878 o Credentials indicating permission to perform the operation at the 879 remote DECADE server. 881 In this way, a DECADE server is also a DECADE client, and requests 882 may instantiate requests via that client. The operations are 883 performed as if the original requestor had its own DECADE client co- 884 located with the DECADE server. It is this mode of operation that 885 provides substantial savings in uplink capacity. 887 6.1. Operational Overview 889 DECADE's server-to-server support is focused on reading and writing 890 data objects between DECADE servers. A DECADE GET or PUT request MAY 891 supply the following additional parameters: 893 REMOTE_SERVER: Address of the remote DECADE server. The format of 894 the address is out-of-scope of this document. 896 REMOTE_USER: The account at the remote server from which to retrieve 897 the object (for a GET), or in which the object is to be stored 898 (for a PUT). 900 TOKEN: Credentials to be used at the remote server. 902 These parameters are used by the DECADE server to instantiate a 903 request to the specified remote server. It is assumed that the data 904 object referred to at the remote server is the same as the original 905 request. It is also assumed that the operation performed at the 906 remote server is the same as the operation in the original request. 907 Though explicitly supplying these may provide additional freedom, it 908 is not clear what benefit they might provide. 910 Note that when a DECADE client invokes a request a DECADE server with 911 these additional parameters, it is giving the DECADE server 912 permission to act on its behalf. Thus, it would be wise for the 913 supplied token to have narrow privileges (e.g., limited to only the 914 necessary data objects) or validity time (e.g., a small expiration 915 time). 917 In the case of a GET operation, the DECADE server is to retrieve the 918 data object from the remote server using the specified credentials 919 (via a GET request to the remote server), and then return the object 920 to the client. In the case of a PUT operation, the DECADE server is 921 to store the object from the client, and then store the object to the 922 remote server using the specified credentials (via a PUT request to 923 the remote server). 925 6.2. Potential Optimizations 927 As a suggestion to the protocol and eventual implementations, we 928 would like to point out particular optimizations that could be made 929 when making use of the server-to-server protocol. 931 6.2.1. Pipelining to Avoid Store-and-Forward Delays 933 A DECADE server may choose to not fully store an object before 934 beginning to serve it. For example, in the case of a GET request, a 935 DECADE server may begin to receive a data object from a remote 936 server, and immediately begin returning it to the DECADE client. 937 This pipelining mode avoids store-and-forward delays, which could be 938 substantial for large objects. A similar behavior could be used for 939 PUT. 941 7. Security Considerations 943 This document currently does not contain any security considerations 944 beyond those mentioned in [I-D.ietf-decade-problem-statement]. 946 8. IANA Considerations 948 This document does not have any IANA considerations. 950 9. Informative References 952 [I-D.ietf-decade-problem-statement] 953 Song, H., Zong, N., Yang, Y., and R. Alimi, "DECoupled 954 Application Data Enroute (DECADE) Problem Statement", 955 draft-ietf-decade-problem-statement-02 (work in progress), 956 January 2011. 958 [I-D.ietf-decade-survey] 959 Alimi, R., Rahman, A., and Y. Yang, "A Survey of In- 960 network Storage Systems", draft-ietf-decade-survey-04 961 (work in progress), March 2011. 963 [I-D.gu-decade-reqs] 964 Yingjie, G., Bryan, D., Yang, Y., and R. Alimi, "DECADE 965 Requirements", draft-gu-decade-reqs-05 (work in progress), 966 July 2010. 968 Appendix A. Appendix: Evaluation of Candidate Existing Protocols for 969 DECADE DRP and SDT 971 In this section we illustrate how the abstract protocol interactions 972 specified in Section 5 for DECADE DRP and SDT can be fulfilled by 973 existing protocols such as HTTP and WEBDAV. (This version of the 974 document considers HTTP only, a future version will provide a 975 possible WEBDAV-based illustration as well.) 977 A.1. HTTP 979 HTTP is a key protocol for the Internet and specifically the World 980 Wide Web. HTTP is a request-response protocol. A typical transaction 981 is when a client (e.g. web browser) requests content (resources) from 982 a web server. Other examples are when the client puts or deletes 983 content from the server. 985 A.1.1. HTTP Support for DECADE Resource Protocol Primitives 987 DRP provides configuration of access control and resource sharing 988 policies on DECADE servers. 990 A.1.1.1. Access Control Primitives 992 Access control requires mechanisms for defining the access policies 993 for the server, and then checking the authorization of a user before 994 it stores or retrieves content. HTTP supports access control via 995 "HTTP Secure" (HTTPS). HTTPS is a combination of HTTP with SSL/TLS. 996 The main purpose of HTTPS is to authenticate the server and encrypt 997 all traffic between the client and the server. HTTPS does not 998 support authentication of the client. 1000 A.1.1.2. Communication Resource Controls Primitives 1002 Communications resources include bandwidth (upload/download) and 1003 number of simultaneous connected clients (connections). HTTP 1004 supports communication resource control through "persistent" HTTP 1005 connections. Persistent HTTP connections allows a client to keep 1006 open the underlying TCP connection to the server to allow streaming 1007 and pipelining (multiple simultaneous requests). HTTP does not 1008 support a mechanism to allow limiting the communciation resources to 1009 a client. 1011 A.1.1.3. Storage Resource Control Primitives 1013 Storage resources include amount of memory and lifetime of storage. 1014 HTTP does not allow direct control of storage at the server end 1015 point. However HTTP supports caching at intermediate points such as 1016 a web proxy. For this purpose, HTTP defines cache control mechanisms 1017 that define how long and in what situations the intermediate point 1018 may store and use the content. 1020 A.1.2. HTTP Support for DECADE Standard Transport Protocol Primitives 1022 SDT is used to write objects and read (download) objects from a 1023 DECADE server. The object can be either a self-contained object such 1024 as a multimedia file or a chunk from a P2P system. 1026 A.1.2.1. Writing Primitives 1028 Writing involves uploading objects to the server. HTTP supports two 1029 methods of writing called PUT and POST. In HTTP the object is called 1030 a resource and can be identified by a URL. PUT uploads a resource to 1031 a specific location on the server. POST, on the other hand, submits 1032 the object to the server and the server decides whether to update an 1033 existing resource or to create a new resource. 1035 A.1.2.2. Downloading Primitives 1037 Downloading involves fetching of an object from the server. HTTP 1038 supports downloading through the GET method. GET fetches a specific 1039 resource as identified by the URL. 1041 A.1.2.3. Other Methods 1043 HTTP supports deleting of content on the server through the DELETE 1044 method. 1046 Authors' Addresses 1048 Richard Alimi 1049 Google 1051 Email: ralimi@google.com 1053 Y. Richard Yang 1054 Yale University 1056 Email: yry@cs.yale.edu 1058 Akbar Rahman 1059 InterDigital Communications, LLC 1061 Email: akbar.rahman@interdigital.com 1063 Dirk Kutscher 1064 NEC 1066 Email: dirk.kutscher@neclab.eu 1068 Hongqiang Liu 1069 Yale University 1071 Email: hongqiang.liu@yale.edu