idnits 2.17.1 draft-ietf-decade-reqs-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Sep 2009 rather than the newer Notice from 28 Dec 2009. (See https://trustee.ietf.org/license-info/) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (May 17, 2011) is 4727 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-06) exists of draft-ietf-decade-problem-statement-00 Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DECADE Y. Gu 3 Internet-Draft Huawei 4 Intended status: Informational D. Bryan 5 Expires: November 18, 2011 Cogent Force, LLC / Huawei 6 Y. Yang 7 Yale University 8 R. Alimi 9 Google 10 May 17, 2011 12 DECADE Requirements 13 draft-ietf-decade-reqs-02 15 Abstract 17 The target of DECoupled Application Data Enroute (DECADE) is to 18 provide an open and standard in-network storage system for 19 applications, primarily P2P applications, to store, retrieve and 20 manage their data. This draft enumerates and explains requirements, 21 not only for store and retrieve, but also for data management, access 22 control and resource control, that should be considered during the 23 design and implementation of a DECADE system. These are requirements 24 on the entire system; some of the requirements may eventually be 25 implemented by an existing protocol with/without some extensions 26 (e.g., the data transport level). A user of DECADE as a complete 27 architecture would be guaranteed complete functionality. 29 Requirements Language 31 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 32 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 33 document are to be interpreted as described in [RFC2119]. 35 Status of this Memo 37 This Internet-Draft is submitted to IETF in full conformance with the 38 provisions of BCP 78 and BCP 79. 40 Internet-Drafts are working documents of the Internet Engineering 41 Task Force (IETF), its areas, and its working groups. Note that 42 other groups may also distribute working documents as Internet- 43 Drafts. 45 Internet-Drafts are draft documents valid for a maximum of six months 46 and may be updated, replaced, or obsoleted by other documents at any 47 time. It is inappropriate to use Internet-Drafts as reference 48 material or to cite them other than as "work in progress." 49 The list of current Internet-Drafts can be accessed at 50 http://www.ietf.org/ietf/1id-abstracts.txt. 52 The list of Internet-Draft Shadow Directories can be accessed at 53 http://www.ietf.org/shadow.html. 55 This Internet-Draft will expire on November 18, 2011. 57 Copyright Notice 59 Copyright (c) 2011 IETF Trust and the persons identified as the 60 document authors. All rights reserved. 62 This document is subject to BCP 78 and the IETF Trust's Legal 63 Provisions Relating to IETF Documents 64 (http://trustee.ietf.org/license-info) in effect on the date of 65 publication of this document. Please review these documents 66 carefully, as they describe your rights and restrictions with respect 67 to this document. Code Components extracted from this document must 68 include Simplified BSD License text as described in Section 4.e of 69 the Trust Legal Provisions and are provided without warranty as 70 described in the BSD License. 72 Table of Contents 74 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 75 2. Terminology and Concepts . . . . . . . . . . . . . . . . . . . 5 76 3. Requirements Structure . . . . . . . . . . . . . . . . . . . . 6 77 4. Protocol Requirements . . . . . . . . . . . . . . . . . . . . 7 78 4.1. Requirements . . . . . . . . . . . . . . . . . . . . . . . 7 79 4.1.1. Overall Protocol Requirements . . . . . . . . . . . . 7 80 4.1.1.1. Cross-platform Access . . . . . . . . . . . . . . 7 81 4.1.1.2. Connectivity Concerns . . . . . . . . . . . . . . 7 82 4.1.1.2.1. NATs and Firewalls . . . . . . . . . . . . . . 7 83 4.1.1.2.2. Connections to Clients . . . . . . . . . . . . 7 84 4.1.1.3. Security . . . . . . . . . . . . . . . . . . . . . 8 85 4.1.1.3.1. Secure Transport . . . . . . . . . . . . . . . 8 86 4.1.1.3.2. Connections to Clients . . . . . . . . . . . . 8 87 4.1.1.4. Error and Failure Conditions . . . . . . . . . . . 8 88 4.1.1.4.1. Overload Condition . . . . . . . . . . . . . . 8 89 4.1.1.4.2. Insufficient Resources . . . . . . . . . . . . 8 90 4.1.1.4.3. Unavailable and Deleted Data . . . . . . . . . 9 91 4.1.1.4.4. Redirection . . . . . . . . . . . . . . . . . 9 92 4.1.2. Transfer and Latency Requirements . . . . . . . . . . 9 93 4.1.2.1. Low-Latency Access . . . . . . . . . . . . . . . . 9 94 4.1.2.2. Indirect Transfer . . . . . . . . . . . . . . . . 10 95 4.1.2.3. Data Object Size . . . . . . . . . . . . . . . . . 10 96 4.1.2.4. Communication among In-network Storage Elements . 10 97 4.1.3. Data Access Requirements . . . . . . . . . . . . . . . 11 98 4.1.3.1. Reading/Writing Own Storage . . . . . . . . . . . 11 99 4.1.3.2. Access by Other Users . . . . . . . . . . . . . . 11 100 4.1.3.3. Negotiable Data Protocol . . . . . . . . . . . . . 11 101 4.1.3.4. Separation of Data Operations from Application 102 Control . . . . . . . . . . . . . . . . . . . . . 12 103 4.1.4. Data Management Requirements . . . . . . . . . . . . . 12 104 4.1.4.1. Agnostic of reliability . . . . . . . . . . . . . 12 105 4.1.4.2. Time-to-live for Stored Data . . . . . . . . . . . 13 106 4.1.4.3. Offline Usage . . . . . . . . . . . . . . . . . . 13 107 4.1.5. Data Naming Requirements . . . . . . . . . . . . . . . 13 108 4.1.5.1. Unique Names . . . . . . . . . . . . . . . . . . . 13 109 4.1.6. Resource Control . . . . . . . . . . . . . . . . . . . 14 110 4.1.6.1. Multiple Applications . . . . . . . . . . . . . . 14 111 4.1.6.2. Per-Remote-Client, Per-Data Control . . . . . . . 14 112 4.1.6.3. Server Involvement . . . . . . . . . . . . . . . . 15 113 4.1.7. Authorization . . . . . . . . . . . . . . . . . . . . 15 114 4.1.7.1. Per-Remote-Client, Per-Data Read Access . . . . . 15 115 4.1.7.2. Per-User Write Access . . . . . . . . . . . . . . 15 116 4.1.7.3. Authorization Checks . . . . . . . . . . . . . . . 16 117 4.1.7.4. Credentials Not IP-Based . . . . . . . . . . . . . 16 118 4.1.7.5. Server Involvement . . . . . . . . . . . . . . . . 16 119 5. Discovery Requirements . . . . . . . . . . . . . . . . . . . . 16 120 5.1. Requirements . . . . . . . . . . . . . . . . . . . . . . . 16 121 5.1.1. Locating DECADE Servers . . . . . . . . . . . . . . . 17 122 5.1.2. Support for Clients Behind NATs and Firewalls . . . . 17 123 5.1.3. Prefer Existing Protocols . . . . . . . . . . . . . . 17 124 6. Storage Requirements . . . . . . . . . . . . . . . . . . . . . 17 125 6.1. Requirements . . . . . . . . . . . . . . . . . . . . . . . 17 126 6.1.1. Explicit Deletion of Stored Data . . . . . . . . . . . 17 127 6.1.2. Multiple writing . . . . . . . . . . . . . . . . . . . 18 128 6.1.3. Multiple reading . . . . . . . . . . . . . . . . . . . 18 129 6.1.4. Reading before completely written . . . . . . . . . . 18 130 6.1.5. Hints concerning usage stored data . . . . . . . . . . 18 131 6.1.6. Writing model . . . . . . . . . . . . . . . . . . . . 19 132 6.1.7. Storage Status . . . . . . . . . . . . . . . . . . . . 19 133 6.2. Non-Requirements . . . . . . . . . . . . . . . . . . . . . 19 134 6.2.1. No ability to update . . . . . . . . . . . . . . . . . 20 135 7. Implementation Considerations . . . . . . . . . . . . . . . . 20 136 7.1. Resource Scheduling . . . . . . . . . . . . . . . . . . . 20 137 7.2. Removal of Duplicate Records . . . . . . . . . . . . . . . 20 138 8. Discussion and Open Issues . . . . . . . . . . . . . . . . . . 21 139 8.1. Discussion . . . . . . . . . . . . . . . . . . . . . . . . 21 140 8.2. Open Issues . . . . . . . . . . . . . . . . . . . . . . . 21 141 9. Security Considerations . . . . . . . . . . . . . . . . . . . 22 142 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 143 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22 144 11.1. Normative References . . . . . . . . . . . . . . . . . . . 22 145 11.2. Informative References . . . . . . . . . . . . . . . . . . 22 146 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . . 22 147 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23 149 1. Introduction 151 The object of DECoupled Application Data Enroute (DECADE) is to 152 provide an open and standard in-network storage system for 153 applications, primarily applications that could be implemented using 154 a content distribution paradigm, where data is broken in to one or 155 more chunks and then distributed. This may already include many 156 types of applications including P2P applications, IPTV, and VoD. 157 Instead of always transferring data directly from a source/owner 158 client to a requesting client, the source/owner client can store and 159 manage its content on its in-network storage. The requesting client 160 can get the address of the in-network storage pertaining to the 161 source/owner client and retrieve data from the storage. 163 This draft enumerates and explains the rationale behind SPECIFIC 164 requirements on the protocol design and on any data store 165 implementation that may be used to implement DECADE servers that 166 should be considered during the design and implementation of a DECADE 167 system. As such, it DOES NOT include general guiding principals. 168 General design considerations, explanation of the problem being 169 addressed, and enumeration of the types of applications to which 170 DECADE may be suited is not considered in this document. For general 171 information, please see the problem statement 172 [I-D.ietf-decade-problem-statement] and architecture drafts. 174 This document enumerates the requirements to enable target 175 applications to utilize in-network storage. In this context, using 176 storage resources includes not only basic capabilities such as 177 storing and retrieving data, and managing data, but also (1) 178 controlling access for particular remote clients with which it is 179 sharing data and (2) controlling the resources used by remote clients 180 when they access data. 182 2. Terminology and Concepts 184 This document uses terms defined in 185 [I-D.ietf-decade-problem-statement]. In particular, IAP refers to 186 the In-network storage Access Protocol, which is the protocol used to 187 communicate between a DECADE client and DECADE server (in-network 188 storage) for access control and resource control. 190 This document also defines additional terminology: 192 Target Application: An application (typically installed at end-hosts) 193 with the ability to explicitly control usage of network and/or 194 storage resources to deliver contents to a large number of users. 195 This includes scenarios where multiple applications or entities 196 cooperate, such as with P2P/CDN hybrid architectures. Such 197 applications distribute large contents (e.g., a large file, or video 198 stream) by dividing the contents into smaller blocks for more 199 flexible distribution (e.g., multipath). The distributed content is 200 typically immutable (though it may be deleted). We use the term 201 Target Application to refer to the type of applications that are 202 explicitly (but not exclusively) supported by DECADE. 204 3. Requirements Structure 206 The DECADE protocol is intended to sit between Target Applications 207 and a back-end storage system. In the development of DECADE, it must 208 be made clear that the intention is to NOT develop yet another 209 storage system, but rather to create a protocol that enables Target 210 Applications to make use of storage within the network, leaving 211 specific storage system considerations to the implementation of the 212 DECADE servers as much as possible. For this reason, we have divided 213 the requirements into two categories: 215 o Protocol Requirements: Protocol requirements for Target 216 Applications to make use of in-network storage within their own 217 data dissemination schemes. Development of these requirements is 218 guided by a study of data access, search and management 219 capabilities used by Target Applications. These requirements may 220 be met by a new protocol to be defined within the DECADE Working 221 Group. 223 o Storage Requirements: Functional requirements necessary for the 224 back-end storage system employed by the DECADE server. 225 Development of these requirements is guided by a study of the data 226 access patterns used by Target Applications. These requirements 227 should be met by the underling data transport used by DECADE. 229 It should also be made clear that the approach is to make DECADE a 230 simple protocol, while still enabling its usage within many Target 231 Applications. For this reason, and to further reinforce the 232 distinction between DECADE and a storage system, in some cases we 233 also highlight the non-requirements of the protocol. These non- 234 requirements are intended to capture behaviors that will NOT be 235 assumed to be needed by DECADE's Target Applications and hence not 236 present in the DECADE protocol. 238 Finally, some implementation considerations are provided, which while 239 strictly are not requirements, are intended to provide guidance and 240 highlight potential points of concern that need to be considered by 241 the protocol developers, and later by implementors. 243 4. Protocol Requirements 245 4.1. Requirements 247 4.1.1. Overall Protocol Requirements 249 4.1.1.1. Cross-platform Access 251 REQUIREMENT(S): If DECADE supports the ability to store metadata 252 associated with data objects, the DECADE protocol(s) MUST 253 transmit any metadata using an operating system-independent and 254 architecture-independent format. 256 RATIONALE: If DECADE supports the possibility for storing metadata 257 (e.g., a description, uploaded date, object size, or access 258 control list), a possible use for the metadata is to help a 259 DECADE client locate a desired data object. Data objects may be 260 stored by DECADE clients running on various platforms. To enable 261 metadata to be readable regardless of its source it must be 262 transmitted to and from the DECADE server in a standard format. 264 4.1.1.2. Connectivity Concerns 266 4.1.1.2.1. NATs and Firewalls 268 REQUIREMENT(S): DECADE SHOULD be usable across firewalls and NATs 269 without requiring additional network support (e.g., Application- 270 level Gateways). 272 RATIONALE: Firewalls and NATs are widely used in the Internet today, 273 both in ISP networks and within households. Deployment of DECADE 274 must not require modifications to such devices (beyond, perhaps, 275 reconfiguration). Note that this requirement applies to both any 276 new protocol developed by the DECADE Working Group and any data 277 transport used with DECADE. 279 4.1.1.2.2. Connections to Clients 281 REQUIREMENT(S): DECADE SHOULD require that network connections be 282 made from DECADE clients to DECADE servers (i.e., not to the 283 DECADE client). 285 RATIONALE: Many household networks and operating systems have 286 firewalls and NATs configured by default. To ease deployment by 287 avoiding configuration changes and help mitigate security risks, 288 DECADE should not require clients to listen for any incoming 289 network connections (beyond what is required by any other 290 already-deployed application). 292 4.1.1.3. Security 294 4.1.1.3.1. Secure Transport 296 REQUIREMENT(S): DECADE MUST contain a mode in which all 297 communication between a DECADE client and server is over a secure 298 transport that provides confidentiality, integrity, and 299 authentication. 301 RATIONALE: Target Applications may wish to store sensitive data. To 302 satisfy this use case, DECADE must provide a mode in which all 303 communication between a DECADE client and server occurs over a 304 secure transport protocol (e.g., SSL/TLS). 306 4.1.1.3.2. Connections to Clients 308 REQUIREMENT(S): DECADE SHOULD require that network connections be 309 made from DECADE clients to DECADE servers (i.e., not to the 310 DECADE client). 312 RATIONALE: Many household networks and operating systems have 313 firewalls and NATs configured by default. To ease deployment by 314 avoiding configuration changes and help mitigate security risks, 315 DECADE should not require clients to listen for any incoming 316 network connections (beyond what is required by any other 317 already-deployed application). 319 4.1.1.4. Error and Failure Conditions 321 4.1.1.4.1. Overload Condition 323 REQUIREMENT(S): In-network storage, which is operating close to its 324 capacity limit (e.g., too busy servicing other requests), MUST be 325 able to reject requests and not be required to generate responses 326 to additional requests. 328 RATIONALE: When in-network storage is operating at a limit where it 329 may not be able to process additional requests, it should not be 330 required to generate responses to such additional requests. 331 Forcing the in-network storage to do so can impair its ability to 332 service existing requests. 334 4.1.1.4.2. Insufficient Resources 335 REQUIREMENT(S): DECADE MUST support an error condition indicating to 336 a DECADE client that resources (e.g., storage space) were not 337 available to service a request (e.g., storage quota exceeded when 338 attempting to store data). 340 RATIONALE: The currently-used resource levels within the in-network 341 storage are not locally-discoverable, since the resources (disk, 342 network interfaces, etc) are not directly attached. In order to 343 allocate resources appropriately amongst remote clients, a client 344 must be able to determine when resource limits have been reached. 345 The client can then respond by explicitly freeing necessary 346 resources or waiting for such resources to be freed. 348 4.1.1.4.3. Unavailable and Deleted Data 350 REQUIREMENT(S): DECADE MUST support error conditions indicating that 351 (1) data was rejected from being stored, (2) deleted, or (3) 352 marked unavailable by a storage provider. 354 RATIONALE: Storage providers may require the ability to (1) avoid 355 storing, (2) delete, or (3) quarantine certain data that has been 356 identified as illegal (or otherwise prohibited). DECADE does not 357 indicate how such data is identified, but applications using 358 DECADE should not break if a storage provider is obligated to 359 enforce such policies. Appropriate error conditions should be 360 indicated to applications. 362 4.1.1.4.4. Redirection 364 REQUIREMENT(S): DECADE SHOULD support the ability for a DECADE 365 server to redirect requests to another DECADE server. This may 366 be in response to either an error or failure condition, or to 367 support capabilities such as load balancing. 369 RATIONALE: A DECADE server may opt to redirect requests to another 370 server to support capabilities such as load balancing, or if the 371 implementation decides that another DECADE server is in a better 372 position to handle the request due to either its location in the 373 network, server status, or other consideration. 375 4.1.2. Transfer and Latency Requirements 377 4.1.2.1. Low-Latency Access 378 REQUIREMENT(S): DECADE SHOULD provide "low-latency" access for 379 application clients. DECADE MUST allow clients to specify at 380 least two classes of services for service: lowest possible 381 latency and latency non-critical. 383 RATIONALE: Some applications may have requirements on delivery time 384 (e.g., live streaming [PPLive]). The user experience may be 385 unsatisfactory if the use of in-network storage results in lower 386 performance than connecting directly to remote clients over a 387 low-speed, possibly congested uplink. Additionally, the overhead 388 required for control-plane operations in DECADE must not cause 389 the latency to be higher than for a low-speed, possibly congested 390 uplink. While it is impossible to make a guarantee that a system 391 using in-network storage will always outperform a system that 392 does not for every transfer, the overall performance of the 393 system should be improved compared with direct connections, even 394 considering control overhead. 396 4.1.2.2. Indirect Transfer 398 REQUIREMENT(S): DECADE MUST allow a user's in-network storage to 399 directly fetch from other user's in-network storage. 401 RATIONALE: As an example, a requesting remote client may get the 402 address of the storage pertaining to the source/owner client and 403 then tell its own in-network storage to fetch the content from 404 the source-owner's in-network storage. This helps to avoid extra 405 transfers across ISP network links where possible. 407 4.1.2.3. Data Object Size 409 REQUIREMENT(S): DECADE MUST allow for efficient data transfer of 410 small objects (e.g., 16KB) between a DECADE client and in-network 411 storage with minimal additional latency required by the protocol. 413 RATIONALE: Though Target Applications are frequently used to share 414 large amounts of data (e.g., continuous streams or large files), 415 the data itself is typically subdivided into smaller chunks that 416 are transferred between clients. Additionally, the small chunks 417 may have requirements on delivery time (e.g., in a live-streaming 418 application). DECADE must enable data to be efficiently 419 transferred amongst clients at this granularity. 421 4.1.2.4. Communication among In-network Storage Elements 422 REQUIREMENT(S): DECADE SHOULD support the ability for two in-network 423 storage elements in different administrative domains to store 424 and/or retrieve data directly between each other. If such a 425 capability is supported, this MAY be the same (or a subset or 426 extension of) as the IAP used by clients to access data. 428 RATIONALE: Allowing server-to-server communication can reduce 429 latency in some common scenarios. Consider a scenario when a 430 DECADE client is downloading data into its own storage from 431 another client's in-network storage. One possibility is for the 432 client to first download the data itself, and then upload it to 433 its own storage. However, this causes unnecessary latency and 434 network traffic. Allowing the data to be downloaded from the 435 remote in-network storage into the client's own in-network 436 storage can alleviate both. 438 4.1.3. Data Access Requirements 440 4.1.3.1. Reading/Writing Own Storage 442 REQUIREMENT(S): DECADE MUST support the ability for a DECADE client 443 to read data from and write data to its own in-network storage. 445 RATIONALE: Two basic capabilities for any storage system are reading 446 and writing data. A DECADE client can read data from and write 447 data to in-network storage space that it owns. 449 4.1.3.2. Access by Other Users 451 REQUIREMENT(S): DECADE MUST support the ability for a user to apply 452 access control policies to users other than itself for its 453 storage. The users with whom access is being shared can be under 454 a different administrative domain than the user who owns the in- 455 network storage. The authorized users may read from or write to 456 the user's storage. 458 RATIONALE: Endpoints in Target Applications may be located across 459 multiple ISPs under multiple administrative domains. Thus, to be 460 useful by Target Applications, DECADE allows a user to specify 461 access control policies for users that may or may not be known to 462 the user's storage provider. 464 4.1.3.3. Negotiable Data Protocol 465 REQUIREMENT(S): DECADE MUST support the ability for a DECADE client 466 to negotiate with its in-network storage about which protocol it 467 can use to read data from and write data to its In-network 468 storage. DECADE MUST specify at least one mandatory protocol to 469 be supported by implementations; usage of a different protocol 470 may be selected via negotiation. 472 RATIONALE: Since typical data transport protocols (e.g., NFS and 473 WebDAV) already provide read and write operations for network 474 storage, it may not be necessary for DECADE to define such 475 operations in a new protocol. However, because of the particular 476 application requirements and deployment considerations, different 477 applications may support different protocols. Thus, a DECADE 478 client must be able to select an appropriate protocol also 479 supported by the in-network storage. This requirement also 480 follows as a result of the requirement of Separation of Control 481 and Data Operations (Section 4.1.3.4). 483 4.1.3.4. Separation of Data Operations from Application Control 485 REQUIREMENT(S): The DECADE IAP MUST only provide a minimal set of 486 core operations to support diverse policies implemented and 487 desired by Target Applications. 489 RATIONALE: Target Applications support many complex behaviors and 490 diverse policies to control and distribute data, such as (e.g., 491 search, index, setting permissions/passing authorization tokens). 492 Thus, to support such Target Applications, these behaviors must 493 be logically separated from the data transfer operations (e.g., 494 retrieve, store). Some minimal overlap (for example obtaining 495 credentials needed to encrypt or authorize data transfer using 496 control operations) may be required to be directly specified by 497 DECADE. 499 4.1.4. Data Management Requirements 501 4.1.4.1. Agnostic of reliability 503 REQUIREMENT(S): DECADE SHOULD remain agnostic of reliability/ 504 fault-tolerance level offered by storage provider. 506 RATIONALE: Providers of a DECADE service may wish to offer varying 507 levels of service for different applications/users. However, a 508 single compliant DECADE client should be able to use multiple 509 DECADE services with differing levels of service. 511 4.1.4.2. Time-to-live for Stored Data 513 REQUIREMENT(S): DECADE MUST support the ability for a DECADE client 514 to indicate a time-to-live value (or expiration time) indicating 515 a length of time until particular data can be deleted by the in- 516 network storage element. 518 RATIONALE: Some data stored by a DECADE client may be usable only 519 within a certain window of time, such as in live-streaming P2P 520 applications. Providing a time-to-live value for stored data 521 (e.g., at the time it is stored) can reduce management overhead 522 by avoiding many 'delete' commands sent to in-network storage. 523 The in-network storage may still keep the data in cache for 524 bandwidth optimization. But this is guided by the privacy policy 525 of the DECADE provider. 527 4.1.4.3. Offline Usage 529 REQUIREMENT(S): DECADE MAY support the ability for a user to provide 530 authorized access to its in-network storage even if the user has 531 no DECADE applications actively running or connected to the 532 network. 534 RATIONALE: If an application desires, it can authorize remote 535 clients to access its storage even after the application exits or 536 network connectivity is lost. An example use case is mobile 537 scenarios, where a client can lose and regain network 538 connectivity very often. 540 4.1.5. Data Naming Requirements 542 4.1.5.1. Unique Names 544 REQUIREMENT(S): DECADE SHOULD use a naming scheme in which the name 545 of a data object is unique from the names of all other data 546 objects with different content. 548 RATIONALE: When storing a data object, a DECADE Client should be 549 able to store it without being concerned over whether an object 550 of the same name already exists, unless the existing object 551 contains the exact same data. Note that it may be reasonable for 552 DECADE to satisfy this requirement probabilistically (e.g., using 553 a hashing mechanism). 555 4.1.6. Resource Control 557 4.1.6.1. Multiple Applications 559 REQUIREMENT(S): DECADE SHOULD support the ability for users to 560 define resource sharing policies for multiple applications 561 (DECADE clients) being run/managed by the user. 563 RATIONALE: A user may own in-network storage and share the in- 564 network storage resources amongst multiple applications. For 565 example, the user may run a video-on-demand application and a 566 live-streaming (or even two different live-streaming 567 applications) application which both make use of the user's in- 568 network storage. The applications may be running on different 569 machines and may not directly communicate. Thus, DECADE should 570 enable the user to determine resource sharing policies between 571 the applications. 573 One possibility is for a user to indicate the particular resource 574 sharing policies between applications out-of-band (not using a 575 standard protocol), but this requirement may manifest itself in 576 passing values over IAP to identify individual applications. 577 Such identifiers can be either user-generated or server-generated 578 and do not need to be registered by IANA. 580 4.1.6.2. Per-Remote-Client, Per-Data Control 582 REQUIREMENT(S): A DECADE client MUST be able to assign resource 583 quotas to individual remote clients for reading from and writing 584 particular data to its in-network storage within a particular 585 range of time. The DECADE server MUST enforce these constraints. 587 RATIONALE: Target Applications can rely on control of resources on a 588 per-remote-client or per-data basis. For example, application 589 policy may indicate that certain remote clients have a higher 590 bandwidth share for receiving data [LLSB08]. Additionally, 591 certain data (e.g., chunks) may be distributed with a higher 592 priority. As another example, when allowing a remote client to 593 write data to a user's in-network storage, the remote client may 594 be restricted to write only a certain amount of data. Since the 595 client may need to manage multiple clients accessing its data, it 596 should be able to indicate the time over which the granted 597 resources are usable. For example, an expiration time for the 598 access could be indicated to the server after which no resources 599 are granted (e.g., indicate error as access denied). 601 4.1.6.3. Server Involvement 603 REQUIREMENT(S): A DECADE client SHOULD be able to indicate to a 604 DECADE server, without itself contacting the server, resource 605 control policies for remote clients' requests. 607 RATIONALE: One important consideration for in-network storage 608 elements is scalability, since a single storage element may be 609 used to support many users. Many Target Applications use small 610 chunk sizes and frequent data exchanges. If such an application 611 employed resource control and contacted the in-network storage 612 element for each data exchange, this could present a scalability 613 challenge for the server as well as additional latency for 614 clients. 616 An alternative is to let requesting users obtain signed resource 617 control policies from the owning user, and then users can then 618 present the policy to the storage directly. This can result in 619 reduced messaging handled by the in-network storage. 621 4.1.7. Authorization 623 4.1.7.1. Per-Remote-Client, Per-Data Read Access 625 REQUIREMENT(S): A DECADE Client MUST be able to control which 626 individual remote clients are authorized to read particular data 627 stored on its in-network storage. 629 RATIONALE: A Target Application can control certain application- 630 level policies by sending particular data (e.g., chunks) to 631 certain remote clients. It is important that remote clients not 632 be able to circumvent such decisions by arbitrarily reading any 633 currently-stored data in in-network storage. 635 4.1.7.2. Per-User Write Access 637 REQUIREMENT(S): A DECADE Client MUST be able to control which 638 individual remote clients are authorized to store data into its 639 in-network storage. 641 RATIONALE: The space managed by a user in in-network storage can be 642 a limited resource. At the same time, it can be useful to allow 643 remote clients to write data directly to a user's in-network 644 storage. Thus, a DECADE client should be able to grant only 645 certain remote clients this privilege. 647 Note that it is not (currently) a requirement to check that a 648 remote client stores a particular set of data (e.g., the check 649 that a remote client writes the expected chunk of a file). 650 Enforcing this as a requirement would require a client to know 651 which data is expected (e.g., the full chunk itself or a hash of 652 the chunk) which may not be available in all applications. 653 Checking for a particular hash could be considered as a 654 requirement in the future that could optionally be employed by 655 applications. 657 4.1.7.3. Authorization Checks 659 REQUIREMENT(S): In-network storage MUST check the authorization of a 660 client before it executes a supplied request. The in-network 661 storage MAY use optimizations to avoid such authorization checks 662 as long as the enforced permissions are the the same. 664 RATIONALE: Authorization granted by a DECADE client are meaningless 665 unless unauthorized requests are denied access. Thus, the in- 666 network storage element must verify the authorization of a 667 particular request before it is executed. 669 4.1.7.4. Credentials Not IP-Based 671 REQUIREMENT(S): Access MUST be able to be granted on other 672 credentials than the IP address 674 RATIONALE: DECADE clients may be operating on hosts without constant 675 network connectivity or without a permanent attachment address 676 (e.g., mobile devices). To support access control with such 677 hosts, DECADE servers must support access control policies that 678 use information other than IP addresses. 680 4.1.7.5. Server Involvement 682 REQUIREMENT(S): A DECADE client SHOULD be able to indicate, without 683 contacting the server itself, access control policies for remote 684 clients' requests. 686 RATIONALE: See discussion in Section 4.1.6.3. 688 5. Discovery Requirements 690 5.1. Requirements 691 5.1.1. Locating DECADE Servers 693 REQUIREMENT(S): The DECADE Discovery mechanism MUST allow a DECADE 694 Client to identify one or more DECADE Servers to which it is 695 authorized to read/write data and to which it may authorize other 696 DECADE Clients to read/write data, or fail if no such DECADE 697 Servers are available. 699 RATIONALE: A basic goal of DECADE is to allow DECADE Clients to 700 read/write data for access by other DECADE Clients or itself. 701 Executing the Discovery process should result in a DECADE Client 702 finding a DECADE Server that it can use for these purposes. It 703 is possible that no such DECADE Servers are available. Note that 704 even if a DECADE Client may not successfully locate a DECADE 705 Server that fits these critieria, it may still read/write data a 706 DECADE Server if authorized by another DECADE Client. 708 5.1.2. Support for Clients Behind NATs and Firewalls 710 REQUIREMENT(S): The Discovery mechanism MUST support DECADE Clients 711 operating behind NATs and Firewalls without requiring additional 712 network support (e.g., Application-level Gateways). 714 RATIONALE: NATs and Firewalls are prevalent in network deployments, 715 and it is common for Target Applications wishing to include a 716 DECADE Client to be deployed behind these devices. 718 5.1.3. Prefer Existing Protocols 720 REQUIREMENT(S): The DECADE Server discovery mechanism SHOULD prefer 721 to use existing mechanisms and protocols wherever possible. 723 RATIONALE: Existing protocols such as DNS and DHCP are widespread. 724 Using existing protocols, or combinations of the protocols that 725 have been specified in other contexts, is strictly preferred over 726 developing a new discovery protocol for DECADE. 728 6. Storage Requirements 730 6.1. Requirements 732 6.1.1. Explicit Deletion of Stored Data 733 REQUIREMENT(S): DECADE MUST support the ability for a DECADE client 734 to explicitly delete data from its own in-network storage. 735 DECADE MAY have an overwrite flag indicating that an object with 736 the same name should be replaced. 738 RATIONALE: A DECADE client may continually be writing data to its 739 in-network storage. Since there may be a limit (e.g., imposed by 740 the storage provider) to how much total storage can be used, some 741 data may need to be removed to make room for additional data. A 742 DECADE client should be able to explicitly remove particular 743 data. This may be implemented using existing protocols. 745 6.1.2. Multiple writing 747 REQUIREMENT(S): DECADE MUST NOT allow multiple writers for the same 748 object. Implementations raise an error to one of the writers. 750 RATIONALE: This avoids data corruption in a simple way while 751 remaining efficient. 753 6.1.3. Multiple reading 755 REQUIREMENT(S): DECADE MUST allow for multiple readers for an 756 object. 758 RATIONALE: One characteristic of Target Applications is the ability 759 to upload an object to multiple clients. Thus, it is natural for 760 DECADE to allow multiple readers to read the content 761 concurrently. 763 6.1.4. Reading before completely written 765 REQUIREMENT(S): DECADE MAY allow readers to read from objects before 766 they have been completely written. 768 RATIONALE: Some Target Applications (in particular, P2P streaming) 769 can be sensitive to latency. A technique to reduce latency is to 770 remove store-and-forward delays for data objects (e.g., make the 771 object available before it is completely stored). Appropriate 772 handling for error conditions (e.g., a disappearing writer) needs 773 to be specified. 775 6.1.5. Hints concerning usage stored data 776 REQUIREMENT(S): DECADE MAY allow writers of new objects to indicate 777 specific hints concerning how the objects are expected to be used 778 (e.g., access frequency or time to live). 780 RATIONALE: Different Target Applications may have different usage 781 patterns for objects stored at in-network storage. For example, 782 a P2P live streaming application may indicate to a DECADE server 783 that the objects are expected to have a shore time-to-live, but 784 read frequently. The DECADE server may then opt to store the 785 objects in memory instead of in disk. 787 6.1.6. Writing model 789 REQUIREMENT(S): DECADE MUST provide at least a writing model (while 790 storing an object) that appends data to data already stored. 792 RATIONALE: Depending on the object size (e.g., chunk size) used by a 793 Target Application, the application may need to send data to the 794 DECADE server in multiple packets. To keep implementation 795 simple, the DECADE must at least support the ability to write the 796 data sequentially in the order received. Implementations MAY 797 allow application to write data in an object out-of-order (but 798 MUST NOT overwrite ranges of the object that have already been 799 stored). 801 6.1.7. Storage Status 803 REQUIREMENT(S): A DECADE client MUST be able to retrieve current 804 resource usage (including list of stored data), resource quotas, 805 and access permissions for its in-network storage. The returned 806 information MUST include resource usage resulting from the 807 client's own usage and usage by other clients that have been 808 authorized to read/write objects or open connections to that 809 client's storage. 811 RATIONALE: The resources used by a client are not directly-attached 812 (e.g., disk, network interface, etc). Thus, the client cannot 813 locally determine how such resources are being used. Before 814 storing and retrieving data, a client should be able to determine 815 which data is available (e.g., after an application restart). 816 Additionally, a client should be able to determine resource 817 availability to better allocate them to remote clients. 819 6.2. Non-Requirements 820 6.2.1. No ability to update 822 REQUIREMENT(S): DECADE SHOULD NOT provide ability to update existing 823 objects. That is, objects are immutable once they are stored. 825 RATIONALE: Reasonable consistency models for updating existing 826 objects would significantly complicate implementation (especially 827 if implementation chooses to replicate data across multiple 828 servers). If a user needs to update a resource, it can store a 829 new resource and then distribute the new resource instead of the 830 old one. 832 7. Implementation Considerations 834 The intent of this section is to collect discussion items and 835 implementation considerations that have been discovered as this 836 requirements document has been produced. The content of this section 837 will be migrated to an appropriate place as the document and the 838 Working Group progress. 840 7.1. Resource Scheduling 842 The particular resource scheduling policy may have important 843 ramifications on the performance of applications. This document has 844 explicitly mentioned simultaneous support for both low-latency 845 applications and latency-tolerant applications. 847 Denial of Service attacks may be another risk. For example, 848 rejecting new requests due to overload conditions may introduce the 849 ability to perform a denial of service attack depending on a 850 particular DECADE server's scheduling implementation and resource 851 allocation policies. 853 7.2. Removal of Duplicate Records 855 There are actually two possible scenarios here. The first is the 856 case of removing duplicates within one particular DECADE server (or 857 logical server). While not a requirement, as it does not impact the 858 protocol and is technically not noticeable on message across the 859 wire, a DECADE server may implement internal mechanisms to monitor 860 for duplicate records and use internal mechanisms to prevent 861 duplication of internal storage. 863 The second scenario is removing duplicates across a distributed set 864 of DECADE servers. This is a more difficult problem, and if the 865 group decides to support this capability, it may require protocol 866 support. See Section 8.2 for more details. 868 8. Discussion and Open Issues 870 8.1. Discussion 872 Sometimes, several logical in-network storages could be deployed on 873 the same physical network device. In this case, in-network storages 874 on the same physical network device can communicate and transfer data 875 through internal communication messages. However in-network storages 876 deployed on different physical network devices SHOULD communicate 877 with in-network storage Access Protocol (IAP). 879 To provide fairness among users, the in-network storage provider 880 should assign resource (e.g., storage, bandwidth, connections) quota 881 for users. This can prevent a small number of clients from occupying 882 large amounts of resources on the in-network storage, while others 883 starve. 885 8.2. Open Issues 887 Gaming of the Resource Control Mechanism: There has been some 888 discussion of how applications may be able game the scheduling 889 system by manipulating the resource control mechanism, for 890 example by specifying many small peers to increase total 891 throughput. This is a serious concern, and we need to identify 892 specific requirements on the protocol (hopefully independent of 893 particular scheduling/resource control schemes) to help address 894 this. 896 Discovery: There needs to be some mechanism for a user to discover 897 that there is a DECADE service available for their use, and to 898 locate that server. This is particularly important in the case 899 of mobile applications, since the actual servers that are 900 available at any given time may differ. However, the specifics 901 of what mechanisms (DHCP, HTTP page, etc.) have not been 902 discussed, or even if the protocol should specify one or leave it 903 as an implementation detail. This needs to be defined, and 904 specific requirements formulated if needed. 906 Removal of Duplicate Records Across Servers: If the group wishes to 907 allow for automated mechanisms to remove duplicates across a 908 number of separate servers, some protocol support may need to be 909 added. In the case of removing duplicates within a single 910 (logical) DECADE server, this is simply an implementation 911 concern. See Section 7 for more details. 913 9. Security Considerations 915 Authorization for access to in-network storage is an important part 916 of the requirements listed in this document. Authorization for 917 access to storage resources and the data itself is important for 918 users to be able to manage and limit distribution of content. For 919 example, a user may only wish to share particular content with 920 certain peers. 922 If the authorization technique implemented in DECADE passes any 923 private information (e.g., user passwords) over the wire, it MUST be 924 passed in a secure way. 926 10. IANA Considerations 928 There are no IANA considerations with this document. 930 11. References 932 11.1. Normative References 934 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 935 Requirement Levels", BCP 14, RFC 2119, March 1997. 937 11.2. Informative References 939 [I-D.ietf-decade-problem-statement] 940 Yongchao, S., Zong, N., Yang, Y., and R. Alimi, "DECoupled 941 Application Data Enroute (DECADE) Problem Statement", 942 draft-ietf-decade-problem-statement-00 (work in progress), 943 August 2010. 945 [LLSB08] Levin, D., LaCurts, K., Spring, N., and B. Bhattacharjee, 946 "BitTorrent is an Auction: Analyzing and Improving 947 BitTorrent's Incentives", SIGCOMM 2008, August 2008. 949 [PPLive] "PPLive", . 951 Appendix A. Acknowledgments 953 We would also like to thank Haibin Song for substantial contributions 954 to earlier versions of this document. We would also like to thank 955 Reinaldo Penno, Alexey Melnikov, Rich Woundy, Ning Zong, Roni Even, 956 David McDysan, Boerje Ohlman and Dirk Kutscher for contributions 957 (including some text used verbatim) and general feedback. 959 Authors' Addresses 961 Yingjie Gu 962 Huawei 963 No. 101 Software Avenue 964 Nanjing, Jiangsu Province 210012 965 P.R.China 967 Phone: +86-25-56624760 968 Email: guyingjie@huawei.com 970 David A. Bryan 971 Cogent Force, LLC / Huawei 973 Email: dbryan@ethernot.org 975 Yang Richard Yang 976 Yale University 978 Email: yry@cs.yale.edu 980 Richard Alimi 981 Google 983 Email: ralimi@google.com