idnits 2.17.1 draft-ietf-decade-reqs-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Sep 2009 rather than the newer Notice from 28 Dec 2009. (See https://trustee.ietf.org/license-info/) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 14, 2011) is 4792 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-06) exists of draft-ietf-decade-problem-statement-00 Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DECADE Y. Gu 3 Internet-Draft Huawei 4 Intended status: Informational D. Bryan 5 Expires: September 15, 2011 Cogent Force, LLC / Huawei 6 Y. Yang 7 Yale University 8 R. Alimi 9 Google 10 March 14, 2011 12 DECADE Requirements 13 draft-ietf-decade-reqs-01 15 Abstract 17 The target of DECoupled Application Data Enroute (DECADE) is to 18 provide an open and standard in-network storage system for 19 applications, primarily P2P applications, to store, retrieve and 20 manage their data. This draft enumerates and explains requirements, 21 not only for store and retrieve, but also for data management, access 22 control and resource control, that should be considered during the 23 design and implementation of a DECADE system. These are requirements 24 on the entire system; some of the requirements may eventually be 25 implemented by an existing protocol with/without some extensions 26 (e.g., the data transport level). A user of DECADE as a complete 27 architecture would be guaranteed complete functionality. 29 Requirements Language 31 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 32 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 33 document are to be interpreted as described in [RFC2119]. 35 Status of this Memo 37 This Internet-Draft is submitted to IETF in full conformance with the 38 provisions of BCP 78 and BCP 79. 40 Internet-Drafts are working documents of the Internet Engineering 41 Task Force (IETF), its areas, and its working groups. Note that 42 other groups may also distribute working documents as Internet- 43 Drafts. 45 Internet-Drafts are draft documents valid for a maximum of six months 46 and may be updated, replaced, or obsoleted by other documents at any 47 time. It is inappropriate to use Internet-Drafts as reference 48 material or to cite them other than as "work in progress." 49 The list of current Internet-Drafts can be accessed at 50 http://www.ietf.org/ietf/1id-abstracts.txt. 52 The list of Internet-Draft Shadow Directories can be accessed at 53 http://www.ietf.org/shadow.html. 55 This Internet-Draft will expire on September 15, 2011. 57 Copyright Notice 59 Copyright (c) 2011 IETF Trust and the persons identified as the 60 document authors. All rights reserved. 62 This document is subject to BCP 78 and the IETF Trust's Legal 63 Provisions Relating to IETF Documents 64 (http://trustee.ietf.org/license-info) in effect on the date of 65 publication of this document. Please review these documents 66 carefully, as they describe your rights and restrictions with respect 67 to this document. Code Components extracted from this document must 68 include Simplified BSD License text as described in Section 4.e of 69 the Trust Legal Provisions and are provided without warranty as 70 described in the BSD License. 72 Table of Contents 74 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 75 2. Terminology and Concepts . . . . . . . . . . . . . . . . . . . 5 76 3. Requirements Structure . . . . . . . . . . . . . . . . . . . . 6 77 4. Protocol Requirements . . . . . . . . . . . . . . . . . . . . 7 78 4.1. Requirements . . . . . . . . . . . . . . . . . . . . . . . 7 79 4.1.1. Overall Protocol Requirements . . . . . . . . . . . . 7 80 4.1.1.1. Cross-platform Access . . . . . . . . . . . . . . 7 81 4.1.1.2. Connectivity Concerns . . . . . . . . . . . . . . 7 82 4.1.1.2.1. NATs and Firewalls . . . . . . . . . . . . . . 7 83 4.1.1.2.2. Connections to Clients . . . . . . . . . . . . 7 84 4.1.1.3. Error and Failure Conditions . . . . . . . . . . . 8 85 4.1.1.3.1. Overload Condition . . . . . . . . . . . . . . 8 86 4.1.1.3.2. Insufficient Resources . . . . . . . . . . . . 8 87 4.1.1.3.3. Unavailable and Deleted Data . . . . . . . . . 8 88 4.1.1.3.4. Redirection . . . . . . . . . . . . . . . . . 9 89 4.1.2. Transfer and Latency Requirements . . . . . . . . . . 9 90 4.1.2.1. Low-Latency Access . . . . . . . . . . . . . . . . 9 91 4.1.2.2. Indirect Transfer . . . . . . . . . . . . . . . . 9 92 4.1.2.3. Data Object Size . . . . . . . . . . . . . . . . . 10 93 4.1.2.4. Communication among In-network Storage Elements . 10 94 4.1.3. Data Access Requirements . . . . . . . . . . . . . . . 10 95 4.1.3.1. Reading/Writing Own Storage . . . . . . . . . . . 10 96 4.1.3.2. Access by Other Users . . . . . . . . . . . . . . 10 97 4.1.3.3. Negotiable Data Protocol . . . . . . . . . . . . . 11 98 4.1.3.4. Separation of Data Operations from Application 99 Control . . . . . . . . . . . . . . . . . . . . . 11 100 4.1.4. Data Management Requirements . . . . . . . . . . . . . 12 101 4.1.4.1. Agnostic of reliability . . . . . . . . . . . . . 12 102 4.1.4.2. Time-to-live for Stored Data . . . . . . . . . . . 12 103 4.1.4.3. Offline Usage . . . . . . . . . . . . . . . . . . 12 104 4.1.5. Resource Control . . . . . . . . . . . . . . . . . . . 12 105 4.1.5.1. Multiple Applications . . . . . . . . . . . . . . 12 106 4.1.5.2. Per-Remote-Client, Per-Data Control . . . . . . . 13 107 4.1.5.3. Server Involvement . . . . . . . . . . . . . . . . 13 108 4.1.6. Authorization . . . . . . . . . . . . . . . . . . . . 14 109 4.1.6.1. Per-Remote-Client, Per-Data Read Access . . . . . 14 110 4.1.6.2. Per-User Write Access . . . . . . . . . . . . . . 14 111 4.1.6.3. Authorization Checks . . . . . . . . . . . . . . . 15 112 4.1.6.4. Credentials Not IP-Based . . . . . . . . . . . . . 15 113 4.1.6.5. Server Involvement . . . . . . . . . . . . . . . . 15 114 5. Storage Requirements . . . . . . . . . . . . . . . . . . . . . 15 115 5.1. Requirements . . . . . . . . . . . . . . . . . . . . . . . 15 116 5.1.1. Explicit Deletion of Stored Data . . . . . . . . . . . 15 117 5.1.2. Multiple writing . . . . . . . . . . . . . . . . . . . 16 118 5.1.3. Multiple reading . . . . . . . . . . . . . . . . . . . 16 119 5.1.4. Reading before completely written . . . . . . . . . . 16 120 5.1.5. Hints concerning usage stored data . . . . . . . . . . 16 121 5.1.6. Writing model . . . . . . . . . . . . . . . . . . . . 17 122 5.1.7. Storage Status . . . . . . . . . . . . . . . . . . . . 17 123 5.2. Non-Requirements . . . . . . . . . . . . . . . . . . . . . 17 124 5.2.1. No ability to update . . . . . . . . . . . . . . . . . 18 125 6. Implementation Considerations . . . . . . . . . . . . . . . . 18 126 6.1. Resource Scheduling . . . . . . . . . . . . . . . . . . . 18 127 6.2. Removal of Duplicate Records . . . . . . . . . . . . . . . 18 128 7. Discussion and Open Issues . . . . . . . . . . . . . . . . . . 19 129 7.1. Discussion . . . . . . . . . . . . . . . . . . . . . . . . 19 130 7.2. Open Issues . . . . . . . . . . . . . . . . . . . . . . . 19 131 8. Security Considerations . . . . . . . . . . . . . . . . . . . 20 132 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 133 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 20 134 10.1. Normative References . . . . . . . . . . . . . . . . . . . 20 135 10.2. Informative References . . . . . . . . . . . . . . . . . . 20 136 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . . 20 137 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 21 139 1. Introduction 141 The object of DECoupled Application Data Enroute (DECADE) is to 142 provide an open and standard in-network storage system for 143 applications, primarily applications that could be implemented using 144 a content distribution paradigm, where data is broken in to one or 145 more chunks and then distributed. This may already include many 146 types of applications including P2P applications, IPTV, and VoD. 147 Instead of always transferring data directly from a source/owner 148 client to a requesting client, the source/owner client can store and 149 manage its content on its in-network storage. The requesting client 150 can get the address of the in-network storage pertaining to the 151 source/owner client and retrieve data from the storage. 153 This draft enumerates and explains the rationale behind SPECIFIC 154 requirements on the protocol design and on any data store 155 implementation that may be used to implement DECADE servers that 156 should be considered during the design and implementation of a DECADE 157 system. As such, it DOES NOT include general guiding principals. 158 General design considerations, explanation of the problem being 159 addressed, and enumeration of the types of applications to which 160 DECADE may be suited is not considered in this document. For general 161 information, please see the problem statement 162 [I-D.ietf-decade-problem-statement] and architecture drafts. 164 This document enumerates the requirements to enable target 165 applications to utilize in-network storage. In this context, using 166 storage resources includes not only basic capabilities such as 167 storing and retrieving data, and managing data, but also (1) 168 controlling access for particular remote clients with which it is 169 sharing data and (2) controlling the resources used by remote clients 170 when they access data. 172 2. Terminology and Concepts 174 This document uses terms defined in 175 [I-D.ietf-decade-problem-statement]. In particular, IAP refers to 176 the In-network storage Access Protocol, which is the protocol used to 177 communicate between a DECADE client and DECADE server (in-network 178 storage) for access control and resource control. 180 This document also defines additional terminology: 182 Target Application: An application (typically installed at end-hosts) 183 with the ability to explicitly control usage of network and/or 184 storage resources to deliver contents to a large number of users. 185 This includes scenarios where multiple applications or entities 186 cooperate, such as with P2P/CDN hybrid architectures. Such 187 applications distribute large contents (e.g., a large file, or video 188 stream) by dividing the contents into smaller blocks for more 189 flexible distribution (e.g., multipath). The distributed content is 190 typically immutable (though it may be deleted). We use the term 191 Target Application to refer to the type of applications that are 192 explicitly (but not exclusively) supported by DECADE. 194 3. Requirements Structure 196 The DECADE protocol is intended to sit between Target Applications 197 and a back-end storage system. In the development of DECADE, it must 198 be made clear that the intention is to NOT develop yet another 199 storage system, but rather to create a protocol that enables Target 200 Applications to make use of storage within the network, leaving 201 specific storage system considerations to the implementation of the 202 DECADE servers as much as possible. For this reason, we have divided 203 the requirements into two categories: 205 o Protocol Requirements: Protocol requirements for Target 206 Applications to make use of in-network storage within their own 207 data dissemination schemes. Development of these requirements is 208 guided by a study of data access, search and management 209 capabilities used by Target Applications. These requirements may 210 be met by a new protocol to be defined within the DECADE Working 211 Group. 213 o Storage Requirements: Functional requirements necessary for the 214 back-end storage system employed by the DECADE server. 215 Development of these requirements is guided by a study of the data 216 access patterns used by Target Applications. These requirements 217 should be met by the underling data transport used by DECADE. 219 It should also be made clear that the approach is to make DECADE a 220 simple protocol, while still enabling its usage within many Target 221 Applications. For this reason, and to further reinforce the 222 distinction between DECADE and a storage system, in some cases we 223 also highlight the non-requirements of the protocol. These non- 224 requirements are intended to capture behaviors that will NOT be 225 assumed to be needed by DECADE's Target Applications and hence not 226 present in the DECADE protocol. 228 Finally, some implementation considerations are provided, which while 229 strictly are not requirements, are intended to provide guidance and 230 highlight potential points of concern that need to be considered by 231 the protocol developers, and later by implementors. 233 4. Protocol Requirements 235 4.1. Requirements 237 4.1.1. Overall Protocol Requirements 239 4.1.1.1. Cross-platform Access 241 REQUIREMENT(S): If DECADE supports the ability to store metadata 242 associated with data objects, the DECADE protocol(s) MUST 243 transmit any metadata using an operating system-independent and 244 architecture-independent format. 246 RATIONALE: If DECADE supports the possibility for storing metadata 247 (e.g., a description, uploaded date, object size, or access 248 control list), a possible use for the metadata is to help a 249 DECADE client locate a desired data object. Data objects may be 250 stored by DECADE clients running on various platforms. To enable 251 metadata to be readable regardless of its source it must be 252 transmitted to and from the DECADE server in a standard format. 254 4.1.1.2. Connectivity Concerns 256 4.1.1.2.1. NATs and Firewalls 258 REQUIREMENT(S): DECADE SHOULD be usable across firewalls and NATs 259 without requiring additional network support (e.g., Application- 260 level Gateways). 262 RATIONALE: Firewalls and NATs are widely used in the Internet today, 263 both in ISP networks and within households. Deployment of DECADE 264 must not require modifications to such devices (beyond, perhaps, 265 reconfiguration). Note that this requirement applies to both any 266 new protocol developed by the DECADE Working Group and any data 267 transport used with DECADE. 269 4.1.1.2.2. Connections to Clients 271 REQUIREMENT(S): DECADE SHOULD require that network connections be 272 made from DECADE clients to DECADE servers (i.e., not to the 273 DECADE client). 275 RATIONALE: Many household networks and operating systems have 276 firewalls and NATs configured by default. To ease deployment by 277 avoiding configuration changes and help mitigate security risks, 278 DECADE should not require clients to listen for any incoming 279 network connections (beyond what is required by any other 280 already-deployed application). 282 4.1.1.3. Error and Failure Conditions 284 4.1.1.3.1. Overload Condition 286 REQUIREMENT(S): In-network storage, which is operating close to its 287 capacity limit (e.g., too busy servicing other requests), MUST be 288 able to reject requests and not be required to generate responses 289 to additional requests. 291 RATIONALE: When in-network storage is operating at a limit where it 292 may not be able to process additional requests, it should not be 293 required to generate responses to such additional requests. 294 Forcing the in-network storage to do so can impair its ability to 295 service existing requests. 297 4.1.1.3.2. Insufficient Resources 299 REQUIREMENT(S): DECADE MUST support an error condition indicating to 300 a DECADE client that resources (e.g., storage space) were not 301 available to service a request (e.g., storage quota exceeded when 302 attempting to store data). 304 RATIONALE: The currently-used resource levels within the in-network 305 storage are not locally-discoverable, since the resources (disk, 306 network interfaces, etc) are not directly attached. In order to 307 allocate resources appropriately amongst remote clients, a client 308 must be able to determine when resource limits have been reached. 309 The client can then respond by explicitly freeing necessary 310 resources or waiting for such resources to be freed. 312 4.1.1.3.3. Unavailable and Deleted Data 314 REQUIREMENT(S): DECADE MUST support error conditions indicating that 315 (1) data was rejected from being stored, (2) deleted, or (3) 316 marked unavailable by a storage provider. 318 RATIONALE: Storage providers may require the ability to (1) avoid 319 storing, (2) delete, or (3) quarantine certain data that has been 320 identified as illegal (or otherwise prohibited). DECADE does not 321 indicate how such data is identified, but applications using 322 DECADE should not break if a storage provider is obligated to 323 enforce such policies. Appropriate error conditions should be 324 indicated to applications. 326 4.1.1.3.4. Redirection 328 REQUIREMENT(S): DECADE SHOULD support the ability for a DECADE 329 server to redirect requests to another DECADE server. This may 330 be in response to either an error or failure condition, or to 331 support capabilities such as load balancing. 333 RATIONALE: A DECADE server may opt to redirect requests to another 334 server to support capabilities such as load balancing, or if the 335 implementation decides that another DECADE server is in a better 336 position to handle the request due to either its location in the 337 network, server status, or other consideration. 339 4.1.2. Transfer and Latency Requirements 341 4.1.2.1. Low-Latency Access 343 REQUIREMENT(S): DECADE SHOULD provide "low-latency" access for 344 application clients. DECADE MUST allow clients to specify at 345 least two classes of services for service: lowest possible 346 latency and latency non-critical. 348 RATIONALE: Some applications may have requirements on delivery time 349 (e.g., live streaming [PPLive]). The user experience may be 350 unsatisfactory if the use of in-network storage results in lower 351 performance than connecting directly to remote clients over a 352 low-speed, possibly congested uplink. Additionally, the overhead 353 required for control-plane operations in DECADE must not cause 354 the latency to be higher than for a low-speed, possibly congested 355 uplink. While it is impossible to make a guarantee that a system 356 using in-network storage will always outperform a system that 357 does not for every transfer, the overall performance of the 358 system should be improved compared with direct connections, even 359 considering control overhead. 361 4.1.2.2. Indirect Transfer 363 REQUIREMENT(S): DECADE MUST allow a user's in-network storage to 364 directly fetch from other user's in-network storage. 366 RATIONALE: As an example, a requesting remote client may get the 367 address of the storage pertaining to the source/owner client and 368 then tell its own in-network storage to fetch the content from 369 the source-owner's in-network storage. This helps to avoid extra 370 transfers across ISP network links where possible. 372 4.1.2.3. Data Object Size 374 REQUIREMENT(S): DECADE MUST allow for efficient data transfer of 375 small objects (e.g., 16KB) between a DECADE client and in-network 376 storage with minimal additional latency required by the protocol. 378 RATIONALE: Though Target Applications are frequently used to share 379 large amounts of data (e.g., continuous streams or large files), 380 the data itself is typically subdivided into smaller chunks that 381 are transferred between clients. Additionally, the small chunks 382 may have requirements on delivery time (e.g., in a live-streaming 383 application). DECADE must enable data to be efficiently 384 transferred amongst clients at this granularity. 386 4.1.2.4. Communication among In-network Storage Elements 388 REQUIREMENT(S): DECADE SHOULD support the ability for two in-network 389 storage elements in different administrative domains to store 390 and/or retrieve data directly between each other. If such a 391 capability is supported, this MAY be the same (or a subset or 392 extension of) as the IAP used by clients to access data. 394 RATIONALE: Allowing server-to-server communication can reduce 395 latency in some common scenarios. Consider a scenario when a 396 DECADE client is downloading data into its own storage from 397 another client's in-network storage. One possibility is for the 398 client to first download the data itself, and then upload it to 399 its own storage. However, this causes unnecessary latency and 400 network traffic. Allowing the data to be downloaded from the 401 remote in-network storage into the client's own in-network 402 storage can alleviate both. 404 4.1.3. Data Access Requirements 406 4.1.3.1. Reading/Writing Own Storage 408 REQUIREMENT(S): DECADE MUST support the ability for a DECADE client 409 to read data from and write data to its own in-network storage. 411 RATIONALE: Two basic capabilities for any storage system are reading 412 and writing data. A DECADE client can read data from and write 413 data to in-network storage space that it owns. 415 4.1.3.2. Access by Other Users 416 REQUIREMENT(S): DECADE MUST support the ability for a user to apply 417 access control policies to users other than itself for its 418 storage. The users with whom access is being shared can be under 419 a different administrative domain than the user who owns the in- 420 network storage. The authorized users may read from or write to 421 the user's storage. 423 RATIONALE: Endpoints in Target Applications may be located across 424 multiple ISPs under multiple administrative domains. Thus, to be 425 useful by Target Applications, DECADE allows a user to specify 426 access control policies for users that may or may not be known to 427 the user's storage provider. 429 4.1.3.3. Negotiable Data Protocol 431 REQUIREMENT(S): DECADE MUST support the ability for a DECADE client 432 to negotiate with its in-network storage about which protocol it 433 can use to read data from and write data to its In-network 434 storage. DECADE MUST specify at least one mandatory protocol to 435 be supported by implementations; usage of a different protocol 436 may be selected via negotiation. 438 RATIONALE: Since typical data transport protocols (e.g., NFS and 439 WebDAV) already provide read and write operations for network 440 storage, it may not be necessary for DECADE to define such 441 operations in a new protocol. However, because of the particular 442 application requirements and deployment considerations, different 443 applications may support different protocols. Thus, a DECADE 444 client must be able to select an appropriate protocol also 445 supported by the in-network storage. This requirement also 446 follows as a result of the requirement of Separation of Control 447 and Data Operations (Section 4.1.3.4). 449 4.1.3.4. Separation of Data Operations from Application Control 451 REQUIREMENT(S): The DECADE IAP MUST only provide a minimal set of 452 core operations to support diverse policies implemented and 453 desired by Target Applications. 455 RATIONALE: Target Applications support many complex behaviors and 456 diverse policies to control and distribute data, such as (e.g., 457 search, index, setting permissions/passing authorization tokens). 458 Thus, to support such Target Applications, these behaviors must 459 be logically separated from the data transfer operations (e.g., 460 retrieve, store). Some minimal overlap (for example obtaining 461 credentials needed to encrypt or authorize data transfer using 462 control operations) may be required to be directly specified by 463 DECADE. 465 4.1.4. Data Management Requirements 467 4.1.4.1. Agnostic of reliability 469 REQUIREMENT(S): DECADE SHOULD remain agnostic of reliability/ 470 fault-tolerance level offered by storage provider. 472 RATIONALE: Providers of a DECADE service may wish to offer varying 473 levels of service for different applications/users. However, a 474 single compliant DECADE client should be able to use multiple 475 DECADE services with differing levels of service. 477 4.1.4.2. Time-to-live for Stored Data 479 REQUIREMENT(S): DECADE MUST support the ability for a DECADE client 480 to indicate a time-to-live value (or expiration time) indicating 481 a length of time until particular data can be deleted by the in- 482 network storage element. 484 RATIONALE: Some data stored by a DECADE client may be usable only 485 within a certain window of time, such as in live-streaming P2P 486 applications. Providing a time-to-live value for stored data 487 (e.g., at the time it is stored) can reduce management overhead 488 by avoiding many 'delete' commands sent to in-network storage. 489 The in-network storage may still keep the data in cache for 490 bandwidth optimization. But this is guided by the privacy policy 491 of the DECADE provider. 493 4.1.4.3. Offline Usage 495 REQUIREMENT(S): DECADE MAY support the ability for a user to provide 496 authorized access to its in-network storage even if the user has 497 no DECADE applications actively running or connected to the 498 network. 500 RATIONALE: If an application desires, it can authorize remote 501 clients to access its storage even after the application exits or 502 network connectivity is lost. An example use case is mobile 503 scenarios, where a client can lose and regain network 504 connectivity very often. 506 4.1.5. Resource Control 508 4.1.5.1. Multiple Applications 509 REQUIREMENT(S): DECADE SHOULD support the ability for users to 510 define resource sharing policies for multiple applications 511 (DECADE clients) being run/managed by the user. 513 RATIONALE: A user may own in-network storage and share the in- 514 network storage resources amongst multiple applications. For 515 example, the user may run a video-on-demand application and a 516 live-streaming (or even two different live-streaming 517 applications) application which both make use of the user's in- 518 network storage. The applications may be running on different 519 machines and may not directly communicate. Thus, DECADE should 520 enable the user to determine resource sharing policies between 521 the applications. 523 One possibility is for a user to indicate the particular resource 524 sharing policies between applications out-of-band (not using a 525 standard protocol), but this requirement may manifest itself in 526 passing values over IAP to identify individual applications. 527 Such identifiers can be either user-generated or server-generated 528 and do not need to be registered by IANA. 530 4.1.5.2. Per-Remote-Client, Per-Data Control 532 REQUIREMENT(S): A DECADE client MUST be able to assign resource 533 quotas to individual remote clients for reading from and writing 534 particular data to its in-network storage within a particular 535 range of time. The DECADE server MUST enforce these constraints. 537 RATIONALE: Target Applications can rely on control of resources on a 538 per-remote-client or per-data basis. For example, application 539 policy may indicate that certain remote clients have a higher 540 bandwidth share for receiving data [LLSB08]. Additionally, 541 certain data (e.g., chunks) may be distributed with a higher 542 priority. As another example, when allowing a remote client to 543 write data to a user's in-network storage, the remote client may 544 be restricted to write only a certain amount of data. Since the 545 client may need to manage multiple clients accessing its data, it 546 should be able to indicate the time over which the granted 547 resources are usable. For example, an expiration time for the 548 access could be indicated to the server after which no resources 549 are granted (e.g., indicate error as access denied). 551 4.1.5.3. Server Involvement 552 REQUIREMENT(S): A DECADE client MUST be able to indicate to a DECADE 553 server, without itself contacting the server, resource control 554 policies for remote clients' requests. 556 RATIONALE: One important consideration for in-network storage 557 elements is scalability, since a single storage element may be 558 used to support many users. Many Target Applications use small 559 chunk sizes and frequent data exchanges. If such an application 560 employed resource control and contacted the in-network storage 561 element for each data exchange, this could present a scalability 562 challenge for the server as well as additional latency for 563 clients. 565 An alternative is to let requesting users get the resource 566 control policies and users can then present the policy to the 567 storage directly. This can result in reduced messaging handled 568 by the in-network storage. 570 4.1.6. Authorization 572 4.1.6.1. Per-Remote-Client, Per-Data Read Access 574 REQUIREMENT(S): A DECADE Client MUST be able to control which 575 individual remote clients are authorized to read particular data 576 stored on its in-network storage. 578 RATIONALE: A Target Application can control certain application- 579 level policies by sending particular data (e.g., chunks) to 580 certain remote clients. It is important that remote clients not 581 be able to circumvent such decisions by arbitrarily reading any 582 currently-stored data in in-network storage. 584 4.1.6.2. Per-User Write Access 586 REQUIREMENT(S): A DECADE Client MUST be able to control which 587 individual remote clients are authorized to store data into its 588 in-network storage. 590 RATIONALE: The space managed by a user in in-network storage can be 591 a limited resource. At the same time, it can be useful to allow 592 remote clients to write data directly to a user's in-network 593 storage. Thus, a DECADE client should be able to grant only 594 certain remote clients this privilege. 596 Note that it is not (currently) a requirement to check that a 597 remote client stores a particular set of data (e.g., the check 598 that a remote client writes the expected chunk of a file). 599 Enforcing this as a requirement would require a client to know 600 which data is expected (e.g., the full chunk itself or a hash of 601 the chunk) which may not be available in all applications. 602 Checking for a particular hash could be considered as a 603 requirement in the future that could optionally be employed by 604 applications. 606 4.1.6.3. Authorization Checks 608 REQUIREMENT(S): In-network storage MUST check the authorization of a 609 client before it executes a supplied request. The in-network 610 storage MAY use optimizations to avoid such authorization checks 611 as long as the enforced permissions are the the same. 613 RATIONALE: Authorization granted by a DECADE client are meaningless 614 unless unauthorized requests are denied access. Thus, the in- 615 network storage element must verify the authorization of a 616 particular request before it is executed. 618 4.1.6.4. Credentials Not IP-Based 620 REQUIREMENT(S): Access MUST be able to be granted on other 621 credentials than the IP address 623 RATIONALE: DECADE clients may be operating on hosts without constant 624 network connectivity or without a permanent attachment address 625 (e.g., mobile devices). To support access control with such 626 hosts, DECADE servers must support access control policies that 627 use information other than IP addresses. 629 4.1.6.5. Server Involvement 631 REQUIREMENT(S): A DECADE client MUST be able to indicate, without 632 contacting the server itself, access control policies for remote 633 clients' requests. 635 RATIONALE: See discussion in Section 4.1.5.3. 637 5. Storage Requirements 639 5.1. Requirements 641 5.1.1. Explicit Deletion of Stored Data 642 REQUIREMENT(S): DECADE MUST support the ability for a DECADE client 643 to explicitly delete data from its own in-network storage. 644 DECADE MAY have an overwrite flag indicating that an object with 645 the same name should be replaced. 647 RATIONALE: A DECADE client may continually be writing data to its 648 in-network storage. Since there may be a limit (e.g., imposed by 649 the storage provider) to how much total storage can be used, some 650 data may need to be removed to make room for additional data. A 651 DECADE client should be able to explicitly remove particular 652 data. This may be implemented using existing protocols. 654 5.1.2. Multiple writing 656 REQUIREMENT(S): DECADE MUST NOT allow multiple writers for the same 657 object. Implementations raise an error to one of the writers. 659 RATIONALE: This avoids data corruption in a simple way while 660 remaining efficient. 662 5.1.3. Multiple reading 664 REQUIREMENT(S): DECADE MUST allow for multiple readers for an 665 object. 667 RATIONALE: One characteristic of Target Applications is the ability 668 to upload an object to multiple clients. Thus, it is natural for 669 DECADE to allow multiple readers to read the content 670 concurrently. 672 5.1.4. Reading before completely written 674 REQUIREMENT(S): DECADE MAY allow readers to read from objects before 675 they have been completely written. 677 RATIONALE: Some Target Applications (in particular, P2P streaming) 678 can be sensitive to latency. A technique to reduce latency is to 679 remove store-and-forward delays for data objects (e.g., make the 680 object available before it is completely stored). Appropriate 681 handling for error conditions (e.g., a disappearing writer) needs 682 to be specified. 684 5.1.5. Hints concerning usage stored data 685 REQUIREMENT(S): DECADE MAY allow writers of new objects to indicate 686 specific hints concerning how the objects are expected to be used 687 (e.g., access frequency or time to live). 689 RATIONALE: Different Target Applications may have different usage 690 patterns for objects stored at in-network storage. For example, 691 a P2P live streaming application may indicate to a DECADE server 692 that the objects are expected to have a shore time-to-live, but 693 read frequently. The DECADE server may then opt to store the 694 objects in memory instead of in disk. 696 5.1.6. Writing model 698 REQUIREMENT(S): DECADE MUST provide at least a writing model (while 699 storing an object) that appends data to data already stored. 701 RATIONALE: Depending on the object size (e.g., chunk size) used by a 702 Target Application, the application may need to send data to the 703 DECADE server in multiple packets. To keep implementation 704 simple, the DECADE must at least support the ability to write the 705 data sequentially in the order received. Implementations MAY 706 allow application to write data in an object out-of-order (but 707 MUST NOT overwrite ranges of the object that have already been 708 stored). 710 5.1.7. Storage Status 712 REQUIREMENT(S): A DECADE client MUST be able to retrieve current 713 resource usage (including list of stored data), resource quotas, 714 and access permissions for its in-network storage. The returned 715 information MUST include resource usage resulting from the 716 client's own usage and usage by other clients that have been 717 authorized to read/write objects or open connections to that 718 client's storage. 720 RATIONALE: The resources used by a client are not directly-attached 721 (e.g., disk, network interface, etc). Thus, the client cannot 722 locally determine how such resources are being used. Before 723 storing and retrieving data, a client should be able to determine 724 which data is available (e.g., after an application restart). 725 Additionally, a client should be able to determine resource 726 availability to better allocate them to remote clients. 728 5.2. Non-Requirements 729 5.2.1. No ability to update 731 REQUIREMENT(S): DECADE SHOULD NOT provide ability to update existing 732 objects. That is, objects are immutable once they are stored. 734 RATIONALE: Reasonable consistency models for updating existing 735 objects would significantly complicate implementation (especially 736 if implementation chooses to replicate data across multiple 737 servers). If a user needs to update a resource, it can store a 738 new resource and then distribute the new resource instead of the 739 old one. 741 6. Implementation Considerations 743 The intent of this section is to collect discussion items and 744 implementation considerations that have been discovered as this 745 requirements document has been produced. The content of this section 746 will be migrated to an appropriate place as the document and the 747 Working Group progress. 749 6.1. Resource Scheduling 751 The particular resource scheduling policy may have important 752 ramifications on the performance of applications. This document has 753 explicitly mentioned simultaneous support for both low-latency 754 applications and latency-tolerant applications. 756 Denial of Service attacks may be another risk. For example, 757 rejecting new requests due to overload conditions may introduce the 758 ability to perform a denial of service attack depending on a 759 particular DECADE server's scheduling implementation and resource 760 allocation policies. 762 6.2. Removal of Duplicate Records 764 There are actually two possible scenarios here. The first is the 765 case of removing duplicates within one particular DECADE server (or 766 logical server). While not a requirement, as it does not impact the 767 protocol and is technically not noticeable on message across the 768 wire, a DECADE server may implement internal mechanisms to monitor 769 for duplicate records and use internal mechanisms to prevent 770 duplication of internal storage. 772 The second scenario is removing duplicates across a distributed set 773 of DECADE servers. This is a more difficult problem, and if the 774 group decides to support this capability, it may require protocol 775 support. See Section 7.2 for more details. 777 7. Discussion and Open Issues 779 7.1. Discussion 781 Sometimes, several logical in-network storages could be deployed on 782 the same physical network device. In this case, in-network storages 783 on the same physical network device can communicate and transfer data 784 through internal communication messages. However in-network storages 785 deployed on different physical network devices SHOULD communicate 786 with in-network storage Access Protocol (IAP). 788 To provide fairness among users, the in-network storage provider 789 should assign resource (e.g., storage, bandwidth, connections) quota 790 for users. This can prevent a small number of clients from occupying 791 large amounts of resources on the in-network storage, while others 792 starve. 794 7.2. Open Issues 796 Gaming of the Resource Control Mechanism: There has been some 797 discussion of how applications may be able game the scheduling 798 system by manipulating the resource control mechanism, for 799 example by specifying many small peers to increase total 800 throughput. This is a serious concern, and we need to identify 801 specific requirements on the protocol (hopefully independent of 802 particular scheduling/resource control schemes) to help address 803 this. 805 Discovery: There needs to be some mechanism for a user to discover 806 that there is a DECADE service available for their use, and to 807 locate that server. This is particularly important in the case 808 of mobile applications, since the actual servers that are 809 available at any given time may differ. However, the specifics 810 of what mechanisms (DHCP, HTTP page, etc.) have not been 811 discussed, or even if the protocol should specify one or leave it 812 as an implementation detail. This needs to be defined, and 813 specific requirements formulated if needed. 815 Removal of Duplicate Records Across Servers: If the group wishes to 816 allow for automated mechanisms to remove duplicates across a 817 number of separate servers, some protocol support may need to be 818 added. In the case of removing duplicates within a single 819 (logical) DECADE server, this is simply an implementation 820 concern. See Section 6 for more details. 822 8. Security Considerations 824 Authorization for access to in-network storage is an important part 825 of the requirements listed in this document. Authorization for 826 access to storage resources and the data itself is important for 827 users to be able to manage and limit distribution of content. For 828 example, a user may only wish to share particular content with 829 certain peers. 831 If the authorization technique implemented in DECADE passes any 832 private information (e.g., user passwords) over the wire, it MUST be 833 passed in a secure way. 835 9. IANA Considerations 837 There are no IANA considerations with this document. 839 10. References 841 10.1. Normative References 843 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 844 Requirement Levels", BCP 14, RFC 2119, March 1997. 846 10.2. Informative References 848 [I-D.ietf-decade-problem-statement] 849 Yongchao, S., Zong, N., Yang, Y., and R. Alimi, "DECoupled 850 Application Data Enroute (DECADE) Problem Statement", 851 draft-ietf-decade-problem-statement-00 (work in progress), 852 August 2010. 854 [LLSB08] Dave Levin, Katrina LaCurts, Neil Spring, Bobby 855 Bhattacharjee., "BitTorrent is an Auction: Analyzing and 856 Improving BitTorrent's Incentives", In SIGCOMM 2008. 858 [PPLive] "PPLive", http://www.pplive.com. 860 Appendix A. Acknowledgments 862 We would also like to thank Haibin Song for substantial contributions 863 to earlier versions of this document. We would also like to thank 864 Reinaldo Penno, Alexey Melnikov, Rich Woundy, Ning Zong, Roni Even, 865 David McDysan, Boerje Ohlman and Dirk Kutscher for contributions 866 (including some text used verbatim) and general feedback. 868 Authors' Addresses 870 Yingjie Gu 871 Huawei 872 No. 101 Software Avenue 873 Nanjing, Jiangsu Province 210012 874 P.R.China 876 Phone: +86-25-56624760 877 Email: guyingjie@huawei.com 879 David A. Bryan 880 Cogent Force, LLC / Huawei 882 Email: dbryan@ethernot.org 884 Yang Richard Yang 885 Yale University 887 Email: yry@cs.yale.edu 889 Richard Alimi 890 Google 892 Email: ralimi@google.com