idnits 2.17.1 draft-ietf-core-block-17.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 09, 2015) is 3329 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFCXXXX' is mentioned on line 1298, but not defined -- Obsolete informational reference (is this intentional?): RFC 7230 (Obsoleted by RFC 9110, RFC 9112) -- Obsolete informational reference (is this intentional?): RFC 7233 (Obsoleted by RFC 9110) Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 CoRE Working Group C. Bormann 3 Internet-Draft Universitaet Bremen TZI 4 Intended status: Standards Track Z. Shelby, Ed. 5 Expires: September 10, 2015 ARM 6 March 09, 2015 8 Block-wise transfers in CoAP 9 draft-ietf-core-block-17 11 Abstract 13 CoAP is a RESTful transfer protocol for constrained nodes and 14 networks. Basic CoAP messages work well for the small payloads we 15 expect from temperature sensors, light switches, and similar 16 building-automation devices. Occasionally, however, applications 17 will need to transfer larger payloads -- for instance, for firmware 18 updates. With HTTP, TCP does the grunt work of slicing large 19 payloads up into multiple packets and ensuring that they all arrive 20 and are handled in the right order. 22 CoAP is based on datagram transports such as UDP or DTLS, which 23 limits the maximum size of resource representations that can be 24 transferred without too much fragmentation. Although UDP supports 25 larger payloads through IP fragmentation, it is limited to 64 KiB 26 and, more importantly, doesn't really work well for constrained 27 applications and networks. 29 Instead of relying on IP fragmentation, this specification extends 30 basic CoAP with a pair of "Block" options, for transferring multiple 31 blocks of information from a resource representation in multiple 32 request-response pairs. In many important cases, the Block options 33 enable a server to be truly stateless: the server can handle each 34 block transfer separately, with no need for a connection setup or 35 other server-side memory of previous block transfers. 37 In summary, the Block options provide a minimal way to transfer 38 larger representations in a block-wise fashion. 40 Status of This Memo 42 This Internet-Draft is submitted in full conformance with the 43 provisions of BCP 78 and BCP 79. 45 Internet-Drafts are working documents of the Internet Engineering 46 Task Force (IETF). Note that other groups may also distribute 47 working documents as Internet-Drafts. The list of current Internet- 48 Drafts is at http://datatracker.ietf.org/drafts/current/. 50 Internet-Drafts are draft documents valid for a maximum of six months 51 and may be updated, replaced, or obsoleted by other documents at any 52 time. It is inappropriate to use Internet-Drafts as reference 53 material or to cite them other than as "work in progress." 55 This Internet-Draft will expire on September 10, 2015. 57 Copyright Notice 59 Copyright (c) 2015 IETF Trust and the persons identified as the 60 document authors. All rights reserved. 62 This document is subject to BCP 78 and the IETF Trust's Legal 63 Provisions Relating to IETF Documents 64 (http://trustee.ietf.org/license-info) in effect on the date of 65 publication of this document. Please review these documents 66 carefully, as they describe your rights and restrictions with respect 67 to this document. Code Components extracted from this document must 68 include Simplified BSD License text as described in Section 4.e of 69 the Trust Legal Provisions and are provided without warranty as 70 described in the Simplified BSD License. 72 Table of Contents 74 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 75 2. Block-wise transfers . . . . . . . . . . . . . . . . . . . . 5 76 2.1. The Block2 and Block1 Options . . . . . . . . . . . . . . 5 77 2.2. Structure of a Block Option . . . . . . . . . . . . . . . 6 78 2.3. Block Options in Requests and Responses . . . . . . . . . 8 79 2.4. Using the Block2 Option . . . . . . . . . . . . . . . . . 10 80 2.5. Using the Block1 Option . . . . . . . . . . . . . . . . . 12 81 2.6. Combining Block-wise Transfers with the Observe Option . 13 82 2.7. Combining Block1 and Block2 . . . . . . . . . . . . . . . 14 83 2.8. Combining Block2 with Multicast . . . . . . . . . . . . . 14 84 2.9. Response Codes . . . . . . . . . . . . . . . . . . . . . 14 85 2.9.1. 2.31 Continue . . . . . . . . . . . . . . . . . . . . 15 86 2.9.2. 4.08 Request Entity Incomplete . . . . . . . . . . . 15 87 2.9.3. 4.13 Request Entity Too Large . . . . . . . . . . . . 15 88 2.10. Caching Considerations . . . . . . . . . . . . . . . . . 15 89 3. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 16 90 3.1. Block2 Examples . . . . . . . . . . . . . . . . . . . . . 16 91 3.2. Block1 Examples . . . . . . . . . . . . . . . . . . . . . 20 92 3.3. Combining Block1 and Block2 . . . . . . . . . . . . . . . 21 93 3.4. Combining Observe and Block2 . . . . . . . . . . . . . . 23 94 4. The Size2 and Size1 Options . . . . . . . . . . . . . . . . . 26 95 5. HTTP Mapping Considerations . . . . . . . . . . . . . . . . . 27 96 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 29 97 7. Security Considerations . . . . . . . . . . . . . . . . . . . 29 98 7.1. Mitigating Resource Exhaustion Attacks . . . . . . . . . 30 99 7.2. Mitigating Amplification Attacks . . . . . . . . . . . . 31 100 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 31 101 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 31 102 9.1. Normative References . . . . . . . . . . . . . . . . . . 31 103 9.2. Informative References . . . . . . . . . . . . . . . . . 32 104 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 32 106 1. Introduction 108 The work on Constrained RESTful Environments (CoRE) aims at realizing 109 the REST architecture in a suitable form for the most constrained 110 nodes (such as microcontrollers with limited RAM and ROM [RFC7228]) 111 and networks (such as 6LoWPAN, [RFC4944]) [RFC7252]. The CoAP 112 protocol is intended to provide RESTful [REST] services not unlike 113 HTTP [RFC7230], while reducing the complexity of implementation as 114 well as the size of packets exchanged in order to make these services 115 useful in a highly constrained network of themselves highly 116 constrained nodes. 118 This objective requires restraint in a number of sometimes 119 conflicting ways: 121 o reducing implementation complexity in order to minimize code size, 123 o reducing message sizes in order to minimize the number of 124 fragments needed for each message (in turn to maximize the 125 probability of delivery of the message), the amount of 126 transmission power needed and the loading of the limited-bandwidth 127 channel, 129 o reducing requirements on the environment such as stable storage, 130 good sources of randomness or user interaction capabilities. 132 CoAP is based on datagram transports such as UDP, which limit the 133 maximum size of resource representations that can be transferred 134 without creating unreasonable levels of IP fragmentation. In 135 addition, not all resource representations will fit into a single 136 link layer packet of a constrained network, which may cause 137 adaptation layer fragmentation even if IP layer fragmentation is not 138 required. Using fragmentation (either at the adaptation layer or at 139 the IP layer) for the transport of larger representations would be 140 possible up to the maximum size of the underlying datagram protocol 141 (such as UDP), but the fragmentation/reassembly process burdens the 142 lower layers with conversation state that is better managed in the 143 application layer. 145 The present specification defines a pair of CoAP options to enable 146 _block-wise_ access to resource representations. The Block options 147 provide a minimal way to transfer larger resource representations in 148 a block-wise fashion. The overriding objective is to avoid the need 149 for creating conversation state at the server for block-wise GET 150 requests. (It is impossible to fully avoid creating conversation 151 state for POST/PUT, if the creation/replacement of resources is to be 152 atomic; where that property is not needed, there is no need to create 153 server conversation state in this case, either.) 155 In summary, this specification adds a pair of Block options to CoAP 156 that can be used for block-wise transfers. Benefits of using these 157 options include: 159 o Transfers larger than what can be accommodated in constrained- 160 network link-layer packets can be performed in smaller blocks. 162 o No hard-to-manage conversation state is created at the adaptation 163 layer or IP layer for fragmentation. 165 o The transfer of each block is acknowledged, enabling individual 166 retransmission if required. 168 o Both sides have a say in the block size that actually will be 169 used. 171 o The resulting exchanges are easy to understand using packet 172 analyzer tools and thus quite accessible to debugging. 174 o If needed, the Block options can also be used (without changes) to 175 provide random access to power-of-two sized blocks within a 176 resource representation. 178 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 179 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 180 "OPTIONAL" in this document are to be interpreted as described in RFC 181 2119, BCP 14 [RFC2119] and indicate requirement levels for compliant 182 CoAP implementations. 184 In this document, the term "byte" is used in its now customary sense 185 as a synonym for "octet". 187 Where bit arithmetic is explained, this document uses the notation 188 familiar from the programming language C, except that the operator 189 "**" stands for exponentiation. 191 2. Block-wise transfers 193 As discussed in the introduction, there are good reasons to limit the 194 size of datagrams in constrained networks: 196 o by the maximum datagram size (~ 64 KiB for UDP) 198 o by the desire to avoid IP fragmentation (MTU of 1280 for IPv6) 200 o by the desire to avoid adaptation layer fragmentation (60-80 bytes 201 for 6LoWPAN [RFC4919]) 203 When a resource representation is larger than can be comfortably 204 transferred in the payload of a single CoAP datagram, a Block option 205 can be used to indicate a block-wise transfer. As payloads can be 206 sent both with requests and with responses, this specification 207 provides two separate options for each direction of payload transfer. 208 In identifying these options, we use the number 1 to refer to the 209 transfer of the resource representation that pertains to the request, 210 and the number 2 to refer to the transfer of the resource 211 representation for the response. 213 In the following, the term "payload" will be used for the actual 214 content of a single CoAP message, i.e. a single block being 215 transferred, while the term "body" will be used for the entire 216 resource representation that is being transferred in a block-wise 217 fashion. The Content-Format option applies to the body, not to the 218 payload, in particular the boundaries between the blocks may be in 219 places that are not separating whole units in terms of the structure, 220 encoding, or content-coding used by the Content-Format. 222 In most cases, all blocks being transferred for a body (except for 223 the last one) will be of the same size. The block size is not fixed 224 by the protocol. To keep the implementation as simple as possible, 225 the Block options support only a small range of power-of-two block 226 sizes, from 2**4 (16) to 2**10 (1024) bytes. As bodies often will 227 not evenly divide into the power-of-two block size chosen, the size 228 need not be reached in the final block (but even for the final block, 229 the chosen power-of-two size will still be indicated in the block 230 size field of the Block option). 232 2.1. The Block2 and Block1 Options 233 +-----+---+---+---+---+--------+--------+--------+---------+ 234 | No. | C | U | N | R | Name | Format | Length | Default | 235 +-----+---+---+---+---+--------+--------+--------+---------+ 236 | 23 | C | U | - | - | Block2 | uint | 0-3 | (none) | 237 | | | | | | | | | | 238 | 27 | C | U | - | - | Block1 | uint | 0-3 | (none) | 239 +-----+---+---+---+---+--------+--------+--------+---------+ 241 Table 1: Block Option Numbers 243 Both Block1 and Block2 options can be present both in request and 244 response messages. In either case, the Block1 Option pertains to the 245 request payload, and the Block2 Option pertains to the response 246 payload. 248 Hence, for the methods defined in [RFC7252], Block1 is useful with 249 the payload-bearing POST and PUT requests and their responses. 250 Block2 is useful with GET, POST, and PUT requests and their payload- 251 bearing responses (2.01, 2.02, 2.04, 2.05 -- see section "Payload" of 252 [RFC7252]). 254 Where Block1 is present in a request or Block2 in a response (i.e., 255 in that message to the payload of which it pertains) it indicates a 256 block-wise transfer and describes how this specific block-wise 257 payload forms part of the entire body being transferred ("descriptive 258 usage"). Where it is present in the opposite direction, it provides 259 additional control on how that payload will be formed or was 260 processed ("control usage"). 262 Implementation of either Block option is intended to be optional. 263 However, when it is present in a CoAP message, it MUST be processed 264 (or the message rejected); therefore it is identified as a critical 265 option. It MUST NOT occur more than once. 267 2.2. Structure of a Block Option 269 Three items of information may need to be transferred in a Block 270 (Block1 or Block2) option: 272 o The size of the block (SZX); 274 o whether more blocks are following (M); 276 o the relative number of the block (NUM) within a sequence of blocks 277 with the given size. 279 The value of the Block Option is a variable-size (0 to 3 byte) 280 unsigned integer (uint, see Section 3.2 of [RFC7252]). This integer 281 value encodes these three fields, see Figure 1. (Due to the CoAP 282 uint encoding rules, when all of NUM, M, and SZX happen to be zero, a 283 zero-byte integer will be sent.) 285 0 286 0 1 2 3 4 5 6 7 287 +-+-+-+-+-+-+-+-+ 288 | NUM |M| SZX | 289 +-+-+-+-+-+-+-+-+ 291 0 1 292 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 293 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 294 | NUM |M| SZX | 295 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 297 0 1 2 298 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 299 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 300 | NUM |M| SZX | 301 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 303 Figure 1: Block option value 305 The block size is encoded using a three-bit unsigned integer (0 for 306 2**4 to 6 for 2**10 bytes), which we call the "SZX" ("size 307 exponent"); the actual block size is then "2**(SZX + 4)". SZX is 308 transferred in the three least significant bits of the option value 309 (i.e., "val & 7" where "val" is the value of the option). 311 The fourth least significant bit, the M or "more" bit ("val & 8"), 312 indicates whether more blocks are following or the current block-wise 313 transfer is the last block being transferred. 315 The option value divided by sixteen (the NUM field) is the sequence 316 number of the block currently being transferred, starting from zero. 317 The current transfer is therefore about the "size" bytes starting at 318 byte "NUM << (SZX + 4)". 320 Implementation note: As an implementation convenience, "(val & ~0xF) 321 << (val & 7)", i.e., the option value with the last 4 bits masked 322 out, shifted to the left by the value of SZX, gives the byte 323 position of the first byte of the block being transferred. 325 More specifically, within the option value of a Block1 or Block2 326 Option, the meaning of the option fields is defined as follows: 328 NUM: Block Number, indicating the block number being requested or 329 provided. Block number 0 indicates the first block of a body 330 (i.e., starting with the first byte of the body). 332 M: More Flag ("not last block"). For descriptive usage, this flag, 333 if unset, indicates that the payload in this message is the last 334 block in the body; when set it indicates that there are one or 335 more additional blocks available. When a Block2 Option is used in 336 a request to retrieve a specific block number ("control usage"), 337 the M bit MUST be sent as zero and ignored on reception. (In a 338 Block1 Option in a response, the M flag is used to indicate 339 atomicity, see below.) 341 SZX: Block Size. The block size is represented as three-bit 342 unsigned integer indicating the size of a block to the power of 343 two. Thus block size = 2**(SZX + 4). The allowed values of SZX 344 are 0 to 6, i.e., the minimum block size is 2**(0+4) = 16 and the 345 maximum is 2**(6+4) = 1024. The value 7 for SZX (which would 346 indicate a block size of 2048) is reserved, i.e. MUST NOT be sent 347 and MUST lead to a 4.00 Bad Request response code upon reception 348 in a request. 350 There is no default value for the Block1 and Block2 Options. Absence 351 of one of these options is equivalent to an option value of 0 with 352 respect to the value of NUM and M that could be given in the option, 353 i.e. it indicates that the current block is the first and only block 354 of the transfer (block number 0, M bit not set). However, in 355 contrast to the explicit value 0, which would indicate an SZX of 0 356 and thus a size value of 16 bytes, there is no specific explicit size 357 implied by the absence of the option -- the size is left unspecified. 358 (As for any uint, the explicit value 0 is efficiently indicated by a 359 zero-length option; this, therefore, is different in semantics from 360 the absence of the option.) 362 2.3. Block Options in Requests and Responses 364 The Block options are used in one of three roles: 366 o In descriptive usage, i.e., a Block2 Option in a response (such as 367 a 2.05 response for GET), or a Block1 Option in a request (a PUT 368 or POST): 370 * The NUM field in the option value describes what block number 371 is contained in the payload of this message. 373 * The M bit indicates whether further blocks need to be 374 transferred to complete the transfer of that body. 376 * The block size implied by SZX MUST match the size of the 377 payload in bytes, if the M bit is set. (SZX does not govern 378 the payload size if M is unset). For Block2, if the request 379 suggested a larger value of SZX, the next request MUST move SZX 380 down to the size given in the response. (The effect is that, 381 if the server uses the smaller of (1) its preferred block size 382 and (2) the block size requested, all blocks for a body use the 383 same block size.) 385 o A Block2 Option in control usage in a request (e.g., GET): 387 * The NUM field in the Block2 Option gives the block number of 388 the payload that is being requested to be returned in the 389 response. 391 * In this case, the M bit has no function and MUST be set to 392 zero. 394 * The block size given (SZX) suggests a block size (in the case 395 of block number 0) or repeats the block size of previous blocks 396 received (in the case of a non-zero block number). 398 o A Block1 Option in control usage in a response (e.g., a 2.xx 399 response for a PUT or POST request): 401 * The NUM field of the Block1 Option indicates what block number 402 is being acknowledged. 404 * If the M bit was set in the request, the server can choose 405 whether to act on each block separately, with no memory, or 406 whether to handle the request for the entire body atomically, 407 or any mix of the two. 409 + If the M bit is also set in the response, it indicates that 410 this response does not carry the final response code to the 411 request, i.e. the server collects further blocks from the 412 same endpoint and plans to implement the request atomically 413 (e.g., acts only upon reception of the last block of 414 payload). In this case, the response MUST NOT carry a 415 Block2 option. 417 + Conversely, if the M bit is unset even though it was set in 418 the request, it indicates the block-wise request was enacted 419 now specifically for this block, and the response carries 420 the final response to this request (and to any previous ones 421 with the M bit set in the response's Block1 Option in this 422 sequence of block-wise transfers); the client is still 423 expected to continue sending further blocks, the request 424 method for which may or may not also be enacted per-block. 426 * Finally, the SZX block size given in a control Block1 Option 427 indicates the largest block size preferred by the server for 428 transfers toward the resource that is the same or smaller than 429 the one used in the initial exchange; the client SHOULD use 430 this block size or a smaller one in all further requests in the 431 transfer sequence, even if that means changing the block size 432 (and possibly scaling the block number accordingly) from now 433 on. 435 Using one or both Block options, a single REST operation can be split 436 into multiple CoAP message exchanges. As specified in [RFC7252], 437 each of these message exchanges uses their own CoAP Message ID. 439 The Content-Format Option sent with the requests or responses MUST 440 reflect the content-format of the entire body. If blocks of a 441 response body arrive with different content-format options, it is up 442 to the client how to handle this error (it will typically abort any 443 ongoing block-wise transfer). If blocks of a request arrive at a 444 server with mismatching content-format options, the server MUST NOT 445 assemble them into a single request; this usually leads to a 4.08 446 (Request Entity Incomplete, Section 2.9.2) error response on the 447 mismatching block. 449 2.4. Using the Block2 Option 451 When a request is answered with a response carrying a Block2 Option 452 with the M bit set, the requester may retrieve additional blocks of 453 the resource representation by sending further requests with the same 454 options as the initial request and a Block2 Option giving the block 455 number and block size desired. In a request, the client MUST set the 456 M bit of a Block2 Option to zero and the server MUST ignore it on 457 reception. 459 To influence the block size used in a response, the requester MAY 460 also use the Block2 Option on the initial request, giving the desired 461 size, a block number of zero and an M bit of zero. A server MUST use 462 the block size indicated or a smaller size. Any further block-wise 463 requests for blocks beyond the first one MUST indicate the same block 464 size that was used by the server in the response for the first 465 request that gave a desired size using a Block2 Option. 467 Once the Block2 Option is used by the requester and a first response 468 has been received with a possibly adjusted block size, all further 469 requests in a single block-wise transfer SHOULD ultimately use the 470 same size, except that there may not be enough content to fill the 471 last block (the one returned with the M bit not set). (Note that the 472 client may start using the Block2 Option in a second request after a 473 first request without a Block2 Option resulted in a Block2 option in 474 the response.) The server SHOULD use the block size indicated in the 475 request option or a smaller size, but the requester MUST take note of 476 the actual block size used in the response it receives to its initial 477 request and proceed to use it in subsequent requests. The server 478 behavior MUST ensure that this client behavior results in the same 479 block size for all responses in a sequence (except for the last one 480 with the M bit not set, and possibly the first one if the initial 481 request did not contain a Block2 Option). 483 Block-wise transfers can be used to GET resources the representations 484 of which are entirely static (not changing over time at all, such as 485 in a schema describing a device), or for dynamically changing 486 resources. In the latter case, the Block2 Option SHOULD be used in 487 conjunction with the ETag Option, to ensure that the blocks being 488 reassembled are from the same version of the representation: The 489 server SHOULD include an ETag option in each response. If an ETag 490 option is available, the client's reassembler, when reassembling the 491 representation from the blocks being exchanged, MUST compare ETag 492 Options. If the ETag Options do not match in a GET transfer, the 493 requester has the option of attempting to retrieve fresh values for 494 the blocks it retrieved first. To minimize the resulting 495 inefficiency, the server MAY cache the current value of a 496 representation for an ongoing sequence of requests. (The server may 497 identify the sequence by the combination of the requesting end-point 498 and the URI being the same in each block-wise request.) Note well 499 that this specification makes no requirement for the server to 500 establish any state; however, servers that offer quickly changing 501 resources may thereby make it impossible for a client to ever 502 retrieve a consistent set of blocks. Clients that want to retrieve 503 all blocks of a resource SHOULD strive to do so without undue delay. 504 Servers can fully expect to be free to discard any cached state after 505 a period of EXCHANGE_LIFETIME ([RFC7252], Section 4.8.2) after the 506 last access to the state, however, there is no requirement to always 507 keep the state for as long. 509 The Block2 option provides no way for a single endpoint to perform 510 multiple concurrently proceeding block-wise response payload transfer 511 (e.g., GET) operations to the same resource. This is rarely a 512 requirement, but as a workaround, a client may vary the cache key 513 (e.g., by using one of several URIs accessing resources with the same 514 semantics, or by varying a proxy-safe elective option). 516 2.5. Using the Block1 Option 518 In a request with a request payload (e.g., PUT or POST), the Block1 519 Option refers to the payload in the request (descriptive usage). 521 In response to a request with a payload (e.g., a PUT or POST 522 transfer), the block size given in the Block1 Option indicates the 523 block size preference of the server for this resource (control 524 usage). Obviously, at this point the first block has already been 525 transferred by the client without benefit of this knowledge. Still, 526 the client SHOULD heed the preference indicated and, for all further 527 blocks, use the block size preferred by the server or a smaller one. 528 Note that any reduction in the block size may mean that the second 529 request starts with a block number larger than one, as the first 530 request already transferred multiple blocks as counted in the smaller 531 size. 533 To counter the effects of adaptation layer fragmentation on packet 534 delivery probability, a client may want to give up retransmitting a 535 request with a relatively large payload even before MAX_RETRANSMIT 536 has been reached, and try restating the request as a block-wise 537 transfer with a smaller payload. Note that this new attempt is then 538 a new message-layer transaction and requires a new Message ID. 539 (Because of the uncertainty whether the request or the 540 acknowledgement was lost, this strategy is useful mostly for 541 idempotent requests.) 543 In a block-wise transfer of a request payload (e.g., a PUT or POST) 544 that is intended to be implemented in an atomic fashion at the 545 server, the actual creation/replacement takes place at the time the 546 final block, i.e. a block with the M bit unset in the Block1 Option, 547 is received. In this case, all success responses to non-final blocks 548 carry the response code 2.31 (Continue, Section 2.9.1). If not all 549 previous blocks are available at the server at the time of processing 550 the final block, the transfer fails and error code 4.08 (Request 551 Entity Incomplete, Section 2.9.2) MUST be returned. A server MAY 552 also return a 4.08 error code for any (final or non-final) Block1 553 transfer that is not in sequence; clients that do not have specific 554 mechanisms to handle this case therefore SHOULD always start with 555 block zero and send the following blocks in order. 557 One reason that a client might encounter a 4.08 error code is that 558 the server has already timed out and discarded the partial request 559 body being assembled. Clients SHOULD strive to send all blocks of a 560 request without undue delay. Servers can fully expect to be free to 561 discard any partial request body when a period of EXCHANGE_LIFETIME 562 ([RFC7252], Section 4.8.2) has elapsed after the most recent block 563 was transferred; however, there is no requirement on a server to 564 always keep the partial request body for as long. 566 The error code 4.13 (Request Entity Too Large) can be returned at any 567 time by a server that does not currently have the resources to store 568 blocks for a block-wise request payload transfer that it would intend 569 to implement in an atomic fashion. (Note that a 4.13 response to a 570 request that does not employ Block1 is a hint for the client to try 571 sending Block1, and a 4.13 response with a smaller SZX in its Block1 572 option than requested is a hint to try a smaller SZX.) 574 The Block1 option provides no way for a single endpoint to perform 575 multiple concurrently proceeding block-wise request payload transfer 576 (e.g., PUT or POST) operations to the same resource. Starting a new 577 block-wise sequence of requests to the same resource (before an old 578 sequence from the same endpoint was finished) simply overwrites the 579 context the server may still be keeping. (This is probably exactly 580 what one wants in this case - the client may simply have restarted 581 and lost its knowledge of the previous sequence.) 583 2.6. Combining Block-wise Transfers with the Observe Option 585 The Observe Option provides a way for a client to be notified about 586 changes over time of a resource [I-D.ietf-core-observe]. Resources 587 observed by clients may be larger than can be comfortably processed 588 or transferred in one CoAP message. The following rules apply to the 589 combination of block-wise transfers with notifications. 591 Observation relationships always apply to an entire resource; the 592 Block2 option does not provide a way to observe a single block of a 593 resource. 595 As with basic GET transfers, the client can indicate its desired 596 block size in a Block2 Option in the GET request establishing or 597 renewing the observation relationship. If the server supports block- 598 wise transfers, it SHOULD take note of the block size and apply it as 599 a maximum size to all notifications/responses resulting from the GET 600 request (until the client is removed from the list of observers or 601 the entry in that list is updated by the server receiving a new GET 602 request for the resource from the client). 604 When sending a 2.05 (Content) notification, the server only sends the 605 first block of the representation. The client retrieves the rest of 606 the representation as if it had caused this first response by a GET 607 request, i.e., by using additional GET requests with Block2 options 608 containing NUM values greater than zero. (This results in the 609 transfer of the entire representation, even if only some of the 610 blocks have changed with respect to a previous notification.) 611 As with other dynamically changing resources, to ensure that the 612 blocks being reassembled are from the same version of the 613 representation, the server SHOULD include an ETag option in each 614 response, and the reassembling client MUST compare the ETag options 615 (Section 2.4). Even more so than for the general case of Block2, 616 clients that want to retrieve all blocks of a resource they have been 617 notified about with a first block SHOULD strive to do so without 618 undue delay. 620 See Section 3.4 for examples. 622 2.7. Combining Block1 and Block2 624 In PUT and particularly in POST exchanges, both the request body and 625 the response body may be large enough to require the use of block- 626 wise transfers. First, the Block1 transfer of the request body 627 proceeds as usual. In the exchange of the last slice of this block- 628 wise transfer, the response carries the first slice of the Block2 629 transfer (NUM is zero). To continue this Block2 transfer, the client 630 continues to send requests similar to the requests in the Block1 631 phase, but leaves out the Block1 options and includes a Block2 632 request option with non-zero NUM. 634 Block2 transfers that retrieve the response body for a request that 635 used Block1 MUST be performed in sequential order. 637 2.8. Combining Block2 with Multicast 639 A client can use the Block2 option in a multicast GET request with 640 NUM = 0 to aid in limiting the size of the response. 642 Similarly, a response to a multicast GET request can use a Block2 643 option with NUM = 0 if the representation is large, or to further 644 limit the size of the response. 646 In both cases, the client retrieves any further blocks using unicast 647 exchanges; in the unicast requests, the client SHOULD heed any block 648 size preferences indicated by the server in the response to the 649 multicast request. 651 Other uses of the Block options in conjunction with multicast 652 messages are for further study. 654 2.9. Response Codes 656 Two response codes are defined by this specification beyond those 657 already defined in [RFC7252], and another response code is extended 658 in its meaning. 660 2.9.1. 2.31 Continue 662 This new success status code indicates that the transfer of this 663 block of the request body was successful and that the server 664 encourages sending further blocks, but that a final outcome of the 665 whole block-wise request cannot yet be determined. No payload is 666 returned with this response code. 668 2.9.2. 4.08 Request Entity Incomplete 670 This new client error status code indicates that the server has not 671 received the blocks of the request body that it needs to proceed. 672 The client has not sent all blocks, not sent them in the order 673 required by the server, or has sent them long enough ago that the 674 server has already discarded them. 676 2.9.3. 4.13 Request Entity Too Large 678 In [RFC7252], section 5.9.2.9, the response code 4.13 (Request Entity 679 Too Large) is defined to be like HTTP 413 "Request Entity Too Large". 680 [RFC7252] also recommends that this response SHOULD include a Size1 681 Option (Section 4) to indicate the maximum size of request entity the 682 server is able and willing to handle, unless the server is not in a 683 position to make this information available. 685 The present specification allows the server to return this response 686 code at any time during a Block1 transfer to indicate that it does 687 not currently have the resources to store blocks for a transfer that 688 it would intend to implement in an atomic fashion. It also allows 689 the server to return a 4.13 response to a request that does not 690 employ Block1 as a hint for the client to try sending Block1. 691 Finally, a 4.13 response to a request with a Block1 option (control 692 usage, see Section 2.3) where the response carries a smaller SZX in 693 its Block1 option is a hint to try that smaller SZX. 695 2.10. Caching Considerations 697 This specification attempts to leave a variety of implementation 698 strategies open for caches, in particular those in caching proxies. 699 E.g., a cache is free to cache blocks individually, but also could 700 wait to obtain the complete representation before it serves parts of 701 it. Partial caching may be more efficient in a cross-proxy 702 (equivalent to a streaming HTTP proxy). A cached block (partial 703 cached response) can be used in place of a complete response to 704 satisfy a block-wise request that is presented to a cache. Note that 705 different blocks can have different Max-Age values, as they are 706 transferred at different times. A response with a block updates the 707 freshness of the complete representation. Individual blocks can be 708 validated, and validating a single block validates the complete 709 representation. A response with a Block1 Option in control usage 710 with the M bit set invalidates cached responses for the target URI. 712 A cache or proxy that combines responses (e.g., to split blocks in a 713 request or increase the block size in a response, or a cross-proxy) 714 may need to combine 2.31 and 2.01/2.04 responses; a stateless server 715 may be responding with 2.01 only on the first Block1 block 716 transferred, which dominates any 2.04 responses for later blocks. 718 If-None-Match only works correctly on Block1 requests with (NUM=0) 719 and MUST NOT be used on Block1 requests with NUM != 0. 721 3. Examples 723 This section gives a number of short examples with message flows for 724 a block-wise GET, and for a PUT or POST. These examples demonstrate 725 the basic operation, the operation in the presence of 726 retransmissions, and examples for the operation of the block size 727 negotiation. 729 In all these examples, a Block option is shown in a decomposed way 730 indicating the kind of Block option (1 or 2) followed by a colon, and 731 then the block number (NUM), more bit (M), and block size exponent 732 (2**(SZX+4)) separated by slashes. E.g., a Block2 Option value of 33 733 would be shown as 2:2/0/32), or a Block1 Option value of 59 would be 734 shown as 1:3/1/128. 736 3.1. Block2 Examples 738 The first example (Figure 2) shows a GET request that is split into 739 three blocks. The server proposes a block size of 128, and the 740 client agrees. The first two ACKs contain 128 bytes of payload each, 741 and third ACK contains between 1 and 128 bytes. 743 CLIENT SERVER 744 | | 745 | CON [MID=1234], GET, /status ------> | 746 | | 747 | <------ ACK [MID=1234], 2.05 Content, 2:0/1/128 | 748 | | 749 | CON [MID=1235], GET, /status, 2:1/0/128 ------> | 750 | | 751 | <------ ACK [MID=1235], 2.05 Content, 2:1/1/128 | 752 | | 753 | CON [MID=1236], GET, /status, 2:2/0/128 ------> | 754 | | 755 | <------ ACK [MID=1236], 2.05 Content, 2:2/0/128 | 757 Figure 2: Simple block-wise GET 759 In the second example (Figure 3), the client anticipates the block- 760 wise transfer (e.g., because of a size indication in the link-format 761 description [RFC6690]) and sends a block size proposal. All ACK 762 messages except for the last carry 64 bytes of payload; the last one 763 carries between 1 and 64 bytes. 765 CLIENT SERVER 766 | | 767 | CON [MID=1234], GET, /status, 2:0/0/64 ------> | 768 | | 769 | <------ ACK [MID=1234], 2.05 Content, 2:0/1/64 | 770 | | 771 | CON [MID=1235], GET, /status, 2:1/0/64 ------> | 772 | | 773 | <------ ACK [MID=1235], 2.05 Content, 2:1/1/64 | 774 : : 775 : ... : 776 : : 777 | CON [MID=1238], GET, /status, 2:4/0/64 ------> | 778 | | 779 | <------ ACK [MID=1238], 2.05 Content, 2:4/1/64 | 780 | | 781 | CON [MID=1239], GET, /status, 2:5/0/64 ------> | 782 | | 783 | <------ ACK [MID=1239], 2.05 Content, 2:5/0/64 | 785 Figure 3: Block-wise GET with early negotiation 787 In the third example (Figure 4), the client is surprised by the need 788 for a block-wise transfer, and unhappy with the size chosen 789 unilaterally by the server. As it did not send a size proposal 790 initially, the negotiation only influences the size from the second 791 message exchange onward. Since the client already obtained both the 792 first and second 64-byte block in the first 128-byte exchange, it 793 goes on requesting the third 64-byte block ("2/0/64"). None of this 794 is (or needs to be) understood by the server, which simply responds 795 to the requests as it best can. 797 CLIENT SERVER 798 | | 799 | CON [MID=1234], GET, /status ------> | 800 | | 801 | <------ ACK [MID=1234], 2.05 Content, 2:0/1/128 | 802 | | 803 | CON [MID=1235], GET, /status, 2:2/0/64 ------> | 804 | | 805 | <------ ACK [MID=1235], 2.05 Content, 2:2/1/64 | 806 | | 807 | CON [MID=1236], GET, /status, 2:3/0/64 ------> | 808 | | 809 | <------ ACK [MID=1236], 2.05 Content, 2:3/1/64 | 810 | | 811 | CON [MID=1237], GET, /status, 2:4/0/64 ------> | 812 | | 813 | <------ ACK [MID=1237], 2.05 Content, 2:4/1/64 | 814 | | 815 | CON [MID=1238], GET, /status, 2:5/0/64 ------> | 816 | | 817 | <------ ACK [MID=1238], 2.05 Content, 2:5/0/64 | 819 Figure 4: Block-wise GET with late negotiation 821 In all these (and the following) cases, retransmissions are handled 822 by the CoAP message exchange layer, so they don't influence the block 823 operations (Figure 5, Figure 6). 825 CLIENT SERVER 826 | | 827 | CON [MID=1234], GET, /status ------> | 828 | | 829 | <------ ACK [MID=1234], 2.05 Content, 2:0/1/128 | 830 | | 831 | CON [MID=1235], GE///////////////////////// | 832 | | 833 | (timeout) | 834 | | 835 | CON [MID=1235], GET, /status, 2:2/0/64 ------> | 836 | | 837 | <------ ACK [MID=1235], 2.05 Content, 2:2/1/64 | 838 : : 839 : ... : 840 : : 841 | CON [MID=1238], GET, /status, 2:5/0/64 ------> | 842 | | 843 | <------ ACK [MID=1238], 2.05 Content, 2:5/0/64 | 845 Figure 5: Block-wise GET with late negotiation and lost CON 847 CLIENT SERVER 848 | | 849 | CON [MID=1234], GET, /status ------> | 850 | | 851 | <------ ACK [MID=1234], 2.05 Content, 2:0/1/128 | 852 | | 853 | CON [MID=1235], GET, /status, 2:2/0/64 ------> | 854 | | 855 | //////////////////////////////////tent, 2:2/1/64 | 856 | | 857 | (timeout) | 858 | | 859 | CON [MID=1235], GET, /status, 2:2/0/64 ------> | 860 | | 861 | <------ ACK [MID=1235], 2.05 Content, 2:2/1/64 | 862 : : 863 : ... : 864 : : 865 | CON [MID=1238], GET, /status, 2:5/0/64 ------> | 866 | | 867 | <------ ACK [MID=1238], 2.05 Content, 2:5/0/64 | 869 Figure 6: Block-wise GET with late negotiation and lost ACK 871 3.2. Block1 Examples 873 The following examples demonstrate a PUT exchange; a POST exchange 874 looks the same, with different requirements on atomicity/idempotence. 875 Note that, similar to GET, the responses to the requests that have a 876 more bit in the request Block1 Option are provisional and carry the 877 response code 2.31 (Continue); only the final response tells the 878 client that the PUT did succeed. 880 CLIENT SERVER 881 | | 882 | CON [MID=1234], PUT, /options, 1:0/1/128 ------> | 883 | | 884 | <------ ACK [MID=1234], 2.31 Continue, 1:0/1/128 | 885 | | 886 | CON [MID=1235], PUT, /options, 1:1/1/128 ------> | 887 | | 888 | <------ ACK [MID=1235], 2.31 Continue, 1:1/1/128 | 889 | | 890 | CON [MID=1236], PUT, /options, 1:2/0/128 ------> | 891 | | 892 | <------ ACK [MID=1236], 2.04 Changed, 1:2/0/128 | 894 Figure 7: Simple atomic block-wise PUT 896 A stateless server that simply builds/updates the resource in place 897 (statelessly) may indicate this by not setting the more bit in the 898 response (Figure 8); in this case, the response codes are valid 899 separately for each block being updated. This is of course only an 900 acceptable behavior of the server if the potential inconsistency 901 present during the run of the message exchange sequence does not lead 902 to problems, e.g. because the resource being created or changed is 903 not yet or not currently in use. 905 CLIENT SERVER 906 | | 907 | CON [MID=1234], PUT, /options, 1:0/1/128 ------> | 908 | | 909 | <------ ACK [MID=1234], 2.04 Changed, 1:0/0/128 | 910 | | 911 | CON [MID=1235], PUT, /options, 1:1/1/128 ------> | 912 | | 913 | <------ ACK [MID=1235], 2.04 Changed, 1:1/0/128 | 914 | | 915 | CON [MID=1236], PUT, /options, 1:2/0/128 ------> | 916 | | 917 | <------ ACK [MID=1236], 2.04 Changed, 1:2/0/128 | 919 Figure 8: Simple stateless block-wise PUT 921 Finally, a server receiving a block-wise PUT or POST may want to 922 indicate a smaller block size preference (Figure 9). In this case, 923 the client SHOULD continue with a smaller block size; if it does, it 924 MUST adjust the block number to properly count in that smaller size. 926 CLIENT SERVER 927 | | 928 | CON [MID=1234], PUT, /options, 1:0/1/128 ------> | 929 | | 930 | <------ ACK [MID=1234], 2.31 Continue, 1:0/1/32 | 931 | | 932 | CON [MID=1235], PUT, /options, 1:4/1/32 ------> | 933 | | 934 | <------ ACK [MID=1235], 2.31 Continue, 1:4/1/32 | 935 | | 936 | CON [MID=1236], PUT, /options, 1:5/1/32 ------> | 937 | | 938 | <------ ACK [MID=1235], 2.31 Continue, 1:5/1/32 | 939 | | 940 | CON [MID=1237], PUT, /options, 1:6/0/32 ------> | 941 | | 942 | <------ ACK [MID=1236], 2.04 Changed, 1:6/0/32 | 944 Figure 9: Simple atomic block-wise PUT with negotiation 946 3.3. Combining Block1 and Block2 948 Block options may be used in both directions of a single exchange. 949 The following example demonstrates a block-wise POST request, 950 resulting in a separate block-wise response. 952 CLIENT SERVER 953 | | 954 | CON [MID=1234], POST, /soap, 1:0/1/128 ------> | 955 | | 956 | <------ ACK [MID=1234], 2.31 Continue, 1:0/1/128 | 957 | | 958 | CON [MID=1235], POST, /soap, 1:1/1/128 ------> | 959 | | 960 | <------ ACK [MID=1235], 2.31 Continue, 1:1/1/128 | 961 | | 962 | CON [MID=1236], POST, /soap, 1:2/0/128 ------> | 963 | | 964 | <------ ACK [MID=1236], 2.04 Changed, 2:0/1/128, 1:2/0/128 | 965 | | 966 | CON [MID=1237], POST, /soap, 2:1/0/128 ------> | 967 | (no payload for requests with Block2 with NUM != 0) | 968 | (could also do late negotiation by requesting e.g. 2:2/0/64) | 969 | | 970 | <------ ACK [MID=1237], 2.04 Changed, 2:1/1/128 | 971 | | 972 | CON [MID=1238], POST, /soap, 2:2/0/128 ------> | 973 | | 974 | <------ ACK [MID=1238], 2.04 Changed, 2:2/1/128 | 975 | | 976 | CON [MID=1239], POST, /soap, 2:3/0/128 ------> | 977 | | 978 | <------ ACK [MID=1239], 2.04 Changed, 2:3/0/128 | 980 Figure 10: Atomic block-wise POST with block-wise response 982 This model does provide for early negotiation input to the Block2 983 block-wise transfer, as shown below. 985 CLIENT SERVER 986 | | 987 | CON [MID=1234], POST, /soap, 1:0/1/128 ------> | 988 | | 989 | <------ ACK [MID=1234], 2.31 Continue, 1:0/1/128 | 990 | | 991 | CON [MID=1235], POST, /soap, 1:1/1/128 ------> | 992 | | 993 | <------ ACK [MID=1235], 2.31 Continue, 1:1/1/128 | 994 | | 995 | CON [MID=1236], POST, /soap, 1:2/0/128, 2:0/0/64 ------> | 996 | | 997 | <------ ACK [MID=1236], 2.04 Changed, 1:2/0/128, 2:0/1/64 | 998 | | 999 | CON [MID=1237], POST, /soap, 2:1/0/64 ------> | 1000 | (no payload for requests with Block2 with NUM != 0) | 1001 | | 1002 | <------ ACK [MID=1237], 2.04 Changed, 2:1/1/64 | 1003 | | 1004 | CON [MID=1238], POST, /soap, 2:2/0/64 ------> | 1005 | | 1006 | <------ ACK [MID=1238], 2.04 Changed, 2:2/1/64 | 1007 | | 1008 | CON [MID=1239], POST, /soap, 2:3/0/64 ------> | 1009 | | 1010 | <------ ACK [MID=1239], 2.04 Changed, 2:3/0/64 | 1012 Figure 11: Atomic block-wise POST with block-wise response, early 1013 negotiation 1015 3.4. Combining Observe and Block2 1017 In the following example, the server first sends a direct response 1018 (Observe sequence number 62350) to the initial GET request (the 1019 resulting block-wise transfer is as in Figure 4 and has therefore 1020 been left out). The second transfer is started by a 2.05 1021 notification that contains just the first block (Observe sequence 1022 number 62354); the client then goes on to obtain the rest of the 1023 blocks. 1025 CLIENT SERVER 1026 | | 1027 +----->| Header: GET 0x41011636 1028 | GET | Token: 0xfb 1029 | | Uri-Path: status-icon 1030 | | Observe: (empty) 1031 | | 1032 |<-----+ Header: 2.05 0x61451636 1033 | 2.05 | Token: 0xfb 1034 | | Block2: 0/1/128 1035 | | Observe: 62350 1036 | | ETag: 6f00f38e 1037 | | Payload: [128 bytes] 1038 | | 1039 | | (Usual GET transfer left out) 1040 ... 1041 | | (Notification of first block:) 1042 | | 1043 |<-----+ Header: 2.05 0x4145af9c 1044 | 2.05 | Token: 0xfb 1045 | | Block2: 0/1/128 1046 | | Observe: 62354 1047 | | ETag: 6f00f392 1048 | | Payload: [128 bytes] 1049 | | 1050 +- - ->| Header: 0x6000af9c 1051 | | 1052 | | (Retrieval of remaining blocks) 1053 | | 1054 +----->| Header: GET 0x41011637 1055 | GET | Token: 0xfc 1056 | | Uri-Path: status-icon 1057 | | Block2: 1/0/128 1058 | | 1059 |<-----+ Header: 2.05 0x61451637 1060 | 2.05 | Token: 0xfc 1061 | | Block2: 1/1/128 1062 | | ETag: 6f00f392 1063 | | Payload: [128 bytes] 1064 | | 1065 +----->| Header: GET 0x41011638 1066 | GET | Token: 0xfc 1067 | | Uri-Path: status-icon 1068 | | Block2: 2/0/128 1069 | | 1070 |<-----+ Header: 2.05 0x61451638 1071 | 2.05 | Token: 0xfc 1072 | | Block2: 2/0/128 1073 | | ETag: 6f00f392 1074 | | Payload: [53 bytes] 1076 Figure 12: Observe sequence with block-wise response 1078 (Note that the choice of token 0xfc in this examples is arbitrary; 1079 tokens are just shown in this example to illustrate that the requests 1080 for additional blocks cannot make use of the token of the Observation 1081 relationship. As a general comment on tokens, there is no other 1082 mention of tokens in this document, as block-wise transfers handle 1083 tokens like any other CoAP exchange. As usual the client is free to 1084 choose tokens for each exchange as it likes.) 1086 In the following example, the client also uses early negotiation to 1087 limit the block size to 64 bytes. 1089 CLIENT SERVER 1090 | | 1091 +----->| Header: GET 0x41011636 1092 | GET | Token: 0xfb 1093 | | Uri-Path: status-icon 1094 | | Observe: (empty) 1095 | | Block2: 0/0/64 1096 | | 1097 |<-----+ Header: 2.05 0x61451636 1098 | 2.05 | Token: 0xfb 1099 | | Block2: 0/1/64 1100 | | Observe: 62350 1101 | | ETag: 6f00f38e 1102 | | Max-Age: 60 1103 | | Payload: [64 bytes] 1104 | | 1105 | | (Usual GET transfer left out) 1106 ... 1107 | | (Notification of first block:) 1108 | | 1109 |<-----+ Header: 2.05 0x4145af9c 1110 | 2.05 | Token: 0xfb 1111 | | Block2: 0/1/64 1112 | | Observe: 62354 1113 | | ETag: 6f00f392 1114 | | Payload: [64 bytes] 1115 | | 1116 +- - ->| Header: 0x6000af9c 1117 | | 1118 | | (Retrieval of remaining blocks) 1119 | | 1120 +----->| Header: GET 0x41011637 1121 | GET | Token: 0xfc 1122 | | Uri-Path: status-icon 1123 | | Block2: 1/0/64 1124 | | 1125 |<-----+ Header: 2.05 0x61451637 1126 | 2.05 | Token: 0xfc 1127 | | Block2: 1/1/64 1128 | | ETag: 6f00f392 1129 | | Payload: [64 bytes] 1130 .... 1131 | | 1132 +----->| Header: GET 0x41011638 1133 | GET | Token: 0xfc 1134 | | Uri-Path: status-icon 1135 | | Block2: 4/0/64 1136 | | 1137 |<-----+ Header: 2.05 0x61451638 1138 | 2.05 | Token: 0xfc 1139 | | Block2: 4/0/64 1140 | | ETag: 6f00f392 1141 | | Payload: [53 bytes] 1143 Figure 13: Observe sequence with early negotiation 1145 4. The Size2 and Size1 Options 1147 In many cases when transferring a large resource representation block 1148 by block, it is advantageous to know the total size early in the 1149 process. Some indication may be available from the maximum size 1150 estimate attribute "sz" provided in a resource description [RFC6690]. 1151 However, the size may vary dynamically, so a more up-to-date 1152 indication may be useful. 1154 This specification defines two CoAP Options, Size1 for indicating the 1155 size of the representation transferred in requests, and Size2 for 1156 indicating the size of the representation transferred in responses. 1157 (Size1 is already defined in [RFC7252] for the narrow case of 1158 indicating in 4.13 responses the maximum size of request payload that 1159 the server is able and willing to handle.) 1161 The Size2 Option may be used for two purposes: 1163 o in a request, to ask the server to provide a size estimate along 1164 with the usual response ("size request"). For this usage, the 1165 value MUST be set to 0. 1167 o in a response carrying a Block2 Option, to indicate the current 1168 estimate the server has of the total size of the resource 1169 representation, measured in bytes ("size indication"). 1171 Similarly, the Size1 Option may be used for two purposes: 1173 o in a request carrying a Block1 Option, to indicate the current 1174 estimate the client has of the total size of the resource 1175 representation, measured in bytes ("size indication"). 1177 o in a 4.13 response, to indicate the maximum size that would have 1178 been acceptable [RFC7252], measured in bytes. 1180 Apart from conveying/asking for size information, the Size options 1181 have no other effect on the processing of the request or response. 1182 If the client wants to minimize the size of the payload in the 1183 resulting response, it should add a Block2 option to the request with 1184 a small block size (e.g., setting SZX=0). 1186 The Size Options are "elective", i.e., a client MUST be prepared for 1187 the server to ignore the size estimate request. The Size Options 1188 MUST NOT occur more than once. 1190 +-----+---+---+---+---+-------+--------+--------+---------+ 1191 | No. | C | U | N | R | Name | Format | Length | Default | 1192 +-----+---+---+---+---+-------+--------+--------+---------+ 1193 | 60 | | | x | | Size1 | uint | 0-4 | (none) | 1194 | | | | | | | | | | 1195 | 28 | | | x | | Size2 | uint | 0-4 | (none) | 1196 +-----+---+---+---+---+-------+--------+--------+---------+ 1198 Table 2: Size Option Numbers 1200 Implementation Notes: 1202 o As a quality of implementation consideration, block-wise transfers 1203 for which the total size considerably exceeds the size of one 1204 block are expected to include size indications, whenever those can 1205 be provided without undue effort (preferably with the first block 1206 exchanged). If the size estimate does not change, the indication 1207 does not need to be repeated for every block. 1209 o The end of a block-wise transfer is governed by the M bits in the 1210 Block Options, _not_ by exhausting the size estimates exchanged. 1212 o As usual for an option of type uint, the value 0 is best expressed 1213 as an empty option (0 bytes). There is no default value for 1214 either Size Option. 1216 o The Size Options are neither critical nor unsafe, and are marked 1217 as No-Cache-Key. 1219 5. HTTP Mapping Considerations 1221 In this subsection, we give some brief examples for the influence the 1222 Block options might have on intermediaries that map between CoAP and 1223 HTTP. 1225 For mapping CoAP requests to HTTP, the intermediary may want to map 1226 the sequence of block-wise transfers into a single HTTP transfer. 1227 E.g., for a GET request, the intermediary could perform the HTTP 1228 request once the first block has been requested and could then 1229 fulfill all further block requests out of its cache. A constrained 1230 implementation may not be able to cache the entire object and may use 1231 a combination of TCP flow control and (in particular if timeouts 1232 occur) HTTP range requests to obtain the information necessary for 1233 the next block transfer at the right time. 1235 For PUT or POST requests, historically there was more variation in 1236 how HTTP servers might implement ranges; recently, [RFC7233] has 1237 defined that Range header fields received with a request method other 1238 than GET are not to be interpreted. So, in general, the CoAP-to-HTTP 1239 intermediary will have to try sending the payload of all the blocks 1240 of a block-wise transfer for these other methods within one HTTP 1241 request. If enough buffering is available, this request can be 1242 started when the last CoAP block is received. A constrained 1243 implementation may want to relieve its buffering by already starting 1244 to send the HTTP request at the time the first CoAP block is 1245 received; any HTTP 408 status code that indicates that the HTTP 1246 server became impatient with the resulting transfer can then be 1247 mapped into a CoAP 4.08 response code (similarly, 413 maps to 4.13). 1249 For mapping HTTP to CoAP, the intermediary may want to map a single 1250 HTTP transfer into a sequence of block-wise transfers. If the HTTP 1251 client is too slow delivering a request body on a PUT or POST, the 1252 CoAP server might time out and return a 4.08 response code, which in 1253 turn maps well to an HTTP 408 status code (again, 4.13 maps to 413). 1254 HTTP range requests received on the HTTP side may be served out of a 1255 cache and/or mapped to GET requests that request a sequence of blocks 1256 overlapping the range. 1258 (Note that, while the semantics of CoAP 4.08 and HTTP 408 differ, 1259 this difference is largely due to the different way the two protocols 1260 are mapped to transport. HTTP has an underlying TCP connection, 1261 which supplies connection state, so a HTTP 408 status code can 1262 immediately be used to indicate that a timeout occurred during 1263 transmitting a request through that active TCP connection. The CoAP 1264 4.08 response code indicates one or more missing blocks, which may be 1265 due to timeouts or resource constraints; as there is no connection 1266 state, there is no way to deliver such a response immediately; 1267 instead, it is delivered on the next block transfer. Still, HTTP 408 1268 is probably the best mapping back to HTTP, as the timeout is the most 1269 likely cause for a CoAP 4.08. Note that there is no way to 1270 distinguish a timeout from a missing block for a server without 1271 creating additional state, the need for which we want to avoid.) 1273 6. IANA Considerations 1275 This draft adds the following option numbers to the CoAP Option 1276 Numbers registry of [RFC7252]: 1278 +--------+--------+-----------+ 1279 | Number | Name | Reference | 1280 +--------+--------+-----------+ 1281 | 23 | Block2 | [RFCXXXX] | 1282 | | | | 1283 | 27 | Block1 | [RFCXXXX] | 1284 | | | | 1285 | 28 | Size2 | [RFCXXXX] | 1286 +--------+--------+-----------+ 1288 Table 3: CoAP Option Numbers 1290 This draft adds the following response code to the CoAP Response 1291 Codes registry of [RFC7252]: 1293 +------+---------------------------+-----------+ 1294 | Code | Description | Reference | 1295 +------+---------------------------+-----------+ 1296 | 2.31 | Continue | [RFCXXXX] | 1297 | | | | 1298 | 4.08 | Request Entity Incomplete | [RFCXXXX] | 1299 +------+---------------------------+-----------+ 1301 Table 4: CoAP Response Codes 1303 7. Security Considerations 1305 Providing access to blocks within a resource may lead to surprising 1306 vulnerabilities. Where requests are not implemented atomically, an 1307 attacker may be able to exploit a race condition or confuse a server 1308 by inducing it to use a partially updated resource representation. 1309 Partial transfers may also make certain problematic data invisible to 1310 intrusion detection systems; it is RECOMMENDED that an intrusion 1311 detection system (IDS) that analyzes resource representations 1312 transferred by CoAP implement the Block options to gain access to 1313 entire resource representations. Still, approaches such as 1314 transferring even-numbered blocks on one path and odd-numbered blocks 1315 on another path, or even transferring blocks multiple times with 1316 different content and obtaining a different interpretation of 1317 temporal order at the IDS than at the server, may prevent an IDS from 1318 seeing the whole picture. These kinds of attacks are well understood 1319 from IP fragmentation and TCP segmentation; CoAP does not add 1320 fundamentally new considerations. 1322 Where access to a resource is only granted to clients making use of 1323 specific security associations, all blocks of that resource MUST be 1324 subject to the same security checks; it MUST NOT be possible for 1325 unprotected exchanges to influence blocks of an otherwise protected 1326 resource. As a related consideration, where object security is 1327 employed, PUT/POST should be implemented in the atomic fashion, 1328 unless the object security operation is performed on each access and 1329 the creation of unusable resources can be tolerated. 1331 A stateless server might be susceptible to an attack where the 1332 adversary sends a Block1 (e.g., PUT) block with a high block number: 1333 A naive implementation might exhaust its resources by creating a huge 1334 resource representation. 1336 Misleading size indications may be used by an attacker to induce 1337 buffer overflows in poor implementations, for which the usual 1338 considerations apply. 1340 7.1. Mitigating Resource Exhaustion Attacks 1342 Certain block-wise requests may induce the server to create state, 1343 e.g. to create a snapshot for the block-wise GET of a fast-changing 1344 resource to enable consistent access to the same version of a 1345 resource for all blocks, or to create temporary resource 1346 representations that are collected until pressed into service by a 1347 final PUT or POST with the more bit unset. All mechanisms that 1348 induce a server to create state that cannot simply be cleaned up 1349 create opportunities for denial-of-service attacks. Servers SHOULD 1350 avoid being subject to resource exhaustion based on state created by 1351 untrusted sources. But even if this is done, the mitigation may 1352 cause a denial-of-service to a legitimate request when it is drowned 1353 out by other state-creating requests. Wherever possible, servers 1354 should therefore minimize the opportunities to create state for 1355 untrusted sources, e.g. by using stateless approaches. 1357 Performing segmentation at the application layer is almost always 1358 better in this respect than at the transport layer or lower (IP 1359 fragmentation, adaptation layer fragmentation), for instance because 1360 there is application layer semantics that can be used for mitigation 1361 or because lower layers provide security associations that can 1362 prevent attacks. However, it is less common to apply timeouts and 1363 keepalive mechanisms at the application layer than at lower layers. 1364 Servers MAY want to clean up accumulated state by timing it out (cf. 1365 response code 4.08), and clients SHOULD be prepared to run block-wise 1366 transfers in an expedient way to minimize the likelihood of running 1367 into such a timeout. 1369 7.2. Mitigating Amplification Attacks 1371 [RFC7252] discusses the susceptibility of CoAP end-points for use in 1372 amplification attacks. 1374 A CoAP server can reduce the amount of amplification it provides to 1375 an attacker by offering large resource representations only in 1376 relatively small blocks. With this, e.g., for a 1000 byte resource, 1377 a 10-byte request might result in an 80-byte response (with a 64-byte 1378 block) instead of a 1016-byte response, considerably reducing the 1379 amplification provided. 1381 8. Acknowledgements 1383 Much of the content of this draft is the result of discussions with 1384 the [RFC7252] authors, and via many CoRE WG discussions. 1386 Charles Palmer provided extensive editorial comments to a previous 1387 version of this draft, some of which the authors hope to have covered 1388 in this version. Esko Dijk reviewed a more recent version, leading 1389 to a number of further editorial improvements, a solution to the 4.13 1390 ambiguity problem, and the section about combining Block and 1391 multicast. Markus Becker proposed getting rid of an ill-conceived 1392 default value for the Block2 and Block1 options. Peter Bigot 1393 insisted on a more systematic coverage of the options and response 1394 code. 1396 Kepeng Li, Linyi Tian, and Barry Leiba wrote up an early version of 1397 the Size Option, which has informed this draft. Klaus Hartke wrote 1398 some of the text describing the interaction of Block2 with Observe. 1399 Matthias Kovatsch provided a number of significant simplifications of 1400 the protocol. 1402 9. References 1404 9.1. Normative References 1406 [I-D.ietf-core-observe] 1407 Hartke, K., "Observing Resources in CoAP", draft-ietf- 1408 core-observe-16 (work in progress), December 2014. 1410 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1411 Requirement Levels", BCP 14, RFC 2119, March 1997. 1413 [RFC7252] Shelby, Z., Hartke, K., and C. Bormann, "The Constrained 1414 Application Protocol (CoAP)", RFC 7252, June 2014. 1416 9.2. Informative References 1418 [REST] Fielding, R., "Architectural Styles and the Design of 1419 Network-based Software Architectures", Ph.D. Dissertation, 1420 University of California, Irvine, 2000, 1421 . 1424 [RFC4919] Kushalnagar, N., Montenegro, G., and C. Schumacher, "IPv6 1425 over Low-Power Wireless Personal Area Networks (6LoWPANs): 1426 Overview, Assumptions, Problem Statement, and Goals", RFC 1427 4919, August 2007. 1429 [RFC4944] Montenegro, G., Kushalnagar, N., Hui, J., and D. Culler, 1430 "Transmission of IPv6 Packets over IEEE 802.15.4 1431 Networks", RFC 4944, September 2007. 1433 [RFC6690] Shelby, Z., "Constrained RESTful Environments (CoRE) Link 1434 Format", RFC 6690, August 2012. 1436 [RFC7228] Bormann, C., Ersue, M., and A. Keranen, "Terminology for 1437 Constrained-Node Networks", RFC 7228, May 2014. 1439 [RFC7230] Fielding, R. and J. Reschke, "Hypertext Transfer Protocol 1440 (HTTP/1.1): Message Syntax and Routing", RFC 7230, June 1441 2014. 1443 [RFC7233] Fielding, R., Lafon, Y., and J. Reschke, "Hypertext 1444 Transfer Protocol (HTTP/1.1): Range Requests", RFC 7233, 1445 June 2014. 1447 Authors' Addresses 1449 Carsten Bormann 1450 Universitaet Bremen TZI 1451 Postfach 330440 1452 Bremen D-28359 1453 Germany 1455 Phone: +49-421-218-63921 1456 Email: cabo@tzi.org 1457 Zach Shelby (editor) 1458 ARM 1459 150 Rose Orchard 1460 San Jose, CA 95134 1461 USA 1463 Phone: +1-408-203-9434 1464 Email: zach.shelby@arm.com