idnits 2.17.1 draft-ietf-core-new-block-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (September 23, 2020) is 1310 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFCXXXX' is mentioned on line 813, but not defined == Outdated reference: A later version (-14) exists of draft-ietf-core-echo-request-tag-10 == Outdated reference: A later version (-25) exists of draft-ietf-dots-telemetry-11 -- Obsolete informational reference (is this intentional?): RFC 8782 (Obsoleted by RFC 9132) Summary: 0 errors (**), 0 flaws (~~), 4 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 CORE M. Boucadair 3 Internet-Draft Orange 4 Intended status: Standards Track J. Shallow 5 Expires: March 27, 2021 September 23, 2020 7 Constrained Application Protocol (CoAP) Block-Wise Transfer Options for 8 Faster Transmission 9 draft-ietf-core-new-block-01 11 Abstract 13 This document specifies alternate Constrained Application Protocol 14 (CoAP) Block-Wise transfer options: Quick-Block1 and Quick-Block2 15 Options. 17 These options are similar to the CoAP Block1 and Block2 Options, not 18 a replacement for them, but do enable faster transmission rates for 19 large amounts of data with less packet interchanges as well as 20 supporting faster recovery should any of the blocks get lost in 21 transmission. 23 Status of This Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at https://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on March 27, 2021. 40 Copyright Notice 42 Copyright (c) 2020 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (https://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 58 1.1. Existing CoAP Block-Wise Transfer Options . . . . . . . . 3 59 1.2. Alternative CoAP Block-Wise Transfer Options . . . . . . 3 60 1.3. An Updated CoAP Response Code . . . . . . . . . . . . . . 4 61 1.4. Applicability Scope . . . . . . . . . . . . . . . . . . . 5 62 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 63 3. The Quick-Block1 and Quick-Block2 Options . . . . . . . . . . 6 64 3.1. Properties of Quick-Block1 and Quick-Block2 Options . . . 6 65 3.2. Structure of Quick-Block1 and Quick-Block2 Options . . . 7 66 3.3. Using the Quick-Block1 Option . . . . . . . . . . . . . . 8 67 3.4. Using the Quick-Block2 Option . . . . . . . . . . . . . . 9 68 3.5. Working with Observe and Quick-Block2 Options . . . . . . 10 69 3.6. Working with Size1 and Size2 Options . . . . . . . . . . 11 70 3.7. Use of Quick-Block1 and Quick-Block2 Options Together . . 11 71 4. The Use of 4.08 (Request Entity Incomplete) Response Code . . 11 72 5. The Use of Tokens . . . . . . . . . . . . . . . . . . . . . . 12 73 6. Congestion Control . . . . . . . . . . . . . . . . . . . . . 12 74 7. Caching Considerations . . . . . . . . . . . . . . . . . . . 13 75 8. HTTP-Mapping Considerations . . . . . . . . . . . . . . . . . 14 76 9. Examples of Selective Block Recovery . . . . . . . . . . . . 14 77 9.1. Quick-Block1 Option: Non-Confirmable Example . . . . . . 15 78 9.2. Quick-Block2 Option: Non-Confirmable Example . . . . . . 16 79 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18 80 10.1. New CoAP Options . . . . . . . . . . . . . . . . . . . . 18 81 10.2. New Content Format . . . . . . . . . . . . . . . . . . . 19 82 11. Security Considerations . . . . . . . . . . . . . . . . . . . 19 83 12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 19 84 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 19 85 13.1. Normative References . . . . . . . . . . . . . . . . . . 19 86 13.2. Informative References . . . . . . . . . . . . . . . . . 20 87 Appendix A. Examples with Confirmable Messages . . . . . . . . . 21 88 A.1. Quick-Block1 Option . . . . . . . . . . . . . . . . . . . 21 89 A.2. Quick-Block2 Option . . . . . . . . . . . . . . . . . . . 23 90 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24 92 1. Introduction 93 1.1. Existing CoAP Block-Wise Transfer Options 95 The Constrained Application Protocol (CoAP) [RFC7252], although 96 inspired by HTTP, was designed to use UDP instead of TCP. The 97 message layer of CoAP over UDP includes support for reliable 98 delivery, simple congestion control, and flow control. [RFC7959] 99 introduced the CoAP Block1 and Block2 Options to handle data records 100 that cannot fit in a single IP packet, so not having to rely on IP 101 fragmentation and further updated by [RFC8323] for use over TCP, TLS, 102 and Websockets. 104 The CoAP Block1 and Block2 Options work well in environments where 105 there are no or minimal packet losses. These options operate 106 synchronously where each block has to be requested and can only ask 107 for (or send) the next block when the request for the previous block 108 has completed. Packet, and hence block transmission rate, is 109 controlled by Round Trip Times (RTTs). 111 There is a requirement for these blocks of data to be transmitted at 112 higher rates under network conditions where there may be asymmetrical 113 transient packet loss. An example is when a network is subject to a 114 Distributed Denial of Service (DDoS) attack and there is a need for 115 DDoS mitigation agents relying upon CoAP to communicate with each 116 other (e.g., [I-D.ietf-dots-telemetry]). As a reminder, [RFC7959] 117 recommends use of Confirmable (CON) responses to handle potential 118 packet loss; which does not work with a flooded pipe DDoS situation. 120 1.2. Alternative CoAP Block-Wise Transfer Options 122 This document introduces the CoAP Quick-Block1 and Quick-Block2 123 Options. These options are similar in operation to the CoAP Block1 124 and Block2 Options respectively, they are not a replacement for them, 125 but have the following benefits: 127 o They can operate in environments where packet loss is highly 128 asymmetrical. 130 o They enable faster transmissions of sets of blocks of data with 131 less packet interchanges. 133 o They support faster recovery should any of the Blocks get lost in 134 transmission. 136 There are the following disadvantages over using CoAP Block 1 and 137 Block2 Options: 139 o Loss of lock-stepping so payloads are not always received in the 140 correct (block ascending) order. 142 o Additional congestion control measures need to be put in place. 144 Using Non-confirmable (NON) messages, the faster transmissions occur 145 as all the Blocks can be transmitted serially (as are IP fragmented 146 packets) without having to wait for an acknowledgement or next 147 request from the remote CoAP peer. Recovery of missing Blocks is 148 faster in that multiple missing Blocks can be requested in a single 149 CoAP packet. Even if there is asymmetrical packet loss, a body can 150 still be sent and received by the peer whether the body compromises 151 of a single or multiple payloads assuming no recovery is required. 153 Note that the same performance benefits can be applied to Confirmable 154 messages if the value of NSTART is increased from 1 (Section 4.7 of 155 [RFC7252]). However, the asymmetrical packet loss is not a benefit 156 here. Some sample examples with Confirmable messages are provided in 157 Appendix A. 159 There is little, if any, benefit of using these options with CoAP 160 running over a reliable connection [RFC8323]. In this case, there is 161 no differentiation between Confirmable and NON as they are not used. 163 A CoAP endpoint can acknowledge all or a subset of the blocks. 164 Concretely, the receiving CoAP endpoint informs the CoAP endpoint 165 sender either successful receipt or reports on all blocks in the body 166 that have been not yet been received. The CoAP endpoint sender will 167 then retransmit only the blocks that have been lost in transmission. 169 Quick-Block1 and Quick-Block2 Options can be used instead of Block1 170 and Block2 Options respectively when the different transmission 171 semantics are required. If the option is not supported by a peer, 172 then transmissions can fall back to using Block1 and Block2 173 respectively. 175 The deviations from Block1 and Block2 Options are specified in 176 Section 3. Pointers to appropriate [RFC7959] sections are provided. 178 The specification refers to the base CoAP methods defined in 179 Section 5.8 of [RFC7252] and the new CoAP methods, FETCH, PATCH, and 180 iPATCH introduced in [RFC8132]. 182 1.3. An Updated CoAP Response Code 184 This document updates the 4.08 (Request Entity Incomplete) by 185 defining an additional message format for reporting on payloads using 186 the Quick-Block1 Option that are not received by the server. 188 See Section 4 for more details. 190 1.4. Applicability Scope 192 The block-wise transfer specified in [RFC7959] covers the general 193 case, but falls short in situations where packet loss is highly 194 asymmetrical. The mechanism specified in the document provides 195 roughly similar features to the Block1/Block2 Options. It provides 196 additional properties that are tailored towards the intended use 197 case. Concretely, this mechanism primarily targets applications such 198 as DDoS Open Threat Signaling (DOTS) that can't use Confirmable (CON) 199 responses to handle potential packet loss and that support 200 application-specific mechanisms to assess whether the remote peer is 201 able to handle the messages sent by a CoAP endpoint (e.g., DOTS 202 heartbeats in Section 4.7 of [RFC8782]). 204 The mechanism includes guards to prevent a CoAP agent from 205 overloading the network by adopting an aggressive sending rate. 206 These guards MUST be followed in addition to the existing CoAP 207 congestion control as specified in Section 4.7 of [RFC7252]. See 208 Section 6 for more details. 210 This mechanism is not intended for general CoAP usage, and any use 211 outside the intended use case should be carefully weighed against the 212 loss of interoperability with generic CoAP applications. It is hoped 213 that the experience gained with this mechanism can feed future 214 extensions of the block-wise mechanism that will both generally 215 applicable and serve this particular use case. 217 2. Terminology 219 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 220 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 221 "OPTIONAL" in this document are to be interpreted as described in BCP 222 14 [RFC2119][RFC8174] when, and only when, they appear in all 223 capitals, as shown here. 225 Readers should be familiar with the terms and concepts defined in 226 [RFC7252]. 228 The terms "payload" and "body" are defined in [RFC7959]. The term 229 "payload" is thus used for the content of a single CoAP message 230 (i.e., a single block being transferred), while the term "body" is 231 used for the entire resource representation that is being transferred 232 in a block-wise fashion. 234 3. The Quick-Block1 and Quick-Block2 Options 236 3.1. Properties of Quick-Block1 and Quick-Block2 Options 238 The properties of Quick-Block1 and Quick-Block2 Options are shown in 239 Table 1. The formatting of this table follows the one used in 240 Table 4 of [RFC7252] (Section 5.10). The C, U, N, and R columns 241 indicate the properties Critical, Unsafe, NoCacheKey, and Repeatable 242 defined in Section 5.4 of [RFC7252]. Only C and U columns are marked 243 for the Quick-Block1 Option. C, U, and R columns are marked for the 244 Quick-Block2 Option. 246 +--------+---+---+---+---+--------------+--------+--------+---------+ 247 | Number | C | U | N | R | Name | Format | Length | Default | 248 +========+===+===+===+===+==============+========+========+=========+ 249 | TBA1 | x | x | | | Quick-Block1 | uint | 0-3 | (none) | 250 | TBA2 | x | x | | x | Quick-Block2 | uint | 0-3 | (none) | 251 +--------+---+---+---+---+--------------+--------+--------+---------+ 253 Table 1: CoAP Quick-Block1 and Quick-Block2 Option Properties 255 The Quick-Block1 and Quick-Block2 Options can be present in both the 256 request and response messages. The Quick-Block1 Option pertains to 257 the request payload and the Quick-Block2 Option pertains to the 258 response payload. The Content-Format Option applies to the body, not 259 to the payload (i.e., it must be the same for all payloads of the 260 same body). 262 Quick-Block1 Option is useful with the payload-bearing POST, PUT, 263 PATCH, and iPATCH requests and their responses (2.01 and 2.04). 265 Quick-Block2 Option is useful with GET, POST, PUT, FETCH, PATCH, and 266 iPATCH requests and their payload-bearing responses (2.01, 2.03, 267 2.04, and 2.05) (Section 5.5 of [RFC7252]). 269 To indicate support for Quick-Block2 responses, the CoAP client MUST 270 include the Quick-Block2 Option in a GET or similar request, or the 271 Quick-Block2 Option in a PUT or similar request, so that the server 272 knows that the client supports this Quick-Block2 functionality should 273 it needs to send back a body that spans multiple payloads. 274 Otherwise, the server would use the Block2 Option (if supported) to 275 send back a message body that is too large to fit into a single IP 276 packet [RFC7959]. 278 If Quick-Block1 Option is present in a request or Quick-Block2 Option 279 in a response (i.e., in that message to the payload of which it 280 pertains), it indicates a block-wise transfer and describes how this 281 specific block-wise payload forms part of the entire body being 282 transferred. If it is present in the opposite direction, it provides 283 additional control on how that payload will be formed or was 284 processed. 286 Implementation of the Quick-Block1 and Quick-Block2 Options is 287 intended to be optional. However, when it is present in a CoAP 288 message, it MUST be processed (or the message rejected). Therefore, 289 Quick-Block1 and Quick-Block2 Options are identified as Critical 290 options. 292 The Quick-Block1 and Quick-Block2 Options are unsafe to forward. 293 That is, a CoAP proxy that does not understand the Quick-Block1 (or 294 Quick-Block2) Option MUST reject the request or response that uses 295 either option. 297 The Quick-Block2 Option is repeatable when requesting re-transmission 298 of missing Blocks, but not otherwise. Except that case, any request 299 carrying multiple Quick-Block1 (or Quick-Block2) Options MUST be 300 handled following the procedure specified in Section 5.4.5 of 301 [RFC7252]. 303 The Quick-Block1 and Quick-Block2 Options, like the Block1 and Block2 304 Options, are both a class E and a class U in terms of OSCORE 305 processing (see Section 4.1 of [RFC8613]): The Quick-Block1 (or 306 Quick-Block2) Option MAY be an Inner or Outer option. The Inner and 307 Outer values are therefore independent of each other. The Inner 308 option is encrypted and integrity protected between clients and 309 servers, and provides message body identification in case of end-to- 310 end fragmentation of requests. The Outer option is visible to 311 proxies and labels message bodies in case of hop-by-hop fragmentation 312 of requests. 314 3.2. Structure of Quick-Block1 and Quick-Block2 Options 316 The structure of Quick-Block1 and Quick-Block2 Options follows the 317 structure defined in Section 2.2 of [RFC7959]. 319 There is no default value for the Quick-Block1 and Quick-Block2 320 Options. Absence of one of these options is equivalent to an option 321 value of 0 with respect to the value of block number (NUM) and more 322 bit (M) that could be given in the option, i.e., it indicates that 323 the current block is the first and only block of the transfer (block 324 number is set to 0, M is unset). However, in contrast to the 325 explicit value 0, which would indicate a size of the block (SZX) of 326 0, and thus a size value of 16 bytes, there is no specific explicit 327 size implied by the absence of the option -- the size is left 328 unspecified. (As for any uint, the explicit value 0 is efficiently 329 indicated by a zero-length option; this, therefore, is different in 330 semantics from the absence of the option). 332 3.3. Using the Quick-Block1 Option 334 The Quick-Block1 Option is used when the client wants to send a large 335 amount of data to the server using the POST, PUT, PATCH, or iPATCH 336 methods where the data and headers do not fit into a single packet. 338 When Quick-Block1 Option is used, the client MUST include a single 339 Request-Tag Option [I-D.ietf-core-echo-request-tag]. The Request-Tag 340 value MUST be the same for all of the blocks in the body of data that 341 is being transferred. It is also used to identify a particular block 342 of a body that needs to be re-transmitted. The Request-Tag is opaque 343 in nature, but it is RECOMMENDED that the client treats it as an 344 unsigned integer of 8 bytes in length. An implementation may want to 345 consider limiting this to 4 bytes to reduce packet overhead size. 346 The server still treats it as an opaque entity. The Request-Tag 347 value MUST be different for distinct bodies or sets of blocks of data 348 and SHOULD be incremented whenever a new body of data is being 349 transmitted for a CoAP session between peers. The initial Request- 350 Tag value SHOULD be randomly generated by the client. 352 For Confirmable transmission, the server MUST continue to acknowledge 353 each packet. NSTART will also need to be increased from the default 354 (1) to get faster transmission rates. 356 Each individual payload of the body is treated as a new request. 358 A 2.01 (Created) or 2.04 (Changed) Response Code indicates successful 359 receipt of the entire body. The 2.31 (Continue) Response Code MUST 360 NOT be used. 362 The 2.31 (Continue) Response is not used. 364 A 4.00 (Bad Request) Response Code MUST be returned if the request 365 does not include a Request-Tag Option but does include a Quick-Block1 366 option. 368 A 4.02 (Bad Option) Response Code MUST be returned if the server does 369 not support the Quick-Block1 Option. 371 A 4.13 (Request Entity Too Large) Response Code can be returned under 372 similar conditions to those discussed in Section 2.9.3 of [RFC7959]. 374 A 4.08 (Request Entity Incomplete) Response Code returned without 375 Content-Type "application/missing-blocks+cbor-seq" (Section 10.2) is 376 handled as in Section 2.9.2 [RFC7959]. 378 A 4.08 (Request Entity Incomplete) Response Code returned with 379 Content-Type "application/missing-blocks+cbor-seq" indicates that 380 some of the payloads are missing and need to be resent. The client 381 then re-transmits the missing payloads using the Request-Tag and 382 Quick-Block1 to specify the block number, SZX, and M bit as 383 appropriate. The Request-Tag value to use is determined from the 384 payload of the 4.08 (Request Entity Incomplete) Response Code. If 385 the client does not recognize the Request-Tag, the client can ignore 386 this response. 388 If the server has not received all the payloads of a body, but one or 389 more payloads have been received, it SHOULD wait for up to 390 MAX_TRANSMIT_SPAN (Section 4.8.2 of [RFC7252]) before sending the 391 4.08 (Request Entity Incomplete) Response Code. However, this time 392 MAY be reduced to two times ACK_TIMEOUT before sending a 4.08 393 (Request Entity Incomplete) Response Code to cover the situation 394 where MAX_PAYLOADS has been triggered by the client causing a break 395 in transmission. 397 If the client transmits a new body of data with a new Request-Tag to 398 the same resource on a server, the server MUST remove any partially 399 received body held for a previous Request-Tag for that resource. 401 If the server receives a duplicate block with the same Request-Tag, 402 it SHOULD silently ignore the packet. 404 A server SHOULD only maintain a partial body (missing payloads) for 405 up to EXCHANGE_LIFETIME (Section 4.8.2 of [RFC7252]). 407 3.4. Using the Quick-Block2 Option 409 In a request, for block number 0, the M bit unset indicates the 410 entire body is requested. If the M bit is set for block number 0, 411 this indicates that this is a repeat request. Otherwise for a 412 request, the Quick-Block2 Option MUST always have the M bit unset. 414 The payloads sent back from the server as a response MUST all have 415 the same ETag (Section 5.10.6 of [RFC7252]) for the same body. The 416 server MUST NOT use the same ETag value for different representations 417 of a resource. 419 The ETag is opaque in nature, but it is RECOMMENDED that the server 420 treats it as an unsigned integer of 8 bytes in length. An 421 implementation may want to consider limiting this to 4 bytes to 422 reduce packet overhead size. The client still treats it as an opaque 423 entity. The ETag value MUST be different for distinct bodies or sets 424 of blocks of data and SHOULD be incremented whenever a new body of 425 data is being transmitted for a CoAP session between peers. The 426 initial ETag value SHOULD be randomly generated by the server. 428 If the client detects that some of the payloads are missing, the 429 missing payloads are requested by issuing a new GET, POST, PUT, 430 FETCH, PATCH, or iPATCH request that contains one or more Quick- 431 Block2 Options that define the missing blocks. 433 The ETag Option MUST NOT be used in the request as the server could 434 respond with a 2.03 (Valid Response) with no payload. If the server 435 responds with a different ETag Option value (as the resource 436 representation has changed), then the client SHOULD drop all the 437 payloads for the current body that are no longer valid. 439 The client may elect to request the missing blocks or just ignore the 440 partial body. It SHOULD wait for up to MAX_TRANSMIT_SPAN 441 (Section 4.8.2 of [RFC7252]) before issuing a GET, POST, PUT, FETCH, 442 PATCH, or iPATCH request for the missing blocks. However, this time 443 MAY be reduced to two times ACK_TIMEOUT before sending the request to 444 cover the situation where MAX_PAYLOADS has been triggered by the 445 server causing a break in transmission. 447 With NON transmission, the client only needs to indicate that some of 448 the payloads are missing by issuing a GET, POST, PUT, FETCH, PATCH, 449 or iPATCH request for the missing blocks. 451 For Confirmable transmission, the client SHOULD continue to 452 acknowledge each packet as well as issuing a separate GET, POST, PUT, 453 FETCH, PATCH, or iPATCH for the missing blocks. 455 If the server transmits a new body of data (e.g., a triggered 456 Observe) with a new ETag to the same client as an additional 457 response, the client MUST remove any partially received body held for 458 a previous ETag. 460 If the client receives a duplicate block with the same ETag, it 461 SHOULD silently ignore the packet. 463 A client SHOULD only maintain a partial body (missing payloads) for 464 up to EXCHANGE_LIFETIME (Section 4.8.2 of [RFC7252]) or as defined by 465 the Max-Age Option whichever is the less. 467 3.5. Working with Observe and Quick-Block2 Options 469 As the blocks of the body are sent without waiting for 470 acknowledgement of the individual blocks, the Observe value [RFC7641] 471 MUST be the same for all the blocks of the same body. 473 If the client requests missing blocks, this is treated as a new 474 request. The Observe value may change but MUST still be reported. 475 If the ETag value changes then the previously received partial body 476 should be destroyed and the whole body re-requested. 478 3.6. Working with Size1 and Size2 Options 480 Section 4 of [RFC7959] defines two CoAP options: Size1 for indicating 481 the size of the representation transferred in requests and Size2 for 482 indicating the size of the representation transferred in responses. 484 The Size1 or Size2 option values MUST exactly represent the size of 485 the data on the body so that any missing data can easily be 486 determined. 488 The Size1 Option MUST be used with the Quick-Block1 Option when used 489 in a request. The Size2 Option MUST be used with the Quick-Block2 490 Option when used in a response. 492 If Size1 or Size2 Options are used, they MUST be used in all payloads 493 of the body and MUST have the same value. 495 3.7. Use of Quick-Block1 and Quick-Block2 Options Together 497 The behavior is similar to the one defined in Section 3.3 of 498 [RFC7959] with Quick-Block1 substituted for Block1 and Quick-Block2 499 for Block2. 501 4. The Use of 4.08 (Request Entity Incomplete) Response Code 503 4.08 (Request Entity Incomplete) Response Code has a new Content-Type 504 "application/missing-blocks+cbor-seq" used to indicate that the 505 server has not received all of the blocks of the request body that it 506 needs to proceed. 508 Likely causes are the client has not sent all blocks, some blocks 509 were dropped during transmission, or the client has sent them 510 sufficiently long ago that the server has already discarded them. 512 The data payload of the 4.08 (Request Entity Incomplete) Response 513 Code is encoded as a CBOR Sequence [RFC8742]. First is CBOR encoded 514 Request-Tag followed by 1 or more missing CBOR encoded missing block 515 numbers. The missing block numbers MUST be unique in each 4.08 516 (Request Entity Incomplete) when created by the server; the client 517 SHOULD drop any duplicates in the same 4.08 (Request Entity 518 Incomplete) message. 520 The Content-Format Option (Section 5.10.3 of [RFC7252]) MUST be used 521 in the 4.08 (Request Entity Incomplete) Response Code. It MUST be 522 set to "application/missing-blocks+cbor-seq" (see Section 10.2). 524 The Concise Data Definition Language [RFC8610] for the data 525 describing these missing blocks is as follows: 527 payload = {request-tag, missing-block-list} 528 ; A copy of the opaque Request-Tag value 529 request-tag = bstr 530 missing-block-list = [1 * missing-block-number] 531 ; A unique block number not received 532 missing-block-number = uint 534 Figure 1: Structure of the Missing Blocks Payload 536 If the size of the 4.08 (Request Entity Incomplete) response packet 537 is larger than that defined by Section 4.6 [RFC7252], then the number 538 of missing blocks MUST be limited so that the response can fit into a 539 single packet. If this is the case, then the server can send 540 subsequent 4.08 (Request Entity Incomplete) responses containing the 541 missing blocks on receipt of a new request providing a missing 542 payload with the same Request-Tag. 544 5. The Use of Tokens 546 Each new request MUST use a unique Token (Section 4 of 547 [I-D.ietf-core-echo-request-tag]). Additional responses may use the 548 same Token. 550 6. Congestion Control 552 PROBING_RATE parameter in CoAP indicates the average data rate that 553 must not be exceeded by a CoAP endpoint in sending to a peer endpoint 554 that does not respond. The body of blocks will be subjected to 555 PROBING_RATE (Section 4.7 of [RFC7252]). 557 Each NON 4.08 (Request Entity Incomplete) Response Codes is subjected 558 to PROBING_RATE. 560 Each NON GET or similar request using Quick-Block2 Option is 561 subjected to PROBING_RATE. 563 As the sending of many payloads of a single body may itself cause 564 congestion, it is RECOMMENDED that after transmission of every set of 565 MAX_PAYLOADS payloads of a single body, a delay is introduced of 566 ACK_TIMEOUT (Section 4.8.2 of [RFC7252]) before the next set of 567 payload transmissions to manage potential congestion issues. 568 MAX_PAYLOADS should be configurable with a default value of 10. 570 Note: The default value is chosen for reasons similar to those 571 discussed in Section 5 of [RFC6928]. 573 For NON transmissions, it is permissible, but not required, to send 574 the ultimate payload of a MAX_PAYLOADS set as a Confirmable packet. 575 If a Confirmable packet is used, then the transmitting peer MUST wait 576 for the ACK to be returned before sending the next set of payloads, 577 which can be in time terms less than the ACK_TIMEOUT delay. 579 Also, for NON transmissions, it is permissible, but not required, to 580 send a Confirmable packet for the final payload of a body (that is, M 581 bit unset). If a Confirmable packet is used, then the transmitting 582 peer MUST wait for the appropriate response to be returned for 583 successful transmission, or respond to requests for the missing 584 blocks (if any). 586 The sending of the set of missing blocks is subject to MAX_PAYLOADS. 588 7. Caching Considerations 590 Caching block based information is not straight forward in a proxy. 591 For Quick-Block1 and Quick-Block2 Options, it is expected that the 592 proxy will reassemble the body (using any appropriate recovery 593 options for packet loss) before passing on the body to the 594 appropriate CoAP endpoint. The onward transmission of the body does 595 not require the use of the Quick-Block1 or Quick-Block2 Options. 596 This means that the proxy must fully support the Quick-Block1 and 597 Quick-Block2 Options. 599 How the body is cached in the initial CoAP client (Quick-Block1) or 600 ultimate CoAP server (Quick-Block2) is implementation specific. 602 As the entire body is being cached in the proxy, the Quick-Block1 and 603 Quick-Block2 Options are not part of the cache key. 605 For Quick-Block2 responses, the ETag Option value is associated with 606 the data (and onward transmitted to the CoAP client), but is not part 607 of the cache key. 609 For requests with Quick-Block1 Option, the Request-Tag Option is 610 associated with the build up of the body from successive payloads, 611 but is not part of the cache key. For the onward transmission of the 612 body using CoAP, a new Request-Tag SHOULD be generated and used. 614 It is possible that two or more CoAP clients are concurrently 615 updating the same resource through a common proxy to the same CoAP 616 server using Quick-Block1 (or Block1) Option. If this is the case, 617 the first client to complete building the body causes that body to 618 start transmitting to the CoAP server with an appropriate Request-Tag 619 value. When the next client completes building the body, any 620 existing partial body transmission to the CoAP server is terminated 621 and the new body representation transmission starts with a new 622 Request-Tag value. 624 A proxy that supports Quick-Block2 Option MUST be prepared to receive 625 a GET or similar message indicating one or more missing blocks. The 626 proxy will serve from its cache the missing blocks that are available 627 in its cache in the same way a server would send all the appropriate 628 Quick-Block2s. If the cache key matching body is not available in 629 the cache, the proxy MUST request the entire body from the CoAP 630 server using the information in the cache key. 632 How long a CoAP endpoint (or proxy) keeps the body in its cache is 633 implementation specific (e.g., it may be based on Max-Age). 635 8. HTTP-Mapping Considerations 637 As a reminder, the basic normative requirements on HTTP/CoAP mappings 638 are defined in Section 10 of [RFC7252]. The implementation 639 guidelines for HTTP/CoAP mappings are elaborated in [RFC8075]. 641 The rules defined in Section 5 of [RFC7959] are to be followed. 643 9. Examples of Selective Block Recovery 645 This section provides some sample flows to illustrate the use of 646 Quick-Block1 and Quick-Block2 Options. Figure 2 lists the 647 conventions that are used in the following subsections. 649 T: Token value 650 O: Observe Option value 651 M: Message ID 652 RT: Request-Tag 653 ET: ETag 654 QB1: Quick-Block1 Option values NUM/More/SZX 655 QB2: Quick-Block2 Option values NUM/More/SZX 656 \: Trimming long lines 657 [[]]: Comments 658 -->X: Message loss 659 X<--: Message loss 661 Figure 2: Notations Used in the Figures 663 9.1. Quick-Block1 Option: Non-Confirmable Example 665 Figure 3 depicts an example of a NON PUT request conveying Quick- 666 Block1 Option. All the blocks are received by the server. 668 CoAP CoAP 669 Client Server 670 | | 671 +--------->| NON PUT /path M:0x01 T:0xf0 RT=10 QB1:0/1/1024 672 +--------->| NON PUT /path M:0x02 T:0xf1 RT=10 QB1:1/1/1024 673 +--------->| NON PUT /path M:0x03 T:0xf2 RT=10 QB1:2/1/1024 674 +--------->| NON PUT /path M:0x04 T:0xf3 RT=10 QB1:3/0/1024 675 |<---------+ NON 2.04 M:0xf1 T:0xf3 676 ... 678 Figure 3: Example of NON Request with Quick-Block1 Option (Without 679 Loss) 681 Consider now a scenario where a new body of data is to be sent by the 682 client, but some blocks are dropped in transmission as illustrated in 683 Figure 4. 685 CoAP CoAP 686 Client Server 687 | | 688 +--------->| NON PUT /path M:0x05 T:0xe0 RT=11 QB1:0/1/1024 689 +--->X | NON PUT /path M:0x06 T:0xe1 RT=11 QB1:1/1/1024 690 +--->X | NON PUT /path M:0x07 T:0xe2 RT=11 QB1:2/1/1024 691 +--------->| NON PUT /path M:0x08 T:0xe3 RT=11 QB1:3/0/1024 692 | | 693 ... 695 Figure 4: Example of NON Request with Quick-Block1 Option (With Loss) 696 The server realizes that some blocks are missing and asks for the 697 missing ones in one go (Figure 5). It does so by indicating which 698 blocks have been received in the data portion of the response. 700 CoAP CoAP 701 Client Server 702 | | 703 ... 704 |<---------+ NON 4.08 M:0xf2 T:0xe3 [Missing 1,2 for RT=11] 705 +--------->| NON PUT /path M:0x09 T:0xe4 RT=11 QB1:1/1/1024 706 +--->X | NON PUT /path M:0x0a T:0xe5 RT=11 QB1:2/1/1024 707 | | 708 |<---------+ NON 4.08 M:0xf3 T:0xe4 [Missing 2 for RT=11] 709 +--------->| NON PUT /path M:0x0b T:0xe6 RT=11 QB1:2/1/1024 710 |<---------+ NON 2.04 M:0xf4 T:0xe6 711 | | 712 ... 714 Figure 5: Example of NON Request with Quick-Block1 Option (Blocks 715 Recovery) 717 Under high levels of traffic loss, the client can elect not to retry 718 sending missing blocks of data. This decision is implementation 719 specific. 721 9.2. Quick-Block2 Option: Non-Confirmable Example 723 Figure 6 illustrates the example of Quick-Block2 Option. The client 724 sends a NON GET carrying an Observe and a Quick-Block2 Options. The 725 Quick-Block2 Option indicates a size hint (1024 bytes). This request 726 is replied by the server using four (4) blocks that are transmitted 727 to the client without any loss. Each of these blocks carries a 728 Quick-Block2 Option. The same process is repeated when an Observe is 729 triggered, but no loss is experienced by any of the notification 730 blocks. 732 CoAP CoAP 733 Client Server 734 | | 735 +--------->| NON GET /path M:0x01 T:0xf0 O:0 QB2:0/0/1024 736 |<---------+ NON 2.05 M:0xf1 T:0xf0 O:1234 ET=21 QB2:0/1/1024 737 |<---------+ NON 2.05 M:0xf2 T:0xf0 O:1234 ET=21 QB2:1/1/1024 738 |<---------+ NON 2.05 M:0xf3 T:0xf0 O:1234 ET=21 QB2:2/1/1024 739 |<---------+ NON 2.05 M:0xf4 T:0xf0 O:1234 ET=21 QB2:3/0/1024 740 ... 741 [[Observe triggered]] 742 |<---------+ NON 2.05 M:0xf5 T:0xf0 O:1235 ET=22 QB2:0/1/1024 743 |<---------+ NON 2.05 M:0xf6 T:0xf0 O:1235 ET=22 QB2:1/1/1024 744 |<---------+ NON 2.05 M:0xf7 T:0xf0 O:1235 ET=22 QB2:2/1/1024 745 |<---------+ NON 2.05 M:0xf8 T:0xf0 O:1235 ET=22 QB2:3/0/1024 746 ... 748 Figure 6: Example of NON Notifications with Quick-Block2 Option 749 (Without Loss) 751 Figure 7 shows the example of an Observe that is triggered but for 752 which some notification blocks are lost. The client detects the 753 missing blocks and request their retransmission. It does so by 754 indicating the blocks that were successfully received. 756 CoAP CoAP 757 Client Server 758 | | 759 ... 760 [[Observe triggered]] 761 |<---------+ NON 2.05 M:0xf9 T:0xf0 O:1236 ET=23 QB2:0/1/1024 762 | X<---+ NON 2.05 M:0xfa T:0xf0 O:1236 ET=23 QB2:1/1/1024 763 | X<---+ NON 2.05 M:0xfb T:0xf0 O:1236 ET=23 QB2:2/1/1024 764 |<---------+ NON 2.05 M:0xfc T:0xf0 O:1236 ET=23 QB2:3/0/1024 765 | | 766 [[Client realizes blocks are missing and asks for the missing 767 ones in one go]] 768 +--------->| NON GET /path M:0x02 T:0xf1 QB2:1/0/1024\ 769 | | QB2:2/0/1024 770 | X<---+ NON 2.05 M:0xfd T:0xf1 ET=23 QB2:1/1/1024 771 |<---------+ NON 2.05 M:0xfe T:0xf1 ET=23 QB2:2/1/1024 772 | | 773 [[Get the final missing block]] 774 +--------->| NON GET /path M:0x03 T:0xf2 QB2:1/0/1024 775 |<---------+ NON 2.05 M:0xff T:0xf2 ET=23 QB2:1/1/1024 776 ... 778 Figure 7: Example of NON Notifications with Quick-Block2 Option 779 (Blocks Recovery) 781 Under high levels of traffic loss, the client can elect not to retry 782 getting missing blocks of data. This decision is implementation 783 specific. 785 10. IANA Considerations 787 10.1. New CoAP Options 789 IANA is requested to add the following entries to the "CoAP Option 790 Numbers" sub-registry [Options]: 792 +--------+------------------+-----------+ 793 | Number | Name | Reference | 794 +========+==================+===========+ 795 | TBA1 | Quick-Block1 | [RFCXXXX] | 796 | TBA2 | Quick-Block2 | [RFCXXXX] | 797 +--------+------------------+-----------+ 799 Table 2: CoAP Quick-Block1 and Quick-Block2 Option Numbers 801 This document suggests 19 (TBA1) and 51 (TBA2) as a values to be 802 assigned for the new option numbers. 804 10.2. New Content Format 806 This document requests IANA to register the CoAP Content-Format ID 807 for the "application/missing-blocks+cbor-seq" media type in the "CoAP 808 Content-Formats" registry [Format]: 810 o Media Type: application/missing-blocks+cbor-seq 811 o Encoding: - 812 o Id: TBD3 813 o Reference: [RFCXXXX] 815 11. Security Considerations 817 Security considerations discussed in Section 9 of [RFC7959] should be 818 taken into account. 820 Security considerations related to the use of Request-Tag are 821 discussed in Section 5 of [I-D.ietf-core-echo-request-tag]. 823 12. Acknowledgements 825 Thanks to Achim Kraus and Jim Schaad for the comments on the mailing 826 list. 828 Special thanks to Christian Amsuess and Carsten Bormann for their 829 suggestions and several reviews, which improved this specification 830 significantly. 832 Some text from [RFC7959] is reused for readers convenience. 834 13. References 836 13.1. Normative References 838 [I-D.ietf-core-echo-request-tag] 839 Amsuess, C., Mattsson, J., and G. Selander, "CoAP: Echo, 840 Request-Tag, and Token Processing", draft-ietf-core-echo- 841 request-tag-10 (work in progress), July 2020. 843 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 844 Requirement Levels", BCP 14, RFC 2119, 845 DOI 10.17487/RFC2119, March 1997, 846 . 848 [RFC7252] Shelby, Z., Hartke, K., and C. Bormann, "The Constrained 849 Application Protocol (CoAP)", RFC 7252, 850 DOI 10.17487/RFC7252, June 2014, 851 . 853 [RFC7641] Hartke, K., "Observing Resources in the Constrained 854 Application Protocol (CoAP)", RFC 7641, 855 DOI 10.17487/RFC7641, September 2015, 856 . 858 [RFC7959] Bormann, C. and Z. Shelby, Ed., "Block-Wise Transfers in 859 the Constrained Application Protocol (CoAP)", RFC 7959, 860 DOI 10.17487/RFC7959, August 2016, 861 . 863 [RFC8075] Castellani, A., Loreto, S., Rahman, A., Fossati, T., and 864 E. Dijk, "Guidelines for Mapping Implementations: HTTP to 865 the Constrained Application Protocol (CoAP)", RFC 8075, 866 DOI 10.17487/RFC8075, February 2017, 867 . 869 [RFC8132] van der Stok, P., Bormann, C., and A. Sehgal, "PATCH and 870 FETCH Methods for the Constrained Application Protocol 871 (CoAP)", RFC 8132, DOI 10.17487/RFC8132, April 2017, 872 . 874 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 875 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 876 May 2017, . 878 [RFC8323] Bormann, C., Lemay, S., Tschofenig, H., Hartke, K., 879 Silverajan, B., and B. Raymor, Ed., "CoAP (Constrained 880 Application Protocol) over TCP, TLS, and WebSockets", 881 RFC 8323, DOI 10.17487/RFC8323, February 2018, 882 . 884 [RFC8613] Selander, G., Mattsson, J., Palombini, F., and L. Seitz, 885 "Object Security for Constrained RESTful Environments 886 (OSCORE)", RFC 8613, DOI 10.17487/RFC8613, July 2019, 887 . 889 [RFC8742] Bormann, C., "Concise Binary Object Representation (CBOR) 890 Sequences", RFC 8742, DOI 10.17487/RFC8742, February 2020, 891 . 893 13.2. Informative References 895 [Format] , . 898 [I-D.ietf-dots-telemetry] 899 Boucadair, M., Reddy.K, T., Doron, E., chenmeiling, c., 900 and J. Shallow, "Distributed Denial-of-Service Open Threat 901 Signaling (DOTS) Telemetry", draft-ietf-dots-telemetry-11 902 (work in progress), July 2020. 904 [Options] , . 907 [RFC6928] Chu, J., Dukkipati, N., Cheng, Y., and M. Mathis, 908 "Increasing TCP's Initial Window", RFC 6928, 909 DOI 10.17487/RFC6928, April 2013, 910 . 912 [RFC8610] Birkholz, H., Vigano, C., and C. Bormann, "Concise Data 913 Definition Language (CDDL): A Notational Convention to 914 Express Concise Binary Object Representation (CBOR) and 915 JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, 916 June 2019, . 918 [RFC8782] Reddy.K, T., Ed., Boucadair, M., Ed., Patil, P., 919 Mortensen, A., and N. Teague, "Distributed Denial-of- 920 Service Open Threat Signaling (DOTS) Signal Channel 921 Specification", RFC 8782, DOI 10.17487/RFC8782, May 2020, 922 . 924 Appendix A. Examples with Confirmable Messages 926 These examples assume NSTART has been increased to at least 4. 928 The notations provided in Figure 2 are used in the following 929 subsections. 931 A.1. Quick-Block1 Option 933 Let's now consider the use Quick-Block1 Option with a CON request as 934 shown in Figure 8. All the blocks are acknowledged (ACK). 936 CoAP CoAP 937 Client Server 938 | | 939 +--------->| CON PUT /path M:0x01 T:0xf0 RT=10 QB1:0/1/1024 940 +--------->| CON PUT /path M:0x02 T:0xf1 RT=10 QB1:1/1/1024 941 +--------->| CON PUT /path M:0x03 T:0xf2 RT=10 QB1:2/1/1024 942 +--------->| CON PUT /path M:0x04 T:0xf3 RT=10 QB1:3/0/1024 943 |<---------+ ACK 0.00 M:0x01 944 |<---------+ ACK 0.00 M:0x02 945 |<---------+ ACK 0.00 M:0x03 946 |<---------+ ACK 2.04 M:0x04 948 Figure 8: Example of CON Request with Quick-Block1 Option (Without 949 Loss) 951 Now, suppose that a new body of data is to sent but with some blocks 952 dropped in transmission as illustrated in Figure 9. The client will 953 retry sending blocks for which no ACK was received. 955 CoAP CoAP 956 Client Server 957 | | 958 +--------->| CON PUT /path M:0x05 T:0xf4 RT=11 QB1:0/1/1024 959 +--->X | CON PUT /path M:0x06 T:0xf5 RT=11 QB1:1/1/1024 960 +--->X | CON PUT /path M:0x07 T:0xf6 RT=11 QB1:2/1/1024 961 +--------->| CON PUT /path M:0x08 T:0xf7 RT=11 QB1:3/1/1024 962 |<---------+ ACK 0.00 M:0x05 963 |<---------+ ACK 0.00 M:0x08 964 | | 965 [[The client retries sending packets not acknowledged]] 966 +--------->| CON PUT /path M:0x06 T:0xf5 RT=11 QB1:1/1/1024 967 +--->X | CON PUT /path M:0x07 T:0xf6 RT=11 QB1:2/1/1024 968 |<---------+ ACK 0.00 M:0x06 969 | | 970 [[The client retransmits messages not acknowledged 971 (exponential backoff)]] 972 +--->? | CON PUT /path M:0x07 T:0xf6 RT=11 QB1:2/1/1024 973 | | 974 [[Either transmission failure (acknowledge retry timeout) 975 or successfully transmitted.]] 977 Figure 9: Example of CON Request with Quick-Block1 Option (Blocks 978 Recovery) 980 It is implementation dependent as to whether a CoAP session is 981 terminated following acknowledge retry timeout, or whether the CoAP 982 session continues to be used under such adverse traffic conditions. 984 If there is likely to be the possibility of network transient losses, 985 then the use of Non-confirmable traffic should be considered. 987 A.2. Quick-Block2 Option 989 An example of the use of Quick-Block2 Option with Confirmable 990 messages is shown in Figure 10. 992 Client Server 993 | | 994 +--------->| CON GET /path M:0x01 T:0xf0 O:0 QB2:0/0/1024 995 |<---------+ ACK 2.05 M:0x01 T:0xf0 O:1234 ET=21 QB2:0/1/1024 996 |<---------+ ACK 2.05 M:0xe1 T:0xf0 O:1234 ET=21 QB2:1/1/1024 997 |<---------+ ACK 2.05 M:0xe2 T:0xf0 O:1234 ET=21 QB2:2/1/1024 998 |<---------+ ACK 2.05 M:0xe3 T:0xf0 O:1234 ET=21 QB2:3/0/1024 999 ... 1000 [[Observe triggered]] 1001 |<---------+ CON 2.05 M:0xe4 T:0xf0 O:1235 ET=22 QB2:0/1/1024 1002 |<---------+ CON 2.05 M:0xe5 T:0xf0 O:1235 ET=22 QB2:1/1/1024 1003 |<---------+ CON 2.05 M:0xe6 T:0xf0 O:1235 ET=22 QB2:2/1/1024 1004 |<---------+ CON 2.05 M:0xe7 T:0xf0 O:1235 ET=22 QB2:3/0/1024 1005 |--------->+ ACK 0.00 M:0xe4 1006 |--------->+ ACK 0.00 M:0xe5 1007 |--------->+ ACK 0.00 M:0xe6 1008 |--------->+ ACK 0.00 M:0xe7 1009 ... 1010 [[Observe triggered]] 1011 |<---------+ CON 2.05 M:0xe8 T:0xf0 O:1236 ET=23 QB2:0/1/1024 1012 | X<---+ CON 2.05 M:0xe9 T:0xf0 O:1236 ET=23 QB2:1/1/1024 1013 | X<---+ CON 2.05 M:0xea T:0xf0 O:1236 ET=23 QB2:2/1/1024 1014 |<---------+ CON 2.05 M:0xeb T:0xf0 O:1236 ET=23 QB2:3/0/1024 1015 |--------->+ ACK 0.00 M:0xe8 1016 |--------->+ ACK 0.00 M:0xeb 1017 | | 1018 [[Server retransmits messages not acknowledged]] 1019 |<---------+ CON 2.05 M:0xe9 T:0xf0 O:1236 ET=23 QB2:1/1/1024 1020 | X<---+ CON 2.05 M:0xea T:0xf0 O:1236 ET=23 QB2:2/1/1024 1021 |--------->+ ACK 0.00 M:0xe9 1022 | | 1023 [[Server retransmits messages not acknowledged 1024 (exponential backoff)]] 1025 | X<---+ CON 2.05 M:0xea T:0xf0 O:1236 ET=23 QB2:2/1/1024 1026 | | 1027 [[Either transmission failure (acknowledge retry timeout) 1028 or successfully transmitted.]] 1030 Figure 10: Example of CON Notifications with Quick-Block2 Option 1032 It is implementation-dependent as to whether a CoAP session is 1033 terminated following acknowledge retry timeout, or whether the CoAP 1034 session continues to be used under such adverse traffic conditions. 1036 If there is likely to be the possibility of network transient losses, 1037 then the use of Non-confirmable traffic should be considered. 1039 Authors' Addresses 1041 Mohamed Boucadair 1042 Orange 1043 Rennes 35000 1044 France 1046 Email: mohamed.boucadair@orange.com 1048 Jon Shallow 1049 United Kingdom 1051 Email: supjps-ietf@jpshallow.com