idnits 2.17.1 draft-ietf-core-new-block-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (August 18, 2020) is 1344 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFCXXXX' is mentioned on line 787, but not defined == Outdated reference: A later version (-14) exists of draft-ietf-core-echo-request-tag-10 == Outdated reference: A later version (-25) exists of draft-ietf-dots-telemetry-11 -- Obsolete informational reference (is this intentional?): RFC 8782 (Obsoleted by RFC 9132) Summary: 0 errors (**), 0 flaws (~~), 4 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 CORE M. Boucadair 3 Internet-Draft Orange 4 Intended status: Standards Track J. Shallow 5 Expires: February 19, 2021 August 18, 2020 7 Constrained Application Protocol (CoAP) Block-Wise Transfer Options for 8 Faster Transmission 9 draft-ietf-core-new-block-00 11 Abstract 13 This document specifies new Constrained Application Protocol (CoAP) 14 Block-Wise transfer options: Block3 and Block4 Options. 16 These options are similar to the CoAP Block1 and Block2 Options, but 17 enable faster transmission rates for large amounts of data with less 18 packet interchanges as well as supporting faster recovery should any 19 of the blocks get lost in transmission. 21 Status of This Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at https://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on February 19, 2021. 38 Copyright Notice 40 Copyright (c) 2020 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (https://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 56 1.1. Existing CoAP Block-Wise Transfer Options . . . . . . . . 2 57 1.2. New CoAP Block-Wise Transfer Options . . . . . . . . . . 3 58 1.3. New CoAP Response Code . . . . . . . . . . . . . . . . . 4 59 1.4. Applicability Scope . . . . . . . . . . . . . . . . . . . 4 60 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 61 3. The Block3 and Block4 Options . . . . . . . . . . . . . . . . 5 62 3.1. Properties of Block3 and Block4 Options . . . . . . . . . 5 63 3.2. Structure of Block3 and Block4 Options . . . . . . . . . 6 64 3.3. Using the Block3 Option . . . . . . . . . . . . . . . . . 7 65 3.4. Using the Block 4 Option . . . . . . . . . . . . . . . . 9 66 3.5. Working with Observe and Block4 Options . . . . . . . . . 11 67 3.6. Working with Size1 and Size2 Options . . . . . . . . . . 11 68 3.7. Use of Block3 and Block4 Options Together . . . . . . . . 11 69 4. TBA3 (Missing Payloads) Response Code . . . . . . . . . . . . 11 70 5. Caching Considerations . . . . . . . . . . . . . . . . . . . 12 71 6. HTTP-Mapping Considerations . . . . . . . . . . . . . . . . . 13 72 7. Examples of Selective Block Recovery . . . . . . . . . . . . 13 73 7.1. Block3 Option: Non-Confirmable Example . . . . . . . . . 13 74 7.2. Block4 Option: Non-Confirmable Example . . . . . . . . . 15 75 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 76 8.1. New CoAP Options . . . . . . . . . . . . . . . . . . . . 16 77 8.2. New CoAP Response Code . . . . . . . . . . . . . . . . . 17 78 8.3. New Content Format . . . . . . . . . . . . . . . . . . . 17 79 9. Security Considerations . . . . . . . . . . . . . . . . . . . 17 80 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 17 81 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 18 82 11.1. Normative References . . . . . . . . . . . . . . . . . . 18 83 11.2. Informative References . . . . . . . . . . . . . . . . . 19 84 Appendix A. Examples with Confirmable Messages . . . . . . . . . 19 85 A.1. Block3 Option . . . . . . . . . . . . . . . . . . . . . . 20 86 A.2. Block4 Option . . . . . . . . . . . . . . . . . . . . . . 21 87 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 23 89 1. Introduction 91 1.1. Existing CoAP Block-Wise Transfer Options 93 The Constrained Application Protocol (CoAP) [RFC7252], although 94 inspired by HTTP, was designed to use UDP instead of TCP. The 95 message layer of CoAP over UDP includes support for reliable 96 delivery, simple congestion control, and flow control. [RFC7959] 97 introduced the CoAP Block1 and Block2 Options to handle data records 98 that cannot fit in a single IP packet, so not having to rely on IP 99 fragmentation and further updated by [RFC8323] for use over TCP, TLS, 100 and Websockets. 102 The CoAP Block1 and Block2 Options work well in environments where 103 there are no or minimal packet losses. These options operate 104 synchronously where each block has to be requested and can only ask 105 for (or send) the next block when the request for the previous block 106 has completed. Packet, and hence block transmission rate, is 107 controlled by Round Trip Times (RTTs). 109 There is a requirement for these blocks of data to be transmitted at 110 higher rates under network conditions where there may be transient 111 packet loss. An example is when a network is subject to a 112 Distributed Denial of Service (DDoS) attack and there is a need for 113 DDoS mitigation agents relying upon CoAP to communicate with each 114 other (e.g., [I-D.ietf-dots-telemetry]). As a reminder, [RFC7959] 115 recommends use of Confirmable (CON) responses to handle potential 116 packet loss; which does not work with a flooded pipe DDoS situation. 118 1.2. New CoAP Block-Wise Transfer Options 120 This document introduces the CoAP Block3 and Block4 Options. These 121 options are similar in operation to the CoAP Block1 and Block2 122 Options respectively, but enable faster transmissions of sets of 123 blocks of data with less packet interchanges as well as supporting 124 faster recovery should any of the Blocks get lost in transmission. 126 Using Non-confirmable (NON) messages, the faster transmissions occur 127 as all the Blocks can be transmitted serially (as are IP fragmented 128 packets) without having to wait for an acknowledgement or next 129 request from the remote CoAP peer. Recovery of missing Blocks is 130 faster in that multiple missing Blocks can be requested in a single 131 CoAP packet. 133 Note that the same performance benefits can be applied to Confirmable 134 messages if the value of NSTART is increased from 1 (Section 4.7 of 135 [RFC7252]). Some sample examples with Confirmable messages are 136 provided in Appendix A. 138 There is little, if any, benefit of using these options with CoAP 139 running over a reliable connection [RFC8323]. In this case, there is 140 no differentiation between Confirmable and NON as they are not used. 142 A CoAP endpoint can acknowledge all or a subset of the blocks. 143 Concretely, the receiving CoAP endpoint informs the CoAP endpoint 144 sender either successful receipt or reports on all blocks in the body 145 that have been not yet been received. The CoAP endpoint sender will 146 then retransmit only the blocks that have been lost in transmission. 148 Block3 and Block4 Options are used instead of Block1 and Block2 149 Options respectively because the transmission semantics and usage 150 have changed. 152 The deviations from Block1 and Block2 Options are specified in 153 Section 3. Pointers to appropriate [RFC7959] sections are provided. 155 The specification refers to the base CoAP methods defined in 156 Section 5.8 of [RFC7252] and the new CoAP methods, FETCH, PATCH, and 157 iPATCH introduced in [RFC8132]. 159 1.3. New CoAP Response Code 161 This document defines a new CoAP Response Code (Section 5.9 of 162 [RFC7252]), called TBA3 (Missing payloads), to report on payloads 163 using the Block3 Option that are not received by the server. 165 See Section 4 for more details. 167 1.4. Applicability Scope 169 The mechanism specified in the document includes guards to prevent a 170 CoAP agent from overloading the network by adopting an aggressive 171 sending rate. These guards MUST be followed in addition to the 172 existing CoAP congestion control as specified in Section 4.7 of 173 [RFC7252]. 175 This mechanism primarily targets applications such as DDoS Open 176 Threat Signaling (DOTS) that can't use Confirmable (CON) responses to 177 handle potential packet loss and that support application-specific 178 mechanisms to assess whether the remote peer is able to handle the 179 messages sent by a CoAP endpoint (e.g., DOTS heartbeats in 180 Section 4.7 of [RFC8782]). 182 2. Terminology 184 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 185 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 186 "OPTIONAL" in this document are to be interpreted as described in BCP 187 14 [RFC2119][RFC8174] when, and only when, they appear in all 188 capitals, as shown here. 190 Readers should be familiar with the terms and concepts defined in 191 [RFC7252]. 193 The terms "payload" and "body" are defined in [RFC7959]. The term 194 "payload" is thus used for the content of a single CoAP message 195 (i.e., a single block being transferred), while the term "body" is 196 used for the entire resource representation that is being transferred 197 in a block-wise fashion. 199 3. The Block3 and Block4 Options 201 3.1. Properties of Block3 and Block4 Options 203 The properties of Block3 and Block4 Options are shown in Table 1. 204 The formatting of this table follows the one used in Table 4 of 205 [RFC7252] (Section 5.10). The C, U, N, and R columns indicate the 206 properties Critical, Unsafe, NoCacheKey, and Repeatable defined in 207 Section 5.4 of [RFC7252]. Only C column is marked for the Block3 208 Option. Only C and R columns are marked for the Block4 Option. 210 +--------+---+---+---+---+-----------+--------+--------+---------+ 211 | Number | C | U | N | R | Name | Format | Length | Default | 212 +========+===+===+===+===+===========+========+========+=========+ 213 | TBA1 | x | | | | Block3 | uint | 0-3 | (none) | 214 | TBA2 | x | | | x | Block4 | uint | 0-3 | (none) | 215 +--------+---+---+---+---+-----------+--------+--------+---------+ 217 Table 1: CoAP Block3 and Block4 Option Properties 219 The Block3 and Block4 Options can be present in both the request and 220 response messages. The Block3 Option pertains to the request payload 221 and the Block4 Option pertains to the response payload. The Content- 222 Format Option applies to the body, not to the payload (i.e., it must 223 be the same for all payloads of the same body). 225 Block3 is useful with the payload-bearing POST, PUT, PATCH, and 226 iPATCH requests and their responses (2.01 and 2.04). Block4 Option 227 is useful with GET, POST, PUT, FETCH, PATCH, and iPATCH requests and 228 their payload-bearing responses (2.01, 2.03, 2.04, and 2.05) 229 (Section 5.5 of [RFC7252]). 231 To indicate support for Block4 responses, the CoAP client MUST 232 include the Block4 Option in a GET or similar requests so that the 233 server knows that the client supports this Block4 functionality 234 should it needs to send back a body that spans multiple payloads. 235 Otherwise, the server would use the Block2 Option (if supported) to 236 send back a message body that is too large to fit into a single IP 237 packet [RFC7959]. 239 If Block3 Option is present in a request or Block4 Option in a 240 response (i.e., in that message to the payload of which it pertains), 241 it indicates a block-wise transfer and describes how this specific 242 block-wise payload forms part of the entire body being transferred. 243 If it is present in the opposite direction, it provides additional 244 control on how that payload will be formed or was processed. 246 Implementation of Block3 (or Block4) Option is intended to be 247 optional. However, when it is present in a CoAP message, it MUST be 248 processed (or the message rejected). Therefore, Block3 and Block4 249 Options are identified as Critical options. 251 The Block3 and Block4 Options are safe to forward. That is, a CoAP 252 proxy that does not understand the Block3 (or Block4) Option should 253 forward the option on. 255 The Block4 Option is repeatable when requesting re-transmission of 256 missing Blocks, but not otherwise. Except that case, any request 257 carrying multiple Block3 (or Block4) Options MUST be handled 258 following the procedure specified in Section 5.4.5 of [RFC7252]. 260 PROBING_RATE parameter in CoAP indicates the average data rate that 261 must not be exceeded by a CoAP endpoint in sending to a peer endpoint 262 that does not respond. The body of blocks will be subjected to 263 PROBING_RATE (Section 4.7 of [RFC7252]). 265 The Block3 and Block4 Options, like the Block1 and Block2 Options, 266 are both a class E and a class U in terms of OSCORE processing (see 267 Section 4.1 of [RFC8613]): The Block3 (or Block4) Option MAY be an 268 Inner or Outer option. The Inner and Outer values are therefore 269 independent of each other. The Inner option is encrypted and 270 integrity protected between clients and servers, and provides message 271 body identification in case of end-to-end fragmentation of requests. 272 The Outer option is visible to proxies and labels message bodies in 273 case of hop-by-hop fragmentation of requests. 275 3.2. Structure of Block3 and Block4 Options 277 The structure of Block3 and Block4 Options follows the structure 278 defined in Section 2.2 of [RFC7959]. 280 There is no default value for the Block3 and Block4 Options. Absence 281 of one of these options is equivalent to an option value of 0 with 282 respect to the value of block number (NUM) and more bit (M) that 283 could be given in the option, i.e., it indicates that the current 284 block is the first and only block of the transfer (block number is 285 set to 0, M is unset). However, in contrast to the explicit value 0, 286 which would indicate a size of the block (SZX) of 0, and thus a size 287 value of 16 bytes, there is no specific explicit size implied by the 288 absence of the option -- the size is left unspecified. (As for any 289 uint, the explicit value 0 is efficiently indicated by a zero-length 290 option; this, therefore, is different in semantics from the absence 291 of the option). 293 3.3. Using the Block3 Option 295 The Block3 Option is used when the client wants to send a large 296 amount of data to the server using the POST, PUT, PATCH, or iPATCH 297 methods where the data and headers do not fit into a single packet. 299 When Block3 Option is used, the client MUST include Request-Tag 300 Option [I-D.ietf-core-echo-request-tag]. The Request-Tag value MUST 301 be the same for all of the blocks in the body of data that is being 302 transferred. It is also used to identify a particular block that 303 needs to be re-transmitted. The Request-Tag is opaque in nature, but 304 it is RECOMMENDED that the client treats it as an unsigned integer of 305 8 bytes in length. An implementation may want to consider limiting 306 this to 4 bytes to reduce packet overhead size. The server still 307 treats it as an opaque entity. The Request-Tag value MUST be 308 different for distinct bodies or sets of blocks of data and SHOULD be 309 incremented whenever a new body of data is being transmitted for a 310 CoAP session between peers. The initial Request-Tag value SHOULD be 311 randomly generated by the client. 313 The client sends all the individual payloads of the body using Block3 314 and Request-Tag Options, only expecting a response when all the 315 payloads have been sent. It is RECOMMENDED that after transmission 316 of every set of MAX_PAYLOADS payloads of a single body, a delay is 317 introduced of ACK_TIMEOUT (Section 4.8.2 of [RFC7252]) before the 318 next set of payload transmissions to manage potential congestion 319 issues. MAX_PAYLOADS should be configurable with a default value of 320 10. 322 Note: The default value is chosen for reasons similar to those 323 discussed in Section 5 of [RFC6928]. 325 For NON transmissions, it is permissible, but not required, to send 326 the ultimate payload of a MAX_PAYLOADS set as a Confirmable packet. 327 If a Confirmable packet is used, then the client MUST wait for the 328 ACK to be returned before sending the next set of payloads, which can 329 be in time terms less than the ACK_TIMEOUT delay. 331 Also, for NON transmissions, it is permissible, but not required, to 332 send a Confirmable packet for the final payload of a body (that is, M 333 bit unset). If a Confirmable packet is used, then the client MUST 334 wait for the 2.01 (Created) or 2.04 (Changed) Response Codes to be 335 returned for successful transmission, or TBA3 (Missing Payloads) 336 Response Code to then resend the missing blocks (if any). 338 With NON transmission, the server acknowledges receipt of all of the 339 payloads that make up the body or can respond at any time during the 340 receipt of the payloads to acknowledge that some of the payloads have 341 arrived, but others are missing. It is RECOMMENDED that, unless 342 there are receipt issues, the server only responds when the final 343 payload (i.e., M bit unset) is received. 345 For Confirmable transmission, the server MUST continue to acknowledge 346 each packet. NSTART will also need to be increased from the default 347 (1) to get faster transmission rates. 349 Tokens MUST be included. Each individual payload of the body MUST 350 have a different Token value. 352 A 2.01 (Created) or 2.04 (Changed) Response Code indicates successful 353 receipt of the entire body. The 2.31 (Continue) Response Code MUST 354 NOT be used. 356 A 4.00 (Bad Request) Response Code MUST be returned if the request 357 does not include a Request-Tag Option but does include a Block3 358 option. 360 A 4.02 (Bad Option) Response Code MUST be returned if the server does 361 not support the Block3 Option. 363 Use of 4.08 (Request Entity Incomplete) Response Code is discouraged 364 when using Block3 Option because packets may arrive out of sequence; 365 TBA3 (Missing Payloads) Response Code (Section 4) SHOULD be used 366 instead. However, 4.08 (Request Entity Incomplete) Response Code is 367 still valid to reject a Content-Format mismatch. 369 A 4.13 (Request Entity Too Large) Response Code can be returned under 370 similar conditions to those discussed in Section 2.9.3 of [RFC7959]. 372 A TBA3 (Missing Payloads) Response Code indicates that some of the 373 payloads are missing and need to be resent. The client then re- 374 transmits the missing payloads using the Request-Tag and Block3 to 375 specify the block number, SZX, and M bit as appropriate. The 376 Request-Tag value to use is determined from the payload of the TBA3 377 (Missing Payloads) Response Code. If the client dos not recognize 378 the Request-Tag, the client can ignore this response. As discussed 379 above, the sending of the list of missing blocks is subject to 380 MAX_PAYLOADS. 382 If the server has not received the final payload (i.e., a block with 383 M bit unset), but one or other payloads have been received, it SHOULD 384 wait for up to MAX_TRANSMIT_SPAN (Section 4.8.2 of [RFC7252]) before 385 sending the TBA3 (Missing Payloads) Response Code. However, this 386 timer MAY be reduced to two times ACK_TIMEOUT before sending a TBA3 387 (Missing Payloads) Response Code to cover the situation where 388 MAX_PAYLOADS has been triggered by the client causing a break in 389 transmission. 391 In all cases, multiple TBA3 (Missing Payloads) Response Codes are 392 traffic limited by PROBING_RATE. 394 If the client transmits a new body of data with a new Request-Tag to 395 the same resource on a server, the server MUST remove any partially 396 received body held for a previous Request-Tag for that resource. 398 If the server receives a duplicate block with the same Request-Tag, 399 it SHOULD silently ignore the packet. 401 A server SHOULD only maintain a partial body (missing payloads) for 402 up to EXCHANGE_LIFETIME (Section 4.8.2 of [RFC7252]). 404 3.4. Using the Block 4 Option 406 Support for the receipt of Block4 Option by the client is indicated 407 by using the Block4 Option in the GET, POST, PUT, FETCH, PATCH or 408 iPATCH request. If the Block4 Option is not included in the request, 409 the server MUST NOT send data using the Block4 Option, but can use 410 Block2 if supported instead. 412 In a request, the Block4 MUST always have the M bit set to 0. 414 The payloads sent back from the server as a response MUST all have 415 the same ETag (Section 5.10.6 of [RFC7252]) for the same body. The 416 server MUST NOT use the same ETag value for different representations 417 of a resource. 419 The sending of the payloads is subject to MAX_PAYLOADS. If 420 MAX_PAYLOADS is exceeded, the server MUST introduce an ACK_TIMEOUT 421 delay before transmitting the next set of payloads. 423 The ETag is opaque in nature, but it is RECOMMENDED that the server 424 treats it as an unsigned integer of 8 bytes in length. An 425 implementation may want to consider limiting this to 4 bytes to 426 reduce packet overhead size. The client still treats it as an opaque 427 entity. The ETag value MUST be different for distinct bodies or sets 428 of blocks of data and SHOULD be incremented whenever a new body of 429 data is being transmitted for a CoAP session between peers. The 430 initial ETag value SHOULD be randomly generated by the server. 432 For NON transmission, it is permissible, but not required, to send 433 the ultimate payload of a MAX_PAYLOADS set as a Confirmable packet. 435 If a Confirmable packet is used, then the server MUST wait for the 436 ACK to be received before sending the next set of payloads, which can 437 be in time terms less than the ACK_TIMEOUT delay. 439 Also, for NON transmission, it is permissible, but not required, to 440 send a Confirmable packet for the final payload of a body (i.e., M 441 bit unset). If a Confirmable packet is used, the server MUST wait 442 for the ACK to be returned for successful transmission. 444 If the client detects that some of the payloads are missing, the 445 missing payloads are requested by issuing a new GET, POST, PUT, 446 FETCH, PATCH, or iPATCH request that contains one or more Block4 447 Options that define the missing blocks. A new Token value MUST be 448 used for this request. The rate of requests for missing blocks is 449 subject to PROBING_RATE. The ETag Option MUST NOT be used in the 450 request as the server could respond with a 2.03 (Valid Response) with 451 no payload. If the server responds with a different ETag Option 452 value (as the resource representation has changed), then the client 453 SHOULD drop all the payloads for the current body that are no longer 454 valid. 456 The client may elect to request the missing blocks or just ignore the 457 partial body. 459 All the payload responses to a specific GET, POST, PUT, FETCH, PATCH, 460 or iPATCH request MUST have the same Token value as in the request. 462 With NON transmission, the client only needs to indicate that some of 463 the payloads are missing by issuing a GET, POST, PUT, FETCH, PATCH, 464 or iPATCH request for the missing blocks. 466 For Confirmable transmission, the client SHOULD continue to 467 acknowledge each packet as well as issuing a separate GET, POST, PUT, 468 FETCH, PATCH, or iPATCH for the missing blocks. NSTART will also 469 need to be increased from the default (1) to get faster transmission 470 rates. 472 If the server transmits a new body of data (e.g., a triggered 473 Observe) with a new ETag to the same client with the same Token 474 value, the client MUST remove any partially received body held for a 475 previous ETag for that Token. 477 If the client receives a duplicate block with the same ETag, it 478 SHOULD silently ignore the packet. 480 A client SHOULD only maintain a partial body (missing payloads) for 481 up to EXCHANGE_LIFETIME (Section 4.8.2 of [RFC7252]) or as defined by 482 the Max-Age Option whichever is the less. 484 3.5. Working with Observe and Block4 Options 486 As the blocks of the body are sent without waiting for 487 acknowledgement of the individual blocks, the Observe value [RFC7641] 488 MUST be the same for all the blocks of the same body. 490 Likewise, the Tokens MUST all have the same value for all the blocks 491 of the same body. This is so that if any of the blocks get lost 492 during transmission (including the first one), the receiving CoAP 493 endpoint can take the appropriate decisions as to how to continue 494 (implementation specific). 496 If the client requests missing blocks, the client MUST use a 497 different Token; all repeated missing blocks for that new request 498 MUST use the new Token. 500 3.6. Working with Size1 and Size2 Options 502 Section 4 of [RFC7959] defines two CoAP options: Size1 for indicating 503 the size of the representation transferred in requests and Size2 for 504 indicating the size of the representation transferred in responses. 506 It is RECOMMENDED that the Size1 Option is used with the Block3 507 Option. It is also RECOMMENDED that the Size2 Option is used with 508 the Block4 Option. 510 If Size1 or Size2 Options are used, they MUST be used in all payloads 511 of the body and MUST have the same value. 513 3.7. Use of Block3 and Block4 Options Together 515 The behavior is similar to the one defined in Section 3.3 of 516 [RFC7959] with Block3 substituted for Block1 and Block4 for Block2. 518 4. TBA3 (Missing Payloads) Response Code 520 TBA3 (Missing Payloads) Response Code is a new client error status 521 code (Section 5.9.2 of [RFC7252]) used to indicate that the server 522 has not received all of the blocks of the request body that it needs 523 to proceed. 525 Likely causes are the client has not sent all blocks, some blocks 526 were dropped during transmission, or the client has sent them 527 sufficiently long ago that the server has already discarded them. 529 The data payload of the TBA3 (Missing Payloads) Response Code is 530 encoded as a CBOR Sequence [RFC8742]. First is CBOR encoded Request- 531 Tag followed by 1 or more missing CBOR encoded missing block numbers. 533 The missing block numbers MUST be unique per TBA3 (Missing Payloads) 534 when created by the server; the client SHOULD drop any duplicates in 535 the same TBA3 (Missing Payloads) message. 537 The Content-Format Option (Section 5.10.3 of [RFC7252]) MUST be used 538 in the TBA3 (Missing Payloads) Response Code. It MUST be set to 539 "application/missing-blocks+cbor-seq" (see Section 8.3). 541 The Concise Data Definition Language [RFC8610] for the data 542 describing these missing blocks is as follows: 544 TBA3-payload = (request-tag, missing-block-list) 545 ; A copy of the opaque Request-Tag value 546 request-tag = bstr 547 missing-block-list = [1 * missing-block-number] 548 ; A unique block number not received 549 missing-block-number = uint 551 Figure 1: Structure of the Missing Blocks Payload 553 If the size of the TBA3 (Missing Payloads) response packet is larger 554 than that defined by Section 4.6 [RFC7252], then the number of 555 missing blocks MUST be limited so that the response can fit into a 556 single packet. If necessary, multiple TBA3 (Missing Payloads) 557 Response Codes can be sent back; each covering a Request-Tag and a 558 unique set of missing blocks. The same Token can be used for the 559 multiple TBA3 (missing Payloads) if this is the case. 561 5. Caching Considerations 563 The Block3 and Block4 Options are part of the cache key. As such, a 564 CoAP proxy that does not understand the Block3 and Block4 Options 565 must follow the recommendations in Section 5.7.1 of [RFC7252] for 566 caching. 568 This specification does not require a proxy to obtain the complete 569 representation before it serves parts of it to the client. 570 Otherwise, the considerations discussed in Section 2.10 of [RFC7959] 571 apply for the Block3 and Block4 Options (with Block3 substituted for 572 Block1 and Block4 substituted for Block2) for proxies that support 573 Block3 and Block4 Options. 575 A proxy that supports Block4 Option MUST be prepared to receive a GET 576 or similar message indicating one or more missing blocks. The proxy 577 can serve from its cache missing blocks that are available in its 578 cache in a set as a server would send all the Block4s. If one or 579 more requested blocks are not available in the cache, the proxy 580 SHOULD update the GET request by removing the blocks that it can 581 serve from the cache, and then forward on the request to the next 582 hop. 584 Alternatively, the original request unmodified (from the missing 585 block perspective) MAY be forwarded on to the server. All the 586 responses are then passed back to the client with the cache getting 587 updated. 589 How long a CoAP endpoint (or proxy) keeps the body in its cache is 590 implementation specific (e.g., it may be based on Max-Age). 592 6. HTTP-Mapping Considerations 594 As a reminder, the basic normative requirements on HTTP/CoAP mappings 595 are defined in Section 10 of [RFC7252]. The implementation 596 guidelines for HTTP/CoAP mappings are elaborated in [RFC8075]. 598 The rules defined in Section 5 of [RFC7959] are to be followed. 600 7. Examples of Selective Block Recovery 602 This section provides some sample flows to illustrate the use of 603 Block3 and Block4 Options. Figure 2 lists the conventions that are 604 used in the following subsections. 606 T: Token value 607 O: Observe Option value 608 M: Message ID 609 RT: Request-Tag 610 ET: ETag 611 B3: Block3 Option values NUM/More/SZX 612 B4: Block3 Option values NUM/More/SZX 613 \: Trimming long lines 614 [[]]: Comments 615 -->X: Message loss 616 X<--: Message loss 618 Figure 2: Notations Used in the Figures 620 7.1. Block3 Option: Non-Confirmable Example 622 Figure 3 depicts an example of a NON PUT request conveying Block3 623 Option. All the blocks are received by the server. 625 CoAP CoAP 626 Client Server 627 | | 628 +--------->| NON PUT /path M:0x01 T:0xf0 RT=10 B3:0/1/1024 629 +--------->| NON PUT /path M:0x02 T:0xf1 RT=10 B3:1/1/1024 630 +--------->| NON PUT /path M:0x03 T:0xf2 RT=10 B3:2/1/1024 631 +--------->| NON PUT /path M:0x04 T:0xf3 RT=10 B3:3/0/1024 632 |<---------+ NON 2.04 M:0xf1 T:0xf3 633 ... 635 Figure 3: Example of NON Request with Block3 Option (Without Loss) 637 Consider now a scenario where a new body of data is to be sent by the 638 client, but some blocks are dropped in transmission as illustrated in 639 Figure 4. 641 CoAP CoAP 642 Client Server 643 | | 644 +--------->| NON PUT /path M:0x05 T:0xe0 RT=11 B3:0/1/1024 645 +--->X | NON PUT /path M:0x06 T:0xe1 RT=11 B3:1/1/1024 646 +--->X | NON PUT /path M:0x07 T:0xe2 RT=11 B3:2/1/1024 647 +--------->| NON PUT /path M:0x08 T:0xe3 RT=11 B3:3/0/1024 648 | | 649 ... 651 Figure 4: Example of NON Request with Block3 Option (With Loss) 653 The server realizes that some blocks are missing and asks for the 654 missing ones in one go (Figure 5). It does so by indicating which 655 blocks have been received in the data portion of the response. 657 CoAP CoAP 658 Client Server 659 | | 660 ... 661 |<---------+ NON TBA3 M:0xf2 T:0xe3 [Missing 1,2 for RT=11] 662 +--------->| NON PUT /path M:0x09 T:0xe4 RT=11 B3:1/1/1024 663 +--->X | NON PUT /path M:0x0a T:0xe5 RT=11 B3:2/1/1024 664 | | 665 |<---------+ NON TBA3 M:0xf3 T:0xe4 [Missing 2 for RT=11] 666 +--------->| NON PUT /path M:0x0b T:0xe6 RT=11 B3:2/1/1024 667 |<---------+ NON 2.04 M:0xf4 T:0xe6 668 | | 669 ... 671 Figure 5: Example of NON Request with Block3 Option (Blocks Recovery) 672 Under high levels of traffic loss, the client can elect not to retry 673 sending missing blocks of data. This decision is implementation 674 specific. 676 7.2. Block4 Option: Non-Confirmable Example 678 Figure 6 illustrates the example of Block4 Option. The client sends 679 a NON GET carrying an Observe and a Block4 Options. The Block4 680 Option indicates a size hint (1024 bytes). This request is replied 681 by the server using four (4) blocks that are transmitted to the 682 client without any loss. Each of these blocks carries a Block4 683 Option. The same process is repeated when an Observe is triggered, 684 but no loss is experienced by any of the notification blocks. 686 CoAP CoAP 687 Client Server 688 | | 689 +--------->| NON GET /path M:0x01 T:0xf0 O:0 B4:0/0/1024 690 |<---------+ NON 2.05 M:0xf1 T:0xf0 O:1234 ET=21 B4:0/1/1024 691 |<---------+ NON 2.05 M:0xf2 T:0xf0 O:1234 ET=21 B4:1/1/1024 692 |<---------+ NON 2.05 M:0xf3 T:0xf0 O:1234 ET=21 B4:2/1/1024 693 |<---------+ NON 2.05 M:0xf4 T:0xf0 O:1234 ET=21 B4:3/0/1024 694 ... 695 [[Observe triggered]] 696 |<---------+ NON 2.05 M:0xf5 T:0xf0 O:1235 ET=22 B4:0/1/1024 697 |<---------+ NON 2.05 M:0xf6 T:0xf0 O:1235 ET=22 B4:1/1/1024 698 |<---------+ NON 2.05 M:0xf7 T:0xf0 O:1235 ET=22 B4:2/1/1024 699 |<---------+ NON 2.05 M:0xf8 T:0xf0 O:1235 ET=22 B4:3/0/1024 700 ... 702 Figure 6: Example of NON Notifications with Block4 Option (Without 703 Loss) 705 Figure 7 shows the example of an Observe that is triggered but for 706 which some notification blocks are lost. The client detects the 707 missing blocks and request their retransmission. It does so by 708 indicating the blocks that were successfully received. 710 CoAP CoAP 711 Client Server 712 | | 713 ... 714 [[Observe triggered]] 715 |<---------+ NON 2.05 M:0xf9 T:0xf0 O:1236 ET=23 B4:0/1/1024 716 | X<---+ NON 2.05 M:0xfa T:0xf0 O:1236 ET=23 B4:1/1/1024 717 | X<---+ NON 2.05 M:0xfb T:0xf0 O:1236 ET=23 B4:2/1/1024 718 |<---------+ NON 2.05 M:0xfc T:0xf0 O:1236 ET=23 B4:3/0/1024 719 | | 720 [[Client realizes blocks are missing and asks for the missing 721 ones in one go]] 722 +--------->| NON GET /path M:0x02 T:0xf1 B4:1/0/1024\ 723 | | B4:2/0/1024 724 | X<---+ NON 2.05 M:0xfd T:0xf1 ET=23 B4:1/1/1024 725 |<---------+ NON 2.05 M:0xfe T:0xf1 ET=23 B4:2/1/1024 726 | | 727 [[Get the final missing block]] 728 +--------->| NON GET /path M:0x03 T:0xf2 B4:1/0/1024 729 |<---------+ NON 2.05 M:0xff T:0xf2 ET=23 B4:1/1/1024 730 ... 732 Figure 7: Example of NON Notifications with Block4 Option (Blocks 733 Recovery) 735 Under high levels of traffic loss, the client can elect not to retry 736 getting missing blocks of data. This decision is implementation 737 specific. 739 8. IANA Considerations 741 8.1. New CoAP Options 743 IANA is requested to add the following entries to the "CoAP Option 744 Numbers" sub-registry available at https://www.iana.org/assignments/ 745 core-parameters/core-parameters.xhtml#option-numbers: 747 +--------+------------------+-----------+ 748 | Number | Name | Reference | 749 +========+==================+===========+ 750 | TBA1 | Block3 | [RFCXXXX] | 751 | TBA2 | Block4 | [RFCXXXX] | 752 +--------+------------------+-----------+ 754 Table 2: CoAP Block3 and Block4 Option Numbers 756 This document suggests 21 (TBA1) and 25 (TBA2) as a values to be 757 assigned for the new option numbers. 759 8.2. New CoAP Response Code 761 IANA is requested to add the following entry to the "CoAP Response 762 Codes" sub-registry available at https://www.iana.org/assignments/ 763 core-parameters/core-parameters.xhtml#response-codes: 765 +------+------------------+-----------+ 766 | Code | Description | Reference | 767 +======+==================+===========+ 768 | TBA3 | Missing Payloads | [RFCXXXX] | 769 +------+------------------+-----------+ 771 Table 3: New CoAP Response Code 773 This document suggests 4.19 (TBA3) as a value to be assigned for the 774 new Response Code. 776 8.3. New Content Format 778 This document requests IANA to register the CoAP Content-Format ID 779 for the "application/missing-blocks+cbor-seq" media type in the "CoAP 780 Content-Formats" registry available at 781 https://www.iana.org/assignments/core-parameters/core- 782 parameters.xhtml#content-formats: 784 o Media Type: application/missing-blocks+cbor-seq 785 o Encoding: - 786 o Id: TBD4 787 o Reference: [RFCXXXX] 789 9. Security Considerations 791 Security considerations discussed in Section 9 of [RFC7959] should be 792 taken into account. 794 Security considerations related to the use of Request-Tag are 795 discussed in Section 5 of [I-D.ietf-core-echo-request-tag]. 797 10. Acknowledgements 799 Thanks to Achim Kraus and Jim Schaad for the comments on the mailing 800 list. 802 Special thanks to Christian Amsuess and Carsten Bormann for their 803 suggestions and several reviews, which improved this specification 804 significantly. 806 Some text from [RFC7959] is reused for readers convenience. 808 11. References 810 11.1. Normative References 812 [I-D.ietf-core-echo-request-tag] 813 Amsuess, C., Mattsson, J., and G. Selander, "CoAP: Echo, 814 Request-Tag, and Token Processing", draft-ietf-core-echo- 815 request-tag-10 (work in progress), July 2020. 817 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 818 Requirement Levels", BCP 14, RFC 2119, 819 DOI 10.17487/RFC2119, March 1997, 820 . 822 [RFC7252] Shelby, Z., Hartke, K., and C. Bormann, "The Constrained 823 Application Protocol (CoAP)", RFC 7252, 824 DOI 10.17487/RFC7252, June 2014, 825 . 827 [RFC7641] Hartke, K., "Observing Resources in the Constrained 828 Application Protocol (CoAP)", RFC 7641, 829 DOI 10.17487/RFC7641, September 2015, 830 . 832 [RFC7959] Bormann, C. and Z. Shelby, Ed., "Block-Wise Transfers in 833 the Constrained Application Protocol (CoAP)", RFC 7959, 834 DOI 10.17487/RFC7959, August 2016, 835 . 837 [RFC8075] Castellani, A., Loreto, S., Rahman, A., Fossati, T., and 838 E. Dijk, "Guidelines for Mapping Implementations: HTTP to 839 the Constrained Application Protocol (CoAP)", RFC 8075, 840 DOI 10.17487/RFC8075, February 2017, 841 . 843 [RFC8132] van der Stok, P., Bormann, C., and A. Sehgal, "PATCH and 844 FETCH Methods for the Constrained Application Protocol 845 (CoAP)", RFC 8132, DOI 10.17487/RFC8132, April 2017, 846 . 848 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 849 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 850 May 2017, . 852 [RFC8323] Bormann, C., Lemay, S., Tschofenig, H., Hartke, K., 853 Silverajan, B., and B. Raymor, Ed., "CoAP (Constrained 854 Application Protocol) over TCP, TLS, and WebSockets", 855 RFC 8323, DOI 10.17487/RFC8323, February 2018, 856 . 858 [RFC8613] Selander, G., Mattsson, J., Palombini, F., and L. Seitz, 859 "Object Security for Constrained RESTful Environments 860 (OSCORE)", RFC 8613, DOI 10.17487/RFC8613, July 2019, 861 . 863 [RFC8742] Bormann, C., "Concise Binary Object Representation (CBOR) 864 Sequences", RFC 8742, DOI 10.17487/RFC8742, February 2020, 865 . 867 11.2. Informative References 869 [I-D.ietf-dots-telemetry] 870 Boucadair, M., Reddy.K, T., Doron, E., chenmeiling, c., 871 and J. Shallow, "Distributed Denial-of-Service Open Threat 872 Signaling (DOTS) Telemetry", draft-ietf-dots-telemetry-11 873 (work in progress), July 2020. 875 [RFC6928] Chu, J., Dukkipati, N., Cheng, Y., and M. Mathis, 876 "Increasing TCP's Initial Window", RFC 6928, 877 DOI 10.17487/RFC6928, April 2013, 878 . 880 [RFC8610] Birkholz, H., Vigano, C., and C. Bormann, "Concise Data 881 Definition Language (CDDL): A Notational Convention to 882 Express Concise Binary Object Representation (CBOR) and 883 JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, 884 June 2019, . 886 [RFC8782] Reddy.K, T., Ed., Boucadair, M., Ed., Patil, P., 887 Mortensen, A., and N. Teague, "Distributed Denial-of- 888 Service Open Threat Signaling (DOTS) Signal Channel 889 Specification", RFC 8782, DOI 10.17487/RFC8782, May 2020, 890 . 892 Appendix A. Examples with Confirmable Messages 894 These examples assume NSTART has been increased to at least 4. 896 The notations provided in Figure 2 are used in the following 897 subsections. 899 A.1. Block3 Option 901 Let's now consider the use Block3 Option with a CON request as shown 902 in Figure 8. All the blocks are acknowledged (ACK). 904 CoAP CoAP 905 Client Server 906 | | 907 +--------->| CON PUT /path M:0x01 T:0xf0 RT=10 B3:0/1/1024 908 +--------->| CON PUT /path M:0x02 T:0xf1 RT=10 B3:1/1/1024 909 +--------->| CON PUT /path M:0x03 T:0xf2 RT=10 B3:2/1/1024 910 +--------->| CON PUT /path M:0x04 T:0xf3 RT=10 B3:3/0/1024 911 |<---------+ ACK 0.00 M:0x01 912 |<---------+ ACK 0.00 M:0x02 913 |<---------+ ACK 0.00 M:0x03 914 |<---------+ ACK 0.00 M:0x04 916 Figure 8: Example of CON Request with Block3 Option (Without Loss) 918 Now, suppose that a new body of data is to sent but with some blocks 919 dropped in transmission as illustrated in Figure 9. The client will 920 retry sending blocks for which no ACK was received. 922 CoAP CoAP 923 Client Server 924 | | 925 +--------->| CON PUT /path M:0x05 T:0xf4 RT=11 B3:0/1/1024 926 +--->X | CON PUT /path M:0x06 T:0xf5 RT=11 B3:1/1/1024 927 +--->X | CON PUT /path M:0x07 T:0xf6 RT=11 B3:2/1/1024 928 +--------->| CON PUT /path M:0x08 T:0xf7 RT=11 B3:3/1/1024 929 |<---------+ ACK 0.00 M:0x05 930 |<---------+ ACK 0.00 M:0x08 931 | | 932 [[The client retries sending packets not acknowledged]] 933 +--------->| CON PUT /path M:0x06 T:0xf5 RT=11 B3:1/1/1024 934 +--->X | CON PUT /path M:0x07 T:0xf6 RT=11 B3:2/1/1024 935 |<---------+ ACK 0.00 M:0x06 936 | | 937 [[The client retransmits messages not acknowledged 938 (exponential backoff)]] 939 +--->? | CON PUT /path M:0x07 T:0xf6 RT=11 B3:2/1/1024 940 | | 941 [[Either transmission failure (acknowledge retry timeout) 942 or successfully transmitted.]] 944 Figure 9: Example of CON Request with Block3 Option (Blocks Recovery) 945 It is implementation dependent as to whether a CoAP session is 946 terminated following acknowledge retry timeout, or whether the CoAP 947 session continues to be used under such adverse traffic conditions. 949 If there is likely to be the possibility of network transient losses, 950 then the use of Non-confirmable traffic should be considered. 952 A.2. Block4 Option 954 An example of the use of Block4 Option with Confirmable messages is 955 shown in Figure 10. 957 Client Server 958 | | 959 +--------->| CON GET /path M:0x01 T:0xf0 O:0 B4:0/0/1024 960 |<---------+ ACK 2.05 M:0x01 T:0xf0 O:1234 ET=21 B4:0/1/1024 961 |<---------+ ACK 2.05 M:0xe1 T:0xf0 O:1234 ET=21 B4:1/1/1024 962 |<---------+ ACK 2.05 M:0xe2 T:0xf0 O:1234 ET=21 B4:2/1/1024 963 |<---------+ ACK 2.05 M:0xe3 T:0xf0 O:1234 ET=21 B4:3/0/1024 964 ... 965 [[Observe triggered]] 966 |<---------+ CON 2.05 M:0xe4 T:0xf0 O:1235 ET=22 B4:0/1/1024 967 |<---------+ CON 2.05 M:0xe5 T:0xf0 O:1235 ET=22 B4:1/1/1024 968 |<---------+ CON 2.05 M:0xe6 T:0xf0 O:1235 ET=22 B4:2/1/1024 969 |<---------+ CON 2.05 M:0xe7 T:0xf0 O:1235 ET=22 B4:3/0/1024 970 |--------->+ ACK 0.00 M:0xe4 971 |--------->+ ACK 0.00 M:0xe5 972 |--------->+ ACK 0.00 M:0xe6 973 |--------->+ ACK 0.00 M:0xe7 974 ... 975 [[Observe triggered]] 976 |<---------+ CON 2.05 M:0xe8 T:0xf0 O:1236 ET=23 B4:0/1/1024 977 | X<---+ CON 2.05 M:0xe9 T:0xf0 O:1236 ET=23 B4:1/1/1024 978 | X<---+ CON 2.05 M:0xea T:0xf0 O:1236 ET=23 B4:2/1/1024 979 |<---------+ CON 2.05 M:0xeb T:0xf0 O:1236 ET=23 B4:3/0/1024 980 |--------->+ ACK 0.00 M:0xe8 981 |--------->+ ACK 0.00 M:0xeb 982 | | 983 [[Server retransmits messages not acknowledged]] 984 |<---------+ CON 2.05 M:0xe9 T:0xf0 O:1236 ET=23 B4:1/1/1024 985 | X<---+ CON 2.05 M:0xea T:0xf0 O:1236 ET=23 B4:2/1/1024 986 |--------->+ ACK 0.00 M:0xe9 987 | | 988 [[Server retransmits messages not acknowledged 989 (exponential backoff)]] 990 | X<---+ CON 2.05 M:0xea T:0xf0 O:1236 ET=23 B4:2/1/1024 991 | | 992 [[Either transmission failure (acknowledge retry timeout) 993 or successfully transmitted.]] 995 Figure 10: Example of CON Notifications with Block4 Option 997 It is implementation-dependent as to whether a CoAP session is 998 terminated following acknowledge retry timeout, or whether the CoAP 999 session continues to be used under such adverse traffic conditions. 1001 If there is likely to be the possibility of network transient losses, 1002 then the use of Non-confirmable traffic should be considered. 1004 Authors' Addresses 1006 Mohamed Boucadair 1007 Orange 1008 Rennes 35000 1009 France 1011 Email: mohamed.boucadair@orange.com 1013 Jon Shallow 1014 United Kingdom 1016 Email: supjps-ietf@jpshallow.com