idnits 2.17.1 draft-bormann-coap-misc-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: o ACK messages that fail to decode are hard to process. The requesting node MAY repeat the request with fewer options in order to receive a simpler answer; if that is not possible, the decoding failure should be treated like a client error. Conversely, nodes SHOULD not send critical options in ACK messages unless the CON message eliciting the ACK contained options that justify this. (There may be exceptions, e.g., a node is always allowed to send a Block option with a large resource representation. A requestor that does not understand Block may never be able to receive that resource representation properly, so it is appropriate to treat the situation as a client error.) -- The document date (July 01, 2010) is 5041 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-02) exists of draft-hartke-coap-observe-00 == Outdated reference: A later version (-18) exists of draft-ietf-core-coap-00 == Outdated reference: A later version (-26) exists of draft-ietf-httpbis-p1-messaging-09 == Outdated reference: A later version (-26) exists of draft-ietf-httpbis-p4-conditional-09 == Outdated reference: A later version (-26) exists of draft-ietf-httpbis-p6-cache-09 ** Obsolete normative reference: RFC 2616 (Obsoleted by RFC 7230, RFC 7231, RFC 7232, RFC 7233, RFC 7234, RFC 7235) Summary: 1 error (**), 0 flaws (~~), 8 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 CoRE Working Group C. Bormann 3 Internet-Draft K. Hartke 4 Intended status: Informational Universitaet Bremen TZI 5 Expires: January 2, 2011 July 01, 2010 7 Miscellaneous additions to CoAP 8 draft-bormann-coap-misc-04 10 Abstract 12 This short I-D makes a number of partially interrelated proposals how 13 to solve certain problems in the CoRE WG's main protocol, CoAP. 15 Status of this Memo 17 This Internet-Draft is submitted in full conformance with the 18 provisions of BCP 78 and BCP 79. 20 Internet-Drafts are working documents of the Internet Engineering 21 Task Force (IETF). Note that other groups may also distribute 22 working documents as Internet-Drafts. The list of current Internet- 23 Drafts is at http://datatracker.ietf.org/drafts/current/. 25 Internet-Drafts are draft documents valid for a maximum of six months 26 and may be updated, replaced, or obsoleted by other documents at any 27 time. It is inappropriate to use Internet-Drafts as reference 28 material or to cite them other than as "work in progress." 30 This Internet-Draft will expire on January 2, 2011. 32 Copyright Notice 34 Copyright (c) 2010 IETF Trust and the persons identified as the 35 document authors. All rights reserved. 37 This document is subject to BCP 78 and the IETF Trust's Legal 38 Provisions Relating to IETF Documents 39 (http://trustee.ietf.org/license-info) in effect on the date of 40 publication of this document. Please review these documents 41 carefully, as they describe your rights and restrictions with respect 42 to this document. Code Components extracted from this document must 43 include Simplified BSD License text as described in Section 4.e of 44 the Trust Legal Provisions and are provided without warranty as 45 described in the Simplified BSD License. 47 Table of Contents 49 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 50 2. A Compact Accept Option . . . . . . . . . . . . . . . . . . . 4 51 3. Representing Durations . . . . . . . . . . . . . . . . . . . . 6 52 3.1. Pseudo-Floating Point . . . . . . . . . . . . . . . . . . 6 53 3.2. A Duration Type for CoAP . . . . . . . . . . . . . . . . . 8 54 4. URI encoding . . . . . . . . . . . . . . . . . . . . . . . . . 9 55 4.1. Stateful URI compression . . . . . . . . . . . . . . . . . 9 56 5. Block-wise transfers . . . . . . . . . . . . . . . . . . . . . 11 57 5.1. The Block Option . . . . . . . . . . . . . . . . . . . . . 11 58 6. Option Encoding . . . . . . . . . . . . . . . . . . . . . . . 15 59 6.1. A More Efficient Option Encoding . . . . . . . . . . . . . 15 60 6.2. Critical Options . . . . . . . . . . . . . . . . . . . . . 16 61 6.3. Errors in Options . . . . . . . . . . . . . . . . . . . . 16 62 6.4. Payload-Length Option . . . . . . . . . . . . . . . . . . 17 63 6.5. Problems with specific options . . . . . . . . . . . . . . 18 64 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19 65 8. Security Considerations . . . . . . . . . . . . . . . . . . . 21 66 8.1. Amplification Attacks . . . . . . . . . . . . . . . . . . 21 67 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22 68 9.1. Normative References . . . . . . . . . . . . . . . . . . . 22 69 9.2. Informative References . . . . . . . . . . . . . . . . . . 23 70 Appendix A. Things we won't do . . . . . . . . . . . . . . . . . 24 71 A.1. An efficient stateless URI encoding . . . . . . . . . . . 24 72 Appendix B. Experimental Options . . . . . . . . . . . . . . . . 26 73 B.1. Options indicating absolute time . . . . . . . . . . . . . 26 74 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 28 76 1. Introduction 78 The CoRE WG is tasked with standardizing an Application Protocol for 79 Constrained Networks/Nodes, CoAP. This protocol is intended to 80 provide RESTful [REST] services not unlike HTTP [RFC2616], while 81 reducing the complexity of implementation as well as the size of 82 packets exchanged in order to make these services useful in a highly 83 constrained network of themselves highly constrained nodes. 85 This objective requires restraint in a number of sometimes 86 conflicting ways: 88 o reducing implementation complexity in order to minimize code size, 90 o reducing message sizes in order to minimize the number of 91 fragments needed for each message (in turn to maximize the 92 probability of delivery of the message), the amount of 93 transmission power needed and the loading of the limited-bandwidth 94 channel, 96 o reducing requirements on the environment such as stable storage, 97 good sources of randomness or user interaction capabilities. 99 This draft attempts to address a number of problems not yet 100 adequately solved in [I-D.ietf-core-coap]. The solutions proposed to 101 these problems are somewhat interrelated and are therefore presented 102 in one draft. 104 In this document, the key words "MUST", "MUST NOT", "REQUIRED", 105 "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", 106 and "OPTIONAL" are to be interpreted as described in BCP 14 [RFC2119] 107 and indicate requirement levels for compliant CoAP implementations. 109 2. A Compact Accept Option 111 A resource may be available in a number of representations. Without 112 some information from the client, a server has no easy way to decide 113 which of these would be best served. HTTP has an Accept: request 114 header that a client can use to indicate the media types supported, 115 allowing the server to decide. This header is somewhat unpopular as, 116 for a web browser, there are too many media types to choose from, so 117 -- even with wildcards -- there is no meaningful information to put 118 there. (This has changed a bit for AJAX calls, which may indeed have 119 a specific media type preference.) It is unlikely that machine-to- 120 machine communication would have the same problem. 122 A similar function to the HTTP Accept: header could be added to CoAP 123 as an option in a much simpler way. The CoAP Accept option would 124 simple carry a value that is a sequence of octets, each of which is 125 an acceptable media type for the client, in the order of preference 126 (see Figure 1). If no Accept option is given, the client does not 127 express a preference. 129 0 130 0 1 2 3 4 5 6 7 131 +-+-+-+-+-+-+-+-+ 132 | mediatype | 133 +-+-+-+-+-+-+-+-+ 135 0 1 136 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 137 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 138 | mediatype1 | mediatype2 | etc. 139 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 141 Figure 1: Accept option value: A sequence of media types 143 Accept also has to be given an option type code, e.g. 7, in Table 2 144 of [I-D.ietf-core-coap]. 146 The other addition that would be required is an error code that 147 mirrors HTTP's "415 Unsupported Media Type". This is indeed already 148 listed as CoAP Code 35 in Table 3 of [I-D.ietf-core-coap]. 150 Proposal: Add an Accept Option. 152 Benefits: A Server does not need to specify one URI each for every 153 possible media type that it wants to serve a resource under. 155 Open Issues: For coap-00, this would have needed a way to handle 156 two-byte media types (easiest if these can be made self- 157 describing, at the cost of about 3 bits in the sub-type field; 158 Figure 2). 160 An self-describing representation for long mediatypes could look like 161 this: 163 0 164 0 1 2 3 4 5 6 7 165 +-+-+-+-+-+-+-+-+ 166 | top | sub | (1-byte: unchanged) 167 +-+-+-+-+-+-+-+-+ 169 0 1 170 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 171 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 172 | 000 | top | sub | (2-byte) 173 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 175 Figure 2: A self-describing media type representation 177 Instead, we assume for now that CoAP-01 will switch to a single-byte 178 media type encoding. 180 3. Representing Durations 182 Various message types used in CoAP need the representation of 183 *durations*, i.e. of the length of a timespan. In SI units, these 184 are measured in seconds. Where CPU power and memory is abundant, a 185 duration can almost always be adequately represented by a non- 186 negative floating-point number representing that number of seconds. 187 Historically, many APIs have also used an integer representation, 188 which limits both the resolution (e.g., if the integer represents the 189 duration in seconds) and often the range (integer machine types have 190 range limits that may become relevant). UNIX's "time_t" (which is 191 used for both absolute time and durations) originally was a signed 192 32-bit value of seconds, but was later complemented by an additional 193 integer to add microsecond ("struct timeval") and then later 194 nanosecond ("struct timespec") resolution. 196 Three decisions need to be made for each application of the concept 197 of duration: 199 o the *resolution*. What rounding error is acceptable? 201 o the *range*. What is the maximum duration that needs to be 202 represented? 204 o the *number of bits* that can be expended. 206 Obviously, these decisions are interrelated. Typically, a large 207 range needs a large number of bits, unless resolution is traded. For 208 most applications, the actual requirement for resolution are limited 209 for longer durations, but can be more acute for shorter durations. 211 3.1. Pseudo-Floating Point 213 Constrained systems typically avoid the use of floating-point (FP) 214 values, as 216 o simple CPUs often don't have support for floating-point datatypes 218 o software floating-point libraries are expensive in code size and 219 slow. 221 In addition, floating-point datatypes used to be a significant 222 element of market differentiation in CPU design; it has taken the 223 industry a long time to agree on a standard floating point 224 representation. 226 These issues have led to protocols that try to constrain themselves 227 to integer representation even where the ability of a floating point 228 representation to trade range for resolution would be beneficial. 230 The idea of introducing _pseudo-FP_ is to obtain the increased range 231 provided by embedding an exponent, without necessarily getting stuck 232 with hardware datatypes or inefficient software floating-point 233 libraries. 235 For the purposes of this draft, we define an (n,e)-pseudo-FP as a 236 fixed-length value of n bits, e of which may be used for an exponent. 237 Figure 3 illustrates an (8,4)-pseudo-FP value. 239 0 1 2 3 4 5 6 7 240 +---+---+---+---+---+---+---+---+ 241 | 0... value | 242 +---+---+---+---+---+---+---+---+ 244 +---+---+---+---+---+---+---+---+ 245 | 1... mantissa | exponent | 246 +---+---+---+---+---+---+---+---+ 248 Figure 3: An (8,4) pseudo-FP representation 250 If the high bit is clear, the entire n-bit value (including the high 251 bit) is the decoded value. If the high bit is set, the mantissa 252 (including the high bit, but with the exponent field cleared out) is 253 shifted left by the exponent to yield the decoded value. 255 The (n,e)-pseudo-FP format can be decoded with a single line of code 256 (plus a couple of constant definition), as demonstrated in Figure 4. 258 #define N 8 259 #define E 4 260 #define HIBIT (1 << (N - 1)) 261 #define EMASK ((1 << E) - 1) 262 #define MMASK ((1 << N) - 1 - EMASK) 264 #define DECODE_8_4(r) (r < HIBIT ? r : (r & MMASK) << (r & EMASK)) 266 Figure 4: Decoding an (8,4) pseudo-FP value 268 Only non-negative numbers can be represented by this format. It is 269 designed to provide full integer resolution for values from 0 to 270 2^(n-1)-1, i.e., 0 to 127 in the (8,4) case, and a mantissa of n-e 271 bits from 2^(n-1) to (2^n-2^e)*2^(2^e-1), i.e., 128 to 7864320 in the 272 (8,4) case. By choosing e carefully, resolution can be traded 273 against range. 275 Note that a pseudo-FP encoder needs to consider rounding; different 276 applications of durations may favor rounding up or rounding down the 277 value encoded in the message. This requires a little more than a 278 single line of code (which is left as an exercise to the reader, as 279 the most efficient expression depends on hardware details). 281 3.2. A Duration Type for CoAP 283 CoAP needs durations in a number of places. In [I-D.ietf-core-coap], 284 durations occur in the option "Subscription-lifetime" as well as in 285 the option "Max-age". (Note that the option "Date" is not a 286 duration, but a point in time.) Other durations of this kind may be 287 added later. 289 Most durations relevant to CoAP are best expressed with a minimum 290 resolution of one second. More detailed resolutions are unlikely to 291 provide much benefit. 293 The range of lifetimes and caching ages are probably best kept below 294 the order of magnitude of months. An (8,4)-pseudo-FP has the maximum 295 value of 7864320, which is about 91 days; this appears to be adequate 296 for a subscription lifetime and probably even for a maximum cache 297 age. (If a larger range for the latter is indeed desired, an (8,5)- 298 pseudo-FP could be used; this would last 15 milleniums, at the cost 299 of having only 3 bits of accuracy for values larger than 127 300 seconds.) 302 Proposal: A single duration type is used throughout CoAP, based on 303 an (8,4)-pseudo-FP giving a duration in seconds. 305 Benefits: Implementations can use a single piece of code for 306 managing all CoAP-related durations. 308 In addition, length information never needs to be managed for 309 durations that are embedded in other data structures: All 310 durations are expressed by a single byte. 312 Open Issues: It might be worthwhile to reserve one duration value, 313 e.g. 0xFF, for an indefinite duration. 315 4. URI encoding 317 In HTTP-based systems, URIs can reach significant lengths. While 318 CoAP-based systems may be able to sidestep the most egregious 319 excesses (mostly by simply applying REST principles), a URI such as 321 /.well-known/resources 323 can use up one third of the available payload in a CoAP message 324 transported in a single 6LoWPAN packet. Is there a way to encode 325 these URIs in a more efficient way? 327 Several proposals have been made on the CoRE mailing list, e.g. 328 applying the principle of base64-encoding [RFC4648] in reverse and 329 using only 6 bits per character. However, due to rounding errors and 330 occasional characters that are not in the 64-character subset chosen 331 to be efficiently encodable, the actual gains are limited. 332 Similarly, using 7 bits per character (assuming URIs contain only 333 ASCII characters) only gives a best-case gain of 12.5 %, and that 334 only in the case the URI is a multiple of 8 characters long. On the 335 other hand, the complexity (and danger of subtle interoperability 336 problems) is not entirely trivial. 338 Appendix A.1 defines a potential URI encoding that is slightly more 339 efficient than the abovementioned ones. However, even that was 340 rejected by the WG for its unconvincing cost-benefit ratio, which 341 then went on to discuss Henning Schulzrinne's proposal to add state. 343 4.1. Stateful URI compression 345 Is the approximately 25 % average saving achievable with Huffman- 346 based URI compression schemes worth the complexity? Probably not, 347 because much higher average savings can be achieved by introducing 348 state. 350 Henning Schulzrinne has proposed for a server to be able to supply a 351 shortened URI once a resource has been requested using the full- 352 length URI. Let's call such a shortened referent a _Temporary 353 Resource Identifier_, _TeRI_ for short. This could be expressed by a 354 response option as shown in Figure 5. 356 0 357 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 358 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 359 | duration | TeRI... 360 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 362 Figure 5: Option for offering a TeRI in a response 364 The TeRI offer option indicates that the server promises to offer 365 this resources under the TeRI given for at least the time given as 366 the duration. Another TeRI offer can be made later to extend the 367 duration. 369 Once a TeRI for a URI is known (and still within its lifetime), the 370 client can supply a TeRI instead of a URI in its requests. The same 371 option format as an offer could be used to allow the client to 372 indicate how long it believes the TeRI will still be valid (so that 373 the server can decide when to update the lifetime duration). TeRIs 374 in requests could be distinguished from URIs e.g. by using a 375 different option number. 377 Proposal: Add a TeRI option (e.g., number 2) that can be used in 378 CoAP requests and responses. 380 Add a way to indicate a TeRI and its duration in a link-value. 382 Do not add any form of stateless URI encoding. 384 Benefits: Much higher reduction of message size than any stateless 385 URI encoding could achieve. 387 As the use of TeRIs is entirely optional, minimal complexity nodes 388 can get by without implementing them. 390 5. Block-wise transfers 392 Not all resource representations will fit into a single link layer 393 packet of a constrained network. Using fragmentation (either at the 394 adaptation layer or at the IP layer) to enable the transport of 395 larger representations is possible up to the maximum size of a UDP 396 datagram, but the fragmentation/reassembly process loads the lower 397 layers with conversation state that is better managed in the 398 application layer. 400 This section proposes options to enable _block-wise_ access to 401 resource representations. The overriding objective is to avoid 402 creating conversation state at the server for block-wise GET 403 requests. (It is impossible to fully avoid creating conversation 404 state for POST/PUT, if the creation/replacement of resources is to be 405 atomic; where that property is not needed, there is no need to create 406 server conversation state in this case, either.) Also, 407 implementation of these options is intended to be optional. (The 408 details of which parts of the behavior need to be mandatory to enable 409 that optionality still are TBD, see below.) 411 The size of the blocks should not be fixed by the protocol. On the 412 other hand, implementation should be as simple as possible. We 413 therefore propose a small range of power-of-two block sizes, from 2^4 414 (16) to 2^11 (2048) bytes. One of these eight values can be encoded 415 in three bits (0 for 2^4 to 7 for 2^11 bytes), the "szx" (size 416 exponent); the actual block size is then "1 << (szx + 4)". 418 5.1. The Block Option 420 When a representation is larger than can be comfortably transferred 421 in a single UDP datagram, the Block option can be used to indicate a 422 block-wise transfer. Block is a 1-, 2- or 3-byte integer, the four 423 least significant bits of which indicate the size and whether the 424 current block-wise transfer is the last block being transferred (M or 425 "more" bit). The value divided by sixteen is the number of the block 426 currently being transferred, starting from zero, i.e., the current 427 transfer is about the "size" bytes starting at "blocknr << (szx + 428 4)". The default value of the Block option is zero, indicating that 429 the current block is the first (block number 0) and only (M bit not 430 set) block of the transfer; however, there is no explicit size 431 implied by this default value. 433 0 434 0 1 2 3 4 5 6 7 435 +-+-+-+-+-+-+-+-+ 436 |blocknr|M| szx | 437 +-+-+-+-+-+-+-+-+ 439 0 1 440 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 441 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 442 | block nr |M| szx | 443 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 445 0 1 2 446 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 447 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 448 | block nr |M| szx | 449 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 451 Figure 6: Block option 453 (Note that the option with the last 4 bits masked out, shifted to the 454 left by the value of szx, gives the byte position of the block. The 455 author is not too sure whether that particularly is a feature.) 457 The block option is used in one of three roles: 459 o In the request for a GET, it gives the block number requested and 460 suggests a block size (block number 0) or echoes the block size of 461 previous blocks received (block numbers other than 0). 463 o In the response for a GET or in the request for a PUT or POST, it 464 describes what block number is contained in the payload, and 465 whether further blocks are part of that body (M bit). If the M 466 bit is set, the size of the payload body in bytes MUST indeed be 467 the power of two given by the block size. All blocks for a 468 transaction MUST use the same block size, except for the last 469 block (M bit not set). 471 o In the response for a PUT or POST, it indicates what block number 472 is being acknowledged. In this case, the M bit is set to indicate 473 that this response does not carry the final response to the 474 request; this can occur when the M bit was set in the request and 475 the server implements PUT/POST atomically (only with the receptin 476 of the last block). 478 In all cases, the block number logically extends the transaction ID, 479 i.e. the same transaction ID can be used for all exchanges for a 480 block-wise transfer. (For GET, and for PUT/POST where atomic 481 semantics are not needed, the requester is free to use different 482 transactions IDs for each block if desired.) 484 When a GET is answered with a response carrying a Block option with 485 the M bit set, the requestor may retrieve additional blocks by 486 sending requests with a Block option giving the block number desired. 487 In such a Block option, the M bit MUST be sent as zero and ignored on 488 reception. 490 To influence the block size used in response to a GET request, the 491 requestor uses the Block option, giving the desired size, a block 492 number of zero and an M bit of zero. A server SHOULD use the block 493 size indicated or a smaller size. Any further block-wise requests 494 for blocks beyond the first one MUST indicate the block size used in 495 the response for the first one. 497 If the Block option is used by the requestor, all GET requests in a 498 single transaction MUST use the same size. The server SHOULD use the 499 block size indicated in the request option, but the requestor MUST 500 take note of the actual block size used in the response; the server 501 MUST ensure that it uses the same block size for all responses in a 502 transaction (except for the last one with the M bit not set). [TBD: 503 decide whether the Block option can only be used in a response if a 504 Block option was in the request. Such a minimal block option could 505 be of length zero, i.e., would occupy just one byte for the type/ 506 length information, but is a bit redundant, so it would be nice to 507 leave this requirement out, but then every GET requestor has the 508 burden of having to cope with receiving Block options.] 510 Block-wise transfers SHOULD be used in conjunction with the Etag 511 option, unless the representation being exchanged is entirely static 512 (not changing over time at all, such as in a schema describing a 513 device). When reassembling the representation from the blocks being 514 exchanged, the reassembler MUST compare Etag options. If the Etag 515 options do not match in a GET transfer, the requestor has the option 516 of attempting to retrieve fresh values for the blocks it retrieved 517 first. To minimize the resulting inefficiency, the server MAY cache 518 the current value of a representation for an ongoing transaction, but 519 there is no requirement for the server to establish any state. The 520 server may offer a TeRI with the initial block to reduce the size of 521 further block-wise GET requests; this TeRI MAY be short-lived and 522 specific to the version of the representation being retrieved (which 523 would in effect freeze the representation of the resource 524 specifically for the purposes of this block-wise transfer). 526 In a PUT or POST transfer, the block option refers to the body in the 527 request, i.e., there is no way to perform a block-wise retrieval of 528 the body of the response. Servers that do need to supply large 529 bodies in response to PUT/POST SHOULD therefore be employing 530 redirects, possibly offering a TeRI. 532 In a PUT or POST transfer that is intended to be implemented in an 533 atomic fashion at the server, the actual creation/replacement takes 534 place at the time a block with the M bit unset is received. If not 535 all previous blocks are available at the server at this time, the 536 transfer fails and error code 4__ (TBD) MUST be returned. The error 537 code 4__ can also be returned at any time by a server that does not 538 currently have the resources to store blocks for a block-wise PUT or 539 POST transfer that it would intend to implement in an atomic fashion. 540 [TBD: a way for a server to derive the equivalent of an Etag for the 541 request body, so that when these do not match in a PUT or POST 542 transfer, the reassembler MUST discard older blocks. For now, the 543 transaction ID will have to suffice.] 545 Proposal: Add a Block option (e.g., number 8) that can be used for 546 block-wise transfers. 548 Benefits: Transfers larger than can be accommodated in constrained- 549 network link-layer packets can be performed in smaller blocks. 551 No hard-to-manage conversation state is created at the adaptation 552 layer or IP layer for fragmentation. 554 The transfer of each block is acknowledged, enabling 555 retransmission if required. 557 Both sides have a say in the block size that actually will be 558 used. 560 TBD: Give examples with detailed message flows for a block-wise GET, 561 PUT and POST. 563 6. Option Encoding 565 The option encoding in [I-D.ietf-core-coap] is neither particularly 566 flexible nor particularly efficient. One important, easily 567 overlooked disadvantage of the encoding is the large number of ways 568 in which the same information can be encoded. This unneeded 569 variability causes problems in interoperability and increases both 570 coding and testing efforts required. 572 6.1. A More Efficient Option Encoding 574 The basic idea of the proposed encoding is to reduce the number of 575 ways the same information can be encoded as far as possible (but not 576 further). This both simplifies decoding (e.g., an implementation 577 that only ever uses short URIs never has to implement long options, 578 because these can only be used with long lengths) and 579 interoperability testing (there is only one way to say a specific 580 thing, so there aren't multiple ways that need testing). 582 One of the undesired variations in packets is the ordering of the 583 options. In this draft, we therefore mandate a total ordering of 584 options, ordered by the option number. 586 As an interesting consequence, the option numbers can now be 587 expressed in delta coding, in turn requiring fewer bits to encode the 588 option number. This frees a number of bits for the length, making 589 the likelihood of actually needing the two-byte form of the option 590 header much smaller. 592 To further reduce variation, the length of the value (as always, not 593 including the option header) is now encoded in such a way that there 594 is only one way to express a given length: The short form (one-byte 595 option tag) can express length values from 0 to 14, and the long form 596 is used for values of 15 to 15+255=270, inclusively (Figure 7). 598 0 1 2 3 4 5 6 7 599 +---+---+---+---+---+---+---+---+ 600 | option delta | length | for 0..14 601 +---+---+---+---+---+---+---+---+ 602 for 15..270: 603 +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 604 | option delta | 1 1 1 1 | length - 15 | 605 +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ 607 Figure 7: Option delta/length representation with small range 609 The small option delta of 0..15 in this encoding limits the 610 difference in option value between two adjacent options (or the value 611 of the option number of the first option). While realistic sequences 612 of options rarely will have a problem here, option numbers 14, 28, 613 ... are reserved for no-op options with no body (implementations will 614 automatically ignore these with zero additional code; see Section 6.2 615 why the reserved values are not 15, 30, ...). Note that the 616 resulting delta that reaches the interim nop option may have any 617 number, e.g., for including option 2 and 27 in one message, the 618 sequence would be: 620 o delta = 2 (option 2, body) 622 o delta = 12 (option 14 = no-op, no body) 624 o delta = 13 (option 27, body) 626 In the unlikely case that only option 40 is needed, the sequence 627 would be: 629 o delta = 14 (option 14 = no-op, no body) 631 o delta = 14 (option 28 = no-op, bo body) 633 o delta = 12 (option 40, body) 635 6.2. Critical Options 637 CoAP is designed to enable the definition of additional options by 638 later extensions. Typically, new options are designed in such a way 639 that they can simply be ignored if not understood, i.e. new options 640 are _elective_. However, some new options may be _critical_, i.e., 641 there is no good way to process the message if the option is not 642 understood. (Actually, half of the options currently on the table 643 are critical in nature.) 645 In the option encoding proposed, odd-numbered options indicate a 646 critical option; even-numbered options indicate elective options. 647 (Note that, again, the even/odd distinction is on the option number 648 resulting from the decoding, not the delta value actually embedded in 649 the packet.) 651 Implementing this proposal requires some renumbering of options from 652 [I-D.ietf-core-coap]. 654 6.3. Errors in Options 656 When a message contains a critical option that is not understood by 657 the receiver, we say that _decoding fails_. 659 When a message contains an option that is defined for a specific 660 length of value (e.g., Max-Age, which is only defined for length 1), 661 this is treated like an unknown option. For a critical option, this 662 causes the decoding to fail. For an elective option, this is not an 663 error, the option with the unsupported structure is just ignored. 664 (In both cases, the intention is to allow extension of the option by 665 different syntax in a later revision of the protocol.) 667 If the decoding of a message fails, the processing depends on the 668 message type: 670 o NCN messages and RST messages with decode failures are always 671 silently ignored. 673 o CON messages with decode failures lead to an RST with error code 674 400 (Bad Request). The payload of the RST SHOULD be a copy of the 675 option bytes that caused decoding to fail. However, nodes with 676 minimal capabilities may choose to restrict their error processing 677 to a minimum, 679 o ACK messages that fail to decode are hard to process. The 680 requesting node MAY repeat the request with fewer options in order 681 to receive a simpler answer; if that is not possible, the decoding 682 failure should be treated like a client error. Conversely, nodes 683 SHOULD not send critical options in ACK messages unless the CON 684 message eliciting the ACK contained options that justify this. 685 (There may be exceptions, e.g., a node is always allowed to send a 686 Block option with a large resource representation. A requestor 687 that does not understand Block may never be able to receive that 688 resource representation properly, so it is appropriate to treat 689 the situation as a client error.) 691 6.4. Payload-Length Option 693 Not all transport mappings may provide an unambiguous length of the 694 CoAP message. For UDP, it may also be desirable to pack more than 695 one CoAP message into one UDP payload (aggregation); in that case, 696 for all but the last message there needs to be a way to delimit the 697 payload of that message. 699 We propose a new option, the Payload-Length option. If this option 700 is present, the value of this option is an unsigned integer giving 701 the length of the payload of the message (note that this integer can 702 be zero for a zero-length payload, which can in turn be represented 703 by a zero-length option value). (In the UDP aggregation case, what 704 would have been in the payload of this message after "payload-length" 705 bytes is then actually one or more additional messages.) 707 6.5. Problems with specific options 709 Problem: The Uri option currently does not provide a way to 710 distinguish an "absolute-URI" from an "absolute-path" [RFC3986], 711 as the leading slash is omitted from the latter. (Ticket #12.) 713 Proposal: Split the option into two variants: "Uri-Full" and 714 "Uri-Path". None (= "Uri-Path" with option value ''), one of 715 these, but never both can be present. 717 Problem: The Etag option only allows for up to four bytes in one 718 Etag. If Etags are computed with a random distribution (e.g., by 719 hashing the resource representation), the birthday paradox makes a 720 collision surprisingly likely already for 1e4 variants in 721 circulation. 723 Proposal: Allow longer Etags (i.e., don't specify a specific upper 724 limit). The default Apache Etag has about 8..12 Bytes of 725 information in it (file ID = inode number, size, timestamp; which 726 interestingly is mostly redundant with information available in 727 Content-Length and Last-Modified). If a tighter upper limit is 728 desired, 8 Bytes should suffice for all practical purposes, but 729 makes two-way gatewaying with HTTP more complex. 731 7. IANA Considerations 733 This draft adds the following option numbers to Table 2 of 734 [I-D.ietf-core-coap]: 736 +------+-----+--------+----------------------------+--------+-------+ 737 | Type | C/E | Name | Data type | Length | Rules | 738 +------+-----+--------+----------------------------+--------+-------+ 739 | 2 | E | TeRI | Duration + Sequence of | 2-n B | | 740 | | | | Bytes | | | 741 | | | | | | | 742 | 7 | E | Accept | Sequence of bytes | 1-n B | | 743 | | | | | | | 744 | 8 | C | Block | Unsigned Integer | 1-3 B | | 745 +------+-----+--------+----------------------------+--------+-------+ 747 With the new option encoding and the proposal for essential options, 748 the total list becomes: 750 +-----+----+---------------+----------------+--------+--------------+ 751 | Typ | C/ | Name | Data type | Length | Default | 752 | e | E | | | | | 753 +-----+----+---------------+----------------+--------+--------------+ 754 | 0 | E | TeRI | Duration + | 2-n B | (none) | 755 | | | | Sequence of | | | 756 | | | | Bytes | | | 757 | | | | | | | 758 | 1 | C | Uri-Path | String | 1-n B | '' | 759 | | | | | | | 760 | 2 | E | Accept | Sequence of | 1-n B | any | 761 | | | | Bytes | | | 762 | | | | | | | 763 | 3 | C | Uri-Full | String | 1-n B | (use | 764 | | | | | | Uri-Path) | 765 | | | | | | | 766 | 4 | E | Max-age | Duration | 1 B | 0 | 767 | | | | | | | 768 | 5 | C | Content-type | Unsigned | 1* B | 0 (= | 769 | | | | Integer | | text/plain) | 770 | | | | | | | 771 | | | | | | | 772 | | | | | | | 773 | 6 | E | Etag | Sequence of | 1-4* B | (none) | 774 | | | | Bytes | | | 775 | | | | | | | 776 | 8 | E | Date | Unsigned | 4-6 B | (none) | 777 | | | | Integer | | | 778 | | | | (Appendix B.1) | | | 779 | 13 | C | Block | Unsigned | 1-3 B | 0 (see | 780 | | | | Integer | | Section 5.1) | 781 | | | | | | | 782 | 14. | E | Nop | None | 0 B | ('') | 783 | . | | | | | | 784 | | | | | | | 785 | 15 | C | Payload-lengt | Unsigned | 0-2 B | (none) | 786 | | | h | Integer | | | 787 +-----+----+---------------+----------------+--------+--------------+ 789 (The upper limit of "n" indicates that the size is limited only by 790 the options encoding. * indicates that this document proposes to 791 change the limit.) Odd option numbers indicate critical options, 792 even option numbers indicate elective options. Option numbers 14, 793 28, 42, ... (any number divisible by 14) are reserved (they are 794 elective and therefore ignored by all implementations). 796 (Subscription-related options are discussed in 797 [I-D.hartke-coap-observe], so the following option from 798 [I-D.ietf-core-coap] is not further discussed here:) 800 +-----+-----+-----------------------+----------+--------+-----------+ 801 | Typ | C/E | Name | Data | Length | Rules | 802 | e | | | type | | | 803 +-----+-----+-----------------------+----------+--------+-----------+ 804 | 6 | E | Subscription-lifetime | Duration | 1 B | With | 805 | | | | | | SUBSCRIBE | 806 | | | | | | or its | 807 | | | | | | response | 808 +-----+-----+-----------------------+----------+--------+-----------+ 810 8. Security Considerations 812 TBD. (Weigh the security implications of application layer block- 813 wise transfer against those of adaptation-layer or IP-layer 814 fragmentation. Discuss the implications of TeRIs. Also: Discuss 815 nodes without clocks.) 817 8.1. Amplification Attacks 819 TBD. (This section discusses how CoAP nodes could become implicated 820 in DoS attacks by using the amplifying properties of the protocol, as 821 well as mitigations for this threat.) 823 9. References 825 9.1. Normative References 827 [I-D.hartke-coap-observe] 828 Hartke, K. and C. Bormann, "Observing Resources in CoAP", 829 draft-hartke-coap-observe-00 (work in progress), 830 June 2010. 832 [I-D.ietf-core-coap] 833 Shelby, Z., Frank, B., and D. Sturek, "Constrained 834 Application Protocol (CoAP)", draft-ietf-core-coap-00 835 (work in progress), June 2010. 837 [I-D.ietf-httpbis-p1-messaging] 838 Fielding, R., Gettys, J., Mogul, J., Nielsen, H., 839 Masinter, L., Leach, P., Berners-Lee, T., and J. Reschke, 840 "HTTP/1.1, part 1: URIs, Connections, and Message 841 Parsing", draft-ietf-httpbis-p1-messaging-09 (work in 842 progress), March 2010. 844 [I-D.ietf-httpbis-p4-conditional] 845 Fielding, R., Gettys, J., Mogul, J., Nielsen, H., 846 Masinter, L., Leach, P., Berners-Lee, T., and J. Reschke, 847 "HTTP/1.1, part 4: Conditional Requests", 848 draft-ietf-httpbis-p4-conditional-09 (work in progress), 849 March 2010. 851 [I-D.ietf-httpbis-p6-cache] 852 Fielding, R., Gettys, J., Mogul, J., Nielsen, H., 853 Masinter, L., Leach, P., Berners-Lee, T., and J. Reschke, 854 "HTTP/1.1, part 6: Caching", 855 draft-ietf-httpbis-p6-cache-09 (work in progress), 856 March 2010. 858 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 859 Requirement Levels", BCP 14, RFC 2119, March 1997. 861 [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., 862 Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext 863 Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. 865 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 866 Resource Identifier (URI): Generic Syntax", STD 66, 867 RFC 3986, January 2005. 869 9.2. Informative References 871 [REST] Fielding, R., "Architectural Styles and the Design of 872 Network-based Software Architectures", 2000. 874 [RFC1951] Deutsch, P., "DEFLATE Compressed Data Format Specification 875 version 1.3", RFC 1951, May 1996. 877 [RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data 878 Encodings", RFC 4648, October 2006. 880 Appendix A. Things we won't do 882 This annex documents roads that the WG decided not to take, in order 883 to spare readers from reinventing them in vain. 885 A.1. An efficient stateless URI encoding 887 There is very little redundancy by repetition in a typical URI, 888 rendering popular compression methods such as LZ77 (as implemented in 889 in the widely used DEFLATE algorithm [RFC1951]) rather ineffective. 891 For the short, non-repetitive data structures that URIs tend to be, 892 efficient stateless compression is pretty much confined to Huffman 893 (or, for even more complexity, arithmetic) coding. The complexity 894 can be reduced significantly by moving to n-ary Huffman coding, i.e., 895 optimizing not to the bit level, but to a larger level of 896 granularity. Informal experiments by the author show that a 16ary 897 Huffman coding is close to optimal for reasonable URI lengths. In 898 other words, basing the encoding on nibbles (4-bit half-bytes) is 899 both nearly optimal and relatively inexpensive to implement. 901 The actual letter frequencies that will occur in CoAP URIs are hard 902 to predict. As a stopgap, the author has analyzed an HTTP-based URI 903 corpus and found the following characters to occur with high 904 frequency: 906 %.aeinorst 908 In the encoding proposed, each of these ten highly-compressed 909 characters is represented by a single 4-bit nibble. As the % 910 character is used for hexadecimal encoding in URIs, two additional 911 nibbles are used to provide the numeric value of the two hexadecimal 912 numbers following the % character (the original URI will only be 913 properly reconstituted if these are upper-case as they should be 914 according to section 2.1 of the URI specification [RFC3986]; the 915 encoder can choose to send all three characters in dual-nibble format 916 if that matters). An encoder could also map non-ASCII characters to 917 this three-nibble form, even though they are not allowed in URIs. 918 This gives compatibility with the %-encoding required by [RFC3986]. 920 All other characters are represented by both of their nibbles. The 921 resulting sequence of nibbles is reconstituted into a sequence of 922 bytes in most-significant-nibble-first order. Any unused nibble in 923 the last byte of an encoding is set to 0. (Upon decoding, this 924 padding can be readily distinguished from another % combination as 925 this would require another byte after the last byte.) The encoding 926 is summarized in Figure 8. 928 0 1 929 0 1 2 3 4 5 6 7 8 9 0 1 930 +---+---+---+---+ 931 | 1, 8-F | .aeinorst 932 +---+---+---+---+ 189ABCDEF 934 +---+---+---+---+---+---+---+---+ 935 | 2-7 | 0-F | other ASCII 936 +---+---+---+---+---+---+---+---+ 938 +---+---+---+---+---+---+---+---+---+---+---+---+ 939 | 0 | 0-F | 0-F | %HH 940 +---+---+---+---+---+---+---+---+---+---+---+---+ 942 Figure 8: A nibble-based URI encoding 944 An example encoding for "/.well-known/resources" (where the initial 945 slash is left out, as proposed for abs-path URIs) is given in 946 Figure 9. While the more than 28 % savings in this example may seem 947 just an accident, the HTTP-based corpus indeed shows an average 948 savings of about 21.8 %, i.e. the sum of the lengths of the encoded 949 version of all URIs in the corpus is about 78.2 % of the sum of the 950 length of all URIs. (The savings should be noticeably higher with a 951 more RESTful selection of URIs than was available for this 952 experiment.) 954 0 1 2 955 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 956 / . w e l l - k n o w n / r e s o u r c e s 958 2e 77 65 6c 6c 2d 6b 6e 6f 77 6e 2f 72 65 73 6f 75 72 63 65 73 959 -> 960 1 77 9 6c 6c 2d 6b b c 77 b 2f d 9 e c 75 d 63 9 e 961 = 17 79 6c 6c 2d 6b bc 77 b2 fd 9e c7 5d 63 9e 963 Figure 9: Nibble-based URI encoding: 21 -> 15 bytes 965 Appendix B. Experimental Options 967 This annex documents proposals that need significant additional 968 discussion before they can become part of (or go back to) the main 969 CoAP specification. They are not dead, but might die if there turns 970 out to be no good way to solve the problem. 972 B.1. Options indicating absolute time 974 HTTP has a number of headers that may indicate absolute time: 976 o "Date", defined in Section 14.18 in [RFC2616] (Section 9.3 in 977 [I-D.ietf-httpbis-p1-messaging]), giving the absolute time a 978 response was generated; 980 o "Last-Modified", defined in Section 14.29 in [RFC2616], (Section 981 6.6 in [I-D.ietf-httpbis-p4-conditional], giving the absolute time 982 of when the origin server believes the resource representation was 983 last modified; 985 o "If-Modified-Since", defined in Section 14.25 in [RFC2616], 986 "If-Unmodified-Since", defined in Section 14.28 in [RFC2616], and 987 "If-Range", defined in Section 14.27 in [RFC2616] can be used to 988 supply absolute time to gate a conditional request; 990 o "Expires", defined in Section 14.21 in [RFC2616] (Section 3.3 in 991 [I-D.ietf-httpbis-p6-cache]), giving the absolute time after which 992 a response is considered stale. 994 o The more obscure headers "Retry-After", defined in Section 14.37 995 in [RFC2616], and "Warning", defined in section 14.46 in 996 [RFC2616], also may employ absolute time. 998 [I-D.ietf-core-coap] defines a single "Date" option, which however 999 "indicates the creation time and date of a given resource 1000 representation", i.e., is closer to a "Last-Modified" HTTP header. 1001 HTTP's caching rules [I-D.ietf-httpbis-p6-cache] make use of both 1002 "Date" and "Last-Modified", combined with "Expires". The specific 1003 semantics required for CoAP needs further consideration. 1005 In addition to the definition of the semantics, an encoding for 1006 absolute times needs to be specified. 1008 In UNIX-related systems, it is customary to indicate absolute time as 1009 an integer number of seconds, after midnight UTC, January 1, 1970. 1010 Unless negative numbers are employed, this time format cannot 1011 represent time values prior to January 1, 1970, which probably is not 1012 required for the uses ob absolute time in CoAP. 1014 If a 32-bit integer is used and allowance is made for a sign-bit in a 1015 local implementation, the latest UTC time value that can be 1016 represented by the resulting 31 bit integer value is 03:14:07 on 1017 January 19, 2038. If the 32-bit integer is used as an unsigned 1018 value, the last date is 2106-02-07, 06:28:15. 1020 The reach can be extended by: - moving the epoch forward, e.g. by 40 1021 years (= 1262304000 seconds) to 2010-01-01. This makes it impossible 1022 to represent Last-Modified times in that past (such as could be 1023 gatewayed in from HTTP). - extending the number of bits, e.g. by one 1024 more byte, either always or as one of two formats, keeping the 32-bit 1025 variant as well. 1027 Also, the resolution can be extended by expressing time in 1028 milliseconds etc., requiring even more bits (e.g., a 48-bit unsigned 1029 integer of milliseconds would last well after year 9999.) 1031 For experiments, an experimental "Date" option is defined with the 1032 semantics of HTTP's "Last-Modified". It can carry an unsigned 1033 integer of 32, 40, or 48 bits; 32- and 40-bit integers indicate the 1034 absolute time in seconds since 1970-01-01 00:00 UTC, while 48-bit 1035 integers indicate the absolute time in milliseconds since 1970-01-01 1036 00:00 UTC. 1038 However, that option is not really that useful until there is a 1039 "If-Modified-Since" option as well. 1041 Authors' Addresses 1043 Carsten Bormann 1044 Universitaet Bremen TZI 1045 Postfach 330440 1046 Bremen D-28359 1047 Germany 1049 Phone: +49-421-218-63921 1050 Fax: +49-421-218-7000 1051 Email: cabo@tzi.org 1053 Klaus Hartke 1054 Universitaet Bremen TZI 1055 Postfach 330440 1056 Bremen D-28359 1057 Germany 1059 Phone: +49-421-218-63905 1060 Fax: +49-421-218-7000 1061 Email: hartke@tzi.org