idnits 2.17.1 draft-ietf-lwig-tcp-constrained-node-networks-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There are 32 instances of too long lines in the document, the longest one being 90 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet has text resembling RFC 2119 boilerplate text. -- The document date (June 4, 2019) is 1787 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'RFC 7228' is mentioned on line 875, but not defined == Unused Reference: 'RFC6092' is defined on line 1195, but no explicit reference was found in the text ** Obsolete normative reference: RFC 793 (Obsoleted by RFC 9293) ** Obsolete normative reference: RFC 2460 (Obsoleted by RFC 8200) == Outdated reference: A later version (-17) exists of draft-ietf-tcpm-rto-consider-08 -- Obsolete informational reference (is this intentional?): RFC 7230 (Obsoleted by RFC 9110, RFC 9112) -- Obsolete informational reference (is this intentional?): RFC 7540 (Obsoleted by RFC 9113) Summary: 4 errors (**), 0 flaws (~~), 5 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 LWIG Working Group C. Gomez 3 Internet-Draft UPC 4 Intended status: Informational J. Crowcroft 5 Expires: December 6, 2019 University of Cambridge 6 M. Scharf 7 Hochschule Esslingen 8 June 4, 2019 10 TCP Usage Guidance in the Internet of Things (IoT) 11 draft-ietf-lwig-tcp-constrained-node-networks-08 13 Abstract 15 This document provides guidance on how to implement and use the 16 Transmission Control Protocol (TCP) in Constrained-Node Networks 17 (CNNs), which are a characterstic of the Internet of Things (IoT). 18 Such environments require a lightweight TCP implementation and may 19 not make use of optional functionality. This document explains a 20 number of known and deployed techniques to simplify a TCP stack as 21 well as corresponding tradeoffs. The objective is to help embedded 22 developers with decisions on which TCP features to use. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at https://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on December 6, 2019. 41 Copyright Notice 43 Copyright (c) 2019 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (https://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 59 2. Conventions used in this document . . . . . . . . . . . . . . 4 60 3. Characteristics of CNNs relevant for TCP . . . . . . . . . . 4 61 3.1. Network and link properties . . . . . . . . . . . . . . . 4 62 3.2. Usage scenarios . . . . . . . . . . . . . . . . . . . . . 5 63 3.3. Communication and traffic patterns . . . . . . . . . . . 6 64 4. TCP implementation and configuration in CNNs . . . . . . . . 6 65 4.1. Addressing path properties . . . . . . . . . . . . . . . 7 66 4.1.1. Maximum Segment Size (MSS) . . . . . . . . . . . . . 7 67 4.1.2. Explicit Congestion Notification (ECN) . . . . . . . 8 68 4.1.3. Explicit loss notifications . . . . . . . . . . . . . 9 69 4.2. TCP guidance for single-segment stacks . . . . . . . . . 9 70 4.2.1. Single-segment stacks - benefits and issues . . . . . 9 71 4.2.2. TCP options for single-segment stacks . . . . . . . . 10 72 4.2.3. Delayed Acknowledgments for single-segment stacks . . 10 73 4.2.4. RTO calculation for single-segment stacks . . . . . . 11 74 4.3. General recommendations for TCP in CNNs . . . . . . . . . 11 75 4.3.1. Loss recovery and congestion/flow control . . . . . . 12 76 4.3.1.1. Selective Acknowledgments (SACK) . . . . . . . . 12 77 4.3.2. Delayed Acknowledgments . . . . . . . . . . . . . . . 13 78 4.3.3. Initial Window . . . . . . . . . . . . . . . . . . . 13 79 5. TCP usage recommendations in CNNs . . . . . . . . . . . . . . 14 80 5.1. TCP connection initiation . . . . . . . . . . . . . . . . 14 81 5.2. Number of concurrent connections . . . . . . . . . . . . 14 82 5.3. TCP connection lifetime . . . . . . . . . . . . . . . . . 15 83 6. Security Considerations . . . . . . . . . . . . . . . . . . . 17 84 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 17 85 8. Annex. TCP implementations for constrained devices . . . . . 18 86 8.1. uIP . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 87 8.2. lwIP . . . . . . . . . . . . . . . . . . . . . . . . . . 19 88 8.3. RIOT . . . . . . . . . . . . . . . . . . . . . . . . . . 19 89 8.4. TinyOS . . . . . . . . . . . . . . . . . . . . . . . . . 19 90 8.5. FreeRTOS . . . . . . . . . . . . . . . . . . . . . . . . 20 91 8.6. uC/OS . . . . . . . . . . . . . . . . . . . . . . . . . . 20 92 8.7. Summary . . . . . . . . . . . . . . . . . . . . . . . . . 20 93 9. Annex. Changes compared to previous versions . . . . . . . . 22 94 9.1. Changes between -00 and -01 . . . . . . . . . . . . . . . 22 95 9.2. Changes between -01 and -02 . . . . . . . . . . . . . . . 22 96 9.3. Changes between -02 and -03 . . . . . . . . . . . . . . . 22 97 9.4. Changes between -03 and -04 . . . . . . . . . . . . . . . 23 98 9.5. Changes between -04 and -05 . . . . . . . . . . . . . . . 23 99 9.6. Changes between -05 and -06 . . . . . . . . . . . . . . . 23 100 9.7. Changes between -06 and -07 . . . . . . . . . . . . . . . 23 101 9.8. Changes between -07 and -08 . . . . . . . . . . . . . . . 23 102 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 103 10.1. Normative References . . . . . . . . . . . . . . . . . . 23 104 10.2. Informative References . . . . . . . . . . . . . . . . . 25 105 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 29 107 1. Introduction 109 The Internet Protocol suite is being used for connecting Constrained- 110 Node Networks (CNNs) to the Internet, enabling the so-called Internet 111 of Things (IoT) [RFC7228]. In order to meet the requirements that 112 stem from CNNs, the IETF has produced a suite of new protocols 113 specifically designed for such environments (see e.g. [RFC8352]). 114 New IETF protocol stack components include the IPv6 over Low-power 115 Wireless Personal Area Networks (6LoWPAN) adaptation layer 116 [RFC4944][RFC6282][RFC6775], the IPv6 Routing Protocol for Low-power 117 and lossy networks (RPL) routing protocol [RFC6550], and the 118 Constrained Application Protocol (CoAP) [RFC7252]. 120 As of the writing, the main current transport layer protocols in IP- 121 based IoT scenarios are UDP and TCP. However, TCP has been 122 criticized (often, unfairly) as a protocol for the IoT. In fact, 123 some TCP features are not optimal for IoT scenarios, such as 124 relatively long header size, unsuitability for multicast, and always- 125 confirmed data delivery. However, many typical claims on TCP 126 unsuitability for IoT (e.g. a high complexity, connection-oriented 127 approach incompatibility with radio duty-cycling, and spurious 128 congestion control activation in wireless links) are not valid, can 129 be solved, or are also found in well accepted IoT end-to-end 130 reliability mechanisms (see [IntComp] for a detailed analysis). 132 At the application layer, CoAP was developed over UDP [RFC7252]. 133 However, the integration of some CoAP deployments with existing 134 infrastructure is being challenged by middleboxes such as firewalls, 135 which may limit and even block UDP-based communications. This is the 136 main reason why a CoAP over TCP specification has been developed 137 [RFC8323]. 139 Other application layer protocols not specifically designed for CNNs 140 are also being considered for the IoT space. Some examples include 141 HTTP/2 and even HTTP/1.1, both of which run over TCP by default 142 [RFC7230] [RFC7540], and the Extensible Messaging and Presence 143 Protocol (XMPP) [RFC6120]. TCP is also used by non-IETF application- 144 layer protocols in the IoT space such as the Message Queue Telemetry 145 Transport (MQTT) and its lightweight variants. 147 TCP is a sophisticated transport protocol that includes optional 148 functionality (e.g. TCP options) that may improve performance in 149 some environments. However, many optional TCP extensions require 150 complex logic inside the TCP stack and increase the codesize and the 151 memory requirements. Many TCP extensions are not required for 152 interoperability with other standard-compliant TCP endpoints. Given 153 the limited resources on constrained devices, careful selection of 154 optional TCP features can make an implementation more lightweight. 156 This document provides guidance on how to implement and configure 157 TCP, as well as on how TCP is advisable to be used by applications, 158 in CNNs. The overarching goal is to offer simple measures to allow 159 for lightweight TCP implementation and suitable operation in such 160 environments. A TCP implementation following the guidance in this 161 document is intended to be compatible with a TCP endpoint that is 162 compliant to the TCP standards, albeit possibly with a lower 163 performance. This implies that such a TCP client would always be 164 able to connect with a standard-compliant TCP server, and a 165 corresponding TCP server would always be able to connect with a 166 standard-compliant TCP client. 168 This document assumes that the reader is familiar with TCP. A 169 comprehensive survey of the TCP standards can be found in [RFC7414]. 170 Similar guidance regarding the use of TCP in special environments has 171 been published before, e.g., for cellular wireless networks 172 [RFC3481]. 174 2. Conventions used in this document 176 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL","SHALL NOT", 177 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 178 document are to be interpreted as described in [RFC2119]. 180 3. Characteristics of CNNs relevant for TCP 182 3.1. Network and link properties 184 CNNs are defined in [RFC7228] as networks whose characteristics are 185 influenced by being composed of a significant portion of constrained 186 nodes. The latter are characterized by significant limitations on 187 processing, memory, and energy resources, among others [RFC7228]. 188 The first two dimensions pose constraints on the complexity and on 189 the memory footprint of the protocols that constrained nodes can 190 support. The latter requires techniques to save energy, such as 191 radio duty-cycling in wireless devices [RFC8352], as well as 192 minimization of the number of messages transmitted/received (and 193 their size). 195 [RFC7228] lists typical network constraints in CNN, including low 196 achievable bitrate/throughput, high packet loss and high variability 197 of packet loss, highly asymmetric link characteristics, severe 198 penalties for using larger packets, limits on reachability over time, 199 etc. CNN may use wireless or wired technologies (e.g., Power Line 200 Communication), and the transmission rates are typically low (e.g. 201 below 1 Mbps). 203 For use of TCP, one challenge is that not all technologies in CNN may 204 be aligned with typical Internet subnetwork design principles 205 [RFC3819]. For instance, constrained nodes often use physical/link 206 layer technologies that have been characterized as 'lossy', i.e., 207 exhibit a relatively high bit error rate. Dealing with corruption 208 loss is one of the open issues in the Internet [RFC6077]. 210 3.2. Usage scenarios 212 There are different deployment and usage scenarios for CNNs. Some 213 CNNs follow the star topology, whereby one or several hosts are 214 linked to a central device that acts as a router connecting the CNN 215 to the Internet. CNNs may also follow the multihop topology 216 [RFC6606]. 218 In constrained environments, there can be different types of devices 219 [RFC7228]. For example, there can be devices with single combined 220 send/receive buffer, devices with a separate send and receive buffer, 221 or devices with a pool of multiple send/receive buffers. In the 222 latter case, it is possible that buffers also be shared for other 223 protocols. 225 One key use case for the use of TCP in CNNs is a model where 226 constrained devices connect to unconstrained servers in the Internet. 227 But it is also possible that both TCP endpoints run on constrained 228 devices. In the first case, communication possibly has to traverse a 229 middlebox (e.g. a firewall, NAT, etc.). Figure 1 illustrates such 230 scenario. Note that the scenario is asymmetric, as the unconstrained 231 device will typically not suffer the severe constraints of the 232 constrained device. The unconstrained device is expected to be 233 mains-powered, to have high amount of memory and processing power, 234 and to be connected to a resource-rich network. 236 Assuming that a majority of constrained devices will correspond to 237 sensor nodes, the amount of data traffic sent by constrained devices 238 (e.g. sensor node measurements) is expected to be higher than the 239 amount of data traffic in the opposite direction. Nevertheless, 240 constrained devices may receive requests (to which they may respond), 241 commands (for configuration purposes and for constrained devices 242 including actuators) and relatively infrequent firmware/software 243 updates. 245 +---------------+ 246 o o <-------- TCP communication -----> | | 247 o o | | 248 o o | Unconstrained | 249 o o +-----------+ | device | 250 o o o ------ | Middlebox | ------- | | 251 o o +-----------+ | (e.g. cloud) | 252 o o o | | 253 +---------------+ 254 constrained devices 256 Figure 1: TCP communication between a constrained device and an 257 unconstrained device, traversing a middlebox. 259 3.3. Communication and traffic patterns 261 IoT applications are characterized by a number of different 262 communication patterns. The following non-comprehensive list 263 explains some typical examples: 265 o Unidirectional transfers: An IoT device (e.g. a sensor) can send 266 (repeatedly) updates to the other endpoint. Not in every case 267 there is a need for an application response back to the IoT 268 device. 270 o Request-response patterns: An IoT device receiving a request from 271 the other endpoint, which triggers a response from the IoT device. 273 o Bulk data transfers: A typical example for a long file transfer 274 would be an IoT device firmware update. 276 A typical communication pattern is that a constrained device 277 communicates with an unconstrained device (cf. Figure 1). But it is 278 also possible that constrained devices communicate amongst 279 themselves. 281 4. TCP implementation and configuration in CNNs 283 This section explains how a TCP stack can deal with typical 284 constraints in CNN. The guidance in this section relates to the TCP 285 implementation and its configuration. 287 4.1. Addressing path properties 289 4.1.1. Maximum Segment Size (MSS) 291 Assuming that IPv6 is used, and for the sake of lightweight 292 implementation and operation, unless applications require handling 293 large data units (i.e. leading to an IPv6 datagram size greater than 294 1280 bytes), it may be desirable to limit the MTU to 1280 bytes in 295 order to avoid the need to support Path MTU Discovery [RFC8201]. In 296 addition, an MTU of 1280 bytes avoids incurring IPv6-layer 297 fragmentation. 299 An IPv6 datagram size exceeding 1280 bytes can be avoided by setting 300 the TCP MSS not larger than 1220 bytes. This assumes that the remote 301 sender will use no TCP options, aside from possibly the MSS option, 302 which is only used in the initial TCP SYN packet. 304 In order to accommodate unrequested TCP options that may be used by 305 some TCP implementations, a constrained device may advertise an MSS 306 smaller than 1220 bytes (e.g. not larger than 1200 bytes). Note 307 that, in many implementations, TCP options generally consume payload 308 space instead of increasing datagram size, therefore this suggestion 309 might be overcautious and its suitability will depend on each 310 specific scenario. 312 Note that setting the MTU to 1280 bytes is possible for link layer 313 technologies in the CNN space, even if some of them are characterized 314 by a short data unit payload size, e.g. up to a few tens or hundreds 315 of bytes. For example, the maximum frame size in IEEE 802.15.4 is 316 127 bytes. 6LoWPAN defined an adaptation layer to support IPv6 over 317 IEEE 802.15.4 networks. The adaptation layer includes a 318 fragmentation mechanism, since IPv6 requires the layer below to 319 support an MTU of 1280 bytes [RFC2460], while IEEE 802.15.4 lacked 320 fragmentation mechanisms. 6LoWPAN defines an IEEE 802.15.4 link MTU 321 of 1280 bytes [RFC4944]. Other technologies, such as Bluetooth LE 322 [RFC7668], ITU-T G.9959 [RFC7428] or DECT-ULE [RFC8105], also use 323 6LoWPAN-based adaptation layers in order to enable IPv6 support. 324 These technologies do support link layer fragmentation. By 325 exploiting this functionality, the adaptation layers that enable IPv6 326 over such technologies also define an MTU of 1280 bytes. 328 On the other hand, there exist technologies also used in the CNN 329 space, such as Master Slave / Token Passing (TP) [RFC8163], 330 Narrowband IoT (NB-IoT) [RFC8376] or IEEE 802.11ah 331 [I-D.delcarpio-6lo-wlanah], that do not suffer the same degree of 332 frame size limitations as the technologies mentioned above. The MTU 333 for MS/TP is recommended to be 1500 bytes [RFC8163], the MTU in NB- 334 IoT is 1600 bytes, and the maximum frame payload size for IEEE 335 802.11ah is 7991 bytes. 337 While many IP-based IoT environments use IPv6, IPv4 can also be in 338 use. In IPv4, the minimum MTU is 576 bytes. In order to avoid 339 exceeding the IPv4 MTU, the MSS needs to be set to a value not larger 340 than the IPv4 MTU minus 40 bytes. Similarly to the recommendations 341 given above for IPv6, a constrained device using IPv4 may advertise 342 an even smaller MSS in order to accommodate unrequested TCP options. 344 Finally, note that using larger MSS (to a suitable extent) may be 345 beneficial, especially when transferring large payloads, as it 346 reduces the number of packets (and packet headers) required for a 347 given payload. 349 4.1.2. Explicit Congestion Notification (ECN) 351 Explicit Congestion Notification (ECN) [RFC3168] ECN allows a router 352 to signal in the IP header of a packet that congestion is arising, 353 for example when a queue size reaches a certain threshold. An ECN- 354 enabled TCP receiver will echo back the congestion signal to the TCP 355 sender by setting a flag in its next TCP ACK. The sender triggers 356 congestion control measures as if a packet loss had happened. 358 The document [RFC8087] outlines the principal gains in terms of 359 increased throughput, reduced delay, and other benefits when ECN is 360 used over a network path that includes equipment that supports 361 Congestion Experienced (CE) marking. In the context of CNNs, a 362 remarkable feature of ECN is that congestion can be signalled without 363 incurring packet drops (which will lead to retransmissions and 364 consumption of limited resources such as energy and bandwitdh). 366 ECN can further reduce packet losses since congestion control 367 measures can be applied earlier [RFC2884]. Less lost packets implies 368 that the number of retransmitted segments decreases, which is 369 particularly beneficial in CNNs, where energy and bandwidth resources 370 are typically limited. Also, it makes sense to try to avoid packet 371 drops for transactional workloads with small data sizes, which are 372 typical for CNNs. In such traffic patterns, it is more difficult and 373 often impossible to detect packet loss without retransmission 374 timeouts (e.g., as there may be no three duplicate ACKs). Any 375 retransmission timeout slows down the data transfer significantly. 376 In addition, if the constrained device uses power saving techniques, 377 a retransmission timeout will incur a wake-up action, in contrast to 378 ACK clock- triggered sending. When the congestion window of a TCP 379 sender has a size of one segment and a TCP ACK with an ECN signal 380 (ECE flag) arrives at the TCP sender, the TCP sender resets the 381 retransmit timer, and the sender will only be able to send a new 382 packet when the retransmit timer expires. Effectively, the TCP 383 sender reduces at that moment its sending rate from 1 segment per 384 Round Trip Time (RTT) to 1 segment per RTO and reduces the sending 385 rate further on each ECN signal received in subsequent TCP ACKs. 386 Otherwise, if an ECN signal is not present in a subsequent TCP ACK 387 the TCP sender resumes the normal ACK-clocked transmission of 388 segments [RFC3168]. 390 ECN can be incrementally deployed in the Internet. Guidance on 391 configuration and usage of ECN is provided in [RFC7567]. Given the 392 benefits, more and more TCP stacks in the Internet support ECN, and 393 it specifically makes sense to leverage ECN in controlled 394 environments such as CNNs. Note, however, that supporting ECN 395 increases implementation complexity. 397 4.1.3. Explicit loss notifications 399 There has been a significant body of research on solutions capable of 400 explicitly indicating whether a TCP segment loss is due to 401 corruption, in order to avoid activation of congestion control 402 mechanisms [ETEN] [RFC2757]. While such solutions may provide 403 significant improvement, they have not been widely deployed and 404 remain as experimental work. In fact, as of today, the IETF has not 405 standardized any such solution. 407 4.2. TCP guidance for single-segment stacks 409 This section discusses TCP stacks that allow transferring only a 410 single segment. More general guidance is provided in Section 4.3. 412 4.2.1. Single-segment stacks - benefits and issues 414 A TCP stack can reduce the memory requirements by advertising a TCP 415 window size of one MSS, and also transmit at most one MSS of 416 unacknowledged data. In that case, both congestion and flow control 417 implementation are quite simple. Such a small receive and send 418 window may be sufficient for simple message exchanges in the CNN 419 space. However, only using a window of one MSS can significantly 420 affect performance. A stop-and-wait operation results in low 421 throughput for transfers that exceed the length of one MSS, e.g., a 422 firmware download. Furthermore, a single-segment solution relies 423 solely on timer-based loss recovery, therefore missing the 424 performance gain of Fast Retransmit and Fast Recovery (which require 425 a larger window size, see Subsection 4.3.1). 427 If CoAP is used over TCP with the default setting for NSTART in 428 [RFC7252], a CoAP endpoint is not allowed to send a new message to a 429 destination until a response for the previous message sent to that 430 destination has been received. This is equivalent to an application- 431 layer window size of 1 data unit. For this use of CoAP, a maximum 432 TCP window of one MSS may be sufficient, as long as the CoAP message 433 size does not exceed one MSS. 435 4.2.2. TCP options for single-segment stacks 437 A TCP implementation needs to support, at a minimum, TCP options 2, 1 438 and 0. These are, respectively, the Maximum Segment Size (MSS) 439 option, the No-Operation option, and the End Of Option List marker 440 [RFC0793]. None of these are a substantial burden to support. These 441 options are sufficient for interoperability with a standard-compliant 442 TCP endpoint, albeit many TCP stacks support additional options and 443 can negotiate their use. A TCP implementation is permitted to 444 silently ignore all other TCP options. 446 A TCP implementation for a constrained device that uses a single- 447 segment TCP receive or transmit window size may not benefit from 448 supporting the following TCP options: Window scale [RFC7323], TCP 449 Timestamps [RFC7323], Selective Acknowledgments (SACK) and SACK- 450 Permitted [RFC2018]. Also other TCP options may not be required on a 451 constrained device with a very lightweight implementation. With 452 regard to the Window scale option, note that it is only useful if a 453 window size greater than 64 kB is needed. 455 Note that a TCP sender can benefit from the TCP Timestamps option 456 [RFC7323] in detecting spurious RTOs. The latter are quite likely to 457 occur in CNN scenarios due to a number of reasons (e.g. route changes 458 in a multihop scenario, link layer retries, etc.). The header 459 overhead incurred by the Timestamps option (of up to 12 bytes) needs 460 to be taken into account. 462 One potentially relevant TCP option in the context of CNNs is TCP 463 Fast Open (TFO) [RFC7413]. As described in Section 5.3, TFO can be 464 used to address the problem of traversing middleboxes that perform 465 early filter state record deletion. 467 4.2.3. Delayed Acknowledgments for single-segment stacks 469 TCP Delayed Acknowledgments are meant to reduce the number of ACKs 470 sent within a TCP connection, thus reducing network overhead, but 471 they may increase the time until a sender may receive an ACK. In 472 general, usefulness of Delayed ACKs depends heavily on the usage 473 scenario (see subsection 4.3.2). There can be interactions with 474 single-segment stacks. 476 When traffic is unidirectional, if the sender can send at most one 477 MSS of data or the receiver advertises a receive window not greater 478 than the MSS, Delayed ACKs may unnecessarily contribute delay (up to 479 500 ms) to the RTT [RFC5681], which limits the throughput and can 480 increase data delivery time. Note that, in some cases, it may not be 481 possible to disable Delayed ACKs. One known workaround is to split 482 the data to be sent into two segments of smaller size. A standard 483 compliant TCP receiver will acknowledge the second MSS of data, which 484 can improve throughput. However, this 'split hack' may not always 485 work since a TCP receiver is required to acknowledge every second 486 full-sized segment, but not two consecutive small segments. 487 Furthermore, the overhead of sending two IP packets instead of one is 488 another downside of the 'split hack'. 490 Similar issues happen when the sender uses the Nagle algorithm. 491 Disabling the algorithm will not have impact if the sender can only 492 handle stop-and-wait operation. 494 For request-response traffic, when the receiver uses Delayed ACKs, a 495 response to a data message can piggyback an ACK, as long as the 496 latter is sent before the Delayed ACK timer expires, thus avoiding 497 unnecessary pure ACKs. Disabling Delayed ACKs at the sender allows 498 an immediate ACK for the data segment carrying the response. 500 4.2.4. RTO calculation for single-segment stacks 502 The Retransmission Timeout (RTO) calculation is one of the 503 fundamental TCP algorithms [RFC6298]. There is a fundamental trade- 504 off: A short, aggressive RTO behavior reduces wait time before 505 retransmissions, but it also increases the probability of spurious 506 timeouts. The latter lead to unnecessary waste of potentially scarce 507 resources in CNNs such as energy and bandwidth. In contrast, a 508 conservative timeout can result in long error recovery times and thus 509 needlessly delay data delivery. 511 If a TCP sender uses a very small window size, and it cannot benefit 512 from Fast Retransmit/Fast Recovery or SACK, the RTO algorithm has a 513 large impact on performance. In that case, RTO algorithm tuning may 514 be considered, although careful assessment of possible drawbacks is 515 recommended [I-D.ietf-tcpm-rto-consider]. 517 As an example, an adaptive RTO algorithm for CoAP over UDP has been 518 defined that has been found to perform well in CNN scenarios 519 [Commag]. 521 4.3. General recommendations for TCP in CNNs 523 This section summarizes some widely used techniques to improve TCP, 524 with a focus on their use in CNNs. The TCP extensions discussed here 525 are useful in a wide range of network scenarios, including CNNs. 527 This section is not comprehensive. A comprehensive survey of TCP 528 extensions is published in [RFC7414]. 530 4.3.1. Loss recovery and congestion/flow control 532 Devices that have enough memory to allow a larger (i.e. more than 3 533 MSS of data) TCP window size can leverage a more efficient loss 534 recovery than the timer-based approach used for smaller TCP window 535 size (see Subsection 3.2.1) by using Fast Retransmit and Fast 536 Recovery [RFC5681], at the expense of slightly greater complexity and 537 TCB size. Assuming that Delayed ACKs are used by the receiver, a 538 window size of up to 5 MSS is required for Fast Retransmit and Fast 539 Recovery to work efficiently: If in a given TCP transmission of 540 segments 1, 2, 3, 4, 5, and 6 segment 2 gets lost, and the ACK for 541 segment 1 is held by the Delayed ACK timer, then the sender should 542 get an ACK for segment 1 when 3 arrives and duplicate ACKs when 543 segments 4, 5, and 6 arrive. It will retransmit segment 2 when the 544 third duplicate ACK arrives. In order to have segments 2, 3, 4, 5, 545 and 6 sent, the window has to be of at least 5 MSS. With an MSS of 546 1220 bytes, a buffer of a size of 5 MSS would require 6100 bytes. 548 Further TCP improvements such as Limited Transmit [RFC3042] may also 549 be useful for any transfer that has more than one segment in flight. 550 Small transfers tend to benefit more from Limited Transmit, because 551 they are more likely to not receive enough duplicate ACKs. Assuming 552 the example in the previous paragraph, Limited Transmit allows 553 sending 5 MSS with a congestion window (cwnd) of 3 segments, plus two 554 additional segments for each one of the first two duplicate ACKs. 556 When a multiple-segment window is used, the receiver will need to 557 manage the reception of possible out-of-order received segments, 558 requiring sufficient buffer space. 560 4.3.1.1. Selective Acknowledgments (SACK) 562 If a device with less severe memory and processing constraints can 563 afford advertising a TCP window size of several MSS, it makes sense 564 to support the SACK option to improve performance. SACK allows a 565 data receiver to inform the data sender of non-contiguous data blocks 566 received, thus a sender (having previously sent the SACK-Permitted 567 option) can avoid performing unnecessary retransmissions, saving 568 energy and bandwidth, as well as reducing latency. In addition, SACK 569 often allows for faster loss recovery when there is more than one 570 lost segment in a window of data, since with SACK recovery requires 571 less RTTs. SACK is particularly useful for bulk data transfers. A 572 receiver supporting SACK will need to keep track of the SACK blocks 573 that need to be received. The sender will also need to keep track of 574 which data segments need to be resent after learning which data 575 blocks are missing at the receiver. SACK adds 8*n+2 bytes to the TCP 576 header, where n denotes the number of data blocks received, up to 4 577 blocks. For a low number of out-of-order segments, the header 578 overhead penalty of SACK is compensated by avoiding unnecessary 579 retransmissions. When the sender discovers the data blocks that have 580 already been received, it needs to also store the necessary state to 581 avoid unnecessary retransmission of data segments that have already 582 been received. 584 4.3.2. Delayed Acknowledgments 586 For certain traffic patterns, Delayed ACKs may have a detrimental 587 effect, as already noted in Section 4.2.3. Advanced TCP stacks may 588 use heuristics to determine the maximum delay for an ACK. For CNNs, 589 the recommendation depends on the expected communication patterns. 591 When traffic over a CNN is expected to mostly be unidirectional 592 messages with a size typically up to one MSS, and the time between 593 two consecutive message transmissions is greater than the Delayed ACK 594 timeout, it may make sense to use a small timeout or disable Delayed 595 ACKs at the receiver. This avoids incurring additional delay, as 596 well as the energy consumption of the sender (which might e.g. keep 597 its radio interface in receive mode) during that time. Note that 598 disabling Delayed ACKs may only be possible if the peer device is 599 administered by the same entity managing the constrained device. For 600 request-response traffic, enabling Delayed ACKs is recommended, in 601 order to allow combining a response with the ACK into a single 602 segment, thus increasing efficiency. In this case, disabling Delayed 603 ACKs at the sender allows an immediate ACK for the data segment 604 carrying the response. 606 In contrast, Delayed ACKs allow to reduce the number of ACKs in bulk 607 transfer type of traffic, e.g. for firmware/software updates or for 608 transferring larger data units containing a batch of sensor readings. 610 Note that, in many scenarios, the peer that a constrained device 611 communicates with will be a general purpose system that communicates 612 with both constrained and unconstrained devices. Since delayed ACKs 613 are often configured through system-wide parameters, delayed ACKs 614 behavior at the peer will be the same regardless of the nature of the 615 endpoints it talks to. Such a peer will typically have delayed ACKs 616 enabled. 618 4.3.3. Initial Window 620 RFC 5681 specifies a TCP Initial Window (IW) of roughly 4 kB 621 [RFC5681]. Subsequently, RFC 6928 defined an experimental new value 622 for the IW, which in practice will result in an IW of 10 MSS 623 [RFC6928]. The latter is nowadays used in many TCP implementations. 625 Note that a 10-MSS IW was recommended for resource-rich environments 626 (e.g. broadband environments), which are significantly different from 627 CNNs. In CNNs, many application layer data units are relatively 628 small (e.g. below one MSS). However, larger objects (e.g. large 629 files containing sensor readings, firmware updates, etc.) may also 630 need to be transferred in CNNs. If such a large object is 631 transferred in CNNs, with an IW setting of 10 MSS, there is 632 significant buffer overflow risk. In order to avoid such problem, in 633 CNNs the IW needs to be carefully set, based on device and network 634 resource constraints. In many cases, a safe IW setting will be 635 smaller than 10 MSS. 637 5. TCP usage recommendations in CNNs 639 This section discusses how TCP can be used by applications that are 640 developed for CNN scenarios. These remarks are by and large 641 independent of how TCP is exactly implemented. 643 5.1. TCP connection initiation 645 In the constrained device to unconstrained device scenario 646 illustrated above, a TCP connection is typically initiated by the 647 constrained device, in order for this device to support possible 648 sleep periods to save energy. 650 5.2. Number of concurrent connections 652 TCP endpoints with a small amount of memory may only support a small 653 number of connections. Each TCP connection requires storing a number 654 of variables in the Transmission Control Block (TCB). Depending on 655 the internal TCP implementation, each connection may result in 656 further memory overhead, and connections may compete for scarce 657 resources (e.g. further memory overhead for send and receive buffers, 658 etc). 660 A careful application design may try to keep the number of concurrent 661 connections as small as possible. A client can for instance limit 662 the number of simultaneous open connections that it maintains to a 663 given server. Multiple connections could for instance be used to 664 avoid the "head-of-line blocking" problem in an application transfer. 665 However, in addition to consuming resources, using multiple 666 connections can also cause undesirable side effects in congested 667 networks. For example, the HTTP/1.1 specification encourages clients 668 to be conservative when opening multiple connections [RFC7230]. 670 Furthermore, each new connection will start with a 3-way handshake, 671 therefore increasing message overhead. 673 Being conservative when opening multiple TCP connections is of 674 particular importance in Constrained-Node Networks. 676 5.3. TCP connection lifetime 678 In order to minimize message overhead, it makes sense to keep a TCP 679 connection open as long as the two TCP endpoints have more data to 680 send. If applications exchange data rather infrequently, i.e., if 681 TCP connections would stay idle for a long time, the idle time can 682 result in problems. For instance, certain middleboxes such as 683 firewalls or NAT devices are known to delete state records after an 684 inactivity interval. RFC 5382 specifies a minimum value for such 685 interval of 124 minutes. Measurement studies have reported that TCP 686 NAT binding timeouts are highly variable across devices, with a 687 median around 60 minutes, the shortest timeout being around 2 688 minutes, and more than 50% of the devices with a timeout shorter than 689 the aforementioned minimum timeout of 124 minutes [HomeGateway]. The 690 timeout duration used by a middlebox implementation may not be known 691 to the TCP endpoints. 693 In CNNs, such middleboxes may e.g. be present at the boundary between 694 the CNN and other networks. If the middlebox can be optimized for 695 CNN use cases, it makes sense to increase the initial value for 696 filter state inactivity timers to avoid problems with idle 697 connections. Apart from that, this problem can be dealt with by 698 different connection handling strategies, each having pros and cons. 700 One approach for infrequent data transfer is to use short-lived TCP 701 connections. Instead of trying to maintain a TCP connection for long 702 time, possibly short-lived connections can be opened between two 703 endpoints, which are closed if no more data needs to be exchanged. 704 For use cases that can cope with the additional messages and the 705 latency resulting from starting new connections, it is recommended to 706 use a sequence of short-lived connections, instead of maintaining a 707 single long-lived connection. 709 The message and latency overhead that stems from using a sequence of 710 short-lived connections could be reduced by TCP Fast Open (TFO) 711 [RFC7413], which is an experimental TCP extension, at the expense of 712 increased implementation complexity and increased TCP Control Block 713 (TCB) size. TFO allows data to be carried in SYN (and SYN-ACK) 714 segments, and to be consumed immediately by the receiving endpoint. 715 This reduces the message and latency overhead compared to the 716 traditional three-way handshake to establish a TCP connection. For 717 security reasons, the connection initiator has to request a TFO 718 cookie from the other endpoint. The cookie, with a size of 4 or 16 719 bytes, is then included in SYN packets of subsequent connections. 720 The cookie needs to be refreshed (and obtained by the client) after a 721 certain amount of time. Nevertheless, TFO is more efficient than 722 frequently opening new TCP connections with the traditional three-way 723 handshake, as long as the cookie can be reused in subsequent 724 connections. However, as stated in RFC 7413, TFO deviates from the 725 standard TCP semantics, since the data in the SYN could be replayed 726 to an application in some rare circumstances. Applications should 727 not use TFO unless they can tolerate this issue, e.g., by using 728 Transport Layer Security (TLS) [RFC7413]. A comprehensive discussion 729 on TFO can be found at RFC 7413. 731 Another approach is to use long-lived TCP connections with 732 application-layer heartbeat messages. Various application protocols 733 support such heartbeat messages (e.g. CoAP over TCP [RFC8323]). 734 Periodic application-layer heartbeats can prevent early filter state 735 record deletion in middleboxes. If the TCP binding timeout for a 736 middlebox to be traversed by a given connection is known, middlebox 737 filter state deletion will be avoided if the heartbeat period is 738 lower than the middlebox TCP binding timeout. Otherwise, the 739 implementer needs to take into account that middlebox TCP binding 740 timeouts fall in a wide range of possible values [HomeGateway], and 741 it may be hard to find a proper heartbeat period for application- 742 layer heartbeat messages. 744 One specific advantage of Heartbeat messages is that they also allow 745 aliveness checks at the application level. In general, it makes 746 sense to realize aliveness checks at the highest protocol layer 747 possible that is meaningful to the application, in order to maximize 748 the depth of the aliveness check. In addition, timely detection of a 749 dead peer may allow savings in terms of TCB memory use. However, the 750 transmission of heartbeat messages consumes resources. This aspect 751 needs to be assessed carefully, considering the characteristics of 752 each specific CNN. 754 A TCP implementation may also be able to send "keep-alive" segments 755 to test a TCP connection. According to [RFC1122], "keep-alives" are 756 an optional TCP mechanism that is turned off by default, i.e., an 757 application must explicitly enable it for a TCP connection. The 758 interval between "keep-alive" messages must be configurable and it 759 must default to no less than two hours. With this large timeout, TCP 760 keep-alive messages might not always be useful to avoid deletion of 761 filter state records in some middleboxes. However, sending TCP keep- 762 alive probes more frequently risks draining power on energy- 763 constrained devices. 765 6. Security Considerations 767 Best current practise for securing TCP and TCP-based communication 768 also applies to CNN. As example, use of Transport Layer Security 769 (TLS) is strongly recommended if it is applicable. 771 There are also TCP options which can improve TCP security. One 772 example is the TCP Authentication Option (TCP-AO) [RFC5925]. 773 However, this option adds overhead and complexity. TCP-AO typically 774 has a size of 16-20 bytes. 776 For the mechanisms discussed in this document, the corresponding 777 considerations apply. For instance, if TFO is used, the security 778 considerations of [RFC7413] apply. 780 Constrained devices are expected to support smaller TCP window sizes 781 than less limited devices. In such conditions, segment 782 retransmission triggered by RTO expiration is expected to be 783 relatively frequent, due to lack of (enough) duplicate ACKs, 784 especially when a constrained device uses a single-segment 785 implementation. For this reason, constrained devices running TCP may 786 appear as particularly appealing victims of the so-called "shrew" 787 Denial of Service (DoS) attack [shrew], whereby one or more sources 788 generate a packet spike targetted to coincide with consecutive RTO- 789 expiration-triggered retry attempts of a victim node. Note that the 790 attack may be performed by Internet-connected devices, including 791 constrained devices in the same CNN as the victim, as well as remote 792 ones. Mitigation techniques include RTO randomization and attack 793 blocking by routers able to detect shrew attacks based on their 794 traffic pattern. 796 7. Acknowledgments 798 Carles Gomez has been funded in part by the Spanish Government 799 (Ministerio de Educacion, Cultura y Deporte) through the Jose 800 Castillejo grants CAS15/00336 and and CAS18/00170, and by European 801 Regional Development Fund (ERDF) and the Spanish Government through 802 project TEC2016-79988-P, AEI/FEDER, UE. Part of his contribution to 803 this work has been carried out during his stays as a visiting scholar 804 at the Computer Laboratory of the University of Cambridge. 806 The authors appreciate the feedback received for this document. The 807 following folks provided comments that helped improve the document: 808 Carsten Bormann, Zhen Cao, Wei Genyu, Ari Keranen, Abhijan 809 Bhattacharyya, Andres Arcia-Moret, Yoshifumi Nishida, Joe Touch, Fred 810 Baker, Nik Sultana, Kerry Lynn, Erik Nordmark, Markku Kojo, Hannes 811 Tschofenig, David Black, Yoshifumi Nishida, Ilpo Jarvinen, Emmanuel 812 Baccelli, Stuart Cheshire, Gorry Fairhurst, and Ingemar Johansson. 814 Simon Brummer provided details, and kindly performed RAM and ROM 815 usage measurements, on the RIOT TCP implementation. Xavi Vilajosana 816 provided details on the OpenWSN TCP implementation. Rahul Jadhav 817 kindly performed code size measurements on the Contiki-NG and lwIP 818 2.1.2 TCP implementations. He also provided details on the uIP TCP 819 implementation. 821 8. Annex. TCP implementations for constrained devices 823 This section overviews the main features of TCP implementations for 824 constrained devices. The survey is limited to open source stacks 825 with small footprint. It is not meant to be all-encompassing. For 826 more powerful embedded systems (e.g., with 32-bit processors), there 827 are further stacks that comprehensively implement TCP. On the other 828 hand, please be aware that this Annex is based on information 829 available as of the writing. 831 8.1. uIP 833 uIP is a TCP/IP stack, targetted for 8 and 16-bit microcontrollers, 834 which pioneered TCP/IP implementations for constrained devices. uIP 835 has been deployed with Contiki and the Arduino Ethernet shield. A 836 code size of ~5 kB (which comprises checksumming, IP, ICMP and TCP) 837 has been reported for uIP [Dunk]. 839 uIP uses the same global buffer for both incoming and outgoing 840 traffic, which has a size of a single packet. In case of a 841 retransmission, an application must be able to reproduce the same 842 user data that had been transmitted. Multiple connections are 843 supported, but need to share the global buffer. 845 The MSS is announced via the MSS option on connection establishment 846 and the receive window size (of one MSS) is not modified during a 847 connection. Stop-and-wait operation is used for sending data. Among 848 other optimizations, this allows to avoid sliding window operations, 849 which use 32-bit arithmetic extensively and are expensive on 8-bit 850 CPUs. 852 Contiki uses the "split hack" technique (see Section 4.2.3) to avoid 853 Delayed ACKs for senders using a single segment. 855 The code size of the TCP implementation in Contiki-NG has been 856 measured to be of 3.2 kB on CC2538DK, cross-compiling on Linux. 858 8.2. lwIP 860 lwIP is a TCP/IP stack, targetted for 8- and 16-bit microcontrollers. 861 lwIP has a total code size of ~14 kB to ~22 kB (which comprises 862 memory management, checksumming, network interfaces, IP, ICMP and 863 TCP), and a TCP code size of ~9 kB to ~14 kB [Dunk]. 865 In contrast with uIP, lwIP decouples applications from the network 866 stack. lwIP supports a TCP transmission window greater than a single 867 segment, as well as buffering of incoming and outcoming data. Other 868 implemented mechanisms comprise slow start, congestion avoidance, 869 fast retransmit and fast recovery. SACK and Window Scale support has 870 been recently added to lwIP. 872 8.3. RIOT 874 The RIOT TCP implementation (called GNRC TCP) has been designed for 875 Class 1 devices [RFC 7228]. The main target platforms are 8- and 876 16-bit microcontrollers, with 32-bit platforms also supported. GNRC 877 TCP offers a similar function set as uIP, but it provides and 878 maintains an independent receive buffer for each connection. In 879 contrast to uIP, retransmission is also handled by GNRC TCP. For 880 simplicity, GNRC TCP uses a single-segment implementation. The 881 application programmer does not need to know anything about the TCP 882 internals, therefore GNRC TCP can be seen as a user-friendly uIP TCP 883 implementation. 885 The MSS is set on connections establishment and cannot be changed 886 during connection lifetime. GNRC TCP allows multiple connections in 887 parallel, but each TCB must be allocated somewhere in the system. By 888 default there is only enough memory allocated for a single TCP 889 connection, but it can be increased at compile time if the user needs 890 multiple parallel connections. 892 The RIOT TCP implementation offers an optional POSIX socket wrapper 893 that enables POSIX compliance, if needed. 895 Further details on RIOT and GNRC can be found in the literature 896 [RIOT], [GNRC]. 898 8.4. TinyOS 900 TinyOS was important as platform for early constrained devices. 901 TinyOS has an experimental TCP stack that uses a simple nonblocking 902 library-based implementation of TCP, which provides a subset of the 903 socket interface primitives. The application is responsible for 904 buffering. The TCP library does not do any receive-side buffering. 905 Instead, it will immediately dispatch new, in-order data to the 906 application and otherwise drop the segment. A send buffer is 907 provided by the application. Multiple TCP connections are possible. 908 Recently there has been little further work on the stack. 910 8.5. FreeRTOS 912 FreeRTOS is a real-time operating system kernel for embedded devices 913 that is supported by 16- and 32-bit microprocessors. Its TCP 914 implementation is based on multiple-segment window size, although a 915 'Tiny-TCP' option, which is a single-segment variant, can be enabled. 916 Delayed ACKs are supported, with a 20-ms Delayed ACK timer as a 917 technique intended 'to gain performance'. 919 8.6. uC/OS 921 uC/OS is a real-time operating system kernel for embedded devices, 922 which is maintained by Micrium. uC/OS is intended for 8-, 16- and 923 32-bit microprocessors. The uC/OS TCP implementation supports a 924 multiple-segment window size. 926 8.7. Summary 927 +---+---------+--------+----+------+--------+-----+ 928 |uIP|lwIP orig|lwIP 2.1|RIOT|TinyOS|FreeRTOS|uC/OS| 929 +------+-------------+---+---------+--------+----+------+--------+-----+ 930 |Memory|Code size(kB)| <5|~9 to ~14| 38 | <7 | N/A | <9.2 | N/A | 931 | | |(a)| (T1) | (T4) |(T3)| | (T2) | | 932 +------+-------------+---+---------+--------+----+------+--------+-----+ 933 | | Single-Segm.|Yes| No | No | Yes| No | No | No | 934 | +-------------+---+---------+--------+----+------+--------+-----+ 935 | | Slow start | No| Yes | Yes | No | Yes | No | Yes | 936 | T +-------------+---+---------+--------+----+------+--------+-----+ 937 | C |Fast rec/retx| No| Yes | Yes | No | Yes | No | Yes | 938 | P +-------------+---+---------+--------+----+------+--------+-----+ 939 | | Keep-alive | No| No | Yes | No | No | Yes | Yes | 940 | +-------------+---+---------+--------+----+------+--------+-----+ 941 | f | Win. Scale | No| No | Yes | No | No | Yes | No | 942 | e +-------------+---+---------+--------+----+------+--------+-----+ 943 | a | TCP timest.| No| No | Yes | No | No | Yes | No | 944 | t +-------------+---+---------+--------+----+------+--------+-----+ 945 | u | SACK | No| No | Yes | No | No | Yes | No | 946 | r +-------------+---+---------+--------+----+------+--------+-----+ 947 | e | Del. ACKs | No| Yes | Yes | No | No | Yes | Yes | 948 | s +-------------+---+---------+--------+----+------+--------+-----+ 949 | | Socket | No| No |Optional|(I) |Subset| Yes | Yes | 950 | +-------------+---+---------+--------+----+------+--------+-----+ 951 | |Concur. Conn.|Yes| Yes | Yes | Yes| Yes | Yes | Yes | 952 +------+-------------+---+---------+--------+----+------+--------+-----+ 953 | TLS supported | No| No | Yes | Yes| Yes | Yes | Yes | 954 +--------------------+---+---------+--------+----+------+--------+-----+ 956 (T1) = TCP-only, on x86 and AVR platforms 957 (T2) = TCP-only, on ARM Cortex-M platform 958 (T3) = TCP-only, on ARM Cortex-M0+ platform (NOTE: RAM usage for the same platform 959 is ~2.5 kB for one TCP connection plus ~1.2 kB for each additional connection) 960 (T4) = TCP-only, on CC2538DK, cross-compiling on Linux 961 (a) = includes IP, ICMP and TCP on x86 and AVR platforms. The Contiki-NG TCP implementation has a code size of 3.2 kB on CC2538DK, cross-compiling on Linux 962 (I) = optional POSIX socket wrapper which enables POSIX compliance if needed 963 Mult. = Multiple 964 N/A = Not Available 966 Figure 2: Summary of TCP features for differrent lightweight TCP 967 implementations. None of the implementations considered in this 968 Annex support ECN or TFO. 970 9. Annex. Changes compared to previous versions 972 RFC Editor: To be removed prior to publication 974 9.1. Changes between -00 and -01 976 o Changed title and abstract 978 o Clarification that communcation with standard-compliant TCP 979 endpoints is required, based on feedback from Joe Touch 981 o Additional discussion on communication patters 983 o Numerous changes to address a comprehensive review from Hannes 984 Tschofenig 986 o Reworded security considerations 988 o Additional references and better distinction between normative and 989 informative entries 991 o Feedback from Rahul Jadhav on the uIP TCP implementation 993 o Basic data for the TinyOS TCP implementation added, based on 994 source code analysis 996 9.2. Changes between -01 and -02 998 o Added text to the Introduction section, and a reference, on 999 traditional bad perception of TCP for IoT 1001 o Added sections on FreeRTOS and uC/OS 1003 o Updated TinyOS section 1005 o Updated summary table 1007 o Reorganized Section 4 (single-MSS vs multiple-MSS window size), 1008 some content now also in new Section 5 1010 9.3. Changes between -02 and -03 1012 o Rewording to better explain the benefit of ECN 1014 o Additional context information on the surveyed implementations 1016 o Added details, but removed "Data size" raw, in the summary table 1017 o Added discussion on shrew attacks 1019 9.4. Changes between -03 and -04 1021 o Addressing the remaining TODOs 1023 o Alignment of the wording on TCP "keep-alives" with related 1024 discussions in the IETF transport area 1026 o Added further discussion on delayed ACKs 1028 o Removed OpenWSN subsection from the Annex 1030 9.5. Changes between -04 and -05 1032 o Addressing comments by Yoshifumi Nishida 1034 o Removed mentioning MD5 as an example (comment by David Black) 1036 o Added memory footprint details of TCP implementations (Contiki-NG 1037 and lwIP 2.1.2) provided by Rahul Jadhav in the Annex 1039 o Addressed comments by Ilpo Jarvinen throughout the whole document 1041 o Improved the RIOT section in the Annex, based on feedback from 1042 Emmanuel Baccelli 1044 9.6. Changes between -05 and -06 1046 o Incorporated suggestions by Stuart Cheshire 1048 9.7. Changes between -06 and -07 1050 o Addressed comments by Gorry Fairhurst 1052 9.8. Changes between -07 and -08 1054 o Addressed WGLC comments by Ilpo Jarvinen, Markku Kojo and Ingemar 1055 Johansson throughout the document, including the addition of a new 1056 subsection on Initial Window considerations. 1058 10. References 1060 10.1. Normative References 1062 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, 1063 RFC 793, DOI 10.17487/RFC0793, September 1981, 1064 . 1066 [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - 1067 Communication Layers", STD 3, RFC 1122, 1068 DOI 10.17487/RFC1122, October 1989, 1069 . 1071 [RFC2018] Mathis, M., Mahdavi, J., Floyd, S., and A. Romanow, "TCP 1072 Selective Acknowledgment Options", RFC 2018, 1073 DOI 10.17487/RFC2018, October 1996, 1074 . 1076 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1077 Requirement Levels", BCP 14, RFC 2119, 1078 DOI 10.17487/RFC2119, March 1997, 1079 . 1081 [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1082 (IPv6) Specification", RFC 2460, DOI 10.17487/RFC2460, 1083 December 1998, . 1085 [RFC3042] Allman, M., Balakrishnan, H., and S. Floyd, "Enhancing 1086 TCP's Loss Recovery Using Limited Transmit", RFC 3042, 1087 DOI 10.17487/RFC3042, January 2001, 1088 . 1090 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 1091 of Explicit Congestion Notification (ECN) to IP", 1092 RFC 3168, DOI 10.17487/RFC3168, September 2001, 1093 . 1095 [RFC3819] Karn, P., Ed., Bormann, C., Fairhurst, G., Grossman, D., 1096 Ludwig, R., Mahdavi, J., Montenegro, G., Touch, J., and L. 1097 Wood, "Advice for Internet Subnetwork Designers", BCP 89, 1098 RFC 3819, DOI 10.17487/RFC3819, July 2004, 1099 . 1101 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 1102 Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, 1103 . 1105 [RFC5925] Touch, J., Mankin, A., and R. Bonica, "The TCP 1106 Authentication Option", RFC 5925, DOI 10.17487/RFC5925, 1107 June 2010, . 1109 [RFC6298] Paxson, V., Allman, M., Chu, J., and M. Sargent, 1110 "Computing TCP's Retransmission Timer", RFC 6298, 1111 DOI 10.17487/RFC6298, June 2011, 1112 . 1114 [RFC6928] Chu, J., Dukkipati, N., Cheng, Y., and M. Mathis, 1115 "Increasing TCP's Initial Window", RFC 6928, 1116 DOI 10.17487/RFC6928, April 2013, 1117 . 1119 [RFC7228] Bormann, C., Ersue, M., and A. Keranen, "Terminology for 1120 Constrained-Node Networks", RFC 7228, 1121 DOI 10.17487/RFC7228, May 2014, 1122 . 1124 [RFC7323] Borman, D., Braden, B., Jacobson, V., and R. 1125 Scheffenegger, Ed., "TCP Extensions for High Performance", 1126 RFC 7323, DOI 10.17487/RFC7323, September 2014, 1127 . 1129 [RFC7413] Cheng, Y., Chu, J., Radhakrishnan, S., and A. Jain, "TCP 1130 Fast Open", RFC 7413, DOI 10.17487/RFC7413, December 2014, 1131 . 1133 10.2. Informative References 1135 [Commag] A. Betzler, C. Gomez, I. Demirkol, J. Paradells, "CoAP 1136 Congestion Control for the Internet of Things", IEEE 1137 Communications Magazine, June 2016. 1139 [Dunk] A. Dunkels, "Full TCP/IP for 8-Bit Architectures", 2003. 1141 [ETEN] R. Krishnan et al, "Explicit transport error notification 1142 (ETEN) for error-prone wireless and satellite networks", 1143 Computer Networks 2004. 1145 [GNRC] M. Lenders et al., "Connecting the World of Embedded 1146 Mobiles: The RIOTApproach to Ubiquitous Networking for the 1147 IoT", 2018. 1149 [HomeGateway] 1150 Haetoenen, S., Nyrhinen, A., Eggert, L., Strowes, S., 1151 Sarolahti, P., and M. Kojo, "An Experimental Study of Home 1152 Gateway Characteristics", Proceedings of the 10th ACM 1153 SIGCOMM conference on Internet measurement 2010. 1155 [I-D.delcarpio-6lo-wlanah] 1156 Vega, L., Robles, I., and R. Morabito, "IPv6 over 1157 802.11ah", draft-delcarpio-6lo-wlanah-01 (work in 1158 progress), October 2015. 1160 [I-D.ietf-tcpm-rto-consider] 1161 Allman, M., "Retransmission Timeout Requirements", draft- 1162 ietf-tcpm-rto-consider-08 (work in progress), February 1163 2019. 1165 [IntComp] C. Gomez, A. Arcia-Moret, J. Crowcroft, "TCP in the 1166 Internet of Things: from ostracism to prominence", IEEE 1167 Internet Computing, January-February 2018. 1169 [RFC2757] Montenegro, G., Dawkins, S., Kojo, M., Magret, V., and N. 1170 Vaidya, "Long Thin Networks", RFC 2757, 1171 DOI 10.17487/RFC2757, January 2000, 1172 . 1174 [RFC2884] Hadi Salim, J. and U. Ahmed, "Performance Evaluation of 1175 Explicit Congestion Notification (ECN) in IP Networks", 1176 RFC 2884, DOI 10.17487/RFC2884, July 2000, 1177 . 1179 [RFC3481] Inamura, H., Ed., Montenegro, G., Ed., Ludwig, R., Gurtov, 1180 A., and F. Khafizov, "TCP over Second (2.5G) and Third 1181 (3G) Generation Wireless Networks", BCP 71, RFC 3481, 1182 DOI 10.17487/RFC3481, February 2003, 1183 . 1185 [RFC4944] Montenegro, G., Kushalnagar, N., Hui, J., and D. Culler, 1186 "Transmission of IPv6 Packets over IEEE 802.15.4 1187 Networks", RFC 4944, DOI 10.17487/RFC4944, September 2007, 1188 . 1190 [RFC6077] Papadimitriou, D., Ed., Welzl, M., Scharf, M., and B. 1191 Briscoe, "Open Research Issues in Internet Congestion 1192 Control", RFC 6077, DOI 10.17487/RFC6077, February 2011, 1193 . 1195 [RFC6092] Woodyatt, J., Ed., "Recommended Simple Security 1196 Capabilities in Customer Premises Equipment (CPE) for 1197 Providing Residential IPv6 Internet Service", RFC 6092, 1198 DOI 10.17487/RFC6092, January 2011, 1199 . 1201 [RFC6120] Saint-Andre, P., "Extensible Messaging and Presence 1202 Protocol (XMPP): Core", RFC 6120, DOI 10.17487/RFC6120, 1203 March 2011, . 1205 [RFC6282] Hui, J., Ed. and P. Thubert, "Compression Format for IPv6 1206 Datagrams over IEEE 802.15.4-Based Networks", RFC 6282, 1207 DOI 10.17487/RFC6282, September 2011, 1208 . 1210 [RFC6550] Winter, T., Ed., Thubert, P., Ed., Brandt, A., Hui, J., 1211 Kelsey, R., Levis, P., Pister, K., Struik, R., Vasseur, 1212 JP., and R. Alexander, "RPL: IPv6 Routing Protocol for 1213 Low-Power and Lossy Networks", RFC 6550, 1214 DOI 10.17487/RFC6550, March 2012, 1215 . 1217 [RFC6606] Kim, E., Kaspar, D., Gomez, C., and C. Bormann, "Problem 1218 Statement and Requirements for IPv6 over Low-Power 1219 Wireless Personal Area Network (6LoWPAN) Routing", 1220 RFC 6606, DOI 10.17487/RFC6606, May 2012, 1221 . 1223 [RFC6775] Shelby, Z., Ed., Chakrabarti, S., Nordmark, E., and C. 1224 Bormann, "Neighbor Discovery Optimization for IPv6 over 1225 Low-Power Wireless Personal Area Networks (6LoWPANs)", 1226 RFC 6775, DOI 10.17487/RFC6775, November 2012, 1227 . 1229 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 1230 Protocol (HTTP/1.1): Message Syntax and Routing", 1231 RFC 7230, DOI 10.17487/RFC7230, June 2014, 1232 . 1234 [RFC7252] Shelby, Z., Hartke, K., and C. Bormann, "The Constrained 1235 Application Protocol (CoAP)", RFC 7252, 1236 DOI 10.17487/RFC7252, June 2014, 1237 . 1239 [RFC7414] Duke, M., Braden, R., Eddy, W., Blanton, E., and A. 1240 Zimmermann, "A Roadmap for Transmission Control Protocol 1241 (TCP) Specification Documents", RFC 7414, 1242 DOI 10.17487/RFC7414, February 2015, 1243 . 1245 [RFC7428] Brandt, A. and J. Buron, "Transmission of IPv6 Packets 1246 over ITU-T G.9959 Networks", RFC 7428, 1247 DOI 10.17487/RFC7428, February 2015, 1248 . 1250 [RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext 1251 Transfer Protocol Version 2 (HTTP/2)", RFC 7540, 1252 DOI 10.17487/RFC7540, May 2015, 1253 . 1255 [RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF 1256 Recommendations Regarding Active Queue Management", 1257 BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015, 1258 . 1260 [RFC7668] Nieminen, J., Savolainen, T., Isomaki, M., Patil, B., 1261 Shelby, Z., and C. Gomez, "IPv6 over BLUETOOTH(R) Low 1262 Energy", RFC 7668, DOI 10.17487/RFC7668, October 2015, 1263 . 1265 [RFC8087] Fairhurst, G. and M. Welzl, "The Benefits of Using 1266 Explicit Congestion Notification (ECN)", RFC 8087, 1267 DOI 10.17487/RFC8087, March 2017, 1268 . 1270 [RFC8105] Mariager, P., Petersen, J., Ed., Shelby, Z., Van de Logt, 1271 M., and D. Barthel, "Transmission of IPv6 Packets over 1272 Digital Enhanced Cordless Telecommunications (DECT) Ultra 1273 Low Energy (ULE)", RFC 8105, DOI 10.17487/RFC8105, May 1274 2017, . 1276 [RFC8163] Lynn, K., Ed., Martocci, J., Neilson, C., and S. 1277 Donaldson, "Transmission of IPv6 over Master-Slave/Token- 1278 Passing (MS/TP) Networks", RFC 8163, DOI 10.17487/RFC8163, 1279 May 2017, . 1281 [RFC8201] McCann, J., Deering, S., Mogul, J., and R. Hinden, Ed., 1282 "Path MTU Discovery for IP version 6", STD 87, RFC 8201, 1283 DOI 10.17487/RFC8201, July 2017, 1284 . 1286 [RFC8323] Bormann, C., Lemay, S., Tschofenig, H., Hartke, K., 1287 Silverajan, B., and B. Raymor, Ed., "CoAP (Constrained 1288 Application Protocol) over TCP, TLS, and WebSockets", 1289 RFC 8323, DOI 10.17487/RFC8323, February 2018, 1290 . 1292 [RFC8352] Gomez, C., Kovatsch, M., Tian, H., and Z. Cao, Ed., 1293 "Energy-Efficient Features of Internet of Things 1294 Protocols", RFC 8352, DOI 10.17487/RFC8352, April 2018, 1295 . 1297 [RFC8376] Farrell, S., Ed., "Low-Power Wide Area Network (LPWAN) 1298 Overview", RFC 8376, DOI 10.17487/RFC8376, May 2018, 1299 . 1301 [RIOT] E. Baccelli et al., "RIOT: an Open Source Operating 1302 Systemfor Low-end Embedded Devices in the IoT", 2018. 1304 [shrew] A. Kuzmanovic, E. Knightly, "Low-Rate TCP-Targeted Denial 1305 of Service Attacks", SIGCOMM'03 2003. 1307 Authors' Addresses 1309 Carles Gomez 1310 UPC 1311 C/Esteve Terradas, 7 1312 Castelldefels 08860 1313 Spain 1315 Email: carlesgo@entel.upc.edu 1317 Jon Crowcroft 1318 University of Cambridge 1319 JJ Thomson Avenue 1320 Cambridge, CB3 0FD 1321 United Kingdom 1323 Email: jon.crowcroft@cl.cam.ac.uk 1325 Michael Scharf 1326 Hochschule Esslingen 1327 Flandernstr. 101 1328 Esslingen 73732 1329 Germany 1331 Email: michael.scharf@hs-esslingen.de