idnits 2.17.1 draft-hui-vasseur-roll-rpl-deployment-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 8) being 66 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 8 instances of too long lines in the document, the longest one being 3 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 296 has weird spacing: '...e nodes had a...' == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (July 2, 2012) is 4315 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC5226' is defined on line 370, but no explicit reference was found in the text ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) == Outdated reference: A later version (-13) exists of draft-ietf-roll-terminology-06 Summary: 2 errors (**), 0 flaws (~~), 6 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Networking Working Group JP. Vasseur, Ed. 3 Internet-Draft J. Hui, Ed. 4 Intended status: Informational S. Dasgupta 5 Expires: January 3, 2013 G. Yoon 6 Cisco Systems, Inc 7 July 2, 2012 9 RPL deployment experience in large scale networks 10 draft-hui-vasseur-roll-rpl-deployment-00 12 Abstract 14 Low power and Lossy Networks (LLNs) exhibit characteristics unlike 15 other more traditional IP links. LLNs are a class of network in 16 which both routers and their interconnect are resource constrained. 17 LLN routers are typically resource constrained in processing power, 18 memory, and energy (i.e. battery power). LLN links are typically 19 exhibit high loss rates, low data rates, are are strongly affected by 20 environmental conditions that change over time. LLNs may be composed 21 of a few dozen to thousands of routers. A new protocol called the 22 IPv6 Routing Protocol for Low-Power and Lossy Networks (RPL) has been 23 specified for routing in LLNs supporting multipoint-to-point, point- 24 to-multipoint traffic, and point-to-point traffic. Since RPL's 25 publication as an RFC, several large scale networks have been 26 succesfully deployed. The aim of this document is to provide 27 deployment experience on real-life deployed RPL-based networks. 29 Requirements Language 31 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 32 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 33 document are to be interpreted as described in RFC 2119 [RFC2119]. 35 Status of this Memo 37 This Internet-Draft is submitted in full conformance with the 38 provisions of BCP 78 and BCP 79. 40 Internet-Drafts are working documents of the Internet Engineering 41 Task Force (IETF). Note that other groups may also distribute 42 working documents as Internet-Drafts. The list of current Internet- 43 Drafts is at http://datatracker.ietf.org/drafts/current/. 45 Internet-Drafts are draft documents valid for a maximum of six months 46 and may be updated, replaced, or obsoleted by other documents at any 47 time. It is inappropriate to use Internet-Drafts as reference 48 material or to cite them other than as "work in progress." 49 This Internet-Draft will expire on January 3, 2013. 51 Copyright Notice 53 Copyright (c) 2012 IETF Trust and the persons identified as the 54 document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's Legal 57 Provisions Relating to IETF Documents 58 (http://trustee.ietf.org/license-info) in effect on the date of 59 publication of this document. Please review these documents 60 carefully, as they describe your rights and restrictions with respect 61 to this document. Code Components extracted from this document must 62 include Simplified BSD License text as described in Section 4.e of 63 the Trust Legal Provisions and are provided without warranty as 64 described in the Simplified BSD License. 66 Table of Contents 68 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 69 2. Objective of this document . . . . . . . . . . . . . . . . . . 6 70 3. RPL Paramaters Settings . . . . . . . . . . . . . . . . . . . 7 71 4. Network Characteristics . . . . . . . . . . . . . . . . . . . 8 72 5. Performance Results . . . . . . . . . . . . . . . . . . . . . 8 73 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 74 7. Security Considerations . . . . . . . . . . . . . . . . . . . 9 75 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 9 76 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 9 77 9.1. Normative references . . . . . . . . . . . . . . . . . . . 9 78 9.2. Informative references . . . . . . . . . . . . . . . . . . 10 79 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 11 81 1. Introduction 83 Low power and Lossy Networks (LLNs) exhibit characteristics unlike 84 other more traditional IP links. LLNs are a class of network in 85 which both routers and their interconnect are resource constrained. 86 LLN routers are typically resource constrained in processing power, 87 memory, and energy (i.e. battery power). LLN links are typically 88 exhibit high loss rates, low data rates, are are strongly affected by 89 environmental conditions that change over time. LLNs may be composed 90 of a few dozen to thousands of routers. 92 A new protocol called the IPv6 Routing Protocol for Low-Power and 93 Lossy Networks (RPL) has been specified for routing in LLNs supporting 94 multipoint-to-point, point-to-multipoint traffic, and point-to-point 95 traffic [RFC6550]. Since RPL's publication as an RFC, several large 96 scale networks have been successfully deployed. The aim of this 97 document is to provide deployment experience on real-life deployed 98 RPL-based networks. 100 In addition to [RFC6550], companion documents have been defined that 101 specify IPv6 packet options required for the proper operation of RPL, 102 including [RFC6553] and [RFC6554]. 104 This document makes use of the terminology defined in 105 [I-D.ietf-roll-terminology]. 107 RPL is a distance-vector routing protocol that builds a Destination 108 Oriented Directed Acyclic Graph (DODAG) according to an Objective 109 Function (OF). The OF is a defined set of rules that optimize paths 110 against a set of metrics and/or constraints. A very basic OF, known 111 as OF0, is specified in [RFC6552]. More involved OFs may be 112 specified, such as the Minimum Rank with Hysteresis Objective 113 Function specified in [I-D.ietf-roll-minrank-hysteresis-of]. 115 Routing requirements documents spelled out in [RFC5673], [RFC5826], 116 [RFC5548] and [RFC5867]) observe that it must be possible to take 117 into account a variety of node metrics and/or constraints during path 118 computation. Thus, a number of routing metrics and constraints for 119 RPL have been specified in [RFC6551] for maximum flexibility 120 according to the objectives and environment of the LLN. 122 RPL supports efficient loop detection using data-path route 123 validation and supports both local and global route repair 124 operations. 126 RPL makes use of the Trickle algorithm, which provides a density- 127 aware mechanism for distributing and maintaining state within a 128 network [RFC6206]. With simple local rules, the Trickle algorithm 129 adjusts the transmission period and suppresses redundant 130 transmissions to minimize control traffic overhead in the steady 131 state while propagating new information quickly. Trickle's 132 suppression mechanisms ensures that control message overhead grows 133 logrithmically with node density. 135 In maintaining point-to-multipoint routes, RPL supports two modes of 136 operations: non-storing and storing. In both cases, the DODAG built 137 by RPL according to the OF is used for hop-by-hop upstream routing 138 towards the DAG Root. In non-storing mode, only the DAG Root 139 maintains downward routes and all data packets must traverse the DAG 140 Root. In storing mode, LLN routers also maintain downward routing 141 state, allowing each LLN router to forward data packets to devices in 142 their sub-DAG. LLN constraints, the network objective, and overall 143 environment typically drives the choice of non-storing or storing 144 mode and is left to the network administrator. 146 RPL, like many other routing protocols, is designed to be deployed in 147 a number of different operational environments and [RFC6550] 148 specifies a number of configuration parameters. Section 17 of 149 [RFC6550] lists the following RPL constants and variables: 151 o DEFAULT_PATH_CONTROL_SIZE: This is the default value used to 152 configure PCS in the DODAG Configuration option, which dictates 153 the number of significant bits in the Path Control field of the 154 Transit Information option. DEFAULT_PATH_CONTROL_SIZE has a value 155 of 0. This configures the simplest case limiting the fan-out to 1 156 and limiting a node to send a DAO message to only one parent. 158 o DEFAULT_DIO_INTERVAL_MIN: This is the default value used to 159 configure Imin for the DIO Trickle timer. 160 DEFAULT_DIO_INTERVAL_MIN has a value of 3. This configuration 161 results in Imin of 8 ms. 163 o DEFAULT_DIO_INTERVAL_DOUBLINGS: This is the default value used to 164 configure Imax for the DIO Trickle timer. 165 DEFAULT_DIO_INTERVAL_DOUBLINGS has a value of 20. This 166 configuration results in a maximum interval of 2.3 hours. 168 o DEFAULT_DIO_REDUNDANCY_CONSTANT: This is the default value used to 169 configure k for the DIO Trickle timer. 170 DEFAULT_DIO_REDUNDANCY_CONSTANT has a value of 10. This 171 configuration is a conservative value for Trickle suppression 172 mechanism. 174 o DEFAULT_MIN_HOP_RANK_INCREASE: This is the default value of 175 MinHopRankIncrease. DEFAULT_MIN_HOP_RANK_INCREASE has a value of 176 256. This configuration results in an 8-bit wide integer part of 177 Rank. 179 o DEFAULT_DAO_DELAY: This is the default value for the DelayDAO 180 Timer. DEFAULT_DAO_DELAY has a value of 1 second. 182 o DIO Timer: One instance per DODAG of which a node is a member. 183 Expiry triggers DIO message transmission. A Trickle timer with 184 variable interval in [0, DIOIntervalMin..2^DIOIntervalDoublings]. 186 o DAG Version Increment Timer: Up to one instance per DODAG of which 187 the node is acting as DODAG root. May not be supported in all 188 implementations. Expiry triggers increment of DODAGVersionNumber, 189 causing a new series of updated DIO message to be sent. Interval 190 should be chosen appropriate to propagation time of DODAG and as 191 appropriate to application requirements (e.g., response time 192 versus overhead). 194 o DelayDAO Timer: Up to one timer per DAO parent (the subset of 195 DODAG parents chosen to receive destination advertisements) per 196 DODAG. Expiry triggers sending of DAO message to the DAO parent. 198 o RemoveTimer: Up to one timer per DAO entry per neighbor (i.e., 199 those neighbors that have given DAO messages to this node as a 200 DODAG parent). Expiry may trigger No-Path advertisements or 201 immediately deallocate the DAO entry if there are no DAO parents. 203 Please refer to the .pdf version of this document to see the figures 204 referred in further sections. 206 2. Objective of this document 208 Since its specification as a standard track RFC in March 2012, a 209 number of RPL-based networks have been deployed in the field, some of 210 small size, others of large scale. The aim of this document is to 211 describe the successful deployment of a RPL-based LLN with 1,000 212 nodes. Other networks of even larger scale (5,000 to 10,000 nodes) 213 are in progress and further revisions of this document will include 214 their details. 216 It is nearly impossible to characterize the absolute performance of a 217 protocol without looking at all the environmental factors and a large 218 number of performance metrics. Furthermore such performance metric 219 not only depends on the environment but also how the various protocol 220 parameters have been configured. Similarly it would not make any 221 sense to provide hard numbers on a performance characteristic of a 222 protocol. For example, Open Shortest Path First (OSPF) routing 223 protocol [RFC2328] may provide convergence times varying between few 224 dozens of milliseconds to seconds depending on the network 225 characteritics and protocol parameters. At one end of the spectrum, 226 fast failure detection with fast Hellos or the use of other protocols 227 such as Bidirectional Forwarding Detection (BFD) [RFC5880], combined 228 with fast LSA generation, LSA prioritization, fast SPF triggering and 229 an optimized SPF calculation (potentially combined with incremental 230 SPF) would lead to a few dozens of milliseconds of convergence times. 231 At the other end of the spectrum slow detection of failure, combined 232 with low priority trigger of Link State Advertisement (LSA), poor 233 implementation of the Shortest Path First (SPF) algorithm, long 234 propagation delays, lack of LSA control plane packets may lead to 235 covergence times of seconds! 237 While convergence time is not the critical performance metric in many 238 LLN deployments, the convergence time example provided above is one 239 that illustrates the challenge of providing performance results. 240 This challenge generally applies to most other performance metrics. 242 As a result, the aim of this document is not to provide absolute 243 performance numbers or parameter setting recommendations, but rather 244 to share successful experience of the large scale deployment of RPL in 245 a real-life deployment scenario. 247 To that end, we first provide several network characteristics such as 248 the network topology, distribution of the link quality providing the 249 link quality distribution according to the Expected Transmission 250 count (ETX) link metric computed by the RPL nodes. Then we provide 251 indications of how RPL was used in that particular network before 252 showing several performance metrics observed in this network. 254 3. RPL Paramaters Settings 256 This RPL network includes the following parameter settings: 258 o The Mode of Operation (MoP) is set to non-storing mode. 260 o Both local and global repair mechanisms are implemented. Note, 261 however, because the network operates in non-storing mode, local 262 repair simply poisons routes and does not create floating DAGs. 264 o MaxRankIncrease is set to 0, which significantly reduces the 265 possibility of routing loops but also limits the capabilities of 266 local repair. 268 o The OF is the Minimum Rank with Hysteresis Object Function using 269 the ETX metric. 271 4. Network Characteristics 273 This network comprises one thousand nodes and the distribution of the 274 hop counts is shown is Figure 1. This has been obtained by observing 275 the topology (shown in Figure 2) for a period of 24 hours and tracking 276 the hop count of all the nodes every 5 minutes. It can be seen that 277 approximately 51% of the nodes are 1 hop away and 30% of nodes are 2 278 hops away on average. Another way of saying this is that approximately 279 81% of the nodes are 2 hops or less from the root. A snapshot of the 280 network topology (the DODAG built by RPL) is depicted in Figure 2. 282 Figure 1: Distribution of average hop count of nodes observed over a 24- 283 hour period. 285 Figure 2: DODAG Topology built by RPL 287 As with any LLN, one can observe that the some links are of good 288 quality while others provides low path delivery rates: this can be 289 seen by observing the link ETX in Figure 3. Note that we observed 290 transient periods during which the ETX was much higher with links 291 providing even intermittent connectivity (which is not always 292 reflected in the ETX value due to the computation of the ETX is using 293 moving averages to avoid network oscillations and over-reactions). 294 Figure 3 was obtained by observing all the nodes periodically for a 24 295 hour period. We tracked the maximum and minimum ETX seen by the node as 296 well. From the figure, we can see that almost 90% of the nodes had an 297 average ETX of 1.25 or less over the 24 hour period. 299 Figure 3: Distribution of average, maximum and minimum ETX observed 300 over a 24-hour period. 302 The LLN routers communicate using IEEE 802.15.4g links. Operating 303 in the 902-928 MHz US ISM band, the links have an effective data rate 304 of 75 kbps and employ frequency hopping to communicate across 64 305 channels with 400 kHz channel spacing. 307 In summary, it can be observed that the network is indeed a LLN, with 308 lossy links. 310 The network is made of constrained nodes with limited processing 311 power and available memory. The root slightly less constrained and 312 main powered. 314 It is worth pointing out that the high density of this topology added 315 stress on the routing protocol. 317 5. Performance Results 319 As pointed out in Section 1, there is not a single performance metric 320 that could be provided to characterize the routing protocol 321 performance. 323 Deep analysis of a number of network management events, logs on 324 routers, and packet inspection operation have shown that the routing 325 topology was quite stable even during unstable conditions. More 326 importantly the observed packet delivery rate was always above 99%: 327 by contrast with non LLN networks where the routing protocol is 328 rarely responsible for non packet delivery because of the absence of 329 routes to reach a destination, several LLN routing protocols have 330 reported low delivery packet rates because of routing issues. In 331 this particular network, the packet delivery rate was as high as 99% 332 in all cases (link local packet retransmissions were handled by the 333 IEEE 802.15.4 reliable link layer). 335 Since questions were raised in the past about the RPL control plane 336 overhead although RPL has been designed for low overhead, we paid a 337 particular attention to this performance metric. RPL has been 338 designed to optimize the control plane overhead (thanks to the use of 339 Neighbor Unreachability Detection (NUD) instead of routing hello 340 packet to detect link/node failure, use of trickle algorithm for the 341 transmission of the DIO packets, ...). 343 Thus we show in Figure 4 the RPL control traffic overhead (both the 344 DIO and the DAO are shown in the network) relative to the available 345 bandwidth provided by the links. Note that the RPL control plane 346 traffic was observed on the most congested area of the network (on 347 the DODAG root). 349 6. IANA Considerations 351 No action is required from IANA. 353 7. Security Considerations 355 This document provides informational data about existing deployments, 356 thus security considerations do not apply. 358 8. Acknowledgements 360 The authors would like to acknowledge the contributions of Ibrahim 361 Mortada for his very valuable contribution. 363 9. References 365 9.1. Normative references 367 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 368 Requirement Levels", BCP 14, RFC 2119, March 1997. 370 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 371 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 372 May 2008. 374 [RFC6550] Winter, T., Thubert, P., Brandt, A., Hui, J., Kelsey, R., 375 Levis, P., Pister, K., Struik, R., Vasseur, JP., and R. 376 Alexander, "RPL: IPv6 Routing Protocol for Low-Power and 377 Lossy Networks", RFC 6550, March 2012. 379 9.2. Informative references 381 [I-D.ietf-roll-minrank-hysteresis-of] 382 Gnawali, O. and P. Levis, "The Minimum Rank with 383 Hysteresis Objective Function", 384 draft-ietf-roll-minrank-hysteresis-of-11 (work in 385 progress), June 2012. 387 [I-D.ietf-roll-terminology] 388 Vasseur, J., "Terminology in Low power And Lossy 389 Networks", draft-ietf-roll-terminology-06 (work in 390 progress), September 2011. 392 [RFC2328] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998. 394 [RFC5548] Dohler, M., Watteyne, T., Winter, T., and D. Barthel, 395 "Routing Requirements for Urban Low-Power and Lossy 396 Networks", RFC 5548, May 2009. 398 [RFC5673] Pister, K., Thubert, P., Dwars, S., and T. Phinney, 399 "Industrial Routing Requirements in Low-Power and Lossy 400 Networks", RFC 5673, October 2009. 402 [RFC5826] Brandt, A., Buron, J., and G. Porcu, "Home Automation 403 Routing Requirements in Low-Power and Lossy Networks", 404 RFC 5826, April 2010. 406 [RFC5867] Martocci, J., De Mil, P., Riou, N., and W. Vermeylen, 407 "Building Automation Routing Requirements in Low-Power and 408 Lossy Networks", RFC 5867, June 2010. 410 [RFC5880] Katz, D. and D. Ward, "Bidirectional Forwarding Detection 411 (BFD)", RFC 5880, June 2010. 413 [RFC6206] Levis, P., Clausen, T., Hui, J., Gnawali, O., and J. Ko, 414 "The Trickle Algorithm", RFC 6206, March 2011. 416 [RFC6551] Vasseur, JP., Kim, M., Pister, K., Dejean, N., and D. 417 Barthel, "Routing Metrics Used for Path Calculation in 418 Low-Power and Lossy Networks", RFC 6551, March 2012. 420 [RFC6552] Thubert, P., "Objective Function Zero for the Routing 421 Protocol for Low-Power and Lossy Networks (RPL)", 422 RFC 6552, March 2012. 424 [RFC6553] Hui, J. and JP. Vasseur, "The Routing Protocol for Low- 425 Power and Lossy Networks (RPL) Option for Carrying RPL 426 Information in Data-Plane Datagrams", RFC 6553, 427 March 2012. 429 [RFC6554] Hui, J., Vasseur, JP., Culler, D., and V. Manral, "An IPv6 430 Routing Header for Source Routes with the Routing Protocol 431 for Low-Power and Lossy Networks (RPL)", RFC 6554, 432 March 2012. 434 Authors' Addresses 436 JP Vasseur (editor) 437 Cisco Systems, Inc 438 11, Rue Camille Desmoulins 439 Issy Les Moulineaux, 92782 440 France 442 Email: jpv@cisco.com 444 Jonathan Hui (editor) 445 Cisco Systems, Inc 446 560 McCarthy Blvd. 447 MILPITAS, CALIFORNIA 95035 448 UNITED STATES 450 Email: johui@cisco.com 452 Sukrit Dasgupta 453 Cisco Systems, Inc 454 300 Beaver Brook Road 455 BOXBOROUGH, MASSACHUSETTS 01719 456 UNITED STATES 458 Email: sukdasgu@cisco.com 459 Giyoung Yoon 460 Cisco Systems, Inc 461 560 McCarthy Blvd. 462 MILPITAS, CALIFORNIA 95035 463 UNITED STATES 465 Email: giyoon@cisco.com