idnits 2.17.1 draft-retana-rtgwg-eacp-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 17, 2013) is 3836 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Routing Area Working Group A. Retana 3 Internet-Draft Cisco Systems, Inc. 4 Intended status: Informational R. White 5 Expires: April 20, 2014 6 M. Paul 7 Deutsche Telekom AG 8 October 17, 2013 10 A Framework and Requirements for Energy Aware Control Planes 11 draft-retana-rtgwg-eacp-02 13 Abstract 15 There has been, for several years, a rising concern over the energy 16 usage of large scale networks. This concern is strongly focused on 17 campus, data center, and other highly concentrated deployments of 18 network infrastructure. Given the steadily increasing demand for 19 higher network speeds, always-on service models, and ubiquitous 20 network coverage, it is also of growing importance for 21 telecommunication networks both local and wide area in scope. One of 22 the issues in moving forward to reduce energy usage is to ensure that 23 the network can still meet the performance specifications required to 24 support the applications running over it. 26 This document provides an overview of the various areas of concern in 27 the interaction between network performance and efforts at energy 28 aware control planes, as a guide for those working on modifying 29 current control planes or designing new control planes to improve the 30 energy efficiency of high density, highly complex, network 31 deployments. 33 Status of this Memo 35 This Internet-Draft is submitted in full conformance with the 36 provisions of BCP 78 and BCP 79. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF). Note that other groups may also distribute 40 working documents as Internet-Drafts. The list of current Internet- 41 Drafts is at http://datatracker.ietf.org/drafts/current/. 43 Internet-Drafts are draft documents valid for a maximum of six months 44 and may be updated, replaced, or obsoleted by other documents at any 45 time. It is inappropriate to use Internet-Drafts as reference 46 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on April 20, 2014. 50 Copyright Notice 52 Copyright (c) 2013 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents 57 (http://trustee.ietf.org/license-info) in effect on the date of 58 publication of this document. Please review these documents 59 carefully, as they describe your rights and restrictions with respect 60 to this document. Code Components extracted from this document must 61 include Simplified BSD License text as described in Section 4.e of 62 the Trust Legal Provisions and are provided without warranty as 63 described in the Simplified BSD License. 65 Table of Contents 67 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 68 2. Requirements Language . . . . . . . . . . . . . . . . . . . . 4 69 3. Background . . . . . . . . . . . . . . . . . . . . . . . . . . 5 70 3.1. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 5 71 3.2. Business Drivers . . . . . . . . . . . . . . . . . . . . . 6 72 3.3. Application Drivers . . . . . . . . . . . . . . . . . . . 6 73 4. Framework . . . . . . . . . . . . . . . . . . . . . . . . . . 7 74 4.1. Modes of Reducing Energy Usage . . . . . . . . . . . . . . 7 75 4.1.1. Example Network . . . . . . . . . . . . . . . . . . . 8 76 4.1.2. Examples of Energy Reduction . . . . . . . . . . . . . 8 77 4.2. Global Verses Local Decisions . . . . . . . . . . . . . . 9 78 5. Considerations and Requirements . . . . . . . . . . . . . . . 9 79 5.1. Energy Efficiency and Bandwidth Reduction . . . . . . . . 9 80 5.1.1. An Example of Lowered Bandwidth . . . . . . . . . . . 10 81 5.1.2. Requirements . . . . . . . . . . . . . . . . . . . . . 10 82 5.2. Energy Efficiency and Stretch . . . . . . . . . . . . . . 10 83 5.2.1. An Example of Stretch . . . . . . . . . . . . . . . . 11 84 5.2.2. Requirements . . . . . . . . . . . . . . . . . . . . . 11 85 5.3. Energy Efficiency and Fast Recovery . . . . . . . . . . . 12 86 5.3.1. An Example of Impact on Fast Recovery . . . . . . . . 12 87 5.3.2. Requirements . . . . . . . . . . . . . . . . . . . . . 12 88 5.4. Introducing Jitter Through Microsleeps . . . . . . . . . . 13 89 5.4.1. An Example of Microsleeps to Reduce Energy Usage . . . 13 90 5.4.2. Requirements . . . . . . . . . . . . . . . . . . . . . 14 91 5.5. Other Operational Aspects . . . . . . . . . . . . . . . . 14 92 5.5.1. An Example of Operational Impact . . . . . . . . . . . 14 93 5.5.2. Requirements . . . . . . . . . . . . . . . . . . . . . 14 94 6. Security Considerations . . . . . . . . . . . . . . . . . . . 14 95 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 15 96 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 15 97 8.1. Normative References . . . . . . . . . . . . . . . . . . . 15 98 8.2. Informative References . . . . . . . . . . . . . . . . . . 15 99 Appendix A. Change Log . . . . . . . . . . . . . . . . . . . . . 15 100 A.1. Changes between the -00 and -01 versions. . . . . . . . . 15 101 A.2. Changes between the -01 and -02 versions. . . . . . . . . 16 102 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 16 104 1. Introduction 106 As energy prices continue to increase, and energy awareness becomes a 107 watchword for most large companies, places where the network 108 infrastructure used a good deal of power have come under increased 109 scrutiny for savings. There is a concern, however, in saving energy 110 at the cost of network operations --to reduce performance along with 111 energy consumption, negatively impacting the operation of a network 112 and the applications reliant on that network. This concern is 113 primarily focused on the network control plane, but will necessarily 114 apply to network performance and energy usage overall. 116 This document provides a background, a framework for understanding 117 and managing the tradeoffs between modifications made to network 118 protocols to conserve energy and network performance metrics and 119 requirements, and a set of requirements for protocol designers to 120 consider in proposals for new control plane protocols or 121 modifications to existing control plane protocols. It is intended to 122 encourage work on mechanisms that will reduce network energy usage 123 while providing perspective on balancing energy usage against 124 performance. The ultimate goal is to provide the tools and knowledge 125 necessary for protocol designers to modify network protocols to best 126 balance efficiency against performance, and to provide the background 127 information network operators will need to intelligently deploy and 128 use protocol modifications to network protocols. 130 The document is organized as follows. Section 3 provides material 131 the reader needs to understand to appreciate the challenges inherent 132 in balancing energy reduction with effective network performance. 133 This section includes subsections considering the application and 134 business requirements that are the basis of the reset of the 135 document. Section 4 provides a framework for understanding 136 mechanisms common to all energy management schemes proposed to date 137 in general terms. Section 5 provides an analysis of the areas 138 highlighted, including an explanation of how the specific area 139 interacts with energy management, and example of the interaction, 140 and, finally, a set of requirements protocol designers should 141 consider when proposing either new protocols or modifications to 142 existing protocols to reduce energy usage. 144 2. Requirements Language 146 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 147 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 148 document are to be interpreted as described in [RFC2119]. 150 3. Background 152 The background covered here describes the underlying business and 153 application drivers for the consideration and requirements sections 154 below. This section also contains a small example network used 155 throughout the remainder of this document for explaining various 156 mechanisms and technical points. 158 3.1. Scope 160 The reader should differentiate between radio based and wireline (or 161 rather, "plugged in"), networks. Radio based networks designed for 162 rapid deployment for highly mobile users (often called Mobile Ad Hoc 163 Networks, or MANETs [MANET]), and sensor networks designed for low 164 power, processing, and memory (such as those described in [ROLL]), 165 are not the target of this document. Readers should refer to the 166 groups working within those areas for energy management requirements 167 based on those specialized environment. While protocol developers 168 for those environments may draw useful information from this 169 document, this work is not intended to address those specialized 170 networks specifically. Mobile cellular networks however are 171 similarly affected by excess energy consumption as wireline networks 172 and seek to save energy by methods as described in the following (see 173 e.g. [3GPP]). 175 The reader should also differentiate between intradomain and 176 interdomain applications. Interdomain applications require more work 177 in policy than in technical and business considerations, and 178 therefore fall outside the scope of this document. Intradomain 179 control planes are (intuitively) where most energy savings will be 180 attained, at any rate. Most high concentrations of routers, such as 181 data centers and campus networks, are under a single administrative 182 domain. Therefore, placing interdomain control planes outside the 183 scope of this document does not limit its usefulness in any 184 meaningful way. 186 The reader should further differentiate between the components of an 187 energy management system, namely energy monitoring and energy 188 control. Energy monitoring deals with the collection of information 189 related to energy utilization and characteristics, as described in 190 [EMAN]. Energy control relates to directly influencing the 191 optimization and/or efficiency of devices in the network. The focus 192 of this document is on understanding the tradeoffs between 193 modifications made to network protocols to conserve energy and 194 network performance metrics and requirements, not on the functions, 195 steps or procedures required for energy monitoring. 197 3.2. Business Drivers 199 Networks are primarily built to support both broad and narrow 200 business requirements. Broad business requirements might include 201 general communication requirements, such as providing email service 202 between internal and external personnel, or providing general access 203 to the World Wide Web for research and business support. Narrow 204 requirements would relate to specific applications, such as 205 supporting a particular financial application in the case of a bank 206 or other financial enterprise, or supporting customer traffic in the 207 case of a service provider. Application requirements will be 208 considered in greater detail in the next section. 210 Another class of requirements business place on networks can be 211 called operational requirements. These include (but are not limited 212 to), capital expense, operational expense, and the restrictions the 213 network architecture places on the growth and operation of the 214 business itself. These, in turn, drive requirements such as change 215 management, total uptime (availability), and the ability of the 216 network to be easily and quickly modified to meet new business 217 demands, or to shed old business demands. Operational expense is the 218 primary area this document covers in relation to business 219 requirements, because this is where energy management most obviously 220 overlaps with network performance. 222 3.3. Application Drivers 224 Applications drivers provide the background for each of the technical 225 sections below. When approaching a specific application, there are 226 only a small number of questions network and protocol designers need 227 to fully understand to shape networks and protocols so a specific 228 application can be supported. The first two questions revolve around 229 bandwidth; how much bandwidth will the application consume, and is 230 this bandwidth consumption fairly steady, or highly variable? For 231 instance, applications such as streaming video tend to have long 232 lasting flows with high bandwidth requirements, file transfers tend 233 to produce shorter flows requiring high bandwidth, and HTML traffic 234 tends to be bursty, with much lower bandwidth requirements. 236 The next question a protocol or network designer might ask about a 237 specific application is it's tolerance to jitter. Real time 238 applications, such as voice and video conferencing, have a very low 239 toleration for jitter. File transfers and streaming video, on the 240 other hand, can often handle large variations in packet arrival 241 times. If packets are delayed long enough, the application may 242 actually time out, shutting down sessions. Users will often "hang 243 up" after a short period of time, as well, causing loss of revenue 244 and productivity. 246 Delay is another crucial factor in the performance of many 247 applications. Many server virtualization protocols, for instance, 248 have very low tolerance for delay, having been written with a short 249 wire local broadcast segment in mind. Applications such as stock and 250 commodity trading, remote medical, and collaborative video editing 251 also exhibit very little tolerance for delay. 253 These last two application drivers, jitter and delay, are normally 254 the result of two underlying causes within a network's control plane: 255 stretch and convergence. Stretch (defined more fully in the section 256 considering stretch below) causes longer paths to be taken through 257 the network. Each hop in the network path adds serialization into 258 and out of a set of queues in device memory, along with the delays of 259 various queuing mechanisms implemented on that device. Each hop in 260 the network increases delay directly, and has the potential to 261 increase jitter as packets pass into and out of the additional 262 devices. 264 Network convergence will also show up as jitter in an application's 265 stream; if packets are held up or looped for hundreds of milliseconds 266 during a network convergence event, applications running over the 267 converging topology will see this convergence time as a massive 268 jitter event, or a short term delay in the delivery of packets. 270 Jitter and delay can also be introduced directly into the packet 271 stream by reducing the throughput of individual links, or putting 272 devices and/or links into energy reduced modes for very short periods 273 of time (microsleeps). If a link is asleep when the first and third 274 packets from a flow arrive at the head end of the link, and not when 275 the second packet from that same flow arrives, each packet is going 276 to be processed differently, and hence will have a different delay 277 across the path. 279 The specific technical problems addressed in the following sections, 280 then, are bandwidth reduction, increasing stretch, network 281 convergence, and introducing jitter through microsleeps. 283 4. Framework 285 4.1. Modes of Reducing Energy Usage 287 Regardless of whether the control plane is centralized (such as some 288 form of centrally computed traffic engineering or software defined 289 network), or distributed (traditional routing protocols), there are 290 four primary ways in which energy usage can be reduced: 292 o Removing redundant links from the network topology 294 o Removing redundant network equipment from the network topology 296 o Reducing the amount of time equipment or links are operational 298 o Reducing the link speed or processing rate of equipment 300 4.1.1. Example Network 302 To illustrate the impacts of link and device removal throughout the 303 rest of this document, the following network is used. 305 /---R2---\ /---\ 306 R1 R4 R5 307 \---R3---/ \---/ 309 This network is overly simplistic so the impact of removing various 310 links and devices from the topology can be more clearly illustrated. 311 More complex topologies will often exhibit these same impacts without 312 being so obvious. 314 4.1.2. Examples of Energy Reduction 316 In the example network above, several different modes of energy 317 reduction might be: 319 o Shutting down one of the two links between R4 and R5 321 o Shutting down one of the two links between R4 and R5, and shutting 322 down any line cards (or part of the nodes themselves) associated 323 with the removal of these links 325 o Shutting down R2 or R3, since these represent alternate paths to 326 reach the same set of destinations 328 o Shutting down the link between R2 and R4, since similar 329 connectivity is provided through R1->R3->R4 331 o Shutting down all links and devices for fractions of time in a 332 coordinated fashion 334 o Shutting down individual links as traffic or the control plane 335 permits for fractions of time (here the momentary shutdown of 336 various links is not coordinated, but undertaken hop by hop) 338 o Reducing the speed of all links and devices for fractions of time 339 in a coordinated fashion 341 o Reducing the speed of individual links as traffic or the control 342 plane permits for fractions of time (here the momentary slowdown 343 of various links is not coordinated, but undertaken hop by hop) 345 4.2. Global Verses Local Decisions 347 Independent of whether the control plane is centralized or 348 distributed, the scope considered when making a decision about energy 349 efficiency may affect the result and effectiveness of the system. 350 There are clearly two extreme options when looking at the scope of 351 the information used to make decisions. The first extreme is that of 352 every device in the network considering only local conditions, and 353 determining the proper local state from that information. An example 354 of this mode of operation might be a local link where the devices on 355 either side of that link measure the link utilization, and 356 independently decide to automatically shut the link down when 357 utilization reaches a specific threshold. An example of the other 358 end of the spectrum might be a network control plane in which all the 359 nodes involved agree before taking a specific action; in the case of 360 two parallel links, the devices on each end not only would have 361 similar configured policies, but would coordinate if one of the links 362 was to be turned off. It is outside the scope of this document to 363 determine which of these two options may be optimal or "best." 365 There are some considerations and tradeoffs which need to be outlined 366 in considering the global versus local decisions in relation to 367 energy efficiency. System designers should take note of the 368 difficulties with preventing pathological conditions when purely 369 localized decisions are made. For instance, in the example network, 370 assume R1 determines to put the R1->R2 link into an energy saving 371 mode, while R4 determines to put the R4->R3 link into an energy 372 saving mode. In this case, no path will remain available through the 373 network. It is also possible for the opposite to occur, that is for 374 no links or devices to be placed into a reduced energy state because 375 R1 and R4 don't agree through the control plane which links and 376 devices should be removed from the topology. 378 Protocol designers should consider these tradeoffs in proposals for 379 energy aware control planes. 381 5. Considerations and Requirements 383 5.1. Energy Efficiency and Bandwidth Reduction 385 Bandwidth is an important consideration in high density networks; 386 most data centers are designed to provide a specific amount of 387 bandwidth into and out of each server and to facilitate virtual 388 server movement among physical devices. In campus and core networks 389 bandwidth is finely coupled with quality of service guarantees for 390 applications and services. It should be obvious that removing links 391 or devices from a network topology will adversely affect the amount 392 of available bandwidth, which could, in turn, cause well thought out 393 quality of service mechanisms to degrade or fail. 395 What might not be so obvious is the relationship between available 396 bandwidth and jitter, or other network quality of service measures. 397 If higher speed links are removed from the topology in order to 398 continue using lower speed (and therefore presumably lower power) 399 links, then serialization delays will have a larger impact on traffic 400 flow. Longer serialization delays can cause input queues to back up, 401 which impacts not only delay but jitter, and possibly even traffic 402 delivery. 404 5.1.1. An Example of Lowered Bandwidth 406 In the network illustrated above, one of the two links between R4 and 407 R5 could be an obvious candidate for removal from the network. 408 Especially if the network load can easily be transferred to the 409 remaining link without failure, and without serious consequences for 410 delay or jitter in the network, there is a strong case to be made for 411 doing so --particularly if the accompanying line cards could also be 412 shut down to add to the energy savings. 414 5.1.2. Requirements 416 Modifications to control plane protocols to achieve network energy 417 efficiency SHOULD provide the ability to set the minimal bandwidth, 418 jitter, and delay through the network, and not shut down links or 419 devices that would violate those minimal requirements. 421 5.2. Energy Efficiency and Stretch 423 In any given network, there is a shortest path between any source and 424 any destination. Network protocols discover these paths from the 425 destination's perspective --routing draws traffic along a path, 426 rather than driving along a path. Along with the shortest path, 427 there are a number of paths that can also carry traffic from a given 428 source to a given destination without the packets passing along the 429 same logical link, or through the same logical device, more than 430 once. These are considered loop-free alternate [RFC5714] paths. 432 The primary difference between the shortest path and the loop-free 433 alternate paths is the total cost of using the path. In simple 434 terms, this difference can be calculated as the number of links and 435 devices a packet must pass through when being carried from the source 436 to the destination --the hop count. While most networks use much 437 more sophisticated metrics based on bandwidth, congestion, and other 438 factors, the hop count will stand in as the only metric used 439 throughout this document. 441 When the control plane causes traffic to pass from the source to the 442 destination along a path which is longer than the shortest path, the 443 network is said to have stretch (see [Krioukov] for a more in depth 444 explanation of network stretch). To measure stretch, simply subtract 445 the metric of the shortest path from the metric of the longer path. 446 For example, in hop count terms, if the best path is three hops, and 447 the current path is four hops, the network exhibits a stretch of 1. 449 5.2.1. An Example of Stretch 451 In the network illustrated above, if a modification is made to the 452 control plane in order to remove the link between R1 and R4 in order 453 to save energy, all the destinations shown in the diagram remain 454 reachable. However, from the perspective of R1, the best path 455 available to reach R2 has increased in length by two hops. The 456 original path is R1->R2, the new path is R1->R3->R4->R2. This 457 represents a stretch of 2. 459 Along with this increased stretch will most likely also come 460 increased delay through the network; each hop in the network 461 represents a measurable amount of delay. This increased stretch 462 might also represent an increased amount of jitter, as there are more 463 queues and more serialization events in the path of each packet 464 carried. There will also be the modifications in jitter as the 465 network switches between the optimal performance configuration and an 466 energy efficient configuration. 468 5.2.2. Requirements 470 Designers who propose modifications to control plane protocols to 471 achieve network energy efficiency SHOULD analyze the impact of their 472 mechanisms on the stretch in typical network topologies, and SHOULD 473 include such analysis when explaining the applicability of their 474 proposals. This analysis may include an examination of the absolute, 475 or maximum, stretch caused by the modifications to the control plane 476 as well as analysis at the 95th percentile, the average stretch 477 increase in a given set of topologies, and/or the mean increase in 478 stretch. 480 Mechanisms that could impact the stretch of a network SHOULD provide 481 the ability for the network administrator to limit the amount of 482 stretch the network will encounter when moving into a more energy 483 efficient mode. 485 5.3. Energy Efficiency and Fast Recovery 487 A final area where modifications to the control plane for energy 488 efficiency is fast convergence or fast recovery. Many networks are 489 now designed to recover from failures quickly enough to only cause a 490 handful of traffic to be lost; recovery on the order of half a second 491 is not an uncommon goal. It should be obvious that removing 492 redundant links and devices from the network to reduce energy 493 consumption could adversely affect these goals. 495 5.3.1. An Example of Impact on Fast Recovery 497 In the network shown, assume R2 and its associated links are removed 498 from the topology in order to save energy. Rather than this second 499 path being available for immediate recovery on the failure of the 500 R1->R3 link, some process must be followed to bring R2 and its 501 associated links back up, reinject them into the topology, and 502 finally begin routing traffic across this path. 504 In many situations, only links and devices which are a "third point 505 of failure" may be acceptable as removal candidates in order to 506 conserve energy. 508 5.3.2. Requirements 510 Modifications to the control plane in order to remove links or nodes 511 to conserve energy SHOULD entail the ability to choose the level of 512 redundancy available after the network topology has been trimmed. 513 For instance, it might be acceptable in some situations to move to 514 single points of failure throughout the network, or in specific 515 sections of the network, for certain periods of time. In other 516 situations, it may only be acceptable to reduce the network to a 517 double point of failure, and never to a single point of failure. 519 The complete removal of nodes or links from the network topology has 520 several impacts on the control plane which must be considered. In 521 these cases, the control plane must: 523 o Modify the network topology so removed links or devices are not 524 used to forward traffic 526 o Remember that such links exist, possibly including the neighbors 527 and destinations reachable through those links or devices 529 5.4. Introducing Jitter Through Microsleeps 531 One proposed mechanism to reduce energy usage in a network is to 532 sleep links or devices for very short periods of time, called 533 microsleeps. For instance, if a particular link is only used at 50% 534 of the actual available bandwidth, it should be possible to place the 535 link in some lower power state for 50% of the time, thus reducing 536 energy usage by something percentage. 538 Such schemes introduce delay and jitter into the network path 539 directly; if a packet arrives while the link to the next hop, or the 540 next hop itself, is in a reduced energy state, the packet must wait 541 until the link or next hop device enter a normal operational mode 542 before it can be forwarded. Most of the time the proposed sleep 543 states are so small as to be presumably inconsequential on overall 544 packet delay, but multiple packets crossing a series of links, each 545 encountering different links in different states, could take very 546 different amounts of time to pass along the path. 548 One possible way to resolve this somewhat random accrual of delays on 549 a per packet basis is to coordinate these sleep states such that 550 packets accepted at the entry of the network are consistently passed 551 through the network when all links and devices are in a normal 552 operating mode, and simply delaying all packets at the entry point 553 into the network while the devices in the network are in some energy 554 reduced state. This solution still introduces some amount of jitter; 555 some packets will be delayed by the sleep state at the edge of the 556 network, while others will not. This solution also requires 557 coordinated timers at the speed of forwarding itself to effectively 558 control the sleep and wake cycles of the network. 560 5.4.1. An Example of Microsleeps to Reduce Energy Usage 562 In the example network, assume the bandwidth utilization along the 563 path R1->R2->R4->R5 is 50% of the actual available bandwidth along 564 this path. It is possible to consider a scheme where R1->R2, R2, 565 R2->R4, and R4->R5 are all put into some energy reduced operational 566 mode 50% of the time, since packets are only available to send 50% of 567 the time. A packet entering at R1 may encounter a short delay at 568 R1->R2, at R2->R4, and at R4->R5, or it might not. Even if these 569 delays are very small, say 200ms at each hop, the accumulated delay 570 through the network due to sleep states may be 0ms (all links and 571 devices awake) or 600ms (all links and devices asleep) as the packet 572 passes through the network. 574 As network paths lengthen to more realistic path lengths in real 575 deployments, the jitter introduced varies more widely, which could 576 cause problems for the operation of a number of applications. 578 5.4.2. Requirements 580 Protocol designers SHOULD analyze the impact of accumulated jitter 581 when proposing mechanisms that rely on microsleeps in either 582 equipment or links. This analysis SHOULD include both worst case and 583 best case scenarios, as well as an analysis of how coordinated clocks 584 are to be handled in the case of coordinated sleep states. 586 5.5. Other Operational Aspects 588 Modification of the network topology in order to save energy needs to 589 consider the operational needs of the network as well as application 590 requirements. Change management, operational downtime, and business 591 usage of the network need to be considered when determining which 592 links and nodes should be placed into a low energy state. Energy 593 provisions have to be assigned and changed for nodes and links, 594 optimally according to network usage profiles over the time of day. 596 Control plane protocol operation, in terms of operational efficiency 597 on the wire, also needs to be considered when modifying protocol 598 parameters. Any changes that negatively impact the operation of the 599 protocol, in terms of the amount of traffic, the size of routing 600 information transmitted over the network, and interaction with 601 network management operations need to be carefully analyzed for 602 scaling and operational implications. 604 5.5.1. An Example of Operational Impact 606 Time of day is an important consideration in business operations. 607 During normal operational hours, the network needs to be fully 608 available, including all available redundancy and bandwidth. During 609 holidays, night hours, and other times when a campus might not be 610 used, or when there are lower traffic and resiliency demands on the 611 network, network elements can be removed to reduce energy usage. 613 5.5.2. Requirements 615 Protocol designers SHOULD analyze operational requirements, such as 616 time of day and network traffic load considerations, and explain how 617 proposed protocols or modifications to protocols will interact with 618 these types of requirements. Protocols designers SHOULD analyze 619 increases in network traffic and the operational efficiency impact of 620 proposed changes or protocols. 622 6. Security Considerations 624 None. 626 7. Acknowledgements 628 The authors of this document would like to acknowledge the 629 suggestions and ideas provided by Sujata Banerjee, Puneet Sharma and 630 Dirk Von Hugo. 632 8. References 634 8.1. Normative References 636 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 637 Requirement Levels", BCP 14, RFC 2119, March 1997. 639 8.2. Informative References 641 [3GPP] 3GPP, "3GPP TR 25-927 Solutions for energy saving within 642 UTRA Node B", 2011, 643 . 645 [EMAN] IETF, "Energy Management Working Group Charter", 2012, 646 . 648 [Krioukov] 649 Krioukov, D., "On Compact Routing for the Internet", 2007, 650 . 653 [MANET] IETF, "Mobile Ad Hoc Networks Charter", 2012, 654 . 656 [RFC5714] Shand, M. and S. Bryant, "IP Fast Reroute Framework", 657 RFC 5714, January 2010. 659 [ROLL] IETF, "Routing Over Lowe power and Lossy networks 660 Charter", 2012, 661 . 663 Appendix A. Change Log 665 A.1. Changes between the -00 and -01 versions. 667 o Updated authors' contact information. 669 o Modified some of the rfc2119 keywords. 671 A.2. Changes between the -01 and -02 versions. 673 o Updated authors' contact information. 675 Authors' Addresses 677 Alvaro Retana 678 Cisco Systems, Inc. 679 7025 Kit Creek Rd. 680 Raleigh, NC 27709 681 USA 683 Email: aretana@cisco.com 685 Russ White 687 Email: russw@riw.us 689 Manuel Paul 690 Deutsche Telekom AG 691 Winterfeldtstr. 21-27 692 Berlin 10781 693 Germany 695 Email: Manuel.Paul@telekom.de