idnits 2.17.1 draft-retana-rtgwg-eacp-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 24, 2014) is 3466 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Routing Area Working Group A. Retana 3 Internet-Draft Cisco Systems, Inc. 4 Intended status: Informational R. White 5 Expires: April 27, 2015 Ericsson 6 M. Paul 7 Deutsche Telekom AG 8 October 24, 2014 10 A Framework and Requirements for Energy Aware Control Planes 11 draft-retana-rtgwg-eacp-03 13 Abstract 15 There has been, for several years, a rising concern over the energy 16 usage of large scale networks. This concern is strongly focused on 17 campus, data center, and other highly concentrated deployments of 18 network infrastructure. Given the steadily increasing demand for 19 higher network speeds, always-on service models, and ubiquitous 20 network coverage, it is also of growing importance for 21 telecommunication networks both local and wide area in scope. One of 22 the issues in moving forward to reduce energy usage is to ensure that 23 the network can still meet the performance specifications required to 24 support the applications running over it. 26 This document provides an overview of the various areas of concern in 27 the interaction between network performance and efforts at energy 28 aware control planes, as a guide for those working on modifying 29 current control planes or designing new control planes to improve the 30 energy efficiency of high density, highly complex, network 31 deployments. 33 Status of this Memo 35 This Internet-Draft is submitted in full conformance with the 36 provisions of BCP 78 and BCP 79. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF). Note that other groups may also distribute 40 working documents as Internet-Drafts. The list of current Internet- 41 Drafts is at http://datatracker.ietf.org/drafts/current/. 43 Internet-Drafts are draft documents valid for a maximum of six months 44 and may be updated, replaced, or obsoleted by other documents at any 45 time. It is inappropriate to use Internet-Drafts as reference 46 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on April 27, 2015. 50 Copyright Notice 52 Copyright (c) 2014 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents 57 (http://trustee.ietf.org/license-info) in effect on the date of 58 publication of this document. Please review these documents 59 carefully, as they describe your rights and restrictions with respect 60 to this document. Code Components extracted from this document must 61 include Simplified BSD License text as described in Section 4.e of 62 the Trust Legal Provisions and are provided without warranty as 63 described in the Simplified BSD License. 65 Table of Contents 67 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 68 2. Requirements Language . . . . . . . . . . . . . . . . . . . . 4 69 3. Background . . . . . . . . . . . . . . . . . . . . . . . . . . 5 70 3.1. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 5 71 3.2. Business Drivers . . . . . . . . . . . . . . . . . . . . . 6 72 3.3. Application Drivers . . . . . . . . . . . . . . . . . . . 6 73 4. Framework . . . . . . . . . . . . . . . . . . . . . . . . . . 7 74 4.1. Modes of Reducing Energy Usage . . . . . . . . . . . . . . 7 75 4.1.1. Example Network . . . . . . . . . . . . . . . . . . . 8 76 4.1.2. Examples of Energy Reduction . . . . . . . . . . . . . 8 77 4.2. Global Verses Local Decisions . . . . . . . . . . . . . . 9 78 5. Considerations and Requirements . . . . . . . . . . . . . . . 9 79 5.1. Energy Efficiency and Bandwidth Reduction . . . . . . . . 9 80 5.1.1. An Example of Lowered Bandwidth . . . . . . . . . . . 10 81 5.1.2. Requirements . . . . . . . . . . . . . . . . . . . . . 10 82 5.2. Energy Efficiency and Stretch . . . . . . . . . . . . . . 10 83 5.2.1. An Example of Stretch . . . . . . . . . . . . . . . . 11 84 5.2.2. Requirements . . . . . . . . . . . . . . . . . . . . . 11 85 5.3. Energy Efficiency and Fast Recovery . . . . . . . . . . . 12 86 5.3.1. An Example of Impact on Fast Recovery . . . . . . . . 12 87 5.3.2. Requirements . . . . . . . . . . . . . . . . . . . . . 12 88 5.4. Introducing Jitter Through Microsleeps . . . . . . . . . . 13 89 5.4.1. An Example of Microsleeps to Reduce Energy Usage . . . 13 90 5.4.2. Requirements . . . . . . . . . . . . . . . . . . . . . 14 91 5.5. Other Operational Aspects . . . . . . . . . . . . . . . . 14 92 5.5.1. An Example of Operational Impact . . . . . . . . . . . 14 93 5.5.2. Requirements . . . . . . . . . . . . . . . . . . . . . 14 94 6. Security Considerations . . . . . . . . . . . . . . . . . . . 14 95 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 15 96 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 15 97 8.1. Normative References . . . . . . . . . . . . . . . . . . . 15 98 8.2. Informative References . . . . . . . . . . . . . . . . . . 15 99 Appendix A. Change Log . . . . . . . . . . . . . . . . . . . . . 15 100 A.1. Changes between the -00 and -01 versions. . . . . . . . . 15 101 A.2. Changes between the -01 and -02 versions. . . . . . . . . 16 102 A.3. Changes between the -02 and -03 versions. . . . . . . . . 16 103 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 16 105 1. Introduction 107 As energy prices continue to increase, and energy awareness becomes a 108 watchword for most large companies, places where the network 109 infrastructure used a good deal of power have come under increased 110 scrutiny for savings. There is a concern, however, in saving energy 111 at the cost of network operations --to reduce performance along with 112 energy consumption, negatively impacting the operation of a network 113 and the applications reliant on that network. This concern is 114 primarily focused on the network control plane, but will necessarily 115 apply to network performance and energy usage overall. 117 This document provides a background, a framework for understanding 118 and managing the tradeoffs between modifications made to network 119 protocols to conserve energy and network performance metrics and 120 requirements, and a set of requirements for protocol designers to 121 consider in proposals for new control plane protocols or 122 modifications to existing control plane protocols. It is intended to 123 encourage work on mechanisms that will reduce network energy usage 124 while providing perspective on balancing energy usage against 125 performance. The ultimate goal is to provide the tools and knowledge 126 necessary for protocol designers to modify network protocols to best 127 balance efficiency against performance, and to provide the background 128 information network operators will need to intelligently deploy and 129 use protocol modifications to network protocols. 131 The document is organized as follows. Section 3 provides material 132 the reader needs to understand to appreciate the challenges inherent 133 in balancing energy reduction with effective network performance. 134 This section includes subsections considering the application and 135 business requirements that are the basis of the reset of the 136 document. Section 4 provides a framework for understanding 137 mechanisms common to all energy management schemes proposed to date 138 in general terms. Section 5 provides an analysis of the areas 139 highlighted, including an explanation of how the specific area 140 interacts with energy management, and example of the interaction, 141 and, finally, a set of requirements protocol designers should 142 consider when proposing either new protocols or modifications to 143 existing protocols to reduce energy usage. 145 2. Requirements Language 147 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 148 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 149 document are to be interpreted as described in [RFC2119]. 151 3. Background 153 The background covered here describes the underlying business and 154 application drivers for the consideration and requirements sections 155 below. This section also contains a small example network used 156 throughout the remainder of this document for explaining various 157 mechanisms and technical points. 159 3.1. Scope 161 The reader should differentiate between radio based and wireline (or 162 rather, "plugged in"), networks. Radio based networks designed for 163 rapid deployment for highly mobile users (often called Mobile Ad Hoc 164 Networks, or MANETs [MANET]), and sensor networks designed for low 165 power, processing, and memory (such as those described in [ROLL]), 166 are not the target of this document. Readers should refer to the 167 groups working within those areas for energy management requirements 168 based on those specialized environment. While protocol developers 169 for those environments may draw useful information from this 170 document, this work is not intended to address those specialized 171 networks specifically. Mobile cellular networks however are 172 similarly affected by excess energy consumption as wireline networks 173 and seek to save energy by methods as described in the following (see 174 e.g. [3GPP]). 176 The reader should also differentiate between intradomain and 177 interdomain applications. Interdomain applications require more work 178 in policy than in technical and business considerations, and 179 therefore fall outside the scope of this document. Intradomain 180 control planes are (intuitively) where most energy savings will be 181 attained, at any rate. Most high concentrations of routers, such as 182 data centers and campus networks, are under a single administrative 183 domain. Therefore, placing interdomain control planes outside the 184 scope of this document does not limit its usefulness in any 185 meaningful way. 187 The reader should further differentiate between the components of an 188 energy management system, namely energy monitoring and energy 189 control. Energy monitoring deals with the collection of information 190 related to energy utilization and characteristics, as described in 191 [EMAN]. Energy control relates to directly influencing the 192 optimization and/or efficiency of devices in the network. The focus 193 of this document is on understanding the tradeoffs between 194 modifications made to network protocols to conserve energy and 195 network performance metrics and requirements, not on the functions, 196 steps or procedures required for energy monitoring. 198 3.2. Business Drivers 200 Networks are primarily built to support both broad and narrow 201 business requirements. Broad business requirements might include 202 general communication requirements, such as providing email service 203 between internal and external personnel, or providing general access 204 to the World Wide Web for research and business support. Narrow 205 requirements would relate to specific applications, such as 206 supporting a particular financial application in the case of a bank 207 or other financial enterprise, or supporting customer traffic in the 208 case of a service provider. Application requirements will be 209 considered in greater detail in the next section. 211 Another class of requirements business place on networks can be 212 called operational requirements. These include (but are not limited 213 to), capital expense, operational expense, and the restrictions the 214 network architecture places on the growth and operation of the 215 business itself. These, in turn, drive requirements such as change 216 management, total uptime (availability), and the ability of the 217 network to be easily and quickly modified to meet new business 218 demands, or to shed old business demands. Operational expense is the 219 primary area this document covers in relation to business 220 requirements, because this is where energy management most obviously 221 overlaps with network performance. 223 3.3. Application Drivers 225 Applications drivers provide the background for each of the technical 226 sections below. When approaching a specific application, there are 227 only a small number of questions network and protocol designers need 228 to fully understand to shape networks and protocols so a specific 229 application can be supported. The first two questions revolve around 230 bandwidth; how much bandwidth will the application consume, and is 231 this bandwidth consumption fairly steady, or highly variable? For 232 instance, applications such as streaming video tend to have long 233 lasting flows with high bandwidth requirements, file transfers tend 234 to produce shorter flows requiring high bandwidth, and HTML traffic 235 tends to be bursty, with much lower bandwidth requirements. 237 The next question a protocol or network designer might ask about a 238 specific application is it's tolerance to jitter. Real time 239 applications, such as voice and video conferencing, have a very low 240 toleration for jitter. File transfers and streaming video, on the 241 other hand, can often handle large variations in packet arrival 242 times. If packets are delayed long enough, the application may 243 actually time out, shutting down sessions. Users will often "hang 244 up" after a short period of time, as well, causing loss of revenue 245 and productivity. 247 Delay is another crucial factor in the performance of many 248 applications. Many server virtualization protocols, for instance, 249 have very low tolerance for delay, having been written with a short 250 wire local broadcast segment in mind. Applications such as stock and 251 commodity trading, remote medical, and collaborative video editing 252 also exhibit very little tolerance for delay. 254 These last two application drivers, jitter and delay, are normally 255 the result of two underlying causes within a network's control plane: 256 stretch and convergence. Stretch (defined more fully in the section 257 considering stretch below) causes longer paths to be taken through 258 the network. Each hop in the network path adds serialization into 259 and out of a set of queues in device memory, along with the delays of 260 various queuing mechanisms implemented on that device. Each hop in 261 the network increases delay directly, and has the potential to 262 increase jitter as packets pass into and out of the additional 263 devices. 265 Network convergence will also show up as jitter in an application's 266 stream; if packets are held up or looped for hundreds of milliseconds 267 during a network convergence event, applications running over the 268 converging topology will see this convergence time as a massive 269 jitter event, or a short term delay in the delivery of packets. 271 Jitter and delay can also be introduced directly into the packet 272 stream by reducing the throughput of individual links, or putting 273 devices and/or links into energy reduced modes for very short periods 274 of time (microsleeps). If a link is asleep when the first and third 275 packets from a flow arrive at the head end of the link, and not when 276 the second packet from that same flow arrives, each packet is going 277 to be processed differently, and hence will have a different delay 278 across the path. 280 The specific technical problems addressed in the following sections, 281 then, are bandwidth reduction, increasing stretch, network 282 convergence, and introducing jitter through microsleeps. 284 4. Framework 286 4.1. Modes of Reducing Energy Usage 288 Regardless of whether the control plane is centralized (such as some 289 form of centrally computed traffic engineering or software defined 290 network), or distributed (traditional routing protocols), there are 291 four primary ways in which energy usage can be reduced: 293 o Removing redundant links from the network topology 295 o Removing redundant network equipment from the network topology 297 o Reducing the amount of time equipment or links are operational 299 o Reducing the link speed or processing rate of equipment 301 4.1.1. Example Network 303 To illustrate the impacts of link and device removal throughout the 304 rest of this document, the following network is used. 306 /---R2---\ /---\ 307 R1 R4 R5 308 \---R3---/ \---/ 310 This network is overly simplistic so the impact of removing various 311 links and devices from the topology can be more clearly illustrated. 312 More complex topologies will often exhibit these same impacts without 313 being so obvious. 315 4.1.2. Examples of Energy Reduction 317 In the example network above, several different modes of energy 318 reduction might be: 320 o Shutting down one of the two links between R4 and R5 322 o Shutting down one of the two links between R4 and R5, and shutting 323 down any line cards (or part of the nodes themselves) associated 324 with the removal of these links 326 o Shutting down R2 or R3, since these represent alternate paths to 327 reach the same set of destinations 329 o Shutting down the link between R2 and R4, since similar 330 connectivity is provided through R1->R3->R4 332 o Shutting down all links and devices for fractions of time in a 333 coordinated fashion 335 o Shutting down individual links as traffic or the control plane 336 permits for fractions of time (here the momentary shutdown of 337 various links is not coordinated, but undertaken hop by hop) 339 o Reducing the speed of all links and devices for fractions of time 340 in a coordinated fashion 342 o Reducing the speed of individual links as traffic or the control 343 plane permits for fractions of time (here the momentary slowdown 344 of various links is not coordinated, but undertaken hop by hop) 346 4.2. Global Verses Local Decisions 348 Independent of whether the control plane is centralized or 349 distributed, the scope considered when making a decision about energy 350 efficiency may affect the result and effectiveness of the system. 351 There are clearly two extreme options when looking at the scope of 352 the information used to make decisions. The first extreme is that of 353 every device in the network considering only local conditions, and 354 determining the proper local state from that information. An example 355 of this mode of operation might be a local link where the devices on 356 either side of that link measure the link utilization, and 357 independently decide to automatically shut the link down when 358 utilization reaches a specific threshold. An example of the other 359 end of the spectrum might be a network control plane in which all the 360 nodes involved agree before taking a specific action; in the case of 361 two parallel links, the devices on each end not only would have 362 similar configured policies, but would coordinate if one of the links 363 was to be turned off. It is outside the scope of this document to 364 determine which of these two options may be optimal or "best." 366 There are some considerations and tradeoffs which need to be outlined 367 in considering the global versus local decisions in relation to 368 energy efficiency. System designers should take note of the 369 difficulties with preventing pathological conditions when purely 370 localized decisions are made. For instance, in the example network, 371 assume R1 determines to put the R1->R2 link into an energy saving 372 mode, while R4 determines to put the R4->R3 link into an energy 373 saving mode. In this case, no path will remain available through the 374 network. It is also possible for the opposite to occur, that is for 375 no links or devices to be placed into a reduced energy state because 376 R1 and R4 don't agree through the control plane which links and 377 devices should be removed from the topology. 379 Protocol designers should consider these tradeoffs in proposals for 380 energy aware control planes. 382 5. Considerations and Requirements 384 5.1. Energy Efficiency and Bandwidth Reduction 386 Bandwidth is an important consideration in high density networks; 387 most data centers are designed to provide a specific amount of 388 bandwidth into and out of each server and to facilitate virtual 389 server movement among physical devices. In campus and core networks 390 bandwidth is finely coupled with quality of service guarantees for 391 applications and services. It should be obvious that removing links 392 or devices from a network topology will adversely affect the amount 393 of available bandwidth, which could, in turn, cause well thought out 394 quality of service mechanisms to degrade or fail. 396 What might not be so obvious is the relationship between available 397 bandwidth and jitter, or other network quality of service measures. 398 If higher speed links are removed from the topology in order to 399 continue using lower speed (and therefore presumably lower power) 400 links, then serialization delays will have a larger impact on traffic 401 flow. Longer serialization delays can cause input queues to back up, 402 which impacts not only delay but jitter, and possibly even traffic 403 delivery. 405 5.1.1. An Example of Lowered Bandwidth 407 In the network illustrated above, one of the two links between R4 and 408 R5 could be an obvious candidate for removal from the network. 409 Especially if the network load can easily be transferred to the 410 remaining link without failure, and without serious consequences for 411 delay or jitter in the network, there is a strong case to be made for 412 doing so --particularly if the accompanying line cards could also be 413 shut down to add to the energy savings. 415 5.1.2. Requirements 417 Modifications to control plane protocols to achieve network energy 418 efficiency SHOULD provide the ability to set the minimal bandwidth, 419 jitter, and delay through the network, and not shut down links or 420 devices that would violate those minimal requirements. 422 5.2. Energy Efficiency and Stretch 424 In any given network, there is a shortest path between any source and 425 any destination. Network protocols discover these paths from the 426 destination's perspective --routing draws traffic along a path, 427 rather than driving along a path. Along with the shortest path, 428 there are a number of paths that can also carry traffic from a given 429 source to a given destination without the packets passing along the 430 same logical link, or through the same logical device, more than 431 once. These are considered loop-free alternate [RFC5714] paths. 433 The primary difference between the shortest path and the loop-free 434 alternate paths is the total cost of using the path. In simple 435 terms, this difference can be calculated as the number of links and 436 devices a packet must pass through when being carried from the source 437 to the destination --the hop count. While most networks use much 438 more sophisticated metrics based on bandwidth, congestion, and other 439 factors, the hop count will stand in as the only metric used 440 throughout this document. 442 When the control plane causes traffic to pass from the source to the 443 destination along a path which is longer than the shortest path, the 444 network is said to have stretch (see [Krioukov] for a more in depth 445 explanation of network stretch). To measure stretch, simply subtract 446 the metric of the shortest path from the metric of the longer path. 447 For example, in hop count terms, if the best path is three hops, and 448 the current path is four hops, the network exhibits a stretch of 1. 450 5.2.1. An Example of Stretch 452 In the network illustrated above, if a modification is made to the 453 control plane in order to remove the link between R1 and R4 in order 454 to save energy, all the destinations shown in the diagram remain 455 reachable. However, from the perspective of R1, the best path 456 available to reach R2 has increased in length by two hops. The 457 original path is R1->R2, the new path is R1->R3->R4->R2. This 458 represents a stretch of 2. 460 Along with this increased stretch will most likely also come 461 increased delay through the network; each hop in the network 462 represents a measurable amount of delay. This increased stretch 463 might also represent an increased amount of jitter, as there are more 464 queues and more serialization events in the path of each packet 465 carried. There will also be the modifications in jitter as the 466 network switches between the optimal performance configuration and an 467 energy efficient configuration. 469 5.2.2. Requirements 471 Designers who propose modifications to control plane protocols to 472 achieve network energy efficiency SHOULD analyze the impact of their 473 mechanisms on the stretch in typical network topologies, and SHOULD 474 include such analysis when explaining the applicability of their 475 proposals. This analysis may include an examination of the absolute, 476 or maximum, stretch caused by the modifications to the control plane 477 as well as analysis at the 95th percentile, the average stretch 478 increase in a given set of topologies, and/or the mean increase in 479 stretch. 481 Mechanisms that could impact the stretch of a network SHOULD provide 482 the ability for the network administrator to limit the amount of 483 stretch the network will encounter when moving into a more energy 484 efficient mode. 486 5.3. Energy Efficiency and Fast Recovery 488 A final area where modifications to the control plane for energy 489 efficiency is fast convergence or fast recovery. Many networks are 490 now designed to recover from failures quickly enough to only cause a 491 handful of traffic to be lost; recovery on the order of half a second 492 is not an uncommon goal. It should be obvious that removing 493 redundant links and devices from the network to reduce energy 494 consumption could adversely affect these goals. 496 5.3.1. An Example of Impact on Fast Recovery 498 In the network shown, assume R2 and its associated links are removed 499 from the topology in order to save energy. Rather than this second 500 path being available for immediate recovery on the failure of the 501 R1->R3 link, some process must be followed to bring R2 and its 502 associated links back up, reinject them into the topology, and 503 finally begin routing traffic across this path. 505 In many situations, only links and devices which are a "third point 506 of failure" may be acceptable as removal candidates in order to 507 conserve energy. 509 5.3.2. Requirements 511 Modifications to the control plane in order to remove links or nodes 512 to conserve energy SHOULD entail the ability to choose the level of 513 redundancy available after the network topology has been trimmed. 514 For instance, it might be acceptable in some situations to move to 515 single points of failure throughout the network, or in specific 516 sections of the network, for certain periods of time. In other 517 situations, it may only be acceptable to reduce the network to a 518 double point of failure, and never to a single point of failure. 520 The complete removal of nodes or links from the network topology has 521 several impacts on the control plane which must be considered. In 522 these cases, the control plane must: 524 o Modify the network topology so removed links or devices are not 525 used to forward traffic 527 o Remember that such links exist, possibly including the neighbors 528 and destinations reachable through those links or devices 530 5.4. Introducing Jitter Through Microsleeps 532 One proposed mechanism to reduce energy usage in a network is to 533 sleep links or devices for very short periods of time, called 534 microsleeps. For instance, if a particular link is only used at 50% 535 of the actual available bandwidth, it should be possible to place the 536 link in some lower power state for 50% of the time, thus reducing 537 energy usage by something percentage. 539 Such schemes introduce delay and jitter into the network path 540 directly; if a packet arrives while the link to the next hop, or the 541 next hop itself, is in a reduced energy state, the packet must wait 542 until the link or next hop device enter a normal operational mode 543 before it can be forwarded. Most of the time the proposed sleep 544 states are so small as to be presumably inconsequential on overall 545 packet delay, but multiple packets crossing a series of links, each 546 encountering different links in different states, could take very 547 different amounts of time to pass along the path. 549 One possible way to resolve this somewhat random accrual of delays on 550 a per packet basis is to coordinate these sleep states such that 551 packets accepted at the entry of the network are consistently passed 552 through the network when all links and devices are in a normal 553 operating mode, and simply delaying all packets at the entry point 554 into the network while the devices in the network are in some energy 555 reduced state. This solution still introduces some amount of jitter; 556 some packets will be delayed by the sleep state at the edge of the 557 network, while others will not. This solution also requires 558 coordinated timers at the speed of forwarding itself to effectively 559 control the sleep and wake cycles of the network. 561 5.4.1. An Example of Microsleeps to Reduce Energy Usage 563 In the example network, assume the bandwidth utilization along the 564 path R1->R2->R4->R5 is 50% of the actual available bandwidth along 565 this path. It is possible to consider a scheme where R1->R2, R2, 566 R2->R4, and R4->R5 are all put into some energy reduced operational 567 mode 50% of the time, since packets are only available to send 50% of 568 the time. A packet entering at R1 may encounter a short delay at 569 R1->R2, at R2->R4, and at R4->R5, or it might not. Even if these 570 delays are very small, say 200ms at each hop, the accumulated delay 571 through the network due to sleep states may be 0ms (all links and 572 devices awake) or 600ms (all links and devices asleep) as the packet 573 passes through the network. 575 As network paths lengthen to more realistic path lengths in real 576 deployments, the jitter introduced varies more widely, which could 577 cause problems for the operation of a number of applications. 579 5.4.2. Requirements 581 Protocol designers SHOULD analyze the impact of accumulated jitter 582 when proposing mechanisms that rely on microsleeps in either 583 equipment or links. This analysis SHOULD include both worst case and 584 best case scenarios, as well as an analysis of how coordinated clocks 585 are to be handled in the case of coordinated sleep states. 587 5.5. Other Operational Aspects 589 Modification of the network topology in order to save energy needs to 590 consider the operational needs of the network as well as application 591 requirements. Change management, operational downtime, and business 592 usage of the network need to be considered when determining which 593 links and nodes should be placed into a low energy state. Energy 594 provisions have to be assigned and changed for nodes and links, 595 optimally according to network usage profiles over the time of day. 597 Control plane protocol operation, in terms of operational efficiency 598 on the wire, also needs to be considered when modifying protocol 599 parameters. Any changes that negatively impact the operation of the 600 protocol, in terms of the amount of traffic, the size of routing 601 information transmitted over the network, and interaction with 602 network management operations need to be carefully analyzed for 603 scaling and operational implications. 605 5.5.1. An Example of Operational Impact 607 Time of day is an important consideration in business operations. 608 During normal operational hours, the network needs to be fully 609 available, including all available redundancy and bandwidth. During 610 holidays, night hours, and other times when a campus might not be 611 used, or when there are lower traffic and resiliency demands on the 612 network, network elements can be removed to reduce energy usage. 614 5.5.2. Requirements 616 Protocol designers SHOULD analyze operational requirements, such as 617 time of day and network traffic load considerations, and explain how 618 proposed protocols or modifications to protocols will interact with 619 these types of requirements. Protocols designers SHOULD analyze 620 increases in network traffic and the operational efficiency impact of 621 proposed changes or protocols. 623 6. Security Considerations 625 None. 627 7. Acknowledgements 629 The authors of this document would like to acknowledge the 630 suggestions and ideas provided by Sujata Banerjee, Puneet Sharma and 631 Dirk Von Hugo. 633 8. References 635 8.1. Normative References 637 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 638 Requirement Levels", BCP 14, RFC 2119, March 1997. 640 8.2. Informative References 642 [3GPP] 3GPP, "3GPP TR 25-927 Solutions for energy saving within 643 UTRA Node B", 2011, 644 . 646 [EMAN] IETF, "Energy Management Working Group Charter", 2012, 647 . 649 [Krioukov] 650 Krioukov, D., "On Compact Routing for the Internet", 2007, 651 . 654 [MANET] IETF, "Mobile Ad Hoc Networks Charter", 2012, 655 . 657 [RFC5714] Shand, M. and S. Bryant, "IP Fast Reroute Framework", 658 RFC 5714, January 2010. 660 [ROLL] IETF, "Routing Over Lowe power and Lossy networks 661 Charter", 2012, 662 . 664 Appendix A. Change Log 666 A.1. Changes between the -00 and -01 versions. 668 o Updated authors' contact information. 670 o Modified some of the rfc2119 keywords. 672 A.2. Changes between the -01 and -02 versions. 674 o Updated authors' contact information. 676 A.3. Changes between the -02 and -03 versions. 678 o Updated authors' contact information. 680 Authors' Addresses 682 Alvaro Retana 683 Cisco Systems, Inc. 684 7025 Kit Creek Rd. 685 Raleigh, NC 27709 686 USA 688 Email: aretana@cisco.com 690 Russ White 691 Ericsson 693 Email: russw@riw.us 695 Manuel Paul 696 Deutsche Telekom AG 697 Winterfeldtstr. 21-27 698 Berlin 10781 699 Germany 701 Email: Manuel.Paul@telekom.de