idnits 2.17.1 draft-behringer-complexity-framework-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 15, 2012) is 4210 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC3439' is defined on line 283, but no explicit reference was found in the text Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Research Task Force M. Behringer 3 Internet-Draft Cisco 4 Intended status: Informational G. Huston 5 Expires: April 18, 2013 Asia Pacific Network Information 6 Centre 7 October 15, 2012 9 A Framework for Defining Network Complexity 10 draft-behringer-complexity-framework-00.txt 12 Abstract 14 Complexity is a widely used parameter in network design, yet there is 15 no generally accepted definition of the term. Complexity metrics 16 exist in a wide range of research, but most of them address only a 17 particular aspect of a network, for example the complexity of a graph 18 or software. There is a desire to define the complexity of a network 19 as a whole, as deployed today to provide Internet services. This 20 document provides a framework to guide research on the topic of 21 network complexity. 23 Status of this Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at http://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on April 18, 2013. 40 Copyright Notice 42 Copyright (c) 2012 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (http://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 55 2. Current Understanding of Network Complexity . . . . . . . . . . 3 56 2.1. The Behavior of a Complex Network . . . . . . . . . . . . . 3 57 2.2. Robust Yet Fragile . . . . . . . . . . . . . . . . . . . . 4 58 2.3. The Complexity Cube . . . . . . . . . . . . . . . . . . . . 4 59 3. Towards Defining Network Complexity . . . . . . . . . . . . . . 4 60 3.1. General Observations . . . . . . . . . . . . . . . . . . . 4 61 3.2. The Problem Space . . . . . . . . . . . . . . . . . . . . . 4 62 3.3. Technical Debt . . . . . . . . . . . . . . . . . . . . . . 5 63 4. Possible Directions of Research . . . . . . . . . . . . . . . . 5 64 4.1. Definitions and Metrics . . . . . . . . . . . . . . . . . . 6 65 4.2. Comparative Analysis . . . . . . . . . . . . . . . . . . . 6 66 4.3. Containment, Control or Reduction of Complexity . . . . . . 6 67 4.4. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . 6 68 5. Security Considerations . . . . . . . . . . . . . . . . . . . . 7 69 6. Informative References . . . . . . . . . . . . . . . . . . . . 7 70 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . . 7 71 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 7 73 1. Introduction 75 During the design phase of a network complexity plays a key role. 76 Network designers generally seek to find the simplest design that 77 fulfils a set of requirements. As no objective definition of network 78 complexity exists, subjective measures are used to come to a 79 conclusion. The resulting diverging views on what constitutes 80 complexity subsequently lead to conflicts in design teams. While 81 most people would agree that complexity is an important factor in 82 network design, today's design decisions are made based on a rough 83 estimation of the network's complexity, rather than a solid 84 understanding. 86 The goal of this document is to define a framework for network 87 complexity research. This framework describes related research and 88 current understanding of the topic, as well as outlining some ways 89 research could be taken forward. Specifically, contributions are 90 invited in all of the areas mentioned. 92 Many references to existing research in the area of network 93 complexity are listed on the Network Complexity Wiki [wiki]. That 94 wiki also contains background information on previous meetings on the 95 subject, previous research, etc. 97 2. Current Understanding of Network Complexity 99 2.1. The Behavior of a Complex Network 101 While there is no generally accepted definition of network 102 complexity, there is some understanding of the behavior of a complex 103 network. It has some or all of the following properties: 104 o Self-Organization: A network runs some protocols and processes 105 without external control; for example a routing process, failover 106 mechanisms, etc. The interaction of those mechanisms can lead to 107 a complex behaviour. 108 o Un-predictability: In a complex network, the effect of a local 109 change on the behaviour of the global network may be 110 unpredictable. 111 o Emergence: A network has an emergent property if a small local 112 change produces a large scale, seemingly unrelated state or 113 result. 114 o Non-linearity: An input into the network produces a non-linear 115 result. 116 o Fragility: A small local input can break the entire system. 118 2.2. Robust Yet Fragile 120 Networks typically follow the "robust yet fragile" paradigm: They are 121 designed to be robust against a set of failures, yet they are very 122 vulnerable to other failures. Doyle [Doyle] explains the concept 123 with an example: The Internet is robust against single component 124 failure, but fragile to targeted attacks. The "robust yet fragile" 125 property also touches on the fact that all network designs are 126 necessarily making trade-offs between different design goals. The 127 simplest one is articulated in "The Twelve Networking Truths" RFC1925 128 [RFC1925]: "Good, Fast, Cheap: Pick any two (you can't have all 129 three)." In real network design, trade-offs between many aspects 130 have to be made. 132 2.3. The Complexity Cube 134 Complex tasks on a network can be done in different components of the 135 network. For example, routing can be controlled by central 136 algorithms, and the result distributed (e.g., OpenFlow model); the 137 routing algorithm can also run completely distributed (e.g., routing 138 protocols such as OSPF or ISIS), or a human operator could calculate 139 routing tables and statically configure routing. Behringer 140 [Behringer] defines these three axes of complexity as a "complexity 141 cube" with three axes: Network elements, central systems, and human 142 operators. While different functions can be shifted between these 143 axes of the network, the overall complexity may change. 145 3. Towards Defining Network Complexity 147 3.1. General Observations 149 Any analysis of practical network complexity must take a wide range 150 of parameters into account, also parameters which are hard to 151 measure, for example the human element. Human error constitutes in 152 most cases of critical outages the trigger condition; therefore any 153 analysis ignoring the human factor cannot address the full picture. 154 [insert a reference that 70%(?) of critical outages have a human 155 origin] 157 3.2. The Problem Space 159 When discussing network complexity, a large number of influencing 160 factors have to be taken into account to arrive at a full picture, 161 for example: 162 o State in the network: Contains the network elements, such as 163 routers, switches (with their OS, including protocols), lines, 164 central systems, etc. The number and algorithmical complexity of 165 the protocols on network devices for example. 166 o Human operators: Complexity manifests itself often by a network 167 that is not completely understood by human operators. Human error 168 is a primary source for catastrophic failures, and therefore must 169 be taken into account. 170 o Classes / templates: Rather than counting the number of lines in a 171 configuration, or the number of hardware elements, more important 172 is the number of classes from which those can be derived. In 173 other words, it is probably less complex to have 1000 interfaces 174 which are identically configured than 5 that are completely 175 different configured. 176 o Dependencies and interactions: The number of dependencies between 177 elements, as well as the interactions between them has influence 178 on the complexity of the network. 179 o TCO (Total cost of ownership): TCO could be a good metric for 180 network complexity, if the TCO calculation takes into accont all 181 influencing factors, for example training time for staff to be 182 able to maintain a network. 183 o Benchmark Unit Cost is a related metric that indicates the cost of 184 operating a certain component. If calculated well, it reflects at 185 least parts of the complexity of this component. Therefore, the 186 way TCO or BUC are calculated can help to derive a complexity 187 metric. 188 o Churn / rate of change: The change rate in a network itself can 189 contribute to complexity, especially if a number of components of 190 the overall network interact. 192 3.3. Technical Debt 194 Many changes in a network are made with a dependency on the existing 195 network. Often, a suboptimal decision is made because the optimal 196 decision is hard or impossible to realise at the time. Over time, 197 the number of suboptimal changes in themselves cause significant 198 complexity, which would not have been there had the optimal solution 199 been implemented. 201 The term "technical debt" refers to the accumulated complexity of 202 sub-optimal changes over time. As with financial debt, the idea is 203 that also technical debt must be repaid one day by cleaning up the 204 network or software. 206 4. Possible Directions of Research 208 The problem space of network complexity is very large, as many 209 influencing factors contribute to the overall complexity of a 210 network. The following sections outline areas for research. 212 4.1. Definitions and Metrics 214 In the context of general network operations, as well as in the 215 context of standardisation of protocols a common definition of the 216 term "network complexity" would be useful. It would also be useful 217 to have a metric for the complexity of a protocol or network design, 218 such that two candidate proposals can be objectively compared.This 219 could happen in a bottom-up approach, where metrics for parts of a 220 network are combined to an overall metric; or in a top-down approach 221 where a global metric or vector of metrics is broken down into the 222 components of a network. 224 4.2. Comparative Analysis 226 In the foreseeable future it is unlikely to define a single, 227 objective metric that includes all the relevant aspects of 228 complexity. In the absence of such a global metric, a comparative 229 approach could be easier. 231 For example, if two network architectures are compared against each 232 other, it may be possible to ignore the network layout and device 233 hardware if those are the same in both cases. In such specific 234 comparisons it should be considerably easier to find valid metrics, 235 and to compare the approaches objectively. 237 4.3. Containment, Control or Reduction of Complexity 239 In some disciplines such as software engineering, complexity is 240 relatively well understood, as well as metrics and methods to reduce 241 it. Such approaches can be applied in the networking industry to 242 achieve the same result. 244 4.4. Use Cases 246 While it is hard to define a universal set of metrics for network 247 complexity, special use cases should be documented to serve as 248 examples, and to stimulate discussion. Such use cases could come out 249 of different areas: 250 o Documented examples of "catastrophic failure": While the cause of 251 complexity is hard to understand, the result may be a catastropic 252 outage, which can be reverse-engineered to understand the root 253 causes. The knowledge from this process may give insight into 254 root causes of complexity. 255 o A detailed complexity analysis of a particular network or 256 protocol. Even if this analysis may not be complete or fully 257 objective, it would be useful to learn about different approaches. 259 o Analysis of existing networks, protocols or components from an 260 insider point of view, discussing in detail where the perceived 261 complexity in the set-up is, and how this could be changed to 262 reduce complexity. 263 o Work in related areas, for example a detailled analysis of the 264 total cost of ownership, and how this could be mapped into a 265 complexity metric. 267 5. Security Considerations 269 This document does not discuss any specific security considerations. 271 6. Informative References 273 [Behringer] 274 Behringer, M., "Classifying Network Complexity", 275 Proceedings of the ACM Re-Arch'09, December 2009. 277 [Doyle] Doyle, J., "The 'robust yet fragile' nature of the 278 Internet", PNAS vol. 102 no. 41 14497-14502, October 2005. 280 [RFC1925] Callon, R., "The Twelve Networking Truths", RFC 1925, 281 April 1996. 283 [RFC3439] Bush, R. and D. Meyer, "Some Internet Architectural 284 Guidelines and Philosophy", RFC 3439, December 2002. 286 [wiki] "Network Complexity Wiki", 287 . 289 Appendix A. Acknowledgements 291 This document is the result of many meetings and discussions, with 292 too many people to provide a full list here. however, key 293 contributions have been made by: John Doyle, Jon Crowcroft, Mark 294 Handley, Fred Baker, Paul Vixie, Lars Eggert, Bob Briscoe, Keith 295 Jones, Bruno Klauser, Steve Youell, Joel Obstfeld. 297 Authors' Addresses 299 Michael H. Behringer 300 Cisco 302 Email: mbehring@cisco.com 304 Geoff Huston 305 Asia Pacific Network Information Centre 307 Email: gih@apnic.net