idnits 2.17.1 draft-davies-fdr-reqs-01.txt: -(183): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(188): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(189): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(280): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(384): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(462): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(702): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(744): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(784): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(809): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(843): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(856): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(875): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(892): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(896): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1022): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1023): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1072): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1073): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1074): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1110): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1113): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1161): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1381): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1395): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1443): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1465): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1714): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1766): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1851): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(1882): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(2041): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(2600): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(2675): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(2684): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(2687): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(2688): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(2729): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(2745): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(2781): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Found some kind of copyright notice around line 41 but it does not match any copyright boilerplate known by this tool. Expected boilerplate is as follows today (2024-04-20) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. == There are 158 instances of lines with non-ascii characters in the document. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 55 longer pages, the longest (page 2) being 59 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Abstract section. ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == Line 1321 has weird spacing: '...-homing and c...' == Line 1589 has weird spacing: '... to descri...' == Line 1809 has weird spacing: '... to the requi...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '1' is defined on line 2661, but no explicit reference was found in the text == Unused Reference: '2' is defined on line 2664, but no explicit reference was found in the text == Unused Reference: '3' is defined on line 2668, but no explicit reference was found in the text == Unused Reference: '5' is defined on line 2675, but no explicit reference was found in the text == Unused Reference: '6' is defined on line 2678, but no explicit reference was found in the text == Unused Reference: '7' is defined on line 2683, but no explicit reference was found in the text == Unused Reference: '9' is defined on line 2691, but no explicit reference was found in the text == Unused Reference: '17' is defined on line 2724, but no explicit reference was found in the text == Unused Reference: '21' is defined on line 2745, but no explicit reference was found in the text ** Downref: Normative reference to an Unknown state RFC: RFC 1102 (ref. '1') ** Downref: Normative reference to an Unknown state RFC: RFC 1125 (ref. '2') ** Downref: Normative reference to an Historic RFC: RFC 1478 (ref. '3') ** Downref: Normative reference to an Unknown state RFC: RFC 1126 (ref. '4') -- Possible downref: Non-RFC (?) normative reference: ref. '5' -- Possible downref: Non-RFC (?) normative reference: ref. '6' ** Downref: Normative reference to an Informational RFC: RFC 1992 (ref. '7') ** Downref: Normative reference to an Informational RFC: RFC 1753 (ref. '8') -- Possible downref: Non-RFC (?) normative reference: ref. '9' -- Possible downref: Non-RFC (?) normative reference: ref. '10' -- Possible downref: Non-RFC (?) normative reference: ref. '11' -- Possible downref: Non-RFC (?) normative reference: ref. '12' -- Possible downref: Non-RFC (?) normative reference: ref. '13' -- Possible downref: Normative reference to a draft: ref. '14' -- Possible downref: Normative reference to a draft: ref. '15' ** Obsolete normative reference: RFC 2547 (ref. '16') (Obsoleted by RFC 4364) ** Downref: Normative reference to an Informational RFC: RFC 1287 (ref. '17') -- Possible downref: Normative reference to a draft: ref. '18' -- Possible downref: Non-RFC (?) normative reference: ref. '19' == Outdated reference: A later version (-01) exists of draft-mcpherson-bgp-route-oscillation-00 -- Possible downref: Normative reference to a draft: ref. '20' ** Downref: Normative reference to an Informational RFC: RFC 2993 (ref. '21') -- Possible downref: Normative reference to a draft: ref. '22' == Outdated reference: A later version (-06) exists of draft-ietf-bgmp-spec-02 ** Downref: Normative reference to an Historic draft: draft-ietf-bgmp-spec (ref. '23') == Outdated reference: A later version (-09) exists of draft-ietf-mpls-generalized-signaling-01 -- Possible downref: Non-RFC (?) normative reference: ref. '26' -- Possible downref: Non-RFC (?) normative reference: ref. '27' -- Possible downref: Non-RFC (?) normative reference: ref. '28' ** Downref: Normative reference to an Informational RFC: RFC 2791 (ref. '29') -- Possible downref: Non-RFC (?) normative reference: ref. '30' == Outdated reference: A later version (-02) exists of draft-many-inference-srlg-00 -- Possible downref: Normative reference to a draft: ref. '31' -- Possible downref: Non-RFC (?) normative reference: ref. '32' -- Possible downref: Normative reference to a draft: ref. '33' -- Possible downref: Non-RFC (?) normative reference: ref. '34' -- Possible downref: Non-RFC (?) normative reference: ref. '36' ** Obsolete normative reference: RFC 2858 (ref. '37') (Obsoleted by RFC 4760) -- No information found for draft-berkowitz-multirqmt - is the name correct? -- Possible downref: Normative reference to a draft: ref. '38' ** Downref: Normative reference to an Informational RFC: RFC 2071 (ref. '39') ** Downref: Normative reference to an Informational RFC: RFC 2072 (ref. '40') Summary: 22 errors (**), 0 flaws (~~), 20 warnings (==), 26 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 IRTF Routing Research Elwyn Davies, Avri Doria, Howard Berkowitz, 3 Internet Draft Dmitri Krioukov, Nortel Networks 4 Malin Carlzon, SUNET 5 Anders Bergsten, Olle Pers, Yong Jiang, 6 Telia Research 7 Lenka Carr Motyckova, Pierre Fransson, Olov Schelen 8 Lulea University of Technology 9 Tove Madsen, Utfors Bredband AB 11 Document: draft-davies-fdr-reqs-01.txt July, 2001 13 Future Domain Routing Requirements 14 16 Status of this Memo 18 This document is an Internet Draft and is in full conformance with 19 all provisions of Section 10 of RFC2026. Internet Drafts are working 20 documents of the Internet Engineering Task Force (IETF), its Areas, 21 and its Working Groups. Note that other groups may also distribute 22 working documents as Internet Drafts. 24 Internet Drafts are draft documents valid for a maximum of six 25 months. Internet Drafts may be updated, replaced, or obsoleted by 26 other documents at any time. It is not appropriate to use Internet 27 Drafts as reference material or to cite them other than as a "working 28 draft" or "work in progress". 30 The list of current Internet-Drafts can be accessed at 31 http://www.ietf.org/ietf/1id-abstracts.txt 33 The list of Internet-Draft Shadow Directories can be accessed at 34 http://www.ietf.org/shadow.html. 36 Discussion and suggestions for improvement are requested. This 37 document will expire before September, 2001. Distribution of this 38 draft is unlimited. 40 Copyright Notice 41 Copyright (C) The Internet Society (2001). All Rights Reserved. 42 Abstract 44 This document sets out a set of requirements which we believe are 45 desirable for the future routing architecture and routing protocols 46 of a successful Internet. This first version is, of necessity, 47 incomplete. It is hoped that this document will be useful as a 48 catalyst to the work that needs to be done in both the IRTF and the 49 IETF. 51 Davies et al 1 52 CONTENTS 54 1. Introduction...............................................4 55 1.1 Background............................................5 56 1.2 Goals.................................................6 57 2. Historical Perspective.....................................9 58 2.1 The legacy of RFC1126.................................9 59 2.2 Nimrod Requirements..................................20 60 2.3 PNNI.................................................21 61 2.4 Recent Research Work.................................22 62 3. Existing problems of BGP and the current EGP/IGP Architecture 63 .....................................................24 64 3.1 BGP and Auto-aggregation.............................24 65 3.2 Convergence and Recovery Issues......................24 66 3.3 Non-locality of Effects of Instability and Misconfiguration 67 ..................................................25 68 3.4 Multihoming Issues...................................25 69 3.5 AS-number exhaustion.................................26 70 3.6 Partitioned AS�s.....................................26 71 3.7 Load Sharing.........................................27 72 3.8 Hold down issues.....................................27 73 3.9 Interaction between Inter domain routing and intra domain 74 routing...........................................27 75 3.10 Policy Issues.....................................28 76 3.11 Security Issues...................................29 77 3.12 Support of MPLS and VPNS..........................29 78 3.13 IPv4 / IPv6 Ships in the Night....................29 79 3.14 Existing Tools to Support Effective Deployment of Inter- 80 Domain Routing....................................30 81 4. Expected Users............................................32 82 4.1 Organisations........................................32 83 4.2 Human Users..........................................34 84 5. Mandated Constraints......................................34 85 5.1 The Federated Environment............................35 86 5.2 Working with different sorts of network..............35 87 5.3 Delivering Diversity.................................35 88 5.4 When will the new solution be required?..............36 89 6. Assumptions...............................................36 90 7. Functional Requirements...................................38 91 7.1 Topology.............................................38 92 7.2 Distribution.........................................39 93 7.3 Addressing...........................................40 94 7.4 Management Requirements..............................42 95 7.5 Mathematical Provability.............................42 96 7.6 Traffic Engineering..................................42 97 7.7 Support for NATs and other Mid-boxes.................43 98 7.8 Statistics support...................................43 99 8. Performance Requirements..................................44 100 9. Backwards compatibility (cutover) and Maintainability.....44 101 10. Security Requirements....................................44 102 11. Open Issues..............................................46 103 11.1 System Modeling...................................46 104 11.2 Advantages and Disadvantages of having the same protocols 105 for EGP and IGP...................................46 107 Davies, et al Expires: January 2002 2 108 11.3 Introduction of new control mechanisms............49 109 11.4 Robustness........................................49 110 11.5 VPN Support.......................................50 111 11.6 End to End Reliability............................50 112 12. Acknowledgements.........................................50 113 13. References...............................................51 114 14. Author's Addresses.......................................54 116 Davies, et al Expires: January 2002 3 117 1. Introduction 119 It is generally accepted that there are major shortcomings in the 120 inter-domain routing of the Internet today and that these may result 121 in meltdown within an unspecified period of time. Remedying these 122 shortcomings will require extensive research to tie down the exact 123 failure modes that lead to these shortcomings and identify the best 124 techniques to remedy the situation. 126 Various developments in the nature and quality of the services that 127 users want from the Internet are difficult to provide within the 128 current framework as they impose requirements never foreseen by the 129 original architects of the Internet routing system. 131 The potential advent of IPv6 and the application of IP mechanisms to 132 private commercial networks offering specific service guarantees as 133 an improvement over the best efforts services of the Public Internet 134 epitomize the kind of radical changes which have to be accommodated: 135 Major changes to the inter-domain routing system are inevitable if it 136 is to provide an efficient underpinning for the radically changed and 137 increasingly, commercially-based networks which rely on the IP 138 protocol suite. 140 Although inter-domain routing is seen as the major source of 141 problems, the interactions with intra-domain routing and the 142 constraints that confining changes to the inter-domain arena would 143 impose, means that we should consider the whole area of routing as an 144 integrated system. This is done for 2 reasons: 146 - Requirements should not presuppose the solution. A continued 147 commitment to the current definitions and split between inter- 148 domain and intra-domain routing would constitute such a 149 presupposition. Therefore the name Future Domain Routing(FDR) is 150 being used in this document, 152 - As an acknowledgement of how intertwined inter-domain and intra- 153 domain routing are within today�s routing architecture. 155 We are aware that using the term Domain Routing is already fraught 156 with danger because of possible misinterpretation due to prior usage: 157 The meaning of Domain Routing will be developed implicitly throughout 158 the document, but a little advance explicit definition of the word 159 �domain� is required, as well as some expansion on the scope of 160 �routing�. 162 This document uses domain in a very broad sense to mean any 163 collection of systems or domains which come under a common authority 164 that determines the attributes defining, and the policies controlling 165 that collection. The use of domain in this context is very similar to 166 the concept of Region that was put forth by John Wroclawski in his 167 Metanet model [10]. The idea includes the notion that within a domain 168 certain system attributes will characterize the behavior of the 169 systems in the domain and that there will be borders between domains. 170 The idea of domain presented does not inherently presuppose that the 172 Davies, et al Expires: January 2002 4 173 identifying behaviors between two domains are to be the same. Nor 174 does it presuppose anything about the hierarchical nature of domains. 175 Finally it does not place restrictions on the nature of the 176 attributes that might be used to determine membership in a domain. 177 Since today�s routing domains are a subset of all domains as 178 conceived of in this document, there has been no attempt to create a 179 new term. 181 Current practice stresses the need to separate the concerns of the 182 control plane in a router and the forwarding plane: This document 183 will follow this practice, but we still use the term �routing� as a 184 global portmanteau to cover all aspects of the system. 186 This draft makes a start on the process of improving domain routing 187 in Section 2 by revisiting the requirements for a future routing 188 system which were last documented in RFC1126 - �Goals and Functional 189 Requirements for Inter-Autonomous System Routing� [4] as a precursor 190 to the design of BGP in 1989. This section also looks at some other 191 work that has been carried out since RFC1126 was published in order 192 to flesh out the historical perspective. Some of the requirements 193 derive from the problems that are currently being experienced in the 194 Internet today. These will be discussed in Section 3. The 195 environment in which the future domain routing system will have to 196 work is covered in Sections 4 - 6. Specific requirements for a 197 future Domain routing system are discussed in Sections 7 - 10. 198 Inevitably this document is incomplete: Some known Open Issues are 199 discussed in Section 11. 201 1.1 Background 203 Today�s Internet uses an addressing and routing structure that has 204 developed in an ad hoc, more or less upwards-compatible fashion. It 205 has progressed from handling a single domain, non-commercial Internet 206 to a solution that is just about controlling today�s multi-domain, 207 federated, combination commercial and not-for-profit Internet. The 208 result is not ideal, particularly as regards inter-domain routing 209 mechanisms that have to implement a host of domain specific routing 210 policies for competing, communicating domains, but it does a pretty 211 fair job at its primary goal of providing any-to-any connectivity to 212 many millions of computers. 214 Based on a large body of anecdotal evidence, but also on experimental 215 evidence [11] and analytic work on the stability of BGP under certain 216 policy specifications [12], the main Internet inter-domain routing 217 protocol, BGP4, appears to have a number of problems that need to be 218 resolved. Additionally, the hierarchical nature of the inter-domain 219 routing problem appears to be changing as the connectivity between 220 domains becomes increasingly meshed [13] which alters some of the 221 scaling and structuring assumptions on which BGP4 is built. Patches 222 and fix-ups may relieve some of these problems but others may require 223 a new architecture and new protocols. The starting point of this work 224 is to step back from the current state, examine how the Internet 225 might develop in the future and derive a new set of requirements for 226 a routing architecture from this work. 228 Davies, et al Expires: January 2002 5 229 The development of the Internet is likely to be driven by a number of 230 changes that will affect the organization and the usage of the 231 network, including: 232 - Ongoing evolution of the commercial relationships between 233 (connectivity) service providers, leading to changes in the way in 234 which peering between providers is organised and the way in which 235 transit traffic is routed 236 - Requirements for traffic engineering within and between domains 237 including coping with multiple paths between domains 238 - Potential addition of a second IP addressing technique through 239 IPv6. 240 - Evolution of the end to end principle to deal with the expanded 241 role of the Internet as discussed in [32]. This paper discusses 242 the possibility that the range of new requirements, especially the 243 social and techno-political ones, that are being placed on the 244 future Internet may compromise the Internet�s original design 245 principles. This might cause the Internet to lose some of its key 246 features, in particular its ability to support new and 247 unanticipated applications. The discussion is linked to the rise 248 of new stakeholders in the Internet, especially ISPs; new 249 government interests; the changing motivations of the ever growing 250 user base; and the tension between the demand for trustworthy 251 overall operation and the inability to trust the behaviour of 252 individual users. 253 - Incorporation of alternative forwarding techniques such as the 254 pipes supplied by the Generalised MPLS environment[25]. 255 - Integrating additional constraints into route determination from 256 interactions with other layers (e.g. Shared Risk Link Groups [31]) 257 - Support for alternative and multiple routing techniques that are 258 better suited to delivering types of content organised other than 259 into IP addressed packets. 261 Philosophically, the Internet has the mission of transferring 262 information from one place to another. Conceptually, this 263 information is rarely organised into conveniently sized, IP-addressed 264 packets and the FDR needs to consider how the information (content) 265 to be carried is identified, named and addressed. Routing techniques 266 can then be adapted to handle the expected types of content. 268 1.2 Goals 270 This section attempts to answer two questions: 271 - What are we trying to achieve in a new architecture? 272 - Why should the Internet community care? 274 There is a third question which needs to be answered as well, but 275 which, for the present, is mostly not explicitly discussed: 276 - How will we know when we have succeeded? 278 1.2.1 Providing a Routing System matched to Domain Organisation 280 Many of today�s routing problems are caused by a routing system which 281 is not well-matched to the organization and policies which it is 283 Davies, et al Expires: January 2002 6 284 trying to support. It is our goal to develop a routing architecture 285 where even domain organization which is not envisioned today can be 286 served by a routing architecture that matches its requirements. 288 We will know when this goal is achieved when the desired policies, 289 rules and organization can be mapped into the routing system in a 290 natural, consistent and simply understood way. 292 1.2.2 Supporting a range of different communication services 294 Today�s routing protocols only support a single data forwarding 295 service which is typically used to deliver a best efforts service in 296 the Public Internet. On the other hand, with, for example, DiffServ 297 it is possible to construct a number of different bit transport 298 services within the network. Using some of the PDBs which have been 299 discussed in the IETF, it should be possible to construct services 300 such as Virtual Wire [18] and Assured Rate [19]. 302 Providers today offer rudimentary promises about how traffic will be 303 handled in the network, for example delay and long term packet loss 304 guarantees, and this will probably be even more relevant in the 305 future. Communicating the service characteristics of paths in routing 306 protocols will be necessary in the near future, and it will be 307 necessary to be able to route packets according to their service 308 requirements. 310 Thus, a goal of this architecture is to allow for adequate 311 information about path service characteristics passed between domains 312 and consequently allow the delivery of bit transport services other 313 than the best-effort datagram connectivity service which is the 314 current common denominator. 316 1.2.3 Scaleable well beyond current predictable needs 318 Any proposed new FDR system should scale beyond the size and 319 performance we can foresee for the next ten years. The previous IDR 320 proposal has, with some massaging, held up for somewhat over ten 321 years in which time the Internet has grown far beyond the predictions 322 that were made in the requirements that were placed on it originally. 324 Unfortunately, we will only know if we have succeeded in this goal if 325 the improved system survives beyond its design lifetime without 326 serious massaging: Failure will be much easier to spot! 328 1.2.4 Supporting alternative forwarding mechanisms 330 With the advent of circuit based technologies (e.g., MPLS [24] used 331 for RSVP-TE or CR-LDP for traffic engineering, G-MPLS [25]) managed 332 by IP routers there are forwarding mechanisms other than the datagram 333 service that need to be supported by the routing architecture. 335 An explicit goal of this architecture is to support more forwarding 336 mechanisms than just hop-by-hop datagram forwarding driven by 337 globally unique IP addresses. 339 Davies, et al Expires: January 2002 7 340 1.2.5 Supporting separation of topology map and connectivity service 342 It is envisioned that an organization can support multiple services 343 on top of a single network. These services can, for example, be of 344 different quality, of different type of connectivity, or different 345 protocols (e.g. IPv4 and IPv6). For all these services there may be 346 common domain topology, even though the policies controlling the 347 routing of information might differ from service to service. 349 Thus, a goal with this architecture is to support separation between 350 creation of a domain (or organization) topology map and service 351 creation. 353 1.2.6 Achieving full/appropriate separation of concerns between 354 routing and forwarding 356 The architecture of a router is composed of two main separable parts; 357 control and forwarding. These components, while inter-dependent, 358 perform functions that are largely independent of each other. 359 Control (routing, signaling, and management) is typically done in 360 software while forwarding typically is done with specialized ASICs or 361 network processors. 363 The nature of an IP based network today is that control and data 364 protocols share the same network and forwarding regime. This may not 365 always be the case in future networks and we should be careful to 366 avoid building this sharing in as an assumption in the FDR. 368 A goal of this architecture is to support full separation of control 369 and forwarding, and to consider what additional concerns might be 370 properly considered separately (e.g. adjacency management). 372 1.2.7 Providing means of using different routing paradigms seamlessly 373 in different areas of the same network 375 A number of different routing paradigms have been used or researched 376 in addition to conventional shortest path hop-by-hop paradigm that is 377 the current mainstay of the Internet. In particular, differences in 378 underlying transport networks may mean that other kinds of routing 379 are more relevant, and the perceived need for traffic engineering 380 will certainly alter the routing chosen in various domains. 382 Implicitly, one of these routing paradigms should be the current 383 routing paradigm, so that the new paradigms will inter-operate in a 384 backwards compatible way with today�s system. This will facilitate a 385 migration strategy which avoids flag days. 387 1.2.8 Preventing denial of service and other security attacks 389 Part of the problem here is that the Internet offers a global, 390 unmoderated connectivity service. Existence of a route to a 392 Davies, et al Expires: January 2002 8 393 destination effectively implies that anybody who can get a packet 394 onto the network is entitled to use that route. Whilst there are 395 limitations to this generalization, there is a clear invitation to 396 denial of service attacks. A goal of the FDR system should be to 397 allow traffic to be specifically linked to whole or partial routes so 398 that a destination or link resources can be protected from 399 unauthorized use. 401 1.2.9 Providing provable convergence with verifiable policy 402 interaction 404 It has been shown both analytically by Griffin et al (see [12]) and 405 practically (see [20]) that BGP will not converge stably or is only 406 meta-stable (i.e. will not reconverge in the face of a single 407 failure) when certain types of policy constraint are applied to 408 categories of network topology. The addition of policy to the basic 409 distance vector algorithm invalidates the mathematical proofs that 410 applied to RIP and could be applied to a policy free BGP 411 implementation. 413 A goal of the FDR should be to achieve mathematically provable 414 convergence of the protocols used which may involve constraining the 415 topologies used, vetting the polices imposed to ensure that they are 416 compatible across domain boundaries and result in a globally 417 consistent policy set. 419 1.2.10 Providing robustness despite errors and failures 421 From time to time in the history of the Internet there have been 422 occurrences where people misconfiguring routers have destroyed global 423 connectivity. This should never be possible. 425 A goal of the FDR is to be robust to configuration errors and 426 failures. This should probably involve ensuring that the effects of 427 misconfiguration and failure can be confined to some suitable 428 locality of the failure or misconfiguration: This is not currently 429 the case as both misconfigurations and problems in any AS can, under 430 the right circumstances, have global effects across the entire 431 Internet. 433 1.2.11 Simplicity in management 435 With the policy work ([26], [27] and [28]) that has been done at IETF 436 there exists an architecture that standardizes and simplifies 437 management of QoS. This kind of simplicity is needed in a future 438 domain routing architecture and its protocols. 440 A goal of this architecture is to make configuration and management 441 of inter-domain routing as simple as possible. 443 2. Historical Perspective 445 2.1 The legacy of RFC1126 447 Davies, et al Expires: January 2002 9 448 RFC1126 outlined a set of requirements that were to guide the 449 development of BGP. While the network is definitely different then it 450 was in 1989, many of the same requirements remain. As a first step 451 in setting requirements for the future, we need to understand the 452 requirements that were originally set for the current protocols. And 453 in charting a future architecture we must first be sure to do no 454 harm. Which means a future domain routing has to support as its base 455 requirement, the level of function that is available today. 457 The following sections each relate to a requirement, or non 458 requirement listed in RFC1126. In fact the section names are direct 459 quotes from the document. The discussion of these requirements 460 covers the following areas: 462 Optionally, interpretation for today�s audience of the intent of the 463 requirement 465 Relevance: Is the requirement of RFC1126 still relevant, and 466 to what degree? Should it be understood 467 differently in today�s environment? 469 Current practice: How well is the requirement met by current 470 protocols and practice? 472 2.1.1 ��General Requirements�� 474 2.1.1.1 ��Route to Destination�� 476 Timely routing to all reachable destinations, including multihoming 477 and multicast. 479 Relevance: Valid, but requirements for multihoming need further 480 discussion and elucidation. The requirement should include 481 multiple source multicast routing. 483 Current practice: Multihoming is not efficient and the proposed 484 inter-domain multicast protocol BGMP is an add-on to BGP 485 following many of the same strategies but not integrated 486 into the BGP framework [23]. 488 2.1.1.2 ��Routing is Assured�� 490 This requires that a user be notified within a reasonable time period 491 of attempts, about inability to provide a service. 493 Relevance: Valid 495 Current practice: There are ICMP messages for this, but in many cases 496 they are not used, either because of fears about creating 497 message storms or uncertainty about whether the end system 498 can do anything useful with the resulting information. 500 Davies, et al Expires: January 2002 10 501 2.1.1.3 ��Large System�� 503 The architecture was designed to accommodate the growth of the 504 Internet. 506 Relevance: Valid. Properties of Internet topology might be an issue 507 for future scalability (topology varies from very sparse 508 to quite dense at present). Instead of setting growth in a 509 time-scale, indefinite growth should be accommodated. On 510 the other hand, such growth has to be accommodated without 511 making the protocols too expensive - trade-offs may be 512 necessary. 514 Current practice: Scalability of the protocols will not be sufficient 515 under the current rate of growth. There are problems with 516 BGP convergence for large dense topologies, problems with 517 routing information propagation between routers in transit 518 domain, limited support for hierarchy, etc. 520 2.1.1.4 ��Autonomous Operation�� 522 Relevance: Valid. There may need to be additional requirements for 523 adjusting policy decisions to the global functionality and 524 to avoid contradictory policies would decrease a 525 possibility of unstable routing behavior. 527 There should also be a separate requirement for handling 528 various degrees of trust in autonomous operation, ranging 529 from no trust (e.g., between separate ISPs) to very high 530 trust where the domains have a common goal of optimizing 531 their mutual policies. 533 Policies for intra domain operations should in some cases 534 be revealed, using suitable abstractions, to a global 535 routing mechanism? 537 Current practice: Policy management is in the control of network 538 managers, as required, but there is little support for 539 handling policies at an abstract level for a domain. 540 Cooperating administrative entities decide about the 541 extent of cooperation independently. Lack of coordination 542 combined with global range of effects results in 543 occasional melt-down of Internet routing. 545 2.1.1.5 ��Distributed System�� 547 The routing environment is a distributed system. The distributed 548 routing environment supports redundancy and diversity of nodes and 549 links. Both data and operations are distributed. 551 Relevance: Valid. RFC1126 is very clear that we should not be using 552 centralized solutions, but maybe we need a requirement on 553 trade-offs between common knowledge and distribution 554 (e.g., to allow for uniform policy routing) (e.g., GSM 556 Davies, et al Expires: January 2002 11 557 systems are in a sense centralized (but with hierarchies) 558 and they work) This requirement should not rule out 559 certain solutions that are needed to meet other 560 requirements in this document. 562 Current practice: Routing is very distributed, but lacking abilities 563 to consider optimization over several hops or domains. 565 2.1.1.6 ��Provide A Credible Environment�� 567 Routing mechanism information must be integral and secure (credible 568 data, reliable operation). Security from unwanted 569 modification and influence is required. 571 Relevance: Valid. 573 Current practice: BGP provides a mechanism for authentication and 574 security. There are however security problems with 575 current practice. 577 2.1.1.7 ��Be A Managed Entity�� 579 Requires that a manager should get enough information on a state of 580 network so that (s)he could make informed decisions. 582 Relevance: The requirement is reasonable, but we might need to be 583 more specific on what information should be available, 584 e.g. to prevent routing oscillations. 586 Current practice: All policies are determined locally, where they may 587 appear reasonable but there is limited global coordination 588 through the routing policy databases operated by the 589 Internet registries (ARIN, RIPE, APNIC etc). Operators 590 are not required to register their policies; even when 591 policies are registered, it is difficult to check that the 592 actual policies in use match the declared policies and 593 therefore a manager cannot guarantee to make a globally 594 consistent decision. 596 2.1.1.8 ��Minimize Required Resources�� 598 Relevance: Valid, however, the paragraph states that assumptions on 599 significant upgrades shouldn�t be made. Although this is 600 reasonable, a new architecture should perhaps be prepared 601 to use upgrades when they occur. 603 Current practice: Most bandwidth is consumed by the exchange of the 604 NLRI. Usage of CPU depends on the stability of the 605 Internet. Both phenomena have a local nature, so there are 606 not scaling problems with bandwidth and CPU usage. 607 Instability of routing increases the consumption of 608 resources in any case. The number of networks in the 609 Internet dominates memory requirements - this is a scaling 610 problem. 612 Davies, et al Expires: January 2002 12 613 2.1.2 ��Functional Requirements�� 615 2.1.2.1 ��Route Synthesis Requirements�� 617 2.1.2.1.1 ��Route around failures dynamically�� 619 Relevance: Valid. Should perhaps be stronger. Only providing a best- 620 effort attempt may not be enough if real-time services are 621 to be provided for. Detections may need to be faster than 622 100ms to avoid being noticed by end-users. 624 Current practice: latency of fail-over is too high (minutes). 626 2.1.2.1.2 ��Provide loop free paths�� 628 Relevance: Valid. Loops should occur only with negligible probability 629 and duration. 631 Current practice: both link-state intra domain routing and BGP inter- 632 domain routing (if correctly configured) are forwarding- 633 loop free after having converged. However, convergence 634 time for BGP can be very long and poorly designed routing 635 policies may result in a number of BGP speakers engaging 636 in a cyclic pattern of advertisements and withdrawals 637 which never converges to a stable result [20]. 639 2.1.2.1.3 ��Know when a path or destination is unavailable�� 641 Relevance: Valid to some extent, but there is a trade-off between 642 aggregation and immediate knowledge of reachability. It 643 requires that routing tables contain enough information to 644 determine that the destination is unknown or a path cannot 645 be constructed to reach it. 647 Current practice: Knowledge about lost reachability propagates slowly 648 through the networks due to slow convergence for route 649 withdrawals. 651 2.1.2.1.4 ��Provide paths sensitive to administrative policies�� 653 Relevance: Valid. Policy control of routing is of increasingly 654 importance as the Internet has turned into business. 656 Current practice: Supported to some extent. Policies are only 657 possible to apply locally in an AS and not globally. At 658 least there is very small possibilities to affect policies 659 in other AS�s. Furthermore, only static policies are 660 supported; between static policies and `policies dependent 661 upon volatile events of great celerity` there should exist 662 events that routing should be aware of. Lastly, there is 663 no support for policies other than route-properties (such 664 as AS-origin, AS-path, destination prefix, MED-values 665 etc). 667 Davies, et al Expires: January 2002 13 668 2.1.2.1.5 ��Provide paths sensitive to user policies�� 670 Relevance: Valid to some extent, as they may conflict with the 671 policies of the network administrator. It is likely that 672 this requirement will be met by means of different bit 673 transport services offered by an operator, but at the cost 674 of adequate provisioning, authentication and policing when 675 utilizing the service. 677 Current practice: not supported in normal routing. Can be 678 accomplished to some extent with loose source routing, 679 resulting in inefficient forwarding in the routers. The 680 various attempts to introduce QoS (Integrated Services and 681 DiffServ) can also be seen as means to support this 682 requirement but they have met with limited success. 684 2.1.2.1.6 ��Provide paths which characterize user quality-of-service 685 requirements�� 687 Relevance: Valid to some extent, as they may conflict with the 688 policies of the operator. It is likely that this 689 requirement will be met by means of different bit 690 transport services offered by an operator, but at the cost 691 of adequate provisioning, authentication and policing when 692 utilizing the service. It has become clear that offering 693 to provide a particular QoS to any arbitrary destination 694 from a particular source is generally impossible: QoS 695 except in very �soft� forms such as overall long term 696 average packet delay, is generally associated with 697 connection oriented routing. 699 Current practice: Creating routes with specified QoS is not generally 700 possible at present. 702 2.1.2.1.7 ��Provide autonomy between inter- and intra-autonomous system 703 route synthesis�� 705 Relevance: Inter and intra domain routing should stay independent, 706 but one should notice that this to some extent contradicts 707 the previous three requirements. There is a trade-off 708 between abstraction and optimality. 710 Current practice: inter-domain routing is performed independently of 711 intra-domain routing. Intra-domain routing is, especially 712 in transit domains, very interrelated to inter-domain 713 routing. 715 2.1.2.2 ��Forwarding Requirements�� 717 2.1.2.2.1 ��Decouple inter- and intra-autonomous system forwarding 718 decisions�� 720 Relevance: Valid. 722 Davies, et al Expires: January 2002 14 723 Current practice: As explained in 2.1.2.1.7, intra-domain forwarding 724 in transit domains is codependent with inter-domain 725 forwarding decisions. 727 2.1.2.2.2 ��Do not forward datagrams deemed administratively 728 inappropriate�� 730 Relevance: Valid, and increasingly important in the context of 731 enforcing policies correctly expressed through routing 732 advertisements but flouted by rogue peers which send 733 traffic for which a route has not been advertised. On the 734 other hand, packets that have been misrouted due to 735 transient routing problems perhaps should be forwarded to 736 reach the destination, although along an unexpected path. 738 Current practice: at stub domains there is packet filtering, e.g., to 739 catch source address spoofing on outgoing traffic or to 740 filter out unwanted incoming traffic. Filtering can in 741 particular reject traffic (such as unauthorized transit 742 traffic) that has been sent to a domain even when it has 743 not advertised a route for such traffic on a given 744 interface. The growing class of �mid boxes� (e.g. NATs) 745 is quite likely to apply administrative rules that will 746 prevent forwarding of packets. Note that security 747 policies may deliberately hide administrative denials. In 748 the backbone, intentional packet dropping based on 749 policies is not common. 751 2.1.2.2.3 ��Do not forward datagrams to failed resources�� 753 Relevance: Unclear, although it is clearly desirable to minimise 754 waste of forwarding resources by discarding datagrams 755 which cannot be delivered at the earliest opportunity. 756 There is a trade-off between scalability and keeping track 757 of unreachable resources. Equipment closest to a failed 758 node has the highest motivation to keep track of failures 759 so that waste can be minimised. 761 Current practice: routing protocols use both internal adjacency 762 management sub-protocols (e.g. Hello protocols) and 763 information from equipment and lower layer link watchdogs 764 to keep track of failures in routers and connecting links. 765 Failures will eventually result in the routing protocol 766 reconfiguring the routing to avoid (if possible) a failed 767 resource, but this is generally very slow (30s or more). 768 In the meantime datagrams may well be forwarded to failed 769 resources. In general terms, end hosts and some non- 770 router midboxes do not participate in these notifications 771 and failures of such boxes will not affect the routing 772 system. 774 2.1.2.2.4 ��Forward datagram according to its characteristics�� 776 Davies, et al Expires: January 2002 15 777 Relevance: Valid. Is necessary in enabling differentiation in the 778 network, based on QoS, precedence, policy or security. 780 Current practice: ingress and egress filtering can be done on policy. 782 2.1.2.3 ��Information Requirements�� 784 2.1.2.3.1 ��Provide a distributed and descriptive information base�� 786 Relevance: Valid, however hierarchical IBs might provide more 787 possibilities. 789 Current practice: IBs are distributed, not sure whether they support 790 all provided routing functionality. 792 2.1.2.3.2 ��Determine resource availability�� 794 Relevance: Valid. It should be possible for resource availability 795 and levels of resource availability to be determined. 796 This prevents needing to discover unavailability through 797 failure. Resource location and discovery is arguably a 798 separate concern which could be addressed outside the core 799 routing requirements. 801 Current practice: Resource availability is predominantly handled 802 outside of the routing system. 804 2.1.2.3.3 ��Restrain transmission utilization�� 806 Relevance: Valid. However certain requirements in the control plane, 807 such as fast detection of faults may be worth consumption 808 of more resources. Similarly, simplicity of 809 implementation may make it cheaper to �back haul� traffic 810 to central locations to minimise the cost of routing if 811 bandwidth is cheaper than processing. 813 Current practice: BGP messages probably do not ordinarily consume 814 excessive resources, but might during erroneous 815 conditions. In the data plane, the near universal 816 adoption of shortest path protocols could be considered to 817 result in minimization of transmission utilization. 819 2.1.2.3.4 ��Allow limited information exchange�� 821 Relevance: Valid. But perhaps routing could be improved if certain 822 information could be globally available. 824 Current practice: Policies are used to determine which reachability 825 information that is exported. 827 2.1.2.4 ��Environmental Requirements�� 829 Davies, et al Expires: January 2002 16 830 2.1.2.4.1 ��Support a packet-switching environment� 832 Relevance: Valid but should not be exclusive. 834 Current practice: supported. 836 2.1.2.4.2 ��Accommodate a connection-less oriented user transport 837 service�� 839 Relevance: Valid, but should not be exclusive. 841 Current practice: accommodated. 843 2.1.2.4.3 ��Accommodate 10K autonomous systems and 100K networks�� 845 Relevance: No longer valid. Needs to be increased potentially 846 indefinitely. It is extremely difficult to foresee the 847 future size expansion of the Internet so that the utopian 848 solution would be to achieve an Internet whose 849 architecture is scale invariant. Regrettably, this may 850 not be achievable without introducing undesirable 851 complexity and a suitable trade off between complexity and 852 scalability is likely to be necessary. 854 Current Practice: Yes but perhaps reaching the limit. 856 2.1.2.4.4 ��Allow for arbitrary interconnection of autonomous systems�� 858 Relevance: Valid. However perhaps not all interconnections should be 859 accessible globally. 861 Current practice: BGP-4 allows for arbitrary interconnections. 863 2.1.2.5 ��General Objectives�� 865 2.1.2.5.1 ��Provide routing services in a timely manner�� 867 Relevance: Valid, as stated before. The more complex a service is the 868 longer it should be allowed to take, but the 869 implementation of services requiring (say) NP-complete 870 calculation should be avoided. 872 Current practice: More or less, with the exception of convergence and 873 fault robustness. 875 2.1.2.5.2 ��Minimize constraints on systems with limited resources�� 877 Relevance: Valid 879 Current practice: Systems with limited resources are typically stub 880 domains that advertise very little information. 882 Davies, et al Expires: January 2002 17 883 2.1.2.5.3 ��Minimize impact of dissimilarities between autonomous 884 systems�� 886 Relevance: Important. This requirement is critical to a future 887 architecture. In a domain routing environment where the 888 internal properties of domains may differ radically, it 889 will be important to be sure that these dissimilarities 890 are minimized at the borders. 892 Current: practice: for the most part this capability isn�t required 893 in today�s networks since the intra-domain attribute are 894 nearly identical to start with. 896 2.1.2.5.4 ��Accommodate the addressing schemes and protocol mechanisms 897 of the autonomous systems�� 899 Relevance: Important, probably more so than when RFC1126 was 900 originally developed because of the potential deployment 901 of IPv6, wider usage of MPLS and the increasing usage of 902 VPNs. 904 Current practice: Largely only one global addressing scheme is 905 supported in most autonomous systems. 907 2.1.2.5.5 ��Must be implementable by network vendors�Ƴ 909 Requirement: Valid, but note that what can be implemented today is 910 different from what was possible when RFC1126 was written: 911 FDR should not be unreasonably constrained by past 912 limitations. 914 Current practice: BGP was implemented. 916 2.1.3 ��Non-Goals� 918 RFC1126 also included a section discussing non goals. To what extent 919 are these still non goals? Does the fact that they were non-goals 920 adversely affect today�s IDR system? 922 2.1.3.1 ��Ubiquity�� 924 In a sense this �non-goal� has effectively been achieved by the 925 Internet and IP protocols. This requirement reflects a different 926 world view where there was serious competition for network protocols 927 which is really no longer the case. Ubiquitous deployment of inter- 928 domain routing in particular has been achieved and must not be undone 929 by any proposed FDR. On the other hand: 930 - ubiquitous connectivity cannot be reached in a policy sensitive 931 environment and should not be an aim, 932 - it must not be required that the same routing mechanisms are 933 used throughout provided that they can interoperate 934 appropriately 935 - the information needed to control routing in a part of the 936 network should not necessarily be ubiquitously available and it 938 Davies, et al Expires: January 2002 18 939 must be possible for an operator to hide commercially sensitive 940 information that is not needed outside a domain. 942 Relevance: De facto essential for a FDR but what is required is 943 ubiquity of the routing system rather than ubiquity of 944 connectivity. 946 Current practice: de facto ubiquity achieved. 948 2.1.3.2 ��Congestion control�� 950 Relevance: It is not clear if this non-goal was to be applied to 951 routing or forwarding. It is definitely a non-goal to 952 adapt the choice of route at transient congestion. 953 However, to add support for congestion avoidance (e.g., 954 ECN and ICMP messages) in the forwarding process would be 955 a useful addition. There is also extensive work going on 956 in traffic engineering which should result in congestion 957 avoidance through routing as well as in forwarding. 959 Current practice: Some ICMP messages (e.g. source quench) exist to 960 deal with congestion control but these are not generally 961 used as they either make the problem worse or there is no 962 mechanism to reflect the message into the application 963 which is providing the source. 965 2.1.3.3 ��Load splitting�� 967 Relevance: This should neither be a non-goal, nor an explicit goal. 968 It might be desirable in some cases. 970 Current practice: Can be implemented by exporting different prefixes 971 on different links, but this requires manual configuration 972 and does not consider actual load. 974 2.1.3.4 ��Maximizing the utilization of resources�� 976 Relevance: Valid. Cost-efficiency should be strived for, maximizing 977 resource utilization does not always lead to greatest 978 cost-efficiency. 980 Current practice: To the extent possible. 982 2.1.3.5 ��Schedule to deadline service�� 984 This non-goal was put in place to ensure that the IDR did not have to 985 meet real time deadline goals such as might apply to CBR services in 986 ATM. 988 Relevance: The hard form of deadline services is still a non-goal for 989 the FDR but overall delay bounds are much more of the 990 essence than was the case when RFC1126 was written. 992 Davies, et al Expires: January 2002 19 993 Current Practice: Service providers are now offering overall 994 probabilistic delay bounds on traffic contracts. To 995 implement these contracts there is a requirement for a 996 rather looser form of delay sensitive routing. 998 2.1.3.6 ��Non-interference policies of resource utilization� 1000 The requirement in RFC1126 is somewhat opaque, but appears to imply 1001 that what we would today call QoS routing is a non-goal and that 1002 routing would not seek to control the elastic characteristics of 1003 Internet traffic whereby a TCP connection can seek to utilize all the 1004 spare bandwidth on a route, possibly to the detriment of other 1005 connections sharing the route or crossing it. 1007 Relevance: Open Issue. It is not clear whether dynamic QoS routing 1008 can or should be implemented. Such a system would seek to 1009 control the admission and routing of traffic depending on 1010 current or recent resource utilization. This would be 1011 particularly problematic where traffic crosses an 1012 ownership boundary because of the need for potentially 1013 commercially sensitive information to be made available 1014 outside the ownership boundary. 1016 Current practice: Routing does not consider dynamic resource 1017 availability. Forwarding can support service 1018 differentiation. 1020 2.2 Nimrod Requirements 1022 Nimrod as expressed by Noel Chiappa in his early document, ��A New IP 1023 Routing and Addressing Architecture�� (1991) and later in the NIMROD 1024 Working Group documents RFC 1753 and RFC1992 established a number of 1025 requirements that need to be considered by any new routing 1026 architecture. The Nimrod requirements took RFC1126 as a starting 1027 point and went further. 1029 The goals of Nimrod, quoted from RFC1992, were as follows: 1031 1. To support a dynamic internetwork of *arbitrary size* (our 1032 emphasis) by providing mechanisms to control the amount of 1033 routing information that must be known throughout an 1034 internetwork. 1036 2. To provide service-specific routing in the presence of multiple 1037 constraints imposed by service providers and users. 1039 3. To admit incremental deployment throughout an internetwork. 1041 It is certain that these goals remain as requirements for any new 1042 domain routing architecture. 1044 - As discussed in other sections of this document the amount of 1045 information needed to maintain the routing system is growing at 1046 a rate that does not scale. And yet, as the services and 1048 Davies, et al Expires: January 2002 20 1049 constraints upon those services grow there is a need for more 1050 information to be maintained by the routing system. 1051 One of the key terms in the first requirements is �control�. 1052 While increasing amounts of information need to be known and 1053 maintained in the Internet, the amounts and kinds of information 1054 that are distributed can be controlled. 1055 This goal will be reflected in the requirements for the future 1056 domain architecture. 1057 - If anything, the demand for specific services in the Internet 1058 has grown since 1996 when the Nimrod architecture was published. 1059 Additionally the kinds of constraints that service providers 1060 need to impose upon their networks and that services need to 1061 impose upon the routing have also increased. Any changes made 1062 to the network in the last half-decade have not significantly 1063 improved this situation. 1064 - The ability to incrementally deploy any new routing architecture 1065 within the Internet is still a absolute necessity. It is 1066 impossible to imagine that a new routing architecture could 1067 supplant the current architecture on a flag day 1069 At one point in time Nimrod, with its addressing and routing 1070 architectures was seen as a candidate for IPng. History shows that 1071 it was not accepted as the IPng, having been ruled out of the 1072 selection process by the IESG in 1994 on the grounds that it was �too 1073 much of a research effort� [35], although input for the requirements 1074 of IPng was explicitly solicited from Chiappa [8]. I�d still like to 1075 know more about what those reasons were� 1076 Instead IPv6 has been put forth as the IPng. Without entering a 1077 discussion of the relative merits of IPv6 versus Nimrod, it is 1078 apparent that IPv6, while it may solve many problems, does not solve 1079 the critical routing problems in the Internet today. In fact in some 1080 sense it exacerbates them by adding a requirements for support of two 1081 internet protocols and their respective addressing methods. In many 1082 ways the addition of IPv6 to the mix of methods in today�s Internet 1083 only points to the fact that the goals, as set forth by the Nimrod 1084 team, remain as necessary goals. 1086 There is another sense in which study of Nimrod and its architecture 1087 may be important to deriving a FDR. Nimrod can be said to have two 1088 derivatives: 1089 - MPLS in that it took the notion of forwarding along well known 1090 paths 1091 - PNNI in that it took the notion of abstracting topological 1092 information and using that information to create connections for 1093 traffic. 1094 It is important to note, that whilst MPLS and PNNI borrowed ideas 1095 from Nimrod, neither of them can be said to be an implementation of 1096 this architecture. 1098 2.3 PNNI 1100 PNNI was developed under the ATM Forum�s auspices as a hierarchical 1101 route determination protocol for ATM, a connection oriented 1102 architecture. It is reputed to have developed several of it methods 1104 Davies, et al Expires: January 2002 21 1105 from a study of the Nimrod architecture. What can be gained from an 1106 analysis of what did and did not succeed in PNNI? 1108 The PNNI protocol includes the assumption that all peer groups are 1109 willing to cooperate, and that the entire network is under the same 1110 top administration. Are there limitations that stem from this �world 1111 node� presupposition? As we know [13], the Internet is no longer a 1112 clean hierarchy and there is a lot of resistance to having any sort 1113 of �ultimate authority� controlling or even brokering communication. 1115 PNNI is the first deployed example of a routing protocol that uses 1116 abstract map exchange (as opposed to distance vector or link state 1117 mechanisms) for inter-domain routing information exchange. One 1118 consequence of this is that domains need not all use the same 1119 mechanism for map creation. What were the results of this 1120 abstraction and source based route calculation mechanism? 1122 Since the authors of this document do not have experience running a 1123 PNNI network, the comments above are from a theoretical perspective. 1124 Information on these issues, and any other relevant issues, is 1125 solicited from those who do have such operational experience. 1127 2.4 Recent Research Work 1129 2.4.1 Developments in Internet Connectivity 1131 The recent work commissioned from Geoff Huston by the Internet 1132 Architecture Board [13] draws a number of conclusions from analysis 1133 of BGP routing tables and routing registry databases: 1134 - The connectivity between provider ASs is becoming more like a 1135 dense mesh than the tree structure which was commonly assumed to 1136 be commonplace a couple of years ago. This has been driven by the 1137 increasing amounts charged for peering and transit traffic by 1138 global service providers. Local direct peering and internet 1139 exchanges are becoming steadily more common as the cost of local 1140 fibre connections drops. 1141 - End user sites are increasingly resorting to multi-homing onto two 1142 or more service providers as a way of improving resiliency. This 1143 has a knock-on effect of spectacularly fast depletion of the 1144 available pool of AS numbers as end user sites require public AS 1145 numbers to become multi-homed and corresponding increase in the 1146 number of prefixes advertised in BGP. 1147 - Multi-homed sites are using advertisement of longer prefixes in 1148 BGP as a means of traffic engineering to load spread across their 1149 multiple external connections with further impact on the size of 1150 the BGP tables. 1151 - Operational practices are not uniform, and in some cases lack of 1152 knowledge or training is leading to instability and/or excessive 1153 advertisement of routes by incorrectly configured BGP speakers. 1154 - All these factors are quickly negating the advantages in limiting 1155 the expansion of BGP routing tables that were gained by the 1156 introduction of CIDR and consequent prefix aggregation in BGP. It 1157 is also now impossible for IPv6 to realize the world view in which 1158 the default free zone would be limited to perhaps 10,000 prefixes. 1160 Davies, et al Expires: January 2002 22 1161 - The typical �width� of the Internet in AS hops is now around five, 1162 and much less in many cases. 1164 These conclusions have a considerable impact on the requirements for 1165 the FDR: 1166 - Topological hierarchy (e.g. mandating a tree structured 1167 connectivity) cannot be relied upon to deliver scalability of a 1168 large Internet routing system 1169 - Aggregation cannot be relied upon to constrain the size of routing 1170 tables for an all-informed routing system 1172 2.4.2 Defending the End To End Principle 1174 DARPA is funding a project to think about a new architecture for 1175 future generation Internet, called imaginatively NewArch 1176 (http://www.isi.edu/newarch/). Work started in the first half of 1177 2000 but the published results are limited to an introductory paper 1178 and some slides. 1180 The main development so far is to conclude that as the Internet 1181 becomes mainstream infrastructure, fewer and fewer of the 1182 requirements are truly global but may apply with different force or 1183 not at all in certain parts of the network. This (it is claimed) 1184 makes the compilation of a single, ordered list of requirements 1185 deeply problematic. Instead we may have to produce multiple 1186 requirement sets with support for differing requirement importance at 1187 different times and in different places. This �meta-requirement� 1188 significantly impacts architectural design. 1190 Potential new technical requirements identified so far include: 1191 - Commercial environment concerns such as richer inter-provider 1192 policy controls and support for a variety of payment models 1193 - Trustworthiness 1194 - Ubiquitous mobility 1195 - Policy driven self-organisation (�deep auto configuration�) 1196 - Extreme short-time-scale resource variability 1197 - Capacity allocation mechanisms 1198 - Speed, propagation delay and Delay/BandWidth Product issues 1200 Non-technical or political �requirements� include: 1201 - Legal and Policy drivers such as 1202 o Privacy and free/anonymous speech 1203 o Intellectual property concerns 1204 o Encryption export controls 1205 o Law enforcement surveillance regulations 1206 o Charging and taxation issues 1207 - Reconciling national variations and consistent operation in a 1208 world wide infrastructure 1210 One of the participants in this work (Dave Clark) with one of his 1211 associates has also just published a very interesting paper analyzing 1212 the impact of some of these new requirements on the end to end 1213 principle that has guided the development of the Internet to date 1214 [32]. Their primary conclusion is that the loss of trust between the 1216 Davies, et al Expires: January 2002 23 1217 users at the ends of end to end has the most fundamental effect on 1218 the Internet. This is clear in the context of the routing system, 1219 where operators are unwilling to reveal the inner workings of their 1220 networks for commercial reasons. Similarly, trusted third parties 1221 and their avatars (mainly mid-boxes of one sort or another) have a 1222 major impact on the end to end principles and the routing mechanisms 1223 that went with them. Overall, the end to end principles should be 1224 defended so far as is possible - some changes are already too deeply 1225 embedded to make it possible to go back to full trust and openness - 1226 at least partly as a means of staving off the day when the network 1227 will ossify into an unchangeable form and function (much as the 1228 telephone network has done). The hope is that by that time a new 1229 Internet will appear to offer a context for unfettered innovation. 1231 3. Existing problems of BGP and the current EGP/IGP Architecture 1233 Although most of the people who have to work with BGP today believe 1234 it to be a useful, working protocol, discussions have brought to 1235 light a number of areas where BGP or the relationship between BGP and 1236 the IGPs in use today could be improved. This section is, to a large 1237 extent, a wish list for the FDR based on those areas where BGP is 1238 seen to be lacking, rather than simply a list of problems with BGP. 1239 The shortcomings of today�s inter-domain routing system have also 1240 been extensively surveyed in �Architectural Requirements for Inter- 1241 Domain Routing in the Internet� [13], particularly with respect to 1242 its stability and the problems produced by explosions in the size of 1243 the Internet. 1245 3.1 BGP and Auto-aggregation 1247 The stability and later linear growth rates of the number of routing 1248 objects (prefixes) that was achieved by the introduction of CIDR 1249 around 1994, has now been once again been replaced by near- 1250 exponential growth of number of routing objects. The granularity of 1251 many of the objects advertised in the DFZ is very small (prefix 1252 length of 22 or longer): This granularity appears to be a by-product 1253 of attempts to perform precision traffic engineering related to 1254 increasing levels of multi-homing. At present there is no mechanism 1255 in BGP that would allow an AS to aggregate such prefixes without 1256 advance knowledge of their existence, even if it was possible to 1257 deduce automatically that they could be aggregated. Achieving 1258 satisfactory auto-aggregation would also significantly reduce the 1259 non-locality problems associated with instability in peripheral ASs. 1261 On the other hand, it may be that alterations to the connectivity of 1262 the net as described in [13] and Section 2.4.1 may limit the 1263 usefulness of auto-aggregation 1265 3.2 Convergence and Recovery Issues 1267 BGP today is a stable protocol under most circumstances but this has 1268 been achieved at the expense of making the convergence time of the 1269 inter-domain routing system very slow under some conditions. This 1271 Davies, et al Expires: January 2002 24 1272 has a detrimental effect on the recovery of the network from 1273 failures. 1275 The timers that control the behavior of BGP are typically set to 1276 values in the region of several tens of seconds to a few minutes, 1277 which constrains the responsiveness of BGP to failure conditions. 1279 In the early days of deployment of BGP, poor network stability and 1280 router software problems lead to storms of withdrawals closely 1281 followed by re-advertisements of many prefices. To control the load 1282 on routing software imposed by these �route flaps�, route flap 1283 damping was introduced into BGP. Most operators have now implemented 1284 a degree of route flap damping in their deployments of BGP. This 1285 restricts the number of times that the routing tables will be rebuilt 1286 even if a route is going up and down very frequently. Unfortunately, 1287 the effect of route flap damping is exponential in its behavior which 1288 can result in some parts of the Internet being inaccessible for hours 1289 at a time. 1291 There is evidence ([13] and our own measurements) that in today�s 1292 network route flap is disproportionately associated with the fine 1293 grain prefices (length 22 or longer) associated with traffic 1294 engineering at the periphery of the network. Auto-aggregation as 1295 previously discussed would tend to mask such instability and prevent 1296 it being propagated across the whole network. 1298 3.3 Non-locality of Effects of Instability and Misconfiguration 1300 There have been a number of instances, some of which are well- 1301 documented (e.g. The April 1997 incident when misconfiguration of BGP 1302 at a small company in Virginia, USA, turned the company into a 1303 traffic magnet for much of the traffic in the Internet resulting in 1304 global problems until it was fixed) of a mistake in BGP configuration 1305 in a single peripheral AS propagating across the whole Internet and 1306 resulting in misrouting of most of the traffic in the Internet. 1308 Similarly, route flap in a single peripheral AS can require route 1309 table recalculation across the entire Internet. 1311 This non-locality of effects is highly undesirable, and it would be a 1312 considerable improvement if such effects were naturally limited to a 1313 small area of the network around the problem. 1315 3.4 Multihoming Issues 1317 As discussed previously, the increasing use of multi-homing as a 1318 robustness technique by peripheral ASs requires that multiple routes 1319 have to be advertised for such domains. These routes must not be 1320 aggregated close in to the multi-homed domain as this would defeat 1321 the traffic engineering implied by multi-homing and currently cannot 1322 be aggregated further away from the multi-homed domain due to the 1323 lack of auto-aggregation capabilities. Consequentially the DFZ 1324 routing table is growing exponentially again. 1326 Davies, et al Expires: January 2002 25 1327 The longest prefix match routing technique introduced by CIDR, and 1328 implemented in BGP4, when combined with provider address allocation 1329 is an obstacle to effective multi-homing if load sharing across the 1330 multiple links is required: If an AS has been allocated its 1331 addresses from an upstream provider, the upstream provider can 1332 aggregate those addresses with those of other customers and need only 1333 advertise a single prefix for a range of customers. But, if the 1334 customer AS is also connected to another provider, the second 1335 provider is not able to aggregate the customer addresses because they 1336 are not taken from his allocation, and will therefore have to 1337 announce a more specific route to the customer AS. The longest match 1338 rule will then direct all traffic through the second provider, which 1339 is not as required. 1341 Example: 1342 AS3 has received its addresses from AS1, which means AS1 can 1343 Aggregate. But if AS3 want its traffic to be seen equally 1344 both ways, AS1 is forced to announce both the aggregate and 1345 the more specific route to AS3. 1347 \ / 1348 AS1 AS2 1349 \ / 1350 AS3 1352 This problem has induced many ASs to apply for their own address 1353 allocation even though they could have been allocated from an 1354 upstream provider further exacerbating the DFZ route table size 1355 explosion. This problem also interferes with the desire of many 1356 providers in the DFZ to route only prefixes that are equal to or 1357 shorter than 20 or 19 bits. 1359 Note that some problems which are referred to as multihoming issues 1360 are not and should not solvable through the routing system (e.g. 1361 where a TCP load distributor is needed), and multihoming is not a 1362 panacea for the general problem of robustness in a routing 1363 system [38]. 1365 3.5 AS-number exhaustion 1367 The domain identifier or AS-number is a 16-bit number. Allocation of 1368 AS-numbers is currently increasing 51% p.a. [13] with the result that 1369 exhaustion is likely around 2005. The IETF is currently studying 1370 proposals to increase the available range of AS-numbers to 32 bits, 1371 but this will present a deployment challenge during transition. 1373 3.6 Partitioned AS�s 1375 Tricks with discontinuous ASs are used by operators, for example, to 1376 implement anycast. Discontinuous ASs may also come into being by 1378 Davies, et al Expires: January 2002 26 1379 chance if a multi-homed domain becomes partitioned as a result of a 1380 fault and part of the domain can access the Internet through each 1381 connection. It may be desirable to make BGP�s support for this kind 1382 of situation more transparent than at present. 1384 3.7 Load Sharing 1386 Load splitting or sharing was not a goal of the original designers of 1387 BGP and it is now a problem for today�s network designers and 1388 managers. Trying to fool BGP into load sharing between several links 1389 is a constantly recurring exercise for most operators today. Traffic 1390 engineering extensions to the FDR which will facilitate load sharing 1391 are essential. 1393 3.8 Hold down issues 1395 As with the interval between �hello� messages in OSPF, the typical 1396 size and defined granularity (seconds to tens of seconds) of the 1397 �keep-alive� time negotiated at start-up for each BGP connection 1398 constrains the responsiveness of BGP to link failures. 1400 The recommended values and the available lower limit for this timer 1401 were set to limit the overhead caused by keep-alive messages when 1402 link bandwidths were typically much lower than today. Analysis and 1403 experiment ([14], [15] & [33]) indicate that faster links could 1404 sustain a much higher rate of keep-alive messages without 1405 significantly impacting normal data traffic. This would improve 1406 BGP�s responsiveness to link and node failures but with a 1407 corresponding increase in the risk of instability, if the error 1408 characteristics of the link are not taken properly into account when 1409 setting the keep-alive interval. 1411 An additional problem with the hold-down mechanism in BGP is the 1412 amount of information that has to be exchanged to re-establish the 1413 database of route advertisements on each side of the link when it is 1414 re-established after a failure. Currently any failure, however brief 1415 forces a full exchange which could perhaps be constrained by 1416 retaining some state across limited time failures and using revision 1417 control, transaction and replication techniques to re-synchonise the 1418 databases. Various techniques have been implemented to try to reduce 1419 this problem but they have not yet been standardised. 1421 3.9 Interaction between Inter domain routing and intra domain routing 1423 Today, many operators� backbone routers run both I-BGP and an IGP 1424 maintain the routes that reach between the borders of the domain. 1425 Exporting routes from BGP into IGP and bringing them back up to BGP 1426 is not recommended [29], but it is still necessary for all backbone 1427 routers to run both protocols. BGP is used to find the egress point 1428 and IGP to find the path (next hop router) to the egress point across 1429 the domain. This is not only a management problem but may also create 1430 other problems: 1432 Davies, et al Expires: January 2002 27 1433 - BGP is a distance vector protocol, as compared with most IGPs 1434 which are link state protocols, and as such it is not optimised 1435 for convergence speed although they generally require less 1436 processing power. Incidentally, more efficient distance vector 1437 algorithms are available such as [34]. 1439 - The metrics used in BGP and the IGP are rarely comparable or 1440 combinable. Whilst there are arguments that the optimizations 1441 inside a domain may be different from those for end-to-end paths, 1442 there are occasions, such as calculating the �topologically 1443 nearest� server when computable or combinable metrics would be of 1444 assistance. 1446 - The policies that can be implemented using BGP are designed for 1447 control of traffic exchange between operators, not for controlling 1448 paths within a domain. Policies for BGP are most conveniently 1449 expressed in RPSL and this could be extended if thought desirable 1450 to include IGP policies. 1452 - If the NEXT HOP destination for a set of BGP routes becomes 1453 inaccessible because of IGP problems, the routes using the 1454 vanished next hop have to be invalidated at the next available 1455 UPDATE. Subsequently, if the next hop route reappears, this would 1456 normally lead to the BGP speaker requesting a full table from its 1457 neighbour(s). Current implementations may attempt to circumvent 1458 the effects of IGP route flap by caching the invalid routes for a 1459 period in case the next hop is restored. 1461 - Synchronization between IGP and EGP is a problem as long as we use 1462 different protocols for IGP and EGP, which will most probably be 1463 the case even in the future because of the differing requirements 1464 in the two situations. Some sort of synchronization between those 1465 two protocols would be useful. The draft �OSPF Transient Blackhole 1466 Avoidance� [22], the IGP side of the story is covered. 1468 - Synchronizing in BGP means waiting for the IGP to know about the 1469 same networks as the EGP, which can take a significant period of 1470 time and slows down the convergence of BGP by adding the IGP 1471 convergence time into each cycle. 1473 3.10 Policy Issues 1475 There are several classes of issue with current BGP policy: 1477 - Policy is installed in an ad-hoc manner in each autonomous 1478 system. There isn�t a method for ensuring that the policy 1479 installed in one router is coherent with policies installed in 1480 other routers. 1481 - As described in Griffin [12] and in McPherson [20] it is 1482 possible to create policies for ASs, and instantiate them in 1483 routers, that will cause BGP to fail to converge in certain 1484 types of topology 1485 - There is no available network model for describing policy in a 1486 coherent manner. 1488 Davies, et al Expires: January 2002 28 1489 Policy management is extremely complex and mostly done without the 1490 aid of any automated procedures. The extreme complexity means that a 1491 highly qualified specialist is required for policy management of 1492 border routers. The training of these specialists is quite lengthy 1493 and needs to involve long periods of hands-on experience. There is, 1494 therefore, a shortage of qualified staff for installing and 1495 maintaining the routing policies. Also many training courses cover 1496 only the basic configuration aspects and do not cover policy issues. 1498 3.11 Security Issues 1500 While many of the issues with BPG security have been traced either to 1501 implementation issues or to operational issues, BGP is vulnerable to 1502 DDOS attacks. Additionally routers can be used as unwitting 1503 forwarders in DDOS attacks on other systems. 1505 Though DDOS attacks can be fought in a variety of ways, most 1506 filtering methods, it is takes constant vigilance. There is nothing 1507 in the current architecture or in the protocols that serves to 1508 protect the forwarders from these attacks. 1510 3.12 Support of MPLS and VPNS 1512 Recently BGP has been modified to function as a signalling protocol 1513 for MPLS and for VPNs [16]. Some people see this over-loading of 1514 the BGP protocol as a boon whilst others see it as a problem. While 1515 it was certainly convenient as a vehicle for vendors to deliver extra 1516 functionality for to their products, it has exacerbated some of the 1517 performance and complexity issues of BGP. Two important problems are, 1518 the additional state that must be retained and refreshed to support 1519 VPN tunnels and that BGP does not provide end-to-end notification 1520 making it difficult to confirm that all necessary state has been 1521 installed or updated. 1523 In creating the future domain routing architecture, serious 1524 consideration must be given to the argument that VPN signaling 1525 protocols should remain separate from the route determination 1526 protocols. 1528 3.13 IPv4 / IPv6 Ships in the Night 1530 The fact that service providers would need to maintain two completely 1531 separate networks; one for IPv4 and one for IPv6 has been a real 1532 hindrance to the introduction of IPv6. Even if IPv6 does get 1533 deployed it will do so without causing the disappearance of IPv4. 1534 This means that unless something is done, service providers would 1535 need to maintain the two networks in perpetuity. 1537 It is possible to use a single set of BGP speakers with multiprotocol 1538 extensions [37] to exchange information about both IPv4 and IPv6 1539 routes between domains, but the use of TCP as the transport protocol 1540 for the information exchange results in an asymmetry when choosing to 1541 use one of TCP over IPv4 or TCP over IPv6. Successful information 1543 Davies, et al Expires: January 2002 29 1544 exchange confirms one of IPv4 or IPv6 reachability between the 1545 speakers but not the other, making it possible that reachability is 1546 being advertised for a protocol for which it is not present. 1548 Also, current implementations do not allow a route to be advertised 1549 for both IPv4 and IPv6 in the same UPDATE message, because it is not 1550 possible to explicitly link the reachability information for an 1551 address family to the corresponding next hop information. This could 1552 be improved, but currently results in independent UPDATEs being 1553 exchanged for each address family. 1555 The tools available to network operators 1557 3.14 Existing Tools to Support Effective Deployment of Inter-Domain 1558 Routing 1560 The tools available to network operators to assist in configuring and 1561 maintaining effective inter-domain routing in line with their defined 1562 policies are limited, and almost entirely passive. 1564 For example: 1565 - there are no tools to facilitate the planning of the routing of a 1566 domain (either intra- or inter-domain); there are a limited 1567 number of display tools which will visualize the routing once it 1568 has been configured 1569 - there are no tools to assist in converting business policy 1570 specifications into the RPSL language; there are limited tools to 1571 convert the RPSL into BGP commands and to check, post-facto, that 1572 the proposed policies are consistent with the policies in adjacent 1573 domains (always provided that these have been revealed and 1574 accurately documented). 1575 - there are no tools to monitor BGP route changes in real time and 1576 warn the operator about policy inconsistencies and/or 1577 instabilities. 1579 The following section summarises the tools that are available to 1580 assist with the use of RPSL. Note they are all batch mode tools used 1581 off-line from a real network. These tools will provide checks for 1582 skilled inter-domain routing configurers but limited assistance for 1583 the novice. 1585 3.14.1 Routing Policy Specification Language RPSL (RFC 2622, 2650) and 1586 RIPE NCC Database (RIPE 157) 1588 Routing Policy Specification Language RPSL enables a network operator 1589 to describe routes, routers and autonomous systems ASs that are 1590 connected to the local AS. 1592 Using the RPSL language a distributed database is created to describe 1593 routing policies in the Internet as described by each AS 1594 independently. The database can be used to check the consistency of 1595 routing policies stored in the database. 1597 Davies, et al Expires: January 2002 30 1598 Tools exist (RIPE 81, 181, 103) that can be applied on the database 1599 to answer requests of the form, e.g. 1601 - flag when two neighboring network operators specify conflicting or 1602 inconsistent routing information exchanges with each other and 1603 also detect global inconsistencies where possible; 1604 - extract all AS-paths between two networks which are allowed by 1605 routing policy from the routing policy database; display the 1606 connectivity a given network has according to current policies. 1608 The database queries enable a partial static solution to the 1609 convergence problem. They analyze routing policies of very limited 1610 part of Internet and verify that they do not contain conflicts that 1611 could lead to protocol divergence. The static analysis of convergence 1612 of the entire system has exponential time complexity, so 1613 approximation algorithms would have to be used. 1615 Davies, et al Expires: January 2002 31 1616 4. Expected Users 1618 This section considers the requirements imposed by the target 1619 audience of the FDR both in terms of organizations that might own 1620 networks, which would use FDR, and the human users who will have to 1621 interact with the FDR. 1623 4.1 Organisations 1625 The organizations that own networks connected to the Internet have 1626 become much more diverse since RFC1126 [4] was published. In 1627 particular a major part of the network is now owned by commercial 1628 service provider organizations in the business of making profits from 1629 carrying data traffic. 1631 4.1.1 Commercial Service Providers 1633 The routing system must take into account their desires for 1634 commercial secrecy and security, as well as allowing them to organize 1635 their business as flexibly as possible. 1637 Service providers will often wish to conceal the details of the 1638 network from other connected networks. So far as is possible, the 1639 routing system should not require the service providers to expose 1640 more details of the topology and capability of their networks than is 1641 strictly necessary. 1643 Many service providers will also offer contracts to their customers 1644 in the form of Service Level Agreements (SLAs) and the routing system 1645 must allow the providers to support these SLAs through traffic 1646 engineering and load balancing as well as multihoming allowing them 1647 to achieve the degree of resilience and robustness that they need. 1649 Service providers can be categorized as 1651 - Global Service Providers (GSPs) with networks which have a 1652 global reach. Such providers may and usually will wish to 1653 constrain traffic between their customers to run entirely on 1654 their networks. Such providers will interchange traffic at 1655 multiple peering points with other GSPs and need extensive 1656 policy-based controls to control the interchange of traffic. 1657 Peering may be through the use of dedicated private lines 1658 between the partners or increasingly through Internet Exchange 1659 Points. 1660 - National Service Providers (NSPs)which are similar to GSPs but 1661 typically cover one country. Such organizations may operate as 1662 a federation which provides similar reach to a GSP and may wish 1663 to be able to steer traffic preferentially to other federation 1664 members to achieve global reach. 1665 - Local Internet Service Providers (ISPs) operate regionally and 1666 will typically purchase transit capacity from NSPs or GSPs to 1667 provide global connectivity, but may also peer with neighbouring 1668 ISPs. 1670 Davies, et al Expires: January 2002 32 1671 The routing system should be sufficiently flexible to accommodate the 1672 continually changing business relationships of the providers, and the 1673 various levels of trustworthiness that they apply to customers and 1674 partners. 1676 Service providers will need to be involved in accounting for Internet 1677 usage, monitoring the traffic, and may be involved in government 1678 action to tax the usage of the Internet, enforce social mores and 1679 intellectual property rules or apply surveillance to the traffic to 1680 detect or prevent crime. 1682 4.1.2 Enterprises 1684 The leaves of the network domain graph are in many cases networks 1685 supporting a single enterprise. Such networks cover an enormous 1686 range of complexity with some multi-national companies owning 1687 networks that rival the complexity and reach of a GSP whereas many 1688 fall into the Small Office-Home Office (SOHO) category. The routing 1689 system should allow simple and robust configuration and operation for 1690 the SOHO category, whilst effectively supporting the larger 1691 enterprise. 1693 Enterprises are particularly likely to lack the capability to 1694 configure and manage a complex routing system and every effort should 1695 be made to provide simple configuration and operation for such 1696 networks. 1698 Enterprises will also wish to be able to change their service 1699 provider with ease. Whilst this is predominantly a naming and 1700 addressing issue, the routing system must be able to support seamless 1701 changeover, for example, by coping with a changeover period when both 1702 sets of addresses are in use. 1704 Enterprises will wish to be able to multihome to one or more 1705 providers as one possible means of enhancing the resilience of their 1706 network. 1708 Enterprises will also frequently wish to control the trust that they 1709 place both in workers and external connections through firewalls and 1710 similar mid-boxes placed at their external connections. 1712 4.1.3 Domestic Networks 1714 Increasingly domestic networks are likely to be �always on� and will 1715 resemble SOHO enterprises networks with no special requirements of 1716 the routing system. 1718 In the meantime, the routing system must support dial-up users. 1720 4.1.4 Internet Exchange Points 1722 Peering of service providers, academic networks and larger 1723 enterprises is increasingly happening at specific Internet Exchange 1724 Points where many networks are linked together in a relatively small 1726 Davies, et al Expires: January 2002 33 1727 physical area. The resources of the exchange may be owned by a 1728 trusted third party or jointly by the connecting networks. The 1729 routing systems should support such exchange points without requiring 1730 the exchange point to either operate as a superior entity with every 1731 connected network logically inferior to it or requiring the exchange 1732 point to be a member of one (or all) connected networks. The 1733 connecting networks have to delegate a certain amount of trust to the 1734 exchange point operator. 1736 4.1.5 Content Providers 1738 Content providers are at one level a special class of enterprise, but 1739 the desire to deliver content efficiently means that a content 1740 provider may provide multiple replicated origin servers or caches 1741 across a network. These may also be provided by a separate content 1742 delivery service. The routing system should facilitate delivering 1743 content from the most efficient location. 1745 4.2 Human Users 1747 This section covers the most important human users of the FDR and 1748 their expected interactions with the system. 1750 4.2.1 Network Planners 1752 The routing system should allow them to plan and implement a network 1753 that can be proved to be stable and will meet their traffic 1754 engineering requirements. 1756 4.2.2 Network Operators 1758 The routing system should, so far as is possible, be simple to 1759 configure and operate, behave in a predictable, stable fashion and 1760 deliver appropriate statistics and events to allow the network to be 1761 managed and upgraded in an efficient and timely fashion. 1763 4.2.3 Mobile End Users 1765 The routing system must support mobile end users. The NewArch team 1766 (see Section 2.4.2) considers that mobility will become �ubiquitous� 1768 5. Mandated Constraints 1770 While many of the requirement to which the protocol must respond are 1771 technical, some aren�t. These mandated constraints are those that 1772 are determined by conditions of the world around us. Understanding 1773 these requirements requires and analysis of the world in which these 1774 systems will be deployed,. The constraints include those that are 1775 determined by: 1776 - Environmental factors. 1777 - Geography 1778 - Political boundaries and considerations 1780 Davies, et al Expires: January 2002 34 1781 - Technological factors such as the prevalence of different 1782 levels of technology in the developed world as opposed to in 1783 the developing or undeveloped world. 1785 5.1 The Federated Environment 1787 The graph of the Internet network with routers and other control 1788 boxes at the nodes and communication links along the edges is today 1789 partitioned administratively into a large number of disjoint domains, 1790 known as Autonomous Systems (ASs). 1792 A common administration may have responsibility for one or more 1793 domains that may or may not be adjacent in the graph. 1795 Commercial and policy constraints affecting the routing system will 1796 typically be exercised at the boundaries of these domains where 1797 traffic is exchanged between domains. 1799 The perceived need for commercial confidentiality will seek to 1800 minimise the information transferred across these boundaries, leading 1801 to requirements for aggregated information, abstracted maps of 1802 connectivity exported from domains and mistrust of supplied 1803 information. 1805 The perceived desire for anonymity may require the use of zero- 1806 knowledge security protocols to allow users to access resources 1807 without exposing their identity. 1809 One possible extension to the requirements would be to require the 1810 protocols to provide the ability for groups of peering domains to be 1811 treated as a (super-)domain. These domains would have a common 1812 administrative policy. 1814 5.2 Working with different sorts of networks 1816 The diverse Layer 2 networks over which the layer 3 routing system is 1817 implemented have typically been operated totally independently from 1818 the layer 3 network. Consideration needs to be given to the degree 1819 and nature of interchange of information between the layers that is 1820 desirable. In particular, the desire for robustness through diverse 1821 routing implies knowledge of the underlying networks to be able to 1822 guarantee the robustness. 1824 Mobile access networks may also impose extra requirements on Layer 3 1825 routing. 1827 5.3 Delivering Diversity 1829 The routing system is operating at Layer 3 in the network. To 1830 achieve robustness and resilience at this layer requires that where 1831 multiple diverse routes are employed as part of delivering the 1832 resilience, the routing system at Layer 3 needs to be assured that 1833 the Layer 2 and lower routes are really diverse. The �diamond 1834 problem� is the simplest form of this problem - layer 3 provider 1836 Davies, et al Expires: January 2002 35 1837 attempting to provide diversity buys layer 2 services from two 1838 separate providers who in turn buy wayleaves from the same provider: 1840 Layer 3 service 1841 / \ 1842 / \ 1843 Layer 2 Layer 2 1844 Provider A Provider B 1845 \ / 1846 \ / 1847 Trench provider 1849 Now when the backhoe cuts the trench, the Layer 3 provider has no 1850 resilience unless he had taken special steps to verify that the 1851 trench wasn�t common. The routing system should facilitate avoidance 1852 of this kind of trap. 1854 Some work is going on to understand the sort of problems that stemm 1855 from this requirement, such as the work on Shared Risk Link Groups 1856 [31]. Unfortunately, the full generality of the problem requires 1857 diversity be maintained over time between an arbitrarily large set of 1858 mutually distrustful providers. For some cases, it may be sufficient 1859 for diversity to be checked at provisioning or route instantiation 1860 time, but this remains a hard problem requiring research work. 1862 5.4 When will the new solution be required? 1864 There is a full range of opinion on this subject. An informal survey 1865 indicates that the range varies from 2 years to 6 years. And while 1866 there are those, possibly outliers, who think there is no need for a 1867 new routing architecture as well as those who think a new 1868 architecture was needed years ago, the median seems to lie at around 1869 4 years. As in all projections of the future this is largely not 1870 provable. 1872 6. Assumptions 1874 In projecting the requirements for the Future Routing Domain a number 1875 of assumptions have been made. The requirements set out should be 1876 consistent with these assumptions, but there are doubtless a number 1877 of other assumptions which are not explicitly articulated here: 1879 1. The number of hosts today is somewhere in the area of 100 Million. 1880 With dial in and NATs this is likely to turn into up to 500 1881 Million users (see [30]). In a number of years, with wireless 1882 accesses and different �gizmos� attaching to the Internet, we are 1883 likely to see a couple of Billion �users� on the Internet. The 1884 number of globally addressable hosts is very much dependent on how 1885 common NATs will be in the future. 1886 2. NATs and other mid-boxes exist and we cannot assume that they will 1887 cease being a presence in the networks. 1888 3. The number of operators in the Internet will probably not grow 1889 very much, as there is a likelihood that operators will tend to 1891 Davies, et al Expires: January 2002 36 1892 merge. However, as Internet-connectivity expands to new countries, 1893 new operators will emerge and then merge again. 1894 4. Today, there are around 9,500 AS�s with a growth rate of around 1895 51% per annum [13]. With current use of AS�s (for e.g., multi- 1896 homing) the number of AS�s grow to 70,000 within 3 - 5 years. 1897 5. In contrast to the number of operators, the number of domains is 1898 likely to grow significantly. Today, each operator has different 1899 domains within an AS, but this also shows in SLAs and policies 1900 internal to the operator. Making this globally visible would 1901 create a number of domains 10-100 times the amount of ASs, i.e., 1902 between 100,000 and 1,000,000. 1903 6. With more and more capacity at the edge of the network the IP 1904 network will expand. Today there are operators with several 1905 thousands of routers, but this is likely to be increased. A domain 1906 will probably contain tens of thousands of routers. 1907 7. The speed of connections in the (fixed) access will technically be 1908 (almost) unconstrained. However, the cost for the links will not 1909 be negligible so that the apparent speed will be effectively 1910 bounded. Within a number of years some will have multi-Gigabit- 1911 speed in the access. 1912 8. At the same time, the bandwidth of wireless access still has a 1913 strict upper-bound. Within the foreseeable future each user will 1914 only have a tiny amount of resources available compared to fixed 1915 accesses (10kbps to 2Mbps for UMTS with only a few achieving the 1916 higher figure as the bandwidth is shared between the active users 1917 in a cell and only small cells can actually reach this speed, but 1918 11Mbps or more for wireless LAN connections). 1919 9. Assumptions 7 and 8 taken together suggest a span of bandwidth 1920 between 10 kbps to 10 Gbps. 1921 10. The speed in the backbone has grown rapidly, and there is no 1922 evidence that the growth will stop in the coming years. Terabit- 1923 speed is likely to be the minimum backbone speed in a couple of 1924 years. The range of bandwidths that might need to be represented 1925 will require some thought to be given to how to represent the 1926 values in the protocols. 1927 11. There have been discussions as to whether Moore's law will 1928 continue to hold for processor speed. If Moore's law does not 1929 hold, then communication circuits might play a more important role 1930 in the future. Also, optical routing is based on circuit 1931 technology, which is the main reason for taking �circuits� into 1932 account when designing an FDR. 1933 12. However, the datagram model still remains the fundamental model 1934 for the Internet. 1935 13. The number of peering points in the network is likely to grow, as 1936 multi-homing becomes important. Also traffic will become more 1937 locally distributed, which will drive the demand for local 1938 peering. 1939 14. The FDR will achieve the same degree of ubiquity as the current 1940 Internet and IP routing. 1942 Davies, et al Expires: January 2002 37 1943 7. Functional Requirements 1945 This section includes a detailed discussion of new requirements for a 1946 future domain routing architecture. As discussed in section 2.1 a 1947 new architecture must build upon the requirements for past routing 1948 architecture. For that reason, the requirements discussed in section 1949 2.1 are not repeated here. In case where the requirement has changed 1950 significantly, was omitted from the discussions in RFC1126 or was 1951 treated as a non-goal in RFC1126 but may now be significant, it will 1952 be discussed in further detail in this section. 1954 7.1 Topology 1956 7.1.1 Routers should be able to know and exploit the domain topology 1958 Routers need to know the domain topology. BGP today operates with a 1959 policy database, but does not provide a link state database for the 1960 connectivity of each AS - the extent to which this is feasible or 1961 desirable needs to be investigated. 1963 The OSI Inter-Domain Routing Protocol (IDRP) [36] utilized a related 1964 capability which allowed a collection of topologically related 1965 domains to be replaced by a domain collection object in a similar way 1966 to Nimrod domain hierarchies, allowing a route to be more compactly 1967 represented by a single collection in place of a sequence of 1968 individual domains. This abstraction and aggregation feature is 1969 similar to but somewhat more powerful than the BGP community 1970 capability. 1972 7.1.2 The same topology information should support different path 1973 selection ideas: 1975 The same topology information needs to provide a more flexible 1976 spectrum of path selection methods that we might expect to find in a 1977 future Internet, including, amongst others, both distributed 1978 techniques such as hop-by-hop, shortest path, local optimization 1979 constraint-based, class of service, source address routing, and 1980 destination address routing as well as the centralized, global 1981 optimization constraint-based �traffic engineering� type (Open 1982 constraints should be allowed). Allowing different path selection 1983 techniques to be used will produce a much more predictable and 1984 comprehensible result than the �clever tricks� that are currently 1985 needed to achieve the same results. Traffic engineering functions 1986 need to be combined. 1988 7.1.3 Separation between the routing information topology from the 1989 data transport topology. 1991 The controlling network should be logically separate from the 1992 controlled network. Physically, the two functional "planes" can 1993 reside in the same nodes and share the same links, but this is not 1994 the only possibility. Other options can also be feasible, and may 1995 sometimes be necessary. An example is a pure circuit switch (that 1996 cannot see individual IP packets), combined with an external 1998 Davies, et al Expires: January 2002 38 1999 controller. Another example may be where there are multiple links 2000 between two routers, and all the links are used for data forwarding, 2001 but only one is used for carrying the routing session. 2003 7.2 Distribution 2005 7.2.1 Distribution mechanisms 2007 The important requirement is that every entity gets the information 2008 it needs in a fast, reliable, and trusted way. 2010 Possible distribution mechanisms for routing information exchange may 2011 be for example full mesh, spanning tree, route reflections, flooding, 2012 and multicast. 2014 The current I-BGP seems to have unnecessary limitations in this 2015 respect, where a router requires full mesh to all other I-BGP 2016 speakers in the domain to obtain all available routes. Route 2017 reflection avoids the need of full meshes but requires very careful 2018 configuration to ensure that the best route available is still 2019 selected as if all routers were connected in a full mesh. 2021 7.2.2 Path advertisement 2023 The inter-domain routing system must be able to advertise more kinds 2024 of information than just connectivity and AS path. The FDR should 2025 support the Service Level Specifications (SLSs) that are being 2026 developed under the Differentiated Services imprimatur. 2028 Careful attention should be paid to ensuring that the distribution of 2029 additional information with path advertisements remains scalable as 2030 domains and the Internet get larger. 2032 Possible examples of such additional information that might be 2033 carried include: 2035 - QoS information 2037 To allow an ISP to sell predictable end-to-end QoS service to any 2038 destination, the routing system should have information about the 2039 end-to-end QoS. This means that the routing system should be able to 2040 support different paths for different services identified by DiffServ 2041 PDB�s or TOS-values. The routing system should also be able to carry 2042 information about the expected (or actually, promised) 2043 characteristics of the entire path and also the price for the 2044 service. (If such information is exchanged at all between network 2045 operators today, it is through bilateral management interfaces, and 2046 not through the routing protocols.) 2048 This would allow for the operator to optimise the choice of path 2049 based on a price/performance trade-off. 2051 It is possible that providing dynamic QoS information to control 2052 routing is not scalable, and an alternative would be to use static 2054 Davies, et al Expires: January 2002 39 2055 class-of-service information such as is suggested in the 2056 Differentiated Services work. 2058 - security information 2060 Security characteristics of other ASs (in the path or in the map) can 2061 allow the routing entity to choose routing decision based on some 2062 political reasons. The information itself is assumed to be so secure 2063 that you can trust it. 2065 - usage and cost information 2067 This can be used for billing and traffic engineering purpose. In 2068 order to support cost based routing policies for customers (ie peer 2069 ISPs), information such as "traffic on this link or path costs XXX 2070 USD per Gigabyte" needs to be advertised, so that the customer can 2071 choose a cheap or an expensive route from an economic perspective. 2073 - monitored performance 2075 Some performance information such as delay and drop frequency can be 2076 carried. (This is may only be suitable inside a domain because of 2077 trust considerations). This should support at least the kind of 2078 delay bound contractual terms that are currently being offered by 2079 service providers. Note that these values refer to the outcome of 2080 carrying bits on the path, whereas the QOS information refers to the 2081 proposed behaviour which results in this outcome. 2083 7.2.3 Stability of Routing Information 2085 7.2.3.1 Avoiding Routing Oscillations 2087 The FDR must minimize oscillations in route advertisements. 2089 7.2.3.2 Providing Loop Free Routing and Forwarding 2091 In line with the separation of concerns of routing and forwarding, 2092 the distribution of routing information should be, so far as is 2093 possible, loop-free, and the forwarding information created from this 2094 routing information should also seek to minimize persistent loops in 2095 the data forwarding paths. It is accepted that transient loops may 2096 occur during convergence of the protocol and that there are trade- 2097 offs between loop avoidance and global scalability. 2099 7.3 Addressing 2101 7.3.1 Support mix of IPv4, IPv6 and other types of addresses 2103 The routing system must support a mix of different kinds of 2104 addresses, including at least IPv4 and IPv6 addresses, and preferably 2105 various types of non-IP addresses too. For instance networks like 2106 SDH/SONET and WDM may prefer to use non-IP addresses. It may also be 2108 Davies, et al Expires: January 2002 40 2109 necessary to support multiple sets of �private� RFC1918 addresses 2110 when dealing with multiple customer VPNs. 2112 The routing system should support the capability to use a single 2113 topology representation to generate routing and forwarding tables for 2114 multiple address families on the same network. This capability would 2115 minimise the protocol overhead when exchanging routes. 2117 Note that both Integrated IS-IS and BGP with multi-protocol 2118 extensions [37] can support different address families. Extended BGP 2119 is used, for example, in RFC2547 [16] to carry the VPN-IPv4 address 2120 family. 2122 7.3.2 Support for domain renumbering/readdressing 2124 The routing system must support readdressing (when a new prefix is 2125 given to an old network, and the change is known in advance) by 2126 maintaining routing during the changeover period [39], [40]. 2128 7.3.3 Multicast and Anycast 2130 The routing system must support multicast addressing, both within a 2131 domain and across multiple domains. It must also support anycast 2132 addressing within a domain, and should support inter-domain anycast 2133 addressing. 2135 7.3.4 Address scoping 2137 The routing system must support scoping of addresses, for each of the 2138 unicast, multicast, and anycast types. 2140 For unicast address scoping as of IPv6, there seems to be no special 2141 problems with respect to routing. Inter-domain routing handles only 2142 global addresses, while intra-domain routing also needs to be aware 2143 of site-local addresses. Link-local addresses are never routed at 2144 all. 2146 For scoping in a more general sense, and for scoping of multicast and 2147 anycast addresses, more study may be needed to identify the 2148 requirements. 2150 7.3.5 Mobility Support 2152 The routing system shall support end system mobility (and movability, 2153 and portability, whatever the differences may be). 2155 We observe that the existing solutions based on re-numbering and/or 2156 tunneling are designed to work with the current routing, so they do 2157 not add any new requirements to future routing. But the requirement 2158 is general, and future solutions may not be restricted to the ones we 2159 have today. 2161 Davies, et al Expires: January 2002 41 2162 7.4 Management Requirements 2164 7.4.1 Simple policy management 2166 - Less manual configuration than today 2167 - Operators/providers want easy handling, but cannot afford to lose 2168 control. 2169 - All the information should be available 2170 - But should not be visible except for when desired. 2171 - Advertise policy (not only the result of policy) 2172 - Policy conflict Resolution 2174 (e g one would like to have one default behavior, and possibilities 2175 to choose other options. But much of this depends on implementation, 2176 and not on the protocols) 2178 7.5 Mathematical Provability 2180 The protocol is required to be resistant to bad routing policy 2181 decisions made by operators. Tools are needed to check compatibility 2182 of routing policies. Routing policies are compatible if their global 2183 interaction does not cause divergence (collection of ASes exchange 2184 routing messages indefinitely never entering a stable state). Tools 2185 must be provided to make routing system convergent. A routing system 2186 is convergent if after an exchange of routing information, routing 2187 tables reach a stable state that does not change until routing 2188 policies change. 2190 To achieve the above mentioned goals a mechanism is needed to publish 2191 and communicate policies so that operational coordination and fault 2192 isolation is possible. Tools are required that verify stable 2193 properties routing system in specified parts of Internet. The tools 2194 should be efficient (fast) and have a broad scope of operation (check 2195 large portions of Internet). 2197 Tools analyzing routing policies can be applied statically or 2198 (preferably) dynamically. Dynamic solution requires tools that can be 2199 used for run time checking for a source of oscillations that arise 2200 from policy conflicts. Research is needed to prove that there is an 2201 efficient solution to the dynamic checking of oscillations. 2203 7.6 Traffic Engineering 2205 7.6.1 Support for and Provision of Traffic Engineering Tools 2207 At present there is an almost total lack of effective traffic 2208 engineering tools, whether on-line at all times for network control 2209 or off-line for network planning. The routing system should 2210 encourage the provision of such tools and generate statistical and 2211 accounting information in such a way that these tools can be used 2212 both in real time and for off-line planning and management. 2214 Davies, et al Expires: January 2002 42 2215 7.6.2 Support of Multiple Parallel Paths 2217 The routing system shall support the controlled distribution over 2218 multiple links or paths, of traffic towards the same destination. 2219 This applies to domains with two or more connections to the same 2220 neighbor domain, and to domains with connections to more than one 2221 neighbor domain. The paths need not have the same metric. 2223 This capability should be provided to support both cases where the 2224 offered traffic is known to exceed the available capacity of a single 2225 link, and also cases where load is to be shared over paths for cost 2226 or resiliency reasons. 2228 Imposition of this requirement on the routing system requires that 2229 the corresponding forwarding should avoid reordering of packets in 2230 individual micro-flows, and should have mechanisms to allow the 2231 traffic to be reallocated back on to a single path when the multiple 2232 paths are not needed. 2234 7.6.3 Peering support 2236 The FDR must support peer-level connectivity as well as purely 2237 hierarchical inter-domain connections. The network is becoming 2238 increasingly complex with private peering arrangements set up between 2239 providers at every level of the hierarchy of service providers and 2240 even by certain large enterprises, in the form of dedicated 2241 extranets. 2243 The FDR must facilitate traffic engineering of these peer routes so 2244 that traffic can be readily constrained to travel as the network 2245 operators desire and they can consequentially make optimal use of the 2246 available connectivity. 2248 7.7 Support for NATs and other Mid-boxes 2250 One of our assumptions is that NATs and other Mid-boxes such as 2251 firewalls, web proxies and address family (e.g. IPv4 to IPv6) 2252 translators are here to stay. 2254 The FDR should seek to work with NATs to aid in bi-directional 2255 connectivity through the NAT without compromising the additional 2256 opacity and privacy which the NAT offers. This problem is closely 2257 analogous to the abstraction problem, which is already under 2258 discussion for the interchange of routing information between 2259 domains. 2261 7.8 Statistics support 2263 Both the routing and forwarding parts of the FDR must maintain 2264 statistical information about the performance of their functions. 2265 This may be an extended version of the MIBs provided for IP 2266 forwarding, BGP and the relevant IGP. 2268 Davies, et al Expires: January 2002 43 2269 8. Performance Requirements 2271 Over the past several years, the perfomance of the routing system has 2272 frequently been discussed. Some of the questions being asked 2273 include: 2274 - How fast does an AS converge (given that we understand what we 2275 mean by convergence)? How fast must domains converge? 2276 - How big are the Areas, the ASs? How big should domains be? How 2277 many peers should a BGP Speaker be able to cope with? Can the 2278 routing protocols manage domains of this size 2279 - How much or how little data may be transferred in a routing 2280 message? 2281 - How much state can be stored and processed in route control 2282 processors. 2283 - Measures of network availability 2284 - Measure of network reliability 2285 - Global and Local measures of network Stability 2286 - Capacity Measurement 2288 In many cases there has been very little data or statistical evidence 2289 for many of the performance claims being made. In recent years 2290 several efforts have been initiated to gather data and do the 2291 analyses required to make scientific assessments of the performance 2292 issues and requirements. In order to complete this section of the 2293 requirements analysis, the data and analyses from these studies needs 2294 to be gathered and collated into this document. This work has been 2295 started but has yet to be completed. 2297 9. Backwards compatibility (cutover) and Maintainability 2299 This area poses a dilemma. On one hand it is an absolute requirement 2300 that introduction of FDR must not require any flag days. The network 2301 currently in place has to keep running at least as well as it does 2302 now while the new network is being brought in around it. 2304 However, at the same time, it is also an absolute requirement that 2305 the new architecture not be limited by the restrictions that plague 2306 today�s network. Those restrictions cannot be allowed to become 2307 permanent baggage on the new architecture. If they do, the effort to 2308 create a new system will come to naught. 2310 These two requirements have significance not only for the transition 2311 strategy, but for the architecture itself implying that it must be 2312 possible for an internet such as today�s BGP controlled network, or 2313 one of its ASs, to exist as a domain within the new FDR. 2315 10. Security Requirements 2317 As previously discussed, one of the major changes to have overtaken 2318 the Internet since its inception, is the erosion of trust between end 2319 users making use of the net, between those users and the suppliers of 2320 services, and between the multiplicity of providers. Hence security, 2321 in all its aspects, will be much more important in the FDR. 2323 Davies, et al Expires: January 2002 44 2324 It must be possible to secure the routing communication: the 2325 communicating entities shall be able to identify who sent and who 2326 received the information (authentication), and verify that the 2327 information has not been changed on the way (integrity). 2329 Security is more important in inter-domain routing where the operator 2330 has no control to the other domains, and less serious in intra-domain 2331 routing since all the links and the nodes are under the 2332 administration of the operator and can be expected to share a trust 2333 relationship. 2335 The routing communication mechanism shall be robust against denial- 2336 of-service attacks. 2338 Further considerations which may impose requirements include: 2339 - Whether no one else but the intended recipient must be able to 2340 access (privacy) or understand (confidentiality) the information. 2341 - Whether it is possible to verify that all the information has been 2342 received (non-repudiation). 2343 - Whether there is a need to separate security of routing from 2344 security of forwarding. 2345 - Whether traffic flow security is needed (i.e. whether there is 2346 value in concealing who can connect to whom, and what volumes of 2347 data are exchanged). 2349 Securing the BGP session, as done today, only secures the exchange of 2350 messages from the peering AS, not the content of the information. In 2351 other words, we can confirm that the information we got is what our 2352 neighbor really sent us, but we do not know if this information (that 2353 originated in some remote AS) is true or not. 2355 A decision has to be made on whether to rely on chains of trust (we 2356 trust our peers who trust their peers who..), or whether we also need 2357 authentication and integrity of the information end-to-end. This 2358 information includes both routes and addresses. There has been 2359 interest in having digital signatures both on originated routes, but 2360 also countersignatures by address authorities to confirm that the 2361 originator has authority to advertise the prefix. Even understanding 2362 who can confirm the authority is non-trivial as it might be the 2363 provider who delegated the prefix (with a whole chain of authority 2364 back to ICANN) or it may be straight to an address registry. Where a 2365 prefix delegated by a provider is being advertised though another 2366 provider as in multi-homing, both may have to be involved to confirm 2367 that the prefix may be advertised through the provider who doesn�t 2368 have any interest in the prefix! 2370 The FDR should seek to cooperate with the security policies of 2371 firewalls and other third-party controlled mid-boxes whenever 2372 possible. This is likely to involve further requirements for 2373 abstraction of information, as, for example, the firewall is seeking 2374 to minimize interchange of information that could lead to a security 2375 breach. The effect of such changes on the end-to-end principle 2376 should be carefully considered as discussed in [32]. 2378 Davies, et al Expires: January 2002 45 2379 Provision may have to be made to override some of these requirements 2380 when local laws mandate interception of communication capabilities. 2382 11. Open Issues 2384 This section covers issues that need to be considered and resolved in 2385 deciding on a future domain routing architecture. While they can�t 2386 be described as requirements, they do affect the types of solution 2387 that are acceptable. The discussions included below are very open- 2388 ended. 2390 11.1 System Modeling 2392 The assumption that object modeling of a system is an essential first 2393 step to creating a new system is still novel in this context. 2394 Frequently the effort to object model becomes an end in itself and 2395 does not lead to system creation. But there is a balance and a lot 2396 that can be discovered in an ongoing effort to model a system such as 2397 the future domain routing system. 2399 It is recommended that this process be included in the requirements. 2400 It should not, however be a gating event to all other work. 2402 Some of the most important realizations will occur during the process 2403 of determining the following: 2404 - Object classification 2405 - Relationships and containment 2406 - Roles and Rules 2408 11.2 Advantages and Disadvantages of having the same protocols for EGP 2409 and IGP 2411 Inter-domain and intra-domain routing have different targets and 2412 business assumptions. An IGP figures out how each node in the network 2413 gets to every other node in the network in the most optimal way. In 2414 this context the word optimal refers to the cost of the path measured 2415 by metrics associated with each link in the network. The area of 2416 network infrastructure (primarily routers) over which an IGP runs is 2417 typically under the same technical and administrative control, and it 2418 defines the boundary of an AS (Autonomous System). The purpose of an 2419 EGP is to allow two different ASs to exchange routing information so 2420 that data traffic can be forwarded across the AS border. Because an 2421 AS border router both separates and attaches two different areas of 2422 technical and administrative control, the specifications and 2423 implementations of EGPs include mechanisms for doing policy routing, 2424 meaning that control can be exerted over which routing information 2425 crosses the border between two ASs. EGPs contain features that are 2426 like metrics in IGPs, but unlike IGPs, the function of an EGP is not 2427 necessarily to optimize the path that data traffic takes through a 2428 backbone. Having different protocols for EGP and IGP reflects this 2429 difference. 2431 However, there is increasing demand in IGP to do policy routing. The 2432 shortest path may not be the best path in the light of the policies. 2434 Davies, et al Expires: January 2002 46 2435 Network operators need to have more flexibility in choosing routes 2436 for reasons such as load balancing. This means both inter-domain 2437 routing and intra-domain routing are for the same purpose of choosing 2438 the best route according to operators' own policies. Having the same 2439 protocol will emphasize the need to do policy control in IGP. 2441 This comment touches on the fact that the level of manual control 2442 (policy) is much larger in EGP. Why is this so? 2444 EGP: 2445 - Manifests business relations to peers, providers and customers. 2446 - Borders to resources outside of our control. We don't trust others 2447 to behave well when configuring routing. These resources are also 2448 often be less stable (eg customer access). 2449 - Network size extremely large. This gives many updates which means 2450 we need to have a simple calculation of paths. It also gives an 2451 extremely large amount of information (due to the network size) 2452 which gives the need for aggregation. Also we need policy to 2453 protect our network from receiving bad announcements causing our 2454 egress traffic to take the "wrong" way and to avoid sending bad 2455 announcements attracting the "wrong" traffic. 2457 IGP: 2458 - The network resources are under our control and we trust ourself 2459 to behave well (in a sense defined by ourselves) when configuring 2460 routing. 2461 - The network resources of today are fairly stable in a backbone 2462 network. 2463 - The size of the network is limited. So, the domain is fairly 2464 stable which gives a limited number of updates. Limited number of 2465 updates gives the option of using processor intensive automation 2466 (distributed link state routing). This gives us fast and easy to 2467 manage dynamic routing. BUT stability and visibility issues still 2468 constrain us from going further down the path of policy routing. 2470 11.2.1 The necessity to clearly identify all identities related to 2471 routing 2473 As in all other fields, the words used to refer to concepts and to 2474 describe operations about routing are important. Rather than describe 2475 concepts using terms that are inaccurate or rarely used in the real 2476 world of networking, it is necessary to make an effort to use the 2477 correct words. Many networking terms are used casually, and the 2478 result is a partial or incorrect understanding of the underlying 2479 concept. Entities such as nodes, interfaces, sub-networks, tunnels, 2480 and the grouping concepts such as ASs, domains, areas, and regions, 2481 need to be clearly identified and defined to avoid mixing from each 2482 other. And, even if they are all identified by IP numbers, the 2483 routing entities should know what kind of entities they are. 2485 There is also a need to separate identifiers (what or who) from 2486 locators (where) from routes (how to reach). One of the problems with 2487 the current BGP is if there is a topology change, the amount of 2488 information circulated is a function of the number of IP prefixes 2490 Davies, et al Expires: January 2002 47 2491 being routed. This is a common problem for a distance vector 2492 protocol. If the topology information is properly separated from 2493 addressing information in a state map, then when a link between two 2494 ASs goes down, this is the only information which needs to be 2495 advertised, instead of advertising the inability to reach some 2496 network prefixes. This example shows the need to separate end node 2497 identifiers from routing information. 2499 11.2.2 Map distribution and/or route Distribution 2501 11.2.2.1 Class of protocol to use 2503 BGP4 is an enhanced distance vector protocol, which is the oldest and 2504 least sophisticated type of mechanism for distributing routes. It 2505 would be possible to retain the basic route distribution mechanism 2506 but use an improved convergence mechanism such as is described in 2507 [34]. 2509 Alternatively, it would be possible to move to the more sophisticated 2510 Map Distribution class of protocol such as PNNI. This has some 2511 advantages in that it probably easier to isolate the routing 2512 mechanisms on a per domain basis when exchanging information by maps 2513 which are a more sophisticated data structure. 2515 11.2.2.2 Map Abstraction 2517 If every detail is advertised throughout the Internet, there will be 2518 a lot of information. Scalable solutions require abstraction. 2520 - If we summarise too much, some information will be lost on the 2521 way. 2523 - If we summarize too little, then more information then required is 2524 available contributing to scaling limitations. 2526 - One can allow more summarisation, if there also is a mechanism to 2527 query for more details within policy limits. 2529 - The basic requirement is not that the information shall be 2530 advertised, but that the information shall be available to those 2531 who need it. (We should not presuppose a solution where 2532 advertising is the only possible mechanism.) 2534 11.2.3 Robustness and redundancy: 2536 The routing association between two domains should survive even if 2537 some individual connection between two ASBR routers goes down. 2539 The "session" should operate between logical "routing entities" on 2540 each domain side, and not necessarily be bound to individual routers 2541 or IP addresses. Such a logical entity can be physically distributed 2542 over multiple network elements. Or it can reside in a single router, 2543 which would default to the current situation. 2545 Davies, et al Expires: January 2002 48 2546 11.2.4 Hierarchy 2548 A more flexible hierarchy with more levels and recursive groupings in 2549 both upward and downward directions allows more structured routing. 2550 The consequence is that no single level will get too big for routers 2551 to handle. 2553 On the other hand, it appears that the real world Internet is 2554 becoming less hierarchical, so that it will be increasingly difficult 2555 to use hierarchy for scaling control. 2557 Note that groupings can look different depending on which aspect we 2558 use to define them. A DiffServ area, a MPLS domain, a trusted domain, 2559 a QoS area, a multicast domain, etc, do not always coincide. And 2560 neither are they strict hierarchical subsets of each other. The basic 2561 distinction at each level is "this grouping versus everything 2562 outside". 2564 Each AS is still independent, and forms the basis for policy 2565 decisions. However, is there a need for a higher level aggregation 2566 which is above AS? If yes, who will be responsible for this level? 2567 Can a network make policy decisions on such aggregated ASs without 2568 seeing the individual ASs? 2570 11.3 Introduction of new control mechanisms 2572 Is it be possible to apply a control theory framework, and analyze 2573 the stability of the control system of the whole network domain, for 2574 e.g. convergence speed and the frequency response, and then use the 2575 results from that analysis to set the timers and other protocol 2576 parameters. 2578 Control theory could also play a part is QoS Routing, by modifying 2579 current link state protocols with link costs dependent on load. 2580 Control theory is used to increase the stability of such systems. 2582 At best, it might be possible to construct a new totally dynamical 2583 routing protocol solely on a control theoretic basis as opposed to 2584 the current protocols which are based in graph theory and static in 2585 nature. 2587 11.4 Robustness 2589 Is solution to the Byzantine Generals problem a requirement? This is 2590 problem of reaching a consensus among distributed units if some of 2591 them give misleading answers. The original problem concerns generals 2592 plotting a coup. Some generals lie about whether they will support a 2593 particular plan and what other generals told them. What percentage of 2594 liars can a decision-making algorithm tolerate and still correctly 2595 determine a consensus? The current intra-domain routing system is, 2596 at one level, totally intolerant of misleading information, but the 2597 effect of different sorts of misleading or incorrect information has 2599 Davies, et al Expires: January 2002 49 2600 vastly varying results, from total collapse after the �small Virginia 2601 ISP� incident through to purely local disconnection of a single AS. 2602 This sort of behaviour is not very desirable. 2604 What are some of the other network robustness issues that must be 2605 resolved? 2607 11.5 VPN Support 2609 Today BGP is also used for VPN and other stuff for example as 2610 described in RFC2547 2612 Internet routing and VPN routing have different purposes, and most 2613 often exchange different information between different devices. Most 2614 Internet routers do not need to know any VPN specific information. 2615 The concepts should be clearly separated. 2617 But when it comes to the mechanisms, VPN routing can share the same 2618 protocol as ordinary Internet routing, it can use a separate instance 2619 of the same protocol, or it can use a different protocol. All 2620 variants are possible and have their own merits. 2622 For example, all the AS Border Routers within one AS participate in a 2623 full-mesh I-BGP process for distributing external IP routes. At the 2624 same time a separate "VPN-routing" protocol can be operating between 2625 all the PE routers of some "VPN provider". These PE routers can be 2626 located in different ASs, and some of them may also be ASBRs. 2628 11.6 End to End Reliability 2630 The existing Internet architecture neither requires or provides end- 2631 to-end reliability of control information dissemination. For 2632 example, in distributing VPN information there is, however, a 2633 requirement for end to end reliability of control information, i.e. 2634 the ends of the VPN established need to have a acknowledgement of the 2635 success in setting up the VPN. While it is not necessarily the 2636 function of a routing architecture to provide end-to-end reliability 2637 for this kind of purpose, we must be clear that end-to-end 2638 reliability becomes a requirement if the network has to support such 2639 reliable control signalling. There may be other requirements that 2640 derive from requiring the FDR to support reliable control signaling. 2642 12. Acknowledgements 2644 The authors would like to acknowledge the helpful comments and 2645 suggestions of the following individuals: Loa Andersson, Tomas 2646 Ahlstr�m, Niklas Borg, Nigel Bragg, Thomas Chmara, Krister Edlund, 2647 Owe Grafford, Torbj�rn Lundberg, Jasminko Mulahusic, Florian-Daniel 2648 Otel Bernhard Stockman, Henrik Villf�r, Tom Worster, Roberto 2649 Zamparo,. 2651 In addition, the authors are indebted to the folks who wrote all the 2652 references we have consulted in putting this paper together. This 2654 Davies, et al Expires: January 2002 50 2655 includes not only the reference explicitly listed below, but those 2656 who contributed to the mailing lists we have been participating in 2657 for years. 2659 13. References 2661 [1] Clark, D., "Policy Routing in Internet 2662 Protocols", RFC 1102, May 1989. 2664 [2] Estrin, D., "Requirements for Policy Based 2665 Routing in the Research Internet", RFC 1125, 2666 November 1989. 2668 [3] Steenstrup, M,. ��An Architecture for Inter- 2669 Domain Policy Routing��, RFC 1478, June 1993 2671 [4] Little, M., "Goals and Functional Requirements 2672 for Inter-Autonomous System Routing", RFC 1126, 2673 July 1989. 2675 [5] Perlman, R., ��Interconnections Second Edition��, 2676 1999, Addison Wesley Longman, Inc. 2678 [6] Perlman, R., "Network Layer Protocols with 2679 Byzantine Robust-ness", Ph.D. Thesis, Department 2680 of Electrical Engineering and Computer Science, 2681 MIT, August 1988. 2683 [7] Castineyra, I., Chiappa, N., Steenstrup, M., 2684 ��the Nimrod Routing Architecture��, RFC1992, Aug 2685 1996 2687 [8] Chiappa, N., ��IPng Technical Requirements of the 2688 Nimrod Routing and Addressing Architecture��, RFC 2689 1753, Dec 1994 2691 [9] Chiappa, N., ��A New IP Routing and Addressing 2692 Architecture�� 2694 [10] Wroclowski, J., The Metanet White Paper - 2695 Workshop on Research Directions for the Next 2696 Generation Internet, 1995 2698 [11] Labovitz, C., Ahuja, A., Farnam J., Bose, A., 2699 Experimental Measurement of Delayed Convergence, 2700 NANOG 2702 [12] Griffin, T.G., Wilfong, G., An Analysis of BGP 2703 Convergence Properties, SIGCOMM 1999 2705 [13] Huston, G., Architectural Requirements for Inter- 2706 Domain Routing in the Internet, Internet Draft - 2707 draft-iab-bgparch-00, Feb 2001, Work in Progress 2709 Davies, et al Expires: January 2002 51 2711 [14] Alaettinoglu, C., Jacobson, V. and Yu, H, , 2712 Towards Milli-Second IGP Convergence, Internet 2713 Draft - draft-alaettinoglu-isis-convergence-00, 2714 Nov 2000 Work in Progress 2716 [15] Sandick, H., Squire, M., Cain, B., Duncan, I., 2717 Haberman, B., Fast LIveness Protocol (FLIP), 2718 Internet Draft - draft-sandiick-flip-00, 2719 Feb 2000, Work in Progress 2721 [16] Rosen, E. and Rekhter, Y., BGP/MPLS VPNs, 2722 RFC2547, March 1999 2724 [17] Clark, D., Chapin, L., Cerf, V., Braden, R., 2725 Hobby, R., ��towards the Future Internet 2726 Architecture��, RFC1287, December 1991 2728 [18] Jacobson, V., Nichols, K. and Poduri, K., The 2729 �Virtual Wire� Behavior Aggregate, Internet Draft 2730 - draft-ietf-diffserv-pdb-vw-00, July 2000, Work 2731 in Progress 2733 [19] Seddigh, N., Nandy, B., and Heinanen, J., 2734 An Assured Rate Per-Domain Behaviour for 2735 Differentiated Services, Internet Draft - 2736 draft-ietf-diffserv-pdb-ar-00, Feb 2001, Work in 2737 Progress 2739 [20] McPherson, D., Gill, V., Walton, D. and Retana, 2740 A., ��BGP Persistent Route Oscillation 2741 Condition��, 2742 Internet Draft - draft-mcpherson-bgp-route- 2743 oscillation-00, Dec 2000, Work in Progress 2745 [21] Hain, T, ��Architectural Implications of NAT��, 2746 RFC 2993, November 2000 2748 [22] McPherson, D. and Przygienda, T., OSPF Transient 2749 Blackhole Avoidance, Internet Draft - draft- 2750 mcpherson-ospf-transient-00, July 2000 Work In 2751 Progress 2753 [23] Thaler, D., Estrin, D. and Meyer, D. (editors), 2754 Border Gateway Multicast Protocol (BGMP): 2755 Protocol Specification, Internet Draft - draft- 2756 ietf-bgmp-spec-02, Nov 2000 Work in progress 2758 [24] Rosen, E. Et al., Multiprotocol Label Switching 2759 Architecture, RFC 3031 2761 [25] Ashwood-Smith, P. Et al., Generalized MPLS - 2762 Signaling Functional Description, Internet Draft 2763 - draft-ietf-mpls-generalized-signaling-01.txt, 2764 Work in progress 2766 Davies, et al Expires: January 2002 52 2768 [26] IETF Resource Allocation Protocol working group, 2769 http://www.ietf.org/html.charters/rap- 2770 charter.html 2772 [27] IETF Configuration management with SNMP working 2773 group, 2774 http://www.ietf.org/html.charters/snmpconf- 2775 charter.html 2777 [28] IETF Policy working group, 2778 http://www.ietf.org/html.charters/policy- 2779 charter.html 2781 [29] Yu J., ��Scalable Routing Design Principles��, 2782 RFC 2791 2784 [30] Telcordia Technologies Netsizer web site 2785 http://www.netsizer.com/ 2787 [31] Inference of Shared Risk Link Groups, 2788 draft-many-inference-srlg-00.txt, 2789 Work in progress 2791 [32] Blumenthal, Marjory S. and Clark, David D., 2792 Rethinking the design of the Internet: 2793 The end to end arguments vs. the brave new world, 2794 May 2001, 2795 http://ana-www.lcs.mit.edu/anaweb/papers.html 2797 [33] Lang et al, Link Management Protocol, 2798 draft-lang-mpls-lmp-02.txt, 2799 Work in progress 2801 [34] Xu, Z., Dai, S. and Garcia-Luna-Aceves, J.J., 2802 A More Efficient Distance Vector Routing 2803 Algorithm, Proc. IEEE MILCOM 97, Monterey, 2804 California, November 2-5, 1997, 2805 http://www.cse.ucsc.edu/research/ccrg/ 2806 publications/zhengyu.milcom97.pdf 2808 [35] Bradner, S. and Mankin, A., "The Recommendation 2809 for the IP Next Generation Protocol", RFC 1752, 2810 January 1995. 2812 [36] ISO/IEC, "Protocol for Exchange of Inter-Domain 2813 Routeing Information among Intermediate 2814 Systems to support Forwarding of ISO 8473 PDUs", 2815 International Standard 10747, 2816 ISO/IEC JTC 1,Switzerland 1993 2818 [37] Bates, T., Rekhter, Y., Chandra, R. and Katz, D, 2819 ��Multiprotocol Extensions to BGP-4�, 2820 RFC2858, June 2000 2822 Davies, et al Expires: January 2002 53 2824 [38] Berkowitz, H. and Krioukov, D, ��To Be 2825 Multihomed: Requirements and Definitions��, 2826 draft-berkowitz-multirqmt-02.txt, 2827 Work in progress. 2829 [39] Ferguson, P. and Berkowitz, H. ��Network 2830 Renumbering Overview: Why would I want it and 2831 what is it anyway?��, RFC2071, January 1997 2833 [40] Berkowitz, H., ��Router Renumbering Guide��, 2834 RFC2072, January 1997 2836 14. Author's Addresses 2838 Elwyn Davies 2839 Nortel Networks 2840 London Road 2841 Harlow, Essex CM17 9NA, UK 2842 Phone: +44-1279-405498 2843 Email: elwynd@nortelnetworks.com 2845 Avri Doria 2846 Nortel Networks 2847 600 Technology Park Drive 2848 Billerica, MA, USA 2849 Phone: +1 978 288 6627 2850 Email: avri@nortelnetworks.com 2852 Howard Berkowitz 2853 Nortel Networks 2854 5012 South 25th St 2855 Arlington 2856 VA 22206, USA 2857 Phone: +1 703 998-5819 2858 Email: hcb@clark.net/hberkowi@nortelnetworks.com 2860 Dmitri Krioukov 2861 Nortel Networks 2862 1st Floor 2863 205 van Buren Street 2864 Herndon 2865 VA 20170, USA 2866 Phone: +1 703 709 8518 2867 Email: dima@nortelnetworks.com 2869 Davies, et al Expires: January 2002 54 2870 Malin Carlzon 2871 Royal Institute of Technology 2872 Network Operating Centre 2873 KTHNOC 2874 SE-100 44 2875 Stockholm, Sweden 2876 Phone: +46 70 269 6519 2877 Email: malin@sunet.se 2879 Anders Bergsten 2880 Telia Research AB 2881 Aurorum 6 2882 S-977 75 Lulea, SWEDEN 2883 Phone: +46 920 754 50 2884 Email: anders.p.bergsten@telia.se 2886 Olle Pers 2887 Telia Research AB 2888 Stockholm, SWEDEN 2889 Phone: +46 8 713 8182 2890 Email: olle.k.pers@telia.se 2892 Yong Jiang 2893 Telia Research AB 2894 123 86 Farsta SWEDEN 2895 Phone: +46 8 713 8125 2896 Email: yong.b.jiang@telia.se 2898 Lenka Carr Motyckova 2899 Div. of Computer 2900 Lulea University of Technology 2901 S-971 87 2902 Lulea, SWEDEN 2903 Phone: (+46) 920 91769 2904 Email: lenka@sm.luth.se 2906 Pierre Fransson 2907 Div. of Computer 2908 Lulea University of Technology 2909 S-971 87 2910 Lulea, SWEDEN 2911 Phone: (+46) 70 646 0384 2912 Email: pierre@cdt.luth.se 2914 Olov Schelen 2915 Div. of Computer 2916 Lulea University of Technology 2917 S-971 87 2918 Lulea, SWEDEN 2919 Phone: (+46) 70 536 2030 2920 Email: Olov.Schelen@cdt.luth.se 2922 Davies, et al Expires: January 2002 55 2923 Tove Madsen 2924 Utfors Bredband AB 2925 R�sundav�gen 12 2926 P.O. Box 525 2927 SE-169 29 Solna 2928 Sweden 2929 Phone: +46 (8) 5270 5040 2930 Email: tove.madsen@utfors.se 2932 Davies, et al Expires: January 2002 56