Network Working Group G. Bernstein Internet Draft Grotto-Networking Expiration Date: August 2003 B. Rajagopalan Document: draft-ietf-ipo-optical-inter-domain-02.txt D. Pendarakis Tellium Angela Chiu John Strand AT&T V. Sharma Metanoia Dean Cheng Polaris Rauf Izmailov NEC Lyndon Ong Ciena Sudheer Dharanikota Frank Hujber February 2003 Optical Inter Domain Routing Considerations Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 [1]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This draft investigates the requirements for general inter-domain and inter-area routing in optical networks and reviews the applicability of existing route protocols in various optical routing applications. Table of Contents: 1 Introduction 2 1.1 Specification of Requirements 3 1.2 Abbreviations 3 Bernstein, G. [Page 1] draft-ipo-optical-inter-domain-02.txt February 2003 2 Background 3 2.1 Basic Concept of Domains and Network Partitioning 3 2.2 Major Differences between Optical and IP datagram Routing 6 2.3 Differences between MPLS and Optical Circuit routing 7 2.4 Diversity in Optical Routing 7 2.4.1 Generalizing Link Diversity 9 2.4.2 Generalizing Node Diversity 9 2.5 Routing Information Categorization 10 2.5.1 Link and Topology Related Information 10 2.5.2 Domain and Node Related Information 11 3 Applications of Inter Domain Optical Routing 12 3.1 Intra Carrier Applications of Optical Inter Domain Routing 12 3.1.1 Intra-Carrier Scalability 12 3.1.2 Intra-Carrier Inter-vendor 13 3.1.3 Inter-Layer Partitioning 16 3.1.4 Interaction with IP Layer Routing 17 3.1.5 Inter-Business Unit 18 3.2 Inter-Carrier Inter-Domain Optical Routing 19 3.3 Multi-Domain Connection Control 21 4 Multiple Layers of Optical Routing 22 5 Conclusion 24 6 Security Considerations 24 6.1.1 24 7 References 25 8 Acknowledgments 26 9 Author's Addresses 26 NOTE: This draft has been updated with more current author contact information and references, but is otherwise unchanged from the previous version. 1 Introduction Multi Protocol Label Switching (MPLS) has received much attention recently for use as a control plane for non-packet switched technologies. In particular, optical technologies have a need to upgrade their control plane as reviewed in reference [2]. Many different optical switching and multiplexing technologies exist and more are sure to come. For the purposes of this draft we only consider non-packet (i.e. circuit switching) forms of optical switching. As the requirements for and extensions to interior gateway protocols such as OSPF and IS-IS have begun to be investigated in the single area case, e.g., reference [3], we consider the requirements that optical networking and switching impose in the inter-domain case. First we explore the definition of a control domain and its use in optical and transport networking. This is explored again later in the document when we consider a number of important applications, networking scenarios, where the notions of inter-control domain routing are important. We review the various differences between routing in the datagram case, the virtual circuit case and the (real) optical circuit case. Then we look at the optical path selection/computation needs to provide for path diversity, switching Bernstein, G. [Page 2] draft-ipo-optical-inter-domain-02.txt February 2003 capabilities, transport capabilities and impairments, and bandwidth/resource status reporting. To add to the concreteness of these considerations we try to illustrate them with one or more specific examples from a particular optical networking layer or technology. This is not to reduce the generality of the requirement but to facilitate the understanding of the requirement or concept. 1.1 Specification of Requirements The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT","SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. 1.2 Abbreviations LSP Label Switched Path (MPLS terminology) LSR Label Switched Router (MPLS terminology) MPLS Multiprotocol Label Switching SDH Synchronous Digital Hierarchy (ITU standard) SONET Synchronous Optical NETwork (ANSI standard) STM(-N) Synchronous Transport Module (-N) STS(-N) Synchronous Transport Signal-Level N (SONET) TU-n Tributary Unit-n (SDH) TUG(-n) Tributary Unit Group (-n) (SDH) VC-n Virtual Container-n (SDH) VTn Virtual Tributary-n (SONET) 2 Background 2.1 Basic Concept of Domains and Network Partitioning In this draft we use the term domain in a general sense, i.e., beyond the BGP interpretation of an Autonomous System. In this draft we will consider domains as the result of partitioning of a network into subnetworks, as shown in the network of Figure 1. A network may be partitioned for a variety of reasons, such as: * Administrative boundaries * Scalability of routing or signaling * Isolation of partitions for security or reliability reasons * Technology differences in the systems in different domains /------------------------------------\ / \ / /-\ \ | Domain |NE2|<--Internal Node | | B /\-/\ | | / \ | | /-\/ \/-\ | Bernstein, G. [Page 3] draft-ipo-optical-inter-domain-02.txt February 2003 | |NE1|-------|NE3| / \+------\-/ \-/ / /\ | | / / \------+-----------+-----------------/ / | |<--- Inter-Domain Links / /------+-----------+-----------------\ | / | | \ | / | /-\ | /-\ \ | | Domain | |NE2|---+---------|NE3| | | | A | /\-/\ | ------/\-/ | | | | / \ | / | | | | /-\/ \/-\/ | | | | |NE1|-------|NE5|---- | / | \ -- \-/ \-/ \ /-\ / | \ / | | --|NE4| / | \/ | | \-/ / | /\-----+-----------+----------------/ Border Nodes | |<--- Inter-Domain Links \ |<----------+---- (External Links) /\------+-----------+-----------------\ / \ | | \ / \ /-\ /-\ | | --|NE1|-----|NE2| | | \-/ \-/ | | Domain | |<--- Intra-Domain | | C | | Links (Internal| | /-\ /-\ Links) | | |NE3|-----|NE3| | | \-/ \-/ | \ / \------------------------------------/ Figure 1. Partitioning a network into domains. The Inter-Domain interface is likely to have different characteristics than the Intra-Domain interface as the domain boundary exists for the purpose of hiding some aspect within the domain from the outside world. For the remainder of this draft we will be concerned with "control domains" that is a control domain will be hiding the internal details of its control plane from other control domains. This definition has a number of important ramifications when we consider inter-domain protocols for routing or signaling. 1. Inter-control domain routing or signaling is required to be independent of a control domainĖs internal signaling or route protocol or lack of either a signaling or routing protocol. This is sometimes referred to as the "domain independence" property. Bernstein, G. [Page 4] draft-ipo-optical-inter-domain-02.txt February 2003 2. Inter-control domain routing or signaling can not count on specific protocols being implemented within the domain. 3. Inter-control domain routing or signaling can not require that a network element participate in the internal protocols of two different control domains. This is sometimes refered to as "boundary on the wire" property. 4. The Inter-control domain routing or signaling protocol does not require that the control domains be interconnected in any particular fashion (except for general graph connectivity). Example #1 (inter-control domain routing) Consider a collection of BGP Autonomous Systems (AS). Each of these may run their own internal IGP. Of these internal IGPs, OSPFv2 and IS-IS are very popular but also incompatible with each other. BGP does not rely on the Autonomous Systems running a particular IGP, i.e., OSPF or IS-IS. Hence it meets the definition of an inter-control domain routing protocol. Note that nothing prevents BGP from being used within an ISP to glue together islands of incompatible IGPs. In fact, the availability of private AS numbers can help in this situation. Example #2 (OSPF Areas) An OSPF Area, on the other hand, is a mechanism within OSPF for providing scalability. It assumes that within each OSPF area that OSPF is being run in addition it requires that all border routers participate in both the backbone area and one or more other their own areas. Hence OSPF with its area concept equated with a control domain doesn't meet the definition of an inter-control domain routing protocol in two ways. It would require OSPF to run within the domains and it requires that a network element (router) participate in the internal routing protocol of two separate domains. Example #3 (PNNI routing Peer Groups) With ATM's PNNI routing hierarchy the concept of a peer group is introduced. PNNI with peer groups does not meet the definition of an a inter-control domain routing protocol since it, like OSPF, requires that within a peer group ATM PNNI routing is running. However, it doesnĖt require that border members of a peer group at a given level in the hierarchy participate in the internal protocols of both peer groups. In other words PNNI's hierarchical routing allows for a "boundary on the wire" model. Bernstein, G. [Page 5] draft-ipo-optical-inter-domain-02.txt February 2003 The domain concept as used here is orthogonal to the transport network concept of layering. When the term layer is used with respect to the transport network we are not referring to the 7- layer OSI model which includes application, presentation, etc., layers. With regard to this model all the optical transport layers would lie at the "physical layer". In the transport network, layers are used for multiplexing, performance monitoring and fault management purposes. Layers tend to be very technology specific. At some point an optical routing protocol must include information particular to the technology layer for which it is being used to acquire/disseminate topology and resource status information. For more information on layering and domain concepts see ITU-T recommendation G.805 [14]. 2.2 Major Differences between Optical and IP datagram Routing Let us first review the major difference between routing for optical (circuit switched networks) and IP datagram networks. In IP datagram networks packet forwarding is done on a hop-by-hop basis for each packet(no connection established ahead of time), while in circuit switched optical networks end-to-end connections must be explicitly established based on network topology and resource status information. This topology and resource status information can be obtained via routing protocols. Note that the routing protocols in the circuit switch case are not involved with data (or bit) forwarding, i.e., they are not "service impacting", while in the IP datagram case the routing protocols are explicitly involved with data plane forwarding decisions and hence are very much service impacting. Note that signaling or label distribution protocols that set up or tear down the virtual or real circuits are service impacting. This does not imply routing is unimportant in the optical case, only that its service impacting effect is secondary. For example, topology and resource status inaccuracies will affect whether a new connection can be established (or a restoration connection can be established) but will not (and should not) cause an existing connection to be torn down. This tends to lead to a slightly different view towards incorporating new information fields (objects, LSA, etc.) into optical routing protocols versus IP routing protocols. In the optical circuit case, any information that can potentially aid in route computations or be used in service differentiation may be incorporated into the route protocol, as either a standard element or a vendor specific extension. Whether a route computation algorithm uses this information and whether two route computation algorithms use this information in the same way doesnĖt matter since the optical connections are explicitly routed (although perhaps loosely). Bernstein, G. [Page 6] draft-ipo-optical-inter-domain-02.txt February 2003 Path selection or route computation in transport case may be based on a different criteria or combination of criteria on a per path basis. For example one connection may requests the lowest economic cost available path while another may request the highest reliability path. One important evaluation criteria for any proposed inter-domain optical routing protocol is the variety of path selection criteria with which it may work. 2.3 Differences between MPLS and Optical Circuit routing The bandwidth accounting needed in optical circuit-switched networks is also different than in packet networks. In packet networks using either ATM QoS or MPLS-TE, complex statistical measures are used to characterize the load on a link, often with varying degrees of accuracy. The inexactness of such measures and the ††compressibilityĖĖ of statistically multiplexed traffic imply that a small percentage change in link utilization can usually be absorbed by the network. By contrast, if an OC-192 link has just one STS-1 path occupied (less than 1% of the link bandwidth), it cannot accommodate an STS- 192c path. Due to the relatively simple finite multiplex structures currently use in optical networks tracking bandwidth resources is much easier than packet switched networks, however much stricter bandwidth accounting is required on circuit switched links. In particular, it is expected that an individual optical circuit switched link can be fully utilized, while due to queuing effects a packet switched link on average can never be run at full capacity and is typically run at less then 80% of capacity. This also affects how protection (or restoration) bandwidth can be committed. In a packet-based network, while the protection path can be preconfigured, resources along it are not used until a failure occurs. In circuit-based networks, a protection path generally implies a committed resource. Such basic differences restrict the direct applicability of some of the traffic engineering mechanisms used in packet-switched networks to circuit-switched networks. 2.4 Diversity in Optical Routing There are two basic demands that drive the need to discover diverse routes for establishing optical paths: 1. Reliability/Robustness 2. Bandwidth capacity. Many times multiple optical connections are set up between the same end points. An important constraint on these connections is that they must be diversely routed in some way [4]. In particular they could be routed over paths that are link diverse, i.e., two connections do not share any common link. Or the more stringent constraint that the two paths should be node diverse, i.e., the two paths do not traverse any common node. Bernstein, G. [Page 7] draft-ipo-optical-inter-domain-02.txt February 2003 Additionally, insufficient bandwidth may exist to set up all the desired connection across the same path (set of links) and hence we need to know about alternative (diverse) ways of reaching the destination that may still have unused capacity. "Diversity" is a relationship between lightpaths. Two lightpaths are said to be diverse if they have no single point of failure. In traditional telephony the dominant transport failure mode is a failure in the interoffice plant, such as a fiber cut inflicted by a backhoe. Data network operators have relied on their private line providers to ensure diversity and so IP routing protocols have not had to deal directly with the problem. GMPLS makes the complexities handled by the private line provisioning process, including diversity, part of the common control plane and so visible to all. Diversity is discussed in the IPO WG document [5]. A key associated concept, "Shared Risk Link Groups", is discussed in a number of other IETF (refs) and OIF (refs) documents. Some implications for routing that are drawn in [5] are: . Dealing with diversity is an unavoidable requirement for routing in the optical layer. It requires dealing with constraints in the routing process but most importantly requires additional state information -- the SRLG relationships and also the routings of any existing circuits from which the new circuit is to be diverse -- to be available to the routing process. . At present SRLG information cannot be self-discovered. Indeed, in a large network it is very difficult to maintain accurate SRLG information. The problem becomes particularly daunting whenever multiple administrative domains are involved, for instance after the acquisition of one network by another, because there normally is a likelihood that there are diversity violations between the domains. It is very unlikely that diversity relationships between carriers will be known any time in the near future. - Considerable variation in what different customers will mean by acceptable diversity should be anticipated. Consequently we suggest that an SRLG should be defined as follows: (i) It is a relationship between two or more links, and (ii) it is characterized by two parameters, the type of compromise (shared conduit, shared ROW, shared optical ring, etc.) and the extent of the compromise (e.g., the number of miles over which the compromise persisted). This will allow the SRLGĖs appropriate to a particular routing request to be easily identified. Bernstein, G. [Page 8] draft-ipo-optical-inter-domain-02.txt February 2003 2.4.1 Generalizing Link Diversity Optical networks may posses a number of hierarchical signaling layers. For example two routers interconnected across an optical network may communicate with IP packets encapsulated within an STS- 48c SONET path layer signal. Within the optical network this STS- 48c signal may be multiplexed at the SONET line layer into an OC-192 line layer signal. In addition this OC-192 may be wavelength division multiplexed onto a fiber with other OC-192 signals at different wavelengths (lambdas). These WDM signals can then be either lambda switched, wave band switched or fiber switched. Hence when we talk about diversity we need to specify the layer to which we are referring. In the previous example we can talk about diversity with respect to the SONET line layer, wave bands, and/or optical fibers. A similar situation arises when we consider the definition of node diversity. For example are we talking with respect to a SONET path layer switch or an optical switch or multiplexer? The Shared Risk Link Group concept in reference [6] generalizes the notion of link diversity (general list of numbers). First it's useful with respect to major outages (cable cuts, natural disasters) to have a few more types of diversity defined: 1. Cable (conduit) diversity (allows us to know which fibers are in the same cable (conduit). This helps avoid sending signals over routes that are most vulnerable to "ordinary" cable cuts (technically known as backhoe fades). 2. Right of Way (ROW) diversity. This helps avoid sending signals over routes that are subject to larger scale disasters such as ship anchor drags, train derailments, etc. 3. Geographic Route diversity. This type of diversity can help one avoid sending signals over routes that are subject to various larger scale disasters such as earthquakes, floods, tornadoes, hurricanes, etc. A route could be approximately described by a piecewise set of latitude/longitude or UTM coordinate pairs. We also have a form of link abstraction/summarization via the link bundling concept [7]. 2.4.2 Generalizing Node Diversity The concept of a node abstraction associated with GMPLS appears in reference [11] where it is used to generalize the concept of an explicitly routed path. In this case an abstract node can be a set of IP addresses or an AS number. From the point of view of node diverse routing specific concepts of interest include: 1. Nodes, i.e., individual switching elements. 2. Switching centers, i.e., a central office or exchange site. 3. Cities, or towns that contain more that one switching center. Bernstein, G. [Page 9] draft-ipo-optical-inter-domain-02.txt February 2003 4. Metro areas, or counties 5. States, 6. Countries, or 7. Geographic Regions 2.5 Routing Information Categorization Different applications of inter-domain optical routing call for different types of information to be shared or hidden between domains. In the following we decompose the information that can be transferred via a routing protocol broadly into link/topology information and node/domain information. We further subdivide these categories and will use this taxonomy of routing information when discussing the routing applications. 2.5.1 Link and Topology Related Information -Internal topology- information is information concerning the nodes and links and their connectivity within a domain. This type of information is traditionally shared within a domain via an intra- domain (interior gateway) routing protocol such as OSPF or IS-IS within a single area. For example the existence of nodes that only have links to other nodes within the domain, i.e., do not have links to other domains, would be strictly internal topology information. These nodes are known as internal nodes, while nodes with links to other domains are known as border nodes. Also included in this information is link/port property information such as whether the link is protected and what type of protection is being used, e.g., linear 1+1, linear 1:N, or some type of ring such as a 4F-BLSR [cite T1 document]. -Internal Resource- Information is concerned with the bandwidth available on links within a domain and possibly other resource related information. This information plays an important role in path selection within a domain. -Inter-Domain Topology- Information is concerned with how the domains are interconnected. This information can be key in inter- domain path selection, for example, in determining diverse routes. For the network in Figure 1 this information would let us know that domain A has two distinct links to domain B, domain A has two distinct links to domain C, but that domains B and C are not directly connected via any links. -Inter-Domain Resource- Information is concerned with the available bandwidth on inter-domain links. This information is important for inter-domain path selection and inter-domain traffic engineering purposes. For example in Figure 1 this information would give us some kind of bandwidth or capacity measure on the links between Bernstein, G. [Page 10] draft-ipo-optical-inter-domain-02.txt February 2003 domain A and domain B, and the links between domains A and C. The exact nature of this information may be application/context dependent. 2.5.2 Domain and Node Related Information -Reachability- information tells us what addresses are directly reachable via a particular domain. These systems can be end systems (clients) to the network or nodes within the network depending upon the application/context. Suppose in domain B of Figure 1 each of the network elements, NE1-NE3, have subtending end systems, and that NE1-NE3 do not represent a valid final destination for a path. Under this assumption the collection of the addresses of all these subtending end systems would form the reachability information for domain B. -Subnetwork Capability- information is concerned with the capabilities or features offered by the domain as a whole. This information is used in some applications where sharing the internal topology and resource information is inappropriate. This information can include: (a) Switching capabilities, (b) Protection capabilities, (c) Some kind of overall available capacity measure, (d) Reliability measures. Examples: 1. For example, in the SONET realm, one subnetwork may switch down to an STS-3c granularity while another switches down to an STS- 1 granularity. Understanding what types of signals within a SDH/SONET multiplex structure can be switched by a subnetwork is important. Similar examples of granularity in switching apply to the waveband case. 2. Some networking technologies, particularly SONET/SDH, provide a wide range of standardized protection technologies. But not all domains will offer all protection options. For example, a 2/4- F BLSR based subnetwork could offer extra data traffic, ring protected traffic and non-preemptible unprotected traffic, (NUT)[8], while a mesh network might offer shared SONET line layer linear protection and some form of mesh protection. 3. Some domains may be in locations that have lower incidences of link failure. Such information could be helpful in computing routes to statistically "share the pain". -End System Capabilities- information: While properties of the subnetwork are very important when trying to decide which domain to use to access a system (in the case of multi-homing), end systems also posses a wide variety of capabilities. Throwing end system capabilities such as a system's ability to support SONET/SDH virtual concatenation for distribution into a routing protocol may not be that advantageous since it somewhat counters the ability to Bernstein, G. [Page 11] draft-ipo-optical-inter-domain-02.txt February 2003 summarize reachability information. Detailed end-system information may alternatively be obtained via a directory service or some type of direct query between the end systems. 3 Applications of Inter Domain Optical Routing In this section we review some of the main applications of optical inter-control domain routing. We also review which aspects of the definition of control domain is most relevant to the application. 3.1 Intra Carrier Applications of Optical Inter Domain Routing Intra Carrier inter domain routing refers to a situation where the network that is to be partitioned into areas is under the control of one administrative entity. The main reasons for this partitioning in optical networks stem from scalability, inter-vendor interoperability, legacy equipment interoperability, and inter-layer partitioning. 3.1.1 Intra-Carrier Scalability As networks grow it is useful to partition a carriers network into separate optical routing domains which share limited or summarized information amongst each other. This reduces the overhead of information exchange across the network as a whole, and reduces the convergence time of routing protocols within a particular area. Hence we see in the inter-carrier scalability application that we will hide or summarize internal topology and resource information, while completely sharing inter-domain topology and resources information so that diverse paths can still be calculated. Note that general domain capabilities/capacity as well as reachability information would tend to be shared as completely as possible. For example the network shown in Figure 1 can be approximately represented as shown in Figure 2. This summarized network topology only has 4 links whose state need to be advertised in a routing protocol versus 17 links in the original network. Note that we may also advertise the capabilities of the three domains in Figure 2 as opposed to the 12 nodes of Figure 1. In Note that this partitioning into domains can recurse, i.e., we can have multiple levels of routing hierarchy to permit larger and larger networks. Such was the motivation behind the extensive hierarchical routing capability within ATM's PNNI routing protocol. The trade off to partitioning into domains for scalability is that less information is available for use in the route selection process, which can lead to inefficient utilization of network resources. On the other hand frequently this partitioning occurs on somewhat "natural" boundaries and as such the potential inefficiencies can be minimized. -------- ------ Bernstein, G. [Page 12] draft-ipo-optical-inter-domain-02.txt February 2003 /- -\ /- -\ ----- // NE 3 \mmmmmm // \\ / \ / Port 2 \ mmmmm NE 3 NE 5 \ /NE 3 \ / \ / Port 4 Port 6 \ mmmmmPort 17 \ | | | mmmmm | | | | | | | | | | | Domain A | | Domain C | | Domain B | | | | | | | | NE 2 NE 1 | | | | | |Port 5 Port 2 mmmmmmmmmNE 1 | | mmm / \Port 21 / \ NE 1 /mmmm \ / \ / \ Port 7 mm \\ // \ / \\ // \- -/ ----- \- -/ ------ -------- Figure 2. Summarized topology for the network of Figure 1. Also in Figure 2 we show the end points of the links between being identified by the triple of (domain, NE address, NE port number). This information is available via the discovery process. Though not strictly necessary including the identification of border nodes in a domain, allowing other nodes to understand whether these links terminate on the same or different nodes is valuable in setting up diverse inter-domain paths. In current intra-domain IP routing protocols, such as OSPF's, a multiple area capability provides for intra-carrier scalability. However, this is currently done by sharing reachability information and using a distance-vector method to obtain routes. This does not discover and propagate inter-domain topology information and hence insufficient information is available for diverse route calculations. In addition OSPF areas are required to be assembled in a particular fashion with all areas bordering on a single backbone area. When the topology within the domain is approximated or hidden then signaling and call processing at the domain border will receive an approximated (loose) route and the border node or signaling entity must then translate this to a precise route through the domain. Hence there is some linkage between multi-domain connection control and inter-area/inter-domain routing. 3.1.2 Intra-Carrier Inter-vendor An important application of intra-carrier optical routing is the intra-carrier inter-vendor scenario. From a carrierĖs perspective, the use of domains provides a clean way to isolate clouds of Bernstein, G. [Page 13] draft-ipo-optical-inter-domain-02.txt February 2003 equipment belonging to different vendors, while at the same time allowing for interoperability between the vendors. An advantage of this method is that it allows the vendors complete freedom to use any combination of routing protocols or traditional management-based methods to propagate topology and resources internal to their domains. In other words, the routing entity in each domain could obtain this information either by participating in a routing protocol like OSPF, or by querying each NE via an EMS, or by simply having the required information manually configured into it. Here the "domain independence" and "boundary on the wire" properties of an inter-control domain routing protocol are being stressed. Note that the routing entity shown in each vendorĖs cloud in Figure 3 is in reality an abstract representation of the routing intelligence within the vendor cloud. This intelligence may either be implemented in a distributed way, by having a routing protocol running at each NE, or in a centralized way, through the use of an intelligent, centralized routing entity that communicates with the individual NEĖs (either via a protocol or by querying individual elements) to retrieve connectivity and resource information that it uses to build a complete topology and resource map of the domain. Therefore, vendors may use different protocols as the primary option between their own devices, adding specialized features or optimizing their performance based on their choice of protocol. /------------------------------------\ / /-\ \ / Domain B |NE3| +-------+ \ | (Vendor 2) /\-/\ |Routing| | | / \ |Entity | | | /-\/ \/-\ +-------+ | \ |NE1|-------|NE2| @ / \ \-/: \-/ @ / \------+-:---------+---------@-------/ Neighbor discovery | : | @ Exchange of routing Between NEs in the | : | @ information between same domain. ----> | : | @ the domains routing | : | @ entities. /------+-:---------+---------@-------\ / | : | @ \ / /-\ /-\ +-------+ \ | |NE1|-----|NE2| |Routing| | | \-/\ /\-/ |Entity | | | \/-\/ +-------+ | | Domain A |NE3| | | (Vendor 1) \-/ / \ / \------------------------------------/ Bernstein, G. [Page 14] draft-ipo-optical-inter-domain-02.txt February 2003 Figure 3. Intra-Carrier Inter-vendor routing domains Even if it is a centralized entity, the routing entity could still be run on a given NE in the vendor cloud. In other words, these entities could be distributed, or centralized onto a single node, or independent of any of the nodes. In ATM's PNNI protocol, for example, this was centralized on a node elected as the "peer group leader". In the inter-vendor case, it can be particularly advantageous to centralize this so that the flow of information can be monitored. A centralized routing entity could apply flooding and summarization mechanisms as if it is a switching system. Since this is optical rather than IP routing, signaling would be carried by a control channel between the routing entity and the neighboring system, rather than being carried over the data links. The functions of the routing entity include: (a) direct reachability exchange (that is, which NEĖs can be directly reached from this domain, (b) verification of area connectedness (that is, understanding how the two domains are interconnected), (c) area/domain topology (possibly summarized) exchange and updates, and (d) topology updates concerning other domains/areas. When a carrier partitions its network for inter-vendor interoperability as described above, it may still share information about the internal topology of the domains in some standardized form that has been agreed upon between the vendors. Although one option is to force both vendors to adopt a new common protocol, another is to require that only a minimum subset of reachability/topology information be shared between the vendor clouds. The latter option helps during fault situations, by providing fault isolation at the domain boundaries. It prevents an outage in a domain composed of one vendorĖs equipment from causing a reaction in an adjacent domain composed of another vendorĖs equipment, thus preventing a situation that would typically degenerate into a process known as ††finger pointingĖĖ between the two vendors. The setup described above takes into account the three most important sub-cases of inter-carrier inter-vendor partitioning: a. The first is where both vendor domains run distributed routing protocols. This is the most flexible case, and is the situation when new equipment capable of running such protocols is deployed. b. The second is where the optical subnetworks or domains (which includes a large number of existing installations) do not run any internal routing protocol (because the NEĖs are not capable of doing so), relying instead on EMS-based topology discovery/resource management. In this case, interoperability Bernstein, G. [Page 15] draft-ipo-optical-inter-domain-02.txt February 2003 with other vendor clouds can be realized by having the routing entity run as a separate software entity with access to the appropriate information. These entities may exchange routing proxy addresses through the neighbor discovery protocol, and then exchange routing information (proxying for the entire domain) with each other. The basic advantage here is that even though the vendor specific element management system (EMS) knows the topology of its subnetwork, it is far easier to get an inter-domain routing protocol to share information than trying to get the separate vendors management systems to communicate. c. The third is where one domain has a centralized routing entity, while the other runs a distributed routing protocol. Once again, the neighbor discovery process between the area border NEĖs could be used to advertise the address of the routing entity. 3.1.3 Inter-Layer Partitioning In transport networks layering is a part of the multiplex and OA&M structure of the signals, playing a role in multiplexing, monitoring and general link management. Layering in the transport network is defined in fairly abstract terms in [G.805] and the concepts are applied to SDH in [G.803]. As explained in a recent ITU SG15 document (WD45 Q.14/15) not all the layers in the transport network are of interest to the control plane, or to routing in particular. Some layers may not contain active switching elements, however this does not mean that information flow concerning a non-switching layer is not valuable in routing. For example in [GB-WDM-SRLG] static WDM layer information was used to set the SRLGs for SONET lines (i.e., information passed around by a link state protocol operating at the SONET line layer). It should be noted that much of the information available from non-switching layers relates to performance monitoring and fault management. In this situation the network is partitioned into sub-networks that operate at different switching layers. One reason for doing this is that not all the information from one layer is necessary or relevant to another layer. For example, between transparent optical switches and SDH/SONET path (VC) layer switches, the optical switches have no direct use for the SONET layer information. In addition optical networks may keep a lot more physical layer information (such as the properties of every optical amplifier on a WDM span) that is of no use to the SONET layer. One again this promotes scalability, but also simplifies the implementation by reducing inter-layer information transfer to that which is actually useful. Bernstein, G. [Page 16] draft-ipo-optical-inter-domain-02.txt February 2003 Now in network planning it is very useful to have a view of the current higher layer traffic matrix [9] being satisfied and higher layer traffic trend measurements over time. Although we can somewhat see this in higher layer resource status changes over time, this represents a link level view when we really desire the trend (change in time) of the traffic matrices between sites. How this information gets distributed is an open issue. Currently individual nodes in a GMPLS network know only about connections that they source or sink. Network planning is generally a longer time horizon process than even traffic engineering hence it is an open question as to whether this would ever be a useful function to incorporate into a network element. Now looking the other way is initially simpler, i.e., it is easier to ask: what can a higher layer use for path selection/computation from a lower layer. The first item that springs to mind is diversity information. For example in setting up a SONET STS-1 path we can use information from a WDM system concerning which SONET lines share the same WDM fiber. This is information, however, is already abstracted into routing protocols via SRLG concept. Other information from a lower layer is of questionable value since it tends to be technology specific and puts more and more burden on the upper layer to be able to effectively understand and use this information. 3.1.4 Interaction with IP Layer Routing The applicability of IP-based routing protocols has, over the years, been constantly expanded to increasingly more circuit-oriented layers. The community began with pure datagram routing, gradually expanded to cover virtual-circuit switched packet routing (for e.g., MPLS), and is finally looking at the application of routing protocols to real circuit switching, e.g. the optical layer. However, as pointed out earlier in this document, it is not clear that the different layers should necessarily share the same instance of the routing protocols. Indeed, there may be significant reasons for not doing so and many carrier tend to partition their networks along switching layer boundaries. For example, IP-layer reachability information is not particularly useful for the optical layer, so it seems an overkill to burden the optical equipment with storing and distributing that information. (It is an extra expense on memory and processing for information that the optical layer does not really care about, so there is little incentive for a vendor to want to do so.) Likewise, information on physical plant (fibers, conduits, ducts) diversity, which is crucial at the optical transport layer, is very unlikely to be used directly by the IP layer. So, it would Bernstein, G. [Page 17] draft-ipo-optical-inter-domain-02.txt February 2003 be quite wasteful of resources to burden the IP layer routing with distributing and manipulating this information. Thus, the extent of interaction or integration with IP layer routing (if any) requires careful consideration. 3.1.5 Inter-Business Unit A slightly different but interesting application of intra-carrier optical routing occurs in the intra-carrier inter-business unit scenario. This arises because a carrier often has multiple administrative domains, with groups of administrative domains being under the purview of independent BUĖs within the carrier. Note that different BUĖs represent independent cost centers with their own profit objectives and sales targets. As a result, while the BUĖs can profitably share topology information and would like to do so, they may not be so inclined to advertise the details of their resource usage into domains belonging to other BUs. Since each BU has its own revenue targets, advertising detailed resource availability information to other, potentially competing, BUs can have a negative impact on a BUs revenue generation. This is because the knowledge of available resources in one BU may enable other BUs within the carrier to requisition capacity from this BU. This would force the BU in question to yield to their request, possibly at the expense of selling capacity to more profitable, external, revenue generating customers. Thus, this scenario is likely to have an additional dimension of information sharing, namely, policy-based information sharing, which does not apply to the other cases that we have discussed so far. Two examples where the inter-business unit scenario could become important are in the case of metro-core-metro networks within a given carrier, and in the case of regional networks within a carrier. In the metro-core-metro situation, such as the one depicted in resource sharing between them. /------------------------------------\ / \ / /-\ \ | Domain |NE2| | Metro BU | B /\-/\<-- Metro Ring | Network | / \ | | /-\/ \/-\ | | |NE1|-------|NE3| / \ \-/ \-/ / \ | | / Bernstein, G. [Page 18] draft-ipo-optical-inter-domain-02.txt February 2003 \------+-----------+-----------------/ | | /------+-----------+-----------------\ / | | \ / | /-\ | /-\ \ | Domain | |NE2|---+---------|NE3| | CORE BU | A | /\-/\ | ------/\-/ | Network | | / \ | / | | | /-\/ \/-\/ |<--Core | | |NE1|-------|NE5|---- | Mesh / \ \-/ \-/ \ /-\ / \ | | --|NE4| / \ | | \-/ / \-----+-----------+----------------/ | | /------+-----------+-----------------\ / | | \ / /-\ /-\ \ | |NE1|-----|NE2| | Metro BU | Domain \-/ \-/ | Network | C | |<--- Metro Ring | | | | | | /-\ /-\ | | |NE3|-----|NE3| | | \-/ \-/ | \ / \------------------------------------/ Figure 4, the metro network domains could be under the jurisdiction Deleted of one BU, while the core network domains belong to a different BU. In this case, for example, it is possible that the metro BU, armed with resource availability information about the core BUĖs domains, could requisition capacity from the core network when needed. This may harm the core networkĖs profit goals, because they may not be able to charge an internal customer the same rates that they could charge for the same capacity from an external customer, thus motivating the need for selective, policy-based resource sharing between them. /------------------------------------\ / \ / /-\ \ | Domain |NE2| | Metro BU | B /\-/\<-- Metro Ring | Network | / \ | | /-\/ \/-\ | | |NE1|-------|NE3| / \ \-/ \-/ / \ | | / \------+-----------+-----------------/ | | /------+-----------+-----------------\ / | | \ Bernstein, G. [Page 19] draft-ipo-optical-inter-domain-02.txt February 2003 / | /-\ | /-\ \ | Domain | |NE2|---+---------|NE3| | CORE BU | A | /\-/\ | ------/\-/ | Network | | / \ | / | | | /-\/ \/-\/ |<--Core | | |NE1|-------|NE5|---- | Mesh / \ \-/ \-/ \ /-\ / \ | | --|NE4| / \ | | \-/ / \-----+-----------+----------------/ | | /------+-----------+-----------------\ / | | \ / /-\ /-\ \ | |NE1|-----|NE2| | Metro BU | Domain \-/ \-/ | Network | C | |<--- Metro Ring | | | | | | /-\ /-\ | | |NE3|-----|NE3| | | \-/ \-/ | \ / \------------------------------------/ Figure 4. Intra-carrier inter-business unit routing domains. 3.2 Inter-Carrier Inter-Domain Optical Routing In this case we are talking about dealing with outside entities, i.e., between service providers. There may be a range of levels of trust here; for example there might be some level of trust between two providers that have formed a marketing alliance or have some other form of business relationship. In general, however, trust can not be assumed. In this case, all the concerns of revealing too much information about one's network come into play. However, not revealing enough, say about diversity capabilities may also lead customers elsewhere. Also there are some other security issues not seen before. For example, in route distribution one carrier might not be inclined to pass on routing information that could point the way to competitive alternatives. This impacts the methods for route updates, etc. With the interest in bandwidth trading [10] we can also look at this as an advertisement of network connectivity and capability with of course any "warts" covered up. This would include reliance on other carrier for fibers or lambdas. Also a fair amount of details such as "unused capacity" would not be advertised since this maybe financially sensitive information. Private line pricing today is based primarily on the service itself (bandwidth, end-points, etc.) and the holding time, and there is no Bernstein, G. [Page 20] draft-ipo-optical-inter-domain-02.txt February 2003 reason to expect that this will change. When multiple service providers are involved the algorithm for dividing up the revenue stream (which can be quite large even for a single connection) must be explicit by connect time. This could be done off-line or could be done at connect time. In either case, the entity or entities doing the routing will need to take provider pricing structures into account whenever there is a choice between providers that needs to be made. The routing logic could do this explicitly if the prices are captured in the advertised metrics or some other advertised data; alternatively it could be done by some sort of policy control, as it is today by BGP. The essence of bandwidth trading is the existence of competing price structures that are known to the entity deciding which competitor to use. It is possible to create plausible bandwidth trading scenarios involving the UNI, the NNI, or both. If the NNI is involved, these price structures will need to be established across it. The situation is further complicated by the fact that bandwidth trading could be realized using any one of a number of business models, each with its own information requirements. To give two examples: If an auction model were used the buyer might repeatedly broadcast the lowest bid received to date and solicit lower bids from the competing providers. On the other hand, if there were a more formal market the providers might post their asking prices in some public fashion and a buyer would be matched by some third party with the lowest offer. In the inter-carrier case notions of hierarchy seem rather sensitive, i.e., he who controls the summarization and advertisement may have an undue advantage over competitors. In addition, a "bandwidth aggregator" may want to advertise capabilities that he has put together via deals with multiple carriers... Notes: We can attempt to extend the SRLG concept to links between ASs but we will need the two ASs to agree on the meaning and number of the list of 32 bit integers that comprise the SRLG, i.e., previously the SRLG concept was one of AS scope. And this is also where things get tricky since it may not be possible to distinguish diverse routes based upon differing path vectors (i.e., AS number traversal list). The reason for this is due the fact that many carriers "fill out" their networks by renting either dark fiber or "lambdas" from a WDM system and hence although the path vectors may be AS diverse they may not even be fiber diverse. Hence there is a need for sharing of diversity information or constraints between ASs when setting up diverse connections across multiple ASs. This gets us somewhat into a quandary over which information needs to be public and how to coordinate its distribution. In this sense geographic link information may be the simplest and least contentious to get various players to disclose and standardize. Bernstein, G. [Page 21] draft-ipo-optical-inter-domain-02.txt February 2003 Notes: (1) The real issue is consistency between the cloud/ASĖs since in many cases they are sharing conduit, ROW, etc. Getting this to happen could be very problematic. It would be preferable to see a diversity option that doesnĖt require this. For example, ensure that there is diversity within each cloud and then do restoration separately within each cloud. (2) See the definition of SRLG in the Carrier Requirements -- an equivalence class of links, the extent of violation, and the level. (3) Flexibility in defining the level of violation seems very desirable -- these historically have drifted in time. There are many others -- eg, if the shared resources are SPRING protected thatĖs less of a problem than otherwise. Notes: Participation in the inter-domain network carries constraints on the carriers. First, in order to participate, each provider network MUST be willing to advertise the destinations that are reachable through his network at each entry point and advertise the formats available. Without providing such information, there is little motivation to participate since it is unlikely that others will be able to access services of which they are not aware. Second, every participating carriers MUST agree to fairly include the information made available by every other carrier so that each carrier has an equal opportunity to provide services. There may be specific exceptions, but the carrier claiming those exceptions MUST advertise the exceptions themselves. In this manner, other carriers that might otherwise be aware of distant services can be prompted to seek those services manually. Note a combination of minimal required information transferred with deferral to the originating subnetwork along with some basic security mechanisms such as integrity and non-repudiation may be useful in helping organizations to "play nice". 3.3 Multi-Domain Connection Control MPLSĖ loose routing capability allows one to specify a route for an optical connection in terms of a sequence of optical AS numbers. This, for example, is handled via RSVP-TEĖs abstract node concept [11]. Currently there is nothing in the GMPLS signaling specification that differentiates between intra AS boundaries, i.e., between two neighbor optical LSRs in the same AS, and inter AS boundaries, i.e. between two neighbor optical LSRs in different ASs. Note that these same notions can apply to separate routing domains within an AS. There may, however, be some useful reasons for differentiating these two cases: 1. Separation of signaling domains, 2. Separation of protection domains. While routing protocols (used for their topology information) in the optical case are not "service impacting", signaling protocols most certainly are. It is desirable to build some type of "wall" between optical ASs so that faults in one that lead to "signaling storms" do not get propagated to other ASs. Note that the same motivation Bernstein, G. [Page 22] draft-ipo-optical-inter-domain-02.txt February 2003 applies for isolating other kinds of clouds, like vendors specific ones. The natural situation where "signaling storms" would be most likely to arise is during network restoration signaling, i.e., signaling to recover connections during major network outages, e.g., natural disasters etc. In this case it may be very advantageous to break up general source reroute forms of restoration into per domain segments or to start reroute at domain boundaries rather than all the way back at the originating node. Note that this has the advantage of reducing the need for globally consistent SRLGĖs. (See earlier SRLG comment.) Such a capability requires some loose coordination between the local, intermediate and global protection mechanisms [12]. This is typically implemented via hold off timers, i.e., one layer of protection will not attempt restoration until a more fundamental (local) form has been given a chance to recover the connection [12]. In other words, prevention of restoration related signaling storms may require the breaking up of a large network into multiple signaling (and hence routing) domains. These domains could be within the same AS. 4 Multiple Layers of Optical Routing As previously discussed, there are multiple layers of signals included in what in the IP model one would call the Physical Layer. One could separate the layers by creating sublayers in the Physical Layer. For example, sublayers in the Physical Layer might be, top to bottom: SDH LOVCs, SDH HOVCs, and Lambdas. If a system supports only one of the three, then isolation of the sublayers is a given; it's geographical. But there are systems which will support more than one physical sublayer, therefore, it is necessary to establish whether or not there is a need to isolate the sublayers in the same manner. Or put another way is there a reason to "integrate" the sublayers for the purposes of routing (topology dissemination). If they are isolated, then there will be separate topological models for each sublayer: one mesh for the LOVC, one for the HOVC, one for the Lambda, and possibly others. The appropriate way to access a sublayer is via the use of sublayer SAPs (service access points). For example, in this way, one may find that use of Lambdas is more efficient because each sublayer can assess the availability of services at its own layer before searching for coarser-granularity services. On the other hand, the control plane must accommodate three separate routing protocols, or at least three separate instances of the same routing protocol, all operating at both intra and inter-domain level. Example (SDH multiplexing) For transport across an SDH network, the lower order signals must be multiplexed into a non-concatenated higher order signal. Hence LOVCs are not routed independently, but only as tributaries of HOVCs. In addition in the SDH hierarchy there is a signal, VC3, that can be Bernstein, G. [Page 23] draft-ipo-optical-inter-domain-02.txt February 2003 treated (multiplexed) as either a LOVC or a HOVC. With this tight and somewhat confused coupling of these layers it may beneficial to sometimes combine them into the same route protocol instance. The alternative to separate routing protocols per sublayer is the original notion behind GMPLS routing and the forwarding adjaciency concept [13]. Rather than separating the route protocols into separate layers (or sublayers) with distinct topologies, each ONE would advertise the services it can provide, along with its topology information. For example, a ONE (optical network element) might advertise that it carries a route to node A with STS-N service and clear-channel lambda service and carries multiple routes to node B with STS-N service. It might, alternatively, advertise its entire network with summarized link capacity information for every included link. Neighboring carriers would, implicitly, be allowed to summarize that information for internal advertisement via its IGP. Further consideration could be given to a query service, where a carrier advertises the geographical area it serves without detailed reachability or capacity information. A second carrier desiring service could query the first carrier as to reachability for a specific destination, and the first carrier would respond with availability and capacity information. Integrating multiple layers into the same routing protocol instance leaves us fewer routing protocols to manage. The downside of this is that more information must be exchanged via this routing protocol and more network elements participate in this single instance of the routing protocol which can lead to scalability concerns. If the equipment working on the different sublayers comes from different vendors there would be little incentive to integrate multiple layers into the routing protocol for a single layer product. Regardless of whether multiple layers are integrated into the same routing protocol instance it can be very useful to share information between layers as illustrated by the following examples: o Drop side links between layers: Capabilities of the links that are between the (client and server) layers need to be propagated into the routing protocol. o Summarize link capabilities: Summarizing the server layer capabilities in the client layer will reduce the amount of information required for multi-layer constraint based path computation. o Send only that are required: Sending only the capabilities that are useful in the constraint path computation in the client layer. 5 Conclusion Bernstein, G. [Page 24] draft-ipo-optical-inter-domain-02.txt February 2003 Key properties of optical control domains and an optical inter control domain routing protocol were reviewed. The stringent properties of "independence of internal domain control plane mechanisms" as well as support for "boundary on the wire" were shown to be crucial to the definition of control domain by showing their use in several mainstream optical inter-networking scenarios. Via examples we saw that BGP is the only existing IP routing protocol for which the "internal domain independence" and "boundary on the wire" property hold. However, it was described how important the ability to diversely route optical connections is to the reliability and resilience of the optical network. In addition it was pointed out that flexibility in choosing/computing paths can be very important to an optical network operator so that they can offer a range of differentiated services. In this light BGP does not currently meet the needs of an optical inter control domain routing protocol. As of this writing we are therefore looking at two alternatives for an optical inter control domain routing protocol. (1) Extensions to a link state type protocol (such as OSPF, IS-IS or PNNI routing) to work at the control domain level rather than node to node level. Or (2) Extensions to a path vector type protocol (such as BGP) to provide diverse routing support and enhance path selection/network optimization. 6 Security Considerations 6.1.1 Protection of the routing information exchanged across an optical inter-domain interface is of high importance; erroneous reachability or topology information may result in connection provisioning requests that either fail or are routed across sub-optimal paths. It is also possible that failed requests may consume significant control and transport resources for a transient amount of time. It follows that erroneous routing information could result in degraded carrier network operation, or even render a carrierĖs network inoperable. Security requirements are expected to be of higher importance in interfaces between different administrative domains. Therefore, an optical inter-domain routing protocol should provide the following: 1. Authenticate entities with which routing information is exchanged. For example, a carrier should authenticate the identity of other carriers it is connected to. The specific mechanisms used for authentication should provide protection against attacks; for example they should not be based on simple clear-text password authentication schemes. 2. Guarantee the integrity of routing information (topology, reachability and resource status) exchanged across the interface. Bernstein, G. [Page 25] draft-ipo-optical-inter-domain-02.txt February 2003 This requirement can be satisfied using security mechanisms at different layers. For example, each routing message could be individually authenticated using a keyed message digest, which is embedded in the message. Both OSPF and BGP provide such options. Alternatively, the two parties could establish a security association at the network layer using IPSEC, which could be used to provide security services to the optical inter-domain routing protocol. From the point of view of routing, information integrity is likely to be the most important requirement. However, in some cases it might be necessary to provide confidentiality of the routing information as well. A possible scenario for this is when a carrier would like to advertise information privately to another carrier, but does not wish to publicly disseminate this information, due to policy constraints. It should be noted than none of the known mechanisms that provide information integrity (such as keyed digests or IPSEC) can provide adequate protection against a compromised node participating in the inter-domain routing protocol. This is an item for further study. 7 References [1] Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. [2] G. Bernstein, J. Yates, D. Saha, "IP-Centric Control and Management of Optical Transport Networks", IEEE Communications Magazine, October 2000. [3] G. Bernstein, E. Mannie, V. Sharma, "Framework for MPLS-based Control of Optical SDH/SONET Networks", , February 2003. [4] Ramesh Bhandari, Survivable Networks: Algorithms for Diverse Routing, Kluwer Academic Publishers,1999. [5] Strand, J. (ed.) "Impairments And Other Constraints On Optical Layer Routing", work in progress, draft-ietf-ipo-impairments- 00.txt, May 2001. [6] Kompella, K., et. al. "IS-IS Extensions in Support of Generalized MPLS", Work in Progress, draft-ietf-isis-gmpls- extensions-16.txt, December 2002. [7] K. Kompella, et. al. "Link Bundling in MPLS Traffic Engineering", Work in Progress, draft-ietf-mpls-bundle- 04.txt, July 2002. Bernstein, G. [Page 26] draft-ipo-optical-inter-domain-02.txt February 2003 [8] ANSI T1.105.01-1995, Synchronous Optical Network (SONET) Automatic Protection Switching, American National Standards institute. [9] Robert S. Cahn, Wide Area Network Design: Concepts and Tools for Optimization, Morgan Kaufmann Publishers, Inc., 1998. [10] Meghan Fuller, "Bandwidth trading no longer a case of 'if' but 'when' says report", Lightwave, June 2001. (www.light-wave.com) [11] RFC 3209, "RSVP-TE: Extensions to RSVP for LSP Tunnels", December 2001. [12] K. Owens, V. Sharma, M. Oommen, "Network Survivability Considerations for Traffic Engineered IP Networks", Work in Progress, draft-owens-te-network-survivability-03.txt, May 2002. [13] K. Kompella and Y. Rekhter, "LSP Hierarchy with MPLS TE", draft-ietf-mpls-lsp-hierarchy-08.txt, Internet Draft, Work in Progress, September 2002. [14] G.805, Generic Functional Architecture of Transport Networks, ITU-T, March 2000. 8 Acknowledgments The authors would like to thank Shezad Mirza of BT and Cheryl Sanchez of Calix for their help. 9 Author's Addresses Greg Bernstein Grotto-Networking Phone: (510) 573-2237 Email: gregb@grotto-networking.com Lyndon Ong Ciena Corporation 5965 Silver Creek Valley Road San Jose, CA 95015 Phone: (408) 834-7894 Email: lyong@ciena.com Bala Rajagopalan, Dimitrios Pendarakis Tellium, Inc 2 Crescent Place Ocean Port, NJ 07757 Email: braja@tellium.com, dpendarakis@Tellium.com Angela Chiu John Strand AT&T Labs Bernstein, G. [Page 27] draft-ipo-optical-inter-domain-02.txt February 2003 200 Laurel Ave. Middletown, NJ 07748 Phone:(732) 420-9061 Angela, (732) 420-9036 John Email: chiu@research.att.com, jls@research.att.com Vishal Sharma Metanoia, Inc. 335 Elan Village Lane, Unit 203 San Jose, CA 95134 Phone: +1 408 943 1794 Email: v.sharma@ieee.org Rauf Izmailov NEC USA, Inc. 4 Independence Way Princeton, NJ 08540 Email: rauf@nec-lab.com Dean Cheng Polaris Networks Inc. 6810 Santa Teresa Bl. San Jose CA 95119 Email: dcheng@polarinetworks.com Sudheer Dharanikota Frank Hujber Bernstein, G. [Page 28]