CCAMP Working Group Eric Mannie - Editor Internet Draft Expiration date: Feb. 2003 August 2002 Generalized Multi-Protocol Label Switching (GMPLS) Architecture draft-ietf-ccamp-gmpls-architecture-03.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 [1]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract Future data and transmission networks will consist of elements such as routers, switches, DWDM systems, Add-Drop Multiplexors (ADMs), photonic cross-connects (PXCs), optical cross-connects (OXCs), etc that will use Generalized MPLS (GMPLS) to dynamically provision resources and to provide network survivability using protection and restoration techniques. This document describes the architecture of GMPLS. GMPLS extends MPLS to encompass time-division (e.g. SDH/SONET, PDH, G.709), wavelength (lambdas), and spatial switching (e.g. incoming port or fiber to outgoing port or fiber). The main focus of GMPLS is on the control plane of these various layers since each of them can use physically diverse data or forwarding planes. The intention is to cover both the signaling and the routing part of that control plane. E. Mannie et. al. 1 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 Table of Contents Status of this Memo................................................1 Abstract...........................................................1 Table of Contents..................................................2 1. Contributors....................................................4 2. Conventions used in this document...............................7 3. Introduction....................................................7 3.1. Acronyms & abbreviations......................................7 3.2. Multiple Types of Switching and Forwarding Hierarchies........8 3.3. Extension of the MPLS Control Plane..........................10 3.4. GMPLS Key Extensions to MPLS-TE..............................12 4. Routing and addressing model...................................13 4.1. Addressing of PSC and non-PSC layers.........................14 4.2. GMPLS scalability enhancements...............................15 4.3. TE Extensions to IP routing protocols........................15 5. Unnumbered links...............................................16 5.1. Unnumbered Forwarding Adjacencies............................17 6. Link bundling..................................................18 6.1. Restrictions on bundling.....................................18 6.2. Routing considerations for bundling..........................18 6.3. Signaling considerations.....................................19 6.3.1. Mechanism 1: Implicit Indication...........................20 6.3.2. Mechanism 2: Explicit Indication by Numbered Interface ID..20 6.3.3. Mechanism 3: Explicit Indication by Unnumbered Interface ID20 6.4. Unnumbered Bundled Link......................................20 6.5. Forming bundled links........................................21 7. Relationship with the UNI......................................21 7.1. Relationship with the OIF UNI................................22 7.2. Reachability across the UNI..................................22 8. Link Management................................................23 8.1. Control channel and control channel management...............23 8.2. Link property correlation....................................25 8.3. Link connectivity verification...............................25 8.4. Fault management.............................................25 8.5 LMP for DWDM Optical Line Systems (OLSs)......................26 9. Generalized Signaling..........................................27 9.1. Overview: How to Request an LSP..............................29 9.2. Generalized Label Request....................................30 9.3. SONET/SDH Traffic Parameters.................................31 9.4. G.709 Traffic Parameters.....................................32 9.5. Bandwidth Encoding...........................................32 9.6. Generalized Label............................................33 9.7. Waveband Switching...........................................33 9.8. Label Suggestion by the Upstream.............................34 9.9. Label Restriction by the Upstream............................34 9.10. Bi-directional LSP..........................................35 9.11. Bi-directional LSP Contention Resolution....................36 9.12. Rapid Notification of Failure...............................36 9.13. Link Protection.............................................37 9.14. Explicit Routing and Explicit Label Control.................37 9.15. Route recording.............................................38 9.16. LSP modification and LSP re-routing.........................39 9.17. LSP administrative status handling..........................39 E. Mannie et. al. Internet-Draft February 2003 2 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 9.18. Control channel separation..................................40 10. Forwarding Adjacencies (FA)...................................41 10.1. Routing and Forwarding Adjacencies..........................41 10.2. Signaling aspects...........................................42 10.3. Cascading of Forwarding Adjacencies.........................42 11. Routing and Signaling Adjacencies.............................43 12. Control Plane Fault Handling..................................44 13. LSP Protection and Restoration................................45 13.1. Protection escalation across domains and layers.............46 13.2. Mapping of Services to P&R Resources........................46 13.3. Classification of P&R mechanism characteristics.............47 13.4. Different Stages in P&R.....................................47 13.5. Recovery Strategies.........................................48 13.6. Recovery mechanisms: Protection schemes.....................48 13.7. Recovery mechanisms: Restoration schemes....................49 13.8. Schema selection criteria...................................50 14. Network Management............................................51 14.1. Network Management Systems (NMS)............................51 14.2. Management Information Base (MIB)...........................52 14.3. Tools.......................................................52 14.4. Fault Correlation Between Multiple Layers...................52 15. Security considerations.......................................53 16. Acknowledgements..............................................54 17. References....................................................55 18. Author's Address..............................................57 Full Copyright Statement..........................................58 E. Mannie et. al. Internet-Draft February 2003 3 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 1. Contributors Peter Ashwood-Smith Eric Mannie Nortel Networks Corp. Ebone (GTS) P.O. Box 3511 Station C, Terhulpsesteenweg 6A Ottawa, ON K1Y 4H7 1560 Hoeilaart Canada Belgium Phone: +1 613 763-4534 Phone: +32 2 658-5652 Email: Email: eric.mannie@gts.com petera@nortelnetworks.com Daniel O. Awduche Thomas D. Nadeau Movaz Networks Cisco Systems, Inc. 7296 Jones Branch Drive 250 Apollo Drive Suite 615 Chelmsford, MA 01824 McLean, VA 22102 USA USA Phone: +1 978 244-3051 Phone: +1 703 847-7350 Email: tnadeau@cisco.com Email: awduche@movaz.com Ayan Banerjee Lyndon Ong Calient Networks Ciena Systems 5853 Rue Ferrari 10480 Ridgeview Ct San Jose, CA 95138 Cupertino, CA 95014 USA USA Phone: +1 408 972-3645 Email: lyong@ciena.com Email: abanerjee@calient.net Debashis Basak Dimitri Papadimitriou Accelight Networks Alcatel 70 Abele Road, Bldg.1200 Francis Wellesplein, 1 Bridgeville, PA 15017 B-2018 Antwerpen USA Belgium Phone: +1 412 220-2102 (ext115) Phone: +32 3 240-8491 email: dbasak@accelight.com Email: dimitri.papadimitriou@alcatel.be Lou Berger Dimitrios Pendarakis Movaz Networks, Inc. Tellium, Inc. 7926 Jones Branch Drive 2 Crescent Place Suite 615 P.O. Box 901 MCLean VA, 22102 Oceanport, NJ 07757-0901 USA USA Phone: +1 703 847-1801 Email: DPendarakis@tellium.com Email: lberger@movaz.com E. Mannie et. al. Internet-Draft February 2003 4 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 Greg Bernstein Bala Rajagopalan Ciena Corporation Tellium, Inc. 10480 Ridgeview Court 2 Crescent Place Cupertino, CA 94014 P.O. Box 901 USA Oceanport, NJ 07757-0901 Phone: +1 408 366-4713 USA Email: greg@ciena.com Phone: +1 732 923-4237 Email: braja@tellium.com Sudheer Dharanikota Yakov Rekhter Nayna Networks Inc. Juniper 481 Sycamore Drive Email: yakov@juniper.net Milpitas, CA 95035 USA Email: sudheer@nayna.com John Drake Debanjan Saha Calient Networks Tellium Optical Systems 5853 Rue Ferrari 2 Crescent Place San Jose, CA 95138 Oceanport, NJ 07757-0901 USA USA Phone: +1 408 972-3720 Phone: +1 732 923-4264 Email: jdrake@calient.net Email: dsaha@tellium.com Yanhe Fan Hal Sandick Axiowave Networks, Inc. Nortel Networks 200 Nickerson Road Email: Marlborough, MA 01752 hsandick@nortelnetworks.com USA Phone: +1 774 348-4627 Email: yfan@axiowave.com Don Fedyk Vishal Sharma Nortel Networks Corp. Metanoia, Inc. 600 Technology Park Drive 305 Elan Village Lane, Unit Billerica, MA 01821 121 USA San Jose, CA 95134-2545 Phone: +1 978 288-4506 USA Email: Phone: +1 408 895-5030 dwfedyk@nortelnetworks.com Email: vsharma87@yahoo.com Gert Grammel George Swallow Alcatel Cisco Systems, Inc. Via Trento, 30 250 Apollo Drive 20059 Vimercate (Mi) Chelmsford, MA 01824 Italy USA Email: gert.grammel@alcatel.it Phone: +1 978 244-8143 Email: swallow@cisco.com E. Mannie et. al. Internet-Draft February 2003 5 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 Dan Guo Z. Bo Tang Turin Networks, Inc. Tellium, Inc. 1415 N. McDowell Blvd, 2 Crescent Place Petaluma, CA 95454 P.O. Box 901 USA Oceanport, NJ 07757-0901 Email: dguo@turinnetworks.com USA Phone: +1 732 923-4231 Email: btang@tellium.com Kireeti Kompella Jennifer Yates Juniper Networks, Inc. AT&T 1194 N. Mathilda Ave. 180 Park Avenue Sunnyvale, CA 94089 Florham Park, NJ 07932 USA USA Email: kireeti@juniper.net Email: jyates@research.att.com Alan Kullberg George R. Young NetPlane Systems, Inc. Edgeflow 888 Washington St. 329 March Road Dedham, MA 02026 Ottawa, Ontario, K2K 2E1 USA Canada Phone: +1 781 251-5319 Email: Email: akullber@netplane.com george.young@edgeflow.com Jonathan P. Lang John Yu Calient Networks Zaffire Inc. 25 Castilian 2630 Orchard Parkway Goleta, CA 93117 San Jose, CA 95134 Email: jplang@calient.net USA Email: jzyu@zaffire.com Fong Liaw Alex Zinin Zaffire Inc. Alcatel 2630 Orchard Parkway 1420 North McDowell Ave San Jose, CA 95134 M/S 1400-HRPB6 USA Petaluma, CA 94954 Email: fliaw@zaffire.com USA Email: alex.zinin@alcatel.com E. Mannie et. al. Internet-Draft February 2003 6 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 2. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [2]. 3. Introduction The architecture presented in this document covers the main building blocks needed to build a consistent control plane for multiple switching layers. It does not restrict the way that these layers work together. Different models can be applied: e.g. overlay, augmented or integrated. Moreover, each pair of contiguous layer may work jointly in a different way, resulting in a number of possible combinations, at the discretion of manufacturers and operators. This architecture clearly separates the control plane and the forwarding plane. In addition, it also clearly separates the control plane in two parts, the signaling plane containing the signaling protocols and the routing plane containing the routing protocols. This document is a generalization of the MPLS architecture [RFC3031], and in some cases may differ slightly from that architecture since non packet-based forwarding planes are now considered. It is not the intention of this document to describe concepts already described in the current MPLS architecture. The goal is to describe specific concepts of Generalized MPLS (GMPLS). However, some of the concepts explained hereafter are not part of the current MPLS architecture and are applicable to both MPLS and GMPLS (i.e. link bundling, unnumbered links, and LSP hierarchy). Since these concepts were introduced together with GMPLS and since they are of paramount importance for an operational GMPLS network, they will be discussed here. The organization of the remainder of this draft is as follows. We begin with an introduction of GMPLS. We then present the specific GMPLS building blocks and explain how they can be combined together to build an operational GMPLS networks. Specific details of the separate building blocks can be found in the corresponding documents. 3.1. Acronyms & abbreviations ABR Area Border Router AS Autonomous System ASBR Autonomous System Boundary Router BGP Border Gateway Protocol CR-LDP Constraint-based Routing LDP CSPF Constraint-based Shortest Path First DWDM Dense Wavelength Division Multiplexing FA Forwarding Adjacency GMPLS Generalized Multi-Protocol Label Switching IGP Interior Gateway Protocol E. Mannie et. al. Internet-Draft February 2003 7 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 LDP Label Distribution Protocol LMP Link Management Protocol LSA Link State Advertisement LSR Label Switching Router LSP Label Switched Path MIB Management Information Base MPLS Multi-Protocol Label Switching NMS Network Management System OXC Optical Cross-Connect PXC Photonic Cross-Connect RSVP ReSource reserVation Protocol SDH Synchronous Digital Hierarchy STM(-N) Synchronous Transport Module (-N) STS(-N) Synchronous Transport Signal-Level N (SONET) TE Traffic Engineering 3.2. Multiple Types of Switching and Forwarding Hierarchies Generalized MPLS (GMPLS) differs from traditional MPLS in that it supports multiple types of switching, i.e. the addition of support for TDM, lambda, and fiber (port) switching. The support for the additional types of switching has driven GMPLS to extend certain base functions of traditional MPLS and, in some cases, to add functionality. These changes and additions impact basic LSP properties, how labels are requested and communicated, the unidirectional nature of LSPs, how errors are propagated, and information provided for synchronizing the ingress and egress LSRs. The MPLS architecture [RFC3031] was defined to support the forwarding of data based on a label. In this architecture, Label Switching Routers (LSRs) were assumed to have a forwarding plane that is capable of (a) recognizing either packet or cell boundaries, and (b) being able to process either packet headers (for LSRs capable of recognizing packet boundaries) or cell headers (for LSRs capable of recognizing cell boundaries). The original MPLS architecture is here being extended to include LSRs whose forwarding plane recognizes neither packet, nor cell boundaries, and therefore, can't forward data based on the information carried in either packet or cell headers. Specifically, such LSRs include devices where the forwarding decision is based on time slots, wavelengths, or physical ports. So, the new set of LSRs, or more precisely interfaces on these LSRs, can be subdivided into the following classes: 1. Packet Switch Capable (PSC) interfaces: Interfaces that recognize packet boundaries and can forward data based on the content of the packet header. Examples include interfaces on routers that forward data based on the content of the IP header and interfaces on routers that forward data based on the content of the MPLS "shim" header. 2. Layer-2 Switch Capable (L2SC) interfaces: E. Mannie et. al. Internet-Draft February 2003 8 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 Interfaces that recognize frame/cell boundaries and can forward data based on the content of the frame/cell header. Examples include interfaces on Ethernet bridges that forward data based on the content of the MAC header and interfaces on ATM-LSRs that forward data based on the ATM VPI/VCI. 3. Time-Division Multiplex Capable (TDM) interfaces: Interfaces that forward data based on the data's time slot in a repeating cycle. An example of such an interface is that of a SDH/SONET Cross-Connect (XC), Terminal Multiplexer (TM), or Add-Drop Multiplexer (ADM). Other examples include interfaces providing G.709 TDM capabilities (the "digital wrapper") and PDH interfaces. 4. Lambda Switch Capable (LSC) interfaces: Interfaces that forward data based on the wavelength on which the data is received. An example of such an interface is that of a Photonic Cross-Connect (PXC) or Optical Cross-Connect (OXC) that can operate at the level of an individual wavelength. Additional examples include PXC interfaces that can operate at the level of a group of wavelengths, i.e. a waveband and G.709 interfaces providing optical capabilities. 5. Fiber-Switch Capable (FSC) interfaces: Interfaces that forward data based on a position of the data in the real world physical spaces. An example of such an interface is that of a PXC or OXC that can operate at the level of a single or multiple fibers. A circuit can be established only between, or through, interfaces of the same type. Depending on the particular technology being used for each interface, different circuit names can be used, e.g. SDH circuit, optical trail, light-path, etc. In the context of GMPLS, all these circuits are referenced by a common name: Label Switched Path (LSP). The concept of nested LSP (LSP within LSP), already available in the traditional MPLS, facilitates building a forwarding hierarchy, i.e. a hierarchy of LSPs. This hierarchy of LSPs can occur on the same interface, or between different interfaces. For example, a hierarchy can be built if an interface is capable of multiplexing several LSPs from the same technology (layer), e.g. a lower order SDH/SONET LSP (VC-12) nested in a higher order SDH/SONET LSP (VC-4). Several levels of signal (LSP) nesting are defined in the SDH/SONET multiplexing hierarchy. The nesting can also occur between interfaces. At the top of the hierarchy are FSC interfaces, followed by LSC interfaces, followed by TDM interfaces, followed by L2SC, and followed by PSC interfaces. This way, an LSP that starts and ends on a PSC interface can be E. Mannie et. al. Internet-Draft February 2003 9 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 nested (together with other LSPs) into an LSP that starts and ends on a L2SC interface. This LSP, in turn, can be nested (together with other LSPs) into an LSP that starts and ends on an TDM interface, which in turn can be nested (together with other LSPs) into an LSP that starts and ends on a LSC interface, which in turn can be nested (together with other LSPs) into an LSP that starts and ends on a FSC interface. 3.3. Extension of the MPLS Control Plane The establishment of LSPs that span only Packet Switch Capable (PSC) or Layer-2 Switch Capable (L2SC) interfaces is defined for the original MPLS and/or MPLS-TE control planes. GMPLS extends these control planes to support each of the five classes of interfaces (i.e. layers) defined in the previous section. Note that the GMPLS control plane supports an overlay model, an augmented model, and a peer (integrated) model. In the near term, GMPLS is very suitable for controlling each layer independently. This elegant approach will facilitate the future deployment of other models. The GMPLS control plane is made of several building blocks are described in more detail in the following sections. These building blocks are based on well-known signaling and routing protocols that have been extended and/or modified to support GMPLS. They use IPv4 and/or IPv6 addresses. Only one new specialized protocol is required to support the operations of GMPLS, a signaling protocol for link management [LMP]. GMPLS is indeed based on the Traffic Engineering (TE) extensions to MPLS, a.k.a. MPLS-TE. This because most of the technologies that can be used below the PSC level require some traffic engineering. The placement of LSPs at these levels needs in general to take several constraints into consideration (such as framing, bandwidth, protection capability, etc) and to bypass the legacy Shortest-Path First (SPF) algorithm. Note, however, that this is not mandatory and that in some cases SPF routing can be applied. In order to facilitate constrained-based SPF routing of LSPs, the nodes performing LSP establishment need more information about the links in the network than standard intra-domain routing protocols provide. These TE attributes are distributed using the transport mechanisms already available in IGPs (e.g. flooding) and taken into consideration by the LSP routing algorithm. Optimization of the LSP routes may also require some external simulations using heuristics that serve as input for the actual path calculation and LSP establishment process. By definition, a TE link is a representation in the ISIS/OSPF Link State advertisements and in the link state database of certain physical resources, and their properties, between two GMPLS nodes. TE Links are used by the GMPLS control plane (routing and signaling) for establishing LSPs. E. Mannie et. al. Internet-Draft February 2003 10 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 Extensions to traditional routing protocols and algorithms are needed to uniformly encode and carry TE link information, and explicit routes (e.g. source routes) are required in the signaling. In addition, the signaling must now be capable of transporting the required circuit (LSP) parameters such as the bandwidth, the type of signal, the desired protection and/or restoration, the position in a particular multiplex, etc. Most of these extensions have already been defined for PSC and L2SC traffic engineering with MPLS. GMPLS primarily defines additional extensions for TDM, LSC, and FSC traffic engineering. A very few elements are technology specific. Thus, GMPLS extends the two signaling protocols defined for MPLS-TE signaling, i.e. RSVP-TE and CR-LDP. However, GMPLS does not specify which one of these two signaling protocols must be used. It is the role of manufacturers and operators to evaluate the two possible solutions for their own interest. Since GMPLS is based on RSVP-TE and CR-LDP, it mandates a downstream-on-demand label allocation and distribution, with an ingress initiated ordered control. Liberal label retention is normally used, but conservative label retention mode could also be used. Furthermore, there is no restriction on the label allocation strategy, it can be request/signaling driven (obvious for circuit switching technologies), traffic/data driven, or even topology driven. There is also no restriction on the route selection; explicit routing is normally used (strict or loose) but hop-by-hop routing could be used as well. GMPLS also extends two traditional intra-domain link-state routing protocols already extended for TE purposes, i.e. OSPF-TE and IS-IS- TE. However, if explicit (source) routing is used, the routing algorithms used by these protocols no longer need to be standardized. Extensions for inter-domain routing (e.g. BGP) are for further study. The use of technologies like DWDM (Dense Wavelength Division Multiplexing) implies that we can now have a very large number of parallel links between two directly adjacent nodes (hundreds of wavelengths, or even thousands of wavelengths if multiple fibers are used). Such a large number of links was not originally considered for an IP or MPLS control plane, although it could be done. Some slight adaptations of that control plane are thus required if we want to better reuse it in the GMPLS context. For instance, the traditional IP routing model assumes the establishment of a routing adjacency over each link connecting two adjacent nodes. Having such a large number of adjacencies does not scale well. Each node needs to maintain each of its adjacencies one by one, and link state routing information must be flooded throughout the network. To solve this issue the concept of link bundling was introduced. Moreover, the manual configuration and control of these links, even E. Mannie et. al. Internet-Draft February 2003 11 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 if they are unnumbered, becomes impractical. The Link Management Protocol (LMP) was specified to solve these issues. LMP runs between data plane adjacent nodes and is used to manage TE links. Specifically, LMP provides mechanisms to maintain control channel connectivity (IP Control Channel Maintenance), verify the physical connectivity of the data-bearing links (Link Verification), correlate the link property information (Link Property Correlation), and manage link failures (Fault Localization and Fault Notification). A unique feature of LMP is that it is able to localize faults in both opaque and transparent networks (i.e. independent of the encoding scheme and bit rate used for the data). LMP is defined in the context of GMPLS, but is specified independently of the GMPLS signaling specification since it is a local protocol running between data-plane adjacent nodes. As a result, LMP can be used in other contexts with non-GMPLS signaling protocols. The MPLS signaling and routing protocols require at least one bi- directional control channel to communicate even if two adjacent nodes are connected by unidirectional links. Several control channels can be used. LMP can be used to establish, maintain and manage these control channels. GMPLS does not specify how these control channels must be implemented, but GMPLS requires IP to transport the signaling and routing protocols over them. Control channels can be either in-band or out-of-band, and several solutions can be used to carry IP. Note also that one type of LMP message (the Test message) is used in-band in the data plane and may not be transported over IP, but this is a particular case, needed to verify connectivity in the data plane. 3.4. GMPLS Key Extensions to MPLS-TE Some key extensions brought by GMPLS to MPLS-TE are highlighted in the following. Some of them are key advantages of GMPLS to control TDM, LSC and FSC layers. - In MPLS-TE, links traversed by an LSP can include an intermix of links with heterogeneous label encoding (e.g. links between routers, links between routers and ATM-LSRs, and links between ATM-LSRs. GMPLS extends this by including links where the label is encoded as a time slot, or a wavelength, or a position in the real world physical space. - In MPLS-TE, an LSP that carries IP has to start and end on a router. GMPLS extends this by requiring an LSP to start and end on similar type of LSR. - The type of a payload that can be carried in GMPLS by an LSP is extended to allow such payloads as SONET/SDH, G.709, 1 or 10Gb Ethernet, etc. E. Mannie et. al. Internet-Draft February 2003 12 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 - The use of Forwarding Adjacencies (FA), provides a mechanism that can improve bandwidth utilization, when bandwidth allocation can be performed only in discrete units, as well as a mechanism to aggregate forwarding state, thus allowing the number of required labels to be reduced. - GMPLS allows for a label to be suggested by an upstream node to reduce the setup latency. This suggestion may be overridden by a downstream node but, in some cases, at the cost of higher LSP setup time. - GMPLS extends on the notion of restricting the range of labels that may be selected by a downstream node. In GMPLS, an upstream node may restrict the labels for an LSP along either a single hop or along the entire LSP path. This feature is useful in photonic networks where wavelength conversion may not be available. - While traditional TE-based (and even LDP-based) LSPs are unidirectional, GMPLS supports the establishment of bi-directional LSPs. - GMPLS supports the termination of an LSP on a specific egress port, i.e. the port selection at the destination side. - GMPLS with RSVP-TE supports an RSVP specific mechanism for rapid failure notification. Note also some other key differences between MPLS-TE and GMPLS: - For TDM, LSC and FSC interfaces, bandwidth allocation for an LSP can be performed only in discrete units. - It is expected to have (much) fewer labels on TDM, LSC or FSC links than on PSC or L2SC links, because the former are physical labels instead of logical labels. 4. Routing and addressing model GMPLS is based on the IP routing and addressing models. This assumes that IPv4 and/or IPv6 addresses are used to identify interfaces and that traditional (distributed) IP routing protocols are also reused. Indeed, the discovery of the topology and the resource state of all links in a routing domain is achieved via these routing protocols. Since control and data planes are de-coupled in GMPLS, control-plane neighbors (i.e. IGP-learnt neighbors) may not be anymore data-plane neighbors, hence mechanisms like LMP are needed to associate TE links with neighboring nodes. IP addresses are not used only to identify interfaces of IP hosts and routers, but more generally to identify any PSC and non-PSC interfaces. Similarly, IP routing protocols are used to find routes for IP datagrams with a SPF algorithm, and are also used to find routes for non-PSC circuits by using a CSPF algorithm. E. Mannie et. al. Internet-Draft February 2003 13 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 However, some additional mechanisms are needed to increase the scalability of these models and to deal with specific traffic engineering requirements of non-PSC layers. These mechanisms will be introduced in the following. Re-using existing IP routing protocols allows for non-PSC layers to take advantages of all the valuable developments that took place since years for IP routing, in particular in the context of intra- domain routing (link-state routing) and inter-domain routing (policy routing). In an overlay model, each particular non-PSC layer can be seen as a set of Autonomous Systems (ASs) interconnected in an arbitrary way. Similarly to the traditional IP routing, each AS is managed by a single administrative authority. For instance, an AS can be an SDH/SONET network operated by a given carrier. The set of interconnected ASs being an SDH/SONET Internetwork. Exchange of routing information between ASs can be done via an inter-domain routing protocol like BGP-4. There is obviously a huge value of re-using well-known policy routing facilities provided by BGP in a non-PSC context. Extensions for BGP traffic engineering (BGP-TE) in the context of non-PSC layers are left for further study. Each AS can be subdivided in different routing domains, and each can run a different intra-domain routing protocol. In turn, each routing-domain can be divided in areas. A routing domain is made of GMPLS enabled nodes (i.e. a network device including a GMPLS entity). These nodes can be either edge nodes (i.e. hosts, ingress LSRs or egress LSRs), or internal LSRs. An example of non-PSC host is an SDH/SONET Terminal Multiplexer (TM). Another example is an SDH/SONET interface card within an IP router or ATM switch. Note that traffic engineering in the intra-domain requires the use of link-state routing protocols like OSPF or IS-IS. GMPLS defines extensions to these protocols. These extensions are needed to disseminate specific TDM, LSC and FSC static and dynamic characteristics related to nodes and links. The current focus is on intra-area traffic engineering. However, inter-area traffic engineering is also under investigation. 4.1. Addressing of PSC and non-PSC layers The fact that IPv4 and/or IPv6 addresses are used doesn't imply at all that they should be allocated in the same addressing space than public IPv4 and/or IPv6 addresses used for the Internet. Private IP addresses can be used if they don't require to be exchanged with any other operator, public IP addresses are otherwise required. Of E. Mannie et. al. Internet-Draft February 2003 14 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 course, if an integrated model is used, two layers could share the same addressing space. Note that there is a benefit of using public IPv4 and/or IPv6 Internet addresses for non-PSC layers if an integrated model with the IP layer is foreseen. If we consider the scalability enhancements proposed in the next section, the IPv4 (32 bits) and the IPv6 (128 bits) addressing spaces are both more than sufficient to accommodate any non-PSC layer. We can reasonably expect to have much less non-PSC devices (e.g. SDH/SONET nodes) than we have today IP hosts and routers. 4.2. GMPLS scalability enhancements TDM, LSC and FSC layers introduce new constraints on the IP addressing and routing models since several hundreds of parallel physical links (e.g. wavelengths) can now connect two nodes. Most of the carriers already have today several tens of wavelengths per fiber between two nodes. New generation of DWDM systems will allow several hundreds of wavelengths per fiber. It becomes rather impractical to associate an IP address with each end of each physical link, to represent each link as a separate routing adjacency, and to advertise and to maintain link states for each of these links. For that purpose, GMPLS enhances the MPLS routing and addressing models to increase their scalability. Two optional mechanisms can be used to increase the scalability of the addressing and the routing: unnumbered links and link bundling. These two mechanisms can also be combined. They require extensions to signaling (RSVP-TE and CR-LDP) and routing (OSPF-TE and IS-IS-TE) protocols. 4.3. TE Extensions to IP routing protocols Traditionally, a TE link is advertised as an adjunct to a "regular" OSPF or IS-IS link, i.e., an adjacency is brought up on the link, and when the link is up, both the regular IGP properties of the link (basically, the SPF metric) and the TE properties of the link are then advertised. However, GMPLS challenges this notion in three ways: - First, links that are non-PSC may yet have TE properties; however, an OSPF adjacency could not be brought up directly on such links. - Second, an LSP can be advertised as a point-to-point TE link in the routing protocol, i.e. as a Forwarding Adjacency (FA); thus, an advertised TE link need no longer be between two OSPF direct neighbors. Forwarding Adjacencies (FA) are further described in a separate section. E. Mannie et. al. Internet-Draft February 2003 15 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 - Third, a number of links may be advertised as a single TE link (e.g. for improved scalability), so again, there is no longer a one- to-one association of a regular adjacency and a TE link. Thus we have a more general notion of a TE link. A TE link is a logical link that has TE properties, some of which may be configured on the advertising LSR, others which may be obtained from other LSRs by means of some protocol, and yet others which may be deduced from the component(s) of the TE link. An important TE property of a TE link is related to the bandwidth accounting for that link. GMPLS will define different accounting rules for different non-PSC layers. Generic bandwidth attributes are however defined by the TE routing extensions and by GMPLS, such as the unreserved bandwidth, the maximum reservable bandwidth, the maximum LSP bandwidth. It is expected in a dynamic environment to have frequent changes of bandwidth accounting information. A flexible policy for triggering link state updates based on bandwidth thresholds and link-dampening mechanism can be implemented. TE properties associated with a link should also capture protection and restoration related characteristics. For instance, shared protection can be elegantly combined with bundling. Protection and restoration are mainly generic mechanisms also applicable to MPLS. It is expected that they will first be developed for MPLS and later on generalized to GMPLS. A TE link between a pair of LSRs doesn't imply the existence of an IGP adjacency between these LSRs. A TE link must also have some means by which the advertising LSR can know of its liveness (e.g. by using LMP hellos). When an LSR knows that a TE link is up, and can determine the TE link's TE properties, the LSR may then advertise that link to its GMPLS enhanced OSPF or IS-IS neighbors using the TE objects/TLVs. We call the interfaces over which GMPLS enhanced OSPF or ISIS adjacencies are established "control channels". 5. Unnumbered links Unnumbered links (or interfaces) are links (or interfaces) that do not have IP addresses. Using such links involves two capabilities: the ability to specify unnumbered links in MPLS TE signaling, and the ability to carry (TE) information about unnumbered links in IGP TE extensions of ISIS-TE and OSPF-TE. A. The ability to specify unnumbered links in MPLS TE signaling requires extensions to RSVP-TE and CR-LDP. The MPLS-TE signaling doesn't provide support for unnumbered links, because it doesnÆt provide a way to indicate an unnumbered link in its Explicit Route Object/TLV and in its Record Route Object (there is no such TLV for CR-LDP). GMPLS defines simple extensions to indicate an unnumbered link in these two Objects/TLVs, using a new Unnumbered Interface ID sub-object/sub-TLV. E. Mannie et. al. Internet-Draft February 2003 16 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 Since unnumbered links are not identified by an IP address, then for the purpose of MPLS TE each end need some other identifier, local to the LSR to which the link belongs. LSRs at the two end points of an unnumbered link exchange with each other the identifiers they assign to the link. Exchanging the identifiers may be accomplished by configuration, by means of a protocol such as LMP ([LMP]), by means of RSVP/CR-LDP (especially in the case where a link is a Forwarding Adjacency, see below), or by means of IS-IS or OSPF extensions ([ISIS-GMPLS], [OSPF-GMPLS]). Consider an (unnumbered) link between LSRs A and B. LSR A chooses an identifier for that link. So is LSR B. From A's perspective we refer to the identifier that A assigned to the link as the "link local identifier" (or just "local identifier"), and to the identifier that B assigned to the link as the "link remote identifier" (or just "remote identifier"). Likewise, from B's perspective the identifier that B assigned to the link is the local identifier, and the identifier that A assigned to the link is the remote identifier. The new Unnumbered Interface ID sub-object/sub-TLV for the ER Object/TLV contains the Router ID of the LSR at the upstream end of the unnumbered link and the outgoing interface identifier or the link local identifier with respect to that upstream LSR. The new Unnumbered Interface ID sub-object for the RR Object contains the outgoing interface identifier with respect to the LSR that adds it in the RR Object. B. The ability to carry (TE) information about unnumbered links in IGP TE extensions requires new sub-TLVs for the extended IS reachability TLV defined in ISIS-TE and for the TE LSA (which is an opaque LSA) defined in OSPF-TE. A Link Local Identifier sub-TLV and a Link Remote Identifier sub-TLV are defined. 5.1. Unnumbered Forwarding Adjacencies If an LSR that originates an LSP advertises this LSP as an unnumbered FA in IS-IS or OSPF, or the LSR uses this FA as an unnumbered component link of a bundled link, the LSR must allocate an Interface ID to that FA. If the LSP is bi-directional, the tail end does the same and allocates an Interface ID to the reverse FA. Signaling has been enhanced to carry the Interface ID of a FA in the new LSP Tunnel Interface ID object/TLV. This object/TLV contains the Router ID of the LSR that generates it, and the Interface ID. It is called the Forward Interface ID when it appears in a Path/REQUEST message, and it is called the Reverse Interface ID when it appears in the Resv/MAPPING message. 6. Link bundling E. Mannie et. al. Internet-Draft February 2003 17 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 The concept of link bundling is essential in certain networks employing the GMPLS control plane as is defined in [BUNDLE]. A typical example is an optical meshed network where adjacent optical cross-connects (LSRs) are connected by several hundreds of parallel wavelengths. In this network, consider the application of link state routing protocols, like OSPF or IS-IS, with suitable extensions for resource discovery and dynamic route computation. Each wavelength must be advertised separately in order to be used, except if link bundling is used. When a pair of LSRs is connected by multiple links, it is possible to advertise several (or all) of these links as a single link into OSPF and/or IS-IS. This process is called link bundling, or just bundling. The resulting logical link is called a bundled link as its physical links are called component links (and are identified by interface indexes). It results that a combination of three identifiers ((bundled) link identifier, component link identifier, label) is sufficient to unambiguously identify the appropriate resources used by an LSP. The purpose of link bundling is to improve routing scalability by reducing the amount of information that has to be handled by OSPF and/or IS-IS. This reduction is accomplished by performing information aggregation/abstraction. As with any other information aggregation/abstraction, this results in losing some of the information. To limit the amount of losses one need to restrict the type of the information that can be aggregated/abstracted. 6.1. Restrictions on bundling The following restrictions are required for bundling links. All component links in a bundle must begin and end on the same pair of LSRs; and share some common characteristics or properties defined in [OSPF-TE] and [ISIS-TE], i.e. they must have the same: - Link Type (i.e. point-to-point or multi-access), - TE Metric (i.e. an administrative cost), - Set of Resource Classes at each end of the links (i.e. colors). Note that an FA may also be a component link. In fact, a bundle can consist of a mix of point-to-point links and FAs, but all sharing some common properties. 6.2. Routing considerations for bundling A bundled link is just another kind of TE link such as those defined by [GMPLS-ROUTING]. The liveness of the bundled link is determined by the liveness of each its component links, a bundled link is alive when at least one of its component links is alive. The liveness of a component link can be determined by any of several means: IS-IS or OSPF hellos over the component link, or RSVP Hello (hop local), or LMP hellos (link local), or from layer 1 or layer 2 indications. E. Mannie et. al. Internet-Draft February 2003 18 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 Note that according to the RSVP-TE Tunnel specification the RSVP Hello mechanism is intended to be used when notification of link layer failures is not available and unnumbered links are not used, or when the failure detection mechanisms provided by the link layer are not sufficient for timely node failure detection. Once a bundled link is determined to be alive, it can be advertised as a TE link and the TE information can be flooded. If IS-IS/OSPF hellos are run over the component links, IS-IS/OSPF flooding can be restricted to just one of the component links. Note that advertising a (bundled) TE link between a pair of LSRs doesn't imply that there is an IGP adjacency between these LSRs that is associated with just that link. In fact, in certain cases a TE link between a pair of LSRs could be advertised even if there is no IGP adjacency at all between the LSR (e.g. when the TE link is an FA). Forming a bundled link consist in aggregating the identical TE parameters of each individual component link to produce aggregated TE parameters. A TE link as defined by [GMPLS-ROUTING] has many parameters, adequate aggregation rules must be defined for each one. Some parameters can be sums of component characteristics such as the unreserved bandwidth and the maximum reservable bandwidth. Bandwidth information is an important part of a bundle advertisement and it must be clearly defined since an abstraction is done. A GMPLS node with bundled links must apply admission control on a per-component link basis. 6.3. Signaling considerations Typically, an LSP's explicit route (e.g. contained in an explicit route TLV/Object) will choose the bundled link to be used for the LSP, but not the component link(s), since information about the bundled link is flooded, but information about the component links is not. The choice of the component link to use is always made by an upstream node. If the LSP is bidirectional, the upstream node chooses a component link in each direction. Three mechanisms for indicating this choice to the downstream node are possible. 6.3.1. Mechanism 1: Implicit Indication This mechanism requires that each component link has a dedicated signaling channel (e.g. the link is a Sonet/SDH link using the DCC for in-band signaling). The upstream node tells the receiver which component link to use by sending the message over the chosen component link's dedicated signaling channel. Note that this signaling channel can be in-band or out-of-band. In this last case, E. Mannie et. al. Internet-Draft February 2003 19 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 the association between the signaling channel and that component link need to be explicitly configured. 6.3.2. Mechanism 2: Explicit Indication by Numbered Interface ID This mechanism requires that the component link has a unique remote IP address. The upstream node indicates the choice of the component link by including a new IF_ID RSVP_HOP object/IF_ID TLV carrying either an IPv4 or an IPv6 address in the Path/Label Request message [RSVP-TE-GMPLS] [CR-LDP-GMPLS]. For a bi-directional LSP, a component link is provided for each direction by the upstream node. This mechanism does not require each component link to have its own control channel. In fact, it doesn't even require the whole (bundled) link to have its own control channel. 6.3.3. Mechanism 3: Explicit Indication by Unnumbered Interface ID With this mechanism, each component link that is unnumbered is assigned a unique Interface Identifier (32 bits value). The upstream node indicates the choice of the component link by including a new IF_ID RSVP_HOP object/IF_ID TLV in the Path/Label Request message [RSVP-TE-GMPLS] [CR-LDP-GMPLS]. This object/TLV carries the component interface ID in the downstream direction for a unidirectional LSP, and in addition the component interface ID in the upstream direction for a bi-directional LSP. The two LSRs at each end of the bundled link exchange these identifiers. Exchanging the identifiers may be accomplished by configuration, by means of a protocol such as LMP (preferred solution), by means of RSVP/CR-LDP (especially in the case where a component link is a Forwarding Adjacency), or by means of IS-IS or OSPF extensions. This mechanism does not require each component link to have its own control channel. In fact, it doesn't even require the whole (bundled) link to have its own control channel. 6.4. Unnumbered Bundled Link A bundled link may itself be numbered or unnumbered independent of whether the component links are numbered or not. This affects how the bundled link is advertised in IS-IS/OSPF, and the format of LSP EROs that traverse the bundled link. Furthermore, unnumbered Interface Identifiers for all unnumbered outgoing links of a given LSR (whether component links, Forwarding Adjacencies or bundled links) must be unique in the context of that LSR. 6.5. Forming bundled links The generic rule for bundling component links is to place those links that are correlated in some manner in the same bundle. If links may be correlated based on multiple properties then the E. Mannie et. al. Internet-Draft February 2003 20 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 bundling may be applied sequentially based on these properties. For instance, links may be first grouped based on the first property. Each of these groups may be then divided into smaller groups based on the second property and so on. The main principle followed in this process is that the properties of the resulting bundles should be concisely summarizable. Link bundling may be done automatically or by configuration. Automatic link bundling can apply bundling rules sequentially to produce bundles. For instance, the first property on which component links may be correlated could be the Interface Switching Capability [GMPLS- ROUTING], the second property could be the Encoding [GMPLS-ROUTING], the third property could be the Administrative Weight (cost), the fourth property could be the Resource Classes and finally links may be correlated based on other metrics such as SRLG (Shared Risk Link Groups). When routing an alternate path for protection purposes, the general principle followed is that the alternate path is not routed over any link belonging to an SRLG that some link in the primary path belongs to. Thus, the rule to be followed is to group links belonging to exactly the same set of SRLGs. This type of sequential sub-division may result in a number of bundles between two adjacent nodes. In practice, however, the link properties may not be very heterogeneous among component links between two adjacent nodes. Thus, the number of bundles in practice may not be large. 7. Relationship with the UNI The interface between an edge GMPLS node and a GMPLS LSR on the network side may be referred to as a User to Network Interface (UNI), while the interface between two network side LSRs may be referred to as a Network to Network Interface (NNI). GMPLS does not specify separately a UNI and an NNI. Edge nodes are connected to LSRs on the network side, and these LSRs are in turn connected between them. Of course, the behavior of an edge node is not exactly the same as the behavior of an LSR on the network side. Note also, that an edge node may run a routing protocol, however it is expected that in most of the cases it will not (see also section 7.2 and the section about signaling with an explicit route). Conceptually, a difference between UNI and NNI make sense either if both interface uses completely different protocols, or if they use the same protocols but with some outstanding differences. In the first case, separate protocols are often defined successively, with more or less success. The GMPLS approach consisted in building a consistent model from day one, considering both the UNI and NNI interfaces at the same time. For that purpose a very few specific UNI particularities have been ignored in a first time. GMPLS has been enhanced to support such E. Mannie et. al. Internet-Draft February 2003 21 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 particularities at the UNI by some other standardization bodies (see hereafter). 7.1. Relationship with the OIF UNI This section is only given for reference to the OIF work related to GMPLS. The current OIF UNI specification [OIF-UNI] defines an interface between a client SDH/SONET equipment and an SDH/SONET network, each belonging to a distinct administrative authority. It is designed for an overlay model. The OIF UNI defines additional mechanisms on the top of GMPLS for the UNI. For instance, the OIF service discovery procedure is a precursor to obtaining UNI services. Service discovery allows a client to determine the static parameters of the interconnection with the network, including the UNI signaling protocol, the type of concatenation, the transparency level as well as the type of diversity (node, link, SRLG) supported by the network. Since the current OIF UNI interface does not cover photonic networks, G.709 Digital Wrapper, etc, it is from that perspective a subset of the GMPLS Architecture at the UNI. 7.2. Reachability across the UNI This section discusses the selection of an explicit route by an edge node. The selection of the first LSR by an edge node connected to multiple LSRs is part of that problem. An edge node (host or LSR) can participate more or less deeply in the GMPLS routing. Four different routing models can be supported at the UNI: configuration based, partial peering, silent listening and full peering. - Configuration based: this routing model requires the manual or automatic configuration of an edge node with a list of neighbor LSRs sorted by preference order. Automatic configuration can be achieved using DHCP for instance. No routing information is exchanged at the UNI, except maybe the ordered list of LSRs. The only routing information used by the edge node is that list. The edge node sends by default an LSP request to the preferred LSR. ICMP redirects could be send by this LSR to redirect some LSP requests to another LSR connected to the edge node. GMPLS does not preclude that model. - Partial peering: limited routing information (mainly reachability) can be exchanged across the UNI using some extensions in the signaling plane. The reachability information exchanged at the UNI may be used to initiate edge node specific routing decision over the network. GMPLS does not have any capability to support this model today. - Silent listening: the edge node can silently listen to routing protocols and take routing decisions based on the information obtained. An edge node receives the full routing information, E. Mannie et. al. Internet-Draft February 2003 22 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 including traffic engineering extensions. One LSR should forward transparently all routing pdus to the edge node. An edge node can now compute a complete explicit route taking into consideration all the end-to-end routing information. GMPLS does not preclude this model. - Full peering: In addition to silent listening, the edge node participates within the routing, establish adjacencies with its neighbors and advertises LSAs. This is useful only if there are benefits for edge nodes to advertise themselves traffic engineering information. GMPLS does not preclude this model. 8. Link Management In the context of GMPLS, a pair of nodes (e.g., a photonic switch) may be connected by tens of fibers, and each fiber may be used to transmit hundreds of wavelengths if DWDM is used. Multiple fibers and/or multiple wavelengths may also be combined into one or more bundled links for routing purposes. Furthermore, to enable communication between nodes for routing, signaling, and link management, control channels must be established between a node pair. Link management is a collection of useful procedures between adjacent nodes that provide local services such as control channel management, link connectivity verification, link property correlation, and fault management. The Link Management Protocol (LMP) has been defined to fulfill these operations. LMP was initiated in the context of GMPLS but is a generic toolbox that can be also used in other contexts. Control channel management and link property correlation are mandatory procedures of LMP. Link connectivity verification and fault management are optional procedures. 8.1. Control channel and control channel management LMP control channel management is used to establish and maintain control channels between nodes. Control channels exist independently of TE links, and can be used to exchange MPLS control-plane information such as signaling, routing, and link management information. An "LMP adjacency" is formed between two nodes that support the same LMP capabilities. Multiple control channels may be active simultaneously for each adjacency. A control channel can be either explicitly configured or automatically selected, however, LMP currently assume that control channels are explicitly configured while the configuration of the control channel capabilities can be dynamically negotiated. For the purposes of LMP, the exact implementation of the control channel is left unspecified. The control channel(s) between two adjacent nodes is no longer required to use the same physical medium E. Mannie et. al. Internet-Draft February 2003 23 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 as the data-bearing links between those nodes. For example, a control channel could use a separate wavelength or fiber, an Ethernet link, or an IP tunnel through a separate management network. A consequence of allowing the control channel(s) between two nodes to be physically diverse from the associated data-bearing links is that the health of a control channel does not necessarily correlate to the health of the data-bearing links, and vice-versa. Therefore, new mechanisms have been developed in LMP to manage links, both in terms of link provisioning and fault isolation. LMP does not specify the signaling transport mechanism used in the control channel, however it states that messages transported over a control channel must be IP encoded. Furthermore, since the messages are IP encoded, the link level encoding is not part of LMP. A 32-bit non-zero integer control channel identifier (CCId) is assigned to each direction of a control channel. Each control channel individually negotiates its control channel parameters and maintains connectivity using a fast Hello protocol. The latter is required if lower-level mechanisms are not available to detect link failures. The Hello protocol of LMP is intended to be a lightweight keep-alive mechanism that will react to control channel failures rapidly so that IGP Hellos are not lost and the associated link-state adjacencies are not removed unnecessarily. The Hello protocol consists of two phases: a negotiation phase and a keep-alive phase. The negotiation phase allows negotiation of some basic Hello protocol parameters, like the Hello frequency. The keep- alive phase consists of a fast lightweight bi-directional Hello message exchange. If a group of control channels share a common node pair and support the same LMP capabilities, then LMP control channel messages (except Configuration messages, and Hello) may be transmitted over any of the active control channels without coordination between the local and remote nodes. For LMP, it is essential that at least one control channel is always available. In the event of a control channel failure, it may be possible to use an alternate active control channel without coordination. 8.2. Link property correlation As part of LMP, a link property correlation exchange is defined. The exchange is used to aggregate multiple data-bearing links (i.e. component links) into a bundled link and exchange, correlate, or change TE link parameters. The link property correlation exchange may be done at any time a link is up and not in the Verification process (see next section). E. Mannie et. al. Internet-Draft February 2003 24 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 It allows for instance to add component links to a link bundle, change a link's protection mechanism, change port identifiers, or change component identifiers in a bundle. This mechanism is supported by an exchange of link summary messages. 8.3. Link connectivity verification Link connectivity verification is an optional procedure that may be used to verify the physical connectivity of data-bearing links as well as to exchange the link identifiers that are used in the GMPLS signaling. The use of this procedure is negotiated as part of the configuration exchange that take place during the negotiation phase of the Hello protocol. This procedure should be done initially when a data- bearing link is first established, and subsequently, on a periodic basis for all unallocated (free) data-bearing links. The verification procedure consists of sending Test messages in-band over the data-bearing links. This requires that the unallocated links must be opaque; however, multiple degrees of opaqueness (e.g., examining overhead bytes, terminating the payload, etc.), and hence different mechanisms to transport the Test messages, are specified. Note that the Test message is the only LMP message that is transmitted over the link, and that Hello messages continue to be exchanged over the control channel during the link verification process. Data-bearing links are tested in the transmit direction as they are unidirectional. As such, it is possible for LMP neighboring nodes to exchange the Test messages simultaneously in both directions. To initiate the link verification procedure, a node must first notify the adjacent node that it will begin sending Test messages over a particular data-bearing link, or over the component links of a particular bundled link. The node must also indicate the number of data-bearing links that are to be verified; the interval at which the test messages will be sent; the encoding scheme, the transport mechanism that are supported, data rate for Test messages; and, in the case where the data-bearing links correspond to fibers, the wavelength over which the Test messages will be transmitted. Furthermore, the local and remote bundled link identifiers are transmitted at this time to perform the component link association with the bundled link identifiers. 8.4. Fault management Fault management is an important requirement from the operational point of view. Fault management includes usually: fault detection, fault localization and fault notification. When a failure occurs and is detected (fault detection), an operator needs to know exactly where it happened (fault localization) and a source node may need to be notified in order to take some actions (fault notification). E. Mannie et. al. Internet-Draft February 2003 25 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 Note that fault localization can also be used to support some specific local protection/restoration mechanisms. In new technologies such as transparent photonic switching currently no method is defined to locate a fault, and the mechanism by which the fault information is propagated must be sent "out of band" (via the control plane). LMP provides a fault localization procedure that can be used to rapidly localize link failures, by notifying a fault up to the node upstream of that fault (i.e. through a fault notification procedure). A downstream LMP neighbor that detects data link failures will send an LMP message to its upstream neighbor notifying it of the failure. When an upstream node receives a failure notification, it can correlate the failure with the corresponding input ports to determine if the failure is between the two nodes. Once the failure has been localized, the signaling protocols can be used to initiate link or path protection/restoration procedures. 8.5 LMP for DWDM Optical Line Systems (OLSs) In an all-optical environment, LMP focuses on peer communications (e.g. OXC-to-OXC). A great deal of information about a link between two OXCs is known by the OLS (Optical Line System or WDM Terminal multiplexer). Exposing this information to the control plane can improve network usability by further reducing required manual configuration and also by greatly enhancing fault detection and recovery. LMP-WDM [LMP-WDM] defines extensions to LMP for use between and OXC and an OLS. These extensions are intended to satisfy the Optical Link Interface Requirements described in [OLI-REQ]. Fault detection is particularly an issue when the network is using all-optical photonic switches (PXC). Once a connection is established, PXCs have only limited visibility into the health of the connection. Even though the PXC is all-optical, long-haul OLSs typically terminate channels electrically and regenerate them optically, which presents an opportunity to monitor the health of a channel between PXCs. LMP-WDM can then be used by the OLS to provide this information to the PXC. In addition to the link information known to the OLS that is exchanged through LMP-WDM, some information known to the OXC may also be exchanged with the OLS through LMP-WDM. This information is useful for alarm management and link monitoring (e.g. trace monitoring). Alarm management is important because the administrative state of a connection, known to the OXC (e.g. this information may be learned through the Admin Status object of GMPLS signaling [GMPLS]), can be used to suppress spurious alarms. For example, the OXC may know that a connection is "up", "down", in a "testing" mode, or being deleted ("deletion-in-progress"). The OXC E. Mannie et. al. Internet-Draft February 2003 26 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 can use this information to inhibit alarm reporting from the OLS when a connection is "down", "testing", or being deleted. It is important to note that an OXC may peer with one or more OLSs and an OLS may peer with one or more OXCs. Although there are many similarities between an OXC-OXC LMP session and an OXC-OLS LMP session, particularly for control management and link verification, there are some differences as well. These differences can primarily be attributed to the nature of an OXC-OLS link, and the purpose of OXC-OLS LMP sessions. The OXC-OXC links can be used to provide the basis for GMPLS signaling and routing at the optical layer. The information exchanged over LMP-WDM sessions is used to augment knowledge about the links between OXCs. In order for the information exchanged over the OXC-OLS LMP sessions to be used by the OXC-OXC session, the information must be coordinated by the OXC. However, the OXC-OXC and OXC-OLS LMP sessions are run independently and must be maintained separately. One critical requirement when running an OXC-OLS LMP session is the ability of the OLS to make a data link transparent when not doing the verification procedure. This is because the same data link may be verified between OXC-OLS and between OXC-OXC. The verification procedure of LMP is used to coordinate the Test procedure (and hence the transparency/opaqueness of the data links). To maintain independence between the sessions, it must be possible for the LMP sessions to come up in any order. In particular, it must be possible for an OXC-OXC LMP session to come up without an OXC-OLS LMP session being brought up, and vice-versa. 9. Generalized Signaling The GMPLS signaling extends certain base functions of the RSVP-TE and CR-LDP signaling and, in some cases, adds functionality. These changes and additions impact basic LSP properties, how labels are requested and communicated, the unidirectional nature of LSPs, how errors are propagated, and information provided for synchronizing the ingress and egress. The core GMPLS signaling specification is available in three parts: 1. A signaling functional description [GMPLS-SIG]. 2. RSVP-TE extensions [RSVP-TE-GMPLS]. 3. CR-LDP extensions [CR-LDP-GMPLS]. In addition, independent parts are available per technology: 1. GMPLS extensions for SONET and SDH control [SONETSDH-GMPLS]. 2. GMPLS extensions for G.709 control [G709-GMPLS]. The following MPLS profile expressed in terms of MPLS features [RFC3031] applies to GMPLS: - Downstream-on-demand label allocation and distribution. - Ingress initiated ordered control. E. Mannie et. al. Internet-Draft February 2003 27 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 - Liberal (typical), or conservative (could) label retention mode. - Request, traffic/data, or topology driven label allocation strategy. - Explicit routing (typical), or hop-by-hop routing. The GMPLS signaling defines the following new building blocks on the top of MPLS-TE: 1. A new generic label request format. 2. Labels for TDM, LSC and FSC interfaces, generically known as Generalized Label. 3. Waveband switching support. 4. Label suggestion by the upstream for optimization purposes (e.g. latency). 5. Label restriction by the upstream to support some optical constraints. 6. Bi-directional LSP establishment with contention resolution. 7. Rapid failure notification extensions. 8. Protection information currently focusing on link protection, plus primary and secondary LSP indication. 9. Explicit routing with explicit label control for a fine degree of control. 10. Specific traffic parameters per technology. 11. LSP administrative status handling. These building blocks will be described in mode details in the following. A complete specification can be found in the corresponding documents. Note that GMPLS is highly generic and has many options. Only building blocks 1, 2 and 10 are mandatory, and only within the specific format that is needed. Typically building blocks 6 and 9 should be implemented. Building blocks 3, 4, 5, 7, 8 and 11 are optional. A typical SDH/SONET switching network would implement building blocks: 1, 2 (the SDH/SONET label), 6, 9, 10 and 11. Building blocks 7 and 8 are optional since the protection/restoration can be achieved using SDH/SONET overhead bytes. A typical wavelength switching network would implement building blocks: 1, 2 (the generic format), 4, 5, 6, 7, 8, 9 and 11. Building block 3 is only needed in the particular case of waveband switching. A typical fiber switching network would implement building blocks: 1, 2 (the generic format), 6, 7, 8, 9 and 11. A typical MPLS-IP network would not implement any of these building blocks, since the absence of building block 1 would indicate regular MPLS-IP. Note however that building block 1 and 8 can be used to signal MPLS-IP as well. In that case, the MPLS-IP network can benefit from the link protection type (not available in CR-LDP, some E. Mannie et. al. Internet-Draft February 2003 28 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 very basic form being available in RSVP-TE). Building block 2 is here a regular MPLS label and no new label format is required. GMPLS does not specify any profile for RSVP-TE and CR-LDP implementations that have to support GMPLS - except for what is directly related to GMPLS procedures. It is to the manufacturer to decide which are the optional elements and procedures of RSVP-TE and CR-LDP that need to be implemented. Some optional MPLS-TE elements can be useful for TDM, LSC and FSC layers, for instance the setup and holding priorities that are inherited from MPLS-TE. 9.1. Overview: How to Request an LSP A TDM, LSC or FSC LSP is established by sending a PATH/Label Request message downstream to the destination. This message contains a Generalized Label Request with the type of LSP (i.e. the layer concerned), and its payload type. An Explicit Route (ERO) is also normally added to the message, but this can be added and/or completed by the first/default LSR. The requested bandwidth is encoded in the RSVP-TE SENDER_TSPEC object, or in the CR-LDP Traffic Parameters TLV. Specific parameters for a given technology are given in these traffic parameters, such as the type of signal, concatenation and/or transparency for a SDH/SONET LSP. For some other technology there be could just one bandwidth parameter indicating the bandwidth as a floating-point value. The requested local protection per link may be requested using the Protection Information Object/TLV. The end-to-end LSP protection is for further study and is introduced LSP protection/restoration section (see after). If the LSP is a bi-directional LSP, an Upstream Label is also specified in the Path/Label Request message. This label will be the one to use in the upstream direction. Additionally, a Suggested Label, a Label Set and a Waveband Label can also be included in the message. Other operations are defined in MPLS-TE. The downstream node will send back a Resv/Label Mapping message including one Generalized Label object/TLV that can contain several Generalized Labels. For instance, if a concatenated SDH/SONET signal is requested, several labels can be returned. In case of SDH/SONET virtual concatenation, a list of labels is returned. Each label identifying one element of the virtual concatenated signal. This limits virtual concatenation to remain within a single (component) link. In case of any type of SDH/SONET contiguous concatenation, only one label is returned. That label is the lowest signal of the contiguous concatenated signal (given an order specified in [SONETSDH-GMPLS]). E. Mannie et. al. Internet-Draft February 2003 29 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 In case of SDH/SONET "multiplication", i.e. co-routing of circuits of the same type but without concatenation but all belonging to the same LSP, the explicit ordered list of all signals that take part in the LSP is returned. 9.2. Generalized Label Request The Generalized Label Request is a new object/TLV to be added in an RSVP-TE Path message instead of the regular Label Request, or in a CR-LDP Request message in addition to the already existing TLVs. Only one label request can be used per message, so a single LSP can be requested at a time per signaling message. The Generalized Label Request gives three major characteristics (parameters) required to support the LSP being requested: the LSP Encoding Type, the Switching Type that must be used and the LSP payload type called Generalized PID (G-PID). The LSP Encoding Type indicates the encoding type that will be used with the data associated with the LSP, i.e. the type of technology being considered. For instance, it can be SDH, SONET, Ethernet, ANSI PDH, etc. It represents the nature of the LSP, and not the nature of the links that the LSP traverses. This is used hop-by-hop by each node. A link may support a set of encoding formats, where support means that a link is able to carry and switch a signal of one or more of these encoding formats. The Switching Type indicates then the type of switching that should be performed on a particular link for that LSP. This information is needed for links that advertise more than one type of switching capability. Nodes must verify that the type indicated in the Switching Type is supported on the corresponding incoming interface; otherwise the node must generate a notification message with a "Routing problem/Switching Type" indication. The LSP payload type (G-PID) identifies the payload carried by the LSP, i.e. an identifier of the client layer of that LSP. For some technologies it also indicates the mapping used by the client layer, e.g. byte synchronous mapping of E1. This must be interpreted according to the LSP encoding type of the LSP and is used by the nodes at the endpoints of the LSP to know to which client layer a request is destined, and in some cases by the penultimate hop. Other technology specific parameters are not transported in the Generalized Label Request but in technology specific traffic parameters as explained hereafter. Currently, two set of traffic parameters are defined, one for SONET/SDH and one for G.709. Note that it is expected than specific traffic parameters will be defined in the future for photonic (all optical) switching. E. Mannie et. al. Internet-Draft February 2003 30 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 9.3. SONET/SDH Traffic Parameters The GMPLS SDH/SONET traffic parameters [SONETSDH-GMPLS] specify a powerful set of capabilities for SONET (ANSI T1.105) and SDH (ITU-T G.707). Optional non-standard capabilities are also available in [SONETSDH-EXT-GMPLS]. The first traffic parameter specifies the type of the elementary SONET/SDH signal that comprises the requested LSP, e.g. VC-11, VT6, VC-4, STS-3c, etc. Several transforms can then be applied successively on the elementary Signal to build the final signal being actually requested for the LSP. These transforms are the contiguous concatenation, the virtual concatenation, the transparency and the multiplication. Each one is optional. They must be applied strictly in the following order: - First, contiguous concatenation can be optionally applied on the Elementary Signal, resulting in a contiguously concatenated signal. - Second, virtual concatenation can be optionally applied either directly on the elementary Signal, or on the contiguously concatenated signal obtained from the previous phase. - Third, some transparency can be optionally specified when requesting a frame as signal rather than a container. Several transparency packages are defined. - Fourth, a multiplication can be optionally applied either directly on the elementary Signal, or on the contiguously concatenated signal obtained from the first phase, or on the virtually concatenated signal obtained from the second phase, or on these signals combined with some transparency. For RSVP-TE, the SONET/SDH traffic parameters are carried in a new SENDER_TSPEC and FLOWSPEC. The same format is used for both. There is no Adspec associated with the SENDER_TSPEC, either it is omitted or a default value is used. The content of the FLOWSPEC object received in a Resv message should be identical to the content of the SENDER_TSPEC of the corresponding Path message. In other words, the receiver is normally not allowed to change the values of the traffic parameters. However some level of negotiation may be achieved as explained in [SONETSDH-GMPLS] For CR-LDP, the SONET/SDH traffic parameters are simply carried in a new TLV. Note that a general discussion on SDH/SONET and GMPLS can be found in [SDH/SONET-GMPLS-FRAMEWORK]. 9.4. G.709 Traffic Parameters Simply said, an ITU-T G.709 based network is decomposed in two major layers: an optical layer (i.e. made of wavelengths) and a digital layer. These two layers are divided into sub-layers and switching occurs at two specific sub-layers: at the OCh (Optical Channel) E. Mannie et. al. Internet-Draft February 2003 31 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 optical layer and at the ODU (Optical channel Data Unit) electrical layer. The ODUk notation is used to denotes ODUs at different bandwidths. The GMPLS G.709 traffic parameters [G709-GMPLS] specify a powerful set of capabilities for ITU-T G.709 networks. The first traffic parameter specifies the type of the elementary G.709 signal that comprises the requested LSP, e.g. ODU1, OCh at 40 Gbps, etc. Several transforms can then be applied successively on the elementary Signal to build the final signal being actually requested for the LSP. These transforms are the virtual concatenation and the multiplication. Each one of these transforms is optional. They must be applied strictly in the following order: - First, virtual concatenation can be optionally applied directly on the elementary Signal, - Second, a multiplication can be optionally applied, either directly on the elementary Signal, or on the virtually concatenated signal obtained from the first phase. Additional ODUk Multiplexing traffic parameters allow indicating an ODUk mapping (ODUj into ODUk) for an ODUk multiplexing LSP request. G.709 supports the following multiplexing capabilities: ODUj into ODUk (k > j); and ODU1 with ODU2 multiplexing into ODU3. For RSVP-TE, the SONET/SDH traffic parameters are carried in a new SENDER-TSPEC and FLOWSPEC. The same format is used for both. There is no Adspec associated with the SENDER_TSPEC, either it is omitted or a default value is used. The content of the FLOWSPEC object received in a Resv message should be identical to the content of the SENDER_TSPEC of the corresponding Path message. For CR-LDP, the SONET/SDH traffic parameters are simply carried in a new TLV. 9.5. Bandwidth Encoding Some technologies that do not have (yet) specific traffic parameters just require a bandwidth encoding transported in a generic form. Bandwidth is carried in 32-bit number in IEEE floating-point format (the unit is bytes per second). Values are carried in a per protocol specific manner. For non-packet LSPs, it is useful to define discrete values to identify the bandwidth of the LSP. It should be noted that this bandwidth encoding do not apply to SONET/SDH and G.709, for which the traffic parameters fully define the requested SONET/SDH or G.709 signal. The bandwidth is coded in the Peak Data Rate field of Int-Serv objects for RSVP-TE in the SENDER_TSPEC and FLOWSPEC objects; and in E. Mannie et. al. Internet-Draft February 2003 32 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 the Peak and Committed Data Rate fields of the CR-LDP Traffic Parameters TLV. 9.6. Generalized Label The Generalized Label extends the traditional MPLS label by allowing the representation of not only labels that travel in-band with associated data packets, but also (virtual) labels that identify time-slots, wavelengths, or space division multiplexed positions. For example, the Generalized Label may identify (a) a single fiber in a bundle, (b) a single waveband within fiber, (c) a single wavelength within a waveband (or fiber), or (d) a set of time-slots within a wavelength (or fiber). It may also be a generic MPLS label, a Frame Relay label, or an ATM label (VCI/VPI). The format of a label can be as simple as an integer value such as a wavelength label or can be more elaborated such as an SDH/SONET or a G.709 label. SDH and SONET define each a multiplexing structure. These multiplexing structures will be used as naming trees to create unique labels. Such a label will identify the exact position (times- lot(s)) of a signal in a multiplexing structure. Since the SONET multiplexing structure may be seen as a subset of the SDH multiplexing structure, the same format of label is used for SDH and SONET. A similar concept is applied to build a label at the G.709 ODU layer. Since the nodes sending and receiving the Generalized Label know what kinds of link they are using, the Generalized Label does not identify its type, instead the nodes are expected to know from the context what type of label to expect. A Generalized Label only carries a single level of label, i.e. it is non-hierarchical. When multiple levels of labels (LSPs within LSPs) are required, each LSP must be established separately. 9.7. Waveband Switching A special case of wavelength switching is waveband switching. A waveband represents a set of contiguous wavelengths, which can be switched together to a new waveband. For optimization reasons it may be desirable for a photonic cross-connect to optically switch multiple wavelengths as a unit. This may reduce the distortion on the individual wavelengths and may allow tighter separation of the individual wavelengths. A Waveband label is defined to support this special case. Waveband switching naturally introduces another level of label hierarchy and as such the waveband is treated the same way all other upper layer labels are treated. As far as the MPLS protocols are concerned there is little difference between a waveband label and a wavelength label except that semantically the waveband can be E. Mannie et. al. Internet-Draft February 2003 33 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 subdivided into wavelengths whereas the wavelength can only be subdivided into time or statistically multiplexed labels. In the context of waveband switching, the generalized label used to indicate a waveband contains three fields, a waveband ID, a Start Label and an End Label. The Start and End Labels are channel identifiers from the sender perspective that identify respectively, the lowest value wavelength and the highest value wavelength making up the waveband. 9.8. Label Suggestion by the Upstream GMPLS allows for a label to be optionally suggested by an upstream node. This suggestion may be overridden by a downstream node but in some cases, at the cost of higher LSP setup time. The suggested label is valuable when establishing LSPs through certain kinds of optical equipment where there may be a lengthy (in electrical terms) delay in configuring the switching fabric. For example micro mirrors may have to be elevated or moved, and this physical motion and subsequent damping takes time. If the labels and hence switching fabric are configured in the reverse direction (the norm) the MAPPING/Resv message may need to be delayed by 10's of milliseconds per hop in order to establish a usable forwarding path. It can be important for restoration purposes where alternate LSPs may need to be rapidly established as a result of network failures. 9.9. Label Restriction by the Upstream An upstream node can optionally restrict (limit) the choice of label of a downstream node to a set of acceptable labels. Giving lists and/or ranges of inclusive (acceptable) or exclusive (unacceptable) labels in a Label Set provides this restriction. If not applied, all labels from the valid label range may be used. There are at least four cases where a label restriction is useful in the "optical" domain. 1. The first case is where the end equipment is only capable of transmitting and receiving on a small specific set of wavelengths/bands. 2. The second case is where there is a sequence of interfaces, which cannot support wavelength conversion and require the same wavelength be used end-to-end over a sequence of hops, or even an entire path. 3. The third case is where it is desirable to limit the amount of wavelength conversion being performed to reduce the distortion on the optical signals. 4. The last case is where two ends of a link support different sets of wavelengths. The receiver of a Label Set must restrict its choice of labels to one that is in the Label Set. A Label Set may be present across multiple hops. In this case each node generates it's own outgoing E. Mannie et. al. Internet-Draft February 2003 34 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 Label Set, possibly based on the incoming Label Set and the node's hardware capabilities. This case is expected to be the norm for nodes with conversion incapable interfaces. 9.10. Bi-directional LSP GMPLS allows establishment of bi-directional symmetric LSPs (not of asymmetric LSPs). A symmetric bi-directional LSP has the same traffic engineering requirements including fate sharing, protection and restoration, LSRs, and resource requirements (e.g. latency and jitter) in each direction. In the remainder of this section, the term "initiator" is used to refer to a node that starts the establishment of an LSP and the term "terminator" is used to refer to the node that is the target of the LSP. For a bi-directional LSPs, there is only one initiator and one terminator. Normally to establish a bi-directional LSP when using [RSVP-TE] or [CR-LDP] two unidirectional paths must be independently established. This approach has the following disadvantages: 1. The latency to establish the bi-directional LSP is equal to one round trip signaling time plus one initiator-terminator signaling transit delay. This not only extends the setup latency for successful LSP establishment, but it extends the worst-case latency for discovering an unsuccessful LSP to as much as two times the initiator-terminator transit delay. These delays are particularly significant for LSPs that are established for restoration purposes. 2. The control overhead is twice that of a unidirectional LSP. This is because separate control messages (e.g. Path and Resv) must be generated for both segments of the bi-directional LSP. 3. Because the resources are established in separate segments, route selection is complicated. There is also additional potential race for conditions in assignment of resources, which decreases the overall probability of successfully establishing the bi-directional connection. 4. It is more difficult to provide a clean interface for SDH/SONET equipment that may rely on bi-directional hop-by-hop paths for protection switching. Note that existing SDH/SONET gear transmits the control information in-band with the data. 5. Bi-directional optical LSPs (or lightpaths) are seen as a requirement for many optical networking service providers. With bi-directional LSPs both the downstream and upstream data paths, i.e. from initiator to terminator and terminator to initiator, are established using a single set of signaling messages. This reduces the setup latency to essentially one initiator- terminator round trip time plus processing time, and limits the E. Mannie et. al. Internet-Draft February 2003 35 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 control overhead to the same number of messages as a unidirectional LSP. For bi-directional LSPs, two labels must be allocated. Bi- directional LSP setup is indicated by the presence of an Upstream Label in the appropriate signaling message. 9.11. Bi-directional LSP Contention Resolution Contention for labels may occur between two bi-directional LSP setup requests traveling in opposite directions. This contention occurs when both sides allocate the same resources (ports) at effectively the same time. The GMPLS signaling defines a procedure to resolve that contention, basically the node with the higher node ID will win the contention. To reduce the probability of contention, some mechanisms are also suggested. 9.12. Rapid Notification of Failure GMPLS defines several signaling extensions that enable expedited notification of failures and other events to nodes responsible for restoring failed LSPs, and error handling. 1. Acceptable Label Set for notification on Label Error: There are cases in traditional MPLS and in GMPLS that result in an error message containing an "Unacceptable label value" indication. When these cases occur, it can useful for the node generating the error message to indicate which labels would be acceptable. To cover this case, GMPLS introduces the ability to convey such information via the "Acceptable Label Set". An Acceptable Label Set is carried in appropriate protocol specific error messages. The format of an Acceptable Label Set is identical to a Label Set. 2. Expedited notification: Extensions to RSVP-TE enable expedited notification of failures and other events to determined nodes. For CR-LDP there is not currently a similar mechanism. The first extension identifies where event notifications are to be sent. The second provides for general expedited event notification with a Notify message. Such extensions can be used by fast restoration mechanisms. Notifications may be requested in both the upstream and downstream directions. The Notify message differs from the currently defined error messages in that it can be "targeted" to a node other than the immediate upstream or downstream neighbor and that it is a generalized notification mechanism. The Notify message does not replace existing error messages. The Notify message may be sent either (a) normally, where non-target nodes just forward the Notify message to the target node, similar to ResvConf processing in [RSVP]; or (b) encapsulated in a new IP header whose destination is equal to the target IP address. E. Mannie et. al. Internet-Draft February 2003 36 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 3. Faster removal of intermediate states: A specific RSVP optimization allowing in some cases the faster removal of intermediate states. This extension is used to deal with specific RSVP mechanisms. 9.13. Link Protection Protection information is carried in the new optional Protection Information Object/TLV. It currently indicates the desired link protection for each link of an LSP. If a particular protection type, i.e., 1+1, or 1:N, is requested, then a connection request is processed only if the desired protection type can be honored. Note that GMPLS advertises the protection capabilities of a link in the routing protocols. Path computation algorithms may take this information into account when computing paths for setting up LSPs. Protection information also indicates if the LSP is a primary or secondary LSP. A secondary LSP is a backup to a primary LSP. The resources of a secondary LSP are normally not used until the primary LSP fails, but they may be used by other LSPs until the primary LSP fails over the secondary LSP. At that point, any LSP that is using the resources for the secondary LSP must be preempted. Six link protection types are currently defined as individual flags and can be combined: enhanced, dedicated 1+1, dedicated 1:1, shared, unprotected, extra traffic. See [GMPLS-SIG] section 7.1 for a precise definition of each. 9.14. Explicit Routing and Explicit Label Control By using an explicit route the path taken by an LSP can be controlled more or less precisely. Typically, the node at the head- end of an LSP finds an explicit route and builds an Explicit Route Object (ERO)/ Explicit Route (ER) TLV that contains that route. Possibly, the edge node does not build any explicit route, and just transmit a signaling request to a default neighbor LSR (as IP/MPLS hosts would). For instance, an explicit route could be added to a signaling message by the first switching node, on behalf of the edge node. Note also that an explicit route is altered by intermediate LSRs during its progression towards the destination. The explicit route is originally defined by MPLS-TE as a list of abstract nodes (i.e. groups of nodes) along the explicit route. Each abstract node can be an IPv4 address prefix, an IPv6 address prefix, or an AS number. This capability allows the generator of the explicit route to have incomplete information about the details of the path. In the simplest case, an abstract node can be a full IP address (32 bits) that identifies a specific node (called a simple abstract node). MPLS-TE allows strict and loose abstract nodes. The path between a strict node and its preceding node must include only network nodes from the strict node and its preceding abstract node. The path E. Mannie et. al. Internet-Draft February 2003 37 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 between a loose node and its preceding abstract node may include other network nodes that are not part of the loose node or its preceding abstract node. This explicit route was extended to include interface numbers as abstract nodes to support unnumbered interfaces; and further extended by GMPLS to include labels as abstract nodes. Having labels in an explicit route is an important feature that allows controlling the placement of an LSP with a very fine granularity. This is more likely to be used for TDM, LSC and FSC links. In particular, the explicit label control in the explicit route allows terminating an LSP on a particular outgoing port of an egress node. Indeed, a label sub-object/TLV must follow a sub-object/TLV containing the IP address, or the interface identifier (in case of unnumbered interface), associated with the link on which it is to be used. This can also be used when it is desirable to "splice" two LSPs together, i.e. where the tail of the first LSP would be "spliced" into the head of the second LSP. Another use is when an optimization algorithm is used for an SDH/SONET network. This algorithm can provide very detailed explicit routes, including the label (time-slot) to use on a link, in order to minimize the fragmentation of the SDH/SONET multiplex on the corresponding interface. 9.15. Route recording In order to improve the reliability and the manageability of the LSP being established, the concept of the route recording was introduced in RSVP-TE to function as: - First, a loop detection mechanism to discover L3 routing loops, or loops inherent in the explicit route (this mechanism is strictly exclusive with the use of explicit routing objects). - Second, a route recording mechanism collects up-to-date detailed path information on a hop-by-hop basis during the LSP setup process. This mechanism provides valuable information to the source and destination nodes. Any intermediate routing change at setup time, in case of loose explicit routing, will be reported. - Third, a recorded route can be used as input for an explicit route. This is useful if a source node receives the recorded route from a destination node and applies it as an explicit route in order to "pin down the path". Within the GMPLS architecture only the second and third functions are mainly applicable for TDM, LSC and FSC layers. 9.16. LSP modification and LSP re-routing E. Mannie et. al. Internet-Draft February 2003 38 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 LSP modification and re-routing are two features already available in MPLS-TE. GMPLS does not add anything new. Elegant re-routing is possible with the concept of "make-before-break" whereby an old path is still used while a new path is set up by avoiding double reservation of resources. Then, the node performing the re-routing can swap on the new path and close the old path. This feature is supported with RSVP-TE (using shared explicit filters) and CR-LDP (using the action indicator flag). LSP modification consists in changing some LSP parameters, but normally without changing the route. It is supported using the same mechanism as re-routing. However, the semantic of LSP modification will differ from one technology to the other. For instance, further studies are required to understand the impact of dynamically changing some SDH/SONET circuit characteristics such as the bandwidth, the protection type, the transparency, the concatenation, etc. 9.17. LSP administrative status handling GMPLS provides the optional capability to indicate the administrative status of an LSP by using a new Admin Status object/TLV. Administrative Status Information is currently used in two ways. In the first usage, Admin Status the object/TLV is carried in a Path/Label Request or Resv/Label Mapping message to indicate the administrative state of an LSP. In this usage, Administrative Status Information indicates the state of the LSP, which include "up" or "down", if it in a "testing" mode, and if deletion is in progress. Based on that administrative status, a node can take local decisions, like to inhibit alarm reporting when an LSP is in "down" or "testing" states, or to report alarms associated with the connection at a priority equal to or less than "Non service affecting". It is possible that some nodes along an LSP will not support the Admin Status Object/TLV. In the case of a non-supporting transit node, the object will pass through the node unmodified and normal processing can continue. In some circumstances, particularly optical networks, it is useful to set the administrative status of an LSP to "being deleted" before tearing it down in order to avoid non-useful generation of alarms. The ingress LSR precedes an LSP deletion by inserting an appropriate Admin Status Object/TLV in a Path/Label Request (with the modification action indicator flag set to modify) message. Transit LSRs process the Admin Status Object/TLV and forward it. The egress LSR answers in a Resv/Label Mapping (with the modification action indicator flag set to modify) message with the Admin Status object. Upon receiving this message and object, the ingress node sends a PathTear/Release message downstream to remove the LSP and normal RSVP/CR-LDP processing takes place. E. Mannie et. al. Internet-Draft February 2003 39 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 In the second usage, the Admin Status object/TLV is carried in a Notification/Label Mapping (with the modification action indicator flag set to modify) message to request that the ingress node change the administrative state of an LSP. This allows intermediate and egress nodes to trigger the setting of administrative status. In particular this allows, intermediate or egress LSRs to request a release of an LSP initiated by the ingress node. 9.18. Control channel separation In GMPLS, a control channel can be separated from the data channel. Indeed, the control channel can be implemented completely out-of- band for various reasons, e.g. when the data channel cannot carry in-band control information. This issue was even originally introduced to MPLS in connection with link bundling. In traditional MPLS there is an implicit one-to-one association of a control channel to a data channel. When such an association is present, no additional or special information is required to associate a particular LSP setup transaction with a particular data channel. Otherwise it is necessary to convey additional information in signaling to identify the particular data channel being controlled. GMPLS supports explicit data channel identification by providing interface identification information. GMPLS allows the use of a number of interface identification schemes including IPv4 or IPv6 addresses, interface indexes (for unnumbered interfaces) and component interfaces (for bundled interfaces), unnumbered bunled interfaces are also supported. The choice of the data interface to use is always made by the sender of the Path/Label Request message, and indicated by including the data channel's interface identifier in the message using a new RSVP_HOP object sub-type/Interface TLV. For bi-directional LSPs, the sender chooses the data interface in each direction. In all cases but bundling, the upstream interface is implied by the downstream interface. For bundling, the Path/Label Request sender explicitly identifies the component interface used in each direction. The new object/TLV is used in Resv/Label Mapping message to indicate the downstream node's usage of the indicated interface(s). The new object/TLV can contain a list of embedded TLVs, each embedded TLV can be an IPv4 address, and IPv6 address, an interface index, a downstream component interface ID or an upstream component interface ID. In the last three cases, the embedded TLV contains itself an IP address plus an Interface ID, the IP address being used to identify the interface ID (it can be the router ID for instance). E. Mannie et. al. Internet-Draft February 2003 40 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 There are cases where it is useful to indicate a specific interface associated with an error. To support these cases the IF_ID ERROR_SPEC RSVP Objects are defined. 10. Forwarding Adjacencies (FA) To improve scalability of MPLS TE (and thus GMPLS) it may be useful to aggregate multiple TE LSPs inside a bigger TE LSP. Intermediate nodes see the external LSP only, they don't have to maintain forwarding states for each internal LSP, less signaling messages need to be exchanged and the external LSP can be somehow protected instead (or in addition) to the internal LSPs. This can considerably increase the scalability of the signaling. The aggregation is accomplished by (a) an LSR creating a TE LSP, (b) the LSR forming a forwarding adjacency out of that LSP (advertising this LSP as a Traffic Engineering (TE) link into ISIS/OSPF), (c) allowing other LSRs to use forwarding adjacencies for their path computation, and (d) nesting of LSPs originated by other LSRs into that LSP (e.g. by using the label stack construct in the case of IP). An LSR may (under its local configuration control) announce an LSP as a TE link into ISIS/OSPF. When this link is advertised into the same instance of ISIS/OSPF as the one that determines the route taken by the LSP, we call such a link a "forwarding adjacency" (FA). We refer to the LSP as the "forwarding adjacency LSP", or just FA- LSP. Note that since the advertised entity is a link in ISIS/OSPF, both the endpoint LSRs of the FA-LSP must belong to the same ISIS level/OSPF area (intra-area improvement of scalability). In general, creation/termination of a FA and its FA-LSP could be driven either by mechanisms outside of MPLS (e.g., via configuration control on the LSR at the head-end of the adjacency), or by mechanisms within MPLS (e.g., as a result of the LSR at the head-end of the adjacency receiving LSP setup requests originated by some other LSRs). ISIS/OSPF floods the information about FAs just as it floods the information about any other links. As a result of this flooding, an LSR has in its TE link state database the information about not just conventional links, but FAs as well. An LSR, when performing path computation, uses not just conventional links, but FAs as well. Once a path is computed, the LSR uses RSVP- TE/CR-LDP for establishing label binding along the path. FAs need simple extensions to signaling and routing protocols. 10.1. Routing and Forwarding Adjacencies Forwarding adjacencies may be represented as either unnumbered or numbered links. A FA can also be a bundle of LSPs between two nodes. E. Mannie et. al. Internet-Draft February 2003 41 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 FAs are advertised as GMPLS TE links such as defined in [HIERARCHY]. GMPLS TE links are advertised in OSPF and ISIS such as defined in [OSPF-TE-GMPLS] and [ISIS-TE-GMPLS]. These last two specifications enhance [OSPF-TE] and [ISIS-TE] that defines a base TE link. When a FA is created dynamically, its TE attributes are inherited from the FA-LSP that induced its creation. [HIERARCHY] specifies how each TE parameter of the FA is inherited from the FA-LSP. Note that the bandwidth of the FA must be at least as big as the FA-LSP that induced it, but may be bigger if only discrete bandwidths are available for the FA-LSP. In general, for dynamically provisioned forwarding adjacencies, a policy-based mechanism may be needed to associate attributes to forwarding adjacencies. A FA advertisement could contain the information about the path taken by the FA-LSP associated with that FA. Other LSRs may use this information for path computation. This information is carried in a new OSPF and IS-IS TLV called the Path TLV. It is possible that the underlying path information might change over time, via configuration updates, or dynamic route modifications, resulting in the change of that TLV. If forwarding adjacencies are bundled (via link bundling), and if the resulting bundled link carries a Path TLV, the underlying path followed by each of the FA-LSPs that form the component links must be the same. It is expected that forwarding adjacencies will not be used for establishing ISIS/OSPF peering relation between the routers at the ends of the adjacency. LSP hierarchy could exist both with the peer and with the overlay models. With the peer model the LSP hierarchy is realized via FAs and an LSP is both created and used as a TE link by exactly the same instance of the control plane. Creating LSP hierarchy with overlays doesn't involve the concept of FA. With the overlay model an LSP created (and maintained) by one instance of the GMPLS control plane is used as a TE link by another instance of the GMPLS control plane. Moreover, the nodes using a TE link are expected to have a routing and signaling adjacency. 10.2. Signaling aspects For the purpose of processing the explicit route in a Path/Request message of an LSP that is to be tunneled over a forwarding adjacency, an LSR at the head-end of the FA-LSP views the LSR at the tail of that FA-LSP as adjacent (one IP hop away). 10.3. Cascading of Forwarding Adjacencies With an integrated model several layers are controlled using the same routing and signaling protocols. A network may then have links with different multiplexing/demultiplexing capabilities. For E. Mannie et. al. Internet-Draft February 2003 42 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 example, a node may be able to multiplex/demultiplex individual packets on a given link, and may be able to multiplex/demultiplex channels within a SONET payload on other links. A new OSPF and IS-IS sub-TLV has been defined to advertise the multiplexing capability of each interface: PSC, L2SC, TDM, LSC or FSC. This sub-TLV is named the Link Multiplex Capability sub-TLV and complements the sub-TLVs defined in [OSPF-TE-GMPLS] and [ISIS-TE- GMPLS]. The information carried in this sub-TLV is used to construct LSP regions, and determine regionÆs boundaries. Path computation may take into account region boundaries when computing a path for an LSP. For example, path computation may restrict the path taken by an LSP to only the links whose multiplexing/demultiplexing capability is PSC. When an LSP need to cross a region boundary, it can trigger the establishment of an FA at the underlying layer (i.e. the L2SC layer). This can trigger a cascading of FAs between layers with the following obvious order: L2SC, then TDM, then LSC, and then finally FSC. 11. Routing and Signaling Adjacencies By definition, two nodes have a routing (ISIS/OSPF) adjacency if they are neighbors in the ISIS/OSPF sense. By definition, two nodes have a signaling (RSVP-TE/CR-LDP) adjacency if they are neighbors in the RSVP-TE/CR-LDP sense. Nodes A and B are RSVP-TE neighbors if they directly exchange RSVP-TE messages (Path/Resv) (e.g., as described in sections 7.1.1 and 7.1.2 of [HIERARCHY]). The neighbor relationship includes exchanging RSVP-TE Hellos. By definition, a Forwarding Adjacency (FA) is a TE Link between two GMPLS nodes whose path transits one or more other (G)MPLS nodes in the same instance of the (G)MPLS control plane. If two nodes have one or more non-FA TE Links between them, these two nodes are expected (although not required) to have a routing adjacency. If two nodes do not have any non-FA TE Links between them, it is expected (although not required) that these two nodes would not have a routing adjacency. To state the obvious, if the TE links between two nodes are to be used for establishing LSPs, the two nodes must have a signaling adjacency. If one wants to establish routing and/or signaling adjacency between two nodes, there must be an IP path between them. This IP path can be, for example, a TE Link with an interface switching capability of PSC, anything that looks likes an IP link (e.g., GRE tunnel, or a (bi-directional) LSP that with an interface switching capability of PSC). A TE link may not be capable of being used directly for maintaining routing and/or signaling adjacencies. This is because GMPLS routing and signaling adjacencies requires exchanging data on a per (IP/ISO) packet basis, and a TE link (e.g. a link between OXCs) may not be E. Mannie et. al. Internet-Draft February 2003 43 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 capable of exchanging data on a per packet basis. In this case the routing and signaling adjacencies are maintained via a set of one or more control channels (see [LMP]). Two nodes may have a TE link between them even if they don't have a routing adjacency. Naturally, each node must run OSPF/ISIS with GMPLS extensions in order for that TE link to be advertised. More precisely, the node needs to run GMPLS extensions for TE Links with an interface switching capability (see [GMPLS-ROUTING]) other than PSC, and it needs to run either GMPLS or MPLS extensions for TE links with an interface switching capability of PSC. The mechanisms for Control Channel Separation [GMPLS-SIG] should be used (even if the IP path between two nodes is a TE link). I.e., RSVP-TE/CR-LDP signaling should use the Interface_ID (IF_ID) object to specify a particular TE link when establishing an LSP. The IP path could consist of multiple IP hops. In this case, the mechanisms of sections 7.1.1 and 7.1.2 of [HIERARCHY] should be used (in addition to Control Channel Separation). 12. Control Plane Fault Handling There are two major types of faults that can impact a control plane. The first, referred to as control channel fault, relates to the case where control communication is lost between two neighboring nodes. If the control channel is embedded with the data channel, data channel recovery procedure should solve the problem. If the control channel is independent of the data channel additional procedures are required to recover from that problem. The second, referred to as nodal faults, relates to the case where a node losses its control state (e.g., after a restart) but does not loose its data forwarding state. In transport networks, such types of control plane faults should not have service impact on the existing connections. Under such circumstances, a mechanism must exist to detect a control communication failure and a recovery procedure must guarantee connection integrity at both ends of the control channel. For a control channel fault, once communication is restored routing protocols are naturally able to recover but the underlying signaling protocols must indicate that the nodes have maintained their state through the failure. The signaling protocol must also ensure that any state changes that were instantiated during the failure are synchronized between the nodes. For a nodal fault, a node's control plane restarts and losses most of it's state information. In this case, both upstream and downstream nodes must synchronize their state information with the restarted node. In order for any resynchronization to occur the node undergoing the restart will need to preserve some information, such as it's mappings of incoming to outgoing labels. E. Mannie et. al. Internet-Draft February 2003 44 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 These issues are addressed in protocol specific fashions, see [RSVP- TE-GMPLS] and [CR-LDP-GMPLS]. Note that these cases only apply when there are mechanisms to detect data channel failures independent of control channel failures. The LDP Fault tolerant draft [LDP-FT] specifies the procedures to recover from a control channel failure. [RSVP-TE-GMPLS] specifies how to recover from both a control channel failure and a node failure. 13. LSP Protection and Restoration This section discusses Protection and Restoration (P&R) issues for GMPLS LSPs. It is driven by the requirements outlined in [TEWG- RESTORE] and some of the principles outlined in [MPLS-RECOVERY]. It will be enhanced, as more GMPLS P&R mechanisms are defined. The scope of this section is clarified hereafter: - This section is only applicable when a fault impacting LSP(s) happens in the data/transport plane. Section 11 deals with control plane fault handling for nodal and control channel faults. - This section focuses on P&R at the TDM, LSC and FSC layers. There are specific P&R requirements at these layers not present at the PSC layer. - This section focuses on intra-area P&R as opposed to inter-area P&R and even inter-domain P&R. Note that the P&R can even be more restricted, e.g. to a collection of like customer equipment, or a collection of equipment of like capabilities, in one single routing area. - This section focuses on intra-layer P&R (horizontal hierarchy as defined in [TEWG-RESTORE]) as opposed to the inter-layer P&R (vertical hierarchy). - P&R mechanisms are in general designed to handle single failures, which makes SRLG diversity a necessity. Recovery from multiple failures requires further study. - Both mesh and ring like topologies are supported. In the following, we assume that: - TDM, LSC and FSC devices are more generally committing recovery resources in a non best effort way. Recovery resources are either allocated and used, or at least logically reserved (used or not by preemptable extra traffic but unavailable anyway for regular working traffic). - Shared P&R mechanisms are valuable to operators in order to maximize their network utilization. E. Mannie et. al. Internet-Draft February 2003 45 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 - Sending preemptable excess traffic on recovery resources is a valuable feature for operators. 13.1. Protection escalation across domains and layers To describe the P&R architecture, one must consider two dimensions of hierarchy [TE-RESTORE]: - A horizontal hierarchy consisting of multiple P&R domains, which is important in an LSP based protection scheme. The scope of P&R may extend over a link (or span), an administrative domain or sub- network, an entire LSP. An administrative domain may consist of a single P&R domain or as a concatenation of several smaller P&R domains. The operator can configure P&R domains, based on customers' requirements, and on network topology and traffic engineering constraints. - A vertical hierarchy consisting of multiple layers of P&R with varying granularities (packet flows, STS trails, lightpaths, fibers, etc). In the absence of adequate P&R coordination, a fault may propagate from one level to the next within a P&R hierarchy. It can lead to "collisions" and simultaneous recovery actions may lead to race conditions, reduced resource utilization, or instabilities [MANCHESTER]. Thus, a consistent escalation strategy is needed to coordinate recovery across domains and layers. The fact that GMPLS can be used at different layers could simplify this coordination. There are two types of escalation strategies: bottom-up and top- down. The bottom-up approach assumes that "lower-level" recovery schemes are more expedient, and therefore we can inhibit or hold off higher-level P&R. The Top-down approach attempts service P&R at the higher levels before invoking "lower level" P&R. Higher- layer P&R is service selective, and permits "per-CoS" or "per-LSP" re-routing. SLA's between network operators and their clients are needed to determine the necessary timescales for P&R at each layer and at each domain. 13.2. Mapping of Services to P&R Resources The choice of a P&R scheme is a tradeoff between network utilization (cost) and service interruption time. In light of this tradeoff, network service providers are expected to support a range of different service offerings or service levels. One can classify LSPs into one of a small set of service levels. Among other things, these service levels define the reliability characteristics of the LSP. The service level associated with a given LSP is mapped to one or more P&R schemes during LSP establishment. An advantage that mapping is that an LSP may use E. Mannie et. al. Internet-Draft February 2003 46 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 different P&E schemes in different segments of a network (e.g. some links may be span protected, whilst other segments of the LSP may utilize ring protection). These details are likely to be service provider specific. An alternative to using service levels is for an application to specify the set of specific P&R mechanisms to be used when establishing the LSP. This allows greater flexibility in using different mechanisms to meet the application requirements. A differentiator between these service levels is service interruption time in the event of network failures, which is defined as the length of time between when a failure occurs and when connectivity is re-established. The choice of service level (or P&R scheme) should be dictated by the service requirements of different applications. 13.3. Classification of P&R mechanism characteristics The following figure provides a classification of the possible provisioning types of recovery LSPs, and of the levels of overbooking that is possible for them. + Computed on +Established +--Resources pre | demand | on demand | allocated Recovery LSP | | | Provisioning -+ Pre computed +Pre established +--Resources allocated on demand +--Dedicated - 1:1, 1 | | +-- Shared - 1:N, Ring, Shared mesh Level of | Overbooking ---+-- Best effort 13.4. Different Stages in P&R Recovery from a network fault or impairment takes place in several stages as discussed in [MPLS-RECOVERY], including fault detection, fault localization, notification, recovery (i.e. the P&R itself) and restoral (i.e. returning the traffic to the original working LSP or to a new one) of traffic. - Fault detection is technology and implementation dependent. In general, failures are detected by lower layer mechanisms (e.g. SONET/SDH, Loss-of-Light (LOL)). When a node detects a failure, an alarm may be passed up to a GMPLS entity, which will take appropriate actions, or the alarm may be propagated at the lower layer (e.g. SONET/SDH AIS). - Fault localization can be done with the help of GMPLS, e.g. using LMP for fault localization (see section 8.4). E. Mannie et. al. Internet-Draft February 2003 47 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 - Fault notification can also be achieved through GMPLS, e.g. using GMPLS RSVP-TE/CR-LDP notification (see section 9.12). - This section focuses on the different mechanisms available for recovery and restoral of traffic once fault detection, localization and notification have taken place. 13.5. Recovery Strategies Network P&R techniques can be divided into Protection and Restoration. In protection, resources between the protection endpoints are established before failure, and connectivity after failure is achieved simply by switching performed at the protection end-points. In contrast, restoration uses signaling after failure to allocate resources along the recovery path. - Protection aims at extremely fast reaction times and may rely on the use of overhead control fields for achieving end-point coordination. Protection for SONET/SDH networks is described in [G.841] and [ANSI-T1.105]. Protection mechanisms can be further classified by the level of redundancy and sharing. - Restoration mechanisms rely on signaling protocols to coordinate switching actions during recovery, and may involve simple re- provisioning, i.e. signaling only at the time of recovery; or pre- signaling, i.e., signaling prior to recovery. In addition, P&R can be applied on a local or end-to-end basis. In the local approach, P&R is focused on the local proximity of the fault in order to reduce delay in restoring service. In the end-to- end approach, the LSP originating and terminating nodes control recovery. Using these strategies, the following recovery mechanisms can be defined. Editor's note: for each mechanism hereafter, it is intended to add references to the appropriate GMPLS and/or technology specific mechanisms. 13.6. Recovery mechanisms: Protection schemes Note that protection schemes are usually defined in technology specific ways, but this does not preclude other solutions. - 1+1 Link Protection: Two pre-provisioned resources are used in parallel. For example, data is transmitted simultaneously on two parallel links and a selector is used at the receiving node to choose the best source. [Mechanisms reference to be added]. - 1:N Link Protection: Working and protecting resources (N working, 1 backup) are pre-provisioned. If a working resource fails, the data is switched to the protecting resource, using a coordination mechanism (e.g. in overhead bytes). More generally, N working and E. Mannie et. al. Internet-Draft February 2003 48 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 M protecting resources can be assigned for M:N link protection. [Mechanisms reference to be added]. - Enhanced Protection: Various mechanisms such as protection rings can be used to enhance the level of protection beyond single link failures to include the ability to switch around a node failure or multiple link failures within a span, based on a pre-established topology of protection resources. [Mechanisms reference to be added]. - 1+1 LSP Protection: Simultaneous data transmission on working and protecting LSPs and tail-end selection can be applied. [Mechanisms reference to be added]. 13.7. Recovery mechanisms: Restoration schemes Restoration is possible thanks to the use of a distributed control plane like GMPLS in multiple of tenths of ms. It is much harder to achieve when only an NMS is used, and can only be done in that case in a multiple of seconds. - End-to-end LSP restoration with re-provisioning: An end-to-end restoration path is established after failure. The restoration path may be dynamically calculated after failure, or pre- calculated before failure (often during LSP establishment). Importantly, no signaling is used along the restoration path before failure, and no restoration bandwidth is reserved. As a result, there is no guarantee that a given restoration path is available when a failure occurs. Thus one may have to crankback to search for an available path. [Mechanisms reference to be added]. - End-to-end LSP restoration with pre-signaled recovery bandwidth reservation and no label pre-selection: An end-to-end restoration path is pre-calculated before failure and a signaling message is sent along this pre-selected path to reserve bandwidth, but labels are not selected. [Mechanisms reference to be added]. The resources reserved on each link of a restoration path may be shared across different working LSPs that are not expected to fail simultaneously. Local node policies can be applied to define the degree to which capacity is shared across independent failures. Upon failure detection, LSP signaling is initiated along the restoration path to select labels, and to initiate the appropriate cross-connections. [Mechanisms reference to be added]. - End-to-end LSP restoration with pre-signaled recovery bandwidth reservation and label pre-selection: An end-to-end restoration path is pre-calculated before failure and a signaling procedure is initiated along this pre-selected path on which bandwidth is reserved and labels are selected. Resources reserved on each link may be shared across different working LSPs that are not expected to fail simultaneously. In networks based on TDM, LSC and FSC technology, LSP signaling is used after failure detection to establish cross-connections at the intermediate switches on the E. Mannie et. al. Internet-Draft February 2003 49 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 restoration path using the pre-selected labels. [Mechanisms reference to be added]. - Local LSP restoration: the above approaches can be applied on a local basis rather than end-to-end, in order to reduce recovery time. [Mechanisms reference to be added]. 13.8. Schema selection criteria This section discusses criteria that could be used by the operator in order to make a choice among the various P&R mechanisms. - Robustness: In general, the less pre-planning of the restoration path, the more robust the restoration scheme is to a variety of failures, provided that adequate resources are available. Restoration schemes with pre-planned paths, will not be able to recover from network failures that simultaneously affect both the working and restoration paths. Thus, these paths should ideally be chosen to be as disjoint as possible (i.e. SRLG and node disjoint), so that any single failure event will not affect both paths. The risk of simultaneous failure of the two paths can be reduced by recalculating the restoration path whenever a failure occurs along it. The pre-selection of a label gives less flexiblity for multiple failure scenarios than no label pre-selection. If failures occur that affect two LSPs that are sharing a label at a common node along their restoration routes, then only one of these LSPs can be recovered, unless the label assignment is changed. The robustness of a restoration scheme is also determined by the amount of reserved restoration bandwidth - as the amount of restoration bandwidth sharing increases (reserved bandwidth decreases), the restoration scheme becomes less robust to failures. Restoration schemes with pre-signaled bandwidth reservation (with or without label pre-selection) can reserve adequate bandwidth to ensure recovery from any specific set of failure events, such as any single SRLG failure, any two SRLG failures etc. Clearly, more restoration capacity is allocated if a greater degree of failure recovery is required. Thus, the degree to which the network is protected is determined by the policy that defines the amount of reserved restoration bandwidth. - Recovery time: In general, the more pre-planning of the restoration route, the more rapid the P&R scheme. Protection schemes generally recover faster than restoration schemes. Restoration with pre-signaled bandwidth reservation are likely to be (significantly) faster than path restoration with re- provisioning, especially because of the elimination of any crankback. Local restoration will generally be faster than end-to- end schemes. Recovery time objectives for SONET/SDH protection switching (not including time to detect failure) are specified in [G.841] at 50 E. Mannie et. al. Internet-Draft February 2003 50 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 ms, taking into account constraints on distance, number of connections involved, and in the case of ring enhanced protection, number of nodes in the ring. Recovery time objectives for restoration mechanisms are being defined through a separate effort [TE-RESTORE]. - Resource Sharing: 1+1 and 1:N link and LSP protection require dedicated recovery paths with limited ability to share resources: 1+1 allows no sharing, 1:N allows some sharing of protection resources and support of extra (preemptable) traffic. Flexibility is limited because of topology restrictions, e.g. fixed ring topology for traditional enhanced protection schemes. The degree to which restoration schemes allow sharing amongst multiple independent failures is directly dictated by the size of the restoration pool. In restoration schemes with re-provisioning, a pool of restoration capacity can be defined from which all restoration routes are selected after failure. Thus, the degree of sharing is defined by the amount of available restoration capacity. In restoration with pre-signaled bandwidth reservation, the amount of reserved restoration capacity is determined by the local bandwidth reservation policies. In all restoration schemes, pre-emptable resources can use spare restoration capacity when that capacity is not being used for failure recovery. 14. Network Management Service Providers (SPs) use network management extensively to configure, monitor or provision various devices in their network. It is important to note that a SPÆs equipment may be distributed across geographically separate sites, making distributed management even more important. The service provider should utilize an NMS system and standard management protocols such as SNMP [RFC1901] [RFC1902] [RFC1903] [RFC1904] [RFC1905] [RFC1906] and its associated MIBs as standard interfaces to configure, monitor and provision devices at various locations. The service provider may also wish to use the command line interface (CLI) provided by vendors with their devices, but this however, is not a standard or recommended solution due to the fact that there is no standard CLI language or interface, which results in N different CLIÆs in a network with devices from N different vendors. In the context of GMPLS, it is extremely important for standard interfaces to the SPÆs devices (SNMP) to exist due to the nature of the technology itself. Since GMPLS comprises many different layers of control-plane and data-plane technology, it is important for management interfaces in this area to be flexible enough to allow the manager to manage GMPLS easily, and in a standard way. 14.1. Network Management Systems (NMS) The NMS system should maintain the collective information about each device within the system. Note that the NMS system may actually be comprised of several distributed applications (i.e.: alarm E. Mannie et. al. Internet-Draft February 2003 51 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 aggregators, configuration consoles, polling applications, etc...) that collectively comprises the SPÆs NMS. In this way, it can make provisioning and maintenance decisions with the full knowledge of the entire SP network. Configuration or provisioning information (i.e.: requests for new services) could be entered into the NMS and subsequently distributed via SNMP to the remote devices, making the SPÆs job of managing the network much more compact and effortless than having to manage each device individually (i.e.: via CLI). Security and access control can be achieved through the use of SNMPv3 and the View Access Control Model [SNMPv3VACM]. This approach can be very effectively used within an SP network, since the SP has access to and control over all devices within its domain. Standardized MIBs will need to be developed before this approach can be used ubiquitously to provision, configure and monitor devices in non-heterogeneous networks or across SP boundaries. 14.2. Management Information Base (MIB) In the context of GMPLS, it is extremely important for standard interfaces to devices to exist due to the nature of the technology itself. Since GMPLS comprises many different layers of control-plane technology, it is important for SNMP MIBs in this area to be flexible enough to allow the manager to manage the entire control plane. This should be through a set of MIBs that may cooperate (i.e.: coordinated row-creation on the agent), or through more generalized MIBs that aggregate some of the desired actions to be taken and push those details down to the devices. It is important to note that in certain circumstances, it may be necessary to duplicate some small subset of manageable objects in new MIBs for the convenience of management. Control of some parts of GMPLS may also be achieved though the use of existing MIB interfaces (i.e.: existing SONET MIB), or though separate ones, which are yet to be defined. MIBs may have been previously defined in the IETF or ITU. Existing MIBs may need to be extended to facilitate some of the new functionality desired by GMPLS. In these cases, the working group should work on new versions of these MIBs so that these extensions can be added. 14.3. Tools As in traditional networks, standard tools such as traceroute [RFC1393] and ping [RFC1739] are needed for debugging and performance monitoring of GMPLS networks, and mainly for the control plane topology that will mimic the data plane topology. Furthermore, such tools provide network reachability information. The GMPLS control protocols will need to expose certain pieces of information in order for these tools to function properly and to provide information germane to GMPLS. These tools should be made available via the CLI. These tools should also be made available for remote invocation via the SNMP interface [RFC2925]. 14.4. Fault Correlation Between Multiple Layers E. Mannie et. al. Internet-Draft February 2003 52 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 Due to the nature of GMPLS and the fact that potential layers may be involved in the control and transmission of GMPLS data and control information, it is therefore required that a fault in one layer be passed to the adjacent higher and lower layers in an effort to notify them of the fault. However, due to nature of these many layers, it is possible and even probable, that hundreds or even thousands of notifications may need to transpire between layers. This is undesirable for several reasons. First, these notifications will overwhelm the device. Second, if the device(s) are programmed to emit SNMP Notifications [RFC1901] then the large number of notifications the device may attempt to emit may overwhelm the network with a storm of notifications. Furthermore, even if the device emits the notifications, the NMS that must process these notifications will either be overwhelmed, or will be processing redundant information. That is, if 1000 interfaces at layer B are stacked above a single interface below it at layer A, and the interface at A goes down, the interfaces at layer B should not emit notifications. Instead, the interface at layer A should emit a single notification. The NMS receiving this notification should be able to correlate the fact that this interface has many others stacked above it and take appropriate action, if necessary. Devices that support GMPLS should provide mechanisms for aggregating, summarizing, enabling and disabling of inter-layer notifications for the reasons described above. In the context of SNMP MIBs, all MIBs that are used by GMPLS must provide enable/disable objects for all notification objects. Furthermore, these MIBs must also provide notification summarization objects or functionality (as described above) as well. NMS systems and standard tools which process notifications or keep track of the many layers on any given devices must be capable of processing the vast amount of information which may potentially be emitted by network devices running GMPLS at any point in time. 15. Security considerations GMPLS defines a new control plane architecture for multiple types of network elements. In general, since LSPs established using GMPLS are expected to carry high volumes of data and consume significant network resources, security mechanisms are required to safeguard the underlying network against attacks on the control plane and/or unauthorized usage of data transport resources. Security requirements depend on the level of trust between nodes that exchange GMPLS control messages as well as the exposure of the control channel to third parties. In general, a network node may apply more relaxed security requirements when exchanging GMPLS control messages with nodes under the same administrative domain than when talking to nodes in a different domain. In this respect, network to user (UNI) and network-to-network interfaces are expected to have higher security requirements than node-to-node interfaces. Security mechanisms can provide two main properties: authentication and confidentiality. Authentication can provide origin verification, E. Mannie et. al. Internet-Draft February 2003 53 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 message integrity and replay protection, while confidentiality ensures that a third party cannot decipher the contents of a message. In situations where GMPLS deployment requires primarily authentication, the respective authentication mechanisms of the GMPLS component protocols may be used ([RFC2747], [RFC3036], [RFC2385], [LMP]). Additionally, the IPSEC suite of protocols ([RFC2402], [RFC2406], [RFC2409]) may be used to provide authentication, confidentiality or both, for a GMPLS control channel; this option offers the benefit of combined protection of all GMPLS component protocols. Note however that GMPLS itself introduces no new security considerations to the current MPLS-TE signaling (RSVP-TE, CR-LDP), routing protocols (OSPF-TE, IS-IS-TE) or network management protocols (SNMP). 16. Acknowledgements This draft is the work of numerous authors and consists of a composition of a number of previous drafts in this area. Many thanks to Ben Mack-Crane (Tellabs) for all the useful SDH/SONET discussions that we had together. Thanks also to Pedro Falcao, Alexandre Geyssens, Michael Moelants, Xavier Neerdaels, and Philippe Noel from Ebone for their SDH/SONET and optical technical advice and support. Finally, many thanks also to Krishna Mitra (Calient) and Curtis Villamizar (Avici). A list of the drafts from which material and ideas were incorporated follows: [GMPLS-SIG] draft-ietf-mpls-generalized-signaling-07.txt Generalized MPLS - Signaling Functional Description [RSVP-TE-GMPLS] draft-ietf-mpls-generalized-rsvp-te-06.txt Generalized MPLS Signaling - RSVP-TE Extensions [CR-LDP-GMPLS] draft-ietf-mpls-generalized-cr-ldp-05.txt Generalized MPLS Signaling - CR-LDP Extensions [SONETSDH-GMPLS] draft-ietf-ccamp-gmpls-sonet-sdh-03.txt GMPLS Extensions for SONET and SDH Control [SONETSDH-EXT-GMPLS] draft-ietf-ccamp-gmpls-sonet-sdh-extensions- 01.txt. GMPLS Extensions to Control Non-Standard SONET and SDH Features [G709-GMPLS] draft-ietf-ccamp-gmpls-g709-01.txt GMPLS Signaling Extensions for G.709 Optical Transport Networks Control [LMP] draft-ietf-mpls-lmp-02.txt Link Management Protocol (LMP) E. Mannie et. al. Internet-Draft February 2003 54 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 [LMP-WDM] draft-ietf-ccamp-lmp-wdm-00.txt LMP for WDM Optical Line Systems (LMP-WDM) [HIERARCHY] draft-ietf-mpls-lsp-hierarchy-04.txt LSP Hierarchy with MPLS TE [RSVP-TE-UNNUM] draft-ietf-mpls-rsvp-unnum-04.txt Signalling Unnumbered Links in RSVP-TE [CR-LDP-UNNUM] draft-ietf-mpls-crldp-unnum-04.txt Signalling Unnumbered Links in CR-LDP [BUNDLE] draft-ietf-mpls-bundle-01.txt Link Bundling in MPLS Traffic Engineering [GMPLS-ROUTING] draft-ietf-ccamp-gmpls-routing-02.txt Routing Extensions in Support of Generalized MPLS [OSPF-TE-GMPLS] draft-ietf-ccamp-ospf-gmpls-extensions-04.txt OSPF Extensions in Support of Generalized MPLS [ISIS-TE-GMPLS] draft-ietf-isis-gmpls-extensions-08.txt IS-IS Extensions in Support of Generalized MPLS 17. References [RFC1393] G. Malkin, "Traceroute Using an IP Option", IETF RFC 1393, January 1993. [RFC1901] Case, J., McCloghrie, K., Rose, M., and S. Waldbusser, "Introduction to Community-based SNMPv2", IETF RFC 1901, January 1996. [RFC1902] Case, J., McCloghrie, K., Rose, M., and S. Waldbusser, "Structure of Management Information for Version 2 of the Simple Network Management Protocol (SNMPv2)", IETF RFC 1902, January 1996. [RFC1903] Case, J., McCloghrie, K., Rose, M., and S. Waldbusser, "Textual Conventions for Version 2 of the Simple Network Management Protocol (SNMPv2)", IETF RFC 1903, January 1996. [RFC1904] Case, J., McCloghrie, K., Rose, M., and S. Waldbusser, "Conformance Statements for Version 2 of the Simple Network Management Protocol (SNMPv2)", IETF RFC 1904, January 1996. [RFC1905] Case, J., McCloghrie, K., Rose, M., and S. Waldbusser, "Protocol Operations for Version 2 of the Simple Network Management Protocol (SNMPv2)", IETF RFC 1905, January 1996. [RFC1906] Case, J., McCloghrie, K., Rose, M., and S. E. Mannie et. al. Internet-Draft February 2003 55 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 Waldbusser, "Transport Mappings for Version 2 of the Simple Network Management Protocol (SNMPv2)", IETF RFC 1906, January 1996. [SNMPv3VACM] Wijnen, B., Presuhn, R., and K. McCloghrie, "View- based Access Control Model (VACM) for the Simple Network Management Protocol (SNMP)", IETF RFC 2575, April 1999. [RFC1739] Kessler G. and Shepard S., "A Primer On Internet and TCP/IP Tools", IETF RFC 1739, December 1994. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, IETF RFC 2119, March 1997. [RFC2328] J. Moy, "OSPF Version 2", IETF RFC 2328, Standard Track, April 1998. [RFC2370] R. Coltun, "The OSPF Opaque LSA Option", IETF RFC 2370 Standard Track, July 1998. [RFC2385] A. Heffernan, "Protection of BGP Sessions via the TCP MD5 Signature Option", IETF RFC 2385, August 1998. [RFC2402] S. Kent and R. Atkinson, "IP Authentication Header", IETF RFC 2402, November 1998. [RFC2406] S. Kent and R. Atkinson, "IP Encapsulating Security Payload (ESP)", IETF RFC 2406, November 1998. [RFC2409] D. Harkins and D. Carrel, "The Internet Key Exchange (IKE)", IETF RFC 2409, November 1998. [RFC2747] F. Baker et al., "RSVP Cryptographic Authentication", IETF RFC 2747, January 2000. [RFC2925] K. White, "Definitions of Managed Objects for Remote Ping, Traceroute, and Lookup Operations", IETF RFC 2925, September 2000. [RFC3031] E. Rosen, A. Viswanathan, R. Callon, "Multiprotocol Label Switching Architecture", IETF RFC 3031, January 2001. [RFC3032] E. Rosen, D. Tappan, G. Fedorkow, Y. Rekhter, D. Farinacci, T. Li, A. Conta, "MPLS Label Stack Encoding", IETF RFC 3032, January 2001. [RFC3036] L. Andersson, P. Doolan, N. Feldman, A. Fredette, B. Thomas, "LDP Specification", IETF RFC 3036, January 2001. [OSPF-TE] D. Katz, D. Yeung, and K. Kompella, "Traffic Engineering Extensions to OSPF", draft-katz-yeung-ospf- E. Mannie et. al. Internet-Draft February 2003 56 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 traffic-06.txt. [MPLS-TEO] D. Awduche et al., "Multi-Protocol Lambda Switching: Combining MPLS Traffic Engineering Control With Optical Crossconnects", Internet Draft, Work in Progress, draft-awduche-mpls-te-optical-03.txt, April 2001. [G.841] ITU-T Recommendation G.841, "Types and Characteristics of SDH Network Protection Architectures", October 1998. [ANSI-T1.105] "Synchronous Optical Network (SONET): Basic Description Including Multiplex Structure, Rates, and Formats", ANSI T1.105, 2000. [TE-RESTORE] W. Lai, D. McDysan, J. Boyle, et al, "Network Hierarchy and Multi-layer Survivability", Internet Draft, Work in Progress, draft-ietf-tewg-restore-hierarchy-00.txt, October 2001. [MPLS-RECOVERY] V. Sharma and F. Hellstrand (Editors), "A Framework for MPLS Recovery", Internet Draft, Work in Progress, draft-ietf-mpls-recovery-frmwrk-06.txt, July 2002. [SDH/SONET-GMPLS-FRAMEWORK] G. Bernstein, E. Mannie, V. Sharma, "Framework for GMPLS-based Control of SDH/SONET Networks", Internet Draft, Work in Progress, draft-ietf-ccamp-sdhsonet-control-01.txt, May 2002. [OLI-REQ] A. Fredette (Editor), "Optical Link Interface Requirements", Internet Draft, Work in Progress, draft-ietf-ccamp-oli-reqts-00.txt, February 2002. [MANCHESTER] J. Manchester, P. Bonenfant, C. Newton, "The Evolution of Transport Network Survivability", IEEE Communications, August 1999. 18. Author's Address Eric Mannie Ebone (GTS) Terhulpsesteenweg 6A 1560 Hoeilaart Belgium Phone: +32 2 658-5652 Email: eric.mannie@gts.com E. Mannie et. al. Internet-Draft February 2003 57 draft-ietf-ccamp-gmpls-architecture-03.txt August 2002 Full Copyright Statement "Copyright (C) The Internet Society (date). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE." E. Mannie et. al. Internet-Draft February 2003 58