idnits 2.17.1 draft-zhang-pce-stateful-pce-app-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (May 24, 2013) is 3983 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'I-D.crabbe-pce-stateful-pce-mpls-te' is defined on line 953, but no explicit reference was found in the text == Unused Reference: 'MPLS-PC' is defined on line 991, but no explicit reference was found in the text == Unused Reference: 'MXMN-TE' is defined on line 995, but no explicit reference was found in the text == Unused Reference: 'NET-REC' is defined on line 1000, but no explicit reference was found in the text == Unused Reference: 'RFC5394' is defined on line 1022, but no explicit reference was found in the text == Outdated reference: A later version (-21) exists of draft-ietf-pce-stateful-pce-04 == Outdated reference: A later version (-10) exists of draft-ietf-ccamp-gmpls-general-constraints-ospf-te-04 == Outdated reference: A later version (-17) exists of draft-ietf-ccamp-wson-signal-compatibility-ospf-11 == Outdated reference: A later version (-16) exists of draft-ietf-pce-gmpls-pcep-extensions-07 == Outdated reference: A later version (-03) exists of draft-ogrcetal-ccamp-flexi-grid-fwk-02 == Outdated reference: A later version (-03) exists of draft-sivabalan-pce-disco-stateful-01 Summary: 1 error (**), 0 flaws (~~), 12 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group X. Zhang, Ed. 3 Internet-Draft Huawei Technologies 4 Intended status: Informational I. Minei, Ed. 5 Expires: November 25, 2013 Juniper Networks, Inc. 6 May 24, 2013 8 Applicability of Stateful Path Computation Element (PCE) 9 draft-zhang-pce-stateful-pce-app-04 11 Abstract 13 A stateful Path Computation Element (PCE) maintains information about 14 Label Switched Path (LSP) characteristics and resource usage within a 15 network in order to provide traffic engineering calculations for its 16 associated Path Computation Clients (PCCs). This document describes 17 general considerations for a stateful PCE deployment and examines its 18 applicability and benefits through a number of use cases. Path 19 Computation Element Protocol (PCEP) extensions required for stateful 20 PCE usage are covered in separate documents. 22 Status of this Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at http://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on November 25, 2013. 39 Copyright Notice 41 Copyright (c) 2013 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (http://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 57 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 58 3. Overview of stateful PCE . . . . . . . . . . . . . . . . . . . 4 59 4. Deployment considerations . . . . . . . . . . . . . . . . . . 5 60 4.1. Multi-PCE deployments . . . . . . . . . . . . . . . . . . 5 61 4.2. LSP State Synchronization . . . . . . . . . . . . . . . . 5 62 4.3. PCE Survivability . . . . . . . . . . . . . . . . . . . . 5 63 5. Application scenarios . . . . . . . . . . . . . . . . . . . . 6 64 5.1. Optimization of LSP placement . . . . . . . . . . . . . . 6 65 5.1.1. Throughput Maximization and Bin Packing . . . . . . . 7 66 5.1.2. Deadlock . . . . . . . . . . . . . . . . . . . . . . . 8 67 5.1.3. Minimum Perturbation . . . . . . . . . . . . . . . . . 10 68 5.1.4. Predictability . . . . . . . . . . . . . . . . . . . . 11 69 5.2. Auto-bandwidth Adjustment . . . . . . . . . . . . . . . . 12 70 5.3. Bandwidth Scheduling . . . . . . . . . . . . . . . . . . . 13 71 5.4. Recovery . . . . . . . . . . . . . . . . . . . . . . . . . 13 72 5.4.1. Protection . . . . . . . . . . . . . . . . . . . . . . 13 73 5.4.2. Restoration . . . . . . . . . . . . . . . . . . . . . 15 74 5.4.3. SRLG Diversity . . . . . . . . . . . . . . . . . . . . 16 75 5.5. Maintenance of Virtual Network Topology (VNT) . . . . . . 16 76 5.6. LSP Re-optimization . . . . . . . . . . . . . . . . . . . 17 77 5.7. Resource Defragmentation . . . . . . . . . . . . . . . . . 17 78 5.8. Impairment-Aware Routing and Wavelength Assignment 79 (IA-RWA) . . . . . . . . . . . . . . . . . . . . . . . . . 18 80 6. Security Considerations . . . . . . . . . . . . . . . . . . . 19 81 7. Contributing Authors . . . . . . . . . . . . . . . . . . . . . 19 82 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 21 83 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 21 84 9.1. Normative References . . . . . . . . . . . . . . . . . . . 21 85 9.2. Informative References . . . . . . . . . . . . . . . . . . 21 86 Appendix A. Editorial notes and open issues . . . . . . . . . . . 23 87 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23 89 1. Introduction 91 [RFC4655] defines the architecture for a Path Computation Element 92 (PCE)-based model for the computation of Multiprotocol Label 93 Switching (MPLS) and Generalized MPLS (GMPLS) Traffic Engineering 94 Label Switched Paths (TE LSPs). To perform such a constrained 95 computation, a PCE stores the network topology (i.e., TE links and 96 nodes) and resource information (i.e., TE attributes) in its TE 97 Database (TED). [RFC5440] describes the Path Computation Element 98 Protocol (PCEP). PCEP defines the communication between a Path 99 Computation Client (PCC) and a Path Control Element (PCE), or between 100 two PCEs, enabling computation of Multiprotocol Label Switching 101 (MPLS) for Traffic Engineering Label Switched Path (TE LSP) 102 characteristics. Extensions for support of GMPLS in PCEP are defined 103 in [I-D.ietf-pce-gmpls-pcep-extensions]. 105 As per [RFC4655], a PCE can be either stateful or stateless. 106 Stateless PCEs have been shown to be useful in many scenarios, 107 including constraint-based path computation in multi-domain/ 108 multi-layer networks. Compared to a stateless PCE, a stateful PCE 109 has access to not only the network state, but also to the set of 110 active paths and their reserved resources. Furthermore, a stateful 111 PCE might also retain information regarding LSPs under construction 112 in order to reduce churn and resource contention. This state allows 113 the PCE to compute constrained paths while considering individual 114 LSPs and their interactions. Note that this requires reliable state 115 synchronization mechanisms between the PCE and the network, PCE and 116 PCC, and between cooperating PCEs, with potentially significant 117 control plane overhead and maintenance of a large amount of state 118 data, as explained in [RFC4655]. 120 This document describes how a stateful PCE can be used to solve 121 various problems for MPLS-TE and GMPLS networks, and the benefits it 122 brings to such deployments. Note that alternative solutions relying 123 on stateless PCEs may also be possible for some of these use cases, 124 and will be mentioned for completeness where appropriate. 126 2. Terminology 128 This document uses the following terms defined in [RFC5440]: PCC, 129 PCE, PCEP Peer. 131 This document uses the following terms defined in 132 [I-D.ietf-pce-stateful-pce]: Passive Stateful PCE, Active Stateful 133 PCE, Delegation, Revocation, Delegation Timeout Interval, LSP State 134 Report, LSP Update Request, LSP State Database. 136 This document defines the following term: 138 Minimum Cut Set: the minimum set of links for a specific source 139 destination pair which, when removed from the network, result in a 140 specific source being completely isolated from specific 141 destination. The summed capacity of these links is equivalent to 142 the maximum capacity from the source to the destination by the 143 max-flow min-cut theorem. 145 3. Overview of stateful PCE 147 This section is included for the convenience of the reader, please 148 refer to the referenced documents for details of the operation. 150 [I-D.ietf-pce-stateful-pce] specifies a set of extensions to PCEP to 151 enable stateful control of tunnels within and across PCEP sessions in 152 compliance with [RFC4657]. It includes mechanisms to effect tunnel 153 state synchronization between PCCs and PCEs, delegation of control 154 over tunnels to PCEs, and PCE control of timing and sequence of path 155 computations within and across PCEP sessions. 157 [I-D.ietf-pce-stateful-pce] applies equally to MPLS-TE and GMPLS 158 LSPs. 160 Several new functions were added in PCEP to support stateful PCEs and 161 are described in [I-D.ietf-pce-stateful-pce]. A function can be 162 initiated either from a PCC towards a PCE (C-E) or from a PCE towards 163 a PCC (E-C). The new functions are: 165 Capability negotiation (E-C,C-E): both the PCC and the PCE must 166 announce during PCEP session establishment that they support PCEP 167 Stateful PCE extensions. 169 LSP state synchronization (C-E): after the session between the PCC 170 and a stateful PCE is initialized, the PCE must learn the state of 171 a PCC's LSPs before it can perform path computations or update LSP 172 attributes in a PCC. 174 LSP Update Request (E-C): A PCE requests modification of attributes 175 on a PCC's LSP. 177 LSP State Report (C-E): a PCC sends an LSP State Report to a PCE 178 whenever the state of an LSP changes. 180 LSP control delegation (C-E,E-C): a PCC grants to a PCE the right to 181 update LSP attributes on one or more LSPs; the PCE becomes the 182 authoritative source of the LSP's attributes as long as the 183 delegation is in effect; the PCC may withdraw the delegation or 184 the PCE may give up the delegation. 186 [I-D.sivabalan-pce-disco-stateful] defines the extensions needed to 187 support autodiscovery of stateful PCEs when using the IGPs for PCE 188 discovery. 190 4. Deployment considerations 192 This section discusses generic issues with Stateful PCE deployments, 193 and how specific protocol mechanisms can be used to address them. 195 4.1. Multi-PCE deployments 197 Stateless and stateful PCEs can co-exist in the same network and be 198 in charge of path computation of different types. To solve the 199 problem of distinguishing between the two types of PCEs, either 200 discovery or configuration may be used. The capability negotiation 201 in [I-D.ietf-pce-stateful-pce] ensures correct operation when the PCE 202 address is configured on the PCC. 204 4.2. LSP State Synchronization 206 A stateful PCE maintains two sets of information for use in path 207 computation. The first is the Traffic Engineering Database (TED) 208 which includes the topology and resource state in the network. This 209 information can be obtained by a stateful PCE using the same 210 mechanisms as a stateless PCE (see [RFC4655]). The second is the LSP 211 State Database (LSP-DB), in which a PCE stores attributes of all 212 active LSPs in the network, such as their paths through the network, 213 bandwidth/resource usage, switching types and LSP constraints. The 214 stateful PCE extensions defined in [I-D.ietf-pce-stateful-pce] 215 support population of this database using information received from 216 the network nodes via LSP State Report messages. Population of the 217 LSP database via other means is not precluded. 219 4.3. PCE Survivability 221 For a stateful PCE, an important issue is to get the LSP state 222 information resynchronized after a restart. 223 [I-D.ietf-pce-stateful-pce] includes support of a synchronization 224 function, allowing the PCC to synchronize its LSP state with the PCE. 225 This can be applied equally to an Label Edge Router (LER) client or 226 another PCE, allowing for support of multiple ways of re-acquiring 227 the LSP database on a restart. For example, the state can be 228 retrieved from the network nodes, or from another stateful PCE. 229 Because synchronization may also be skipped, if a PCE implementation 230 has the means to retrieve its database in a different way (for 231 example from a backup copy stored locally), the state can be restored 232 without further overhead in the network. Note that locally 233 recovering the state would still require some degree of 234 resynchronization to ensure that the recovered state is indeed up-to- 235 date. 237 5. Application scenarios 239 In the following sections, several use cases are described, 240 showcasing scenarios that benefit from the deployment of a stateful 241 PCE. 243 5.1. Optimization of LSP placement 245 The following use cases demonstrate a need for visibility into global 246 inter-PCC LSP state in PCE path computations, and for a PCE control 247 of sequence and timing in altering LSP path characteristics within 248 and across PCEP sessions. Reference topologies for the use cases 249 described later in this section are shown in Figures 1 and 2. 251 Some of the use cases below are focused on MPLS-TE deployments, but 252 may also apply to GMPLS. Unless otherwise cited, use cases assume 253 that all LSPs listed exist at the same LSP priority. 255 The main benefit in the cases below comes from moving away from an 256 asynchronous PCC-driven mode of operation to a model that allows for 257 central control over LSP computations and setup, and focuses 258 specifically on the active stateful PCE model of operation. 260 +-----+ 261 | A | 262 +-----+ 263 \ 264 +-----+ +-----+ 265 | C |----------------------| E | 266 +-----+ +-----+ 267 / \ +-----+ / 268 +-----+ +-----| D |-----+ 269 | B | +-----+ 270 +-----+ 272 Figure 1: Reference topology 1 274 +-----+ +-----+ +-----+ 275 | A | | B | | C | 276 +--+--+ +--+--+ +--+--+ 277 | | | 278 | | | 279 +--+--+ +--+--+ +--+--+ 280 | E +--------+ F +--------+ G | 281 +-----+ +-----+ +-----+ 283 Figure 2: Reference topology 2 285 5.1.1. Throughput Maximization and Bin Packing 287 Because LSP attribute changes in [RFC5440] are driven by PCReq 288 messages under control of a PCC's local timers, the sequence of RSVP 289 reservation arrivals occurring in the network will be randomized. 290 This, coupled with a lack of global LSP state visibility on the part 291 of a stateless PCE may result in suboptimal throughput in a given 292 network topology, as will be shown in the example below. 294 Reference topology 2 in Figure 2 and Tables 1 and 2 show an example 295 in which throughput is at 50% of optimal as a result of lack of 296 visibility and synchronized control across PCC's. In this scenario, 297 the decision must be made as to whether to route any portion of the 298 E-G demand, as any demand routed for this source and destination will 299 decrease system throughput. 301 +------+--------+----------+ 302 | Link | Metric | Capacity | 303 +------+--------+----------+ 304 | A-E | 1 | 10 | 305 | B-F | 1 | 10 | 306 | C-G | 1 | 10 | 307 | E-F | 1 | 10 | 308 | F-G | 1 | 10 | 309 +------+--------+----------+ 311 Table 1: Link parameters for Throughput use case 313 +------+-----+-----+-----+--------+----------+-------+ 314 | Time | LSP | Src | Dst | Demand | Routable | Path | 315 +------+-----+-----+-----+--------+----------+-------+ 316 | 1 | 1 | E | G | 10 | Yes | E-F-G | 317 | 2 | 2 | A | B | 10 | No | --- | 318 | 3 | 1 | F | C | 10 | No | --- | 319 +------+-----+-----+-----+--------+----------+-------+ 321 Table 2: Throughput use case demand time series 323 In many cases throughput maximization becomes a bin packing problem. 324 While bin packing itself is an NP-hard problem, a number of common 325 heuristics which run in polynomial time can provide significant 326 improvements in throughput over random reservation event 327 distribution, especially when traversing links which are members of 328 the minimum cut set for a large subset of source destination pairs. 330 Tables 3 and 4 show a simple use case using Reference Topology 1 in 331 Figure 1, where LSP state visibility and control of reservation order 332 across PCCs would result in significant improvement in total 333 throughput. 335 +------+--------+----------+ 336 | Link | Metric | Capacity | 337 +------+--------+----------+ 338 | A-C | 1 | 10 | 339 | B-C | 1 | 10 | 340 | C-E | 10 | 5 | 341 | C-D | 1 | 10 | 342 | D-E | 1 | 10 | 343 +------+--------+----------+ 345 Table 3: Link parameters for Bin Packing use case 347 +------+-----+-----+-----+--------+----------+---------+ 348 | Time | LSP | Src | Dst | Demand | Routable | Path | 349 +------+-----+-----+-----+--------+----------+---------+ 350 | 1 | 1 | A | E | 5 | Yes | A-C-D-E | 351 | 2 | 2 | B | E | 10 | No | --- | 352 +------+-----+-----+-----+--------+----------+---------+ 354 Table 4: Bin Packing use case demand time series 356 5.1.2. Deadlock 358 This section discusses a use case of cross-LSP impact under degraded 359 operation. Most existing RSVP-TE implementations will not tear down 360 established LSPs in the event of the failure of the bandwidth 361 increase procedure detailed in [RFC3209]. This behavior is directly 362 implied to be correct in [RFC3209] and is often desirable from an 363 operator's perspective, because either a) the destination prefixes 364 are not reachable via any means other than MPLS or b) this would 365 result in significant packet loss as demand is shifted to other LSPs 366 in the overlay mesh. 368 In addition, there are currently few implementations offering dynamic 369 ingress admission control (policing of the traffic volume mapped onto 370 an LSP) at the LER. Having ingress admission control on a per LSP 371 basis is not necessarily desirable from an operational perspective, 372 as a) one must over-provision tunnels significantly in order to avoid 373 deleterious effects resulting from stacked transport and flow control 374 systems and b) there is currently no efficient commonly available 375 northbound interface for dynamic configuration of per LSP ingress 376 admission control (such an interface could easily be defined using 377 the extensions for stateful PCE, but has not been yet at the time of 378 this writing). 380 Lack of ingress admission control coupled with the behavior in 381 [RFC3209] may result in LSPs operating out of profile for significant 382 periods of time. It is reasonable to expect that these out-of- 383 profile LSPs will be operating in a degraded state and experience 384 traffic loss, but because they end up sharing common network 385 interfaces with other LSPs operating within their bandwidth 386 reservations, they will end up impacting the operation of the in- 387 profile LSPs, even when there is unused network capacity elsewhere in 388 the network. Furthermore, this behavior will cause information loss 389 in the TED with regards to the actual available bandwidth on the 390 links used by the out-of-profile LSPs, as the reservations on the 391 links no longer reflect the capacity used. 393 Reference Topology 1 in Figure 1 and Tables 5 and 6 show a use case 394 that demonstrates this behavior. Two LSPs, LSP 1 and LSP 2 are 395 signaled with demand 2 and routed along paths A-C-D-E and B-C-D-E 396 respectively. At a later time, the demand of LSP 1 increases to 20. 397 Under such a demand, the LSP cannot be resignaled. However, the 398 existing LSP will not be torn down. In the absence of ingress 399 policing, traffic on LSP 1 will cause degradation for traffic of LSP 400 2 (due to oversubscription on the links C-D and D-E), as well as 401 information loss in the TED with regard to the actual network state. 403 The problem could be easily ameliorated by global visibility of LSP 404 state coupled with PCC-external demand measurements and placement of 405 two LSPs on disjoint links. Note that while the demand of 20 for LSP 406 1 could never be satisfied in the given topology, what could be 407 achieved would be isolation from the ill-effects of the 408 (unsatisfiable) increased demand. 410 +------+--------+----------+ 411 | Link | Metric | Capacity | 412 +------+--------+----------+ 413 | A-C | 1 | 10 | 414 | B-C | 1 | 10 | 415 | C-E | 10 | 5 | 416 | C-D | 1 | 10 | 417 | D-E | 1 | 10 | 418 +------+--------+----------+ 420 Table 5: Link parameters for the 'Degraded operation' example 422 +------+-----+-----+-----+--------+----------+---------+ 423 | Time | LSP | Src | Dst | Demand | Routable | Path | 424 +------+-----+-----+-----+--------+----------+---------+ 425 | 1 | 1 | A | E | 2 | Yes | A-C-D-E | 426 | 2 | 2 | B | E | 2 | Yes | B-C-D-E | 427 | 3 | 1 | A | E | 20 | No | --- | 428 +------+-----+-----+-----+--------+----------+---------+ 430 Table 6: Degraded operation demand time series 432 5.1.3. Minimum Perturbation 434 As a result of both the lack of visibility into global LSP state and 435 the lack of control over event ordering across PCE sessions, 436 unnecessary perturbations may be introduced into the network by a 437 stateless PCE. Tables 7 and 8 show an example of an unnecessary 438 network perturbation using Reference Topology 1 in Figure 1. In this 439 case an unimportant (high LSP priority value) LSP (LSP1) is first set 440 up along the shortest path. At time 2, which is assumed to be 441 relatively close to time 1, a second more important (lower LSP- 442 priority value) LSP (LSP2) is established, preempting LSP1, 443 potentially causing traffic loss. LSP1 is then reestablished on the 444 longer A-C-E path. 446 +------+--------+----------+ 447 | Link | Metric | Capacity | 448 +------+--------+----------+ 449 | A-C | 1 | 10 | 450 | B-C | 1 | 10 | 451 | C-E | 10 | 10 | 452 | C-D | 1 | 10 | 453 | D-E | 1 | 10 | 454 +------+--------+----------+ 456 Table 7: Link parameters for the 'Minimum-Perturbation' example 458 +------+-----+-----+-----+--------+----------+----------+---------+ 459 | Time | LSP | Src | Dst | Demand | LSP Prio | Routable | Path | 460 +------+-----+-----+-----+--------+----------+----------+---------+ 461 | 1 | 1 | A | E | 7 | 7 | Yes | A-C-D-E | 462 | 2 | 2 | B | E | 7 | 0 | Yes | B-C-D-E | 463 | 3 | 1 | A | E | 7 | 7 | Yes | A-C-E | 464 +------+-----+-----+-----+--------+----------+----------+---------+ 466 Table 8: Minimum-Perturbation LSP and demand time series 468 A stateful PCE can help in this scenario by evaluating both requests 469 at the same time (due to their proximity in time). This will ensure 470 placement of the more important LSP along the shortest path, avoiding 471 the preemption of the lower priority LSP. 473 5.1.4. Predictability 475 Randomization of reservation events caused by lack of control over 476 event ordering across PCE sessions results in poor predictability in 477 LSP routing. An offline system applying a consistent optimization 478 method will produce predictable results to within either the boundary 479 of forecast error when reservations are over-provisioned by 480 reasonable margins or to the variability of the signal and the 481 forecast error when applying some hysteresis in order to minimize 482 churn. Predictable results are valuable for being able to simulate 483 the network and reliably test it under various scenarios, especially 484 under various failure modes and planned maintenances when predictable 485 path characteristics are desired under contention for network 486 resources. 488 Reference Topology 1 and Tables 9, 10 and 11 show the impact of event 489 ordering and predictability of LSP routing. 491 +------+--------+----------+ 492 | Link | Metric | Capacity | 493 +------+--------+----------+ 494 | A-C | 1 | 10 | 495 | B-C | 1 | 10 | 496 | C-E | 1 | 10 | 497 | C-D | 1 | 10 | 498 | D-E | 1 | 10 | 499 +------+--------+----------+ 501 Table 9: Link parameters for the 'Predictability' example 502 +------+-----+-----+-----+--------+----------+---------+ 503 | Time | LSP | Src | Dst | Demand | Routable | Path | 504 +------+-----+-----+-----+--------+----------+---------+ 505 | 1 | 1 | A | E | 7 | Yes | A-C-E | 506 | 2 | 2 | B | E | 7 | Yes | B-C-D-E | 507 +------+-----+-----+-----+--------+----------+---------+ 509 Table 10: Predictability LSP and demand time series 1 511 +------+-----+-----+-----+--------+----------+---------+ 512 | Time | LSP | Src | Dst | Demand | Routable | Path | 513 +------+-----+-----+-----+--------+----------+---------+ 514 | 1 | 2 | B | E | 7 | Yes | B-C-E | 515 | 2 | 1 | A | E | 7 | Yes | A-C-D-E | 516 +------+-----+-----+-----+--------+----------+---------+ 518 Table 11: Predictability LSP and demand time series 2 520 As can be shown in the example, both LSPs were routed in both cases, 521 but along very different paths. This would be a challenge if 522 reliable simulation of the network was attempted. A stateful PCE can 523 solve this through control over LSP ordering. 525 5.2. Auto-bandwidth Adjustment 527 The bandwidth requirement of LSPs often change over time, requiring 528 resizing the LSP. Currently the head-end node performs this function 529 by monitoring the actual bandwidth usage, triggering a recomputation 530 and resignaling when a threshold is reached. This operation is 531 referred as auto-bandwidth adjustment. The head-end node either 532 recomputes the path locally, or it requests a recomputation from a 533 PCE by sending a PCReq message. In the latter case, the PCE computes 534 a new path and provides the new route suggestion. Upon receiving the 535 reply from the PCE, the PCC re-signals the LSP in Shared-Explicit 536 (SE) mode along the newly computed path. If a passive stateful PCE 537 is used, only the new bandwidth information is needed to trigger a 538 path re-computation since the LSP information is already known to the 539 PCE. Note that in this scenario, the head-end node is the one that 540 drives the LSP resizing based on local information, and that the 541 difference between using a stateless and a passive stateful PCE is in 542 the level of optimization of the LSP placement as discussed in the 543 previous section. 545 A more interesting smart bandwidth adjustment case is one where the 546 LSP resizing decision is done by an external entity, with access to 547 additional information such as historical trending data, application- 548 specific information about expected demands or policy information, as 549 well as knowledge of the actual desired flow volumes. In this case 550 an active stateful PCE provides an advantage in both the computation 551 with knowledge of all LSPs in the domain and in the ability to 552 trigger bandwidth modification of the LSP. 554 5.3. Bandwidth Scheduling 556 Bandwidth scheduling allows network operators to reserve resources in 557 advance according to the agreements with their customers, and allow 558 them to transmit data with specified starting time and duration, for 559 example for a scheduled bulk data replication between data centers. 561 Traditionally, this can be supported by NMS operation through path 562 pre-establishment and activation on the agreed starting time. 563 However, this does not provide efficient network usage since the 564 established paths exclude the possibility of being used by other 565 services even when they are not used for undertaking any service. It 566 can also be accomplished through GMPLS protocol extensions by 567 carrying the related request information (e.g., starting time and 568 duration) across the network. Nevertheless, this method inevitably 569 increases the complexity of signaling and routing process. 571 A passive stateful PCE can support this application with better 572 efficiency since it can alleviate the burden of processing on network 573 elements. This requires the PCE to maintain the scheduled LSPs and 574 their associated resource usage, as well as the ability of head-ends 575 to trigger signaling for LSP setup/deletion at the correct time. 576 This approach requires coarse time synchronization between PCEs and 577 PCCs. If an active stateful PCE is available, the PCE can trigger 578 the setup/deletion of scheduled requests in a centralized manner, 579 without modification of existing head-end behaviors. 581 5.4. Recovery 583 The recovery use cases discussed in the following sections show how 584 leveraging a stateful PCE can simplify the computation of recovery 585 path(s). In particular, two characteristics of a stateful PCE are 586 used: 1) using information stored in the LSP-DB for determining 587 shared protection resources and 2) performing computations with 588 knowledge of all LSPs in a domain. 590 5.4.1. Protection 592 For protection purposes, a PCC may send a request to a PCE for 593 computing a set of paths for a given LSP. Alternatively, the PCC can 594 send multiple requests to the PCE, asking for working and backup LSPs 595 separately. Either way, the resources bound to backup paths can be 596 shared by different LSPs to improve the overall network efficiency, 597 such as m:n protection or pre-configured shared mesh recovery 598 techniques as specified in [RFC4427]. If resource sharing is 599 supported for LSP protection, the information relating to existing 600 LSPs is required to avoid allocation of shared protection resources 601 to two LSPs that might fail together and cause protection contention 602 issues. A stateless PCE can accommodate this use case by having the 603 PCC pass in this information as a constraint to the path computation 604 request. A stateful PCE can more easily accommodate this need using 605 the information stored in its LSP-DB. 607 +----+ 608 |PCE | 609 +----+ 611 +------+ +------+ +------+ 612 | N1 +----------+ N2 +----------+ N3 | 613 +--+---+ +---+--+ +---+--+ 614 | | | 615 | +---------+ | 616 | | | 617 | +--+---+ +------+ | 618 +-----+ N5 +----------+ N4 +-----+ 619 +------+ +------+ 621 Figure 3: Reference topology 3 623 For example, in the network depicted in Figure 3 , suppose there 624 exists LSP1 with working path LSP1_working following N1->N5 and with 625 backup path LSP1_backup following N1->N2->N5. A request arrives 626 asking for a working and backup path pair to be computed for LSP2, 627 for a request from N2 to N5. If the PCE decides LSP2_working follows 628 N2->N1->N5, then the backup path LSP2_backup should not use the same 629 protection resource with LSP1 since LSP2 shares part of its resource 630 (specifically N1->N5) with LSP1 (i.e., these two LSPs are in the same 631 shared risk group). Alternatively, there is no such constraint if 632 N2->N3->N4->N5 is chosen for LSP2_working. 634 If a stateless PCE is used, the head node N2 needs to be aware of the 635 existence of LSPs which share the route of LSP2_working and of the 636 details of their protection resources. N2 must pass this information 637 to the PCE as a constraint so as to request a path with SRLG 638 diversity. On the other hand, a stateful PCE can get the LSPs 639 information by itself and can achieve the goal of finding SRLG- 640 diversified protection paths for both LSPs. This is made possible by 641 comparing the LSP resource usage exploiting the LSP DB accessible by 642 the stateful PCE. 644 5.4.2. Restoration 646 In case of a link failure, such as fiber cut, multiple LSPs may fail 647 at the same time. Thus, the source nodes of the affected LSPs will 648 be informed of the failure by the nodes detecting the failure. These 649 source nodes will send requests to a PCE for rerouting. In order to 650 reuse the resource taken by an existing LSP, the source node can send 651 a PCReq message including the XRO object with F bit set, together 652 with RRO object, as specified in [RFC5521]. 654 If a stateless PCE is exploited, it might respond to the rerouting 655 requests separately if they arrive at different times. Thus, it 656 might result in sub-optimal resource usage. Even worse, it might 657 unnecessarily block some of the rerouting requests due to 658 insufficient resources for later-arrived rerouting messages. If a 659 stateful PCE is used to fulfill this task, it can re-compute the 660 affected LSPs concurrently while reusing part of the existing LSPs 661 resources when it is informed of the failed link identifier provided 662 by the first request. This is made possible since the stateful PCE 663 can check what other LSPs are affected by the failed link and their 664 route information by inspecting its LSP-DB. As a result, a better 665 performance, such as better resource usage, minimal probability of 666 blocking upcoming new rerouting requests sent as a result of the link 667 failure, can be achieved. 669 In order to further reduce the amount of LSP rerouting messages flow 670 in the network, the notification can be performed at the node(s) 671 which detect the link failure. For example, suppose there are two 672 LSPs in the network as shown in Figure 3: (i) LSP1: N1->N5->N4->N3; 673 (ii) LSP2: N2->N5->N4. They traverse the failed link between N5-N4. 674 When N4 detects the failure, it can send a notification message to a 675 stateful PCE. Note that the stateful PCE stores the path information 676 of the LSPs that are affected by the link failure, so it does not 677 need to acquire this information from N4. Moreover, it can make use 678 of the bandwidth resources occupied by the affected LSPs when 679 performing path recalculation. After N4 receives the new paths from 680 the PCE, it notifies the ingress nodes of the LSPs, i.e., N1 and N2, 681 and specifies the new paths which should be used as the rerouting 682 paths. To support this, it would require extensions to the existing 683 signaling protocols. 685 Alternatively, if the target is to avoid resource contention within 686 the time-window of high LSP requests, a stateful PCE can retain the 687 under-construction LSP resource usage information for a given time 688 and exclude it from being used for forthcoming LSPs request. In this 689 way, it can ensure that the resource will not be double-booked and 690 thus the issue of resource contention and computation crank-backs can 691 be resolved. 693 5.4.3. SRLG Diversity 695 An alternative way to achieve efficient resilience is to maintain 696 SRLG disjointness between LSPs, irrespective of whether these LSPs 697 share the source and destination nodes or not. This can be achieved 698 at provisioning time, if the routes of all the LSPs are requested 699 together, using a synchronized computation of the different LSPs with 700 SRLG disjointness constraint. If the LSPs need to be provisioned at 701 different times (more general, the routes are requested at different 702 times, e.g. in the case of a restoration), the PCC can specify, as 703 constraints to the path computation a set of Shared Risk Link Groups 704 (SRLGs) using the Explicit Route Object [RFC5521]. However, for the 705 latter to be effective, it is needed that the entity that requests 706 the route to the PCE maintains updated SRLG information of all the 707 LSPs to which it must maintain the disjointness. A stateless PCE can 708 compute an SRLG-disjoint path by inspecting the TED and precluding 709 the links with the same SRLG values specified in the PCReq message 710 sent by a PCC. 712 A stateful PCE maintains the updated SRLG information of the 713 established LSPs in a centralized manner. Therefore, the PCC can 714 specify as constraints to the path computation the SRLG disjointness 715 of a set of already established LSPs by only providing the LSP 716 identifiers. 718 5.5. Maintenance of Virtual Network Topology (VNT) 720 In Multi-Layer Networks (MLN), a Virtual Network Topology (VNT) 721 [RFC5212] consists of a set of one or more TE LSPs in the lower layer 722 which provides TE links to the upper layer. In [RFC5623], the PCE- 723 based architecture is proposed to support path computation in MLN 724 networks in order to achieve inter-layer TE. 726 The establishment/teardown of a TE link in VNT needs to take into 727 consideration the state of existing LSPs and/or new LSP request(s) in 728 the higher layer. As specified in [RFC5623], a VNT manager (VNTM) is 729 in charge of setting up connections in the lower layer to provide TE 730 links for upper layer. Hence, when a stateless PCE cannot find the 731 route for a request based on the upper layer topology information, it 732 needs to interact with the VNTM and rely on the VNTM to decide 733 whether to set up or remove a TE link or not. On the other hand, a 734 stateful PCE can make the decision of when and how to modify the VNT 735 either to accommodate new LSP requests or to re-optimize resource 736 usage across layers irrespective of the PCE models as described in 737 [RFC5623]. 739 5.6. LSP Re-optimization 741 In order to make efficient usage of network resources, it is 742 sometimes desirable to re-optimize one or more LSPs dynamically. In 743 the case of a stateless PCE, in order to optimize network resource 744 usage dynamically through online planning, a PCC must send a request 745 to the PCE together with detailed path/bandwidth information of the 746 LSPs that need to be concurrently optimized. This means the PCC must 747 be able to determine when and which LSPs should be optimized. In the 748 case of a stateful PCE, given the LSP state information in the LSP 749 database, the process of dynamic optimization of network resources 750 can be automated without requiring the PCC to supply LSP state 751 information or to trigger the request. Moreover, since a stateful 752 PCE can maintain information for all LSPs that are in the process of 753 being set up and since it may have the ability to control timing and 754 sequence of LSP setup/deletion, the optimization procedures can be 755 performed more intelligently and effectively. 757 A special case of LSP re-optimization is Global Concurrent 758 Optimization (GCO) [RFC5557]. Global control of LSP operation 759 sequence in [RFC5557] is predicated on the use of what is effectively 760 a stateful (or semi-stateful) NMS. The NMS can be either not local 761 to the switch, in which case another northbound interface is required 762 for LSP attribute changes, or local/collocated, in which case there 763 are significant issues with efficiency in resource usage. A stateful 764 PCE adds a few features that: 766 o Roll the NMS visibility into the PCE and remove the requirement 767 for an additional northbound interface 769 o Allow the PCE to determine when re-optimization is needed, with 770 which level (GCO or a more incremental optimization) 772 o Allow the PCE to determine which LSPs should be re-optimized 774 o Allow a PCE to control the sequence of events across multiple 775 PCCs, allowing for bulk (and truly global) optimization, LSP 776 shuffling etc. 778 5.7. Resource Defragmentation 780 In networks with link bundles, if LSPs are dynamically allocated and 781 released over time, the resource becomes fragmented. The overall 782 available resource on a (bundle) link might be sufficient for a new 783 LSP request, but if the available resource is not continuous, the 784 request is rejected. In order to perform the defragmentation 785 procedure, stateful PCEs cam be used, since global visibility of LSPs 786 in the network is required to accurately assess resources on the 787 LSPs, and perform de-fragmentation while ensuring a minimal 788 disruption of the network. This use case cannot be accommodated by a 789 stateless PCE since it does not possess the detailed information of 790 existing LSPs in the network. 792 A case of particular interest to GMPLS-based transport networks is 793 the frequency defragmentation in flexible grid. In Flexible grid 794 networks [I-D.ogrcetal-ccamp-flexi-grid-fwk], LSPs with different 795 slot widths (such as 12.5G, 25G etc.) can co-exist so as to 796 accommodate the services with different bandwidth requests. 797 Therefore, even if the overall spectrum can meet the service request, 798 it may not be usable if it is not contiguous. Thus, with the help of 799 existing LSP state information, stateful PCE can make the resource 800 grouped together to be usable. Moreover, stateful PCE can 801 proactively choose routes for upcoming path requests to reduce the 802 chance of spectrum fragmentation. 804 5.8. Impairment-Aware Routing and Wavelength Assignment (IA-RWA) 806 In WSONs [RFC6163], a wavelength-switched LSP traverses one or more 807 fiber links. The bit rates of the client signals carried by the 808 wavelength LSPs may be the same or different. Hence, a fiber link 809 may transmit a number of wavelength LSPs with equal or mixed bit rate 810 signals. For example, a fiber link may multiplex the wavelengths 811 with only 10G signals, mixed 10G and 40G signals, or mixed 40G and 812 100G signals. 814 IA-RWA in WSONs refers to the RWA process (i.e., lightpath 815 computation) that takes into account the optical layer/transmission 816 imperfections by considering as additional (i.e., physical layer) 817 constraints. To be more specific, linear and non-linear effects 818 associated with the optical network elements should be incorporated 819 into the route and wavelength assignment procedure. For example, the 820 physical imperfection can result in the interference of two adjacent 821 lightpaths. Thus, a guard band should be reserved between them to 822 alleviate these effects. The width of the guard band between two 823 adjacent wavelengths depends on their characteristics, such as 824 modulation formats and bit rates. Two adjacent wavelengths with 825 different characteristics (e.g., different bit rates) may need a 826 wider guard band and with same characteristics may need a narrower 827 guard band. For example, 50GHz spacing may be acceptable for two 828 adjacent wavelengths with 40G signals. But for two adjacent 829 wavelengths with different bit rates (e.g., 10G and 40G), a larger 830 spacing such as 300GHz spacing may be needed. Hence, the 831 characteristics (states) of the existing wavelength LSPs should be 832 considered for a new RWA request in WSON. 834 In summary, when stateful PCEs are used to perform the IA-RWA 835 procedure, they need to know the characteristics of the existing 836 wavelength LSPs. The impairment information relating to existing and 837 to-be-established LSPs can be obtained by nodes in WSON networks via 838 external configuration or other means such as monitoring or 839 estimation based on a vendor-specific impair model. However, WSON 840 related routing protocols, i.e., 841 [I-D.ietf-ccamp-wson-signal-compatibility-ospf] and 842 [I-D.ietf-ccamp-gmpls-general-constraints-ospf-te], only advertise 843 limited information (i.e., availability) of the existing wavelengths, 844 without defining the supported client bit rates. It will incur 845 substantial amount of control plane overhead if routing protocols are 846 extended to support dissemination of the new information relevant for 847 the IA-RWA process. In this scenario, stateful PCE(s) would be a 848 more appropriate mechanism to solve this problem. Stateful PCE(s) 849 can exploit impairment information of LSPs stored in LSP-DB to 850 provide accurate RWA calculation. 852 6. Security Considerations 854 This document does not introduce any new security considerations 855 beyond those discussed in [I-D.ietf-pce-stateful-pce]. 857 The following topics will be discussed in a future version of this 858 document: whether use of a stateful PCE makes the network more or 859 less secure, and security use cases if any. 861 7. Contributing Authors 863 The following people all contributed significantly to this document 864 and are listed below in alphabetical order: 866 Ramon Casellas 867 CTTC - Centre Tecnologic de Telecomunicacions de Catalunya 868 Av. Carl Friedrich Gauss n7 869 Castelldefels, Barcelona 08860 870 Spain 871 Email: ramon.casellas@cttc.es 873 Edward Crabbe 874 Google, Inc. 875 1600 Amphitheatre Parkway 876 Mountain View, CA 94043 877 US 878 Email: edc@google.com 880 Dhruv Dhody 881 Huawei Technology 882 Leela Palace 883 Bangalore, Karnataka 560008 884 INDIA 885 EMail: dhruvd@huawei.com 887 Oscar Gonzalez de Dios 888 Telefonica Investigacion y Desarrollo 889 Emilio Vargas 6 890 Madrid, 28045 891 Spain 892 Phone: +34 913374013 893 Email: ogondio@tid.es 895 Young Lee 896 Huawei 897 1700 Alma Drive, Suite 100 898 Plano, TX 75075 899 US 900 Phone: +1 972 509 5599 x2240 901 Fax: +1 469 229 5397 902 EMail: ylee@huawei.com 904 Jan Medved 905 Cisco Systems, Inc. 906 170 West Tasman Dr. 907 San Jose, CA 95134 908 US 909 Email: jmedved@cisco.com 911 Robert Varga 912 Pantheon Technologies LLC 913 Mlynske Nivy 56 914 Bratislava 821 05 915 Slovakia 916 Email: robert.varga@pantheon.sk 918 Fatai Zhang 919 Huawei Technologies 920 F3-5-B R&D Center, Huawei Base 921 Bantian, Longgang District 922 Shenzhen 518129 P.R.China 923 Phone: +86-755-28972912 924 Email: zhangfatai@huawei.com 926 Xiaobing Zi 927 Email: unknown 929 8. Acknowledgements 931 We would like to thank Cyril Margaria, Adrian Farrel and JP Vasseur 932 for the useful comments and discussions. 934 9. References 936 9.1. Normative References 938 [I-D.ietf-pce-stateful-pce] 939 Crabbe, E., Medved, J., Minei, I., and R. Varga, "PCEP 940 Extensions for Stateful PCE", 941 draft-ietf-pce-stateful-pce-04 (work in progress), 942 May 2013. 944 [RFC4655] Farrel, A., Vasseur, J., and J. Ash, "A Path Computation 945 Element (PCE)-Based Architecture", RFC 4655, August 2006. 947 [RFC5440] Vasseur, JP. and JL. Le Roux, "Path Computation Element 948 (PCE) Communication Protocol (PCEP)", RFC 5440, 949 March 2009. 951 9.2. Informative References 953 [I-D.crabbe-pce-stateful-pce-mpls-te] 954 Crabbe, E., Medved, J., Minei, I., and R. Varga, "Stateful 955 PCE extensions for MPLS-TE LSPs", 956 draft-crabbe-pce-stateful-pce-mpls-te-01 (work in 957 progress), May 2013. 959 [I-D.ietf-ccamp-gmpls-general-constraints-ospf-te] 960 Zhang, F., Lee, Y., Han, J., Bernstein, G., and Y. Xu, 961 "OSPF-TE Extensions for General Network Element 962 Constraints", 963 draft-ietf-ccamp-gmpls-general-constraints-ospf-te-04 964 (work in progress), July 2012. 966 [I-D.ietf-ccamp-wson-signal-compatibility-ospf] 967 Lee, Y. and G. Bernstein, "GMPLS OSPF Enhancement for 968 Signal and Network Element Compatibility for Wavelength 969 Switched Optical Networks", 970 draft-ietf-ccamp-wson-signal-compatibility-ospf-11 (work 971 in progress), February 2013. 973 [I-D.ietf-pce-gmpls-pcep-extensions] 974 Margaria, C., Dios, O., and F. Zhang, "PCEP extensions for 975 GMPLS", draft-ietf-pce-gmpls-pcep-extensions-07 (work in 976 progress), October 2012. 978 [I-D.ogrcetal-ccamp-flexi-grid-fwk] 979 Dios, O., Casellas, R., Zhang, F., Fu, X., Ceccarelli, D., 980 and I. Hussain, "Framework and Requirements for GMPLS 981 based control of Flexi-grid DWDM networks", 982 draft-ogrcetal-ccamp-flexi-grid-fwk-02 (work in progress), 983 February 2013. 985 [I-D.sivabalan-pce-disco-stateful] 986 Sivabalan, S., Medved, J., and X. Zhang, "IGP Extensions 987 for Stateful PCE Discovery", 988 draft-sivabalan-pce-disco-stateful-01 (work in progress), 989 April 2013. 991 [MPLS-PC] Chaieb, I., Le Roux, JL., and B. Cousin, "Improved MPLS-TE 992 LSP Path Computation using Preemption", Global 993 Information Infrastructure Symposium, July 2007. 995 [MXMN-TE] Danna, E., Mandal, S., and A. Singh, "Practical linear 996 programming algorithm for balancing the max-min fairness 997 and throughput objectives in traffic engineering", pre- 998 print, 2011. 1000 [NET-REC] Vasseur, JP., Pickavet, M., and P. Demeester, "Network 1001 Recovery: Protection and Restoration of Optical, SONET- 1002 SDH, IP, and MPLS", The Morgan Kaufmann Series in 1003 Networking, June 2004. 1005 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1006 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1007 Tunnels", RFC 3209, December 2001. 1009 [RFC4427] Mannie, E. and D. Papadimitriou, "Recovery (Protection and 1010 Restoration) Terminology for Generalized Multi-Protocol 1011 Label Switching (GMPLS)", RFC 4427, March 2006. 1013 [RFC4657] Ash, J. and J. Le Roux, "Path Computation Element (PCE) 1014 Communication Protocol Generic Requirements", RFC 4657, 1015 September 2006. 1017 [RFC5212] Shiomoto, K., Papadimitriou, D., Le Roux, JL., Vigoureux, 1018 M., and D. Brungard, "Requirements for GMPLS-Based Multi- 1019 Region and Multi-Layer Networks (MRN/MLN)", RFC 5212, 1020 July 2008. 1022 [RFC5394] Bryskin, I., Papadimitriou, D., Berger, L., and J. Ash, 1023 "Policy-Enabled Path Computation Framework", RFC 5394, 1024 December 2008. 1026 [RFC5521] Oki, E., Takeda, T., and A. Farrel, "Extensions to the 1027 Path Computation Element Communication Protocol (PCEP) for 1028 Route Exclusions", RFC 5521, April 2009. 1030 [RFC5557] Lee, Y., Le Roux, JL., King, D., and E. Oki, "Path 1031 Computation Element Communication Protocol (PCEP) 1032 Requirements and Protocol Extensions in Support of Global 1033 Concurrent Optimization", RFC 5557, July 2009. 1035 [RFC5623] Oki, E., Takeda, T., Le Roux, JL., and A. Farrel, 1036 "Framework for PCE-Based Inter-Layer MPLS and GMPLS 1037 Traffic Engineering", RFC 5623, September 2009. 1039 [RFC6163] Lee, Y., Bernstein, G., and W. Imajuku, "Framework for 1040 GMPLS and Path Computation Element (PCE) Control of 1041 Wavelength Switched Optical Networks (WSONs)", RFC 6163, 1042 April 2011. 1044 Appendix A. Editorial notes and open issues 1046 This section will be removed prior to publication. 1048 The following open issues remain: 1050 Use cases from draft-ietf-pce-stateful-pce To avoid loss of 1051 information, the use cases will be removed from 1052 [I-D.ietf-pce-stateful-pce] only after this document becomes a 1053 working group document. 1055 This document WILL NOT repeat terminology defined in other documents 1056 or attempt to place any additional requirements on stateful PCE. 1058 Authors' Addresses 1060 Xian Zhang (editor) 1061 Huawei Technologies 1062 F3-5-B R&D Center, Huawei Base Bantian, Longgang District 1063 Shenzhen, Guangdong 518129 1064 P.R.China 1066 Email: zhang.xian@huawei.com 1067 Ina Minei (editor) 1068 Juniper Networks, Inc. 1069 1194 N. Mathilda Ave. 1070 Sunnyvale, CA 94089 1071 US 1073 Email: ina@juniper.net