Optimizations of Label Switched Path State Synchronization Procedures for a Stateful PCEedward.crabbe@gmail.comGoogle, Inc.1600 Amphitheatre ParkwayMountain ViewCA94043USinaminei@google.comCisco Systems, Inc.170 West Tasman Dr.San JoseCA95134USjmedved@cisco.comPantheon Technologies SROMlynske Nivy 56Bratislava821 05Slovakiarobert.varga@pantheon.skHuawei TechnologiesF3-5-B R&D Center, Huawei Industrial Base, Bantian, Longgang District
ShenzhenGuangdong518129P.R.Chinazhang.xian@huawei.comHuawei TechnologiesDivyashree Techno Park, WhitefieldBangaloreKarnataka560037Indiadhruv.ietf@gmail.comPCE Working Group A stateful Path Computation Element (PCE) has access to not only the
information disseminated by the network's Interior Gateway Protocol (IGP), but also
the set of active paths and their reserved resources for its computation.
The additional Label Switched Path (LSP) state information allows
the PCE to compute constrained
paths while considering individual LSPs and their interactions.
This requires a reliable state synchronization mechanism between the
PCE and the network, PCE and path computation clients (PCCs), and between cooperating PCEs.
The basic mechanism for state synchronization is part of the stateful PCE specification.
This draft presents motivations for optimizations to the base state synchronization procedure
and specifies the required Path Computation Element Communication Protocol (PCEP) extensions.The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in .
The Path Computation Element Communication Protocol (PCEP) provides
mechanisms for Path Computation Elements (PCEs) to perform path
computations in response to Path Computation Clients (PCCs) requests. describes a set of extensions to PCEP to
provide stateful control. A stateful PCE has access to not only the
information carried by the network's Interior Gateway Protocol (IGP),
but also the set of active paths and their reserved resources for its
computations. The additional state allows the PCE to compute constrained
paths while considering individual LSPs and their interactions.
This requires a reliable state synchronization mechanism between the
PCE and the network, PCE and PCC, and between cooperating PCEs.
describes the basic mechanism
for state synchronization. This draft specifies following optimizations for state synchronization
and the corresponding PCEP procedures and extensions:
State Synchronization Avoidance: To skip state synchronization if the state has survived and not changed during session restart. (See .)Incremental State Synchronization: To do incremental (delta) state synchronization when possible. (See .)PCE-triggered Initial Synchronization: To let PCE control the timing of the initial state synchronization. (See .)PCE-triggered Re-synchronization: To let PCE re-synchronize the state for sanity check. (See .)This document uses the following terms defined in : PCC, PCE, PCEP Peer.This document uses the following terms defined in :
Delegation, Redelegation Timeout Interval, LSP State Report, LSP Update Request, LSP State Database.Within this document, when describing PCE-PCE communications, the
requesting PCE fills the role of a PCC. This provides a saving in
documentation without loss of function.The purpose of state synchronization is to provide a
checkpoint-in-time state replica of a PCC's LSP state in a stateful PCE. State
synchronization is performed immediately after the initialization phase
(). describes
the basic mechanism for state synchronization. State synchronization is not always necessary following a PCEP session restart. If the
state of both PCEP peers did not change, the synchronization phase may be skipped. This
can result in significant savings in both control-plane data exchanges and the time it
takes for the stateful PCE to become fully operational. State synchronization MAY be skipped following a PCEP session
restart if the state of both PCEP peers did not change during the
period prior to session re-initialization. To be able to make this
determination, state must be exchanged and maintained by both PCE and
PCC during normal operation. This is accomplished by keeping track of
the changes to the LSP state database, using a version tracking field called
the LSP State Database Version Number. The LSP State Database Version Number, carried in LSP-DB-VERSION TLV
(see ), is owned by a PCC and it MUST be incremented
by 1 for each successive change in the PCC's LSP state database. The LSP State
Database Version Number MUST start at 1 and may wrap around. Values 0 and
0xFFFFFFFFFFFFFFFF are reserved. If either of the two values are used during LSP state
(re)-synchronization, the PCE speaker receiving this node should send back a PCErr with
Error-type 20 Error-value TBD (suggested value - 6) 'Received an invalid LSP DB Version Number', and close
the PCEP session. Operations that trigger a change to the local
LSP state database include a change in the LSP operational state, delegation of an LSP,
removal or setup of an LSP or change in any of the LSP attributes
that would trigger a report to the PCE. State synchronization avoidance is advertised on a PCEP session
during session startup using the INCLUDE-DB-VERSION (S) bit in the
capabilities TLV (see ). The peer may
move in the network, either physically or logically, which may cause
its connectivity details and transport-level identity (such as IP
address) to change. To ensure that a PCEP peer can recognize a
previously connected peer even in face of such mobility, each PCEP
peer includes the SPEAKER-ENTITY-ID TLV described in
in the OPEN message.If both PCEP speakers set the S flag in the OPEN
object's STATEFUL-PCE-CAPABILITY TLV to 1, the PCC MUST include the
LSP-DB-VERSION TLV in each LSP object of the PCRpt message. If the LSP-DB-VERSION
TLV is missing in a PCRpt message, the PCE will generate an error with Error-Type 6
(mandatory object missing) and Error-Value TBD (suggested value - 12) 'LSP-DB-VERSION TLV missing' and
close the session. If state synchronization avoidance has not been enabled
on a PCEP session, the PCC SHOULD NOT include the LSP-DB-VERSION TLV in
the LSP Object and the PCE SHOULD ignore it were to receive one. If a PCE's LSP state database survived the restart of a PCEP
session, the PCE will include the LSP-DB-VERSION TLV in its OPEN
object, and the TLV will contain the last LSP State Database Version Number
received on an LSP State Report from the PCC in the previous PCEP
session. If a PCC's LSP State Database survived the restart of a PCEP
session, the PCC will include the LSP-DB-VERSION TLV in its OPEN object
and the TLV will contain the latest LSP State Database Version Number.
If a PCEP speaker's LSP state database did not survive the restart of
a PCEP session, the PCEP speaker MUST NOT include the LSP-DB-VERSION
TLV in the OPEN object.If both PCEP speakers include the LSP-DB-VERSION TLV in the OPEN
Object and the TLV values match, the PCC MAY skip state
synchronization. Otherwise, the PCC MUST perform full state
synchronization (see )
or incremental state synchronization (see ) to
the stateful PCE. If the PCC attempts to skip state synchronization
(i.e., the SYNC Flag = 0 on the first LSP State Report from the PCC
as per ),
the PCE MUST send back a PCErr with Error-Type 20 Error-Value TBD (suggested value - 2) 'LSP
Database version mismatch', and close the PCEP session. If state synchronization is required, then prior to completing the
initialization phase, the PCE MUST mark any LSPs in the LSP database
that were previously reported by the PCC as stale. When the PCC
reports an LSP during state synchronization, if the LSP already exists
in the LSP database, the PCE MUST update the LSP database and clear
the stale marker from the LSP. When it has finished state
synchronization, the PCC MUST immediately send an end of synchronization
marker. The end of synchronization marker is a Path Computation State Report (PCRpt) message
with an LSP object containing a PLSP-ID of 0 and with the SYNC flag set to 0
(). The LSP-DB-VERSION TLV
MUST be included in this PCRpt message. On receiving this state report, the PCE MUST
purge any LSPs from the LSP database that are still marked as stale.Note that a PCE/PCC MAY force state synchronization by not including
the LSP-DB-VERSION TLV in its OPEN object.Since a PCE does not make changes to the LSP State Database Version Number,
a PCC should never encounter this TLV in a message from the PCE (other than the OPEN message).
A PCC SHOULD ignore the LSP-DB-VERSION TLV, were it to receive one from a PCE. If state synchronization avoidance is enabled, a PCC MUST increment
its LSP State Database Version Number when the 'Redelegation Timeout
Interval' timer expires (see )
for the use of the Redelegation Timeout Interval). shows an example
sequence where the state synchronization is skipped. shows an example sequence
where the state synchronization is performed due to LSP state database
version mismatch during the PCEP session setup. Note that the same
state synchronization sequence would happen if either the PCC or the
PCE would not include the LSP-DB-VERSION TLV in their respective Open
messages. shows an example
sequence where the state synchronization is skipped, but because one or
both PCEP speakers set the S Flag to 0, the PCC does
not send LSP-DB-VERSION TLVs in subsequent PCRpt messages to the PCE. If the
current PCEP session restarts, the PCEP speakers will have to perform state
synchronization, since the PCE does not know the PCC's latest LSP
State Database Version Number information. A new INCLUDE-DB-VERSION (S) bit is added in the stateful
capabilities TLV (see for details). The LSP State Database Version Number (LSP-DB-VERSION) TLV is an optional TLV
that MAY be included in the OPEN object and the LSP object. The format of the LSP-DB-VERSION TLV is shown in the following
figure:The type of the TLV is [TBD] and it has a fixed length of 8 octets.
The value contains a 64-bit unsigned integer, representing the LSP State
DB Version Number.The Speaker Entity Identifier TLV (SPEAKER-ENTITY-ID) is an optional TLV that
MAY be included in the OPEN Object when a PCEP speaker wishes to determine if state
synchronization can be skipped when a PCEP session is restarted. It
contains a unique identifier for the node that does not change during
the lifetime of the PCEP speaker. It identifies the PCEP speaker to
its peers even if the speaker's IP address is changed.In case of a remote peer IP address change, a PCEP speaker would
learn the speaker entity identifier on receiving the open message but
it MAY have already sent its open message without realizing that it is
a known PCEP peer. In such a case, either a full synchronization is
done or PCEP session is terminated. This may be a local policy decision.
The new IP address is
associated with the speaker entity identifier for future either way. In
the latter case when PCEP session is re-established, it would be
correctly associated with speaker entity identifier and not be considered
as an unknown peer. The format of the SPEAKER-ENTITY-ID TLV is shown in the
following figure:The type of the TLV is [TBD] and it has a variable length, which
MUST be greater than 0. The Value is padded to 4-octet alignment. The padding
is not included in the Length field. The value contains the entity identifier of
the speaker transmitting this TLV. This identifier is required to be
unique within its scope of visibility, which is usually limited to a single
domain. It MAY be configured by the operator. Alternatively, it can be
derived automatically from a suitably-stable unique identifier, such as
a MAC address, serial number, Traffic Engineering Router ID, or similar.
In the case of inter-domain connections, the speaker SHOULD prefix its
usual identifier with the domain identifier of its residence, such as
Autonomous System number, IGP area identifier, or similar.The relationship between this identifier and entities in the Traffic
Engineering database is intentionally left undefined.From a manageability point of view, a PCE or PCC implementation SHOULD
allow the operator to configure this Speaker Entity Identifier. describes the LSP state
synchronization mechanism between PCCs and stateful PCEs. During the state
synchronization, a PCC sends the information of all its LSPs (i.e., the full LSP-DB) to the stateful PCE.
In order to reduce the state synchronization overhead when there is a small number of LSP state change
in the network between PCEP session restart, this section defines a mechanism for incremental (Delta) LSP Database
(LSP-DB) synchronization.According to , if a PCE restarts
and its LSP-DB survived, PCCs with mismatched LSP State Database Version Number
will send all their LSPs information (full LSP-DB) to the stateful PCE, even if
only a small number of LSPs underwent state change. It can take a long time and consume
large communication channel bandwidth. shows an example of LSP state synchronization. Assuming there are 320 LSPs in the network, with each PCC having 80
LSPs. During the time when the PCEP session is down, 20 LSPs of each
PCC (i.e., 80 LSPs in total), are changed. Hence when PCEP session
restarts, the stateful PCE needs to synchronize 320 LSPs with all
PCCs. But actually, 240 LSPs stay the same. If performing full LSP
state synchronization, it can take a long time to carry out the
synchronization of all LSPs. It is especially true when only a low
bandwidth communication channel is available and there is a
substantial number of LSPs in the network. Another disadvantage of
full LSP synchronization is that it is a waste of communication
bandwidth to perform full LSP synchronization given the fact that
the number of LSP changes can be small during the time when PCEP
session is down. An incremental (Delta) LSP Database (LSP-DB) state synchronization
is described in this section, where only the LSPs underwent state
change are synchronized between the session restart. This may
include new/modified/deleted LSPs. PCEP extensions for stateful PCEs to perform LSP synchronization
SHOULD allow: incremental LSP state synchronization between session restarts.
Note this does not exclude the need for a stateful PCE to request a full LSP DB synchronization.
describes state
synchronization and describes state
synchronization avoidance by using LSP-DB-VERSION TLV in its OPEN
object. This section extends this idea to only synchronize the
delta (changes) in case of version mismatch. If both PCEP speakers include the LSP-DB-VERSION TLV in the OPEN
object and the LSP-DB-VERSION TLV values match, the PCC MAY skip state synchronization. Otherwise,
the PCC MUST perform state synchronization.
Incremental State synchronization capability is advertised on a PCEP session
during session startup using the DELTA-LSP-SYNC-CAPABILITY (D) bit in the
capabilities TLV (see ).
Instead of dumping full LSP-DB to the stateful PCE
again, the PCC synchronizes the delta (changes) as described in
when D flag and S flag is set to 1 by both PCC and PCE.
Other combinations of D and S flags setting by PCC and PCE result in full LSP-DB synchronization
procedure as described in .
If a PCC has
to force full LSP DB synchronization due to reasons including but not limited: (1) local policy
configured at the PCC; (2) no sufficient LSP state caches for incremental update, the
PCC can set the D flag to 0. Note a PCC may have to bring down the current session and
force a full LSP-DB synchronization with D flag set to 0 in the subsequent open message.
As per , the LSP State Database Version Number is
incremented each time a change is made to the PCC's local LSP State
Database. Each LSP is associated with the DB version at the time of
its state change. This is needed to determine which LSP and what
information needs to be synchronized in incremental state synchronization. It is not necessary for a PCC to store a complete history of LSP Database
change, but rather remember the LSP state changes (including LSP
modification, setup and deletion) that happened between the PCEP session(s) restart in order to carry out
incremental state synchronization. After the synchronization procedure finishes,
the PCC can dump this history information. In the example shown in , the PCC
needs to store the LSP state changes that happened between DB Version 43 to 46 and synchronizes
these changes only when performing incremental LSP state update. So a PCC needs to remember
the LSP state changes that happened when an existing PCEP session to a stateful PCE goes down
in the hope of doing incremental synchronisation when the session is re-established. If a PCC finds out it does not have sufficient information to complete incremental
synchronisation after advertising incremental LSP state synchronization capability, it MUST send
a PCErr with Error-Type 20 and Error-Value 5 'A PCC indicates to a PCE that it can not
complete the state synchronization' (defined in ) and
terminate the session.The other procedures and error checks remain unchanged from the
full state synchronization (). In networks such as optical transport networks, the control channel between network nodes can be
realized through in-band overhead thus has limited bandwidth. With a stateful PCE connected to the network
via one network node, it is desirable to control the timing of PCC state synchronization so as not to overload
the low communication channel available in the network during the initial synchronization (be it incremental or full)
when the session restarts , when there is comparatively large amount of control information needing to be
synchronized between the stateful PCE and the network. The method proposed, i.e., allowing PCE to trigger
the state synchronization, is similar to the function proposed in
but is used in different scenarios and for different purposes. Support of PCE-triggered state synchronization is advertised during session
startup using the TRIGGERED-INITIAL-SYNC (F) bit in the STATEFUL-PCE-CAPABILITY TLV
(see ). If the TRIGGERED-INITIAL-SYNC capability is not advertised and the PCC
receives a PCUpd with the SYNC flag set to 1, it MUST send a PCErr with the SRP-ID-number
of the PCUpd, Error-Type 20 and Error-Value TBD (suggested value - 4) 'Attempt to trigger synchronization when the
TRIGGERED-SYNC capability has not been advertised' (see ). A stateful PCE MAY choose to control the LSP-DB synchronization process. To allow PCE to do so,
PCEP speakers MUST set T bit to 1 to indicate this (as described in ).
If the LSP-DB Version is mis-matched, it can send a PCUpd message with PLSP-ID = 0 and SYNC = 1
in order to trigger the LSP-DB synchronization process. In this way, the PCE can control the sequence of
LSP synchronization among all the PCCs that are re-establishing PCEP sessions with it. When the capability of PCE
control is enabled, only after a PCC receives this message, it will start sending information to the PCE. The PCC
SHOULD NOT send PCRpt messages to the stateful PCE before it triggers the State Synchronization. This PCE-triggering
capability can be applied to both full and incremental state synchronization. If applied to
the later, the PCCs only send information that PCE does not possess, which is inferred from the LSP-DB version
information exchanged in the OPEN message (see for detailed
procedure). Once the initial state synchronization is triggered by the PCE,
the procedures and error checks remain unchanged from the full
state synchronization (). The accuracy of the computations performed by the PCE is
tied to the accuracy of the view the PCE has on the state of the LSPs.
Therefore, it can be beneficial to be able to re-synchronize this state even after
the session has been established. The PCE may use this approach to
continuously sanity check its state against the network, or to recover
from error conditions without having to tear down sessions. Support of PCE-triggered state synchronization is advertised during session
startup using the TRIGGERED-RESYNC (T) bit in the STATEFUL-PCE-CAPABILITY TLV
(see ). The PCE can choose to re-synchronize its entire
LSP database or a single LSP. To trigger re-synchronization for an LSP, the PCE MUST first mark
the LSP as stale and then send a Path Computation State Update (PCUpd) for it,
with the SYNC flag in the LSP object set to 1. The PCE SHOULD NOT include any
parameter updates for the LSP, and the PCC
SHOULD ignore such updates if the SYNC flag is set. The PCC MUST respond with a
PCRpt message with the LSP state, SYNC Flag set to 0 and MUST include the
SRP-ID-number of the PCUpd message that triggered the resynchronization. The PCE can also trigger re-synchronization of the entire LSP database.
The PCE MUST first mark all LSPs in the LSP database that were previously reported
by the PCC as stale and then send a PCUpd with an LSP object containing a PLSP-ID
of 0 and with the SYNC flag set to 1. This PCUpd message is the
trigger for the PCC to enter the synchronization phase as described in
and start sending PCRpt
messages. After the receipt of the end-of-synchronization marker, the
PCE will purge LSPs which were not refreshed. The SRP-ID-number of the PCUpd that
triggered the re-synchronization SHOULD be included in each of the PCRpt messages. If the TRIGGERED-RESYNC capability is not advertised and the PCC
receives a PCUpd with the SYNC flag set to 1, it MUST send a PCErr with the SRP-ID-number
of the PCUpd, Error-Type 20 and Error-Value TBD (suggested value - 4) 'Attempt to trigger synchronization when the
TRIGGERED-SYNC capability has not been advertised' (see ). Once the state re-synchronization is triggered by the PCE,
the procedures and error checks remain unchanged from the full
state synchronization ().
This would also include PCE triggering multiple state re-synchronization requests
while synchronization is in progress. Support for each of the optimizations described in this document requires
advertising the corresponding capabilities during session establishment time. New flags are defined for the STATEFUL-PCE-CAPABILITY TLV defined in
. Its format is
shown in the following figure:The value comprises a single field - Flags (32 bits):
defined in
.if set to 1 by both
PCEP Speakers, the PCC will include the LSP-DB-VERSION TLV in each
LSP Object. See for
details.defined in
.if set to 1 by both
PCEP Speakers, the PCE can trigger re-synchronization of LSPs at
any point in the life of the session. See for details.if set to 1 by a PCEP
speaker, it indicates that the PCEP speaker allows incremental (delta) state
synchronization. See for details.if set to 1 by both
PCEP Speakers, the PCE SHOULD trigger initial (first) state synchronization.
See for details.This document requests IANA actions to allocate code points for the
protocol elements defined in this document. IANA is requested to make the following allocation in the "PCEP-ERROR
Object Error Types and Values" registry.This document defines the following new PCEP TLVs:ValueMeaning Reference TBD(suggested value 23) LSP-DB-VERSIONThis documentTBD(suggested value 24) SPEAKER-ENTITY-IDThis documentThe following values are defined in this document for the Flags field in the
STATEFUL-PCE-CAPABILITY-TLV in the OPEN object:BitDescription Reference TBD(suggested value 26)TRIGGERED-INITIAL-SYNCThis documentTBD(suggested value 27)DELTA-LSP-SYNC-CAPABILITYThis documentTBD(suggested value 28)TRIGGERED-RESYNCThis documentTBD(suggested value 30)INCLUDE-DB-VERSIONThis documentAll manageability requirements and considerations listed in
and
apply to PCEP protocol extensions defined in this document. In
addition, requirements and considerations listed in this section
apply.A PCE or PCC implementation MUST allow configuring the
state synchronization optimization capabilities as described in
this document. The implementation SHOULD also allow the
operator to configure the Speaker Entity Identifier ().The PCEP MIB module
SHOULD be extended to include advertised stateful capabilities,
and synchronization status.Mechanisms defined in this document do not imply any new liveness detection
and monitoring requirements in addition to those already listed in
.Mechanisms defined in this document do not imply any new operation
verification requirements in addition to those already listed in
and
.Mechanisms defined in this document do not imply any new requirements
on other protocols.Mechanisms defined in this document do not have any impact on
network operations in addition to those already listed in
and
.The security considerations listed in
apply to this document as well. However, because the protocol modifications
outlined in this document allow the PCE to control state (re)-synchronization
timing and sequence, it
also introduces a new attack vector: an attacker may flood the PCC
with triggered re-synchronization request at a rate which exceeds the PCC's ability
to process them, either by spoofing messages or by
compromising the PCE itself. The PCC is free to drop any trigger
re-synchronization request without additional processing.We would like to thank Young Lee, Jonathan Hardwick, Sergio Belotti and Cyril Margaria for their
comments and discussions.
Gang Xie
Huawei Technologies
F3-5-B R&D Center, Huawei Industrial Base, Bantian, Longgang District
Shenzhen, Guangdong, 518129
P.R. China
Email: xiegang09@huawei.com