< draft-ietf-bess-evpn-ipvpn-interworking-02.txt   draft-ietf-bess-evpn-ipvpn-interworking-03.txt >
BESS Workgroup J. Rabadan, Ed. BESS Workgroup J. Rabadan, Ed.
Internet Draft Nokia Internet-Draft Nokia
Intended status: Standards Track A. Sajassi, Ed. Intended status: Standards Track A. Sajassi, Ed.
Cisco Expires: November 26, 2020 Cisco
E. Rosen E. Rosen
Individual Individual
J. Drake J. Drake
W. Lin W. Lin
Juniper Juniper
J. Uttaro J. Uttaro
AT&T AT&T
A. Simpson A. Simpson
Nokia Nokia
May 25, 2020
Expires: June 1, 2020 November 29, 2019
EVPN Interworking with IPVPN EVPN Interworking with IPVPN
draft-ietf-bess-evpn-ipvpn-interworking-02 draft-ietf-bess-evpn-ipvpn-interworking-03
Abstract Abstract
EVPN is used as a unified control plane for tenant network intra and EVPN is used as a unified control plane for tenant network intra and
inter-subnet forwarding. When a tenant network spans not only EVPN inter-subnet forwarding. When a tenant network spans not only EVPN
domains but also domains where IPVPN provides inter-subnet domains but also domains where IPVPN provides inter-subnet
forwarding, there is a need to specify the interworking aspects forwarding, there is a need to specify the interworking aspects
between both EVPN and IPVPN domains, so that the end to end tenant between both EVPN and IPVPN domains, so that the end to end tenant
connectivity can be accomplished. This document specifies how EVPN connectivity can be accomplished. This document specifies how EVPN
should interwork with VPN-IPv4/VPN-IPv6 and IPv4/IPv6 BGP families should interwork with VPN-IPv4/VPN-IPv6 and IPv4/IPv6 BGP families
for inter-subnet forwarding. for inter-subnet forwarding.
Status of this Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF). Note that other groups may also distribute
other groups may also distribute working documents as Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at This Internet-Draft will expire on November 26, 2020.
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
This Internet-Draft will expire on June 1, 2020.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction and Problem Statement . . . . . . . . . . . . . . 3 1. Introduction and Problem Statement . . . . . . . . . . . . . 2
2. Terminology and Interworking PE Components . . . . . . . . . . 3 2. Terminology and Interworking PE Components . . . . . . . . . 3
3. Domain Path Attribute (D-PATH) . . . . . . . . . . . . . . . . 9 3. Domain Path Attribute (D-PATH) . . . . . . . . . . . . . . . 9
3.1. D-PATH and Loop Prevention . . . . . . . . . . . . . . . . 11 3.1. D-PATH and Loop Prevention . . . . . . . . . . . . . . . 11
4. BGP Path Attribute Propagation across ISF SAFIs . . . . . . . . 12 4. BGP Path Attribute Propagation across ISF SAFIs . . . . . . . 12
4.1. No-Propagation-Mode . . . . . . . . . . . . . . . . . . . . 12 4.1. No-Propagation-Mode . . . . . . . . . . . . . . . . . . . 12
4.2. Uniform-Propagation-Mode . . . . . . . . . . . . . . . . . 12 4.2. Uniform-Propagation-Mode . . . . . . . . . . . . . . . . 12
4.3. Aggregation of Routes and Path Attribute Propagation . . . 13 4.3. Aggregation of Routes and Path Attribute Propagation . . 13
5. Route Selection Process between EVPN and other ISF SAFIs . . . 14 5. Route Selection Process between EVPN and other ISF SAFIs . . 14
6. Composite PE Procedures . . . . . . . . . . . . . . . . . . . . 15 6. Composite PE Procedures . . . . . . . . . . . . . . . . . . . 15
7. Gateway PE Procedures . . . . . . . . . . . . . . . . . . . . . 17 7. Gateway PE Procedures . . . . . . . . . . . . . . . . . . . . 17
8. Interworking Use-Cases . . . . . . . . . . . . . . . . . . . . 19 8. Interworking Use-Cases . . . . . . . . . . . . . . . . . . . 19
9. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 21 9. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 21
10. Conventions used in this document . . . . . . . . . . . . . . 21 10. Conventions used in this document . . . . . . . . . . . . . . 21
11. Security Considerations . . . . . . . . . . . . . . . . . . . 21 11. Security Considerations . . . . . . . . . . . . . . . . . . . 21
12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21
13. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22 13. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 22
13.1. Normative References . . . . . . . . . . . . . . . . . . . 22 14. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 22
13.2. Informative References . . . . . . . . . . . . . . . . . . 22 15. References . . . . . . . . . . . . . . . . . . . . . . . . . 22
14. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 23 15.1. Normative References . . . . . . . . . . . . . . . . . . 22
15. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 23 15.2. Informative References . . . . . . . . . . . . . . . . . 22
16. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 23 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 23
1. Introduction and Problem Statement 1. Introduction and Problem Statement
EVPN is used as a unified control plane for tenant network intra and EVPN is used as a unified control plane for tenant network intra and
inter-subnet forwarding. When a tenant network spans not only EVPN inter-subnet forwarding. When a tenant network spans not only EVPN
domains but also domains where IPVPN provides inter-subnet domains but also domains where IPVPN provides inter-subnet
forwarding, there is a need to specify the interworking aspects forwarding, there is a need to specify the interworking aspects
between both EVPN and IPVPN domains, so that the end to end tenant between both EVPN and IPVPN domains, so that the end to end tenant
connectivity can be accomplished. This document specifies how EVPN connectivity can be accomplished. This document specifies how EVPN
should interwork with VPN-IPv4/VPN-IPv6 and IPv4/IPv6 BGP families should interwork with VPN-IPv4/VPN-IPv6 and IPv4/IPv6 BGP families
for inter-subnet forwarding. for inter-subnet forwarding.
EVPN supports the advertisement of IPv4 or IPv6 prefixes in two EVPN supports the advertisement of IPv4 or IPv6 prefixes in two
different route types: different route types:
o Route Type 2 - MAC/IP route (only for /32 and /128 host routes), as o Route Type 2 - MAC/IP route (only for /32 and /128 host routes),
described by [INTER-SUBNET]. as described by [I-D.ietf-bess-evpn-inter-subnet-forwarding].
o Route Type 5 - IP Prefix route, as described by [IP-PREFIX]. o Route Type 5 - IP Prefix route, as described by
[I-D.ietf-bess-evpn-prefix-advertisement].
When interworking with other BGP address families (AFIs/SAFIs) for When interworking with other BGP address families (AFIs/SAFIs) for
inter-subnet forwarding, the IP prefixes in those two EVPN route inter-subnet forwarding, the IP prefixes in those two EVPN route
types must be propagated to other domains using different SAFIs. Some types must be propagated to other domains using different SAFIs.
aspects of that propagation must be clarified. Examples of these Some aspects of that propagation must be clarified. Examples of
aspects or procedures across BGP families are: route selection, loop these aspects or procedures across BGP families are: route selection,
prevention or BGP Path attribute propagation. The Interworking PE loop prevention or BGP Path attribute propagation. The Interworking
concepts are defined in section 2, and the rest of the document PE concepts are defined in section 2, and the rest of the document
describes the interaction between Interworking PEs and other PEs for describes the interaction between Interworking PEs and other PEs for
end-to-end inter-subnet forwarding. end-to-end inter-subnet forwarding.
2. Terminology and Interworking PE Components 2. Terminology and Interworking PE Components
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
In addition, this section summarizes the terminology related to the This section summarizes the terminology related to the "Interworking
"Interworking PE" concept that will be used throughout the rest of PE" concept that will be used throughout the rest of the document.
the document.
+-------------------------------------------------------------+ +-------------------------------------------------------------+
| | | |
| +------------------+ Interworking PE | | +------------------+ Interworking PE |
| Attachment | +------------------+ | | Attachment | +------------------+ |
| Circuit(AC1) | | +----------+ | MPLS/NVO tnl | Circuit(AC1) | | +----------+ | MPLS/NVO tnl
----------------------*Bridge | | +------ ----------------------*Bridge | | +------
| | | |Table(BT1)| | +-----------+ / \ \ | | | |Table(BT1)| | +-----------+ / \ \
MPLS/NVO tnl +-------->| *---------* |<--> | Eth | MPLS/NVO tnl +-------->| *---------* |<--> | Eth |
-------+ | | | |Eth-Tag x + |IRB1| | \ / / -------+ | | | |Eth-Tag x + |IRB1| | \ / /
skipping to change at page 4, line 33 skipping to change at page 4, line 31
| AC2 | | +----------+ | AC3| +------ | AC2 | | +----------+ | AC3| +------
| | | MAC-VRF1 | | | | | | MAC-VRF1 | | |
| +-+ RD1/RT1 | | | | +-+ RD1/RT1 | | |
| +------------------+ | SAFIs | | +------------------+ | SAFIs |
| | 1 +---+ | | | 1 +---+ |
-------------------------------------------------+ 128 |BGP| | -------------------------------------------------+ 128 |BGP| |
| EVPN +---+ | | EVPN +---+ |
| | | |
+-------------------------------------------------------------+ +-------------------------------------------------------------+
Figure 1 EVPN-IPVPN Interworking PE Figure 1: EVPN-IPVPN Interworking PE
o ISF SAFI: Inter-Subnet Forwarding (ISF) SAFI is a MP-BGP Sub-
Address Family that advertises reachability for IP prefixes and can
be used for inter-subnet forwarding within a given tenant network.
The ISF SAFIs are 1 (including IPv4 and IPv6 AFIs), 128 (including
IPv4 and IPv6 AFIs) and 70 (EVPN, including only AFI 25).
o ISF route: a route for a given prefix whose ISF SAFI may change as o ISF SAFI: Inter-Subnet Forwarding (ISF) SAFI is a MP-BGP Sub-
it transits different domains. Address Family that advertises reachability for IP prefixes and
can be used for inter-subnet forwarding within a given tenant
network. The ISF SAFIs are 1 (including IPv4 and IPv6 AFIs), 128
(including IPv4 and IPv6 AFIs) and 70 (EVPN, including only AFI
25).
o IP-VRF: an IP Virtual Routing and Forwarding table, as defined in o ISF route: a route for a given prefix whose ISF SAFI may change as
[RFC4364]. It is also the instantiation of an IPVPN in a PE. Route it transits different domains.
Distinguisher and Route Target(s) are required properties of an IP-
VRF.
o MAC-VRF: a MAC Virtual Routing and Forwarding table, as defined in o IP-VRF: an IP Virtual Routing and Forwarding table, as defined in
[RFC4364]. It is also the instantiation of an IPVPN in a PE.
Route Distinguisher and Route Target(s) are required properties of
an IP- VRF.
[RFC7432]. It is also the instantiation of an EVI (EVPN Instance) o MAC-VRF: a MAC Virtual Routing and Forwarding table, as defined in
in a PE. Route Distinguisher and Route Target(s) are required [RFC7432]. It is also the instantiation of an EVI (EVPN Instance)
properties and they are normally different than the ones defined in in a PE. Route Distinguisher and Route Target(s) are required
the associated IP-VRF. properties and they are normally different than the ones defined
in the associated IP-VRF.
o BT: a Bridge Table, as defined in [RFC7432]. A BT is the o BT: a Bridge Table, as defined in [RFC7432]. A BT is the
instantiation of a Broadcast Domain in a PE. When there is a single instantiation of a Broadcast Domain in a PE. When there is a
Broadcast Domain in a given EVI, the MAC-VRF in each PE will single Broadcast Domain in a given EVI, the MAC-VRF in each PE
contain a single BT. When there are multiple BTs within the same will contain a single BT. When there are multiple BTs within the
MAC-VRF, each BT is associated to a different Ethernet Tag. The same MAC-VRF, each BT is associated to a different Ethernet Tag.
EVPN routes specific to a BT, will indicate which Ethernet Tag the The EVPN routes specific to a BT, will indicate which Ethernet Tag
route corresponds to. the route corresponds to.
Example: In Figure 1, MAC-VRF1 has two BTs: BT1 and BT2. Ethernet Example: In Figure 1, MAC-VRF1 has two BTs: BT1 and BT2. Ethernet
Tag x is defined in BT1 and Ethernet Tag y in BT2. Tag x is defined in BT1 and Ethernet Tag y in BT2.
o AC: Attachment Circuit or logical interface associated to a given o AC: Attachment Circuit or logical interface associated to a given
BT or IP-VRF. To determine the AC on which a packet arrived, the PE BT or IP-VRF. To determine the AC on which a packet arrived, the
will examine the combination of a physical port and VLAN tags PE will examine the combination of a physical port and VLAN tags
(where the VLAN tags can be individual c-tags, s-tags or ranges of (where the VLAN tags can be individual c-tags, s-tags or ranges of
both). both).
Example: In Figure 1, AC1 is associated to BT1, AC2 to BT2 and AC3 Example: In Figure 1, AC1 is associated to BT1, AC2 to BT2 and AC3
to IP-VRF1. to IP-VRF1.
o IRB: Integrated Routing and Bridging interface. It refers to the o IRB: Integrated Routing and Bridging interface. It refers to the
logical interface that connects a BT to an IP-VRF and allows to logical interface that connects a BT to an IP-VRF and allows to
forward packets with destination in a different subnet. forward packets with destination in a different subnet.
o MPLS/NVO tnl: It refers to a tunnel that can be MPLS or NVO-based o MPLS/NVO tnl: It refers to a tunnel that can be MPLS or NVO-based
(Network Virtualization Overlays) and it is used by MAC-VRFs and (Network Virtualization Overlays) and it is used by MAC-VRFs and
IP-VRFs. Irrespective of the type, the tunnel may carry an Ethernet IP-VRFs. Irrespective of the type, the tunnel may carry an
or an IP payload. MAC-VRFs can only use tunnels with Ethernet Ethernet or an IP payload. MAC-VRFs can only use tunnels with
payloads (setup by EVPN), whereas IP-VRFs can use tunnels with Ethernet payloads (setup by EVPN), whereas IP-VRFs can use tunnels
Ethernet (setup by EVPN) or IP payloads (setup by EVPN or IPVPN). with Ethernet (setup by EVPN) or IP payloads (setup by EVPN or
IPVPN-only PEs have IP-VRFs but they cannot send or receive traffic IPVPN). IPVPN-only PEs have IP-VRFs but they cannot send or
on tunnels with Ethernet payloads. receive traffic on tunnels with Ethernet payloads.
Example: Figure 1 shows an MPLS/NVO tunnel that is used to Example: Figure 1 shows an MPLS/NVO tunnel that is used to
transport Ethernet frames to/from MAC-VRF1. The PE determines the transport Ethernet frames to/from MAC-VRF1. The PE determines the
MAC-VRF and BT the packets belong to based on the EVPN label (MPLS MAC-VRF and BT the packets belong to based on the EVPN label (MPLS
or VNI). Figure 1 also shows two MPLS/NVO tunnels being used by IP- or VNI). Figure 1 also shows two MPLS/NVO tunnels being used by
VRF1, one carrying Ethernet frames and the other one carrying IP IP- VRF1, one carrying Ethernet frames and the other one carrying
packets. IP packets.
o RT-2: Route Type 2 or MAC/IP route, as per [RFC7432]. o RT-2: Route Type 2 or MAC/IP route, as per [RFC7432].
o RT-5: Route Type 5 or IP Prefix route, as per [IP-PREFIX]. o RT-5: Route Type 5 or IP Prefix route, as per
[I-D.ietf-bess-evpn-prefix-advertisement].
o Domain: Two PEs are in the same domain if they are attached to the o Domain: Two PEs are in the same domain if they are attached to the
same tenant and the packets between them do not require a data path same tenant and the packets between them do not require a data
IP lookup (in the tenant space) in any intermediate router. A path IP lookup (in the tenant space) in any intermediate router.
gateway PE is always configured with multiple Domain-IDs. A gateway PE is always configured with multiple Domain-IDs.
Example 1: Figure 2 depicts an example where TS1 and TS2 belong to Example 1: Figure 2 depicts an example where TS1 and TS2 belong to
the same tenant, and they are located in different Data Centers the same tenant, and they are located in different Data Centers
that are connected by gateway PEs (see the gateway PE definition that are connected by gateway PEs (see the gateway PE definition
later). These gateway PEs use IPVPN in the WAN. When TS1 sends later). These gateway PEs use IPVPN in the WAN. When TS1 sends
traffic to TS2, the intermediate routers between PE1 and PE2 traffic to TS2, the intermediate routers between PE1 and PE2
require a tenant IP lookup in their IP-VRFs so that the packets can require a tenant IP lookup in their IP-VRFs so that the packets
be forwarded. In this example there are three different domains. can be forwarded. In this example there are three different
The gateway PEs connect the EVPN domains to the IPVPN domain. domains. The gateway PEs connect the EVPN domains to the IPVPN
domain.
GW1------------GW3 GW1------------GW3
+------+ +------+ +------+ +------+
+-------------|IP-VRF| |IP-VRF|-------------+ +-------------|IP-VRF| |IP-VRF|-------------+
PE1 +------+ +------+ PE2 PE1 +------+ +------+ PE2
+------+ DC1 | WAN | DC2 +------+ +------+ DC1 | WAN | DC2 +------+
TS1-|IP-VRF| EVPN | IPVPN | EVPN |IP-VRF|-TS2 TS1-|IP-VRF| EVPN | IPVPN | EVPN |IP-VRF|-TS2
+------+ GW2 GW4 +---+--+ +------+ GW2 GW4 +---+--+
| +------+ +------+ | | +------+ +------+ |
+-------------|IP-VRF| |IP-VRF|-------------+ +-------------|IP-VRF| |IP-VRF|-------------+
+------+ +------+ +------+ +------+
+--------------+ +--------------+
DOMAIN 1 DOMAIN 2 DOMAIN 3 DOMAIN 1 DOMAIN 2 DOMAIN 3
<---------------> <------------> <----------------> <---------------> <------------> <---------------->
Figure 2 Multiple domain DCI example Figure 2: Multiple domain DCI example
Example 2: Figure 3 illustrates a similar example, but PE1 and PE2 Example 2: Figure 3 illustrates a similar example, but PE1 and PE2
are now connected by a BGP-LU (BGP Labeled Unicast) tunnel, and are now connected by a BGP-LU (BGP Labeled Unicast) tunnel, and
they have a BGP peer relationship for EVPN. Contrary to Example 1, they have a BGP peer relationship for EVPN. Contrary to Example 1,
there is no need for tenant IP lookups on the intermediate routers there is no need for tenant IP lookups on the intermediate routers
in order to forward packets between PE1 and PE2. Therefore, there in order to forward packets between PE1 and PE2. Therefore, there
is only one domain in the network and PE1/PE2 belong to it. is only one domain in the network and PE1/PE2 belong to it.
EVPN EVPN
<-------------------------------------------------> <------------------------------------------------->
BGP-LU BGP-LU
<-------------------------------------------------> <------------------------------------------------->
ASBR------------ASBR ASBR------------ASBR
+------+ +------+ +------+ +------+
+-------------| | | |-------------+ +-------------| | | |-------------+
skipping to change at page 7, line 24 skipping to change at page 7, line 24
+------+ DC1 | WAN | DC2 +------+ +------+ DC1 | WAN | DC2 +------+
TS1-|IP-VRF| EVPN | | EVPN |IP-VRF|-TS2 TS1-|IP-VRF| EVPN | | EVPN |IP-VRF|-TS2
+------+ ASBR ASBR +---+--+ +------+ ASBR ASBR +---+--+
| +------+ +------+ | | +------+ +------+ |
+-------------| | | |-------------+ +-------------| | | |-------------+
+------+ +------+ +------+ +------+
+--------------+ +--------------+
<--------------------DOMAIN-1---------------------> <--------------------DOMAIN-1--------------------->
Figure 3 Single domain DCI example Figure 3: Single domain DCI example
o Regular Domain: a domain in which a single control plane, IPVPN or o Regular Domain: a domain in which a single control plane, IPVPN or
EVPN, is used and which is composed of regular PEs, see below. In EVPN, is used and which is composed of regular PEs, see below. In
Figures 2 and 3, above, all domains are regular domains. Figures 2 and 3, above, all domains are regular domains.
o Composite Domain: a domain in which multiple control planes, IPVPN o Composite Domain: a domain in which multiple control planes, IPVPN
and EVPN, are used and which is composed of regular PEs, see below, and EVPN, are used and which is composed of regular PEs, see
and composite PEs, see below. below, and composite PEs, see below.
o Regular PE: a PE that is attached to a domain, either regular or o Regular PE: a PE that is attached to a domain, either regular or
composite, and which uses one of the control plane protocols (IPVPN composite, and which uses one of the control plane protocols
or EVPN) operating in the domain. (IPVPN or EVPN) operating in the domain.
o Interworking PE: a PE that may advertise a given prefix with an o Interworking PE: a PE that may advertise a given prefix with an
EVPN ISF route (RT-2 or RT-5) and/or an IPVPN ISF route. An EVPN ISF route (RT-2 or RT-5) and/or an IPVPN ISF route. An
interworking PE has one IP-VRF per tenant, and one or multiple MAC- interworking PE has one IP-VRF per tenant, and one or multiple
VRFs per tenant. Each MAC-VRF may contain one or more BTs, where MAC- VRFs per tenant. Each MAC-VRF may contain one or more BTs,
each BT may be attached to that IP-VRF via IRB. There are two types where each BT may be attached to that IP-VRF via IRB. There are
of Interworking PEs: composite PEs and gateway PEs. Both PE two types of Interworking PEs: composite PEs and gateway PEs.
functions can be independently implemented per tenant and they may Both PE functions can be independently implemented per tenant and
both be implemented for the same tenant. they may both be implemented for the same tenant.
Example: Figure 1 shows an interworking PE of type gateway, where Example: Figure 1 shows an interworking PE of type gateway, where
ISF SAFIs 1, 128 and 70 are enabled. IP-VRF1 and MAC-VRF1 are ISF SAFIs 1, 128 and 70 are enabled. IP-VRF1 and MAC-VRF1 are
instantiated on the PE, and together provide inter-subnet instantiated on the PE, and together provide inter-subnet
forwarding for the tenant. forwarding for the tenant.
o Composite PE: an interworking PE that is attached to a composite o Composite PE: an interworking PE that is attached to a composite
domain and which advertises a given prefix to an IPVPN peer with an domain and which advertises a given prefix to an IPVPN peer with
IPVPN ISF route, to an EVPN peer with an EVPN ISF route, and to a an IPVPN ISF route, to an EVPN peer with an EVPN ISF route, and to
route reflector with both an IPVPN and EVPN ISF route. A composite a route reflector with both an IPVPN and EVPN ISF route. A
PE performs the procedures of Sections 5 and 6. composite PE performs the procedures of Sections 5 and 6.
Example: Figure 4 shows an example where PE1 is a composite PE Example: Figure 4 shows an example where PE1 is a composite PE
since PE1 has EVPN and another ISF SAFI enabled to the same route- since PE1 has EVPN and another ISF SAFI enabled to the same route-
reflector, and PE1 advertises a given IP prefix IPn/x twice, one reflector, and PE1 advertises a given IP prefix IPn/x twice, one
using EVPN and another one using ISF SAFI 128. PE2 and PE3 are not using EVPN and another one using ISF SAFI 128. PE2 and PE3 are
composite PEs. not composite PEs.
+---+ +---+
|PE2| |PE2|
+---+ +---+
^ ^
|EVPN |EVPN
IW EVPN v IW EVPN v
+---+ IPVPN ++-+ +---+ +---+ IPVPN ++-+ +---+
|PE1| <----> |RR| <---> |PE3| |PE1| <----> |RR| <---> |PE3|
+---+ +--+ IPVPN +---+ +---+ +--+ IPVPN +---+
Composite Composite
Figure 4 Interworking composite PE example Figure 4: Interworking composite PE example
o Gateway PE: an interworking PE that is attached to two domains, o Gateway PE: an interworking PE that is attached to two domains,
each either regular or composite, and which, based on each either regular or composite, and which, based on
configuration, does one of the following: configuration, does one of the following:
- Propagates the same control plane protocol, either IPVPN or EVPN, - Propagates the same control plane protocol, either IPVPN or
between the two domains. EVPN, between the two domains.
- Propagates an ISF route with different ISF SAFIs between the two - Propagates an ISF route with different ISF SAFIs between the
domains. E.g., propagate an EVPN ISF route in one domain as an two domains. E.g., propagate an EVPN ISF route in one domain
IPVPN ISF route in the other domain and vice versa. A gateway PE as an IPVPN ISF route in the other domain and vice versa. A
performs the procedures of Sections 3, 4, 5 and 7. gateway PE performs the procedures of Sections 3, 4, 5 and 7.
A gateway PE is always configured with multiple Domain-IDs. The A gateway PE is always configured with multiple Domain-IDs.
Domain-ID is encoded in the Domain Path Attribute (D-PATH), and The Domain-ID is encoded in the Domain Path Attribute (D-PATH),
advertised along with EVPN and other ISF SAFI routes. Section 3 and advertised along with EVPN and other ISF SAFI routes.
describes the D-PATH attribute. Section 3 describes the D-PATH attribute.
Example: Figure 5 illustrates an example where PE1 is a gateway Example: Figure 5 illustrates an example where PE1 is a gateway
PE since the EVPN and IPVPN SAFIs are enabled on different BGP PE since the EVPN and IPVPN SAFIs are enabled on different BGP
peers, and a given local IP prefix IPn/x is sent to both BGP peers, and a given local IP prefix IPn/x is sent to both BGP
peers for the same tenant. PE2 and PE1 are in one domain and PE3 peers for the same tenant. PE2 and PE1 are in one domain and
and PE1 are in another domain. PE3 and PE1 are in another domain.
IW IW
+---+ EVPN +---+ IPVPN +---+ +---+ EVPN +---+ IPVPN +---+
|PE2| <----> |PE1| <----> |PE3| |PE2| <----> |PE1| <----> |PE3|
+---+ +---+ +---+ +---+ +---+ +---+
Gateway Gateway
Figure 5 Interworking gateway PE example Figure 5: Interworking gateway PE example
o Composite/Gateway PE: an interworking PE that is both a composite o Composite/Gateway PE: an interworking PE that is both a composite
PE and a gateway PE that is attached to two domains, one regular PE and a gateway PE that is attached to two domains, one regular
and one composite, and which does the following: and one composite, and which does the following:
- Propagates an ISF route, either IPVPN or EVPN, from the regular - Propagates an ISF route, either IPVPN or EVPN, from the regular
domain into the composite domain. Within the composite domain it domain into the composite domain. Within the composite domain
acts as a composite PE. it acts as a composite PE.
- Propagates an ISF route, either IPVPN or EVPN, from the composite - Propagates an ISF route, either IPVPN or EVPN, from the
domain into the regular domain. Within the regular domain it is composite domain into the regular domain. Within the regular
propagated as an ISF route using the ISF SAFI for that domain. domain it is propagated as an ISF route using the ISF SAFI for
that domain.
This is particularly useful when a tenant network is attached to This is particularly useful when a tenant network is attached
both IPVPN and EVPN domains, any-to-any connectivity is required, to both IPVPN and EVPN domains, any-to-any connectivity is
and end-to-end control plane consistency, when possible, is required, and end-to-end control plane consistency, when
desired. possible, is desired.
It would be instantiated by attaching the disparate, regular It would be instantiated by attaching the disparate, regular
IPVPN and EVPN domains via these PEs to a central composite IPVPN and EVPN domains via these PEs to a central composite
domain. domain.
3. Domain Path Attribute (D-PATH) 3. Domain Path Attribute (D-PATH)
The BGP Domain Path (D-PATH) attribute is an optional and transitive The BGP Domain Path (D-PATH) attribute is an optional and transitive
BGP path attribute. BGP path attribute.
Similar to AS_PATH, D-PATH is composed of a sequence of Domain Similar to AS_PATH, D-PATH is composed of a sequence of Domain
segments. Each Domain segment is comprised of <domain segment length, segments. Each Domain segment is comprised of <domain segment
domain segment value>, where the domain segment value is a sequence length, domain segment value>, where the domain segment value is a
of one or more Domains. Each domain is represented by <DOMAIN- sequence of one or more Domains. Each domain is represented by
ID:ISF_SAFI_TYPE>. <DOMAIN-ID:ISF_SAFI_TYPE>.
o The domain segment length field is a 1-octet field, containing the o The domain segment length field is a 1-octet field, containing the
number of domains in the segment. number of domains in the segment.
o DOMAIN-ID is a 6-octet field that represents a domain. It is o DOMAIN-ID is a 6-octet field that represents a domain. It is
composed of a 4-octet Global Administrator sub-field and a 2-octet composed of a 4-octet Global Administrator sub-field and a 2-octet
Local Administrator sub-field. The Global Administrator sub-field Local Administrator sub-field. The Global Administrator sub-field
MAY be filled with an Autonomous System Number (ASN), an IPv4 MAY be filled with an Autonomous System Number (ASN), an IPv4
address, or any value that guarantees the uniqueness of the DOMAIN- address, or any value that guarantees the uniqueness of the
ID when the tenant network is connected to multiple Operators. DOMAIN- ID when the tenant network is connected to multiple
Operators.
o ISF_SAFI_TYPE is a 1-octet field that indicates the Inter-Subnet o ISF_SAFI_TYPE is a 1-octet field that indicates the Inter-Subnet
Forwarding SAFI type in which a route was advertised in the DOMAIN. Forwarding SAFI type in which a route was advertised in the
The following types are valid in this document: DOMAIN. The following types are valid in this document:
Value Type Value Type
1 SAFI 1 1 SAFI 1
70 EVPN 70 EVPN
128 SAFI 128 128 SAFI 128
About the BGP D-PATH attribute: About the BGP D-PATH attribute:
a) Identifies the sequence of domains, each identified by a <DOMAIN- a) Identifies the sequence of domains, each identified by a
ID:ISF_SAFI_TYPE> through which a given ISF route has passed. <DOMAIN-ID:ISF_SAFI_TYPE> through which a given ISF route has
passed.
- This attribute list may contain zero, one or more segments. - This attribute list may contain zero, one or more segments.
- The first entry in the list (leftmost) is the <DOMAIN- - The first entry in the list (leftmost) is the <DOMAIN-
ID:ISF_SAFI_TYPE> from which a gateway PE is propagating an ISF ID:ISF_SAFI_TYPE> from which a gateway PE is propagating an ISF
route. The last entry in the list (rightmost) is the <DOMAIN- route. The last entry in the list (rightmost) is the <DOMAIN-
ID:ISF_SAFI_TYPE> from which a gateway PE received an ISF route ID:ISF_SAFI_TYPE> from which a gateway PE received an ISF route
without a D-PATH attribute. Intermediate entries in the list are without a D-PATH attribute. Intermediate entries in the list
domains that the ISF route has transited. are domains that the ISF route has transited.
- As an example, an ISF route received with a D-PATH attribute - As an example, an ISF route received with a D-PATH attribute
containing a domain segment of {length=2, containing a domain segment of {length=2,
<6500:2:IPVPN>,<6500:1:EVPN>} indicates that the ISF route was <6500:2:IPVPN>,<6500:1:EVPN>} indicates that the ISF route was
originated in EVPN domain 6500:1, and propagated into IPVPN originated in EVPN domain 6500:1, and propagated into IPVPN
domain 6500:2. domain 6500:2.
b) It is added/modified by a gateway PE when propagating an update to b) It is added/modified by a gateway PE when propagating
a different domain: an update to a different domain:
- A gateway PE's IP-VRF, that connects two domains, belongs to two - A gateway PE's IP-VRF, that connects two domains, belongs to
DOMAIN-IDs, e.g. 6500:1 for EVPN and 6500:2 for IPVPN. two DOMAIN-IDs, e.g. 6500:1 for EVPN and 6500:2 for IPVPN.
- Whenever a prefix arrives at a gateway PE in a particular ISF - Whenever a prefix arrives at a gateway PE in a particular ISF
SAFI route, if the gateway PE needs to export that prefix to a SAFI route, if the gateway PE needs to export that prefix to a
BGP peer, the gateway PE will prepend a <DOMAIN- BGP peer, the gateway PE will prepend a <DOMAIN-
ID:ISF_SAFI_TYPE> to the list of domains in the received D-PATH. ID:ISF_SAFI_TYPE> to the list of domains in the received
D-PATH.
- For instance, in an IP-VRF configured with DOMAIN-IDs 6500:1 for - For instance, in an IP-VRF configured with DOMAIN-IDs 6500:1
EVPN and 6500:2 for IPVPN, if an EVPN route for prefix P is for EVPN and 6500:2 for IPVPN, if an EVPN route for prefix P is
received and P installed in the IP-VRF, the IPVPN route for P received and P installed in the IP-VRF, the IPVPN route for P
that is exported to an IPVPN peer will prepend the domain that is exported to an IPVPN peer will prepend the domain
<6500:1:EVPN> to the previously received D-PATH attribute. <6500:1:EVPN> to the previously received D-PATH attribute.
Likewise, IP-VRF prefixes that are received from IP-VPN, will be Likewise, IP-VRF prefixes that are received from IP-VPN, will
exported to EVPN peers with the domain <6500:2:IPVPN> added to be exported to EVPN peers with the domain <6500:2:IPVPN> added
the segment. to the segment.
- In the above example, if the EVPN route is received without D- - In the above example, if the EVPN route is received without D-
PATH, the gateway PE will add the D-PATH attribute with one PATH, the gateway PE will add the D-PATH attribute with one
segment {length=1, <6500:1:EVPN>} when re-advertising to domain segment {length=1, <6500:1:EVPN>} when re-advertising to domain
6500:2. 6500:2.
- Within the originating domain, the update does not contain a D- - Within the originating domain, the update does not contain a D-
PATH attribute because the update has not passed through a PATH attribute because the update has not passed through a
gateway PE yet. gateway PE yet.
c) The gateway PE MUST NOT add the D-PATH attribute to ISF routes c) The gateway PE MUST NOT add the D-PATH attribute to ISF routes
generated for IP-VRF prefixes that are not learned via any ISF generated for IP-VRF prefixes that are not learned via any ISF
SAFI, for instance, local prefixes. SAFI, for instance, local prefixes.
d) An ISF route received by a gateway PE with a D-PATH attribute that d) An ISF route received by a gateway PE with a D-PATH
contains one or more of its locally configured domains for the IP- attribute that contains one or more of its locally configured
VRF is considered to be a looped ISF route and MUST be dropped. domains for the IP-VRF is considered to be a looped ISF route and
MUST be dropped.
e) The number of domains in the D-PATH attribute indicates the number e) The number of domains in the D-PATH attribute
of gateway PEs that the ISF route update has transited. indicates the number of gateway PEs that the ISF route update has
transited.
3.1. D-PATH and Loop Prevention 3.1. D-PATH and Loop Prevention
The D-PATH attribute is used to prevent loops in interworking PE The D-PATH attribute is used to prevent loops in interworking PE
networks. For instance, in the example of Figure 4, gateway GW1 networks. For instance, in the example of Figure 4, gateway GW1
receives TS1 prefix in two different ISF routes: receives TS1 prefix in two different ISF routes:
o In an EVPN RT-5 with next-hop PE1 and no D-PATH attribute. o In an EVPN RT-5 with next-hop PE1 and no D-PATH attribute.
o In a SAFI 128 route with next-hop GW2 and D-PATH = {length=1, o In a SAFI 128 route with next-hop GW2 and D-PATH = {length=1,
<6500:1:EVPN>}, assuming that DOMAIN-ID for domain 1 is 6500:1. <6500:1:EVPN>}, assuming that DOMAIN-ID for domain 1 is 6500:1.
Gateway GW1 flags the SAFI 128 route as a loop, and does not re- Gateway GW1 flags the SAFI 128 route as a loop, and does not re-
advertise it to the EVPN neighbors since the route includes the GW1's advertise it to the EVPN neighbors since the route includes the GW1's
local domain. local domain.
In general, any interworking PE that imports an ISF route MUST flag In general, any interworking PE that imports an ISF route MUST flag
the route as "looped" if its D-PATH contains a <DOMAIN- the route as "looped" if its D-PATH contains a <DOMAIN-
ID:ISF_SAFI_TYPE> segment, where DOMAIN-ID matches a local DOMAIN-ID ID:ISF_SAFI_TYPE> segment, where DOMAIN-ID matches a local DOMAIN-ID
in the tenant IP-VRF. in the tenant IP-VRF.
4. BGP Path Attribute Propagation across ISF SAFIs 4. BGP Path Attribute Propagation across ISF SAFIs
Based on configurations a gateway PE is required to propagate an ISF Based on configurations a gateway PE is required to propagate an ISF
route with different ISF SAFIs between two domains. This requires a route with different ISF SAFIs between two domains. This requires a
definition of what a gateway PE has to do with Path attributes definition of what a gateway PE has to do with Path attributes
attached to the ISF route that it is propagating. attached to the ISF route that it is propagating.
4.1. No-Propagation-Mode 4.1. No-Propagation-Mode
This is the default mode of operation. In this mode, the gateway PE This is the default mode of operation. In this mode, the gateway PE
will simply re-initialize the Path Attributes when propagating an ISF will simply re-initialize the Path Attributes when propagating an ISF
route, as though it would for direct or local IP prefixes. This model route, as though it would for direct or local IP prefixes. This
may be enough in those use-cases where the EVPN domain is considered model may be enough in those use-cases where the EVPN domain is
an "abstracted" CE and remote IPVPN/IP PEs don't need to consider the considered an "abstracted" CE and remote IPVPN/IP PEs don't need to
original EVPN Attributes for path calculations. consider the original EVPN Attributes for path calculations.
Since this mode of operation does not propagate the D-PATH attribute Since this mode of operation does not propagate the D-PATH attribute
either, redundant gateway PEs are exposed to routing loops. Those either, redundant gateway PEs are exposed to routing loops. Those
loops may be resolved by policies and the use of other attributes, loops may be resolved by policies and the use of other attributes,
such as the Route Origin extended community [RFC4360], however not such as the Route Origin extended community [RFC4360], however not
all the loop situations may be solved. all the loop situations may be solved.
4.2. Uniform-Propagation-Mode 4.2. Uniform-Propagation-Mode
In this mode, the gateway PE simply keeps accumulating or mapping In this mode, the gateway PE simply keeps accumulating or mapping
certain key commonly used Path Attributes when propagating an ISF certain key commonly used Path Attributes when propagating an ISF
route. This mode is typically used in networks where EVPN and IPVPN route. This mode is typically used in networks where EVPN and IPVPN
SAFIs are used seamlessly to distribute IP prefixes. SAFIs are used seamlessly to distribute IP prefixes.
The following rules MUST be observed by the gateway PE when The following rules MUST be observed by the gateway PE when
propagating Path Attributes: propagating Path Attributes:
o The gateway PE imports an ISF route in the IP-VRF and stores the o The gateway PE imports an ISF route in the IP-VRF and stores the
original Path Attributes. The following set of Path Attributes original Path Attributes. The following set of Path Attributes
SHOULD be propagated by the gateway PE to other ISF SAFIs (other SHOULD be propagated by the gateway PE to other ISF SAFIs (other
Path Attributes SHOULD NOT be propagated): Path Attributes SHOULD NOT be propagated):
- AS_PATH - AS_PATH
- D-PATH - D-PATH
- IBGP-only Path Attributes: LOCAL_PREF, ORIGINATOR_ID, CLUSTER_ID - IBGP-only Path Attributes: LOCAL_PREF, ORIGINATOR_ID, CLUSTER_ID
- MED - MED
- AIGP - AIGP
- Communities, (non-EVPN) Extended Communities and Large
Communities
o When propagating an ISF route to a different ISF SAFI and IBGP o Communities, (non-EVPN) Extended Communities and Large Communities
peer, the gateway PE SHOULD copy the AS_PATH of the originating - When propagating an ISF route to a different ISF SAFI and IBGP
family and add it to the destination family without any peer, the gateway PE SHOULD copy the AS_PATH of the originating
modification. When re-advertising to a different ISF SAFI and EBGP family and add it to the destination family without any
peer, the gateway PE SHOULD copy the AS_PATH of the originating modification. When re-advertising to a different ISF SAFI and
family and prepend the IP-VRF's AS before sending the route. EBGP peer, the gateway PE SHOULD copy the AS_PATH of the
originating family and prepend the IP-VRF's AS before sending
the route.
o When propagating an ISF route to IBGP peers, the gateway PE SHOULD - When propagating an ISF route to IBGP peers, the gateway PE
copy the IBGP-only Path Attributes from the originating SAFI to the SHOULD copy the IBGP-only Path Attributes from the originating
re-advertised route. SAFI to the re-advertised route.
o Communities, non-EVPN Extended Communities and Large Communities - Communities, non-EVPN Extended Communities and Large
SHOULD be copied by the gateway PE from the originating SAFI route. Communities SHOULD be copied by the gateway PE from the
originating SAFI route.
4.3. Aggregation of Routes and Path Attribute Propagation 4.3. Aggregation of Routes and Path Attribute Propagation
Instead of propagating a high number of (host) ISF routes between ISF Instead of propagating a high number of (host) ISF routes between ISF
SAFIs, a gateway PE that receives multiple ISF routes of one ISF SAFI SAFIs, a gateway PE that receives multiple ISF routes of one ISF SAFI
MAY choose to propagate a single ISF aggregate route with a different MAY choose to propagate a single ISF aggregate route with a different
ISF SAFI. In this document, aggregation is used to combine the ISF SAFI. In this document, aggregation is used to combine the
characteristics of multiple ISF routes of the same ISF SAFI in such characteristics of multiple ISF routes of the same ISF SAFI in such
way that a single aggregate ISF route of a different ISF SAFI can be way that a single aggregate ISF route of a different ISF SAFI can be
propagated. Aggregation of multiple ISF routes of one ISF SAFI into propagated. Aggregation of multiple ISF routes of one ISF SAFI into
an aggregate ISF route of a different ISF SAFI is only done by a an aggregate ISF route of a different ISF SAFI is only done by a
gateway PE. gateway PE.
Aggregation on gateway PEs may use either the No-Propagation-Mode or Aggregation on gateway PEs may use either the No-Propagation-Mode or
the Uniform-Propagation-Mode explained in Sections 4.1. and 4.2, the Uniform-Propagation-Mode explained in Sections 4.1. and 4.2,
respectively. respectively.
When using Uniform-Propagation-Mode, Path Attributes of the same type When using Uniform-Propagation-Mode, Path Attributes of the same type
code MAY be aggregated according to the following rules: code MAY be aggregated according to the following rules:
o AS_PATH is aggregated based on the rules in [RFC4271]. The gateway o AS_PATH is aggregated based on the rules in [RFC4271]. The
PEs SHOULD NOT receive AS_PATH attributes with path segments of gateway PEs SHOULD NOT receive AS_PATH attributes with path
type AS_SET [RFC6472]. Routes received with AS_PATH attributes segments of type AS_SET [RFC6472]. Routes received with AS_PATH
including AS_SET path segments MUST NOT be aggregated. attributes including AS_SET path segments MUST NOT be aggregated.
o ISF routes that have different attributes of the following type o ISF routes that have different attributes of the following type
codes MUST NOT be aggregated: D-PATH, LOCAL_PREF, ORIGINATOR_ID, codes MUST NOT be aggregated: D-PATH, LOCAL_PREF, ORIGINATOR_ID,
CLUSTER_ID, MED or AIGP. CLUSTER_ID, MED or AIGP.
o The Community, Extended Community and Large Community attributes of o The Community, Extended Community and Large Community attributes
the aggregate ISF route MUST contain all the Communities/Extended of the aggregate ISF route MUST contain all the Communities/
Communities/Large Communities from all of the aggregated ISF Extended Communities/Large Communities from all of the aggregated
routes. ISF routes.
Assuming the aggregation can be performed (the above rules are Assuming the aggregation can be performed (the above rules are
applied), the operator should consider aggregation to deal with applied), the operator should consider aggregation to deal with
scaled tenant networks where a significant number of host routes scaled tenant networks where a significant number of host routes
exists. For a example, large Data Centers. exists. For a example, large Data Centers.
5. Route Selection Process between EVPN and other ISF SAFIs 5. Route Selection Process between EVPN and other ISF SAFIs
A PE may receive an IP prefix in ISF routes with different ISF SAFIs, A PE may receive an IP prefix in ISF routes with different ISF SAFIs,
from the same or different BGP peer. It may also receive the same IP from the same or different BGP peer. It may also receive the same IP
prefix (host route) in an EVPN RT-2 and RT-5. A route selection prefix (host route) in an EVPN RT-2 and RT-5. A route selection
algorithm across all ISF SAFIs is needed so that: algorithm across all ISF SAFIs is needed so that:
o Different gateway and composite PEs have a consistent and o Different gateway and composite PEs have a consistent and
deterministic view on how to reach a given prefix. deterministic view on how to reach a given prefix.
o Prefixes advertised in EVPN and other ISF SAFIs can be compared o Prefixes advertised in EVPN and other ISF SAFIs can be compared
based on path attributes commonly used by operators across based on path attributes commonly used by operators across
networks. networks.
o Equal Cost Multi-Path (ECMP) is allowed across EVPN and other ISF o Equal Cost Multi-Path (ECMP) is allowed across EVPN and other ISF
SAFI routes. SAFI routes.
For a given prefix advertised in one or more non-EVPN ISF routes, the For a given prefix advertised in one or more non-EVPN ISF routes, the
BGP best path selection procedure will produce a set of "non-EVPN BGP best path selection procedure will produce a set of "non-EVPN
best paths". For a given prefix advertised in one or more EVPN ISF best paths". For a given prefix advertised in one or more EVPN ISF
routes, the BGP best path selection procedure will produce a set of routes, the BGP best path selection procedure will produce a set of
"EVPN best paths". To support IP/EVPN interworking, it is then "EVPN best paths". To support IP/EVPN interworking, it is then
necessary to run a tie-breaking selection algorithm on the union of necessary to run a tie-breaking selection algorithm on the union of
these two sets. This tie-breaking algorithm begins by considering all these two sets. This tie-breaking algorithm begins by considering
EVPN and other ISF SAFI routes, equally preferable routes to the same all EVPN and other ISF SAFI routes, equally preferable routes to the
destination, and then selects routes to be removed from same destination, and then selects routes to be removed from
consideration. The process terminates as soon as only one route consideration. The process terminates as soon as only one route
remains in consideration. remains in consideration.
The route selection algorithm must remove from consideration the The route selection algorithm must remove from consideration the
routes following the rules and the order defined in [RFC4271], with routes following the rules and the order defined in [RFC4271], with
the following exceptions and in the following order: the following exceptions and in the following order:
1- Immediately after removing from consideration all routes that are 1- Immediately after removing from consideration all routes that are
not tied for having the highest Local Preference, any routes that not tied for having the highest Local Preference, any routes that
do not have the shortest D-PATH are also removed from do not have the shortest D-PATH are also removed from
consideration. Routes with no D-PATH are considered to have a consideration. Routes with no D-PATH are considered to have a
zero-length D-PATH. zero-length D-PATH.
2- Then regular [RFC4271] selection criteria is followed. 2- Then regular [RFC4271] selection criteria is followed.
3- At the end of the selection algorithm, if at least one route still 3- At the end of the selection algorithm, if at least one route still
under consideration is an RT-2 route, remove from consideration under consideration is an RT-2 route, remove from consideration
any RT-5 routes. any RT-5 routes.
4- Steps 1-3 could possibly leave Equal Cost Multi-Path (ECMP) 4- Steps 1-3 could possibly leave Equal Cost Multi-Path (ECMP)
between IP and EVPN paths. By default, the EVPN path is considered between IP and EVPN paths. By default, the EVPN path is
(and the IP path removed from consideration). However, if ECMP considered (and the IP path removed from consideration). However,
across ISF SAFIs is enabled by policy, and an "IP path" and an if ECMP across ISF SAFIs is enabled by policy, and an "IP path"
"EVPN path" remain at the end of step 3, both path types will be and an "EVPN path" remain at the end of step 3, both path types
used. will be used.
Example 1 - PE1 receives the following routes for IP1/32, that are Example 1 - PE1 receives the following routes for IP1/32, that are
candidate to be imported in IP-VRF-1: candidate to be imported in IP-VRF-1:
{SAFI=EVPN, RT-2, Local-Pref=100, AS-Path=(100,200)} {SAFI=EVPN, RT-2, Local-Pref=100, AS-Path=(100,200)}
{SAFI=EVPN, RT-5, Local-Pref=100, AS-Path=(100,200)} {SAFI=EVPN, RT-5, Local-Pref=100, AS-Path=(100,200)}
{SAFI=128, Local-Pref=100, AS-Path=(100,200)} {SAFI=128, Local-Pref=100, AS-Path=(100,200)}
Selected route: {SAFI=EVPN, RT-2, Local-Pref=100, AS-Path=100,200] Selected route: {SAFI=EVPN, RT-2, Local-Pref=100, AS-Path=100,200]
(due to step 3, and no ECMP) (due to step 3, and no ECMP)
skipping to change at page 15, line 39 skipping to change at page 15, line 38
candidate to be imported in IP-VRF-1: candidate to be imported in IP-VRF-1:
{SAFI=EVPN, RT-5, D-PATH=(6500:3:IPVPN), AS-Path=(100,200), {SAFI=EVPN, RT-5, D-PATH=(6500:3:IPVPN), AS-Path=(100,200),
MED=10} MED=10}
{SAFI=128, D-PATH=(6500:1:EVPN,6500:2:IPVPN), AS-Path=(200), {SAFI=128, D-PATH=(6500:1:EVPN,6500:2:IPVPN), AS-Path=(200),
MED=200} MED=200}
Selected route: {SAFI=EVPN, RT-5, D-PATH=(6500:3:IPVPN), AS- Selected route: {SAFI=EVPN, RT-5, D-PATH=(6500:3:IPVPN), AS-
Path=(100,200), MED=10} (due to step 1) Path=(100,200), MED=10} (due to step 1)
6. Composite PE Procedures 6. Composite PE Procedures
As described in Section 2, composite PEs are typically used in tenant As described in Section 2, composite PEs are typically used in tenant
networks where EVPN and IPVPN are both used to provide inter-subnet networks where EVPN and IPVPN are both used to provide inter-subnet
forwarding within the same composite domain. forwarding within the same composite domain.
Figure 6 depicts an example of a composite domain, where PE1/PE2/PE4 Figure 6 depicts an example of a composite domain, where PE1/PE2/PE4
are composite PEs (they support EVPN and IPVPN ISF SAFIs on their are composite PEs (they support EVPN and IPVPN ISF SAFIs on their
peering to the Route Reflector), and PE3 is a regular IPVPN PE. peering to the Route Reflector), and PE3 is a regular IPVPN PE.
+-----------------------------------+ +-----------------------------------+
skipping to change at page 16, line 32 skipping to change at page 16, line 32
+---+ | | +---------------+ | +---+ | | +---------------+ |
|CE1|--+ +----| +------+ +--------------+ |CE1|--+ +----| +------+ +--------------+
+---+ | |IP-VRF| | +---+ | |IP-VRF| |
| | +----| | | | | +----| | |
| | | +------+ | | | | +------+ |
+--------------|MAC-VRF| | +--------------|MAC-VRF| |
| +-------+ | | +-------+ |
+---------------+ +---------------+
Composite PE2 Composite PE2
Figure 6 Composite PE example Figure 6: Composite PE example
In a composite domain with composite and regular PEs: In a composite domain with composite and regular PEs:
o The composite PEs advertise the same IP prefixes in each ISF SAFI o The composite PEs advertise the same IP prefixes in each ISF SAFI
to the RR. For example, in Figure 6, the prefix IP1/24 is to the RR. For example, in Figure 6, the prefix IP1/24 is
advertised by PE1 and PE2 to the RR in two separate NLRIs, one for advertised by PE1 and PE2 to the RR in two separate NLRIs, one for
AFI/SAFI 1/128 and another one for EVPN. AFI/SAFI 1/128 and another one for EVPN.
o The RR does not forward EVPN routes to PE3 (since the RR does not o The RR does not forward EVPN routes to PE3 (since the RR does not
have the EVPN SAFI enabled on its BGP session to PE3), whereas the have the EVPN SAFI enabled on its BGP session to PE3), whereas the
IPVPN routes are forwarded to all the PEs. IPVPN routes are forwarded to all the PEs.
o PE3 receives only the IPVPN route for IP1/24 and resolves the BGP o PE3 receives only the IPVPN route for IP1/24 and resolves the BGP
next-hop to an MPLS tunnel (with IP payload) to PE1 and/or PE2. next-hop to an MPLS tunnel (with IP payload) to PE1 and/or PE2.
o Composite PE4 receives IP1/24 encoded in EVPN and another ISF SAFI o Composite PE4 receives IP1/24 encoded in EVPN and another ISF SAFI
route (EVPN RT-5 and IPVPN). The route selection follows the route (EVPN RT-5 and IPVPN). The route selection follows the
procedures in Section 5. Assuming an EVPN route is selected, PE4 procedures in Section 5. Assuming an EVPN route is selected, PE4
resolves the BGP next-hop to an MPLS tunnel (with Ethernet or IP resolves the BGP next-hop to an MPLS tunnel (with Ethernet or IP
payload) to PE1 and/or PE2. As described in Section 2, two EVPN PEs payload) to PE1 and/or PE2. As described in Section 2, two EVPN
may use tunnels with Ethernet or IP payloads to connect their IP- PEs may use tunnels with Ethernet or IP payloads to connect their
VRFs, depending on the [IP-PREFIX] model implemented. If some IP- VRFs, depending on the
attributes are modified so that the route selection process [I-D.ietf-bess-evpn-prefix-advertisement] model implemented. If
(Section 5) results in PE4 selecting the IPVPN path instead of the some attributes are modified so that the route selection process
EVPN path, the operator should be aware that the EVPN advanced (Section 5) results in PE4 selecting the IPVPN path instead of the
forwarding features, e.g. recursive resolution to overlay indexes, EVPN path, the operator should be aware that the EVPN advanced
will be lost for PE4. forwarding features, e.g. recursive resolution to overlay indexes,
will be lost for PE4.
o The other composite PEs (PE1 and PE2) receive also the same IP o The other composite PEs (PE1 and PE2) receive also the same IP
prefix via EVPN and IPVPN SAFIs and they also follow the route prefix via EVPN and IPVPN SAFIs and they also follow the route
selection in Section 5. selection in Section 5.
o When a given route has been selected as the route for a particular o When a given route has been selected as the route for a particular
packet, the transmission of the packet is done according to the packet, the transmission of the packet is done according to the
rules for that route's AFI/SAFI. rules for that route's AFI/SAFI.
o It is important to note that in composite domains, such as the one o It is important to note that in composite domains, such as the one
in Figure 6, the EVPN advanced forwarding features will only be in Figure 6, the EVPN advanced forwarding features will only be
available to composite and EVPN PEs (assuming they select an RT-5 available to composite and EVPN PEs (assuming they select an RT-5
to forward packets for a given IP prefix), and not to IPVPN PEs. to forward packets for a given IP prefix), and not to IPVPN PEs.
For example, assuming PE1 sends IP1/24 in an EVPN and an IPVPN For example, assuming PE1 sends IP1/24 in an EVPN and an IPVPN
route and the EVPN route is the best one in the selection, the route and the EVPN route is the best one in the selection, the
recursive resolution of the EVPN RT-5s can only be used in PE2 and recursive resolution of the EVPN RT-5s can only be used in PE2 and
PE4 (composite PEs), and not in PE3 (IPVPN PE). As a consequence of PE4 (composite PEs), and not in PE3 (IPVPN PE). As a consequence
this, the indirection provided by the RT5's recursive resolution of this, the indirection provided by the RT5's recursive
and its benefits in a scaled network, will not be available in all resolution and its benefits in a scaled network, will not be
the PEs in the network. available in all the PEs in the network.
7. Gateway PE Procedures 7. Gateway PE Procedures
Section 2 defines a gateway PE as an Interworking PE that advertises Section 2 defines a gateway PE as an Interworking PE that advertises
IP prefixes to different BGP peers, using EVPN to one BGP peer and IP prefixes to different BGP peers, using EVPN to one BGP peer and
another ISF SAFI to another BGP peer. Examples of gateway PEs are another ISF SAFI to another BGP peer. Examples of gateway PEs are
Data Center gateways connecting domains that make use of EVPN and Data Center gateways connecting domains that make use of EVPN and
other ISF SAFIs for a given tenant. Figure 7 illustrates this use- other ISF SAFIs for a given tenant. Figure 7 illustrates this use-
case, in which PE1 and PE2 (and PE3/PE4) are gateway PEs case, in which PE1 and PE2 (and PE3/PE4) are gateway PEs
interconnecting domains for the same tenant. interconnecting domains for the same tenant.
<----EVPN----> <----------IPVPN---------> <----EVPN----> <----EVPN----> <----------IPVPN---------> <----EVPN---->
6500:1:EVPN 6500:2:IPVPN 6500:3:EVPN 6500:1:EVPN 6500:2:IPVPN 6500:3:EVPN
<DOMAIN-ID:ISF_SAFI_TYPE> <DOMAIN-ID:ISF_SAFI_TYPE>
+-----------------------+ +-----------------------+
Gateway PE1 Gateway PE3 Gateway PE1 Gateway PE3
+----------+ +----------+ +----------+ +----------+
+-----------|+------+ | MPLS tnls |+------+ |-------------+ +-----------|+------+ | MPLS tnls |+------+ |-------------+
| ||IP-VRF| | ||IP-VRF| | | | ||IP-VRF| | ||IP-VRF| | |
PE5 |+------+ | |+------+ | PE6 PE5 |+------+ | |+------+ | PE6
+------+ +----------+ +----------+ +------+ +------+ +----------+ +----------+ +------+
|IP-VRF| NVO tnls | | | | NVO tnls |IP-VRF| |IP-VRF| NVO tnls | | | | NVO tnls |IP-VRF|
| | | | | | | | | | | | | | | |
+------+ +----------+ +----------+ +------+ +------+ +----------+ +----------+ +------+
IP1/24--> |+------+ | |+------+ | | IP1/24--> |+------+ | |+------+ | |
| ||IP-VRF| | ||IP-VRF| | | | ||IP-VRF| | ||IP-VRF| | |
+-----------|+------+ | |+------+ |-------------+ +-----------|+------+ | |+------+ |-------------+
+----------+ +----------+ +----------+ +----------+
Gateway PE2 +------+ Gateway PE4 Gateway PE2 +------+ Gateway PE4
+-------|IP-VRF|---------+ +-------|IP-VRF|---------+
| | | |
+------+ +------+
PE7 PE7
Figure 7 Gateway PE example Figure 7: Gateway PE example
The gateway PE procedures are described as follows: The gateway PE procedures are described as follows:
o A gateway PE that imports an ISF SAFI-x route to prefix P in an IP- o A gateway PE that imports an ISF SAFI-x route to prefix P in an
VRF, MUST export P in ISF SAFI-y if: IP-VRF, MUST export P in ISF SAFI-y if:
1. P is installed in the IP-VRF (hence the SAFI-x route is the best 1. P is installed in the IP-VRF (hence the SAFI-x route is the
one for P) and best one for P) and
2. PE has a BGP peer for SAFI-y (enabled for the same IP-VRF) and 2. PE has a BGP peer for SAFI-y (enabled for the same IP-VRF) and
3. Either x or y is EVPN. 3. Either x or y is EVPN
In the example of Figure 7, gateway PE1 and PE2 receive an EVPN In the example of Figure 7, gateway PE1 and PE2 receive an EVPN
RT-5 with IP1/24, install the prefix in the IP-VRF and re- RT-5 with IP1/24, install the prefix in the IP-VRF and re-
advertise it using SAFI 128. advertise it using SAFI 128.
o ISF SAFI routes advertised by a gateway PE MUST include a D-PATH o ISF SAFI routes advertised by a gateway PE MUST include a D-PATH
attribute, so that loops can be detected in remote gateway PEs. attribute, so that loops can be detected in remote gateway PEs.
When a gateway PE propagates an IP prefix between EVPN and another When a gateway PE propagates an IP prefix between EVPN and another
ISF SAFI, it MUST prepend a <DOMAIN-ID:ISF_SAFI_TYPE> to the ISF SAFI, it MUST prepend a <DOMAIN-ID:ISF_SAFI_TYPE> to the
received D-PATH attribute. The DOMAIN-ID and ISF_SAFI_TYPE fields received D-PATH attribute. The DOMAIN-ID and ISF_SAFI_TYPE fields
refer to the domain over which the gateway PE received the IP refer to the domain over which the gateway PE received the IP
prefix and the ISF SAFI of the route, respectively. If the received prefix and the ISF SAFI of the route, respectively. If the
IP prefix route did not include any D-PATH attribute, the gateway received IP prefix route did not include any D-PATH attribute, the
IP MUST add the D-PATH when readvertising. The D-PATH in this case gateway IP MUST add the D-PATH when readvertising. The D-PATH in
will have only one segment on the list, the <DOMAIN- this case will have only one segment on the list, the <DOMAIN-
ID:ISF_SAFI_TYPE> of the received route. ID:ISF_SAFI_TYPE> of the received route.
In the example of Figure 7, gateway PE1/PE2 receive the EVPN RT-5 In the example of Figure 7, gateway PE1/PE2 receive the EVPN RT-5
with no D-PATH attribute since the route is originated at PE5. with no D-PATH attribute since the route is originated at PE5.
Therefore PE1 and PE2 will add the D-PATH attribute including Therefore PE1 and PE2 will add the D-PATH attribute including
<DOMAIN-ID:ISF_SAFI_TYPE> = <6500:1:EVPN>. Gateways PE3/PE4 will <DOMAIN-ID:ISF_SAFI_TYPE> = <6500:1:EVPN>. Gateways PE3/PE4 will
propagate the route again, now prepending their <DOMAIN- propagate the route again, now prepending their <DOMAIN-
ID:ISF_SAFI_TYPE> = <6500:2:IPVPN>. PE6 receives the EVPN RT-5 ID:ISF_SAFI_TYPE> = <6500:2:IPVPN>. PE6 receives the EVPN RT-5
routes with D-PATH = {<6500:2:IPVPN>,<6500:1:EVPN>} and can use routes with D-PATH = {<6500:2:IPVPN>,<6500:1:EVPN>} and can use
that information to make BGP path decisions. that information to make BGP path decisions.
o The gateway PE MAY use the Route Distinguisher of the IP-VRF to o The gateway PE MAY use the Route Distinguisher of the IP-VRF to
readvertise IP prefixes in EVPN or the other ISF SAFI. readvertise IP prefixes in EVPN or the other ISF SAFI.
o The label allocation used by each gateway PE is a local o The label allocation used by each gateway PE is a local
implementation matter. The IP-VRF advertising IP prefixes for EVPN implementation matter. The IP-VRF advertising IP prefixes for
and another ISF SAFI may use a label per-VRF, per-prefix, etc. EVPN and another ISF SAFI may use a label per-VRF, per-prefix,
etc.
o The gateway PE MUST be able to use the same or different set of o The gateway PE MUST be able to use the same or different set of
Route Targets per ISF SAFI on the same IP-VRF. In particular, if Route Targets per ISF SAFI on the same IP-VRF. In particular, if
different domains use different set of Route Targets for the same different domains use different set of Route Targets for the same
tenant, the gateway PE MUST be able to import and export routes tenant, the gateway PE MUST be able to import and export routes
with the different sets. with the different sets.
o Even though Figure 7 only shows two domains per gateway PE, the o Even though Figure 7 only shows two domains per gateway PE, the
gateway PEs may be connected to more than two domains. gateway PEs may be connected to more than two domains.
o There is no limitation of gateway PEs that a given IP prefix can o There is no limitation of gateway PEs that a given IP prefix can
pass through until it reaches a given PE. pass through until it reaches a given PE.
o It is worth noting that an IP prefix that was originated in an EVPN o It is worth noting that an IP prefix that was originated in an
domain but traversed a different ISF SAFI domain, will lose EVPN- EVPN domain but traversed a different ISF SAFI domain, will lose
specific attributes that are used in advanced EVPN procedures. For EVPN- specific attributes that are used in advanced EVPN
example, even if PE1 advertises IP1/24 along with a given non-zero procedures. For example, even if PE1 advertises IP1/24 along with
ESI (for recursive resolution to that ESI), when PE6 receives the a given non-zero ESI (for recursive resolution to that ESI), when
IP prefix in an EVPN route, the ESI value will be zero. This is PE6 receives the IP prefix in an EVPN route, the ESI value will be
because the route traverses an ISF SAFI domain that is different zero. This is because the route traverses an ISF SAFI domain that
than EVPN. is different than EVPN.
8. Interworking Use-Cases 8. Interworking Use-Cases
While Interworking PE networks may well be similar to the examples While Interworking PE networks may well be similar to the examples
described in Sections 6 and 7, in some cases a combination of both described in Sections 6 and 7, in some cases a combination of both
functions may be required. Figure 8 illustrates an example where the functions may be required. Figure 8 illustrates an example where the
gateway PEs are also composite PEs, since not only they need to re- gateway PEs are also composite PEs, since not only they need to re-
advertise IP prefixes from EVPN routes to another ISF SAFI routes, advertise IP prefixes from EVPN routes to another ISF SAFI routes,
but they also need to interwork with IPVPN-only PEs in a domain with but they also need to interwork with IPVPN-only PEs in a domain with
a mix of composite and IPVPN-only PEs. a mix of composite and IPVPN-only PEs.
+-----------------------------------+ +-----------------------------------+
| | | |
| MPLS IPVPN PE3 | MPLS IPVPN PE3
| Network +---------+ | Network +---------+
| IPVPN |+------+ | | IPVPN |+------+ |
skipping to change at page 20, line 37 skipping to change at page 20, line 37
| NVO tnls +----| +------+ |-------------+ | NVO tnls +----| +------+ |-------------+
| | |IP+VRF| | | | |IP+VRF| |
| | +----| | | | | +----| | |
| | | +------+ | | | | +------+ |
| +----+ | |MAC+VRF| | | +----+ | |MAC+VRF| |
+-----|NVE2|---------| +-------+ | +-----|NVE2|---------| +-------+ |
+----+ +---------------+ +----+ +---------------+
| (GW+composite) PE2 | (GW+composite) PE2
TS2 TS2
Figure 8 Gateway and composite combined functions - example Figure 8: Gateway and composite combined functions - example
In the example above, PE1 and PE2 MUST follow the procedures In the example above, PE1 and PE2 MUST follow the procedures
described in Sections 6 and 7. Compared to section 7, PE1 and PE2 now described in Sections 6 and 7. Compared to section 7, PE1 and PE2
need to also propagate prefixes from EVPN to EVPN, in addition to now need to also propagate prefixes from EVPN to EVPN, in addition to
propagating prefixes from EVPN to IPVPN. propagating prefixes from EVPN to IPVPN.
It is worth noting that PE1 and PE2 will receive TS4's IP prefix via It is worth noting that PE1 and PE2 will receive TS4's IP prefix via
IPVPN and RT-5 routes. When readvertising to NVE1 and NVE2, PE1 and IPVPN and RT-5 routes. When readvertising to NVE1 and NVE2, PE1 and
PE2 will consider the D-PATH rules and attributes of the selected PE2 will consider the D-PATH rules and attributes of the selected
route for TS4 (Section 5 describes the Route Selection Process). route for TS4 (Section 5 describes the Route Selection Process).
9. Conclusion 9. Conclusion
This document describes the procedures required in PEs that use EVPN This document describes the procedures required in PEs that use EVPN
and another Inter-Subnet Forwarding SAFI to import and export IP and another Inter-Subnet Forwarding SAFI to import and export IP
prefixes for a given tenant. In particular, this document defines: prefixes for a given tenant. In particular, this document defines:
o A route selection algorithm so that a PE can determine what path to o A route selection algorithm so that a PE can determine what path
choose between EVPN paths and other ISF SAFI paths. to choose between EVPN paths and other ISF SAFI paths.
o A new BGP Path attribute called D-PATH that provides loop o A new BGP Path attribute called D-PATH that provides loop
protection and visibility on the domains a particular route has protection and visibility on the domains a particular route has
traversed. traversed.
o The way Path attributes should be propagated between EVPN and o The way Path attributes should be propagated between EVPN and
another ISF SAFI. another ISF SAFI.
o The procedures that must be followed on Interworking PEs that o The procedures that must be followed on Interworking PEs that
behave as composite PEs, gateway PEs or a combination of both. behave as composite PEs, gateway PEs or a combination of both.
The above procedures provide an operator with the required tools to The above procedures provide an operator with the required tools to
build large tenant networks that may span multiple domains, use build large tenant networks that may span multiple domains, use
different ISF SAFIs to handle IP prefixes, in a deterministic way and different ISF SAFIs to handle IP prefixes, in a deterministic way and
with routing loop protection. with routing loop protection.
10. Conventions used in this document 10. Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP "OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all 14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here. capitals, as shown here.
11. Security Considerations 11. Security Considerations
This section will be added in future versions. This section will be added in future versions.
12. IANA Considerations 12. IANA Considerations
This document defines a new BGP path attribute known as the BGP This document defines a new BGP path attribute known as the BGP
Domain Path (D-PATH) attribute. Domain Path (D-PATH) attribute.
IANA has assigned a new attribute code type from the "BGP Path IANA has assigned a new attribute code type from the "BGP Path
Attributes" subregistry under the "Border Gateway Protocol (BGP) Attributes" subregistry under the "Border Gateway Protocol (BGP)
Parameters" registry: Parameters" registry:
Path Attribute Value Code Reference Path Attribute Value Code Reference
-------------------- ------------------------ --------------- -------------------- ------------------------ ---------------
36 BGP Domain Path (D-PATH) [This document] 36 BGP Domain Path (D-PATH) [This document]
13. References 13. Acknowledgments
13.1. Normative References The authors want to thank Russell Kelly for his review of the D-PATH
section and suggestions.
[RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., 14. Contributors
Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based Ethernet
VPN", RFC 7432, DOI 10.17487/RFC7432, February 2015, <http://www.rfc-
editor.org/info/rfc7432>.
[RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A 15. References
Border Gateway Protocol 4 (BGP-4)", RFC 4271, DOI 10.17487/RFC4271,
January 2006, <http://www.rfc-editor.org/info/rfc4271>. 15.1. Normative References
[RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A.,
Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based
Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February
2015, <https://www.rfc-editor.org/info/rfc7432>.
[RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A
Border Gateway Protocol 4 (BGP-4)", RFC 4271,
DOI 10.17487/RFC4271, January 2006,
<https://www.rfc-editor.org/info/rfc4271>.
[RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private
Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February 2006, Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February
<https://www.rfc-editor.org/info/rfc4364>. 2006, <https://www.rfc-editor.org/info/rfc4364>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March Requirement Levels", BCP 14, RFC 2119,
1997, <https://www.rfc-editor.org/info/rfc2119>. DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
<https://www.rfc-editor.org/info/rfc8174>. May 2017, <https://www.rfc-editor.org/info/rfc8174>.
13.2. Informative References 15.2. Informative References
[RFC4360] Sangli, S., Tappan, D., and Y. Rekhter, "BGP Extended [RFC4360] Sangli, S., Tappan, D., and Y. Rekhter, "BGP Extended
Communities Attribute", RFC 4360, DOI 10.17487/RFC4360, February Communities Attribute", RFC 4360, DOI 10.17487/RFC4360,
2006, <http://www.rfc-editor.org/info/rfc4360>. February 2006, <https://www.rfc-editor.org/info/rfc4360>.
[IP-PREFIX] Rabadan et al., "IP Prefix Advertisement in EVPN", [I-D.ietf-bess-evpn-prefix-advertisement]
draft-ietf-bess-evpn-prefix-advertisement-11, May, 2018. Rabadan, J., Henderickx, W., Drake, J., Lin, W., and A.
Sajassi, "IP Prefix Advertisement in EVPN", draft-ietf-
bess-evpn-prefix-advertisement-11 (work in progress), May
2018.
[INTER-SUBNET] Sajassi et al., "IP Inter-Subnet Forwarding in EVPN", [I-D.ietf-bess-evpn-inter-subnet-forwarding]
draft-ietf-bess-evpn-inter-subnet-forwarding-08.txt, work in Sajassi, A., Salam, S., Thoria, S., Drake, J., and J.
progress, March, 2019. Rabadan, "Integrated Routing and Bridging in EVPN", draft-
ietf-bess-evpn-inter-subnet-forwarding-08 (work in
progress), March 2019.
[RFC6472] Kumari, W. and K. Sriram, "Recommendation for Not Using [RFC6472] Kumari, W. and K. Sriram, "Recommendation for Not Using
AS_SET and AS_CONFED_SET in BGP", BCP 172, RFC 6472, DOI AS_SET and AS_CONFED_SET in BGP", BCP 172, RFC 6472,
10.17487/RFC6472, December 2011, <https://www.rfc- DOI 10.17487/RFC6472, December 2011,
editor.org/info/rfc6472>. <https://www.rfc-editor.org/info/rfc6472>.
14. Acknowledgments
The authors want to thank Russell Kelly for his review of the D-PATH
section and suggestions.
15. Contributors
16. Authors' Addresses Authors' Addresses
Jorge Rabadan (editor) J. Rabadan (editor)
Nokia Nokia
777 E. Middlefield Road 777 Middlefield Road
Mountain View, CA 94043 USA Mountain View, CA 94043
USA
Email: jorge.rabadan@nokia.com Email: jorge.rabadan@nokia.com
Ali Sajassi (editor) A. Sajassi (editor)
Cisco Cisco
170 West Tasman Drive 225 West Tasman Drive
San Jose, CA 95134, US San Jose, CA 95134
EMail: sajassi@cisco.com USA
Eric C. Rosen Email: sajassi@cisco.com
EMail: erosen52@gmail.com
John Drake E. Rosen
Juniper Networks, Inc. Individual
EMail: jdrake@juniper.net
Wen Lin Email: erosen52@gmail.com
Juniper Networks, Inc.
EMail: wlin@juniper.net
Jim Uttaro J. Drake
Juniper
Email: jdrake@juniper.net
W. Lin
Juniper
Email: wlin@juniper.net
J. Uttaro
AT&T AT&T
Email: ju1738@att.com Email: ju1738@att.com
Adam Simpson A. Simpson
Nokia Nokia
Email: adam.1.simpson@nokia.com Email: adam.1.simpson@nokia.com
 End of changes. 185 change blocks. 
528 lines changed or deleted 540 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/