< draft-ietf-bess-evpn-inter-subnet-forwarding-10.txt   draft-ietf-bess-evpn-inter-subnet-forwarding-11.txt >
BESS WorkGroup A. Sajassi BESS WorkGroup A. Sajassi
Internet-Draft S. Salam Internet-Draft S. Salam
Intended status: Standards Track S. Thoria Intended status: Standards Track S. Thoria
Expires: March 7, 2021 Cisco Systems Expires: April 16, 2021 Cisco Systems
J. Drake J. Drake
Juniper Juniper
J. Rabadan J. Rabadan
Nokia Nokia
September 3, 2020 October 13, 2020
Integrated Routing and Bridging in EVPN Integrated Routing and Bridging in EVPN
draft-ietf-bess-evpn-inter-subnet-forwarding-10 draft-ietf-bess-evpn-inter-subnet-forwarding-11
Abstract Abstract
Ethernet VPN (EVPN) provides an extensible and flexible multi-homing Ethernet VPN (EVPN) provides an extensible and flexible multi-homing
VPN solution over an MPLS/IP network for intra-subnet connectivity VPN solution over an MPLS/IP network for intra-subnet connectivity
among Tenant Systems and End Devices that can be physical or virtual. among Tenant Systems and End Devices that can be physical or virtual.
However, there are scenarios for which there is a need for a dynamic However, there are scenarios for which there is a need for a dynamic
and efficient inter-subnet connectivity among these Tenant Systems and efficient inter-subnet connectivity among these Tenant Systems
and End Devices while maintaining the multi-homing capabilities of and End Devices while maintaining the multi-homing capabilities of
EVPN. This document describes an Integrated Routing and Bridging EVPN. This document describes an Integrated Routing and Bridging
skipping to change at page 2, line 4 skipping to change at page 2, line 4
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on March 7, 2021. This Internet-Draft will expire on April 16, 2021.
Copyright Notice Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 32 skipping to change at page 2, line 32
1. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4
3. EVPN PE Model for IRB Operation . . . . . . . . . . . . . . . 6 3. EVPN PE Model for IRB Operation . . . . . . . . . . . . . . . 6
4. Symmetric and Asymmetric IRB . . . . . . . . . . . . . . . . 7 4. Symmetric and Asymmetric IRB . . . . . . . . . . . . . . . . 7
4.1. IRB Interface and its MAC and IP addresses . . . . . . . 10 4.1. IRB Interface and its MAC and IP addresses . . . . . . . 10
5. Symmetric IRB Procedures . . . . . . . . . . . . . . . . . . 12 5. Symmetric IRB Procedures . . . . . . . . . . . . . . . . . . 12
5.1. Control Plane - Advertising PE . . . . . . . . . . . . . 12 5.1. Control Plane - Advertising PE . . . . . . . . . . . . . 12
5.2. Control Plane - Receiving PE . . . . . . . . . . . . . . 13 5.2. Control Plane - Receiving PE . . . . . . . . . . . . . . 13
5.3. Subnet route advertisement . . . . . . . . . . . . . . . 14 5.3. Subnet route advertisement . . . . . . . . . . . . . . . 14
5.4. Data Plane - Ingress PE . . . . . . . . . . . . . . . . . 14 5.4. Data Plane - Ingress PE . . . . . . . . . . . . . . . . . 15
5.5. Data Plane - Egress PE . . . . . . . . . . . . . . . . . 15 5.5. Data Plane - Egress PE . . . . . . . . . . . . . . . . . 15
6. Asymmetric IRB Procedures . . . . . . . . . . . . . . . . . . 16 6. Asymmetric IRB Procedures . . . . . . . . . . . . . . . . . . 16
6.1. Control Plane - Advertising PE . . . . . . . . . . . . . 16 6.1. Control Plane - Advertising PE . . . . . . . . . . . . . 16
6.2. Control Plane - Receiving PE . . . . . . . . . . . . . . 16 6.2. Control Plane - Receiving PE . . . . . . . . . . . . . . 17
6.3. Data Plane - Ingress PE . . . . . . . . . . . . . . . . . 18 6.3. Data Plane - Ingress PE . . . . . . . . . . . . . . . . . 18
6.4. Data Plane - Egress PE . . . . . . . . . . . . . . . . . 18 6.4. Data Plane - Egress PE . . . . . . . . . . . . . . . . . 18
7. Mobility Procedure . . . . . . . . . . . . . . . . . . . . . 19 7. Mobility Procedure . . . . . . . . . . . . . . . . . . . . . 19
7.1. Initiating a gratutious ARP upon a Move . . . . . . . . . 20 7.1. Initiating a gratutious ARP upon a Move . . . . . . . . . 20
7.2. Sending Data Traffic without an ARP Request . . . . . . . 20 7.2. Sending Data Traffic without an ARP Request . . . . . . . 21
7.3. Silent Host . . . . . . . . . . . . . . . . . . . . . . . 22 7.3. Silent Host . . . . . . . . . . . . . . . . . . . . . . . 22
8. BGP Encoding . . . . . . . . . . . . . . . . . . . . . . . . 23 8. BGP Encoding . . . . . . . . . . . . . . . . . . . . . . . . 23
8.1. Router's MAC Extended Community . . . . . . . . . . . . . 23 8.1. Router's MAC Extended Community . . . . . . . . . . . . . 23
9. Operational Models for Symmetric Inter-Subnet Forwarding . . 24 9. Operational Models for Symmetric Inter-Subnet Forwarding . . 24
9.1. IRB forwarding on NVEs for Tenant Systems . . . . . . . . 24 9.1. IRB forwarding on NVEs for Tenant Systems . . . . . . . . 24
9.1.1. Control Plane Operation . . . . . . . . . . . . . . . 25 9.1.1. Control Plane Operation . . . . . . . . . . . . . . . 25
9.1.2. Data Plane Operation . . . . . . . . . . . . . . . . 27 9.1.2. Data Plane Operation . . . . . . . . . . . . . . . . 27
9.2. IRB forwarding on NVEs for Subnets behind Tenant Systems 28 9.2. IRB forwarding on NVEs for Subnets behind Tenant Systems 28
9.2.1. Control Plane Operation . . . . . . . . . . . . . . . 29 9.2.1. Control Plane Operation . . . . . . . . . . . . . . . 29
9.2.2. Data Plane Operation . . . . . . . . . . . . . . . . 30 9.2.2. Data Plane Operation . . . . . . . . . . . . . . . . 30
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 31 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 31
11. Security Considerations . . . . . . . . . . . . . . . . . . . 31 11. Security Considerations . . . . . . . . . . . . . . . . . . . 32
12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32
13. References . . . . . . . . . . . . . . . . . . . . . . . . . 32 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 32
13.1. Normative References . . . . . . . . . . . . . . . . . . 32 13.1. Normative References . . . . . . . . . . . . . . . . . . 33
13.2. Informative References . . . . . . . . . . . . . . . . . 33 13.2. Informative References . . . . . . . . . . . . . . . . . 34
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 34 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 34
1. Terminology 1. Terminology
AC: Attachment Circuit AC: Attachment Circuit
ARP: Address Resolution Protocol ARP: Address Resolution Protocol
BD: Broadcast Domain. As per [RFC7432], an EVI consists of a single BD: Broadcast Domain. As per [RFC7432], an EVI consists of a single
or multiple BDs. In case of VLAN-bundle and VLAN-based service or multiple BDs. In the case of VLAN-bundle and VLAN-based service
models (see [RFC7432]), a BD is equivalent to an EVI. In case of models (see [RFC7432]), a BD is equivalent to an EVI. In the case of
VLAN-aware bundle service model, an EVI contains multiple BDs. Also, VLAN-aware bundle service model, an EVI contains multiple BDs. Also,
in this document, BD and subnet are equivalent terms and wherever in this document, BD and subnet are equivalent terms and wherever
"subnet" is used, it means "IP subnet" "subnet" is used, it means "IP subnet"
BD Route Target: refers to the Broadcast Domain assigned Route Target BD Route Target: refers to the Broadcast Domain assigned Route Target
[RFC4364]. In case of VLAN-aware bundle service model, all the BD [RFC4364]. In the case of VLAN-aware bundle service model, all the
instances in the MAC-VRF share the same Route Target BD instances in the MAC-VRF share the same Route Target
BT: Bridge Table. The instantiation of a BD in a MAC-VRF, as per BT: Bridge Table. The instantiation of a BD in a MAC-VRF, as per
[RFC7432]. [RFC7432].
Ethernet NVO tunnel: refers to Network Virtualization Overlay tunnels Ethernet NVO tunnel: refers to Network Virtualization Overlay tunnels
with Ethernet payload. Examples of this type of tunnels are VXLAN or with Ethernet payload as specified for VxLAN in [RFC7348] and for
GENEVE. NVGRE in [RFC7637].
EVI: EVPN Instance spanning the NVE/PE devices that are participating EVI: EVPN Instance spanning the NVE/PE devices that are participating
on that EVPN, as per [RFC7432]. on that EVPN, as per [RFC7432].
EVPN: Ethernet Virtual Private Networks, as per [RFC7432]. EVPN: Ethernet Virtual Private Networks, as per [RFC7432].
IP NVO tunnel: it refers to Network Virtualization Overlay tunnels IP NVO tunnel: it refers to Network Virtualization Overlay tunnels
with IP payload (no MAC header in the payload). with IP payload (no MAC header in the payload) as specified for GPE
in [I-D.ietf-nvo3-vxlan-gpe].
IP-VRF: A VPN Routing and Forwarding table for IP routes on an NVE/ IP-VRF: A Virtual Routing and Forwarding table for IP routes on an
PE. The IP routes could be populated by EVPN and IP-VPN address NVE/PE. The IP routes could be populated by EVPN and IP-VPN address
families. An IP-VRF is also an instantiation of a layer 3 VPN in an families. An IP-VRF is also an instantiation of a layer 3 VPN in an
NVE/PE. NVE/PE.
IRB: Integrated Routing and Bridging interface. It connects an IP- IRB: Integrated Routing and Bridging interface. It connects an IP-
VRF to a BD (or subnet). VRF to a BD (or subnet).
MAC-VRF: A Virtual Routing and Forwarding table for Media Access MAC-VRF: A Virtual Routing and Forwarding table for Media Access
Control (MAC) addresses on an NVE/PE, as per [RFC7432]. A MAC-VRF is Control (MAC) addresses on an NVE/PE, as per [RFC7432]. A MAC-VRF is
also an instantiation of an EVI in an NVE/PE. also an instantiation of an EVI in an NVE/PE.
ND: Neighbor Discovery Protocol ND: Neighbor Discovery Protocol
NVE: Network Virtualization Edge NVE: Network Virtualization Edge
GENEVE: Generic Network Virtualization Encapsulation, [GENEVE] NVGRE: Network Virtualization Generic Routing Encapsulation,
[RFC7637]
NVGRE: Network Virtualization Generic Routing Encapsulation
NVO: Network Virtualization Overlays NVO: Network Virtualization Overlays
RT-2: EVPN route type 2, i.e., MAC/IP Advertisement route, as defined RT-2: EVPN route type 2, i.e., MAC/IP Advertisement route, as defined
in [RFC7432] in [RFC7432]
RT-5: EVPN route type 5, i.e., IP Prefix route. As defined in RT-5: EVPN route type 5, i.e., IP Prefix route. As defined in
Section 3 of [I-D.ietf-bess-evpn-prefix-advertisement] Section 3 of [I-D.ietf-bess-evpn-prefix-advertisement]
TS: Tenant System TS: Tenant System
VA: Virtual Appliance VA: Virtual Appliance
VNI: Virtual Network Identifier. As in [RFC8365], the term is used VNI: Virtual Network Identifier. As in [RFC8365], the term is used
as a representation of a 24-bit NVO instance identifier, with the as a representation of a 24-bit NVO instance identifier, with the
understanding that VNI will refer to a VXLAN Network Identifier in understanding that VNI will refer to a VXLAN Network Identifier in
VXLAN, or Virtual Network Identifier in GENEVE, etc. unless it is VXLAN, or Virtual Subnet Identifier in NVGRE, etc. unless it is
stated otherwise. stated otherwise.
VTEP: VXLAN Termination End Point, as in [RFC7348]. VTEP: VXLAN Termination End Point, as in [RFC7348].
VXLAN: Virtual Extensible LAN, as in [RFC7348]. VXLAN: Virtual Extensible LAN, as in [RFC7348].
This document also assumes familiarity with the terminology of This document also assumes familiarity with the terminology of
[RFC7432], [RFC8365] and [RFC7365]. [RFC7432], [RFC8365] and [RFC7365].
2. Introduction 2. Introduction
EVPN [RFC7432] provides an extensible and flexible multi-homing VPN EVPN [RFC7432] provides an extensible and flexible multi-homing VPN
solution over an MPLS/IP network for intra-subnet connectivity among solution over an MPLS/IP network for intra-subnet connectivity among
Tenant Systems (TSes) and End Devices that can be physical or Tenant Systems (TSes) and End Devices that can be physical or
virtual; where an IP subnet is represented by an EVI for a VLAN-based virtual; where an IP subnet is represented by an EVPN Instance (EVI)
service or by an (EVI, VLAN) for a VLAN-aware bundle service. for a VLAN-based service or by an (EVI, VLAN) for a VLAN-aware bundle
However, there are scenarios for which there is a need for a dynamic service. However, there are scenarios for which there is a need for
and efficient inter-subnet connectivity among these Tenant Systems a dynamic and efficient inter-subnet connectivity among these Tenant
and End Devices while maintaining the multi-homing capabilities of Systems and End Devices while maintaining the multi-homing
EVPN. This document describes an Integrated Routing and Bridging capabilities of EVPN. This document describes an Integrated Routing
(IRB) solution based on EVPN to address such requirements. and Bridging (IRB) solution based on EVPN to address such
requirements.
The inter-subnet communication is traditionally achieved at The inter-subnet communication is traditionally achieved at
centralized L3 Gateway (L3GW) devices where all the inter-subnet centralized L3 Gateway (L3GW) devices where all the inter-subnet
forwarding is performed and all the inter-subnet communication forwarding is performed and all the inter-subnet communication
policies are enforced. When two TSes belonging to two different policies are enforced. When two TSes belonging to two different
subnets connected to the same PE wanted to communicate with each subnets connected to the same PE wanted to communicate with each
other, their traffic needed to be back hauled from the PE all the way other, their traffic needed to be backhauled from the PE all the way
to the centralized gateway where inter-subnet switching is performed to the centralized gateway where inter-subnet switching is performed
and then back to the PE. For today's large multi-tenant data center, and then back to the PE. For today's large multi-tenant data center,
this scheme is very inefficient and sometimes impractical. this scheme is very inefficient and sometimes impractical.
In order to overcome the drawback of centralized layer-3 GW approach, In order to overcome the drawback of the centralized layer-3 GW
IRB functionality is needed on the PEs (also referred to as EVPN approach, IRB functionality is needed on the PEs (also referred to as
NVEs) attached to TSes in order to avoid inefficient forwarding of EVPN NVEs) attached to TSes in order to avoid inefficient forwarding
tenant traffic (i.e., avoid back-hauling and hair-pinning). When a of tenant traffic (i.e., avoid back-hauling and hair-pinning). When
PE with IRB capability receives tenant traffic over an Attachment a PE with IRB capability receives tenant traffic over an Attachment
Circuit (AC), it can not only locally bridge the tenant intra-subnet Circuit (AC), it can not only locally bridge the tenant intra-subnet
traffic but also can locally route the tenant inter-subnet traffic on traffic but also can locally route the tenant inter-subnet traffic on
a packet by packet basis thus meeting the requirements for both intra a packet by packet basis thus meeting the requirements for both intra
and inter-subnet forwarding and avoiding non-optimal traffic and inter-subnet forwarding and avoiding non-optimal traffic
forwarding associated with centralized layer-3 GW approach. forwarding associated with centralized layer-3 GW approach.
Some TSes run non-IP protocols in conjunction with their IP traffic. Some TSes run non-IP protocols in conjunction with their IP traffic.
Therefore, it is important to handle both kinds of traffic optimally Therefore, it is important to handle both kinds of traffic optimally
- e.g., to bridge non-IP and intra-subnet traffic and to route inter- - e.g., to bridge non-IP and intra-subnet traffic and to route inter-
subnet IP traffic. Therefore, the solution needs to meet the subnet IP traffic. Therefore, the solution needs to meet the
following requirements: following requirements:
R1: The solution MUST allow for both inter-subnet and intra-subnet R1: The solution must allow for both inter-subnet and intra-subnet
traffic belonging to the same tenant to be locally routed and bridged traffic belonging to the same tenant to be locally routed and bridged
respectively. The solution MUST provide IP routing for inter-subnet respectively. The solution must provide IP routing for inter-subnet
traffic and Ethernet Bridging for intra-subnet traffic. It should be traffic and Ethernet Bridging for intra-subnet traffic. It should be
noted that if an IP-VRF in a NVE is configured for IPv6 and that NVE noted that if an IP-VRF in a NVE is configured for IPv6 and that NVE
receives IPv4 traffic on the corresponding VLAN, then the IPv4 receives IPv4 traffic on the corresponding VLAN, then the IPv4
traffic is treated as L2 traffic and it is bridged. Also vise versa, traffic is treated as L2 traffic and it is bridged. Also vise versa,
if an IP-VRF in a NVE is configured for IPv4 and that NVE receives if an IP-VRF in a NVE is configured for IPv4 and that NVE receives
IPv6 traffic on the corresponding VLAN, then the IPv6 traffic is IPv6 traffic on the corresponding VLAN, then the IPv6 traffic is
treated as L2 traffic and it is bridged. treated as L2 traffic and it is bridged.
R2: The solution MUST support bridging for non-IP traffic. R2: The solution must support bridging for non-IP traffic.
R3: The solution MUST allow inter-subnet switching to be disabled on R3: The solution must allow inter-subnet switching to be disabled on
a per VLAN basis on PEs where the traffic needs to be back hauled to a per VLAN basis on PEs where the traffic needs to be backhauled to
another node (i.e., for performing FW or DPI functionality). another node (i.e., for performing FW or DPI functionality).
3. EVPN PE Model for IRB Operation 3. EVPN PE Model for IRB Operation
Since this document discusses IRB operation in relationship to EVPN Since this document discusses IRB operation in relationship to EVPN
MAC-VRF, IP-VRF, EVI, Bridge Domain (BD), Bridge Table (BT), and IRB MAC-VRF, IP-VRF, EVI, Bridge Domain (BD), Bridge Table (BT), and IRB
interfaces, it is important to understand the relationship among interfaces, it is important to understand the relationship between
these components. Therefore, the following PE model is illustrated these components. Therefore, the following PE model is illustrated
below to a) describe these components and b) illustrate the below to a) describe these components and b) illustrate the
relationship among them. relationship among them.
+-------------------------------------------------------------+ +-------------------------------------------------------------+
| | | |
| +------------------+ IRB PE | | +------------------+ IRB PE |
| Attachment | +------------------+ | | Attachment | +------------------+ |
| Circuit(AC1) | | +----------+ | MPLS/NVO tnl | Circuit(AC1) | | +----------+ | MPLS/NVO tnl
----------------------*Bridge | | +----- ----------------------*Bridge | | +-----
skipping to change at page 7, line 8 skipping to change at page 7, line 8
Instance) in a PE. A MAC-VRF consists of one or more Bridge Tables Instance) in a PE. A MAC-VRF consists of one or more Bridge Tables
(BTs) where each BT corresponds to a VLAN (broadcast domain - BD). (BTs) where each BT corresponds to a VLAN (broadcast domain - BD).
If service interfaces for an EVPN PE are configured in VLAN- Based If service interfaces for an EVPN PE are configured in VLAN- Based
mode (i.e., section 6.1 of RFC7432), then there is only a single BT mode (i.e., section 6.1 of RFC7432), then there is only a single BT
per MAC-VRF (per EVI) - i.e., there is only one tenant VLAN per EVI. per MAC-VRF (per EVI) - i.e., there is only one tenant VLAN per EVI.
However, if service interfaces for an EVPN PE are configured in VLAN- However, if service interfaces for an EVPN PE are configured in VLAN-
Aware Bundle mode (i.e., section 6.3 of RFC7432), then there are Aware Bundle mode (i.e., section 6.3 of RFC7432), then there are
several BTs per MAC-VRF (per EVI) - i.e., there are several tenant several BTs per MAC-VRF (per EVI) - i.e., there are several tenant
VLANs per EVI. VLANs per EVI.
Each BT is connected to a IP-VRF via a L3 interface called IRB Each BT is connected to an IP-VRF via an L3 interface called IRB
interface. Since a single tenant subnet is typically (and in this interface. Since a single tenant subnet is typically (and in this
document) represented by a VLAN (and thus supported by a single BT), document) represented by a VLAN (and thus supported by a single BT),
for a given tenant there are as many BTs as there are subnets and for a given tenant there are as many BTs as there are subnets and
thus there are also as many IRB interfaces between the tenant IP-VRF thus there are also as many IRB interfaces between the tenant IP-VRF
and the associated BTs as shown in the PE model above. and the associated BTs as shown in the PE model above.
IP-VRF is identified by its corresponding route target and route IP-VRF is identified by its corresponding route target and route
distinguisher and MAC-VRF is also identified by its corresponding distinguisher and MAC-VRF is also identified by its corresponding
route target and route distinguisher. If operating in EVPN VLAN- route target and route distinguisher. If operating in EVPN VLAN-
Based mode, then a receiving PE that receives an EVPN route with MAC- Based mode, then a receiving PE that receives an EVPN route with MAC-
skipping to change at page 8, line 4 skipping to change at page 8, line 4
to ARP or ND protocol tables. to ARP or ND protocol tables.
o references to host IP lookup followed by a host MAC lookup in the o references to host IP lookup followed by a host MAC lookup in the
context of asymmetric IRB MAY be collapsed into a single IP lookup context of asymmetric IRB MAY be collapsed into a single IP lookup
in a hardware implementation. in a hardware implementation.
In symmetric IRB as its name implies, the lookup operation is In symmetric IRB as its name implies, the lookup operation is
symmetric at both ingress and egress PEs - i.e., both ingress and symmetric at both ingress and egress PEs - i.e., both ingress and
egress PEs perform lookups on both MAC and IP addresses. The ingress egress PEs perform lookups on both MAC and IP addresses. The ingress
PE performs a MAC lookup followed by an IP lookup and the egress PE PE performs a MAC lookup followed by an IP lookup and the egress PE
performs a IP lookup followed by a MAC lookup as depicted in the performs an IP lookup followed by a MAC lookup as depicted in the
following figure. following figure.
Ingress PE Egress PE Ingress PE Egress PE
+-------------------+ +------------------+ +-------------------+ +------------------+
| | | | | | | |
| +-> IP-VRF ----|---->---|-----> IP-VRF -+ | | +-> IP-VRF ----|---->---|-----> IP-VRF -+ |
| | | | | | | | | | | |
| BT1 BT2 | | BT3 BT2 | | BT1 BT2 | | BT3 BT2 |
| | | | | | | | | | | |
| ^ | | v | | ^ | | v |
| | | | | | | | | | | |
+-------------------+ +------------------+ +-------------------+ +------------------+
^ | ^ |
| | | |
TS1->-+ +->-TS2 TS1->-+ +->-TS2
Figure 2: Symmetric IRB Figure 2: Symmetric IRB
In symmetric IRB as shown in figure-2, the inter-subnet forwarding In symmetric IRB as shown in figure-2, the inter-subnet forwarding
between two PEs is done between their associated IP-VRFs. Therefore, between two PEs is done between their associated IP-VRFs. Therefore,
the tunnel connecting these IP-VRFs can be either IP-only tunnel the tunnel connecting these IP-VRFs can be either IP-only tunnel
(e.g., in case of MPLS or GENEVE encapsulation) or Ethernet NVO (e.g., in case of MPLS or GPE encapsulation) or Ethernet NVO tunnel
tunnel (e.g., in case of VxLAN encapsulation). If it is an Ethernet (e.g., in case of VxLAN encapsulation). If it is an Ethernet NVO
NVO tunnel, the TS1's IP packet is encapsulated in an Ethernet header tunnel, the TS1's IP packet is encapsulated in an Ethernet header
consisting of ingress and egress PEs MAC addresses - i.e., there is consisting of ingress and egress PEs MAC addresses - i.e., there is
no need for ingress PE to use the destination TS2's MAC address. no need for ingress PE to use the destination TS2's MAC address.
Therefore, in symmetric IRB, there is no need for the ingress PE to Therefore, in symmetric IRB, there is no need for the ingress PE to
maintain ARP entries for destination TS2's IP and MAC addresses maintain ARP entries for destination TS2's IP and MAC addresses
association in its ARP table. Each PE participating in symmetric IRB association in its ARP table. Each PE participating in symmetric IRB
only maintains ARP entries for locally connected hosts and maintains only maintains ARP entries for locally connected hosts and maintains
MAC-VRFs/BTs for only locally configured subnets. MAC-VRFs/BTs for only locally configured subnets.
In asymmetric IRB, the lookup operation is asymmetric and the ingress In asymmetric IRB, the lookup operation is asymmetric and the ingress
PE performs three lookups; whereas the egress PE performs a single PE performs three lookups; whereas the egress PE performs a single
skipping to change at page 9, line 36 skipping to change at page 9, line 36
ingress PE to perform such encapsulation, it needs to maintain TS2's ingress PE to perform such encapsulation, it needs to maintain TS2's
IP and MAC address association in its ARP table. Furthermore, it IP and MAC address association in its ARP table. Furthermore, it
needs to maintain destination TS2's MAC address in the corresponding needs to maintain destination TS2's MAC address in the corresponding
BT even though it may not have any TSes of the corresponding subnet BT even though it may not have any TSes of the corresponding subnet
locally attached. In other words, each PE participating in locally attached. In other words, each PE participating in
asymmetric IRB MUST maintain ARP entries for remote hosts (hosts asymmetric IRB MUST maintain ARP entries for remote hosts (hosts
connected to other PEs) as well as maintain MAC-VRFs/BTs and IRB connected to other PEs) as well as maintain MAC-VRFs/BTs and IRB
interfaces for ALL subnets in an IP VRF including subnets that may interfaces for ALL subnets in an IP VRF including subnets that may
not be locally attached. Therefore, careful consideration of PE not be locally attached. Therefore, careful consideration of PE
scale aspects for its ARP table size, its IRB interfaces, number and scale aspects for its ARP table size, its IRB interfaces, number and
size of its bridge tables should be given for application of size of its bridge tables should be given for the application of
asymmetric IRB. asymmetric IRB.
It should be noted that whenever a PE performs a host IP lookup for a
packet, IPv4 TTL or IPv6 hop limit for that packet is decremented by
one and if it reaches zero, the packet is discarded. In the case of
symmetric IRB, the TTL/hop limit is decremented by both ingress and
egress PEs (once by each); whereas, in the case of asymmetric IRB,
the TTL/hop limit is decremented only once by the ingress PE.
The following subsection defines the control and data planes The following subsection defines the control and data planes
procedures for symmetric and asymmetric IRB on ingress and egress procedures for symmetric and asymmetric IRB on ingress and egress
PEs. The following figure is used in description of these procedures PEs. The following figure is used to describe these procedures where
where it shows a single IP-VRF and a number of BTs on each PE for a it shows a single IP-VRF and a number of BTs on each PE for a given
given tenant. The IP-VRF of the tenant (i.e., IP-VRF1) is connected tenant. The IP-VRF of the tenant (i.e., IP-VRF1) is connected to
to each BT via its associated IRB interface. Each BT on a PE is each BT via its associated IRB interface. Each BT on a PE is
associated with a unique VLAN (e.g., with a BD) where in turn it is associated with a unique VLAN (e.g., with a BD) where in turn it is
associated with a single MAC-VRF in case of VLAN-Based mode or a associated with a single MAC-VRF in the case of VLAN-Based mode or a
number of BTs can be associated with a single MAC-VRF in case of number of BTs can be associated with a single MAC-VRF in the case of
VLAN-Aware Bundle mode. Whether the service interface on a PE is VLAN-Aware Bundle mode. Whether the service interface on a PE is
VLAN-Based or VLAN-Aware Bundle mode does not impact the IRB VLAN-Based or VLAN-Aware Bundle mode does not impact the IRB
operation and procedures. It mainly impacts the setting of Ethernet operation and procedures. It mainly impacts the setting of Ethernet
tag field in EVPN BGP routes as described in section 6 of [RFC7432]. tag field in EVPN BGP routes as described in section 6 of [RFC7432].
PE 1 +---------+ PE 1 +---------+
+-------------+ | | +-------------+ | |
TS1-----| MACx| | | PE2 TS1-----| MACx| | | PE2
(IP1/M1) |(BT1) | | | +-------------+ (IP1/M1) |(BT1) | | | +-------------+
TS5-----| \ | | MPLS/ | |MACy (BT3) |-----TS3 TS5-----| \ | | MPLS/ | |MACy (BT3) |-----TS3
skipping to change at page 12, line 19 skipping to change at page 12, line 31
5. Symmetric IRB Procedures 5. Symmetric IRB Procedures
5.1. Control Plane - Advertising PE 5.1. Control Plane - Advertising PE
When a PE (e.g., PE1 in figure 4 above) learns MAC and IP address of When a PE (e.g., PE1 in figure 4 above) learns MAC and IP address of
a TS (e.g., via an ARP request or Neighbor Solicitation), it adds the a TS (e.g., via an ARP request or Neighbor Solicitation), it adds the
MAC address to the corresponding MAC-VRF/BT of that tenant's subnet MAC address to the corresponding MAC-VRF/BT of that tenant's subnet
and adds the IP address to the IP-VRF for that tenant. Furthermore, and adds the IP address to the IP-VRF for that tenant. Furthermore,
it adds this TS's MAC and IP address association to its ARP table or it adds this TS's MAC and IP address association to its ARP table or
NDP cahce. It then builds an EVPN MAC/IP Advertisement route (type NDP cache. It then builds an EVPN MAC/IP Advertisement route (type
2) as follows and advertises it to other PEs participating in that 2) as follows and advertises it to other PEs participating in that
tenant's VPN. tenant's VPN.
o The Length field of the BGP EVPN NLRI for an EVPN MAC/IP o The Length field of the BGP EVPN NLRI for an EVPN MAC/IP
Advertisement route MUST be either 40 (if IPv4 address is carried) Advertisement route MUST be either 40 (if IPv4 address is carried)
or 52 (if IPv6 address is carried). or 52 (if IPv6 address is carried).
o Route Distinguisher (RD), Ethernet Segment Identifier, Ethernet o Route Distinguisher (RD), Ethernet Segment Identifier, Ethernet
Tag ID, MAC Address Length, MAC Address, IP Address Length, IP Tag ID, MAC Address Length, MAC Address, IP Address Length, IP
Address, and MPLS Label1 fields MUST be set per [RFC7432] and Address, and MPLS Label1 fields MUST be set per [RFC7432] and
[RFC8365]. [RFC8365].
o The MPLS Label2 field is set to either an MPLS label or a VNI o The MPLS Label2 field is set to either an MPLS label or a VNI
corresponding to the tenant's IP-VRF. In case of an MPLS label, corresponding to the tenant's IP-VRF. In the case of an MPLS
this field is encoded as 3 octets, where the high-order 20 bits label, this field is encoded as 3 octets, where the high-order 20
contain the label value. bits contain the label value.
Just as in [RFC7432], the RD, Ethernet Tag ID, MAC Address Length, Just as in [RFC7432], the RD, Ethernet Tag ID, MAC Address Length,
MAC Address, IP Address Length, and IP Address fields are part of the MAC Address, IP Address Length, and IP Address fields are part of the
route key used by BGP to compare routes. The rest of the fields are route key used by BGP to compare routes. The rest of the fields are
not part of the route key. not part of the route key.
This route is advertised along with the following two extended This route is advertised along with the following two extended
communities: communities:
1. Tunnel Type Extended Community 1. Encapsulation Extended Community
2. Router's MAC Extended Community 2. Router's MAC Extended Community
For symmetric IRB mode, Router's MAC EC is needed to carry the PE's For symmetric IRB mode, Router's MAC EC is needed to carry the PE's
overlay MAC address (e.g., inner MAC address in NVO encapsulation) overlay MAC address (e.g., inner MAC address in NVO encapsulation)
which is used for IP-VRF to IP-VRF communications with Ethernet NVO which is used for IP-VRF to IP-VRF communications with Ethernet NVO
tunnel. If MPLS or IP-only NVO tunnel is used, then there is no need tunnel. If MPLS or IP-only NVO tunnel is used, then there is no need
to send Router's MAC Extended Community along with this route. to send Router's MAC Extended Community along with this route.
This route MUST be advertised with two route targets, one This route MUST be advertised with two route targets, one
skipping to change at page 14, line 7 skipping to change at page 14, line 18
If the receiving PE receives this route with both the MAC-VRF and IP- If the receiving PE receives this route with both the MAC-VRF and IP-
VRF route targets and the MAC/IP Advertisement route includes MPLS VRF route targets and the MAC/IP Advertisement route includes MPLS
label2 field but the receiving PE only supports asymmetric IRB mode, label2 field but the receiving PE only supports asymmetric IRB mode,
then the receiving PE MUST ignore MPLS label2 field and install the then the receiving PE MUST ignore MPLS label2 field and install the
MAC address in the corresponding MAC-VRF and (IP, MAC) association in MAC address in the corresponding MAC-VRF and (IP, MAC) association in
the ARP table for that tenant (identified by the corresponding IP-VRF the ARP table for that tenant (identified by the corresponding IP-VRF
route target). route target).
5.3. Subnet route advertisement 5.3. Subnet route advertisement
In case of symmetric IRB, a layer-3 subnet and IRB interface In the case of symmetric IRB, a layer-3 subnet and IRB interface
corresponding to a MAC-VRF/BT is required to be provisioned at a PE corresponding to a MAC-VRF/BT is required to be provisioned at a PE
only if that PE has locally attached hosts in that subnet. In order only if that PE has locally attached hosts in that subnet. In order
to enable inter-subnet routing across PEs in a deployment where not to enable inter-subnet routing across PEs in a deployment where not
all subnets are provisioned at all PEs participating in an EVPN IRB all subnets are provisioned at all PEs participating in an EVPN IRB
instance, PEs MUST advertise local subnet routes as RT-5. These instance, PEs MUST advertise local subnet routes as EVPN RT-5. These
subnet routes are required for bootstrapping host (MAC,IP) learning subnet routes are required for bootstrapping host (MAC,IP) learning
using gleaning procedures initiated by an inter-subnet data packet. using gleaning procedures initiated by an inter-subnet data packet.
Consider a subnet A that is locally attached to PE1 and subnet B that Consider a subnet A that is locally attached to PE1 and subnet B that
is locally attached to PE2 and to PE3. Host A in subnet A, that is is locally attached to PE2 and to PE3. Host A in subnet A, that is
attached to PE1 initiates a data packet destined to host B in subnet attached to PE1 initiates a data packet destined to host B in subnet
B that is attached to PE3. If host B's (MAC, IP) has not yet been B that is attached to PE3. If host B's (MAC, IP) has not yet been
learnt either via a gratuitous ARP OR via a prior gleaning procedure, learnt either via a gratuitous ARP OR via a prior gleaning procedure,
a new gleaning procedure MUST be triggered for host B's (MAC, IP) to a new gleaning procedure MUST be triggered for host B's (MAC, IP) to
be learnt and advertised across the EVPN network. Since host B's be learnt and advertised across the EVPN network. Since host B's
subnet is not local to PE1, an IP lookup for host B at PE1 will not subnet is not local to PE1, an IP lookup for host B at PE1 will not
trigger this gleaning procedure for host B's (MAC, IP). Therefore, trigger this gleaning procedure for host B's (MAC, IP). Therefore,
PE1 MUST learn subnet B's prefix route via RT-5 advertised from PE2 PE1 MUST learn subnet B's prefix route via EVPN RT-5 advertised from
and PE3, so it can route the packet to one of the PEs that have PE2 and PE3, so it can route the packet to one of the PEs that have
subnet B locally attached. Once the packet is received at PE2 OR subnet B locally attached. Once the packet is received at PE2 OR
PE3, and the route lookup yields a glean result, an ARP request is PE3, and the route lookup yields a glean result, an ARP request is
triggered and flooded across the layer-2 overlay. This ARP request triggered and flooded across the layer-2 overlay. This ARP request
would be received and replied to by host B, resulting in host B (MAC, would be received and replied to by host B, resulting in host B (MAC,
IP) learning at PE3, and its advertisement across the EVPN network. IP) learning at PE3, and its advertisement across the EVPN network.
Packets from host A to host B can now be routed directly from PE1 to Packets from host A to host B can now be routed directly from PE1 to
PE3. Advertisement of local subnet RT-5 for an IP VRF MAY typically PE3. Advertisement of local subnet EVPN RT-5 for an IP VRF MAY
be achieved via provisioning connected route redistribution to BGP. typically be achieved via provisioning connected route redistribution
to BGP.
5.4. Data Plane - Ingress PE 5.4. Data Plane - Ingress PE
When an Ethernet frame is received by an ingress PE (e.g., PE1 in When an Ethernet frame is received by an ingress PE (e.g., PE1 in
figure 4 above), the PE uses the AC ID (e.g., VLAN ID) to identify figure 4 above), the PE uses the AC ID (e.g., VLAN ID) to identify
the associated MAC-VRF/BT and it performs a lookup on the destination the associated MAC-VRF/BT and it performs a lookup on the destination
MAC address. If the MAC address corresponds to its IRB Interface MAC MAC address. If the MAC address corresponds to its IRB Interface MAC
address, the ingress PE deduces that the packet must be inter-subnet address, the ingress PE deduces that the packet must be inter-subnet
routed. Hence, the ingress PE performs an IP lookup in the routed. Hence, the ingress PE performs an IP lookup in the
associated IP-VRF table. The lookup identifies BGP next hop of associated IP-VRF table. The lookup identifies BGP next hop of
egress PE along with the tunnel/encapsulation type and the associated egress PE along with the tunnel/encapsulation type and the associated
MPLS/VNI values. MPLS/VNI values. The ingress PE also decrements the TTL/hop limit
for that packet by one and if it reaches zero, the ingress PE
discards the packet.
If the tunnel type is that of MPLS or IP-only NVO tunnel, then TS's If the tunnel type is that of MPLS or IP-only NVO tunnel, then TS's
IP packet is sent over the tunnel without any Ethernet header. IP packet is sent over the tunnel without any Ethernet header.
However, if the tunnel type is that of Ethernet NVO tunnel, then an However, if the tunnel type is that of Ethernet NVO tunnel, then an
Ethernet header needs to be added to the TS's IP packet. The source Ethernet header needs to be added to the TS's IP packet. The source
MAC address of this inner Ethernet header is set to the ingress PE's MAC address of this inner Ethernet header is set to the ingress PE's
router MAC address and the destination MAC address of this inner router MAC address and the destination MAC address of this inner
Ethernet header is set to the egress PE's router MAC address learnt Ethernet header is set to the egress PE's router MAC address learnt
via Router's MAC extended community attached to the route. MPLS VPN via Router's MAC extended community attached to the route. MPLS VPN
label is set to the received label2 in the route. In case of label is set to the received label2 in the route. In the case of
Ethernet NVO tunnel type, VNI may be set one of two ways: Ethernet NVO tunnel type, VNI may be set one of two ways:
o downstream mode: VNI is set to the received label2 in the route o downstream mode: VNI is set to the received label2 in the route
which is downstream assigned. which is downstream assigned.
o global mode: VNI is set to the received label2 in the route which o global mode: VNI is set to the received label2 in the route which
is domain-wide assigned. This VNI value from received label2 MUST is domain-wide assigned. This VNI value from received label2 MUST
be the same as the locally configured VNI for the IP VRF as all be the same as the locally configured VNI for the IP VRF as all
PEs in the NVO MUST be configured with the same IP VRF VNI for PEs in the NVO MUST be configured with the same IP VRF VNI for
this mode of operation. this mode of operation.
PEs may be configured to operate in one of these two modes depending PEs may be configured to operate in one of these two modes depending
on the administrative domain boundaries across PEs participating in on the administrative domain boundaries across PEs participating in
the NVO, and PE's capability to support downstream VNI mode. the NVO, and PE's capability to support downstream VNI mode.
In case of NVO tunnel encapsulation, the outer source and destination In the case of NVO tunnel encapsulation, the outer source and
IP addresses are set to the ingress and egress PE BGP next-hop IP destination IP addresses are set to the ingress and egress PE BGP
addresses respectively. next-hop IP addresses respectively.
5.5. Data Plane - Egress PE 5.5. Data Plane - Egress PE
When the tenant's MPLS or NVO encapsulated packet is received over an When the tenant's MPLS or NVO encapsulated packet is received over an
MPLS or NVO tunnel by the egress PE, the egress PE removes NVO tunnel MPLS or NVO tunnel by the egress PE, the egress PE removes NVO tunnel
encapsulation and uses the VPN MPLS label (for MPLS encapsulation) or encapsulation and uses the VPN MPLS label (for MPLS encapsulation) or
VNI (for NVO encapsulation) to identify the IP-VRF in which IP lookup VNI (for NVO encapsulation) to identify the IP-VRF in which IP lookup
needs to be performed. If the VPN MPLS label or VNI identifies a needs to be performed. If the VPN MPLS label or VNI identifies a
MAC- VRF instead of an IP-VRF, then the procedures in section 6.4 for MAC- VRF instead of an IP-VRF, then the procedures in section 6.4 for
asymmetric IRB are executed. asymmetric IRB are executed.
The lookup in the IP-VRF identifies a local adjacency to the IRB The lookup in the IP-VRF identifies a local adjacency to the IRB
interface associated with the egress subnet's MAC-VRF/BT. interface associated with the egress subnet's MAC-VRF/BT. The egress
PE also decrements the TTL/hop limit for that packet by one and if it
reaches zero, the egress PE discards the packet.
The egress PE gets the destination TS's MAC address for that TS's IP The egress PE gets the destination TS's MAC address for that TS's IP
address from its ARP table or NDP cache, it encapsulates the packet address from its ARP table or NDP cache, it encapsulates the packet
with that destination MAC address and a source MAC address with that destination MAC address and a source MAC address
corresponding to that IRB interface and sends the packet to its corresponding to that IRB interface and sends the packet to its
destination subnet MAC-VRF/BT. destination subnet MAC-VRF/BT.
The destination MAC address lookup in the MAC-VRF/BT results in local The destination MAC address lookup in the MAC-VRF/BT results in local
adjacency (e.g., local interface) over which the Ethernet frame is adjacency (e.g., local interface) over which the Ethernet frame is
sent on. sent on.
skipping to change at page 17, line 31 skipping to change at page 17, line 51
o Using MAC-VRF route target, the receiving PE identifies the o Using MAC-VRF route target, the receiving PE identifies the
corresponding ARP table or NDP cache for the tenant and it adds an corresponding ARP table or NDP cache for the tenant and it adds an
entry to the ARP table or NDP cache for the TS's MAC and IP entry to the ARP table or NDP cache for the TS's MAC and IP
address association. It should be noted that the tenant's ARP address association. It should be noted that the tenant's ARP
table or NDP cache at the receiving PE is identified by all the table or NDP cache at the receiving PE is identified by all the
MAC- VRF route targets for that tenant. MAC- VRF route targets for that tenant.
o If IP-VRF route target is included, it may be used to import the o If IP-VRF route target is included, it may be used to import the
route to IP-VRF. If IP-VRF route-target is not included, MAC-VRF route to IP-VRF. If IP-VRF route-target is not included, MAC-VRF
is used to derive corresponding IP-VRF for import, as explained in is used to derive corresponding IP-VRF for import, as explained in
prior section. In both cases, IP-VRF route is installed with the the prior section. In both cases, IP-VRF route is installed with
TS MAC binding included in the received route. the TS MAC binding included in the received route.
If the receiving PE receives the MAC/IP Advertisement route with MPLS If the receiving PE receives the MAC/IP Advertisement route with MPLS
label2 field but the receiving PE only supports asymmetric IRB mode, label2 field but the receiving PE only supports asymmetric IRB mode,
then the receiving PE MUST ignore MPLS label2 field and install the then the receiving PE MUST ignore MPLS label2 field and install the
MAC address in the corresponding MAC-VRF and (IP, MAC) association in MAC address in the corresponding MAC-VRF and (IP, MAC) association in
the ARP table or NDP cache for that tenant (with IRB interface the ARP table or NDP cache for that tenant (with IRB interface
identified by the MAC-VRF). identified by the MAC-VRF).
If the receiving PE receives the MAC/IP Advertisement route with MPLS If the receiving PE receives the MAC/IP Advertisement route with MPLS
label2 field and it can support symmetric IRB mode, then it should label2 field and it can support symmetric IRB mode, then it should
skipping to change at page 18, line 15 skipping to change at page 18, line 30
6.3. Data Plane - Ingress PE 6.3. Data Plane - Ingress PE
When an Ethernet frame is received by an ingress PE (e.g., PE1 in When an Ethernet frame is received by an ingress PE (e.g., PE1 in
figure 4 above), the PE uses the AC ID (e.g., VLAN ID) to identify figure 4 above), the PE uses the AC ID (e.g., VLAN ID) to identify
the associated MAC-VRF/BT and it performs a lookup on the destination the associated MAC-VRF/BT and it performs a lookup on the destination
MAC address. If the MAC address corresponds to its IRB Interface MAC MAC address. If the MAC address corresponds to its IRB Interface MAC
address, the ingress PE deduces that the packet must be inter-subnet address, the ingress PE deduces that the packet must be inter-subnet
routed. Hence, the ingress PE performs an IP lookup in the routed. Hence, the ingress PE performs an IP lookup in the
associated IP-VRF table. The lookup identifies a local adjacency to associated IP-VRF table. The lookup identifies a local adjacency to
the IRB interface associated with the egress subnet's MAC-VRF/BT. the IRB interface associated with the egress subnet's MAC-VRF/BT.
The ingress PE also decrements the TTL/hop limit for that packet by
one and if it reaches zero, the ingress PE discards the packet.
The ingress PE gets the destination TS's MAC address for that TS's IP The ingress PE gets the destination TS's MAC address for that TS's IP
address from its ARP table or NDP cache, it encapsulates the packet address from its ARP table or NDP cache, it encapsulates the packet
with that destination MAC address and a source MAC address with that destination MAC address and a source MAC address
corresponding to that IRB interface and sends the packet to its corresponding to that IRB interface and sends the packet to its
destination subnet MAC-VRF/BT. destination subnet MAC-VRF/BT.
The destination MAC address lookup in the MAC-VRF/BT results in BGP The destination MAC address lookup in the MAC-VRF/BT results in BGP
next hop address of egress PE along with label-1 (L2 VPN MPLS label next hop address of egress PE along with label1 (L2 VPN MPLS label or
or VNI). The ingress PE encapsulates the packet using Ethernet NVO VNI). The ingress PE encapsulates the packet using Ethernet NVO
tunnel of the choice (e.g., VxLAN or GENEVE) and sends the packet to tunnel of the choice (e.g., VxLAN or NVGRE) and sends the packet to
the egress PE. Since the packet forwarding is between ingress PE's the egress PE. Because the packet forwarding is between ingress PE's
MAC-VRF/BT and egress PE's MAC-VRF/BT, the packet encapsulation MAC-VRF/BT and egress PE's MAC-VRF/BT, the packet encapsulation
procedures follows that of [RFC7432] for MPLS and [RFC8365] for VxLAN procedures follow that of [RFC7432] for MPLS and [RFC8365] for VxLAN
encapsulations. encapsulations.
6.4. Data Plane - Egress PE 6.4. Data Plane - Egress PE
When a tenant's Ethernet frame is received over an NVO tunnel by the When a tenant's Ethernet frame is received over an NVO tunnel by the
egress PE, the egress PE removes NVO tunnel encapsulation and uses egress PE, the egress PE removes NVO tunnel encapsulation and uses
the VPN MPLS label (for MPLS encapsulation) or VNI (for NVO the VPN MPLS label (for MPLS encapsulation) or VNI (for NVO
encapsulation) to identify the MAC-VRF/BT in which MAC lookup needs encapsulation) to identify the MAC-VRF/BT in which MAC lookup needs
to be performed. to be performed.
The MAC lookup results in local adjacency (e.g., local interface) The MAC lookup results in local adjacency (e.g., local interface)
over which the packet needs to get sent. over which the packet needs to get sent.
Note that the forwarding behavior on the egress PE is the same as Note that the forwarding behavior on the egress PE is the same as
EVPN intra-subnet forwarding described in [RFC7432] for MPLS and EVPN intra-subnet forwarding described in [RFC7432] for MPLS and
[RFC8365] for NVO networks. In other words, all the packet [RFC8365] for NVO networks. In other words, all the packet
processing associated with the inter-subnet forwarding semantics is processing associated with the inter-subnet forwarding semantics is
confined to the ingress PE for asymmetric IRB mode. confined to the ingress PE for asymmetric IRB mode.
It should also be noted that [RFC7432] provides different level of It should also be noted that [RFC7432] provides a different level of
granularity for the EVPN label. Besides identifying bridge domain granularity for the EVPN label. Besides identifying the bridge
table, it can be used to identify the egress interface or a domain table, it can be used to identify the egress interface or a
destination MAC address on that interface. If EVPN label is used for destination MAC address on that interface. If EVPN label is used for
egress interface or individual MAC address identification, then no egress interface or individual MAC address identification, then no
MAC lookup is needed in the egress PE for MPLS encapsulation and the MAC lookup is needed in the egress PE for MPLS encapsulation and the
packet can be directly forwarded to the egress interface just based packet can be directly forwarded to the egress interface just based
on EVPN label lookup. on EVPN label lookup.
7. Mobility Procedure 7. Mobility Procedure
When a TS moves from one NVE (aka source NVE) to another NVE (aka When a TS moves from one NVE (aka source NVE) to another NVE (aka
target NVE), it is important that the MAC mobility procedures are target NVE), it is important that the MAC mobility procedures are
skipping to change at page 19, line 42 skipping to change at page 20, line 11
3. TS is a silent host and neither initiates an ARP request nor 3. TS is a silent host and neither initiates an ARP request nor
sends any packets sends any packets
Depending on the expexted TS's behavior, an NVE needs to handle at Depending on the expexted TS's behavior, an NVE needs to handle at
least the first bullet and should be able to handle the 2nd and the least the first bullet and should be able to handle the 2nd and the
3rd bullet. The following subsections describe the procedures for 3rd bullet. The following subsections describe the procedures for
each of them where it is assumed that the MAC and IP addresses of a each of them where it is assumed that the MAC and IP addresses of a
TS have one-to-one relationship (i.e., there is one IP address per TS have one-to-one relationship (i.e., there is one IP address per
MAC address and vice versa). If there is many- to-one relationship MAC address and vice versa). If there is many- to-one relationship
such that there are many host IP addresses correspond to a single such that there are many host IP addresses (non-link-local unicast
host MAC address or there are many host MAC addresses correspond to a addresses for IPv6) corresponding to a single host MAC address or
single IP address, then to detect host mobility, the procedures in there are many host MAC addresses corresponding to a single IP
address (non-link-local unicast address for IPv6), then to detect
host mobility, the procedures in
[I-D.ietf-bess-evpn-irb-extended-mobility] must be exercised followed [I-D.ietf-bess-evpn-irb-extended-mobility] must be exercised followed
by the procedures described below. by the procedures described below.
7.1. Initiating a gratutious ARP upon a Move 7.1. Initiating a gratutious ARP upon a Move
In this scenario when a TS moves from a source NVE to a target NVE, In this scenario when a TS moves from a source NVE to a target NVE,
the TS initiates a gratuitous ARP upon the move to the target NVE. the TS initiates a gratuitous ARP upon the move to the target NVE.
The target NVE upon receiving this ARP message, updates its MAC-VRF, The target NVE upon receiving this ARP message, updates its MAC-VRF,
IP-VRF, and ARP table with the host MAC, IP, and local adjacency IP-VRF, and ARP table with the host MAC, IP, and local adjacency
skipping to change at page 20, line 26 skipping to change at page 20, line 40
it initiates MAC mobility procedures per [RFC7432] by advertising an it initiates MAC mobility procedures per [RFC7432] by advertising an
EVPN MAC/IP Advertisement route with both the MAC and IP addresses EVPN MAC/IP Advertisement route with both the MAC and IP addresses
filled in (per sections 5.1 and 6.1) along with MAC Mobility Extended filled in (per sections 5.1 and 6.1) along with MAC Mobility Extended
Community with the sequence number incremented by one. The target Community with the sequence number incremented by one. The target
NVE also exercises the MAC duplication detection procedure in section NVE also exercises the MAC duplication detection procedure in section
15.1 of [RFC7432]. 15.1 of [RFC7432].
The source NVE upon receiving this MAC/IP Advertisement route, The source NVE upon receiving this MAC/IP Advertisement route,
realizes that the MAC has moved to the target NVE. It updates its realizes that the MAC has moved to the target NVE. It updates its
MAC-VRF and IP-VRF table accordingly with the adjacency information MAC-VRF and IP-VRF table accordingly with the adjacency information
of the target NVE. In case of the asymmetric IRB, the source NVE of the target NVE. In the case of the asymmetric IRB, the source NVE
also updates its ARP table with the received adjacency information also updates its ARP table with the received adjacency information
and in case of the symmetric IRB, the source NVE removes the entry and in the case of the symmetric IRB, the source NVE removes the
associated with the received (MAC, IP) from its local ARP table. It entry associated with the received (MAC, IP) from its local ARP
then withdraws its EVPN MAC/IP Advertisement route. Furthermore, it table. It then withdraws its EVPN MAC/IP Advertisement route.
sends an ARP probe locally to ensure that the MAC is gone. If an ARP Furthermore, it sends an ARP probe locally to ensure that the MAC is
response is received, the source NVE updates its ARP entry for that gone. If an ARP response is received, the source NVE updates its ARP
(IP, MAC) and re-advertises an EVPN MAC/IP Advertisement route for entry for that (IP, MAC) and re-advertises an EVPN MAC/IP
that (IP, MAC) along with MAC Mobility Extended Community with the Advertisement route for that (IP, MAC) along with MAC Mobility
sequence number incremented by one. The source NVE also exercises Extended Community with the sequence number incremented by one. The
the MAC duplication detection procedure in section 15.1 of [RFC7432]. source NVE also exercises the MAC duplication detection procedure in
section 15.1 of [RFC7432].
All other remote NVE devices upon receiving the MAC/IP Advertisement All other remote NVE devices upon receiving the MAC/IP Advertisement
route with MAC Mobility extended community compare the sequence route with MAC Mobility extended community compare the sequence
number in this advertisement with the one previously received. If number in this advertisement with the one previously received. If
the new sequence number is greater than the old one, then they update the new sequence number is greater than the old one, then they update
the MAC/IP addresses of the TS in their corresponding MAC-VRF and IP- the MAC/IP addresses of the TS in their corresponding MAC-VRF and IP-
VRF tables to point to the target NVE. Furthermore, upon receiving VRF tables to point to the target NVE. Furthermore, upon receiving
the MAC/IP withdraw for the TS from the source NVE, these remote PEs the MAC/IP withdraw for the TS from the source NVE, these remote PEs
perform the cleanups for their BGP tables. perform the cleanups for their BGP tables.
7.2. Sending Data Traffic without an ARP Request 7.2. Sending Data Traffic without an ARP Request
In this scenario when a TS moves from a source NVE to a target NVE, In this scenario when a TS moves from a source NVE to a target NVE,
the TS starts sending data traffic without first initiating an ARP the TS starts sending data traffic without first initiating an ARP
request. request.
The target NVE upon receiving the first data packet, learns the MAC The target NVE upon receiving the first data packet, learns the MAC
address of the TS in data plane and updates its MAC-VRF table with address of the TS in the data plane and updates its MAC-VRF table
the MAC address and the local adjacency information (e.g., local with the MAC address and the local adjacency information (e.g., local
interface) accordingly. The target NVE realizes that there has been interface) accordingly. The target NVE realizes that there has been
a MAC move because the same MAC address has been learned remotely a MAC move because the same MAC address has been learned remotely
from the source NVE. from the source NVE.
If EVPN-IRB NVEs are configured to advertise MAC-only routes in If EVPN-IRB NVEs are configured to advertise MAC-only routes in
addition to MAC-and-IP EVPN routes, then the following steps are addition to MAC-and-IP EVPN routes, then the following steps are
taken: taken:
o The target NVE upon learning this MAC address in data-plane, o The target NVE upon learning this MAC address in the data plane,
updates this MAC address entry in the corresponding MAC-VRF with updates this MAC address entry in the corresponding MAC-VRF with
the local adjacency information (e.g., local interface). It also the local adjacency information (e.g., local interface). It also
recognizes that this MAC has moved and initiates MAC mobility recognizes that this MAC has moved and initiates MAC mobility
procedures per [RFC7432] by advertising an EVPN MAC/IP procedures per [RFC7432] by advertising an EVPN MAC/IP
Advertisement route with only the MAC address filled in along with Advertisement route with only the MAC address filled in along with
MAC Mobility Extended Community with the sequence number MAC Mobility Extended Community with the sequence number
incremented by one. incremented by one.
o The source NVE upon receiving this MAC/IP Advertisement route, o The source NVE upon receiving this MAC/IP Advertisement route,
realizes that the MAC has moved to the new NVE. It updates its realizes that the MAC has moved to the new NVE. It updates its
skipping to change at page 21, line 48 skipping to change at page 22, line 15
o The target NVE passes the ARP request to its locally attached TSes o The target NVE passes the ARP request to its locally attached TSes
and when it receives the ARP response, it updates its IP-VRF and and when it receives the ARP response, it updates its IP-VRF and
ARP table with the host (MAC, IP) information. It also sends an ARP table with the host (MAC, IP) information. It also sends an
EVPN MAC/IP Advertisement route with both the MAC and IP addresses EVPN MAC/IP Advertisement route with both the MAC and IP addresses
filled in along with MAC Mobility Extended Community with the filled in along with MAC Mobility Extended Community with the
sequence number set to the same value as the one for MAC-only sequence number set to the same value as the one for MAC-only
advertisement route it sent previously. advertisement route it sent previously.
o When the source NVE receives the EVPN MAC/IP Advertisement route, o When the source NVE receives the EVPN MAC/IP Advertisement route,
it updates its IP-VRF table with the new adjacency information it updates its IP-VRF table with the new adjacency information
(pointing to the target NVE). In case of the asymmetric IRB, the (pointing to the target NVE). In the case of the asymmetric IRB,
source NVE also updates its ARP table with the received adjacency the source NVE also updates its ARP table with the received
information and in case of the symmetric IRB, the source NVE adjacency information and in the case of the symmetric IRB, the
removes the entry associated with the received (MAC, IP) from its source NVE removes the entry associated with the received (MAC,
local ARP table. Furthermore, it withdraws its previously IP) from its local ARP table. Furthermore, it withdraws its
advertised EVPN MAC/IP route with both the MAC and IP address previously advertised EVPN MAC/IP route with both the MAC and IP
fields filled in. address fields filled in.
o All other remote NVE devices upon receiving the MAC/IP o All other remote NVE devices upon receiving the MAC/IP
advertisement route with MAC Mobility extended community compare advertisement route with MAC Mobility extended community compare
the sequence number in this advertisement with the one previously the sequence number in this advertisement with the one previously
received. If the new sequence number is greater than the old one, received. If the new sequence number is greater than the old one,
then they update the MAC/IP addresses of the TS in their then they update the MAC/IP addresses of the TS in their
corresponding MAC-VRF, IP-VRF, and ARP tables (in case of corresponding MAC-VRF, IP-VRF, and ARP tables (in the case of
asymmetric IRB) to point to the new NVE. Furthermore, upon asymmetric IRB) to point to the new NVE. Furthermore, upon
receiving the MAC/IP withdraw for the TS from the old NVE, these receiving the MAC/IP withdraw for the TS from the old NVE, these
remote PEs perform the cleanups for their BGP tables. remote PEs perform the cleanups for their BGP tables.
If EVPN-IRB NVEs are configured not to advertise MAC-only routes, If EVPN-IRB NVEs are configured not to advertise MAC-only routes,
then upon receiving the first data packet, it learns the MAC address then upon receiving the first data packet, it learns the MAC address
of the TS and updates the MAC entry in the corresponding MAC-VRF of the TS and updates the MAC entry in the corresponding MAC-VRF
table with the local adjacency information (e.g., local interface). table with the local adjacency information (e.g., local interface).
It also realizes that there has been a MAC move because the same MAC It also realizes that there has been a MAC move because the same MAC
address has been learned remotely from the source NVE. It uses the address has been learned remotely from the source NVE. It uses the
skipping to change at page 23, line 9 skipping to change at page 23, line 24
The target NVE passes the ARP request to its locally attached TSes The target NVE passes the ARP request to its locally attached TSes
and when it receives the ARP response, it updates its MAC-VRF, IP- and when it receives the ARP response, it updates its MAC-VRF, IP-
VRF, and ARP table with the host (MAC, IP) and local adjacency VRF, and ARP table with the host (MAC, IP) and local adjacency
information (e.g., local interface). It also sends an EVPN MAC/IP information (e.g., local interface). It also sends an EVPN MAC/IP
advertisement route with both the MAC and IP address fields filled in advertisement route with both the MAC and IP address fields filled in
along with MAC Mobility Extended Community with the sequence number along with MAC Mobility Extended Community with the sequence number
incremented by one. incremented by one.
When the source NVE receives the EVPN MAC/IP Advertisement route, it When the source NVE receives the EVPN MAC/IP Advertisement route, it
updates its IP-VRF table with the new adjacency information (pointing updates its IP-VRF table with the new adjacency information (pointing
to the target NVE). In case of the asymmetric IRB, the source NVE to the target NVE). In the case of the asymmetric IRB, the source
also updates its ARP table with the received adjacency information NVE also updates its ARP table with the received adjacency
and in case of the symmetric IRB, the source NVE removes the entry information and in the case of the symmetric IRB, the source NVE
associated with the received (MAC, IP) from its local ARP table. removes the entry associated with the received (MAC, IP) from its
Furthermore, it withdraws its previously advertised EVPN MAC/IP route local ARP table. Furthermore, it withdraws its previously advertised
with both the MAC and IP address fields filled in. EVPN MAC/IP route with both the MAC and IP address fields filled in.
All other remote NVE devices upon receiving the MAC/IP Advertisement All other remote NVE devices upon receiving the MAC/IP Advertisement
route route with MAC Mobility extended community compare the sequence route with MAC Mobility extended community compare the sequence
number in this advertisement with the one previously received. If number in this advertisement with the one previously received. If
the new sequence number is greater than the old one, then they update the new sequence number is greater than the old one, then they update
the MAC/IP addresses of the TS in their corresponding MAC-VRF, IP- the MAC/IP addresses of the TS in their corresponding MAC-VRF, IP-
VRF, and ARP (in case of asymmetric IRB) tables to point to the new VRF, and ARP (in the case of asymmetric IRB) tables to point to the
NVE. Furthermore, upon receiving the MAC/IP withdraw for the TS from new NVE. Furthermore, upon receiving the MAC/IP withdraw for the TS
the old NVE, these remote PEs perform the cleanups for their BGP from the old NVE, these remote PEs perform the cleanups for their BGP
tables. tables.
8. BGP Encoding 8. BGP Encoding
This document defines one new BGP Extended Community for EVPN. This document defines one new BGP Extended Community for EVPN.
8.1. Router's MAC Extended Community 8.1. Router's MAC Extended Community
A new EVPN BGP Extended Community called Router's MAC is introduced A new EVPN BGP Extended Community called Router's MAC is introduced
here. This new extended community is a transitive extended community here. This new extended community is a transitive extended community
skipping to change at page 24, line 6 skipping to change at page 24, line 19
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type=0x06 | Sub-Type=0x03 | Router's MAC | | Type=0x06 | Sub-Type=0x03 | Router's MAC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Router's MAC Cont'd | | Router's MAC Cont'd |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 5: Router's MAC Extended Community Figure 5: Router's MAC Extended Community
This extended community is used to carry the PE's MAC address for This extended community is used to carry the PE's MAC address for
symmetric IRB scenarios and it is sent with RT-2. symmetric IRB scenarios and it is sent with EVPN RT-2. The
advertising PE SHALL only attach a single Router's MAC Extended
Community to a route. In case the receiving PE receives more than
one Router's MAC Extended Community with a route, it SHALL process
the first one in the list and not store and propagate the others.
9. Operational Models for Symmetric Inter-Subnet Forwarding 9. Operational Models for Symmetric Inter-Subnet Forwarding
The following sections describe two main symmetric IRB forwarding The following sections describe two main symmetric IRB forwarding
scenarios (within a DC -- i.e., intra-DC) along with the scenarios (within a DC -- i.e., intra-DC) along with the
corresponding procedures. In the following scenarios, without loss corresponding procedures. In the following scenarios, without loss
of generality, it is assumed that a given tenant is represented by a of generality, it is assumed that a given tenant is represented by a
single IP-VPN instance. Therefore, on a given PE, a tenant is single IP-VPN instance. Therefore, on a given PE, a tenant is
represented by a single IP-VRF table and one or more MAC-VRF tables. represented by a single IP-VRF table and one or more MAC-VRF tables.
skipping to change at page 24, line 29 skipping to change at page 24, line 46
This section covers the symmetric IRB procedures for the scenario This section covers the symmetric IRB procedures for the scenario
where each Tenant System (TS) is attached to one or more NVEs and its where each Tenant System (TS) is attached to one or more NVEs and its
host IP and MAC addresses are learned by the attached NVEs and are host IP and MAC addresses are learned by the attached NVEs and are
distributed to all other NVEs that are interested in participating in distributed to all other NVEs that are interested in participating in
both intra-subnet and inter-subnet communications with that TS. both intra-subnet and inter-subnet communications with that TS.
In this scenario, without loss of generality, it is assumed that NVEs In this scenario, without loss of generality, it is assumed that NVEs
operate in VLAN-based service interface mode with one Bridge operate in VLAN-based service interface mode with one Bridge
Table (BT) per MAC-VRF. Thus, for a given tenant, an NVE has one Table (BT) per MAC-VRF. Thus, for a given tenant, an NVE has one
MAC-VRF for each tenant subnet (e.g., each VLAN) that is configured MAC-VRF for each tenant subnet (e.g., each VLAN) that is configured
for extension via VxLAN or GENEVE encapsulation. In case of VLAN- for extension via VxLAN or NVGRE encapsulation. In the case of VLAN-
aware bundling, then each MAC-VRF consists of multiple Bridge Tables aware bundling, then each MAC-VRF consists of multiple Bridge Tables
(e.g., one BT per VLAN). The MAC-VRFs on an NVE for a given tenant (e.g., one BT per VLAN). The MAC-VRFs on an NVE for a given tenant
are associated with an IP-VRF corresponding to that tenant (or IP-VPN are associated with an IP-VRF corresponding to that tenant (or IP-VPN
instance) via their IRB interfaces. instance) via their IRB interfaces.
Since VxLAN and GENEVE encapsulations require inner Ethernet header Since VxLAN and NVGRE encapsulations require inner Ethernet header
(inner MAC SA/DA), and since for inter-subnet traffic, TS MAC address (inner MAC SA/DA), and since for inter-subnet traffic, TS MAC address
cannot be used, the ingress NVE's MAC address is used as inner MAC cannot be used, the ingress NVE's MAC address is used as inner MAC
SA. The NVE's MAC address is the device MAC address and it is common SA. The NVE's MAC address is the device MAC address and it is common
across all MAC-VRFs and IP-VRFs. This MAC address is advertised across all MAC-VRFs and IP-VRFs. This MAC address is advertised
using the new EVPN Router's MAC Extended Community (section 8.1). using the new EVPN Router's MAC Extended Community (section 8.1).
Figure 6 below illustrates this scenario where a given tenant (e.g., Figure 6 below illustrates this scenario where a given tenant (e.g.,
an IP-VPN instance) has three subnets represented by MAC-VRF1, MAC- an IP-VPN instance) has three subnets represented by MAC-VRF1, MAC-
VRF2, and MAC-VRF3 across two NVEs. There are five TSes that are VRF2, and MAC-VRF3 across two NVEs. There are five TSes that are
associated with these three MAC-VRFs -- i.e., TS1, TS4, and TS5 are associated with these three MAC-VRFs -- i.e., TS1, TS4, and TS5 are
on the same subnet (e.g., same MAC-VRF/VLAN). TS1 and TS5 are on the same subnet (e.g., same MAC-VRF/VLAN). TS1 and TS5 are
associated with MAC-VRF1 on NVE1, while TS4 is associated with MAC- associated with MAC-VRF1 on NVE1, while TS4 is associated with MAC-
VRF1 on NVE2. TS2 is associated with MAC-VRF2 on NVE1, and TS3 is VRF1 on NVE2. TS2 is associated with MAC-VRF2 on NVE1, and TS3 is
associated with MAC-VRF3 on NVE2. MAC-VRF1 and MAC-VRF2 on NVE1 are associated with MAC-VRF3 on NVE2. MAC-VRF1 and MAC-VRF2 on NVE1 are
in turn associated with IP-VRF1 on NVE1 and MAC-VRF1 and MAC-VRF3 on in turn associated with IP-VRF1 on NVE1 and MAC-VRF1 and MAC-VRF3 on
NVE2 are associated with IP-VRF1 on NVE2. When TS1, TS5, and TS4 NVE2 are associated with IP-VRF1 on NVE2. When TS1, TS5, and TS4
exchange traffic with each other, only L2 forwarding (bridging) part exchange traffic with each other, only the L2 forwarding (bridging)
of the IRB solution is exercised because all these TSes belong to the part of the IRB solution is exercised because all these TSes belong
same subnet. However, when TS1 wants to exchange traffic with TS2 or to the same subnet. However, when TS1 wants to exchange traffic with
TS3 which belong to different subnets, both bridging and routing TS2 or TS3 which belong to different subnets, both bridging and
parts of the IRB solution are exercised. The following subsections routing parts of the IRB solution are exercised. The following
describe the control and data planes operations for this IRB scenario subsections describe the control and data planes operations for this
in details. IRB scenario in details.
NVE1 +---------+ NVE1 +---------+
+-------------+ | | +-------------+ | |
TS1-----| MACx| | | NVE2 TS1-----| MACx| | | NVE2
(IP1/M1) |(MAC- | | | +-------------+ (IP1/M1) |(MAC- | | | +-------------+
TS5-----| VRF1)\ | | MPLS/ | |MACy (MAC- |-----TS3 TS5-----| VRF1)\ | | MPLS/ | |MACy (MAC- |-----TS3
(IP5/M5) | \ | | VxLAN/ | | / VRF3) | (IP3/M3) (IP5/M5) | \ | | VxLAN/ | | / VRF3) | (IP3/M3)
| (IP-VRF1)|----| NVGRE |---|(IP-VRF1) | | (IP-VRF1)|----| NVGRE |---|(IP-VRF1) |
| / | | | | \ | | / | | | | \ |
TS2-----|(MAC- / | | | | (MAC- |-----TS4 TS2-----|(MAC- / | | | | (MAC- |-----TS4
skipping to change at page 25, line 44 skipping to change at page 26, line 17
o Ethernet Tag = 0; assuming VLAN-based service o Ethernet Tag = 0; assuming VLAN-based service
o MAC Address Length = 48 o MAC Address Length = 48
o MAC Address = Mi ; where i = 1,2,3,4, or 5 in the above example o MAC Address = Mi ; where i = 1,2,3,4, or 5 in the above example
o IP Address Length = 32 or 128 o IP Address Length = 32 or 128
o IP Address = IPi ; where i = 1,2,3,4, or 5 in the above example o IP Address = IPi ; where i = 1,2,3,4, or 5 in the above example
o Label-1 = MPLS Label or VNI corresponding to MAC-VRF o Label1 = MPLS Label or VNI corresponding to MAC-VRF
o Label-2 = MPLS Label or VNI corresponding to IP-VRF o Label2 = MPLS Label or VNI corresponding to IP-VRF
Each NVE advertises an RT-2 route with two Route Targets (one Each NVE advertises an EVPN RT-2 route with two Route Targets (one
corresponding to its MAC-VRF and the other corresponding to its IP- corresponding to its MAC-VRF and the other corresponding to its IP-
VRF. Furthermore, the RT-2 is advertised with two BGP Extended VRF. Furthermore, the EVPN RT-2 is advertised with two BGP Extended
Communities. The first BGP Extended Community identifies the tunnel Communities. The first BGP Extended Community identifies the tunnel
type per section 4.5 of [I-D.ietf-idr-tunnel-encaps] and the second type and it is called Encapsulation Extended Community as defined in
BGP Extended Community includes the MAC address of the NVE (e.g., [I-D.ietf-idr-tunnel-encaps] and the second BGP Extended Community
MACx for NVE1 or MACy for NVE2) as defined in section 8.1. This includes the MAC address of the NVE (e.g., MACx for NVE1 or MACy for
second Extended Community (for the MAC address of NVE) is only NVE2) as defined in section 8.1. The Router's MAC Extended community
required when Ethernet NVO tunnel type is used. If IP NVO tunnel MUST be added when Ethernet NVO tunnel is used. If IP NVO tunnel
type is used, then there is no need to send this second Extended type is used, then there is no need to send this second Extended
Community. It should be noted that IP NVO tunnel type is only Community. It should be noted that IP NVO tunnel type is only
applicable to symmetric IRB procedures. applicable to symmetric IRB procedures.
Upon receiving this advertisement, the receiving NVE performs the Upon receiving this advertisement, the receiving NVE performs the
following: following:
o It uses Route Targets corresponding to its MAC-VRF and IP-VRF for o It uses Route Targets corresponding to its MAC-VRF and IP-VRF for
identifying these tables and subsequently importing the MAC and IP identifying these tables and subsequently importing the MAC and IP
addresses into them respectively. addresses into them respectively.
o It imports the MAC address from MAC/IP Advertisement route into o It imports the MAC address from MAC/IP Advertisement route into
the MAC-VRF with BGP Next Hop address as underlay tunnel the MAC-VRF with BGP Next Hop address as the underlay tunnel
destination address (e.g., VTEP DA for VxLAN encapsulation) and destination address (e.g., VTEP DA for VxLAN encapsulation) and
Label-1 as VNI for VxLAN encapsulation or EVPN label for MPLS Label1 as VNI for VxLAN encapsulation or EVPN label for MPLS
encapsulation. encapsulation.
o If the route carries the new Router's MAC Extended Community, and o If the route carries the new Router's MAC Extended Community, and
if the receiving NVE uses Ethernet NVO tunnel, then the receiving if the receiving NVE uses Ethernet NVO tunnel, then the receiving
NVE imports the IP address into IP-VRF with NVE's MAC address NVE imports the IP address into IP-VRF with NVE's MAC address
(from the new Router's MAC Extended Community) as inner MAC DA and (from the new Router's MAC Extended Community) as inner MAC DA and
BGP Next Hop address as underlay tunnel destination address, VTEP BGP Next Hop address as the underlay tunnel destination address,
DA for VxLAN encapsulation and Label-2 as IP-VPN VNI for VxLAN VTEP DA for VxLAN encapsulation and Label2 as IP-VPN VNI for VxLAN
encapsulation. encapsulation.
o If the receiving NVE uses MPLS encapsulation, then the receiving o If the receiving NVE uses MPLS encapsulation, then the receiving
NVE imports the IP address into IP-VRF with BGP Next Hop address NVE imports the IP address into IP-VRF with BGP Next Hop address
as underlay tunnel destination address, and Label-2 as IP-VPN as the underlay tunnel destination address, and Label2 as IP-VPN
label for MPLS encapsulation. label for MPLS encapsulation.
If the receiving NVE receives a RT-2 with only Label-1 and only a If the receiving NVE receives an EVPN RT-2 with only Label1 and only
single Route Target corresponding to IP-VRF, or if it receives a RT-2 a single Route Target corresponding to IP-VRF, or if it receives an
with only a single Route Target corresponding to MAC-VRF but with EVPN RT-2 with only a single Route Target corresponding to MAC-VRF
both Label-1 and Label-2, or if it receives a RT-2 with MAC Address but with both Label1 and Label2, or if it receives an EVPN RT-2 with
Length of zero, then it MUST treat the route as withdraw [RFC7606] MAC Address Length of zero, then it MUST use the treat-as-withdraw
and SHOULD log an error message. approach [RFC7606] and SHOULD log an error message.
9.1.2. Data Plane Operation 9.1.2. Data Plane Operation
The following description of the data-plane operation describes just The following description of the data-plane operation describes just
the logical functions and the actual implementation may differ. Lets the logical functions and the actual implementation may differ. Lets
consider data-plane operation when TS1 in subnet-1 (MAC-VRF1) on NVE1 consider data-plane operation when TS1 in subnet-1 (MAC-VRF1) on NVE1
wants to send traffic to TS3 in subnet-3 (MAC-VRF3) on NVE2. wants to send traffic to TS3 in subnet-3 (MAC-VRF3) on NVE2.
o NVE1 receives a packet with MAC DA corresponding to the MAC-VRF1 o NVE1 receives a packet with MAC DA corresponding to the MAC-VRF1
IRB interface on NVE1 (the interface between MAC-VRF1 and IP- IRB interface on NVE1 (the interface between MAC-VRF1 and IP-
VRF1), and VLAN-tag corresponding to MAC-VRF1. VRF1), and VLAN-tag corresponding to MAC-VRF1.
o Upon receiving the packet, the NVE1 uses VLAN-tag to identify the o Upon receiving the packet, the NVE1 uses VLAN-tag to identify the
MAC-VRF1. It then looks up the MAC DA and forwards the frame to MAC-VRF1. It then looks up the MAC DA and forwards the frame to
its IRB interface. its IRB interface.
o The Ethernet header of the packet is stripped and the packet is o The Ethernet header of the packet is stripped and the packet is
fed to the IP-VRF where IP lookup is performed on the destination fed to the IP-VRF where an IP lookup is performed on the
IP address. This lookup yields the outgoing NVO tunnel and the destination IP address. NVE1 also decrements the TTL/hop limit
for that packet by one and if it reaches zero, NVE1 discards the
packet. This lookup yields the outgoing NVO tunnel and the
required encapsulation. If the encapsulation is for Ethernet NVO required encapsulation. If the encapsulation is for Ethernet NVO
tunnel, then it includes the egress NVE's MAC address as inner MAC tunnel, then it includes the egress NVE's MAC address as inner MAC
DA, the egress NVE's IP address (e.g., BGP Next Hop address) as DA, the egress NVE's IP address (e.g., BGP Next Hop address) as
the VTEP DA, and the VPN-ID as the VNI. The inner MAC SA and VTEP the VTEP DA, and the VPN-ID as the VNI. The inner MAC SA and VTEP
SA are set to NVE's MAC and IP addresses respectively. If it is a SA are set to NVE's MAC and IP addresses respectively. If it is a
MPLS encapsulation, then corresponding EVPN and LSP labels are MPLS encapsulation, then corresponding EVPN and LSP labels are
added to the packet. The packet is then forwarded to the egress added to the packet. The packet is then forwarded to the egress
NVE. NVE.
o On the egress NVE, if the packet arrives on Ethernet NVO tunnel o On the egress NVE, if the packet arrives on Ethernet NVO tunnel
(e.g., it is VxLAN encapsulated), then the NVO tunnel header is (e.g., it is VxLAN encapsulated), then the NVO tunnel header is
removed. Since the inner MAC DA is the egress NVE's MAC address, removed. Since the inner MAC DA is the egress NVE's MAC address,
the egress NVE knows that it needs to perform an IP lookup. It the egress NVE knows that it needs to perform an IP lookup. It
uses the VNI to identify the IP-VRF table. If the packet is MPLS uses the VNI to identify the IP-VRF table. If the packet is MPLS
encapsulated, then the EVPN label lookup identifies the IP-VRF encapsulated, then the EVPN label lookup identifies the IP-VRF
table. Next, an IP lookup is performed for the destination TS table. Next, an IP lookup is performed for the destination TS
(TS3) which results in access-facing IRB interface over which the (TS3) which results in an access-facing IRB interface over which
packet is sent. Before sending the packet over this interface, the packet is sent. Before sending the packet over this
the ARP table is consulted to get the destination TS's MAC interface, the ARP table is consulted to get the destination TS's
address. MAC address. NVE2 also decrements the TTL/hop limit for that
packet by one and if it reaches zero, NVE2 discards the packet.
o The IP packet is encapsulated with an Ethernet header with MAC SA o The IP packet is encapsulated with an Ethernet header with MAC SA
set to that of IRB interface MAC address (i.e, IRB interface set to that of IRB interface MAC address (i.e, IRB interface
between MAC-VRF3 and IP-VRF1 on NVE2) and MAC DA set to that of between MAC-VRF3 and IP-VRF1 on NVE2) and MAC DA set to that of
destination TS (TS3) MAC address. The packet is sent to the destination TS (TS3) MAC address. The packet is sent to the
corresponding MAC-VRF (i.e., MAC-VRF3) and after a lookup of MAC corresponding MAC-VRF (i.e., MAC-VRF3) and after a lookup of MAC
DA, is forwarded to the destination TS (TS3) over the DA, is forwarded to the destination TS (TS3) over the
corresponding interface. corresponding interface.
In this symmetric IRB scenario, inter-subnet traffic between NVEs In this symmetric IRB scenario, inter-subnet traffic between NVEs
skipping to change at page 28, line 21 skipping to change at page 28, line 37
This section covers the symmetric IRB procedures for the scenario This section covers the symmetric IRB procedures for the scenario
where some Tenant Systems (TSes) support one or more subnets and where some Tenant Systems (TSes) support one or more subnets and
these TSes are associated with one or more NVEs. Therefore, besides these TSes are associated with one or more NVEs. Therefore, besides
the advertisement of MAC/IP addresses for each TS which can be multi- the advertisement of MAC/IP addresses for each TS which can be multi-
homed with All-Active redundancy mode, the associated NVE needs to homed with All-Active redundancy mode, the associated NVE needs to
also advertise the subnets statically configured on each TS. also advertise the subnets statically configured on each TS.
The main difference between this solution and the previous one is the The main difference between this solution and the previous one is the
additional advertisement corresponding to each subnet. These subnet additional advertisement corresponding to each subnet. These subnet
advertisements are accomplished using EVPN IP Prefix route defined in advertisements are accomplished using the EVPN IP Prefix route
[I-D.ietf-bess-evpn-prefix-advertisement]. These subnet prefixes are defined in [I-D.ietf-bess-evpn-prefix-advertisement]. These subnet
advertised with the IP address of their associated TS (which is in prefixes are advertised with the IP address of their associated TS
overlay address space) as their next hop. The receiving NVEs perform (which is in overlay address space) as their next hop. The receiving
recursive route resolution to resolve the subnet prefix with its NVEs perform recursive route resolution to resolve the subnet prefix
advertising NVE so that they know which NVE to forward the packets to with its advertising NVE so that they know which NVE to forward the
when they are destined for that subnet prefix. packets to when they are destined for that subnet prefix.
The advantage of this recursive route resolution is that when a TS The advantage of this recursive route resolution is that when a TS
moves from one NVE to another, there is no need to re-advertise any moves from one NVE to another, there is no need to re-advertise any
of the subnet prefixes for that TS. All it is needed is to advertise of the subnet prefixes for that TS. All it is needed is to advertise
the IP/MAC addresses associated with the TS itself and exercise MAC the IP/MAC addresses associated with the TS itself and exercise MAC
mobility procedures for that TS. The recursive route resolution mobility procedures for that TS. The recursive route resolution
automatically takes care of the updates for the subnet prefixes of automatically takes care of the updates for the subnet prefixes of
that TS. that TS.
Figure below illustrates this scenario where a given tenant (e.g., an Figure 7 illustrates this scenario where a given tenant (e.g., an IP-
IP-VPN service) has three subnets represented by MAC-VRF1, MAC-VRF2, VPN service) has three subnets represented by MAC-VRF1, MAC-VRF2, and
and MAC-VRF3 across two NVEs. There are four TSes associated with MAC-VRF3 across two NVEs. There are four TSes associated with these
these three MAC-VRFs -- i.e., TS1 is connected to MAC-VRF1 on NVE1, three MAC-VRFs -- i.e., TS1 is connected to MAC-VRF1 on NVE1, TS2 is
TS2 is connected to MAC-VRF2 on NVE1, TS3 is connected to MAC- VRF3 connected to MAC-VRF2 on NVE1, TS3 is connected to MAC- VRF3 on NVE2,
on NVE2, and TS4 is connected to MAC-VRF1 on NVE2. TS1 has two and TS4 is connected to MAC-VRF1 on NVE2. TS1 has two subnet
subnet prefixes (SN1 and SN2) and TS3 has a single subnet prefix, prefixes (SN1 and SN2) and TS3 has a single subnet prefix, SN3. The
SN3. The MAC-VRFs on each NVE are associated with their MAC-VRFs on each NVE are associated with their corresponding IP-VRF
corresponding IP-VRF using their IRB interfaces. When TS4 and TS1 using their IRB interfaces. When TS4 and TS1 exchange intra- subnet
exchange intra- subnet traffic, only L2 forwarding (bridging) part of traffic, only L2 forwarding (bridging) part of the IRB solution is
the IRB solution is used (i.e., the traffic only goes through their used (i.e., the traffic only goes through their MAC- VRFs); however,
MAC- VRFs); however, when TS3 wants to forward traffic to SN1 or SN2 when TS3 wants to forward traffic to SN1 or SN2 sitting behind TS1
sitting behind TS1 (inter-subnet traffic), then both bridging and (inter-subnet traffic), then both bridging and routing parts of the
routing parts of the IRB solution are exercised (i.e., the traffic IRB solution are exercised (i.e., the traffic goes through the
goes through the corresponding MAC-VRFs and IP-VRFs). The following corresponding MAC-VRFs and IP-VRFs). The following subsections
subsections describe the control and data planes operations for this describe the control and data planes operations for this IRB scenario
IRB scenario in details. in details.
NVE1 +----------+ NVE1 +----------+
SN1--+ +-------------+ | | SN1--+ +-------------+ | |
|--TS1-----|(MAC- \ | | | |--TS1-----|(MAC- \ | | |
SN2--+ IP1/M1 | VRF1) \ | | | SN2--+ IP1/M1 | VRF1) \ | | |
| (IP-VRF)|---| | | (IP-VRF)|---| |
| / | | | | / | | |
TS2-----|(MAC- / | | MPLS/ | TS2-----|(MAC- / | | MPLS/ |
IP2/M2 | VRF2) | | VxLAN/ | IP2/M2 | VRF2) | | VxLAN/ |
+-------------+ | NVGRE | +-------------+ | NVGRE |
skipping to change at page 29, line 30 skipping to change at page 29, line 46
| / | | | | / | | |
TS4-----|(MAC- / | | | TS4-----|(MAC- / | | |
IP4/M4 | VRF1) | | | IP4/M4 | VRF1) | | |
+-------------+ +----------+ +-------------+ +----------+
NVE2 NVE2
Figure 7: IRB forwarding on NVEs for subnets behind TSes Figure 7: IRB forwarding on NVEs for subnets behind TSes
9.2.1. Control Plane Operation 9.2.1. Control Plane Operation
Each NVE advertises a Route Type-5 (RT-5, IP Prefix Route defined in Each NVE advertises a Route Type-5 (EVPN RT-5, IP Prefix Route
[I-D.ietf-bess-evpn-prefix-advertisement]) for each of its subnet defined in [I-D.ietf-bess-evpn-prefix-advertisement]) for each of its
prefixes with the IP address of its TS as the next hop (gateway subnet prefixes with the IP address of its TS as the next hop
address field) as follow: (gateway address field) as follows:
o RD associated with the IP-VRF o RD associated with the IP-VRF
o ESI = 0 o ESI = 0
o Ethernet Tag = 0; o Ethernet Tag = 0;
o IP Prefix Length = 0 to 32 or 0 to 128 o IP Prefix Length = 0 to 32 or 0 to 128
o IP Prefix = SNi o IP Prefix = SNi
o Gateway Address = IPi; IP address of TS o Gateway Address = IPi; IP address of TS
o MPLS Label = 0 o MPLS Label = 0
This RT-5 is advertised with one or more Route Targets associated
with the IP-VRF from which the route is originated.
Each NVE also advertises an RT-2 (MAC/IP Advertisement Route) along This EVPN RT-5 is advertised with one or more Route Targets
with their associated Route Targets and Extended Communities for each associated with the IP-VRF from which the route is originated.
of its TSes exactly as described in section 9.1.1.
Upon receiving the RT-5 advertisement, the receiving NVE performs the Each NVE also advertises an EVPN RT-2 (MAC/IP Advertisement Route)
following: along with their associated Route Targets and Extended Communities
for each of its TSes exactly as described in section 9.1.1.
Upon receiving the EVPN RT-5 advertisement, the receiving NVE
performs the following:
o It uses the Route Target to identify the corresponding IP-VRF o It uses the Route Target to identify the corresponding IP-VRF
o It imports the IP prefix into its corresponding IP-VRF that is o It imports the IP prefix into its corresponding IP-VRF that is
configured with an import RT that is one of the RTs being carried configured with an import RT that is one of the RTs being carried
by the RT-5 route along with the IP address of the associated TS by the EVPN RT-5 route along with the IP address of the associated
as its next hop. TS as its next hop.
When receiving the RT-2 advertisement, the receiving NVE imports MAC/ When receiving the EVPN RT-2 advertisement, the receiving NVE imports
IP addresses of the TS into the corresponding MAC-VRF and IP-VRF per MAC/IP addresses of the TS into the corresponding MAC-VRF and IP-VRF
section 9.1.1. When both routes exist, recursive route resolution is per section 9.1.1. When both routes exist, recursive route
performed to resolve the IP prefix (received in RT-5) to its resolution is performed to resolve the IP prefix (received in EVPN
corresponding NVE's IP address (e.g., its BGP next hop). BGP next RT-5) to its corresponding NVE's IP address (e.g., its BGP next hop).
hop will be used as underlay tunnel destination address (e.g., VTEP BGP next hop will be used as the underlay tunnel destination address
DA for VxLAN encapsulation) and Router's MAC will be used as inner (e.g., VTEP DA for VxLAN encapsulation) and Router's MAC will be used
MAC for VxLAN encapsulation. as inner MAC for VxLAN encapsulation.
9.2.2. Data Plane Operation 9.2.2. Data Plane Operation
The following description of the data-plane operation describes just The following description of the data-plane operation describes just
the logical functions and the actual implementation may differ. Lets the logical functions and the actual implementation may differ. Lets
consider data-plane operation when a host on SN1 sitting behind TS1 consider data-plane operation when a host on SN1 sitting behind TS1
wants to send traffic to a host sitting behind SN3 behind TS3. wants to send traffic to a host sitting behind SN3 behind TS3.
o TS1 send a packet with MAC DA corresponding to the MAC-VRF1 IRB o TS1 send a packet with MAC DA corresponding to the MAC-VRF1 IRB
interface of NVE1, and VLAN-tag corresponding to MAC-VRF1. interface of NVE1, and VLAN-tag corresponding to MAC-VRF1.
o Upon receiving the packet, the ingress NVE1 uses VLAN-tag to o Upon receiving the packet, the ingress NVE1 uses VLAN-tag to
identify the MAC-VRF1. It then looks up the MAC DA and forwards identify the MAC-VRF1. It then looks up the MAC DA and forwards
the frame to its IRB interface just like section 9.1.1. the frame to its IRB interface just like section 9.1.1.
o The Ethernet header of the packet is stripped and the packet is o The Ethernet header of the packet is stripped and the packet is
fed to the IP-VRF; where, IP lookup is performed on the fed to the IP-VRF; where, IP lookup is performed on the
destination address. This lookup yields the fields needed for destination address. This lookup yields the fields needed for
VxLAN encapsulation with NVE2's MAC address as the inner MAC DA, VxLAN encapsulation with NVE2's MAC address as the inner MAC DA,
NVE'2 IP address as the VTEP DA, and the VNI. MAC SA is set to NVE'2 IP address as the VTEP DA, and the VNI. MAC SA is set to
NVE1's MAC address and VTEP SA is set to NVE1's IP address. NVE1's MAC address and VTEP SA is set to NVE1's IP address. NVE1
also decrements the TTL/hop limit for that packet by one and if it
reaches zero, NVE1 discards the packet.
o The packet is then encapsulated with the proper header based on o The packet is then encapsulated with the proper header based on
the above info and is forwarded to the egress NVE (NVE2). the above info and is forwarded to the egress NVE (NVE2).
o On the egress NVE (NVE2), assuming the packet is VxLAN o On the egress NVE (NVE2), assuming the packet is VxLAN
encapsulated, the VxLAN and the inner Ethernet headers are removed encapsulated, the VxLAN and the inner Ethernet headers are removed
and the resultant IP packet is fed to the IP-VRF associated with and the resultant IP packet is fed to the IP-VRF associated with
that the VNI. that the VNI.
o Next, a lookup is performed based on IP DA (which is in SN3) in o Next, a lookup is performed based on IP DA (which is in SN3) in
the associated IP-VRF of NVE2. The IP lookup yields the access- the associated IP-VRF of NVE2. The IP lookup yields the access-
facing IRB interface over which the packet needs to be sent. facing IRB interface over which the packet needs to be sent.
Before sending the packet over this interface, the ARP table is Before sending the packet over this interface, the ARP table is
consulted to get the destination TS (TS3) MAC address. consulted to get the destination TS (TS3) MAC address. NVE2 also
decrements the TTL/hop limit for that packet by one and if it
reaches zero, NVE2 discards the packet.
o The IP packet is encapsulated with an Ethernet header with the MAC o The IP packet is encapsulated with an Ethernet header with the MAC
SA set to that of the access-facing IRB interface of the egress SA set to that of the access-facing IRB interface of the egress
NVE (NVE2) and the MAC DA is set to that of destination TS (TS3) NVE (NVE2) and the MAC DA is set to that of destination TS (TS3)
MAC address. The packet is sent to the corresponding MAC-VRF3 and MAC address. The packet is sent to the corresponding MAC-VRF3 and
after a lookup of MAC DA, is forwarded to the destination TS (TS3) after a lookup of MAC DA, is forwarded to the destination TS (TS3)
over the corresponding interface. over the corresponding interface.
10. Acknowledgements 10. Acknowledgements
The authors would like to thank Sami Boutros, Jeffrey Zhang, The authors would like to thank Sami Boutros, Jeffrey Zhang,
Krzysztof Szarkowicz, Lukas Krattiger and Neeraj Malhotra for their Krzysztof Szarkowicz, Lukas Krattiger and Neeraj Malhotra for their
valuable comments. The authors would also like to thank Linda valuable comments. The authors would also like to thank Linda
Dunbar, Florin Balus, Yakov Rekhter, Wim Henderickx, Lucy Yong, and Dunbar, Florin Balus, Yakov Rekhter, Wim Henderickx, Lucy Yong, and
Dennis Cai for their feedbacks and contributions. Dennis Cai for their feedback and contributions.
11. Security Considerations 11. Security Considerations
The security considerations for layer-2 forwarding in this document The security considerations for layer-2 forwarding in this document
follow that of [RFC7432] for MPLS encapsulation and it follows that follow that of [RFC7432] for MPLS encapsulation and it follows that
of [RFC8365] for VxLAN or GENEVE encapsulations. This section of [RFC8365] for VxLAN or NVGRE encapsulations. This section
describes additional considerations. describes additional considerations.
This document describes a set of procedures for Inter-Subnet This document describes a set of procedures for Inter-Subnet
Forwarding of tenant traffic across PEs (or NVEs). These procedures Forwarding of tenant traffic across PEs (or NVEs). These procedures
include both layer-2 forwarding and layer-3 routing on a packet by include both layer-2 forwarding and layer-3 routing on a packet by
packet basis. The security consideration for layer-3 routing is this packet basis. The security consideration for layer-3 routing in this
document follows that of [RFC4365] with the exception for application document follows that of [RFC4365] with the exception for the
of routing protocols between CEs and PEs. Contrary to [RFC4364], application of routing protocols between CEs and PEs. Contrary to
this document does not describe route distribution techniques between [RFC4364], this document does not describe route distribution
CEs and PEs, but rather considers the CEs as TSes or VAs that do not techniques between CEs and PEs, but rather considers the CEs as TSes
run dynamic routing protocols. This can be considered a security or VAs that do not run dynamic routing protocols. This can be
advantage, since dynamic routing protocols can be blocked on the NVE/ considered a security advantage, since dynamic routing protocols can
PE ACs, not allowing the tenant to interact with the infrastructure's be blocked on the NVE/PE ACs, not allowing the tenant to interact
dynamic routing protocols. with the infrastructure's dynamic routing protocols.
The VPN scheme described in this document does not provide the The VPN scheme described in this document does not provide the
quartet of security properties mentioned in [RFC4365] quartet of security properties mentioned in [RFC4365]
(confidentiality protection, source authentication, integrity (confidentiality protection, source authentication, integrity
protection, replay protection). If these are desired, they must be protection, replay protection). If these are desired, they must be
provided by mechanisms that are outside the scope of the VPN provided by mechanisms that are outside the scope of the VPN
mechanisms. mechanisms.
In this document, the RT-5 is used for certain scenarios. This route In this document, the EVPN RT-5 is used for certain scenarios. This
uses an Overlay Index that requires a recursive resolution to a route uses an Overlay Index that requires a recursive resolution to a
different EVPN route (an RT-2). Because of this, it is worth noting different EVPN route (an EVPN RT-2). Because of this, it is worth
that any action that ends up filtering or modifying the RT-2 route noting that any action that ends up filtering or modifying the EVPN
used to convey the Overlay Indexes, will modify the resolution of the RT-2 route used to convey the Overlay Indexes, will modify the
RT-5 and therefore the forwarding of packets to the remote subnet. resolution of the EVPN RT-5 and therefore the forwarding of packets
to the remote subnet.
12. IANA Considerations 12. IANA Considerations
IANA has allocated a new transitive extended community Type of 0x06 IANA has allocated a new transitive extended community Type of 0x06
and Sub-Type of 0x03 for EVPN Router's MAC Extended Community. and Sub-Type of 0x03 for EVPN Router's MAC Extended Community.
13. References 13. References
13.1. Normative References 13.1. Normative References
[I-D.ietf-bess-evpn-prefix-advertisement] [I-D.ietf-bess-evpn-prefix-advertisement]
Rabadan, J., Henderickx, W., Drake, J., Lin, W., and A. Rabadan, J., Henderickx, W., Drake, J., Lin, W., and A.
Sajassi, "IP Prefix Advertisement in EVPN", draft-ietf- Sajassi, "IP Prefix Advertisement in EVPN", draft-ietf-
bess-evpn-prefix-advertisement-11 (work in progress), May bess-evpn-prefix-advertisement-11 (work in progress), May
2018. 2018.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>. <https://www.rfc-editor.org/info/rfc2119>.
[RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private
Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February
2006, <https://www.rfc-editor.org/info/rfc4364>. 2006, <https://www.rfc-editor.org/info/rfc4364>.
[RFC7348] Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger,
L., Sridhar, T., Bursell, M., and C. Wright, "Virtual
eXtensible Local Area Network (VXLAN): A Framework for
Overlaying Virtualized Layer 2 Networks over Layer 3
Networks", RFC 7348, DOI 10.17487/RFC7348, August 2014,
<https://www.rfc-editor.org/info/rfc7348>.
[RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A.,
Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based
Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February
2015, <https://www.rfc-editor.org/info/rfc7432>. 2015, <https://www.rfc-editor.org/info/rfc7432>.
[RFC7606] Chen, E., Ed., Scudder, J., Ed., Mohapatra, P., and K. [RFC7606] Chen, E., Ed., Scudder, J., Ed., Mohapatra, P., and K.
Patel, "Revised Error Handling for BGP UPDATE Messages", Patel, "Revised Error Handling for BGP UPDATE Messages",
RFC 7606, DOI 10.17487/RFC7606, August 2015, RFC 7606, DOI 10.17487/RFC7606, August 2015,
<https://www.rfc-editor.org/info/rfc7606>. <https://www.rfc-editor.org/info/rfc7606>.
[RFC7637] Garg, P., Ed. and Y. Wang, Ed., "NVGRE: Network
Virtualization Using Generic Routing Encapsulation",
RFC 7637, DOI 10.17487/RFC7637, September 2015,
<https://www.rfc-editor.org/info/rfc7637>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>. May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8365] Sajassi, A., Ed., Drake, J., Ed., Bitar, N., Shekhar, R., [RFC8365] Sajassi, A., Ed., Drake, J., Ed., Bitar, N., Shekhar, R.,
Uttaro, J., and W. Henderickx, "A Network Virtualization Uttaro, J., and W. Henderickx, "A Network Virtualization
Overlay Solution Using Ethernet VPN (EVPN)", RFC 8365, Overlay Solution Using Ethernet VPN (EVPN)", RFC 8365,
DOI 10.17487/RFC8365, March 2018, DOI 10.17487/RFC8365, March 2018,
<https://www.rfc-editor.org/info/rfc8365>. <https://www.rfc-editor.org/info/rfc8365>.
skipping to change at page 33, line 28 skipping to change at page 34, line 18
Malhotra, N., Sajassi, A., Pattekar, A., Lingala, A., Malhotra, N., Sajassi, A., Pattekar, A., Lingala, A.,
Rabadan, J., and J. Drake, "Extended Mobility Procedures Rabadan, J., and J. Drake, "Extended Mobility Procedures
for EVPN-IRB", draft-ietf-bess-evpn-irb-extended- for EVPN-IRB", draft-ietf-bess-evpn-irb-extended-
mobility-03 (work in progress), May 2020. mobility-03 (work in progress), May 2020.
[I-D.ietf-idr-tunnel-encaps] [I-D.ietf-idr-tunnel-encaps]
Patel, K., Velde, G., Sangli, S., and J. Scudder, "The BGP Patel, K., Velde, G., Sangli, S., and J. Scudder, "The BGP
Tunnel Encapsulation Attribute", draft-ietf-idr-tunnel- Tunnel Encapsulation Attribute", draft-ietf-idr-tunnel-
encaps-17 (work in progress), July 2020. encaps-17 (work in progress), July 2020.
[I-D.ietf-nvo3-vxlan-gpe]
Maino, F., Kreeger, L., and U. Elzur, "Generic Protocol
Extension for VXLAN (VXLAN-GPE)", draft-ietf-nvo3-vxlan-
gpe-10 (work in progress), July 2020.
[RFC4365] Rosen, E., "Applicability Statement for BGP/MPLS IP [RFC4365] Rosen, E., "Applicability Statement for BGP/MPLS IP
Virtual Private Networks (VPNs)", RFC 4365, Virtual Private Networks (VPNs)", RFC 4365,
DOI 10.17487/RFC4365, February 2006, DOI 10.17487/RFC4365, February 2006,
<https://www.rfc-editor.org/info/rfc4365>. <https://www.rfc-editor.org/info/rfc4365>.
[RFC5798] Nadas, S., Ed., "Virtual Router Redundancy Protocol (VRRP) [RFC5798] Nadas, S., Ed., "Virtual Router Redundancy Protocol (VRRP)
Version 3 for IPv4 and IPv6", RFC 5798, Version 3 for IPv4 and IPv6", RFC 5798,
DOI 10.17487/RFC5798, March 2010, DOI 10.17487/RFC5798, March 2010,
<https://www.rfc-editor.org/info/rfc5798>. <https://www.rfc-editor.org/info/rfc5798>.
[RFC7348] Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger,
L., Sridhar, T., Bursell, M., and C. Wright, "Virtual
eXtensible Local Area Network (VXLAN): A Framework for
Overlaying Virtualized Layer 2 Networks over Layer 3
Networks", RFC 7348, DOI 10.17487/RFC7348, August 2014,
<https://www.rfc-editor.org/info/rfc7348>.
[RFC7365] Lasserre, M., Balus, F., Morin, T., Bitar, N., and Y. [RFC7365] Lasserre, M., Balus, F., Morin, T., Bitar, N., and Y.
Rekhter, "Framework for Data Center (DC) Network Rekhter, "Framework for Data Center (DC) Network
Virtualization", RFC 7365, DOI 10.17487/RFC7365, October Virtualization", RFC 7365, DOI 10.17487/RFC7365, October
2014, <https://www.rfc-editor.org/info/rfc7365>. 2014, <https://www.rfc-editor.org/info/rfc7365>.
Authors' Addresses Authors' Addresses
Ali Sajassi Ali Sajassi
Cisco Systems Cisco Systems
MILPITAS, CALIFORNIA 95035 MILPITAS, CALIFORNIA 95035
 End of changes. 92 change blocks. 
223 lines changed or deleted 263 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/