BFD for VXLAN
VMwaresantosh.pallagatti@gmail.comIndividual Contributorsudarsan.225@gmail.comCiscovenggovi@cisco.comCiscommudigon@cisco.comZTE Corp.gregimirsky@gmail.com
Routing
BFDBFDBFD for VXLANThis document describes the use of the Bidirectional Forwarding Detection (BFD) protocol
in point-to-point Virtual eXtensible Local Area Network (VXLAN) tunnels
used to form an overlay network.
"Virtual eXtensible Local Area Network" (VXLAN) provides
an encapsulation scheme that allows building an overlay network by
decoupling the address space of the attached virtual hosts from that of the network.
One use of VXLAN is in data centers interconnecting virtual machines (VMs)
of a tenant. VXLAN addresses requirements of the Layer 2 and
Layer 3 data center network infrastructure in the presence of VMs in a multi-tenant environment by
providing a Layer 2 overlay scheme on a Layer 3 network .
Another use is as an encapsulation for Ethernet VPN .
This document is written assuming the use of VXLAN for virtualized
hosts and refers to VMs and VXLAN Tunnel End Points (VTEPs) in hypervisors. However, the
concepts are equally applicable to non-virtualized hosts attached to
VTEPs in switches.
In the absence of a router in the overlay, a VM can communicate with another VM only if they are on the same VXLAN segment.
VMs are unaware of VXLAN tunnels as a VXLAN tunnel is terminated on a VTEP.
VTEPs are responsible for encapsulating and decapsulating frames exchanged among
VMs.
The ability to monitor path continuity, i.e., perform proactive continuity check (CC) for point-to-point (p2p) VXLAN tunnels, is important.
The asynchronous mode of BFD, as defined in , is used to monitor a p2p VXLAN tunnel.
In the case where a Multicast Service Node (MSN) (as described in Section 3.3
of ) resides behind a Network Virtualization Endpoint (NVE), the mechanisms described in this
document apply and can, therefore, be used to test the connectivity from
the source NVE to the MSN.
This document describes the use of Bidirectional Forwarding Detection (BFD) protocol
to enable monitoring continuity of the path between VXLAN VTEPs,
performing as Network Virtualization Endpoints,
and/or availability of a replicator multicast service node.
BFD Bidirectional Forwarding Detection CC Continuity Checkp2p Point-to-pointMSN Multicast Service NodeNVE Network Virtualization EndpointVFI Virtual Forwarding InstanceVM Virtual MachineVNI VXLAN Network Identifier (or VXLAN Segment ID)VTEP VXLAN Tunnel End PointVXLAN Virtual eXtensible Local Area Network
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED",
"MAY", and "OPTIONAL" in this document are to be interpreted as
described in BCP 14
when, and only when, they appear in all capitals, as shown here.
illustrates the scenario with two servers, each of them hosting two VMs.
The servers host VTEPs that terminate two VXLAN tunnels with VXLAN Network Identifier (VNI) number 100
and 200 respectively. Separate BFD sessions can be
established between the VTEPs (IP1 and IP2) for monitoring
each of the VXLAN tunnels (VNI 100 and 200). Using a BFD session to monitor a set of VXLAN VNIs
between the same pair of VTEPs might help to detect and localize problems caused by misconfiguration.
An implementation that supports this specification MUST
be able to control the number of BFD sessions
that can be created between the same pair of VTEPs.
BFD packets intended for a VTEP MUST NOT be forwarded to a VM, as a VM may drop BFD packets,
leading to a false negative. This method is applicable whether the VTEP is a virtual or physical device.
At the same time, a service layer BFD session may be used between the tenants of VTEPs IP1 and IP2
to provide end-to-end fault management (this use case is outside the scope of this document). In
such case, for VTEPs BFD Control packets of that session are indistinguishable from data packets.
For BFD Control packets encapsulated in VXLAN (),
the inner destination IP address
SHOULD be set to one of the loopback addresses from
127/8 range for IPv4 or to one of IPv4-mapped IPv4 loopback addresses from ::ffff:127.0.0.0/104 range for IPv6.
There could be a firewall configured on VTEP to block loopback addresses if set as the destination
IP in the inner IP header. It is RECOMMENDED to allow addresses from the loopback range through a firewall only
if they are used as the destination IP addresses in the inner IP header
and the destination UDP port is set to 3784 .
BFD packets MUST be encapsulated and sent to a remote VTEPs as explained in this section.
Implementations SHOULD ensure that the BFD packets follow the same
forwarding path as VXLAN data packets within the sender system.
BFD packets are encapsulated in VXLAN as described below.
The VXLAN packet format is defined in Section 5 of .
The Outer IP/UDP and VXLAN headers MUST
be encoded by the sender as defined in .The BFD packet MUST be carried inside the inner Ethernet frame of the VXLAN packet.
The choice of Destination MAC and Destination IP addresses for the inner Ethernet frame MUST
ensure that the BFD Control packet is not forwarded to a tenant but is processed locally at the remote VTEP.
The inner Ethernet frame carrying the BFD Control packet- has the following format:
Ethernet Header:
Destination MAC: This MUST NOT be of one of tenant's MAC addresses.
The destination MAC address MAY be the address associated with the destination VTEP.
The MAC address may be either configured or learned via a control plane protocol. The
details of how the MAC address is obtained
are outside the scope of this document. Source MAC: MAC address associated with the originating VTEP IP header:
Destination IP:
IP address MUST NOT be of one of tenant's IP addresses.
The IP address SHOULD be selected
from the range 127/8 for IPv4, for IPv6 - from the range ::ffff:127.0.0.0/104.
Alternatively, the destination IP address MAY be set to VTEP's IP address.Source IP: IP address of the originating VTEP.TTL or Hop Limit: MUST be set to 1 to ensure that the
BFD packet is not routed within the Layer 3 underlay network.
This addresses the scenario when the inner IP destination address is
of the VXLAN gateway and there is a router in the underlay which removes
the VXLAN header, then it is possible to route the packet as VXLAN
gateway address is routable address.
The fields of the UDP header and the BFD Control packet are encoded as specified
in .
Once a packet is received, the VTEP MUST validate the packet.
If the Destination MAC of the inner Ethernet frame matches one of the MAC addresses
associated with the VTEP the packet
MUST be processed further. If the Destination MAC of the inner Ethernet frame doesn't match any of VTEP's
MAC addresses, then the processing of the received VXLAN packet MUST
follow the procedures described in Section 4.1 of .
If the BFD session is using the Management VNI (),
BFD Control packets with unknown MAC address MUST NOT be forwarded to VMs.
The UDP destination port and the TTL or Hop Limit of the inner IP packet MUST be validated to determine if the received packet
can be processed by BFD.Demultiplexing of IP BFD packet has been defined in Section 3 of .
Since multiple BFD sessions may be running between two VTEPs, there
needs to be a mechanism for demultiplexing received BFD packets to
the proper session. For demultiplexing packets with
Your Discriminator equal to 0, a BFD session MUST be identified using
the logical link over which the BFD Control packet is received. In the case of VXLAN, the VNI number
identifies that logical link.
If BFD packet is received with non-zero Your Discriminator, then the BFD session MUST
be demultiplexed only with Your Discriminator as the key.
In most cases, a single BFD session is sufficient for the given VTEP to monitor
the reachability of a remote VTEP, regardless of the number of VNIs.
When the single BFD session is used to monitor the reachability of the remote VTEP,
an implementation SHOULD choose any of the VNIs. An implementation MAY support the use
of the Management VNI as control and management channel between VTEPs. The selection of the VNI number
of the Management VNI MUST be controlled through management plane. An implementation MAY use VNI number 1 as
the default value for the Management VNI. All VXLAN packets received on the Management VNI MUST be processed locally
and MUST NOT be forwarded to a tenant.
Support for echo BFD is outside the scope of this document.
This specification has no IANA action requested. This section may be deleted before the publication.
The document requires setting the inner IP TTL or Hop Limit to 1, which could be used as a DDoS attack vector.
Thus the implementation MUST have throttling in place to control the rate of BFD Control packets sent to the control plane.
On the other hand, over-aggressive throttling of BFD Control packets may become the cause of the inability to form and maintain
BFD session at scale. Hence, throttling of BFD Control packets SHOULD be adjusted to permit BFD to work according to its
procedures.
This document recommends using an address from the Internal host loopback addresses
127/8 range for IPv4 or an IP4-mapped IPv4 loopback address from ::ffff:127.0.0.0/104 range for IPv6
as the destination IP address in the inner IP header. Using such an address prevents
the forwarding of the encapsulated BFD control message by a transient node in case the VXLAN tunnel is broken as
according to :
A router SHOULD NOT forward, except over a loopback interface, any
packet that has a destination address on network 127. A router
MAY have a switch that allows the network manager to disable these
checks. If such a switch is provided, it MUST default to
performing the checks.
If the implementation supports establishing multiple BFD sessions
between the same pair of VTEPs, there SHOULD be a mechanism
to control the maximum number of such sessions that can be active
at the same time.
Other than setting the value of inner IP TTL or Hop Limit to 1 and limit the number of BFD sessions between the same pair of VTEPs,
this specification does not raise any additional security issues
beyond those discussed in , , and .
Authors would like to thank Jeff Haas of Juniper Networks for his reviews and feedback on this material.Authors would also like to thank Nobo Akiya, Marc Binderberger, Shahram Davari,
Donald E. Eastlake 3rd, and Anoop Ghanwani for the extensive reviews
and the most detailed and helpful comments.