< draft-ietf-nvo3-encap-06.txt   draft-ietf-nvo3-encap-07.txt >
NVO3 Workgroup S. Boutros, Ed. NVO3 Workgroup S. Boutros, Ed.
Internet-Draft Ciena Internet-Draft Ciena
Intended Status: Informational D. Eastlake, Ed. Intended Status: Informational D. Eastlake, Ed.
Futurewei Futurewei
Expires: December 8, 2021 June 9, 2021 Expires: January 28, 2022 July 29, 2021
NVO3 Encapsulation Considerations NVO3 Encapsulation Considerations
draft-ietf-nvo3-encap-06 draft-ietf-nvo3-encap-07
Abstract Abstract
As communicated by the WG Chairs, the IETF NVO3 chairs and Routing As communicated by the WG Chairs, the IETF NVO3 chairs and Routing
Area director have chartered a design team to take forward the Area director have chartered a design team to take forward the
encapsulation discussion and see if there is potential to design a encapsulation discussion and see if there is potential to design a
common encapsulation that addresses the various technical concerns. common encapsulation that addresses the various technical concerns.
There are implications of different encapsulations in real There are implications of different encapsulations in real
environments consisting of both software and hardware implementations environments consisting of both software and hardware implementations
and spanning multiple data centers. For example, OAM functions such and spanning multiple data centers. For example, OAM functions such
skipping to change at page 2, line 4 skipping to change at page 2, line 6
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
https://www.ietf.org/1id-abstracts.html. The list of Internet-Draft https://www.ietf.org/1id-abstracts.html. The list of Internet-Draft
Shadow Directories can be accessed at Shadow Directories can be accessed at
https://www.ietf.org/shadow.html. https://www.ietf.org/shadow.html.
Copyright Notice Copyright Notice
Internet-Draft NVO3 Encapsulation Considerations
Copyright (c) 2021 IETF Trust and the persons identified as the Copyright (c) 2021 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Internet-Draft NVO3 Encapsulation Considerations
Table of Contents Table of Contents
1. Introduction............................................4 1. Introduction............................................4
2. Design Team Goals.......................................4 2. Design Team Goals.......................................4
3. Terminology.............................................5 3. Terminology.............................................5
4. Abbreviations and Acronyms..............................5 4. Abbreviations and Acronyms..............................5
5. Issues with Current Encapsulations......................6 5. Issues with Current Encapsulations......................6
5.1. Geneve................................................6 5.1. Geneve................................................6
5.2. GUE...................................................6 5.2. GUE (Generic UDP Encapsulation).......................6
5.3. VXLAN-GPE.............................................6 5.3. VXLAN-GPE.............................................6
6. Common Encapsulation Considerations.....................7 6. Common Encapsulation Considerations.....................7
6.1. Current Encapsulations................................7 6.1. Current Encapsulations................................7
6.2. Useful Extensions Use Cases...........................7 6.2. Useful Extensions Use Cases...........................7
6.2.1. Telemetry Extensions................................7 6.2.1. Telemetry Extensions................................7
6.2.2. Security/Integrity Extensions.......................8 6.2.2. Security/Integrity Extensions.......................8
6.2.3 Group Base Policy....................................8 6.2.3 Group Based Policy...................................8
6.3. Hardware Considerations...............................9 6.3. Hardware Considerations...............................9
6.4. Extension Size........................................9 6.4. Extension Size........................................9
6.5. Extension Ordering...................................10 6.5. Ordering of Extension Headers........................10
6.6. TLV versus Bit Fields................................10 6.6. TLV versus Bit Fields................................10
6.7. Control Plane Considerations.........................11 6.7. Control Plane Considerations.........................11
6.8. Split NVE............................................12 6.8. Split NVE............................................12
6.9. Larger VNI Considerations............................12 6.9. Larger VNI Considerations............................12
7. Design Team Recommendations............................13 7. Design Team Recommendations............................14
8. Acknowledgements.......................................16
9. Security Considerations................................16
10. IANA Considerations...................................16
11. References............................................17 8. Acknowledgements.......................................17
11.1 Normative References.................................17 9. Security Considerations................................17
11.2 Informative References...............................17 10. IANA Considerations...................................17
Appendix A: Encapsulations Comparison.....................19 11. References............................................18
A.1. Overview.............................................19 11.1 Normative References.................................18
A.2. Extensibility........................................19 11.2 Informative References...............................18
A.2.1. Native Extensibility Support.......................19
A.2.2. Extension Parsing..................................19
A.2.3. Critical Extensions................................20
A.2.4. Maximal Header Length..............................20
A.3. Encapsulation Header.................................20
A.3.1. Virtual Network Identifier (VNI)...................20
A.3.2. Next Protocol......................................20
A.3.3. Other Header Fields................................21
A.4. Comparison Summary...................................21
Contributors..............................................23 Appendix A: Encapsulations Comparison.....................20
A.1. Overview.............................................20
A.2. Extensibility........................................20
A.2.1. Native Extensibility Support.......................20
A.2.2. Extension Parsing..................................20
A.2.3. Critical Extensions................................21
A.2.4. Maximal Header Length..............................21
A.3. Encapsulation Header.................................21
A.3.1. Virtual Network Identifier (VNI)...................21
A.3.2. Next Protocol......................................21
A.3.3. Other Header Fields................................22
A.4. Comparison Summary...................................23
Internet-Draft NVO3 Encapsulation Considerations Contributors..............................................25
1. Introduction 1. Introduction
As communicated by the WG Chairs, the NVO3 WG Charter states that it As communicated by the WG Chairs, the NVO3 WG Charter states that it
may produce requirements for network virtualization data planes based may produce requirements for network virtualization data planes based
on encapsulation of virtual network traffic over an IP-based underlay on encapsulation of virtual network traffic over an IP-based underlay
data plane. Such requirements should consider OAM and security. data plane. Such requirements should consider OAM and security.
Based on these requirements the WG will select, extend, and/or Based on these requirements the WG will select, extend, and/or
develop one or more data plane encapsulation format(s). develop one or more data plane encapsulation format(s).
skipping to change at page 4, line 39 skipping to change at page 4, line 37
Berlin that it is undesirable for the working group to progress more Berlin that it is undesirable for the working group to progress more
than one data plane encapsulation. Although consensus could not be than one data plane encapsulation. Although consensus could not be
reached on the list, the overall consensus was for a single reached on the list, the overall consensus was for a single
encapsulation [RFC2418], Section 3.3. encapsulation [RFC2418], Section 3.3.
Nonetheless there has been resistance to converging on a single Nonetheless there has been resistance to converging on a single
encapsulation format. encapsulation format.
2. Design Team Goals 2. Design Team Goals
As communicated by the WG Chairs, the design team should take one of As communicated by the WG Chairs, the design team (DT) should take
the proposed encapsulations and enhance it to address the technical one of the proposed encapsulations and enhance it to address the
concerns. The simple evolution of deployed networks as well as technical concerns. The simple evolution of deployed networks as
applicability to all locations in the NVO3 architecture are goals. well as applicability to all locations in the NVO3 architecture are
The DT should specifically avoid a design that is burdensome on goals. The DT should specifically avoid a design that is burdensome
hardware implementations but should allow future extensibility. The on hardware implementations but should allow future extensibility.
chosen design should also operate well with ICMP and in ECMP The chosen design should also operate well with ICMP and in ECMP
environments. If further extensibility is required, then it should environments. If further extensibility is required, then it should
be done in such a manner that it does not require the consent of an be done in such a manner that it does not require the consent of an
entity outside of the IETF. entity outside of the IETF.
Internet-Draft NVO3 Encapsulation Considerations
3. Terminology 3. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP "OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all 14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here. capitals, as shown here.
4. Abbreviations and Acronyms 4. Abbreviations and Acronyms
DT NVO3 encapsulation Design Team DT NVO3 encapsulation Design Team
EVPN Ethernet VPN [RFC8365]
GUE Generic UDP Encapsulation [I-D.ietf-intarea-gue]
NVO3 Network Virtualization Overlays over Layer 3 NVO3 Network Virtualization Overlays over Layer 3
OAM Operations, Administration, and Maintenance OAM Operations, Administration, and Maintenance
TLV Type, Length, and Value TLV Type, Length, and Value
VNI Virtual Network Identifier VNI Virtual Network Identifier
NVE Network Virtualization Edge NVE Network Virtualization Edge
NVA Network Virtualization Authority NVA Network Virtualization Authority
NIC Network interface card NIC Network interface card
TCAM Ternary Content-Addressable Memory TCAM Ternary Content-Addressable Memory
Transit device - Underlay network devices between NVE(s). Transit device - Underlay network devices between NVE(s).
Internet-Draft NVO3 Encapsulation Considerations
5. Issues with Current Encapsulations 5. Issues with Current Encapsulations
The following subsections describe issues with current encapsulations The following subsections describe issues with current encapsulations
as summarized by the WG Chairs: as summarized by the WG Chairs:
5.1. Geneve 5.1. Geneve
- Can't be implemented cost-effectively in all use cases because - Can't be implemented cost-effectively in all use cases because
variable length header and order of the TLVs makes is costly (in variable length header and order of the TLVs makes is costly (in
terms of number of gates) to implement in hardware. terms of number of gates) to implement in hardware.
- Header doesn't fit into largest commonly available parse buffer - Header doesn't fit into largest commonly available parse buffer
(256 bytes in NIC). Cannot justify doubling buffer size unless it is (256 bytes in NIC). Cannot justify doubling buffer size unless it is
mandatory for hardware to process additional option fields. mandatory for hardware to process additional option fields.
5.2. GUE 5.2. GUE (Generic UDP Encapsulation)
- There were a significant number of objections related to the - There were a significant number of objections to GUE
complexity of implementation in hardware, similar to those noted for [I-D.ietf-intarea-gue] related to the complexity of implementation in
Geneve above. hardware, similar to those noted for Geneve above.
5.3. VXLAN-GPE 5.3. VXLAN-GPE
- GPE is not day-1 backwards compatible with VXLAN. Although the - GPE is not day-1 backwards compatible with VXLAN [RFC7348].
frame format is similar, it uses a different UDP port, so would Although the frame format is similar, it uses a different UDP port,
require changes to existing implementations even if the rest of the so would require changes to existing implementations even if the rest
GPE frame is the same. of the GPE frame is the same.
- GPE is insufficiently extensible. Numerous extensions and options - GPE is insufficiently extensible. Numerous extensions and options
have been designed for GUE and Geneve. Note that these have not yet have been designed for GUE and Geneve. Note that these have not yet
been validated by the WG. been validated by the WG.
- Security, e.g., of the VNI, has not been addressed by GPE. - Security, e.g., of the VNI, has not been addressed by GPE.
Although a shim header could be used for security and other Although a shim header could be used for security and other
extensions, this has not been defined yet and its implications on extensions, this has not been defined yet and its implications on
offloading in NICs are not understood. offloading in NICs are not understood.
Internet-Draft NVO3 Encapsulation Considerations
6. Common Encapsulation Considerations 6. Common Encapsulation Considerations
6.1. Current Encapsulations 6.1. Current Encapsulations
Appendix A includes a detailed comparison between the three proposed Appendix A includes a detailed comparison between the three proposed
encapsulations. The comparison indicates several common properties encapsulations. The comparison indicates several common properties
but also three major differences among the encapsulations: but also three major differences among the encapsulations:
- Extensibility: Geneve and GUE were defined with built-in - Extensibility: Geneve and GUE were defined with built-in
extensibility, while VXLAN-GPE is not inherently extensible. Note extensibility, while VXLAN-GPE is not inherently extensible. Note
skipping to change at page 7, line 47 skipping to change at page 7, line 45
In several scenarios it is beneficial to make information about the In several scenarios it is beneficial to make information about the
path a packet took through the network or through a network device as path a packet took through the network or through a network device as
well as associated telemetry information available to the operator. well as associated telemetry information available to the operator.
This includes not only tasks like debugging, troubleshooting, and This includes not only tasks like debugging, troubleshooting, and
network planning and optimization but also policy or service level network planning and optimization but also policy or service level
agreement compliance checks. agreement compliance checks.
Packet scheduling algorithms, especially for balancing traffic across Packet scheduling algorithms, especially for balancing traffic across
equal cost paths or links, often leverage information contained equal cost paths or links, often leverage information contained
within the packet, such as protocol number, IP-address, or MAC- within the packet, such as protocol number, IP address, or MAC
address. Probe packets would thus either need to be sent between the address. Probe packets would thus either need to be sent between the
exact same endpoints with the exact same parameters, or probe packets exact same endpoints with the exact same parameters, or probe packets
would need to be artificially constructed as "fake" packets and would need to be artificially constructed as "fake" packets and
Internet-Draft NVO3 Encapsulation Considerations
inserted along the path. Both approaches are often not feasible from inserted along the path. Both approaches are often not feasible from
an operational perspective, be it that access to the end-system is an operational perspective, be it that access to the end-system is
not feasible, or that the diversity of parameters and associated not feasible, or that the diversity of parameters and associated
probe packets to be created is simply too large. An extension probe packets to be created is simply too large. An extension
providing an in-band telemetry mechanism is an alternative in those providing an in-band telemetry mechanism [I-D.ietf-ippm-ioam-data] is
cases. an alternative in those cases.
6.2.2. Security/Integrity Extensions 6.2.2. Security/Integrity Extensions
Since the currently proposed NVO3 encapsulations do not protect their Since the currently proposed NVO3 encapsulations do not protect their
headers, a single bit corruption in the VNI field could deliver a headers, a single bit corruption in the VNI field could deliver a
packet to the wrong tenant. Extensions are needed to use any packet to the wrong tenant. Extension headers are needed to use any
sophisticated security. sophisticated security.
The possibility of VNI spoofing with an NVO3 protocol is exacerbated The possibility of VNI spoofing with an NVO3 protocol is exacerbated
by using UDP. Systems typically have no restrictions on applications by using UDP. Systems typically have no restrictions on applications
being able to send to any UDP port so an unprivileged application can being able to send to any UDP port so an unprivileged application can
trivially spoof VXLAN packets for instance, including using arbitrary trivially spoof VXLAN [RFC7348] packets for instance, including using
VNIs. arbitrary VNIs.
One can envision HMAC-like support in some NVO3 extension to One can envision HMAC-like support in an NVO3 extension to
authenticate the header and the outer IP addresses, thereby authenticate the header and the outer IP addresses, thereby
preventing attackers from injecting packets with spoofed VNIs. preventing attackers from injecting packets with spoofed VNIs.
Another aspect of security is payload security. Essentially this is Another aspect of security is payload security. Essentially this is
to make packets that look like IP|UDP|NVO3 Encap|DTLS/IPSEC-ESP to make packets that look like IP|UDP|NVO3 Encap|DTLS/IPSEC-ESP
Extension|payload. This is nice since we still have the UDP header Extension|payload. This is desireable since we still have the UDP
for ECMP, the NVO3 header is in plain text so it can be read by header for ECMP, the NVO3 header is in plain text so it can be read
network elements, and different security or other payload transforms by network elements, and different security or other payload
can be supported on a single UDP port (we don't need a separate UDP transforms can be supported on a single UDP port (we don't need a
for DTLS/IPSEC). separate UDP port for DTLS/IPSEC).
6.2.3 Group Base Policy 6.2.3 Group Based Policy
Another use case would be to carry the Group Based Policy (GBP) Another use case would be to carry the Group Based Policy (GBP)
source group information within a NVO3 header extension in a similar source group information within a NVO3 header extension in a similar
manner as has been implemented for VXLAN manner as has been implemented for VXLAN
[I-D.smith-vxlan-group-policy]. This allows various forms of policy [I-D.smith-vxlan-group-policy]. This allows various forms of policy
such as access control and QoS to be applied between abstract groups such as access control and QoS to be applied between abstract groups
rather than coupled to specific endpoint addresses. rather than coupled to specific endpoint addresses.
Internet-Draft NVO3 Encapsulation Considerations
6.3. Hardware Considerations 6.3. Hardware Considerations
Hardware restrictions should be taken into consideration along with Hardware restrictions should be taken into consideration along with
future hardware enhancements that may provide more flexible metadata future hardware enhancements that may provide more flexible metadata
processing. However, the set of options that need to and will be processing. However, the set of options that need to and will be
implemented in hardware will be a subset of what is implemented in implemented in hardware will be a subset of what is implemented in
software, since software NVEs are likely to grow features, and hence software, since software NVEs are likely to grow features, and hence
option support, at a more rapid rate. option support, at a more rapid rate.
We note that it is hard to predict which options will be implemented We note that it is hard to predict which options will be implemented
skipping to change at page 9, line 32 skipping to change at page 9, line 30
purposes. purposes.
A result of this is that it doesn't look useful to prescribe some A result of this is that it doesn't look useful to prescribe some
order of the option so that the ones that are likely to be order of the option so that the ones that are likely to be
implemented in hardware come first; we can't decide such an order implemented in hardware come first; we can't decide such an order
when we define the options, however a control plane can enforce such when we define the options, however a control plane can enforce such
an order for some hardware implementation. an order for some hardware implementation.
We do know that hardware needs to initially be able to efficiently We do know that hardware needs to initially be able to efficiently
skip over the NVO3 header to find the inner payload. That is needed skip over the NVO3 header to find the inner payload. That is needed
both for NICs doing TCP offload and for transit devices and NVEs both for NICs implementing various TCP offload mechanisms and for
applying policy/ACLs to the inner payload. transit devices and NVEs applying policy/ACLs to the inner payload.
6.4. Extension Size 6.4. Extension Size
Extension header length has a significant impact on hardware and Extension header length has a significant impact on hardware and
software implementations. A total header length that is too small software implementations. A total header length that is too small
will unnecessarily constrained software flexibility. A total header will unnecessarily constrain software flexibility. A total header
length that is too large will place a nontrivial cost on hardware length that is too large will place a nontrivial cost on hardware
implementations. Thus, the design team recommends that there be a implementations. Thus, the DT recommends that there be a minimum and
minimum and maximum total extension header length selected. The maximum total extension header length specified. The maximum total
maximum total header length is determined by the bits allocated for header length is determined by the size of the bit field allocated
the total extension header length field. The risk with this approach for the total extension header length field. The risk with this
is that it may be difficult to extend the total header size in the approach is that it may be difficult to extend the total header size
future. The minimum total header length is determined by a in the future. The minimum total header length is determined by a
requirement in the specifications that all implementations must meet. requirement in the specifications that all implementations must meet.
The risk with this approach is that all implementations will only The risk with this approach is that all implementations will only
implement the minimum total header length which would then become the implement the minimum total header length which would then become the
de facto maximum total header length. The recommended minimum total de facto maximum total header length. The recommended minimum total
header length is 64 bytes. header length is 64 bytes.
Single Extension size should always be 4 byte aligned. The size of an extension header should always be 4 byte aligned.
Internet-Draft NVO3 Encapsulation Considerations
The maximum length of a single option should be large enough to meet The maximum length of a single option should be large enough to meet
the different extension use case requirements, e.g., in-band the different extension use case requirements, e.g., in-band
telemetry and future use. telemetry and future use.
6.5. Extension Ordering 6.5. Ordering of Extension Headers
To support hardware nodes at the tunnel endpoint or at a transit To support hardware nodes at the target NVE or at a transit device
device that can process one or a few extensions TLVs in TCAM, a that can process one or a few extension headers in TCAM, a control
control plane in such a deployment can signal a capability to ensure plane in such a deployment can signal a capability to ensure a
a specific TLV will always appear in a specific order, for example specific extension header will always appear in a specific order, for
the first one in the packet. example the first one in the packet.
The order of the TLVs should be hardware friendly for both the sender The order of the extension headers should be hardware friendly for
and the receiver and possibly the transit device also. both the sender and the receiver and possibly the transit device
also.
Transit devices doesn't participate in control plane communication Transit devices don't participate in control plane communication
between the end points and are not required to process the options; between the end points and are not required to process the extension
however, if they do, they need to process only a small subset of headers; however, if they do, they may need to process only a small
options that will be consumed by tunnel endpoints. subset of extension headers that will be consumed by target NVEs.
6.6. TLV versus Bit Fields 6.6. TLV versus Bit Fields
If there is a well-known initial set of options that are likely to be If there is a well-known initial set of options that are likely to be
implemented in software and in hardware, it can be efficient to use implemented in software and in hardware, it can be efficient to use
the bit-field approach as in GUE. However, as described in section the bit fields approach as in GUE. However, as described in section
6.3, if options are added over time and different subsets of options 6.3, if options are added over time and different subsets of options
are likely to be implemented in different pieces of hardware, then it are likely to be implemented in different pieces of hardware, then it
would be hard for the IETF to specify which options should get the would be hard for the IETF to specify which options should get the
early bit fields. TLVs are a lot more flexible, which avoids the early bit fields. TLVs are a lot more flexible, which avoids the
need to determine the relative importance different options. need to determine the relative importance different options.
However, general TLV of arbitrary order, size, and repetition of the However, general TLV of arbitrary order, size, and repetition of the
same order is difficult to implement in hardware. A middle ground is same order is difficult to implement in hardware. A middle ground is
to use TLVs with restrictions on their size and alignment, observing to use TLVs with restrictions on their size and alignment, observing
that individual TLVs can have a fixed length, and support in the that individual TLVs can have a fixed length, and support via the
control plane such that an NVE will only receive options that it control plane a method such that an NVE will only receive options
needs and implements. The control plane approach can potentially be that it needs and implements. The control plane approach can
used to control the order of the TLVs sent to a particular NVE. Note potentially be used to control the order of the TLVs sent to a
that transit devices are not likely to participate in the control particular NVE. Note that transit devices are not likely to
plane; hence, to the extent that they need to participate in option participate in the control plane; hence, to the extent that they need
processing, they need more effort. Transit devices would have issues to participate in option processing, some other method must be used.
with future GUE bits being defined for future options as well. Transit devices would have issues with future GUE bit fields being
defined for future options as well.
A benefit of TLVs from a hardware perspective is that they are self A benefit of TLVs from a hardware perspective is that they are self
describing, i.e., all the information is in the TLV. In a Bit fields describing, i.e., all the information is in the TLV. In a bit field
approach the hardware needs to look up the bit to determine the approach, the hardware needs to look up the bit to determine the
length of the data associated with the bit through some separate length of the data associated with the bit through some separate
Internet-Draft NVO3 Encapsulation Considerations
table, which would add hardware complexity. table, which would add hardware complexity.
There are use cases where multiple modules of software are running on There are use cases where multiple modules of software are running on
an NVE. This can be modules such as a diagnostic module by one an NVE. This can be modules such as a diagnostic module by one
vendor that does packet sampling and another module from a different vendor that does packet sampling and another module from a different
vendor that does a firewall. Using a TLV format, it is easier to vendor that implements a firewall. Using a TLV format, it is easier
have different software modules process different TLVs, which could to have different software modules process different TLVs, which
be standard extensions or vendor specific extensions defined by the could be standard extensions or vendor specific extensions defined by
different vendors, without conflicting with each other. This can the different vendors, without conflicting with each other. This can
help with hardware modularity as well. There are some help with hardware modularity as well. There are some
implementations with options that allows different software, like MAC implementations with options that allows different software modules,
learning and security, to handle different options. like MAC learning and security, to process different options.
6.7. Control Plane Considerations 6.7. Control Plane Considerations
Given that we want to allow considerable flexibility and Given that we want to allow considerable flexibility and
extensibility for, e.g., software NVEs, yet be able to support extensibility for, e.g., software NVEs, yet be able to support
important extensions in less flexible contexts such as hardware NVEs, important extensions in less flexible contexts such as hardware NVEs,
it is useful to consider the control plane. By control plane in this it is useful to consider the control plane. By control plane in this
section we mean both protocols, such as EVPN and others, and section we mean both protocols, such as EVPN [RFC8365] and others,
deployment specific configuration. and deployment specific configuration.
If each NVE can express in the control plane that they only care If each NVE can express in the control plane that it only supports
about particular extensions (could be a single extension, or a few), certain extensions (could be a single extension, or a few), and the
and the source NVEs only include requested extensions in the NVO3 source NVEs only include supported extensions in the NVO3 packets,
packets, then the target NVE can both use a simpler parser (e.g., a then the target NVE can both use a simpler parser (e.g., a TCAM might
TCAM might be usable to look for a single NVO3 extension) and the be usable to look for a single NVO3 extension) and the depth of the
depth of the inner payload in the NVO3 packet will be minimized. inner payload in the NVO3 packet will be minimized. Furthermore, if
Furthermore, if the target NVE cares about a few extensions and can the target NVE cares about a few extensions and can express in the
express in the control plane the desired order of those extensions in control plane the desired order of those extensions in the NVO3
the NVO3 packets, then it can provide useful functionality with packets, then it can provide useful functionality with simplified
minimal hardware requirements. hardware requirements for the target NVE.
Note that transit devices that are not aware of the NVO3 extensions Note that transit devices that are not aware of the NVO3 extensions
somewhat benefit from such an approach, since the inner payload is somewhat benefit from such an approach, since the inner payload is
less deep in the packet if no extraneous extensions are included in less deep in the packet if no extraneous extension headers are
the packet. In general, a transit device is not likely to included in the packet. In general, a transit device is not likely
participate in the NVO3 control plane. (However, configuration to participate in the NVO3 control plane. (However, configuration
mechanisms can take into account limitations of the transit devices mechanisms can take into account limitations of the transit devices
used in particular deployments.) used in particular deployments.)
Note that in this approach different NVEs could desire different Note that with this approach different NVEs could desire different
extensions or sets of extensions, which means that the source NVE extensions or sets of extensions, which means that the source NVE
needs to be able to place different sets of extensions in different needs to be able to place different sets of extensions in different
NVO3 packets, and perhaps in different order. It also assumes that NVO3 packets, and perhaps in different order. It also assumes that
underlay multicast or replication servers are not used together with underlay multicast or replication servers are not used together with
NVO3 extensions. NVO3 extension headers.
Internet-Draft NVO3 Encapsulation Considerations
There is a need to consider mandatory extensions versus optional There is a need to consider mandatory extensions versus optional
extensions. Mandatory extensions require the receiver to drop the extensions. Mandatory extensions require the receiver to drop the
packet if the extension is unknown. A control plane mechanism can packet if the extension is unknown. A control plane mechanism can
prevent the need for dropping unknown extensions, since they would prevent the need for dropping unknown extensions, since they would
not be included to targets that do not support them. not be included to target NVEs that do not support them.
The control planes defined today need to add the ability to describe The control planes defined today need to add the ability to describe
the different encapsulations. Thus, perhaps EVPN and any other the different encapsulations. Thus, perhaps EVPN [RFC8365] and any
control plane protocol that the IETF defines should have a way to other control plane protocol that the IETF defines should have a way
enumerate the supported NVO3 extensions and their order. to indicate the supported NVO3 extensions and their order, for each
of the encapsulations supported.
The WG should consider developing a separate draft on guidance for The WG should consider developing a separate draft on guidance for
option processing and control plane participation. This should option processing and control plane participation. This should
provide examples/guidance on range of usage models and deployments provide examples/guidance on range of usage models and deployments
scenarios for specific options and ordering that are relevant for scenarios for specific options and ordering that are relevant for
that specific deployment. This includes end points and middle boxes that specific deployment. This includes end points and middle boxes
using the options. So, having the control plane negotiate the using the options. So, having the control plane negotiate the
constraints is the most appropriate and flexible way to address these constraints is the most appropriate and flexible way to address these
requirements. requirements.
6.8. Split NVE 6.8. Split NVE
If the working group sees a need for having the hosts send and If the working group sees a need for having the hosts send and
receive options in a split NVE case, this is possible using any of receive options in a split NVE case [RFC8394], this is possible using
the existing extensible encapsulations (Geneve, GUE, GPE+NSH) by any of the existing extensible encapsulations (Geneve, GUE, GPE+NSH)
defining a way to carry those over other transports. NSH can already by defining a way to carry those over other transports. NSH can
be used over different transports. already be used over different transports.
If we need to do this with other encapsulations it can be done by If we need to do this with other encapsulations it can be done by
defining an Ether type for other encapsulations so that it can be defining an Ethertype for other encapsulations so that it can be
carried over Ethernet and 802.1Q. carried over Ethernet and 802.1Q.
If we need to carry other encapsulations over MPLS, it would require If we need to carry other encapsulations over MPLS, it would require
an EVPN control plane to signal that other encapsulation header + an EVPN control plane to signal that other encapsulation header +
options will be present in front of the L2 packet. The VNI can be options will be present in front of the L2 packet. The VNI can be
ignored in the header, and the MPLS label will be the one used to ignored in the header, and the MPLS label will be the one used to
identify the EVPN L2 instance. identify the EVPN L2 instance.
6.9. Larger VNI Considerations 6.9. Larger VNI Considerations
We discussed whether we should make the VNI 32-bits or larger. The We discussed whether we should make the VNI 32-bits or larger. The
benefit of a 24-bit VNI would be to avoid unnecessary changes with benefit of a 24-bit VNI would be to avoid unnecessary changes with
existing proposals and implementations that are almost all, if not existing proposals and implementations that are almost all, if not
all, using 24-bit VNI. If we need a larger VNI, an extension can be all, using 24-bit VNI. If we need a larger VNI, an extension can be
used to support that. used to support that.
Internet-Draft NVO3 Encapsulation Considerations
7. Design Team Recommendations 7. Design Team Recommendations
We concluded that Geneve is most suitable as a starting point for a We concluded that Geneve is most suitable as a starting point for a
proposed standard for network virtualization, for the following proposed standard for network virtualization, for the following
reasons: reasons:
1. We studied whether VNI should be in the base header or in 1. We studied whether VNI should be in the base header or in an
extensions and whether it should be 24-bit or 32-bit. The design extension header and whether it should be a 24-bit or 32-bit field.
team agreed that VNI is critical information for network The design team agreed that VNI is critical information for network
virtualization and MUST be present in all packets. The design team virtualization and MUST be present in all packets. The design team
also agreed that a 24-bit VNI matches the existing widely used also agreed that a 24-bit VNI matches the existing widely used
encapsulation formats, i.e., VxLAN and NVGRE, and hence is more encapsulation formats, i.e., VXLAN [RFC7348] and NVGRE [RFC7637], and
suitable to use going forward. hence is more suitable to use going forward.
2. The Geneve header has the total options length which allows 2. The Geneve header has the total options length which allows
skipping over the options for NIC offload operations and will allow skipping over the options for NIC offload operations and will allow
transit devices to view flow information in the inner payload. transit devices to view flow information in the inner payload.
3. We considered the option of using NSH [RFC8300] with VxLAN-GPE 3. We considered the option of using NSH [RFC8300] with VXLAN-GPE
but given that NSH is targeted at service chaining and contains but given that NSH is targeted at service chaining and contains
service chaining information, it is less suitable for the network service chaining information, it is less suitable for the network
virtualization use case. The other downside for VxLAN-GPE was lack virtualization use case. The other downside for VXLAN-GPE was lack
of header length in VxLAN-GPE which makes skipping over the headers of a header length in VXLAN-GPE which makes skipping over the headers
to process inner payload more difficult. Total Option Length is to process inner payload more difficult. Total Option Length is
present in Geneve. It is not possible to skip any options in the present in Geneve. It is not possible to skip any options in the
middle with VxLAN-GPE. In principle a split between a base header middle with VXLAN-GPE. In principle a split between a base header
and a header with options is interesting (whether that options header and a header with options is interesting (whether that options header
is NSH or some new header without ties to a service path). We is NSH or some new header without ties to a service path). We
explored whether it would make sense to either use NSH for this, or explored whether it would make sense to either use NSH for this, or
define a new NVO3 options header. However, we observed that this define a new NVO3 options header. However, we observed that this
makes it slightly harder to find the inner payload since the length makes it slightly harder to find the inner payload since the length
field is not in the NVO3 header itself. Thus, one more field would field is not in the NVO3 header itself. Thus, one more field would
have to be extracted to compute the start of the inner payload. have to be extracted to compute the start of the inner payload.
Also, if the experience with IPv6 extension headers is a guide, there Also, if the experience with IPv6 extension headers is a guide, there
would be a risk that key pieces of hardware might not implement the would be a risk that key pieces of hardware might not implement the
options header, resulting in future calls to deprecate its use. options header, resulting in future calls to deprecate its use.
Making the options part of the base NVO3 header has less of those Making the options part of the base NVO3 header has less of those
issues. Even though the implementation of any particular option can issues. Even though the implementation of any particular option can
not be predicted ahead of time, the option mechanism and ability to not be predicted ahead of time, the option mechanism and ability to
skip the options is likely to be broadly implemented. skip the options is likely to be broadly implemented.
4. We compared the TLV vs Bit-fields style extension and it was 4. We compared the TLV vs bit fields style extension and it was
deemed that parsing both TLV and bit-fields is expensive and while deemed that parsing both TLV and bit fields is expensive and while
bit-fields may be simpler to parse, it is also more restrictive and bit fields may be simpler to parse, it is also more restrictive and
requires guessing which extensions will be widely implemented so they requires guessing which extensions will be widely implemented so they
can get early bit assignments, given that half the bits are already can get early bit assignments, given that half the bits are already
assigned in GUE, a widely deployed extension may appear in a flag assigned in GUE, a widely deployed extension may appear in a flag
extension, and this will require extra processing, to dig the flag extension, and this will require extra processing, to dig the flag
from the flag extension and then look for the extension itself. Also from the flag extension and then look for the extension itself. Also
Bit-fields are not flexible enough to address the requirements from bit fields are not flexible enough to address the requirements from
Internet-Draft NVO3 Encapsulation Considerations
OAM, Telemetry, and security extensions, for variable length option OAM, Telemetry, and security extensions, for variable length option
and different subtypes of the same option. While TLV are more and different subtypes of the same option. While TLV are more
flexible, a control plane can restrict the number of option TLVs as flexible, a control plane can restrict the number of option TLVs as
well the order and size of the TLVs to make it simpler for a well the order and size of the TLVs to make it simpler for a
dataplane implementation to handle. dataplane implementation to handle.
5. We briefly discussed the multi-vendor NVE case, and the need to 5. We briefly discussed the multi-vendor NVE case, and the need to
allow vendors to put their own extensions in the NVE header. This is allow vendors to put their own extensions in the NVE header. This is
possible with TLVs. possible with TLVs.
6. We also agreed that the C bit in Geneve is helpful to allow a 6. We also agreed that the C bit in Geneve is helpful to allow a
receiver NVE to easily decide whether to process options or not, for receiver NVE to easily decide whether to process options or not, for
example a UUID based packet trace, and how an optional extension such example a UUID based packet trace, and how an optional extension such
as that can be ignored by a receiver NVE and thus make it easy for as that can be ignored by a receiver NVE and thus make it easy for
NVE to skip over the options. Thus, the C-bit remains as defined in NVE to skip over the options. Thus, the C bit remains as defined in
Geneve. Geneve.
7. There are already some extensions that are being discussed (see 7. There are already some extensions that are being discussed (see
section 6.2) of varying sizes. By using Geneve option it is possible section 6.2) of varying sizes. By using Geneve options it is possible
to get in band parameters like switch id, ingress port, egress port, to get in band parameters like switch id, ingress port, egress port,
internal delay, and queue in telemetry defined extension TLV from internal delay, and queue in telemetry defined extension TLV from
switches. It is also possible to add Security extension TLVs like switches. It is also possible to add security extension TLVs like
HMAC and DTLS/IPSEC to authenticate the Geneve packet header and HMAC and DTLS/IPSEC to authenticate the Geneve packet header and
secure the Geneve packet payload by software or hardware tunnel secure the Geneve packet payload by software or hardware tunnel
endpoints. A Group Based Policy extension TLV can be carried as endpoints. A Group Based Policy extension TLV can be carried as
well. well.
8. There are implemented Geneve options today in production. There 8. There are already implementations of Geneve options deployed in
are as well new hardware supporting Geneve TLV parsing. In addition, production networks as of this writing. There are as well new
an In-band Telemetry (INT) specification is being developed by P4.org hardware supporting Geneve TLV parsing. In addition, an In-band
that illustrates the option of INT meta data carried over Geneve. Telemetry [INT] specification is being developed by P4.org that
OVN/OVS have also defined some option TLV(s) for Geneve. illustrates the option of INT meta data carried over Geneve. OVN/OVS
have also defined some option TLV(s) for Geneve.
9. The DT has addressed the usage models while considering the 9. The DT has addressed the usage models while considering the
requirements and implementations in general that includes software requirements and implementations in general that includes software
and hardware. and hardware.
There seems to be interest to standardize some well-known secure There seems to be interest to standardize some well-known secure
option TLVs to secure the header and payload to guarantee option TLVs to secure the header and payload to guarantee
encapsulation header integrity and tenant data privacy. The design encapsulation header integrity and tenant data privacy. The
team recommends that the working group consider standardizing such design team recommends that the working group consider
option(s). standardizing such option(s).
We recommend the following enhancements to Geneve to make it more
suitable to hardware and yet provide the flexibility for software:
We would propose a text such as, while TLV are more flexible, a
control plane can restrict the number of option TLVs as well the
order and size of the TLVs to make it simpler for a data plane
implementation in software or hardware to handle. For example, there
Internet-Draft NVO3 Encapsulation Considerations
may be some critical information such as a secure hash that must be We recommend the following enhancements to Geneve to make it more
processed in a certain order at lowest latency. suitable to hardware and yet provide the flexibility for software:
A control plane can negotiate a subset of option TLVs and certain TLV We would propose a text such as, while TLV are more flexible, a
ordering, as well as limiting the total number of option TLVs present control plane can restrict the number of option TLVs as well the
in the packet, for example, to allow for hardware capable of order and size of the TLVs to make it simpler for a data plane
processing fewer options. Hence, the control plane needs to have the implementation in software or hardware to handle. For example,
ability to describe the supported TLVs subset and their order. there may be some critical information such as a secure hash that
must be processed in a certain order at lowest latency.
The Geneve draft could specify that the subset and order of option A control plane can negotiate a subset of option TLVs and certain
TLVs should be configurable for each remote NVE in the absence of a TLV ordering, as well as limiting the total number of option TLVs
protocol control plane. present in the packet, for example, to allow for hardware capable
of processing fewer options. Hence, the control plane needs to
have the ability to describe the supported TLVs subset and their
order.
We recommend that Geneve follow fragmentation recommendations in The Geneve draft should specify that the subset and order of
overlay services like PWE3 and the L2/L3 VPN recommendations to option TLVs should be configurable for each remote NVE in the
guarantee larger MTU for the tunnel overhead ([RFC3985] Section 5.3). absence of a protocol control plane.
We request that Geneve provide a recommendation for critical bit We recommend that Geneve follow fragmentation recommendations in
processing - text could specify how critical bits can be used with overlay services like PWE3 and the L2/L3 VPN recommendations to
control plane specifying the critical options. guarantee larger MTU for the tunnel overhead ([RFC3985] Section
5.3).
Given that there is a telemetry option use case for a length of 256 We request that Geneve provide a recommendation for critical bit
bytes, we recommend that Geneve increase the Single TLV option length processing - text could specify how critical bits can be used with
to 256. control plane specifying the critical options.
We request that Geneve address Requirements for OAM considerations Given that there is a telemetry option use case for a length of
for alternate marking and for performance measurements that need 2 256 bytes, we recommend that Geneve increase the Single TLV option
bits in the header and clarify the need for the current OAM bit in length to 256.
the Geneve Header.
We recommend that the WG work on security options for Geneve. We request that Geneve address Requirements for OAM considerations
for alternate marking and for performance measurements that need a
2 bit field in the header and clarify the need for the current OAM
bit in the Geneve Header.
Internet-Draft NVO3 Encapsulation Considerations We recommend that the WG work on security options for Geneve.
8. Acknowledgements 8. Acknowledgements
The authors would like to thank Tom Herbert for providing the The authors would like to thank Tom Herbert for providing the
motivation for the Security/Integrity extension, and for his valuable motivation for the Security/Integrity extension, and for his valuable
comments, and would like to thank T. Sridhar for his valuable comments, T. Sridhar for his valuable comments and feedback, and
comments and feedback. Anoop Ghanwani for his extensive comments.
9. Security Considerations 9. Security Considerations
This document does not introduce any additional security constraints. This document does not introduce any additional security constraints.
10. IANA Considerations 10. IANA Considerations
This document has no actions for IANA. This document requires no IANA actions.
Internet-Draft NVO3 Encapsulation Considerations
11. References 11. References
11.1 Normative References 11.1 Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, DOI Requirement Levels", BCP 14, RFC 2119, DOI
10.17487/RFC2119, March 1997, <https://www.rfc- 10.17487/RFC2119, March 1997, <https://www.rfc-
editor.org/info/rfc2119>. editor.org/info/rfc2119>.
skipping to change at page 17, line 31 skipping to change at page 18, line 29
[I-D.herbert-gue-extensions] Herbert, T., Yong, L., and F. Templin, [I-D.herbert-gue-extensions] Herbert, T., Yong, L., and F. Templin,
"Extensions for Generic UDP Encapsulation", "Extensions for Generic UDP Encapsulation",
draft-herbert-gue-extensions-01 (work in progress), October draft-herbert-gue-extensions-01 (work in progress), October
2016. 2016.
[I-D.ietf-intarea-gue] Herbert, T., Yong, L., and O. Zia, "Generic [I-D.ietf-intarea-gue] Herbert, T., Yong, L., and O. Zia, "Generic
UDP Encapsulation", draft-ietf-intarea-gue (work in UDP Encapsulation", draft-ietf-intarea-gue (work in
progress), October 2019. progress), October 2019.
[I-D.ietf-ippm-ioam-data] F. Brockers, S. Bhandari, T. Mizrahi, "Data
Fields for In-situ OAM", draft-ietf-ippm-ioam-data (work in
progress), June 2021.
[I-D.ietf-nvo3-vxlan-gpe] Maino, F., Kreeger, L., and U. Elzur, [I-D.ietf-nvo3-vxlan-gpe] Maino, F., Kreeger, L., and U. Elzur,
"Generic Protocol Extension for VXLAN", "Generic Protocol Extension for VXLAN",
draft-ietf-nvo3-vxlan-gpe (work in progress), March 2021. draft-ietf-nvo3-vxlan-gpe (work in progress), March 2021.
[I-D.smith-vxlan-group-policy] Smith, M. and L. Kreeger, "VXLAN Group [I-D.smith-vxlan-group-policy] Smith, M. and L. Kreeger, "VXLAN Group
Policy Option", draft-smith-vxlan-group-policy-05 (work in Policy Option", draft-smith-vxlan-group-policy-05 (work in
progress), October 2018. progress), October 2018.
[INT] P4.org, "In-band Network Telemetry (INT) Dataplane
Specification", November 2020,
https://p4.org/p4-spec/docs/INT_v2_1.pdf
[RFC2418] Bradner, S., "IETF Working Group Guidelines and [RFC2418] Bradner, S., "IETF Working Group Guidelines and
Procedures", BCP 25, RFC 2418, DOI 10.17487/RFC2418, Procedures", BCP 25, RFC 2418, DOI 10.17487/RFC2418,
September 1998, <https://www.rfc-editor.org/info/rfc2418>. September 1998, <https://www.rfc-editor.org/info/rfc2418>.
[RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation
Edge-to-Edge (PWE3) Architecture", RFC 3985, DOI Edge-to-Edge (PWE3) Architecture", RFC 3985, DOI
10.17487/RFC3985, March 2005, <https://www.rfc- 10.17487/RFC3985, March 2005,
editor.org/info/rfc3985>. <https://www.rfc-editor.org/info/rfc3985>.
[RFC7348] Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger,
L., Sridhar, T., Bursell, M., and C. Wright, "Virtual
eXtensible Local Area Network (VXLAN): A Framework for
Overlaying Virtualized Layer 2 Networks over Layer 3
Networks", RFC 7348, DOI 10.17487/RFC7348, August 2014,
<https://www.rfc- editor.org/info/rfc7348>.
[RFC7637] Garg, P., Ed., and Y. Wang, Ed., "NVGRE: Network
Virtualization Using Generic Routing Encapsulation", RFC
7637, DOI 10.17487/RFC7637, September 2015,
<https://www.rfc-editor.org/info/rfc7637>.
[RFC8300] Quinn, P., Ed., Elzur, U., Ed., and C. Pignataro, Ed., [RFC8300] Quinn, P., Ed., Elzur, U., Ed., and C. Pignataro, Ed.,
"Network Service Header (NSH)", RFC 8300, DOI "Network Service Header (NSH)", RFC 8300, DOI
10.17487/RFC8300, January 2018, <https://www.rfc- 10.17487/RFC8300, January 2018,
editor.org/info/rfc8300>. <https://www.rfc-editor.org/info/rfc8300>.
Internet-Draft NVO3 Encapsulation Considerations [RFC8365] Sajassi, A., Ed., Drake, J., Ed., Bitar, N., Shekhar, R.,
Uttaro, J., and W. Henderickx, "A Network Virtualization
Overlay Solution Using Ethernet VPN (EVPN)", RFC 8365, DOI
10.17487/RFC8365, March 2018,
<https://www.rfc-editor.org/info/rfc8365>.
[RFC8394] Li, Y., Eastlake 3rd, D., Kreeger, L., Narten, T., and D.
Black, "Split Network Virtualization Edge (Split-NVE)
Control-Plane Requirements", RFC 8394, DOI
10.17487/RFC8394, May 2018,
<https://www.rfc-editor.org/info/rfc8394>.
[RFC8926] Gross, J., Ed., Ganga, I., Ed., and T. Sridhar, Ed., [RFC8926] Gross, J., Ed., Ganga, I., Ed., and T. Sridhar, Ed.,
"Geneve: Generic Network Virtualization Encapsulation", RFC "Geneve: Generic Network Virtualization Encapsulation", RFC
8926, DOI 10.17487/RFC8926, November 2020, 8926, DOI 10.17487/RFC8926, November 2020,
<https://www.rfc-editor.org/info/rfc8926>. <https://www.rfc-editor.org/info/rfc8926>.
Internet-Draft NVO3 Encapsulation Considerations
Appendix A: Encapsulations Comparison Appendix A: Encapsulations Comparison
A.1. Overview A.1. Overview
This section presents a comparison of the three NVO3 encapsulation This section presents a comparison of the three NVO3 encapsulation
proposals, Geneve, GUE, and VXLAN-GPE. The three encapsulations use proposals, Geneve, GUE, and VXLAN-GPE. The three encapsulations use
an outer UDP/IP transport. Geneve and VXLAN-GPE use an 8-octet an outer UDP/IP transport. Geneve and VXLAN-GPE use an 8-octet
header, while GUE uses a 4-octet header. In addition to the base header, while GUE uses a 4-octet header. In addition to the base
header, optional extensions may be included in the encapsulation, as header, optional extensions may be included in the encapsulation, as
discussed in Section A.2 below. discussed in Section A.2 below.
skipping to change at page 20, line 5 skipping to change at page 21, line 5
possible extension fields (proposed in [I-D.herbert-gue-extensions]), possible extension fields (proposed in [I-D.herbert-gue-extensions]),
and a set of flags in the GUE header that indicate for each extension and a set of flags in the GUE header that indicate for each extension
type whether it is present or not. type whether it is present or not.
TLV-based extensions, as defined in Geneve, provide the flexibility TLV-based extensions, as defined in Geneve, provide the flexibility
for a large number of possible extension types. Similar behavior can for a large number of possible extension types. Similar behavior can
be supported in NSH-over-VXLAN-GPE when using MD Type 2. The flag- be supported in NSH-over-VXLAN-GPE when using MD Type 2. The flag-
based approach taken in GUE strives to simplify implementations by based approach taken in GUE strives to simplify implementations by
defining a small number of possible extensions used in a fixed order. defining a small number of possible extensions used in a fixed order.
Internet-Draft NVO3 Encapsulation Considerations
The Geneve and GUE headers both include a length field, defining the The Geneve and GUE headers both include a length field, defining the
total length of the encapsulation, including the optional extensions. total length of the encapsulation, including the optional extensions.
The length field simplifies the parsing of transit devices that skip The length field simplifies the parsing of transit devices that skip
the encapsulation header without parsing its extensions. the encapsulation header without parsing its extensions.
A.2.3. Critical Extensions A.2.3. Critical Extensions
The Geneve encapsulation header includes the 'C' field, which The Geneve encapsulation header includes the 'C' field, which
indicates whether the current Geneve header includes critical indicates whether the current Geneve header includes critical
options, that is to say, options which must be parsed by the tunnel options, that is to say, options which must be parsed by the target
endpoint. If the endpoint is not able to process a critical option, NVE. If the endpoint is not able to process a critical option, the
the packet is discarded. packet is discarded.
A.2.4. Maximal Header Length A.2.4. Maximal Header Length
The maximal header length in Geneve, including options, is 260 The maximal header length in Geneve, including options, is 260
octets. GUE defines the maximal header to be 128 octets. VXLAN-GPE octets. GUE defines the maximal header to be 128 octets. VXLAN-GPE
uses a fixed-length header of 8 octets, unless NSH-over-VXLAN-GPE is uses a fixed-length header of 8 octets, unless NSH-over-VXLAN-GPE is
used, yielding an encapsulation header of up to 264 octets. used, yielding an encapsulation header of up to 264 octets.
A.3. Encapsulation Header A.3. Encapsulation Header
skipping to change at page 21, line 4 skipping to change at page 22, line 4
The VXLAN-GPE header includes the 'I' bit, indicating that the VNI The VXLAN-GPE header includes the 'I' bit, indicating that the VNI
field is valid in the current header. A similar indicator is defined field is valid in the current header. A similar indicator is defined
as a flag in the GUE header [I-D.herbert-gue-extensions]. as a flag in the GUE header [I-D.herbert-gue-extensions].
A.3.2. Next Protocol A.3.2. Next Protocol
The three encapsulation headers include a field that specifies the The three encapsulation headers include a field that specifies the
type of the next protocol header, which resides after the NVO3 type of the next protocol header, which resides after the NVO3
encapsulation header. The Geneve header includes a 16-bit field that encapsulation header. The Geneve header includes a 16-bit field that
uses the IEEE Ethertype convention. GUE uses an 8-bit field, which uses the IEEE Ethertype convention. GUE uses an 8-bit field, which
Internet-Draft NVO3 Encapsulation Considerations
uses the IANA Internet protocol numbering. The VXLAN-GPE header uses the IANA Internet protocol numbering. The VXLAN-GPE header
incorporates an 8-bit Next Protocol field, using a VXLAN-GPE-specific incorporates an 8-bit Next Protocol field, using a VXLAN-GPE-specific
registry, defined in [I-D.ietf-nvo3-vxlan-gpe]. registry, defined in [I-D.ietf-nvo3-vxlan-gpe].
The VXLAN-GPE header also includes the 'P' bit, which explicitly The VXLAN-GPE header also includes the 'P' bit, which explicitly
indicates whether the Next Protocol field is present in the current indicates whether the Next Protocol field is present in the current
header. header.
A.3.3. Other Header Fields A.3.3. Other Header Fields
skipping to change at page 22, line 4 skipping to change at page 23, line 6
protocols that are not OAM protocols. However, the control packet protocols that are not OAM protocols. However, the control packet
examples discussed in [I-D.ietf-intarea-gue] are OAM-related. examples discussed in [I-D.ietf-intarea-gue] are OAM-related.
Each of the three NVO3 encapsulation headers includes a 2-bit Version Each of the three NVO3 encapsulation headers includes a 2-bit Version
field, which is currently defined to be zero. field, which is currently defined to be zero.
The Geneve and VXLAN-GPE headers include reserved fields; 14 bits in The Geneve and VXLAN-GPE headers include reserved fields; 14 bits in
the Geneve header, and 27 bits in the VXLAN-GPE header are reserved. the Geneve header, and 27 bits in the VXLAN-GPE header are reserved.
A.4. Comparison Summary A.4. Comparison Summary
Internet-Draft NVO3 Encapsulation Considerations
The following table summarizes the comparison between the three NVO3 The following table summarizes the comparison between the three NVO3
encapsulations: encapsulations:
+----------------+----------------+----------------+----------------+ +----------------+----------------+----------------+----------------+
| | Geneve | GUE | VXLAN-GPE | | | Geneve | GUE | VXLAN-GPE |
+----------------+----------------+----------------+----------------+ +----------------+----------------+----------------+----------------+
| Outer transport| UDP/IP | UDP/IP | UDP/IP | | Outer transport| UDP/IP | UDP/IP | UDP/IP |
+----------------+----------------+----------------+----------------+ +----------------+----------------+----------------+----------------+
| Base header | 8 octets | 4 octets | 8 octets | | Base header | 8 octets | 4 octets | 8 octets |
| length | | | (16 octets | | length | | | (16 octets |
skipping to change at page 22, line 54 skipping to change at page 24, line 4
| Next protocol | - | - | + | | Next protocol | - | - | + |
| indicator | | | | | indicator | | | |
+----------------+----------------+----------------+----------------+ +----------------+----------------+----------------+----------------+
| OAM / control | OAM bit | Control bit | OAM bit | | OAM / control | OAM bit | Control bit | OAM bit |
| field | | | | | field | | | |
+----------------+----------------+----------------+----------------+ +----------------+----------------+----------------+----------------+
| Version field | 2 bits | 2 bits | 2 bits | | Version field | 2 bits | 2 bits | 2 bits |
+----------------+----------------+----------------+----------------+ +----------------+----------------+----------------+----------------+
| Reserved bits | 14 bits | - | 27 bits | | Reserved bits | 14 bits | - | 27 bits |
+----------------+----------------+----------------+----------------+ +----------------+----------------+----------------+----------------+
Figure 1: NVO3 Encapsulations Comparison Figure 1: NVO3 Encapsulations Comparison
Internet-Draft NVO3 Encapsulation Considerations
Contributors Contributors
The following co-authors have contributed to this document: The following co-authors have contributed to this document:
Ilango Ganga Intel Email: ilango.s.ganga@intel.com Ilango Ganga Intel Email: ilango.s.ganga@intel.com
Pankaj Garg Microsoft Email: pankajg@microsoft.com Pankaj Garg Microsoft Email: pankajg@microsoft.com
Rajeev Manur Broadcom Email: rajeev.manur@broadcom.com Rajeev Manur Broadcom Email: rajeev.manur@broadcom.com
Tal Mizrahi Marvell Email: talmi@marvell.com Tal Mizrahi Marvell Email: talmi@marvell.com
David Mozes Email: mosesster@gmail.com David Mozes Email: mosesster@gmail.com
Erik Nordmark Email: nordmark@sonic.net Erik Nordmark ZEDEDA Email: nordmark@sonic.net
Michael Smith Cisco Email: michsmit@cisco.com Michael Smith Cisco Email: michsmit@cisco.com
Sam Aldrin Google Email: aldrin.ietf@gmail.com Sam Aldrin Google Email: aldrin.ietf@gmail.com
Ignas Bagdonas Equinix Email: ibagdona.ietf@gmail.com Ignas Bagdonas Equinix Email: ibagdona.ietf@gmail.com
Internet-Draft NVO3 Encapsulation Considerations
Authors' Addresses Authors' Addresses
Sami Boutros (editor) Sami Boutros (editor)
Ciena Ciena
USA USA
Email: sboutros@ciena.com Email: sboutros@ciena.com
Donald E. Eastlake, 3rd (editor) Donald E. Eastlake, 3rd (editor)
Futurewei Technologies Futurewei Technologies
 End of changes. 90 change blocks. 
234 lines changed or deleted 226 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/