idnits 2.17.1 draft-ietf-tsvwg-gre-in-udp-encap-12.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 29, 2016) is 2857 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC7637' is defined on line 1051, but no explicit reference was found in the text ** Obsolete normative reference: RFC 6347 (Obsoleted by RFC 9147) -- Obsolete informational reference (is this intentional?): RFC 793 (Obsoleted by RFC 9293) -- Obsolete informational reference (is this intentional?): RFC 2460 (Obsoleted by RFC 8200) == Outdated reference: A later version (-15) exists of draft-ietf-tsvwg-circuit-breaker-13 Summary: 1 error (**), 0 flaws (~~), 3 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group Lucy Yong(Ed.) 2 Internet-Draft Huawei Technologies 3 Intended status: Standard Track E. Crabbe 4 Oracle 5 X. Xu 6 Huawei Technologies 7 T. Herbert 8 Facebook 10 Expires: December 2016 June 29, 2016 12 GRE-in-UDP Encapsulation 13 draft-ietf-tsvwg-gre-in-udp-encap-12 15 Abstract 17 This document specifies a method of encapsulating network protocol 18 packet within GRE and UDP headers. This GRE-in-UDP encapsulation 19 allows the UDP source port field to be used as an entropy field. 20 This may be used for load balancing of GRE traffic in transit 21 networks using existing ECMP mechanisms. This document also 22 specifies GRE-in-UDP tunnel requirements for two applicability 23 scenarios: (1) general Internet; (2) a traffic-managed controlled 24 environment. The controlled environment has less restrictive 25 requirements than the general Internet. 27 Status of This Document 29 This Internet-Draft is submitted in full conformance with the 30 provisions of BCP 78 and BCP 79. 32 Internet-Drafts are working documents of the Internet Engineering 33 Task Force (IETF). Note that other groups may also distribute 34 working documents as Internet-Drafts. The list of current Internet- 35 Drafts is at http://datatracker.ietf.org/drafts/current/. 37 Internet-Drafts are draft documents valid for a maximum of six 38 months and may be updated, replaced, or obsoleted by other documents 39 at any time. It is inappropriate to use Internet-Drafts as reference 40 material or to cite them other than as "work in progress." 42 This Internet-Draft will expire on December 29,2016. 44 Copyright Notice 46 Copyright (c) 2016 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents 51 (http://trustee.ietf.org/license-info) in effect on the date of 52 publication of this document. Please review these documents 53 carefully, as they describe your rights and restrictions with 54 respect to this document. Code Components extracted from this 55 document must include Simplified BSD License text as described in 56 Section 4.e of the Trust Legal Provisions and are provided without 57 warranty as described in the Simplified BSD License. 59 Table of Contents 61 1. Introduction...................................................3 62 1.1. Terminology...............................................3 63 1.2. Requirements Language.....................................4 64 2. Applicability Statement........................................4 65 2.1. GRE-in-UDP Tunnel Requirements............................5 66 2.1.1. Requirements for Default GRE-in-UDP Tunnel...........5 67 2.1.2. Requirements for TMCE GRE-in-UDP Tunnel..............6 68 3. GRE-in-UDP Encapsulation.......................................6 69 3.1. IP Header.................................................9 70 3.2. UDP Header................................................9 71 3.2.1. Source Port..........................................9 72 3.2.2. Destination Port....................................10 73 3.2.3. Checksum............................................10 74 3.2.4. Length..............................................10 75 3.3. GRE Header...............................................10 76 4. Encapsulation Process Procedures..............................11 77 4.1. MTU and Fragmentation....................................11 78 4.2. Differentiated Services and ECN Marking..................12 79 5. Use of DTLS...................................................12 80 6. UDP Checksum Handling.........................................13 81 6.1. UDP Checksum with IPv4...................................13 82 6.2. UDP Checksum with IPv6...................................13 83 7. Middlebox Considerations......................................16 84 7.1. Middlebox Considerations for Zero Checksums..............17 85 8. Congestion Considerations.....................................17 86 9. Backward Compatibility........................................18 87 10. IANA Considerations..........................................19 88 11. Security Considerations......................................20 89 12. Acknowledgements.............................................20 90 13. Contributors.................................................21 91 14. References...................................................22 92 14.1. Normative References....................................22 93 14.2. Informative References..................................23 94 15. Authors' Addresses...........................................24 96 1. Introduction 98 This document specifies a generic GRE-in-UDP encapsulation for 99 tunneling network protocol packets across an IP network. This 100 encapsulation uses Generic Routing Encapsulation (GRE) 101 [RFC2784][RFC7676] and User Datagram Protocol(UDP) [RFC768] headers. 102 The GRE header provides payload protocol type as an EtherType in the 103 protocol type field, and the source port field in the UDP header may 104 be used to provide additional entropy that may be used for load 105 balancing GRE traffic in transit networks using existing Equal-Cost 106 Multi-Path (ECMP) mechanisms. Existing ECMP mechanisms, when the IP 107 payload is a UDP or Transmission Control Protocol (TCP)[RFC793] 108 packet, frequently use of a hash of the five-tuple of source IP 109 address, destination IP address, UDP/TCP source port, UDP/TCP 110 destination port, and protocol/next-header. A GRE-in-UDP tunnel 111 offers the additional possibility of using GRE across networks that 112 might otherwise disallow it; for instance GRE-in-UDP may be used to 113 bridge two islands where GRE is not supported natively across the 114 middleboxes. 116 This encapsulation method requires no changes to the transit IP 117 network. Hash functions in most existing IP routers may utilize and 118 benefit from the use of a GRE-in-UDP tunnel without needing any 119 change or upgrade to their ECMP implementation. The encapsulation 120 mechanism is applicable to a variety of IP networks including Data 121 Center and wide area networks. 123 GRE-in-UDP encapsulation may be used to encapsulate already tunneled 124 traffic, i.e. tunnel-in-tunnel. In this case, GRE-in-UDP tunnel do 125 not differentiate such end hosts from other end hosts, i.e., 126 applying the same treatment for traffic from hosts and tunnel 127 endpoints. 129 This document specifies GRE-in-UDP tunnel requirements for two 130 applicability scenarios: (1) general Internet; (2) a traffic-managed 131 controlled environment. The controlled environment has less 132 restrictive requirements than the general Internet. 134 The document also specifies Datagram Transport Layer Security (DTLS) 135 version of GRE-in-UDP tunnel to be used where/when security is a 136 concern. 138 1.1. Terminology 140 The terms defined in [RFC768] and [RFC2784] are used in this 141 document. Following are additional terms used in this draft. 143 ECMP: Equal-Cost Multi-Path. 145 Flow Entropy: The information to be derived from traffic or 146 applications and to be used by network devices in ECMP process 147 [RFC6438]. 149 Default GRE-in-UDP Tunnel: A GRE-in-UDP tunnel that can apply to the 150 general Internet. 152 TMCE: A Traffic-managed controlled environment, i.e. an IP network 153 that is traffic-engineered and/or otherwise managed (e.g., via use 154 of traffic rate limiters) to avoid congestion, as defined in Section 155 2. 157 TMCE GRE-in-UDP Tunnel: A GRE-in-UDP tunnel that can only apply to a 158 traffic-managed controlled environment that is defined in Section 2. 160 1.2. Requirements Language 162 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 163 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 164 document are to be interpreted as described in [RFC2119]. 166 2. Applicability Statement 168 GRE-in-UDP encapsulation as specified herein applies to IPv4 and 169 IPv6 networks. When using GRE-in-UDP encapsulation, packets so 170 encapsulated are treated as UDP datagrams by an IP network. As such, 171 a GRE-in-UDP tunnel needs to meet the UDP requirements specified in 172 [RFC5405bis], which imposes requirements on GRE-in-UDP tunnel usage. 173 These requirements depend on both the delivery network and the 174 nature of the encapsulated traffic. For example, the GRE-in-UDP 175 tunnel protocol does not provide any congestion control 176 functionality beyond that of the encapsulated traffic. Therefore, a 177 GRE-in-UDP tunnel MUST be used only with congestion controlled 178 traffic (e.g., IP unicast traffic) and/or within a network that has 179 traffic management capability to avoid congestion. 181 [RFC5405bis] considers two types of IETF UDP applications: 1) 182 General Internet and 2) A controlled environment. The controlled 183 environment means a single administrative domain or bilaterally 184 agreed connection between domains. A network forming a controlled 185 environment can be managed/operated to meet certain conditions while 186 the general Internet cannot be; thus the requirements for a tunnel 187 protocol operating under a controlled environment can be less 188 restrictive than the requirements in the general Internet. 190 For the purpose of this document, a traffic-managed controlled 191 environment is defined as an IP network that is traffic-engineered 192 and/or otherwise managed (e.g., via use of traffic rate limiters) to 193 avoid congestion. 195 This document specifies GRE-in-UDP tunnel usage in the general 196 Internet and GRE-in-UDP tunnel usage in a traffic-managed controlled 197 environment and uses "default GRE-in-UDP tunnel" and "TMCE GRE-in- 198 UDP tunnel" terms to refer to each usage. 200 2.1. GRE-in-UDP Tunnel Requirements 202 This section states out the requirements for a GRE-in-UDP tunnel. 203 Section 2.1.1 describes the requirements for a default GRE-in-UDP 204 tunnel that is suitable for the general Internet; Section 2.1.2 205 describes a set of relaxed requirements for a TMCE GRE-in-UDP tunnel 206 used in a traffic-managed controlled environment. They are 207 applicable to an IPv4 or IPv6 delivery network. 209 2.1.1. Requirements for Default GRE-in-UDP Tunnel 211 The following is a summary of the default GRE-in-UDP tunnel 212 requirements: 214 1. A UDP checksum SHOULD be used when encapsulating in IPv4. 216 2. A UDP checksum MUST be used when encapsulating in IPv6. 218 3. GRE-in-UDP tunnel MUST NOT be used for traffic that does not 219 implement congestion control. As stated in [RFC5405bis], IP-based 220 unicast traffic is generally assumed to be congestion-controlled, 221 i.e., it is assumed that the transport protocols generating IP-based 222 traffic at the sender already employ mechanisms that are sufficient 223 to address congestion on the path. GRE-in-UDP tunnels are not 224 appropriate for traffic that is not known to be congestion- 225 controlled (e.g., IP multicast traffic). 227 4. UDP source port values that are used as a source of flow entropy 228 SHOULD be chosen from the ephemeral port range (49152- 229 65535).[RFC5405bis] 231 5. The use of the UDP source port MUST be configurable so that a 232 single value can be set for all traffic within the tunnel (this 233 disables use of the UDP source port to provide flow entropy). When a 234 single value is set, a random port SHOULD be selected in order to 235 minimize the vulnerability to off-path attacks [RFC6056]. 237 6. For IPv6 delivery networks, the flow entropy SHOULD also be 238 placed in the flow label field for ECMP per [RFC6438]. 240 7. At the tunnel ingress, any fragmentation of the incoming packet 241 (e.g., because the tunnel has an MTU that is smaller than the packet 242 SHOULD be performed before encapsulation. In addition, the tunnel 243 ingress MUST apply the UDP checksum to all encapsulated fragments so 244 that the tunnel egress can validate reassembly of the fragments; it 245 MUST set the same DSCP value as in the DS field of the payload 246 packet in all fragments [RFC2474]. To avoid unwanted forwarding over 247 multiple paths, the same source UDP port value SHOULD be set in all 248 packet fragments. 250 2.1.2. Requirements for TMCE GRE-in-UDP Tunnel 252 The section contains the TMCE GRE-in-UDP tunnel requirements. It 253 lists the changed requirements, compared with a Default GRE-in-UDP 254 Tunnel, for a TMCE GRE-in-UDP Tunnel, which corresponds to the 255 requirements 1-3 listed in Section 2.1.1. 257 1. A UDP checksum SHOULD be used when encapsulating in IPv4. A 258 tunnel endpoint sending GRE-in-UDP MAY disable the UDP checksum, 259 since GRE has been designed to work without a UDP checksum [RFC2784]. 260 However, a checksum also offers protection from mis-delivery to 261 another port. 263 2. Use of UDP checksum MUST be the default when encapsulating in 264 IPv6. This default MAY be overridden via configuration of UDP zero- 265 checksum mode. All usage of UDP zero-checksum mode with IPv6 is 266 subject to the additional requirements specified in Section 6.2. 268 3. A GRE-in-UDP tunnel MAY encapsulate traffic that is not 269 congestion controlled. 271 The requirements 4-7 listed in Section 2.1.1 also apply to a TMCE 272 GRE-in-UDP Tunnel. 274 3. GRE-in-UDP Encapsulation 276 The GRE-in-UDP encapsulation format contains a UDP header [RFC768] 277 and a GRE header [RFC2890]. The format is shown as follows: 278 (presented in bit order) 279 0 1 2 3 280 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 282 IPv4 Header: 283 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 284 |Version| IHL |Type of Service| Total Length | 285 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 286 | Identification |Flags| Fragment Offset | 287 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 288 | Time to Live |Protcol=17(UDP)| Header Checksum | 289 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 290 | Source IPv4 Address | 291 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 292 | Destination IPv4 Address | 293 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 295 UDP Header: 296 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 297 | Source Port = Entropy Value | Dest. Port = TBD1/TBD2 | 298 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 299 | UDP Length | UDP Checksum | 300 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 302 GRE Header: 303 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 304 |C| |K|S| Reserved0 | Ver | Protocol Type | 305 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 306 | Checksum (optional) | Reserved1 (Optional) | 307 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 308 | Key (optional) | 309 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 310 | Sequence Number (optional) | 311 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 313 Figure 1 UDP+GRE Headers in IPv4 315 0 1 2 3 316 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 318 IPv6 Header: 319 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 320 |Version| Traffic Class | Flow Label | 321 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 322 | Payload Length | NxtHdr=17(UDP)| Hop Limit | 323 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 324 | | 325 + + 326 | | 327 + Outer Source IPv6 Address + 328 | | 329 + + 330 | | 331 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 332 | | 333 + + 334 | | 335 + Outer Destination IPv6 Address + 336 | | 337 + + 338 | | 339 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 341 UDP Header: 342 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 343 | Source Port = entropy value | Dest. Port = TBD1/TBD2 | 344 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 345 | UDP Length | UDP Checksum | 346 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 348 GRE Header: 349 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 350 |C| |K|S| Reserved0 | Ver | Protocol Type | 351 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 352 | Checksum (optional) | Reserved1 (Optional) | 353 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 354 | Key (optional) | 355 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 356 | Sequence Number (optional) | 357 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 359 Figure 2 UDP+GRE Headers in IPv6 361 The contents of the IP, UDP, and GRE headers that are relevant in 362 this encapsulation are described below. 364 3.1. IP Header 366 An encapsulator MUST encode its own IP address as the source IP 367 address and the decapsulator's IP address as the destination IP 368 address. A sufficiently large value is needed in the IPv4 TTL field 369 or IPv6 Hop Count field to allow delivery of the encapsulated packet 370 to the peer of the encapsulation. 372 3.2. UDP Header 374 3.2.1. Source Port 376 GRE-in-UDP permits the UDP source port value to be used to encode an 377 entropy value. The UDP source port contains a 16-bit entropy value 378 that is generated by the encapsulator to identify a flow for the 379 encapsulated packet. The port value SHOULD be within the ephemeral 380 port range, i.e., 49152 to 65535, where the high order two bits of 381 the port are set to one. This provides fourteen bits of entropy for 382 the inner flow identifier. In the case that an encapsulator is 383 unable to derive flow entropy from the payload header or the entropy 384 usage has to be disabled to meet operational requirements (see 385 Section 7), to avoid reordering with a packet flow, the encapsulator 386 SHOULD use the same UDP source port value for all packets assigned 387 to a flow e.g., the result of an algorithm that perform a hash of 388 the tunnel ingress and egress IP address. 390 The source port value for a flow set by an encapsulator MAY change 391 over the lifetime of the encapsulated flow. For instance, an 392 encapsulator may change the assignment for Denial of Service (DOS) 393 mitigation or as a means to effect routing through the ECMP network. 394 An encapsulator SHOULD NOT change the source port selected for a 395 flow more than once every thirty seconds. 397 An IPv6 GRE-in-UDP tunnel endpoint should copy a flow entropy value 398 in the IPv6 flow label field (requirement 6). This permits network 399 equipment to inspect this value and utilize it during forwarding, 400 e.g. to perform ECMP [RFC6438]. 402 This document places requirements on the generation of the flow 403 entropy value [RFC5405bis] but does not specify the algorithm that 404 an implementation should use to derive this value. 406 3.2.2. Destination Port 408 The destination port of the UDP header is set either GRE-in-UDP 409 (TBD1) or GRE-UDP-DTLS (TBD2) (see Section 5). 411 3.2.3. Checksum 413 The UDP checksum is set and processed per [RFC768] and [RFC1122] for 414 IPv4, and [RFC2460] for IPv6. Requirements for checksum handling and 415 use of zero UDP checksums are detailed in Section 6. 417 3.2.4. Length 419 The usage of this field is in accordance with the current UDP 420 specification in [RFC768]. This length will include the UDP header 421 (eight bytes), GRE header, and the GRE payload (encapsulated packet). 423 3.3. GRE Header 425 An encapsulator sets the protocol type (EtherType) of the packet 426 being encapsulated in the GRE Protocol Type field. 428 An encapsulator MAY set the GRE Key Present, Sequence Number Present, 429 and Checksum Present bits and associated fields in the GRE header as 430 defined by [RFC2784] and [RFC2890]. Usage of the reserved bits, i.e., 431 Reserved0, is specified in [RFC2784]. 433 The GRE checksum MAY be enabled to protect the GRE header and 434 payload. When the UDP checksum is enabled, it protects the GRE 435 payload, resulting in the GRE checksum being mostly redundant. 436 Enabling both checksums may result in unnecessary processing. Since 437 the UDP checksum covers the pseudo-header and the packet payload, 438 including the GRE header and its payload, the UDP checksum SHOULD be 439 used in preference to using the GRE checksum. 441 An implementation MAY use the GRE keyid to authenticate the 442 encapsulator.(See Security Considerations Section) In this model, a 443 shared value is either configured or negotiated between an 444 encapsulator and decapsulator. When a decapsulator determines a 445 presented keyid is not valid for the source, the packet MUST be 446 dropped. 448 Although GRE-in-UDP encapsulation protocol uses both UDP header and 449 GRE header, it is one tunnel encapsulation protocol. GRE and UDP 450 headers MUST be applied and removed as a pair at the encapsulation 451 and decapsulation points. This specification does not support UDP 452 encapsulation of a GRE header where that GRE header is applied or 453 removed at a network node other than the UDP tunnel ingress or 454 egress. 456 4. Encapsulation Process Procedures 458 The procedures specified in this section apply to both a default 459 GRE-in-UDP tunnel and a TMCE GRE-in-UDP tunnel. 461 The GRE-in-UDP encapsulation allows encapsulated packets to be 462 forwarded through "GRE-in-UDP tunnels". The encapsulator MUST set 463 the UDP and GRE header according to Section 3. 465 Intermediate routers, upon receiving these UDP encapsulated packets, 466 could load balance these packets based on the hash of the five-tuple 467 of UDP packets. 469 Upon receiving these UDP encapsulated packets, the decapsulator 470 decapsulates them by removing the UDP and GRE headers and then 471 processes them accordingly. 473 GRE-in-UDP allows encapsulation of unicast, IPv4 broadcast, or 474 multicast traffic. Entropy may be generated from the header of 475 encapsulated packets at an encapsulator. The mapping mechanism 476 between the encapsulated multicast traffic and the multicast 477 capability in the IP network is transparent and independent of the 478 encapsulation and is otherwise outside the scope of this document. 480 To provide entropy for ECMP, GRE-in-UDP does not rely on GRE keep- 481 alive. It is RECOMMENED not to use GRE keep-alive in the GRE-in-UDP 482 tunnel. This aligns with middlebox traversal guidelines in Section 483 3.5 of [RFC5405bis]. 485 4.1. MTU and Fragmentation 487 Regarding packet fragmentation, an encapsulator/decapsulator SHOULD 488 perform fragmentation before the encapsulation. The size of 489 fragments SHOULD be less or equal to the PMTU associated with the 490 path between the GRE ingress and the GRE egress tunnel endpoints 491 minus the GRE and UDP overhead, assuming the egress resemble MTU is 492 larger than PMTU. When applying payload fragmentation, the UDP 493 checksum MUST be used so that the receiving endpoint can validate 494 reassembly of the fragments; the same source UDP port SHOULD be used 495 for all packet fragments to ensure the transit routers will forward 496 the fragments on the same path. 498 If the operator of the transit network supporting the tunnel is able 499 to control the payload MTU size, the MTU SHOULD be configured to 500 avoid fragmentation, i.e., sufficient for the largest supported size 501 of packet, including all additional bytes introduced by the tunnel 502 overhead [RFC5405bis]. 504 4.2. Differentiated Services and ECN Marking 506 To ensure that tunneled traffic receives the same treatment over the 507 IP network as traffic that is not tunneled, prior to the 508 encapsulation process, an encapsulator processes the tunneled IP 509 packet headers to retrieve appropriate parameters for the 510 encapsulating IP packet header such as DiffServ [RFC2983]. 511 Encapsulation end points that support Explicit Congestion 512 Notification (ECN) must use the method described in [RFC6040] for 513 ECN marking propagation. The congestion control process is outside 514 of the scope of this document. 516 Additional information on IP header processing is provided in 517 Section 3.1. 519 5. Use of DTLS 521 Datagram Transport Layer Security (DTLS) [RFC6347] can be used for 522 application security and can preserve network and transport layer 523 protocol information. Specifically, if DTLS is used to secure the 524 GRE-in-UDP tunnel, the destination port of the UDP header MUST be 525 set to an IANA-assigned value (TBD2) indicating GRE-in-UDP with DTLS, 526 and that UDP port MUST NOT be used for other traffic. The UDP source 527 port field can still be used to add entropy, e.g., for load-sharing 528 purposes. DTLS applies to a default GRE-in-UDP tunnel and a TMCE 529 GRE-in-UDP tunnel. 531 Use of DTLS is limited to a single DTLS session for any specific 532 tunnel encapsulator/decapsulator pair (identified by source and 533 destination IP addresses). Both IP addresses MUST be unicast 534 addresses - multicast traffic is not supported when DTLS is used. A 535 GRE-in-UDP tunnel decapsulator that supports DTLS is expected to be 536 able to establish DTLS sessions with multiple tunnel encapsulators, 537 and likewise a GRE-in-UDP tunnel encapsulator is expected to be able 538 to establish DTLS sessions with multiple decapsulators. Different 539 source and/or destination IP addresses will be involved (see Section 540 6.2) for discussion of one situation where use of different source 541 IP addresses is important. 543 If an application already performs encryption, no need to encrypt 544 traffic again. Applying DTLS to a GRE-in-UDP tunnel requires both 545 tunnel end points to configure use of DTLS. 547 6. UDP Checksum Handling 549 6.1. UDP Checksum with IPv4 551 For UDP in IPv4, the UDP checksum MUST be processed as specified in 552 [RFC768] and [RFC1122] for both transmit and receive. The IPv4 553 header includes a checksum that protects against mis-delivery of the 554 packet due to corruption of IP addresses. The UDP checksum 555 potentially provides protection against corruption of the UDP header, 556 GRE header, and GRE payload. Disabling the use of checksums is a 557 deployment consideration that should take into account the risk and 558 effects of packet corruption. 560 When a decapsulator receives a packet, the UDP checksum field MUST 561 be processed. If the UDP checksum is non-zero, the decapsulator MUST 562 verify the checksum before accepting the packet. By default a 563 decapsulator SHOULD accept UDP packets with a zero checksum. A node 564 MAY be configured to disallow zero checksums per [RFC1122]; this may 565 be done selectively, for instance disallowing zero checksums from 566 certain hosts that are known to be sending over paths subject to 567 packet corruption. If verification of a non-zero checksum fails, a 568 decapsulator lacks the capability to verify a non-zero checksum, or 569 a packet with a zero-checksum was received and the decapsulator is 570 configured to disallow, the packet MUST be dropped and an event MAY 571 be logged. 573 6.2. UDP Checksum with IPv6 575 For UDP in IPv6, the UDP checksum MUST be processed as specified in 576 [RFC768] and [RFC2460] for both transmit and receive. 578 When UDP is used over IPv6, the UDP checksum is relied upon to 579 protect both the IPv6 and UDP headers from corruption. As such, A 580 default GRE-in-UDP Tunnel MUST perform UDP checksum; A TMCE GRE-in- 581 UDP Tunnel MAY be configured with the UDP zero-checksum mode if the 582 traffic-managed controlled environment or a set of closely 583 cooperating traffic-managed controlled environments (such as by 584 network operators who have agreed to work together in order to 585 jointly provide specific services) meet at least one of following 586 conditions: 588 a. It is known (perhaps through knowledge of equipment types and 589 lower layer checks) that packet corruption is exceptionally 590 unlikely and where the operator is willing to take the risk of 591 undetected packet corruption. 593 b. It is judged through observational measurements (perhaps of 594 historic or current traffic flows that use a non-zero checksum) 595 that the level of packet corruption is tolerably low and where 596 the operator is willing to take the risk of undetected packet 597 corruption. 599 c. Carrying applications that are tolerant of mis-delivered or 600 corrupted packets (perhaps through higher layer checksum, 601 validation, and retransmission or transmission redundancy) where 602 the operator is willing to rely on the applications using the 603 tunnel to survive any corrupt packets. 605 The following requirements apply to a TMCE GRE-in-UDP tunnel that 606 uses UDP zero-checksum mode: 608 a. Use of the UDP checksum with IPv6 MUST be the default 609 configuration of all GRE-in-UDP tunnels. 611 b. The GRE-in-UDP tunnel implementation MUST comply with all 612 requirements specified in Section 4 of [RFC6936] and with 613 requirement 1 specified in Section 5 of [RFC6936]. 615 c. The tunnel decapsulator SHOULD only allow the use of UDP zero- 616 checksum mode for IPv6 on a single received UDP Destination 617 Port regardless of the encapsulator. The motivation for this 618 requirement is possible corruption of the UDP Destination Port, 619 which may cause packet delivery to the wrong UDP port. If that 620 other UDP port requires the UDP checksum, the mis-delivered 621 packet will be discarded. 623 d. It is RECOMMENDED that the UDP zero-checksum mode for IPv6 is 624 only enabled for certain selected source addresses. The tunnel 625 decapsulator MUST check that the source and destination IPv6 626 addresses are valid for the GRE-in-UDP tunnel on which the 627 packet was received if that tunnel uses UDP zero-checksum mode 628 and discard any packet for which this check fails. 630 e. The tunnel encapsulator SHOULD use different IPv6 addresses for 631 each GRE-in-UDP tunnel that uses UDP zero-checksum mode 632 regardless of the decapsulator in order to strengthen the 633 decapsulator's check of the IPv6 source address (i.e., the same 634 IPv6 source address SHOULD NOT be used with more than one IPv6 635 destination address, independent of whether that destination 636 address is a unicast or multicast address). When this is not 637 possible, it is RECOMMENDED to use each source IPv6 address for 638 as few UDP zero-checksum mode GRE-in-UDP tunnels as is feasible. 640 f. When any middlebox exists on the path of a GRE-in-UDP tunnel, 641 it is RECOMMENDED to use the default mode, i.e. use UDP 642 checksum, to reduce the chance that the encapsulated packets 643 will be dropped. 645 g. Any middlebox that allows the UDP zero-checksum mode for IPv6 646 MUST comply with requirement 1 and 8-10 in Section 5 of 647 [RFC6936]. 649 h. Measures SHOULD be taken to prevent IPv6 traffic with zero UDP 650 checksums from "escaping" to the general Internet; see Section 651 8 for examples of such measures. 653 i. IPv6 traffic with zero UDP checksums MUST be actively monitored 654 for errors by the network operator. For example, the operator 655 may monitor Ethernet layer packet error rates. 657 j. If a packet with a non-zero checksum is received, the checksum 658 MUST be verified before accepting the packet. This is 659 regardless of whether the tunnel encapsulator and decapsulator 660 have been configured with UDP zero-checksum mode. 662 The above requirements do not change either the requirements 663 specified in [RFC2460] as modified by [RFC6935] or the requirements 664 specified in [RFC6936]. 666 The requirement to check the source IPv6 address in addition to the 667 destination IPv6 address, plus the strong recommendation against 668 reuse of source IPv6 addresses among GRE-in-UDP tunnels collectively 669 provide some mitigation for the absence of UDP checksum coverage of 670 the IPv6 header. A traffic-managed controlled environment that 671 satisfies at least one of three conditions listed at the beginning 672 of this section provides additional assurance. 674 A GRE-in-UDP tunnel is suitable for transmission over lower layers 675 in the traffic-managed controlled environments that are allowed by 676 the exceptions stated above and the rate of corruption of the inner 677 IP packet on such networks is not expected to increase by comparison 678 to GRE traffic that is not encapsulated in UDP. For these reasons, 679 GRE-in-UDP does not provide an additional integrity check except 680 when GRE checksum is used when UDP zero-checksum mode is used with 681 IPv6, and this design is in accordance with requirements 2, 3 and 5 682 specified in Section 5 of [RFC6936]. 684 Generic Router Encapsulation (GRE) does not accumulate incorrect 685 transport layer state as a consequence of GRE header corruption. A 686 corrupt GRE packet may result in either packet discard or forwarding 687 of the packet without accumulation of GRE state. Active monitoring 688 of GRE-in-UDP traffic for errors is REQUIRED as occurrence of errors 689 will result in some accumulation of error information outside the 690 protocol for operational and management purposes. This design is in 691 accordance with requirement 4 specified in Section 5 of [RFC6936]. 693 The remaining requirements specified in Section 5 of [RFC6936] are 694 not applicable to GRE-in-UDP. Requirements 6 and 7 do not apply 695 because GRE does not include a control feedback mechanism. 696 Requirements 8-10 are middlebox requirements that do not apply to 697 GRE-in-UDP tunnel endpoints (see Section 7.1 for further middlebox 698 discussion). 700 It is worth mentioning that the use of a zero UDP checksum should 701 present the equivalent risk of undetected packet corruption when 702 sending similar packet using GRE-in-IPv6 without UDP [RFC7676] and 703 without GRE checksums. 705 In summary, a TMCE GRE-in-UDP Tunnel is allowed to use UDP-zero- 706 checksum mode for IPv6 when the conditions and requirements stated 707 above are met. Otherwise the UDP checksum need to be used for IPv6 708 as specified in [RFC768] and [RFC2460]. Use of GRE checksum is 709 RECOMMENED when the UDP checksum is not used. 711 7. Middlebox Considerations 713 Many middleboxes read or update UDP port information of the packets 714 that they forward. Network Address/Port Translator (NAPT) is the 715 most commonly deployed Network Address Translation (NAT) device 716 [RFC4787]. An NAPT device establishes a NAT session to translate the 717 {private IP address, private source port number} tuple to a {public 718 IP address, public source port number} tuple, and vice versa, for 719 the duration of the UDP session. This provides a UDP application 720 with the "NAT-pass-through" function. NAPT allows multiple internal 721 hosts to share a single public IP address. The port number, i.e., 722 the UDP Source Port number, is used as the demultiplexer of the 723 multiple internal hosts. However, the above NAPT behaviors conflict 724 with the behavior a GRE-in-UDP tunnel that is configured to use the 725 UDP source port value to provide entropy. 727 A GRE-in-UDP tunnel is unidirectional; the tunnel traffic is not 728 expected to be returned back to the UDP source port values used to 729 generate entropy. However some middleboxes (e.g., firewall) assume 730 that bidirectional traffic uses a common pair of UDP ports. This 731 assumption also conflicts with the use of the UDP source port field 732 as entropy. 734 Hence, use of the UDP source port for entropy may impact middleboxes 735 behavior. If a GRE-in-UDP tunnel is expected to be used on a path 736 with a middlebox, the tunnel can be configured to either disable use 737 of the UDP source port for entropy or to configure middleboxes to 738 pass packets with UDP source port entropy. 740 7.1. Middlebox Considerations for Zero Checksums 742 IPv6 datagrams with a zero UDP checksum will not be passed by any 743 middlebox that validates the checksum based on [RFC2460] or that 744 updates the UDP checksum field, such as NATs or firewalls. Changing 745 this behavior would require such middleboxes to be updated to 746 correctly handle datagrams with zero UDP checksums. The GRE-in-UDP 747 encapsulation does not provide a mechanism to safely fall back to 748 using a checksum when a path change occurs redirecting a tunnel over 749 a path that includes a middlebox that discards IPv6 datagrams with a 750 zero UDP checksum. In this case the GRE-in-UDP tunnel will be black- 751 holed by that middlebox. 753 As such, when any middlebox exists on the path of GRE-in-UDP tunnel, 754 use of the UDP checksum is RECOMMENDED to increase the probability 755 of successful transmission of GRE-in-UDP packets. Recommended 756 changes to allow firewalls, NATs and other middleboxes to support 757 use of an IPv6 zero UDP checksum are described in Section 5 of 758 [RFC6936]. 760 8. Congestion Considerations 762 Section 3.1.9 of [RFC5405bis] discusses the congestion 763 considerations for design and use of UDP tunnels; this is important 764 because other flows could share the path with one or more UDP 765 tunnels, necessitating congestion control [RFC2914] to avoid 766 distractive interference. 768 Congestion has potential impacts both on the rest of the network 769 containing a UDP tunnel, and on the traffic flows using the UDP 770 tunnels. These impacts depend upon what sort of traffic is carried 771 over the tunnel, as well as the path of the tunnel. A default GRE- 772 in-UDP tunnel MAY be used to carry IP traffic that is known to be 773 congestion controlled on the Internet. IP unicast traffic is 774 generally assumed to be congestion-controlled. A default GRE-in-UDP 775 tunnel MUST NOT be used to carry traffic that is not known to be 776 congestion-controlled. 778 A TMCE GRE-in-UDP tunnel can be used to carry traffic that is known 779 not to be congestion controlled. For example, GRE-in-UDP may be used 780 to carry MPLS that carries pseudowire or VPN traffic where specific 781 bandwidth guarantees are provided to each pseudowire or to each VPN. 782 In such cases, network operators may avoid congestion by careful 783 provisioning of their networks, by rate limiting of user data 784 traffic, and traffic engineering according to path capacity. 786 When a TMCE GRE-in-UDP tunnel carries traffic that is not known to 787 be congestion controlled, the tunnel MUST be used within a traffic- 788 managed controlled environment (e.g., single operator network that 789 utilizes careful provisioning such as rate limiting at the entries 790 of the network while over-provisioning network capacity) to manage 791 congestion, or within a limited number of networks whose operators 792 closely cooperate in order to jointly provide this same careful 793 provisioning. When a TMCE GRE-in-UDP tunnel is used to carry the 794 traffic that is not known to be congestion controlled, measures 795 SHOULD be taken to prevent the GRE-in-UDP traffic from "escaping" to 796 the general Internet, e.g.: 798 o Physical or logical isolation of the links carrying GRE-in-UDP 799 from the general Internet. 801 o Deployment of packet filters that block the UDP ports assigned 802 for GRE-in-UDP. 804 o Imposition of restrictions on GRE-in-UDP traffic by software 805 tools used to set up GRE-in-UDP tunnels between specific end 806 systems (as might be used within a single data center) or by 807 tunnel ingress nodes for tunnels that don't terminate at end 808 systems. 810 o Use of a "Circuit Breaker" for the tunneled traffic as described 811 in [CB]. 813 9. Backward Compatibility 815 In general, tunnel ingress routers have to be upgraded in order to 816 support the encapsulations described in this document. 818 No change is required at transit routers to support forwarding of 819 the encapsulation described in this document. 821 If a tunnel endpoint (a host or router) that is intended for use as 822 a decapsulator does not support or enable the GRE-in-UDP 823 encapsulation described in this document, that endpoint will not 824 listen on the destination port assigned to the GRE-encapsulation 825 (TBD1 and TBD2). In these cases, the endpoint will perform normal 826 UDP processing and respond to an encapsulator with an ICMP message 827 indicating "port unreachable" according to [RFC792]. Upon receiving 828 this ICMP message, the node MUST NOT continue to use GRE-in-UDP 829 encapsulation toward this peer without management intervention. 831 10. IANA Considerations 833 IANA is requested to make the following allocations: 835 One UDP destination port number for the indication of GRE, 837 Service Name: GRE-in-UDP 838 Transport Protocol(s): UDP 839 Assignee: IESG 840 Contact: IETF Chair 841 Description: GRE-in-UDP Encapsulation 842 Reference: [This.I-D] 843 Port Number: TBD1 844 Service Code: N/A 845 Known Unauthorized Uses: N/A 846 Assignment Notes: N/A 848 Editor Note: replace "TBD1" with IANA assigned number in this 849 document. 851 One UDP destination port number for the indication of GRE with DTLS, 853 Service Name: GRE-UDP-DTLS 854 Transport Protocol(s): UDP 855 Assignee: IESG 856 Contact: IETF Chair 857 Description: GRE-in-UDP Encapsulation with DTLS 858 Reference: [This.I-D] 859 Port Number: TBD2 860 Service Code: N/A 861 Known Unauthorized Uses: N/A 862 Assignment Notes: N/A 864 Editor Note: replace "TBD2" with IANA assigned number in this 865 document. 867 11. Security Considerations 869 GRE-in-UDP encapsulation does not affect security for the payload 870 protocol. When using GRE-in-UDP, Network Security in a network is 871 mostly equivalent to that of a network using GRE. 873 To secure original traffic, DTLS SHOULD be used as specified in 874 Section 5. 876 In the case that UDP source port for entropy usage is disabled, a 877 random port SHOULD be selected in order to minimize the 878 vulnerability to off-path attacks.[RFC6056] The random port may also 879 be periodically changed to mitigate certain denial of service 880 attacks as mentioned in Section 3.2.1. 882 Using one standardized value as the UDP destination port to indicate 883 an encapsulation may increase the vulnerability of off-path attack. 884 To overcome this, an alternate port may be agreed upon to use 885 between an encapsulator and decapsulator [RFC6056]. How the 886 encapsulator end points communicate the value is outside scope of 887 this document. 889 This document does not require that a decapsulator validates the IP 890 source address of the tunneled packets (with the exception that the 891 IPv6 source address MUST be validated when UDP zero-checksum mode is 892 used with IPv6), but it should be understood that failure to do so 893 presupposes that there is effective destination-based (or a 894 combination of source-based and destination-based) filtering at the 895 boundaries. 897 Corruption of a GRE header can cause a privacy and security concern 898 for some applications that rely on the key field for traffic 899 segregation. When the GRE key field is used for privacy and security, 900 ether UDP checksum or GRE checksum SHOULD be used for GRE-in-UDP 901 with both IPv4 and IPv6, and in particular, when UDP zero-checksum 902 mode is used, GRE checksum SHOULD be used. 904 12. Acknowledgements 906 Authors like to thank Vivek Kumar, Ron Bonica, Joe Touch, Ruediger 907 Geib, Lar Edds, Lloyd Wood, Bob Briscoe, and many others for their 908 review and valuable input on this draft. 910 Thank Donald Eastlake, Eliot Lear, and Martin Stiemerling for their 911 detail reviews and valuable suggestions in WGLC process. 913 Thank the design team led by David Black (members: Ross Callon, 914 Gorry Fairhurst, Xiaohu Xu, Lucy Yong) to efficiently work out the 915 descriptions for the congestion considerations and IPv6 UDP zero 916 checksum. 918 Thank David Black and Gorry Fairhurst for their great help in 919 document content and editing. 921 13. Contributors 923 The following people all contributed significantly to this document 924 and are listed below in alphabetical order: 926 David Black 927 EMC Corporation 928 176 South Street 929 Hopkinton, MA 01748 930 USA 932 Email: david.black@emc.com 934 Ross Callon 935 Juniper Networks 936 10 Technology Park Drive 937 Westford, MA 01886 938 USA 940 Email: rcallon@juniper.net 942 John E. Drake 943 Juniper Networks 945 Email: jdrake@juniper.net 947 Gorry Fairhurst 948 University of Aberdeen 950 Email: gorry@erg.abdn.ac.uk 952 Yongbing Fan 953 China Telecom 954 Guangzhou, China. 956 Phone: +86 20 38639121 958 Email: fanyb@gsta.com 960 Adrian Farrel 961 Juniper Networks 963 Email: adrian@olddog.co.uk 965 Vishwas Manral 966 Hewlett-Packard Corp. 967 3000 Hanover St, Palo Alto. 969 Email: vishwas.manral@hp.com 971 Carlos Pignataro 972 Cisco Systems 973 7200-12 Kit Creek Road 974 Research Triangle Park, NC 27709 USA 976 EMail: cpignata@cisco.com 978 14. References 980 14.1. Normative References 982 [RFC768] Postel, J., "User Datagram Protocol", STD 6, RFC 768, 983 August 1980. 985 [RFC1122] Braden, R., "Requirements for Internet Hosts -- 986 Communication Layers", RFC1122, October 1989. 988 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 989 Requirement Levels", BCP 14, RFC2119, March 1997. 991 [RFC2784] Farinacci, D., Li, T., Hanks, S., Meyer, D., and P. 992 Traina, "Generic Routing Encapsulation (GRE)", RFC 2784, 993 March 2000. 995 [RFC2890] Dommety, G., "Key and Sequence Number Extensions to GRE", 996 RFC2890, September 2000. 998 [RFC5405bis] Eggert, L., "Unicast UDP Usage Guideline for 999 Application Designers", draft-ietf-tsvwg-rfc5405bis, work 1000 in progress. 1002 [RFC6040] Briscoe, B., "Tunneling of Explicit Congestion 1003 Notification", RFC6040, November 2010. 1005 [RFC6347] Rescoria, E., Modadugu, N., "Datagram Transport Layer 1006 Security Version 1.2", RFC6347, 2012. 1008 [RFC6438] Carpenter, B., Amante, S., "Using the IPv6 Flow Label for 1009 Equal Cost Multipath Routing and Link Aggregation in 1010 tunnels", RFC6438, November, 2011. 1012 [RFC6935] Eubanks, M., Chimento, P., and M. Westerlund, "IPv6 and 1013 UDP Checksums for Tunneled Packets", RFC 6935, April 2013. 1015 [RFC6936] Fairhurst, G. and M. Westerlund, "Applicability Statement 1016 for the Use of IPv6 UDP Datagrams with Zero Checksums", 1017 RFC 6936, April 2013. 1019 14.2. Informative References 1021 [RFC792] Postel, J., "Internet Control Message Protocol", STD 5, RFC 1022 792, September 1981. 1024 [RFC793] DARPA, "Transmission Control Protocol", RFC793, September 1025 1981. 1027 [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1028 (IPv6) Specification", RFC 2460, December 1998. 1030 [RFC2474] Nichols K., Blake S., Baker F., Black D., "Definition of 1031 the Differentiated Services Field (DS Field) in the IPv4 1032 and IPv6 Headers", December 1998. 1034 [RFC2914] Floyd, S.,"Congestion Control Principles", RFC2914, 1035 September 2000. 1037 [RFC2983] Black, D., "Differentiated Services and Tunnels", RFC2983, 1038 October 2000. 1040 [RFC4787] Audet, F., et al, "network Address Translation (NAT) 1041 Behavioral Requirements for Unicast UDP", RFC4787, January 1042 2007. 1044 [RFC6056] Larsen, M. and Gont, F., "Recommendations for Transport- 1045 Protocol Port Randomization", RFC6056, January 2011. 1047 [RFC6438] Carpenter, B., Amante, S., "Using the Ipv6 Flow Label for 1048 Equal Cost Multipath Routing and Link Aggreation in 1049 Tunnels", RFC6438, November 2011. 1051 [RFC7637] Garg, P. and Wang, Y., "NVGRE: Network Virtualization 1052 Using Generic Routing Encapsulation", RFC7637, September 1053 2015. 1055 [RFC7676] Pignataro, C., Bonica, R., Krishnan, S., "IPv6 Support for 1056 Generic Routing Encapsulation (GRE)", RFC7676, October 1057 2015. 1059 [CB] Fairhurst, G., "Network Transport Circuit Breakers", 1060 draft-ietf-tsvwg-circuit-breaker-13, work in progress. 1062 15. Authors' Addresses 1064 Lucy Yong 1065 Huawei Technologies, USA 1067 Email: lucy.yong@huawei.com 1069 Edward Crabbe 1070 Oracle 1072 Email: edward.crabbe@gmail.com 1074 Xiaohu Xu 1075 Huawei Technologies, 1076 Beijing, China 1078 Email: xuxiaohu@huawei.com 1080 Tom Herbert 1081 Facebook 1082 1 Hacker Way 1083 Menlo Park, CA 1084 Email : tom@herbertland.com