INTERNET-DRAFT Margaret Cullen Intended Status: Proposed Standard Painless Security Updates: 7177, 7178 Donald Eastlake Mingui Zhang Huawei Dacheng Zhang Alibaba Expires: January 5, 2016 July 6, 2015 Transparent Interconnection of Lots of Links (TRILL) over IP Abstract The Transparent Interconnection of Lots of Links (TRILL) protocol is implemented by devices called TRILL Switches or RBridges (Routing Bridges). TRILL supports both point-to-point and multi-access links and is designed so that a variety of link protocols can be used between TRILL switch ports. This document standardizes methods for encapsulating TRILL in IP (v4 or v6) so as to use IP as a TRILL link protocol in a unified TRILL campus. It updates RFC 7177 and RFC 7178. Status of This Document This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Distribution of this document is unlimited. Comments should be sent to the author or the DNSEXT mailing list . Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Margaret Cullen, et al [Page 1] INTERNET-DRAFT TRILL over IP Table of Contents 1. Introduction............................................4 2. Terminology.............................................4 3. Use Cases for TRILL over IP.............................5 3.1 Remote Office Scenario.................................5 3.2 IP Backbone Scenario...................................5 3.3 Important Properties of the Scenarios..................5 3.3.1 Security Requirements................................6 3.3.2 Multicast Handling...................................6 3.3.3 RBridge Neighbor Discovery...........................7 4. TRILL Packet Formats....................................8 5. Some Link Protocol Specifics............................9 6. TRILL over IP Port Configuration.......................10 6.1 Per IP Port Configuration.............................10 6.2 Additional per IP Address Cofiguration................10 6.2.1 Native Multicast Configuration......................10 6.2.2 Serial Unicast Configuration........................11 6.2.3 Encapsulation Specific Configuration................11 6.2.3.1 VXLAN Configuration...............................11 6.2.3.2 Other Encapsulation Configuration.................12 6.2.4 Security Configuration..............................12 7. TRILL over IP Encapsulation Formats....................13 7.1 Encapsulation Agreement...............................14 7.2 IPsec ESP Format......................................14 7.3 Broadcast Link Encapsulation Considerations...........15 7.4 Native Encapsulaton...................................15 7.5 VXLAN Encapsulation...................................16 7.6 Other Encpaulsations..................................17 8. Handling Multicast.....................................18 9. Use of IPsec...........................................19 9.1 Default Keys..........................................19 9.2 Mandatory-to-Implement Algorithms.....................19 10. Transport Considerations..............................20 10.1 Recursive Ingress....................................20 10.2 Fat Flows............................................20 10.3 Congestion Considerations............................21 10.4 MTU Considerations...................................22 10.5 QoS Considerations...................................23 11. Middlebox Considerations..............................24 12. Security Considerations...............................25 Margaret Cullen, et al [Page 2] INTERNET-DRAFT TRILL over IP Table of Contents (continued) 13. IANA Considerations...................................26 13.1 Port Assignments.....................................26 13.2 Multicast Address Assignments........................26 13.3 Encapsulation Method Support Indication..............26 Normative References......................................28 Informative References....................................30 Acknowledgements..........................................31 Authors' Addresses........................................32 Margaret Cullen, et al [Page 3] INTERNET-DRAFT TRILL over IP 1. Introduction TRILL switches (RBridges) are devices that implement the IETF TRILL protocol [RFC6325] [RFC7177] [rfc7180bis]. RBridges provide transparent forwarding of frames within an arbitrary network topology, using least cost paths for unicast traffic. They support not only VLANs and Fine Grained Labels [RFC7172] but also multipathing of unicast and multi-destination traffic. They use IS-IS link state routing and encapsulation with a hop count. Ports on different RBridges can communicate with each other over various link types, such as Ethernet [RFC6325], pseudowires [RFC7173], or PPP [RFC6361]. This document defines a method for RBridges to communicate over IP (v4 or v6). TRILL over IP will allow Internet-connected RBridges to form a single TRILL campus, or multiple TRILL over IP networks within a campus to be connected as a single TRILL campus via a TRILL over IP backbone. TRILL over IP connects RBridge ports using IPv4 or IPv6 as a transport in such a way that the ports appear to TRILL to be connected by a single multi-access link. Therefore, if more than two RBridge ports are connected via a single TRILL over IP link, any pair of them can communicate. To support the scenarios where RBridges are connected via IP paths (such as over the public Internet) that are not under the same administrative control as the TRILL campus and/or not physically secure, this document specifies the use of IPsec [RFC4301] Encapsulating Security Protocol [RFC4303] to secure all or part of such paths. To support the use of TRILL over IP encapsulation with good fast path hardware support, a method is provided for agreement between adjacent TRILL switches as to what encapsulation to use. This document updates [RFC7177] and [RFC7178] as described in Section 7 by redefining an interval of RBridge Channel protocol numbers to indicate encapsulation method support for TRILL over IP and by making adjacency between TRILL over IP ports dependent on having a method of encapsulation in common. 2. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. Margaret Cullen, et al [Page 4] INTERNET-DRAFT TRILL over IP 3. Use Cases for TRILL over IP This section introduces two application scenarios (a remote office scenario and an IP backbone scenario) which cover typical situations where network administrators may choose to use TRILL over an IP network to connect TRILL switches. 3.1 Remote Office Scenario In the Remote Office Scenario, a remote TRILL network is connected to a TRILL campus across a multihop IP network, such as the public Internet. The TRILL network in the remote office becomes a part of TRILL campus, and nodes in the remote office can be attached to the same VLANs or Fine Grained Labels [RFC7172] as local campus nodes. In many cases, a remote office may be attached to the TRILL campus by a single pair of RBridges, one on the campus end, and the other in the remote office. In this use case, the TRILL over IP link will often cross logical and physical IP networks that do not support TRILL, and are not under the same administrative control as the TRILL campus. 3.2 IP Backbone Scenario In the IP Backbone Scenario, TRILL over IP is used to connect a number of TRILL networks to form a single TRILL campus. For example, a TRILL over IP backbone could be used to connect multiple TRILL networks on different floors of a large building, or to connect TRILL networks in separate buildings of a multi-building site. In this use case, there may often be several TRILL switches on a single TRILL over IP link, and the IP link(s) used by TRILL over IP are typically under the same administrative control as the rest of the TRILL campus. 3.3 Important Properties of the Scenarios There are a number of differences between the above two application scenarios, some of which drive features of this specification. These differences are especially pertinent to the security requirements of the solution, how multicast data frames are handled, and how the TRILL switch ports discover each other. Margaret Cullen, et al [Page 5] INTERNET-DRAFT TRILL over IP 3.3.1 Security Requirements In the IP Backbone Scenario, TRILL over IP is used between a number of RBridge ports, on a network link that is in the same administrative control as the remainder of the TRILL campus. While it is desirable in this scenario to prevent the association of unauthorized RBridges, this can be accomplished using existing IS-IS security mechanisms. There may be no need to protect the data traffic, beyond any protections that are already in place on the local network. In the Remote Office Scenario, TRILL over IP may run over a network that is not under the same administrative control as the TRILL network. Nodes on the network may think that they are sending traffic locally, while that traffic is actually being sent, in an IP tunnel, over the public Internet. It is necessary in this scenario to protect the integrity and confidentiality of user traffic, as well as ensuring that no unauthorized RBridges can gain access to the RBridge campus. The issues of protecting integrity and confidentiality of user traffic are addressed by using IPsec for both TRILL IS-IS and TRILL Data packets between RBridges in this scenario. 3.3.2 Multicast Handling In the IP Backbone scenario, native IP multicast may be supported on the TRILL over IP link. If so, it can be used to send TRILL IS-IS and multicast data packets, as discussed later in this document. Alternatively, multi-destination packets can be transmitted serially by IP unicast to the intended recipients. In the Remote Office Scenario there will often be only one pair of RBridges connecting a given site and, even when multiple RBridges are used to connect a Remote Office to the TRILL campus, the intervening network may not provide reliable (or any) multicast connectivity. Issues such as complex key management also make it difficult to provide strong data integrity and confidentiality protections for multicast traffic. For all of these reasons, the connections between local and remote RBridges will commonly be treated like point-to- point links, and all TRILL IS-IS control messages and multicast data packets that are transmitted between the Remote Office and the TRILL campus will be serially transmitted by IP unicast, as discussed later in this document. Margaret Cullen, et al [Page 6] INTERNET-DRAFT TRILL over IP 3.3.3 RBridge Neighbor Discovery In the IP Backbone Scenario, RBridges that use TRILL over IP can use the normal TRILL IS-IS Hello mechanisms to discover the existence of other RBridges on the link [RFC7177], and to establish authenticated communication with those RBridges. In the Remote Office Scenario, an IPsec session will need to be established before TRILL IS-IS traffic can be exchanged, as discussed below. In this case, one end will need to be configured to establish a IPSEC session with the other. This will typically be accomplished by configuring the RBridge or a border device at a Remote Office to initiate an IPsec session and subsequent TRILL exchanges with a TRILL over IP-enabled RBridge attached to the TRILL campus. Margaret Cullen, et al [Page 7] INTERNET-DRAFT TRILL over IP 4. TRILL Packet Formats To support the TRILL base protocol standard [RFC6325], two types of packets are transmitted between RBridges: TRILL Data packets and TRILL IS-IS packets. The on-the-wire form of a TRILL Data packet in transit between two neighboring RBridges is as shown below: +--------------+----------+----------------+-----------+ | TRILL Data | TRILL | Native Frame | Link | | Link Header | Header | Payload | Trailer | +--------------+----------+----------------+-----------+ Where the encapsulated Native Frame Payload is similar to an Ethernet frame with a VLAN tag or Fine Grained Label [RFC7172] but with no trailing Frame Check Sequence (FCS). TRILL IS-IS packets are formatted on-the-wire as follows: +--------------+---------------+-----------+ | TRILL IS-IS | TRILL IS-IS | Link | | Link Header | Payload | Trailer | +--------------+---------------+-----------+ The Link Header and Link Trailer in these formats depend on the specific link technology. The Link Header contains one or more fields that distinguish TRILL Data from TRILL IS-IS. For example, over Ethernet, the TRILL Data Link Header ends with the TRILL Ethertype while the TRILL IS-IS Link Header ends with the L2-IS-IS Ethertype; on the other hand, over PPP, there are no Ethertypes but PPP protocol code points are included that distinguish TRILL Data from TRILL IS- IS. In TRILL over IP, we will use IP (v4 or v6) in the link header. (On the wire, the IP header will be preceeded by the lower layer protocol that is carrying IP, such as Ethernet.) However, there are several IP based encapsulations usable for TRILL over IP as further discussed in Section 7 that differ in exactly what appears after the IP header and before the TRILL header. Margaret Cullen, et al [Page 8] INTERNET-DRAFT TRILL over IP 5. Some Link Protocol Specifics TRILL Data packets can be unicast to a specific RBridge or multicast to all RBridges on a link. TRILL IS-IS packets are always multicast to all other RBridge on the link (except for MTU PDUs, which may be unicast [RFC7177]. On Ethernet links, the Ethernet multicast address All-RBridges is used for TRILL Data and All-IS-IS-RBridges for TRILL IS-IS. To properly handle TRILL base protocol packets on a TRILL over IP link in the general case, either native IP multicast mode must be used on that link, or multicast must be simulated using serial IP unicast, as discussed in Section 8. (Of course, if the IP link happens to actually be point-to-point no special provision is needed for handling multicast addressed packets.) In TRILL Hello PDUs used on TRILL IP links, the IP addresses of the connected IP ports are their real SNPA (SubNetwork Point of Attachment [IS-IS]) addresses and, for IPv6, the 16-byte IPv6 address is used as the SNPA; however, for easy in re-using code designed for common 48-bit IS-IS SNPAs, for TRILL over IPv4, a 48-bit synthetic SNPA that looks like a unicast MAC address is constructed for use in the SNPA field of TRILL Neighbor TLVs [RFC7176] [RFC7177] on that link. This synthetic SNPA derived from an IPv4 address is as follows: 1 1 1 1 1 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 0xFE | 0x00 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPv4 upper half | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPv4 lower half | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ This synthetic SNPA (MAC) address has the local (0x02) bit on in the first byte and so cannot conflict with any globally unique 48-bit Ethernet MAC. However, at the IP level, where TRILL operates on an IP link, TRILL sees only IP stations, not MAC stations, even if the TRILL over IP Link is being carried over Ethernet, so conflict on the link in TRILL IS-IS between a real MAC address and the synethic SNPA (MAC) address as above would be impossible in any case. Margaret Cullen, et al [Page 9] INTERNET-DRAFT TRILL over IP 6. TRILL over IP Port Configuration This section specifies the configuration information needed at a TRILL over IP port beyond that needed for a general RBridge port. 6.1 Per IP Port Configuration Each RBridge port used for a TRILL over IP link should have at least one IP (v4 or v6) address. If no IP address is associated with the port, perhaps as a transient condition during re-configuration, the port is disabled. Implementations MAY allow a single port to operate as multiple IPv4 and/or IPv6 logical ports. Each IP address constitutes a different logical port and the RBridge with those ports MUST associate a different Port ID (see Section 4.4.2 of [RFC6325]) with each logical port. By default an TRILL over IP port discards output packets that fail the possible recursive ingress test (see Section 10.1) unless configured to disable that test. 6.2 Additional per IP Address Cofiguration The configuration information specified below is per IP address at a TRILL over IP port. The mapping from TRILL packet priority to Differentiated Services Code Point (DSCP [RFC2474]) can be configured (see Section 10.5). Each TRILL over IP port has a list of acceptable encapsulations it will use. By default this list consists of one entry for native encapsulation. (See Section 7.) Additional configuration is possible for specific encapsulations as described in Section 6.2.3. Each IP address at a TRILL over IP port uses native IP multicast by default but may be configured whether to use serial IP unicast (Section 6.2.2) or native IP multicast (Section 6.2.1). Each IP address at a TRILL over IP is configured whether or not to use IPsec (Section 6.2.3). 6.2.1 Native Multicast Configuration If a TRILL over IP port address is using native IP multicast for multi-destination TRILL packets (IS-IS and data), by default transmissions from that IP address use the appropriate IP multicast Margaret Cullen, et al [Page 10] INTERNET-DRAFT TRILL over IP address (IPv4 or IPv6) specified in Section 13.2. The TRILL over IP port may be configured to use a different IP multicast address for multi-destination packets. 6.2.2 Serial Unicast Configuration If a TRILL over IP port address has been configured to use serial unicast for multi-destination packets (IS-IS and data), it should have associated with it a non-empty list of unicast IP destination addresses with the same IP version as the version of the ports IP address (IPv4 or IPv6). Multi-destination TRILL packets are serially unicast to the addresses in this list. Such a TRILL over IP port will only be able to form adjacencies [RFC7177] with the RBridges at the addresses in this list as those are the only RBridges to which it will send TRILL Hellos. If this list of destination IP addresses is empty, there is no way to transmit a multi-destination TRILL over IP packet such as a TRILL Hello. Thus it is impossible to achieve adjacency [RFC7177] or if adjacency had been achieved (perhaps the list was non-empty and has just been configured to be empty), no way to maintain such adjacency. Thus, in the empty list case, TRILL Data multi-destination packets cannot be sent and TRILL Data unicast packets will not start flowing or, if they are already flowing, will soon cease, effectively disabling the port. 6.2.3 Encapsulation Specific Configuration Specific TRILL over IP encapsulation methods may provide for futher configuration as specified in the subsections below. 6.2.3.1 VXLAN Configuration A TRILL over IP port using VXLAN encapsulation can be configured with a non-default VXLAN Network Identifier (VNI) which is used in that field of the VXLAN header for all TRILL packets sent using the encapsulation and required in all TRILL packets received using the encapsulation. In this case, a TRILL packet received with the wrong VNI is discarded. A TRILL over IP port using VXLAN encapsulation can also be configured to place the Inner.VLAN or Inner.FGL of a TRILL Data packet being transported in the VNI field. Margaret Cullen, et al [Page 11] INTERNET-DRAFT TRILL over IP 6.2.3.2 Other Encapsulation Configuration [Specific configuration for other encapsulation methods will be added here.] 6.2.4 Security Configuration tbd ... Margaret Cullen, et al [Page 12] INTERNET-DRAFT TRILL over IP 7. TRILL over IP Encapsulation Formats There are a variety of TRILL over IP formats possible. In all cases, there must be a method specified, with each format, to distinguish TRILL Data and TRILL IS-IS packets, or that format is not useful for TRILL. The following criteria can be helpful is choosing between different encapsulations: a) Fast path support - For most applications, it is highly desireable to be able to encapsulate/decpasulate TRILL over IP at line speed so a format where existing or anticipated fast path hardware can do that is best. b) Ease of multi-pathing - The IP path between TRILL over IP port may include internal equal cost multipath routes so a method of encapsulation that provides variable fields available for existing or anticipated fast path hardware multi-pathing is better. c) Fragmentaton and robust ID support - tbd d) Checksum strength - Depending on the particular circumstances of the TRILL over IP link, a checksum provided by the encapsulation may be an important factor. Use of IPsec as provided herein can also provide a strong integrity check. TRILL over IP adopts a hybrid encapsulation approach by default. There is one format, called "native encapsulation" that MUST be implemented. Although native encapsulation does not typically have good fast path support, as a lowest common denominator it can be used with low bandwidth control messages to detetmine a preferred encapsulation with better performance. In particular, by default all TRILL IS-IS Hellos are sent using native encapsulation and those Hellos are used to determine the encpasulation used for all TRILL Data packets and all other TRILL IS-IS PDUs with the possible exception of IS-IS MTU-probe and MTU-ack PDUs as discussed in Section 7. Alternatively, the network operator can pre-configure a TRILL over IP port to use a particular encapsulation choosen for their particular network needs and TRILL over IP port capabiliies for all TRILL data and IS-IS packets. Section 7.1 discusses encapsulation agreement. Section 7.2 discusses TRILL over IP IPsec ESP format, which is independent of encapsulation. Section 7.3 discusses broadcast link encapsulation considerations. The subsequent subsections discuss particular encapsulations. Margaret Cullen, et al [Page 13] INTERNET-DRAFT TRILL over IP 7.1 Encapsulation Agreement TRILL Hellos sent out a TRILL over IP port indicate the encapsulations that port is willing to use through the mechanism described in [RFC7178] and [RFC7176]. RBridge Channel Protocol numbers 0xFC0 through 0xFF7 are hereby redefined to be link technology dependent flags that, for TRILL over IP, indicate support for different encapsulations, allowing for up to 24 encapsulations to be specified. Support for an encapsulation is indicated in the Hello PDU in the same way that support for an RBridge Channel was indicated. (See also section 13.3.) "Support" indicates willingness to use that encapsulation for TRILL data and TRILL IS-IS other than Hellos. Even if support is not indicated for native encapsulation, by default support for native encapsulation of TRILL Hellos is assumed. If no encapsulation support is indicated in a TRILL Hello, then the port from which it was sent is assumed to support only native encapsualtion (see Section 7.4). An adjacency is formed between two TRILL over IP ports ONLY if the intersection of the sets of encapsulation methods they support is not null. If that intersection is null, then no adjaceny is formed. In particular, for a TRILL over IP link, the adjacency state machine MUST NOT advance to the Report state unless the ports share an encapsulation [RFC7177]. If any TRILL over IP packet, other than a IS-IS Hello or MTU PDU in native encapsulation, is received in an encapsulation for which support is not being indicated, it MUST be discarded (see Section 7.3). It expected to normally be the case in a well configured network that all the TRILL over IP ports connected to an IP network that are intended to communicate with each other will support the same encapsulation. But the network will operate correctly if this is not true. 7.2 IPsec ESP Format TRILL over IP link security uses IPsec Encapsulating Security Protocol (ESP) in tunnel mode [RFC4303]. Since TRILL over IP always starts with an IP Header (on the wire this appears right after any required Layer 2 header), the modifications when IPsec is in effect are independent of the TRILL over IP encapsulation fields that occur after that IP Header and before the TRILL Header. The resulting packet formats are as follows for IPv4 and IPv6: Margaret Cullen, et al [Page 14] INTERNET-DRAFT TRILL over IP ------------------------------------------------------------ IPv4 | new IP hdr | | TRILL IP hdr | TRILL | ESP | ESP| |(any options)| ESP | (any options) | Encap2 |Trailer| ICV| ------------------------------------------------------------ |<--------- encryption ---------->| |<------------- integrity ------------->| ------------------------------------------------------------- IPv6 | new |new ext | | orig |orig ext | TRILL | ESP | ESP| |IP hdr| hdrs |ESP|IP hdr| hdrs | Encap2 |Trailer| ICV| ------------------------------------------------------------. |<--------- encryption ----------->| |<------------ integrity ------------->| The "TRILL Encap2" above includes whatever additional fields are required by the encapsulation in use followed by the TRILL Header and then the native frame payload (see Section 4). This architecture permits the ESP tunnel termination to be separated from the TRILL over IP RBridge port and, for example, placed at a physical or administrative security boundary. 7.3 Broadcast Link Encapsulation Considerations It is possible for the Hellos from a TRILL over IP port P1 to establish adjacency with multiple other TRILL over IP ports (P2, P3, ...) forming a broadcast link. In a well configured network one would expect such multiple other IP ports to support the same encapsulation but, if P1 supports multiple encapsulations, it is possible that P2 and P3, for example, do not have an encapsulation in common that is supported by P1. IS-IS can handle such non-transitive adjacencies which are reported as specified in [RFC7177]. If serial IP unicast is being used by P1, it can use different encapsulatons for different transmission. If native IP multicast is being used by P1, it will have to send one transmission per encapsulation method by which it has an adjanceny on the link. (It is for this reason that a TRILL over IP port MUST discard any packet received with the wrong encapsulation. Otherwise, packets would be duplicated.) 7.4 Native Encapsulaton The mandatory to implement "native encapsulaton" format of a TRILL over IP packet, when used without security, is TRILL over UDP as shown below. Margaret Cullen, et al [Page 15] INTERNET-DRAFT TRILL over IP +----------+--------+-----------------------+ | IP | UDP | TRILL | | Header | Header | Payload | +----------+--------+-----------------------+ Where the UDP Header is as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port = Entropy | Destination Port | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | UDP Length | UDP Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TRILL Payload ... Source Port - see Section 10.2 Destination Port - indicates TRILL Data or IS-IS, see Section 13 UDP Length - as specified in [RFC0768] UDP Checksum - as specified in [RFC0768] The TRILL Payload starts with the TRILL Header (not including the TRILL Ethertype) for TRILL Data packets and starts with the 0x83 Intradomain Routeing Protocol Discriminator byte (thus not including the L2-IS-IS Ethertype) for TRILL IS-IS packets. 7.5 VXLAN Encapsulation VXLAN [RFC7348] IP encapsulation of TRILL looks, on the wire, as TRILL over Ethernet over VXLAN over UDP over IP. The outer UDP uses a destination port number indicating VXLAN and the outer UDP source port may be used for entropy as with native encapsulation (see Section 7.2). The VXLAN header after the outer UDP header adds a 24 bit Virtual Network Identifier. The Ethernet header after the VXLAN header and before the TRILL header provides an Ethertype field that distinguishes TRILL data from TRILL IS-IS; however, the destination and source MAC addresses in this inner Ethernet header before the TRILL header are not used and represent 12 wasted bytes. A TRILL over IP port using VXLAN encapsulation by default uses a VNI of 1 but can be configured as described in Section 6.2.3.1. Margaret Cullen, et al [Page 16] INTERNET-DRAFT TRILL over IP 7.6 Other Encpaulsations [Additional encpasulations will be added here as additional subsections.] Margaret Cullen, et al [Page 17] INTERNET-DRAFT TRILL over IP 8. Handling Multicast By default, both TRILL IS-IS packets and multi-destination TRILL Data packets are sent to an All-RBridges IPv4 or IPv6 IP multicast Address as appropriate (see Section 13.2); however, a TRILL over IP port may be configured (see Section 6) to use serial IP unicast with a list of one or more unicast IP addresses of other TRILL over IP ports to which multi-destination packets are sent. In that case the outer IP header shows the IP unicast even through the TRILL header has the M bit set to one to indicate multi-destination. Serial unicast configuration is necessary if the TRILL over IP port is connected to an IP network that does not support IP multicast. In any case, unicast TRILL packets are sent by unicast IP. When a TRILL over IP port is using IP multicast, it MUST periodically transmit appropriate IGMP (IPv4 [RFC3376] or MLD (IPv6 [RFC2710]) packets so that the TRILL multicast IP traffic will be sent to it. Although TRILL fully supports broadcast links with more than 2 RBridges connected to the link, even where native IP multicast is available, there may be good reasons for configuring TRILL over IP ports to use serial unicast. In some networks, unicast is more reliable than multicast. If multiple unicast connections between parts of a TRILL campus are configured, TRILL will in any case spread traffic across them, treating them as parallel links, and appropriately fail over traffic if a link ceases to operate or incorporate a new link that comes up. Margaret Cullen, et al [Page 18] INTERNET-DRAFT TRILL over IP 9. Use of IPsec All RBridges that support TRILL over IP MUST implement IPsec [RFC4301] and support the use of IPsec Encapsulating Security Protocol (ESP [RFC4303]) to secure both TRILL IS-IS and TRILL data packets. When IPsec is used to secure a TRILL over IP link and no IS- IS security is enabled, the IPsec session MUST be fully established before any TRILL IS-IS or data packets are exchanged. When there is IS-IS security [RFC5310] provided, implementors may elect use IS-IS security to protect TRILL IS-IS packets. However, in this case, the IPsec session still MUST be fully established before any data packets transmission since IS-IS security does not provide any protection to data packets. 9.1 Default Keys The default pre-shared keys for IPsec usage are derived as follows: HMAC-SHA256 ("TRILL IP"| IS-IS-shared key ) In the above "|" indicates concatenation, HMAC-SHA256 is as described in [FIPS180] [RFC6234] and "TRILL IP" is the eight byte US ASCII [RFC0020] string indicated. "IS-IS-shared key" is a link (or wider scope) IS-IS key usable for IS-IS security of link local IS-IS local PDUs such as Hello, CSNP, and PSNP. With [RFC5310] there could be multiple keys identified with 16-bit key IDs. In this case, the Key ID of IS-IS-shared key is also used to identify the derived key. Although we are using pre-shared keys at the IPsec level, the IS-IS- shared keys from which they are derived expire and can be updated as described in RFC 5310. The derived keys MUST expire within the lifetime as the IS-IS-shared keys from which they were derived. 9.2 Mandatory-to-Implement Algorithms All RBridges that support TRILL over IP MUST implement the following algorithms for IPsec ESP, as recommended in [RFC4308]: Protocol ESP [RFC4303] ESP encryption AES with 128-bit keys in CBC mode [RFC3602] ESP integrity AES-XCBC-MAC-96 [RFC3566] Margaret Cullen, et al [Page 19] INTERNET-DRAFT TRILL over IP 10. Transport Considerations This section discusses a variety of transport considerations. 10.1 Recursive Ingress TRILL is designed to transport end station traffic to and from end stations over IEEE 802.3 and IP is frequently transported over IEEE 802.3 or similar protocols. Thus, an end station native data frame EF might get TRILL ingressed to TRILL(EF) which was then sent oout a TRILL over IP over 802.3 port resulting in an 802.3 frame of the form 802.3(IP(TRILL(EF))). There is a risk of such a packet being re- ingressed by the same TRILL campus, due to physical or logical misconfiguration, looping round, being further re-ingressed, etc. The packet might get discarded if it got too large but if fragmentation is enabled, it would just keep getting split into fragments that would continue to loop and grow and re-fragment until the path was saturated with junk and packets were being discarded due to queue overflow. The TRILL Header TTL would provide no protection because each TRILL ingress adds a new Header and TTL. To protect against this scenario, a TRILL over IP port MUST by, default, test whether a TRILL packet it is about to transmit is, in fact a TRILL ingress of a TRILL over IP over 802.3 or the like packets. That is, is it of the form TRILL(802.3(IP(TRILL(...)))? If so, the default action of the TRILL over IP output port is to discard the packet rather than transmit it. However, there are cases where some level of nested ingress is desired so it MUST be possible to configure the port to allow such packets. 10.2 Fat Flows For the purpose of load balancing, it is worthwhile to consider how to transport the TRILL packets over the Equal Cost Multiple Paths (ECMPs) existing internal to the IP path between two TRILL over IP ports. The ECMP election for the IP traffic could be based, at least for IPv4, on the quintuple of the outer IP header { Source IP, Destination IP, Source Port, Destination Port, and IP protocol }. Such tuples, however, could be exactly the same for all TRILL Data packets between two RBridge ports, even if there is a huge amount of data being sent between a variety of ingress and egress RBridges. Therefore, in order to better support ECMP, a RBridge SHOULD set the Source Port as an entropy field for ECMP decisions. (This idea is also introduced in [gre-in-udp].) For example, for TRILL Data this Margaret Cullen, et al [Page 20] INTERNET-DRAFT TRILL over IP entropy field could be based on the Inner.MacDA, Inner.MacSA, and Inner.VLAN or Inner.FGL. 10.3 Congestion Considerations Section 3.1.3 of [RFC5405] discussed the congestion implications of UDP tunnels. As discussed in [RFC5405], because other flows can share the path with one or more UDP tunnels, congestion control [RFC2914] needs to be considered. The default initial determination of the TRILL over IP encapsulation to be used through the exchange of TRILL IS-IS Hellos is a low bandwidth process. Hellos are not permitted to be sent any more often than once per second, and so are unlikely to cause congestion. One motivation for including UDP in a TRILL encapsulation is to improve the use of multipath (such as ECMP) in cases where traffic is to traverse routers which are able to hash on UDP Port and IP address. In many cases this may reduce the occurrence of congestion and improve usage of available network capacity. However, it is also necessary to ensure that the network, including applications that use the network, responds appropriately in more difficult cases, such as when link or equipment failures have reduced the available capacity. The impact of congestion must be considered both in terms of the effect on the rest of the network of a UDP tunnel that is consuming excessive capacity, and in terms of the effect on the flows using the UDP tunnels. The potential impact of congestion from a UDP tunnel depends upon what sort of traffic is carried over the tunnel, as well as the path of the tunnel. TRILL is used to carry a wide range of traffic. In many cases TRILL is used to carry IP traffic. IP traffic is generally assumed to be congestion controlled, and thus a tunnel carrying general IP traffic (as might be expected to be carried across the Internet) generally does not need additional congestion control mechanisms. As specified in [RFC5405]: "IP-based traffic is generally assumed to be congestion- controlled, i.e., it is assumed that the transport protocols generating IP-based traffic at the sender already employ mechanisms that are sufficient to address congestion on the path. Consequently, a tunnel carrying IP-based traffic should already interact appropriately with other traffic sharing the path, and specific congestion control mechanisms for the tunnel are not necessary". For this reason, where TRILL is sent using UDP and used to carry IP traffic that is known to be congestion controlled, the UDP paths MAY Margaret Cullen, et al [Page 21] INTERNET-DRAFT TRILL over IP be used across any combination of a single or cooperating service providers or across the general Internet. However, TRILL is also used to carry traffic that is not necessarily congestion controlled. For example, TRILL may be used to carry traffic where specific bandwidth guarantees are provided. In such cases congestion may be avoided by careful provisioning of the network and/or by rate limiting of user data traffic. Where TRILL is carried, directly or indirectly, over UDP over IP, the identity of each individual TRILL flow is in general lost. For this reason, where the TRILL traffic is not congestion controlled, TRILL over UDP/IP MUST only be used within a single service provider that utilizes careful provisioning (e.g., rate limiting at the entries of the network while over-provisioning network capacity) to ensure against congestion, or within a limited number of service providers who closely cooperate in order to jointly provide this same careful provisioning. As such, TRILL over UDP/IP MUST NOT be used over the general Internet, or over non-cooperating service providers, to carry traffic that is not congestion- controlled. Measures SHOULD be taken to prevent non-congestion-controlled TRILL over UDP/IP traffic from "escaping" to the general Internet, for example the following: a. Physical or logical isolation of the TRILL over IP links from the general Internet. b. Deployment of packet filters that block the UDP ports assigned for TRILL-over-UDP. c. Imposition of restrictions on TRILL over UDP/IP traffic by software tools used to set up TRILL over UDP paths between specific end systems (as might be used within a single data center). d. Use of a "Managed Circuit Breaker" for the TRILL traffic as described in [circuit-breaker]. 10.4 MTU Considerations In TRILL each RBridge advertises in its LSP number zero the largest LSP frame it can accept (but not less than 1,470 bytes) on any of its interfaces (at least those interfaces with adjacencies to other RBridges in the campus) through the originatingLSPBufferSize TLV [RFC6325] [RFC7177]. The campus minimum MTU, denoted Sz, is then Margaret Cullen, et al [Page 22] INTERNET-DRAFT TRILL over IP established by taking the minimum of this advertised MTU for all RBridges in the campus. Links that do not meet the Sz MTU are not included in the routing topology. This protects the operation of IS- IS from links that would be unable to accommodate some LSPs. A method of determining originatingLSPBufferSize for an RBridge with one or more TRILL over IP ports is described in [rfc7180bis]. However, if an IP link either can accommodate jumbo frames or is a link on which IP fragmentation is enabled and acceptable, then it is unlikely that the IP link will be a constraint on the originatingLSPBufferSize of an RBridge using the link. On the other hand, if the IP link can only handle smaller frames and fragmentation is to be avoided when possible, a TRILL over IP port might constrain the RBridge's originatingLSPBufferSize. Because TRILL sets the minimum values of Sz at 1,470 bytes, there may be links that meet the minimum MTU for the IP protocol (1,280 bytes for IPv6, theoretically 68 bytes for IPv4) on which it would be necessary to enable fragmentation for TRILL use. The optional use of TRILL IS-IS MTU PDUs, as specified in [RFC6325] and [RFC7177] can provide added assurance of the actual MTU of a link. 10.5 QoS Considerations Within TRILL, priority is indicated by a three bit (0 through 7) priority field in TRILL data packets and by configuration for TRILL IS-IS packets. When TRILL packets are sent on a TRILL over IP link, this priority is mapped to a Differential Services Code Point (DSCP [RFC2474] Section 4.2.2). The default mapping, which may be configured per TRILL over IP port, is an follows. Note that, to provide a potentially lower priority service than the default 0, priority 1 is considered lower priority than 0. So the priority squence from lower to higher priority is 1, 0, 2, 3, 4, 5, 6, 7. TRILL Priority DiffServ Field (Binary/decimal) -------------- ------------------------------- 0 00100000 / 8 1 00000000 / 0 2 01000000 / 16 3 01100000 / 24 4 10000000 / 32 5 10100000 / 40 6 11000000 / 48 7 11100000 / 56 Margaret Cullen, et al [Page 23] INTERNET-DRAFT TRILL over IP 11. Middlebox Considerations TBD ... Margaret Cullen, et al [Page 24] INTERNET-DRAFT TRILL over IP 12. Security Considerations TRILL over IP is subject to all of the security considerations for the base TRILL protocol [RFC6325]. In addition, there are specific security requirements for different TRILL deployment scenarios, as discussed in the "Use Cases for TRILL over IP" section above. This document specifies that all RBridges that support TRILL over IP MUST implement IPsec, and makes it clear that it is both wise and good to use IPsec in all cases where a TRILL over IP link will traverse a network that is not under the same administrative control as the rest of the TRILL campus or is not physically secure. IPsec is necessary, in these cases to protect the privacy and integrity of data traffic. TRILL over IP is compatible with the use of IS-IS Security [RFC5310], which can be used to authenticate RBridges before allowing them to join a TRILL campus. This is sufficient to protect against rogue RBridges, but is not sufficient to protect data packets that may be sent in IP outside of the local network, or even across the public Internet. To protect the privacy and integrity of that traffic, use IPsec. In cases were IPsec is used, the use of IS-IS security may not be necessary, but there is nothing about this specification that would prevent using both IPsec and IS-IS security together. In cases where both types of security are enabled, by default, a key derived from the IS-IS key will be used for IPsec. Margaret Cullen, et al [Page 25] INTERNET-DRAFT TRILL over IP 13. IANA Considerations IANA considerations are given below. 13.1 Port Assignments IANA has allocated the following destination UDP Ports for the TRILL IS-IS and Data channels: UDP Port Protocol ---------- --------------------- (TBD1) TRILL IS-IS Channel (TBD2) TRILL Data Channel 13.2 Multicast Address Assignments IANA has allocated one IPv4 and one IPv6 multicast address, as shown below, which correspond to the All-RBridges and All-IS-IS-RBridges multicast MAC addresses that the IEEE Registration Authority has assigned for TRILL. Because the low level hardware MAC address dispatch considerations for TRILL over Ethernet do not apply to TRILL over IP, one IP multicast address for each version of IP is sufficient. (Values recommended to IANA in square brackets) Name IPv4 IPv6 ------------ ------------------ -------------------------- All-RBridges TBD3[233.252.14.0] TBD4[FF0X:0:0:0:0:0:0:205] Note: when these IPv4 and IPv6 multicast addresses are used and the resulting IP frame is sent over Ethernet, the usual IP derived MAC address is used. [Need to discuss scopes for IPv6 multicast (the "X" in the addresses) somewhere. Default to "site" scope but MUST be configurable?] 13.3 Encapsulation Method Support Indication The existing "RBridge Channel Protocols" registry is re-named and a new sub-registry under that registry added as follows: The TRILL Parameters registry for "RBridge Channel Protocols" is renamed the "RBridge Channel Protocols and Link Technology Flags" Margaret Cullen, et al [Page 26] INTERNET-DRAFT TRILL over IP registry. [this document] is added as a second reference for this registry. The first part of the table is changed to the following: Range Registration Note ----------- ---------------- ---------------------------- 0x002-0x0FF Standards Action 0x100-0xFBF RFC Required allocation of a single value 0x100-0xFBF IESG Approval allocation of multiple values 0xFC0 0xFF7 see Note link technology dependent, see subregistry In the existing table of RBridge Channel Protocols, the following line is changed to two lines as shown: OLD 0x004-0xFF7 Unassigned NEW 0x004-0xFBF Unassigned 0xFC0-0xFF7 (link technology dependent, see subregistry) A new subregistry under the newly named "RBridge Channel Protocols and Link Technology Flags" registry is added as follows: Name: TRILL over IP Link Flags Registration Procedure: IETF Review Reference: [this document] Flag Meaning ----------- ------------------------------ 0xFC0 Native encapsulation supported 0xFC1 VXLAN encapsulation supported 0xFC2-0xFF7 Unassigned Margaret Cullen, et al [Page 27] INTERNET-DRAFT TRILL over IP Normative References [IS-IS] - "Intermediate system to Intermediate system routeing information exchange protocol for use in conjunction with the Protocol for providing the Connectionless-mode Network Service (ISO 8473)", ISO/IEC 10589:2002, 2002". [RFC0020] - Cerf, V., "ASCII format for network interchange", STD 80, RFC 20, DOI 10.17487/RFC0020, October 1969, . [RFC0768] - Postel, J., "User Datagram Protocol", STD 6, RFC 768, DOI 10.17487/RFC0768, August 1980, . [RFC2119] - Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC2474] - Nichols, K., Blake, S., Baker, F., and D. Black, "Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers", RFC 2474, DOI 10.17487/RFC2474, December 1998, . [RFC2710] - Deering, S., Fenner, W., and B. Haberman, "Multicast Listener Discovery (MLD) for IPv6", RFC 2710, DOI 10.17487/RFC2710, October 1999, . [RFC2914] - Floyd, S., "Congestion Control Principles", BCP 41, RFC 2914, DOI 10.17487/RFC2914, September 2000, . [RFC3376] - Cain, B., Deering, S., Kouvelas, I., Fenner, B., and A. Thyagarajan, "Internet Group Management Protocol, Version 3", RFC 3376, DOI 10.17487/RFC3376, October 2002, . [RFC3566] - Frankel, S. and H. Herbert, "The AES-XCBC-MAC-96 Algorithm and Its Use With IPsec", RFC 3566, DOI 10.17487/RFC3566, September 2003, . [RFC3602] - Frankel, S., Glenn, R., and S. Kelly, "The AES-CBC Cipher Algorithm and Its Use with IPsec", RFC 3602, DOI 10.17487/RFC3602, September 2003, . [RFC4301] - Kent, S. and K. Seo, "Security Architecture for the Internet Protocol", RFC 4301, DOI 10.17487/RFC4301, December Margaret Cullen, et al [Page 28] INTERNET-DRAFT TRILL over IP 2005, . [RFC4303] - Kent, S., "IP Encapsulating Security Payload (ESP)", RFC 4303, DOI 10.17487/RFC4303, December 2005, . [RFC4308] - Hoffman, P., "Cryptographic Suites for IPsec", RFC 4308, DOI 10.17487/RFC4308, December 2005, . [RFC5405] - Li, T. and R. Atkinson, "IS-IS Cryptographic Authentication", RFC 5304, DOI 10.17487/RFC5304, October 2008, . [RFC5310] - Bhatia, M., Manral, V., Li, T., Atkinson, R., White, R., and M. Fanto, "IS-IS Generic Cryptographic Authentication", RFC 5310, DOI 10.17487/RFC5310, February 2009, . [RFC6325] - Perlman, R., Eastlake 3rd, D., Dutt, D., Gai, S., and A. Ghanwani, "Routing Bridges (RBridges): Base Protocol Specification", RFC 6325, DOI 10.17487/RFC6325, July 2011, . [RFC7176] - Eastlake 3rd, D., Senevirathne, T., Ghanwani, A., Dutt, D., and A. Banerjee, "Transparent Interconnection of Lots of Links (TRILL) Use of IS-IS", RFC 7176, DOI 10.17487/RFC7176, May 2014, . [RFC7177] - Eastlake 3rd, D., Perlman, R., Ghanwani, A., Yang, H., and V. Manral, "Transparent Interconnection of Lots of Links (TRILL): Adjacency", RFC 7177, DOI 10.17487/RFC7177, May 2014, . [RFC7178] - Eastlake 3rd, D., Manral, V., Li, Y., Aldrin, S., and D. Ward, "Transparent Interconnection of Lots of Links (TRILL): RBridge Channel Support", RFC 7178, DOI 10.17487/RFC7178, May 2014, . [RFC7348] - Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger, L., Sridhar, T., Bursell, M., and C. Wright, "Virtual eXtensible Local Area Network (VXLAN): A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks", RFC 7348, DOI 10.17487/RFC7348, August 2014, . [rfc7180bis] - Eastlake, D., et al, "TRILL: Clarifications, Corrections, and Updates", draft-ietf-trill-rfc7180bis, work in progress. Margaret Cullen, et al [Page 29] INTERNET-DRAFT TRILL over IP [FIPS180] - "Secure Hash Standard (SHS)", United States of American, National Institute of Science and Technology, Federal Information Processing Standard (FIPS) 180-4, March 2012. Informative References [RFC6234] - Eastlake 3rd, D. and T. Hansen, "US Secure Hash Algorithms (SHA and SHA-based HMAC and HKDF)", RFC 6234, DOI 10.17487/RFC6234, May 2011, . [RFC6361] - Carlson, J. and D. Eastlake 3rd, "PPP Transparent Interconnection of Lots of Links (TRILL) Protocol Control Protocol", RFC 6361, DOI 10.17487/RFC6361, August 2011, . [RFC7172] - Eastlake 3rd, D., Zhang, M., Agarwal, P., Perlman, R., and D. Dutt, "Transparent Interconnection of Lots of Links (TRILL): Fine-Grained Labeling", RFC 7172, DOI 10.17487/RFC7172, May 2014, . [RFC7173] - Yong, L., Eastlake 3rd, D., Aldrin, S., and J. Hudson, "Transparent Interconnection of Lots of Links (TRILL) Transport Using Pseudowires", RFC 7173, DOI 10.17487/RFC7173, May 2014, . [gre-in-udp] - Crabbe, E., Yong, L., and X. Xu, "Generic UDP Encapsulation for IP Tunneling", draft-yong-tsvwg-gre-in-udp- encap, work in progress. [circuit-breaker] - Fairhurst, G., "Network Transport Circuit Breakers", draft-ietf-tsvwg-circuit-breaker, work in progress. Margaret Cullen, et al [Page 30] INTERNET-DRAFT TRILL over IP Acknowledgements The following people have provided useful feedback on the contents of this document: Sam Hartman, Adrian Farrel. Some material in Section 10.2 is derived from draft-ietf-mpls-in-udp by Xiaohu Xu, Nischal Sheth, Lucy Yong, Carlos Pignataro, and Yongbing Fan. The document was prepared in raw nroff. All macros used were defined within the source file. Margaret Cullen, et al [Page 31] INTERNET-DRAFT TRILL over IP Authors' Addresses Margaret Cullen Painless Security 356 Abbott Street North Andover, MA 01845 USA Phone: +1 781 405-7464 Email: margaret@painless-security.com URI: http://www.painless-security.com Donald Eastlake Huawei Technologies 155 Beaver Street Milford, MA 01757 USA Phone: +1 508 333-2270 Email: d3e3e3@gmail.com Mingui Zhang Huawei Technologies No.156 Beiqing Rd. Haidian District, Beijing 100095 P.R. China EMail: zhangmingui@huawei.com Dacheng Zhang Alibaba Beijing, Chao yang District P.R. China Email: dacheng.zdc@alibaba-inc.com Copyright, Disclaimer, and Additional IPR Provisions Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect Margaret Cullen, et al [Page 32] INTERNET-DRAFT TRILL over IP to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Margaret Cullen, et al [Page 33]