idnits 2.17.1 draft-schwartz-tls-lb-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 31, 2019) is 1638 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-18) exists of draft-ietf-tls-esni-04 ** Downref: Normative reference to an Informational draft: draft-kinnear-tls-client-net-address (ref. 'I-D.kinnear-tls-client-net-address') Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 tls B. Schwartz 3 Internet-Draft Google LLC 4 Intended status: Standards Track October 31, 2019 5 Expires: May 3, 2020 7 TLS Metadata for Load Balancers 8 draft-schwartz-tls-lb-02 10 Abstract 12 A load balancer that does not terminate TLS may wish to provide some 13 information to the backend server, in addition to forwarding TLS 14 data. This draft proposes a protocol between load balancers and 15 backends that enables secure, efficient delivery of TLS with 16 additional information. The need for such a protocol has recently 17 become apparent in the context of split mode ESNI. 19 Status of This Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF). Note that other groups may also distribute 26 working documents as Internet-Drafts. The list of current Internet- 27 Drafts is at https://datatracker.ietf.org/drafts/current/. 29 Internet-Drafts are draft documents valid for a maximum of six months 30 and may be updated, replaced, or obsoleted by other documents at any 31 time. It is inappropriate to use Internet-Drafts as reference 32 material or to cite them other than as "work in progress." 34 This Internet-Draft will expire on May 3, 2020. 36 Copyright Notice 38 Copyright (c) 2019 IETF Trust and the persons identified as the 39 document authors. All rights reserved. 41 This document is subject to BCP 78 and the IETF Trust's Legal 42 Provisions Relating to IETF Documents 43 (https://trustee.ietf.org/license-info) in effect on the date of 44 publication of this document. Please review these documents 45 carefully, as they describe your rights and restrictions with respect 46 to this document. Code Components extracted from this document must 47 include Simplified BSD License text as described in Section 4.e of 48 the Trust Legal Provisions and are provided without warranty as 49 described in the Simplified BSD License. 51 Table of Contents 53 1. Conventions and Definitions . . . . . . . . . . . . . . . . . 2 54 2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 2 55 3. Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 56 4. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 4 57 5. Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . 4 58 6. Defined ProxyExtensions . . . . . . . . . . . . . . . . . . . 6 59 6.1. padding . . . . . . . . . . . . . . . . . . . . . . . . . 6 60 6.2. client_address . . . . . . . . . . . . . . . . . . . . . 6 61 6.3. destination_address . . . . . . . . . . . . . . . . . . . 6 62 6.4. esni_inner . . . . . . . . . . . . . . . . . . . . . . . 6 63 6.5. certificate_padding . . . . . . . . . . . . . . . . . . . 7 64 6.6. overload . . . . . . . . . . . . . . . . . . . . . . . . 7 65 6.7. ratchet . . . . . . . . . . . . . . . . . . . . . . . . . 8 66 7. Protocol wire format . . . . . . . . . . . . . . . . . . . . 9 67 8. Security considerations . . . . . . . . . . . . . . . . . . . 10 68 8.1. Integrity . . . . . . . . . . . . . . . . . . . . . . . . 10 69 8.2. Confidentiality . . . . . . . . . . . . . . . . . . . . . 10 70 8.3. Fingerprinting . . . . . . . . . . . . . . . . . . . . . 11 71 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 72 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 73 10.1. Normative References . . . . . . . . . . . . . . . . . . 11 74 10.2. Informative References . . . . . . . . . . . . . . . . . 12 75 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 12 76 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 12 78 1. Conventions and Definitions 80 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 81 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 82 "OPTIONAL" in this document are to be interpreted as described in BCP 83 14 [RFC2119] [RFC8174] when, and only when, they appear in all 84 capitals, as shown here. 86 Data encodings are expressed in the TLS 1.3 presentation language, as 87 defined in Section 3 of [TLS13]. 89 2. Background 91 A load balancer is a server or bank of servers that acts as an 92 intermediary between the client and a range of backend servers. As 93 the name suggests, a load balancer's primary function is to ensure 94 that client traffic is spread evenly across the available backend 95 servers. However load balancers also serve many other functions, 96 such as identifying connections intended for different backends and 97 forwarding them appropriately, or dropping connections that are 98 deemed malicious. 100 A load balancer operates at a specific point in the protocol stack, 101 forwarding e.g. IP packets, TCP streams, TLS contents, HTTP 102 requests, etc. Most relevant to this proposal are TCP and TLS load 103 balancers. TCP load balancers terminate the TCP connection with the 104 client and establish a new TCP connection to the selected backend, 105 bidirectionally copying the TCP contents between these two 106 connections. TLS load balancers additionally terminate the TLS 107 connection, forwarding the plaintext to the backend server (typically 108 inside a new TLS connection). TLS load balancers must therefore hold 109 the private keys for the domains they serve. 111 When a TCP load balancer forwards a TLS stream, the load balancer has 112 no way to incorporate additional information into the stream. 113 Insertion of any additional data would cause the connection to fail. 114 However, the load-balancer and backend can share additional 115 information if they agree to speak a new protocol. The most popular 116 protocol used for this purpose is currently the PROXY protocol 117 [PROXY], developed by HAPROXY. This protocol prepends a plaintext 118 collection of metadata (e.g. client IP address) onto the TCP socket. 119 The backend can parse this metadata, then pass the remainder of the 120 stream to its TLS library. 122 The PROXY protocol is effective and widely used, but it offers no 123 confidentiality or integrity protection, and therefore might not be 124 suitable when the load balancer and backend communicate over the 125 public internet. It also does not offer a way for the backend to 126 reply. 128 3. Goals 130 o Enable TCP load balancers to forward metadata to the backend. 132 o Enable backends to reply. 134 o Reduce the need for TLS-terminating load balancers. 136 o Ensure confidentiality and integrity for all forwarded metadata. 138 o Enable split ESNI architectures. 140 o Prove to the backend that the load balancer intended to associate 141 this metadata with this connection. 143 o Achieve good CPU and memory efficiency. 145 o Don't impose additional latency. 147 o Support backends that receive a mixture of direct and load- 148 balanced TLS. 150 o Enable simple and safe implementation. 152 4. Overview 154 The proposed protocol supports a two-way exchange between a load 155 balancer and a backend server. It works by prepending information to 156 the TLS handshake: 158 +-----------+ +-----------+ +-----------+ 159 | Backend A | | Backend B | | Backend C | 160 +-----------+ +-----------+ +-----------+ 161 \/ /\ 162 4. EncryptedProxyData[ \/ /\ 3. ClientHello (verbatim) 163 got SNI info] \/ /\ 2. EncryptedProxyData[ 164 5. ServerHello, etc. \/ /\ SNI="secret.b", 165 \/ /\ client=2, etc.] 166 \/ /\ 167 +---------------+ 168 | Load balancer | 169 +---------------+ 170 \/ /\ 171 6. ServerHello, etc. \/ /\ 1. ClientHello[ 172 (verbatim) \/ /\ ESNI=enc("secret.b")] 173 \/ /\ 174 +-----------+ +-----------+ +-----------+ 175 | Client 1 | | Client 2 | | Client 3 | 176 +-----------+ +-----------+ +-----------+ 178 Figure 1: Data flow diagram 180 5. Encoding 182 A ProxyExtension is identical in form to a standard TLS Extension 183 (Section 4.2 of [TLS13]), with a new identifier space for the 184 extension types. 186 struct { 187 ProxyExtensionType extension_type; 188 opaque extension_data<0..2^16-1>; 189 } ProxyExtension; 191 ProxyExtensions can be sent in an upstream (to the backend) or 192 downstream (to the load balancer) direction 193 enum { 194 upstream(0), 195 downstream(1), 196 (255) 197 } ProxyDataDirection; 199 The ProxyData contains a set of ProxyExtensions. 201 struct { 202 ProxyDataDirection direction; 203 ProxyExtension proxy_data<0..2^16-1>; 204 } ProxyData; 206 The EncryptedProxyData structure contains metadata associated with 207 the original ClientHello (Section 4.1.2 of [TLS13]), encrypted with a 208 pre-shared key that is configured out of band. 210 struct { 211 opaque psk_identity<1..2^16-1>; 212 opaque nonce<8..2^16-1> 213 opaque encrypted_proxy_data<1..2^16-1>; 214 } EncryptedProxyData; 216 o "psk_identity": The identity of a PSK previously agreed upon by 217 the load balancer and the backend. Including the PSK identity 218 allows for updating the PSK without disruption. 220 o "nonce": Non-repeating initializer for the AEAD. This prevents an 221 attacker from observing whether the same ClientHello is marked 222 with different metadata over time. 224 o "encrypted_proxy_data": "AEAD-Encrypt(key, nonce, additional_data, 225 plaintext=ProxyData)". The key and AEAD function are agreed out 226 of band and associated with "psk_identity". The "additional_data" 227 is context-dependent. 229 When the load balancer receives a ClientHello, it serializes any 230 relevant metadata into an upstream ProxyData, then encrypts it with 231 the ClientHello as "additional_data" to produce the 232 EncryptedProxyData. The backend's reply is a downstream ProxyData 233 struct, also transmitted as an EncryptedProxyData, using the upstream 234 EncryptedProxyData as "additional_data". Recipients in each case 235 MUST verify that "ProxyData.direction" has the expected value, and 236 discard the connection if it does not. 238 The downstream ProxyData SHOULD NOT contain any ProxyExtensionType 239 values that were not present in the upstream ProxyData. 241 6. Defined ProxyExtensions 243 Like a standard TLS Extension, a ProxyExtension is identified by a 244 uint16 type number. Load balancers MUST only include extensions that 245 are registered for use in ProxyData. Backends MUST ignore any 246 extensions that they do not recognize. 248 There are initially seven type numbers allocated: 250 enum { 251 padding(0), 252 client_address(1), 253 destination_address(2), 254 esni_inner(3), 255 certificate_padding(4), 256 overload(5), 257 ratchet(6), 258 (65535) 259 } ProxyExtensionType; 261 6.1. padding 263 The "padding" extension functions as described in [RFC7685]. It is 264 used here to avoid leaking information about the other extensions. 265 It can be used in upstream and downstream ProxyData. 267 6.2. client_address 269 The "client_address" extension functions as described in 270 [I-D.kinnear-tls-client-net-address]. It conveys the client IP 271 address observed by the load balancer. Backends that make use of 272 this extension SHOULD include an empty "client_address" extension in 273 the downstream ProxyData. 275 6.3. destination_address 277 The "destination_address" extension is identical to the 278 "client_address" extension, except that it contains the load 279 balancer's server IP address that received this connection. 281 6.4. esni_inner 283 The "esni_inner" extension is only sent upstream, and can only be 284 used if the ClientHello contains the encrypted_server_name extension 285 [ESNI]. The "extension_data" is the ClientESNIInner (Section 5.1.1 286 of [ESNI]), which contains the true SNI and nonce. This is useful 287 when the load balancer knows the ESNI private key and the backend 288 does not, i.e. split mode ESNI. 290 6.5. certificate_padding 292 The "certificate_padding" extension always contains a single uint32 293 value. The upstream value conveys the padding granularity "G", and 294 the downstream value indicates the unpadded size of the Certificate 295 struct (Section 4.4.2 of [TLS13]). 297 To pad the Handshake message (Section 4 of [TLS13]) containing the 298 Certificate struct, the backend SHOULD select the smallest 299 "length_of_padding" (Section 5.2 of [TLS13]) such that 300 "Handshake.length + length_of_padding" is a multiple of "G". 302 The load balancer SHOULD include this extension whenever it sends the 303 "esni_inner" extension. 305 Padding certificates from many backends to the same length is 306 important to avoid revealing which backend is responding to a 307 ClientHello. Load balancer operators SHOULD ensure that no backend 308 has a unique certificate size after padding, and MAY set "G" large 309 enough to make all responses have equal size. 311 6.6. overload 313 In the upstream ProxyData, the "overload" extension contains a single 314 uint16 indicating the approximate proportion of connections that are 315 being routed to this server as a fraction of 65535. If there is only 316 one server, load balancers SHOULD set the value to 65535. 318 In the downstream ProxyData, the value is an OverloadValue: 320 enum { 321 accepted(0), 322 overloaded(1), 323 rejected(2), 324 (255) 325 } OverloadState; 326 struct { 327 OverloadState state; 328 uint16 load; 329 uint32 ttl; 330 } OverloadValue; 332 When "OverloadValue.state" is "accepted", the backend is accepting 333 connections normally. The "overloaded" state indicates that the 334 backend is accepting this connection, but would prefer not to receive 335 additional connections. A value of "rejected" indicates that the 336 backend did not accept this connection. When sending a "rejected" 337 response, the backend SHOULD close the connection without sending a 338 ServerHello. 340 "OverloadValue.load" indicates the load fraction of the responding 341 backend server, with 65535 indicating maximum load. 343 The load balancer SHOULD treat this information as valid for 344 "OverloadValue.ttl" seconds, or until it receives another 345 OverloadValue from that server. 347 Load balancers that have multiple available backends for an origin 348 SHOULD avoid connecting to servers that are in the "overloaded" or 349 "rejected" state. When a connection is rejected, the load balancer 350 MAY retry that connection by sending the ClientHello to a different 351 backend server. When multiple servers are in the "accepted" state, 352 the load balancer MAY use "OverloadValue.load" to choose among them. 354 When there is a server in an unknown state (i.e. a new server or one 355 whose last TTL has expired), the load balancer SHOULD direct at least 356 one connection to it, in order to refresh its OverloadState. 358 If all servers are in the "overloaded" or "rejected" state, the load 359 balancer SHOULD drop the connection. 361 6.7. ratchet 363 If the backend server is reachable without traversing the load 364 balancer, and an adversary can observe packets on the link between 365 the load balancer and the backend, then that adversary can execute a 366 replay flooding attack, sending the backend server duplicate copies 367 of observed EncryptedProxyData and ClientHello. This attack can 368 waste server resources on the Diffie-Hellman operations required to 369 process the ClientHello, resulting in denial of service. 371 The "ratchet" extension reduces the impact of such an attack on the 372 backend server by allowing the backend to reject these duplicates 373 after decrypting the ProxyData. (This decryption uses only a 374 symmetric cipher, so it is expected to be much faster than typical 375 Diffie-Hellman operations.) Its upstream payload consists of a 376 RatchetValue: 378 struct { 379 uint64 index; 380 uint64 floor; 381 } RatchetValue; 382 A RatchetValue is scoped to a single backend server and 383 "psk_identity". Within that scope, the load balancer initializes 384 "index" to a random value, and executes the following procedure: 386 1. For each new forwarded connection (to the same server under the 387 same "psk_identity"), increment "index". 389 2. Set "floor" to the "index" of the earliest connection that has 390 not yet been connected or closed. 392 The backend server initializes "floor" to the first 393 "RatchetValue.floor" it receives (under a "psk_identity"), and then 394 executes the following procedure for each incoming connection: 396 1. Define "a >= b" if the most significant bit of "a - b" is 0. 398 2. Let "newValue" be the RatchetValue in the ProxyData. 400 3. If "newValue.index < floor", ignore the connection. 402 4. If "newValue.floor >= floor", set "floor" to "newValue.floor". 404 5. OPTIONALLY, ignore the connection if "newValue.index" has been 405 seen recently. This can be implemented efficiently by keeping 406 track of any "index" values greater than "floor" that appear to 407 have been skipped. 409 With these measures in place, replays can be rejected without 410 processing the ClientHello. 412 In principle, this replay protection fails after 2^64 connections 413 when the "floor" value wraps. On a backend server that averages 10^9 414 new connections per second, this would occur after 584 years. To 415 avoid this replay attack, load balancers and backends SHOULD 416 establish a new PSK at least this often. 418 Backends that are making use of the "ratchet" extension SHOULD 419 include an empty "ratchet" extension in their downstream ProxyData. 421 7. Protocol wire format 423 When forwarding a TLS stream over TCP, the load balancer SHOULD 424 prepend a TLSPlaintext whose "content_type" is XX (proxy_header) and 425 whose "fragment" is the EncryptedProxyData. 427 Following this proxy header, the load balancer MUST send the full 428 contents of the TCP stream, exactly as received from the client. The 429 backend will observe the proxy header, immediately followed by a 430 TLSPlaintext containing the ClientHello. The backend will decrypt 431 the EncryptedProxyData using the ClientHello as associated data, and 432 process the ClientHello and the remainder of the stream as standard 433 TLS. 435 Similarly, the backend SHOULD reply with the downstream 436 EncryptedProxyData in a proxy header, followed by the normal TLS 437 stream, beginning with a TLSPlaintext frame containing the 438 ServerHello. If the downstream ProxyHeader is not present, has an 439 unrecognized version number, or produces an error, the load balancer 440 SHOULD proxy the rest of the stream regardless. 442 8. Security considerations 444 8.1. Integrity 446 This protocol is intended to provide both parties with a strong 447 guarantee of integrity for the metadata they receive. For example, 448 an active attacker cannot take metadata intended for one stream and 449 attach it to another, because each stream will have a unique 450 ClientHello, and the metadata is bound to the ClientHello by AEAD. 452 One exception to this protection is in the case of an attacker who 453 deliberately reissues identical ClientHello messages. An attacker 454 who reuses a ClientHello can also reuse the metadata associated with 455 it, if they can first observe the EncryptedProxyData transferred 456 between the load balancer and the backend. This could be used by an 457 attacker to reissue data originally generated by a true client (e.g. 458 as part of a 0-RTT replay attack), or it could be used by a group of 459 adversaries who are willing to share a single set of client secrets 460 while initiating different sessions, in order to reuse metadata that 461 they find helpful. 463 Backends that are sensitive to this attack SHOULD implement the 464 "ratchet" mechanism in Section 6.7, including the optional defenses. 466 8.2. Confidentiality 468 This protocol is intended to maintain confidentiality of the metadata 469 transferred between the load balancer and backend, especially the 470 ESNI plaintext and the client IP address. An observer between the 471 client and the load balancer does not observe this protocol at all, 472 and an observer between the load balancer and backend observes only 473 ciphertext. 475 However, an adversary who can monitor both of these links can easily 476 observe that a connection from the client to the load balancer is 477 shortly followed by a connection from the load balancer to a backend, 478 with the same ClientHello. This reveals which backend server the 479 client intended to visit. In many cases, the choice of backend 480 server could be the sensitive information that ESNI is intended to 481 protect. 483 8.3. Fingerprinting 485 Connections to different domains might be distinguishable by the 486 cleartext contents of the ServerHello, such as "cipher_suite" and 487 "server_share.group". Load balancer operators with ESNI support 488 SHOULD provide backend operators with a list of cipher suites and 489 groups to support, and a preference order, to avoid different 490 backends having distinctive behaviors. 492 9. IANA Considerations 494 IANA will be directed to add the following allocation to the TLS 495 ContentType registry: 497 +-------+--------------+---------+---------------+ 498 | Value | Description | DTLS-OK | Reference | 499 +-------+--------------+---------+---------------+ 500 | XX | proxy_header | N | This document | 501 +-------+--------------+---------+---------------+ 503 IANA will be directed to create a new "TLS ProxyExtensionType Values" 504 registry on the TLS Extensions page. Values less than 0x8000 will be 505 subject to the "RFC Required" registration procedure, and the rest 506 will be "First Come First Served". To avoid codepoint exhaustion, 507 proxy developers SHOULD pack all their nonstandard information into a 508 single ProxyExtension. 510 10. References 512 10.1. Normative References 514 [ESNI] Rescorla, E., Oku, K., Sullivan, N., and C. Wood, 515 "Encrypted Server Name Indication for TLS 1.3", draft- 516 ietf-tls-esni-04 (work in progress), July 2019. 518 [I-D.kinnear-tls-client-net-address] 519 Kinnear, E., Pauly, T., and C. Wood, "TLS Client Network 520 Address Extension", draft-kinnear-tls-client-net- 521 address-00 (work in progress), March 2019. 523 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 524 Requirement Levels", BCP 14, RFC 2119, 525 DOI 10.17487/RFC2119, March 1997, 526 . 528 [RFC7685] Langley, A., "A Transport Layer Security (TLS) ClientHello 529 Padding Extension", RFC 7685, DOI 10.17487/RFC7685, 530 October 2015, . 532 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 533 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 534 May 2017, . 536 [TLS13] Rescorla, E., "The Transport Layer Security (TLS) Protocol 537 Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018, 538 . 540 10.2. Informative References 542 [PROXY] Tarreau, W., "The PROXY protocol", March 2017, 543 . 546 Appendix A. Acknowledgements 548 This is an elaboration of an idea proposed by Eric Rescorla during 549 the development of ESNI. Thanks to David Schinazi, David Benjamin, 550 and Piotr Sikora for suggesting important improvements. 552 Author's Address 554 Benjamin M. Schwartz 555 Google LLC 557 Email: bemasc@google.com