idnits 2.17.1 draft-oliver-pluggable-transports-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) == There are 1 instance of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 08, 2019) is 1754 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 2 errors (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group B. Wiley 3 Internet-Draft Operator Foundation 4 Intended status: Informational D. Oliver 5 Expires: January 9, 2020 Guardian Project 6 July 08, 2019 8 Enabling Network Traffic Obfuscation - Pluggable Transports 9 draft-oliver-pluggable-transports-00 11 Abstract 13 Pluggable Transports (PTs) are a mechanism enabling the rapid 14 development and deployment of network traffic obfuscation techniques 15 used to circumvent surveillance and censorship. This specification 16 does not define or limit the techniques themselves, but rather 17 focuses on the startup, shutdown, and inter-process communication 18 mechanisms required to make these technologies interoperable with 19 applications. 21 This document is based heavily on [PT2.1]. 23 Status of This Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at https://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on January 9, 2020. 40 Copyright Notice 42 Copyright (c) 2019 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (https://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 58 2. Conventions and Definitions . . . . . . . . . . . . . . . . . 3 59 3. Background . . . . . . . . . . . . . . . . . . . . . . . . . 3 60 4. Architecture Overview . . . . . . . . . . . . . . . . . . . . 4 61 5. Specification . . . . . . . . . . . . . . . . . . . . . . . . 6 62 5.1. Pluggable Transport Naming . . . . . . . . . . . . . . . 6 63 5.2. Transports API Interface . . . . . . . . . . . . . . . . 6 64 5.2.1. Goals for interface design . . . . . . . . . . . . . 6 65 5.2.2. Abstract Interfaces . . . . . . . . . . . . . . . . . 7 66 6. Adapters . . . . . . . . . . . . . . . . . . . . . . . . . . 8 67 6.1. API to IPC Adapter . . . . . . . . . . . . . . . . . . . 8 68 6.2. PT 1.0 Compatibility . . . . . . . . . . . . . . . . . . 9 69 6.3. Cross-language Linking . . . . . . . . . . . . . . . . . 9 70 6.3.1. Using the Dispatcher IPC Interface In-process . . . . 9 71 6.4. Anonymity Considerations . . . . . . . . . . . . . . . . 10 72 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 73 7.1. Normative References . . . . . . . . . . . . . . . . . . 10 74 7.2. Informative References . . . . . . . . . . . . . . . . . 10 75 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 12 76 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 12 78 1. Introduction 80 The increased interest in network traffic obfuscation technologies 81 mirrors the increase in usage of Deep Packet Inspection (DPI) to 82 actively monitor the content of application data in addition to that 83 data's routing information. Deep Packet Inspection inspects each 84 packet based on the header of its request and the data it carries. 85 It can identify the type of protocol the connection is using even if 86 it was encrypted. DPI is not a mechanism to decrypt what is inside 87 packets but to identify the 'protocol' or the application it 88 represents. 90 Deep packet inspection has become the prime tool of censors and 91 surveillance entities who block, log, and/or traffic-shape access to 92 sites and services they deem undesirable. 93 As deep packet inspection has become more routine, the sophistication 94 of monitoring has increased to include active probing that 95 fingerprints and classifies application protocols. Thus, even as 96 conventional care in application design has improved (via encryption 97 and other protocol design features that encourage privacy), network 98 traffic is still under attack. 100 The techniques of network monitoring are changing and improving day 101 by day. The development of traffic obfuscation techniques that foil 102 these efforts is slowed by the lack of common agreement on how these 103 techniques are invoked, made easily interoperable with applications, 104 and deployed quickly. This specification addresses those issues. 106 This specification describes a method for decoupling protocol-level 107 obfuscation from an application's client/server code, in a manner 108 that promotes rapid development of obfuscation/circumvention tools 109 and promotes reuse across privacy tools such as VPNs and secure 110 proxies. 112 This decoupling is accomplished by utilizing helper code, either in- 113 process through a language-specific API or in a separate sub- 114 processes, that implements the necessary forward/reverse proxy 115 services that handle the censorship circumvention, with a well 116 defined and standardized configuration and management interface. Any 117 application code that implements the interfaces as specified in this 118 document will be able to use all specification-compliant Pluggable 119 Transports. 121 2. Conventions and Definitions 123 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 124 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 125 "OPTIONAL" in this document are to be interpreted as described in BCP 126 14 [RFC2119] [RFC8174] when, and only when, they appear in all 127 capitals, as shown here. 129 3. Background 131 We define an Internet censor as any network intermediary that seeks 132 to block, divert or traffic-manage Internet network connections for 133 the purpose of eliminating, frustrating and/or logging access to 134 Internet resources that have been deemed (by the censor) to be 135 undesirable (either on a temporary or permanent basis). A variety of 136 techniques are commonly applied by Internet censors to block such 137 traffic. These include: 139 1. DNS Blocking 141 2. IP Blocking 143 3. Port Blocking 144 These techniques are applicable to a connection's metadata (IP 145 routing information) and do not require inspecting the connection's 146 datastream. 148 DPI, in contrast, actually looks at the connection's datastream - 149 often, specifically, the initial data elements in the stream (or 150 within blocks of the stream). These elements of the stream can 151 contain clues as to the application-level protocol employed, even 152 when the data itself is encrypted. Through observation over time, 153 these clues ("fingerprints") can be learned by the censor and (along 154 with the routing information) used to block or divert targeted 155 traffic. 157 A defense against this type of active probing is traffic obfuscation 158 - disguising the application data itself in a manner that is less- 159 easily fingerprinted. However, in early experiments it quickly 160 became clear that repeated use of the same obfuscation technique 161 would, itself, be learned. Methods were developed by which a single 162 obfuscation technique could transform on its own TODO: cite FTE 163 proxy, ScrambleSuit, Dust. This approach proved expensive in terms 164 of computational load. Interest gathered in solving this problem and 165 as more ideas arose so to did the need for a mechanism supporting 166 rapid deploying of new obfuscation techniques. 168 While intense work on network traffic obfuscation commenced initially 169 and continues within the Tor Project (and across a wider set of 170 external parties using Tor as a vehicle for research), vendors of 171 other privacy-enhancing software (such as VPNs) quickly found their 172 products also foiled by DPI. Thus, it becomes important to see 173 transport pluggability as a mechanism implemented in a manner 174 independent of a specific product or service. The notion of 175 "Pluggable Transports" (PT) was born from these requirements. 177 4. Architecture Overview 179 The PT Server software exposes a public proxy that accepts 180 connections from PT Clients. The PT Client transforms the traffic 181 before it hits the public Internet and the PT Server reverses this 182 transformation before passing the traffic on to its next destination. 183 The PT Server directly forwards this data to the Server App, but the 184 Server App itself may itself be a proxy server and expect the 185 forwarded traffic it receives to conform to a proxy communication 186 protocol such as SOCKS or TURN. There is also an optional 187 lightweight protocol to facilitate communicating connection metadata 188 that would otherwise be lost such as the source IP address and port 189 EXTORPORT. 191 When using an in-process, language-specific API ("Transport API 192 Interface") to integrate PTs into an application on both client and 193 server, the PT Client Library is integrated directly into the Client 194 App and the PT Server Library is integrated directly into the Server 195 App. The Client App and Server App communicate through socket-like 196 APIs, with all communication between them going through the PT 197 library, which only sends transformed traffic over the public 198 Internet. 200 +------------+ +---------------------------+ 201 | Client App +-- Socket-like API --+ PT Client (Library) +--+ 202 +------------+ +---------------------------+ | 203 | 204 Public Internet (Obfuscated/Transformed traffic) ==> | 205 | 206 +------------+ +-------------------------+ | 207 | Server App +-- Socket-like API --+ PT Server (Library) +--+ 208 +------------+ +-------------------------+ 210 Figure 1. API Architecture Overview 212 When using the transports as a separate process on both client and 213 server, the Dispatcher IPC Interface is used. On the client device, 214 the PT Client software exposes a local proxy to the client 215 application, and transforms traffic before forwarding it to the PT 216 Server. The PT Dispatcher can be configured to provide different 217 proxy types, supporting proxying of both TCP and UDP traffic. 219 +------------+ +---------------------------+ 220 | Client App +---- Local Proxy ----+ PT Client (Dispatcher) +--+ 221 +------------+ +---+-------------------+---+ | 222 | PT Client Library | | 223 +-------------------+ | 224 | 225 Public Internet (Transformed/Proxied traffic) =====> | 226 | 227 +------------+ +---------------------------+ | 228 | Server App +---- Local Proxy ----+ PT Server (Dispatcher ) +--+ 229 +------------+ +---+-------------------+---+ 230 |PT Server (Library)| 231 +-------------------+ 233 Figure 2. IPC Architecture Overview 235 A PT client-server setup may also mix and match interfaces, using 236 Dispatcher IPC on one end of the connection and the Transport API on 237 the other, as below (or vice-versa): 239 +------------+ +---------------------------+ 240 | Client App +---- Local Proxy ----+ PT Dispatcher Client +-+ 241 +------------+ +---+-------------------+---+ | 242 | 243 | 244 +------------+ +--------------------------+ | 245 | Server App +-- Socket-like API --+ PT Server (Library ) +--+ 246 +------------+ +--------------------------+ 248 Figure 3. Mixed IPC and Transport API example 250 Each invocation of a PT MUST be either a client OR a server. 252 PT dispatchers MAY support any of the following proxy modes: PT 1.0 253 with SOCKS4, PT 1.0 with SOCKS5, or any of the PT 2.1 modes: 254 transparent TCP, transparent UDP, or STUN-aware UDP. Clients SHOULD 255 prefer PT 2.1 over PT 1.0. 257 5. Specification 259 5.1. Pluggable Transport Naming 261 Pluggable Transport names serve as unique identifiers, and every PT 262 MUST have a unique name. PT names MUST be valid C identifiers, which 263 means that PT names MUST begin with a letter or underscore, and the 264 remaining characters MUST be ASCII letters, numbers or underscores. 265 No length limit is imposed. PT names MUST therefore satisfy the 266 regular expression [a-zA-Z_][a-zA-Z0-9_]*. 268 5.2. Transports API Interface 270 5.2.1. Goals for interface design 272 The goal for the interface design is to achieve the following 273 properties: 275 - Transport implementers have to do the minimum amount of work in 276 addition to implementing the core transform logic. 278 - Transport users have to do the minimum amount of work to add PT 279 support to code that uses standard networking primitives from the 280 language or platform. 282 - Transports require an explicit destination address to be 283 specified. However, this can be either an explicit PT server 284 destination with the Server App is already known implicitly, or an 285 explicit Server App destination with the PT server destination 286 already known implicity. 288 - Transports may or may not generate, send, receive, store, and/or 289 update persistent or ephemeral state. 291 - Transports that do not need persistence or negotiation can 292 interact with the application through the simplest possible 293 interface 295 - Transports that do need persistence or negotiation can rely on the 296 application to provide it through the specified interface, so the 297 transport does not need to implement persistence or negotiation 298 internally. 300 - Applications should be able to use a PT Client implementation to 301 establish several independent transport connections with different 302 parameters, with a minimum of complexity and latency. 304 - The interface in each language should be idiomatic and performant, 305 including reproducing blocking behavior and interaction with 306 nonblocking IO subsystems when possible. 308 5.2.2. Abstract Interfaces 310 This section presents high-level pseudocode descriptions of the 311 interfaces exposed by different types of transport components. 312 Implementations for different languages should provide equivalent 313 functionality, but should use the idioms for each language, mimicking 314 the existing networking libraries. 316 5.2.2.1. Transport 318 - Transport takes a transport configuration and provides a Client 319 Factory and a Server Factory. 321 - Transports may provide additional language-specific configuration 322 methods. 324 - The only way to obtain Client Factories and Server Factories is 325 from the Transport. 327 - The Server Factory of the Transport can fail if the Transport does 328 not provide a server-side implementation. However, most 329 transports provide both a client and server implementation. 331 - The transport configuration is specific to each Transport. Using 332 a Transport requires knowing the correct parameters to initialize 333 that Transport. 335 5.2.2.1.1. Client Factory 337 - Client Factory takes the connection settings and produces 338 a Connection to that server. 340 - The connection settings are specific to each transport. Some 341 transports will also require an argument indicating 342 the destination endpoint. Producing a Connection may fail if the 343 server is unreachable or if the transport configuration was 344 incorrect. 346 5.2.2.1.2. Server Factory 348 - Server Factory takes the address on which the PT server should 349 listen for incoming client connections and produces a Listener for 350 that address 352 5.2.2.1.3. Listener 354 - Listener produces a stream of Connections 356 - New Connections are available whenever an incoming network 357 connection from the PT client has been established. The language- 358 specific API can adopt either a blocking or non-blocking API for 359 accepting new connections, depending on what is idiomatic for the 360 language. 3.2.2.2. Connection 362 - Connection provides an API similar to the environment's native 363 socket type 365 - Connection is what is used to read and write data over the 366 transport connection 368 - The transport-specific logic for obfuscating network traffic is 369 implemented inside the Connection. 371 6. Adapters 373 This section covers the various different ways that the Pluggable 374 Transport interfaces (both API and IPC) can be adapted to different 375 use cases. 377 6.1. API to IPC Adapter 379 When an application and the transports it uses are written in the 380 same language, either the Transports API or Dispatcher IPC can be 381 used. When they are in different languages, they must communicate 382 through the Dispatcher IPC interface. For maximum flexibility and to 383 minimize duplication of effort across languages, dispatcher can be 384 implemented by wrapping transport implementations that implement the 385 Transports API. For an example of this approach, see the 386 Shapeshifter Dispatcher [PT2-DISPATCHER], which wraps transports 387 implementing the Transports API in the Go language and provides a 388 Dispatcher IPC interface to use them from other languages. 390 6.2. PT 1.0 Compatibility 392 The only interface defined in the PT 1.0 specification is an IPC 393 interface. No standard API is defined. Therefore, PT 1.0 394 compatibility refers to compatibility between applications and 395 transports where one side conforms to the PT 1.0 specification and 396 the other conforms to the PT 2.1 specification. Fortunately, an 397 adapter is not needed in this case as both the PT 1.0 and PT 2.1 398 specifications allow for version negotiation. The 399 TOR_PT_MANAGED_TRANSPORT_VER environment variable or -ptversion 400 command line flag is used by the application to specify a list of 401 supported versions, for instance "1.0,2.1". The PT provider responds 402 with the VERSION command on stdout in order to specify which version 403 is supported by the PT provider, for instance "VERSION 2.1". Since 404 the application can specify a list of supported versions, the PT 405 provider can respond dynamically, supporting PT 1.0 when required and 406 automatically upgrading to a PT 2.1 implementation when that is an 407 available option. It is up to applications whether they want to 408 support PT 2.1 exclusively or maintain backwards compatibility with 409 PT 1.0 implementations. 411 6.3. Cross-language Linking 413 If two languages are compatible via cross-language linking, then a 414 suitable adapter can be written that wraps the implementation of the 415 Transports API in one language with an API for a compatible language. 416 For example, on Android the Go implementation of the Transports API 417 is wrapped in a Java API to create Java language bindings without the 418 need for a native Java implementation or use of Dispatcher IPC. 420 6.3.1. Using the Dispatcher IPC Interface In-process 422 When using a transport that exposes the Dispatcher IPC interface, it 423 may be more convenient to run the transport in a separate thread but 424 in the same process as the application. Packets can still be routed 425 through the transport's SOCKS5 or TURN port on localhost. However, 426 it may be inconvenient or impossible to use STDIN and STDOUT for 427 communication between these two threads. Therefore, in some 428 languages it may be appropriate to produce an "inter-thread 429 interface" that reproduces the Dispatcher IPC interface's semantics, 430 but replaces STDIN and STDOUT with language-native function-call and 431 event primitives. This is the approach used by OnionBrowser 432 [ONION_BROWSER], the Tor implementation on iOS. This approach is 433 used because Tor uses the Dispatcher IPC mechanism to talk to the 434 transports instead of the Transports API. However, iOS does not 435 allow for applications to have multiple processes. Therefore, an in- 436 process Dispatcher IPC approach must be used instead of traditional 437 separate process Dispatcher IPC. An alternative would be to use the 438 Transports API directly instead of Dispatcher IPC. 440 6.4. Anonymity Considerations 442 When designing and implementing a Pluggable Transport, care should be 443 taken to preserve the privacy of clients and to avoid leaking 444 personally identifying information. Examples of client related 445 considerations are: 447 - Not logging client IP addresses to disk. 449 - Not leaking DNS addresses except when necessary. 451 - Ensuring that "TOR_PT_PROXY"'s "fail closed" behavior is 452 implemented correctly. 454 Additionally, certain obfuscation mechanisms rely on information such 455 as the server IP address and port being confidential, so clients also 456 need to take care to preserve server side information confidential 457 when applicable. 459 7. References 461 7.1. Normative References 463 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 464 Requirement Levels", BCP 14, RFC 2119, 465 DOI 10.17487/RFC2119, March 1997, 466 . 468 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 469 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 470 May 2017, . 472 7.2. Informative References 474 [ONION_BROWSER] 475 "Onion Browser", 2019, 476 . 478 [PT2-DISPATCHER] 479 Wiley, B., "Shapeshifter Dispatcher", 2018, 480 . 483 [PT2.1] Wiley, B., "Pluggable Transport Base Specification", 2018, 484 . 490 Acknowledgments 492 Many people contributed to the PT 2.1 specification. Major 493 contributions were made by Dr. Brandon Wiley (Operator Foundation), 494 Nick Mathewson (Tor), and Ben Schwartz (Jigsaw). Valuable feedback 495 was provided by the attendees at the Pluggable Transport Implementers 496 Meetings and the traffic-obf and tor-dev mailing lists. The PT 2.1 497 specification expands upon the "Pluggable Transport Specification 498 (Version 1)" document authored by Yawning Angel (Tor). Inspiration 499 for the PT 2.1 Go API was also inspired by the obfs4proxy 500 implementation of the PT 1.0 specification in Go, also developed by 501 Yawning Angel (Tor). 503 Authors' Addresses 505 Brandon Wiley 506 Operator Foundation 508 EMail: brandon@operatorfoundation.org 509 URI: https://operatorfoundation.org 511 David M. Oliver 512 Guardian Project 514 EMail: david@guardianproject.info 515 URI: https://guardianproject.info