idnits 2.17.1 draft-murchison-nntp-compress-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The exact meaning of the all-uppercase expression 'MAY NOT' is not defined in RFC 2119. If it is intended as a requirements expression, it should be rewritten using one of the combinations defined in RFC 2119; otherwise it should not be all-uppercase. == The expression 'MAY NOT', while looking like RFC 2119 requirements text, is not defined in RFC 2119, and should not be used. Consider using 'MUST NOT' instead (if that is what you mean). Found 'MAY NOT' in this paragraph: Additionally, implementations MAY NOT compress together the contents of two distinct confidential articles. This can be achieved for instance with DEFLATE by clearing the compression dictionary before sending any article in these confidential newsgroups. More complex strategies are of course possible, and encouraged. -- The document date (June 18, 2016) is 2869 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'C' is mentioned on line 434, but not defined == Missing Reference: 'S' is mentioned on line 435, but not defined -- Looks like a reference, but probably isn't: '1' on line 230 ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) -- Obsolete informational reference (is this intentional?): RFC 5246 (Obsoleted by RFC 8446) -- Obsolete informational reference (is this intentional?): RFC 7525 (Obsoleted by RFC 9325) Summary: 1 error (**), 0 flaws (~~), 4 warnings (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Independent Submission K. Murchison 3 Internet-Draft Carnegie Mellon University 4 Intended status: Standards Track J. Elie 5 Expires: December 20, 2016 June 18, 2016 7 Network News Transfer Protocol (NNTP) Extension for Compression 8 draft-murchison-nntp-compress-04 10 Abstract 12 This document defines an extension to the Network News Transport 13 Protocol (NNTP) that allows a connection to be effectively and 14 efficiently compressed between an NNTP client and server. 16 Status of This Memo 18 This Internet-Draft is submitted in full conformance with the 19 provisions of BCP 78 and BCP 79. 21 Internet-Drafts are working documents of the Internet Engineering 22 Task Force (IETF). Note that other groups may also distribute 23 working documents as Internet-Drafts. The list of current Internet- 24 Drafts is at http://datatracker.ietf.org/drafts/current/. 26 Internet-Drafts are draft documents valid for a maximum of six months 27 and may be updated, replaced, or obsoleted by other documents at any 28 time. It is inappropriate to use Internet-Drafts as reference 29 material or to cite them other than as "work in progress." 31 This Internet-Draft will expire on December 20, 2016. 33 Copyright Notice 35 Copyright (c) 2016 IETF Trust and the persons identified as the 36 document authors. All rights reserved. 38 This document is subject to BCP 78 and the IETF Trust's Legal 39 Provisions Relating to IETF Documents 40 (http://trustee.ietf.org/license-info) in effect on the date of 41 publication of this document. Please review these documents 42 carefully, as they describe your rights and restrictions with respect 43 to this document. Code Components extracted from this document must 44 include Simplified BSD License text as described in Section 4.e of 45 the Trust Legal Provisions and are provided without warranty as 46 described in the Simplified BSD License. 48 Table of Contents 50 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 51 1.1. About TLS-level Compression . . . . . . . . . . . . . . . 3 52 1.2. Conventions Used in This Document . . . . . . . . . . . . 4 53 1.3. Authors' Note . . . . . . . . . . . . . . . . . . . . . . 4 54 2. The COMPRESS Extension . . . . . . . . . . . . . . . . . . . 4 55 2.1. Advertising the COMPRESS Extension . . . . . . . . . . . 4 56 2.2. COMPRESS Command . . . . . . . . . . . . . . . . . . . . 5 57 2.2.1. Usage . . . . . . . . . . . . . . . . . . . . . . . . 5 58 2.2.2. Description . . . . . . . . . . . . . . . . . . . . . 6 59 2.2.3. Examples . . . . . . . . . . . . . . . . . . . . . . 7 60 3. Compression Efficiency . . . . . . . . . . . . . . . . . . . 10 61 3.1. DEFLATE Specificities . . . . . . . . . . . . . . . . . . 12 62 4. Augmented BNF Syntax for the COMPRESS Extension . . . . . . . 12 63 4.1. Commands . . . . . . . . . . . . . . . . . . . . . . . . 13 64 4.2. Capability Entries . . . . . . . . . . . . . . . . . . . 13 65 4.3. General Non-terminals . . . . . . . . . . . . . . . . . . 13 66 5. Summary of Response Codes . . . . . . . . . . . . . . . . . . 13 67 6. Security Considerations . . . . . . . . . . . . . . . . . . . 13 68 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 69 7.1. NNTP Compression Algorithm Registry . . . . . . . . . . . 14 70 7.1.1. Algorithm Name Registration Procedure . . . . . . . . 15 71 7.1.2. Comments on Algorithm Registrations . . . . . . . . . 16 72 7.1.3. Change Control . . . . . . . . . . . . . . . . . . . 16 73 7.2. Registration of the DEFLATE Compression Algorithm . . . . 17 74 7.3. Registration of the NNTP COMPRESS Extension . . . . . . . 17 75 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 18 76 8.1. Normative References . . . . . . . . . . . . . . . . . . 18 77 8.2. Informative References . . . . . . . . . . . . . . . . . 19 78 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 21 79 Appendix B. Document History (to be removed by RFC Editor before 80 publication) . . . . . . . . . . . . . . . . . . . . 21 81 B.1. Changes since -03 . . . . . . . . . . . . . . . . . . . . 21 82 B.2. Changes since -02 . . . . . . . . . . . . . . . . . . . . 21 83 B.3. Changes since -01 . . . . . . . . . . . . . . . . . . . . 22 84 B.4. Changes since -00 . . . . . . . . . . . . . . . . . . . . 22 85 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 23 87 1. Introduction 89 The goal of COMPRESS is to reduce the bandwidth usage of NNTP. 91 Compared to PPP compression [RFC1962] and modem-based compression 92 ([MNP] and [V42bis]), COMPRESS offers greater compression efficiency. 93 COMPRESS can be used together with Transport Layer Security (TLS) 94 [RFC5246], Simple Authentication and Security Layer (SASL) encryption 95 [RFC4422], Virtual Private Networks (VPNs), etc. 97 The point of COMPRESS as an NNTP extension is to act as a compression 98 layer, similarly to a security layer like the one negotiated by 99 STARTTLS [RFC4642]. Compression can therefore benefit to all NNTP 100 commands sent or received after the use of COMPRESS. This facility 101 responds to a long-standing need for NNTP to compress data, that has 102 partially been addressed by unstandardized commands like XZVER, 103 XZHDR, XFEATURE COMPRESS, or MODE COMPRESS. These commands are not 104 wholly satisfactory because they enable compression only for the 105 responses sent by the news server. On the contrary, the COMPRESS 106 command permits to compress data sent by both the client and the 107 server, and removes the constraint of having to implement compression 108 separately in each NNTP command. Besides, the compression level can 109 be dynamically adjusted and optimized at any time during the 110 connection, which even allows to disable compression for certain 111 commands, if need be. If the news client wants to stop compression 112 on a particular connection, it can simply use QUIT ([RFC3977] 113 Section 5.4), and establish a new connection. For these reasons, 114 using other NNTP commands than COMPRESS to enable compression is 115 discouraged once COMPRESS is supported. 117 In order to increase interoperability, it is desirable to have as few 118 different compression algorithms as possible, so this document 119 specifies only one. The DEFLATE algorithm (defined in [RFC1951]) 120 MUST be implemented as part of this extension. This compression 121 algorithm is standard, widely available, and fairly efficient. 123 This specification should be read in conjunction with the NNTP base 124 specification [RFC3977]. In the case of a conflict between these two 125 documents, [RFC3977] takes precedence. 127 1.1. About TLS-level Compression 129 Though data compression is made possible via the use of TLS with NNTP 130 [RFC4642], the best current practice is to disable TLS-level 131 compression as explained in Section 3.3 of [RFC7525]. The COMPRESS 132 command will permit to keep the compression facility in NNTP and 133 control when it is available during a connection. 135 Compared to TLS-level compression [RFC3749], NNTP COMPRESS has the 136 following advantages: 138 o COMPRESS can be implemented easily both by NNTP servers and 139 clients. 141 o COMPRESS benefits from an intimate knowledge of the NNTP 142 protocol's state machine, allowing for dynamic and aggressive 143 optimization of the underlying compression algorithm's parameters. 145 o COMPRESS can be activated after authentication has completed thus 146 reducing the chances that authentication credentials can be leaked 147 via for instance a CRIME attack ([RFC7457] Section 2.6). 149 1.2. Conventions Used in This Document 151 The notational conventions used in this document are the same as 152 those in [RFC3977], and any term not defined in this document has the 153 same meaning as it does in that one. 155 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 156 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 157 document are to be interpreted as described in [RFC2119]. 159 In the examples, commands from the client are indicated with [C], and 160 responses from the server are indicated with [S]. The client is the 161 initiator of the NNTP connection; the server is the other endpoint. 163 1.3. Authors' Note 165 Please write the first letter of "Elie" and the penultimate letter of 166 "allee" with an acute accent wherever possible -- they are 167 respectively U+00C9 ("É" in XML) and U+00E9 ("é" in XML). 168 Also, the letters "ae" in "Baeuerle" should be written as an a-umlaut 169 (U+00E4, "ä" in XML), and the first letter of "Angel" as well as 170 the fifth letter of "Gonzalez" should be written with an acute accent 171 (respectively U+00C1 and U+00E1, that is to say "Á" and "á" 172 in XML). 174 2. The COMPRESS Extension 176 The COMPRESS extension is used to enable data compression on an NNTP 177 connection. 179 This extension provides a new COMPRESS command and has capability 180 label COMPRESS. 182 2.1. Advertising the COMPRESS Extension 184 A server supporting the COMPRESS command as defined in this document 185 will advertise the "COMPRESS" capability label in response to the 186 CAPABILITIES command ([RFC3977] Section 5.2). However, this 187 capability MUST NOT be advertised once a compression layer is active 188 (see Section 2.2.2). This capability MAY be advertised both before 189 and after any use of the MODE READER command ([RFC3977] Section 5.3), 190 with the same semantics. 192 The COMPRESS capability label contains a whitespace-separated list of 193 available compression algorithms. This document defines one 194 compression algorithm: DEFLATE. This algorithm is mandatory to 195 implement and MUST be supported in order to advertise the COMPRESS 196 extension. 198 Future extensions may add additional compression algorithms to this 199 capability. Unrecognized algorithms MUST be ignored by the client. 201 As the COMPRESS command is related to security because it can weaken 202 encryption, cached results of CAPABILITIES from a previous session 203 MUST NOT be relied on, as per Section 12.6 of [RFC3977]. 205 Example: 207 [C] CAPABILITIES 208 [S] 101 Capability list: 209 [S] VERSION 2 210 [S] READER 211 [S] IHAVE 212 [S] COMPRESS DEFLATE X-SHRINK 213 [S] LIST ACTIVE NEWSGROUPS 214 [S] . 216 2.2. COMPRESS Command 218 2.2.1. Usage 220 This command MUST NOT be pipelined. 222 Syntax 223 COMPRESS algorithm 225 Responses 226 206 Compression active 227 403 Unable to activate compression 228 502 Command unavailable [1] 230 [1] If a compression layer is already active, COMPRESS is not a valid 231 command (see Section 2.2.2). 233 Parameters 234 algorithm = Name of compression algorithm (e.g. "DEFLATE") 236 2.2.2. Description 238 The COMPRESS command instructs the server to use the named 239 compression algorithm ("DEFLATE" is the only one defined in this 240 document) for all commands and/or responses after COMPRESS. 242 The client MUST NOT send any further commands until it has seen the 243 result of COMPRESS. 245 If the requested compression algorithm is syntactically incorrect, 246 the server MUST reject the COMPRESS command with a 501 response code 247 ([RFC3977] Section 3.2.1). If the requested compression algorithm is 248 invalid (e.g., is not supported), the server MUST reject the COMPRESS 249 command with a 503 response code ([RFC3977] Section 3.2.1). If the 250 server is unable to activate compression for any reason (e.g., a 251 server configuration or resource problem), the server MUST reject the 252 COMPRESS command with a 403 response code ([RFC3977] Section 3.2.1). 253 Otherwise, the server issues a 206 response code and the compression 254 layer takes effect for both client and server immediately following 255 the CRLF of the success reply. 257 Additionally, the client MUST NOT issue a MODE READER command after 258 activating a compression layer, and a server MUST NOT advertise the 259 MODE-READER capability. 261 Both the client and the server MUST know if there is a compression 262 layer active (for instance via the previous use of the COMPRESS 263 command or the negotiation of a TLS-level compression [RFC3749]). A 264 client MUST NOT attempt to activate compression or negotiate a TLS 265 layer (for instance via the use of the COMPRESS or STARTTLS [RFC4642] 266 commands) if a compression layer is already active. A server MUST 267 NOT return the COMPRESS or STARTTLS capability labels in response to 268 a CAPABILITIES command received after a compression layer is active, 269 and a server MUST reply with a 502 response code if a syntactically 270 valid COMPRESS or STARTTLS command is received while a compression 271 layer is already active. 273 In order to help mitigate leaking authentication credentials via for 274 instance a CRIME attack [CRIME], authentication SHOULD NOT be 275 attempted when a compression layer is active. Consequently, a server 276 SHOULD NOT return any arguments with the AUTHINFO capability label 277 (or SHOULD NOT advertise it at all) in response to a CAPABILITIES 278 command received from an unauthenticated client after a compression 279 layer is active, and such a client SHOULD NOT attempt to utilize any 280 AUTHINFO [RFC4643] commands. It implies that a server SHOULD reply 281 with a 502 response code if a syntactically valid AUTHINFO command is 282 received while a compression layer is already active. 284 For DEFLATE [RFC1951] (as for many other compression algorithms), the 285 compressor can trade speed against quality. The decompressor MUST 286 automatically adjust to the parameters selected by the sender. 287 Consequently, the client and server are both free to pick the best 288 reasonable rate of compression for the data they send. 290 When COMPRESS is combined with TLS [RFC5246] or SASL [RFC4422] 291 security layers, the processing order of the three layers MUST be 292 first COMPRESS, then SASL, and finally TLS. That is, before data is 293 transmitted, it is first compressed. Second, if a SASL security 294 layer has been negotiated, the compressed data is then signed and/or 295 encrypted accordingly. Third, if a TLS security layer has been 296 negotiated, the data from the previous step is signed and/or 297 encrypted accordingly. When receiving data, the processing order 298 MUST be reversed. This ensures that before sending, data is 299 compressed before it is encrypted, independent of the order in which 300 the client issues the COMPRESS, AUTHINFO SASL [RFC4643], and STARTTLS 301 [RFC4642] commands. 303 When compression is active and either the client or the server 304 receives invalid or corrupted compressed data, the receiving end 305 SHOULD immediately close the connection. (Then the sending end will 306 in turn do the same.) 308 2.2.3. Examples 310 Example of layering TLS and NNTP compression: 312 [C] CAPABILITIES 313 [S] 101 Capability list: 314 [S] VERSION 2 315 [S] READER 316 [S] STARTTLS 317 [S] AUTHINFO 318 [S] COMPRESS DEFLATE 319 [S] LIST ACTIVE NEWSGROUPS 320 [S] . 321 [C] STARTTLS 322 [S] 382 Continue with TLS negotiation 323 [TLS negotiation without compression occurs here] 324 [Following successful negotiation, all traffic is encrypted] 325 [C] CAPABILITIES 326 [S] 101 Capability list: 327 [S] VERSION 2 328 [S] READER 329 [S] AUTHINFO USER 330 [S] COMPRESS DEFLATE 331 [S] LIST ACTIVE NEWSGROUPS 332 [S] . 333 [C] AUTHINFO USER fred 334 [S] 381 Enter passphrase 335 [C] AUTHINFO PASS flintstone 336 [S] 281 Authentication accepted 337 [C] CAPABILITIES 338 [S] 101 Capability list: 339 [S] VERSION 2 340 [S] READER 341 [S] POST 342 [S] COMPRESS DEFLATE 343 [S] LIST ACTIVE NEWSGROUPS 344 [S] . 345 [C] COMPRESS DEFLATE 346 [S] 206 Compression active 347 [Henceforth, all traffic is compressed before being encrypted] 349 Example of a server failing to activate compression: 351 [C] CAPABILITIES 352 [S] 101 Capability list: 353 [S] VERSION 2 354 [S] IHAVE 355 [S] COMPRESS DEFLATE 356 [S] . 357 [C] COMPRESS DEFLATE 358 [S] 403 Unable to activate compression 360 Example of attempting to use an unsupported compression algorithm: 362 [C] CAPABILITIES 363 [S] 101 Capability list: 364 [S] VERSION 2 365 [S] IHAVE 366 [S] COMPRESS DEFLATE 367 [S] . 368 [C] COMPRESS X-SHRINK 369 [S] 503 Compression algorithm not supported 371 Example of a server refusing to compress twice: 373 [C] CAPABILITIES 374 [S] 101 Capability list: 375 [S] VERSION 2 376 [S] IHAVE 377 [S] STARTTLS 378 [S] COMPRESS DEFLATE 379 [S] . 380 [C] STARTTLS 381 [S] 382 Continue with TLS negotiation 382 [TLS negotiation with compression occurs here] 383 [Following successful negotiation, all traffic is encrypted] 384 [C] CAPABILITIES 385 [S] 101 Capability list: 386 [S] VERSION 2 387 [S] IHAVE 388 [S] . 389 [C] COMPRESS DEFLATE 390 [S] 502 Compression already active via TLS 392 Example of a server refusing to negotiate a TLS layer after 393 compression has been activated: 395 [C] CAPABILITIES 396 [S] 101 Capability list: 397 [S] VERSION 2 398 [S] IHAVE 399 [S] STARTTLS 400 [S] COMPRESS DEFLATE 401 [S] . 402 [C] COMPRESS DEFLATE 403 [S] 206 Compression active 404 [Henceforth, all traffic is compressed] 405 [C] CAPABILITIES 406 [S] 101 Capability list: 407 [S] VERSION 2 408 [S] IHAVE 409 [S] . 410 [C] STARTTLS 411 [S] 502 DEFLATE compression already active 413 Example of a server not advertising AUTHINFO arguments after 414 compression has been activated: 416 [C] CAPABILITIES 417 [S] 101 Capability list: 418 [S] VERSION 2 419 [S] READER 420 [S] AUTHINFO USER 421 [S] COMPRESS DEFLATE 422 [S] LIST ACTIVE NEWSGROUPS 423 [S] . 424 [C] COMPRESS DEFLATE 425 [S] 206 Compression active 426 [Henceforth, all traffic is compressed] 427 [C] CAPABILITIES 428 [S] 101 Capability list: 429 [S] VERSION 2 430 [S] READER 431 [S] AUTHINFO 432 [S] LIST ACTIVE NEWSGROUPS 433 [S] . 434 [C] AUTHINFO USER fred 435 [S] 502 DEFLATE compression already active 437 3. Compression Efficiency 439 This section is informative, not normative. 441 NNTP poses some unusual problems for a compression layer. 443 Upstream traffic is fairly simple. Most NNTP clients send the same 444 few commands again and again, so any compression algorithm that can 445 exploit repetition works efficiently. The article posting and 446 transfer commands (e.g., POST, IHAVE, and TAKETHIS [RFC4644]) are 447 exceptions; clients that send many article posting or transfer 448 commands may want to surround large multi-line data blocks with a 449 dictionary flush and/or, depending on the compression algorithm, a 450 change of compression level in the same way as is recommended for 451 servers later in this document (Section 3.1). 453 Downstream traffic has the unusual property that several kinds of 454 data are sent, possibly confusing a dictionary-based compression 455 algorithm. 457 One type is NNTP simple responses and NNTP multi-line responses not 458 related to article header/body retrieval (e.g, CAPABILITIES, GROUP, 459 LISTGROUP, LAST, NEXT, STAT, DATE, NEWNEWS, NEWGROUPS, LIST, CHECK 460 [RFC4644], etc). These are highly compressible; zlib using its least 461 CPU-intensive setting compresses typical responses to 25-40% of their 462 original size. 464 Another type is article headers (as retrieved via the HEAD, HDR, 465 OVER, or ARTICLE commands). These are equally compressible, and 466 benefit from using the same dictionary as the NNTP responses. 468 A third type is article body text (as retrieved via the BODY or 469 ARTICLE commands). Text is usually fairly short and includes much 470 ASCII, so the same compression dictionary will do a good job here, 471 too. When multiple messages in the same thread are read at the same 472 time, quoted lines, etc. can often be compressed almost to zero. 474 Finally, non-text article bodies or attachments (as retrieved via the 475 BODY and ARTICLE commands) are transmitted in encoded form, usually 476 Base64 [RFC4648], UUencode [IEEE.1003-2.1992], or yEnc [yEnc]. 478 When already compressed articles or attachments are retrieved, a 479 compression algorithm may be able to compress them, but the format of 480 their encoding is usually not NNTP-like, so the dictionary built 481 while compressing NNTP does not help much. The compressor has to 482 adapt its dictionary from NNTP to the attachment's encoding format, 483 and then back. 485 When attachments are retrieved in Base64 or UUencode form, the 486 Huffman coding usually compresses those to approximatively only 75% 487 of their encoding size. 8-bit compression algorithms such as DEFLATE 488 work well on 8-bit file formats; however, both Base64 and UUencode 489 transform a file into something resembling 6-bit bytes, hiding most 490 of the 8-bit file format from the compressor. 492 On the other end, attachments encoded using a compression algorithm 493 that retains the full 8-bit spectrum, like yEnc, are much more likely 494 to be incompressible. 496 3.1. DEFLATE Specificities 498 When using the zlib library (see [RFC1951]), the functions 499 deflateInit2(), deflate(), inflateInit2(), and inflate() suffice to 500 implement this extension. The windowBits value must be in the range 501 -8 to -15 for deflateInit2(), or else it will use the wrong format. 502 The windowBits value should be -15 for inflateInit2(), or else it 503 will not be able to decompress a stream with a larger window size. 504 deflateParams() can be used to improve compression rate and resource 505 use. In order to improve compression efficiency, the Z_PARTIAL_FLUSH 506 argument to deflate() should always be used to flush data. As far as 507 DEFLATE is concerned, clearing the dictionary never improves 508 compression over the other flushes. On the contrary, having the 32kB 509 dictionary from previous data, no matter how unrelated, can only 510 help. If there are no matching strings in there, then it is simply 511 not referenced. Using Z_FULL_FLUSH clears the dictionary, and 512 consequently always results in compression that is less effective 513 than a Z_PARTIAL_FLUSH. 515 A server can improve downstream compression and the CPU efficiency 516 both of the server and the client if it adjusts the compression level 517 (e.g., using the deflateParams() function in zlib) at the start and 518 end of large non-text multi-line data blocks (before and after 519 'content-lines' in the definition of 'multi-line-data-block' in 520 [RFC3977] Section 9.8). It permits to avoid trying to compress 521 incompressible attachments. Small multi-line data blocks are best 522 left alone. A possible boundary is 5kB. 524 A very simple strategy is to change the compression level to 0 at the 525 start of a multi-line data block provided the first two bytes are 526 either 0x1F 0x8B (as in deflate-compressed files) or 0xFF 0xD8 527 (JPEG), and to keep it at 1-5 the rest of the time. More complex 528 strategies are of course possible, and encouraged. 530 4. Augmented BNF Syntax for the COMPRESS Extension 532 This section describes the formal syntax of the COMPRESS extension 533 using ABNF [RFC5234]. It extends the syntax in Section 9 of 534 [RFC3977], and non-terminals not defined in this document are defined 535 there. The [RFC3977] ABNF should be imported first, before 536 attempting to validate these rules. 538 4.1. Commands 540 This syntax extends the non-terminal , which represents an 541 NNTP command. 543 command =/ compress-command 545 compress-command = "COMPRESS" WS algorithm 547 4.2. Capability Entries 549 This syntax extends the non-terminal , which 550 represents a capability that may be advertised by the server. 552 capability-entry =/ compress-capability 554 compress-capability = "COMPRESS" 1*(WS algorithm) 556 4.3. General Non-terminals 558 algorithm = "DEFLATE" / 1*20alg-char 559 alg-char = UPPER / DIGIT / "-" / "_" 561 5. Summary of Response Codes 563 This section contains a list of each new response code defined in 564 this document and indicates whether it is multi-line, which commands 565 can generate it, what arguments it has, and what its meaning is. 567 Response code 206 568 Generated by: COMPRESS 569 Meaning: compression layer activated 571 6. Security Considerations 573 Security issues are discussed throughout this document. 575 In general, the security considerations of the NNTP core 576 specification ([RFC3977] Section 12) and the DEFLATE compressed data 577 format specification ([RFC1951] Section 6) are applicable here. 579 Implementers should be aware that combining compression with 580 encryption like TLS can sometimes reveal information that would not 581 have been revealed without compression, as explained in Section 6 of 582 [RFC3749]. As a matter of fact, adversaries that observe the length 583 of the compressed data might be able to derive information about the 584 corresponding uncompressed data. The CRIME and the BREACH attacks 585 ([RFC7457] Section 2.6) are examples of such case. 587 In order to help mitigate leaking authentication credentials, this 588 document states in Section 2.2.2 that authentication SHOULD NOT be 589 attempted when a compression layer is active. Therefore, when a 590 client wants to authenticate, compress data, and negotiate a TLS 591 layer (without TLS-level compression) in the same NNTP connection, it 592 SHOULD use the STARTTLS, AUTHINFO, and COMPRESS commands in that 593 order. Of course instead of using the STARTTLS command, a client can 594 also use implicit TLS, that is to say it begins the TLS negotiation 595 immediately upon connection on a separate port dedicated to NNTP over 596 TLS. 598 NNTP commands other than AUTHINFO are not believed to divulgate 599 confidential information as long as only public Netnews newsgroups 600 and articles are accessed. That is why this specification only adds 601 a restriction to the use of AUTHINFO when a compression layer is 602 active. In case confidential articles are accessed in private 603 newsgroups, special care is needed: implementations SHOULD NOT 604 compress confidential data together with public data when a security 605 layer is active, for the same reasons as mentioned above in this 606 Section. 608 Additionally, implementations MAY NOT compress together the contents 609 of two distinct confidential articles. This can be achieved for 610 instance with DEFLATE by clearing the compression dictionary before 611 sending any article in these confidential newsgroups. More complex 612 strategies are of course possible, and encouraged. 614 Implementations SHOULD use a default configuration with disabled 615 compression when a security layer is active, and MUST support an 616 option to allow compression to be enabled when a security layer is 617 active. Such an option can be either with global scope or server/ 618 connection based. Implementations MAY unconditionally allow 619 compression to be enabled when no security layer is active. 621 Future extensions to NNTP that define commands conveying confidential 622 data SHOULD ensure to state that these confidential data SHOULD NOT 623 be compressed together with public data when a security layer is 624 active. 626 7. IANA Considerations 628 7.1. NNTP Compression Algorithm Registry 630 The NNTP Compression Algorithm registry will be maintained by IANA. 631 The registry will be available at . 634 The purpose of this registry is not only to ensure uniqueness of 635 values used to name NNTP compression algorithms, but also to provide 636 a definitive reference to technical specifications detailing each 637 NNTP compression algorithm available for use on the Internet. 639 An NNTP compression algorithm is either a private algorithm, or its 640 name is included in the IANA NNTP Compression Algorithm registry (in 641 which case it is a "registered NNTP compression algorithm"). 642 Different entries in the registry MUST use different names. 644 Any name that conforms to the syntax of an NNTP compression algorithm 645 name (Section 4.3) can be used. However, the name of a registered 646 NNTP compression algorithm MUST NOT begin with "X". 648 To avoid the risk of a clash with a future registered NNTP 649 compression algorithm, the names of private compression algorithms 650 SHOULD begin with "X-". 652 If the server advertises a registered NNTP compression algorithm, it 653 MUST implement the compression algorithm so as to fully conform with 654 its related specification. If it does not implement the compression 655 algorithm as specified, it MUST NOT list its registered name in the 656 arguments list of the COMPRESS capability label. In that case, it 657 MAY, but SHOULD NOT, provide a private algorithm (not listed, or 658 listed with a different name) that implements the compression 659 algorithm differently. 661 A server MUST NOT send different response codes to COMPRESS in 662 response to the availability or use of a private compression 663 algorithm. 665 The procedure detailed in Section 7.1.1 is to be used for 666 registration of a value naming a specific individual compression 667 algorithm. 669 Comments may be included in the registry as discussed in 670 Section 7.1.2 and may be changed as discussed in Section 7.1.3. 672 7.1.1. Algorithm Name Registration Procedure 674 IANA will register new NNTP compression algorithm names on a First 675 Come First Served basis, as defined in BCP 26 [RFC5226]. IANA has 676 the right to reject obviously bogus registration requests, but will 677 perform no review of claims made in the registration form. 679 Registration of an NNTP compression algorithm is requested by filling 680 in the following template and sending it via electronic mail to IANA 681 at : 683 Subject: Registration of NNTP compression algorithm X 685 NNTP compression algorithm name: 687 Security considerations: 689 Published specification (recommended): 691 Contact for further information: 693 Intended usage: (One of COMMON, LIMITED USE, or OBSOLETE) 695 Owner/Change controller: 697 Note: (Any other information that the author deems relevant may be 698 added here.) 700 While this registration procedure does not require expert review, 701 authors of NNTP compression algorithms are encouraged to seek 702 community review and comment whenever that is feasible. Authors may 703 seek community review by posting a specification of their proposed 704 algorithm as an Internet-Draft. NNTP compression algorithms intended 705 for widespread use should be standardized through the normal IETF 706 process, when appropriate. 708 7.1.2. Comments on Algorithm Registrations 710 Comments on a registered NNTP compression algorithm should first be 711 sent to the "owner" of the algorithm and/or to the mailing list for 712 the now concluded IETF NNTPEXT working group (). 715 Submitters of comments may, after a reasonable attempt to contact the 716 owner and/or the above mailing list, request IANA to attach their 717 comment to the NNTP compression algorithm registration itself by 718 sending mail to . At IANA's sole discretion, IANA may 719 attach the comment to the NNTP compression algorithm's registration. 721 7.1.3. Change Control 723 Once an NNTP compression algorithm registration has been published by 724 IANA, the owner may request a change to its definition. The change 725 request follows the same procedure as the initial registration 726 request. 728 The owner of an NNTP compression algorithm may pass responsibility 729 for the algorithm to another person or agency by informing IANA; this 730 can be done without discussion or review. 732 The IESG may reassign responsibility for an NNTP compression 733 algorithm. The most common case of this will be to enable changes to 734 be made to algorithms where the owner of the registration has died, 735 has moved out of contact, or is otherwise unable to make changes that 736 are important to the community. 738 NNTP compression algorithm registrations MUST NOT be deleted; 739 algorithms that are no longer believed appropriate for use can be 740 declared OBSOLETE by a change to their "intended usage" field; such 741 algorithms will be clearly marked in the registry published by IANA. 743 The IESG is considered to be the owner of all NNTP compression 744 algorithms that are on the IETF standards track. 746 7.2. Registration of the DEFLATE Compression Algorithm 748 This section gives a formal definition of the DEFLATE compression 749 algorithm as required by Section 7.1.1 for the IANA registry. 751 NNTP compression algorithm name: DEFLATE 753 Security considerations: See Section 6 of this document 755 Published specification: This document 757 Contact for further information: Authors of this document 759 Intended usage: COMMON 761 Owner/Change controller: IESG 763 Note: This algorithm is mandatory to implement 765 7.3. Registration of the NNTP COMPRESS Extension 767 This section gives a formal definition of the COMPRESS extension as 768 required by Section 3.3.3 of [RFC3977] for the IANA registry. 770 o The COMPRESS extension allows an NNTP connection to be effectively 771 and efficiently compressed. 773 o The capability label for this extension is "COMPRESS", whose 774 arguments list the available compression algorithms. 776 o This extension defines one new command, COMPRESS, whose behavior, 777 arguments, and responses are defined in Section 2.2. 779 o This extension does not associate any new responses with pre- 780 existing NNTP commands. 782 o This extension does affect the overall behavior of both server and 783 client, in that after successful use of the COMPRESS command, all 784 communication is transmitted in a compressed format. 786 o This extension does not affect the maximum length of commands or 787 initial response lines. 789 o This extension does not alter pipelining, but the COMPRESS command 790 cannot be pipelined. 792 o Use of this extension does alter the capabilities list; once the 793 COMPRESS command has been used successfully, the COMPRESS 794 capability can no longer be advertised by CAPABILITIES. 795 Additionally, the STARTTLS and MODE-READER capabilities MUST NOT 796 be advertised, and the AUTHINFO capability label SHOULD either 797 return no arguments or no longer be advertised after successful 798 execution of the COMPRESS command. 800 o This extension does not cause any pre-existing command to produce 801 a 401, 480, or 483 response code. 803 o This extension is unaffected by any use of the MODE READER 804 command; however, the MODE READER command MUST NOT be used in the 805 same session following a successful execution of the COMPRESS 806 command. 808 o The STARTTLS command MUST NOT be used, and the AUTHINFO command 809 SHOULD NOT be used, in the same session following a successful 810 execution of the COMPRESS command. 812 o Published Specification: This document. 814 o Contact for Further Information: Authors of this document. 816 o Change Controller: IESG . 818 8. References 820 8.1. Normative References 822 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 823 Requirement Levels", BCP 14, RFC 2119, 824 DOI 10.17487/RFC2119, March 1997, 825 . 827 [RFC3977] Feather, C., "Network News Transfer Protocol (NNTP)", 828 RFC 3977, DOI 10.17487/RFC3977, October 2006, 829 . 831 [RFC4642] Murchison, K., Vinocur, J., and C. Newman, "Using 832 Transport Layer Security (TLS) with Network News Transfer 833 Protocol (NNTP)", RFC 4642, DOI 10.17487/RFC4642, October 834 2006, . 836 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 837 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 838 DOI 10.17487/RFC5226, May 2008, 839 . 841 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 842 Specifications: ABNF", STD 68, RFC 5234, 843 DOI 10.17487/RFC5234, January 2008, 844 . 846 8.2. Informative References 848 [CRIME] Rizzo, J. and T. Duong, "The CRIME Attack", Ekoparty 849 Security Conference, 2012. 851 [IEEE.1003-2.1992] 852 Institute of Electrical and Electronics Engineers, 853 "Information Technology - Portable Operating System 854 Interface (POSIX) - Part 2: Shell and Utilities (Vol. 1)", 855 IEEE Standard 1003.2, 1992. 857 [MNP] Held, G., "The Complete Modem Reference", Second 858 Edition, Wiley Professional Computing, May 1994. 860 [RFC1951] Deutsch, P., "DEFLATE Compressed Data Format Specification 861 version 1.3", RFC 1951, DOI 10.17487/RFC1951, May 1996, 862 . 864 [RFC1962] Rand, D., "The PPP Compression Control Protocol (CCP)", 865 RFC 1962, DOI 10.17487/RFC1962, June 1996, 866 . 868 [RFC3749] Hollenbeck, S., "Transport Layer Security Protocol 869 Compression Methods", RFC 3749, DOI 10.17487/RFC3749, May 870 2004, . 872 [RFC4422] Melnikov, A., Ed. and K. Zeilenga, Ed., "Simple 873 Authentication and Security Layer (SASL)", RFC 4422, 874 DOI 10.17487/RFC4422, June 2006, 875 . 877 [RFC4643] Vinocur, J. and K. Murchison, "Network News Transfer 878 Protocol (NNTP) Extension for Authentication", RFC 4643, 879 DOI 10.17487/RFC4643, October 2006, 880 . 882 [RFC4644] Vinocur, J. and K. Murchison, "Network News Transfer 883 Protocol (NNTP) Extension for Streaming Feeds", RFC 4644, 884 DOI 10.17487/RFC4644, October 2006, 885 . 887 [RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data 888 Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006, 889 . 891 [RFC4978] Gulbrandsen, A., "The IMAP COMPRESS Extension", RFC 4978, 892 DOI 10.17487/RFC4978, August 2007, 893 . 895 [RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer Security 896 (TLS) Protocol Version 1.2", RFC 5246, 897 DOI 10.17487/RFC5246, August 2008, 898 . 900 [RFC7457] Sheffer, Y., Holz, R., and P. Saint-Andre, "Summarizing 901 Known Attacks on Transport Layer Security (TLS) and 902 Datagram TLS (DTLS)", RFC 7457, DOI 10.17487/RFC7457, 903 February 2015, . 905 [RFC7525] Sheffer, Y., Holz, R., and P. Saint-Andre, 906 "Recommendations for Secure Use of Transport Layer 907 Security (TLS) and Datagram Transport Layer Security 908 (DTLS)", BCP 195, RFC 7525, DOI 10.17487/RFC7525, May 909 2015, . 911 [V42bis] International Telecommunications Union, "Data compression 912 procedures for data circuit-terminating equipment (DCE) 913 using error correction procedures", ITU-T Recommendation 914 V.42bis, January 1990. 916 [yEnc] Helbing, J., "yEnc - Efficient encoding for Usenet and 917 eMail", March 2002, . 919 Appendix A. Acknowledgements 921 This document draws heavily on ideas in [RFC4978] by Arnt Gulbrandsen 922 and a large portion of this text was borrowed from that 923 specification. 925 The authors would like to thank the following individuals for 926 contributing their ideas and support for writing this specification: 927 Mark Adler, Russ Allbery, Michael Baeuerle, Angel Gonzalez, and Brian 928 Peterson. 930 Appendix B. Document History (to be removed by RFC Editor before 931 publication) 933 B.1. Changes since -03 935 o Added a naming convention for NNTP compression algorithms. 936 Improve the wording of registered vs private compression 937 algorithms. 939 o If a registered NNTP compression algorithm is advertised, it MUST 940 fully conform with its related specification. 942 o Fixed the wording of security considerations to reflect that the 943 threat appears when public and confidential data are compressed 944 together inside a security layer. Thanks to Angel Gonzalez for 945 pointing that. 947 o The default configuration SHOULD be disabled compression when a 948 security layer is active. 950 o COMPRESS acts as a compression layer, not a transport layer. 952 o Minor editorial changes. 954 B.2. Changes since -02 956 o Added text stating that the receiving end SHOULD terminate the 957 connection when receiving invalid or corrupted compressed data. 959 o Explained why COMPRESS permits to do better than existing 960 unstandardized commands like XZVER, XZHDR, MODE COMPRESS, and 961 XFEATURE GZIP. 963 o Added an example of AUTHINFO command when compression is active. 965 o The LIST capability label was missing in the examples when READER 966 was also advertised. 968 o Improved an example to send CAPABILITIES after successful 969 authentication. 971 o Mentioned that COMPRESS is related to security. CAPABILITIES is 972 therefore sent again after COMPRESS. 974 o Re-added discussion of attachments in binary form and 975 incompressible file formats. Improve the discussion about 976 flushes, and add a specific section about DEFLATE. 978 o Changed a MUST NOT to SHOULD NOT for the use of AUTHINFO after 979 COMPRESS. 981 o Algorithm names are case-insensitive. 983 o Mentioned the use of the 501 response code. 985 o Added the Security Considerations Section. 987 o Added Julien Elie as co-author of this document. 989 o Minor editorial changes. 991 B.3. Changes since -01 993 o Switched to using 206 response code when compression has been 994 activated. 996 o Added text stating that TLS-level compression is susceptible to 997 CRIME attack and current BCP is to disable it. 999 o Added text stating that AUTHINFO shouldn't be advertised or used 1000 after COMPRESS to prevent possible CRIME attack (with example). 1002 o Added text stating that a windowBits value of -15 should be used 1003 for inflateInit2(). 1005 o Minor editorial changes. 1007 B.4. Changes since -00 1009 o Made DEFLATE the mandatory to implement compression algorithm. 1011 o Removed the requirement that clients/servers implementing COMPRESS 1012 also implement TLS compression. 1014 o Added an example of a client trying to use an unsupported 1015 compression algorithm. 1017 o Rewrote Compression Efficiency (Section 3) as follows: 1019 * Included a sample listing of which NNTP commands produce which 1020 type of data to be compressed. 1022 * Removed discussion of attachments in binary form and 1023 incompressible file formats. 1025 * Mentioned UUencode and yEnc encoding of attachments. 1027 o Added IANA registry of NNTP compression algorithms. 1029 o Miscellaneous editorial changes submitted by Julien Elie. 1031 Authors' Addresses 1033 Kenneth Murchison 1034 Carnegie Mellon University 1035 5000 Forbes Avenue 1036 Pittsburgh, PA 15213 1037 US 1039 Phone: +1 412 268 1982 1040 EMail: murch@andrew.cmu.edu 1042 Julien Elie 1043 10 allee Clovis 1044 Noisy-le-Grand 93160 1045 France 1047 EMail: julien@trigofacile.com 1048 URI: http://www.trigofacile.com/