idnits 2.17.1 draft-murchison-nntp-compress-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The expression 'MAY NOT', while looking like RFC 2119 requirements text, is not defined in RFC 2119, and should not be used. Consider using 'MUST NOT' instead (if that is what you mean). Found 'MAY NOT' in this paragraph: o Reworded a sentence wrongly using "MAY NOT" (not a key word defined in [RFC2119]). -- The document date (June 23, 2016) is 2863 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'C' is mentioned on line 436, but not defined == Missing Reference: 'S' is mentioned on line 437, but not defined -- Looks like a reference, but probably isn't: '1' on line 232 ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) -- Obsolete informational reference (is this intentional?): RFC 5246 (Obsoleted by RFC 8446) -- Obsolete informational reference (is this intentional?): RFC 7525 (Obsoleted by RFC 9325) Summary: 1 error (**), 0 flaws (~~), 4 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Independent Submission K. Murchison 3 Internet-Draft Carnegie Mellon University 4 Intended status: Standards Track J. Elie 5 Expires: December 25, 2016 June 23, 2016 7 Network News Transfer Protocol (NNTP) Extension for Compression 8 draft-murchison-nntp-compress-05 10 Abstract 12 This document defines an extension to the Network News Transport 13 Protocol (NNTP) that allows a connection to be effectively and 14 efficiently compressed between an NNTP client and server. 16 Status of This Memo 18 This Internet-Draft is submitted in full conformance with the 19 provisions of BCP 78 and BCP 79. 21 Internet-Drafts are working documents of the Internet Engineering 22 Task Force (IETF). Note that other groups may also distribute 23 working documents as Internet-Drafts. The list of current Internet- 24 Drafts is at http://datatracker.ietf.org/drafts/current/. 26 Internet-Drafts are draft documents valid for a maximum of six months 27 and may be updated, replaced, or obsoleted by other documents at any 28 time. It is inappropriate to use Internet-Drafts as reference 29 material or to cite them other than as "work in progress." 31 This Internet-Draft will expire on December 25, 2016. 33 Copyright Notice 35 Copyright (c) 2016 IETF Trust and the persons identified as the 36 document authors. All rights reserved. 38 This document is subject to BCP 78 and the IETF Trust's Legal 39 Provisions Relating to IETF Documents 40 (http://trustee.ietf.org/license-info) in effect on the date of 41 publication of this document. Please review these documents 42 carefully, as they describe your rights and restrictions with respect 43 to this document. Code Components extracted from this document must 44 include Simplified BSD License text as described in Section 4.e of 45 the Trust Legal Provisions and are provided without warranty as 46 described in the Simplified BSD License. 48 Table of Contents 50 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 51 1.1. About TLS-level Compression . . . . . . . . . . . . . . . 3 52 1.2. Conventions Used in This Document . . . . . . . . . . . . 4 53 1.3. Authors' Note . . . . . . . . . . . . . . . . . . . . . . 4 54 2. The COMPRESS Extension . . . . . . . . . . . . . . . . . . . 4 55 2.1. Advertising the COMPRESS Extension . . . . . . . . . . . 4 56 2.2. COMPRESS Command . . . . . . . . . . . . . . . . . . . . 5 57 2.2.1. Usage . . . . . . . . . . . . . . . . . . . . . . . . 5 58 2.2.2. Description . . . . . . . . . . . . . . . . . . . . . 6 59 2.2.3. Examples . . . . . . . . . . . . . . . . . . . . . . 7 60 3. Compression Efficiency . . . . . . . . . . . . . . . . . . . 10 61 3.1. DEFLATE Specificities . . . . . . . . . . . . . . . . . . 12 62 4. Augmented BNF Syntax for the COMPRESS Extension . . . . . . . 12 63 4.1. Commands . . . . . . . . . . . . . . . . . . . . . . . . 13 64 4.2. Capability Entries . . . . . . . . . . . . . . . . . . . 13 65 4.3. General Non-terminals . . . . . . . . . . . . . . . . . . 13 66 5. Summary of Response Codes . . . . . . . . . . . . . . . . . . 13 67 6. Security Considerations . . . . . . . . . . . . . . . . . . . 13 68 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 69 7.1. NNTP Compression Algorithm Registry . . . . . . . . . . . 14 70 7.1.1. Algorithm Name Registration Procedure . . . . . . . . 15 71 7.1.2. Comments on Algorithm Registrations . . . . . . . . . 16 72 7.1.3. Change Control . . . . . . . . . . . . . . . . . . . 16 73 7.2. Registration of the DEFLATE Compression Algorithm . . . . 17 74 7.3. Registration of the NNTP COMPRESS Extension . . . . . . . 17 75 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 18 76 8.1. Normative References . . . . . . . . . . . . . . . . . . 18 77 8.2. Informative References . . . . . . . . . . . . . . . . . 19 78 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 21 79 Appendix B. Document History (to be removed by RFC Editor before 80 publication) . . . . . . . . . . . . . . . . . . . . 21 81 B.1. Changes since -04 . . . . . . . . . . . . . . . . . . . . 21 82 B.2. Changes since -03 . . . . . . . . . . . . . . . . . . . . 21 83 B.3. Changes since -02 . . . . . . . . . . . . . . . . . . . . 21 84 B.4. Changes since -01 . . . . . . . . . . . . . . . . . . . . 22 85 B.5. Changes since -00 . . . . . . . . . . . . . . . . . . . . 23 86 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 23 88 1. Introduction 90 The goal of COMPRESS is to reduce the bandwidth usage of NNTP. 92 Compared to PPP compression [RFC1962] and modem-based compression 93 ([MNP] and [V42bis]), COMPRESS offers greater compression efficiency. 94 COMPRESS can be used together with Transport Layer Security (TLS) 96 [RFC5246], Simple Authentication and Security Layer (SASL) encryption 97 [RFC4422], Virtual Private Networks (VPNs), etc. 99 The point of COMPRESS as an NNTP extension is to act as a compression 100 layer, similarly to a security layer like the one negotiated by 101 STARTTLS [RFC4642]. Compression can therefore benefit to all NNTP 102 commands sent or received after the use of COMPRESS. This facility 103 responds to a long-standing need for NNTP to compress data, that has 104 partially been addressed by unstandardized commands like XZVER, 105 XZHDR, XFEATURE COMPRESS, or MODE COMPRESS. These commands are not 106 wholly satisfactory because they enable compression only for the 107 responses sent by the news server. On the contrary, the COMPRESS 108 command permits to compress data sent by both the client and the 109 server, and removes the constraint of having to implement compression 110 separately in each NNTP command. Besides, the compression level can 111 be dynamically adjusted and optimized at any time during the 112 connection, which even allows to disable compression for certain 113 commands, if need be. If the news client wants to stop compression 114 on a particular connection, it can simply use QUIT ([RFC3977] 115 Section 5.4), and establish a new connection. For these reasons, 116 using other NNTP commands than COMPRESS to enable compression is 117 discouraged once COMPRESS is supported. 119 In order to increase interoperability, it is desirable to have as few 120 different compression algorithms as possible, so this document 121 specifies only one. The DEFLATE algorithm (defined in [RFC1951]) 122 MUST be implemented as part of this extension. This compression 123 algorithm is standard, widely available, and fairly efficient. 125 This specification should be read in conjunction with the NNTP base 126 specification [RFC3977]. In the case of a conflict between these two 127 documents, [RFC3977] takes precedence. 129 1.1. About TLS-level Compression 131 Though data compression is made possible via the use of TLS with NNTP 132 [RFC4642], the best current practice is to disable TLS-level 133 compression as explained in Section 3.3 of [RFC7525]. The COMPRESS 134 command will permit to keep the compression facility in NNTP and 135 control when it is available during a connection. 137 Compared to TLS-level compression [RFC3749], NNTP COMPRESS has the 138 following advantages: 140 o COMPRESS can be implemented easily both by NNTP servers and 141 clients. 143 o COMPRESS benefits from an intimate knowledge of the NNTP 144 protocol's state machine, allowing for dynamic and aggressive 145 optimization of the underlying compression algorithm's parameters. 147 o COMPRESS can be activated after authentication has completed thus 148 reducing the chances that authentication credentials can be leaked 149 via for instance a CRIME attack ([RFC7457] Section 2.6). 151 1.2. Conventions Used in This Document 153 The notational conventions used in this document are the same as 154 those in [RFC3977], and any term not defined in this document has the 155 same meaning as it does in that one. 157 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 158 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 159 document are to be interpreted as described in [RFC2119]. 161 In the examples, commands from the client are indicated with [C], and 162 responses from the server are indicated with [S]. The client is the 163 initiator of the NNTP connection; the server is the other endpoint. 165 1.3. Authors' Note 167 Please write the first letter of "Elie" and the penultimate letter of 168 "allee" with an acute accent wherever possible -- they are 169 respectively U+00C9 ("É" in XML) and U+00E9 ("é" in XML). 170 Also, the letters "ae" in "Baeuerle" should be written as an a-umlaut 171 (U+00E4, "ä" in XML), and the first letter of "Angel" as well as 172 the fifth letter of "Gonzalez" should be written with an acute accent 173 (respectively U+00C1 and U+00E1, that is to say "Á" and "á" 174 in XML). 176 2. The COMPRESS Extension 178 The COMPRESS extension is used to enable data compression on an NNTP 179 connection. 181 This extension provides a new COMPRESS command and has capability 182 label COMPRESS. 184 2.1. Advertising the COMPRESS Extension 186 A server supporting the COMPRESS command as defined in this document 187 will advertise the "COMPRESS" capability label in response to the 188 CAPABILITIES command ([RFC3977] Section 5.2). However, this 189 capability MUST NOT be advertised once a compression layer is active 190 (see Section 2.2.2). This capability MAY be advertised both before 191 and after any use of the MODE READER command ([RFC3977] Section 5.3), 192 with the same semantics. 194 The COMPRESS capability label contains a whitespace-separated list of 195 available compression algorithms. This document defines one 196 compression algorithm: DEFLATE. This algorithm is mandatory to 197 implement and MUST be supported in order to advertise the COMPRESS 198 extension. 200 Future extensions may add additional compression algorithms to this 201 capability. Unrecognized algorithms MUST be ignored by the client. 203 As the COMPRESS command is related to security because it can weaken 204 encryption, cached results of CAPABILITIES from a previous session 205 MUST NOT be relied on, as per Section 12.6 of [RFC3977]. 207 Example: 209 [C] CAPABILITIES 210 [S] 101 Capability list: 211 [S] VERSION 2 212 [S] READER 213 [S] IHAVE 214 [S] COMPRESS DEFLATE X-SHRINK 215 [S] LIST ACTIVE NEWSGROUPS 216 [S] . 218 2.2. COMPRESS Command 220 2.2.1. Usage 222 This command MUST NOT be pipelined. 224 Syntax 225 COMPRESS algorithm 227 Responses 228 206 Compression active 229 403 Unable to activate compression 230 502 Command unavailable [1] 232 [1] If a compression layer is already active, COMPRESS is not a valid 233 command (see Section 2.2.2). 235 Parameters 236 algorithm = Name of compression algorithm (e.g. "DEFLATE") 238 2.2.2. Description 240 The COMPRESS command instructs the server to use the named 241 compression algorithm ("DEFLATE" is the only one defined in this 242 document) for all commands and/or responses after COMPRESS. 244 The client MUST NOT send any further commands until it has seen the 245 result of COMPRESS. 247 If the requested compression algorithm is syntactically incorrect, 248 the server MUST reject the COMPRESS command with a 501 response code 249 ([RFC3977] Section 3.2.1). If the requested compression algorithm is 250 invalid (e.g., is not supported), the server MUST reject the COMPRESS 251 command with a 503 response code ([RFC3977] Section 3.2.1). If the 252 server is unable to activate compression for any reason (e.g., a 253 server configuration or resource problem), the server MUST reject the 254 COMPRESS command with a 403 response code ([RFC3977] Section 3.2.1). 255 Otherwise, the server issues a 206 response code and the compression 256 layer takes effect for both client and server immediately following 257 the CRLF of the success reply. 259 Additionally, the client MUST NOT issue a MODE READER command after 260 activating a compression layer, and a server MUST NOT advertise the 261 MODE-READER capability. 263 Both the client and the server MUST know if there is a compression 264 layer active (for instance via the previous use of the COMPRESS 265 command or the negotiation of a TLS-level compression [RFC3749]). A 266 client MUST NOT attempt to activate compression or negotiate a TLS 267 layer (for instance via the use of the COMPRESS or STARTTLS [RFC4642] 268 commands) if a compression layer is already active. A server MUST 269 NOT return the COMPRESS or STARTTLS capability labels in response to 270 a CAPABILITIES command received after a compression layer is active, 271 and a server MUST reply with a 502 response code if a syntactically 272 valid COMPRESS or STARTTLS command is received while a compression 273 layer is already active. 275 In order to help mitigate leaking authentication credentials via for 276 instance a CRIME attack [CRIME], authentication SHOULD NOT be 277 attempted when a compression layer is active. Consequently, a server 278 SHOULD NOT return any arguments with the AUTHINFO capability label 279 (or SHOULD NOT advertise it at all) in response to a CAPABILITIES 280 command received from an unauthenticated client after a compression 281 layer is active, and such a client SHOULD NOT attempt to utilize any 282 AUTHINFO [RFC4643] commands. It implies that a server SHOULD reply 283 with a 502 response code if a syntactically valid AUTHINFO command is 284 received while a compression layer is already active. 286 For DEFLATE [RFC1951] (as for many other compression algorithms), the 287 compressor can trade speed against quality. The decompressor MUST 288 automatically adjust to the parameters selected by the sender. 289 Consequently, the client and server are both free to pick the best 290 reasonable rate of compression for the data they send. 292 When COMPRESS is combined with TLS [RFC5246] or SASL [RFC4422] 293 security layers, the processing order of the three layers MUST be 294 first COMPRESS, then SASL, and finally TLS. That is, before data is 295 transmitted, it is first compressed. Second, if a SASL security 296 layer has been negotiated, the compressed data is then signed and/or 297 encrypted accordingly. Third, if a TLS security layer has been 298 negotiated, the data from the previous step is signed and/or 299 encrypted accordingly. When receiving data, the processing order 300 MUST be reversed. This ensures that before sending, data is 301 compressed before it is encrypted, independent of the order in which 302 the client issues the COMPRESS, AUTHINFO SASL [RFC4643], and STARTTLS 303 [RFC4642] commands. 305 When compression is active and either the client or the server 306 receives invalid or corrupted compressed data, the receiving end 307 SHOULD immediately close the connection. (Then the sending end will 308 in turn do the same.) 310 2.2.3. Examples 312 Example of layering TLS and NNTP compression: 314 [C] CAPABILITIES 315 [S] 101 Capability list: 316 [S] VERSION 2 317 [S] READER 318 [S] STARTTLS 319 [S] AUTHINFO 320 [S] COMPRESS DEFLATE 321 [S] LIST ACTIVE NEWSGROUPS 322 [S] . 323 [C] STARTTLS 324 [S] 382 Continue with TLS negotiation 325 [TLS negotiation without compression occurs here] 326 [Following successful negotiation, all traffic is encrypted] 327 [C] CAPABILITIES 328 [S] 101 Capability list: 329 [S] VERSION 2 330 [S] READER 331 [S] AUTHINFO USER 332 [S] COMPRESS DEFLATE 333 [S] LIST ACTIVE NEWSGROUPS 334 [S] . 335 [C] AUTHINFO USER fred 336 [S] 381 Enter passphrase 337 [C] AUTHINFO PASS flintstone 338 [S] 281 Authentication accepted 339 [C] CAPABILITIES 340 [S] 101 Capability list: 341 [S] VERSION 2 342 [S] READER 343 [S] POST 344 [S] COMPRESS DEFLATE 345 [S] LIST ACTIVE NEWSGROUPS 346 [S] . 347 [C] COMPRESS DEFLATE 348 [S] 206 Compression active 349 [Henceforth, all traffic is compressed before being encrypted] 351 Example of a server failing to activate compression: 353 [C] CAPABILITIES 354 [S] 101 Capability list: 355 [S] VERSION 2 356 [S] IHAVE 357 [S] COMPRESS DEFLATE 358 [S] . 359 [C] COMPRESS DEFLATE 360 [S] 403 Unable to activate compression 362 Example of attempting to use an unsupported compression algorithm: 364 [C] CAPABILITIES 365 [S] 101 Capability list: 366 [S] VERSION 2 367 [S] IHAVE 368 [S] COMPRESS DEFLATE 369 [S] . 370 [C] COMPRESS X-SHRINK 371 [S] 503 Compression algorithm not supported 373 Example of a server refusing to compress twice: 375 [C] CAPABILITIES 376 [S] 101 Capability list: 377 [S] VERSION 2 378 [S] IHAVE 379 [S] STARTTLS 380 [S] COMPRESS DEFLATE 381 [S] . 382 [C] STARTTLS 383 [S] 382 Continue with TLS negotiation 384 [TLS negotiation with compression occurs here] 385 [Following successful negotiation, all traffic is encrypted] 386 [C] CAPABILITIES 387 [S] 101 Capability list: 388 [S] VERSION 2 389 [S] IHAVE 390 [S] . 391 [C] COMPRESS DEFLATE 392 [S] 502 Compression already active via TLS 394 Example of a server refusing to negotiate a TLS layer after 395 compression has been activated: 397 [C] CAPABILITIES 398 [S] 101 Capability list: 399 [S] VERSION 2 400 [S] IHAVE 401 [S] STARTTLS 402 [S] COMPRESS DEFLATE 403 [S] . 404 [C] COMPRESS DEFLATE 405 [S] 206 Compression active 406 [Henceforth, all traffic is compressed] 407 [C] CAPABILITIES 408 [S] 101 Capability list: 409 [S] VERSION 2 410 [S] IHAVE 411 [S] . 412 [C] STARTTLS 413 [S] 502 DEFLATE compression already active 415 Example of a server not advertising AUTHINFO arguments after 416 compression has been activated: 418 [C] CAPABILITIES 419 [S] 101 Capability list: 420 [S] VERSION 2 421 [S] READER 422 [S] AUTHINFO USER 423 [S] COMPRESS DEFLATE 424 [S] LIST ACTIVE NEWSGROUPS 425 [S] . 426 [C] COMPRESS DEFLATE 427 [S] 206 Compression active 428 [Henceforth, all traffic is compressed] 429 [C] CAPABILITIES 430 [S] 101 Capability list: 431 [S] VERSION 2 432 [S] READER 433 [S] AUTHINFO 434 [S] LIST ACTIVE NEWSGROUPS 435 [S] . 436 [C] AUTHINFO USER fred 437 [S] 502 DEFLATE compression already active 439 3. Compression Efficiency 441 This section is informative, not normative. 443 NNTP poses some unusual problems for a compression layer. 445 Upstream traffic is fairly simple. Most NNTP clients send the same 446 few commands again and again, so any compression algorithm that can 447 exploit repetition works efficiently. The article posting and 448 transfer commands (e.g., POST, IHAVE, and TAKETHIS [RFC4644]) are 449 exceptions; clients that send many article posting or transfer 450 commands may want to surround large multi-line data blocks with a 451 dictionary flush and/or, depending on the compression algorithm, a 452 change of compression level in the same way as is recommended for 453 servers later in this document (Section 3.1). 455 Downstream traffic has the unusual property that several kinds of 456 data are sent, possibly confusing a dictionary-based compression 457 algorithm. 459 One type is NNTP simple responses and NNTP multi-line responses not 460 related to article header/body retrieval (e.g, CAPABILITIES, GROUP, 461 LISTGROUP, LAST, NEXT, STAT, DATE, NEWNEWS, NEWGROUPS, LIST, CHECK 462 [RFC4644], etc). These are highly compressible; zlib using its least 463 CPU-intensive setting compresses typical responses to 25-40% of their 464 original size. 466 Another type is article headers (as retrieved via the HEAD, HDR, 467 OVER, or ARTICLE commands). These are equally compressible, and 468 benefit from using the same dictionary as the NNTP responses. 470 A third type is article body text (as retrieved via the BODY or 471 ARTICLE commands). Text is usually fairly short and includes much 472 ASCII, so the same compression dictionary will do a good job here, 473 too. When multiple messages in the same thread are read at the same 474 time, quoted lines, etc. can often be compressed almost to zero. 476 Finally, non-text article bodies or attachments (as retrieved via the 477 BODY and ARTICLE commands) are transmitted in encoded form, usually 478 Base64 [RFC4648], UUencode [IEEE.1003-2.1992], or yEnc [yEnc]. 480 When already compressed articles or attachments are retrieved, a 481 compression algorithm may be able to compress them, but the format of 482 their encoding is usually not NNTP-like, so the dictionary built 483 while compressing NNTP does not help much. The compressor has to 484 adapt its dictionary from NNTP to the attachment's encoding format, 485 and then back. 487 When attachments are retrieved in Base64 or UUencode form, the 488 Huffman coding usually compresses those to approximatively only 75% 489 of their encoding size. 8-bit compression algorithms such as DEFLATE 490 work well on 8-bit file formats; however, both Base64 and UUencode 491 transform a file into something resembling 6-bit bytes, hiding most 492 of the 8-bit file format from the compressor. 494 On the other end, attachments encoded using a compression algorithm 495 that retains the full 8-bit spectrum, like yEnc, are much more likely 496 to be incompressible. 498 3.1. DEFLATE Specificities 500 When using the zlib library (see [RFC1951]), the functions 501 deflateInit2(), deflate(), inflateInit2(), and inflate() suffice to 502 implement this extension. The windowBits value MUST be in the range 503 -8 to -15 for deflateInit2(), or else it will use the wrong format. 504 The windowBits value SHOULD be -15 for inflateInit2(), or else it 505 will not be able to decompress a stream with a larger window size. 506 deflateParams() can be used to improve compression rate and resource 507 use. In order to improve compression efficiency, the Z_PARTIAL_FLUSH 508 argument to deflate() should always be used to flush data. As far as 509 DEFLATE is concerned, clearing the dictionary never improves 510 compression over the other flushes. On the contrary, having the 32kB 511 dictionary from previous data, no matter how unrelated, can only 512 help. If there are no matching strings in there, then it is simply 513 not referenced. Using Z_FULL_FLUSH clears the dictionary, and 514 consequently always results in compression that is less effective 515 than a Z_PARTIAL_FLUSH. 517 A server can improve downstream compression and the CPU efficiency 518 both of the server and the client if it adjusts the compression level 519 (e.g., using the deflateParams() function in zlib) at the start and 520 end of large non-text multi-line data blocks (before and after 521 'content-lines' in the definition of 'multi-line-data-block' in 522 [RFC3977] Section 9.8). It permits to avoid trying to compress 523 incompressible attachments. Small multi-line data blocks are best 524 left alone. A possible boundary is 5kB. 526 A very simple strategy is to change the compression level to 0 at the 527 start of a multi-line data block provided the first two bytes are 528 either 0x1F 0x8B (as in deflate-compressed files) or 0xFF 0xD8 529 (JPEG), and to keep it at 1-5 the rest of the time. More complex 530 strategies are of course possible, and encouraged. 532 4. Augmented BNF Syntax for the COMPRESS Extension 534 This section describes the formal syntax of the COMPRESS extension 535 using ABNF [RFC5234]. It extends the syntax in Section 9 of 536 [RFC3977], and non-terminals not defined in this document are defined 537 there. The [RFC3977] ABNF should be imported first, before 538 attempting to validate these rules. 540 4.1. Commands 542 This syntax extends the non-terminal , which represents an 543 NNTP command. 545 command =/ compress-command 547 compress-command = "COMPRESS" WS algorithm 549 4.2. Capability Entries 551 This syntax extends the non-terminal , which 552 represents a capability that may be advertised by the server. 554 capability-entry =/ compress-capability 556 compress-capability = "COMPRESS" 1*(WS algorithm) 558 4.3. General Non-terminals 560 algorithm = "DEFLATE" / 1*20alg-char 561 alg-char = UPPER / DIGIT / "-" / "_" 563 5. Summary of Response Codes 565 This section contains a list of each new response code defined in 566 this document and indicates whether it is multi-line, which commands 567 can generate it, what arguments it has, and what its meaning is. 569 Response code 206 570 Generated by: COMPRESS 571 Meaning: compression layer activated 573 6. Security Considerations 575 Security issues are discussed throughout this document. 577 In general, the security considerations of the NNTP core 578 specification ([RFC3977] Section 12) and the DEFLATE compressed data 579 format specification ([RFC1951] Section 6) are applicable here. 581 Implementers should be aware that combining compression with 582 encryption like TLS can sometimes reveal information that would not 583 have been revealed without compression, as explained in Section 6 of 584 [RFC3749]. As a matter of fact, adversaries that observe the length 585 of the compressed data might be able to derive information about the 586 corresponding uncompressed data. The CRIME and the BREACH attacks 587 ([RFC7457] Section 2.6) are examples of such case. 589 In order to help mitigate leaking authentication credentials, this 590 document states in Section 2.2.2 that authentication SHOULD NOT be 591 attempted when a compression layer is active. Therefore, when a 592 client wants to authenticate, compress data, and negotiate a TLS 593 layer (without TLS-level compression) in the same NNTP connection, it 594 SHOULD use the STARTTLS, AUTHINFO, and COMPRESS commands in that 595 order. Of course instead of using the STARTTLS command, a client can 596 also use implicit TLS, that is to say it begins the TLS negotiation 597 immediately upon connection on a separate port dedicated to NNTP over 598 TLS. 600 NNTP commands other than AUTHINFO are not believed to divulgate 601 confidential information as long as only public Netnews newsgroups 602 and articles are accessed. That is why this specification only adds 603 a restriction to the use of AUTHINFO when a compression layer is 604 active. In case confidential articles are accessed in private 605 newsgroups, special care is needed: implementations SHOULD NOT 606 compress confidential data together with public data when a security 607 layer is active, for the same reasons as mentioned above in this 608 Section. 610 Additionally, implementations MAY ensure that the contents of two 611 distinct confidential articles are not compressed together. This can 612 be achieved for instance with DEFLATE by clearing the compression 613 dictionary each time a confidential article is sent. More complex 614 implementations are of course possible, and encouraged. 616 Implementations SHOULD use a default configuration with disabled 617 compression when a security layer is active, and MUST support an 618 option to allow compression to be enabled when a security layer is 619 active. Such an option can be either with global scope or server/ 620 connection based. Implementations MAY unconditionally allow 621 compression to be enabled when no security layer is active. 623 Future extensions to NNTP that define commands conveying confidential 624 data SHOULD ensure to state that these confidential data SHOULD NOT 625 be compressed together with public data when a security layer is 626 active. 628 7. IANA Considerations 630 7.1. NNTP Compression Algorithm Registry 632 The NNTP Compression Algorithm registry will be maintained by IANA. 633 The registry will be available at . 636 The purpose of this registry is not only to ensure uniqueness of 637 values used to name NNTP compression algorithms, but also to provide 638 a definitive reference to technical specifications detailing each 639 NNTP compression algorithm available for use on the Internet. 641 An NNTP compression algorithm is either a private algorithm, or its 642 name is included in the IANA NNTP Compression Algorithm registry (in 643 which case it is a "registered NNTP compression algorithm"). 644 Different entries in the registry MUST use different names. 646 Any name that conforms to the syntax of an NNTP compression algorithm 647 name (Section 4.3) can be used. However, the name of a registered 648 NNTP compression algorithm MUST NOT begin with "X". 650 To avoid the risk of a clash with a future registered NNTP 651 compression algorithm, the names of private compression algorithms 652 SHOULD begin with "X-". 654 If the server advertises a registered NNTP compression algorithm, it 655 MUST implement the compression algorithm so as to fully conform with 656 its related specification. If it does not implement the compression 657 algorithm as specified, it MUST NOT list its registered name in the 658 arguments list of the COMPRESS capability label. In that case, it 659 MAY, but SHOULD NOT, provide a private algorithm (not listed, or 660 listed with a different name) that implements the compression 661 algorithm differently. 663 A server MUST NOT send different response codes to COMPRESS in 664 response to the availability or use of a private compression 665 algorithm. 667 The procedure detailed in Section 7.1.1 is to be used for 668 registration of a value naming a specific individual compression 669 algorithm. 671 Comments may be included in the registry as discussed in 672 Section 7.1.2 and may be changed as discussed in Section 7.1.3. 674 7.1.1. Algorithm Name Registration Procedure 676 IANA will register new NNTP compression algorithm names on a First 677 Come First Served basis, as defined in BCP 26 [RFC5226]. IANA has 678 the right to reject obviously bogus registration requests, but will 679 perform no review of claims made in the registration form. 681 Registration of an NNTP compression algorithm is requested by filling 682 in the following template and sending it via electronic mail to IANA 683 at : 685 Subject: Registration of NNTP compression algorithm X 687 NNTP compression algorithm name: 689 Security considerations: 691 Published specification (recommended): 693 Contact for further information: 695 Intended usage: (One of COMMON, LIMITED USE, or OBSOLETE) 697 Owner/Change controller: 699 Note: (Any other information that the author deems relevant may be 700 added here.) 702 While this registration procedure does not require expert review, 703 authors of NNTP compression algorithms are encouraged to seek 704 community review and comment whenever that is feasible. Authors may 705 seek community review by posting a specification of their proposed 706 algorithm as an Internet-Draft. NNTP compression algorithms intended 707 for widespread use should be standardized through the normal IETF 708 process, when appropriate. 710 7.1.2. Comments on Algorithm Registrations 712 Comments on a registered NNTP compression algorithm should first be 713 sent to the "owner" of the algorithm and/or to the mailing list for 714 the now concluded IETF NNTPEXT working group (). 717 Submitters of comments may, after a reasonable attempt to contact the 718 owner and/or the above mailing list, request IANA to attach their 719 comment to the NNTP compression algorithm registration itself by 720 sending mail to . At IANA's sole discretion, IANA may 721 attach the comment to the NNTP compression algorithm's registration. 723 7.1.3. Change Control 725 Once an NNTP compression algorithm registration has been published by 726 IANA, the owner may request a change to its definition. The change 727 request follows the same procedure as the initial registration 728 request. 730 The owner of an NNTP compression algorithm may pass responsibility 731 for the algorithm to another person or agency by informing IANA; this 732 can be done without discussion or review. 734 The IESG may reassign responsibility for an NNTP compression 735 algorithm. The most common case of this will be to enable changes to 736 be made to algorithms where the owner of the registration has died, 737 has moved out of contact, or is otherwise unable to make changes that 738 are important to the community. 740 NNTP compression algorithm registrations MUST NOT be deleted; 741 algorithms that are no longer believed appropriate for use can be 742 declared OBSOLETE by a change to their "intended usage" field; such 743 algorithms will be clearly marked in the registry published by IANA. 745 The IESG is considered to be the owner of all NNTP compression 746 algorithms that are on the IETF standards track. 748 7.2. Registration of the DEFLATE Compression Algorithm 750 This section gives a formal definition of the DEFLATE compression 751 algorithm as required by Section 7.1.1 for the IANA registry. 753 NNTP compression algorithm name: DEFLATE 755 Security considerations: See Section 6 of this document 757 Published specification: This document 759 Contact for further information: Authors of this document 761 Intended usage: COMMON 763 Owner/Change controller: IESG 765 Note: This algorithm is mandatory to implement 767 7.3. Registration of the NNTP COMPRESS Extension 769 This section gives a formal definition of the COMPRESS extension as 770 required by Section 3.3.3 of [RFC3977] for the IANA registry. 772 o The COMPRESS extension allows an NNTP connection to be effectively 773 and efficiently compressed. 775 o The capability label for this extension is "COMPRESS", whose 776 arguments list the available compression algorithms. 778 o This extension defines one new command, COMPRESS, whose behavior, 779 arguments, and responses are defined in Section 2.2. 781 o This extension does not associate any new responses with pre- 782 existing NNTP commands. 784 o This extension does affect the overall behavior of both server and 785 client, in that after successful use of the COMPRESS command, all 786 communication is transmitted in a compressed format. 788 o This extension does not affect the maximum length of commands or 789 initial response lines. 791 o This extension does not alter pipelining, but the COMPRESS command 792 cannot be pipelined. 794 o Use of this extension does alter the capabilities list; once the 795 COMPRESS command has been used successfully, the COMPRESS 796 capability can no longer be advertised by CAPABILITIES. 797 Additionally, the STARTTLS and MODE-READER capabilities MUST NOT 798 be advertised, and the AUTHINFO capability label SHOULD either 799 return no arguments or no longer be advertised after successful 800 execution of the COMPRESS command. 802 o This extension does not cause any pre-existing command to produce 803 a 401, 480, or 483 response code. 805 o This extension is unaffected by any use of the MODE READER 806 command; however, the MODE READER command MUST NOT be used in the 807 same session following a successful execution of the COMPRESS 808 command. 810 o The STARTTLS command MUST NOT be used, and the AUTHINFO command 811 SHOULD NOT be used, in the same session following a successful 812 execution of the COMPRESS command. 814 o Published Specification: This document. 816 o Contact for Further Information: Authors of this document. 818 o Change Controller: IESG . 820 8. References 822 8.1. Normative References 824 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 825 Requirement Levels", BCP 14, RFC 2119, 826 DOI 10.17487/RFC2119, March 1997, 827 . 829 [RFC3977] Feather, C., "Network News Transfer Protocol (NNTP)", 830 RFC 3977, DOI 10.17487/RFC3977, October 2006, 831 . 833 [RFC4642] Murchison, K., Vinocur, J., and C. Newman, "Using 834 Transport Layer Security (TLS) with Network News Transfer 835 Protocol (NNTP)", RFC 4642, DOI 10.17487/RFC4642, October 836 2006, . 838 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 839 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 840 DOI 10.17487/RFC5226, May 2008, 841 . 843 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 844 Specifications: ABNF", STD 68, RFC 5234, 845 DOI 10.17487/RFC5234, January 2008, 846 . 848 8.2. Informative References 850 [CRIME] Rizzo, J. and T. Duong, "The CRIME Attack", Ekoparty 851 Security Conference, 2012. 853 [IEEE.1003-2.1992] 854 Institute of Electrical and Electronics Engineers, 855 "Information Technology - Portable Operating System 856 Interface (POSIX) - Part 2: Shell and Utilities (Vol. 1)", 857 IEEE Standard 1003.2, 1992. 859 [MNP] Held, G., "The Complete Modem Reference", Second 860 Edition, Wiley Professional Computing, May 1994. 862 [RFC1951] Deutsch, P., "DEFLATE Compressed Data Format Specification 863 version 1.3", RFC 1951, DOI 10.17487/RFC1951, May 1996, 864 . 866 [RFC1962] Rand, D., "The PPP Compression Control Protocol (CCP)", 867 RFC 1962, DOI 10.17487/RFC1962, June 1996, 868 . 870 [RFC3749] Hollenbeck, S., "Transport Layer Security Protocol 871 Compression Methods", RFC 3749, DOI 10.17487/RFC3749, May 872 2004, . 874 [RFC4422] Melnikov, A., Ed. and K. Zeilenga, Ed., "Simple 875 Authentication and Security Layer (SASL)", RFC 4422, 876 DOI 10.17487/RFC4422, June 2006, 877 . 879 [RFC4643] Vinocur, J. and K. Murchison, "Network News Transfer 880 Protocol (NNTP) Extension for Authentication", RFC 4643, 881 DOI 10.17487/RFC4643, October 2006, 882 . 884 [RFC4644] Vinocur, J. and K. Murchison, "Network News Transfer 885 Protocol (NNTP) Extension for Streaming Feeds", RFC 4644, 886 DOI 10.17487/RFC4644, October 2006, 887 . 889 [RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data 890 Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006, 891 . 893 [RFC4978] Gulbrandsen, A., "The IMAP COMPRESS Extension", RFC 4978, 894 DOI 10.17487/RFC4978, August 2007, 895 . 897 [RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer Security 898 (TLS) Protocol Version 1.2", RFC 5246, 899 DOI 10.17487/RFC5246, August 2008, 900 . 902 [RFC7457] Sheffer, Y., Holz, R., and P. Saint-Andre, "Summarizing 903 Known Attacks on Transport Layer Security (TLS) and 904 Datagram TLS (DTLS)", RFC 7457, DOI 10.17487/RFC7457, 905 February 2015, . 907 [RFC7525] Sheffer, Y., Holz, R., and P. Saint-Andre, 908 "Recommendations for Secure Use of Transport Layer 909 Security (TLS) and Datagram Transport Layer Security 910 (DTLS)", BCP 195, RFC 7525, DOI 10.17487/RFC7525, May 911 2015, . 913 [V42bis] International Telecommunications Union, "Data compression 914 procedures for data circuit-terminating equipment (DCE) 915 using error correction procedures", ITU-T Recommendation 916 V.42bis, January 1990. 918 [yEnc] Helbing, J., "yEnc - Efficient encoding for Usenet and 919 eMail", March 2002, . 921 Appendix A. Acknowledgements 923 This document draws heavily on ideas in [RFC4978] by Arnt Gulbrandsen 924 and a large portion of this text was borrowed from that 925 specification. 927 The authors would like to thank the following individuals for 928 contributing their ideas and support for writing this specification: 929 Mark Adler, Russ Allbery, Michael Baeuerle, Angel Gonzalez, and Brian 930 Peterson. 932 Appendix B. Document History (to be removed by RFC Editor before 933 publication) 935 B.1. Changes since -04 937 o Reworded a sentence wrongly using "MAY NOT" (not a key word 938 defined in [RFC2119]). 940 o Uppercased a "must" and a "should" in Section 3.1. 942 B.2. Changes since -03 944 o Added a naming convention for NNTP compression algorithms. 945 Improve the wording of registered vs private compression 946 algorithms. 948 o If a registered NNTP compression algorithm is advertised, it MUST 949 fully conform with its related specification. 951 o Fixed the wording of security considerations to reflect that the 952 threat appears when public and confidential data are compressed 953 together inside a security layer. Thanks to Angel Gonzalez for 954 pointing that. 956 o The default configuration SHOULD be disabled compression when a 957 security layer is active. 959 o COMPRESS acts as a compression layer, not a transport layer. 961 o Minor editorial changes. 963 B.3. Changes since -02 965 o Added text stating that the receiving end SHOULD terminate the 966 connection when receiving invalid or corrupted compressed data. 968 o Explained why COMPRESS permits to do better than existing 969 unstandardized commands like XZVER, XZHDR, MODE COMPRESS, and 970 XFEATURE GZIP. 972 o Added an example of AUTHINFO command when compression is active. 974 o The LIST capability label was missing in the examples when READER 975 was also advertised. 977 o Improved an example to send CAPABILITIES after successful 978 authentication. 980 o Mentioned that COMPRESS is related to security. CAPABILITIES is 981 therefore sent again after COMPRESS. 983 o Re-added discussion of attachments in binary form and 984 incompressible file formats. Improve the discussion about 985 flushes, and add a specific section about DEFLATE. 987 o Changed a MUST NOT to SHOULD NOT for the use of AUTHINFO after 988 COMPRESS. 990 o Algorithm names are case-insensitive. 992 o Mentioned the use of the 501 response code. 994 o Added the Security Considerations Section. 996 o Added Julien Elie as co-author of this document. 998 o Minor editorial changes. 1000 B.4. Changes since -01 1002 o Switched to using 206 response code when compression has been 1003 activated. 1005 o Added text stating that TLS-level compression is susceptible to 1006 CRIME attack and current BCP is to disable it. 1008 o Added text stating that AUTHINFO shouldn't be advertised or used 1009 after COMPRESS to prevent possible CRIME attack (with example). 1011 o Added text stating that a windowBits value of -15 should be used 1012 for inflateInit2(). 1014 o Minor editorial changes. 1016 B.5. Changes since -00 1018 o Made DEFLATE the mandatory to implement compression algorithm. 1020 o Removed the requirement that clients/servers implementing COMPRESS 1021 also implement TLS compression. 1023 o Added an example of a client trying to use an unsupported 1024 compression algorithm. 1026 o Rewrote Compression Efficiency (Section 3) as follows: 1028 * Included a sample listing of which NNTP commands produce which 1029 type of data to be compressed. 1031 * Removed discussion of attachments in binary form and 1032 incompressible file formats. 1034 * Mentioned UUencode and yEnc encoding of attachments. 1036 o Added IANA registry of NNTP compression algorithms. 1038 o Miscellaneous editorial changes submitted by Julien Elie. 1040 Authors' Addresses 1042 Kenneth Murchison 1043 Carnegie Mellon University 1044 5000 Forbes Avenue 1045 Pittsburgh, PA 15213 1046 US 1048 Phone: +1 412 268 1982 1049 EMail: murch@andrew.cmu.edu 1051 Julien Elie 1052 10 allee Clovis 1053 Noisy-le-Grand 93160 1054 France 1056 EMail: julien@trigofacile.com 1057 URI: http://www.trigofacile.com/