idnits 2.17.1 draft-preston-ftpext-deflate-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents -- however, there's a paragraph with a matching beginning. Boilerplate error? == No 'Intended status' indicated for this document; assuming Proposed Standard == It seems as if not all pages are separated by form feeds - found 0 form feeds but 12 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (April 2004) is 7315 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '6' is defined on line 504, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2234 (ref. '3') (Obsoleted by RFC 4234) ** Downref: Normative reference to an Informational RFC: RFC 1950 (ref. '5') ** Downref: Normative reference to an Informational RFC: RFC 1951 (ref. '6') ** Downref: Normative reference to an Informational RFC: RFC 2577 (ref. '9') Summary: 8 errors (**), 0 flaws (~~), 5 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 J. Preston 2 Internet Draft NSC 3 Document: draft-preston-ftpext-deflate-02.txt TJ Saunders 4 Expires: November 2004 April 2004 6 Deflate transmission mode for FTP 8 Status of this Memo 10 This document is an Internet-Draft and is subject to all 11 provisions of Section 10 of RFC 2026. 13 Internet-Drafts are working documents of the Internet Engineering 14 Task Force (IETF), its areas, and its working groups. Note that 16 other groups may also distribute working documents as Internet- 17 Drafts. 19 Internet-Drafts are draft documents valid for a maximum of six 20 months and may be updated, replaced, or obsoleted by other 21 documents at any time. It is inappropriate to use Internet-Drafts 22 as reference material or to cite them other than as "work in 23 progress." 25 The list of current Internet-Drafts can be accessed at 26 http://www.ietf.org/ietf/1id-abstracts.txt 27 The list of Internet-Draft Shadow Directories can be accessed at 28 http://www.ietf.org/shadow.html. 30 Abstract 32 This document defines an optional extension to RFC 959, "FILE 33 TRANSFER PROTOCOL (FTP)" (October 1985). It specifies a new 34 "deflate" transmission mode designed to increase network bandwidth 35 by compressing data using existing techniques. 37 Table of Contents 39 1. Introduction...................................................2 40 2. Document Conventions...........................................2 41 2.1 Basic Tokens...............................................3 42 3. Deflate Transmission Mode......................................3 43 3.1 Client-server Interaction..................................4 44 3.2 Overview...................................................4 45 3.3 Compression Engine.........................................4 46 3.3.1 ZLIB Compression Engine..............................4 47 3.4 Syntax.....................................................5 48 3.5 FEAT Response..............................................6 49 3.5.1 FEAT Examples........................................6 51 3.6 OPTS Features..............................................7 52 3.6.1 Standard Opt-names...................................8 53 3.6.1.1 ENGINE Syntax..................................8 54 3.6.1.2 METHOD Syntax..................................8 55 3.6.1.3 LEVEL Syntax...................................8 56 3.6.1.4 EXTRA Syntax...................................8 57 3.6.1.5 BLOCKSIZE Syntax...............................9 58 3.6.2 OPTS Examples........................................9 59 3.7 Error Recovery and Restart.................................9 60 4. Security Considerations.......................................10 61 5. References....................................................10 62 6. Copyright.....................................................11 63 7. Authors' Addresses............................................12 65 1. Introduction 67 As the Internet grows, modern devices and networking environments 68 create new performance challenges for the File Transfer Protocol 69 (FTP) [1]. One solution to this problem, which is addressed in the 70 FTP "compress" transmission mode, is to compress file and system 71 data to maximize network resources. However, the original system is 72 designed to reduce ASCII text with repetitive characters and is 73 unsuitable in many applications because it can add significant 74 network overhead to binary transfers. This document enhances the 75 capabilities of FTP by introducing a new "deflate" transmission mode 76 that: 78 * increases network throughput and decreases transfer time 80 * effectively compresses ASCII and binary data 82 * requires a minimum amount of control information 84 * provides error recovery and data integrity options 86 * includes a mechanism to negotiate compression parameters to 87 balance CPU, memory and data requirements 89 * is extensible to accommodate future compression techniques 91 2. Document Conventions 93 This document makes use of the conventions defined in BCP 14 [2] 94 which includes the explanation of capitalized imperative words MUST, 95 SHOULD, MAY, SHOULD NOT and MUST NOT. Any syntax is defined using 96 Augmented BNF (ABNF) as specified in RFC 2234 [3]. 98 The terms "reply", "user", "file", "pathname", "FTP commands", 99 "DTP", "user-FTP process", "user-PI", "user-DTP", "server-FTP 100 process", "server-PI", "server-DTP", "mode", "type", "NVT", 101 "control connection", "data connection", "transmission mode", 102 "binary" and "ASCII" are all used here as defined in STD 9 [1]. 104 In addition, this specification makes use of the terms "compression 105 engine" and "compression method." A compression engine is a 106 hardware or software component that implements a compression method. 108 The compression method is a process that reduces the size of 109 computer data. 111 2.1 Basic Tokens 113 This document imports the core definitions given in Appendix A of 114 RFC 2234 [3] which includes the ABNF elements like ALPHA, DIGIT, SP, 115 etc. The following terms are added for use in this document: 117 TCHAR = VCHAR / SP / HTAB ; visible plus white space 118 RCHAR = ALPHA / DIGIT / "," / "." / ":" / "!" / 119 "@" / "#" / "$" / "%" / "^" / 120 "&" / "(" / ")" / "-" / "_" / 121 "+" / "?" / "/" / "\" / "'" / 122 DQUOTE ; <"> -- double quote character (%x22) 123 SCHAR = RCHAR / "=" ; 125 The VCHAR (from [3]), RCHAR, SCHAR, and TCHAR types give basic 126 character types from varying sub-sets of the ASCII character set for 127 use in various commands and responses. 129 token = 1*RCHAR 131 A "token" is a string whose precise meaning depends upon the context 132 in which it is used. In some cases it will be a value from a set of 133 possible values and in others it might be a string invented by one 134 party for an FTP conversation. 136 Note that in ABNF, string literals are case insensitive. That 137 convention is preserved in this document, and implies that FTP 138 commands added by this specification have names that can be 139 represented in any case. For example, "MODE" is the same as "mode" 140 and "Mode". However, ALPHA characters are case sensitive which 141 implies a token can have an exact value. That implication is 142 correct, except where explicitly stated to the contrary in this 143 document, or in some other specification which defines the values 144 this document specifies be used in a particular context. 146 3. Deflate Transmission Mode 148 The deflate extension introduces a fourth transmission mode to FTP 149 by updating the transfer mode (MODE) command. It employs general 150 purpose compression methods to reduce data for efficient transfers. 152 The following codes are assigned for transfer modes: 154 S Stream 155 B Block 156 C Compressed 157 Z Deflate 159 The default transfer mode remains Stream. 161 3.1 Client-server Interaction 163 The user-FTP process sends the MODE Z command to request compressed 164 data transfers. If the server-FTP process accepts the request, then 165 deflate transmission mode will be used for all data transfers until 166 the client switches to another mode. 168 3.2 Overview 170 In deflate transmission mode, data is compressed and transmitted as 171 a stream of octets (8 bit bytes). The sender and receiver rely on a 172 compression engine to perform compression operations 173 (deflate/inflate) and maintain state. There is no restriction on 174 the representation type used; record structures are allowed. 176 Since there is no fixed compression format, both FTP hosts MUST 177 process data until the compression engine reports an end-of-file 178 (EOF) state or data error. Closing the data connection is not a 179 sufficient method to end transfers because there may be pending 180 information. 182 If an FTP process encounters an error while compressing or 183 decompressing the data stream, it SHOULD discard all information 184 after that point and cancel the transfer using the procedures 185 described in STD 9 [1]. 187 3.3 Compression Engine 189 Each compression engine generates a unique data stream that MAY 190 consist of the following parts: header and control information, 191 compressed data, integrity checkpoints and end-of-file (EOF) 192 markers. The compression engine MUST support an EOF mechanism and 193 MUST NOT send non-essential structures like version headers. 195 3.3.1 ZLIB Compression Engine 197 All FTP processes that support deflate transmission mode MUST 198 support the ZLIB compressed data format specified in RFC 1950 [5]. 200 The ZLIB compression method, an LZ77 variant called deflation, 201 provides a lossless compressed data format that: 203 * is independent of CPU type, operating system, file system 204 and character set and is therefore ideal for network 205 communications 207 * provides a number of different compression settings 208 (ratios are in the order of 2:1 to 5:1) that accommodates 209 a wide range of CPU, memory and data requirements 211 * minimizes control data overhead (approximately 0.02% for 212 large data streams) 214 * provides integrity checks 216 * can be implemented readily in a manner not covered by 217 patents, and hence can be practiced freely 219 In the worst case, ZLIB reverts to stored (uncompressed) blocks 220 making the deflate data stream analogous to STREAM transmission 221 mode. 223 By default, compliant FTP processes MUST support compression method 224 8 and transmit the CMF, FLG and ADLER32 information in the data 225 stream. If bandwidth or processing requirements are a concern, 226 these restrictions can be negotiated with the OPTS command. 228 3.4 Syntax 230 The deflate extension modifies the MODE command by adding the 231 parameter "Z": 233 Request: 235 mode = "MODE" SP "Z" CRLF 237 Response: 239 mode-response = mode-good / mode-bad 240 mode-good = "200" SP response-message CRLF 241 mode-bad = "451" SP response-message CRLF / 242 = "501" SP response-message CRLF 243 response-message = *TCHAR 245 A "mode-good" response (200 reply) MUST be sent when the "Z" 246 parameter is recognized and the current compression settings are 247 appropriate. An "mode-bad" response is sent in other cases. The 248 451 reply should be used when the current compression settings or 249 some other temporary condition at the server prevent the command 250 from being accepted; but a changed environment for the server-FTP 251 process may permit the command to succeed. A 501 reply is 252 appropriate for a permanent error. 254 3.5 FEAT Response 256 If the server-FTP process supports the feature (FEAT) command 257 specified in RFC 2389 [4], then it MUST include a "MODE Z" feature 258 line. This string indicates required support for the extension and 259 lists the names of additional compression engines: 261 mode-feat = SP "MODE" SP "Z" [SP eng-list] CRLF 262 eng-list = *(eng-desc ",") 263 eng-desc = 1*(eng-name eng-opts) 264 eng-name = ALPHA*(ALPHA / DIGIT / "-" / ".") 265 eng-opts = *("(" opts-list ")") 266 opts-list = 1*(opt-name ",") 267 opt-name = ALPHA*(ALPHA / DIGIT) 269 If eng-list is not present, then the server-FTP process is informing 270 the client that ZLIB is the only engine available. 272 The "MODE Z" feature line string is not case sensitive, but SHOULD 273 be transmitted in upper case. 275 In the case where the server-FTP process does not support the FEAT 276 command, the user-FTP can negotiate the deflate extension by sending 277 the "MODE Z" request. The server would respond with a positive 278 (200) reply and the default compression settings would be effective. 280 3.5.1 FEAT Examples 282 The following examples contrast three servers with deflate support. 283 The first server advertises ZLIB and BZIP2 capabilities, while the 284 second indicates the required ZLIB engine and the third implements 285 ZLIB and a proprietary compression engine. 287 C> FEAT 288 S> 211-Extensions supported: 289 S> =85 290 S> MODE Z BZIP2(LEVEL,BLOCKSIZE) 291 S> =85 292 S> 211 End. 294 and 296 C> FEAT 297 S> 211-Extensions supported: 299 S> =85 300 S> MODE Z 301 S> =85 302 S> 211 End. 304 and 306 C> FEAT 307 S> 211-Extensions supported: 308 S> =85 309 S> MODE Z ENG(SETTING1,SETTING2) 310 S> =85 311 S> 211 End. 313 The ellipses indicate place holders where other features may be 314 included, and are not required. A one space indentation of the 315 feature line is mandatory [4]. 317 3.6 OPTS Features 319 The user-FTP process may specify alternate compression settings with 320 the OPTS command [4]. All subsequent transfers will use these 321 settings until another OPTS request is sent. The format is 322 specified by: 324 opts = "OPTS" SP "MODE" SP "Z" [ SP 1*(name SP value ",") ] 325 name = ALPHA*(ALPHA / DIGIT) 326 value = 1*RCHAR 328 When the client sends an "OPTS MODE Z" command, the server will 329 examine each opt-name and opt-value pair and update the compression 330 engine. An OPTS request with no parameters will cause the server- 331 FTP process to revert to the default compression settings outlined 332 in this document. If the server encounters an invalid or 333 unsupported opt-name or opt-value the OPTS request will be rejected. 335 Note the server-FTP process MUST reject any MODE Z requests during 336 data transfers. 338 If the server-FTP process accepts an OPTS request, it MUST respond 339 with a positive (200) reply. Otherwise, a negative (501) response 340 should be sent. 342 The deflate extension does not require the server to support all of 343 the opt-name and opt-value parameters defined in this documented. 344 If the server-FTP process encounters an invalid or unsupported 345 option, it SHOULD return the opt-name in the error reply. In the 346 worst case, where all OPTS requests are rejected, the FTP processes 347 revert to the default compression settings. 349 If new opt-name parameters are required, the server-FTP process MUST 350 include the labels in the feature string enclosed in brackets and 351 separated by commas. For example, "MODE Z ENG(SETTING1,SETTING2)." 352 The apparatus defined in this specification should be able to handle 353 any routine compression setting. 355 3.6.1 Standard Opt-names 357 This document defines a standard set of opt-names as follows: 358 ENGINE, METHOD, LEVEL, EXTRA and BLOCKSIZE. Each opt-name is case 359 insensitive, or in other words, "ENGINE" is the same as "Engine" and 360 "engine". 362 3.6.1.1 ENGINE Syntax 364 The syntax of the ENGINE option follows: 366 eng-option = eng-label SP eng-value 367 eng-label = "ENGINE" 368 eng-value = ALPHA*(ALPHA / DIGIT / "-" / ".") 370 3.6.1.2 METHOD Syntax 372 The METHOD option allows the FTP processes to negotiate the 373 compression method. The syntax of the METHOD option follows: 375 mth-option = mth-label SP mth-value 376 mth-label = "METHOD" 377 mth-value = 1*DIGIT 379 3.6.1.3 LEVEL Syntax 381 The LEVEL option allows the FTP processes to negotiate the 382 compression level. It will influence the processing requirements 383 and length of the compressed stream. The syntax of the LEVEL option 384 follows: 386 lvl-option = lvl-label SP lvl-value 387 lvl-label = "LEVEL" 388 lvl-value = 1*DIGIT 390 3.6.1.4 EXTRA Syntax 392 The EXTRA option allows the FTP processes to negotiate the 393 transmission of non-essential compression information (like version 394 headers and trailers). This option does not apply to all compression 395 engines. The syntax of the EXTRA option follows: 397 ext-option = ext-label SP ext-value 398 ext-label = "EXTRA" 399 ext-value = ext-enable / ext-disable 400 ext-enable = "ON" 401 ext-disable = "OFF" 403 3.6.1.5 BLOCKSIZE Syntax 405 The BLOCKSIZE option is for compression engines that use block 406 sorting algorithms. It influences the compression ratio and 407 processing requirements. The syntax of the BLOCKSIZE option 408 follows: 410 blk-option = blk-label SP blk-value 411 blk-label = "BLOCKSIZE" 412 blk-value = 1*DIGIT 414 The blk-value is specified in octets. 416 3.6.2 OPTS Examples 418 The following examples illustrate how a client would change the ZLIB 419 compression options and configure a new compression engine. 421 C> OPTS MODE Z LEVEL 9 422 S> 200 MODE Z LEVEL set to 9. 423 C> OPTS MODE Z BLOCKSIZE 8192 424 S> 501 MODE Z BLOCKSIZE is not available. 425 C> OPTS MODE Z METHOD 9 426 S> 501 MODE Z METHOD 9 is invalid. 427 C> OPTS MODE Z ENGINE ZLIB LEVEL 9 EXTRA OFF 428 S> 200- MODE Z ENGINE set to ZLIB. 429 S> 200- MODE Z LEVEL set to 9. 430 S> 200 MODE Z EXTRA set to OFF. 431 C> OPTS MODE Z ENGINE ZLIB LEVEL 9 METHOD 15 432 S> 501 MODE Z METHOD 15 is invalid. 434 and 436 C> OPTS MODE Z ENGINE BZIP2 BLOCKSIZE 8192 437 S> 200- MODE Z ENGINE set to BZIP2. 438 S> 200 MODE Z BLOCKSIZE set to 8192. 439 C> OPTS MODE Z ENGINE ZLIB BLOCKSIZE 8192 440 S> 501 MODE Z BLOCKSIZE is not available. 442 3.7 Error Recovery and Restart 443 In deflate transmission mode, it is not possible to insert restart 444 markers into the data stream because they would be indistinguishable 445 from compressed data, and the user-FTP can alter the data 446 representation by changing compression settings between transfers. 447 However, it is possible to define a restart mechanism by specifying 448 a byte offset into the uncompressed data stream. 450 The logic for this system is similar to the restart mechanisms 451 specified in RFC 1123 [7] and other Internet Drafts [8] for STREAM 452 transmission mode with the following addendum. In a compressed data 453 stream, the output will always be exactly the same as the input, 454 thus an offset will always represent the same position within a 455 file. 457 If the user-FTP process plans to restart a retrieve (RETR) request, 458 it will directly calculate the restart marker, and send the 459 uncompressed offset in the restart (REST) command. The server will 460 skip to the specified file position. When the transfer continues, 461 both FTP processes will operate with a new compressed data stream. 463 The store (STOR) process works in the same manner as the retrieve 464 system. However, the sender must determine how much data was 465 previously received and expanded, with the SIZE [8] command or an 466 alternate method, before resuming the transfer. 468 4. Security Considerations 470 The deflate extension does not introduce any protocol related 471 security issues. However, some compression settings may impose a 472 considerable load on the FTP server, which could lead to denial of 473 service attacks, and compression engines not described in this 474 document may contain security vulnerabilities. If these operational 475 risks are a concern, then implementers should consider limiting 476 server resources or denying problematic settings. 478 In some cases, deflate transmission mode can reduce the demands on 479 the server. For example, in a secure FTP session, the combined 480 process of compressing and encrypting data is less expensive than 481 sending raw encrypted data; and fewer secure renegotiations are 482 required because of the shorter transfer times. 484 A general discussion of issues related to the security of FTP can be 485 found in RFC 2577 [9]. 487 5. References 489 [1] Postel, J., Reynolds, J., "File Transfer Protocol (FTP)", STD 9, 490 RFC 959, October 1985 492 [2] Bradner, S., "Key words for use in RFCs to Indicate Requirement 493 Levels", BCP 14, RFC 2119, March 1997 495 [3] Crocker, D., Overell, P., "Augmented BNF for Syntax 496 Specifications: ABNF", RFC 2234, November 1997 498 [4] Hethmon, P., Elz, R., "Feature negotiation mechanism for the 499 File Transfer Protocol", RFC 2389, August 1998 501 [5] Deutsch, P., "ZLIB Compressed Data Format Specification version 502 3.3", RFC 1950, May 1996 504 [6] Deutsch, P., "DEFLATE Compressed Data Format Specification 505 version 1.3", RFC 1951, May 1996 507 [7] Braden, R., "Requirements for Internet Hosts -- Application and 508 Support", RFC 1123, October 1989 510 [8] P., Elz, R., Hethmon, "Extensions to FTP", Internet Draft 511 (draft-ietf-ftpext-mlst-16), September 2002 513 [9] Allman, M., Ostermann, S., "FTP Security Considerations", RFC 514 2577, May 1999 516 6. Copyright 518 Copyright (C) The Internet Society (1999). All Rights Reserved. 520 This document and translations of it may be copied and furnished to 521 others, and derivative works that comment on or otherwise explain it 522 or assist in its implementation may be prepared, copied, published 523 and distributed, in whole or in part, without restriction of any 524 kind, provided that the above copyright notice and this paragraph 525 are included on all such copies and derivative works. However, this 526 document itself may not be modified in any way, such as by removing 527 the copyright notice or references to the Internet Society or other 528 Internet organizations, except as needed for the purpose of 529 developing Internet standards in which case the procedures for 530 copyrights defined in the Internet Standards process must be 531 followed, or as required to translate it into languages other than 532 English. 534 The limited permissions granted above are perpetual and will not be 535 revoked by the Internet Society or its successors or assigns. 537 This document and the information contained herein is provided on an 538 "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING 539 TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING 540 BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION 541 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF 542 MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 544 7. Authors' Addresses 546 Jeff Preston 547 NSC 548 23 Fielding Drive 549 Aurora, Ontario. L4G 4Z4 551 EMail: jpreston@nsctech.com 553 TJ Saunders 554 23525 24th Ave W 555 Brier, WA 98036 557 EMail: tj@castaglia.org