idnits 2.17.1 draft-preston-ftpext-deflate-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == There are 4 instances of lines with non-ascii characters in the document. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 12 instances of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (September 2003) is 7523 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '6' is defined on line 469, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2234 (ref. '3') (Obsoleted by RFC 4234) ** Downref: Normative reference to an Informational RFC: RFC 1950 (ref. '5') ** Downref: Normative reference to an Informational RFC: RFC 1951 (ref. '6') ** Downref: Normative reference to an Informational RFC: RFC 2577 (ref. '9') Summary: 8 errors (**), 0 flaws (~~), 5 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 FTPEXT Working Group J. Preston 2 Internet Draft Noisette Software 3 Document: draft-preston-ftpext-deflate-00.txt TJ Saunders 4 Expires: March 2004 September 2003 6 Deflate transmission mode for FTP 8 Status of this Memo 10 This document is an Internet-Draft and is subject to all 11 provisions of Section 10 of RFC 2026. 13 Internet-Drafts are working documents of the Internet Engineering 14 Task Force (IETF), its areas, and its working groups. Note that 15 other groups may also distribute working documents as Internet- 16 Drafts. 18 Internet-Drafts are draft documents valid for a maximum of six 19 months and may be updated, replaced, or obsoleted by other 20 documents at any time. It is inappropriate to use Internet-Drafts 21 as reference material or to cite them other than as "work in 22 progress." 24 The list of current Internet-Drafts can be accessed at 25 http://www.ietf.org/ietf/1id-abstracts.txt 26 The list of Internet-Draft Shadow Directories can be accessed at 27 http://www.ietf.org/shadow.html. 29 Abstract 31 This document defines an optional extension to RFC 959, "FILE 32 TRANSFER PROTOCOL (FTP)" (October 1985). It specifies a new 33 "deflate" transmission mode designed to increase network bandwidth 34 by compressing data using existing techniques. 36 Table of Contents 38 1. Introduction...................................................2 39 2. Document Conventions...........................................2 40 2.1 Basic Tokens...............................................3 41 3. Deflate Transmission Mode......................................3 42 3.1 Client-server Interaction..................................4 43 3.2 Overview...................................................4 44 3.3 Compression Engine.........................................4 45 3.3.1 ZLIB Compression Engine..............................4 46 3.3.2 Default ZLIB Settings................................5 47 3.4 Syntax.....................................................5 48 3.5 FEAT Response..............................................6 49 3.5.1 FEAT Examples........................................6 50 3.6 OPTS Features..............................................7 51 3.6.1 Standard Opt-names...................................8 52 3.6.1.1 ENGINE Syntax..................................8 53 3.6.1.2 LEVEL Syntax...................................8 54 3.6.1.3 BLOCKSIZE Syntax...............................8 55 3.6.2 OPTS Examples........................................9 56 3.7 Error Recovery and Restart.................................9 57 4. Security Considerations.......................................10 58 5. References....................................................10 59 6. Copyright.....................................................11 60 7. Authors' Addresses............................................11 62 1. Introduction 64 As the Internet grows, modern devices and networking environments 65 create new performance challenges for the File Transfer Protocol 66 (FTP) [1]. One solution to this problem, which is addressed in the 67 FTP "compress" transmission mode, is to compress file and system 68 data to maximize network resources. However, the original system is 69 designed to reduce ASCII text with repetitive characters and is 70 unsuitable in many applications because it can add significant 71 network overhead to binary transfers. This document enhances the 72 capabilities of FTP by introducing a new "deflate" transmission mode 73 that: 75 * increases network throughput and decreases transfer time 76 * effectively compresses ASCII and binary data 77 * requires a minimum amount of control information 78 * provides error recovery and data integrity options 79 * includes a mechanism to negotiate compression parameters to 80 balance CPU, memory and data requirements 81 * is extensible to accommodate future compression techniques 83 2. Document Conventions 85 This document makes use of the conventions defined in BCP 14 [2] 86 which includes the explanation of capitalized imperative words MUST, 87 SHOULD, MAY, SHOULD NOT and MUST NOT. Any syntax is defined using 88 Augmented BNF (ABNF) as specified in RFC 2234 [3]. 90 The terms "reply", "user", "file", "pathname", "FTP commands", 91 "DTP", "user-FTP process", "user-PI", "user-DTP", "server-FTP 92 process", "server-PI", "server-DTP", "mode", "type", "NVT", 93 "control connection", "data connection", "transmission mode", 94 "binary" and "ASCII" are all used here as defined in STD 9 [1]. 96 In addition, this specification makes use of the terms "compression 97 engine" and "compression method." A compression engine is a 98 hardware or software component that implements a compression method. 99 The compression method is a process that reduces the size of 100 computer data. 102 2.1 Basic Tokens 104 This document imports the core definitions given in Appendix A of 105 RFC 2234 [3] which includes the ABNF elements like ALPHA, DIGIT, SP, 106 etc. The following terms are added for use in this document: 108 TCHAR = VCHAR / SP / HTAB ; visible plus white space 109 RCHAR = ALPHA / DIGIT / "," / "." / ":" / "!" / 110 "@" / "#" / "$" / "%" / "^" / 111 "&" / "(" / ")" / "-" / "_" / 112 "+" / "?" / "/" / "\" / "'" / 113 DQUOTE ; <"> -- double quote character (%x22) 114 SCHAR = RCHAR / "=" ; 116 The VCHAR (from [3]), RCHAR, SCHAR, and TCHAR types give basic 117 character types from varying sub-sets of the ASCII character set for 118 use in various commands and responses. 120 token = 1*RCHAR 122 A "token" is a string whose precise meaning depends upon the context 123 in which it is used. In some cases it will be a value from a set of 124 possible values and in others it might be a string invented by one 125 party for an FTP conversation. 127 Note that in ABNF, string literals are case insensitive. That 128 convention is preserved in this document, and implies that FTP 129 commands added by this specification have names that can be 130 represented in any case. For example, "MODE" is the same as "mode" 131 and "Mode". However, ALPHA characters are case sensitive which 132 implies a token can have an exact value. That implication is 133 correct, except where explicitly stated to the contrary in this 134 document, or in some other specification which defines the values 135 this document specifies be used in a particular context. 137 3. Deflate Transmission Mode 139 The deflate extension introduces a fourth transmission mode to FTP 140 by updating the transfer mode (MODE) command. It employs general 141 purpose compression methods to reduce data for efficient transfers. 143 The following codes are assigned for transfer modes: 145 S � Stream 146 B � Block 147 C � Compressed 148 Z � Deflate 150 The default transfer mode remains Stream. 152 3.1 Client-server Interaction 154 The user-FTP process sends the MODE Z command to request compressed 155 data transfers. If the server-FTP process accepts the request, then 156 deflate transmission mode will be used for all data transfers until 157 the client switches to another mode. 159 3.2 Overview 161 In deflate transmission mode, data is compressed and transmitted as 162 a stream of octets (8 bit bytes). The sender and receiver rely on a 163 compression engine to perform compression operations 164 (deflate/inflate) and maintain state. There is no restriction on 165 the representation type used; record structures are allowed. 167 Since there is no fixed compression format, both FTP hosts MUST 168 process data until the compression engine reports an end-of-file 169 (EOF) state or data error. Closing the data connection is not a 170 sufficient method to end transfers because there may be pending 171 information. 173 If an FTP process encounters an error while compressing or 174 decompressing the data stream, it SHOULD discard all information 175 after that point and cancel the transfer using the procedures 176 described in STD 9 [1]. 178 3.3 Compression Engine 180 Each compression engine generates a unique data stream that MAY 181 consist of the following parts: header and control information, 182 compressed data, integrity checkpoints and end-of-file (EOF) 183 markers. The compression engine MUST support an EOF mechanism and 184 MUST NOT send non-essential structures like version headers. 186 3.3.1 ZLIB Compression Engine 188 All FTP processes that support deflate transmission mode MUST 189 support the ZLIB compressed data format specified in RFC 1950 [5]. 190 The ZLIB compression method, an LZ77 variant called deflation, 191 provides a lossless compressed data format that: 193 * is independent of CPU type, operating system, file system 194 and character set and is therefore ideal for network 195 communications 196 * provides a number of different compression settings 197 (ratios are in the order of 2:1 to 5:1) that accommodates 198 a wide range of CPU, memory and data requirements 199 * minimizes control data overhead (approximately 0.02% for 200 large data streams) 201 * provides integrity checks 202 * can be implemented readily in a manner not covered by 203 patents, and hence can be practiced freely 205 In the worst case, ZLIB reverts to stored (uncompressed) blocks 206 making the deflate data stream analogous to STREAM transmission 207 mode. 209 3.3.2 Default ZLIB Settings 211 The following ZLIB [5] parameters are recommended for deflate 212 transmission mode: 214 Compression level: 7 215 Compression method: Z_DEFLATED 216 Window bits: -15 217 Memory level: 8 218 Strategy: Z_DEFAULT_STRATEGY 220 Some server-FTP processes may use different compression and memory 221 levels to satisfy capacity limitations. These parameters are 222 transparent to the user-FTP process. 224 Note the window bits value is negative to suppress the optional 225 version header. 227 3.4 Syntax 229 The deflate extension modifies the MODE command by adding the 230 parameter "Z": 232 Request: 234 mode = "MODE" SP "Z" CRLF 236 Response: 238 mode-response = mode-good / mode-bad 239 mode-good = "200" SP response-message CRLF 240 mode-bad = "451" SP response-message CRLF / 241 = "501" SP response-message CRLF 242 response-message = *TCHAR 244 A "mode-good" response (200 reply) MUST be sent when the "Z" 245 parameter is recognized and the current compression settings are 246 appropriate. An "mode-bad" response is sent in other cases. The 247 451 reply should be used when the current compression settings or 248 some other temporary condition at the server prevent the command 249 from being accepted; but a changed environment for the server-FTP 250 process may permit the command to succeed. A 501 reply is 251 appropriate for a permanent error. 253 3.5 FEAT Response 255 If the server-FTP process supports the feature (FEAT) command 256 specified in RFC 2389 [4], then it MUST include a "MODE Z" feature 257 line. This string indicates required support for the extension and 258 lists the names of additional compression engines: 260 mode-feat = SP "MODE" SP "Z" [SP eng-list] CRLF 261 eng-list = *(eng-desc ",") 262 eng-desc = 1*(eng-name eng-opts) 263 eng-name = ALPHA*(ALPHA / DIGIT / "-" / ".") 264 eng-opts = *("(" opts-list ")") 265 opts-list = 1*(opt-name ",") 266 opt-name = ALPHA*(ALPHA / DIGIT) 268 If eng-list is not present, then the server-FTP process is informing 269 the client that ZLIB is the only engine available. 271 The "MODE Z" feature line string is not case sensitive, but SHOULD 272 be transmitted in upper case. 274 In the case where the server-FTP process does not support the FEAT 275 command, the user-FTP can negotiate the deflate extension by sending 276 the "MODE Z" request. The server would respond with a positive 277 (200) reply and the default compression settings would be effective. 279 3.5.1 FEAT Examples 281 The following examples contrast three servers with deflate support. 282 The first server advertises ZLIB and BZIP2 capabilities, while the 283 second indicates the required ZLIB engine and the third implements 284 ZLIB and a proprietary compression engine. 286 C> FEAT 287 S> 211-Extensions supported: 289 S> ... 290 S> MODE Z BZIP2 291 S> ... 292 S> 211 End. 294 and 296 C> FEAT 297 S> 211-Extensions supported: 298 S> ... 299 S> MODE Z 300 S> ... 301 S> 211 End. 303 and 305 C> FEAT 306 S> 211-Extensions supported: 307 S> ... 308 S> MODE Z ENG(SETTING1,SETTING2) 309 S> ... 310 S> 211 End. 312 The ellipses indicate place holders where other features may be 313 included, and are not required. A one space indentation of the 314 feature line is mandatory [4]. 316 3.6 OPTS Features 318 The user-FTP process may specify alternate compression settings with 319 the OPTS command [4]. All subsequent transfers will use these 320 settings until another OPTS request is sent. The format is 321 specified by: 323 opts = "OPTS" SP "MODE" SP "Z" [ SP 1*(name SP value ",") ] 324 name = ALPHA*(ALPHA / DIGIT) 325 value = 1*RCHAR 327 When the client sends an "OPTS MODE Z" command, the server will 328 examine each opt-name and opt-value pair and update the compression 329 engine. An OPTS request with no parameters will cause the server- 330 FTP process to revert to the default compression settings outlined 331 in this document. If the server encounters an invalid or 332 unsupported opt-name or opt-value the OPTS request will be rejected. 334 Note the server-FTP process MUST reject any MODE Z requests during 335 data transfers. 337 If the server-FTP process accepts an OPTS request, it MUST respond 338 with a positive (200) reply. Otherwise, a negative (501) response 339 should be sent. 341 The deflate extension does not require the server to support all of 342 the opt-name and opt-value parameters defined in this documented. 343 If the server-FTP process encounters an invalid or unsupported 344 option, it SHOULD return the opt-name in the error reply. In the 345 worst case, where all OPTS requests are rejected, the FTP processes 346 revert to the default compression settings. 348 If new opt-name parameters are required, the server-FTP process MUST 349 include the labels in the feature string enclosed in brackets and 350 separated by commas. For example, "MODE Z ENG(SETTING1,SETTING2)." 351 The apparatus defined in this specification should be able to handle 352 any routine compression setting. 354 3.6.1 Standard Opt-names 356 This document defines a standard set of opt-names as follows: 357 ENGINE, LEVEL and BLOCKSIZE. Each opt-name is case insensitive, or 358 in other words, "ENGINE" is the same as "Engine" and "engine". 360 3.6.1.1 ENGINE Syntax 362 The syntax of the ENGINE option follows: 364 eng-option = eng-label SP eng-value 365 eng-label = "ENGINE" 366 eng-value = ALPHA*(ALPHA / DIGIT / "-" / ".") 368 3.6.1.2 LEVEL Syntax 370 The syntax of the LEVEL option follows: 372 lvl-option = lvl-label SP lvl-value 373 lvl-label = "LEVEL" 374 lvl-value = 1*DIGIT 376 3.6.1.3 BLOCKSIZE Syntax 378 The syntax of the BLOCKSIZE option follows: 380 blk-option = blk-label SP blk-value 381 blk-label = "BLOCKSIZE" 382 blk-value = 1*DIGIT 384 The blk-value is specified in octets. 386 3.6.2 OPTS Examples 388 The following examples illustrate how a client would change the ZLIB 389 compression level and configure a new compression engine. 391 C> OPTS MODE Z LEVEL 9 392 S> 200 MODE Z LEVEL set to 9. 393 C> OPTS MODE Z BLOCKSIZE 8192 394 S> 501 MODE Z BLOCKSIZE option is not applicable. 396 and 398 C> OPTS MODE Z ENGINE BZIP2 BLOCKSIZE 8192 399 S> 200 MODE Z ENGINE set to BZIP2. 401 and 403 C> OPTS MODE Z ENGINE ZLIB BLOCKSIZE 8192 404 S> 501 MODE Z BLOCKSIZE option is not applicable. 406 3.7 Error Recovery and Restart 408 In deflate transmission mode, it is not possible to insert restart 409 markers into the data stream because they would be indistinguishable 410 from compressed data, and the user-FTP can alter the data 411 representation by changing compression settings between transfers. 412 However, it is possible to define a restart mechanism by specifying 413 a byte offset into the uncompressed data stream. 415 The logic for this system is similar to the restart mechanisms 416 specified in RFC 1123 [7] and other Internet Drafts [8] for STREAM 417 transmission mode with the following addendum. In a compressed data 418 stream, the output will always be exactly the same as the input, 419 thus an offset will always represent the same position within a 420 file. 422 If the user-FTP process plans to restart a retrieve (RETR) request, 423 it will directly calculate the restart marker, and send the 424 uncompressed offset in the restart (REST) command. The server will 425 skip to the specified file position. When the transfer continues, 426 both FTP processes will operate with a new compressed data stream. 428 The store (STOR) process works in the same manner as the retrieve 429 system. However, the sender must determine how much data was 430 previously received and expanded, with the SIZE [8] command or an 431 alternate method, before resuming the transfer. 433 4. Security Considerations 435 The deflate extension does not introduce any protocol related 436 security issues. However, some compression settings may impose a 437 considerable load on the FTP server, which could lead to denial of 438 service attacks, and compression engines not described in this 439 document may contain security vulnerabilities. If these operational 440 risks are a concern, then implementers should consider limiting 441 server resources or denying problematic settings. 443 In some cases, deflate transmission mode can reduce the demands on 444 the server. For example, in a secure FTP session, the combined 445 process of compressing and encrypting data is less expensive than 446 sending raw encrypted data; and fewer secure renegotiations are 447 required because of the shorter transfer times. 449 A general discussion of issues related to the security of FTP can be 450 found in RFC 2577 [9]. 452 5. References 454 [1] Postel, J., Reynolds, J., "File Transfer Protocol (FTP)", STD 9, 455 RFC 959, October 1985 457 [2] Bradner, S., "Key words for use in RFCs to Indicate Requirement 458 Levels", BCP 14, RFC 2119, March 1997 460 [3] Crocker, D., Overell, P., "Augmented BNF for Syntax 461 Specifications: ABNF", RFC 2234, November 1997 463 [4] Hethmon, P., Elz, R., "Feature negotiation mechanism for the 464 File Transfer Protocol", RFC 2389, August 1998 466 [5] Deutsch, P., "ZLIB Compressed Data Format Specification version 467 3.3", RFC 1950, May 1996 469 [6] Deutsch, P., "DEFLATE Compressed Data Format Specification 470 version 1.3", RFC 1951, May 1996 472 [7] Braden, R., "Requirements for Internet Hosts -- Application and 473 Support", RFC 1123, October 1989 475 [8] P., Elz, R., Hethmon, "Extensions to FTP", Internet Draft 476 (draft-ietf-ftpext-mlst-16), September 2002 478 [9] Allman, M., Ostermann, S., "FTP Security Considerations", RFC 479 2577, May 1999 481 6. Copyright 483 Copyright (C) The Internet Society (1999). All Rights Reserved. 485 This document and translations of it may be copied and furnished to 486 others, and derivative works that comment on or otherwise explain it 487 or assist in its implementation may be prepared, copied, published 488 and distributed, in whole or in part, without restriction of any 489 kind, provided that the above copyright notice and this paragraph 490 are included on all such copies and derivative works. However, this 491 document itself may not be modified in any way, such as by removing 492 the copyright notice or references to the Internet Society or other 493 Internet organizations, except as needed for the purpose of 494 developing Internet standards in which case the procedures for 495 copyrights defined in the Internet Standards process must be 496 followed, or as required to translate it into languages other than 497 English. 499 The limited permissions granted above are perpetual and will not be 500 revoked by the Internet Society or its successors or assigns. 502 This document and the information contained herein is provided on an 503 "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING 504 TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING 505 BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION 506 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF 507 MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 509 7. Authors' Addresses 511 Jeff Preston 512 Noisette Software Corporation 513 2112-135 Hillcrest Avenue 514 Mississauga, Ontario. L5B 4B1 516 EMail: jpreston@noisette-software.com 518 TJ Saunders 519 23525 24th Ave W 520 Brier, WA 98036 522 EMail: tj@castaglia.org