idnits 2.17.1 draft-preston-ftpext-deflate-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 1) being 64 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 12 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 30, 2010) is 5046 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '6' is defined on line 517, but no explicit reference was found in the text == Unused Reference: 'RFC5797' is defined on line 531, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2234 (ref. '3') (Obsoleted by RFC 4234) ** Downref: Normative reference to an Informational RFC: RFC 1950 (ref. '5') ** Downref: Normative reference to an Informational RFC: RFC 1951 (ref. '6') ** Downref: Normative reference to an Informational RFC: RFC 2577 (ref. '9') Summary: 5 errors (**), 0 flaws (~~), 6 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet Draft J. Preston 2 Document: draft-preston-ftpext-deflate-04.txt NSC 3 Intended status: Standards Track TJ Saunders 4 Expires: December 30, 2010 June 30, 2010 6 Deflate transmission mode for FTP 7 draft-preston-ftpext-deflate 9 Abstract 11 This document defines an optional extension to RFC 959, "FILE 12 TRANSFER PROTOCOL (FTP)" (October 1985). It specifies a new 13 "deflate" transmission mode designed to increase network bandwidth 14 by compressing data using existing techniques. 16 Status of this Memo 18 This Internet-Draft is submitted in full conformance with the 19 provisions of BCP 78 and BCP 79. 21 Internet-Drafts are working documents of the Internet Engineering 22 Task Force (IETF). Note that other groups may also distribute 23 working documents as Internet-Drafts. The list of current Internet- 24 Drafts is at http://datatracker.ietf.org/drafts/current/. 26 Internet-Drafts are draft documents valid for a maximum of six months 27 and may be updated, replaced, or obsoleted by other documents at any 28 time. It is inappropriate to use Internet-Drafts as reference 29 material or to cite them other than as "work in progress." 31 This Internet-Draft will expire on December 30, 2010. 33 Copyright Notice 35 Copyright (c) 2010 IETF Trust and the persons identified as the 36 document authors. All rights reserved. 38 This document is subject to BCP 78 and the IETF Trust's Legal 39 Provisions Relating to IETF Documents 40 (http://trustee.ietf.org/license-info) in effect on the date of 41 publication of this document. Please review these documents 42 carefully, as they describe your rights and restrictions with respect 43 to this document. Code Components extracted from this document must 44 include Simplified BSD License text as described in Section 4.e of 45 the Trust Legal Provisions and are provided without warranty as 46 described in the Simplified BSD License. 48 Table of Contents 50 1. Introduction...................................................2 51 2. Document Conventions...........................................2 52 2.1 Basic Tokens...............................................3 53 3. Deflate Transmission Mode......................................3 54 3.1 Client-server Interaction..................................4 55 3.2 Overview...................................................4 56 3.3 Compression Engine.........................................4 57 3.3.1 ZLIB Compression Engine..............................4 58 3.4 Syntax.....................................................5 59 3.5 FEAT Response..............................................6 60 3.5.1 FEAT Examples........................................6 62 3.6 OPTS Features..............................................7 63 3.6.1 Standard Opt-names...................................8 64 3.6.1.1 ENGINE Syntax..................................8 65 3.6.1.2 METHOD Syntax..................................8 66 3.6.1.3 LEVEL Syntax...................................8 67 3.6.1.4 EXTRA Syntax...................................8 68 3.6.1.5 BLOCKSIZE Syntax...............................9 69 3.6.2 OPTS Examples........................................9 70 3.7 Error Recovery and Restart.................................9 71 4. Security Considerations.......................................10 72 5. References....................................................10 73 6. Copyright.....................................................11 74 7. Authors' Addresses............................................12 76 1. Introduction 78 As the Internet grows, modern devices and networking environments 79 create new performance challenges for the File Transfer Protocol 80 (FTP) [1]. One solution to this problem, which is addressed in the 81 FTP "compress" transmission mode, is to compress file and system 82 data to maximize network resources. However, the original system is 83 designed to reduce ASCII text with repetitive characters and is 84 unsuitable in many applications because it can add significant 85 network overhead to binary transfers. This document enhances the 86 capabilities of FTP by introducing a new "deflate" transmission mode 87 that: 89 * increases network throughput and decreases transfer time 91 * effectively compresses ASCII and binary data 93 * requires a minimum amount of control information 95 * provides error recovery and data integrity options 97 * includes a mechanism to negotiate compression parameters to 98 balance CPU, memory and data requirements 100 * is extensible to accommodate future compression techniques 102 2. Document Conventions 104 This document makes use of the conventions defined in BCP 14 [2] 105 which includes the explanation of capitalized imperative words MUST, 106 SHOULD, MAY, SHOULD NOT and MUST NOT. Any syntax is defined using 107 Augmented BNF (ABNF) as specified in RFC 2234 [3]. 109 The terms "reply", "user", "file", "pathname", "FTP commands", 110 "DTP", "user-FTP process", "user-PI", "user-DTP", "server-FTP 111 process", "server-PI", "server-DTP", "mode", "type", "NVT", 112 "control connection", "data connection", "transmission mode", 113 "binary" and "ASCII" are all used here as defined in STD 9 [1]. 115 In addition, this specification makes use of the terms "compression 116 engine" and "compression method." A compression engine is a 117 hardware or software component that implements a compression method. 119 The compression method is a process that reduces the size of 120 computer data. 122 2.1 Basic Tokens 124 This document imports the core definitions given in Appendix A of 125 RFC 2234 [3] which includes the ABNF elements like ALPHA, DIGIT, SP, 126 etc. The following terms are added for use in this document: 128 TCHAR = VCHAR / SP / HTAB ; visible plus white space 129 RCHAR = ALPHA / DIGIT / "," / "." / ":" / "!" / 130 "@" / "#" / "$" / "%" / "^" / 131 "&" / "(" / ")" / "-" / "_" / 132 "+" / "?" / "/" / "\" / "'" / 133 DQUOTE ; <"> -- double quote character (%x22) 134 SCHAR = RCHAR / "=" ; 136 The VCHAR (from [3]), RCHAR, SCHAR, and TCHAR types give basic 137 character types from varying sub-sets of the ASCII character set for 138 use in various commands and responses. 140 token = 1*RCHAR 142 A "token" is a string whose precise meaning depends upon the context 143 in which it is used. In some cases it will be a value from a set of 144 possible values and in others it might be a string invented by one 145 party for an FTP conversation. 147 Note that in ABNF, string literals are case insensitive. That 148 convention is preserved in this document, and implies that FTP 149 commands added by this specification have names that can be 150 represented in any case. For example, "MODE" is the same as "mode" 151 and "Mode". However, ALPHA characters are case sensitive which 152 implies a token can have an exact value. That implication is 153 correct, except where explicitly stated to the contrary in this 154 document, or in some other specification which defines the values 155 this document specifies be used in a particular context. 157 3. Deflate Transmission Mode 159 The deflate extension introduces a fourth transmission mode to FTP 160 by updating the transfer mode (MODE) command. It employs general 161 purpose compression methods to reduce data for efficient transfers. 163 The following codes are assigned for transfer modes: 165 S Stream 166 B Block 167 C Compressed 168 Z Deflate 170 The default transfer mode remains Stream. 172 3.1 Client-server Interaction 174 The user-FTP process sends the MODE Z command to request compressed 175 data transfers. If the server-FTP process accepts the request, then 176 deflate transmission mode will be used for all data transfers until 177 the client switches to another mode. 179 3.2 Overview 181 In deflate transmission mode, data is compressed and transmitted as 182 a stream of octets (8 bit bytes). The sender and receiver rely on a 183 compression engine to perform compression operations 184 (deflate/inflate) and maintain state. There is no restriction on 185 the representation type used; record structures are allowed. 187 Since there is no fixed compression format, both FTP hosts MUST 188 process data until the compression engine reports an end-of-file 189 (EOF) state or data error. Closing the data connection is not a 190 sufficient method to end transfers because there may be pending 191 information. 193 If an FTP process encounters an error while compressing or 194 decompressing the data stream, it SHOULD discard all information 195 after that point and cancel the transfer using the procedures 196 described in STD 9 [1]. 198 3.3 Compression Engine 200 Each compression engine generates a unique data stream that MAY 201 consist of the following parts: header and control information, 202 compressed data, integrity checkpoints and end-of-file (EOF) 203 markers. The compression engine MUST support an EOF mechanism and 204 MUST NOT send non-essential structures like version headers. 206 3.3.1 ZLIB Compression Engine 208 All FTP processes that support deflate transmission mode MUST 209 support the ZLIB compressed data format specified in RFC 1950 [5]. 211 The ZLIB compression method, an LZ77 variant called deflation, 212 provides a lossless compressed data format that: 214 * is independent of CPU type, operating system, file system 215 and character set and is therefore ideal for network 216 communications 218 * provides a number of different compression settings 219 (ratios are in the order of 2:1 to 5:1) that accommodates 220 a wide range of CPU, memory and data requirements 222 * minimizes control data overhead (approximately 0.02% for 223 large data streams) 225 * provides integrity checks 227 * can be implemented readily in a manner not covered by 228 patents, and hence can be practiced freely 230 In the worst case, ZLIB reverts to stored (uncompressed) blocks 231 making the deflate data stream analogous to STREAM transmission 232 mode. 234 By default, compliant FTP processes MUST support compression method 235 8 and transmit the CMF, FLG and ADLER32 information in the data 236 stream. If bandwidth or processing requirements are a concern, 237 these restrictions can be negotiated with the OPTS command. 239 3.4 Syntax 241 The deflate extension modifies the MODE command by adding the 242 parameter "Z": 244 Request: 246 mode = "MODE" SP "Z" CRLF 248 Response: 250 mode-response = mode-good / mode-bad 251 mode-good = "200" SP response-message CRLF 252 mode-bad = "451" SP response-message CRLF / 253 = "501" SP response-message CRLF 254 response-message = *TCHAR 256 A "mode-good" response (200 reply) MUST be sent when the "Z" 257 parameter is recognized and the current compression settings are 258 appropriate. An "mode-bad" response is sent in other cases. The 259 451 reply should be used when the current compression settings or 260 some other temporary condition at the server prevent the command 261 from being accepted; but a changed environment for the server-FTP 262 process may permit the command to succeed. A 501 reply is 263 appropriate for a permanent error. 265 3.5 FEAT Response 267 If the server-FTP process supports the feature (FEAT) command 268 specified in RFC 2389 [4], then it MUST include a "MODE Z" feature 269 line. This string indicates required support for the extension and 270 lists the names of additional compression engines: 272 mode-feat = SP "MODE" SP "Z" [SP eng-list] CRLF 273 eng-list = *(eng-desc ",") 274 eng-desc = 1*(eng-name eng-opts) 275 eng-name = ALPHA*(ALPHA / DIGIT / "-" / ".") 276 eng-opts = *("(" opts-list ")") 277 opts-list = 1*(opt-name ",") 278 opt-name = ALPHA*(ALPHA / DIGIT) 280 If eng-list is not present, then the server-FTP process is informing 281 the client that ZLIB is the only engine available. 283 The "MODE Z" feature line string is not case sensitive, but SHOULD 284 be transmitted in upper case. 286 In the case where the server-FTP process does not support the FEAT 287 command, the user-FTP can negotiate the deflate extension by sending 288 the "MODE Z" request. The server would respond with a positive 289 (200) reply and the default compression settings would be effective. 291 3.5.1 FEAT Examples 293 The following examples contrast three servers with deflate support. 294 The first server advertises ZLIB and BZIP2 capabilities, while the 295 second indicates the required ZLIB engine and the third implements 296 ZLIB and a proprietary compression engine. 298 C> FEAT 299 S> 211-Extensions supported: 300 S> =85 301 S> MODE Z BZIP2(LEVEL,BLOCKSIZE) 302 S> =85 303 S> 211 End. 305 and 307 C> FEAT 308 S> 211-Extensions supported: 310 S> =85 311 S> MODE Z 312 S> =85 313 S> 211 End. 315 and 317 C> FEAT 318 S> 211-Extensions supported: 319 S> =85 320 S> MODE Z ENG(SETTING1,SETTING2) 321 S> =85 322 S> 211 End. 324 The ellipses indicate place holders where other features may be 325 included, and are not required. A one space indentation of the 326 feature line is mandatory [4]. 328 3.6 OPTS Features 330 The user-FTP process may specify alternate compression settings with 331 the OPTS command [4]. All subsequent transfers will use these 332 settings until another OPTS request is sent. The format is 333 specified by: 335 opts = "OPTS" SP "MODE" SP "Z" [ SP 1*(name SP value ",") ] 336 name = ALPHA*(ALPHA / DIGIT) 337 value = 1*RCHAR 339 When the client sends an "OPTS MODE Z" command, the server will 340 examine each opt-name and opt-value pair and update the compression 341 engine. An OPTS request with no parameters will cause the server- 342 FTP process to revert to the default compression settings outlined 343 in this document. If the server encounters an invalid or 344 unsupported opt-name or opt-value the OPTS request will be rejected. 346 Note the server-FTP process MUST reject any MODE Z requests during 347 data transfers. 349 If the server-FTP process accepts an OPTS request, it MUST respond 350 with a positive (200) reply. Otherwise, a negative (501) response 351 should be sent. 353 The deflate extension does not require the server to support all of 354 the opt-name and opt-value parameters defined in this documented. 355 If the server-FTP process encounters an invalid or unsupported 356 option, it SHOULD return the opt-name in the error reply. In the 357 worst case, where all OPTS requests are rejected, the FTP processes 358 revert to the default compression settings. 360 If new opt-name parameters are required, the server-FTP process MUST 361 include the labels in the feature string enclosed in brackets and 362 separated by commas. For example, "MODE Z ENG(SETTING1,SETTING2)." 363 The apparatus defined in this specification should be able to handle 364 any routine compression setting. 366 3.6.1 Standard Opt-names 368 This document defines a standard set of opt-names as follows: 369 ENGINE, METHOD, LEVEL, EXTRA and BLOCKSIZE. Each opt-name is case 370 insensitive, or in other words, "ENGINE" is the same as "Engine" and 371 "engine". 373 3.6.1.1 ENGINE Syntax 375 The syntax of the ENGINE option follows: 377 eng-option = eng-label SP eng-value 378 eng-label = "ENGINE" 379 eng-value = ALPHA*(ALPHA / DIGIT / "-" / ".") 381 3.6.1.2 METHOD Syntax 383 The METHOD option allows the FTP processes to negotiate the 384 compression method. The syntax of the METHOD option follows: 386 mth-option = mth-label SP mth-value 387 mth-label = "METHOD" 388 mth-value = 1*DIGIT 390 3.6.1.3 LEVEL Syntax 392 The LEVEL option allows the FTP processes to negotiate the 393 compression level. It will influence the processing requirements 394 and length of the compressed stream. The syntax of the LEVEL option 395 follows: 397 lvl-option = lvl-label SP lvl-value 398 lvl-label = "LEVEL" 399 lvl-value = 1*DIGIT 401 3.6.1.4 EXTRA Syntax 403 The EXTRA option allows the FTP processes to negotiate the 404 transmission of non-essential compression information (like version 405 headers and trailers). This option does not apply to all compression 406 engines. The syntax of the EXTRA option follows: 408 ext-option = ext-label SP ext-value 409 ext-label = "EXTRA" 410 ext-value = ext-enable / ext-disable 411 ext-enable = "ON" 412 ext-disable = "OFF" 414 3.6.1.5 BLOCKSIZE Syntax 416 The BLOCKSIZE option is for compression engines that use block 417 sorting algorithms. It influences the compression ratio and 418 processing requirements. The syntax of the BLOCKSIZE option 419 follows: 421 blk-option = blk-label SP blk-value 422 blk-label = "BLOCKSIZE" 423 blk-value = 1*DIGIT 425 The blk-value is specified in octets. 427 3.6.2 OPTS Examples 429 The following examples illustrate how a client would change the ZLIB 430 compression options and configure a new compression engine. 432 C> OPTS MODE Z LEVEL 9 433 S> 200 MODE Z LEVEL set to 9. 434 C> OPTS MODE Z BLOCKSIZE 8192 435 S> 501 MODE Z BLOCKSIZE is not available. 436 C> OPTS MODE Z METHOD 9 437 S> 501 MODE Z METHOD 9 is invalid. 438 C> OPTS MODE Z ENGINE ZLIB LEVEL 9 EXTRA OFF 439 S> 200- MODE Z ENGINE set to ZLIB. 440 S> 200- MODE Z LEVEL set to 9. 441 S> 200 MODE Z EXTRA set to OFF. 442 C> OPTS MODE Z ENGINE ZLIB LEVEL 9 METHOD 15 443 S> 501 MODE Z METHOD 15 is invalid. 445 and 447 C> OPTS MODE Z ENGINE BZIP2 BLOCKSIZE 8192 448 S> 200- MODE Z ENGINE set to BZIP2. 449 S> 200 MODE Z BLOCKSIZE set to 8192. 450 C> OPTS MODE Z ENGINE ZLIB BLOCKSIZE 8192 451 S> 501 MODE Z BLOCKSIZE is not available. 453 3.7 Error Recovery and Restart 454 In deflate transmission mode, it is not possible to insert restart 455 markers into the data stream because they would be indistinguishable 456 from compressed data, and the user-FTP can alter the data 457 representation by changing compression settings between transfers. 458 However, it is possible to define a restart mechanism by specifying 459 a byte offset into the uncompressed data stream. 461 The logic for this system is similar to the restart mechanisms 462 specified in RFC 1123 [7] and other Internet Drafts [8] for STREAM 463 transmission mode with the following addendum. In a compressed data 464 stream, the output will always be exactly the same as the input, 465 thus an offset will always represent the same position within a 466 file. 468 If the user-FTP process plans to restart a retrieve (RETR) request, 469 it will directly calculate the restart marker, and send the 470 uncompressed offset in the restart (REST) command. The server will 471 skip to the specified file position. When the transfer continues, 472 both FTP processes will operate with a new compressed data stream. 474 The store (STOR) process works in the same manner as the retrieve 475 system. However, the sender must determine how much data was 476 previously received and expanded, with the SIZE [8] command or an 477 alternate method, before resuming the transfer. 479 4. Security Considerations 481 The deflate extension does not introduce any protocol related 482 security issues. However, some compression settings may impose a 483 considerable load on the FTP server, which could lead to denial of 484 service attacks, and compression engines not described in this 485 document may contain security vulnerabilities. If these operational 486 risks are a concern, then implementers should consider limiting 487 server resources or denying problematic settings. 489 In some cases, deflate transmission mode can reduce the demands on 490 the server. For example, in a secure FTP session, the combined 491 process of compressing and encrypting data is less expensive than 492 sending raw encrypted data; and fewer secure renegotiations are 493 required because of the shorter transfer times. 495 A general discussion of issues related to the security of FTP can be 496 found in RFC 2577 [9]. 498 5. References 500 5.1. Normative References 502 [1] Postel, J., Reynolds, J., "File Transfer Protocol (FTP)", STD 9, 503 RFC 959, October 1985 505 [2] Bradner, S., "Key words for use in RFCs to Indicate Requirement 506 Levels", BCP 14, RFC 2119, March 1997 508 [3] Crocker, D., Overell, P., "Augmented BNF for Syntax 509 Specifications: ABNF", RFC 2234, November 1997 511 [4] Hethmon, P., Elz, R., "Feature negotiation mechanism for the 512 File Transfer Protocol", RFC 2389, August 1998 514 [5] Deutsch, P., "ZLIB Compressed Data Format Specification version 515 3.3", RFC 1950, May 1996 517 [6] Deutsch, P., "DEFLATE Compressed Data Format Specification 518 version 1.3", RFC 1951, May 1996 520 [7] Braden, R., "Requirements for Internet Hosts -- Application and 521 Support", RFC 1123, October 1989 523 [8] P., Elz, R., Hethmon, "Extensions to FTP", Internet Draft 524 (draft-ietf-ftpext-mlst-16), September 2002 526 [9] Allman, M., Ostermann, S., "FTP Security Considerations", RFC 527 2577, May 1999 529 5.2. Informative References 531 [RFC5797] Klensin, J. and A. Hoenes, "FTP Command and Extension 532 Registry", RFC 5797, March 2010. 534 6. Authors' Addresses 536 Jeff Preston 537 NSC 538 23 Fielding Drive 539 Aurora, Ontario. L4G 4Z4 541 EMail: jpreston@nsctech.com 543 TJ Saunders 544 23525 24th Ave W 545 Brier, WA 98036 547 EMail: tj@castaglia.org