idnits 2.17.1 draft-bryan-ftp-hash-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 11, 2010) is 5068 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'FIPS-180-3' -- Obsolete informational reference (is this intentional?): RFC 3230 (Obsoleted by RFC 9530) Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Bryan 3 Internet-Draft T. Kosse 4 Intended status: Standards Track D. Stenberg 5 Expires: December 13, 2010 June 11, 2010 7 FTP Extensions for Cryptographic Hashes 8 draft-bryan-ftp-hash-04 10 Abstract 12 The File Transfer Protocol does not offer any method to verify the 13 integrity of a transferred file, nor can two files be compared 14 against each other without actually transferring them first. 15 Cryptographic hashes are a possible solution to this problem. In the 16 past, several attempts have been made to add commands to obtain 17 checksums and hashes, however none have been formally specified, 18 leading to non-interoperability and confusion. To solve these 19 issues, this document specifies a new FTP command to be used by 20 clients to request cryptographic hashes of files. 22 Status of this Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at http://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on December 13, 2010. 39 Copyright Notice 41 Copyright (c) 2010 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (http://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 57 1.1. Example . . . . . . . . . . . . . . . . . . . . . . . . . 3 58 2. Notational Conventions . . . . . . . . . . . . . . . . . . . . 3 59 3. The HASH Command (HASH) . . . . . . . . . . . . . . . . . . . 4 60 3.1. User-PI usage of HASH . . . . . . . . . . . . . . . . . . 4 61 3.2. Changing the HASH algorithm . . . . . . . . . . . . . . . 5 62 3.3. FEAT Command Response for HASH Command . . . . . . . . . . 6 63 3.4. HASH Command Errors . . . . . . . . . . . . . . . . . . . 7 64 4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7 65 5. Implementation Requirements . . . . . . . . . . . . . . . . . 7 66 6. Security Considerations . . . . . . . . . . . . . . . . . . . 7 67 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 8 68 7.1. Normative References . . . . . . . . . . . . . . . . . . . 8 69 7.2. Informative References . . . . . . . . . . . . . . . . . . 9 70 Appendix A. Acknowledgements and Contributors . . . . . . . . . . 9 71 Appendix B. List of Non-standard Cryptographic Hash or 72 Checksum Commands and Implementations . . . . . . . . 9 73 Appendix C. Document History . . . . . . . . . . . . . . . . . . 12 74 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12 76 1. Introduction 78 The File Transfer Protocol [RFC0959] does not offer any method to 79 verify the integrity of a transferred file, nor can two files be 80 compared against each other without actually transferring them first. 81 Cryptographic hashes are a possible solution to this problem. In the 82 past, several attempts have been made to add commands to obtain 83 checksums and hashes, however none have been formally specified, 84 leading to non-interoperability and confusion. To solve these 85 issues, this document specifies a new FTP command to be used by 86 clients to request cryptographic hashes of files. HTTP has a similar 87 feature named Instance Digests [RFC3230] which allows a client to 88 request the cryptographic hash of a file. 90 [[ Discussion of this draft should take place on ftpext@ietf.org (or 91 apps-discuss@ietf.org if necessary). ]] 93 1.1. Example 95 Example of HASH client request: 97 C> HASH filename.ext 99 HASH server response with Positive Completion code and the requested 100 hash using the currently selected algorithm: 102 S> 213 80bc95fd391772fa61c91ed68567f0980bb45fd9 104 2. Notational Conventions 106 This specification describes conformance of FTP Extensions for 107 cryptographic hashes. 109 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 110 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 111 document are to be interpreted as described in BCP 14, [RFC2119], as 112 scoped to those conformance targets. 114 In examples, the "C>" lines are commands from user-PI to server-PI, 115 and the "S>" lines are server-PI replies. 117 This document also uses notation defined in STD 9, [RFC0959]. In 118 particular, the terms "reply", "user", "file", "pathname", "FTP 119 commands", "user-PI", "server-FTP process", "server-PI", "mode", 120 "type", and "ASCII", are all used here as defined there. 122 Syntax required is defined using the Augmented BNF defined in 124 [RFC5234]. 126 3. The HASH Command (HASH) 128 A new command "HASH" is added to the FTP command set to request the 129 cryptographic hash of a file from a server-FTP process. 131 The syntax for the HASH command is: 133 hash-command = "HASH" SP 135 As with all FTP commands, the "HASH" command word is case 136 independent, and MAY be specified in any character case desired. 138 The HASH command keyword MUST be followed by a single space (ASCII 139 32) followed by the pathname. 141 The pathname argument should reference the same file as other file 142 based commands such as STOR or RETR which the same argument would 143 reference. 145 The text returned in response to the HASH command MUST be: 147 hash-response = "213" SP 1*HEXDIGIT CRLF 149 All hash values MUST be encoded in lowercase hexadecimal format. 151 The HASH command is meant to be used for files transmitted in Image 152 type mode (TYPE I) and Stream transfer mode (MODE S). The returned 153 hash MUST be calculated over the raw octet data of the file 154 irrespective of the selected data type, transfer mode or any other 155 state affecting the transfer. In other words, if a client were to 156 download a full file using TYPE I and MODE S and were to calculate 157 the hash on the received octet data, it would be identical to the 158 hash returned by HASH. 160 3.1. User-PI usage of HASH 162 The user-PI issues the FEAT command to query the server-PI about 163 which algorithm is currently selected. This also reveals the other 164 algorithms that the server supports. In this example, the SHA-1 165 algorithm is currently selected. 167 C> FEAT 168 S> 211-Extensions supported: 169 S> ... 170 S> HASH SHA-256;SHA-512;SHA-1*;MD5 171 S> ... 172 S> 211 END 174 OPTS HASH is an alternative method for the user-PI to query the 175 server-PI about which algorithm is currently selected. 177 C> OPTS HASH 178 S> 200 SHA-1 180 In this example, we wish to select SHA-256, a different algorithm. 182 C> OPTS HASH SHA-256 183 S> 200 SHA-256 185 The user-PI requests the cryptographic hash of a file with HASH 186 command. Server-PI replies with cryptographic hash of file. 188 C> HASH filename.ext 189 S> 213 f0ad929cd259957e160ea442eb80986b5f01... 191 Client downloads file. Client hashes the downloaded file and 192 compares its hash to the hash obtained from the server. The HASH 193 command could also be used to verify that an uploaded file is an 194 exact copy. 196 3.2. Changing the HASH algorithm 198 To query the current hash algorithm and to change it, the OPTS 199 command as defined in [RFC2389] is used with HASH as the first 200 argument. If no second argument is passed, OPTS HASH simply returns 201 the currently selected hash algorithm. To change the algorithm, a 202 valid hashtype MUST be given as second argument. If the command is 203 successful, all future calls to HASH until the next successful OPTS 204 HASH command or until the session is reinitialized (REIN) will use 205 the selected hash algorithm. 207 C> OPTS HASH 208 S> 200 SHA-1 209 C> OPTS HASH SHA-512 210 S> 200 SHA-512 211 C> OPTS HASH CRC-37 212 S> 501 Unknown algorithm, current selection not changed 214 hashopts-cmd = "OPTS HASH" [ SP hashtype ] CRLF 215 hashopts-response = "200" SP hashtype CRLF 217 3.3. FEAT Command Response for HASH Command 219 A server-FTP process that supports the HASH command MUST include, in 220 the response to the FEAT command [RFC2389], a feature line indicating 221 that the HASH command is supported, along with a list of all 222 supported hash algorithms in a semicolon separated list. The hash 223 algorithm that is currently selected MUST be marked with an asterisk. 224 The order of hash algorithms is insignificant. This command word is 225 case insensitive, but it SHOULD be transmitted in upper case only. 226 That is, the response SHOULD be: 228 C> FEAT 229 S> 211-Extensions supported: 230 S> ... 231 S> HASH SHA-256;SHA-512;SHA-1*;MD5 232 S> ... 233 S> 211 END 235 The ellipses indicate place holders where other features MAY be 236 listed, but this is OPTIONAL. The single space indentation of each 237 feature line is REQUIRED by [RFC2389]. 239 The IANA registry named "Hash Function Textual Names" defines values 240 for hash types. Hash names should be presented in uppercase, but 241 comparisons should be case-insensitive, e.g. MD5, md5, Md5 are all 242 the same. 244 hash-feat = SP "HASH" SP hashlist CRLF 245 hashlist = 1*( hashname ["*"] ";" ) 246 hashname = 1*( hchar ) 247 hchar = ALPHA / DIGIT / "-" / "_" / "/" / "." / "," 249 3.4. HASH Command Errors 251 The server-PI SHOULD reply with a 500 reply if the HASH command is 252 unrecognized or unimplemented. 254 The server-PI SHOULD reply with a 501 reply to the OPTS HASH command 255 if the user-PI has requested an unknown or unsupported algorithm. 257 The server-PI SHOULD reply with a 550 reply if the HASH command is 258 used on a file that can not be found. 260 The server-PI SHOULD reply with a 552 reply if the user is not 261 allowed to use the HASH command. 263 The server-PI SHOULD reply with a 450 reply if the server is busy, 264 e.g. already hashing other files yet inviting the client to retry in 265 future. 267 4. IANA Considerations 269 This new command is added to the "FTP Commands and Extensions" 270 registry created by [RFC5797]. 272 Command Name: HASH 274 Description: Cryptographic Hash of a file 276 FEAT String: HASH 278 Command Type: Service execution 280 Conformance Requirements: Optional 282 Reference: This specification 284 5. Implementation Requirements 286 All conforming implementations MUST at least support the SHA-1 287 algorithm [FIPS-180-3]. Implementations SHOULD NOT make any 288 algorithm the default that is known to be weaker than SHA-1. Support 289 for any additional algorithms is OPTIONAL. 291 6. Security Considerations 293 Calculating a file's hash is a CPU intensive operation and can easily 294 consume the available disk I/O resources. If the HASH command isn't 295 implemented carefully, a server could be vulnerable to a denial of 296 service attack. On an affected server a malicious user could, for 297 example, continuously send HASH commands over multiple connections 298 and thus consume most of the FTP server's CPU and disk I/O resources, 299 leaving little room for other operations. To mitigate this risk, a 300 server SHOULD cache the calculated hashes so that the hash of a file 301 is only calculated once even if multiple hash requests are sent for 302 that file. 304 The performance of commonly used hard disk drives is adversely 305 affected by the amount of time the device needs to reposition its 306 read-and-write heads. A server SHOULD therefore avoid hashing 307 multiple files at the same time which are located on the same 308 physical media and SHOULD instead hash them sequentially. The FTP 309 server's right to refuse to calculate the hash is of course important 310 to help against DOS risks. A possible solution is to use the 450 311 reply code of HASH to indicate that the server is already busy with 312 another HASH operation. 314 In addition, the HASH command can be used to draw conclusions about 315 the contents of a file. If the hash of a file on some server matches 316 the hash of some known, local file, both files are likely identical. 317 To prevent this scenario it suffices to limit use of the HASH command 318 to users who would already be able to download the file. 320 Currently, some of the hash types defined in the IANA registry named 321 "Hash Function Textual Names" are considered insecure. These include 322 the whole Message Digest family of algorithms that are not suitable 323 for cryptographically strong verification. Malicious people could 324 provide files that appear to be identical to another file because of 325 a collision, i.e., the weak cryptographic hashes of the intended file 326 and a substituted malicious file could match. 328 7. References 330 7.1. Normative References 332 [FIPS-180-3] 333 National Institute of Standards and Technology (NIST), 334 "Secure Hash Standard (SHS)", FIPS PUB 180-3, 335 October 2008. 337 [RFC0959] Postel, J. and J. Reynolds, "File Transfer Protocol", 338 STD 9, RFC 0959, October 1985. 340 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 341 Requirement Levels", BCP 14, RFC 2119, March 1997. 343 [RFC2389] Hethmon, P. and R. Elz, "Feature negotiation mechanism for 344 the File Transfer Protocol", RFC 2389, August 1998. 346 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 347 Specifications: ABNF", STD 68, RFC 5234, January 2008. 349 7.2. Informative References 351 [RFC3230] Mogul, J. and A. Van Hoff, "Instance Digests in HTTP", 352 RFC 3230, January 2002. 354 [RFC5797] Klensin, J. and A. Hoenes, "FTP Command and Extension 355 Registry", RFC 5797, March 2010. 357 [draft-twine-ftpmd5] 358 Twine, J., "The MD5 and MMD5 FTP Command Extensions", 359 draft-twine-ftpmd5-00 (work in progress), May 2002. 361 Appendix A. Acknowledgements and Contributors 363 Thanks to John C. Klensin, Alfred Hoenes, James Twine, Robert 364 McMurray, and Mathias Berchtold. 366 Appendix B. List of Non-standard Cryptographic Hash or Checksum 367 Commands and Implementations 369 [[ to be removed by the RFC editor before publication as an RFC. ]] 371 A number of similar checksum or hash commands exist, but are not 372 formally specified, leading to non-interoperability and confusion. 373 The commands, any specifications, and relevant details: 375 o CKSM: GridFTP v2 Protocol Description 376 http://www.ogf.org/documents/GFD.47.pdf Usage: OPTS CKSM 377 CRLF. Supports ADLER32, MD5, CRC32. 378 o MD5/MMD5: Expired Internet Draft [draft-twine-ftpmd5] from 2002. 379 Usage: MD5 Algorithm specific command. Response codes: 380 251 positive completion, 500 Command Not Recognized, 502 Command 381 Not Implemented, 504 Command Not Implemented for the Specified 382 Argument. 383 o SITE CHECKSUM: Usage: SITE check_login SP CHECKSUM SP pathname 384 CRLF. Supports CRC32 and MD5. 386 o SITE SHOHASH: Usage: site shohash [filename]. Supports MD5. 387 Response codes: 200 positive completion. 388 o XCRC: By GlobalSCAPE in 2001. http://help.globalscape.com/help/ 389 secureserver2/File_Integrity_Checking.htm Usage: XCRC 390 SP EP. SP is starting point and EP is ending point in bytes and 391 are optional parameters. Algorithm specific command. Response 392 codes: 250 positive completion, 450 Requested file action not 393 taken. (File is busy), 550 Requested action not taken. (File not 394 found, no read permission, SP or EP not correct). 395 o XMD5: XMD5 SP EP. Similar to XCRC. Algorithm specific 396 command. 397 o XSHA, XSHA1, XSHA256, XSHA512: Usage similar to XCRC, although 398 SP/EP usage unknown. Algorithm specific commands. 400 An incomplete list of FTP clients and servers that have implemented 401 these commands: 403 o Akamai NetStorage (supports SITE CHKHSH/SITE SHOHASH) p17-18 404 http://pigdogslow.dyndns.org/NetStorage_UserGuide.pdf 405 o Apache Ftp Server (supports MD5/MMD5 from draft-twine-ftpmd5) 406 http://cwiki.apache.org/FTPSERVER/documentation.html 407 o Backup4all Pro (supports XCRC) 408 o Backup to FTP (supports XCRC) 409 o BlackMoon FTP Server (supports XCRC) 410 http://www.blackmoonftpserver.com/portal/readmore/features.html 411 o C.P.A. Secure (supports XCRC) 412 http://www.cpasecure.com/CPASecureVsSecureFTP.html 413 o Cerberus FTP server (supports XCRC, XMD5, XSHA1, XSHA256, XSHA512) 414 http://www.softpedia.com/progChangelog/ 415 Cerberus-FTP-Server-Changelog-1904.html 416 o Core FTP Pro (supports XCRC) 417 o Cross FTP Server (supports MD5/MMD5) 418 o FileCOPA FTP Server (supports XCRC, XMD5, XSHA1) 419 http://www.filecopa-ftpserver.com/features.html 420 o File Watchdogs FTP Server (supports XCRC, XMD5, XSHA1, XSHA256, 421 XSHA512) 422 http://www.filewatchdogs.com/ftpsitehosting/help/15559.htm 423 o FireFTP (supports XMD5, XSHA1) 424 http://fireftp.mozdev.org/features.html 425 o FTP Daemon (supports SITE CHECKMETHOD/SITE CHECKSUM) 426 http://www.pro-bono-publico.de/projects/ftpd.html 427 o FTP Voyager (supports XCRC) http://www.ftpvoyager.com/XCRC.asp 428 o Gene6 FTP Server 429 http://www.g6ftpserver.com/en/information#features 430 o GlobalSCAPE's Secure FTP Server / EFT Server / CuteFTP clients 431 (supports XCRC) 433 o Globus FTP client / Globus Toolkit(supports CKSM) http:// 434 www.globus.org/toolkit/releasenotes/3.2.0/gridftp_notes.html 435 o GoldenGate FTP (Ftp Full Java Server) (supports XCRC, XMD5, XSHA1) 436 o IceWarp FTP Server http://www.icewarp.com/products/ftp_server/ 437 o ICS FTP client (supports XCRC, XMD5) 438 http://www.magsys.co.uk/delphi/magics.asp 439 o ioFTPD (supports XCRC) 440 o JAFS (supports XCRC and MD5) 441 http://www.sbbi.net/site/jafs/features.html 442 o Kellerman FTP (supports XCRC) 443 http://sharptoolbox.com/tools/kellerman-ftp 444 o Limagito FTP server (supports XCRC, XMD5, XSHA1) 445 http://www.limagito.com/file-mover-features.html 446 o MOVEit DMZ (supports XSHA1) 447 o Nofeel FTP server (supports XCRC, XMD5, XSHA1) 448 http://www.nftpserver.com/history.php 449 o Null FTP (supports XCRC, XMD5, XSHA) 450 http://www.sharewareconnection.com/null-ftp-client-pro.htm 451 o Orenosv FTP Client (supports XCRC, XMD5) 452 http://www.orenosv.com/orenosv/ftpcli_en.html 453 o ProFTPD module mod_digest (supports XCRC, XMD5, XSHA1, SHA256) 454 http://www.smartftp.com/oss/proftpd/mod_digest.html 455 o PSFTPd Secure FTP Server (supports XCRC, XMD5, XSHA) 456 http://www.psftp.de/psftpd_fo.php 457 o Quick 'n Easy FTP Server (supports XCRC) http:// 458 www.pablosoftwaresolutions.com/html/ 459 quick__n_easy_ftp_server_pro.html 460 o RaidenFTPD32 FTP server (supports XCRC, XMD5) 461 o Robo-FTP Server (supports XCRC, XMD5, XSHA1) 462 http://kb.robo-ftp.com/change_log/show/61 463 o SyncBackPro and SyncBackSE (supports XCRC) 464 http://www.2brightsparks.com/syncback/sbpro-changes.html 465 o Secure FTP Factory (supports XCRC) 466 o Serv-U FTP Server (supports XCRC) http://www.serv-u.com/help/ 467 serv_u_help/additional_ftp_commands_supported_by_serv_u.htm 468 o SmartFTP client (supports XCRC, XMD5, XSHA, CKSM) 469 http://www.smartftp.com/features/ 470 o Starksoft Ftp Component for .NET / Mono (supports XCRC, XMD5, 471 XSHA1) http://www.starksoft.com/prod_ftp.html 472 o Titan FTP Server (supports XCRC) 473 o Turbo FTP (supports XCRC) 474 o WISE-FTP (supports XCRC) http://www.wise-ftp.com/news/ 475 o WS_FTP client / server (supports XSHA1, server also XMD5, XSHA1, 476 XSHA256, XSHA512) http://ipswitchft.custhelp.com/app/answers/ 477 detail/a_id/671/kw/xmd5/r_id/166/sno/1 478 o wuftpd (supports SITE CHECKMETHOD/SITE CHECKSUM) 479 o wzdFTPd (supports XCRC, XMD5) 480 http://www.wzdftpd.net/wiki/index.php/Commands 481 o Zalman FTP Client (supports XCRC) 482 http://www.zalmansoftware.com/download.html 483 o zFTPServer 485 Appendix C. Document History 487 [[ to be removed by the RFC editor before publication as an RFC. ]] 489 Known issues concerning this draft: 490 o Should HASH support partial file hashes, similar to the Content- 491 MD5 HTTP Header. 492 o Underspecification of the representation of the file that shall 493 undergo the hash calculation. 494 o Should the server response include the algorithm? i.e. "S> 213 495 SHA-256 xxxxxxxxxxxxxxx" 497 -04 : June 11, 2010. 498 o User-PI usage and command errors sections updated. 500 -03 : May 21, 2010. 501 o List of non-standard checksum and hash commands and their 502 implementations. 504 -02 : April 16, 2010. 505 o Error codes section. 507 -01 : April 7, 2010. 508 o Changing HASH algorithm with OPTS. 509 o Reference RFC 5797 and add IANA Considerations section. 510 o Informative Reference to expired Internet Draft 511 (draft-twine-ftpmd5) which attempted to address this issue (it 512 only supported one hash, MD5). 514 -00 : October 19, 2009. 515 o Initial draft. 517 Authors' Addresses 519 Anthony Bryan 520 Pompano Beach, FL 521 USA 523 Email: anthonybryan@gmail.com 524 URI: http://www.metalinker.org 526 Tim Kosse 528 Email: tim.kosse@filezilla-project.org 529 URI: http://filezilla-project.org/ 531 Daniel Stenberg 533 Email: daniel@haxx.se 534 URI: http://www.haxx.se/