idnits 2.17.1 draft-ietf-ftpext-mlst-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-25) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There are 6 instances of too long lines in the document, the longest one being 7 characters in excess of 72. ** The abstract seems to contain references ([2], [1]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (March 1998) is 9538 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '8' is defined on line 1673, but no explicit reference was found in the text -- Possible downref: Non-RFC (?) normative reference: ref. '1' ** Obsolete normative reference: RFC 2044 (ref. '2') (Obsoleted by RFC 2279) ** Obsolete normative reference: RFC 2234 (ref. '3') (Obsoleted by RFC 4234) -- Possible downref: Non-RFC (?) normative reference: ref. '4' -- Possible downref: Non-RFC (?) normative reference: ref. '5' -- Possible downref: Non-RFC (?) normative reference: ref. '8' -- Possible downref: Non-RFC (?) normative reference: ref. '10' ** Obsolete normative reference: RFC 1766 (ref. '12') (Obsoleted by RFC 3066, RFC 3282) -- Possible downref: Non-RFC (?) normative reference: ref. '13' Summary: 12 errors (**), 0 flaws (~~), 3 warnings (==), 8 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 FTPEXT Working Group R. Elz 3 Internet Draft University of Melbourne 4 Expiration Date: September 1998 5 P. Hethmon 6 Hethmon Brothers 8 March 1998 10 Extended Directory Listing, TVFS, and Restart Mechanism for FTP 12 draft-ietf-ftpext-mlst-04.txt 14 Status of this Memo 16 This document is an Internet-Draft. Internet-Drafts are working 17 documents of the Internet Engineering Task Force (IETF), its areas, 18 and its working groups. Note that other groups may also distribute 19 working documents as Internet-Drafts. 21 Internet-Drafts are draft documents valid for a maximum of six months 22 and may be updated, replaced, or obsoleted by other documents at any 23 time. It is inappropriate to use Internet-Drafts as reference 24 material or to cite them other than as "work in progress." 26 To view the entire list of current Internet-Drafts, please check the 27 "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow 28 Directories on ftp.is.co.za (Africa), ftp.nordu.net (Northern 29 Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au (Pacific 30 Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast). 32 Abstract 34 In order to overcome the problems caused by the undefined format of 35 the current FTP LIST command output, a new command is needed to 36 transfer standardized listing information from Server-FTP to Client- 37 FTP. Commands to enable this are defined in this document. 39 In order to allow consenting clients and servers to interact more 40 freely, a quite basic, and optional, virtual file store structure is 41 defined. 43 This proposal also extends the FTP protocol to allow character sets 44 other than US-ASCII[1] by allowing the transmission of 8-bit 45 characters and the recommended use of UTF-8[2] encoding. 47 Much implemented, but long undocumented, mechanisms to permit 48 restarts of interrupted data transfers in STREAM mode, are also 49 included here. 51 This version contains editorial, grammar, spelling, and punctuation 52 corrections. The Trivial Virtual File Store first appears in this 53 version. It gives some semantics to path names, and modifies the 54 specification of the ''cdir'' file type listing to make use of TVFS 55 semantics when available, and be undefined otherwise. The ''Open 56 Questions'' section has not been altered. All previously open 57 questions remain open. There are still no examples (other than one 58 for TVFS). This paragraph will be deleted from the final version of 59 this document. 61 Table of Contents 63 Abstract ................................................ 1 64 1 Introduction ............................................ 4 65 2 Document Conventions .................................... 4 66 2.1 Basic Tokens ............................................ 5 67 2.2 Pathnames ............................................... 5 68 2.3 Times ................................................... 7 69 2.4 Server Replies .......................................... 8 70 3 File Modification Time (MDTM) ........................... 8 71 3.1 Syntax .................................................. 9 72 3.2 Error responses ......................................... 9 73 3.3 FEAT response for MDTM .................................. 9 74 4 File SIZE ............................................... 10 75 4.1 Syntax .................................................. 10 76 4.2 Error responses ......................................... 10 77 4.3 FEAT response for SIZE .................................. 11 78 5 Restart of Interrupted Transfer (REST) .................. 11 79 5.1 Restarting in STREAM Mode ............................... 11 80 5.2 Error Recovery and Restart .............................. 12 81 5.3 Syntax .................................................. 13 82 5.4 FEAT response for REST .................................. 13 83 6 A Trivial Virtual File Store (TVFS) ..................... 14 84 6.1 TVFS File Names ......................................... 14 85 6.2 TVFS Path Names ......................................... 15 86 6.3 FEAT Response for TVFS .................................. 16 87 6.4 OPTS for TVFS ........................................... 17 88 6.5 TVFS Examples ........................................... 17 89 7 Listings for Machine Processing (MLST and MLSD) ......... 19 90 7.1 Format of MLST Request .................................. 20 91 7.2 Format of MLST Response ................................. 20 92 7.3 Filename encoding ....................................... 22 93 7.4 Format of Facts ......................................... 23 94 7.5 Standard Facts .......................................... 24 95 7.6 FEAT response for MLST .................................. 31 96 7.7 OPTS parameters for MLST ................................ 32 97 8 Interpretation of STAT command output ................... 32 98 8.1 FEAT response for STAT .................................. 33 99 9 Impact On Other FTP Commands ............................ 33 100 9.1 Impact on Pathnames and Filenames ....................... 34 101 10 Character sets and Internationalization ................. 34 102 11 IANA Considerations ..................................... 35 103 12 Security Non-Considerations ............................. 35 104 13 Open Issues ............................................. 35 105 13.1 General ................................................. 35 106 13.2 With MDTM ............................................... 35 107 13.3 With SIZE ............................................... 35 108 13.4 With REST ............................................... 35 109 13.5 With MLST/MLSD .......................................... 35 110 13.6 With STAT modifications ................................. 36 111 14 Very sketchy proposal ................................... 36 112 14.1 Syntax .................................................. 37 113 15 References .............................................. 37 114 Acknowledgements ........................................ 38 115 Copyright ............................................... 38 116 Editors' Addresses ...................................... 38 118 1. Introduction 120 This document amends the File Transfer Protocol (FTP) [6]. Four new 121 commands are added: "SIZE", "MDTM", "MLST", and "MLSD". Two existing 122 commands are modified, those are "REST" and "STAT". Of those, the 123 "SIZE" and "MDTM" commands, and the modifications to "REST" have been 124 in wide use for many years. The others are new. 126 These commands allow a client to restart an interrupted transfer in 127 transfer modes not previously supported in any documented way, and to 128 obtain a directory listing in a machine friendly, predictable, 129 format. 131 An optional structure for the server's file store (NVFS) is also 132 defined, allowing servers that support such a structure to convey 133 that information to clients in a standard way, thus allowing clients 134 more certainty in constructing and interpreting path names. 136 2. Document Conventions 138 This document makes use of the document conventions defined in BCP14 139 [9]. That provides the interpretation of capitalized imperative 140 words like MUST, SHOULD, etc. 142 This document also uses notation defined in STD 9 [6]. In 143 particular, the terms "reply", "user", "NVFS", "file", "pathname", 144 "FTP commands", "DTP", "user-FTP process", "user-PI", "user-DTP", 145 "server-FTP process", "server-PI", "server-DTP", "mode", "type", 146 "NVT", "control connection", "data connection", and "ASCII", are all 147 used here as defined there. 149 Syntax required is defined using the Augmented BNF defined in [3]. 150 Some general ABNF definitions are required throughout the document, 151 those will be defined later in this section. At first reading, it 152 may be wise to simply recall that these definitions exist here, and 153 skip to the next section. 155 2.1. Basic Tokens 157 This document imports the core definitions given in Appendix A of 158 [3]. There definitions will be found for basic ABNF elements like 159 ALPHA, DIGIT, SP, etc. To that, the following terms are added for 160 use in this document. 162 TCHAR = VCHAR / SP / HTAB ; visible plus white space 163 RCHAR = ALPHA / DIGIT / "," / "." / ":" / "!" / 164 "@" / "#" / "$" / "%" / "^" / 165 "&" / "(" / ")" / "-" / "_" / 166 "+" / "?" / "/" / "\" / "'" / 167 DQUOTE ; <"> -- double quote character (%x22) 169 The VCHAR (from [3]), TCHAR, and RCHAR types give basic character 170 types from varying sub-sets of the ASCII character set for use in 171 various commands and responses. 173 token = 1*RCHAR 175 A "token" is a string whose precise meaning depends upon the context 176 in which it is used. In some cases it will be a value from a set of 177 possible values maintained elsewhere. In others it might be a string 178 invented by one party to an FTP conversation from whatever sources it 179 finds relevant. 181 error-response = error-code SP *TCHAR CRLF 182 error-code = ("4" / "5") 2DIGIT 184 Note that in ABNF, string literals are case insensitive. That 185 convention is preserved in this document, and implies that FTP 186 commands added by this specification have names that can be 187 represented in any case. That is, "MDTM" is the same as "mdtm", 188 "Mdtm" and "MdTm" etc. However note that ALPHA, in particular, is 189 case sensitive. That implies that a "token" is a case sensitive 190 value. That implication is correct. 192 2.2. Pathnames 194 Various FTP commands take pathnames as arguments, or return pathnames 195 in responses. When the MLST command is supported, as indicated in 196 the response to the FEAT command [10], pathnames are to be 197 transferred in one of the following two formats. 199 pathname = utf-8-name / raw 200 utf-8-name = 201 raw = 203 Which format is used is at the option of the user-PI or server-PI 204 sending the pathname. UTF-8 encodings contain enough internal 205 structure that it is always, in practice, possible to determine 206 whether a UTF-8 or raw encoding has been used, in those cases where 207 it matters. While it is useful for the user-PI to be able to 208 correctly display a pathname received from the server-PI to the user, 209 it is far more important for the user-PI to be able to retain and 210 retransmit the identical pathname when required. Implementations are 211 advised against converting a UTF-8 pathname to a local encoding, and 212 then attempting to invert the encoding later. Note that ASCII is a 213 subset of UTF-8. 215 Unless otherwise specified, the pathname is terminated by the CRLF 216 that terminates the FTP command, or by the CRLF that ends a reply. 217 Any trailing spaces preceding that CRLF form part of the name. 218 Exactly one space will precede the pathname and serve as a separator 219 from the preceding syntax element. Any additional spaces form part 220 of the pathname. See [4] for a fuller explanation of the character 221 encoding issues. All implementations supporting MLST MUST support 222 [4]. 224 Implementations should also beware that the control connection uses 225 Telnet NVT conventions [11], and that the Telnet IAC character, if 226 part of a pathname sent over the control connection, MUST be 227 correctly escaped as defined by the Telnet protocol. 229 Implementors should also be aware that although Telnet NVT 230 conventions are used over the control connections, Telnet option 231 negotiation MUST NOT be attempted. See section 4.1.2.12 of [7]. 233 Except where TVFS is supported (see section 6) this specification 234 imposes no syntax upon pathnames. Nor does it restrict the the 235 character set from which pathnames are created. This does not imply 236 that the NVFS is required to make sense of all possible pathnames. 237 Server-PIs may restrict the syntax of valid pathnames in their NVFS 238 in any manner appropriate to their implementation or underlying 239 filesystem. Similarly, a server-PI may parse the pathname, and 240 assign meaning to the components detected. 242 2.3. Times 244 The syntax of a time value is: 246 time-val = 12DIGIT [ "." 1*DIGIT ] 248 The leading, mandatory, twelve digits are to be interpreted as, in 249 order from the leftmost, four digits giving the year, with a range of 250 1000-9999, two digits giving the month of the year, with a range of 251 01-12, two digits giving the day of the month, with a range of 01-31, 252 two digits giving the hour of the day, with a range of 00-23, two 253 digits giving minutes past the hour, with a range of 00-59, and 254 finally, two digits giving seconds past the minute, with a range of 255 00-60 (with 60 being used only at a leap second). Years in the tenth 256 century, and earlier, cannot be expressed. This is not considered a 257 serious defect of the protocol. 259 [ Ed-Note: Should we permit 12*DIGIT (or maybe 260 12*13DIGIT) so times in the 101st century and beyond can 261 be represented? ] 263 The optional digits, which are preceded by a period, give decimal 264 fractions of a second. These may be given to whatever precision is 265 appropriate to the circumstance, however implementations MUST NOT add 266 precision to time-vals where that precision does not exist in the 267 underlying value being transmitted. 269 Symbolically, a time-val may be viewed as 271 YYYYMMDDHHMMSS.sss 273 The "." and subsequent digits are optional. 275 Time values are always represented in UTC (GMT), and in the Gregorian 276 calendar regardless of what calendar may have been in use at the date 277 and time indicated at the location of the server-PI. 279 The technical differences between GMT, UTC, UT1, UT2, etc, are not 280 considered here. A server-FTP process should always use the same 281 time reference, so the times it returns will be consistent. Clients 282 are not expected to be time synchronized with the server, so the 283 possible difference in times that might be reported by the different 284 time standards is not considered important. 286 2.4. Server Replies 288 Section 4.2 of [6] defines the format and meaning of replies by the 289 server-PI to FTP commands from the user-PI. Those reply conventions 290 are used here without change. Implementors should note that the ABNF 291 syntax (which was not used in [6]) in this document, and other FTP 292 related documents, sometimes shows replies using the one line format. 293 Unless otherwise explicitly stated, that is not intended to imply 294 that multi-line responses are not permitted. Implementors should 295 assume that, unless stated to the contrary, any reply to any FTP 296 command (including QUIT) may be of the multiline format described in 297 [6]. 299 Throughout this document, replies will be identified by the three 300 digit code that is their first element. Thus the term "500 reply" 301 means a reply from the server-PI using the three digit code "500". 303 3. File Modification Time (MDTM) 305 The FTP command, MODIFICATION TIME (MDTM), can be used to determine 306 when a file in the server NVFS was last modified. This command has 307 existed in many FTP servers for many years, as an adjunct to the REST 308 command for STREAM mode, thus is widely available. However, where 309 supported, the "mtime" fact which can be provided in the result from 310 the new MLST command is recommended as a superior alternative. 312 When attempting to restart a RETRieve, if the User FTP makes use of 313 the MDTM command, or "mtime" fact, it can check and see if the 314 modification time of the source file is more recent than the 315 modification time of the partially transferred file. If it is, then 316 most likely the source file has changed and it would be unsafe to 317 restart in the middle of the file transfer. 319 When attempting to restart a STORe, the User FTP can use the MDTM 320 command to discover the modification time of the partially 321 transferred file. If it is older than the modification time of the 322 file that is about to be STORed, then most likely the source file has 323 changed and it would be unsafe to restart in the middle of the file 324 transfer. 326 Note that using MLST (described below) where available, can provide 327 this information, and much more, thus giving an even better 328 indication that a file has changed, and that restarting a transfer 329 would not give valid results. 331 Note that this is applicable to any RESTart attempt, regardless of 332 the mode of the file transfer. 334 3.1. Syntax 336 The syntax for the MDTM command is: 338 mdtm = "MdTm" SP pathname CRLF 340 As with all FTP commands, the "MDTM" command label is interpreted in 341 a case insensitive manner. 343 The server-PI will respond to the MDTM command with a 213 reply 344 giving the last modification time of the file whose pathname was 345 supplied, or an error response if the file does not exist, the 346 modification time is unavailable, or some other error has occurred. 348 mdtm-response = "213" SP time-val CRLF / 349 error-response 351 3.2. Error responses 353 Where the command is correctly parsed, but the modification time is 354 not available, either because the pathname identifies no existing 355 entity, or because the information is not available for the entity 356 named, then a 550 reply should be sent. Where the command cannot be 357 correctly parsed, a 500 or 501 reply should be sent, as specified in 358 [6]. 360 3.3. FEAT response for MDTM 362 When replying to the FEAT command [10], a FTP server process that 363 supports the MDTM command MUST include a line containing the single 364 word "MDTM". This MAY be sent in upper or lower case, or a mixture 365 of both (it is case insensitive) but SHOULD be transmitted in upper 366 case only. That is, the response SHOULD be 368 C> Feat 369 S> 211- 370 S> ... 371 S> MDTM 372 S> ... 373 S> 211 End 375 The ellipses indicate placeholders where other features may be 376 included, and are not required. The one space indentation of the 377 feature lines is mandatory [10]. 379 4. File SIZE 381 The FTP command, SIZE OF FILE (SIZE), is used to obtain the transfer 382 size of a file from the server-FTP process. That is, the exact 383 number of octets (8 bit bytes) which would be transmitted over the 384 data connection should that file be transmitted. This value will 385 change depending on the current STRUcture, MODE and TYPE of the data 386 connection, or a data connection which would be created were one 387 created now. Thus, the result of the SIZE command is dependent on 388 the currently established STRU, MODE and TYPE parameters. 390 The SIZE command returns how many octets would be transferred if the 391 file were to be transferred using the current transfer structure, 392 mode and type. This command is normally used in conjunction with the 393 RESTART (REST) command. The server-PI might need to read the 394 partially transferred file, do any appropriate conversion, and count 395 the number of octets that would be generated when sending the file in 396 order to correctly respond to this command. Estimates of the file 397 transfer size MUST NOT be returned, only precise information is 398 acceptable. 400 4.1. Syntax 402 The syntax of the SIZE command is: 404 size = "Size" SP pathname CRLF 406 The server-PI will respond to the SIZE command with a 213 reply 407 giving the transfer size of the file whose pathname was supplied, or 408 an error response if the file does not exist, the size is 409 unavailable, or some other error has occurred. The value returned is 410 in a format suitable for use with the RESTART (REST) command for mode 411 STREAM, provided the transfer mode and type are not altered. 413 size-response = "213" SP 1*DIGIT CRLF / 414 error-response 416 4.2. Error responses 418 Where the command is correctly parsed, but the size is not available, 419 either because the pathname identifies no existing entity, or because 420 the entity named cannot be transferred in the current MODE and TYPE 421 (or at all), then a 550 reply should be sent. Where the command 422 cannot be correctly parsed, a 500 or 501 reply should be sent, as 423 specified in [6]. 425 4.3. FEAT response for SIZE 427 When replying to the FEAT command [10], a FTP server process that 428 supports the SIZE command MUST include a line containing the single 429 word "SIZE". This word is case insensitive, and MAY be sent in any 430 mixture of upper or lower case, however it SHOULD be sent in upper 431 case. That is, the response SHOULD be 433 C> FEAT 434 S> 211- 435 S> ... 436 S> SIZE 437 S> ... 438 S> 211 END 440 The ellipses indicate placeholders where other features may be 441 included, and are not required. The one space indentation of the 442 feature lines is mandatory [10]. 444 5. Restart of Interrupted Transfer (REST) 446 To avoid having to resend the entire file if the file is only 447 partially transferred, both sides need some way to be able to agree 448 on where in the data stream to restart the data transfer. 450 The FTP specification [6] includes three modes of data transfer, 451 Stream, Block and Compressed. In Block and Compressed modes, the 452 data stream that is transferred over the data connection is 453 formatted, allowing the embedding of restart markers into the stream. 454 The sending DTP can include a restart marker with whatever 455 information it needs to be able to restart a file transfer at that 456 point. The receiving DTP can keep a list of these restart markers, 457 and correlate them with how the file is being saved. To restart the 458 file transfer, the receiver just sends back that last restart marker, 459 and both sides know how to resume the data transfer. Note that there 460 are some flaws in the description of the restart mechanism in RFC 959 461 [6]. See section 4.1.3.4 of RFC 1123 [7] for the corrections. 463 5.1. Restarting in STREAM Mode 465 In Stream mode, the data connection contains just a stream of 466 unformatted octets of data. Explicit restart markers thus cannot be 467 inserted into the data stream, they would be indistinguishable from 468 data. For this reason, the FTP specification [6] did not provide the 469 ability to do restarts in stream mode. However, there is not really 470 a need to have explicit restart markers in this case, as restart 471 markers can be implied by the octet offset into the data stream. 473 Because the data stream defines the file in STREAM mode, a different 474 data stream would represent a different file. Thus, an offset will 475 always represent the same position within a file. On the other hand, 476 in other modes than STREAM, the same file can be transferred using 477 quite different octet sequences, and yet be reconstructed into the 478 one identical file. Thus an offset into the data stream in transfer 479 modes other than STREAM would not give an unambiguous restart point. 481 If the data representation TYPE is IMAGE, and the STRUcture is File, 482 for many systems the file will be stored exactly in the same format 483 as it is sent across the data connection. It is then usually very 484 easy for the receiver to determine how much data was previously 485 received, and notify the sender the offset where the transfer should 486 be restarted. In other representation types and structures more 487 effort will be required, but it remains always possible to determine 488 the offset with finite, but perhaps non-negligible, effort. In the 489 worst case an FTP process may need to open a data connection to 490 itself, set the appropriate transfer type and structure, and actually 491 transmit the file, counting the transmitted octets. 493 If the user-FTP process is intending to restart a retrieve, it will 494 directly calculate the restart marker, and send that information in 495 the RESTart command. However, if the user-FTP process is intending 496 to restart sending the file, it needs to be able to determine how 497 much data was previously sent, and correctly received and saved. A 498 new FTP command is needed to get this information. This is the 499 purpose of the SIZE command, as documented in section 4. 501 5.2. Error Recovery and Restart 503 STREAM MODE transfers with FILE STRUcture may be restarted even 504 though no restart marker has been transferred in addition to the data 505 itself. This is done by perhaps the SIZE command, if needed, in 506 combination with the RESTART (REST) command, and one of the standard 507 file transfer commands. 509 When using TYPE ASCII or IMAGE, the SIZE command will return the 510 number of octets that would actually be transferred if the file were 511 to be sent between the two systems. I.e. with type IMAGE, the SIZE 512 normally would be the number of octets in the file. With type ASCII, 513 the SIZE would be the number of octets in the file including any 514 modifications required to satisfy the TYPE ASCII CR-LF end of line 515 convention. 517 5.3. Syntax 519 The syntax for the REST command when the current transfer mode is 520 STREAM is: 522 rest = "Rest" SP 1*DIGIT CRLF 524 The numeric value gives the number of octets of the immediately 525 following transfer to not actually send, effectively causing the 526 transmission to be restarted at a later point. A value of zero 527 effectively disables restart, causing the entire file to be 528 transmitted. The server-PI will respond to the REST command with a 529 350 reply, indicating that the REST parameter has been saved, and 530 that another command, which should be either RETR or STOR, should 531 then follow to complete the restart. 533 rest-response = "350" SP *TCHAR CRLF / 534 error-response 536 Server-FTP processes may permit transfer commands other than RETR and 537 STOR, such as APPE and STOU, to complete a restart, however, this is 538 not recommended. STOU (store unique) is undefined in this usage, as 539 storing the remainder of a file into a unique filename is rarely 540 going to be useful. If APPE (append) is permitted, it MUST act 541 identically to STOR when a restart marker has been set. That is, in 542 both cases, octets from the data connection are placed into the file 543 at the location indicated by the restart marker value. 545 An error-response will follow a REST command only when the server 546 does not implement the command, or the restart marker value is 547 syntactically invalid for the current transfer mode. That is, in 548 STREAM mode, if something other than one or more digits appears in 549 the parameter to the REST command. Any other errors, including such 550 problems as restart marker out of range, should be reported when the 551 following transfer command is issued. 553 5.4. FEAT response for REST 555 Where a server-FTP process supports RESTart in STREAM mode, as 556 specified here, it MUST include in the response to the FEAT command 557 [10], a line containing exactly the string "REST STREAM". This 558 string is not case sensitive, but SHOULD be transmitted in upper 559 case. Where REST is not supported at all, or supported only in block 560 or compressed modes, the REST line MUST NOT be included in the FEAT 561 response. Where required, the response SHOULD be 562 C> feat 563 S> 211- 564 S> ... 565 S> REST STREAM 566 S> ... 567 S> 211 end 569 The ellipses indicate placeholders where other features may be 570 included, and are not required. The one space indentation of the 571 feature lines is mandatory [10]. 573 6. A Trivial Virtual File Store (TVFS) 575 Traditionally, FTP has placed almost no constraints upon the file 576 store (NVFS) provided by a server. This specification does not alter 577 that. However, it has become common for servers to attempt to 578 provide at least file system naming conventions modeled loosely upon 579 those of the UNIX(TM) filesystem. That is, a tree structured 580 filesystem, built of directories, each of which can contain other 581 directories, or other kinds of files, or both. Each file and 582 directory has a file name relative to the directory that contains it, 583 except for the directory at the root of the tree, which is contained 584 in no other directory, and hence has no name of its own. 586 That which has so far been described is perfectly consistent with the 587 standard FTP NVFS and access mechanisms. The "CWD" command is used 588 to move from one directory to an embedded directory. "CDUP" may be 589 provided to return to the parent directory, and the various file 590 manipulation commands ("RETR", "STOR", the rename commands, etc) are 591 used to manipulate files within the current directory. 593 However, it is often useful to be able to reference files other than 594 by changing directories, especially as FTP provides no guaranteed 595 mechanism to return to a previous directory. The Trivial Virtual 596 File Store (TVFS), if implemented, provides that mechanism. 598 6.1. TVFS File Names 600 Where a server implements the TVFS, no elementary filename shall 601 contain the character "/". Where the underlying natural file store 602 permits files, or directories, to contain the "/" character in their 603 names, a server implementing TVFS must encode that character in some 604 manner whenever file or directory names are being returned to the 605 client, and reverse that encoding whenever such names are being 606 accepted from the client. 608 The encoding method to be used is not specified here. Where some 609 other character is illegal in file and directory names in the 610 underlying filestore, a simple transliteration may be sufficient. 611 Where there is no suitable substitute character a more complex 612 encoding scheme, possibly using an escape character, is likely to be 613 required. 615 With the one exception of the unnamed root directory, a TVFS file 616 name may not be empty. That is, all other file names contain at 617 least one character. 619 With the sole exception of the "/" character, any valid IS10646 620 character may be used in a TVFS filename. When transmitted, file 621 name characters are encoded using the UTF-8 encoding. 623 6.2. TVFS Path Names 625 A TVFS "Path Name" combines the file or directory name of a target 626 file or directory, with the directory names of zero or more enclosing 627 directories, so as to allow the target file or directory to be 628 referenced other than when the server's "current working directory" 629 is the directory directly containing the target file or directory. 631 By definition, every TVFS file or directory name is also a TVFS path 632 name. Such a path name is valid to reference the file from the 633 directory containing the name, that is, when that directory is the 634 server-FTP's current working directory. 636 Other TVFS path names are constructed by prefixing a path name by a 637 name of a directory from which the path is valid, and separating the 638 two with the "/" character. Such a path name is valid to reference 639 the file or directory from the directory containing the newly added 640 directory name. 642 Where a path name has been extended to the point where the directory 643 added is the unnamed root directory, the path name will begin with 644 the "/" character. Such a path is known as a fully qualified path 645 name. Fully qualified paths may, obviously, not be further extended, 646 as, by definition, no directory contains the root directory. Being 647 unnamed, it cannot be represented in any other directory. A fully 648 qualified path name is valid to reference the named file or directory 649 from any location (that is, regardless of what the current working 650 directory may be) in the virtual file store. 652 Any path name which is not a fully qualified path name may be 653 referred to as a "relative path name" and will only correctly 654 reference the intended file when the current working directory of the 655 server-FTP is a directory from which the relative path name is valid. 657 As a special case, the path name "/" is defined to be a fully 658 qualified path name referring to the root directory. That is, the 659 root directory does not have a directory (or file) name, but does 660 have a path name. 662 6.2.1. Notes 664 + It is not required, or expected, that there be only one fully 665 qualified path name that will reference any particular file or 666 directory. 667 + As a caveat, though the TVFS file store is basically tree 668 structured, there is no requirement that any file or directory 669 have only one parent directory. 670 + As defined, no TVFS path name will ever contain two consecutive 671 "/" characters. Such a name is not illegal however, and may be 672 defined by the server for any purpose that suits it. Clients 673 implementing this specification should not assume any semantics 674 at all for such names. 675 + Similarly, other than the special case path that refers to the 676 root directory, no TVFS path name constructed as defined here 677 will ever end with the "/" character. Such names are also not 678 illegal, but are undefined. 679 + While any legal IS10646 character is permitted to occur in a TVFS 680 file or directory name, other than "/", server FTP 681 implementations are not required to support all possible IS10646 682 characters. The subset supported is entirely at the discretion 683 of the server. The case (where it exists) of the characters that 684 make up file, directory, and path names may be significant. 685 Unless determined otherwise by means unspecified here, clients 686 should assume that all such names are comprised of characters 687 whose case is significant. Servers are free to treat case (or 688 any other attribute) of a name as irrelevant, and hence map two 689 names which appear to be distinct onto the same underlying file. 690 + There are no defined "magic" names, like ".", ".." or "C:". 691 Servers may implement such names, with any semantics they choose, 692 but are not required to do so. 693 + TVFS imposes no particular semantics or properties upon files, 694 guarantees no access control schemes, or any of the other common 695 properties of a file store. Only the naming scheme is defined. 697 6.3. FEAT Response for TVFS 699 In response to the the FEAT command [10] a server that wishes to 700 indicate support for the TVFS as defined here will include a line 701 that begins with the four characters "TVFS" (in any case, or mixture 702 of cases, upper case is not required). Servers SHOULD send upper 703 case. 705 Such a response to the FEAT command MUST NOT be returned unless the 706 server implements TVFS as defined here. 708 Later specifications may add to the TVFS definition. Such additions 709 should be notified by means of additional text appended to the TVFS 710 feature line. Such specifications, if any, will define the extra 711 text. 713 Until such a specification is defined, servers should not include 714 anything after "TVFS" in the TVFS feature line. Clients, however, 715 should be prepared to deal with arbitrary text following the four 716 defined characters, and simply ignore it if unrecognized. 718 A typical response to the FEAT command issued by a server 719 implementing only this specification would be: 721 C> feat 722 S> 211- 723 S> ... 724 S> TVFS 725 S> ... 726 S> 211 end 728 The ellipses indicate placeholders where other features may be 729 included, and are not required. The one space indentation of the 730 feature lines is mandatory [10], and is not counted as one of the 731 first four characters for the purposes of this feature listing. 733 The TVFS feature adds no new commands to the FTP command repertoire. 735 6.4. OPTS for TVFS 737 There are no options in this TVFS specification, and hence there is 738 no OPTS command defined. 740 6.5. TVFS Examples 742 Assume a TVFS file store is comprised of a root directory, which 743 contains two directories (A and B) and two non-directory files (X and 744 Y). The A directory contains two directories (C and D) and one other 745 file (Z). The B directory contains just two non-directory files (P 746 and Q) and the C directory also two non-directory files (also named P 747 and Q, by chance). The D directory is empty, that is, contains no 748 files or directories. 750 This structure may depicted graphically as... 752 (unnamed root) 753 / | \ \ 754 / | \ \ 755 A X B Y 756 /|\ / \ 757 / | \ / \ 758 C D Z P Q 759 / \ 760 / \ 761 P Q 763 Given this structure, the following fully qualified path names exist. 765 / 766 /A 767 /B 768 /X 769 /Y 770 /A/C 771 /A/D 772 /A/Z 773 /A/C/P 774 /A/C/Q 775 /B/P 776 /B/Q 778 It is clear that none of the paths / /A /B or /A/D refer to the same 779 directory, as the contents of each is different. Nor do any of / /A 780 /A/C or /A/D. However /A/C and /B might be the same directory, there 781 is insufficient information given to tell. Any of the other path 782 names (/X /Y /A/Z /A/C/P /A/C/Q /B/P and /B/Q) may refer to the same 783 underlying files, in almost any combination. 785 If the current working directory of the server-FTP is /A then the 786 following path names, in addition to all the fully qualified path 787 names, are valid 789 C 790 D 791 Z 792 C/P 793 C/Q 795 These all refer to the same files or directories as the corresponding 796 fully qualified path with "/A/" prepended. 798 That those path names all exist does not imply that the TVFS sever 799 will necessarily grant any kind of access rights to the named paths, 800 or that access to the same file via different path names will 801 necessarily be granted equal rights. 803 None of the following relative paths are valid when the current 804 directory is /A 806 A 807 B 808 X 809 Y 810 B/P 811 B/Q 812 P 813 Q 815 Any of those could be made valid by changing the server-FTP's current 816 working directory to the appropriate directory. Note that the paths 817 "P" and "Q" might refer to different files depending upon which 818 directory is selected to cause those to become valid TVFS relative 819 paths. 821 7. Listings for Machine Processing (MLST and MLSD) 823 The MLST and MLSD commands are intended to standardize the file and 824 directory information returned by the Server-FTP process. These 825 commands differ from the LIST command in that the format of the 826 replies is strictly defined although extensible. 828 Two commands are defined, MLST which provides data about exactly the 829 object named on its command line, and no others. MLSD on the other 830 hand will list the contents of a directory if a directory is named, 831 otherwise a 501 error reply will be returned. In either case, if no 832 object is named, the current directory is assumed. That will cause 833 MLST to send a one line response, describing the current directory 834 itself, and MLSD to list the contents of the current directory. 836 [ Ed-Note: Do we need something for recursive listings ? 837 ] 839 In the sequel only MLST will be described. Other than as previously 840 mentioned, MLSD is identical. 842 The MLST and MLSD commands also extend the FTP protocol as presented 843 in RFC 959 [6] and RFC 1123 [7] to allow that transmission of 8-bit 844 data over the control connection. Note this is not specifying 845 character sets which are 8-bit, but specifying that FTP 846 implementations are to specifically allow the transmission and 847 reception of 8-bit bytes, with all bits significant, over the control 848 connection. That is, all 256 possible octet values are permitted. 849 The MLST command allows both UTF-8/Unicode and "raw" forms as 850 arguments, and in responses to the MLST and MLSD commands, and all 851 other FTP commands which take pathnames as arguments. 853 7.1. Format of MLST Request 855 The MLST and MLSD commands each allow a single optional argument. 856 This argument may be either a directory name or a filename. For 857 these purposes, a "filename" is the name of any entity in the server 858 NVFS which is not a directory. Where TVFS is supported, any TVFS 859 path name valid in the current working directory may be given. If a 860 directory name is given then MLSD must return a listing of the 861 contents of the named directory, otherwise it issues a 501 reply, and 862 does not open a data connection. In all cases for MLST, only a 863 single fact line containing the information about the named file or 864 directory shall be returned. 866 If no argument is given then MLSD must return a listing of the 867 contents of the current working directory, and MLST must return a 868 listing giving information about the current working directory 869 itself. 871 No title, or header, lines, or any other formatting, other than as is 872 specified below, is ever returned in the output of an MLST or MLSD 873 command. 875 If the Client-FTP sends an invalid argument, the Server-FTP MUST 876 reply with an error code of 501. 878 The syntax for the MLST command is: 880 mlst = "MLst" [ SP pathname ] CRLF 881 mlsd = "MLsD" [ SP pathname ] CRLF 883 7.2. Format of MLST Response 885 The format of a response to the MLST command is as follows: 887 mlst-response = ( initial-response final-response ) / 888 error-response 889 mlsd-response = mlst-response 890 initial-response = "150" [ SP response-message ] CRLF 891 response-message = *TCHAR 892 final-response = "226" SP response-message CRLF 894 data-response = *( entry CRLF ) 895 entry = [ facts ] SP pathname 896 facts = fact *( ";" fact ) 897 fact = factname "=" value 898 factname = "Size" / "Modify" / "Create" / 899 "Type" / "Unique" / "Perm" / 900 "Lang" / "Media-Type" / "CharSet" / 901 os-depend-fact / local-fact 902 os-depend-fact = "." 1*RCHAR 903 local-fact = "X." 1*RCHAR 904 value = 1*RCHAR 906 Upon receipt of a MLST or MLSD command, the server will verify the 907 parameter, and if invalid return an error-response. If valid, the 908 server will open a data connection as indicated in section 3.2 of 909 RFC959 [6]. If that fails, the server will return an error-response. 910 If all is OK, the server will return the initial-response, send the 911 appropriate data-response over the new data connection, close that 912 connection, and then send the final-response. The grammar above 913 defines the format for the data-response. 915 The data connection opened for a MLST or MLSD response shall be a 916 connection as if the "TYPE L 8", "MODE S", and "STRU F" commands had 917 been given, whatever FTP transfer type, mode and structure had 918 actually been set, and without causing those settings to be altered 919 for future commands. That is, this transfer type shall be set for 920 the duration of the data connection established for this command 921 only. While the content of the data sent can be viewed as a series 922 of lines, implementations should note that there is no maximum line 923 length defined. Implementations should be prepared to deal with 924 arbitrarily long lines. 926 The facts part of the specification would contain a series of "file 927 facts" about the file or directory named on the same line. Typical 928 information to be presented would include file size, last 929 modification time, creation time, a unique identifier, and a 930 file/directory flag. 932 The complete format for a successful reply to the MLSD command would 933 be: 935 facts SP pathname CRLF 936 facts SP pathname CRLF 937 facts SP pathname CRLF 938 ... 940 Note that the format is intended for machine processing, not human 941 viewing, and as such the format is very rigid. Implementations MUST 942 NOT vary the format by, for example, inserting extra spaces for 943 readability, replacing spaces by tabs, including header or title 944 lines, or inserting blank lines, or in any other way alter this 945 format. Exactly one space is always required after the set of facts 946 (which may be empty). More spaces may be present on a line if, and 947 only if, the file name presented contains significant spaces. The 948 set of facts must not contain any spaces anywhere inside it. 950 7.3. Filename encoding 952 A FTP implementation supporting the MLST command must be 8-bit clean. 953 This is necessary in order to transmit UTF-8 encoded filenames. This 954 specification recommends the use of UTF-8 encoded filenames. FTP 955 implementations SHOULD use UTF-8 whenever possible to encourage the 956 maximum interoperability. 958 Filenames are not restricted to UTF-8, however treatment of arbitrary 959 character encodings is not specified by this standard. Applications 960 are encouraged to treat non-UTF-8 encodings of filenames as octet 961 sequences. 963 Note that this encoding is unrelated to that of the contents of the 964 file, even if the file contains character data. 966 Further information about filename encoding for FTP may be found in 967 "Internationalization of the File Transfer Protocol" [4]. 969 7.3.1. Notes about the Filename 971 The filename returned in the MLST response should be the same name as 972 was specified in the MLST command, or, where TVFS is supported, a 973 fully qualified TVFS path naming the same file. Where no argument 974 was given to the MLST command, the server-PI may either include an 975 empty filename in the response, or it may supply a name that refers 976 to the current directory, if such a name is available. Where TVFS is 977 supported, the fully qualified path name of the current directory 978 SHOULD be returned. 980 Filenames returned in the output from an MLSD command should be 981 unqualified names within the directory named, or the current 982 directory if no argument was given. That is, the directory named in 983 the MLSD command should not appear as a component of the filenames 984 returned. 986 If the server-FTP process is able, and the "type" fact is being 987 returned, it MAY return in the MLSD response, an entry whose type is 988 "cdir", which names the directory from which the contents of the 989 listing were obtained. Where TVFS is supported, the name may be the 990 fully qualified path name of the directory, or may be any other path 991 name which is valid to refer to that directory from the current 992 working directory of the server-FTP. Where more than one name 993 exists, multiple of these entries may be returned. In a sense, the 994 "cdir" entry can be viewed as a heading for the MLSD output. 995 However, it is not required to be the first entry returned, and may 996 occur anywhere withing the listing. 998 Where TVFS is not supported, type "cdir" names are not terribly 999 useful, as no defined meaning is attached to them. When TVFS is 1000 supported, a user-PI can refer to any file or directory in the 1001 listing by taking a type "cdir" name, appending a "/", and following 1002 that by the appropriate name from the directory listing. 1004 Alternatively, the user-PI can issue a CWD command ([6]) giving the 1005 name of type "cdir", and from that point reference the files returned 1006 in the MLSD response from which the cdir was obtained by using the 1007 filename components of the listing. Once having attempted any CWD 1008 command however, it is no longer guaranteed that a file can be 1009 referenced by the combination of type "cdir" and other names, whether 1010 using CWD or name concatenation, unless the type "cdir" name was a 1011 fully qualified path name, and TVFS is supported. 1013 [ Ed-Note: This whole scheme is (yet again) open to 1014 revision or removal - more discussion of its worth and if 1015 worthwhile, the details of the scheme is needed ] 1017 7.3.2. Examples 1019 Once upon a (future) time, examples existed here. 1021 7.4. Format of Facts 1023 The "facts" for a file in a reply to a MLST command consist of 1024 information about that file. The facts are a series of keyword=value 1025 pairs separated by a semi-colon (";") character. The complete series 1026 of facts may not contain the space character. 1028 A sample of a typical series of facts would be: (spread over two 1029 lines for presentation here only) 1031 size=4161;lang=en-us;modify=19970214165800;create=19961001124534; 1032 type=file;x.myfact=foo,bar 1034 7.5. Standard Facts 1036 This document defines a standard set of facts as follows: 1038 size -- Size in octets 1039 modify -- Last modification time 1040 create -- Creation time 1041 type -- Entry type 1042 unique -- Unique id of file/directory 1043 perm -- File permissions, whether read, write, execute is 1044 allowed for the login id. 1045 lang -- Language of the filename per IANA[5] registry. 1046 media-type -- MIME media-type of file contents per IANA registry. 1047 charset -- Character set per IANA registry (if not UTF-8) 1049 Fact names are case-insensitive. Size, size, SIZE, and SiZe are the 1050 same fact. 1052 Further operating system specific keywords could be specified by 1053 using the IANA operating system name as a prefix (examples only): 1055 OS/2.ea -- OS/2 extended attributes 1056 MACOS.rf -- MacIntosh resource forks 1057 UNIX.mode -- Unix file modes (permissions) 1059 Implementations may define keywords for experimental, or private use. 1060 All such keywords MUST begin with the two character sequence "x.". 1061 As type names are case independent, "x." and "X." are equivalent. 1062 For example: 1064 x.ver -- Version information 1065 x.desc -- File description 1066 x.type -- File type 1068 7.5.1. The type Fact 1070 The type fact needs a special description. Part of the problem with 1071 current practices is deciding when a file is a directory. If it is a 1072 directory, is it the current directory, a regular directory, or a 1073 parent directory? The MLST specification makes this unambiguous 1074 using the type fact. The type fact given specifies information about 1075 the object listed on the same line of the MLST response. 1077 Five values are possible for the type fact: 1079 file -- a file entry 1080 cdir -- the listed directory 1081 pdir -- the parent directory 1082 dir -- a directory or sub-directory 1083 OS.name=type -- an OS or file system dependent file type 1085 The syntax is defined to be: 1087 type-fact = type-label "=" type-val 1088 type-label = "Type" 1089 type-val = "File" / "cdir" / "pdir" / "dir" / 1090 os-type 1092 7.5.1.1. type=file 1094 The presence of the type=file fact indicates the listed entry is a 1095 file containing non-system data. That is, it may be transferred from 1096 one system to another of quite different characteristics, and perhaps 1097 still be meaningful. 1099 [ Ed-Note: There is an open question on the best way to 1100 indicate objects that are both files and directories. ] 1102 7.5.1.2. type=cdir 1104 The type=cdir fact indicates the listed entry is a pathname of the 1105 directory whose contents are listed. This type will only be returned 1106 as a part of the result of an MLSD command, and provides a name for 1107 the listed directory, and facts about that directory. In a sense, it 1108 can be viewed as representing the title of the listing, in a machine 1109 friendly format. It may appear at any point of the listing, it is 1110 not restricted to appearing at the start, though frequently may do 1111 so, and may occur multiple times. 1113 Where TVFS is supported by the server-FTP, this name may be used to 1114 construct path names with which to refer to the files and directories 1115 returned in the same MLSD output. These commands are only expected 1116 to work when the server-PIs position in the NFVS file tree is the 1117 same as its position when the MLSD command was issued. 1119 Where TVFS is not supported, there are no defined semantics 1120 associated with a "type=cdir" entry. 1122 7.5.1.3. type=dir 1124 If present, the type=dir entry is the name of a directory. When 1125 executed with the current directory in the same place in the NVFS as 1126 it was when the MLST or MLSD command was issued, a CWD with its 1127 argument being the formed by appending the name with type=pdir to a 1128 name with type=cdir should succeed (assuming the user has the 1129 appropriate access rights). 1131 7.5.1.4. type=pdir 1133 If present, which will occur only in the response to a MLSD command, 1134 the type=pdir entry represents a pathname of the parent directory of 1135 the listed directory. As well as having the properties of a 1136 type=dir, a CWD command with the appropriate value formed from the 1137 name given with this type should change the user to the parent 1138 directory of the listed directory. If the listed directory is the 1139 current directory, a CDUP command may also have the effect of 1140 changing to the named directory. User-FTP processes should note not 1141 all responses will include this information, and that some systems 1142 may provide multiple type=pdir responses. 1144 Where TVFS is supported, a "type=pdir" name may be a relative path 1145 name, or a fully qualified path name. 1147 For the purposes of this type value, a "parent directory" is any 1148 directory in which there is an entry of type=dir which refers to the 1149 directory in which the type=pdir entity was found. Thus it is not 1150 required that all entities with type=pdir refer to the same 1151 directory, the "unique" fact can be used to determine whether there 1152 is a relationship between the type=pdir entries or not. 1154 7.5.1.5. System defined types 1156 Files types that are specific to a specific operating system, or file 1157 system, can be encoded using the "OS." type names. The format is: 1159 os-type = "OS." os-name "=" localtype 1160 os-name = 1161 localtype = 1*RCHAR 1163 The "os-name" indicates the specific system type which supports the 1164 particular localtype. It should be taken from the IANA maintained 1165 list of operating systems wherever possible. The "localtype" 1166 provides the system dependent information as to the type of the file 1167 listed. The os-name and localtype strings in an os-type are case 1168 independent. "OS.unix=block" and "OS.Unix=BLOCK" represent the same 1169 type. 1171 Note: Where the underlying system supports a file type which is 1172 essentially an indirect pointer to another file, the NVFS 1173 representation of that type should normally be to represent the file 1174 which the reference indicates. That is, the underlying basic file 1175 will appear more than once in the NVFS, each time with the "unique" 1176 fact (see immediately following section) containing the same value, 1177 indicating that the same file is represented by all such names. 1178 User-PIs transferring the file need then transfer it only once, and 1179 then insert their own form of indirect reference to construct 1180 alternate names where desired, or perhaps even copy the local file if 1181 that is the only way to provide two names with the same content. A 1182 file which would be a reference to another file, if only the other 1183 file actually existed, may be represented in any OS dependent manner 1184 appropriate, or not represented at all. 1186 7.5.2. The unique Fact 1188 The unique fact is used to present a unique identifier for a file or 1189 directory in the NVFS accessed via a server-FTP process. The value 1190 of this fact should be the same for any number of pathnames that 1191 refer to the same underlying file. The fact should have different 1192 values for names which reference distinct files. The mapping between 1193 files, and unique fact tokens should be maintained, and remain 1194 consistent, for at least the lifetime of the control connection from 1195 user-PI to server-PI. 1197 unique-fact = "Unique" "=" token 1199 This fact would be expected to be used by Server-FTPs whose host 1200 system allows things such as symbolic links so that the same file may 1201 be represented in more than one directory on the server. The only 1202 conclusion that should be drawn is that if two different names each 1203 have the same value for the unique fact, they refer to the same 1204 underlying object. The value of the unique fact (the token) should 1205 be considered an opaque string for comparison purposes, and is a case 1206 dependent value. The tokens "A" and "a" do not represent the same 1207 underlying object. 1209 7.5.3. The modify Fact 1211 The modify fact is used to determine the last time the content of the 1212 file (or directory) indicated was modified. Any change of substance 1213 to the file should cause this value to alter. That is, if a change 1214 is made to a file such that the results of a RETR command would 1215 differ, then the value of the modify fact should alter. User-PIs 1216 should not assume that a different modify fact value indicates that 1217 the file contents are necessarily different than when last retrieved. 1218 Some systems may alter the value of the modify fact for other 1219 reasons, though this is discouraged wherever possible. Also a file 1220 may alter, and then be returned to its previous content, which would 1221 often be indicated as two incremental alterations to the value of the 1222 modify fact. 1224 For directories, this value should alter whenever a change occurs to 1225 the directory such that different filenames would (or might) be 1226 included in MLSD output of that directory. 1228 modify-fact = "Modify" "=" time-val 1230 7.5.4. The create Fact 1232 The create fact indicates when a file, or directory, was first 1233 created. Exactly what "creation" is for this purpose is not 1234 specified here, and may vary from server to server. About all that 1235 can be said about the value returned is that it can never indicate a 1236 later time than the mtime fact. 1238 create-fact = "Create" "=" time-val 1240 Implementation Note: Implementors of this fact on UNIX(TM) systems 1241 should note that the unix "stat" "st_ctime" field does not give 1242 creation time, and that unix filesystems do not record creation 1243 time at all. Unix (and POSIX) implementations will normally not 1244 include this fact. 1246 7.5.5. The perm Fact 1248 The perm fact is used to indicate access rights the current FTP user 1249 has over the object listed. Its value is always an unordered 1250 sequence of alphabetic characters. 1252 perm-fact = "Perm" "=" pvals 1253 pvals = "a" / "c" / "d" / "e" / "f" / 1254 "l" / "m" / "p" / "r" / "w" 1256 There are ten permission indicators currently defined. Many are 1257 meaningful only when used with a particular type of object. The 1258 indicators are case independent, "d" and "D" are the same indicator. 1260 The "a" permission applies to objects of type=file, and indicates 1261 that the APPE (append) command may be applied to the file named. 1263 The "c" permission applies to objects of type=dir (and type=pdir, 1264 type=cdir). It indicates that files may be created in the directory 1265 named. That is, that a STOU command is likely to succeed, and that 1266 STOR and APPE commands might succeed if the file named did not 1267 previously exist, but is to be created in the directory object that 1268 has the "c" permission. It also indicates that the RNTO command is 1269 likely to succeed for names in the directory. 1271 The "d" permission applies to all types. It indicates that the 1272 object named may be deleted, that is, that the RMD command may be 1273 applied to it if it is a directory, and otherwise that the DELE 1274 command may be applied to it. 1276 The "e" permission applies to the directory types. When set on an 1277 object of type=dir, type=cdir, or type=pdir it indicates that a CWD 1278 command naming the object should succeed, and the user should be able 1279 to enter the directory named. For type=pdir it also indicates that 1280 the CDUP command should succeed. 1282 The "f" permission for objects indicates that the object named may be 1283 renamed - that is, may be the object of an RNFR command. 1285 The "l" permission applies to the directory file types, and indicates 1286 that the listing commands, LIST, NLST, and MLSD may be applied to the 1287 directory in question, and that MLST, LIST, NLST, and STAT may be 1288 applied to objects in the directory. 1290 The "m" permission applies to directory types, and indicates that the 1291 MKD command may be used to create a new directory within the 1292 directory under consideration. 1294 The "p" permission applies to directory types, and indicates that 1295 objects in the directory may be deleted, or (stretching naming a 1296 little) that the directory may be purged. Note: it does not indicate 1297 that the RMD command may be used to remove the directory named, the 1298 "d" permission indicator indicates that. 1300 The "r" permission applies to type=file objects, and for some 1301 systems, perhaps to other types of objects, and indicates that the 1302 RETR command may be applied to that object. 1304 The "w" permission applies to type=file objects, and for some 1305 systems, perhaps to other types of objects, and indicates that the 1306 STOR command may be applied to the object named. 1308 Note: That a permission indicator is set can never imply that the 1309 appropriate command is guaranteed to work - just that it might. 1310 Other system specific limitations, such as limitations on 1311 available space for storing files, may cause an operation to 1312 fail, where the permission flags may have indicated that it was 1313 likely to succeed. The permissions are a guide only. 1315 Implementation note: The permissions are described here as they apply 1316 to FTP commands. They may not map easily into particular 1317 permissions available on the server's operating system. Servers 1318 are expected to synthesize these permission bits from the 1319 permission information available from operating system. For 1320 example, to correctly determine whether the "D" permission bit 1321 should be set on a directory for a server running on the 1322 UNIX(TM) operating system, the server should check that the 1323 directory named is empty, and that the user has write permission 1324 on both the directory under consideration, and its parent 1325 directory. 1327 Some systems may have more specific permissions than those 1328 listed here, such systems should map those to the flags defined 1329 as best they are able. Other systems may have only more broad 1330 access controls. They will generally have just a few possible 1331 permutations of permission flags, however they should attempt to 1332 correctly represent what is permitted. 1334 7.5.6. The lang Fact 1336 The lang fact describes the natural language of the filename for use 1337 in display purposes. Values used here should be taken from the 1338 language registry of the IANA. See [12] for the syntax, and 1339 procedures, related to language tags. 1341 lang-fact = "Lang" "=" token 1343 Server-FTP implementations MUST NOT guess language values. Language 1344 values must be determined in an unambiguous way such as filesystem 1345 tagging of language or by user configuration. Note that the lang 1346 fact provides no information at all about the content of a file, only 1347 about the encoding of its name. 1349 7.5.7. The size Fact 1351 The size should always reflect the approximate size of the file. 1352 This should be as accurate as the server can make it, without going 1353 to extraordinary lengths, such as reading the entire file. The size 1354 is expressed in units of octets. 1356 Given limitations in some systems, Client-FTP implementations must 1357 understand this size may not be precise and may change between the 1358 time of a MLST and RETR operation. 1360 Clients that need highly accurate size information for some 1361 particular reason should use the SIZE command as defined in section 1362 4. The most common need for this accuracy is likely to be in 1363 conjunction with the REST command described in section 5. The size 1364 fact, on the other hand, should be used for purposes such as 1365 indicating to a human user the approximate size of the file to be 1366 transferred, and perhaps to give an idea of expected transfer 1367 completion time. 1369 size-fact = "Size" "=" 1*DIGIT 1371 7.5.8. The media-type Fact 1373 The media-type fact represents the IANA media type of the file. The 1374 list of values used must follow the guidelines set by the IANA 1375 registry. 1377 media-type = "Media-Type" "=" 1379 Server-FTP implementations MUST NOT guess media type values. Media 1380 type values must be determined in an unambiguous way such as 1381 filesystem tagging of media-type or by user configuration. 1383 7.5.9. The charset Fact 1385 The charset fact represents the IANA character set name for the 1386 encoded names in a MLST response. The default character set is UTF-8 1387 unless specified otherwise. FTP implementations SHOULD use UTF-8 if 1388 possible to encourage maximum interoperability. 1390 charset-type = "Charset" "=" token 1392 7.6. FEAT response for MLST 1394 When responding to the FEAT command, a server-FTP process that 1395 supports MLST, and the related commands, MLSD, and the modified STAT, 1396 plus internationalization of pathnames, MUST indicate that this 1397 support exists. It does this by including a MLST feature line. As 1398 well as indicating the basic support, the MLST feature line indicates 1399 which MLST facts are available from the server, and which of those 1400 will be returned if no subsequent "OPTS MLST" command is sent. 1402 mlst-feat = SP "MLST" [SP factlist] CRLF 1403 factlist = factname ["*"] *( ";" factname ["*"] ) 1405 The initial space shown in the mlst-feat response is that required by 1406 the FEAT command, two spaces are not permitted. If no factlist is 1407 given, then the server-FTP process is indicating that it supports 1408 MLST, but implements no facts. Only pathnames can be returned. This 1409 would be a minimal MLST implementation, and useless for most 1410 practical purposes. Where the factlist is present, the factnames 1411 included indicate the facts supported by the server. Where the 1412 optional asterisk appears after a factname, that fact will be 1413 included in MLST format responses, until an "OPTS MLST" is given to 1414 alter the list of facts returned. 1416 [ Ed-Note: Perhaps the sense of the "*" should be 1417 reversed? That is, make the asterisk indicate those 1418 facts not returned? ] 1420 Note that there is no distinct FEAT output for MLSD. The presence of 1421 the MLST feature indicates that both MLST and MLSD are both 1422 supported. 1424 7.7. OPTS parameters for MLST 1426 For the MLST command, the Client-FTP may specify a list of facts it 1427 wishes to be returned in all subsequent MLST commands until another 1428 OPTS MLST command is sent. The format is specified by: 1430 mlst-opts = "OPTS" SP "MLST" 1431 [ SP factname *(";" factname) ] 1433 By sending the "OPTS MLST" command, the client requests the server to 1434 include only the facts listed as arguments to the command in 1435 subsequent output from MLST commands. Facts not included in the 1436 "OPTS MLST" command must not be returned by the server. Facts that 1437 are included should be returned for each entry returned from the MLST 1438 command where they apply. Facts requested that are not supported, or 1439 which are inappropriate to the file or directory being listed should 1440 simply be omitted from the MLST output. This is not an error. Note 1441 that where no factname arguments are present, the client is 1442 requesting that only the file names be returned. In this case, and 1443 in any other case where no facts are included in the result, the 1444 space that separates the fact names and their values from the file 1445 name is still required. That is, the first character of the output 1446 line will be a space, and the file name will start immediately 1447 thereafter. 1449 Note, there is no "OPTS MLSD" command, the fact names set with the 1450 "OPTS MLST" command apply to both MLST and MLSD commands, and to the 1451 STAT command when used with a file name argument and no transfer in 1452 progress. 1454 8. Interpretation of STAT command output 1456 Where a server-FTP process supports the MLST and MLSD commands, it 1457 MUST also support the format of the STAT command that allows a 1458 pathname to be given ([6] section 4.1.3). Further, the response to 1459 that STAT command MUST be in MLST format, just as if an MLST command 1460 for the same argument had been given, but slightly modified for 1461 transport over the control connection rather than over a data 1462 connection. 1464 To construct the response to this form of the STAT command, the 1465 server-PI should first construct the MLST output for the file named 1466 by the argument. That should then be broken that output into 1467 segments no longer than 79 octets each. Each segment should have a 1468 space prepended, and CRLF appended. Then send a multi-line reply, 1469 where the first line is "211-", the subsequent lines 1470 are those created above, with NUL after CR insertion (other than the 1471 CR in the end of line CRLF) and IAC escaping, as required. Finally a 1472 terminating line "211 " is sent. 1474 The leading space on each line guarantees that none of the MLST 1475 output can be mis-interpreted as the terminating line. Server-PIs 1476 are free to be creative in splitting the MLST output in creative ways 1477 should they desire, however this should be relevant only to human 1478 end-users who happen to see the raw form of the output. User-PIs 1479 receiving this form of STAT output should simply reconstruct the MLST 1480 format response by ignoring the leading and terminating lines, after 1481 checking that no error occurred of course, deleting the leading space 1482 from each interior line, deleting the terminating CRLF, and 1483 performing escape character reduction (remove the NUL after each CR, 1484 and delete any IAC escapes) then join the remaining lines in order, 1485 to produce the original MLST response. 1487 [ Ed-Note: this is a very cumbersome description of a 1488 very simple procedure... ] 1490 8.1. FEAT response for STAT 1492 There is no output in the FEAT command that specifically indicates 1493 that the STAT command behaves as described above. Implementations 1494 must infer this from support of the MLST command by the server, which 1495 is indicated in the FEAT output. 1497 9. Impact On Other FTP Commands 1499 Along with the introduction of MLST, traditional FTP commands must be 1500 extended to allow for the use of more than US-ASCII or EBCDIC 1501 character sets. In general, the support of MLST requires support for 1502 arbitrary character sets wherever filenames and directory names are 1503 allowed. This applies equally to both arguments given to the 1504 following commands and to the replies from them, as appropriate. 1506 CWD 1507 RETR 1508 STOR 1509 STOU 1510 APPE 1511 RNFR 1512 RNTO 1513 DELE 1514 RMD 1515 MKD 1516 PWD 1517 STAT 1519 The arguments to all of these commands should be processed the same 1520 way that MLST commands and responses are processed with respect to 1521 handling embedded CRs and NULs. See section 2.2. 1523 9.1. Impact on Pathnames and Filenames 1525 The design of MLST requires the Server-FTP to allow concatenation of 1526 certain elements of a MLST response. Specifically, a typical 1527 response would include an element which indicates the current 1528 directory and one or more elements which are files in the indicated 1529 directory. A Server-FTP must be able to accept a simple 1530 concatenation of these two names even if the underlying operating 1531 system does not accept a simple concatenation. The Server-FTP must 1532 perform any translation of the concatenated name to local 1533 equivalents. 1535 10. Character sets and Internationalization 1537 FTP commands and responses are protocol elements, and are always 1538 expressed in ASCII. It is not expected that users normally interact 1539 directly with those elements, rather the user FTP-process constructs 1540 the commands, and interprets the results, in the manner best suited 1541 for the particular user. Explanatory text in responses generally has 1542 no particular meaning to the protocol. The numeric codes provide all 1543 necessary information. Server-PIs are free to provide the text in 1544 any language that can be adequately represented in ASCII. 1546 Pathnames are expected to be encoded in UTF-8 allowing essentially 1547 any character to be represented in a pathname. Meaningful pathnames 1548 are defined by the server NFVS. 1550 No restrictions at all are placed upon the contents of files 1551 transferred using the FTP protocols. Nor is any advice given here 1552 which would allow determining the content type. That information is 1553 assumed to be obtained via other means. 1555 11. IANA Considerations 1557 This specification makes use of some lists of values currently 1558 maintained by the IANA, but does not create any new lists for the 1559 IANA to maintain. It also does not add any values to any existing 1560 lists. 1562 12. Security Non-Considerations 1564 This memo does not yet discuss security. It is possible that no new 1565 security concerns are raised in this memo above what already exists 1566 within the FTP protocol. However, the working group needs to 1567 consider this carefully. 1569 A general discussion of issues related to the security of FTP can be 1570 found in [13]. 1572 13. Open Issues 1574 13.1. General 1576 + Should date/time format be more general? 1578 13.2. With MDTM 1580 + None known (other than precise syntax of result, as above) 1582 13.3. With SIZE 1584 + None known 1586 13.4. With REST 1588 + None known 1590 13.5. With MLST/MLSD 1592 + Mandatory to implement fact sets? 1593 + Recursive listings? How to request? How to parse result? 1594 + Use of data connection for MLST. 1595 + Wildcards? Almost certainly not define, but permit at all? 1596 Meaning if permitted. What to say about them? 1597 + What filename should appear in MLST/MLSD results, for MLST must 1598 it be the same as the parameter? For MLSD, what is expected of 1599 cdir type names? 1601 + cdir/filename concatenation - Does this have any real use? Can 1602 it be properly defined? If so, how? 1603 + Multi-type objects? Define new type labels, or represent in 1604 listing multiple times? 1605 + "type" fact - should it have defined values, and "other", with 1606 "other" types distinguished in OS dependent facts, or defined 1607 values and undefined values, with undefined values recognized by 1608 those for which they are known, and ignored by others. 1609 + Labels for permission indicators. 1610 + FEAT result for MLST - which sense is the * to have? 1611 + OPTS for MLST - add a way to add/subtract facts without repeating 1612 entire desired fact set? 1614 13.6. With STAT modifications 1616 + Is this necessary, or could MLST use control connection and STAT 1617 be left alone? 1618 + Line wrapping? Needed or not? (This applies generally to 1619 whatever uses the control connection.) 1621 14. Very sketchy proposal 1623 One idea for some of the above, may be to permit multiple response to 1624 MLST and MLSD commands. 1626 For MLST, this would occur if the server implemented some kind of 1627 wildcarding, and the user managed to input the right kind of pattern, 1628 in which case the filenames returned would be different on the 1629 different lines. Or, it could also occur if the WG decides that 1630 multiple listings for one filename are the right way to handle 1631 schizophrenic files. In that case, all lines would contain the same 1632 filename. 1634 For MLSD, this could indicate that wildcarding happened, and that 1635 multiple directories are being listed. Or, it could indicate the 1636 result of a recursive listing, with sub-directories being listed. 1637 How to request that recursive listings be returned, or not returned, 1638 and how to indicate that even though requested, and sometimes 1639 available, and that sub-directories actually exist, the server 1640 refuses to recursively list the current tree, are all open questions 1641 (if this goes anywhere at all). 1643 14.1. Syntax 1645 I would suggest for MLSD that each single directory listing be 1646 returned as a block, with a blank line (extra CRLF) between the 1647 blocks. For MLST, no output change is required, though a CRLF could 1648 be inserted between outputs from wildcards. 1650 15. References 1652 [1] Coded Character Set--7-bit American Standard Code for Information 1653 Interchange, ANSI X3.4-1986. 1655 [2] Yergeau, F., "UTF-8, a transformation format of Unicode and ISO 1656 10646", RFC 2044, October 1996. 1658 [3] Crocker, D., Overell, P., "Augmented BNF for Syntax 1659 Specifications: ABNF", RFC 2234, November 1997 1661 [4] Curtin, W., "Internationalization of the File Transfer Protocol", 1662 Work In Progress , June 1997 1664 [5] Internet Assigned Numbers Authority. http://www.isi.edu/div7/iana/ 1665 Email: iana@iana.org. 1667 [6] Postel, J., Reynolds, J., "File Transfer Protocol (FTP)", 1668 STD 9, RFC 959, October 1985 1670 [7] Braden, R,. "Requirements for Internet Hosts -- Application 1671 and Support", STD 3, RFC 1123, October 1989 1673 [8] ISO 3307 (need a citation for this please!) 1675 [9] Bradner, S., "Key words for use in RFCs to Indicate 1676 Requirement Levels", BCP 14, RFC 2119, March 1997 1678 [10] Hethmon, P., Elz, R., "Feature negotiation mechanism for the 1679 File Transfer Protocol", Work in progress, 1680 November 1997. 1682 [11] Postel, J., Reynolds, J., "Telnet protocol Specification" 1683 STD 8, RFC 854, May 1983 1685 [12] Alvestrand, H., "Tags for the Identification of Languages" 1686 RFC 1766, March 1995 1688 [13] Allman, M., "FTP Security Considerations" 1689 Work in progress, , January 1998 1691 Acknowledgements 1693 This document is a product of the FTPEXT working group of the IETF. 1695 The following people are among those who have contributed to this 1696 document: 1698 Alex Belits 1699 D. J. Berstein 1700 Dave Cridland 1701 Martin J. Duerst 1702 Mark Harris 1703 Alun Jones 1704 James Matthews 1705 Keith Moore 1706 Buz Owen 1707 Stephen Tihor 1708 and the entire FTPEXT working group of the IETF. 1710 Apologies are offered to any inadvertently omitted. 1712 The description of the modifications to the REST command and the MDTM 1713 and SIZE commands comes from a set of modifications suggested for 1714 RFC959 by Rick Adams in 1989. A draft containing just those 1715 commands, edited by David Borman, has been merged with this document. 1717 Copyright 1719 This document is in the public domain. Any and all copyright 1720 protection that might apply in any jurisdiction is expressly 1721 disclaimed. 1723 Editors' Addresses 1725 Robert Elz 1726 University of Melbourne 1727 Department of Computer Science 1728 Parkville, Vic 3052 1729 Australia 1731 Email: kre@munnari.OZ.AU 1732 Paul Hethmon 1733 Hethmon Brothers 1734 2305 Chukar Road 1735 Knoxville, TN 37923 USA 1737 Phone: +1 423 690 8990 1738 Email: phethmon@hethmon.com