idnits 2.17.1 draft-ietf-ips-command-ordering-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 17 instances of too long lines in the document, the longest one being 3 characters in excess of 72. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 315: '... reflected in [RFC3347], "iSCSI MUST specify strictly ordered...' Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == Line 660 has weird spacing: '... be rev...' == Unrecognized Status in '', assuming Proposed Standard (Expected one of 'Standards Track', 'Full Standard', 'Draft Standard', 'Proposed Standard', 'Best Current Practice', 'Informational', 'Experimental', 'Informational', 'Historic'.) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC3347' is mentioned on line 612, but not defined == Missing Reference: 'SPC3' is mentioned on line 553, but not defined == Missing Reference: 'RFC793' is mentioned on line 608, but not defined ** Obsolete undefined reference: RFC 793 (Obsoleted by RFC 9293) == Missing Reference: 'RFC2119' is mentioned on line 610, but not defined == Unused Reference: 'SAM' is defined on line 602, but no explicit reference was found in the text == Unused Reference: 'SBC' is defined on line 604, but no explicit reference was found in the text -- Possible downref: Non-RFC (?) normative reference: ref. 'SAM' -- Possible downref: Non-RFC (?) normative reference: ref. 'SAM2' -- Possible downref: Non-RFC (?) normative reference: ref. 'SBC' Summary: 7 errors (**), 0 flaws (~~), 9 warnings (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 IPS Mallikarjun Chadalapaka 3 Internet Draft Rob Elliott 4 draft-ietf-ips-command-ordering-02.txt Hewlett-Packard Co. 5 Category: Informational-track 7 SCSI Command Ordering Considerations with iSCSI 9 Status of this Memo 11 This document is an Internet-Draft and fully conforms to all 12 provisions of Section 10 of RFC2026. 14 Internet-Drafts are working documents of the Internet 15 Engineering Task Force (IETF), its areas, and its working 16 groups. Note that other groups may also distribute working 17 documents as Internet-Drafts. Internet-Drafts are draft 18 documents valid for at most six months and may be updated, 19 replaced, or made obsolete by other documents at any time. It is 20 inappropriate to use Internet- Drafts as reference material or 21 to cite them except as "work in progress." 22 The list of Internet-Drafts can be accessed at http:// 23 www.ietf.org/ietf/1id-abstracts.txt. 24 The list of Internet-Draft Shadow Directories can be accessed at 25 http://www.ietf.org/shadow.html. 27 Abstract 29 iSCSI is a SCSI transport protocol designed to run on top of 30 TCP. The iSCSI session abstraction is equivalent to the classic 31 SCSI "I_T nexus" which represents the logical relationship 32 between an Initiator and a Target (I and T) required in order to 33 communicate via the SCSI family of protocols. The iSCSI session 34 provides an ordered command delivery from the SCSI initiator to 35 the SCSI target. This document goes into the design 37 Mallikarjun Chadalapaka Expires July 2004 1 38 Command Ordering 20-January-04 40 considerations that led to the iSCSI session model as it is 41 defined today, relates the SCSI command ordering features 42 defined in T10 specifications to the iSCSI concepts, and finally 43 provides guidance to system designers on how true command 44 ordering solutions can be built based on iSCSI. 46 Acknowledgments 48 We are grateful to the IPS working group whose work defined the 49 iSCSI protocol. Thanks also to David Black (EMC) who encouraged 50 the publication of this document. Special thanks to Randy 51 Haagens (HP) for his insights on the topic of command ordering. 52 Thanks are also due to Elizabeth Rodriguez for carefully 53 reviewing this document. 55 Mallikarjun Chadalapaka Expires July 2004 2 56 Command Ordering 20-January-04 58 Status of this Memo . . . . . . . . . . . . . . . . . . . . . . . . . 1 59 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 60 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 61 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 62 2. Definitions and Acronyms . . . . . . . . . . . . . . . . . . . . . 5 63 2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . 5 64 2.2 Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 65 3. Overview of the iSCSI Protocol . . . . . . . . . . . . . . . . . . 7 66 3.1 Protocol Mapping Description . . . . . . . . . . . . . . . . . . 7 67 3.2 The I_T Nexus Model . . . . . . . . . . . . . . . . . . . . . . 7 68 3.3 Ordered Command Delivery . . . . . . . . . . . . . . . . . . . . 9 69 3.3.1 Questions . . . . . . . . . . . . . . . . . . . . . . . . . 9 70 3.3.2 The Session Guarantee . . . . . . . . . . . . . . . . . . . 9 71 3.3.3 Ordering Onus . . . . . . . . . . . . . . . . . . . . . . .10 72 3.3.4 Design Intent . . . . . . . . . . . . . . . . . . . . . . .10 73 4. The Command Ordering Scenario . . . . . . . . . . . . . . . . . . .11 74 4.1 SCSI Layer . . . . . . . . . . . . . . . . . . . . . . . . . . .11 75 4.1.1 Command Reference Number (CRN) . . . . . . . . . . . . . .11 76 4.1.2 Task Attributes . . . . . . . . . . . . . . . . . . . . . .11 77 4.1.3 Auto Contingent Allegiance (ACA) . . . . . . . . . . . . .11 78 4.1.4 UA Interlock . . . . . . . . . . . . . . . . . . . . . . .12 79 4.2 iSCSI Layer . . . . . . . . . . . . . . . . . . . . . . . . . .12 80 5. Connection Failure Considerations . . . . . . . . . . . . . . . . .13 81 6. Command Ordering System Considerations . . . . . . . . . . . . . .14 82 7. Reservation Considerations . . . . . . . . . . . . . . . . . . . .15 83 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . . .16 84 9. Security Considerations . . . . . . . . . . . . . . . . . . . . . .17 85 10. References and Bibliography . . . . . . . . . . . . . . . . . . .18 86 10.1 Normative References . . . . . . . . . . . . . . . . . . . . .18 87 10.2 Informative References: . . . . . . . . . . . . . . . . . . . .18 88 11. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . .19 89 Full Copyright Statement . . . . . . . . . . . . . . . . . . . . . .20 91 Mallikarjun Chadalapaka Expires July 2004 3 92 Command Ordering 20-January-04 94 1. Introduction 96 iSCSI is a SCSI transport protocol ([iSCSI]) designed to enable 97 running SCSI application protocols on TCP/IP networks, including 98 potentially the Internet. Given the size and scope of Internet, 99 iSCSI thus enables some exciting new SCSI applications. 100 Potential new application areas for exploiting iSCSI's value 101 include the following. 103 a) Larger (diameter) Storage Area Networks (SANs) than 104 had been possible until now. 105 b) Asynchronous remote mirroring 106 c) Remote tape vaulting 108 Each of these applications takes advantage of the practically 109 unlimited geographical distance that iSCSI enables between a 110 SCSI initiator and a SCSI target. In each of these cases, 111 because of the long delays involved, there is a very high 112 incentive for the initiator to stream SCSI commands back-to-back 113 without waiting for the SCSI status of previous commands. 114 Command streaming may be employed primarily by two classes of 115 applications - while one class may not particularly care about 116 ordered command execution, the other class does rely on ordered 117 command execution (i.e. there is an application-level dependency 118 on the ordering among SCSI commands). As an example, cases b) 119 and c) listed earlier clearly require ordered command execution. 120 A mirroring application does not want the writes to be committed 121 out of order on the remote SCSI target, so as to preserve the 122 transactional integrity of the data on that target. To 123 summarize, SCSI command streaming when coupled with the 124 guarantee of ordered command execution on the SCSI target is 125 extremely valuable for a critical class of applications in long- 126 latency networks. 128 This document reviews the various protocol considerations in 129 designing storage solutions that employ SCSI command ordering. 130 This document also analyzes and explains the design intent of 131 [iSCSI] with respect to command ordering. 133 Mallikarjun Chadalapaka Expires July 2004 4 134 Command Ordering 20-January-04 136 2. Definitions and Acronyms 138 2.1 Definitions 140 - I_T nexus: [SAM2] defines the I_T nexus as a relationship 141 between a SCSI initiator port and a SCSI target port. [iSCSI] 142 defines an iSCSI session as the iSCSI representation of an I_T 143 nexus. In the iSCSI context, the I_T nexus (i.e. the iSCSI 144 session) is a relationship between an iSCSI initiator's end of 145 the session (SCSI Initiator Port) and the iSCSI target's Portal 146 Group (SCSI Target Port). 148 - PDU (Protocol Data Unit): An iSCSI initiator and iSCSI target 149 communicate using iSCSI protocol messages. These messages are 150 called "iSCSI protocol data units" (iSCSI PDUs). 152 - SCSI device: A SCSI device is an entity that contains one or 153 more SCSI ports that are connected to a service delivery 154 subsystem and supports SCSI application protocols. In the iSCSI 155 context, the SCSI Device is the component within an iSCSI Node 156 that provides the SCSI functionality. The SCSI Device Name is 157 defined to be the iSCSI Name of the node. 159 - Session: A group of logically related iSCSI connections that 160 link an initiator with a target form a session (equivalent to a 161 SCSI I-T nexus). The number of participating iSCSI connections 162 within an iSCSI session may vary over time. The multiplicity of 163 connections at the iSCSI level is completely hidden for the SCSI 164 layer - each SCSI port in an I_T nexus sees only one peer SCSI 165 port across all the connections of a session. 167 Mallikarjun Chadalapaka Expires July 2004 5 168 Command Ordering 20-January-04 170 2.2 Acronyms 172 Acronym Definition 173 -------------------------------------------------------------- 174 ACA Auto Contingent Allegiance 175 ASC Additional Sense Code 176 ASCQ Additional Sense Code Qualifier 177 CRN Command Reference Number 178 IETF Internet Engineering Task Force 179 ISID Initiator Session Identifier 180 ITT Initiator Task Tag 181 LU Logical Unit 182 LUN Logical Unit Number 183 NIC Network Interface Card 184 PDU Protocol Data Unit 185 TMF Task Management Function 186 TSIH Target Session Identifying Handle 187 SAM-2 SCSI Architecture Model - 2 188 SAN Storage Area Network 189 SCSI Small Computer Systems Interface 190 TCP Transmission Control Protocol 191 UA Unit Attention 192 WG Working Group 194 Mallikarjun Chadalapaka Expires July 2004 6 195 Command Ordering 20-January-04 197 3. Overview of the iSCSI Protocol 199 3.1 Protocol Mapping Description 201 The iSCSI protocol is a mapping of the SCSI remote procedure 202 invocation model (see [SAM2]) over the TCP protocol. 204 SCSI's notion of a task maps to an iSCSI task. Each iSCSI task 205 is uniquely identified within that I_T nexus by a 32-bit unique 206 identifier called Initiator Task Tag (ITT). The ITT is both an 207 iSCSI identifier of the task and a classic SCSI task tag. 209 SCSI commands from the initiator to the target are carried in 210 iSCSI requests called SCSI Command PDUs. SCSI status back to 211 the initiator is carried in iSCSI responses called SCSI Response 212 PDUs. SCSI Data-out from the initiator to the target is carried 213 in SCSI Data-Out PDUs, and the SCSI Data-in back to the 214 initiator is carried in SCSI Data-in PDUs. 216 3.2 The I_T Nexus Model 218 In the iSCSI model, the SCSI I_T nexus maps directly to the 219 iSCSI session which is an iSCSI protocol abstraction spanning 220 one or more TCP connections. The iSCSI protocol defines the 221 semantics in order to realize one logical flow of bidirectional 222 communication on the I_T nexus potentially spanning multiple TCP 223 connections (as many as 2^16). The multiplicity of iSCSI 224 connections is thus completely contained at the iSCSI layer, 225 while the SCSI layer is presented with a single I_T nexus even 226 in a multi-connection session. A session between a pair of 227 given iSCSI nodes is identified by the session identifier (SSID) 228 and each connection within a given session is uniquely 229 identified by a connection identifier (CID) in iSCSI. The SSID 230 itself has two components - Initiator Session Identifier (ISID) 231 and a Target Session Identifying Handler (TSIH) - each 232 identifying one end of the same session. 234 There are four crucial functional facets of iSCSI that together 235 present this single logical flow abstraction to the SCSI layer 236 even with an iSCSI session spanning across multiple iSCSI 237 connections. 239 a) Ordered command delivery: A sequence of SCSI commands 240 that is striped across all the connections in the 242 Mallikarjun Chadalapaka Expires July 2004 7 243 Command Ordering 20-January-04 245 session is "reordered" by the target iSCSI layer into 246 an identical sequence based on a Command Sequence 247 Number (CmdSN) that is unique across the session. The 248 goal is to achieve bandwidth aggregation from multiple 249 TCP connections, but to still make it appear to the 250 target SCSI layer as if all the commands had travelled 251 in one flow. 252 b) Connection allegiance: All the PDU exchanges for a 253 SCSI Command, up to and including the SCSI Response PDU 254 for the Command, are required to flow on the same iSCSI 255 connection at any given time. This again is intended 256 to hide the multi-connection nature of a session 257 because the SCSI layer on either side will never see 258 the PDU contents out of order (e.g., status cannot 259 bypass read data for an initiator). 260 c) Task set management function handling: [iSCSI] 261 specifies an ordered sequence of steps for the iSCSI 262 layer on the SCSI target in handling the two SCSI task 263 management functions (TMFs) that manage SCSI task sets. 264 The two TMFs are ABORT TASK SET that aborts all active 265 tasks in a session and CLEAR TASK SET that clears the 266 tasks in the task set. The goal of the sequence of 267 steps is to guarantee that the initiator receives the 268 SCSI Response PDUs of all unaffected tasks before the 269 TMF Response itself arrives, regardless of the number 270 of connections in the iSCSI session. This operational 271 model is again intended to preserve the single flow 272 abstraction to the SCSI layer. 273 d) Immediate task management function handling: Even when 274 a TMF request is marked as "immediate" (i.e. only has a 275 position in the command stream, but does not consume a 276 CmdSN), [iSCSI] defines semantics that require the 277 target iSCSI layer to ensure that the TMF request is 278 executed as if the commands and the TMF request were 279 all flowing on a single logical channel. This ensures 280 that the TMF request will act on tasks that it was 281 meant to manage. 283 The following sections will analyze the "Ordered command 284 delivery" aspect in more detail, since command ordering is the 285 focus of this document. 287 Mallikarjun Chadalapaka Expires July 2004 8 288 Command Ordering 20-January-04 290 3.3 Ordered Command Delivery 292 3.3.1 Questions 294 A couple of important questions related to iSCSI command 295 ordering were considered early on in the design of the iSCSI 296 protocol. The questions were: 298 a) What should be the command ordering behavior required 299 of iSCSI implementations in the presence of transport 300 errors, such as errors that corrupt the data in a 301 fashion that is not detected by the TCP checksum (e.g., 302 two offsetting bit flips in the same bit position), but 303 is detected by the iSCSI CRC digest? 304 b) Should [iSCSI] require both initiators and targets to 305 use ordered command delivery? 307 Since the answers to these questions are critical to the 308 understanding of the ordering behavior required by the iSCSI 309 protocol, the following sub-sections consider them in more 310 detail. 312 3.3.2 The Session Guarantee 314 The final disposition of question a) in section 3.3.1 was 315 reflected in [RFC3347], "iSCSI MUST specify strictly ordered 316 delivery of SCSI commands over an iSCSI session between an 317 initiator/target pair, even in the presence of transport 318 errors.". Stated differently, an iSCSI digest failure, or an 319 iSCSI connection termination must not cause the iSCSI layer on a 320 target to allow executing the commands in an order different 321 from that intended (as indicated by the CmdSN order) by the 322 initiator. This design choice is enormously helpful in building 323 storage systems and solutions that can now always assume command 324 ordering to be a service characteristic of an iSCSI substrate. 326 Note that by taking the position that an iSCSI session always 327 guarantees command ordering, [iSCSI] was indirectly implying 328 that the principal reason for the multi-connection iSCSI session 329 abstraction was to allow ordered bandwidth aggregation for an 330 I_T nexus. In deployment models where this cross-connection 331 ordering mandated by [iSCSI] is deemed expensive, a serious 332 consideration should be given to deploying multiple single- 333 connection sessions in stead. 335 Mallikarjun Chadalapaka Expires July 2004 9 336 Command Ordering 20-January-04 338 3.3.3 Ordering Onus 340 The final resolution of b) in section 3.3.1 by the iSCSI 341 protocol designers was in favor of not requiring the initiators 342 to use command ordering always. This resolution is reflected in 343 dropping the mandatory ACA usage requirement on the initiators, 344 and allowing an ABORT TASK TMF to plug a command hole etc., 345 since these are conscious choices an initiator makes in favor of 346 not using ordered command delivery. The net result can be 347 discerned by a careful reader of [iSCSI] - the onus of ensuring 348 ordered command delivery is always on the iSCSI targets, while 349 the initiators may or may not utilize command ordering. iSCSI 350 targets being the servers in the client-server model do not 351 really attempt to establish whether or not a client (initiator) 352 intends to take advantage of command ordering service, but 353 instead simply always provide the guaranteed delivery service. 354 The rationale here is that there are inherent SCSI and 355 application-level dependencies as we shall see in building a 356 command ordered solution that are beyond the scope of [iSCSI], 357 to mandate or even discern the intent with respect to the usage 358 of command ordering. 360 3.3.4 Design Intent 362 To summarize the design intent of [iSCSI] - 364 The service delivery subsystem (see [SAM2]) abstraction 365 provided by an iSCSI session is guaranteed to have the intrinsic 366 property of ordered delivery of commands to the target SCSI 367 layer under all conditions. Consequently, the guarantee of the 368 ordered command delivery is across the entire I_T nexus spanning 369 all the LUs that the nexus is authorized to access. It is the 370 initiator's discretion to whether or not make use of this 371 property. 373 Mallikarjun Chadalapaka Expires July 2004 10 374 Command Ordering 20-January-04 376 4. The Command Ordering Scenario 378 A storage systems designer working with SCSI and iSCSI has to 379 consider the following protocol features in SCSI and iSCSI 380 layers, each of which has a role to play in realizing the 381 command ordering goal. 383 4.1 SCSI Layer 385 The SCSI application layer has several tools to enforce 386 ordering. 388 4.1.1 Command Reference Number (CRN) 390 CRN is an ordered sequence number which when enabled for a 391 device server, increments by one for each I_T_L nexus (see 392 [SAM2]). The one notable drawback with CRN is that there is no 393 SCSI-generic way (such as through mode pages) to enable or 394 disable the CRN feature. [SAM2] also leaves the usage semantics 395 of CRN for the SCSI transport protocol, such as iSCSI, to 396 specify. [iSCSI] chose not to support the CRN feature for 397 various reasons. 399 4.1.2 Task Attributes 401 SAM-2 defines the following four task attributes - SIMPLE, 402 ORDERED, HEAD OF QUEUE, and ACA. Each task to an LU may be 403 assigned an attribute. [SAM2] defines the ordering constraints 404 that each of these attributes conveys to the device server that 405 is servicing the task. In particular, judicious use of ORDERED 406 and SIMPLE attributes applied to a stream of pipelined commands 407 could convey the precise execution schema for the commands that 408 the initiator issues, provided the commands are received in the 409 same order on the target. 411 4.1.3 Auto Contingent Allegiance (ACA) 413 ACA is an LU-level condition that is triggered when a command 414 (with the NACA bit set to 1) completes with CHECK CONDITION. 415 When ACA is triggered, it prevents all commands other than those 416 with the ACA attribute from executing until the CLEAR ACA task 417 management function is executed, while blocking all the other 418 tasks already in the task set. See [SAM2] for the detailed 419 semantics of ACA. Since ACA is closely tied to the notion of a 421 Mallikarjun Chadalapaka Expires July 2004 11 422 Command Ordering 20-January-04 424 task set, one would ideally have to select the scope of the task 425 set (by setting the TST bit to 1 in the control mode page of the 426 LU) to be per-initiator in order to prevent command failures in 427 one I_T_L nexus from impacting other I_T_L nexuses through ACA. 429 4.1.4 UA Interlock 431 When UA interlock is enabled, the logical unit does not clear 432 any standard Unit Attention condition reported with autosense 433 and in addition, establishes a Unit Attention condition when a 434 task is terminated with one of BUSY, TASK SET FULL, or 435 RESERVATION CONFLICT statuses. This so-called "interlocked UA" 436 is cleared only when the device server executes an explicit 437 REQUEST SENSE ([SPC3]) command from the same initiator. From a 438 functionality perspective, the scope of UA interlock today is 439 slightly different from ACA's because it enforces ordering 440 behavior for completion statuses other than CHECK CONDITION, but 441 otherwise conceptually has the same design intent as ACA. On 442 the other hand, ACA is somewhat more sophisticated because it 443 allows special "cleanup" tasks (ones with ACA attribute) to 444 execute when ACA is active. One of the principal reasons UA 445 interlock came into being was that SCSI designers wanted a 446 command ordering feature without the side effects of using the 447 aforementioned TST bit in the control mode page. 449 4.2 iSCSI Layer 451 As noted in section 3.2 and section 3.3, the iSCSI protocol 452 enforces and guarantees ordered command delivery per iSCSI 453 session using the CmdSN, and this is an attribute of the SCSI 454 transport layer. Note further that any command ordering 455 solution that seeks to realize ordering from the initiator SCSI 456 layer to the target SCSI layer would be of practical value only 457 when the command ordering is guaranteed by the SCSI transport 458 layer. In other words, the related SCSI application layer 459 protocol features such as ACA etc. are based on the premise of 460 an ordered SCSI transport. Thus iSCSI's command ordering is the 461 last piece in completing the puzzle of building solutions that 462 rely on ordered command execution, by providing the crucial 463 guarantee that all the commands handed to the initiator iSCSI 464 layer will be transported and handed to the target SCSI layer in 465 the same order. 467 Mallikarjun Chadalapaka Expires July 2004 12 468 Command Ordering 20-January-04 470 5. Connection Failure Considerations 472 [iSCSI] mandates that when an iSCSI connection fails, the active 473 tasks on that connection must be terminated if not recovered 474 within a certain negotiated time limit. When an iSCSI target 475 does terminate some subset of tasks due to iSCSI connection 476 dynamics, there is a danger that the SCSI layer would simply 477 move on to the next tasks waiting to be processed and execute 478 them out-of-order unbeknownst to the initiator SCSI layer. To 479 preclude this danger, [iSCSI] further mandates the following - 481 a) The tasks terminated due to the connection failure must 482 be internally terminated by the iSCSI target "as if" due to a 483 CHECK CONDITION. While this particular completion status is 484 never communicated back to the initiator, the "as if" is 485 still meaningful and required because if the initiator were 486 using ACA as the command ordering mechanism of choice, a 487 SCSI-level ACA will be triggered due to this mandatory CHECK 488 CONDITION. This addresses the aforementioned danger. 489 b) After the tasks are terminated due to the connection 490 failure, the iSCSI target must report a Unit Attention 491 condition on the next command processed on any connection for 492 each affected I_T_L nexus of that session. This is required 493 because if the initiator were using UA interlock as the 494 command ordering mechanism of choice, a SCSI-level UA will 495 trigger a UA-interlock. This again addresses the 496 aforementioned danger. iSCSI targets must report this UA with 497 the status of CHECK CONDITION, and the ASC/ASCQ value of 47h/ 498 7Fh ("SOME COMMANDS CLEARED BY ISCSI PROTOCOL EVENT"). 500 Mallikarjun Chadalapaka Expires July 2004 13 501 Command Ordering 20-January-04 503 6. Command Ordering System Considerations 505 In general, command ordering is automatically enforced if 506 targets and initiators comply with the iSCSI specification. 507 However, listed below are certain additional related 508 implementation considerations for the iSCSI initiators and 509 targets to take note of. 511 a) Even when all iSCSI and SCSI command ordering 512 considerations earlier noted in this draft were applied, it 513 is beneficial for iSCSI initiators to proactively avoid 514 scenarios that would otherwise lead to out-of-order command 515 execution. This is simply because the SCSI command ordering 516 features such as UA interlock are likely to be costlier in 517 performance when they are allowed to be triggered. [iSCSI] 518 provides enough guidance on how to implement this proactive 519 detection of PDU ordering errors. 520 b) The whole notion of command streaming does of course 521 assume that the target in question supports command queueing. 522 An iSCSI target desirous of supporting command ordering 523 solutions should ensure that the SCSI layer on the target 524 supports command queuing. Especially the remote backup (tape 525 vaulting) applications that iSCSI enables make a compelling 526 case that tape devices should give a very serious 527 consideration to supporting command queuing, at least when 528 used in conjunction with iSCSI. 529 c) An iSCSI target desirous of supporting high-performance 530 command ordering solutions that involve specifying a 531 description of execution schema should ensure that the SCSI 532 layer on the target in fact does support the ORDERED and 533 SIMPLE task attributes. 534 d) There is some consideration of expanding the scope of UA 535 interlock to encompass CHECK CONDITION status and thus make 536 it the only required command ordering functionality of 537 implementations to build command ordering solutions. Until 538 this is resolved in T10, the currently defined semantics of 539 UA interlock and ACA warrant implementing both features by 540 iSCSI targets desirous of supporting command ordering 541 solutions. 543 Mallikarjun Chadalapaka Expires July 2004 14 544 Command Ordering 20-January-04 546 7. Reservation Considerations 548 [iSCSI] describes a "principle of conservative reuse" that 549 encourages iSCSI initiators to reuse the same ISIDs (see section 550 3.2) to various SCSI target ports, in order to present the same 551 SCSI initiator port name to those target ports. This is in fact 552 a very crucial implementation consideration that must be 553 complied with. [SPC3] mandates the SCSI targets to associate 554 persistent reservations and the related registrations with the 555 SCSI initiator port names whenever they are required by the SCSI 556 transport protocol. Since [iSCSI] requires the mandatory SCSI 557 initiator port names based on ISIDs, iSCSI targets are required 558 to work off the SCSI initiator port names and thus indirectly 559 the ISIDs in enforcing the persistent reservations. 561 This fact has the following implications for the 562 implementations. 564 a) If a persistent reservation/registration is intended to 565 be used across multiple SCSI ports of a SCSI device, the 566 initiator iSCSI implementation must use the same ISID across 567 associated iSCSI sessions connecting to different iSCSI 568 target portal groups of the SCSI device. 569 b) If a persistent reservation/registration is intended to 570 be used across the power loss of a SCSI target, the initiator 571 iSCSI implementation must use the same ISIDs as before in re- 572 establishing the associated iSCSI sessions upon subsequent 573 reboot in order to rely on the persist through power loss 574 capability. 576 Mallikarjun Chadalapaka Expires July 2004 15 577 Command Ordering 20-January-04 579 8. IANA Considerations 581 This document does not have any IANA considerations. 583 Mallikarjun Chadalapaka Expires July 2004 16 584 Command Ordering 20-January-04 586 9. Security Considerations 588 For security considerations in using the iSCSI protocol, refer 589 to the Security Considerations section in [iSCSI]. This 590 document does not introduce any additional secuirty 591 considerations other than those already discussed in [iSCSI]. 593 Mallikarjun Chadalapaka Expires July 2004 17 594 Command Ordering 20-January-04 596 10. References and Bibliography 598 10.1 Normative References 600 [iSCSI] J. Satran et. al. draft-ietf-ips-iscsi-20.txt 601 (work in progress) 602 [SAM] ANSI X3.270-1998, SCSI-3 Architecture Model (SAM). 603 [SAM2] T10/1157D, SCSI Architecture Model - 2 (SAM-2). 604 [SBC] NCITS.306-1998, SCSI-3 Block Commands (SBC). 606 10.2 Informative References: 608 [RFC793] TRANSMISSION CONTROL PROTOCOL, DARPA INTERNET 609 PROGRAM PROTOCOL SPECIFICATION, September 1981. 610 [RFC2119] Bradner, S. "Key Words for use in RFCs to Indi- 611 cate Requirement Levels", BCP 14, RFC 2119, March 1997. 612 [RFC3347] M. Krueger et. al., "iSCSI Requirements and 613 Design Considerations" 614 [SPC3]T10/1416-D, SCSI Primary Commands-3. 616 Mallikarjun Chadalapaka Expires July 2004 18 617 Command Ordering 20-January-04 619 11. Authors' Addresses 621 Mallikarjun Chadalapaka 622 Hewlett-Packard Company 623 8000 Foothills Blvd. 624 Roseville, CA 95747-5668, USA 625 Phone: +1.916.785.5621 626 E-mail: cbm@rose.hp.com 628 Rob Elliott 629 Hewlett-Packard Company 630 MC 150801 631 PO Box 692000 632 Houston, TX 77269-2000 USA 633 Phone: +1.281.518.5037 634 E-mail: elliott@hp.com 636 Comments may be sent to Mallikarjun Chadalapaka. 638 Mallikarjun Chadalapaka Expires July 2004 19 639 Command Ordering 20-January-04 641 Full Copyright Statement 643 "Copyright (C) The Internet Society (2004). All Rights Reserved. 644 This document and translations of it may be copied and furnished 645 to others, and derivative works that comment on or otherwise 646 explain it or assist in its implementation may be prepared, 647 copied, published and distributed, in whole or in part, without 648 restriction of any kind, provided that the above copyright 649 notice and this paragraph are included on all such copies and 650 derivative works. However, this document itself may not be 651 modified in any way, such as by removing the copyright notice or 652 references to the Internet Society or other Internet 653 organizations, except as needed for the purpose of developing 654 Internet standards in which case the procedures for copyrights 655 defined in the Internet Standards process must be followed, or 656 as required to translate it into languages other than 657 English. 659 The limited permissions granted above are perpetual and will not 660 be revoked by the Internet Society or its successors or 661 assigns. 663 This document and the information contained herein is provided 664 on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET 665 ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR 666 IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE 667 OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY 668 IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A 669 PARTICULAR PURPOSE." 671 The IETF has been notified of intellectual property rights 672 claimed in regard to some or all of the specification contained 673 in this document. For more information consult the online list 674 of claimed rights. 676 Mallikarjun Chadalapaka Expires July 2004 20