idnits 2.17.1 draft-ietf-storm-iscsi-cons-09.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 1 instance of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. == There are 1 instance of lines with non-RFC3849-compliant IPv6 addresses in the document. If these are example addresses, they should be changed. -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? -- The abstract seems to indicate that this document obsoletes RFC3720, but the header doesn't have an 'Obsoletes:' line to match this. -- The abstract seems to indicate that this document obsoletes RFC3980, but the header doesn't have an 'Obsoletes:' line to match this. -- The abstract seems to indicate that this document obsoletes RFC5048, but the header doesn't have an 'Obsoletes:' line to match this. -- The abstract seems to indicate that this document obsoletes RFC4850, but the header doesn't have an 'Obsoletes:' line to match this. -- The abstract seems to indicate that this document updates RFC3721, but the header doesn't have an 'Updates:' line to match this. -- The abstract seems to indicate that this document updates RFC3980, but the header doesn't have an 'Updates:' line to match this. -- The abstract seems to indicate that this document updates RFC5048, but the header doesn't have an 'Updates:' line to match this. -- The abstract seems to indicate that this document updates RFC4850, but the header doesn't have an 'Updates:' line to match this. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- Couldn't find a document date in the document -- date freshness check skipped. -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '1' on line 2914 -- Looks like a reference, but probably isn't: '0' on line 6873 -- Looks like a reference, but probably isn't: '7' on line 6873 == Missing Reference: 'MaxMissingDPDU' is mentioned on line 12089, but not defined == Missing Reference: 'MaxOutStandingR2T' is mentioned on line 12097, but not defined == Missing Reference: 'MaxMissingSPDU' is mentioned on line 12110, but not defined == Missing Reference: 'MaxSupportedConns' is mentioned on line 12120, but not defined -- Possible downref: Non-RFC (?) normative reference: ref. 'EUI' -- Possible downref: Non-RFC (?) normative reference: ref. 'FC-FS3' == Outdated reference: A later version (-04) exists of draft-ietf-storm-ipsec-ips-update-01 -- Possible downref: Non-RFC (?) normative reference: ref. 'OUI' ** Obsolete normative reference: RFC 3454 (Obsoleted by RFC 7564) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) ** Obsolete normative reference: RFC 5996 (Obsoleted by RFC 7296) -- Possible downref: Non-RFC (?) normative reference: ref. 'SAM2' -- Possible downref: Non-RFC (?) normative reference: ref. 'SAM4' -- Possible downref: Non-RFC (?) normative reference: ref. 'SPC2' -- Possible downref: Non-RFC (?) normative reference: ref. 'SPC3' -- Possible downref: Non-RFC (?) normative reference: ref. 'UML' -- Possible downref: Non-RFC (?) normative reference: ref. 'UNICODE' -- Obsolete informational reference (is this intentional?): RFC 2401 (Obsoleted by RFC 4301) -- Obsolete informational reference (is this intentional?): RFC 2406 (Obsoleted by RFC 4303, RFC 4305) -- Obsolete informational reference (is this intentional?): RFC 2407 (Obsoleted by RFC 4306) -- Obsolete informational reference (is this intentional?): RFC 2409 (Obsoleted by RFC 4306) -- Obsolete informational reference (is this intentional?): RFC 3720 (Obsoleted by RFC 7143) -- Obsolete informational reference (is this intentional?): RFC 4850 (Obsoleted by RFC 7143) -- Obsolete informational reference (is this intentional?): RFC 5046 (Obsoleted by RFC 7145) -- Obsolete informational reference (is this intentional?): RFC 5048 (Obsoleted by RFC 7143) Summary: 3 errors (**), 0 flaws (~~), 8 warnings (==), 31 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Storage Maintenance (storm) WG Mallikarjun Chadalapaka 2 Internet Draft Microsoft 3 draft-ietf-storm-iscsi-cons-09.txt 4 Intended status: Proposed Standard Julian Satran 5 Expires: December 2013 Infinidat Ltd. 6 Obsoletes: RFC3720, RFC3980, RFC4850, RFC5048 7 Updates: RFC3721 Kalman Meth 8 IBM 10 David Black 11 EMC 13 iSCSI Protocol (Consolidated) 15 Status of this Memo 17 This Internet-Draft is submitted to IETF in full conformance with 18 the provisions of BCP 78 and BCP 79. 20 Internet-Drafts are working documents of the Internet Engineering 21 Task Force (IETF), its areas, and its working groups. Note that 22 other groups may also distribute working documents as Internet- 23 Drafts. 25 Internet-Drafts are draft documents valid for a maximum of six 26 months and may be updated, replaced, or obsoleted by other 27 documents at any time. It is inappropriate to use Internet-Drafts 28 as reference material or to cite them other than as "work in 29 progress." 31 The list of current Internet-Drafts can be accessed at 32 http://www.ietf.org/ietf/1id-abstracts.txt 33 The list of Internet-Draft Shadow Directories can be accessed at 34 http://www.ietf.org/shadow.html 36 This Internet-Draft will expire on December 31, 2013. 38 Copyright Notice 40 Copyright (c) 2013 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (http://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with 48 respect to this document. Code Components extracted from this 49 document must include Simplified BSD License text as described in 50 Section 4.e of the Trust Legal Provisions and are provided without 51 warranty as described in the Simplified BSD License. 53 Abstract 55 This document describes a transport protocol for SCSI that works 56 on top of TCP. The iSCSI protocol aims to be fully compliant with 57 the standardized SCSI Architecture Model (SAM-2). RFC 3720 58 defined the original iSCSI protocol. RFC 3721 discusses iSCSI 59 Naming examples and discovery techniques. Subsequently, RFC 3980 60 added an additional naming format to iSCSI protocol. RFC 4850 61 followed up by adding a new public extension key to iSCSI. RFC 62 5048 offered a number of clarifications and a few improvements and 63 corrections to the original iSCSI protocol. 65 This document obsoletes RFCs 3720, 3980, 4850 and 5048 by 66 consolidating them into a single document and making additional 67 updates to the consolidated specification. This document also 68 updates RFC 3721. The text in this document thus supersedes the 69 text in all the noted RFCs wherever there is a difference in 70 semantics. 72 Note: This version of the draft does not yet incorporate planned 73 resolutions to some Last Call comments regarding Kerberos and 74 IPsec-related security considerations. 76 1. Introduction.................................................... 15 78 2. Acronyms, Definitions and Document Summary...................... 16 79 2.1. Acronyms ................................................ 16 80 2.2. Definitions ............................................. 18 81 2.3. Summary of Changes ...................................... 25 82 2.4. Conventions ............................................. 26 83 3. UML Conventions................................................. 27 84 3.1. UML Conventions Overview ................................ 27 85 3.2. Multiplicity Notion ..................................... 27 86 3.3. Class Diagram Conventions ............................... 28 87 3.4. Class Diagram Notation for Associations ................. 29 88 3.5. Class Diagram Notation for Aggregations ................. 30 89 3.6. Class Diagram Notation for Generalizations .............. 30 90 4. Overview........................................................ 32 91 4.1. SCSI Concepts ........................................... 32 92 4.2. iSCSI Concepts and Functional Overview .................. 33 93 4.2.1. Layers and Sessions ................................. 34 94 4.2.2. Ordering and iSCSI Numbering ........................ 35 95 4.2.2.1. Command Numbering and Acknowledging .............. 35 96 4.2.2.2. Response/Status Numbering and Acknowledging ...... 39 97 4.2.2.3. Response Ordering ................................ 40 98 4.2.2.3.1. Need for Response Ordering ................... 40 99 4.2.2.3.2. Response Ordering Model Description .......... 40 100 4.2.2.3.3. iSCSI Semantics with the Interface Model ..... 41 101 4.2.2.3.4. Current List of Fenced Response Use Cases .... 42 102 4.2.2.4. Data Sequencing .................................. 43 103 4.2.3. iSCSI Task Management ............................... 44 104 4.2.3.1. Task Management Overview ......................... 44 105 4.2.3.2. Notion of Affected Tasks ......................... 44 106 4.2.3.3. Standard Multi-task Abort Semantics .............. 45 107 4.2.3.4. FastAbort Multi-task Abort Semantics ............. 46 108 4.2.3.5. Affected Tasks Shared across Standard and FastAbort 109 Sessions .................................................. 48 110 4.2.3.6. Rationale behind the FastAbort Semantics ......... 49 111 4.2.4. iSCSI Login ......................................... 51 112 4.2.5. iSCSI Full Feature Phase ............................ 52 113 4.2.5.1. Command Connection Allegiance .................... 53 114 4.2.5.2. Data Transfer Overview ........................... 54 115 4.2.5.3. Tags and Integrity Checks ........................ 55 116 4.2.5.4. Task Management .................................. 56 117 4.2.6. iSCSI Connection Termination ........................ 56 118 4.2.7. iSCSI Names ......................................... 57 119 4.2.7.1. iSCSI Name Properties ............................ 58 120 4.2.7.2. iSCSI Name Encoding .............................. 60 121 4.2.7.3. iSCSI Name Structure ............................. 61 122 4.2.7.4. Type "iqn." (iSCSI Qualified Name) ............... 62 123 4.2.7.5. Type "eui." (IEEE EUI-64 format) ................. 64 124 4.2.7.6. Type "naa." - Network Address Authority .......... 64 125 4.2.8. Persistent State .................................... 65 126 4.2.9. Message Synchronization and Steering ................ 66 127 4.2.9.1. Sync/Steering and iSCSI PDU Length ............... 67 128 4.3. iSCSI Session Types ..................................... 67 129 4.4. SCSI to iSCSI Concepts Mapping Model .................... 68 130 4.4.1. iSCSI Architecture Model ............................ 69 131 4.4.2. SCSI Architecture Model ............................. 72 132 4.4.3. Consequences of the Model ........................... 74 133 4.4.3.1. I_T Nexus State .................................. 75 134 4.4.3.2. Reservations ..................................... 75 135 4.5. iSCSI UML Model ......................................... 76 136 4.6. Request/Response Summary ................................ 79 137 4.6.1. Request/Response Types Carrying SCSI Payload ........ 79 138 4.6.1.1. SCSI-Command ..................................... 79 139 4.6.1.2. SCSI-Response .................................... 80 140 4.6.1.3. Task Management Function Request ................. 80 141 4.6.1.4. Task Management Function Response ................ 81 142 4.6.1.5. SCSI Data-out and SCSI Data-in ................... 81 143 4.6.1.6. Ready To Transfer (R2T) .......................... 82 144 4.6.2. Requests/Responses carrying SCSI and iSCSI Payload .. 83 145 4.6.2.1. Asynchronous Message ............................. 83 147 4.6.3. Requests/Responses Carrying iSCSI Only Payload ...... 83 148 4.6.3.1. Text Request and Text Response ................... 83 149 4.6.3.2. Login Request and Login Response ................. 84 150 4.6.3.3. Logout Request and Response ...................... 85 151 4.6.3.4. SNACK Request .................................... 85 152 4.6.3.5. Reject ........................................... 85 153 4.6.3.6. NOP-Out Request and NOP-In Response .............. 86 154 5. SCSI Mode Parameters for iSCSI................................. 87 156 6. Login and Full Feature Phase Negotiation....................... 88 157 6.1. Text Format ................................................ 89 158 6.2. Text Mode Negotiation ...................................... 93 159 6.2.1. List negotiations ...................................... 97 160 6.2.2. Simple-value Negotiations .............................. 98 161 6.3. Login Phase ................................................ 98 162 6.3.1. Login Phase Start ..................................... 102 163 6.3.2. iSCSI Security Negotiation ............................ 105 164 6.3.3. Operational Parameter Negotiation During the Login Phase 106 165 6.3.4. Connection Reinstatement .............................. 107 166 6.3.5. Session Reinstatement, Closure, and Timeout ........... 107 167 6.3.5.1. Loss of Nexus Notification ........................ 108 168 6.3.6. Session Continuation and Failure ...................... 108 169 6.4. Operational Parameter Negotiation Outside the Login Phase . 109 170 7. iSCSI Error Handling and Recovery............................. 111 171 7.1. Overview ............................................... 111 172 7.1.1. Background ......................................... 111 173 7.1.2. Goals .............................................. 111 174 7.1.3. Protocol Features and State Expectations ........... 112 175 7.1.4. Recovery Classes ................................... 113 176 7.1.4.1. Recovery Within-command ......................... 114 177 7.1.4.2. Recovery Within-connection ...................... 115 178 7.1.4.3. Connection Recovery ............................. 116 179 7.1.4.4. Session Recovery ................................ 117 180 7.1.5. Error Recovery Hierarchy ........................... 117 181 7.2. Retry and Reassign in Recovery ......................... 119 182 7.2.1. Usage of Retry ..................................... 119 183 7.2.2. Allegiance Reassignment ............................ 120 184 7.3. Usage Of Reject PDU in Recovery ........................ 121 185 7.4. Error Recovery Considerations for Discovery Sessions ... 122 186 7.4.1. ErrorRecoveryLevel for Discovery Sessions .......... 122 187 7.4.2. Reinstatement Semantics for Discovery Sessions ..... 122 188 7.4.2.1. Unnamed Discovery Sessions ...................... 123 189 7.4.2.2. Named Discovery Session ......................... 124 190 7.4.3. Target PDUs During Discovery ....................... 124 191 7.5. Connection Timeout Management .......................... 124 192 7.5.1. Timeouts on Transport Exception Events ............. 125 193 7.5.2. Timeouts on Planned Decommissioning ................ 125 194 7.6. Implicit Termination of Tasks .......................... 125 195 7.7. Format Errors .......................................... 126 196 7.8. Digest Errors .......................................... 127 197 7.9. Sequence Errors ........................................ 129 198 7.10. Message Error Checking ................................ 129 199 7.11. SCSI Timeouts ......................................... 130 200 7.12. Negotiation Failures .................................. 131 201 7.13. Protocol Errors ....................................... 131 202 7.14. Connection Failures ................................... 132 203 7.15. Session Errors ........................................ 133 204 8. State Transitions............................................. 134 205 8.1. Standard Connection State Diagrams ..................... 134 206 8.1.1. State Descriptions for Initiators and Targets ...... 134 207 8.1.2. State Transition Descriptions for Initiators and Targets135 208 8.1.3. Standard Connection State Diagram for an Initiator . 139 209 8.1.4. Standard Connection State Diagram for a Target ..... 141 210 8.2. Connection Cleanup State Diagram for Initiators and Targets 143 211 8.2.1. State Descriptions for Initiators and Targets ...... 145 212 8.2.2. State Transition Descriptions for Initiators and Targets146 213 8.3. Session State Diagrams .................................. 148 214 8.3.1. Session State Diagram for an Initiator .............. 148 215 8.3.2. Session State Diagram for a Target .................. 149 216 8.3.3. State Descriptions for Initiators and Targets ....... 150 217 8.3.4. State Transition Descriptions for Initiators and Targets 151 218 9. Security Considerations...................................... 153 219 9.1. iSCSI Security Mechanisms ............................. 153 220 9.2. In-band Initiator-Target Authentication ............... 154 221 9.2.1. CHAP Considerations ............................... 156 222 9.2.2. SRP Considerations ................................ 159 223 Kerberos Considerations ................................. 159 224 9.2.3. ................................................... 159 225 9.3. IPsec ................................................. 160 226 9.3.1. Data Integrity and Authentication ................. 160 227 9.3.2. Confidentiality ................................... 161 228 9.3.3. Policy, Security Associations, and Cryptographic Key 229 Management ............................................... 162 230 9.4. Security Considerations for the X#NodeArchitecture Key 164 231 9.5. SCSI Access Control Considerations .................... 166 232 10. Notes to Implementers....................................... 167 233 10.1. Multiple Network Adapters ............................... 167 234 10.1.1. Conservative Reuse of ISIDs ......................... 167 235 10.1.2. iSCSI Name, ISID, and TPGT Use ...................... 168 236 10.2. Autosense and Auto Contingent Allegiance (ACA) .......... 170 237 10.3. iSCSI Timeouts .......................................... 170 238 10.4. Command Retry and Cleaning Old Command Instances ........ 171 239 10.5. Synch and Steering Layer and Performance ................ 172 240 10.6. Considerations for State-dependent Devices and Long-lasting 241 SCSI Operations ............................................... 172 242 10.6.1. Determining the Proper ErrorRecoveryLevel ........... 173 243 10.7. Multi-task Abort Implementation Considerations .......... 174 244 11. iSCSI PDU Formats........................................... 175 245 11.1. iSCSI PDU Length and Padding ......................... 175 246 11.2. PDU Template, Header, and Opcodes .................... 175 247 11.2.1. Basic Header Segment (BHS) ....................... 176 248 11.2.1.1. I ............................................. 177 249 11.2.1.2. Opcode ........................................ 177 250 11.2.1.3. Final (F) bit ................................. 179 251 11.2.1.4. Opcode-specific Fields ........................ 179 252 11.2.1.5. TotalAHSLength ................................ 179 253 11.2.1.6. DataSegmentLength ............................. 179 254 11.2.1.7. LUN ........................................... 179 255 11.2.1.8. Initiator Task Tag ............................ 180 256 11.2.2. Additional Header Segment (AHS) .................. 180 257 11.2.2.1. AHSType ....................................... 180 258 11.2.2.2. AHSLength ..................................... 181 259 11.2.2.3. Extended CDB AHS .............................. 181 260 11.2.2.4. Bidirectional Expected Read-Data Length AHS ... 181 261 11.2.3. Header Digest and Data Digest .................... 182 262 11.2.4. Data Segment ..................................... 182 263 11.3. SCSI Command ......................................... 182 264 11.3.1. Flags and Task Attributes (byte 1) ............... 183 265 11.3.2. CmdSN - Command Sequence Number .................. 184 266 11.3.3. ExpStatSN ........................................ 185 267 11.3.4. Expected Data Transfer Length .................... 185 268 11.3.5. CDB - SCSI Command Descriptor Block .............. 186 269 11.3.6. Data Segment - Command Data ...................... 186 270 11.4. SCSI Response ........................................ 186 271 11.4.1. Flags (byte 1) ................................... 187 272 11.4.2. Status ........................................... 188 273 11.4.3. Response ......................................... 189 274 11.4.4. SNACK Tag ........................................ 190 275 11.4.5. Residual Count ................................... 190 276 11.4.5.1. Field Semantics ............................... 190 277 11.4.5.2. Residuals Concepts Overview ................... 191 278 11.4.5.3. SCSI REPORT LUNS and Residual Overflow ........ 191 279 11.4.6. Bidirectional Read Residual Count ................ 193 280 11.4.7. Data Segment - Sense and Response Data Segment ... 193 281 11.4.7.1. SenseLength ................................... 194 282 11.4.7.2. Sense Data .................................... 194 283 11.4.8. ExpDataSN ........................................ 195 284 11.4.9. StatSN - Status Sequence Number .................. 195 285 11.4.10. ExpCmdSN - Next Expected CmdSN from this Initiator 196 286 11.4.11. MaxCmdSN - Maximum CmdSN from this Initiator .... 196 288 11.5. Task Management Function Request ..................... 197 289 11.5.1. Function ......................................... 197 290 11.5.2. TotalAHSLength and DataSegmentLength ............. 201 291 11.5.3. LUN .............................................. 201 292 11.5.4. Referenced Task Tag .............................. 201 293 11.5.5. RefCmdSN ......................................... 201 294 11.5.6. ExpDataSN ........................................ 202 295 11.6. Task Management Function Response .................... 202 296 11.6.1. Response ......................................... 203 297 11.6.2. TotalAHSLength and DataSegmentLength ............. 205 298 11.7. SCSI Data-out & SCSI Data-in ......................... 205 299 11.7.1. F (Final) Bit .................................... 208 300 11.7.2. A (Acknowledge) bit .............................. 208 301 11.7.3. Flags (byte 1) ................................... 209 302 11.7.4. Target Transfer Tag and LUN ...................... 210 303 11.7.5. DataSN ........................................... 210 304 11.7.6. Buffer Offset .................................... 210 305 11.7.7. DataSegmentLength ................................ 211 306 11.8. Ready To Transfer (R2T) .............................. 212 307 11.8.1. TotalAHSLength and DataSegmentLength ............. 214 308 11.8.2. R2TSN ............................................ 214 309 11.8.3. StatSN ........................................... 214 310 11.8.4. Desired Data Transfer Length and Buffer Offset ... 214 311 11.8.5. Target Transfer Tag .............................. 214 312 11.9. Asynchronous Message ................................. 215 313 11.9.1. AsyncEvent ....................................... 216 314 11.9.2. AsyncVCode ....................................... 219 315 11.9.3. LUN .............................................. 219 316 11.9.4. Sense Data and iSCSI Event Data .................. 219 317 11.9.4.1. SenseLength ................................... 220 318 11.10. Text Request ........................................ 220 319 11.10.1. F (Final) Bit ................................... 222 320 11.10.2. C (Continue) Bit ................................ 222 321 11.10.3. Initiator Task Tag .............................. 222 322 11.10.4. Target Transfer Tag ............................. 222 323 11.10.5. Text ............................................ 223 325 11.11. Text Response ....................................... 224 326 11.11.1. F (Final) Bit ................................... 225 327 11.11.2. C (Continue) Bit ................................ 226 328 11.11.3. Initiator Task Tag .............................. 226 329 11.11.4. Target Transfer Tag ............................. 226 330 11.11.5. StatSN .......................................... 227 331 11.11.6. Text Response Data .............................. 227 332 11.12. Login Request ....................................... 227 333 11.12.1. T (Transit) Bit ................................. 228 334 11.12.2. C (Continue) Bit ................................ 229 335 11.12.3. CSG and NSG ..................................... 229 336 11.12.4. Version ......................................... 229 337 11.12.4.1. Version-max .................................. 229 338 11.12.4.2. Version-min .................................. 230 339 11.12.5. ISID ............................................ 230 340 11.12.6. TSIH ............................................ 232 341 11.12.7. Connection ID - CID ............................. 232 342 11.12.8. CmdSN ........................................... 232 343 11.12.9. ExpStatSN ....................................... 233 344 11.12.10. Login Parameters ............................... 233 345 11.13. Login Response ...................................... 233 346 11.13.1. Version-max ..................................... 234 347 11.13.2. Version-active .................................. 235 348 11.13.3. TSIH ............................................ 235 349 11.13.4. StatSN .......................................... 235 350 11.13.5. Status-Class and Status-Detail .................. 235 351 11.13.6. T (Transit) bit ................................. 239 352 11.13.7. C (Continue) Bit ................................ 240 353 11.13.8. Login Parameters ................................ 240 354 11.14. Logout Request ...................................... 240 355 11.14.1. Reason Code ..................................... 243 356 11.14.2. TotalAHSLength and DataSegmentLength ............ 244 357 11.14.3. CID ............................................. 244 358 11.14.4. ExpStatSN ....................................... 244 359 11.14.5. Implicit termination of tasks ................... 244 360 11.15. Logout Response ..................................... 245 361 11.15.1. Response ........................................ 246 362 11.15.2. TotalAHSLength and DataSegmentLength ............ 247 363 11.15.3. Time2Wait ....................................... 247 364 11.15.4. Time2Retain ..................................... 247 365 11.16. SNACK Request ....................................... 248 366 11.16.1. Type ............................................ 249 367 11.16.2. Data Acknowledgement ............................ 250 368 11.16.3. Resegmentation .................................. 250 369 11.16.4. Initiator Task Tag .............................. 251 370 11.16.5. Target Transfer Tag or SNACK Tag ................ 251 371 11.16.6. BegRun .......................................... 252 372 11.16.7. RunLength ....................................... 252 373 11.17. Reject .............................................. 253 374 11.17.1. Reason .......................................... 254 375 11.17.2. DataSN/R2TSN .................................... 255 376 11.17.3. StatSN, ExpCmdSN and MaxCmdSN ................... 255 377 11.17.4. Complete Header of Bad PDU ...................... 256 378 11.18. NOP-Out ............................................. 256 379 11.18.1. Initiator Task Tag .............................. 257 380 11.18.2. Target Transfer Tag ............................. 257 381 11.18.3. Ping Data ....................................... 258 382 11.19. NOP-In .............................................. 259 383 11.19.1. Target Transfer Tag ............................. 260 384 11.19.2. StatSN .......................................... 260 385 11.19.3. LUN ............................................. 260 386 12. iSCSI Security Text Keys and Authentication Methods.......... 261 387 12.1. AuthMethod ........................................... 261 388 12.1.1. Kerberos ......................................... 263 389 12.1.2. Secure Remote Password (SRP) ..................... 264 390 12.1.3. Challenge Handshake Authentication Protocol (CHAP) 266 391 13. Login/Text Operational Text Keys............................. 268 392 13.1. HeaderDigest and DataDigest .......................... 268 393 13.2. MaxConnections ....................................... 271 394 13.3. SendTargets .......................................... 271 395 13.4. TargetName ........................................... 271 396 13.5. InitiatorName ......................................... 272 397 13.6. TargetAlias ........................................... 273 398 13.7. InitiatorAlias ........................................ 273 399 13.8. TargetAddress ......................................... 274 400 13.9. TargetPortalGroupTag .................................. 275 401 13.10. InitialR2T ........................................... 275 402 13.11. ImmediateData ........................................ 276 403 13.12. MaxRecvDataSegmentLength ............................. 277 404 13.13. MaxBurstLength ....................................... 278 405 13.14. FirstBurstLength ..................................... 278 406 13.15. DefaultTime2Wait ..................................... 279 407 13.16. DefaultTime2Retain ................................... 279 408 13.17. MaxOutstandingR2T .................................... 280 409 13.18. DataPDUInOrder ....................................... 280 410 13.19. DataSequenceInOrder .................................. 281 411 13.20. ErrorRecoveryLevel ................................... 281 412 13.21. SessionType .......................................... 282 413 13.22. The Private Extension Key Format ..................... 283 414 13.23. TaskReporting ........................................ 283 415 13.24. iSCSIProtocolLevel Negotiation ....................... 284 416 13.25. Obsoleted Keys ....................................... 284 417 13.26. X#NodeArchitecture ................................... 285 418 13.26.1. Definition ....................................... 285 419 13.26.2. Implementation Requirements ...................... 286 420 14. Rationale for revised IANA Considerations................. 287 422 15. IANA Considerations....................................... 289 424 Appendix A. Examples ......................................... 296 425 Read Operation Example ...................................... 296 426 Write Operation Example ..................................... 297 427 R2TSN/DataSN Use Examples ................................... 297 428 CRC Examples ................................................ 301 429 Appendix B. Login Phase Examples ............................... 303 431 Appendix C. SendTargets Operation ............................ 313 432 Appendix D. Algorithmic Presentation of Error Recovery Classes . 318 433 D.2.1. Procedure Descriptions ............................... 320 434 Appendix E. Clearing Effects of Various Events on Targets ...... 337 435 1. Introduction 437 The Small Computer Systems Interface (SCSI) is a popular family of 438 protocols for communicating with I/O devices, especially storage 439 devices. SCSI is a client-server architecture. Clients of a SCSI 440 interface are called "initiators". Initiators issue SCSI 441 "commands" to request services from components, logical units of a 442 server known as a "target". A "SCSI transport" maps the client- 443 server SCSI protocol to a specific interconnect. An Initiator is 444 one endpoint of a SCSI transport and a target is the other 445 endpoint. 447 The SCSI protocol has been mapped over various transports, 448 including Parallel SCSI, IPI, IEEE-1394 (firewire) and Fibre 449 Channel. These transports are I/O specific and have limited 450 distance capabilities. 452 The iSCSI protocol defined in this document describes a means of 453 transporting of the SCSI packets over TCP/IP, providing for an 454 interoperable solution which can take advantage of existing 455 Internet infrastructure, Internet management facilities and 456 address distance limitations. 458 2. Acronyms, Definitions and Document Summary 460 2.1. Acronyms 462 Acronym Definition 463 -------------------------------------------------------------- 464 3DES Triple Data Encryption Standard 465 ACA Auto Contingent Allegiance 466 AEN Asynchronous Event Notification 467 AES Advanced Encryption Standard 468 AH Additional Header (not the IPsec AH!) 469 AHS Additional Header Segment 470 API Application Programming Interface 471 ASC Additional Sense Code 472 ASCII American Standard Code for Information Interchange 473 ASCQ Additional Sense Code Qualifier 474 BHS Basic Header Segment 475 CBC Cipher Block Chaining 476 CD Compact Disk 477 CDB Command Descriptor Block 478 CHAP Challenge Handshake Authentication Protocol 479 CID Connection ID 480 CO Connection Only 481 CRC Cyclic Redundancy Check 482 CRL Certificate Revocation List 483 CSG Current Stage 484 CSM Connection State Machine 485 DES Data Encryption Standard 486 DNS Domain Name Server 487 DOI Domain of Interpretation 488 DVD Digital Versatile Disk 489 EDTL Expected Data Transfer Length 490 ESP Encapsulating Security Payload 491 EUI Extended Unique Identifier 492 FFP Full Feature Phase 493 FFPO Full Feature Phase Only 494 Gbps Gigabits per Second 495 HBA Host Bus Adapter 496 HMAC Hashed Message Authentication Code 497 I_T Initiator_Target 499 I_T_L Initiator_Target_LUN 500 IANA Internet Assigned Numbers Authority 501 IB InfiniBand 502 ID Identifier 503 IDN Internationalized Domain Name 504 IEEE Institute of Electrical & Electronics Engineers 505 IETF Internet Engineering Task Force 506 IKE Internet Key Exchange 507 I/O Input-Output 508 IO Initialize Only 509 IP Internet Protocol 510 IPsec Internet Protocol Security 511 IPv4 Internet Protocol Version 4 512 IPv6 Internet Protocol Version 6 513 IQN iSCSI Qualified Name 514 iSCSI Internet SCSI 515 iSER iSCSI Extensions for RDMA 516 ISID Initiator Session ID 517 iSNS Internet Storage Name Service (see [RFC4171]) 518 ITN iSCSI Target Name 519 ITT Initiator Task Tag 520 KRB5 Kerberos V5 521 LFL Lower Functional Layer 522 LTDS Logical-Text-Data-Segment 523 LO Leading Only 524 LU Logical Unit 525 LUN Logical Unit Number 526 MAC Message Authentication Codes 527 NA Not Applicable 528 NAA Network Address Authority 529 NIC Network Interface Card 530 NOP No Operation 531 NSG Next Stage 532 OS Operating System 533 PDU Protocol Data Unit 534 PKI Public Key Infrastructure 535 R2T Ready To Transfer 536 R2TSN Ready To Transfer Sequence Number 537 RDMA Remote Direct Memory Access 538 RFC Request For Comments 539 SAM SCSI Architecture Model 540 SAM2 SCSI Architecture Model - 2 541 SAN Storage Area Network 542 SAS Serial Attached SCSI 543 SCSI Small Computer Systems Interface 544 SLP Service Location Protocol 545 SN Sequence Number 546 SNACK Selective Negative Acknowledgment - also 547 Sequence Number Acknowledgement for data 548 SPDTL SCSI-Presented Data Transfer Length 549 SPKM Simple Public-Key Mechanism 550 SRP Secure Remote Password 551 SSID Session ID 552 SW Session-Wide 553 TCB Task Control Block 554 TCP Transmission Control Protocol 555 TMF Task Management Function 556 TPGT Target Portal Group Tag 557 TSIH Target Session Identifying Handle 558 TTT Target Transfer Tag 559 UA Unit Attention 560 UFL Upper Functional Layer 561 ULP Upper Level Protocol 562 URN Uniform Resource Names 563 UTF Universal Transformation Format 564 WG Working Group 566 2.2. Definitions 568 - Alias: An alias string can also be associated with an iSCSI 569 Node. The alias allows an organization to associate a user- 570 friendly string with the iSCSI Name. However, the alias string is 571 not a substitute for the iSCSI Name. 573 - CID (Connection ID): Connections within a session are identified 574 by a connection ID. It is a unique ID for this connection within 575 the session for the initiator. It is generated by the initiator 576 and presented to the target during login requests and during 577 logouts that close connections. 579 - Connection: A connection is a TCP connection. Communication 580 between the initiator and target occurs over one or more TCP 582 connections. The TCP connections carry control messages, SCSI 583 commands, parameters, and data within iSCSI Protocol Data Units 584 (iSCSI PDUs). 586 - I/O Buffer:A buffer that is used in a SCSI Read or Write 587 operation so SCSI data may be sent from or received into that 588 buffer. For a read or write data transfer to take place for a 589 task, an I/O Buffer is required on the initiator and at least one 590 is required on the 591 target. 593 - INCITS: INCITS stands for InterNational Committee of Information 594 Technology Standards. The INCITS has a broad standardization scope 595 within the field of Information and Communications Technologies 596 (ICT), encompassing storage, processing, transfer, display, 597 management, organization, and retrieval of information. INCITS 598 serves as ANSI's Technical Advisory Group for the ISO/IEC Joint 599 Technical Committee 1 (JTC 1). See http://www.incits.org. 601 - InfiniBand: An I/O architecture originally intended to replace 602 PCI and to address high performance server interconnectivity [IB]. 604 - iSCSI Device: A SCSI Device using an iSCSI service delivery 605 subsystem. Service Delivery Subsystem is defined by [SAM2] as a 606 transport mechanism for SCSI commands and responses. 608 - iSCSI Initiator Name: The iSCSI Initiator Name specifies the 609 worldwide unique name of the initiator. 611 - iSCSI Initiator Node: The "initiator" device. The word 612 "initiator" has been appropriately qualified as either a port or a 613 device in the rest of the document when the context is ambiguous. 614 All unqualified usages of "initiator" refer to an initiator port 615 (or device) depending on the context. 617 - iSCSI Layer: This layer builds/receives iSCSI PDUs and 618 relays/receives them to/from one or more TCP connections that form 619 an initiator-target "session". 621 - iSCSI Name: The name of an iSCSI initiator or iSCSI target. 623 - iSCSI Node: The iSCSI Node represents a single iSCSI initiator 624 or iSCSI target or a single instance of each. There are one or 625 more iSCSI Nodes within a Network Entity. The iSCSI Node is 626 accessible via one or more Network Portals. An iSCSI Node is 627 identified by its iSCSI Name. The separation of the iSCSI Name 628 from the addresses used by and for the iSCSI Node allows multiple 629 iSCSI nodes to use the same address, and the same iSCSI node to 630 use multiple addresses. 632 - iSCSI Target Name: The iSCSI Target Name specifies the worldwide 633 unique name of the target. 635 - iSCSI Target Node: The "target" device. The word "target" has 636 been appropriately qualified as either a port or a device in the 637 rest of the document when the context is ambiguous. All 638 unqualified usages of "target" refer to a target port (or device) 639 depending on the context. 641 - iSCSI Task: An iSCSI task is an iSCSI request for which a 642 response is expected. 644 - iSCSI Transfer Direction: The iSCSI transfer direction is 645 defined with regard to the initiator. Outbound or outgoing 646 transfers are transfers from the initiator to the target, while 647 inbound or incoming transfers are from the target to the 648 initiator. 650 - ISID: The initiator part of the Session Identifier. It is 651 explicitly specified by the initiator during Login. 653 - I_T nexus: According to [SAM2], the I_T nexus is a relationship 654 between a SCSI Initiator Port and a SCSI Target Port. For iSCSI, 655 this relationship is a session, defined as a relationship between 656 an iSCSI Initiator's end of the session (SCSI Initiator Port) and 657 the iSCSI Target's Portal Group. The I_T nexus can be identified 658 by the conjunction of the SCSI port names; that is, the I_T nexus 659 identifier is the tuple (iSCSI Initiator Name + ',i,'+ ISID, iSCSI 660 Target Name + ',t,'+ Portal Group Tag). 662 - I_T_L nexus: An I_T_L nexus is a SCSI concept, and is defined as 663 the relationship between a SCSI Initiator Port, a SCSI Target 664 Port, and a Logical Unit (LU). 666 - NAA: Network Address Authority, a naming format defined by the 667 INCITS T11 Fibre Channel protocols [FC-FS3]. 669 - Network Entity: The Network Entity represents a device or 670 gateway that is accessible from the IP network. A Network Entity 671 must have one or more Network Portals, each of which can be used 672 to gain access to the IP network by some iSCSI Nodes contained in 673 that Network Entity. 675 - Network Portal: The Network Portal is a component of a Network 676 Entity that has a TCP/IP network address and that may be used by 677 an iSCSI Node within that Network Entity for the connection(s) 678 within one of its iSCSI sessions. A Network Portal in an initiator 679 is identified by its IP address. A Network Portal in a target is 680 identified by its IP address and its listening TCP port. 682 - Originator: In a negotiation or exchange, the party that 683 initiates the negotiation or exchange. 685 - PDU (Protocol Data Unit): The initiator and target divide their 686 communications into messages. The term "iSCSI protocol data unit" 687 (iSCSI PDU) is used for these messages. 689 - Portal Groups: iSCSI supports multiple connections within the 690 same session; some implementations will have the ability to 691 combine connections in a session across multiple Network Portals. 692 A Portal Group defines a set of Network Portals within an iSCSI 693 Network Entity that collectively supports the capability of 694 coordinating a session with connections spanning these portals. 695 Not all Network Portals within a Portal Group need participate in 696 every session connected through that Portal Group. One or more 697 Portal Groups may provide access to an iSCSI Node. Each Network 698 Portal, as utilized by a given iSCSI Node, belongs to exactly one 699 portal group within that node. 701 - Portal Group Tag: This 16-bit quantity identifies a Portal Group 702 within an iSCSI Node. All Network Portals with the same portal 703 group tag in the context of a given iSCSI Node are in the same 704 Portal Group. 706 - Recovery R2T: An R2T generated by a target upon detecting the 707 loss of one or more Data-Out PDUs through one of the following 708 means: a digest error, a sequence error, or a sequence reception 709 timeout. A recovery R2T carries the next unused R2TSN, but 710 requests all or part of the data burst that an earlier R2T (with a 711 lower R2TSN) had already requested. 713 - Responder: In a negotiation or exchange, the party that responds 714 to the originator of the negotiation or exchange. 716 - SAS: Serial Attached SCSI. The Serial Attached SCSI (SAS) 717 standard contains both a physical layer compatible with Serial 718 ATA, and protocols for transporting SCSI commands to SAS devices 719 and ATA commands to SATA devices [SAS]. 721 - SCSI Device: This is the SAM2 term for an entity that contains 722 one or more SCSI ports that are connected to a service delivery 723 subsystem and supports a SCSI application protocol. For example, a 724 SCSI Initiator Device contains one or more SCSI Initiator Ports 725 and zero or more application clients. A Target Device contains one 726 or more SCSI Target Ports and one or more device servers and 727 associated logical units. For iSCSI, the SCSI Device is the 728 component within an iSCSI Node that provides the SCSI 729 functionality. As such, there can be, at most, one SCSI Device 730 within a given iSCSI Node. Access to the SCSI Device can only be 731 achieved in an iSCSI normal operational session. The SCSI Device 732 Name is defined to be the iSCSI Name of the node. 734 - SCSI Layer: This builds/receives SCSI CDBs (Command Descriptor 735 Blocks) and relays/receives them with the remaining command 736 execute [SAM2] parameters to/from the iSCSI Layer. 738 - Session: The group of TCP connections that link an initiator 739 with a target form a session (loosely equivalent to a SCSI I-T 740 nexus). TCP connections can be added and removed from a session. 742 Across all connections within a session, an initiator sees one and 743 the same target. 745 - SCSI Port: This is the SAM2 term for an entity in a SCSI Device 746 that provides the SCSI functionality to interface with a service 747 delivery subsystem. For iSCSI, the definition of the SCSI 748 Initiator Port and the SCSI Target Port are different. 750 - SCSI Initiator Port: This maps to the endpoint of an iSCSI 751 normal operational session. An iSCSI normal operational session is 752 negotiated through the login process between an iSCSI initiator 753 node and an iSCSI target node. At successful completion of this 754 process, a SCSI Initiator Port is created within the SCSI 755 Initiator Device. The SCSI Initiator Port Name and SCSI Initiator 756 Port Identifier are both defined to be the iSCSI Initiator Name 757 together with (a) a label that identifies it as an initiator port 758 name/identifier and (b) the ISID portion of the session 759 identifier. 761 - SCSI Port Name: A name consisting of UTF-8 [RFC3629] encoding of 762 Unicode [UNICODE] characters and includes the iSCSI Name + 'i' or 763 't' + ISID or Portal Group Tag. 765 - SCSI-Presented Data Transfer Length (SPDTL): SPDTL is the 766 aggregate data length of the data that the SCSI layer logically 767 "presents" to the iSCSI layer for a Data-In or Data-Out transfer 768 in the context of a SCSI task. For a bidirectional task, there are 769 two SPDTL values -- one for Data-In and one for Data-Out. Note 770 that the notion of "presenting" includes immediate data per the 771 data transfer model in [SAM2], and excludes overlapping data 772 transfers, if any, requested by the SCSI layer. 774 - SCSI Target Port: This maps to an iSCSI Target Portal Group. 776 - SCSI Target Port Name and SCSI Target Port Identifier: These are 777 both defined to be the iSCSI Target Name together with (a) a label 778 that identifies it as a target port name/identifier and (b) the 779 portal group tag. 781 - SSID (Session ID): A session between an iSCSI initiator and an 782 iSCSI target is defined by a session ID that is a tuple composed 783 of an initiator part (ISID) and a target part (Target Portal Group 784 Tag). The ISID is explicitly specified by the initiator at session 785 establishment. The Target Portal Group Tag is implied by the 786 initiator through the selection of the TCP endpoint at connection 787 establishment. The TargetPortalGroupTag key must also be returned 788 by the target as a confimation during connection establishment. 790 - T10: A technical committee within INCITS that develops standards 791 and technical reports on I/O interfaces, particularly the series 792 of SCSI (Small Computer Systems Interface) standards. See 793 http://www.t10.org. 795 - T11: A technical committee within INCITS responsible for 796 standards development in the areas of Intelligent Peripheral 797 Interface (IPI), High-Performance Parallel Interface (HIPPI) and 798 Fibre Channel (FC). See http://www.t11.org. 800 - Target Portal Group Tag: A numerical identifier (16-bit) for an 801 iSCSI Target Portal Group. 803 -Target Transfer Tag (TTT): An iSCSI protocol field used in a few 804 iSCSI PDUs (e.g. R2T, NOP-In) which is always sent from the target 805 to the initiator first and then quoted as a reference in 806 initiator-sent PDUs back to the target relating to the same 807 task/exchange. So effectively, TTT acts as an opaque handle to an 808 existing task/exchange to help target associate the incoming PDUs 809 from the initiator to the proper execution context. 811 - Third-party: A term used in this document as a qualifier to 812 nexus objects (I_T or I_T_L) and iSCSI sessions, to indicate that 813 these objects and sessions reap the side effects of actions that 814 take place in the context of a separate iSCSI session. One 815 example of a third-party session is an iSCSI session discovering 816 that its I_T_L nexus to an LU got reset due to an LU Reset 817 operation orchestrated via a separate I_T nexus. 819 - TSIH (Target Session Identifying Handle): A target assigned tag 820 for a session with a specific named initiator. The target 821 generates it during session establishment. Other than defining it 822 as a 16 bit binary string, its internal format and content are not 823 defined by this protocol but for the all 0 value that is reserved 824 and used by the initiator to indicate a new session. It is given 825 to the target during additional connection establishment for the 826 same session. 828 2.3. Summary of Changes 830 1) Consolidated RFCs 3720, 3980, 4850 and 5048, and made the 831 necessary editorial changes 832 2) iSCSIProtocolLevel is specified as "1" in Section 13.24, and 833 added a related normative reference to [iSCSI-SAM] draft 834 3) Markers and related keys were removed 835 4) SPKM authentication and related keys were removed 836 5) Added a new Section 13.25 on responding to obsoleted keys 837 6) Have explicitly allowed initiator+target implementations 838 throughout the text 839 7) Clarified in Section 4.2.7 that implementations SHOULD NOT 840 rely on SLP-based discovery 841 8) Added UML diagrams and related conventions in Section 3 842 9) FastAbort implementation is made a "SHOULD" requirement in 843 Section 4.2.3.4 from the previous "MUST" requirement. 844 10) Required in Section 4.2.7.1 that iSCSI Target Name must be 845 the same as iSCSI Initiator Name for SCSI (composite) devices 846 with both roles 847 11) Changed the "MUST NOT" to "should avoid" in Section 4.2.7.2 848 regarding usage of characters such as punctuation marks in 849 iSCSI Names. 850 12) Updated Section 9.3 to require the following: MUST implement 851 IPsec, 2400-series RFCs (IPsec v2, IKEv1) and SHOULD implement 852 IPsec, 4300-series RFCs (IPsec v3, IKEv2). 853 13) Clarified in Section 10.2 that ACA is a SHOULD requirement 854 only for iSCSI targets 855 14) Prohibited usage of X# name prefix for new public keys in 856 Section 6.2 857 15) Prohibited usage of Y# name prefix for new digest extensions 858 in Section 13.1, and Z# name prefix for new authentication 859 method extensions in Section 12.1 860 16) Added a SHOULD requirement in Section 6.2 that initiators and 861 targets support at least six (6) exchanges during text 862 negotiation. 863 17) Added a clarification that Appendix.C is normative. 865 18) Added a normative requirement on [IPSEC-IPS] draft, and made 866 a few related changes in Section 9.3 to align the text in this 867 document with that of [IPSEC-IPS] 868 19) Added a new Section 9.2.3 covering Kerberos authentication 869 considerations 871 2.4. Conventions 873 In examples, "I->" and "T->" show iSCSI PDUs sent by the initiator 874 and target respectively. 876 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 877 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" 878 in this document are to be interpreted as described in RFC 2119 879 [RFC2119]. 881 3. UML Conventions 883 3.1. UML Conventions Overview 885 The SCSI Architecture Model (SAM) uses class diagrams and object 886 diagrams with notation that is based on the Unified Modeling 887 Language [UML]. Therefore, this document also uses UML to model 888 the relationships for SCSI and iSCSI objects. 890 A treatise on the graphical notation used in UML is beyond the 891 scope of this document. However, given the use of ASCII drawing 892 for UML static class diagrams, a description of the notational 893 conventions used in this document is included in the remainder of 894 this Section. 896 3.2. Multiplicity Notion 898 Not specified The number of instances of an attribute is not 899 specified. 901 1 One instance of the class or attribute exists. 903 0..* Zero or more instances of the class or attribute exist. 905 1..* One or more instances of the class or attribute exist. 907 0..1 Zero or one instance of the class or attribute exists. 909 n..m n to m instances of the class or attribute exist 910 (e.g., 2..8). 912 x, n..m Multiple disjoint instances of the class or 913 attribute exist (e.g., 2, 8..15). 915 3.3. Class Diagram Conventions 917 +--------------+ +--------------+ +--------------+ 918 | Class Name | | Class Name | | Class Name | 919 +--------------+ +--------------+ +--------------+ 920 | | | | 921 +--------------+ +--------------+ 922 | | 923 +--------------+ 924 The previous three diagrams are examples of a class with no 925 attributes and with no operations. 927 +-------------------+ +-------------------+ 928 | Class Name | | Class Name | 929 +-------------------+ +-------------------+ 930 | attribute 01[1] | | attribute 01[1] | 931 | attribute 02[1] | | attribute 02[1] | 932 +-------------------+ +-------------------+ 933 | | 934 +-------------------+ 935 The preceding two diagrams are examples of a class with attributes 936 and with no operations. 938 +------------------------+ 939 | Class Name | 940 +------------------------+ 941 | attribute 01[1..*] | 942 | attribute 02[1] | 943 +------------------------+ 944 | operation 01() | 945 | operation 02() | 946 +------------------------+ 947 The preceding diagram is an example of a class with attributes 948 that have a specified multiplicity and operations. 950 3.4. Class Diagram Notation for Associations 952 +-----------------+ 953 | Class A | 954 +-----------------+ association_name +-----------------+ 955 | attribute 01[1] |<------------------>| Class B | 956 | attribute 02[1] | 1..* 0..1 +-----------------+ 957 +-----------------+ | attribute 03[1] | 958 | operation 1() | +-----------------+ 959 +-----------------+ 960 The preceding diagram is an example where Class A knows about 961 Class B (i.e., read as "Class A association_name ClassB") and 962 Class B knows about Class A (i.e., read as "Class B 963 association_name Class A"). The use of association_name is 964 optional. The multiplicity notation (1..* and 0..1) indicates the 965 number of instances of the object. 967 +--------------------+ 968 | Class A | 969 +--------------------+ +--------------------+ 970 | attribute 01[1] |<-------------| Class B | 971 | attribute 02[1] | 1 0..1 +--------------------+ 972 +--------------------+ | attribute 03[1] | 973 | operation 1() | +--------------------+ 974 +--------------------+ 975 The preceding diagram is an example where Class B knows about 976 Class A (i.e., read as "Class B knows about Class A") but Class A 977 does not know about Class B. 979 +----------------------+ 980 | Class A | 981 +----------------------+ +--------------------+ 982 | attribute 01[1] |----------->| Class B | 983 | attribute 02[1] | 0..* 1 +--------------------+ 984 +----------------------+ | attribute 03[1] | 985 | operation 1() | +--------------------+ 986 +----------------------+ 987 The preceding diagram is an example where Class A knows about 988 Class B (i.e., read as "Class A knows about Class B") but Class B 989 does not know about Class A. 991 3.5. Class Diagram Notation for Aggregations 993 +---------------+ +--------------+ 994 | Class whole |o------------| Class part | 995 +---------------+ +--------------+ 996 The preceding diagram is an example where Class whole is an 997 aggregate that contains Class part and where Class part may 998 continue to exist even if Class whole is removed (i.e., read as 999 "the whole contains the part"). 1001 +---------------+ +--------------+ 1002 | Class whole |@------------| Class part | 1003 +---------------+ +--------------+ 1004 The preceding diagram is an example where Class whole is an 1005 aggregate that contains Class part where Class part only belongs 1006 to one Class whole, and the Class part does not continue to exist 1007 if the Class whole is removed (i.e., read as "the whole contains 1008 the part"). 1010 +-------------+ 1011 | | 1012 +-------------+ 1013 | | 1014 + =(a)= + 1015 | | 1016 The preceding diagram is an example where there is a constraint 1017 between the associations where the (a) footnote describes the 1018 constraint. 1020 3.6. Class Diagram Notation for Generalizations 1022 +---------------+ 1023 | Superclass | 1024 +-------^-------+ 1025 /_\ 1026 | 1027 +---------------+ 1028 | Subclass | 1029 +---------------+ 1030 The preceding diagram is an example where the subclass is a kind 1031 of superclass. A subclass shares all the attributes and 1032 operations of the superclass (i.e., the subclass inherits from the 1033 superclass). 1035 4. Overview 1037 4.1. SCSI Concepts 1039 The SCSI Architecture Model-2 [SAM2] describes in detail the 1040 architecture of the SCSI family of I/O protocols. This Section 1041 provides a brief background of the SCSI architecture and is 1042 intended to familiarize readers with its terminology. 1044 At the highest level, SCSI is a family of interfaces for 1045 requesting services from I/O devices, including hard drives, tape 1046 drives, CD and DVD drives, printers, and scanners. In SCSI 1047 terminology, an individual I/O device is called a "logical unit" 1048 (LU). 1050 SCSI is a client-server architecture. Clients of a SCSI interface 1051 are called "initiators". Initiators issue SCSI "commands" to 1052 request services from components, logical units, of a server known 1053 as a "target". The "device server" on the logical unit accepts 1054 SCSI commands and processes them. 1056 A "SCSI transport" maps the client-server SCSI protocol to a 1057 specific interconnect. Initiator is one endpoint of a SCSI 1058 transport. The "target" is the other endpoint. A target can 1059 contain multiple Logical Units (LUs). Each Logical Unit has an 1060 address within a target called a Logical Unit Number (LUN). 1062 A SCSI task is a SCSI command or possibly a linked set of SCSI 1063 commands. Some LUs support multiple pending (queued) tasks, but 1064 the queue of tasks is managed by the logical unit. The target uses 1065 an initiator provided "task tag" to distinguish between tasks. 1066 Only one command in a task can be outstanding at any given time. 1068 Each SCSI command results in an optional data phase and a required 1069 response phase. In the data phase, information can travel from the 1070 initiator to target (e.g., WRITE), target to initiator (e.g., 1071 READ), or in both directions. In the response phase, the target 1072 returns the final status of the operation, including any errors. 1074 Command Descriptor Blocks (CDB) are the data structures used to 1075 contain the command parameters that an initiator sends to a 1077 target. The CDB content and structure is defined by [SAM2] and 1078 device-type specific SCSI standards. 1080 4.2. iSCSI Concepts and Functional Overview 1082 The iSCSI protocol is a mapping of the SCSI command, event and 1083 task management model (see [SAM2]) over the TCP protocol. SCSI 1084 commands are carried by iSCSI requests and SCSI responses and 1085 status are carried by iSCSI responses. iSCSI also uses the request 1086 response mechanism for iSCSI protocol mechanisms. 1088 For the remainder of this document, the terms "initiator" and 1089 "target" refer to "iSCSI initiator node" and "iSCSI target node", 1090 respectively (see iSCSI) unless otherwise qualified. 1092 As its title suggests, Section 4 presents an overview of the iSCSI 1093 concepts, and later Sections in the rest of the specification 1094 contain the normative requirements - in many cases covering the 1095 same concepts discussed in Section 4. Such normative requirements 1096 text overrides the overview text in Section 4 if there is a 1097 disagreement between the two. 1099 In keeping with similar protocols, the initiator and target divide 1100 their communications into messages. This document uses the term 1101 "iSCSI protocol data unit" (iSCSI PDU) for these messages. 1103 For performance reasons, iSCSI allows a "phase-collapse". A 1104 command and its associated data may be shipped together from 1105 initiator to target, and data and responses may be shipped 1106 together from targets. 1108 The iSCSI transfer direction is defined with respect to the 1109 initiator. Outbound or outgoing transfers are transfers from an 1110 initiator to a target, while inbound or incoming transfers are 1111 from a target to an initiator. 1113 An iSCSI task is an iSCSI request for which a response is 1114 expected. 1116 In this document "iSCSI request", "iSCSI command", request, or 1117 (unqualified) command have the same meaning. Also, unless 1118 otherwise specified, status, response, or numbered response have 1119 the same meaning. 1121 4.2.1. Layers and Sessions 1123 The following conceptual layering model is used to specify 1124 initiator and target actions and the way in which they relate to 1125 transmitted and received Protocol Data Units: 1127 - The SCSI layer builds/receives SCSI CDBs (Command Descriptor 1128 Blocks) and passes/receives them with the remaining command 1129 execute parameters ([SAM2]) to/from 1131 - the iSCSI layer that builds/receives iSCSI PDUs and 1132 relays/receives them to/from one or more TCP connections; 1133 the group of connections form an initiator-target "session". 1135 Communication between the initiator and target occurs over one or 1136 more TCP connections. The TCP connections carry control messages, 1137 SCSI commands, parameters, and data within iSCSI Protocol Data 1138 Units (iSCSI PDUs). The group of TCP connections that link an 1139 initiator with a target form a session (equivalent to a SCSI I_T 1140 nexus, see Section 4.4.2). A session is defined by a session ID 1141 that is composed of an initiator part and a target part. TCP 1142 connections can be added and removed from a session. Each 1143 connection within a session is identified by a connection ID 1144 (CID). 1146 Across all connections within a session, an initiator sees one 1147 "target image". All target identifying elements, such as LUN, are 1148 the same. A target also sees one "initiator image" across all 1149 connections within a session. Initiator-identifying elements, such 1150 as the Initiator Task Tag, are global across the session 1151 regardless of the connection on which they are sent or received. 1153 iSCSI targets and initiators MUST support at least one TCP 1154 connection and MAY support several connections in a session. For 1155 error recovery purposes, targets and initiators that support a 1156 single active connection in a session SHOULD support two 1157 connections during recovery. 1159 4.2.2. Ordering and iSCSI Numbering 1161 iSCSI uses Command and Status numbering schemes and a Data 1162 sequencing scheme. 1164 Command numbering is session-wide and is used for ordered command 1165 delivery over multiple connections. It can also be used as a 1166 mechanism for command flow control over a session. 1168 Status numbering is per connection and is used to enable missing 1169 status detection and recovery in the presence of transient or 1170 permanent communication errors. 1172 Data sequencing is per command or part of a command (R2T-triggered 1173 sequence) and is used to detect missing data and/or R2T PDUs due 1174 to header digest errors. 1176 Typically, fields in the iSCSI PDUs communicate the Sequence 1177 Numbers between the initiator and target. During periods when 1178 traffic on a connection is unidirectional, iSCSI NOP-Out/In PDUs 1179 may be utilized to synchronize the command and status ordering 1180 counters of the target and initiator. 1182 The iSCSI session abstraction is equivalent to the SCSI I_T nexus, 1183 and the iSCSI session provides an ordered command delivery from 1184 the SCSI initiator to the SCSI target. For detailed design 1185 considerations that led to the iSCSI session model as it is 1186 defined here and how it relates the SCSI command ordering features 1187 defined in SCSI specifications to the iSCSI concepts see 1188 [RFC3783]. 1190 4.2.2.1. Command Numbering and Acknowledging 1192 iSCSI performs ordered command delivery within a session. All 1193 commands (initiator-to-target PDUs) in transit from the initiator 1194 to the target are numbered. 1196 iSCSI considers a task to be instantiated on the target in 1197 response to every request issued by the initiator. A set of task 1198 management operations including abort and reassign (see Section 1199 11.5) may be performed on an iSCSI task - however an abort 1200 operation cannot be performed on a task management operation, and 1201 usage of reassign operation has certain constraints. See Section 1202 11.5.1 for the details. 1204 Some iSCSI tasks are SCSI tasks, and many SCSI activities are 1205 related to a SCSI task ([SAM2]). In all cases, the task is 1206 identified by the Initiator Task Tag for the life of the task. 1208 The command number is carried by the iSCSI PDU as CmdSN (Command- 1209 Sequence-Number). The numbering is session-wide. Outgoing iSCSI 1210 PDUs carry this number. The iSCSI initiator allocates CmdSNs with 1211 a 32-bit unsigned counter (modulo 2**32). Comparisons and 1212 arithmetic on CmdSN use Serial Number Arithmetic as defined in 1213 [RFC1982] where SERIAL_BITS = 32. 1215 Commands meant for immediate delivery are marked with an immediate 1216 delivery flag; they MUST also carry the current CmdSN. CmdSN MUST 1217 NOT advance after a command marked for immediate delivery is sent. 1219 Command numbering starts with the first login request on the first 1220 connection of a session (the leading login on the leading 1221 connection) and CmdSN MUST be incremented by 1, in a Serial Number 1222 Arithmetic sense as defined in [RFC1982], for every non-immediate 1223 command issued afterwards. 1225 If immediate delivery is used with task management commands, these 1226 commands may reach the target before the tasks on which they are 1227 supposed to act. However their CmdSN serves as a marker of their 1228 position in the stream of commands. The initiator and target MUST 1229 ensure that the SCSI task management functions specified in [SAM2] 1230 act in accordance with the [SAM2] specification. For example, both 1231 commands and responses appear as if delivered in order. Whenever 1232 CmdSN for an outgoing PDU is not specified by an explicit rule, 1233 CmdSN will carry the current value of the local CmdSN variable 1234 (see later in this Section). 1236 The means by which an implementation decides to mark a PDU for 1237 immediate delivery or by which iSCSI decides by itself to mark a 1238 PDU for immediate delivery are beyond the scope of this document. 1240 The number of commands used for immediate delivery is not limited 1241 and their delivery to execution is not acknowledged through the 1242 numbering scheme. An iSCSI target MAY reject immediate commands, 1243 e.g., due to lack of resources to accommodate additional commands. 1244 An iSCSI target MUST be able to handle at least one immediate task 1245 management command and one immediate non-task-management iSCSI 1246 command per connection at any time. 1248 In this document, delivery for execution means delivery to the 1249 SCSI execution engine or an iSCSI protocol specific execution 1250 engine (e.g., for text requests with public or private extension 1251 keys involving an execution component). With the exception of the 1252 commands marked for immediate delivery, the iSCSI target layer 1253 MUST deliver the commands for execution in the order specified by 1254 CmdSN. Commands marked for immediate delivery may be delivered by 1255 the iSCSI target layer for execution as soon as detected. iSCSI 1256 may avoid delivering some commands to the SCSI target layer if 1257 required by a prior SCSI or iSCSI action (e.g., CLEAR TASK SET 1258 Task Management request received before all the commands on which 1259 it was supposed to act). 1261 On any connection, the iSCSI initiator MUST send the commands in 1262 increasing order of CmdSN, except for commands that are 1263 retransmitted due to digest error recovery and connection 1264 recovery. 1266 For the numbering mechanism, the initiator and target maintain the 1267 following three variables for each session: 1269 - CmdSN - the current command Sequence Number, advanced by 1 1270 on each command shipped except for commands marked for 1271 immediate delivery (see Section 4.2.2.1). CmdSN always 1272 contains the number to be assigned to the next Command PDU. 1274 - ExpCmdSN - the next expected command by the target. The 1275 target acknowledges all commands up to, but not including, 1276 this number. The initiator treats all commands with CmdSN 1277 less than ExpCmdSN as acknowledged. The target iSCSI layer 1278 sets the ExpCmdSN to the largest non-immediate CmdSN that it 1279 can deliver for execution "plus 1" per [RFC1982]. There 1280 MUST NOT be any holes in the acknowledged CmdSN sequence. 1282 - MaxCmdSN - the maximum number to be shipped. The queuing 1283 capacity of the receiving iSCSI layer is MaxCmdSN - ExpCmdSN 1284 + 1. 1286 The initiator's ExpCmdSN and MaxCmdSN are derived from target-to- 1287 initiator PDU fields. Comparisons and arithmetic on ExpCmdSN and 1288 MaxCmdSN MUST use Serial Number Arithmetic as defined in [RFC1982] 1289 where SERIAL_BITS = 32. 1291 The target MUST NOT transmit a MaxCmdSN that is less than 1292 ExpCmdSN-1. For non-immediate commands, the CmdSN field can take 1293 any value from ExpCmdSN to MaxCmdSN inclusive. The target MUST 1294 silently ignore any non-immediate command outside of this range or 1295 non-immediate duplicates within the range. The CmdSN carried by 1296 immediate commands may lie outside the ExpCmdSN to MaxCmdSN range. 1297 For example, if the initiator has previously sent a non-immediate 1298 command carrying the CmdSN equal to MaxCmdSN, the target window is 1299 closed. For group task management commands issued as immediate 1300 commands, CmdSN indicates the scope of the group action (e.g., on 1301 ABORT TASK SET indicates which commands are to be aborted). 1303 MaxCmdSN and ExpCmdSN fields are processed by the initiator as 1304 follows: 1306 -If the PDU MaxCmdSN is less than the PDU ExpCmdSN-1 (in 1307 Serial Arithmetic Sense), they are both ignored. 1309 -If the PDU MaxCmdSN is greater than the local MaxCmdSN (in 1310 Serial Arithmetic Sense), it updates the local MaxCmdSN; 1311 otherwise, it is ignored. 1313 -If the PDU ExpCmdSN is greater than the local ExpCmdSN (in 1314 Serial Arithmetic Sense), it updates the local ExpCmdSN; 1315 otherwise, it is ignored. 1317 This sequence is required because updates may arrive out of order 1318 (e.g., the updates are sent on different TCP connections). 1320 iSCSI initiators and targets MUST support the command numbering 1321 scheme. 1323 A numbered iSCSI request will not change its allocated CmdSN, 1324 regardless of the number of times and circumstances in which it is 1325 reissued (see Section 7.2.1). At the target, CmdSN is only 1326 relevant while the command has not created any state related to 1327 its execution (execution state); afterwards, CmdSN becomes 1328 irrelevant. Testing for the execution state (represented by 1329 identifying the Initiator Task Tag) MUST precede any other action 1330 at the target. If no execution state is found, it is followed by 1331 ordering and delivery. If an execution state is found, it is 1332 followed by delivery if it has not already been delivered. 1334 If an initiator issues a command retry for a command with CmdSN R 1335 on a connection when the session CmdSN value is Q, it MUST NOT 1336 advance the CmdSN past R + 2**31 -1 unless the connection is no 1337 longer operational (i.e., it has returned to the FREE state, see 1338 Section 8.1.3), the connection has been reinstated (see Section 1339 6.3.4), or a non-immediate command with CmdSN equal or greater 1340 than Q was issued subsequent to the command retry on the same 1341 connection and the reception of that command is acknowledged by 1342 the target (see Section 10.4). 1344 A target command response or Data-in PDU with status MUST NOT 1345 precede the command acknowledgement. However, the acknowledgement 1346 MAY be included in the response or the Data-in PDU. 1348 4.2.2.2. Response/Status Numbering and Acknowledging 1350 Responses in transit from the target to the initiator are 1351 numbered. The StatSN (Status Sequence Number) is used for this 1352 purpose. StatSN is a counter maintained per connection. ExpStatSN 1353 is used by the initiator to acknowledge status. The status 1354 sequence number space is 32-bit unsigned-integers and the 1355 arithmetic operations are the regular mod(2**32) arithmetic. 1357 Status numbering starts with the Login response to the first Login 1358 request of the connection. The Login response includes an initial 1359 value for status numbering (any initial value is valid). 1361 To enable command recovery, the target MAY maintain enough state 1362 information for data and status recovery after a connection 1363 failure. A target doing so can safely discard all of the state 1364 information maintained for recovery of a command after the 1365 delivery of the status for the command (numbered StatSN) is 1366 acknowledged through ExpStatSN. 1368 A large absolute difference between StatSN and ExpStatSN may 1369 indicate a failed connection. Initiators MUST undertake recovery 1370 actions if the difference is greater than an implementation 1371 defined constant that MUST NOT exceed 2**31-1. 1373 Initiators and Targets MUST support the response-numbering scheme. 1375 4.2.2.3. Response Ordering 1377 4.2.2.3.1. Need for Response Ordering 1379 Whenever an iSCSI session is composed of multiple connections, the 1380 Response PDUs (task responses or TMF responses) originating in 1381 the target SCSI layer are distributed onto the multiple 1382 connections by the target iSCSI layer according to iSCSI 1383 connection allegiance rules. This process generally may not 1384 preserve the ordering of the responses by the time they are 1385 delivered to the initiator SCSI layer. 1387 Since ordering is not expected across SCSI responses anyway, this 1388 approach works fine in the general case. However, to address the 1389 special cases where some ordering is desired by the SCSI layer, we 1390 introduce the notion of a "Response Fence": Response Fence is 1391 logically the attribute/property of a SCSI response message handed 1392 off to a target iSCSI layer which indicates that there are special 1393 SCSI-level ordering considerations associated with this particular 1394 response message - whenever Response Fence is set or required on a 1395 SCSI response message, we define the semantics in 4.2.2.3.2 with 1396 respect to target iSCSI layer's handling of such SCSI response 1397 messages. 1399 4.2.2.3.2. Response Ordering Model Description 1401 The target SCSI protocol layer hands off the SCSI response 1402 messages to the target iSCSI layer by invoking the "Send Command 1403 Complete" protocol data service ([SAM2], clause 5.4.2) and "Task 1404 Management Function Executed" ([SAM2], clause 6.9) service. On 1405 receiving the SCSI response message, the iSCSI layer exhibits the 1406 Response Fence behavior for certain SCSI response messages 1407 (Section 4.2.2.3.4 describes the specific instances where the 1408 semantics must be realized). 1410 Whenever the Response Fence behavior is required for a SCSI 1411 response message, the target iSCSI layer MUST ensure that the 1412 following conditions are met in delivering the response message to 1413 the initiator iSCSI layer: 1415 - Response with Response Fence MUST be delivered 1416 chronologically after all the "preceding" responses on the 1417 I_T_L nexus, if the preceding responses are delivered at 1418 all, to the initiator iSCSI layer. 1420 - Response with Response Fence MUST be delivered 1421 chronologically prior to all the "following" responses on 1422 the I_T_L nexus. 1424 The "preceding" and "following" notions refer to the order of 1425 handoff of a response message from the target SCSI protocol layer 1426 to the target iSCSI layer. 1428 4.2.2.3.3. iSCSI Semantics with the Interface Model 1430 Whenever the TaskReporting key (Section 13.23) is negotiated to 1431 ResponseFence or FastAbort for an iSCSI session and the Response 1432 Fence behavior is required for a SCSI response message, the target 1433 iSCSI layer MUST perform the actions described in this Section for 1434 that session. 1436 a) If it is a single-connection session, no special 1437 processing is required. The standard SCSI Response PDU 1438 build and dispatch process happens. 1439 b) If it is a multi-connection session, the target iSCSI 1440 layer takes note of the last-sent and unacknowledged 1441 StatSN on each of the connections in the iSCSI session, 1442 and waits for an acknowledgement (NOP-In PDUs MAY be used 1443 to solicit acknowledgements as needed in order to 1444 accelerate this process) of each such StatSN to clear the 1445 fence. The SCSI response requiring Response Fence 1446 behavior MUST NOT be sent to the initiator before 1447 acknowledgements are received for each of the 1448 unacknowledged StatSNs. 1449 c) The target iSCSI layer must wait for an acknowledgement 1450 of the SCSI Response PDU that carried the SCSI response 1451 requiring the Response Fence behavior. The fence MUST be 1452 considered cleared only after receiving the 1453 acknowledgement. 1454 d) All further status processing for the LU is resumed only 1455 after clearing the fence. If any new responses for the 1456 I_T_L nexus are received from the SCSI layer before the 1457 fence is cleared, those Response PDUs MUST be held and 1458 queued at the iSCSI layer until the fence is cleared. 1460 4.2.2.3.4. Current List of Fenced Response Use Cases 1462 This Section lists the situations in which fenced response 1463 behavior is REQUIRED in iSCSI target implementations. Note that 1464 the following list is an exhaustive enumeration as currently 1465 identified - it is expected that as SCSI protocol specifications 1466 evolve, the specifications will specify when response fencing is 1467 required on a case-by-case basis. 1469 Whenever the TaskReporting key (Section 13.23) is negotiated to 1470 ResponseFence or FastAbort for an iSCSI session, the target iSCSI 1471 layer MUST assume that the Response Fence is required for the 1472 following SCSI completion messages: 1474 1. The first completion message carrying the UA after the 1475 multi-task abort on issuing and third-party sessions. See 1476 Section 4.2.3.2 for related TMF discussion. 1478 2. The TMF Response carrying the multi-task TMF Response on 1479 the issuing session. 1481 3. The completion message indicating ACA establishment on the 1482 issuing session. 1484 4. The first completion message carrying the ACA ACTIVE status 1485 after ACA establishment on issuing and third-party 1486 sessions. 1488 5. The TMF Response carrying the Clear ACA response on the 1489 issuing session. 1491 6. The response to a PERSISTENT RESERVE OUT/PREEMPT AND ABORT 1492 command. 1494 Note: 1495 - Due to the absence of ACA-related fencing requirements in 1496 [RFC3720], initiator implementations SHOULD NOT use ACA on 1497 multi-connection iSCSI sessions with targets complying only 1498 with [RFC3720]. This can be determined via TaskReporting key 1499 (Section 13.23) negotiation - when the negotiation results 1500 in either "RFC3720" or "NotUnderstood". 1502 - Initiators that want to employ ACA on multi-connection iSCSI 1503 sessions SHOULD first assess response-fencing behavior via 1504 negotiating for "ResponseFence" or "FastAbort" value for the 1505 TaskReporting (Section 13.23) key. 1507 4.2.2.4. Data Sequencing 1509 Data and R2T PDUs transferred as part of some command execution 1510 MUST be sequenced. The DataSN field is used for data sequencing. 1511 For input (read) data PDUs, DataSN starts with 0 for the first 1512 data PDU of an input command and advances by 1 for each subsequent 1513 data PDU. For output data PDUs, DataSN starts with 0 for the first 1514 data PDU of a sequence (the initial unsolicited sequence or any 1515 data PDU sequence issued to satisfy an R2T) and advances by 1 for 1516 each subsequent data PDU. R2Ts are also sequenced per command. For 1517 example, the first R2T has an R2TSN of 0 and advances by 1 for 1518 each subsequent R2T. For bidirectional commands, the target uses 1519 the DataSN/R2TSN to sequence Data-In and R2T PDUs in one 1520 continuous sequence (undifferentiated). Unlike command and status, 1521 data PDUs and R2Ts are not acknowledged by a field in regular 1522 outgoing PDUs. Data-In PDUs can be acknowledged on demand by a 1523 special form of the SNACK PDU. Data and R2T PDUs are implicitly 1524 acknowledged by status for the command. The DataSN/R2TSN field 1525 enables the initiator to detect missing data or R2T PDUs. 1527 For any read or bidirectional command, a target MUST issue less 1528 than 2**32 combined R2T and Data-In PDUs. Any output data sequence 1529 MUST contain less than 2**32 Data-Out PDUs. 1531 4.2.3. iSCSI Task Management 1533 4.2.3.1. Task Management Overview 1535 iSCSI task management features allow an initiator to control the 1536 active iSCSI tasks on an operational iSCSI session that it has 1537 with an iSCSI target. Section 11.5 defines the task management 1538 function types that this specification defines - ABORT TASK, ABORT 1539 TASK SET, CLEAR ACA, CLEAR TASK SET, LOGICAL UNIT RESET, TARGET 1540 WARM RESET, TARGET COLD RESET, and TASK REASSIGN. 1542 Out of these function types, ABORT TASK and TASK REASSIGN 1543 functions manage a single active task, whereas ABORT TASK SET, 1544 CLEAR TASK SET, LOGICAL UNIT RESET, TARGET WARM RESET and TARGET 1545 COLD RESET functions can each potentially affect multiple active 1546 tasks. 1548 4.2.3.2. Notion of Affected Tasks 1550 This Section defines the notion of "affected tasks" in multi-task 1551 abort scenarios. Scope definitions in this Section apply to both 1552 the Standard Multi-task Abort semantics (Section 4.2.3.3) and the 1553 FastAbort Multi-task Abort semantics behavior (Section 4.2.3.4). 1555 ABORT TASK SET: All outstanding tasks for the I_T_L nexus 1556 identified by the LUN field in the ABORT TASK SET TMF Request PDU 1557 (Section 11.5). 1559 CLEAR TASK SET: All outstanding tasks in the task set for the LU 1560 identified by the LUN field in the CLEAR TASK SET TMF Request PDU. 1561 See [SPC3] for the definition of a "task set". 1563 LOGICAL UNIT RESET: All outstanding tasks from all initiators for 1564 the LU identified by the LUN field in the LOGICAL UNIT RESET 1565 Request PDU. 1567 TARGET WARM RESET/TARGET COLD RESET: All outstanding tasks from 1568 all initiators across all LUs to which the TMF-issuing session has 1569 access on the SCSI target device hosting the iSCSI session. 1571 Usage: An "ABORT TASK SET TMF Request PDU" in the preceding text 1572 is an iSCSI TMF Request PDU with the "Function" field set to 1573 "ABORT TASK SET" as defined in Section 11.5. Similar usage is 1574 employed for other scope descriptions. 1576 4.2.3.3. Standard Multi-task Abort Semantics 1578 All iSCSI implementations MUST support the protocol behavior 1579 defined in this Section as the default behavior. The execution of 1580 ABORT TASK SET, CLEAR TASK SET, LOGICAL UNIT RESET, TARGET WARM 1581 RESET, and TARGET COLD RESET TMF Requests consists of the 1582 following sequence of actions in the specified order on the 1583 specified party. 1585 The initiator iSCSI layer: 1586 a. MUST continue to respond to each TTT received for the 1587 affected tasks. 1588 b. SHOULD process any responses received for affected tasks in 1589 the normal fashion. This is acceptable because the 1590 responses are guaranteed to have been sent prior to the TMF 1591 response. 1592 c. SHOULD receive the TMF Response concluding all the tasks in 1593 the set of affected tasks unless the initiator has done 1594 something (e.g., LU reset, connection drop) that may 1595 prevent the TMF Response from being sent or received. The 1596 initiator MUST thus conclude all affected tasks as part of 1597 this step in either case, and MUST discard any TMF Response 1598 received after the affected tasks are concluded. 1600 The target iSCSI layer: 1601 a. MUST wait for responses on currently valid target-transfer 1602 tags of the affected tasks from the issuing initiator. MAY 1603 wait for responses on currently valid target-transfer tags 1604 of the affected tasks from third-party initiators. 1605 b. MUST wait (concurrent with the wait in Step a) for all 1606 commands of the affected tasks to be received based on the 1607 CmdSN ordering. SHOULD NOT wait for new commands on third- 1608 party affected sessions -- only the instantiated tasks have 1609 to be considered for the purpose of determining the 1610 affected tasks. In the case of target-scoped requests 1611 (i.e., TARGET WARM RESET and TARGET COLD RESET), all of the 1612 commands that are not yet received on the issuing session 1613 in the command stream however can be considered to have 1614 been received with no command waiting period -- i.e., the 1615 entire CmdSN space up to the CmdSN of the task management 1616 function can be "plugged". 1617 c. MUST propagate the TMF request to and receive the response 1618 from the target SCSI layer. 1619 d. MUST provide the Response Fence behavior for the TMF 1620 Response on the issuing session as specified in Section 1621 4.2.2.3.2. 1622 e. MUST provide the Response Fence behavior on the first post- 1623 TMF Response on third-party sessions as specified in 1624 Section 4.2.2.3.3. If some tasks originate from non-iSCSI 1625 I_T_L nexuses, then the means by which the target ensures 1626 that all affected tasks have returned their status to the 1627 initiator are defined by the specific non-iSCSI transport 1628 protocol(s). 1630 Technically, the TMF servicing is complete in Step d. Data 1631 transfers corresponding to terminated tasks may however still be 1632 in progress on third-party iSCSI sessions even at the end of Step 1633 e. The TMF Response MUST NOT be sent by the target iSCSI layer 1634 before the end of Step d, and MAY be sent at the end of Step d 1635 despite these outstanding data transfers until after Step e. 1637 4.2.3.4. FastAbort Multi-task Abort Semantics 1639 Protocol behavior defined in this Section SHOULD be implemented by 1640 all iSCSI implementations complying with this document, noting 1641 that some steps below may not be compatible with [RFC3720] 1642 semantics. However, protocol behavior defined in this Section MUST 1643 be exhibited by iSCSI implementations on an iSCSI session when 1644 they negotiate the TaskReporting (Section 13.23) key to 1645 "FastAbort" on that session. The execution of ABORT TASK SET, 1646 CLEAR TASK SET, LOGICAL UNIT RESET, TARGET WARM RESET, and TARGET 1647 COLD RESET TMF Requests consists of the following sequence of 1648 actions in the specified order on the specified party. 1650 The initiator iSCSI layer: 1652 a. MUST NOT send any more Data-Out PDUs for affected tasks on 1653 the issuing connection of the issuing iSCSI session once 1654 the TMF is sent to the target. 1655 b. SHOULD process any responses received for affected tasks in 1656 the normal fashion. This is acceptable because the 1657 responses are guaranteed to have been sent prior to the TMF 1658 response. 1659 c. MUST respond to each Async Message PDU with FAST_ABORT 1660 AsyncEvent as defined in Section 11.9. 1661 d. MUST treat the TMF response as terminating all affected 1662 tasks for which responses have not been received, and MUST 1663 discard any responses for affected tasks received after the 1664 TMF response is passed to the SCSI layer (although the 1665 semantics defined in this Section ensure that such an out- 1666 of-order scenario will never happen with a compliant target 1667 implementation). 1669 The target iSCSI layer: 1670 a. MUST wait for all commands of the affected tasks to be 1671 received based on the CmdSN ordering on the issuing 1672 session. SHOULD NOT wait for new commands on third-party 1673 affected sessions - only the instantiated tasks have to be 1674 considered for the purpose of determining the affected 1675 tasks. In the case of target-scoped requests (i.e., TARGET 1676 WARM RESET and TARGET COLD RESET), all the commands that 1677 are not yet received on the issuing session in the command 1678 stream can be considered to have been received with no 1679 command waiting period -- i.e., the entire CmdSN space up 1680 to the CmdSN of the task management function can be 1681 "plugged". 1682 b. MUST propagate the TMF request to and receive the response 1683 from the target SCSI layer. 1684 c. MUST leave all active "affected TTTs" (i.e., active TTTs 1685 associated with affected tasks) valid. 1686 d. MUST send an Asynchronous Message PDU with AsyncEvent=5 1687 (Section 11.9) on: 1688 i) each connection of each third-party session to 1689 which at least one affected task is allegiant if 1690 TaskReporting=FastAbort is operational on that third- 1691 party session, and, 1693 ii) each connection except the issuing connection of 1694 the issuing session that has at least one allegiant 1695 affected task. 1696 If there are multiple affected LUs (say, due to a target 1697 reset), then one Async Message PDU MUST be sent for each 1698 such LU on each connection that has at least one allegiant 1699 affected task. The LUN field in the Asynchronous Message PDU 1700 MUST be set to match the LUN for each such LU. 1701 e. MUST address the Response Fence flag on the TMF Response on 1702 the issuing session as defined in Section 4.2.2.3.3. 1703 f. MUST address the Response Fence flag on the first post-TMF 1704 Response on third-party sessions as defined in Section 1705 4.2.2.3.3. If some tasks originate from non-iSCSI I_T_L 1706 nexuses, then the means by which the target ensures that 1707 all affected tasks have returned their status to the 1708 initiator are defined by the specific non-iSCSI transport 1709 protocol(s). 1710 g. MUST free up the affected TTTs (and STags, if applicable) 1711 and the corresponding buffers, if any, once it receives 1712 each associated NOP-Out acknowledgement that the initiator 1713 generated in response to each Async Message. 1715 Technically, the TMF servicing is complete in Step e. Data 1716 transfers corresponding to terminated tasks may however still be 1717 in progress even at the end of Step f. A TMF Response MUST NOT be 1718 sent by the target iSCSI layer before the end of Step e, and MAY 1719 be sent at the end of Step e despite these outstanding Data 1720 transfers until Step g. Step g specifies an event to free up any 1721 such resources that may have been reserved to support outstanding 1722 data transfers. 1724 4.2.3.5. Affected Tasks Shared across Standard and FastAbort Sessions 1726 If an iSCSI target implementation is capable of supporting 1727 TaskReporting=FastAbort functionality (Section 13.23), it may end 1728 up in a situation where some sessions have TaskReporting=RFC3720 1729 operational (RFC 3720 sessions) while some other sessions have 1730 TaskReporting=FastAbort operational (FastAbort sessions) even 1731 while accessing a shared set of affected tasks (Section 4.2.3.2). 1732 If the issuing session is an RFC 3720 session, the iSCSI target 1733 implementation is FastAbort-capable, and the third-party affected 1734 session is a FastAbort session, the following behavior SHOULD be 1735 exhibited by the iSCSI target layer: 1736 a. Between Steps c and d of the target behavior in Section 1737 4.2.3.3, send an Asynchronous Message PDU with AsyncEvent=5 1738 (Section 11.9) on each connection of each third-party 1739 session to which at least one affected task is allegiant. 1740 If there are multiple affected LUs, then send one Async 1741 Message PDU for each such LU on each connection that has at 1742 least one allegiant affected task. When sent, the LUN field 1743 in the Asynchronous Message PDU MUST be set to match the 1744 LUN for each such LU. 1745 b. After Step e of the target behavior in Section 4.2.3.3, 1746 free up the affected TTTs (and STags, if applicable) and 1747 the corresponding buffers, if any, once each associated 1748 NOP-Out acknowledgement is received that the third-party 1749 initiator generated in response to each Async Message sent 1750 in Step a. 1752 If the issuing session is a FastAbort session, the iSCSI target 1753 implementation is FastAbort-capable, and the third-party affected 1754 session is an RFC 3720 session, iSCSI target layer MUST NOT send 1755 Asynchronous Message PDUs on the third-party session to prompt the 1756 FastAbort behavior. 1758 If the third-party affected session is a FastAbort session and the 1759 issuing session is a FastAbort session, the initiator in the 1760 third-party role MUST respond to each Async Message PDU with 1761 AsyncEvent=5 as defined in Section 11.9. Note that an initiator 1762 MAY thus receive these Async Messages on a third-party affected 1763 session even if the session is a single-connection session. 1765 4.2.3.6. Rationale behind the FastAbort Semantics 1767 There are fundamentally three basic objectives behind the 1768 semantics 1769 specified in Sections 4.2.3.3 and 4.2.3.4. 1770 1. Maintaining an ordered command flow I_T nexus abstraction 1771 to the target SCSI layer even with multi-connection 1772 sessions. 1773 - Target iSCSI processing of a TMF request must 1774 maintain the single flow illusion. Target behavior in 1776 Step b of Section 4.2.3.3 and Step a of Section 4.2.3.4 1777 correspond to this objective. 1778 2. Maintaining a single ordered response flow I_T nexus 1779 abstraction to the initiator SCSI layer even with multi- 1780 connection sessions when one response (i.e., TMF response) 1781 could imply the status of other unfinished tasks from the 1782 initiator's perspective. 1783 - The target must ensure that the initiator does not 1784 see "old" task responses (that were placed on the wire 1785 chronologically earlier than the TMF Response) after 1786 seeing the TMF response. The target behavior in Step d 1787 of Section 4.2.3.3 and Step e of Section 4.2.3.4 1788 correspond to this objective. 1789 - Whenever the result of a TMF action is visible across 1790 multiple I_T_L nexuses, [SAM2] requires the SCSI device 1791 server to trigger a UA on each of the other I_T_L 1792 nexuses. Once an initiator is notified of such an UA, 1793 the application client on the receiving initiator is 1794 required to clear its task state (clause 5.5 in [SAM2]) 1795 for the affected tasks. It would thus be inappropriate 1796 to deliver a SCSI Response for a task after the task 1797 state is cleared on the initiator, i.e., after the UA 1798 is notified. The UA notification contained in the first 1799 SCSI Response PDU on each affected Third-party I_T_L 1800 nexus after the TMF action thus MUST NOT pass the 1801 affected task responses on any of the iSCSI sessions 1802 accessing the LU. The target behavior in Step e of 1803 Section 4.2.3.3 and Step f of Section 4.2.3.4 1804 correspond to this objective. 1805 3. Draining all active TTTs corresponding to affected tasks in 1806 a deterministic fashion. 1807 - Data-Out PDUs with stale TTTs arriving after the 1808 tasks are terminated can create a buffer management 1809 problem even for traditional iSCSI implementations, and 1810 is fatal for the connection for iSCSI/iSER 1811 implementations. Either the termination of affected 1812 tasks should be postponed until the TTTs are retired 1813 (as in Step a of Section 4.2.3.3), or the TTTs and the 1814 buffers should stay allocated beyond task termination 1815 to be deterministically freed up later (as in Steps c 1816 and g of Section 4.2.3.4). 1818 The only other notable optimization is the plugging. If all tasks 1819 on an I_T nexus will be aborted anyway (as with a target reset), 1820 there is no need to wait to receive all commands to plug the CmdSN 1821 holes. The target iSCSI layer can simply plug all missing CmdSN 1822 slots and move on with TMF processing. The first objective 1823 (maintaining a single ordered command flow) is still met with this 1824 optimization because the target SCSI layer only sees ordered 1825 commands. 1827 4.2.4. iSCSI Login 1829 The purpose of the iSCSI login is to enable a TCP connection for 1830 iSCSI use, authentication of the parties, negotiation of the 1831 session's parameters and marking of the connection as belonging to 1832 an iSCSI session. 1834 A session is used to identify to a target all the connections with 1835 a given initiator that belong to the same I_T nexus. (For more 1836 details on how a session relates to an I_T nexus, see Section 1837 4.4.2). 1839 The targets listen on a well-known TCP port or other TCP port for 1840 incoming connections. The initiator begins the login process by 1841 connecting to one of these TCP ports. 1843 As part of the login process, the initiator and target SHOULD 1844 authenticate each other and MAY set a security association 1845 protocol for the session. This can occur in many different ways 1846 and is subject to negotiation - see Section 12. 1848 To protect the TCP connection, an IPsec security association MAY 1849 be established before the Login request. For information on using 1850 IPsec security for iSCSI see Section 9, [RFC3723] and [IPSEC-IPS}. 1852 The iSCSI Login Phase is carried through Login requests and 1853 responses. Once suitable authentication has occurred and 1854 operational parameters have been set, the session transitions to 1855 Full Feature Phase and the initiator may start to send SCSI 1856 commands. The security policy for whether, and by what means, a 1858 target chooses to authorize an initiator is beyond the scope of 1859 this document. For a more detailed description of the Login Phase, 1860 see Section 6. 1862 The login PDU includes the ISID part of the session ID (SSID). The 1863 target portal group that services the login is implied by the 1864 selection of the connection endpoint. For a new session, the TSIH 1865 is zero. As part of the response, the target generates a TSIH. 1867 During session establishment, the target identifies the SCSI 1868 initiator port (the "I" in the "I_T nexus") through the value pair 1869 (InitiatorName, ISID). We describe InitiatorName later in this 1870 Section. Any persistent state (e.g., persistent reservations) on 1871 the target that is associated with a SCSI initiator port is 1872 identified based on this value pair. Any state associated with the 1873 SCSI target port (the "T" in the "I_T nexus") is identified 1874 externally by the TargetName and portal group tag (see Section 1875 4.4.1). ISID is subject to reuse restrictions because it is used 1876 to identify a persistent state (see Section 4.4.3). 1878 Before the Full Feature Phase is established, only Login Request 1879 and Login Response PDUs are allowed. Login requests and responses 1880 MUST be used exclusively during Login. On any connection, the 1881 login phase MUST immediately follow TCP connection establishment 1882 and a subsequent Login Phase MUST NOT occur before tearing down a 1883 connection. 1885 A target receiving any PDU except a Login request before the Login 1886 phase is started MUST immediately terminate the connection on 1887 which the PDU was received. Once the Login phase has started, if 1888 the target receives any PDU except a Login request, it MUST send a 1889 Login reject (with Status "invalid during login") and then 1890 disconnect. If the initiator receives any PDU except a Login 1891 response, it MUST immediately terminate the connection. 1893 4.2.5. iSCSI Full Feature Phase 1895 Once the two sides successfully conclude the Login on the first - 1896 also called the leading - connection in the session, the iSCSI 1897 session is in the iSCSI Full Feature Phase. A connection is in 1898 Full Feature Phase if the session is in Full Feature Phase and the 1899 connection login has completed successfully. An iSCSI connection 1900 is not in Full Feature Phase 1902 a) when it does not have an established transport 1903 connection, 1904 OR 1905 b) when it has a valid transport connection, but a 1906 successful login was not performed or the connection is 1907 currently logged out. 1909 In a normal Full Feature Phase, the initiator may send SCSI 1910 commands and data to the various LUs on the target by 1911 encapsulating them in iSCSI PDUs that go over the established 1912 iSCSI session. 1914 4.2.5.1. Command Connection Allegiance 1916 For any iSCSI request issued over a TCP connection, the 1917 corresponding response and/or other related PDU(s) MUST be sent 1918 over the same connection. We call this "connection allegiance". If 1919 the original connection fails before the command is completed, the 1920 connection allegiance of the command may be explicitly reassigned 1921 to a different transport connection as described in detail in 1922 Section 7.2. 1924 Thus, if an initiator issues a READ command, the target MUST send 1925 the requested data, if any, followed by the status to the 1926 initiator over the same TCP connection that was used to deliver 1927 the SCSI command. If an initiator issues a WRITE command, the 1928 initiator MUST send the data, if any, for that command over the 1929 same TCP connection that was used to deliver the SCSI command. The 1930 target MUST return Ready To Transfer (R2T), if any, and the status 1931 over the same TCP connection that was used to deliver the SCSI 1932 command. Retransmission requests (SNACK PDUs) and the data and 1933 status that they generate MUST also use the same connection. 1935 However, consecutive commands that are part of a SCSI linked 1936 command-chain task (see [SAM2]) MAY use different connections. 1937 Connection allegiance is strictly per-command and not per-task. 1938 During the iSCSI Full Feature Phase, the initiator and target MAY 1939 interleave unrelated SCSI commands, their SCSI Data, and responses 1940 over the session. 1942 4.2.5.2. Data Transfer Overview 1944 Outgoing SCSI data (initiator to target user data or command 1945 parameters) is sent as either solicited data or unsolicited data. 1946 Solicited data are sent in response to R2T PDUs. Unsolicited data 1947 can be sent as part of an iSCSI command PDU ("immediate data") or 1948 in separate iSCSI data PDUs. 1950 Immediate data are assumed to originate at offset 0 in the 1951 initiator SCSI write-buffer (outgoing data buffer). All other Data 1952 PDUs have the buffer offset set explicitly in the PDU header. 1954 An initiator may send unsolicited data up to FirstBurstLength 1955 (see Section 13.14) as immediate (up to the negotiated maximum PDU 1956 length), in a separate PDU sequence or both. All subsequent data 1957 MUST be solicited. The maximum length of an individual data PDU or 1958 the immediate-part of the first unsolicited burst MAY be 1959 negotiated at login. 1961 The maximum amount of unsolicited data that can be sent with a 1962 command is negotiated at login through the FirstBurstLength (see 1963 Section 13.14) key. A target MAY separately enable immediate data 1964 (through the ImmediateData key) without enabling the more general 1965 (separate data PDUs) form of unsolicited data (through the 1966 InitialR2T key). 1968 Unsolicited data on write are meant to reduce the effect of 1969 latency on throughput (no R2T is needed to start sending data). In 1970 addition, immediate data is meant to reduce the protocol overhead 1971 (both bandwidth and execution time). 1973 An iSCSI initiator MAY choose not to send unsolicited data, only 1974 immediate data or FirstBurstLength bytes of unsolicited data with 1975 a command. If any non-immediate unsolicited data is sent, the 1976 total unsolicited data MUST be either FirstBurstLength, or all of 1977 the data if the total amount is less than the FirstBurstLength. 1979 It is considered an error for an initiator to send unsolicited 1980 data PDUs to a target that operates in R2T mode (only solicited 1981 data are allowed). It is also an error for an initiator to send 1982 more unsolicited data, whether immediate or as separate PDUs, than 1983 FirstBurstLength. 1985 An initiator MUST honor an R2T data request for a valid 1986 outstanding command (i.e., carrying a valid Initiator Task Tag) 1987 and deliver all the requested data provided the command is 1988 supposed to deliver outgoing data and the R2T specifies data 1989 within the command bounds. The initiator action is unspecified for 1990 receiving an R2T request that specifies data, all or part, outside 1991 of the bounds of the command. 1993 A target SHOULD NOT silently discard data and then request 1994 retransmission through R2T. Initiators SHOULD NOT keep track of 1995 the data transferred to or from the target (scoreboarding). SCSI 1996 targets perform residual count calculation to check how much data 1997 was actually transferred to or from the device by a command. This 1998 may differ from the amount the initiator sent and/or received for 1999 reasons such as retransmissions and errors. Read or bidirectional 2000 commands implicitly solicit the transmission of the entire amount 2001 of data covered by the command. SCSI data packets are matched to 2002 their corresponding SCSI commands by using tags specified in the 2003 protocol. 2005 In addition, iSCSI initiators and targets MUST enforce some 2006 ordering rules. When unsolicited data is used, the order of the 2007 unsolicited data on each connection MUST match the order in which 2008 the commands on that connection are sent. Command and unsolicited 2009 data PDUs may be interleaved on a single connection as long as the 2010 ordering requirements of each are maintained (e.g., command N+1 2011 MAY be sent before the unsolicited Data-Out PDUs for command N, 2012 but the unsolicited Data-Out PDUs for command N MUST precede the 2013 unsolicited Data-Out PDUs of command N+1). A target that receives 2014 data out of order MAY terminate the session. 2016 4.2.5.3. Tags and Integrity Checks 2018 Initiator tags for pending commands are unique initiator-wide for 2019 a session. Target tags are not strictly specified by the protocol. 2020 It is assumed that target tags are used by the target to tag 2021 (alone or in combination with the LUN) the solicited data. Target 2022 tags are generated by the target and "echoed" by the initiator. 2024 These mechanisms are designed to accomplish efficient data 2025 delivery along with a large degree of control over the data flow. 2027 As the Initiator Task Tag is used to identify a task during its 2028 execution the iSCSI initiator and target MUST verify that all 2029 other fields used in task related PDUs have values that are 2030 consistent with the values used at the task instantiation based on 2031 Initiator Task Tag (e.g., the LUN used in an R2T PDU MUST be the 2032 same as the one used in the SCSI command PDU used to instantiate 2033 the task). Using inconsistent field values is considered a 2034 protocol error. 2036 4.2.5.4. Task Management 2038 SCSI task management assumes that individual tasks and task groups 2039 can be aborted solely based on the task tags (for individual 2040 tasks) or the timing of the task management command (for task 2041 groups) and that the task management action is executed 2042 synchronously - i.e, no message involving an aborted task will be 2043 seen by the SCSI initiator after receiving the task management 2044 response. In iSCSI initiators and targets interact asynchronously 2045 over several connections. iSCSI specifies the protocol mechanism 2046 and implementation requirements needed to present a synchronous 2047 view while using an asynchronous infrastructure. 2049 4.2.6. iSCSI Connection Termination 2051 An iSCSI connection may be terminated by use of a transport 2052 connection shutdown or a transport reset. Transport reset is 2053 assumed to be an exceptional event. 2055 Graceful TCP connection shutdowns are done by sending TCP FINs. A 2056 graceful transport connection shutdown SHOULD only be initiated by 2057 either party when the connection is not in iSCSI Full Feature 2058 Phase. A target MAY terminate a Full Feature Phase connection on 2059 internal exception events, but it SHOULD announce the fact through 2060 an Asynchronous Message PDU. Connection termination with 2061 outstanding commands may require recovery actions. 2063 If a connection is terminated while in Full Feature Phase, 2064 connection cleanup (see Section 7) is required prior to recovery. 2065 By doing connection cleanup before starting recovery, the 2066 initiator and target will avoid receiving stale PDUs after 2067 recovery. 2069 4.2.7. iSCSI Names 2071 Both targets and initiators require names for the purpose of 2072 identification. In addition, names enable iSCSI storage resources 2073 to be managed regardless of location (address). An iSCSI node name 2074 is also the SCSI device name contained in the iSCSI Node. The 2075 iSCSI name of a SCSI device is the principal object used in 2076 authentication of targets to initiators and initiators to targets. 2077 This name is also used to identify and manage iSCSI storage 2078 resources. 2080 iSCSI names must be unique within the operation domain of the end 2081 user. However, because the operation domain of an IP network is 2082 potentially worldwide, the iSCSI name formats are architected to 2083 be worldwide unique. To assist naming authorities in the 2084 construction of worldwide unique names, iSCSI provides three name 2085 formats for different types of naming authorities. 2087 iSCSI names are associated with iSCSI nodes, and not iSCSI network 2088 adapter cards, to ensure that the replacement of network adapter 2089 cards does not require reconfiguration of all SCSI and iSCSI 2090 resource allocation information. 2092 Some SCSI commands require that protocol-specific identifiers be 2093 communicated within SCSI CDBs. See SCSI for the definition of the 2094 SCSI port name/identifier for iSCSI ports. 2096 An initiator may discover the iSCSI Target Names to which it has 2097 access, along with their addresses, using the SendTargets text 2098 request, or other techniques discussed in [RFC3721]. 2100 iSCSI equipment that needs discovery functions beyond SendTargets 2101 SHOULD implement iSNS (see [RFC4171]) for extended discovery 2102 management capabilities and interoperability. Although [RFC3721] 2103 implies an SLP ([RFC2608]) implementation requirement, SLP has not 2104 been widely implemented or deployed for use with iSCSI in 2105 practice. iSCSI implementations therefore SHOULD NOT rely on SLP- 2106 based discovery interoperability. 2108 4.2.7.1. iSCSI Name Properties 2110 Each iSCSI node, whether it is an initiator or target or both, 2111 MUST have an iSCSI name. Whenever an iSCSI Node contains an iSCSI 2112 Initiator Node and an iSCSI Target Node, the iSCSI Initiator Name 2113 MUST be the same as the iSCSI Target Name for the contained Nodes 2114 such that there is only one iSCSI Node Name for the iSCSI Node 2115 overall. Note the related requirements in Section 9.2.1 on how to 2116 map CHAP names to iSCSI Names in such a scenario. 2118 Initiators and targets MUST support the receipt of iSCSI names of 2119 up to the maximum length of 223 bytes. 2121 The initiator MUST present both its iSCSI Initiator Name and the 2122 iSCSI Target Name to which it wishes to connect in the first login 2123 request of a new session or connection. The only exception is if a 2124 discovery session (see Section 4.3) is to be established. In this 2125 case, the iSCSI Initiator Name is still required, but the iSCSI 2126 Target Name MAY be omitted. 2128 iSCSI names have the following properties: 2130 - iSCSI names are globally unique. No two initiators or 2131 targets can have the same name. 2133 - iSCSI names are permanent. An iSCSI initiator node or target 2134 node has the same name for its lifetime. 2136 - iSCSI names do not imply a location or address. An iSCSI 2137 initiator or target can move, or have multiple addresses. A 2138 change of address does not imply a change of name. 2140 - iSCSI names do not rely on a central name broker; the naming 2141 authority is distributed. 2143 - iSCSI names support integration with existing unique naming 2144 schemes. 2146 - iSCSI names rely on existing naming authorities. iSCSI does 2147 not create any new naming authority. 2149 The encoding of an iSCSI name has the following properties: 2151 - iSCSI names have the same encoding method regardless of the 2152 underlying protocols. 2154 - iSCSI names are relatively simple to compare. The algorithm 2155 for comparing two iSCSI names for equivalence does not rely 2156 on an external server. 2158 - iSCSI names are composed only of printable ASCII and Unicode 2159 characters. iSCSI names allow the use of international 2160 character sets but uppercase characters are prohibited. The 2161 iSCSI stringprep profile [RFC3722] maps uppercase characters 2162 to lowercase and SHOULD be used to prepare iSCSI names from 2163 input that may include uppercase characters. No whitespace 2164 characters are used in iSCSI names, see [RFC3722] for 2165 details. 2167 - iSCSI names may be transported using both binary and ASCII- 2168 based protocols. 2170 An iSCSI name really names a logical software entity, and is not 2171 tied to a port or other hardware that can be changed. For 2172 instance, an initiator name should name the iSCSI initiator node, 2173 not a particular NIC or HBA. When multiple NICs are used, they 2174 should generally all present the same iSCSI initiator name to the 2175 targets, because they are simply paths to the same SCSI layer. In 2176 most operating systems, the named entity is the operating system 2177 image. 2179 Similarly, a target name should not be tied to hardware interfaces 2180 that can be changed. A target name should identify the logical 2181 target and must be the same for the target regardless of the 2182 physical portion being addressed. This assists iSCSI initiators in 2183 determining that the two targets it has discovered are really two 2184 paths to the same target. 2186 The iSCSI name is designed to fulfill the functional requirements 2187 for Uniform Resource Names (URN) [RFC1737]. For example, it is 2188 required that the name have a global scope, be independent of 2189 address or location, and be persistent and globally unique. Names 2190 must be extensible and scalable with the use of naming 2191 authorities. The name encoding should be both human and machine 2192 readable. See [RFC1737] for further requirements. 2194 4.2.7.2. iSCSI Name Encoding 2196 An iSCSI name MUST be a UTF-8 (see [RFC3629]) encoding of a string 2197 of Unicode characters with the following properties: 2199 - It is in Normalization Form C (see "Unicode Normalization 2200 Forms" [UNICODE]). 2202 - It only contains characters allowed by the output of the 2203 iSCSI stringprep template (described in [RFC3722]). 2205 - The following characters are used for formatting iSCSI 2206 names: 2208 - dash ('-'=U+002d) 2209 - dot ('.'=U+002e) 2210 - colon (':'=U+003a) 2212 - The UTF-8 encoding of the name is not larger than 223 bytes. 2214 The stringprep process is described in [RFC3454]; iSCSI's use of 2215 the stringprep process is described in [RFC3722]. Stringprep is a 2216 method designed by the Internationalized Domain Name (IDN) working 2217 group to translate human-typed strings into a format that can be 2218 compared as opaque strings. iSCSI names are expected to be used by 2219 administrators for purposes such as system configuration - for 2220 this reason, characters that may lead to human confusion among 2221 different iSCSI names (e.g., punctuation, spacing, diacritical 2222 marks) should be avoided, even when such characters are allowed as 2223 stringprep processing output by [RFC3722]. The stringprep process 2224 also converts strings into equivalent strings of lower-case 2225 characters. 2227 The stringprep process does not need to be implemented if the 2228 names are generated using only characters allowed as output by the 2229 stringprep processing specified in [RFC3722]. Those allowed 2230 characters include all ASCII lowercase and numeric characters, as 2231 well as lowercase Unicode characters as specified in [RFC3722]. 2232 Once iSCSI names encoded in UTF-8 are "normalized" as described in 2233 this Section, they may be safely compared byte-for-byte. 2235 4.2.7.3. iSCSI Name Structure 2237 An iSCSI name consists of two parts--a type designator followed by 2238 a unique name string. 2240 iSCSI uses three existing naming authorities in constructing 2241 globally unique iSCSI names. Type designator in an iSCSI name 2242 indicates the naming authority on which the name is based. The 2243 three iSCSI name formats are the following: 2245 a) iSCSI-Qualified Name: it is based on domain names to 2246 identify a naming authority, 2247 b) NAA format Name: it is based on a naming format defined 2248 by [FC-FS3] for constructing globally unique identifiers, 2249 referred to as the Network Address Authority (NAA), and, 2250 c) EUI format Name: it is based on EUI names where the IEEE 2251 Registration Authority assists in the formation of 2252 worldwide unique names (EUI-64 format). 2254 The corresponding type designator strings currently defined are: 2256 a) iqn. - iSCSI Qualified name 2258 b) naa. - Remainder of the string is an INCITS T11-defined 2259 Network Address Authority identifier, in ASCII-encoded 2260 hexadecimal. 2262 c) eui. - Remainder of the string is an IEEE EUI-64 2263 identifier, in ASCII-encoded hexadecimal. 2265 These three naming authority designators were considered 2266 sufficient at the time of writing this document. The creation of 2267 additional naming type designators for iSCSI may be considered by 2268 the IETF and detailed in separate RFCs. 2270 The following table summarizes the current SCSI transport 2271 protocols and their naming formats. 2273 SCSI Transport Protocol Naming Format 2274 +----------------------------+-------+-----+----+ 2275 | | EUI-64| NAA |IQN | 2276 |----------------------------|-------|-----|----| 2277 | iSCSI (Internet SCSI) | X | X | X | 2278 |----------------------------|-------|-----|----| 2279 | FCP (Fibre Channel) | | X | | 2280 |----------------------------|-------|-----|----| 2281 | SAS (Serial Attached SCSI) | | X | | 2282 +----------------------------+-------+-----+----+ 2284 4.2.7.4. Type "iqn." (iSCSI Qualified Name) 2286 This iSCSI name type can be used by any organization that owns a 2287 domain name. This naming format is useful when an end user or 2288 service provider wishes to assign iSCSI names for targets and/or 2289 initiators. 2291 To generate names of this type, the person or organization 2292 generating the name must own a registered domain name. This domain 2293 name does not have to resolve to an address; it just needs to be 2294 reserved to prevent others from generating iSCSI names using the 2295 same domain name. 2297 Since a domain name can expire, be acquired by another entity, or 2298 may be used to generate iSCSI names by both owners, the domain 2299 name must be additionally qualified by a date during which the 2300 naming authority owned the domain name. A date code is provided as 2301 part of the "iqn." format for this reason. 2303 The iSCSI qualified name string consists of: 2305 - The string "iqn.", used to distinguish these names from 2306 "eui." formatted names. 2308 - A date code, in yyyy-mm format. This date MUST be a date 2309 during which the naming authority owned the domain name used 2310 in this format, and SHOULD be the first month in which the 2311 domain name was owned by this naming authority at 00:01 GMT 2312 of the first day of the month. This date code uses the 2313 Gregorian calendar. All four digits in the year must be 2314 present. Both digits of the month must be present, with 2315 January == "01" and December == "12". The dash must be 2316 included. 2318 - A dot "." 2320 - The reversed domain name of the naming authority (person or 2321 organization) creating this iSCSI name. 2323 - An optional, colon (:) prefixed, string within the character 2324 set and length boundaries that the owner of the domain name 2325 deems appropriate. This may contain product types, serial 2326 numbers, host identifiers, or software keys (e.g, it may 2327 include colons to separate organization boundaries). With 2328 the exception of the colon prefix, the owner of the domain 2329 name can assign everything after the reversed domain name as 2330 desired. It is the responsibility of the entity that is the 2331 naming authority to ensure that the iSCSI names it assigns 2332 are worldwide unique. For example, "Example Storage Arrays, 2333 Inc.", might own the domain name "example.com". 2335 The following are examples of iSCSI qualified names that might be 2336 generated by "EXAMPLE Storage Arrays, Inc." 2338 Naming String defined by 2339 Type Date Auth "example.com" naming authority 2340 +--++-----+ +---------+ +--------------------------------+ 2341 | || | | | | | 2343 iqn.2001-04.com.example:storage:diskarrays-sn-a8675309 2344 iqn.2001-04.com.example 2345 iqn.2001-04.com.example:storage.tape1.sys1.xyz 2346 iqn.2001-04.com.example:storage.disk2.sys1.xyz 2348 4.2.7.5. Type "eui." (IEEE EUI-64 format) 2350 The IEEE Registration Authority provides a service for assigning 2351 globally unique identifiers [EUI]. The EUI-64 format is used to 2352 build a global identifier in other network protocols. For example, 2353 Fibre Channel defines a method of encoding it into a 2354 WorldWideName. For more information on registering for EUI 2355 identifiers, see [OUI]. 2357 The format is "eui." followed by an EUI-64 identifier (16 ASCII- 2358 encoded hexadecimal digits). 2360 Example iSCSI name: 2362 Type EUI-64 identifier (ASCII-encoded hexadecimal) 2364 +--++--------------+ 2366 | || | 2368 eui.02004567A425678D 2370 The IEEE EUI-64 iSCSI name format might be used when a 2371 manufacturer is already registered with the IEEE Registration 2372 Authority and uses EUI-64 formatted worldwide unique names for its 2373 products. 2375 More examples of name construction are discussed in [RFC3721]. 2377 4.2.7.6. Type "naa." - Network Address Authority 2379 The INCITS T11 Framing and Signaling Specification [FC-FS3] 2380 defines a format called the Network Address Authority (NAA) format 2381 for constructing worldwide unique identifiers that use various 2382 identifier registration authorities. This identifier format is 2383 used by the Fibre Channel and SAS SCSI transport protocols. As FC 2384 and SAS constitute a large fraction of networked SCSI ports, the 2385 NAA format is a widely used format for SCSI transports. The 2386 objective behind iSCSI supporting a direct representation of an 2387 NAA-format name is to facilitate construction of a target device 2388 name that translates easily across multiple namespaces for a SCSI 2389 storage device containing ports served by different transports. 2390 More specifically, this format allows implementations wherein one 2391 NAA identifier can be assigned as the basis for the SCSI device 2392 name for a SCSI target with both SAS ports and iSCSI ports. 2394 The iSCSI NAA naming format is "naa.", followed by an NAA 2395 identifier represented in ASCII-encoded hexadecimal digits. 2397 An example of an iSCSI name with a 64-bit NAA value follows: 2399 Type NAA identifier (ASCII-encoded hexadecimal) 2400 +--++--------------+ 2401 | || | 2402 naa.52004567BA64678D 2404 An example of an iSCSI name with a 128-bit NAA value follows: 2406 Type NAA identifier (ASCII-encoded hexadecimal) 2407 +--++------------------------------+ 2408 | || | 2409 naa.62004567BA64678D0123456789ABCDEF 2411 The iSCSI NAA naming format might be used in an implementation 2412 when the infrastructure for generating NAA worldwide unique names 2413 is already in place because the device contains both SAS and iSCSI 2414 SCSI ports. 2416 The NAA identifier formatted in an ASCII-hexadecimal 2417 representation has a maximum size of 32 characters (128 bit NAA 2418 format). As a result, there is no issue with this naming format 2419 exceeding the maximum size for iSCSI node names. 2421 4.2.8. Persistent State 2423 iSCSI does not require any persistent state maintenance across 2424 sessions. However, in some cases, SCSI requires persistent 2425 identification of the SCSI initiator port name (See Section 4.4.2 2426 and Section 4.4.3). 2428 iSCSI sessions do not persist through power cycles and boot 2429 operations. 2431 All iSCSI session and connection parameters are re-initialized on 2432 session and connection creation. 2434 Commands persist beyond connection termination if the session 2435 persists and command recovery within the session is supported. 2436 However, when a connection is dropped, command execution, as 2437 perceived by iSCSI (i.e., involving iSCSI protocol exchanges for 2438 the affected task), is suspended until a new allegiance is 2439 established by the 'task reassign' task management function. (See 2440 Section 11.5.) 2442 4.2.9. Message Synchronization and Steering 2444 iSCSI presents a mapping of the SCSI protocol onto TCP. This 2445 encapsulation is accomplished by sending iSCSI PDUs of varying 2446 lengths. Unfortunately, TCP does not have a built-in mechanism for 2447 signaling message boundaries at the TCP layer. iSCSI overcomes 2448 this obstacle by placing the message length in the iSCSI message 2449 header. This serves to delineate the end of the current message as 2450 well as the beginning of the next message. 2452 In situations where IP packets are delivered in order from the 2453 network, iSCSI message framing is not an issue and messages are 2454 processed one after the other. In the presence of IP packet 2455 reordering (i.e., frames being dropped), legacy TCP 2456 implementations store the "out of order" TCP segments in temporary 2457 buffers until the missing TCP segments arrive, upon which the data 2458 must be copied to the application buffers. In iSCSI, it is 2459 desirable to steer the SCSI data within these out of order TCP 2460 segments into the pre-allocated SCSI buffers rather than store 2461 them in temporary buffers. This decreases the need for dedicated 2462 reassembly buffers as well as the latency and bandwidth related to 2463 extra copies. 2465 Relying solely on the "message length" information from the iSCSI 2466 message header may make it impossible to find iSCSI message 2467 boundaries in subsequent TCP segments due to the loss of a TCP 2468 segment that contains the iSCSI message length. The missing TCP 2469 segment(s) must be received before any of the following segments 2470 can be steered to the correct SCSI buffers (due to the inability 2471 to determine the iSCSI message boundaries). Since these segments 2472 cannot be steered to the correct location, they must be saved in 2473 temporary buffers that must then be copied to the SCSI buffers. 2475 Different schemes can be used to recover synchronization. The 2476 details of any such schemes are beyond this protocol 2477 specification, but it suffices to note that [RFC4297] provides an 2478 overview of the direct data placement problem on IP networks, and 2479 [RFC5046] specifies a protocol extension for iSCSI that 2480 facilitates this direct data placement objective. Rest of this 2481 document refers to any such direct data placement protocol usage 2482 as an example of a "Synch and Steering layer". 2484 Under normal circumstances (no PDU loss or data reception out of 2485 order), iSCSI data steering can be accomplished by using the 2486 identifying tag and the data offset fields in the iSCSI header in 2487 addition to the TCP sequence number from the TCP header. The 2488 identifying tag helps associate the PDU with a SCSI buffer address 2489 while the data offset and TCP sequence number are used to 2490 determine the offset within the buffer. 2492 4.2.9.1. Sync/Steering and iSCSI PDU Length 2494 When a large iSCSI message is sent, the TCP segment(s) that 2495 contain the iSCSI header may be lost. The remaining TCP segment(s) 2496 up to the next iSCSI message must be buffered (in temporary 2497 buffers) because the iSCSI header that indicates to which SCSI 2498 buffers the data are to be steered was lost. To minimize the 2499 amount of buffering, it is recommended that the iSCSI PDU length 2500 be restricted to a small value (perhaps a few TCP segments in 2501 length). During login, each end of the iSCSI session specifies the 2502 maximum iSCSI PDU length it will accept. 2504 4.3. iSCSI Session Types 2506 iSCSI defines two types of sessions: 2508 a) Normal operational session - an unrestricted session. 2510 b) Discovery-session - a session only opened for target 2511 discovery. The target MUST ONLY accept text requests with 2512 the SendTargets key and a logout request with reason 2513 "close the session". All other requests MUST be rejected. 2515 The session type is defined during login with SessionType=value 2516 parameter in the login command. 2518 4.4. SCSI to iSCSI Concepts Mapping Model 2520 The following diagram shows an example of how multiple iSCSI Nodes 2521 (targets in this case) can coexist within the same Network Entity 2522 and can share Network Portals (IP addresses and TCP ports). Other 2523 more complex configurations are also possible. For detailed 2524 descriptions of the components of these diagrams, see Section 2525 4.4.1. 2527 +-----------------------------------+ 2528 | Network Entity (iSCSI Client) | 2529 | | 2530 | +-------------+ | 2531 | | iSCSI Node | | 2532 | | (Initiator) | | 2533 | +-------------+ | 2534 | | | | 2535 | +--------------+ +--------------+ | 2536 | |Network Portal| |Network Portal| | 2537 | | 192.0.2.4 | | 192.0.2.5 | | 2538 +-+--------------+-+--------------+-+ 2539 | | 2540 | IP Networks | 2541 | | 2542 +-+--------------+-+--------------+-+ 2543 | |Network Portal| |Network Portal| | 2544 | |198.51.100.21 | |198.51.100.3 | | 2545 | | TCP Port 3260| | TCP Port 3260| | 2546 | +--------------+ +--------------+ | 2547 | | | | 2548 | ----------------- | 2549 | | | | 2550 | +-------------+ +--------------+ | 2551 | | iSCSI Node | | iSCSI Node | | 2552 | | (Target) | | (Target) | | 2553 | +-------------+ +--------------+ | 2554 | | 2555 | Network Entity (iSCSI Server) | 2556 +-----------------------------------+ 2558 4.4.1. iSCSI Architecture Model 2560 This Section describes the part of the iSCSI architecture model 2561 that has the most bearing on the relationship between iSCSI and 2562 the SCSI Architecture Model. 2564 - Network Entity - represents a device or gateway that is 2565 accessible from the IP network. A Network Entity must have 2566 one or more Network Portals (see a following item), each of 2567 which can be used by some iSCSI Nodes (see the following 2568 item) contained in that Network Entity to gain access to the 2570 IP network. 2572 - iSCSI Node - represents a single iSCSI initiator or iSCSI 2573 target or an instance of each. There are one or more iSCSI 2574 Nodes within a Network Entity. The iSCSI Node is accessible 2575 via one or more Network Portals (see item d). An iSCSI Node 2576 is identified by its iSCSI Name (see Section 4.2.7 and 2577 Section 13). The separation of the iSCSI Name from the 2578 addresses used by and for the iSCSI node allows multiple 2579 iSCSI nodes to use the same addresses, and the same iSCSI 2580 node to use multiple addresses. 2582 - An alias string may also be associated with an iSCSI Node. 2583 The alias allows an organization to associate a user 2584 friendly string with the iSCSI Name. However, the alias 2585 string is not a substitute for the iSCSI Name. 2587 - Network Portal - a component of a Network Entity that has a 2588 TCP/IP network address and that may be used by an iSCSI Node 2589 within that Network Entity for the connection(s) within one 2590 of its iSCSI sessions. In an initiator, it is identified by 2591 its IP address. In a target, it is identified by its IP 2592 address and its listening TCP port. 2594 - Portal Groups - iSCSI supports multiple connections within 2595 the same session; some implementations will have the ability 2596 to combine connections in a session across multiple Network 2597 Portals. A Portal Group defines a set of Network Portals 2598 within an iSCSI Node that collectively supports the 2599 capability of coordinating a session with connections that 2600 span these portals. Not all Network Portals within a Portal 2601 Group need to participate in every session connected through 2602 that Portal Group. One or more Portal Groups may provide 2603 access to an iSCSI Node. Each Network Portal, as utilized by 2604 a given iSCSI Node, belongs to exactly one portal group 2605 within that node. Portal Groups are identified within an 2606 iSCSI Node by a portal group tag, a simple unsigned-integer 2607 between 0 and 65535 (see Section 13.3). All Network Portals 2608 with the same portal group tag in the context of a given 2609 iSCSI Node are in the same Portal Group. 2611 Both iSCSI Initiators and iSCSI Targets have portal groups, 2612 though only the iSCSI Target Portal Groups are used directly 2613 in the iSCSI protocol (e.g., in SendTargets). For references 2614 to the Initiator Portal Groups, see Section 10.1.1. 2616 - Portals within a Portal Group should support similar session 2617 parameters, because they may participate in a common 2618 session. 2620 The following diagram shows an example of one such configuration 2621 on a target and how a session that shares Network Portals within a 2622 Portal Group may be established. 2624 ----------------------------IP Network--------------------- 2625 | | | 2626 +----|----------------|----+ +----|---------+ 2627 | +---------+ +---------+ | | +---------+ | 2628 | | Network | | Network | | | | Network | | 2629 | | Portal | | Portal | | | | Portal | | 2630 | +--|------+ +---------+ | | +---------+ | 2631 | | | | | | | 2632 | | Portal | | | | Portal | 2633 | | Group 1 | | | | Group 2 | 2634 +--------------------------+ +--------------+ 2635 | | | 2636 +--------|----------------|------------------|------------------+ 2637 | | | | | 2638 | +----------------------------+ +----------------------------+ | 2639 | | iSCSI Session (Target side)| | iSCSI Session (Target side)| | 2640 | | | | | | 2641 | | (TSIH = 56) | | (TSIH = 48) | | 2642 | +----------------------------+ +----------------------------+ | 2643 | | 2644 | iSCSI Target Node | 2645 | (within Network Entity, not shown) | 2646 +---------------------------------------------------------------+ 2648 4.4.2. SCSI Architecture Model 2650 This Section describes the relationship between the SCSI 2651 Architecture Model [SAM2] and constructs of the SCSI device, SCSI 2652 port and I_T nexus, and the iSCSI constructs described in Section 2653 4.4.1. 2655 This relationship implies implementation requirements in order to 2656 conform to the SAM2 model and other SCSI operational functions. 2657 These requirements are detailed in Section 4.4.3. 2659 The following list outlines mappings of SCSI architectural 2660 elements to iSCSI. 2662 a) SCSI Device - the SAM2 term for an entity that contains 2663 one or more SCSI ports that are connected to a service 2664 delivery subsystem and supports a SCSI application 2665 protocol. For example, a SCSI Initiator Device contains 2666 one or more SCSI Initiator Ports and zero or more 2667 application clients. A SCSI Target Device contains one or 2668 more SCSI Target Ports and one or more logical units. For 2669 iSCSI, the SCSI Device is the component within an iSCSI 2670 Node that provides the SCSI functionality. As such, there 2671 can be one SCSI Device, at most, within an iSCSI Node. 2672 Access to the SCSI Device can only be achieved in an 2673 iSCSI normal operational session (see Section 4.3). The 2674 SCSI Device Name is defined to be the iSCSI Name of the 2675 node and MUST be used in the iSCSI protocol. 2677 b) SCSI Port - the SAM2 term for an entity in a SCSI Device 2678 that provides the SCSI functionality to interface with a 2679 service delivery subsystem or transport. For iSCSI, the 2680 definition of SCSI Initiator Port and SCSI Target Port 2681 are different. 2683 SCSI Initiator Port: This maps to one endpoint of an 2684 iSCSI normal operational session (see Section 4.3). An 2685 iSCSI normal operational session is negotiated through 2686 the login process between an iSCSI initiator node and an 2687 iSCSI target node. At successful completion of this 2688 process, a SCSI Initiator Port is created within the SCSI 2689 Initiator Device. The SCSI Initiator Port Name and SCSI 2690 Initiator Port Identifier are both defined to be the 2691 iSCSI Initiator Name together with (a) a label that 2692 identifies it as an initiator port name/identifier and 2693 (b) the ISID portion of the session identifier. 2695 SCSI Target Port: This maps to an iSCSI Target Portal 2696 Group. The SCSI Target Port Name and the SCSI Target Port 2697 Identifier are both defined to be the iSCSI Target Name 2698 together with (a) a label that identifies it as a target 2699 port name/identifier and (b) the portal group tag. 2701 The SCSI Port Name MUST be used in iSCSI. When used in 2702 SCSI parameter data, the SCSI port name MUST be encoded 2703 as: 2704 1. The iSCSI Name in UTF-8 format, followed by 2705 2. a comma separator (1 byte), followed by 2706 3. the ASCII character 'i' (for SCSI Initiator Port) 2707 or the ASCII character 't' (for SCSI Target Port) 2708 (1 byte), followed by 2709 4. a comma separator (1 byte), followed by 2710 5. a text encoding as a hex-constant (see Section 6.1) 2711 of the ISID (for SCSI initiator port) or the 2712 portal group tag (for SCSI target port) including 2713 the initial 0X or 0x and the terminating null (14 2714 bytes). 2716 The ASCII character 'i' or 't' is the label that 2717 identifies this port as either a SCSI Initiator 2718 Port or a SCSI Target Port. 2720 c) I_T nexus - a relationship between a SCSI Initiator Port 2721 and a SCSI Target Port, according to [SAM2]. For iSCSI, 2722 this relationship is a session, defined as a relationship 2723 between an iSCSI Initiator's end of the session (SCSI 2724 Initiator Port) and the iSCSI Target's Portal Group. The 2725 I_T nexus can be identified by the conjunction of the 2726 SCSI port names or by the iSCSI session identifier SSID. 2727 iSCSI defines the I_T nexus identifier to be the tuple 2728 (iSCSI Initiator Name + ",i,0x" + ISID in text format, 2729 iSCSI Target Name + ",t,0x" + Portal Group Tag in text 2730 format) - an upper case hex prefix "0X" may alternatively 2731 be used in place of "0x". 2733 NOTE: The I_T nexus identifier is not equal to the 2734 session identifier (SSID). 2736 4.4.3. Consequences of the Model 2738 This Section describes implementation and behavioral requirements 2739 that result from the mapping of SCSI constructs to the iSCSI 2740 constructs defined above. Between a given SCSI initiator port and 2741 a given SCSI target port, only one I_T nexus (session) can exist. 2742 No more than one nexus relationship (parallel nexus) is allowed by 2743 [SAM2]. Therefore, at any given time, only one session with the 2744 same session identifier (SSID) can exist between a given iSCSI 2745 initiator node and an iSCSI target node. 2747 These assumptions lead to the following conclusions and 2748 requirements: 2750 ISID RULE: Between a given iSCSI Initiator and iSCSI Target Portal 2751 Group (SCSI target port), there can only be one session with a 2752 given value for ISID that identifies the SCSI initiator port. See 2753 Section 11.12.5. 2755 The structure of the ISID that contains a naming authority 2756 component (see Section 11.12.5 and [RFC3721]) provides a mechanism 2757 to facilitate compliance with the ISID rule. (See Section 10.1.1) 2759 The iSCSI Initiator Node should manage the assignment of ISIDs 2760 prior to session initiation. The "ISID RULE" does not preclude the 2761 use of the same ISID from the same iSCSI Initiator with different 2762 Target Portal Groups on the same iSCSI target or on other iSCSI 2763 targets (see Section 10.1.1). Allowing this would be analogous to 2764 a single SCSI Initiator Port having relationships (nexus) with 2765 multiple SCSI target ports on the same SCSI target device or SCSI 2766 target ports on other SCSI target devices. It is also possible to 2767 have multiple sessions with different ISIDs to the same Target 2768 Portal Group. Each such session would be considered to be with a 2769 different initiator even when the sessions originate from the same 2770 initiator device. The same ISID may be used by a different iSCSI 2771 initiator because it is the iSCSI Name together with the ISID that 2772 identifies the SCSI Initiator Port. 2774 NOTE: A consequence of the ISID RULE and the specification for the 2775 I_T nexus identifier is that two nexus with the same identifier 2776 should never exist at the same time. 2778 TSIH RULE: The iSCSI Target selects a non-zero value for the TSIH 2779 at session creation (when an initiator presents a 0 value at 2780 Login). After being selected, the same TSIH value MUST be used 2781 whenever initiator or target refers to the session and a TSIH is 2782 required. 2784 4.4.3.1. I_T Nexus State 2786 Certain nexus relationships contain an explicit state (e.g., 2787 initiator-specific mode pages) that may need to be preserved by 2788 the device server [SAM2] in a logical unit through changes or 2789 failures in the iSCSI layer (e.g., session failures). In order for 2790 that state to be restored, the iSCSI initiator should reestablish 2791 its session (re-login) to the same Target Portal Group using the 2792 previous ISID. That is, it should reinstate the session via iSCSI 2793 session reinstatement (Section 6.3.5) or continue via session 2794 continuation (Section 6.3.6). This is because the SCSI initiator 2795 port identifier and the SCSI target port identifier (or relative 2796 target port) form the datum that the SCSI logical unit device 2797 server uses to identify the I_T nexus. 2799 4.4.3.2. Reservations 2801 There are two reservation management methods defined in the SCSI 2802 standards, reserve/release reservations, based on the RESERVE and 2803 RELEASE commands [SPC2], and persistent reservations, based on the 2804 PERSISTENT RESERVE IN and PERSISTENT RESERVE OUT commands [SPC3]. 2805 Reserve/release reservations are obsolete [SPC3] and should not be 2806 used; persistent reservations are suggested as an alternative, see 2807 Annex B of [SPC4]. 2809 State for persistent reservations is required to persist through 2810 changes and failures at the iSCSI layer that result in I_T nexus 2811 failures, see [SPC3] for details and specific requirements. 2813 In contrast, [SPC2] does not specify detailed persistence 2814 requirements for reserve/release reservation state after an I_T 2815 nexus failure. Nonetheless, when reserve/release reservations are 2816 supported by an iSCSI target, the preferred implementation 2817 approach is to preserve reserve/release reservation state for 2818 iSCSI session reinstatement (see Section 6.3.5) or session 2819 continuation (see Section 6.3.6). 2821 Two additional caveats apply to reserve/release reservations: 2823 - Retention of a failed session's reserve/release reservation 2824 state by an iSCSI target, even after that failed iSCSI 2825 session is not reinstated or continued, may require an 2826 initiator to issue a reset (e.g., LOGICAL UNIT RESET, see 2827 Section 11.5) in order to remove that reservation state. 2829 - Reserve/release reservations may not behave as expected when 2830 persistent reservations are also used on the same logical 2831 unit; see the discussion of "Exceptions to SPC-2 RESERVE and 2832 RELEASE behavior" in [SPC4]. 2834 4.5. iSCSI UML Model 2836 This Section presents the application of the UML modeling concepts 2837 discussed in Section 3 to the iSCSI and SCSI architecture model 2838 discussed in Section 4.4. 2840 +----------------+ 2841 | Network Entity | 2842 +----------------+ 2843 @ 1 @ 1 2844 | | 2845 +--------------------+ | 2846 | | 2847 | | 0..* 2848 | +------------------+ 2849 | | iSCSI Node | 2850 | +------------------+ 2851 | @ @ 2852 | | | 2853 | +-----------+ =(a)= +-----------+ 2854 | | | 2855 | | 0..1 | 0..1 2856 | +------------------------+ +----------------------+ 2857 | | iSCSI Target Node | | iSCSI Initiator Node | 2858 | +------------------------+ +----------------------+ 2859 | @ 1 @ 1 2860 | +--------------+ | 2861 | 1..* | | 1..* 2862 | +-----------------------------+ 2863 | | Portal Group | 2864 | +-----------------------------+ 2865 | O 1 2866 | | 2867 | | 1..* 2868 | 1..* +------------------------+ 2869 +--------------------| Network Portal | 2870 +------------------------+ 2872 (a) Each instance of an iSCSI Node class MUST contain one iSCSI 2873 Target Node instance or one iSCSI Initiator Node instance, or 2874 both. 2876 +----------------+ 2877 | Network Entity | 2878 +----------------+ 2879 @ 1 @ 1 2880 | | +-------------------+ 2881 +---------------------+ | | iSCSI Session | 2882 | | +-------------------+ 2883 | | 0..* | SSID[1] | 2884 | +--------------------+ | ISID[1] | 2885 | | iSCSI Node | +-------------------+ 2886 | +--------------------+ @ 1 2887 | | iSCSI Node Name[1] | | 2888 | | Alias [0..1] | | 0..* 2889 | +--------------------+ +------------------+ 2890 | | | | iSCSI Connection | 2891 | +--------------------+ +------------------+ 2892 | @ 1 @ 1 | CID[1] | 2893 | | | +------------------+ 2894 | +-------------+ ==(b)== +----------+ 0..* | 2895 | | 1 | 1 | 2896 | +------------------------+ +------------------------+ | 2897 | | iSCSI Target Node | | iSCSI Initiator Node | | 2898 | +------------------------+ +------------------------+ | 2899 | | iSCSI Target name [1] | |iSCSI Initiator Name [1]| | 2900 | +------------------------+ +------------------------+ | 2901 | @ 1 @ 1 | 2902 | | 1..* | 1..* | 2903 | +--------------------------+ +------------------------+ | 2904 | | Target Portal Group | | Initiator Portal Group | | 2905 | +--------------------------+ +------------------------+ | 2906 | |Target Portal Group Tag[1]| | Portal group tag[1] | | 2907 | +--------------------------+ +------------------------+ | 2908 | o 1 o 1 | 2909 | +------------+ +---------+ | 2910 | 1..* | | 1..* | 2911 | +-------------------------+ | 2912 | | Network Portal | | 2913 | +-------------------------+ | 2914 | 1..* | IP Address [1] | 1 | 2915 +----------------| TCP Port [0..1] |<----------------------+ 2916 +-------------------------+ 2918 (b) Each instance of an iSCSI Node class MUST contain one iSCSI 2919 Target Node instance or one iSCSI Initiator Node instance, or 2920 both. However, in all scenarios, note that an iSCSI Node MUST 2921 only have a single iSCSI Name. Note the related requirement in 2922 Section 4.2.7.1. 2924 4.6. Request/Response Summary 2926 This Section lists and briefly describes all the iSCSI PDU types 2927 (request and responses). 2929 All iSCSI PDUs are built as a set of one or more header segments 2930 (basic and auxiliary) and zero or one data segments. The header 2931 group and the data segment may each be followed by a CRC (digest) 2932 (see [CRC]). 2934 The basic header segment has a fixed length of 48 bytes. 2936 4.6.1. Request/Response Types Carrying SCSI Payload 2938 4.6.1.1. SCSI-Command 2940 This request carries the SCSI CDB and all the other SCSI execute 2941 command procedure call (see [SAM2]) IN arguments such as task 2942 attributes, Expected Data Transfer Length for one or both transfer 2943 directions (the latter for bidirectional commands), and Task Tag 2944 (as part of the I_T_L_x nexus). The I_T_L nexus is derived by the 2945 initiator and target from the LUN field in the request and the I_T 2946 nexus is implicit in the session identification. 2948 In addition, the SCSI-command PDU carries information required for 2949 the proper operation of the iSCSI protocol - the command sequence 2950 number (CmdSN) and the expected status number (ExpStatSN) on the 2951 connection it is issued. 2953 All or part of the SCSI output (write) data associated with the 2954 SCSI command may be sent as part of the SCSI-Command PDU as a data 2955 segment. 2957 4.6.1.2. SCSI-Response 2959 The SCSI-Response carries all the SCSI execute-command procedure 2960 call (see [SAM2]) OUT arguments and the SCSI execute-command 2961 procedure call return value. 2963 The SCSI-Response contains the residual counts from the operation, 2964 if any, an indication of whether the counts represent an overflow 2965 or an underflow, and the SCSI status if the status is valid or a 2966 response code (a non-zero return value for the execute-command 2967 procedure call) if the status is not valid. 2969 For a valid status that indicates that the command has been 2970 processed, but resulted in an exception (e.g., a SCSI CHECK 2971 CONDITION), the PDU data segment contains the associated sense 2972 data. The use of Autosense ([SAM2]) is REQUIRED by iSCSI. 2974 Some data segment content may also be associated (in the data 2975 segment) with a non-zero response code. 2977 In addition, the SCSI-Response PDU carries information required 2978 for the proper operation of the iSCSI protocol: 2980 - The number of Data-In PDUs that a target has sent (to enable 2981 the initiator to check that all have arrived). 2983 - StatSN - the Status Sequence Number on this connection. 2985 - ExpCmdSN - the next Expected Command Sequence Number at the 2986 target. 2988 - MaxCmdSN - the maximum CmdSN acceptable at the target from 2989 this initiator. 2991 4.6.1.3. Task Management Function Request 2993 The Task Management function request provides an initiator with a 2994 way to explicitly control the execution of one or more SCSI Tasks 2995 or iSCSI functions. The PDU carries a function identifier (which 2996 task management function to perform) and enough information to 2997 unequivocally identify the task or task-set on which to perform 2998 the action, even if the task(s) to act upon has not yet arrived or 2999 has been discarded due to an error. 3001 The referenced tag identifies an individual task if the function 3002 refers to an individual task. 3004 The I_T_L nexus identifies task sets. In iSCSI the I_T_L nexus is 3005 identified by the LUN and the session identification (the session 3006 identifies an I_T nexus). 3008 For task sets, the CmdSN of the Task Management function request 3009 helps identify the tasks upon which to act, namely all tasks 3010 associated with a LUN and having a CmdSN preceding the Task 3011 Management function request CmdSN. 3013 For a Task Management function, the coordination between responses 3014 to the tasks affected and the Task Management function response is 3015 done by the target. 3017 4.6.1.4. Task Management Function Response 3019 The Task Management function response carries an indication of 3020 function completion for a Task Management function request 3021 including how it completed (response and qualifier) and additional 3022 information for failure responses. 3024 After the Task Management response indicates Task Management 3025 function completion, the initiator will not receive any additional 3026 responses from the affected tasks. 3028 4.6.1.5. SCSI Data-out and SCSI Data-in 3030 SCSI Data-out and SCSI Data-in are the main vehicles by which SCSI 3031 data payload is carried between initiator and target. Data payload 3032 is associated with a specific SCSI command through the Initiator 3033 Task Tag. For target convenience, outgoing solicited data also 3034 carries a Target Transfer Tag (copied from R2T) and the LUN. Each 3035 PDU contains the payload length and the data offset relative to 3036 the buffer address contained in the SCSI execute command procedure 3037 call. 3039 In each direction, the data transfer is split into "sequences". An 3040 end-of-sequence is indicated by the F bit. 3042 An outgoing sequence is either unsolicited (only the first 3043 sequence can be unsolicited) or consists of all the Data-Out PDUs 3044 sent in response to an R2T. 3046 Input sequences enable the switching of direction for 3047 bidirectional commands as required. 3049 For input, the target may request positive acknowledgement of 3050 input data. This is limited to sessions that support error 3051 recovery and is implemented through the A bit in the SCSI Data-in 3052 PDU header. 3054 Data-in and Data-out PDUs also carry the DataSN to enable the 3055 initiator and target to detect missing PDUs (discarded due to an 3056 error). 3058 In addition, StatSN is carried by the Data-In PDUs. 3060 To enable a SCSI command to be processed while involving a minimum 3061 number of messages, the last SCSI Data-in PDU passed for a command 3062 may also contain the status if the status indicates termination 3063 with no exceptions (no sense or response involved). 3065 4.6.1.6. Ready To Transfer (R2T) 3067 R2T is the mechanism by which the SCSI target "requests" the 3068 initiator for output data. R2T specifies to the initiator the 3069 offset of the requested data relative to the buffer address from 3070 the execute command procedure call and the length of the solicited 3071 data. 3073 To help the SCSI target associate the resulting Data-out with an 3074 R2T, the R2T carries a Target Transfer Tag that will be copied by 3075 the initiator in the solicited SCSI Data-out PDUs. There are no 3076 protocol specific requirements with regard to the value of these 3077 tags, but it is assumed that together with the LUN, they will 3078 enable the target to associate data with an R2T. 3080 R2T also carries information required for proper operation of the 3081 iSCSI protocol, such as: 3083 - R2TSN (to enable an initiator to detect a missing R2T) 3085 - StatSN 3087 - ExpCmdSN 3089 - MaxCmdSN 3091 4.6.2. Requests/Responses carrying SCSI and iSCSI Payload 3093 4.6.2.1. Asynchronous Message 3095 Asynchronous Messages are used to carry SCSI asynchronous events 3096 (AEN) and iSCSI asynchronous messages. 3098 When carrying an AEN, the event details are reported as sense data 3099 in the data segment. 3101 4.6.3. Requests/Responses Carrying iSCSI Only Payload 3103 4.6.3.1. Text Request and Text Response 3105 Text requests and responses are designed as a parameter 3106 negotiation vehicle and as a vehicle for future extension. 3108 In the data segment Text Requests/Responses carry text information 3109 using a simple "key=value" syntax. 3111 Text Request/Responses may form extended sequences using the same 3112 Initiator Task Tag. The initiator uses the F (Final) flag bit in 3113 the text request header to indicate its readiness to terminate a 3114 sequence. The target uses the F (Final) flag bit in the text 3115 response header to indicate its consent to sequence termination. 3117 Text Request and Responses also use the Target Transfer Tag to 3118 indicate continuation of an operation or a new beginning. A target 3119 that wishes to continue an operation will set the Target Transfer 3120 Tag in a Text Response to a value different from the default 3121 0xffffffff. An initiator willing to continue will copy this value 3122 into the Target Transfer Tag of the next Text Request. If the 3123 initiator wants to restart the current target negotiation (start 3124 fresh) will set the Target Transfer Tag to 0xffffffff. 3126 Although a complete exchange is always started by the initiator, 3127 specific parameter negotiations may be initiated by the initiator 3128 or target. 3130 4.6.3.2. Login Request and Login Response 3132 Login Requests and Responses are used exclusively during the Login 3133 Phase of each connection to set up the session and connection 3134 parameters. (The Login Phase consists of a sequence of login 3135 requests and responses carrying the same Initiator Task Tag.) 3137 A connection is identified by an arbitrarily selected connection- 3138 ID (CID) that is unique within a session. 3140 Similar to the Text Requests and Responses, Login 3141 Requests/Responses carry key=value text information with a simple 3142 syntax in the data segment. 3144 The Login Phase proceeds through several stages (security 3145 negotiation, operational parameter negotiation) that are selected 3146 with two binary coded fields in the header -- the "current stage" 3147 (CSG) and the "next stage" (NSG) with the appearance of the latter 3148 being signaled by the "transit" flag (T). 3150 The first Login Phase of a session plays a special role, called 3151 the leading login, which determines some header fields (e.g., the 3152 version number, the maximum number of connections, and the session 3153 identification). 3155 The CmdSN initial value is also set by the leading login. 3157 StatSN for each connection is initiated by the connection login. 3159 A login request may indicate an implied logout (cleanup) of the 3160 connection to be logged in (a connection restart) by using the 3161 same Connection ID (CID) as an existing connection as well as the 3162 same session identifying elements of the session to which the old 3163 connection was associated. 3165 4.6.3.3. Logout Request and Response 3167 Logout Requests and Responses are used for the orderly closing of 3168 connections for recovery or maintenance. The logout request may be 3169 issued following a target prompt (through an asynchronous message) 3170 or at an initiators initiative. When issued on the connection to 3171 be logged out no other request may follow it. 3173 The Logout response indicates that the connection or session 3174 cleanup is completed and no other responses will arrive on the 3175 connection (if received on the logging out connection). In 3176 addition, the Logout Response indicates how long the target will 3177 continue to hold resources for recovery (e.g., command execution 3178 that continues on a new connection) in the Time2Retain field and 3179 how long the initiator must wait before proceeding with recovery 3180 in the Time2Wait field. 3182 4.6.3.4. SNACK Request 3184 With the SNACK Request, the initiator requests retransmission of 3185 numbered-responses or data from the target. A single SNACK request 3186 covers a contiguous set of missing items, called a run, of a given 3187 type of items. The type is indicated in a type field in the PDU 3188 header. The run is composed of an initial item (StatSN, DataSN, 3189 R2TSN) and the number of missed Status, Data, or R2T PDUs. For 3190 long data-in sequences, the target may request (at predefined 3191 minimum intervals) a positive acknowledgement for the data sent. A 3192 SNACK request with a type field that indicates ACK and the number 3193 of Data-In PDUs acknowledged conveys this positive 3194 acknowledgement. 3196 4.6.3.5. Reject 3198 Reject enables the target to report an iSCSI error condition 3199 (e.g., protocol, unsupported option) that uses a Reason field in 3200 the PDU header and includes the complete header of the bad PDU in 3201 the Reject PDU data segment. 3203 4.6.3.6. NOP-Out Request and NOP-In Response 3205 This request/response pair may be used by an initiator and target 3206 as a "ping" mechanism to verify that a connection/session is still 3207 active and all of its components are operational. Such a ping may 3208 be triggered by the initiator or target. The triggering party 3209 indicates that it wants a reply by setting a value different from 3210 the default 0xffffffff in the corresponding Initiator/Target 3211 Transfer Tag. 3213 NOP-In/NOP-Out may also be used "unidirectional" to convey to the 3214 initiator/target command, status or data counter values when there 3215 is no other "carrier" and there is a need to update the 3216 initiator/target. 3218 5. SCSI Mode Parameters for iSCSI 3220 There are no iSCSI specific mode pages. 3222 6. Login and Full Feature Phase Negotiation 3224 iSCSI parameters are negotiated at session or connection 3225 establishment by using Login Requests and Responses (see Section 3226 4.2.4) and during Full Feature Phase (Section 4.2.5) by using Text 3227 Requests and Responses. In both cases the mechanism used is an 3228 exchange of iSCSI-text-key=value pairs. For brevity, iSCSI-text- 3229 keys are called just keys in the rest of this document. 3231 Keys are either declarative or require negotiation and the key 3232 description indicates if the key is declarative or requires 3233 negotiation. 3235 For the declarative keys the declaring party sets a value for the 3236 key. The key specification indicates if the key can be declared by 3237 the initiator, target or both. 3239 For the keys that require negotiation, one of the parties (the 3240 proposing party) proposes a value or set of values by including 3241 the key=value in the data part of a Login or Text Request or 3242 Response. The other party (the accepting party) makes a selection 3243 based on the value or list of values proposed and includes the 3244 selected value in a key=value in the data part of the following 3245 Login or Text Response or Request. For most of the keys, both the 3246 initiator and target can be proposing parties. 3248 The login process proceeds in two stages - the security 3249 negotiation stage and the operational parameter negotiation stage. 3250 Both stages are optional but at least one of them has to be 3251 present to enable setting some mandatory parameters. 3253 If present, the security negotiation stage precedes the 3254 operational parameter negotiation stage. 3256 Progression from stage to stage is controlled by the T 3257 (Transition) bit in the Login Request/Response PDU header. Through 3258 the T bit set to 1, the initiator indicates that it would like to 3259 transition. The target agrees to the transition (and selects the 3260 next stage) when ready. A field in the Login PDU header indicates 3261 the current stage (CSG) and during transition, another field 3262 indicates the next stage (NSG) proposed (initiator) and selected 3263 (target). 3265 The Text negotiation process is used to negotiate or declare 3266 operational parameters. The negotiation process is controlled by 3267 the F (final) bit in the PDU header. During text negotiations, the 3268 F bit is used by the initiator to indicate that it is ready to 3269 finish the negotiation and by the Target to acquiesce the end of 3270 negotiation. 3272 Since some key=value pairs may not fit entirely in a single PDU, 3273 the C (continuation) bit is used (both in Login and Text) to 3274 indicate that "more follows". 3276 The text negotiation uses an additional mechanism by which a 3277 target may deliver larger amounts of data to an enquiring 3278 initiator. The target sets a Target Task Tag to be used as a 3279 bookmark which when returned by the initiator, means "go on". If 3280 reset to a "neutral value", it means "forget about the rest". 3282 This Section details types of keys and values used, the syntax 3283 rules for parameter formation, and the negotiation schemes to be 3284 used with different types of parameters. 3286 6.1. Text Format 3288 The initiator and target send a set of key=value pairs encoded in 3289 UTF-8 Unicode. All the text keys and text values specified in this 3290 document are to be presented and interpreted in the case in which 3291 they appear in this document. They are case sensitive. 3293 The following character symbols are used in this document for text 3294 items (the hexadecimal values represent Unicode code points): 3296 (a-z, A-Z) ) (0x61-0x7a, 0x41-0x5a) - letters 3297 (0-9) (0x30-0x39) - digits 3298 " " (0x20) - space 3299 "." (0x2e) - dot 3300 "-" (0x2d) - minus 3301 "+" (0x2b) - plus 3302 "@" (0x40) - commercial at 3303 "_" (0x5f) - underscore 3304 "=" (0x3d) - equal 3305 ":" (0x3a) - colon 3307 "/" (0x2f) - solidus or slash 3308 "[" (0x5b) - left bracket 3309 "]" (0x5d) - right bracket 3310 null (0x00) - null separator 3311 "," (0x2c) - comma 3312 "~" (0x7e) - tilde 3314 Key=value pairs may span PDU boundaries. An initiator or target 3315 that sends partial key=value text within a PDU indicates that more 3316 text follows by setting the C bit in the Text or Login Request or 3317 Text or Login Response to 1. Data segments in a series of PDUs 3318 that have the C bit set to 1 and end with a PDU that have the C 3319 bit set to 0, or include a single PDU that has the C bit set to 0 3320 have to be considered as forming a single logical-text-data- 3321 segment (LTDS). 3323 Every key=value pair, including the last or only pair in a LTDS, 3324 MUST be followed by one null (0x00) delimiter. 3326 A key-name is whatever precedes the first = in the key=value pair. 3327 The term key is used frequently in this document in place of key- 3328 name. 3330 A value is whatever follows the first = in the key=value pair up 3331 to the end of the key=value pair, but not including the null 3332 delimiter. 3334 The following definitions will be used in the rest of this 3335 document: 3337 - standard-label: A string of one or more characters that 3338 consist of letters, digits, dot, minus, plus, commercial at, 3339 or underscore. A standard-label MUST begin with a capital 3340 letter and must not exceed 63 characters. 3342 - key-name: A standard-label. 3344 - text-value: A string of zero or more characters that consist 3345 of letters, digits, dot, minus, plus, commercial at, 3346 underscore, slash, left bracket, right bracket, or colon. 3348 - iSCSI-name-value: A string of one or more characters that 3349 consist of minus, dot, colon, or any character allowed by 3350 the output of the iSCSI string-prep template as specified in 3351 [RFC3722] (see also Section 4.2.7.2). 3353 - iSCSI-local-name-value: A UTF-8 string; no null characters 3354 are allowed in the string. This encoding is to be used for 3355 localized (internationalized) aliases. 3357 - boolean-value: The string "Yes" or "No". 3359 - hex-constant: A hexadecimal constant encoded as a string 3360 that starts with "0x" or "0X" followed by one or more digits 3361 or the letters a, b, c, d, e, f, A, B, C, D, E, or F. Hex- 3362 constants are used to encode numerical values or binary 3363 strings. When used to encode numerical values, the excessive 3364 use of leading 0 digits is discouraged. The string following 3365 0X (or 0x) represents a base16 number that starts with the 3366 most significant base16 digit, followed by all other digits 3367 in decreasing order of significance and ending with the 3368 least-significant base16 digit. When used to encode binary 3369 strings, hexadecimal constants have an implicit byte-length 3370 that includes four bits for every hexadecimal digit of the 3371 constant, including leading zeroes. For example, a hex- 3372 constant of n hexadecimal digits has a byte-length of (the 3373 integer part of) (n+1)/2. 3375 - decimal-constant: An unsigned decimal number with the digit 3376 0 or a string of one or more digits that start with a non- 3377 zero digit. Decimal-constants are used to encode numerical 3378 values or binary strings. Decimal constants can only be used 3379 to encode binary strings if the string length is explicitly 3380 specified. There is no implicit length for decimal strings. 3381 Decimal-constant MUST NOT be used for parameter values if 3382 the values can be equal or greater than 2**64 (numerical) or 3383 for binary strings that can be longer than 64 bits. 3385 - base64-constant: base64 constant encoded as a string that 3386 starts with "0b" or "0B" followed by 1 or more digits or 3387 letters or plus or slash or equal. The encoding is done 3388 according to [RFC4648]. 3390 - numerical-value: An unsigned integer always less than 2**64 3391 encoded as a decimal-constant or a hex-constant. Unsigned 3392 integer arithmetic applies to numerical-values. 3394 - large-numerical-value: An unsigned integer that can be 3395 larger than or equal to 2**64 encoded as a hex constant, or 3396 base64-constant. Unsigned integer arithmetic applies to 3397 large-numeric-values. 3399 - numeric-range: Two numerical-values separated by a tilde 3400 where the value to the right of tilde must not be lower than 3401 the value to the left. 3403 - regular-binary-value: A binary string not longer than 64 3404 bits encoded as a decimal constant, hex constant, or base64- 3405 constant. The length of the string is either specified by 3406 the key definition or is the implicit byte-length of the 3407 encoded string. 3409 - large-binary-value: A binary string longer than 64 bits 3410 encoded as a hex-constant or base64-constant. The length of 3411 the string is either specified by the key definition or is 3412 the implicit byte-length of the encoded string. 3414 - binary-value: A regular-binary-value or a large-binary- 3415 value. Operations on binary values are key specific. 3417 - simple-value: Text-value, iSCSI-name-value, boolean-value, 3418 numeric-value, a numeric-range, or a binary-value. 3420 - list-of-values: A sequence of text-values separated by a 3421 comma. 3423 If not otherwise specified, the maximum length of a simple-value 3424 (not its encoded representation) is 255 bytes not including the 3425 delimiter (comma or zero byte). 3427 Any iSCSI target or initiator MUST support receiving at least 8192 3428 bytes of key=value data in a negotiation sequence. When proposing 3429 or accepting authentication methods that explicitly require 3430 support for very long authentication items, the initiator and 3431 target MUST support receiving of at least 64 kilobytes of 3432 key=value data. 3434 6.2. Text Mode Negotiation 3436 During login, and thereafter, some session or connection 3437 parameters are either declared or negotiated through an exchange 3438 of textual information. 3440 The initiator starts the negotiation and/or declaration through a 3441 Text or Login request and indicates when it is ready for 3442 completion (by setting the F bit to 1 and keeping it to 1 in a 3443 Text Request or the T bit in the Login Request). As negotiation 3444 text may span PDU boundaries, a Text or Login Request or Text or 3445 Login Response PDU that have the C bit set to 1 MUST NOT have the 3446 F/T bit set to 1. 3448 A target receiving a Text or Login Request with the C bit set to 1 3449 MUST answer with a Text or Login Response with no data segment 3450 (DataSegmentLength 0). An initiator receiving a Text or Login 3451 Response with the C bit set to 1 MUST answer with a Text or Login 3452 Request with no data segment (DataSegmentLength 0). 3454 A target or initiator SHOULD NOT use a Text or Login Response or 3455 Text or Login Request with no data segment (DataSegmentLength 0) 3456 unless explicitly required by a general or a key-specific 3457 negotiation rule. 3459 There MUST NOT be more than one outstanding Text Request, or Text 3460 Response PDU on an iSCSI connection. An outstanding PDU in this 3461 context is one that has not been acknowledged by the remote iSCSI 3462 side. 3464 The format of a declaration is: 3466 Declarer-> = 3468 The general format of text negotiation is: 3470 Proposer-> = 3472 Acceptor-> ={|NotUnderstood|Irrelevant|Reject} 3474 Thus a declaration is a one-way textual exchange (unless the key 3475 is not understood by the receiver) while a negotiation is a two- 3476 way exchange. 3478 The proposer or declarer can either be the initiator or the 3479 target, and the acceptor can either be the target or initiator, 3480 respectively. Targets are not limited to respond to key=value 3481 pairs as proposed by the initiator. The target may propose 3482 key=value pairs of its own. 3484 All negotiations are explicit (i.e., the result MUST only be based 3485 on newly exchanged or declared values). There are no implicit 3486 proposals. If a proposal is not made, then a reply cannot be 3487 expected. Conservative design also requires that default values 3488 should not be relied upon when use of some other value has serious 3489 consequences. 3491 The value proposed or declared can be a numerical-value, a 3492 numerical-range defined by lower and upper value with both 3493 integers separated by tilde, a binary value, a text-value, an 3494 iSCSI-name-value, an iSCSI-local-name-value, a boolean-value (Yes 3495 or No), or a list of comma separated text-values. A range, a 3496 large-numerical-value, an iSCSI-name-value and an iSCSI-local- 3497 name-value MAY ONLY be used if it is explicitly allowed. An 3498 accepted value can be a numerical-value, a large-numerical-value, 3499 a text-value, or a boolean-value. 3501 If a specific key is not relevant for the current negotiation, the 3502 acceptor may answer with the constant "Irrelevant" for all types 3503 of negotiation. However the negotiation is not considered as 3504 failed if the answer is "Irrelevant". The "Irrelevant" answer is 3505 meant for those cases in which several keys are presented by a 3506 proposing party but the selection made by the acceptor for one of 3507 the keys makes other keys irrelevant. The following example 3508 illustrates the use of "Irrelevant": 3510 I->T InitialR2T=No,ImmediateData=Yes,FirstBurstLength=4192 3511 T->I InitialR2T=Yes,ImmediateData=No,FirstBurstLength=Irrelevant 3512 I->T X-rdname-vkey1=(bla,alb,None), X-rdname-vkey2=(bla,alb) 3513 T->I X-rdname-vkey1=None, X-rdname-vkey2=Irrelevant 3515 Any key not understood by the acceptor may be ignored by the 3516 acceptor without affecting the basic function. However, the answer 3517 for a key not understood MUST be key=NotUnderstood. Note that 3518 NotUnderstood is a valid answer for both declarative and 3519 negotiated keys. The general iSCSI philosophy is that 3520 comprehension precedes processing for any iSCSI key. A proposer 3521 of an iSCSI key, negotiated or declarative, in a text key exchange 3522 MUST thus be able to properly handle a NotUnderstood response. 3524 The proper way to handle a NotUnderstood response depends on where 3525 the key is specified and whether the key is declarative vs. 3526 negotiated. An iSCSI implementation MUST comprehend all text keys 3527 defined in this document. Returning a NotUnderstood response on 3528 any of these text keys therefore MUST be considered a protocol 3529 error and handled accordingly. For all other "later" keys, i.e. 3530 text keys defined in later specifications, a NotUnderstood answer 3531 concludes the negotiation for a negotiated key whereas for a 3532 declarative key, a NotUnderstood answer simply informs the 3533 declarer of a lack of comprehension by the receiver. 3535 In either case, a NotUnderstood answer always requires that the 3536 protocol behavior associated with that key not be used within the 3537 scope of the key (connection/session) by either side. 3539 The constants "None", "Reject", "Irrelevant", and "NotUnderstood" 3540 are reserved and MUST ONLY be used as described here. Violation of 3541 this rule is a protocol error (in particular the use of "Reject", 3542 "Irrelevant", and "NotUnderstood" as proposed values). 3544 Reject or Irrelevant are legitimate negotiation options where 3545 allowed but their excessive use is discouraged. A negotiation is 3546 considered complete when the acceptor has sent the key value pair 3547 even if the value is "Reject", "Irrelevant", or "NotUnderstood". 3548 Sending the key again would be a re-negotiation and is forbidden 3549 for many keys. 3551 If the acceptor sends "Reject" as an answer the negotiated key is 3552 left at its current value (or default if no value was set). If the 3553 current value is not acceptable to the proposer on the connection 3554 or to the session it is sent, the proposer MAY choose to terminate 3555 the connection or session. 3557 All keys in this document MUST be supported by iSCSI initiators 3558 and targets when used as specified here. If used as specified, 3559 these keys MUST NOT be answered with NotUnderstood. 3561 Implementers may introduce new private keys by prefixing them with 3562 X- followed by their (reversed) domain name, or with new public 3563 keys registered with IANA. For example, the entity owning the 3564 domain example.com can issue: 3566 X-com.example.bar.foo.do_something=3 3568 Each new public key in the course of standardization MUST define 3569 the acceptable responses to the key, including NotUnderstood as 3570 appropriate. Unlike [RFC3720], note that this document prohibits 3571 the X# prefix for new public keys. Based on iSCSI implementation 3572 experience, we know that there is no longer a need for a standard 3573 name prefix for keys that allow NotUnderstood response. Note that 3574 NotUnderstood will generally have to be allowed for new public 3575 keys for backwards compatibility, as well as for private X- keys. 3576 Thus the name prefix "X#" in new public key names does not carry 3577 any significance. New public key names MUST NOT begin with "X#" 3578 prefix to avoid confusion. 3580 Implementers MAY also introduce new values, but ONLY for new keys 3581 or authentication methods (see Section 12), or digests (see 3582 Section 13.1). 3584 Whenever parameter action or acceptance are dependent on other 3585 parameters, the dependency rules and parameter sequence must be 3586 specified with the parameters. 3588 In the Login Phase (see Section 6.3), every stage is a separate 3589 negotiation. In the FullFeaturePhase, a Text Request Response 3590 sequence is a negotiation. Negotiations MUST be handled as atomic 3591 operations. For example, all negotiated values go into effect 3592 after the negotiation concludes in agreement or are ignored if the 3593 negotiation fails. 3595 Some parameters may be subject to integrity rules (e.g., 3596 parameter-x must not exceed parameter-y or parameter-u not 1 3597 implies parameter-v be Yes). Whenever required, integrity rules 3598 are specified with the keys. Checking for compliance with the 3599 integrity rule must only be performed after all the parameters are 3600 available (the existent and the newly negotiated). An iSCSI target 3601 MUST perform integrity checking before the new parameters take 3602 effect. An initiator MAY perform integrity checking. 3604 An iSCSI initiator or target MAY terminate a negotiation that does 3605 not terminate within an implementation-specific reasonable time or 3606 number of exchanges, but SHOULD allow at least six (6) exchanges. 3608 6.2.1. List negotiations 3610 In list negotiation, the originator sends a list of values (which 3611 may include "None") in its order of preference. 3613 The responding party MUST respond with the same key and the first 3614 value that it supports (and is allowed to use for the specific 3615 originator) selected from the originator list. 3617 The constant "None" MUST always be used to indicate a missing 3618 function. However, "None" is only a valid selection if it is 3619 explicitly proposed. When "None" is proposed as a selection item 3620 in a negotiation for a key, it indicates to the responder that not 3621 supporting any functionality related to that key is legal, and if 3622 "None" is the negotiation result for such a key, it means that 3623 key-specific semantics are not operational for the negotiation 3624 scope (connection or session) of that key. 3626 If an acceptor does not understand any particular value in a list, 3627 it MUST ignore it. If an acceptor does not support, does not 3628 understand, or is not allowed to use any of the proposed options 3629 with a specific originator, it may use the constant "Reject" or 3630 terminate the negotiation. The selection of a value not proposed 3631 MUST be handled by the originator as a protocol error. 3633 6.2.2. Simple-value Negotiations 3635 For simple-value negotiations, the accepting party MUST answer 3636 with the same key. The value it selects becomes the negotiation 3637 result. 3639 Proposing a value not admissible (e.g., not within the specified 3640 bounds) MAY be answered with the constant "Reject", otherwise the 3641 acceptor MUST select an admissible value. 3643 The selection, by the acceptor, of a value not admissible under 3644 the selection rules is considered a protocol error. The selection 3645 rules are key-specific. 3647 For a numerical range the value selected MUST be an integer within 3648 the proposed range or "Reject" (if the range is unacceptable). 3650 For Boolean negotiations (i.e., keys taking the values Yes or No), 3651 the accepting party MUST answer with the same key and the result 3652 of the negotiation when the received value does not determine that 3653 result by itself. The last value transmitted becomes the 3654 negotiation result. The rules for selecting the value to answer 3655 with are expressed as Boolean functions of the value received, and 3656 the value that the accepting party would have selected if given a 3657 choice. 3659 Specifically, the two cases in which answers are OPTIONAL are: 3661 - The Boolean function is "AND" and the value "No" is 3662 received. The outcome of the negotiation is "No". 3664 - The Boolean function is "OR" and the value "Yes" is 3665 received. The outcome of the negotiation is "Yes". 3667 Responses are REQUIRED in all other cases, and the value chosen 3668 and sent by the acceptor becomes the outcome of the negotiation. 3670 6.3. Login Phase 3672 The Login Phase establishes an iSCSI connection between an 3673 initiator and a target; it creates also a new session or 3675 associates the connection to an existing session. The Login Phase 3676 sets the iSCSI protocol parameters, security parameters, and 3677 authenticates the initiator and target to each other. 3679 The Login Phase is only implemented via Login request and 3680 responses. The whole Login Phase is considered as a single task 3681 and has a single Initiator Task Tag (similar to the linked SCSI 3682 commands). 3684 There MUST NOT be more than one outstanding Login Request, or 3685 Login Response on an iSCSI connection. An outstanding PDU in this 3686 context is one that has not been acknowledged by the remote iSCSI 3687 side. 3689 The default MaxRecvDataSegmentLength is used during Login. 3691 The Login Phase sequence of requests and responses proceeds as 3692 follows: 3694 - Login initial request 3696 - Login partial response (optional) 3698 - More Login requests and responses (optional) 3700 - Login Final-Response (mandatory) 3702 The initial login request of any connection MUST include the 3703 InitiatorName key=value pair. The initial login request of the 3704 first connection of a session MAY also include the SessionType 3705 key=value pair. For any connection within a session whose type is 3706 not "Discovery", the first login request MUST also include the 3707 TargetName key=value pair. 3709 The Login Final-response accepts or rejects the Login request. 3711 The Login Phase MAY include a SecurityNegotiation stage and a 3712 LoginOperationalNegotiation stage and MUST include at least one of 3713 them, but the included stage MAY be empty except for the mandatory 3714 names. 3716 The login requests and responses contain a field (CSG) that 3717 indicates the current negotiation stage (SecurityNegotiation or 3718 LoginOperationalNegotiation). If both stages are used, the 3719 SecurityNegotiation MUST precede the LoginOperationalNegotiation. 3721 Some operational parameters can be negotiated outside the login 3722 through Text requests and responses. 3724 Authentication-related security keys (Section 12 ) MUST be 3725 completely negotiated within the Login Phase. The use of 3726 underlying IPsec security is specified in Section 9.3, in 3727 [RFC3723], and in [IPSEC-IPS}. iSCSI support for security within 3728 the protocol only consists of authentication in the Login Phase. 3730 In some environments, a target or an initiator is not interested 3731 in authenticating its counterpart. It is possible to bypass 3732 authentication through the Login request and response. 3734 The initiator and target MAY want to negotiate iSCSI 3735 authentication parameters. Once this negotiation is completed, the 3736 channel is considered secure. 3738 Most of the negotiation keys are only allowed in a specific stage. 3739 The SecurityNegotiation keys appear in Section 12 and the 3740 LoginOperationalNegotiation keys appear in Section 13. Only a 3741 limited set of keys (marked as Any-Stage in Section 13) may be 3742 used in any of the two stages. 3744 Any given Login request or response belongs to a specific stage; 3745 this determines the negotiation keys allowed with the request or 3746 response. It is considered to be a protocol error to send a key 3747 not allowed in the current stage. 3749 Stage transition is performed through a command exchange 3750 (request/response) that carries the T bit and the same CSG code. 3751 During this exchange, the next stage is selected by the target 3752 through the "next stage" code (NSG). The selected NSG MUST NOT 3753 exceed the value stated by the initiator. The initiator can 3754 request a transition whenever it is ready, but a target can only 3755 respond with a transition after one is proposed by the initiator. 3757 In a negotiation sequence, the T bit settings in one pair of login 3758 request-responses have no bearing on the T bit settings of the 3759 next pair. An initiator that has a T bit set to 1 in one pair and 3760 is answered with a T bit setting of 0 may issue the next request 3761 with T bit set to 0. 3763 When a transition is requested by the initiator and acknowledged 3764 by the target, both the initiator and target switch to the 3765 selected stage. 3767 Targets MUST NOT submit parameters that require an additional 3768 initiator login request in a login response with the T bit set to 3769 1. 3771 Stage transitions during login (including entering and exit) are 3772 only possible as outlined in the following table: 3774 +-----------------------------------------------------------+ 3775 |From To -> | Security | Operational | FullFeature | 3776 | | | | | | 3777 | V | | | | 3778 +-----------------------------------------------------------+ 3779 | (start) | yes | yes | no | 3780 +-----------------------------------------------------------+ 3781 | Security | no | yes | yes | 3782 +-----------------------------------------------------------+ 3783 | Operational | no | no | yes | 3784 +-----------------------------------------------------------+ 3786 The Login Final-Response that accepts a Login Request can only 3787 come as a response to a Login request with the T bit set to 1, and 3788 both the request and response MUST indicate FullFeaturePhase as 3789 the next phase via the NSG field. 3791 Neither the initiator nor the target should attempt to declare or 3792 negotiate a parameter more than once during login except for 3793 responses to specific keys that explicitly allow repeated key 3794 declarations (e.g., TargetAddress). An attempt to 3795 renegotiate/redeclare parameters not specifically allowed MUST be 3796 detected by the initiator and target. If such an attempt is 3797 detected by the target, the target MUST respond with Login reject 3798 (initiator error); if detected by the initiator, the initiator 3799 MUST drop the connection. 3801 6.3.1. Login Phase Start 3803 The Login Phase starts with a login request from the initiator to 3804 the target. The initial login request includes: 3806 -Protocol version supported by the initiator. 3808 -iSCSI Initiator Name and iSCSI Target Name 3810 -ISID, TSIH, and connection Ids 3812 -Negotiation stage that the initiator is ready to enter. 3814 A login may create a new session or it may add a connection to an 3815 existing session. Between a given iSCSI Initiator Node (selected 3816 only by an InitiatorName) and a given iSCSI target defined by an 3817 iSCSI TargetName and a Target Portal Group Tag, the login results 3818 are defined by the following table: 3820 +----------------------------------------------------------------+ 3821 |ISID | TSIH | CID | Target action | 3822 +----------------------------------------------------------------+ 3823 |new | non-zero | any | fail the login | 3824 | | | | ("session does not exist") | 3825 +----------------------------------------------------------------+ 3826 |new | zero | any | instantiate a new session | 3827 +----------------------------------------------------------------+ 3828 |existing| zero | any | do session reinstatement | 3829 | | | | (see Section 6.3.5) | 3830 +----------------------------------------------------------------+ 3831 |existing| non-zero | new | add a new connection to | 3832 | | existing | | the session | 3833 +----------------------------------------------------------------+ 3834 |existing| non-zero |existing| do connection reinstatement| 3835 | | existing | | (see Section 7.1.4.3) | 3836 +----------------------------------------------------------------+ 3837 |existing| non-zero | any | fail the login | 3838 | | new | | ("session does not exist") | 3839 +----------------------------------------------------------------+ 3841 Determination of "existing" or "new" are made by the target. 3843 Optionally, the login request may include: 3845 -Security parameters 3846 OR 3848 -iSCSI operational parameters 3849 AND/OR 3851 -The next negotiation stage that the initiator is ready to 3852 enter. 3854 The target can answer the login in the following ways: 3856 -Login Response with Login reject. This is an immediate 3857 rejection from the target that causes the connection to 3858 terminate and the session to terminate if this is the first 3859 (or only) connection of a new session. The T bit and the CSG 3860 and NSG fields are reserved. 3862 -Login Response with Login accept as a final response (T bit 3863 set to 1 and the NSG in both request and response are set to 3864 FullFeaturePhase). The response includes the protocol 3865 version supported by the target and the session ID, and may 3866 include iSCSI operational or security parameters (that 3867 depend on the current stage). 3869 -Login Response with Login Accept as a partial response (NSG 3870 not set to FullFeaturePhase in both request and response) 3871 that indicates the start of a negotiation sequence. The 3872 response includes the protocol version supported by the 3873 target and either security or iSCSI parameters (when no 3874 security mechanism is chosen) supported by the target. 3876 If the initiator decides to forego the SecurityNegotiation stage, 3877 it issues the Login with the CSG set to 3878 LoginOperationalNegotiation and the target may reply with a Login 3879 Response that indicates that it is unwilling to accept the 3880 connection (see Section 11.13) without SecurityNegotiation and 3881 will terminate the connection with a response of Authentication 3882 failure (see Section 11.13.5). 3884 If the initiator is willing to negotiate iSCSI security, but is 3885 unwilling to make the initial parameter proposal and may accept a 3886 connection without iSCSI security, it issues the Login with the T 3887 bit set to 1, the CSG set to SecurityNegotiation, and NSG set to 3888 LoginOperationalNegotiation. If the target is also ready to skip 3889 security, the login response only contains the 3890 TargetPortalGroupTag key (see Section 13.9), the T bit set to 1, 3891 the CSG set to SecurityNegotiation, and NSG set to 3892 LoginOperationalNegotiation. 3894 An initiator that chooses to operate without iSCSI security and 3895 with all the operational parameters taking the default values 3896 issues the Login with the T bit set to 1, the CSG set to 3897 LoginOperationalNegotiation, and NSG set to FullFeaturePhase. If 3898 the target is also ready to forego security and can finish its 3899 LoginOperationalNegotiation, the Login response has T bit set to 3900 1, the CSG set to LoginOperationalNegotiation, and NSG set to 3901 FullFeaturePhase in the next stage. 3903 During the Login Phase the iSCSI target MUST return the 3904 TargetPortalGroupTag key with the first Login Response PDU with 3905 which it is allowed to do so (i.e., the first Login Response 3906 issued after the first Login Request with the C bit set to 0) for 3907 all session types. The TargetPortalGroupTag key value indicates 3908 the iSCSI portal group servicing the Login Request PDU. If the 3909 reconfiguration of iSCSI portal groups is a concern in a given 3910 environment, the iSCSI initiator should use this key to ascertain 3911 that it had indeed initiated the Login Phase with the intended 3912 target portal group. 3914 6.3.2. iSCSI Security Negotiation 3916 The security exchange sets the security mechanism and 3917 authenticates the initiator user and the target to each other. The 3918 exchange proceeds according to the authentication method chosen in 3919 the negotiation phase and is conducted using the login requests' 3920 and responses' key=value parameters. 3922 An initiator directed negotiation proceeds as follows: 3924 -The initiator sends a login request with an ordered list of 3925 the options it supports (authentication algorithm). The 3926 options are listed in the initiator's order of preference. 3927 The initiator MAY also send private or public extension 3928 options. 3929 -The target MUST reply with the first option in the list it 3930 supports and is allowed to use for the specific initiator 3931 unless it does not support any in which case it MUST answer 3932 with "Reject" (see Section 6.2). The parameters are encoded 3933 in UTF8 as key=value. For security parameters, see Section 3934 12. 3936 -When the initiator considers that it is ready to conclude the 3937 SecurityNegotiation stage, it sets the T bit to 1 and the 3938 NSG to what it would like the next stage to be. The target 3939 will then set the T bit to 1 and set NSG to the next stage 3940 in the Login response when it finishes sending its security 3941 keys. The next stage selected will be the one the target 3942 selected. If the next stage is FullFeaturePhase, the target 3943 MUST respond with a Login Response with the TSIH value. 3945 If the security negotiation fails at the target, then the target 3946 MUST send the appropriate Login Response PDU. If the security 3947 negotiation fails at the initiator, the initiator SHOULD close the 3948 connection. 3950 It should be noted that the negotiation might also be directed by 3951 the target if the initiator does support security, but is not 3952 ready to direct the negotiation (propose options) - see Appendix B 3953 for an example. 3955 6.3.3. Operational Parameter Negotiation During the Login Phase 3957 Operational parameter negotiation during the login MAY be done: 3959 - Starting with the first Login request if the initiator does 3960 not propose any security/ integrity option. 3962 - Starting immediately after the security negotiation if the 3963 initiator and target perform such a negotiation. 3965 Operational parameter negotiation MAY involve several Login 3966 request-response exchanges started and terminated by the 3967 initiator. The initiator MUST indicate its intent to terminate the 3968 negotiation by setting the T bit to 1; the target sets the T bit 3969 to 1 on the last response. 3971 Even when the initiator indicates its intent to switch stage by 3972 setting the T bit to 1 in a Login request, the target MAY respond 3973 with a Login response with the T bit set to 0. In that case, the 3974 initiator SHOULD continue to set the T bit to 1 in subsequent 3975 Login requests (even empty) that it sends, until target sends a 3976 Login response with the T bit set to 1 or sends a key that 3977 requires initiator to set the T bit to 0. 3979 Some session specific parameters can only be specified during the 3980 Login Phase of the first connection of a session (i.e., begun by a 3981 login request that contains a zero-valued TSIH) - the leading 3982 Login Phase (e.g., the maximum number of connections that can be 3983 used for this session). 3985 A session is operational once it has at least one connection in 3986 FullFeaturePhase. New or replacement connections can only be added 3987 to a session after the session is operational. 3989 For operational parameters, see Section 13. 3991 6.3.4. Connection Reinstatement 3993 Connection reinstatement is the process of an initiator logging in 3994 with a ISID-TSIH-CID combination that is possibly active from the 3995 target's perspective, which causes the implicit logging out of the 3996 connection corresponding to the CID and reinstating a new Full 3997 Feature Phase iSCSI connection in its place (with the same CID). 3998 Thus, the TSIH in the Login Request PDU MUST be non-zero and CID 3999 does not change during a connection reinstatement. The Login 4000 request performs the logout function of the old connection if an 4001 explicit logout was not performed earlier. In sessions with a 4002 single connection, this may imply the opening of a second 4003 connection with the sole purpose of cleaning up the first. Targets 4004 MUST support opening a second connection even when they do not 4005 support multiple connections in Full Feature Phase if 4006 ErrorRecoveryLevel is 2 and SHOULD support opening a second 4007 connection if ErrorRecoveryLevel is less than 2. 4009 If the operational ErrorRecoveryLevel is 2, connection 4010 reinstatement enables future task reassignment. If the operational 4011 ErrorRecoveryLevel is less than 2, connection reinstatement is the 4012 replacement of the old CID without enabling task reassignment. In 4013 this case, all the tasks that were active on the old CID must be 4014 immediately terminated without further notice to the initiator. 4016 The initiator connection state MUST be CLEANUP_WAIT (Section 4017 8.1.3) when the initiator attempts a connection reinstatement. 4019 In practical terms, in addition to the implicit logout of the old 4020 connection, reinstatement is equivalent to a new connection login. 4022 6.3.5. Session Reinstatement, Closure, and Timeout 4024 Session reinstatement is the process of the initiator logging in 4025 with an ISID that is possibly active from the target's 4026 perspective. Thus implicitly logging out the session that 4027 corresponds to the ISID and reinstating a new iSCSI session in its 4028 place (with the same ISID). Therefore, the TSIH in the Login PDU 4029 MUST be zero to signal session reinstatement. Session 4030 reinstatement causes all the tasks that were active on the old 4031 session to be immediately terminated by the target without further 4032 notice to the initiator. 4034 The initiator session state MUST be FAILED (Section 8.3) when the 4035 initiator attempts a session reinstatement. 4037 Session closure is an event defined to be one of the following: 4039 - A successful "session close" logout. 4041 - A successful "connection close" logout for the last Full 4042 Feature Phase connection when no other connection in the 4043 session is waiting for cleanup (Section 8.2) and no tasks in 4044 the session are waiting for reassignment. 4046 Session timeout is an event defined to occur when the last 4047 connection state timeout expires and no tasks are waiting for 4048 reassignment. This takes the session to the FREE state (N6 4049 transition in the session state diagram). 4051 6.3.5.1. Loss of Nexus Notification 4053 The iSCSI layer provides the SCSI layer with the "I_T nexus loss" 4054 notification when any one of the following events happens: 4056 Successful completion of session reinstatement. 4057 Session closure event. 4058 Session timeout event. 4060 Certain SCSI object clearing actions may result due to the 4061 notification in the SCSI end nodes, as documented in Appendix E. 4063 6.3.6. Session Continuation and Failure 4065 Session continuation is the process by which the state of a 4066 preexisting session continues to be used by connection 4067 reinstatement (Section 6.3.4), or by adding a connection with a 4068 new CID. Either of these actions associates the new transport 4069 connection with the session state. 4071 Session failure is an event where the last Full Feature Phase 4072 connection reaches the CLEANUP_WAIT state (Section 8.2), or 4073 completes a successful recovery logout thus causing all active 4074 tasks (that are formerly allegiant to the connection) to start 4075 waiting for task reassignment. 4077 6.4. Operational Parameter Negotiation Outside the Login Phase 4079 Some operational parameters MAY be negotiated outside (after) the 4080 Login Phase. 4082 Parameter negotiation in Full Feature Phase is done through Text 4083 requests and responses. Operational parameter negotiation MAY 4084 involve several Text request-response exchanges, which the 4085 initiator always starts, terminates, and uses the same Initiator 4086 Task Tag. The initiator MUST indicate its intent to finish the 4087 negotiation by setting the F bit to 1; the target sets the F bit 4088 to 1 on the last response. 4090 If the target responds to a Text request with the F bit set to 1 4091 with a Text response with the F bit set to 0, the initiator should 4092 keep sending the Text request (even empty) with the F bit set to 4093 1, while it still wants to finish the negotiation, until it 4094 receives the Text response with the F bit set to 1. Responding to 4095 a Text request with the F bit set to 1 with an empty (no key=value 4096 pairs) response with the F bit set to 0 is discouraged. 4098 Even when the initiator indicates its intent to finish the 4099 negotiation by setting the F bit to 1 in a Text request, the 4100 target MAY respond with a Text response with the F bit set to 0. 4101 In that case, the initiator SHOULD continue to set the F bit to 1 4102 in subsequent Text requests (even empty) that it sends, until 4103 target sends the final Text response with the F bit set to 1. Note 4104 that in the same case of Text request with the F bit set to 1, 4105 target SHOULD NOT respond with an empty (no key=value pairs) Text 4106 response with the F bit set to 0, because such a response may 4107 cause the initiator to abandon negotiation. 4109 Targets MUST NOT submit parameters that require an additional 4110 initiator Text request in a Text response with the F bit set to 1. 4112 In a negotiation sequence, the F bit settings in one pair of Text 4113 request-responses have no bearing on the F bit settings of the 4114 next pair. An initiator that has the F bit set to 1 in a request 4115 and is being answered with an F bit setting of 0 may issue the 4116 next request with the F bit set to 0. 4118 Whenever the target responds with the F bit set to 0, it MUST set 4119 the Target Transfer Tag to a value other than the default 4120 0xffffffff. 4122 An initiator MAY reset an operational parameter negotiation by 4123 issuing a Text request with the Target Transfer Tag set to the 4124 value 0xffffffff after receiving a response with the Target 4125 Transfer Tag set to a value other than 0xffffffff. A target may 4126 reset an operational parameter negotiation by answering a Text 4127 request with a Reject PDU. 4129 Neither the initiator nor the target should attempt to declare or 4130 negotiate a parameter more than once during any negotiation 4131 sequence, except for responses to specific keys that explicitly 4132 allow repeated key declarations (e.g., TargetAddress). If detected 4133 by the target, this MUST result in a Reject PDU with a reason of 4134 "protocol error". The initiator MUST reset the negotiation as 4135 outlined above. 4137 Parameters negotiated by a text exchange negotiation sequence only 4138 become effective after the negotiation sequence is completed. 4140 7. iSCSI Error Handling and Recovery 4142 7.1. Overview 4144 7.1.1. Background 4146 The following two considerations prompted the design of much of 4147 the error recovery functionality in iSCSI: 4149 An iSCSI PDU may fail the digest check and be dropped, despite 4150 being received by the TCP layer. The iSCSI layer must 4151 optionally be allowed to recover such dropped PDUs. 4153 A TCP connection may fail at any time during the data 4154 transfer. All the active tasks must optionally be allowed 4155 to be continued on a different TCP connection within the 4156 same session. 4158 Implementations have considerable flexibility in deciding what 4159 degree of error recovery to support, when to use it and by which 4160 mechanisms to achieve the required behavior. Only the externally 4161 visible actions of the error recovery mechanisms must be 4162 standardized to ensure interoperability. 4164 This Section describes a general model for recovery in support of 4165 interoperability. See Appendix D for further detail on how the 4166 described model may be implemented. Compliant implementations do 4167 not have to match the implementation details of this model as 4168 presented, but the external behavior of such implementations must 4169 correspond to the externally observable characteristics of the 4170 presented model. 4172 7.1.2. Goals 4174 The major design goals of the iSCSI error recovery scheme are as 4175 follows: 4177 Allow iSCSI implementations to meet different requirements by 4178 defining a collection of error recovery mechanisms that 4179 implementations may choose from. 4181 Ensure interoperability between any two implementations 4182 supporting different sets of error recovery capabilities. 4184 Define the error recovery mechanisms to ensure command 4185 ordering even in the face of errors, for initiators that 4186 demand ordering. 4188 Do not make additions in the fast path, but allow moderate 4189 complexity in the error recovery path. 4191 Prevent both the initiator and target from attempting to 4192 recover the same set of PDUs at the same time. For example, 4193 there must be a clear "error recovery functionality 4194 distribution" between the initiator and target. 4196 7.1.3. Protocol Features and State Expectations 4198 The initiator mechanisms defined in connection with error recovery 4199 are: 4201 a) NOP-OUT to probe sequence numbers of the target (Section 4202 11.18) 4203 b) Command retry (Section 7.2.1) 4204 c) Recovery R2T support (Section 7.8) 4205 d) Requesting retransmission of status/data/R2T using the 4206 SNACK facility (Section 11.16) 4207 e) Acknowledging the receipt of the data (Section 11.16) 4208 f) Reassigning the connection allegiance of a task to a 4209 different TCP connection (Section 7.2.2) 4210 g) Terminating the entire iSCSI session to start afresh 4211 (Section 7.1.4.4) 4213 The target mechanisms defined in connection with error recovery 4214 are: 4216 a) NOP-IN to probe sequence numbers of the initiator (Section 4217 11.19) 4218 b) Requesting retransmission of data using the recovery R2T 4219 feature (Section 7) 4220 c) SNACK support (Section 11.16) 4221 d) Requesting that parts of read data be acknowledged (Section 4222 11.7.2) 4223 e) Allegiance reassignment support (Section 7.2.2) 4224 f) Terminating the entire iSCSI session to force the initiator 4225 to start over (Section 7.1.4.4) 4227 For any outstanding SCSI command, it is assumed that iSCSI, in 4228 conjunction with SCSI at the initiator, is able to keep enough 4229 information to be able to rebuild the command PDU, and that 4230 outgoing data is available (in host memory) for retransmission 4231 while the command is outstanding. It is also assumed that at the 4232 target, incoming data (read data) MAY be kept for recovery or it 4233 can be reread from a device server. 4235 It is further assumed that a target will keep the "status & sense" 4236 for a command it has executed if it supports status 4237 retransmission. 4239 A target that agrees to support data retransmission is expected to 4240 be prepared to retransmit the outgoing data (i.e., Data-In) on 4241 request until either the status for the completed command is 4242 acknowledged, or the data in question has been separately 4243 acknowledged. 4245 7.1.4. Recovery Classes 4247 iSCSI enables the following classes of recovery (in the order of 4248 increasing scope of affected iSCSI tasks): 4250 - Within a command (i.e., without requiring command restart). 4252 - Within a connection (i.e., without requiring the connection 4253 to be rebuilt, but perhaps requiring command restart). 4255 - Connection recovery (i.e., perhaps requiring connections to 4256 be rebuilt and commands to be reissued). 4258 - Session recovery. 4260 The recovery scenarios detailed in the rest of this Section are 4261 representative rather than exclusive. In every case, they detail 4262 the lowest class recovery that MAY be attempted. The implementer 4263 is left to decide under which circumstances to escalate to the 4264 next recovery class and/or what recovery classes to implement. 4265 Both the iSCSI target and initiator MAY escalate the error 4266 handling to an error recovery class, which impacts a larger number 4267 of iSCSI tasks in any of the cases identified in the following 4268 discussion. 4270 In all classes, the implementer has the choice of deferring errors 4271 to the SCSI initiator (with an appropriate response code), in 4272 which case the task, if any, has to be removed from the target and 4273 all the side-effects, such as ACA, must be considered. 4275 Use of within-connection and within-command recovery classes MUST 4276 NOT be attempted before the connection is in Full Feature Phase. 4278 In the detailed description of the recovery classes, the mandating 4279 terms (MUST, SHOULD, MAY, etc.) indicate normative actions to be 4280 executed if the recovery class is supported (see Section 7.1.5 for 4281 the related negotiation semantics) and used. 4283 7.1.4.1. Recovery Within-command 4285 At the target, the following cases lend themselves to within- 4286 command recovery: 4288 a) Lost data PDU - realized through one of the following: 4289 b) Data digest error - dealt with as specified in Section 7.8, 4290 using the option of a recovery R2T. 4291 c) Sequence reception timeout (no data or partial-data-and-no-F- 4292 bit) - considered an implicit sequence error and dealt with 4293 as specified in Section 7.9, using the option of a recovery 4294 R2T. 4295 d) Header digest error, which manifests as a sequence reception 4296 timeout or a sequence error - dealt with as specified in 4297 Section 7.9, using the option of a recovery R2T. 4299 At the initiator, the following cases lend themselves to within- 4300 command recovery: 4302 a) Lost data PDU or lost R2T - realized through one of the 4303 following: 4304 b) Data digest error - dealt with as specified in Section 7.8, 4305 using the option of a SNACK. 4306 c) Sequence reception timeout (no status) or response reception 4307 timeout - dealt with as specified in Section 7.9, using the 4308 option of a SNACK. 4309 d) Header digest error, which manifests as a sequence reception 4310 timeout or a sequence error - dealt with as specified in 4311 Section 7.9, using the option of a SNACK. 4313 To avoid a race with the target, which may already have a recovery 4314 R2T or a termination response on its way, an initiator SHOULD NOT 4315 originate a SNACK for an R2T based on its internal timeouts (if 4316 any). Recovery in this case is better left to the target. 4318 The timeout values used by the initiator and target are outside 4319 the scope of this document. Sequence reception timeout is 4320 generally a large enough value to allow the data sequence transfer 4321 to be complete. 4323 7.1.4.2. Recovery Within-connection 4325 At the initiator, the following cases lend themselves to within- 4326 connection recovery: 4328 a) Requests not acknowledged for a long time. Requests are 4329 acknowledged explicitly through ExpCmdSN or implicitly by 4330 receiving data and/or status. The initiator MAY retry non- 4331 acknowledged commands as specified in Section 7.2. 4332 b) Lost iSCSI numbered Response. It is recognized by either 4333 identifying a data digest error on a Response PDU or a Data- 4334 In PDU carrying the status, or by receiving a Response PDU 4335 with a higher StatSN than expected. In the first case, digest 4336 error handling is done as specified in Section 7.8 using the 4337 option of a SNACK. In the second case, sequence error 4338 handling is done as specified in Section 7.9, using the 4339 option of a SNACK. 4341 At the target, the following cases lend themselves to within- 4342 connection recovery: 4344 - Status/Response not acknowledged for a long time. The target 4345 MAY issue a NOP-IN (with a valid Target Transfer Tag or 4346 otherwise) that carries the next status sequence number it 4347 is going to use in the StatSN field. This helps the 4348 initiator detect any missing StatSN(s) and issue a SNACK for 4349 the status. 4351 The timeout values used by the initiator and the target are 4352 outside the scope of this document. 4354 7.1.4.3. Connection Recovery 4356 At an iSCSI initiator, the following cases lend themselves to 4357 connection recovery: 4359 a) TCP connection failure: The initiator MUST close the 4360 connection. It then MUST either implicitly or explicitly 4361 logout the failed connection with the reason code "remove the 4362 connection for recovery" and reassign connection allegiance 4363 for all commands still in progress associated with the failed 4364 connection on one or more connections (some or all of which 4365 MAY be newly established connections) using the "Task 4366 reassign" task management function (see Section 11.5.1). For 4367 an initiator, a command is in progress as long as it has not 4368 received a response or a Data-In PDU including status. 4370 Note: The logout function is mandatory. However, a new 4371 connection establishment is only mandatory if the failed 4372 connection was the last or only connection in the session. 4373 b) Receiving an Asynchronous Message that indicates one or all 4374 connections in a session has been dropped. The initiator 4375 MUST handle it as a TCP connection failure for the 4376 connection(s) referred to in the Message. 4378 At an iSCSI target, the following cases lend themselves to 4379 connection recovery: 4381 - TCP connection failure. The target MUST close the connection 4382 and, if more than one connection is available, the target 4383 SHOULD send an Asynchronous Message that indicates it has 4384 dropped the connection. Then, the target will wait for the 4385 initiator to continue recovery. 4387 7.1.4.4. Session Recovery 4389 Session recovery should be performed when all other recovery 4390 attempts have failed. Very simple initiators and targets MAY 4391 perform session recovery on all iSCSI errors and rely on recovery 4392 on the SCSI layer and above. 4394 Session recovery implies the closing of all TCP connections, 4395 internally aborting all executing and queued tasks for the given 4396 initiator at the target, terminating all outstanding SCSI commands 4397 with an appropriate SCSI service response at the initiator, and 4398 restarting a session on a new set of connection(s) (TCP connection 4399 establishment and login on all new connections). 4401 For possible clearing effects of session recovery on SCSI and 4402 iSCSI objects, refer to Appendix E. 4404 7.1.5. Error Recovery Hierarchy 4406 The error recovery classes described so far are organized into a 4407 hierarchy for ease in understanding and to limit the 4408 implementation complexity. With few and well defined recovery 4409 levels interoperability is easier to achieve. The attributes of 4410 this hierarchy are as follows: 4412 a) Each level is a superset of the capabilities of the 4413 previous level. For example, Level 1 support implies 4414 supporting all capabilities of Level 0 and more. 4415 b) As a corollary, supporting a higher error recovery level 4416 means increased sophistication and possibly an increase 4417 in resource requirements. 4418 c) Supporting error recovery level "n" is advertised and 4419 negotiated by each iSCSI entity by exchanging the text 4420 key "ErrorRecoveryLevel=n". The lower of the two 4421 exchanged values is the operational ErrorRecoveryLevel 4422 for the session. 4424 The following diagram represents the error recovery hierarchy. 4426 + 4427 / \ 4428 / 2 \ <-- Connection recovery 4429 +-----+ 4430 / 1 \ <-- Digest failure recovery 4431 +----------+ 4432 / 0 \ <-- Session failure recovery 4433 +---------------+ 4435 The following table lists the error recovery capabilities expected 4436 from the implementations that support each error recovery level. 4438 +-------------------+--------------------------------------------+ 4439 |ErrorRecoveryLevel | Associated Error recovery capabilities | 4440 +-------------------+--------------------------------------------+ 4441 | 0 | Session recovery class | 4442 | | (Session Recovery) | 4443 +-------------------+--------------------------------------------+ 4444 | 1 | Digest failure recovery (See Note below.) | 4445 | | plus the capabilities of ER Level 0 | 4446 +-------------------+--------------------------------------------+ 4447 | 2 | Connection recovery class | 4448 | | (Connection Recovery) | 4449 | | plus the capabilities of ER Level 1 | 4450 +-------------------+--------------------------------------------+ 4452 Note: Digest failure recovery is comprised of two recovery 4453 classes: Within-Connection recovery class (Recovery Within- 4454 connection) and Within-Command recovery class (Recovery Within- 4455 command). 4457 When a defined value of ErrorRecoveryLevel is proposed by an 4458 originator in a text negotiation, the originator MUST support the 4459 functionality defined for the proposed value and additionally, 4460 functionality corresponding to any defined value numerically less 4461 than the proposed. When a defined value of ErrorRecoveryLevel is 4462 returned by a responder in a text negotiation, the responder MUST 4463 support the functionality corresponding to the ErrorRecoveryLevel 4464 it is accepting. 4466 When either party attempts to use error recovery functionality 4467 beyond what is negotiated, the recovery attempts MAY fail unless 4468 an apriori agreement outside the scope of this document exists 4469 between the two parties to provide such support. 4471 Implementations MUST support error recovery level "0", while the 4472 rest are OPTIONAL to implement. In implementation terms, the 4473 above striation means that the following incremental 4474 sophistication with each level is required. 4476 +-------------------+--------------------------------------------+ 4477 |Level transition | Incremental requirement | 4478 +-------------------+--------------------------------------------+ 4479 | 0->1 | PDU retransmissions on the same connection| 4480 +-------------------+--------------------------------------------+ 4481 | 1->2 | Retransmission across connections and | 4482 | | allegiance reassignment | 4483 +-------------------+--------------------------------------------+ 4485 7.2. Retry and Reassign in Recovery 4487 This Section summarizes two important and somewhat related iSCSI 4488 protocol features used in error recovery. 4490 7.2.1. Usage of Retry 4492 By resending the same iSCSI command PDU ("retry") in the absence 4493 of a command acknowledgement (by way of an ExpCmdSN update) or a 4494 response, an initiator attempts to "plug" (what it thinks are) the 4495 discontinuities in CmdSN ordering on the target end. Discarded 4496 command PDUs, due to digest errors, may have created these 4497 discontinuities. 4499 Retry MUST NOT be used for reasons other than plugging command 4500 sequence gaps, and in particular, cannot be used for requesting 4501 PDU retransmissions from a target. Any such PDU retransmission 4502 requests for a currently allegiant command in progress may be made 4503 using the SNACK mechanism described in Section 11.16, although the 4504 usage of SNACK is OPTIONAL. 4506 If initiators, as part of plugging command sequence gaps as 4507 described above, inadvertently issue retries for allegiant 4508 commands already in progress (i.e., targets did not see the 4509 discontinuities in CmdSN ordering), the duplicate commands are 4510 silently ignored by targets as specified in Section 4.2.2.1. 4512 When an iSCSI command is retried, the command PDU MUST carry the 4513 original Initiator Task Tag and the original operational 4514 attributes (e.g., flags, function names, LUN, CDB etc.) as well as 4515 the original CmdSN. The command being retried MUST be sent on the 4516 same connection as the original command unless the original 4517 connection was already successfully logged out. 4519 7.2.2. Allegiance Reassignment 4521 By issuing a "task reassign" task management request (Section 4522 11.5.1), the initiator signals its intent to continue an already 4523 active command (but with no current connection allegiance) as part 4524 of connection recovery. This means that a new connection 4525 allegiance is requested for the command, which seeks to associate 4526 it to the connection on which the task management request is being 4527 issued. Before the allegiance reassignment is attempted for a 4528 task, an implicit or explicit Logout with the reason code "remove 4529 the connection for recovery" (see Section 11.14.1) MUST be 4530 successfully completed for the previous connection to which the 4531 task was allegiant. 4533 In reassigning connection allegiance for a command, the targets 4534 SHOULD continue the command from its current state. For example, 4535 when reassigning read commands, the target SHOULD take advantage 4536 of the ExpDataSN field provided by the Task Management function 4537 request (which must be set to zero if there was no data transfer) 4538 and bring the read command to completion by sending the remaining 4539 data and sending (or resending) the status. ExpDataSN 4540 acknowledges all data sent up to, but not including, the Data-In 4541 PDU and or R2T with DataSN (or R2TSN) equal to ExpDataSN. However, 4542 targets may choose to send/receive all unacknowledged data or all 4543 of the data on a reassignment of connection allegiance if unable 4544 to recover or maintain accurate state. Initiators MUST NOT 4545 subsequently request data retransmission through Data SNACK for 4546 PDUs numbered less than ExpDataSN (i.e., prior to the acknowledged 4547 sequence number). For all types of commands, a reassignment 4548 request implies that the task is still considered in progress by 4549 the initiator and the target must conclude the task appropriately 4550 if the target returns the "Function Complete" response to the 4551 reassignment request. This might possibly involve retransmission 4552 of data/R2T/status PDUs as necessary, but MUST involve the 4553 (re)transmission of the status PDU. 4555 It is OPTIONAL for targets to support the allegiance reassignment. 4556 This capability is negotiated via the ErrorRecoveryLevel text key 4557 during the login time. When a target does not support allegiance 4558 reassignment, it MUST respond with a Task Management response code 4559 of "Allegiance reassignment not supported". If allegiance 4560 reassignment is supported by the target, but the task is still 4561 allegiant to a different connection, or a successful recovery 4562 Logout of the previously allegiant connection was not performed, 4563 the target MUST respond with a Task Management response code of 4564 "Task still allegiant". 4566 If allegiance reassignment is supported by the target, the Task 4567 Management response to the reassignment request MUST be issued 4568 before the reassignment becomes effective. 4570 If a SCSI Command that involves data input is reassigned, any 4571 SNACK Tag it holds for a final response from the original 4572 connection is deleted and the default value of 0 MUST be used 4573 instead. 4575 7.3. Usage Of Reject PDU in Recovery 4577 Targets MUST NOT implicitly terminate an active task by sending a 4578 Reject PDU for any PDU exchanged during the life of the task. If 4579 the target decides to terminate the task, a Response PDU (SCSI, 4580 Text, Task, etc.) must be returned by the target to conclude the 4581 task. If the task had never been active before the Reject (i.e., 4582 the Reject is on the command PDU), targets should not send any 4583 further responses because the command itself is being discarded. 4585 The above rule means that the initiator can eventually expect a 4586 response on receiving Rejects, if the received Reject is for a PDU 4587 other than the command PDU itself. The non-command Rejects only 4588 have diagnostic value in logging the errors, and they can be used 4589 for retransmission decisions by the initiators. 4591 The CmdSN of the rejected command PDU (if it is a non-immediate 4592 command) MUST NOT be considered received by the target (i.e., a 4593 command sequence gap must be assumed for the CmdSN), even though 4594 the CmdSN of the rejected command PDU may be reliably ascertained. 4595 Upon receiving the Reject, the initiator MUST plug the CmdSN gap 4596 in order to continue to use the session. The gap may be plugged 4597 either by transmitting a command PDU with the same CmdSN, or by 4598 aborting the task (see SCS on how an abort may plug a CmdSN gap). 4600 When a data PDU is rejected and its DataSN can be ascertained, a 4601 target MUST advance ExpDataSN for the current data burst if a 4602 recovery R2T is being generated. The target MAY advance its 4603 ExpDataSN if it does not attempt to recover the lost data PDU. 4605 7.4. Error Recovery Considerations for Discovery Sessions 4607 7.4.1. ErrorRecoveryLevel for Discovery Sessions 4609 The negotiation of the key ErrorRecoveryLevel is not required for 4610 Discovery sessions -- i.e., for sessions that negotiated 4611 "SessionType=Discovery" -- because the default value of 0 is 4612 necessary and sufficient for Discovery sessions. It is however 4613 possible that some legacy iSCSI implementations might attempt to 4614 negotiate the ErrorRecoveryLevel key on Discovery sessions. When 4615 such a negotiation attempt is made by the remote side, a compliant 4616 iSCSI implementation MUST propose a value of 0 (zero) in response. 4617 The operational ErrorRecoveryLevel for Discovery sessions thus 4618 MUST 4619 be 0. This naturally follows from the functionality constraints 4620 that Section 4.3 imposes on Discovery sessions. 4622 7.4.2. Reinstatement Semantics for Discovery Sessions 4624 Discovery sessions are intended to be relatively short-lived. 4625 Initiators are not expected to establish multiple Discovery 4626 sessions to the same iSCSI Network Portal. An initiator may use 4627 the same iSCSI Initiator Name and ISID when establishing different 4628 unique sessions with different targets and/or different portal 4629 groups. This behavior is discussed in Section 10.1.1 and is, in 4630 fact, encouraged as conservative reuse of ISIDs. 4632 The ISID RULE in Section 4.4.3 states that there must not be more 4633 than one session with a matching 4-tuple: . While the spirit of the ISID 4635 RULE applies to Discovery sessions the same as it does for Normal 4636 sessions, note that some Discovery sessions differ from the Normal 4637 sessions in two important aspects: 4639 a) Because Appendix C allows a Discovery session to be 4640 established without specifying a TargetName key in the 4641 Login Request PDU (let us call such a session an "Unnamed" 4642 Discovery session), there is no Target Node context to 4643 enforce the ISID RULE. 4645 b) Portal Groups are defined only in the context of a Target 4646 Node. When the TargetName key is NULL-valued (i.e., not 4647 specified), the TargetPortalGroupTag thus cannot be 4648 ascertained to enforce the ISID RULE. 4650 The following two sections describe each of the two scenarios -- 4651 Named Discovery sessions and Unnamed Discovery sessions. 4653 7.4.2.1. Unnamed Discovery Sessions 4655 For Unnamed Discovery sessions, neither the TargetName nor the 4656 TargetPortalGroupTag is available to the targets in order to 4657 enforce the ISID RULE. So the following rule applies. 4659 UNNAMED ISID RULE: Targets MUST enforce the uniqueness of the 4660 following 4-tuple for Unnamed Discovery sessions: . The following semantics are implied by 4662 this uniqueness requirement. 4664 Targets SHOULD allow concurrent establishment of one Discovery 4665 session with each of its Network Portals by the same initiator 4666 port with a given iSCSI Node Name and an ISID. Each of the 4667 concurrent Discovery sessions, if established by the same 4668 initiator port to other Network Portals, MUST be treated as 4669 independent sessions -- i.e., one session MUST NOT reinstate the 4670 other. 4672 A new Unnamed Discovery session that has a matching 4673 to an existing 4674 Discovery session MUST reinstate the existing Unnamed Discovery 4675 session. Note thus that only an Unnamed Discovery session may 4676 reinstate an Unnamed Discovery session. 4678 7.4.2.2. Named Discovery Session 4680 For a Named Discovery session, the TargetName key is specified by 4681 the initiator and thus the target can unambiguously ascertain the 4682 TargetPortalGroupTag as well. Since all the four elements of the 4683 4-tuple are known, the ISID RULE MUST be enforced by targets with 4684 no changes from Section 4.4.3 semantics. A new session with a 4685 matching 4686 thus will reinstate an existing session. Note in this case that 4687 any new iSCSI session (Discovery or Normal) with the matching 4- 4688 tuple may reinstate an existing Named Discovery iSCSI session. 4690 7.4.3. Target PDUs During Discovery 4692 Targets SHOULD NOT send any responses other than a Text Response 4693 and Logout Response on a Discovery session, once in Full Feature 4694 Phase. 4696 Implementation Note: A target may simply drop the connection in a 4697 Discovery session when it would have requested a Logout via an 4698 Async Message on Normal sessions. 4700 7.5. Connection Timeout Management 4702 iSCSI defines two session-global timeout values (in seconds) - 4703 Time2Wait and Time2Retain - that are applicable when an iSCSI Full 4704 Feature Phase connection is taken out of service either 4705 intentionally or by an exception. Time2Wait is the initial 4706 "respite time" before attempting an explicit/implicit Logout for 4707 the CID in question or task reassignment for the affected tasks 4708 (if any). Time2Retain is the maximum time after the initial 4709 respite interval that the task and/or connection state(s) is/are 4710 guaranteed to be maintained on the target to cater to a possible 4711 recovery attempt. Recovery attempts for the connection and/or 4712 task(s) SHOULD NOT be made before Time2Wait seconds, but MUST be 4713 completed within Time2Retain seconds after that initial Time2Wait 4714 waiting period. 4716 7.5.1. Timeouts on Transport Exception Events 4718 A transport connection shutdown or a transport reset without any 4719 preceding iSCSI protocol interactions informing the end-points of 4720 the fact causes a Full Feature Phase iSCSI connection to be 4721 abruptly terminated. The timeout values to be used in this case 4722 are the negotiated values of DefaultTime2Wait (Section 13.15) and 4723 DefaultTime2Retain (Section 13.16) text keys for the session. 4725 7.5.2. Timeouts on Planned Decommissioning 4727 Any planned decommissioning of a Full Feature Phase iSCSI 4728 connection is preceded by either a Logout Response PDU, or an 4729 Async Message PDU. The Time2Wait and Time2Retain field values 4730 (Section 11.15) in a Logout Response PDU, and the Parameter2 and 4731 Parameter3 fields of an Async Message (AsyncEvent types "drop the 4732 connection" or "drop all the connections"; Section 11.9.1) specify 4733 the timeout values to be used in each of these cases. 4735 These timeout values are only applicable for the affected 4736 connection, and the tasks active on that connection. These 4737 timeout values have no bearing on initiator timers (if any) that 4738 are already running on connections or tasks associated with that 4739 session. 4741 7.6. Implicit Termination of Tasks 4743 A target implicitly terminates the active tasks due to iSCSI 4744 protocol dynamics in the following cases: 4746 a) When a connection is implicitly or explicitly logged out 4747 with the reason code of "Close the connection" and there 4748 are active tasks allegiant to that connection. 4750 b) When a connection fails and eventually the connection 4751 state times out (state transition M1 in Section 8.2.2) 4752 and there are active tasks allegiant to that connection. 4754 c) When a successful Logout with the reason code of "remove 4755 the connection for recovery" is performed while there are 4756 active tasks allegiant to that connection, and those 4757 tasks eventually time out after the Time2Wait and 4758 Time2Retain periods without allegiance reassignment. 4760 d) When a connection is implicitly or explicitly logged out 4761 with the reason code of "Close the session" and there are 4762 active tasks in that session. 4764 If the tasks terminated in the above cases a), b), c) and d)are 4765 SCSI tasks, they must be internally terminated as if with CHECK 4766 CONDITION status. This status is only meaningful for appropriately 4767 handling the internal SCSI state and SCSI side effects with 4768 respect to ordering because this status is never communicated back 4769 as a terminating status to the initiator. However additional 4770 actions may have to be taken at SCSI level depending on the SCSI 4771 context as defined by the SCSI standards (e.g., queued commands 4772 and ACA, UA for the next command on the I_T nexus in cases a), b), 4773 and c) etc. - see [SAM2] and [SPC3]). 4775 7.7. Format Errors 4777 The following two explicit violations of PDU layout rules are 4778 format errors: 4780 a) Illegal contents of any PDU header field except the 4781 Opcode (legal values are specified in Section 11). 4782 b) Inconsistent field contents (consistent field contents 4783 are specified in Section 11). 4785 Format errors indicate a major implementation flaw in one of the 4786 parties. 4788 When a target or an initiator receives an iSCSI PDU with a format 4789 error, it MUST immediately terminate all transport connections in 4790 the session either with a connection close or with a connection 4791 reset and escalate the format error to session recovery (see 4792 Section 7.1.4.4). 4794 All initiator-detected PDU construction errors MUST be considered 4795 as format errors. Some examples of such errors are: 4796 - NOP-In with a valid TTT but an invalid LUN 4798 - NOP-In with a valid ITT (i.e., a NOP-In response) and also a 4799 valid TTT 4801 - SCSI Response PDU with Status=CHECK CONDITION, but 4802 DataSegmentLength = 0 4804 7.8. Digest Errors 4806 The discussion of the legal choices in handling digest errors 4807 below excludes session recovery as an explicit option, but either 4808 party detecting a digest error may choose to escalate the error to 4809 session recovery. 4811 When a target or an initiator receives any iSCSI PDU, with a 4812 header digest error, it MUST either discard the header and all 4813 data up to the beginning of a later PDU or close the connection. 4814 Because the digest error indicates that the length field of the 4815 header may have been corrupted, the location of the beginning of a 4816 later PDU needs to be reliably ascertained by other means such as 4817 the operation of a sync and steering layer. 4819 When a target receives any iSCSI PDU with a payload digest error, 4820 it MUST answer with a Reject PDU with a reason code of Data- 4821 Digest-Error and discard the PDU. 4823 - If the discarded PDU is a solicited or unsolicited iSCSI 4824 data PDU (for immediate data in a command PDU, non-data PDU 4825 rule below applies), the target MUST do one of the 4826 following: 4828 i) Request retransmission with a recovery R2T. 4829 ii) Terminate the task with a response PDU with a CHECK 4830 CONDITION Status and an iSCSI Condition of "protocol 4831 service CRC error" (Section 11.4.7.2). If the target 4832 chooses to implement this option, it MUST wait to 4833 receive all the data (signaled by a Data PDU with the 4834 final bit set for all outstanding R2Ts) before sending 4836 the response PDU. A task management command (such as an 4837 abort task) from the initiator during this wait may 4838 also conclude the task. 4839 - No further action is necessary for targets if the discarded 4840 PDU is a non-data PDU. In case of immediate data being 4841 present on a discarded command, the immediate data is 4842 implicitly recovered when the task is retried (see Section 4843 7.2.1) followed by the entire data transfer for the task. 4845 When an initiator receives any iSCSI PDU with a payload digest 4846 error, it MUST discard the PDU. 4848 - If the discarded PDU is an iSCSI data PDU, the initiator 4849 MUST do one of the following: 4851 a) Request the desired data PDU through SNACK. In 4852 response to the SNACK, the target MUST either resend 4853 the data PDU or reject the SNACK with a Reject PDU 4854 with a reason code of "SNACK reject" in which case: 4855 b) If the status has not already been sent for the 4856 command, the target MUST terminate the command with a 4857 CHECK CONDITION Status and an iSCSI Condition of 4858 "SNACK rejected" (Section 11.4.7.2). 4859 c) If the status was already sent, no further action is 4860 necessary for the target. The initiator in this case 4861 MUST wait for the status to be received and then 4862 discard it, so as to internally signal the completion 4863 with CHECK CONDITION Status and an iSCSI Condition of 4864 "protocol service CRC error" (Section 11.4.7.2). 4865 d) Abort the task and terminate the command with an 4866 error. 4868 - If the discarded PDU is a response PDU or an unsolicited PDU 4869 (e.g. Async, Reject), the initiator MUST do one of the 4870 following: 4872 a) Request PDU retransmission with a status SNACK. 4873 b) Logout the connection for recovery and continue the 4874 tasks on a different connection instance as described 4875 in Section 7.2. 4877 c) Logout to close the connection (abort all the 4878 commands associated with the connection). 4880 Note that an unsolicited PDU carries the next StatSN value on 4881 an iSCSI connection, thereby advancing the StatSN. When an 4882 initiator discards one of these PDUs due to a payload digest 4883 error, the entire PDU including the header MUST be discarded. 4884 Consequently, the initiator MUST treat the exception like a 4885 loss of any other solicited response PDU. 4887 7.9. Sequence Errors 4889 When an initiator receives an iSCSI R2T/data PDU with an out of 4890 order R2TSN/DataSN or a SCSI response PDU with an ExpDataSN that 4891 implies missing data PDU(s), it means that the initiator must have 4892 detected a header or payload digest error on one or more earlier 4893 R2T/data PDUs. The initiator MUST address these implied digest 4894 errors as described in Section 7.8. When a target receives a data 4895 PDU with an out of order DataSN, it means that the target must 4896 have hit a header or payload digest error on at least one of the 4897 earlier data PDUs. The target MUST address these implied digest 4898 errors as described in Section 7.8. 4900 When an initiator receives an iSCSI status PDU with an out of 4901 order StatSN that implies missing responses, it MUST address the 4902 one or more missing status PDUs as described in Section 7.8. As a 4903 side effect of receiving the missing responses, the initiator may 4904 discover missing data PDUs. If the initiator wants to recover the 4905 missing data for a command, it MUST NOT acknowledge the received 4906 responses that start from the StatSN of the relevant command, 4907 until it has completed receiving all the data PDUs of the command. 4909 When an initiator receives duplicate R2TSNs (due to proactive 4910 retransmission of R2Ts by the target) or duplicate DataSNs (due to 4911 proactive SNACKs by the initiator), it MUST discard the 4912 duplicates. 4914 7.10. Message Error Checking 4916 In the iSCSI implementations till date, there has been some 4917 uncertainty on the extent to which incoming messages have to be 4918 checked for protocol errors, beyond what is strictly required for 4919 processing the inbound message. This Section addresses this 4920 question. 4922 Unless this document requires it, an iSCSI implementation is not 4923 required to do an exhaustive protocol conformance check on an 4924 incoming iSCSI PDU. The iSCSI implementation especially is not 4925 required to double-check the remote iSCSI implementation's 4926 conformance to protocol requirements. 4928 7.11. SCSI Timeouts 4930 An iSCSI initiator MAY attempt to plug a command sequence gap on 4931 the target end (in the absence of an acknowledgement of the 4932 command by way of ExpCmdSN) before the ULP timeout by retrying the 4933 unacknowledged command, as described in Section 7.2. 4935 On a ULP timeout for a command (that carried a CmdSN of n), if the 4936 iSCSI initiator intends to continue the session it MUST abort the 4937 command by either using an appropriate Task Management function 4938 request for the specific command, or a "close the connection" 4939 Logout. When using an ABORT TASK, if the ExpCmdSN is still less 4940 than (n+1), the target may see the abort request while missing the 4941 original command itself due to one of the following reasons: 4943 - Original command was dropped due to digest error. 4945 - Connection on which the original command was sent was 4946 successfully logged out. On logout, the unacknowledged 4947 commands issued on the connection being logged out are 4948 discarded. 4950 If the abort request is received and the original command is 4951 missing, targets MUST consider the original command with that 4952 RefCmdSN to be received and issue a Task Management response with 4953 the response code: "Function Complete". This response concludes 4954 the task on both ends. If the abort request is received and the 4955 target can determine (based on the Referenced Task Tag) that the 4956 command was received and executed and also that the response was 4957 sent prior to the abort, then the target MUST respond with the 4958 response code of "Task Does Not Exist". 4960 7.12. Negotiation Failures 4962 Text request and response sequences, when used to set/negotiate 4963 operational parameters, constitute the negotiation/parameter 4964 setting. A negotiation failure is considered to be one or more of 4965 the following: 4967 - None of the choices, or the stated value, is acceptable to 4968 one of the sides in the negotiation. 4970 - The text request timed out and possibly terminated. 4972 - The text request was answered with a Reject PDU. 4974 The following two rules should be used to address negotiation 4975 failures: 4977 a) During Login, any failure in negotiation MUST be 4978 considered a login process failure and the Login Phase 4979 MUST be terminated, and with it, the connection. If the 4980 target detects the failure, it must terminate the login 4981 with the appropriate login response code. 4983 b) A failure in negotiation, while in the Full Feature 4984 Phase, will terminate the entire negotiation sequence 4985 that may consist of a series of text requests that use 4986 the same Initiator Task Tag. The operational parameters 4987 of the session or the connection MUST continue to be the 4988 values agreed upon during an earlier successful 4989 negotiation (i.e., any partial results of this 4990 unsuccessful negotiation MUST NOT take effect and MUST be 4991 discarded). 4993 7.13. Protocol Errors 4995 Mapping framed messages over a "stream" connection, such as TCP, 4996 makes the proposed mechanisms vulnerable to simple software 4997 framing errors. On the other hand, the introduction of framing 4998 mechanisms to limit the effects of these errors may be onerous on 4999 performance for simple implementations. Command Sequence Numbers 5000 and the above mechanisms for connection drop and reestablishment 5001 help handle this type of mapping errors. 5003 All violations of iSCSI PDU exchange sequences specified in this 5004 draft are also protocol errors. This category of errors can only 5005 be 5006 addressed by fixing the implementations; iSCSI defines Reject and 5007 response codes to enable this. 5009 7.14. Connection Failures 5011 iSCSI can keep a session in operation if it is able to 5012 keep/establish at least one TCP connection between the initiator 5013 and the target in a timely fashion. Targets and/or initiators may 5014 recognize a failing connection by either transport level means 5015 (TCP), a gap in the command sequence number, a response stream 5016 that is not filled for a long time, or by a failing iSCSI NOP 5017 (acting as a ping). The latter MAY be used periodically to 5018 increase the speed and likelihood of detecting connection 5019 failures. As an example for transport level means, initiators and 5020 targets MAY also use the keep-alive option, see [RFC1122], on the 5021 TCP connection to enable early link failure detection on otherwise 5022 idle links. 5024 On connection failure, the initiator and target MUST do one of the 5025 following: 5027 a) Attempt connection recovery within the session 5028 (Connection Recovery). 5030 b) Logout the connection with the reason code "closes the 5031 connection" (Section 10.14.5), re-issue missing commands, 5032 and implicitly terminate all active commands. This option 5033 requires support for the within-connection recovery class 5034 (Recovery Within-connection). 5036 c) Perform session recovery (Session Recovery). 5038 Either side may choose to escalate to session recovery (via the 5039 initiator dropping all the connections, or via an Async Message 5040 that announces the similar intent from a target), and the other 5041 side MUST give it precedence. On a connection failure, a target 5042 MUST terminate and/or discard all of the active immediate commands 5043 regardless of which of the above options is used (i.e., immediate 5044 commands are not recoverable across connection failures). 5046 7.15. Session Errors 5048 If all of the connections of a session fail and cannot be 5049 reestablished in a short time, or if initiators detect protocol 5050 errors repeatedly, an initiator may choose to terminate a session 5051 and establish a new session. 5053 In this case, the initiator takes the following actions: 5055 - Resets or closes all the transport connections. 5057 - Terminates all outstanding requests with an appropriate 5058 response before initiating a new session. If the same I_T 5059 nexus is intended to be reestablished, the initiator MUST 5060 employ session reinstatement (see Section 6.3.5). 5062 When the session timeout (the connection state timeout for the 5063 last failed connection) happens on the target, it takes the 5064 following actions: 5066 - Resets or closes the TCP connections (closes the session). 5068 - Terminates all active tasks that were allegiant to the 5069 connection(s) that constituted the session. 5071 A target MUST also be prepared to handle a session reinstatement 5072 request from the initiator that may be addressing session errors. 5074 8. State Transitions 5076 iSCSI connections and iSCSI sessions go through several well- 5077 defined states from the time they are created to the time they are 5078 cleared. 5080 The connection state transitions are described in two separate but 5081 dependent state diagrams for ease in understanding. The first 5082 diagram, "standard connection state diagram", describes the 5083 connection state transitions when the iSCSI connection is not 5084 waiting for, or undergoing, a cleanup by way of an explicit or 5085 implicit Logout. The second diagram, "connection cleanup state 5086 diagram", describes the connection state transitions while 5087 performing the iSCSI connection cleanup. 5089 The "session state diagram" describes the state transitions an 5090 iSCSI session would go through during its lifetime, and it depends 5091 on the states of possibly multiple iSCSI connections that 5092 participate in the session. 5094 States and transitions are described in text, tables and diagrams. 5095 The diagrams are used for illustration. The text and the tables 5096 are the governing specification. 5098 8.1. Standard Connection State Diagrams 5100 8.1.1. State Descriptions for Initiators and Targets 5102 State descriptions for the standard connection state diagram are 5103 as follows: 5104 -S1: FREE 5105 -initiator: State on instantiation, or after successful 5106 connection closure. 5107 -target: State on instantiation, or after successful 5108 connection closure. 5109 -S2: XPT_WAIT 5110 -initiator: Waiting for a response to its transport 5111 connection establishment request. 5112 -target: Illegal 5113 -S3: XPT_UP 5114 -initiator: Illegal 5115 -target: Waiting for the Login process to commence. 5117 -S4: IN_LOGIN 5118 -initiator: Waiting for the Login process to conclude, 5119 possibly involving several PDU exchanges. 5120 -target: Waiting for the Login process to conclude, 5121 possibly involving several PDU exchanges. 5122 -S5: LOGGED_IN 5123 -initiator: In Full Feature Phase, waiting for all 5124 internal, iSCSI, and transport events. 5125 -target: In Full Feature Phase, waiting for all internal, 5126 iSCSI, and transport events. 5127 -S6: IN_LOGOUT 5128 -initiator: Waiting for a Logout response. 5129 -target: Waiting for an internal event signaling completion 5130 of logout processing. 5131 -S7: LOGOUT_REQUESTED 5132 -initiator: Waiting for an internal event signaling 5133 readiness to proceed with Logout. 5134 -target: Waiting for the Logout process to start after 5135 having requested a Logout via an Async Message. 5136 -S8: CLEANUP_WAIT 5137 -initiator: Waiting for the context and/or resources to 5138 initiate the cleanup processing for this CSM. 5139 -target: Waiting for the cleanup process to start for this 5140 CSM. 5141 8.1.2. State Transition Descriptions for Initiators and Targets 5143 -T1: 5144 -initiator: Transport connect request was made (e.g., TCP 5145 SYN sent). 5146 -target: Illegal 5147 -T2: 5148 -initiator: Transport connection request timed out, a 5149 transport reset was received, or an internal event of 5150 receiving a Logout response (success) on another connection 5151 for a "close the session" Logout request was received. 5152 -target:Illegal 5153 -T3: 5154 -initiator: Illegal 5155 -target: Received a valid transport connection request that 5156 establishes the transport connection. 5157 -T4: 5159 -initiator: Transport connection established, thus 5160 prompting the initiator to start the iSCSI Login. 5161 -target: Initial iSCSI Login request was received. 5162 -T5: 5163 -initiator: The final iSCSI Login response with a Status- 5164 Class of zero was received. 5165 -target: The final iSCSI Login request to conclude the 5166 Login Phase was received, thus prompting the target to send 5167 the final iSCSI Login response with a Status-Class of zero. 5168 -T6: 5169 -initiator: Illegal 5170 -target: Timed out waiting for an iSCSI Login, transport 5171 disconnect indication was received, transport reset was 5172 received, or an internal event indicating a transport 5173 timeout was received. In all these cases, the connection is 5174 to be closed. 5175 -T7: 5176 -initiator - one of the following events caused the 5177 transition: 5178 a) The final iSCSI Login response was received with a 5179 non-zero Status-Class. 5180 b) Login timed out. 5181 c) A transport disconnect indication was received. 5182 d) A transport reset was received. 5183 e) An internal event indicating a transport timeout was 5184 received. 5185 f) An internal event of receiving a Logout response 5186 (success) on another connection for a "close the 5187 session" Logout request was received. 5189 In all these cases, the transport connection is closed. 5191 -target - one of the following events caused the 5192 transition: 5193 a) The final iSCSI Login request to conclude the Login 5194 Phase was received, prompting the target to send the 5195 final iSCSI Login response with a non-zero Status- 5196 Class. 5197 b) Login timed out. 5198 c) Transport disconnect indication was received. 5200 d) Transport reset was received. 5201 e) An internal event indicating a transport timeout was 5202 received. 5203 f) On another connection, a "close the session" Logout 5204 request was received. 5206 In all these cases, the connection is to be closed. 5207 -T8: 5208 -initiator: An internal event of receiving a Logout 5209 response (success) on another connection for a "close the 5210 session" Logout request was received, thus closing this 5211 connection requiring no further cleanup. 5212 -target: An internal event of sending a Logout response 5213 (success) on another connection for a "close the session" 5214 Logout request was received, or an internal event of a 5215 successful connection/session reinstatement is received, 5216 thus prompting the target to close this connection cleanly. 5217 -T9, T10: 5218 -initiator: An internal event that indicates the readiness 5219 to start the Logout process was received, thus prompting an 5220 iSCSI Logout to be sent by the initiator. 5221 -target: An iSCSI Logout request was received. 5222 -T11, T12: 5223 -initiator: Async PDU with AsyncEvent "Request Logout" was 5224 received. 5225 -target: An internal event that requires the 5226 decommissioning of the connection is received, thus causing 5227 an Async PDU with an AsyncEvent "Request Logout" to be 5228 sent. 5229 -T13: 5230 -initiator: An iSCSI Logout response (success) was 5231 received, or an internal event of receiving a Logout 5232 response (success) on another connection for a "close the 5233 session" Logout request was received. 5234 -target: An internal event was received that indicates 5235 successful processing of the Logout, which prompts an iSCSI 5236 Logout response (success) to be sent; an internal event of 5237 sending a Logout response (success) on another connection 5238 for a "close the session" Logout request was received; or 5239 an internal event of a successful connection/session 5240 reinstatement is received. In all these cases, the 5241 transport connection is closed. 5243 -T14: 5244 -initiator: Async PDU with AsyncEvent "Request Logout" was 5245 received again. 5246 -target: Illegal 5247 -T15, T16: 5248 -initiator: One or more of the following events caused this 5249 transition: 5250 a) Internal event that indicates a transport connection 5251 timeout was received thus prompting transport RESET 5252 or transport connection closure. 5253 b) A transport RESET. 5254 c) A transport disconnect indication. 5255 d) Async PDU with AsyncEvent "Drop connection" (for 5256 this CID). 5257 e) Async PDU with AsyncEvent "Drop all connections". 5258 -target: One or more of the following events caused this 5259 transition: 5260 a) Internal event that indicates a transport connection 5261 timeout was received, thus prompting transport RESET 5262 or transport connection closure. 5263 b) An internal event of a failed connection/session 5264 reinstatement is received. 5265 c) A transport RESET. 5266 d) A transport disconnect indication. 5267 e) Internal emergency cleanup event was received which 5268 prompts an Async PDU with AsyncEvent "Drop 5269 connection" (for this CID), or event "Drop all 5270 connections". 5272 -T17: 5273 -initiator: One or more of the following events caused this 5274 transition: 5275 a) Logout response, (failure i.e., a non-zero status) 5276 was received, or Logout timed out. 5277 b) Any of the events specified for T15 and T16. 5278 -target: One or more of the following events caused this 5279 transition: 5281 a) Internal event that indicates a failure of the 5282 Logout processing was received, which prompts a 5283 Logout response (failure, i.e., a non-zero status) 5284 to be sent. 5285 b) Any of the events specified for T15 and T16. 5286 -T18: 5287 -initiator: An internal event of receiving a Logout 5288 response (success) on another connection for a "close the 5289 session" Logout request was received. 5291 -target: An internal event of sending a Logout response 5292 (success) on another connection for a "close the session" 5293 Logout request was received, or an internal event of a 5294 successful connection/session reinstatement is received. 5295 In both these cases, the connection is closed. 5297 The CLEANUP_WAIT state (S8) implies that there are possible iSCSI 5298 tasks that have not reached conclusion and are still considered 5299 busy. 5301 8.1.3. Standard Connection State Diagram for an Initiator 5303 Symbolic names for States: 5305 S1: FREE 5307 S2: XPT_WAIT 5309 S4: IN_LOGIN 5311 S5: LOGGED_IN 5313 S6: IN_LOGOUT 5315 S7: LOGOUT_REQUESTED 5317 S8: CLEANUP_WAIT 5319 States S5, S6, and S7 constitute the Full Feature Phase operation 5320 of the connection. 5322 The state diagram is as follows: 5324 -------<-------------+ 5325 +--------->/ S1 \<----+ | 5326 T13| +->\ /<-+ \ | 5327 | / ---+--- \ \ | 5328 | / | T2 \ | | 5329 | T8 | |T1 | | | 5330 | | | / |T7 | 5331 | | | / | | 5332 | | | / | | 5333 | | V / / | 5334 | | ------- / / | 5335 | | / S2 \ / | 5336 | | \ / / | 5337 | | ---+--- / | 5338 | | |T4 / | 5339 | | V / | T18 5340 | | ------- / | 5341 | | / S4 \ | 5342 | | \ / | 5343 | | ---+--- | T15 5344 | | |T5 +--------+---------+ 5345 | | | /T16+-----+------+ | 5346 | | | / -+-----+--+ | | 5347 | | | / / S7 \ |T12| | 5348 | | | / +->\ /<-+ V V 5349 | | | / / -+----- ------- 5350 | | | / /T11 |T10 / S8 \ 5351 | | V / / V +----+ \ / 5352 | | ---+-+- ----+-- | ------- 5353 | | / S5 \T9 / S6 \<+ ^ 5354 | +-----\ /--->\ / T14 | 5355 | ------- --+----+------+T17 5356 +---------------------------+ 5358 The following state transition table represents the above diagram. 5359 Each row represents the starting state for a given transition, 5360 which after taking a transition marked in a table cell would end 5361 in the state represented by the column of the cell. For example, 5362 from state S1, the connection takes the T1 transition to arrive at 5363 state S2. The fields marked "-" correspond to undefined 5364 transitions. 5366 +----+---+---+---+---+----+---+ 5367 |S1 |S2 |S4 |S5 |S6 |S7 |S8 | 5368 ---+----+---+---+---+---+----+---+ 5369 S1| - |T1 | - | - | - | - | - | 5370 ---+----+---+---+---+---+----+---+ 5371 S2|T2 |- |T4 | - | - | - | - | 5372 ---+----+---+---+---+---+----+---+ 5373 S4|T7 |- |- |T5 | - | - | - | 5374 ---+----+---+---+---+---+----+---+ 5375 S5|T8 |- |- | - |T9 |T11 |T15| 5376 ---+----+---+---+---+---+----+---+ 5377 S6|T13 |- |- | - |T14|- |T17| 5378 ---+----+---+---+---+---+----+---+ 5379 S7|T18 |- |- | - |T10|T12 |T16| 5380 ---+----+---+---+---+---+----+---+ 5381 S8| - |- |- | - | - | - | - | 5382 ---+----+---+---+---+---+----+---+ 5384 8.1.4. Standard Connection State Diagram for a Target 5386 Symbolic names for States: 5387 S1: FREE 5389 S3: XPT_UP 5391 S4: IN_LOGIN 5393 S5: LOGGED_IN 5395 S6: IN_LOGOUT 5397 S7: LOGOUT_REQUESTED 5399 S8: CLEANUP_WAIT 5401 States S5, S6, and S7 constitute the Full Feature Phase operation 5402 of the connection. 5404 The state diagram is as follows: 5406 -------<-------------+ 5407 +--------->/ S1 \<----+ | 5408 T13| +->\ /<-+ \ | 5409 | / ---+--- \ \ | 5410 | / | T6 \ | | 5411 | T8 | |T3 | | | 5412 | | | / |T7 | 5413 | | | / | | 5414 | | | / | | 5415 | | V / / | 5416 | | ------- / / | 5417 | | / S3 \ / | 5418 | | \ / / | T18 5419 | | ---+--- / | 5420 | | |T4 / | 5421 | | V / | 5422 | | ------- / | 5423 | | / S4 \ | 5424 | | \ / | 5425 | | ---+--- T15 | 5426 | | |T5 +--------+---------+ 5427 | | | /T16+-----+------+ | 5428 | | | / -+-----+---+ | | 5429 | | | / / S7 \ |T12| | 5430 | | | / +->\ /<-+ V V 5431 | | | / / -+----- ------- 5432 | | | / /T11 |T10 / S8 \ 5433 | | V / / V \ / 5434 | | ---+-+- ------- ------- 5435 | | / S5 \T9 / S6 \ ^ 5436 | +-----\ /--->\ / | 5437 | ------- --+----+--------+T17 5438 +---------------------------+ 5440 The following state transition table represents the above diagram, 5441 and follows the conventions described for the initiator diagram. 5443 +----+---+---+---+---+----+---+ 5444 |S1 |S3 |S4 |S5 |S6 |S7 |S8 | 5445 ---+----+---+---+---+---+----+---+ 5446 S1| - |T3 | - | - | - | - | - | 5447 ---+----+---+---+---+---+----+---+ 5448 S3|T6 |- |T4 | - | - | - | - | 5449 ---+----+---+---+---+---+----+---+ 5450 S4|T7 |- |- |T5 | - | - | - | 5451 ---+----+---+---+---+---+----+---+ 5452 S5|T8 |- |- | - |T9 |T11 |T15| 5453 ---+----+---+---+---+---+----+---+ 5454 S6|T13 |- |- | - |- |- |T17| 5455 ---+----+---+---+---+---+----+---+ 5456 S7|T18 |- |- | - |T10|T12 |T16| 5457 ---+----+---+---+---+---+----+---+ 5458 S8| - |- |- | - | - | - | - | 5459 ---+----+---+---+---+---+----+---+ 5461 8.2. Connection Cleanup State Diagram for Initiators and Targets 5463 Symbolic names for states: 5465 R1: CLEANUP_WAIT (same as S8) 5467 R2: IN_CLEANUP 5469 R3: FREE (same as S1) 5471 Whenever a connection state machine in cleanup (let's call it CSM- 5472 C) enters the CLEANUP_WAIT state (S8), it must go through the 5473 state transitions described in the connection cleanup state 5474 diagram either a) using a separate full-feature phase connection 5475 (let's call it CSM-E, for explicit) in the LOGGED_IN state in the 5476 same session, or b) using a new transport connection (let's call 5477 it CSM-I, for implicit) in the FREE state that is to be added to 5478 the same session. In the CSM-E case, an explicit logout for the 5479 CID that corresponds to CSM-C (either as a connection or session 5480 logout) needs to be performed to complete the cleanup. In the CSM- 5481 I case, an implicit logout for the CID that corresponds to CSM-C 5482 needs to be performed by way of connection reinstatement (Section 5483 6.3.4) for that CID. In either case, the protocol exchanges on 5484 CSM-E or CSM-I determine the state transitions for CSM-C. 5485 Therefore, this cleanup state diagram is only applicable to the 5486 instance of the connection in cleanup (i.e., CSM-C). In the case 5487 of an implicit logout for example, CSM-C reaches FREE (R3) at the 5488 time CSM-I reaches LOGGED_IN. In the case of an explicit logout, 5489 CSM-C reaches FREE (R3) when CSM-E receives a successful logout 5490 response while continuing to be in the LOGGED_IN state. 5492 An initiator must initiate an explicit or implicit connection 5493 logout for a connection in the CLEANUP_WAIT state, if the 5494 initiator intends to continue using the associated iSCSI session. 5496 The following state diagram applies to both initiators and 5497 targets. 5499 ------- 5500 / R1 \ 5501 +--\ /<-+ 5502 / ---+--- \ 5503 / | \ M3 5504 M1 | |M2 | 5505 | | / 5506 | | / 5507 | | / 5508 | V / 5509 | ------- / 5510 | / R2 \ 5511 | \ / 5512 | ------- 5513 | | 5514 | |M4 5515 | | 5516 | | 5517 | | 5518 | V 5519 | ------- 5520 | / R3 \ 5521 +---->\ / 5522 ------- 5524 The following state transition table represents the above diagram, 5525 and follows the same conventions as in earlier sections. 5527 +----+----+----+ 5528 |R1 |R2 |R3 | 5529 -----+----+----+----+ 5530 R1 | - |M2 |M1 | 5531 -----+----+----+----+ 5532 R2 |M3 | - |M4 | 5533 -----+----+----+----+ 5534 R3 | - | - | - | 5535 -----+----+----+----+ 5537 8.2.1. State Descriptions for Initiators and Targets 5539 -R1: CLEANUP_WAIT (Same as S8) 5540 -initiator: Waiting for the internal event to initiate the 5541 cleanup processing for CSM-C. 5542 -target: Waiting for the cleanup process to start for CSM- 5543 C. 5544 -R2: IN_CLEANUP 5545 -initiator: Waiting for the connection cleanup process to 5546 conclude for CSM-C. 5547 -target: Waiting for the connection cleanup process to 5548 conclude for CSM-C. 5549 -R3: FREE (Same as S1) 5550 -initiator: End state for CSM-C. 5551 -target: End state for CSM-C. 5553 8.2.2. State Transition Descriptions for Initiators and Targets 5555 -M1: One or more of the following events was received: 5556 -initiator: 5557 -An internal event that indicates connection state 5558 timeout. 5559 -An internal event of receiving a successful Logout 5560 response on a different connection for a "close the 5561 session" Logout. 5562 -target: 5563 -An internal event that indicates connection state 5564 timeout. 5565 -An internal event of sending a Logout response 5566 (success) on a different connection for a "close the 5567 session" Logout request. 5569 -M2: An implicit/explicit logout process was initiated by the 5570 initiator. 5571 -In CSM-I usage: 5572 -initiator: An internal event requesting the connection 5573 (or session) reinstatement was received, thus prompting 5574 a connection (or session) reinstatement Login to be 5575 sent transitioning CSM-I to state IN_LOGIN. 5576 -target: A connection/session reinstatement Login was 5577 received while in state XPT_UP. 5578 -In CSM-E usage: 5580 -initiator: An internal event that indicates that an 5581 explicit logout was sent for this CID in state 5582 LOGGED_IN. 5583 -target: An explicit logout was received for this CID 5584 in state LOGGED_IN. 5585 -M3: Logout failure detected 5586 -In CSM-I usage: 5587 -initiator: CSM-I failed to reach LOGGED_IN and arrived 5588 into FREE instead. 5589 -target: CSM-I failed to reach LOGGED_IN and arrived 5590 into FREE instead. 5591 -In CSM-E usage: 5592 -initiator: CSM-E either moved out of LOGGED_IN, or 5593 Logout timed out and/or aborted, or Logout response 5594 (failure) was received. 5595 -target: CSM-E either moved out of LOGGED_IN, Logout 5596 timed out and/or aborted, or an internal event that 5597 indicates a failed Logout processing was received. A 5598 Logout response (failure) was sent in the last case. 5600 -M4: Successful implicit/explicit logout was performed. 5601 - In CSM-I usage: 5602 -initiator: CSM-I reached state LOGGED_IN, or an 5603 internal event of receiving a Logout response (success) 5604 on another connection for a "close the session" Logout 5605 request was received. 5606 -target: CSM-I reached state LOGGED_IN, or an internal 5607 event of sending a Logout response (success) on a 5608 different connection for a "close the session" Logout 5609 request was received. 5610 - In CSM-E usage: 5611 -initiator: CSM-E stayed in LOGGED_IN and received a 5612 Logout response (success), or an internal event of 5613 receiving a Logout response (success) on another 5614 connection for a "close the session" Logout request was 5615 received. 5616 -target: CSM-E stayed in LOGGED_IN and an internal 5617 event indicating a successful Logout processing was 5618 received, or an internal event of sending a Logout 5619 response (success) on a different connection for a 5620 "close the session" Logout request was received. 5622 8.3. Session State Diagrams 5624 8.3.1. Session State Diagram for an Initiator 5626 Symbolic Names for States: 5628 Q1: FREE 5630 Q3: LOGGED_IN 5632 Q4: FAILED 5634 State Q3 represents the Full Feature Phase operation of the 5635 session. 5637 The state diagram is as follows: 5639 ------- 5640 / Q1 \ 5641 +------>\ /<-+ 5642 / ---+--- | 5643 / | |N3 5644 N6 | |N1 | 5645 | | | 5646 | N4 | | 5647 | +----------+ | / 5648 | | | | / 5649 | | | | / 5650 | | V V / 5651 -+--+-- -----+- 5652 / Q4 \ N5 / Q3 \ 5653 \ /<------\ / 5654 ------- ------- 5656 The state transition table is as follows: 5658 +---+---+---+ 5659 |Q1 |Q3 |Q4 | 5660 -----+---+---+---+ 5661 Q1 | - |N1 | - | 5662 -----+---+---+---+ 5663 Q3 |N3 | - |N5 | 5664 -----+---+---+---+ 5665 Q4 |N6 |N4 | - | 5666 -----+---+---+---+ 5668 8.3.2. Session State Diagram for a Target 5670 Symbolic Names for States: 5672 Q1: FREE 5674 Q2: ACTIVE 5676 Q3: LOGGED_IN 5678 Q4: FAILED 5680 Q5: IN_CONTINUE 5682 State Q3 represents the Full Feature Phase operation of the 5683 session. 5685 The state diagram is as follows: 5687 ------- 5688 +------------------>/ Q1 \ 5689 / +-------------->\ /<-+ 5690 | | ---+--- | 5691 | | ^ | |N3 5692 N 6 | |N11 N9| V N1 | 5693 | | +------ | 5694 | | / Q2 \ | 5695 | | \ / | 5696 | --+---- +--+--- | 5697 | / Q5 \ | | 5698 | \ / N10 | | 5699 | +-+---+------------+ | N2 / 5700 | ^ | | | / 5701 | N7| |N8 | | / 5702 | | | | V / 5703 -+---+-V V------+- 5704 / Q4 \ N5 / Q3 \ 5705 \ /<-------------\ / 5706 ------- ------- 5708 The state transition table is as follows: 5710 +----+----+----+----+----+ 5711 |Q1 |Q2 |Q3 |Q4 |Q5 | 5712 -----+----+----+----+----+----+ 5713 Q1 | - |N1 | - | - | - | 5714 -----+----+----+----+----+----+ 5715 Q2 |N9 | - |N2 | - | - | 5716 -----+----+----+----+----+----+ 5717 Q3 |N3 | - | - |N5 | - | 5718 -----+----+----+----+----+----+ 5719 Q4 |N6 | - | - | - |N7 | 5720 -----+----+----+----+----+----+ 5721 Q5 |N11 | - |N10 |N8 | - | 5722 -----+----+----+----+----+----+ 5724 8.3.3. State Descriptions for Initiators and Targets 5726 -Q1: FREE 5727 -initiator: State on instantiation or after cleanup. 5729 -target: State on instantiation or after cleanup. 5730 -Q2: ACTIVE 5731 -initiator: Illegal. 5732 -target: The first iSCSI connection in the session 5733 transitioned to IN_LOGIN, waiting for it to complete the 5734 login process. 5735 -Q3: LOGGED_IN 5736 -initiator: Waiting for all session events. 5737 -target: Waiting for all session events. 5738 -Q4: FAILED 5739 -initiator: Waiting for session recovery or session 5740 continuation. 5741 -target: Waiting for session recovery or session 5742 continuation. 5743 -Q5: IN_CONTINUE 5744 -initiator: Illegal. 5745 -target: Waiting for session continuation attempt to reach 5746 a conclusion. 5748 8.3.4. State Transition Descriptions for Initiators and Targets 5750 -N1: 5751 -initiator: At least one transport connection reached the 5752 LOGGED_IN state. 5753 -target: The first iSCSI connection in the session had 5754 reached the IN_LOGIN state. 5755 -N2: 5756 -initiator: Illegal. 5757 -target: At least one iSCSI connection reached the 5758 LOGGED_IN state. 5759 -N3: 5760 -initiator: Graceful closing of the session via session 5761 closure (Section 6.3.6). 5762 -target: Graceful closing of the session via session 5763 closure (Section 6.3.6) or a successful session 5764 reinstatement cleanly closed the session. 5765 -N4: 5766 -initiator: A session continuation attempt succeeded. 5767 -target: Illegal. 5768 -N5: 5770 -initiator: Session failure (Section 6.3.6) occurred. 5771 -target: Session failure (Section 6.3.6) occurred. 5772 -N6: 5773 -initiator: Session state timeout occurred, or a session 5774 reinstatement cleared this session instance. This results 5775 in the freeing of all associated resources and the session 5776 state is discarded. 5777 -target: Session state timeout occurred, or a session 5778 reinstatement cleared this session instance. This results 5779 in the freeing of all associated resources and the session 5780 state is discarded. 5781 -N7: 5782 -initiator: Illegal. 5783 -target: A session continuation attempt is initiated. 5784 -N8: 5785 -initiator: Illegal. 5786 -target: The last session continuation attempt failed. 5787 -N9: 5788 -initiator: Illegal. 5789 -target: Login attempt on the leading connection failed. 5790 -N10: 5791 -initiator: Illegal. 5792 -target: A session continuation attempt succeeded. 5793 -N11: 5794 -initiator: Illegal. 5795 -target: A successful session reinstatement cleanly closed 5796 the session. 5798 9. Security Considerations 5800 Historically, native storage systems have not had to consider 5801 security because their environments offered minimal security 5802 risks. That is, these environments consisted of storage devices 5803 either directly attached to hosts or connected via a Storage Area 5804 Network (SAN) distinctly separate from the communications network. 5805 The use of storage protocols, such as SCSI, over IP-networks 5806 requires that security concerns be addressed. iSCSI 5807 implementations must provide means of protection against active 5808 attacks (e.g., pretending to be another identity, message 5809 insertion, deletion, modification, and replaying) and passive 5810 attacks (e.g.,eavesdropping, gaining advantage by analyzing the 5811 data sent over the line). 5813 Although technically possible, iSCSI SHOULD NOT be configured 5814 without security, specifically in-band authentication, see Section 5815 9.2. iSCSI configured without security should be confined to 5816 closed environments that have very limited and well controlled 5817 security risks. [RFC3723] specifies the mechanisms that must be 5818 used in order to mitigate risks fully described in that document. 5820 The following Section describes the security mechanisms provided 5821 by an iSCSI implementation. 5823 9.1. iSCSI Security Mechanisms 5825 The entities involved in iSCSI security are the initiator, target, 5826 and the IP communication end points. iSCSI scenarios in which 5827 multiple initiators or targets share a single communication end 5828 point are expected. To accommodate such scenarios, iSCSI uses two 5829 separate security mechanisms: In-band authentication between the 5830 initiator and the target at the iSCSI connection level (carried 5831 out by exchange of iSCSI Login PDUs), and packet protection 5832 (integrity, authentication, and confidentiality) by IPsec at the 5833 IP level. The two security mechanisms complement each other. The 5834 in-band authentication provides end-to-end trust (at login time) 5835 between the iSCSI initiator and the target while IPsec provides a 5836 secure channel between the IP communication end points. iSCSI can 5837 be used to access sensitive information for which significant 5838 security protection is appropriate. As further specified in the 5839 rest of this security considerations section, both iSCSI security 5840 mechanisms are mandatory to implement (MUST). Use of in-band 5841 authentication is strongly recommended (SHOULD). In contrast, use 5842 of IPsec is optional (MAY) as the security risks that it addresses 5843 may only be present over a subset of the networks used by an iSCSI 5844 connection or a session; a specific example is that when an iSCSI 5845 session spans data centers, IPsec VPN gateways at the data center 5846 boundaries to protect the WAN connectivity between data centers 5847 may be appropriate in combination with in-band iSCSI 5848 authentication. 5850 Further details on typical iSCSI scenarios and the relation 5851 between the initiators, targets, and the communication end points 5852 can be found in [RFC3723]. 5854 9.2. In-band Initiator-Target Authentication 5856 During login, the target MAY authenticate the initiator and the 5857 initiator MAY authenticate the target. The authentication is 5858 performed on every new iSCSI connection by an exchange of iSCSI 5859 Login PDUs using a negotiated authentication method. 5861 The authentication method cannot assume an underlying IPsec 5862 protection, because IPsec is optional to use. An attacker should 5863 gain as little advantage as possible by inspecting the 5864 authentication phase PDUs. Therefore, a method using clear text 5865 (or equivalent) passwords MUST NOT be used; on the other hand, 5866 identity protection is not strictly required. 5868 The authentication mechanism protects against an unauthorized 5869 login to storage resources by using a false identity (spoofing). 5870 Once the authentication phase is completed, if the underlying 5871 IPsec is not used, all PDUs are sent and received in clear. The 5872 authentication mechanism alone (without underlying IPsec) should 5873 only be used when there is no risk of eavesdropping, message 5874 insertion, deletion, modification, and replaying. 5876 Section 11 defines several authentication methods and the exact 5877 steps that must be followed in each of them, including the iSCSI- 5878 text-keys and their allowed values in each step. Whenever an iSCSI 5879 initiator gets a response whose keys, or their values, are not 5880 according to the step definition, it MUST abort the connection. 5882 Whenever an iSCSI target gets a response whose keys, or their 5883 values, are not according to the step definition, it MUST answer 5884 with a Login reject with the "Initiator Error" or "Missing 5885 Parameter" status. These statuses are not intended for 5886 cryptographically incorrect values such as the CHAP response, for 5887 which "Authentication Failure" status MUST be specified. The 5888 importance of this rule can be illustrated in CHAP with target 5889 authentication (see Section 12.1.3) where the initiator would have 5890 been able to conduct a reflection attack by omitting his response 5891 key (CHAP_R) using the same CHAP challenge as the target and 5892 reflecting the target's response back to the target. In CHAP, this 5893 is prevented because the target must answer the missing CHAP_R key 5894 with a Login reject with the "Missing Parameter" status. 5896 For some of the authentication methods, a key specifies the 5897 identity of the iSCSI initiator or target for authentication 5898 purposes. The value associated with that key MAY be different from 5899 the iSCSI Name and SHOULD be configurable. (CHAP_N, see Section 5900 12.1.3 and SRP_U, see Section 12.1.2). For this reason, iSCSI 5901 implementations SHOULD manage authentication in a way that 5902 impersonation across iSCSI Names via these authentication 5903 identities is not possible. Specifically, implementations SHOULD 5904 allow configuration of an authentication identity for a Name if 5905 different, and authentication credentials for that identity. 5906 During the login time, implementations SHOULD verify the Name-to- 5907 identity relationship in addition to authenticating the identity 5908 through the negotiated authentication method. 5910 When an iSCSI session has multiple TCP connections, either 5911 concurrently or sequentially, the authentication method and 5912 identities should not vary among the connections. Therefore, all 5913 connections in an iSCSI session SHOULD use the same authentication 5914 method, iSCSI Name and authentication identity (for authentication 5915 methods that use an authentication identity). Implementations 5916 SHOULD check this and cause an authentication failure on a new 5917 connection that uses a different authentication method, iSCSI 5918 Name or authentication identity from those already used in the 5919 session. In addition, implementations SHOULD NOT support both 5920 authenticated and unauthenticated TCP connections in the same 5921 iSCSI session, added either concurrently or sequentially to the 5922 session. 5924 9.2.1. CHAP Considerations 5926 Compliant iSCSI initiators and targets MUST implement the CHAP 5927 authentication method [RFC1994] (according to Section 12.1.3 5928 including the target authentication option). 5930 When CHAP is performed over a non-encrypted channel, it is 5931 vulnerable to an off-line dictionary attack. Implementations MUST 5932 support use of up to 128 bit random CHAP secrets, including the 5933 means to generate such secrets and to accept them from an external 5934 generation source. Implementations MUST NOT provide secret 5935 generation (or expansion) means other than random generation. 5937 An administrative entity of an environment in which CHAP is used 5938 with a secret that has less than 96 random bits MUST enforce IPsec 5939 encryption (according to the implementation requirements in 5940 Confidentiality) to protect the connection. Moreover, in this case 5941 IKE authentication with group pre-shared cryptographic keys SHOULD 5942 NOT be used unless it is not essential to protect group members 5943 against off-line dictionary attacks by other members. 5945 CHAP secrets MUST be an integral number of bytes (octets). A 5946 compliant implementation SHOULD NOT continue with the login step 5947 in which it should send a CHAP response (CHAP_R, Section 12.1.3) 5948 unless it can verify that the CHAP secret is at least 96 bits, or 5949 that IPsec encryption is being used to protect the connection. 5951 Any CHAP secret used for initiator authentication MUST NOT be 5952 configured for authentication of any target, and any CHAP secret 5953 used for target authentication MUST NOT be configured for 5954 authentication of any initiator. If the CHAP response received by 5955 one end of an iSCSI connection is the same as the CHAP response 5956 that the receiving endpoint would have generated for the same CHAP 5957 challenge, the response MUST be treated as an authentication 5958 failure and cause the connection to close (this ensures that the 5959 same CHAP secret is not used for authentication in both 5960 directions). Also, if an iSCSI implementation can function as 5961 both initiator and target, different CHAP secrets and identities 5962 MUST be configured for these two roles. The following is an 5963 example of the attacks prevented by the above requirements: 5965 a) Rogue wants to impersonate Storage to Alice, and knows 5966 that a single secret is used for both directions of 5967 Storage-Alice authentication. 5969 b) Rogue convinces Alice to open two connections to Rogue, 5970 and Rogue identifies itself as Storage on both 5971 connections. 5973 c) Rogue issues a CHAP challenge on connection 1, waits for 5974 Alice to respond, and then reflects Alice's challenge as 5975 the initial challenge to Alice on connection 2. 5977 d) If Alice doesn't check for the reflection across 5978 connections, Alice's response on connection 2 enables 5979 Rogue to impersonate Storage on connection 1, even though 5980 Rogue does not know the Alice-Storage CHAP secret. 5982 Originators MUST NOT reuse the CHAP challenge sent by the 5983 Responder for the other direction of a bidirectional 5984 authentication. Responders MUST check for this condition and close 5985 the iSCSI TCP connection if it occurs. 5987 The same CHAP secret SHOULD NOT be configured for authentication 5988 of multiple initiators or multiple targets, as this enables any of 5989 them to impersonate any other one of them, and compromising one of 5990 them enables the attacker to impersonate any of them. It is 5991 recommended that iSCSI implementations check for use of identical 5992 CHAP secrets by different peers when this check is feasible, and 5993 take appropriate measures to warn users and/or administrators when 5994 this is detected. 5996 When an iSCSI initiator or target authenticates itself to 5997 counterparts in multiple administrative domains, it SHOULD use a 5998 different CHAP secret for each administrative domain to avoid 5999 propagating security compromises across domains. 6001 Within a single administrative domain: 6003 - A single CHAP secret MAY be used for authentication of an 6004 initiator to multiple targets. 6006 - A single CHAP secret MAY be used for an authentication of a 6007 target to multiple initiators when the initiators use an 6008 external server (e.g., RADIUS, [RFC2865]) to verify the 6009 target's CHAP responses and do not know the target's CHAP 6010 secret. 6012 If an external response verification server (e.g., RADIUS) is not 6013 used, employing a single CHAP secret for authentication of a 6014 target to multiple initiators requires that all such initiators 6015 know that target secret. Any of these initiators can impersonate 6016 the target to any other such initiator, and compromise of such an 6017 initiator enables an attacker to impersonate the target to all 6018 such initiators. Targets SHOULD use separate CHAP secrets for 6019 authentication to each initiator when such risks are of concern; 6020 in this situation it may be useful to configure a separate logical 6021 iSCSI target with its own iSCSI Node Name for each initiator or 6022 group of initiators among which such separation is desired. 6024 The above requirements strengthen the security properties of CHAP 6025 authentication for iSCSI by comparison to the basic CHAP 6026 authentication mechanism [RFC1994]. It is very important to 6027 adhere to these requirements, especially the requirements for 6028 strong (large randomly generated) CHAP secrets, as iSCSI 6029 implementations and deployments that fail to use strong CHAP 6030 secrets are likely to be highly vulnerable to off-line dictionary 6031 attacks on CHAP secrets. 6033 Replacement of CHAP with a better authentication mechanism is 6034 anticipated in a future version of iSCSI. The FC-SP-2 standard 6035 [FC-SP-2] has specified the EAP-GPSK authentication mechanism 6036 [RFC5433] as an alternative to (and possible future replacement 6037 for) Fibre Channel's similar usage of strengthened CHAP. Another 6038 possible replacement for CHAP is a secure password mechanism, 6039 e.g., an updated version of iSCSI's current SRP authentication 6040 mechanism. 6042 9.2.2. SRP Considerations 6044 The strength of the SRP authentication method (specified in 6045 [RFC2945]) is dependent on the characteristics of the group being 6046 used (i.e., the prime modulus N and generator g). As described in 6047 [RFC2945], N is required to be a Sophie-German prime (of the form 6048 N = 2q + 1, where q is also prime) and the generator g is a 6049 primitive root of GF(n). In iSCSI authentication, the prime 6050 modulus N MUST be at least 768 bits. 6052 The list of allowed SRP groups is provided in [RFC3723]. 6054 9.2.3. Kerberos Considerations 6056 iSCSI uses raw Kerberos V5 [RFC4120] for authenticating a client 6057 (iSCSI initiator) principal to a service (iSCSI target) principal. 6058 Note that iSCSI does not use the Generic Security Services 6059 Application Programming Interface (GSS-API) [RFC2743] nor the 6060 Kerberos V5 GSS-API security mechanism [RFC4121]. This means that 6061 iSCSI implementations supporting the KRB5 AuthMethod (Section 6062 12.1) are directly involved in the Kerberos protocol. When 6063 Kerberos V5 is used for authentication, the following actions MUST 6064 be performed as specified in [RFC4120]: 6066 Target MUST validate the KRB_AP_REQ to ensure that the 6067 initiator can be trusted 6069 When mutual authentication is selected, the initiator MUST 6070 validate KRB_AP_REP to determine the outcome of mutual 6071 authentication 6073 As Kerberos V5 is capable of providing mutual authentication, 6074 implementations SHOULD support mutual authentication by default 6075 for login authentication. 6077 Note however that Kerberos authentication only assures that the 6078 server (iSCSI target) can be trusted by the Kerberos client 6079 (initiator) and vice-versa; an initiator should employ 6080 appropriately secured service discovery techniques (e.g. iSNS, 6081 Section 4.2.7) to ensure that it is talking to the intended target 6082 principal. 6084 iSCSI does not use Kerberos v5 for either integrity or 6085 confidentiality protection of the iSCSI protocol. iSCSI uses IPsec 6086 for those purposes as specified in Section 9.3. 6088 9.3. IPsec 6090 iSCSI uses the IPsec mechanism for packet protection 6091 (cryptographic integrity, authentication, and confidentiality) at 6092 the IP level between the iSCSI communicating end points. The 6093 following sections describe the IPsec protocols that must be 6094 implemented for data integrity and authentication, 6095 confidentiality, and cryptographic key management. 6097 An iSCSI initiator or target may provide the required IPsec 6098 support fully integrated or in conjunction with an IPsec front-end 6099 device. In the latter case, the compliance requirements with 6100 regard to IPsec support apply to the "combined device". Only the 6101 "combined device" is to be considered an iSCSI device. 6103 Detailed considerations and recommendations for using IPsec for 6104 iSCSI are provided in [RFC3723] as updated by [IPSEC-IPS]. The 6105 IPsec requirements are reproduced here for convenience and are 6106 intended to match those in [IPSEC-IPS]; in the event of a 6107 discrepancy, the requirements in [IPSEC-IPS] apply. 6109 9.3.1. Data Integrity and Authentication 6111 Data authentication and integrity is provided by a cryptographic 6112 keyed Message Authentication Code in every sent packet. This code 6113 protects against message insertion, deletion, and modification. 6114 Protection against message replay is realized by using a sequence 6115 counter. 6117 An iSCSI-compliant initiator or target MUST provide data integrity 6118 and authentication by implementing IPsec v2 [RFC2401] with ESPv2 6119 [RFC2406] in tunnel mode, SHOULD provide data integrity and 6120 authentication by implementing IPsec v3 [RFC4301] with ESPv3 6121 [RFC4303] in tunnel mode, and MAY provide data integrity and 6122 authentication by implementing either IPsec v2 or v3 with the 6123 appropriate version of ESP in transport mode. The IPsec 6125 implementation MUST fulfill the following iSCSI-specific 6126 requirements: 6128 - HMAC-SHA1 MUST be implemented in the specific form of HMAC- 6129 SHA-1-96 [RFC2404]. 6131 - AES CBC MAC with XCBC extensions using 128-bit keys SHOULD 6132 be implemented [RFC3566]. 6134 - Implementations that support IKEv2 [RFC5996] SHOULD also 6135 implement AES GMAC [RFC4543] using 128-bit keys. 6137 The ESP anti-replay service MUST also be implemented. 6139 At the high speeds iSCSI is expected to operate, a single IPsec SA 6140 could rapidly cycle through the ESP 32-bit sequence number space. 6141 In view of this, an iSCSI implementation that is capable of 6142 operating at speeds of 1 Gbps and that implements both IKEv2 6143 [RFC5996] and ESPv3 [RFC4303] MUST also implement extended (64- 6144 bit) sequence numbers for ESPv3 and SHOULD use ESPv3 extended 6145 sequence numbers for all security associations that use ESPv3 to 6146 protect iSCSI connections. 6148 9.3.2. Confidentiality 6150 Confidentiality is provided by encrypting the data in every 6151 packet. When confidentiality is used it MUST be accompanied by 6152 data integrity and authentication to provide comprehensive 6153 protection against eavesdropping, message insertion, deletion, 6154 modification, and replaying. 6156 An iSCSI-compliant initiator or target MUST provide 6157 confidentiality by implementing IPsec v2 [RFC2401] with ESPv2 6158 [RFC2406] in tunnel mode, SHOULD provide confidentiality by 6159 implementing IPsec v3 [RFC4301] with ESPv3 [RFC4303] in tunnel 6160 mode and MAY provide confidentiality by implementing either IPsec 6161 v2 or v3 with the appropriate version of ESP in transport mode, 6162 with the following iSCSI specific requirements that apply to IPsec 6163 v2 and IPsec v3: 6164 - 3DES in CBC mode MAY be implemented [RFC2451]. 6166 - AES in CBC mode with 128-bit keys MUST be implemented 6167 [RFC3602]; other key sizes MAY be supported. 6169 - AES in Counter mode MAY be implemented [RFC3686]. 6171 - Implementations that support IKEv2 [RFC5996] SHOULD also 6172 implement AES GCM with 128-bit keys [RFC4106] ]; other key 6173 sizes MAY be supported. 6175 DES in CBC mode MUST NOT be used due to its inherent weakness. 6177 The NULL encryption algorithm MUST also be implemented. 6179 9.3.3. Policy, Security Associations, and Cryptographic Key 6180 Management 6182 A compliant iSCSI implementation MUST meet the cryptographic key 6183 management requirements of the IPsec protocol suite. 6184 Authentication, security association negotiation, and 6185 cryptographic key management MUST be provided by implementing IKE 6186 [RFC2409] using the IPsec DOI [RFC2407], and SHOULD be provided by 6187 implementing IKEv2 [RFC5996], with the following iSCSI-specific 6188 requirements: 6190 a) Peer authentication using a pre-shared cryptographic key MUST 6191 be supported. Certificate-based peer authentication using 6192 digital signatures MAY be supported. For IKEv1 ([RFC2409]), 6193 peer authentication using the public key encryption methods 6194 outlined in IKE sections 5.2 and 5.3 of [RFC2409] SHOULD NOT 6195 be used. 6196 b) When digital signatures are used to achieve authentication, 6197 an IKE negotiator SHOULD use IKE Certificate Request 6198 Payload(s) to specify the certificate authority. IKE 6199 negotiators SHOULD check the pertinent Certificate Revocation 6200 List (CRL) before accepting a PKI certificate for use in IKE 6201 authentication procedures. These checks may not be needed in 6202 environments where a small number of certificates are 6203 statically configured as trust anchors. 6204 c) Conformant iSCSI implementations of IKEv1 MUST support Main 6205 Mode and SHOULD support Aggressive Mode. Main Mode with pre- 6206 shared key authentication method SHOULD NOT be used when 6207 either the initiator or the target uses dynamically assigned 6208 addresses. While in many cases pre-shared keys offer good 6209 security, situations in which dynamically assigned addresses 6210 are used force the use of a group pre-shared key, which 6211 creates vulnerability to a man-in-the-middle attack. 6212 d) In the IKEv1 Phase 2 Quick Mode, exchanges for creating the 6213 Phase 2 SA, the Identification Payload MUST be present. 6214 e) The following identification type requirements apply to 6215 IKEv1. ID_IPV4_ADDR, ID_IPV6_ADDR (if the protocol stack 6216 supports IPv6) and ID_FQDN Identification Types MUST be 6217 supported; ID_USER_FQDN SHOULD be supported. The IP Subnet, 6218 IP Address Range, ID_DER_ASN1_DN, and ID_DER_ASN1_GN 6219 Identification Types SHOULD NOT be used. The ID_KEY_ID 6220 Identification Type MUST NOT be used. 6221 f) If IKEv2 is supported, the following identification 6222 requirements apply. ID_IPV4_ADDR, ID_IPV6_ADDR (if the 6223 protocol stack supports IPv6) and ID_FQDN Identification 6224 Types MUST be supported; ID_RFC822_ADDR SHOULD be supported. 6225 The ID_DER_ASN1_DN, and ID_DER_ASN1_GN Identification Types 6226 SHOULD NOT be used. The ID_KEY_ID Identification Type MUST 6227 NOT be used. 6229 The reasons for the "MUST NOT" and "SHOULD NOT" requirements for 6230 identification type requirements in preceding bullets e) and f) 6231 are: 6233 - IP Subnet and IP Address Range are too broad to usefully 6234 identify an iSCSI endpoint. 6236 - The DN and GN types are X.500 identities; it is usually 6237 better to use an identity from subjectAltName in a PKI 6238 certificate. 6240 - ID_KEY_ID is not interoperable as specified. 6242 Manual cryptographic keying MUST NOT be used because it does not 6243 provide the necessary re-keying support. 6245 When DH groups are used, a DH group of at least 2048 bits SHOULD 6246 be offered as a part of all proposals to create IPsec Security 6247 Associations to protect iSCSI traffic, with both IKEv1 and IKEv2. 6249 When IPsec is used, the receipt of an IKEv1 Phase 2 delete message 6250 or an IKEv2 INFORMATIONAL exchange that deletes the SA SHOULD NOT 6251 be interpreted as a reason for tearing down the iSCSI TCP 6252 connection. If additional traffic is sent on it, a new IKE SA will 6253 be created to protect it. 6255 The method used by the initiator to determine whether the target 6256 should be connected using IPsec is regarded as an issue of IPsec 6257 policy administration, and thus not defined in the iSCSI standard. 6259 The method used by an initiator that supports both IPsec v2 and v3 6260 to determine which versions of IPsec are supported by the target 6261 is also regarded as an issue of IPsec policy administration, and 6262 thus not defined in the iSCSI standard. If both IPsec v2 and v3 6263 are supported by both the initiator and target, use of IPsec v3 is 6264 recommended. 6266 If an iSCSI target is discovered via a SendTargets request in a 6267 discovery session not using IPsec, the initiator should assume 6268 that it does not need IPsec to establish a session to that target. 6269 If an iSCSI target is discovered using a discovery session that 6270 does use IPsec, the initiator SHOULD use IPsec when establishing a 6271 session to that target. 6273 9.4. Security Considerations for the X#NodeArchitecture Key 6275 The security considerations in this Section are specific to the 6276 X#NodeArchitecture discussed in Section 13.26. 6278 This extension key transmits specific implementation details about 6279 the node that sends it; such details may be considered sensitive 6280 in some environments. For example, if a certain software or 6281 firmware version is known to contain security weaknesses, 6282 announcing the presence of that version via this key may not be 6283 desirable. The countermeasures for this security concern are: 6285 a) sending less detailed information in the key values, 6286 b) not sending the extension key, or 6287 c) using IPsec ([RFC4303]) to provide confidentiality for the 6288 iSCSI connection on which the key is sent 6290 To support the first and second countermeasures, all 6291 implementations of this extension key MUST provide an 6292 administrative mechanism to disable sending the key. In addition, 6293 all implementations SHOULD provide an administrative mechanism to 6294 configure a verbosity level of the key value, thereby controlling 6295 the amount of information sent. 6297 For example, a lower verbosity might enable transmission of node 6298 architecture component names only, but no version numbers. The 6299 choice of which countermeasure is most appropriate depends on the 6300 environment. However, sending less detailed information in the key 6301 values may be an acceptable countermeasure in many environments, 6302 since it provides a compromise between sending too much 6303 information and the other more complete countermeasures of not 6304 sending the key at all or using IPsec. 6306 In addition to security considerations involving transmission of 6307 the key contents, any logging method(s) used for the key values 6308 MUST keep the information secure from intruders. For all 6309 implementations, the requirements to address this security concern 6310 are: 6312 a) Display of the log MUST only be possible with administrative 6313 rights to the node. 6314 b) Options to disable logging to disk and to keep logs for a 6315 fixed duration SHOULD be provided. 6317 Finally, it is important to note that different nodes may have 6318 different levels of risk, and these differences may affect the 6319 implementation. The components of risk include assets, threats, 6320 and vulnerabilities. Consider the following example iSCSI nodes, 6321 which demonstrate differences in assets and vulnerabilities of the 6322 nodes, and as a result, differences in implementation: 6324 a) One iSCSI target based on a special-purpose operating 6325 system: Since the iSCSI target controls access to the 6326 data storage containing company assets, the asset level 6327 is seen as very high. Also, because of the special- 6328 purpose operating system, in which vulnerabilities are 6329 less well-known, the vulnerability level is viewed as 6330 low. 6332 b) Multiple iSCSI initiators in a blade farm, each running a 6333 general purpose operating system: The asset level of each 6334 node is viewed as low, since blades are replaceable and 6335 low cost. However, the vulnerability level is viewed as 6336 high, since there may be many well-known vulnerabilities 6337 to that general-purpose operating system. For this 6338 target, an appropriate implementation might be logging of 6339 received key values, but no transmission of the key. For 6340 this initiator, an appropriate implementation might be 6341 transmission of the key, but no logging of received key 6342 values. 6344 9.5. SCSI Access Control Considerations 6346 iSCSI is a SCSI transport protocol and as such does not apply any 6347 access controls on SCSI-level operations such as SCSI task 6348 management functions (e.g. LU Reset, see Section 11.5.1). SCSI- 6349 level access controls (e.g. ACCESS CONTROL OUT, see [SPC3]) have 6350 to be appropriately deployed in practice to address SCSI-level 6351 security considerations, in addition to security at iSCSI 6352 connection and packet protection mechanisms that were already 6353 discussed in preceding Sections. 6355 10. Notes to Implementers 6357 This Section notes some of the performance and reliability 6358 considerations of the iSCSI protocol. This protocol was designed 6359 to allow efficient silicon and software implementations. The iSCSI 6360 task tag mechanism was designed to enable Direct Data Placement 6361 (DDP - a DMA form) at the iSCSI level or lower. 6363 The guiding assumption made throughout the design of this protocol 6364 is that targets are resource constrained relative to initiators. 6366 Implementers are also advised to consider the implementation 6367 consequences of the iSCSI to SCSI mapping model as outlined in 6368 Section 4.4.3. 6370 10.1. Multiple Network Adapters 6372 The iSCSI protocol allows multiple connections, not all of which 6373 need to go over the same network adapter. If multiple network 6374 connections are to be utilized with hardware support, the iSCSI 6375 protocol command-data-status allegiance to one TCP connection 6376 ensures that there is no need to replicate information across 6377 network adapters or otherwise require them to cooperate. 6379 However, some task management commands may require some loose form 6380 of cooperation or replication at least on the target. 6382 10.1.1. Conservative Reuse of ISIDs 6384 Historically, the SCSI model (and implementations and applications 6385 based on that model) has assumed that SCSI ports are static, 6386 physical entities. Recent extensions to the SCSI model have taken 6387 advantage of persistent worldwide unique names for these ports. In 6388 iSCSI however, the SCSI initiator ports are the endpoints of 6389 dynamically created sessions, so the presumptions of "static and 6390 physical" do not apply. In any case, the model clauses 6391 (particularly, Section 4.4.1) provide for persistent, reusable 6392 names for the iSCSI-type SCSI initiator ports even though there 6393 does not need to be any physical entity bound to these names. 6395 To both minimize the disruption of legacy applications and to 6396 better facilitate the SCSI features that rely on persistent names 6397 for SCSI ports, iSCSI implementations SHOULD attempt to provide a 6398 stable presentation of SCSI Initiator Ports (both to the upper OS- 6399 layers and to the targets to which they connect). This can be 6400 achieved in an initiator implementation by conservatively reusing 6401 ISIDs. In other words, the same ISID should be used in the Login 6402 process to multiple target portal groups (of the same iSCSI Target 6403 or different iSCSI Targets). The ISID RULE (Section 4.4.3) only 6404 prohibits reuse to the same target portal group. It does not 6405 "preclude" reuse to other target portal groups. 6406 The principle of conservative reuse "encourages" reuse to other 6407 target portal groups. When a SCSI target device sees the same 6408 (InitiatorName, ISID) pair in different sessions to different 6409 target portal groups, it can identify the underlying SCSI 6410 Initiator Port on each session as the same SCSI port. In effect, 6411 it can recognize multiple paths from the same source. 6413 10.1.2. iSCSI Name, ISID, and TPGT Use 6415 The designers of the iSCSI protocol are aware that legacy SCSI 6416 transports rely on initiator identity to assign access to storage 6417 resources. Although newer techniques are available and simplify 6418 access control, support for configuration and authentication 6419 schemes that are based on initiator identity is deemed important 6420 in order to support legacy systems and administration software. 6421 iSCSI thus supports the notion that it should be possible to 6422 assign access to storage resources based on "initiator device" 6423 identity. 6425 When there are multiple hardware or software components 6426 coordinated as a single iSCSI Node, there must be some (logical) 6427 entity that represents the iSCSI Node that makes the iSCSI Node 6428 Name available to all components involved in session creation and 6429 login. Similarly, this entity that represents the iSCSI Node must 6430 be able to coordinate session identifier resources (ISID for 6431 initiators) to enforce both the ISID and TSIH RULES (see Section 6432 4.4.3). 6434 For targets, because of the closed environment, implementation of 6435 this entity should be straightforward. However, vendors of iSCSI 6436 hardware (e.g., NICs or HBAs) intended for targets, SHOULD provide 6437 mechanisms for configuration of the iSCSI Node Name across the 6438 portal groups instantiated by multiple instances of these 6439 components within a target. 6441 However, complex targets making use of multiple Target Portal 6442 Group Tags may reconfigure them to achieve various quality goals. 6443 The initiators have two mechanisms at their disposal to discover 6444 and/or check reconfiguring targets - the discovery session type 6445 and a key returned by the target during login to confirm the TPGT. 6446 An initiator should attempt to "rediscover" the target 6447 configuration anytime a session is terminated unexpectedly. 6449 For initiators, in the long term, it is expected that operating 6450 system vendors will take on the role of this entity and provide 6451 standard APIs that can inform components of their iSCSI Node Name 6452 and can configure and/or coordinate ISID allocation, use, and 6453 reuse. 6455 Recognizing that such initiator APIs are not available today, 6456 other implementations of the role of this entity are possible. For 6457 example, a human may instantiate the (common) Node name as part of 6458 the installation process of each iSCSI component involved in 6459 session creation and login. This may be done either by pointing 6460 the component to a vendor-specific location for this datum or to a 6461 system-wide location. The structure of the ISID namespace (see 6462 Section 11.12.5 and [RFC3721]) facilitates implementation of the 6463 ISID coordination by allowing each component vendor to 6464 independently (of other vendor's components) coordinate 6465 allocation, use, and reuse of its own partition of the ISID 6466 namespace in a vendor-specific manner. Partitioning of the ISID 6467 namespace within initiator portal groups managed by that vendor 6468 allows each such initiator portal group to act independently of 6469 all other portal groups when selecting an ISID for a login; this 6470 facilitates enforcement of the ISID RULE (see Section 4.4.3) at 6471 the initiator. 6473 A vendor of iSCSI hardware (e.g., NICs or HBAs) intended for use 6474 in initiators MUST implement a mechanism for configuring the iSCSI 6475 Node Name. Vendors, and administrators must ensure that iSCSI Node 6476 Names are unique worldwide. It is therefore important that when 6477 one chooses to reuse the iSCSI Node Name of a disabled unit, not 6478 to re-assign that name to the original unit unless its worldwide 6479 uniqueness can be ascertained again. 6481 In addition, a vendor of iSCSI hardware must implement a mechanism 6482 to configure and/or coordinate ISIDs for all sessions managed by 6483 multiple instances of that hardware within a given iSCSI Node. 6484 Such configuration might be either permanently pre-assigned at the 6485 factory (in a necessarily globally unique way), statically 6486 assigned (e.g., partitioned across all the NICs at initialization 6487 in a locally unique way), or dynamically assigned (e.g., on-line 6488 allocator, also in a locally unique way). In the latter two cases, 6489 the configuration may be via public APIs (perhaps driven by an 6490 independent vendor's software, such as the OS vendor) or via 6491 private APIs driven by the vendor's own software. 6493 The process of name assignment and coordination has to be as 6494 encompassing and automated as possible as years of legacy usage 6495 have shown it to be highly error-prone. It is to be mentioned 6496 that SCSI has today alternative schemes of access control that can 6497 be used by all transports and their security is not dependent on 6498 strict naming coordination. 6500 10.2. Autosense and Auto Contingent Allegiance (ACA) 6502 Autosense refers to the automatic return of sense data to the 6503 initiator in case a command did not complete successfully. iSCSI 6504 initiators and targets MUST support and use autosense. 6506 ACA helps preserve ordered command execution in the presence of 6507 errors. As there can be many commands in-flight between an 6508 initiator and a target, SCSI initiator functionality in some 6509 operating systems depends on ACA to enforce ordered command 6510 execution during error recovery, and hence iSCSI initiator 6511 implementations for those operating systems need to support ACA. 6512 In order to support error recovery for these operating systems and 6513 iSCSI initiators, iSCSI targets SHOULD support ACA. 6515 10.3. iSCSI Timeouts 6517 iSCSI recovery actions are often dependent on iSCSI time-outs 6518 being recognized and acted upon before SCSI time-outs. Determining 6519 the right time-outs to use for various iSCSI actions (command 6520 acknowledgements expected, status acknowledgements, etc.) is very 6521 much dependent on infrastructure (hardware, links, TCP/IP stack, 6522 iSCSI driver). As a guide, the implementer may use an average Nop- 6523 Out/Nop-In turnaround delay multiplied by a "safety factor" (e.g., 6524 4) as a good estimate for the basic delay of the iSCSI stack for a 6525 given connection. The safety factor should account for the network 6526 load variability. For connection teardown the implementer may 6527 want to consider also TCP common practice for the given 6528 infrastructure. 6530 Text negotiations MAY also be subject to either time-limits or 6531 limits in the number of exchanges. Those SHOULD be generous enough 6532 to avoid affecting interoperability (e.g., allowing each key to be 6533 negotiated on a separate exchange). 6535 The relation between iSCSI timeouts and SCSI timeouts should also 6536 be considered. SCSI timeouts should be longer than iSCSI timeouts 6537 plus the time required for iSCSI recovery whenever iSCSI recovery 6538 is planned. Alternatively, an implementer may choose to interlock 6539 iSCSI timeouts and recovery with SCSI timeouts so that SCSI 6540 recovery will become active only where iSCSI is not planned to, or 6541 failed to, recover. 6543 The implementer may also want to consider the interaction between 6544 various iSCSI exception events - such as a digest failure - and 6545 subsequent timeouts. When iSCSI error recovery is active, a digest 6546 failure is likely to result in discovering a missing command or 6547 data PDU. In these cases, an implementer may want to lower the 6548 timeout values to enable faster initiation for recovery 6549 procedures. 6551 10.4. Command Retry and Cleaning Old Command Instances 6553 To avoid having old, retried command instances appear in a valid 6554 command window after a command sequence number wrap around, the 6555 protocol requires (see Section 4.2.2.1) that on every connection 6556 on which a retry has been issued, a non-immediate command be 6557 issued and acknowledged within a 2**31-1 commands interval from 6558 the CmdSN of the retried command. This requirement can be 6559 fulfilled by an implementation in several ways. 6561 The simplest technique to use is to send a (non-retry) non- 6562 immediate SCSI command (or a NOP if no SCSI command is available 6563 for a while) after every command retry on the connection on which 6564 the retry was attempted. As errors are deemed rare events, this 6565 technique is probably the most effective, as it does not involve 6566 additional checks at the initiator when issuing commands. 6568 10.5. Synch and Steering Layer and Performance 6570 While a synch and steering layer is optional, an initiator/target 6571 that does not have it working against a target/initiator that 6572 demands synch and steering may experience performance degradation 6573 caused by packet reordering and loss. Providing a synch and 6574 steering mechanism is recommended for all high-speed 6575 implementations. 6577 10.6. Considerations for State-dependent Devices and Long-lasting 6578 SCSI Operations 6580 Sequential access devices operate on the principle that the 6581 position of the device is based on the last command processed. As 6582 such, command processing order and knowledge of whether or not the 6583 previous command was processed is of the utmost importance to 6584 maintain data integrity. For example, inadvertent retries of SCSI 6585 commands when it is not known if the previous SCSI command was 6586 processed is a potential data integrity risk. 6588 For a sequential access device, consider the scenario in which a 6589 SCSI SPACE command to backspace one filemark is issued and then 6590 re-issued due to no status received for the command. If the first 6591 SPACE command was actually processed, the re-issued SPACE command, 6592 if processed, will cause the position to change. Thus, a 6593 subsequent write operation will write data to the wrong position 6594 and any previous data at that position will be overwritten. 6596 For a medium changer device, consider the scenario in which an 6597 EXCHANGE MEDIUM command (the SOURCE ADDRESS and DESTINATION 6598 ADDRESS are the same thus performing a swap) is issued and then 6599 re-issued due to no status received for the command. If the first 6600 EXCHANGE MEDIUM command was actually processed, the re-issued 6601 EXCHANGE MEDIUM command, if processed, will perform the swap 6602 again. The net effect is no swap was performed thus leaving a data 6603 integrity exposure. 6605 All commands that change the state of the device (as in SPACE 6606 commands for sequential access devices, and EXCHANGE MEDIUM for 6607 medium changer device), MUST be issued as non-immediate commands 6608 for deterministic and in order delivery to iSCSI targets. 6610 For many of those state changing commands, the execution model 6611 also assumes that the command is executed exactly once. Devices 6612 implementing READ POSITION and LOCATE provide a means for SCSI 6613 level command recovery and new tape-class devices should support 6614 those commands. In their absence a retry at SCSI level is 6615 difficult and error recovery at iSCSI level is advisable. 6617 Devices operating on long latency delivery subsystems and 6618 performing long lasting SCSI operations may need mechanisms that 6619 enable connection replacement while commands are running (e.g., 6620 during an extended copy operation). 6622 10.6.1. Determining the Proper ErrorRecoveryLevel 6624 The implementation and use of a specific ErrorRecoveryLevel should 6625 be determined based on the deployment scenarios of a given iSCSI 6626 implementation. Generally, the following factors must be 6627 considered before deciding on the proper level of recovery: 6629 a) Application resilience to I/O failures. 6630 b) Required level of availability in the face of transport 6631 connection failures. 6632 c) Probability of transport layer "checksum escape" (message 6633 error undetected by TCP checksum - see [RFC3385] for 6634 related discussion). This in turn decides the iSCSI 6635 digest failure frequency, and thus the criticality of 6636 iSCSI-level error recovery. The details of estimating 6637 this probability are outside the scope of this document. 6639 A consideration of the above factors for SCSI tape devices as an 6640 example suggests that implementations SHOULD use 6641 ErrorRecoveryLevel=1 when transport connection failure is not a 6642 concern and SCSI level recovery is unavailable, and 6643 ErrorRecoveryLevel=2 when the connection failure is also of high 6644 likelihood during a backup/retrieval. 6646 For extended copy operations, implementations SHOULD use 6647 ErrorRecoveryLevel=2 whenever there is a relatively high 6648 likelihood of connection failure. 6650 10.7. Multi-task Abort Implementation Considerations 6652 Multi-task abort operations are typically issued in emergencies - 6653 such as clearing a device lock-up, HA failover/failback etc. In 6654 these circumstances, it is desirable to rapidly go through the 6655 error handling process as opposed to the target waiting on 6656 multiple third-party initiators who may not even be functional 6657 anymore - especially if this emergency is triggered because of one 6658 such initiator failure. Therefore, both iSCSI target and 6659 initiator implementations SHOULD support FastAbort multi-task 6660 abort semantics (Section 4.2.3.4). 6662 Note that both in standard semantics (Section 4.2.3.3) and 6663 FastAbort semantics (Section 4.2.3.4), there may be outstanding 6664 data transfers even after the TMF completion is reported on the 6665 issuing session. In the case of iSCSI/iSER [iSER], these would be 6666 tagged data transfers for STags not owned by any active tasks. 6667 Whether or not real buffers support these data transfers is 6668 implementation-dependent. However, the data transfers logically 6669 MUST be silently discarded by the target iSCSI layer in all cases. 6670 A target MAY, on an implementation-defined internal timeout, also 6671 choose to drop the connections on which it did not receive the 6672 expected Data-Out sequences (Section 4.2.3.3) or NOP-Out 6673 acknowledgements (Section 4.2.3.4) so as to reclaim the associated 6674 buffer, STag, and TTT resources as appropriate. 6676 11. iSCSI PDU Formats 6678 All multi-byte integers that are specified in formats defined in 6679 this document are to be represented in network byte order (i.e., 6680 big endian). Any field that appears in this document assumes that 6681 the most significant byte is the lowest numbered byte and the most 6682 significant bit (within byte or field) is the lowest numbered bit 6683 unless specified otherwise. 6685 Any compliant sender MUST set all bits not defined and all 6686 reserved fields to zero unless specified otherwise. Any compliant 6687 receiver MUST ignore any bit not defined and all reserved fields 6688 unless specified otherwise. Receipt of reserved code values in 6689 defined fields MUST be reported as a protocol error. 6691 Reserved fields are marked by the word "reserved", some 6692 abbreviation of "reserved", or by "." for individual bits when no 6693 other form of marking is technically feasible. 6695 11.1. iSCSI PDU Length and Padding 6697 iSCSI PDUs are padded to the closest integer number of four byte 6698 words. The padding bytes SHOULD be sent as 0. 6700 11.2. PDU Template, Header, and Opcodes 6702 All iSCSI PDUs have one or more header segments and, optionally, a 6703 data segment. After the entire header segment group a header- 6704 digest MAY follow. The data segment MAY also be followed by a 6705 data-digest. 6707 The Basic Header Segment (BHS) is the first segment in all of the 6708 iSCSI PDUs. The BHS is a fixed-length 48-byte header segment. It 6709 MAY be followed by Additional Header Segments (AHS), a Header- 6710 Digest, a Data Segment, and/or a Data-Digest. 6712 The overall structure of an iSCSI PDU is as follows: 6714 Byte/ 0 | 1 | 2 | 3 | 6715 / | | | | 6716 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 6717 +---------------+---------------+---------------+--------------+ 6718 0/ Basic Header Segment (BHS) / 6719 +/ / 6720 +---------------+---------------+---------------+--------------+ 6721 48/ Additional Header Segment 1 (AHS) (optional) / 6722 +/ / 6723 +---------------+---------------+---------------+--------------+ 6724 / Additional Header Segment 2 (AHS) (optional) / 6725 +/ / 6726 +---------------+---------------+---------------+--------------+ 6727 +---------------+---------------+---------------+--------------+ 6728 / Additional Header Segment n (AHS) (optional) / 6729 +/ / 6730 +---------------+---------------+---------------+--------------+ 6731 k/ Header-Digest (optional) / 6732 +/ / 6733 +---------------+---------------+---------------+--------------+ 6734 l/ Data Segment(optional) / 6735 +/ / 6736 +---------------+---------------+---------------+--------------+ 6737 m/ Data-Digest (optional) / 6738 +/ / 6739 +---------------+---------------+---------------+--------------+ 6741 All PDU segments and digests are padded to the closest integer 6742 number of four byte words. For example, all PDU segments and 6743 digests start at a four byte word boundary and the padding ranges 6744 from 0 to 3 bytes. The padding bytes SHOULD be sent as 0. 6746 iSCSI response PDUs do not have AH Segments. 6748 11.2.1. Basic Header Segment (BHS) 6750 The BHS is 48 bytes long. The Opcode and DataSegmentLength fields 6751 appear in all iSCSI PDUs. In addition, when used, the Initiator 6752 Task Tag and Logical Unit Number always appear in the same 6753 location in the header. 6755 The format of the BHS is: 6757 Byte/ 0 | 1 | 2 | 3 | 6758 / | | | | 6759 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 6760 +---------------+---------------+---------------+--------------+ 6761 0|.|I| Opcode |F| Opcode-specific fields | 6762 +---------------+---------------+---------------+--------------+ 6763 4|TotalAHSLength | DataSegmentLength | 6764 +---------------+---------------+---------------+--------------+ 6765 8| LUN or Opcode-specific fields | 6766 + + 6767 12| | 6768 +---------------+---------------+---------------+--------------+ 6769 16| Initiator Task Tag | 6770 +---------------+---------------+---------------+--------------+ 6771 20/ Opcode-specific fields / 6772 +/ / 6773 +---------------+---------------+---------------+--------------+ 6774 48 6776 11.2.1.1. I 6778 For request PDUs, the I bit set to 1 is an immediate delivery 6779 marker. 6781 11.2.1.2. Opcode 6783 The Opcode indicates the type of iSCSI PDU the header 6784 encapsulates. 6786 The Opcodes are divided into two categories: initiator opcodes and 6787 target opcodes. Initiator opcodes are in PDUs sent by the 6788 initiator (request PDUs). Target opcodes are in PDUs sent by the 6789 target (response PDUs). 6791 Initiators MUST NOT use target opcodes and targets MUST NOT use 6792 initiator opcodes. 6794 Initiator opcodes defined in this specification are: 6796 0x00 NOP-Out 6798 0x01 SCSI Command (encapsulates a SCSI Command Descriptor 6799 Block) 6801 0x02 SCSI Task Management function request 6803 0x03 Login Request 6805 0x04 Text Request 6807 0x05 SCSI Data-out (for WRITE operations) 6809 0x06 Logout Request 6811 0x10 SNACK Request 6813 0x1c-0x1e Vendor specific codes 6815 Target opcodes are: 6817 0x20 NOP-In 6819 0x21 SCSI Response - contains SCSI status and possibly sense 6820 information or other response information. 6822 0x22 SCSI Task Management function response 6824 0x23 Login Response 6826 0x24 Text Response 6828 0x25 SCSI Data-in - for READ operations. 6830 0x26 Logout Response 6832 0x31 Ready To Transfer (R2T) - sent by target when it is ready 6833 to receive data. 6835 0x32 Asynchronous Message - sent by target to indicate certain 6836 special conditions. 6838 0x3c-0x3e Vendor specific codes 6840 0x3f Reject 6842 All other opcodes are reserved. 6844 11.2.1.3. Final (F) bit 6846 When set to 1 it indicates the final (or only) PDU of a sequence. 6848 11.2.1.4. Opcode-specific Fields 6850 These fields have different meanings for different opcode types. 6852 11.2.1.5. TotalAHSLength 6854 Total length of all AHS header segments in units of four byte 6855 words including padding, if any. 6857 The TotalAHSLength is only used in PDUs that have an AHS and MUST 6858 be 0 in all other PDUs. 6860 11.2.1.6. DataSegmentLength 6862 This is the data segment payload length in bytes (excluding 6863 padding). The DataSegmentLength MUST be 0 whenever the PDU has no 6864 data segment. 6866 11.2.1.7. LUN 6868 Some opcodes operate on a specific Logical Unit. The Logical Unit 6869 Number (LUN) field identifies which Logical Unit. If the opcode 6870 does not relate to a Logical Unit, this field is either ignored or 6871 may be used in an opcode specific way. The LUN field is 64-bits 6872 and should be formatted in accordance with [SAM2]. For example, 6873 LUN[0] from [SAM2] is BHS byte 8 and so on up to LUN[7] from 6874 [SAM2], which is BHS byte 15. 6876 11.2.1.8. Initiator Task Tag 6878 The initiator assigns a Task Tag to each iSCSI task it issues. 6879 While a task exists, this tag MUST uniquely identify the task 6880 session-wide. SCSI may also use the initiator task tag as part of 6881 the SCSI task identifier when the timespan during which an iSCSI 6882 initiator task tag must be unique extends over the timespan during 6883 which a SCSI task tag must be unique. However, the iSCSI Initiator 6884 Task Tag must exist and be unique even for untagged SCSI commands. 6886 An ITT value of 0xffffffff is reserved and MUST NOT be assigned 6887 for a task by the initiator. The only instance in which it may be 6888 seen on the wire is in a target-initiated NOP-In PDU (Section 6889 11.19) and in the initiator response to that PDU, if necessary. 6891 11.2.2. Additional Header Segment (AHS) 6893 The general format of an AHS is: 6895 Byte/ 0 | 1 | 2 | 3 | 6896 / | | | | 6897 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 6898 +---------------+---------------+---------------+--------------+ 6899 0| AHSLength | AHSType | AHS-Specific | 6900 +---------------+---------------+---------------+--------------+ 6901 4/ AHS-Specific / 6902 +/ / 6903 +---------------+---------------+---------------+--------------+ 6904 x 6906 11.2.2.1. AHSType 6908 The AHSType field is coded as follows: 6910 bit 0-1 - Reserved 6912 bit 2-7 - AHS code 6914 0 - Reserved 6916 1 - Extended CDB 6917 2 - Expected Bidirectional Read Data Length 6919 3 - 63 Reserved 6921 11.2.2.2. AHSLength 6923 This field contains the effective length in bytes of the AHS 6924 excluding AHSType and AHSLength and padding, if any. The AHS is 6925 padded to the smallest integer number of 4 byte words (i.e., from 6926 0 up to 3 padding bytes). 6928 11.2.2.3. Extended CDB AHS 6930 The format of the Extended CDB AHS is: 6932 Byte/ 0 | 1 | 2 | 3 | 6933 / | | | | 6934 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 6935 +---------------+---------------+---------------+--------------+ 6936 0| AHSLength (CDBLength-15) | 0x01 | Reserved | 6937 +---------------+---------------+---------------+--------------+ 6938 4/ ExtendedCDB...+padding / 6939 +/ / 6940 +---------------+---------------+---------------+--------------+ 6941 x 6943 This type of AHS MUST NOT be used if the CDBLength is less than 6944 17. 6945 The length includes the reserved byte 3. 6947 11.2.2.4. Bidirectional Expected Read-Data Length AHS 6949 The format of the Bidirectional Read Expected Data Transfer Length 6950 AHS is: 6952 Byte/ 0 | 1 | 2 | 3 | 6953 / | | | | 6954 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 6955 +---------------+---------------+---------------+--------------+ 6956 0| AHSLength (0x0005) | 0x02 | Reserved | 6957 +---------------+---------------+---------------+--------------+ 6958 4| Expected Read-Data Length | 6959 +---------------+---------------+---------------+--------------+ 6960 8 6962 11.2.3. Header Digest and Data Digest 6964 Optional header and data digests protect the integrity of the 6965 header and data, respectively. The digests, if present, are 6966 located, respectively, after the header and PDU-specific data, and 6967 cover respectively the header and the PDU data, each including 6968 the padding bytes, if any. 6970 The existence and type of digests are negotiated during the Login 6971 Phase. 6973 The separation of the header and data digests is useful in iSCSI 6974 routing applications, in which only the header changes when a 6975 message is forwarded. In this case, only the header digest should 6976 be recalculated. 6978 Digests are not included in data or header length fields. 6980 A zero-length Data Segment also implies a zero-length data-digest. 6982 11.2.4. Data Segment 6984 The (optional) Data Segment contains PDU associated data. Its 6985 payload effective length is provided in the BHS field - 6986 DataSegmentLength. The Data Segment is also padded to an integer 6987 number of 4 byte words. 6989 11.3. SCSI Command 6991 The format of the SCSI Command PDU is: 6993 Byte/ 0 | 1 | 2 | 3 | 6994 / | | | | 6995 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 6996 +---------------+---------------+---------------+--------------+ 6997 0|.|I| 0x01 |F|R|W|. .|ATTR | Reserved | 6998 +---------------+---------------+---------------+--------------+ 6999 4|TotalAHSLength | DataSegmentLength | 7000 +---------------+---------------+---------------+--------------+ 7001 8| Logical Unit Number (LUN) | 7002 + + 7003 12| | 7004 +---------------+---------------+---------------+--------------+ 7005 16| Initiator Task Tag | 7006 +---------------+---------------+---------------+--------------+ 7007 20| Expected Data Transfer Length | 7008 +---------------+---------------+---------------+--------------+ 7009 24| CmdSN | 7010 +---------------+---------------+---------------+--------------+ 7011 28| ExpStatSN | 7012 +---------------+---------------+---------------+--------------+ 7013 32/ SCSI Command Descriptor Block (CDB) / 7014 +/ / 7015 +---------------+---------------+---------------+--------------+ 7016 48/ AHS (Optional) / 7017 +---------------+---------------+---------------+--------------+ 7018 x/ Header Digest (Optional) / 7019 +---------------+---------------+---------------+--------------+ 7020 y/ (DataSegment, Command Data) (Optional) / 7021 +/ / 7022 +---------------+---------------+---------------+--------------+ 7023 z/ Data Digest (Optional) / 7024 +---------------+---------------+---------------+--------------+ 7026 11.3.1. Flags and Task Attributes (byte 1) 7028 The flags for a SCSI Command are: 7030 bit 0 (F) is set to 1 when no unsolicited SCSI Data-Out PDUs 7031 follow this PDU. When F=1 for a write and if Expected Data 7032 Transfer Length is larger than the DataSegmentLength, the 7033 target may solicit additional data through R2T. 7035 bit 1 (R) is set to 1 when the command is expected to input 7036 data. 7038 bit 2 (W) is set to 1 when the command is expected to output 7039 data. 7041 bit 3-4 Reserved. 7043 bit 5-7 contains Task Attributes. 7045 Task Attributes (ATTR) have one of the following integer values 7046 (see [SAM2] for details): 7048 0 - Untagged 7050 1 - Simple 7052 2 - Ordered 7054 3 - Head of Queue 7056 4 - ACA 7058 5-7 - Reserved 7060 At least one of the W and F bits MUST be set to 1. 7061 Either or both of R and W MAY be 1 when either the Expected Data 7062 Transfer Length and/or Bidirectional Read Expected Data Transfer 7063 Length are 0, but they MUST NOT both be 0 when the Expected Data 7064 Transfer Length and/or Bidirectional Read Expected Data Transfer 7065 Length are not 0 (i.e., when some data transfer is expected the 7066 transfer direction is indicated by the R and/or W bit). 7068 11.3.2. CmdSN - Command Sequence Number 7070 Enables ordered delivery across multiple connections in a single 7071 session. 7073 11.3.3. ExpStatSN 7075 Command responses up to ExpStatSN-1 (mod 2**32) have been received 7076 (acknowledges status) on the connection. 7078 11.3.4. Expected Data Transfer Length 7080 For unidirectional operations, the Expected Data Transfer Length 7081 field contains the number of bytes of data involved in this SCSI 7082 operation. For a unidirectional write operation (W flag set to 1 7083 and R flag set to 0), the initiator uses this field to specify the 7084 number of bytes of data it expects to transfer for this operation. 7085 For a unidirectional read operation (W flag set to 0 and R flag 7086 set to 1), the initiator uses this field to specify the number of 7087 bytes of data it expects the target to transfer to the initiator. 7088 It corresponds to the SAM2 byte count. 7090 For bidirectional operations (both R and W flags are set to 1), 7091 this field contains the number of data bytes involved in the write 7092 transfer. For bidirectional operations, an additional header 7093 segment MUST be present in the header sequence that indicates the 7094 Bidirectional Read Expected Data Transfer Length. The Expected 7095 Data Transfer Length field and the Bidirectional Read Expected 7096 Data Transfer Length field correspond to the SAM2 byte count. 7098 If the Expected Data Transfer Length for a write and the length of 7099 the immediate data part that follows the command (if any) are the 7100 same, then no more data PDUs are expected to follow. In this 7101 case, the F bit MUST be set to 1. 7103 If the Expected Data Transfer Length is higher than the 7104 FirstBurstLength (the negotiated maximum amount of unsolicited 7105 data the target will accept), the initiator MUST send the maximum 7106 amount of unsolicited data OR ONLY the immediate data, if any. 7108 Upon completion of a data transfer, the target informs the 7109 initiator (through residual counts) of how many bytes were 7110 actually processed (sent and/or received) by the target. 7112 11.3.5. CDB - SCSI Command Descriptor Block 7114 There are 16 bytes in the CDB field to accommodate the commonly 7115 used CDBs. Whenever the CDB is larger than 16 bytes, an Extended 7116 CDB AHS MUST be used to contain the CDB spillover. 7118 11.3.6. Data Segment - Command Data 7120 Some SCSI commands require additional parameter data to accompany 7121 the SCSI command. This data may be placed beyond the boundary of 7122 the iSCSI header in a data segment. Alternatively, user data 7123 (e.g., from a WRITE operation) can be placed in the data segment 7124 (both cases are referred to as immediate data). These data are 7125 governed by the rules for solicited vs. unsolicited data outlined 7126 in Section 4.2.5.2. 7128 11.4. SCSI Response 7130 The format of the SCSI Response PDU is: 7132 Byte/ 0 | 1 | 2 | 3 | 7133 / | | | | 7134 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 7135 +---------------+---------------+---------------+--------------+ 7136 0|.|.| 0x21 |1|. .|o|u|O|U|.| Response | Status | 7137 +---------------+---------------+---------------+--------------+ 7138 4|TotalAHSLength | DataSegmentLength | 7139 +---------------+---------------+---------------+--------------+ 7140 8| Reserved | 7141 + + 7142 12| | 7143 +---------------+---------------+---------------+--------------+ 7144 16| Initiator Task Tag | 7145 +---------------+---------------+---------------+--------------+ 7146 20| SNACK Tag or Reserved | 7147 +---------------+---------------+---------------+--------------+ 7148 24| StatSN | 7149 +---------------+---------------+---------------+--------------+ 7150 28| ExpCmdSN | 7151 +---------------+---------------+---------------+--------------+ 7152 32| MaxCmdSN | 7153 +---------------+---------------+---------------+--------------+ 7154 36| ExpDataSN or Reserved | 7155 +---------------+---------------+---------------+--------------+ 7156 40| Bidirectional Read Residual Count or Reserved | 7157 +---------------+---------------+---------------+--------------+ 7158 44| Residual Count or Reserved | 7159 +---------------+---------------+---------------+--------------+ 7160 48| Header-Digest (Optional) | 7161 +---------------+---------------+---------------+--------------+ 7162 / Data Segment (Optional) / 7163 +/ / 7164 +---------------+---------------+---------------+--------------+ 7165 | Data-Digest (Optional) | 7166 +---------------+---------------+---------------+--------------+ 7168 11.4.1. Flags (byte 1) 7170 bit 1-2 Reserved. 7172 bit 3 - (o) set for Bidirectional Read Residual Overflow. In 7173 this case, the Bidirectional Read Residual Count indicates 7174 the number of bytes that were not transferred to the 7175 initiator because the initiator's Expected Bidirectional 7176 Read Data Transfer Length was not sufficient. 7178 bit 4 - (u) set for Bidirectional Read Residual Underflow. In 7179 this case, the Bidirectional Read Residual Count indicates 7180 the number of bytes that were not transferred to the 7181 initiator out of the number of bytes expected to be 7182 transferred. 7184 bit 5 - (O) set for Residual Overflow. In this case, the 7185 Residual Count indicates the number of bytes that were not 7186 transferred because the initiator's Expected Data Transfer 7187 Length was not sufficient. For a bidirectional operation, 7188 the Residual Count contains the residual for the write 7189 operation. 7191 bit 6 - (U) set for Residual Underflow. In this case, the 7192 Residual Count indicates the number of bytes that were not 7193 transferred out of the number of bytes that were expected to 7194 be transferred. For a bidirectional operation, the Residual 7195 Count contains the residual for the write operation. 7197 bit 7 - (0) Reserved. 7199 Bits O and U and bits o and u are mutually exclusive (i.e., having 7200 both o and u or O and U set to 1 is a protocol error). 7202 For a response other than "Command Completed at Target", bits 3-6 7203 MUST be 0. 7205 11.4.2. Status 7207 The Status field is used to report the SCSI status of the command 7208 (as specified in [SAM2]) and is only valid if the Response Code is 7209 Command Completed at target. 7211 Some of the status codes defined in [SAM2] are: 7213 0x00 GOOD 7214 0x02 CHECK CONDITION 7216 0x08 BUSY 7218 0x18 RESERVATION CONFLICT 7220 0x28 TASK SET FULL 7222 0x30 ACA ACTIVE 7224 0x40 TASK ABORTED 7226 See [SAM2] for the complete list and definitions. 7228 If a SCSI device error is detected while data from the initiator 7229 is still expected (the command PDU did not contain all the data 7230 and the target has not received a Data PDU with the final bit 7231 Set), the target MUST wait until it receives a Data PDU with the F 7232 bit set in the last expected sequence before sending the Response 7233 PDU. 7235 11.4.3. Response 7237 This field contains the iSCSI service response. 7239 iSCSI service response codes defined in this specification are: 7241 0x00 - Command Completed at Target 7243 0x01 - Target Failure 7245 0x80-0xff - Vendor specific 7247 All other response codes are reserved. 7249 The Response is used to report a Service Response. The mapping of 7250 the response code into a SCSI service response code value, if 7251 needed, is outside the scope of this document. However, in 7252 symbolic terms response value 0x00 maps to the SCSI service 7253 response (see [SAM2] and [SPC3]) of TASK COMPLETE or LINKED 7254 COMMAND COMPLETE. All other Response values map to the SCSI 7255 service response of SERVICE DELIVERY OR TARGET FAILURE. 7257 If a SCSI Response PDU does not arrive before the session is 7258 terminated, the SCSI service response is SERVICE DELIVERY OR 7259 TARGET FAILURE. 7261 A non-zero response field indicates a failure to execute the 7262 command in which case the Status and Flag fields are undefined and 7263 MUST be ignored on reception. 7265 11.4.4. SNACK Tag 7267 This field contains a copy of the SNACK Tag of the last SNACK Tag 7268 accepted by the target on the same connection and for the command 7269 for which the response is issued. Otherwise it is reserved and 7270 should be set to 0. 7272 After issuing a R-Data SNACK the initiator must discard any SCSI 7273 status unless contained in an SCSI Response PDU carrying the same 7274 SNACK Tag as the last issued R-Data SNACK for the SCSI command on 7275 the current connection. 7277 For a detailed discussion on R-Data SNACK see 11.16.3. 7279 11.4.5. Residual Count 7281 11.4.5.1. Field Semantics 7283 The Residual Count field MUST be valid in the case where either 7284 the U bit or the O bit is set. If neither bit is set, the Residual 7285 Count field MUST be ignored on reception and SHOULD be set to 0 7286 when sending. Targets may set the residual count and initiators 7287 may use it when the response code is "completed at target" (even 7288 if the status returned is not GOOD). If the O bit is set, the 7289 Residual Count indicates the number of bytes that were not 7290 transferred because the initiator's Expected Data Transfer Length 7291 was not sufficient. If the U bit is set, the Residual Count 7292 indicates the number of bytes that were not transferred out of the 7293 number of bytes expected to be transferred. 7295 11.4.5.2. Residuals Concepts Overview 7297 SCSI-Presented Data Transfer Length (SPDTL) is the term this 7298 document uses (see Section 2.1 for definition) to represent the 7299 aggregate data length that the target SCSI layer attempts to 7300 transfer using the local iSCSI layer for a task. Expected Data 7301 Transfer Length (EDTL) is the iSCSI term that represents the 7302 length of data that the iSCSI layer expects to transfer for a 7303 task. EDTL is specified in the SCSI Command PDU. 7305 When SPDTL = EDTL for a task, the target iSCSI layer completes the 7306 task with no residuals. Whenever SPDTL differs from EDTL for a 7307 task, that task is said to have a residual. 7309 If SPDTL > EDTL for a task, iSCSI Overflow MUST be signaled in the 7310 SCSI Response PDU as specified in Section 11.4.5.1. The Residual 7311 Count MUST be set to the numerical value of (SPDTL - EDTL). 7313 If SPDTL < EDTL for a task, iSCSI Underflow MUST be signaled in 7314 the SCSI Response PDU as specified in Section 11.4.5.1. The 7315 Residual Count MUST be set to the numerical value of (EDTL - 7316 SPDTL). 7318 Note that the Overflow and Underflow scenarios are independent of 7319 Data-In and Data-Out. Either scenario is logically possible in 7320 either direction of data transfer. 7322 11.4.5.3. SCSI REPORT LUNS and Residual Overflow 7324 This Section discusses the residual overflow issues citing the 7325 example of the SCSI REPORT LUNS command. Note however that there 7326 are several SCSI commands (e.g., INQUIRY) with ALLOCATION LENGTH 7327 fields following the same underlying rules. The semantics in the 7328 rest of the Section apply to all such SCSI commands. 7330 The specification of the SCSI REPORT LUNS command requires that 7331 the SCSI target limit the amount of data transferred to a maximum 7332 size (ALLOCATION LENGTH) provided by the initiator in the REPORT 7333 LUNS CDB. 7335 If the Expected Data Transfer Length (EDTL) in the iSCSI header of 7336 the SCSI Command PDU for a REPORT LUNS command is set to at least 7337 as large as that ALLOCATION LENGTH, the SCSI layer truncation 7338 prevents an iSCSI Residual Overflow from occurring. A SCSI 7339 initiator can detect that such truncation has occurred via other 7340 information at the SCSI layer. The rest of the Section elaborates 7341 this required behavior. 7343 The SCSI REPORT LUNS command requests a target SCSI layer to 7344 return a logical unit inventory (LUN list) to the initiator SCSI 7345 layer (see Section 6.21 of [SPC3]). The size of this LUN list may 7346 not be known to the initiator SCSI layer when it issues the REPORT 7347 LUNS command; to avoid transferring more LUN list data than the 7348 initiator is prepared for, the REPORT LUNS CDB contains an 7349 ALLOCATION LENGTH field to specify the maximum amount of data to 7350 be transferred to the initiator for this command. If the initiator 7351 SCSI layer has underestimated the number of logical units at the 7352 target, it is possible that the complete logical unit inventory 7353 does not fit in the specified ALLOCATION LENGTH. In this 7354 situation, Section 4.3.3.6 in [SPC3] requires that the target SCSI 7355 layer "shall terminate transfers to the Data-In Buffer" when the 7356 number of bytes specified by the ALLOCATION LENGTH field have been 7357 transferred. 7359 Therefore, in response to a REPORT LUNS command, the SCSI layer at 7360 the target presents at most ALLOCATION LENGTH bytes of data 7361 (logical unit inventory) to iSCSI for transfer to the initiator. 7362 For a REPORT LUNS command, if the iSCSI EDTL is at least as large 7363 as the ALLOCATION LENGTH, the SCSI truncation ensures that the 7364 EDTL will accommodate all of the data to be transferred. If all of 7365 the logical unit inventory data presented to the iSCSI layer -- 7366 i.e., the data remaining after any SCSI truncation -- is 7367 transferred to the initiator by the iSCSI layer, an iSCSI Residual 7368 Overflow has not occurred and the iSCSI (O) bit MUST NOT be set in 7369 the SCSI Response or final SCSI Data-Out PDU. Note that this 7370 behavior is implied by the combination of Section 11.4.5.1 along 7371 with the specification of the REPORT LUNS command in [SPC3]. 7372 However, if the iSCSI EDTL is larger than the ALLOCATION LENGTH in 7373 this scenario, note that the iSCSI Underflow MUST be signaled in 7374 the SCSI Response PDU. An iSCSI Underflow MUST also be signaled 7375 when the iSCSI EDTL is equal to the ALLOCATION LENGTH but the 7376 logical unit inventory data presented to the iSCSI layer is 7377 smaller than the ALLOCATION LENGTH. 7379 The LUN LIST LENGTH field in the logical unit inventory (the first 7380 field in the inventory) is not affected by truncation of the 7381 inventory to fit in ALLOCATION LENGTH; this enables a SCSI 7382 initiator to determine that the received inventory is incomplete 7383 by noticing that the LUN LIST LENGTH in the inventory is larger 7384 than the ALLOCATION LENGTH that was sent in the REPORT LUNS CDB. A 7385 common initiator behavior in this situation is to re-issue the 7386 REPORT LUNS command with a larger ALLOCATION LENGTH. 7388 11.4.6. Bidirectional Read Residual Count 7390 The Bidirectional Read Residual Count field MUST be valid in the 7391 case where either the u bit or the o bit is set. If neither bit is 7392 set, the Bidirectional Read Residual Count field is reserved. 7393 Targets may set the Bidirectional Read Residual Count and 7394 initiators may use it when the response code is "completed at 7395 target". If the o bit is set, the Bidirectional Read Residual 7396 Count indicates the number of bytes that were not transferred to 7397 the initiator because the initiator's Expected Bidirectional Read 7398 Transfer Length was not sufficient. If the u bit is set, the 7399 Bidirectional Read Residual Count indicates the number of bytes 7400 that were not transferred to the initiator out of the number of 7401 bytes expected to be transferred. 7403 11.4.7. Data Segment - Sense and Response Data Segment 7405 iSCSI targets MUST support and enable autosense. If Status is 7406 CHECK CONDITION (0x02), then the Data Segment MUST contain sense 7407 data for the failed command. 7409 For some iSCSI responses, the response data segment MAY contain 7410 some response related information, (e.g., for a target failure, it 7411 may contain a vendor specific detailed description of the 7412 failure). 7414 If the DataSegmentLength is not 0, the format of the Data Segment 7415 is as follows: 7417 Byte/ 0 | 1 | 2 | 3 | 7418 / | | | | 7419 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 7420 +---------------+---------------+---------------+--------------+ 7421 0|SenseLength | Sense Data | 7422 +---------------+---------------+---------------+--------------+ 7423 x/ Sense Data / 7424 +---------------+---------------+---------------+--------------+ 7425 y/ Response Data / 7426 / / 7427 +---------------+---------------+---------------+--------------+ 7429 11.4.7.1. SenseLength 7431 Length of Sense Data. 7433 11.4.7.2. Sense Data 7435 The Sense Data contains detailed information about a check 7436 condition and [SPC3] specifies the format and content of the Sense 7437 Data. 7439 Certain iSCSI conditions result in the command being terminated at 7440 the target (response Command Completed at Target) with a SCSI 7441 Check Condition Status as outlined in the next table: 7443 +--------------------------+----------+--------------------------+ 7444 | iSCSI Condition |Sense | Additional Sense Code & | 7445 | |Key | Qualifier | 7446 +--------------------------+----------+--------------------------+ 7447 | Unexpected unsolicited |Aborted | ASC = 0x0c ASCQ = 0x0c | 7448 | data |Command-0B| Write Error | 7449 +--------------------------+----------+--------------------------+ 7450 | Incorrect amount of data |Aborted | ASC = 0x0c ASCQ = 0x0d | 7451 | |Command-0B| Write Error | 7452 +--------------------------+----------+--------------------------+ 7453 | Protocol Service CRC |Aborted | ASC = 0x47 ASCQ = 0x05 | 7454 | error |Command-0B| CRC Error Detected | 7455 +--------------------------+----------+--------------------------+ 7456 | SNACK rejected |Aborted | ASC = 0x11 ASCQ = 0x13 | 7457 | |Command-0B| Read Error | 7458 +--------------------------+----------+--------------------------+ 7460 The target reports the "Incorrect amount of data" condition if 7461 during data output the total data length to output is greater than 7462 FirstBurstLength and the initiator sent unsolicited non-immediate 7463 data but the total amount of unsolicited data is different than 7464 FirstBurstLength. The target reports the same error when the 7465 amount of data sent as a reply to an R2T does not match the amount 7466 requested. 7468 11.4.8. ExpDataSN 7470 The number of Data-In (read) PDUs the target has sent for the 7471 command. 7473 This field MUST be 0 if the response code is not Command Completed 7474 at Target or the target sent no Data-In PDUs for the command. 7476 11.4.9. StatSN - Status Sequence Number 7478 StatSN is a Sequence Number that the target iSCSI layer generates 7479 per connection and that in turn, enables the initiator to 7480 acknowledge status reception. StatSN is incremented by 1 for every 7481 response/status sent on a connection except for responses sent as 7482 a result of a retry or SNACK. In the case of responses sent due to 7483 a retransmission request, the StatSN MUST be the same as the first 7484 time the PDU was sent unless the connection has since been 7485 restarted. 7486 11.4.10. ExpCmdSN - Next Expected CmdSN from this Initiator 7488 ExpCmdSN is a Sequence Number that the target iSCSI returns to the 7489 initiator to acknowledge command reception. It is used to update a 7490 local variable with the same name. An ExpCmdSN equal to MaxCmdSN+1 7491 indicates that the target cannot accept new commands. 7493 11.4.11. MaxCmdSN - Maximum CmdSN from this Initiator 7495 MaxCmdSN is a Sequence Number that the target iSCSI returns to the 7496 initiator to indicate the maximum CmdSN the initiator can send. It 7497 is used to update a local variable with the same name. If MaxCmdSN 7498 is equal to ExpCmdSN-1, this indicates to the initiator that the 7499 target cannot receive any additional commands. When MaxCmdSN 7500 changes at the target while the target has no pending PDUs to 7501 convey this information to the initiator, it MUST generate a NOP- 7502 IN to carry the new MaxCmdSN. 7504 11.5. Task Management Function Request 7506 Byte/ 0 | 1 | 2 | 3 | 7507 / | | | | 7508 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 7509 +---------------+---------------+---------------+--------------+ 7510 0|.|I| 0x02 |1| Function | Reserved | 7511 +---------------+---------------+---------------+--------------+ 7512 4|TotalAHSLength | DataSegmentLength | 7513 +---------------+---------------+---------------+--------------+ 7514 8| Logical Unit Number (LUN) or Reserved | 7515 + + 7516 12| | 7517 +---------------+---------------+---------------+--------------+ 7518 16| Initiator Task Tag | 7519 +---------------+---------------+---------------+--------------+ 7520 20| Referenced Task Tag or 0xffffffff | 7521 +---------------+---------------+---------------+--------------+ 7522 24| CmdSN | 7523 +---------------+---------------+---------------+--------------+ 7524 28| ExpStatSN | 7525 +---------------+---------------+---------------+--------------+ 7526 32| RefCmdSN or Reserved | 7527 +---------------+---------------+---------------+--------------+ 7528 36| ExpDataSN or Reserved | 7529 +---------------+---------------+---------------+--------------+ 7530 40/ Reserved / 7531 +/ / 7532 +---------------+---------------+---------------+--------------+ 7533 48| Header-Digest (Optional) | 7534 +---------------+---------------+---------------+--------------+ 7536 11.5.1. Function 7538 The Task Management functions provide an initiator with a way to 7539 explicitly control the execution of one or more Tasks (SCSI and 7540 iSCSI tasks). The Task Management function codes are listed below. 7541 For a more detailed description of SCSI task management, see 7542 [SAM2]. 7544 1 - ABORT TASK - aborts the task identified by the 7545 Referenced Task Tag field. 7547 2 - ABORT TASK SET - aborts all Tasks issued via this 7548 session on the logical unit. 7550 3 - CLEAR ACA - clears the Auto Contingent Allegiance 7551 condition. 7553 4 - CLEAR TASK SET - aborts all Tasks in the appropriate 7554 task set as defined by the TST field in the Control mode 7555 page (see [SPC3]). 7557 5 - LOGICAL UNIT RESET 7559 6 - TARGET WARM RESET 7561 7 - TARGET COLD RESET 7563 8 - TASK REASSIGN - reassigns connection allegiance for the 7564 task identified by the Initiator Task Tag field to this 7565 connection, thus resuming the iSCSI exchanges for the task. 7567 All other possible values for Function field are reserved. 7569 For all these functions, the Task Management function response 7570 MUST be returned as detailed in Section 11.6. All these functions 7571 apply to the referenced tasks regardless of whether they are 7572 proper SCSI tasks or tagged iSCSI operations. Task management 7573 requests must act on all the commands from the same session having 7574 a CmdSN lower than the task management CmdSN. LOGICAL UNIT RESET, 7575 TARGET WARM RESET and TARGET COLD RESET may affect commands from 7576 other sessions or commands from the same session regardless of 7577 their CmdSN value. 7579 If the task management request is marked for immediate delivery, 7580 it must be considered immediately for execution, but the 7581 operations involved (all or part of them) may be postponed to 7582 allow the target to receive all relevant tasks. According to 7583 [SAM2], for all the tasks covered by the Task Management response 7584 (i.e., with CmdSN lower than the task management command CmdSN) 7585 but except the Task Management response to a TASK REASSIGN, 7586 additional responses MUST NOT be delivered to the SCSI layer after 7587 the Task Management response. The iSCSI initiator MAY deliver to 7588 the SCSI layer all responses received before the Task Management 7589 response (i.e., it is a matter of implementation if the SCSI 7590 responses, received before the Task Management response but after 7591 the task management request was issued, are delivered to the SCSI 7592 layer by the iSCSI layer in the initiator). The iSCSI target MUST 7593 ensure that no responses for the tasks covered by a task 7594 management function are delivered to the iSCSI initiator after the 7595 Task Management response except for a task covered by a TASK 7596 REASSIGN. 7598 For ABORT TASK SET and CLEAR TASK SET, the issuing initiator MUST 7599 continue to respond to all valid target transfer tags (received 7600 via R2T, Text Response, NOP-In, or SCSI Data-in PDUs) related to 7601 the affected task set, even after issuing the task management 7602 request. The issuing initiator SHOULD however terminate (i.e., by 7603 setting the F-bit to 1) these response sequences as quickly as 7604 possible. The target on its part MUST wait for responses on all 7605 affected target transfer tags before acting on either of these two 7606 task management requests. In case all or part of the response 7607 sequence is not received (due to digest errors) for a valid TTT, 7608 the target MAY treat it as a case of within-command error recovery 7609 class (see Section 7.1.4.1) if it is supporting ErrorRecoveryLevel 7610 >= 1, or alternatively may drop the connection to complete the 7611 requested task set function. 7613 If an ABORT TASK is issued for a task created by an immediate 7614 command then RefCmdSN MUST be that of the Task Management request 7615 itself (i.e. CmdSN and RefCmdSN are equal); otherwise RefCmdSN 7616 MUST be set to the CmdSN of the task to be aborted (lower than 7617 CmdSN). 7619 If the connection is still active (it is not undergoing an 7620 implicit or explicit logout), ABORT TASK MUST be issued on the 7621 same connection to which the task to be aborted is allegiant at 7622 the time the Task Management Request is issued. If the connection 7623 is implicitly or explicitly logged out (i.e., no other request 7624 will be issued on the failing connection and no other response 7625 will be received on the failing connection), then an ABORT TASK 7626 function request may be issued on another connection. This Task 7627 Management request will then establish a new allegiance for the 7628 command to be aborted as well as abort it (i.e., the task to be 7629 aborted will not have to be retried or reassigned, and its status, 7630 if sent but not acknowledged, will be resent followed by the Task 7631 Management response). 7633 At the target an ABORT TASK function MUST NOT be executed on a 7634 Task Management request; such a request MUST result in Task 7635 Management response of "Function rejected". 7637 For the LOGICAL UNIT RESET function, the target MUST behave as 7638 dictated by the Logical Unit Reset function in [SAM2]. 7640 The implementation of the TARGET WARM RESET function and the 7641 TARGET COLD RESET function is OPTIONAL and when implemented, 7642 should act as described below. The TARGET WARM RESET is also 7643 subject to SCSI access controls on the requesting initiator as 7644 defined in [SPC3]. When authorization fails at the target, the 7645 appropriate response as described in Section 11.6.1 MUST be 7646 returned by the target. The TARGET COLD RESET function is not 7647 subject to SCSI access controls, but its execution privileges may 7648 be managed by iSCSI mechanisms such as login authentication. 7650 When executing the TARGET WARM RESET and TARGET COLD RESET 7651 functions, the target cancels all pending operations on all 7652 Logical Units known by the issuing initiator. Both functions are 7653 equivalent to the Target Reset function specified by [SAM2]. They 7654 can affect many other initiators logged in with the servicing SCSI 7655 target port. 7657 The target MUST treat the TARGET COLD RESET function additionally 7658 as a power on event, thus terminating all of its TCP connections 7659 to all initiators (all sessions are terminated). For this reason, 7660 the Service Response (defined by [SAM2]) for this SCSI task 7661 management function may not be reliably delivered to the issuing 7662 initiator port. 7664 For the TASK REASSIGN function, the target should reassign the 7665 connection allegiance to this new connection (and thus resume 7666 iSCSI exchanges for the task). TASK REASSIGN MUST ONLY be received 7667 by the target after the connection on which the command was 7668 previously executing has been successfully logged-out. The Task 7669 Management response MUST be issued before the reassignment becomes 7670 effective. 7672 For additional usage semantics see Section 7.2. 7674 At the target a TASK REASSIGN function request MUST NOT be 7675 executed to reassign the connection allegiance of a Task 7676 Management function request, an active text negotiation task, or a 7677 Logout task; such a request MUST result in Task Management 7678 response of "Function rejected". 7680 TASK REASSIGN MUST be issued as an immediate command. 7682 11.5.2. TotalAHSLength and DataSegmentLength 7684 For this PDU, TotalAHSLength and DataSegmentLength MUST be 0. 7686 11.5.3. LUN 7688 This field is required for functions that address a specific LU 7689 (ABORT TASK, CLEAR TASK SET, ABORT TASK SET, CLEAR ACA, LOGICAL 7690 UNIT RESET) and is reserved in all others. 7692 11.5.4. Referenced Task Tag 7694 The Initiator Task Tag of the task to be aborted for the ABORT 7695 TASK function or reassigned for the TASK REASSIGN function. For 7696 all the other functions this field MUST be set to the reserved 7697 value 0xffffffff. 7699 11.5.5. RefCmdSN 7701 If an ABORT TASK is issued for a task created by an immediate 7702 command then RefCmdSN MUST be that of the Task Management request 7703 itself (i.e. CmdSN and RefCmdSN are equal). 7705 For an ABORT TASK of a task created by non-immediate command 7706 RefCmdSN MUST be set to the CmdSN of the task identified by the 7707 Referenced Task Tag field. Targets must use this field as 7708 described in Section 11.6.1 when the task identified by the 7709 Referenced Task Tag field is not with the target. 7711 Otherwise, this field is reserved. 7713 11.5.6. ExpDataSN 7715 For recovery purposes, the iSCSI target and initiator maintain a 7716 data acknowledgement reference number - the first input DataSN 7717 number unacknowledged by the initiator. When issuing a new 7718 command, this number is set to 0. If the function is TASK 7719 REASSIGN, which establishes a new connection allegiance for a 7720 previously issued Read or Bidirectional command, ExpDataSN will 7721 contain an updated data acknowledgement reference number or the 7722 value 0; the latter indicating that the data acknowledgement 7723 reference number is unchanged. The initiator MUST discard any data 7724 PDUs from the previous execution that it did not acknowledge and 7725 the target MUST transmit all Data-in PDUs (if any) starting with 7726 the data acknowledgement reference number. The number of 7727 retransmitted PDUs may or may not be the same as the original 7728 transmission depending on if there was a change in 7729 MaxRecvDataSegmentLength in the reassignment. The target MAY also 7730 send no more Data-In PDUs if all data has been acknowledged. 7732 The value of ExpDataSN MUST be 0 or higher than the DataSN of the 7733 last acknowledged Data-In PDU, but not larger than DataSN+1 of the 7734 last Data-IN PDU sent by the target. Any other value MUST be 7735 ignored by the target. 7737 For other functions this field is reserved. 7739 11.6. Task Management Function Response 7740 Byte/ 0 | 1 | 2 | 3 | 7741 / | | | | 7742 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 7743 +---------------+---------------+---------------+--------------+ 7744 0|.|.| 0x22 |1| Reserved | Response | Reserved | 7745 +---------------+---------------+---------------+--------------+ 7746 4|TotalAHSLength | DataSegmentLength | 7747 +--------------------------------------------------------------+ 7748 8/ Reserved / 7749 / / 7750 +---------------+---------------+---------------+--------------+ 7751 16| Initiator Task Tag | 7752 +---------------+---------------+---------------+--------------+ 7753 20| Reserved | 7754 +---------------+---------------+---------------+--------------+ 7755 24| StatSN | 7756 +---------------+---------------+---------------+--------------+ 7757 28| ExpCmdSN | 7758 +---------------+---------------+---------------+--------------+ 7759 32| MaxCmdSN | 7760 +---------------+---------------+---------------+--------------+ 7761 36/ Reserved / 7762 +/ / 7763 +---------------+---------------+---------------+--------------+ 7764 48| Header-Digest (Optional) | 7765 +---------------+---------------+---------------+--------------+ 7767 For the functions ABORT TASK, ABORT TASK SET, CLEAR ACA, CLEAR 7768 TASK SET, LOGICAL UNIT RESET, TARGET COLD RESET, TARGET WARM RESET 7769 and TASK REASSIGN, the target performs the requested Task 7770 Management function and sends a Task Management response back to 7771 the initiator. For TASK REASSIGN, the new connection allegiance 7772 MUST ONLY become effective at the target after the target issues 7773 the Task Management Response. 7775 11.6.1. Response 7777 The target provides a Response, which may take on the following 7778 values: 7780 0 - Function complete. 7781 1 - Task does not exist. 7783 2 - LUN does not exist. 7784 3 - Task still allegiant. 7785 4 - Task allegiance reassignment not supported. 7786 5 - Task management function not supported. 7787 6 - Function authorization failed. 7788 255 - Function rejected. 7790 All other values are reserved. 7792 For a discussion on usage of response codes 3 and 4, see Section 7793 7.2.2. 7795 For the TARGET COLD RESET and TARGET WARM RESET functions, the 7796 target cancels all pending operations across all Logical Units 7797 known to the issuing initiator. For the TARGET COLD RESET 7798 function, the target MUST then close all of its TCP connections to 7799 all initiators (terminates all sessions). 7801 The mapping of the response code into a SCSI service response code 7802 value, if needed, is outside the scope of this document. However, 7803 in symbolic terms, Response values 0 and 1 map to the SCSI service 7804 response of FUNCTION COMPLETE. Response value 2 maps to SCSI 7805 service response of INCORRECT LOGICAL UNIT NUMBER. All other 7806 Response values map to the SCSI service response of FUNCTION 7807 REJECTED. If a Task Management function response PDU does not 7808 arrive before the session is terminated, the SCSI service response 7809 is SERVICE DELIVERY OR TARGET FAILURE. 7811 The response to ABORT TASK SET and CLEAR TASK SET MUST only be 7812 issued by the target after all of the commands affected have been 7813 received by the target, the corresponding task management 7814 functions have been executed by the SCSI target, and the delivery 7815 of all responses delivered until the task management function 7816 completion have been confirmed (acknowledged through ExpStatSN) by 7817 the initiator on all connections of this session. For the exact 7818 timeline of events, refer to Section 4.2.3.3 and Section 4.2.3.4. 7820 For the ABORT TASK function, 7821 a) If the Referenced Task Tag identifies a valid task leading 7822 to a successful termination, then targets must return the 7823 "Function complete" response. 7824 b) If the Referenced Task Tag does not identify an existing 7825 task, but if the CmdSN indicated by the RefCmdSN field in 7826 the Task Management function request is within the valid 7827 CmdSN window and less than the CmdSN of the Task Management 7828 function request itself, then targets must consider the 7829 CmdSN received and return the "Function complete" response. 7830 c) If the Referenced Task Tag does not identify an existing 7831 task and if the CmdSN indicated by the RefCmdSN field in 7832 the Task Management function request is outside the valid 7833 CmdSN window, then targets must return the "Task does not 7834 exist" response. 7836 For response semantics on function types that can potentially 7837 impact multiple active tasks on the target, see Section 4.2.3. 7839 11.6.2. TotalAHSLength and DataSegmentLength 7841 For this PDU TotalAHSLength and DataSegmentLength MUST be 0. 7843 11.7. SCSI Data-out & SCSI Data-in 7845 The SCSI Data-out PDU for WRITE operations has the following 7846 format: 7848 Byte/ 0 | 1 | 2 | 3 | 7849 / | | | | 7850 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 7851 +---------------+---------------+---------------+--------------+ 7852 0|.|.| 0x05 |F| Reserved | 7853 +---------------+---------------+---------------+--------------+ 7854 4|TotalAHSLength | DataSegmentLength | 7855 +---------------+---------------+---------------+--------------+ 7856 8| LUN or Reserved | 7857 + + 7858 12| | 7859 +---------------+---------------+---------------+--------------+ 7860 16| Initiator Task Tag | 7861 +---------------+---------------+---------------+--------------+ 7862 20| Target Transfer Tag or 0xffffffff | 7863 +---------------+---------------+---------------+--------------+ 7864 24| Reserved | 7865 +---------------+---------------+---------------+--------------+ 7866 28| ExpStatSN | 7867 +---------------+---------------+---------------+--------------+ 7868 32| Reserved | 7869 +---------------+---------------+---------------+--------------+ 7870 36| DataSN | 7871 +---------------+---------------+---------------+--------------+ 7872 40| Buffer Offset | 7873 +---------------+---------------+---------------+--------------+ 7874 44| Reserved | 7875 +---------------+---------------+---------------+--------------+ 7876 48| Header-Digest (Optional) | 7877 +---------------+---------------+---------------+--------------+ 7878 / DataSegment / 7879 +/ / 7880 +---------------+---------------+---------------+--------------+ 7881 | Data-Digest (Optional) | 7882 +---------------+---------------+---------------+--------------+ 7884 The SCSI Data-in PDU for READ operations has the following format: 7886 Byte/ 0 | 1 | 2 | 3 | 7887 / | | | | 7888 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 7889 +---------------+---------------+---------------+--------------+ 7890 0|.|.| 0x25 |F|A|0 0 0|O|U|S| Reserved |Status or Rsvd| 7891 +---------------+---------------+---------------+--------------+ 7892 4|TotalAHSLength | DataSegmentLength | 7893 +---------------+---------------+---------------+--------------+ 7894 8| LUN or Reserved | 7895 + + 7896 12| | 7897 +---------------+---------------+---------------+--------------+ 7898 16| Initiator Task Tag | 7899 +---------------+---------------+---------------+--------------+ 7900 20| Target Transfer Tag or 0xffffffff | 7901 +---------------+---------------+---------------+--------------+ 7902 24| StatSN or Reserved | 7903 +---------------+---------------+---------------+--------------+ 7904 28| ExpCmdSN | 7905 +---------------+---------------+---------------+--------------+ 7906 32| MaxCmdSN | 7907 +---------------+---------------+---------------+--------------+ 7908 36| DataSN | 7909 +---------------+---------------+---------------+--------------+ 7910 40| Buffer Offset | 7911 +---------------+---------------+---------------+--------------+ 7912 44| Residual Count | 7913 +---------------+---------------+---------------+--------------+ 7914 48| Header-Digest (Optional) | 7915 +---------------+---------------+---------------+--------------+ 7916 / DataSegment / 7917 +/ / 7918 +---------------+---------------+---------------+--------------+ 7919 | Data-Digest (Optional) | 7920 +---------------+---------------+---------------+--------------+ 7922 Status can accompany the last Data-in PDU if the command did not 7923 end with an exception (i.e., the status is "good status" - GOOD, 7924 CONDITION MET or INTERMEDIATE CONDITION MET). The presence of 7925 status (and of a residual count) is signaled though the S flag 7926 bit. Although targets MAY choose to send even non-exception 7927 status in separate responses, initiators MUST support non- 7928 exception status in Data-In PDUs. 7930 11.7.1. F (Final) Bit 7932 For outgoing data, this bit is 1 for the last PDU of unsolicited 7933 data or the last PDU of a sequence that answers an R2T. 7935 For incoming data, this bit is 1 for the last input (read) data 7936 PDU of a sequence. Input can be split into several sequences, 7937 each having its own F bit. Splitting the data stream into 7938 sequences does not affect DataSN counting on Data-In PDUs. It MAY 7939 be used as a "change direction" indication for Bidirectional 7940 operations that need such a change. 7942 DataSegmentLength MUST NOT exceed MaxRecvDataSegmentLength for the 7943 direction it is sent and the total of all the DataSegmentLength of 7944 all PDUs in a sequence MUST NOT exceed MaxBurstLength (or 7945 FirstBurstLength for unsolicited data). However the number of 7946 individual PDUs in a sequence (or in total) may be higher than the 7947 MaxBurstLength (or FirstBurstLength) to MaxRecvDataSegmentLength 7948 ratio (as PDUs may be limited in length by the sender 7949 capabilities). Using DataSegmentLength of 0 may increase beyond 7950 what is reasonable for the number of PDUs and should therefore be 7951 avoided. 7953 For Bidirectional operations, the F bit is 1 for both the end of 7954 the input sequences and the end of the output sequences. 7956 11.7.2. A (Acknowledge) bit 7958 For sessions with ErrorRecoveryLevel 1 or higher, the target sets 7959 this bit to 1 to indicate that it requests a positive 7960 acknowledgement from the initiator for the data received. The 7961 target should use the A bit moderately; it MAY only set the A bit 7962 to 1 once every MaxBurstLength bytes, or on the last Data-In PDU 7963 that concludes the entire requested read data transfer for the 7964 task from the target's perspective, and it MUST NOT do so more 7965 frequently. The target MUST NOT set to 1 the A bit for sessions 7966 with ErrorRecoveryLevel=0. The initiator MUST ignore the A bit set 7967 to 1 for sessions with ErrorRecoveryLevel=0. 7969 On receiving a Data-In PDU with the A bit set to 1 on a session 7970 with ErrorRecoveryLevel greater than 0, if there are no holes in 7971 the read data until that Data-In PDU, the initiator MUST issue a 7972 SNACK of type DataACK except when it is able to acknowledge the 7973 status for the task immediately via ExpStatSN on other outbound 7974 PDUs if the status for the task is also received. In the latter 7975 case (acknowledgement through ExpStatSN), sending a SNACK of type 7976 DataACK in response to the A bit is OPTIONAL, but if it is done, 7977 it must not be sent after the status acknowledgement through 7978 ExpStatSN. If the initiator has detected holes in the read data 7979 prior to that Data-In PDU, it MUST postpone issuing the SNACK of 7980 type DataACK until the holes are filled. An initiator also MUST 7981 NOT acknowledge the status for the task before those holes are 7982 filled. A status acknowledgement for a task that generated the 7983 Data-In PDUs is considered by the target as an implicit 7984 acknowledgement of the Data-In PDUs if such an acknowledgement was 7985 requested by the target. 7987 11.7.3. Flags (byte 1) 7989 The last SCSI Data packet sent from a target to an initiator for a 7990 SCSI command that completed successfully (with a status of GOOD, 7991 CONDITION MET, INTERMEDIATE or INTERMEDIATE CONDITION MET) may 7992 also optionally contain the Status for the data transfer. In this 7993 case, Sense Data cannot be sent together with the Command Status. 7994 If the command is completed with an error, then the response and 7995 sense data MUST be sent in a SCSI Response PDU (i.e., MUST NOT be 7996 sent in a SCSI Data packet). For Bidirectional commands, the 7997 status MUST be sent in a SCSI Response PDU. 7999 bit 2-4 - Reserved. 8001 bit 5-6 - used the same as in a SCSI Response. These bits are 8002 only valid when S is set to 1. For details see SNACK . 8004 bit 7 S (status)- set to indicate that the Command Status 8005 field contains status. If this bit is set to 1, the F bit 8006 MUST also be set to 1. 8008 The fields StatSN, Status, and Residual Count only have meaningful 8009 content if the S bit is set to 1 and their values are defined in 8010 SNACK . 8012 11.7.4. Target Transfer Tag and LUN 8014 On outgoing data, the Target Transfer Tag is provided to the 8015 target if the transfer is honoring an R2T. In this case, the 8016 Target Transfer Tag field is a replica of the Target Transfer Tag 8017 provided with the R2T. 8019 On incoming data, the Target Transfer Tag and LUN MUST be provided 8020 by the target if the A bit is set to 1; otherwise they are 8021 reserved. The Target Transfer Tag and LUN are copied by the 8022 initiator into the SNACK of type DataACK that it issues as a 8023 result of receiving a SCSI Data-in PDU with the A bit set to 1. 8025 The Target Transfer Tag values are not specified by this protocol 8026 except that the value 0xffffffff is reserved and means that the 8027 Target Transfer Tag is not supplied. If the Target Transfer Tag 8028 is provided, then the LUN field MUST hold a valid value and be 8029 consistent with whatever was specified with the command; 8030 otherwise, the LUN field is reserved. 8032 11.7.5. DataSN 8034 For input (read) or bidirectional Data-In PDUs, the DataSN is the 8035 input PDU number within the data transfer for the command 8036 identified by the Initiator Task Tag. 8038 R2T and Data-In PDUs, in the context of bidirectional commands, 8039 share the numbering sequence (see Section 4.2.2.4). 8041 For output (write) data PDUs, the DataSN is the Data-Out PDU 8042 number within the current output sequence. The current output 8043 sequence is either identified by the Initiator Task Tag (for 8044 unsolicited data) or is a data sequence generated for one R2T (for 8045 data solicited through R2T). 8047 11.7.6. Buffer Offset 8049 The Buffer Offset field contains the offset of this PDU payload 8050 data within the complete data transfer. The sum of the buffer 8051 offset and length should not exceed the expected transfer length 8052 for the command. 8054 The order of data PDUs within a sequence is determined by 8055 DataPDUInOrder. When set to Yes, it means that PDUs have to be in 8056 increasing Buffer Offset order and overlays are forbidden. 8058 The ordering between sequences is determined by 8059 DataSequenceInOrder. When set to Yes, it means that sequences have 8060 to be in increasing Buffer Offset order and overlays are 8061 forbidden. 8063 11.7.7. DataSegmentLength 8065 This is the data payload length of a SCSI Data-In or SCSI Data-Out 8066 PDU. The sending of 0 length data segments should be avoided, but 8067 initiators and targets MUST be able to properly receive 0 length 8068 data segments. 8070 The Data Segments of Data-in and Data-out PDUs SHOULD be filled to 8071 the integer number of 4 byte words (real payload) unless the F bit 8072 is set to 1. 8074 11.8. Ready To Transfer (R2T) 8076 Byte/ 0 | 1 | 2 | 3 | 8077 / | | | | 8078 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 8079 +---------------+---------------+---------------+--------------+ 8080 0|.|.| 0x31 |1| Reserved | 8081 +---------------+---------------+---------------+--------------+ 8082 4|TotalAHSLength | DataSegmentLength | 8083 +---------------+---------------+---------------+--------------+ 8084 8| LUN | 8085 + + 8086 12| | 8087 +---------------+---------------+---------------+--------------+ 8088 16| Initiator Task Tag | 8089 +---------------+---------------+---------------+--------------+ 8090 20| Target Transfer Tag | 8091 +---------------+---------------+---------------+--------------+ 8092 24| StatSN | 8093 +---------------+---------------+---------------+--------------+ 8094 28| ExpCmdSN | 8095 +---------------+---------------+---------------+--------------+ 8096 32| MaxCmdSN | 8097 +---------------+---------------+---------------+--------------+ 8098 36| R2TSN | 8099 +---------------+---------------+---------------+--------------+ 8100 40| Buffer Offset | 8101 +---------------+---------------+---------------+--------------+ 8102 44| Desired Data Transfer Length | 8103 +--------------------------------------------------------------+ 8104 48| Header-Digest (Optional) | 8105 +---------------+---------------+---------------+--------------+ 8107 When an initiator has submitted a SCSI Command with data that 8108 passes from the initiator to the target (WRITE), the target may 8109 specify which blocks of data it is ready to receive. The target 8110 may request that the data blocks be delivered in whichever order 8111 is convenient for the target at that particular instant. This 8112 information is passed from the target to the initiator in the 8113 Ready To Transfer (R2T) PDU. 8115 In order to allow write operations without an explicit initial 8116 R2T, the initiator and target MUST have negotiated the key 8117 InitialR2T to No during Login. 8119 An R2T MAY be answered with one or more SCSI Data-out PDUs with a 8120 matching Target Transfer Tag. If an R2T is answered with a single 8121 Data-out PDU, the Buffer Offset in the Data PDU MUST be the same 8122 as the one specified by the R2T, and the data length of the Data 8123 PDU MUST be the same as the Desired Data Transfer Length specified 8124 in the R2T. If the R2T is answered with a sequence of Data PDUs, 8125 the Buffer Offset and Length MUST be within the range of those 8126 specified by R2T, and the last PDU MUST have the F bit set to 1. 8127 If the last PDU (marked with the F bit) is received before the 8128 Desired Data Transfer Length is transferred, a target MAY choose 8129 to Reject that PDU with "Protocol error" reason code. 8130 DataPDUInOrder governs the Data-Out PDU ordering. If 8131 DataPDUInOrder is set to Yes, the Buffer Offsets and Lengths for 8132 consecutive PDUs MUST form a continuous non-overlapping range and 8133 the PDUs MUST be sent in increasing offset order. 8135 The target may send several R2T PDUs. It, therefore, can have a 8136 number of pending data transfers. The number of outstanding R2T 8137 PDUs are limited by the value of the negotiated key 8138 MaxOutstandingR2T. Within a task, outstanding R2Ts MUST be 8139 fulfilled by the initiator in the order in which they were 8140 received. 8142 R2T PDUs MAY also be used to recover Data Out PDUs. Such an R2T 8143 (Recovery-R2T) is generated by a target upon detecting the loss of 8144 one or more Data-Out PDUs due to: 8146 - Digest error 8148 - Sequence error 8150 - Sequence reception timeout 8152 A Recovery-R2T carries the next unused R2TSN, but requests part of 8153 or the entire data burst that an earlier R2T (with a lower R2TSN) 8154 had already requested. 8156 DataSequenceInOrder governs the buffer offset ordering in 8157 consecutive R2Ts. If DataSequenceInOrder is Yes, then consecutive 8158 R2Ts MUST refer to continuous non-overlapping ranges except for 8159 Recovery-R2Ts. 8161 11.8.1. TotalAHSLength and DataSegmentLength 8163 For this PDU TotalAHSLength and DataSegmentLength MUST be 0. 8165 11.8.2. R2TSN 8167 R2TSN is the R2T PDU input PDU number within the command 8168 identified by the Initiator Task Tag. 8170 For bidirectional commands R2T and Data-In PDUs share the input 8171 PDU numbering sequence (see Section 4.2.2.4). 8173 11.8.3. StatSN 8175 The StatSN field will contain the next StatSN. The StatSN for this 8176 connection is not advanced after this PDU is sent. 8178 11.8.4. Desired Data Transfer Length and Buffer Offset 8180 The target specifies how many bytes it wants the initiator to send 8181 because of this R2T PDU. The target may request the data from the 8182 initiator in several chunks, not necessarily in the original order 8183 of the data. The target, therefore, also specifies a Buffer Offset 8184 that indicates the point at which the data transfer should begin, 8185 relative to the beginning of the total data transfer. The Desired 8186 Data Transfer Length MUST NOT be 0 and MUST NOT exceed 8187 MaxBurstLength. 8189 11.8.5. Target Transfer Tag 8191 The target assigns its own tag to each R2T request that it sends 8192 to the initiator. This tag can be used by the target to easily 8193 identify the data it receives. The Target Transfer Tag and LUN are 8194 copied in the outgoing data PDUs and are only used by the target. 8195 There is no protocol rule about the Target Transfer Tag except 8196 that the value 0xffffffff is reserved and MUST NOT be sent by a 8197 target in an R2T. 8199 11.9. Asynchronous Message 8201 An Asynchronous Message may be sent from the target to the 8202 initiator without correspondence to a particular command. The 8203 target specifies the reason for the event and sense data. 8205 Byte/ 0 | 1 | 2 | 3 | 8206 / | | | | 8207 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 8208 +---------------+---------------+---------------+--------------+ 8209 0|.|.| 0x32 |1| Reserved | 8210 +---------------+---------------+---------------+--------------+ 8211 4|TotalAHSLength | DataSegmentLength | 8212 +---------------+---------------+---------------+--------------+ 8213 8| LUN or Reserved | 8214 + + 8215 12| | 8216 +---------------+---------------+---------------+--------------+ 8217 16| 0xffffffff | 8218 +---------------+---------------+---------------+--------------+ 8219 20| Reserved | 8220 +---------------+---------------+---------------+--------------+ 8221 24| StatSN | 8222 +---------------+---------------+---------------+--------------+ 8223 28| ExpCmdSN | 8224 +---------------+---------------+---------------+--------------+ 8225 32| MaxCmdSN | 8226 +---------------+---------------+---------------+--------------+ 8227 36| AsyncEvent | AsyncVCode | Parameter1 or Reserved | 8228 +---------------+---------------+---------------+--------------+ 8229 40| Parameter2 or Reserved | Parameter3 or Reserved | 8230 +---------------+---------------+---------------+--------------+ 8231 44| Reserved | 8232 +---------------+---------------+---------------+--------------+ 8233 48| Header-Digest (Optional) | 8234 +---------------+---------------+---------------+--------------+ 8235 / DataSegment - Sense Data and iSCSI Event Data / 8236 +/ / 8237 +---------------+---------------+---------------+--------------+ 8238 | Data-Digest (Optional) | 8239 +---------------+---------------+---------------+--------------+ 8241 Some Asynchronous Messages are strictly related to iSCSI while 8242 others are related to SCSI [SAM2]. 8244 StatSN counts this PDU as an acknowledgeable event (StatSN is 8245 advanced), which allows for initiator and target state 8246 synchronization. 8248 11.9.1. AsyncEvent 8250 The codes used for iSCSI Asynchronous Messages (events) are: 8252 0 (SCSI_ASYNC) - a SCSI Asynchronous Event is reported in the 8253 sense data. Sense Data that accompanies the report, in the 8254 data segment, identifies the condition. The sending of a 8255 SCSI Event (Asynchronous Event Reporting in SCSI 8256 terminology) is dependent on the target support for SCSI 8257 asynchronous event reporting (see [SAM2]) as indicated in 8258 the standard INQUIRY data (see [SPC3]). Its use may be 8259 enabled by parameters in the SCSI Control mode page (see 8260 [SPC3]). 8262 1 (REQUEST_LOGOUT) - target requests Logout. This Async 8263 Message MUST be sent on the same connection as the one 8264 requesting to be logged out. The initiator MUST honor this 8265 request by issuing a Logout as early as possible, but no 8266 later than Parameter3 seconds. Initiator MUST send a 8267 Logout with a reason code of "Close the connection" OR 8268 "Close the session" to close all the connections. Once this 8269 message is received, the initiator SHOULD NOT issue new 8270 iSCSI commands on the connection to be logged out. The 8271 target MAY reject any new I/O requests that it receives 8272 after this Message with the reason code "Waiting for 8273 Logout". If the initiator does not Logout in Parameter3 8274 seconds, the target should send an Async PDU with iSCSI 8275 event code "Dropped the connection" if possible, or simply 8276 terminate the transport connection. Parameter1 and 8277 Parameter2 are reserved. 8279 2 (CONNECTION_DROP) - target indicates it will drop the 8280 connection. 8282 The Parameter1 field indicates the CID of the connection 8283 going to be dropped. 8285 The Parameter2 field (Time2Wait) indicates, in seconds, the 8286 minimum time to wait before attempting to reconnect or 8287 reassign. 8289 The Parameter3 field (Time2Retain) indicates the maximum 8290 time allowed to reassign commands after the initial wait (in 8291 Parameter2). 8293 If the initiator does not attempt to reconnect and/or 8294 reassign the outstanding commands within the time specified 8295 by Parameter3, or if Parameter3 is 0, the target will 8296 terminate all outstanding commands on this connection. In 8297 this case, no other responses should be expected from the 8298 target for the outstanding commands on this connection. 8300 A value of 0 for Parameter2 indicates that reconnect can be 8301 attempted immediately. 8303 3 (SESSION_DROP) - target indicates it will drop all the 8304 connections of this session. 8306 Parameter1 field is reserved. 8308 The Parameter2 field (Time2Wait) indicates, in seconds, the 8309 minimum time to wait before attempting to reconnect. 8310 The Parameter3 field (Time2Retain) indicates the maximum 8311 time allowed to reassign commands after the initial wait (in 8312 Parameter2). 8314 If the initiator does not attempt to reconnect and/or 8315 reassign the outstanding commands within the time specified 8316 by Parameter3, or if Parameter3 is 0, the session is 8317 terminated. In this case, the target will terminate all 8318 outstanding commands in this session; no other responses 8319 should be expected from the target for the outstanding 8320 commands in this session. A value of 0 for Parameter2 8321 indicates that reconnect can be attempted immediately. 8323 4 (RENEGOTIATE) - target requests parameter negotiation on 8324 this connection. The initiator MUST honor this request by 8325 issuing a Text Request (that can be empty) on the same 8326 connection as early as possible, but no later than 8327 Parameter3 seconds, unless a Text Request is already pending 8328 on the connection, or by issuing a Logout Request. If the 8329 initiator does not issue a Text Request the target may 8330 reissue the Asynchronous Message requesting parameter 8331 negotiation. 8333 5 (FAST_ABORT) - all active tasks for LU with a matching LUN 8334 field in the Async Message PDU are being terminated. The 8335 receiving initiator iSCSI layer MUST respond to this Message 8336 by taking the following steps in order. 8338 - Stop Data-Out transfers on that connection for all active 8339 TTTs for the affected LUN quoted in the Async Message 8340 PDU. 8341 - Acknowledge the StatSN of the Async Message PDU via a NOP- 8342 Out PDU with ITT=0xffffffff (i.e., non-ping flavor), 8343 while copying the LUN field from the Async Message to 8344 NOP-Out. 8346 This value of AsyncEvent however MUST NOT be used on an 8347 iSCSI session unless the new TaskReporting text key defined 8348 in Section 13.23 was negotiated to FastAbort on the session. 8350 255 - vendor-specific iSCSI Event. The AsyncVCode details the 8351 vendor code, and data MAY accompany the report. 8353 All other event codes are reserved. 8355 11.9.2. AsyncVCode 8357 AsyncVCode is a vendor specific detail code that is only valid if 8358 the AsyncEvent field indicates a vendor specific event. Otherwise, 8359 it is reserved. 8361 11.9.3. LUN 8363 The LUN field MUST be valid if AsyncEvent is 0. Otherwise, this 8364 field is reserved. 8366 11.9.4. Sense Data and iSCSI Event Data 8368 For a SCSI event, this data accompanies the report in the data 8369 segment and identifies the condition. 8371 For an iSCSI event, additional vendor-unique data MAY accompany 8372 the Async event. Initiators MAY ignore the data when not 8373 understood while processing the rest of the PDU. 8375 If the DataSegmentLength is not 0, the format of the DataSegment 8376 is as follows: 8378 Byte/ 0 | 1 | 2 | 3 | 8379 / | | | | 8380 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 8381 +---------------+---------------+---------------+--------------+ 8382 0|SenseLength | Sense Data | 8383 +---------------+---------------+---------------+--------------+ 8384 x/ Sense Data / 8385 +---------------+---------------+---------------+--------------+ 8386 y/ iSCSI Event Data / 8387 / / 8388 +---------------+---------------+---------------+--------------+ 8389 z| 8391 11.9.4.1. SenseLength 8393 This is the length of Sense Data. When the Sense Data field is 8394 empty (e.g., the event is not a SCSI event) SenseLength is 0. 8396 11.10. Text Request 8398 The Text Request is provided to allow for the exchange of 8399 information and for future extensions. It permits the initiator to 8400 inform a target of its capabilities or to request some special 8401 operations. 8403 Byte/ 0 | 1 | 2 | 3 | 8404 / | | | | 8405 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 8406 +---------------+---------------+---------------+--------------+ 8407 0|.|I| 0x04 |F|C| Reserved | 8408 +---------------+---------------+---------------+--------------+ 8409 4|TotalAHSLength | DataSegmentLength | 8410 +---------------+---------------+---------------+--------------+ 8411 8| LUN or Reserved | 8412 + + 8413 12| | 8414 +---------------+---------------+---------------+--------------+ 8415 16| Initiator Task Tag | 8416 +---------------+---------------+---------------+--------------+ 8417 20| Target Transfer Tag or 0xffffffff | 8418 +---------------+---------------+---------------+--------------+ 8419 24| CmdSN | 8420 +---------------+---------------+---------------+--------------+ 8421 28| ExpStatSN | 8422 +---------------+---------------+---------------+--------------+ 8423 32/ Reserved / 8424 +/ / 8425 +---------------+---------------+---------------+--------------+ 8426 48| Header-Digest (Optional) | 8427 +---------------+---------------+---------------+--------------+ 8428 / DataSegment (Text) / 8429 +/ / 8430 +---------------+---------------+---------------+--------------+ 8431 | Data-Digest (Optional) | 8432 +---------------+---------------+---------------+--------------+ 8434 An initiator MUST NOT have more than one outstanding Text Request 8435 on a connection at any given time. 8437 On a connection failure, an initiator must either explicitly abort 8438 any active allegiant text negotiation task or must cause such a 8439 task to be implicitly terminated by the target. 8441 11.10.1. F (Final) Bit 8443 When set to 1, indicates that this is the last or only text 8444 request in a sequence of Text Requests; otherwise, it indicates 8445 that more Text Requests will follow. 8447 11.10.2. C (Continue) Bit 8449 When set to 1, indicates that the text (set of key=value pairs) in 8450 this Text Request is not complete (it will be continued on 8451 subsequent Text Requests); otherwise, it indicates that this Text 8452 Request ends a set of key=value pairs. A Text Request with the C 8453 bit set to 1 MUST have the F bit set to 0. 8455 11.10.3. Initiator Task Tag 8457 The initiator assigned identifier for this Text Request. If the 8458 command is sent as part of a sequence of text requests and 8459 responses, the Initiator Task Tag MUST be the same for all the 8460 requests within the sequence (similar to linked SCSI commands). 8461 The I bit for all requests in a sequence also MUST be the same. 8463 11.10.4. Target Transfer Tag 8465 When the Target Transfer Tag is set to the reserved value 8466 0xffffffff, it tells the target that this is a new request and the 8467 target resets any internal state associated with the Initiator 8468 Task Tag (resets the current negotiation state). 8470 The target sets the Target Transfer Tag in a text response to a 8471 value other than the reserved value 0xffffffff whenever it 8472 indicates that it has more data to send or more operations to 8473 perform that are associated with the specified Initiator Task Tag. 8474 It MUST do so whenever it sets the F bit to 0 in the response. By 8475 copying the Target Transfer Tag from the response to the next Text 8476 Request, the initiator tells the target to continue the operation 8477 for the specific Initiator Task Tag. The initiator MUST ignore the 8478 Target Transfer Tag in the Text Response when the F bit is set to 8479 1. 8481 This mechanism allows the initiator and target to transfer a large 8482 amount of textual data over a sequence of text-command/text- 8483 response exchanges, or to perform extended negotiation sequences. 8485 If the Target Transfer Tag is not 0xffffffff, the LUN field MUST 8486 be sent by the target in the Text Response. 8488 A target MAY reset its internal negotiation state if an exchange 8489 is stalled by the initiator for a long time or if it is running 8490 out of resources. 8492 Long text responses are handled as in the following example: 8494 I->T Text SendTargets=All (F=1,TTT=0xffffffff) 8496 T->I Text (F=0,TTT=0x12345678) 8498 I->T Text (F=1, TTT=0x12345678) 8500 T->I Text (F=0, TTT=0x12345678) 8502 I->T Text (F=1, TTT=0x12345678) 8504 ... 8506 T->I Text (F=1, TTT=0xffffffff) 8508 11.10.5. Text 8510 The data lengths of a text request MUST NOT exceed the iSCSI 8511 target MaxRecvDataSegmentLength (a per connection and per 8512 direction negotiated parameter). The text format is specified in 8513 Section 6.2. 8515 Section 12 and Section 13 list some basic Text key=value pairs, 8516 some of which can be used in Login Request/Response and some in 8517 Text Request/Response. 8519 A key=value pair can span Text request or response boundaries. A 8520 key=value pair can start in one PDU and continue on the next. In 8522 other words the end of a PDU does not necessarily signal the end 8523 of a key=value pair. 8525 The target responds by sending its response back to the initiator. 8526 The response text format is similar to the request text format. 8527 The text response MAY refer to key=value pairs presented in an 8528 earlier text request and the text in the request may refer to 8529 earlier responses. 8531 Section 6.2 details the rules for the Text Requests and Responses. 8533 Text operations are usually meant for parameter 8534 setting/negotiations, but can also be used to perform some long 8535 lasting operations. 8537 Text operations that take a long time should be placed in their 8538 own Text request. 8540 11.11. Text Response 8542 The Text Response PDU contains the target's responses to the 8543 initiator's Text request. The format of the Text field matches 8544 that of the Text request. 8546 Byte/ 0 | 1 | 2 | 3 | 8547 / | | | | 8548 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 8549 +---------------+---------------+---------------+--------------+ 8550 0|.|.| 0x24 |F|C| Reserved | 8551 +---------------+---------------+---------------+--------------+ 8552 4|TotalAHSLength | DataSegmentLength | 8553 +---------------+---------------+---------------+--------------+ 8554 8| LUN or Reserved | 8555 + + 8556 12| | 8557 +---------------+---------------+---------------+--------------+ 8558 16| Initiator Task Tag | 8559 +---------------+---------------+---------------+--------------+ 8560 20| Target Transfer Tag or 0xffffffff | 8561 +---------------+---------------+---------------+--------------+ 8562 24| StatSN | 8563 +---------------+---------------+---------------+--------------+ 8564 28| ExpCmdSN | 8565 +---------------+---------------+---------------+--------------+ 8566 32| MaxCmdSN | 8567 +---------------+---------------+---------------+--------------+ 8568 36/ Reserved / 8569 +/ / 8570 +---------------+---------------+---------------+--------------+ 8571 48| Header-Digest (Optional) | 8572 +---------------+---------------+---------------+--------------+ 8573 / DataSegment (Text) / 8574 +/ / 8575 +---------------+---------------+---------------+--------------+ 8576 | Data-Digest (Optional) | 8577 +---------------+---------------+---------------+--------------+ 8579 11.11.1. F (Final) Bit 8581 When set to 1, in response to a Text Request with the Final bit 8582 set to 1, the F bit indicates that the target has finished the 8583 whole operation. Otherwise, if set to 0 in response to a Text 8584 Request with the Final Bit set to 1, it indicates that the target 8585 has more work to do (invites a follow-on text request). A Text 8586 Response with the F bit set to 1 in response to a Text Request 8587 with the F bit set to 0 is a protocol error. 8589 A Text Response with the F bit set to 1 MUST NOT contain key=value 8590 pairs that may require additional answers from the initiator. 8592 A Text Response with the F bit set to 1 MUST have a Target 8593 Transfer Tag field set to the reserved value of 0xffffffff. 8595 A Text Response with the F bit set to 0 MUST have a Target 8596 Transfer Tag field set to a value other than the reserved 8597 0xffffffff. 8599 11.11.2. C (Continue) Bit 8601 When set to 1, indicates that the text (set of key=value pairs) in 8602 this Text Response is not complete (it will be continued on 8603 subsequent Text Responses); otherwise, it indicates that this Text 8604 Response ends a set of key=value pairs. A Text Response with the C 8605 bit set to 1 MUST have the F bit set to 0. 8607 11.11.3. Initiator Task Tag 8609 The Initiator Task Tag matches the tag used in the initial Text 8610 Request. 8612 11.11.4. Target Transfer Tag 8614 When a target has more work to do (e.g., cannot transfer all the 8615 remaining text data in a single Text Response or has to continue 8616 the negotiation) and has enough resources to proceed, it MUST set 8617 the Target Transfer Tag to a value other than the reserved value 8618 of 0xffffffff. Otherwise, the Target Transfer Tag MUST be set to 8619 0xffffffff. 8621 When the Target Transfer Tag is not 0xffffffff, the LUN field may 8622 be significant. 8624 The initiator MUST copy the Target Transfer Tag and LUN in its 8625 next request to indicate that it wants the rest of the data. 8627 When the target receives a Text Request with the Target Transfer 8628 Tag set to the reserved value of 0xffffffff, it resets its 8629 internal information (resets state) associated with the given 8630 Initiator Task Tag (restarts the negotiation). 8632 When a target cannot finish the operation in a single Text 8633 Response, and does not have enough resources to continue, it 8634 rejects the Text Request with the appropriate Reject code. 8636 A target may reset its internal state associated with an Initiator 8637 Task Tag (the current negotiation state), state expressed through 8638 the Target Transfer Tag if the initiator fails to continue the 8639 exchange for some time. The target may reject subsequent Text 8640 Requests with the Target Transfer Tag set to the "stale" value. 8642 11.11.5. StatSN 8644 The target StatSN variable is advanced by each Text Response sent. 8646 11.11.6. Text Response Data 8648 The data lengths of a text response MUST NOT exceed the iSCSI 8649 initiator MaxRecvDataSegmentLength (a per connection and per 8650 direction negotiated parameter). 8652 The text in the Text Response Data is governed by the same rules 8653 as the text in the Text Request Data (see Section 11.11.2). 8655 Although the initiator is the requesting party and controls the 8656 request-response initiation and termination, the target can offer 8657 key=value pairs of its own as part of a sequence and not only in 8658 response to the initiator. 8660 11.12. Login Request 8662 After establishing a TCP connection between an initiator and a 8663 target, the initiator MUST start a Login Phase to gain further 8664 access to the target's resources. 8666 The Login Phase (see Section 6.3) consists of a sequence of Login 8667 requests and responses that carry the same Initiator Task Tag. 8669 Login requests are always considered as immediate. 8671 Byte/ 0 | 1 | 2 | 3 | 8672 / | | | | 8673 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 8674 +---------------+---------------+---------------+--------------+ 8675 0|.|1| 0x03 |T|C|.|.|CSG|NSG| Version-max | Version-min | 8676 +---------------+---------------+---------------+--------------+ 8677 4|TotalAHSLength | DataSegmentLength | 8678 +---------------+---------------+---------------+--------------+ 8679 8| ISID | 8680 + +---------------+--------------+ 8681 12| | TSIH | 8682 +---------------+---------------+---------------+--------------+ 8683 16| Initiator Task Tag | 8684 +---------------+---------------+---------------+--------------+ 8685 20| CID | Reserved | 8686 +---------------+---------------+---------------+--------------+ 8687 24| CmdSN | 8688 +---------------+---------------+---------------+--------------+ 8689 28| ExpStatSN or Reserved | 8690 +---------------+---------------+---------------+--------------+ 8691 32| Reserved | 8692 +---------------+---------------+---------------+--------------+ 8693 36| Reserved | 8694 +---------------+---------------+---------------+--------------+ 8695 40/ Reserved / 8696 +/ / 8697 +---------------+---------------+---------------+--------------+ 8698 48/ DataSegment - Login Parameters in Text request Format / 8699 +/ / 8700 +---------------+---------------+---------------+--------------+ 8702 11.12.1. T (Transit) Bit 8704 If set to 1, indicates that the initiator is ready to transit to 8705 the next stage. 8707 If the T bit is set to 1 and NSG is FullFeaturePhase, then this 8708 also indicates that the initiator is ready for the Final Login 8709 Response (see Section 6.3). 8711 11.12.2. C (Continue) Bit 8713 When set to 1, this bit indicates that the text (set of key=value 8714 pairs) in this Login Request is not complete (it will be continued 8715 on subsequent Login Requests); otherwise, it indicates that this 8716 Login Request ends a set of key=value pairs. A Login Request with 8717 the C bit set to 1 MUST have the T bit set to 0. 8719 11.12.3. CSG and NSG 8721 Through these fields, Current Stage (CSG) and Next Stage (NSG), 8722 the Login negotiation requests and responses are associated with a 8723 specific stage in the session (SecurityNegotiation, 8724 LoginOperationalNegotiation, FullFeaturePhase) and may indicate 8725 the next stage to which they want to move (see Section 6.3). The 8726 next stage value is only valid when the T bit is 1; otherwise, it 8727 is reserved. 8729 The stage codes are: 8731 - 0 - SecurityNegotiation 8733 - 1 - LoginOperationalNegotiation 8735 - 3 - FullFeaturePhase 8737 All other codes are reserved. 8739 11.12.4. Version 8741 The version number of the current draft is 0x00. As such, all 8742 devices MUST carry version 0x00 for both Version-min and Version- 8743 max. 8745 11.12.4.1. Version-max 8747 Maximum Version number supported. 8749 All Login requests within the Login Phase MUST carry the same 8750 Version-max. 8752 The target MUST use the value presented with the first login 8753 request. 8755 11.12.4.2. Version-min 8757 All Login requests within the Login Phase MUST carry the same 8758 Version-min. The target MUST use the value presented with the 8759 first login request. 8761 11.12.5. ISID 8763 This is an initiator-defined component of the session identifier 8764 and is structured as follows (see Section 10.1.1 for details): 8766 Byte/ 0 | 1 | 2 | 3 | 8767 / | | | | 8768 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 8769 +---------------+---------------+---------------+--------------+ 8770 8| T | A | B | C | 8771 +---------------+---------------+---------------+--------------+ 8772 12| D | 8773 +---------------+---------------+ 8775 The T field identifies the format and usage of A, B, C, and D as 8776 indicated below: 8778 T 8780 00b OUI-Format 8782 A&B are a 22 bit OUI 8784 (the I/G & U/L bits are omitted) 8786 C&D 24 bit qualifier 8788 01b EN - Format (IANA Enterprise Number) 8790 A - Reserved 8792 B&C EN (IANA Enterprise Number) 8794 D - Qualifier 8796 10b "Random" 8798 A - Reserved 8800 B&C Random 8802 D - Qualifier 8804 11b A,B,C&D Reserved 8806 For the T field values 00b and 01b, a combination of A and B (for 8807 00b) or B and C (for 01b) identifies the vendor or organization 8808 whose component (software or hardware) generates this ISID. A 8809 vendor or organization with one or more OUIs, or one or more 8810 Enterprise Numbers, MUST use at least one of these numbers and 8811 select the appropriate value for the T field when its components 8812 generate ISIDs. An OUI or EN MUST be set in the corresponding 8813 fields in network byte order (byte big-endian). 8815 If the T field is 10b, B and C are set to a random 24-bit unsigned 8816 integer value in network byte order (byte big-endian). See 8817 [RFC3721] for how this affects the principle of "conservative 8818 reuse". 8820 The Qualifier field is a 16 or 24-bit unsigned integer value that 8821 provides a range of possible values for the ISID within the 8822 selected namespace. It may be set to any value within the 8823 constraints specified in the iSCSI protocol (see Section 4.4.3 and 8824 Section 10.1.1). 8826 The T field value of 11b is reserved. 8828 If the ISID is derived from something assigned to a hardware 8829 adapter or interface by a vendor, as a preset default value, it 8830 MUST be configurable to a value assigned according to the SCSI 8831 port behavior desired by the system in which it is installed (see 8832 Section 10.1.1 and Section 10.1.2). The resultant ISID MUST also 8833 be persistent over power cycles, reboot, card swap, etc. 8835 11.12.6. TSIH 8837 TSIH must be set in the first Login Request. The reserved value 0 8838 MUST be used on the first connection for a new session. Otherwise, 8839 the TSIH sent by the target at the conclusion of the successful 8840 login of the first connection for this session MUST be used. The 8841 TSIH identifies to the target the associated existing session for 8842 this new connection. 8844 All Login requests within a Login Phase MUST carry the same TSIH. 8846 The target MUST check the value presented with the first login 8847 request and act as specified in Section 5.3.1. 8849 11.12.7. Connection ID - CID 8851 A unique ID for this connection within the session. 8853 All Login requests within the Login Phase MUST carry the same CID. 8855 The target MUST use the value presented with the first login 8856 request. 8858 A Login request with a non-zero TSIH and a CID equal to that of an 8859 existing connection implies a logout of the connection followed by 8860 a Login (see Section 6.3.4). For the details of the implicit 8861 Logout Request, see Section 11.14. 8863 11.12.8. CmdSN 8865 CmdSN is either the initial command sequence number of a session 8866 (for the first Login request of a session - the "leading" login), 8867 or the command sequence number in the command stream if the login 8868 is for a new connection in an existing session. 8870 Examples: 8872 - Login on a leading connection - if the leading login carries 8873 the CmdSN 123, all other login requests in the same login 8874 phase carry the CmdSN 123 and the first non-immediate 8875 command in FullFeaturePhase also carries the CmdSN 123. 8877 - Login on other than a leading connection - if the current 8878 CmdSN at the time the first login on the connection is 8879 issued is 500, then that PDU carries CmdSN=500. Subsequent 8880 login requests that are needed to complete this login phase 8881 may carry a CmdSN higher than 500 if non-immediate requests 8882 that were issued on other connections in the same session 8883 advance CmdSN. 8885 If the login request is a leading login request, the target MUST 8886 use the value presented in CmdSN as the target value for ExpCmdSN. 8888 11.12.9. ExpStatSN 8890 For the first Login Request on a connection this is ExpStatSN for 8891 the old connection and this field is only valid if the Login 8892 request restarts a connection (see Section 6.3.4). 8894 For subsequent Login Requests it is used to acknowledge the Login 8895 Responses with their increasing StatSN values. 8897 11.12.10. Login Parameters 8899 The initiator MUST provide some basic parameters in order to 8900 enable the target to determine if the initiator may use the 8901 target's resources and the initial text parameters for the 8902 security exchange. 8904 All the rules specified in Section 11.10.5 for text requests also 8905 hold for login requests. Keys and their explanations are listed 8906 in Section 12 (security negotiation keys) and Section 13 8907 (operational parameter negotiation keys). All keys in Section 13, 8908 except for the X extension formats, MUST be supported by iSCSI 8909 initiators and targets. Keys in Section 12 only need to be 8910 supported when the function to which they refer is mandatory to 8911 implement. 8913 11.13. Login Response 8915 The Login Response indicates the progress and/or end of the Login 8916 Phase. 8918 Byte/ 0 | 1 | 2 | 3 | 8919 / | | | | 8920 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 8921 +---------------+---------------+---------------+--------------+ 8922 0|.|.| 0x23 |T|C|.|.|CSG|NSG| Version-max |Version-active| 8923 +---------------+---------------+---------------+--------------+ 8924 4|TotalAHSLength | DataSegmentLength | 8925 +---------------+---------------+---------------+--------------+ 8926 8| ISID | 8927 + +---------------+--------------+ 8928 12| | TSIH | 8929 +---------------+---------------+---------------+--------------+ 8930 16| Initiator Task Tag | 8931 +---------------+---------------+---------------+--------------+ 8932 20| Reserved | 8933 +---------------+---------------+---------------+--------------+ 8934 24| StatSN | 8935 +---------------+---------------+---------------+--------------+ 8936 28| ExpCmdSN | 8937 +---------------+---------------+---------------+--------------+ 8938 32| MaxCmdSN | 8939 +---------------+---------------+---------------+--------------+ 8940 36| Status-Class | Status-Detail | Reserved | 8941 +---------------+---------------+---------------+--------------+ 8942 40/ Reserved / 8943 +/ / 8944 +---------------+---------------+---------------+--------------+ 8945 48/ DataSegment - Login Parameters in Text request Format / 8946 +/ / 8947 +---------------+---------------+---------------+--------------+ 8949 11.13.1. Version-max 8951 This is the highest version number supported by the target. 8953 All Login responses within the Login Phase MUST carry the same 8954 Version-max. 8956 The initiator MUST use the value presented as a response to the 8957 first login request. 8959 11.13.2. Version-active 8961 Indicates the highest version supported by the target and 8962 initiator. If the target does not support a version within the 8963 range specified by the initiator, the target rejects the login and 8964 this field indicates the lowest version supported by the target. 8966 All Login responses within the Login Phase MUST carry the same 8967 Version-active. 8969 The initiator MUST use the value presented as a response to the 8970 first login request. 8972 11.13.3. TSIH 8974 The TSIH is the target assigned session identifying handle. Its 8975 internal format and content are not defined by this protocol 8976 except for the value 0 that is reserved. With the exception of the 8977 Login Final-Response in a new session, this field should be set to 8978 the TSIH provided by the initiator in the Login Request. For a 8979 new session, the target MUST generate a non-zero TSIH and ONLY 8980 return it in the Login Final-Response (see Section 6.3). 8982 11.13.4. StatSN 8984 For the first Login Response (the response to the first Login 8985 Request), this is the starting status Sequence Number for the 8986 connection. The next response of any kind, including the next 8987 login response, if any, in the same Login Phase, will carry this 8988 number + 1. This field is only valid if the Status-Class is 0. 8990 11.13.5. Status-Class and Status-Detail 8992 The Status returned in a Login Response indicates the execution 8993 status of the Login Phase. The status includes: 8995 Status-Class 8997 Status-Detail 8999 0 Status-Class indicates success. 9001 A non-zero Status-Class indicates an exception. In this case, 9002 Status-Class is sufficient for a simple initiator to use when 9003 handling exceptions, without having to look at the Status-Detail. 9004 The Status-Detail allows finer-grained exception handling for more 9005 sophisticated initiators and for better information for logging. 9007 The status classes are as follows: 9009 0 - Success - indicates that the iSCSI target successfully 9010 received, understood, and accepted the request. The 9011 numbering fields (StatSN, ExpCmdSN, MaxCmdSN) are only valid 9012 if Status-Class is 0. 9014 1 - Redirection - indicates that the initiator must take 9015 further action to complete the request. This is usually due 9016 to the target moving to a different address. All of the 9017 redirection status class responses MUST return one or more 9018 text key parameters of the type "TargetAddress", which 9019 indicates the target's new address. A redirection response 9020 MAY be issued by a target prior or after completing a 9021 security negotiation if a security negotiation is required. 9022 A redirection SHOULD be accepted by an initiator even 9023 without having the target complete a security negotiation if 9024 any security negotiation is required, and MUST be accepted 9025 by the initiator after the completion of the security 9026 negotiation if any security negotiation is required. 9028 2 - Initiator Error (not a format error) - indicates that the 9029 initiator most likely caused the error. This MAY be due to a 9030 request for a resource for which the initiator does not have 9031 permission. The request should not be tried again. 9033 3 - Target Error - indicates that the target sees no errors in 9034 the initiator's login request, but is currently incapable of 9035 fulfilling the request. The initiator may re-try the same 9036 login request later. 9038 The table below shows all of the currently allocated status codes. 9039 The codes are in hexadecimal; the first byte is the status class 9040 and the second byte is the status detail. 9042 ----------------------------------------------------------------- 9043 Status | Code | Description 9044 |(hex) | 9045 ----------------------------------------------------------------- 9046 Success | 0000 | Login is proceeding OK (*1). 9047 ----------------------------------------------------------------- 9048 Target moved | 0101 | The requested iSCSI Target Name (ITN) 9049 temporarily | | has temporarily moved 9050 | | to the address provided. 9051 ----------------------------------------------------------------- 9052 Target moved | 0102 | The requested ITN has permanently moved 9053 permanently | | to the address provided. 9054 ----------------------------------------------------------------- 9055 Initiator | 0200 | Miscellaneous iSCSI initiator 9056 error | | errors. 9057 ---------------------------------------------------------------- 9058 Authentication| 0201 | The initiator could not be 9059 failure | | successfully authenticated or target 9060 | | authentication is not supported. 9061 ----------------------------------------------------------------- 9062 Authorization | 0202 | The initiator is not allowed access 9063 failure | | to the given target. 9064 ----------------------------------------------------------------- 9065 Not found | 0203 | The requested ITN does not 9066 | | exist at this address. 9067 ----------------------------------------------------------------- 9068 Target removed| 0204 | The requested ITN has been removed and 9069 | | no forwarding address is provided. 9070 ----------------------------------------------------------------- 9071 Unsupported | 0205 | The requested iSCSI version range is 9072 version | | not supported by the target. 9073 ----------------------------------------------------------------- 9074 Too many | 0206 | Too many connections on this SSID. 9075 connections | | 9076 ----------------------------------------------------------------- 9077 Missing | 0207 | Missing parameters (e.g., iSCSI 9078 parameter | | Initiator and/or Target Name). 9079 ----------------------------------------------------------------- 9080 Can't include | 0208 | Target does not support session 9081 in session | | spanning to this connection (address). 9082 ----------------------------------------------------------------- 9083 Session type | 0209 | Target does not support this type of 9084 not supported | | of session or not from this Initiator. 9085 ----------------------------------------------------------------- 9086 Session does | 020a | Attempt to add a connection 9087 not exist | | to a non-existent session. 9088 ----------------------------------------------------------------- 9089 Invalid during| 020b | Invalid Request type during Login. 9090 login | | 9091 ----------------------------------------------------------------- 9092 Target error | 0300 | Target hardware or software error. 9093 ----------------------------------------------------------------- 9094 Service | 0301 | The iSCSI service or target is not 9095 unavailable | | currently operational. 9096 ----------------------------------------------------------------- 9097 Out of | 0302 | The target has insufficient session, 9098 resources | | connection, or other resources. 9099 ----------------------------------------------------------------- 9101 (*1)If the response T bit is 1 in both the request and the 9102 matching response, and the NSG is FullFeaturePhase in both the 9103 request and the matching response, the Login Phase is finished and 9104 the initiator may proceed to issue SCSI commands. 9106 If the Status Class is not 0, the initiator and target MUST close 9107 the TCP connection. 9109 If the target wishes to reject the login request for more than one 9110 reason, it should return the primary reason for the rejection. 9112 11.13.6. T (Transit) bit 9114 The T bit is set to 1 as an indicator of the end of the stage. If 9115 the T bit is set to 1 and NSG is FullFeaturePhase, then this is 9116 also the Final Login Response (see Section 6.3). A T bit of 0 9117 indicates a "partial" response, which means "more negotiation 9118 needed". 9120 A login response with a T bit set to 1 MUST NOT contain key=value 9121 pairs that may require additional answers from the initiator 9122 within the same stage. 9124 If the status class is 0, the T bit MUST NOT be set to 1 if the T 9125 bit in the request was set to 0. 9127 11.13.7. C (Continue) Bit 9129 When set to 1, indicates that the text (set of key=value pairs) in 9130 this Login Response is not complete (it will be continued on 9131 subsequent Login Responses); otherwise, it indicates that this 9132 Login Response ends a set of key=value pairs. A Login Response 9133 with the C bit set to 1 MUST have the T bit set to 0. 9135 11.13.8. Login Parameters 9137 The target MUST provide some basic parameters in order to enable 9138 the initiator to determine if it is connected to the correct port 9139 and the initial text parameters for the security exchange. 9141 All the rules specified in Section 11.11.6 for text responses also 9142 hold for login responses. Keys and their explanations are listed 9143 in Section 12(security negotiation keys) and Section 13 9144 (operational parameter negotiation keys). All keys in Section 13, 9145 except for the X extension formats, MUST be supported by iSCSI 9146 initiators and targets. Keys in Section 12, only need to be 9147 supported when the function to which they refer is mandatory to 9148 implement. 9150 11.14. Logout Request 9152 The Logout request is used to perform a controlled closing of a 9153 connection. 9155 An initiator MAY use a logout request to remove a connection from 9156 a session or to close an entire session. 9158 After sending the Logout request PDU, an initiator MUST NOT send 9159 any new iSCSI requests on the closing connection. If the Logout 9160 request is intended to close the session, new iSCSI requests MUST 9161 NOT be sent on any of the connections participating in the 9162 session. 9164 When receiving a Logout request with the reason code of "close the 9165 connection" or "close the session", the target MUST terminate all 9166 pending commands, whether acknowledged via ExpCmdSN or not, on 9167 that connection or session respectively. 9169 When receiving a Logout request with the reason code "remove 9170 connection for recovery", the target MUST discard all requests not 9171 yet acknowledged via ExpCmdSN that were issued on the specified 9172 connection, and suspend all data/status/R2T transfers on behalf of 9173 pending commands on the specified connection. 9175 The target then issues the Logout response and half-closes the TCP 9176 connection (sends FIN). After receiving the Logout response and 9177 attempting to receive the FIN (if still possible), the initiator 9178 MUST completely close the logging-out connection. For the 9179 terminated commands, no additional responses should be expected. 9181 A Logout for a CID may be performed on a different transport 9182 connection when the TCP connection for the CID has already been 9183 terminated. In such a case, only a logical "closing" of the iSCSI 9184 connection for the CID is implied with a Logout. 9186 All commands that were not terminated or not completed (with 9187 status) and acknowledged when the connection is closed completely 9188 can be reassigned to a new connection if the target supports 9189 connection recovery. 9191 If an initiator intends to start recovery for a failing 9192 connection, it MUST use the Logout request to "clean-up" the 9193 target end of a failing connection and enable recovery to start, 9194 or the Login request with a non-zero TSIH and the same CID on a 9195 new connection for the same effect. In sessions with a single 9196 connection, the connection can be closed and then a new connection 9197 reopened. A connection reinstatement login can be used for 9198 recovery (see Section 6.3.4). 9200 A successful completion of a logout request with the reason code 9201 of "close the connection" or "remove the connection for recovery" 9202 results at the target in the discarding of unacknowledged commands 9203 received on the connection being logged out. These are commands 9204 that have arrived on the connection being logged out, but have not 9205 been delivered to SCSI because one or more commands with a smaller 9206 CmdSN has not been received by iSCSI. See Section 4.2.2.1. The 9207 resulting holes in the command sequence numbers will have to be 9208 handled by appropriate recovery (see Section 7) unless the session 9209 is also closed. 9211 The entire logout discussion in this Section is also applicable 9212 for an implicit Logout realized by way of a connection 9213 reinstatement or session reinstatement. When a Login Request 9214 performs an implicit Logout, the implicit Logout is performed as 9215 if having the reason codes specified below: 9217 Reason code Type of implicit Logout 9219 ------------------------------------------- 9221 0 session reinstatement 9223 1 connection reinstatement when the operational 9224 ErrorRecoveryLevel < 2 9226 2 connection reinstatement when the operational 9227 ErrorRecoveryLevel = 2 9229 Byte/ 0 | 1 | 2 | 3 | 9230 / | | | | 9231 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 9232 +---------------+---------------+---------------+--------------+ 9233 0|.|I| 0x06 |1| Reason Code | Reserved | 9234 +---------------+---------------+---------------+--------------+ 9235 4|TotalAHSLength | DataSegmentLength | 9236 +--------------------------------------------------------------+ 9237 8/ Reserved / 9238 +/ / 9239 +---------------+---------------+---------------+--------------+ 9240 16| Initiator Task Tag | 9241 +---------------+---------------+---------------+--------------+ 9242 20| CID or Reserved | Reserved | 9243 +---------------+---------------+---------------+--------------+ 9244 24| CmdSN | 9245 +---------------+---------------+---------------+--------------+ 9246 28| ExpStatSN | 9247 +---------------+---------------+---------------+--------------+ 9248 32/ Reserved / 9249 +/ / 9250 +---------------+---------------+---------------+--------------+ 9251 48| Header-Digest (Optional) | 9252 +---------------+---------------+---------------+--------------+ 9254 11.14.1. Reason Code 9256 Reason Code indicates the reason for Logout as follows: 9258 0 - close the session. All commands associated with the 9259 session (if any) are terminated. 9261 1 - close the connection. All commands associated with 9262 connection (if any) are terminated. 9264 2 - remove the connection for recovery. Connection is closed 9265 and all commands associated with it, if any, are to be 9266 prepared for a new allegiance. 9268 All other values are reserved. 9270 11.14.2. TotalAHSLength and DataSegmentLength 9272 For this PDU TotalAHSLength and DataSegmentLength MUST be 0. 9274 11.14.3. CID 9276 This is the connection ID of the connection to be closed 9277 (including closing the TCP stream). This field is only valid if 9278 the reason code is not "close the session". 9280 11.14.4. ExpStatSN 9282 This is the last ExpStatSN value for the connection to be closed. 9284 11.14.5. Implicit termination of tasks 9286 A target implicitly terminates the active tasks due to the iSCSI 9287 protocol in the following cases: 9289 a) When a connection is implicitly or explicitly logged out 9290 with the reason code of "Close the connection" and there 9291 are active tasks allegiant to that connection. 9293 b) When a connection fails and eventually the connection state 9294 times out (state transition M1 in Section 8.2.2) and there 9295 are active tasks allegiant to that connection. 9297 c) When a successful recovery Logout is performed while there 9298 are active tasks allegiant to that connection, and those 9299 tasks eventually time out after the Time2Wait and 9300 Time2Retain periods without allegiance reassignment. 9302 d) When a connection is implicitly or explicitly logged out 9303 with the reason code of "Close the session" and there are 9304 active tasks in that session. 9306 If the tasks terminated in any of the above cases are SCSI tasks, 9307 they must be internally terminated as if with CHECK CONDITION 9308 status. This status is only meaningful for appropriately handling 9310 the internal SCSI state and SCSI side effects with respect to 9311 ordering because this status is never communicated back as a 9312 terminating status to the initiator. However additional actions 9313 may have to be taken at SCSI level depending on the SCSI context 9314 as defined by the SCSI standards (e.g., queued commands and ACA, 9315 UA for the next command on the I_T nexus in cases a), b), and c), 9316 after the tasks are terminated, the target MUST report a Unit 9317 Attention condition on the next command processed on any 9318 connection for each affected I_T_L nexus with the status of CHECK 9319 CONDITION, and the ASC/ASCQ value of 47h/7Fh - "SOME COMMANDS 9320 CLEARED BY ISCSI PROTOCOL EVENT" - etc. - see [SPC3]). 9322 11.15. Logout Response 9324 The logout response is used by the target to indicate if the 9325 cleanup operation for the connection(s) has completed. 9327 After Logout, the TCP connection referred by the CID MUST be 9328 closed at both ends (or all connections must be closed if the 9329 logout reason was session close). 9331 Byte/ 0 | 1 | 2 | 3 | 9332 / | | | | 9333 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 9334 +---------------+---------------+---------------+--------------+ 9335 0|.|.| 0x26 |1| Reserved | Response | Reserved | 9336 +---------------+---------------+---------------+--------------+ 9337 4|TotalAHSLength | DataSegmentLength | 9338 +--------------------------------------------------------------+ 9339 8/ Reserved / 9340 +/ / 9341 +---------------+---------------+---------------+--------------+ 9342 16| Initiator Task Tag | 9343 +---------------+---------------+---------------+--------------+ 9344 20| Reserved | 9345 +---------------+---------------+---------------+--------------+ 9346 24| StatSN | 9347 +---------------+---------------+---------------+--------------+ 9348 28| ExpCmdSN | 9349 +---------------+---------------+---------------+--------------+ 9350 32| MaxCmdSN | 9351 +---------------+---------------+---------------+--------------+ 9352 36| Reserved | 9353 +--------------------------------------------------------------+ 9354 40| Time2Wait | Time2Retain | 9355 +---------------+---------------+---------------+--------------+ 9356 44| Reserved | 9357 +---------------+---------------+---------------+--------------+ 9358 48| Header-Digest (Optional) | 9359 +---------------+---------------+---------------+--------------+ 9361 11.15.1. Response 9363 Logout response: 9365 0 - connection or session closed successfully. 9367 1 - CID not found. 9369 2 - connection recovery is not supported. If Logout reason 9370 code was recovery and target does not support it as 9371 indicated by the ErrorRecoveryLevel. 9372 3 - cleanup failed for various reasons. 9374 11.15.2. TotalAHSLength and DataSegmentLength 9376 For this PDU TotalAHSLength and DataSegmentLength MUST be 0. 9378 11.15.3. Time2Wait 9380 If the Logout response code is 0 and if the operational 9381 ErrorRecoveryLevel is 2, this is the minimum amount of time, in 9382 seconds, to wait before attempting task reassignment. If the 9383 Logout response code is 0 and if the operational 9384 ErrorRecoveryLevel is less than 2, this field is to be ignored. 9386 This field is invalid if the Logout response code is 1. 9388 If the Logout response code is 2 or 3, this field specifies the 9389 minimum time to wait before attempting a new implicit or explicit 9390 logout. 9392 If Time2Wait is 0, the reassignment or a new Logout may be 9393 attempted immediately. 9395 11.15.4. Time2Retain 9397 If the Logout response code is 0 and if the operational 9398 ErrorRecoveryLevel is 2, this is the maximum amount of time, in 9399 seconds, after the initial wait (Time2Wait), the target waits for 9400 the allegiance reassignment for any active task after which the 9401 task state is discarded. If the Logout response code is 0 and if 9402 the operational ErrorRecoveryLevel is less than 2, this field is 9403 to be ignored. 9405 This field is invalid if the Logout response code is 1. 9407 If the Logout response code is 2 or 3, this field specifies the 9408 maximum amount of time, in seconds, after the initial wait 9409 (Time2Wait), the target waits for a new implicit or explicit 9410 logout. 9412 If it is the last connection of a session, the whole session state 9413 is discarded after Time2Retain. 9415 If Time2Retain is 0, the target has already discarded the 9416 connection (and possibly the session) state along with the task 9417 states. No reassignment or Logout is required in this case. 9419 11.16. SNACK Request 9421 Byte/ 0 | 1 | 2 | 3 | 9422 / | | | | 9423 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 9424 +---------------+---------------+---------------+--------------+ 9425 0|.|.| 0x10 |1|.|.|.| Type | Reserved | 9426 +---------------+---------------+---------------+--------------+ 9427 4|TotalAHSLength | DataSegmentLength | 9428 +---------------+---------------+---------------+--------------+ 9429 8| LUN or Reserved | 9430 + + 9431 12| | 9432 +---------------+---------------+---------------+--------------+ 9433 16| Initiator Task Tag or 0xffffffff | 9434 +---------------+---------------+---------------+--------------+ 9435 20| Target Transfer Tag or SNACK Tag or 0xffffffff | 9436 +---------------+---------------+---------------+--------------+ 9437 24| Reserved | 9438 +---------------+---------------+---------------+--------------+ 9439 28| ExpStatSN | 9440 +---------------+---------------+---------------+--------------+ 9441 32/ Reserved / 9442 +/ / 9443 +---------------+---------------+---------------+--------------+ 9444 40| BegRun | 9445 +--------------------------------------------------------------+ 9446 44| RunLength | 9447 +--------------------------------------------------------------+ 9448 48| Header-Digest (Optional) | 9449 +---------------+---------------+---------------+--------------+ 9451 If the implementation supports ErrorRecoveryLevel greater than 9452 zero, it MUST support all SNACK types. 9454 The SNACK is used by the initiator to request the retransmission 9455 of numbered-responses, data, or R2T PDUs from the target. The 9456 SNACK request indicates the numbered-responses or data "runs" 9457 whose retransmission is requested by the target, where the run 9458 starts with the first StatSN, DataSN, or R2TSN whose 9459 retransmission is requested and indicates the number of Status, 9460 Data, or R2T PDUs requested including the first. 0 has special 9461 meaning when used as a starting number and length: 9463 - When used in RunLength, it means all PDUs starting with the 9464 initial. 9466 - When used in both BegRun and RunLength, it means all 9467 unacknowledged PDUs. 9469 The numbered-response(s) or R2T(s), requested by a SNACK, MUST be 9470 delivered as exact replicas of the ones that the target 9471 transmitted originally except for the fields ExpCmdSN, MaxCmdSN, 9472 and ExpDataSN, which MUST carry the current values. 9473 R2T(s)requested by SNACK MUST also carry the current value of 9474 StatSN. 9476 The numbered Data-In PDUs, requested by a Data SNACK MUST be 9477 delivered as exact replicas of the ones that the target 9478 transmitted originally except for the fields ExpCmdSN and 9479 MaxCmdSN, which MUST carry the current values and except for 9480 resegmentation (see Section 11.16.3). 9482 Any SNACK that requests a numbered-response, Data, or R2T that was 9483 not sent by the target or was already acknowledged by the 9484 initiator, MUST be rejected with a reason code of "Protocol 9485 error". 9487 11.16.1. Type 9489 This field encodes the SNACK function as follows: 9491 0-Data/R2T SNACK - requesting retransmission of one or more 9492 Data-In or R2T PDUs. 9494 1-Status SNACK - requesting retransmission of one or more 9495 numbered responses. 9497 2-DataACK - positively acknowledges Data-In PDUs. 9499 3-R-Data SNACK - requesting retransmission of Data-In PDUs 9500 with possible resegmentation and status tagging. 9502 All other values are reserved. 9504 Data/R2T SNACK, Status SNACK, or R-Data SNACK for a command MUST 9505 precede status acknowledgement for the given command. 9507 11.16.2. Data Acknowledgement 9509 If an initiator operates at ErrorRecoveryLevel 1 or higher, it 9510 MUST issue a SNACK of type DataACK after receiving a Data-In PDU 9511 with the A bit set to 1. However, if the initiator has detected 9512 holes in the input sequence, it MUST postpone issuing the SNACK of 9513 type DataACK until the holes are filled. An initiator MAY ignore 9514 the A bit if it deems that the bit is being set aggressively by 9515 the target (i.e., before the MaxBurstLength limit is 9516 reached). 9518 The DataACK is used to free resources at the target and not to 9519 request or imply data retransmission. 9521 An initiator MUST NOT request retransmission for any data it had 9522 already acknowledged. 9524 11.16.3. Resegmentation 9526 If the initiator MaxRecvDataSegmentLength changed between the 9527 original transmission and the time the initiator requests 9528 retransmission, the initiator MUST issue a R-Data SNACK (see 9529 Section 11.16.1). With R-Data SNACK, the initiator indicates that 9530 it discards all the unacknowledged data and expects the target to 9531 resend it. It also expects resegmentation. In this case, the 9532 retransmitted Data-In PDUs MAY be different from the ones 9533 originally sent in order to reflect changes in 9534 MaxRecvDataSegmentLength. Their DataSN starts with the BegRun of 9535 the last DataACK received by the target if any was received; 9536 otherwise it starts with 0 and is increased by 1 for each resent 9537 Data-In PDU. 9539 A target that has received a R-Data SNACK MUST return a SCSI 9540 Response that contains a copy of the SNACK Tag field from the R- 9541 Data SNACK in the SCSI Response SNACK Tag field as its last or 9542 only Response. For example, if it has already sent a response 9543 containing another value in the SNACK Tag field or had the status 9544 included in the last Data-In PDU, it must send a new SCSI Response 9545 PDU. If a target sends more than one SCSI Response PDU due to this 9546 rule, all SCSI responses must carry the same StatSN (see Section 9547 11.4.4). If an initiator attempts to recover a lost SCSI Response 9548 (with a Status-SNACK, see Section 11.16.1) when more than one 9549 response has been sent, the target will send the SCSI Response 9550 with the latest content known to the target, including the last 9551 SNACK Tag for the command. 9553 For considerations in allegiance reassignment of a task to a 9554 connection with a different MaxRecvDataSegmentLength, refer to 9555 Section 7.2.2. 9557 11.16.4. Initiator Task Tag 9559 For Status SNACK and DataACK, the Initiator Task Tag MUST be set 9560 to the reserved value 0xffffffff. In all other cases, the 9561 Initiator Task Tag field MUST be set to the Initiator Task Tag of 9562 the referenced command. 9564 11.16.5. Target Transfer Tag or SNACK Tag 9566 For an R-Data SNACK, this field MUST contain a value that is 9567 different from 0 or 0xffffffff and is unique for the task 9568 (identified by the Initiator Task Tag). This value MUST be copied 9569 by the iSCSI target in the last or only SCSI Response PDU it 9570 issues for the command. 9572 For DataACK, the Target Transfer Tag MUST contain a copy of the 9573 Target Transfer Tag and LUN provided with the SCSI Data-In PDU 9574 with the A bit set to 1. 9576 In all other cases, the Target Transfer Tag field MUST be set to 9577 the reserved value of 0xffffffff. 9579 11.16.6. BegRun 9581 The DataSN, R2TSN, or StatSN of the first PDU whose retransmission 9582 is requested (Data/R2T and Status SNACK), or the next expected 9583 DataSN (DataACK SNACK). 9585 BegRun 0 when used in conjunction with RunLength 0 means resend 9586 all unacknowledged Data-In, R2T or Response PDUs. 9588 BegRun MUST be 0 for a R-Data SNACK. 9590 11.16.7. RunLength 9592 The number of PDUs whose retransmission is requested. 9594 RunLength 0 signals that all Data-In, R2T, or Response PDUs 9595 carrying the numbers equal to or greater than BegRun have to be 9596 resent. 9598 The RunLength MUST also be 0 for a DataACK SNACK in addition to R- 9599 Data SNACK. 9601 11.17. Reject 9603 Byte/ 0 | 1 | 2 | 3 | 9604 / | | | | 9605 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 9606 +---------------+---------------+---------------+--------------+ 9607 0|.|.| 0x3f |1| Reserved | Reason | Reserved | 9608 +---------------+---------------+---------------+--------------+ 9609 4|TotalAHSLength | DataSegmentLength | 9610 +---------------+---------------+---------------+--------------+ 9611 8/ Reserved / 9612 +/ / 9613 +---------------+---------------+---------------+--------------+ 9614 16| 0xffffffff | 9615 +---------------+---------------+---------------+--------------+ 9616 20| Reserved | 9617 +---------------+---------------+---------------+--------------+ 9618 24| StatSN | 9619 +---------------+---------------+---------------+--------------+ 9620 28| ExpCmdSN | 9621 +---------------+---------------+---------------+--------------+ 9622 32| MaxCmdSN | 9623 +---------------+---------------+---------------+--------------+ 9624 36| DataSN/R2TSN or Reserved | 9625 +---------------+---------------+---------------+--------------+ 9626 40| Reserved | 9627 +---------------+---------------+---------------+--------------+ 9628 44| Reserved | 9629 +---------------+---------------+---------------+--------------+ 9630 48| Header-Digest (Optional) | 9631 +---------------+---------------+---------------+--------------+ 9632 xx/ Complete Header of Bad PDU / 9633 +/ / 9634 +---------------+---------------+---------------+--------------+ 9635 yy/Vendor specific data (if any) / 9636 / / 9637 +---------------+---------------+---------------+--------------+ 9638 zz| Data-Digest (Optional) | 9639 +---------------+---------------+---------------+--------------+ 9641 Reject is used to indicate an iSCSI error condition (protocol, 9642 unsupported option, etc.). 9644 11.17.1. Reason 9646 The reject Reason is coded as follows: 9648 +------+----------------------------------------+----------------+ 9649 | Code | Explanation |Can the original| 9650 | (hex)| |PDU be re-sent? | 9651 +------+----------------------------------------+----------------+ 9652 | 0x01 | Reserved | no | 9653 | | | | 9654 | 0x02 | Data (payload) Digest Error | yes (Note 1) | 9655 | | | | 9656 | 0x03 | SNACK Reject | yes | 9657 | | | | 9658 | 0x04 | Protocol Error (e.g., SNACK request for| no | 9659 | | a status that was already acknowledged)| | 9660 | | | | 9661 | 0x05 | Command not supported | no | 9662 | | | | 9663 | 0x06 | Immediate Command Reject - too many | yes | 9664 | | immediate commands | | 9665 | | | | 9666 | 0x07 | Task in progress | no | 9667 | | | | 9668 | 0x08 | Invalid Data ACK | no | 9669 | | | | 9670 | 0x09 | Invalid PDU field | no (Note 2) | 9671 | | | | 9672 | 0x0a | Long Operation Reject - Can't generate | yes | 9673 | | Target Transfer Tag - out of resources | | 9674 | | | | 9675 | 0x0c | Waiting for Logout | no | 9676 +------+----------------------------------------+----------------+ 9678 Note 1: For iSCSI, Data-Out PDU retransmission is only done if the 9679 target requests retransmission with a recovery R2T. However, if 9680 this is the data digest error on immediate data, the initiator may 9681 choose to retransmit the whole PDU including the immediate data. 9683 Note 2: A target should use this reason code for all invalid 9684 values of PDU fields that are meant to describe a task, a 9685 response, or a data transfer. Some examples are invalid TTT/ITT, 9686 buffer offset, LUN qualifying a TTT, and an invalid sequence 9687 number in a SNACK. 9689 Note 3: Reason code 0x0b ("Negotiation reset") defined in 9690 [RFC3720] is deprecated and MUST NOT be used by implementations. 9691 An implementation receiving reason code 0x0b MUST treat it as a 9692 negotiation failure that terminates the Login Phase and the TCP 9693 connection, as specified in Section 7.12. 9695 All other values for Reason are reserved. 9697 In all the cases in which a pre-instantiated SCSI task is 9698 terminated because of the reject, the target MUST issue a proper 9699 SCSI command response with CHECK CONDITION as described in Section 9700 11.4.3. In these cases in which a status for the SCSI task was 9701 already sent before the reject, no additional status is required. 9702 If the error is detected while data from the initiator is still 9703 expected (i.e., the command PDU did not contain all the data and 9704 the target has not received a Data-out PDU with the Final bit set 9705 to 1 for the unsolicited data, if any, and all outstanding R2Ts, 9706 if any), the target MUST wait until it receives the last expected 9707 Data-out PDUs with the F bit set to 1 before sending the Response 9708 PDU. 9710 For additional usage semantics of Reject PDU, see Section 7.3. 9712 11.17.2. DataSN/R2TSN 9714 This field is only valid if the rejected PDU is a Data/R2T SNACK 9715 and the Reject reason code is "Protocol error" (see Section 9716 11.16). The DataSN/R2TSN is the next Data/R2T sequence number 9717 that the target would send for the task, if any. 9719 11.17.3. StatSN, ExpCmdSN and MaxCmdSN 9721 These fields carry their usual values and are not related to the 9722 rejected command. StatSN is advanced after a Reject. 9724 11.17.4. Complete Header of Bad PDU 9726 The target returns the header (not including digest) of the PDU in 9727 error as the data of the response. 9729 11.18. NOP-Out 9731 Byte/ 0 | 1 | 2 | 3 | 9732 / | | | | 9733 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 9734 +---------------+---------------+---------------+--------------+ 9735 0|.|I| 0x00 |1| Reserved | 9736 +---------------+---------------+---------------+--------------+ 9737 4|TotalAHSLength | DataSegmentLength | 9738 +---------------+---------------+---------------+--------------+ 9739 8| LUN or Reserved | 9740 + + 9741 12| | 9742 +---------------+---------------+---------------+--------------+ 9743 16| Initiator Task Tag or 0xffffffff | 9744 +---------------+---------------+---------------+--------------+ 9745 20| Target Transfer Tag or 0xffffffff | 9746 +---------------+---------------+---------------+--------------+ 9747 24| CmdSN | 9748 +---------------+---------------+---------------+--------------+ 9749 28| ExpStatSN | 9750 +---------------+---------------+---------------+--------------+ 9751 32/ Reserved / 9752 +/ / 9753 +---------------+---------------+---------------+--------------+ 9754 48| Header-Digest (Optional) | 9755 +---------------+---------------+---------------+--------------+ 9756 / DataSegment - Ping Data (optional) / 9757 +/ / 9758 +---------------+---------------+---------------+--------------+ 9759 | Data-Digest (Optional) | 9760 +---------------+---------------+---------------+--------------+ 9762 A NOP-Out may be used by an initiator as a "ping request" to 9763 verify that a connection/session is still active and all its 9764 components are operational. The NOP-In response is the "ping 9765 echo". 9767 A NOP-Out is also sent by an initiator in response to a NOP-In. 9769 A NOP-Out may also be used to confirm a changed ExpStatSN if 9770 another PDU will not be available for a long time. 9772 Upon receipt of a NOP-In with the Target Transfer Tag set to a 9773 valid value (not the reserved 0xffffffff), the initiator MUST 9774 respond with a NOP-Out. In this case, the NOP-Out Target Transfer 9775 Tag MUST contain a copy of the NOP-In Target Transfer Tag. The 9776 initiator SHOULD NOT send a NOP-Out in response to any other 9777 received NOP-In in order to avoid lengthy sequences of NOP-In and 9778 NOP-Out PDUs sent in response to each other. 9780 11.18.1. Initiator Task Tag 9782 The NOP-Out MUST have the Initiator Task Tag set to a valid value 9783 only if a response in the form of NOP-In is requested (i.e., the 9784 NOP-Out is used as a ping request). Otherwise, the Initiator Task 9785 Tag MUST be set to 0xffffffff. 9787 When a target receives the NOP-Out with a valid Initiator Task 9788 Tag, it MUST respond with a Nop-In Response (see Section 6). 9790 If the Initiator Task Tag contains 0xffffffff, the I bit MUST be 9791 set to 1 and the CmdSN is not advanced after this PDU is sent. 9793 11.18.2. Target Transfer Tag 9795 A target assigned identifier for the operation. 9797 The NOP-Out MUST only have the Target Transfer Tag set if it is 9798 issued in response to a NOP-In with a valid Target Transfer Tag. 9799 In this case, it copies the Target Transfer Tag from the NOP-In 9800 PDU. Otherwise, the Target Transfer Tag MUST be set to 0xffffffff. 9802 When the Target Transfer Tag is set to a value other than 9803 0xffffffff, the LUN field MUST also be copied from the NOP-In. 9805 11.18.3. Ping Data 9807 Ping data is reflected in the NOP-In Response. The length of the 9808 reflected data is limited to MaxRecvDataSegmentLength. The length 9809 of ping data is indicated by the DataSegmentLength. 0 is a valid 9810 value for the DataSegmentLength and indicates the absence of ping 9811 data. 9813 11.19. NOP-In 9815 Byte/ 0 | 1 | 2 | 3 | 9816 / | | | | 9817 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 67| 9818 +---------------+---------------+---------------+--------------+ 9819 0|.|.| 0x20 |1| Reserved | 9820 +---------------+---------------+---------------+--------------+ 9821 4|TotalAHSLength | DataSegmentLength | 9822 +---------------+---------------+---------------+--------------+ 9823 8| LUN or Reserved | 9824 + + 9825 12| | 9826 +---------------+---------------+---------------+--------------+ 9827 16| Initiator Task Tag or 0xffffffff | 9828 +---------------+---------------+---------------+--------------+ 9829 20| Target Transfer Tag or 0xffffffff | 9830 +---------------+---------------+---------------+--------------+ 9831 24| StatSN | 9832 +---------------+---------------+---------------+--------------+ 9833 28| ExpCmdSN | 9834 +---------------+---------------+---------------+--------------+ 9835 32| MaxCmdSN | 9836 +---------------+---------------+---------------+--------------+ 9837 36/ Reserved / 9838 +/ / 9839 +---------------+---------------+---------------+--------------+ 9840 48| Header-Digest (Optional) | 9841 +---------------+---------------+---------------+--------------+ 9842 / DataSegment - Return Ping Data / 9843 +/ / 9844 +---------------+---------------+---------------+--------------+ 9845 | Data-Digest (Optional) | 9846 +---------------+---------------+---------------+--------------+ 9848 NOP-In is either sent by a target as a response to a NOP-Out, as a 9849 "ping" to an initiator, or as a means to carry a changed ExpCmdSN 9850 and/or MaxCmdSN if another PDU will not be available for a long 9851 time (as determined by the target). 9853 When a target receives the NOP-Out with a valid Initiator Task Tag 9854 (not the reserved value 0xffffffff), it MUST respond with a NOP-In 9855 with the same Initiator Task Tag that was provided in the NOP-Out 9856 request. It MUST also duplicate up to the first 9857 MaxRecvDataSegmentLength bytes of the initiator provided Ping 9858 Data. For such a response, the Target Transfer Tag MUST be 9859 0xffffffff. The target SHOULD NOT send a NOP-In in response to any 9860 other received NOP-Out in order to avoid lengthy sequences of NOP- 9861 In and NOP-Out PDUs sent in response to each other. 9863 Otherwise, when a target sends a NOP-In that is not a response to 9864 a Nop-Out received from the initiator, the Initiator Task Tag MUST 9865 be set to 0xffffffff and the Data Segment MUST NOT contain any 9866 data (DataSegmentLength MUST be 0). 9868 11.19.1. Target Transfer Tag 9870 If the target is responding to a NOP-Out, this is set to the 9871 reserved value 0xffffffff. 9873 If the target is sending a NOP-In as a Ping (intending to receive 9874 a corresponding NOP-Out), this field is set to a valid value (not 9875 the reserved 0xffffffff). 9877 If the target is initiating a NOP-In without wanting to receive a 9878 corresponding NOP-Out, this field MUST hold the reserved value of 9879 0xffffffff. 9881 11.19.2. StatSN 9883 The StatSN field will always contain the next StatSN. However, 9884 when the Initiator Task Tag is set to 0xffffffff StatSN for the 9885 connection is not advanced after this PDU is sent. 9887 11.19.3. LUN 9889 A LUN MUST be set to a correct value when the Target Transfer Tag 9890 is valid (not the reserved value 0xffffffff). 9892 12. iSCSI Security Text Keys and Authentication Methods 9894 Only the following keys are used during the SecurityNegotiation 9895 stage of the Login Phase: 9897 SessionType 9899 InitiatorName 9901 TargetName 9903 TargetAddress 9905 InitiatorAlias 9907 TargetAlias 9909 TargetPortalGroupTag 9911 AuthMethod and the keys used by the authentication methods 9912 specified under Section 12.1 along with all of their 9913 associated keys as well as Vendor-Specific Authentication 9914 Methods. 9916 Other keys MUST NOT be used. 9918 SessionType, InitiatorName, TargetName, InitiatorAlias, 9919 TargetAlias, and TargetPortalGroupTag are described in Section 13 9920 as they can be used also in the OperationalNegotiation stage. 9922 All security keys have connection-wide applicability. 9924 12.1. AuthMethod 9926 Use: During Login - Security Negotiation 9927 Senders: Initiator and Target 9928 Scope: connection 9930 AuthMethod = 9932 The main item of security negotiation is the authentication method 9933 (AuthMethod). 9935 The authentication methods that can be used (appear in the list- 9936 of-values) are either those listed in the following table or are 9937 vendor-unique methods: 9939 +------------------------------------------------------------+ 9940 | Name | Description | 9941 +------------------------------------------------------------+ 9942 | KRB5 | Kerberos V5 - defined in [RFC4120] | 9943 +------------------------------------------------------------+ 9944 | SRP | Secure Remote Password | 9945 | | defined in [RFC2945] | 9946 +------------------------------------------------------------+ 9947 | CHAP | Challenge Handshake Authentication Protocol| 9948 | | defined in [RFC1994] | 9949 +------------------------------------------------------------+ 9950 | None | No authentication | 9951 +------------------------------------------------------------+ 9953 The AuthMethod selection is followed by an "authentication 9954 exchange" specific to the authentication method selected. 9956 The authentication method proposal may be made by either the 9957 initiator or the target. However the initiator MUST make the first 9958 step specific to the selected authentication method as soon as it 9959 is selected. It follows that if the target makes the 9960 authentication method proposal the initiator sends the first 9961 key(s) of the exchange together with its authentication method 9962 selection. 9964 The authentication exchange authenticates the initiator to the 9965 target, and optionally, the target to the initiator. 9966 Authentication is OPTIONAL to use but MUST be supported by the 9967 target and initiator. 9969 The initiator and target MUST implement CHAP. All other 9970 authentication methods are OPTIONAL. 9972 Private or public extension algorithms MAY also be negotiated for 9973 authentication methods. Whenever a private or public extension 9974 algorithm is part of the default offer (the offer made in absence 9975 of explicit administrative action) the implementer MUST ensure 9976 that CHAP is listed as an alternative in the default offer and 9977 "None" is not part of the default offer. 9979 Extension authentication methods MUST be named using one of the 9980 following two formats: 9982 i) Z-reversed.vendor.dns_name.do_something= 9983 ii) New public key with no name prefix constraints 9985 Authentication methods named using the Z- format are used as 9986 private extensions. New public keys must be registered with IANA 9987 using IETF Review process ([RFC5226]). New public extensions for 9988 authentication methods MUST NOT use the Z# name prefix. 9990 For all of the public or private extension authentication methods, 9991 the method specific keys MUST conform to the format specified in 9992 Section 6.1 for standard-label. 9994 To identify the vendor for private extension authentication 9995 methods, we suggest you use the reversed DNS-name as a prefix to 9996 the proper digest names. 9998 The part of digest-name following Z- MUST conform to the format 9999 for standard-label specified in Section 6.1. 10001 Support for public or private extension authentication methods is 10002 OPTIONAL. 10004 The following subsections define the specific exchanges for each 10005 of the standardized authentication methods. As mentioned earlier 10006 the first step is always done by the initiator. 10008 12.1.1. Kerberos 10010 For KRB5 (Kerberos V5) [RFC4120] and [RFC1964], the initiator MUST 10011 use: 10013 KRB_AP_REQ= 10015 where KRB_AP_REQ is the client message as defined in [RFC4120]. 10017 The default principal name assumed by an iSCSI initiator or target 10018 (prior to any administrative configuration action) MUST be the 10019 iSCSI Initiator Name or iSCSI Target Name respectively, prefixed 10020 by the string "iscsi/". 10022 If the initiator authentication fails, the target MUST respond 10023 with a Login reject with "Authentication Failure" status. 10024 Otherwise, if the initiator has selected the mutual authentication 10025 option (by setting MUTUAL-REQUIRED in the ap-options field of the 10026 KRB_AP_REQ), the target MUST reply with: 10028 KRB_AP_REP= 10030 where KRB_AP_REP is the server's response message as defined in 10031 [RFC4120]. 10033 If mutual authentication was selected and target authentication 10034 fails, the initiator MUST close the connection. 10036 KRB_AP_REQ and KRB_AP_REP are binary-values and their binary 10037 length (not the length of the character string that represents 10038 them in encoded form) MUST NOT exceed 65536 bytes. Hex or Base64 10039 encoding may be used for KRB_AP_REQ and KRB_AP_REP, see Section 10040 6.1. 10042 12.1.2. Secure Remote Password (SRP) 10044 For SRP [RFC2945], the initiator MUST use: 10046 SRP_U= TargetAuth=Yes /* or TargetAuth=No */ 10048 The target MUST answer with a Login reject with the "Authorization 10049 Failure" status or reply with: 10051 SRP_GROUP= SRP_s= 10053 Where G1,G2... are proposed groups, in order of preference. 10055 The initiator MUST either close the connection or continue with: 10057 SRP_A= SRP_GROUP= 10059 Where G is one of G1,G2... that were proposed by the target. 10061 The target MUST answer with a Login reject with the 10062 "Authentication Failure" status or reply with: 10064 SRP_B= 10066 The initiator MUST close the connection or continue with: 10068 SRP_M= 10070 If the initiator authentication fails, the target MUST answer with 10071 a Login reject with "Authentication Failure" status. Otherwise, if 10072 the initiator sent TargetAuth=Yes in the first message (requiring 10073 target authentication), the target MUST reply with: 10075 SRP_HM= 10077 If the target authentication fails, the initiator MUST close the 10078 connection. 10080 Where U, s, A, B, M, and H(A | M | K) are defined in [RFC2945] 10081 (using the SHA1 hash function, such as SRP-SHA1) and G,Gn (Gn 10082 stands for G1,G2...) are identifiers of SRP groups specified in 10083 [RFC3723]. G, Gn, and U are text strings, s,A,B,M, and H(A | M | 10084 K) are binary-values. The length of s,A,B,M and H(A | M | K) in 10085 binary form (not the length of the character string that 10086 represents them in encoded form) MUST NOT exceed 1024 bytes. Hex 10087 or Base64 encoding may be used for s,A,B,M and H(A | M | K), see 10088 Section 6.1. 10090 See Appendix B for the related login example. 10092 For the SRP_GROUP, all the groups specified in [RFC3723] up to 10093 1536 bits (i.e., SRP-768, SRP-1024, SRP-1280, SRP-1536) must be 10094 supported by initiators and targets. To guarantee 10095 interoperability, targets MUST always offer "SRP-1536" as one of 10096 the proposed groups. 10098 12.1.3. Challenge Handshake Authentication Protocol (CHAP) 10100 For CHAP [RFC1994], the initiator MUST use: 10102 CHAP_A= 10104 Where A1,A2... are proposed algorithms, in order of preference. 10106 The target MUST answer with a Login reject with the 10107 "Authentication Failure" status or reply with: 10109 CHAP_A= CHAP_I= CHAP_C= 10111 Where A is one of A1,A2... that were proposed by the initiator. 10113 The initiator MUST continue with: 10115 CHAP_N= CHAP_R= 10117 or, if it requires target authentication, with: 10119 CHAP_N= CHAP_R= CHAP_I= CHAP_C= 10121 If the initiator authentication fails, the target MUST answer with 10122 a Login reject with "Authentication Failure" status. Otherwise, if 10123 the initiator required target authentication, the target MUST 10124 either answer with a Login reject with "Authentication Failure" or 10125 reply with: 10127 CHAP_N= CHAP_R= 10129 If target authentication fails, the initiator MUST close the 10130 connection. 10132 Where N, (A,A1,A2), I, C, and R are (correspondingly) the Name, 10133 Algorithm, Identifier, Challenge, and Response as defined in 10134 [RFC1994], N is a text string, A,A1,A2, and I are numbers, and C 10135 and R are binary-values and their binary length (not the length of 10136 the character string that represents them in encoded form) MUST 10137 NOT exceed 1024 bytes. Hex or Base64 encoding may be used for C 10138 and R, see Section 6.1. 10140 See Appendix B for the related login example. 10142 For the Algorithm, as stated in [RFC1994], one value is required 10143 to be implemented: 10145 5 (CHAP with MD5) 10147 To guarantee interoperability, initiators MUST always offer it as 10148 one of the proposed algorithms. 10150 13. Login/Text Operational Text Keys 10152 Some session specific parameters MUST only be carried on the 10153 leading connection and cannot be changed after the leading 10154 connection login (e.g., MaxConnections, the maximum number of 10155 connections). This holds for a single connection session with 10156 regard to connection restart. The keys that fall into this 10157 category have the use: LO (Leading Only). 10159 Keys that can only be used during login have the use: IO 10160 (initialize only), while those that can be used in both the Login 10161 Phase and Full Feature Phase have the use: ALL. 10163 Keys that can only be used during Full Feature Phase use FFPO 10164 (Full Feature Phase only). 10166 Keys marked as Any-Stage may also appear in the 10167 SecurityNegotiation stage while all other keys described in this 10168 Section are operational keys. 10170 Keys that do not require an answer are marked as Declarative. 10172 Key scope is indicated as session-wide (SW) or connection-only 10173 (CO). 10175 Result function, wherever mentioned, states the function that can 10176 be applied to check the validity of the responder selection. 10177 Minimum means that the selected value cannot exceed the offered 10178 value. Maximum means that the selected value cannot be lower than 10179 the offered value. AND means that the selected value must be a 10180 possible result of a Boolean "and" function with an arbitrary 10181 Boolean value (e.g., if the offered value is No the selected value 10182 must be No). OR means that the selected value must be a possible 10183 result of a Boolean "or" function with an arbitrary Boolean value 10184 (e.g., if the offered value is Yes the selected value must be 10185 Yes). 10187 13.1. HeaderDigest and DataDigest 10189 Use: IO 10190 Senders: Initiator and Target 10191 Scope: CO 10192 HeaderDigest = 10193 DataDigest = 10195 Default is None for both HeaderDigest and DataDigest. 10197 Digests enable the checking of end-to-end, non-cryptographic data 10198 integrity beyond the integrity checks provided by the link layers 10199 and the covering of the whole communication path including all 10200 elements that may change the network level PDUs such as routers, 10201 switches, and proxies. 10203 The following table lists cyclic integrity checksums that can be 10204 negotiated for the digests and that MUST be implemented by every 10205 iSCSI initiator and target. These digest options only have error 10206 detection significance. 10208 +---------------------------------------------+ 10209 | Name | Description | Generator | 10210 +---------------------------------------------+ 10211 | CRC32C | 32 bit CRC |0x11edc6f41| 10212 +---------------------------------------------+ 10213 | None | no digest | 10214 +---------------------------------------------+ 10216 The generator polynomial for this digest is given in hex-notation 10217 (e.g., 0x3b stands for 0011 1011 and the polynomial is 10218 x**5+X**4+x**3+x+1). 10220 When the Initiator and Target agree on a digest, this digest MUST 10221 be used for every PDU in Full Feature Phase. 10223 Padding bytes, when present in a segment covered by a CRC, SHOULD 10224 be set to 0 and are included in the CRC. 10226 The CRC MUST be calculated by a method that produces the same 10227 results as the following process: 10229 - The PDU bits are considered as the coefficients of a 10230 polynomial M(x) of degree n-1; bit 7 of the lowest numbered 10231 byte is considered the most significant bit (x^n-1), 10232 followed by bit 6 of the lowest numbered byte through bit 0 10233 of the highest numbered byte (x^0). 10235 - The most significant 32 bits are complemented. 10237 - The polynomial is multiplied by x^32 then divided by G(x). 10238 The generator polynomial produces a remainder R(x) of degree 10239 <= 31. 10241 - The coefficients of R(x) are considered a 32 bit sequence. 10243 - The bit sequence is complemented and the result is the CRC. 10245 - The CRC bits are mapped into the digest word. The x^31 10246 coefficient in bit 7 of the lowest numbered byte of the 10247 digest continuing through to the byte up to the x^24 10248 coefficient in bit 0 of the lowest numbered byte, continuing 10249 with the x^23 coefficient in bit 7 of next byte through x^0 10250 in bit 0 of the highest numbered byte. 10252 - Computing the CRC over any segment (data or header) extended 10253 to include the CRC built using the generator 0x11edc6f41 10254 will always get the value 0x1c2d19ed as its final remainder 10255 (R(x)). This value is given here in its polynomial form 10256 (i.e., not mapped as the digest word). 10258 For a discussion about selection criteria for the CRC, see 10259 [RFC3385]. For a detailed analysis of the iSCSI polynomial, see 10260 [Castagnoli93]. 10262 Private or public extension algorithms MAY also be negotiated for 10263 digests. Whenever a private or public digest extension algorithm 10264 is part of the default offer (the offer made in absence of 10265 explicit administrative action) the implementer MUST ensure that 10266 CRC32C is listed as an alternative in the default offer and "None" 10267 is not part of the default offer. 10269 Extension digest algorithms MUST be named using one of the 10270 following two formats: 10272 i) Y-reversed.vendor.dns_name.do_something= 10273 ii) New public key with no name prefix constraints 10275 Digests named using the Y- format are used for private purposes 10276 (unregistered). New public keys must be registered with IANA using 10277 IETF Review process ([RFC5226]). New public extensions for digests 10278 MUST NOT use the Y# name prefix. 10280 For private extension digests, to identify the vendor, we suggest 10281 you use the reversed DNS-name as a prefix to the proper digest 10282 names. 10284 The part of digest-name following Y- MUST conform to the format 10285 for standard-label specified in Section 6.1. 10287 Support for public or private extension digests is OPTIONAL. 10289 13.2. MaxConnections 10291 Use: LO 10292 Senders: Initiator and Target 10293 Scope: SW 10294 Irrelevant when: SessionType=Discovery 10296 MaxConnections= 10298 Default is 1. 10299 Result function is Minimum. 10301 Initiator and target negotiate the maximum number of connections 10302 requested/acceptable. 10304 13.3. SendTargets 10306 Use: FFPO 10307 Senders: Initiator 10308 Scope: SW 10310 For a complete description, see Appendix C. 10312 13.4. TargetName 10314 Use: IO by initiator, FFPO by target - only as response to a 10315 SendTargets, Declarative, Any-Stage 10316 Senders: Initiator and Target 10317 Scope: SW 10319 TargetName= 10321 Examples: 10323 TargetName=iqn.1993-11.com.disk-vendor:diskarrays.sn.45678 10325 TargetName=eui.020000023B040506 10327 TargetName=naa.62004567BA64678D0123456789ABCDEF 10329 The initiator of the TCP connection MUST provide this key to the 10330 remote endpoint in the first login request if the initiator is not 10331 establishing a discovery session. The iSCSI Target Name specifies 10332 the worldwide unique name of the target. 10334 The TargetName key may also be returned by the "SendTargets" text 10335 request (which is its only use when issued by a target). 10337 TargetName MUST NOT be redeclared within the login phase. 10339 13.5. InitiatorName 10341 Use: IO, Declarative, Any-Stage 10342 Senders: Initiator 10343 Scope: SW 10345 InitiatorName= 10347 Examples: 10349 InitiatorName=iqn.1992-04.com.os-vendor.plan9:cdrom.12345 10351 InitiatorName=iqn.2001-02.com.ssp.users:customer235.host90 10353 InitiatorName=naa.52004567BA64678D 10355 The initiator of the TCP connection MUST provide this key to the 10356 remote endpoint at the first Login of the Login Phase for every 10357 connection. The InitiatorName key enables the initiator to 10358 identify itself to the remote endpoint. 10360 InitiatorName MUST NOT be redeclared within the login phase. 10362 13.6. TargetAlias 10364 Use: ALL, Declarative, Any-Stage 10365 Senders: Target 10366 Scope: SW 10368 TargetAlias= 10370 Examples: 10372 TargetAlias=Bob-s Disk 10374 TargetAlias=Database Server 1 Log Disk 10376 TargetAlias=Web Server 3 Disk 20 10378 If a target has been configured with a human-readable name or 10379 description, this name SHOULD be communicated to the initiator 10380 during a Login Response PDU if SessionType=Normal (see 13.21). 10381 This string is not used as an identifier, nor is it meant to be 10382 used for authentication or authorization decisions. It can be 10383 displayed by the initiator's user interface in a list of targets 10384 to which it is connected. 10386 13.7. InitiatorAlias 10388 Use: ALL, Declarative, Any-Stage 10389 Senders: Initiator 10390 Scope: SW 10392 InitiatorAlias= 10394 Examples: 10396 InitiatorAlias=Web Server 4 10398 InitiatorAlias=spyalley.nsa.gov 10400 InitiatorAlias=Exchange Server 10402 If an initiator has been configured with a human-readable name or 10403 description, it SHOULD be communicated to the target during a 10404 Login Request PDU. If not, the host name can be used instead. This 10405 string is not used as an identifier, nor is meant to be used for 10406 authentication or authorization decisions. It can be displayed by 10407 the target's user interface in a list of initiators to which it is 10408 connected. 10410 13.8. TargetAddress 10412 Use: ALL, Declarative, Any-Stage 10413 Senders: Target 10414 Scope: SW 10416 TargetAddress=domainname[:port][,portal-group-tag] 10418 The domainname can be specified as either a DNS host name, a 10419 dotted-decimal IPv4 address, or a bracketed IPv6 address as 10420 specified in [RFC3986]. 10422 If the TCP port is not specified, it is assumed to be the IANA- 10423 assigned default port for iSCSI (see Section 14). 10425 If the TargetAddress is returned as the result of a redirect 10426 status in a login response, the comma and portal group tag MUST be 10427 omitted. 10429 If the TargetAddress is returned within a SendTargets response, 10430 the portal group tag MUST be included. 10432 Examples: 10434 TargetAddress=10.0.0.1:5003,1 10436 TargetAddress=[1080:0:0:0:8:800:200C:417A],65 10437 TargetAddress=[1080::8:800:200C:417A]:5003,1 10439 TargetAddress=computingcenter.example.com,23 10441 Use of the portal-group-tag is described in Appendix C. The 10442 formats for the port and portal-group-tag are the same as the one 10443 specified in TargetPortalGroupTag. 10445 13.9. TargetPortalGroupTag 10447 Use: IO by target, Declarative, Any-Stage 10448 Senders: Target 10449 Scope: SW 10451 TargetPortalGroupTag=<16-bit-binary-value> 10453 Examples: 10454 TargetPortalGroupTag=1 10456 The target portal group tag is a 16-bit binary-value that uniquely 10457 identifies a portal group within an iSCSI target node. This key 10458 carries the value of the tag of the portal group that is servicing 10459 the Login request. The iSCSI target returns this key to the 10460 initiator in the Login Response PDU to the first Login Request PDU 10461 that has the C bit set to 0 when TargetName is given by the 10462 initiator. 10464 [SAM2] notes in its informative text that TPGT value should be 10465 non-zero, note that it is incorrect. A zero value is allowed as a 10466 legal value for TPGT. This discrepancy currently stands corrected 10467 in [SAM4]. 10469 For the complete usage expectations of this key see Section 6.3. 10471 13.10. InitialR2T 10473 Use: LO 10474 Senders: Initiator and Target 10475 Scope: SW 10476 Irrelevant when: SessionType=Discovery 10478 InitialR2T= 10480 Examples: 10482 I->InitialR2T=No 10484 T->InitialR2T=No 10486 Default is Yes. 10487 Result function is OR. 10489 The InitialR2T key is used to turn off the default use of R2T for 10490 unidirectional and the output part of bidirectional commands, thus 10491 allowing an initiator to start sending data to a target as if it 10492 has received an initial R2T with Buffer Offset=Immediate Data 10493 Length and Desired Data Transfer Length=(min(FirstBurstLength, 10494 Expected Data Transfer Length) - Received Immediate Data Length). 10496 The default action is that R2T is required, unless both the 10497 initiator and the target send this key-pair attribute specifying 10498 InitialR2T=No. Only the first outgoing data burst (immediate data 10499 and/or separate PDUs) can be sent unsolicited (i.e., not requiring 10500 an explicit R2T). 10502 13.11. ImmediateData 10504 Use: LO 10505 Senders: Initiator and Target 10506 Scope: SW 10507 Irrelevant when: SessionType=Discovery 10509 ImmediateData= 10511 Default is Yes. 10512 Result function is AND. 10514 The initiator and target negotiate support for immediate data. To 10515 turn immediate data off, the initiator or target must state its 10516 desire to do so. ImmediateData can be turned on if both the 10517 initiator and target have ImmediateData=Yes. 10519 If ImmediateData is set to Yes and InitialR2T is set to Yes 10520 (default), then only immediate data are accepted in the first 10521 burst. 10523 If ImmediateData is set to No and InitialR2T is set to Yes, then 10524 the initiator MUST NOT send unsolicited data and the target MUST 10525 reject unsolicited data with the corresponding response code. 10527 If ImmediateData is set to No and InitialR2T is set to No, then 10528 the initiator MUST NOT send unsolicited immediate data, but MAY 10529 send one unsolicited burst of Data-OUT PDUs. 10531 If ImmediateData is set to Yes and InitialR2T is set to No, then 10532 the initiator MAY send unsolicited immediate data and/or one 10533 unsolicited burst of Data-OUT PDUs. 10535 The following table is a summary of unsolicited data options: 10537 +----------+-------------+------------------+--------------+ 10538 |InitialR2T|ImmediateData| Unsolicited |Immediate Data| 10539 | | | Data Out PDUs | | 10540 +----------+-------------+------------------+--------------+ 10541 | No | No | Yes | No | 10542 +----------+-------------+------------------+--------------+ 10543 | No | Yes | Yes | Yes | 10544 +----------+-------------+------------------+--------------+ 10545 | Yes | No | No | No | 10546 +----------+-------------+------------------+--------------+ 10547 | Yes | Yes | No | Yes | 10548 +----------+-------------+------------------+--------------+ 10550 13.12. MaxRecvDataSegmentLength 10552 Use: ALL, Declarative 10553 Senders: Initiator and Target 10554 Scope: CO 10556 MaxRecvDataSegmentLength= 10558 Default is 8192 bytes. 10560 The initiator or target declares the maximum data segment length 10561 in bytes it can receive in an iSCSI PDU. 10563 The transmitter (initiator or target) is required to send PDUs 10564 with a data segment that does not exceed MaxRecvDataSegmentLength 10565 of the receiver. 10567 A target receiver is additionally limited by MaxBurstLength for 10568 solicited data and FirstBurstLength for unsolicited data. An 10569 initiator MUST NOT send solicited PDUs exceeding MaxBurstLength 10570 nor unsolicited PDUs exceeding FirstBurstLength (or 10571 FirstBurstLength-Immediate Data Length if immediate data were 10572 sent). 10574 13.13. MaxBurstLength 10576 Use: LO 10577 Senders: Initiator and Target 10578 Scope: SW 10579 Irrelevant when: SessionType=Discovery 10581 MaxBurstLength= 10583 Default is 262144 (256 Kbytes). 10584 Result function is Minimum. 10586 The initiator and target negotiate maximum SCSI data payload in 10587 bytes in a Data-In or a solicited Data-Out iSCSI sequence. A 10588 sequence consists of one or more consecutive Data-In or Data-Out 10589 PDUs that end with a Data-In or Data-Out PDU with the F bit set to 10590 one. 10592 13.14. FirstBurstLength 10594 Use: LO 10595 Senders: Initiator and Target 10596 Scope: SW 10597 Irrelevant when: SessionType=Discovery 10598 Irrelevant when: ( InitialR2T=Yes and ImmediateData=No ) 10600 FirstBurstLength= 10601 Default is 65536 (64 Kbytes). 10602 Result function is Minimum. 10604 The initiator and target negotiate the maximum amount in bytes of 10605 unsolicited data an iSCSI initiator may send to the target during 10606 the execution of a single SCSI command. This covers the immediate 10607 data (if any) and the sequence of unsolicited Data-Out PDUs (if 10608 any) that follow the command. 10610 FirstBurstLength MUST NOT exceed MaxBurstLength. 10612 13.15. DefaultTime2Wait 10614 Use: LO 10615 Senders: Initiator and Target 10616 Scope: SW 10618 DefaultTime2Wait= 10620 Default is 2. 10621 Result function is Maximum. 10623 The initiator and target negotiate the minimum time, in seconds, 10624 to wait before attempting an explicit/implicit logout or an active 10625 task reassignment after an unexpected connection termination or a 10626 connection reset. 10628 A value of 0 indicates that logout or active task reassignment can 10629 be attempted immediately. 10631 13.16. DefaultTime2Retain 10633 Use: LO 10634 Senders: Initiator and Target 10635 Scope: SW 10637 DefaultTime2Retain= 10639 Default is 20. 10640 Result function is Minimum. 10642 The initiator and target negotiate the maximum time, in seconds 10643 after an initial wait (Time2Wait), before which an active task 10644 reassignment is still possible after an unexpected connection 10645 termination or a connection reset. 10647 This value is also the session state timeout if the connection in 10648 question is the last LOGGED_IN connection in the session. 10650 A value of 0 indicates that connection/task state is immediately 10651 discarded by the target. 10653 13.17. MaxOutstandingR2T 10655 Use: LO 10656 Senders: Initiator and Target 10657 Scope: SW 10659 MaxOutstandingR2T= 10660 Irrelevant when: SessionType=Discovery 10662 Default is 1. 10663 Result function is Minimum. 10665 Initiator and target negotiate the maximum number of outstanding 10666 R2Ts per task, excluding any implied initial R2T that might be 10667 part of that task. An R2T is considered outstanding until the last 10668 data PDU (with the F bit set to 1) is transferred, or a sequence 10669 reception timeout (Section 7.1.4.1) is encountered for that data 10670 sequence. 10672 13.18. DataPDUInOrder 10674 Use: LO 10675 Senders: Initiator and Target 10676 Scope: SW 10677 Irrelevant when: SessionType=Discovery 10679 DataPDUInOrder= 10681 Default is Yes. 10682 Result function is OR. 10684 No is used by iSCSI to indicate that the data PDUs within 10685 sequences can be in any order. Yes is used to indicate that data 10686 PDUs within sequences have to be at continuously increasing 10687 addresses and overlays are forbidden. 10689 13.19. DataSequenceInOrder 10691 Use: LO 10692 Senders: Initiator and Target 10693 Scope: SW 10694 Irrelevant when: SessionType=Discovery 10696 DataSequenceInOrder= 10698 Default is Yes. 10699 Result function is OR. 10701 A Data Sequence is a sequence of Data-In or Data-Out PDUs that end 10702 with a Data-In or Data-Out PDU with the F bit set to one. A Data- 10703 out sequence is sent either unsolicited or in response to an R2T. 10704 Sequences cover an offset-range. 10706 If DataSequenceInOrder is set to No, Data PDU sequences may be 10707 transferred in any order. 10709 If DataSequenceInOrder is set to Yes, Data Sequences MUST be 10710 transferred using continuously non-decreasing sequence offsets 10711 (R2T buffer offset for writes, or the smallest SCSI Data-In buffer 10712 offset within a read data sequence). 10714 If DataSequenceInOrder is set to Yes, a target may retry at most 10715 the last R2T, and an initiator may at most request retransmission 10716 for the last read data sequence. For this reason, if 10717 ErrorRecoveryLevel is not 0 and DataSequenceInOrder is set to Yes 10718 then MaxOustandingR2T MUST be set to 1. 10720 13.20. ErrorRecoveryLevel 10722 Use: LO 10723 Senders: Initiator and Target 10724 Scope: SW 10725 ErrorRecoveryLevel= 10727 Default is 0. 10728 Result function is Minimum. 10730 The initiator and target negotiate the recovery level supported. 10732 Recovery levels represent a combination of recovery capabilities. 10733 Each recovery level includes all the capabilities of the lower 10734 recovery levels and adds some new ones to them. 10736 In the description of recovery mechanisms, certain recovery 10737 classes are specified. Section 7.1.5 describes the mapping between 10738 the classes and the levels. 10740 13.21. SessionType 10742 Use: LO, Declarative, Any-Stage 10743 Senders: Initiator 10744 Scope: SW 10746 SessionType= 10748 Default is Normal. 10750 The Initiator indicates the type of session it wants to create. 10751 The target can either accept it or reject it. 10753 A Discovery session indicates to the Target that the only purpose 10754 of this Session is discovery. The only requests a target accepts 10755 in this type of session are a text request with a SendTargets key 10756 and a logout request with reason "close the session". 10758 The Discovery session implies MaxConnections = 1 and overrides 10759 both the default and an explicit setting. As Section 7.4.1 10760 states, ErrorRecoveryLevel MUST be 0 (zero) for Discovery 10761 sessions. 10763 Depending on the type of the session, a target may decide on 10764 resources to allocate and the security to enforce, etc. for thion. 10765 If the SessionType key is thus going to be offered as "Discovery", 10767 it SHOULD be offered in the initial Login request by the 10768 initiator. 10770 13.22. The Private Extension Key Format 10772 Use: ALL 10773 Senders: Initiator and Target 10774 Scope: specific key dependent 10776 X-reversed.vendor.dns_name.do_something= 10778 Keys with this format are used for private extension purposes. 10779 These keys always start with X- if unregistered with IANA 10780 (private). New public keys (if registered with IANA via an IETF 10781 Review, [RFC5226]) no longer have an X# name prefix requirement, 10782 implementers may propose any intuitive unique name. 10784 For unregistered keys, to identify the vendor, we suggest you use 10785 the reversed DNS-name as a prefix to the key-proper. 10787 The part of key-name following X- MUST conform to the format for 10788 key-name specified in Section 6.1. 10790 Vendor specific keys MUST ONLY be used in normal sessions. 10792 Support for public or private extension keys is OPTIONAL. 10794 13.23. TaskReporting 10796 Use: LO 10797 Senders: Initiator and Target 10798 Scope: SW 10799 Irrelevant when: SessionType=Discovery 10800 TaskReporting= 10802 Default is RFC3720. 10803 Result function is AND. 10805 This key is used to negotiate the task completion reporting 10806 semantics from the SCSI target. The following table describes the 10807 semantics that an iSCSI target MUST support for respective 10808 negotiated key values. Whenever this key is negotiated, at least 10809 the RFC3720 and ResponseFence values MUST be offered as options by 10810 the negotiation originator. 10811 +--------------+------------------------------------------+ 10812 | Name | Description | 10813 +--------------+------------------------------------------+ 10814 | RFC3720 | RFC 3720-compliant semantics. Response | 10815 | | fencing is not guaranteed and fast | 10816 | | completion of multi-task aborting is not | 10817 | | supported | 10818 +--------------+------------------------------------------+ 10819 | ResponseFence| Response Fence (Section 4.2.2.3.3) | 10820 | | semantics MUST be supported in reporting | 10821 | | task completions | 10822 +--------------+------------------------------------------+ 10823 | FastAbort | Updated fast multi-task abort semantics | 10824 | | defined in Section 4.2.3.4 MUST be | 10825 | | supported. Support for Response Fence is | 10826 | | implied -- i.e., (Section 4.2.2.3.3) | 10827 | | semantics MUST be supported as well | 10828 +--------------+------------------------------------------+ 10829 When TaskReporting is not negotiated to FastAbort, the standard 10830 multi-task abort semantics in Section 4.2.3.3 MUST be used. 10832 13.24. iSCSIProtocolLevel Negotiation 10834 The iSCSIProtocolLevel associated with this document is "1". As a 10835 responder or an originator in a negotiation of this key, an iSCSI 10836 implementation compliant to this document alone, without any 10837 future protocol extensions, MUST use this value as defined by the 10838 [iSCSI-SAM] document. 10840 13.25. Obsoleted Keys 10842 This document obsoletes the following keys defined in [RFC3720]: 10843 IFMarker, OFMarker, OFMarkInt, IFMarkInt. However, iSCSI 10844 implementations compliant to this document may still receive these 10845 obsoleted keys - i.e. in a responder role - in a text negotiation. 10847 When IFMarker or OFMarker key is received, a compliant iSCSI 10848 implementation SHOULD respond with the constant "Reject" value. 10849 The implementation MAY alternatively respond with a "No" value. 10851 However, the implementation MUST NOT respond with a 10852 "NotUnderstood" value for either of these keys. 10854 When IFMarkInt or OFMarkInt key is received, a compliant iSCSI 10855 implementation MUST respond with the constant "Reject" value. 10856 The implementation MUST NOT respond with a "NotUnderstood" value 10857 for either of these keys. 10859 13.26. X#NodeArchitecture 10861 13.26.1. Definition 10863 Use: LO, Declarative 10864 Senders: Initiator and Target 10865 Scope: SW 10867 X#NodeArchitecture= 10869 Default is none. 10871 Examples: 10872 X#NodeArchitecture=ExampleOS/v1234,ExampleInc_SW_Initiator/1.05a 10873 X#NodeArchitecture=ExampleInc_HW_Initiator/4010,Firmware/2.0.0.5 10874 X#NodeArchitecture=ExampleInc_SW_Initiator/2.1,CPU_Arch/i686 10876 This document does not define the structure or content of the list 10877 of values. 10879 The initiator or target declares the details of its iSCSI node 10880 architecture to the remote endpoint. These details may include, 10881 but are not limited to, iSCSI vendor software, firmware, or 10882 hardware versions, the OS version, or hardware architecture. This 10883 key may be declared on a Discovery session or a Normal session. 10885 The length of the key value (total length of the list-of-values) 10886 MUST NOT be greater than 255 bytes. 10888 X#NodeArchitecture MUST NOT be redeclared during the login phase. 10890 13.26.2. Implementation Requirements 10892 Functional behavior of the iSCSI node (this includes the iSCSI 10893 protocol logic -- the SCSI, iSCSI, and TCP/IP protocols) MUST NOT 10894 depend on the presence, absence, or content of the 10895 X#NodeArchitecture key. The key MUST NOT be used by iSCSI nodes 10896 for interoperability, or exclusion of other nodes. To ensure 10897 proper use, key values SHOULD be set by the node itself, and there 10898 SHOULD NOT be provisions for the key values to contain user- 10899 defined text. 10901 Nodes implementing this key MUST choose one of the following 10902 implementation options: 10904 - only transmit the key, 10906 - only log the key values received from other nodes, or 10908 - both transmit and log the key values. 10910 Each node choosing to implement transmission of the key values 10911 MUST be prepared to handle the response of iSCSI Nodes that do not 10912 understand the key. 10914 Nodes that implement transmission and/or logging of the key values 10915 may also implement administrative mechanisms that disable and/or 10916 change the logging and key transmission detail (see Section 9.4). 10917 Thus, a valid behavior for this key may be that a node is 10918 completely silent (the node does not transmit any key value, and 10919 simply discards any key values it receives without issuing a 10920 NotUnderstood response). 10922 14. Rationale for revised IANA Considerations 10924 This document makes rather significant changes in this area, and 10925 this Section outlines the reasons behind the changes. As 10926 previously specified in [RFC3720], iSCSI had used text string 10927 prefixes, such as X- and X#, to distinguish extended login/text 10928 keys, digest algorithms and authentication methods from their 10929 standardized counterparts. Based on experience with other 10930 protocols, [RFC6648] however strongly recommends against this 10931 practice, in large part because extensions that use such prefixes 10932 may become standard over time at which point it can be infeasible 10933 to change their text string names due to widespread usage under 10934 the existing text string name. 10936 iSCSI's experience with public extensions supports the 10937 recommendations in [RFC6648], as the only extension item ever 10938 registered with IANA, the X#NodeArchitecture key, was specified as 10939 a standard key in a standards-track RFC [RFC4850], and hence did 10940 not require the X# prefix. In addition, that key is the only 10941 public iSCSI extension that has been registered with IANA since 10942 RFC 3720 was originally published, so there has been effectively 10943 no use of the X#, Y# and Z# public extension formats. 10945 Therefore, this document makes the following changes to the IANA 10946 registration procedures for iSCSI: 10948 (1) The separate registries for X#, Y# and Z# public 10949 extensions are removed. The single entry in the registry for 10950 X# login/text keys(X#NodeArchitecture) is transferred to the 10951 main login/text key registry. IANA has never created the 10952 latter two registries because there have been no 10953 registration requests for them. These public extension 10954 formats (X#, Y#, Z#) MUST NOT be used with the exception of 10955 the existing X#NodeArchitecture key. 10957 (2) The Registration Procedures for the main login/text key, 10958 digest algorithm and authentication method IANA registries 10959 are changed to IETF Review [RFC5226] for possible future 10960 extensions to iSCSI. This change includes a deliberate 10961 decision to remove the possibility of specifying an IANA- 10962 registered iSCSI extension in an RFC published via an RFC 10964 Editor independent submission, as the level of review in 10965 that process is insufficient for iSCSI extensions. 10967 (3) The restriction against registering items using the 10968 private extension formats (X-, Y-, Z-) in the main IANA 10969 registries is removed. Extensions using these formats MAY be 10970 registered under the IETF Review registration procedures, 10971 but each format is restricted to the type of extension for 10972 which it is specified in this RFC and MUST NOT be used for 10973 other types. For example, the X- extension format for 10974 extension login/text keys MUST NOT be used for digest 10975 algorithms or authentication methods. 10977 15. IANA Considerations 10979 The well-known TCP port number for iSCSI connections assigned by 10980 IANA is 3260 and this is the default iSCSI port. Implementations 10981 needing a system TCP port number may use port 860, the port 10982 assigned by IANA as the iSCSI system port; however in order to use 10983 port 860, it MUST be explicitly specified - implementations MUST 10984 NOT default to use of port 860, as 3260 is the only allowed 10985 default. 10987 IANA is requested to update all references to RFC 3720, RFC 4850 10988 and RFC 5048 to instead reference this RFC in all of the iSCSI 10989 registries that are part of the "Internet Small Computer System 10990 Interface (iSCSI) Parameters" set of registries. This change 10991 reflects the fact that those three RFCs are obsoleted by this RFC. 10992 References to other RFCs that are not being obsoleted (e.g., RFC 10993 3723, RFC 5046) should not be changed. 10995 IANA is requested to perform the following actions on the iSCSI 10996 Login/Text Keys registry: 10998 - Change the registration procedure to IETF Review from 10999 Standard Required. 11001 - Change the RFC 5048 reference for the registry to reference 11002 this RFC. 11004 - Add the X#NodeArchitecture Key from the iSCSI extended key 11005 registry and change its reference to this RFC. 11007 - Change all references of RFC 3720 and RFC 5048 to reference 11008 this RFC. 11010 IANA is requested to change the Registration Procedures for the 11011 iSCSI authentication methods and iSCSI digests registries to IETF 11012 Review from RFC Required. 11014 IANA is requested to remove the iSCSI extended key registry, as 11015 its one entry is to be added to the iSCSI login/text keys 11016 registry. 11018 IANA is requested to mark obsolete the values 4 and 5, for SPKM1 11019 and SPKM2 respectively, in the iSCSI authentication methods 11020 subregistry of the Internet Small Computer System Interface 11021 (iSCSI) Parameters registries. 11023 All the other IANA considerations stated in [RFC3720] and 11024 [RFC5048] remain unchanged. 11026 This document obsoletes the SPKM1 and SPKM2 key values for the 11027 AuthMethod text key. Consequently, the SPKM_ text key prefix MUST 11028 be treated as obsolete and be not reused. 11030 References 11032 Normative References 11034 [EUI] "Guidelines for 64-bit Global Identifier (EUI-64)", 11035 http://standards.ieee.org/regauth/oui/tutorials/EUI64.html 11037 [FC-FS3] INCITS 470-2011, Fibre Channel - Framing and 11038 Signaling - 3 (FC-FS-3). 11040 [IPSEC-IPS] Black, D., Koning, P., "IP Storage: IPsec 11041 Requirements Update for IPsec v3", draft-ietf-storm-ipsec- 11042 ips-update-01.txt (work in progress), June 2013 11044 [iSCSI-SAM] Knight, F., Chadalapaka, M., "Internet Small 11045 Computer Systems Interface (iSCSI) SCSI Architecture 11046 Features Update", draft-ietf-storm-iscsi-sam-06.txt (work in 11047 progress), July 2012 11049 [OUI] "IEEE OUI and Company_Id Assignments", 11050 http://standards.ieee.org/regauth/oui 11052 [RFC1122] R.Braden, "Requirements for Internet Hosts -- 11053 Communication Layers", October 1989. 11055 [RFC1964] J. Linn, "The Kerberos Version 5 GSS-API Mechanism", 11056 June 1996. 11058 [RFC1982] Elz, R., Bush, R., "Serial Number Arithmetic", 11059 August 1996. 11061 [RFC1994] W. Simpson, "PPP Challenge Handshake Authentication 11062 Protocol (CHAP)", August 1996. 11064 [RFC2119] Bradner, S. "Key Words for use in RFCs to Indicate 11065 Requirement Levels", BCP 14, March 1997. 11067 [RFC2404] C. Madson, R. Glenn, "The Use of HMAC-SHA-1-96 11068 within ESP and AH", November 1998. 11070 [RFC2451] R. Pereira, R. Adams "The ESP CBC-Mode Cipher 11071 Algorithms". 11073 [RFC2945] Wu, T., "The SRP Authentication and Key Exchange 11074 System", September 2000. 11076 [RFC3454] Hoffman, P. and M. Blanchet, "Preparation of 11077 Internationalized Strings ("stringprep")", RFC 3454, 11078 December 2002. 11080 [RFC3566] Frankel, S. and H. Herbert, "The AES-XCBC-MAC-96 11081 Algorithm and Its Use With IPsec", RFC 3566, September 2003. 11083 [RFC3629] Yergeau, F., "UTF-8, a Transformation Format of ISO 11084 10646", RFC 3629, November 2003 11086 [RFC3686] Housley, R., "Using Advanced Encryption Standard 11087 (AES) Counter Mode with IPsec Encapsulating Security Payload 11088 (ESP)", RFC 3686, January 2004. 11090 [RFC3722] Bakke, M., "String Profile for Internet Small 11091 Computer Systems Interface (iSCSI) Names", RFC 3722, March 11092 2004. 11094 [RFC3723] Aboba, B., Tseng, J., Walker, J., Rangan, V. and F. 11095 Travostino, "Securing Block Storage Protocols over IP", RFC 11096 3723, March 2004. 11098 [RFC3986] T. Berners-Lee, R. Fielding, L. Masinter "Uniform 11099 Resource Identifier (URI): Generic Syntax", January 2005. 11101 [RFC4106] J. Viega, D. McGrew, "The Use of Galois/Counter Mode 11102 (GCM) in IPsec Encapsulating Security Payload (ESP)", RFC 11103 4106, June 2005. 11105 [RFC4120] Neuman, C., Yu, T., Hartman, S., Raeburn, K, "The 11106 Kerberos Network Authentication Service (V5)", RFC 4120, 11107 July 2005. 11109 [RFC4171] J. Tseng, K. Gibbons, F. Travostino, C. Du Laney, J. 11110 Souza, "Internet Storage Name Service (iSNS)", September 11111 2005. 11113 [RFC4291] Hinden, R. and S. Deering, "IP Version 6 Addressing 11114 Architecture", February 2006. 11116 [RFC4301] S. Kent, K.Seo, "Security Architecture for the 11117 Internet Protocol", December 2005. 11119 [RFC4303] Kent, S., "IP Encapsulating Security Payload (ESP)", 11120 RFC 4303, December 2005 11122 [RFC4543] D. McGrew, J. Viega, "The Use of Galois Message 11123 Authentication Code (GMAC) in IPsec ESP and AH", RFC 4543, 11124 May 2006 11126 [RFC4648] S. Josefsson, "The Base16, Base32, and Base64 Data 11127 Encodings", RFC 4648, October 2006 11129 [RFC5226] H. Alvestrand, T. Narten, "Guidelines for Writing an 11130 IANA Considerations Section in RFCs", RFC 5226, May 2008 11132 [RFC5996] C. Kaufman, P. Hoffman, Y. Nir, P. Eronen, "Internet 11133 Key Exchange Protocol Version 2 (IKEv2) ", RFC 5996, 11134 September 2010. 11136 [SAM2] T10/1157-D, SCSI Architecture Model - 2 (SAM-2). 11138 [SAM4] T10/1683-D, SCSI Architecture Model - 4 (SAM-4). 11140 [SPC2] T10/1236-D, SCSI Primary Commands-2. 11142 [SPC3] T10/1416-D, SCSI Primary Commands-3. 11144 [UML] ISO/IEC 19501, Unified Modeling Language 11145 Specification Version 1.4.2. 11147 [UNICODE] Unicode Standard Annex #15, "Unicode Normalization 11148 Forms", http://www.unicode.org/unicode/reports/tr15 11150 Informative References 11152 [FC-SP-2] T11/1835-D, Fibre Channel Security Protocols- 2 (FC- 11153 SP-2). 11155 [RFC1737] K. Sollins, L. Masinter "Functional Requirements for 11156 Uniform Resource Names". 11158 [RFC5433] T. Clancy, H. Tschofenig "Extensible Authentication 11159 Protocol - Generalized Pre-Shared Key (EAP-GPSK) Method", 11160 RFC 5433, February 2009. 11162 [IB] InfiniBand{tm} Architecture Specification, Vol. 1, 11163 Rel.1.0.a, InfiniBand Trade 11164 Association(http://www.infinibandta.org). 11166 [Castagnoli93] G. Castagnoli, S. Braeuer and M. Herrman 11167 "Optimization of Cyclic Redundancy-Check Codes with 24 and 11168 32 Parity Bits", IEEE Transact. on Communications, Vol. 41, 11169 No. 6, June 1993. 11171 [CRC] ISO 3309, High-Level Data Link Control (CRC 32). 11173 [RFC2401] S. Kent, R. Atkinson, "Security Architecture for the 11174 Internet Protocol ", November 1998. 11176 [RFC2406] S. Kent, R. Atkinson, "IP Encapsulating Security 11177 Payload (ESP)", November 1998. 11179 [RFC2407] D. Piper, "The Internet IP Security Domain of 11180 Interpretation for ISAKMP", November 1998. 11182 [RFC2409] D. Harkins, D. Carrel, "The Internet Key Exchange 11183 (IKE)", November 1998. 11185 [RFC2608] E. Guttman, C. Perkins, J. Veizades, M. Day, 11186 "Service Location Protocol, Version 2", June 1999. 11188 [RFC2743] J.Linn, "Generic Security Service Application Program 11189 Interface Version 2, Update 1", January 2000. 11191 [RFC2865] C. Rigney, S. Willens, A. Rubens, W. Simpson, 11192 "Remote Authentication Dial In User Service (RADIUS)", June 11193 2000. 11195 [RFC3385] Sheinwald, D., Satran, J., Thaler, P. and V. 11196 Cavanna, "Internet Protocol Small Computer System Interface 11197 (iSCSI) Cyclic Redundancy Check (CRC)/Checksum 11198 Considerations", RFC 3385, September 2002. 11200 [RFC3602] S. Frankel, R. Glenn, S. Kelly, "The AES-CBC Cipher 11201 Algorithm and Its Use with IPsec", RFC 3602, September 2003. 11203 [RFC3720] Satran, J., Meth, K., Sapuntzakis, C., Chadalapaka, 11204 M., and E. Zeidner, "Internet Small Computer Systems 11205 Interface (iSCSI)", RFC 3720, April 2004. 11207 [RFC3721] Bakke, M., Hafner, J., Hufferd, J., Voruganti, K. 11208 and M. Krueger, "Internet Small Computer Systems Interface 11209 (iSCSI) Naming and Discovery", RFC 3721, March 2004 11211 [RFC3783] M. Chadalapaka, R. Elliott, "Small Computer Systems 11212 Interface (SCSI) Command Ordering Considerations with 11213 iSCSI", RFC 3783, May 2004. 11215 [RFC4121] L. Zhu, K. Jaganathan, S. Hartman, "The Kerberos 11216 Version 5 Generic Security Service Application Program 11217 Interface (GSS-API) Mechanism: Version 2", RFC 4121, July 11218 2005. 11220 [RFC4297] Romanow, A., Mogul, J., Talpey, T., and S. Bailey, 11221 "Remote Direct Memory Access (RDMA) over IP Problem 11222 Statement", RFC 4297, October 2004 11224 [RFC4850] Wysochanski, D., "Declarative Public Extension Key 11225 for Internet Small Computer Systems Interface (iSCSI) Node 11226 Architecture", RFC 4850, April 2007. 11228 [RFC5046] Ko, M., Chadalapaka, M., Hufferd, J., Elzur, U., 11229 Shah, H., and P. Thaler, "Internet Small Computer System 11230 Interface (iSCSI) Extensions for Remote Direct Memory Access 11231 (RDMA)", RFC 5046, October 2007 11233 [RFC5048] Chadalapaka, M., "Internet Small Computer Systems 11234 Interface (iSCSI) Corrections and Clarifications", RFC 5048, 11235 October 2007. 11237 [RFC6648] P. Saint-Andre, D. Crocker, M. Nottingham, 11238 "Deprecating the X- Prefix and Similar Constructs in 11239 Application Protocols", RFC 6648, June 2012 11241 [SAS] T10/2125-D, Serial Attached SCSI - 2.1 (SAS-2.1); 11242 T10/2124-D, SAS Protocol Layer (SPL); T10/2124-M, SAS 11243 Protocol Layer (SPL) Amendment #1 (SPL AM1). 11245 [SBC2] NCITS.405-205, SCSI Block Commands - 2 (SBC-2). 11247 [SPC4] T10/1731-D, SCSI Primary Commands-4. 11249 Appendix A. Examples 11251 Read Operation Example 11253 +------------------+-----------------------+---------------------+ 11254 |Initiator Function| PDU Type | Target Function | 11255 +------------------+-----------------------+---------------------+ 11256 | Command request |SCSI Command (READ)>>> | | 11257 | (read) | | | 11258 +------------------+-----------------------+---------------------+ 11259 | | |Prepare Data Transfer| 11260 +------------------+-----------------------+---------------------+ 11261 | Receive Data | <<< SCSI Data-in | Send Data | 11262 +------------------+-----------------------+---------------------+ 11263 | Receive Data | <<< SCSI Data-in | Send Data | 11264 +------------------+-----------------------+---------------------+ 11265 | Receive Data | <<< SCSI Data-in | Send Data | 11266 +------------------+-----------------------+---------------------+ 11267 | | <<< SCSI Response |Send Status and Sense| 11268 +------------------+-----------------------+---------------------+ 11269 | Command Complete | | | 11270 +------------------+-----------------------+---------------------+ 11272 Write Operation Example 11274 +------------------+-----------------------+---------------------+ 11275 |Initiator Function| PDU Type | Target Function | 11276 +------------------+-----------------------+---------------------+ 11277 | Command request |SCSI Command (WRITE)>>>| Receive command | 11278 | (write) | | and queue it | 11279 +------------------+-----------------------+---------------------+ 11280 | | | Process old commands| 11281 +------------------+-----------------------+---------------------+ 11282 | | | Ready to process | 11283 | | <<< R2T | WRITE command | 11284 +------------------+-----------------------+---------------------+ 11285 | Send Data | SCSI Data-out >>> | Receive Data | 11286 +------------------+-----------------------+---------------------+ 11287 | | <<< R2T | Ready for data | 11288 +------------------+-----------------------+---------------------+ 11289 | | <<< R2T | Ready for data | 11290 +------------------+-----------------------+---------------------+ 11291 | Send Data | SCSI Data-out >>> | Receive Data | 11292 +------------------+-----------------------+---------------------+ 11293 | Send Data | SCSI Data-out >>> | Receive Data | 11294 +------------------+-----------------------+---------------------+ 11295 | | <<< SCSI Response |Send Status and Sense| 11296 +------------------+-----------------------+---------------------+ 11297 | Command Complete | | | 11298 +------------------+-----------------------+---------------------+ 11300 R2TSN/DataSN Use Examples 11302 Output (write) data DataSN/R2TSN Example 11303 +------------------+-----------------------+---------------------+ 11304 |Initiator Function| PDU Type & Content | Target Function | 11305 +------------------+-----------------------+---------------------+ 11306 | Command request |SCSI Command (WRITE)>>>| Receive command | 11307 | (write) | | and queue it | 11308 +------------------+-----------------------+---------------------+ 11309 | | | Process old commands| 11310 +------------------+-----------------------+---------------------+ 11311 | | <<< R2T | Ready for data | 11312 | | R2TSN = 0 | | 11313 +------------------+-----------------------+---------------------+ 11314 | | <<< R2T | Ready for more data | 11315 | | R2TSN = 1 | | 11316 +------------------+-----------------------+---------------------+ 11317 | Send Data | SCSI Data-out >>> | Receive Data | 11318 | for R2TSN 0 | DataSN = 0, F=0 | | 11319 +------------------+-----------------------+---------------------+ 11320 | Send Data | SCSI Data-out >>> | Receive Data | 11321 | for R2TSN 0 | DataSN = 1, F=1 | | 11322 +------------------+-----------------------+---------------------+ 11323 | Send Data | SCSI Data >>> | Receive Data | 11324 | for R2TSN 1 | DataSN = 0, F=1 | | 11325 +------------------+-----------------------+---------------------+ 11326 | | <<< SCSI Response |Send Status and Sense| 11327 | | ExpDataSN = 0 | | 11328 +------------------+-----------------------+---------------------+ 11329 | Command Complete | | | 11330 +------------------+-----------------------+---------------------+ 11332 Input (read) data DataSN Example 11334 +------------------+-----------------------+---------------------+ 11335 |Initiator Function| PDU Type | Target Function | 11336 +------------------+-----------------------+---------------------+ 11337 | Command request |SCSI Command (READ)>>> | | 11338 | (read) | | | 11339 +------------------+-----------------------+---------------------+ 11340 | | |Prepare Data Transfer| 11341 +------------------+-----------------------+---------------------+ 11342 | Receive Data | <<< SCSI Data-in | Send Data | 11343 | | DataSN = 0, F=0 | | 11344 +------------------+-----------------------+---------------------+ 11345 | Receive Data | <<< SCSI Data-in | Send Data | 11346 | | DataSN = 1, F=0 | | 11347 +------------------+-----------------------+---------------------+ 11348 | Receive Data | <<< SCSI Data-in | Send Data | 11349 | | DataSN = 2, F=1 | | 11350 +------------------+-----------------------+---------------------+ 11351 | | <<< SCSI Response |Send Status and Sense| 11352 | | ExpDataSN = 3 | | 11353 +------------------+-----------------------+---------------------+ 11354 | Command Complete | | | 11355 +------------------+-----------------------+---------------------+ 11357 Bidirectional DataSN Example 11359 +------------------+-----------------------+---------------------+ 11360 |Initiator Function| PDU Type | Target Function | 11361 +------------------+-----------------------+---------------------+ 11362 | Command request |SCSI Command >>> | | 11363 | (Read-Write) | Read-Write | | 11364 +------------------+-----------------------+---------------------+ 11365 | | | Process old commands| 11366 +------------------+-----------------------+---------------------+ 11367 | | <<< R2T | Ready to process | 11368 | | R2TSN = 0 | WRITE command | 11369 +------------------+-----------------------+---------------------+ 11370 | * Receive Data | <<< SCSI Data-in | Send Data | 11371 | | DataSN = 0, F=0 | | 11372 +------------------+-----------------------+---------------------+ 11373 | * Receive Data | <<< SCSI Data-in | Send Data | 11374 | | DataSN = 1, F=1 | | 11375 +------------------+-----------------------+---------------------+ 11376 | * Send Data | SCSI Data-out >>> | Receive Data | 11377 | for R2TSN 0 | DataSN = 0, F=1 | | 11378 +------------------+-----------------------+---------------------+ 11379 | | <<< SCSI Response |Send Status and Sense| 11380 | | ExpDataSN = 2 | | 11381 +------------------+-----------------------+---------------------+ 11382 | Command Complete | | | 11383 +------------------+-----------------------+---------------------+ 11385 *) Send data and Receive Data may be transferred simultaneously as 11386 in an atomic Read-Old-Write-New or sequentially as in an atomic 11387 Read-Update-Write (in the latter case the R2T may follow the 11388 received data). 11390 Unsolicited and immediate output (write) data with DataSN Example 11391 +------------------+-----------------------+---------------------+ 11392 |Initiator Function| PDU Type & Content | Target Function | 11393 +------------------+-----------------------+---------------------+ 11394 | Command request |SCSI Command (WRITE)>>>| Receive command | 11395 | (write) |F=0 | and data | 11396 |+ immediate data | | and queue it | 11397 +------------------+-----------------------+---------------------+ 11398 | Send Unsolicited | SCSI Write Data >>> | Receive more Data | 11399 | Data | DataSN = 0, F=1 | | 11400 +------------------+-----------------------+---------------------+ 11401 | | | Process old commands| 11402 +------------------+-----------------------+---------------------+ 11403 | | <<< R2T | Ready for more data | 11404 | | R2TSN = 0 | | 11405 +------------------+-----------------------+---------------------+ 11406 | Send Data | SCSI Write Data >>> | Receive Data | 11407 | for R2TSN 0 | DataSN = 0, F=1 | | 11408 +------------------+-----------------------+---------------------+ 11409 | | <<< SCSI Response |Send Status and Sense| 11410 | | | | 11411 +------------------+-----------------------+---------------------+ 11412 | Command Complete | | | 11413 +------------------+-----------------------+---------------------+ 11415 CRC Examples 11417 N.B. all Values are Hexadecimal 11419 32 bytes of zeroes: 11421 Byte: 0 1 2 3 11423 0: 00 00 00 00 11424 ... 11425 28: 00 00 00 00 11427 CRC: aa 36 91 8a 11429 32 bytes of ones: 11431 Byte: 0 1 2 3 11432 0: ff ff ff ff 11433 ... 11434 28: ff ff ff ff 11436 CRC: 43 ab a8 62 11438 32 bytes of incrementing 00..1f: 11440 Byte: 0 1 2 3 11442 0: 00 01 02 03 11443 ... 11444 28: 1c 1d 1e 1f 11446 CRC: 4e 79 dd 46 11448 32 bytes of decrementing 1f..00: 11450 Byte: 0 1 2 3 11452 0: 1f 1e 1d 1c 11453 ... 11454 28: 03 02 01 00 11456 CRC: 5c db 3f 11 11458 An iSCSI - SCSI Read (10) Command PDU 11460 Byte: 0 1 2 3 11462 0: 01 c0 00 00 11463 4: 00 00 00 00 11464 8: 00 00 00 00 11465 12: 00 00 00 00 11466 16: 14 00 00 00 11467 20: 00 00 04 00 11468 24: 00 00 00 14 11469 28: 00 00 00 18 11470 32: 28 00 00 00 11471 36: 00 00 00 00 11472 40: 02 00 00 00 11473 44: 00 00 00 00 11475 CRC: 56 3a 96 d9 11477 Appendix B. Login Phase Examples 11479 In the first example, the initiator and target authenticate each 11480 other via Kerberos: 11482 I-> Login (CSG,NSG=0,1 T=1) 11483 InitiatorName=iqn.1999-07.com.os:hostid.77 11484 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11485 AuthMethod=KRB5,SRP,None 11487 T-> Login (CSG,NSG=0,0 T=0) 11488 AuthMethod=KRB5 11490 I-> Login (CSG,NSG=0,1 T=1) 11491 KRB_AP_REQ= 11493 (krb_ap_req contains the Kerberos V5 ticket and authenticator with 11494 MUTUAL-REQUIRED set in the ap-options field) 11496 If the authentication is successful, the target proceeds with: 11498 T-> Login (CSG,NSG=0,1 T=1) 11499 KRB_AP_REP= 11501 (krb_ap_rep is the Kerberos V5 mutual authentication reply) 11503 If the authentication is successful, the initiator may proceed 11504 with: 11506 I-> Login (CSG,NSG=1,0 T=0) FirstBurstLength=8192 11507 T-> Login (CSG,NSG=1,0 T=0) FirstBurstLength=4096 11508 MaxBurstLength=8192 11509 I-> Login (CSG,NSG=1,0 T=0) MaxBurstLength=8192 11510 ... more iSCSI Operational Parameters 11512 T-> Login (CSG,NSG=1,0 T=0) 11513 ... more iSCSI Operational Parameters 11515 And at the end: 11517 I-> Login (CSG,NSG=1,3 T=1) 11518 optional iSCSI parameters 11520 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11522 If the initiator's authentication by the target is not successful, 11523 the target responds with: 11525 T-> Login "login reject" 11527 instead of the Login KRB_AP_REP message, and terminates the 11528 connection. 11530 If the target's authentication by the initiator is not successful, 11531 the initiator terminates the connection (without responding to the 11532 Login KRB_AP_REP message). 11534 In the next example only the initiator is authenticated by the 11535 target via Kerberos: 11537 I-> Login (CSG,NSG=0,1 T=1) 11538 InitiatorName=iqn.1999-07.com.os:hostid.77 11539 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11540 AuthMethod=SRP,KRB5,None 11542 T-> Login-PR (CSG,NSG=0,0 T=0) 11543 AuthMethod=KRB5 11545 I-> Login (CSG,NSG=0,1 T=1) 11546 KRB_AP_REQ=krb_ap_req 11548 (MUTUAL-REQUIRED not set in the ap-options field of krb_ap_req) 11550 If the authentication is successful, the target proceeds with: 11552 T-> Login (CSG,NSG=0,1 T=1) 11554 I-> Login (CSG,NSG=1,0 T=0) 11555 ... iSCSI parameters 11557 T-> Login (CSG,NSG=1,0 T=0) 11558 ... iSCSI parameters 11560 . . . 11562 T-> Login (CSG,NSG=1,3 T=1)"login accept" 11564 In the next example, the initiator and target authenticate each 11565 other via SRP: 11567 I-> Login (CSG,NSG=0,1 T=1) 11568 InitiatorName=iqn.1999-07.com.os:hostid.77 11569 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11570 AuthMethod=KRB5,SRP,None 11572 T-> Login-PR (CSG,NSG=0,0 T=0) 11573 AuthMethod=SRP 11575 I-> Login (CSG,NSG=0,0 T=0) 11576 SRP_U= 11577 TargetAuth=Yes 11579 T-> Login (CSG,NSG=0,0 T=0) 11580 SRP_N= 11581 SRP_g= 11582 SRP_s= 11584 I-> Login (CSG,NSG=0,0 T=0) 11585 SRP_A= 11587 T-> Login (CSG,NSG=0,0 T=0) 11588 SRP_B= 11590 I-> Login (CSG,NSG=0,1 T=1) 11591 SRP_M= 11593 If the initiator authentication is successful, the target 11594 proceeds: 11596 T-> Login (CSG,NSG=0,1 T=1) 11597 SRP_HM= 11599 Where N, g, s, A, B, M, and H(A | M | K) are defined in 11600 [RFC2945]. 11602 If the target authentication is not successful, the initiator 11603 terminates the connection; otherwise, it proceeds. 11605 I-> Login (CSG,NSG=1,0 T=0) 11606 ... iSCSI parameters 11608 T-> Login (CSG,NSG=1,0 T=0) 11609 ... iSCSI parameters 11611 And at the end: 11613 I-> Login (CSG,NSG=1,3 T=1) 11614 optional iSCSI parameters 11616 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11618 If the initiator authentication is not successful, the target 11619 responds with: 11621 T-> Login "login reject" 11623 Instead of the T-> Login SRP_HM= message and 11624 terminates the connection. 11626 In the next example, only the initiator is authenticated by the 11627 target via SRP: 11629 I-> Login (CSG,NSG=0,1 T=1) 11630 InitiatorName=iqn.1999-07.com.os:hostid.77 11631 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11632 AuthMethod=KRB5,SRP,None 11634 T-> Login-PR (CSG,NSG=0,0 T=0) 11635 AuthMethod=SRP 11637 I-> Login (CSG,NSG=0,0 T=0) 11638 SRP_U= 11639 TargetAuth=No 11641 T-> Login (CSG,NSG=0,0 T=0) 11642 SRP_N= 11643 SRP_g= 11644 SRP_s= 11646 I-> Login (CSG,NSG=0,0 T=0) 11647 SRP_A= 11649 T-> Login (CSG,NSG=0,0 T=0) 11650 SRP_B= 11652 I-> Login (CSG,NSG=0,1 T=1) 11653 SRP_M= 11655 If the initiator authentication is successful, the target 11656 proceeds: 11658 T-> Login (CSG,NSG=0,1 T=1) 11660 I-> Login (CSG,NSG=1,0 T=0) 11662 ... iSCSI parameters 11664 T-> Login (CSG,NSG=1,0 T=0) 11666 ... iSCSI parameters 11668 And at the end: 11670 I-> Login (CSG,NSG=1,3 T=1) 11672 optional iSCSI parameters 11674 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11676 In the next example the initiator and target authenticate each 11677 other via CHAP: 11679 I-> Login (CSG,NSG=0,0 T=0) 11681 InitiatorName=iqn.1999-07.com.os:hostid.77 11683 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11685 AuthMethod=KRB5,CHAP,None 11687 T-> Login-PR (CSG,NSG=0,0 T=0) 11689 AuthMethod=CHAP 11691 I-> Login (CSG,NSG=0,0 T=0) 11693 CHAP_A= 11695 T-> Login (CSG,NSG=0,0 T=0) 11696 CHAP_A= 11697 CHAP_I= 11698 CHAP_C= 11700 I-> Login (CSG,NSG=0,1 T=1) 11701 CHAP_N= 11703 CHAP_R= 11705 CHAP_I= 11707 CHAP_C= 11709 If the initiator authentication is successful, the target 11710 proceeds: 11712 T-> Login (CSG,NSG=0,1 T=1) 11714 CHAP_N= 11716 CHAP_R= 11718 If the target authentication is not successful, the initiator 11719 aborts the connection; otherwise, it proceeds. 11721 I-> Login (CSG,NSG=1,0 T=0) 11723 ... iSCSI parameters 11725 T-> Login (CSG,NSG=1,0 T=0) 11727 ... iSCSI parameters 11729 And at the end: 11731 I-> Login (CSG,NSG=1,3 T=1) 11733 optional iSCSI parameters 11735 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11737 If the initiator authentication is not successful, the target 11738 responds with: 11740 T-> Login "login reject" 11742 Instead of the Login CHAP_R= "proceed and change 11743 stage" message and terminates the connection. 11745 In the next example, only the initiator is authenticated by the 11746 target via CHAP: 11748 I-> Login (CSG,NSG=0,1 T=0) 11750 InitiatorName=iqn.1999-07.com.os:hostid.77 11752 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11754 AuthMethod=KRB5,CHAP,None 11756 T-> Login-PR (CSG,NSG=0,0 T=0) 11758 AuthMethod=CHAP 11760 I-> Login (CSG,NSG=0,0 T=0) 11762 CHAP_A= 11764 T-> Login (CSG,NSG=0,0 T=0) 11765 CHAP_A= 11766 CHAP_I= 11767 CHAP_C= 11769 I-> Login (CSG,NSG=0,1 T=1) 11771 CHAP_N= 11773 CHAP_R= 11775 If the initiator authentication is successful, the target 11776 proceeds: 11778 T-> Login (CSG,NSG=0,1 T=1) 11780 I-> Login (CSG,NSG=1,0 T=0) 11782 ... iSCSI parameters 11784 T-> Login (CSG,NSG=1,0 T=0) 11786 ... iSCSI parameters 11788 And at the end: 11790 I-> Login (CSG,NSG=1,3 T=1) 11792 optional iSCSI parameters 11794 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11796 In the next example, the initiator does not offer any security 11797 parameters. It therefore may offer iSCSI parameters on the Login 11798 PDU with the T bit set to 1, and the target may respond with a 11799 final Login Response PDU immediately: 11801 I-> Login (CSG,NSG=1,3 T=1) 11803 InitiatorName=iqn.1999-07.com.os:hostid.77 11805 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11807 ... iSCSI parameters 11809 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11811 ... ISCSI parameters 11813 In the next example, the initiator does offer security 11814 parameters on the Login PDU, but the target does not choose 11815 any (i.e., chooses the "None" values): 11817 I-> Login (CSG,NSG=0,1 T=1) 11819 InitiatorName=iqn.1999-07.com.os:hostid.77 11821 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11823 AuthMethod=KRB5,SRP,None 11825 T-> Login-PR (CSG,NSG=0,1 T=1) 11827 AuthMethod=None 11829 I-> Login (CSG,NSG=1,0 T=0) 11831 ... iSCSI parameters 11833 T-> Login (CSG,NSG=1,0 T=0) 11835 ... iSCSI parameters 11837 And at the end: 11839 I-> Login (CSG,NSG=1,3 T=1) 11841 optional iSCSI parameters 11843 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11845 Appendix C. SendTargets Operation 11847 The text in this Appendix is a normative part of this document. 11849 To reduce the amount of configuration required on an initiator, 11850 iSCSI provides the SendTargets text request. The initiator uses 11851 the SendTargets request to get a list of targets to which it may 11852 have access, as well as the list of addresses (IP address and TCP 11853 port) on which these targets may be accessed. 11855 To make use of SendTargets, an initiator must first establish one 11856 of two types of sessions. If the initiator establishes 11857 the session using the key "SessionType=Discovery", the session is 11858 a discovery session, and a target name does not need to be 11859 specified. Otherwise, the session is a normal, operational 11860 session. The SendTargets command MUST only be sent during the 11861 Full Feature Phase of a normal or discovery session. 11863 A system that contains targets MUST support discovery sessions on 11864 each of its iSCSI IP address-port pairs, and MUST support the 11865 SendTargets command on the discovery session. In a discovery 11866 session, a target MUST return all path information (IP address- 11867 port pairs and portal group tags) for the targets on the target 11868 network entity which the requesting initiator is authorized to 11869 access. 11871 A target MUST support the SendTargets command on operational 11872 sessions; these will only return path information about the target 11873 to which the session is connected, and do not need to return 11874 information about other target names that may be defined in the 11875 responding system. 11877 An initiator MAY make use of the SendTargets as it sees fit. 11879 A SendTargets command consists of a single Text request PDU. 11880 This PDU contains exactly one text key and value. The text key 11881 MUST be SendTargets. The expected response depends upon the 11882 value, as well as whether the session is a discovery or 11883 operational session. 11885 The value must be one of: 11887 All 11889 The initiator is requesting that information on all relevant 11890 targets known to the implementation be returned. This value 11891 MUST be supported on a discovery session, and MUST NOT be 11892 supported on an operational session. 11894 11896 If an iSCSI target name is specified, the session should 11897 respond with addresses for only the named target, if 11898 possible. This value MUST be supported on discovery 11899 sessions. A discovery session MUST be capable of returning 11900 addresses for those targets that would have been returned 11901 had value=All been designated. 11903 11905 The session should only respond with addresses for the target 11906 to which the session is logged in. This MUST be supported 11907 on operational sessions, and MUST NOT return targets other 11908 than the one to which the session is logged in. 11910 The response to this command is a text response that contains a 11911 list of zero or more targets and, optionally, their addresses. 11912 Each target is returned as a target record. A target record 11913 begins with the TargetName text key, followed by a list of 11914 TargetAddress text keys, and bounded by the end of the text 11915 response or the next TargetName key, which begins a new record. 11916 No text keys other than TargetName and TargetAddress are permitted 11917 within a SendTargets response. 11919 For the format of the TargetName, see Section 13.4. 11921 A discovery session MAY respond to a SendTargets request with its 11922 complete list of targets, or with a list of targets that is based 11923 on the name of the initiator logged in to the session. 11925 A SendTargets response MUST NOT contain target names if there are 11926 no targets for the requesting initiator to access. 11928 Each target record returned includes zero or more TargetAddress 11929 fields. 11931 Each target record starts with one text key of the form: 11933 TargetName= 11935 Followed by zero or more address keys of the form: 11937 TargetAddress=[:], 11940 The hostname-or-ipaddress contains a domain name, IPv4 address, or 11941 IPv6 address ([RFC4291]), as specified for the TargetAddress key. 11943 A hostname-or-ipaddress duplicated in TargetAddress responses for 11944 a given node (the port is absent or equal) would probably indicate 11945 that multiple address families are in use at once (IPv6 and IPv4). 11947 Each TargetAddress belongs to a portal group, identified by its 11948 numeric portal group tag (as in Section 13.9). The iSCSI target 11949 name, together with this tag, constitutes the SCSI port 11950 identifier; the tag only needs to be unique within a given 11951 target's name list of addresses. 11953 Multiple-connection sessions can span iSCSI addresses that belong 11954 to the same portal group. 11956 Multiple-connection sessions cannot span iSCSI addresses that 11957 belong to different portal groups. 11959 If a SendTargets response reports an iSCSI address for a target, 11960 it SHOULD also report all other addresses in its portal group in 11961 the same response. 11963 A SendTargets text response can be longer than a single Text 11964 Response PDU, and makes use of the long text responses as 11965 specified. 11967 After obtaining a list of targets from the discovery target 11968 session, 11969 an iSCSI initiator may initiate new sessions to log in to the 11970 discovered targets for full operation. The initiator MAY keep the 11971 discovery session open, and MAY send subsequent SendTargets 11972 commands to discover new targets. 11974 Examples: 11976 This example is the SendTargets response from a single target that 11977 has no other interface ports. 11979 Initiator sends text request that contains: 11981 SendTargets=All 11983 Target sends a text response that contains: 11985 TargetName=iqn.1993-11.com.example:diskarray.sn.8675309 11987 All the target had to return in the simple case was the target 11988 name. It is assumed by the initiator that the IP address and TCP 11989 port for this target are the same as used on the current 11990 connection to the default iSCSI target. 11992 The next example has two internal iSCSI targets, each accessible 11993 via two different ports with different IP addresses. The 11994 following is the text response: 11996 TargetName=iqn.1993-11.com.example:diskarray.sn.8675309 11998 TargetAddress=10.1.0.45:3000,1 12000 TargetAddress=10.1.1.45:3000,2 12002 TargetName=iqn.1993-11.com.example:diskarray.sn.1234567 12004 TargetAddress=10.1.0.45:3000,1 12006 TargetAddress=10.1.1.45:3000,2 12008 Both targets share both addresses; the multiple addresses are 12009 likely used to provide multi-path support. The initiator may 12010 connect to either target name on either address. Each of the 12011 addresses has its own portal group tag; they do not support 12012 spanning multiple-connection sessions with each other. Keep in 12013 mind that the portal group tags for the two named targets are 12014 independent of one another; portal group "1" on the first target 12015 is not necessarily the same as portal group "1" on the second 12016 target. 12018 In the above example, a DNS host name or an IPv6 address could 12019 have been returned instead of an IPv4 address. 12021 The next text response shows a target that supports spanning 12022 sessions across multiple addresses, and further illustrates the 12023 use of the portal group tags: 12025 TargetName=iqn.1993-11.com.example:diskarray.sn.8675309 12027 TargetAddress=10.1.0.45:3000,1 12029 TargetAddress=10.1.1.46:3000,1 12031 TargetAddress=10.1.0.47:3000,2 12033 TargetAddress=10.1.1.48:3000,2 12035 TargetAddress=10.1.1.49:3000,3 12037 In this example, any of the target addresses can be used to reach 12038 the same target. A single-connection session can be established 12039 to any of these TCP addresses. A multiple-connection session 12040 could span addresses .45 and .46 or .47 and .48, but cannot span 12041 any other combination. A TargetAddress with its own tag (.49) 12042 cannot be combined with any other address within the same session. 12044 This SendTargets response does not indicate whether .49 supports 12045 multiple connections per session; it is communicated via the 12046 MaxConnections text key upon login to the target. 12048 Appendix D. Algorithmic Presentation of Error Recovery Classes 12050 This Appendix illustrates the error recovery classes using a 12051 pseudo-programming-language. The procedure names are chosen to be 12052 obvious to most implementers. Each of the recovery classes 12053 described has initiator procedures as well as target procedures. 12054 These algorithms focus on outlining the mechanics of error 12055 recovery classes, and do not exhaustively describe all other 12056 aspects/cases. Examples of this approach are: 12058 - Handling for only certain Opcode types is shown. 12060 - Only certain reason codes (e.g., Recovery in Logout command) 12061 are outlined. 12063 - Resultant cases, such as recovery of Synchronization on a 12064 header digest error are considered out-of-scope in these 12065 algorithms. In this particular example, a header digest 12066 error may lead to connection recovery if some type of sync 12067 and steering layer is not implemented. 12069 These algorithms strive to convey the iSCSI error recovery 12070 concepts in the simplest terms, and are not designed to be 12071 optimal. 12073 D.1. General Data Structure and Procedure Description 12075 This Section defines the procedures and data structures that are 12076 commonly used by all the error recovery algorithms. The structures 12077 may not be the exhaustive representations of what is required for 12078 a typical implementation. 12080 Data structure definitions - 12081 struct TransferContext { 12082 int TargetTransferTag; 12083 int ExpectedDataSN; 12084 }; 12086 struct TCB { /* task control block */ 12087 Boolean SoFarInOrder; 12088 int ExpectedDataSN; /* used for both R2Ts, and Data */ 12089 int MissingDataSNList[MaxMissingDPDU]; 12090 Boolean FbitReceived; 12091 Boolean StatusXferd; 12092 Boolean CurrentlyAllegiant; 12093 int ActiveR2Ts; 12094 int Response; 12095 char *Reason; 12096 struct TransferContext 12097 TransferContextList[MaxOutStandingR2T]; 12098 int InitiatorTaskTag; 12099 int CmdSN; 12100 int SNACK_Tag; 12101 }; 12103 struct Connection { 12104 struct Session SessionReference; 12105 Boolean SoFarInOrder; 12106 int CID; 12107 int State; 12108 int CurrentTimeout; 12109 int ExpectedStatSN; 12110 int MissingStatSNList[MaxMissingSPDU]; 12111 Boolean PerformConnectionCleanup; 12112 }; 12114 struct Session { 12115 int NumConnections; 12116 int CmdSN; 12117 int Maxconnections; 12118 int ErrorRecoveryLevel; 12119 struct iSCSIEndpoint OtherEndInfo; 12120 struct Connection ConnectionList[MaxSupportedConns]; 12121 }; 12123 Procedure descriptions - 12124 Receive-a-In-PDU(transport connection, inbound PDU); 12125 check-basic-validity(inbound PDU); 12126 Start-Timer(timeout handler, argument, timeout value); 12127 Build-And-Send-Reject(transport connection, bad PDU, reason 12128 code); 12130 D.2. Within-command Error Recovery Algorithms 12132 D.2.1. Procedure Descriptions 12134 Recover-Data-if-Possible(last required DataSN, task control 12135 block); 12136 Build-And-Send-DSnack(task control block); 12137 Build-And-Send-RDSnack(task control block); 12138 Build-And-Send-Abort(task control block); 12139 SCSI-Task-Completion(task control block); 12140 Build-And-Send-A-Data-Burst(transport connection, data- 12141 descriptor, 12142 task control 12143 block); 12144 Build-And-Send-R2T(transport connection, data-descriptor, 12145 task control 12146 block); 12147 Build-And-Send-Status(transport connection, task control block); 12148 Transfer-Context-Timeout-Handler(transfer context); 12150 Notes: 12152 - One procedure used in this Section: Handle-Status-SNACK- 12153 request is defined in Within-connection recovery algorithms. 12155 - The Response processing pseudo-code, shown in the target 12156 algorithms, applies to all solicited PDUs that carry StatSN 12157 - SCSI Response, Text Response etc. 12159 D.2.2. Initiator Algorithms 12161 Recover-Data-if-Possible(LastRequiredDataSN, TCB) 12162 { 12163 if (operational ErrorRecoveryLevel > 0) { 12164 if (# of missing PDUs is trackable) { 12165 Note the missing DataSNs in TCB. 12166 if (the task spanned a change in 12167 MaxRecvDataSegmentLength) { 12168 if (TCB.StatusXferd is TRUE) 12169 drop the status PDU; 12170 Build-And-Send-RDSnack(TCB); 12171 } else { 12172 Build-And-Send-DSnack(TCB); 12173 } 12175 } else { 12176 TCB.Reason = "Protocol service CRC error"; 12177 } 12178 } else { 12179 TCB.Reason = "Protocol service CRC error"; 12180 } 12181 if (TCB.Reason == "Protocol service CRC error") { 12182 Clear the missing PDU list in the TCB. 12183 if (TCB.StatusXferd is not TRUE) 12184 Build-And-Send-Abort(TCB); 12185 } 12186 } 12188 Receive-a-In-PDU(Connection, CurrentPDU) 12189 { 12190 check-basic-validity(CurrentPDU); 12191 if (Header-Digest-Bad) discard, return; 12192 Retrieve TCB for CurrentPDU.InitiatorTaskTag. 12193 if ((CurrentPDU.type == Data) 12194 or (CurrentPDU.type = R2T)) { 12195 if (Data-Digest-Bad for Data) { 12196 send-data-SNACK = TRUE; 12197 LastRequiredDataSN = CurrentPDU.DataSN; 12198 } else { 12199 if (TCB.SoFarInOrder = TRUE) { 12200 if (current DataSN is expected) { 12201 Increment TCB.ExpectedDataSN. 12202 } else { 12203 TCB.SoFarInOrder = FALSE; 12204 send-data-SNACK = TRUE; 12205 } 12206 } else { 12207 if (current DataSN was considered 12208 missing) { 12209 remove current DataSN from missing 12210 PDU list. 12211 } else if (current DataSN is higher than 12212 expected) { 12213 send-data-SNACK = TRUE; 12214 } else { 12215 discard, return; 12216 } 12217 Adjust TCB.ExpectedDataSN if 12218 appropriate. 12219 } 12220 LastRequiredDataSN = CurrentPDU.DataSN - 1; 12221 } 12222 if (send-data-SNACK is TRUE and 12223 task is not already considered failed) { 12224 Recover-Data-if-Possible(LastRequiredDataSN, TCB); 12225 } 12226 if (missing data PDU list is empty) { 12227 TCB.SoFarInOrder = TRUE; 12228 } 12229 if (CurrentPDU.type == R2T) { 12230 Increment ActiveR2Ts for this task. 12231 Create a data-descriptor for the data burst. 12232 Build-And-Send-A-Data-Burst(Connection, data- 12233 descriptor, 12234 TCB); 12235 } 12236 } else if (CurrentPDU.type == Response) { 12237 if (Data-Digest-Bad) { 12238 send-status-SNACK = TRUE; 12239 } else { 12240 TCB.StatusXferd = TRUE; 12241 Store the status information in TCB. 12242 if (ExpDataSN does not match) { 12243 TCB.SoFarInOrder = FALSE; 12244 Recover-Data-if-Possible(current DataSN, TCB); 12245 } 12246 if (missing data PDU list is empty) { 12247 TCB.SoFarInOrder = TRUE; 12248 } 12249 } 12250 } else { /* REST UNRELATED TO WITHIN-COMMAND-RECOVERY, NOT 12251 SHOWN */ 12252 } 12253 if ((TCB.SoFarInOrder == TRUE) and 12254 (TCB.StatusXferd == TRUE)) { 12256 SCSI-Task-Completion(TCB); 12257 } 12258 } 12260 D.2.3. Target Algorithms 12262 Receive-a-In-PDU(Connection, CurrentPDU) 12263 { 12264 check-basic-validity(CurrentPDU); 12265 if (Header-Digest-Bad) discard, return; 12266 Retrieve TCB for CurrentPDU.InitiatorTaskTag. 12267 if (CurrentPDU.type == Data) { 12268 Retrieve TContext from CurrentPDU.TargetTransferTag; 12269 if (Data-Digest-Bad) { 12270 Build-And-Send-Reject(Connection, CurrentPDU, 12271 Payload-Digest-Error); 12272 Note the missing data PDUs in MissingDataRange[]. 12273 send-recovery-R2T = TRUE; 12274 } else { 12275 if (current DataSN is not expected) { 12276 Note the missing data PDUs in MissingDataRange[]. 12277 send-recovery-R2T = TRUE; 12278 } 12279 if (CurrentPDU.Fbit == TRUE) { 12280 if (current PDU is solicited) { 12281 Decrement TCB.ActiveR2Ts. 12282 } 12283 if ((current PDU is unsolicited and 12284 data received is less than I/O length and 12285 data received is less than 12286 FirstBurstLength) 12287 or (current PDU is solicited and the length of 12288 this burst is less than expected)) { 12289 send-recovery-R2T = TRUE; 12290 Note the missing data in MissingDataRange[]. 12291 } 12292 } 12293 } 12294 Increment TContext.ExpectedDataSN. 12295 if (send-recovery-R2T is TRUE and 12296 task is not already considered failed) { 12297 if (operational ErrorRecoveryLevel > 0) { 12298 Increment TCB.ActiveR2Ts. 12299 Create a data-descriptor for the data burst 12300 from MissingDataRange. 12301 Build-And-Send-R2T(Connection, data-descriptor, 12302 TCB); 12303 } else { 12304 if (current PDU is the last unsolicited) 12305 TCB.Reason = "Not enough unsolicited data"; 12306 else 12307 TCB.Reason = "Protocol service CRC error"; 12308 } 12309 } 12310 if (TCB.ActiveR2Ts == 0) { 12311 Build-And-Send-Status(Connection, TCB); 12312 } 12313 } else if (CurrentPDU.type == SNACK) { 12314 snack-failure = FALSE; 12315 if (operational ErrorRecoveryLevel > 0) { 12316 if (CurrentPDU.type == Data/R2T) { 12317 if (the request is satisfiable) { 12318 if (request for Data) { 12319 Create a data-descriptor for the data burst 12320 from BegRun and RunLength. 12321 Build-And-Send-A-Data-Burst(Connection, 12322 data-descriptor, TCB); 12323 } else { /* R2T */ 12324 Create a data-descriptor for the data burst 12325 from BegRun and RunLength. 12326 Build-And-Send-R2T(Connection, data- 12327 descriptor, 12328 TCB); 12329 } 12330 } else { 12331 snack-failure = TRUE; 12332 } 12333 } else if (CurrentPDU.type == status) { 12334 Handle-Status-SNACK-request(Connection, 12335 CurrentPDU); 12336 } else if (CurrentPDU.type == DataACK) { 12337 Consider all data upto CurrentPDU.BegRun as 12338 acknowledged. 12339 Free up the retransmission resources for that data. 12340 } else if (CurrentPDU.type == R-Data SNACK) { 12341 Create a data descriptor for a data burst 12342 covering 12343 all unacknowledged data. 12344 Build-And-Send-A-Data-Burst(Connection, 12345 data-descriptor, TCB); 12346 TCB.SNACK_Tag = CurrentPDU.SNACK_Tag; 12347 if (there's no more data to send) { 12348 Build-And-Send-Status(Connection, TCB); 12349 } 12350 } 12351 } else { /* operational ErrorRecoveryLevel = 0 */ 12352 snack-failure = TRUE; 12353 } 12354 if (snack-failure == TRUE) { 12355 Build-And-Send-Reject(Connection, CurrentPDU, 12356 SNACK-Reject); 12357 if (TCB.StatusXferd != TRUE) { 12358 TCB.Reason = "SNACK Rejected"; 12359 Build-And-Send-Status(Connection, TCB); 12360 } 12361 } 12363 } else { /* REST UNRELATED TO WITHIN-COMMAND-RECOVERY, NOT 12364 SHOWN */ 12365 } 12366 } 12368 Transfer-Context-Timeout-Handler(TContext) 12369 { 12370 Retrieve TCB and Connection from TContext. 12371 Decrement TCB.ActiveR2Ts. 12372 if (operational ErrorRecoveryLevel > 0 and 12373 task is not already considered failed) { 12374 Note the missing data PDUs in MissingDataRange[]. 12375 Create a data-descriptor for the data burst 12376 from MissingDataRange[]. 12377 Build-And-Send-R2T(Connection, data-descriptor, TCB); 12379 } else { 12380 TCB.Reason = "Protocol service CRC error"; 12381 if (TCB.ActiveR2Ts = 0) { 12382 Build-And-Send-Status(Connection, TCB); 12383 } 12384 } 12385 } 12387 D.3. Within-connection Recovery Algorithms 12389 D.3.1. Procedure Descriptions 12391 Procedure descriptions: 12392 Recover-Status-if-Possible(transport connection, 12393 currently received PDU); 12394 Evaluate-a-StatSN(transport connection, currently received PDU); 12395 Retransmit-Command-if-Possible(transport connection, CmdSN); 12396 Build-And-Send-SSnack(transport connection); 12397 Build-And-Send-Command(transport connection, task control 12398 block); 12399 Command-Acknowledge-Timeout-Handler(task control block); 12400 Status-Expect-Timeout-Handler(transport connection); 12401 Build-And-Send-Nop-Out(transport connection); 12402 Handle-Status-SNACK-request(transport connection, status SNACK 12403 PDU); 12404 Retransmit-Status-Burst(status SNACK, task control block); 12405 Is-Acknowledged(beginning StatSN, run length); 12407 Implementation-specific tunables: 12408 InitiatorProactiveSNACKEnabled 12410 Notes: 12411 - The initiator algorithms only deal with unsolicited Nop-In 12412 PDUs for generating status SNACKs. A solicited Nop-In PDU 12413 has an assigned StatSN, which, when out of order, could 12414 trigger the out of order StatSN handling in Within-command 12415 algorithms, again leading to Recover-Status-if-Possible. 12417 - The pseudo-code shown may result in the retransmission of 12418 unacknowledged commands in more cases than necessary. This 12419 will not, however, affect the correctness of the operation 12420 because the target is required to discard the duplicate 12421 CmdSNs. 12423 - The procedure Build-And-Send-Async is defined in the 12424 Connection recovery algorithms. 12426 - The procedure Status-Expect-Timeout-Handler describes how 12427 initiators may proactively attempt to retrieve the Status if 12428 they so choose. This procedure is assumed to be triggered 12429 much before the standard ULP timeout. 12431 D.3.2. Initiator Algorithms 12433 Recover-Status-if-Possible(Connection, CurrentPDU) 12434 { 12435 if ((Connection.state == LOGGED_IN) and 12436 connection is not already considered 12437 failed) { 12438 if (operational ErrorRecoveryLevel > 0) { 12439 if (# of missing PDUs is trackable) { 12440 Note the missing StatSNs in Connection 12441 that were not already requested with SNACK; 12442 Build-And-Send-SSnack(Connection); 12443 } else { 12444 Connection.PerformConnectionCleanup = TRUE; 12445 } 12446 } else { 12447 Connection.PerformConnectionCleanup = TRUE; 12448 } 12449 if (Connection.PerformConnectionCleanup == TRUE) { 12450 Start-Timer(Connection-Cleanup-Handler, Connection, 12451 0); 12452 } 12453 } 12455 } 12457 Retransmit-Command-if-Possible(Connection, CmdSN) 12458 { 12459 if (operational ErrorRecoveryLevel > 0) { 12460 Retrieve the InitiatorTaskTag, and thus TCB for the 12461 CmdSN. 12462 Build-And-Send-Command(Connection, TCB); 12463 } 12464 } 12466 Evaluate-a-StatSN(Connection, CurrentPDU) 12467 { 12468 send-status-SNACK = FALSE; 12469 if (Connection.SoFarInOrder == TRUE) { 12470 if (current StatSN is the expected) { 12471 Increment Connection.ExpectedStatSN. 12472 } else { 12473 Connection.SoFarInOrder = FALSE; 12474 send-status-SNACK = TRUE; 12475 } 12476 } else { 12477 if (current StatSN was considered missing) { 12478 remove current StatSN from the missing list. 12479 } else { 12480 if (current StatSN is higher than expected){ 12481 send-status-SNACK = TRUE; 12482 } else { 12483 send-status-SNACK = FALSE; 12484 discard the PDU; 12485 } 12486 } 12487 Adjust Connection.ExpectedStatSN if appropriate. 12488 if (missing StatSN list is empty) { 12489 Connection.SoFarInOrder = TRUE; 12490 } 12491 } 12492 return send-status-SNACK; 12493 } 12494 Receive-a-In-PDU(Connection, CurrentPDU) 12495 { 12496 check-basic-validity(CurrentPDU); 12497 if (Header-Digest-Bad) discard, return; 12498 Retrieve TCB for CurrentPDU.InitiatorTaskTag. 12499 if (CurrentPDU.type == Nop-In) { 12500 if (the PDU is unsolicited) { 12501 if (current StatSN is not expected) { 12502 Recover-Status-if-Possible(Connection, 12503 CurrentPDU); 12504 } 12505 if (current ExpCmdSN is not Session.CmdSN) { 12506 Retransmit-Command-if-Possible(Connection, 12507 CurrentPDU.ExpCmdSN); 12508 } 12509 } 12510 } else if (CurrentPDU.type == Reject) { 12511 if (it is a data digest error on immediate data) { 12512 Retransmit-Command-if-Possible(Connection, 12514 CurrentPDU.BadPDUHeader.CmdSN); 12515 } 12516 } else if (CurrentPDU.type == Response) { 12517 send-status-SNACK = Evaluate-a-StatSN(Connection, 12518 CurrentPDU); 12519 if (send-status-SNACK == TRUE) 12520 Recover-Status-if-Possible(Connection, CurrentPDU); 12521 } else { /* REST UNRELATED TO WITHIN-CONNECTION-RECOVERY, 12522 * NOT SHOWN */ 12523 } 12524 } 12526 Command-Acknowledge-Timeout-Handler(TCB) 12527 { 12528 Retrieve the Connection for TCB. 12529 Retransmit-Command-if-Possible(Connection, TCB.CmdSN); 12530 } 12532 Status-Expect-Timeout-Handler(Connection) 12533 { 12534 if (operational ErrorRecoveryLevel > 0) { 12535 Build-And-Send-Nop-Out(Connection); 12536 } else if (InitiatorProactiveSNACKEnabled){ 12537 if ((Connection.state == LOGGED_IN) and 12538 connection is not already considered 12539 failed) { 12540 Build-And-Send-SSnack(Connection); 12541 } 12542 } 12543 } 12545 D.3.3.Target Algorithms 12546 Handle-Status-SNACK-request(Connection, CurrentPDU) 12547 { 12548 if (operational ErrorRecoveryLevel > 0) { 12549 if (request for an acknowledged run) { 12550 Build-And-Send-Reject(Connection, CurrentPDU, 12551 Protocol-Error); 12552 } else if (request for an untransmitted run) { 12553 discard, return; 12554 } else { 12555 Retransmit-Status-Burst(CurrentPDU, TCB); 12556 } 12557 } else { 12558 Build-And-Send-Async(Connection, DroppedConnection, 12559 DefaultTime2Wait, 12560 DefaultTime2Retain); 12561 } 12562 } 12564 D.4. Connection Recovery Algorithms 12566 D.4.1. Procedure Descriptions 12568 Build-And-Send-Async(transport connection, reason code, 12569 minimum time, maximum time); 12570 Pick-A-Logged-In-Connection(session); 12571 Build-And-Send-Logout(transport connection, logout connection 12572 identifier, reason code); 12573 PerformImplicitLogout(transport connection, logout connection 12574 identifier, target information); 12576 PerformLogin(transport connection, target information); 12577 CreateNewTransportConnection(target information); 12578 Build-And-Send-Command(transport connection, task control 12579 block); 12580 Connection-Cleanup-Handler(transport connection); 12581 Connection-Resource-Timeout-Handler(transport connection); 12582 Quiesce-And-Prepare-for-New-Allegiance(session, task control 12583 block); 12584 Build-And-Send-Logout-Response(transport connection, 12585 CID of connection in recovery, reason 12586 code); 12587 Build-And-Send-TaskMgmt-Response(transport connection, 12588 task mgmt command PDU, response code); 12589 Establish-New-Allegiance(task control block, transport 12590 connection); 12591 Schedule-Command-To-Continue(task control block); 12593 Notes: 12594 - Transport exception conditions, such as unexpected 12595 connection termination, connection reset, and hung 12596 connection while the connection is in the full-feature 12597 phase, are all assumed to be asynchronously signaled to the 12598 iSCSI layer using the Transport_Exception_Handler procedure. 12600 D.4.2. Initiator Algorithms 12602 Receive-a-In-PDU(Connection, CurrentPDU) 12603 { 12604 check-basic-validity(CurrentPDU); 12605 if (Header-Digest-Bad) discard, return; 12606 Retrieve TCB from CurrentPDU.InitiatorTaskTag. 12607 if (CurrentPDU.type == Async) { 12608 if (CurrentPDU.AsyncEvent == ConnectionDropped) { 12609 Retrieve the AffectedConnection for 12610 CurrentPDU.Parameter1. 12611 AffectedConnection.CurrentTimeout = 12612 CurrentPDU.Parameter3; 12613 AffectedConnection.State = CLEANUP_WAIT; 12614 Start-Timer(Connection-Cleanup-Handler, 12615 AffectedConnection, 12616 CurrentPDU.Parameter2); 12617 } else if (CurrentPDU.AsyncEvent == LogoutRequest)) { 12618 AffectedConnection = Connection; 12619 AffectedConnection.State = LOGOUT_REQUESTED; 12620 AffectedConnection.PerformConnectionCleanup = TRUE; 12621 AffectedConnection.CurrentTimeout = 12622 CurrentPDU.Parameter3; 12623 Start-Timer(Connection-Cleanup-Handler, 12624 AffectedConnection, 0); 12625 } else if (CurrentPDU.AsyncEvent == SessionDropped)) { 12626 for (each Connection) { 12627 Connection.State = CLEANUP_WAIT; 12628 Connection.CurrentTimeout = CurrentPDU.Parameter3; 12629 Start-Timer(Connection-Cleanup-Handler, 12630 Connection, CurrentPDU.Parameter2); 12631 } 12632 Session.state = FAILED; 12633 } 12635 } else if (CurrentPDU.type == LogoutResponse) { 12636 Retrieve the CleanupConnection for CurrentPDU.CID. 12637 if (CurrentPDU.Response = failure) { 12638 CleanupConnection.State = CLEANUP_WAIT; 12639 } else { 12640 CleanupConnection.State = FREE; 12641 } 12642 } else if (CurrentPDU.type == LoginResponse) { 12643 if (this is a response to an implicit Logout) { 12644 Retrieve the CleanupConnection. 12645 if (successful) { 12646 CleanupConnection.State = FREE; 12647 Connection.State = LOGGED_IN; 12648 } else { 12649 CleanupConnection.State = CLEANUP_WAIT; 12650 DestroyTransportConnection(Connection); 12651 } 12652 } 12653 } else { /* REST UNRELATED TO CONNECTION-RECOVERY, 12654 * NOT SHOWN */ 12656 } 12657 if (CleanupConnection.State == FREE) { 12658 for (each command that was active on CleanupConnection) { 12659 /* Establish new connection allegiance */ 12660 NewConnection = Pick-A-Logged-In- 12661 Connection(Session); 12662 Build-And-Send-Command(NewConnection, TCB); 12663 } 12664 } 12665 } 12667 Connection-Cleanup-Handler(Connection) 12668 { 12669 Retrieve Session from Connection. 12670 if (Connection can still exchange iSCSI PDUs) { 12671 NewConnection = Connection; 12672 } else { 12673 Start-Timer(Connection-Resource-Timeout-Handler, 12674 Connection, Connection.CurrentTimeout); 12675 if (there are other logged-in connections) { 12676 NewConnection = Pick-A-Logged-In- 12677 Connection(Session); 12678 } else { 12679 NewConnection = 12681 CreateTransportConnection(Session.OtherEndInfo); 12682 Initiate an implicit Logout on NewConnection for 12683 Connection.CID. 12684 return; 12685 } 12686 } 12687 Build-And-Send-Logout(NewConnection, Connection.CID, 12688 RecoveryRemove); 12689 } 12691 Transport_Exception_Handler(Connection) 12692 { 12693 Connection.PerformConnectionCleanup = TRUE; 12694 if (the event is an unexpected transport disconnect) { 12695 Connection.State = CLEANUP_WAIT; 12696 Connection.CurrentTimeout = DefaultTime2Retain; 12697 Start-Timer(Connection-Cleanup-Handler, Connection, 12698 DefaultTime2Wait); 12700 } else { 12701 Connection.State = FREE; 12702 } 12703 } 12705 D.4.3. Target Algorithms 12707 Receive-a-In-PDU(Connection, CurrentPDU) 12708 { 12709 check-basic-validity(CurrentPDU); 12710 if (Header-Digest-Bad) discard, return; 12711 else if (Data-Digest-Bad) { 12712 Build-And-Send-Reject(Connection, CurrentPDU, 12713 Payload-Digest-Error); 12714 discard, return; 12715 } 12716 Retrieve TCB and Session. 12717 if (CurrentPDU.type == Logout) { 12718 if (CurrentPDU.ReasonCode = RecoveryRemove) { 12719 Retrieve the CleanupConnection from CurrentPDU.CID). 12720 for (each command active on CleanupConnection) { 12721 Quiesce-And-Prepare-for-New-Allegiance(Session, 12722 TCB); 12723 TCB.CurrentlyAllegiant = FALSE; 12724 } 12725 Cleanup-Connection-State(CleanupConnection); 12726 if ((quiescing successful) and (cleanup successful)) 12727 { 12728 Build-And-Send-Logout-Response(Connection, 12729 CleanupConnection.CID, 12730 Success); 12731 } else { 12732 Build-And-Send-Logout-Response(Connection, 12733 CleanupConnection.CID, 12734 Failure); 12735 } 12737 } 12738 } else if ((CurrentPDU.type == Login) and 12739 operational ErrorRecoveryLevel == 2) { 12740 Retrieve the CleanupConnection from CurrentPDU.CID). 12741 for (each command active on CleanupConnection) { 12742 Quiesce-And-Prepare-for-New-Allegiance(Session, 12743 TCB); 12744 TCB.CurrentlyAllegiant = FALSE; 12745 } 12746 Cleanup-Connection-State(CleanupConnection); 12747 if ((quiescing successful) and (cleanup successful)) 12748 { 12749 Continue with the rest of the Login processing; 12750 } else { 12751 Build-And-Send-Login-Response(Connection, 12752 CleanupConnection.CID, Target 12753 Error); 12754 } 12755 } 12756 } else if (CurrentPDU.type == TaskManagement) { 12757 if (CurrentPDU.function == "TaskReassign") { 12758 if (Session.ErrorRecoveryLevel < 2) { 12759 Build-And-Send-TaskMgmt-Response(Connection, 12760 CurrentPDU, "Allegiance reassignment 12761 not supported"); 12762 } else if (task is not found) { 12763 Build-And-Send-TaskMgmt-Response(Connection, 12764 CurrentPDU, "Task not in task set"); 12765 } else if (task is currently allegiant) { 12766 Build-And-Send-TaskMgmt-Response(Connection, 12767 CurrentPDU, "Task still allegiant"); 12768 } else { 12769 Establish-New-Allegiance(TCB, Connection); 12770 TCB.CurrentlyAllegiant = TRUE; 12771 Schedule-Command-To-Continue(TCB); 12772 } 12773 } 12774 } else { /* REST UNRELATED TO CONNECTION-RECOVERY, 12775 * NOT SHOWN */ 12776 } 12778 } 12780 Transport_Exception_Handler(Connection) 12781 { 12782 Connection.PerformConnectionCleanup = TRUE; 12783 if (the event is an unexpected transport disconnect) { 12784 Connection.State = CLEANUP_WAIT; 12785 Start-Timer(Connection-Resource-Timeout-Handler, 12786 Connection, 12788 (DefaultTime2Wait+DefaultTime2Retain)); 12789 if (this Session has full-feature phase connections 12790 left) { 12791 DifferentConnection = 12792 Pick-A-Logged-In-Connection(Session); 12793 Build-And-Send-Async(DifferentConnection, 12794 DroppedConnection, DefaultTime2Wait, 12795 DefaultTime2Retain); 12796 } 12797 } else { 12798 Connection.State = FREE; 12799 } 12800 } 12801 Appendix E. Clearing Effects of Various Events on Targets 12803 E.1. Clearing Effects on iSCSI Objects 12805 The following tables describe the target behavior on receiving the 12806 events specified in the rows of the table. The second table is 12807 an extension of the first table and defines clearing actions for 12808 more objects on the same events. The legend is: 12810 Y = Yes (cleared/discarded/reset on the event specified in 12811 the row). Unless otherwise noted, the clearing action is 12812 only applicable for the issuing initiator port. 12814 N = No (not affected on the event specified in the row, 12815 i.e., stays at previous value). 12817 NA = Not Applicable or Not Defined. 12819 +-----+-----+-----+-----+-----+ 12820 |IT(1)|IC(2)|CT(5)|ST(6)|PP(7)| 12821 +---------------------+-----+-----+-----+-----+-----+ 12822 |connection failure(8)|Y |Y |N |N |Y | 12823 +---------------------+-----+-----+-----+-----+-----+ 12824 |connection state |NA |NA |Y |N |NA | 12825 |timeout (9) | | | | | | 12826 +---------------------+-----+-----+-----+-----+-----+ 12827 |session timeout/ |Y |Y |Y |Y |Y(14)| 12828 |closure/reinstatement| | | | | | 12829 |(10) | | | | | | 12830 +---------------------+-----+-----+-----+-----+-----+ 12831 |session continuation |NA |NA |N(11)|N |NA | 12832 |(12) | | | | | | 12833 +---------------------+-----+-----+-----+-----+-----+ 12834 |successful connection|Y |Y |Y |N |Y(13)| 12835 |close logout | | | | | | 12836 +---------------------+-----+-----+-----+-----+-----+ 12837 |session failure (18) |Y |Y |N |N |Y | 12838 +---------------------+-----+-----+-----+-----+-----+ 12839 |successful recovery |Y |Y |N |N |Y(13)| 12840 |Logout | | | | | | 12841 +---------------------+-----+-----+-----+-----+-----+ 12842 |failed Logout |Y |Y |N |N |Y | 12843 +---------------------+-----+-----+-----+-----+-----+ 12844 |connection Login |NA |NA |NA |Y(15)|NA | 12845 |(leading) | | | | | | 12846 +---------------------+-----+-----+-----+-----+-----+ 12847 |connection Login |NA |NA |N(11)|N |Y | 12848 |(non-leading) | | | | | | 12849 +---------------------+-----+-----+-----+-----+-----+ 12850 |target cold reset(16)|Y(20)|Y |Y |Y |Y | 12851 +---------------------+-----+-----+-----+-----+-----+ 12852 |target warm reset(16)|Y(20)|Y |Y |Y |Y | 12853 +---------------------+-----+-----+-----+-----+-----+ 12854 |LU reset(19) |Y(20)|Y |Y |Y |Y | 12855 +---------------------+-----+-----+-----+-----+-----+ 12856 |powercycle(16) |Y |Y |Y |Y |Y | 12857 +---------------------+-----+-----+-----+-----+-----+ 12859 1. Incomplete TTTs - Target Transfer Tags on which the target is 12860 still expecting PDUs to be received. Examples include TTTs 12861 received via R2T, NOP-IN, etc. 12863 2. Immediate Commands - immediate commands, but waiting for 12864 execution on a target. For example, Abort Task Set. 12866 5. Connection Tasks - tasks that are active on the iSCSI connection 12867 in question. 12869 6. Session Tasks - tasks that are active on the entire iSCSI 12870 session. A union of "connection tasks" on all participating 12871 connections. 12873 7. Partial PDUs (if any) - PDUs that are partially sent and waiting 12874 for transport window credit to complete the transmission. 12876 8. Connection failure is a connection exception condition - one of 12877 the transport connections shutdown, transport connections 12878 reset, or transport connections timed out, which abruptly 12879 terminated the iSCSI full-feature phase connection. A 12880 connection failure always takes the connection state machine to 12881 the CLEANUP_WAIT state. 12883 9. Connection state timeout happens if a connection spends more 12884 time than agreed upon during Login negotiation in the 12885 CLEANUP_WAIT state, and this takes the connection to the FREE 12886 state (M1 transition in connection cleanup state diagram). 12888 10.These are defined in Section 6.3.5. 12890 11.This clearing effect is "Y" only if it is a connection 12891 reinstatement and the operational ErrorRecoveryLevel is less 12892 than 2. 12894 12.Session continuation is defined in Section 6.3.5. 12896 13.This clearing effect is only valid if the connection is being 12897 logged out on a different connection and when the connection 12898 being logged out on the target may have some partial PDUs 12899 pending to be sent. In all other cases, the effect is "NA". 12901 14.This clearing effect is only valid for a "close the session" 12902 logout in a multi-connection session. In all other cases, the 12903 effect is "NA". 12904 15.Only applicable if this leading connection login is a session 12905 reinstatement. If this is not the case, it is "NA". 12906 16.This operation affects all logged-in initiators. 12907 18.Session failure is defined in Section 6.3.5. 12908 19.This operation affects all logged-in initiators and the clearing 12909 effects are only applicable to the LU being reset. 12910 20.With Standard multi-task abort semantics (Section 4.2.3.3), a 12911 target warm reset or a target cold reset or an LU reset would 12912 clear the active TTTs upon completion. However, the FastAbort 12913 multi-task abort semantics defined by Section 4.2.3.4 do not 12914 guarantee that the active TTTs are cleared by the end of the 12915 reset operations. In fact, the FastAbort semantics are designed 12916 to allow clearing the TTTs in a "lazy" fashion after the TMF 12917 Response is delivered. Thus, when TaskReporting=FastAbort 12918 (Section 13.23) is operational on a session, the clearing 12919 effects of reset operations on "Incomplete TTTs" is "N". 12921 +-----+-----+-----+-----+-----+ 12922 |DC(1)|DD(2)|SS(3)|CS(4)|DS(5)| 12923 +---------------------+-----+-----+-----+-----+-----+ 12924 |connection failure |N |Y |N |N |N | 12925 +---------------------+-----+-----+-----+-----+-----+ 12926 |connection state |Y |NA |Y |N |NA | 12927 |timeout | | | | | | 12928 +---------------------+-----+-----+-----+-----+-----+ 12929 |session timeout/ |Y |Y |Y(7) |Y |NA | 12930 |closure/reinstatement| | | | | | 12931 +---------------------+-----+-----+-----+-----+-----+ 12932 |session continuation |N(11)|NA*12|NA |N |NA*13| 12933 +---------------------+-----+-----+-----+-----+-----+ 12934 |successful connection|Y |Y |Y |N |NA | 12935 |close Logout | | | | | | 12936 +---------------------+-----+-----+-----+-----+-----+ 12937 |session failure |N |Y |N |N |N | 12938 +---------------------+-----+-----+-----+-----+-----+ 12939 |successful recovery |Y |Y |Y |N |N | 12940 |Logout | | | | | | 12941 +---------------------+-----+-----+-----+-----+-----+ 12942 |failed Logout |N |Y(9) |N |N |N | 12943 +---------------------+-----+-----+-----+-----+-----+ 12944 |connection Login |NA |NA |N(8) |N(8) |NA | 12945 |(leading | | | | | | 12946 +---------------------+-----+-----+-----+-----+-----+ 12947 |connection Login |N(11)|NA*12|N(8) |N |NA*13| 12948 |(non-leading) | | | | | | 12949 +---------------------+-----+-----+-----+-----+-----+ 12950 |target cold reset |Y |Y |Y |Y(10)|NA | 12951 +---------------------+-----+-----+-----+-----+-----+ 12952 |target warm reset |Y |Y |N |N |NA | 12953 +---------------------+-----+-----+-----+-----+-----+ 12954 |LU reset |N |Y |N |N |N | 12955 +---------------------+-----+-----+-----+-----+-----+ 12956 |powercycle |Y |Y |Y |Y(10)|NA | 12957 +---------------------+-----+-----+-----+-----+-----+ 12959 1. Discontiguous Commands - commands allegiant to the connection 12960 in question and waiting to be reordered in the iSCSI layer. All 12961 "Y"s in this column assume that the task causing the event (if 12962 indeed the event is the result of a task) is issued as an 12963 immediate command, because the discontiguities can be ahead of the 12964 task. 12966 2. Discontiguous Data - data PDUs received for the task in 12967 question and waiting to be reordered due to prior discontiguities 12968 in DataSN. 12970 3. StatSN 12972 4. CmdSN 12974 5. DataSN 12976 7. It clears the StatSN on all the connections. 12978 8. This sequence number is instantiated on this event. 12980 9. A logout failure drives the connection state machine to the 12981 CLEANUP_WAIT state, similar to the connection failure event. 12982 Hence, it has a similar effect on this and several other protocol 12983 aspects. 12985 10. This is cleared by virtue of the fact that all sessions with 12986 all initiators are terminated. 12988 11. This clearing effect is "Y" if it is a connection 12989 reinstatement. 12991 12. This clearing effect is "Y" only if it is a connection 12992 reinstatement and the operational ErrorRecoveryLevel is 2. 12994 13. This clearing effect is "N" only if it is a connection 12995 reinstatement and the operational ErrorRecoveryLevel is 2. 12997 E.2. Clearing Effects on SCSI Objects 12999 The only iSCSI protocol action that can effect clearing actions on 13000 SCSI objects is the "I_T nexus loss" notification (Section 6.3.5.1 13001 Loss of Nexus notification). [SPC3] describes the clearing effects 13002 of this notification on a variety of SCSI attributes. In addition, 13003 SCSI standards documents (such as [SAM2] and [SBC2]) define 13004 additional clearing actions that may take place for several SCSI 13005 objects on SCSI events such as LU resets and power-on resets. 13007 Since iSCSI defines a target cold reset as a protocol-equivalent 13008 to a target power-cycle, the iSCSI target cold reset must also be 13009 considered as the power-on reset event in interpreting the actions 13010 defined in the SCSI standards. 13012 When the iSCSI session is reconstructed (between the same SCSI 13013 ports with the same nexus identifier) reestablishing the same I_T 13014 nexus, all SCSI objects that are defined to not clear on the "I_T 13015 nexus loss" notification event, such as persistent reservations, 13016 are automatically associated to this new session. 13018 Acknowledgments 13020 Several individuals on the original IPS Working Group made 13021 significant contributions to the original RFCs 3720, 3980, 4850 13022 and 5048. 13024 Specifically, the authors of the original RFCs - which this draft 13025 consolidates into a single document - were the following: 13027 RFC 3720: Julian Satran, Kalman Meth, Costa Sapuntzakis, 13028 Mallikarjun Chadalapaka, Efri Zeidner 13030 RFC 3980: Marjorie Krueger, Mallikarjun Chadalapaka, Rob Elliott 13032 RFC 4850: David Wysochanski 13034 RFC 5048: Mallikarjun Chadalapaka 13036 Many thanks to Fred Knight for contributing to the UML notations 13037 and drawings in this draft. 13039 We would in addition like to acknowledge the following individuals 13040 who contributed to this revised draft: David Harrington, Paul 13041 Koning, Mark Edwards, Rob Elliott, Martin Stiemerling. 13043 Thanks to Yi Zeng and Nico for suggesting and/or reviewing 13044 Kerberos-related security considerations text. 13046 Finally, this draft also benefited from significant review 13047 contributions from the Storm Working Group at large. 13049 Authors' Addresses 13051 Mallikarjun Chadalapaka 13052 Microsoft 13053 One Microsoft Way 13054 Redmond WA 98052 USA 13055 E-mail: cbm@chadalapaka.com 13057 Julian Satran 13058 Infinidat Ltd. 13059 E-mail: julians@infinidat.com, julian@satran.net 13061 Kalman Meth 13062 IBM Haifa Research Lab 13063 Haifa University Campus - Mount Carmel 13064 Haifa 31905, Israel 13065 Phone +972.4.829.6341 13066 E-mail: meth@il.ibm.com 13068 David L. Black, 13069 EMC Corporation, 13070 176 South St., Hopkinton, MA 01748 13071 Phone +1 (508) 293-7953 13072 Email: david.black@emc.com 13074 Comments may be sent to Mallikarjun Chadalapaka 13076 Acknowledgement 13078 Funding for the RFC Editor function is currently provided by the 13079 Internet Society