idnits 2.17.1 draft-ietf-storm-iscsi-cons-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** The document seems to lack a License Notice according IETF Trust Provisions of 28 Dec 2009, Section 6.b.i or Provisions of 12 Sep 2009 Section 6.b -- however, there's a paragraph with a matching beginning. Boilerplate error? (You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Feb 2009 rather than one of the newer Notices. See https://trustee.ietf.org/license-info/.) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 1 instance of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. == There are 2 instances of lines with private range IPv4 addresses in the document. If these are generic example addresses, they should be changed to use any of the ranges defined in RFC 6890 (or successor): 192.0.2.x, 198.51.100.x or 203.0.113.x. == There are 1 instance of lines with non-RFC3849-compliant IPv6 addresses in the document. If these are example addresses, they should be changed. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 1441 has weird spacing: '...essions only ...' == Line 1990 has weird spacing: '...e three nami...' == Line 1991 has weird spacing: '...he time of w...' == Line 1992 has weird spacing: '...ng type desi...' == Line 1993 has weird spacing: '...iled in sepa...' == (17 more instances...) == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- Couldn't find a document date in the document -- date freshness check skipped. -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '7' on line 6327 -- Looks like a reference, but probably isn't: '0' on line 6326 == Missing Reference: 'RFC1510' is mentioned on line 9493, but not defined ** Obsolete undefined reference: RFC 1510 (Obsoleted by RFC 4120, RFC 6649) == Missing Reference: 'MaxMissingDPDU' is mentioned on line 11677, but not defined == Missing Reference: 'MaxOutStandingR2T' is mentioned on line 11685, but not defined == Missing Reference: 'MaxMissingSPDU' is mentioned on line 11698, but not defined == Missing Reference: 'MaxSupportedConns' is mentioned on line 11708, but not defined == Unused Reference: 'RFC791' is defined on line 10439, but no explicit reference was found in the text == Unused Reference: 'RFC793' is defined on line 10442, but no explicit reference was found in the text == Unused Reference: 'RFC1035' is defined on line 10445, but no explicit reference was found in the text == Unused Reference: 'RFC1122' is defined on line 10448, but no explicit reference was found in the text == Unused Reference: 'RFC2373' is defined on line 10471, but no explicit reference was found in the text == Unused Reference: 'RFC3629' is defined on line 10490, but no explicit reference was found in the text == Unused Reference: 'RFC3980' is defined on line 10513, but no explicit reference was found in the text == Unused Reference: 'RFC4646' is defined on line 10533, but no explicit reference was found in the text == Unused Reference: 'RFC5226' is defined on line 10544, but no explicit reference was found in the text == Unused Reference: 'RFC4173' is defined on line 10569, but no explicit reference was found in the text == Unused Reference: 'CRC' is defined on line 10578, but no explicit reference was found in the text == Unused Reference: 'RFC3347' is defined on line 10580, but no explicit reference was found in the text == Unused Reference: 'Schneier' is defined on line 10594, but no explicit reference was found in the text -- Possible downref: Non-RFC (?) normative reference: ref. 'EUI' -- Possible downref: Non-RFC (?) normative reference: ref. 'FC-FS' -- Possible downref: Non-RFC (?) normative reference: ref. 'OUI' ** Obsolete normative reference: RFC 793 (Obsoleted by RFC 9293) ** Obsolete normative reference: RFC 2373 (Obsoleted by RFC 3513) ** Obsolete normative reference: RFC 3454 (Obsoleted by RFC 7564) ** Obsolete normative reference: RFC 3720 (Obsoleted by RFC 7143) ** Downref: Normative reference to an Informational RFC: RFC 3783 ** Obsolete normative reference: RFC 3980 (Obsoleted by RFC 7143) ** Obsolete normative reference: RFC 4306 (Obsoleted by RFC 5996) ** Obsolete normative reference: RFC 4646 (Obsoleted by RFC 5646) ** Obsolete normative reference: RFC 4850 (Obsoleted by RFC 7143) ** Obsolete normative reference: RFC 5048 (Obsoleted by RFC 7143) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) -- Possible downref: Non-RFC (?) normative reference: ref. 'SAM2' -- Possible downref: Non-RFC (?) normative reference: ref. 'SAM3' -- Possible downref: Non-RFC (?) normative reference: ref. 'SAM4' -- Possible downref: Non-RFC (?) normative reference: ref. 'SBC' -- Possible downref: Non-RFC (?) normative reference: ref. 'SPC3' -- Possible downref: Non-RFC (?) normative reference: ref. 'UNICODE' ** Downref: Normative reference to an Informational RFC: RFC 1737 -- Possible downref: Non-RFC (?) normative reference: ref. 'IB' -- Possible downref: Non-RFC (?) normative reference: ref. 'Castagnoli93' -- Possible downref: Non-RFC (?) normative reference: ref. 'CRC' ** Downref: Normative reference to an Informational RFC: RFC 3385 ** Downref: Normative reference to an Informational RFC: RFC 3721 -- Possible downref: Non-RFC (?) normative reference: ref. 'Schneier' -- Possible downref: Non-RFC (?) normative reference: ref. 'SAS' -- Possible downref: Non-RFC (?) normative reference: ref. 'SRP' Summary: 16 errors (**), 0 flaws (~~), 29 warnings (==), 19 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Storage Maintenance (storm) WG Mallikarjun Chadalapaka 2 Internet Draft Hewlett-Packard Co. 3 draft-ietf-storm-iscsi-cons-00.txt 4 Intended status: Proposed Standard Julian Satran 5 Expires: June 2010 6 Updates: RFC 3720, 3980, 4850, 5048 Kalman Meth 7 IBM 9 iSCSI Protocol (Consolidated) 11 Status of this Memo 13 This Internet-Draft is submitted to IETF in full conformance with 14 the provisions of BCP 78 and BCP 79. 16 Internet-Drafts are working documents of the Internet Engineering 17 Task Force (IETF), its areas, and its working groups. Note that 18 other groups may also distribute working documents as Internet- 19 Drafts. 21 Internet-Drafts are draft documents valid for a maximum of six 22 months and may be updated, replaced, or obsoleted by other 23 documents at any time. It is inappropriate to use Internet- 24 Drafts as reference material or to cite them other than as "work 25 in progress." 27 The list of current Internet-Drafts can be accessed at 28 http://www.ietf.org/ietf/1id-abstracts.txt 30 The list of Internet-Draft Shadow Directories can be accessed at 31 http://www.ietf.org/shadow.html 32 This Internet-Draft will expire on June 30, 2010. 34 Copyright Notice 36 Copyright (c) 2009 IETF Trust and the persons identified as the 37 document authors. All rights reserved. 39 This document is subject to BCP 78 and the IETF Trust's Legal 40 Provisions Relating to IETF Documents in effect on the date of 41 publication of this document (http://trustee.ietf.org/license- 42 info). Please review these documents carefully, as they describe 43 your rights and restrictions with respect to this document. Code 44 Components extracted from this document must include Simplified 45 BSD License text as described in Section 4.e of the Trust Legal 46 Provisions and are provided without warranty as described in the 47 BSD License. 49 Abstract 51 This document describes a transport protocol for SCSI that works on 52 top of TCP. The iSCSI protocol aims to be fully compliant with the 53 standardized SCSI Architecture Model (SAM). RFC 3720 defined the 54 original iSCSI protocol. Subsequently, RFC 3980 added an 55 additional naming format to iSCSI protocol. RFC 4850 followed up 56 by adding a new public extension key to iSCSI. RFC 5048 offered a 57 number of clarifications and a few improvements and corrections to 58 the original iSCSI protocol. 60 This document consolidates RFCs 3720, 3980, 4850 and 5048 into a 61 single document and makes additional updates to the consolidated 62 specification. The text in this document supersedes the text in 63 RFCs 3720, 3980, 4850 and 5048 whenever there is such a question. 65 1. Introduction..................................................... 13 67 2. Definitions and Acronyms.........................................14 68 2.1. Definitions .................................................. 14 69 2.2. Acronyms ..................................................... 20 70 2.3. Conventions .................................................. 22 71 2.3.1. Word Rule ................................................ 23 72 2.3.2. Half-Word Rule ........................................... 23 73 2.3.3. Byte Rule ................................................ 24 74 3. Overview.........................................................25 75 3.1. SCSI Concepts ................................................ 25 76 3.2. iSCSI Concepts and Functional Overview ....................... 26 77 3.2.1. Layers and Sessions ...................................... 26 78 3.2.2. Ordering and iSCSI Numbering ............................. 27 79 3.2.2.1. Command Numbering and Acknowledging ...................28 80 3.2.2.2. Response/Status Numbering and Acknowledging ...........32 81 3.2.2.3. Response Ordering .....................................32 82 3.2.2.3.1. Need for Response Ordering ........................32 83 3.2.2.3.2. Response Ordering Model Description ...............33 84 3.2.2.3.3. iSCSI Semantics with the Interface Model ..........34 85 3.2.2.3.4. Current List of Fenced Response Use Cases .........34 86 3.2.2.4. Data Sequencing .......................................35 87 3.2.3. iSCSI Task Management .................................... 36 88 3.2.3.1. Task Management Overview ..............................36 89 3.2.3.2. Notion of Affected Tasks ..............................36 90 3.2.3.3. Standard Multi-task Abort Semantics ...................37 91 3.2.3.4. FastAbort Multi-task Abort Semantics ..................38 92 3.2.3.5. Affected Tasks Shared across Standard and FastAbort 93 Sessions .......................................................40 94 3.2.3.6. Rationale behind the FastAbort Semantics ..............41 95 3.2.4. iSCSI Login .............................................. 43 96 3.2.5. iSCSI Full Feature Phase ................................. 45 97 3.2.5.1. Command Connection Allegiance .........................45 98 3.2.5.2. Data Transfer Overview ................................46 99 3.2.5.3. Tags and Integrity Checks .............................47 100 3.2.5.4. Task Management .......................................48 101 3.2.6. iSCSI Connection Termination ............................. 48 102 3.2.7. iSCSI Names .............................................. 49 103 3.2.7.1. iSCSI Name Properties .................................49 104 3.2.7.2. iSCSI Name Encoding ...................................51 105 3.2.7.3. iSCSI Name Structure ..................................52 106 3.2.7.4. Type "iqn." (iSCSI Qualified Name) ....................53 107 3.2.7.5. Type "eui." (IEEE EUI-64 format) ......................55 108 3.2.7.6. Type "naa." - Network Address Authority ...............56 109 3.2.8. Persistent State ......................................... 57 110 3.2.9. Message Synchronization and Steering ..................... 57 111 3.2.9.1. Sync/Steering and iSCSI PDU Length ....................59 112 3.3. iSCSI Session Types .......................................... 59 113 3.4. SCSI to iSCSI Concepts Mapping Model ......................... 59 114 3.4.1. iSCSI Architecture Model ................................. 60 115 3.4.2. SCSI Architecture Model .................................. 63 116 3.4.3. Consequences of the Model ................................ 65 117 3.4.3.1. I_T Nexus State .......................................67 118 3.5. Request/Response Summary ..................................... 67 119 3.5.1. Request/Response Types Carrying SCSI Payload ............. 67 120 3.5.1.1. SCSI-Command ..........................................67 121 3.5.1.2. SCSI-Response .........................................68 122 3.5.1.3. Task Management Function Request ......................69 123 3.5.1.4. Task Management Function Response .....................69 124 3.5.1.5. SCSI Data-out and SCSI Data-in ........................69 125 3.5.1.6. Ready To Transfer (R2T) ...............................70 126 3.5.2. Requests/Responses carrying SCSI and iSCSI Payload ....... 71 127 3.5.2.1. Asynchronous Message ..................................71 128 3.5.3. Requests/Responses Carrying iSCSI Only Payload ........... 71 129 3.5.3.1. Text Request and Text Response ........................71 130 3.5.3.2. Login Request and Login Response ......................72 131 3.5.3.3. Logout Request and Response ...........................73 132 3.5.3.4. SNACK Request .........................................73 133 3.5.3.5. Reject ................................................74 134 3.5.3.6. NOP-Out Request and NOP-In Response ...................74 136 4. SCSI Mode Parameters for iSCSI...................................75 138 5. Login and Full Feature Phase Negotiation.........................76 139 5.1. Text Format .................................................. 77 140 5.2. Text Mode Negotiation ........................................ 82 141 5.2.1. List negotiations ........................................ 86 142 5.2.2. Simple-value Negotiations ................................ 86 143 5.3. Login Phase .................................................. 87 144 5.3.1. Login Phase Start ........................................ 90 145 5.3.2. iSCSI Security Negotiation ............................... 94 146 5.3.3. Operational Parameter Negotiation During the Login Phase . 95 147 5.3.4. Connection Reinstatement ................................. 96 148 5.3.5. Session Reinstatement, Closure, and Timeout .............. 97 149 5.3.5.1. Loss of Nexus Notification ............................97 150 5.3.6. Session Continuation and Failure ......................... 98 151 5.4. Operational Parameter Negotiation Outside the Login Phase .... 98 152 6. iSCSI Error Handling and Recovery...............................100 153 6.1. Overview .................................................... 100 154 6.1.1. Background .............................................. 100 155 6.1.2. Goals ................................................... 100 156 6.1.3. Protocol Features and State Expectations ................ 101 157 6.1.4. Recovery Classes ........................................ 102 158 6.1.4.1. Recovery Within-command ..............................103 159 6.1.4.2. Recovery Within-connection ...........................104 160 6.1.4.3. Connection Recovery ..................................105 161 6.1.4.4. Session Recovery .....................................106 162 6.1.5. Error Recovery Hierarchy ................................ 106 163 6.2. Retry and Reassign in Recovery .............................. 108 164 6.2.1. Usage of Retry .......................................... 108 165 6.2.2. Allegiance Reassignment ................................. 109 166 6.3. Usage Of Reject PDU in Recovery ............................. 110 167 6.4. Error Recovery Considerations for Discovery Sessions ........ 111 168 6.4.1. ErrorRecoveryLevel for Discovery Sessions ............... 111 169 6.4.2. Reinstatement Semantics for Discovery Sessions .......... 111 170 6.4.2.1. Unnamed Discovery Sessions ...........................112 171 6.4.2.2. Named Discovery Session ..............................113 172 6.4.3. Target PDUs During Discovery ............................ 113 173 6.5. Connection Timeout Management ............................... 113 174 6.5.1. Timeouts on Transport Exception Events .................. 114 175 6.5.2. Timeouts on Planned Decommissioning ..................... 114 176 6.6. Implicit Termination of Tasks ............................... 114 177 6.7. Format Errors ............................................... 115 178 6.8. Digest Errors ............................................... 116 179 6.9. Sequence Errors ............................................. 118 180 6.10. Message Error Checking ..................................... 118 181 6.11. SCSI Timeouts .............................................. 119 182 6.12. Negotiation Failures ....................................... 120 183 6.13. Protocol Errors ............................................ 120 184 6.14. Connection Failures ........................................ 121 185 6.15. Session Errors ............................................. 122 186 7. State Transitions...............................................123 187 7.1. Standard Connection State Diagrams .......................... 123 188 7.1.1. State Descriptions for Initiators and Targets ........... 123 189 7.1.2. State Transition Descriptions for Initiators and Targets 124 190 7.1.3. Standard Connection State Diagram for an Initiator ...... 128 191 7.1.4. Standard Connection State Diagram for a Target .......... 130 192 7.2. Connection Cleanup State Diagram for Initiators and Targets . 132 193 7.2.1. State Descriptions for Initiators and Targets ........... 134 194 7.2.2. State Transition Descriptions for Initiators and Targets 134 195 7.3. Session State Diagrams ...................................... 136 196 7.3.1. Session State Diagram for an Initiator .................. 136 197 7.3.2. Session State Diagram for a Target ...................... 137 198 7.3.3. State Descriptions for Initiators and Targets ........... 139 199 7.3.4. State Transition Descriptions for Initiators and Targets 140 200 8. Security Considerations.........................................142 201 8.1. iSCSI Security Mechanisms ................................... 142 202 8.2. In-band Initiator-Target Authentication ..................... 143 203 8.2.1. CHAP Considerations ..................................... 144 204 8.2.2. SRP Considerations ...................................... 146 205 8.3. IPsec ....................................................... 147 206 8.3.1. Data Integrity and Authentication .......................147 207 8.3.2. Confidentiality .........................................148 208 8.3.3. Policy, Security Associations, and Cryptographic Key 209 Management .....................................................148 210 8.4. Security Considerations for the X#NodeArchitecture Key ......150 211 9. Notes to Implementers...........................................152 212 9.1. Multiple Network Adapters ...................................152 213 9.1.1. Conservative Reuse of ISIDs .............................152 214 9.1.2. iSCSI Name, ISID, and TPGT Use ..........................153 215 9.2. Autosense and Auto Contingent Allegiance (ACA) ..............155 216 9.3. iSCSI Timeouts ..............................................155 217 9.4. Command Retry and Cleaning Old Command Instances ............156 218 9.5. Synch and Steering Layer and Performance ....................157 219 9.6. Considerations for State-dependent Devices and Long-lasting SCSI 220 Operations .......................................................157 221 9.6.1. Determining the Proper ErrorRecoveryLevel ...............158 222 9.7. Multi-task Abort Implementation Considerations ..............159 223 10. iSCSI PDU Formats..............................................160 224 10.1. iSCSI PDU Length and Padding ...............................160 225 10.2. PDU Template, Header, and Opcodes ..........................160 226 10.2.1. Basic Header Segment (BHS) .............................161 227 10.2.1.1. I ...................................................162 228 10.2.1.2. Opcode ..............................................162 229 10.2.1.3. Final (F) bit .......................................164 230 10.2.1.4. Opcode-specific Fields ..............................164 231 10.2.1.5. TotalAHSLength ......................................164 232 10.2.1.6. DataSegmentLength ...................................164 233 10.2.1.7. LUN .................................................164 234 10.2.1.8. Initiator Task Tag ..................................165 235 10.2.2. Additional Header Segment (AHS) ........................165 236 10.2.2.1. AHSType .............................................165 237 10.2.2.2. AHSLength ...........................................166 238 10.2.2.3. Extended CDB AHS ....................................166 239 10.2.2.4. Bidirectional Expected Read-Data Length AHS .........166 240 10.2.3. Header Digest and Data Digest ..........................167 241 10.2.4. Data Segment ...........................................167 242 10.3. SCSI Command ...............................................167 243 10.3.1. Flags and Task Attributes (byte 1) .....................168 244 10.3.2. CmdSN - Command Sequence Number ........................169 245 10.3.3. ExpStatSN ..............................................170 246 10.3.4. Expected Data Transfer Length ..........................170 247 10.3.5. CDB - SCSI Command Descriptor Block ....................171 248 10.3.6. Data Segment - Command Data ............................171 249 10.4. SCSI Response ..............................................171 250 10.4.1. Flags (byte 1) .........................................172 251 10.4.2. Status .................................................173 252 10.4.3. Response ...............................................174 253 10.4.4. SNACK Tag ..............................................175 254 10.4.5. Residual Count .........................................175 255 10.4.5.1. Field Semantics ....................................175 256 10.4.5.2. Residuals Concepts Overview ........................176 257 10.4.5.3. SCSI REPORT LUNS and Residual Overflow .............176 258 10.4.6. Bidirectional Read Residual Count ......................178 259 10.4.7. Data Segment - Sense and Response Data Segment .........178 260 10.4.7.1. SenseLength ........................................179 261 10.4.7.2. Sense Data .........................................179 262 10.4.8. ExpDataSN ..............................................180 263 10.4.9. StatSN - Status Sequence Number ........................180 264 10.4.10. ExpCmdSN - Next Expected CmdSN from this Initiator ....181 265 10.4.11. MaxCmdSN - Maximum CmdSN from this Initiator ..........181 266 10.5. Task Management Function Request ...........................182 267 10.5.1. Function ...............................................182 268 10.5.2. TotalAHSLength and DataSegmentLength ...................186 269 10.5.3. LUN ....................................................186 270 10.5.4. Referenced Task Tag ....................................186 271 10.5.5. RefCmdSN ...............................................186 272 10.5.6. ExpDataSN ..............................................187 273 10.6. Task Management Function Response ..........................187 274 10.6.1. Response ...............................................188 275 10.6.2. TotalAHSLength and DataSegmentLength ...................190 276 10.7. SCSI Data-out & SCSI Data-in ...............................190 277 10.7.1. F (Final) Bit ..........................................193 278 10.7.2. A (Acknowledge) bit ....................................193 279 10.7.3. Flags (byte 1) .........................................194 280 10.7.4. Target Transfer Tag and LUN ............................195 281 10.7.5. DataSN .................................................195 282 10.7.6. Buffer Offset ..........................................196 283 10.7.7. DataSegmentLength ......................................196 284 10.8. Ready To Transfer (R2T) ....................................197 285 10.8.1. TotalAHSLength and DataSegmentLength ...................199 286 10.8.2. R2TSN ..................................................199 287 10.8.3. StatSN .................................................199 288 10.8.4. Desired Data Transfer Length and Buffer Offset .........199 289 10.8.5. Target Transfer Tag ....................................199 290 10.9. Asynchronous Message .......................................200 291 10.9.1. AsyncEvent .............................................202 292 10.9.2. AsyncVCode .............................................205 293 10.9.3. LUN ....................................................205 294 10.9.4. Sense Data and iSCSI Event Data ........................205 295 10.9.4.1. SenseLength ........................................205 296 10.10. Text Request ..............................................206 297 10.10.1. F (Final) Bit .........................................207 298 10.10.2. C (Continue) Bit ......................................207 299 10.10.3. Initiator Task Tag ....................................207 300 10.10.4. Target Transfer Tag ...................................207 301 10.10.5. Text ..................................................208 302 10.11. Text Response .............................................209 303 10.11.1. F (Final) Bit .........................................210 304 10.11.2. C (Continue) Bit ......................................211 305 10.11.3. Initiator Task Tag ....................................211 306 10.11.4. Target Transfer Tag ...................................211 307 10.11.5. StatSN ................................................212 308 10.11.6. Text Response Data ....................................212 309 10.12. Login Request .............................................212 310 10.12.1. T (Transit) Bit .......................................213 311 10.12.2. C (Continue) Bit ......................................214 312 10.12.3. CSG and NSG ...........................................214 313 10.12.4. Version ...............................................214 314 10.12.4.1. Version-max .......................................214 315 10.12.4.2. Version-min .......................................215 316 10.12.5. ISID ..................................................215 317 10.12.6. TSIH ..................................................217 318 10.12.7. Connection ID - CID ...................................217 319 10.12.8. CmdSN .................................................217 320 10.12.9. ExpStatSN .............................................218 321 10.12.10. Login Parameters .....................................218 322 10.13. Login Response ............................................219 323 10.13.1. Version-max ...........................................219 324 10.13.2. Version-active ........................................220 325 10.13.3. TSIH ..................................................220 326 10.13.4. StatSN ................................................220 327 10.13.5. Status-Class and Status-Detail ........................220 328 10.13.6. T (Transit) bit .......................................224 329 10.13.7. C (Continue) Bit ......................................225 330 10.13.8. Login Parameters ......................................225 331 10.14. Logout Request ............................................225 332 10.14.1. Reason Code ...........................................228 333 10.14.2. TotalAHSLength and DataSegmentLength ..................229 334 10.14.3. CID ...................................................229 335 10.14.4. ExpStatSN .............................................229 336 10.14.5. Implicit termination of tasks .........................229 337 10.15. Logout Response ...........................................230 338 10.15.1. Response ..............................................231 339 10.15.2. TotalAHSLength and DataSegmentLength ..................232 340 10.15.3. Time2Wait .............................................232 341 10.15.4. Time2Retain ...........................................232 342 10.16. SNACK Request .............................................234 343 10.16.1. Type ..................................................235 344 10.16.2. Data Acknowledgement ..................................236 345 10.16.3. Resegmentation ........................................236 346 10.16.4. Initiator Task Tag ....................................237 347 10.16.5. Target Transfer Tag or SNACK Tag ......................237 348 10.16.6. BegRun ................................................238 349 10.16.7. RunLength .............................................238 350 10.17. Reject ....................................................239 351 10.17.1. Reason ................................................240 352 10.17.2. DataSN/R2TSN ..........................................243 353 10.17.3. StatSN, ExpCmdSN and MaxCmdSN .........................243 354 10.17.4. Complete Header of Bad PDU ............................243 355 10.18. NOP-Out ...................................................244 356 10.18.1. Initiator Task Tag ....................................245 357 10.18.2. Target Transfer Tag ...................................245 358 10.18.3. Ping Data .............................................245 359 10.19. NOP-In ....................................................246 360 10.19.1. Target Transfer Tag ...................................247 361 10.19.2. StatSN ................................................247 362 10.19.3. LUN ...................................................247 363 11. iSCSI Security Text Keys and Authentication Methods............248 364 11.1. AuthMethod .................................................248 365 11.1.1. Kerberos ...............................................251 366 11.1.2. Simple Public-Key Mechanism (SPKM) .....................251 367 11.1.3. Secure Remote Password (SRP) ...........................253 368 11.1.4. Challenge Handshake Authentication Protocol (CHAP) .....254 369 12. Login/Text Operational Text Keys...............................256 370 12.1. HeaderDigest and DataDigest ................................256 371 12.2. MaxConnections .............................................259 372 12.3. SendTargets ................................................259 373 12.4. TargetName .................................................260 374 12.5. InitiatorName ..............................................260 375 12.6. TargetAlias ................................................261 376 12.7. InitiatorAlias .............................................261 377 12.8. TargetAddress ..............................................262 378 12.9. TargetPortalGroupTag .......................................263 379 12.10. InitialR2T ................................................264 380 12.11. ImmediateData .............................................264 381 12.12. MaxRecvDataSegmentLength ..................................265 382 12.13. MaxBurstLength ............................................266 383 12.14. FirstBurstLength ..........................................266 384 12.15. DefaultTime2Wait ..........................................267 385 12.16. DefaultTime2Retain ........................................267 386 12.17. MaxOutstandingR2T .........................................268 387 12.18. DataPDUInOrder ............................................268 388 12.19. DataSequenceInOrder .......................................269 389 12.20. ErrorRecoveryLevel ........................................270 390 12.21. SessionType ...............................................270 391 12.22. The Private or Public Extension Key Format ................271 392 12.23. Task Reporting ............................................271 393 12.24. X#NodeArchitecture ........................................272 394 12.24.1. Definition ............................................272 395 12.24.2. Implementation Requirements ...........................273 396 13. IANA Considerations............................................275 397 1. Introduction 399 The Small Computer Systems Interface (SCSI) is a popular family of 400 protocols for communicating with I/O devices, especially storage 401 devices. SCSI is a client-server architecture. Clients of a SCSI 402 interface are called "initiators". Initiators issue SCSI 403 "commands" to request services from components, logical units of a 404 server known as a "target". A "SCSI transport" maps the client- 405 server SCSI protocol to a specific interconnect. An Initiator is 406 one endpoint of a SCSI transport and a target is the other 407 endpoint. 409 The SCSI protocol has been mapped over various transports, 410 including Parallel SCSI, IPI, IEEE-1394 (firewire) and Fibre 411 Channel. These transports are I/O specific and have limited 412 distance capabilities. 414 The iSCSI protocol defined in this document describes a means of 415 transporting of the SCSI packets over TCP/IP, providing for an 416 interoperable solution which can take advantage of existing 417 Internet infrastructure, Internet management facilities and address 418 distance limitations. 420 2. Definitions and Acronyms 422 2.1. Definitions 424 - Alias: An alias string can also be associated with an iSCSI Node. 425 The alias allows an organization to associate a user-friendly 426 string with the iSCSI Name. However, the alias string is not a 427 substitute for the iSCSI Name. 429 - CID (Connection ID): Connections within a session are identified 430 by a connection ID. It is a unique ID for this connection within 431 the session for the initiator. It is generated by the initiator and 432 presented to the target during login requests and during logouts 433 that close connections. 435 - Connection: A connection is a TCP connection. Communication 436 between the initiator and target occurs over one or more TCP 437 connections. The TCP connections carry control messages, SCSI 438 commands, parameters, and data within iSCSI Protocol Data Units 439 (iSCSI PDUs). 441 - I/O Buffer:A buffer that is used in a SCSI Read or Write 442 operation so SCSI data may be sent from or received into that 443 buffer. For a read or write data transfer to take place for a task, 444 an I/O Buffer is required on the initiator and at least one is 445 required on the 446 target. 448 - INCITS: INCITS stands for InterNational Committee of Information 449 Technology Standards. The INCITS has a broad standardization scope 450 within the field of Information and Communications Technologies 451 (ICT), encompassing storage, processing, transfer, display, 452 management, organization, and retrieval of information. INCITS 453 serves as ANSIs Technical Advisory Group for the ISO/IEC Joint 454 Technical Committee 1 (JTC 1). See http://www.incits.org. 456 - InfiniBand: An I/O architecture originally intended to replace 457 PCI and to address high performance server interconnectivity [IB]. 459 - iSCSI Device: A SCSI Device using an iSCSI service delivery 460 subsystem. Service Delivery Subsystem is defined by [SAM2] as a 461 transport mechanism for SCSI commands and responses. 463 - iSCSI Initiator Name: The iSCSI Initiator Name specifies the 464 worldwide unique name of the initiator. 466 - iSCSI Initiator Node: The "initiator" device. The word 467 "initiator" has been appropriately qualified as either a port or a 468 device in the rest of the document when the context is ambiguous. 469 All unqualified usages of "initiator" refer to an initiator port 470 (or device) depending on the context. 472 - iSCSI Layer: This layer builds/receives iSCSI PDUs and 473 relays/receives them to/from one or more TCP connections that form 474 an initiator-target "session". 476 - iSCSI Name: The name of an iSCSI initiator or iSCSI target. 478 - iSCSI Node: The iSCSI Node represents a single iSCSI initiator or 479 iSCSI target. There are one or more iSCSI Nodes within a Network 480 Entity. The iSCSI Node is accessible via one or more Network 481 Portals. An iSCSI Node is identified by its iSCSI Name. The 482 separation of the iSCSI Name from the addresses used by and for the 483 iSCSI Node allows multiple iSCSI nodes to use the same address, and 484 the same iSCSI node to use multiple addresses. 486 - iSCSI Target Name: The iSCSI Target Name specifies the worldwide 487 unique name of the target. 489 - iSCSI Target Node: The "target" device. The word "target" has 490 been appropriately qualified as either a port or a device in the 491 rest of the document when the context is ambiguous. All 492 unqualified usages of "target" refer to an target port (or device) 493 depending on the context. 495 - iSCSI Task: An iSCSI task is an iSCSI request for which a 496 response is expected. 498 - iSCSI Transfer Direction: The iSCSI transfer direction is defined 499 with regard to the initiator. Outbound or outgoing transfers are 500 transfers from the initiator to the target, while inbound or 501 incoming transfers are from the target to the initiator. 503 - ISID: The initiator part of the Session Identifier. It is 504 explicitly specified by the initiator during Login. 506 - I_T nexus: According to [SAM2], the I_T nexus is a relationship 507 between a SCSI Initiator Port and a SCSI Target Port. For iSCSI, 508 this relationship is a session, defined as a relationship between 509 an iSCSI Initiator's end of the session (SCSI Initiator Port) and 510 the iSCSI Target's Portal Group. The I_T nexus can be identified by 511 the conjunction of the SCSI port names; that is, the I_T nexus 512 identifier is the tuple (iSCSI Initiator Name + ',i,'+ ISID, iSCSI 513 Target Name + ',t,'+ Portal Group Tag). 515 - NAA: Network Address Authority, a naming format defined by the 516 INCITS T11 Fibre Channel protocols [FC-FS]. 518 - Network Entity: The Network Entity represents a device or gateway 519 that is accessible from the IP network. A Network Entity must have 520 one or more Network Portals, each of which can be used to gain 521 access to the IP network by some iSCSI Nodes contained in that 522 Network Entity. 524 - Network Portal: The Network Portal is a component of a Network 525 Entity that has a TCP/IP network address and that may be used by an 526 iSCSI Node within that Network Entity for the connection(s) within 527 one of its iSCSI sessions. A Network Portal in an initiator is 528 identified by its IP address. A Network Portal in a target is 529 identified by its IP address and its listening TCP port. 531 - Originator: In a negotiation or exchange, the party that 532 initiates the negotiation or exchange. 534 - PDU (Protocol Data Unit): The initiator and target divide their 535 communications into messages. The term "iSCSI protocol data unit" 536 (iSCSI PDU) is used for these messages. 538 - Portal Groups: iSCSI supports multiple connections within the 539 same session; some implementations will have the ability to combine 540 connections in a session across multiple Network Portals. A Portal 541 Group defines a set of Network Portals within an iSCSI Network 542 Entity that collectively supports the capability of coordinating a 543 session with connections spanning these portals. Not all Network 544 Portals within a Portal Group need participate in every session 545 connected through that Portal Group. One or more Portal Groups may 546 provide access to an iSCSI Node. Each Network Portal, as utilized 547 by a given iSCSI Node, belongs to exactly one portal group within 548 that node. 550 - Portal Group Tag: This 16-bit quantity identifies a Portal Group 551 within an iSCSI Node. All Network Portals with the same portal 552 group tag in the context of a given iSCSI Node are in the same 553 Portal Group. 555 - Recovery R2T: An R2T generated by a target upon detecting the 556 loss of one or more Data-Out PDUs through one of the following 557 means: a digest error, a sequence error, or a sequence reception 558 timeout. A recovery R2T carries the next unused R2TSN, but requests 559 all or part of the data burst that an earlier R2T (with a lower 560 R2TSN) had already requested. 562 - Responder: In a negotiation or exchange, the party that responds 563 to the originator of the negotiation or exchange. 565 - SAS: Serial Attached SCSI. The Serial Attached SCSI (SAS) 566 standard 567 contains both a physical layer compatible with Serial ATA, and 568 protocols for transporting SCSI commands to SAS devices and ATA 569 commands to SATA devices [SAS]. 571 - SCSI Device: This is the SAM2 term for an entity that contains 572 one or more SCSI ports that are connected to a service delivery 573 subsystem and supports a SCSI application protocol. For example, a 574 SCSI Initiator Device contains one or more SCSI Initiator Ports and 575 zero or more application clients. A Target Device contains one or 576 more SCSI Target Ports and one or more device servers and 577 associated logical units. For iSCSI, the SCSI Device is the 578 component within an iSCSI Node that provides the SCSI 579 functionality. As such, there can be, at most, one SCSI Device 580 within a given iSCSI Node. Access to the SCSI Device can only be 581 achieved in an iSCSI normal operational session. The SCSI Device 582 Name is defined to be the iSCSI Name of the node. 584 - SCSI Layer: This builds/receives SCSI CDBs (Command Descriptor 585 Blocks) and relays/receives them with the remaining command execute 586 [SAM2] parameters to/from the iSCSI Layer. 588 - Session: The group of TCP connections that link an initiator with 589 a target form a session (loosely equivalent to a SCSI I-T nexus). 590 TCP connections can be added and removed from a session. Across all 591 connections within a session, an initiator sees one and the same 592 target. 594 - SCSI Initiator Port: This maps to the endpoint of an iSCSI normal 595 operational session. An iSCSI normal operational session is 596 negotiated through the login process between an iSCSI initiator 597 node and an iSCSI target node. At successful completion of this 598 process, a SCSI Initiator Port is created within the SCSI Initiator 599 Device. The SCSI Initiator Port Name and SCSI Initiator Port 600 Identifier are both defined to be the iSCSI Initiator Name together 601 with (a) a label that identifies it as an initiator port 602 name/identifier and (b) the ISID portion of the session identifier. 604 - SCSI Port: This is the SAM2 term for an entity in a SCSI Device 605 that provides the SCSI functionality to interface with a service 606 delivery subsystem. For iSCSI, the definition of the SCSI Initiator 607 Port and the SCSI Target Port are different. 609 - SCSI Port Name: A name made up as UTF-8 characters and includes 610 the iSCSI Name + 'i' or 't' + ISID or Portal Group Tag. 612 - SCSI-Presented Data Transfer Length (SPDTL): SPDTL is the 613 aggregate data length of the data that the SCSI layer logically 614 "presents" to the iSCSI layer for a Data-In or Data-Out transfer in 615 the context of a SCSI task. For a bidirectional task, there are two 616 SPDTL values -- one for Data-In and one for Data-Out. Note that the 617 notion of "presenting" includes immediate data per the data 618 transfer model in [SAM2], and excludes overlapping data transfers, 619 if any, requested by the SCSI layer. 621 - SCSI Target Port: This maps to an iSCSI Target Portal Group. 623 - SCSI Target Port Name and SCSI Target Port Identifier: These are 624 both defined to be the iSCSI Target Name together with (a) a label 625 that identifies it as a target port name/identifier and (b) the 626 portal group tag. 628 - SRP: SCSI RDMA Protocol. SRP defines a SCSI protocol mapping onto 629 the InfiniBand (tm) Architecture and/or functionally similar Remote 630 DMA-capable protocols [SRP]. 632 - SSID (Session ID): A session between an iSCSI initiator and an 633 iSCSI target is defined by a session ID that is a tuple composed of 634 an initiator part (ISID) and a target part (Target Portal Group 635 Tag). The ISID is explicitly specified by the initiator at session 636 establishment. The Target Portal Group Tag is implied by the 637 initiator through the selection of the TCP endpoint at connection 638 establishment. The TargetPortalGroupTag key must also be returned 639 by the target as a confimation during connection establishment. 641 - T10: A technical committee within INCITS that develops standards 642 and technical reports on I/O interfaces, particularly the series of 643 SCSI (Small Computer Systems Interface) standards. See 644 http://www.t10.org. 646 - T11: A technical committee within INCITS responsible for 647 standards development in the areas of Intelligent Peripheral 648 Interface (IPI), High-Performance Parallel Interface (HIPPI) and 649 Fibre Channel (FC). See http://www.t11.org. 651 - Target Portal Group Tag: A numerical identifier (16-bit) for an 652 iSCSI Target Portal Group. 654 - Third-party: A term used in this document to denote nexus objects 655 (I_T or I_T_L) and iSCSI sessions that reap the side effects of 656 actions that take place in the context of a separate iSCSI session, 657 while being third parties to the action that caused the side 658 effects. One example of a third-party session is an iSCSI session 659 hosting an I_T_L nexus to an LU that is reset with an LU Reset TMF 660 via a separate I_T nexus. 662 - TSIH (Target Session Identifying Handle): A target assigned tag 663 for a session with a specific named initiator. The target generates 664 it during session establishment. Other than defining it as a 16 bit 665 binary string, its internal format and content are not defined by 666 this protocol but for the all 0 value that is reserved and used by 667 the initiator to indicate a new session. It is given to the target 668 during additional connection establishment for the same session. 670 2.2. Acronyms 672 Acronym Definition 673 -------------------------------------------------------------- 674 3DES Triple Data Encryption Standard 675 ACA Auto Contingent Allegiance 676 AEN Asynchronous Event Notification 677 AES Advanced Encryption Standard 678 AH Additional Header (not the IPsec AH!) 679 AHS Additional Header Segment 680 API Application Programming Interface 681 ASC Additional Sense Code 682 ASCII American Standard Code for Information Interchange 683 ASCQ Additional Sense Code Qualifier 684 BHS Basic Header Segment 685 CBC Cipher Block Chaining 686 CD Compact Disk 687 CDB Command Descriptor Block 688 CHAP Challenge Handshake Authentication Protocol 689 CID Connection ID 690 CO Connection Only 691 CRC Cyclic Redundancy Check 692 CRL Certificate Revocation List 693 CSG Current Stage 694 CSM Connection State Machine 695 DES Data Encryption Standard 696 DNS Domain Name Server 697 DOI Domain of Interpretation 698 DVD Digital Versatile Disk 699 EDTL Expected Data Transfer Length 700 ESP Encapsulating Security Payload 701 EUI Extended Unique Identifier 702 FFP Full Feature Phase 703 FFPO Full Feature Phase Only 704 FIM Fixed Interval Marker 705 Gbps Gigabits per Second 706 HBA Host Bus Adapter 707 HMAC Hashed Message Authentication Code 709 I_T Initiator_Target 710 I_T_L Initiator_Target_LUN 711 IANA Internet Assigned Numbers Authority 712 IB InfiniBand 713 ID Identifier 714 IDN Internationalized Domain Name 715 IEEE Institute of Electrical & Electronics Engineers 716 IETF Internet Engineering Task Force 717 IKE Internet Key Exchange 718 I/O Input Output 719 IO Initialize Only 720 IP Internet Protocol 721 IPsec Internet Protocol Security 722 IPv4 Internet Protocol Version 4 723 IPv6 Internet Protocol Version 6 724 IQN iSCSI Qualified Name 725 iSCSI Internet SCSI 726 iSER iSCSI Extensions for RDMA 727 ISID Initiator Session ID 728 ITN iSCSI Target Name 729 ITT Initiator Task Tag 730 KRB5 Kerberos V5 731 LFL Lower Functional Layer 732 LTDS Logical-Text-Data-Segment 733 LO Leading Only 734 LU Logical Unit 735 LUN Logical Unit Number 736 MAC Message Authentication Codes 737 NA Not Applicable 738 NAA Network Address Authority 739 NIC Network Interface Card 740 NOP No Operation 741 NSG Next Stage 742 OS Operating System 743 PDU Protocol Data Unit 744 PKI Public Key Infrastructure 745 R2T Ready To Transfer 746 R2TSN Ready To Transfer Sequence Number 747 RDMA Remote Direct Memory Access 748 RFC Request For Comments 749 SAM SCSI Architecture Model 750 SAM2 SCSI Architecture Model - 2 751 SAN Storage Area Network 752 SAS Serial Attached SCSI 753 SCSI Small Computer Systems Interface 754 SN Sequence Number 755 SNACK Selective Negative Acknowledgment - also 756 Sequence Number Acknowledgement for data 757 SPDTL SCSI-Presented Data Transfer Length 758 SPKM Simple Public-Key Mechanism 759 SRP Secure Remote Password, also SCSI RDMA Protocol 760 SSID Session ID 761 SW Session Wide 762 TCB Task Control Block 763 TCP Transmission Control Protocol 764 TMF Task Management Function 765 TPGT Target Portal Group Tag 766 TSIH Target Session Identifying Handle 767 TTT Target Transfer Tag 768 UFL Upper Functional Layer 769 ULP Upper Level Protocol 770 URN Uniform Resource Names 771 UTF Universal Transformation Format 772 WG Working Group 774 2.3. Conventions 776 In examples, "I->" and "T->" show iSCSI PDUs sent by the initiator 777 and target respectively. 779 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 780 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in 781 this document are to be interpreted as described in BCP 14 782 [RFC2119]. 784 iSCSI messages - PDUs - are represented by diagrams as in the 785 following example: 787 Byte/ 0 | 1 | 2 | 3 | 788 / | | | | 789 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 790 +---------------+---------------+---------------+---------------+ 791 0| Basic Header Segment (BHS) | 792 +---------------+---------------+---------------+---------------+ 793 ---------- 794 +| | 795 +---------------+---------------+---------------+---------------+ 797 The diagrams include byte and bit numbering. 799 The following representation and ordering rules are observed in 800 this document: 802 - Word Rule 804 - Half-word Rule 806 - Byte Rule 808 2.3.1. Word Rule 810 A word holds four consecutive bytes. Whenever a word has numeric 811 content, it is considered an unsigned number in base 2 positional 812 representation with the lowest numbered byte (e.g., byte 0) bit 0 813 representing 2**31 and bit 1 representing 2**30 through lowest 814 numbered byte + 3 (e.g., byte 3) bit 7 representing 2**0. 816 Decimal and hexadecimal representation of word values map this 817 representation to decimal or hexadecimal positional notation. 819 2.3.2. Half-Word Rule 821 A half-word holds two consecutive bytes. Whenever a half-word has 822 numeric content it is considered an unsigned number in base 2 823 positional representation with the lowest numbered byte (e.g., byte 824 0) bit 0 representing 2**15 and bit 1 representing 2**14 through 825 lowest numbered byte + 1 (e.g., byte 1) bit 7 representing 2**0. 827 Decimal and hexadecimal representation of half-word values map this 828 representation to decimal or hexadecimal positional notation. 830 2.3.3. Byte Rule 832 For every PDU, bytes are sent and received in increasing numbered 833 order (network order). 835 Whenever a byte has numerical content it is considered an unsigned 836 number in base 2 positional representation with bit 0 representing 837 2**7 and bit 1 representing 2**6 through bit 7 representing 2**0. 839 3. Overview 841 3.1. SCSI Concepts 843 The SCSI Architecture Model-2 [SAM2] describes in detail the 844 architecture of the SCSI family of I/O protocols. This section 845 provides a brief background of the SCSI architecture and is 846 intended to familiarize readers with its terminology. 848 At the highest level, SCSI is a family of interfaces for requesting 849 services from I/O devices, including hard drives, tape drives, CD 850 and DVD drives, printers, and scanners. In SCSI terminology, an 851 individual I/O device is called a "logical unit" (LU). 853 SCSI is a client-server architecture. Clients of a SCSI interface 854 are called "initiators". Initiators issue SCSI "commands" to 855 request services from components, logical units, of a server known 856 as a "target". The "device server" on the logical unit accepts SCSI 857 commands and processes them. 859 A "SCSI transport" maps the client-server SCSI protocol to a 860 specific interconnect. Initiator is one endpoint of a SCSI 861 transport. The "target" is the other endpoint. A target can contain 862 multiple Logical Units (LUs). Each Logical Unit has an address 863 within a target called a Logical Unit Number (LUN). 865 A SCSI task is a SCSI command or possibly a linked set of SCSI 866 commands. Some LUs support multiple pending (queued) tasks, but the 867 queue of tasks is managed by the logical unit. The target uses an 868 initiator provided "task tag" to distinguish between tasks. Only 869 one command in a task can be outstanding at any given time. 871 Each SCSI command results in an optional data phase and a required 872 response phase. In the data phase, information can travel from the 873 initiator to target (e.g., WRITE), target to initiator (e.g., 874 READ), or in both directions. In the response phase, the target 875 returns the final status of the operation, including any errors. 877 Command Descriptor Blocks (CDB) are the data structures used to 878 contain the command parameters that an initiator sends to a target. 879 The CDB content and structure is defined by [SAM2] and device-type 880 specific SCSI standards. 882 3.2. iSCSI Concepts and Functional Overview 884 The iSCSI protocol is a mapping of the SCSI command, event and task 885 management model (see [SAM2]) over the TCP protocol. SCSI commands 886 are carried by iSCSI requests and SCSI responses and status are 887 carried by iSCSI responses. iSCSI also uses the request response 888 mechanism for iSCSI protocol mechanisms. 890 For the remainder of this document, the terms "initiator" and 891 "target" refer to "iSCSI initiator node" and "iSCSI target node", 892 respectively (see iSCS) unless otherwise qualified. 894 In keeping with similar protocols, the initiator and target divide 895 their communications into messages. This document uses the term 896 "iSCSI protocol data unit" (iSCSI PDU) for these messages. 898 For performance reasons, iSCSI allows a "phase-collapse". A command 899 and its associated data may be shipped together from initiator to 900 target, and data and responses may be shipped together from 901 targets. 903 The iSCSI transfer direction is defined with respect to the 904 initiator. Outbound or outgoing transfers are transfers from an 905 initiator to a target, while inbound or incoming transfers are from 906 a target to an initiator. 908 An iSCSI task is an iSCSI request for which a response is expected. 910 In this document "iSCSI request", "iSCSI command", request, or 911 (unqualified) command have the same meaning. Also, unless otherwise 912 specified, status, response, or numbered response have the same 913 meaning. 915 3.2.1. Layers and Sessions 917 The following conceptual layering model is used to specify 918 initiator and target actions and the way in which they relate to 919 transmitted and received Protocol Data Units: 921 the SCSI layer builds/receives SCSI CDBs (Command Descriptor 922 Blocks) and passes/receives them with the remaining command 923 execute parameters ([SAM2]) to/from 924 the iSCSI layer that builds/receives iSCSI PDUs and 925 relays/receives them to/from one or more TCP connections; 926 the group of connections form an initiator-target "session". 928 Communication between the initiator and target occurs over one or 929 more TCP connections. The TCP connections carry control messages, 930 SCSI commands, parameters, and data within iSCSI Protocol Data 931 Units (iSCSI PDUs). The group of TCP connections that link an 932 initiator with a target form a session (equivalent to a SCSI I_T 933 nexus, see SCSI ). A session is defined by a session ID that is 934 composed of an initiator part and a target part. TCP connections 935 can be added and removed from a session. Each connection within a 936 session is identified by a connection ID (CID). 938 Across all connections within a session, an initiator sees one 939 "target image". All target identifying elements, such as LUN, are 940 the same. A target also sees one "initiator image" across all 941 connections within a session. Initiator-identifying elements, such 942 as the Initiator Task Tag, are global across the session regardless 943 of the connection on which they are sent or received. 945 iSCSI targets and initiators MUST support at least one TCP 946 connection and MAY support several connections in a session. For 947 error recovery purposes, targets and initiators that support a 948 single active connection in a session SHOULD support two 949 connections during recovery. 951 3.2.2. Ordering and iSCSI Numbering 953 iSCSI uses Command and Status numbering schemes and a Data 954 sequencing scheme. 956 Command numbering is session-wide and is used for ordered command 957 delivery over multiple connections. It can also be used as a 958 mechanism for command flow control over a session. 960 Status numbering is per connection and is used to enable missing 961 status detection and recovery in the presence of transient or 962 permanent communication errors. 964 Data sequencing is per command or part of a command (R2T-triggered 965 sequence) and is used to detect missing data and/or R2T PDUs due to 966 header digest errors. 968 Typically, fields in the iSCSI PDUs communicate the Sequence 969 Numbers between the initiator and target. During periods when 970 traffic on a connection is unidirectional, iSCSI NOP-Out/In PDUs 971 may be utilized to synchronize the command and status ordering 972 counters of the target and initiator. 974 The iSCSI session abstraction is equivalent to the SCSI I_T nexus, 975 and the iSCSI session provides an ordered command delivery from the 976 SCSI initiator to the SCSI target. For detailed design 977 considerations that led to the iSCSI session model as it is defined 978 here and how it relates the SCSI command ordering features defined 979 in SCSI specifications to the iSCSI concepts see [RFC3783]. 981 3.2.2.1. Command Numbering and Acknowledging 983 iSCSI performs ordered command delivery within a session. All 984 commands (initiator-to-target PDUs) in transit from the initiator 985 to the target are numbered. 987 iSCSI considers a task to be instantiated on the target in response 988 to every request issued by the initiator. A set of task management 989 operations including abort and reassign (see Section 10.5 "Task 990 Management Function Request") may be performed on any iSCSI task. 992 Some iSCSI tasks are SCSI tasks, and many SCSI activities are 993 related to a SCSI task ([SAM2]). In all cases, the task is 994 identified by the Initiator Task Tag for the life of the task. 996 The command number is carried by the iSCSI PDU as CmdSN (Command- 997 Sequence-Number). The numbering is session-wide. Outgoing iSCSI 998 PDUs carry this number. The iSCSI initiator allocates CmdSNs with a 999 32-bit unsigned counter (modulo 2**32). Comparisons and arithmetic 1000 on CmdSN use Serial Number Arithmetic as defined in [RFC1982] where 1001 SERIAL_BITS = 32. 1003 Commands meant for immediate delivery are marked with an immediate 1004 delivery flag; they MUST also carry the current CmdSN. CmdSN does 1005 not advance after a command marked for immediate delivery is sent. 1007 Command numbering starts with the first login request on the first 1008 connection of a session (the leading login on the leading 1009 connection) and command numbers are incremented by 1 for every non- 1010 immediate command issued afterwards. 1012 If immediate delivery is used with task management commands, these 1013 commands may reach the target before the tasks on which they are 1014 supposed to act. However their CmdSN serves as a marker of their 1015 position in the stream of commands. The initiator and target must 1016 ensure that the task management commands act as specified by 1017 [SAM2]. For example, both commands and responses appear as if 1018 delivered in order. Whenever CmdSN for an outgoing PDU is not 1019 specified by an explicit rule, CmdSN will carry the current value 1020 of the local CmdSN variable (see later in this section). 1022 The means by which an implementation decides to mark a PDU for 1023 immediate delivery or by which iSCSI decides by itself to mark a 1024 PDU for immediate delivery are beyond the scope of this document. 1026 The number of commands used for immediate delivery is not limited 1027 and their delivery to execution is not acknowledged through the 1028 numbering scheme. Immediate commands MAY be rejected by the iSCSI 1029 target layer due to lack of resources. An iSCSI target MUST be able 1030 to handle at least one immediate task management command and one 1031 immediate non-task-management iSCSI command per connection at any 1032 time. 1034 In this document, delivery for execution means delivery to the SCSI 1035 execution engine or an iSCSI protocol specific execution engine 1036 (e.g., for text requests with public or private extension keys 1037 involving an execution component). With the exception of the 1038 commands marked for immediate delivery, the iSCSI target layer MUST 1039 deliver the commands for execution in the order specified by CmdSN. 1040 Commands marked for immediate delivery may be delivered by the 1041 iSCSI target layer for execution as soon as detected. iSCSI may 1042 avoid delivering some commands to the SCSI target layer if required 1043 by a prior SCSI or iSCSI action (e.g., CLEAR TASK SET Task 1044 Management request received before all the commands on which it was 1045 supposed to act). 1047 On any connection, the iSCSI initiator MUST send the commands in 1048 increasing order of CmdSN, except for commands that are 1049 retransmitted due to digest error recovery and connection recovery. 1051 For the numbering mechanism, the initiator and target maintain the 1052 following three variables for each session: 1054 - CmdSN - the current command Sequence Number, advanced by 1 1055 on each command shipped except for commands marked for 1056 immediate delivery. CmdSN always contains the number to be 1057 assigned to the next Command PDU. 1059 - ExpCmdSN - the next expected command by the target. The 1060 target acknowledges all commands up to, but not including, 1061 this number. The initiator treats all commands with CmdSN 1062 less than ExpCmdSN as acknowledged. The target iSCSI layer 1063 sets the ExpCmdSN to the largest non-immediate CmdSN that it 1064 can deliver for execution "plus 1" per [RFC1982]. There MUST 1065 NOT be any holes in the acknowledged CmdSN sequence. 1067 - MaxCmdSN - the maximum number to be shipped. The queuing 1068 capacity of the receiving iSCSI layer is MaxCmdSN - ExpCmdSN 1069 + 1. 1071 The initiators ExpCmdSN and MaxCmdSN are derived from target-to- 1072 initiator PDU fields. Comparisons and arithmetic on ExpCmdSN and 1073 MaxCmdSN MUST use Serial Number Arithmetic as defined in [RFC1982] 1074 where SERIAL_BITS = 32. 1076 The target MUST NOT transmit a MaxCmdSN that is less than ExpCmdSN- 1077 1. For non-immediate commands, the CmdSN field can take any value 1078 from ExpCmdSN to MaxCmdSN inclusive. The target MUST silently 1079 ignore any non-immediate command outside of this range or non- 1080 immediate duplicates within the range. The CmdSN carried by 1081 immediate commands may lie outside the ExpCmdSN to MaxCmdSN range. 1082 For example, if the initiator has previously sent a non-immediate 1083 command carrying the CmdSN equal to MaxCmdSN, the target window is 1084 closed. For group task management commands issued as immediate 1085 commands, CmdSN indicates the scope of the group action (e.g., on 1086 ABORT TASK SET indicates which commands are to be aborted). 1088 MaxCmdSN and ExpCmdSN fields are processed by the initiator as 1089 follows: 1091 -If the PDU MaxCmdSN is less than the PDU ExpCmdSN-1 (in Serial 1092 Arithmetic Sense), they are both ignored. 1094 -If the PDU MaxCmdSN is greater than the local MaxCmdSN (in 1095 Serial Arithmetic Sense), it updates the local MaxCmdSN; 1096 otherwise, it is ignored. 1098 -If the PDU ExpCmdSN is greater than the local ExpCmdSN (in 1099 Serial Arithmetic Sense), it updates the local ExpCmdSN; 1100 otherwise, it is ignored. 1102 This sequence is required because updates may arrive out of order 1103 (e.g., the updates are sent on different TCP connections). 1105 iSCSI initiators and targets MUST support the command numbering 1106 scheme. 1108 A numbered iSCSI request will not change its allocated CmdSN, 1109 regardless of the number of times and circumstances in which it is 1110 reissued (see Section 6.2.1 "Usage of Retry"). At the target, CmdSN 1111 is only relevant while the command has not created any state 1112 related to its execution (execution state); afterwards, CmdSN 1113 becomes irrelevant. Testing for the execution state (represented by 1114 identifying the Initiator Task Tag) MUST precede any other action 1115 at the target. If no execution state is found, it is followed by 1116 ordering and delivery. If an execution state is found, it is 1117 followed by delivery if it has not already been delivered. 1119 If an initiator issues a command retry for a command with CmdSN R 1120 on 1121 a connection when the session CmdSN value is Q, it MUST NOT advance 1122 the CmdSN past R + 2**31 -1 unless the connection is no longer 1123 operational (i.e., it has returned to the FREE state, see Section 1124 7.1.3 "Standard Connection State Diagram for an Initiator"), the 1125 connection has been reinstated (see Section 5.3.4 "Connection 1126 Reinstatement"), or a non-immediate command with CmdSN equal or 1127 greater than Q was issued subsequent to the command retry on the 1128 same connection and the reception of that command is acknowledged 1129 by the target (see Section 9.4 "Command Retry and Cleaning Old 1130 Command Instances"). 1132 A target command response or Data-in PDU with status MUST NOT 1133 precede the command acknowledgement. However, the acknowledgement 1134 MAY be included in the response or the Data-in PDU. 1136 3.2.2.2. Response/Status Numbering and Acknowledging 1138 Responses in transit from the target to the initiator are numbered. 1139 The StatSN (Status Sequence Number) is used for this purpose. 1140 StatSN is a counter maintained per connection. ExpStatSN is used by 1141 the initiator to acknowledge status. The status sequence number 1142 space is 32-bit unsigned-integers and the arithmetic operations are 1143 the regular mod(2**32) arithmetic. 1145 Status numbering starts with the Login response to the first Login 1146 request of the connection. The Login response includes an initial 1147 value for status numbering (any initial value is valid). 1149 To enable command recovery, the target MAY maintain enough state 1150 information for data and status recovery after a connection 1151 failure. A target doing so can safely discard all of the state 1152 information maintained for recovery of a command after the delivery 1153 of the status for the command (numbered StatSN) is acknowledged 1154 through ExpStatSN. 1156 A large absolute difference between StatSN and ExpStatSN may 1157 indicate a failed connection. Initiators MUST undertake recovery 1158 actions if the difference is greater than an implementation defined 1159 constant that MUST NOT exceed 2**31-1. 1161 Initiators and Targets MUST support the response-numbering scheme. 1163 3.2.2.3. Response Ordering 1165 3.2.2.3.1. Need for Response Ordering 1166 Whenever an iSCSI session is composed of multiple connections, the 1167 Response PDUs (task responses or TMF responses) originating in the 1168 target SCSI layer are distributed onto the multiple connections by 1169 the target iSCSI layer according to iSCSI connection allegiance 1170 rules. This process generally may not preserve the ordering of the 1171 responses by the time they are delivered to the initiator SCSI 1172 layer. 1174 Since ordering is not expected across SCSI responses anyway, this 1175 approach works fine in the general case. However, to address the 1176 special cases where some ordering is desired by the SCSI layer, the 1177 following "Response Fence" semantics are defined with respect to 1178 handling SCSI response messages as they are handed off from the 1179 SCSI protocol layer to the iSCSI layer. 1181 3.2.2.3.2. Response Ordering Model Description 1183 The target SCSI protocol layer hands off the SCSI response messages 1184 to the target iSCSI layer by invoking the "Send Command Complete" 1185 protocol data service ([SAM2], clause 5.4.2) and "Task Management 1186 Function Executed" ([SAM2], clause 6.9) service. On receiving the 1187 SCSI response message, the iSCSI layer exhibits the Response Fence 1188 behavior for certain SCSI response messages (Section 3.2.2.3.4 1189 describes the specific instances where the semantics must be 1190 realized). 1192 Whenever the Response Fence behavior is required for a SCSI 1193 response message, the target iSCSI layer MUST ensure that the 1194 following conditions are met in delivering the response message to 1195 the initiator iSCSI layer: 1197 Response with Response Fence MUST be delivered chronologically 1198 after all the "preceding" responses on the I_T_L nexus, if 1199 the preceding responses are delivered at all, to the 1200 initiator iSCSI layer. 1201 Response with Response Fence MUST be delivered chronologically 1202 prior to all the "following" responses on the I_T_L nexus. 1204 The "preceding" and "following" notions refer to the order of 1205 handoff of a response message from the target SCSI protocol layer 1206 to the target iSCSI layer. 1208 3.2.2.3.3. iSCSI Semantics with the Interface Model 1210 Whenever the TaskReporting key (Section 12.23 "Task Reporting") is 1211 negotiated to ResponseFence or FastAbort for an iSCSI session and 1212 the Response Fence behavior is required for a SCSI response 1213 message, the target iSCSI layer MUST perform the actions described 1214 in this section for that session. 1216 If it is a single-connection session, no special processing is 1217 required. The standard SCSI Response PDU build and dispatch 1218 process happens. 1219 If it is a multi-connection session, the target iSCSI layer 1220 takes note of the last-sent and unacknowledged StatSN on 1221 each of the connections in the iSCSI session, and waits for 1222 an acknowledgement (NOP-In PDUs MAY be used to solicit 1223 acknowledgements as needed in order to accelerate this 1224 process) of each such StatSN to clear the fence. The SCSI 1225 response requiring Response Fence behavior MUST NOT be sent 1226 to the initiator before acknowledgements are received for 1227 each of the unacknowledged StatSNs. 1228 The target iSCSI layer must wait for an acknowledgement of the 1229 SCSI Response PDU that carried the SCSI response requiring 1230 the Response Fence behavior. The fence MUST be considered 1231 cleared only after receiving the acknowledgement. 1232 All further status processing for the LU is resumed only after 1233 clearing the fence. If any new responses for the I_T_L nexus 1234 are received from the SCSI layer before the fence is 1235 cleared, those Response PDUs MUST be held and queued at the 1236 iSCSI layer until the fence is cleared. 1238 3.2.2.3.4. Current List of Fenced Response Use Cases 1240 This section lists the fenced response use cases that iSCSI 1241 implementations MUST comply with. However, this is not an 1242 exhaustive enumeration. It is expected that as SCSI protocol 1243 specifications evolve, the specifications will specify when 1244 response fencing is required on a case-by-case basis. 1246 Whenever the TaskReporting key (Section 12.23) is negotiated to 1247 ResponseFence or FastAbort for an iSCSI session, the target iSCSI 1248 layer MUST assume that the Response Fence is required for the 1249 following SCSI completion messages: 1251 1. The first completion message carrying the UA after the 1252 multi-task abort on issuing and third-party sessions. See 1253 Section 3.2.3.2 for related TMF discussion. 1255 2. The TMF Response carrying the multi-task TMF Response on the 1256 issuing session. 1258 3. The completion message indicating ACA establishment on the 1259 issuing session. 1261 4. The first completion message carrying the ACA ACTIVE status 1262 after ACA establishment on issuing and third-party sessions. 1264 5. The TMF Response carrying the Clear ACA response on the 1265 issuing session. 1267 6. The response to a PERSISTENT RESERVE OUT/PREEMPT AND ABORT 1268 command. 1270 Note: 1271 - Due to the absence of ACA-related fencing requirements in 1272 [RFC3720], initiator implementations SHOULD NOT use ACA on 1273 multi-connection iSCSI sessions with targets complying only 1274 with [RFC3720]. 1276 - Initiators that want to employ ACA on multi-connection iSCSI 1277 sessions SHOULD first assess response-fencing behavior via 1278 negotiating for ResponseFence or FastAbort values for the 1279 TaskReporting (Section 12.23) key. 1281 3.2.2.4. Data Sequencing 1283 Data and R2T PDUs transferred as part of some command execution 1284 MUST be sequenced. The DataSN field is used for data sequencing. 1285 For input (read) data PDUs, DataSN starts with 0 for the first data 1286 PDU of an input command and advances by 1 for each subsequent data 1287 PDU. For output data PDUs, DataSN starts with 0 for the first data 1288 PDU of a sequence (the initial unsolicited sequence or any data PDU 1289 sequence issued to satisfy an R2T) and advances by 1 for each 1290 subsequent data PDU. R2Ts are also sequenced per command. For 1291 example, the first R2T has an R2TSN of 0 and advances by 1 for each 1292 subsequent R2T. For bidirectional commands, the target uses the 1293 DataSN/R2TSN to sequence Data-In and R2T PDUs in one continuous 1294 sequence (undifferentiated). Unlike command and status, data PDUs 1295 and R2Ts are not acknowledged by a field in regular outgoing PDUs. 1296 Data-In PDUs can be acknowledged on demand by a special form of the 1297 SNACK PDU. Data and R2T PDUs are implicitly acknowledged by status 1298 for the command. The DataSN/R2TSN field enables the initiator to 1299 detect missing data or R2T PDUs. 1301 For any read or bidirectional command, a target MUST issue less 1302 than 2**32 combined R2T and Data-In PDUs. Any output data sequence 1303 MUST contain less than 2**32 Data-Out PDUs. 1305 3.2.3. iSCSI Task Management 1307 3.2.3.1. Task Management Overview 1309 iSCSI task management features allow an initiator to control the 1310 active iSCSI tasks on an operational iSCSI session that it has with 1311 an iSCSI target. Section 10.5 defines the task management function 1312 types that this specification defines - ABORT TASK, ABORT TASK SET, 1313 CLEAR ACA, CLEAR TASK SET, LOGICAL UNIT RESET, TARGET WARM RESET, 1314 TARGET COLD RESET, and TASK REASSIGN. 1316 Out of these function types, ABORT TASK and TASK REASSIGN functions 1317 manage a single active task, whereas ABORT TASK SET, CLEAR TASK 1318 SET, LOGICAL UNIT RESET, TARGET WARM RESET and TARGET COLD RESET 1319 functions can each potentially affect multiple active tasks. 1321 3.2.3.2. Notion of Affected Tasks 1323 This section defines the notion of "affected tasks" in multi-task 1324 abort scenarios. Scope definitions in this section apply to both 1325 the Standard Multi-task Abort semantics (Section 3.2.3.3) and the 1326 FastAbort Multi-task Abort semantics behavior (Section 3.2.3.4). 1328 ABORT TASK SET: All outstanding tasks for the I_T_L nexus 1329 identified by the LUN field in the ABORT TASK SET TMF Request PDU 1330 (Section 10.5). 1332 CLEAR TASK SET: All outstanding tasks in the task set for the LU 1333 identified by the LUN field in the CLEAR TASK SET TMF Request PDU. 1334 See [SPC3] for the definition of a "task set". 1336 LOGICAL UNIT RESET: All outstanding tasks from all initiators for 1337 the LU identified by the LUN field in the LOGICAL UNIT RESET 1338 Request PDU. 1340 TARGET WARM RESET/TARGET COLD RESET: All outstanding tasks from all 1341 initiators across all LUs to which the TMF-issuing session has 1342 access on the SCSI target device hosting the iSCSI session. 1344 Usage: An "ABORT TASK SET TMF Request PDU" in the preceding text is 1345 an iSCSI TMF Request PDU with the "Function" field set to "ABORT 1346 TASK SET" as defined in Section 10.5. Similar usage is employed 1347 for other scope descriptions. 1349 3.2.3.3. Standard Multi-task Abort Semantics 1351 All iSCSI implementations MUST support the protocol behavior 1352 defined in this section as the default behavior. The execution of 1353 ABORT TASK SET, CLEAR TASK SET, LOGICAL UNIT RESET, TARGET WARM 1354 RESET, and TARGET COLD RESET TMF Requests consists of the following 1355 sequence of actions in the specified order on the specified party. 1357 The initiator iSCSI layer: 1358 a. MUST continue to respond to each TTT received for the 1359 affected tasks. 1360 b. SHOULD process any responses received for affected tasks in 1361 the normal fashion. This is acceptable because the responses 1362 are guaranteed to have been sent prior to the TMF response. 1363 c. SHOULD receive the TMF Response concluding all the tasks in 1364 the set of affected tasks unless the initiator has done 1365 something (e.g., LU reset, connection drop) that may prevent 1366 the TMF Response from being sent or received. The initiator 1367 MUST thus conclude all affected tasks as part of this step 1368 in either case, and MUST discard any TMF Response received 1369 after the affected tasks are concluded. 1371 The target iSCSI layer: 1373 a. MUST wait for responses on currently valid target-transfer 1374 tags of the affected tasks from the issuing initiator. MAY 1375 wait for responses on currently valid target-transfer tags 1376 of the affected tasks from third-party initiators. 1377 b. MUST wait (concurrent with the wait in Step a) for all 1378 commands of the affected tasks to be received based on the 1379 CmdSN ordering. SHOULD NOT wait for new commands on third- 1380 party affected sessions -- only the instantiated tasks have 1381 to be considered for the purpose of determining the affected 1382 tasks. In the case of target-scoped requests (i.e., TARGET 1383 WARM RESET and TARGET COLD RESET), all of the commands that 1384 are not yet received on the issuing session in the command 1385 stream however can be considered to have been received with 1386 no command waiting period -- i.e., the entire CmdSN space up 1387 to the CmdSN of the task management function can be 1388 "plugged". 1389 c. MUST propagate the TMF request to and receive the response 1390 from the target SCSI layer. 1391 d. MUST provide the Response Fence behavior for the TMF 1392 Response on the issuing session as specified in Section 1393 3.2.2.3.2. 1394 e. MUST provide the Response Fence behavior on the first post- 1395 TMF Response on third-party sessions as specified in Section 1396 3.2.2.3.3. If some tasks originate from non-iSCSI I_T_L 1397 nexuses, then the means by which the target ensures that all 1398 affected tasks have returned their status to the initiator 1399 are defined by the specific non-iSCSI transport protocol(s). 1401 Technically, the TMF servicing is complete in Step d. Data 1402 transfers corresponding to terminated tasks may however still be in 1403 progress on third-party iSCSI sessions even at the end of Step e. 1404 The TMF Response MUST NOT be sent by the target iSCSI layer before 1405 the end of Step d, and MAY be sent at the end of Step d despite 1406 these outstanding data transfers until after Step e. 1408 3.2.3.4. FastAbort Multi-task Abort Semantics 1410 Protocol behavior defined in this section MUST be implemented by 1411 all iSCSI implementations complying with this document. Protocol 1412 behavior defined in this section MUST be exhibited by iSCSI 1413 implementations on an iSCSI session when they negotiate the 1414 TaskReporting (Section 12.23) key to "FastAbort" on that session. 1415 The execution of ABORT TASK SET, CLEAR TASK SET, LOGICAL UNIT 1416 RESET, TARGET WARM RESET, and TARGET COLD RESET TMF Requests 1417 consists of the following sequence of actions in the specified 1418 order on the specified party. 1420 The initiator iSCSI layer: 1421 a. MUST NOT send any more Data-Out PDUs for affected tasks on 1422 the issuing connection of the issuing iSCSI session once the 1423 TMF is sent to the target. 1424 b. SHOULD process any responses received for affected tasks in 1425 the normal fashion. This is acceptable because the responses 1426 are guaranteed to have been sent prior to the TMF response. 1427 c. MUST respond to each Async Message PDU with FAST_ABORT 1428 AsyncEvent as defined in Section 10.9. 1429 d. MUST treat the TMF response as terminating all affected 1430 tasks for which responses have not been received, and MUST 1431 discard any responses for affected tasks received after the 1432 TMF response is passed to the SCSI layer (although the 1433 semantics defined in this section ensure that such an out- 1434 of-order scenario will never happen with a compliant target 1435 implementation). 1437 The target iSCSI layer: 1438 a. MUST wait for all commands of the affected tasks to be 1439 received based on the CmdSN ordering on the issuing session. 1440 SHOULD NOT wait for new commands on third-party affected 1441 sessions only the instantiated tasks have to be considered 1442 for the purpose of determining the affected tasks. In the 1443 case of target-scoped requests (i.e., TARGET WARM RESET and 1444 TARGET COLD RESET), all the commands that are not yet 1445 received on the issuing session in the command stream can be 1446 considered to have been received with no command waiting 1447 period -- i.e., the entire CmdSN space up to the CmdSN of 1448 the task management function can be "plugged". 1449 b. MUST propagate the TMF request to and receive the response 1450 from the target SCSI layer. 1451 c. MUST leave all active "affected TTTs" (i.e., active TTTs 1452 associated with affected tasks) valid. 1454 d. MUST send an Asynchronous Message PDU with AsyncEvent=5 1455 (Section 10.9) on: 1456 i) each connection of each third-party session to which 1457 at least one affected task is allegiant if 1458 TaskReporting=FastAbort is operational on that third- 1459 party session, and, 1460 ii) each connection except the issuing connection of the 1461 issuing session that has at least one allegiant affected 1462 task. 1463 If there are multiple affected LUs (say, due to a target 1464 reset), then one Async Message PDU MUST be sent for each such 1465 LU on each connection that has at least one allegiant 1466 affected task. The LUN field in the Asynchronous Message PDU 1467 MUST be set to match the LUN for each such LU. 1468 e. MUST address the Response Fence flag on the TMF Response on 1469 the issuing session as defined in Section 3.2.2.3.3. 1470 f. MUST address the Response Fence flag on the first post-TMF 1471 Response on third-party sessions as defined in Section 1472 3.2.2.3.3. If some tasks originate from non-iSCSI I_T_L 1473 nexuses, then the means by which the target ensures that all 1474 affected tasks have returned their status to the initiator 1475 are defined by the specific non-iSCSI transport protocol(s). 1476 g. MUST free up the affected TTTs (and STags, if applicable) 1477 and the corresponding buffers, if any, once it receives each 1478 associated NOP-Out acknowledgement that the initiator 1479 generated in response to each Async Message. 1481 Technically, the TMF servicing is complete in Step e. Data 1482 transfers corresponding to terminated tasks may however still be in 1483 progress even at the end of Step f. A TMF Response MUST NOT be 1484 sent by the target iSCSI layer before the end of Step e, and MAY be 1485 sent at the end of Step e despite these outstanding Data transfers 1486 until Step g. Step g specifies an event to free up any such 1487 resources that may have been reserved to support outstanding data 1488 transfers. 1490 3.2.3.5. Affected Tasks Shared across Standard and FastAbort Sessions 1492 If an iSCSI target implementation is capable of supporting 1493 TaskReporting=FastAbort functionality (Section 12.23), it may end 1494 up in a situation where some sessions have TaskReporting=RFC3720 1495 operational (RFC 3720 sessions) while some other sessions have 1496 TaskReporting=FastAbort operational (FastAbort sessions) even while 1497 accessing a shared set of affected tasks (Section 3.2.3.2). If the 1498 issuing session is an RFC 3720 session, the iSCSI target 1499 implementation is FastAbort-capable, and the third-party affected 1500 session is a FastAbort session, the following behavior SHOULD be 1501 exhibited by the iSCSI target layer: 1502 a. Between Steps c and d of the target behavior in Section 1503 3.2.3.3, send an Asynchronous Message PDU with AsyncEvent=5 1504 (Section 10.9) on each connection of each third-party 1505 session to which at least one affected task is allegiant. If 1506 there are multiple affected LUs, then send one Async Message 1507 PDU for each such LU on each connection that has at least 1508 one allegiant affected task. When sent, the LUN field in the 1509 Asynchronous Message PDU MUST be set to match the LUN for 1510 each such LU. 1511 b. After Step e of the target behavior in Section 3.2.3.3, free 1512 up the affected TTTs (and STags, if applicable) and the 1513 corresponding buffers, if any, once each associated NOP-Out 1514 acknowledgement is received that the third-party initiator 1515 generated in response to each Async Message sent in Step a. 1517 If the issuing session is a FastAbort session, the iSCSI target 1518 implementation is FastAbort-capable, and the third-party affected 1519 session is an RFC 3720 session, the following behavior MUST be 1520 exhibited by the iSCSI target layer: Asynchronous Message PDUs MUST 1521 NOT be sent on the third-party session to prompt the FastAbort 1522 behavior. 1524 If the third-party affected session is a FastAbort session and the 1525 issuing session is a FastAbort session, the initiator in the third- 1526 party role MUST respond to each Async Message PDU with AsyncEvent=5 1527 as defined in Section 10.9. Note that an initiator MAY thus receive 1528 these Async Messages on a third-party affected session even if the 1529 session is a single-connection session. 1531 3.2.3.6. Rationale behind the FastAbort Semantics 1533 There are fundamentally three basic objectives behind the semantics 1534 specified in Sections 3.2.3.3 and 3.2.3.4. 1536 1. Maintaining an ordered command flow I_T nexus abstraction to 1537 the target SCSI layer even with multi-connection sessions. 1538 - Target iSCSI processing of a TMF request must maintain 1539 the single flow illusion. Target behavior in Step b of 1540 Section 3.2.3.3 and Step a of Section 3.2.3.4 correspond 1541 to this objective. 1542 2. Maintaining a single ordered response flow I_T nexus 1543 abstraction to the initiator SCSI layer even with multi- 1544 connection sessions when one response (i.e., TMF response) 1545 could imply the status of other unfinished tasks from the 1546 initiators perspective. 1547 - The target must ensure that the initiator does not see 1548 "old" task responses (that were placed on the wire 1549 chronologically earlier than the TMF Response) after 1550 seeing the TMF response. The target behavior in Step d 1551 of Section 3.2.3.3 and Step e of Section 3.2.3.4 1552 correspond to this objective. 1553 - Whenever the result of a TMF action is visible across 1554 multiple I_T_L nexuses, [SAM2] requires the SCSI device 1555 server to trigger a UA on each of the other I_T_L 1556 nexuses. Once an initiator is notified of such an UA, 1557 the application client on the receiving initiator is 1558 required to clear its task state (clause 5.5 in [SAM2]) 1559 for the affected tasks. It would thus be inappropriate 1560 to deliver a SCSI Response for a task after the task 1561 state is cleared on the initiator, i.e., after the UA is 1562 notified. The UA notification contained in the first 1563 SCSI Response PDU on each affected Third-party I_T_L 1564 nexus after the TMF action thus MUST NOT pass the 1565 affected task responses on any of the iSCSI sessions 1566 accessing the LU. The target behavior in Step e of 1567 Section 3.2.3.3 and Step f of Section 3.2.3.4 correspond 1568 to this objective. 1569 3. Draining all active TTTs corresponding to affected tasks in 1570 a deterministic fashion. 1571 - Data-Out PDUs with stale TTTs arriving after the tasks 1572 are terminated can create a buffer management problem 1573 even for traditional iSCSI implementations, and is fatal 1574 for the connection for iSCSI/iSER implementations. 1575 Either the termination of affected tasks should be 1576 postponed until the TTTs are retired (as in Step a of 1577 Section 3.2.3.3), or the TTTs and the buffers should 1578 stay allocated beyond task termination to be 1579 deterministically freed up later (as in Steps c and g of 1580 Section 3.2.3.4). 1582 The only other notable optimization is the plugging. If all tasks 1583 on an I_T nexus will be aborted anyway (as with a target reset), 1584 there is no need to wait to receive all commands to plug the CmdSN 1585 holes. The target iSCSI layer can simply plug all missing CmdSN 1586 slots and move on with TMF processing. The first objective 1587 (maintaining a single ordered command flow) is still met with this 1588 optimization because the target SCSI layer only sees ordered 1589 commands. 1591 3.2.4. iSCSI Login 1593 The purpose of the iSCSI login is to enable a TCP connection for 1594 iSCSI use, authentication of the parties, negotiation of the 1595 session's parameters and marking of the connection as belonging to 1596 an iSCSI session. 1598 A session is used to identify to a target all the connections with 1599 a given initiator that belong to the same I_T nexus. (For more 1600 details on how a session relates to an I_T nexus, see section 1601 3.4.2). 1603 The targets listen on a well-known TCP port or other TCP port for 1604 incoming connections. The initiator begins the login process by 1605 connecting to one of these TCP ports. 1607 As part of the login process, the initiator and target SHOULD 1608 authenticate each other and MAY set a security association protocol 1609 for the session. This can occur in many different ways and is 1610 subject to negotiation. 1612 To protect the TCP connection, an IPsec security association MAY be 1613 established before the Login request. For information on using 1614 IPsec security for iSCSI see Chapter 8 and [RFC3723]. 1616 The iSCSI Login Phase is carried through Login requests and 1617 responses. Once suitable authentication has occurred and 1618 operational parameters have been set, the session transitions to 1619 Full Feature Phase and the initiator may start to send SCSI 1620 commands. The security policy for whether, and by what means, a 1621 target chooses to authorize an initiator is beyond the scope of 1622 this document. For a more detailed description of the Login Phase, 1623 see Chapter 5. 1625 The login PDU includes the ISID part of the session ID (SSID). The 1626 target portal group that services the login is implied by the 1627 selection of the connection endpoint. For a new session, the TSIH 1628 is zero. As part of the response, the target generates a TSIH. 1630 During session establishment, the target identifies the SCSI 1631 initiator port (the "I" in the "I_T nexus") through the value pair 1632 (InitiatorName, ISID). We describe InitiatorName later in this 1633 section. Any persistent state (e.g., persistent reservations) on 1634 the target that is associated with a SCSI initiator port is 1635 identified based on this value pair. Any state associated with the 1636 SCSI target port (the "T" in the "I_T nexus") is identified 1637 externally by the TargetName and portal group tag (see Section 1638 3.4.1). ISID is subject to reuse restrictions because it is used to 1639 identify a persistent state (see Section 3.4.3). 1641 Before the Full Feature Phase is established, only Login Request 1642 and Login Response PDUs are allowed. Login requests and responses 1643 MUST be used exclusively during Login. On any connection, the login 1644 phase MUST immediately follow TCP connection establishment and a 1645 subsequent Login Phase MUST NOT occur before tearing down a 1646 connection. 1648 A target receiving any PDU except a Login request before the Login 1649 phase is started MUST immediately terminate the connection on which 1650 the PDU was received. Once the Login phase has started, if the 1651 target receives any PDU except a Login request, it MUST send a 1652 Login reject (with Status "invalid during login") and then 1653 disconnect. If the initiator receives any PDU except a Login 1654 response, it MUST immediately terminate the connection. 1656 3.2.5. iSCSI Full Feature Phase 1658 Once the initiator is authorized to do so, the iSCSI session is in 1659 the iSCSI Full Feature Phase. A session is in Full Feature Phase 1660 after successfully finishing the Login Phase on the first (leading) 1661 connection of a session. A connection is in Full Feature Phase if 1662 the session is in Full Feature Phase and the connection login has 1663 completed successfully. An iSCSI connection is not in Full Feature 1664 Phase 1666 when it does not have an established transport connection, 1667 OR 1668 when it has a valid transport connection, but a successful 1669 login was not performed or the connection is currently 1670 logged out. 1672 In a normal Full Feature Phase, the initiator may send SCSI 1673 commands and data to the various LUs on the target by encapsulating 1674 them in iSCSI PDUs that go over the established iSCSI session. 1676 3.2.5.1. Command Connection Allegiance 1678 For any iSCSI request issued over a TCP connection, the 1679 corresponding response and/or other related PDU(s) MUST be sent 1680 over the same connection. We call this "connection allegiance". If 1681 the original connection fails before the command is completed, the 1682 connection allegiance of the command may be explicitly reassigned 1683 to a different transport connection as described in detail in 1684 Section 6.2 "Retry and Reassign in Recovery". 1686 Thus, if an initiator issues a READ command, the target MUST send 1687 the requested data, if any, followed by the status to the initiator 1688 over the same TCP connection that was used to deliver the SCSI 1689 command. If an initiator issues a WRITE command, the initiator MUST 1690 send the data, if any, for that command over the same TCP 1691 connection that was used to deliver the SCSI command. The target 1692 MUST return Ready To Transfer (R2T), if any, and the status over 1693 the same TCP connection that was used to deliver the SCSI command. 1694 Retransmission requests (SNACK PDUs) and the data and status that 1695 they generate MUST also use the same connection. 1697 However, consecutive commands that are part of a SCSI linked 1698 command-chain task (see [SAM2]) MAY use different connections. 1699 Connection allegiance is strictly per-command and not per-task. 1700 During the iSCSI Full Feature Phase, the initiator and target MAY 1701 interleave unrelated SCSI commands, their SCSI Data, and responses 1702 over the session. 1704 3.2.5.2. Data Transfer Overview 1706 Outgoing SCSI data (initiator to target user data or command 1707 parameters) is sent as either solicited data or unsolicited data. 1708 Solicited data are sent in response to R2T PDUs. Unsolicited data 1709 can be sent as part of an iSCSI command PDU ("immediate data") or 1710 in separate iSCSI data PDUs. 1712 Immediate data are assumed to originate at offset 0 in the 1713 initiator SCSI write-buffer (outgoing data buffer). All other Data 1714 PDUs have the buffer offset set explicitly in the PDU header. 1716 An initiator may send unsolicited data up to FirstBurstLength as 1717 immediate (up to the negotiated maximum PDU length), in a separate 1718 PDU sequence or both. All subsequent data MUST be solicited. The 1719 maximum length of an individual data PDU or the immediate-part of 1720 the first unsolicited burst MAY be negotiated at login. 1722 The maximum amount of unsolicited data that can be sent with a 1723 command is negotiated at login through the FirstBurstLength key. A 1724 target MAY separately enable immediate data (through the 1725 ImmediateData key) without enabling the more general (separate data 1726 PDUs) form of unsolicited data (through the InitialR2T key). 1728 Unsolicited data on write are meant to reduce the effect of latency 1729 on throughput (no R2T is needed to start sending data). In 1730 addition, immediate data is meant to reduce the protocol overhead 1731 (both bandwidth and execution time). 1733 An iSCSI initiator MAY choose not to send unsolicited data, only 1734 immediate data or FirstBurstLength bytes of unsolicited data with a 1735 command. If any non-immediate unsolicited data is sent, the total 1736 unsolicited data MUST be either FirstBurstLength, or all of the 1737 data if the total amount is less than the FirstBurstLength. 1739 It is considered an error for an initiator to send unsolicited data 1740 PDUs to a target that operates in R2T mode (only solicited data are 1741 allowed). It is also an error for an initiator to send more 1742 unsolicited data, whether immediate or as separate PDUs, than 1743 FirstBurstLength. 1745 An initiator MUST honor an R2T data request for a valid outstanding 1746 command (i.e., carrying a valid Initiator Task Tag) and deliver all 1747 the requested data provided the command is supposed to deliver 1748 outgoing data and the R2T specifies data within the command bounds. 1749 The initiator action is unspecified for receiving an R2T request 1750 that specifies data, all or part, outside of the bounds of the 1751 command. 1753 A target SHOULD NOT silently discard data and then request 1754 retransmission through R2T. Initiators SHOULD NOT keep track of the 1755 data transferred to or from the target (scoreboarding). SCSI 1756 targets perform residual count calculation to check how much data 1757 was actually transferred to or from the device by a command. This 1758 may differ from the amount the initiator sent and/or received for 1759 reasons such as retransmissions and errors. Read or bidirectional 1760 commands implicitly solicit the transmission of the entire amount 1761 of data covered by the command. SCSI data packets are matched to 1762 their corresponding SCSI commands by using tags specified in the 1763 protocol. 1765 In addition, iSCSI initiators and targets MUST enforce some 1766 ordering rules. When unsolicited data is used, the order of the 1767 unsolicited data on each connection MUST match the order in which 1768 the commands on that connection are sent. Command and unsolicited 1769 data PDUs may be interleaved on a single connection as long as the 1770 ordering requirements of each are maintained (e.g., command N+1 MAY 1771 be sent before the unsolicited Data-Out PDUs for command N, but the 1772 unsolicited Data-Out PDUs for command N MUST precede the 1773 unsolicited Data-Out PDUs of command N+1). A target that receives 1774 data out of order MAY terminate the session. 1776 3.2.5.3. Tags and Integrity Checks 1778 Initiator tags for pending commands are unique initiator-wide for a 1779 session. Target tags are not strictly specified by the protocol. It 1780 is assumed that target tags are used by the target to tag (alone or 1781 in combination with the LUN) the solicited data. Target tags are 1782 generated by the target and "echoed" by the initiator. These 1783 mechanisms are designed to accomplish efficient data delivery along 1784 with a large degree of control over the data flow. 1786 As the Initiator Task Tag is used to identify a task during its 1787 execution the iSCSI initiator and target MUST verify that all other 1788 fields used in task related PDUs have values that are consistent 1789 with the values used at the task instantiation based on Initiator 1790 Task Tag (e.g., the LUN used in an R2T PDU MUST be the same as the 1791 one used in the SCSI command PDU used to instantiate the task). 1792 Using inconsistent field values is considered a protocol error. 1794 3.2.5.4. Task Management 1796 SCSI task management assumes that individual tasks and task groups 1797 can be aborted solely based on the task tags (for individual tasks) 1798 or the timing of the task management command (for task groups) and 1799 that the task management action is executed synchronously - i.e, no 1800 message involving an aborted task will be seen by the SCSI 1801 initiator after receiving the task management response. In iSCSI 1802 initiators and targets interact asynchronously over several 1803 connections. iSCSI specifies the protocol mechanism and 1804 implementation requirements needed to present a synchronous view 1805 while using an asynchronous infrastructure. 1807 3.2.6. iSCSI Connection Termination 1809 An iSCSI connection may be terminated by use of a transport 1810 connection shutdown or a transport reset. Transport reset is 1811 assumed to be an exceptional event. 1813 Graceful TCP connection shutdowns are done by sending TCP FINs. A 1814 graceful transport connection shutdown SHOULD only be initiated by 1815 either party when the connection is not in iSCSI Full Feature 1816 Phase. A target MAY terminate a Full Feature Phase connection on 1817 internal exception events, but it SHOULD announce the fact through 1818 an Asynchronous Message PDU. Connection termination with 1819 outstanding commands may require recovery actions. 1821 If a connection is terminated while in Full Feature Phase, 1822 connection cleanup (see section 7) is required prior to recovery. 1824 By doing connection cleanup before starting recovery, the initiator 1825 and target will avoid receiving stale PDUs after recovery. 1827 3.2.7. iSCSI Names 1829 Both targets and initiators require names for the purpose of 1830 identification. In addition, names enable iSCSI storage resources 1831 to be managed regardless of location (address). An iSCSI node name 1832 is also the SCSI device name contained in the iSCSI Node. The iSCSI 1833 name of a SCSI device is the principal object used in 1834 authentication of targets to initiators and initiators to targets. 1835 This name is also used to identify and manage iSCSI storage 1836 resources. 1838 iSCSI names must be unique within the operation domain of the end 1839 user. However, because the operation domain of an IP network is 1840 potentially worldwide, the iSCSI name formats are architected to be 1841 worldwide unique. To assist naming authorities in the construction 1842 of worldwide unique names, iSCSI provides three name formats for 1843 different types of naming authorities. 1845 iSCSI names are associated with iSCSI nodes, and not iSCSI network 1846 adapter cards, to ensure that the replacement of network adapter 1847 cards does not require reconfiguration of all SCSI and iSCSI 1848 resource allocation information. 1850 Some SCSI commands require that protocol-specific identifiers be 1851 communicated within SCSI CDBs. See SCSI for the definition of the 1852 SCSI port name/identifier for iSCSI ports. 1854 An initiator may discover the iSCSI Target Names to which it has 1855 access, along with their addresses, using the SendTargets text 1856 request, or other techniques discussed in [RFC3721]. 1858 3.2.7.1. iSCSI Name Properties 1860 Each iSCSI node, whether it is an initiator or target, MUST have an 1861 iSCSI name. 1863 Initiators and targets MUST support the receipt of iSCSI names of 1864 up to the maximum length of 223 bytes. 1866 The initiator MUST present both its iSCSI Initiator Name and the 1867 iSCSI Target Name to which it wishes to connect in the first login 1868 request of a new session or connection. The only exception is if a 1869 discovery session (see Section 2.3 iSCSI Session Types) is to be 1870 established. In this case, the iSCSI Initiator Name is still 1871 required, but the iSCSI Target Name MAY be omitted. 1873 iSCSI names have the following properties: 1875 iSCSI names are globally unique. No two initiators or targets 1876 can have the same name. 1877 iSCSI names are permanent. An iSCSI initiator node or target 1878 node has the same name for its lifetime. 1879 iSCSI names do not imply a location or address. An iSCSI 1880 initiator or target can move, or have multiple addresses. A 1881 change of address does not imply a change of name. 1882 iSCSI names do not rely on a central name broker; the naming 1883 authority is distributed. 1884 iSCSI names support integration with existing unique naming 1885 schemes. 1886 iSCSI names rely on existing naming authorities. iSCSI does not 1887 create any new naming authority. 1889 The encoding of an iSCSI name has the following properties: 1891 iSCSI names have the same encoding method regardless of the 1892 underlying protocols. 1893 iSCSI names are relatively simple to compare. The algorithm for 1894 comparing two iSCSI names for equivalence does not rely on 1895 an external server. 1896 iSCSI names are composed only of displayable characters. iSCSI 1897 names allow the use of international character sets but are 1898 not case sensitive. No whitespace characters are used in 1899 iSCSI names. 1900 iSCSI names may be transported using both binary and ASCII- 1901 based protocols. 1903 An iSCSI name really names a logical software entity, and is not 1904 tied to a port or other hardware that can be changed. For instance, 1905 an initiator name should name the iSCSI initiator node, not a 1906 particular NIC or HBA. When multiple NICs are used, they should 1907 generally all present the same iSCSI initiator name to the targets, 1908 because they are simply paths to the same SCSI layer. In most 1909 operating systems, the named entity is the operating system image. 1911 Similarly, a target name should not be tied to hardware interfaces 1912 that can be changed. A target name should identify the logical 1913 target and must be the same for the target regardless of the 1914 physical portion being addressed. This assists iSCSI initiators in 1915 determining that the two targets it has discovered are really two 1916 paths to the same target. 1918 The iSCSI name is designed to fulfill the functional requirements 1919 for Uniform Resource Names (URN) [RFC1737]. For example, it is 1920 required that the name have a global scope, be independent of 1921 address or location, and be persistent and globally unique. Names 1922 must be extensible and scalable with the use of naming authorities. 1923 The name encoding should be both human and machine readable. See 1924 [RFC1737] for further requirements. 1926 3.2.7.2. iSCSI Name Encoding 1928 An iSCSI name MUST be a UTF-8 encoding of a string of Unicode 1929 characters with the following properties: 1931 - It is in Normalization Form C (see "Unicode Normalization 1932 Forms" [UNICODE]). 1934 - It only contains characters allowed by the output of the 1935 iSCSI stringprep template (described in [RFC3722]). 1937 - The following characters are used for formatting iSCSI names: 1939 - dash ('-'=U+002d) 1940 - dot ('.'=U+002e) 1941 - colon (':'=U+003a) 1943 - The UTF-8 encoding of the name is not larger than 223 bytes. 1945 The stringprep process is described in [RFC3454]; iSCSI's use of 1946 the stringprep process is described in [RFC3722]. Stringprep is a 1947 method designed by the Internationalized Domain Name (IDN) working 1948 group to translate human-typed strings into a format that can be 1949 compared as opaque strings. Strings MUST NOT include punctuation, 1950 spacing, diacritical marks, or other characters that could get in 1951 the way of readability. The stringprep process also converts 1952 strings into equivalent strings of lower-case characters. 1954 The stringprep process does not need to be implemented if the names 1955 are only generated using numeric and lower-case (any character set) 1956 alphabetic characters. 1958 Once iSCSI names encoded in UTF-8 are "normalized" they may be 1959 safely compared byte-for-byte. 1961 3.2.7.3. iSCSI Name Structure 1963 An iSCSI name consists of two partsa type designator followed by a 1964 unique name string. 1966 iSCSI uses three existing naming authorities in constructing 1967 globally unique iSCSI names. Type designator in an iSCSI name 1968 indicates the naming authority on which the name is based. The 1969 three iSCSI name formats are the following: 1971 iSCSI-Qualified Name: it is based on domain names to identify a 1972 naming authority, 1973 NAA format Name: it is based on a naming format defined by [FC- 1974 FS] for constructing globally unique identifiers, referred 1975 to as the Network Address Authority (NAA), and, 1976 EUI format Name: it is based on EUI names where the IEEE 1977 Registration Authority assists in the formation of worldwide 1978 unique names (EUI-64 format). 1980 The corresponding type designator strings currently defined are: 1982 iqn. - iSCSI Qualified name 1983 naa. - Remainder of the string is an INCITS T11-defined 1984 Network Address Authority identifier, in ASCII-encoded 1985 hexadecimal. 1987 eui. - Remainder of the string is an IEEE EUI-64 identifier, in 1988 ASCII-encoded hexadecimal. 1990 These three naming authority designators were considered sufficient 1991 at the time of writing this document. The creation of additional 1992 naming type designators for iSCSI may be considered by the IETF and 1993 detailed in separate RFCs. 1995 The following table summarizes the current SCSI transport protocols 1996 and their naming formats. 1998 SCSI Transport Protocol Naming Format 1999 +----------------------------+-------+-----+----+ 2000 | | EUI-64| NAA |IQN | 2001 |----------------------------|-------|-----|----| 2002 | iSCSI (Internet SCSI) | X | X | X | 2003 |----------------------------|-------|-----|----| 2004 | FCP (Fibre Channel) | | X | | 2005 |----------------------------|-------|-----|----| 2006 | SAS (Serial Attached SCSI) | | X | | 2007 |----------------------------|-------|-----|----| 2008 | SRP (for InfiniBand) | X | | | 2009 +----------------------------+-------+-----+----+ 2011 3.2.7.4. Type "iqn." (iSCSI Qualified Name) 2013 This iSCSI name type can be used by any organization that owns a 2014 domain name. This naming format is useful when an end user or 2015 service provider wishes to assign iSCSI names for targets and/or 2016 initiators. 2018 To generate names of this type, the person or organization 2019 generating the name must own a registered domain name. This domain 2020 name does not have to be active, and does not have to resolve to an 2021 address; it just needs to be reserved to prevent others from 2022 generating iSCSI names using the same domain name. 2024 Since a domain name can expire, be acquired by another entity, or 2025 may be used to generate iSCSI names by both owners, the domain name 2026 must be additionally qualified by a date during which the naming 2027 authority owned the domain name. A date code is provided as part of 2028 the "iqn." format for this reason. 2030 The iSCSI qualified name string consists of: 2032 - The string "iqn.", used to distinguish these names from 2033 "eui." formatted names. 2035 - A date code, in yyyy-mm format. This date MUST be a date 2036 during which the naming authority owned the domain name used 2037 in this format, and SHOULD be the first month in which the 2038 domain name was owned by this naming authority at 00:01 GMT 2039 of the first day of the month. This date code uses the 2040 Gregorian calendar. All four digits in the year must be 2041 present. Both digits of the month must be present, with 2042 January == "01" and December == "12". The dash must be 2043 included. 2045 - A dot "." 2047 - The reversed domain name of the naming authority (person or 2048 organization) creating this iSCSI name. 2050 - An optional, colon (:) prefixed, string within the character 2051 set and length boundaries that the owner of the domain name 2052 deems appropriate. This may contain product types, serial 2053 numbers, host identifiers, or software keys (e.g, it may 2054 include colons to separate organization boundaries). With the 2055 exception of the colon prefix, the owner of the domain name 2056 can assign everything after the reversed domain name as 2057 desired. It is the responsibility of the entity that is the 2058 naming authority to ensure that the iSCSI names it assigns 2059 are worldwide unique. For example, "Example Storage Arrays, 2060 Inc.", might own the domain name "example.com". 2062 The following are examples of iSCSI qualified names that might be 2063 generated by "EXAMPLE Storage Arrays, Inc." 2065 Naming String defined by 2066 Type Date Auth "example.com" naming authority 2067 +--++-----+ +---------+ +--------------------------------+ 2068 | || | | | | | 2070 iqn.2001-04.com.example:storage:diskarrays-sn-a8675309 2071 iqn.2001-04.com.example 2072 iqn.2001-04.com.example:storage.tape1.sys1.xyz 2073 iqn.2001-04.com.example:storage.disk2.sys1.xyz 2075 3.2.7.5. Type "eui." (IEEE EUI-64 format) 2077 The IEEE Registration Authority provides a service for assigning 2078 globally unique identifiers [EUI]. The EUI-64 format is used to 2079 build a global identifier in other network protocols. For example, 2080 Fibre Channel defines a method of encoding it into a WorldWideName. 2081 For more information on registering for EUI identifiers, see [OUI]. 2083 The format is "eui." followed by an EUI-64 identifier (16 ASCII- 2084 encoded hexadecimal digits). 2086 Example iSCSI name: 2088 Type EUI-64 identifier (ASCII-encoded hexadecimal) 2090 +--++--------------+ 2092 | || | 2094 eui.02004567A425678D 2096 The IEEE EUI-64 iSCSI name format might be used when a manufacturer 2097 is already registered with the IEEE Registration Authority and uses 2098 EUI-64 formatted worldwide unique names for its products. 2100 More examples of name construction are discussed in [RFC3721]. 2102 3.2.7.6. Type "naa." - Network Address Authority 2104 The INCITS T11 Framing and Signaling Specification [FC-FS] defines 2105 a format called the Network Address Authority (NAA) format for 2106 constructing worldwide unique identifiers that use various 2107 identifier registration authorities. This identifier format is used 2108 by the Fibre Channel and SAS SCSI transport protocols. As FC and 2109 SAS constitute a large fraction of networked SCSI ports, the NAA 2110 format is a widely used format for SCSI transports. The objective 2111 behind iSCSI supporting a direct representation of an NAA-format 2112 name is to facilitate construction of a target device name that 2113 translates easily across multiple namespaces for a SCSI storage 2114 device containing ports served by different transports. More 2115 specifically, this format allows implementations wherein one NAA 2116 identifier can be assigned as the basis for the SCSI device name 2117 for a SCSI target with both SAS ports and iSCSI ports. 2119 The iSCSI NAA naming format is "naa.", followed by an NAA 2120 identifier represented in ASCII-encoded hexadecimal digits. 2122 An example of an iSCSI name with a 64-bit NAA value follows: 2124 Type NAA identifier (ASCII-encoded hexadecimal) 2125 +--++--------------+ 2126 | || | 2127 naa.52004567BA64678D 2129 An example of an iSCSI name with a 128-bit NAA value follows: 2131 Type NAA identifier (ASCII-encoded hexadecimal) 2132 +--++------------------------------+ 2133 | || | 2134 naa.62004567BA64678D0123456789ABCDEF 2136 The iSCSI NAA naming format might be used in an implementation when 2137 the infrastructure for generating NAA worldwide unique names is 2138 already in place because the device contains both SAS and iSCSI 2139 SCSI ports. 2141 The NAA identifier formatted in an ASCII-hexadecimal representation 2142 has a maximum size of 32 characters (128 bit NAA format). As a 2143 result, there is no issue with this naming format exceeding the 2144 maximum size for iSCSI node names. 2146 3.2.8. Persistent State 2148 iSCSI does not require any persistent state maintenance across 2149 sessions. However, in some cases, SCSI requires persistent 2150 identification of the SCSI initiator port name (See Section 3.4.2 2151 and Section 3.4.3). 2153 iSCSI sessions do not persist through power cycles and boot 2154 operations. 2156 All iSCSI session and connection parameters are re-initialized on 2157 session and connection creation. 2159 Commands persist beyond connection termination if the session 2160 persists and command recovery within the session is supported. 2161 However, when a connection is dropped, command execution, as 2162 perceived by iSCSI (i.e., involving iSCSI protocol exchanges for 2163 the affected task), is suspended until a new allegiance is 2164 established by the 'task reassign' task management function. (See 2165 Section 10.5 "Task Management Function Request".) 2167 3.2.9. Message Synchronization and Steering 2169 iSCSI presents a mapping of the SCSI protocol onto TCP. This 2170 encapsulation is accomplished by sending iSCSI PDUs of varying 2171 lengths. Unfortunately, TCP does not have a built-in mechanism for 2172 signaling message boundaries at the TCP layer. iSCSI overcomes this 2173 obstacle by placing the message length in the iSCSI message header. 2174 This serves to delineate the end of the current message as well as 2175 the beginning of the next message. 2177 In situations where IP packets are delivered in order from the 2178 network, iSCSI message framing is not an issue and messages are 2179 processed one after the other. In the presence of IP packet 2180 reordering (i.e., frames being dropped), legacy TCP implementations 2181 store the "out of order" TCP segments in temporary buffers until 2182 the missing TCP segments arrive, upon which the data must be copied 2183 to the application buffers. In iSCSI, it is desirable to steer the 2184 SCSI data within these out of order TCP segments into the pre- 2185 allocated SCSI buffers rather than store them in temporary buffers. 2186 This decreases the need for dedicated reassembly buffers as well as 2187 the latency and bandwidth related to extra copies. 2189 Relying solely on the "message length" information from the iSCSI 2190 message header may make it impossible to find iSCSI message 2191 boundaries in subsequent TCP segments due to the loss of a TCP 2192 segment that contains the iSCSI message length. The missing TCP 2193 segment(s) must be received before any of the following segments 2194 can be steered to the correct SCSI buffers (due to the inability to 2195 determine the iSCSI message boundaries). Since these segments 2196 cannot be steered to the correct location, they must be saved in 2197 temporary buffers that must then be copied to the SCSI buffers. 2199 Different schemes can be used to recover synchronization. To make 2200 these schemes work, iSCSI implementations have to make sure that 2201 the appropriate protocol layers are provided with enough 2202 information to implement a synchronization and/or data steering 2203 mechanism. One of these schemes is detailed in Appendix A. - Sync 2204 and Steering with Fixed Interval Markers. 2206 The Fixed Interval Markers (FIM) scheme works by inserting markers 2207 in the payload stream at fixed intervals that contain the offset to 2208 the start of the next iSCSI PDU. 2210 Under normal circumstances (no PDU loss or data reception out of 2211 order), iSCSI data steering can be accomplished by using the 2212 identifying tag and the data offset fields in the iSCSI header in 2213 addition to the TCP sequence number from the TCP header. The 2214 identifying tag helps associate the PDU with a SCSI buffer address 2215 while the data offset and TCP sequence number are used to determine 2216 the offset within the buffer. 2218 When the part of the TCP data stream containing an iSCSI PDU header 2219 is delayed or lost, markers may be used to minimize the damage as 2220 follows: 2222 - Markers indicate where the next iSCSI PDU starts and enable 2223 continued processing when iSCSI headers have to be dropped 2224 due to data errors discovered at iSCSI level (e.g., iSCSI 2225 header CRC errors). 2227 - Markers help minimize the amount of data that has to be kept 2228 by the TCP/iSCSI layer while waiting for a late TCP packet 2229 arrival or recovery, because later they might help find iSCSI 2230 PDU headers and use the information contained in those to 2231 steer data to SCSI buffers. 2233 3.2.9.1. Sync/Steering and iSCSI PDU Length 2235 When a large iSCSI message is sent, the TCP segment(s) that contain 2236 the iSCSI header may be lost. The remaining TCP segment(s) up to 2237 the next iSCSI message must be buffered (in temporary buffers) 2238 because the iSCSI header that indicates to which SCSI buffers the 2239 data are to be steered was lost. To minimize the amount of 2240 buffering, it is recommended that the iSCSI PDU length be 2241 restricted to a small value (perhaps a few TCP segments in length). 2242 During login, each end of the iSCSI session specifies the maximum 2243 iSCSI PDU length it will accept. 2245 3.3. iSCSI Session Types 2247 iSCSI defines two types of sessions: 2249 Normal operational session - an unrestricted session. 2251 Discovery-session - a session only opened for target discovery. 2252 The target MUST ONLY accept text requests with the 2253 SendTargets key and a logout request with reason "close the 2254 session". All other requests MUST be rejected. 2256 The session type is defined during login with key=value parameter 2257 in the login command. 2259 3.4. SCSI to iSCSI Concepts Mapping Model 2261 The following diagram shows an example of how multiple iSCSI Nodes 2262 (targets in this case) can coexist within the same Network Entity 2263 and can share Network Portals (IP addresses and TCP ports). Other 2264 more complex configurations are also possible. For detailed 2265 descriptions of the components of these diagrams, see iSCS . 2267 +-----------------------------------+ 2268 | Network Entity (iSCSI Client) | 2269 | | 2270 | +-------------+ | 2271 | | iSCSI Node | | 2272 | | (Initiator) | | 2273 | +-------------+ | 2274 | | | | 2275 | +--------------+ +--------------+ | 2276 | |Network Portal| |Network Portal| | 2277 | | 10.1.30.4 | | 10.1.40.6 | | 2278 +-+--------------+-+--------------+-+ 2279 | | 2280 | IP Networks | 2281 | | 2282 +-+--------------+-+--------------+-+ 2283 | |Network Portal| |Network Portal| | 2284 | | 10.1.30.21 | | 10.1.40.3 | | 2285 | | TCP Port 3260| | TCP Port 3260| | 2286 | +--------------+ +--------------+ | 2287 | | | | 2288 | ----------------- | 2289 | | | | 2290 | +-------------+ +--------------+ | 2291 | | iSCSI Node | | iSCSI Node | | 2292 | | (Target) | | (Target) | | 2293 | +-------------+ +--------------+ | 2294 | | 2295 | Network Entity (iSCSI Server) | 2296 +-----------------------------------+ 2298 3.4.1. iSCSI Architecture Model 2300 This section describes the part of the iSCSI architecture model 2301 that has the most bearing on the relationship between iSCSI and the 2302 SCSI Architecture Model. 2304 Network Entity - represents a device or gateway that is 2305 accessible from the IP network. A Network Entity must have 2306 one or more Network Portals (see item (d)), each of which 2307 can be used by some iSCSI Nodes (see item (b)) contained in 2309 that Network Entity to gain access to the IP network. 2311 iSCSI Node - represents a single iSCSI initiator or iSCSI 2312 target. There are one or more iSCSI Nodes within a Network 2313 Entity. The iSCSI Node is accessible via one or more Network 2314 Portals (see item d). An iSCSI Node is identified by its 2315 iSCSI Name (see Section 3.2.7 and Section 12). The 2316 separation of the iSCSI Name from the addresses used by and 2317 for the iSCSI node allows multiple iSCSI nodes to use the 2318 same addresses, and the same iSCSI node to use multiple 2319 addresses. 2321 An alias string may also be associated with an iSCSI Node. The 2322 alias allows an organization to associate a user friendly 2323 string with the iSCSI Name. However, the alias string is not 2324 a substitute for the iSCSI Name. 2326 Network Portal - a component of a Network Entity that has a 2327 TCP/IP network address and that may be used by an iSCSI Node 2328 within that Network Entity for the connection(s) within one 2329 of its iSCSI sessions. In an initiator, it is identified by 2330 its IP address. In a target, it is identified by its IP 2331 address and its listening TCP port. 2333 Portal Groups - iSCSI supports multiple connections within the 2334 same session; some implementations will have the ability to 2335 combine connections in a session across multiple Network 2336 Portals. A Portal Group defines a set of Network Portals 2337 within an iSCSI Node that collectively supports the 2338 capability of coordinating a session with connections that 2339 span these portals. Not all Network Portals within a Portal 2340 Group need to participate in every session connected through 2341 that Portal Group. One or more Portal Groups may provide 2342 access to an iSCSI Node. Each Network Portal, as utilized by 2343 a given iSCSI Node, belongs to exactly one portal group 2344 within that node. Portal Groups are identified within an 2345 iSCSI Node by a portal group tag, a simple unsigned-integer 2346 between 0 and 65535 (see Section 12.3 "SendTargets"). All 2347 Network Portals with the same portal group tag in the 2348 context of a given iSCSI Node are in the same Portal Group. 2350 Both iSCSI Initiators and iSCSI Targets have portal groups, 2351 though only the iSCSI Target Portal Groups are used directly 2352 in the iSCSI protocol (e.g., in SendTargets). For references 2353 to the Initiator Portal Groups, see Section 9.1.1 2354 "Conservative Reuse of ISIDs". 2356 Portals within a Portal Group should support similar session 2357 parameters, because they may participate in a common 2358 session. 2360 The following diagram shows an example of one such configuration on 2361 a target and how a session that shares Network Portals within a 2362 Portal Group may be established. 2364 ----------------------------IP Network--------------------- 2365 | | | 2366 +----|---------------|-----+ +----|---------+ 2367 | +---------+ +---------+ | | +---------+ | 2368 | | Network | | Network | | | | Network | | 2369 | | Portal | | Portal | | | | Portal | | 2370 | +--|------+ +---------+ | | +---------+ | 2371 | | | | | | | 2372 | | Portal | | | | Portal | 2373 | | Group 1 | | | | Group 2 | 2374 +--------------------------+ +--------------+ 2375 | | | 2376 +--------|---------------|--------------------|-------------------- 2377 + 2378 | | | | 2379 | 2380 | +----------------------------+ +-----------------------------+ 2381 | 2382 | | iSCSI Session (Target side)| | iSCSI Session (Target side) | 2383 | 2384 | | | | | 2385 | 2386 | | (TSIH = 56) | | (TSIH = 48) | 2387 | 2388 | +----------------------------+ +-----------------------------+ 2389 | 2390 | 2391 | 2392 | iSCSI Target Node 2393 | 2394 | (within Network Entity, not shown) 2395 | 2396 +------------------------------------------------------------------ 2397 + 2399 3.4.2. SCSI Architecture Model 2401 This section describes the relationship between the SCSI 2402 Architecture Model [SAM2] and constructs of the SCSI device, SCSI 2403 port and I_T nexus, and the iSCSI constructs described in Section 2404 3.4.1. 2406 This relationship implies implementation requirements in order to 2407 conform to the SAM2 model and other SCSI operational functions. 2408 These requirements are detailed in Section 3.4.3. 2410 The following list outlines mappings of SCSI architectural elements 2411 to iSCSI. 2413 SCSI Device - the SAM2 term for an entity that contains one or 2414 more SCSI ports that are connected to a service delivery 2415 subsystem and supports a SCSI application protocol. For 2416 example, a SCSI Initiator Device contains one or more SCSI 2417 Initiator Ports and zero or more application clients. A SCSI 2418 Target Device contains one or more SCSI Target Ports and one 2419 or more logical units. For iSCSI, the SCSI Device is the 2420 component within an iSCSI Node that provides the SCSI 2421 functionality. As such, there can be one SCSI Device, at 2422 most, within an iSCSI Node. Access to the SCSI Device can 2423 only be achieved in an iSCSI normal operational session (see 2424 Section 3.3). The SCSI Device Name is defined to be the 2425 iSCSI Name of the node and MUST be used in the iSCSI 2426 protocol. 2428 SCSI Port - the SAM2 term for an entity in a SCSI Device that 2429 provides the SCSI functionality to interface with a service 2430 delivery subsystem or transport. For iSCSI, the definition 2431 of SCSI Initiator Port and SCSI Target Port are different. 2433 SCSI Initiator Port: This maps to one endpoint of an iSCSI 2434 normal operational session (see Section 3.3). An iSCSI 2435 normal operational session is negotiated through the login 2436 process between an iSCSI initiator node and an iSCSI target 2437 node. At successful completion of this process, a SCSI 2438 Initiator Port is created within the SCSI Initiator Device. 2439 The SCSI Initiator Port Name and SCSI Initiator Port 2440 Identifier are both defined to be the iSCSI Initiator Name 2441 together with (a) a label that identifies it as an initiator 2442 port name/identifier and (b) the ISID portion of the session 2443 identifier. 2445 SCSI Target Port: This maps to an iSCSI Target Portal Group. 2447 The SCSI Target Port Name and the SCSI Target Port 2448 Identifier are both defined to be the iSCSI Target Name 2449 together with (a) a label that identifies it as a target 2450 port name/identifier and (b) the portal group tag. 2452 The SCSI Port Name MUST be used in iSCSI. When used in SCSI 2453 parameter data, the SCSI port name MUST be encoded as: 2454 - The iSCSI Name in UTF-8 format, followed by 2455 - a comma separator (1 byte), followed by 2456 - the ASCII character 'i' (for SCSI Initiator Port) or the 2457 ASCII character 't' (for SCSI Target Port) (1 byte), 2458 followed by 2459 - a comma separator (1 byte), followed by 2460 - a text encoding as a hex-constant (see Section 5.1 "Text 2461 Format") of the ISID (for SCSI initiator port) or the 2462 portal group tag (for SCSI target port) including the 2463 initial 0X or 0x and the terminating null (14 bytes). 2465 The ASCII character 'i' or 't' is the label that 2466 identifies this port as either a SCSI Initiator Port or a 2467 SCSI Target Port. 2469 I_T nexus - a relationship between a SCSI Initiator Port and a 2470 SCSI Target Port, according to [SAM2]. For iSCSI, this 2471 relationship is a session, defined as a relationship between 2472 an iSCSI Initiator's end of the session (SCSI Initiator 2473 Port) and the iSCSI Target's Portal Group. The I_T nexus can 2474 be identified by the conjunction of the SCSI port names or 2475 by the iSCSI session identifier SSID. iSCSI defines the I_T 2476 nexus identifier to be the tuple (iSCSI Initiator Name + 'i' 2477 + ISID, iSCSI Target Name + 't' + Portal Group Tag). 2479 NOTE: The I_T nexus identifier is not equal to the session 2480 identifier (SSID). 2482 3.4.3. Consequences of the Model 2484 This section describes implementation and behavioral requirements 2485 that result from the mapping of SCSI constructs to the iSCSI 2486 constructs defined above. Between a given SCSI initiator port and a 2487 given SCSI target port, only one I_T nexus (session) can exist. No 2488 more than one nexus relationship (parallel nexus) is allowed by 2489 [SAM2]. Therefore, at any given time, only one session with the 2490 same session identifier (SSID) can exist between a given iSCSI 2491 initiator node and an iSCSI target node. 2493 These assumptions lead to the following conclusions and 2494 requirements: 2496 ISID RULE: Between a given iSCSI Initiator and iSCSI Target Portal 2497 Group (SCSI target port), there can only be one session with a 2498 given value for ISID that identifies the SCSI initiator port. See 2499 Section 10.12.5 "ISID". 2501 The structure of the ISID that contains a naming authority 2502 component (see Section 10.12.5 "ISID" and [RFC3721]) provides a 2503 mechanism to facilitate compliance with the ISID rule. (See Section 2504 9.1.1 "Conservative Reuse of ISIDs".) 2506 The iSCSI Initiator Node should manage the assignment of ISIDs 2507 prior to session initiation. The "ISID RULE" does not preclude the 2508 use of the same ISID from the same iSCSI Initiator with different 2509 Target Portal Groups on the same iSCSI target or on other iSCSI 2510 targets (see Section 9.1.1 "Conservative Reuse of ISIDs"). Allowing 2511 this would be analogous to a single SCSI Initiator Port having 2512 relationships (nexus) with multiple SCSI target ports on the same 2513 SCSI target device or SCSI target ports on other SCSI target 2514 devices. It is also possible to have multiple sessions with 2515 different ISIDs to the same Target Portal Group. Each such session 2516 would be considered to be with a different initiator even when the 2517 sessions originate from the same initiator device. The same ISID 2518 may be used by a different iSCSI initiator because it is the iSCSI 2519 Name together with the ISID that identifies the SCSI Initiator 2520 Port. 2522 NOTE: A consequence of the ISID RULE and the specification for the 2523 I_T nexus identifier is that two nexus with the same identifier 2524 should never exist at the same time. 2526 TSIH RULE: The iSCSI Target selects a non-zero value for the TSIH 2527 at session creation (when an initiator presents a 0 value at 2528 Login). After being selected, the same TSIH value MUST be used 2529 whenever initiator or target refers to the session and a TSIH is 2530 required. 2532 3.4.3.1. I_T Nexus State 2534 Certain nexus relationships contain an explicit state (e.g., 2535 initiator-specific mode pages) that may need to be preserved by the 2536 device server [SAM2] in a logical unit through changes or failures 2537 in the iSCSI layer (e.g., session failures). In order for that 2538 state to be restored, the iSCSI initiator should reestablish its 2539 session (re-login) to the same Target Portal Group using the 2540 previous ISID. That is, it should perform session recovery as 2541 described in Chapter 6. This is because the SCSI initiator port 2542 identifier and the SCSI target port identifier (or relative target 2543 port) form the datum that the SCSI logical unit device server uses 2544 to identify the I_T nexus. 2546 3.5. Request/Response Summary 2548 This section lists and briefly describes all the iSCSI PDU types 2549 (request and responses). 2551 All iSCSI PDUs are built as a set of one or more header segments 2552 (basic and auxiliary) and zero or one data segments. The header 2553 group and the data segment may each be followed by a CRC (digest). 2555 The basic header segment has a fixed length of 48 bytes. 2557 3.5.1. Request/Response Types Carrying SCSI Payload 2559 3.5.1.1. SCSI-Command 2561 This request carries the SCSI CDB and all the other SCSI execute 2562 command procedure call (see [SAM2]) IN arguments such as task 2563 attributes, Expected Data Transfer Length for one or both transfer 2564 directions (the latter for bidirectional commands), and Task Tag 2565 (as part of the I_T_L_x nexus). The I_T_L nexus is derived by the 2566 initiator and target from the LUN field in the request and the I_T 2567 nexus is implicit in the session identification. 2569 In addition, the SCSI-command PDU carries information required for 2570 the proper operation of the iSCSI protocol - the command sequence 2571 number (CmdSN) and the expected status number (ExpStatSN) on the 2572 connection it is issued. 2574 All or part of the SCSI output (write) data associated with the 2575 SCSI command may be sent as part of the SCSI-Command PDU as a data 2576 segment. 2578 3.5.1.2. SCSI-Response 2580 The SCSI-Response carries all the SCSI execute-command procedure 2581 call (see [SAM2]) OUT arguments and the SCSI execute-command 2582 procedure call return value. 2584 The SCSI-Response contains the residual counts from the operation, 2585 if any, an indication of whether the counts represent an overflow 2586 or an underflow, and the SCSI status if the status is valid or a 2587 response code (a non-zero return value for the execute-command 2588 procedure call) if the status is not valid. 2590 For a valid status that indicates that the command has been 2591 processed, but resulted in an exception (e.g., a SCSI CHECK 2592 CONDITION), the PDU data segment contains the associated sense 2593 data. The use of Autosense ([SAM2]) is REQUIRED by iSCSI. 2595 Some data segment content may also be associated (in the data 2596 segment) with a non-zero response code. 2598 In addition, the SCSI-Response PDU carries information required for 2599 the proper operation of the iSCSI protocol: 2601 - The number of Data-In PDUs that a target has sent (to enable 2602 the initiator to check that all have arrived). 2604 - StatSN - the Status Sequence Number on this connection. 2606 - ExpCmdSN - the next Expected Command Sequence Number at the 2607 target. 2609 - MaxCmdSN - the maximum CmdSN acceptable at the target from 2610 this initiator. 2612 3.5.1.3. Task Management Function Request 2614 The Task Management function request provides an initiator with a 2615 way to explicitly control the execution of one or more SCSI Tasks 2616 or iSCSI functions. The PDU carries a function identifier (which 2617 task management function to perform) and enough information to 2618 unequivocally identify the task or task-set on which to perform the 2619 action, even if the task(s) to act upon has not yet arrived or has 2620 been discarded due to an error. 2622 The referenced tag identifies an individual task if the function 2623 refers to an individual task. 2625 The I_T_L nexus identifies task sets. In iSCSI the I_T_L nexus is 2626 identified by the LUN and the session identification (the session 2627 identifies an I_T nexus). 2629 For task sets, the CmdSN of the Task Management function request 2630 helps identify the tasks upon which to act, namely all tasks 2631 associated with a LUN and having a CmdSN preceding the Task 2632 Management function request CmdSN. 2634 For a Task Management function, the coordination between responses 2635 to the tasks affected and the Task Management function response is 2636 done by the target. 2638 3.5.1.4. Task Management Function Response 2640 The Task Management function response carries an indication of 2641 function completion for a Task Management function request 2642 including how it completed (response and qualifier) and additional 2643 information for failure responses. 2645 After the Task Management response indicates Task Management 2646 function completion, the initiator will not receive any additional 2647 responses from the affected tasks. 2649 3.5.1.5. SCSI Data-out and SCSI Data-in 2651 SCSI Data-out and SCSI Data-in are the main vehicles by which SCSI 2652 data payload is carried between initiator and target. Data payload 2653 is associated with a specific SCSI command through the Initiator 2654 Task Tag. For target convenience, outgoing solicited data also 2655 carries a Target Transfer Tag (copied from R2T) and the LUN. Each 2656 PDU contains the payload length and the data offset relative to the 2657 buffer address contained in the SCSI execute command procedure 2658 call. 2660 In each direction, the data transfer is split into "sequences". An 2661 end-of-sequence is indicated by the F bit. 2663 An outgoing sequence is either unsolicited (only the first sequence 2664 can be unsolicited) or consists of all the Data-Out PDUs sent in 2665 response to an R2T. 2667 Input sequences enable the switching of direction for bidirectional 2668 commands as required. 2670 For input, the target may request positive acknowledgement of input 2671 data. This is limited to sessions that support error recovery and 2672 is implemented through the A bit in the SCSI Data-in PDU header. 2674 Data-in and Data-out PDUs also carry the DataSN to enable the 2675 initiator and target to detect missing PDUs (discarded due to an 2676 error). 2678 In addition, StatSN is carried by the Data-In PDUs. 2680 To enable a SCSI command to be processed while involving a minimum 2681 number of messages, the last SCSI Data-in PDU passed for a command 2682 may also contain the status if the status indicates termination 2683 with no exceptions (no sense or response involved). 2685 3.5.1.6. Ready To Transfer (R2T) 2687 R2T is the mechanism by which the SCSI target "requests" the 2688 initiator for output data. R2T specifies to the initiator the 2689 offset of the requested data relative to the buffer address from 2690 the execute command procedure call and the length of the solicited 2691 data. 2693 To help the SCSI target associate the resulting Data-out with an 2694 R2T, the R2T carries a Target Transfer Tag that will be copied by 2695 the initiator in the solicited SCSI Data-out PDUs. There are no 2696 protocol specific requirements with regard to the value of these 2697 tags, but it is assumed that together with the LUN, they will 2698 enable the target to associate data with an R2T. 2700 R2T also carries information required for proper operation of the 2701 iSCSI protocol, such as: 2703 - R2TSN (to enable an initiator to detect a missing R2T) 2705 - StatSN 2707 - ExpCmdSN 2709 - MaxCmdSN 2711 3.5.2. Requests/Responses carrying SCSI and iSCSI Payload 2713 3.5.2.1. Asynchronous Message 2715 Asynchronous Messages are used to carry SCSI asynchronous events 2716 (AEN) and iSCSI asynchronous messages. 2718 When carrying an AEN, the event details are reported as sense data 2719 in the data segment. 2721 3.5.3. Requests/Responses Carrying iSCSI Only Payload 2723 3.5.3.1. Text Request and Text Response 2725 Text requests and responses are designed as a parameter negotiation 2726 vehicle and as a vehicle for future extension. 2728 In the data segment Text Requests/Responses carry text information 2729 using a simple "key=value" syntax. 2731 Text Request/Responses may form extended sequences using the same 2732 Initiator Task Tag. The initiator uses the F (Final) flag bit in 2733 the text request header to indicate its readiness to terminate a 2734 sequence. The target uses the F (Final) flag bit in the text 2735 response header to indicate its consent to sequence termination. 2737 Text Request and Responses also use the Target Transfer Tag to 2738 indicate continuation of an operation or a new beginning. A target 2739 that wishes to continue an operation will set the Target Transfer 2740 Tag in a Text Response to a value different from the default 2741 0xffffffff. An initiator willing to continue will copy this value 2742 into the Target Transfer Tag of the next Text Request. If the 2743 initiator wants to restart the current target negotiation (start 2744 fresh) will set the Target Transfer Tag to 0xffffffff. 2746 Although a complete exchange is always started by the initiator, 2747 specific parameter negotiations may be initiated by the initiator 2748 or target. 2750 3.5.3.2. Login Request and Login Response 2752 Login Requests and Responses are used exclusively during the Login 2753 Phase of each connection to set up the session and connection 2754 parameters. (The Login Phase consists of a sequence of login 2755 requests and responses carrying the same Initiator Task Tag.) 2757 A connection is identified by an arbitrarily selected connection-ID 2758 (CID) that is unique within a session. 2760 Similar to the Text Requests and Responses, Login 2761 Requests/Responses carry key=value text information with a simple 2762 syntax in the data segment. 2764 The Login Phase proceeds through several stages (security 2765 negotiation, operational parameter negotiation) that are selected 2766 with two binary coded fields in the header the "current stage" 2767 (CSG) and the "next stage" (NSG) with the appearance of the latter 2768 being signaled by the "transit" flag (T). 2770 The first Login Phase of a session plays a special role, called the 2771 leading login, which determines some header fields (e.g., the 2772 version number, the maximum number of connections, and the session 2773 identification). 2775 The CmdSN initial value is also set by the leading login. 2777 StatSN for each connection is initiated by the connection login. 2779 A login request may indicate an implied logout (cleanup) of the 2780 connection to be logged in (a connection restart) by using the same 2781 Connection ID (CID) as an existing connection as well as the same 2782 session identifying elements of the session to which the old 2783 connection was associated. 2785 3.5.3.3. Logout Request and Response 2787 Logout Requests and Responses are used for the orderly closing of 2788 connections for recovery or maintenance. The logout request may be 2789 issued following a target prompt (through an asynchronous message) 2790 or at an initiators initiative. When issued on the connection to be 2791 logged out no other request may follow it. 2793 The Logout response indicates that the connection or session 2794 cleanup is completed and no other responses will arrive on the 2795 connection (if received on the logging out connection). In 2796 addition, the Logout Response indicates how long the target will 2797 continue to hold resources for recovery (e.g., command execution 2798 that continues on a new connection) in the text key Time2Retain and 2799 how long the initiator must wait before proceeding with recovery in 2800 the text key Time2Wait. 2802 3.5.3.4. SNACK Request 2804 With the SNACK Request, the initiator requests retransmission of 2805 numbered-responses or data from the target. A single SNACK request 2806 covers a contiguous set of missing items, called a run, of a given 2807 type of items. The type is indicated in a type field in the PDU 2808 header. The run is composed of an initial item (StatSN, DataSN, 2809 R2TSN) and the number of missed Status, Data, or R2T PDUs. For long 2810 data-in sequences, the target may request (at predefined minimum 2811 intervals) a positive acknowledgement for the data sent. A SNACK 2812 request with a type field that indicates ACK and the number of 2813 Data-In PDUs acknowledged conveys this positive acknowledgement. 2815 3.5.3.5. Reject 2817 Reject enables the target to report an iSCSI error condition (e.g., 2818 protocol, unsupported option) that uses a Reason field in the PDU 2819 header and includes the complete header of the bad PDU in the 2820 Reject PDU data segment. 2822 3.5.3.6. NOP-Out Request and NOP-In Response 2824 This request/response pair may be used by an initiator and target 2825 as a "ping" mechanism to verify that a connection/session is still 2826 active and all of its components are operational. Such a ping may 2827 be triggered by the initiator or target. The triggering party 2828 indicates that it wants a reply by setting a value different from 2829 the default 0xffffffff in the corresponding Initiator/Target 2830 Transfer Tag. 2832 NOP-In/NOP-Out may also be used "unidirectional" to convey to the 2833 initiator/target command, status or data counter values when there 2834 is no other "carrier" and there is a need to update the 2835 initiator/target. 2837 4. SCSI Mode Parameters for iSCSI 2839 There are no iSCSI specific mode pages. 2841 5. Login and Full Feature Phase Negotiation 2843 iSCSI parameters are negotiated at session or connection 2844 establishment by using Login Requests and Responses (see Section 2845 3.2.3 - "iSCSI Login") and during Full Feature Phase (Section 3.2.4 2846 - "iSCSI Full Feature Phase") by using Text Requests and Responses. 2847 In both cases the mechanism used is an exchange of iSCSI-text- 2848 key=value pairs. For brevity, iSCSI-text-keys are called just keys 2849 in the rest of this document. 2851 Keys are either declarative or require negotiation and the key 2852 description indicates if the key is declarative or requires 2853 negotiation. 2855 For the declarative keys the declaring party sets a value for the 2856 key. The key specification indicates if the key can be declared by 2857 the initiator, target or both. 2859 For the keys that require negotiation, one of the parties (the 2860 proposing party) proposes a value or set of values by including the 2861 key=value in the data part of a Login or Text Request or Response. 2862 The other party (the accepting party) makes a selection based on 2863 the value or list of values proposed and includes the selected 2864 value in a key=value in the data part of the following Login or 2865 Text Response or Request. For most of the keys, both the initiator 2866 and target can be proposing parties. 2868 The login process proceeds in two stages - the security negotiation 2869 stage and the operational parameter negotiation stage. Both stages 2870 are optional but at least one of them has to be present to enable 2871 setting some mandatory parameters. 2873 If present, the security negotiation stage precedes the operational 2874 parameter negotiation stage. 2876 Progression from stage to stage is controlled by the T (Transition) 2877 bit in the Login Request/Response PDU header. Through the T bit set 2878 to 1, the initiator indicates that it would like to transition. The 2879 target agrees to the transition (and selects the next stage) when 2880 ready. A field in the Login PDU header indicates the current stage 2881 (CSG) and during transition, another field indicates the next stage 2882 (NSG) proposed (initiator) and selected (target). 2884 The Text negotiation process is used to negotiate or declare 2885 operational parameters. The negotiation process is controlled by 2886 the F (final) bit in the PDU header. During text negotiations, the 2887 F bit is used by the initiator to indicate that it is ready to 2888 finish the negotiation and by the Target to acquiesce the end of 2889 negotiation. 2891 Since some key=value pairs may not fit entirely in a single PDU, 2892 the C (continuation) bit is used (both in Login and Text) to 2893 indicate that "more follows". 2895 The text negotiation uses an additional mechanism by which a target 2896 may deliver larger amounts of data to an enquiring initiator. The 2897 target sets a Target Task Tag to be used as a bookmark which when 2898 returned by the initiator, means "go on". If reset to a "neutral 2899 value", it means "forget about the rest". 2901 This chapter details types of keys and values used, the syntax 2902 rules for parameter formation, and the negotiation schemes to be 2903 used with different types of parameters. 2905 5.1. Text Format 2907 The initiator and target send a set of key=value pairs encoded in 2908 UTF-8 Unicode. All the text keys and text values specified in this 2909 document are to be presented and interpreted in the case in which 2910 they appear in this document. They are case sensitive. 2912 The following character symbols are used in this document for text 2913 items (the hexadecimal values represent Unicode code points): 2915 (a-z, A-Z) - letters 2916 (0-9) - digits 2917 " " (0x20) - space 2918 "." (0x2e) - dot 2919 "-" (0x2d) - minus 2920 "+" (0x2b) - plus 2921 "@" (0x40) - commercial at 2922 "_" (0x5f) - underscore 2923 "=" (0x3d) - equal 2924 ":" (0x3a) - colon 2926 "/" (0x2f) - solidus or slash 2927 "[" (0x5b) - left bracket 2928 "]" (0x5d) - right bracket 2929 null (0x00) - null separator 2930 "," (0x2c) - comma 2931 "~" (0x7e) - tilde 2933 Key=value pairs may span PDU boundaries. An initiator or target 2934 that sends partial key=value text within a PDU indicates that more 2935 text follows by setting the C bit in the Text or Login Request or 2936 Text or Login Response to 1. Data segments in a series of PDUs that 2937 have the C bit set to 1 and end with a PDU that have the C bit set 2938 to 0, or include a single PDU that has the C bit set to 0 have to 2939 be considered as forming a single logical-text-data-segment (LTDS). 2941 Every key=value pair, including the last or only pair in a LTDS, 2942 MUST be followed by one null (0x00) delimiter. 2944 A key-name is whatever precedes the first = in the key=value pair. 2945 The term key is used frequently in this document in place of key- 2946 name. 2948 A value is whatever follows the first = in the key=value pair up to 2949 the end of the key=value pair, but not including the null 2950 delimiter. 2952 The following definitions will be used in the rest of this 2953 document: 2955 standard-label: A string of one or more characters that consist 2956 of letters, digits, dot, minus, plus, commercial at, or 2957 underscore. A standard-label MUST begin with a capital letter 2958 and must not exceed 63 characters. 2960 key-name: A standard-label. 2962 text-value: A string of zero or more characters that consist of 2963 letters, digits, dot, minus, plus, commercial at, underscore, 2964 slash, left bracket, right bracket, or colon. 2966 iSCSI-name-value: A string of one or more characters that 2967 consist of minus, dot, colon, or any character allowed by the 2968 output of the iSCSI string-prep template as specified in 2969 [RFC3722] (see also Section 3.2.6.2 - "iSCSI Name Encoding"). 2971 iSCSI-local-name-value: A UTF-8 string; no null characters are 2972 allowed in the string. This encoding is to be used for 2973 localized (internationalized) aliases. 2975 boolean-value: The string "Yes" or "No". 2977 hex-constant: A hexadecimal constant encoded as a string that 2978 starts with "0x" or "0X" followed by one or more digits or 2979 the letters a, b, c, d, e, f, A, B, C, D, E, or F. Hex- 2980 constants are used to encode numerical values or binary 2981 strings. When used to encode numerical values, the excessive 2982 use of leading 0 digits is discouraged. The string following 2983 0X (or 0x) represents a base16 number that starts with the 2984 most significant base16 digit, followed by all other digits 2985 in decreasing order of significance and ending with the 2986 least-significant base16 digit. When used to encode binary 2987 strings, hexadecimal constants have an implicit byte-length 2988 that includes four bits for every hexadecimal digit of the 2989 constant, including leading zeroes. For example, a hex- 2990 constant of n hexadecimal digits has a byte-length of (the 2991 integer part of) (n+1)/2. 2993 decimal-constant: An unsigned decimal number with the digit 0 2994 or a string of one or more digits that start with a non-zero 2996 digit. Decimal-constants are used to encode numerical values 2997 or binary strings. Decimal constants can only be used to 2998 encode binary strings if the string length is explicitly 2999 specified. There is no implicit length for decimal strings. 3000 Decimal-constant MUST NOT be used for parameter values if the 3001 values can be equal or greater than 2**64 (numerical) or for 3002 binary strings that can be longer than 64 bits. 3004 base64-constant: base64 constant encoded as a string that 3005 starts with "0b" or "0B" followed by 1 or more digits or 3006 letters or plus or slash or equal. The encoding is done 3007 according to [RFC2045] and each character, except equal, 3008 represents a base64 digit or a 6-bit binary string. Base64- 3009 constants are used to encode numerical-values or binary 3010 strings. When used to encode numerical values, the excessive 3011 use of leading 0 digits (encoded as A) is discouraged. The 3012 string following 0B (or 0b) represents a base64 number that 3013 starts with the most significant base64 digit, followed by 3014 all other digits in decreasing order of significance and 3015 ending with the least-significant base64 digit; the least 3016 significant base64 digit may be optionally followed by pad 3017 digits (encoded as equal) that are not considered as part of 3018 the number. When used to encode binary strings, base64- 3019 constants have an implicit byte-length that includes six bits 3020 for every character of the constant, excluding trailing 3021 equals (i.e., a base64-constant of n base64 characters 3022 excluding the trailing equals has a byte-length of ((the 3023 integer part of) (n*3/4)). Correctly encoded base64 strings 3024 cannot have n values of 1, 5 ... k*4+1. 3026 numerical-value: An unsigned integer always less than 2**64 3027 encoded as a decimal-constant or a hex-constant. Unsigned 3028 integer arithmetic applies to numerical-values. 3030 large-numerical-value: An unsigned integer that can be larger 3031 than or equal to 2**64 encoded as a hex constant, or base64- 3032 constant. Unsigned integer arithmetic applies to large- 3033 numeric-values. 3035 numeric-range: Two numerical-values separated by a tilde where 3036 the value to the right of tilde must not be lower than the 3037 value to the left. 3039 regular-binary-value: A binary string not longer than 64 bits 3040 encoded as a decimal constant, hex constant, or base64- 3041 constant. The length of the string is either specified by the 3042 key definition or is the implicit byte-length of the encoded 3043 string. 3045 large-binary-value: A binary string longer than 64 bits encoded 3046 as a hex-constant or base64-constant. The length of the 3047 string is either specified by the key definition or is the 3048 implicit byte-length of the encoded string. 3050 binary-value: A regular-binary-value or a large-binary-value. 3051 Operations on binary values are key specific. 3053 simple-value: Text-value, iSCSI-name-value, boolean-value, 3054 numeric-value, a numeric-range, or a binary-value. 3056 list-of-values: A sequence of text-values separated by a comma. 3058 If not otherwise specified, the maximum length of a simple-value 3059 (not its encoded representation) is 255 bytes not including the 3060 delimiter (comma or zero byte). 3062 Any iSCSI target or initiator MUST support receiving at least 8192 3063 bytes of key=value data in a negotiation sequence. When proposing 3064 or accepting authentication methods that explicitly require support 3065 for very long authentication items, the initiator and target MUST 3066 support receiving of at least 64 kilobytes of key=value data (see 3067 Appendix 11.1.2 - Simple Public-Key Mechanism (SPKM) that requires 3068 support for public key certificates). 3070 5.2. Text Mode Negotiation 3072 During login, and thereafter, some session or connection parameters 3073 are either declared or negotiated through an exchange of textual 3074 information. 3076 The initiator starts the negotiation and/or declaration through a 3077 Text or Login request and indicates when it is ready for completion 3078 (by setting the F bit to 1 and keeping it to 1 in a Text Request or 3079 the T bit in the Login Request). As negotiation text may span PDU 3080 boundaries, a Text or Login Request or Text or Login Response PDU 3081 that have the C bit set to 1 MUST NOT have the F/T bit set to 1. 3083 A target receiving a Text or Login Request with the C bit set to 1 3084 MUST answer with a Text or Login Response with no data segment 3085 (DataSegmentLength 0). An initiator receiving a Text or Login 3086 Response with the C bit set to 1 MUST answer with a Text or Login 3087 Request with no data segment (DataSegmentLength 0). 3089 A target or initiator SHOULD NOT use a Text or Login Response or 3090 Text or Login Request with no data segment (DataSegmentLength 0) 3091 unless explicitly required by a general or a key-specific 3092 negotiation rule. 3094 There MUST NOT be more than one outstanding Text Request, or Text 3095 Response PDU on an iSCSI connection. An outstanding PDU in this 3096 context is one that has not been acknowledged by the remote iSCSI 3097 side. 3099 The format of a declaration is: 3101 Declarer-> = 3103 The general format of text negotiation is: 3105 Proposer-> = 3107 Acceptor-> ={|NotUnderstood|Irrelevant|Reject} 3109 Thus a declaration is a one-way textual exchange (unless the key is 3110 not understood by the receiver) while a negotiation is a two-way 3111 exchange. 3113 The proposer or declarer can either be the initiator or the target, 3114 and the acceptor can either be the target or initiator, 3115 respectively. Targets are not limited to respond to key=value pairs 3116 as proposed by the initiator. The target may propose key=value 3117 pairs of its own. 3119 All negotiations are explicit (i.e., the result MUST only be based 3120 on newly exchanged or declared values). There are no implicit 3121 proposals. If a proposal is not made, then a reply cannot be 3122 expected. Conservative design also requires that default values 3123 should not be relied upon when use of some other value has serious 3124 consequences. 3126 The value proposed or declared can be a numerical-value, a 3127 numerical-range defined by lower and upper value with both integers 3128 separated by tilde, a binary value, a text-value, an iSCSI-name- 3129 value, an iSCSI-local-name-value, a boolean-value (Yes or No), or a 3130 list of comma separated text-values. A range, a large-numerical- 3131 value, an iSCSI-name-value and an iSCSI-local-name-value MAY ONLY 3132 be used if it is explicitly allowed. An accepted value can be a 3133 numerical-value, a large-numerical-value, a text-value, or a 3134 boolean-value. 3136 If a specific key is not relevant for the current negotiation, the 3137 acceptor may answer with the constant "Irrelevant" for all types of 3138 negotiation. However the negotiation is not considered as failed if 3139 the answer is "Irrelevant". The "Irrelevant" answer is meant for 3140 those cases in which several keys are presented by a proposing 3141 party but the selection made by the acceptor for one of the keys 3142 makes other keys irrelevant. The following example illustrates the 3143 use of "Irrelevant": 3145 I->T OFMarker=Yes,OFMarkInt=2048~8192 3146 T->I OFMarker=No,OFMarkInt=Irrelevant 3148 I->T X#vkey1=(bla,alb,None),X#vkey2=(bla,alb) 3149 T->I X#vkey1=None,X#vkey2=Irrelevant 3151 Any key not understood by the acceptor may be ignored by the 3152 acceptor without affecting the basic function. However, the answer 3153 for a key not understood MUST be key=NotUnderstood. Note that 3154 NotUnderstood is a valid answer for both declarative and negotiated 3155 keys. The general iSCSI philosophy is that comprehension precedes 3156 processing for any iSCSI key. A proposer of an iSCSI key, 3157 negotiated or declarative, in a text key exchange MUST thus be able 3158 to properly handle a NotUnderstood response. 3160 The proper way to handle a NotUnderstood response depends on where 3161 the key is specified and whether the key is declarative vs. 3162 negotiated. All keys defined in [RFC3720] MUST be supported by all 3163 compliant implementations; a NotUnderstood answer on any of the 3164 [RFC3720] keys therefore MUST be considered a protocol error and 3165 handled accordingly. For all other later keys, a NotUnderstood 3166 answer concludes the negotiation for a negotiated key whereas for a 3167 declarative key, a NotUnderstood answer simply informs the declarer 3168 of a lack of comprehension by the receiver. 3170 In either case, a NotUnderstood answer always requires that the 3171 protocol behavior associated with that key not be used within the 3172 scope of the key (connection/session) by either side. 3174 The constants "None", "Reject", "Irrelevant", and "NotUnderstood" 3175 are reserved and MUST ONLY be used as described here. Violation of 3176 this rule is a protocol error (in particular the use of "Reject", 3177 "Irrelevant", and "NotUnderstood" as proposed values). 3179 Reject or Irrelevant are legitimate negotiation options where 3180 allowed but their excessive use is discouraged. A negotiation is 3181 considered complete when the acceptor has sent the key value pair 3182 even if the value is "Reject", "Irrelevant", or "NotUnderstood. 3183 Sending the key again would be a re-negotiation and is forbidden 3184 for many keys. 3186 If the acceptor sends "Reject" as an answer the negotiated key is 3187 left at its current value (or default if no value was set). If the 3188 current value is not acceptable to the proposer on the connection 3189 or to the session it is sent, the proposer MAY choose to terminate 3190 the connection or session. 3192 All keys in this document, except for the X extension formats, MUST 3193 be supported by iSCSI initiators and targets when used as specified 3194 here. If used as specified, these keys MUST NOT be answered with 3195 NotUnderstood. 3197 Implementers may introduce new keys by prefixing them with X- 3198 followed by their (reversed) domain name, or with new keys 3199 registered with IANA prefixing them with X#. For example, the 3200 entity owning the domain example.com can issue: 3202 X-com.example.bar.foo.do_something=3 3204 or a new registered key may be used as in: 3206 X#SuperCalyPhraGilistic=Yes 3208 Implementers MAY also introduce new values, but ONLY for new keys 3209 or authentication methods (see Section 11 - "iSCSI Security Text 3210 Keys and Authentication Methods"), or digests (see Section 12.1 - 3211 "HeaderDigest and DataDigest"). 3213 Whenever parameter action or acceptance are dependent on other 3214 parameters, the dependency rules and parameter sequence must be 3215 specified with the parameters. 3217 In the Login Phase (see Login Phase), every stage is a separate 3218 negotiation. In the FullFeaturePhase, a Text Request Response 3219 sequence is a negotiation. Negotiations MUST be handled as atomic 3220 operations. For example, all negotiated values go into effect after 3221 the negotiation concludes in agreement or are ignored if the 3222 negotiation fails. 3224 Some parameters may be subject to integrity rules (e.g., parameter- 3225 x must not exceed parameter-y or parameter-u not 1 implies 3226 parameter-v be Yes). Whenever required, integrity rules are 3227 specified with the keys. Checking for compliance with the integrity 3228 rule must only be performed after all the parameters are available 3229 (the existent and the newly negotiated). An iSCSI target MUST 3230 perform integrity checking before the new parameters take effect. 3231 An initiator MAY perform integrity checking. 3233 An iSCSI initiator or target MAY terminate a negotiation that does 3234 not end within a reasonable time or number of exchanges. 3236 5.2.1. List negotiations 3238 In list negotiation, the originator sends a list of values (which 3239 may include "None") in its order of preference. 3241 The responding party MUST respond with the same key and the first 3242 value that it supports (and is allowed to use for the specific 3243 originator) selected from the originator list. 3245 The constant "None" MUST always be used to indicate a missing 3246 function. However, "None" is only a valid selection if it is 3247 explicitly proposed. 3249 If an acceptor does not understand any particular value in a list, 3250 it MUST ignore it. If an acceptor does not support, does not 3251 understand, or is not allowed to use any of the proposed options 3252 with a specific originator, it may use the constant "Reject" or 3253 terminate the negotiation. The selection of a value not proposed 3254 MUST be handled as a protocol error. 3256 5.2.2. Simple-value Negotiations 3258 For simple-value negotiations, the accepting party MUST answer with 3259 the same key. The value it selects becomes the negotiation result. 3261 Proposing a value not admissible (e.g., not within the specified 3262 bounds) MAY be answered with the constant "Reject" or the acceptor 3263 MAY select an admissible value. 3265 The selection, by the acceptor, of a value not admissible under the 3266 selection rules is considered a protocol error. The selection rules 3267 are key-specific. 3269 For a numerical range the value selected must be an integer within 3270 the proposed range or "Reject" (if the range is unacceptable). 3272 For Boolean negotiations (i.e., keys taking the values Yes or No), 3273 the accepting party MUST answer with the same key and the result of 3274 the negotiation when the received value does not determine that 3275 result by itself. The last value transmitted becomes the 3276 negotiation result. The rules for selecting the value to answer 3277 with are expressed as Boolean functions of the value received, and 3278 the value that the accepting party would have selected if given a 3279 choice. 3281 Specifically, the two cases in which answers are OPTIONAL are: 3283 - The Boolean function is "AND" and the value "No" is received. 3284 The outcome of the negotiation is "No". 3286 - The Boolean function is "OR" and the value "Yes" is received. 3287 The outcome of the negotiation is "Yes". 3289 Responses are REQUIRED in all other cases, and the value chosen and 3290 sent by the acceptor becomes the outcome of the negotiation. 3292 5.3. Login Phase 3294 The Login Phase establishes an iSCSI connection between an 3295 initiator and a target; it creates also a new session or associates 3296 the connection to an existing session. The Login Phase sets the 3297 iSCSI protocol parameters, security parameters, and authenticates 3298 the initiator and target to each other. 3300 The Login Phase is only implemented via Login request and 3301 responses. The whole Login Phase is considered as a single task and 3303 has a single Initiator Task Tag (similar to the linked SCSI 3304 commands). 3306 There MUST NOT be more than one outstanding Login Request, or Login 3307 Response on an iSCSI connection. An outstanding PDU in this 3308 context is one that has not been acknowledged by the remote iSCSI 3309 side. 3311 The default MaxRecvDataSegmentLength is used during Login. 3313 The Login Phase sequence of requests and responses proceeds as 3314 follows: 3316 - Login initial request 3318 - Login partial response (optional) 3320 - More Login requests and responses (optional) 3322 - Login Final-Response (mandatory) 3324 The initial login request of any connection MUST include the 3325 InitiatorName key=value pair. The initial login request of the 3326 first connection of a session MAY also include the SessionType 3327 key=value pair. For any connection within a session whose type is 3328 not "Discovery", the first login request MUST also include the 3329 TargetName key=value pair. 3331 The Login Final-response accepts or rejects the Login request. 3333 The Login Phase MAY include a SecurityNegotiation stage and a 3334 LoginOperationalNegotiation stage and MUST include at least one of 3335 them, but the included stage MAY be empty except for the mandatory 3336 names. 3338 The login requests and responses contain a field (CSG) that 3339 indicates the current negotiation stage (SecurityNegotiation or 3340 LoginOperationalNegotiation). If both stages are used, the 3341 SecurityNegotiation MUST precede the LoginOperationalNegotiation. 3343 Some operational parameters can be negotiated outside the login 3344 through Text requests and responses. 3346 Security MUST be completely negotiated within the Login Phase. The 3347 use of underlying IPsec security is specified in Chapter 8 and in 3348 [RFC3723]. iSCSI support for security within the protocol only 3349 consists of authentication in the Login Phase. 3351 In some environments, a target or an initiator is not interested in 3352 authenticating its counterpart. It is possible to bypass 3353 authentication through the Login request and response. 3355 The initiator and target MAY want to negotiate iSCSI authentication 3356 parameters. Once this negotiation is completed, the channel is 3357 considered secure. 3359 Most of the negotiation keys are only allowed in a specific stage. 3360 The SecurityNegotiation keys appear in Chapter 11 and the 3361 LoginOperationalNegotiation keys appear in Chapter 12. Only a 3362 limited set of keys (marked as Any-Stage in Chapter 12) may be used 3363 in any of the two stages. 3365 Any given Login request or response belongs to a specific stage; 3366 this determines the negotiation keys allowed with the request or 3367 response. It is considered to be a protocol error to send a key not 3368 allowed in the current stage. 3370 Stage transition is performed through a command exchange 3371 (request/response) that carries the T bit and the same CSG code. 3372 During this exchange, the next stage is selected by the target 3373 through the "next stage" code (NSG). The selected NSG MUST NOT 3374 exceed the value stated by the initiator. The initiator can request 3375 a transition whenever it is ready, but a target can only respond 3376 with a transition after one is proposed by the initiator. 3378 In a negotiation sequence, the T bit settings in one pair of login 3379 request-responses have no bearing on the T bit settings of the next 3380 pair. An initiator that has a T bit set to 1 in one pair and is 3381 answered with a T bit setting of 0 may issue the next request with 3382 T bit set to 0. 3384 When a transition is requested by the initiator and acknowledged by 3385 the target, both the initiator and target switch to the selected 3386 stage. 3388 Targets MUST NOT submit parameters that require an additional 3389 initiator login request in a login response with the T bit set to 3390 1. 3392 Stage transitions during login (including entering and exit) are 3393 only possible as outlined in the following table: 3395 +-----------------------------------------------------------+ 3396 |From To -> | Security | Operational | FullFeature | 3397 | | | | | | 3398 | V | | | | 3399 +-----------------------------------------------------------+ 3400 | (start) | yes | yes | no | 3401 +-----------------------------------------------------------+ 3402 | Security | no | yes | yes | 3403 +-----------------------------------------------------------+ 3404 | Operational | no | no | yes | 3405 +-----------------------------------------------------------+ 3407 The Login Final-Response that accepts a Login Request can only come 3408 as a response to a Login request with the T bit set to 1, and both 3409 the request and response MUST indicate FullFeaturePhase as the next 3410 phase via the NSG field. 3412 Neither the initiator nor the target should attempt to declare or 3413 negotiate a parameter more than once during login except for 3414 responses to specific keys that explicitly allow repeated key 3415 declarations (e.g., TargetAddress). An attempt to 3416 renegotiate/redeclare parameters not specifically allowed MUST be 3417 detected by the initiator and target. If such an attempt is 3418 detected by the target, the target MUST respond with Login reject 3419 (initiator error); if detected by the initiator, the initiator MUST 3420 drop the connection. 3422 5.3.1. Login Phase Start 3424 The Login Phase starts with a login request from the initiator to 3425 the target. The initial login request includes: 3427 -Protocol version supported by the initiator. 3429 -iSCSI Initiator Name and iSCSI Target Name 3431 -ISID, TSIH, and connection Ids 3433 -Negotiation stage that the initiator is ready to enter. 3435 A login may create a new session or it may add a connection to an 3436 existing session. Between a given iSCSI Initiator Node (selected 3437 only by an InitiatorName) and a given iSCSI target defined by an 3438 iSCSI TargetName and a Target Portal Group Tag, the login results 3439 are defined by the following table: 3441 +------------------------------------------------------------------ 3442 + 3443 |ISID | TSIH | CID | Target action 3444 | 3445 +------------------------------------------------------------------ 3446 + 3447 |new | non-zero | any | fail the login 3448 | 3449 | | | | ("session does not exist") 3450 | 3451 +------------------------------------------------------------------ 3452 + 3453 |new | zero | any | instantiate a new session 3454 | 3455 +------------------------------------------------------------------ 3456 + 3457 |existing | zero | any | do session reinstatement 3458 | 3459 | | | | (see Section 5.3.5) 3460 | +---------------------------------------------------------------- 3461 --+ 3462 |existing | non-zero | new | add a new connection to 3463 | 3464 | | existing | | the session 3465 | 3466 +------------------------------------------------------------------ 3467 + 3468 |existing | non-zero |existing| do connection 3469 reinstatement| 3470 | | existing | | (see Conne) | 3471 +------------------------------------------------------------------ 3472 + 3473 |existing | non-zero | any | fail the login 3474 | 3475 | | new | | ("session does not exist") 3476 | 3477 +------------------------------------------------------------------ 3478 + 3480 Determination of "existing" or "new" are made by the target. 3482 Optionally, the login request may include: 3484 -Security parameters 3485 OR 3487 -iSCSI operational parameters 3488 AND/OR 3490 -The next negotiation stage that the initiator is ready to 3491 enter. 3493 The target can answer the login in the following ways: 3495 -Login Response with Login reject. This is an immediate 3496 rejection from the target that causes the connection to 3497 terminate and the session to terminate if this is the first 3498 (or only) connection of a new session. The T bit and the CSG 3499 and NSG fields are reserved. 3501 -Login Response with Login accept as a final response (T bit 3502 set to 1 and the NSG in both request and response are set to 3503 FullFeaturePhase). The response includes the protocol version 3504 supported by the target and the session ID, and may include 3505 iSCSI operational or security parameters (that depend on the 3506 current stage). 3508 -Login Response with Login Accept as a partial response (NSG 3509 not set to FullFeaturePhase in both request and response) 3510 that indicates the start of a negotiation sequence. The 3511 response includes the protocol version supported by the 3512 target and either security or iSCSI parameters (when no 3513 security mechanism is chosen) supported by the target. 3515 If the initiator decides to forego the SecurityNegotiation stage, 3516 it issues the Login with the CSG set to LoginOperationalNegotiation 3517 and the target may reply with a Login Response that indicates that 3518 it is unwilling to accept the connection (see Section 10.13 - 3519 "Login Response") without SecurityNegotiation and will terminate 3520 the connection with a response of Authentication failure (see 3521 Section 10.13.5 - "Status-Class and Status-Detail"). 3523 If the initiator is willing to negotiate iSCSI security, but is 3524 unwilling to make the initial parameter proposal and may accept a 3525 connection without iSCSI security, it issues the Login with the T 3526 bit set to 1, the CSG set to SecurityNegotiation, and NSG set to 3527 LoginOperationalNegotiation. If the target is also ready to skip 3528 security, the login response only contains the TargetPortalGroupTag 3529 key (see Section 12.9 - "TargetPortalGroupTag"), the T bit set to 3530 1, the CSG set to SecurityNegotiation, and NSG set to 3531 LoginOperationalNegotiation. 3533 An initiator that chooses to operate without iSCSI security and 3534 with all the operational parameters taking the default values 3535 issues the Login with the T bit set to 1, the CSG set to 3536 LoginOperationalNegotiation, and NSG set to FullFeaturePhase. If 3537 the target is also ready to forego security and can finish its 3538 LoginOperationalNegotiation, the Login response has T bit set to 1, 3539 the CSG set to LoginOperationalNegotiation, and NSG set to 3540 FullFeaturePhase in the next stage. 3542 During the Login Phase the iSCSI target MUST return the 3543 TargetPortalGroupTag key with the first Login Response PDU with 3544 which it is allowed to do so (i.e., the first Login Response issued 3545 after the first Login Request with the C bit set to 0) for all 3546 session types. The TargetPortalGroupTag key value indicates the 3547 iSCSI portal group servicing the Login Request PDU. If the 3548 reconfiguration of iSCSI portal groups is a concern in a given 3549 environment, the iSCSI initiator should use this key to ascertain 3550 that it had indeed initiated the Login Phase with the intended 3551 target portal group. 3553 5.3.2. iSCSI Security Negotiation 3555 The security exchange sets the security mechanism and authenticates 3556 the initiator user and the target to each other. The exchange 3557 proceeds according to the authentication method chosen in the 3558 negotiation phase and is conducted using the login requests and 3559 responses key=value parameters. 3561 An initiator directed negotiation proceeds as follows: 3563 -The initiator sends a login request with an ordered list of 3564 the options it supports (authentication algorithm). The 3565 options are listed in the initiator's order of preference. 3566 The initiator MAY also send private or public extension 3567 options. 3569 -The target MUST reply with the first option in the list it 3570 supports and is allowed to use for the specific initiator 3571 unless it does not support any in which case it MUST answer 3572 with "Reject" (see Text Mode Negotiation). The parameters are 3573 encoded in UTF8 as key=value. For security parameters, see 3574 Chapter 11. 3576 -When the initiator considers that it is ready to conclude the 3577 SecurityNegotiation stage, it sets the T bit to 1 and the NSG 3578 to what it would like the next stage to be. The target will 3579 then set the T bit to 1 and set NSG to the next stage in the 3580 Login response when it finishes sending its security keys. 3582 The next stage selected will be the one the target selected. 3583 If the next stage is FullFeaturePhase, the target MUST 3584 respond with a Login Response with the TSIH value. 3586 If the security negotiation fails at the target, then the target 3587 MUST send the appropriate Login Response PDU. If the security 3588 negotiation fails at the initiator, the initiator SHOULD close the 3589 connection. 3591 It should be noted that the negotiation might also be directed by 3592 the target if the initiator does support security, but is not ready 3593 to direct the negotiation (propose options). 3595 5.3.3. Operational Parameter Negotiation During the Login Phase 3597 Operational parameter negotiation during the login MAY be done: 3599 - Starting with the first Login request if the initiator does 3600 not propose any security/ integrity option. 3602 - Starting immediately after the security negotiation if the 3603 initiator and target perform such a negotiation. 3605 Operational parameter negotiation MAY involve several Login 3606 request-response exchanges started and terminated by the initiator. 3607 The initiator MUST indicate its intent to terminate the negotiation 3608 by setting the T bit to 1; the target sets the T bit to 1 on the 3609 last response. 3611 If the target responds to a Login request that has the T bit set to 3612 1 with a Login response that has the T bit set to 0, the initiator 3613 should keep sending the Login request (even empty) with the T bit 3614 set to 1, while it still wants to switch stage, until it receives 3615 the Login Response that has the T bit set to 1 or it receives a key 3616 that requires it to set the T bit to 0. 3618 Some session specific parameters can only be specified during the 3619 Login Phase of the first connection of a session (i.e., begun by a 3620 login request that contains a zero-valued TSIH) - the leading Login 3621 Phase (e.g., the maximum number of connections that can be used for 3622 this session). 3624 A session is operational once it has at least one connection in 3625 FullFeaturePhase. New or replacement connections can only be added 3626 to a session after the session is operational. 3628 For operational parameters, see Chapter 12. 3630 5.3.4. Connection Reinstatement 3632 Connection reinstatement is the process of an initiator logging in 3633 with a ISID-TSIH-CID combination that is possibly active from the 3634 targets perspective, which causes the implicit logging out of the 3635 connection corresponding to the CID and reinstating a new Full 3636 Feature Phase iSCSI connection in its place (with the same CID). 3637 Thus, the TSIH in the Login Request PDU MUST be non-zero and CID 3638 does not change during a connection reinstatement. The Login 3639 request performs the logout function of the old connection if an 3640 explicit logout was not performed earlier. In sessions with a 3641 single connection, this may imply the opening of a second 3642 connection with the sole purpose of cleaning up the first. Targets 3643 MUST support opening a second connection even when they do not 3644 support multiple connections in Full Feature Phase if 3645 ErrorRecoveryLevel is 2 and SHOULD support opening a second 3646 connection if ErrorRecoveryLevel is less than 2. 3648 If the operational ErrorRecoveryLevel is 2, connection 3649 reinstatement enables future task reassignment. If the operational 3650 ErrorRecoveryLevel is less than 2, connection reinstatement is the 3651 replacement of the old CID without enabling task reassignment. In 3652 this case, all the tasks that were active on the old CID must be 3653 immediately terminated without further notice to the initiator. 3655 The initiator connection state MUST be CLEANUP_WAIT (section 7.1.3) 3656 when the initiator attempts a connection reinstatement. 3658 In practical terms, in addition to the implicit logout of the old 3659 connection, reinstatement is equivalent to a new connection login. 3661 5.3.5. Session Reinstatement, Closure, and Timeout 3663 Session reinstatement is the process of the initiator logging in 3664 with an ISID that is possibly active from the targets perspective. 3665 Thus implicitly logging out the session that corresponds to the 3666 ISID and reinstating a new iSCSI session in its place (with the 3667 same ISID). Therefore, the TSIH in the Login PDU MUST be zero to 3668 signal session reinstatement. Session reinstatement causes all the 3669 tasks that were active on the old session to be immediately 3670 terminated by the target without further notice to the initiator. 3672 The initiator session state MUST be FAILED (Section 7.3 - "Session 3673 State Diagrams") when the initiator attempts a session 3674 reinstatement. 3676 Session closure is an event defined to be one of the following: 3678 - A successful "session close" logout. 3680 - A successful "connection close" logout for the last Full 3681 Feature Phase connection when no other connection in the 3682 session is waiting for cleanup (Section 7.2 - "Connection 3683 Cleanup State Diagram for Initiators and Targets") and no 3684 tasks in the session are waiting for reassignment. 3686 Session timeout is an event defined to occur when the last 3687 connection state timeout expires and no tasks are waiting for 3688 reassignment. This takes the session to the FREE state (N6 3689 transition in the session state diagram). 3691 5.3.5.1. Loss of Nexus Notification 3693 The iSCSI layer provides the SCSI layer with the "I_T nexus loss" 3694 notification when any one of the following events happens: 3696 Successful completion of session reinstatement. 3697 Session closure event. 3699 Session timeout event. 3701 Certain SCSI object clearing actions may result due to the 3702 notification in the SCSI end nodes, as documented in Appendix F. - 3703 Clearing Effects of Various Events on Targets. 3705 5.3.6. Session Continuation and Failure 3707 Session continuation is the process by which the state of a 3708 preexisting session continues to be used by connection 3709 reinstatement (Section 5.3.4), or by adding a connection with a new 3710 CID. Either of these actions associates the new transport 3711 connection with the session state. 3713 Session failure is an event where the last Full Feature Phase 3714 connection reaches the CLEANUP_WAIT state (Section 7.2), or 3715 completes a successful recovery logout thus causing all active 3716 tasks (that are formerly allegiant to the connection) to start 3717 waiting for task reassignment. 3719 5.4. Operational Parameter Negotiation Outside the Login Phase 3721 Some operational parameters MAY be negotiated outside (after) the 3722 Login Phase. 3724 Parameter negotiation in Full Feature Phase is done through Text 3725 requests and responses. Operational parameter negotiation MAY 3726 involve several Text request-response exchanges, which the 3727 initiator always starts, terminates, and uses the same Initiator 3728 Task Tag. The initiator MUST indicate its intent to terminate the 3729 negotiation by setting the F bit to 1; the target sets the F bit to 3730 1 on the last response. 3732 If the target responds to a Text request with the F bit set to 1 3733 with a Text response with the F bit set to 0 , the initiator should 3734 keep sending the Text request (even empty) with the F bit set to 1, 3735 while it still wants to finish the negotiation, until it receives 3736 the Text response with the F bit set to 1. Responding to a Text 3737 request with the F bit set to 1 with an empty (no key=value pairs) 3738 response with the F bit set to 0 is discouraged. 3740 Targets MUST NOT submit parameters that require an additional 3741 initiator Text request in a Text response with the F bit set to 1. 3743 In a negotiation sequence, the F bit settings in one pair of Text 3744 request-responses have no bearing on the F bit settings of the next 3745 pair. An initiator that has the F bit set to 1 in a request and is 3746 being answered with an F bit setting of 0 may issue the next 3747 request with the F bit set to 0. 3749 Whenever the target responds with the F bit set to 0, it MUST set 3750 the Target Transfer Tag to a value other than the default 3751 0xffffffff. 3753 An initiator MAY reset an operational parameter negotiation by 3754 issuing a Text request with the Target Transfer Tag set to the 3755 value 0xffffffff after receiving a response with the Target 3756 Transfer Tag set to a value other than 0xffffffff. A target may 3757 reset an operational parameter negotiation by answering a Text 3758 request with a Reject PDU. 3760 Neither the initiator nor the target should attempt to declare or 3761 negotiate a parameter more than once during any negotiation 3762 sequence, except for responses to specific keys that explicitly 3763 allow repeated key declarations (e.g., TargetAddress). If detected 3764 by the target, this MUST result in a Reject PDU with a reason of 3765 "protocol error". The initiator MUST reset the negotiation as 3766 outlined above. 3768 Parameters negotiated by a text exchange negotiation sequence only 3769 become effective after the negotiation sequence is completed. 3771 6. iSCSI Error Handling and Recovery 3773 6.1. Overview 3775 6.1.1. Background 3777 The following two considerations prompted the design of much of the 3778 error recovery functionality in iSCSI: 3780 An iSCSI PDU may fail the digest check and be dropped, despite 3781 being received by the TCP layer. The iSCSI layer must 3782 optionally be allowed to recover such dropped PDUs. 3784 A TCP connection may fail at any time during the data transfer. 3785 All the active tasks must optionally be allowed to be 3786 continued on a different TCP connection within the same 3787 session. 3789 Implementations have considerable flexibility in deciding what 3790 degree of error recovery to support, when to use it and by which 3791 mechanisms to achieve the required behavior. Only the externally 3792 visible actions of the error recovery mechanisms must be 3793 standardized to ensure interoperability. 3795 This chapter describes a general model for recovery in support of 3796 interoperability. See Appendix E. - "Algorithmic Presentation of 3797 Error Recovery Classes" for further detail on how the described 3798 model may be implemented. Compliant implementations do not have to 3799 match the implementation details of this model as presented, but 3800 the external behavior of such implementations must correspond to 3801 the externally observable characteristics of the presented model. 3803 6.1.2. Goals 3805 The major design goals of the iSCSI error recovery scheme are as 3806 follows: 3808 Allow iSCSI implementations to meet different requirements 3809 by defining a collection of error recovery mechanisms that 3810 implementations may choose from. 3811 Ensure interoperability between any two implementations 3812 supporting different sets of error recovery capabilities. 3814 Define the error recovery mechanisms to ensure command 3815 ordering even in the face of errors, for initiators that 3816 demand ordering. 3817 Do not make additions in the fast path, but allow moderate 3818 complexity in the error recovery path. 3819 Prevent both the initiator and target from attempting to 3820 recover the same set of PDUs at the same time. For example, 3821 there must be a clear "error recovery functionality 3822 distribution" between the initiator and target. 3824 6.1.3. Protocol Features and State Expectations 3826 The initiator mechanisms defined in connection with error recovery 3827 are: 3829 NOP-OUT to probe sequence numbers of the target (Section 10.18) 3830 Command retry (Section 6.2.1) 3831 Recovery R2T support (Section 6.8) 3832 Requesting retransmission of status/data/R2T using the SNACK 3833 facility (section 10.16) 3834 Acknowledging the receipt of the data (section 10.16) 3835 Reassigning the connection allegiance of a task to a 3836 different TCP connection (Section 6.2.2) 3837 Terminating the entire iSCSI session to start afresh (Session 3838 Recovery) 3840 The target mechanisms defined in connection with error recovery 3841 are: 3843 NOP-IN to probe sequence numbers of the initiator (section 3844 10.19) 3845 Requesting retransmission of data using the recovery R2T 3846 feature (iSCSI Erro) 3847 SNACK support (section 10.16) 3848 Requesting that parts of read data be acknowledged (section 3849 10.7.2) 3850 Allegiance reassignment support (Section 6.2.2) 3851 Terminating the entire iSCSI session to force the initiator 3852 to start over (Session Recovery) 3854 For any outstanding SCSI command, it is assumed that iSCSI, in 3855 conjunction with SCSI at the initiator, is able to keep enough 3856 information to be able to rebuild the command PDU, and that 3857 outgoing data is available (in host memory) for retransmission 3858 while the command is outstanding. It is also assumed that at the 3859 target, incoming data (read data) MAY be kept for recovery or it 3860 can be reread from a device server. 3862 It is further assumed that a target will keep the "status & sense" 3863 for a command it has executed if it supports status retransmission. 3864 A target that agrees to support data retransmission is expected to 3865 be prepared to retransmit the outgoing data (i.e., Data-In) on 3866 request until either the status for the completed command is 3867 acknowledged, or the data in question has been separately 3868 acknowledged. 3870 6.1.4. Recovery Classes 3872 iSCSI enables the following classes of recovery (in the order of 3873 increasing scope of affected iSCSI tasks): 3875 - Within a command (i.e., without requiring command restart). 3877 - Within a connection (i.e., without requiring the connection 3878 to be rebuilt, but perhaps requiring command restart). 3880 - Connection recovery (i.e., perhaps requiring connections to 3881 be rebuilt and commands to be reissued). 3883 - Session recovery. 3885 The recovery scenarios detailed in the rest of this section are 3886 representative rather than exclusive. In every case, they detail 3887 the lowest class recovery that MAY be attempted. The implementer is 3888 left to decide under which circumstances to escalate to the next 3889 recovery class and/or what recovery classes to implement. Both the 3890 iSCSI target and initiator MAY escalate the error handling to an 3891 error recovery class, which impacts a larger number of iSCSI tasks 3892 in any of the cases identified in the following discussion. 3894 In all classes, the implementer has the choice of deferring errors 3895 to the SCSI initiator (with an appropriate response code), in which 3896 case the task, if any, has to be removed from the target and all 3897 the side-effects, such as ACA, must be considered. 3899 Use of within-connection and within-command recovery classes MUST 3900 NOT be attempted before the connection is in Full Feature Phase. 3902 In the detailed description of the recover classes the 3903 mandating terms (MUST, SHOULD, MAY, etc.) indicate normative 3904 actions to be executed if the recovery class is supported and used. 3906 6.1.4.1. Recovery Within-command 3908 At the target, the following cases lend themselves to within- 3909 command recovery: 3911 - Lost data PDU - realized through one of the following: 3913 Data digest error - dealt with as specified in Section 6.8, 3914 using the option of a recovery R2T. 3915 Sequence reception timeout (no data or partial-data-and-no-F- 3916 bit) - considered an implicit sequence error and dealt with 3917 as specified in Section 6.9, using the option of a recovery 3918 R2T. 3919 Header digest error, which manifests as a sequence reception 3920 timeout or a sequence error - dealt with as specified in 3921 Section 6.9, using the option of a recovery R2T. 3923 At the initiator, the following cases lend themselves to within- 3924 command recovery: 3926 Lost data PDU or lost R2T - realized through one of the 3927 following: 3929 Data digest error - dealt with as specified in Section 6.8, 3930 using the option of a SNACK. 3931 Sequence reception timeout (no status) or response reception 3932 timeout - dealt with as specified in Section 6.9, using the 3933 option of a SNACK. 3935 Header digest error, which manifests as a sequence reception 3936 timeout or a sequence error - dealt with as specified in 3937 Section 6.9, using the option of a SNACK. 3939 To avoid a race with the target, which may already have a recovery 3940 R2T or a termination response on its way, an initiator SHOULD NOT 3941 originate a SNACK for an R2T based on its internal timeouts (if 3942 any). Recovery in this case is better left to the target. 3944 The timeout values used by the initiator and target are outside the 3945 scope of this document. Sequence reception timeout is generally a 3946 large enough value to allow the data sequence transfer to be 3947 complete. 3949 6.1.4.2. Recovery Within-connection 3951 At the initiator, the following cases lend themselves to within- 3952 connection recovery: 3954 - Requests not acknowledged for a long time. Requests are 3955 acknowledged explicitly through ExpCmdSN or implicitly by 3956 receiving data and/or status. The initiator MAY retry non- 3957 acknowledged commands as specified in Retry an. 3959 - Lost iSCSI numbered Response. It is recognized by either 3960 identifying a data digest error on a Response PDU or a Data- 3961 In PDU carrying the status, or by receiving a Response PDU 3962 with a higher StatSN than expected. In the first case, digest 3963 error handling is done as specified in Section 6.8 using the 3964 option of a SNACK. In the second case, sequence error 3965 handling is done as specified in Section 6.9, using the 3966 option of a SNACK. 3968 At the target, the following cases lend themselves to within- 3969 connection recovery: 3971 - Status/Response not acknowledged for a long time. The target 3972 MAY issue a NOP-IN (with a valid Target Transfer Tag or 3973 otherwise) that carries the next status sequence number it is 3974 going to use in the StatSN field. This helps the initiator 3975 detect any missing StatSN(s) and issue a SNACK for the 3976 status. 3978 The timeout values used by the initiator and the target are outside 3979 the scope of this document. 3981 6.1.4.3. Connection Recovery 3983 At an iSCSI initiator, the following cases lend themselves to 3984 connection recovery: 3986 - TCP connection failure: The initiator MUST close the 3987 connection. It then MUST either implicitly or explicitly 3988 logout the failed connection with the reason code "remove the 3989 connection for recovery" and reassign connection allegiance 3990 for all commands still in progress associated with the failed 3991 connection on one or more connections (some or all of which 3992 MAY be newly established connections) using the "Task 3993 reassign" task management function (see Section 10.5.1 - 3994 "Function"). For an initiator, a command is in progress as 3995 long as it has not received a response or a Data-In PDU 3996 including status. 3998 Note: The logout function is mandatory. However, a new 3999 connection establishment is only mandatory if the failed 4000 connection was the last or only connection in the session. 4002 - Receiving an Asynchronous Message that indicates one or all 4003 connections in a session has been dropped. The initiator 4004 MUST handle it as a TCP connection failure for the 4005 connection(s) referred to in the Message. 4007 At an iSCSI target, the following cases lend themselves to 4008 connection recovery: 4010 - TCP connection failure. The target MUST close the connection 4011 and, if more than one connection is available, the target 4012 SHOULD send an Asynchronous Message that indicates it has 4013 dropped the connection. Then, the target will wait for the 4014 initiator to continue recovery. 4016 6.1.4.4. Session Recovery 4018 Session recovery should be performed when all other recovery 4019 attempts have failed. Very simple initiators and targets MAY 4020 perform session recovery on all iSCSI errors and rely on recovery 4021 on the SCSI layer and above. 4023 Session recovery implies the closing of all TCP connections, 4024 internally aborting all executing and queued tasks for the given 4025 initiator at the target, terminating all outstanding SCSI commands 4026 with an appropriate SCSI service response at the initiator, and 4027 restarting a session on a new set of connection(s) (TCP connection 4028 establishment and login on all new connections). 4030 For possible clearing effects of session recovery on SCSI and iSCSI 4031 objects, refer to Appendix F. - "Clearing Effects of Various Events 4032 on Targets". 4034 6.1.5. Error Recovery Hierarchy 4036 The error recovery classes described so far are organized into a 4037 hierarchy for ease in understanding and to limit the implementation 4038 complexity. With few and well defined recovery levels 4039 interoperability is easier to achieve. The attributes of this 4040 hierarchy are as follows: 4042 Each level is a superset of the capabilities of the previous 4043 level. For example, Level 1 support implies supporting all 4044 capabilities of Level 0 and more. 4045 As a corollary, supporting a higher error recovery level means 4046 increased sophistication and possibly an increase in 4047 resource requirements. 4048 Supporting error recovery level "n" is advertised and 4049 negotiated by each iSCSI entity by exchanging the text key 4051 "ErrorRecoveryLevel=n". The lower of the two exchanged 4052 values is the operational ErrorRecoveryLevel for the 4053 session. 4055 The following diagram represents the error recovery hierarchy. 4057 + 4058 / \ 4059 / 2 \ <-- Connection recovery 4060 +-----+ 4061 / 1 \ <-- Digest failure recovery 4062 +---------+ 4063 / 0 \ <-- Session failure recovery 4064 +-------------+ 4066 The following table lists the error recovery capabilities expected 4067 from the implementations that support each error recovery level. 4069 +-------------------+--------------------------------------------+ 4070 |ErrorRecoveryLevel | Associated Error recovery capabilities | 4071 +-------------------+--------------------------------------------+ 4072 | 0 | Session recovery class | 4073 | | (Session Recovery) | 4074 +-------------------+--------------------------------------------+ 4075 | 1 | Digest failure recovery (See Note below.) | 4076 | | plus the capabilities of ER Level 0 | 4077 +-------------------+--------------------------------------------+ 4078 | 2 | Connection recovery class | 4079 | | (Connection Recovery) | 4080 | | plus the capabilities of ER Level 1 | 4081 +-------------------+--------------------------------------------+ 4083 Note: Digest failure recovery is comprised of two recovery classes: 4084 Within-Connection recovery class (Recovery Within-connection) and 4085 Within-Command recovery class (Recovery Within-command ). 4087 When a defined value of ErrorRecoveryLevel is proposed by an 4088 originator in a text negotiation, the originator MUST support the 4089 functionality defined for the proposed value and additionally, 4090 functionality corresponding to any defined value numerically less 4091 than the proposed. When a defined value of ErrorRecoveryLevel is 4092 returned by a responder in a text negotiation, the responder MUST 4093 support the functionality corresponding to the ErrorRecoveryLevel 4094 it is accepting. 4096 When either party attempts to use error recovery functionality 4097 beyond what is negotiated, the recovery attempts MAY fail unless an 4098 apriori agreement outside the scope of this document exists between 4099 the two parties to provide such support. 4101 Implementations MUST support error recovery level "0", while the 4102 rest are OPTIONAL to implement. In implementation terms, the above 4103 striation means that the following incremental sophistication with 4104 each level is required. 4106 +-------------------+---------------------------------------------+ 4107 |Level transition | Incremental requirement | 4108 +-------------------+---------------------------------------------+ 4109 | 0->1 | PDU retransmissions on the same connection | 4110 +-------------------+---------------------------------------------+ 4111 | 1->2 | Retransmission across connections and | 4112 | | allegiance reassignment | 4113 +-------------------+---------------------------------------------+ 4115 6.2. Retry and Reassign in Recovery 4117 This section summarizes two important and somewhat related iSCSI 4118 protocol features used in error recovery. 4120 6.2.1. Usage of Retry 4122 By resending the same iSCSI command PDU ("retry") in the absence of 4123 a command acknowledgement (by way of an ExpCmdSN update) or a 4124 response, an initiator attempts to "plug" (what it thinks are) the 4125 discontinuities in CmdSN ordering on the target end. Discarded 4126 command PDUs, due to digest errors, may have created these 4127 discontinuities. 4129 Retry MUST NOT be used for reasons other than plugging command 4130 sequence gaps, and in particular, cannot be used for requesting PDU 4131 retransmissions from a target. Any such PDU retransmission requests 4132 for a currently allegiant command in progress may be made using the 4133 SNACK mechanism described in section 10.16, although the usage of 4134 SNACK is OPTIONAL. 4136 If initiators, as part of plugging command sequence gaps as 4137 described above, inadvertently issue retries for allegiant commands 4138 already in progress (i.e., targets did not see the discontinuities 4139 in CmdSN ordering), the duplicate commands are silently ignored by 4140 targets as specified in section 3.2.2.1. 4142 When an iSCSI command is retried, the command PDU MUST carry the 4143 original Initiator Task Tag and the original operational attributes 4144 (e.g., flags, function names, LUN, CDB etc.) as well as the 4145 original CmdSN. The command being retried MUST be sent on the same 4146 connection as the original command unless the original connection 4147 was already successfully logged out. 4149 6.2.2. Allegiance Reassignment 4151 By issuing a "task reassign" task management request (Section 4152 10.5.1 - "Function"), the initiator signals its intent to continue 4153 an already active command (but with no current connection 4154 allegiance) as part of connection recovery. This means that a new 4155 connection allegiance is requested for the command, which seeks to 4156 associate it to the connection on which the task management request 4157 is being issued. Before the allegiance reassignment is attempted 4158 for a task, an implicit or explicit Logout with the reason code 4159 "remove the connection for recovery" ( see section 10.14) MUST be 4160 successfully completed for the previous connection to which the 4161 task was allegiant. 4163 In reassigning connection allegiance for a command, the targets 4164 SHOULD continue the command from its current state. For example, 4165 when reassigning read commands, the target SHOULD take advantage of 4166 the ExpDataSN field provided by the Task Management function 4167 request (which must be set to zero if there was no data transfer) 4168 and bring the read command to completion by sending the remaining 4169 data and sending (or resending) the status. ExpDataSN acknowledges 4170 all data sent up to, but not including, the Data-In PDU and or R2T 4171 with DataSN (or R2TSN) equal to ExpDataSN. However, targets may 4172 choose to send/receive all unacknowledged data or all of the data 4173 on a reassignment of connection allegiance if unable to recover or 4174 maintain accurate an state. Initiators MUST NOT subsequently 4175 request data retransmission through Data SNACK for PDUs numbered 4176 less than ExpDataSN (i.e., prior to the acknowledged sequence 4177 number). For all types of commands, a reassignment request implies 4178 that the task is still considered in progress by the initiator and 4179 the target must conclude the task appropriately if the target 4180 returns the "Function Complete" response to the reassignment 4181 request. This might possibly involve retransmission of 4182 data/R2T/status PDUs as necessary, but MUST involve the 4183 (re)transmission of the status PDU. 4185 It is OPTIONAL for targets to support the allegiance reassignment. 4186 This capability is negotiated via the ErrorRecoveryLevel text key 4187 during the login time. When a target does not support allegiance 4188 reassignment, it MUST respond with a Task Management response code 4189 of "Allegiance reassignment not supported". If allegiance 4190 reassignment is supported by the target, but the task is still 4191 allegiant to a different connection, or a successful recovery 4192 Logout of the previously allegiant connection was not performed, 4193 the target MUST respond with a Task Management response code of 4194 "Task still allegiant". 4196 If allegiance reassignment is supported by the target, the Task 4197 Management response to the reassignment request MUST be issued 4198 before the reassignment becomes effective. 4200 If a SCSI Command that involves data input is reassigned, any SNACK 4201 Tag it holds for a final response from the original connection is 4202 deleted and the default value of 0 MUST be used instead. 4204 6.3. Usage Of Reject PDU in Recovery 4206 Targets MUST NOT implicitly terminate an active task by sending a 4207 Reject PDU for any PDU exchanged during the life of the task. If 4208 the target decides to terminate the task, a Response PDU (SCSI, 4209 Text, Task, etc.) must be returned by the target to conclude the 4210 task. If the task had never been active before the Reject (i.e., 4211 the Reject is on the command PDU), targets should not send any 4212 further responses because the command itself is being discarded. 4214 The above rule means that the initiator can eventually expect a 4215 response on receiving Rejects, if the received Reject is for a PDU 4216 other than the command PDU itself. The non-command Rejects only 4217 have diagnostic value in logging the errors, and they can be used 4218 for retransmission decisions by the initiators. 4220 The CmdSN of the rejected command PDU (if it is a non-immediate 4221 command) MUST NOT be considered received by the target (i.e., a 4222 command sequence gap must be assumed for the CmdSN), even though 4223 the CmdSN of the rejected command PDU may be reliably ascertained. 4224 Upon receiving the Reject, the initiator MUST plug the CmdSN gap in 4225 order to continue to use the session. The gap may be plugged either 4226 by transmitting a command PDU with the same CmdSN, or by aborting 4227 the task (see SCS on how an abort may plug a CmdSN gap). 4229 When a data PDU is rejected and its DataSN can be ascertained, a 4230 target MUST advance ExpDataSN for the current data burst if a 4231 recovery R2T is being generated. The target MAY advance its 4232 ExpDataSN if it does not attempt to recover the lost data PDU. 4234 6.4. Error Recovery Considerations for Discovery Sessions 4236 6.4.1. ErrorRecoveryLevel for Discovery Sessions 4238 The negotiation of the key ErrorRecoveryLevel is not required for 4239 Discovery sessions -- i.e., for sessions that negotiated 4240 "SessionType=Discovery" -- because the default value of 0 is 4241 necessary and sufficient for Discovery sessions. It is however 4242 possible that some legacy iSCSI implementations might attempt to 4243 negotiate the ErrorRecoveryLevel key on Discovery sessions. When 4244 such a negotiation attempt is made by the remote side, a compliant 4245 iSCSI implementation MUST propose a value of 0 (zero) in response. 4246 The operational ErrorRecoveryLevel for Discovery sessions thus MUST 4247 be 0. This naturally follows from the functionality constraints 4248 that Section 3.3 imposes on Discovery sessions. 4250 6.4.2. Reinstatement Semantics for Discovery Sessions 4252 Discovery sessions are intended to be relatively short-lived. 4253 Initiators are not expected to establish multiple Discovery 4254 sessions to the same iSCSI Network Portal. An initiator may use the 4255 same iSCSI Initiator Name and ISID when establishing different 4256 unique sessions with different targets and/or different portal 4257 groups. This behavior is discussed in Section 9.1.1 and is, in 4258 fact, encouraged as conservative reuse of ISIDs. 4260 The ISID RULE in Section 3.4.3 states that there must not be more 4261 than one session with a matching 4-tuple: . While the spirit of the ISID 4263 RULE applies to Discovery sessions the same as it does for Normal 4264 sessions, note that some Discovery sessions differ from the Normal 4265 sessions in two important aspects: 4267 Because Appendix D allows a Discovery session to be established 4268 without specifying a TargetName key in the Login Request PDU 4269 (let us call such a session an "Unnamed" Discovery session), 4270 there is no Target Node context to enforce the ISID RULE. 4272 Portal Groups are defined only in the context of a Target Node. 4273 When the TargetName key is NULL-valued (i.e., not specified), 4274 the TargetPortalGroupTag thus cannot be ascertained to 4275 enforce the ISID RULE. 4277 The following two sections describe each of the two scenarios -- 4278 Named Discovery sessions and Unnamed Discovery sessions. 4280 6.4.2.1. Unnamed Discovery Sessions 4282 For Unnamed Discovery sessions, neither the TargetName nor the 4283 TargetPortalGroupTag is available to the targets in order to 4284 enforce the ISID RULE. So the following rule applies. 4286 UNNAMED ISID RULE: Targets MUST enforce the uniqueness of the 4287 following 4-tuple for Unnamed Discovery sessions: . The following semantics are implied by 4289 this uniqueness requirement. 4291 Targets SHOULD allow concurrent establishment of one Discovery 4292 session with each of its Network Portals by the same initiator port 4293 with a given iSCSI Node Name and an ISID. Each of the concurrent 4294 Discovery sessions, if established by the same initiator port to 4295 other Network Portals, MUST be treated as independent sessions -- 4296 i.e., one session MUST NOT reinstate the other. 4298 A new Unnamed Discovery session that has a matching to an existing Discovery session MUST 4300 reinstate the existing Unnamed Discovery session. Note thus that 4301 only an Unnamed Discovery session may reinstate an Unnamed 4302 Discovery session. 4304 6.4.2.2. Named Discovery Session 4306 For a Named Discovery session, the TargetName key is specified by 4307 the initiator and thus the target can unambiguously ascertain the 4308 TargetPortalGroupTag as well. Since all the four elements of the 4- 4309 tuple are known, the ISID RULE MUST be enforced by targets with no 4310 changes from Section 3.4.3 semantics. A new session with a matching 4311 thus will 4312 reinstate an existing session. Note in this case that any new iSCSI 4313 session (Discovery or Normal) with the matching 4-tuple may 4314 reinstate an existing Named Discovery iSCSI session. 4316 6.4.3. Target PDUs During Discovery 4318 Targets SHOULD NOT send any responses other than a Text Response 4319 and Logout Response on a Discovery session, once in Full Feature 4320 Phase. 4322 Implementation Note: A target may simply drop the connection in a 4323 Discovery session when it would have requested a Logout via an 4324 Async Message on Normal sessions. 4326 6.5. Connection Timeout Management 4328 iSCSI defines two session-global timeout values (in seconds) - 4329 Time2Wait and Time2Retain - that are applicable when an iSCSI Full 4330 Feature Phase connection is taken out of service either 4331 intentionally or by an exception. Time2Wait is the initial "respite 4332 time" before attempting an explicit/implicit Logout for the CID in 4333 question or task reassignment for the affected tasks (if any). 4334 Time2Retain is the maximum time after the initial respite interval 4335 that the task and/or connection state(s) is/are guaranteed to be 4336 maintained on the target to cater to a possible recovery attempt. 4337 Recovery attempts for the connection and/or task(s) SHOULD NOT be 4338 made before Time2Wait seconds, but MUST be completed within 4339 Time2Retain seconds after that initial Time2Wait waiting period. 4341 6.5.1. Timeouts on Transport Exception Events 4343 A transport connection shutdown or a transport reset without any 4344 preceding iSCSI protocol interactions informing the end-points of 4345 the fact causes a Full Feature Phase iSCSI connection to be 4346 abruptly terminated. The timeout values to be used in this case are 4347 the negotiated values of defaultTime2Wait (Section 12.15 - 4348 "DefaultTime2Wait") and DefaultTime2Retain (Section 12.16 - 4349 "DefaultTime2Retain") text keys for the session. 4351 6.5.2. Timeouts on Planned Decommissioning 4353 Any planned decommissioning of a Full Feature Phase iSCSI 4354 connection is preceded by either a Logout Response PDU, or an Async 4355 Message PDU. The Time2Wait and Time2Retain field values (section 4356 10.15) in a Logout Response PDU, and the Parameter2 and Parameter3 4357 fields of an Async Message (AsyncEvent types "drop the connection" 4358 or "drop all the connections"; section 10.9.1) specify the timeout 4359 values to be used in each of these cases. 4361 These timeout values are only applicable for the affected 4362 connection, and the tasks active on that connection. These timeout 4363 values have no bearing on initiator timers (if any) that are 4364 already running on connections or tasks associated with that 4365 session. 4367 6.6. Implicit Termination of Tasks 4369 A target implicitly terminates the active tasks due to iSCSI 4370 protocol dynamics in the following cases: 4372 When a connection is implicitly or explicitly logged out with 4373 the reason code of "Close the connection" and there are 4374 active tasks allegiant to that connection. 4376 When a connection fails and eventually the connection state 4377 times out (state transition M1 in Section 7.2.2 - "State 4378 Transition Descriptions for Initiators and Targets") and 4379 there are active tasks allegiant to that connection. 4381 When a successful Logout with the reason code of "remove the 4382 connection for recovery" is performed while there are active 4383 tasks allegiant to that connection, and those tasks 4384 eventually time out after the Time2Wait and Time2Retain 4385 periods without allegiance reassignment. 4387 When a connection is implicitly or explicitly logged out with 4388 the reason code of "Close the session" and there are active 4389 tasks in that session. 4391 If the tasks terminated in the above cases a), b, c) and d)are SCSI 4392 tasks, they must be internally terminated as if with CHECK 4393 CONDITION status. This status is only meaningful for appropriately 4394 handling the internal SCSI state and SCSI side effects with respect 4395 to ordering because this status is never communicated back as a 4396 terminating status to the initiator. However additional actions may 4397 have to be taken at SCSI level depending on the SCSI context as 4398 defined by the SCSI standards (e.g., queued commands and ACA, UA 4399 for the next command on the I_T nexus in cases a), b), and c) etc. 4400 - see [SAM2] and [SPC3]). 4402 6.7. Format Errors 4404 The following two explicit violations of PDU layout rules are 4405 format errors: 4407 Illegal contents of any PDU header field except the Opcode 4408 (legal values are specified in Section 10 - "iSCSI PDU 4409 Formats"). 4410 Inconsistent field contents (consistent field contents are 4411 specified in Section 10 - "iSCSI PDU Formats"). 4413 Format errors indicate a major implementation flaw in one of the 4414 parties. 4416 When a target or an initiator receives an iSCSI PDU with a format 4417 error, it MUST immediately terminate all transport connections in 4418 the session either with a connection close or with a connection 4419 reset and escalate the format error to session recovery (see 4420 Section Session Recovery). 4422 All initiator-detected PDU construction errors MUST be considered 4423 as format errors. Some examples of such errors are: 4424 - NOP-In with a valid TTT but an invalid LUN 4426 - NOP-In with a valid ITT (i.e., a NOP-In response) and also a 4427 valid TTT 4429 - SCSI Response PDU with Status=CHECK CONDITION, but 4430 DataSegmentLength = 0 4432 6.8. Digest Errors 4434 The discussion of the legal choices in handling digest errors below 4435 excludes session recovery as an explicit option, but either party 4436 detecting a digest error may choose to escalate the error to 4437 session recovery. 4439 When a target or an initiator receives any iSCSI PDU, with a header 4440 digest error, it MUST either discard the header and all data up to 4441 the beginning of a later PDU or close the connection. Because the 4442 digest error indicates that the length field of the header may have 4443 been corrupted, the location of the beginning of a later PDU needs 4444 to be reliably ascertained by other means such as the operation of 4445 a sync and steering layer. 4447 When a target receives any iSCSI PDU with a payload digest error, 4448 it MUST answer with a Reject PDU with a reason code of Data-Digest- 4449 Error and discard the PDU. 4451 - If the discarded PDU is a solicited or unsolicited iSCSI data 4452 PDU (for immediate data in a command PDU, non-data PDU rule 4453 below applies), the target MUST do one of the following: 4455 i) Request retransmission with a recovery R2T. 4456 ii) Terminate the task with a response PDU with a CHECK 4457 CONDITION Status and an iSCSI Condition of "protocol 4458 service CRC error" (Section 10.4.7.2 - "Sense Data"). If 4459 the target chooses to implement this option, it MUST 4460 wait to receive all the data (signaled by a Data PDU 4461 with the final bit set for all outstanding R2Ts) before 4462 sending the response PDU. A task management command 4463 (such as an abort task) from the initiator during this 4464 wait may also conclude the task. 4465 - No further action is necessary for targets if the discarded 4466 PDU is a non-data PDU. In case of immediate data being 4467 present on a discarded command, the immediate data is 4468 implicitly recovered when the task is retried (see Section 4469 6.2.1) followed by the entire data transfer for the task. 4471 When an initiator receives any iSCSI PDU with a payload digest 4472 error, it MUST discard the PDU. 4474 - If the discarded PDU is an iSCSI data PDU, the initiator MUST 4475 do one of the following: 4477 Request the desired data PDU through SNACK. In response to 4478 the SNACK, the target MUST either resend the data 4479 PDU or reject the SNACK with a Reject PDU with a 4480 reason code of "SNACK reject" in which case: 4481 If the status has not already been sent for the command, 4482 the target MUST terminate the command with a CHECK 4483 CONDITION Status and an iSCSI Condition of "SNACK 4484 rejected" (Section 10.4.7.2 - "Sense Data"). 4485 If the status was already sent, no further action is 4486 necessary for the target. The initiator in this 4487 case MUST wait for the status to be received and 4488 then discard it, so as to internally signal the 4489 completion with CHECK CONDITION Status and an 4490 iSCSI Condition of "protocol service CRC error" 4491 (Section 10.4.7.2 - "Sense Data"). 4492 Abort the task and terminate the command with an error. 4494 - If the discarded PDU is a response PDU or an unsolicited PDU 4495 (e.g. Async, Reject), the initiator MUST do one of the 4496 following: 4498 Request PDU retransmission with a status SNACK. 4499 Logout the connection for recovery and continue the tasks 4500 on a different connection instance as described in 4501 Retry an. 4503 Logout to close the connection (abort all the commands 4504 associated with the connection). 4506 Note that an unsolicited PDU carries the next StatSN value on 4507 an iSCSI connection, thereby advancing the StatSN. When an 4508 initiator discards one of these PDUs due to a payload digest 4509 error, the entire PDU including the header MUST be discarded. 4510 Consequently, the initiator MUST treat the exception like a 4511 loss of any other solicited response PDU. 4513 6.9. Sequence Errors 4515 When an initiator receives an iSCSI R2T/data PDU with an out of 4516 order R2TSN/DataSN or a SCSI response PDU with an ExpDataSN that 4517 implies missing data PDU(s), it means that the initiator must have 4518 detected a header or payload digest error on one or more earlier 4519 R2T/data PDUs. The initiator MUST address these implied digest 4520 errors as described in Section 6.8. When a target receives a data 4521 PDU with an out of order DataSN, it means that the target must have 4522 hit a header or payload digest error on at least one of the earlier 4523 data PDUs. The target MUST address these implied digest errors as 4524 described in Section 6.8. 4526 When an initiator receives an iSCSI status PDU with an out of order 4527 StatSN that implies missing responses, it MUST address the one or 4528 more missing status PDUs as described in Section 6.8. As a side 4529 effect of receiving the missing responses, the initiator may 4530 discover missing data PDUs. If the initiator wants to recover the 4531 missing data for a command, it MUST NOT acknowledge the received 4532 responses that start from the StatSN of the relevant command, until 4533 it has completed receiving all the data PDUs of the command. 4535 When an initiator receives duplicate R2TSNs (due to proactive 4536 retransmission of R2Ts by the target) or duplicate DataSNs (due to 4537 proactive SNACKs by the initiator), it MUST discard the duplicates. 4539 6.10. Message Error Checking 4541 In the iSCSI implementations till date, there has been some 4542 uncertainty on the extent to which incoming messages have to be 4543 checked for protocol errors, beyond what is strictly required for 4544 processing the inbound message. This section addresses this 4545 question. 4547 Unless this document requires it, an iSCSI implementation is not 4548 required to do an exhaustive protocol conformance check on an 4549 incoming iSCSI PDU. The iSCSI implementation especially is not 4550 required to double-check the remote iSCSI implementations 4551 conformance to protocol requirements. 4553 6.11. SCSI Timeouts 4555 An iSCSI initiator MAY attempt to plug a command sequence gap on 4556 the target end (in the absence of an acknowledgement of the command 4557 by way of ExpCmdSN) before the ULP timeout by retrying the 4558 unacknowledged command, as described in Section 6.2. 4560 On a ULP timeout for a command (that carried a CmdSN of n), if the 4561 iSCSI initiator intends to continue the session it MUST abort the 4562 command by either using an appropriate Task Management function 4563 request for the specific command, or a "close the connection" 4564 Logout. When using an ABORT TASK, if the ExpCmdSN is still less 4565 than (n+1), the target may see the abort request while missing the 4566 original command itself due to one of the following reasons: 4568 - Original command was dropped due to digest error. 4570 - Connection on which the original command was sent was 4571 successfully logged out. On logout, the unacknowledged 4572 commands issued on the connection being logged out are 4573 discarded. 4575 If the abort request is received and the original command is 4576 missing, targets MUST consider the original command with that 4577 RefCmdSN to be received and issue a Task Management response with 4578 the response code: "Function Complete". This response concludes the 4579 task on both ends. If the abort request is received and the target 4580 can determine (based on the Referenced Task Tag) that the command 4581 was received and executed and also that the response was sent prior 4582 to the abort, then the target MUST respond with the response code 4583 of "Task Does Not Exist". 4585 6.12. Negotiation Failures 4587 Text request and response sequences, when used to set/negotiate 4588 operational parameters, constitute the negotiation/parameter 4589 setting. A negotiation failure is considered to be one or more of 4590 the following: 4592 - None of the choices, or the stated value, is acceptable to 4593 one of the sides in the negotiation. 4595 - The text request timed out and possibly terminated. 4597 - The text request was answered with a Reject PDU. 4599 The following two rules should be used to address negotiation 4600 failures: 4602 - During Login, any failure in negotiation MUST be considered a 4603 login process failure and the Login Phase must be terminated, 4604 and with it, the connection. If the target detects the 4605 failure, it must terminate the login with the appropriate 4606 login response code. 4608 - A failure in negotiation, while in the Full Feature Phase, 4609 will terminate the entire negotiation sequence that may 4610 consist of a series of text requests that use the same 4611 Initiator Task Tag. The operational parameters of the 4612 session or the connection MUST continue to be the values 4613 agreed upon during an earlier successful negotiation (i.e., 4614 any partial results of this unsuccessful negotiation MUST NOT 4615 take effect and MUST be discarded). 4617 6.13. Protocol Errors 4619 Mapping framed messages over a "stream" connection, such as TCP, 4620 makes the proposed mechanisms vulnerable to simple software framing 4621 errors. On the other hand, the introduction of framing mechanisms 4622 to limit the effects of these errors may be onerous on performance 4623 for simple implementations. Command Sequence Numbers and the above 4624 mechanisms for connection drop and reestablishment help handle this 4625 type of mapping errors. 4627 All violations of iSCSI PDU exchange sequences specified in this 4628 draft are also protocol errors. This category of errors can only 4629 be 4630 addressed by fixing the implementations; iSCSI defines Reject and 4631 response codes to enable this. 4633 6.14. Connection Failures 4635 iSCSI can keep a session in operation if it is able to 4636 keep/establish at least one TCP connection between the initiator 4637 and the target in a timely fashion. Targets and/or initiators may 4638 recognize a failing connection by either transport level means 4639 (TCP), a gap in the command sequence number, a response stream that 4640 is not filled for a long time, or by a failing iSCSI NOP (acting as 4641 a ping). The latter MAY be used periodically to increase the speed 4642 and likelihood of detecting connection failures. Initiators and 4643 targets MAY also use the keep-alive option on the TCP connection to 4644 enable early link failure detection on otherwise idle links. 4646 On connection failure, the initiator and target MUST do one of the 4647 following: 4649 - Attempt connection recovery within the session (Connection 4650 Recovery). 4652 - Logout the connection with the reason code "closes the 4653 connection" (Section 10.14.5 - "Implicit termination of 4654 tasks"), re-issue missing commands, and implicitly terminate 4655 all active commands. This option requires support for the 4656 within-connection recovery class (Recovery Within- 4657 connection). 4659 - Perform session recovery (Session Recovery). 4661 Either side may choose to escalate to session recovery (via the 4662 initiator dropping all the connections, or via an Async Message 4663 that announces the similar intent from a target), and the other 4664 side MUST give it precedence. On a connection failure, a target 4665 MUST terminate and/or discard all of the active immediate commands 4666 regardless of which of the above options is used (i.e., immediate 4667 commands are not recoverable across connection failures). 4669 6.15. Session Errors 4671 If all of the connections of a session fail and cannot be 4672 reestablished in a short time, or if initiators detect protocol 4673 errors repeatedly, an initiator may choose to terminate a session 4674 and establish a new session. 4676 In this case, the initiator takes the following actions: 4678 - Resets or closes all the transport connections. 4680 - Terminates all outstanding requests with an appropriate 4681 response before initiating a new session. If the same I_T 4682 nexus is intended to be reestablished, the initiator MUST 4683 employ session reinstatement (see section 5.3.5). 4685 When the session timeout (the connection state timeout for the last 4686 failed connection) happens on the target, it takes the following 4687 actions: 4689 - Resets or closes the TCP connections (closes the session). 4691 - Terminates all active tasks that were allegiant to the 4692 connection(s) that constituted the session. 4694 A target MUST also be prepared to handle a session reinstatement 4695 request from the initiator, that may be addressing session errors. 4697 7. State Transitions 4699 iSCSI connections and iSCSI sessions go through several well- 4700 defined states from the time they are created to the time they are 4701 cleared. 4703 The connection state transitions are described in two separate but 4704 dependent state diagrams for ease in understanding. The first 4705 diagram, "standard connection state diagram", describes the 4706 connection state transitions when the iSCSI connection is not 4707 waiting for, or undergoing, a cleanup by way of an explicit or 4708 implicit Logout. The second diagram, "connection cleanup state 4709 diagram", describes the connection state transitions while 4710 performing the iSCSI connection cleanup. 4712 The "session state diagram" describes the state transitions an 4713 iSCSI session would go through during its lifetime, and it depends 4714 on the states of possibly multiple iSCSI connections that 4715 participate in the session. 4717 States and transitions are described in text, tables and diagrams. 4718 The diagrams are used for illustration. The text and the tables are 4719 the governing specification. 4721 7.1. Standard Connection State Diagrams 4723 7.1.1. State Descriptions for Initiators and Targets 4725 State descriptions for the standard connection state diagram are as 4726 follows: 4727 -S1: FREE 4728 -initiator: State on instantiation, or after successful 4729 connection closure. 4730 -target: State on instantiation, or after successful 4731 connection closure. 4732 -S2: XPT_WAIT 4733 -initiator: Waiting for a response to its transport 4734 connection establishment request. 4735 -target: Illegal 4736 -S3: XPT_UP 4737 -initiator: Illegal 4738 -target: Waiting for the Login process to commence. 4740 -S4: IN_LOGIN 4741 -initiator: Waiting for the Login process to conclude, 4742 possibly involving several PDU exchanges. 4743 -target: Waiting for the Login process to conclude, possibly 4744 involving several PDU exchanges. 4745 -S5: LOGGED_IN 4746 -initiator: In Full Feature Phase, waiting for all internal, 4747 iSCSI, and transport events. 4748 -target: In Full Feature Phase, waiting for all internal, 4749 iSCSI, and transport events. 4750 -S6: IN_LOGOUT 4751 -initiator: Waiting for a Logout response. 4752 -target: Waiting for an internal event signaling completion 4753 of logout processing. 4754 -S7: LOGOUT_REQUESTED 4755 -initiator: Waiting for an internal event signaling 4756 readiness to proceed with Logout. 4757 -target: Waiting for the Logout process to start after 4758 having requested a Logout via an Async Message. 4759 -S8: CLEANUP_WAIT 4760 -initiator: Waiting for the context and/or resources to 4761 initiate the cleanup processing for this CSM. 4762 -target: Waiting for the cleanup process to start for this 4763 CSM. 4764 7.1.2. State Transition Descriptions for Initiators and Targets 4766 -T1: 4767 -initiator: Transport connect request was made (e.g., TCP 4768 SYN sent). 4769 -target: Illegal 4770 -T2: 4771 -initiator: Transport connection request timed out, a 4772 transport reset was received, or an internal event of 4773 receiving a Logout response (success) on another connection 4774 for a "close the session" Logout request was received. 4775 -target:Illegal 4776 -T3: 4777 -initiator: Illegal 4778 -target: Received a valid transport connection request that 4779 establishes the transport connection. 4780 -T4: 4782 -initiator: Transport connection established, thus prompting 4783 the initiator to start the iSCSI Login. 4784 -target: Initial iSCSI Login request was received. 4785 -T5: 4786 -initiator: The final iSCSI Login response with a Status- 4787 Class of zero was received. 4788 -target: The final iSCSI Login request to conclude the Login 4789 Phase was received, thus prompting the target to send the 4790 final iSCSI Login response with a Status-Class of zero. 4791 -T6: 4792 -initiator: Illegal 4793 -target: Timed out waiting for an iSCSI Login, transport 4794 disconnect indication was received, transport reset was 4795 received, or an internal event indicating a transport 4796 timeout was received. In all these cases, the connection is 4797 to be closed. 4798 -T7: 4799 -initiator - one of the following events caused the 4800 transition: 4801 - The final iSCSI Login response was received with a 4802 non-zero Status-Class. 4803 - Login timed out. 4804 - A transport disconnect indication was received. 4805 - A transport reset was received. 4806 - An internal event indicating a transport timeout was 4807 received. 4808 - An internal event of receiving a Logout response 4809 (success) on another connection for a "close the 4810 session" Logout request was received. 4812 In all these cases, the transport connection is closed. 4814 -target - one of the following events caused the transition: 4815 - The final iSCSI Login request to conclude the Login 4816 Phase was received, prompting the target to send the 4817 final iSCSI Login response with a non-zero Status- 4818 Class. 4819 - Login timed out. 4820 - Transport disconnect indication was received. 4821 - Transport reset was received. 4823 - An internal event indicating a transport timeout was 4824 received . 4825 - On another connection a "close the session" Logout 4826 request was received. 4828 In all these cases, the connection is to be closed. 4829 -T8: 4830 -initiator: An internal event of receiving a Logout response 4831 (success) on another connection for a "close the session" 4832 Logout request was received, thus closing this connection 4833 requiring no further cleanup. 4834 -target: An internal event of sending a Logout response 4835 (success) on another connection for a "close the session" 4836 Logout request was received, or an internal event of a 4837 successful connection/session reinstatement is received, 4838 thus prompting the target to close this connection cleanly. 4839 -T9, T10: 4840 -initiator: An internal event that indicates the readiness 4841 to start the Logout process was received, thus prompting an 4842 iSCSI Logout to be sent by the initiator. 4843 -target: An iSCSI Logout request was received. 4844 -T11, T12: 4845 -initiator: Async PDU with AsyncEvent "Request Logout" was 4846 received. 4847 -target: An internal event that requires the decommissioning 4848 of the connection is received, thus causing an Async PDU 4849 with an AsyncEvent "Request Logout" to be sent. 4850 -T13: 4851 -initiator: An iSCSI Logout response (success) was received, 4852 or an internal event of receiving a Logout response 4853 (success) on another connection for a "close the session" 4854 Logout request was received. 4855 -target: An internal event was received that indicates 4856 successful processing of the Logout, which prompts an iSCSI 4857 Logout response (success) to be sent; an internal event of 4858 sending a Logout response (success) on another connection 4859 for a "close the session" Logout request was received; or an 4860 internal event of a successful connection/session 4861 reinstatement is received. In all these cases, the transport 4862 connection is closed. 4864 -T14: 4865 -initiator: Async PDU with AsyncEvent "Request Logout" was 4866 received again. 4867 -target: Illegal 4868 -T15, T16: 4869 -initiator: One or more of the following events caused this 4870 transition: 4871 -Internal event that indicates a transport connection 4872 timeout was received thus prompting transport RESET or 4873 transport connection closure. 4874 -A transport RESET. 4875 -A transport disconnect indication. 4876 -Async PDU with AsyncEvent "Drop connection" (for this 4877 CID). 4878 -Async PDU with AsyncEvent "Drop all connections". 4879 -target: One or more of the following events caused this 4880 transition: 4881 -Internal event that indicates a transport connection 4882 timeout was received, thus prompting transport RESET or 4883 transport connection closure. 4884 -An internal event of a failed connection/session 4885 reinstatement is received. 4886 -A transport RESET. 4887 -A transport disconnect indication. 4888 -Internal emergency cleanup event was received which 4889 prompts an Async PDU with AsyncEvent "Drop connection" 4890 (for this CID), or event "Drop all connections". 4892 -T17: 4893 -initiator: One or more of the following events caused this 4894 transition: 4895 -Logout response, (failure i.e., a non-zero status) was 4896 received, or Logout timed out. 4897 -Any of the events specified for T15 and T16. 4898 -target: One or more of the following events caused this 4899 transition: 4900 -Internal event that indicates a failure of the Logout 4901 processing was received, which prompts a Logout response 4902 (failure, i.e., a non-zero status) to be sent. 4904 -Any of the events specified for T15 and T16. 4905 -T18: 4906 -initiator: An internal event of receiving a Logout response 4907 (success) on another connection for a "close the session" 4908 Logout request was received. 4910 -target: An internal event of sending a Logout response 4911 (success) on another connection for a "close the session" 4912 Logout request was received, or an internal event of a 4913 successful connection/session reinstatement is received. In 4914 both these cases, the connection is closed. 4916 The CLEANUP_WAIT state (S8) implies that there are possible iSCSI 4917 tasks that have not reached conclusion and are still considered 4918 busy. 4920 7.1.3. Standard Connection State Diagram for an Initiator 4922 Symbolic names for States: 4924 S1: FREE 4926 S2: XPT_WAIT 4928 S4: IN_LOGIN 4930 S5: LOGGED_IN 4932 S6: IN_LOGOUT 4934 S7: LOGOUT_REQUESTED 4936 S8: CLEANUP_WAIT 4938 States S5, S6, and S7 constitute the Full Feature Phase operation 4939 of the connection. 4941 The state diagram is as follows: 4943 -------<-------------+ 4944 +--------->/ S1 \<----+ | 4945 T13| +->\ /<-+ \ | 4946 | / ---+--- \ \ | 4947 | / | T2 \ | | 4948 | T8 | |T1 | | | 4949 | | | / |T7 | 4950 | | | / | | 4951 | | | / | | 4952 | | V / / | 4953 | | ------- / / | 4954 | | / S2 \ / | 4955 | | \ / / | 4956 | | ---+--- / | 4957 | | |T4 / | 4958 | | V / | T18 4959 | | ------- / | 4960 | | / S4 \ | 4961 | | \ / | 4962 | | ---+--- | T15 4963 | | |T5 +--------+---------+ 4964 | | | /T16+-----+------+ | 4965 | | | / -+-----+--+ | | 4966 | | | / / S7 \ |T12| | 4967 | | | / +->\ /<-+ V V 4968 | | | / / -+----- ------- 4969 | | | / /T11 |T10 / S8 \ 4970 | | V / / V +----+ \ / 4971 | | ---+-+- ----+-- | ------- 4972 | | / S5 \T9 / S6 \<+ ^ 4973 | +-----\ /--->\ / T14 | 4974 | ------- --+----+------+T17 4975 +---------------------------+ 4977 The following state transition table represents the above diagram. 4978 Each row represents the starting state for a given transition, 4979 which after taking a transition marked in a table cell would end in 4980 the state represented by the column of the cell. For example, from 4981 state S1, the connection takes the T1 transition to arrive at state 4982 S2. The fields marked "-" correspond to undefined transitions. 4984 +----+---+---+---+---+----+---+ 4985 |S1 |S2 |S4 |S5 |S6 |S7 |S8 | 4986 ---+----+---+---+---+---+----+---+ 4987 S1| - |T1 | - | - | - | - | - | 4988 ---+----+---+---+---+---+----+---+ 4989 S2|T2 |- |T4 | - | - | - | - | 4990 ---+----+---+---+---+---+----+---+ 4991 S4|T7 |- |- |T5 | - | - | - | 4992 ---+----+---+---+---+---+----+---+ 4993 S5|T8 |- |- | - |T9 |T11 |T15| 4994 ---+----+---+---+---+---+----+---+ 4995 S6|T13 |- |- | - |T14|- |T17| 4996 ---+----+---+---+---+---+----+---+ 4997 S7|T18 |- |- | - |T10|T12 |T16| 4998 ---+----+---+---+---+---+----+---+ 4999 S8| - |- |- | - | - | - | - | 5000 ---+----+---+---+---+---+----+---+ 5002 7.1.4. Standard Connection State Diagram for a Target 5004 Symbolic names for States: 5005 S1: FREE 5007 S3: XPT_UP 5009 S4: IN_LOGIN 5011 S5: LOGGED_IN 5013 S6: IN_LOGOUT 5015 S7: LOGOUT_REQUESTED 5017 S8: CLEANUP_WAIT 5019 States S5, S6, and S7 constitute the Full Feature Phase operation 5020 of the connection. 5022 The state diagram is as follows: 5024 -------<-------------+ 5025 +--------->/ S1 \<----+ | 5026 T13| +->\ /<-+ \ | 5027 | / ---+--- \ \ | 5028 | / | T6 \ | | 5029 | T8 | |T3 | | | 5030 | | | / |T7 | 5031 | | | / | | 5032 | | | / | | 5033 | | V / / | 5034 | | ------- / / | 5035 | | / S3 \ / | 5036 | | \ / / | T18 5037 | | ---+--- / | 5038 | | |T4 / | 5039 | | V / | 5040 | | ------- / | 5041 | | / S4 \ | 5042 | | \ / | 5043 | | ---+--- T15 | 5044 | | |T5 +--------+---------+ 5045 | | | /T16+-----+------+ | 5046 | | | / -+-----+---+ | | 5047 | | | / / S7 \ |T12| | 5048 | | | / +->\ /<-+ V V 5049 | | | / / -+----- ------- 5050 | | | / /T11 |T10 / S8 \ 5051 | | V / / V \ / 5052 | | ---+-+- ------- ------- 5053 | | / S5 \T9 / S6 \ ^ 5054 | +-----\ /--->\ / | 5055 | ------- --+----+--------+T17 5056 +---------------------------+ 5058 The following state transition table represents the above diagram, 5059 and follows the conventions described for the initiator diagram. 5061 +----+---+---+---+---+----+---+ 5062 |S1 |S3 |S4 |S5 |S6 |S7 |S8 | 5063 ---+----+---+---+---+---+----+---+ 5064 S1| - |T3 | - | - | - | - | - | 5065 ---+----+---+---+---+---+----+---+ 5066 S3|T6 |- |T4 | - | - | - | - | 5067 ---+----+---+---+---+---+----+---+ 5068 S4|T7 |- |- |T5 | - | - | - | 5069 ---+----+---+---+---+---+----+---+ 5070 S5|T8 |- |- | - |T9 |T11 |T15| 5071 ---+----+---+---+---+---+----+---+ 5072 S6|T13 |- |- | - |- |- |T17| 5073 ---+----+---+---+---+---+----+---+ 5074 S7|T18 |- |- | - |T10|T12 |T16| 5075 ---+----+---+---+---+---+----+---+ 5076 S8| - |- |- | - | - | - | - | 5077 ---+----+---+---+---+---+----+---+ 5079 7.2. Connection Cleanup State Diagram for Initiators and Targets 5081 Symbolic names for states: 5083 R1: CLEANUP_WAIT (same as S8) 5085 R2: IN_CLEANUP 5087 R3: FREE (same as S1) 5089 Whenever a connection state machine in cleanup (lets call it CSM- 5090 C) enters the CLEANUP_WAIT state (S8), it must go through the state 5091 transitions described in the connection cleanup state diagram 5092 either a) using a separate full-feature phase connection (lets 5093 call it CSM-E, for explicit) in the LOGGED_IN state in the same 5094 session, or b) using a new transport connection (lets call it CSM- 5095 I, for implicit) in the FREE state that is to be added to the same 5096 session. In the CSM-E case, an explicit logout for the CID that 5097 corresponds to CSM-C (either as a connection or session logout) 5098 needs to be performed to complete the cleanup. In the CSM-I case, 5099 an implicit logout for the CID that corresponds to CSM-C needs to 5100 be performed by way of connection reinstatement (section 5.3.4) for 5101 that CID. In either case, the protocol exchanges on CSM-E or CSM-I 5102 determine the state transitions for CSM-C. Therefore, this cleanup 5103 state diagram is only applicable to the instance of the connection 5104 in cleanup (i.e., CSM-C). In the case of an implicit logout for 5105 example, CSM-C reaches FREE (R3) at the time CSM-I reaches 5106 LOGGED_IN. In the case of an explicit logout, CSM-C reaches FREE 5107 (R3) when CSM-E receives a successful logout response while 5108 continuing to be in the LOGGED_IN state. 5110 An initiator must initiate an explicit or implicit connection 5111 logout for a connection in the CLEANUP_WAIT state, if the initiator 5112 intends to continue using the associated iSCSI session. 5114 The following state diagram applies to both initiators and targets. 5116 ------- 5117 / R1 \ 5118 +--\ /<-+ 5119 / ---+--- \ 5120 / | \ M3 5121 M1 | |M2 | 5122 | | / 5123 | | / 5124 | | / 5125 | V / 5126 | ------- / 5127 | / R2 \ 5128 | \ / 5129 | ------- 5130 | | 5131 | |M4 5132 | | 5133 | | 5134 | | 5135 | V 5136 | ------- 5137 | / R3 \ 5138 +---->\ / 5139 ------- 5141 The following state transition table represents the above diagram, 5142 and follows the same conventions as in earlier sections. 5144 +----+----+----+ 5145 |R1 |R2 |R3 | 5146 -----+----+----+----+ 5147 R1 | - |M2 |M1 | 5148 -----+----+----+----+ 5149 R2 |M3 | - |M4 | 5150 -----+----+----+----+ 5151 R3 | - | - | - | 5152 -----+----+----+----+ 5154 7.2.1. State Descriptions for Initiators and Targets 5156 -R1: CLEANUP_WAIT (Same as S8) 5157 -initiator: Waiting for the internal event to initiate the 5158 cleanup processing for CSM-C. 5159 -target: Waiting for the cleanup process to start for CSM-C. 5160 -R2: IN_CLEANUP 5161 -initiator: Waiting for the connection cleanup process to 5162 conclude for CSM-C. 5163 -target: Waiting for the connection cleanup process to 5164 conclude for CSM-C. 5165 -R3: FREE (Same as S1) 5166 -initiator: End state for CSM-C. 5167 -target: End state for CSM-C. 5169 7.2.2. State Transition Descriptions for Initiators and Targets 5171 -M1: One or more of the following events was received: 5172 -initiator: 5173 -An internal event that indicates connection state 5174 timeout. 5175 -An internal event of receiving a successful Logout 5176 response on a different connection for a "close the 5177 session" Logout. 5178 -target: 5179 -An internal event that indicates connection state 5180 timeout. 5182 -An internal event of sending a Logout response 5183 (success) on a different connection for a "close the 5184 session" Logout request. 5186 -M2: An implicit/explicit logout process was initiated by the 5187 initiator. 5188 -In CSM-I usage: 5189 -initiator: An internal event requesting the connection 5190 (or session) reinstatement was received, thus prompting 5191 a connection (or session) reinstatement Login to be sent 5192 transitioning CSM-I to state IN_LOGIN. 5193 -target: A connection/session reinstatement Login was 5194 received while in state XPT_UP. 5195 -In CSM-E usage: 5196 -initiator: An internal event that indicates that an 5197 explicit logout was sent for this CID in state 5198 LOGGED_IN. 5199 -target: An explicit logout was received for this CID in 5200 state LOGGED_IN. 5201 -M3: Logout failure detected 5202 -In CSM-I usage: 5203 -initiator: CSM-I failed to reach LOGGED_IN and arrived 5204 into FREE instead. 5205 -target: CSM-I failed to reach LOGGED_IN and arrived 5206 into FREE instead. 5207 -In CSM-E usage: 5208 -initiator: CSM-E either moved out of LOGGED_IN, or 5209 Logout timed out and/or aborted, or Logout response 5210 (failure) was received. 5211 -target: CSM-E either moved out of LOGGED_IN, Logout 5212 timed out and/or aborted, or an internal event that 5213 indicates a failed Logout processing was received. A 5214 Logout response (failure) was sent in the last case. 5216 -M4: Successful implicit/explicit logout was performed. 5217 - In CSM-I usage: 5218 -initiator: CSM-I reached state LOGGED_IN, or an 5219 internal event of receiving a Logout response (success) 5220 on another connection for a "close the session" Logout 5221 request was received. 5222 -target: CSM-I reached state LOGGED_IN, or an internal 5223 event of sending a Logout response (success) on a 5224 different connection for a "close the session" Logout 5225 request was received. 5226 - In CSM-E usage: 5227 -initiator: CSM-E stayed in LOGGED_IN and received a 5228 Logout response (success), or an internal event of 5229 receiving a Logout response (success) on another 5230 connection for a "close the session" Logout request was 5231 received. 5232 -target: CSM-E stayed in LOGGED_IN and an internal event 5233 indicating a successful Logout processing was received, 5234 or an internal event of sending a Logout response 5235 (success) on a different connection for a "close the 5236 session" Logout request was received. 5238 7.3. Session State Diagrams 5240 7.3.1. Session State Diagram for an Initiator 5242 Symbolic Names for States: 5244 Q1: FREE 5246 Q3: LOGGED_IN 5248 Q4: FAILED 5250 State Q3 represents the Full Feature Phase operation of the 5251 session. 5253 The state diagram is as follows: 5255 ------- 5256 / Q1 \ 5257 +------>\ /<-+ 5258 / ---+--- | 5259 / | |N3 5260 N6 | |N1 | 5261 | | | 5262 | N4 | | 5263 | +--------+ | / 5264 | | | | / 5265 | | | | / 5266 | | V V / 5267 -+--+-- -----+- 5268 / Q4 \ N5 / Q3 \ 5269 \ /<---\ / 5270 ------- ------- 5272 The state transition table is as follows: 5274 +----+----+----+ 5275 |Q1 |Q3 |Q4 | 5276 -----+----+----+----+ 5277 Q1 | - |N1 | - | 5278 -----+----+----+----+ 5279 Q3 |N3 | - |N5 | 5280 -----+----+----+----+ 5281 Q4 |N6 |N4 | - | 5282 -----+----+----+----+ 5284 7.3.2. Session State Diagram for a Target 5286 Symbolic Names for States: 5288 Q1: FREE 5290 Q2: ACTIVE 5292 Q3: LOGGED_IN 5294 Q4: FAILED 5295 Q5: IN_CONTINUE 5297 State Q3 represents the Full Feature Phase operation of the 5298 session. 5300 The state diagram is as follows: 5302 ------- 5303 +------------------>/ Q1 \ 5304 / +-------------->\ /<-+ 5305 | | ---+--- | 5306 | | ^ | |N3 5307 N6 | |N11 N9| V N1 | 5308 | | +------ | 5309 | | / Q2 \ | 5310 | | \ / | 5311 | --+---- +--+--- | 5312 | / Q5 \ | | 5313 | \ / N10 | | 5314 | +-+---+------------+ |N2 / 5315 | ^ | | | / 5316 |N7| |N8 | | / 5317 | | | | V / 5318 -+--+-V V----+- 5319 / Q4 \ N5 / Q3 \ 5320 \ /<-------------\ / 5321 ------- ------- 5323 The state transition table is as follows: 5325 +----+----+----+----+----+ 5326 |Q1 |Q2 |Q3 |Q4 |Q5 | 5327 -----+----+----+----+----+----+ 5328 Q1 | - |N1 | - | - | - | 5329 -----+----+----+----+----+----+ 5330 Q2 |N9 | - |N2 | - | - | 5331 -----+----+----+----+----+----+ 5332 Q3 |N3 | - | - |N5 | - | 5333 -----+----+----+----+----+----+ 5334 Q4 |N6 | - | - | - |N7 | 5335 -----+----+----+----+----+----+ 5336 Q5 |N11 | - |N10 |N8 | - | 5337 -----+----+----+----+----+----+ 5339 7.3.3. State Descriptions for Initiators and Targets 5341 -Q1: FREE 5342 -initiator: State on instantiation or after cleanup. 5343 -target: State on instantiation or after cleanup. 5344 -Q2: ACTIVE 5345 -initiator: Illegal. 5346 -target: The first iSCSI connection in the session 5347 transitioned to IN_LOGIN, waiting for it to complete the 5348 login process. 5349 -Q3: LOGGED_IN 5350 -initiator: Waiting for all session events. 5351 -target: Waiting for all session events. 5352 -Q4: FAILED 5353 -initiator: Waiting for session recovery or session 5354 continuation. 5355 -target: Waiting for session recovery or session 5356 continuation. 5357 -Q5: IN_CONTINUE 5358 -initiator: Illegal. 5359 -target: Waiting for session continuation attempt to reach a 5360 conclusion. 5362 7.3.4. State Transition Descriptions for Initiators and Targets 5364 -N1: 5365 -initiator: At least one transport connection reached the 5366 LOGGED_IN state. 5367 -target: The first iSCSI connection in the session had 5368 reached the IN_LOGIN state. 5369 -N2: 5370 -initiator: Illegal. 5371 -target: At least one iSCSI connection reached the LOGGED_IN 5372 state. 5373 -N3: 5374 -initiator: Graceful closing of the session via session 5375 closure (Section 5.3.6 - "Session Continuation and 5376 Failure"). 5377 -target: Graceful closing of the session via session closure 5378 (Section 5.3.6 - "Session Continuation and Failure") or a 5379 successful session reinstatement cleanly closed the session. 5380 -N4: 5381 -initiator: A session continuation attempt succeeded. 5382 -target: Illegal. 5383 -N5: 5384 -initiator: Session failure (Section 5.3.6 - "Session 5385 Continuation and Failure") occurred. 5386 -target: Session failure (Section 5.3.6 - "Session 5387 Continuation and Failure") occurred. 5388 -N6: 5389 -initiator: Session state timeout occurred, or a session 5390 reinstatement cleared this session instance. This results 5391 in the freeing of all associated resources and the session 5392 state is discarded. 5393 -target: Session state timeout occurred, or a session 5394 reinstatement cleared this session instance. This results 5395 in the freeing of all associated resources and the session 5396 state is discarded. 5397 -N7: 5398 -initiator: Illegal. 5399 -target: A session continuation attempt is initiated. 5400 -N8: 5401 -initiator: Illegal. 5402 -target: The last session continuation attempt failed. 5404 -N9: 5405 -initiator: Illegal. 5406 -target: Login attempt on the leading connection failed. 5407 -N10: 5408 -initiator: Illegal. 5409 -target: A session continuation attempt succeeded. 5410 -N11: 5411 -initiator: Illegal. 5412 -target: A successful session reinstatement cleanly closed 5413 the session. 5415 8. Security Considerations 5417 Historically, native storage systems have not had to consider 5418 security because their environments offered minimal security risks. 5419 That is, these environments consisted of storage devices either 5420 directly attached to hosts or connected via a Storage Area Network 5421 (SAN) distinctly separate from the communications network. The use 5422 of storage protocols, such as SCSI, over IP-networks requires that 5423 security concerns be addressed. iSCSI implementations MUST provide 5424 means of protection against active attacks (e.g., pretending to be 5425 another identity, message insertion, deletion, modification, and 5426 replaying) and passive attacks (e.g.,eavesdropping, gaining 5427 advantage by analyzing the data sent over the line). 5429 Although technically possible, iSCSI SHOULD NOT be configured 5430 without security. iSCSI configured without security should be 5431 confined, in extreme cases, to closed environments without any 5432 security risk. [RFC3723] specifies the mechanisms that must be used 5433 in order to mitigate risks fully described in that document. 5435 The following section describes the security mechanisms provided by 5436 an iSCSI implementation. 5438 8.1. iSCSI Security Mechanisms 5440 The entities involved in iSCSI security are the initiator, target, 5441 and the IP communication end points. iSCSI scenarios in which 5442 multiple initiators or targets share a single communication end 5443 point are expected. To accommodate such scenarios, iSCSI uses two 5444 separate security mechanisms: In-band authentication between the 5445 initiator and the target at the iSCSI connection level (carried out 5446 by exchange of iSCSI Login PDUs), and packet protection (integrity, 5447 authentication, and confidentiality) by IPsec at the IP level. The 5448 two security mechanisms complement each other. The in-band 5449 authentication provides end-to-end trust (at login time) between 5450 the iSCSI initiator and the target while IPsec provides a secure 5451 channel between the IP communication end points. 5453 Further details on typical iSCSI scenarios and the relation between 5454 the initiators, targets, and the communication end points can be 5455 found in [RFC3723]. 5457 8.2. In-band Initiator-Target Authentication 5459 During login, the target MUST authenticate the initiator and the 5460 initiator MAY authenticate the target. The authentication is 5461 performed on every new iSCSI connection by an exchange of iSCSI 5462 Login PDUs using a negotiated authentication method. 5464 The authentication method cannot assume an underlying IPsec 5465 protection, because IPsec is optional to use. An attacker should 5466 gain as little advantage as possible by inspecting the 5467 authentication phase PDUs. Therefore, a method using clear text (or 5468 equivalent) passwords is not acceptable; on the other hand, 5469 identity protection is not strictly required. 5471 The authentication mechanism protects against an unauthorized login 5472 to storage resources by using a false identity (spoofing). Once the 5473 authentication phase is completed, if the underlying IPsec is not 5474 used, all PDUs are sent and received in clear. The authentication 5475 mechanism alone (without underlying IPsec) should only be used when 5476 there is no risk of eavesdropping, message insertion, deletion, 5477 modification, and replaying. 5479 Section 11 - "iSCSI Security Text Keys and Authentication Methods" 5480 defines several authentication methods and the exact steps that 5481 must be followed in each of them, including the iSCSI-text-keys and 5482 their allowed values in each step. Whenever an iSCSI initiator gets 5483 a response whose keys, or their values, are not according to the 5484 step definition, it MUST abort the connection. Whenever an iSCSI 5485 target gets a response whose keys, or their values, are not 5486 according to the step definition, it MUST answer with a Login 5487 reject with the "Initiator Error" or "Missing Parameter" status. 5488 These statuses are not intended for cryptographically incorrect 5489 values such as the CHAP response, for which "Authentication 5490 Failure" status MUST be specified. The importance of this rule can 5491 be illustrated in CHAP with target authentication (see Section 5492 11.1.4 - "Challenge Handshake Authentication Protocol (CHAP)") 5493 where the initiator would have been able to conduct a reflection 5494 attack by omitting his response key (CHAP_R) using the same CHAP 5495 challenge as the target and reflecting the target's response back 5496 to the target. In CHAP, this is prevented because the target must 5497 answer the missing CHAP_R key with a Login reject with the "Missing 5498 Parameter" status. 5500 For some of the authentication methods, a key specifies the 5501 identity of the iSCSI initiator or target for authentication 5502 purposes. The value associated with that key MAY be different from 5503 the iSCSI name and SHOULD be configurable. (CHAP_N, see Section 5504 11.1.4 - "Challenge Handshake Authentication Protocol (CHAP)" and 5505 SRP_U, see Section 11.1.3 - "Secure Remote Password (SRP)"). 5507 8.2.1. CHAP Considerations 5509 Compliant iSCSI initiators and targets MUST implement the CHAP 5510 authentication method [RFC1994] (according to Section 11.1.4 - 5511 "Challenge Handshake Authentication Protocol (CHAP)" including the 5512 target authentication option). 5514 When CHAP is performed over a non-encrypted channel, it is 5515 vulnerable to an off-line dictionary attack. Implementations MUST 5516 support use of up to 128 bit random CHAP secrets, including the 5517 means to generate such secrets and to accept them from an external 5518 generation source. Implementations MUST NOT provide secret 5519 generation (or expansion) means other than random generation. 5521 An administrative entity of an environment in which CHAP is used 5522 with a secret that has less than 96 random bits MUST enforce IPsec 5523 encryption (according to the implementation requirements in 5524 Confidentiality) to protect the connection. Moreover, in this case 5525 IKE authentication with group pre-shared cryptographic keys SHOULD 5526 NOT be used unless it is not essential to protect group members 5527 against off-line dictionary attacks by other members. 5529 CHAP secrets MUST be an integral number of bytes (octets). A 5530 compliant implementation SHOULD NOT continue with the login step in 5531 which it should send a CHAP response (CHAP_R, Section 11.1.4 - 5532 "Challenge Handshake Authentication Protocol (CHAP)") unless it can 5533 verify that the CHAP secret is at least 96 bits, or that IPsec 5534 encryption is being used to protect the connection. 5536 Any CHAP secret used for initiator authentication MUST NOT be 5537 configured for authentication of any target, and any CHAP secret 5538 used for target authentication MUST NOT be configured for 5539 authentication of any initiator. If the CHAP response received by 5540 one end of an iSCSI connection is the same as the CHAP response 5541 that the receiving endpoint would have generated for the same CHAP 5542 challenge, the response MUST be treated as an authentication 5543 failure and cause the connection to close (this ensures that the 5544 same CHAP secret is not used for authentication in both 5545 directions). Also, if 5546 an iSCSI implementation can function as both initiator and target, 5547 different CHAP secrets and identities MUST be configured for these 5548 two roles. The following is an example of the attacks prevented by 5549 the above requirements: 5551 Rogue wants to impersonate Storage to Alice, and knows that a 5552 single secret is used for both directions of Storage-Alice 5553 authentication. 5555 Rogue convinces Alice to open two connections to Rogue, and 5556 Rogue identifies itself as Storage on both connections. 5558 Rogue issues a CHAP challenge on connection 1, waits for Alice 5559 to respond, and then reflects Alice's challenge as the 5560 initial challenge to Alice on connection 2. 5562 If Alice doesn't check for the reflection across connections, 5563 Alice's response on connection 2 enables Rogue to impersonate 5564 Storage on connection 1, even though Rogue does not know the 5565 Alice-Storage CHAP secret. 5567 Originators MUST NOT reuse the CHAP challenge sent by the Responder 5568 for the other direction of a bidirectional authentication. 5569 Responders MUST check for this condition and close the iSCSI TCP 5570 connection if it occurs. 5572 The same CHAP secret SHOULD NOT be configured for authentication of 5573 multiple initiators or multiple targets, as this enables any of 5574 them to impersonate any other one of them, and compromising one of 5575 them enables the attacker to impersonate any of them. It is 5576 recommended that iSCSI implementations check for use of identical 5577 CHAP secrets by different peers when this check is feasible, and 5578 take appropriate measures to warn users and/or administrators when 5579 this is detected. 5581 When an iSCSI initiator or target authenticates itself to 5582 counterparts in multiple administrative domains, it SHOULD use a 5583 different CHAP secret for each administrative domain to avoid 5584 propagating security compromises across domains. 5586 Within a single administrative domain: 5587 - A single CHAP secret MAY be used for authentication of an 5588 initiator to multiple targets. 5590 - A single CHAP secret MAY be used for an authentication of a 5591 target to multiple initiators when the initiators use an 5592 external server (e.g., RADIUS) to verify the target's CHAP 5593 responses and do not know the target's CHAP secret. 5595 If an external response verification server (e.g., RADIUS) is not 5596 used, employing a single CHAP secret for authentication of a target 5597 to multiple initiators requires that all such initiators know that 5598 target secret. Any of these initiators can impersonate the target 5599 to any other such initiator, and compromise of such an initiator 5600 enables an attacker to impersonate the target to all such 5601 initiators. Targets SHOULD use separate CHAP secrets for 5602 authentication to each initiator when such risks are of concern; in 5603 this situation it may be useful to configure a separate logical 5604 iSCSI target with its own iSCSI Node Name for each initiator or 5605 group of initiators among which such separation is desired. 5607 8.2.2. SRP Considerations 5609 The strength of the SRP authentication method (specified in 5610 [RFC2945]) is dependent on the characteristics of the group being 5611 used (i.e., the prime modulus N and generator g). As described in 5612 [RFC2945], N is required to be a Sophie-German prime (of the form N 5613 = 2q + 1, where q is also prime) and the generator g is a primitive 5614 root of GF(n). In iSCSI authentication, the prime modulus N MUST be 5615 at least 768 bits. 5617 The list of allowed SRP groups is provided in [RFC3723]. 5619 8.3. IPsec 5621 iSCSI uses the IPsec mechanism for packet protection (cryptographic 5622 integrity, authentication, and confidentiality) at the IP level 5623 between the iSCSI communicating end points. The following sections 5624 describe the IPsec protocols that must be implemented for data 5625 integrity and authentication, confidentiality, and cryptographic 5626 key management. 5628 An iSCSI initiator or target may provide the required IPsec support 5629 fully integrated or in conjunction with an IPsec front-end device. 5630 In the latter case, the compliance requirements with regard to 5631 IPsec support apply to the "combined device". Only the "combined 5632 device" is to be considered an iSCSI device. 5634 Detailed considerations and recommendations for using IPsec for 5635 iSCSI are provided in [RFC3723]. 5637 8.3.1. Data Integrity and Authentication 5639 Data authentication and integrity is provided by a cryptographic 5640 keyed Message Authentication Code in every sent packet. This code 5641 protects against message insertion, deletion, and modification. 5642 Protection against message replay is realized by using a sequence 5643 counter. 5645 An iSCSI compliant initiator or target MUST provide data integrity 5646 and authentication by implementing IPsec [RFC4301] with ESP 5647 [RFC4303] in tunnel mode and MAY provide data integrity and 5648 authentication by implementing IPsec with ESP in transport mode. 5649 The IPsec implementation MUST fulfill the following iSCSI specific 5650 requirements: 5652 - HMAC-SHA1 MUST be implemented [RFC2404]. 5654 - AES CBC MAC with XCBC extensions SHOULD be implemented 5655 [RFC3566]. 5657 The ESP anti-replay service MUST also be implemented. 5659 At the high speeds iSCSI is expected to operate, a single IPsec SA 5660 could rapidly cycle through the 32-bit IPsec sequence number space. 5661 In view of this, an iSCSI implementation that operates at speeds of 5662 1 Gbps or greater MUST implement the IPsec sequence number 5663 extension [RFC4303] and SHOULD use it on iSCSI connections. 5665 8.3.2. Confidentiality 5667 Confidentiality is provided by encrypting the data in every packet. 5668 When confidentiality is used it MUST be accompanied by data 5669 integrity and authentication to provide comprehensive protection 5670 against eavesdropping, message insertion, deletion, modification, 5671 and replaying. 5673 An iSCSI compliant initiator or target MUST provide confidentiality 5674 by implementing IPsec [RFC4301] with ESP [RFC4303] in tunnel mode 5675 and MAY provide confidentiality by implementing IPsec with ESP in 5676 transport mode, with the following iSCSI specific requirements: 5678 - 3DES in CBC mode MUST be implemented [RFC2451]. 5680 - AES in Counter mode SHOULD be implemented [RFC3686]. 5682 DES in CBC mode SHOULD NOT be used due to its inherent weakness. 5683 The NULL encryption algorithm MUST also be implemented. 5685 8.3.3. Policy, Security Associations, and Cryptographic Key Management 5687 A compliant iSCSI implementation MUST meet the cryptographic key 5688 management requirements of the IPsec protocol suite. 5689 Authentication, security association negotiation, and cryptographic 5690 key management MUST be provided by implementing IKE [RFC4306] using 5691 the IPsec DOI [RFC4306] with the following iSCSI specific 5692 requirements: 5694 - Peer authentication using a pre-shared cryptographic key MUST 5695 be supported. Certificate-based peer authentication using 5696 digital signatures MAY be supported. Peer authentication 5697 using the public key encryption methods outlined in IKE 5698 sections 5.2 and 5.3[7] SHOULD NOT be used. 5700 - When digital signatures are used to achieve authentication, 5701 an IKE negotiator SHOULD use IKE Certificate Request 5702 Payload(s) to specify the certificate authority. IKE 5703 negotiators SHOULD check the pertinent Certificate Revocation 5704 List (CRL) before accepting a PKI certificate for use in IKE 5705 authentication procedures. 5707 - Conformant iSCSI implementations MUST support IKE Main Mode 5708 and SHOULD support Aggressive Mode. IKE main mode with pre- 5709 shared key authentication method SHOULD NOT be used when 5710 either the initiator or the target uses dynamically assigned 5711 IP addresses. While in many cases pre-shared keys offer good 5712 security, situations in which dynamically assigned addresses 5713 are used force the use of a group pre-shared key, which 5714 creates vulnerability to a man-in-the-middle attack. 5716 - In the IKE Phase 2 Quick Mode, exchanges for creating the 5717 Phase 2 SA, the Identity Payload, fields MUST be present. 5718 ID_IPV4_ADDR, ID_IPV6_ADDR (if the protocol stack supports 5719 IPv6) and ID_FQDN Identity payloads MUST be supported; 5720 ID_USER_FQDN SHOULD be supported. The IP Subnet, IP Address 5721 Range, ID_DER_ASN1_DN, and ID_DER_ASN1_GN formats SHOULD NOT 5722 be used. The ID_KEY_ID Identity Payload MUST NOT be used. 5724 Manual cryptographic keying MUST NOT be used because it does not 5725 provide the necessary re-keying support. 5727 When IPsec is used, the receipt of an IKE Phase 2 delete message 5728 SHOULD NOT be interpreted as a reason for tearing down the iSCSI 5729 TCP connection. If additional traffic is sent on it, a new IKE 5730 Phase 2 SA will be created to protect it. 5732 The method used by the initiator to determine whether the target 5733 should be connected using IPsec is regarded as an issue of IPsec 5734 policy administration, and thus not defined in the iSCSI standard. 5736 If an iSCSI target is discovered via a SendTargets request in a 5737 discovery session not using IPsec, the initiator should assume that 5738 it does not need IPsec to establish a session to that target. If an 5739 iSCSI target is discovered using a discovery session that does use 5740 IPsec, the initiator SHOULD use IPsec when establishing a session 5741 to that target. 5743 8.4. Security Considerations for the X#NodeArchitecture Key 5745 The security considerations in this section are specific to the 5746 X#NodeArchitecture discussed in Section 12.24 - 5747 "X#NodeArchitecture". 5749 This extension key transmits specific implementation details about 5750 the node that sends it; such details may be considered sensitive in 5751 some environments. For example, if a certain software or firmware 5752 version is known to contain security weaknesses, announcing the 5753 presence of that version via this key may not be desirable. The 5754 countermeasures for this security concern are: 5756 - sending less detailed information in the key values, 5757 - not sending the extension key, or 5758 - using IPsec ([RFC4303]) to provide confidentiality for 5759 the iSCSI connection on which the key is sent 5760 To support the first and second countermeasures, all 5761 implementations of this extension key MUST provide an 5762 administrative mechanism to disable sending the key. In addition, 5763 all implementations SHOULD provide an administrative mechanism to 5764 configure a verbosity level of the key value, thereby controlling 5765 the amount of information sent. 5767 For example, a lower verbosity might enable transmission of node 5768 architecture component names only, but no version numbers. The 5769 choice of which countermeasure is most appropriate depends on the 5770 environment. However, sending less detailed information in the key 5771 values may be an acceptable countermeasure in many environments, 5772 since it provides a compromise between sending too much information 5773 and the other more complete countermeasures of not sending the key 5774 at all or using IPsec. 5776 In addition to security considerations involving transmission of 5777 the key contents, any logging method(s) used for the key values 5778 MUST keep the information secure from intruders. For all 5779 implementations, the requirements to address this security concern 5780 are: 5782 - Display of the log MUST only be possible with administrative 5783 rights to the node. 5785 - Options to disable logging to disk and to keep logs for a 5786 fixed duration SHOULD be provided. 5788 Finally, it is important to note that different nodes may have 5789 different levels of risk, and these differences may affect the 5790 implementation. The components of risk include assets, threats, and 5791 vulnerabilities. Consider the following example iSCSI nodes, which 5792 demonstrate differences in assets and vulnerabilities of the nodes, 5793 and as a result, differences in implementation: 5795 One iSCSI target based on a special-purpose operating system: 5796 Since the iSCSI target controls access to the data storage 5797 containing company assets, the asset level is seen as very 5798 high. Also, because of the special-purpose operating 5799 system, in which vulnerabilities are less well-known, the 5800 vulnerability level is viewed as low. 5802 Multiple iSCSI initiators in a blade farm, each running a 5803 general purpose operating system: The asset level of each 5804 node is viewed as low, since blades are replaceable and low 5805 cost. However, the vulnerability level is viewed as high, 5806 since there may be many wellknown vulnerabilities to that 5807 general-purpose operating system. For this target, an 5808 appropriate implementation might be logging of received key 5809 values, but no transmission of the key. For this initiator, 5810 an appropriate implementation might be transmission of the 5811 key, but no logging of received key values. 5813 9. Notes to Implementers 5815 This section notes some of the performance and reliability 5816 considerations of the iSCSI protocol. This protocol was designed to 5817 allow efficient silicon and software implementations. The iSCSI 5818 task tag mechanism was designed to enable Direct Data Placement 5819 (DDP - a DMA form) at the iSCSI level or lower. 5821 The guiding assumption made throughout the design of this protocol 5822 is that targets are resource constrained relative to initiators. 5824 Implementers are also advised to consider the implementation 5825 consequences of the iSCSI to SCSI mapping model as outlined in 5826 Section 3.4.3 - "Consequences of the Model". 5828 9.1. Multiple Network Adapters 5830 The iSCSI protocol allows multiple connections, not all of which 5831 need to go over the same network adapter. If multiple network 5832 connections are to be utilized with hardware support, the iSCSI 5833 protocol command-data-status allegiance to one TCP connection 5834 ensures that there is no need to replicate information across 5835 network adapters or otherwise require them to cooperate. 5837 However, some task management commands may require some loose form 5838 of cooperation or replication at least on the target. 5840 9.1.1. Conservative Reuse of ISIDs 5842 Historically, the SCSI model (and implementations and applications 5843 based on that model) has assumed that SCSI ports are static, 5844 physical entities. Recent extensions to the SCSI model have taken 5845 advantage of persistent worldwide unique names for these ports. In 5846 iSCSI however, the SCSI initiator ports are the endpoints of 5847 dynamically created sessions, so the presumptions of "static and 5848 physical" do not apply. In any case, the model clauses 5849 (particularly, Section 3.4.2 - "SCSI Architecture Model") provide 5850 for persistent, reusable names for the iSCSI-type SCSI initiator 5851 ports even though there does not need to be any physical entity 5852 bound to these names. 5854 To both minimize the disruption of legacy applications and to 5855 better facilitate the SCSI features that rely on persistent names 5856 for SCSI ports, iSCSI implementations SHOULD attempt to provide a 5857 stable presentation of SCSI Initiator Ports (both to the upper OS- 5858 layers and to the targets to which they connect). This can be 5859 achieved in an initiator implementation by conservatively reusing 5860 ISIDs. In other words, the same ISID should be used in the Login 5861 process to multiple target portal groups (of the same iSCSI Target 5862 or different iSCSI Targets). The ISID RULE (Section 3.4.3 - 5863 "Consequences of the Model") only prohibits reuse to the same 5864 target portal group. It does not "preclude" reuse to other target 5865 portal groups. 5866 The principle of conservative reuse "encourages" reuse to other 5867 target portal groups. When a SCSI target device sees the same 5868 (InitiatorName, ISID) pair in different sessions to different 5869 target portal groups, it can identify the underlying SCSI Initiator 5870 Port on each session as the same SCSI port. In effect, it can 5871 recognize multiple paths from the same source. 5873 9.1.2. iSCSI Name, ISID, and TPGT Use 5875 The designers of the iSCSI protocol are aware that legacy SCSI 5876 transports rely on initiator identity to assign access to storage 5877 resources. Although newer techniques are available and simplify 5878 access control, support for configuration and authentication 5879 schemes that are based on initiator identity is deemed important in 5880 order to support legacy systems and administration software. iSCSI 5881 thus supports the notion that it should be possible to assign 5882 access to storage resources based on "initiator device" identity. 5884 When there are multiple hardware or software components coordinated 5885 as a single iSCSI Node, there must be some (logical) entity that 5886 represents the iSCSI Node that makes the iSCSI Node Name available 5887 to all components involved in session creation and login. 5888 Similarly, this entity that represents the iSCSI Node must be able 5889 to coordinate session identifier resources (ISID for initiators) to 5890 enforce both the ISID and TSIH RULES (see Section 3.4.3 - 5891 "Consequences of the Model"). 5893 For targets, because of the closed environment, implementation of 5894 this entity should be straightforward. However, vendors of iSCSI 5895 hardware (e.g., NICs or HBAs) intended for targets, SHOULD provide 5896 mechanisms for configuration of the iSCSI Node Name across the 5897 portal groups instantiated by multiple instances of these 5898 components within a target. 5900 However, complex targets making use of multiple Target Portal Group 5901 Tags may reconfigure them to achieve various quality goals. The 5902 initiators have two mechanisms at their disposal to discover and/or 5903 check reconfiguring targets - the discovery session type and a key 5904 returned by the target during login to confirm the TPGT. An 5905 initiator should attempt to "rediscover" the target configuration 5906 anytime a session is terminated unexpectedly. 5908 For initiators, in the long term, it is expected that operating 5909 system vendors will take on the role of this entity and provide 5910 standard APIs that can inform components of their iSCSI Node Name 5911 and can configure and/or coordinate ISID allocation, use, and 5912 reuse. 5914 Recognizing that such initiator APIs are not available today, other 5915 implementations of the role of this entity are possible. For 5916 example, a human may instantiate the (common) Node name as part of 5917 the installation process of each iSCSI component involved in 5918 session creation and login. This may be done either by pointing the 5919 component to a vendor-specific location for this datum or to a 5920 system-wide location. The structure of the ISID namespace (see 5921 Section 10.12.5 - "ISID" and [RFC3721]) facilitates implementation 5922 of the ISID coordination by allowing each component vendor to 5923 independently (of other vendor's components) coordinate allocation, 5924 use, and reuse of its own partition of the ISID namespace in a 5925 vendor-specific manner. Partitioning of the ISID namespace within 5926 initiator portal groups managed by that vendor allows each such 5927 initiator portal group to act independently of all other portal 5928 groups when selecting an ISID for a login; this facilitates 5929 enforcement of the ISID RULE (see Section 3.4.3 - "Consequences of 5930 the Model") at the initiator. 5932 A vendor of iSCSI hardware (e.g., NICs or HBAs) intended for use in 5933 initiators MUST implement a mechanism for configuring the iSCSI 5934 Node Name. Vendors, and administrators must ensure that iSCSI Node 5935 Names are unique worldwide. It is therefore important that when one 5936 chooses to reuse the iSCSI Node Name of a disabled unit, not to re- 5937 assign that name to the original unit unless its worldwide 5938 uniqueness can be ascertained again. 5940 In addition, a vendor of iSCSI hardware must implement a mechanism 5941 to configure and/or coordinate ISIDs for all sessions managed by 5942 multiple instances of that hardware within a given iSCSI Node. Such 5943 configuration might be either permanently pre-assigned at the 5944 factory (in a necessarily globally unique way), statically assigned 5945 (e.g., partitioned across all the NICs at initialization in a 5946 locally unique way), or dynamically assigned (e.g., on-line 5947 allocator, also in a locally unique way). In the latter two cases, 5948 the configuration may be via public APIs (perhaps driven by an 5949 independent vendor's software, such as the OS vendor) or via 5950 private APIs driven by the vendor's own software. 5952 The process of name assignment and coordination has to be as 5953 encompassing and automated as possible as years of legacy usage 5954 have shown it to be highly error-prone. It is to be mentioned that 5955 SCSI has today alternative schemes of access control that can be 5956 used by all transports and their security is not dependent on 5957 strict naming coordination. 5959 9.2. Autosense and Auto Contingent Allegiance (ACA) 5961 Autosense refers to the automatic return of sense data to the 5962 initiator in case a command did not complete successfully. iSCSI 5963 initiators and targets MUST support and use autosense. 5965 ACA helps preserve ordered command execution in the presence of 5966 errors. As iSCSI can have many commands in-flight between initiator 5967 and target, iSCSI initiators and targets SHOULD support ACA. 5969 9.3. iSCSI Timeouts 5971 iSCSI recovery actions are often dependent on iSCSI time-outs being 5972 recognized and acted upon before SCSI time-outs. Determining the 5973 right time-outs to use for various iSCSI actions (command 5974 acknowledgements expected, status acknowledgements, etc.) is very 5975 much dependent on infrastructure (hardware, links, TCP/IP stack, 5976 iSCSI driver). As a guide, the implementer may use an average Nop- 5977 Out/Nop-In turnaround delay multiplied by a "safety factor" (e.g., 5978 4) as a good estimate for the basic delay of the iSCSI stack for a 5980 given connection. The safety factor should account for the network 5981 load variability. For connection teardown the implementer may want 5982 to consider also TCP common practice for the given infrastructure. 5984 Text negotiations MAY also be subject to either time-limits or 5985 limits in the number of exchanges. Those SHOULD be generous enough 5986 to avoid affecting interoperability (e.g., allowing each key to be 5987 negotiated on a separate exchange). 5989 The relation between iSCSI timeouts and SCSI timeouts should also 5990 be considered. SCSI timeouts should be longer than iSCSI timeouts 5991 plus the time required for iSCSI recovery whenever iSCSI recovery 5992 is planned. Alternatively, an implementer may choose to interlock 5993 iSCSI timeouts and recovery with SCSI timeouts so that SCSI 5994 recovery will become active only where iSCSI is not planned to, or 5995 failed to, recover. 5997 The implementer may also want to consider the interaction between 5998 various iSCSI exception events - such as a digest failure - and 5999 subsequent timeouts. When iSCSI error recovery is active, a digest 6000 failure is likely to result in discovering a missing command or 6001 data PDU. In these cases, an implementer may want to lower the 6002 timeout values to enable faster initiation for recovery procedures. 6004 9.4. Command Retry and Cleaning Old Command Instances 6006 To avoid having old, retried command instances appear in a valid 6007 command window after a command sequence number wrap around, the 6008 protocol requires (see Section 3.2.2.1 - "Command Numbering and 6009 Acknowledging") that on every connection on which a retry has been 6010 issued, a non-immediate command be issued and acknowledged within a 6011 2**31-1 commands interval from the CmdSN of the retried command. 6012 This requirement can be fulfilled by an implementation in several 6013 ways. 6015 The simplest technique to use is to send a (non-retry) non- 6016 immediate SCSI command (or a NOP if no SCSI command is available 6017 for a while) after every command retry on the connection on which 6018 the retry was attempted. As errors are deemed rare events, this 6019 technique is probably the most effective, as it does not involve 6020 additional checks at the initiator when issuing commands. 6022 9.5. Synch and Steering Layer and Performance 6024 While a synch and steering layer is optional, an initiator/target 6025 that does not have it working against a target/initiator that 6026 demands synch and steering may experience performance degradation 6027 caused by packet reordering and loss. Providing a synch and 6028 steering mechanism is recommended for all high-speed 6029 implementations. 6031 9.6. Considerations for State-dependent Devices and Long-lasting SCSI 6032 Operations 6034 Sequential access devices operate on the principle that the 6035 position of the device is based on the last command processed. As 6036 such, command processing order and knowledge of whether or not the 6037 previous command was processed is of the utmost importance to 6038 maintain data integrity. For example, inadvertent retries of SCSI 6039 commands when it is not known if the previous SCSI command was 6040 processed is a potential data integrity risk. 6042 For a sequential access device, consider the scenario in which a 6043 SCSI SPACE command to backspace one filemark is issued and then re- 6044 issued due to no status received for the command. If the first 6045 SPACE command was actually processed, the re-issued SPACE command, 6046 if processed, will cause the position to change. Thus, a subsequent 6047 write operation will write data to the wrong position and any 6048 previous data at that position will be overwritten. 6050 For a medium changer device, consider the scenario in which an 6051 EXCHANGE MEDIUM command (the SOURCE ADDRESS and DESTINATION ADDRESS 6052 are the same thus performing a swap) is issued and then re-issued 6053 due to no status received for the command. If the first EXCHANGE 6054 MEDIUM command was actually processed, the re-issued EXCHANGE 6055 MEDIUM command, if processed, will perform the swap again. The net 6056 effect is no swap was performed thus leaving a data integrity 6057 exposure. 6059 All commands that change the state of the device (as in SPACE 6060 commands for sequential access devices, and EXCHANGE MEDIUM for 6061 medium changer device), MUST be issued as non-immediate commands 6062 for deterministic and in order delivery to iSCSI targets. 6064 For many of those state changing commands, the execution model also 6065 assumes that the command is executed exactly once. Devices 6066 implementing READ POSITION and LOCATE provide a means for SCSI 6067 level command recovery and new tape-class devices 6068 should support those commands. In their absence a retry at SCSI 6069 level is difficult and error recovery at iSCSI level is advisable. 6071 Devices operating on long latency delivery subsystems and 6072 performing long lasting SCSI operations may need mechanisms that 6073 enable connection replacement while commands are running (e.g., 6074 during an extended copy operation). 6076 9.6.1. Determining the Proper ErrorRecoveryLevel 6078 The implementation and use of a specific ErrorRecoveryLevel should 6079 be determined based on the deployment scenarios of a given iSCSI 6080 implementation. Generally, the following factors must be 6081 considered before deciding on the proper level of recovery: 6083 Application resilience to I/O failures. 6084 Required level of availability in the face of transport 6085 connection failures. 6086 Probability of transport layer "checksum escape". This in turn 6087 decides the iSCSI digest failure frequency, and thus the 6088 criticality of iSCSI-level error recovery. The details of 6089 estimating this probability are outside the scope of this 6090 document. 6092 A consideration of the above factors for SCSI tape devices as an 6093 example suggests that implementations SHOULD use 6094 ErrorRecoveryLevel=1 when transport connection failure is not a 6095 concern and SCSI level recovery is unavailable, and 6096 ErrorRecoveryLevel=2 when the connection failure is also of high 6097 likelihood during a backup/retrieval. 6099 For extended copy operations, implementations SHOULD use 6100 ErrorRecoveryLevel=2 whenever there is a relatively high likelihood 6101 of connection failure. 6103 9.7. Multi-task Abort Implementation Considerations 6105 Multi-task abort operations are typically issued in emergencies 6106 such as clearing a device lock-up, HA failover/failback etc. In 6107 these circumstances, it is desirable to rapidly go through the 6108 error handling process as opposed to the target waiting on multiple 6109 third-party initiators who may not even be functional anymore 6110 especially if this emergency is triggered because of one such 6111 initiator failure. Therefore, both iSCSI target and initiator 6112 implementations SHOULD support FastAbort multi-task abort semantics 6113 (Section 3.2.3.4). 6115 Note that both in standard semantics (Section 3.2.3.3) and 6116 FastAbort semantics (Section 3.2.3.4), there may be outstanding 6117 data transfers even after the TMF completion is reported on the 6118 issuing session. In the case of iSCSI/iSER [iSER], these would be 6119 tagged data transfers for STags not owned by any active tasks. 6120 Whether or not real buffers support these data transfers is 6121 implementation-dependent. However, the data transfers logically 6122 MUST be silently discarded by the target iSCSI layer in all cases. 6123 A target MAY, on an implementation-defined internal timeout, also 6124 choose to drop the connections on which it did not receive the 6125 expected Data-Out sequences (Section 3.2.3.3) or NOP-Out 6126 acknowledgements (Section 3.2.3.4) so as to reclaim the associated 6127 buffer, STag, and TTT resources as appropriate. 6129 10. iSCSI PDU Formats 6131 All multi-byte integers that are specified in formats defined in 6132 this document are to be represented in network byte order (i.e., 6133 big endian). Any field that appears in this document assumes that 6134 the most significant byte is the lowest numbered byte and the most 6135 significant bit (within byte or field) is the lowest numbered bit 6136 unless specified otherwise. 6138 Any compliant sender MUST set all bits not defined and all reserved 6139 fields to zero unless specified otherwise. Any compliant receiver 6140 MUST ignore any bit not defined and all reserved fields unless 6141 specified otherwise. Receipt of reserved code values in defined 6142 fields MUST be reported as a protocol error. 6144 Reserved fields are marked by the word "reserved", some 6145 abbreviation of "reserved", or by "." for individual bits when no 6146 other form of marking is technically feasible. 6148 10.1. iSCSI PDU Length and Padding 6150 iSCSI PDUs are padded to the closest integer number of four byte 6151 words. The padding bytes SHOULD be sent as 0. 6153 10.2. PDU Template, Header, and Opcodes 6155 All iSCSI PDUs have one or more header segments and, optionally, a 6156 data segment. After the entire header segment group a header-digest 6157 MAY follow. The data segment MAY also be followed by a data-digest. 6159 The Basic Header Segment (BHS) is the first segment in all of the 6160 iSCSI PDUs. The BHS is a fixed-length 48-byte header segment. It 6161 MAY be followed by Additional Header Segments (AHS), a Header- 6162 Digest, a Data Segment, and/or a Data-Digest. 6164 The overall structure of an iSCSI PDU is as follows: 6166 Byte/ 0 | 1 | 2 | 3 | 6167 / | | | | 6168 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 6169 +---------------+---------------+---------------+---------------+ 6170 0/ Basic Header Segment (BHS) / 6171 +/ / 6172 +---------------+---------------+---------------+---------------+ 6173 48/ Additional Header Segment 1 (AHS) (optional) / 6174 +/ / 6175 +---------------+---------------+---------------+---------------+ 6176 / Additional Header Segment 2 (AHS) (optional) / 6177 +/ / 6178 +---------------+---------------+---------------+---------------+ 6179 ---- 6180 +---------------+---------------+---------------+---------------+ 6181 / Additional Header Segment n (AHS) (optional) / 6182 +/ / 6183 +---------------+---------------+---------------+---------------+ 6184 ---- 6185 +---------------+---------------+---------------+---------------+ 6186 k/ Header-Digest (optional) / 6187 +/ / 6188 +---------------+---------------+---------------+---------------+ 6189 l/ Data Segment(optional) / 6190 +/ / 6191 +---------------+---------------+---------------+---------------+ 6192 m/ Data-Digest (optional) / 6193 +/ / 6194 +---------------+---------------+---------------+---------------+ 6196 All PDU segments and digests are padded to the closest integer 6197 number of four byte words. For example, all PDU segments and 6198 digests start at a four byte word boundary and the padding ranges 6199 from 0 to 3 bytes. The padding bytes SHOULD be sent as 0. 6201 iSCSI response PDUs do not have AH Segments. 6203 10.2.1. Basic Header Segment (BHS) 6205 The BHS is 48 bytes long. The Opcode and DataSegmentLength fields 6206 appear in all iSCSI PDUs. In addition, when used, the Initiator 6207 Task Tag and Logical Unit Number always appear in the same location 6208 in the header. 6210 The format of the BHS is: 6212 Byte/ 0 | 1 | 2 | 3 | 6213 / | | | | 6214 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 6215 +---------------+---------------+---------------+---------------+ 6216 0|.|I| Opcode |F| Opcode-specific fields | 6217 +---------------+---------------+---------------+---------------+ 6218 4|TotalAHSLength | DataSegmentLength | 6219 +---------------+---------------+---------------+---------------+ 6220 8| LUN or Opcode-specific fields | 6221 + + 6222 12| | 6223 +---------------+---------------+---------------+---------------+ 6224 16| Initiator Task Tag | 6225 +---------------+---------------+---------------+---------------+ 6226 20/ Opcode-specific fields / 6227 +/ / 6228 +---------------+---------------+---------------+---------------+ 6229 48 6231 10.2.1.1. I 6233 For request PDUs, the I bit set to 1 is an immediate delivery 6234 marker. 6236 10.2.1.2. Opcode 6238 The Opcode indicates the type of iSCSI PDU the header encapsulates. 6240 The Opcodes are divided into two categories: initiator opcodes and 6241 target opcodes. Initiator opcodes are in PDUs sent by the initiator 6242 (request PDUs). Target opcodes are in PDUs sent by the target 6243 (response PDUs). 6245 Initiators MUST NOT use target opcodes and targets MUST NOT use 6246 initiator opcodes. 6248 Initiator opcodes defined in this specification are: 6250 0x00 NOP-Out 6252 0x01 SCSI Command (encapsulates a SCSI Command Descriptor 6253 Block) 6255 0x02 SCSI Task Management function request 6257 0x03 Login Request 6259 0x04 Text Request 6261 0x05 SCSI Data-out (for WRITE operations) 6263 0x06 Logout Request 6265 0x10 SNACK Request 6267 0x1c-0x1e Vendor specific codes 6269 Target opcodes are: 6271 0x20 NOP-In 6273 0x21 SCSI Response - contains SCSI status and possibly sense 6274 information or other response information. 6276 0x22 SCSI Task Management function response 6278 0x23 Login Response 6280 0x24 Text Response 6282 0x25 SCSI Data-in - for READ operations. 6284 0x26 Logout Response 6286 0x31 Ready To Transfer (R2T) - sent by target when it is ready 6287 to receive data. 6289 0x32 Asynchronous Message - sent by target to indicate certain 6290 special conditions. 6292 0x3c-0x3e Vendor specific codes 6294 0x3f Reject 6296 All other opcodes are reserved. 6298 10.2.1.3. Final (F) bit 6300 When set to 1 it indicates the final (or only) PDU of a sequence. 6302 10.2.1.4. Opcode-specific Fields 6304 These fields have different meanings for different opcode types. 6306 10.2.1.5. TotalAHSLength 6308 Total length of all AHS header segments in units of four byte words 6309 including padding, if any. 6311 The TotalAHSLength is only used in PDUs that have an AHS and MUST 6312 be 0 in all other PDUs. 6314 10.2.1.6. DataSegmentLength 6316 This is the data segment payload length in bytes (excluding 6317 padding). The DataSegmentLength MUST be 0 whenever the PDU has no 6318 data segment. 6320 10.2.1.7. LUN 6322 Some opcodes operate on a specific Logical Unit. The Logical Unit 6323 Number (LUN) field identifies which Logical Unit. If the opcode 6324 does not relate to a Logical Unit, this field is either ignored or 6325 may be used in an opcode specific way. The LUN field is 64-bits and 6326 should be formatted in accordance with [SAM2]. For example, LUN[0] 6327 from [SAM2] is BHS byte 8 and so on up to LUN[7] from [SAM2], which 6328 is BHS byte 15. 6330 10.2.1.8. Initiator Task Tag 6332 The initiator assigns a Task Tag to each iSCSI task it issues. 6333 While a task exists, this tag MUST uniquely identify the task 6334 session-wide. SCSI may also use the initiator task tag as part of 6335 the SCSI task identifier when the timespan during which an iSCSI 6336 initiator task tag must be unique extends over the timespan during 6337 which a SCSI task tag must be unique. However, the iSCSI Initiator 6338 Task Tag must exist and be unique even for untagged SCSI commands. 6340 An ITT value of 0xffffffff is reserved and MUST NOT be assigned for 6341 a task by the initiator. The only instance in which it may be seen 6342 on the wire is in a target-initiated NOP-In PDU (Section 10.19) and 6343 in the initiator response to that PDU, if necessary. 6345 10.2.2. Additional Header Segment (AHS) 6347 The general format of an AHS is: 6349 Byte/ 0 | 1 | 2 | 3 | 6350 / | | | | 6351 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 6352 +---------------+---------------+---------------+---------------+ 6353 0| AHSLength | AHSType | AHS-Specific | 6354 +---------------+---------------+---------------+---------------+ 6355 4/ AHS-Specific / 6356 +/ / 6357 +---------------+---------------+---------------+---------------+ 6358 x 6360 10.2.2.1. AHSType 6362 The AHSType field is coded as follows: 6364 bit 0-1 - Reserved 6366 bit 2-7 - AHS code 6368 0 - Reserved 6370 1 - Extended CDB 6371 2 - Expected Bidirectional Read Data Length 6373 3 - 63 Reserved 6375 10.2.2.2. AHSLength 6377 This field contains the effective length in bytes of the AHS 6378 excluding AHSType and AHSLength and padding, if any. The AHS is 6379 padded to the smallest integer number of 4 byte words (i.e., from 0 6380 up to 3 padding bytes). 6382 10.2.2.3. Extended CDB AHS 6384 The format of the Extended CDB AHS is: 6386 Byte/ 0 | 1 | 2 | 3 | 6387 / | | | | 6388 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 6389 +---------------+---------------+---------------+---------------+ 6390 0| AHSLength (CDBLength-15) | 0x01 | Reserved | 6391 +---------------+---------------+---------------+---------------+ 6392 4/ ExtendedCDB...+padding / 6393 +/ / 6394 +---------------+---------------+---------------+---------------+ 6395 x 6397 This type of AHS MUST NOT be used if the CDBLength is less than 17. 6398 The length includes the reserved byte 3. 6400 10.2.2.4. Bidirectional Expected Read-Data Length AHS 6402 The format of the Bidirectional Read Expected Data Transfer Length 6403 AHS is: 6405 Byte/ 0 | 1 | 2 | 3 | 6406 / | | | | 6407 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 6408 +---------------+---------------+---------------+---------------+ 6409 0| AHSLength (0x0005) | 0x02 | Reserved | 6410 +---------------+---------------+---------------+---------------+ 6411 4| Expected Read-Data Length | 6412 +---------------+---------------+---------------+---------------+ 6413 8 6415 10.2.3. Header Digest and Data Digest 6417 Optional header and data digests protect the integrity of the 6418 header and data, respectively. The digests, if present, are 6419 located, respectively, after the header and PDU-specific data, and 6420 cover respectively the header and the PDU data, each including the 6421 padding bytes, if any. 6423 The existence and type of digests are negotiated during the Login 6424 Phase. 6426 The separation of the header and data digests is useful in iSCSI 6427 routing applications, in which only the header changes when a 6428 message is forwarded. In this case, only the header digest should 6429 be recalculated. 6431 Digests are not included in data or header length fields. 6433 A zero-length Data Segment also implies a zero-length data-digest. 6435 10.2.4. Data Segment 6437 The (optional) Data Segment contains PDU associated data. Its 6438 payload effective length is provided in the BHS field - 6439 DataSegmentLength. The Data Segment is also padded to an integer 6440 number of 4 byte words. 6442 10.3. SCSI Command 6444 The format of the SCSI Command PDU is: 6446 Byte/ 0 | 1 | 2 | 3 | 6447 / | | | | 6448 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 6449 +---------------+---------------+---------------+---------------+ 6450 0|.|I| 0x01 |F|R|W|. .|ATTR | Reserved | 6451 +---------------+---------------+---------------+---------------+ 6452 4|TotalAHSLength | DataSegmentLength | 6453 +---------------+---------------+---------------+---------------+ 6454 8| Logical Unit Number (LUN) | 6455 + + 6456 12| | 6457 +---------------+---------------+---------------+---------------+ 6458 16| Initiator Task Tag | 6459 +---------------+---------------+---------------+---------------+ 6460 20| Expected Data Transfer Length | 6461 +---------------+---------------+---------------+---------------+ 6462 24| CmdSN | 6463 +---------------+---------------+---------------+---------------+ 6464 28| ExpStatSN | 6465 +---------------+---------------+---------------+---------------+ 6466 32/ SCSI Command Descriptor Block (CDB) / 6467 +/ / 6468 +---------------+---------------+---------------+---------------+ 6469 48/ AHS (Optional) / 6470 +---------------+---------------+---------------+---------------+ 6471 x/ Header Digest (Optional) / 6472 +---------------+---------------+---------------+---------------+ 6473 y/ (DataSegment, Command Data) (Optional) / 6474 +/ / 6475 +---------------+---------------+---------------+---------------+ 6476 z/ Data Digest (Optional) / 6477 +---------------+---------------+---------------+---------------+ 6479 10.3.1. Flags and Task Attributes (byte 1) 6481 The flags for a SCSI Command are: 6483 bit 0 (F) is set to 1 when no unsolicited SCSI Data-Out PDUs 6484 follow this PDU. When F=1 for a write and if Expected Data 6485 Transfer Length is larger than the DataSegmentLength, the 6486 target may solicit additional data through R2T. 6488 bit 1 (R) is set to 1 when the command is expected to input 6489 data. 6491 bit 2 (W) is set to 1 when the command is expected to output 6492 data. 6494 bit 3-4 Reserved. 6496 bit 5-7 contains Task Attributes. 6498 Task Attributes (ATTR) have one of the following integer values 6499 (see [SAM2] for details): 6501 0 - Untagged 6503 1 - Simple 6505 2 - Ordered 6507 3 - Head of Queue 6509 4 - ACA 6511 5-7 - Reserved 6513 Setting both the W and the F bit to 0 is an error. 6514 Either or both of R and W MAY be 1 when either the Expected Data 6515 Transfer Length and/or Bidirectional Read Expected Data Transfer 6516 Length are 0, but they MUST NOT both be 0 when the Expected Data 6517 Transfer Length and/or Bidirectional Read Expected Data Transfer 6518 Length are not 0 (i.e., when some data transfer is expected the 6519 transfer direction is indicated by the R and/or W bit). 6521 10.3.2. CmdSN - Command Sequence Number 6523 Enables ordered delivery across multiple connections in a single 6524 session. 6526 10.3.3. ExpStatSN 6528 Command responses up to ExpStatSN-1 (mod 2**32) have been received 6529 (acknowledges status) on the connection. 6531 10.3.4. Expected Data Transfer Length 6533 For unidirectional operations, the Expected Data Transfer Length 6534 field contains the number of bytes of data involved in this SCSI 6535 operation. For a unidirectional write operation (W flag set to 1 6536 and R flag set to 0), the initiator uses this field to specify the 6537 number of bytes of data it expects to transfer for this operation. 6538 For a unidirectional read operation (W flag set to 0 and R flag set 6539 to 1), the initiator uses this field to specify the number of bytes 6540 of data it expects the target to transfer to the initiator. It 6541 corresponds to the SAM2 byte count. 6543 For bidirectional operations (both R and W flags are set to 1), 6544 this field contains the number of data bytes involved in the write 6545 transfer. For bidirectional operations, an additional header 6546 segment MUST be present in the header sequence that indicates the 6547 Bidirectional Read Expected Data Transfer Length. The Expected 6548 Data Transfer Length field and the Bidirectional Read Expected Data 6549 Transfer Length field correspond to the SAM2 byte count. 6551 If the Expected Data Transfer Length for a write and the length of 6552 the immediate data part that follows the command (if any) are the 6553 same, then no more data PDUs are expected to follow. In this case, 6554 the F bit MUST be set to 1. 6556 If the Expected Data Transfer Length is higher than the 6557 FirstBurstLength (the negotiated maximum amount of unsolicited data 6558 the target will accept), the initiator MUST send the maximum amount 6559 of unsolicited data OR ONLY the immediate data, if any. 6561 Upon completion of a data transfer, the target informs the 6562 initiator (through residual counts) of how many bytes were actually 6563 processed (sent and/or received) by the target. 6565 10.3.5. CDB - SCSI Command Descriptor Block 6567 There are 16 bytes in the CDB field to accommodate the commonly 6568 used CDBs. Whenever the CDB is larger than 16 bytes, an Extended 6569 CDB AHS MUST be used to contain the CDB spillover. 6571 10.3.6. Data Segment - Command Data 6573 Some SCSI commands require additional parameter data to accompany 6574 the SCSI command. This data may be placed beyond the boundary of 6575 the iSCSI header in a data segment. Alternatively, user data 6576 (e.g., from a WRITE operation) can be placed in the data segment 6577 (both cases are referred to as immediate data). These data are 6578 governed by the rules for solicited vs. unsolicited data outlined 6579 in Section 3.2.4.2 - "Data Transfer Overview". 6581 10.4. SCSI Response 6583 The format of the SCSI Response PDU is: 6585 Byte/ 0 | 1 | 2 | 3 | 6586 / | | | | 6587 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 6588 +---------------+---------------+---------------+---------------+ 6589 0|.|.| 0x21 |1|. .|o|u|O|U|.| Response | Status | 6590 +---------------+---------------+---------------+---------------+ 6591 4|TotalAHSLength | DataSegmentLength | 6592 +---------------+---------------+---------------+---------------+ 6593 8| Reserved | 6594 + + 6595 12| | 6596 +---------------+---------------+---------------+---------------+ 6597 16| Initiator Task Tag | 6598 +---------------+---------------+---------------+---------------+ 6599 20| SNACK Tag or Reserved | 6600 +---------------+---------------+---------------+---------------+ 6601 24| StatSN | 6602 +---------------+---------------+---------------+---------------+ 6603 28| ExpCmdSN | 6604 +---------------+---------------+---------------+---------------+ 6605 32| MaxCmdSN | 6606 +---------------+---------------+---------------+---------------+ 6607 36| ExpDataSN or Reserved | 6608 +---------------+---------------+---------------+---------------+ 6609 40| Bidirectional Read Residual Count or Reserved | 6610 +---------------+---------------+---------------+---------------+ 6611 44| Residual Count or Reserved | 6612 +---------------+---------------+---------------+---------------+ 6613 48| Header-Digest (Optional) | 6614 +---------------+---------------+---------------+---------------+ 6615 / Data Segment (Optional) / 6616 +/ / 6617 +---------------+---------------+---------------+---------------+ 6618 | Data-Digest (Optional) | 6619 +---------------+---------------+---------------+---------------+ 6621 10.4.1. Flags (byte 1) 6623 bit 1-2 Reserved. 6625 bit 3 - (o) set for Bidirectional Read Residual Overflow. In 6626 this case, the Bidirectional Read Residual Count indicates 6627 the number of bytes that were not transferred to the 6628 initiator because the initiator's Expected Bidirectional Read 6629 Data Transfer Length was not sufficient. 6631 bit 4 - (u) set for Bidirectional Read Residual Underflow. In 6632 this case, the Bidirectional Read Residual Count indicates 6633 the number of bytes that were not transferred to the 6634 initiator out of the number of bytes expected to be 6635 transferred. 6637 bit 5 - (O) set for Residual Overflow. In this case, the 6638 Residual Count indicates the number of bytes that were not 6639 transferred because the initiator's Expected Data Transfer 6640 Length was not sufficient. For a bidirectional operation, the 6641 Residual Count contains the residual for the write operation. 6643 bit 6 - (U) set for Residual Underflow. In this case, the 6644 Residual Count indicates the number of bytes that were not 6645 transferred out of the number of bytes that were expected to 6646 be transferred. For a bidirectional operation, the Residual 6647 Count contains the residual for the write operation. 6649 bit 7 - (0) Reserved. 6651 Bits O and U and bits o and u are mutually exclusive (i.e., having 6652 both o and u or O and U set to 1 is a protocol error). 6653 For a response other than "Command Completed at Target", bits 3-6 6654 MUST be 0. 6656 10.4.2. Status 6658 The Status field is used to report the SCSI status of the command 6659 (as specified in [SAM2]) and is only valid if the Response Code is 6660 Command Completed at target. 6662 Some of the status codes defined in [SAM2] are: 6664 0x00 GOOD 6665 0x02 CHECK CONDITION 6667 0x08 BUSY 6669 0x18 RESERVATION CONFLICT 6671 0x28 TASK SET FULL 6673 0x30 ACA ACTIVE 6675 0x40 TASK ABORTED 6677 See [SAM2] for the complete list and definitions. 6679 If a SCSI device error is detected while data from the initiator is 6680 still expected (the command PDU did not contain all the data and 6681 the target has not received a Data PDU with the final bit Set), the 6682 target MUST wait until it receives a Data PDU with the F bit set in 6683 the last expected sequence before sending the Response PDU. 6685 10.4.3. Response 6687 This field contains the iSCSI service response. 6689 iSCSI service response codes defined in this specification are: 6691 0x00 - Command Completed at Target 6693 0x01 - Target Failure 6695 0x80-0xff - Vendor specific 6697 All other response codes are reserved. 6699 The Response is used to report a Service Response. The mapping of 6700 the response code into a SCSI service response code value, if 6701 needed, is outside the scope of this document. However, in symbolic 6702 terms response value 0x00 maps to the SCSI service response (see 6703 [SAM2] and [SPC3]) of TASK COMPLETE or LINKED COMMAND COMPLETE. All 6704 other Response values map to the SCSI service response of SERVICE 6705 DELIVERY OR TARGET FAILURE. 6707 If a SCSI Response PDU does not arrive before the session is 6708 terminated, the SCSI service response is SERVICE DELIVERY OR TARGET 6709 FAILURE. 6711 A non-zero response field indicates a failure to execute the 6712 command in which case the Status and Flag fields are undefined. 6714 10.4.4. SNACK Tag 6716 This field contains a copy of the SNACK Tag of the last SNACK Tag 6717 accepted by the target on the same connection and for the command 6718 for which the response is issued. Otherwise it is reserved and 6719 should be set to 0. 6721 After issuing a R-Data SNACK the initiator must discard any SCSI 6722 status unless contained in an SCSI Response PDU carrying the same 6723 SNACK Tag as the last issued R-Data SNACK for the SCSI command on 6724 the current connection. 6726 For a detailed discussion on R-Data SNACK see SNACK. 6728 10.4.5. Residual Count 6730 10.4.5.1. Field Semantics 6732 The Residual Count field MUST be valid in the case where either the 6733 U bit or the O bit is set. If neither bit is set, the Residual 6734 Count field is reserved. Targets may set the residual count and 6735 initiators may use it when the response code is "completed at 6736 target" (even if the status returned is not GOOD). If the O bit is 6737 set, the Residual Count indicates the number of bytes that were not 6738 transferred because the initiator's Expected Data Transfer Length 6739 was not sufficient. If the U bit is set, the Residual Count 6740 indicates the number of bytes that were not transferred out of the 6741 number of bytes expected to be transferred. 6743 10.4.5.2. Residuals Concepts Overview 6745 SCSI-Presented Data Transfer Length (SPDTL) is the term this 6746 document uses (see Section 1.1 for definition) to represent the 6747 aggregate data length that the target SCSI layer attempts to 6748 transfer using the local iSCSI layer for a task. Expected Data 6749 Transfer Length (EDTL) is the iSCSI term that represents the length 6750 of data that the iSCSI layer expects to transfer for a task. EDTL 6751 is specified in the SCSI Command PDU. 6753 When SPDTL = EDTL for a task, the target iSCSI layer completes the 6754 task with no residuals. Whenever SPDTL differs from EDTL for a 6755 task, that task is said to have a residual. 6757 If SPDTL > EDTL for a task, iSCSI Overflow MUST be signaled in the 6758 SCSI Response PDU as specified in Section 10.4.5.1. The Residual 6759 Count MUST be set to the numerical value of (SPDTL - EDTL). 6761 If SPDTL < EDTL for a task, iSCSI Underflow MUST be signaled in the 6762 SCSI Response PDU as specified in Section 10.4.5.1. The Residual 6763 Count MUST be set to the numerical value of (EDTL - SPDTL). 6765 Note that the Overflow and Underflow scenarios are independent of 6766 Data-In and Data-Out. Either scenario is logically possible in 6767 either direction of data transfer. 6769 10.4.5.3. SCSI REPORT LUNS and Residual Overflow 6771 This section discusses the residual overflow issues citing the 6772 example of the SCSI REPORT LUNS command. Note however that there 6773 are several SCSI commands (e.g., INQUIRY) with ALLOCATION LENGTH 6774 fields following the same underlying rules. The semantics in the 6775 rest of the section apply to all such SCSI commands. 6777 The specification of the SCSI REPORT LUNS command requires that the 6778 SCSI target limit the amount of data transferred to a maximum size 6779 (ALLOCATION LENGTH) provided by the initiator in the REPORT LUNS 6780 CDB. 6782 If the Expected Data Transfer Length (EDTL) in the iSCSI header of 6783 the SCSI Command PDU for a REPORT LUNS command is set to at least 6784 as large as that ALLOCATION LENGTH, the SCSI layer truncation 6785 prevents an iSCSI Residual Overflow from occurring. A SCSI 6786 initiator can detect that such truncation has occurred via other 6787 information at theS CSI layer. The rest of the section elaborates 6788 this required behavior. 6790 The SCSI REPORT LUNS command requests a target SCSI layer to return 6791 a logical unit inventory (LUN list) to the initiator SCSI layer 6792 (see Section 6.21 of [SPC3]). The size of this LUN list may not be 6793 known to the initiator SCSI layer when it issues the REPORT LUNS 6794 command; to avoid transferring more LUN list data than the 6795 initiator is prepared for, the REPORT LUNS CDB contains an 6796 ALLOCATION LENGTH field to specify the maximum amount of data to be 6797 transferred to the initiator for this command. If the initiator 6798 SCSI layer has underestimated the number of logical units at the 6799 target, it is possible that the complete logical unit inventory 6800 does not fit in the specified ALLOCATION LENGTH. In this situation, 6801 Section 4.3.3.6 in [SPC3] requires that the target SCSI layer 6802 "shall terminate transfers to the Data-In Buffer" when the number 6803 of bytes specified by the ALLOCATION LENGTH field have been 6804 transferred. 6806 Therefore, in response to a REPORT LUNS command, the SCSI layer at 6807 the target presents at most ALLOCATION LENGTH bytes of data 6808 (logical unit inventory) to iSCSI for transfer to the initiator. 6809 For a REPORT LUNS command, if the iSCSI EDTL is at least as large 6810 as the ALLOCATION LENGTH, the SCSI truncation ensures that the EDTL 6811 will accommodate all of the data to be transferred. If all of the 6812 logical unit inventory data presented to the iSCSI layer -- i.e., 6813 the data remaining after any SCSI truncation -- is transferred to 6814 the initiator by the iSCSI layer, an iSCSI Residual Overflow has 6815 not occurred and the iSCSI (O) bit MUST NOT be set in the SCSI 6816 Response or final SCSI Data-Out PDU. Note that this behavior is 6817 implied by the combination of Section 10.4.5.1 along with the 6818 specification of the REPORT LUNS command in [SPC3]. However, if the 6819 iSCSI EDTL is larger than the ALLOCATION LENGTH in this scenario, 6820 note that the iSCSI Underflow MUST be signaled in the SCSI Response 6821 PDU. An iSCSI Underflow MUST also be signaled when the iSCSI EDTL 6822 is equal to the ALLOCATION LENGTH but the logical unit inventory 6823 data presented to the iSCSI layer is smaller than the ALLOCATION 6824 LENGTH. 6826 The LUN LIST LENGTH field in the logical unit inventory (the first 6827 field in the inventory) is not affected by truncation of the 6828 inventory to fit in ALLOCATION LENGTH; this enables a SCSI 6829 initiator to determine that the received inventory is incomplete by 6830 noticing that the LUN LIST LENGTH in the inventory is larger than 6831 the ALLOCATION LENGTH that was sent in the REPORT LUNS CDB. A 6832 common initiator behavior in this situation is to re-issue the 6833 REPORT LUNS command with a larger ALLOCATION LENGTH. 6835 10.4.6. Bidirectional Read Residual Count 6837 The Bidirectional Read Residual Count field MUST be valid in the 6838 case where either the u bit or the o bit is set. If neither bit is 6839 set, the Bidirectional Read Residual Count field is reserved. 6840 Targets may set the Bidirectional Read Residual Count and 6841 initiators may use it when the response code is "completed at 6842 target". If the o bit is set, the Bidirectional Read Residual Count 6843 indicates the number of bytes that were not transferred to the 6844 initiator because the initiator's Expected Bidirectional Read 6845 Transfer Length was not sufficient. If the u bit is set, the 6846 Bidirectional Read Residual Count indicates the number of bytes 6847 that were not transferred to the initiator out of the number of 6848 bytes expected to be transferred. 6850 10.4.7. Data Segment - Sense and Response Data Segment 6852 iSCSI targets MUST support and enable autosense. If Status is CHECK 6853 CONDITION (0x02), then the Data Segment MUST contain sense data for 6854 the failed command. 6856 For some iSCSI responses, the response data segment MAY contain 6857 some response related information, (e.g., for a target failure, it 6858 may contain a vendor specific detailed description of the failure). 6860 If the DataSegmentLength is not 0, the format of the Data Segment 6861 is as follows: 6863 Byte/ 0 | 1 | 2 | 3 | 6864 / | | | | 6865 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 6866 +---------------+---------------+---------------+---------------+ 6867 0|SenseLength | Sense Data | 6868 +---------------+---------------+---------------+---------------+ 6869 x/ Sense Data / 6870 +---------------+---------------+---------------+---------------+ 6871 y/ Response Data / 6872 / / 6873 +---------------+---------------+---------------+---------------+ 6874 z| 6876 10.4.7.1. SenseLength 6878 Length of Sense Data. 6880 10.4.7.2. Sense Data 6882 The Sense Data contains detailed information about a check 6883 condition and [SPC3] specifies the format and content of the Sense 6884 Data. 6886 Certain iSCSI conditions result in the command being terminated at 6887 the target (response Command Completed at Target) with a SCSI Check 6888 Condition Status as outlined in the next table: 6890 +--------------------------+----------+---------------------------+ 6891 | iSCSI Condition |Sense | Additional Sense Code & | 6892 | |Key | Qualifier | 6893 +--------------------------+----------+---------------------------+ 6894 | Unexpected unsolicited |Aborted | ASC = 0x0c ASCQ = 0x0c | 6895 | data |Command-0B| Write Error | 6896 +--------------------------+----------+---------------------------+ 6897 | Incorrect amount of data |Aborted | ASC = 0x0c ASCQ = 0x0d | 6898 | |Command-0B| Write Error | 6899 +--------------------------+----------+---------------------------+ 6900 | Protocol Service CRC |Aborted | ASC = 0x47 ASCQ = 0x05 | 6901 | error |Command-0B| CRC Error Detected | 6902 +--------------------------+----------+---------------------------+ 6903 | SNACK rejected |Aborted | ASC = 0x11 ASCQ = 0x13 | 6904 | |Command-0B| Read Error | 6905 +--------------------------+----------+---------------------------+ 6907 The target reports the "Incorrect amount of data" condition if 6908 during data output the total data length to output is greater than 6909 FirstBurstLength and the initiator sent unsolicited non-immediate 6910 data but the total amount of unsolicited data is different than 6911 FirstBurstLength. The target reports the same error when the amount 6912 of data sent as a reply to an R2T does not match the amount 6913 requested. 6915 10.4.8. ExpDataSN 6917 The number of Data-In (read) PDUs the target has sent for the 6918 command. 6920 This field MUST be 0 if the response code is not Command Completed 6921 at Target or the target sent no Data-In PDUs for the command. 6922 10.4.9. StatSN - Status Sequence Number 6924 StatSN is a Sequence Number that the target iSCSI layer generates 6925 per connection and that in turn, enables the initiator to 6926 acknowledge status reception. StatSN is incremented by 1 for every 6927 response/status sent on a connection except for responses sent as a 6928 result of a retry or SNACK. In the case of responses sent due to a 6929 retransmission request, the StatSN MUST be the same as the first 6930 time the PDU was sent unless the connection has since been 6931 restarted. 6933 10.4.10. ExpCmdSN - Next Expected CmdSN from this Initiator 6935 ExpCmdSN is a Sequence Number that the target iSCSI returns to the 6936 initiator to acknowledge command reception. It is used to update a 6937 local variable with the same name. An ExpCmdSN equal to MaxCmdSN+1 6938 indicates that the target cannot accept new commands. 6940 10.4.11. MaxCmdSN - Maximum CmdSN from this Initiator 6942 MaxCmdSN is a Sequence Number that the target iSCSI returns to the 6943 initiator to indicate the maximum CmdSN the initiator can send. It 6944 is used to update a local variable with the same name. If MaxCmdSN 6945 is equal to ExpCmdSN-1, this indicates to the initiator that the 6946 target cannot receive any additional commands. When MaxCmdSN 6947 changes at the target while the target has no pending PDUs to 6948 convey this information to the initiator, it MUST generate a NOP-IN 6949 to carry the new MaxCmdSN. 6951 10.5. Task Management Function Request 6953 Byte/ 0 | 1 | 2 | 3 | 6954 / | | | | 6955 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 6956 +---------------+---------------+---------------+---------------+ 6957 0|.|I| 0x02 |1| Function | Reserved | 6958 +---------------+---------------+---------------+---------------+ 6959 4|TotalAHSLength | DataSegmentLength | 6960 +---------------+---------------+---------------+---------------+ 6961 8| Logical Unit Number (LUN) or Reserved | 6962 + + 6963 12| | 6964 +---------------+---------------+---------------+---------------+ 6965 16| Initiator Task Tag | 6966 +---------------+---------------+---------------+---------------+ 6967 20| Referenced Task Tag or 0xffffffff | 6968 +---------------+---------------+---------------+---------------+ 6969 24| CmdSN | 6970 +---------------+---------------+---------------+---------------+ 6971 28| ExpStatSN | 6972 +---------------+---------------+---------------+---------------+ 6973 32| RefCmdSN or Reserved | 6974 +---------------+---------------+---------------+---------------+ 6975 36| ExpDataSN or Reserved | 6976 +---------------+---------------+---------------+---------------+ 6977 40/ Reserved / 6978 +/ / 6979 +---------------+---------------+---------------+---------------+ 6980 48| Header-Digest (Optional) | 6981 +---------------+---------------+---------------+---------------+ 6983 10.5.1. Function 6985 The Task Management functions provide an initiator with a way to 6986 explicitly control the execution of one or more Tasks (SCSI and 6987 iSCSI tasks). The Task Management function codes are listed below. 6988 For a more detailed description of SCSI task management, see 6989 [SAM2]. 6991 1 - ABORT TASK - aborts the task identified by the Referenced 6992 Task Tag field. 6994 2 - ABORT TASK SET - aborts all Tasks issued via this session 6995 on the logical unit. 6997 3 - CLEAR ACA - clears the Auto Contingent Allegiance 6998 condition. 7000 4 - CLEAR TASK SET - aborts all Tasks in the appropriate task 7001 set as defined by the TST field in the Control mode page (see 7002 [SPC3]). 7004 5 - LOGICAL UNIT RESET 7006 6 - TARGET WARM RESET 7008 7 - TARGET COLD RESET 7010 8 - TASK REASSIGN - reassigns connection allegiance for the 7011 task identified by the Initiator Task Tag field to this 7012 connection, thus resuming the iSCSI exchanges for the task. 7014 For all these functions, the Task Management function response MUST 7015 be returned as detailed in Section 10.6. All these functions apply 7016 to the referenced tasks regardless of whether they are proper SCSI 7017 tasks or tagged iSCSI operations. Task management requests must 7018 act on all the commands from the same session having a CmdSN lower 7019 than the task management CmdSN. LOGICAL UNIT RESET, TARGET WARM 7020 RESET and TARGET COLD RESET may affect commands from other sessions 7021 or commands from the same session regardless of their CmdSN value. 7023 If the task management request is marked for immediate delivery, it 7024 must be considered immediately for execution, but the operations 7025 involved (all or part of them) may be postponed to allow the target 7026 to receive all relevant tasks. According to [SAM2], for all the 7027 tasks covered by the Task Management response (i.e., with CmdSN 7028 lower than the task management command CmdSN) but except the Task 7029 Management response to a TASK REASSIGN, additional responses MUST 7030 NOT be delivered to the SCSI layer after the Task Management 7031 response. The iSCSI initiator MAY deliver to the SCSI layer all 7032 responses received before the Task Management response (i.e., it is 7033 a matter of implementation if the SCSI responses, received before 7034 the Task Management response but after the task management request 7035 was issued, are delivered to the SCSI layer by the iSCSI layer in 7036 the initiator). The iSCSI target MUST ensure that no responses for 7037 the tasks covered by a task management function are delivered to 7038 the iSCSI initiator after the Task Management response except for a 7039 task covered by a TASK REASSIGN. 7041 For ABORT TASK SET and CLEAR TASK SET, the issuing initiator MUST 7042 continue to respond to all valid target transfer tags (received via 7043 R2T, Text Response, NOP-In, or SCSI Data-in PDUs) related to the 7044 affected task set, even after issuing the task management request. 7045 The issuing initiator SHOULD however terminate (i.e., by setting 7046 the F-bit to 1) these response sequences as quickly as possible. 7047 The target on its part MUST wait for responses on all affected 7048 target transfer tags before acting on either of these two task 7049 management requests. In case all or part of the response sequence 7050 is not received (due to digest errors) for a valid TTT, the target 7051 MAY treat it as a case of within-command error recovery class (see 7052 Section 6.1.4.1 - "Recovery Within-command") if it is supporting 7053 ErrorRecoveryLevel >= 1, or alternatively may drop the connection 7054 to complete the requested task set function. 7056 If an ABORT TASK is issued for a task created by an immediate 7057 command then RefCmdSN MUST be that of the Task Management request 7058 itself (i.e. CmdSN and RefCmdSN are equal); otherwise RefCmdSN MUST 7059 be set to the CmdSN of the task to be aborted (lower than CmdSN). 7061 If the connection is still active (it is not undergoing an implicit 7062 or explicit logout), ABORT TASK MUST be issued on the same 7063 connection to which the task to be aborted is allegiant at the time 7064 the Task Management Request is issued. If the connection is 7065 implicitly or explicitly logged out (i.e., no other request will be 7066 issued on the failing connection and no other response will be 7067 received on the failing connection), then an ABORT TASK function 7068 request may be issued on another connection. This Task Management 7069 request will then establish a new allegiance for the command to be 7070 aborted as well as abort it (i.e., the task to be aborted will not 7071 have to be retried or reassigned, and its status, if sent but not 7072 acknowledged, will be resent followed by the Task Management 7073 response). 7075 At the target an ABORT TASK function MUST NOT be executed on a Task 7076 Management request; such a request MUST result in Task Management 7077 response of "Function rejected". 7079 For the LOGICAL UNIT RESET function, the target MUST behave as 7080 dictated by the Logical Unit Reset function in [SAM2]. 7082 The implementation of the TARGET WARM RESET function and the TARGET 7083 COLD RESET function is OPTIONAL and when implemented, should act as 7084 described below. The TARGET WARM RESET is also subject to SCSI 7085 access controls on the requesting initiator as defined in [SPC3]. 7086 When authorization fails at the target, the appropriate response as 7087 described in Targe MUST be returned by the target. The TARGET COLD 7088 RESET function is not subject to SCSI access controls, but its 7089 execution privileges may be managed by iSCSI mechanisms such as 7090 login authentication. 7092 When executing the TARGET WARM RESET and TARGET COLD RESET 7093 functions, the target cancels all pending operations on all Logical 7094 Units known by the issuing initiator. Both functions are equivalent 7095 to the Target Reset function specified by [SAM2]. They can affect 7096 many other initiators logged in with the servicing SCSI target 7097 port. 7099 The target MUST treat the TARGET COLD RESET function additionally 7100 as a power on event, thus terminating all of its TCP connections to 7101 all initiators (all sessions are terminated). For this reason, the 7102 Service Response (defined by [SAM2]) for this SCSI task management 7103 function may not be reliably delivered to the issuing initiator 7104 port. 7106 For the TASK REASSIGN function, the target should reassign the 7107 connection allegiance to this new connection (and thus resume iSCSI 7108 exchanges for the task). TASK REASSIGN MUST ONLY be received by the 7109 target after the connection on which the command was previously 7110 executing has been successfully logged-out. The Task Management 7111 response MUST be issued before the reassignment becomes effective. 7112 For additional usage semantics see Section 6.2 - "Retry and 7113 Reassign in Recovery". 7115 At the target a TASK REASSIGN function request MUST NOT be executed 7116 to reassign the connection allegiance of a Task Management function 7117 request, an active text negotiation task, or a Logout task; such a 7118 request MUST result in Task Management response of "Function 7119 rejected". 7121 TASK REASSIGN MUST be issued as an immediate command. 7123 10.5.2. TotalAHSLength and DataSegmentLength 7125 For this PDU, TotalAHSLength and DataSegmentLength MUST be 0. 7127 10.5.3. LUN 7129 This field is required for functions that address a specific LU 7130 (ABORT TASK, CLEAR TASK SET, ABORT TASK SET, CLEAR ACA, LOGICAL 7131 UNIT RESET) and is reserved in all others. 7133 10.5.4. Referenced Task Tag 7135 The Initiator Task Tag of the task to be aborted for the ABORT TASK 7136 function or reassigned for the TASK REASSIGN function. For all the 7137 other functions this field MUST be set to the reserved value 7138 0xffffffff. 7140 10.5.5. RefCmdSN 7142 If an ABORT TASK is issued for a task created by an immediate 7143 command then RefCmdSN MUST be that of the Task Management request 7144 itself (i.e. CmdSN and RefCmdSN are equal). 7146 For an ABORT TASK of a task created by non-immediate command 7147 RefCmdSN MUST be set to the CmdSN of the task identified by the 7148 Referenced Task Tag field. Targets must use this field as described 7149 in Res when the task identified by the Referenced Task Tag field is 7150 not with the target. 7152 Otherwise, this field is reserved. 7154 10.5.6. ExpDataSN 7156 For recovery purposes, the iSCSI target and initiator maintain a 7157 data acknowledgement reference number - the first input DataSN 7158 number unacknowledged by the initiator. When issuing a new command, 7159 this number is set to 0. If the function is TASK REASSIGN, which 7160 establishes a new connection allegiance for a previously issued 7161 Read or Bidirectional command, ExpDataSN will contain an updated 7162 data acknowledgement reference number or the value 0; the latter 7163 indicating that the data acknowledgement reference number is 7164 unchanged. The initiator MUST discard any data PDUs from the 7165 previous execution that it did not acknowledge and the target MUST 7166 transmit all Data-in PDUs (if any) starting with the data 7167 acknowledgement reference number. The number of retransmitted PDUs 7168 may or may not be the same as the original transmission depending 7169 on if there was a change in MaxRecvDataSegmentLength in the 7170 reassignment. The target MAY also send no more Data-In PDUs if all 7171 data has been acknowledged. 7173 The value of ExpDataSN MUST be 0 or higher than the DataSN of the 7174 last acknowledged Data-In PDU, but not larger than DataSN+1 of the 7175 last Data-IN PDU sent by the target. Any other value MUST be 7176 ignored by the target. 7178 For other functions this field is reserved. 7180 10.6. Task Management Function Response 7181 Byte/ 0 | 1 | 2 | 3 | 7182 / | | | | 7183 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 7184 +---------------+---------------+---------------+---------------+ 7185 0|.|.| 0x22 |1| Reserved | Response | Reserved | 7186 +---------------+---------------+---------------+---------------+ 7187 4|TotalAHSLength | DataSegmentLength | 7188 +---------------------------------------------------------------+ 7189 8/ Reserved / 7190 / / 7191 +---------------+---------------+---------------+---------------+ 7192 16| Initiator Task Tag | 7193 +---------------+---------------+---------------+---------------+ 7194 20| Reserved | 7195 +---------------+---------------+---------------+---------------+ 7196 24| StatSN | 7197 +---------------+---------------+---------------+---------------+ 7198 28| ExpCmdSN | 7199 +---------------+---------------+---------------+---------------+ 7200 32| MaxCmdSN | 7201 +---------------+---------------+---------------+---------------+ 7202 36/ Reserved / 7203 +/ / 7204 +---------------+---------------+---------------+---------------+ 7205 48| Header-Digest (Optional) | 7206 +---------------+---------------+---------------+---------------+ 7208 For the functions ABORT TASK, ABORT TASK SET, CLEAR ACA, CLEAR TASK 7209 SET, LOGICAL UNIT RESET, TARGET COLD RESET, TARGET WARM RESET and 7210 TASK REASSIGN, the target performs the requested Task Management 7211 function and sends a Task Management response back to the 7212 initiator. For TASK REASSIGN, the new connection allegiance MUST 7213 ONLY become effective at the target after the target issues the 7214 Task Management Response. 7216 10.6.1. Response 7218 The target provides a Response, which may take on the following 7219 values: 7221 0 - Function complete. 7222 1 - Task does not exist. 7224 2 - LUN does not exist. 7225 3 - Task still allegiant. 7226 4 - Task allegiance reassignment not supported. 7227 5 - Task management function not supported. 7228 6 - Function authorization failed. 7229 255 - Function rejected. 7231 All other values are reserved. 7233 For a discussion on usage of response codes 3 and 4, see Section 7234 6.2.2 - "Allegiance Reassignment". 7236 For the TARGET COLD RESET and TARGET WARM RESET functions, the 7237 target cancels all pending operations across all Logical Units 7238 known to the issuing initiator. For the TARGET COLD RESET 7239 function, the target MUST then close all of its TCP connections to 7240 all initiators (terminates all sessions). 7242 The mapping of the response code into a SCSI service response code 7243 value, if needed, is outside the scope of this document. However, 7244 in symbolic terms Response values 0 and 1 map to the SCSI service 7245 response of FUNCTION COMPLETE. All other Response values map to 7246 the SCSI service response of FUNCTION REJECTED. If a Task 7247 Management function response PDU does not arrive before the session 7248 is terminated, the SCSI service response is SERVICE DELIVERY OR 7249 TARGET FAILURE. 7251 The response to ABORT TASK SET and CLEAR TASK SET MUST only be 7252 issued by the target after all of the commands affected have been 7253 received by the target, the corresponding task management functions 7254 have been executed by the SCSI target, and the delivery of all 7255 responses delivered until the task management function completion 7256 have been confirmed (acknowledged through ExpStatSN) by the 7257 initiator on all connections of this session. For the exact 7258 timeline of events, refer to Section 3.2.3.3 and Section 3.2.3.4. 7260 For the ABORT TASK function, 7262 If the Referenced Task Tag identifies a valid task leading to a 7263 successful termination, then targets must return the 7264 "Function complete" response. 7266 If the Referenced Task Tag does not identify an existing task, 7267 but if the CmdSN indicated by the RefCmdSN field in the Task 7268 Management function request is within the valid CmdSN window 7269 and less than the CmdSN of the Task Management function 7270 request itself, then targets must consider the CmdSN 7271 received and return the "Function complete" response. 7272 If the Referenced Task Tag does not identify an existing task 7273 and if the CmdSN indicated by the RefCmdSN field in the Task 7274 Management function request is outside the valid CmdSN 7275 window, then targets must return the "Task does not exist" 7276 response. 7278 For response semantics on function types that can potentially 7279 impact multiple active tasks on the target, see Section 3.2.3. 7281 10.6.2. TotalAHSLength and DataSegmentLength 7283 For this PDU TotalAHSLength and DataSegmentLength MUST be 0. 7285 10.7. SCSI Data-out & SCSI Data-in 7287 The SCSI Data-out PDU for WRITE operations has the following 7288 format: 7290 Byte/ 0 | 1 | 2 | 3 | 7291 / | | | | 7292 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 7293 +---------------+---------------+---------------+---------------+ 7294 0|.|.| 0x05 |F| Reserved | 7295 +---------------+---------------+---------------+---------------+ 7296 4|TotalAHSLength | DataSegmentLength | 7297 +---------------+---------------+---------------+---------------+ 7298 8| LUN or Reserved | 7299 + + 7300 12| | 7301 +---------------+---------------+---------------+---------------+ 7302 16| Initiator Task Tag | 7303 +---------------+---------------+---------------+---------------+ 7304 20| Target Transfer Tag or 0xffffffff | 7305 +---------------+---------------+---------------+---------------+ 7306 24| Reserved | 7307 +---------------+---------------+---------------+---------------+ 7308 28| ExpStatSN | 7309 +---------------+---------------+---------------+---------------+ 7310 32| Reserved | 7311 +---------------+---------------+---------------+---------------+ 7312 36| DataSN | 7313 +---------------+---------------+---------------+---------------+ 7314 40| Buffer Offset | 7315 +---------------+---------------+---------------+---------------+ 7316 44| Reserved | 7317 +---------------+---------------+---------------+---------------+ 7318 48| Header-Digest (Optional) | 7319 +---------------+---------------+---------------+---------------+ 7320 / DataSegment / 7321 +/ / 7322 +---------------+---------------+---------------+---------------+ 7323 | Data-Digest (Optional) | 7324 +---------------+---------------+---------------+---------------+ 7326 The SCSI Data-in PDU for READ operations has the following format: 7328 Byte/ 0 | 1 | 2 | 3 | 7329 / | | | | 7330 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 7331 +---------------+---------------+---------------+---------------+ 7332 0|.|.| 0x25 |F|A|0 0 0|O|U|S| Reserved |Status or Rsvd | 7333 +---------------+---------------+---------------+---------------+ 7334 4|TotalAHSLength | DataSegmentLength | 7335 +---------------+---------------+---------------+---------------+ 7336 8| LUN or Reserved | 7337 + + 7338 12| | 7339 +---------------+---------------+---------------+---------------+ 7340 16| Initiator Task Tag | 7341 +---------------+---------------+---------------+---------------+ 7342 20| Target Transfer Tag or 0xffffffff | 7343 +---------------+---------------+---------------+---------------+ 7344 24| StatSN or Reserved | 7345 +---------------+---------------+---------------+---------------+ 7346 28| ExpCmdSN | 7347 +---------------+---------------+---------------+---------------+ 7348 32| MaxCmdSN | 7349 +---------------+---------------+---------------+---------------+ 7350 36| DataSN | 7351 +---------------+---------------+---------------+---------------+ 7352 40| Buffer Offset | 7353 +---------------+---------------+---------------+---------------+ 7354 44| Residual Count | 7355 +---------------+---------------+---------------+---------------+ 7356 48| Header-Digest (Optional) | 7357 +---------------+---------------+---------------+---------------+ 7358 / DataSegment / 7359 +/ / 7360 +---------------+---------------+---------------+---------------+ 7361 | Data-Digest (Optional) | 7362 +---------------+---------------+---------------+---------------+ 7364 Status can accompany the last Data-in PDU if the command did not 7365 end with an exception (i.e., the status is "good status" - GOOD, 7366 CONDITION MET or INTERMEDIATE CONDITION MET). The presence of 7367 status (and of a residual count) is signaled though the S flag bit. 7369 Although targets MAY choose to send even non-exception status in 7370 separate responses, initiators MUST support non-exception status in 7371 Data-In PDUs. 7373 10.7.1. F (Final) Bit 7375 For outgoing data, this bit is 1 for the last PDU of unsolicited 7376 data or the last PDU of a sequence that answers an R2T. 7378 For incoming data, this bit is 1 for the last input (read) data PDU 7379 of a sequence. Input can be split into several sequences, each 7380 having its own F bit. Splitting the data stream into sequences does 7381 not affect DataSN counting on Data-In PDUs. It MAY be used as a 7382 "change direction" indication for Bidirectional operations that 7383 need such a change. 7385 DataSegmentLength MUST NOT exceed MaxRecvDataSegmentLength for the 7386 direction it is sent and the total of all the DataSegmentLength of 7387 all PDUs in a sequence MUST NOT exceed MaxBurstLength (or 7388 FirstBurstLength for unsolicited data). However the number of 7389 individual PDUs in a sequence (or in total) may be higher than the 7390 MaxBurstLength (or FirstBurstLength) to MaxRecvDataSegmentLength 7391 ratio (as PDUs may be limited in length by the sender 7392 capabilities). Using DataSegmentLength of 0 may increase beyond 7393 what is reasonable for the number of PDUs and should therefore be 7394 avoided. 7396 For Bidirectional operations, the F bit is 1 for both the end of 7397 the input sequences and the end of the output sequences. 7399 10.7.2. A (Acknowledge) bit 7401 For sessions with ErrorRecoveryLevel 1 or higher, the target sets 7402 this bit to 1 to indicate that it requests a positive 7403 acknowledgement from the initiator for the data received. The 7404 target should use the A bit moderately; it MAY only set the A bit 7405 to 1 once every MaxBurstLength bytes, or on the last Data-In PDU 7406 that concludes the entire requested read data transfer for the task 7407 from the target's perspective, and it MUST NOT do so more 7408 frequently. The target MUST NOT set to 1 the A bit for sessions 7409 with ErrorRecoveryLevel=0. The initiator MUST ignore the A bit set 7410 to 1 for sessions with ErrorRecoveryLevel=0. 7412 On receiving a Data-In PDU with the A bit set to 1 on a session 7413 with ErrorRecoveryLevel greater than 0, if there are no holes in 7414 the read data until that Data-In PDU, the initiator MUST issue a 7415 SNACK of type DataACK except when it is able to acknowledge the 7416 status for the task immediately via ExpStatSN on other outbound 7417 PDUs if the status for the task is also received. In the latter 7418 case (acknowledgement through ExpStatSN), sending a SNACK of type 7419 DataACK in response to the A bit is OPTIONAL, but if it is done, it 7420 must not be sent after the status acknowledgement through 7421 ExpStatSN. If the initiator has detected holes in the read data 7422 prior to that Data-In PDU, it MUST postpone issuing the SNACK of 7423 type DataACK until the holes are filled. An initiator also MUST NOT 7424 acknowledge the status for the task before those holes are filled. 7425 A status acknowledgement for a task that generated the Data-In PDUs 7426 is considered by the target as an implicit acknowledgement of the 7427 Data-In PDUs if such an acknowledgement was requested by the 7428 target. 7430 10.7.3. Flags (byte 1) 7432 The last SCSI Data packet sent from a target to an initiator for a 7433 SCSI command that completed successfully (with a status of GOOD, 7434 CONDITION MET, INTERMEDIATE or INTERMEDIATE CONDITION MET) may also 7435 optionally contain the Status for the data transfer. In this case, 7436 Sense Data cannot be sent together with the Command Status. If the 7437 command is completed with an error, then the response and sense 7438 data MUST be sent in a SCSI Response PDU (i.e., MUST NOT be sent in 7439 a SCSI Data packet). For Bidirectional commands, the status MUST be 7440 sent in a SCSI Response PDU. 7442 bit 2-4 - Reserved. 7444 bit 5-6 - used the same as in a SCSI Response. These bits are 7445 only valid when S is set to 1. For details see SNACK . 7447 bit 7 S (status)- set to indicate that the Command Status field 7448 contains status. If this bit is set to 1, the F bit MUST also 7449 be set to 1. 7451 The fields StatSN, Status, and Residual Count only have meaningful 7452 content if the S bit is set to 1 and their values are defined in 7453 SNACK . 7455 10.7.4. Target Transfer Tag and LUN 7457 On outgoing data, the Target Transfer Tag is provided to the target 7458 if the transfer is honoring an R2T. In this case, the Target 7459 Transfer Tag field is a replica of the Target Transfer Tag provided 7460 with the R2T. 7462 On incoming data, the Target Transfer Tag and LUN MUST be provided 7463 by the target if the A bit is set to 1; otherwise they are 7464 reserved. The Target Transfer Tag and LUN are copied by the 7465 initiator into the SNACK of type DataACK that it issues as a result 7466 of receiving a SCSI Data-in PDU with the A bit set to 1. 7468 The Target Transfer Tag values are not specified by this protocol 7469 except that the value 0xffffffff is reserved and means that the 7470 Target Transfer Tag is not supplied. If the Target Transfer Tag is 7471 provided, then the LUN field MUST hold a valid value and be 7472 consistent with whatever was specified with the command; otherwise, 7473 the LUN field is reserved. 7475 10.7.5. DataSN 7477 For input (read) or bidirectional Data-In PDUs, the DataSN is the 7478 input PDU number within the data transfer for the command 7479 identified by the Initiator Task Tag. 7481 R2T and Data-In PDUs, in the context of bidirectional commands, 7482 share the numbering sequence (see Section 3.2.2.4 - "Data 7483 Sequencing"). 7485 For output (write) data PDUs, the DataSN is the Data-Out PDU number 7486 within the current output sequence. The current output sequence is 7487 either identified by the Initiator Task Tag (for unsolicited data) 7488 or is a data sequence generated for one R2T (for data solicited 7489 through R2T). 7491 10.7.6. Buffer Offset 7493 The Buffer Offset field contains the offset of this PDU payload 7494 data within the complete data transfer. The sum of the buffer 7495 offset and length should not exceed the expected transfer length 7496 for the command. 7498 The order of data PDUs within a sequence is determined by 7499 DataPDUInOrder. When set to Yes, it means that PDUs have to be in 7500 increasing Buffer Offset order and overlays are forbidden. 7502 The ordering between sequences is determined by 7503 DataSequenceInOrder. When set to Yes, it means that sequences have 7504 to be in increasing Buffer Offset order and overlays are forbidden. 7506 10.7.7. DataSegmentLength 7508 This is the data payload length of a SCSI Data-In or SCSI Data-Out 7509 PDU. The sending of 0 length data segments should be avoided, but 7510 initiators and targets MUST be able to properly receive 0 length 7511 data segments. 7513 The Data Segments of Data-in and Data-out PDUs SHOULD be filled to 7514 the integer number of 4 byte words (real payload) unless the F bit 7515 is set to 1. 7517 10.8. Ready To Transfer (R2T) 7519 Byte/ 0 | 1 | 2 | 3 | 7520 / | | | | 7521 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 7522 +---------------+---------------+---------------+---------------+ 7523 0|.|.| 0x31 |1| Reserved | 7524 +---------------+---------------+---------------+---------------+ 7525 4|TotalAHSLength | DataSegmentLength | 7526 +---------------+---------------+---------------+---------------+ 7527 8| LUN | 7528 + + 7529 12| | 7530 +---------------+---------------+---------------+---------------+ 7531 16| Initiator Task Tag | 7532 +---------------+---------------+---------------+---------------+ 7533 20| Target Transfer Tag | 7534 +---------------+---------------+---------------+---------------+ 7535 24| StatSN | 7536 +---------------+---------------+---------------+---------------+ 7537 28| ExpCmdSN | 7538 +---------------+---------------+---------------+---------------+ 7539 32| MaxCmdSN | 7540 +---------------+---------------+---------------+---------------+ 7541 36| R2TSN | 7542 +---------------+---------------+---------------+---------------+ 7543 40| Buffer Offset | 7544 +---------------+---------------+---------------+---------------+ 7545 44| Desired Data Transfer Length | 7546 +---------------------------------------------------------------+ 7547 48| Header-Digest (Optional) | 7548 +---------------+---------------+---------------+---------------+ 7550 When an initiator has submitted a SCSI Command with data that 7551 passes from the initiator to the target (WRITE), the target may 7552 specify which blocks of data it is ready to receive. The target may 7553 request that the data blocks be delivered in whichever order is 7554 convenient for the target at that particular instant. This 7555 information is passed from the target to the initiator in the Ready 7556 To Transfer (R2T) PDU. 7558 In order to allow write operations without an explicit initial R2T, 7559 the initiator and target MUST have negotiated the key InitialR2T to 7560 No during Login. 7562 An R2T MAY be answered with one or more SCSI Data-out PDUs with a 7563 matching Target Transfer Tag. If an R2T is answered with a single 7564 Data-out PDU, the Buffer Offset in the Data PDU MUST be the same as 7565 the one specified by the R2T, and the data length of the Data PDU 7566 MUST be the same as the Desired Data Transfer Length specified in 7567 the R2T. If the R2T is answered with a sequence of Data PDUs, the 7568 Buffer Offset and Length MUST be within the range of those 7569 specified by R2T, and the last PDU MUST have the F bit set to 1. If 7570 the last PDU (marked with the F bit) is received before the Desired 7571 Data Transfer Length is transferred, a target MAY choose to Reject 7572 that PDU with "Protocol error" reason code. DataPDUInOrder governs 7573 the Data-Out PDU ordering. If DataPDUInOrder is set to Yes, the 7574 Buffer Offsets and Lengths for consecutive PDUs MUST form a 7575 continuous non-overlapping range and the PDUs MUST be sent in 7576 increasing offset order. 7578 The target may send several R2T PDUs. It, therefore, can have a 7579 number of pending data transfers. The number of outstanding R2T 7580 PDUs are limited by the value of the negotiated key 7581 MaxOutstandingR2T. Within a task, outstanding R2Ts MUST be 7582 fulfilled by the initiator in the order in which they were 7583 received. 7585 R2T PDUs MAY also be used to recover Data Out PDUs. Such an R2T 7586 (Recovery-R2T) is generated by a target upon detecting the loss of 7587 one or more Data-Out PDUs due to: 7589 - Digest error 7591 - Sequence error 7593 - Sequence reception timeout 7595 A Recovery-R2T carries the next unused R2TSN, but requests part of 7596 or the entire data burst that an earlier R2T (with a lower R2TSN) 7597 had already requested. 7599 DataSequenceInOrder governs the buffer offset ordering in 7600 consecutive R2Ts. If DataSequenceInOrder is Yes, then consecutive 7601 R2Ts MUST refer to continuous non-overlapping ranges except for 7602 Recovery-R2Ts. 7604 10.8.1. TotalAHSLength and DataSegmentLength 7606 For this PDU TotalAHSLength and DataSegmentLength MUST be 0. 7608 10.8.2. R2TSN 7610 R2TSN is the R2T PDU input PDU number within the command identified 7611 by the Initiator Task Tag. 7613 For bidirectional commands R2T and Data-In PDUs share the input PDU 7614 numbering sequence (see Section 3.2.2.4 - "Data Sequencing"). 7616 10.8.3. StatSN 7618 The StatSN field will contain the next StatSN. The StatSN for this 7619 connection is not advanced after this PDU is sent. 7621 10.8.4. Desired Data Transfer Length and Buffer Offset 7623 The target specifies how many bytes it wants the initiator to send 7624 because of this R2T PDU. The target may request the data from the 7625 initiator in several chunks, not necessarily in the original order 7626 of the data. The target, therefore, also specifies a Buffer Offset 7627 that indicates the point at which the data transfer should begin, 7628 relative to the beginning of the total data transfer. The Desired 7629 Data Transfer Length MUST NOT be 0 and MUST NOT exceed 7630 MaxBurstLength. 7632 10.8.5. Target Transfer Tag 7634 The target assigns its own tag to each R2T request that it sends to 7635 the initiator. This tag can be used by the target to easily 7636 identify the data it receives. The Target Transfer Tag and LUN are 7637 copied in the outgoing data PDUs and are only used by the target. 7638 There is no protocol rule about the Target Transfer Tag except that 7639 the value 0xffffffff is reserved and MUST NOT be sent by a target 7640 in an R2T. 7642 10.9. Asynchronous Message 7644 An Asynchronous Message may be sent from the target to the 7645 initiator without correspondence to a particular command. The 7646 target specifies the reason for the event and sense data. 7648 Byte/ 0 | 1 | 2 | 3 | 7649 / | | | | 7650 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 7651 +---------------+---------------+---------------+---------------+ 7652 0|.|.| 0x32 |1| Reserved | 7653 +---------------+---------------+---------------+---------------+ 7654 4|TotalAHSLength | DataSegmentLength | 7655 +---------------+---------------+---------------+---------------+ 7656 8| LUN or Reserved | 7657 + + 7658 12| | 7659 +---------------+---------------+---------------+---------------+ 7660 16| 0xffffffff | 7661 +---------------+---------------+---------------+---------------+ 7662 20| Reserved | 7663 +---------------+---------------+---------------+---------------+ 7664 24| StatSN | 7665 +---------------+---------------+---------------+---------------+ 7666 28| ExpCmdSN | 7667 +---------------+---------------+---------------+---------------+ 7668 32| MaxCmdSN | 7669 +---------------+---------------+---------------+---------------+ 7670 36| AsyncEvent | AsyncVCode | Parameter1 or Reserved | 7671 +---------------+---------------+---------------+---------------+ 7672 40| Parameter2 or Reserved | Parameter3 or Reserved | 7673 +---------------+---------------+---------------+---------------+ 7674 44| Reserved | 7675 +---------------+---------------+---------------+---------------+ 7676 48| Header-Digest (Optional) | 7677 +---------------+---------------+---------------+---------------+ 7678 / DataSegment - Sense Data and iSCSI Event Data / 7679 +/ / 7680 +---------------+---------------+---------------+---------------+ 7681 | Data-Digest (Optional) | 7682 +---------------+---------------+---------------+---------------+ 7684 Some Asynchronous Messages are strictly related to iSCSI while 7685 others are related to SCSI [SAM2]. 7687 StatSN counts this PDU as an acknowledgeable event (StatSN is 7688 advanced), which allows for initiator and target state 7689 synchronization. 7691 10.9.1. AsyncEvent 7693 The codes used for iSCSI Asynchronous Messages (events) are: 7695 0 (SCSI_ASYNC) - a SCSI Asynchronous Event is reported in the 7696 sense data. Sense Data that accompanies the report, in the 7697 data segment, identifies the condition. The sending of a SCSI 7698 Event (Asynchronous Event Reporting in SCSI terminology) is 7699 dependent on the target support for SCSI asynchronous event 7700 reporting (see [SAM2]) as indicated in the standard INQUIRY 7701 data (see [SPC3]). Its use may be enabled by parameters in 7702 the SCSI Control mode page (see [SPC3]). 7704 1 (REQUEST_LOGOUT) - target requests Logout. This Async Message 7705 MUST be sent on the same connection as the one requesting to 7706 be logged out. The initiator MUST honor this request by 7707 issuing a Logout as early as possible, but no later than 7708 Parameter3 seconds. Initiator MUST send a Logout with a 7709 reason code of "Close the connection" OR "Close the session" 7710 to close all the connections. Once this message is received, 7711 the initiator SHOULD NOT issue new iSCSI commands on the 7712 connection to be logged out. The target MAY reject any new 7713 I/O requests that it receives after this Message with the 7714 reason code "Waiting for Logout". If the initiator does not 7715 Logout in Parameter3 seconds, the target should send an Async 7716 PDU with iSCSI event code "Dropped the connection" if 7717 possible, or simply terminate the transport connection. 7718 Parameter1 and Parameter2 are reserved. 7720 2 (CONNECTION_DROP) - target indicates it will drop the 7721 connection. 7722 The Parameter1 field indicates the CID of the connection 7723 going to be dropped. 7725 The Parameter2 field (Time2Wait) indicates, in seconds, the 7727 minimum time to wait before attempting to reconnect or 7728 reassign. 7730 The Parameter3 field (Time2Retain) indicates the maximum time 7731 allowed to reassign commands after the initial wait (in 7732 Parameter2). 7734 If the initiator does not attempt to reconnect and/or 7735 reassign the outstanding commands within the time specified 7736 by Parameter3, or if Parameter3 is 0, the target will 7737 terminate all outstanding commands on this connection. In 7738 this case, no other responses should be expected from the 7739 target for the outstanding commands on this connection. 7741 A value of 0 for Parameter2 indicates that reconnect can be 7742 attempted immediately. 7744 3 (SESSION_DROP) - target indicates it will drop all the 7745 connections of this session. 7747 Parameter1 field is reserved. 7749 The Parameter2 field (Time2Wait) indicates, in seconds, the 7750 minimum time to wait before attempting to reconnect. 7751 The Parameter3 field (Time2Retain) indicates the maximum time 7752 allowed to reassign commands after the initial wait (in 7753 Parameter2). 7755 If the initiator does not attempt to reconnect and/or 7756 reassign the outstanding commands within the time specified 7757 by Parameter3, or if Parameter3 is 0, the session is 7758 terminated. In this case, the target will terminate all 7759 outstanding commands in this session; no other responses 7760 should be expected from the target for the outstanding 7761 commands in this session. A value of 0 for Parameter2 7762 indicates that reconnect can be attempted immediately. 7764 4 (RENEGOTIATE) - target requests parameter negotiation on this 7765 connection. The initiator MUST honor this request by issuing 7766 a Text Request (that can be empty) on the same connection as 7767 early as possible, but no later than Parameter3 seconds, 7768 unless a Text Request is already pending on the connection, 7769 or by issuing a Logout Request. If the initiator does not 7770 issue a Text Request the target may reissue the Asynchronous 7771 Message requesting parameter negotiation. 7773 5 (FAST_ABORT) - all active tasks for LU with a matching LUN 7774 field in the Async Message PDU are being terminated. The 7775 receiving initiator iSCSI layer MUST respond to this Message 7776 by taking the following steps in order. 7778 - Stop Data-Out transfers on that connection for all active 7779 TTTs for the affected LUN quoted in the Async Message PDU. 7780 - Acknowledge the StatSN of the Async Message PDU via a NOP- 7781 Out PDU with ITT=0xffffffff (i.e., non-ping flavor), while 7782 copying the LUN field from the Async Message to NOP-Out. 7784 This value of AsyncEvent however MUST NOT be used on an iSCSI 7785 session unless the new TaskReporting text key defined in 7786 Section 12.23 was negotiated to FastAbort on the session. 7788 255 vendor-specific iSCSI Event. The AsyncVCode details the 7789 vendor code, and data MAY accompany the report. 7791 All other event codes are reserved. 7793 10.9.2. AsyncVCode 7795 AsyncVCode is a vendor specific detail code that is only valid if 7796 the AsyncEvent field indicates a vendor specific event. Otherwise, 7797 it is reserved. 7799 10.9.3. LUN 7801 The LUN field MUST be valid if AsyncEvent is 0. Otherwise, this 7802 field is reserved. 7803 10.9.4. Sense Data and iSCSI Event Data 7805 For a SCSI event, this data accompanies the report in the data 7806 segment and identifies the condition. 7808 For an iSCSI event, additional vendor-unique data MAY accompany the 7809 Async event. Initiators MAY ignore the data when not understood 7810 while processing the rest of the PDU. 7812 If the DataSegmentLength is not 0, the format of the DataSegment is 7813 as follows: 7814 Byte/ 0 | 1 | 2 | 3 | 7815 / | | | | 7816 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 7817 +---------------+---------------+---------------+---------------+ 7818 0|SenseLength | Sense Data | 7819 +---------------+---------------+---------------+---------------+ 7820 x/ Sense Data / 7821 +---------------+---------------+---------------+---------------+ 7822 y/ iSCSI Event Data / 7823 / / 7824 +---------------+---------------+---------------+---------------+ 7825 z| 7827 10.9.4.1. SenseLength 7829 This is the length of Sense Data. When the Sense Data field is 7830 empty (e.g., the event is not a SCSI event) SenseLength is 0. 7832 10.10. Text Request 7834 The Text Request is provided to allow for the exchange of 7835 information and for future extensions. It permits the initiator to 7836 inform a target of its capabilities or to request some special 7837 operations. 7839 Byte/ 0 | 1 | 2 | 3 | 7840 / | | | | 7841 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 7842 +---------------+---------------+---------------+---------------+ 7843 0|.|I| 0x04 |F|C| Reserved | 7844 +---------------+---------------+---------------+---------------+ 7845 4|TotalAHSLength | DataSegmentLength | 7846 +---------------+---------------+---------------+---------------+ 7847 8| LUN or Reserved | 7848 + + 7849 12| | 7850 +---------------+---------------+---------------+---------------+ 7851 16| Initiator Task Tag | 7852 +---------------+---------------+---------------+---------------+ 7853 20| Target Transfer Tag or 0xffffffff | 7854 +---------------+---------------+---------------+---------------+ 7855 24| CmdSN | 7856 +---------------+---------------+---------------+---------------+ 7857 28| ExpStatSN | 7858 +---------------+---------------+---------------+---------------+ 7859 32/ Reserved / 7860 +/ / 7861 +---------------+---------------+---------------+---------------+ 7862 48| Header-Digest (Optional) | 7863 +---------------+---------------+---------------+---------------+ 7864 / DataSegment (Text) / 7865 +/ / 7866 +---------------+---------------+---------------+---------------+ 7867 | Data-Digest (Optional) | 7868 +---------------+---------------+---------------+---------------+ 7870 An initiator MUST NOT have more than one outstanding Text Request 7871 on a connection at any given time. 7873 On a connection failure, an initiator must either explicitly abort 7874 any active allegiant text negotiation task or must cause such a 7875 task to be implicitly terminated by the target. 7877 10.10.1. F (Final) Bit 7879 When set to 1, indicates that this is the last or only text 7880 request in a sequence of Text Requests; otherwise, it indicates 7881 that more Text Requests will follow. 7883 10.10.2. C (Continue) Bit 7885 When set to 1, indicates that the text (set of key=value pairs) in 7886 this Text Request is not complete (it will be continued on 7887 subsequent Text Requests); otherwise, it indicates that this Text 7888 Request ends a set of key=value pairs. A Text Request with the C 7889 bit set to 1 MUST have the F bit set to 0. 7891 10.10.3. Initiator Task Tag 7893 The initiator assigned identifier for this Text Request. If the 7894 command is sent as part of a sequence of text requests and 7895 responses, the Initiator Task Tag MUST be the same for all the 7896 requests within the sequence (similar to linked SCSI commands). The 7897 I bit for all requests in a sequence also MUST be the same. 7899 10.10.4. Target Transfer Tag 7901 When the Target Transfer Tag is set to the reserved value 7902 0xffffffff, it tells the target that this is a new request and the 7903 target resets any internal state associated with the Initiator Task 7904 Tag (resets the current negotiation state). 7906 The target sets the Target Transfer Tag in a text response to a 7907 value other than the reserved value 0xffffffff whenever it 7908 indicates that it has more data to send or more operations to 7909 perform that are associated with the specified Initiator Task Tag. 7910 It MUST do so whenever it sets the F bit to 0 in the response. By 7911 copying the Target Transfer Tag from the response to the next Text 7912 Request, the initiator tells the target to continue the operation 7913 for the specific Initiator Task Tag. The initiator MUST ignore the 7914 Target Transfer Tag in the Text Response when the F bit is set to 7915 1. 7917 This mechanism allows the initiator and target to transfer a large 7918 amount of textual data over a sequence of text-command/text- 7919 response exchanges, or to perform extended negotiation sequences. 7921 If the Target Transfer Tag is not 0xffffffff, the LUN field MUST be 7922 sent by the target in the Text Response. 7924 A target MAY reset its internal negotiation state if an exchange is 7925 stalled by the initiator for a long time or if it is running out of 7926 resources. 7928 Long text responses are handled as in the following example: 7930 I->T Text SendTargets=All (F=1,TTT=0xffffffff) 7932 T->I Text (F=0,TTT=0x12345678) 7934 I->T Text (F=1, TTT=0x12345678) 7936 T->I Text (F=0, TTT=0x12345678) 7938 I->T Text (F=1, TTT=0x12345678) 7940 ... 7942 T->I Text (F=1, TTT=0xffffffff) 7944 10.10.5. Text 7946 The data lengths of a text request MUST NOT exceed the iSCSI target 7947 MaxRecvDataSegmentLength (a per connection and per direction 7948 negotiated parameter). The text format is specified in Section 5.2 7949 - "Text Mode Negotiation". 7951 Chapter 11 and Chapter 12 list some basic Text key=value pairs, 7952 some of which can be used in Login Request/Response and some in 7953 Text Request/Response. 7955 A key=value pair can span Text request or response boundaries. A 7956 key=value pair can start in one PDU and continue on the next. In 7957 other words the end of a PDU does not necessarily signal the end of 7958 a key=value pair. 7960 The target responds by sending its response back to the initiator. 7961 The response text format is similar to the request text format. 7962 The text response MAY refer to key=value pairs presented in an 7963 earlier text request and the text in the request may refer to 7964 earlier responses. 7966 Chapter 5 details the rules for the Text Requests and Responses. 7968 Text operations are usually meant for parameter 7969 setting/negotiations, but can also be used to perform some long 7970 lasting operations. 7972 Text operations that take a long time should be placed in their own 7973 Text request. 7975 10.11. Text Response 7977 The Text Response PDU contains the target's responses to the 7978 initiator's Text request. The format of the Text field matches that 7979 of the Text request. 7981 Byte/ 0 | 1 | 2 | 3 | 7982 / | | | | 7983 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 7984 +---------------+---------------+---------------+---------------+ 7985 0|.|.| 0x24 |F|C| Reserved | 7986 +---------------+---------------+---------------+---------------+ 7987 4|TotalAHSLength | DataSegmentLength | 7988 +---------------+---------------+---------------+---------------+ 7989 8| LUN or Reserved | 7990 + + 7991 12| | 7992 +---------------+---------------+---------------+---------------+ 7993 16| Initiator Task Tag | 7994 +---------------+---------------+---------------+---------------+ 7995 20| Target Transfer Tag or 0xffffffff | 7996 +---------------+---------------+---------------+---------------+ 7997 24| StatSN | 7998 +---------------+---------------+---------------+---------------+ 7999 28| ExpCmdSN | 8000 +---------------+---------------+---------------+---------------+ 8001 32| MaxCmdSN | 8002 +---------------+---------------+---------------+---------------+ 8003 36/ Reserved / 8004 +/ / 8005 +---------------+---------------+---------------+---------------+ 8006 48| Header-Digest (Optional) | 8007 +---------------+---------------+---------------+---------------+ 8008 / DataSegment (Text) / 8009 +/ / 8010 +---------------+---------------+---------------+---------------+ 8011 | Data-Digest (Optional) | 8012 +---------------+---------------+---------------+---------------+ 8014 10.11.1. F (Final) Bit 8016 When set to 1, in response to a Text Request with the Final bit set 8017 to 1, the F bit indicates that the target has finished the whole 8018 operation. Otherwise, if set to 0 in response to a Text Request 8019 with the Final Bit set to 1, it indicates that the target has more 8020 work to do (invites a follow-on text request). A Text Response with 8021 the F bit set to 1 in response to a Text Request with the F bit set 8022 to 0 is a protocol error. 8024 A Text Response with the F bit set to 1 MUST NOT contain key=value 8025 pairs that may require additional answers from the initiator. 8027 A Text Response with the F bit set to 1 MUST have a Target Transfer 8028 Tag field set to the reserved value of 0xffffffff. 8030 A Text Response with the F bit set to 0 MUST have a Target Transfer 8031 Tag field set to a value other than the reserved 0xffffffff. 8033 10.11.2. C (Continue) Bit 8035 When set to 1, indicates that the text (set of key=value pairs) in 8036 this Text Response is not complete (it will be continued on 8037 subsequent Text Responses); otherwise, it indicates that this Text 8038 Response ends a set of key=value pairs. A Text Response with the C 8039 bit set to 1 MUST have the F bit set to 0. 8041 10.11.3. Initiator Task Tag 8043 The Initiator Task Tag matches the tag used in the initial Text 8044 Request. 8046 10.11.4. Target Transfer Tag 8048 When a target has more work to do (e.g., cannot transfer all the 8049 remaining text data in a single Text Response or has to continue 8050 the negotiation) and has enough resources to proceed, it MUST set 8051 the Target Transfer Tag to a value other than the reserved value of 8052 0xffffffff. Otherwise, the Target Transfer Tag MUST be set to 8053 0xffffffff. 8055 When the Target Transfer Tag is not 0xffffffff, the LUN field may 8056 be significant. 8058 The initiator MUST copy the Target Transfer Tag and LUN in its next 8059 request to indicate that it wants the rest of the data. 8061 When the target receives a Text Request with the Target Transfer 8062 Tag set to the reserved value of 0xffffffff, it resets its internal 8063 information (resets state) associated with the given Initiator Task 8064 Tag (restarts the negotiation). 8066 When a target cannot finish the operation in a single Text 8067 Response, and does not have enough resources to continue, it 8068 rejects the Text Request with the appropriate Reject code. 8070 A target may reset its internal state associated with an Initiator 8071 Task Tag (the current negotiation state), state expressed through 8072 the Target Transfer Tag if the initiator fails to continue the 8073 exchange for some time. The target may reject subsequent Text 8074 Requests with the Target Transfer Tag set to the "stale" value. 8076 10.11.5. StatSN 8078 The target StatSN variable is advanced by each Text Response sent. 8080 10.11.6. Text Response Data 8082 The data lengths of a text response MUST NOT exceed the iSCSI 8083 initiator MaxRecvDataSegmentLength (a per connection and per 8084 direction negotiated parameter). 8086 The text in the Text Response Data is governed by the same rules as 8087 the text in the Text Request Data (see C (Con). 8089 Although the initiator is the requesting party and controls the 8090 request-response initiation and termination, the target can offer 8091 key=value pairs of its own as part of a sequence and not only in 8092 response to the initiator. 8094 10.12. Login Request 8096 After establishing a TCP connection between an initiator and a 8097 target, the initiator MUST start a Login Phase to gain further 8098 access to the target's resources. 8100 The Login Phase (see Chapter 5) consists of a sequence of Login 8101 requests and responses that carry the same Initiator Task Tag. 8103 Login requests are always considered as immediate. 8105 Byte/ 0 | 1 | 2 | 3 | 8106 / | | | | 8107 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 8108 +---------------+---------------+---------------+---------------+ 8109 0|.|1| 0x03 |T|C|.|.|CSG|NSG| Version-max | Version-min | 8110 +---------------+---------------+---------------+---------------+ 8111 4|TotalAHSLength | DataSegmentLength | 8112 +---------------+---------------+---------------+---------------+ 8113 8| ISID | 8114 + +---------------+---------------+ 8115 12| | TSIH | 8116 +---------------+---------------+---------------+---------------+ 8117 16| Initiator Task Tag | 8118 +---------------+---------------+---------------+---------------+ 8119 20| CID | Reserved | 8120 +---------------+---------------+---------------+---------------+ 8121 24| CmdSN | 8122 +---------------+---------------+---------------+---------------+ 8123 28| ExpStatSN or Reserved | 8124 +---------------+---------------+---------------+---------------+ 8125 32| Reserved | 8126 +---------------+---------------+---------------+---------------+ 8127 36| Reserved | 8128 +---------------+---------------+---------------+---------------+ 8129 40/ Reserved / 8130 +/ / 8131 +---------------+---------------+---------------+---------------+ 8132 48/ DataSegment - Login Parameters in Text request Format / 8133 +/ / 8134 +---------------+---------------+---------------+---------------+ 8136 10.12.1. T (Transit) Bit 8138 If set to 1, indicates that the initiator is ready to transit to 8139 the next stage. 8141 If the T bit is set to 1 and NSG is FullFeaturePhase, then this 8142 also indicates that the initiator is ready for the Final Login 8143 Response (see Chapter 5). 8145 10.12.2. C (Continue) Bit 8147 When set to 1, indicates that the text (set of key=value pairs) in 8148 this Login Request is not complete (it will be continued on 8149 subsequent Login Requests); otherwise, it indicates that this Login 8150 Request ends a set of key=value pairs. A Login Request with the C 8151 bit set to 1 MUST have the T bit set to 0. 8153 10.12.3. CSG and NSG 8155 Through these fields, Current Stage (CSG) and Next Stage (NSG), the 8156 Login negotiation requests and responses are associated with a 8157 specific stage in the session (SecurityNegotiation, 8158 LoginOperationalNegotiation, FullFeaturePhase) and may indicate the 8159 next stage to which they want to move (see Chapter 5). The next 8160 stage value is only valid when the T bit is 1; otherwise, it is 8161 reserved. 8163 The stage codes are: 8165 - 0 - SecurityNegotiation 8167 - 1 - LoginOperationalNegotiation 8169 - 3 - FullFeaturePhase 8171 All other codes are reserved. 8173 10.12.4. Version 8175 The version number of the current draft is 0x00. As such, all 8176 devices MUST carry version 0x00 for both Version-min and Version- 8177 max. 8179 10.12.4.1. Version-max 8181 Maximum Version number supported. 8183 All Login requests within the Login Phase MUST carry the same 8184 Version-max. 8186 The target MUST use the value presented with the first login 8187 request. 8189 10.12.4.2. Version-min 8191 All Login requests within the Login Phase MUST carry the same 8192 Version-min. The target MUST use the value presented with the first 8193 login request. 8195 10.12.5. ISID 8197 This is an initiator-defined component of the session identifier 8198 and is structured as follows (see Section 9.1.1 for details): 8200 Byte/ 0 | 1 | 2 | 3 | 8201 / | | | | 8202 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 8203 +---------------+---------------+---------------+---------------+ 8204 8| T | A | B | C | 8205 +---------------+---------------+---------------+---------------+ 8206 12| D | 8207 +---------------+---------------+ 8209 The T field identifies the format and usage of A, B, C, and D as 8210 indicated below: 8212 T 8214 00b OUI-Format 8216 A&B are a 22 bit OUI 8218 (the I/G & U/L bits are omitted) 8220 C&D 24 bit qualifier 8222 01b EN - Format (IANA Enterprise Number) 8224 A - Reserved 8226 B&C EN (IANA Enterprise Number) 8228 D - Qualifier 8230 10b "Random" 8232 A - Reserved 8234 B&C Random 8236 D - Qualifier 8238 11b A,B,C&D Reserved 8240 For the T field values 00b and 01b, a combination of A and B (for 8241 00b) or B and C (for 01b) identifies the vendor or organization 8242 whose component (software or hardware) generates this ISID. A 8243 vendor or organization with one or more OUIs, or one or more 8244 Enterprise Numbers, MUST use at least one of these numbers and 8245 select the appropriate value for the T field when its components 8246 generate ISIDs. An OUI or EN MUST be set in the corresponding 8247 fields in network byte order (byte big-endian). 8249 If the T field is 10b, B and C are set to a random 24-bit unsigned 8250 integer value in network byte order (byte big-endian). See 8251 [RFC3721] for how this affects the principle of "conservative 8252 reuse". 8254 The Qualifier field is a 16 or 24-bit unsigned integer value that 8255 provides a range of possible values for the ISID within the 8256 selected namespace. It may be set to any value within the 8257 constraints specified in the iSCSI protocol (see Section 3.4.3 - 8258 "Consequences of the Model" and Section 9.1.1 - "Conservative Reuse 8259 of ISIDs"). 8261 The T field value of 11b is reserved. 8263 If the ISID is derived from something assigned to a hardware 8264 adapter or interface by a vendor, as a preset default value, it 8265 MUST be configurable to a value assigned according to the SCSI port 8266 behavior desired by the system in which it is installed (see 8267 Section 9.1.1 - "Conservative Reuse of ISIDs" and Section 9.1.2 - 8268 "iSCSI Name, ISID, and TPGT Use"). The resultant ISID MUST also be 8269 persistent over power cycles, reboot, card swap, etc. 8271 10.12.6. TSIH 8273 TSIH must be set in the first Login Request. The reserved value 0 8274 MUST be used on the first connection for a new session. Otherwise, 8275 the TSIH sent by the target at the conclusion of the successful 8276 login of the first connection for this session MUST be used. The 8277 TSIH identifies to the target the associated existing session for 8278 this new connection. 8280 All Login requests within a Login Phase MUST carry the same TSIH. 8282 The target MUST check the value presented with the first login 8283 request and act as specified in Section 5.3.1 - "Login Phase 8284 Start". 8286 10.12.7. Connection ID - CID 8288 A unique ID for this connection within the session. 8290 All Login requests within the Login Phase MUST carry the same CID. 8292 The target MUST use the value presented with the first login 8293 request. 8295 A Login request with a non-zero TSIH and a CID equal to that of an 8296 existing connection implies a logout of the connection followed by 8297 a Login (see Section 5.3.4). For the details of the implicit Logout 8298 Request, see Section 10.14. 8300 10.12.8. CmdSN 8302 CmdSN is either the initial command sequence number of a session 8303 (for the first Login request of a session - the "leading" login), 8304 or the command sequence number in the command stream if the login 8305 is for a new connection in an existing session. 8307 Examples: 8309 - Login on a leading connection - if the leading login carries 8310 the CmdSN 123, all other login requests in the same login 8311 phase carry the CmdSN 123 and the first non-immediate command 8312 in FullFeaturePhase also carries the CmdSN 123. 8314 - Login on other than a leading connection - if the current 8315 CmdSN at the time the first login on the connection is issued 8316 is 500, then that PDU carries CmdSN=500. Subsequent login 8317 requests that are needed to complete this login phase may 8318 carry a CmdSN higher than 500 if non-immediate requests that 8319 were issued on other connections in the same session advance 8320 CmdSN. 8322 If the login request is a leading login request, the target MUST 8323 use the value presented in CmdSN as the target value for ExpCmdSN. 8325 10.12.9. ExpStatSN 8327 For the first Login Request on a connection this is ExpStatSN for 8328 the old connection and this field is only valid if the Login 8329 request restarts a connection (see Section 5.3.4 - "Connection 8330 Reinstatement"). 8332 For subsequent Login Requests it is used to acknowledge the Login 8333 Responses with their increasing StatSN values. 8335 10.12.10. Login Parameters 8337 The initiator MUST provide some basic parameters in order to enable 8338 the target to determine if the initiator may use the target's 8339 resources and the initial text parameters for the security 8340 exchange. 8342 All the rules specified in Section 10.10.5 for text requests also 8343 hold for login requests. Keys and their explanations are listed in 8344 Chapter 11 (security negotiation keys) and Section 12 (operational 8345 parameter negotiation keys). All keys in Section 12, except for the 8346 X extension formats, MUST be supported by iSCSI initiators and 8347 targets. Keys in Section 11 only need to be supported when the 8348 function to which they refer is mandatory to implement. 8350 10.13. Login Response 8352 The Login Response indicates the progress and/or end of the Login 8353 Phase. 8355 Byte/ 0 | 1 | 2 | 3 | 8356 / | | | | 8357 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 8358 +---------------+---------------+---------------+---------------+ 8359 0|.|.| 0x23 |T|C|.|.|CSG|NSG| Version-max | Version-active| 8360 +---------------+---------------+---------------+---------------+ 8361 4|TotalAHSLength | DataSegmentLength | 8362 +---------------+---------------+---------------+---------------+ 8363 8| ISID | 8364 + +---------------+---------------+ 8365 12| | TSIH | 8366 +---------------+---------------+---------------+---------------+ 8367 16| Initiator Task Tag | 8368 +---------------+---------------+---------------+---------------+ 8369 20| Reserved | 8370 +---------------+---------------+---------------+---------------+ 8371 24| StatSN | 8372 +---------------+---------------+---------------+---------------+ 8373 28| ExpCmdSN | 8374 +---------------+---------------+---------------+---------------+ 8375 32| MaxCmdSN | 8376 +---------------+---------------+---------------+---------------+ 8377 36| Status-Class | Status-Detail | Reserved | 8378 +---------------+---------------+---------------+---------------+ 8379 40/ Reserved / 8380 +/ / 8381 +---------------+---------------+---------------+---------------+ 8382 48/ DataSegment - Login Parameters in Text request Format / 8383 +/ / 8384 +---------------+---------------+---------------+---------------+ 8386 10.13.1. Version-max 8388 This is the highest version number supported by the target. 8390 All Login responses within the Login Phase MUST carry the same 8391 Version-max. 8393 The initiator MUST use the value presented as a response to the 8394 first login request. 8396 10.13.2. Version-active 8398 Indicates the highest version supported by the target and 8399 initiator. If the target does not support a version within the 8400 range specified by the initiator, the target rejects the login and 8401 this field indicates the lowest version supported by the target. 8403 All Login responses within the Login Phase MUST carry the same 8404 Version-active. 8406 The initiator MUST use the value presented as a response to the 8407 first login request. 8409 10.13.3. TSIH 8411 The TSIH is the target assigned session identifying handle. Its 8412 internal format and content are not defined by this protocol except 8413 for the value 0 that is reserved. With the exception of the Login 8414 Final-Response in a new session, this field should be set to the 8415 TSIH provided by the initiator in the Login Request. For a new 8416 session, the target MUST generate a non-zero TSIH and ONLY return 8417 it in the Login Final-Response (see Section 5.3 - "Login Phase"). 8419 10.13.4. StatSN 8421 For the first Login Response (the response to the first Login 8422 Request), this is the starting status Sequence Number for the 8423 connection. The next response of any kind, including the next login 8424 response, if any, in the same Login Phase, will carry this number + 8425 1. This field is only valid if the Status-Class is 0. 8427 10.13.5. Status-Class and Status-Detail 8429 The Status returned in a Login Response indicates the execution 8430 status of the Login Phase. The status includes: 8432 Status-Class 8434 Status-Detail 8436 0 Status-Class indicates success. 8438 A non-zero Status-Class indicates an exception. In this case, 8439 Status-Class is sufficient for a simple initiator to use when 8440 handling exceptions, without having to look at the Status-Detail. 8441 The Status-Detail allows finer-grained exception handling for more 8442 sophisticated initiators and for better information for logging. 8444 The status classes are as follows: 8446 0 - Success - indicates that the iSCSI target successfully 8447 received, understood, and accepted the request. The numbering 8448 fields (StatSN, ExpCmdSN, MaxCmdSN) are only valid if Status- 8449 Class is 0. 8451 1 - Redirection - indicates that the initiator must take 8452 further action to complete the request. This is usually due 8453 to the target moving to a different address. All of the 8454 redirection status class responses MUST return one or more 8455 text key parameters of the type "TargetAddress", which 8456 indicates the target's new address. A redirection response 8457 MAY be issued by a target prior or after completing a 8458 security negotiation if a security negotiation is required. A 8459 redirection SHOULD be accepted by an initiator even without 8460 having the target complete a security negotiation if any 8461 security negotiation is required, and MUST be accepted by the 8462 initiator after the completion of the security negotiation if 8463 any security negotiation is required. 8465 2 - Initiator Error (not a format error) - indicates that the 8466 initiator most likely caused the error. This MAY be due to a 8467 request for a resource for which the initiator does not have 8468 permission. The request should not be tried again. 8470 3 - Target Error - indicates that the target sees no errors in 8471 the initiator's login request, but is currently incapable of 8473 fulfilling the request. The initiator may re-try the same 8474 login request later. 8476 The table below shows all of the currently allocated status codes. 8477 The codes are in hexadecimal; the first byte is the status class 8478 and the second byte is the status detail. 8480 ----------------------------------------------------------------- 8481 Status | Code | Description 8482 |(hex) | 8483 ----------------------------------------------------------------- 8484 Success | 0000 | Login is proceeding OK (*1). 8485 ----------------------------------------------------------------- 8486 Target moved | 0101 | The requested iSCSI Target Name (ITN) 8487 temporarily | | has temporarily moved 8488 | | to the address provided. 8489 ----------------------------------------------------------------- 8490 Target moved | 0102 | The requested ITN has permanently moved 8491 permanently | | to the address provided. 8492 ----------------------------------------------------------------- 8493 Initiator | 0200 | Miscellaneous iSCSI initiator 8494 error | | errors. 8495 ---------------------------------------------------------------- 8496 Authentication| 0201 | The initiator could not be 8497 failure | | successfully authenticated or target 8498 | | authentication is not supported. 8499 ----------------------------------------------------------------- 8500 Authorization | 0202 | The initiator is not allowed access 8501 failure | | to the given target. 8502 ----------------------------------------------------------------- 8503 Not found | 0203 | The requested ITN does not 8504 | | exist at this address. 8505 ----------------------------------------------------------------- 8506 Target removed| 0204 | The requested ITN has been removed and 8507 | |no forwarding address is provided. 8508 ----------------------------------------------------------------- 8509 Unsupported | 0205 | The requested iSCSI version range is 8510 version | | not supported by the target. 8511 ----------------------------------------------------------------- 8512 Too many | 0206 | Too many connections on this SSID. 8513 connections | | 8514 ----------------------------------------------------------------- 8515 Missing | 0207 | Missing parameters (e.g., iSCSI 8516 parameter | | Initiator and/or Target Name). 8517 ----------------------------------------------------------------- 8518 Can't include | 0208 | Target does not support session 8519 in session | | spanning to this connection (address). 8520 ----------------------------------------------------------------- 8521 Session type | 0209 | Target does not support this type of 8522 not supported | | of session or not from this Initiator. 8523 ----------------------------------------------------------------- 8524 Session does | 020a | Attempt to add a connection 8525 not exist | | to a non-existent session. 8526 ----------------------------------------------------------------- 8527 Invalid during| 020b | Invalid Request type during Login. 8528 login | | 8529 ----------------------------------------------------------------- 8530 Target error | 0300 | Target hardware or software error. 8531 ----------------------------------------------------------------- 8532 Service | 0301 | The iSCSI service or target is not 8533 unavailable | | currently operational. 8534 ----------------------------------------------------------------- 8535 Out of | 0302 | The target has insufficient session, 8536 resources | | connection, or other resources. 8537 ----------------------------------------------------------------- 8539 (*1)If the response T bit is 1 in both the request and the matching 8540 response, and the NSG is FullFeaturePhase in both the request and 8541 the matching response, the Login Phase is finished and the 8542 initiator may proceed to issue SCSI commands. 8544 If the Status Class is not 0, the initiator and target MUST close 8545 the TCP connection. 8547 If the target wishes to reject the login request for more than one 8548 reason, it should return the primary reason for the rejection. 8550 10.13.6. T (Transit) bit 8552 The T bit is set to 1 as an indicator of the end of the stage. If 8553 the T bit is set to 1 and NSG is FullFeaturePhase, then this is 8554 also the Final Login Response (see Chapter 5). A T bit of 0 8555 indicates a "partial" response, which means "more negotiation 8556 needed". 8558 A login response with a T bit set to 1 MUST NOT contain key=value 8559 pairs that may require additional answers from the initiator within 8560 the same stage. 8562 If the status class is 0, the T bit MUST NOT be set to 1 if the T 8563 bit in the request was set to 0. 8565 10.13.7. C (Continue) Bit 8567 When set to 1, indicates that the text (set of key=value pairs) in 8568 this Login Response is not complete (it will be continued on 8569 subsequent Login Responses); otherwise, it indicates that this 8570 Login Response ends a set of key=value pairs. A Login Response with 8571 the C bit set to 1 MUST have the T bit set to 0. 8573 10.13.8. Login Parameters 8575 The target MUST provide some basic parameters in order to enable 8576 the initiator to determine if it is connected to the correct port 8577 and the initial text parameters for the security exchange. 8579 All the rules specified in Section 10.11.6 for text responses also 8580 hold for login responses. Keys and their explanations are listed 8581 in Chapter 11 (security negotiation keys) and Chapter 12 8582 (operational parameter negotiation keys). All keys in Section 12, 8583 except for the X extension formats, MUST be supported by iSCSI 8584 initiators and targets. Keys in Section 11, only need to be 8585 supported when the function to which they refer is mandatory to 8586 implement. 8588 10.14. Logout Request 8590 The Logout request is used to perform a controlled closing of a 8591 connection. 8593 An initiator MAY use a logout request to remove a connection from a 8594 session or to close an entire session. 8596 After sending the Logout request PDU, an initiator MUST NOT send 8597 any new iSCSI requests on the closing connection. If the Logout 8598 request is intended to close the session, new iSCSI requests MUST 8599 NOT be sent on any of the connections participating in the session. 8601 When receiving a Logout request with the reason code of "close the 8602 connection" or "close the session", the target MUST terminate all 8603 pending commands, whether acknowledged via ExpCmdSN or not, on that 8604 connection or session respectively. 8606 When receiving a Logout request with the reason code "remove 8607 connection for recovery", the target MUST discard all requests not 8608 yet acknowledged via ExpCmdSN that were issued on the specified 8609 connection, and suspend all data/status/R2T transfers on behalf of 8610 pending commands on the specified connection. 8612 The target then issues the Logout response and half-closes the TCP 8613 connection (sends FIN). After receiving the Logout response and 8614 attempting to receive the FIN (if still possible), the initiator 8615 MUST completely close the logging-out connection. For the 8616 terminated commands, no additional responses should be expected. 8618 A Logout for a CID may be performed on a different transport 8619 connection when the TCP connection for the CID has already been 8620 terminated. In such a case, only a logical "closing" of the iSCSI 8621 connection for the CID is implied with a Logout. 8623 All commands that were not terminated or not completed (with 8624 status) and acknowledged when the connection is closed completely 8625 can be reassigned to a new connection if the target supports 8626 connection recovery. 8628 If an initiator intends to start recovery for a failing connection, 8629 it MUST use the Logout request to "clean-up" the target end of a 8630 failing connection and enable recovery to start, or the Login 8631 request with a non-zero TSIH and the same CID on a new connection 8632 for the same effect. In sessions with a single connection, the 8633 connection can be closed and then a new connection reopened. A 8634 connection reinstatement login can be used for recovery (see 8635 Section 5.3.4 - "Connection Reinstatement"). 8637 A successful completion of a logout request with the reason code of 8638 "close the connection" or "remove the connection for recovery" 8639 results at the target in the discarding of unacknowledged commands 8640 received on the connection being logged out. These are commands 8641 that have arrived on the connection being logged out, but have not 8642 been delivered to SCSI because one or more commands with a smaller 8643 CmdSN has not been received by iSCSI. See Section 3.2.2.1 - 8644 "Command Numbering and Acknowledging". The resulting holes in the 8645 command sequence numbers will have to be handled by appropriate 8646 recovery (see Chapter 6) unless the session is also closed. 8648 The entire logout discussion in this section is also applicable for 8649 an implicit Logout realized by way of a connection reinstatement or 8650 session reinstatement. When a Login Request performs an implicit 8651 Logout, the implicit Logout is performed as if having the reason 8652 codes specified below: 8654 Reason code Type of implicit Logout 8656 ------------------------------------------- 8658 0 session reinstatement 8660 1 connection reinstatement when the operational 8661 ErrorRecoveryLevel < 2 8663 2 connection reinstatement when the operational 8664 ErrorRecoveryLevel = 2 8666 Byte/ 0 | 1 | 2 | 3 | 8667 / | | | | 8668 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 8669 +---------------+---------------+---------------+---------------+ 8670 0|.|I| 0x06 |1| Reason Code | Reserved | 8671 +---------------+---------------+---------------+---------------+ 8672 4|TotalAHSLength | DataSegmentLength | 8673 +---------------------------------------------------------------+ 8674 8/ Reserved / 8675 +/ / 8676 +---------------+---------------+---------------+---------------+ 8677 16| Initiator Task Tag | 8678 +---------------+---------------+---------------+---------------+ 8679 20| CID or Reserved | Reserved | 8680 +---------------+---------------+---------------+---------------+ 8681 24| CmdSN | 8682 +---------------+---------------+---------------+---------------+ 8683 28| ExpStatSN | 8684 +---------------+---------------+---------------+---------------+ 8685 32/ Reserved / 8686 +/ / 8687 +---------------+---------------+---------------+---------------+ 8688 48| Header-Digest (Optional) | 8689 +---------------+---------------+---------------+---------------+ 8691 10.14.1. Reason Code 8693 Reason Code indicates the reason for Logout as follows: 8695 0 - close the session. All commands associated with the session 8696 (if any) are terminated. 8698 1 - close the connection. All commands associated with 8699 connection (if any) are terminated. 8701 2 - remove the connection for recovery. Connection is closed 8702 and all commands associated with it, if any, are to be 8703 prepared for a new allegiance. 8705 All other values are reserved. 8707 10.14.2. TotalAHSLength and DataSegmentLength 8709 For this PDU TotalAHSLength and DataSegmentLength MUST be 0. 8711 10.14.3. CID 8713 This is the connection ID of the connection to be closed (including 8714 closing the TCP stream). This field is only valid if the reason 8715 code is not "close the session". 8717 10.14.4. ExpStatSN 8719 This is the last ExpStatSN value for the connection to be closed. 8721 10.14.5. Implicit termination of tasks 8723 A target implicitly terminates the active tasks due to the iSCSI 8724 protocol in the following cases: 8726 When a connection is implicitly or explicitly logged out with 8727 the reason code of "Close the connection" and there are 8728 active tasks allegiant to that connection. 8730 When a connection fails and eventually the connection state 8731 times out (state transition M1 in Section 7.2.2 - "State 8732 Transition Descriptions for Initiators and Targets") and 8733 there are active tasks allegiant to that connection. 8735 When a successful recovery Logout is performed while there are 8736 active tasks allegiant to that connection, and those tasks 8737 eventually time out after the Time2Wait and Time2Retain 8738 periods without allegiance reassignment. 8740 When a connection is implicitly or explicitly logged out with 8741 the reason code of "Close the session" and there are active 8742 tasks in that session. 8744 If the tasks terminated in any of the above cases are SCSI tasks, 8745 they must be internally terminated as if with CHECK CONDITION 8746 status. This status is only meaningful for appropriately handling 8747 the internal SCSI state and SCSI side effects with respect to 8748 ordering because this status is never communicated back as a 8749 terminating status to the initiator. However additional actions may 8750 have to be taken at SCSI level depending on the SCSI context as 8751 defined by the SCSI standards (e.g., queued commands and ACA, UA 8752 for the next command on the I_T nexus in cases a), b), and c), 8753 after the tasks are terminated, the target MUST report a Unit 8754 Attention condition on the next command processed on any connection 8755 for each affected I_T_L nexus with the status of CHECK CONDITION, 8756 and the ASC/ASCQ value of 47h/7Fh - "SOME COMMANDS CLEARED BY ISCSI 8757 PROTOCOL EVENT" - etc. - see [SAM4] and [SPC3]). 8759 10.15. Logout Response 8761 The logout response is used by the target to indicate if the 8762 cleanup operation for the connection(s) has completed. 8764 After Logout, the TCP connection referred by the CID MUST be closed 8765 at both ends (or all connections must be closed if the logout 8766 reason was session close). 8768 Byte/ 0 | 1 | 2 | 3 | 8769 / | | | | 8770 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 8771 +---------------+---------------+---------------+---------------+ 8772 0|.|.| 0x26 |1| Reserved | Response | Reserved | 8773 +---------------+---------------+---------------+---------------+ 8774 4|TotalAHSLength | DataSegmentLength | 8775 +---------------------------------------------------------------+ 8776 8/ Reserved / 8777 +/ / 8778 +---------------+---------------+---------------+---------------+ 8779 16| Initiator Task Tag | 8780 +---------------+---------------+---------------+---------------+ 8781 20| Reserved | 8782 +---------------+---------------+---------------+---------------+ 8783 24| StatSN | 8784 +---------------+---------------+---------------+---------------+ 8785 28| ExpCmdSN | 8786 +---------------+---------------+---------------+---------------+ 8787 32| MaxCmdSN | 8788 +---------------+---------------+---------------+---------------+ 8789 36| Reserved | 8790 +---------------------------------------------------------------+ 8791 40| Time2Wait | Time2Retain | 8792 +---------------+---------------+---------------+---------------+ 8793 44| Reserved | 8794 +---------------+---------------+---------------+---------------+ 8795 48| Header-Digest (Optional) | 8796 +---------------+---------------+---------------+---------------+ 8798 10.15.1. Response 8800 Logout response: 8802 0 - connection or session closed successfully. 8804 1 - CID not found. 8806 2 - connection recovery is not supported. If Logout reason code 8807 was recovery and target does not support it as indicated by 8808 the ErrorRecoveryLevel. 8810 3 - cleanup failed for various reasons. 8812 10.15.2. TotalAHSLength and DataSegmentLength 8814 For this PDU TotalAHSLength and DataSegmentLength MUST be 0. 8816 10.15.3. Time2Wait 8818 If the Logout response code is 0 and if the operational 8819 ErrorRecoveryLevel is 2, this is the minimum amount of time, in 8820 seconds, to wait before attempting task reassignment. If the Logout 8821 response code is 0 and if the operational ErrorRecoveryLevel is 8822 less than 2, this field is to be ignored. 8824 This field is invalid if the Logout response code is 1. 8826 If the Logout response code is 2 or 3, this field specifies the 8827 minimum time to wait before attempting a new implicit or explicit 8828 logout. 8830 If Time2Wait is 0, the reassignment or a new Logout may be 8831 attempted immediately. 8833 10.15.4. Time2Retain 8835 If the Logout response code is 0 and if the operational 8836 ErrorRecoveryLevel is 2, this is the maximum amount of time, in 8837 seconds, after the initial wait (Time2Wait), the target waits for 8838 the allegiance reassignment for any active task after which the 8839 task state is discarded. If the Logout response code is 0 and if 8840 the operational ErrorRecoveryLevel is less than 2, this field is to 8841 be ignored. 8843 This field is invalid if the Logout response code is 1. 8845 If the Logout response code is 2 or 3, this field specifies the 8846 maximum amount of time, in seconds, after the initial wait 8847 (Time2Wait), the target waits for a new implicit or explicit 8848 logout. 8850 If it is the last connection of a session, the whole session state 8851 is discarded after Time2Retain. 8853 If Time2Retain is 0, the target has already discarded the 8854 connection (and possibly the session) state along with the task 8855 states. No reassignment or Logout is required in this case. 8857 10.16. SNACK Request 8859 Byte/ 0 | 1 | 2 | 3 | 8860 / | | | | 8861 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 8862 +---------------+---------------+---------------+---------------+ 8863 0|.|.| 0x10 |1|.|.|.| Type | Reserved | 8864 +---------------+---------------+---------------+---------------+ 8865 4|TotalAHSLength | DataSegmentLength | 8866 +---------------+---------------+---------------+---------------+ 8867 8| LUN or Reserved | 8868 + + 8869 12| | 8870 +---------------+---------------+---------------+---------------+ 8871 16| Initiator Task Tag or 0xffffffff | 8872 +---------------+---------------+---------------+---------------+ 8873 20| Target Transfer Tag or SNACK Tag or 0xffffffff | 8874 +---------------+---------------+---------------+---------------+ 8875 24| Reserved | 8876 +---------------+---------------+---------------+---------------+ 8877 28| ExpStatSN | 8878 +---------------+---------------+---------------+---------------+ 8879 32/ Reserved / 8880 +/ / 8881 +---------------+---------------+---------------+---------------+ 8882 40| BegRun | 8883 +---------------------------------------------------------------+ 8884 44| RunLength | 8885 +---------------------------------------------------------------+ 8886 48| Header-Digest (Optional) | 8887 +---------------+---------------+---------------+---------------+ 8889 If the implementation supports ErrorRecoveryLevel greater than 8890 zero, it MUST support all SNACK types. 8892 The SNACK is used by the initiator to request the retransmission of 8893 numbered-responses, data, or R2T PDUs from the target. The SNACK 8894 request indicates the numbered-responses or data "runs" whose 8895 retransmission is requested by the target, where the run starts 8896 with the first StatSN, DataSN, or R2TSN whose retransmission is 8897 requested and indicates the number of Status, Data, or R2T PDUs 8898 requested including the first. 0 has special meaning when used as a 8899 starting number and length: 8901 - When used in RunLength, it means all PDUs starting with the 8902 initial. 8904 - When used in both BegRun and RunLength, it means all 8905 unacknowledged PDUs. 8907 The numbered-response(s) or R2T(s), requested by a SNACK, MUST be 8908 delivered as exact replicas of the ones that the target transmitted 8909 originally except for the fields ExpCmdSN, MaxCmdSN, and ExpDataSN, 8910 which MUST carry the current values. R2T(s)requested by SNACK MUST 8911 also carry the current value of StatSN. 8913 The numbered Data-In PDUs, requested by a Data SNACK MUST be 8914 delivered as exact replicas of the ones that the target transmitted 8915 originally except for the fields ExpCmdSN and MaxCmdSN, which MUST 8916 carry the current values and except for resegmentation (see 8917 Resegmentation). 8919 Any SNACK that requests a numbered-response, Data, or R2T that was 8920 not sent by the target or was already acknowledged by the 8921 initiator, MUST be rejected with a reason code of "Protocol error". 8923 10.16.1. Type 8925 This field encodes the SNACK function as follows: 8927 0-Data/R2T SNACK - requesting retransmission of one or more 8928 Data-In or R2T PDUs. 8930 1-Status SNACK - requesting retransmission of one or more 8931 numbered responses. 8933 2-DataACK - positively acknowledges Data-In PDUs. 8935 3-R-Data SNACK - requesting retransmission of Data-In PDUs with 8936 possible resegmentation and status tagging. 8938 All other values are reserved. 8940 Data/R2T SNACK, Status SNACK, or R-Data SNACK for a command MUST 8941 precede status acknowledgement for the given command. 8943 10.16.2. Data Acknowledgement 8945 If an initiator operates at ErrorRecoveryLevel 1 or higher, it MUST 8946 issue a SNACK of type DataACK after receiving a Data-In PDU with 8947 the A bit set to 1. However, if the initiator has detected holes in 8948 the input sequence, it MUST postpone issuing the SNACK of type 8949 DataACK until the holes are filled. An initiator MAY ignore the A 8950 bit if it deems that the bit is being set aggressively by the 8951 target (i.e., before the MaxBurstLength limit is reached). 8953 The DataACK is used to free resources at the target and not to 8954 request or imply data retransmission. 8956 An initiator MUST NOT request retransmission for any data it had 8957 already acknowledged. 8959 10.16.3. Resegmentation 8961 If the initiator MaxRecvDataSegmentLength changed between the 8962 original transmission and the time the initiator requests 8963 retransmission, the initiator MUST issue a R-Data SNACK (see Type). 8964 With R-Data SNACK, the initiator indicates that it discards all the 8965 unacknowledged data and expects the target to resend it. It also 8966 expects resegmentation. In this case, the retransmitted Data-In 8967 PDUs MAY be different from the ones originally sent in order to 8968 reflect changes in MaxRecvDataSegmentLength. Their DataSN starts 8969 with the BegRun of the last DataACK received by the target if any 8970 was received; otherwise it starts with 0 and is increased by 1 for 8971 each resent Data-In PDU. 8973 A target that has received a R-Data SNACK MUST return a SCSI 8974 Response that contains a copy of the SNACK Tag field from the R- 8975 Data SNACK in the SCSI Response SNACK Tag field as its last or only 8976 Response. For example, if it has already sent a response containing 8977 another value in the SNACK Tag field or had the status included in 8978 the last Data-In PDU, it must send a new SCSI Response PDU. If a 8979 target sends more than one SCSI Response PDU due to this rule, all 8980 SCSI responses must carry the same StatSN (see SNACK ). If an 8981 initiator attempts to recover a lost SCSI Response (with a Status- 8982 SNACK, see Type) when more than one response has been sent, the 8983 target will send the SCSI Response with the latest content known to 8984 the target, including the last SNACK Tag for the command. 8986 For considerations in allegiance reassignment of a task to a 8987 connection with a different MaxRecvDataSegmentLength, refer to 8988 Section 6.2.2 - "Allegiance Reassignment". 8990 10.16.4. Initiator Task Tag 8992 For Status SNACK and DataACK, the Initiator Task Tag MUST be set to 8993 the reserved value 0xffffffff. In all other cases, the Initiator 8994 Task Tag field MUST be set to the Initiator Task Tag of the 8995 referenced command. 8997 10.16.5. Target Transfer Tag or SNACK Tag 8999 For an R-Data SNACK, this field MUST contain a value that is 9000 different from 0 or 0xffffffff and is unique for the task 9001 (identified by the Initiator Task Tag). This value MUST be copied 9002 by the iSCSI target in the last or only SCSI Response PDU it issues 9003 for the command. 9005 For DataACK, the Target Transfer Tag MUST contain a copy of the 9006 Target Transfer Tag and LUN provided with the SCSI Data-In PDU with 9007 the A bit set to 1. 9009 In all other cases, the Target Transfer Tag field MUST be set to 9010 the reserved value of 0xffffffff. 9012 10.16.6. BegRun 9014 The DataSN, R2TSN, or StatSN of the first PDU whose retransmission 9015 is requested (Data/R2T and Status SNACK), or the next expected 9016 DataSN (DataACK SNACK). 9018 BegRun 0 when used in conjunction with RunLength 0 means resend all 9019 unacknowledged Data-In, R2T or Response PDUs. 9021 BegRun MUST be 0 for a R-Data SNACK. 9023 10.16.7. RunLength 9025 The number of PDUs whose retransmission is requested. 9027 RunLength 0 signals that all Data-In, R2T, or Response PDUs 9028 carrying the numbers equal to or greater than BegRun have to be 9029 resent. 9031 The RunLength MUST also be 0 for a DataACK SNACK in addition to R- 9032 Data SNACK. 9034 10.17. Reject 9036 Byte/ 0 | 1 | 2 | 3 | 9037 / | | | | 9038 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 9039 +---------------+---------------+---------------+---------------+ 9040 0|.|.| 0x3f |1| Reserved | Reason | Reserved | 9041 +---------------+---------------+---------------+---------------+ 9042 4|TotalAHSLength | DataSegmentLength | 9043 +---------------+---------------+---------------+---------------+ 9044 8/ Reserved / 9045 +/ / 9046 +---------------+---------------+---------------+---------------+ 9047 16| 0xffffffff | 9048 +---------------+---------------+---------------+---------------+ 9049 20| Reserved | 9050 +---------------+---------------+---------------+---------------+ 9051 24| StatSN | 9052 +---------------+---------------+---------------+---------------+ 9053 28| ExpCmdSN | 9054 +---------------+---------------+---------------+---------------+ 9055 32| MaxCmdSN | 9056 +---------------+---------------+---------------+---------------+ 9057 36| DataSN/R2TSN or Reserved | 9058 +---------------+---------------+---------------+---------------+ 9059 40| Reserved | 9060 +---------------+---------------+---------------+---------------+ 9061 44| Reserved | 9062 +---------------+---------------+---------------+---------------+ 9063 48| Header-Digest (Optional) | 9064 +---------------+---------------+---------------+---------------+ 9065 xx/ Complete Header of Bad PDU / 9066 +/ / 9067 +---------------+---------------+---------------+---------------+ 9068 yy/Vendor specific data (if any) / 9069 / / 9070 +---------------+---------------+---------------+---------------+ 9071 zz| Data-Digest (Optional) | 9072 +---------------+---------------+---------------+---------------+ 9074 Reject is used to indicate an iSCSI error condition (protocol, 9075 unsupported option, etc.). 9077 10.17.1. Reason 9079 The reject Reason is coded as follows: 9081 +------+----------------------------------------+------------------ 9082 + 9083 | Code | Explanation | Can the original 9084 | 9085 | (hex)| | PDU be re-sent? 9086 | 9087 +------+----------------------------------------+------------------ 9088 + 9089 | 0x01 | Reserved | no 9090 | 9091 | | | 9092 | 9093 | 0x02 | Data (payload) Digest Error | yes (Note 1) 9094 | 9095 | | | 9096 | 9097 | 0x03 | SNACK Reject | yes 9098 | 9099 | | | 9100 | 9101 | 0x04 | Protocol Error (e.g., SNACK request for| no 9102 | 9103 | | a status that was already acknowledged)| 9104 | 9105 | | | 9106 | 9107 | 0x05 | Command not supported | no 9108 | 9109 | | | 9110 | 9111 | 0x06 | Immediate Command Reject - too many | yes 9112 | 9113 | | immediate commands | 9114 | 9115 | | | 9116 | 9117 | 0x07 | Task in progress | no 9118 | 9119 | | | 9120 | 9121 | 0x08 | Invalid Data ACK | no 9122 | 9123 | | | 9124 | 9125 | 0x09 | Invalid PDU field | no (Note 2) 9126 | 9127 | | | 9128 | 9129 | 0x0a | Long Operation Reject - Can't generate | yes 9130 | 9131 | | Target Transfer Tag - out of resources | 9132 | 9133 | | | 9134 | 9135 | 0x0c | Waiting for Logout | no 9136 | 9137 +------+----------------------------------------+------------------ 9138 + 9140 Note 1: For iSCSI, Data-Out PDU retransmission is only done if the 9141 target requests retransmission with a recovery R2T. However, if 9142 this is the data digest error on immediate data, the initiator may 9143 choose to retransmit the whole PDU including the immediate data. 9145 Note 2: A target should use this reason code for all invalid values 9146 of PDU fields that are meant to describe a task, a response, or a 9147 data transfer. Some examples are invalid TTT/ITT, buffer offset, 9148 LUN qualifying a TTT, and an invalid sequence number in a SNACK. 9150 Note 3: Reason code 0x0b (Negotiation reset) defined in [RFC3720] 9151 is deprecated and MUST NOT be used by implementations. An 9152 implementation receiving reason code 0x0b MUST treat it as a 9153 negotiation failure that terminates the Login Phase and the TCP 9154 connection, as specified in Section 6.12. 9156 All other values for Reason are reserved. 9158 In all the cases in which a pre-instantiated SCSI task is 9159 terminated because of the reject, the target MUST issue a proper 9160 SCSI command response with CHECK CONDITION as described in Section 9161 10.4.3. In these cases in which a status for the SCSI task was 9162 already sent before the reject, no additional status is required. 9163 If the error is detected while data from the initiator is still 9164 expected (i.e., the command PDU did not contain all the data and 9165 the target has not received a Data-out PDU with the Final bit set 9166 to 1 for the unsolicited data, if any, and all outstanding R2Ts, if 9167 any), the target MUST wait until it receives the last expected 9168 Data-out PDUs with the F bit set to 1 before sending the Response 9169 PDU. 9171 For additional usage semantics of Reject PDU, see Section 6.3 - 9172 "Usage Of Reject PDU in Recovery". 9174 10.17.2. DataSN/R2TSN 9176 This field is only valid if the rejected PDU is a Data/R2T SNACK 9177 and the Reject reason code is "Protocol error" (see SNACK). The 9178 DataSN/R2TSN is the next Data/R2T sequence number that the target 9179 would send for the task, if any. 9181 10.17.3. StatSN, ExpCmdSN and MaxCmdSN 9183 These fields carry their usual values and are not related to the 9184 rejected command. StatSN is advanced after a Reject. 9186 10.17.4. Complete Header of Bad PDU 9188 The target returns the header (not including digest) of the PDU in 9189 error as the data of the response. 9191 10.18. NOP-Out 9193 Byte/ 0 | 1 | 2 | 3 | 9194 / | | | | 9195 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 9196 +---------------+---------------+---------------+---------------+ 9197 0|.|I| 0x00 |1| Reserved | 9198 +---------------+---------------+---------------+---------------+ 9199 4|TotalAHSLength | DataSegmentLength | 9200 +---------------+---------------+---------------+---------------+ 9201 8| LUN or Reserved | 9202 + + 9203 12| | 9204 +---------------+---------------+---------------+---------------+ 9205 16| Initiator Task Tag or 0xffffffff | 9206 +---------------+---------------+---------------+---------------+ 9207 20| Target Transfer Tag or 0xffffffff | 9208 +---------------+---------------+---------------+---------------+ 9209 24| CmdSN | 9210 +---------------+---------------+---------------+---------------+ 9211 28| ExpStatSN | 9212 +---------------+---------------+---------------+---------------+ 9213 32/ Reserved / 9214 +/ / 9215 +---------------+---------------+---------------+---------------+ 9216 48| Header-Digest (Optional) | 9217 +---------------+---------------+---------------+---------------+ 9218 / DataSegment - Ping Data (optional) / 9219 +/ / 9220 +---------------+---------------+---------------+---------------+ 9221 | Data-Digest (Optional) | 9222 +---------------+---------------+---------------+---------------+ 9224 A NOP-Out may be used by an initiator as a "ping request" to verify 9225 that a connection/session is still active and all its components 9226 are operational. The NOP-In response is the "ping echo". 9228 A NOP-Out is also sent by an initiator in response to a NOP-In. 9230 A NOP-Out may also be used to confirm a changed ExpStatSN if 9231 another PDU will not be available for a long time. 9233 Upon receipt of a NOP-In with the Target Transfer Tag set to a 9234 valid value (not the reserved 0xffffffff), the initiator MUST 9235 respond with a NOP-Out. In this case, the NOP-Out Target Transfer 9236 Tag MUST contain a copy of the NOP-In Target Transfer Tag. 9238 10.18.1. Initiator Task Tag 9240 The NOP-Out MUST have the Initiator Task Tag set to a valid value 9241 only if a response in the form of NOP-In is requested (i.e., the 9242 NOP-Out is used as a ping request). Otherwise, the Initiator Task 9243 Tag MUST be set to 0xffffffff. 9245 When a target receives the NOP-Out with a valid Initiator Task Tag, 9246 it MUST respond with a Nop-In Response (see Login and Full Feature 9247 Phase Negotiation). 9249 If the Initiator Task Tag contains 0xffffffff, the I bit MUST be 9250 set to 1 and the CmdSN is not advanced after this PDU is sent. 9252 10.18.2. Target Transfer Tag 9254 A target assigned identifier for the operation. 9256 The NOP-Out MUST only have the Target Transfer Tag set if it is 9257 issued in response to a NOP-In with a valid Target Transfer Tag. In 9258 this case, it copies the Target Transfer Tag from the NOP-In PDU. 9259 Otherwise, the Target Transfer Tag MUST be set to 0xffffffff. 9261 When the Target Transfer Tag is set to a value other than 9262 0xffffffff, the LUN field MUST also be copied from the NOP-In. 9264 10.18.3. Ping Data 9266 Ping data are reflected in the NOP-In Response. The length of the 9267 reflected data are limited to MaxRecvDataSegmentLength. The length 9268 of ping data are indicated by the DataSegmentLength. 0 is a valid 9269 value for the DataSegmentLength and indicates the absence of ping 9270 data. 9272 10.19. NOP-In 9274 Byte/ 0 | 1 | 2 | 3 | 9275 / | | | | 9276 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 9277 +---------------+---------------+---------------+---------------+ 9278 0|.|.| 0x20 |1| Reserved | 9279 +---------------+---------------+---------------+---------------+ 9280 4|TotalAHSLength | DataSegmentLength | 9281 +---------------+---------------+---------------+---------------+ 9282 8| LUN or Reserved | 9283 + + 9284 12| | 9285 +---------------+---------------+---------------+---------------+ 9286 16| Initiator Task Tag or 0xffffffff | 9287 +---------------+---------------+---------------+---------------+ 9288 20| Target Transfer Tag or 0xffffffff | 9289 +---------------+---------------+---------------+---------------+ 9290 24| StatSN | 9291 +---------------+---------------+---------------+---------------+ 9292 28| ExpCmdSN | 9293 +---------------+---------------+---------------+---------------+ 9294 32| MaxCmdSN | 9295 +---------------+---------------+---------------+---------------+ 9296 36/ Reserved / 9297 +/ / 9298 +---------------+---------------+---------------+---------------+ 9299 48| Header-Digest (Optional) | 9300 +---------------+---------------+---------------+---------------+ 9301 / DataSegment - Return Ping Data / 9302 +/ / 9303 +---------------+---------------+---------------+---------------+ 9304 | Data-Digest (Optional) | 9305 +---------------+---------------+---------------+---------------+ 9307 NOP-In is either sent by a target as a response to a NOP-Out, as a 9308 "ping" to an initiator, or as a means to carry a changed ExpCmdSN 9309 and/or MaxCmdSN if another PDU will not be available for a long 9310 time (as determined by the target). 9312 When a target receives the NOP-Out with a valid Initiator Task Tag 9313 (not the reserved value 0xffffffff), it MUST respond with a NOP-In 9314 with the same Initiator Task Tag that was provided in the NOP-Out 9315 request. It MUST also duplicate up to the first 9316 MaxRecvDataSegmentLength bytes of the initiator provided Ping Data. 9317 For such a response, the Target Transfer Tag MUST be 0xffffffff. 9319 Otherwise, when a target sends a NOP-In that is not a response to a 9320 Nop-Out received from the initiator, the Initiator Task Tag MUST be 9321 set to 0xffffffff and the Data Segment MUST NOT contain any data 9322 (DataSegmentLength MUST be 0). 9324 10.19.1. Target Transfer Tag 9326 If the target is responding to a NOP-Out, this is set to the 9327 reserved value 0xffffffff. 9329 If the target is sending a NOP-In as a Ping (intending to receive a 9330 corresponding NOP-Out), this field is set to a valid value (not the 9331 reserved 0xffffffff). 9333 If the target is initiating a NOP-In without wanting to receive a 9334 corresponding NOP-Out, this field MUST hold the reserved value of 9335 0xffffffff. 9337 10.19.2. StatSN 9339 The StatSN field will always contain the next StatSN. However, when 9340 the Initiator Task Tag is set to 0xffffffff StatSN for the 9341 connection is not advanced after this PDU is sent. 9343 10.19.3. LUN 9345 A LUN MUST be set to a correct value when the Target Transfer Tag 9346 is valid (not the reserved value 0xffffffff). 9348 11. iSCSI Security Text Keys and Authentication Methods 9350 Only the following keys are used during the SecurityNegotiation 9351 stage of the Login Phase: 9353 SessionType 9355 InitiatorName 9357 TargetName 9359 TargetAddress 9361 InitiatorAlias 9363 TargetAlias 9365 TargetPortalGroupTag 9367 AuthMethod and the keys used by the authentication methods 9368 specified under Section 11.1 along with all of their 9369 associated keys as well as Vendor Specific Authentication 9370 Methods. 9372 Other keys MUST NOT be used. 9374 SessionType, InitiatorName, TargetName, InitiatorAlias, 9375 TargetAlias, and TargetPortalGroupTag are described in Section 12 9376 as they can be used also in the OperationalNegotiation stage. 9378 All security keys have connection-wide applicability. 9380 11.1. AuthMethod 9382 Use: During Login - Security Negotiation 9383 Senders: Initiator and Target 9384 Scope: connection 9386 AuthMethod = 9388 The main item of security negotiation is the authentication method 9389 (AuthMethod). 9391 The authentication methods that can be used (appear in the list-of- 9392 values) are either those listed in the following table or are 9393 vendor-unique methods: 9395 +------------------------------------------------------------+ 9396 | Name | Description | 9397 +------------------------------------------------------------+ 9398 | KRB5 | Kerberos V5 - defined in [RFC4120] | 9399 +------------------------------------------------------------+ 9400 | SPKM1 | Simple Public-Key GSS-API Mechanism | 9401 | | defined in [RFC2025] | 9402 +------------------------------------------------------------+ 9403 | SPKM2 | Simple Public-Key GSS-API Mechanism | 9404 | | defined in [RFC2025] | 9405 +------------------------------------------------------------+ 9406 | SRP | Secure Remote Password | 9407 | | defined in [RFC2945] | 9408 +------------------------------------------------------------+ 9409 | CHAP | Challenge Handshake Authentication Protocol| 9410 | | defined in [RFC1994] | 9411 +------------------------------------------------------------+ 9412 | None | No authentication | 9413 +------------------------------------------------------------+ 9415 The AuthMethod selection is followed by an "authentication 9416 exchange" specific to the authentication method selected. 9418 The authentication method proposal may be made by either the 9419 initiator or the target. However the initiator MUST make the first 9420 step specific to the selected authentication method as soon as it 9421 is selected. It follows that if the target makes the authentication 9422 method proposal the initiator sends the first key(s) of the 9423 exchange together with its authentication method selection. 9425 The authentication exchange authenticates the initiator to the 9426 target, and optionally, the target to the initiator. 9427 Authentication is OPTIONAL to use but MUST be supported by the 9428 target and initiator. 9430 The initiator and target MUST implement CHAP. All other 9431 authentication methods are OPTIONAL. 9433 Private or public extension algorithms MAY also be negotiated for 9434 authentication methods. Whenever a private or public extension 9435 algorithm is part of the default offer (the offer made in absence 9436 of explicit administrative action) the implementer MUST ensure that 9437 CHAP is listed as an alternative in the default offer and "None" 9438 is not part of the default offer. 9440 Extension authentication methods MUST be named using one of the 9441 following two formats: 9443 iii) Z-reversed.vendor.dns_name.do_something= 9444 iv) Z<#>= 9446 Authentication methods named using the Z- format are used as 9447 private extensions. Authentication methods named using the Z# 9448 format are used as public extensions that must be registered with 9449 IANA and MUST be described by a standards track RFC, an 9450 experimental RFC, or an informational RFC. 9452 For all of the public or private extension authentication methods, 9453 the method specific keys MUST conform to the format specified in 9454 Section 5.1 - "Text Format" for standard-label. 9456 To identify the vendor for private extension authentication 9457 methods, we suggest you use the reversed DNS-name as a prefix to 9458 the proper digest names. 9460 The part of digest-name following Z- and Z# MUST conform to the 9461 format for standard-label specified in Section 5.1 - "Text Format". 9463 Support for public or private extension authentication methods is 9464 OPTIONAL. 9466 The following subsections define the specific exchanges for each of 9467 the standardized authentication methods. As mentioned earlier the 9468 first step is always done by the initiator. 9470 11.1.1. Kerberos 9472 For KRB5 (Kerberos V5) [RFC1510] and [RFC1964], the initiator MUST 9473 use: 9475 KRB_AP_REQ= 9477 where KRB_AP_REQ is the client message as defined in [RFC1510]. 9479 The default principal name assumed by an iSCSI initiator or target 9480 (prior to any administrative configuration action) MUST be the 9481 iSCSI Initiator Name or iSCSI Target Name respectively, prefixed by 9482 the string "iscsi/". 9484 If the initiator authentication fails, the target MUST respond with 9485 a Login reject with "Authentication Failure" status. Otherwise, if 9486 the initiator has selected the mutual authentication option (by 9487 setting MUTUAL-REQUIRED in the ap-options field of the KRB_AP_REQ), 9488 the target MUST reply with: 9490 KRB_AP_REP= 9492 where KRB_AP_REP is the server's response message as defined in 9493 [RFC1510]. 9495 If mutual authentication was selected and target authentication 9496 fails, the initiator MUST close the connection. 9498 KRB_AP_REQ and KRB_AP_REP are binary-values and their binary length 9499 (not the length of the character string that represents them in 9500 encoded form) MUST NOT exceed 65536 bytes. 9502 11.1.2. Simple Public-Key Mechanism (SPKM) 9504 For SPKM1 and SPKM2 [RFC2025], the initiator MUST use: 9506 SPKM_REQ= 9508 where SPKM-REQ is the first initiator token as defined in 9509 [RFC2025]. 9511 [RFC2025] defines situations where each side may send an error 9512 token 9513 that may cause the peer to re-generate and resend its last token. 9514 This scheme is followed in iSCSI, and the error token syntax is: 9516 SPKM_ERROR= 9518 However, SPKM-DEL tokens that are defined by [RFC2025] for fatal 9519 errors will not be used by iSCSI. If the target needs to send a 9520 SPKM-DEL token, it will, instead, send a Login "login reject" 9521 message with the "Authentication Failure" status and terminate the 9522 connection. If the initiator needs to send a SPKM-DEL token, it 9523 will close the connection. 9525 In the following sections, we assume that no SPKM-ERROR tokens are 9526 required. 9528 If the initiator authentication fails, the target MUST return an 9529 error. Otherwise, if the AuthMethod is SPKM1 or if the initiator 9530 has selected the mutual authentication option (by setting mutual- 9531 state bit in the options field of the REQ-TOKEN in the SPKM-REQ), 9532 the target MUST reply with: 9534 SPKM_REP_TI= 9536 where SPKM-REP-TI is the target token as defined in [RFC2025]. 9538 If mutual authentication was selected and target authentication 9539 fails, the initiator MUST close the connection. Otherwise, if the 9540 AuthMethod is SPKM1, the initiator MUST continue with: 9542 SPKM_REP_IT= 9544 where SPKM-REP-IT is the second initiator token as defined in 9545 [RFC2025]. If the initiator authentication fails, the target MUST 9546 answer with a Login reject with "Authentication Failure" status. 9548 SPKM requires support for very long authentication items. 9550 All the SPKM-* tokens are binary-values and their binary length 9551 (not the length of the character string that represents them in 9552 encoded form) MUST NOT exceed 65536 bytes. 9554 11.1.3. Secure Remote Password (SRP) 9556 For SRP [RFC2945], the initiator MUST use: 9558 SRP_U= TargetAuth=Yes /* or TargetAuth=No */ 9560 The target MUST answer with a Login reject with the "Authorization 9561 Failure" status or reply with: 9563 SRP_GROUP= SRP_s= 9565 Where G1,G2... are proposed groups, in order of preference. 9567 The initiator MUST either close the connection or continue with: 9569 SRP_A= SRP_GROUP= 9571 Where G is one of G1,G2... that were proposed by the target. 9573 The target MUST answer with a Login reject with the "Authentication 9574 Failure" status or reply with: 9576 SRP_B= 9578 The initiator MUST close the connection or continue with: 9580 SRP_M= 9582 If the initiator authentication fails, the target MUST answer with 9583 a Login reject with "Authentication Failure" status. Otherwise, if 9584 the initiator sent TargetAuth=Yes in the first message (requiring 9585 target authentication), the target MUST reply with: 9587 SRP_HM= 9589 If the target authentication fails, the initiator MUST close the 9590 connection. 9592 Where U, s, A, B, M, and H(A | M | K) are defined in [RFC2945] 9593 (using the SHA1 hash function, such as SRP-SHA1) and G,Gn (Gn 9594 stands for G1,G2...) are identifiers of SRP groups specified in 9595 [RFC3723]. G, Gn, and U are text strings, s,A,B,M, and H(A | M | K) 9596 are binary-values. The length of s,A,B,M and H(A | M | K) in binary 9597 form (not the length of the character string that represents them 9598 in encoded form) MUST NOT exceed 1024 bytes. 9600 For the SRP_GROUP, all the groups specified in [RFC3723] up to 1536 9601 bits (i.e., SRP-768, SRP-1024, SRP-1280, SRP-1536) must be 9602 supported by initiators and targets. To guarantee interoperability, 9603 targets MUST always offer "SRP-1536" as one of the proposed groups. 9605 11.1.4. Challenge Handshake Authentication Protocol (CHAP) 9607 For CHAP [RFC1994], the initiator MUST use: 9609 CHAP_A= 9611 Where A1,A2... are proposed algorithms, in order of preference. 9613 The target MUST answer with a Login reject with the "Authentication 9614 Failure" status or reply with: 9616 CHAP_A= CHAP_I= CHAP_C= 9618 Where A is one of A1,A2... that were proposed by the initiator. 9620 The initiator MUST continue with: 9622 CHAP_N= CHAP_R= 9624 or, if it requires target authentication, with: 9626 CHAP_N= CHAP_R= CHAP_I= CHAP_C= 9628 If the initiator authentication fails, the target MUST answer with 9629 a Login reject with "Authentication Failure" status. Otherwise, if 9630 the initiator required target authentication, the target MUST 9631 either answer with a Login reject with "Authentication Failure" or 9632 reply with: 9634 CHAP_N= CHAP_R= 9636 If target authentication fails, the initiator MUST close the 9637 connection. 9639 Where N, (A,A1,A2), I, C, and R are (correspondingly) the Name, 9640 Algorithm, Identifier, Challenge, and Response as defined in 9641 [RFC1994], N is a text string, A,A1,A2, and I are numbers, and C 9642 and R are binary-values and their binary length (not the length of 9643 the character string that represents them in encoded form) MUST NOT 9644 exceed 1024 bytes. 9646 For the Algorithm, as stated in [RFC1994], one value is required 9647 to be implemented: 9649 5 (CHAP with MD5) 9651 To guarantee interoperability, initiators MUST always offer it as 9652 one of the proposed algorithms. 9654 12. Login/Text Operational Text Keys 9656 Some session specific parameters MUST only be carried on the 9657 leading connection and cannot be changed after the leading 9658 connection login (e.g., MaxConnections, the maximum number of 9659 connections). This holds for a single connection session with 9660 regard to connection restart. The keys that fall into this category 9661 have the use: LO (Leading Only). 9663 Keys that can only be used during login have the use: IO 9664 (initialize only), while those that can be used in both the Login 9665 Phase and Full Feature Phase have the use: ALL. 9667 Keys that can only be used during Full Feature Phase use FFPO (Full 9668 Feature Phase only). 9670 Keys marked as Any-Stage may also appear in the SecurityNegotiation 9671 stage while all other keys described in this chapter are 9672 operational keys. 9674 Keys that do not require an answer are marked as Declarative. 9676 Key scope is indicated as session-wide (SW) or connection-only 9677 (CO). 9679 Result function, wherever mentioned, states the function that can 9680 be applied to check the validity of the responder selection. 9681 Minimum means that the selected value cannot exceed the offered 9682 value. Maximum means that the selected value cannot be lower than 9683 the offered value. AND means that the selected value must be a 9684 possible result of a Boolean "and" function with an arbitrary 9685 Boolean value (e.g., if the offered value is No the selected value 9686 must be No). OR means that the selected value must be a possible 9687 result of a Boolean "or" function with an arbitrary Boolean value 9688 (e.g., if the offered value is Yes the selected value must be Yes). 9690 12.1. HeaderDigest and DataDigest 9692 Use: IO 9693 Senders: Initiator and Target 9694 Scope: CO 9695 HeaderDigest = 9696 DataDigest = 9698 Default is None for both HeaderDigest and DataDigest. 9700 Digests enable the checking of end-to-end, non-cryptographic data 9701 integrity beyond the integrity checks provided by the link layers 9702 and the covering of the whole communication path including all 9703 elements that may change the network level PDUs such as routers, 9704 switches, and proxies. 9706 The following table lists cyclic integrity checksums that can be 9707 negotiated for the digests and that MUST be implemented by every 9708 iSCSI initiator and target. These digest options only have error 9709 detection significance. 9711 +---------------------------------------------+ 9712 | Name | Description | Generator | 9713 +---------------------------------------------+ 9714 | CRC32C | 32 bit CRC |0x11edc6f41| 9715 +---------------------------------------------+ 9716 | None | no digest | 9717 +---------------------------------------------+ 9719 The generator polynomial for this digest is given in hex-notation 9720 (e.g., 0x3b stands for 0011 1011 and the polynomial is 9721 x**5+X**4+x**3+x+1). 9723 When the Initiator and Target agree on a digest, this digest MUST 9724 be used for every PDU in Full Feature Phase. 9726 Padding bytes, when present in a segment covered by a CRC, SHOULD 9727 be set to 0 and are included in the CRC. 9729 The CRC MUST be calculated by a method that produces the same 9730 results as the following process: 9732 - The PDU bits are considered as the coefficients of a 9733 polynomial M(x) of degree n-1; bit 7 of the lowest numbered 9734 byte is considered the most significant bit (x^n-1), followed 9736 by bit 6 of the lowest numbered byte through bit 0 of the 9737 highest numbered byte (x^0). 9739 - The most significant 32 bits are complemented. 9741 - The polynomial is multiplied by x^32 then divided by G(x). 9742 The generator polynomial produces a remainder R(x) of degree 9743 <= 31. 9745 - The coefficients of R(x) are considered a 32 bit sequence. 9747 - The bit sequence is complemented and the result is the CRC. 9749 - The CRC bits are mapped into the digest word. The x^31 9750 coefficient in bit 7 of the lowest numbered byte of the 9751 digest continuing through to the byte up to the x^24 9752 coefficient in bit 0 of the lowest numbered byte, continuing 9753 with the x^23 coefficient in bit 7 of next byte through x^0 9754 in bit 0 of the highest numbered byte. 9756 - Computing the CRC over any segment (data or header) extended 9757 to include the CRC built using the generator 0x11edc6f41 will 9758 always get the value 0x1c2d19ed as its final remainder 9759 (R(x)). This value is given here in its polynomial form 9760 (i.e., not mapped as the digest word). 9762 For a discussion about selection criteria for the CRC, see 9763 [RFC3385]. For a detailed analysis of the iSCSI polynomial, see 9764 [Castagnoli93]. 9766 Private or public extension algorithms MAY also be negotiated for 9767 digests. Whenever a private or public digest extension algorithm is 9768 part of the default offer (the offer made in absence of explicit 9769 administrative action) the implementer MUST ensure that CRC32C is 9770 listed as an alternative in the default offer and "None" is not 9771 part of the default offer. 9773 Extension digest algorithms MUST be named using one of the 9774 following two formats: 9776 v) Y-reversed.vendor.dns_name.do_something= 9777 vi) Y<#>= 9779 Digests named using the Y- format are used for private purposes 9780 (unregistered). Digests named using the Y# format (public 9781 extension) must be registered with IANA and MUST be described by a 9782 standards track RFC, an experimental RFC, or an informational RFC. 9784 For private extension digests, to identify the vendor, we suggest 9785 you use the reversed DNS-name as a prefix to the proper digest 9786 names. 9788 The part of digest-name following Y- and Y# MUST conform to the 9789 format for standard-label specified in Section 5.1. 9791 Support for public or private extension digests is OPTIONAL. 9793 12.2. MaxConnections 9795 Use: LO 9796 Senders: Initiator and Target 9797 Scope: SW 9798 Irrelevant when: SessionType=Discovery 9800 MaxConnections= 9802 Default is 1. 9803 Result function is Minimum. 9805 Initiator and target negotiate the maximum number of connections 9806 requested/acceptable. 9808 12.3. SendTargets 9810 Use: FFPO 9811 Senders: Initiator 9812 Scope: SW 9814 For a complete description, see Appendix D. - "SendTargets 9815 Operation". 9817 12.4. TargetName 9819 Use: IO by initiator, FFPO by target - only as response to a 9820 SendTargets, Declarative, Any-Stage 9821 Senders: Initiator and Target 9822 Scope: SW 9824 TargetName= 9826 Examples: 9828 TargetName=iqn.1993-11.com.disk-vendor:diskarrays.sn.45678 9830 TargetName=eui.020000023B040506 9832 TargetName=naa.62004567BA64678D0123456789ABCDEF 9834 The initiator of the TCP connection MUST provide this key to the 9835 remote endpoint in the first login request if the initiator is not 9836 establishing a discovery session. The iSCSI Target Name specifies 9837 the worldwide unique name of the target. 9839 The TargetName key may also be returned by the "SendTargets" text 9840 request (which is its only use when issued by a target). 9842 TargetName MUST NOT be redeclared within the login phase. 9844 12.5. InitiatorName 9846 Use: IO, Declarative, Any-Stage 9847 Senders: Initiator 9848 Scope: SW 9850 InitiatorName= 9852 Examples: 9854 InitiatorName=iqn.1992-04.com.os-vendor.plan9:cdrom.12345 9856 InitiatorName=iqn.2001-02.com.ssp.users:customer235.host90 9857 InitiatorName=naa.52004567BA64678D 9859 The initiator of the TCP connection MUST provide this key to the 9860 remote endpoint at the first Login of the Login Phase for every 9861 connection. The InitiatorName key enables the initiator to identify 9862 itself to the remote endpoint. 9864 InitiatorName MUST NOT be redeclared within the login phase. 9866 12.6. TargetAlias 9868 Use: ALL, Declarative, Any-Stage 9869 Senders: Target 9870 Scope: SW 9872 TargetAlias= 9874 Examples: 9876 TargetAlias=Bob-s Disk 9878 TargetAlias=Database Server 1 Log Disk 9880 TargetAlias=Web Server 3 Disk 20 9882 If a target has been configured with a human-readable name or 9883 description, this name SHOULD be communicated to the initiator 9884 during a Login Response PDU if SessionType=Normal (see 12.21). This 9885 string is not used as an identifier, nor is it meant to be used for 9886 authentication or authorization decisions. It can be displayed by 9887 the initiator's user interface in a list of targets to which it is 9888 connected. 9890 12.7. InitiatorAlias 9892 Use: ALL, Declarative, Any-Stage 9893 Senders: Initiator 9894 Scope: SW 9896 InitiatorAlias= 9897 Examples: 9899 InitiatorAlias=Web Server 4 9901 InitiatorAlias=spyalley.nsa.gov 9903 InitiatorAlias=Exchange Server 9905 If an initiator has been configured with a human-readable name or 9906 description, it SHOULD be communicated to the target during a Login 9907 Request PDU. If not, the host name can be used instead. This string 9908 is not used as an identifier, nor is meant to be used for 9909 authentication or authorization decisions. It can be displayed by 9910 the target's user interface in a list of initiators to which it is 9911 connected. 9913 12.8. TargetAddress 9915 Use: ALL, Declarative, Any-Stage 9916 Senders: Target 9917 Scope: SW 9919 TargetAddress=domainname[:port][,portal-group-tag] 9921 The domainname can be specified as either a DNS host name, a 9922 dotted-decimal IPv4 address, or a bracketed IPv6 address as 9923 specified in [RFC3986]. 9925 If the TCP port is not specified, it is assumed to be the IANA- 9926 assigned default port for iSCSI (see Section 13 -"IANA 9927 Considerations"). 9929 If the TargetAddress is returned as the result of a redirect status 9930 in a login response, the comma and portal group tag MUST be 9931 omitted. 9933 If the TargetAddress is returned within a SendTargets response, the 9934 portal group tag MUST be included. 9936 Examples: 9938 TargetAddress=10.0.0.1:5003,1 9940 TargetAddress=[1080:0:0:0:8:800:200C:417A],65 9942 TargetAddress=[1080::8:800:200C:417A]:5003,1 9944 TargetAddress=computingcenter.example.com,23 9946 Use of the portal-group-tag is described in Appendix D. - 9947 "SendTargets Operation". The formats for the port and portal-group- 9948 tag are the same as the one specified in TargetPortalGroupTag. 9950 12.9. TargetPortalGroupTag 9952 Use: IO by target, Declarative, Any-Stage 9953 Senders: Target 9954 Scope: SW 9956 TargetPortalGroupTag=<16-bit-binary-value> 9958 Examples: 9959 TargetPortalGroupTag=1 9961 The target portal group tag is a 16-bit binary-value that uniquely 9962 identifies a portal group within an iSCSI target node. This key 9963 carries the value of the tag of the portal group that is servicing 9964 the Login request. The iSCSI target returns this key to the 9965 initiator in the Login Response PDU to the first Login Request PDU 9966 that has the C bit set to 0 when TargetName is given by the 9967 initiator. 9969 [SAM2] and [SAM3] specifications note in their informative text 9970 that TPGT value should be non-zero, note that it is incorrect. A 9971 zero value is allowed as a legal value for TPGT. This discrepancy 9972 currently stands corrected in [SAM4]. 9974 For the complete usage expectations of this key see Section 5.3. 9976 12.10. InitialR2T 9978 Use: LO 9979 Senders: Initiator and Target 9980 Scope: SW 9981 Irrelevant when: SessionType=Discovery 9983 InitialR2T= 9985 Examples: 9987 I->InitialR2T=No 9989 T->InitialR2T=No 9991 Default is Yes. 9992 Result function is OR. 9994 The InitialR2T key is used to turn off the default use of R2T for 9995 unidirectional and the output part of bidirectional commands, thus 9996 allowing an initiator to start sending data to a target as if it 9997 has received an initial R2T with Buffer Offset=Immediate Data 9998 Length and Desired Data Transfer Length=(min(FirstBurstLength, 9999 Expected Data Transfer Length) - Received Immediate Data Length). 10001 The default action is that R2T is required, unless both the 10002 initiator and the target send this key-pair attribute specifying 10003 InitialR2T=No. Only the first outgoing data burst (immediate data 10004 and/or separate PDUs) can be sent unsolicited (i.e., not requiring 10005 an explicit R2T). 10007 12.11. ImmediateData 10009 Use: LO 10010 Senders: Initiator and Target 10011 Scope: SW 10012 Irrelevant when: SessionType=Discovery 10014 ImmediateData= 10016 Default is Yes. 10018 Result function is AND. 10020 The initiator and target negotiate support for immediate data. To 10021 turn immediate data off, the initiator or target must state its 10022 desire to do so. ImmediateData can be turned on if both the 10023 initiator and target have ImmediateData=Yes. 10025 If ImmediateData is set to Yes and InitialR2T is set to Yes 10026 (default), then only immediate data are accepted in the first 10027 burst. 10029 If ImmediateData is set to No and InitialR2T is set to Yes, then 10030 the initiator MUST NOT send unsolicited data and the target MUST 10031 reject unsolicited data with the corresponding response code. 10033 If ImmediateData is set to No and InitialR2T is set to No, then the 10034 initiator MUST NOT send unsolicited immediate data, but MAY send 10035 one unsolicited burst of Data-OUT PDUs. 10037 If ImmediateData is set to Yes and InitialR2T is set to No, then 10038 the initiator MAY send unsolicited immediate data and/or one 10039 unsolicited burst of Data-OUT PDUs. 10041 The following table is a summary of unsolicited data options: 10043 +----------+-------------+------------------+--------------+ 10044 |InitialR2T|ImmediateData| Unsolicited |Immediate Data| 10045 | | | Data Out PDUs | | 10046 +----------+-------------+------------------+--------------+ 10047 | No | No | Yes | No | 10048 +----------+-------------+------------------+--------------+ 10049 | No | Yes | Yes | Yes | 10050 +----------+-------------+------------------+--------------+ 10051 | Yes | No | No | No | 10052 +----------+-------------+------------------+--------------+ 10053 | Yes | Yes | No | Yes | 10054 +----------+-------------+------------------+--------------+ 10056 12.12. MaxRecvDataSegmentLength 10058 Use: ALL, Declarative 10059 Senders: Initiator and Target 10060 Scope: CO 10062 MaxRecvDataSegmentLength= 10064 Default is 8192 bytes. 10066 The initiator or target declares the maximum data segment length in 10067 bytes it can receive in an iSCSI PDU. 10069 The transmitter (initiator or target) is required to send PDUs with 10070 a data segment that does not exceed MaxRecvDataSegmentLength of the 10071 receiver. 10073 A target receiver is additionally limited by MaxBurstLength for 10074 solicited data and FirstBurstLength for unsolicited data. An 10075 initiator MUST NOT send solicited PDUs exceeding MaxBurstLength nor 10076 unsolicited PDUs exceeding FirstBurstLength (or FirstBurstLength- 10077 Immediate Data Length if immediate data were sent). 10079 12.13. MaxBurstLength 10081 Use: LO 10082 Senders: Initiator and Target 10083 Scope: SW 10084 Irrelevant when: SessionType=Discovery 10086 MaxBurstLength= 10088 Default is 262144 (256 Kbytes). 10089 Result function is Minimum. 10091 The initiator and target negotiate maximum SCSI data payload in 10092 bytes in a Data-In or a solicited Data-Out iSCSI sequence. A 10093 sequence consists of one or more consecutive Data-In or Data-Out 10094 PDUs that end with a Data-In or Data-Out PDU with the F bit set to 10095 one. 10097 12.14. FirstBurstLength 10099 Use: LO 10100 Senders: Initiator and Target 10101 Scope: SW 10102 Irrelevant when: SessionType=Discovery 10103 Irrelevant when: ( InitialR2T=Yes and ImmediateData=No ) 10105 FirstBurstLength= 10107 Default is 65536 (64 Kbytes). 10108 Result function is Minimum. 10110 The initiator and target negotiate the maximum amount in bytes of 10111 unsolicited data an iSCSI initiator may send to the target during 10112 the execution of a single SCSI command. This covers the immediate 10113 data (if any) and the sequence of unsolicited Data-Out PDUs (if 10114 any) that follow the command. 10116 FirstBurstLength MUST NOT exceed MaxBurstLength. 10118 12.15. DefaultTime2Wait 10120 Use: LO 10121 Senders: Initiator and Target 10122 Scope: SW 10124 DefaultTime2Wait= 10126 Default is 2. 10127 Result function is Maximum. 10129 The initiator and target negotiate the minimum time, in seconds, to 10130 wait before attempting an explicit/implicit logout or an active 10131 task reassignment after an unexpected connection termination or a 10132 connection reset. 10134 A value of 0 indicates that logout or active task reassignment can 10135 be attempted immediately. 10137 12.16. DefaultTime2Retain 10139 Use: LO 10140 Senders: Initiator and Target 10141 Scope: SW 10142 DefaultTime2Retain= 10144 Default is 20. 10145 Result function is Minimum. 10147 The initiator and target negotiate the maximum time, in seconds 10148 after an initial wait (Time2Wait), before which an active task 10149 reassignment is still possible after an unexpected connection 10150 termination or a connection reset. 10152 This value is also the session state timeout if the connection in 10153 question is the last LOGGED_IN connection in the session. 10155 A value of 0 indicates that connection/task state is immediately 10156 discarded by the target. 10158 12.17. MaxOutstandingR2T 10160 Use: LO 10161 Senders: Initiator and Target 10162 Scope: SW 10164 MaxOutstandingR2T= 10165 Irrelevant when: SessionType=Discovery 10167 Default is 1. 10168 Result function is Minimum. 10170 Initiator and target negotiate the maximum number of outstanding 10171 R2Ts per task, excluding any implied initial R2T that might be part 10172 of that task. An R2T is considered outstanding until the last data 10173 PDU (with the F bit set to 1) is transferred, or a sequence 10174 reception timeout (Section 6.1.4.1) is encountered for that data 10175 sequence. 10177 12.18. DataPDUInOrder 10179 Use: LO 10180 Senders: Initiator and Target 10181 Scope: SW 10182 Irrelevant when: SessionType=Discovery 10183 DataPDUInOrder= 10185 Default is Yes. 10186 Result function is OR. 10188 No is used by iSCSI to indicate that the data PDUs within sequences 10189 can be in any order. Yes is used to indicate that data PDUs within 10190 sequences have to be at continuously increasing addresses and 10191 overlays are forbidden. 10193 12.19. DataSequenceInOrder 10195 Use: LO 10196 Senders: Initiator and Target 10197 Scope: SW 10198 Irrelevant when: SessionType=Discovery 10200 DataSequenceInOrder= 10202 Default is Yes. 10203 Result function is OR. 10205 A Data Sequence is a sequence of Data-In or Data-Out PDUs that end 10206 with a Data-In or Data-Out PDU with the F bit set to one. A Data- 10207 out sequence is sent either unsolicited or in response to an R2T. 10208 Sequences cover an offset-range. 10210 If DataSequenceInOrder is set to No, Data PDU sequences may be 10211 transferred in any order. 10213 If DataSequenceInOrder is set to Yes, Data Sequences MUST be 10214 transferred using continuously non-decreasing sequence offsets (R2T 10215 buffer offset for writes, or the smallest SCSI Data-In buffer 10216 offset within a read data sequence). 10218 If DataSequenceInOrder is set to Yes, a target may retry at most 10219 the last R2T, and an initiator may at most request retransmission 10220 for the last read data sequence. For this reason, if 10221 ErrorRecoveryLevel is not 0 and DataSequenceInOrder is set to Yes 10222 then MaxOustandingR2T MUST be set to 1. 10224 12.20. ErrorRecoveryLevel 10226 Use: LO 10227 Senders: Initiator and Target 10228 Scope: SW 10230 ErrorRecoveryLevel= 10232 Default is 0. 10233 Result function is Minimum. 10235 The initiator and target negotiate the recovery level supported. 10237 Recovery levels represent a combination of recovery capabilities. 10238 Each recovery level includes all the capabilities of the lower 10239 recovery levels and adds some new ones to them. 10241 In the description of recovery mechanisms, certain recovery classes 10242 are specified. Section 6.1.5 describes the mapping between the 10243 classes and the levels. 10245 12.21. SessionType 10247 Use: LO, Declarative, Any-Stage 10248 Senders: Initiator 10249 Scope: SW 10251 SessionType= 10253 Default is Normal. 10255 The Initiator indicates the type of session it wants to create. The 10256 target can either accept it or reject it. 10258 A Discovery session indicates to the Target that the only purpose 10259 of this Session is discovery. The only requests a target accepts in 10260 this type of session are a text request with a SendTargets key and 10261 a logout request with reason "close the session". 10263 The Discovery session implies MaxConnections = 1 and overrides both 10264 the default and an explicit setting. As section 6.4.1 states, 10265 ErrorRecoveryLevel MUST be 0 (zero) for Discovery sessions. 10267 Depending on the type of the session, a target may decide on 10268 resources to allocate and the security to enforce, etc. for the 10269 session. If the SessionType key is thus going to be offered as 10270 "Discovery", it SHOULD be offered in the initial Login request by 10271 the initiator. 10273 12.22. The Private or Public Extension Key Format 10275 Use: ALL 10276 Senders: Initiator and Target 10277 Scope: specific key dependent 10279 X-reversed.vendor.dns_name.do_something= 10281 or 10283 X<#>= 10285 Keys with this format are used for public or private extension 10286 purposes. These keys always start with X- if unregistered with IANA 10287 (private) or X# if registered with IANA (public). 10289 For unregistered keys, to identify the vendor, we suggest you use 10290 the reversed DNS-name as a prefix to the key-proper. 10292 The part of key-name following X- and X# MUST conform to the format 10293 for key-name specified in Section 5.1 -"Text Format". 10295 For IANA registered keys the string following X# must be registered 10296 with IANA and the use of the key MUST be described by a standards 10297 track RFC, an experimental RFC, or an informational RFC. 10299 Vendor specific keys MUST ONLY be used in normal sessions. 10301 Support for public or private extension keys is OPTIONAL. 10303 12.23. Task Reporting 10305 Use: LO 10306 Senders: Initiator and Target 10307 Scope: SW 10308 Irrelevant when: SessionType=Discovery 10309 TaskReporting= 10311 Default is RFC3720. 10312 Result function is AND. 10314 This key is used to negotiate the task completion reporting 10315 semantics from the SCSI target. The following table describes the 10316 semantics that an iSCSI target MUST support for respective 10317 negotiated key values. Whenever this key is negotiated, at least 10318 the RFC3720 and ResponseFence values MUST be offered as options by 10319 the negotiation originator. 10320 +--------------+------------------------------------------+ 10321 | Name | Description | 10322 +--------------+------------------------------------------+ 10323 | RFC3720 | RFC 3720-compliant semantics. Response | 10324 | | fencing is not guaranteed and fast | 10325 | | completion of multi-task aborting is not | 10326 | | supported | 10327 +--------------+------------------------------------------+ 10328 | ResponseFence| Response Fence (section 3.2.2.3.3) | 10329 | | semantics MUST be supported in reporting | 10330 | | task completions | 10331 +--------------+------------------------------------------+ 10332 | FastAbort | Updated fast multi-task abort semantics | 10333 | | defined in Section 3.2.3.4 MUST be | 10334 | | supported. Support for Response Fence is | 10335 | | implied -- i.e., (Section 3.2.2.3.3) | 10336 | | semantics MUST be supported as well | 10337 +--------------+------------------------------------------+ 10338 When TaskReporting is not negotiated to FastAbort, the standard 10339 multi-task abort semantics in Section 3.2.3.3 MUST be used. 10341 12.24. X#NodeArchitecture 10343 12.24.1. Definition 10345 Use: LO, Declarative 10346 Senders: Initiator and Target 10347 Scope: SW 10349 X#NodeArchitecture= 10351 Default is none. 10353 Examples: 10354 X#NodeArchitecture=ExampleOS/v1234,ExampleInc_SW_Initiator/1.05a 10355 X#NodeArchitecture=ExampleInc_HW_Initiator/4010,Firmware/2.0.0.5 10356 X#NodeArchitecture=ExampleInc_SW_Initiator/2.1,CPU_Arch/i686 10358 This document does not define the structure or content of the list 10359 of values. 10361 The initiator or target declares the details of its iSCSI node 10362 architecture to the remote endpoint. These details may include, but 10363 are not limited to, iSCSI vendor software, firmware, or hardware 10364 versions, the OS version, or hardware architecture. This key may 10365 be declared on a Discovery session or a Normal session. 10367 The length of the key value (total length of the list-of-values) 10368 MUST NOT be greater than 255 bytes. 10370 X#NodeArchitecture MUST NOT be redeclared during the login phase. 10372 12.24.2. Implementation Requirements 10374 Functional behavior of the iSCSI node (this includes the iSCSI 10375 protocol logic -- the SCSI, iSCSI, and TCP/IP protocols) MUST NOT 10376 depend on the presence, absence, or content of the 10377 X#NodeArchitecture key. The key MUST NOT be used by iSCSI nodes for 10378 interoperability, or exclusion of other nodes. To ensure proper 10379 use, key values SHOULD be set by the node itself, and there SHOULD 10380 NOT be provisions for the key values to contain user-defined text. 10382 Nodes implementing this key MUST choose one of the following 10383 implementation options: 10385 only transmit the key, 10386 only log the key values received from other nodes, or 10387 both transmit and log the key values. 10388 Each node choosing to implement transmission of the key values MUST 10389 be prepared to handle the response of iSCSI Nodes that do not 10390 understand the key. 10392 Nodes that implement transmission and/or logging of the key values 10393 may also implement administrative mechanisms that disable and/or 10394 change the logging and key transmission detail (see Section 8.4 - 10395 "Security Considerations for the X#NodeArchitecture Key"). Thus, a 10396 valid behavior for this key may be that a node is completely silent 10397 (the node does not transmit any key value, and simply discards any 10398 key values it receives without issuing a NotUnderstood response). 10400 13. IANA Considerations 10402 The well-known TCP port number for iSCSI connections assigned by 10403 IANA is 3260 and this is the default iSCSI port. Implementations 10404 needing a system TCP port number may use port 860, the port 10405 assigned by IANA as the iSCSI system port; however in order to use 10406 port 860, it MUST be explicitly specified - implementations MUST 10407 NOT default to use of port 860, as 3260 is the only allowed 10408 default. 10410 [RFC3720] instructs that three text key registries be set up, one 10411 for each of Extension keys, authentication methods, or digest keys 10412 with the stipulation that the key prefix (X#, Y# or Z#) be not 10413 recorded. However, [RFC4850] indicates that the key prefix was 10414 recorded by IANA as part of the key name. Consequently, storm 10415 working group (which published this document) instructs IANA that: 10416 (i) It maintain a single text key registry for iSCSI keys, and, 10417 (ii) MUST always record the key prefix as part of the recorded 10418 string 10420 This is being done with the intent to not have to change what IANA 10421 already did while publishing [RFC4850]. 10423 All the other IANA considerations stated in [RFC3720] and [RFC5048] 10424 remain unchanged. 10426 References 10428 Normative References 10430 [EUI] "Guidelines for 64-bit Global Identifier (EUI-64)", 10431 http://standards.ieee.org/regauth/oui/tutorials/EUI64.html 10433 [FC-FS] INCITS 373-2003, Fibre Channel Framing and Signaling 10434 Interface (FC-FS). 10436 [OUI] "IEEE OUI and Company_Id Assignments", 10437 http://standards.ieee.org/regauth/oui 10439 [RFC791] INTERNET PROTOCOL, DARPA INTERNET PROGRAM PROTOCOL 10440 SPECIFICATION, September 1981. 10442 [RFC793] TRANSMISSION CONTROL PROTOCOL, DARPA INTERNET PROGRAM 10443 PROTOCOL SPECIFICATION, September 1981. 10445 [RFC1035] P. Mockapetris, DOMAIN NAMES - IMPLEMENTATION AND 10446 SPECIFICATION, November 1987. 10448 [RFC1122] Requirements for Internet Hosts-Communication Layer 10449 RFC1122, R. Braden (editor). 10451 [RFC1964] J. Linn, "The Kerberos Version 5 GSS-API Mechanism", 10452 June 1996. 10454 [RFC1982] Elz, R., Bush, R., "Serial Number Arithmetic", August 10455 1996. 10457 [RFC1994] W. Simpson, PPP Challenge Handshake Authentication 10458 Protocol (CHAP)", August 1996. 10460 [RFC2025] C. Adams, "The Simple Public-Key GSS-API Mechanism 10461 (SPKM)", October 1996. 10463 [RFC2045] N. Borenstein, N. Freed, "MIME (Multipurpose Internet 10464 Mail Extensions) Part One: Mechanisms for Specifying and 10465 Describing the Format of Internet Message Bodies", November 10466 1996. 10468 [RFC2119] Bradner, S. "Key Words for use in RFCs to Indicate 10469 Requirement Levels", BCP 14, March 1997. 10471 [RFC2373] Hinden, R. and S. Deering, "IP Version 6 Addressing 10472 Architecture", July 1998. 10474 [RFC2404] C. Madson, R. Glenn, "The Use of HMAC-SHA-1-96 within 10475 ESP and AH", November 1998. 10477 [RFC2451] R. Pereira, R. Adams " The ESP CBC-Mode Cipher 10478 Algorithms". 10480 [RFC2945] Wu, T., "The SRP Authentication and Key Exchange 10481 System", September 2000. 10483 [RFC3454] Hoffman, P. and M. Blanchet, "Preparation of 10484 Internationalized Strings ("stringprep")", RFC 3454, December 10485 2002. 10487 [RFC3566] Frankel, S. and H. Herbert, "The AES-XCBC-MAC-96 10488 Algorithm and Its Use With IPsec", RFC 3566, September 2003. 10490 [RFC3629] Yergeau, F., "UTF-8, a Transformation Format of ISO 10491 10646", RFC 3629, November 2003 10493 [RFC3686] Housley, R., "Using Advanced Encryption Standard 10494 (AES) Counter Mode with IPsec Encapsulating Security Payload 10495 (ESP)", RFC 3686, January 2004. 10497 [RFC3720] Satran, J., Meth, K., Sapuntzakis, C., Chadalapaka, 10498 M., and E. Zeidner, "Internet Small Computer Systems 10499 Interface (iSCSI)", RFC 3720, April 2004. 10501 [RFC3722] Bakke, M., "String Profile for Internet Small 10502 Computer Systems Interface (iSCSIs", RFC 3722, March 10503 2004. 10505 [RFC3723] Aboba, B., Tseng, J., Walker, J., Rangan, V. and F. 10506 Travostino, "Securing Block Storage Protocols over IP", RFC 10507 3723, March 2004. 10509 [RFC3783] M. Chadalapaka, R. Elliott, Small Computer Systems 10510 Interface (SCSI) Command Ordering Considerations with iSCSI, 10511 RFC 3783, May 2004. 10513 [RFC3980] Krueger, M., Chadalapaka, M., Elliott, R., "T11 10514 Network Address Authority (NAA) Naming Format for iSCSI Node 10515 Names", RFC 3980, February 2005. 10517 [RFC3986] T. Berners-Lee, R. Fielding, L. Masinter "Uniform 10518 Resource Identifier (URI): Generic Syntax", January 2005. 10520 [RFC4301] S. Kent, K.Seo, "Security Architecture for the 10521 Internet Protocol", December 2005. 10523 [RFC4303] Kent, S., "IP Encapsulating Security Payload (ESP)", 10524 RFC 4303, December 2005 10526 [RFC4306] C. Kaufman (Ed), "Internet Key Exchange (IKEv2) 10527 Protocol", December 2005. 10529 [RFC4120] Neuman, C., Yu, T., Hartman, S., Raeburn, K, "The 10530 Kerberos Network Authentication Service (V5)", RFC 4120, July 10531 2005. 10533 [RFC4646] A. Phillips, M. Davis, "Tags for the Identification 10534 of Languages", RFC 4646, September 2006. 10536 [RFC4850] Wysochanski, D., "Declarative Public Extension Key 10537 for Internet Small Computer Systems Interface (iSCSI) Node 10538 Architecture", RFC 4850, April 2007. 10540 [RFC5048] Chadalapaka, M., "Internet Small Computer Systems 10541 Interface (iSCSI) Corrections and Clarifications", RFC 5048, 10542 October 2007. 10544 [RFC5226] T. Narten, and H. Avestrand, "Guidelines for Writing 10545 an IANA Considerations Section in RFCs.", May 2008. 10547 [SAM2] T10/1157-D, SCSI Architecture Model - 2 (SAM-2). 10549 [SAM3] T10/1561-D, SCSI Architecture Model - 3 (SAM-3). 10551 [SAM4] T10/1683-D, SCSI Architecture Model - 4 (SAM-4). 10553 [SBC] NCITS.306-1998, SCSI-3 Block Commands (SBC). 10555 [SPC3] T10/1416-D, SCSI Primary Commands-3. 10557 [UNICODE] Unicode Standard Annex #15, "Unicode Normalization 10558 Forms", http://www.unicode.org/unicode/reports/tr15 10560 Informative References: 10562 [RFC1737] K. Sollins, L. Masinter "Functional Requirements for 10563 Uniform Resource Names". 10565 [IB] InfiniBand{tm} Architecture Specification, Vol. 1, 10566 Rel.1.0.a, InfiniBand 10567 TradezAssociation(http://www.infinibandta.org). 10569 [RFC4173] P. Sarkar, D. Missimer, C. Sapuntzakis, 10570 "Bootstrapping Clients using the Internet Small Computer 10571 System Interface (iSCSI) Protocol", RFC 4173, September 2005. 10573 [Castagnoli93] G. Castagnoli, S. Braeuer and M. Herrman 10574 "Optimization of Cyclic Redundancy-Check Codes with 24 and 32 10575 Parity Bits", IEEE Transact. on Communications, Vol. 41, No. 10576 6, June 1993. 10578 [CRC] ISO 3309, High-Level Data Link Control (CRC 32). 10580 [RFC3347] Krueger, M., Haagens, R., Sapuntzakis, C. and M. 10581 Bakke, "Small Computer Systems Interface protocol over the 10582 Internet (iSCSI) Requirements and Design Considerations", RFC 10583 3347, July 2002. 10585 [RFC3385] Sheinwald, D., Staran, J., Thaler, P. and V. Cavanna, 10586 "Internet Protocol Small Computer System Interface (iSCSI) 10587 Cyclic Redundancy Check (CRC)/Checksum Considerations", RFC 10588 3385, September 2002. 10590 [RFC3721] Bakke M., Hafner, J., Hufferd, J., Voruganti, K. and 10591 M. Krueger, "Internet Small Computer Systems Interface 10592 (iSCSI) Naming and Discovery, RFC 3721, March 2004 10594 [Schneier] B. Schneier, "Applied Cryptography: Protocols, 10595 Algorithms, and Source Code in C", 2nd edition, John Wiley & 10596 Sons, New York, NY, 1996. 10598 [SAS] INCITS 376-2003, Serial Attached SCSI (SAS). 10600 [SRP] INCITS 365-2002, SCSI RDMA Protocol (SRP). 10602 Appendix A. Sync and Steering with Fixed Interval Markers 10604 This appendix presents a simple scheme for synchronization (PDU 10605 boundary retrieval). It uses markers that include synchronization 10606 information placed at fixed intervals in the TCP stream. 10608 A Marker consists of: 10610 Byte / 0 | 1 | 2 | 3 | 10611 / | | | | 10612 |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7| 10613 +---------------+---------------+---------------+---------------+ 10614 0| Next-iSCSI-PDU-start pointer - copy #1 | 10615 +---------------+---------------+---------------+---------------+ 10616 4| Next-iSCSI-PDU-start pointer - copy #2 | 10617 +---------------+---------------+---------------+---------------+ 10619 The Marker scheme uses payload byte stream counting that includes 10620 every byte placed by iSCSI in the TCP stream except for the markers 10621 themselves. It also excludes any bytes that TCP counts but are not 10622 originated by iSCSI. 10624 Markers MUST NOT be included in digest calculation. 10626 The Marker indicates the offset to the next iSCSI PDU header. The 10627 Marker is eight bytes in length and contains two 32-bit offset 10628 fields that indicate how many bytes to skip in the TCP stream in 10629 order to find the next iSCSI PDU header. The marker uses two copies 10630 of the pointer so that a marker that spans a TCP packet boundary 10631 should leave at least one valid copy in one of the packets. 10633 The structure and semantics of an inserted marker are independent 10634 of the marker interval. 10636 The use of markers is negotiable. The initiator and target MAY 10637 indicate their readiness to receive and/or send markers during 10638 login separately for each connection. The default is No. 10640 A.1 Markers At Fixed Intervals 10642 A marker is inserted at fixed intervals in the TCP byte stream. 10643 During login, each end of the iSCSI session specifies the interval 10644 at which it is willing to receive the marker, or it disables the 10645 marker altogether. If a receiver indicates that it desires a 10646 marker, the sender MAY agree (during negotiation) and provide the 10647 marker at the desired interval. However, in certain environments, a 10648 sender that does not provide markers to a receiver that wants 10649 markers may suffer an appreciable performance degradation. 10651 The marker interval and the initial marker-less interval are 10652 counted in terms of the bytes placed in the TCP stream data by 10653 iSCSI. 10655 When reduced to iSCSI terms, markers MUST indicate the offset to a 10656 4-byte word boundary in the stream. The least significant two bits 10657 of each marker word are reserved and are considered 0 for offset 10658 computation. 10660 Padding iSCSI PDU payloads to 4-byte word boundaries simplifies 10661 marker manipulation. 10663 A.2 Initial Marker-less Interval 10665 To enable the connection setup including the Login Phase 10666 negotiation, marking (if any) is only started at the first marker 10667 interval after the end of the Login Phase. However, in order to 10668 enable the marker inclusion and exclusion mechanism to work without 10669 knowledge of the length of the Login Phase, the first marker will 10670 be placed in the TCP stream as if the Marker-less interval had 10671 included markers. 10673 Thus, all markers appear in the stream at locations conforming to 10674 the formula: [(MI + 8) * n - 8] where MI = Marker Interval, n = 10675 integer number. 10677 For example, if the marker interval is 512 bytes and the login 10678 ended at byte 1003 (first iSCSI placed byte is 0), the first marker 10679 will be inserted after byte 1031 in the stream. 10681 A.3 Negotiation 10683 The following operational key=value pairs are used to negotiate the 10684 fixed interval markers. The direction (output or input) is relative 10685 to the initiator. 10687 A.3.1 OFMarker, IFMarker 10689 Use: IO 10690 Senders: Initiator and Target 10691 Scope: CO 10693 OFMarker= 10694 IFMarker= 10696 Default is No. 10698 Result function is AND. 10700 OFMarker is used to turn on or off the initiator to target markers 10701 on the connection. IFMarker is used to turn on or off the target 10702 to initiator markers on the connection. 10704 Examples: 10706 I->OFMarker=Yes,IFMarker=Yes 10708 T->OFMarker=Yes,IFMarker=Yes 10710 Results in the Marker being used in both directions while: 10712 I->OFMarker=Yes,IFMarker=Yes 10714 T->OFMarker=Yes,IFMarker=No 10716 Results in Marker being used from the initiator to the target, but 10717 not from the target to initiator. 10719 A.3.2 OFMarkInt, IFMarkInt 10721 Use: IO 10722 Senders: Initiator and Target 10723 Scope: CO 10724 OFMarkInt is Irrelevant when: OFMarker=No 10725 IFMarkInt is Irrelevant when: IFMarker=No 10727 Offering: 10729 OFMarkInt= 10730 IFMarkInt= 10732 Responding: 10734 OFMarkInt=|Reject 10735 IFMarkInt=|Reject 10737 OFMarkInt is used to set the interval for the initiator to target 10738 markers on the connection. IFMarkInt is used to set the interval 10739 for the target to initiator markers on the connection. 10741 For the offering, the initiator or target indicates the minimum to 10742 maximum interval (in 4-byte words) it wants the markers for one or 10743 both directions. In case it only wants a specific value, only a 10744 single value has to be specified. The responder selects a value 10745 within the minimum and maximum offered or the only value offered or 10746 indicates through the xFMarker key=value its inability to set 10747 and/or receive markers. When the interval is unacceptable the 10748 responder answers with "Reject". Reject is resetting the marker 10749 function in the specified direction (Output or Input) to No. 10751 The interval is measured from the end of a marker to the beginning 10752 of the next marker. For example, a value of 1024 means 1024 words 10753 (4096 bytes of iSCSI payload between markers). 10755 The default is 2048. 10757 Appendix B. Examples 10759 B.1 Read Operation Example 10761 +------------------+-----------------------+----------------------+ 10762 |Initiator Function| PDU Type | Target Function | 10763 +------------------+-----------------------+----------------------+ 10764 | Command request |SCSI Command (READ)>>> | | 10765 | (read) | | | 10766 +------------------+-----------------------+----------------------+ 10767 | | |Prepare Data Transfer | 10768 +------------------+-----------------------+----------------------+ 10769 | Receive Data | <<< SCSI Data-in | Send Data | 10770 +------------------+-----------------------+----------------------+ 10771 | Receive Data | <<< SCSI Data-in | Send Data | 10772 +------------------+-----------------------+----------------------+ 10773 | Receive Data | <<< SCSI Data-in | Send Data | 10774 +------------------+-----------------------+----------------------+ 10775 | | <<< SCSI Response |Send Status and Sense | 10776 +------------------+-----------------------+----------------------+ 10777 | Command Complete | | | 10778 +------------------+-----------------------+----------------------+ 10780 B.2 Write Operation Example 10782 +------------------+-----------------------+---------------------+ 10783 |Initiator Function| PDU Type | Target Function | 10784 +------------------+-----------------------+---------------------+ 10785 | Command request |SCSI Command (WRITE)>>>| Receive command | 10786 | (write) | | and queue it | 10787 +------------------+-----------------------+---------------------+ 10788 | | | Process old commands| 10789 +------------------+-----------------------+---------------------+ 10790 | | | Ready to process | 10791 | | <<< R2T | WRITE command | 10792 +------------------+-----------------------+---------------------+ 10793 | Send Data | SCSI Data-out >>> | Receive Data | 10794 +------------------+-----------------------+---------------------+ 10795 | | <<< R2T | Ready for data | 10796 +------------------+-----------------------+---------------------+ 10797 | | <<< R2T | Ready for data | 10798 +------------------+-----------------------+---------------------+ 10799 | Send Data | SCSI Data-out >>> | Receive Data | 10800 +------------------+-----------------------+---------------------+ 10801 | Send Data | SCSI Data-out >>> | Receive Data | 10802 +------------------+-----------------------+---------------------+ 10803 | | <<< SCSI Response |Send Status and Sense| 10804 +------------------+-----------------------+---------------------+ 10805 | Command Complete | | | 10806 +------------------+-----------------------+---------------------+ 10808 B.3 R2TSN/DataSN Use Examples 10810 Output (write) data DataSN/R2TSN Example 10811 +------------------+-----------------------+----------------------+ 10812 |Initiator Function| PDU Type & Content | Target Function | 10813 +------------------+-----------------------+----------------------+ 10814 | Command request |SCSI Command (WRITE)>>>| Receive command | 10815 | (write) | | and queue it | 10816 +------------------+-----------------------+----------------------+ 10817 | | | Process old commands | 10818 +------------------+-----------------------+----------------------+ 10819 | | <<< R2T | Ready for data | 10820 | | R2TSN = 0 | | 10821 +------------------+-----------------------+----------------------+ 10822 | | <<< R2T | Ready for more data | 10823 | | R2TSN = 1 | | 10824 +------------------+-----------------------+----------------------+ 10825 | Send Data | SCSI Data-out >>> | Receive Data | 10826 | for R2TSN 0 | DataSN = 0, F=0 | | 10827 +------------------+-----------------------+----------------------+ 10828 | Send Data | SCSI Data-out >>> | Receive Data | 10829 | for R2TSN 0 | DataSN = 1, F=1 | | 10830 +------------------+-----------------------+----------------------+ 10831 | Send Data | SCSI Data >>> | Receive Data | 10832 | for R2TSN 1 | DataSN = 0, F=1 | | 10833 +------------------+-----------------------+----------------------+ 10834 | | <<< SCSI Response |Send Status and Sense | 10835 | | ExpDataSN = 0 | | 10836 +------------------+-----------------------+----------------------+ 10837 | Command Complete | | | 10838 +------------------+-----------------------+----------------------+ 10840 Input (read) data DataSN Example 10842 +------------------+-----------------------+----------------------+ 10843 |Initiator Function| PDU Type | Target Function | 10844 +------------------+-----------------------+----------------------+ 10845 | Command request |SCSI Command (READ)>>> | | 10846 | (read) | | | 10847 +------------------+-----------------------+----------------------+ 10848 | | | Prepare Data Transfer| 10849 +------------------+-----------------------+----------------------+ 10850 | Receive Data | <<< SCSI Data-in | Send Data | 10851 | | DataSN = 0, F=0 | | 10852 +------------------+-----------------------+----------------------+ 10853 | Receive Data | <<< SCSI Data-in | Send Data | 10854 | | DataSN = 1, F=0 | | 10855 +------------------+-----------------------+----------------------+ 10856 | Receive Data | <<< SCSI Data-in | Send Data | 10857 | | DataSN = 2, F=1 | | 10858 +------------------+-----------------------+----------------------+ 10859 | | <<< SCSI Response |Send Status and Sense | 10860 | | ExpDataSN = 3 | | 10861 +------------------+-----------------------+----------------------+ 10862 | Command Complete | | | 10863 +------------------+-----------------------+----------------------+ 10865 Bidirectional DataSN Example 10867 +------------------+-----------------------+----------------------+ 10868 |Initiator Function| PDU Type | Target Function | 10869 +------------------+-----------------------+----------------------+ 10870 | Command request |SCSI Command >>> | | 10871 | (Read-Write) | Read-Write | | 10872 +------------------+-----------------------+----------------------+ 10873 | | | Process old commands | 10874 +------------------+-----------------------+----------------------+ 10875 | | <<< R2T | Ready to process | 10876 | | R2TSN = 0 | WRITE command | 10877 +------------------+-----------------------+----------------------+ 10878 | * Receive Data | <<< SCSI Data-in | Send Data | 10879 | | DataSN = 0, F=0 | | 10880 +------------------+-----------------------+----------------------+ 10881 | * Receive Data | <<< SCSI Data-in | Send Data | 10882 | | DataSN = 1, F=1 | | 10883 +------------------+-----------------------+----------------------+ 10884 | * Send Data | SCSI Data-out >>> | Receive Data | 10885 | for R2TSN 0 | DataSN = 0, F=1 | | 10886 +------------------+-----------------------+----------------------+ 10887 | | <<< SCSI Response |Send Status and Sense | 10888 | | ExpDataSN = 2 | | 10889 +------------------+-----------------------+----------------------+ 10890 | Command Complete | | | 10891 +------------------+-----------------------+----------------------+ 10893 *) Send data and Receive Data may be transferred simultaneously as 10894 in an atomic Read-Old-Write-New or sequentially as in an atomic 10895 Read-Update-Write (in the latter case the R2T may follow the 10896 received data). 10898 Unsolicited and immediate output (write) data with DataSN Example 10899 +------------------+-----------------------+----------------------+ 10900 |Initiator Function| PDU Type & Content | Target Function | 10901 +------------------+-----------------------+----------------------+ 10902 | Command request |SCSI Command (WRITE)>>>| Receive command | 10903 | (write) |F=0 | and data | 10904 |+ immediate data | | and queue it | 10905 +------------------+-----------------------+----------------------+ 10906 | Send Unsolicited | SCSI Write Data >>> | Receive more Data | 10907 | Data | DataSN = 0, F=1 | | 10908 +------------------+-----------------------+----------------------+ 10909 | | | Process old commands | 10910 +------------------+-----------------------+----------------------+ 10911 | | <<< R2T | Ready for more data | 10912 | | R2TSN = 0 | | 10913 +------------------+-----------------------+----------------------+ 10914 | Send Data | SCSI Write Data >>> | Receive Data | 10915 | for R2TSN 0 | DataSN = 0, F=1 | | 10916 +------------------+-----------------------+----------------------+ 10917 | | <<< SCSI Response |Send Status and Sense | 10918 | | | | 10919 +------------------+-----------------------+----------------------+ 10920 | Command Complete | | | 10921 +------------------+-----------------------+----------------------+ 10923 B.4 CRC Examples 10925 N.B. all Values are Hexadecimal 10927 32 bytes of zeroes: 10929 Byte: 0 1 2 3 10931 0: 00 00 00 00 10932 ... 10933 28: 00 00 00 00 10935 CRC: aa 36 91 8a 10937 32 bytes of ones: 10939 Byte: 0 1 2 3 10940 0: ff ff ff ff 10941 ... 10942 28: ff ff ff ff 10944 CRC: 43 ab a8 62 10946 32 bytes of incrementing 00..1f: 10948 Byte: 0 1 2 3 10950 0: 00 01 02 03 10951 ... 10952 28: 1c 1d 1e 1f 10954 CRC: 4e 79 dd 46 10956 32 bytes of decrementing 1f..00: 10958 Byte: 0 1 2 3 10960 0: 1f 1e 1d 1c 10961 ... 10962 28: 03 02 01 00 10964 CRC: 5c db 3f 11 10966 An iSCSI - SCSI Read (10) Command PDU 10968 Byte: 0 1 2 3 10970 0: 01 c0 00 00 10971 4: 00 00 00 00 10972 8: 00 00 00 00 10973 12: 00 00 00 00 10974 16: 14 00 00 00 10975 20: 00 00 04 00 10976 24: 00 00 00 14 10977 28: 00 00 00 18 10978 32: 28 00 00 00 10979 36: 00 00 00 00 10980 40: 02 00 00 00 10981 44: 00 00 00 00 10983 CRC: 56 3a 96 d9 10985 Appendix C. Login Phase Examples 10987 In the first example, the initiator and target authenticate each 10988 other via Kerberos: 10990 I-> Login (CSG,NSG=0,1 T=1) 10991 InitiatorName=iqn.1999-07.com.os:hostid.77 10992 TargetName=iqn.1999-07.com.example:diskarray.sn.88 10993 AuthMethod=KRB5,SRP,None 10995 T-> Login (CSG,NSG=0,0 T=0) 10996 AuthMethod=KRB5 10998 I-> Login (CSG,NSG=0,1 T=1) 10999 KRB_AP_REQ= 11001 (krb_ap_req contains the Kerberos V5 ticket and authenticator with 11002 MUTUAL-REQUIRED set in the ap-options field) 11004 If the authentication is successful, the target proceeds with: 11006 T-> Login (CSG,NSG=0,1 T=1) 11007 KRB_AP_REP= 11009 (krb_ap_rep is the Kerberos V5 mutual authentication reply) 11011 If the authentication is successful, the initiator may proceed 11012 with: 11014 I-> Login (CSG,NSG=1,0 T=0) FirstBurstLength=8192 11015 T-> Login (CSG,NSG=1,0 T=0) FirstBurstLength=4096 11016 MaxBurstLength=8192 11017 I-> Login (CSG,NSG=1,0 T=0) MaxBurstLength=8192 11018 ... more iSCSI Operational Parameters 11020 T-> Login (CSG,NSG=1,0 T=0) 11021 ... more iSCSI Operational Parameters 11023 And at the end: 11025 I-> Login (CSG,NSG=1,3 T=1) 11026 optional iSCSI parameters 11028 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11030 If the initiators authentication by the target is not successful, 11031 the target responds with: 11033 T-> Login "login reject" 11035 instead of the Login KRB_AP_REP message, and terminates the 11036 connection. 11038 If the targets authentication by the initiator is not successful, 11039 the initiator terminates the connection (without responding to the 11040 Login KRB_AP_REP message). 11042 In the next example only the initiator is authenticated by the 11043 target via Kerberos: 11045 I-> Login (CSG,NSG=0,1 T=1) 11046 InitiatorName=iqn.1999-07.com.os:hostid.77 11047 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11048 AuthMethod=SRP,KRB5,None 11050 T-> Login-PR (CSG,NSG=0,0 T=0) 11051 AuthMethod=KRB5 11053 I-> Login (CSG,NSG=0,1 T=1) 11054 KRB_AP_REQ=krb_ap_req 11056 (MUTUAL-REQUIRED not set in the ap-options field of krb_ap_req) 11058 If the authentication is successful, the target proceeds with: 11060 T-> Login (CSG,NSG=0,1 T=1) 11062 I-> Login (CSG,NSG=1,0 T=0) 11063 ... iSCSI parameters 11065 T-> Login (CSG,NSG=1,0 T=0) 11066 ... iSCSI parameters 11068 . . . 11070 T-> Login (CSG,NSG=1,3 T=1)"login accept" 11072 In the next example, the initiator and target authenticate each 11073 other via SPKM1: 11075 I-> Login (CSG,NSG=0,1 T=1) 11076 InitiatorName=iqn.1999-07.com.os:hostid.77 11077 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11078 AuthMethod=SPKM1,KRB5,None 11080 T-> Login (CSG,NSG=0,0 T=0) 11081 AuthMethod=SPKM1 11083 I-> Login (CSG,NSG=0,0 T=0) 11084 SPKM_REQ= 11086 (spkm-req is the SPKM-REQ token with the mutual-state bit in the 11087 options field of the REQ-TOKEN set) 11089 T-> Login (CSG,NSG=0,0 T=0) 11090 SPKM_REP_TI= 11092 If the authentication is successful, the initiator proceeds: 11094 I-> Login (CSG,NSG=0,1 T=1) 11095 SPKM_REP_IT= 11097 If the authentication is successful, the target proceeds with: 11099 T-> Login (CSG,NSG=0,1 T=1) 11101 The initiator may proceed: 11103 I-> Login (CSG,NSG=1,0 T=0) ... iSCSI parameters 11104 T-> Login (CSG,NSG=1,0 T=0) ... iSCSI parameters 11106 And at the end: 11108 I-> Login (CSG,NSG=1,3 T=1) 11109 optional iSCSI parameters 11111 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11113 If the targets authentication by the initiator is not successful, 11114 the initiator terminates the connection (without responding to the 11115 Login SPKM_REP_TI message). 11117 If the initiators authentication by the target is not successful, 11118 the target responds with: 11120 T-> Login "login reject" 11122 instead of the Login "proceed and change stage" message, and 11123 terminates the connection. 11125 In the next example, the initiator and target authenticate each 11126 other via SPKM2: 11128 I-> Login (CSG,NSG=0,0 T=0) 11129 InitiatorName=iqn.1999-07.com.os:hostid.77 11130 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11131 AuthMethod=SPKM1,SPKM2 11133 T-> Login-PR (CSG,NSG=0,0 T=0) 11134 AuthMethod=SPKM2 11136 I-> Login (CSG,NSG=0,1 T=1) 11137 SPKM_REQ= 11139 (spkm-req is the SPKM-REQ token with the mutual-state bit in the 11140 options field of the REQ-TOKEN not set) 11142 If the authentication is successful, the target proceeds with: 11144 T-> Login (CSG,NSG=0,1 T=1) 11146 The initiator may proceed: 11148 I-> Login (CSG,NSG=1,0 T=0) 11149 ... iSCSI parameters 11151 T-> Login (CSG,NSG=1,0 T=0) 11152 ... iSCSI parameters 11154 And at the end: 11156 I-> Login (CSG,NSG=1,3 T=1) 11157 optional iSCSI parameters 11159 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11161 In the next example, the initiator and target authenticate each 11162 other via SRP: 11164 I-> Login (CSG,NSG=0,1 T=1) 11165 InitiatorName=iqn.1999-07.com.os:hostid.77 11166 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11167 AuthMethod=KRB5,SRP,None 11169 T-> Login-PR (CSG,NSG=0,0 T=0) 11170 AuthMethod=SRP 11172 I-> Login (CSG,NSG=0,0 T=0) 11173 SRP_U= 11174 TargetAuth=Yes 11176 T-> Login (CSG,NSG=0,0 T=0) 11177 SRP_N= 11178 SRP_g= 11179 SRP_s= 11181 I-> Login (CSG,NSG=0,0 T=0) 11182 SRP_A= 11184 T-> Login (CSG,NSG=0,0 T=0) 11185 SRP_B= 11187 I-> Login (CSG,NSG=0,1 T=1) 11188 SRP_M= 11190 If the initiator authentication is successful, the target proceeds: 11192 T-> Login (CSG,NSG=0,1 T=1) 11193 SRP_HM= 11195 Where N, g, s, A, B, M, and H(A | M | K) are defined in 11196 [RFC2945]. 11198 If the target authentication is not successful, the initiator 11199 terminates the connection; otherwise, it proceeds. 11201 I-> Login (CSG,NSG=1,0 T=0) 11202 ... iSCSI parameters 11204 T-> Login (CSG,NSG=1,0 T=0) 11205 ... iSCSI parameters 11207 And at the end: 11209 I-> Login (CSG,NSG=1,3 T=1) 11210 optional iSCSI parameters 11212 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11214 If the initiator authentication is not successful, the target 11215 responds with: 11217 T-> Login "login reject" 11219 Instead of the T-> Login SRP_HM= message and 11220 terminates the connection. 11222 In the next example, only the initiator is authenticated by the 11223 target via SRP: 11225 I-> Login (CSG,NSG=0,1 T=1) 11226 InitiatorName=iqn.1999-07.com.os:hostid.77 11227 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11228 AuthMethod=KRB5,SRP,None 11230 T-> Login-PR (CSG,NSG=0,0 T=0) 11231 AuthMethod=SRP 11233 I-> Login (CSG,NSG=0,0 T=0) 11234 SRP_U= 11235 TargetAuth=No 11237 T-> Login (CSG,NSG=0,0 T=0) 11238 SRP_N= 11239 SRP_g= 11240 SRP_s= 11242 I-> Login (CSG,NSG=0,0 T=0) 11243 SRP_A= 11245 T-> Login (CSG,NSG=0,0 T=0) 11246 SRP_B= 11248 I-> Login (CSG,NSG=0,1 T=1) 11249 SRP_M= 11251 If the initiator authentication is successful, the target 11252 proceeds: 11254 T-> Login (CSG,NSG=0,1 T=1) 11256 I-> Login (CSG,NSG=1,0 T=0) 11258 ... iSCSI parameters 11260 T-> Login (CSG,NSG=1,0 T=0) 11262 ... iSCSI parameters 11264 And at the end: 11266 I-> Login (CSG,NSG=1,3 T=1) 11268 optional iSCSI parameters 11270 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11272 In the next example the initiator and target authenticate each 11273 other via CHAP: 11275 I-> Login (CSG,NSG=0,0 T=0) 11277 InitiatorName=iqn.1999-07.com.os:hostid.77 11279 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11281 AuthMethod=KRB5,CHAP,None 11283 T-> Login-PR (CSG,NSG=0,0 T=0) 11285 AuthMethod=CHAP 11287 I-> Login (CSG,NSG=0,0 T=0) 11289 CHAP_A= 11291 T-> Login (CSG,NSG=0,0 T=0) 11292 CHAP_A= 11293 CHAP_I= 11294 CHAP_C= 11296 I-> Login (CSG,NSG=0,1 T=1) 11298 CHAP_N= 11300 CHAP_R= 11302 CHAP_I= 11304 CHAP_C= 11306 If the initiator authentication is successful, the target 11307 proceeds: 11309 T-> Login (CSG,NSG=0,1 T=1) 11311 CHAP_N= 11313 CHAP_R= 11315 If the target authentication is not successful, the initiator 11316 aborts the connection; otherwise, it proceeds. 11318 I-> Login (CSG,NSG=1,0 T=0) 11320 ... iSCSI parameters 11322 T-> Login (CSG,NSG=1,0 T=0) 11324 ... iSCSI parameters 11326 And at the end: 11328 I-> Login (CSG,NSG=1,3 T=1) 11330 optional iSCSI parameters 11332 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11334 If the initiator authentication is not successful, the target 11335 responds with: 11337 T-> Login "login reject" 11339 Instead of the Login CHAP_R= "proceed and change 11340 stage" message and terminates the connection. 11342 In the next example, only the initiator is authenticated by the 11343 target via CHAP: 11345 I-> Login (CSG,NSG=0,1 T=0) 11347 InitiatorName=iqn.1999-07.com.os:hostid.77 11349 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11351 AuthMethod=KRB5,CHAP,None 11353 T-> Login-PR (CSG,NSG=0,0 T=0) 11354 AuthMethod=CHAP 11356 I-> Login (CSG,NSG=0,0 T=0) 11358 CHAP_A= 11360 T-> Login (CSG,NSG=0,0 T=0) 11361 CHAP_A= 11362 CHAP_I= 11363 CHAP_C= 11365 I-> Login (CSG,NSG=0,1 T=1) 11367 CHAP_N= 11369 CHAP_R= 11371 If the initiator authentication is successful, the target 11372 proceeds: 11374 T-> Login (CSG,NSG=0,1 T=1) 11376 I-> Login (CSG,NSG=1,0 T=0) 11378 ... iSCSI parameters 11380 T-> Login (CSG,NSG=1,0 T=0) 11382 ... iSCSI parameters 11384 And at the end: 11386 I-> Login (CSG,NSG=1,3 T=1) 11388 optional iSCSI parameters 11390 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11392 In the next example, the initiator does not offer any security 11393 parameters. It therefore may offer iSCSI parameters on the Login 11394 PDU with the T bit set to 1, and the target may respond with a 11395 final Login Response PDU immediately: 11397 I-> Login (CSG,NSG=1,3 T=1) 11399 InitiatorName=iqn.1999-07.com.os:hostid.77 11401 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11403 ... iSCSI parameters 11405 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11407 ... ISCSI parameters 11409 In the next example, the initiator does offer security 11410 parameters on the Login PDU, but the target does not choose 11411 any (i.e., chooses the "None" values): 11413 I-> Login (CSG,NSG=0,1 T=1) 11415 InitiatorName=iqn.1999-07.com.os:hostid.77 11416 TargetName=iqn.1999-07.com.example:diskarray.sn.88 11418 AuthMethod=KRB5,SRP,None 11420 T-> Login-PR (CSG,NSG=0,1 T=1) 11422 AuthMethod=None 11424 I-> Login (CSG,NSG=1,0 T=0) 11426 ... iSCSI parameters 11428 T-> Login (CSG,NSG=1,0 T=0) 11430 ... iSCSI parameters 11432 And at the end: 11434 I-> Login (CSG,NSG=1,3 T=1) 11436 optional iSCSI parameters 11438 T-> Login (CSG,NSG=1,3 T=1) "login accept" 11440 Appendix D. SendTargets Operation 11442 To reduce the amount of configuration required on an initiator, 11443 iSCSI provides the SendTargets text request. The initiator uses 11444 the SendTargets request to get a list of targets to which it may 11445 have access, as well as the list of addresses (IP address and TCP 11446 port) on which these targets may be accessed. 11448 To make use of SendTargets, an initiator must first establish one 11449 of two types of sessions. If the initiator establishes the 11450 session using the key "SessionType=Discovery", the session is a 11451 discovery session, and a target name does not need to be specified. 11452 Otherwise, the session is a normal, operational session. The 11453 SendTargets command MUST only be sent during the Full Feature Phase 11454 of a normal or discovery session. 11456 A system that contains targets MUST support discovery sessions on 11457 each of its iSCSI IP address-port pairs, and MUST support the 11458 SendTargets command on the discovery session. In a discovery 11459 session, a target MUST return all path information (IP address-port 11460 pairs and portal group tags) for the targets on the target network 11461 entity which the requesting initiator is authorized to access. 11463 A target MUST support the SendTargets command on operational 11464 sessions; these will only return path information about the target 11465 to which the session is connected, and do not need to return 11466 information about other target names that may be defined in the 11467 responding system. 11469 An initiator MAY make use of the SendTargets as it sees fit. 11471 A SendTargets command consists of a single Text request PDU. 11472 This PDU contains exactly one text key and value. The text key 11473 MUST be SendTargets. The expected response depends upon the value, 11474 as well as whether the session is a discovery or operational 11475 session. 11477 The value must be one of: 11479 All 11480 The initiator is requesting that information on all relevant 11481 targets known to the implementation be returned. This value 11482 MUST be supported on a discovery session, and MUST NOT be 11483 supported on an operational session. 11485 11487 If an iSCSI target name is specified, the session should 11488 respond with addresses for only the named target, if 11489 possible. This value MUST be supported on discovery 11490 sessions. A discovery session MUST be capable of returning 11491 addresses for those targets that would have been returned had 11492 value=All been designated. 11494 11496 The session should only respond with addresses for the target 11497 to which the session is logged in. This MUST be supported on 11498 operational sessions, and MUST NOT return targets other than 11499 the one to which the session is logged in. 11501 The response to this command is a text response that contains a 11502 list of zero or more targets and, optionally, their addresses. 11503 Each target is returned as a target record. A target record begins 11504 with the TargetName text key, followed by a list of TargetAddress 11505 text keys, and bounded by the end of the text response or the next 11506 TargetName key, which begins a new record. No text keys other than 11507 TargetName and TargetAddress are permitted within a SendTargets 11508 response. 11510 For the format of the TargetName, see Section 12.4 - "TargetName". 11512 A discovery session MAY respond to a SendTargets request with its 11513 complete list of targets, or with a list of targets that is based 11514 on the name of the initiator logged in to the session. 11516 A SendTargets response MUST NOT contain target names if there are 11517 no targets for the requesting initiator to access. 11519 Each target record returned includes zero or more TargetAddress 11520 fields. 11522 Each target record starts with one text key of the form: 11524 TargetName= 11526 Followed by zero or more address keys of the form: 11528 TargetAddress=[:], 11531 The hostname-or-ipaddress contains a domain name, IPv4 address, or 11532 IPv6 address, as specified for the TargetAddress key. 11534 A hostname-or-ipaddress duplicated in TargetAddress responses for a 11535 given node (the port is absent or equal) would probably indicate 11536 that multiple address families are in use at once (IPv6 and IPv4). 11538 Each TargetAddress belongs to a portal group, identified by its 11539 numeric portal group tag (as in Section 12.9 - 11540 "TargetPortalGroupTag"). The iSCSI target name, together with this 11541 tag, constitutes the SCSI port identifier; the tag only needs to be 11542 unique within a given targets name list of addresses. 11544 Multiple-connection sessions can span iSCSI addresses that belong 11545 to the same portal group. 11547 Multiple-connection sessions cannot span iSCSI addresses that 11548 belong to different portal groups. 11550 If a SendTargets response reports an iSCSI address for a target, it 11551 SHOULD also report all other addresses in its portal group in the 11552 same response. 11554 A SendTargets text response can be longer than a single Text 11555 Response PDU, and makes use of the long text responses as 11556 specified. 11558 After obtaining a list of targets from the discovery target 11559 session, 11560 an iSCSI initiator may initiate new sessions to log in to the 11561 discovered targets for full operation. The initiator MAY keep the 11562 discovery session open, and MAY send subsequent SendTargets 11563 commands to discover new targets. 11565 Examples: 11567 This example is the SendTargets response from a single target that 11568 has no other interface ports. 11570 Initiator sends text request that contains: 11572 SendTargets=All 11574 Target sends a text response that contains: 11576 TargetName=iqn.1993-11.com.example:diskarray.sn.8675309 11578 All the target had to return in the simple case was the target 11579 name. It is assumed by the initiator that the IP address and TCP 11580 port for this target are the same as used on the current connection 11581 to the default iSCSI target. 11583 The next example has two internal iSCSI targets, each accessible 11584 via two different ports with different IP addresses. The following 11585 is the text response: 11587 TargetName=iqn.1993-11.com.example:diskarray.sn.8675309 11589 TargetAddress=10.1.0.45:3000,1 11591 TargetAddress=10.1.1.45:3000,2 11593 TargetName=iqn.1993-11.com.example:diskarray.sn.1234567 11595 TargetAddress=10.1.0.45:3000,1 11596 TargetAddress=10.1.1.45:3000,2 11598 Both targets share both addresses; the multiple addresses are 11599 likely used to provide multi-path support. The initiator may 11600 connect to either target name on either address. Each of the 11601 addresses has its own portal group tag; they do not support 11602 spanning multiple-connection sessions with each other. Keep in mind 11603 that the portal group tags for the two named targets are 11604 independent of one another; portal group "1" on the first target is 11605 not necessarily the same as portal group "1" on the second target. 11607 In the above example, a DNS host name or an IPv6 address could have 11608 been returned instead of an IPv4 address. 11610 The next text response shows a target that supports spanning 11611 sessions across multiple addresses, and further illustrates the use 11612 of the portal group tags: 11614 TargetName=iqn.1993-11.com.example:diskarray.sn.8675309 11616 TargetAddress=10.1.0.45:3000,1 11618 TargetAddress=10.1.1.46:3000,1 11620 TargetAddress=10.1.0.47:3000,2 11622 TargetAddress=10.1.1.48:3000,2 11624 TargetAddress=10.1.1.49:3000,3 11626 In this example, any of the target addresses can be used to reach 11627 the same target. A single-connection session can be established to 11628 any of these TCP addresses. A multiple-connection session could 11629 span addresses .45 and .46 or .47 and .48, but cannot span any 11630 other combination. A TargetAddress with its own tag (.49) cannot 11631 be combined with any other address within the same session. 11633 This SendTargets response does not indicate whether .49 supports 11634 multiple connections per session; it is communicated via the 11635 MaxConnections text key upon login to the target. 11637 Appendix E. Algorithmic Presentation of Error Recovery Classes 11639 This appendix illustrates the error recovery classes using a 11640 pseudo-programming-language. The procedure names are chosen to be 11641 obvious to most implementers. Each of the recovery classes 11642 described has initiator procedures as well as target procedures. 11643 These algorithms focus on outlining the mechanics of error recovery 11644 classes, and do not exhaustively describe all other aspects/cases. 11645 Examples of this approach are: 11647 - Handling for only certain Opcode types is shown. 11649 - Only certain reason codes (e.g., Recovery in Logout command) 11650 are outlined. 11652 - Resultant cases, such as recovery of Synchronization on a 11653 header digest error are considered out-of-scope in these 11654 algorithms. In this particular example, a header digest 11655 error may lead to connection recovery if some type of sync 11656 and steering layer is not implemented. 11658 These algorithms strive to convey the iSCSI error recovery concepts 11659 in the simplest terms, and are not designed to be optimal. 11661 E.1 General Data Structure and Procedure Description 11663 This section defines the procedures and data structures that are 11664 commonly used by all the error recovery algorithms. The structures 11665 may not be the exhaustive representations of what is required for a 11666 typical implementation. 11668 Data structure definitions - 11669 struct TransferContext { 11670 int TargetTransferTag; 11671 int ExpectedDataSN; 11672 }; 11674 struct TCB { /* task control block */ 11675 Boolean SoFarInOrder; 11676 int ExpectedDataSN; /* used for both R2Ts, and Data */ 11677 int MissingDataSNList[MaxMissingDPDU]; 11678 Boolean FbitReceived; 11679 Boolean StatusXferd; 11680 Boolean CurrentlyAllegiant; 11681 int ActiveR2Ts; 11682 int Response; 11683 char *Reason; 11684 struct TransferContext 11685 TransferContextList[MaxOutStandingR2T]; 11686 int InitiatorTaskTag; 11687 int CmdSN; 11688 int SNACK_Tag; 11689 }; 11691 struct Connection { 11692 struct Session SessionReference; 11693 Boolean SoFarInOrder; 11694 int CID; 11695 int State; 11696 int CurrentTimeout; 11697 int ExpectedStatSN; 11698 int MissingStatSNList[MaxMissingSPDU]; 11699 Boolean PerformConnectionCleanup; 11700 }; 11702 struct Session { 11703 int NumConnections; 11704 int CmdSN; 11705 int Maxconnections; 11706 int ErrorRecoveryLevel; 11707 struct iSCSIEndpoint OtherEndInfo; 11708 struct Connection ConnectionList[MaxSupportedConns]; 11709 }; 11711 Procedure descriptions - 11712 Receive-a-In-PDU(transport connection, inbound PDU); 11713 check-basic-validity(inbound PDU); 11714 Start-Timer(timeout handler, argument, timeout value); 11715 Build-And-Send-Reject(transport connection, bad PDU, reason 11716 code); 11718 E.2 Within-command Error Recovery Algorithms 11720 E.2.1 Procedure Descriptions 11722 Recover-Data-if-Possible(last required DataSN, task control 11723 block); 11724 Build-And-Send-DSnack(task control block); 11725 Build-And-Send-RDSnack(task control block); 11726 Build-And-Send-Abort(task control block); 11727 SCSI-Task-Completion(task control block); 11728 Build-And-Send-A-Data-Burst(transport connection, data- 11729 descriptor, 11730 task control 11731 block); 11732 Build-And-Send-R2T(transport connection, data-descriptor, 11733 task control block); 11734 Build-And-Send-Status(transport connection, task control block); 11735 Transfer-Context-Timeout-Handler(transfer context); 11737 Notes: 11739 - One procedure used in this section: Handle-Status-SNACK- 11740 request is defined in Within-connection recovery algorithms. 11742 - The Response processing pseudo-code, shown in the target 11743 algorithms, applies to all solicited PDUs that carry StatSN - 11744 SCSI Response, Text Response etc. 11746 E.2.2 Initiator Algorithms 11748 Recover-Data-if-Possible(LastRequiredDataSN, TCB) 11749 { 11750 if (operational ErrorRecoveryLevel > 0) { 11751 if (# of missing PDUs is trackable) { 11752 Note the missing DataSNs in TCB. 11753 if (the task spanned a change in 11754 MaxRecvDataSegmentLength) { 11755 if (TCB.StatusXferd is TRUE) 11756 drop the status PDU; 11758 Build-And-Send-RDSnack(TCB); 11759 } else { 11760 Build-And-Send-DSnack(TCB); 11761 } 11762 } else { 11763 TCB.Reason = "Protocol service CRC error"; 11764 } 11765 } else { 11766 TCB.Reason = "Protocol service CRC error"; 11767 } 11768 if (TCB.Reason == "Protocol service CRC error") { 11769 Clear the missing PDU list in the TCB. 11770 if (TCB.StatusXferd is not TRUE) 11771 Build-And-Send-Abort(TCB); 11772 } 11773 } 11775 Receive-a-In-PDU(Connection, CurrentPDU) 11776 { 11777 check-basic-validity(CurrentPDU); 11778 if (Header-Digest-Bad) discard, return; 11779 Retrieve TCB for CurrentPDU.InitiatorTaskTag. 11780 if ((CurrentPDU.type == Data) 11781 or (CurrentPDU.type = R2T)) { 11782 if (Data-Digest-Bad for Data) { 11783 send-data-SNACK = TRUE; 11784 LastRequiredDataSN = CurrentPDU.DataSN; 11785 } else { 11786 if (TCB.SoFarInOrder = TRUE) { 11787 if (current DataSN is expected) { 11788 Increment TCB.ExpectedDataSN. 11789 } else { 11790 TCB.SoFarInOrder = FALSE; 11791 send-data-SNACK = TRUE; 11792 } 11793 } else { 11794 if (current DataSN was considered 11795 missing) { 11796 remove current DataSN from missing PDU 11797 list. 11799 } else if (current DataSN is higher than 11800 expected) { 11801 send-data-SNACK = TRUE; 11802 } else { 11803 discard, return; 11804 } 11805 Adjust TCB.ExpectedDataSN if appropriate. 11806 } 11807 LastRequiredDataSN = CurrentPDU.DataSN - 1; 11808 } 11809 if (send-data-SNACK is TRUE and 11810 task is not already considered failed) { 11811 Recover-Data-if-Possible(LastRequiredDataSN, TCB); 11812 } 11813 if (missing data PDU list is empty) { 11814 TCB.SoFarInOrder = TRUE; 11815 } 11816 if (CurrentPDU.type == R2T) { 11817 Increment ActiveR2Ts for this task. 11818 Create a data-descriptor for the data burst. 11819 Build-And-Send-A-Data-Burst(Connection, data-descriptor, 11820 TCB); 11821 } 11822 } else if (CurrentPDU.type == Response) { 11823 if (Data-Digest-Bad) { 11824 send-status-SNACK = TRUE; 11825 } else { 11826 TCB.StatusXferd = TRUE; 11827 Store the status information in TCB. 11828 if (ExpDataSN does not match) { 11829 TCB.SoFarInOrder = FALSE; 11830 Recover-Data-if-Possible(current DataSN, TCB); 11831 } 11832 if (missing data PDU list is empty) { 11833 TCB.SoFarInOrder = TRUE; 11834 } 11835 } 11836 } else { /* REST UNRELATED TO WITHIN-COMMAND-RECOVERY, NOT 11837 SHOWN */ 11838 } 11839 if ((TCB.SoFarInOrder == TRUE) and 11840 (TCB.StatusXferd == TRUE)) { 11841 SCSI-Task-Completion(TCB); 11842 } 11843 } 11845 E.2.3 Target Algorithms 11847 Receive-a-In-PDU(Connection, CurrentPDU) 11848 { 11849 check-basic-validity(CurrentPDU); 11850 if (Header-Digest-Bad) discard, return; 11851 Retrieve TCB for CurrentPDU.InitiatorTaskTag. 11852 if (CurrentPDU.type == Data) { 11853 Retrieve TContext from CurrentPDU.TargetTransferTag; 11854 if (Data-Digest-Bad) { 11855 Build-And-Send-Reject(Connection, CurrentPDU, 11856 Payload-Digest-Error); 11857 Note the missing data PDUs in MissingDataRange[]. 11858 send-recovery-R2T = TRUE; 11859 } else { 11860 if (current DataSN is not expected) { 11861 Note the missing data PDUs in MissingDataRange[]. 11862 send-recovery-R2T = TRUE; 11863 } 11864 if (CurrentPDU.Fbit == TRUE) { 11865 if (current PDU is solicited) { 11866 Decrement TCB.ActiveR2Ts. 11867 } 11868 if ((current PDU is unsolicited and 11869 data received is less than I/O length and 11870 data received is less than 11871 FirstBurstLength) 11872 or (current PDU is solicited and the length of 11873 this burst is less than expected)) { 11874 send-recovery-R2T = TRUE; 11875 Note the missing data in MissingDataRange[]. 11876 } 11877 } 11878 } 11879 Increment TContext.ExpectedDataSN. 11880 if (send-recovery-R2T is TRUE and 11881 task is not already considered failed) { 11882 if (operational ErrorRecoveryLevel > 0) { 11883 Increment TCB.ActiveR2Ts. 11884 Create a data-descriptor for the data burst 11885 from MissingDataRange. 11886 Build-And-Send-R2T(Connection, data-descriptor, 11887 TCB); 11888 } else { 11889 if (current PDU is the last unsolicited) 11890 TCB.Reason = "Not enough unsolicited data"; 11891 else 11892 TCB.Reason = "Protocol service CRC error"; 11893 } 11894 } 11895 if (TCB.ActiveR2Ts == 0) { 11896 Build-And-Send-Status(Connection, TCB); 11897 } 11898 } else if (CurrentPDU.type == SNACK) { 11899 snack-failure = FALSE; 11900 if (operational ErrorRecoveryLevel > 0) { 11901 if (CurrentPDU.type == Data/R2T) { 11902 if (the request is satisfiable) { 11903 if (request for Data) { 11904 Create a data-descriptor for the data burst 11905 from BegRun and RunLength. 11906 Build-And-Send-A-Data-Burst(Connection, 11907 data-descriptor, TCB); 11908 } else { /* R2T */ 11909 Create a data-descriptor for the data burst 11910 from BegRun and RunLength. 11911 Build-And-Send-R2T(Connection, data- 11912 descriptor, 11913 TCB); 11914 } 11915 } else { 11916 snack-failure = TRUE; 11917 } 11918 } else if (CurrentPDU.type == status) { 11919 Handle-Status-SNACK-request(Connection, 11920 CurrentPDU); 11921 } else if (CurrentPDU.type == DataACK) { 11922 Consider all data upto CurrentPDU.BegRun as 11923 acknowledged. 11924 Free up the retransmission resources for that data. 11925 } else if (CurrentPDU.type == R-Data SNACK) { 11926 Create a data descriptor for a data burst 11927 covering 11928 all unacknowledged data. 11929 Build-And-Send-A-Data-Burst(Connection, 11930 data-descriptor, TCB); 11931 TCB.SNACK_Tag = CurrentPDU.SNACK_Tag; 11932 if (theres no more data to send) { 11933 Build-And-Send-Status(Connection, TCB); 11934 } 11935 } 11936 } else { /* operational ErrorRecoveryLevel = 0 */ 11937 snack-failure = TRUE; 11938 } 11939 if (snack-failure == TRUE) { 11940 Build-And-Send-Reject(Connection, CurrentPDU, 11941 SNACK-Reject); 11942 if (TCB.StatusXferd != TRUE) { 11943 TCB.Reason = "SNACK Rejected"; 11944 Build-And-Send-Status(Connection, TCB); 11945 } 11946 } 11948 } else { /* REST UNRELATED TO WITHIN-COMMAND-RECOVERY, NOT 11949 SHOWN */ 11950 } 11951 } 11953 Transfer-Context-Timeout-Handler(TContext) 11954 { 11955 Retrieve TCB and Connection from TContext. 11956 Decrement TCB.ActiveR2Ts. 11957 if (operational ErrorRecoveryLevel > 0 and 11958 task is not already considered failed) { 11959 Note the missing data PDUs in MissingDataRange[]. 11961 Create a data-descriptor for the data burst 11962 from MissingDataRange[]. 11963 Build-And-Send-R2T(Connection, data-descriptor, TCB); 11964 } else { 11965 TCB.Reason = "Protocol service CRC error"; 11966 if (TCB.ActiveR2Ts = 0) { 11967 Build-And-Send-Status(Connection, TCB); 11968 } 11969 } 11970 } 11972 E.3 Within-connection Recovery Algorithms 11974 E.3.1 Procedure Descriptions 11976 Procedure descriptions: 11977 Recover-Status-if-Possible(transport connection, 11978 currently received PDU); 11979 Evaluate-a-StatSN(transport connection, currently received PDU); 11980 Retransmit-Command-if-Possible(transport connection, CmdSN); 11981 Build-And-Send-SSnack(transport connection); 11982 Build-And-Send-Command(transport connection, task control block); 11983 Command-Acknowledge-Timeout-Handler(task control block); 11984 Status-Expect-Timeout-Handler(transport connection); 11985 Build-And-Send-Nop-Out(transport connection); 11986 Handle-Status-SNACK-request(transport connection, status SNACK 11987 PDU); 11988 Retransmit-Status-Burst(status SNACK, task control block); 11989 Is-Acknowledged(beginning StatSN, run length); 11991 Implementation-specific tunables: 11992 InitiatorProactiveSNACKEnabled 11994 Notes: 11995 - The initiator algorithms only deal with unsolicited Nop-In 11996 PDUs for generating status SNACKs. A solicited Nop-In PDU 11997 has an assigned StatSN, which, when out of order, could 11998 trigger the out of order StatSN handling in Within-command 11999 algorithms, again leading to Recover-Status-if-Possible. 12001 - The pseudo-code shown may result in the retransmission of 12002 unacknowledged commands in more cases than necessary. This 12003 will not, however, affect the correctness of the operation 12004 because the target is required to discard the duplicate 12005 CmdSNs. 12007 - The procedure Build-And-Send-Async is defined in the 12008 Connection recovery algorithms. 12010 - The procedure Status-Expect-Timeout-Handler describes how 12011 initiators may proactively attempt to retrieve the Status if 12012 they so choose. This procedure is assumed to be triggered 12013 much before the standard ULP timeout. 12015 E.3.2 Initiator Algorithms 12017 Recover-Status-if-Possible(Connection, CurrentPDU) 12018 { 12019 if ((Connection.state == LOGGED_IN) and 12020 connection is not already considered failed) 12021 { 12022 if (operational ErrorRecoveryLevel > 0) { 12023 if (# of missing PDUs is trackable) { 12024 Note the missing StatSNs in Connection 12025 that were not already requested with SNACK; 12026 Build-And-Send-SSnack(Connection); 12027 } else { 12028 Connection.PerformConnectionCleanup = TRUE; 12029 } 12030 } else { 12031 Connection.PerformConnectionCleanup = TRUE; 12032 } 12033 if (Connection.PerformConnectionCleanup == TRUE) { 12034 Start-Timer(Connection-Cleanup-Handler, Connection, 0); 12035 } 12036 } 12037 } 12038 Retransmit-Command-if-Possible(Connection, CmdSN) 12039 { 12040 if (operational ErrorRecoveryLevel > 0) { 12041 Retrieve the InitiatorTaskTag, and thus TCB for the CmdSN. 12042 Build-And-Send-Command(Connection, TCB); 12043 } 12044 } 12046 Evaluate-a-StatSN(Connection, CurrentPDU) 12047 { 12048 send-status-SNACK = FALSE; 12049 if (Connection.SoFarInOrder == TRUE) { 12050 if (current StatSN is the expected) { 12051 Increment Connection.ExpectedStatSN. 12052 } else { 12053 Connection.SoFarInOrder = FALSE; 12054 send-status-SNACK = TRUE; 12055 } 12056 } else { 12057 if (current StatSN was considered missing) { 12058 remove current StatSN from the missing list. 12059 } else { 12060 if (current StatSN is higher than expected){ 12061 send-status-SNACK = TRUE; 12062 } else { 12063 send-status-SNACK = FALSE; 12064 discard the PDU; 12065 } 12066 } 12067 Adjust Connection.ExpectedStatSN if appropriate. 12068 if (missing StatSN list is empty) { 12069 Connection.SoFarInOrder = TRUE; 12070 } 12071 } 12072 return send-status-SNACK; 12073 } 12075 Receive-a-In-PDU(Connection, CurrentPDU) 12076 { 12077 check-basic-validity(CurrentPDU); 12078 if (Header-Digest-Bad) discard, return; 12079 Retrieve TCB for CurrentPDU.InitiatorTaskTag. 12080 if (CurrentPDU.type == Nop-In) { 12081 if (the PDU is unsolicited) { 12082 if (current StatSN is not expected) { 12083 Recover-Status-if-Possible(Connection, 12084 CurrentPDU); 12085 } 12086 if (current ExpCmdSN is not Session.CmdSN) { 12087 Retransmit-Command-if-Possible(Connection, 12088 CurrentPDU.ExpCmdSN); 12089 } 12090 } 12091 } else if (CurrentPDU.type == Reject) { 12092 if (it is a data digest error on immediate data) { 12093 Retransmit-Command-if-Possible(Connection, 12095 CurrentPDU.BadPDUHeader.CmdSN); 12096 } 12097 } else if (CurrentPDU.type == Response) { 12098 send-status-SNACK = Evaluate-a-StatSN(Connection, 12099 CurrentPDU); 12100 if (send-status-SNACK == TRUE) 12101 Recover-Status-if-Possible(Connection, CurrentPDU); 12102 } else { /* REST UNRELATED TO WITHIN-CONNECTION-RECOVERY, 12103 * NOT SHOWN */ 12104 } 12105 } 12107 Command-Acknowledge-Timeout-Handler(TCB) 12108 { 12109 Retrieve the Connection for TCB. 12110 Retransmit-Command-if-Possible(Connection, TCB.CmdSN); 12111 } 12113 Status-Expect-Timeout-Handler(Connection) 12114 { 12115 if (operational ErrorRecoveryLevel > 0) { 12116 Build-And-Send-Nop-Out(Connection); 12118 } else if (InitiatorProactiveSNACKEnabled){ 12119 if ((Connection.state == LOGGED_IN) and 12120 connection is not already considered failed) 12121 { 12122 Build-And-Send-SSnack(Connection); 12123 } 12124 } 12125 } 12127 E.3.3 Target Algorithms 12129 Handle-Status-SNACK-request(Connection, CurrentPDU) 12130 { 12131 if (operational ErrorRecoveryLevel > 0) { 12132 if (request for an acknowledged run) { 12133 Build-And-Send-Reject(Connection, CurrentPDU, 12134 Protocol-Error); 12135 } else if (request for an untransmitted run) { 12136 discard, return; 12137 } else { 12138 Retransmit-Status-Burst(CurrentPDU, TCB); 12139 } 12140 } else { 12141 Build-And-Send-Async(Connection, DroppedConnection, 12142 DefaultTime2Wait, 12143 DefaultTime2Retain); 12144 } 12145 } 12147 E.4 Connection Recovery Algorithms 12149 E.4.1 Procedure Descriptions 12151 Build-And-Send-Async(transport connection, reason code, 12152 minimum time, maximum time); 12153 Pick-A-Logged-In-Connection(session); 12154 Build-And-Send-Logout(transport connection, logout connection 12155 identifier, reason code); 12156 PerformImplicitLogout(transport connection, logout connection 12157 identifier, target information); 12158 PerformLogin(transport connection, target information); 12159 CreateNewTransportConnection(target information); 12160 Build-And-Send-Command(transport connection, task control block); 12161 Connection-Cleanup-Handler(transport connection); 12162 Connection-Resource-Timeout-Handler(transport connection); 12163 Quiesce-And-Prepare-for-New-Allegiance(session, task control 12164 block); 12165 Build-And-Send-Logout-Response(transport connection, 12166 CID of connection in recovery, reason 12167 code); 12168 Build-And-Send-TaskMgmt-Response(transport connection, 12169 task mgmt command PDU, response code); 12170 Establish-New-Allegiance(task control block, transport 12171 connection); 12172 Schedule-Command-To-Continue(task control block); 12174 Notes: 12175 - Transport exception conditions, such as unexpected connection 12176 termination, connection reset, and hung connection while the 12177 connection is in the full-feature phase, are all assumed to 12178 be asynchronously signaled to the iSCSI layer using the 12179 Transport_Exception_Handler procedure. 12181 E.4.2 Initiator Algorithms 12183 Receive-a-In-PDU(Connection, CurrentPDU) 12184 { 12185 check-basic-validity(CurrentPDU); 12186 if (Header-Digest-Bad) discard, return; 12187 Retrieve TCB from CurrentPDU.InitiatorTaskTag. 12188 if (CurrentPDU.type == Async) { 12189 if (CurrentPDU.AsyncEvent == ConnectionDropped) { 12190 Retrieve the AffectedConnection for 12191 CurrentPDU.Parameter1. 12192 AffectedConnection.CurrentTimeout = 12193 CurrentPDU.Parameter3; 12194 AffectedConnection.State = CLEANUP_WAIT; 12195 Start-Timer(Connection-Cleanup-Handler, 12196 AffectedConnection, 12197 CurrentPDU.Parameter2); 12198 } else if (CurrentPDU.AsyncEvent == LogoutRequest)) { 12199 AffectedConnection = Connection; 12200 AffectedConnection.State = LOGOUT_REQUESTED; 12201 AffectedConnection.PerformConnectionCleanup = TRUE; 12202 AffectedConnection.CurrentTimeout = 12203 CurrentPDU.Parameter3; 12204 Start-Timer(Connection-Cleanup-Handler, 12205 AffectedConnection, 0); 12206 } else if (CurrentPDU.AsyncEvent == SessionDropped)) { 12207 for (each Connection) { 12208 Connection.State = CLEANUP_WAIT; 12209 Connection.CurrentTimeout = CurrentPDU.Parameter3; 12210 Start-Timer(Connection-Cleanup-Handler, 12211 Connection, CurrentPDU.Parameter2); 12212 } 12213 Session.state = FAILED; 12214 } 12216 } else if (CurrentPDU.type == LogoutResponse) { 12217 Retrieve the CleanupConnection for CurrentPDU.CID. 12218 if (CurrentPDU.Response = failure) { 12219 CleanupConnection.State = CLEANUP_WAIT; 12220 } else { 12221 CleanupConnection.State = FREE; 12222 } 12223 } else if (CurrentPDU.type == LoginResponse) { 12224 if (this is a response to an implicit Logout) { 12225 Retrieve the CleanupConnection. 12226 if (successful) { 12227 CleanupConnection.State = FREE; 12228 Connection.State = LOGGED_IN; 12229 } else { 12230 CleanupConnection.State = CLEANUP_WAIT; 12231 DestroyTransportConnection(Connection); 12232 } 12233 } 12234 } else { /* REST UNRELATED TO CONNECTION-RECOVERY, 12235 * NOT SHOWN */ 12236 } 12237 if (CleanupConnection.State == FREE) { 12238 for (each command that was active on CleanupConnection) { 12239 /* Establish new connection allegiance */ 12240 NewConnection = Pick-A-Logged-In-Connection(Session); 12241 Build-And-Send-Command(NewConnection, TCB); 12242 } 12243 } 12244 } 12246 Connection-Cleanup-Handler(Connection) 12247 { 12248 Retrieve Session from Connection. 12249 if (Connection can still exchange iSCSI PDUs) { 12250 NewConnection = Connection; 12251 } else { 12252 Start-Timer(Connection-Resource-Timeout-Handler, 12253 Connection, Connection.CurrentTimeout); 12254 if (there are other logged-in connections) { 12255 NewConnection = Pick-A-Logged-In- 12256 Connection(Session); 12257 } else { 12258 NewConnection = 12260 CreateTransportConnection(Session.OtherEndInfo); 12261 Initiate an implicit Logout on NewConnection for 12262 Connection.CID. 12263 return; 12264 } 12265 } 12266 Build-And-Send-Logout(NewConnection, Connection.CID, 12267 RecoveryRemove); 12268 } 12270 Transport_Exception_Handler(Connection) 12271 { 12272 Connection.PerformConnectionCleanup = TRUE; 12273 if (the event is an unexpected transport disconnect) { 12274 Connection.State = CLEANUP_WAIT; 12275 Connection.CurrentTimeout = DefaultTime2Retain; 12276 Start-Timer(Connection-Cleanup-Handler, Connection, 12277 DefaultTime2Wait); 12279 } else { 12280 Connection.State = FREE; 12281 } 12282 } 12284 E.4.3 Target Algorithms 12286 Receive-a-In-PDU(Connection, CurrentPDU) 12287 { 12288 check-basic-validity(CurrentPDU); 12289 if (Header-Digest-Bad) discard, return; 12290 else if (Data-Digest-Bad) { 12291 Build-And-Send-Reject(Connection, CurrentPDU, 12292 Payload-Digest-Error); 12293 discard, return; 12294 } 12295 Retrieve TCB and Session. 12296 if (CurrentPDU.type == Logout) { 12297 if (CurrentPDU.ReasonCode = RecoveryRemove) { 12298 Retrieve the CleanupConnection from CurrentPDU.CID). 12299 for (each command active on CleanupConnection) { 12300 Quiesce-And-Prepare-for-New-Allegiance(Session, 12301 TCB); 12302 TCB.CurrentlyAllegiant = FALSE; 12303 } 12304 Cleanup-Connection-State(CleanupConnection); 12305 if ((quiescing successful) and (cleanup successful)) { 12306 Build-And-Send-Logout-Response(Connection, 12307 CleanupConnection.CID, 12308 Success); 12309 } else { 12310 Build-And-Send-Logout-Response(Connection, 12311 CleanupConnection.CID, 12312 Failure); 12313 } 12314 } 12315 } else if ((CurrentPDU.type == Login) and 12316 operational ErrorRecoveryLevel == 2) { 12317 Retrieve the CleanupConnection from CurrentPDU.CID). 12319 for (each command active on CleanupConnection) { 12320 Quiesce-And-Prepare-for-New-Allegiance(Session, 12321 TCB); 12322 TCB.CurrentlyAllegiant = FALSE; 12323 } 12324 Cleanup-Connection-State(CleanupConnection); 12325 if ((quiescing successful) and (cleanup successful)) { 12326 Continue with the rest of the Login processing; 12327 } else { 12328 Build-And-Send-Login-Response(Connection, 12329 CleanupConnection.CID, Target 12330 Error); 12331 } 12332 } 12333 } else if (CurrentPDU.type == TaskManagement) { 12334 if (CurrentPDU.function == "TaskReassign") { 12335 if (Session.ErrorRecoveryLevel < 2) { 12336 Build-And-Send-TaskMgmt-Response(Connection, 12337 CurrentPDU, "Allegiance reassignment 12338 not supported"); 12339 } else if (task is not found) { 12340 Build-And-Send-TaskMgmt-Response(Connection, 12341 CurrentPDU, "Task not in task set"); 12342 } else if (task is currently allegiant) { 12343 Build-And-Send-TaskMgmt-Response(Connection, 12344 CurrentPDU, "Task still allegiant"); 12345 } else { 12346 Establish-New-Allegiance(TCB, Connection); 12347 TCB.CurrentlyAllegiant = TRUE; 12348 Schedule-Command-To-Continue(TCB); 12349 } 12350 } 12351 } else { /* REST UNRELATED TO CONNECTION-RECOVERY, 12352 * NOT SHOWN */ 12353 } 12354 } 12356 Transport_Exception_Handler(Connection) 12357 { 12358 Connection.PerformConnectionCleanup = TRUE; 12359 if (the event is an unexpected transport disconnect) { 12360 Connection.State = CLEANUP_WAIT; 12361 Start-Timer(Connection-Resource-Timeout-Handler, 12362 Connection, 12364 (DefaultTime2Wait+DefaultTime2Retain)); 12365 if (this Session has full-feature phase connections left) 12366 { 12367 DifferentConnection = 12368 Pick-A-Logged-In-Connection(Session); 12369 Build-And-Send-Async(DifferentConnection, 12370 DroppedConnection, DefaultTime2Wait, 12371 DefaultTime2Retain); 12372 } 12373 } else { 12374 Connection.State = FREE; 12375 } 12376 } 12378 Appendix F. Clearing Effects of Various Events on Targets 12380 F.1 Clearing Effects on iSCSI Objects 12382 The following tables describe the target behavior on receiving the 12383 events specified in the rows of the table. The second table is an 12384 extension of the first table and defines clearing actions for more 12385 objects on the same events. The legend is: 12387 Y = Yes (cleared/discarded/reset on the event specified in 12388 the row). Unless otherwise noted, the clearing action is 12389 only applicable for the issuing initiator port. 12391 N = No (not affected on the event specified in the row, 12392 i.e., stays at previous value). 12394 NA = Not Applicable or Not Defined. 12396 +-----+-----+-----+-----+-----+ 12397 |IT(1)|IC(2)|CT(5)|ST(6)|PP(7)| 12398 +---------------------+-----+-----+-----+-----+-----+ 12399 |connection failure(8)|Y |Y |N |N |Y | 12400 +---------------------+-----+-----+-----+-----+-----+ 12401 |connection state |NA |NA |Y |N |NA | 12402 |timeout (9) | | | | | | 12403 +---------------------+-----+-----+-----+-----+-----+ 12404 |session timeout/ |Y |Y |Y |Y |Y(14)| 12405 |closure/reinstatement| | | | | | 12406 |(10) | | | | | | 12407 +---------------------+-----+-----+-----+-----+-----+ 12408 |session continuation |NA |NA |N(11)|N |NA | 12409 |(12) | | | | | | 12410 +---------------------+-----+-----+-----+-----+-----+ 12411 |successful connection|Y |Y |Y |N |Y(13)| 12412 |close logout | | | | | | 12413 +---------------------+-----+-----+-----+-----+-----+ 12414 |session failure (18) |Y |Y |N |N |Y | 12415 +---------------------+-----+-----+-----+-----+-----+ 12416 |successful recovery |Y |Y |N |N |Y(13)| 12417 |Logout | | | | | | 12418 +---------------------+-----+-----+-----+-----+-----+ 12419 |failed Logout |Y |Y |N |N |Y | 12420 +---------------------+-----+-----+-----+-----+-----+ 12421 |connection Login |NA |NA |NA |Y(15)|NA | 12422 |(leading) | | | | | | 12423 +---------------------+-----+-----+-----+-----+-----+ 12424 |connection Login |NA |NA |N(11)|N |Y | 12425 |(non-leading) | | | | | | 12426 +---------------------+-----+-----+-----+-----+-----+ 12427 |target cold reset(16)|Y(20)|Y |Y |Y |Y | 12428 +---------------------+-----+-----+-----+-----+-----+ 12429 |target warm reset(16)|Y(20)|Y |Y |Y |Y | 12430 +---------------------+-----+-----+-----+-----+-----+ 12431 |LU reset(19) |Y(20)|Y |Y |Y |Y | 12432 +---------------------+-----+-----+-----+-----+-----+ 12433 |powercycle(16) |Y |Y |Y |Y |Y | 12434 +---------------------+-----+-----+-----+-----+-----+ 12436 1. Incomplete TTTs - Target Transfer Tags on which the target is 12437 still expecting PDUs to be received. Examples include TTTs 12438 received via R2T, NOP-IN, etc. 12440 2. Immediate Commands - immediate commands, but waiting for 12441 execution on a target. For example, Abort Task Set. 12443 5. Connection Tasks - tasks that are active on the iSCSI connection 12444 in question. 12446 6. Session Tasks - tasks that are active on the entire iSCSI 12447 session. A union of "connection tasks" on all participating 12448 connections. 12450 7. Partial PDUs (if any) - PDUs that are partially sent and waiting 12451 for transport window credit to complete the transmission. 12453 8. Connection failure is a connection exception condition - one of 12454 the transport connections shutdown, transport connections reset, 12455 or transport connections timed out, which abruptly terminated 12456 the iSCSI full-feature phase connection. A connection failure 12457 always takes the connection state machine to the CLEANUP_WAIT 12458 state. 12460 9. Connection state timeout happens if a connection spends more time 12461 than agreed upon during Login negotiation in the CLEANUP_WAIT 12462 state, and this takes the connection to the FREE state (M1 12463 transition in connection cleanup state diagram). 12465 10. These are defined in Section 5.3.5. 12467 11. This clearing effect is "Y" only if it is a connection 12468 reinstatement and the operational ErrorRecoveryLevel is less 12469 than 2. 12471 12. Session continuation is defined in Section 5.3.5. 12473 13. This clearing effect is only valid if the connection is being 12474 logged out on a different connection and when the connection 12475 being logged out on the target may have some partial PDUs 12476 pending to be sent. In all other cases, the effect is "NA". 12477 14. This clearing effect is only valid for a "close the session" 12478 logout in a multi-connection session. In all other cases, the 12479 effect is "NA". 12480 15. Only applicable if this leading connection login is a session 12481 reinstatement. If this is not the case, it is "NA". 12482 16. This operation affects all logged-in initiators. 12483 18. Session failure is defined in Section 5.3.5. 12484 19. This operation affects all logged-in initiators and the clearing 12485 effects are only applicable to the LU being reset. 12487 20. With Standard multi-task abort semantics (Section 3.2.3.3), a 12488 target warm reset or a target cold reset or an LU reset would 12489 clear the active TTTs upon completion. However, the FastAbort 12490 multi-task abort semantics defined by Section 3.2.3.4 do not 12491 guarantee that the active TTTs are cleared by the end of the 12492 reset operations. In fact, the FastAbort semantics are designed 12493 to allow clearing the TTTs in a "lazy" fashion after the TMF 12494 Response is delivered. Thus, when TaskReporting=FastAbort 12495 (Section 12.23) is operational on a session, the clearing 12496 effects of reset operations on "Incomplete TTTs" is "N". 12498 +-----+-----+-----+-----+-----+ 12499 |DC(1)|DD(2)|SS(3)|CS(4)|DS(5)| 12500 +---------------------+-----+-----+-----+-----+-----+ 12501 |connection failure |N |Y |N |N |N | 12502 +---------------------+-----+-----+-----+-----+-----+ 12503 |connection state |Y |NA |Y |N |NA | 12504 |timeout | | | | | | 12505 +---------------------+-----+-----+-----+-----+-----+ 12506 |session timeout/ |Y |Y |Y(7) |Y |NA | 12507 |closure/reinstatement| | | | | | 12508 +---------------------+-----+-----+-----+-----+-----+ 12509 |session continuation |N(11)|NA*12|NA |N |NA*13| 12510 +---------------------+-----+-----+-----+-----+-----+ 12511 |successful connection|Y |Y |Y |N |NA | 12512 |close Logout | | | | | | 12513 +---------------------+-----+-----+-----+-----+-----+ 12514 |session failure |N |Y |N |N |N | 12515 +---------------------+-----+-----+-----+-----+-----+ 12516 |successful recovery |Y |Y |Y |N |N | 12517 |Logout | | | | | | 12518 +---------------------+-----+-----+-----+-----+-----+ 12519 |failed Logout |N |Y(9) |N |N |N | 12520 +---------------------+-----+-----+-----+-----+-----+ 12521 |connection Login |NA |NA |N(8) |N(8) |NA | 12522 |(leading | | | | | | 12523 +---------------------+-----+-----+-----+-----+-----+ 12524 |connection Login |N(11)|NA*12|N(8) |N |NA*13| 12525 |(non-leading) | | | | | | 12526 +---------------------+-----+-----+-----+-----+-----+ 12527 |target cold reset |Y |Y |Y |Y(10)|NA | 12528 +---------------------+-----+-----+-----+-----+-----+ 12529 |target warm reset |Y |Y |N |N |NA | 12530 +---------------------+-----+-----+-----+-----+-----+ 12531 |LU reset |N |Y |N |N |N | 12532 +---------------------+-----+-----+-----+-----+-----+ 12533 |powercycle |Y |Y |Y |Y(10)|NA | 12534 +---------------------+-----+-----+-----+-----+-----+ 12536 1. Discontiguous Commands - commands allegiant to the connection in 12537 question and waiting to be reordered in the iSCSI layer. All Ys 12538 in this column assume that the task causing the event (if indeed 12539 the event is the result of a task) is issued as an immediate 12540 command, because the discontiguities can be ahead of the task. 12542 2. Discontiguous Data - data PDUs received for the task in question 12543 and waiting to be reordered due to prior discontiguities in DataSN. 12545 3. StatSN 12547 4. CmdSN 12549 5. DataSN 12551 7. It clears the StatSN on all the connections. 12553 8. This sequence number is instantiated on this event. 12555 9. A logout failure drives the connection state machine to the 12556 CLEANUP_WAIT state, similar to the connection failure event. Hence, 12557 it has a similar effect on this and several other protocol aspects. 12559 10. This is cleared by virtue of the fact that all sessions with 12560 all initiators are terminated. 12562 11. This clearing effect is "Y" if it is a connection 12563 reinstatement. 12565 12. This clearing effect is "Y" only if it is a connection 12566 reinstatement and the operational ErrorRecoveryLevel is 2. 12568 13. This clearing effect is "N" only if it is a connection 12569 reinstatement and the operational ErrorRecoveryLevel is 2. 12571 F.2 Clearing Effects on SCSI Objects 12573 The only iSCSI protocol action that can effect clearing actions on 12574 SCSI objects is the "I_T nexus loss" notification (Section 4.3.5.1 12575 Loss of Nexus notification). [SPC3] describes the clearing effects 12576 of this notification on a variety of SCSI attributes. In addition, 12577 SCSI standards documents (such as [SAM2] and [SBC]) define 12578 additional clearing actions that may take place for several SCSI 12579 objects on SCSI events such as LU resets and power-on resets. 12581 Since iSCSI defines a target cold reset as a protocol-equivalent to 12582 a target power-cycle, the iSCSI target cold reset must also be 12583 considered as the power-on reset event in interpreting the actions 12584 defined in the SCSI standards. 12586 When the iSCSI session is reconstructed (between the same SCSI 12587 ports with the same nexus identifier) reestablishing the same I_T 12588 nexus, all SCSI objects that are defined to not clear on the "I_T 12589 nexus loss" notification event, such as persistent reservations, 12590 are automatically associated to this new session. 12592 Acknowledgments 12594 Several individuals on the original IPS Working Group made 12595 significant contributions to the original RFCs 3720, 3980, 4850 and 12596 5048. 12598 Specifically, the authors of the original RFCs - which this draft 12599 consolidates into a single document - were the following: 12601 RFC 3720: Julian Satran, Kalman Meth, Costa Sapuntzakis, 12602 Mallikarjun Chadalapaka, Efri Zeidner 12604 RFC 3980: Marjorie Krueger, Mallikarjun Chadalapaka, Rob Elliott 12606 RFC 4850: David Wysochanski 12608 RFC 5048: Mallikarjun Chadalapaka 12610 Place holder: We would like to acknowledge the following 12611 individuals who contributed to this revised draft. 12613 Authors' Addresses 12615 Mallikarjun Chadalapaka 12616 Hewlett-Packard Company 12617 8000 Foothills Blvd. 12618 Roseville, CA 95747-5668, USA 12619 Phone: +1.916.785.5621 12620 E-mail: cbm@chadalapaka.com 12621 Julian Satran 12622 E-mail: Julian.Satran@gmail.com 12624 Kalman Meth 12625 Haifa University Campus - Mount Carmel 12626 MATAM - Advanced Technology Center 12627 Haifa 31905, Israel 12628 Phone +972.4.829.6341 12629 E-mail: meth@il.ibm.com 12631 Comments may be sent to Mallikarjun Chadalapaka 12633 Acknowledgement 12635 Funding for the RFC Editor function is currently provided by the 12636 Internet Society