idnits 2.17.1 draft-ietf-ips-iscsi-impl-guide-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 16. -- Found old boilerplate from RFC 3978, Section 5.5, updated by RFC 4748 on line 1143. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 1155. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 1163. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 1169. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The abstract seems to indicate that this document updates RFC3720, but the header doesn't have an 'Updates:' line to match this. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (July 2007) is 6130 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 3720 (Obsoleted by RFC 7143) ** Downref: Normative reference to an Informational RFC: RFC 3721 -- Possible downref: Non-RFC (?) normative reference: ref. 'SPC3' Summary: 3 errors (**), 0 flaws (~~), 2 warnings (==), 9 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTERNET DRAFT Mallikarjun Chadalapaka 3 draft-ietf-ips-iscsi-impl-guide-04.txt Hewlett-Packard Co. 4 Editor 6 Expires 7 July 2007 9 iSCSI Corrections and Clarifications 11 Status of this Memo 12 By submitting this Internet-Draft, each author represents 13 that any applicable patent or other IPR claims of which he or 14 she is aware have been or will be disclosed, and any of which 15 he or she becomes aware will be disclosed, in accordance with 16 Section 6 of BCP 79. 18 Internet-Drafts are working documents of the Internet 19 Engineering Task Force (IETF), its areas, and its working 20 groups. Note that other groups may also distribute working 21 documents as Internet-Drafts. 23 Internet-Drafts are draft documents valid for a maximum of 24 six months and may be updated, replaced, or obsoleted by 25 other documents at any time. It is inappropriate to use 26 Internet-Drafts as reference material or to cite them other 27 than as "work in progress." 29 The list of current Internet-Drafts can be accessed at 30 http://www.ietf.org/1id-abstracts.html 32 The list of Internet-Draft Shadow Directories can be accessed 33 at http://www.ietf.org/shadow.html. 35 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 36 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 37 "OPTIONAL" in this document are to be interpreted as 38 described in [RFC2119]. 40 Abstract 41 iSCSI is a SCSI transport protocol and maps the SCSI family 42 of application protocols onto TCP/IP. RFC 3720 defines the 43 iSCSI protocol. This document compiles the clarifications to 44 the original protocol definition in RFC 3720 to serve as a 45 companion document for the iSCSI implementers. This document 46 updates RFC 3720 and the text in this document supersedes the 47 text in RFC 3720 when the two differ. 49 Table of Contents 51 1 Definitions and acronyms ...............................5 52 1.1 Definitions ............................................5 53 1.2 Acronyms ...............................................5 54 2 Introduction ...........................................7 55 3 iSCSI semantics for SCSI tasks .........................8 56 3.1 Residual handling ......................................8 57 3.1.1 Overview..............................................8 58 3.1.2 SCSI REPORT LUNS and Residual Overflow................9 59 3.2 R2T Ordering ..........................................10 60 3.3 SCSI Protocol Interface Model for Response Ordering ...11 61 3.3.1 Model Description....................................11 62 3.3.2 iSCSI Semantics with the Interface Model.............11 63 3.3.3 Current List of Fenced Response Use Cases............12 64 4 Task Management .......................................14 65 4.1 Requests Affecting Multiple Tasks .....................14 66 4.1.1 Scope of affected tasks..............................14 67 4.1.2 Clarified multi-task abort semantics.................14 68 4.1.3 Updated multi-task abort semantics...................16 69 4.1.4 Implementation considerations........................18 70 4.1.5 Rationale behind the new semantics...................19 71 5 Discovery semantics ...................................21 72 5.1 Error Recovery for Discovery Sessions .................21 73 5.2 Reinstatement Semantics of Discovery Sessions .........21 74 5.2.1 Unnamed Discovery Sessions...........................22 75 5.2.2 Named Discovery Sessions.............................22 76 5.3 Target PDUs during Discovery ..........................23 77 6 Negotiation and Others ................................24 78 6.1 TPGT Values ...........................................24 79 6.2 SessionType Negotiation ...............................24 80 6.3 Understanding NotUnderstood ...........................24 81 6.4 Outstanding Negotiation Exchanges .....................25 82 7 iSCSI Error Handling and Recovery .....................26 83 7.1 ITT ...................................................26 84 7.2 Format Errors .........................................26 85 7.3 Digest Errors .........................................26 86 7.4 Message Error Checking ................................27 87 8 iSCSI PDUs ............................................28 88 8.1 Asynchronous Message ..................................28 89 8.2 Reject ................................................28 90 9 Login/Text Operational Text Keys ......................29 91 9.1 TaskReporting .........................................29 92 10 Security Considerations ...............................31 93 11 IANA Considerations ...................................32 94 12 References and Bibliography ...........................33 95 12.1 Normative References.................................33 96 12.2 Informative References...............................33 97 13 Editor's Address ......................................34 98 14 Acknowledgements ......................................35 99 15 Full Copyright Statement ..............................36 100 16 Intellectual Property Statement .......................37 102 1 Definitions and acronyms 104 1.1 Definitions 106 I/O Buffer - A buffer that is used in a SCSI Read or Write 107 operation so SCSI data may be sent from or received into 108 that buffer. For a read or write data transfer to take 109 place for a task, an I/O Buffer is required on the 110 initiator and at least one required on the target. 112 SCSI-Presented Data Transfer Length (SPDTL): SPDTL is the 113 aggregate data length of the data that SCSI layer 114 logically "presents" to iSCSI layer for a Data-in or 115 Data-out transfer in the context of a SCSI task. For a 116 bidirectional task, there are two SPDTL values - one for 117 Data-in and one for Data-out. Note that the notion of 118 "presenting" includes immediate data per the data 119 transfer model in [SAM2], and excludes overlapping data 120 transfers, if any, requested by the SCSI layer. 122 Third-party: A term used in this document to denote nexus 123 objects (I_T or I_T_L) and iSCSI sessions which reap the 124 side-effects of actions that take place in the context of 125 a separate iSCSI session, while being third parties to 126 the action that caused the side-effects. One example of 127 a Third-party session is an iSCSI session hosting an 128 I_T_L nexus to an LU that is reset with an LU Reset TMF 129 via a separate I_T nexus. 131 1.2 Acronyms 133 Acronym Definition 135 ------------------------------------------------------------- 137 EDTL Expected Data Transfer Length 139 IANA Internet Assigned Numbers Authority 141 IETF Internet Engineering Task Force 143 I/O Input - Output 145 IP Internet Protocol 147 iSCSI Internet SCSI 149 iSER iSCSI Extensions for RDMA 151 ITT Initiator Task Tag 153 LO Leading Only 155 LU Logical Unit 157 LUN Logical Unit Number 159 PDU Protocol Data Unit 161 RDMA Remote Direct Memory Access 163 R2T Ready To Transfer 165 R2TSN Ready To Transfer Sequence Number 167 RFC Request For Comments 169 SAM SCSI Architecture Model 171 SCSI Small Computer Systems Interface 173 SN Sequence Number 175 SNACK Selective Negative Acknowledgment - also 177 Sequence Number Acknowledgement for data 179 TCP Transmission Control Protocol 181 TMF Task Management Function 183 TTT Target Transfer Tag 185 UA Unit Attention 187 2 Introduction 189 Several iSCSI implementations had been built after [RFC3720] was 190 published and the iSCSI community is now richer by the resulting 191 implementation expertise. The goal of this document is to 192 leverage this expertise both to offer clarifications to the 193 [RFC3720] semantics and to address defects in [RFC3720] as 194 appropriate. This document intends to offer critical guidance 195 to implementers with regard to non-obvious iSCSI implementation 196 aspects so as to improve interoperability and accelerate iSCSI 197 adoption. This document, however, does not purport to be an 198 all-encompassing iSCSI how-to guide for implementers, nor a 199 complete revision of [RFC3720]. This document instead is 200 intended as a companion document to [RFC3720] for the iSCSI 201 implementers. 203 iSCSI implementers are required to reference [RFC3722] and 204 [RFC3723] in addition to [RFC3720] for mandatory requirements. 205 In addition, [RFC3721] also contains useful information for 206 iSCSI implementers. The text in this document, however, updates 207 and supersedes the text in [RFC3720] and [RFC3721] whenever 208 there is such a question. 210 3 iSCSI semantics for SCSI tasks 212 3.1 Residual handling 214 Section 10.4.1 of [RFC3720] defines the notion of "residuals" 215 and specifies how the residual information should be encoded 216 into the SCSI Response PDU in Counts and Flags fields. Section 217 3.1.1 clarifies the intent of [RFC3720] and explains the general 218 principles. Section 3.1.2 describes the residual handling in 219 the REPORT LUNS scenario. 221 3.1.1 Overview 223 SCSI-Presented Data Transfer Length (SPDTL) is the term this 224 document uses (see section 1.1 for definition) to represent the 225 aggregate data length that the target SCSI layer attempts to 226 transfer using the local iSCSI layer for a task. Expected Data 227 Transfer Length (EDTL) is the iSCSI term that represents the 228 length of data that the iSCSI layer expects to transfer for a 229 task. EDTL is specified in the SCSI Command PDU. 231 When SPDTL = EDTL for a task, the target iSCSI layer completes 232 the task with no residuals. Whenever SPDTL differs from EDTL 233 for a task, that task is said to have a residual. 235 If SPDTL > EDTL for a task, iSCSI Overflow MUST be signaled in 236 the SCSI Response PDU as specified in [RFC3720]. Residual Count 237 MUST be set to the numerical value of (SPDTL - EDTL). 239 If SPDTL < EDTL for a task, iSCSI Underflow MUST be signaled in 240 the SCSI Response PDU as specified in [RFC3720]. Residual Count 241 MUST be set to the numerical value of (EDTL - SPDTL). 243 Note that the Overflow and Underflow scenarios are independent 244 of Data-in and Data-out. Either scenario is logically possible 245 in either direction of data transfer. 247 3.1.2 SCSI REPORT LUNS and Residual Overflow 249 This section discusses the residual overflow issues citing the 250 example of SCSI REPORT LUNS command. Note however that there 251 are several SCSI commands (e.g. INQUIRY) with ALLOCATION LENGTH 252 fields following the same underlying rules. The semantics in 253 the rest of the section apply to all such SCSI commands. 255 The specification of the SCSI REPORT LUNS command requires that 256 the SCSI target limit the amount of data transferred to a 257 maximum size (ALLOCATION LENGTH) provided by the initiator in 258 the REPORT LUNS CDB. If the Expected Data Transfer Length 259 (EDTL) in the iSCSI header of the SCSI Command PDU for a REPORT 260 LUNS command is set to at least as large as that ALLOCATION 261 LENGTH, the SCSI layer truncation prevents an iSCSI Residual 262 Overflow from occurring. A SCSI initiator can detect that such 263 truncation has occurred via other information at the SCSI layer. 264 The rest of the section elaborates this required behavior. 266 iSCSI uses the (O) bit (bit 5) in the Flags field of the SCSI 267 Response and the last SCSI Data-In PDUs to indicate that that an 268 iSCSI target was unable to transfer all of the SCSI data for a 269 command to the initiator because the amount of data to be 270 transferred exceeded the EDTL in the corresponding SCSI Command 271 PDU (see Section 10.4.1 of [RFC3720]). 273 The SCSI REPORT LUNS command requests a target SCSI layer to 274 return a logical unit inventory (LUN list) to the initiator SCSI 275 layer (see section 6.21 of SPC-3 [SPC3]). The size of this LUN 276 list may not be known to the initiator SCSI layer when it issues 277 the REPORT LUNS command; to avoid transfer of more LUN list data 278 than the initiator is prepared for, the REPORT LUNS CDB contains 279 an ALLOCATION LENGTH field to specify the maximum amount of data 280 to be transferred to the initiator for this command. If the 281 initiator SCSI layer has under-estimated the number of logical 282 units at the target, it is possible that the complete logical 283 unit inventory does not fit in the specified ALLOCATION LENGTH. 284 In this situation, section 4.3.3.6 in [SPC3] requires that the 285 target SCSI layer "shall terminate transfers to the Data-In 286 Buffer" when the number of bytes specified by the ALLOCATION 287 LENGTH field have been transferred. 289 Therefore, in response to a REPORT LUNS command, the SCSI layer 290 at the target presents at most ALLOCATION LENGTH bytes of data 291 (logical unit inventory) to iSCSI for transfer to the initiator. 292 For a REPORT LUNS command, if the iSCSI EDTL is at least as 293 large as the ALLOCATION LENGTH, the SCSI truncation ensures that 294 the EDTL will accommodate all of the data to be transferred. If 295 all of the logical unit inventory data presented to the iSCSI 296 layer - i.e. the data remaining after any SCSI truncation - is 297 transferred to the initiator by the iSCSI layer, an iSCSI 298 Residual Overflow has not occurred and the iSCSI (O) bit MUST 299 NOT be set in the SCSI Response or final SCSI Data-Out PDU. 300 This is not a new requirement but is already required by the 301 combination of [RFC3720] with the specification of the REPORT 302 LUNS command in [SPC3]. If the iSCSI EDTL is larger than the 303 ALLOCATION LENGTH however in this scenario, note that the iSCSI 304 Underflow MUST be signaled in the SCSI Response PDU. An iSCSI 305 Underflow MUST also be signaled when the iSCSI EDTL is equal to 306 ALLOCATION LENGTH but the logical unit inventory data presented 307 to the iSCSI layer is smaller than ALLOCATION LENGTH. 309 The LUN LIST LENGTH field in the logical unit inventory (first 310 field in the inventory) is not affected by truncation of the 311 inventory to fit in ALLOCATION LENGTH; this enables a SCSI 312 initiator to determine that the received inventory is incomplete 313 by noticing that the LUN LIST LENGTH in the inventory is larger 314 than the ALLOCATION LENGTH that was sent in the REPORT LUNS CDB. 315 A common initiator behavior in this situation is to re-issue the 316 REPORT LUNS command with a larger ALLOCATION LENGTH. 318 3.2 R2T Ordering 320 Section 10.8 in [RFC3720] says the following: 322 The target may send several R2T PDUs. It, therefore, can have 323 a number of pending data transfers. The number of outstanding 324 R2T PDUs are limited by the value of the negotiated key 325 MaxOutstandingR2T. Within a connection, outstanding R2Ts MUST 326 be fulfilled by the initiator in the order in which they were 327 received. 329 The quoted [RFC3720] text was unclear on the scope of 330 applicability - either per task, or across all tasks on a 331 connection - and may be interpreted as either. This section is 332 intended to clarify that the scope of applicability of the 333 quoted text is a task. No R2T ordering relationship - either in 334 generation at the target or in fulfilling at the initiator - 335 across tasks is implied. I.e., outstanding R2Ts within a task 336 MUST be fulfilled by the initiator in the order in which they 337 were received on a connection. 339 3.3 SCSI Protocol Interface Model for Response Ordering 341 Whenever an iSCSI session is composed of multiple connections, 342 the Response PDUs (task responses or TMF responses) originating 343 in the target SCSI layer are distributed onto the multiple 344 connections by the target iSCSI layer according to iSCSI 345 connection allegiance rules. This process generally may not 346 preserve the ordering of the responses by the time they are 347 delivered to the initiator SCSI layer. Since ordering is not 348 expected across SCSI responses anyway, this approach works fine 349 in the general case. However to address the special cases where 350 some ordering is desired by the SCSI layer, the following SCSI 351 protocol interface model is assumed. 353 3.3.1 Model Description 355 SCSI protocol layer instructs the SCSI transport layer of a 356 "Response Fence" associated with the response in question when 357 the "Send Command Complete" protocol data service (SAM-2, clause 358 5.4.2) and "Task Management Function Executed" (SAM-2, clause 359 6.9) service are invoked. The Response Fence flag instructs the 360 SCSI transport layer that the following conditions must be met 361 in delivering the response message: 363 (1) Response with Response Fence MUST chronologically be 364 delivered after all the "preceding" responses on the 365 I_T_L nexus, if the preceding responses are delivered at 366 all, to the application client on the initiator. 368 (2) Response with Response Fence MUST chronologically be 369 delivered prior to all the "following" responses on the 370 I_T_L nexus. 372 The "preceding" and "following" notions refer to the order of 373 hand-off of a response message from the target SCSI protocol 374 layer to the target SCSI transport (e.g. iSCSI) layer. 376 3.3.2 iSCSI Semantics with the Interface Model 378 Whenever the TaskReporting key (section 9.1) is negotiated to 379 ResponseFence or FastAbort for an iSCSI session, the target 380 iSCSI layer MUST perform the actions described in this section 381 on all tasks related to that session. A target iSCSI layer MUST 382 do the following on sensing the "Response Fence" flag associated 383 with a response being handed down from the target SCSI layer: 385 a) If it is a single-connection session, no special processing 386 is required. Standard SCSI Response PDU build process 387 happens. 389 b) If it is a multi-connection session, target iSCSI layer 390 takes note of last-sent and unacknowledged StatSN on each 391 of the connections in the iSCSI session, and waits for 392 acknowledgement (SHOULD solicit for acknowledgement by way 393 of a Nop-In) of each such StatSN to clear the fence. SCSI 394 response with the Response Fence flag must be sent to the 395 initiator only after receiving acknowledgements for each of 396 the unacknowledged StatSNs. 398 c) Target iSCSI layer must wait for an acknowledgement of the 399 SCSI Response PDU that carried the response which the 400 target SCSI layer marked with the Response Fence flag. The 401 fence must be considered cleared after receiving the 402 acknowledgement. 404 d) All further status processing for the LU is resumed only 405 after clearing the fence. If any new responses for the 406 I_T_L nexus are received from the SCSI layer before the 407 fence is cleared, those Response PDUs must be held and 408 queued at the iSCSI layer until the fence is cleared. 410 3.3.3 Current List of Fenced Response Use Cases 412 This section lists the fenced response use cases that iSCSI 413 implementations must comply with. However, this is not an 414 exhaustive enumeration. It is expected that as SCSI protocol 415 specifications evolve, the specifications will specify when 416 response fencing is required on a case-by-case basis. 418 Whenever the TaskReporting key (section 9.1) is negotiated to 419 ResponseFence or FastAbort for an iSCSI session, target iSCSI 420 layer MUST assume that Response Fence flag is set by the target 421 SCSI layer on the following SCSI completion messages handed down 422 to it: 424 1. The first completion message carrying the UA after the 425 multi-task abort on issuing and third-party sessions. 427 2. The TMF Response carrying the multi-task TMF Response on 428 the issuing session. 430 3. The completion message indicating ACA establishment on the 431 issuing session. 433 4. The first completion message carrying the ACA ACTIVE status 434 after ACA establishment on issuing and third-party 435 sessions. 437 5. The TMF Response carrying the Clear ACA response on the 438 issuing session. 440 6. The response to a PERSISTENT RESERVE OUT/PREEMPT AND ABORT 441 command 443 Note: Due to the absence of ACA-related fencing requirements in 444 [RFC3720], initiator implementations SHOULD NOT use ACA on 445 multi-connection iSCSI sessions to targets complying only with 446 [RFC3720]. Initiators which want to employ ACA on multi- 447 connection iSCSI sessions SHOULD first assess response fencing 448 behavior via negotiating for ResponseFence or FastAbort values 449 for the TaskReporting (section 9.1) key. 451 4 Task Management 453 4.1 Requests Affecting Multiple Tasks 455 This section clarifies and updates the original text in section 456 10.6.2 of [RFC3720]. The clarified semantics (section 4.1.2) 457 are a superset of the protocol behavior required in the original 458 text and all iSCSI implementations MUST support the new 459 behavior. The updated semantics (section 4.1.3) on the other 460 hand are mandatory only when the new key TaskReporting (section 461 9.1) is negotiated to "FastAbort". 463 4.1.1 Scope of affected tasks 465 This section defines the notion of "affected tasks" in multi- 466 task abort scenarios. Scope definitions in this section apply 467 to both the clarified protocol behavior (section 4.1.2) and the 468 updated protocol behavior (section 4.1.3). 470 ABORT TASK SET: All outstanding tasks for the I_T_L nexus 471 identified by the LUN field in the ABORT TASK SET TMF 472 Request PDU. 474 CLEAR TASK SET: All outstanding tasks in the task set for 475 the LU identified by the LUN field in the CLEAR TASK SET 476 TMF Request PDU. See [SPC3] for the definition of a "task 477 set". 479 LOGICAL UNIT RESET: All outstanding tasks from all 480 initiators for the LU identified by the LUN field in the 481 LOGICAL UNIT RESET Request PDU. 483 TARGET WARM RESET/TARGET COLD RESET: All outstanding tasks 484 from all initiators across all LUs to which the TMF-issuing 485 session has access to on the SCSI target device hosting the 486 iSCSI session. 488 Usage: an "ABORT TASK SET TMF Request PDU" in the preceding text 489 is an iSCSI TMF Request PDU with the "Function" field set to 490 "ABORT TASK SET" as defined in [RFC3720]. Similar usage is 491 employed for other scope descriptions. 493 4.1.2 Clarified multi-task abort semantics 495 All iSCSI implementations MUST support the protocol behavior 496 defined in this section as the default behavior. The execution 497 of ABORT TASK SET, CLEAR TASK SET, LOGICAL UNIT RESET, TARGET 498 WARM RESET, and TARGET COLD RESET TMF Requests consists of the 499 following sequence of actions in the specified order on the 500 specified party. 502 The initiator iSCSI layer: 504 a. MUST continue to respond to each TTT received for the 505 affected tasks. 507 b. Should receive any responses that the target may provide 508 for some tasks among the affected tasks (may process them 509 as usual because they are guaranteed to have 510 chronologically originated prior to the TMF response). 512 c. Should receive the TMF Response concluding all the tasks in 513 the set of affected tasks. 515 The target iSCSI layer: 517 a. MUST wait for responses on currently valid target transfer 518 tags of the affected tasks from the issuing initiator. MAY 519 wait for responses on currently valid target transfer tags 520 of the affected tasks from third-party initiators. 522 b. MUST wait (concurrent with the wait in Step.a) for all 523 commands of the affected tasks to be received based on the 524 CmdSN ordering. SHOULD NOT wait for new commands on 525 third-party affected sessions - only the instantiated tasks 526 have to be considered for the purpose of determining the 527 affected tasks. In the case of target-scoped requests 528 (i.e. TARGET WARM RESET and TARGET COLD RESET), all the 529 commands that are not yet received on the issuing session 530 in the command stream however can be considered to have 531 been received with no command waiting period - i.e. the 532 entire CmdSN space up to the CmdSN of the task management 533 function can be "plugged". 535 c. MUST propagate the TMF request to and receive the response 536 from the target SCSI layer. 538 d. MUST address the Response Fence flag on the TMF Response on 539 issuing session as defined in 3.3.2. 541 e. MUST address the Response Fence flag on the first post-TMF 542 Response on third-party sessions as defined in 3.3.2. If 543 some tasks originate from non-iSCSI I_T_L nexuses then the 544 means by which the target ensures that all affected tasks 545 have returned their status to the initiator are defined by 546 the specific non-iSCSI transport protocol(s). 548 Implementation note: Technically, the TMF servicing is complete 549 in Step.d. Data transfers corresponding to terminated tasks may 550 however still be in progress on third-party iSCSI sessions even 551 at the end of Step.e. TMF Response MUST NOT be sent by the 552 target iSCSI layer before the end of Step.d, and MAY be sent at 553 the end of Step.d despite these outstanding data transfers until 554 after Step.e. 556 4.1.3 Updated multi-task abort semantics 558 Protocol behavior defined in this section MUST be implemented by 559 all iSCSI implementations complying with this document. 560 Protocol behavior defined in this section MUST be exhibited by 561 iSCSI implementations on an iSCSI session when they negotiate 562 the TaskReporting (section 9.1) key to "FastAbort" on that 563 session. The execution of ABORT TASK SET, CLEAR TASK SET, 564 LOGICAL UNIT RESET, TARGET WARM RESET, and TARGET COLD RESET TMF 565 Requests consists of the following sequence of actions in the 566 specified order on the specified party. 568 The initiator iSCSI layer: 570 a. MUST NOT send any more Data-Out PDUs for affected tasks on 571 the issuing connection of the issuing iSCSI session once 572 the TMF is sent to the target. 574 b. Should receive any responses that the target may provide 575 for some tasks among the affected tasks (may process them 576 as usual because they are guaranteed to have 577 chronologically originated prior to the TMF response). 579 c. MUST respond to each Async Message PDU with AsyncEvent=5 as 580 defined in section 8.1. 582 d. Should receive the TMF Response concluding all the tasks in 583 the set of affected tasks. 585 The target iSCSI layer: 587 a. MUST wait for all commands of the affected tasks to be 588 received based on the CmdSN ordering on the issuing 589 session. SHOULD NOT wait for new commands on third-party 590 affected sessions - only the instantiated tasks have to be 591 considered for the purpose of determining the affected 592 tasks. In the case of target-scoped requests (i.e. TARGET 593 WARM RESET and TARGET COLD RESET), all the commands that 594 are not yet received on the issuing session in the command 595 stream can be considered to have been received with no 596 command waiting period - i.e. the entire CmdSN space up to 597 the CmdSN of the task management function can be "plugged". 598 b. MUST propagate the TMF request to and receive the response 599 from the target SCSI layer. 601 c. MUST leave all active "affected TTTs" (i.e. active TTTs 602 associated with affected tasks) valid. 604 d. MUST generate an Asynchronous Message PDU with AsyncEvent=5 605 (section 8.1) on: 606 i) each connection of each third-party session to which at 607 least one affected task is allegiant, and 608 ii) each connection except the issuing connection of the 609 issuing session that has at least one allegiant affected 610 task. 612 If there are multiple affected LUs (say due to a target 613 reset), then one Async Message PDU MUST be sent for each 614 such LU on each connection that has at least one allegiant 615 affected task. 617 e. MUST address the Response Fence flag on the TMF Response on 618 issuing session as defined in 3.3.2. 620 f. MUST address the Response Fence flag on the first post-TMF 621 Response on third-party sessions as defined in 3.3.2. If 622 some tasks originate from non-iSCSI I_T_L nexuses then the 623 means by which the target ensures that all affected tasks 624 have returned their status to the initiator are defined by 625 the specific non-iSCSI transport protocol(s). 627 g. MUST free up the affected TTTs (and STags, if applicable) 628 and the corresponding buffers, if any, once it receives 629 each associated Nop-Out acknowledgement that the initiator 630 generated in response to each Async Message. 632 Implementation note: Technically, the TMF servicing is complete 633 in Step.e. Data transfers corresponding to terminated tasks may 634 however still be in progress even at the end of Step.f. TMF 635 Response MUST NOT be sent by the target iSCSI layer before the 636 end of Step.e, and MAY be sent at the end of Step.e despite 637 these outstanding Data transfers until Step.g. Step.g specifies 638 an event to free up any such resources that may have been 639 reserved to support outstanding data transfers. 641 4.1.3.1 Clearing effects update 643 Appendix F.1 of [RFC3720] specifies the clearing effects of 644 target and LU resets on "Incomplete TTTs" as "Y". This meant 645 that a target warm reset or a target cold reset or an LU reset 646 would clear the active TTTs upon completion. The 647 TaskReporting=FastAbort (section 9.1) semantics defined by this 648 section however do not guarantee that the active TTTs are 649 cleared by the end of the reset operations. In fact, the new 650 semantics are designed to allow clearing the TTTs in a "lazy" 651 fashion after the TMF Response is delivered. Thus, when 652 TaskReporting=FastAbort is operational on a session, the 653 clearing effects of reset operations on "Incomplete TTTs" is 654 "N". 656 4.1.4 Implementation considerations 658 Both in clarified semantics (section 4.1.2) and updated 659 semantics (section 4.1.3), there may be outstanding data 660 transfers even after the TMF completion is reported on the 661 issuing session. In the case of iSCSI/iSER [iSER], these would 662 be tagged data transfers for STags not owned by any active 663 tasks. Whether or not real buffers support these data transfers 664 is implementation-dependent. However, the data transfers 665 logically MUST be silently discarded by the target iSCSI layer 666 in all cases. A target MAY, on an implementation-defined 667 internal timeout, also choose to drop the connections on which 668 it did not receive the expected Data-out sequences (section 669 4.1.2) or Nop-Out acknowledgements (section 4.1.3) so as to 670 reclaim the associated buffer, STag and TTT resources as 671 appropriate. 673 4.1.5 Rationale behind the new semantics 675 There are fundamentally three basic objectives behind the 676 semantics specified in section 4.1.2 and section 4.1.3. 678 1. Maintaining an ordered command flow I_T nexus abstraction 679 to the target SCSI layer even with multi-connection 680 sessions. 682 o Target iSCSI processing of a TMF request must maintain 683 the single flow illusion. Target behavior in Step.b 684 of section 4.1.2 and Step.a of section 4.1.3 685 correspond to this objective. 687 2. Maintaining a single ordered response flow I_T nexus 688 abstraction to the initiator SCSI layer even with multi- 689 connection sessions when one response (i.e. TMF response) 690 could imply the status of other unfinished tasks from the 691 initiator's perspective. 693 o Target must ensure that the initiator does not see 694 "old" task responses (that were placed on the wire 695 chronologically earlier than the TMF Response) after 696 seeing the TMF response. Target behavior in Step.d of 697 section 4.1.2 and Step.e of section 4.1.3 correspond 698 to this objective. 700 o Whenever the result of a TMF action is visible across 701 multiple I_T_L nexuses, [SAM2] requires the SCSI 702 device server to trigger a UA on each of the other 703 I_T_L nexuses. Once an initiator is notified of such 704 an UA, the application client on the receiving 705 initiator is required to clear its task state (clause 706 5.5 in [SAM2]) for the affected tasks. It would thus 707 be inappropriate to deliver a SCSI Response for a task 708 after the task state is cleared on the initiator, i.e. 709 after the UA is notified. The UA notification 710 contained in the first SCSI Response PDU on each 711 affected Third-party I_T_L nexus after the TMF action 712 thus MUST NOT pass the affected task responses on any 713 of the iSCSI sessions accessing the LU. Target 714 behavior in Step.e of section 4.1.2 and Step.f of 715 section 4.1.3 correspond to this objective. 717 3. Draining all active TTTs corresponding to affected tasks 718 in a deterministic fashion. 720 o Data-out PDUs with stale TTTs arriving after the tasks 721 are terminated can create a buffer management problem 723 even for traditional iSCSI implementations, and is 724 fatal for the connection for iSCSI/iSER 725 implementations. Either the termination of affected 726 tasks should be postponed until the TTTs are retired 727 (as in Step.a of section 4.1.2), or the TTTs and the 728 buffers should stay allocated beyond task termination 729 to be deterministically freed up later (as in Step.c 730 and Step.g of section 4.1.3). 732 The only other notable optimization is the plugging. If all 733 tasks on an I_T nexus will be aborted anyway (as with a target 734 reset), there is no need to wait to receive all commands to plug 735 the CmdSN holes. Target iSCSI layer can simply plug all missing 736 CmdSN slots and move on with TMF processing. The first 737 objective (maintaining a single ordered command flow) is still 738 met with this optimization because target SCSI layer only sees 739 ordered commands. 741 5 Discovery semantics 743 5.1 Error Recovery for Discovery Sessions 745 The negotiation of the key ErrorRecoveryLevel is not required 746 for Discovery sessions - i.e. for sessions that negotiated 747 "SessionType=Discovery" - because the default value of 0 is 748 necessary and sufficient for Discovery sessions. It is however 749 possible that some legacy iSCSI implementations might attempt to 750 negotiate the ErrorRecoveryLevel key on Discovery sessions. 751 When such a negotiation attempt is made by the remote side, a 752 compliant iSCSI implementation MUST propose a value of 0 (zero) 753 in response. The operational ErrorRecoveryLevel for Discovery 754 sessions thus MUST be 0. This naturally follows from the 755 functionality constraints [RFC3720] imposes on Discovery 756 sessions. 758 5.2 Reinstatement Semantics of Discovery Sessions 760 Discovery sessions are intended to be relatively short-lived. 761 Initiators are not expected to establish multiple Discovery 762 sessions to the same iSCSI Network Portal (see [RFC3720]). An 763 initiator may use the same iSCSI Initiator Name and ISID when 764 establishing different unique sessions with different targets 765 and/or different portal groups. This behavior is discussed in 766 Section 9.1.1 of [RFC3720] and is, in fact, encouraged as 767 conservative reuse of ISIDs. ISID RULE in [RFC3720] states that 768 there must not be more than one session with a matching 4-tuple: 769 . While 770 the spirit of the ISID RULE applies to Discovery sessions the 771 same as it does for Normal sessions, note that some Discovery 772 sessions differ from the Normal sessions in two important 773 aspects: 775 Because [RFC3720] allows a Discovery session to be 776 established without specifying a TargetName key in the 777 Login Request PDU (let us call such a session an "Unnamed" 778 Discovery session), there is no Target Node context to 779 enforce the ISID RULE. 781 Portal Groups are defined only in the context of a Target 782 Node. When the TargetName key is NULL-valued (i.e. not 783 specified), the TargetPortalGroupTag thus cannot be 784 ascertained to enforce the ISID RULE. 786 The following sections describe the two scenarios - Named 787 Discovery sessions and Unnamed Discovery sessions - separately. 789 5.2.1 Unnamed Discovery Sessions 791 For Unnamed Discovery sessions, neither the TargetName nor the 792 TargetPortalGroupTag is available to the targets in order to 793 enforce the ISID RULE. So the following rule applies. 795 UNNAMED ISID RULE: Targets MUST enforce the uniqueness of the 796 following 4-tuple for Unnamed Discovery sessions: 797 . The following 798 semantics are implied by this uniqueness requirement. 800 Targets SHOULD allow concurrent establishment of one Discovery 801 session with each of its Network Portals by the same initiator 802 port with a given iSCSI Node Name and an ISID. Each of the 803 concurrent Discovery sessions, if established by the same 804 initiator port to other Network Portals, MUST be treated as 805 independent sessions - i.e. one session MUST NOT reinstate the 806 other. 808 A new Unnamed Discovery session that has a matching 809 to an existing 810 discovery session MUST reinstate the existing Unnamed Discovery 811 session. Note thus that only an Unnamed Discovery session may 812 reinstate an Unnamed Discovery session. 814 5.2.2 Named Discovery Sessions 816 For a Named Discovery session, the TargetName key is specified 817 by the initiator and thus the target can unambiguously ascertain 818 the TargetPortalGroupTag as well. Since all the four elements 819 of the 4-tuple are known, the ISID RULE MUST be enforced by 820 targets with no changes from [RFC3720] semantics. A new session 821 with a matching thus will reinstate an existing session. 823 Note in this case that any new iSCSI session (Discovery or 824 Normal) with the matching 4-tuple may reinstate an existing 825 Named Discovery iSCSI session. 827 5.3 Target PDUs during Discovery 829 Targets SHOULD NOT send any responses other than a Text Response 830 and Logout Response on a Discovery session, once in full feature 831 phase. 833 Implementation Note: A target may simply drop the connection in 834 a Discovery session when it would have requested a Logout via an 835 Async Message on Normal sessions. 837 6 Negotiation and Others 839 6.1 TPGT Values 841 [SAM2] and [SAM3] specifications incorrectly note in their 842 informative text that TPGT value should be non-zero, although 843 [RFC3720] allows the value of zero for TPGT. This section is to 844 clarify that zero value is expressly allowed as a legal value 845 for TPGT. This discrepancy currently stands corrected in 846 [SAM4]. 848 6.2 SessionType Negotiation 850 During the Login phase, the SessionType key is offered by the 851 initiator to choose the type of session it wants to create with 852 the target. The target may accept or reject the offer. 853 Depending on the type of the session, a target may decide on 854 resources to allocate and the security to enforce etc. for the 855 session. If the SessionType key is thus going to be offered as 856 "Discovery", it SHOULD be offered in the initial Login request 857 by the initiator. 859 6.3 Understanding NotUnderstood 861 [RFC3720] defines NotUnderstood as a valid answer during a 862 negotiation text key exchange between two iSCSI nodes. 863 NotUnderstood has the reserved meaning that the sending side did 864 not understand the key semantics. This section seeks to clarify 865 that NotUnderstood is a valid answer for both declarative and 866 negotiated keys. The general iSCSI philosophy is that 867 comprehension precedes processing for any iSCSI key. A proposer 868 of an iSCSI key, negotiated or declarative, in a text key 869 exchange MUST thus be able to properly handle a NotUnderstood 870 response. 872 The proper way to handle a NotUnderstood response varies 873 depending on the lineage and type of the key. All keys defined 874 in [RFC3720] MUST be supported by all compliant implementations; 875 a NotUnderstood answer on any of the [RFC3720] keys therefore 876 MUST be considered a protocol error and handled accordingly. 877 For all other later keys, a NotUnderstood answer concludes the 878 negotiation for a negotiated key whereas for a declarative key, 879 a NotUnderstood answer simply informs the declarer of lack of 880 comprehension by the receiver. In either case, a NotUnderstood 881 answer always requires that the protocol behavior associated 882 with that key be not used within the scope of the key 883 (connection/session) by either side. 885 6.4 Outstanding Negotiation Exchanges 887 There was some uncertainty around the number of outstanding 888 Login Response PDUs on a connection. [RFC3720] offers the 889 analogy of SCSI linked commands to Login and Text negotiations 890 in sections 5.3 and 10.10.3 respectively, but does not make it 891 fully explicit. This section is to offer a clarification in 892 this regard. 894 There MUST NOT be more than one outstanding Login Request or 895 Login Response or Text Request or Text Response PDU on an iSCSI 896 connection. An outstanding PDU in this context is one that has 897 not been acknowledged by the remote iSCSI side. 899 7 iSCSI Error Handling and Recovery 901 7.1 ITT 903 Section 10.19 in [RFC3720] mentions this in passing but noted 904 here again for making it obvious since the semantics apply to 905 the initiators in general. An ITT value of 0xffffffff is 906 reserved and MUST NOT be assigned for a task by the initiator. 907 The only instance it may be seen on the wire is in a target- 908 initiated NOP-In PDU (and in the initiator response to that PDU 909 if necessary). 911 7.2 Format Errors 913 Section 6.6 of [RFC3720] discusses format error handling. This 914 section elaborates on the "inconsistent" PDU field contents 915 noted in [RFC3720]. 917 All initiator-detected PDU construction errors MUST be 918 considered as format errors. Some examples of such errors are: 920 - NOP-In with a valid TTT but an invalid LUN 922 - NOP-In with a valid ITT (i.e. a NOP-In response) and also a 923 valid TTT 925 - SCSI Response PDU with Status=CHECK CONDITION, but 926 DataSegmentLength = 0 928 7.3 Digest Errors 930 Section 6.7 of [RFC3720] discusses digest error handling. It 931 states that "No further action is necessary for initiators if 932 the discarded PDU is an unsolicited PDU (e.g., Async, Reject)" 933 on detecting a payload digest error. This is incorrect. 935 An Asynchronous Message PDU or a Reject PDU carries the next 936 StatSN value on an iSCSI connection, advancing the StatSN. When 937 an initiator discards one of these PDUs due to a payload digest 938 error, the entire PDU including the header MUST be discarded. 939 Consequently, the initiator MUST treat the exception like a loss 940 of any other solicited response PDU - i.e. it MUST use one of 941 the following options noted in [RFC3720]: 943 a) Request PDU retransmission with a status SNACK. 945 b) Logout the connection for recovery and continue the 946 tasks on a different connection instance. 948 c) Logout to close the connection (abort all the commands 949 associated with the connection). 951 7.4 Message Error Checking 953 There has been some uncertainty on the extent to which incoming 954 messages have to be checked for protocol errors, beyond what is 955 strictly required for processing the inbound message. This 956 section addresses that question. 958 Unless [RFC3720] or this draft requires it, an iSCSI 959 implementation is not required to do an exhaustive protocol 960 conformance checking on an incoming iSCSI PDU. The iSCSI 961 implementation especially is not required to double-check the 962 remote iSCSI implementation's conformance to protocol 963 requirements. 965 8 iSCSI PDUs 967 8.1 Asynchronous Message 969 This section defines additional semantics for the Asynchronous 970 Message PDU defined in section 10.9 of [RFC3720] using the same 971 conventions. 973 The following new legal value for AsyncEvent is defined: 975 5: all active tasks for LU with matching LUN field in the Async 976 Message PDU are being terminated. 978 The receiving initiator iSCSI layer MUST respond to this Message 979 by taking the following steps in order. 981 i) Stop Data-Out transfers on that connection for all active 982 TTTs for the affected LUN quoted in the Async Message 983 PDU. 984 ii) Acknowledge the StatSN of the Async Message PDU via a 985 Nop-Out PDU with ITT=0xffffffff (i.e. non-ping flavor), 986 while copying the LUN field from Async Message to Nop- 987 Out. 989 8.2 Reject 991 Section 10.17.1 of [RFC3720] specifies the Reject reason code of 992 0x0b with an explanation of "Negotiation Reset". At this point, 993 we do not see any legitimate iSCSI protocol use case for using 994 this reason code. Thus reason code 0x0b MUST be considered as 995 deprecated and MUST NOT be used by any new implementations. 997 9 Login/Text Operational Text Keys 999 This section follows the same conventions as section 12 of 1000 [RFC3720]. 1002 9.1 TaskReporting 1004 Use: LO 1005 Senders: Initiator and Target 1006 Scope: SW 1008 Irrelevant when: SessionType=Discovery 1009 TaskReporting= 1011 Default is Legacy. 1012 Result function is AND. 1014 This key is used to negotiate the task completion reporting 1015 semantics from the SCSI target. Following table describes the 1016 semantics an iSCSI target MUST support for respective negotiated 1017 key values. Whenever this key is negotiated, at least the 1018 Legacy and ResponseFence values MUST be offered as options by 1019 the negotiation originator. 1021 +--------------+------------------------------------------+ 1022 | Name | Description | 1023 +--------------+------------------------------------------+ 1024 | Legacy | RFC 3720-compliant semantics. Response | 1025 | | fencing is not guaranteed and fast | 1026 | | completion of multi-task aborting is not | 1027 | | supported | 1028 +--------------+------------------------------------------+ 1029 | ResponseFence| Response Fence (section 3.3.1) semantics | 1030 | | MUST be supported in reporting task | 1031 | | completions | 1032 +--------------+------------------------------------------+ 1033 | FastAbort | Updated fast multi-task abort semantics | 1034 | | defined in section 4.1.3 MUST be | 1035 | | supported. Support for Response Fence is| 1036 | | implied - i.e. section 3.3.1 semantics | 1037 | | MUST be supported as well | 1038 +--------------+------------------------------------------+ 1039 When TaskReporting is not negotiated to FastAbort, the default 1040 behavior is to use the [RFC3720] TMF semantics as clarified in 1041 section 4.1.2. 1043 10 Security Considerations 1045 This document does not introduce any new security considerations 1046 other than those already noted in [RFC3720]. Consequently, all 1047 the iSCSI-related security text in [RFC3723] is also directly 1048 applicable to this document. 1050 11 IANA Considerations 1052 This draft does not have any specific IANA considerations. 1054 12 References and Bibliography 1056 12.1 Normative References 1058 [RFC3720] Satran, J., Meth, K., Sapuntzakis, C., Chadalapaka, 1059 M., and E. Zeidner, "Internet Small Computer Systems 1060 Interface (iSCSI)", RFC 3720, April 2004. 1062 [RFC3721] Bakke, M., Hafner, J., Hufferd, J., Voruganti, K., 1063 and M. Krueger, "Internet Small Computer Systems Interface 1064 (iSCSI) Naming and Discovery", RFC 3721, April 2004. 1066 [SPC3] T10/1416-D, SCSI Primary Commands-3. 1068 [RFC2119] S. Bradner, "Key words for use in RFCs to Indicate 1069 Requirement Levels", March 1997. 1071 12.2 Informative References 1073 [RFC3723] Aboba, B., Tseng, J., Walker, J., Rangan, V., and 1074 F. Travostino, "Securing Block Storage Protocols over IP", 1075 RFC 3723, April 2004. 1077 [RFC3722] Bakke, M., "String Profile for Internet Small 1078 Computer Systems Interface (iSCSI) Names", RFC 3722, April 1079 2004. 1081 [iSER] Ko, M., Chadalapaka, M., Elzur, U., Shah, H., Thaler, 1082 P., J. Hufferd, "iSCSI Extensions for RDMA", IETF Internet 1083 Draft draft-ietf-ips-iser-04.txt (work in progress), June 1084 2005. 1086 [RFC2119] Bradner, S. "Key Words for use in RFCs to Indicate 1087 Requirement Levels", BCP 14, RFC 2119, March 1997. 1089 [SAM2] ANSI INCITS 366-2003, SCSI Architecture Model-2 (SAM- 1090 2). 1092 [SAM3] ANSI INCITS 402-2005, SCSI Architecture Model-3 (SAM- 1093 3). 1095 [SAM4] T10 Project: 1683-D, SCSI Architecture Model-4 (SAM- 1096 4), Work in Progress. 1098 13 Editor's Address 1100 Mallikarjun Chadalapaka 1101 Hewlett-Packard Company 1102 8000 Foothills Blvd. 1103 Roseville, CA 95747-5668, USA 1104 Phone: +1-916-785-5621 1105 E-mail: cbm@rose.hp.com 1107 14 Acknowledgements 1109 The IP Storage (ips) Working Group in the Transport Area of 1110 IETF has been responsible for defining the iSCSI protocol 1111 (apart from a host of other relevant IP Storage protocols). 1112 The editor acknowledges the contributions of the entire 1113 working group. 1115 The following individuals directly contributed to identifying 1116 [RFC3720] issues and/or suggesting resolutions to the issues 1117 clarified in this document: David Black (REPORT LUNS/overflow 1118 semantics, ACA semantics, TMF semantics), Gwendal Grignou 1119 (TMF scope), Mike Ko (digest error handling for Asynchronous 1120 Message), Dmitry Fomichev (reserved ITT), Bill Studenmund 1121 (residual handling, discovery semantics), Ken Sandars 1122 (discovery semantics), Bob Russell (discovery semantics), 1123 Julian Satran (discovery semantics), Rob Elliott (T10 1124 liaison, R2T ordering), Joseph Pittman(TMF scope), Somesh 1125 Gupta (multi-task abort semantics). This document benefited 1126 from all these contributions. 1128 15 Full Copyright Statement 1130 Copyright (C) The IETF Trust (2007). This document is 1131 subject to the rights, licenses and restrictions contained in 1132 BCP 78, and except as set forth therein, the authors retain 1133 all their rights. 1135 This document and the information contained herein are 1136 provided on an "AS IS" basis and THE CONTRIBUTOR, THE 1137 ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), 1138 THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET 1139 ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR 1140 IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE 1141 USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR 1142 ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A 1143 PARTICULAR PURPOSE. 1145 16 Intellectual Property Statement 1147 The IETF takes no position regarding the validity or scope of 1148 any Intellectual Property Rights or other rights that might 1149 be claimed to pertain to the implementation or use of the 1150 technology described in this document or the extent to which 1151 any license under such rights might or might not be 1152 available; nor does it represent that it has made any 1153 independent effort to identify any such rights. Information 1154 on the procedures with respect to rights in RFC documents can 1155 be found in BCP 78 and BCP 79. 1157 Copies of IPR disclosures made to the IETF Secretariat and 1158 any assurances of licenses to be made available, or the 1159 result of an attempt made to obtain a general license or 1160 permission for the use of such proprietary rights by 1161 implementers or users of this specification can be obtained 1162 from the IETF on-line IPR repository at 1163 http://www.ietf.org/ipr. 1165 The IETF invites any interested party to bring to its 1166 attention any copyrights, patents or patent applications, or 1167 other proprietary rights that may cover technology that may 1168 be required to implement this standard. Please address the 1169 information to the IETF at ietf-ipr@ietf.org.