Julian,
Your primary concern appears to be around unpredictability of interactions between issuing and third-party sessions.
Issuing and third-party sessions can assume one of three values for TaskReporting: Legacy, ResponseFence, FastAbort
That makes for 9 combinations of issuing+third-party modes. The homogeneous combinations (Legacy+Legacy, ResponseFence+ResponseFence, FastAbort+FastAbort) I assume are not what you are concerned about.
That leaves us with 6 heterogeneous combinations of issuing+third-party modes. AFAICT, the heterogeneous combinations appear benign as well, but I am obviously missing something. Of course, whenever ResponseFence is not in effect, the inter-connection network skew could deliver the UAs on any iSCSI session for certain SCSI events (e.g. ACA) in an unpredictable order, but that is only to be expected (and that's what prompted us to put in the notion of response fence).
What would greatly help me is if you could identify a specific heterogeneous combination, its unpredictable behavior with updated TMF text and the consequences of such unpredictability that you think are undesirable.
Your secondary concern appears to be around complexity. As always, I would welcome any suggestions from you on that front as well.
I'll wait until the end of this week before I publish an updated draft revision as David suggested.
Thanks!
Mallikarjun
----- Original Message ----
From: Julian Satran <Julian_Satran at il.ibm.com>
To: Mallikarjun C. <cb_mallikarjun at yahoo.com>
Cc: ips at ietf.org
Sent: Monday, January 1, 2007 5:28:14 AM
Subject: Re: [Ips] TMF text with updates
Mallikarjun,
My reading was slightly different as bellow
"Mallikarjun C." <cb_mallikarjun at yahoo.com> wrote on 29/12/2006 23:42:09:
> David and Julian,
>
> Just to be clear on the semantics:
>
> "Fast abort" = early reporting of TMF completion despite outstandingtransfers
>
> RFC 3720 Not allowed for issuing session, Not allowed for
> third-party sessions
>
RFC3270 pays only lip-service to third party
sessions (there is little you can do beside it.
> IG (Last Call):
> FastMultiTaskAbort !=Yes
> Same as RFC3720
I would clarify/correct it to say that it is allowed for third party but I can go for your interpretation
> FastMultiTaskAbort =Yes
> Allowed for issuing session, Allowed for
> third-party sessions
See above
> Latest updates:
> TaskReporting !=FastAbort
>
Not allowed for issuing session, Allowed
> for third-party sessions
> TaskReporting=FastAbort
> Allowed for issuing session, Allowed for
> third-party sessions
>
If the mechanics is TM then as soon as TM rsponses are collected.
> Before we sanction more combinations of legal behavior, let us
> analyze why the last two combinations do not address the use cases
> we are concerned about. More combinations mean more complexity.....
>
> As far as Julian's reference to "target-scoped" queues, I assume it
> is about the scope of a task set when TST=0. IMHO, iSCSI needs to
> offer a reasonable processing model for this case with Clear Task
> Set TMF as well as that of a Logical Reset TMF (which always
affects
> tasks from multiple initiators even if TST=1). That is where the
> value of fast abort comes in.
>
I agree except that I would say that the effects on initiators other than the issuing may be unexpected if they use a mechanism with different semantics.
> Mallikarjun
>
>
>
> ----- Original Message ----
> From: "Black_David at emc.com" <Black_David at emc.com>
> To: Julian_Satran at il.ibm.com
> Cc: ips at ietf.org
> Sent: Friday, December 29, 2006 9:45:25 AM
> Subject: RE: [Ips] TMF text with updates
> Julian,
>
> I'm not sure I completely understand what you wrote, but if you're
> suggesting that for a TMF that affects tasks from
multiple initiators:
> - Fast abort (early termination of data transfers) is used for the sending
> initiator.
> - The existing 3270 abort mechanism is used with other initiators, with
> the update that the TMF response does not have to wait for data transfer
> completion.
> I think that suggestion is reasonable, and it actually helps with the use
> case that started this (3rd party initiator is dead). The draft text would
> need to change to allow this (right now it mandates fast abort in all cases
> if the key is negotiated to support it)
>
> I'm not sure how target-scoped queues enter into this.
>
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 176 South St., Hopkinton, MA 01748
> +1 (508) 293-7953 FAX: +1 (508) 293-7786
> black_david at emc.com Mobile: +1 (978) 394-7754
> ----------------------------------------------------
>
> From: Julian Satran [mailto:Julian_Satran at il.ibm.com]
> Sent: Friday, December 29, 2006 6:16 AM
> To: Black, David
> Cc: cb_mallikarjun at yahoo.com; ips at ietf.org
> Subject: RE: [Ips] TMF text with updates
>
>
David,
>
> My point was that we can solve the TM issue only for a single
> initiator. So besides puting the burden on SCSI to cleanly finish
> other and advise that fast solutions really work if there are no
> target scoped queues there is little we can do. And if we limit
> ourselves to a single initiator the multiple TM solution is a
> simpler (and basically does not deviate from 3270). If you consider
> how software stacks are layered I assume that some clever
> implementers have figured that already (and did it).
>
> Regards,
> Julo
>
>
> Black_David at emc.com
> 28/12/06 21:54
>
> To
>
> Julian Satran/Haifa/IBM at IBMIL, <cb_mallikarjun at yahoo.com>
>
> cc
>
> <ips at ietf.org>
>
> Subject
>
> RE: [Ips] TMF text with updates
>
>
>
>
> I agree with Julian that we should avoid discussing "buffer allocations" and
> the like, even though we know that something like that has to happen in at
> least iSER implementations. A general discussion of "resources" works.
>
> > You could achive the same effect by issuing the TM command on
> every affected connection.
>
> Not for TMF's that affect commands from other initiators. Also, asking the
> target to coordinate receipt of the TM command on every connection in a
> multi-connection session is also a bit much.
>
> > The third party story is even more puzzling as in order to negotiate any
of
> > the new TM modes the taget will have to ascertain that all other initiators
> > support it. I fail to understand how would you handle downgrading the mode
> > for those that don't.
>
> I think the target has to track this initiator by initiator, and notissue the
> new async message to old initiators. This increases the importance of being
> able to complete a TMF in the face of an uncooperative third party "Legacy"
> initiator.
>
> > And if you don't downgrade why not state that fast-abort and target scoped
> > queues don't go together and simplify the mechanics to a multiple issue TM.
> This didn't parse for me - could you explain in more detail?
>
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Senior Technologist
> EMC
Corporation, 176 South St., Hopkinton, MA 01748
> +1 (508) 293-7953 FAX: +1 (508) 293-7786
> black_david at emc.com Mobile: +1 (978) 394-7754
> ----------------------------------------------------
>
> From: Julian Satran [mailto:Julian_Satran at il.ibm.com]
> Sent: Thursday, December 28, 2006 5:32 AM
> To: Mallikarjun C.
> Cc: ips at ietf.org
> Subject: Re: [Ips] TMF text with updates
>
>
> Mallikarjun,
>
> I would consider the text:
>
> c. MUST leave all active "affected TTTs" (i.e. active TTTs
> associated with affected tasks) valid along with any buffer
> allocations for the TTTs intact.
>
> somewhat excessive as it relates too much to implementation. I would
> rather say:
>
> c. MUST leave all active "affected TTTs" (i.e.
active TTTs
> associated with affected tasks) valid.
>
> and leave the buffer issue to the implementer (as I have stated
> already on this list).
>
> I also keep thinking (and mildly objecting to) that the whole fast
> abort is excesively complex.
>
> You could achive the same effect by issuing the TM command on every
> affected connection.
> The third party story is even more puzzling as in order to negotiate
> any of the nem TM modes the taget will have to ascertain that all
> other initiators support it. I fail to understand how would you
> handle downgrading the mode for those that don't. And if you don't
> downgrade why not state that fast-abort and target scoped queues
> don't go together and simplify the mechanics to a multiple issue TM.
>
> Thanks,
> Julo
>
> "Mallikarjun C."
<cb_mallikarjun at yahoo.com>
> 28/12/06 06:41
>
> To
>
> ips at ietf.org
>
> cc
>
> Subject
>
> [Ips] TMF text with updates
>
>
>
>
>
>
>
> Attached is the latest text that incorporates David's proposed
> enhancement. Please review and comment. Note especially two
> things: new section 4.1.4 that summarizes generic implementation
> considerations for both "clarified" and "updated" semantics, the
> changed text in 4.1.2 that says "MAY wait for ....target transfer
> tags.....from third-party initiators" from the previous blanket MUST.
>
>
Mallikarjun
>
>
>
> 4.1.2 Clarified multi-task abort semantics
> All iSCSI implementations MUST support the protocol behavior defined
> in this section as the default behavior. The execution of ABORT
> TASK SET, CLEAR TASK SET, LOGICAL UNIT RESET, TARGET WARM RESET, and
> TARGET COLD RESET TMF Requests consists of the following sequence of
> actions in the specified order on the specified party.
> The initiator iSCSI layer:
> a. MUST continue to respond to each TTT received for the affected tasks.
> b. Should receive any responses that the target may provide for some
> tasks among the affected tasks (may process them as usual because
> they are guaranteed to have chronologically originated prior to the
> TMF response).
> c. Should receive the TMF Response concluding all the tasks in the
> set of affected tasks.
>
> The target iSCSI
layer:
> a. MUST wait for currently valid target transfer tags of the
> affected tasks from the issuing initiator to be responded to. MAY
> wait for responses on currently valid target transfer tags of the
> affected tasks from third-party initiators.
> b. MUST wait (concurrent with the wait in Step.a) for all commands
> of the affected tasks to be received based on the CmdSN ordering.
> SHOULD NOT wait for new commands on third-party affected sessions -
> only the instantiated tasks have to be considered for the purpose of
> determining the affected tasks. In the case of target-scoped
> requests (i.e. TARGET WARM RESET and TARGET COLD RESET), all the
> commands that are not yet received on the issuing session in the
> command stream however can be considered to have been received with
> no command waiting period - i.e. the entire CmdSN space up to the
> CmdSN
of the task management function can be "plugged".
> c. MUST propagate the TMF request to and receive the response from
> the target SCSI layer.
> d. MUST address the Response Fence flag on the TMF Response on
> issuing session as defined in 3.3.2.
> e. MUST address the Response Fence flag on the first post-TMF
> Response on third-party sessions as defined in 3.3.2. If some tasks
> originate from non-iSCSI I_T_L nexuses then the means by which the
> target ensures that all affected tasks have returned their status to
> the initiator are defined by the specific non-iSCSI transport protocol(s).
> Implementation note: Technically, the TMF servicing is complete in
> Step.d. Data transfers corresponding to terminated tasks may
> however still be in progress on third-party iSCSI sessiosn even at
> the end of Step.e. TMF Response MUST NOT be sent by the target
> iSCSI
layer before the end of Step.d, and MAY be sent at the end of
> Step.d despite these outstanding Data transfers until after Step.e.
>
> 4.1.3 Updated multi-task abort semantics
> Protocol behavior defined in this section MUST be implemented by all
> iSCSI implementations complying with this document. Protocol
> behavior defined in this section MUST be exhibited by iSCSI
> implementations on an iSCSI session when they negotiate the
> TaskReporting (section 9.1) key to “FastAbort” on that session. The
> execution of ABORT TASK SET, CLEAR TASK SET, LOGICAL UNIT RESET,
> TARGET WARM RESET, and TARGET COLD RESET TMF Requests consists of
> the following sequence of actions in the specified order on the
> specified party.
> The initiator iSCSI layer:
> a. MUST NOT send any more Data-Out PDUs for affected tasks on the
> issuing connection of the issuing iSCSI session
once the TMF is sent
> to the target.
> b. Should receive any responses that the target may provide for some
> tasks among the affected tasks (may process them as usual because
> they are guaranteed to have chronologically originated prior to the
> TMF response).
> c. MUST respond to Async Message PDU with AsyncEvent=5 as defined in
> section 8.1.
> d. Should receive the TMF Response concluding all the tasks in the
> set of affected tasks.
>
> The target iSCSI layer:
> a. MUST wait for all commands of the affected tasks to be received
> based on the CmdSN ordering on the issuing session. SHOULD NOT wait
> for new commands on third-party affected sessions - only the
> instantiated tasks have to be considered for the purpose of
> determining the affected tasks. In the case of target-scoped
> requests (i.e. TARGET WARM RESET and TARGET COLD RESET), all the
> commands that are not yet received on the issuing session in the
> command stream however can be considered to have been received with
> no command waiting period - i.e. the entire CmdSN space up to the
> CmdSN of the task management function can be "plugged".
> b. MUST propagate the TMF request to and receive the response from
> the target SCSI layer.
> c. MUST leave all active "affected TTTs" (i.e. active TTTs
> associated with affected tasks) valid along with any buffer
> allocations for the TTTs intact.
> d. MUST generate an Asynchronous Message PDU with AsyncEvent=5
> (section 8.1) on:
> i) each connection of each third-party session that at least one
> affected task is allegiant to, and
> ii) each connection except the issuing connection of the issuing
> session that has at least one allegiant affected task.
> If there are multiple affected LUs (say due to a
target reset), then
> one Async Message PDU MUST be sent for each such LU on each
> connection that has at least one allegiant affected task.
> e. MUST address the Response Fence flag on the TMF Response on
> issuing session as defined in 3.3.2.
> f. MUST address the Response Fence flag on the first post-TMF
> Response on third-party sessions as defined in 3.3.2. If some tasks
> originate from non-iSCSI I_T_L nexuses then the means by which the
> target ensures that all affected tasks have returned their status to
> the initiator are defined by the specific non-iSCSI transport protocol(s).
> g. MUST free up the affected TTTs (and STags, if applicable) and the
> corresponding buffers once it receives the associated Nop-Out
> acknowledgement that the initiator generated in response to the
> Async Message.
> Implementation note: Technically, the TMF servicing is complete in
> Step.e. Data transfers corresponding to terminated tasks may
> however still be in progress even at the end of Step.f. TMF
> Response MUST NOT be sent by the target iSCSI layer before the end
> of Step.e, and MAY be sent at the end of Step.e despite these
> outstanding Data transfers until Step.g. Step.g specifies an event
> to free up any such resources that may have been reserved to support
> outstanding data transfers.
> 4.1.3.1 Clearing effects update
> Appendix F.1 of [RFC3720] specifies the clearing effects of target
> and LU resets on “Incomplete TTTs” as “Y”. This meant that a target
> warm reset or a target cold reset or an LU reset would clear the
> active TTTs upon completion. The TaskReporting=FastAbort (section
> 9.1) semantics defined by this section however do not guarantee that
> the active TTTs are cleared by the end
of the reset operations. In
> fact, the new semantics are designed to allow clearing the TTTs in a
> “lazy” fashion after the TMF Response is delivered. Thus, when
> TaskReporting=FastAbort is operational on a session, the clearing
> effects of reset operations on “Incomplete TTTs” is “N”.
> 4.1.4 Implementation considerations
> Both in clarified semantics (section 4.1.2) and updated semantics
> (section 4.1.3), there may be outstanding data transfers even after
> the TMF completion is reported on the issuing session. In the case
> of iSCSI/iSER [iSER], these would be tagged data transfers for STags
> not owned by any active tasks. Whether or not real buffers support
> these data transfers is implementation-dependent. However, the data
> transfers logically MUST be silently discarded by the target iSCSI
> layer in all cases. A target
MAY, on an implementation-defined
> internal timeout, also choose to drop the connections on which it
> did not receive the expected Data-out sequences (section 4.1.2) or
> Nop-Out acknowledgements (section 4.1.3) so as to reclaim the
> associated buffer, STag and TTT resources as appropriate.
>
>
> ----- Original Message ----
> From: "Black_David at emc.com" <Black_David at emc.com>
> To: Black_David at emc.com; cb_mallikarjun at yahoo.com; ips at ietf.org
> Sent: Wednesday, December 20, 2006 2:33:43 PM
> Subject: RE: [Ips] Implementer's Guide - Task Management Issue
>
>
> > Let's try this line of reasoning - the target issues the Nop-Out, now
> when
> > can it free the resources? Answer:
> > - The Nop-In response comes back, OR
> > - The connection times out and is torn down.
> > Now, what if the Nop-Out is not
issued - what does the target wait for
> to
> > free the resources? Answer:
> > - The transfers complete, OR
> > - The connection times out and is torn down.
> > Those look similar enough at the target (the worst case is the same -
> the
> > resources are tied up until an uncooperative initiator times out) that
> > I don't see the harm in allowing the early TMF return without the new
> > key. The clear distinction is that the first two bullets are
> different;
> > if the new key is not negotiated, the target has to wait for the
> transfers
> > to complete; the new key and the Nop-Out are necessary to walk away
> earlier
> > when the initiator involved is able to continue the transfers.
>
> That got slightly twisted - what it should have talked about was the
> target
> issuing the new Async
Message and the Nop-Out response coming back.
>
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 176 South St., Hopkinton, MA 01748
> +1 (508) 293-7953 FAX: +1 (508) 293-7786
> black_david at emc.com Mobile: +1 (978) 394-7754
> ----------------------------------------------------
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
> _______________________________________________
> Ips mailing list
> Ips at ietf.org
> https://www1.ietf.org/mailman/listinfo/ips
> _______________________________________________
> Ips mailing list
>
Ips at ietf.org
> https://www1.ietf.org/mailman/listinfo/ips
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com _______________________________________________
> Ips mailing list
> Ips at ietf.org
> https://www1.ietf.org/mailman/listinfo/ips