<?xml version="1.0" encoding="utf-8"?>
<!-- This is built from a template for a generic Internet Draft. Suggestions for
     improvement welcome - write to Brian Carpenter, brian.e.carpenter @ gmail.com 
     This can be converted using the Web service at https://xml2rfc.tools.ietf.org/ -->
<!DOCTYPE rfc SYSTEM "rfc2629-xhtml.ent">
<?rfc toc="yes"?>
<!-- You want a table of contents -->
<?rfc symrefs="yes"?>
<!-- Use symbolic labels for references -->
<?rfc sortrefs="yes"?>
<!-- This sorts the references -->
<?rfc iprnotified="no" ?>
<!-- Change to "yes" if someone has disclosed IPR for the draft -->
<?rfc compact="yes"?>
<!-- This defines the specific filename and version number of your draft (and inserts the appropriate IETF boilerplate -->
<rfc xmlns:xi="http://www.w3.org/2001/XInclude" category="info" docName="draft-carpenter-anima-grasp-bulk-05" ipr="trust200902" obsoletes="" updates="" submissionType="IETF" xml:lang="en" tocInclude="true" symRefs="true" sortRefs="true" version="3">
  <!-- xml2rfc v2v3 conversion 2.37.3 -->
  <front>
    <title abbrev="Bulk data over GRASP">Transferring Bulk Data over the GeneRic Autonomic Signaling Protocol (GRASP)</title>
    <seriesInfo name="Internet-Draft" value="draft-carpenter-anima-grasp-bulk-05"/>
    <author fullname="Brian Carpenter" initials="B. E." surname="Carpenter">
      <organization abbrev="Univ. of Auckland"/>
      <address>
        <postal>
          <street>School of Computer Science</street>
          <street>University of Auckland</street>
          <street>PB 92019</street>
          <city>Auckland</city>
          <region/>
          <code>1142</code>
          <country>New Zealand</country>
        </postal>
        <email>brian.e.carpenter@gmail.com</email>
      </address>
    </author>
    <author fullname="Sheng Jiang" initials="S." surname="Jiang">
      <organization>Huawei Technologies Co., Ltd</organization>
      <address>
        <postal>
          <street>Q14 Huawei Campus</street>
          <street>156 Beiqing Road</street>
          <street>Hai-Dian District</street>
          <city>Beijing</city>
          <code>100095</code>
          <country>China</country>
        </postal>
        <email>jiangsheng@huawei.com</email>
      </address>
    </author>
    <author fullname="Bing Liu" initials="B." surname="Liu">
      <organization>Huawei Technologies Co., Ltd</organization>
      <address>
        <postal>
          <street>Q14 Huawei Campus</street>
          <street>156 Beiqing Road</street>
          <street>Hai-Dian District</street>
          <city>Beijing</city>
          <code>100095</code>
          <country>China</country>
        </postal>
        <email>leo.liubing@huawei.com</email>
      </address>
    </author>
    <date day="10" month="January" year="2020"/>
    <abstract>
      <t>This document describes how bulk data may be transferred between Autonomic Service Agents
      via the GeneRic Autonomic Signaling Protocol (GRASP). Although not an equivalent of a file
      transfer protocol, such a technique may be used for non-urgent transfer of data too large
      to fit into a normal GRASP message.
      </t>
    </abstract>
  </front>
  <middle>
    <section anchor="intro" numbered="true" toc="default">
      <name>Introduction</name>
      <t>
      The document <xref target="I-D.liu-anima-grasp-distribution" format="default"/> discusses how information may be distributed
      within the secure Autonomic Networking Infrastructure (ANI) <xref target="I-D.ietf-anima-reference-model" format="default"/>.
      Specifically, it describes using the Synchronization and Flood Synchronization mechanisms of the GeneRic
      Autonomic Signaling Protocol  (GRASP) <xref target="I-D.ietf-anima-grasp" format="default"/> for this purpose
      as well as proposing GRASP extensions to support a publish/subscribe model. However,
      those mechanisms are limited to distributing GRASP Objective Options contained in messages that cannot
      exceed the GRASP maximum message size of 2048 bytes. This places a limit on the size of data
      that can be transferred in a Synchronization or Flood operation.
      </t>
      <t>There are scenarios in autonomic networks where this restriction is a problem.
     One example is the distribution of network policy in lengthy formats such as YANG or JSON.
     <!-- Intent <xref target="I-D.du-anima-an-intent"/>.-->
     Another case might be an
     Autonomic Service Agent (ASA) uploading a log file to the Network Operations Center (NOC). A third case might
     be a supervisory system downloading a software upgrade to an autonomic node.
     A related case might be installing the code of a new or updated ASA to a target node (see the
     discussion of ASA life cycles in <xref target="I-D.carpenter-anima-asa-guidelines" format="default"/>).</t>
      <t>Naturally, an existing solution such as a secure file transfer protocol or secure HTTP might be used for
     this. Other management protocols such as syslog <xref target="RFC5424" format="default"/> or NETCONF <xref target="RFC6241" format="default"/>
     might also be used for related purposes, or might be mapped directly over GRASP.
     The present document, however, applies to any scenario where it is preferable to re-use the
     autonomic networking infrastructure itself to transfer a significant amount of data,
     rather than install and configure an additional mechanism. The basic model is to
     use the GRASP Negotiation process to transfer and acknowledge multiple
     blocks of data in successive negotiation steps, thereby overcoming the GRASP
     message size limitation.</t>
      <t>The emphasis is placed on simplicity rather than efficiency, high throughput, or advanced
     functionality. For example, if a transfer gets out of step or data packets are lost, the
     strategy is to abort the transfer and try again. In an enterprise network with low bit
     error rates, and with GRASP running over TCP, this is not considered a serious issue.
     Clearly, a more sophisticated approach could be designed but if the application requires
     that, existing protocols could be used, as indicated in the preceding paragraph.</t>
      <t>This is an informational description of a class of solutions. Standards track
     solutions could be published as detailed specifications of the corresponding GRASP
     objectives.</t>
    </section>
    <section anchor="method" numbered="true" toc="default">
      <name>General Method for Bulk Transfer</name>
      <t>As for any GRASP operation, the two participants are considered to be Autonomic Service Agents (ASAs)
    and they communicate using a specific GRASP Objective Option, containing its own name, some flag bits,
    a loop count, and a value. In bulk transfer, we can model the ASA acting as the source of the transfer as
    a download server, and the destination as a download client. No changes or extensions are required
    to GRASP itself, but compared to a normal GRASP negotiation, the communication
    pattern is slightly asymmetric:
      </t>
      <ol spacing="normal" type="1">
        <li>The client first discovers the server by the GRASP discovery mechanism (M_DISCOVERY and M_RESPONSE messages).</li>
        <li>The client then sends a GRASP negotiation request (M_REQ_NEG message).
    The value of the objective expresses the requested item (e.g., a file name - see the next
    section for a detailed example).</li>
        <li>The server replies with a negotiation step (M_NEGOTIATE message). The value of the objective is the first section of
    the requested item (e.g., the first block of the requested file as a raw byte string).</li>
        <li>The client replies with a negotiation step (M_NEGOTIATE message). The value of the objective is a simple
    acknowledgement (e.g., the text string 'ACK').</li>
      </ol>
      <t>The last two steps repeat until the transfer is complete. The server signals the end by
    transferring an empty byte string as the final value. In this case the client responds with
    a normal end to the negotiation (M_END message with an O_ACCEPT option).</t>
      <t>Errors of any kind are handled with the normal GRASP mechanisms, in particular by an M_END message
    with an O_DECLINE option in either direction. In this case the GRASP session terminates.
    It is then the client's choice whether to retry the operation from the start, as a new GRASP
    session, or to abandon the transfer.</t>
      <t>The block size must be chosen such that each step does not exceed the GRASP message size limit of 2048
    bits.</t>
      <t>This approach is safe since each block must be positively acknowledged, and data transfer errors
    will be detected by TCP. If a future variant of GRASP runs over UDP, the mandatory UDP checksum for IPv6
    will detect such errors. The method does not specify retransmission for failed blocks, so the ASA that
    detects the error must signal the error as above. </t>
      <t>An observant reader will notice that the GRASP loop count mechanism, intended to terminate
    endless negotiations, will cause a problem for large transfers. For this reason, both the client
    and server must artificially increment the loop count by 1 before each negotiation step, cancelling
    out the normal decrement at each step.</t>
      <t>If network load is a concern, the data rate can be limited by inserting a delay before each
    negotiation step, with the GRASP timeout set accordingly. Either the server or the client,
    or both, could insert such a delay. Also, either side could use the GRASP Confirm Waiting (M_WAIT)
    message to slow the other side down.</t>
      <t>The description above concerns bulk download from a server (responding ASA) to a client (requesting ASA).
    The data transfer could also be in the opposite (upload) direction with minor modifications to
    the procedure: the client would send the file name and the data blocks, and the server would send
    acknowledgements.</t>
    </section>
    <section anchor="example" numbered="true" toc="default">
      <name>Example for File Transfer</name>
      <t>This example describes a client ASA requesting a file download from a server ASA.</t>
      <t>Firstly we define a GRASP objective informally:
      </t>
      <t>
    ["411:mvFile", 3, 6, value]
      </t>
      <t>
    The formal CDDL definition <xref target="RFC8610" format="default"/> is:
      </t>
      <artwork name="" type="" align="left" alt=""><![CDATA[
mvfile-objective = ["411:mvFile", objective-flags, loop-count, value]
  
objective-flags = ; as in the GRASP specification
loop-count = ; as in the GRASP specification
value = any
]]></artwork>
      <t>The objective-flags field is set to indicate negotiation.</t>
      <t>Dry run mode must not be used.</t>
      <t>The loop-count is set to a suitable value to limit the scope of discovery. A suggested default value is 6.</t>
      <t>The value takes the following forms:
      </t>
      <ul spacing="normal">
        <li>In the initial request from the client, a UTF-8 string containing the requested file name (with file path if appropriate).</li>
        <li>
          <t>In negotiation steps from the server, a byte string containing at most 1024 bytes. However:
          </t>
          <ul spacing="normal">
            <li>If the file does not exist, the first negotiation step will return an M_END, O_DECLINE response.</li>
            <li>After sending the last block, the next and final negotiation step will send an empty byte string as the value.</li>
          </ul>
        </li>
        <li>In negotiation steps from the client, the value is the UTF-8 string 'ACK'.</li>
      </ul>
      <t>Note that the block size of 1024 is chosen to guarantee not only that each GRASP message is below the size
    limit, but also that only one TCP data packet will be needed, even on an IPv6 network with a minimum link MTU.</t>
      <t>We now present outline pseudocode for the client and the server ASA. The API documented in
    <xref target="I-D.ietf-anima-grasp-api" format="default"/> is used in a simplified way, and
    error handling is not shown in detail.</t>
      <t>Pseudo code for client ASA (request and receive a file):</t>
      <artwork name="" type="" align="left" alt=""><![CDATA[
requested_obj = objective('411:mvFile')
locator = discover(requested_obj)
requested_obj.value = 'etc/test.pdf'
received_obj = request_negotiate(requested_obj, locator)
if error_code == declined:
    #no such file
    exit

file = open(requested_obj.value)
file.write(received_obj.value) #write to file
eof = False
while not eof:
    received_obj.value = 'ACK'
    received_obj.loop_count = received_obj.loop_count + 1
    received_obj = negotiate_step(received_obj)
    if received_obj.value == null:
        end_negotiate(True)
        file.close()
        eof = True     
    else:
        file.write(received_obj.value) #write to file

#file received
exit
]]></artwork>
      <t>Pseudo code for server ASA (await request and send a file):</t>
      <artwork name="" type="" align="left" alt=""><![CDATA[
supported_obj = objective('411:mvFile')
requested_obj = listen_negotiate(supported_obj)
file = open(requested_obj.value) #open the source file
if no such file:
    end_negotiate(False) #decline negotiation
    exit

eof = False
while not eof:
    chunk = file.read(1024) #next block of file
    requested_obj.value = chunk
    requested_obj.loop_count = requested_obj.loop_count + 1
    requested_obj = negotiate_step(requested_obj)
    if chunk == null:
        file.close()
        eof = True
        end_negotiate(True)
        exit
    if requested_obj.value != 'ACK':
        #unexpected reply...
]]></artwork>
    </section>
    <section anchor="UDP" numbered="true" toc="default">
      <name>Loss Detection</name>
      <t>The above description and example assume that GRASP is implemented over a
    reliable transport layer such as TCP, such that lost or corrupted messages are not
    likely. Rarely, an error might be detected via a missing ACK, in which case
    the transfer would be aborted and restarted. In the event that GRASP is implemented
    over an unreliable transport layer such as UDP, it would be possible to add a block
    number to both the data block and acknowledgement objectives, so that missing blocks
    can be retransmitted, or duplicate blocks can be ignored.
    For example, the objective in <xref target="example" format="default"/> would become:</t>
      <artwork name="" type="" align="left" alt=""><![CDATA[
  mvfile-objective = ["411:mvFile", objective-flags, loop-count, value]
  
  objective-flags = ; as in the GRASP specification
  loop-count = ; as in the GRASP specification
  value = [block-number, any]
  block-number = uint
]]></artwork>
      <t>It would also be necessary for the transport layer to detect data errors, for example
    by enabling UDP checksums.</t>
    </section>
    <section anchor="MTU" numbered="true" toc="default">
      <name>Maximum Transmission Unit</name>
      <t>In an IPv6 environment, a minimal MTU of 1280 bytes can be assumed, and assuming that
    high throughput is not a requirement, bulk transfers can be designed to match that MTU. However,
    there are environments where the underlying physical MTU is much smaller. For example,
    on an IEEE 802.15.4 network it may be less than 100 bytes <xref target="RFC4944" format="default"/>. Even in
    a 5G network, the Transport Block Size may be quite small, depending on the radio parameters.
    In such a case, a bulk transfer solution has several choices: 
      </t>
      <ol spacing="normal" type="1">
        <li>Accept the overhead of fragmentation in an adaptation layer, and therefore
       assume a network-layer MTU of 1280 bytes.
       Indeed, the presence of such an adaptation layer may be impossible to detect.</li>
        <li>Attempt to determine the actual MTU available without lower-layer fragmentation.
       This however will be impossible without using low-level functions of the socket interface.</li>
        <li>Attempt to determine a message size that provides optimum performance, by some sort of
       trial-and-error solution.</li>
      </ol>
      <t>These complexities suggest that using a GRASP-based mechanism is unlikely to be
    optimal in environments with a very small physical MTU.</t>
    </section>
    <section anchor="pipe" numbered="true" toc="default">
      <name>Pipelining</name>
      <t>The above description and example descibe a simple handshake model where each
    block is acknowledged before the next block is sent. For the scenarios
    discussed in <xref target="intro" format="default"/>, this should be acceptable. Therefore we do not
    suggest adding a pipelining or windowing mechanism. If high throughput is
    required, a conventional file transfer protocol should be used.</t>
    </section>
    <section anchor="other" numbered="true" toc="default">
      <name>Other Considerations</name>
      <t>If multiple transfers are requested simultaneously, each one will proceed as a separate
    GRASP negotiation session. The ASA acting as the server must be coded accordingly, like
    any ASA that needs to handle simultaneous sessions <xref target="I-D.carpenter-anima-asa-guidelines" format="default"/>.</t>
      <t>Bulk transfer might become a utility function for use by various ASAs, such as those
    supporting YANG or JSON distribution, log file uploads, or code downloads. In this case
    some form of user space API for bulk transfer will be required. This could be in the form
    of an inter-process communication call between the ASA in question and the ASA implementing
    the bulk transfer mechanism. The details are out of scope for this document.</t>
    </section>
    <section anchor="future" numbered="true" toc="default">
      <name>Possible Future Work</name>
      <t>The simple file transfer mechanism described above is only an example. Other
    application scenarios should be developed.
      </t>
      <t>The mechanism described in this document is suitable for simple unicast scenarios
    where GRASP runs over TCP and can be treated as a reliable protocol. A more sophisticated
    approach would be needed in at least two cases:
      </t>
      <ol spacing="normal" type="1">
        <li>A scenario where GRASP runs over UDP, where error detection and retransmission would be essential.</li>
        <li>A scenario where multicast data distribution is required, so that a mechanism such as Trickle
    <xref target="RFC6206" format="default"/> would be appropriate.</li>
      </ol>
      <t>These solutions might also require extensions to the GRASP protocol itself.</t>
    </section>
    <section numbered="true" toc="default">
      <name>Implementation Status [RFC Editor: please remove]</name>
      <t>A prototype open source Python implementation of simple file transfer has been used
    to verify the mechanism described above. It may be found at
    https://github.com/becarpenter/graspy/blob/master/getter.py
    and https://github.com/becarpenter/graspy/blob/master/pusher.py .
      </t>
    </section>
    <section anchor="security" numbered="true" toc="default">
      <name>Security Considerations</name>
      <t>All GRASP transactions are secured by the mandatory security substrate required by
      <xref target="I-D.ietf-anima-grasp" format="default"/>. No additional security issues are created by
      the application of GRASP described in this document.</t>
    </section>
    <section anchor="iana" numbered="true" toc="default">
      <name>IANA Considerations</name>
      <t>This document makes no request of the IANA.</t>
    </section>
    <section anchor="ack" numbered="true" toc="default">
      <name>Acknowledgements</name>
      <t>Thanks to Joel Halpern and other members of the ANIMA WG.</t>
    </section>
  </middle>
  <back>
    <references>
      <name>References</name>
      <references>
        <name>Normative References</name>
      <xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.ietf-anima-grasp.xml"/>
        <xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml/reference.RFC.8610.xml"/>
      </references>
      <references>
        <name>Informative References</name>
      <xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.ietf-anima-reference-model.xml"/>
      <xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.ietf-anima-grasp-api.xml"/>
      <xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.liu-anima-grasp-distribution.xml"/>
      <xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.carpenter-anima-asa-guidelines.xml"/>
        <!-- <?rfc include='reference.I-D.du-anima-an-intent'?> -->
        <xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml/reference.RFC.5424.xml"/>
        <xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml/reference.RFC.6241.xml"/>
        <xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml/reference.RFC.4944.xml"/>
        <xi:include href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml/reference.RFC.6206.xml"/>
      </references>
    </references>
    <section anchor="changes" numbered="true" toc="default">
      <name>Change log [RFC Editor: Please remove]</name>
      <t>draft-carpenter-anima-grasp-bulk-05, 2020-01-10:</t>
      <ul spacing="compact">
      <li>Minor technical clarifications.</li>
      <li>Converted to v3 format.</li>
      </ul>
      <t>draft-carpenter-anima-grasp-bulk-04, 2019-07-03:</t>
      <ul spacing="compact">
      <li>Updated description of very small link-layer MTU issue.</li>
      <li>Clarified informational status, updated reference.</li>
      </ul>
      <t>draft-carpenter-anima-grasp-bulk-03, 2019-01-07:</t>
      <ul spacing="compact">       
      <li>Added future work section, implementation status.</li>
      </ul>
      <t>draft-carpenter-anima-grasp-bulk-02, 2018-06-30:</t>
      <ul spacing="compact">
      <li>Update reference, fix TBDs.</li>
      </ul>
      <t>draft-carpenter-anima-grasp-bulk-01, 2018-03-03:</t>
      <ul spacing="compact"> 
      <li>Updates after IETF100 discussion.</li>
      </ul>
      <t>draft-carpenter-anima-grasp-bulk-00, 2017-09-12:</t>
      <ul spacing="compact">
      <li>Initial version.</li>
      </ul>
    </section>
  </back>
</rfc>
