idnits 2.17.1 draft-carpenter-anima-grasp-bulk-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There is 1 instance of too long lines in the document, the longest one being 2 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (10 January 2020) is 1565 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-09) exists of draft-carpenter-anima-asa-guidelines-07 == Outdated reference: A later version (-10) exists of draft-ietf-anima-grasp-api-04 Summary: 1 error (**), 0 flaws (~~), 3 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group B. E. Carpenter 3 Internet-Draft Univ. of Auckland 4 Intended status: Informational S. Jiang 5 Expires: 13 July 2020 B. Liu 6 Huawei Technologies Co., Ltd 7 10 January 2020 9 Transferring Bulk Data over the GeneRic Autonomic Signaling Protocol 10 (GRASP) 11 draft-carpenter-anima-grasp-bulk-05 13 Abstract 15 This document describes how bulk data may be transferred between 16 Autonomic Service Agents via the GeneRic Autonomic Signaling Protocol 17 (GRASP). Although not an equivalent of a file transfer protocol, 18 such a technique may be used for non-urgent transfer of data too 19 large to fit into a normal GRASP message. 21 Status of This Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at https://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on 13 July 2020. 38 Copyright Notice 40 Copyright (c) 2020 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 45 license-info) in effect on the date of publication of this document. 46 Please review these documents carefully, as they describe your rights 47 and restrictions with respect to this document. Code Components 48 extracted from this document must include Simplified BSD License text 49 as described in Section 4.e of the Trust Legal Provisions and are 50 provided without warranty as described in the Simplified BSD License. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 55 2. General Method for Bulk Transfer . . . . . . . . . . . . . . 3 56 3. Example for File Transfer . . . . . . . . . . . . . . . . . . 4 57 4. Loss Detection . . . . . . . . . . . . . . . . . . . . . . . 7 58 5. Maximum Transmission Unit . . . . . . . . . . . . . . . . . . 7 59 6. Pipelining . . . . . . . . . . . . . . . . . . . . . . . . . 8 60 7. Other Considerations . . . . . . . . . . . . . . . . . . . . 8 61 8. Possible Future Work . . . . . . . . . . . . . . . . . . . . 8 62 9. Implementation Status [RFC Editor: please remove] . . . . . . 9 63 10. Security Considerations . . . . . . . . . . . . . . . . . . . 9 64 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 65 12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 9 66 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 9 67 13.1. Normative References . . . . . . . . . . . . . . . . . . 9 68 13.2. Informative References . . . . . . . . . . . . . . . . . 9 69 Appendix A. Change log [RFC Editor: Please remove] . . . . . . . 10 70 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11 72 1. Introduction 74 The document [I-D.liu-anima-grasp-distribution] discusses how 75 information may be distributed within the secure Autonomic Networking 76 Infrastructure (ANI) [I-D.ietf-anima-reference-model]. Specifically, 77 it describes using the Synchronization and Flood Synchronization 78 mechanisms of the GeneRic Autonomic Signaling Protocol (GRASP) 79 [I-D.ietf-anima-grasp] for this purpose as well as proposing GRASP 80 extensions to support a publish/subscribe model. However, those 81 mechanisms are limited to distributing GRASP Objective Options 82 contained in messages that cannot exceed the GRASP maximum message 83 size of 2048 bytes. This places a limit on the size of data that can 84 be transferred in a Synchronization or Flood operation. 86 There are scenarios in autonomic networks where this restriction is a 87 problem. One example is the distribution of network policy in 88 lengthy formats such as YANG or JSON. Another case might be an 89 Autonomic Service Agent (ASA) uploading a log file to the Network 90 Operations Center (NOC). A third case might be a supervisory system 91 downloading a software upgrade to an autonomic node. A related case 92 might be installing the code of a new or updated ASA to a target node 93 (see the discussion of ASA life cycles in 94 [I-D.carpenter-anima-asa-guidelines]). 96 Naturally, an existing solution such as a secure file transfer 97 protocol or secure HTTP might be used for this. Other management 98 protocols such as syslog [RFC5424] or NETCONF [RFC6241] might also be 99 used for related purposes, or might be mapped directly over GRASP. 100 The present document, however, applies to any scenario where it is 101 preferable to re-use the autonomic networking infrastructure itself 102 to transfer a significant amount of data, rather than install and 103 configure an additional mechanism. The basic model is to use the 104 GRASP Negotiation process to transfer and acknowledge multiple blocks 105 of data in successive negotiation steps, thereby overcoming the GRASP 106 message size limitation. 108 The emphasis is placed on simplicity rather than efficiency, high 109 throughput, or advanced functionality. For example, if a transfer 110 gets out of step or data packets are lost, the strategy is to abort 111 the transfer and try again. In an enterprise network with low bit 112 error rates, and with GRASP running over TCP, this is not considered 113 a serious issue. Clearly, a more sophisticated approach could be 114 designed but if the application requires that, existing protocols 115 could be used, as indicated in the preceding paragraph. 117 This is an informational description of a class of solutions. 118 Standards track solutions could be published as detailed 119 specifications of the corresponding GRASP objectives. 121 2. General Method for Bulk Transfer 123 As for any GRASP operation, the two participants are considered to be 124 Autonomic Service Agents (ASAs) and they communicate using a specific 125 GRASP Objective Option, containing its own name, some flag bits, a 126 loop count, and a value. In bulk transfer, we can model the ASA 127 acting as the source of the transfer as a download server, and the 128 destination as a download client. No changes or extensions are 129 required to GRASP itself, but compared to a normal GRASP negotiation, 130 the communication pattern is slightly asymmetric: 132 1. The client first discovers the server by the GRASP discovery 133 mechanism (M_DISCOVERY and M_RESPONSE messages). 135 2. The client then sends a GRASP negotiation request (M_REQ_NEG 136 message). The value of the objective expresses the requested 137 item (e.g., a file name - see the next section for a detailed 138 example). 140 3. The server replies with a negotiation step (M_NEGOTIATE message). 141 The value of the objective is the first section of the requested 142 item (e.g., the first block of the requested file as a raw byte 143 string). 145 4. The client replies with a negotiation step (M_NEGOTIATE message). 146 The value of the objective is a simple acknowledgement (e.g., the 147 text string 'ACK'). 149 The last two steps repeat until the transfer is complete. The server 150 signals the end by transferring an empty byte string as the final 151 value. In this case the client responds with a normal end to the 152 negotiation (M_END message with an O_ACCEPT option). 154 Errors of any kind are handled with the normal GRASP mechanisms, in 155 particular by an M_END message with an O_DECLINE option in either 156 direction. In this case the GRASP session terminates. It is then 157 the client's choice whether to retry the operation from the start, as 158 a new GRASP session, or to abandon the transfer. 160 The block size must be chosen such that each step does not exceed the 161 GRASP message size limit of 2048 bits. 163 This approach is safe since each block must be positively 164 acknowledged, and data transfer errors will be detected by TCP. If a 165 future variant of GRASP runs over UDP, the mandatory UDP checksum for 166 IPv6 will detect such errors. The method does not specify 167 retransmission for failed blocks, so the ASA that detects the error 168 must signal the error as above. 170 An observant reader will notice that the GRASP loop count mechanism, 171 intended to terminate endless negotiations, will cause a problem for 172 large transfers. For this reason, both the client and server must 173 artificially increment the loop count by 1 before each negotiation 174 step, cancelling out the normal decrement at each step. 176 If network load is a concern, the data rate can be limited by 177 inserting a delay before each negotiation step, with the GRASP 178 timeout set accordingly. Either the server or the client, or both, 179 could insert such a delay. Also, either side could use the GRASP 180 Confirm Waiting (M_WAIT) message to slow the other side down. 182 The description above concerns bulk download from a server 183 (responding ASA) to a client (requesting ASA). The data transfer 184 could also be in the opposite (upload) direction with minor 185 modifications to the procedure: the client would send the file name 186 and the data blocks, and the server would send acknowledgements. 188 3. Example for File Transfer 190 This example describes a client ASA requesting a file download from a 191 server ASA. 193 Firstly we define a GRASP objective informally: 195 ["411:mvFile", 3, 6, value] 197 The formal CDDL definition [RFC8610] is: 199 mvfile-objective = ["411:mvFile", objective-flags, loop-count, value] 201 objective-flags = ; as in the GRASP specification 202 loop-count = ; as in the GRASP specification 203 value = any 205 The objective-flags field is set to indicate negotiation. 207 Dry run mode must not be used. 209 The loop-count is set to a suitable value to limit the scope of 210 discovery. A suggested default value is 6. 212 The value takes the following forms: 214 * In the initial request from the client, a UTF-8 string containing 215 the requested file name (with file path if appropriate). 217 * In negotiation steps from the server, a byte string containing at 218 most 1024 bytes. However: 220 - If the file does not exist, the first negotiation step will 221 return an M_END, O_DECLINE response. 223 - After sending the last block, the next and final negotiation 224 step will send an empty byte string as the value. 226 * In negotiation steps from the client, the value is the UTF-8 227 string 'ACK'. 229 Note that the block size of 1024 is chosen to guarantee not only that 230 each GRASP message is below the size limit, but also that only one 231 TCP data packet will be needed, even on an IPv6 network with a 232 minimum link MTU. 234 We now present outline pseudocode for the client and the server ASA. 235 The API documented in [I-D.ietf-anima-grasp-api] is used in a 236 simplified way, and error handling is not shown in detail. 238 Pseudo code for client ASA (request and receive a file): 240 requested_obj = objective('411:mvFile') 241 locator = discover(requested_obj) 242 requested_obj.value = 'etc/test.pdf' 243 received_obj = request_negotiate(requested_obj, locator) 244 if error_code == declined: 245 #no such file 246 exit 248 file = open(requested_obj.value) 249 file.write(received_obj.value) #write to file 250 eof = False 251 while not eof: 252 received_obj.value = 'ACK' 253 received_obj.loop_count = received_obj.loop_count + 1 254 received_obj = negotiate_step(received_obj) 255 if received_obj.value == null: 256 end_negotiate(True) 257 file.close() 258 eof = True 259 else: 260 file.write(received_obj.value) #write to file 262 #file received 263 exit 265 Pseudo code for server ASA (await request and send a file): 267 supported_obj = objective('411:mvFile') 268 requested_obj = listen_negotiate(supported_obj) 269 file = open(requested_obj.value) #open the source file 270 if no such file: 271 end_negotiate(False) #decline negotiation 272 exit 274 eof = False 275 while not eof: 276 chunk = file.read(1024) #next block of file 277 requested_obj.value = chunk 278 requested_obj.loop_count = requested_obj.loop_count + 1 279 requested_obj = negotiate_step(requested_obj) 280 if chunk == null: 281 file.close() 282 eof = True 283 end_negotiate(True) 284 exit 285 if requested_obj.value != 'ACK': 286 #unexpected reply... 288 4. Loss Detection 290 The above description and example assume that GRASP is implemented 291 over a reliable transport layer such as TCP, such that lost or 292 corrupted messages are not likely. Rarely, an error might be 293 detected via a missing ACK, in which case the transfer would be 294 aborted and restarted. In the event that GRASP is implemented over 295 an unreliable transport layer such as UDP, it would be possible to 296 add a block number to both the data block and acknowledgement 297 objectives, so that missing blocks can be retransmitted, or duplicate 298 blocks can be ignored. For example, the objective in Section 3 would 299 become: 301 mvfile-objective = ["411:mvFile", objective-flags, loop-count, value] 303 objective-flags = ; as in the GRASP specification 304 loop-count = ; as in the GRASP specification 305 value = [block-number, any] 306 block-number = uint 308 It would also be necessary for the transport layer to detect data 309 errors, for example by enabling UDP checksums. 311 5. Maximum Transmission Unit 313 In an IPv6 environment, a minimal MTU of 1280 bytes can be assumed, 314 and assuming that high throughput is not a requirement, bulk 315 transfers can be designed to match that MTU. However, there are 316 environments where the underlying physical MTU is much smaller. For 317 example, on an IEEE 802.15.4 network it may be less than 100 bytes 318 [RFC4944]. Even in a 5G network, the Transport Block Size may be 319 quite small, depending on the radio parameters. In such a case, a 320 bulk transfer solution has several choices: 322 1. Accept the overhead of fragmentation in an adaptation layer, and 323 therefore assume a network-layer MTU of 1280 bytes. Indeed, the 324 presence of such an adaptation layer may be impossible to detect. 326 2. Attempt to determine the actual MTU available without lower-layer 327 fragmentation. This however will be impossible without using 328 low-level functions of the socket interface. 330 3. Attempt to determine a message size that provides optimum 331 performance, by some sort of trial-and-error solution. 333 These complexities suggest that using a GRASP-based mechanism is 334 unlikely to be optimal in environments with a very small physical 335 MTU. 337 6. Pipelining 339 The above description and example descibe a simple handshake model 340 where each block is acknowledged before the next block is sent. For 341 the scenarios discussed in Section 1, this should be acceptable. 342 Therefore we do not suggest adding a pipelining or windowing 343 mechanism. If high throughput is required, a conventional file 344 transfer protocol should be used. 346 7. Other Considerations 348 If multiple transfers are requested simultaneously, each one will 349 proceed as a separate GRASP negotiation session. The ASA acting as 350 the server must be coded accordingly, like any ASA that needs to 351 handle simultaneous sessions [I-D.carpenter-anima-asa-guidelines]. 353 Bulk transfer might become a utility function for use by various 354 ASAs, such as those supporting YANG or JSON distribution, log file 355 uploads, or code downloads. In this case some form of user space API 356 for bulk transfer will be required. This could be in the form of an 357 inter-process communication call between the ASA in question and the 358 ASA implementing the bulk transfer mechanism. The details are out of 359 scope for this document. 361 8. Possible Future Work 363 The simple file transfer mechanism described above is only an 364 example. Other application scenarios should be developed. 366 The mechanism described in this document is suitable for simple 367 unicast scenarios where GRASP runs over TCP and can be treated as a 368 reliable protocol. A more sophisticated approach would be needed in 369 at least two cases: 371 1. A scenario where GRASP runs over UDP, where error detection and 372 retransmission would be essential. 374 2. A scenario where multicast data distribution is required, so that 375 a mechanism such as Trickle [RFC6206] would be appropriate. 377 These solutions might also require extensions to the GRASP protocol 378 itself. 380 9. Implementation Status [RFC Editor: please remove] 382 A prototype open source Python implementation of simple file transfer 383 has been used to verify the mechanism described above. It may be 384 found at https://github.com/becarpenter/graspy/blob/master/getter.py 385 and https://github.com/becarpenter/graspy/blob/master/pusher.py . 387 10. Security Considerations 389 All GRASP transactions are secured by the mandatory security 390 substrate required by [I-D.ietf-anima-grasp]. No additional security 391 issues are created by the application of GRASP described in this 392 document. 394 11. IANA Considerations 396 This document makes no request of the IANA. 398 12. Acknowledgements 400 Thanks to Joel Halpern and other members of the ANIMA WG. 402 13. References 404 13.1. Normative References 406 [I-D.ietf-anima-grasp] 407 Bormann, C., Carpenter, B., and B. Liu, "A Generic 408 Autonomic Signaling Protocol (GRASP)", Work in Progress, 409 Internet-Draft, draft-ietf-anima-grasp-15, 13 July 2017, 410 . 412 [RFC8610] Birkholz, H., Vigano, C., and C. Bormann, "Concise Data 413 Definition Language (CDDL): A Notational Convention to 414 Express Concise Binary Object Representation (CBOR) and 415 JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, 416 June 2019, . 418 13.2. Informative References 420 [I-D.carpenter-anima-asa-guidelines] 421 Carpenter, B., Ciavaglia, L., Jiang, S., and P. Pierre, 422 "Guidelines for Autonomic Service Agents", Work in 423 Progress, Internet-Draft, draft-carpenter-anima-asa- 424 guidelines-07, 6 July 2019, . 427 [I-D.ietf-anima-grasp-api] 428 Carpenter, B., Liu, B., Wang, W., and X. Gong, "Generic 429 Autonomic Signaling Protocol Application Program Interface 430 (GRASP API)", Work in Progress, Internet-Draft, draft- 431 ietf-anima-grasp-api-04, 6 October 2019, 432 . 435 [I-D.ietf-anima-reference-model] 436 Behringer, M., Carpenter, B., Eckert, T., Ciavaglia, L., 437 and J. Nobre, "A Reference Model for Autonomic 438 Networking", Work in Progress, Internet-Draft, draft-ietf- 439 anima-reference-model-10, 22 November 2018, 440 . 443 [I-D.liu-anima-grasp-distribution] 444 Liu, B., Xiao, X., Hecker, A., Jiang, S., and Z. 445 Despotovic, "Information Distribution in Autonomic 446 Networking", Work in Progress, Internet-Draft, draft-liu- 447 anima-grasp-distribution-13, 12 December 2019, 448 . 451 [RFC4944] Montenegro, G., Kushalnagar, N., Hui, J., and D. Culler, 452 "Transmission of IPv6 Packets over IEEE 802.15.4 453 Networks", RFC 4944, DOI 10.17487/RFC4944, September 2007, 454 . 456 [RFC5424] Gerhards, R., "The Syslog Protocol", RFC 5424, 457 DOI 10.17487/RFC5424, March 2009, 458 . 460 [RFC6206] Levis, P., Clausen, T., Hui, J., Gnawali, O., and J. Ko, 461 "The Trickle Algorithm", RFC 6206, DOI 10.17487/RFC6206, 462 March 2011, . 464 [RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., 465 and A. Bierman, Ed., "Network Configuration Protocol 466 (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, 467 . 469 Appendix A. Change log [RFC Editor: Please remove] 471 draft-carpenter-anima-grasp-bulk-05, 2020-01-10: 473 * Minor technical clarifications. 474 * Converted to v3 format. 476 draft-carpenter-anima-grasp-bulk-04, 2019-07-03: 478 * Updated description of very small link-layer MTU issue. 479 * Clarified informational status, updated reference. 481 draft-carpenter-anima-grasp-bulk-03, 2019-01-07: 483 * Added future work section, implementation status. 485 draft-carpenter-anima-grasp-bulk-02, 2018-06-30: 487 * Update reference, fix TBDs. 489 draft-carpenter-anima-grasp-bulk-01, 2018-03-03: 491 * Updates after IETF100 discussion. 493 draft-carpenter-anima-grasp-bulk-00, 2017-09-12: 495 * Initial version. 497 Authors' Addresses 499 Brian Carpenter 500 School of Computer Science 501 University of Auckland 502 PB 92019 503 Auckland 1142 504 New Zealand 506 Email: brian.e.carpenter@gmail.com 508 Sheng Jiang 509 Huawei Technologies Co., Ltd 510 Q14 Huawei Campus 511 156 Beiqing Road 512 Hai-Dian District 513 Beijing 514 100095 515 China 517 Email: jiangsheng@huawei.com 519 Bing Liu 520 Huawei Technologies Co., Ltd 521 Q14 Huawei Campus 522 156 Beiqing Road 523 Hai-Dian District 524 Beijing 525 100095 526 China 528 Email: leo.liubing@huawei.com