idnits 2.17.1 draft-carpenter-anima-grasp-bulk-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (January 7, 2019) is 1936 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-08) exists of draft-ietf-cbor-cddl-06 == Outdated reference: A later version (-09) exists of draft-carpenter-anima-asa-guidelines-05 == Outdated reference: A later version (-10) exists of draft-ietf-anima-grasp-api-02 == Outdated reference: A later version (-13) exists of draft-liu-anima-grasp-distribution-09 Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group B. Carpenter 3 Internet-Draft Univ. of Auckland 4 Intended status: Informational S. Jiang 5 Expires: July 11, 2019 B. Liu 6 Huawei Technologies Co., Ltd 7 January 7, 2019 9 Transferring Bulk Data over the GeneRic Autonomic Signaling Protocol 10 (GRASP) 11 draft-carpenter-anima-grasp-bulk-03 13 Abstract 15 This document describes how bulk data may be transferred between 16 Autonomic Service Agents via the GeneRic Autonomic Signaling Protocol 17 (GRASP). 19 Status of This Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF). Note that other groups may also distribute 26 working documents as Internet-Drafts. The list of current Internet- 27 Drafts is at https://datatracker.ietf.org/drafts/current/. 29 Internet-Drafts are draft documents valid for a maximum of six months 30 and may be updated, replaced, or obsoleted by other documents at any 31 time. It is inappropriate to use Internet-Drafts as reference 32 material or to cite them other than as "work in progress." 34 This Internet-Draft will expire on July 11, 2019. 36 Copyright Notice 38 Copyright (c) 2019 IETF Trust and the persons identified as the 39 document authors. All rights reserved. 41 This document is subject to BCP 78 and the IETF Trust's Legal 42 Provisions Relating to IETF Documents 43 (https://trustee.ietf.org/license-info) in effect on the date of 44 publication of this document. Please review these documents 45 carefully, as they describe your rights and restrictions with respect 46 to this document. Code Components extracted from this document must 47 include Simplified BSD License text as described in Section 4.e of 48 the Trust Legal Provisions and are provided without warranty as 49 described in the Simplified BSD License. 51 Table of Contents 53 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 54 2. General Method for Bulk Transfer . . . . . . . . . . . . . . 3 55 3. Example for File Transfer . . . . . . . . . . . . . . . . . . 4 56 4. Loss Detection . . . . . . . . . . . . . . . . . . . . . . . 7 57 5. Maximum Transmission Unit . . . . . . . . . . . . . . . . . . 8 58 6. Pipelining . . . . . . . . . . . . . . . . . . . . . . . . . 8 59 7. Other Considerations . . . . . . . . . . . . . . . . . . . . 8 60 8. Possible Future Work . . . . . . . . . . . . . . . . . . . . 9 61 9. Implementation Status [RFC Editor: please remove] . . . . . . 9 62 10. Security Considerations . . . . . . . . . . . . . . . . . . . 9 63 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 64 12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 9 65 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 9 66 13.1. Normative References . . . . . . . . . . . . . . . . . . 9 67 13.2. Informative References . . . . . . . . . . . . . . . . . 10 68 Appendix A. Change log [RFC Editor: Please remove] . . . . . . . 11 69 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11 71 1. Introduction 73 The document [I-D.liu-anima-grasp-distribution] discusses how 74 information may be distributed within the secure Autonomic Networking 75 Infrastructure (ANI) [I-D.ietf-anima-reference-model]. Specifically, 76 it describes using the Synchronization and Flood Synchronization 77 mechanisms of the GeneRic Autonomic Signaling Protocol (GRASP) 78 [I-D.ietf-anima-grasp] for this purpose. However, those mechanisms 79 are limited to distributing GRASP Objective Options contained in 80 messages that cannot exceed the GRASP maximum message size of 2048 81 bytes. 83 There are scenarios in autonomic networks where this restriction is a 84 problem. One example is the distribution of network policy in 85 lengthy formats such as YANG or JSON. Another case might be an 86 Autonomic Service Agent (ASA) uploading a log file to the Network 87 Operations Center (NOC). A third case might be a supervisory system 88 downloading a software upgrade to an autonomic node. A related case 89 might be installing the code of a new or updated ASA to a target node 90 (see the discussion of ASA life cycles in 91 [I-D.carpenter-anima-asa-guidelines]). 93 Naturally, an existing solution such as a secure file transfer 94 protocol or secure HTTP might be used for this. Other management 95 protocols such as syslog [RFC5424] or NETCONF [RFC6241] might also be 96 used for related purposes, or might be mapped directly over GRASP. 97 The present document, however, applies to any scenario where it is 98 preferable to re-use the autonomic networking infrastructure itself 99 to transfer a significant amount of data, rather than install and 100 configure an additional mechanism. The basic model is to use the 101 GRASP Negotiation process to transfer and acknowledge multiple blocks 102 of data in successive negotiation steps. 104 The emphasis is placed on simplicity rather than efficiency, high 105 throughput, or advanced functionality. For example, if a transfer 106 gets out of step or data packets are lost, the strategy is to abort 107 the transfer and try again. In an enterprise network with low bit 108 error rates, and with GRASP running over TCP, this is not considered 109 a serious issue. Clearly, a more sophisticated approach could be 110 designed but if the application requires that, existing protocols 111 could be used, as indicated in the preceding paragraph. 113 NOTE: This is an early draft of a solution. As the specification 114 becomes more mature, the authors expect it to become precise enough 115 to be placed on the standards track. 117 2. General Method for Bulk Transfer 119 As for any GRASP operation, the two participants are considered to be 120 Autonomic Service Agents (ASAs) and they communicate using a specific 121 GRASP Objective Option, containing its own name, some flag bits, a 122 loop count, and a value. In bulk transfer, we can model the ASA 123 acting as the source of the transfer as a download server, and the 124 destination as a download client. No changes or extensions are 125 required to GRASP itself, but compared to a normal GRASP negotiation, 126 the communication pattern is slightly asymmetric: 128 1. The client first discovers the server by the GRASP discovery 129 mechanism (M_DISCOVERY and M_RESPONSE messages). 131 2. The client then sends a GRASP negotiation request (M_REQ_NEG 132 message). The value of the objective expresses the requested 133 item (e.g., a file name - see the next section for a detailed 134 example). 136 3. The server replies with a negotiation step (M_NEGOTIATE message). 137 The value of the objective is the first section of the requested 138 item (e.g., the first block of the requested file as a raw byte 139 string). 141 4. The client replies with a negotiation step (M_NEGOTIATE message). 142 The value of the objective is a simple acknowledgement (e.g., the 143 text string 'ACK'). 145 The last two steps repeat until the transfer is complete. The server 146 signals the end by transferring an empty byte string as the final 147 value. In this case the client responds with a normal end to the 148 negotiation (M_END message with an O_ACCEPT option). 150 Errors of any kind are handled with the normal GRASP mechanisms, in 151 particular by an M_END message with an O_DECLINE option in either 152 direction. 154 The block size must be chosen such that each step does not exceed the 155 GRASP message size limit of 2048 bits. 157 This approach is safe since each block must be positively 158 acknowledged, and data transfer errors will be detected by TCP. If a 159 future variant of GRASP runs over UDP, the mandatory UDP checksum for 160 IPv6 will detect such errors. The method does not specify 161 retransmission for failed blocks, so a failed transfer will need to 162 be restarted. 164 An observant reader will notice that the GRASP loop count mechanism, 165 intended to terminate endless negotiations, will cause a problem for 166 large transfers. For this reason, both the client and server must 167 artificially increment the loop count by 1 before each negotiation 168 step, cancelling out the normal decrement at each step. 170 If network load is a concern, the data rate can be limited by 171 inserting a delay before each negotiation step, with the GRASP 172 timeout set accordingly. Either the server or the client, or both, 173 could insert such a delay. Also, either side could use the GRASP 174 Confirm Waiting (M_WAIT) message to slow the other side down. 176 The description above concerns bulk download from a server 177 (responding ASA) to a client (requesting ASA). The data transfer 178 could also be in the opposite (upload) direction with minor 179 modifications to the procedure: the client would send the file name 180 and the data blocks, and the server would send acknowledgements. 182 3. Example for File Transfer 184 This example describes a client ASA requesting a file download from a 185 server ASA. 187 Firstly we define a GRASP objective informally: 189 ["411:mvFile", 3, 6, value] 191 The formal CDDL definition [I-D.ietf-cbor-cddl] is: 193 mvfile-objective = ["411:mvFile", objective-flags, loop-count, value] 195 objective-flags = ; as in the GRASP specification 196 loop-count = ; as in the GRASP specification 197 value = any 199 The objective-flags field is set to indicate negotiation. 201 Dry run mode must not be used. 203 The loop-count is set to a suitable value to limit the scope of 204 discovery. A suggested default value is 6. 206 The value takes the following forms: 208 o In the initial request from the client, a UTF-8 string containing 209 the requested file name (with file path if appropriate). 211 o In negotiation steps from the server, a byte string containing at 212 most 1024 bytes. However: 214 * If the file does not exist, the first negotiation step will 215 return an M_END, O_DECLINE response. 217 * After sending the last block, the next and final negotiation 218 step will send an empty byte string as the value. 220 o In negotiation steps from the client, the value is the UTF-8 221 string 'ACK'. 223 Note that the block size of 1024 is chosen to guarantee not only that 224 each GRASP message is below the size limit, but also that only one 225 TCP data packet will be needed, even on an IPv6 network with a 226 minimum link MTU. 228 We now present outline pseudocode for the client and the server ASA. 229 The API documented in [I-D.ietf-anima-grasp-api] is used in a 230 simplified way, and error handling is not shown in detail. 232 Pseudo code for client ASA (request and receive a file): 234 requested_obj = objective('411:mvFile') 235 locator = discover(requested_obj) 236 requested_obj.value = 'etc/test.pdf' 237 received_obj = request_negotiate(requested_obj, locator) 238 if error_code == declined: 239 #no such file 240 exit 242 file = open(requested_obj.value) 243 file.write(received_obj.value) #write to file 244 eof = False 245 while not eof: 246 received_obj.value = 'ACK' 247 received_obj.loop_count = received_obj.loop_count + 1 248 received_obj = negotiate_step(received_obj) 249 if received_obj.value == null: 250 end_negotiate(True) 251 file.close() 252 eof = True 253 else: 254 file.write(received_obj.value) #write to file 256 #file received 257 exit 258 Pseudo code for server ASA (await request and send a file): 260 supported_obj = objective('411:mvFile') 261 requested_obj = listen_negotiate(supported_obj) 262 file = open(requested_obj.value) #open the source file 263 if no such file: 264 end_negotiate(False) #decline negotiation 265 exit 267 eof = False 268 while not eof: 269 chunk = file.read(1024) #next block of file 270 requested_obj.value = chunk 271 requested_obj.loop_count = requested_obj.loop_count + 1 272 requested_obj = negotiate_step(requested_obj) 273 if chunk == null: 274 file.close() 275 eof = True 276 end_negotiate(True) 277 exit 278 if requested_obj.value != 'ACK': 279 #unexpected reply... 281 4. Loss Detection 283 The above description and example assume that GRASP is implemented 284 over a reliable transport layer such as TCP, such that lost or 285 corrupted messages are not likely. Rarely, an error might be 286 detected via a missing ACK, in which case the transfer would be 287 aborted and restarted. In the event that GRASP is implemented over 288 an unreliable transport layer such as UDP, it would be possible to 289 add a block number to both the data block and acknowledgement 290 objectives, so that missing blocks can be retransmitted, or duplicate 291 blocks can be ignored. For example, the objective in Section 3 would 292 become: 294 mvfile-objective = ["411:mvFile", objective-flags, loop-count, value] 296 objective-flags = ; as in the GRASP specification 297 loop-count = ; as in the GRASP specification 298 value = [block-number, any] 299 block-number = uint 301 It would also be necessary for the transport layer to detect data 302 errors, for example by enabling UDP checksums. 304 5. Maximum Transmission Unit 306 In an IPv6 environment, a minimal MTU of 1280 bytes can be assumed, 307 and assuming that high throughput is not a requirement, bulk 308 transfers can be designed to match that MTU. However, there are 309 environments where the underlying physical MTU is much smaller. For 310 example, on an IEEE 802.15.4 network it may be less than 100 bytes 311 [RFC4944]. In such a case, a bulk transfer solution has several 312 choices: 314 1. Accept the overhead of an adaptation layer, and therefore assume 315 a network-layer MTU of 1280 bytes. Indeed, the presence of such 316 an adaptation layer may be impossible to detect. 318 2. Attempt to determine the actual MTU available without lower-layer 319 fragmentation. This however will be impossible without using 320 low-level functions of the socket interface. 322 3. Attempt to determine a message size that provides optimum 323 performance, by some sort of trial-and-error solution. 325 These complexities suggest that using a GRASP-based mechanism is 326 unlikely to be optimal in environments with a very small physical 327 MTU. 329 6. Pipelining 331 The above description and example descibe a simple handshake model 332 where each block is acknowledged before the next block is sent. For 333 the scenarios discussed in Section 1, this should be acceptable. 334 Therefore we do not suggest adding a pipelining or windowing 335 mechanism. If high throughput is required, a conventional file 336 transfer protocol should be used. 338 7. Other Considerations 340 If multiple transfers are requested simultaneously, each one will 341 proceed as a separate GRASP negotiation session. The ASA acting as 342 the server must be coded accordingly, like any ASA that needs to 343 handle simultaneous sessions [I-D.carpenter-anima-asa-guidelines]. 345 Bulk transfer might become a utility function for use by various 346 ASAs, such as those supporting YANG or JSON distribution, log file 347 uploads, or code downloads. In this case some form of user space API 348 for bulk transfer will be required. This could be in the form of an 349 inter-process communication call between the ASA in question and the 350 ASA implementing the bulk transfer mechanism. The details are out of 351 scope for this document. 353 8. Possible Future Work 355 The simple file transfer mechanism described above is only an 356 example. Other application scenarios should be developed. 358 The mechanism described in this document is suitable for simple 359 unicast scenarios where GRASP runs over TCP and can be treated as a 360 reliable protocol. A more sophisticated approach would be needed in 361 at least two cases: 363 1. A scenario where GRASP runs over UDP, where error detection and 364 retransmission would be essential. 366 2. A scenario where multicast data distribution is required, so that 367 a mechanism such as Trickle [RFC6206] would be appropriate. 369 These solutions might also require extensions to the GRASP protocol 370 itself. 372 9. Implementation Status [RFC Editor: please remove] 374 A prototype open source Python implementation of simple file transfer 375 has been used to verify the mechanism described above. It may be 376 found at https://github.com/becarpenter/graspy/blob/master/getter.py 377 and https://github.com/becarpenter/graspy/blob/master/pusher.py . 379 10. Security Considerations 381 All GRASP transactions are secured by the mandatory security 382 substrate required by [I-D.ietf-anima-grasp]. No additional security 383 issues are created by the application of GRASP described in this 384 document. 386 11. IANA Considerations 388 This document makes no request of the IANA. 390 12. Acknowledgements 392 Thanks to Joel Halpern and other members of the ANIMA WG. 394 13. References 396 13.1. Normative References 398 [I-D.ietf-anima-grasp] 399 Bormann, C., Carpenter, B., and B. Liu, "A Generic 400 Autonomic Signaling Protocol (GRASP)", draft-ietf-anima- 401 grasp-15 (work in progress), July 2017. 403 [I-D.ietf-cbor-cddl] 404 Birkholz, H., Vigano, C., and C. Bormann, "Concise data 405 definition language (CDDL): a notational convention to 406 express CBOR and JSON data structures", draft-ietf-cbor- 407 cddl-06 (work in progress), November 2018. 409 13.2. Informative References 411 [I-D.carpenter-anima-asa-guidelines] 412 Carpenter, B., Ciavaglia, L., Jiang, S., and P. Pierre, 413 "Guidelines for Autonomic Service Agents", draft- 414 carpenter-anima-asa-guidelines-05 (work in progress), June 415 2018. 417 [I-D.ietf-anima-grasp-api] 418 Carpenter, B., Liu, B., Wang, W., and X. Gong, "Generic 419 Autonomic Signaling Protocol Application Program Interface 420 (GRASP API)", draft-ietf-anima-grasp-api-02 (work in 421 progress), June 2018. 423 [I-D.ietf-anima-reference-model] 424 Behringer, M., Carpenter, B., Eckert, T., Ciavaglia, L., 425 and J. Nobre, "A Reference Model for Autonomic 426 Networking", draft-ietf-anima-reference-model-10 (work in 427 progress), November 2018. 429 [I-D.liu-anima-grasp-distribution] 430 Liu, B., Jiang, S., Xiao, X., Hecker, A., and Z. 431 Despotovic, "Information Distribution in Autonomic 432 Networking", draft-liu-anima-grasp-distribution-09 (work 433 in progress), October 2018. 435 [RFC4944] Montenegro, G., Kushalnagar, N., Hui, J., and D. Culler, 436 "Transmission of IPv6 Packets over IEEE 802.15.4 437 Networks", RFC 4944, DOI 10.17487/RFC4944, September 2007, 438 . 440 [RFC5424] Gerhards, R., "The Syslog Protocol", RFC 5424, 441 DOI 10.17487/RFC5424, March 2009, 442 . 444 [RFC6206] Levis, P., Clausen, T., Hui, J., Gnawali, O., and J. Ko, 445 "The Trickle Algorithm", RFC 6206, DOI 10.17487/RFC6206, 446 March 2011, . 448 [RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., 449 and A. Bierman, Ed., "Network Configuration Protocol 450 (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, 451 . 453 Appendix A. Change log [RFC Editor: Please remove] 455 draft-carpenter-anima-grasp-bulk-03, 2019-01-07: 457 Added future work section, implementation status. 459 draft-carpenter-anima-grasp-bulk-02, 2018-06-30: 461 Update reference, fix TBDs. 463 draft-carpenter-anima-grasp-bulk-01, 2018-03-03: 465 Updates after IETF100 discussion. 467 draft-carpenter-anima-grasp-bulk-00, 2017-09-12: 469 Initial version. 471 Authors' Addresses 473 Brian Carpenter 474 Department of Computer Science 475 University of Auckland 476 PB 92019 477 Auckland 1142 478 New Zealand 480 Email: brian.e.carpenter@gmail.com 482 Sheng Jiang 483 Huawei Technologies Co., Ltd 484 Q14, Huawei Campus, No.156 Beiqing Road 485 Hai-Dian District, Beijing, 100095 486 P.R. China 488 Email: jiangsheng@huawei.com 489 Bing Liu 490 Huawei Technologies Co., Ltd 491 Q14, Huawei Campus 492 No.156 Beiqing Road 493 Hai-Dian District, Beijing 100095 494 P.R. China 496 Email: leo.liubing@huawei.com