idnits 2.17.1 draft-ietf-netconf-udp-pub-channel-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (September 23, 2017) is 2400 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '1' on line 494 -- Looks like a reference, but probably isn't: '2' on line 363 -- Looks like a reference, but probably isn't: '3' on line 361 -- Looks like a reference, but probably isn't: '6' on line 361 -- Looks like a reference, but probably isn't: '7' on line 363 -- Looks like a reference, but probably isn't: '4' on line 363 -- Looks like a reference, but probably isn't: '5' on line 363 -- Looks like a reference, but probably isn't: '8' on line 363 -- Looks like a reference, but probably isn't: '9' on line 359 -- Looks like a reference, but probably isn't: '0' on line 363 ** Downref: Normative reference to an Informational RFC: RFC 2104 ** Downref: Normative reference to an Informational RFC: RFC 2202 ** Downref: Normative reference to an Informational RFC: RFC 4493 ** Downref: Normative reference to an Informational RFC: RFC 6151 ** Obsolete normative reference: RFC 7049 (Obsoleted by RFC 8949) == Outdated reference: A later version (-26) exists of draft-ietf-netconf-subscribed-notifications-04 == Outdated reference: A later version (-25) exists of draft-ietf-netconf-yang-push-09 Summary: 5 errors (**), 0 flaws (~~), 4 warnings (==), 11 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 NETCONF G. Zheng 3 Internet-Draft T. Zhou 4 Intended status: Standards Track A. Clemm 5 Expires: March 27, 2018 Huawei 6 September 23, 2017 8 UDP based Publication Channel for Streaming Telemetry 9 draft-ietf-netconf-udp-pub-channel-00 11 Abstract 13 This document describes a UDP-based publication channel for streaming 14 telemetry use to collect data from devices. A new shim header is 15 proposed to facilitate the distributed data collection mechanism 16 which directly pushes data from line cards to the collector. Because 17 of the lightweight UDP encapsulation, higher frequency and better 18 transit performance can be achieved. 20 Requirements Language 22 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 23 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 24 document are to be interpreted as described in RFC 2119 [RFC2119]. 26 Status of This Memo 28 This Internet-Draft is submitted in full conformance with the 29 provisions of BCP 78 and BCP 79. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF). Note that other groups may also distribute 33 working documents as Internet-Drafts. The list of current Internet- 34 Drafts is at https://datatracker.ietf.org/drafts/current/. 36 Internet-Drafts are draft documents valid for a maximum of six months 37 and may be updated, replaced, or obsoleted by other documents at any 38 time. It is inappropriate to use Internet-Drafts as reference 39 material or to cite them other than as "work in progress." 41 This Internet-Draft will expire on March 27, 2018. 43 Copyright Notice 45 Copyright (c) 2017 IETF Trust and the persons identified as the 46 document authors. All rights reserved. 48 This document is subject to BCP 78 and the IETF Trust's Legal 49 Provisions Relating to IETF Documents 50 (https://trustee.ietf.org/license-info) in effect on the date of 51 publication of this document. Please review these documents 52 carefully, as they describe your rights and restrictions with respect 53 to this document. Code Components extracted from this document must 54 include Simplified BSD License text as described in Section 4.e of 55 the Trust Legal Provisions and are provided without warranty as 56 described in the Simplified BSD License. 58 Table of Contents 60 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 61 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 62 3. Solution Overview . . . . . . . . . . . . . . . . . . . . . . 4 63 4. UDP Transport for Publication Channel . . . . . . . . . . . . 6 64 4.1. Data Format . . . . . . . . . . . . . . . . . . . . . . . 6 65 4.2. Options . . . . . . . . . . . . . . . . . . . . . . . . . 8 66 4.2.1. Reliability Option . . . . . . . . . . . . . . . . . 8 67 4.2.2. Authentication Option . . . . . . . . . . . . . . . . 9 68 4.3. Data Encoding . . . . . . . . . . . . . . . . . . . . . . 10 69 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 70 6. Operational Considerations . . . . . . . . . . . . . . . . . 10 71 7. Security Considerations . . . . . . . . . . . . . . . . . . . 10 72 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 10 73 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 74 9.1. Normative References . . . . . . . . . . . . . . . . . . 10 75 9.2. Informative References . . . . . . . . . . . . . . . . . 11 76 9.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 12 77 Appendix A. Change Log . . . . . . . . . . . . . . . . . . . . . 12 78 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 12 80 1. Introduction 82 Streaming telemetry refers to sending a continuous stream of 83 operational data from a device to a remote receiver. This provides 84 an ability to monitor a network from remote and to provide network 85 analytics. Devices generate telemetry data and push that data to a 86 collector for further analysis. By streaming the data, much better 87 performance, finer-grained sampling, monitoring accuracy, and 88 bandwidth utilization can be achieved than with polling-based 89 alternatives. 91 Sub-Notif [I-D.ietf-netconf-subscribed-notifications] and YANG-Push 92 [I-D.ietf-netconf-yang-push] defines a mechanism that allows a 93 collector to subscribe to updates of YANG-defined data that is 94 maintained in a YANG [RFC7950] datastore. The mechanism separates 95 the management and control of subscriptions from the transport that 96 is used to actually stream and deliver the data. Two transports have 97 been defined so far, NETCONF [RFC6241] and RESTCONF [RFC8040]. 99 While powerful in its features and general in its architecture, in 100 its current form the mechanism needs to be extended to stream 101 telemetry data at high velocity from devices that feature a 102 distributed architecture. Specifically, there are two aspects that 103 need to be addressed: 105 1. The transports that have been defined so far, NETCONF and 106 RESTCONF, are ultimately based on TCP (Transmission Control 107 Protocol) and lack the efficiency needed to stream data 108 continuously at high velocity. A lighter-weight, more efficient 109 transport, e.g. a transport based on UDP (User Datagram Protocol) 110 is needed. 112 * Firstly, data collector will suffer a lot of TCP connection 113 from, for example, many line cards equipped on different 114 devices. 116 * Secondly, as no connection state needs to be maintained, UDP 117 encapsulation can be easily implemented by hardware which will 118 further improve the performance. 120 * Thirdly, because of the lightweight UDP encapsulation, higher 121 frequency and better transit performance can be achieved, 122 which is important for streaming telemetry. 124 2. The current design involves a single push server. In the case of 125 data originating from multiple line cards, the design requires 126 data to be internally forwarded from those line cards to the push 127 server, presumably on a main board, which then combines the 128 individual data items into a single consolidated stream. This 129 centralized data collection mechanism can result in a performance 130 bottleneck, especially when large amounts of data are involved. 131 What is needed instead is support for a distributed mechanism 132 that allows to directly push multiple individual substreams, e.g. 133 one from each line card, without needing to first pass them 134 through an additional processing stage for internal 135 consolidation, but still allowing those substreams to be managed 136 and controlled via a single subscription. 138 This document specifies a distributed data collection mechanism which 139 can directly push data from line cards to a collector by using a UDP 140 based publication channel. Specifically, a higher-performance 141 transport option for YANG-Push that leverages UDP is specified. 143 While this document will focus on the data publication channel, the 144 subscription can be used in conjunction with the mechanism proposed 145 in [I-D.ietf-netconf-yang-push] with necessary extensions. 147 Although the distributed data streaming from device line cards is one 148 typical scenario that the proposed UDP based publication channel can 149 be useful, the proposal is general enough to fit more scenarios that 150 require UDP transport for data collections, e.g. the IoT (Internet of 151 Things) use case. 153 2. Terminology 155 Streaming telemetry: refers to sending a continuous stream of 156 operational data from a device to a remote receiver. This provides 157 an ability to monitor a network from remote and to provide network 158 analytics. 160 Component subscription: A subscription that defines the data from 161 each individual entity which is managed and controlled by a single 162 subscription server. 164 Subscription agent: An agent that streams telemetry data per the 165 terms of a component subscription. 167 3. Solution Overview 169 The typical distributed data collection solution is shown in figure 170 1. The subscription server located in the main board receives the 171 subscription requests or configurations. It may be colocated, not 172 necessary, with a NETCONF server which interacts with outside 173 clients. When receiving a subscription request, the subscription 174 server decomposes the subscription into multiple component 175 subscriptions, each involving data from a separate internal telemetry 176 source, for example a line card. The component subscriptions are 177 distributed within the device to the subscription agents located in 178 line cards. Subsequently, each line card generates its own stream of 179 telemetry data, collecting and encapsulating the packets per the 180 component subscription and streaming it to the designated data 181 collector. 183 The publication channel supports the reliable data streaming, for 184 example for some alarm events. The subscriber has the option of 185 deducing the packet loss and the disorder based on the information 186 carried by the notification data. And the subscriber will decide the 187 behavior to request retransmission. The subscriber can send the 188 retransmission request to the subscriber server for further 189 processing. 191 Subscription server and subscription agents interact with each other 192 in several ways: 194 o Subscription agents need to have a registration or announcement 195 handshake with the subscription server, so the subscription server 196 is aware of them and of lifecycle events (such as subscription 197 agents appearing and disappearing). 199 o The subscription server relays the component subscriptions to the 200 subscription agents. 202 o The subscription agents indicate status of component subscriptions 203 to the subscription server. The status of the overall "master" 204 subscription is maintained by the subscription server. The 205 subscription server is also responsible for notifying the 206 subscriber in case of any problems of component subscriptions. 208 The rest of the draft describes the UDP based publication channel. 210 retransmission + + 211 request | | subscription 212 +------------------------+ 213 | | | Main Board| 214 | +--v----v--------+ | 215 | | subscription | | 216 | | server | | 217 | +--+----+-----+--+ | 218 | | | | | internal 219 +------------------------+ subscription 220 | | | distribution 221 +---------------+ | +--------------+ 222 | | | 223 +------------------+ +------------------+ +------------------+ 224 | | | | | | | | | 225 | +-------v------+ | | +------v-------+ | | +-----v--------+ | 226 | | subscription | | | | subscription | | | | subscription | | 227 | | agent | | | | agent | | | | agent | | 228 | +--------------+ | | +--------------+ | | +--------------+ | 229 | Line Card 1 | | Line Card 2 | | Line Card n | 230 +---------+--------+ +--------+---------+ +----------+-------+ 231 | | | 232 | | Publication Channel | 233 +--------------+ | +-----------------+ 234 | | | 235 +-v-----v-----v-+ 236 | | 237 | Collector | 238 | | 239 +---------------+ 241 4. UDP Transport for Publication Channel 243 In [I-D.voit-netconf-notification-messages], the transport 244 independent message header is proposed for the notification use. The 245 following shim header refers to and implements that message header 246 definition. 248 4.1. Data Format 250 The data format of the UDP based based publication transport is shown 251 as follows. 253 0 1 2 3 254 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 255 +---------------------------------------------------------------+ 256 ~ UDP Header ~ 257 +-------+---------------+-------+-------------------------------+ 258 | Vers. | Flag | Rsvd | Length | 259 +-------+---------------+-------+-------------------------------+ 260 | Notification-Time | 261 +---------------------------------------------------------------+ 262 | Message-Generator-ID | 263 +---------------------------------------------------------------+ 264 ~ Options ~ 265 +---------------------------------------------------------------+ 266 ~ Message Content ~ 267 +---------------------------------------------------------------+ 269 Right after the UDP header, a simple inform header is attached to 270 carry the necessary information with regard to the streaming mode. 272 o The Vers. field represents the PDU (Protocol Data Unit) encoding 273 version. The initial version value is 0. 275 o The Flag is a bitmap indicating what features this packet has and 276 the corresponding options attached. Each bit associates to one 277 feature and one option data. When the bit is set to 1, the 278 associated feature is enabled and the option data is attached. 279 The sequence of the presence of the options follows the bit order 280 of the bitmap. In this document, 2 flags are specified as 281 follows: 283 * bit 0, the reliability flag; 285 * bit 1, the authentication flag; 287 * other bits are reserved. 289 o The Length field is the total length of the message, measured in 290 octets, including message header. 292 o The Message-Generator-ID is a 32-bit identifier of the process 293 which created the message notification. This allows 294 disambiguation of an information source, such as the 295 identification of different line cards sending the notification 296 messages. 298 o The Notification-Time, is the time at which the message leaves the 299 exporter, expressed in seconds since the UNIX epoch of 1 January 300 1970 at 00:00 UTC, encoded as an unsigned 32-bit integer. 302 o The Options is a variable-length field. The details of the 303 Options will be described in the respective sections below. 305 After the inform header is the real content which is encoded. The 306 actual encoding is based on the subscription, e.g., in binary with 307 GPB [1] or CBOR [RFC7049]. 309 4.2. Options 311 The order of packing the data fields in the Options field follows the 312 bit order of the Flag field. 314 4.2.1. Reliability Option 316 The UDP based publication transport described in this document 317 provides two streaming modes, the reliable mode an the unreliable 318 mode, for different SLA (Service Level Agreement) and telemetry 319 requirements. 321 In the unreliable streaming mode, the line card pushes the 322 encapsulated data to the data collector without any sequence 323 information. So the subscriber does not know whether the data is 324 correctly received or not. Hence no retransmission happens. 326 The reliable streaming mode provides sequence information in the UDP 327 packet, based on which the subscriber can deduce the packet loss and 328 disorder. Then the subscriber can decide whether to request the 329 retransmission of the lost packets. 331 In most case, the unreliable streaming mode is preferred. Because 332 the reliable streaming mode will cost more network bandwidth and 333 precious device resource. Different from the unreliable streaming 334 mode, the line card cannot remove the sent reliable notifications 335 immediately, but to keep them in the memory for a while. Reliable 336 notifications may be pushed multiple times, which will increase the 337 traffic. When choosing the reliable streaming mode or the unreliable 338 streaming mode, the operate need to consider the reliable requirement 339 together with the resource usage. 341 When the reliability flag bit is set to 1 in the Flag field, the 342 following option data will be attached 343 0 1 2 3 344 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 345 +---------------------------------------------------------------+ 346 | Notification ID | 347 +---------------------------------------------------------------+ 348 | Previous Notification ID | 349 +---------------------------------------------------------------+ 351 The notification ID is generated continuously by the message 352 generator. Different subscribers share the same notification ID 353 sequence. Current ID and previous ID will be added in the packets. 355 For example, there are two subscriber A and B, 357 o Notification IDs for the generator are : [1, 2, 3, 4, 5, 6, 7, 8, 358 9], in which Subscriber A subscribes [1,2,3,6,7] and Subscriber B 359 subscribes [1,2,4,5,7,8,9]. 361 o Subscriber A will receive : [0,1][1,2][2,3][3,6][6,7]. 363 o Subscriber B will receive : [0,1][1,2][2,4][4,5][5,7][7,8]. 365 4.2.2. Authentication Option 367 When the authentication flag bit is set to 1 in the Flag field, a 24 368 octets data field will be included in the Options. The message is 369 signed, and the signature is filled in the 24 octets Authentication 370 Option field. So that a receiver can verify the authenticity of the 371 message. 373 HMAC [RFC2104] defines a mechanism for message authentication using 374 cryptographic hash functions. Any message digest algorithm can be 375 used, but the cryptographic strength of HMAC depends on the 376 properties of the underlying hash function. As suggested by 377 [RFC6151], new protocol designs should not employ HMAC-MD5 [RFC2202]. 378 Alternatives to HMAC-MD5 include HMAC-SHA256 [RFC4231] and AES-CMAC 379 [RFC4493]. 381 Implementations permit multiple acceptable algorithms, while the 382 HMAC-SHA256 algorithm is mandatory in this document. The resulting 383 message digest (output of HMAC) is truncated to 24 octets, which is 384 the 192 leftmost bits of the HMAC computation, to fit the size of the 385 Authentication Option field. It is recommended in [RFC2104] that the 386 truncated output length should be not less than half the length of 387 the hash output to match the birthday attack bound. 389 4.3. Data Encoding 391 Subscribed data can be encoded in GPB, CBOR, XML or JSON format. It 392 is conceivable that additional encodings may be supported as options 393 in the future. This can be accomplished by augmenting the 394 subscription data model with additional identity statements used to 395 refer to requested encodings. 397 5. IANA Considerations 399 TBD 401 6. Operational Considerations 403 While efficient, UDP has no build-in congestion-avoidance mechanism. 404 It is not recommended to use the UDP based publication channel over 405 congestion-sensitive network paths. The deployments require the 406 communications from exporters to collectors are always congestion 407 controllable, i.e., the transport is over dedicated links or the 408 streaming rate can be limited. 410 7. Security Considerations 412 The security of the UDP based publication channel depends on the 413 subscription channel. Typically, both NETCONF and RESTCONF support 414 the secure configuration of the private key for the publication 415 channel. So that the message data can be encrypted by using 416 symmetric key algorithms. 418 8. Acknowledgements 420 The authors of this documents would like to thank Eric Voit, Tim 421 Jenkins, and Huiyang Yang for the initial comments. 423 9. References 425 9.1. Normative References 427 [RFC2104] Krawczyk, H., Bellare, M., and R. Canetti, "HMAC: Keyed- 428 Hashing for Message Authentication", RFC 2104, 429 DOI 10.17487/RFC2104, February 1997, 430 . 432 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 433 Requirement Levels", BCP 14, RFC 2119, 434 DOI 10.17487/RFC2119, March 1997, 435 . 437 [RFC2202] Cheng, P. and R. Glenn, "Test Cases for HMAC-MD5 and HMAC- 438 SHA-1", RFC 2202, DOI 10.17487/RFC2202, September 1997, 439 . 441 [RFC4231] Nystrom, M., "Identifiers and Test Vectors for HMAC-SHA- 442 224, HMAC-SHA-256, HMAC-SHA-384, and HMAC-SHA-512", 443 RFC 4231, DOI 10.17487/RFC4231, December 2005, 444 . 446 [RFC4493] Song, JH., Poovendran, R., Lee, J., and T. Iwata, "The 447 AES-CMAC Algorithm", RFC 4493, DOI 10.17487/RFC4493, June 448 2006, . 450 [RFC6151] Turner, S. and L. Chen, "Updated Security Considerations 451 for the MD5 Message-Digest and the HMAC-MD5 Algorithms", 452 RFC 6151, DOI 10.17487/RFC6151, March 2011, 453 . 455 [RFC6241] Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., 456 and A. Bierman, Ed., "Network Configuration Protocol 457 (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011, 458 . 460 [RFC7049] Bormann, C. and P. Hoffman, "Concise Binary Object 461 Representation (CBOR)", RFC 7049, DOI 10.17487/RFC7049, 462 October 2013, . 464 [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", 465 RFC 7950, DOI 10.17487/RFC7950, August 2016, 466 . 468 [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF 469 Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, 470 . 472 9.2. Informative References 474 [I-D.ietf-netconf-subscribed-notifications] 475 Voit, E., Clemm, A., Prieto, A., Nilsen-Nygaard, E., and 476 A. Tripathy, "Custom Subscription to Event Notifications", 477 draft-ietf-netconf-subscribed-notifications-04 (work in 478 progress), September 2017. 480 [I-D.ietf-netconf-yang-push] 481 Clemm, A., Voit, E., Prieto, A., Tripathy, A., Nilsen- 482 Nygaard, E., Bierman, A., and B. Lengyel, "Subscribing to 483 YANG datastore push updates", draft-ietf-netconf-yang- 484 push-09 (work in progress), September 2017. 486 [I-D.voit-netconf-notification-messages] 487 Voit, E., Bierman, A., Clemm, A., and T. Jenkins, 488 "Notification Message Headers and Bundles", draft-voit- 489 netconf-notification-messages-01 (work in progress), July 490 2017. 492 9.3. URIs 494 [1] https://developers.google.com/protocol-buffers/ 496 Appendix A. Change Log 498 (To be removed by RFC editor prior to publication) 500 A.1. draft-ietf-zheng-udp-pub-channel-00 to v00 502 o Modified the telemetry header format. 504 o Add a section on the Authentication Option. 506 o Cleaned up the text and removed unnecessary TBDs. 508 Authors' Addresses 510 Guangying Zheng 511 Huawei 512 101 Yu-Hua-Tai Software Road 513 Nanjing, Jiangsu 514 China 516 Email: zhengguangying@huawei.com 518 Tianran Zhou 519 Huawei 520 156 Beiqing Rd., Haidian District 521 Beijing 522 China 524 Email: zhoutianran@huawei.com 525 Alexander Clemm 526 Huawei 527 2330 Central Expressway 528 Santa Clara, California 529 USA 531 Email: alexander.clemm@huawei.com