idnits 2.17.1 draft-bormann-t2trg-stp-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 18, 2018) is 2231 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-08) exists of draft-ietf-netconf-notification-messages-03 == Outdated reference: A later version (-15) exists of draft-ietf-netconf-restconf-notif-04 == Outdated reference: A later version (-26) exists of draft-ietf-netconf-subscribed-notifications-10 == Outdated reference: A later version (-05) exists of draft-ietf-netconf-udp-pub-channel-01 == Outdated reference: A later version (-25) exists of draft-ietf-netconf-yang-push-15 -- Obsolete informational reference (is this intentional?): RFC 7230 (Obsoleted by RFC 9110, RFC 9112) Summary: 0 errors (**), 0 flaws (~~), 6 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group C. Bormann 3 Internet-Draft Universitaet Bremen TZI 4 Intended status: Informational March 18, 2018 5 Expires: September 19, 2018 7 The Series Transfer Pattern (STP) 8 draft-bormann-t2trg-stp-00 10 Abstract 12 Many applications make use of Series of data items, i.e., an array of 13 data items where new items can be added over time. Where such Series 14 are to be made available using REST protocols such as CoAP or HTTP, 15 the Series has to be mapped into a structure of one or more resources 16 and a protocol for a client to obtain the Series and to learn about 17 new items. 19 Various protocols have been standardized that make Series-shaped data 20 available, with rather different properties and objectives. The 21 present document is an attempt to extract a common underlying pattern 22 and to define media types and an access scheme that can be used right 23 away for further protocols that provide Series-shaped data. 25 Status of This Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at https://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on September 19, 2018. 42 Copyright Notice 44 Copyright (c) 2018 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (https://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 60 2. Objectives . . . . . . . . . . . . . . . . . . . . . . . . . 4 61 3. A REST Series Transfer Pattern (STP) . . . . . . . . . . . . 4 62 4. IANA considerations . . . . . . . . . . . . . . . . . . . . . 5 63 5. Security considerations . . . . . . . . . . . . . . . . . . . 5 64 6. Informative References . . . . . . . . . . . . . . . . . . . 6 65 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 7 66 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 7 68 1. Introduction 70 (TO DO: Insert an extended form of the abstract first here, expanding 71 the reference to [RFC7230] and [RFC7252] in the process.) 73 Examples for protocols that provide Series-shaped data are: 75 o The Atom Syndication Format [RFC4287] defines _feeds_ as Series of 76 _entries_ (links plus some metadata, which can often be much of 77 the content of an entry), where a feed is represented by a 78 collection resource that contains just a small number of the most 79 recent entries. By polling a feed, a client can contain a fresh 80 view of the Series, with a focus on recent items. If the client 81 does not poll often enough, it will _miss_ items. 83 o Messaging protocols such as XMPP or SIMPLE transfer series of what 84 is often called "Instant Messages". A publish/subscribe mechanism 85 allows a client to select sequences of messages that it is 86 interested in. 88 o Mail servers that provide interactive access to stored messages 89 present a Series to their clients. Obviously, loss of messages is 90 frowned upon. 92 o CoAP Observe allows a client to observe a resource as it changes; 93 the client can collect the changes into a Series. Observe is 94 focused on eventual consistency: a fresher data items simply 95 overwrites an older one. The present document uses the observe 96 pattern to build a more general Series Transfer Pattern. 98 o Syslog is an interesting case of a Series Transfer. 100 o [I-D.ietf-netconf-yang-push], 101 [I-D.voit-netmod-yang-notifications2], 102 [I-D.ietf-netconf-subscribed-notifications], 103 [I-D.ietf-netconf-notification-messages], 104 [I-D.ietf-netconf-restconf-notif]. 106 o An RTP stream can be viewed as an (somehat extreme) case of a 107 Series; new items are just sent inside separate UDP packets. A 108 sequence number allows to detect (but not normally ask for 109 retransmission of) missing items. A timestamp as well as source 110 data (SSRC, CSRC) provide further common metadata that aid in the 111 processing of the Series items. 113 o An example of an ad-hoc design of a series transfer protocol is 114 [I-D.ietf-netconf-udp-pub-channel]. 116 o Server-sent events [sse] are a somewhat bizarre version of a 117 series transfer protocol. 119 o The Interface for Metadata Access Points (IF-MAP) specified by the 120 Trusted Computing Group and emerging derivates of that protocol 121 create a series of updates to a graph representation of related 122 network-related security information. The requests created by IF- 123 MAP clients are bundled operations of updates to a MAP Graph, 124 which compose a Series Transfer Pattern of bundled atomic 125 operations that ensure the integrity of the MAP Graph. [Henk 126 Birkholz] 128 o netflow/IPFIX was defined to transfer a series of data items about 129 flows. Information about PDU flows accounted by network 130 interfaces of endpoints is emitted in a series of counter bundles 131 via the IPFIX protocol. Only a series of these continuous Flow 132 Records creates a ' meaningful bigger picture about the current 133 traffic in the network topology of an administrative domain. 134 Depending on the characteristics measured, loss of a Flow Record 135 can range from harmless to missing the only vital counter 136 measurement. [Henk Birkholz] 138 o TO DO: Add more items. 140 [I-D.birkholz-yang-push-coap-problemstatement] is a problem statement 141 that will require the design of another scheme to transfer Series- 142 shaped data. 144 2. Objectives 146 Series transfer applications may have rather different objectives. 148 o The completeness of the Series transfer may be of utmost 149 importance (e.g., if each item represents a sale), it may be 150 desirable but can be jettisoned in an overload situation, or it 151 may just be a likely outcome with a very active client (e.g., with 152 Atom). Note that there is never a way to _guarantee_ completeness 153 unless all of the rate and size of incoming new items, the 154 transfer capacity available, and the processing capabilities of 155 the client are controlled; however, system designs may want to 156 give the illusion of "reliability". 158 o Mimimizing the latency of the transfer may be important, as may be 159 limiting it below a defined maximum (note that these are different 160 objectives). The latter can be supported in a polling system by 161 polling at least as often as that maximum latency; this may be 162 considered inefficient and "push" mechanisms may be developed. 163 Mail environments have developed "push" services to enable 164 minimizing the latency. Where latency requirements go below the 165 time that might be needed for an end-to-end retransmission, error 166 concealment may provide an acceptable user experience (e.g., in 167 RTP). 169 In general, minimizing latency and ensuring completeness are 170 competing objectives. 172 Series transfer environments sometimes centralize information 173 distribution functions, leading to "broker" architectures (often 174 combined with the "publish/subscribe" pattern). With brokers, Series 175 publishers may use an entirely different interface to the brokers 176 from that used by the receiving clients, or the interfaces can be 177 designed so they are similar for all the forwarding steps. 179 3. A REST Series Transfer Pattern (STP) 181 A GET on a resource representing a Series may return a collection 182 item that contains the following pieces of information 184 o An array of Series items, either as an array of media-typed 185 objects in a suitable representation format (e.g., CBOR, MIME) or 186 by using an array-like media type (e.g., SenML). 188 * Items may be full items or limit themselves to some metadata 189 and a link; the client can then follow that link if it is 190 interested in the data (possibly basing that decision on the 191 metadata and/or a measure of load). 193 o A "cursor" that can then be used as a parameter in further GET 194 requests (see below) in order to receive only newer items than 195 those received with this transfer. 197 o A "more bit" that indicates whether such further items already 198 exist but could not be returned in the present response. 200 A GET may be enhanced with additional parameters (possibly turning it 201 into a FETCH): 203 o The cursor. 205 o A "wait bit" that indicates whether a (possibly empty) reply 206 should be given right away or the server should wait for new items 207 to become available. (To avoid the equivalence of the "silly 208 window syndrome", the wait bit may be enhanced by a minimum number 209 of items and a timeout after which even a smaller number is made 210 available). 212 A server may implement a form of custody transfer by interpreting the 213 cursor as an acknowledgement that the client has received all data up 214 to the cursor. This is not necessarily acting as an unsafe request 215 ("destructive GET"), as other clients may be active that have not yet 216 received all these data. To implement a full custody semantics, the 217 server needs to be aware of all the clients that expect a full Series 218 Transfer (a classical group management problem). 220 (Explain how Observe can help. Can it?) 222 4. IANA considerations 224 This memo registers a number of media types: TO DO. 226 o A media type for FETCH selectors (Section 3): 228 * An alternative way to encode this information into the URI of a 229 GET should also be available. 231 o A Series media type as alluded to in Section 3. 233 5. Security considerations 235 TO DO 237 6. Informative References 239 [I-D.birkholz-yang-push-coap-problemstatement] 240 Birkholz, H., Zhou, T., Liu, X., and E. Voit, "YANG Push 241 Operations for CoMI", draft-birkholz-yang-push-coap- 242 problemstatement-00 (work in progress), October 2017. 244 [I-D.ietf-netconf-notification-messages] 245 Voit, E., Birkholz, H., Bierman, A., Clemm, A., and T. 246 Jenkins, "Notification Message Headers and Bundles", 247 draft-ietf-netconf-notification-messages-03 (work in 248 progress), February 2018. 250 [I-D.ietf-netconf-restconf-notif] 251 Voit, E., Tripathy, A., Nilsen-Nygaard, E., Clemm, A., 252 Prieto, A., and A. Bierman, "RESTCONF and HTTP Transport 253 for Event Notifications", draft-ietf-netconf-restconf- 254 notif-04 (work in progress), January 2018. 256 [I-D.ietf-netconf-subscribed-notifications] 257 Voit, E., Clemm, A., Prieto, A., Nilsen-Nygaard, E., and 258 A. Tripathy, "Custom Subscription to Event Streams", 259 draft-ietf-netconf-subscribed-notifications-10 (work in 260 progress), February 2018. 262 [I-D.ietf-netconf-udp-pub-channel] 263 Zheng, G., Zhou, T., and A. Clemm, "UDP based Publication 264 Channel for Streaming Telemetry", draft-ietf-netconf-udp- 265 pub-channel-01 (work in progress), November 2017. 267 [I-D.ietf-netconf-yang-push] 268 Clemm, A., Voit, E., Prieto, A., Tripathy, A., Nilsen- 269 Nygaard, E., Bierman, A., and B. Lengyel, "YANG Datastore 270 Subscription", draft-ietf-netconf-yang-push-15 (work in 271 progress), February 2018. 273 [I-D.voit-netmod-yang-notifications2] 274 Voit, E., Bierman, A., Clemm, A., and T. Jenkins, "YANG 275 Notification Headers and Bundles", draft-voit-netmod-yang- 276 notifications2-00 (work in progress), February 2017. 278 [RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom 279 Syndication Format", RFC 4287, DOI 10.17487/RFC4287, 280 December 2005, . 282 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 283 Protocol (HTTP/1.1): Message Syntax and Routing", 284 RFC 7230, DOI 10.17487/RFC7230, June 2014, 285 . 287 [RFC7252] Shelby, Z., Hartke, K., and C. Bormann, "The Constrained 288 Application Protocol (CoAP)", RFC 7252, 289 DOI 10.17487/RFC7252, June 2014, 290 . 292 [sse] WHATWG, "HTML Living Standard -- 9.2 Server-sent events", 293 n.d., . 296 Acknowledgements 298 The need for a Series Transfer Pattern has been made clear by a 299 number of people that contribute to the IRTF Thing-to-Thing Research 300 Group (T2TRG), e.g. Matthias Kovatsch and Henk Birkholz (both of 301 whom also provided feedback on an early draft). Henk also 302 contributed further examples for the use of Series Transfers in 303 protocols. 305 Author's Address 307 Carsten Bormann 308 Universitaet Bremen TZI 309 Postfach 330440 310 Bremen D-28359 311 Germany 313 Phone: +49-421-218-63921 314 Email: cabo@tzi.org