idnits 2.17.1 draft-rfced-exp-rupp-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-24) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 1) being 68 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 8 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** The abstract seems to contain references ([2], [4], [5], [6], [7], [8], [9], [10], [11], [1]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '3' is defined on line 357, but no explicit reference was found in the text ** Obsolete normative reference: RFC 977 (ref. '1') (Obsoleted by RFC 3977) -- Possible downref: Non-RFC (?) normative reference: ref. '4' -- Possible downref: Non-RFC (?) normative reference: ref. '5' -- Possible downref: Non-RFC (?) normative reference: ref. '6' ** Obsolete normative reference: RFC 1036 (ref. '7') (Obsoleted by RFC 5536, RFC 5537) ** Downref: Normative reference to an Informational RFC: RFC 1950 (ref. '8') -- Possible downref: Non-RFC (?) normative reference: ref. '9' ** Obsolete normative reference: RFC 793 (ref. '10') (Obsoleted by RFC 9293) -- Possible downref: Non-RFC (?) normative reference: ref. '11' Summary: 12 errors (**), 0 flaws (~~), 4 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 INTERNET-DRAFT EXPIRES OCTOBER 1997 INTERNET-DRAFT 2 Network Working Group Heiko W.Rupp 3 INTERNET-DRAFT 4 Experimental 29.04.1997 6 A Protocol for the Transmission of Net News Articles 7 over IP multicast. 8 10 Status of this Memo 12 This document is an Internet-Draft. Internet-Drafts are working 13 documents of the Internet Engineering task Force (IETF), its areas, 14 and its working groups. Note that other groups may also distribute 15 working documents as Internet-Drafts. 17 Internet-Drafts are draft documents valid for a maximum of six 18 months and may be updated, replaced, or obsoleted by other 19 documents at any time. It is inappropriate to use Internet-Drafts 20 as reference material or to cite them other than as "work in 21 progress". 23 To learn the current status of any Internet-Draft, please check the 24 "1id-abstract.txt" listing contained in the Internet-Drafts Shadow 25 Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), 26 munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or 27 ftp.isi.edu (US West Coast). 29 Abstract 31 This protocol provides a way to use the IP multicast infrastructure 32 to transmit NetNews articles between news servers. Doing so will 33 reduce the bandwidth that is actually needed for transmission via 34 NNTP. This does not affect how news reading clients communicate with 35 servers. 37 Overview and Rationale 39 NetNews are bulk data that are produced in large quantities every day 40 around the world. Distribution of NetNews on the Internet are usually 41 distributed with NNTP[1]. In order to get a fast and redundant 42 distribution many news servers communicate with many others, thus 43 imposing a higher load on the underlying network than necessary. 45 Assume the following scenario: 47 +--------- R1 48 / 49 S -- A ------- B -------- R2 50 \ 51 +--------- R3 53 A sender S which wants to transmit articles via NNTP to receivers 54 R1...R3 will thus transmit them three times across the link from A to 55 B. 57 With IP Multicast[2], an efficient way to distribute datagrams to 58 groups of users exists in the Internet. Thus articles would traverse 59 the link A to B only once, thus reducing load on that link. 61 This cannot be done with existing news transfer technology, as it is 63 =0C 64 I/D NETNEWS OVER IP MULTICAST 29 April 1997 66 based on TCP[10] which cannot be multicasted. The protocol described 67 in this memo is designed to put news articles into datagrams and 68 distribute them via IP multicast to receivers that are interested in 69 the specific newsgroup. For more information about NetNews, refer to 70 [7] and [1]. 72 In the rest of this memo I will refer to the protocol as ``Multicast 73 News Transport Protocol'' (Mcntp). 75 Protocol overview 77 This paragraph will show how news articles are propagated with Mcntp. 78 Basically, three parties are involved: 80 + Multicast directory service, MD, coordinates the assignment 81 between multicast and news groups. 83 + A Multicast sender, MS, that sends news articles over an 84 IP multicast infrastructure 86 + A Multicast receiver, MR, gets packets from the IP multicast 87 infrastructure and processes them further. 89 So this can be seen as follows: 91 directory +---------+ directory 92 +------------- > | | ----------------+ 93 \|/ | MD | \|/ 94 +---------+ | | +--------+ 95 | | +---------+ | | 96 | MS | | MR | 97 | | ------------ articles -------> | | 98 +---------+ +--------+ 100 MS and MR can be implemented into existing news server software, or 101 can be implemented as separate processes that communicate with the 102 news servers (e.g. via NNTP); this does not matter to the protocol. 104 MD can either be implemented within MS, or as separate processes that 105 communicate with each other. A practical way is to have on MD per 106 sender host so that communication between MD and MS is fast and 107 reliable, while not too many resources are needed. 109 The protocol itself consist of two parts that will be presented in 110 the next two chapters -- Distribution of articles and the directory 111 service. 113 =0C 114 I/D NETNEWS OVER IP MULTICAST 29 April 1997 116 Packet format -- Distribution of articles 118 To send articles via IP multicast, they have to be encapsulated into 119 UDP packets. The following diagram shows how this can be done: 121 +---------------------------------------+ 122 | Magic | 123 +----+----+---------+---------+---------+ 124 | Ver|Rev | Comp | Crypt | Offset | 125 +----+----+---------+---------+---------+ 126 | Original length | 127 +---------------------------------------+ 128 | Length as sent | 129 +---------------------------------------+ 130 | Crc | 131 +---------------------------------------+ 132 | Message-id | 133 +---------------------------------------+ 134 | Data | 135 +---------------------------------------+ 137 All entries are in network byte order. The fields have the following 138 meaning and types: 140 + Magic (32-bit): The String ``McNt'' 142 + Ver (4-bit): Protocol version -- currently 1 144 + Rev (4-bit): Protocol revision -- currently 1 146 + Comp (8-bit): Compression method used. Currently are only 2 147 methods defined: 149 0 Article is not compressed 151 1 Article is compressed via zlib [8] 153 + Crypt (8-bit): Encryption method used. See below 155 + Offset (8-bit): Offset of article data from packet start 157 + Original length (32-bit): Length of article before compression 158 and encryption 160 + Length as sent (32-bit): 162 + Length of the ``Data'' field (see below) 164 =0C 165 I/D NETNEWS OVER IP MULTICAST 29 April 1997 167 + Crc (32-bit): Checksum over the original articles. See also 168 below. 170 + Message-ID: The message id of the article in the form it is 171 defined in RFC 1036 [7], terminated by a null byte. 173 + Data: The article data after possible compression and 174 encryption. 176 This memo does not specify a encryption method for the case that the 177 field ``Crypt'' is set to anything other than 0; the involved parties 178 (i.e. the senders of encrypted news and their receivers) have to 179 agree on a method they want to use. If encryption and compression is 180 used then the article data is first to be compressed and then the 181 result to be encrypted. 183 The checksum is computed over the original article data (i.e. before 184 possible compression and encryption) by the algorithm of IEEE 185 Std1003.2-1992 which is also used in 4.4 BSD cksum(1) tool. See also 186 [11]. 188 To send an article off, it is encapsulated and then just sent to the 189 appropriate multicast group. There is no feedback from the receiver 190 to the sender when an article is received. 192 Directory service 194 In order to get a relation between newsgroups and multicast groups, a 195 directory service exists; this has been referenced as MD above. When 196 a sender MS wants to propagate a news group, it asks the directory 197 service for a multicast group it can use to distribute articles, 198 waits for the reply, and starts to send. The directory server 199 registers this group in its tables and periodically distributes this 200 table over IP multicast. For this purpose, the multicast group 201 ``mcntp-directory.mcast.net'' has been officially been assigned by 202 the IANA. The UDP port which announcements are sent to, has 203 officially been assigned by the IANA as UDP port number 5418 with the 204 name ``mcntp''. 206 Announcements should not be sent too often to keep traffic low, but 207 often enough that new receivers don't have to wait to long to be able 208 to receive articles. Once a minute is assumed to be a good value 209 here. Announcements can be sent less often if they are transmitted 210 immediately after a change in the directory. 212 If more than one directory server is involved (e.g. if there is more 213 than one sender site), the directory servers have to listen to 214 announcement packets on ``mcntp-directory.mcast.net''. If it does not 216 =0C 217 I/D NETNEWS OVER IP MULTICAST 29 April 1997 219 receive a packet after five times the waiting period (e.g., five 220 minutes) it can consider itself alone on the net and can choose the 221 multicast groups as it wishes. If it does receive a packet, it must 222 register the contained groups in its tables and has to select new 223 groups so that different newsgroups are not mapped on the same 224 multicast group. 226 When a receiver (MR) wants to receive a newsgroup, it listens on 227 ``mcntp-directory.mcast.net'' for announcements, parses them, and 228 then joins the appropriate multicast groups. 230 Multicast groups that are no longer in use (e.g. because the sender 231 has stopped working) must be removed from the announcement. 233 The format of those announcement packets is: 235 +-----------+------+-----+--------+ 236 | Magic | Vers | Rev | Offset | 237 +-----------+------+-----+--------+ 238 | Length | 239 +---------------------------------+ 240 | Crc | 241 +---------------------------------+ 242 | Sender-ID | 243 +-----------------+------+---+-----------+ -+ 244 | Multicast group | Port |TTL| Newsgroup | | 245 +-----------------+------+---+-----------+ | 246 ... repeat ... | NG lines 247 +-----------------+------+---+-----------+ | 248 | Multicast group | Port |TTL| Newsgroup | | 249 +-----------------+------+---+-----------+ -+ 251 All numbers are in network byte order. The fields have the following 252 meaning and types: 254 + Magic (16-bit): The Bytes 0xab 256 + Vers (4-bit): Protocol version 258 + Rev (4-bit): Protocol revision (currently 1) 260 + Offset (8-bit): Offset of NG-lines from packet start. 262 + Length (32-bit): Total packet length. 264 + Crc (32-bit): Checksum over the rest of the packet. 266 =0C 267 I/D NETNEWS OVER IP MULTICAST 29 April 1997 269 + Sender-ID : Identification of sender host, terminated by a 270 null byte (see below). 272 + Multicast group (32-bit *): The associated multicast group 274 + Port (8-bit): UDP Port to use for this group 276 + TTL (8-bit): Time to live for multicast packets. 278 + Newsgroup: Name of the Newsgroup in wildmat(3) format, 279 terminated by a null byte. 281 The protocol version (Vers) is currently 1 for IPv4 and 11 for IPv6. 282 The multicast group field (*) is 32 bit in size for IPv4 and 128-bit 283 for IPv6 in size. 285 The lenght field is 32 bit in size to support IPv6 jumbo datagrams. 287 The sender-ID is normally the fully qualified domain name of the 288 hosts that sends the announcement. As is common practice with 289 NetNews, this can also be the (possibly shorter) entry that the host 290 puts in the ``Path:'' header when an article passes through it. 292 The checksum is computed over all lines with newsgroup to multicast 293 group relations in the packet by the algorithm of IEEE Std1003.2-1992 294 which is also used in 4.4 BSD cksum(1) tool. See also [11]. 296 The lines with newsgroup to multicast group relations are repeated as 297 often as needed to announce all groups. The TTL can be used by 298 clients to find out if packets that come from this source can reach 299 them, or if the sender is too far away. Note that all entries have 300 to fit into one UDP packet. 302 Reliability Considerations 304 As UDP is a unreliable service, provisions for reliable distribution 305 of articles are needed. There exist some approaches to reliable 306 multicast (XTP [4], KLG [5] or RMTP [6]) which all suffer from some 307 problem or other. Specifically, additional hard- or software is 308 needed and usually requires kernel modification. 310 As there is already a reliable transport of NetNews via NNTP, there 311 is no need for a reliable transport via IP multicast: articles need 312 not be in order, so it is no problem if one is missing in the 313 multicast. Since articles need not arrive in order, lost or missing 314 articles can easily be transmitted via an additional NNTP feed. 316 As UDP packets can be at maximum 64kBytes in size and every Mcntp 318 =0C 319 I/D NETNEWS OVER IP MULTICAST 29 April 1997 321 packet has to fit in one UDP packet, there is no provision given to 322 distribute news articles larger than about 63kBytes in size (other 323 than compressing them). This does not matter much in practise as 324 recent research has shown that more than 95% off all news articles 325 are smaller 64kBytes [9]. The remaining 5% can still be transferred 326 via NNTP. Some hosts may have problems in receiving UDP packets as 327 large as 64kBytes, so in practical use article sizes of 16kBytes 328 would be appropriate. 330 Summary 332 The distribution of NetNews articles via IP multicast can be a way to 333 decrease the network bandwidth used to distribute them. Articles are 334 delivered fast via a nonreliable protocol; later, the holes are 335 filled via a reliable, already existing protocol. Compression of 336 articles can further reduce the network Load. With encryption private 337 news groups can be established on a public IP multicast 338 infrastructure. A prototype of a reference implementation already 339 shows that Mcntp is fast and can be used as an alternative to 340 classical transports. 342 Security Considerations 344 This memo touches security only as far as the provision for 345 encryption of article data is given in the protocol. 347 No further security issues are discussed. 349 References 351 [1] RFC 977 -- B. Kantor, P. Lapsley, "Network News Transfer Protocol: 352 A Proposed Standard for the Stream-Based Transmission of News". 354 [2] RFC 1112 -- S. Deering, "Host extensions for IP multicasting", 355 08/01/1989. 357 [3] RFC 768 -- J. Postel, "User Datagram Protocol", 08/28/1980. 359 [4] XTP -- W. T. Strayer, D.J. Dmepsey, B.C. Weaver, "XTP: The Xpress 360 Transfer Protocol", Addison-Wesley 362 [5] KLG -- M. Hofmann, "Zuverlaessige Kommunikation in heterogenen 363 Netzen", Thesis at "Institut fuer Telematik, CS Dept. Univ 364 Karlsruhe" 366 [6] RMTP -- Lin, John C., Paul Sanjoy, "RMTP: A Reliable Multicast 367 Transport Protocol". 369 =0C 370 I/D NETNEWS OVER IP MULTICAST 29 April 1997 372 [7] RFC 1036 -- M. Horton, R. Adams, "Standard for interchange of 373 USENET messages", 12/01/1987. 375 [8] RFC 1950 -- L. Deutsch, J. Gailly, "ZLIB Compressed Data Format 376 Specification version 3.3", 05/23/1996. 378 [9] http://www.xlink.net/~hwr/histo/ -- Some Statistics about size dis- 379 tribution of NetNews 381 [10] RFC 793 -- J. Postel, "Transmission Control Protocol", 09/01/1981. 383 [11] Dilip V. Sarwate, "Computation of Cyclic Redundancy Checks Via 384 Table Lookup", Communications of the ACM, August 1988. 386 Author's Address 388 Heiko W.Rupp 389 Gerwigstr. 5 390 D-76131 Karlsruhe 392 Phone: +49 721 9661524 394 EMail: hwr@pilhuhn.de