idnits 2.17.1 draft-rfced-exp-rupp-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-19) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 2 longer pages, the longest (page 1) being 68 lines == It seems as if not all pages are separated by form feeds - found 9 form feeds but 12 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** There is 1 instance of too long lines in the document, the longest one being 1 character in excess of 72. ** There are 2 instances of lines with control characters in the document. ** The abstract seems to contain references ([2], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [1]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '3' is defined on line 481, but no explicit reference was found in the text ** Obsolete normative reference: RFC 977 (ref. '1') (Obsoleted by RFC 3977) -- Possible downref: Non-RFC (?) normative reference: ref. '4' -- Possible downref: Non-RFC (?) normative reference: ref. '5' -- Possible downref: Non-RFC (?) normative reference: ref. '6' ** Obsolete normative reference: RFC 1036 (ref. '7') (Obsoleted by RFC 5536, RFC 5537) ** Downref: Normative reference to an Informational RFC: RFC 1950 (ref. '8') -- Possible downref: Non-RFC (?) normative reference: ref. '9' ** Obsolete normative reference: RFC 793 (ref. '10') (Obsoleted by RFC 9293) -- Possible downref: Non-RFC (?) normative reference: ref. '11' == Outdated reference: A later version (-05) exists of draft-ietf-mboned-admin-ip-space-04 -- Possible downref: Non-RFC (?) normative reference: ref. '13' -- Possible downref: Non-RFC (?) normative reference: ref. '14' Summary: 14 errors (**), 0 flaws (~~), 5 warnings (==), 9 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 INTERNET DRAFT EXPIRES SETP 1998 INTERNET DRAFT 2 Network Working Group Heiko W.Rupp 3 Experimental 8.3.1998 5 A Protocol for the Transmission of Net News Articles 6 over IP multicast. 7 9 Status of This Memo 11 This document is an Internet-Draft. Internet-Drafts are working 12 documents of the Internet Engineering Task Force (IETF), its 13 areas, and its working groups. Note that other groups may also 14 distribute working documents as Internet-Drafts. 16 Internet-Drafts are draft documents valid for a maximum of six 17 months and may be updated, replaced, or obsoleted by other 18 documents at any time. It is inappropriate to use Internet- 19 Drafts as reference material or to cite them other than as 20 "work in progress." 22 To learn the current status of any Internet-Draft, please check 23 the "1id-abstracts.txt" listing contained in the Internet- 24 Drafts Shadow Directories on ftp.is.co.za (Africa), 25 ftp.nordu.net (Europe), munnari.oz.au (Pacific Rim), 26 ds.internic.net (US East Coast), or ftp.isi.edu (US West Coast). 28 Distribution of this document is unlimited. 30 Abstract 32 Mcntp (Multicast News Transfer Protocol) provides a way to use the IP 33 multicast infrastructure to transmit NetNews articles between news 34 servers. Doing so will reduce the bandwidth that is actually needed 35 for transmission of articles which is mostly done via NNTP. This does 36 not affect how news reading clients communicate with servers. 38 Overview and Rationale 40 NetNews are bulk data that are produced in large quantities every day 41 around the world. Distribution of NetNews on the Internet are usually 42 distributed with NNTP[1]. In order to get a fast and redundant 43 distribution many news servers communicate with many others, thus 44 imposing a higher load on the underlying network than necessary. 46 Assume the following scenario: 48 +--------- R1 49 / 50 S -- A ------- B -------- R2 51 \ 52 +--------- R3 54 A sender S which wants to transmit articles via NNTP to receivers 55 R1...R3 will thus transmit them three times across the link from 56 A to B. 58 With IP Multicast[2], an efficient way to distribute datagrams to 59 groups of users exists in the Internet. Thus articles would traverse 60 the link A to B only once, thus reducing load on that link. 62 This cannot be done with existing news transfer technology, as it is 63 based on TCP[10] which cannot be multicasted. The protocol described 64 in this memo is designed to put news articles into datagrams and 65 distribute them via IP multicast to receivers that are interested in 66 the specific newsgroup. For more information about NetNews, refer to 67 [1] and [7]. 69 Protocol overview 71 This paragraph will show how news articles are propagated with Mcntp. 72 Basically, three parties are involved: 74 + Multicast directory service, MD, coordinates the assignment 75 between multicast and news groups. 77 + A Multicast sender, MS, that sends news articles over an 78 IP multicast infrastructure 80 + A Multicast receiver, MR, gets packets from the IP multicast 81 infrastructure and processes them further. 83 So this can be seen as follows: 85 directory +---------+ directory 86 +------------- > | | ----------------+ 87 \|/ | MD | \|/ 88 +---------+ | | +--------+ 89 | | +---------+ | | 90 | MS | | MR | 91 | | ------------ articles -------> | | 92 +---------+ +--------+ 94 MS and MR can be implemented into existing news server software, or 95 can be implemented as separate processes that communicate with the 96 news servers (e.g. via NNTP); this does not matter to the protocol. 98 MD can either be implemented within MS, or as separate processes that 99 communicate with each other. A practical way is to have on MD per 100 sender host so that communication between MD and MS is fast and 101 reliable, while not too many resources are needed. 103 The protocol itself consist of two parts that will be presented in 104 the next two chapters -- Distribution of articles and the directory 105 service. 107 Packet format -- Distribution of articles 109 To send articles via IP multicast, they have to be encapsulated into 110 UDP packets. The following diagram shows how this can be done: 112 +---------------------------------------+ 113 | Magic | 114 +----+----+----+----+---------+---------+ 115 | Ver|Rev |Comp|Cryp|Reserved | Offset | 116 +----+----+----+----+---------+---------+ 117 | Original length | 118 +---------------------------------------+ 119 | Length as sent | 120 +---------------------------------------+ 121 | Sender-ID | 122 +---------------------------------------+ 123 | Message-id | 124 +---------------------------------------+ 125 | Data | 126 +---------------------------------------+ 128 All entries are in network byte order. The fields have the following 129 meaning and types: 131 + Magic (32-bit): The String ``McNt'' 133 + Ver (4-bit): Protocol version -- currently 1 135 + Rev (4-bit): Protocol revision -- currently 1 137 + Comp (4-bit): Compression method used. Currently are only 2 138 methods defined: 140 0 Article is not compressed 142 1 Article is compressed via zlib [8] 144 + Cryp (4-bit): Encryption method used. See below 146 + Reserved (8-bit): Reserved for future extensions. 148 + Offset (8-bit): Offset of article data from packet start 150 + Original length (32-bit): Length of article before compression, 151 encryption and signing. 153 + Length as sent (including digital signature) (32-bit): Size of 154 the ``Data'' field (see below) 156 + Sender-ID: Identification of the sender host terminated by a 157 null byte (see below). 159 + Message-ID: The message id of the article in the form it is 160 defined in RFC 1036 [7], terminated by a null byte. 162 + Data: The signed article data after possible compression and 163 encryption. 165 This memo does not specify a encryption method for the case that the 166 field ``Crypt'' is set to anything other than 0; the involved parties 167 (i.e. the senders of encrypted news and their receivers) have to 168 agree on a method they want to use. If encryption and compression is 169 used then the article data is first to be compressed and then the 170 result to be encrypted. 172 All articles must be signed before sending them off the net. This is 173 accomplished by running the RIPEMD-160 message digest [11] algorithm 174 over the (possibly compressed and encrypted) article and then RSA- 175 encrypting the message digest with the private key that is suitable 176 for sender-id. The receiver decrypts the signature of the article 177 with the public key of sender-id and runs RIPEMD-160 over the data to 178 see if it has been altered on the way. An article with an invalid 179 signature or a non matching message digest has to be thrown away. The 180 sender-id can be the path entry or the hostname of the sending site; 181 there can also be more than one key pair per site e.g. to have 182 different keys for different newsgroups. The sender-id has to be 183 treated in a case independent manner. 185 Encryption of the message digest is done the following way. The 20 186 Bytes RIPEMD-160 message digest and the first 28 bytes of the 187 (possibly compressed and encrypted) message are tacked together to 188 form a 48 Bytes buffer. This buffer is then encrypted with the right 189 RSA private key and prepended to the original message without the 190 first 28 bytes: 192 +----------------+---------------------------------+ 193 | Message digest | message | 194 +----------------+-----------+---------------------+ 195 1 28 n 196 | \ 198 +-----------------------+----------------------------------+ 199 | Signature | message without first 28 Bytes | 200 +-----------------------+----------------------------------+ 202 To send an article off, it is encapsulated and then just sent to the 203 appropriate multicast group. There is no feedback from the receiver 204 to the sender when an article is received. 206 Directory service 208 In order to get a relation between newsgroups and multicast groups, a 209 directory service exists; this has been referenced as MD above. When 210 a sender MS wants to propagate a news group, it asks the directory 211 service for a multicast group it can use to distribute articles, 212 waits for the reply, and starts to send. The directory server 213 registers this group in its tables and periodically distributes this 214 table over IP multicast. For this purpose, the multicast group 215 ``mcntp-directory.mcast.net'' has been officially been assigned by 216 the IANA. The UDP port which announcements are sent to, has 217 officially been assigned by the IANA as UDP port number 5418 with the 218 name ``mcntp''. 220 Announcements should not be sent too often to keep traffic low, but 221 often enough that new receivers don't have to wait to long to be able 222 to receive articles. Once a minute is assumed to be a good value 223 here. Announcements can be sent less often if they are transmitted 224 immediately after a change in the directory. 226 If more than one directory server is involved (e.g. if there is more 227 than one sender site), the directory servers have to listen to 228 announcement packets on ``mcntp-directory.mcast.net''. If it does not 229 receive a packet after five times the waiting period (e.g., five 230 minutes) it can consider itself alone on the net and can choose the 231 multicast groups as it wishes. See below on usage scenarios which 232 further explain this. 234 Groups that are local to an organisation (e.g. an ISP) or should stay 235 within their bounds, must be transported within the range of the 236 administratively scoped multicast addresses [12]. 238 When a receiver (MR) wants to receive a newsgroup, it listens on 239 ``mcntp-directory.mcast.net'' for announcements, parses them, and 240 then joins the appropriate multicast groups. 242 Multicast groups that are no longer in use (e.g. because the sender 243 has stopped working) must be removed from the announcement. 245 The format of those announcement packets is: 247 +-----------+------+-----+--------+ 248 | Magic | Vers | Rev | Offset | 249 +-----------+------+-----+--------+ 250 | Length | 251 +---------------------------------+ 252 | rmd160 | 253 +---------------------------------+------+ 254 | Sender-ID | pad1 | 255 +-----------------+------+---+-----------+---+ -+ 256 | Multicast group | Port |TTL| Newsgroup |pad| | 257 +-----------------+------+---+-----------+---+ | 258 ... repeat ... | NG lines 259 +-----------------+------+---+-----------+---+ | 260 | Multicast group | Port |TTL| Newsgroup |pad| | 261 +-----------------+------+---+-----------+---+ -+ 263 All numbers are in network byte order. The fields have the following 264 meaning and types: 266 + Magic (16-bit): The Bytes 0xabba. 268 + Vers (4-bit): Protocol version (see below). 270 + Rev (4-bit): Protocol revision (currently 1). 272 + Offset (8-bit): Offset of NG-lines from packet start. 274 + Length (32-bit): Total packet length. 276 + rmd160 (160-bit): RIPEMD-160 message digest over the rest of 277 the packet. 279 + Sender-ID : Identification of sender host, terminated by a 280 null byte (see below). 282 + Pad1: Padding to next 4-Byte boundary filled with null 283 bytes. 285 + Multicast group (32-bit *): The associated multicast group. 287 + Port (16-bit): UDP Port to use for this group. 289 + TTL (8-bit): Time to live for multicast packets. 291 + Newsgroup: Name of the Newsgroup(s), terminated by a null 292 byte. See also below. 294 + Pad: Padding of the string to the next 4-bytes boundary 295 filled with null bytes. 297 The protocol version (Vers) is currently 1 for IPv4 and 11 for IPv6. 298 The multicast group field (*) is 32 bit in size for IPv4 and 128-bit 299 for IPv6 in size. 301 The length field is 32 bit in size to support IPv6 jumbo datagrams. 303 The sender-ID is normally the fully qualified domain name of the 304 hosts that sends the announcement. As is common practice with 305 NetNews, this can also be the (possibly shorter) entry that the host 306 puts in the ``Path:'' header when an article passes through it. This 307 entry has to be treated in a case independent manner. 309 The rmd160 is computed over the sender-id field and all lines with 310 newsgroup to multicast group relations in the packet with the 311 RIPEMD-160 message digest algorithm. 313 The lines with newsgroup to multicast group relations are repeated as 314 often as needed to announce all groups. The TTL can be used by 315 clients to find out if packets that come from this source can reach 316 them, or if the sender is too far away. Note that all entries have 317 to fit into one UDP packet. 319 The sender-id and the newsgroups entries are padded to the next 320 4-bytes boundary in order to make processing easier. 322 TTL values of articles have to be chosen, especially for use on the 323 MBONE, in a way that newsgroups that are only of local relevance 324 (e.g. campus groups or groups local to a town) are not distributed 325 out of their normal distribution area. As already mentioned above, 326 articles that are only of a local meaning or of local relevance, must 327 be distributed within the administratively scoped group range [12]. 329 The relation between multicast and newsgroups can range from one 330 multicast group per newsgroup over one multicast group per news 331 hierarchy (e.g. comp.*) to all articles in only one group. As current 332 implementations of kernels and routers get inefficient with too many 333 multicast groups, the use of one multicast group per newsgroup is 334 deprecated. 336 Reliability Considerations 338 As UDP is a unreliable service, provisions for reliable distribution 339 of articles are needed. There exist some approaches to reliable 340 multicast (XTP [4], KLG [5] RMTP [6] and others) which all suffer 341 from some problem or other. Specifically, additional hard- or 342 software is needed and usually requires kernel modification. 344 As there is already a reliable transport of NetNews via NNTP, there 345 is no need for a reliable transport via IP multicast: articles need 346 not be in order, so it is no problem if one is missing in the 347 multicast. Since articles need not arrive in order, lost or missing 348 articles can easily be transmitted via an additional NNTP feed. 350 As UDP packets can be at maximum 64kBytes in size and every Mcntp 351 packet has to fit in one UDP packet, there is no provision given to 352 distribute news articles larger than about 63kBytes in size (other 353 than compressing them). This does not matter much in practise as 354 recent research has shown that more than 95% off all news articles 355 are smaller 64kBytes [9]. The remaining 5% can still be transferred 356 via NNTP. Some hosts may have problems in receiving UDP packets as 357 large as 64kBytes, so in practical use article sizes of 16kBytes 358 would be appropriate. These are still over 90% of all articles. 360 Usage Scenarios 362 These scenarios show how mcntp can be used in daily use. The main 363 difference between local and MBone wide usage are the multicast 364 groups that are used for distribution as stated above. 366 For a local use within an organisation there could be one central 367 sending site that redistributes all news articles it receives via 368 mcntp. No further action is needed. 370 When more than one directory server (MD) gets involved, directory 371 servers must wait on startup for announcement packets from other MD 372 processes, register the contained groups in its tables and make 373 decisions involving that tables. Decisions can be divided into the 374 following: 376 Use 378 If the group in which the sender (MS) wants to send is already 379 distributed over multicast, then the articles are distributed in 380 the existing group else a new multicast group is used. For 381 example: if de.* is already distributed over multicast group 382 a.b.c.d then use that group. 384 New 386 Always create own multicast groups that don't clash with the ones 387 that are already existing. For example: if comp.* is already 388 distributed on group a.b.c.e and the sender (MS) wants to 389 distribute comp.foo, don't use group a.b.d.e, but create a new 390 one. 392 Standby 394 Only send articles for a specified newsgroup when no one other is 395 doing it. This can be used to implement backup functionality. For 396 example: Sender A is sending comp.*. If now a directory packet 397 arrives at site B, which no longer has comp.* in it, B can start 398 to send comp.*. When it sees again announcements from A, then it 399 stops the distribution of comp.*. 401 For use in an environment, where multiple organisations are involved 402 (e.g. on the MBone), the following could deployed: everyone that is 403 participating utilizes the ``use'' method described above. It only 404 sends articles that are locally produced (e.g. customers) and which 405 are not distributed via mcntp by another site. No articles received 406 from news peers should be distributed that way. After some delay (at 407 least 10 seconds), articles which are distributed via mcntp are 408 offered to peers over nntp as usual. The set of groups that is 409 distributed must be negotiated between the involved organisations. 410 With the current Usenet groups this could be: 412 - rec.* 413 - comp.*,news.*,gnu.* 414 - talk.*,misc.* 415 - humanities.*,sci.*,soc.* 416 - alt.* 418 Usage over Satellite connections 420 While in some regions of the world, terrestial bandwidth is cheap, 421 there are other regions where this is not the case. But those regions 422 can be reached by satellite beams. There are already some NetNews 423 over satellite mechanisms in place which often have their proprietary 424 protocol in place. With mcntp, transport of articles can go over the 425 same equipement, which is in place for IP communications. A possible 426 setup can be found in [13]. This setup has also the advantage that no 427 backchannel is needed, which allows the use of small and cheap anten- 428 nas on the receiver side. 430 Summary 432 The distribution of NetNews articles via IP multicast can be a way to 433 decrease the network bandwidth used to distribute them. Articles are 434 delivered fast via a nonreliable protocol; later, the holes are 435 filled via a reliable, already existing protocol. Compression of 436 articles can further reduce the network Load. With encryption private 437 news groups can be established on a public IP multicast 438 infrastructure. A prototype of a reference implementation [13] 439 already shows that Mcntp is fast and can be used as an alternative to 440 classical transports. The use of zlib for compressing articles shows 441 a reduction of transferred volume (including protocol headers) to 442 about 65% of the original article volumes. 443 In cooperation with Orion Network Systems [14], mcntp has proven its 444 use for distribution of NetNews over a unidirectional satellite 445 connection. 447 Security Considerations 449 With the classical NNTP based distribution, every host on the path of 450 an article keeps track of it in the logfiles, making it possible to 451 find the sender of forged or abusive articles with the aid of the 452 administrators of the newshosts along the path. For the distribution 453 of NetNews over IP multicast, this is no longer true: routers don't 454 log packets flowing by and as the sender address of IP packets can be 455 forged, a sender can't be traced. This fact can be used to inject 456 forged news articles without being traceable. 458 To prevent the unnoticed injection of articles, a mcntp receiver only 459 accepts articles from senders that it trusts. This trust is build by 460 digitally signing the article with the private key of the sender and 461 verifying the signature at the receiver site. Receivers have to 462 accept only articles with good signatures 464 The RIPEMD-160 message digest algorithm has been chosen, as it is 465 more secure than MD5 while still being fast enough. The RSA 466 encryption algorithm has been chosen as there exist reference 467 implementations for usage inside US (from RSA Inc.) and outside 468 (rsaeuro by J.S.A.Kapp). 470 The key size for the RSA algorithm must be at least 512 bit in size 471 to prevent cracking of the key. 473 References 475 [1] RFC 977 -- B. Kantor, P. Lapsley, "Network News Transfer Protocol: 476 A Proposed Standard for the Stream-Based Transmission of News". 478 [2] RFC 1112 -- S. Deering, "Host extensions for IP multicasting", 479 08/01/1989. 481 [3] RFC 768 -- J. Postel, "User Datagram Protocol", 08/28/1980. 483 [4] XTP -- W. T. Strayer, D.J. Dmepsey, B.C. Weaver, "XTP: The Xpress 484 Transfer Protocol", Addison-Wesley 486 [5] KLG -- M. Hofmann, "Zuverlaessige Kommunikation in heterogenen 487 Netzen", Thesis at "Institut fuer Telematik, CS Dept. Univ 488 Karlsruhe" 490 [6] RMTP -- Lin, John C., Paul Sanjoy, "RMTP: A Reliable Multicast 491 Transport Protocol". 493 [7] RFC 1036 -- M. Horton, R. Adams, "Standard for interchange of 494 USENET messages", 12/01/1987. 496 [8] RFC 1950 -- L. Deutsch, J. Gailly, "ZLIB Compressed Data Format 497 Specification version 3.3", 05/23/1996. 499 [9] http://www.pilhuhn.de/mcntp/histo/ -- Some Statistics about size 500 distribution of NetNews 502 [10] RFC 793 -- J. Postel, "Transmission Control Protocol", 09/01/1981. 504 [11] H. Dobbertin, A. Bosselaers, B. Preneel, "RIPEMD-160: A 505 Strengthened Version of RIPEMD" 04/18/1996. An earlier version 506 appeared in "Fast Software Encryption,LNCS 1039" Springer Verlag, 507 1996, pp. 71-82. 509 [12] [draft-ietf-mboned-admin-ip-space-04.txt/number of rfc] David 510 Meyer, "Administratively Scoped IP Multicast", [date of rfc] 512 [13] http://www.pilhuhn.de/mcntp/ 514 [14] http://www.onsi.com/ 516 Author's Address 518 Heiko W.Rupp 519 Gerwigstr. 5 520 D-76131 Karlsruhe 522 Phone: +49 721 9661524 523 EMail: hwr@pilhuhn.de