idnits 2.17.1 draft-sam-nntpupdates-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 18, 2020) is 1500 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'C' is mentioned on line 475, but not defined == Missing Reference: 'S' is mentioned on line 476, but not defined == Unused Reference: 'RFC4644' is defined on line 539, but no explicit reference was found in the text Summary: 0 errors (**), 0 flaws (~~), 4 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group E. Sam 3 Internet-Draft March 18, 2020 4 Intended status: Informational 5 Expires: September 19, 2020 7 Updates to the NNTP Protocol 8 draft-sam-nntpupdates-00 10 Abstract 12 This Internet Draft proposes and suggests updates to the Network News 13 Transfer protocol, or NNTP. The NNTP powers Usenet, one of the 14 largest social media platforms in the world. It is primarily used 15 for transferring text and binary posts. The aim of these updates is 16 to improve efficiency between NNTP peers and NNTP clients. 18 Status of This Memo 20 This Internet-Draft is submitted in full conformance with the 21 provisions of BCP 78 and BCP 79. 23 Internet-Drafts are working documents of the Internet Engineering 24 Task Force (IETF). Note that other groups may also distribute 25 working documents as Internet-Drafts. The list of current Internet- 26 Drafts is at https://datatracker.ietf.org/drafts/current/. 28 Internet-Drafts are draft documents valid for a maximum of six months 29 and may be updated, replaced, or obsoleted by other documents at any 30 time. It is inappropriate to use Internet-Drafts as reference 31 material or to cite them other than as "work in progress." 33 This Internet-Draft will expire on September 19, 2020. 35 Copyright Notice 37 Copyright (c) 2020 IETF Trust and the persons identified as the 38 document authors. All rights reserved. 40 This document is subject to BCP 78 and the IETF Trust's Legal 41 Provisions Relating to IETF Documents 42 (https://trustee.ietf.org/license-info) in effect on the date of 43 publication of this document. Please review these documents 44 carefully, as they describe your rights and restrictions with respect 45 to this document. Code Components extracted from this document must 46 include Simplified BSD License text as described in Section 4.e of 47 the Trust Legal Provisions and are provided without warranty as 48 described in the Simplified BSD License. 50 Table of Contents 52 1. Background . . . . . . . . . . . . . . . . . . . . . . . . . 2 53 2. Conventions and Definitions . . . . . . . . . . . . . . . . . 3 54 3. Updates to Article Posting and Retrieval . . . . . . . . . . 3 55 4. Command Updates/Additions . . . . . . . . . . . . . . . . . . 3 56 4.1. XPAT . . . . . . . . . . . . . . . . . . . . . . . . . . 3 57 4.1.1. Usage . . . . . . . . . . . . . . . . . . . . . . . . 3 58 4.1.2. Description . . . . . . . . . . . . . . . . . . . . . 4 59 4.1.3. Examples . . . . . . . . . . . . . . . . . . . . . . 4 60 4.2. WHOAMI . . . . . . . . . . . . . . . . . . . . . . . . . 5 61 4.2.1. Usage . . . . . . . . . . . . . . . . . . . . . . . . 5 62 4.2.2. Description . . . . . . . . . . . . . . . . . . . . . 5 63 4.2.3. Examples . . . . . . . . . . . . . . . . . . . . . . 6 64 5. Dynamic Feeds . . . . . . . . . . . . . . . . . . . . . . . . 6 65 5.1. LIST CRITERIA command . . . . . . . . . . . . . . . . . . 7 66 5.2. MAXARTSIZE attribute . . . . . . . . . . . . . . . . . . 7 67 5.3. GROUPWILDMAT attribute . . . . . . . . . . . . . . . . . 7 68 5.4. MAXGROUPS attribute . . . . . . . . . . . . . . . . . . . 8 69 5.5. DIST attribute . . . . . . . . . . . . . . . . . . . . . 8 70 5.6. Example . . . . . . . . . . . . . . . . . . . . . . . . . 8 71 6. Header Related Commands . . . . . . . . . . . . . . . . . . . 8 72 6.1. IHAVEHDR . . . . . . . . . . . . . . . . . . . . . . . . 9 73 6.1.1. Usage . . . . . . . . . . . . . . . . . . . . . . . . 9 74 6.1.2. Description . . . . . . . . . . . . . . . . . . . . . 9 75 6.1.3. Examples . . . . . . . . . . . . . . . . . . . . . . 10 76 6.2. Changes to Certain Commands with HEADER capability . . . 11 77 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 78 8. Security Considerations . . . . . . . . . . . . . . . . . . . 11 79 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 80 9.1. Normative References . . . . . . . . . . . . . . . . . . 11 81 9.2. Informative References . . . . . . . . . . . . . . . . . 12 82 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 12 83 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 12 85 1. Background 87 Usenet is one of the largest social media networks in the world, with 88 many endpoints and users posting both text and binary content into 89 the network. This draft proposes various changes and improvements to 90 keep up with the demand for Usenet. 92 Today, many posts are transmitted through the Usenet network and this 93 can cause stress on a server. Header-only feeds, which are 94 standardized in this document, help test servers capability to handle 95 a Usenet feed with billions of articles. 97 2. Conventions and Definitions 99 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 100 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 101 "OPTIONAL" in this document are to be interpreted as described in BCP 102 14 [RFC2119] [RFC8174] when, and only when, they appear in all 103 capitals, as shown here. 105 3. Updates to Article Posting and Retrieval 107 News servers store and index articles with three primary keys, a 108 Message ID (which is globally unique and non reusable), a article 109 number (a unique key for an article), and an arrival timestamp. More 110 information can be found in [RFC3977]. This draft aims to make 111 changes to some of the rules set in [RFC3977]. 113 Article numbers MUST lie between 1 and 9,223,372,036,854,775,807, 114 inclusive. The client and server MAY use leading zeroes in 115 specifying article numbers but MUST NOT use more than 19 digits. 117 4. Command Updates/Additions 119 This draft proposes additional commands and changes to existing 120 commands put forth by previous NNTP RFCs. 122 4.1. XPAT 124 4.1.1. Usage 126 Syntax 128 XPAT [extended-wildmat] header range| (pat 129 [pat...])|wildmat 131 Responses 133 221 Header follows 134 430 no such article 135 502 no permission 137 Parameters 139 pat wildmat-pattern 140 message-id Article message ID 141 extended-wildmat Use RFC 3977 extended wildmat 143 4.1.2. Description 145 The XPAT command is used to retrieve specific headers from specific 146 articles, based on pattern matching on the contents of the header. 147 This command was first available in INN. This command is backward 148 compatible with the XPAT specification in [RFC2980]. 150 In this command, a pat (pattern) is equivalent to a wildmat-pattern 151 in [RFC3977]. 153 The required header parameter is the name of a header line (e.g. 154 "subject") in a news group article. See [RFC5536] for a list of 155 valid header lines. The required range argument may be any of the 156 following: 158 o an article number 160 o an article number followed by a dash to indicate all following 162 o an article number followed by a dash followed another article 163 number 165 The required message-id argument indicates a specific article. The 166 range and message-id arguments are mutually exclusive. 168 If the optional extended-wildmat parameter was specified in the 169 command, then the last argument MUST be ONE [RFC3977] wildmat. 171 At least one single wildmat-pattern (as defined by [RFC3977]) must be 172 specified as well if the extended-wildmat parameter is not specified. 173 If there are additional wildmat-patterns they are joined together 174 separated by a single space to form a collection of wildmat- 175 pattern(s). Successful responses start with a 221 response followed 176 by a the headers from all messages in which the pattern/wildmat 177 matched the contents of the specified header line. This includes an 178 empty list. Once the output is complete, a period is sent on a line 179 by itself. If the optional argument is a message-id and no such 180 article exists, the 430 error response is returned. A 502 response 181 will be returned if the client only has permission to transfer 182 articles. 184 4.1.3. Examples 186 Examples of a client using the PAT command to get the Path header for 187 2 posts: 189 [C] GROUP alt.example 190 [S] 211 80 1 80 alt.example 192 [C] PAT path 79-80 *example* 193 [S] 221 Header information for path follows (from articles) 194 [S] 79 patton.com!example.com!not-for-mail 195 [S] 80 juju.org!kubellis.com!bowdoin.com!example.com!not-for-mail 196 [S] . 198 Examples of a client using the PAT command with extended-wildmat to 199 filter out all path headers with "patton.com": 201 [C] GROUP alt.example 202 [S] 211 80 1 80 alt.example 203 [C] PAT path 79-80 *,!*patton.com* 204 [S] 221 Header information for path follows (from articles) 205 [S] 80 juju.org!kubellis.com!bowdoin.com!example.com!not-for-mail 206 [S] . 208 4.2. WHOAMI 210 4.2.1. Usage 212 Syntax 214 WHOAMI 216 Responses 218 102 username server-name Logged in 219 103 server-name Not logged in 221 Parameters 223 username The username of the current user (specified in AUTHINFO 224 USER) 225 server-name The name of the NNTP server 227 4.2.2. Description 229 Upon receiving the WHOAMI command, the server will send the response 230 code 102 along with the username and mnemonic name of the server if 231 the user is authenticated. If the user is not logged in, then the 232 response code 103 must be returned with the server-name. 234 Severs MUST only send the 102 response code when successful 235 authentication has been achieved through the AUTHINFO command in 236 [RFC4643]. 238 This command MUST always return the response code 102 or 103, 239 regardless of the clients authentication status. 241 The server-name can be the hostname, FQDN, or mnemonic name of the 242 server. It can also be the name that the news server injects into 243 its path header. Multiple news servers can have the same server-name 244 (server operators might due this for load balancing), but news 245 servers MUST make sure servers with the same server-name have the 246 same article numbering for newsgroups. News clients SHOULD assume a 247 different server-name means the article numbering is different. News 248 clients can be assured that receiving the same server-name on the 249 invoking of the WHOAMI command means the article numbering is the 250 same. 252 This command SHOULD NOT be used to confirm if posting/reading is 253 allowed on the server. To see if those abilities are allowed, 254 clients should use the specific commands (POST, GROUP, ARTICLE) to 255 see if they can post and read articles. 257 4.2.3. Examples 259 Examples of a client using the WHOAMI command while unauthenticated: 261 [C] WHOAMI 262 [S] 103 example.com Not logged in 264 Examples of a client using the WHOAMI command while authenticated: 266 [C] AUTHINFO USER ama 267 [S] 381 PASS required 268 [C] AUTHINFO PASS dublin 269 [S] 281 Authentication accepted 270 [C] WHOAMI 271 [S] 102 ama example.com Logged in 273 5. Dynamic Feeds 275 A news server may like to filter incoming feeds to conserve space and 276 reduce spam on their server. The LIST CRITERIA capability will aim 277 to accomplish this by allowing clients to request from the server 278 what types of articles it is looking for. This is not intended to be 279 a complete replacement for out of band feed control, but should be 280 good enough for temporary feed adjustment. On the absence of certain 281 attributes in the LIST CRETERIA command or the absence of the LIST 282 CRETERIA capability altogether, news servers and clients SHOULD 283 revert to the out of band peering info they have stored or their 284 default behavior. 286 5.1. LIST CRITERIA command 288 When the LIST CRITERIA command is sent to a server, the server will 289 respond with one of the following attributes. Clients must send the 290 LIST CRITERIA command ONLY if the server advertises the LIST CRITERIA 291 capability. 293 MAXARTSIZE nnn 294 GROUPWILDMAT wildmat 295 MAXGROUPS nnn 296 DIST string[,string] 298 When it is done with all of its responses, the server should 299 terminate the sequence with a single dot (".") on a line. 301 5.2. MAXARTSIZE attribute 303 MAXARTSIZE nnn 305 Argument: byte amount of the maximum size of desired articles. 307 The MAXARTSIZE attribute specifies the maximum size of the articles 308 the server wishes to recieve (in bytes). Clients SHOULD NOT send 309 articles bigger than then the size specified in MAXARTSIZE. 311 5.3. GROUPWILDMAT attribute 313 GROUPWILDMAT wildmat 315 Argument:[RFC3977] wildmat 317 The GROUPWILDMAT attribute specifies the newsgroups the server wishes 318 to receive. 319 Using the extended wildmat format, servers can specify the articles 320 they want and don't want in one attribute. Clients SHOULD NOT send 321 articles if all of the newsgroups in the article are prohibited by 322 the wildmat. Clients SHOULD send articles if at least one of the 323 newsgroups in the article are allowed by the wildmat. 325 Some examples of some wildmats that might be used in GROUPWILDMAT: 327 * Send all newsgroups 328 *,!alt.usenet.kooks Send all newsgroups except "alt.usenet.kooks" 329 alt.example Only send articles that were posted to "alt.example" 330 *bin* Only send articles that were posted to newsgroups with the 331 phrase "bin" 333 5.4. MAXGROUPS attribute 335 MAXGROUPS nnn 337 Argument: A positive integer specifying the maximum number of 338 newsgroups to which an article may be posted for the site to be 339 willing to receive it. 341 The MAXGROUPS attribute allows for a sever to inform the client that 342 it should not send articles that have been cross posted to more 343 newsgroups than the argument in MAXGROUPS. 345 5.5. DIST attribute 347 DIST string[,string] 349 Argument: A group of strings containing article distributions the 350 server does not want to receive. 352 The DIST attribute allows for the server to inform the client that it 353 should not send any articles that have the following distributions 354 (determined by the Distribution: header). 356 5.6. Example 358 This is an example of a server who specifies and returns all 359 attributes listed above after the client sends the LIST CRETERIA 360 command. 362 [C] LIST CRETERIA 363 [S] MAXARTSIZE 256000 364 [S] GROUPWILDMAT *,!alt.example,!alt.test 365 [S] MAXGROUPS 7 366 [S] DIST local,internal 367 [S] . 369 6. Header Related Commands 371 The massive growth of Usenet has caused some peers to engage in 372 header-only feeds. Header-only feeds exist to help test system 373 reliability and test systems between two peers before a full binary 374 feed is established. 376 In addition, caching software and proxy servers may wish to have a 377 copy of headers and then request the actual article from a upstream 378 server when requested from a user. This may be accomplished from a 379 two servers: a faster one that only has headers, and a slower one 380 that actually has the articles request. 382 These commands aim to standardize commands that can be used in the 383 transmission of header only feeds. In order to use any of the 384 commands listed below, a server MUST advertise the HEADERS capability 385 and the client MUST be in MODE TRANSIT. More information about 386 adding capabilities and modes can be found in [RFC3977]. 388 6.1. IHAVEHDR 390 6.1.1. Usage 392 In order to use this command, the server MUST advertise the HEADER 393 capability. This command MUST NOT be pipelined. 395 Syntax 397 IHAVEHDR message-id 399 Responses 401 Initial Responses 403 331 Send article headers to be transferred 404 431 Headers not wanted 405 432 Transfer not possible; try again later 407 Subsequent Responses 409 232 Header transfer successful 410 432 Transfer failed; try again later 411 433 Transfer failed; do not retry 413 Parameters 415 message-id Article message ID 417 6.1.2. Description 419 The IHAVEHDR command informs the server that the client has the 420 headers of an article with the specified message-id. If the server 421 wants a copy of the article headers, it MUST return a 331 response 422 code. If the server does not want a copy from 423 the client (example - the server already has a copy of the article 424 headers or has the full article), the server MUST return a 431 425 response code. 426 If the server does not wish to receive the headers at the moment 427 (example - if it is in the process of receiving a full article/ 428 headers from another client), then the serer MUST send a 432 response 429 code. 431 If the server indicates it wants to receive the headers by a 331 432 response code, then the client will then send back a copy of the 433 headers. The end of header transmission is marked by a single dot 434 (".") on a line by itself. The actual content of the article MUST 435 NOT be transmitted using IHAVEHDR, instead the client can use IHAVE 436 to transmit both the headers and content of an article. 438 Once the transmission of the articles have been completed, a server 439 MUST send a 232 response code if the header transfer is successful. 440 Unless a 232 response code is sent, clients MUST NOT assume that the 441 transfer of the headers was successful. A lack of response SHOULD BE 442 treated the same as a 432 response. Once received, a server MUST add 443 the article to the newsgroups and assign the article a article 444 number, like it would with a complete article. However, certain 445 commands must return a different response code if only headers are 446 available. 448 6.1.3. Examples 450 Example of a successful header transfer to another site: 452 [C] IHAVEHDR 453 [S] 331 Send headers; end with . 454 [C] From: "A Usenet User" 455 [C] Newsgroups: alt.test 456 [C] Path: example.com!not-for-mail 457 [C] Subject: Test Article 458 [C] Date: 20 Jan 2020 09:30:40 -0500 459 [C] Organization: Example Corp 460 [C] Message-ID: 461 [C] . 462 [S] 232 Header transfer successful 464 Example of a unsuccessful header transfer to another site: 466 [C] IHAVEHDR 467 [S] 331 Send headers; end with . 468 [C] From: "A Usenet User" 469 [C] Newsgroups: alt.test 470 [C] Path: example.com!not-for-mail 471 [C] Subject: Test Article 472 [C] Date: 20 Jan 2020 09:30:40 -0500 473 [C] Organization: Example Corp 474 [C] Message-ID: 475 [C] . 476 [S] 433 Header transfer unsuccessful; don't try again 478 6.2. Changes to Certain Commands with HEADER capability 480 Advertising the HEADER capability means that certain commands will 481 return an extra response code when dealing with a header-only message 482 ID. There are also certain commands that must return a successful 483 response code only if a full article is available: 485 o STAT SHOULD respond with code 226 (Header only article) along with 486 the article number and message ID in that order. 488 o ARTICLE SHOULD respond with code 227 for header only articles. It 489 should then follow the conventional format (article number, 490 message-id, content of article). To maintain compatibility with 491 newsreaders, a empty line in between the headers and the dot 492 terminator (".") MUST be present. A news server can also opt, but 493 SHOULD NOT, to place a message in the body explaining it only has 494 the headers for that article. If the server manages to get the 495 full article, then it can respond normally with a 220 code and the 496 article information. 498 o When a server receives a IHAVE, CHECK or TAKETHIS command, it 499 SHOULD act like the article does not exist if the server only has 500 the headers. This will allow the server to get the full article. 502 o IHAVE MUST only be used to send full articles. For sending 503 headers only, use IHAVEHDR. 505 o The header only article MUST be assigned an article number and the 506 headers MUST be available in header commands like PAT, OVER, HDR, 507 etc. 509 7. IANA Considerations 511 This document has no IANA actions. 513 8. Security Considerations 515 This document has no security considerations. 517 9. References 519 9.1. Normative References 521 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 522 Requirement Levels", BCP 14, RFC 2119, 523 DOI 10.17487/RFC2119, March 1997, 524 . 526 [RFC2980] Barber, S., "Common NNTP Extensions", RFC 2980, 527 DOI 10.17487/RFC2980, October 2000, 528 . 530 [RFC3977] Feather, C., "Network News Transfer Protocol (NNTP)", 531 RFC 3977, DOI 10.17487/RFC3977, October 2006, 532 . 534 [RFC4643] Vinocur, J. and K. Murchison, "Network News Transfer 535 Protocol (NNTP) Extension for Authentication", RFC 4643, 536 DOI 10.17487/RFC4643, October 2006, 537 . 539 [RFC4644] Vinocur, J. and K. Murchison, "Network News Transfer 540 Protocol (NNTP) Extension for Streaming Feeds", RFC 4644, 541 DOI 10.17487/RFC4644, October 2006, 542 . 544 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 545 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 546 May 2017, . 548 9.2. Informative References 550 [RFC5536] Murchison, K., Ed., Lindsey, C., and D. Kohn, "Netnews 551 Article Format", RFC 5536, DOI 10.17487/RFC5536, November 552 2009, . 554 Acknowledgments 556 Thanks to everyone who helped create the tools that let us use 557 Markdown to create Internet Drafts. 559 Thanks to everyone who contributed to ideas for this draft. 561 Thanks to everyone in the DISPATCH WG. 563 Author's Address 565 Ekow Sam 567 Email: winshell64@gmail.com