idnits 2.17.1 draft-irtf-asrg-bcp-blacklists-09.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 15 instances of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to use 'NOT RECOMMENDED' as an RFC 2119 keyword, but does not include the phrase in its RFC 2119 key words list. == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: Therefore, negative-connotation DNSBLs MUST not charge fees or require donations for delisting or "faster handling", and it is RECOMMENDED that such DNSBLs that do charge fees or require donations not be used. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (May 5, 2011) is 4739 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Obsolete informational reference (is this intentional?): RFC 3330 (Obsoleted by RFC 5735) -- Obsolete informational reference (is this intentional?): RFC 5156 (Obsoleted by RFC 6890) Summary: 0 errors (**), 0 flaws (~~), 4 warnings (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Anti-Spam Research Group - IRTF C. Lewis 3 Internet-Draft Nortel Networks 4 Intended status: Informational M. Sergeant 5 Expires: November 6, 2011 Symantec Corporation 6 May 5, 2011 8 Overview of Email DNSBL Best Practise 9 draft-irtf-asrg-bcp-blacklists-09 11 Abstract 13 The rise of spam and other anti-social behavior on the Internet has 14 led to the creation of shared DNS-based lists ("DNSBLs") of IP 15 addresses or domain names intended to help guide email filtering. 16 This memo summarizes guidelines of accepted best practise for the 17 management of public DNSBLs by their operators as well as for the 18 proper use of such lists by mail server administrators (DNSBL users), 19 and it provides useful background for both parties. It is not 20 intended to advise on the utility or efficacy of particular DNSBLs or 21 the DNSBL concept in general, nor to assist end users with questions 22 about spam. 24 This document is a product of the Anti-Spam Research Group and 25 represents the consensus of that group. 27 Status of this Memo 29 This Internet-Draft is submitted in full conformance with the 30 provisions of BCP 78 and BCP 79. 32 Internet-Drafts are working documents of the Internet Engineering 33 Task Force (IETF). Note that other groups may also distribute 34 working documents as Internet-Drafts. The list of current Internet- 35 Drafts is at http://datatracker.ietf.org/drafts/current/. 37 Internet-Drafts are draft documents valid for a maximum of six months 38 and may be updated, replaced, or obsoleted by other documents at any 39 time. It is inappropriate to use Internet-Drafts as reference 40 material or to cite them other than as "work in progress." 42 This Internet-Draft will expire on November 6, 2011. 44 Copyright Notice 46 Copyright (c) 2011 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents 51 (http://trustee.ietf.org/license-info) in effect on the date of 52 publication of this document. Please review these documents 53 carefully, as they describe your rights and restrictions with respect 54 to this document. Code Components extracted from this document must 55 include Simplified BSD License text as described in Section 4.e of 56 the Trust Legal Provisions and are provided without warranty as 57 described in the Simplified BSD License. 59 Table of Contents 61 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 62 1.1. DNS-Based Reputation Systems . . . . . . . . . . . . . . . 4 63 1.2. Guidance for DNSBL Users . . . . . . . . . . . . . . . . . 6 64 1.3. Requirements Language . . . . . . . . . . . . . . . . . . 8 65 1.4. Background . . . . . . . . . . . . . . . . . . . . . . . . 8 66 2. DNSBL Policies . . . . . . . . . . . . . . . . . . . . . . . . 8 67 2.1. Transparency . . . . . . . . . . . . . . . . . . . . . . . 8 68 2.1.1. Listing/Delisting Criteria SHOULD Be Easily 69 Available . . . . . . . . . . . . . . . . . . . . . . 9 70 2.1.2. Audit Trail SHOULD be maintained . . . . . . . . . . . 9 71 2.1.3. The Scope and Aggressiveness of Listings MUST be 72 Disclosed. . . . . . . . . . . . . . . . . . . . . . . 10 73 2.2. Listings and Removals . . . . . . . . . . . . . . . . . . 10 74 2.2.1. Listings SHOULD Be Temporary . . . . . . . . . . . . . 10 75 2.2.2. A Direct Non-Public Way to Request Removal SHOULD 76 Be Available . . . . . . . . . . . . . . . . . . . . . 12 77 2.2.3. Response SHOULD Be Prompt . . . . . . . . . . . . . . 12 78 2.2.4. SHOULD Have Similar Criteria for Listing and 79 Delisting . . . . . . . . . . . . . . . . . . . . . . 13 80 2.2.5. Conflict of Interest . . . . . . . . . . . . . . . . . 13 81 3. Operational Issues . . . . . . . . . . . . . . . . . . . . . . 14 82 3.1. DNSBL Query Root Domain Name SHOULD be a Subdomain . . . . 14 83 3.2. DNSBLs SHOULD be Adequately Provisioned . . . . . . . . . 14 84 3.3. DNSBLs SHOULD Provide Operational Flags . . . . . . . . . 15 85 3.4. Shutdowns MUST Be Done Gracefully . . . . . . . . . . . . 16 86 3.5. Listing of Special and Reserved IP Addresses MUST be 87 disclosed . . . . . . . . . . . . . . . . . . . . . . . . 17 88 3.6. Considerations for DNSBLs Listing Insecure Hosts . . . . . 17 89 3.6.1. MUST NOT scan without provocation . . . . . . . . . . 18 90 3.6.2. Re-scan Periods SHOULD be Reasonable . . . . . . . . . 18 91 3.6.3. Scans MUST NOT be Destructive . . . . . . . . . . . . 18 92 3.7. Removals SHOULD Be Possible in Absence of the DNSBL 93 Operator . . . . . . . . . . . . . . . . . . . . . . . . . 18 94 3.8. Protect Against Misconfiguration/Outages . . . . . . . . . 18 95 3.9. Error Handling . . . . . . . . . . . . . . . . . . . . . . 19 97 4. Security Considerations . . . . . . . . . . . . . . . . . . . 20 98 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 99 6. References . . . . . . . . . . . . . . . . . . . . . . . . . . 21 100 6.1. Normative References . . . . . . . . . . . . . . . . . . . 21 101 6.2. Informative References . . . . . . . . . . . . . . . . . . 21 102 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 21 103 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 22 105 1. Introduction 107 1.1. DNS-Based Reputation Systems 109 Due to the rising amount of spam and other forms of network abuse on 110 the Internet, many community members and companies began to create, 111 publish and maintain DNS-based reputation systems (DNS-Based Lists) 112 of IP addresses or domain names and make reputation suggestions or 113 assertions about email sourced from these IP addresses or domain 114 names. 116 The first DNS-based Lists were almost exclusively intended to be used 117 (by email administrators) as lists of abusive IP addresses to block, 118 however the DNS publication method has proven to be so robust, 119 popular and simple to use, that it has been extended for use in many 120 different ways, far beyond the imaginings of the designers of DNS or 121 DNS-based blocking IP lists. For example, today, the same basic DNS- 122 based listing technology is commonly used for: 124 DNSWL listings of well-behaving email source IP/domain addresses 125 (whitelist). 127 RHSBL listings of well/ill behaving email source domain names (often 128 applied against the domain name part of the originating email 129 address or DNS PTR (reverse IP) lookups) 131 URIBL listings of well/ill behaving web link domain names or host 132 names used in email 134 Further, the DNSBL user using the list doesn't have to use a listing 135 as a pass/fail binary decision, it can use a listing as one factor in 136 email filters that make decisions based on scoring multiple factors 137 together. 139 The DNS-based list technology has even been extended to purely 140 informational purposes. For example, there are implementations that 141 return results based on what geographic region an IP/domain is 142 putatively allocated in, implementations that translate an IP/domain 143 address into an ASN number and/or allocation block, implementations 144 that indicate whether the queried domain name is registered through a 145 given Domain registrar, implementations that return aggregate numeric 146 reputation for an IP address or domain name from another system's 147 email system, and so on. The possibilities are virtually endless. 149 As well, DNS-based listing technology has also been used in areas 150 other than email filtering, such as IRC, web access control, and 151 transaction verification. 153 As the terminology in this area has never been well formalized, often 154 overlaps, and lacks precision, this document has been written to use 155 the term "DNSBL" to refer to DNS-based lists generally, not just DNS- 156 based block (or black) lists. This document is not applicable to 157 some DNSBLs in some areas (mentioned as appropriate) but it is the 158 authors' belief that most of the practises are applicable to almost 159 all DNSBLs. 161 DNSBLs may be either public or private. A public DNSBL makes its 162 data available to any party seeking information about data on the 163 list, while a private DNSBL is used solely by an organization for its 164 own use and the data is not made available publicly. There are also 165 commercial DNSBLs, available for a fee. Furthermore, some are free 166 yet require a fee for higher numbers of queries or certain classes of 167 DNSBL users. 169 The first publicly available DNSBL using the Domain Name System (DNS) 170 for distributing reputation data about email senders emerged in 1997, 171 shortly after spam became a problem for network operators and email 172 administrators. This pioneer DNSBL focused on identifying known spam 173 sources situated at static (unchanging) IP/domain addresses. Due to 174 the broad adoption of this DNSBL, it had a major impact on static 175 spam sources. Consequently, abusers found other methods for 176 distributing their spam, such as relaying messages through unsecured 177 email servers or flawed formmail scripts on web pages. Additional 178 DNSBLs were developed by others in order to address these changing 179 tactics, and today more than 700 public DNSBLs are known to be in 180 operation. 182 These DNSBLs vary widely in purpose for which the list was intended, 183 the method the list uses to achieve the purpose, the integrity of 184 those overseeing the method, and the stability of the technology used 185 to create and distribute the data. Listing criteria can sometimes be 186 quite controversial, therefore this document deliberately does not 187 discuss the rightness or wrongness of any criteria. We assert that 188 DNSBL operators are free to choose whatever listing criteria they 189 wish, as long as those criteria are clearly and accurately 190 communicated. It is the responsibility of the DNSBL user to ensure 191 that the listing criteria and other aspects of a DNSBL meets their 192 needs. 194 This document is intended to provide guidance to DNSBL operators so 195 that they may be able to identify what features users would be 196 interested in seeing as part of a high-quality, well-managed DNSBL, 197 for example, a clear listing and delisting policy to which the DNSBL 198 operator adheres strictly. This document is intended to be normative 199 rather than prescriptive: it seeks to characterize the features of a 200 well-managed DNSBL rather than setting out rules for how DNSBLs 201 should be operated. 203 This document is not intended as a protocol specification of DNSBL 204 queries. (See [RFC5782].) 206 The DNS has been the most popular distribution method for DNSBLs due 207 to its ubiquity and its good scaling and performance characteristics. 208 It is also common to make private arrangements to distribute DNSBL 209 data in bulk to high volume users, typically by rsync [RSYNC], 210 [RSYNCTHESIS]. The data is the same in either case, the 211 recommendations in this document apply regardless of distribution 212 method, other than the ones in Section 3.1 and Section 3.2 that 213 specifically refer to DNS distribution. 215 1.2. Guidance for DNSBL Users 217 When choosing to adopt a DNSBL, a DNSBL user SHOULD keep the 218 following questions in mind: 220 1. What is the intended use of the list? 222 2. Does the list have a web site? 224 3. Are the list's policies stated on the web site? 226 4. Are the policies stated clearly and understandably? 228 5. Does the web site function properly, e.g., hyperlinks? 230 6. Are web pages for removal requirements accessible and working 231 properly? 233 7. How long has the list been in operation? 235 8. What are the demographics and quantity of the list's user base? 236 In other words, do other sites like my own use this DNSBL? 238 9. Are comparative evaluations of the list available? Note: all 239 such evaluations depend on the mail mix used as well as local 240 policy. DNSBL users SHOULD consider trial periods and/or 241 ongoing local monitoring of DNSBL suitability. 243 10. What do your peers or members of the Internet community say 244 about the list? DNSBLs can sometimes be quite controversial and 245 sometimes considerable misinformation is spread. Ensure that 246 the opinions are knowledgeable, and reflect similar goals to 247 yours. 249 11. Does the DNSBL have a mailing list for announcing changes, 250 outages etc? 252 DNSBLs can, and have, ceased operation without notice. DNSBL users 253 SHOULD periodically check the correct operation of the DNSBL, and 254 cease using DNSBLs that are working incorrectly. See Section 3.3 256 The DNSBL user MUST ensure that they understand the intended use of 257 the DNSBL. For example, some IP address-based DNSBLs are appropriate 258 only for assessment of the peer IP address of the machine connecting 259 to the DNSBL user's mail server, and not other IP addresses appearing 260 in an email (such as header Received lines or web links), or IRC 261 connections etc. While a DNSBL user may choose to ignore the intent 262 of the DNSBL, they SHOULD implement any variance in compliance with 263 the DNSBL usage instructions. 265 For example, one of the requirements of some DNSBLs is that if the 266 DNSBL is used contrary to the usage instructions, then the DNSBL user 267 should not identify the DNSBL being used. Furthermore, it is the 268 DNSBL user's responsibility to mitigate the effect of the listing 269 locally. 271 It is the responsibility of the system administrators who adopt one 272 or more DNSBLs to evaluate, understand, and make a determination of 273 which DNSBLs are appropriate for the sites they administer. If you 274 are going to allow a third party's information to guide your 275 filtering decision-making process, you MUST understand the policies 276 and practises of those third parties because responsibility for 277 filter decisions remains ultimately with you, the postmaster. 279 A DNSBL without DNSBL users does not block (or otherwise impair) 280 email or any other Internet service. A DNSBL user voluntarily uses 281 the DNSBL data to guide their decisions, and the DNSBL user therefore 282 MUST assume responsibility for dealing with the consequences. 284 DNSBL operators are expressing an opinion through the publication of 285 a DNSBL. However, it is through abiding by the guidelines set forth 286 in this BCP that the operators of a DNSBL may gain the trust of their 287 users. 289 These guidelines address only public DNSBLs and do not apply to 290 private access DNSBLs, however, implementers and users of private 291 access DNSBLs may wish to use these guidelines as a starting point of 292 things to consider. 294 1.3. Requirements Language 296 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 297 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 298 document are to be interpreted as described in [RFC2119]. 300 1.4. Background 302 The Anti Spam Research Group (ASRG) was chartered to address the spam 303 problem. The ASRG charter includes: 305 "codification of best current practises in spam management" 307 This note falls within that category by listing guidelines for 308 management of public DNSBLs. 310 NOTE: This document is a product of the Anti-Spam Research Group 311 (ASRG) of the IRTF. As per section 3 of [RFC2014] IRTF groups do 312 not require consensus to publish documents. Therefore readers 313 should be aware that this document does not necessarily represent 314 the consensus of the entire ASRG. 316 NOTE: This document is intended to evolve, based on comments from 317 the Anti-Spam Research Group (ASRG) mailing list. It is certain 318 that the current draft is incomplete and entirely possible that it 319 is inaccurate. Hence, comments are eagerly sought, preferably in 320 the form of suggested text changes, and preferably on the ASRG 321 mailing list, at . 323 2. DNSBL Policies 325 2.1. Transparency 327 A DNSBL SHOULD carefully describe the criteria that are the cause for 328 adding, and the criteria for removing an entry from the list. Such 329 listing and delisting criteria SHOULD be presented in a clear and 330 readable manner easily accessible to the public on the DNSBL's web 331 site. A DNSBL MUST abide by its stated listing and delisting 332 criteria. Entries that do not meet the published criteria MUST NOT 333 be added to the DNSBL. 335 In other words, be direct and honest and clear about the listing 336 criteria, and make certain that only entries meeting the published 337 criteria are added to the list. For example, some DNSBL operators 338 have been known to include "spite listings" in the lists they 339 administer -- listings of IP addresses or domain names associated 340 with someone who has insulted them, rather than actually violating 341 technical criteria for inclusion in the list. There is nothing 342 inherently wrong with this practise so long as it is clearly 343 disclosed - and thus becomes part of the published criteria. For 344 example, a DNSBL described as only listing open relays MUST NOT 345 include IP addresses for any other reason. This transparency 346 principle does not require DNSBL operators to disclose the precise 347 algorithms and data involved in a listing, but rather the intent 348 behind choosing those algorithms and data. 350 Furthermore, the DNSBL documentation SHOULD be clear on the intended 351 use of the DNSBL - whether it be intended for peer addresses of 352 email, IRC, etc. 354 Availability of documentation concerning a DNSBL SHOULD NOT be 355 dependent on the continued operation of DNS for DNSBL queries. 357 In other words, if the DNSBL documentation is at 358 "http://dnsbl.example.com", the documentation for the web site should 359 not become unavailable if the DNSBL query name servers are not 360 available (or shut down). See Section 3.1. 362 2.1.1. Listing/Delisting Criteria SHOULD Be Easily Available 364 Listing and delisting criteria for DNSBLs SHOULD be easily available 365 and SHOULD be located in a place clearly marked in its own section of 366 the web site affiliated with the DNSBL. 368 DNSBLs often publish their listing criteria along with additional 369 technical information about using the DNSBL. This additional 370 technical information can confuse end users, so a separate page, 371 section or query function on its own SHOULD be dedicated to detailing 372 why a specific entry appears in the DNSBL. 374 2.1.2. Audit Trail SHOULD be maintained 376 A DNSBL SHOULD maintain an audit trail for all listings and it is 377 RECOMMENDED that it is made publicly available in an easy to find 378 location, preferably on the DNSBL's web site. Please note that 379 making an audit trail data public does not entail revealing all 380 information in the DNSBL operator's possession relating to the 381 listing; e.g., a DNSBL operator MAY make the audit trail data 382 selectively accessible in such a way as to not disclose information 383 that might assist spammers, such as the location or identity of a 384 spam trap. 386 2.1.3. The Scope and Aggressiveness of Listings MUST be Disclosed. 388 Some DNSBLs have adopted policies of listing entries that are broader 389 in scope than they have evidence of being involved in abuse. 390 Similarly, some DNSBLs list entries that are "mixed", in that the 391 entry may be behaving in a manner that is both abusive and non- 392 abusive. This is inherent to the techniques that many DNSBLs use. 394 Examples: Some DNSBLs will list IP address ranges if there is reason 395 to believe that abusive behavior seen from a few IP addresses within 396 the range is (or will be) reflected in the rest of the range. Some 397 DNSBLs utilize scoring to list IP addresses, IP ranges or domain 398 names that have abusive behavior above some threshold - often meaning 399 that some of the email corresponding to the listing is not abusive. 400 Even an entry demonstrably infected with email spam or virus emitting 401 malware may emit non-abusive email. 403 Inevitably, some of these listings may impact non-abusive email. 404 This has resulted in some labeling such practises by the emotionally 405 loaded term "collateral damage". No filtering technique is perfect, 406 and an occasional mistake is inevitable no matter what is used, 407 DNSBLs or otherwise. 409 There is nothing wrong with this practise, because mail server 410 administrators may wish to implement such policies or use them in 411 combination with other techniques (such as scoring). However, a 412 diligent administrator needs information about these policies in 413 order to make an informed decision as to the risk and benefit of 414 using any particularly DNSBL, and in many cases guide them in how to 415 use it for results best reflecting the DNSBL user's requirements. 417 Therefore, DNSBL listing policies MUST include statements as to the 418 scope and aggressiveness of listings, and include, as appropriate, 419 whether the DNSBL operator intends the listings to be used in scoring 420 or other techniques. 422 2.2. Listings and Removals 424 2.2.1. Listings SHOULD Be Temporary 426 In many cases, listings can exist for long periods of time past the 427 conditions leading to the listing's creation, and/or the listed 428 entity has putatively changed ownership. 430 Generally speaking, listings SHOULD be considered temporary, and 431 should expire on their own at some point in the future unless reasons 432 for listing still exist. 434 Expiration intervals SHOULD be chosen to be reasonable for the type 435 of listing. For example: 437 1. It does not make sense to remove entries from DNSBLs where the 438 existence of an entry does not have a direct meaning, that is, 439 DNSBLs that return information in addition to just existence/ 440 non-existence. For example: entries in DNSBLs that return 441 geographic or assignment information on where the IP address or 442 domain name is located or owned, or DNSBLs that return flow 443 statistics from the DNSBL operator that are intended for the 444 DNSBL user to interpret, need not ever be removed, just kept 445 reasonably current. 447 2. DNSBLs based on relatively static information, such as block 448 assignment or domain names of demonstrably bad actors MAY have 449 very long expiration intervals or be removed only upon request 450 after verification that the removal criteria have been met. 452 3. Automated DNSBLs with highly effective detection and fast listing 453 mechanisms can benefit from very short expiration intervals. 454 Many of the things that these DNSBLs look for are of relatively 455 short duration, and even if they do expire, a resumption of the 456 behavior will be caught quickly by the DNSBL's detection 457 mechanisms and relisted. By utilizing a short expiration 458 interval, after reassignment/problem correction, the listing will 459 automatically expire in short order without manual intervention. 461 4. Manually created DNSBL entries SHOULD be periodically reviewed in 462 some manner. 464 It is RECOMMENDED that DNSBL operators publish in general terms their 465 expiration policy, even if it's only "delist on request" or no 466 expiration is performed. In information-only lists, a method for 467 users requesting corrections to the information (if appropriate) 468 SHOULD be published. Abusers may be able to "game" policy that is 469 too explicit; on the other hand, many DNSBL users wish to have an 470 idea of how "current" the DNSBL is. It is the authors' experience 471 that some automated DNSBLs have increasingly higher error rates as 472 the "last detection date" gets older. 474 Note that listings being temporary does not mean that some listings 475 will not remain after the initial time out period. If the DNSBL 476 operator determines that the conditions triggering listing still 477 exist, then the timer for determining time outs can be renewed. 479 2.2.2. A Direct Non-Public Way to Request Removal SHOULD Be Available 481 Discussions about whether a DNSBL should remove an entry MAY include 482 activity in a public forum. Methods for processing removal requests 483 through private, direct exchanges, such as person-to-person email or 484 a combination of web page requests and email responses, SHOULD be 485 available. As a minimum, the DNSBL SHOULD have a web page that has a 486 removal request function (separate from the page describing listing 487 criteria as per Section 2.1.1). The DNSBL SHOULD also make available 488 an email address to handle issues other than blocking issues. 490 The DNSBL operator MUST NOT use the list in question in such a way 491 that removal requests would be blocked, and, moreover, SHOULD make 492 mailboxes available in order to allow affected users to submit their 493 requests. In some cases it is impractical not to filter email to 494 accounts due to the amount of spam those mailboxes receive. If 495 filtering should be necessary in such circumstances, filtering 496 methods with as low false positive rate as practical SHOULD be 497 chosen. 499 DNSBL operators SHOULD be prepared to provide alternate means of 500 contact in case of system failure due to DDoS or other reasons. 502 2.2.3. Response SHOULD Be Prompt 504 A response to removal requests or queries about a listing SHOULD be 505 prompt. A DNSBL operator SHOULD respond within two days and MUST 506 respond within 7 days, unless the DNSBL operator has deemed that 507 further discussion of the issue will not result in meeting the 508 conditions for removal, and notifies the requestor of that decision. 510 Consequent removals (if the conditions for removal are met) should be 511 similarly prompt. 513 A DNSBL MAY impose restrictions on who (e.g. network operator's 514 representative or domain name owner) may make valid removal requests. 515 However, in many DNSBLs this is inadvisable because it requires 516 impractical amounts of effort and is hence NOT RECOMMENDED in most 517 cases. 519 Many DNSBLs (especially those with highly effective detection and 520 fast listing mechanisms) greatly benefit from a "no questions asked" 521 removal policy. 523 Although this approach allows people to submit a request and have any 524 listed IP address/domain name removed immediately, it does not 525 prevent the DNSBL operator from re-listing the IP address/domain name 526 at a later time. 528 Many DNSBLs can effectively use a "no questions asked" removal policy 529 because by their very nature they will redetect or relist problems 530 almost immediately. They can mitigate more organized attempts to 531 "game" the system by elementary checking and rate-limiting 532 procedures, increasing lockout periods, rescans etc. Furthermore, a 533 few IP addresses more or less usually do not make a significant 534 difference in the overall effectiveness of a DNSBL. Moreover, a "no 535 questions asked" removal policy provides the huge benefit of a swift 536 reaction to incorrect listings. 538 As an example, one popular DNSBL uses a "no questions asked" removal 539 policy, but does perform rate-limiting and malicious removal 540 detection and mitigation. 542 Another important consideration supporting a "no questions asked" 543 self-removal policy is that it forestalls many conflicts between 544 DNSBL operators and organizations whose IP addresses/domain names 545 have been listed. Such a policy may be an effective measure to 546 prevent small issues from becoming big problems. 548 2.2.4. SHOULD Have Similar Criteria for Listing and Delisting 550 The criteria for being removed from a DNSBL SHOULD bear a reasonable 551 relationship to the factors that were the cause of the addition to 552 the DNSBL. If a listed entity fulfills all published requirements 553 for removal from a DNSBL, then the DNSBL operator SHOULD NOT impose 554 any additional obstacles to remove a given entry from the DNSBL. 555 There SHOULD NOT be any extra rules for de-listing other than the 556 ones listed in the published listing criteria. 558 2.2.5. Conflict of Interest 560 Some DNSBLs used for blocking/negative reputation have had a practise 561 of requiring fees or donations to charities from the listee for 562 delisting. 564 It is generally considered entirely appropriate for a DNSBL to charge 565 for access to it by its users - the definition of a commercial DNSBL. 567 However, the practise of requiring a listee to pay for delisting from 568 a negative connotation DNSBL steers perilously close to notions of 569 extortion, blackmail or a "protection racket". Even when such 570 accusations are entirely unjustified the practise causes uproar and 571 damage to the DNSBL's reputation, if not the entire DNSBL mechanism 572 as a whole. Colloquially, "it smells bad". 574 Therefore, negative-connotation DNSBLs MUST not charge fees or 575 require donations for delisting or "faster handling", and it is 576 RECOMMENDED that such DNSBLs that do charge fees or require donations 577 not be used. 579 3. Operational Issues 581 3.1. DNSBL Query Root Domain Name SHOULD be a Subdomain 583 By virtue of using domain names, a DNSBL is a hierarchy with a root 584 anchored in the global Internet. The DNSBL "query root" SHOULD be 585 below the registered domain name, so that the DNSBL information is 586 not conflated with domain name housekeeping information (e.g., name 587 server or MX records) for the domain name. By using this approach, 588 DNSBL queries would take the form of ".dnsbl.example.com" 589 rather than ".example.com". Further, this sub-tree should 590 have its own name servers. Thus, the DNSBL query root has its own 591 zone file containing the DNSBL information, and the registered domain 592 name has its own name servers containing the information (MX records 593 etc.) for the domain name. This approach facilitates clear 594 delineation of function as well as orderly DNSBL shutdown because the 595 DNSBL name server records can be specified separately from the domain 596 name's principal name servers. 598 Many DNSBLs support more than one logical zone (DNSBL entries with 599 different meanings) that DNSBL users may wish to treat differently 600 (or even ignore). It is RECOMMENDED that, even if there is a single 601 DNSBL zone with entry type distinguished by return code, that 602 separate subdomain names (of the query root) consist only of the 603 corresponding entries. For example, entry types "A" and "B" might 604 return 127.0.0.2 and 127.0.0.3 from the consolidated zone (eg: 605 dnsbl.example.com), but there should also be zones 606 typeA.dnsbl.example.com and typeB.dnsbl.example.com that contain 607 their respective types only. See also Section 3.3. 609 3.2. DNSBLs SHOULD be Adequately Provisioned 611 The DNSBL SHOULD have sufficient name server capacity to handle the 612 expected loading, and have sufficient redundancy to handle normal 613 outages. 615 Name servers SHOULD provide appropriate glue records, possibly in 616 different TLDs to protect against single-TLD issues. 618 If the DNSBL offers zone transfers (in addition to or instead of 619 standard DNSBL query mechanisms), it SHOULD be sufficiently 620 provisioned to handle the expected loading. 622 Note that some DNSBLs have been subject to distributed denial of 623 service attacks. Provisioning SHOULD take the likelihood of this 624 into account, and include plans for dealing with it. 626 3.3. DNSBLs SHOULD Provide Operational Flags 628 Most IP address-based DNSBLs follow a convention of query entries for 629 IP addresses in 127.0.0.0/8 (127.0.0.0-127.255.255.255) to provide on 630 line indication of whether the DNSBL is operational. Many, if not 631 most, DNSBLs arrange to have a query of 127.0.0.2 return an A record 632 indicating that the IP address is listed. This appears to be a de 633 facto standard. (See [RFC5782].) 635 If this indicator is missing (query of 127.0.0.2 returns NXDOMAIN), 636 the DNSBL should be considered non-functional. 638 There does not appear to be a de facto standard for test entries 639 within domain name-based DNSBLs. A number use the same 127.0.0.2 640 query test mechanism as IP address-based DNSBLs, and others use a 641 variety of domain name-based test entries. Due to the way many 642 domain name-based DNSBLs are used (eg: hostname parts of URIs in 643 email bodies), using anything likely to appear in a legitimate email 644 message is a bad idea (eg: http://example.com), especially 645 considering that some email readers will transform bare IP addresses 646 or domain names appearing in the body of an email into links. So, 647 even 127.0.0.2 may be problematic. But a common testing method is 648 desirable. 650 In the absence of new emerging standards, it is RECOMMENDED that 651 domain name-based DNSBLs use a test entry of "test". Test is chosen 652 because it is a reserved top-level-domain. 654 Note: In Section 3.4 it is noted that some DNSBLs have shut down in 655 such a way to list all of the Internet. Further, in Section 3.5, 656 DNSBL operators MUST NOT list 127.0.0.1. Therefore, a positive 657 listing for 127.0.0.1 SHOULD indicate that the DNSBL has started 658 listing the world and is non-functional. Similarly, a domain-based 659 DNSBL SHOULD NOT ever list the reserved domain INVALID, and a 660 positive listing for INVALID SHOULD indicate that the DNSBL is non- 661 functional. 663 Other results, such as 127.0.0.3, may have different meanings. This 664 operational flag usage and meaning SHOULD be published on the DNSBL's 665 web site, and the DNSBL user SHOULD periodically test the DNSBL. 667 Some mail systems are unable to differentiate between these various 668 results or flags, however, so a public DNSBL SHOULD NOT include 669 opposing or widely different meanings -- such as 127.0.0.23 for 670 "sends good mail" and 127.0.0.99 for "sends bad mail" -- within the 671 same DNS zone. 673 3.4. Shutdowns MUST Be Done Gracefully 675 A number of DNSBLs have shut down operations in such a way as to list 676 the entire Internet, sometimes without warning. These were usually 677 done this way to force DNSBL users (mail administrators) to adjust 678 their DNSBL client configurations to omit the now inoperative DNSBL 679 and to shed the DNS query load from the registered domain name 680 servers for the DNSBL. Popular DNSBLs are used by tens of thousands 681 of sites, yet, the correct operation of the DNSBLs are not well 682 monitored by their users. The DNSBL query clients are often not 683 compliant with DNSBL query conventions (e.g.: will treat any A record 684 returned as being "listed", instead of specific 127/8 A record 685 returns) hence shutdowns (or even ordinary domain name expiration) 686 can be quite destructive to all email flow if not done properly. 688 The DNSBL operator MUST issue impending shutdown warnings (on the 689 DNSBL web site, appropriate mailing lists, newsgroups, vendor 690 newsletters etc), and indicate that the DNSBL is inoperative using 691 the signaling given in Section 3.3. 693 Only after these warnings have been issued for a significant period 694 of time (RECOMMENDED: one or more months), should the DNSBL operator 695 finally shutdown the DNSBL. 697 The shutdown procedure should have the following properties: 699 1. MUST NOT list the entire Internet 701 2. SHOULD shed the DNSBL query load from the DNSBL name servers, 702 permitting the registered domain name to continue being usable. 704 3. SHOULD, perhaps through increased delays, indicate to the Mail 705 administrator that the DNSBL is no longer functional. 707 4. Name server or query lookups MUST NOT be aimed at third parties 708 unrelated to DNSBL operation. Such behavior is similar to 709 inflicting a DDOS attack. 711 5. The base domain name SHOULD be registered indefinitely, so as to 712 ensure that the domain name doesn't represent a "booby trap" for 713 future owners, and/or provide a means by which a new owner could 714 maliciously list the entire Internet. 716 One way of satisfying the points 1-4 above is to change the DNS name 717 servers for the DNSBL to point at "TEST-NET" addresses (see 718 [RFC3330]). The below suggested [BIND] declarations will cause a 719 DNSBL query to query non-existent name servers in TEST-NET addresses, 720 which will result in a significant delay (usually more delay as the 721 number of non-existent TEST-NET name servers is increased, but not 722 return any A records except in very unusual circumstances. 724 BIND-equivalent DNS declarations for DNSBL shutdown. 726 dnsbl.example.com. 604800 IN NS u1.example.com. 727 u1.example.com. 604800 IN A 192.0.2.1 729 dnsbl.example.com. 604800 IN NS u2.example.com. 730 u2.example.com. 604800 IN A 192.0.2.2 732 dnsbl.example.com. 604800 IN NS u3.example.com. 733 u3.example.com. 604800 IN A 192.0.2.3 735 ... [as many NS/A record pairs as you like] 737 This example assumes that the DNSBL is named "dnsbl.example.com". 738 Replace "example.com" and "dnsbl.example.com" as appropriate for the 739 DNSBL. 741 NOTE: Of course, the above shutdown procedure cannot be implemented 742 if Section 3.1 is not followed. 744 3.5. Listing of Special and Reserved IP Addresses MUST be disclosed 746 The DNSBL MAY list loopback, [RFC1918], LINK-LOCAL class [RFC3927], 747 class D/E, and any other permanently reserved or special-use IP 748 addresses [RFC3330] (and [RFC5156] for IPv6), [RFC5156]. Such use 749 MUST be disclosed in the documentation related to the DNSBL. 751 As additional insurance against listings of space that should not be 752 listed through testing or other unforeseen events, DNSBL operators 753 SHOULD consider implementing facilities to prevent them. At least 754 one popular automated DNSBL has implemented permanent exclusions for 755 such addresses. 757 A functioning DNSBL MUST NOT list 127.0.0.1. There are a number of 758 mail server implementations that do not cope with this well, and many 759 will use a positive response for 127.0.0.1 as an indication that the 760 DNSBL is shut down and listing the entire Internet. 762 3.6. Considerations for DNSBLs Listing Insecure Hosts 764 Some DNSBLs list IP addresses of hosts that are insecure in various 765 ways (e.g. open relays, open proxies). The following recommendations 766 for such DNSBLs may not be relevant to other types of DNSBLs. 768 The practise of scanning for vulnerabilities can represent a risk in 769 some jurisdictions. The following recommendations for such DNSBLs 770 MAY help alleviate this risk. 772 3.6.1. MUST NOT scan without provocation 774 DNSBLs MUST NOT automatically probe for insecure hosts without 775 provocation. There is little agreement in the community as to 776 whether or not such activity should be allowed, so this BCP errs on 777 the side of caution. 779 Therefore, scanning MUST be targeted, rather than broad-based, where 780 a given scan is motivated by a specific reason to have concern about 781 the address being scanned. Examples of such reasons include delivery 782 of an email, delivery to a spam trap address, receipt of a user 783 complaint, or periodic testing of an address that is already listed. 785 3.6.2. Re-scan Periods SHOULD be Reasonable 787 If the DNSBL operator re-scans a host in order to determine whether 788 the listing SHOULD time out or not, the re-scan period SHOULD be 789 reasonable. Automated scanning SHOULD NOT occur more often than once 790 every 24 hours. 792 It is RECOMMENDED that automated re-scanning should cease within a 793 reasonable period of the vulnerability no longer existing, and 794 targeting conditions no longer being met. 796 3.6.3. Scans MUST NOT be Destructive 798 In the past, some scanning mechanisms have proven to adversely impact 799 the scanned host, sometimes in severe fashion. Scanning 800 methodologies MUST NOT negatively impact the scanned host. 802 3.7. Removals SHOULD Be Possible in Absence of the DNSBL Operator 804 If removals cannot be automated (e.g., via robot re-testing or self- 805 removal) then the DNSBL SHOULD have multiple administrators so that a 806 removal request can be processed if the principal list administrator 807 is on vacation or otherwise unavailable. 809 3.8. Protect Against Misconfiguration/Outages 811 It is not altogether uncommon for DNSBL users to configure their 812 systems improperly for DNSBL queries. The consequences of an error 813 can range from undue (or even damaging) load on the DNSBL servers, to 814 accidentally blocking all incoming email. 816 DNSBL users MUST test their initial DNSBL configurations to ensure 817 that they're working correctly, and SHOULD periodically recheck the 818 status of the DNSBLs they use and adjust their configuration as 819 necessary. 821 Common types of misconfigurations include: 823 1. Using wrong (sub-)zones for querying (e.g. 4.3.2.1.example.com or 824 4.3.2.1.dnsbl.exmple.cm instead of 4.3.2.1.dnsbl.example.com). 826 2. Downloading a local mirror of the data, but failing to set up the 827 local name server infrastructure appropriately, and thus 828 continuing to query the public name servers. 830 3. Downloading a local mirror of the data, but misconfiguring the 831 local name server infrastructure to query a locally invented zone 832 name (4.3.2.1.dnsbl.local) at the public name servers. 834 4. Misconfiguring local name servers to not do meaningful caching, 835 thus heavily increasing load on the public name servers. 837 5. Using the DNSBL query root domain name as the name server for 838 queries. 840 6. Using the DNSBL incorrectly; e.g. Some DNSBLs are suitable only 841 for certain types of filtering. Improper use may result in 842 excessive incorrect filtering. 844 While in many cases, it can be difficult to detect such situations, 845 to protect against such misconfiguration it is RECOMMENDED that DNSBL 846 operators make design decisions to mitigate the impact of such 847 mistakes, and make efforts to contact administrative contacts to 848 remedy the situation where appropriate. But the DNSBL operator 849 SHOULD also prepare to take appropriate steps to protect the 850 operational infrastructure (e.g., have the ability to block abusive 851 users from causing further damage). 853 Appropriate use of the DNSBL (e.g. email, not IRC, not against 854 authenticated local users) SHOULD be documented on the web site. 856 3.9. Error Handling 858 From time to time, DNSBLs have encountered operational data integrity 859 or data collection problems that have resulted in improper listings. 860 For example: data corruption, erroneous restoration of since-resolved 861 listings, or grossly misfiring detection heuristics. This has often 862 results in great consternation over what appear to be nonsensical 863 listings or listings for previously resolved issues. 865 Many DNSBLs have implemented policies and procedures whereby such 866 situations result in the purging of even slightly doubtful entries, 867 disconnection of untrustworthy components until the entries' validity 868 or correct operation of the component can be verified or corrected, 869 as well as provide notification of the issue on the DNSBL's web 870 pages. 872 As an example, one popular DNSBL has a demonstrated track record of 873 disabling faulty data collection mechanisms, purging all listings 874 generated by the faulty mechanism, and publishing a brief description 875 of the problem and course of remediation. 877 Therefore, DNSBLs SHOULD have policies and procedures in place to 878 treat operational problems conservatively, be prepared to mass purge 879 dubious entries, prevent future erroneous entries, and notify their 880 users by the DNSBL's web page. 882 4. Security Considerations 884 Any system manager that uses DNSBLs is entrusting part of his or her 885 server management to the parties that run the lists. A DNSBL manager 886 that decided to list 0/0 (which has actually happened) could cause 887 every server that uses the DNSBL to reject all mail. Conversely, if 888 a DNSBL manager removes all of the entries (which has also happened), 889 systems that depend on the DNSBL will find that their filtering 890 doesn't work as they want it to. 892 If a registered domain name used for a DNSBL is allowed to lapse, or 893 the DNSBL user spells the DNSBL domain name incorrectly, the system 894 manager's server management is now subject to an entirely different 895 party than was intended. Further, even if there is no malicious 896 intent, some DNSBL query clients will interpret any A record being 897 returned as being listed. DNSBL users SHOULD be prepared to 898 periodically test the DNSBLs they use for correct operation. 900 Like all DNS-based mechanisms, DNSBLs are subject to various threats 901 outlined in [RFC3833]. 903 5. IANA Considerations 905 This document has no actions for IANA. [This section may be removed 906 before publishing as an RFC.] 908 6. References 909 6.1. Normative References 911 [RFC1918] Rekhter, Y., Moskowitz, R., Karrenberg, D., Groot, G., and 912 E. Lear, "Address Allocation for Private Internets", 913 BCP 5, RFC 1918, February 1996. 915 [RFC2014] Weinrib, A. and J. Postel, "IRTF Research Group Guidelines 916 and Procedures", BCP 8, RFC 2014, October 1996. 918 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 919 Requirement Levels", BCP 14, RFC 2119, March 1997. 921 [RFC3927] Cheshire, S., Aboba, B., and E. Guttman, "Dynamic 922 Configuration of IPv4 Link-Local Addresses", RFC 3927, 923 May 2005. 925 6.2. Informative References 927 [BIND] Internet Systems Corporation, "ISC BIND", 928 . 930 [RFC3330] IANA, "Special-Use IPv4 Addresses", RFC 3330, 931 September 2002. 933 [RFC3833] Atkins, D. and R. Austein, "Threat Analysis of the Domain 934 Name System (DNS)", RFC 3833, August 2004. 936 [RFC5156] Blanchet, M., "Special-Use IPv6 Addresses", RFC 5156, 937 April 2008. 939 [RFC5782] Levine, J., "DNS Blacklists and Whitelists", RFC 5782, 940 February 2010. 942 [RSYNC] Tridgell, A., "rsync". 944 [RSYNCTHESIS] 945 Tridgell, A., "Efficient Algorithms for Sorting and 946 Synchronization". 948 Appendix A. Acknowledgements 950 We would like to thank John R. Levine, Alan Murphy and Dave Crocker 951 for their insightful comments. 953 We would also like to thank Yakov Shafranovich and Nick Nicholas for 954 editing previous versions of this document. 956 Authors' Addresses 958 Chris Lewis 959 Nortel Networks 961 Email: clewisbcp@cauce.org 963 Matt Sergeant 964 Symantec Corporation 966 Email: matt_sergeant@symantec.com