idnits 2.17.1 draft-irtf-asrg-bcp-blacklists-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 16. -- Found old boilerplate from RFC 3978, Section 5.5, updated by RFC 4748 on line 900. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 911. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 918. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 924. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 15 instances of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust Copyright Line does not match the current year == The document seems to use 'NOT RECOMMENDED' as an RFC 2119 keyword, but does not include the phrase in its RFC 2119 key words list. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (November 17, 2008) is 5610 days in the past. Is this intentional? Checking references for intended status: Best Current Practice ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Obsolete informational reference (is this intentional?): RFC 3330 (Obsoleted by RFC 5735) -- Obsolete informational reference (is this intentional?): RFC 5156 (Obsoleted by RFC 6890) Summary: 1 error (**), 0 flaws (~~), 3 warnings (==), 10 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Anti-Spam Research Group - IRTF C. Lewis 3 Internet-Draft Nortel Networks 4 Intended status: BCP M. Sergeant 5 Expires: May 21, 2009 MessageLabs, Inc 6 November 17, 2008 8 Guidelines for Management of DNSBLs for Email 9 draft-irtf-asrg-bcp-blacklists-05 11 Status of this Memo 13 By submitting this Internet-Draft, each author represents that any 14 applicable patent or other IPR claims of which he or she is aware 15 have been or will be disclosed, and any of which he or she becomes 16 aware will be disclosed, in accordance with Section 6 of BCP 79. 18 Internet-Drafts are working documents of the Internet Engineering 19 Task Force (IETF), its areas, and its working groups. Note that 20 other groups may also distribute working documents as Internet- 21 Drafts. 23 Internet-Drafts are draft documents valid for a maximum of six months 24 and may be updated, replaced, or obsoleted by other documents at any 25 time. It is inappropriate to use Internet-Drafts as reference 26 material or to cite them other than as "work in progress." 28 The list of current Internet-Drafts can be accessed at 29 http://www.ietf.org/ietf/1id-abstracts.txt. 31 The list of Internet-Draft Shadow Directories can be accessed at 32 http://www.ietf.org/shadow.html. 34 This Internet-Draft will expire on May 21, 2009. 36 Abstract 38 The rise of spam and other anti-social behavior on the Internet has 39 led to the creation of shared DNS-based lists ("DNSBLs") of IP 40 addresses or domain names intended to help guide email filtering. 41 This memo discusses guidelines for the management of public DNSBLs by 42 their operators as well as for the proper use of such lists by mail 43 server administrators (DNSBL users), and it provides useful 44 background for both parties. It is not intended to advise on the 45 utility or efficacy of particular DNSBLs or the DNSBL concept in 46 general, nor to assist end users with questions about spam. 48 The document will seek BCP status. Comments and discussion of this 49 document should be addressed to the asrg@ietf.org mailing list. 51 Table of Contents 53 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 54 1.1. DNS-Based Reputation Systems . . . . . . . . . . . . . . . 3 55 1.2. Guidance for DNSBL Users . . . . . . . . . . . . . . . . . 5 56 1.3. Requirements Language . . . . . . . . . . . . . . . . . . 6 57 1.4. Background . . . . . . . . . . . . . . . . . . . . . . . . 6 58 2. DNSBL Policies . . . . . . . . . . . . . . . . . . . . . . . . 7 59 2.1. Transparency . . . . . . . . . . . . . . . . . . . . . . . 7 60 2.1.1. Listing/Delisting Criteria SHOULD Be Easily 61 Available . . . . . . . . . . . . . . . . . . . . . . 8 62 2.1.2. Audit Trail SHOULD be maintained . . . . . . . . . . . 8 63 2.1.3. The Scope and Aggressiveness of Listings MUST be 64 Disclosed. . . . . . . . . . . . . . . . . . . . . . . 8 65 2.2. Listings and Removals . . . . . . . . . . . . . . . . . . 9 66 2.2.1. Listings SHOULD Be Temporary . . . . . . . . . . . . . 9 67 2.2.2. A Direct Non-Public Way to Request Removal SHOULD 68 Be Available . . . . . . . . . . . . . . . . . . . . . 10 69 2.2.3. Removals SHOULD Be Prompt . . . . . . . . . . . . . . 11 70 2.2.4. SHOULD Have Similar Criteria for Listing and 71 Delisting . . . . . . . . . . . . . . . . . . . . . . 11 72 3. Operational Issues . . . . . . . . . . . . . . . . . . . . . . 12 73 3.1. DNSBL Query Root Domain Name SHOULD be a Subdomain . . . . 12 74 3.2. DNSBLs SHOULD be Adequately Provisioned . . . . . . . . . 12 75 3.3. DNSBLs SHOULD Provide Operational Flags . . . . . . . . . 13 76 3.4. Shutdowns MUST Be Done Gracefully . . . . . . . . . . . . 14 77 3.5. Listing of Special and Reserved IP Addresses MUST be 78 disclosed . . . . . . . . . . . . . . . . . . . . . . . . 15 79 3.6. Considerations for DNSBLs Listing Insecure Hosts . . . . . 15 80 3.6.1. MUST NOT scan without provocation . . . . . . . . . . 16 81 3.6.2. Re-scan Periods SHOULD be Reasonable . . . . . . . . . 16 82 3.6.3. Scans MUST NOT be Destructive . . . . . . . . . . . . 16 83 3.7. Removals SHOULD Be Possible in Absence of the DNSBL 84 Operator . . . . . . . . . . . . . . . . . . . . . . . . . 16 85 3.8. Protect Against Misconfiguration/Outages . . . . . . . . . 16 86 4. Security Considerations . . . . . . . . . . . . . . . . . . . 17 87 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18 88 6. References . . . . . . . . . . . . . . . . . . . . . . . . . . 18 89 6.1. Normative References . . . . . . . . . . . . . . . . . . . 18 90 6.2. Informative References . . . . . . . . . . . . . . . . . . 18 91 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 19 92 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 19 93 Intellectual Property and Copyright Statements . . . . . . . . . . 20 95 1. Introduction 97 1.1. DNS-Based Reputation Systems 99 Due to the rising amount of spam and other forms of network abuse on 100 the Internet, many community members and companies began to create, 101 publish and maintain DNS-based reputation systems (DNS-Based Lists) 102 of IP addresses or domain names and make reputation suggestions or 103 assertions about email sourced from these IP addresses or domain 104 names. 106 The first DNS-based Lists were almost exclusively intended to be used 107 (by email administrators) as lists of abusive IP addresses to block, 108 however the DNS publication method has proven to be so robust, 109 popular and simple to use, that it has been extended for use in many 110 different ways, far beyond the imaginings of the designers of DNS or 111 DNS-based blocking IP lists. For example, today, the same basic DNS- 112 based listing technology is commonly used for: 114 DNSWL listings of well-behaving email source IP/domain addresses 115 (whitelist). 117 RHSBL listings of well/ill behaving email source domain names (often 118 applied against the domain name part of the originating email 119 address or DNS PTR (reverse IP) lookups) 121 URIBL listings of well/ill behaving web link domain names or host 122 names used in email 124 Further, the DNSBL user using the list doesn't have to use a listing 125 as a pass/fail binary decision, it can use a listing as one factor in 126 email filters that make decisions based on scoring multiple factors 127 together. 129 The DNS-based list technology has even been extended to purely 130 informational purposes. For example, there are implementations that 131 return results based on what geographic region an IP/domain is 132 putatively allocated in, implementations that translate an IP/domain 133 address into an ASN number and/or allocation block, implementations 134 that indicate whether the queried domain name is registered through a 135 given Domain registrar, implementations that return aggregate numeric 136 reputation for an IP address or domain name from another system's 137 email system, and so on. The possibilities are virtually endless. 139 As well, DNS-based listing technology has also been used in areas 140 other than email filtering, such as IRC, web access control, and 141 transaction verification. 143 As the terminology in this area has never been well formalized, often 144 overlaps, and lacks precision, this document has been written to use 145 the term "DNSBL" to refer to DNS-based lists generally, not just DNS- 146 based block (or black) lists. This document is not applicable to 147 some DNSBLs in some areas (mentioned as appropriate) but it is the 148 authors' belief that most of the practises are applicable to almost 149 all DNSBLs. 151 DNSBLs may be either public or private. A public DNSBL makes its 152 data available to any party seeking information about data on the 153 list, while a private DNSBL is used solely by an organization for its 154 own use and the data is not made available publicly. There are also 155 commercial DNSBLs, available for a fee. Furthermore, some are free 156 yet require a fee for higher numbers of queries or certain classes of 157 DNSBL users. 159 The first publicly available DNSBL using the Domain Name System (DNS) 160 for distributing reputation data about email senders emerged in 1997, 161 shortly after spam became a problem for network operators and email 162 administrators. This pioneer DNSBL focused on identifying known spam 163 sources situated at static (unchanging) IP/domain addresses. Due to 164 the broad adoption of this DNSBL, it had a major impact on static 165 spam sources. Consequently, abusers found other methods for 166 distributing their spam, such as relaying messages through unsecured 167 email servers or flawed formmail scripts on web pages. Additional 168 DNSBLs were developed by others in order to address these changing 169 tactics, and today more than 700 public DNSBLs are known to be in 170 operation. 172 These DNSBLs vary widely in purpose for which the list was intended, 173 the method the list uses to achieve the purpose, the integrity of 174 those overseeing the method, and the stability of the technology used 175 to create and distribute the data. Listing criteria can sometimes be 176 quite controversial, therefore this document deliberately does not 177 discuss the rightness or wrongness of any criteria. We assert that 178 DNSBL operators are free to choose whatever listing criteria they 179 wish, as long as those criteria are clearly and accurately 180 communicated. It is the responsibility of the DNSBL user to ensure 181 that the listing criteria and other aspects of a DNSBL meets their 182 needs. 184 This document is intended to provide guidance to DNSBL operators so 185 that they may be able to identify what features users would be 186 interested in seeing as part of a high-quality, well-managed DNSBL, 187 for example, a clear listing and delisting policy to which the DNSBL 188 operator adheres strictly. This document is intended to be normative 189 rather than prescriptive: it seeks to characterize the features of a 190 well-managed DNSBL rather than setting out rules for how DNSBLs 191 should be operated. 193 This document is not intended as a protocol specification of DNSBL 194 queries. (See [DNSBL-EMAIL].) 196 1.2. Guidance for DNSBL Users 198 When choosing to adopt a DNSBL, a DNSBL user SHOULD keep the 199 following questions in mind: 201 1. What is the intended use of the list? 203 2. Does the list have a web site? 205 3. Are the list's policies stated on the web site? 207 4. Are the policies stated clearly and understandably? 209 5. Does the web site function properly, e.g., hyperlinks? 211 6. Are web pages for removal requirements accessible and working 212 properly? 214 7. How long has the list been in operation? 216 8. What are the demographics and quantity of the list's user base? 217 In other words, do other sites like my own use this DNSBL? 219 9. Are comparative evaluations of the list available? Note: all 220 such evaluations depend on the mail mix used as well as local 221 policy. DNSBL users SHOULD consider trial periods and/or 222 ongoing local monitoring of DNSBL suitability. 224 10. What do your peers or members of the Internet community say 225 about the list? DNSBLs can sometimes be quite controversial and 226 sometimes considerable misinformation is spread. Ensure that 227 the opinions are knowledgeable, and reflect similar goals to 228 yours. 230 11. Does the DNSBL have a mailing list for announcing changes, 231 outages etc? 233 DNSBLs can, and have, ceased operation without notice. DNSBL users 234 SHOULD periodically check the correct operation of the DNSBL, and 235 cease using DNSBLs that are working incorrectly. See Section 3.3 237 The DNSBL user MUST ensure that they understand the intended use of 238 the DNSBL. For example, some IP address-based DNSBLs are appropriate 239 only for assessment of the peer IP address of the machine connecting 240 to the DNSBL user's mail server, and not other IP addresses appearing 241 in an email (such as header Received lines or web links), or IRC 242 connections etc. While a DNSBL user may choose to ignore the intent 243 of the DNSBL, they SHOULD implement any variance in compliance with 244 the DNSBL usage instructions. 246 For example, one of the requirements of some DNSBLs is that if the 247 DNSBL is used contrary to the usage instructions, then the DNSBL user 248 should not identify the DNSBL being used. Furthermore, it is the 249 DNSBL user's responsibility to mitigate the effect of the listing 250 locally. 252 It is the responsibility of the system administrators who adopt one 253 or more DNSBLs to evaluate, understand, and make a determination of 254 which DNSBLs are appropriate for the sites they administer. If you 255 are going to allow a third party's information to guide your 256 filtering decision-making process, you MUST understand the policies 257 and practices of those third parties because responsibility for 258 filter decisions remains ultimately with you, the postmaster. 260 A DNSBL without DNSBL users does not block (or otherwise impair) 261 email or any other Internet service. A DNSBL user voluntarily uses 262 the DNSBL data to guide their decisions, and the DNSBL user therefore 263 MUST assume responsibility for dealing with the consequences. 265 DNSBL operators are expressing an opinion through the publication of 266 a DNSBL. However, it is through abiding by the guidelines set forth 267 in this BCP that the operators of a DNSBL may gain the trust of their 268 users. 270 These guidelines only address public DNSBLs and do not apply to 271 private access DNSBLs, however, implementors and users of private 272 access DNSBLs may wish to use these guidelines as a starting point of 273 things to consider. 275 1.3. Requirements Language 277 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 278 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 279 document are to be interpreted as described in [RFC2119]. 281 1.4. Background 283 The Anti Spam Research Group (ASRG) was chartered to address the spam 284 problem. The ASRG charter includes: 286 "codification of best current practices in spam management" 287 This note falls within that category by listing guidelines for 288 management of public DNSBLs. This document will seek BCP status. 290 NOTE: This document is a product of the Anti-Spam Research Group 291 (ASRG) of the IRTF. As per section 3 of [RFC2014] IRTF groups do 292 not require consensus to publish documents. Therefore readers 293 should be aware that this document does not necessarily represent 294 the consensus of the entire ASRG. 296 NOTE: This document is intended to evolve, based on comments from 297 the Anti-Spam Research Group (ASRG) mailing list. It is certain 298 that the current draft is incomplete and entirely possible that it 299 is inaccurate. Hence, comments are eagerly sought, preferably in 300 the form of suggested text changes, and preferably on the ASRG 301 mailing list, at . 303 2. DNSBL Policies 305 2.1. Transparency 307 A DNSBL SHOULD carefully describe the criteria that are the cause for 308 adding, and the criteria for removing an entry from the list. Such 309 listing and delisting criteria SHOULD be presented in a clear and 310 readable manner easily accessible to the public on the DNSBL's web 311 site. A DNSBL MUST abide by its stated listing and delisting 312 criteria. Entries that do not meet the published criteria MUST NOT 313 be added to the DNSBL. 315 In other words, be direct and honest and clear about the listing 316 criteria, and make certain that only entries meeting the published 317 criteria are added to the list. For example, some DNSBL operators 318 have been known to include "spite listings" in the lists they 319 administer -- listings of IP addresses or domain names associated 320 with someone who has insulted them, rather than actually violating 321 the published criteria for inclusion in the list. There is nothing 322 inherently wrong with this practice so long as it is clearly 323 disclosed. For example, a DNSBL described as only listing open 324 relays MUST NOT include IP addresses for any other reason. This 325 transparency principle does not require DNSBL operators to disclose 326 the precise algorithms and data involved in a listing, but rather the 327 intent behind choosing those algorithms and data. 329 Furthermore, the DNSBL documentation SHOULD be clear on the intended 330 use of the DNSBL - whether it be intended for peer addresses of 331 email, IRC, etc. 333 Availability of documentation concerning a DNSBL SHOULD NOT be 334 dependent on the continued operation of DNS for DNSBL queries. 336 In other words, if the DNSBL documentation is at 337 "http://dnsbl.example.com", the documentation for the web site should 338 not become unavailable if the DNSBL query name servers are not 339 available (or shut down). See Section 3.1. 341 2.1.1. Listing/Delisting Criteria SHOULD Be Easily Available 343 Listing and delisting criteria for DNSBLs SHOULD be easily available 344 and SHOULD be located in a place clearly marked in its own section of 345 the web site affiliated with the DNSBL. 347 DNSBLs often publish their listing criteria along with additional 348 technical information about using the DNSBL. This additional 349 technical information can confuse end users, so a separate page, 350 section or query function on its own SHOULD be dedicated to detailing 351 why a specific entry appears in the DNSBL. 353 2.1.2. Audit Trail SHOULD be maintained 355 A DNSBL SHOULD maintain an audit trail for all listings and it is 356 RECOMMENDED that it is made publicly available in an easy to find 357 location, preferably on the DNSBL's web site. Please note that 358 making an audit trail data public does not entail revealing all 359 information in the DNSBL operator's possession relating to the 360 listing; e.g., a DNSBL operator MAY make the audit trail data 361 selectively accessible in such a way as to not disclose information 362 that might assist spammers, such as the location or identity of a 363 spam trap. 365 2.1.3. The Scope and Aggressiveness of Listings MUST be Disclosed. 367 Some DNSBLs have adopted policies of listing entries that are broader 368 in scope than they have evidence of being involved in abuse. 369 Similarly, some DNSBLs list entries that are "mixed", in that the 370 entry may be behaving in a manner that is both abusive and non- 371 abusive. This is inherent to the techniques that many DNSBLs use. 373 Examples: Some DNSBLs will list IP address ranges if there is reason 374 to believe that abusive behaviour seen from a few IP addresses within 375 the range is (or will be) reflected in the rest of the range. Some 376 DNSBLs utilize scoring to list IP addresses, IP ranges or domain 377 names that have abusive behaviour above some threshold - often 378 meaning that some of the email corresponding to the listing is not 379 abusive. Even an entry demonstrably infected with email spam or 380 virus emitting malware may emit non-abusive email. 382 Inevitably, some of these listings may impact non-abusive email. 383 This has resulted in some labelling such practises by the emotionally 384 loaded term "collateral damage". No filtering technique is perfect, 385 and that an occasional mistake is inevitable no matter what is used, 386 DNSBLs or otherwise. 388 There is nothing wrong with this practise, because mail server 389 administrators may wish to implement such policies or use them in 390 combination with other techniques (such as scoring). However, a 391 diligent administrator needs information about these policies in 392 order to make an informed decision as to the risk and benefit of 393 using any particularly DNSBL, and in many cases guide them in how to 394 use it for results best reflecting the DNSBL user's requirements. 396 Therefore, DNSBL listing policies MUST include statements as to the 397 scope and aggressiveness of listings, and include, as appropriate, 398 whether the DNSBL operator intends the listings to be used in scoring 399 or other techniques. 401 2.2. Listings and Removals 403 2.2.1. Listings SHOULD Be Temporary 405 In many cases, listings can exist for long periods of time past the 406 conditions leading to the listing's creation, and/or the listed 407 entity has putatively changed ownership. 409 Generally speaking, listings SHOULD be considered temporary, and 410 should expire on their own at some point in the future unless reasons 411 for listing still exist. 413 Expiration intervals SHOULD be chosen to be reasonable for the type 414 of listing. For example: 416 1. It does not make sense to remove entries from DNSBL where the 417 existance of an entry is not of direct meaning. In other words, 418 DNSBLs that return information in addition to just existance/ 419 non-existance. For example: entries in DNSBLs that return 420 geographic or assignment information on where the IP address or 421 domain name is located or owned, or DNSBLs that return flow 422 statistics from the DNSBL operator that are intended for the 423 DNSBL user to interpret, need not ever be removed, just kept 424 reasonably current. 426 2. DNSBLs based on relatively static information, such as block 427 assignment or domain names of demonstrably bad actors MAY have 428 very long expiration intervals or only be removed upon request 429 after verification that the removal criteria has been met. 431 3. Automated DNSBLs with highly effective detection and fast listing 432 mechanisms can benefit from very short expiration intervals. 433 Many of the things that these DNSBLs look for are of relatively 434 short duration, and even if they do expire, a resumption of the 435 behaviour will be caught quickly by the DNSBL's detection 436 mechanisms and relisted. By utilizing a short expiration 437 interval, after reassignment/problem correction, the listing will 438 automatically expire in short order without manual intervention. 440 4. Manually created DNSBL entries SHOULD be periodically reviewed in 441 some manner. 443 It is RECOMMENDED that DNSBL operators publish in general terms their 444 expiration policy, even if its only "delist on request" or no 445 expiration is performed. In information-only lists, a method for 446 users requesting corrections to the information (if appropriate) 447 SHOULD be published. Abusers may be able to "game" policy that is 448 too explicit; on the other hand, many DNSBL users wish to have an 449 idea of how "current" the DNSBL is. It is the authors' experience 450 that some automated DNSBLs have increasingly higher error rates as 451 the "last detection date" gets older. 453 Note that listings being temporary does not mean that some listings 454 will not remain after the initial timeout period. If the DNSBL 455 operator determines that the conditions triggering listing still 456 exist, then the timer for determining timeouts can be renewed. 458 2.2.2. A Direct Non-Public Way to Request Removal SHOULD Be Available 460 Discussions about whether a DNSBL should remove an entry MAY include 461 activity in a public forum. Methods for processing removal requests 462 through private, direct exchanges, such as person-to-person email or 463 a combination of web page requests and email responses, SHOULD be 464 available. As a minimum, the DNSBL SHOULD have a web page that has a 465 removal request function (separate from the page describing listing 466 criteria as per Section 2.1.1). The DNSBL SHOULD also make available 467 an email address to handle issues other than blocking issues. 469 The DNSBL operator MUST NOT use the list in question in such a way 470 that removal requests would be blocked, and, moreover, SHOULD make 471 mailboxes available in order to allow affected users to submit their 472 requests. In some cases it is impractical not to filter email to 473 accounts due to the amount of spam those mailboxes receive. If 474 filtering should be necessary in such circumstances, filtering 475 methods with as low false positive rate as practical SHOULD be 476 chosen. 478 2.2.3. Removals SHOULD Be Prompt 480 The response to removal requests (if the conditions for list removal 481 are present) SHOULD be prompt. 483 A DNSBL MAY impose restrictions on who (e.g. network operator's 484 representative or domain name owner) may make valid removal requests. 485 However, in many DNSBLs this is inadvisable because it requires 486 impractical amounts of effort and is hence NOT RECOMMENDED in most 487 cases. 489 Many DNSBLs (especially those with highly effective detection and 490 fast listing mechanisms) greatly benefit from a "no questions asked" 491 removal policy. 493 Although this approach allows people to submit a request and have any 494 listed IP address/domain name removed immediately, it does not 495 prevent the DNSBL operator from re-listing the IP address/domain name 496 at a later time. 498 Many DNSBLs can effectively use a "no questions asked" removal policy 499 because by their very nature they will redetect or relist problems 500 almost immediately. They can mitigate more organized attempts to 501 "game" the system by elementary checking and rate-limiting 502 procedures, increasing lockout periods, rescans etc. Furthermore, a 503 few IP addresses more or less usually do not make a significant 504 difference in the overall effectiveness of a DNSBL. Moreover, a "no 505 questions asked" removal policy provides the huge benefit of a swift 506 reaction to incorrect listings. 508 As an example, one popular DNSBL uses a "no questions asked" removal 509 policy, but does perform rate-limiting and malicious removal 510 detection and mitigation. 512 Another important consideration supporting a "no questions asked" 513 self-removal policy is that it forestalls many conflicts between 514 DNSBL operators and organizations whose IP addresses/domain names 515 have been listed. Such a policy may be an effective measure to 516 prevent small issues from becoming big problems. 518 2.2.4. SHOULD Have Similar Criteria for Listing and Delisting 520 The criteria for being removed from a DNSBL SHOULD bear a reasonable 521 relationship to the factors that were the cause of the addition to 522 the DNSBL. If a listed entity fulfills all published requirements 523 for removal from a DNSBL, then the DNSBL operator SHOULD NOT impose 524 any additional obstacles to remove a given entry from the DNSBL. 525 There SHOULD NOT be any extra rules for de-listing other than the 526 ones listed in the published listing criteria. 528 3. Operational Issues 530 3.1. DNSBL Query Root Domain Name SHOULD be a Subdomain 532 By virtue of using domain names, a DNSBL is a hierarchy with a root 533 anchored in the global Internet. The DNSBL "query root" SHOULD be 534 below the registered domain name, so that the DNSBL information is 535 not conflated with domain name housekeeping information (e.g., name 536 server or MX records) for the domain name. By using this approach, 537 DNSBL queries would take the form of ".dnsbl.example.com" 538 rather than ".example.com". Further, this sub-tree should 539 have its own name servers. Thus, the DNSBL query root has its own 540 zone file containing the DNSBL information, and the registered domain 541 name has its own name servers containing the information (MX records 542 etc.) for the domain name. This approach facilitates clear 543 delineation of function as well as orderly DNSBL shutdown because the 544 DNSBL nameserver records can be specified separately from the domain 545 name's principal nameservers. 547 Many DNSBLs support more than one logical zone (DNSBL entries with 548 different meanings) that DNSBL users may wish to treat differently 549 (or even ignore). It is RECOMMENDED that, even if there is a single 550 DNSBL zone with entry type distinguished by return code, that 551 separate subdomain names (of the query root) consist only of the 552 corresponding entries. For example, entry types "A" and "B" might 553 return 127.0.0.2 and 127.0.0.3 from the consolidated zone (eg: 554 dnsbl.example.com), but there should also be zones 555 typeA.dnsbl.example.com and typeB.dnsbl.example.com that contain 556 their respective types only. See also Section 3.3. 558 3.2. DNSBLs SHOULD be Adequately Provisioned 560 The DNSBL SHOULD have sufficient name server capacity to handle the 561 expected loading, and have sufficient redundancy to handle normal 562 outages. 564 Nameservers SHOULD provide appropriate glue records, possibly in 565 different TLDs to protect against single-TLD issues. 567 If the DNSBL offers zone transfers (in addition to or instead of 568 standard DNSBL query mechanisms), it SHOULD be sufficiently 569 provisioned to handle the expected loading. 571 Note that some DNSBLs have been subject to distributed denial of 572 service attacks. Provisioning SHOULD take the likelyhood of this 573 into account, and include plans for dealing with it. 575 3.3. DNSBLs SHOULD Provide Operational Flags 577 Most IP address-based DNSBLs follow a convention of query entries for 578 IP addresses in 127.0.0.0/8 (127.0.0.0-127.255.255.255) to provide 579 online indication of whether the DNSBL is operational. Many, if not 580 most, DNSBLs arrange to have a query of 127.0.0.2 return an A record 581 indicating that the IP address is listed. This appears to be a 582 defacto standard. (See [DNSBL-EMAIL].) 584 If this indicator is missing (query of 127.0.0.2 returns NXDOMAIN), 585 the DNSBL should be considered non-functional. 587 There does not appear to be a defacto standard for test entries 588 within domain name-based DNSBLs. A number use the same 127.0.0.2 589 query test mechanism as IP address-based DNSBLs, and others use a 590 variety of domain name-based test entries. Due to the way many 591 domain name-based DNSBLs are used (eg: hostname parts of URIs in 592 email bodies), using anything likely to appear in a legitimate email 593 is a bad idea (eg: http://example.com), especially considering that 594 some email readers will transform bare IP addresses or domain names 595 appearing in the body of an email into links. So, even 127.0.0.2 may 596 be problemmatic. But a common testing method is desirable. 598 In the absence of new emerging standards, it is RECOMMENDED that 599 domain name-based DNSBLs use a test entry of "_DNSBL_.test". Test is 600 chosen because it is a reserved top-level-domain, and the underscores 601 because they are generally prohibited in hostnames, and are highly 602 unlikely to appear in valid URIs. 604 Note: In Section 3.4 it is noted that some DNSBLs have shut down in 605 such a way to list all of the Internet. Further, in Section 3.5, 606 DNSBL operators MUST NOT list 127.0.0.1. Therefore, a positive 607 listing for 127.0.0.1 SHOULD be interpretable as an indicator that 608 the DNSBL has started listing the world and is non-functional. 610 Other results, such as 127.0.0.3, may have different meanings. This 611 operational flag usage and meaning SHOULD be published on the DNSBL's 612 web site, and the DNSBL user SHOULD periodically test the DNSBL. 614 Some mail systems are unable to differentiate between these various 615 results or flags, however, so a public DNSBL SHOULD NOT include 616 opposing or widely different meanings -- such as 127.0.0.23 for 617 "sends good mail" and 127.0.0.99 for "sends bad mail" -- within the 618 same DNS zone. 620 3.4. Shutdowns MUST Be Done Gracefully 622 A number of DNSBLs have shut down operations in such a way as to list 623 the entire Internet, sometimes without warning. These were usually 624 done this way to force DNSBL users (mail administrators) to adjust 625 their DNSBL client configurations to omit the now inoperative DNSBL 626 and to shed the DNS query load from the registered domain name 627 servers for the DNSBL. Popular DNSBLs are used by tens of thousands 628 of sites, yet, the correct operation of the DNSBLs are not well 629 monitored by their users. The DNSBL query clients are often not 630 compliant with DNSBL query conventions (e.g.: will treat any A record 631 returned as being "listed", instead of specific 127/8 A record 632 returns) hence shutdowns (or even ordinary domain name expiration) 633 can be quite destructive to all email flow if not done properly. 635 The DNSBL operator MUST issue impending shutdown warnings (on the 636 DNSBL web site, appropriate mailing lists, newsgroups, vendor 637 newsletters etc), and indicate that the DNSBL is inoperative using 638 the signalling given in Section 3.3. 640 Only after these warnings have been issued for a significant period 641 of time (RECOMMENDED: one or more months), should the DNSBL operator 642 finally shutdown the DNSBL. 644 The shutdown procedure should have the following properties: 646 1. MUST NOT list the entire Internet 648 2. SHOULD shed the DNSBL query load from the DNSBL name servers, 649 permitting the registered domain name to continue being useable. 651 3. SHOULD, perhaps through increased delays, indicate to the Mail 652 administrator that the DNSBL is no longer functional. 654 4. Name server or query lookups MUST NOT be aimed at third parties 655 unrelated to DNSBL operation. Such behaviour is similar to 656 inflicting a DDOS attack. 658 5. The base domain name SHOULD be registered indefinitely, so as to 659 ensure that the domain name doesn't represent a "booby trap" for 660 future owners, and/or provide a means by which a new owner could 661 maliciously list the entire Internet. 663 One way of satisfying the points 1-4 above is to change the DNS name 664 servers for the DNSBL to point at "TEST-NET" addresses (see 665 [RFC3330]). The below suggested [BIND] declarations will cause a 666 DNSBL query to query non-existant name servers in TEST-NET addresses, 667 which will result in a significant delay (usually more delay as the 668 number of non-existant TEST-NET name servers is increased, but not 669 return any A records except in very unusual circumstances. 671 BIND-equivalent DNS declarations for DNSBL shutdown. 673 dnsbl.example.com. 604800 IN NS u1.example.com. 674 u1.example.com. 604800 IN A 192.0.2.1 676 dnsbl.example.com. 604800 IN NS u2.example.com. 677 u2.example.com. 604800 IN A 192.0.2.2 679 dnsbl.example.com. 604800 IN NS u3.example.com. 680 u3.example.com. 604800 IN A 192.0.2.3 682 ... [as many NS/A record pairs as you like] 684 This example assumes that the DNSBL is named "dnsbl.example.com". 685 Replace "example.com" and "dnsbl.example.com" as appropriate for the 686 DNSBL. 688 NOTE: Of course, the above shutdown procedure cannot be implemented 689 if Section 3.1 is not followed. 691 3.5. Listing of Special and Reserved IP Addresses MUST be disclosed 693 The DNSBL MAY list loopback, [RFC1918], LINK-LOCAL class [RFC3927], 694 class D/E, and any other permanently reserved or special-use IP 695 addresses [RFC3330] (and [RFC5156] for IPv6), [RFC5156]. Such use 696 MUST be disclosed in the documentation related to the DNSBL. 698 As additional insurance against listings of space that should not be 699 listed through testing or other unforeseen events, DNSBL operators 700 SHOULD consider implementing facilities to prevent them. At least 701 one popular automated DNSBL has implemented permanent exclusions for 702 such addresses. 704 A functioning DNSBL MUST NOT list 127.0.0.1. There are a number of 705 mail server implementations that do not cope with this well, and many 706 will use a positive response for 127.0.0.1 as an indication that the 707 DNSBL is shut down and listing the entire Internet. 709 3.6. Considerations for DNSBLs Listing Insecure Hosts 711 Some DNSBLs list IP addresses of hosts that are insecure in various 712 ways (e.g. open relays, open proxies). The following recommendations 713 for such DNSBLs may not be relevant to other types of DNSBLs. 715 The practise of scanning for vulnerabilities can represent a risk in 716 some jurisdictions. The following recommendations for such DNSBLs 717 MAY help alleviate this risk. 719 3.6.1. MUST NOT scan without provocation 721 DNSBLs MUST NOT automatically probe for insecure hosts without 722 provocation. There is little agreement in the community as to 723 whether or not such activity should be allowed, so this BCP errs on 724 the side of caution. 726 Therefore, scanning MUST be targeted, rather than broad-based, where 727 a given scan is motivated by a specific reason to have concern about 728 the address being scanned. Examples of such reasons include delivery 729 of an email, delivery to a spam trap address, receipt of a user 730 complaint, or periodic testing of an address that is already listed. 732 3.6.2. Re-scan Periods SHOULD be Reasonable 734 If the DNSBL operator re-scans a host in order to determine whether 735 the listing SHOULD timeout or not, the re-scan period SHOULD be 736 reasonable. Automated scanning SHOULD NOT occur more often than once 737 every 24 hours. 739 It is RECOMMENDED that automated re-scanning should cease within a 740 reasonable period of the vulnerability no longer existing, and 741 targetting conditions no longer being met. 743 3.6.3. Scans MUST NOT be Destructive 745 In the past, some scanning mechanisms have proven to adversely impact 746 the scanned host, sometimes in severe fashion. Scanning 747 methodologies MUST NOT negatively impact the scanned host. 749 3.7. Removals SHOULD Be Possible in Absence of the DNSBL Operator 751 If removals cannot be automated (e.g., via robot re-testing or self- 752 removal) then the DNSBL SHOULD have multiple administrators so that a 753 removal request can be processed if the principal list administrator 754 is on vacation or otherwise unavailable. 756 3.8. Protect Against Misconfiguration/Outages 758 It is not altogether uncommon for DNSBL users to configure their 759 systems improperly for DNSBL queries. The consequences of an error 760 can range from undue (or even damaging) load on the DNSBL servers, to 761 accidentally blocking all incoming email. 763 DNSBL users MUST test their initial DNSBL configurations to ensure 764 that they're working correctly, and SHOULD periodically recheck the 765 status of the DNSBLs they use and adjust their configuration as 766 necessary. 768 Common types of misconfigurations include: 770 1. Using wrong (sub-)zones for querying (e.g. 4.3.2.1.example.com or 771 4.3.2.1.dnsbl.exmple.cm instead of 4.3.2.1.dnsbl.example.com). 773 2. Downloading a local mirror of the data, but failing to set up the 774 local nameserver infrastructure appropriately, and thus 775 continuing to query the public nameservers. 777 3. Downloading a local mirror of the data, but misconfiguring the 778 local nameserver infrastructure to query a locally invented zone 779 name (4.3.2.1.dnsbl.local) at the public nameservers. 781 4. Misconfigured local nameservers to not do meaningful caching, 782 thus heavily increasing load on the public nameservers. 784 5. Using the DNSBL query root domain name as the name server for 785 queries. 787 6. Using the DNSBL incorrectly; e.g. Some DNSBLs are suitable only 788 for certain types of filtering. Improper use may result in 789 excessive incorrect filtering. 791 While in many cases, it can be difficult to detect such situations, 792 to protect against such misconfiguration it is RECOMMENDED that DNSBL 793 operators make design decisions to mitigate the impact of such 794 mistakes, and make efforts to contact administrative contacts to 795 remedy the situation where appropriate. But the DNSBL operator 796 SHOULD also prepare to take appropriate steps to protect the 797 operational infrastructure (e.g., have the ability to block abusive 798 users from causing further damage). 800 Appropriate use of the DNSBL (e.g. email, not IRC, not against 801 authenticated local users) SHOULD be documented on the web site. 803 4. Security Considerations 805 Any system manager that uses DNSBLs is entrusting part of his or her 806 server management to the parties that run the lists. A DNSBL manager 807 that decided to list 0/0 (which has actually happened) could cause 808 every server that uses the DNSBL to reject all mail. Conversely, if 809 a DNSBL manager removes all of the entries (which has also happened), 810 systems that depend on the DNSBL will find that their filtering 811 doesn't work as they want it to. 813 If a registered domain name used for a DNSBL is allowed to lapse, or 814 the DNSBL user spells the DNSBL domain name incorrectly, the system 815 manager's server management is now subject to an entirely different 816 party than was intended. Further, even if there is no malicious 817 intent, some DNSBL query clients will interpret any A record being 818 returned as being listed. DNSBL users SHOULD be prepared to 819 periodically test the DNSBLs they use for correct operation. 821 Like all DNS-based mechanisms, DNSBLs are subject to various threats 822 outlined in [RFC3833]. 824 5. IANA Considerations 826 This document has no actions for IANA. [This section may be removed 827 before publishing as an RFC.] 829 6. References 831 6.1. Normative References 833 [RFC1918] Rekhter, Y., Moskowitz, R., Karrenberg, D., Groot, G., and 834 E. Lear, "Address Allocation for Private Internets", 835 BCP 5, RFC 1918, February 1996. 837 [RFC2014] Weinrib, A. and J. Postel, "IRTF Research Group Guidelines 838 and Procedures", BCP 8, RFC 2014, October 1996. 840 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 841 Requirement Levels", BCP 14, RFC 2119, March 1997. 843 [RFC3927] Cheshire, S., Aboba, B., and E. Guttman, "Dynamic 844 Configuration of IPv4 Link-Local Addresses", RFC 3927, 845 May 2005. 847 6.2. Informative References 849 [BIND] Internet Systems Corporation, "ISC BIND", 850 . 852 [DNSBL-EMAIL] 853 Levine, J., "DNS-based blacklists and Whitelists for 854 E-Mail", November 2008, . 857 [RFC3330] IANA, "Special-Use IPv4 Addresses", RFC 3330, 858 September 2002. 860 [RFC3833] Atkins, D. and R. Austein, "Threat Analysis of the Domain 861 Name System (DNS)", RFC 3833, August 2004. 863 [RFC5156] Blanchet, M., "Special-Use IPv6 Addresses", RFC 5156, 864 April 2008. 866 Appendix A. Acknowledgements 868 We would like to thank John R. Levine, Alan Murphy and Dave Crocker 869 for their insightful comments. 871 We would also like to thank Yakov Shafranovich and Nick Nicholas for 872 editing previous versions of this document. 874 Authors' Addresses 876 Chris Lewis 877 Nortel Networks 879 Email: clewis@nortel.com 881 Matt Sergeant 882 MessageLabs, Inc 884 Email: msergeant@messagelabs.com 886 Full Copyright Statement 888 Copyright (C) The IETF Trust (2008). 890 This document is subject to the rights, licenses and restrictions 891 contained in BCP 78, and except as set forth therein, the authors 892 retain all their rights. 894 This document and the information contained herein are provided on an 895 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 896 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND 897 THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS 898 OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 899 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 900 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 902 Intellectual Property 904 The IETF takes no position regarding the validity or scope of any 905 Intellectual Property Rights or other rights that might be claimed to 906 pertain to the implementation or use of the technology described in 907 this document or the extent to which any license under such rights 908 might or might not be available; nor does it represent that it has 909 made any independent effort to identify any such rights. Information 910 on the procedures with respect to rights in RFC documents can be 911 found in BCP 78 and BCP 79. 913 Copies of IPR disclosures made to the IETF Secretariat and any 914 assurances of licenses to be made available, or the result of an 915 attempt made to obtain a general license or permission for the use of 916 such proprietary rights by implementers or users of this 917 specification can be obtained from the IETF on-line IPR repository at 918 http://www.ietf.org/ipr. 920 The IETF invites any interested party to bring to its attention any 921 copyrights, patents or patent applications, or other proprietary 922 rights that may cover technology that may be required to implement 923 this standard. Please address the information to the IETF at 924 ietf-ipr@ietf.org.