Internet Draft M. Stout VB NetConsult November 2003 Anti-Spam Recommendations II for SMTP MTAs draft-stout-antispam-00.txt Status of this Memo This memo provides information for the Internet community. It does not specify an Internet standard of any kind. Distribution of this memo is unlimited. This document is an Internet-Draft and is subject to all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html Copyright Notice Copyright (C) The Internet Society (2003). All Rights Reserved. Abstract This memo gives an implementation recommendation for SMTP [1] MTAs (Mail Transfer Agents) to make them more capable of reducing the impact of spam(*). Although this recommendation is not the 'final solution', if applied on enough SMTP MTAs on the Internet, the recommendation will force spammers into the open, where they can be dealt with. A brief summary of the recommendation is: o Register all MTAs within DNS o Verify the combination of sender and sending MTA Stout Internet Draft [Page 1] Anti-Spam Recommendations II November 2003 1. Introduction The following terms will be used: Receiving MTA: this is the host system, setup as a mail server, that will act as the receiver of a message being transferred via (E-)SMTP. Sending MTA: this is the host system that tries to deliver an e-mail message on behalf of the Sender. Sender: FQDN (Fully qualified domain name) that is stated to be the sender of the e-mail message, as provided via the 'MAIL FROM' part of the (E-)SMTP dialogue (sender@domain.name). Mail servers exchange e-mail with each other by way of the (E-)SMTP protocol [1]. For any message to be received, the Receiving MTA will acquire the following details during connection setup and the SMTP dialogue [2]: o Sending MTA's IP address o Sending MTA's host name o Sender of the message (as provided via the 'MAIL FROM') Some steps have been suggested in earlier anti-SPAM recommendations [3], and these are in operation at some or most sites: a. Perform a reverse-DNS lookup (PTR DNS request) on the IP address of the Sending MTA. Some mail systems refuse to receive a message if no reverse-DNS information is available. b. Check the supplied host name of the Sending MTA with that of the result of reverse-DNS lookup. Many mail systems react to a mismatch with information within the dialogue. This does not solve anything. c. Verify the validity of the Sender of the message. This means that either DNS is checked for the existence of a mailbox or the MX-servers for the Sender's domain name are checked for the existence of the Sender. Unfortunately many spammers abuse valid e-mail addresses, so checking will not help. Other steps have been implemented in some systems, for instance to check an IP address or host name against black-lists that are maintained at different locations. Black lists are however very demanding and allow for error. Stout Internet Draft [Page 2] Anti-Spam Recommendations II November 2003 After an extensive analysis the conclusions to determine if you are spammed are: 1. There can be no conclusion based on the existence of a PTR-record for a Sending MTA's IP address. 2. There can be no conclusion based on a (mis-)match between the supplied Sending MTA's host name and the reverse-DNS name (PTR). 3. There can be no conclusion based on the fact that the Sender's e-mail address is valid. It is the combination of the Sender and the Sending MTA that has to be analysed. ******************************************************************** The real problem any Receiving MTA is facing is that there is currently no way to verify if the Sending MTA is a valid host for sending a particular message. (if there is, then please let me know) ******************************************************************** So, when a Sending MTA contacts a Receiving MTA, the receiver has no technical way to verify that the Sending MTA is indeed part of the technical infrastructure of the e-mail message's Sender. (Is this host a valid host for trying to send this message from that Sender to me?). Using DNS to obtain the MX-records of the Sender's domain name does not help most of the time, because many mail systems use different host systems for sending and receiving of messages. And only the receiving host systems will be registered within DNS as MX records. Stout Internet Draft [Page 3] Anti-Spam Recommendations II November 2003 2. Solution 2.1 Register all mail servers If - at any Receiving MTA - we can technically verify that the Sending MTA is a valid host for the Sender, then: a. a spammer would have to operate in the 'open' where he/she can be dealt with. b. we can discard any message where this match does not exist. ****************************************************************** The solution is to require from all domain names to register ALL their mail servers (host names) within the DNS MX records. Send-only mail servers MUST also be registered in the MX records, and they should be given the lowest possible MX priority (65535). ****************************************************************** Domains that do not comply with the requirement to register all their mail servers, will risk not being able to send messages via unregistered mail servers. 2.2 Verification process at the Receiving MTA During a SMTP session, for any Receiving MTA the Sender/Sending-MTA verification procedure is then to: 1. determine the Sender's domain name (everything after the '@' as provided with the MAIL FROM). 2. determine all valid mail servers for the Sender's domain name, which means obtain a list of all MX records via DNS. 3. check the IP address of the Sending MTA against the obtained list of valid mail servers. 4. if the IP address is not present, then discard the message. 5. if the IP address is present, then assume a valid message. Further checking may of course still be applied, e.g. to check for incoming virusses or filter specific content etc, but that should be dealt with at a later time by the mail server or the receiving client's applications involved. Stout Internet Draft [Page 4] Anti-Spam Recommendations II November 2003 2.3 Sending MTA Any Sending MTA should understand the special status of MX priority of 65535, and should never try to contact these hosts in order to try to deliver an e-mail message. However, older systems that do not understand the special status of MX priority 65535 will unsuccessfully try to contact these hosts. 2.4 Fall back mail server Special consideration is to be given to the Receiving MTA's possible fall back mail servers. If the fall back server acts accordingly to the suggested way of working, then messages would be filtered already by the time the fall back server tries to deliver remaining messages to the primary mail server. The primary mail server should therefore provide special status (clearance) to its fall back servers and not perform the verification process again. A Receiving MTA should be configurable to know which fall back servers are defined. This could be generalised by the Receiving MTA by obtaining a list of MX records for its own domain name(s). Any host with a lower priority than itself should then be provided a 'special' clearance. Any host with a higher priority will not normally make a connection, but may be set to special status anyway. Stout Internet Draft [Page 5] Anti-Spam Recommendations II November 2003 3. Conclusion If this way of working would be adopted, then this will eliminate: a. the necessity of PTR lookups of the Sending MTA's IP address b. the use of (external) spam black lists c. verifying the Sender's e-mail address. ad a. The system's name does not matter at all. ad b. If a Receiving MTA wishes to maintain its own black lists or apply content filters, that remains to be at the discretion of the system's administrator. ad c. Since the Sender will be part of the Sending MTA's domain, one can assume a valid Sender. There is no need to change anything to DNS itself. The proposed way of working is backwards compatible. Nothing changes for systems that do not have their software updated yet. In the worst case Sending MTAs will try to connect to send-only mail servers, but that will inflict no real harm. For this proposed solution to work it will need support from a considerable group of important mail systems, such as hotmail, AOL and Yahoo (**). If these mail systems would adopt the suggested way of working, then all other mail systems will quickly follow to have all their mail systems registered, so to ensure that one is still able to get a message sent. Stout Internet Draft [Page 6] Anti-Spam Recommendations II November 2003 4. Examples Please note that the following examples are in NO way intended to harm or discredit any person or organisation. Also note that the provided info may be subject to change. 4.1 hotmail.com A user from 'hotmail.com' sends a message to someone outside of the 'hotmail.com' domain. One of the (many) Sending MTAs for 'hotmail.com' will pick up the message and contact the appropriate Receiving MTA. Examples of mail servers that act as Sending MTAs for 'hotmail.com', the names are determined via PTR lookup of the IP address: 65.54.173.21 bay5-f21.bay5.hotmail.com 65.54.173.40 bay5-f40.bay5.hotmail.com 64.4.11.58 bay7-f58.bay7.hotmail.com 64.4.11.82 bay7-f82.bay7.hotmail.com The fact that these servers have a name relative to the 'hotmail.com' domain may not be coincidence, but this is unfortunately not true for many examples that could be given. A Receiving MTA will be contacted by a mail server that identifies itself as 'hotmail.com', but reverse DNS of the Sending MTA's IP address will possibly show one of the above mentioned names. MX info of the 'hotmail.com' domain (= mail servers that act as Receiving MTAs for 'hotmail.com'): 5 mx1.hotmail.com 65.54.252.99 64.4.50.99 5 mx2.hotmail.com 65.54.254.145 65.54.252.230 5 mx3.hotmail.com 65.54.167.5 64.4.50.239 5 mx4.hotmail.com 65.54.253.230 65.54.254.151 As is not so clear in this example, but true nonetheless, none of the Sending MTAs exist as MX record within DNS for 'hotmail.com'. A Receiving MTA has therefore no way of verifying that the Sending MTA is part of the 'hotmail.com' infrastructure. We could in this example determine that the Sending MTA is part of the IP-network for 'hotmail.com', i.e. that they either belong to '64.4.0.0' or '65.54.0.0'. However, in many other possible examples this rule would not be applicable. Stout Internet Draft [Page 7] Anti-Spam Recommendations II November 2003 4.2 vb.net A user from 'vb.net' sends a message to someone outside of the 'vb.net' domain. The mail server (there is only one) will pick up the message and contact the appropriate Receiving MTA. The mail server that acts as Sending MTA for 'vb.net': 80.127.133.149 a80-127-133-149.adsl.xs4all.nl A Receiving MTA will be contacted by a mail server that identifies itself as 'smtp.vb.net', but reverse DNS of the Sending MTA's IP address will show the above mentioned name. MX info of the 'vb.net' domain (= mail servers that act as Receiving MTAs for 'vb.net'): 10 smtp.vb.net 80.127.133.149 As is clear in this example, when a Receving MTA is contacted by the host 'smtp.vb.net' at '80.127.133.149' with a message from someone within the 'vb.net' domain, the verification process shows an easy match between the Sending MTA and the one registered mail server. Stout Internet Draft [Page 8] Anti-Spam Recommendations II November 2003 5. References [1] Postel, J., "Simple Mail Transfer Protocol", STD 10, RFC 821, August 1982. [2] Crocker, D., "Standard for the format of ARPA Internet text messages", STD 11, RFC 822, August 1982. [3] Lindberg, G., "Anti-Spam Recommendations for SMTP MTAs", BCP, RFC 2505, February 1999. * Spam (R) (capitalized) is a registered trademark of a meat product made by Hormel. Use of the term spam (uncapitalized) in the Internet community comes from a Monty Python sketch and is almost Internet folklore. The term spam is usually pejorative, however this is not in any way intended to describe the Hormel product. ** hotmail, AOL and Yahoo are registered trademarks of their respective holders. 6. Contact information Marthin Stout Havenstraat 39 2652 BR BERKEL EN RODENRIJS NETHERLANDS Tel: +31-10-5190528 Fax: +31-10-5199449 E-mail: stout@vb.net Stout Internet Draft [Page 9] Anti-Spam Recommendations II November 2003 Full Copyright Statement Copyright (C) The Internet Society (2003). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Stout Internet Draft [Page 10]