idnits 2.17.1 draft-dkg-openpgp-pgpmime-message-mangling-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (May 22, 2019) is 1795 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Looks like a reference, but probably isn't: '0' on line 455 -- Looks like a reference, but probably isn't: '1' on line 450 -- Looks like a reference, but probably isn't: '2' on line 452 == Missing Reference: '-1' is mentioned on line 456, but not defined -- Obsolete informational reference (is this intentional?): RFC 7601 (Obsoleted by RFC 8601) Summary: 0 errors (**), 0 flaws (~~), 3 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 openpgp D. Gillmor 3 Internet-Draft ACLU 4 Intended status: Informational May 22, 2019 5 Expires: November 23, 2019 7 Common PGP/MIME Message Mangling 8 draft-dkg-openpgp-pgpmime-message-mangling-00 10 Abstract 12 An e-mail message with OpenPGP encryption and/or signature has 13 standard form. However, some mail transport agents damage those 14 forms in transit. A user-friendly mail user agent that encounters 15 such a mangled message may wish to try to process it anyway, despite 16 the message's non-standard structure. 18 This document aims to be a sort of bestiary of common message 19 mangling patterns, along with guidance for implementers for how to 20 cope with messages structured in these common ways. 22 Status of This Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at https://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on November 23, 2019. 39 Copyright Notice 41 Copyright (c) 2019 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (https://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 57 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 58 1.2. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 59 2. Problem Statement . . . . . . . . . . . . . . . . . . . . . . 3 60 3. Cryptographic Properties . . . . . . . . . . . . . . . . . . 4 61 4. Encrypted Messages . . . . . . . . . . . . . . . . . . . . . 4 62 4.1. "Mixed Up" Encryption . . . . . . . . . . . . . . . . . . 4 63 4.1.1. Detection of "Mixed Up" Encryption . . . . . . . . . 5 64 4.1.2. Repair of "Mixed Up" Encryption . . . . . . . . . . . 5 65 5. Clearsigned Messages . . . . . . . . . . . . . . . . . . . . 5 66 5.1. Extra MIME footer . . . . . . . . . . . . . . . . . . . . 6 67 5.2. Content-Re-Encoding . . . . . . . . . . . . . . . . . . . 6 68 6. Performance Considerations . . . . . . . . . . . . . . . . . 6 69 7. Security Considerations . . . . . . . . . . . . . . . . . . . 6 70 7.1. Only Repair for Cryptographic Improvement . . . . . . . . 6 71 7.2. Avoiding Another E-Fail . . . . . . . . . . . . . . . . . 6 72 7.3. Do Not Attempt Repair Inside End-to-End Encryption . . . 7 73 7.4. Interactions with DKIM Verification . . . . . . . . . . . 7 74 7.5. "Helpful" MTA mangling . . . . . . . . . . . . . . . . . 7 75 8. Privacy Considerations . . . . . . . . . . . . . . . . . . . 7 76 9. User Considerations . . . . . . . . . . . . . . . . . . . . . 8 77 9.1. Indications of Mangling Can Be Confusing . . . . . . . . 8 78 9.2. Repair Indicators . . . . . . . . . . . . . . . . . . . . 8 79 9.3. Undoing Repair . . . . . . . . . . . . . . . . . . . . . 8 80 9.4. Attempting Decryption During Repair . . . . . . . . . . . 8 81 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8 82 11. Document Considerations . . . . . . . . . . . . . . . . . . . 8 83 12. Example Messages . . . . . . . . . . . . . . . . . . . . . . 9 84 12.1. Example Input Encrypted Message . . . . . . . . . . . . 9 85 12.2. Input Message Mangled by "Mixed Up" . . . . . . . . . . 9 86 13. Example Implemenations . . . . . . . . . . . . . . . . . . . 10 87 13.1. Example: Detecting "Mixed Up" Encryption . . . . . . . . 10 88 13.2. Example: Repairing "Mixed Up" Encryption . . . . . . . . 11 89 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 90 14.1. Normative References . . . . . . . . . . . . . . . . . . 11 91 14.2. Informative References . . . . . . . . . . . . . . . . . 12 92 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 12 94 1. Introduction 96 1.1. Requirements Language 98 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 99 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 100 "OPTIONAL" in this document are to be interpreted as described in BCP 101 14 [RFC2119] [RFC8174] when, and only when, they appear in all 102 capitals, as shown here. 104 1.2. Terminology 106 o "Mail User Agent" (or "MUA") refers to a program designed to send 107 and receive e-mail. Common examples include Thunderbird, Outlook, 108 and Mail.app. Webmail implementations, like Roundcube or the 109 Gmail interface can also be considered a MUA. 111 o "Mail Transport Agent" (or "MTA") refers to mail-handling software 112 within the network. Common examples include Postfix and Exchange. 114 o "PGP/MIME" is the message structure for OpenPGP protections 115 described in [RFC3156]. 117 o "encrypted message" is used to mean any message where the 118 outermost PGP/MIME cryptographic feature of the message itself is 119 encryption. Some encrypted messages might be signed, but the 120 inner signature cannot typically be mangled by an "MTA" that can't 121 decrypt. 123 o "clearsigned message" is used here to refer specifically to 124 messages that are not encrypted, but are signed with a PGP/MIME 125 signature. 127 o "signed message" is used to refer to a message that has a valid 128 OpenPGP cryptographic signature applied by the message sender, 129 whether the message is encrypted or not. 131 2. Problem Statement 133 Some MTAs modify message headers, bodies, and structure in transit. 134 Some of those modifications ("manglings") can transform a message in 135 such a way that the end-to-end cryptographic properties of the 136 message are lost. 138 Clearsigned messages can become unverifiable, and encrypted messages 139 can become impossible to decrypt. MUAs that receive these mangled 140 messages are unable to fulfil their users goals because of damage 141 done to the message by the mangling MTA. 143 In some cases, those manglings can take a regular enough form that a 144 clever MUA can "repair" them, restoring the cryptographic properties 145 of the message for the user. 147 This document aims to collect common manglings, and document how an 148 MUA can detect them and (if possible) repair them safely for the 149 user. 151 3. Cryptographic Properties 153 TODO: describe message-wide cryptographic envelope vs. cryptographic 154 payload and their relevance to this draft. 156 TODO: we're assuming a single layer of cryptographic protection. 157 These steps get more complicated if there are multiple nested layers 158 (encrypted and then signed? signed and then encrypted? triple- 159 wrapped?). Should this document address that situation? 161 4. Encrypted Messages 163 This section aims to collect examples of repairable mangling known to 164 be performed by some currently-active MTA on encrypted messages. 165 Each section offers a brief description of the mangling, a way to 166 detect it, and a proposed method of repair. 168 4.1. "Mixed Up" Encryption 170 One common mangling takes an incoming multipart/encrypted e-mail and 171 transforms it into a multipart/mixed e-mail, shuffling body parts at 172 the same time. 174 An encrypted message typically has this clean structure C (see 175 example in Section 12.1): 177 C multipart/encrypted 178 C.1 application/pgp-encrypted 179 C.2 application/octet-stream 181 But after processing, it has mangled structure M (see example in 182 Section 12.2): 184 M multipart/mixed 185 M.1 text/plain (0 bytes) 186 M.2 application/pgp-encrypted 187 M.3 application/octet-stream 189 Some versions of Microsoft Exchange are known to do this. 191 4.1.1. Detection of "Mixed Up" Encryption 193 Check that all of the following conditions hold: 195 o The message itself (M) is Content-Type: multipart/mixed. 197 o The message has exactly three MIME subparts, all direct children 198 of the message itself. 200 o The first part (M.1) is Content-Type: text/plain, and has 0 bytes. 202 o The second part (M.2) is Content-Type: application/pgp-encrypted, 203 and contains (after removing any Content-Transfer-Encoding) 204 exactly the string "Version: 1" 206 o The third part (M.3) is Content-Type: application/octet-stream. 208 o After removing any Content-Transfer-Encoding, the decoded third 209 part (M.3) is an ASCII-armored OpenPGP block of type "PGP MESSAGE" 210 (see see section 6.2 of [RFC4880]) 212 Please see Section 13.1 for an example implementation. 214 4.1.2. Repair of "Mixed Up" Encryption 216 If the detection is successful, repair can be attempted with the 217 following steps: 219 o Convert the message's Content-Type: value to multipart/encrypted, 220 while leaving any other parameters untouched. 222 o Add (or reset) the parameter protocol=application/pgp-encrypted to 223 the Content-Type: header. 225 o Drop the first sub-part (M.1). 227 Please see Section 13.2 for an example implementation. 229 5. Clearsigned Messages 231 This section aims to collect examples of repairable mangling known to 232 be performed by some currently-active MTA on clearsigned messages. 233 Each section offers a brief description of the mangling, a way to 234 detect it, and a proposed method of repair. 236 5.1. Extra MIME footer 238 TODO: describe the mailman case 240 5.2. Content-Re-Encoding 242 TODO: despite the content of multipart/signed parts being ostensibly 243 invariant (see page 4 of [RFC1847]), some MTAs mangle them. Describe 244 some reversible manglings, like gratuitous application of Content- 245 Transfer-Encoding. 247 6. Performance Considerations 249 Trying multiple repair techniques can be expensive. A resource- 250 constrained MUA may want to limit itself to detection of possible 251 mangling, rather than trying every possible repair it knows of and 252 indicating 254 7. Security Considerations 256 There are several ways that attempting repair on a mangled message 257 can go dangerously wrong. 259 7.1. Only Repair for Cryptographic Improvement 261 If a message repair produces a seemingly-better-structured message, 262 but the repaired message still cannot be decrypted or verified, then 263 the MUA should abandon the repair and instead render the message in 264 its original form. Additional message modification that does not 265 provide a cryptographic advantage over the received form of the 266 message offers no benefit to the user. 268 TODO: is a decryptable, unsigned message "better" than a verifiably 269 signed message? What is "an improvement" isn't necessarily clear 270 here. 272 7.2. Avoiding Another E-Fail 274 Promiscuous message repair may inject new vulnerabilities, or modify 275 a message beyond recognition. An MUA attempting message repair 276 should use caution to avoid re-combining potentially malicious 277 material with material that has better cryptographic guarantees. 279 In particular, a responsible MUA should take care not to claim 280 cryptographic protections over material that does not have them. 282 7.3. Do Not Attempt Repair Inside End-to-End Encryption 284 If a message or part of a message arrived end-to-end encrypted, none 285 of the recommendations in this draft should be read as encouraging 286 their application to any cleartext material within that end-to-end 287 encryption. These techniques are specifically for recovering from 288 damage done by the MTA, which by design does not have access to the 289 cleartext of an end-to-end encrypted message. 291 7.4. Interactions with DKIM Verification 293 A mangling MTA may also add DKIM headers after mangling. A 294 subsequent MTA that handles the message may verify the message by 295 checking DKIM headers, storing the results of that check in an 296 Authentication-Results header (see [RFC7601]). If the MUA performs 297 message repair, the repaired message might not pass DKIM check. An 298 MUA that relies on Authentication-Results headers for any purpose on 299 such a repaired message should consider re-validating the DKIM header 300 itself directly after message repair (though this introduces new 301 problems for stored/old messages, as older DKIM signing keys may no 302 longer be published in the DNS). 304 Note that a repaired message may have cryptographic properties that 305 obviates the need for DKIM verification; an MUA that requires 306 messages to have a valid Authentication-Results: header might waive 307 that requirement for a signed message (whether any repair routine was 308 invoked or not). 310 7.5. "Helpful" MTA mangling 312 An MTA may intend to help the user by modifying the message. For 313 example, in a clearsigned message, the MTA may modify a potentially- 314 malicious URL in the text, or it might change an attachment name or 315 MIME-type in an attachment. If an MUA attempts message repair in a 316 way that reverses these modifications, it may negate whatever 317 security advantages the MTA hoped to offer. 319 It is not clear that this is a risk for repair of an encrypted 320 message, though, since the MTA by design cannot perform such a 321 modification on the cleartext of an encrypted message. 323 8. Privacy Considerations 325 TODO: privacy considerations? 327 9. User Considerations 329 Cryptographic properties of messages are only useful if the user 330 understands them. 332 9.1. Indications of Mangling Can Be Confusing 334 o TODO: about how indicators of mangled messages can cause user- 335 experience confusion (e.g. parts of https://efail.de/ and 336 https://github.com/RUB-NDS/Johnny-You-Are-Fired ) 338 9.2. Repair Indicators 340 o TODO: how should a repairing MUA indicate that a repair has 341 happened (or is possible) to the user? 343 9.3. Undoing Repair 345 o TODO: Should a repairing MUA allow the user to reverse the repair 346 and see the broken message? If so, what else should happen if the 347 user takes this action (e.g., how is a subsequent "Reply" or 348 "Forward" action affected?) 350 9.4. Attempting Decryption During Repair 352 Attempting decryption may require access to secret key material, 353 which itself may cause interaction with the user. If multiple 354 repairs are attempted, each attempt to decrypt may require multiple 355 decryptions. In some scenarios, multiple use of the secret key may 356 be annoying to the user. It might be preferable to try to decrypt 357 the apparent underlying block of text exactly once, regardless of 358 message structure or repair, and if successful, either stash the 359 session key, or store the cleartext in memory, while manipulating 360 exterior structure. 362 10. IANA Considerations 364 This document requires no actions from IANA. 366 11. Document Considerations 368 [ RFC Editor: please remove this section before publication ] 370 This document is currently edited as markdown. Minor editorial 371 changes can be suggested via merge requests at 372 https://gitlab.com/dkg/draft-openpgp-pgpmime-message-mangling or by 373 e-mail to the author. Please direct all significant commentary to 374 the public IETF OpenPGP mailing list: openpgp@ietf.org 376 12. Example Messages 378 12.1. Example Input Encrypted Message 380 From: Michael Elkins 381 To: Michael Elkins 382 Mime-Version: 1.0 383 Content-Type: multipart/encrypted; boundary=foo; 384 protocol="application/pgp-encrypted" 386 --foo 387 Content-Type: application/pgp-encrypted 389 Version: 1 391 --foo 392 Content-Type: application/octet-stream 394 -----BEGIN PGP MESSAGE----- 396 hIwDY32hYGCE8MkBA/wOu7d45aUxF4Q0RKJprD3v5Z9K1YcRJ2fve87lMlDlx4Oj 397 eW4GDdBfLbJE7VUpp13N19GL8e/AqbyyjHH4aS0YoTk10QQ9nnRvjY8nZL3MPXSZ 398 g9VGQxFeGqzykzmykU6A26MSMexR4ApeeON6xzZWfo+0yOqAq6lb46wsvldZ96YA 399 AABH78hyX7YX4uT1tNCWEIIBoqqvCeIMpp7UQ2IzBrXg6GtukS8NxbukLeamqVW3 400 1yt21DYOjuLzcMNe/JNsD9vDVCvOOG3OCi8= 401 =zzaA 402 -----END PGP MESSAGE----- 404 --foo-- 406 12.2. Input Message Mangled by "Mixed Up" 407 From: Michael Elkins 408 To: Michael Elkins 409 Mime-Version: 1.0 410 Content-Type: multipart/mixed; boundary=foo 412 --foo 413 Content-Type: text/plain; charset="us-ascii" 415 --foo 416 Content-Type: application/pgp-encrypted 418 Version: 1 420 --foo 421 Content-Type: application/octet-stream 423 -----BEGIN PGP MESSAGE----- 425 hIwDY32hYGCE8MkBA/wOu7d45aUxF4Q0RKJprD3v5Z9K1YcRJ2fve87lMlDlx4Oj 426 eW4GDdBfLbJE7VUpp13N19GL8e/AqbyyjHH4aS0YoTk10QQ9nnRvjY8nZL3MPXSZ 427 g9VGQxFeGqzykzmykU6A26MSMexR4ApeeON6xzZWfo+0yOqAq6lb46wsvldZ96YA 428 AABH78hyX7YX4uT1tNCWEIIBoqqvCeIMpp7UQ2IzBrXg6GtukS8NxbukLeamqVW3 429 1yt21DYOjuLzcMNe/JNsD9vDVCvOOG3OCi8= 430 =zzaA 431 -----END PGP MESSAGE----- 433 --foo-- 435 13. Example Implemenations 437 These examples use python3. As Code Examples, they are licensed 438 under the Simplified BSD License as noted above. 440 13.1. Example: Detecting "Mixed Up" Encryption 441 import email.message 443 def is_mixed_up(msg: email.message.Message) -> bool: 444 if msg.get_content_type() == 'multipart/mixed': 445 pts = msg.get_payload() 446 if len(pts) == 3 and \ 447 pts[0].get_content_type() == 'text/plain' and \ 448 pts[0].get_payload(decode=True) == b'' and \ 449 pts[1].get_content_type() == 'application/pgp-encrypted' and \ 450 pts[1].get_payload(decode=True).strip() == b'Version: 1' and \ 451 pts[2].get_content_type() == 'application/octet-stream': 452 encpart = pts[2].get_payload(decode=True) 453 # FIXME: this is not a true check for RFC4880 ASCII armor: 454 lines = encpart.replace(b'\r\n', b'\n').split(b'\n') 455 if lines[0] == b'-----BEGIN PGP MESSAGE-----' and \ 456 lines[-1] == b'-----END PGP MESSAGE-----': 457 return True 458 return False 460 13.2. Example: Repairing "Mixed Up" Encryption 462 import copy 463 import email.message 464 import typing 466 def un_mix_up(msg: email.message.Message) -> \ 467 typing.Optional[email.message.Message]: 468 if is_mixed_up(msg): 469 ret = copy.deepcopy(msg) 470 ret.set_type('multipart/encrypted') 471 ret.set_param('protocol', 'application/pgp-encrypted') 472 ret.set_payload(msg.get_payload()[1:]) 473 return ret 474 return None 476 14. References 478 14.1. Normative References 480 [RFC1847] Galvin, J., Murphy, S., Crocker, S., and N. Freed, 481 "Security Multiparts for MIME: Multipart/Signed and 482 Multipart/Encrypted", RFC 1847, DOI 10.17487/RFC1847, 483 October 1995, . 485 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 486 Requirement Levels", BCP 14, RFC 2119, 487 DOI 10.17487/RFC2119, March 1997, 488 . 490 [RFC3156] Elkins, M., Del Torto, D., Levien, R., and T. Roessler, 491 "MIME Security with OpenPGP", RFC 3156, 492 DOI 10.17487/RFC3156, August 2001, 493 . 495 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 496 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 497 May 2017, . 499 14.2. Informative References 501 [RFC4880] Callas, J., Donnerhacke, L., Finney, H., Shaw, D., and R. 502 Thayer, "OpenPGP Message Format", RFC 4880, 503 DOI 10.17487/RFC4880, November 2007, 504 . 506 [RFC7601] Kucherawy, M., "Message Header Field for Indicating 507 Message Authentication Status", RFC 7601, 508 DOI 10.17487/RFC7601, August 2015, 509 . 511 Author's Address 513 Daniel Kahn Gillmor 514 American Civil Liberties Union 515 125 Broad St. 516 New York, NY 10004 517 USA 519 Email: dkg@fifthhorseman.net