[Spasm] Chunking ciphertexts

Adam Langley <agl@imperialviolet.org> Fri, 02 September 2016 00:22 UTC

MIME-Version: 1.0
Sender: alangley@gmail.com
From: Adam Langley <agl@imperialviolet.org>
Date: Thu, 01 Sep 2016 17:22:18 -0700
Message-ID: <CAMfhd9VyNzoR1F8zUaxqni0RAD=n3VtPF9ku0phkod2j+CBD+Q@mail.gmail.com>
To: spasm@ietf.org
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/spasm/QKbdr01z4dq20T7aKdlfDNVHq_c>
Subject: [Spasm] Chunking ciphertexts
Precedence: list

Since the WG is currently considering changes to S/MIME, I'd like to
mention the benefits of chunking ciphertexts. (Or, rather, the
problems with not chunking them.)

As I understand it, S/MIME and associated standards typically encrypt
payloads as a single ciphertext (i.e. RFC 5084). Since the plaintexts
might be quite large, this encourages implementations to decrypt these
ciphertexts in a "streaming" fashion, i.e. they return plaintext
before it's authenticated.

Encouraging APIs that return unauthenticated plaintext is an ecosystem
cost since such APIs are dangerous. One similarity is with the
MAC-then-encrypt constructions used in TLS, which caused many problems
by processing unauthenticated ciphertext. At a higher level, it's not
hard to find people on the internet suggesting things like:

gpg -d your_archive.tgz.gpg | tar xz

because the gpg command also provides a "streaming" interface.
(http://askubuntu.com/questions/95920/encrypt-tar-gz-file-on-create)

With constructions like AES-GCM it's, of course, very easy to
manipulate the plaintext at will if the application doesn't
authenticate it before processing. Even if the application is careful
not to "release" plaintext to the UI until the authenticity has been
established, a streaming design exposes more program attack surface.

By normalising large ciphertexts and thus streaming APIs, the next
protocol that comes along is more likely to use them without realising
the issues and thus the problem persists.


Preferably, plaintext inputs would be chunked into reasonably large
parts (say 16KiB) and encrypted separately. The chunks only need to be
large enough that the overhead from the additional authenticators is
negligible. With such a design, large messages can be incrementally
processed without having to deal with unauthenticated plaintext, and
AEAD APIs can be safer. (Not to mention that larger messages can be
processed since AES-GCM, for one, has a ~64GiB limit for a single
plaintext.)

Some thought is needed to ensure that the chunks are in the correct
order (i.e. by counting nonces), that the first chunk should be first
(i.e. by starting the nonce at zero) and that the last chunk should be
last (i.e. by appending an empty, terminator chunk with special
additional data). But that's not hard.


Even with such a design it's still the case that an attacker can cause
the message to be detectably truncated. If you want to aim higher, an
all-or-nothing transform can be used, although that requires two
passes over the input and I understand if it's sufficiently
problematic that you don't want to do that.


Cheers

AGL

-- 
Adam Langley agl@imperialviolet.org https://www.imperialviolet.org

[Spasm] Chunking ciphertexts Adam Langley
Re: [Spasm] Chunking ciphertexts Wei Chuang
Re: [Spasm] Chunking ciphertexts Russ Housley
Re: [Spasm] [Curdle] Chunking ciphertexts Russ Housley
Re: [Spasm] Chunking ciphertexts Jim Schaad
Re: [Spasm] Chunking ciphertexts Adam Langley
Re: [Spasm] Chunking ciphertexts Jim Schaad
Re: [Spasm] Chunking ciphertexts Adam Langley