ba64: A Binary-to-Text Encoding That Is Never Larger Than Base64

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 17 February 2027.¶

Copyright Notice

Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved.¶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document.¶

Table of Contents

1. Introduction

Base64 [RFC4648] expands binary data by 33%. Applications that embed binary in JSON, cookies, logs, or size-limited message channels pay that tax even when the underlying bytes are highly compressible. Ad hoc solutions (gzip-then-base64) fork the wire format and lose base64 compatibility.¶

ba64 addresses this with a single rule: emit compressed output only when it is strictly shorter than plain base64. The result is an encoding that is never larger than base64, is byte-identical to base64 in the common (incompressible) case, and is unambiguously self-describing.¶

The emitted length depends on the compressibility of the input, which exposes the compression side channel of [CRIME], summarized for TLS in [RFC7457], where an adversary controls part of an input carried alongside secret data. Section 6 defines an OPTIONAL padding method that decouples the emitted length from compressibility, Section 10 specifies the policy for selecting a pad, and Section 12 states the limits of the mitigation.¶

This document specifies version 1 of the ba64 format. Version 1 is intended to be frozen: evolution occurs only through new method identifiers assigned by a future revision.¶

2. Conventions and Definitions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶

Throughout, "base64" means canonical base64 as defined in [RFC4648], Section 4: the alphabet A-Z a-z 0-9 + /, "=" padding REQUIRED, no whitespace, no other characters, and the unused trailing bits of the final symbol set to zero. There is no URL-safe variant of ba64 (Section 14).¶

3. Encoded Forms

A ba64 text is an ASCII string in exactly one of two forms:¶

plain form:       base64(input)        ; byte-identical to base64
compressed form:  "=" + base64(frame)

Dispatch is by the first character: "=" selects the compressed form; anything else (including the empty string, which decodes to zero bytes) selects the plain form.¶

4. Frame Layout

frame = version || method || decoded_len || crc32 || payload

offset  size       field        value
0       1 byte     version      0x01
1       1 byte     method       see below
2       1-9 bytes  decoded_len  unsigned LEB128
...     4 bytes    crc32        CRC-32 of DECODED bytes, LE
...     rest       payload      method-specific compressed data

5. Method 0x01 - Raw DEFLATE

The payload is exactly one complete raw DEFLATE stream per [RFC1951], with no zlib or gzip wrapper. Normative outcomes:¶

• Inflater produces MORE than decoded_len bytes: abort at the cap, E_LENGTH_MISMATCH.¶
• Stream completes with FEWER than decoded_len bytes: E_LENGTH_MISMATCH.¶
• Payload exhausted before the final block completes, or malformed: E_PAYLOAD.¶
• Bytes remain after the final block: E_PAYLOAD.¶

6. Method 0x02 - Raw DEFLATE With Padding

Method 0x02 is method 0x01 with one additional header field and a trailing pad. It allows an encoder to select the emitted text length independently of the compressibility of the input (Section 10, Section 12).¶

frame = version || 0x02 || decoded_len || crc32
                       || pad_len || payload || pad

• pad_len is an unsigned LEB128 under the same constraints as decoded_len: minimally encoded, at most 9 bytes; violations yield E_HEADER.¶
• pad is exactly pad_len bytes at the end of the frame. If fewer than pad_len bytes remain after pad_len itself, the result is E_TRUNCATED.¶
• payload is everything between pad_len and the pad: one complete raw DEFLATE stream, judged by the Section 5 rules. The pad is removed BEFORE inflating and is never fed to the inflater.¶
• Encoders MUST write zero bytes for the pad. Decoders MUST NOT inspect the pad's content and MUST NOT reject a frame because of it. Pad content is therefore not integrity-protected: the CRC covers the decoded bytes only.¶
• pad_len = 0 is valid, and is grammatically distinct from a method-0x01 frame carrying the same payload.¶

A padded frame occupies the length its encoder selected. Padding therefore trades the compression saving for length uniformity.¶

7. Decoding Algorithm

Input: string s, caller limit max_decoded_len (default 67,108,864). Checks MUST run in this order, so any invalid input yields one deterministic code:¶

 1. s does not start with "=": canonical-base64-decode s
      (fail -> E_BASE64); return the bytes ("" -> empty bytes).
 2. canonical-base64-decode s[1:]                fail -> E_BASE64
 3. read version (1 byte)  missing -> E_TRUNCATED; !=0x01 -> E_VERSION
 4. read method  (1 byte)  missing -> E_TRUNCATED; unknown -> E_METHOD
 5. read decoded_len (LEB128)
      exhausted -> E_TRUNCATED; non-minimal or >9 bytes -> E_HEADER
 6. decoded_len > max_decoded_len  -> E_LIMIT_EXCEEDED
      (MUST precede any allocation proportional to decoded_len)
 7. read crc32 (4 bytes)    missing -> E_TRUNCATED
7b. method 0x02 only: read pad_len (LEB128)
      exhausted -> E_TRUNCATED; non-minimal or >9 bytes -> E_HEADER
      fewer than pad_len bytes remain -> E_TRUNCATED
      trim the trailing pad_len bytes; the payload is what is left
 8. inflate payload with hard output cap = decoded_len
      (a cap of zero still caps)  -> E_PAYLOAD / E_LENGTH_MISMATCH
 9. CRC32(output) != crc32  -> E_CHECKSUM
10. return output

E_TRUNCATED refers only to the fixed header fields (steps 3-7b); truncation inside the DEFLATE stream is E_PAYLOAD. A decoder MUST NOT reject a frame merely because a conforming encoder would have chosen the plain form: decoders validate the grammar, not encoder optimality.¶

8. Error Taxonomy

Conformance means the exact code, not merely "an error."¶

Implementation note: the boundary between E_PAYLOAD and E_LENGTH_MISMATCH for a truncated DEFLATE stream depends on the underlying inflate library (some treat premature end of input as end of stream). Both codes reject the frame and the CRC still prevents silently-wrong output; callers SHOULD treat the two as equivalent "corrupt frame" outcomes.¶

9. Encoding Requirements

1. Output MUST be one of the two forms, using canonical base64 throughout.¶
2. Floor rule: the compressed form MUST be emitted only if its final text length is strictly less than the plain form's; ties take plain. Consequently len(ba64(x)) <= len(base64(x)) for every input.¶
3. decoded_len MUST equal the true input length, minimally encoded; crc32 MUST be over the input bytes.¶
4. An encoder MAY skip compression entirely. An encoder that always emits plain base64 is degenerate but fully conforming.¶
5. DEFLATE level is the encoder's choice (SHOULD default to 6). Encoded output is therefore NOT canonical: the same input may yield different valid texts. Any equality, deduplication, cache-keying, or MAC comparison MUST operate on the decoded bytes, never on ba64 text.¶
6. Padding is OPTIONAL: an encoder MAY emit method 0x02 (Section 10). The floor rule still binds. A decoder MUST accept method 0x02 whether or not the encoder ever produces it.¶

10. The Padding Policy

Let n be the input length and P = 4*ceil(n/3) the plain-form length. A compressed text is 1 + 4k characters and is therefore congruent to 1 modulo 4, while P is congruent to 0 modulo 4. A compressed text can never equal the plain-form length, and the greatest length the floor rule permits is P - 3.¶

The RECOMMENDED policy takes one parameter, a quantum Q, a positive multiple of 4 in characters:¶

1. Build the method-0x02 frame with pad_len = 0; call its text length L0.¶
2. Target T = Q*ceil((L0 - 1)/Q) + 1, clamped down to P - 3.¶
3. If T < L0 the compressed form cannot fit under the floor: emit the plain form.¶
4. Otherwise pad the frame to F = 3*(T - 1)/4 bytes. Frame lengths F, F - 1 and F - 2 all encode to T characters, so an encoder that cannot hit F exactly (the pad_len varint changes size as the pad grows) uses whichever of the three it can reach.¶

Q = 4 pads by at most three bytes. A quantum greater than or equal to P yields P - 3 characters for every input that fits under the floor rule, so the emitted length is then a function of n alone.¶

One bit remains observable at every quantum: whether the input compressed enough to fit under the floor rule. An encoder that must not leak that bit uses the always-plain mode, whose length is a function of n alone.¶

11. Limits and Resource Safety

Decoders MUST enforce max_decoded_len before any allocation proportional to the claimed size (step 6) and MUST cap inflation at decoded_len (step 8). A cap of zero MUST still cap: some inflate APIs treat an output limit of 0 as unlimited, so a frame claiming decoded_len = 0 whose payload hides megabytes MUST fail after producing at most one byte. A 100-byte frame claiming 8 GiB costs O(header) work to reject.¶

12. Security Considerations

Integrity is not authenticity: CRC-32 detects accidental corruption, not tampering. Where malice is in scope, verify a MAC or signature over the decoded bytes at the application layer.¶

Compression side channel (CRIME/BREACH class, [CRIME], [RFC7457]): where the emitted length is observable to an adversary who controls part of an input carried alongside secret data, compression leaks the secret through the length. Implementations MUST NOT ba64-compress attacker-influenced data concatenated with secret data in such settings; the always-plain encoder mode applies there, and its length, 4*ceil(n/3), is a function of the input length alone.¶

Where compression cannot be disabled, the padding of Section 6 and Section 10 quantizes the emitted length. At quantum Q, a one-byte change in compressibility is unobservable unless it crosses a bucket boundary; at Q >= P all inputs of a given length that fit under the floor rule emit P - 3 characters. Padding increases the number of observations an attack requires. It does not remove the channel: bucket boundaries are still crossed under repeated queries, and the bit identified in Section 10 still leaks. Padding is therefore a secondary control and not a substitute for separating attacker-influenced data from secret data.¶

The pad is not covered by the CRC-32, which is computed over the decoded bytes. An attacker able to rewrite the text can rewrite the pad undetected, as with any other part of an unauthenticated message. The decoded output is unaffected.¶

Decompression bombs are handled structurally by the limit and the hard inflate cap. Decoder error messages MUST NOT echo payload or decoded content by default: inputs may be secrets and errors end up in logs; the code and an offset suffice.¶

Canonical base64 plus minimal LEB128 plus the ordered checks of Section 7 ensure every ba64 text has exactly one interpretation (specific bytes or one specific error). There is no decode-side malleability.¶

13. Interoperability and Migration

Compatibility is one-directional: a ba64 decoder reads every legacy base64 string unchanged (the plain form is base64); a plain base64 decoder cannot read the compressed form. Therefore deploy ba64 decoders everywhere first (drop-in, no behavior change), enable compressing encoders per channel only once every reader decodes ba64, and note that reverting encoders to the always-plain mode restores universal readability instantly.¶

All ba64 characters are safe verbatim in JSON strings and cookie values. In URL components, "+", "/", and "=" MUST be percent-encoded. Transport artifacts (MIME line-wrapping, trailing newlines) MUST be stripped by the caller before decoding.¶

14. Non-Goals

No URL-safe or alternative alphabets, no streaming, no compression negotiation, no dictionary registry, no public methods beyond the two defined here. No version 2 is planned; the version byte is an escape hatch, not a roadmap.¶

15. IANA Considerations

This document has no IANA actions. The method registry in this document is maintained by revisions of this document, not by IANA.¶

Appendix A. Golden Examples

Every conforming decoder MUST reproduce these exactly.¶

""                              -> ""     (empty plain form)
"SGVsbG8sIHdvcmxkIQ=="          -> "Hello, world!" (13 bytes)
"=AQECDg4XTQECAP3/SGk="         -> "Hi"   (stored block)
"=AQEAAAAAAAEAAP//"             -> ""     (decoded_len 0)
"=AQICDg4XTQABAgD9/0hp"         -> "Hi"   (0x02, pad_len 0)
"=AQICDg4XTQMBAgD9/0hpAAAA"     -> "Hi"   (0x02, pad_len 3)
"=AQICDg4XTQQBAgD9/0hp/wDerQ==" -> "Hi"   (0x02, nonzero pad)

Error-direction anchors:¶

"="            -> E_TRUNCATED          "=="        -> E_BASE64
"SGVsbG8"      -> E_BASE64 (padding)   "QR==" -> E_BASE64 (bits)
"SGVs bG8="    -> E_BASE64 (space)     version=2   -> E_VERSION
varint 80 00   -> E_HEADER             claimed 1TB -> E_LIMIT_EXCEEDED
pad_len past end of frame -> E_TRUNCATED
"=AQEAAAAAAAEBAP7/WA==" -> E_LENGTH_MISMATCH (zero-cap trap)

Appendix B. Conformance Vectors

The normative conformance corpus is a set of machine-readable JSON vector files (decode passthrough, valid frames of both methods, error cases with required codes, encoder invariants, resource-exhaustion frames, and a large differential set). They are the executable form of this specification: an implementation conforms if and only if it reproduces every decode vector's bytes exactly and raises every error vector's exact code. Six independent implementations (Python, TypeScript, Go, Rust, Java, C#) are maintained against this corpus and agree pairwise over the full differential set. The corpus is published alongside this document.¶

Acknowledgments

The format's testing discipline follows the SQLite and zlib approach: many more lines of tests and vectors than of codec, continuous fuzzing ([OSSFUZZ]), and every historical finding frozen into the corpus.¶

Author's Address