README
¶
envelope
Chunked AES-256-GCM envelope encryption for payloads at rest — a small, dependency-free Go package for encrypting blobs (object-storage objects, database columns) under a per-object data key, where key management is the caller's job.
- Stdlib-only. No third-party dependencies. Nothing to audit but the standard
cryptolibrary. - Vault-agnostic. The package never talks to a KMS. You supply an
UnwrapFuncthat turns the stored wrapped data-encryption key back into raw key bytes however you like — a cloud KMS, a Vault/OpenBao transit engine, an HSM, or an in-memory key for tests. So the same format works everywhere and is fully testable without any external service. - One format, two call sites. One-shot
Seal/Openfor small values (e.g. a DB column), and streamingWriter/Readerfor large blobs you don't want to hold in memory. - Tamper-evident + context-bound. Every chunk is GCM-authenticated, and a
bindingstring (e.g. a tenant or object id) is mixed into the additional authenticated data, so ciphertext can't be silently moved between contexts.
How it works
The plaintext is split into fixed-size chunks, each sealed with AES-256-GCM under a random
data-encryption key (DEK). The DEK is never stored in the clear — you wrap it (with your key
system) and the wrapped DEK + its key id ride in the ciphertext header. On open, the header's wrapped
DEK is handed back to your UnwrapFunc to recover the raw key, then each chunk is verified and
decrypted.
type UnwrapFunc func(ctx context.Context, tenant string, wrapped []byte, keyID string) ([]byte, error)
Install
go get github.com/getotium/envelope
Usage
// One-shot (small values):
sealed, err := envelope.Seal(params, plaintext) // params carries the wrapped DEK + key id
plain, err := envelope.Open(ctx, sealed, binding, unwrap) // unwrap recovers the raw DEK
// Streaming (large blobs):
w, err := envelope.NewWriter(dst, params) // dst is any io.Writer (e.g. an object-store upload)
// ... io.Copy(w, src) ...; w.Close()
r, err := envelope.NewReader(ctx, src, binding, unwrap) // src is any io.Reader; read decrypted bytes out
Seal needs no context or vault call (it encrypts under a key you already hold, wrapped); Open
takes the UnwrapFunc because recovering the key is the only step that touches your key system.
Provenance
Extracted from Otium's payload-encryption layer, where it encrypts customer job payloads at rest under per-tenant keys. Published standalone so the data path is auditable.
License
Documentation
¶
Overview ¶
Package envelope is Otium's payload encryption format: AES-256-GCM under a per-payload data key, with the wrapped data key carried in the ciphertext's own header (docs/payload-encryption.md).
One format serves both call sites — object storage (streaming, via Writer/Reader) and Postgres payload columns (one-shot, via Seal/Open) — so there is a single thing to get right and a single thing to test.
The package is stdlib-only and holds no Otium types: it knows nothing about keyrings, jobs, or tenants beyond an opaque tenant string. Key wrapping is the caller's business (pkg/keyring), which keeps this lift-ready for go-toolkit and keeps the crypto testable without a vault.
Format ¶
offset size field
0 8 magic "OTIUMENC"
8 1 version
9 1 algorithm
10 2 tenant length (big-endian uint16)
12 2 key id length (big-endian uint16)
14 4 wrapped DEK length (big-endian uint32)
18 7 nonce prefix
25 n tenant
. m key id
. k wrapped DEK
---- chunks follow ----
repeated: 4-byte big-endian length, then that many ciphertext bytes
Each chunk is sealed with the same DEK under nonce = prefix(7) || counter(4) || final(1), with the header and a caller-supplied binding string as additional authenticated data. Three properties follow:
- Nonce reuse — GCM's catastrophic failure — cannot happen by accident: the prefix is random per payload and the counter is monotonic.
- Truncation is detectable: only the last chunk sets the final flag, so a cut-short stream fails to authenticate instead of decrypting to a plausible short payload.
- Relocation is detectable: the binding (an object key, or a job id and column) is authenticated, so ciphertext moved elsewhere — or to another tenant — will not open.
Index ¶
- Constants
- Variables
- func HeaderTenant(b []byte) (string, error)
- func IsEnvelope(b []byte) bool
- func Open(ctx context.Context, data []byte, binding string, unwrap UnwrapFunc) ([]byte, error)
- func OpenString(ctx context.Context, data []byte, binding string, unwrap UnwrapFunc) (string, error)
- func Seal(p Params, plaintext []byte) ([]byte, error)
- func SealString(p Params, plaintext string) ([]byte, error)
- func SealedSize(p Params, n int64) int64
- type Params
- type Reader
- type UnwrapFunc
- type Writer
Examples ¶
Constants ¶
const ( // Version1 is the current envelope version. Version1 = 1 // AlgAES256GCM is AES-256-GCM with 1 MiB chunks. AlgAES256GCM = 1 // ChunkSize is the plaintext bytes per sealed chunk. Bounds the working set: a // multi-gigabyte batch file encrypts in a megabyte of memory. ChunkSize = 1 << 20 )
const DEKSize = 32
DEKSize is the required data-key length (AES-256).
Variables ¶
var ( // ErrNotEnvelope means the data does not begin with Magic — i.e. it is plaintext, // not corrupted ciphertext. Callers use this to pass legacy data through. ErrNotEnvelope = errors.New("envelope: not an envelope") // ErrCorrupt covers every failure to parse or authenticate: bad header, wrong // binding, tampered or truncated ciphertext. Deliberately undifferentiated — telling // a caller which check failed tells an attacker the same thing. ErrCorrupt = errors.New("envelope: corrupt or tampered ciphertext") // ErrUnsupported means the envelope was written by a newer version or with an // algorithm this build does not know. Distinct from ErrCorrupt because the fix is // to upgrade, not to restore from backup. ErrUnsupported = errors.New("envelope: unsupported version or algorithm") // ErrBadDEK means the supplied data key is not 32 bytes. ErrBadDEK = errors.New("envelope: data key must be 32 bytes") // ErrNoTenant means no tenant was supplied. Encrypting without one would defeat the // isolation the format exists to provide. ErrNoTenant = errors.New("envelope: tenant required") )
Errors reported by this package.
var Magic = []byte("OTIUMENC")
Magic prefixes every envelope. Data that does not start with it is not an envelope — which is what lets the decrypt path pass legacy plaintext through untouched during rollout (docs/payload-encryption.md §6).
Functions ¶
func HeaderTenant ¶
HeaderTenant returns the tenant recorded in an envelope header without decrypting anything. Useful for operator tooling and for asserting that a stored object belongs to the tenant that asked for it. Returns ErrNotEnvelope for plaintext.
func IsEnvelope ¶
IsEnvelope reports whether b begins with the envelope magic. Used by the decrypt paths to pass through data written before encryption was enabled.
func Open ¶
Open decrypts an envelope produced by Seal (or Writer) back into plaintext.
It returns ErrNotEnvelope when data is not an envelope at all, which is how the rollout reads payloads written before encryption was enabled: the caller treats that error as "this is plaintext, use it verbatim" (docs/payload-encryption.md §6).
func OpenString ¶
func OpenString(ctx context.Context, data []byte, binding string, unwrap UnwrapFunc) (string, error)
OpenString decrypts to a string.
func Seal ¶
Seal encrypts plaintext into a single self-contained envelope. It is the one-shot form of Writer, for payloads that are already fully in memory — the jobs.payload and jobs.result columns, which are bounded by the submit API's request-size cap.
The output is the same wire format Writer produces, so anything Seal writes, Reader can read and vice versa. There is exactly one format in the system.
Example ¶
package main
import (
"bytes"
"fmt"
"github.com/getotium/envelope"
)
func main() {
p := envelope.Params{
Tenant: "acme",
KeyID: "otium-tenant-acme:v1",
DEK: bytes.Repeat([]byte{0x2A}, envelope.DEKSize),
Wrapped: []byte("wrapped-by-the-keyring"),
Binding: "t/acme/batch/input/file-1",
}
sealed, err := envelope.Seal(p, []byte("a customer prompt"))
if err != nil {
panic(err)
}
fmt.Println(envelope.IsEnvelope(sealed))
}
Output: true
func SealString ¶
SealString and OpenString are string-typed conveniences for the Postgres payload columns, which are TEXT. The ciphertext is binary, so callers that need a text-safe representation must encode it — pkg/store does, and its column comment says so.
func SealedSize ¶
SealedSize returns the exact ciphertext length for a plaintext of n bytes under these params.
It exists so a streaming writer can still declare its length. An S3 client given an unknown size cannot choose a part size, so it allocates a worst-case buffer per object — which OOM-killed the re-encryption Job after three files. The format is fully deterministic, so there is no reason to make the caller guess.
Layout: the header, then one framed chunk per ChunkSize of plaintext plus a final chunk that is always emitted (possibly empty), each costing a 4-byte length prefix and a 16-byte tag.
Types ¶
type Params ¶
type Params struct {
Tenant string
KeyID string
DEK []byte
Wrapped []byte
// Binding is authenticated but not stored: an object key, or "job:<id>:payload".
// Ciphertext will not open under a different binding, which is what prevents a
// storage-layer attacker from relocating one tenant's payload into another's slot.
Binding string
}
Params describe the key material and identity for one payload. The DEK encrypts the bytes; Wrapped is the same key sealed under the tenant's KEK and is stored in the header so the payload carries its own key. KeyID records which KEK version produced Wrapped, so a rotation never strands existing ciphertext.
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader decrypts a chunked envelope stream.
func NewReader ¶
func NewReader(ctx context.Context, r io.Reader, binding string, unwrap UnwrapFunc) (*Reader, error)
NewReader reads the envelope header from r, unwraps its data key via unwrap, and returns a Reader over the plaintext.
It returns ErrNotEnvelope if r does not begin with the magic — the caller decides whether that is legacy plaintext to pass through or an error. Note that the bytes already consumed from r cannot be un-read, so callers that need passthrough should use a buffered peek (objectstore.Encrypted does exactly this).
type UnwrapFunc ¶
type UnwrapFunc func(ctx context.Context, tenant string, wrapped []byte, keyID string) ([]byte, error)
UnwrapFunc unwraps a data key that was sealed under a tenant's KEK. Its signature matches keyring.Keyring.Unwrap exactly, so a Keyring's method value satisfies it — which is how this package stays free of any Otium import.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer encrypts a stream into chunked envelope format. Callers write plaintext and must Close to flush the final chunk — Close is what marks the stream complete, so a Writer that is never closed produces ciphertext that deliberately fails to open.
func NewWriter ¶
NewWriter returns a Writer that encrypts to w. The header is emitted on the first Write or on Close, so an empty payload still produces a well-formed, authenticated envelope.
Source Files