Documentation
¶
Overview ¶
Package cereal provides context-aware serialization with field transformation.
The package offers a Codec interface for marshaling/unmarshaling data, along with a generic Processor that adds context-aware field transformation including encryption, hashing, masking, and redaction based on struct tags.
Contexts ¶
Cereal operates on four contexts representing boundary crossings:
- receive: Ingress from external sources (API requests, events)
- load: Ingress from storage (database, cache)
- store: Egress to storage
- send: Egress to external destinations (API responses, events)
Tag Syntax ¶
Field behavior is declared via struct tags:
{context}.{action}:"{capability}"
Valid combinations:
receive.hash:"argon2" - Hash on receive (passwords) load.decrypt:"aes" - Decrypt on load store.encrypt:"aes" - Encrypt on store send.mask:"email" - Mask on send send.redact:"***" - Redact on send
Basic Usage ¶
type User struct {
ID string `json:"id"`
Password string `json:"password" receive.hash:"argon2" send.redact:"***"`
Email string `json:"email" store.encrypt:"aes" load.decrypt:"aes" send.mask:"email"`
}
func (u User) Clone() User { return u }
proc, _ := cereal.NewProcessor[User]()
enc, _ := cereal.AES(aesKey)
proc.SetEncryptor(cereal.EncryptAES, enc)
// Primary API: T -> T boundary transforms
received, _ := proc.Receive(ctx, user) // hashes password
stored, _ := proc.Store(ctx, received) // encrypts email
loaded, _ := proc.Load(ctx, stored) // decrypts email
sent, _ := proc.Send(ctx, loaded) // masks email, redacts password
// Secondary API: codec-aware (requires SetCodec)
proc.SetCodec(json.New())
user, _ := proc.Decode(ctx, requestBody) // unmarshal + hash
data, _ := proc.Write(ctx, &user) // encrypt + marshal
loaded, _ := proc.Read(ctx, data) // unmarshal + decrypt
response, _ := proc.Encode(ctx, &loaded) // mask/redact + marshal
Capability Types ¶
Capabilities are constrained to predefined constants:
- EncryptAlgo: EncryptAES, EncryptRSA, EncryptEnvelope
- HashAlgo: HashArgon2, HashBcrypt, HashSHA256, HashSHA512
- MaskType: MaskSSN, MaskEmail, MaskPhone, MaskCard, MaskIP, MaskUUID, MaskIBAN, MaskName
Auto-Registration ¶
Hashers and maskers are auto-registered. Only encryption keys need manual registration:
cereal.WithKey(cereal.EncryptAES, key)
Override Interfaces ¶
Types can bypass reflection by implementing action-specific interfaces:
- Encryptable: Custom encryption logic
- Decryptable: Custom decryption logic
- Hashable: Custom hashing logic
- Maskable: Custom masking logic
- Redactable: Custom redaction logic
Codec Providers ¶
The following codec implementations are available as submodules:
- json - JSON encoding (application/json)
- xml - XML encoding (application/xml)
- yaml - YAML encoding (application/yaml)
- msgpack - MessagePack encoding (application/msgpack)
- bson - BSON encoding (application/bson)
Encryption Algorithms ¶
Built-in encryptors:
- AES(key) - AES-GCM symmetric encryption
- RSA(pub, priv) - RSA-OAEP asymmetric encryption
- Envelope(masterKey) - Envelope encryption with per-message data keys
Hash Algorithms ¶
Built-in hashers:
- Argon2() - Argon2id password hashing (salted)
- Bcrypt() - bcrypt password hashing (salted)
- SHA256Hasher() - SHA-256 deterministic hashing
- SHA512Hasher() - SHA-512 deterministic hashing
Masking ¶
Built-in content-aware maskers:
- ssn: 123-45-6789 → ***-**-6789
- email: alice@example.com → a***@example.com
- phone: (555) 123-4567 → (***) ***-4567
- card: 4111111111111111 → ************1111
- ip: 192.168.1.100 → 192.168.xxx.xxx
- uuid: 550e8400-e29b-... → 550e8400-****-****-****-************
- iban: GB82WEST12345698765432 → GB82**************5432
- name: John Smith → J*** S****
Index ¶
- Variables
- func IsValidEncryptAlgo(algo EncryptAlgo) bool
- func IsValidHashAlgo(algo HashAlgo) bool
- func IsValidMaskType(mt MaskType) bool
- func ResetPlansCache()
- type Argon2Params
- type BcryptCost
- type Cloner
- type Codec
- type CodecError
- type ConfigError
- type Decryptable
- type EncryptAlgo
- type Encryptable
- type Encryptor
- type HashAlgo
- type Hashable
- type Hasher
- type MaskType
- type Maskable
- type Masker
- type Processor
- func (p *Processor[T]) Decode(ctx context.Context, data []byte) (*T, error)
- func (p *Processor[T]) Encode(ctx context.Context, obj *T) ([]byte, error)
- func (p *Processor[T]) Load(ctx context.Context, obj T) (T, error)
- func (p *Processor[T]) Read(ctx context.Context, data []byte) (*T, error)
- func (p *Processor[T]) Receive(ctx context.Context, obj T) (T, error)
- func (p *Processor[T]) Send(ctx context.Context, obj T) (T, error)
- func (p *Processor[T]) SetCodec(codec Codec) *Processor[T]
- func (p *Processor[T]) SetEncryptor(algo EncryptAlgo, enc Encryptor) *Processor[T]
- func (p *Processor[T]) SetHasher(algo HashAlgo, h Hasher) *Processor[T]
- func (p *Processor[T]) SetMasker(mt MaskType, m Masker) *Processor[T]
- func (p *Processor[T]) Store(ctx context.Context, obj T) (T, error)
- func (p *Processor[T]) Validate() error
- func (p *Processor[T]) Write(ctx context.Context, obj *T) ([]byte, error)
- type Redactable
- type TransformError
Constants ¶
This section is empty.
Variables ¶
var ( // ErrMissingEncryptor indicates a required encryptor was not registered. ErrMissingEncryptor = errors.New("missing encryptor") // ErrMissingHasher indicates a required hasher was not registered. ErrMissingHasher = errors.New("missing hasher") // ErrMissingMasker indicates a required masker was not registered. ErrMissingMasker = errors.New("missing masker") // ErrInvalidTag indicates a struct tag has an invalid format or value. ErrInvalidTag = errors.New("invalid tag") // ErrUnmarshal indicates the codec failed to unmarshal input data. ErrUnmarshal = errors.New("unmarshal failed") // ErrMarshal indicates the codec failed to marshal output data. ErrMarshal = errors.New("marshal failed") // ErrEncrypt indicates encryption of a field failed. ErrEncrypt = errors.New("encrypt failed") // ErrDecrypt indicates decryption of a field failed. ErrDecrypt = errors.New("decrypt failed") // ErrHash indicates hashing of a field failed. ErrHash = errors.New("hash failed") // ErrMask indicates masking of a field failed. ErrMask = errors.New("mask failed") // ErrRedact indicates redaction of a field failed. ErrRedact = errors.New("redact failed") // ErrInvalidKey indicates an encryption key has invalid size or format. ErrInvalidKey = errors.New("invalid key") // ErrMissingCodec indicates a codec operation was called without a configured codec. ErrMissingCodec = errors.New("missing codec") )
Sentinel errors for programmatic error handling. Use errors.Is() to check for these error types.
var ( SignalProcessorCreated = capitan.NewSignal("codec.processor.created", "Processor instantiated") SignalReceiveStart = capitan.NewSignal("codec.receive.start", "Receive operation beginning") SignalReceiveComplete = capitan.NewSignal("codec.receive.complete", "Receive operation finished") SignalLoadStart = capitan.NewSignal("codec.load.start", "Load operation beginning") SignalLoadComplete = capitan.NewSignal("codec.load.complete", "Load operation finished") SignalStoreStart = capitan.NewSignal("codec.store.start", "Store operation beginning") SignalStoreComplete = capitan.NewSignal("codec.store.complete", "Store operation finished") SignalSendStart = capitan.NewSignal("codec.send.start", "Send operation beginning") SignalSendComplete = capitan.NewSignal("codec.send.complete", "Send operation finished") )
Signals for codec events.
var ( KeyContentType = capitan.NewStringKey("content_type") KeyTypeName = capitan.NewStringKey("type_name") KeySize = capitan.NewIntKey("size") KeyDuration = capitan.NewDurationKey("duration") KeyError = capitan.NewErrorKey("error") KeyEncryptedCount = capitan.NewIntKey("encrypted_count") KeyDecryptedCount = capitan.NewIntKey("decrypted_count") KeyHashedCount = capitan.NewIntKey("hashed_count") KeyMaskedCount = capitan.NewIntKey("masked_count") KeyRedactedCount = capitan.NewIntKey("redacted_count") )
Keys for typed event data.
var ( // ErrCiphertextShort indicates the ciphertext is too short to decrypt. ErrCiphertextShort = errors.New("ciphertext too short") )
Encryption-specific errors (extend the base sentinel errors).
Functions ¶
func IsValidEncryptAlgo ¶
func IsValidEncryptAlgo(algo EncryptAlgo) bool
IsValidEncryptAlgo returns true if the algorithm is a known encryption algorithm.
func IsValidHashAlgo ¶
IsValidHashAlgo returns true if the algorithm is a known hash algorithm.
func IsValidMaskType ¶
IsValidMaskType returns true if the type is a known mask type.
func ResetPlansCache ¶
func ResetPlansCache()
ResetPlansCache clears the field plans cache. This is primarily useful for test isolation.
Types ¶
type Argon2Params ¶
type Argon2Params struct {
Time uint32 // Number of iterations
Memory uint32 // Memory usage in KiB
Threads uint8 // Parallelism factor
KeyLen uint32 // Output key length
SaltLen uint32 // Salt length
}
Argon2Params configures Argon2id hashing.
func DefaultArgon2Params ¶
func DefaultArgon2Params() Argon2Params
DefaultArgon2Params returns recommended Argon2id parameters. Based on OWASP recommendations for password hashing.
type BcryptCost ¶
type BcryptCost int
BcryptCost represents the bcrypt cost factor.
const ( BcryptMinCost BcryptCost = BcryptCost(bcrypt.MinCost) BcryptMaxCost BcryptCost = BcryptCost(bcrypt.MaxCost) // BcryptDefaultCost is the default cost used by Bcrypt(). // Set to 12 per OWASP recommendations (2024+) for password hashing. // The standard library default is 10, but modern hardware warrants higher costs. BcryptDefaultCost BcryptCost = 12 )
Bcrypt cost constants.
type Cloner ¶
type Cloner[T any] interface { Clone() T }
Cloner allows types to provide deep copy logic. Implementing this interface is required for use with Processor.
Deep Copy Requirement ¶
The Clone method MUST return a deep copy where modifications to the clone do not affect the original value. This is critical for the processor's non-destructive behavior: Store() and Send() transform clones, leaving originals untouched.
WARNING: A shallow copy (simply returning the receiver) is only safe for types with NO reference fields. If your type contains pointers, slices, maps, or nested structs with reference fields, you MUST deep copy them.
Simple Value Types ¶
For types with only primitive fields (string, int, bool, etc.), a shallow copy is sufficient because Go copies these by value:
type User struct {
ID string
Name string
Age int
}
func (u User) Clone() User { return u } // Safe: all fields are values
Types with Reference Fields ¶
For types containing slices, maps, or pointers, you MUST allocate new backing storage and copy elements:
type Order struct {
ID string
Items []Item // Slice: needs deep copy
Metadata map[string]string // Map: needs deep copy
Billing *Address // Pointer: needs deep copy
}
func (o Order) Clone() Order {
clone := Order{ID: o.ID}
// Deep copy slice
if o.Items != nil {
clone.Items = make([]Item, len(o.Items))
copy(clone.Items, o.Items)
}
// Deep copy map
if o.Metadata != nil {
clone.Metadata = make(map[string]string, len(o.Metadata))
for k, v := range o.Metadata {
clone.Metadata[k] = v
}
}
// Deep copy pointer
if o.Billing != nil {
addr := *o.Billing
clone.Billing = &addr
}
return clone
}
Nested Structs ¶
If a nested struct itself contains reference fields, recursively apply the same deep copy logic. Consider implementing Clone() on nested types and calling it from the parent:
func (o Order) Clone() Order {
clone := Order{ID: o.ID}
if o.Billing != nil {
billingClone := o.Billing.Clone()
clone.Billing = &billingClone
}
return clone
}
Verification ¶
To verify your Clone implementation is correct, test that modifying the clone does not affect the original:
original := Order{Items: []Item{{Name: "A"}}}
clone := original.Clone()
clone.Items[0].Name = "B"
assert(original.Items[0].Name == "A") // Must still be "A"
type Codec ¶
type Codec interface {
// ContentType returns the MIME type for this codec (e.g., "application/json").
ContentType() string
// Marshal encodes v into bytes.
Marshal(v any) ([]byte, error)
// Unmarshal decodes data into v.
Unmarshal(data []byte, v any) error
}
Codec provides content-type aware marshaling.
type CodecError ¶
type CodecError struct {
Err error // Underlying sentinel error (ErrMarshal, ErrUnmarshal)
Cause error // Original error from the codec
}
CodecError represents a marshal/unmarshal error.
func (*CodecError) Error ¶
func (e *CodecError) Error() string
func (*CodecError) Unwrap ¶
func (e *CodecError) Unwrap() error
type ConfigError ¶
type ConfigError struct {
Err error // Underlying sentinel error (ErrMissingEncryptor, etc.)
Field string // Field name that triggered the error
Algorithm string // Algorithm or type that was missing/invalid
}
ConfigError represents a processor configuration error. It wraps a sentinel error with additional context about the field and algorithm.
func (*ConfigError) Error ¶
func (e *ConfigError) Error() string
func (*ConfigError) Unwrap ¶
func (e *ConfigError) Unwrap() error
type Decryptable ¶
type Decryptable interface {
// Decrypt transforms the receiver's fields that require decryption.
// The encryptors map contains all registered encryptors keyed by algorithm.
// Called on freshly unmarshaled data.
Decrypt(encryptors map[EncryptAlgo]Encryptor) error
}
Decryptable bypasses reflection for load.decrypt actions. Implement this to handle all decryption for a type.
type EncryptAlgo ¶
type EncryptAlgo string
EncryptAlgo represents a supported encryption algorithm. Use these constants in struct tags: `store.encrypt:"aes"`
const ( // EncryptAES uses AES-GCM symmetric encryption. EncryptAES EncryptAlgo = "aes" // EncryptRSA uses RSA-OAEP asymmetric encryption. EncryptRSA EncryptAlgo = "rsa" // EncryptEnvelope uses envelope encryption with per-message data keys. EncryptEnvelope EncryptAlgo = "envelope" )
type Encryptable ¶
type Encryptable interface {
// Encrypt transforms the receiver's fields that require encryption.
// The encryptors map contains all registered encryptors keyed by algorithm.
// The receiver is a clone, so mutations are safe.
Encrypt(encryptors map[EncryptAlgo]Encryptor) error
}
Encryptable bypasses reflection for store.encrypt actions. Implement this to handle all encryption for a type.
type Encryptor ¶
type Encryptor interface {
// Encrypt encrypts plaintext and returns ciphertext.
Encrypt(plaintext []byte) ([]byte, error)
// Decrypt decrypts ciphertext and returns plaintext.
Decrypt(ciphertext []byte) ([]byte, error)
}
Encryptor handles encryption/decryption operations.
func AES ¶
AES returns an AES-GCM encryptor. Key must be 16, 24, or 32 bytes for AES-128, AES-192, or AES-256.
type HashAlgo ¶
type HashAlgo string
HashAlgo represents a supported hashing algorithm. Use these constants in struct tags: `receive.hash:"argon2"`
const ( // HashArgon2 uses Argon2id for password hashing (salted, slow). HashArgon2 HashAlgo = "argon2" // HashBcrypt uses bcrypt for password hashing (salted, slow). HashBcrypt HashAlgo = "bcrypt" // HashSHA256 uses SHA-256 for deterministic hashing (fast, no salt). // Use for fingerprinting/identification, NOT for passwords. HashSHA256 HashAlgo = "sha256" // HashSHA512 uses SHA-512 for deterministic hashing (fast, no salt). // Use for fingerprinting/identification, NOT for passwords. HashSHA512 HashAlgo = "sha512" )
type Hashable ¶
type Hashable interface {
// Hash transforms the receiver's fields that require hashing.
// The hashers map contains all registered hashers keyed by algorithm.
// Called on freshly unmarshaled data.
Hash(hashers map[HashAlgo]Hasher) error
}
Hashable bypasses reflection for receive.hash actions. Implement this to handle all hashing for a type.
type Hasher ¶
type Hasher interface {
// Hash returns the hash of plaintext as a string.
// For password hashers (argon2, bcrypt), the result includes salt and parameters.
// For deterministic hashers (sha256, sha512), the result is a hex-encoded hash.
Hash(plaintext []byte) (string, error)
}
Hasher performs one-way hashing.
func Argon2WithParams ¶
func Argon2WithParams(params Argon2Params) Hasher
Argon2WithParams returns an Argon2id hasher with custom parameters.
func Bcrypt ¶
func Bcrypt() Hasher
Bcrypt returns a bcrypt hasher with default cost (12). This cost is recommended by OWASP for password hashing as of 2024.
func BcryptWithCost ¶
func BcryptWithCost(cost BcryptCost) Hasher
BcryptWithCost returns a bcrypt hasher with a specific cost factor.
func SHA256Hasher ¶
func SHA256Hasher() Hasher
SHA256Hasher returns a SHA-256 hasher. The result is a hex-encoded 64-character string. Use for fingerprinting/identification, NOT for passwords.
func SHA512Hasher ¶
func SHA512Hasher() Hasher
SHA512Hasher returns a SHA-512 hasher. The result is a hex-encoded 128-character string. Use for fingerprinting/identification, NOT for passwords.
type MaskType ¶
type MaskType string
MaskType represents a known data format with masking rules.
const ( MaskSSN MaskType = "ssn" // 123-45-6789 -> ***-**-6789 MaskEmail MaskType = "email" // alice@example.com -> a***@example.com MaskPhone MaskType = "phone" // (555) 123-4567 -> (***) ***-4567 MaskCard MaskType = "card" // 4111111111111111 -> ************1111 MaskIP MaskType = "ip" // 192.168.1.100 -> 192.168.xxx.xxx MaskUUID MaskType = "uuid" // 550e8400-e29b-41d4-a716-446655440000 -> 550e8400-****-****-****-************ MaskIBAN MaskType = "iban" // GB82WEST12345698765432 -> GB82************5432 MaskName MaskType = "name" // John Smith -> J*** S**** )
Mask type constants for content-aware masking.
type Maskable ¶
type Maskable interface {
// Mask transforms the receiver's fields that require masking.
// The maskers map contains all registered maskers keyed by type.
// The receiver is a clone, so mutations are safe.
Mask(maskers map[MaskType]Masker) error
}
Maskable bypasses reflection for send.mask actions. Implement this to handle all masking for a type.
type Masker ¶
type Masker interface {
// Mask applies masking to the value.
// Returns an error if the value doesn't match the expected format.
Mask(value string) (string, error)
}
Masker applies content-aware masking.
func CardMasker ¶
func CardMasker() Masker
CardMasker returns a masker for credit card numbers. Preserves the last 4 digits, masks everything else.
func EmailMasker ¶
func EmailMasker() Masker
EmailMasker returns a masker for email addresses. Preserves first character of local part and full domain.
func IBANMasker ¶
func IBANMasker() Masker
IBANMasker returns a masker for IBANs. Preserves country code + check digits (first 4) and last 4 chars.
func IPMasker ¶
func IPMasker() Masker
IPMasker returns a masker for IP addresses. Supports both IPv4 and IPv6. IPv4: Preserves first two octets (network), masks last two (host). IPv6: Preserves first four groups (network prefix), masks last four (interface ID).
func NameMasker ¶
func NameMasker() Masker
NameMasker returns a masker for personal names. Preserves first letter of each word, masks the rest.
func PhoneMasker ¶
func PhoneMasker() Masker
PhoneMasker returns a masker for phone numbers. Preserves the last 4 digits, masks everything else.
func SSNMasker ¶
func SSNMasker() Masker
SSNMasker returns a masker for Social Security Numbers. Preserves the last 4 digits, masks everything else.
func UUIDMasker ¶
func UUIDMasker() Masker
UUIDMasker returns a masker for UUIDs. Preserves first segment, masks the rest.
type Processor ¶
type Processor[T Cloner[T]] struct { // contains filtered or unexported fields }
Processor provides context-aware serialization with field transformation. Use Receive/Load for ingress and Store/Send for egress.
Processors are safe for concurrent use. Configuration methods (SetEncryptor, SetHasher, SetMasker) may be called at any time to update or rotate keys.
Validation occurs automatically on first operation. Configure all required handlers before the first call to Receive, Load, Store, or Send.
func NewProcessor ¶
NewProcessor creates a new Processor for type T.
The processor is created with builtin hashers and maskers. Encryptors must be configured via SetEncryptor before using Store/Load operations on fields with encryption tags.
A codec is not required for the primary T -> T API (Receive, Load, Store, Send). Use SetCodec to enable the codec-aware methods (Decode, Read, Write, Encode).
Use Validate() to check that all required capabilities are configured.
func (*Processor[T]) Decode ¶
Decode unmarshals data and applies receive context actions (hash). Requires a codec to be configured via SetCodec. Use for data coming from external sources (API requests, events).
func (*Processor[T]) Encode ¶
Encode applies send context actions (mask, redact) and marshals the result. Requires a codec to be configured via SetCodec. Use for data going to external destinations (API responses, events).
func (*Processor[T]) Load ¶
Load applies load context actions (decrypt) to a value. Returns a transformed clone, leaving the original untouched. Use for data coming from storage (database, cache).
func (*Processor[T]) Read ¶
Read unmarshals data and applies load context actions (decrypt). Requires a codec to be configured via SetCodec. Use for data coming from storage (database, cache).
func (*Processor[T]) Receive ¶
Receive applies receive context actions (hash) to a value. Returns a transformed clone, leaving the original untouched. Use for data coming from external sources (API requests, events).
func (*Processor[T]) Send ¶
Send applies send context actions (mask, redact) to a value. Returns a transformed clone, leaving the original untouched. Use for data going to external destinations (API responses, events).
func (*Processor[T]) SetCodec ¶
SetCodec registers a codec for marshal/unmarshal operations. Required for Decode, Read, Write, and Encode methods. Returns the processor for chaining. Safe for concurrent use.
func (*Processor[T]) SetEncryptor ¶
func (p *Processor[T]) SetEncryptor(algo EncryptAlgo, enc Encryptor) *Processor[T]
SetEncryptor registers an encryptor for the given algorithm. Returns the processor for chaining. Safe for concurrent use.
func (*Processor[T]) SetHasher ¶
SetHasher registers a hasher for the given algorithm. Returns the processor for chaining. Safe for concurrent use.
func (*Processor[T]) SetMasker ¶
SetMasker registers a masker for the given type. Returns the processor for chaining. Safe for concurrent use.
func (*Processor[T]) Store ¶
Store applies store context actions (encrypt) to a value. Returns a transformed clone, leaving the original untouched. Use for data going to storage (database, cache).
func (*Processor[T]) Validate ¶
Validate checks that all required capabilities are configured. Returns an error if any field's required encryptor, hasher, or masker is not registered.
Validation also runs automatically on first operation. Calling Validate explicitly allows catching configuration errors at startup.
type Redactable ¶
type Redactable interface {
// Redact transforms the receiver's fields that require redaction.
// The receiver is a clone, so mutations are safe.
// Redaction values are typically hardcoded based on struct tag values.
Redact() error
}
Redactable bypasses reflection for send.redact actions. Implement this to handle all redaction for a type.
type TransformError ¶
type TransformError struct {
Err error // Underlying sentinel error (ErrEncrypt, ErrDecrypt, etc.)
Field string // Field name that failed
Operation string // Operation that failed (encrypt, decrypt, hash, mask, redact)
Cause error // Original error from the underlying operation
}
TransformError represents an error during field transformation. It wraps a sentinel error with context about which field and operation failed.
func (*TransformError) Error ¶
func (e *TransformError) Error() string
func (*TransformError) Unwrap ¶
func (e *TransformError) Unwrap() error