avro

README

avro

Encode and decode Avro binary data.

This project aims to be the "best" Avro encoder/decoder in the Go ecosystem by:

• Keeping the API tight but comprehensive
• Combining features that exist in only one of linkedin/goavro or hamba/avro
• Being safe above all, while being fast with unsafe specialization functions internally
• Running round after round of AI audits to shake out any bug / DoS / huge alloc that exists
• Being documented thoroughly for both human and AI users

For the "why" of this project, see the Why section.

Index

• Quick Start
• Type Mapping
• Struct Tags
• Schema Inference
• Schema Introspection
• Logical Types
• Schema Evolution
• Schema Cache
• Custom Types
• Type name constants
• Object Container Files
• JSON Encoding
• Single Object Encoding
• Fingerprinting
• Errors
• Performance
• Encode/decode behavior contract
• Why

Quick Start

package main

import (
	"fmt"
	"log"

	"github.com/twmb/avro"
)

var schema = avro.MustParse(`{
    "type": "record",
    "name": "User",
    "fields": [
        {"name": "name", "type": "string"},
        {"name": "age",  "type": "int"}
    ]
}`)

type User struct {
	Name string `avro:"name"`
	Age  int    `avro:"age"`
}

func main() {
	data, err := schema.Encode(&User{Name: "Alice", Age: 30})
	if err != nil {
		log.Fatal(err)
	}

	var u User
	_, err = schema.Decode(data, &u)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(u) // {Alice 30}
}

Type Mapping

All types can decode into any.

Avro Type	Encode	Decode
null	anything nillable	zeroes the target
boolean	bool	bool
int, long, float, double	numeric †	numeric †
string	string []byte TextAppender TextMarshaler	string []byte TextUnmarshaler
bytes	bytes-like	bytes-like ‡
enum	string integer TextAppender TextMarshaler	string integer TextUnmarshaler
fixed	bytes-like ‡	bytes-like ‡
array	slice or [N]array	slice or [N]array ‡
map	map[string]T	map[string]T
union	*T, tagged-union map, or the matched branch	*T or the matched branch
record	struct or map[string]any	struct or map[string]any

Shorthands used above: numeric = int int8–int64 uint uint8–uint64 float32 float64 json.Number. integer = int int8–int64 uint uint8–uint64. bytes-like = []byte [N]byte string. TextAppender / TextMarshaler / TextUnmarshaler are the standard encoding interfaces.

† Numeric types accept any numeric Go type, but coercion has precision rules — whole-number/range checks into integers, silent IEEE rounding into floats. See Encode/decode behavior contract. ‡ Length must match: decoding bytes into [N]byte, or encoding/decoding any value as fixed, requires the byte length to equal N; [N]array requires exactly N elements.

Decoding into any yields the natural Go type — int32/int64 for int/long, float32/float64 for float/double, []any for arrays, map[string]any for maps and records; logical types use the Go types in Logical Types. A null (e.g. a union's null branch) decodes to the target's zero value, replacing any prior contents — use *T to tell null from zero.

Struct Tags

Struct fields are matched to Avro record fields by name. Use the avro struct tag to control the mapping:

type Example struct {
    Name    string  `avro:"name"`          // maps to Avro field "name"
    Ignored int     `avro:"-"`             // excluded from encoding/decoding
    Inner   Nested  `avro:",inline"`       // inline Nested's fields into this record
    Value   int     `avro:"val,omitzero"`  // encode zero value as Avro default
}

The tag format is:

avro:"[name][,option][,option]..."

The name portion maps the struct field to the Avro field with that name. If empty, the Go field name is used as-is. A tag of "-" excludes the field entirely.

Supported options:

• inline: flatten a nested struct's fields into the parent record, as if they were declared directly on the parent. The field must be a struct or pointer to struct. This works like anonymous (embedded) struct fields, but for named fields. When using inline, the name portion of the tag must be empty and no other tag options are allowed — the flattened embed has no field of its own for default=, alias=, logical-type tags, etc. to apply to. Put those options on the embed's child fields directly.

• omitzero: when encoding, a zero value (or a field whose IsZero() bool method returns true) is encoded as if its key were omitted from a map[string]any — the field's schema default is used. A nullable field with no default encodes null; a non-nullable field with no default keeps its zero value (there is nothing to fill with). Useful for optional fields in ["null", T] / ["T", "null"] unions or fields with explicit defaults.

Embedded (anonymous) struct fields are automatically inlined — their fields are promoted into the parent as if declared directly. To prevent inlining an embedded struct, give it an explicit name tag:

type Parent struct {
    Nested                    // inlined: Nested's fields are promoted
    Other  Aux `avro:"other"` // not inlined: treated as a single field
}

When multiple fields resolve to the same Avro field name, a tagged field wins over an untagged one at any depth; among fields with the same tagged status, the shallowest field wins. Two fields that resolve to the same name at the same depth with the same tagged status are an ambiguous collision (Go itself makes such a field reference a compile error). twmb errors rather than silently selecting one: SchemaFor rejects the type, while encode and decode reject only when the schema actually resolves a field to the ambiguous name — a coincidental collision on a name the schema never references does not break the type.

Schema Inference

SchemaFor infers an Avro schema from a Go struct type, using the same struct tags as encoding/decoding:

type User struct {
    Name      string     `avro:"name"`
    Age       int32      `avro:"age,default=18"`
    Email     *string    `avro:"email"`
    CreatedAt time.Time  `avro:"created_at"`
}

schema := avro.MustSchemaFor[User](avro.WithNamespace("com.example"))

This produces the equivalent of:

{
  "type": "record",
  "name": "User",
  "namespace": "com.example",
  "fields": [
    {"name": "name", "type": "string"},
    {"name": "age", "type": "int", "default": 18},
    {"name": "email", "type": ["null", "string"]},
    {"name": "created_at", "type": {"type": "long", "logicalType": "timestamp-millis"}}
  ]
}

Go types map to Avro types automatically: *T becomes a ["null", T] union with "default": null (backward-compatible by default), [N]byte becomes a fixed type named after the Go type (or fixed_N for unnamed arrays), time.Time becomes timestamp-millis, and so on (see Type Mapping).

Additional tag options for schema inference:

Tag	Example	Description
default=	avro:",default=0"	Default value (must be last; scalars only)
alias=	avro:",alias=old"	Field alias for schema evolution (repeatable, or alias=[a,b])
type-alias=	avro:",type-alias=old"	Alias for the field's named type — record, enum, or fixed (repeatable, or type-alias=[a,b])
timestamp-micros	avro:",timestamp-micros"	Override logical type
decimal(p,s)	avro:",decimal(10,2)"	Decimal logical type (requires *big.Rat or big.Rat)
uuid	avro:",uuid"	UUID logical type (requires Go string, [16]byte, or a text marshaler type)
date	avro:",date"	Date logical type (requires time.Time, time.Duration, or an int8/16/32/uint8/uint16)

alias and type-alias serve different purposes in schema evolution. alias adds an alias to the field — it lets a writer field with a different name match this reader field. type-alias adds an alias to the named type (record, enum, or fixed) that the field references — it lets a writer type with a different name match the reader type. The alias is applied to the innermost named type, walking through pointers, slices, and maps:

type FieldSummary struct {
    ContainsNull bool    `avro:"contains_null"`
    ContainsNaN  *bool   `avro:"contains_nan"`
}

type ManifestFile struct {
    // alias=old_partitions: match writer field named "old_partitions"
    // type-alias=r508:      match writer record type named "r508" for FieldSummary
    Partitions *[]FieldSummary `avro:"partitions,type-alias=r508"`
}

Options:

• WithNamespace(ns) sets the Avro namespace for the record.
• WithName(name) overrides the record name (defaults to the Go struct name).

Schema Introspection

Schema.Root() returns a SchemaNode representing the parsed schema. This provides read access to all schema metadata including field types, logical types, doc strings, and custom properties:

schema, _ := avro.Parse(schemaJSON)
root := schema.Root()

for _, f := range root.Fields {
    fmt.Printf("field %s: type=%s\n", f.Name, f.Type.Type)
    if cn, ok := f.Props["connect.name"].(string); ok {
        fmt.Printf("  kafka connect type: %s\n", cn)
    }
}

SchemaNode can also be used to build schemas programmatically:

node := &avro.SchemaNode{
    Type: "record",
    Name: "User",
    Fields: []avro.SchemaField{
        {Name: "name", Type: avro.SchemaNode{Type: "string"}},
        {Name: "age", Type: avro.SchemaNode{Type: "int"}, Default: 18},
    },
}
schema, err := node.Schema()

Reserved attribute names (type, name, items, ...) match only their exact lowercase spelling, as in the Java, Python (fastavro), and goavro implementations. A key differing from a reserved name only by letter case ("ITEMS", "Namespace") is an ordinary custom property, preserved in Props. One migration note: hamba/avro matches these keys case-insensitively, so a schema that parsed there but fails here has a miscased reserved key. A miscased structural key fails loudly at parse time — {"type":"array","Items":"int"} errors with "array is missing items schema"; fix the casing — while a miscased non-structural key ("Doc", "Aliases") simply becomes a harmless custom property instead of binding the attribute.

Logical Types

Logical types decode to their natural Go equivalents:

Logical Type	Avro Type	Encode	Decode
date	int	time.Time, RFC 3339 or YYYY-MM-DD string, or int	time.Time (UTC)
time-millis	int	time.Duration, time.Time (lossy †), or int	time.Duration or time.Time (lossy †)
time-micros	long	time.Duration, time.Time (lossy †), or int	time.Duration or time.Time (lossy †)
timestamp-millis	long	time.Time, RFC 3339 string, or int	time.Time (UTC)
timestamp-micros	long	time.Time, RFC 3339 string, or int	time.Time (UTC)
timestamp-nanos	long	time.Time, RFC 3339 string, or int	time.Time (UTC)
local-timestamp-millis	long	time.Time, RFC 3339 string, or int	time.Time (UTC)
local-timestamp-micros	long	time.Time, RFC 3339 string, or int	time.Time (UTC)
local-timestamp-nanos	long	time.Time, RFC 3339 string, or int	time.Time (UTC)
uuid (string)	string	[16]byte or string	string into any; [16]byte or string into typed target
uuid (fixed(16))	fixed(16)	[16]byte, []byte, or hex-dash string	[16]byte into any or [16]byte target; string into string target
decimal	bytes or fixed	*big.Rat, float64, numeric string, json.Number, or underlying type	*big.Rat, float64/float32, numeric string, json.Number, or underlying type
big-decimal	bytes	*big.Rat, float64, numeric string, json.Number	*big.Rat, float64/float32, numeric string, json.Number
duration	fixed(12)	avro.Duration or underlying type	avro.Duration or underlying type

When encoding, timestamp and date fields accept RFC 3339 strings, and decimal fields accept float64 and numeric strings (e.g. "3.14"). Values that don't match the expected format fall through to the underlying type's encoder, which will return an error.

† time-millis / time-micros with time.Time is a convenience escape hatch for users whose Go data already lives in time.Time. Avro's time-millis and time-micros logical types are time-of-day only — the wire bytes physically cannot represent a date or zone. On encode, only the wall-clock fields (hour/minute/second/nanosecond) are written; year/month/day/location are silently discarded. On decode into a time.Time target, the wire value is materialized at the Unix epoch (1970-01-01 UTC) plus the time-of-day. A round-trip through time.Time therefore preserves the time-of-day but resets the date and zone: 2024-01-15 12:34:56 PST → wire → 1970-01-01 12:34:56 UTC. If round-trip date fidelity matters, use timestamp-millis / timestamp-micros (which preserve the full instant) or convert to and from time.Duration explicitly. time.Duration round-trips exactly when its nanosecond component is a whole multiple of the schema's resolution unit (millisecond for time-millis, microsecond for time-micros); sub-resolution nanoseconds are silently truncated toward zero on encode (integer division by the resolution unit, dropping the remainder).

big-decimal carries no schema-level precision or scale; scale is derived per value, and rationals with no finite decimal expansion (e.g. big.NewRat(1, 3)) return an error.

Unknown logical types are silently ignored per the Avro spec, and the underlying type is used as-is.

Schema Evolution

Avro data is always written with a specific schema — the writer schema. When you read that data later, your application may expect a different schema — the reader schema. You may have added a field, removed one, or widened a type from int to long.

Resolve bridges this gap. Given the writer and reader schemas, it returns a new schema that decodes data in the old wire format and produces values in the reader's layout:

• Fields in the reader but not the writer are filled from defaults.
• Fields in the writer but not the reader are skipped.
• Fields that exist in both are matched by name (or alias) and decoded, with type promotion applied where needed (e.g. int → long).

Example

Suppose v1 of your application wrote User records with just a name:

var writerSchema = avro.MustParse(`{
    "type": "record", "name": "User",
    "fields": [
        {"name": "name", "type": "string"}
    ]
}`)

In v2 you added an email field with a default:

var readerSchema = avro.MustParse(`{
    "type": "record", "name": "User",
    "fields": [
        {"name": "name",  "type": "string"},
        {"name": "email", "type": "string", "default": ""}
    ]
}`)

type User struct {
    Name  string `avro:"name"`
    Email string `avro:"email"`
}

To read old v1 data with your v2 struct, resolve the two schemas:

resolved, err := avro.Resolve(writerSchema, readerSchema)

var u User
_, err = resolved.Decode(v1Data, &u)
// u == User{Name: "Alice", Email: ""}

The following type promotions are supported:

Writer → Reader
int → long, float, double
long → float, double
float → double
string ↔ bytes

CheckCompatibility checks whether two schemas are compatible without building a resolved schema. The direction you check depends on the guarantee you need:

// Backward: new schema can read old data.
avro.CheckCompatibility(oldSchema, newSchema)

// Forward: old schema can read new data.
avro.CheckCompatibility(newSchema, oldSchema)

// Full: check both directions.
avro.CheckCompatibility(oldSchema, newSchema)
avro.CheckCompatibility(newSchema, oldSchema)

Schema Cache

When working with a schema registry, schemas often reference types defined in other schemas. SchemaCache accumulates named types across multiple Parse calls so they can be resolved:

var cache avro.SchemaCache

// Parse referenced schema first — order matters.
_, err := cache.Parse(`{
    "type": "record",
    "name": "Address",
    "fields": [{"name": "city", "type": "string"}]
}`)

// Now parse a schema that references Address.
schema, err := cache.Parse(`{
    "type": "record",
    "name": "User",
    "fields": [
        {"name": "name",    "type": "string"},
        {"name": "address", "type": "Address"}
    ]
}`)

Parsing the same schema string multiple times returns the cached result, handling diamond dependencies without caller-side deduplication. The returned *Schema is independent of the cache and safe to use concurrently.

Custom Types

Register custom Go type conversions with NewCustomType for type-safe primitive conversions, or CustomType for advanced cases:

type Money struct {
    Cents    int64
    Currency string
}

moneyType := avro.NewCustomType[Money, int64]("money",
    func(m Money, _ *avro.SchemaNode) (int64, error) { return m.Cents, nil },
    func(c int64, _ *avro.SchemaNode) (Money, error) {
        return Money{Cents: c, Currency: "USD"}, nil
    },
)

schema := avro.MustParse(moneySchema, moneyType)

// Encode and decode — Money fields are automatically converted.
data, _ := schema.Encode(&order)
var out Order
schema.Decode(data, &out) // out.Price is Money{Cents: 500, ...}

// Works with SchemaFor too.
schema = avro.MustSchemaFor[Order](moneyType)

A matching custom type replaces the built-in logical type deserializer. Decode callbacks receive raw Avro-native values (int64 for long, int32 for int, etc.). A nil Decode suppresses the built-in handler with zero overhead, producing raw values directly:

// Decode timestamps as raw int64 instead of time.Time.
schema := avro.MustParse(raw, avro.CustomType{
    LogicalType: "timestamp-millis",
    AvroType:    "long",
})

For property-based dispatch (e.g., Kafka Connect / Debezium types), use an empty matching criteria with ErrSkipCustomType:

avro.CustomType{
    Decode: func(v any, node *avro.SchemaNode) (any, error) {
        name, _ := node.Props["connect.name"].(string)
        switch name {
        case "io.debezium.time.Timestamp":
            return time.UnixMilli(v.(int64)).UTC(), nil
        default:
            return nil, avro.ErrSkipCustomType
        }
    },
}

Type name constants

The atype sub-package exports constants for Avro primitive type names, complex type names, logical type names, and field sort orders — the string values used in SchemaNode, SchemaField, and CustomType:

import (
    "github.com/twmb/avro"
    "github.com/twmb/avro/atype"
)

node := avro.SchemaNode{
    Type:        atype.Long,
    LogicalType: atype.TimestampMicros,
}

ct := avro.CustomType{LogicalType: atype.Decimal}

They are untyped string constants and can be used anywhere a string is expected. Helps catch typos (atype.TimestampMicros vs "timestam-micros") at compile time.

Object Container Files

The ocf sub-package reads and writes Avro Object Container Files — self-describing binary files that embed the schema in the header and store data in compressed blocks.

Writing

var schema = avro.MustParse(`{
    "type": "record",
    "name": "User",
    "fields": [
        {"name": "name", "type": "string"},
        {"name": "age",  "type": "int"}
    ]
}`)

f, _ := os.Create("users.avro")
w, err := ocf.NewWriter(f, schema, ocf.WithCodec(ocf.SnappyCodec()))
if err != nil {
    log.Fatal(err)
}
w.Encode(&User{Name: "Alice", Age: 30})
w.Encode(&User{Name: "Bob", Age: 25})
w.Close()
f.Close()

Reading

f, _ := os.Open("users.avro")
r, err := ocf.NewReader(f)
if err != nil {
    log.Fatal(err)
}
defer r.Close()
for {
    var u User
    err := r.Decode(&u)
    if err == io.EOF {
        break
    }
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(u)
}

The reader's Schema() method returns the schema parsed from the file header, which you can pass as the writer schema to Resolve.

Codecs

Built-in codecs: null (default, no compression), deflate (DeflateCodec), snappy (SnappyCodec), and zstandard (ZstdCodec). Custom codecs can be provided via the Codec interface.

Memory bounds. WithMaxBlockBytes (default 64 MiB) caps the compressed block size, not the decompressed size — snappy, deflate, and zstd all allocate the declared decompressed length before validating the payload, so a malicious block can drive a larger allocation than the cap allows. For OCF read from untrusted sources, bound memory at the transport or process layer (request size cap, cgroup limit).

Appending

NewAppendWriter opens an existing OCF for appending — it reads the header to recover the schema, codec, and sync marker, then seeks to the end.

JSON Encoding

EncodeJSON is a schema-aware JSON serializer. By default it produces standard JSON with bare union values and \uXXXX-encoded bytes:

// Standard JSON (default): bare unions
jsonBytes, err := schema.EncodeJSON(&user)
// {"name":"Alice","email":"a@b.com"}

// Avro JSON: unions wrapped as {"type_name": value}
jsonBytes, err = schema.EncodeJSON(&user, avro.TaggedUnions())
// {"name":"Alice","email":{"string":"a@b.com"}}

The tagged form is the Avro JSON spec's representation and what Java and other Avro tools require; the bare default is plainer JSON for non-Avro consumers.

DecodeJSON accepts both formats (tagged and bare unions) and all NaN/Infinity conventions:

var user User
err = schema.DecodeJSON(jsonBytes, &user)

Decode and DecodeJSON also accept TaggedUnions() to wrap union values when decoding into *any:

var native any
schema.Decode(binary, &native, avro.TaggedUnions())
// native["email"] is map[string]any{"string": "a@b.com"}

Encode and DecodeJSON accept both tagged and bare union input, so tagged union output from Decode can round-trip through Encode directly.

Pass TagLogicalTypes() with TaggedUnions() to qualify union branch names with their logical type (e.g. "long.timestamp-millis" instead of "long"), matching the linkedin/goavro naming convention.

NaN and Infinity float values are encoded as "NaN", "Infinity", "-Infinity" strings by default (Java Avro convention). Pass LinkedinFloats() for the linkedin/goavro convention (null for NaN, ±1e999 for Infinity).

Single Object Encoding

For sending self-describing values over the wire (as opposed to files, where OCF is preferred), use Single Object Encoding. Each message is a 2-byte magic header, an 8-byte CRC-64-AVRO fingerprint, and the Avro binary payload.

// Encode with fingerprint header
data, err := schema.AppendSingleObject(nil, &user)

// Decode (schema known)
_, err = schema.DecodeSingleObject(data, &user)

// Decode (schema unknown): extract fingerprint, look up schema
fp, payload, err := avro.SingleObjectFingerprint(data)
schema := registry.Lookup(fp) // your schema registry
_, err = schema.Decode(payload, &user)

Fingerprinting

Canonical returns the Parsing Canonical Form of a schema — a deterministic JSON representation stripped of doc, aliases, defaults, and other non-essential attributes. Use it for schema comparison and fingerprinting.

canonical := schema.Canonical() // []byte

// CRC-64-AVRO (Rabin) — the Avro-standard fingerprint
fp := schema.Fingerprint(avro.NewRabin())

// SHA-256 — common for cross-language registries
fp256 := schema.Fingerprint(sha256.New())

Errors

Encode and decode errors can be inspected with errors.As:

• *SemanticError: type mismatch between Go and Avro (includes a dotted field path for nested records, e.g. "address.zip").
• *ShortBufferError: input truncated mid-value.
• *CompatibilityError: schema evolution incompatibility (from Resolve or CheckCompatibility).

Performance

Struct field access uses unsafe pointer arithmetic (similar to encoding/json v2) to avoid reflect.Value overhead on every encode/decode. All schemas, type mappings, and codec state are cached after first use so repeated operations pay no extra allocation cost.

Encode/decode behavior contract

The encoder and decoder are mostly symmetric: any Go shape the encoder accepts as input is a Go shape the decoder accepts as a target, and an encode→decode round-trip through the same Go type yields the same value. The cases below are deliberate exceptions.

Round-trips that lose data

• time.Time → time-millis / time-micros keeps only the time-of-day; date and zone are dropped. The decoded time.Time sits at the Unix epoch with the original time-of-day. Use timestamp-* to keep the full instant.
• time.Time → date keeps only the UTC date; time-of-day and zone are dropped.
• big-decimal trailing-zero scale isn't preserved — scale is derived per value, and big.Rat can't carry the distinction.

Numbers

• Precision follows the reader schema. A float/double schema rounds silently into a Go float (overflowing to ±Inf on encode), but decoding one into a Go integer still requires a whole number in range. An int/long schema never loses precision silently — decoding into a Go type that can't hold the value exactly errors. For exact large-integer round-trips keep the schema long with an int64 target; evolving to double opts into rounding.
• Union branch is chosen by the Go type, not the value. A Go int always selects a union's long branch, never int, even when the value would fit — dispatch keys off the static type, keeping wire size deterministic. Force the int branch with an int32 value or map[string]any{"int": v}.
• Encoding a json.Number into an int/long works even in fractional or exponent form, as long as the value is whole — json.Number("9.5e17") succeeds (it's exactly 950000000000000000). Caveat: the same literal as a schema default ("default":9.5e17) won't load in Java; use the plain integer form if you publish schemas to Java consumers.

Why

I had efficient encode-side code lying around for two years. I'd written it one way, started rewriting it another, got most of the way through, and decided it wasn't worth the effort — hamba/avro was around by then and really good, so I stopped.

When I started using LLMs in January 2026, I figured I'd touch up all the old projects I had lying around. Did it for this one — and coincidentally, hamba/avro got archived right around then. I wanted one library that did it all: hamba/avro missed things linkedin/goavro had, and vice versa. Here we are.

Documentation

Overview

Package avro encodes and decodes Avro specification data.

Parse an Avro JSON schema with Parse (or MustParse for package-level vars), then call Schema.Encode / Schema.Decode for binary encoding, or Schema.EncodeJSON / Schema.DecodeJSON for JSON encoding. Use SchemaFor to infer a schema from a Go struct type, or Schema.Root to inspect a parsed schema's structure.

Basic usage

schema := avro.MustParse(`{
    "type": "record",
    "name": "User",
    "fields": [
        {"name": "name", "type": "string"},
        {"name": "age",  "type": "int"}
    ]
}`)

type User struct {
    Name string `avro:"name"`
    Age  int    `avro:"age"`
}

// Encode
data, err := schema.Encode(&User{Name: "Alice", Age: 30})

// Decode
var u User
_, err = schema.Decode(data, &u)

JSON encoding

Schema.EncodeJSON is schema-aware and handles bytes, unions, and NaN/Infinity floats correctly — use it instead of a generic JSON encoder when serializing decoded Avro data to JSON. Options control the output format: TaggedUnions for Avro JSON union wrappers ({"type": value}), TagLogicalTypes for qualified branch names, and LinkedinFloats for the goavro NaN/Infinity convention.

Encoding from JSON input

Generically-decoded JSON data (map[string]any with float64 numbers and string timestamps) can be encoded directly. Missing map keys are filled from schema defaults, encoding/json.Number is accepted for numeric Avro types only (string, bytes, fixed, and enum reject it — use a Go string or []byte for those), and timestamp fields accept RFC 3339 strings. String fields accept encoding.TextAppender and encoding.TextMarshaler implementations (with encoding.TextUnmarshaler on decode).

Schema evolution

Avro data is always written with a specific schema — the "writer schema." When you read it later your application may expect a different one — the "reader schema" — having added a field, removed one, or widened an int to a long.

Resolve bridges the two: given writer and reader, it returns a schema that decodes the old wire format into the reader's layout.

• Fields in the reader but not the writer are filled from defaults.
• Fields in the writer but not the reader are skipped.
• Fields that exist in both are matched by name (or alias) and decoded, with type promotion applied where needed (e.g. int → long).

You typically get the writer schema from the data itself: an OCF file header embeds it, and schema registries store it by ID or fingerprint.

As a concrete example, suppose v1 of your application wrote User records with just a name:

var writerSchema = avro.MustParse(`{
    "type": "record", "name": "User",
    "fields": [
        {"name": "name", "type": "string"}
    ]
}`)

In v2, you added an email field with a default:

var readerSchema = avro.MustParse(`{
    "type": "record", "name": "User",
    "fields": [
        {"name": "name",  "type": "string"},
        {"name": "email", "type": "string", "default": ""}
    ]
}`)

type User struct {
    Name  string `avro:"name"`
    Email string `avro:"email"`
}

To read old v1 data with your v2 struct, resolve the two schemas:

resolved, err := avro.Resolve(writerSchema, readerSchema)

// Decode v1 data: "email" is absent in the old data, so it gets
// the reader default ("").
var u User
_, err = resolved.Decode(v1Data, &u)
// u == User{Name: "Alice", Email: ""}

If you just want to check whether two schemas are compatible without building a resolved schema, use CheckCompatibility.

A null union branch decodes to the target's Go zero value, always replacing any prior value. Use *T to distinguish null from zero.

The reader schema is your contract for precision. A lossy reader schema — float or double — silently IEEE-rounds on both encode and decode, and an out-of-range finite input becomes ±Inf on the wire. An exact one — int, long, bytes, string — requires the Go decode target to hold the wire value without loss: a value outside the target's range, or one it cannot represent exactly such as a long above 2^53 decoded into a float64, is an error. For exact round-trip of large integers choose a long reader schema with an int64 target rather than relying on a float to round.

Struct tags

Use the "avro" struct tag to control field mapping and schema inference. The format is avro:"[name][,option]..." where the name maps the Go field to the Avro field name (empty = use Go field name, "-" = exclude).

Encoding/decoding options:

avro:"name"           // map to Avro field "name"
avro:"-"              // exclude field
avro:",inline"        // flatten nested struct fields into parent record
avro:",omitzero"      // encode a zero value as the field's default (or null)

Schema inference options (used by SchemaFor):

avro:",default=value"         // set field default (must be last option; scalars only)
avro:",alias=old_name"        // field alias for evolution (repeatable, or alias=[a,b])
avro:",type-alias=old_name"   // named type alias (record/enum/fixed) for evolution (repeatable, or type-alias=[a,b])
avro:",timestamp-micros"      // override logical type (also: timestamp-nanos, date, time-millis, time-micros)
avro:",decimal(10,2)"         // decimal logical type with precision and scale
avro:",uuid"                  // UUID logical type

The alias tag adds an alias to the field itself. The type-alias tag adds an alias to the named type (record, enum, or fixed) that the field references, walking through pointers, slices, and maps to find it. This is needed when a writer schema uses a different name for the same type — for example, a legacy schema naming a record "r508" instead of "FieldSummary".

When encoding a map[string]any as a record, missing keys fill from the schema's defaults. A ["null", T] field declared without a default has an implicit null default, so a missing key there fills null rather than erroring. The omitzero tag applies the same fill to a struct's zero-valued fields, and to fields whose IsZero() reports true: a zero value encodes the field's default, or null for a nullable field with no default, or — for a non-nullable field with no default — the zero value itself, there being nothing to fill with. It differs from map fill in one case, a [T, "null"] union declared without a default: no null default can exist there, since a union default must match the first branch, so omitzero encodes null where map fill errors on the missing key.

Embedded (anonymous) struct fields are inlined automatically; an explicit name tag prevents it. When several fields resolve to one name, a tagged field wins over an untagged one at any depth, and among equally tagged fields the shallowest wins. Two fields at the same depth with the same tagged status are an ambiguous collision, and this package errors rather than pick one: SchemaFor rejects the type, while encode and decode reject only when the schema actually resolves a field to that name, so a coincidental collision on a name the schema never references does not break the type.

Custom types

CustomType registers custom Go type conversions for logical types, domain types, or to replace built-in behavior. A matching custom type replaces the built-in logical type deserializer — Decode callbacks receive raw Avro-native values, not enriched types like time.Time. A CustomType with nil Decode suppresses the built-in handler with zero overhead, producing raw values directly. Use NewCustomType for type-safe primitive conversions, or the CustomType struct directly for complex cases (records, fixed types, property-based dispatch). Custom types are registered per-schema via SchemaOpt.

Parsing options

Parse and SchemaCache.Parse accept WithLaxNames to allow non-standard characters in type and field names.

Errors

Encode and decode errors can be inspected with errors.As:

• *SemanticError: type mismatch (includes a dotted field path for nested records)
• *ShortBufferError: input truncated mid-value
• *CompatibilityError: schema evolution incompatibility

Other features

• Schema Cache: SchemaCache accumulates named types across Parse calls for schema registry workflows
• Schema Introspection: Schema.Root returns a SchemaNode; Schema.String returns the original JSON
• Single Object Encoding: Schema.AppendSingleObject, Schema.DecodeSingleObject
• Fingerprinting: Schema.Canonical, Schema.Fingerprint, NewRabin
• Object Container Files: the github.com/twmb/avro/ocf sub-package

The repository README's "Encode/decode behavior contract" section documents the intentional asymmetries between the encoder and decoder (lossy-by-design conversions, spec/interop choices, and decoder-only leniencies).

Example

package main

import (
	"fmt"
	"log"

	"github.com/twmb/avro"
)

func main() {
	schema := avro.MustParse(`{
		"type": "record",
		"name": "User",
		"fields": [
			{"name": "name", "type": "string"},
			{"name": "age",  "type": "int"}
		]
	}`)

	type User struct {
		Name string `avro:"name"`
		Age  int32  `avro:"age"`
	}

	data, err := schema.Encode(&User{Name: "Alice", Age: 30})
	if err != nil {
		log.Fatal(err)
	}

	var u User
	if _, err := schema.Decode(data, &u); err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%s is %d\n", u.Name, u.Age)
}

Output:
Alice is 30

Index

• Variables
• func CheckCompatibility(writer, reader *Schema) error
• func NewRabin() hash.Hash64
• func RatFromBytes(b []byte, scale int) *big.Rat
• func SingleObjectFingerprint(data []byte) (fp [8]byte, rest []byte, err error)
• type CompatibilityError
  • func (e *CompatibilityError) Error() string
• type CustomType
  • func NewCustomType[G, A any](logicalType string, encode func(G, *SchemaNode) (A, error), ...) CustomType
• type Duration
  • func DurationFromBytes(b []byte) Duration
  • func (d Duration) Bytes() [12]byte
  • func (d Duration) String() string
• type Opt
  • func LinkedinFloats() Opt
  • func TagLogicalTypes() Opt
  • func TaggedUnions() Opt
• type Schema
  • func MustParse(schema string, opts ...SchemaOpt) *Schema
  • func MustSchemaFor[T any](opts ...SchemaOpt) *Schema
  • func Parse(schema string, opts ...SchemaOpt) (*Schema, error)
  • func Resolve(writer, reader *Schema) (*Schema, error)
  • func SchemaFor[T any](opts ...SchemaOpt) (*Schema, error)
  • func (s *Schema) AppendEncode(dst []byte, v any, opts ...Opt) ([]byte, error)
  • func (s *Schema) AppendEncodeJSON(dst []byte, v any, opts ...Opt) ([]byte, error)
  • func (s *Schema) AppendSingleObject(dst []byte, v any, opts ...Opt) ([]byte, error)
  • func (s *Schema) Canonical() []byte
  • func (s *Schema) Decode(src []byte, v any, opts ...Opt) ([]byte, error)
  • func (s *Schema) DecodeJSON(src []byte, v any, opts ...Opt) error
  • func (s *Schema) DecodeSingleObject(data []byte, v any, opts ...Opt) ([]byte, error)
  • func (s *Schema) Encode(v any, opts ...Opt) ([]byte, error)
  • func (s *Schema) EncodeJSON(v any, opts ...Opt) ([]byte, error)
  • func (s *Schema) Fingerprint(h hash.Hash) []byte
  • func (s *Schema) Root() *SchemaNode
  • func (s *Schema) String() string
• type SchemaCache
  • func (c *SchemaCache) Parse(schema string, opts ...SchemaOpt) (*Schema, error)
• type SchemaField
• type SchemaNode
  • func (n *SchemaNode) Schema(opts ...SchemaOpt) (*Schema, error)
• type SchemaOpt
  • func WithCustomType(ct CustomType) SchemaOpt
  • func WithLaxNames(fn func(string) error) SchemaOpt
  • func WithName(name string) SchemaOpt
  • func WithNamespace(ns string) SchemaOpt
• type SemanticError
  • func (e *SemanticError) Error() string
  • func (e *SemanticError) Unwrap() error
• type ShortBufferError
  • func (e *ShortBufferError) Error() string

Examples

• Package
• CustomType (Override)
• CustomType (PropertyDispatch)
• CustomType (SchemaFor)
• NewCustomType
• Resolve
• Schema.AppendEncode
• Schema.AppendSingleObject
• Schema.DecodeJSON
• Schema.Encode (TextMarshaler)
• Schema.EncodeJSON
• SchemaCache
• SchemaFor

Variables

var ErrSkipCustomType = errors.New("avro: skip custom type")

ErrSkipCustomType is returned from a CustomType Encode or Decode function to indicate the value is not handled by this custom type. The library falls through to the next matching custom type or to built-in behavior.

Functions

func CheckCompatibility

func CheckCompatibility(writer, reader *Schema) error

CheckCompatibility reports whether data written with the writer schema can be read by the reader schema. It returns nil on success or a *CompatibilityError describing the first incompatibility.

See Resolve for a note on argument order.

func NewRabin

func NewRabin() hash.Hash64

NewRabin returns a hash.Hash64 computing the CRC-64-AVRO (Rabin) fingerprint defined by the Avro specification.

func RatFromBytes

func RatFromBytes(b []byte, scale int) *big.Rat

RatFromBytes converts Avro decimal bytes (big-endian two's complement) to *big.Rat with the given scale — the conversion a CustomType Decode callback overriding the built-in decimal handling would otherwise write itself, since such a callback receives the raw []byte.

Negative scale is interpreted as `unscaled * 10^|scale|` (matching Java/avro-rs big-decimal semantics). |scale| is bounded by decimalScaleLimit and the unscaled byte length by maxDecimalUnscaledBytes; inputs beyond either bound produce a zero *big.Rat rather than allocating / base-converting unbounded.

func SingleObjectFingerprint

func SingleObjectFingerprint(data []byte) (fp [8]byte, rest []byte, err error)

SingleObjectFingerprint extracts the 8-byte CRC-64-AVRO fingerprint and returns the remaining payload.

Types

type CompatibilityError

type CompatibilityError struct {
	// Path is the dotted path to the incompatible element (e.g. "User.address.zip").
	Path string
	// ReaderType is the Avro type in the reader schema.
	ReaderType string
	// WriterType is the Avro type in the writer schema.
	WriterType string
	// Detail describes the specific incompatibility.
	Detail string
}

CompatibilityError describes an incompatibility between a reader and writer schema, as returned by CheckCompatibility and Resolve.

func (*CompatibilityError) Error

func (e *CompatibilityError) Error() string

type CustomType

type CustomType struct {
	// LogicalType narrows matching to schema nodes with this logicalType.
	LogicalType string

	// AvroType narrows matching to schema nodes of this Avro type
	// (e.g. "long", "bytes", "record"). Also used by SchemaFor to
	// infer the underlying Avro type.
	AvroType string

	// GoType adds an encode-time filter: when set, the Encode function
	// only fires when the value's concrete type matches GoType. Values
	// of other types pass through to the underlying serializer unchanged.
	// If nil, Encode fires for all values on matched schema nodes
	// (those matching LogicalType/AvroType).
	//
	// [SchemaFor] uses GoType to match struct fields: when a field's Go
	// type equals GoType, SchemaFor emits AvroType + LogicalType (or
	// Schema) instead of the default type mapping. Because the custom
	// supplies the whole field schema, a logical-type tag on a matched
	// field has no effect and is rejected — set LogicalType (or Schema)
	// here instead. If nil, the custom type does not affect schema
	// generation, but is still wired into the returned [*Schema] for
	// encode/decode.
	GoType reflect.Type

	// Schema is the full schema to emit in SchemaFor. Only needed for
	// types requiring extra metadata (fixed needs name+size, decimal
	// needs precision+scale, records need fields). If nil, SchemaFor
	// infers from AvroType + LogicalType.
	//
	// SchemaFor preserves every fullname the schema declares: a
	// namespaced type keeps its namespace, and a null-namespace type
	// embedded under [WithNamespace] keeps its null namespace (the
	// emitted definition carries the "namespace":"" inheritance escape).
	// One combination is unrepresentable and errors: a null-namespace
	// type used on two or more fields under WithNamespace, because Avro
	// has no reference spelling that reaches the null namespace from
	// inside another namespace.
	//
	// SchemaFor composes a private copy of the rendered schema, so the
	// SchemaNode and everything reachable from it (including Props
	// container values) are never mutated by a build, and it fails the
	// build with the walk's named error when the schema exceeds the
	// schema-tree budgets or contains an unnamed pointer cycle.
	//
	// A union branch's type may be written in either spelling Avro
	// admits — the bare name ("null") or the wrapped object
	// ({"type":"null"}) — and composition treats the two as the one type
	// they are. A null branch is recognized in both spellings, and in the
	// wrapped form regardless of any properties or logicalType it carries
	// (Avro defines no null logical type, so both are inert): a nullable
	// union collapses through a pointer field and receives its null
	// default identically either way.
	Schema *SchemaNode

	// Encode converts a caller-provided Go value to an Avro-native
	// value, called before serialization. The callback receives the
	// value as passed to [Schema.Encode] (e.g. a custom Money type),
	// and should return the corresponding Avro-native value (e.g.
	// int64 cents). Return [ErrSkipCustomType] to fall through to the
	// next matching custom type or built-in behavior. Any other
	// non-nil error is fatal.
	//
	// If nil, the built-in logical type encoder is used, which accepts
	// both enriched types ([time.Time], [time.Duration]) and raw
	// values (int64, int32, etc.).
	//
	// The schema argument is built once at Parse and shared across all
	// concurrent invocations. Treat it as read-only; in particular, do
	// not mutate schema.Props or schema.Symbols — those slices/maps
	// alias the parser's internal state and concurrent writes from
	// multiple goroutines decoding the same [*Schema] will race.
	Encode func(v any, schema *SchemaNode) (any, error)

	// Decode converts a raw Avro-native value to a custom Go value,
	// called after deserialization. The callback receives the raw
	// Avro-native value (int32 for int, int64 for long, []byte for
	// bytes/fixed, etc.) and should return the desired Go type.
	// Return [ErrSkipCustomType] to fall through. Any other non-nil
	// error is fatal.
	//
	// When all matching decoders skip at a node, the wire is re-decoded
	// into the target faithfully (identical to a no-custom decode); a
	// wildcard custom (empty LogicalType and AvroType) that matches leaf
	// nodes but skips containers therefore makes decoding into a
	// deeply-nested TYPED target (struct/slice/map) cost O(depth^2) — for
	// untrusted deeply-nested data decode into an interface / map[string]any
	// (single-pass) or register against a specific LogicalType/AvroType.
	//
	// If nil, the built-in logical type handler is bypassed and the
	// base Avro type decoder is used directly, producing raw
	// Avro-native values (int32, int64, etc.) rather than enriched
	// types ([time.Time], [time.Duration], etc.).
	//
	// The schema argument is shared across concurrent callback invocations;
	// see [CustomType.Encode] for the read-only contract.
	Decode func(v any, schema *SchemaNode) (any, error)
	// contains filtered or unexported fields
}

CustomType defines a custom conversion between a Go type and an Avro type. NewCustomType covers the primitive-backing-type case with the wiring inferred from its type parameters; this struct is the general form, and the only one that reaches records, fixed types, and property-based dispatch.

Pass to Parse or SchemaFor as a SchemaOpt.

Matching at parse time: LogicalType and AvroType are checked against schema nodes. All non-empty criteria must match.

• LogicalType only: matches any schema node with that logicalType
• LogicalType + AvroType: matches that logicalType on that Avro type
• AvroType only: matches all nodes of that Avro type
• Neither: matches ALL schema nodes (use with ErrSkipCustomType for property-based dispatch like Kafka Connect types)

At encode time, GoType is also checked: the Encode function only fires when the value's type matches GoType. This prevents the codec from intercepting native values (e.g. a raw int64 passes through without conversion for a custom-typed long field).

A matching CustomType replaces the built-in logical type deserializer. Among user registrations, first match wins.

Backed by a complex Avro type, the Encode function returns map[string]any, []any, and so on.

Example (Override)

package main

import (
	"fmt"

	"github.com/twmb/avro"
)

func main() {
	// Use CustomType directly to override a built-in logical type handler.
	// Here we suppress the timestamp-millis → time.Time conversion and
	// keep the raw int64 epoch millis.
	schema := avro.MustParse(`{
		"type": "record", "name": "Event",
		"fields": [
			{"name": "ts", "type": {"type": "long", "logicalType": "timestamp-millis"}}
		]
	}`, avro.CustomType{
		LogicalType: "timestamp-millis",
		Decode: func(v any, _ *avro.SchemaNode) (any, error) {
			return v, nil // pass through raw int64
		},
	})

	data, _ := schema.Encode(map[string]any{"ts": int64(1767225600000)})
	var out any
	schema.Decode(data, &out)
	m := out.(map[string]any)
	fmt.Printf("ts type: %T\n", m["ts"])
}

Output:
ts type: int64

Example (PropertyDispatch)

package main

import (
	"fmt"

	"github.com/twmb/avro"
)

func main() {
	// CustomType with no LogicalType/AvroType/GoType matches ALL schema
	// nodes. Use ErrSkipCustomType to selectively handle nodes based on
	// schema properties, e.g. Kafka Connect type annotations.
	ct := avro.CustomType{
		Decode: func(v any, node *avro.SchemaNode) (any, error) {
			if node.Props["connect.type"] == "double-it" {
				return v.(int64) * 2, nil
			}
			return nil, avro.ErrSkipCustomType
		},
	}

	// Properties on the type object are available via node.Props in the
	// custom type callback.
	schema := avro.MustParse(`{
		"type": "record", "name": "R",
		"fields": [
			{"name": "x", "type": {"type": "long", "connect.type": "double-it"}},
			{"name": "y", "type": "long"}
		]
	}`, ct)

	data, _ := schema.Encode(map[string]any{"x": int64(5), "y": int64(5)})
	var out any
	schema.Decode(data, &out)
	m := out.(map[string]any)
	fmt.Printf("x=%d y=%d\n", m["x"], m["y"])
}

Output:
x=10 y=5

Example (SchemaFor)

package main

import (
	"fmt"
	"log"
	"reflect"

	"github.com/twmb/avro"
)

func main() {
	// Setting GoType lets SchemaFor infer the Avro schema for struct
	// fields of that type. Without GoType, SchemaFor doesn't know that
	// a Cents field should map to {"type":"long","logicalType":"money"}.
	type Cents int64
	ct := avro.CustomType{
		LogicalType: "money",
		AvroType:    "long",
		GoType:      reflect.TypeFor[Cents](),
		Encode: func(v any, _ *avro.SchemaNode) (any, error) {
			return int64(v.(Cents)), nil
		},
		Decode: func(v any, _ *avro.SchemaNode) (any, error) {
			return Cents(v.(int64)), nil
		},
	}

	type Order struct {
		Price Cents `avro:"price"`
	}
	schema, err := avro.SchemaFor[Order](ct)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(schema.Root().Fields[0].Type.LogicalType)
}

Output:
money

func NewCustomType

func NewCustomType[G, A any](
	logicalType string,
	encode func(G, *SchemaNode) (A, error),
	decode func(A, *SchemaNode) (G, error),
) CustomType

NewCustomType returns a type-safe CustomType for the common case of mapping a custom Go type to/from a primitive Avro type.

G is the custom Go type (e.g. Money). A is the Avro-native Go type: int32 for int, int64 for long, float32 for float, float64 for double, string for string, []byte for bytes, bool for boolean. A may also be a named type whose underlying kind is one of these (e.g. type Cents int64); the Avro type is inferred from A's kind and the decoded value is converted to A.

GoType and AvroType are inferred from the type parameters. If A is not a supported Avro-native type, Parse or SchemaFor returns an error.

Note: AvroType is inferred from A's Go kind, which may not match the Avro schema's type for logical types backed by smaller types. For example, time-millis uses Avro "int" but time.Duration is int64 (which infers "long"). Use int32 as A, or use the CustomType struct directly with an explicit AvroType.

For fixed, records, or types needing extra schema metadata, use the CustomType struct directly.

Example

package main

import (
	"fmt"
	"log"

	"github.com/twmb/avro"
)

type ExMoney struct {
	Cents int64
}

func main() {
	// NewCustomType is the easiest way to map a custom Go type to and from a
	// primitive Avro type: G is your Go type, A the Avro-native Go type it maps
	// to, and A decides the wire type —
	//   int32 → int    float32 → float    bool   → boolean
	//   int64 → long   float64 → double   string → string   []byte → bytes
	// The first argument is the logicalType to match; "" matches all schema
	// nodes of the inferred Avro type.
	moneyType := avro.NewCustomType[ExMoney, int64]("money",
		func(m ExMoney, _ *avro.SchemaNode) (int64, error) { return m.Cents, nil },
		func(c int64, _ *avro.SchemaNode) (ExMoney, error) { return ExMoney{Cents: c}, nil },
	)

	schema := avro.MustParse(`{
		"type": "record", "name": "Order",
		"fields": [
			{"name": "price", "type": {"type": "long", "logicalType": "money"}}
		]
	}`, moneyType)

	type Order struct {
		Price ExMoney `avro:"price"`
	}

	data, err := schema.Encode(&Order{Price: ExMoney{Cents: 1999}})
	if err != nil {
		log.Fatal(err)
	}

	var out Order
	if _, err := schema.Decode(data, &out); err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%d cents\n", out.Price.Cents)
}

Output:
1999 cents

type Duration

type Duration struct {
	Months       uint32
	Days         uint32
	Milliseconds uint32
}

Duration represents the Avro duration logical type: a 12-byte fixed value containing three little-endian unsigned 32-bit integers representing months, days, and milliseconds.

func DurationFromBytes

func DurationFromBytes(b []byte) Duration

DurationFromBytes decodes a 12-byte little-endian fixed value into a Duration. Returns zero Duration if b is shorter than 12 bytes. This is useful in CustomType Decode callbacks that override the default duration handling: the callback receives raw []byte and can use this function to interpret the value before converting to a custom Go type.

func (Duration) Bytes

func (d Duration) Bytes() [12]byte

Bytes encodes the Duration as a 12-byte little-endian fixed value, matching the Avro duration wire format.

func (Duration) String

func (d Duration) String() string

String returns an ISO 8601 duration string. Zero components are omitted. Examples: "P1Y3M15DT1H30M0.500S", "P30D", "PT1H".

type Opt

type Opt interface {
	// contains filtered or unexported methods
}

Opt configures encoding and decoding behavior. See each option's documentation for which functions it affects. Inapplicable options are silently ignored.

func LinkedinFloats

func LinkedinFloats() Opt

LinkedinFloats encodes NaN as JSON null and ±Infinity as ±1e999 in Schema.EncodeJSON, matching the linkedin/goavro convention. Without this option, NaN is encoded as the JSON string "NaN" and ±Infinity as "Infinity"/"-Infinity", following the Java Avro convention.

Schema.DecodeJSON accepts both conventions for a float/double decoded directly or as a tagged union branch ({"float":null} → NaN). One exception: a NaN inside a BARE union does not round-trip. It encodes as a bare null, and on decode the union's null branch claims that null — or rejects it, if the union has none — before the float branch is tried. The ambiguity is inherent to the null-for-NaN convention when null is also a structural union value; use TaggedUnions for a round-trip-safe NaN union member. ±Infinity is a number token and round-trips in a bare union regardless.

func TagLogicalTypes

func TagLogicalTypes() Opt

TagLogicalTypes qualifies union branch names with their logical type (e.g. "long.timestamp-millis" instead of "long"). This applies to Schema.EncodeJSON with TaggedUnions and to Schema.Decode with TaggedUnions. Without this option, branch names use the base Avro type per the specification. This option has no effect without TaggedUnions.

func TaggedUnions

func TaggedUnions() Opt

TaggedUnions wraps non-null union values as {"type_name": value}.

Schema.EncodeJSON emits the tagged form. Schema.Decode and Schema.DecodeJSON wrap union values as map[string]any{branchName: value}, but only when the decode target is *any. A typed target — a concrete struct field, *T, or a non-empty interface — cannot hold the wrapper, so it receives the bare branch value.

Schema.DecodeJSON and Schema.Encode accept both tagged and bare union input regardless of this option.

Spec note: Avro 1.12 defines non-null union values as {"type_name": value}. The default here emits bare values, which Java's JsonDecoder and fastavro's JSON decoder both REJECT ("Expected start-union" or equivalent) on the first non-null union field. Pass TaggedUnions for interop with Java, fastavro, or avro-tools fromjson. The bare default serves goavro's bare-JSON codecs (NewCodecForStandardJSON / NewCodecForStandardJSONFull; goavro's plain codec wants the tagged form too) and the natural Go map[string]any shape. See AVRO-2899 for the upstream discussion.

Branch identity: a bare union value does not name its branch, so Schema.DecodeJSON cannot tell which branch the writer used when several share a JSON token class — a bare 7 matches int, long, float and double; a bare "x" matches string, bytes, fixed and enum. The decoder commits to the FIRST declaration-order branch of that class, for every target shape. That can differ from the writer's branch, and it silently bypasses a CustomType registered on a later branch of the same class: its Decode never runs, and a typed target is filled by plain coercion from the first branch's value. Binary Schema.Decode is unaffected — the wire carries the branch index. When branch identity or a branch-bound CustomType matters, encode AND decode with TaggedUnions; the envelope names the branch and decode dispatches to the writer's branch exactly as binary does.

type Schema

type Schema struct {
	// contains filtered or unexported fields
}

Schema is a compiled Avro schema. Create one with Parse or MustParse, then use Schema.Encode / Schema.Decode to convert between Go values and Avro binary. A Schema is safe for concurrent use.

func MustParse

func MustParse(schema string, opts ...SchemaOpt) *Schema

MustParse is like Parse but panics on error.

func MustSchemaFor

func MustSchemaFor[T any](opts ...SchemaOpt) *Schema

MustSchemaFor is like SchemaFor but panics on error.

func Parse

func Parse(schema string, opts ...SchemaOpt) (*Schema, error)

Parse parses an Avro JSON schema string and returns a compiled *Schema. The input can be a primitive name (e.g. `"string"`), a JSON object (record, enum, array, map, fixed), or a JSON array (union). Named types may self-reference. The schema is fully validated: unknown types, duplicate names, invalid defaults, etc. all return errors.

To parse schemas that reference named types from other schemas, use SchemaCache.

func Resolve

func Resolve(writer, reader *Schema) (*Schema, error)

Resolve returns a schema that decodes data written with the writer schema and produces values matching the reader schema's layout. The writer schema is what the data was encoded with (typically from an OCF file header or a schema registry); the reader schema is what your application expects now.

Decoding with the returned schema handles field addition (defaults), field removal (skip), renaming (aliases), reordering, and type promotion. Encoding with it uses the reader's format.

CheckCompatibility runs first, and any incompatibility comes back as a *CompatibilityError; if it passes and the canonical forms are identical, reader is returned as-is. The check must precede that fast path: the parsing canonical form strips logicalType, precision and scale, so two schemas with equal canonical forms can still be logically incompatible — a decimal precision/scale mismatch most of all — and would otherwise pass the fast path and silently rescale the decoded value.

Note: the argument order is (writer, reader), matching source-then-destination convention and Java's GenericDatumReader. This differs from the Avro spec text and hamba/avro, which put reader first.

Example

package main

import (
	"fmt"
	"log"

	"github.com/twmb/avro"
)

func main() {
	// v1 wrote User with just a name.
	writerSchema := avro.MustParse(`{
		"type": "record", "name": "User",
		"fields": [{"name": "name", "type": "string"}]
	}`)

	// v2 added an email field with a default.
	readerSchema := avro.MustParse(`{
		"type": "record", "name": "User",
		"fields": [
			{"name": "name",  "type": "string"},
			{"name": "email", "type": "string", "default": ""}
		]
	}`)

	resolved, err := avro.Resolve(writerSchema, readerSchema)
	if err != nil {
		log.Fatal(err)
	}

	// Encode a v1 record (name only).
	v1Data, err := writerSchema.Encode(map[string]any{"name": "Alice"})
	if err != nil {
		log.Fatal(err)
	}

	// Decode old data into the new layout; email gets the default.
	type User struct {
		Name  string `avro:"name"`
		Email string `avro:"email"`
	}
	var u User
	if _, err := resolved.Decode(v1Data, &u); err != nil {
		log.Fatal(err)
	}
	fmt.Printf("name=%s email=%q\n", u.Name, u.Email)
}

Output:
name=Alice email=""

func SchemaFor

func SchemaFor[T any](opts ...SchemaOpt) (*Schema, error)

SchemaFor infers an Avro schema from the Go type T. T must be a struct.

Field names are taken from the avro struct tag, falling back to the Go field name. The following tag options are supported:

• avro:"-" excludes the field
• avro:",inline" flattens a nested struct's fields into the parent
• avro:",omitzero" is recorded but does not affect the schema
• avro:",alias=old_name" adds a field alias (repeatable)
• avro:",type-alias=old_name" adds an alias to the field's named type (record, enum, fixed; repeatable)
• avro:",default=value" sets the field's default value (must be last option; scalars only)
• avro:",timestamp-millis" overrides the logical type (also: timestamp-micros, timestamp-nanos, date, time-millis, time-micros)
• avro:",decimal(precision,scale)" sets the decimal logical type
• avro:",uuid" sets the uuid logical type

Type inference:

• bool → boolean
• int8, int16, int32 → int
• int, int64, uint32 → long
• uint8, uint16 → int
• float32 → float
• float64 → double
• string → string
• []byte → bytes
• [N]byte → fixed (size N, name from Go type name or "fixed_N")
• *T → ["null", T] union with default null (a pointer chain of any depth — **T, ***T — collapses to the same single nullable union)
• []T → array
• map[string]T → map
• struct → record (recursive)
• time.Time → long with timestamp-millis (override with tag)
• time.Duration → int with time-millis (override with tag; a Duration is a span of time, so it is only meaningful with the time-millis/time-micros logicals — overriding it onto date or a timestamp-* logical maps a duration onto a point in time, and a large Duration overflows the narrower wire type)
• avro.Duration → fixed(12) with the duration logical type (the dedicated Go type for the Avro duration logical — little-endian months/days/ milliseconds; recognized by type, takes no tag, and does not accept one)
• *big.Rat → requires explicit decimal(p,s) tag
• [16]byte with uuid tag → fixed(16) with uuid logical type
• string (or text marshaler type) with uuid tag → string with uuid logical type

Example

package main

import (
	"fmt"
	"log"
	"time"

	"github.com/twmb/avro"
)

func main() {
	type Event struct {
		ID     int64     `avro:"id"`
		Name   string    `avro:"name,default=unnamed"`
		Source string    `avro:"source,default=web"`
		Time   time.Time `avro:"ts"`
		Meta   *string   `avro:"meta"` // *T becomes ["null", T] union
	}

	schema := avro.MustSchemaFor[Event](avro.WithNamespace("com.example"))

	// Encode, then decode back.
	meta := "test"
	data, err := schema.Encode(&Event{
		ID:     1,
		Name:   "click",
		Source: "mobile",
		Time:   time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC),
		Meta:   &meta,
	})
	if err != nil {
		log.Fatal(err)
	}

	var out Event
	if _, err := schema.Decode(data, &out); err != nil {
		log.Fatal(err)
	}
	fmt.Printf("id=%d name=%s source=%s meta=%s\n", out.ID, out.Name, out.Source, *out.Meta)

	// Inspect the inferred schema.
	root := schema.Root()
	for _, f := range root.Fields {
		if f.HasDefault {
			fmt.Printf("field %s: default=%v\n", f.Name, f.Default)
		}
	}
}

Output:
id=1 name=click source=mobile meta=test
field name: default=unnamed
field source: default=web
field meta: default=<nil>

func (*Schema) AppendEncode

func (s *Schema) AppendEncode(dst []byte, v any, opts ...Opt) ([]byte, error)

AppendEncode appends the Avro binary encoding of v to dst. See Schema.Decode for the Go-to-Avro type mapping. In addition to the types listed there, encoding also accepts:

• encoding/json.Number for any numeric Avro type (int, long, float, double)
• RFC 3339 strings for timestamp and date logical types
• *big.Rat, big.Rat, float32, float64, encoding/json.Number, and numeric strings for decimal logical types
• encoding.TextAppender, encoding.TextMarshaler, and []byte for string types (and vice versa for encoding.TextUnmarshaler)
• string (hex-dash UUID format) for fixed(16) UUID logical types
• Tagged union maps (map[string]any{"typeName": value}) for union types, as produced by Schema.Decode with TaggedUnions

Example

package main

import (
	"fmt"
	"log"

	"github.com/twmb/avro"
)

func main() {
	schema := avro.MustParse(`"string"`)

	// AppendEncode reuses a buffer across calls, avoiding allocation.
	var buf []byte
	var err error
	for _, s := range []string{"hello", "world"} {
		buf, err = schema.AppendEncode(buf[:0], s)
		if err != nil {
			log.Fatal(err)
		}
		fmt.Printf("encoded %q: %d bytes\n", s, len(buf))
	}
}

Output:
encoded "hello": 6 bytes
encoded "world": 6 bytes

func (*Schema) AppendEncodeJSON

func (s *Schema) AppendEncodeJSON(dst []byte, v any, opts ...Opt) ([]byte, error)

AppendEncodeJSON is like Schema.EncodeJSON but appends to dst.

func (*Schema) AppendSingleObject

func (s *Schema) AppendSingleObject(dst []byte, v any, opts ...Opt) ([]byte, error)

AppendSingleObject appends a Single Object Encoding of v to dst: 2-byte magic, 8-byte CRC-64-AVRO fingerprint, then the Avro binary payload.

Example

package main

import (
	"fmt"
	"log"

	"github.com/twmb/avro"
)

func main() {
	schema := avro.MustParse(`{
		"type": "record",
		"name": "Event",
		"fields": [
			{"name": "id",   "type": "long"},
			{"name": "name", "type": "string"}
		]
	}`)

	type Event struct {
		ID   int64  `avro:"id"`
		Name string `avro:"name"`
	}

	// Encode: 2-byte magic + 8-byte fingerprint + Avro payload.
	data, err := schema.AppendSingleObject(nil, &Event{ID: 1, Name: "click"})
	if err != nil {
		log.Fatal(err)
	}

	// Decode.
	var e Event
	if _, err := schema.DecodeSingleObject(data, &e); err != nil {
		log.Fatal(err)
	}
	fmt.Printf("id=%d name=%s\n", e.ID, e.Name)
}

Output:
id=1 name=click

func (*Schema) Canonical

func (s *Schema) Canonical() []byte

Canonical returns the Parsing Canonical Form of the schema, stripping doc, aliases, defaults, and other non-essential attributes. The result is deterministic and matches Java's reference output byte-for-byte, so Schema.Fingerprint values are interoperable across implementations.

func (*Schema) Decode

func (s *Schema) Decode(src []byte, v any, opts ...Opt) ([]byte, error)

Decode reads Avro binary from src into v and returns the remaining bytes. v must be a non-nil pointer to a type compatible with the schema:

• null: any (always decodes to nil)
• boolean: bool, any
• int, long: int, int8–int64, uint8–uint64, any
• float: float32, float64, any
• double: float64, float32, any
• string: string, []byte, any; also encoding.TextUnmarshaler
• bytes: []byte, string, any
• enum: string, int/uint (ordinal), any
• fixed: [N]byte, []byte, any
• array: slice, any
• map: map[string]T, any
• union: any, *T (for ["null", T] unions), or the matched branch type
• record: struct (matched by field name or `avro` tag), map[string]any, any

When decoding into *any, primitive types become nil, bool, int32, int64, float32, float64, string, []byte, []any, or map[string]any (for records). Logical types decode to their natural Go equivalents:

• date, timestamp-millis/micros/nanos: time.Time (UTC)
• local-timestamp-millis/micros/nanos: time.Time (UTC; wall-clock fields encode/decode as if UTC, matching Java's reference impl)
• time-millis, time-micros: time.Duration
• decimal: *math/big.Rat
• uuid on string: string
• uuid on fixed(16): [16]byte
• duration: Duration

To produce JSON from decoded *any data use Schema.EncodeJSON, not a generic JSON encoder: it is schema-aware and converts these types back to their Avro representations (time.Time to epoch integers, []byte to \uXXXX strings).

Decode is liberal in what it accepts. Non-canonical input, such as a non-0/1 boolean byte that Java also reads as false, is tolerated rather than rejected. Encode is canonical, so such input round-trips to the canonical form.

func (*Schema) DecodeJSON

func (s *Schema) DecodeJSON(src []byte, v any, opts ...Opt) error

DecodeJSON decodes Avro JSON from src into v. It unwraps union wrappers, converts bytes/fixed strings, and coerces numeric types to match the schema. When v is *any, the result is returned directly.

DecodeJSON also accepts the non-standard union branch naming used by linkedin/goavro (e.g. "long.timestamp-millis" instead of "long").

DecodeJSON accepts all input formats (tagged and bare unions, Java and goavro NaN/Infinity conventions). Pass TaggedUnions to wrap decoded union values when the target is *any.

A bare union value whose JSON token class matches several branches (e.g. a bare number against ["long","int"]) decodes via the first matching branch in declaration order — the bare form does not name the writer's branch, so it cannot be recovered; see the TaggedUnions doc for the branch-identity and CustomType consequences.

On a schema returned by Resolve, src is WRITER-shaped JSON (the JSON a producer using the writer schema would emit) and full writer→reader resolution is applied — promotion, enum-symbol remapping to the reader default, field add/drop, and aliases — matching Java's ResolvingDecoder over a JsonDecoder constructed with the writer schema. (For binary, Schema.Decode resolves directly; JSON resolution composes the writer's JSON decode with the resolving binary decode, so it is not on a hot path.)

Example

package main

import (
	"fmt"
	"log"

	"github.com/twmb/avro"
)

func main() {
	schema := avro.MustParse(`{
		"type": "record",
		"name": "User",
		"fields": [
			{"name": "name",  "type": "string"},
			{"name": "email", "type": ["null", "string"]}
		]
	}`)

	type User struct {
		Name  string  `avro:"name"`
		Email *string `avro:"email"`
	}

	// DecodeJSON accepts both bare and tagged union formats.
	var u1, u2 User
	if err := schema.DecodeJSON([]byte(`{"name":"Alice","email":"a@b.com"}`), &u1); err != nil {
		log.Fatal(err)
	}
	if err := schema.DecodeJSON([]byte(`{"name":"Bob","email":{"string":"b@c.com"}}`), &u2); err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%s: %s\n", u1.Name, *u1.Email)
	fmt.Printf("%s: %s\n", u2.Name, *u2.Email)
}

Output:
Alice: a@b.com
Bob: b@c.com

func (*Schema) DecodeSingleObject

func (s *Schema) DecodeSingleObject(data []byte, v any, opts ...Opt) ([]byte, error)

DecodeSingleObject decodes a Single Object Encoding message into v after verifying the magic and fingerprint match this schema.

For a schema returned by Resolve, the writer's fingerprint is also accepted — wire bytes carry the writer's fingerprint per the SOE spec, and a resolved schema is the right place to decode them.

func (*Schema) Encode

func (s *Schema) Encode(v any, opts ...Opt) ([]byte, error)

Encode encodes v as Avro binary. It is shorthand for AppendEncode(nil, v).

Example (TextMarshaler)

package main

import (
	"fmt"
	"log"
	"net"

	"github.com/twmb/avro"
)

func main() {
	// Types implementing encoding.TextMarshaler are encoded as Avro
	// strings, and encoding.TextUnmarshaler types decode from them.
	schema := avro.MustParse(`{
		"type": "record",
		"name": "Server",
		"fields": [
			{"name": "name", "type": "string"},
			{"name": "ip",   "type": "string"}
		]
	}`)

	type Server struct {
		Name string `avro:"name"`
		IP   net.IP `avro:"ip"`
	}

	data, err := schema.Encode(&Server{
		Name: "web-1",
		IP:   net.IPv4(192, 168, 1, 1),
	})
	if err != nil {
		log.Fatal(err)
	}

	var out Server
	if _, err := schema.Decode(data, &out); err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%s: %s\n", out.Name, out.IP)
}

Output:
web-1: 192.168.1.1

func (*Schema) EncodeJSON

func (s *Schema) EncodeJSON(v any, opts ...Opt) ([]byte, error)

EncodeJSON encodes v as JSON using the schema for type-aware encoding. By default, union values are written as bare JSON values and bytes/fixed fields use \uXXXX escapes for non-ASCII bytes. Options can modify the output format; see Opt for details.

NaN and Infinity encode as the JSON strings "NaN", "Infinity" and "-Infinity" by default (Java convention), or as null / ±1e999 with LinkedinFloats. A generic JSON encoder rejects non-finite floats outright; these forms keep the output valid JSON for any strict parser.

String content that is not valid UTF-8 has each invalid byte replaced by U+FFFD. A JSON string cannot carry arbitrary non-UTF-8 bytes, so the JSON wire is lossy for such content while Schema.Encode preserves it verbatim; Java behaves the same on both formats. Applies to string values and map keys at any depth.

EncodeJSON accepts the same Go types as Schema.Encode. Map keys are not sorted, so their output order is non-deterministic.

Interop: the default bare-union output is NOT readable by Java's JsonDecoder, fastavro's JSON decoder, or avro-tools fromjson — all require the {"type_name": value} envelope and reject bare values. Pass TaggedUnions for those tools; see its doc and AVRO-2899.

Example

package main

import (
	"fmt"
	"log"

	"github.com/twmb/avro"
)

func main() {
	schema := avro.MustParse(`{
		"type": "record",
		"name": "User",
		"fields": [
			{"name": "name",  "type": "string"},
			{"name": "email", "type": ["null", "string"]}
		]
	}`)

	type User struct {
		Name  string  `avro:"name"`
		Email *string `avro:"email"`
	}
	email := "alice@example.com"
	u := User{Name: "Alice", Email: &email}

	// Default: bare union values.
	bare, err := schema.EncodeJSON(&u)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(string(bare))

	// TaggedUnions: wrapped as {"type": value}.
	tagged, err := schema.EncodeJSON(&u, avro.TaggedUnions())
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(string(tagged))
}

Output:
{"name":"Alice","email":"alice@example.com"}
{"name":"Alice","email":{"string":"alice@example.com"}}

func (*Schema) Fingerprint

func (s *Schema) Fingerprint(h hash.Hash) []byte

Fingerprint hashes the schema's canonical form with h and returns h's digest. Use NewRabin for the spec's CRC-64-AVRO algorithm, or crypto/sha256 for its 256-bit recommendation.

Byte order matters for CRC-64-AVRO. Go writes integer hashes high byte first, as crc32/crc64/adler32/fnv all do, so NewRabin returns the fingerprint BIG-ENDIAN; Java, fastavro and the single-object header write that same 64-bit value LITTLE-ENDIAN. Only the order differs — compare as a uint64, or reverse the bytes.

A crypto/sha256 fingerprint is a byte string with no byte order and already matches Java and fastavro byte for byte; reversing it would break that.

No call returns the little-endian CRC-64-AVRO form. Schema.AppendSingleObject writes it into the message header, SingleObjectFingerprint reads it back, Schema.DecodeSingleObject verifies.

func (*Schema) Root

func (s *Schema) Root() *SchemaNode

Root returns a SchemaNode tree describing the parsed schema. All metadata is preserved (doc strings, namespaces, custom properties, numeric defaults). See SchemaNode.Props and SchemaField.Default for how values decode.

Reserved Avro attribute names ("type", "name", "namespace", "doc", "aliases", …) match only by their exact lowercase spelling, as in the Avro reference implementations. A case variant such as "Aliases" is an ordinary custom property: it never binds the attribute and is reported verbatim in SchemaNode.Props. Parsing applies the same rule, so a schema whose only spelling of a structural key is a case variant ("ITEMS" on an array) fails Parse — the structural attribute is absent.

A field in the flat (goavro-style) format — a bare-string complex-kind type with the kind's defining key (symbols, items, values, fields, size) alongside the field's own keys — is described post-lift, exactly as it parses: the field's type is the lifted nested definition (named after the field for record/error/enum/fixed), and the keys the lift routed into the type appear on the type node rather than in SchemaField.Props. SchemaNode.Schema rebuilds the nested form, which parses identically.

Every node converts back to a usable *Schema via SchemaNode.Schema, name-reference nodes included: the tree carries the schema's named-type definitions, so any extracted subtree is self-contained.

Root re-parses the JSON on each call. Cache the result if you access it repeatedly (e.g. in a per-message loop).

func (*Schema) String

func (s *Schema) String() string

String returns the original JSON passed to Parse, preserving all attributes (doc, aliases, defaults, etc.) unlike Schema.Canonical.

type SchemaCache

type SchemaCache struct {
	// contains filtered or unexported fields
}

SchemaCache accumulates named types across multiple SchemaCache.Parse calls, allowing schemas to reference types defined in previously parsed schemas — the shape a Schema Registry's inter-schema references take.

Schemas must be parsed in dependency order: referenced types must be parsed before the schemas that reference them.

Parsing the same schema string more than once is allowed and returns the previously parsed result, so diamond dependencies (A→B→D, A→C→D) need no caller-side tracking. Options that change what the string compiles to — custom types or WithLaxNames — skip this deduplication and re-parse, since the string alone no longer identifies the result. Deduplication normalizes JSON whitespace and key order but not the Avro canonical form: schemas differing only in formatting dedupe, while differences in non-canonical fields like doc or aliases return a duplicate type error.

Each returned *Schema is fully resolved and independent of the cache. That extends to sub-schemas: a node extracted from Schema.Root converts via SchemaNode.Schema with every cross-parse reference resolved, so the cache is never needed again once Parse returns.

WithLaxNames is sticky: if a type is defined with it, pass it to every later Parse that references that type. A schema containing a lax name is not parseable without it, cache or no cache, so the referencing Parse's Schema.String and Schema.Canonical output also needs WithLaxNames to re-parse. Schema.Encode and Schema.Decode are unaffected either way.

The zero value is ready to use. A SchemaCache is safe for concurrent use.

Example

package main

import (
	"fmt"
	"log"

	"github.com/twmb/avro"
)

func main() {
	cache := new(avro.SchemaCache)

	// Parse the Address type first.
	if _, err := cache.Parse(`{
		"type": "record",
		"name": "Address",
		"fields": [
			{"name": "street", "type": "string"},
			{"name": "city",   "type": "string"}
		]
	}`); err != nil {
		log.Fatal(err)
	}

	// User references Address by name.
	schema, err := cache.Parse(`{
		"type": "record",
		"name": "User",
		"fields": [
			{"name": "name",    "type": "string"},
			{"name": "address", "type": "Address"}
		]
	}`)
	if err != nil {
		log.Fatal(err)
	}

	type Address struct {
		Street string `avro:"street"`
		City   string `avro:"city"`
	}
	type User struct {
		Name    string  `avro:"name"`
		Address Address `avro:"address"`
	}

	data, err := schema.Encode(&User{
		Name:    "Alice",
		Address: Address{Street: "123 Main St", City: "Springfield"},
	})
	if err != nil {
		log.Fatal(err)
	}

	var u User
	if _, err := schema.Decode(data, &u); err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%s lives at %s, %s\n", u.Name, u.Address.Street, u.Address.City)
}

Output:
Alice lives at 123 Main St, Springfield

func (*SchemaCache) Parse

func (c *SchemaCache) Parse(schema string, opts ...SchemaOpt) (*Schema, error)

Parse parses a schema string, registering any named types (records, enums, fixed) in the cache. Named types from previous Parse calls are available for reference resolution. On failure, the cache is not modified.

type SchemaField

type SchemaField struct {
	Name string     // field name
	Type SchemaNode // field schema

	// Default is the field's default value, present when HasDefault is
	// true. The Go type matches the schema:
	//
	//   - int schemas give int32, long schemas give int64. Out-of-range
	//     defaults are rejected at parse.
	//   - float schemas give float32, double float64. Overflows narrow to
	//     ±Inf; NaN, ±Inf, and a float-syntax "-0.0" round-trip. An
	//     integer-syntax "-0" is the sign-less integer 0 and surfaces as +0.0
	//     (matching Java/fastavro), though the wire encoder writes -0.0 for it.
	//   - string and enum schemas give string.
	//   - bytes and fixed give []byte, already decoded from the JSON spec's
	//     codepoint-per-byte form.
	//   - record, array, and map give map[string]any or []any, each leaf
	//     following these same rules.
	//
	// Union defaults pick the first branch that accepts the value, and the Go
	// type tells you which: ["float","int"] with default 42 gives float32(42).
	//
	// Unlike Props, a numeric Default is never json.Number — parse rejects
	// defaults that do not fit the declared type.
	Default any

	HasDefault bool     // true if a default value is defined in the schema
	Aliases    []string // field aliases for schema evolution
	Order      string   // sort order: "ascending" (default), "descending", or "ignore"
	Doc        string   // documentation string

	// Props holds custom (non-reserved) field properties; numbers decode as in
	// [SchemaNode.Props]. Field-level "logicalType", "precision", and "scale"
	// appear here as written — the wire-side lift is a codec concession that
	// never removes them from this surface. An unconsumed precision/scale (no
	// field logicalType, a non-decimal one, or a decimal whose lift target is
	// not a bytes/fixed carrier) is an ordinary property whatever its JSON
	// shape; only a consumed placement shape-validates the pair at parse.
	Props map[string]any
	// contains filtered or unexported fields
}

SchemaField represents a field in an Avro record schema.

type SchemaNode

type SchemaNode struct {
	Type        string // Avro type or named type reference
	LogicalType string // e.g. date, timestamp-millis, decimal, uuid; empty if none (or if the attribute's value is not a string — see Props)

	Name string // name for record, enum, fixed

	// Namespace is the named type's RESOLVED namespace: [Schema.Root] fills it
	// for every named type, a child inheriting its enclosing namespace
	// surfaces that namespace here, and "" always means the null namespace,
	// never "inherit". [SchemaNode.Schema] emits a "namespace":"" escape when
	// a null-namespace type sits inside a namespaced scope, so the distinction
	// survives the round trip. A dotted Name takes precedence over this field.
	Namespace string

	Aliases []string // alternate names for named types (record, enum, fixed)
	Doc     string   // documentation string

	Fields   []SchemaField // record fields
	Items    *SchemaNode   // array element schema
	Values   *SchemaNode   // map value schema
	Branches []SchemaNode  // union member schemas
	Symbols  []string      // enum symbols
	Size     int           // fixed byte size

	EnumDefault    string // default symbol for enum schema evolution
	HasEnumDefault bool   // true if an enum default is defined

	// Precision and Scale are the decimal logical type's parameters, set and
	// validated exactly when LogicalType is "decimal" on a bytes or fixed
	// carrier. Anywhere else — no logical type, an unknown or non-decimal one,
	// or a decimal on a carrier it soft-drops from — the attributes are inert
	// metadata surfaced in Props, matching the field level.
	Precision int // decimal precision
	Scale     int // decimal scale

	// Props holds custom (non-reserved) schema attributes — anything in the
	// schema JSON that is not a standard Avro field (e.g. "com.example.tag").
	// It is also the ONLY surface for a reserved structural key on a kind that
	// does not bind it whose body does not parse as that key's schema shape (a
	// stray "items":3 on an "int"): the matching structural field stays zero.
	// A schema-shaped stray body instead surfaces as-written on Items / Values
	// / Fields. A non-string logicalType is likewise inert and appears here
	// verbatim, since nothing but a string can name a logical.
	//
	// Values use the natural Go types from JSON: string, bool, nil, []any,
	// map[string]any, int64 for whole numbers, float64 for fractional. A
	// number stays json.Number when neither fits — a whole number too large
	// for int64, or a fractional literal over 1024 bytes, whose digits are
	// kept verbatim rather than rounded. Whole-valued exponents collapse to
	// int64 (1e3 reads as int64(1000)); exponents overflowing float64 give
	// ±Inf. math.NaN() re-reads as the string "NaN" after Schema()/Root(),
	// because JSON has no NaN literal; ±Inf round-trips as float64(±Inf).
	Props map[string]any
	// contains filtered or unexported fields
}

SchemaNode is a read-write representation of an Avro schema. It can be obtained from a parsed schema via Schema.Root, or constructed directly and converted to a *Schema via the SchemaNode.Schema method.

The Type field determines which other fields are relevant:

• Primitives (null, boolean, int, long, float, double, string, bytes): LogicalType, Precision, Scale, Props optional; other fields ignored.
• record/error: Name, Fields required; Namespace, Doc, Props optional.
• enum: Name, Symbols required; Namespace, Doc, Props optional.
• array: Items required.
• map: Values required.
• fixed: Name, Size required; LogicalType, Precision, Scale, Namespace, Props optional.
• union: Branches lists the member schemas.

A named type (record, enum, fixed) already defined elsewhere in the schema can be referenced by setting Type to its full name (e.g. com.example.Address) with no other fields. In a Schema.Root tree, references also resolve outward: converting ANY node with SchemaNode.Schema resolves names against the schema the tree came from, so a field type, union branch, or deeper node converts even when the definition lives outside the extracted node. A hand-built tree has no enclosing schema, so there every referenced name must be defined within the tree being converted, or Schema returns an error.

func (*SchemaNode) Schema

func (n *SchemaNode) Schema(opts ...SchemaOpt) (*Schema, error)

Schema parses the SchemaNode into a *Schema that can be used for encoding and decoding. Returns an error if the node is invalid.

Named types appearing multiple times are deduplicated by FULLNAME: the first occurrence emits the definition, later ones emit the fullname as a reference. Two types sharing a short name across namespaces are distinct and both emit definitions.

A node extracted from a Schema.Root tree may reference definitions living elsewhere in the enclosing schema — an earlier field, a prior SchemaCache parse, or the enclosing type itself for a recursive schema. Those resolve automatically: the definition is emitted at the reference's first occurrence, so the result needs neither the enclosing schema nor any cache. A name the tree defines itself wins over the enclosing schema's definition, and custom properties on a wrapped reference ride onto the emitted definition (reserved usage-site attributes do not survive, matching the SchemaCache splice). Hand-built nodes carry no enclosing schema, so there a reference the tree does not define is an error.

opts pass through to the internal Parse. A schema originally parsed with [SchemaOpt]s that change what Parse accepts or wires — WithLaxNames, CustomType registrations — needs the same opts here, or the rebuilt schema fails to parse or silently lacks the custom wiring.

type SchemaOpt

type SchemaOpt interface {
	// contains filtered or unexported methods
}

SchemaOpt configures schema construction via Parse, SchemaCache.Parse, or SchemaFor. Inapplicable options are silently ignored.

func WithCustomType

func WithCustomType(ct CustomType) SchemaOpt

WithCustomType registers a custom type conversion for use with Parse, SchemaCache.Parse, or SchemaFor. CustomType and NewCustomType both satisfy SchemaOpt directly, so this wrapper is optional.

func WithLaxNames

func WithLaxNames(fn func(string) error) SchemaOpt

WithLaxNames relaxes name validation in Parse and SchemaCache.Parse, overriding the default requirement that names match the Avro strict name regex [A-Za-z_][A-Za-z0-9_]*. If fn is nil, only non-empty names are required. If fn is non-nil, it is called for each name component and should return an error for invalid names. Dot-separated fullnames are split before calling fn. Ignored by SchemaFor.

func WithName

func WithName(name string) SchemaOpt

WithName overrides the Avro record name in SchemaFor. By default the Go struct name is used. Ignored by Parse.

func WithNamespace

func WithNamespace(ns string) SchemaOpt

WithNamespace sets the Avro namespace for the top-level record in SchemaFor. Ignored by Parse.

type SemanticError

type SemanticError struct {
	// GoType is the Go type involved, if applicable.
	GoType reflect.Type
	// AvroType is the Avro schema type (e.g. "int", "record", "boolean").
	AvroType string
	// Field is the dotted path to the record field (e.g. "address.zip"),
	// if the error occurred within a record.
	Field string
	// Err is the underlying error.
	Err error
}

SemanticError indicates a Go type is incompatible with an Avro schema type during encoding or decoding.

func (*SemanticError) Error

func (e *SemanticError) Error() string

func (*SemanticError) Unwrap

func (e *SemanticError) Unwrap() error

type ShortBufferError

type ShortBufferError struct {
	// Type is what was being read (e.g. "boolean", "string", "uint32").
	Type string
	// Need is the number of bytes required (0 if unknown).
	Need int
	// Have is the number of bytes available.
	Have int
}

ShortBufferError indicates the input buffer is too short for the value being decoded.

func (*ShortBufferError) Error

func (e *ShortBufferError) Error() string