gobspect

package module
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 24, 2026 License: MIT Imports: 20 Imported by: 0

README

gobspect

gobspect is a decode-only introspection library for Go's encoding/gob wire format. It reads arbitrary gob streams without requiring the original Go types and produces a structured AST and human-readable output. Included in this repo is a query package and CLI inspection tool.

gq — command-line tool

gq is a jq-inspired CLI for inspecting gob streams from the terminal. No Go code required:

go install github.com/codepuke/gobspect/cmd/gq@latest
gq --schema data.gob          # print the Go-style type schema
gq .Orders.*.ID orders.gob    # navigate to a field across all slice elements
cat data.gob | gq --format json .Header

See the gq README for the full flag reference, query syntax, and examples.

Library overview

Standard encoding/gob decoding requires the original type definitions at decode time. gobspect removes this requirement: it parses the wire format directly, reconstructs the type graph from the inline type definitions present in every gob stream, and yields a structured representation of the encoded data.

This is useful for debugging serialized data, building inspection tools, or reading gob streams from code you do not control.

Two output layers are provided:

  • Structural AST (Value and its subtypes): a complete representation of the wire data, preserving type IDs, type names, field names, and raw bytes for opaque blobs. This layer does not lose information.
  • Human-readable formatting (Format): a text rendering of a Value tree, with built-in decoders for common opaque types.

Installation

go get github.com/codepuke/gobspect

Requires Go 1.26 or later.

Usage

Schema inspection

When you encounter an unknown .gob file, Stream.Schema is usually the first call to make. It drains the stream, reads the type definitions embedded in every gob stream, and renders them as Go-style type declarations:

ins := gobspect.New()
schema, err := ins.Stream(r).Schema() // r is any io.Reader
if err != nil {
    log.Fatal(err)
}
fmt.Println(schema)

For a stream that encodes an Order struct referencing LineItem and opaque types, the output looks like:

type LineItem struct {
  Price     Decimal  // GobEncoder
  Quantity  int
  SKU       string
}

type Order struct {
  Customer  string
  ID        uint
  Items     LineItem
  PlacedAt  Time  // GobEncoder
}

Opaque types — values that implement GobEncoder or BinaryMarshaler — appear as inline comments on fields that reference them, and as standalone declarations:

type Decimal // GobEncoder
type Time    // GobEncoder

The gob wire format records only the short type name and raw encoded bytes for opaque types — no underlying structure and no import path — so no valid Go type declaration can be produced.

Stream.Schema drains the stream and returns a *Schema. Use Stream.Types to access the structured []TypeInfo slice directly, or call FormatSchema to convert it:

ins := gobspect.New()
stream := ins.Stream(r)
values, err := stream.Collect()
if err != nil {
    log.Fatal(err)
}
schema := gobspect.FormatSchema(stream.Types())
fmt.Println(schema)

Schema.Format and Schema.FormatTo accept FormatOption values (e.g. WithColor, WithIndent) to control rendering.

Decode a stream and format the output
ins := gobspect.New()
values, err := ins.Stream(r).Collect() // r is any io.Reader
if err != nil {
    log.Fatal(err)
}
for _, v := range values {
    fmt.Println(gobspect.Format(v))
}

New returns an Inspector with all built-in opaque decoders pre-registered. Collect returns one Value per top-level Encode call in the original stream. A stream may contain multiple values.

Stream values one at a time with an iterator
ins := gobspect.New()
stream := ins.Stream(r)
for v, err := range stream.Values() {
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(gobspect.Format(v))
}

Values returns an iter.Seq2[Value, error] that yields each decoded value as it is read, without buffering the entire stream first. An early break is safe; the iterator stops reading immediately.

Note: A Stream is single-use. Calling Values on an already-consumed Stream panics. Create a new Stream with ins.Stream(r) for each pass.

When to prefer Values over Collect:

  • The stream is large and you want to process or discard each value before reading the next.
  • You want to stop partway through (e.g., search for the first matching value).
  • You want to integrate with other range-based pipelines.

Use Collect when you need all values as a slice.

Inspect type definitions alongside values
ins := gobspect.New()
stream := ins.Stream(r)
for v, err := range stream.Values() {
    if err != nil {
        log.Fatal(err)
    }
    // stream.Types() grows as the stream is consumed.
    fmt.Printf("decoded value; %d types known so far\n", len(stream.Types()))
    fmt.Println(gobspect.Format(v))
}
// After the loop, stream.Types() contains all type definitions.
for _, ti := range stream.Types() {
    fmt.Printf("type %s kind=%v fields=%d\n", ti.Name, ti.Kind, len(ti.Fields))
}

Stream.Types returns the live slice of TypeInfo for every type definition encountered in stream order. It grows incrementally as the stream is consumed.

Inspecting raw framing

Stream.Messages returns an iter.Seq2[MessageInfo, error] that yields one entry per length-prefixed frame in the stream without decoding the value body. Each MessageInfo carries the byte Offset, BodyLen, signed TypeID, and the Body bytes. This is the right tool for size profiling, building a frame index, or comparing wire layouts:

for m, err := range ins.Stream(r).Messages() {
    if err != nil { log.Fatal(err) }
    fmt.Printf("msg %d @%d len=%d typeID=%d typeDef=%v\n",
        m.Index, m.Offset, m.BodyLen, m.TypeID, m.IsTypeDef())
}

Stream.Stats wraps a full decode pass and returns population-level counts (per-type record totals, body byte consumption, struct field presence rates, opaque decoder coverage). Use it for a quick profile of an unknown file.

Structural diff

The gobspect/diff subpackage compares two Value trees or two streams position-by-position, producing a Delta AST that can be rendered as text or JSON. The CLI exposes it via gq -diff PATH (exit code 1 when there are changes).

Compressed streams

Inspector.Stream accepts any io.Reader, so compressed streams work by wrapping the reader before passing it in. For gzip, use compress/gzip.NewReader; apply the same pattern for any other compression format.

Register a custom opaque decoder

Types that implement GobEncoder or BinaryMarshaler are serialized as opaque byte blobs. gobspect ships decoders for common standard library and third-party types (see Built-in opaque decoders). For application-specific types, register a decoder by the type's CommonType.Name as it appears in the gob wire format (the short type name, not the full import path):

ins := gobspect.New()
ins.RegisterDecoder("SessionToken", func(data []byte) (any, error) {
    if len(data) < 8 {
        return nil, errors.New("session token too short")
    }
    created := binary.BigEndian.Uint64(data[:8])
    payload := data[8:]
    return map[string]any{
        "created": time.Unix(int64(created), 0).Format(time.RFC3339),
        "payload": hex.EncodeToString(payload),
    }, nil
})

The returned value is stored in OpaqueValue.Decoded and used by Format. Registered decoders override built-in decoders for the same type name.

Note: CommonType.Name is only populated in the wire format when a GobEncoder type is transmitted through an interface field. When such a type is encoded directly (not via an interface), gob sends an empty name and registry lookup by name cannot match. See docs/opaque-types.md for details.

JSON output

Individual values can be serialized with gobspect.ToJSON(v) (compact) or gobspect.ToJSONIndent(v, "", " ") (pretty-printed).

type Point struct{ X, Y int }

var buf bytes.Buffer
gob.NewEncoder(&buf).Encode(Point{X: 3, Y: 7})

ins := gobspect.New()
values, err := ins.Stream(&buf).Collect()
if err != nil {
    log.Fatal(err)
}

// Pretty-print a single value.
b, err := gobspect.ToJSONIndent(values[0], "", "  ")

The above produces output like:

{
  "fields": [
    {"name": "X", "value": {"kind": "int", "v": 3}},
    {"name": "Y", "value": {"kind": "int", "v": 7}}
  ],
  "kind": "struct",
  "typeId": 64,
  "typeName": "Point"
}

Note: typeId is session-scoped. The numeric value depends on the order type definitions appear in the stream and will differ between sessions.

Every node carries a "kind" discriminator. The full field mapping per kind is documented in docs/api.md.

Format options

These options are passed to Format(v, ...FormatOption):

Option Type Description
WithIndent(s) FormatOption Indentation string for nested output. Default: " "
WithMaxBytes(n) FormatOption Max bytes rendered for BytesValue and OpaqueValue.Raw. Default: 64. Zero = no limit. Applies to all byte formats.
WithRawOpaques(bool) FormatOption Always show raw bytes even when OpaqueValue.Decoded is set.
WithBytesFormat(f) FormatOption How BytesValue and OpaqueValue.Raw are rendered: BytesHex (default), BytesBase64, or BytesLiteral. When set explicitly, the printable-UTF-8 shortcut is suppressed.
WithRedactKeys(cfg) FormatOption Redact values at render time when the field or map-key name matches. See Redacting sensitive fields.
WithRedactTypes(cfg) FormatOption Redact values whose type name matches. Supports custom fill character and length. May be combined with WithRedactKeys.

WithTimeFormat(layout) is an Inspector-level option passed to New(), not to Format(). It re-registers the time.Time decoder with a custom Go time layout. Default: time.RFC3339Nano.

ins := gobspect.New(gobspect.WithTimeFormat("2006-01-02"))

out := gobspect.Format(v,
    gobspect.WithIndent("\t"),      // default: two spaces
    gobspect.WithMaxBytes(128),     // max bytes shown for opaque/bytes, default: 64
    gobspect.WithRawOpaques(true),  // always show raw bytes even when Decoded is set
    gobspect.WithBytesFormat(gobspect.BytesBase64), // base64 instead of hex
)
Redacting sensitive fields

WithRedactKeys replaces the rendered value of matching struct fields or map entries with a fill character string. The AST is never modified — redaction happens at render time only.

out := gobspect.Format(v,
    gobspect.WithRedactKeys(gobspect.RedactConfig{
        Keys:       []string{"Password", "Token"},
        Char:       '*',
        TextLength: 0, // 0 = preserve visual length of the original rendered value
    }),
)

WithRedactTypes redacts all values whose TypeName matches, regardless of where they appear. It accepts a RedactTypesConfig that controls which types to redact and how the fill characters are rendered:

out := gobspect.Format(v,
    gobspect.WithRedactTypes(gobspect.RedactTypesConfig{
        Types:      []string{"Sensitive", "SecretKey"},
        Char:       '*',
        TextLength: 0, // 0 = preserve visual length of the original rendered value
    }),
)

Both options may be combined; a value is redacted if it matches either rule.

Key matching for struct fields is by exact field name. For map entries, matching is by the formatted key string (e.g., "\"password\"" for a string map key "password"). Case-sensitive exact match only.

Built-in opaque decoders

The following types are decoded automatically when encountered in a stream:

Type Encoding Formatted as
time.Time BinaryMarshaler RFC 3339 with nanosecond precision
math/big.Int GobEncoder Decimal string
math/big.Float GobEncoder Decimal string
math/big.Rat GobEncoder numerator/denominator or decimal
github.com/google/uuid.UUID BinaryMarshaler Standard UUID string
github.com/gofrs/uuid.UUID BinaryMarshaler Standard UUID string
github.com/shopspring/decimal.Decimal GobEncoder Reconstructed decimal string
Any TextMarshaler (pre-Go 1.26 streams) TextMarshaler UTF-8 string as-is

Unknown GobEncoder and BinaryMarshaler types are stored as OpaqueValue with Decoded = nil and rendered as (TypeName) <hex>.

Value AST types

All AST node types implement the sealed Value interface and can be inspected with a type switch:

switch v := v.(type) {
case gobspect.StructValue:
    for _, f := range v.Fields {
        fmt.Printf("%s = %v\n", f.Name, f.Value)
    }
case gobspect.IntValue:
    fmt.Println(v.V)
case gobspect.OpaqueValue:
    fmt.Printf("opaque %s: %v\n", v.TypeName, v.Decoded)
// ... InterfaceValue, MapValue, SliceValue, ArrayValue,
//     UintValue, FloatValue, ComplexValue, BoolValue,
//     StringValue, BytesValue, NilValue
}

The full type definitions are documented in docs/api.md.

Decoding limits

Limits can be set at construction time to bound resource use on untrusted input:

ins := gobspect.New(gobspect.WithReadLimit(4 * 1024 * 1024)) // 4 MiB

WithReadLimit caps the total bytes read across the entire stream. Zero means no limit.

Hard limits are always enforced regardless of options: 64 MiB per message, 65536 struct fields, and 2^30 elements in slices, maps, and arrays.

Error handling

The decoder does not panic on malformed input. All errors are returned. Stream.Collect returns partial results alongside any error: a stream that decodes successfully up to a corrupt message returns those values plus the error. The Values iterator similarly stops and yields (nil, err) on the first error.

Query

The query subpackage (github.com/codepuke/gobspect/query) provides path-based navigation of decoded Value trees without manual type switches:

v, ok    := query.Get(root, "Orders.0.Customer.Name")
names    := query.All(root, "Orders.*.Customer.Name")
keys, _  := query.Keys(root, "Orders.0")

For hot paths or explicit error handling, compile the expression once with query.Parse and reuse it with query.GetPath or query.AllPath:

p, err := query.Parse("Orders.*.Customer.Name")
if err != nil { ... }
for _, root := range roots {
    names := query.AllPath(root, p)
}

For lazy, streaming enumeration (early-break safe), use query.AllPathSeq:

for v := range query.AllPathSeq(root, p) {
    // process v; break at any time to stop early
}

See query/README.md for the full path syntax, filter expressions, and API reference.

Sorting

The sortval subpackage (github.com/codepuke/gobspect/sortval) sorts sequences of Value nodes by struct field keys:

spec, err := sortval.ParseSortSpec("Name,Score", false, false, false)
if err != nil { ... }

sorted := sortval.SortMatches(sortval.SeqOf(results), spec)

ParseSortSpec accepts a comma-separated list of field names plus flags for descending order (desc), case-insensitive comparison (fold), and exclusion of rows missing all sort keys (dropMissing). The comparison delegates to gobspect.CompareValues / gobspect.CompareValuesFold.

See sortval/README.md for the full API reference.

Tabular output

The tabular subpackage (github.com/codepuke/gobspect/tabular) writes Value nodes as CSV or TSV rows:

tp := tabular.NewPrinter(&buf,
    tabular.WithStream(stream),
    tabular.WithDelimiter(','),
    tabular.WithHeterogeneousMode(tabular.HeterogeneousUnion),
)

for v, err := range stream.Values() {
    if err != nil { ... }
    if err := tp.WriteValue(v); err != nil { ... }
}
tp.Flush()

The printer derives a header row from the first struct's field definitions, aligns sparse gob rows to the canonical column order, and supports four strategies for mixed-type streams: FirstWins, Reject, Union, and Partition.

See tabular/README.md for all options and heterogeneous-mode details.

Documentation

Contributing

See CONTRIBUTING.md for guidelines on opening issues, making minimal focused changes, and the pull request process.

License

MIT

Documentation

Overview

Package gobspect is a decode-only introspection library for Go's encoding/gob wire format. It reads arbitrary gob streams without requiring the original Go types, producing a structured Value AST and human-readable output via Format.

Basic usage 

ins := gobspect.New()
stream := ins.Stream(r)

for v, err := range stream.Values() {
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(gobspect.Format(v))
}

// stream.Types() now contains every type encountered

To collect all values at once: 

values, err := ins.Stream(r).Collect()

Value AST

Stream.Values returns an iterator over Value nodes. Each Value is one of the concrete node types listed below; use a type switch to dispatch on them:

Type metadata

Stream.Types returns TypeInfo for every type definition in the stream as the stream is consumed. By the time a value is yielded by Stream.Values, all type definitions it references are already present in Stream.Types and all [TypeRef.Name] fields are resolved. Stream.TypeByID provides O(1) lookup by stream-scoped type ID.

Stream.Schema drains the stream and returns all type definitions formatted as a Schema.

Looking up types during iteration 

stream := ins.Stream(r)
for v, err := range stream.Values() {
    if err != nil { log.Fatal(err) }
    sv, ok := v.(gobspect.StructValue)
    if !ok { continue }
    ti, ok := stream.TypeByID(sv.TypeID())
    if ok {
        fmt.Printf("type %s has %d fields\n", ti.Name, len(ti.Fields))
    }
}

Opaque types

Types that implement GobEncoder, BinaryMarshaler, or TextMarshaler are represented as OpaqueValue. New pre-registers decoders for common types:

  • time.Time (encoded as RFC 3339 with nanosecond precision)
  • math/big.Int and math/big.Rat (auto-detected by wire format)
  • math/big.Float (registered under "math/big.Float")
  • UUID types from github.com/google/uuid and github.com/gofrs/uuid
  • github.com/shopspring/decimal.Decimal

Additional decoders can be registered with Inspector.RegisterDecoder. TextMarshaler blobs are always decoded as UTF-8 strings automatically.

Formatting

Format renders any Value as a human-readable string. FormatTo writes the same output to an io.Writer and propagates write errors. Structs are always indented; maps, slices, and arrays are inlined when short (≤72 chars) and indented otherwise. Map entries are sorted by key for deterministic output. Use WithIndent, WithMaxBytes, and WithRawOpaques to adjust the output.

Decoding limits

Pass WithReadLimit to New to cap total bytes read from a stream: 

ins := gobspect.New(gobspect.WithReadLimit(10 << 20)) // 10 MiB

Path-based navigation

The companion subpackage github.com/codepuke/gobspect/query provides path-based navigation of decoded Value trees. Use query.AllPathSeq for lazy, streaming enumeration of matching values (early-break safe), or query.AllPath to collect all matches into a slice.

Index

Constants

This section is empty.

Variables

View Source 
var ANSIColorScheme = ColorScheme{
	FieldName:    Style{Prefix: "\x1b[32m", Suffix: "\x1b[0m"},
	TypeHeader:   Style{Prefix: "\x1b[1;36m", Suffix: "\x1b[0m"},
	CloseBrace:   Style{},
	String:       Style{Prefix: "\x1b[33m", Suffix: "\x1b[0m"},
	Number:       Style{Prefix: "\x1b[35m", Suffix: "\x1b[0m"},
	Bool:         Style{Prefix: "\x1b[36m", Suffix: "\x1b[0m"},
	Nil:          Style{Prefix: "\x1b[36m", Suffix: "\x1b[0m"},
	OpaquePrefix: Style{Prefix: "\x1b[2m", Suffix: "\x1b[0m"},
	OpaqueValue:  Style{},
	Bytes:        Style{Prefix: "\x1b[2m", Suffix: "\x1b[0m"},
}

ANSIColorScheme is a pre-built ColorScheme that uses ANSI escape codes to produce syntax-highlighted terminal output. The colors match the coloring used by cmd/gq.

View Source 
var NoColorScheme = ColorScheme{}

NoColorScheme is a zero-valued ColorScheme that produces no color output.

Functions

func CompareValues 

func CompareValues(a, b Value) int

CompareValues returns -1, 0, or +1 ordering a before, equal to, or after b. InterfaceValue is unwrapped from both sides before dispatch. Same-kind numerics: Int vs Int uses int64, Uint vs Uint uses uint64. Cross-numeric comparisons use float64; large integer values near the limits of float64 precision may not compare correctly. Composite types (struct, map, slice, array) fall back to Format output.

func CompareValuesFold 

func CompareValuesFold(a, b Value) int

CompareValuesFold is identical to CompareValues except strings are compared case-insensitively using strings.ToLower. It does not handle all Unicode case equivalences (e.g. ß vs SS).

func Equal added in v0.2.0 

func Equal(a, b Value) bool

Equal reports whether a and b are structurally equal Values. Equality is strict: the two kinds must match, composite shapes must line up exactly, and primitives compare by native value. Cross-kind numeric equivalence (e.g. IntValue{5} vs FloatValue{5}) returns false — use CompareValues if you need the permissive numeric coercion.

InterfaceValue wrappers are unwrapped on both sides before comparison. An InterfaceValue's outer TypeName does not participate in equality: only the inner concrete value matters.

func Format 

func Format(v Value, opts ...FormatOption) string

Format renders v as a human-readable string. Structs are always rendered as indented field trees. Maps, slices, and arrays are rendered inline when their formatted length fits within the inline width (default 72, overridden by WithInlineWidth) and none of their elements require multi-line rendering; otherwise they are indented. Map entries are sorted by formatted key for deterministic output.

func FormatBytes 

func FormatBytes(b []byte, format BytesFormat, maxBytes int) string

FormatBytes encodes b using the given BytesFormat, truncating to maxBytes raw bytes before encoding when maxBytes > 0 and len(b) exceeds it. A '…' suffix is appended when truncation occurs. An empty slice always returns "[]".

The printable-UTF-8 shortcut used by Format for BytesValue is intentionally absent here; the requested format is always applied.

func FormatTo 

func FormatTo(w io.Writer, v Value, opts ...FormatOption) error

FormatTo renders v as a human-readable string and writes it to w. Structs are always rendered as indented field trees. Maps, slices, and arrays are rendered inline when their formatted length fits within the inline width (default 72, overridden by WithInlineWidth) and none of their elements require multi-line rendering; otherwise they are indented. Map entries are sorted by formatted key for deterministic output. The first write error aborts rendering and is returned.

func ToJSON 

func ToJSON(v Value) ([]byte, error)

ToJSON serializes a Value as a discriminated-union JSON object. Every node carries a "kind" field with the concrete type name in lowercase snake_case. See the package documentation for the full field mapping per kind.

func ToJSONIndent 

func ToJSONIndent(v Value, prefix, indent string) ([]byte, error)

ToJSONIndent is like ToJSON but applies indentation. prefix and indent follow the same semantics as encoding/json.MarshalIndent.

func ValueKind 

func ValueKind(v Value) string

ValueKind returns a short lowercase string identifying the concrete type of v: "struct", "map", "slice", "array", "int", "uint", "float", "complex", "bool", "string", "bytes", "nil", "interface", "opaque", or "unknown".

Types

type ArrayValue 

type ArrayValue struct {
	TypeName  string
	GobTypeID int
	ElemType  string
	Len       int
	Elems     []Value
}

ArrayValue represents a decoded gob array.

func (ArrayValue) TypeID 

func (v ArrayValue) TypeID() int

type BoolValue 

type BoolValue struct{ V bool }

BoolValue holds a decoded boolean.

func (BoolValue) TypeID 

func (BoolValue) TypeID() int

type BytesFormat 

type BytesFormat int

BytesFormat controls how raw byte slices are rendered by Format. 

const (
	BytesHex     BytesFormat = iota // lowercase hex string (default)
	BytesBase64                     // standard base64 (RFC 4648, with padding)
	BytesLiteral                    // Go-style: []byte{0xde, 0xad, ...}
)

func ParseBytesFormat 

func ParseBytesFormat(s string) (BytesFormat, bool)

ParseBytesFormat converts a string to a BytesFormat constant. Accepts "hex", "base64", or "literal" (case-insensitive). An empty string maps to BytesHex. Returns (BytesHex, false) for any unrecognised value.

type BytesValue 

type BytesValue struct{ V []byte }

BytesValue holds a decoded []byte.

func (BytesValue) TypeID 

func (BytesValue) TypeID() int

type ColorScheme 

type ColorScheme struct {
	FieldName    Style // struct field names, map keys for string-keyed maps
	TypeHeader   Style // struct/map/slice/array type name ("Foo", "[]int", "map[string]int")
	CloseBrace   Style // "{" and "}" — opening and closing braces
	String       Style // quoted string values
	Number       Style // int, uint, float, complex
	Bool         Style // true, false
	Nil          Style // nil
	OpaquePrefix Style // "(TypeName)" marker
	OpaqueValue  Style // the rendered opaque value after the prefix
	Bytes        Style // hex / base64 / literal byte output
}

ColorScheme controls how different token roles are wrapped during rendering. A zero-valued ColorScheme produces no change to the output (identity).

type ComplexValue 

type ComplexValue struct{ Real, Imag float64 }

ComplexValue holds a decoded complex number.

func (ComplexValue) TypeID 

func (ComplexValue) TypeID() int

type DecoderFunc 

type DecoderFunc func([]byte) (any, error)

DecoderFunc decodes the raw bytes of a GobEncoder, BinaryMarshaler, or TextMarshaler blob into a human-meaningful value.

The returned value should be a simple Go type (string, int, float, map, etc.) suitable for display. It does not need to reconstruct the original Go type.

type Field 

type Field struct {
	Name  string
	Value Value
}

Field is a single named field within a StructValue.

type FieldDecl 

type FieldDecl struct {
	Name       string
	Type       string
	Annotation string
}

FieldDecl represents a field within a struct declaration.

type FieldInfo 

type FieldInfo struct {
	Name   string
	TypeID int
}

FieldInfo describes one field of a struct type.

type FloatValue 

type FloatValue struct{ V float64 }

FloatValue holds a decoded floating-point number.

func (FloatValue) TypeID 

func (FloatValue) TypeID() int

type FormatOption 

type FormatOption func(*formatConfig)

FormatOption configures the behavior of Format.

func WithBytesFormat 

func WithBytesFormat(f BytesFormat) FormatOption

WithBytesFormat controls how BytesValue and [OpaqueValue.Raw] are rendered. When set explicitly, the printable UTF-8 shortcut (render as a quoted string) is suppressed and the requested format is always used.

func WithColor 

func WithColor(scheme ColorScheme) FormatOption

WithColor applies the given ColorScheme to the rendered output. Each token role is wrapped with its corresponding Style's Prefix and Suffix. A zero-valued scheme (NoColorScheme) is the identity: no escape sequences are emitted and the output is byte-identical to calling Format without this option.

func WithIndent 

func WithIndent(indent string) FormatOption

WithIndent sets the indentation string used for nested output. Default: " ".

func WithInlineWidth 

func WithInlineWidth(n int) FormatOption

WithInlineWidth sets the maximum character count at which maps, slices, and arrays are rendered inline rather than indented. The plain-text (no-color) length of the full inline form is compared against n. Default: 72. Pass 0 to use the default explicitly.

func WithMapOrder 

func WithMapOrder(order MapOrder) FormatOption

WithMapOrder sets the ordering of map entries in Format output. The default (MapOrderSorted) sorts entries by their plain-text formatted key for deterministic output. MapOrderInsertion skips sorting and iterates entries in the order they appear in [MapValue.Entries] (wire order).

func WithMaxBytes 

func WithMaxBytes(n int) FormatOption

WithMaxBytes sets the maximum number of raw bytes rendered for OpaqueValue and BytesValue output. Default: 64. Zero means no limit. Applies to all byte formats (hex, base64, literal): the byte slice is truncated before encoding when this limit is exceeded.

func WithRawOpaques 

func WithRawOpaques(raw bool) FormatOption

WithRawOpaques controls whether decoded [OpaqueValue]s still show their raw bytes. When true, the raw bytes are always shown even when [OpaqueValue.Decoded] is set.

func WithRedactKeys 

func WithRedactKeys(cfg RedactConfig) FormatOption

WithRedactKeys redacts the rendered value of any struct field or map entry whose key matches one of cfg.Keys (case-sensitive exact match). The value is replaced at render time; the AST is never modified.

Key matching for struct fields is by field name. For map entries, matching is by the formatted key string (the result of rendering the key through Format). Glob and regex matching are intentionally out of scope.

func WithRedactTypes 

func WithRedactTypes(cfg RedactTypesConfig) FormatOption

WithRedactTypes redacts all values whose type name matches one of the names in cfg.Types. Checked against [StructValue.TypeName], [InterfaceValue.TypeName], and [OpaqueValue.TypeName]. May be combined with WithRedactKeys; both rules apply. cfg.Char and cfg.TextLength control the fill character and output length, identical to their meaning in RedactConfig.

type Inspector 

type Inspector struct {
	// contains filtered or unexported fields
}

Inspector is the top-level entry point. It holds the opaque decoder registry and decoding options. Create one with New.

func New 

func New(opts ...Option) *Inspector

New returns an Inspector with all built-in opaque decoders pre-registered.

func (*Inspector) RegisterDecoder 

func (ins *Inspector) RegisterDecoder(typeName string, dec DecoderFunc)

RegisterDecoder adds or overrides the opaque decoder for the given type name.

func (*Inspector) RegisterUnnamedDecoder 

func (ins *Inspector) RegisterUnnamedDecoder(dec DecoderFunc)

RegisterUnnamedDecoder appends dec to the list of decoders tried for opaque values whose gob wire type name is empty. Decoders are tried in registration order; the first one that returns a non-error result wins.

func (*Inspector) Stream 

func (ins *Inspector) Stream(r io.Reader) *Stream

Stream begins decoding the gob stream from r. Decoding is lazy: nothing is read from r until the iterator returned by Values is advanced, or a helper like Collect or Schema drains the stream. Calling Types before advancing returns an empty slice.

type IntValue 

type IntValue struct{ V int64 }

IntValue holds a decoded signed integer.

func (IntValue) TypeID 

func (IntValue) TypeID() int

type InterfaceValue 

type InterfaceValue struct {
	TypeName string // concrete type name from the stream
	Value    Value  // the concrete value, or NilValue for nil
}

InterfaceValue represents a decoded interface value.

func (InterfaceValue) TypeID 

func (InterfaceValue) TypeID() int

type MapEntry 

type MapEntry struct {
	Key   Value
	Value Value
}

MapEntry is a single key-value pair within a MapValue.

type MapOrder 

type MapOrder int

MapOrder controls the ordering of map entries in Format output. 

const (
	MapOrderSorted    MapOrder = iota // default: sort entries by formatted key
	MapOrderInsertion                 // skip sorting; use wire (insertion) order
)

type MapValue 

type MapValue struct {
	TypeName  string
	GobTypeID int
	KeyType   string // descriptive label for the key type
	ElemType  string // descriptive label for the element type
	Entries   []MapEntry
}

MapValue represents a decoded gob map.

func (MapValue) TypeID 

func (v MapValue) TypeID() int

type MessageInfo added in v0.2.0 

type MessageInfo struct {
	Index   int    // 0-based counter of messages in the stream
	Offset  int64  // byte offset of the length prefix from the start of the stream
	BodyLen int    // length of the message body (the bytes after the length prefix)
	TypeID  int    // signed type ID from the start of the body: negative = type def, positive = value
	Body    []byte // raw body bytes (including the type-ID prefix at the start)
}

MessageInfo describes a single length-prefixed message as it appears on the wire, without requiring the message body to be decoded into a Value. It is yielded by Stream.Messages and can be used for size profiling, stream indexing, or recovering framing information during error diagnostics.

func (MessageInfo) IsTypeDef added in v0.2.0 

func (m MessageInfo) IsTypeDef() bool

IsTypeDef reports whether the message carries a type definition.

type NilValue 

type NilValue struct{}

NilValue represents a nil pointer or nil interface.

func (NilValue) TypeID 

func (NilValue) TypeID() int

type OpaqueValue 

type OpaqueValue struct {
	TypeName  string // from CommonType.Name
	GobTypeID int    // stream-scoped type ID; use with Stream.TypeByID
	Encoding  string // "gob", "binary", or "text"
	Raw       []byte // the undecoded blob
	Decoded   any    // best-effort decoded form, nil if no decoder matched
}

OpaqueValue holds the raw bytes for a GobEncoder, BinaryMarshaler, or TextMarshaler value, along with any best-effort decoded form.

func (OpaqueValue) TypeID 

func (v OpaqueValue) TypeID() int

type Option 

type Option func(*Inspector)

Option is a functional option for New.

func WithReadLimit 

func WithReadLimit(n int64) Option

WithReadLimit sets the maximum total bytes read from a stream. Zero means no limit.

func WithSkipCorruptValues added in v0.2.0 

func WithSkipCorruptValues(b bool) Option

WithSkipCorruptValues configures the inspector to continue past individual value-message decode failures instead of aborting the stream. Each skipped message is counted and available via Stream.SkipCount. Errors in type- definition messages remain fatal because they would leave the type registry inconsistent and subsequent values undecodable.

Enable this when inspecting archived logs that may contain occasional bad records; leave it off for strict validation.

func WithTimeFormat 

func WithTimeFormat(layout string) Option

WithTimeFormat sets the layout used to render time.Time opaque values. The layout must be a valid Go time format string (see the time package). Default: time.RFC3339Nano.

type RedactConfig 

type RedactConfig struct {
	Keys       []string // exact field/key names that trigger redaction
	Char       rune     // fill character; defaults to '*'
	TextLength int      // number of Char runes to emit; 0 = see type-level doc
}

RedactConfig controls value redaction by field or map key name.

When TextLength is 0 (the default), the number of fill characters emitted depends on the rendered form of the value being redacted:

  • Single-line values: emit len([]rune(rendered)) fill chars, preserving the visual width of the original text.
  • Multi-line values (e.g. nested structs): emit exactly 3 fill chars ("***"). Counting runes across a multi-line rendering is not meaningful — the rune total includes indentation, newlines, and braces rather than content length — and inserting a flat string where a multi-line value was breaks visual alignment regardless. A short placeholder is unambiguous and clean.

Set TextLength > 0 to always emit exactly that many fill chars, regardless of the original rendered length or whether it is single- or multi-line.

Note: Char is repeated by Unicode code point, not by terminal display width. Multibyte fill characters (e.g. '█') produce the requested number of code points; terminal column width may differ.

type RedactTypesConfig 

type RedactTypesConfig struct {
	Types      []string // type names that trigger redaction
	Char       rune     // fill character; defaults to '*'
	TextLength int      // number of Char runes to emit; 0 = preserve original length
}

RedactTypesConfig controls value redaction by type name.

type Schema 

type Schema struct {
	Types  []TypeDecl
	Indent string
}

Schema represents a complete parsed Gob schema comprising multiple type declarations.

func FormatSchema 

func FormatSchema(types []TypeInfo) *Schema

FormatSchema converts a []TypeInfo slice into an AST-based *Schema, covering all top-level named types. Named types with mechanically generated names (containing brackets, e.g. "[]int") are safely excluded from the top-level schema output. Anonymous (unnamed) types appear only inline within other declarations.

To control rendering, pass SchemaFormatOption values to Schema.Format or Schema.FormatTo after calling FormatSchema.

func (*Schema) Format 

func (s *Schema) Format(opts ...SchemaFormatOption) string

Format renders the Schema as a string, applying any provided SchemaFormatOptions. A zero-valued ColorScheme (the default) produces plain text identical to Schema.String.

func (*Schema) FormatTo 

func (s *Schema) FormatTo(w io.Writer, opts ...SchemaFormatOption) error

FormatTo renders the Schema to w, applying any provided SchemaFormatOptions. The first write error aborts rendering and is returned.

func (*Schema) JSON added in v0.2.0 

func (s *Schema) JSON() ([]byte, error)

JSON returns a compact machine-readable representation of the schema. The output is an array of type declarations with stable keys; downstream tools (code generators, documentation, compatibility checkers) can consume it without parsing Go-style syntax.

func (*Schema) JSONIndent added in v0.2.0 

func (s *Schema) JSONIndent(prefix, indent string) ([]byte, error)

JSONIndent is like Schema.JSON but formats the output with the given indentation (see encoding/json.MarshalIndent).

func (*Schema) String 

func (s *Schema) String() string

String renders the Schema as a human-readable Go-style type declaration block, matching the original plain-text formatting exactly.

func (*Schema) TypeByName 

func (s *Schema) TypeByName(name string) (*TypeDecl, bool)

TypeByName locates a top-level type declaration by its name.

type SchemaFormatOption 

type SchemaFormatOption func(*formatConfig)

SchemaFormatOption configures the rendering of a Schema. Unlike FormatOption, which applies to value trees, SchemaFormatOption covers only the subset of formatting options meaningful for type-declaration output.

func SchemaWithColor 

func SchemaWithColor(scheme ColorScheme) SchemaFormatOption

SchemaWithColor sets the ColorScheme used when rendering a Schema. A zero-valued ColorScheme (the default) produces plain text identical to Schema.String.

func SchemaWithIndent 

func SchemaWithIndent(indent string) SchemaFormatOption

SchemaWithIndent sets the indentation string used inside struct bodies. The default is two spaces.

type SliceValue 

type SliceValue struct {
	TypeName  string
	GobTypeID int
	ElemType  string
	Elems     []Value
}

SliceValue represents a decoded gob slice.

func (SliceValue) TypeID 

func (v SliceValue) TypeID() int

type Stats added in v0.2.0 

type Stats struct {
	// TotalMessages counts every length-prefixed message in the stream
	// (type definitions + values).
	TotalMessages int
	// TotalBodyBytes is the sum of every message body's length in bytes
	// (excluding the length prefix itself). It is a close approximation of
	// "bytes on the wire" for the stream.
	TotalBodyBytes int64
	// TypeDefMessages counts messages that carry a type definition.
	TypeDefMessages int
	// ValueMessages counts messages that carry a top-level value.
	ValueMessages int

	// ByType summarises value messages grouped by the top-level type ID.
	// Entries are sorted by ValueCount descending, then by Name ascending for
	// determinism. TypeInfo from the stream's registry is also included.
	ByType []TypeStats

	// DecodedOpaques is the number of opaque values (anywhere in the tree)
	// whose registered decoder returned a non-nil Decoded value.
	DecodedOpaques int
	// UndecodedOpaques is the number of opaque values (anywhere in the tree)
	// for which no decoder matched — their Decoded field is nil.
	UndecodedOpaques int

	// Skipped is the count of corrupt value messages silently skipped because
	// [WithSkipCorruptValues] was enabled. Zero in strict mode.
	Skipped int
}

Stats is a population-level summary of a gob stream: message counts, byte consumption, type distribution, struct-field presence rates, and opaque decoder coverage. Obtain one with Stream.Stats.

A Stats describes the complete stream after Stream.Stats has drained it; partial stats from an aborted pass are not exposed.

func (*Stats) Format added in v0.2.0 

func (s *Stats) Format(w io.Writer) error

Format writes a human-readable summary of s to w. The output is a small number of lines — total message and byte counts, then a per-type table, then opaque coverage.

func (*Stats) JSON added in v0.2.0 

func (s *Stats) JSON() ([]byte, error)

JSON returns the stats as a JSON document. Convenient for downstream tools that want to aggregate stats across many files.

func (*Stats) JSONIndent added in v0.2.0 

func (s *Stats) JSONIndent(prefix, indent string) ([]byte, error)

JSONIndent is like Stats.JSON but with indentation.

type Stream 

type Stream struct {
	// contains filtered or unexported fields
}

Stream represents an in-progress decoding of a gob stream. Obtain one with Inspector.Stream. A Stream is consumed by calling Stream.Values, which returns an iterator over top-level decoded values. Type definitions encountered during decoding are accumulated and accessible via Stream.Types at any point.

A Stream is single-use: calling Values a second time panics. It is not safe for concurrent use.

A Stream does not own its reader. The caller is responsible for closing the underlying io.Reader if needed.

func (*Stream) Collect 

func (s *Stream) Collect() ([]Value, error)

Collect drains the remainder of the stream and returns all decoded values. If Values has been partially consumed, Collect returns only the values that had not yet been yielded. On error, returns the values collected before the error alongside it.

func (*Stream) Messages added in v0.2.0 

func (s *Stream) Messages() iter.Seq2[MessageInfo, error]

Messages returns an iterator that yields one MessageInfo per length- prefixed gob message in the stream, *without* decoding values. This is a cheap way to profile a stream (per-message byte count, type-ID distribution) or to locate message boundaries for tooling that processes the raw frames.

The stream is consumed by Messages just like by Values: you cannot call Values on a stream after Messages has drained it, and you cannot call Messages twice.

Unlike Stream.Values, Messages does not register type definitions against the decoder's type registry. If you need both decoded values and per- message framing, use Values and call Stream.TypeByID / Stream.Types at each step, or read the underlying Stream.Messages separately on a second Inspector.Stream over a rewound reader.

func (*Stream) Schema 

func (s *Stream) Schema() (*Schema, error)

Schema drains the remainder of the stream and returns a formatted Schema over every type definition encountered. All value messages are decoded and discarded; only type information survives. Returns the partial schema alongside any error.

This implementation decodes value bodies fully and throws them away. It is not optimized for the types-only case.

func (*Stream) SkipCount added in v0.2.0 

func (s *Stream) SkipCount() int

SkipCount reports the number of value messages silently skipped so far because WithSkipCorruptValues is enabled and they failed to decode. When WithSkipCorruptValues is disabled this is always zero.

func (*Stream) Stats added in v0.2.0 

func (s *Stream) Stats() (*Stats, error)

Stats drains the remainder of the stream, accumulating per-type and per-field statistics. Returns a partial Stats alongside any error. Value- level decode failures count toward Skipped when WithSkipCorruptValues is enabled and otherwise abort the pass.

Stats consumes the stream — a subsequent call to Values or Messages will panic just as after a normal Collect.

func (*Stream) TypeByID 

func (s *Stream) TypeByID(id int) (TypeInfo, bool)

TypeByID returns the TypeInfo for the given stream-scoped type ID, if known.

func (*Stream) Types 

func (s *Stream) Types() []TypeInfo

Types returns the live slice of type definitions encountered so far, in the order they appeared on the wire. The slice grows as the stream is consumed; callers iterating Values may call Types() at each step to see types defined up to and including the value just yielded.

TypeRef.Name fields are resolved incrementally as new type definitions arrive. When a type references another type that hasn't been seen yet, the referring TypeRef.Name is initially empty; it is filled in as soon as the referenced type's definition is processed.

The returned slice is owned by the Stream and must not be mutated. If a stable snapshot is needed, the caller should copy it with slices.Clone.

func (*Stream) Values 

func (s *Stream) Values() iter.Seq2[Value, error]

Values returns an iterator yielding one decoded Value per top-level gob value in the stream. On error it yields (nil, err) and stops. Early break from the loop is safe; the iterator will not read past the break point.

By the time a value v is yielded, every TypeInfo that v's type graph references has been appended to the slice returned by Types, and all TypeRef.Name fields within those types have been resolved against definitions seen so far. Callers may call Types() from inside the loop body to look up the type definition for the value just received.

A Stream is single-use. Calling Values on an already-consumed Stream panics.

type StringValue 

type StringValue struct{ V string }

StringValue holds a decoded string.

func (StringValue) TypeID 

func (StringValue) TypeID() int

type StructValue 

type StructValue struct {
	TypeName  string
	GobTypeID int
	Fields    []Field
}

StructValue represents a decoded gob struct.

func (StructValue) TypeID 

func (v StructValue) TypeID() int

TypeID implementations — composites and OpaqueValue return their stream-scoped type ID; pure scalars return 0.

type Style 

type Style struct {
	Prefix, Suffix string
}

Style is a prefix/suffix pair used to wrap rendered tokens with markup or escape sequences. A zero-valued Style applies no wrapping.

func (Style) Apply added in v0.2.0 

func (st Style) Apply(s string) string

Apply wraps s with the Style's Prefix and Suffix. A zero-valued Style returns s unchanged — it's the identity function, not an empty wrap — so consumers can thread a ColorScheme through a rendering pipeline without branching on whether color is on.

type TypeDecl 

type TypeDecl struct {
	Name       string
	Kind       TypeKind
	Fields     []FieldDecl // Populated if Kind is KindStruct
	TargetType string      // Target expression for slices, maps, arrays
	Annotation string      // Opaque type annotations (GobEncoder, etc.)
}

TypeDecl represents a top-level type declaration in the schema.

type TypeInfo 

type TypeInfo struct {
	ID     int
	Name   string
	Kind   TypeKind
	Fields []FieldInfo // non-nil only for KindStruct
	Key    *TypeRef    // non-nil only for KindMap
	Elem   *TypeRef    // non-nil for KindMap, KindSlice, KindArray
	Len    int         // non-zero only for KindArray
}

TypeInfo describes a single type definition encountered in a gob stream.

type TypeKind 

type TypeKind int

TypeKind classifies a type definition found in a gob stream. 

const (
	KindStruct TypeKind = iota
	KindMap
	KindSlice
	KindArray
	KindGobEncoder
	KindBinaryMarshaler
	KindTextMarshaler
)

func (TypeKind) String 

func (k TypeKind) String() string

type TypeRef 

type TypeRef struct {
	ID   int
	Name string // resolved name if available in the registry
}

TypeRef is a reference to another type by its stream-scoped ID.

type TypeStats added in v0.2.0 

type TypeStats struct {
	TypeID     int
	Name       string
	Kind       TypeKind
	ValueCount int
	// BodyBytes is the total body-byte count of all value messages of this
	// type (excluding length prefixes). Useful for spotting which types
	// dominate the stream.
	BodyBytes int64
	// FieldPresence counts the number of value messages of this type in
	// which each struct field was present (gob omits zero-valued fields).
	// Keyed by field name; non-struct types leave it nil.
	FieldPresence map[string]int
}

TypeStats describes how a single top-level type contributed to the stream.

type UintValue 

type UintValue struct{ V uint64 }

UintValue holds a decoded unsigned integer.

func (UintValue) TypeID 

func (UintValue) TypeID() int

type Value 

type Value interface {

	// TypeID returns the stream-scoped type ID for composite and opaque values
	// (StructValue, MapValue, SliceValue, ArrayValue, OpaqueValue).
	// Pure scalar values (IntValue, UintValue, FloatValue, ComplexValue,
	// BoolValue, StringValue, BytesValue, NilValue, InterfaceValue) return 0.
	TypeID() int
	// contains filtered or unexported methods
}

Value is the sealed interface implemented by all AST node types. Use a type switch to dispatch on the concrete type.

Source Files

View all Source files

Directories

Path Synopsis
cmd
demo command
gq command
Command gq is a jq-inspired CLI for inspecting gob binary streams.
 Command gq is a jq-inspired CLI for inspecting gob binary streams.
Package diff computes structural differences between two gobspect.Value trees or two streams of gobspect.Values.
 Package diff computes structural differences between two gobspect.Value trees or two streams of gobspect.Values.
Package query provides path-based navigation of decoded gobspect Value trees.
 Package query provides path-based navigation of decoded gobspect Value trees.
Package sortval provides sorting utilities for sequences of gobspect.Value nodes.
 Package sortval provides sorting utilities for sequences of gobspect.Value nodes.
Package tabular writes gobspect.Value nodes as CSV or TSV rows.
 Package tabular writes gobspect.Value nodes as CSV or TSV rows.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.