protoon

README

protoon-go

ProtoJSON for TOON — a Go package that converts protocol buffer messages to the TOON format.

proto + toon = protoon

This package is modeled after ProtoJSON, the canonical JSON mapping for Protocol Buffers. Just as ProtoJSON defines how proto.Message values are serialized as JSON, protoon defines how they are serialized as TOON — a compact, human-readable format designed for LLM workflows.

Efficiently convert protocol buffer messages to the TOON format in Go — without an intermediate JSON representation.

Features

• Direct conversion: Uses google.golang.org/protobuf/reflect/protoreflect to inspect proto messages and build TOON data structures directly.
• Tabular optimization: Repeated proto messages with uniform schemas are automatically encoded in TOON's compact tabular form (key[N]{field1,field2}:).
• Well-known types: Supports google.protobuf.Timestamp, Duration, Struct, Value, ListValue, FieldMask, and Any.
• Configurable: Choose between enum numbers (default) or enum names, and optionally emit default/zero values.

Installation

go get github.com/apstndb/protoon-go

Usage

package main

import (
    "fmt"
    "log"

    "github.com/apstndb/protoon-go"
    "google.golang.org/protobuf/types/known/timestamppb"
)

type Person struct {
    Name  string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
    Age   int32  `protobuf:"varint,2,opt,name=age,proto3" json:"age,omitempty"`
}

func main() {
    p := &Person{Name: "Alice", Age: 30}

    b, err := protoon.Marshal(p)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(string(b))
    // Output:
    // name: Alice
    // age: 30
}

Options

// Emit enum names instead of numbers
b, err := protoon.MarshalOptions{EmitEnumNames: true}.Marshal(msg)

// Emit default/zero values so repeated messages stay in tabular form
b, err := protoon.MarshalOptions{EmitDefaultValues: true}.Marshal(msg)

// Forward toon encoder options (indent, length markers, etc.)
b, err := protoon.MarshalOptions{
    EncoderOptions: []toon.EncoderOption{toon.WithLengthMarkers(true)},
}.Marshal(msg)

How it works

1. protoon walks the proto.Message using protoreflect.Message.Range (or Descriptor().Fields() when EmitDefaultValues is set).
2. Each field value is mapped to plain Go values (string, int64, float64, bool, []any, map[string]any, or toon.Object).
3. The resulting structure is passed to toon.Marshal, which applies TOON-specific optimizations such as tabular arrays.

Because there is no JSON intermediate step, field ordering from the proto descriptor is preserved and numeric precision is maintained for int64/uint64 values (which JSON cannot represent exactly).

License

MIT

Documentation

Overview

Package protoon converts protocol buffer messages to the TOON format without an intermediate JSON representation, preserving field order and enabling TOON-specific tabular optimization for repeated messages.

Index

• func Marshal(m proto.Message) ([]byte, error)
• func MarshalAny(v any, opts ...MarshalOption) ([]byte, error)
• type MarshalOption
  • func WithEmitDefaultValues() MarshalOption
  • func WithProtoJSONCompat() MarshalOption
• type MarshalOptions
  • func (o MarshalOptions) Marshal(m proto.Message) ([]byte, error)
  • func (o MarshalOptions) MarshalAny(v any, opts ...MarshalOption) ([]byte, error)
• type UnmarshalOptions
  • func (o UnmarshalOptions) Unmarshal(data []byte, m proto.Message) error

Examples

• Marshal
• Marshal (Any)
• MarshalOptions (EmitDefaultValues)
• MarshalOptions (EmitEnumNames)

Functions

func Marshal

func Marshal(m proto.Message) ([]byte, error)

Marshal converts a proto.Message to TOON using default options.

Example

p := &testdata.Person{
	Name:   "Alice",
	Age:    30,
	Active: true,
	Tags:   []string{"go", "proto"},
	Status: testdata.Status_STATUS_ACTIVE,
}

b, err := Marshal(p)
if err != nil {
	log.Fatal(err)
}
fmt.Println(string(b))

Output:
name: Alice
age: 30
active: true
status: 1
tags[2]: go,proto

Example (Any)

inner := &testdata.Person{Name: "Alice", Age: 30}
anyMsg, err := anypb.New(inner)
if err != nil {
	log.Fatal(err)
}

b, err := Marshal(anyMsg)
if err != nil {
	log.Fatal(err)
}
fmt.Println(string(b))

Output:
name: Alice
age: 30

func MarshalAny

func MarshalAny(v any, opts ...MarshalOption) ([]byte, error)

MarshalAny converts any Go value to TOON, recursively converting proto.Message fields using protoon logic.

Types

type MarshalOption

type MarshalOption func(*MarshalOptions)

MarshalOption is a functional option for MarshalAny.

func WithEmitDefaultValues

func WithEmitDefaultValues() MarshalOption

WithEmitDefaultValues causes default-valued fields to be emitted.

func WithProtoJSONCompat

func WithProtoJSONCompat() MarshalOption

WithProtoJSONCompat enables ProtoJSON-compatible encoding for MarshalAny.

type MarshalOptions

type MarshalOptions struct {
	// EncoderOptions are forwarded to the toon encoder.
	EncoderOptions []toon.EncoderOption

	// EmitEnumNames emits the protobuf enum name as a string instead of the
	// numeric value. When false (default), enums are emitted as numbers.
	EmitEnumNames bool

	// EmitDefaultValues causes fields set to their zero value to be emitted.
	// By default, only populated fields are emitted (matching proto.Range).
	EmitDefaultValues bool

	// EmitDefaultValuesForTypes emits default fields only for messages whose
	// descriptor FullName matches one of the listed names. This is useful for
	// making specific repeated row-like message types tabular without adding
	// noise to the entire protobuf tree. If EmitDefaultValues is true, this
	// option is ignored (global wins).
	EmitDefaultValuesForTypes []protoreflect.FullName

	// EmitDefaultValuesForMessage is a predicate that can be used for more
	// advanced control over which message types should emit default values.
	// If both EmitDefaultValuesForTypes and EmitDefaultValuesForMessage are
	// set, either matching condition enables default emission for that type.
	// If EmitDefaultValues is true, this option is ignored.
	EmitDefaultValuesForMessage func(protoreflect.MessageDescriptor) bool

	// ProtoJSONCompat causes the output to match the JSON representation
	// produced by google.golang.org/protobuf/encoding/protojson.
	// This affects well-known types (Timestamp, Duration, FieldMask, Any,
	// wrapper types) and enum encoding.
	ProtoJSONCompat bool
}

MarshalOptions configures how a proto.Message is converted to TOON.

Example (EmitDefaultValues)

c := &testdata.Company{
	Name: "Acme",
	Employees: []*testdata.Person{
		{Name: "Alice", Age: 30, Active: true, Status: testdata.Status_STATUS_ACTIVE},
		{Name: "Bob", Age: 25, Active: false, Status: testdata.Status_STATUS_INACTIVE},
	},
}

b, err := MarshalOptions{EmitDefaultValues: true}.Marshal(c)
if err != nil {
	log.Fatal(err)
}
fmt.Println(string(b))

Output:
name: Acme
employees[2]{name,age,active,status}:
  Alice,30,true,1
  Bob,25,false,2

Example (EmitEnumNames)

p := &testdata.Person{
	Name:   "Alice",
	Status: testdata.Status_STATUS_ACTIVE,
}

b, err := MarshalOptions{EmitEnumNames: true}.Marshal(p)
if err != nil {
	log.Fatal(err)
}
fmt.Println(string(b))

Output:
name: Alice
status: STATUS_ACTIVE

func (MarshalOptions) Marshal

func (o MarshalOptions) Marshal(m proto.Message) ([]byte, error)

Marshal converts a proto.Message to TOON.

func (MarshalOptions) MarshalAny

func (o MarshalOptions) MarshalAny(v any, opts ...MarshalOption) ([]byte, error)

MarshalAny converts any Go value to TOON.

type UnmarshalOptions

type UnmarshalOptions struct {
	// ProtoJSONCompat causes the input to be interpreted using the same
	// conventions as google.golang.org/protobuf/encoding/protojson.
	ProtoJSONCompat bool
}

UnmarshalOptions configures how a TOON document is converted to proto.Message.

func (UnmarshalOptions) Unmarshal

func (o UnmarshalOptions) Unmarshal(data []byte, m proto.Message) error

Unmarshal parses a TOON document and populates the given proto.Message.