generator

package module
v0.23.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 21, 2026 License: MIT Imports: 1 Imported by: 0

README

SofaBuffers

SofaBuffers

Structured Objects For Anyone
... so optimized, feels amazing.

Would you like to know more?

SofaBuffers Code Generator

This repository holds the SofaBuffers code generator — the tool that turns a declarative YAML/JSON object definition (validated against schema/sofabuffers-schema-v1.json) into idiomatic, typed source code for every supported language. The generated code is a thin, zero-overhead wrapper that calls into the highly-optimized corelib runtime for its language, so the hard part — a fast, portable, footprint-tuned wire codec with a uniform streaming API — is owned by the corelibs, not the generated code.

The generator (sofabgen) emits typed code for C, Go, Python, TypeScript, C++, Rust, C#, Java, Kotlin, Zig, and Dart. Every backend is built against its real corelib, JSON round-trips every field kind, and is byte-exact against the shared wire vectors — so code generated for one language interoperates with any other.

Installation

sofabgen is a single static binary with no runtime dependencies. Every channel below ships the same binary from the same release, so they are interchangeable — pick whichever fits how you already manage tools.

Channel Install Best for
Script curl -fsSL …/install.sh | sh a workstation or a container image
npm npm install --save-dev @sofa-buffers/generator JS/TS projects, pinned per project
PyPI uv tool install sofabgen Python projects, or a standalone CLI
GitHub Actions uses: …/setup-sofabgen@v0.22.0 CI
Go toolchain go install …/cmd/sofabgen@latest you already build Go
From source go build -o sofabgen ./cmd/sofabgen hacking on the generator
One-line install

Prebuilt static binaries for Linux, Windows and macOS (x86 and ARM, 32- and 64-bit) are attached to every release. This grabs the latest — with OS/arch detection and SHA-256 verification:

curl -fsSL https://raw.githubusercontent.com/sofa-buffers/generator/main/install.sh | sh

Set SOFABGEN_VERSION=vX.Y.Z to pin a release, or SOFABGEN_INSTALL_DIR=<dir> to choose where it lands.

npm

A project-local dev dependency: the prebuilt binary ships as a per-platform optional dependency, so there is no Go toolchain and no postinstall download, and the binary is integrity-hashed in your lockfile. The package is @sofa-buffers/generator, the command is sofabgen (see packaging/npm/):

npm install --save-dev @sofa-buffers/generator
npx sofabgen --lang typescript --in messages/ --out src/generated/
PyPI

A plain CLI — one wheel per platform, each holding nothing but the binary, so pip puts sofabgen straight on your PATH (no Python code, no build step; see packaging/pypi/):

uv tool install sofabgen     # or: pipx install sofabgen / pip install sofabgen
uvx sofabgen --lang python --in messages/ --out src/generated/
GitHub Actions

The bundled composite action installs the CLI and puts it on PATH for later steps:

- uses: sofa-buffers/generator/.github/actions/setup-sofabgen@v0.22.0
  with:
    version: v0.22.0   # optional; defaults to the latest release
- run: sofabgen --version
Go toolchain
go install github.com/sofa-buffers/generator/cmd/sofabgen@latest

The binary self-reports the installed version.

From source
go build -o sofabgen ./cmd/sofabgen

Quick start

# Generate typed sources for one language from a definition.
sofabgen --lang go --in examples/messages/example.yaml --out out/go

# Generate for every language.
for lang in c cpp go python typescript rust csharp java kotlin zig dart; do
  sofabgen --lang "$lang" --in examples/messages/example.yaml --out "out/$lang"
done

# Render the definitions as a self-contained HTML reference page instead:
sofabgen --lang docs --in examples/messages/example.yaml --out out/docs

# Scaffold a full buildable project + encode/decode harness:
#   sofabgen --config myconfig.yaml --lang rust --in examples --out out

Examples:

Each backend has a one-command conformance harness at tests/conformance/<lang>/run.sh that generates the example, builds it against the real corelib, and round-trips it.

The "minimal-footprint vs. maximum-throughput" claim below is measured, not asserted: tests/bench/ records instructions/op and embedded .text for every (language × corelib) combination in a committed results.txt, so a codegen change that costs or saves shows up as a diff.

What it does
  1. Reads an object/message definition in YAML (JSON also accepted).
  2. Validates it against the JSON Schema first — a hard, non-optional gate. Invalid input produces a clear, located error, a non-zero exit, and no output. Invalid definitions are never code-generated.
  3. Resolves $ref into a shared-type graph and lowers the definition into a language-neutral Intermediate Representation (IR).
  4. Emits one typed serialize/deserialize type per object for the selected target language, tuned to that corelib's profile (minimal-footprint vs. maximum-throughput).
  5. Ships as a single, statically-linked, cross-platform binary (sofabgen).
Supported targets

The generator emits code against one corelib per language:

Target Corelib Profile
C (object.h) corelib-c-cpp Embedded, minimal footprint, no heap
C++ (embedded) corelib-c-cpp (sofab.hpp) Embedded friendly
C++ (max speed) corelib-cpp Max speed, zero-copy decode
Rust (no_std) corelib-rs-no-std Embedded, no alloc by default
Rust (std) corelib-rs Max speed
Go corelib-go Max speed
Python corelib-py Max speed
Java corelib-java Max speed
Kotlin (Multiplatform) corelib-kotlin-mp Max speed; one source set for JVM, JS and native
C# / .NET corelib-cs Max speed
TypeScript corelib-ts Max speed
Zig corelib-zig Max speed, zero-copy decode
Dart corelib-dart Max speed

Because every corelib speaks the same wire format, code generated for one language interoperates with code generated for any other for free.

Beyond the language backends there is one non-code target: docs renders the definitions as a self-contained HTML reference page (messages, field tables, cross-linked named types — see docs/generator/docs.md).

CLI

The CLI is deliberately tiny — everything configurable lives in a config file:

sofabgen --config <file> --lang <c|cpp|rust|go|python|java|kotlin|csharp|typescript|zig|dart|docs> \
        [--in <dir>] [--out <dir>]
Argument Required Purpose
--config <file> yes YAML/JSON config carrying all other options
--lang <target> yes Which backend to generate
--in <dir> no Override the config's input definition folder
--out <dir> no Override the config's output folder

Repository layout

.
├── cmd/sofabgen/        # the `sofabgen` CLI entry point
├── generators/        # one backend per target (c, cpp, rust, golang, python, java, kotlin, csharp, typescript, zig, dart, docs)
├── internal/          # parser, validator, IR, analysis, generation pipeline
├── schema/            # the message-definition JSON Schema (draft-07)
├── examples/          # example definitions (config/ + messages/)
├── tests/             # conformance harnesses, the hermetic matrix, and bench/ (Ir/op + footprint)
├── docs/              # architecture & design
├── assets/            # logo & icon
├── .devcontainer/     # dev container: Go + the conformance-harness toolchains
└── LICENSE            # MIT

The definition schema (schema/sofabuffers-schema-v1.json) is authoritative for the input format: field types (u8–u64, i8–i64, fp32/fp64, boolean, string, blob, array, enum, bitfield, struct, union), unique field ids, $ref-able $defs, and the custom keywords uniqueIds and defaultMatchesEnum.

Development

A .devcontainer (Ubuntu 26.04 + Go, plus the toolchains the per-language conformance harnesses shell out to: C/C++, Zig, Rust, Java/Maven, Kotlin/Gradle, .NET, Python, Node) is provided. To use it locally, copy the secrets template first (the real .env is gitignored):

cp .devcontainer/.env.example .devcontainer/.env

Then open the folder in a devcontainer-aware editor, or build the image directly via the scripts under .devcontainer/. The generator is written in Go — a single static binary with frictionless GOOS/GOARCH cross-compilation.

Documentation

  • docs/ARCHITECTURE.md — how the generator is structured: the parser → validator → IR → backend pipeline, the per-corelib decode models, and the design patterns (Composite / Visitor / Builder / Strategy) that hold it together.
  • docs/PLAN.md — the design rationale: input format, corelib runtime contract, generated-code shape, optimization strategy, and configuration.

Documentation

Overview

Package generator is the module root. It exists only to embed the shipped JSON Schema artifacts so the single static binary carries them with no runtime file dependency. Code that needs a schema reads it from SchemaFS.

Index

Constants

View Source 
const (
	MessageSchemaPath = "schema/sofabuffers-schema-v1.json"
	ConfigSchemaPath  = "schema/sofabgen-config-schema.json"
)

ConfigSchemaPath / MessageSchemaPath are the in-FS names.

Variables

View Source 
var SchemaFS embed.FS

SchemaFS holds the authoritative message-definition schema and the config schema. They remain the source of truth in schema/; this just bundles them.

Functions

This section is empty.

Types

This section is empty.

Source Files

View all Source files

Directories

Path Synopsis
cmd
sofabgen command
Command sofabgen is the SofaBuffers code generator CLI (PLAN §8.8).
 Command sofabgen is the SofaBuffers code generator CLI (PLAN §8.8).
generators
c
Package c is the embedded-C backend (PLAN §6.2): it emits descriptor-driven code against corelib-c-cpp's object.h API — a struct + a static sofab_object_descr_field_t[] table + a sofab_object_descr_t per object, plus thin encode/decode/init wrappers.
 Package c is the embedded-C backend (PLAN §6.2): it emits descriptor-driven code against corelib-c-cpp's object.h API — a struct + a static sofab_object_descr_field_t[] table + a sofab_object_descr_t per object, plus thin encode/decode/init wrappers.
cpp
Package cpp is the max-speed C++ backend (PLAN §6.3): header-only output against corelib-cpp.
 Package cpp is the max-speed C++ backend (PLAN §6.3): header-only output against corelib-cpp.
csharp
Package csharp is the C#/.NET throughput backend (PLAN §6.4): classes with Serialize over OStream and a flat-visitor decode (IVisitor) against corelib-cs.
 Package csharp is the C#/.NET throughput backend (PLAN §6.4): classes with Serialize over OStream and a flat-visitor decode (IVisitor) against corelib-cs.
dart
Package dart is the Dart throughput backend (PLAN §6.4): plain classes with a streaming `serialize` over the corelib Encoder and a push child-visitor decode against corelib-dart.
 Package dart is the Dart throughput backend (PLAN §6.4): plain classes with a streaming `serialize` over the corelib Encoder and a push child-visitor decode against corelib-dart.
docs
Package docs is the documentation backend: it renders the message definitions as human-readable reference documentation instead of source code.
 Package docs is the documentation backend: it renders the message definitions as human-readable reference documentation instead of source code.
golang
Package golang is the Go throughput backend (PLAN §6.4): it emits one struct per object with Marshal (streaming Encoder) and a pull-parser Unmarshal (Decoder.Next/Skip + typed readers) against corelib-go.
 Package golang is the Go throughput backend (PLAN §6.4): it emits one struct per object with Marshal (streaming Encoder) and a pull-parser Unmarshal (Decoder.Next/Skip + typed readers) against corelib-go.
java
Package java is the Java throughput backend (PLAN §6.4): classes with serialize over OStream and a flat-visitor decode (Visitor) against corelib-java.
 Package java is the Java throughput backend (PLAN §6.4): classes with serialize over OStream and a flat-visitor decode (Visitor) against corelib-java.
kotlin
Package kotlin is the Kotlin Multiplatform throughput backend: classes with serialize over OStream and a flat-visitor decode (Visitor) against corelib-kotlin-mp.
 Package kotlin is the Kotlin Multiplatform throughput backend: classes with serialize over OStream and a flat-visitor decode (Visitor) against corelib-kotlin-mp.
python
Package python is the Python throughput backend (PLAN §6.4): it emits dataclasses with _marshal (Encoder) and a pull-parser _unmarshal (Decoder.next/skip + typed readers) against corelib-py, plus JSON helpers for the conformance harness.
 Package python is the Python throughput backend (PLAN §6.4): it emits dataclasses with _marshal (Encoder) and a pull-parser _unmarshal (Decoder.next/skip + typed readers) against corelib-py, plus JSON helpers for the conformance harness.
rust
Package rust is the Rust backend (PLAN §6.2, embedded/no_std-capable): structs with serialize() over OStream and a flat-visitor decode.
 Package rust is the Rust backend (PLAN §6.2, embedded/no_std-capable): structs with serialize() over OStream and a flat-visitor decode.
typescript
Package typescript is the TypeScript throughput backend (PLAN §6.4): it emits one class per object with serialize(OStream) and a visitor-based decode against corelib-ts (@sofa-buffers/corelib).
 Package typescript is the TypeScript throughput backend (PLAN §6.4): it emits one class per object with serialize(OStream) and a visitor-based decode against corelib-ts (@sofa-buffers/corelib).
zig
Package zig is the Zig backend (corelib-zig, the max-speed port): structs with schema defaults in the type declaration, marshal() over OStream, and a flat-visitor decode.
 Package zig is the Zig backend (corelib-zig, the max-speed port): structs with schema defaults in the type declaration, marshal() over OStream, and a flat-visitor decode.
internal
analysis
Package analysis implements stage [3] of the pipeline: it resolves the IR's $ref/shared-type graph in place and runs the language-independent semantic checks, then freezes the IR (PLAN §8.2).
 Package analysis implements stage [3] of the pipeline: it resolves the IR's $ref/shared-type graph in place and runs the language-independent semantic checks, then freezes the IR (PLAN §8.2).
config
Package config loads and validates sofabgen configuration (PLAN §7).
 Package config loads and validates sofabgen configuration (PLAN §7).
generator
Package generator holds the backend CONTRACT only — interfaces, no language code (PLAN §8.6/§8.7).
 Package generator holds the backend CONTRACT only — interfaces, no language code (PLAN §8.6/§8.7).
ir
Package ir is the language-neutral Intermediate Representation that every backend consumes (PLAN §8.2).
 Package ir is the language-neutral Intermediate Representation that every backend consumes (PLAN §8.2).
model
Package model implements stage [2] of the pipeline: it lowers a validated, unresolved definition document into the language-neutral ir.Schema (Composite).
 Package model implements stage [2] of the pipeline: it lowers a validated, unresolved definition document into the language-neutral ir.Schema (Composite).
parser
Package parser implements stage [1] of the generation pipeline: it loads a SofaBuffers message-definition document (YAML or JSON) and validates it against the v1 schema as a HARD GATE (PLAN §1, §8.1).
 Package parser implements stage [1] of the generation pipeline: it loads a SofaBuffers message-definition document (YAML or JSON) and validates it against the v1 schema as a HARD GATE (PLAN §1, §8.1).
pipeline
Package pipeline orchestrates the generation stages (PLAN §8 diagram):
 Package pipeline orchestrates the generation stages (PLAN §8 diagram):

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.