generator

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 29, 2026 License: MIT Imports: 1 Imported by: 0

README

SofaBuffers

SofaBuffers

Structured Objects For Anyone
... so optimized, feels amazing.

Would you like to know more?

SofaBuffers Code Generator

This repository holds the SofaBuffers code generator — the tool that turns a declarative YAML/JSON object definition (validated against schema/sofabuffers-schema-v1.json) into idiomatic, typed source code for every supported language. The generated code is a thin, zero-overhead wrapper that calls into the highly-optimized corelib runtime for its language, so the hard part — a fast, portable, footprint-tuned wire codec with a uniform streaming API — is owned by the corelibs, not the generated code.

Status: implemented — all 8 language backends complete and CI-green. The generator (sbufgen) emits typed code for C, Go, Python, TypeScript, C++, Rust, C#, and Java; each is built against its real corelib in CI, JSON round-trips every field kind, and is byte-exact against the shared wire vectors. The full design lives in docs/PLAN.md and the living architecture in docs/ARCHITECTURE.md.

Quick start

# Build the single static binary (or grab one from a release).
go build -o sbufgen ./cmd/sbufgen

# Generate typed sources for one language from a definition.
./sbufgen --lang go --in examples/messages/example.yaml --out out/go

# Generate for every language.
for lang in c cpp go python typescript rust csharp java; do
  ./sbufgen --lang "$lang" --in examples/messages/example.yaml --out "out/$lang"
done

# Scaffold a full buildable project + encode/decode harness (per PLAN §7):
#   sbufgen --config myconfig.yaml --lang rust --in examples --out out

Examples:

Want to see the generated output without running anything? Every CI run attaches the generated sources for each language as a downloadable artifact named generated-<lang> (see the Artifacts section of any CI run). Each backend's one-command conformance harness is tests/<lang>/run.sh.

What it does
  1. Reads an object/message definition in YAML (JSON also accepted).
  2. Validates it against the JSON Schema first — a hard, non-optional gate. Invalid input produces a clear, located error, a non-zero exit, and no output. Invalid definitions are never code-generated.
  3. Resolves $ref into a shared-type graph and lowers the definition into a language-neutral Intermediate Representation (IR).
  4. Emits one typed serialize/deserialize type per object for the selected target language, tuned to that corelib's profile (minimal-footprint vs. maximum-throughput).
  5. Ships as a single, statically-linked, cross-platform binary (sbufgen).
Supported targets

The generator emits code against one corelib per language:

Target Corelib Profile
C (object.h) corelib-c-cpp Embedded, minimal footprint, no heap
C++ (embedded) corelib-c-cpp (sofab.hpp) Embedded
C++ (max speed) corelib-cpp Header-only C++20, zero-copy
Rust (no_std) corelib-rs-no-std Embedded, no alloc by default
Go corelib-go High throughput
Python corelib-py High throughput
Java corelib-java High throughput
C# / .NET corelib-cs High throughput
TypeScript corelib-ts High throughput

Because every corelib speaks the same wire format, code generated for one language interoperates with code generated for any other for free.

CLI (planned)

The CLI is deliberately tiny — everything configurable lives in a config file:

sbufgen --config <file> --lang <c|cpp-embedded|cpp|rust|go|python|java|csharp|ts> \
        [--in <dir>] [--out <dir>]
Argument Required Purpose
--config <file> yes YAML/JSON config carrying all other options
--lang <target> yes Which backend to generate
--in <dir> no Override the config's input definition folder
--out <dir> no Override the config's output folder

Repository layout

.
├── schema/
│   └── sofabuffers-schema-v1.json   # the message-definition JSON Schema (draft-07)
├── docs/
│   ├── PLAN.md                      # full implementation plan & design
│   └── ARCHITECTURE.md              # living architecture doc (created at M0)
├── assets/                          # logo & icon
├── .devcontainer/                   # Go toolchain dev container
└── LICENSE                          # MIT

The definition schema (schema/sofabuffers-schema-v1.json) is authoritative for the input format: field types (u8–u64, i8–i64, fp32/fp64, boolean, string, blob, array, enum, bitfield, struct, union), unique field ids, $ref-able $defs, and the custom keywords uniqueIds and defaultMatchesEnum.

Development

A .devcontainer (Ubuntu 24.04 + Go) is provided. To use it locally, copy the secrets template first (the real .env is gitignored):

cp .devcontainer/.env.example .devcontainer/.env

Then open the folder in a devcontainer-aware editor, or build the image directly via the scripts under .devcontainer/. The recommended generator implementation language is Go (single static binary, frictionless GOOS/GOARCH cross-compilation); see docs/PLAN.md §2.1 for the rationale.

Documentation

  • docs/PLAN.md — the complete design: input format, corelib runtime contract, generated-code shape, optimization strategy, configuration, generator architecture (Composite / Visitor / Builder / Strategy), the testing & conformance matrix, and the phased roadmap (M0–M8).
  • docs/ARCHITECTURE.md — the maintained, up-to-date description of how the generator works (kept current per the per-milestone rule).

Documentation

Overview

Package generator is the module root. It exists only to embed the shipped JSON Schema artifacts so the single static binary carries them with no runtime file dependency. Code that needs a schema reads it from SchemaFS.

Index

Constants

View Source 
const (
	MessageSchemaPath = "schema/sofabuffers-schema-v1.json"
	ConfigSchemaPath  = "schema/sbufgen-config-schema.json"
)

ConfigSchemaPath / MessageSchemaPath are the in-FS names.

Variables

View Source 
var SchemaFS embed.FS

SchemaFS holds the authoritative message-definition schema and the config schema. They remain the source of truth in schema/; this just bundles them.

Functions

This section is empty.

Types

This section is empty.

Source Files

View all Source files

Directories

Path Synopsis
cmd
sbufgen command
Command sbufgen is the SofaBuffers code generator CLI (PLAN §8.8).
 Command sbufgen is the SofaBuffers code generator CLI (PLAN §8.8).
generators
c
Package c is the embedded-C backend (PLAN §6.2): it emits descriptor-driven code against corelib-c-cpp's object.h API — a struct + a static sofab_object_descr_field_t[] table + a sofab_object_descr_t per object, plus thin encode/decode/init wrappers.
 Package c is the embedded-C backend (PLAN §6.2): it emits descriptor-driven code against corelib-c-cpp's object.h API — a struct + a static sofab_object_descr_field_t[] table + a sofab_object_descr_t per object, plus thin encode/decode/init wrappers.
cpp
Package cpp is the max-speed C++ backend (PLAN §6.3): header-only output against corelib-cpp.
 Package cpp is the max-speed C++ backend (PLAN §6.3): header-only output against corelib-cpp.
csharp
Package csharp is the C#/.NET throughput backend (PLAN §6.4): classes with Marshal over OStream and a flat-visitor decode (IVisitor) against corelib-cs.
 Package csharp is the C#/.NET throughput backend (PLAN §6.4): classes with Marshal over OStream and a flat-visitor decode (IVisitor) against corelib-cs.
golang
Package golang is the Go throughput backend (PLAN §6.4): it emits one struct per object with Marshal (streaming Encoder) and a pull-parser Unmarshal (Decoder.Next/Skip + typed readers) against corelib-go.
 Package golang is the Go throughput backend (PLAN §6.4): it emits one struct per object with Marshal (streaming Encoder) and a pull-parser Unmarshal (Decoder.Next/Skip + typed readers) against corelib-go.
java
Package java is the Java throughput backend (PLAN §6.4): classes with marshal over OStream and a flat-visitor decode (Visitor) against corelib-java.
 Package java is the Java throughput backend (PLAN §6.4): classes with marshal over OStream and a flat-visitor decode (Visitor) against corelib-java.
python
Package python is the Python throughput backend (PLAN §6.4): it emits dataclasses with _marshal (Encoder) and a pull-parser _unmarshal (Decoder.next/skip + typed readers) against corelib-py, plus JSON helpers for the conformance harness.
 Package python is the Python throughput backend (PLAN §6.4): it emits dataclasses with _marshal (Encoder) and a pull-parser _unmarshal (Decoder.next/skip + typed readers) against corelib-py, plus JSON helpers for the conformance harness.
rust
Package rust is the Rust backend (PLAN §6.2, embedded/no_std-capable): structs with marshal() over OStream and a flat-visitor decode against corelib-rs-no-std.
 Package rust is the Rust backend (PLAN §6.2, embedded/no_std-capable): structs with marshal() over OStream and a flat-visitor decode against corelib-rs-no-std.
typescript
Package typescript is the TypeScript throughput backend (PLAN §6.4): it emits one class per object with marshal(OStream) and a visitor-based decode against corelib-ts (@sofabuffers/corelib).
 Package typescript is the TypeScript throughput backend (PLAN §6.4): it emits one class per object with marshal(OStream) and a visitor-based decode against corelib-ts (@sofabuffers/corelib).
internal
analysis
Package analysis implements stage [3] of the pipeline: it resolves the IR's $ref/shared-type graph in place and runs the language-independent semantic checks, then freezes the IR (PLAN §8.2).
 Package analysis implements stage [3] of the pipeline: it resolves the IR's $ref/shared-type graph in place and runs the language-independent semantic checks, then freezes the IR (PLAN §8.2).
config
Package config loads and validates sbufgen configuration (PLAN §7).
 Package config loads and validates sbufgen configuration (PLAN §7).
generator
Package generator holds the backend CONTRACT only — interfaces, no language code (PLAN §8.6/§8.7).
 Package generator holds the backend CONTRACT only — interfaces, no language code (PLAN §8.6/§8.7).
ir
Package ir is the language-neutral Intermediate Representation that every backend consumes (PLAN §8.2).
 Package ir is the language-neutral Intermediate Representation that every backend consumes (PLAN §8.2).
model
Package model implements stage [2] of the pipeline: it lowers a validated, unresolved definition document into the language-neutral ir.Schema (Composite).
 Package model implements stage [2] of the pipeline: it lowers a validated, unresolved definition document into the language-neutral ir.Schema (Composite).
parser
Package parser implements stage [1] of the generation pipeline: it loads a SofaBuffers message-definition document (YAML or JSON) and validates it against the v1 schema as a HARD GATE (PLAN §1, §8.1).
 Package parser implements stage [1] of the generation pipeline: it loads a SofaBuffers message-definition document (YAML or JSON) and validates it against the v1 schema as a HARD GATE (PLAN §1, §8.1).
pipeline
Package pipeline orchestrates the generation stages (PLAN §8 diagram):
 Package pipeline orchestrates the generation stages (PLAN §8 diagram):

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.