apery

package module
v0.0.1-rc1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 20, 2026 License: MIT Imports: 7 Imported by: 0

README

Apery — Synthetic Data Generator for Agents

Apery

Deterministic synthetic data generation for agents.

CI status GitHub release Discord MIT License

Apery generates synthetic data from declarative plans. Same plan, same seed, same output — every time. 1M rows in under 2 seconds.

AI agents are a first-class citizen. Silent stdout by default, structured logging, machine-parseable output, clean exit codes.

Install

Pre-built binary (Linux / macOS / Windows, amd64 / arm64):

# Linux x86_64 — pick the asset for your platform from the releases page
curl -L https://github.com/compuficial/apery/releases/latest/download/apery_<version>_linux_x86_64.tar.gz \
  | tar xz -C /tmp && sudo mv /tmp/apery /usr/local/bin/

See releases for all platforms.

Go install (requires Go 1.24+):

go install github.com/compuficial/apery/cmd/apery@latest

From source (requires Go 1.24+):

git clone https://github.com/compuficial/apery.git
cd apery
make install   # builds and installs to ~/.local/bin

30-Second Demo

# plan.yaml
seed: 42
entities:
  - name: User
    count: 1000
    fields:
      - name: id
        gen: seq
      - name: email
        gen: regex
        config:
          pattern: "[a-z]{5,10}@(gmail|yahoo|outlook)\\.com"
      - name: department
        gen: pick
        config:
          values: [engineering, sales, marketing, support]
          weights: [40, 30, 20, 10]

$ apery generate -f plan.yaml | head -3
{"_entity":"User","id":1,"email":"kczvbmih@outlook.com","department":"engineering"}
{"_entity":"User","id":2,"email":"yzdevl@yahoo.com","department":"engineering"}
{"_entity":"User","id":3,"email":"eoikwpvxu@gmail.com","department":"sales"}

Why Apery
Deterministic Plan + Seed = Identical Output. Always. Across parallel workers, platforms, runs.
Fast 1M rows in ~1.6s with chunked parallel execution across all cores.
Composable 20 generators that nest and combine. Objects, lists, templates, conditional dispatch.
Relational Foreign keys, 1:M parent-child, M:N junction tables. Zipf distributions for realistic skew.
Agent-first YAML/JSON plans, stdout piping, structured slog output, exit codes. No GUI, no server.
Zero config Single binary. No database, no runtime dependencies. make install and go.

Generators

Run apery list generators to see all available generators, or apery describe generator <name> for full config docs.

Scalar
Generator Description Example Config
seq Sequential integers start: 1, step: 1
int Uniform random integer min: 0, max: 100
float Uniform random float min: 0.0, max: 1.0
bool Weighted boolean probability: 0.8
pick Random from list/file/URL values: [a, b, c], weights: [5, 3, 2]
const Fixed value value: active
regex String from pattern pattern: "[A-Z]{2}-\\d{6}"
time Timestamp in range start: "2024-01-01", end: "2024-12-31"
uuid UUID v4 —
ulid ULID —
normal_int Gaussian integer mu: 50, sigma: 10
normal_float Gaussian float mu: 0.0, sigma: 1.0
zipf Zipf distribution s: 1.1, imax: 100
Composite
Generator Description
object Nested object with sub-generators per field
list Array of N items from one generator
sample N unique items without replacement
one_of Weighted random dispatch to sub-generators
template String interpolation: "{first} {last}"
switch Conditional dispatch based on another field
Relational
Generator Description
rel_ref Foreign key from a previously generated entity (uniform or zipf, optional unique: true)
driven_by 1:M parent-child — generate Min to Max children per parent row

Relational Example

seed: 99
entities:
  - name: User
    count: 100
    fields:
      - name: id
        gen: seq
      - name: name
        gen: pick
        config:
          values: [Alice, Bob, Carol, Dave]

  - name: Product
    count: 50
    fields:
      - name: id
        gen: seq
      - name: sku
        gen: regex
        config:
          pattern: "[A-Z]{2}-\\d{6}"

  - name: Order                    # 1:M — each User gets 1-5 Orders
    driven_by:
      entity: User
      field: id
      as: user_id
      min: 1
      max: 5
    fields:
      - name: order_id
        gen: seq
      - name: product_id
        gen: rel_ref
        config:
          entity: Product
          field: id
      - name: quantity
        gen: int
        config:
          min: 1
          max: 10

  - name: Review                   # M:1 with zipf skew
    count: 500
    fields:
      - name: user_id
        gen: rel_ref
        config:
          entity: User
          field: id
          distribution: zipf
          s: 1.5
      - name: product_id
        gen: rel_ref
        config:
          entity: Product
          field: id
      - name: rating
        gen: int
        config:
          min: 1
          max: 5

$ apery generate -f ecommerce.yaml --output-dir ./out --split-entities
$ ls out/
Order.jsonl  Product.jsonl  Review.jsonl  User.jsonl

$ head -1 out/Order.jsonl | jq .
{
  "user_id": 1,
  "order_id": 1,
  "product_id": 34,
  "quantity": 7
}

CLI Reference

apery generate -f plan.yaml              # JSONL to stdout
apery generate -f plan.yaml -o csv       # CSV to stdout
apery generate -f plan.yaml --output-dir ./out
apery generate -f plan.yaml --output-dir ./out --split-entities
apery generate -f plan.yaml --dry-run    # validate only
apery generate -f plan.yaml --seed 123   # override seed
apery generate -f plan.yaml --verbose    # entity progress to stderr
apery generate -f plan.yaml --debug      # full debug output to stderr

apery validate -f plan.yaml              # validate a plan file
apery list generators                    # list all generators
apery describe generator <name>          # show config schema + example
apery version                            # print version
apery help <command>                     # help for any command

Exit codes: 0 success, 1 validation error, 2 generation error, 3 I/O error.

Performance

$ cat bench.yaml
seed: 1
entities:
  - name: Row
    count: 1000000
    fields:
      - name: id
        gen: seq
      - name: value
        gen: int
        config: { min: 0, max: 1000000 }
      - name: label
        gen: pick
        config: { values: [a, b, c, d, e] }

$ time apery generate -f bench.yaml --workers 16 > /dev/null
real    0m1.6s

~600,000 rows/second on a modern desktop. Scales linearly with cores.

Determinism

$ apery generate -f plan.yaml --seed 42 | md5sum
fc8756b572010e94b46afc81ecbe6a02  -
$ apery generate -f plan.yaml --seed 42 | md5sum
fc8756b572010e94b46afc81ecbe6a02  -

Hierarchical seed derivation ensures identical output regardless of worker count or chunk size. See the spec for the full seed derivation model.

Go Library

import "apery"

p, _ := apery.LoadPlanFile("plan.yaml")

w, _ := apery.NewJSONLWriter("output.jsonl")
apery.Run(ctx, p, w,
    apery.WithWorkers(16),
    apery.WithChunkSize(100000),
)

Architecture

Plan (YAML/JSON) --> Validation --> Registry (20 Generators) --> Runtime (Parallel Executor) --> Writer (JSONL/CSV)
                                         |                            |
                                    GeneratorInfo               slog structured
                                    (self-describing)            logging

Documentation

License

MIT

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewCSVWriter 

func NewCSVWriter(path string) (*writer.CSVWriter, error)

func NewCSVWriterFromWriter 

func NewCSVWriterFromWriter(w io.Writer) *writer.CSVWriter

func NewJSONLWriter 

func NewJSONLWriter(path string) (*writer.JSONLWriter, error)

func NewJSONLWriterFromWriter 

func NewJSONLWriterFromWriter(w io.Writer) *writer.JSONLWriter

func NewSplitWriter 

func NewSplitWriter(dir, format string) *writer.SplitWriter

func Run 

func Run(ctx context.Context, p *Plan, w Writer, opts ...Option) error

func ValidatePlan 

func ValidatePlan(p *Plan) error

Types

type ConfigKey 

type ConfigKey = registry.ConfigKey

type DrivenBy 

type DrivenBy = plan.DrivenBy

type EntitySpec 

type EntitySpec = plan.EntitySpec

type FieldSpec 

type FieldSpec = plan.FieldSpec

type GeneratorInfo 

type GeneratorInfo = registry.GeneratorInfo

func ListGenerators 

func ListGenerators() []GeneratorInfo

type Option 

type Option = runtime.Option

func WithChunkSize 

func WithChunkSize(n int64) Option

func WithLogger 

func WithLogger(logger *slog.Logger) Option

func WithWorkers 

func WithWorkers(n int) Option

type OrderedMap 

type OrderedMap = writer.OrderedMap

type Plan 

type Plan = plan.Plan

func LoadPlanFile 

func LoadPlanFile(path string) (*Plan, error)

type Writer 

type Writer = writer.Writer

Source Files

View all Source files

Directories

Path Synopsis
cmd
apery command
internal
plan
Package plan defines the declarative schema for synthetic data generation.
 Package plan defines the declarative schema for synthetic data generation.
registry
Package registry provides the generator plugin system for synthetic data generation.
 Package registry provides the generator plugin system for synthetic data generation.
rng
Package rng provides deterministic random number generation with hierarchical seed derivation.
 Package rng provides deterministic random number generation with hierarchical seed derivation.
runtime
Package runtime orchestrates the execution of synthetic data generation plans.
 Package runtime orchestrates the execution of synthetic data generation plans.
writer
Package writer provides output format abstraction for synthetic data generation.
 Package writer provides output format abstraction for synthetic data generation.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.