prost

README

go-protoc-gen-prost

A Go module that embeds protoc-gen-prost as a WebAssembly module for pure-Go Protocol Buffer code generation.

Related Projects

• aperturerobotics/protoc-gen-prost - Fork with WASI build support
• neoeinstein/protoc-gen-prost - Original protoc-gen-prost repository
• tetratelabs/wazero - Zero-dependency WebAssembly runtime for Go

About

This module provides the protoc-gen-prost Protocol Buffers code generator compiled to WebAssembly with WASI support. The WASM binary is embedded directly in the Go module, enabling Rust/Prost code generation in Go applications without external dependencies or native binaries.

Exported Functions

The WASM module exports the following functions:

• prost_malloc(size) - Allocate memory for input data
• prost_free(ptr, size) - Free allocated memory
• prost_execute(input_ptr, input_len) - Execute the plugin, returns output length
• prost_get_output_ptr() - Get pointer to output buffer
• prost_get_output_len() - Get output buffer length
• prost_clear_output() - Clear the output buffer

How It Works

1. The host allocates memory in WASM using prost_malloc
2. The host writes the serialized CodeGeneratorRequest to that memory
3. The host calls prost_execute with the pointer and length
4. The plugin processes the request and stores the CodeGeneratorResponse internally
5. The host reads the response using prost_get_output_ptr and prost_get_output_len
6. The host calls prost_clear_output to free the internal buffer

Features

• Embeds protoc-gen-prost as a ~600KB WASI WebAssembly binary
• Pure Go execution via wazero (no CGO required)
• Thread-safe with mutex protection
• Supports repeated executions without reloading

Usage

package main

import (
    "context"
    "fmt"

    "github.com/tetratelabs/wazero"
    "google.golang.org/protobuf/proto"
    "google.golang.org/protobuf/types/pluginpb"
    prost "github.com/aperturerobotics/go-protoc-gen-prost"
)

func main() {
    ctx := context.Background()
    r := wazero.NewRuntime(ctx)
    defer r.Close(ctx)

    // Create the protoc-gen-prost instance
    p, err := prost.NewProtocGenProst(ctx, r)
    if err != nil {
        panic(err)
    }
    defer p.Close(ctx)

    // Create a CodeGeneratorRequest
    req := &pluginpb.CodeGeneratorRequest{
        FileToGenerate: []string{"example.proto"},
        ProtoFile: []*descriptorpb.FileDescriptorProto{
            // ... your proto file descriptors
        },
    }

    // Serialize the request
    input, err := proto.Marshal(req)
    if err != nil {
        panic(err)
    }

    // Execute the plugin
    output, err := p.Execute(ctx, input)
    if err != nil {
        panic(err)
    }

    // Parse the response
    resp := &pluginpb.CodeGeneratorResponse{}
    if err := proto.Unmarshal(output, resp); err != nil {
        panic(err)
    }

    // Process generated files
    for _, file := range resp.GetFile() {
        fmt.Printf("Generated: %s\n", file.GetName())
    }
}

Updating the WASM Binary

To update to a new version of protoc-gen-prost:

./update-prost.bash

This script:

1. Fetches the latest release from aperturerobotics/protoc-gen-prost
2. Downloads the protoc-gen-prost.wasm artifact
3. Updates version.go with the new version info

Building the WASM Binary

The WASM binary is built from aperturerobotics/protoc-gen-prost:

# Clone the repository
git clone https://github.com/aperturerobotics/protoc-gen-prost.git
cd protoc-gen-prost

# Build for WASI
./build-wasi.sh

# Output: dist/protoc-gen-prost.wasm

Build Requirements

• Rust toolchain with wasm32-wasip1 target
• binaryen (for wasm-opt)
• wabt (for wasm-strip)

Testing

go test -v ./...

License

MIT

The embedded protoc-gen-prost WASM is covered by the Apache 2.0 license.

Documentation

Overview

Package prost provides a Go wrapper for running protoc-gen-prost via WASI/wazero.

Index

• Constants
• Variables
• func CompileProtocGenProst(ctx context.Context, r wazero.Runtime) (wazero.CompiledModule, error)
• type ProtocGenProst
  • func NewProtocGenProst(ctx context.Context, r wazero.Runtime) (*ProtocGenProst, error)
  • func NewProtocGenProstWithModule(ctx context.Context, r wazero.Runtime, compiled wazero.CompiledModule) (*ProtocGenProst, error)
  • func NewProtocGenProstWithWASI(ctx context.Context, r wazero.Runtime) (*ProtocGenProst, error)
  • func NewProtocGenProstWithWASIAndModule(ctx context.Context, r wazero.Runtime, compiled wazero.CompiledModule) (*ProtocGenProst, error)
  • func (p *ProtocGenProst) Close(ctx context.Context) error
  • func (p *ProtocGenProst) Execute(ctx context.Context, input []byte) ([]byte, error)

Constants

const (
	// ExportProstExecute executes the prost plugin.
	// Signature: prost_execute(input_ptr: i32, input_len: i32) -> i32 (output_len)
	ExportProstExecute = "prost_execute"

	// ExportProstGetOutputPtr returns the pointer to the output buffer.
	// Signature: prost_get_output_ptr() -> i32 (ptr)
	ExportProstGetOutputPtr = "prost_get_output_ptr"

	// ExportProstGetOutputLen returns the length of the output buffer.
	// Signature: prost_get_output_len() -> i32 (len)
	ExportProstGetOutputLen = "prost_get_output_len"

	// ExportProstClearOutput clears the output buffer.
	// Signature: prost_clear_output() -> void
	ExportProstClearOutput = "prost_clear_output"
)

Prost plugin exports

const (
	// ExportProstMalloc allocates memory in WASM linear memory.
	// Signature: prost_malloc(size: i32) -> i32 (pointer)
	ExportProstMalloc = "prost_malloc"

	// ExportProstFree frees memory in WASM linear memory.
	// Signature: prost_free(ptr: i32, size: i32) -> void
	ExportProstFree = "prost_free"
)

Memory management exports

const (
	// Version is the protoc-gen-prost version
	Version = "v0.5.0-wasi"
	// DownloadURL is the URL where this WASM file was downloaded from
	DownloadURL = "https://github.com/aperturerobotics/protoc-gen-prost/releases/download/v0.5.0-wasi/protoc-gen-prost.wasm"
)

protoc-gen-prost WASI version information

const ProtocGenProstWASMFilename = "protoc-gen-prost.wasm"

ProtocGenProstWASMFilename is the filename for ProtocGenProstWASM.

Variables

var ProtocGenProstWASM []byte

ProtocGenProstWASM contains the binary contents of the protoc-gen-prost WASI build.

This is a WASM binary that exports functions for executing the Prost protobuf code generator. The module uses the standard WASI preview1 interface.

Functions

func CompileProtocGenProst

func CompileProtocGenProst(ctx context.Context, r wazero.Runtime) (wazero.CompiledModule, error)

CompileProtocGenProst compiles the embedded protoc-gen-prost WASM module. The compiled module can be reused across multiple ProtocGenProst instances.

Types

type ProtocGenProst

type ProtocGenProst struct {
	// contains filtered or unexported fields
}

ProtocGenProst wraps a protoc-gen-prost WASI module providing a high-level API for executing the Prost protobuf code generator.

func NewProtocGenProst

func NewProtocGenProst(ctx context.Context, r wazero.Runtime) (*ProtocGenProst, error)

NewProtocGenProst creates a new ProtocGenProst instance using the embedded WASM. This instantiates WASI on the runtime. For shared runtimes where WASI is already instantiated, use NewProtocGenProstWithWASI instead. Call Close() when done to release resources.

func NewProtocGenProstWithModule

func NewProtocGenProstWithModule(ctx context.Context, r wazero.Runtime, compiled wazero.CompiledModule) (*ProtocGenProst, error)

NewProtocGenProstWithModule creates a new ProtocGenProst instance using a pre-compiled module. This instantiates WASI on the runtime. For shared runtimes where WASI is already instantiated, use NewProtocGenProstWithWASIAndModule instead.

func NewProtocGenProstWithWASI

func NewProtocGenProstWithWASI(ctx context.Context, r wazero.Runtime) (*ProtocGenProst, error)

NewProtocGenProstWithWASI creates a new ProtocGenProst instance on a runtime that already has WASI instantiated. Use this when sharing a runtime with other WASM modules (e.g., protoc).

func NewProtocGenProstWithWASIAndModule

func NewProtocGenProstWithWASIAndModule(ctx context.Context, r wazero.Runtime, compiled wazero.CompiledModule) (*ProtocGenProst, error)

NewProtocGenProstWithWASIAndModule creates a new ProtocGenProst instance using a pre-compiled module on a runtime that already has WASI instantiated.

func (*ProtocGenProst) Close

func (p *ProtocGenProst) Close(ctx context.Context) error

Close releases resources associated with the ProtocGenProst instance.

func (*ProtocGenProst) Execute

func (p *ProtocGenProst) Execute(ctx context.Context, input []byte) ([]byte, error)

Execute runs the protoc-gen-prost plugin with the given CodeGeneratorRequest. The input should be a serialized google.protobuf.compiler.CodeGeneratorRequest. Returns a serialized google.protobuf.compiler.CodeGeneratorResponse.