wazeropool

README

Wazero Pool

A wazero module instance pool.

When working with webassembly modules, it is frequently more advantageous to use a homogeneous set of module instances rather than a single instance. This package provides a common approach and interface to manage pools of wazero module instances.

package main

import (
	"context"
	_ "embed"

	"github.com/tetratelabs/wazero"
	"github.com/tetratelabs/wazero/api"
	"github.com/tetratelabs/wazero/imports/wasi_snapshot_preview1"

	"github.com/pantopic/wazero-pool"
)

//go:embed test\.wasm
var src []byte

func main() {
	ctx := context.Background()
	runtime := wazero.NewRuntimeWithConfig(ctx, wazero.NewRuntimeConfig())
	wasi_snapshot_preview1.MustInstantiate(ctx, runtime)
	cfg := wazero.NewModuleConfig()

	pool, err := wazeropool.New(ctx, runtime, src, cfg)
	if err != nil {
		panic(err)
	}
	pool.With(func(mod api.Module) {
		_, err := mod.ExportedFunction(`test1`).Call(ctx)
		if err != nil {
			panic(err)
		}
		_, err := mod.ExportedFunction(`test2`).Call(ctx)
		if err != nil {
			panic(err)
		}
	})

	// ...
}

Options

The constructor has one functional option.

Limit

Supports explicit limit to prevent unbounded memory growth.

pool, err := wazeropool.New(ctx, runtime, src, cfg, 
	wazeropool.WithLimit(2))
if err != nil {
	panic(err)
}
mod1 := pool.Get()
mod2 := pool.Get()
mod3 := pool.Get() // Blocks until a module is returned to the pool with `pool.Put`

Performance

Setting limit explicitly is recommended unless you are already strictly limiting the concurrency of the calling code in other ways.

> make bench
BenchmarkModule/add/raw-16                      31112986                38.95 ns/op
BenchmarkModule/add/wrapped-16                  26988614                45.59 ns/op
BenchmarkModule/add/pooled-16                    7362813               160.9 ns/op
BenchmarkModule/add/parallel-2-16                2847218               445.3 ns/op
BenchmarkModule/add/parallel-4-16                2397896               508.7 ns/op
BenchmarkModule/add/parallel-8-16                1846426               656.6 ns/op
BenchmarkModule/add/parallel-16-16               1676278               745.8 ns/op
BenchmarkModule/add/parallel-0-16                3551059               331.8 ns/op
BenchmarkModule/microsleep/raw-16                  30891             40574 ns/op
BenchmarkModule/microsleep/wrapped-16              25706             48160 ns/op
BenchmarkModule/microsleep/pooled-16               12202             94872 ns/op
BenchmarkModule/microsleep/parallel-2-16           83138             14738 ns/op
BenchmarkModule/microsleep/parallel-4-16          109393             11234 ns/op
BenchmarkModule/microsleep/parallel-8-16          731980              1714 ns/op
BenchmarkModule/microsleep/parallel-16-16        1258392               969.3 ns/op
BenchmarkModule/microsleep/parallel-0-16         2359347               517.5 ns/op

These benchmark results illustrate the throughput advantages pooling can have for high latency operations and the throughput disadvantages it can introduce for low latency operations.

Generally, this instance pool will be more useful for modules that make callbacks to host modules and less useful for pure functional modules.

Roadmap

This project is in Alpha. Breaking API changes should be expected until Beta.

• v0.0.x - Alpha
  • Stabilize API
• v0.x.x - Beta
  • Finalize API
  • Test in production
• v1.x.x - General Availability
  • Proven long term stability in production

Documentation

Index

• func New(ctx context.Context, r wazero.Runtime, src []byte, opts ...Option) (m *instance, err error)
• type Factory
• type Instance
• type Option
  • func WithLimit(n int) Option
  • func WithModuleConfig(cfg wazero.ModuleConfig) Option

Functions

func New

func New(ctx context.Context, r wazero.Runtime, src []byte, opts ...Option) (m *instance, err error)

New returns a new module instance pool.

Types

type Factory

type Factory interface {
	// New returns a new module instance pool.
	New(ctx context.Context, src []byte, opts ...Option) (m *instance, err error)
}

type Instance

type Instance interface {
	// Compiled returns the compiled module for this instance.
	Compiled() wazero.CompiledModule

	// Get returns a module instance from the pool.
	// If limit is non-zero, [Get] will block until an instance becomes available.
	// If limit is non-zero and the module is not [Put] back, a deadlock may occur.
	Get() api.Module

	// Put puts a module instance back into the pool.
	Put(mod api.Module)

	// Run automatically [Get]s a module instance from the pool and [Put]s it back after the function returns.
	Run(fn func(mod api.Module))
}

Instance represents a module instance pool

type Option

type Option func(*instance)

Option represents a constructor option.

func WithLimit

func WithLimit(n int) Option

WithLimit sets the maximum pool size. Set to 0 for unlimited pool (default)

func WithModuleConfig

func WithModuleConfig(cfg wazero.ModuleConfig) Option

WithModuleConfig sets the module config.