README
¶
gpu
GPU access for go-wasmcontainers
WASI containers: wasi-nn inference, wasi:webgpu compute, and
rendering, on WebGPU in the browser (the playground) and on the CPU
natively. Pure Go, CGO_ENABLED=0.
Design and references: docs/DESIGN.md.
Status
| Part | State |
|---|---|
onnx: ONNX model loader and CPU operators |
done (v0.1.0) — see below |
nn: wasi-nn host module (wasi_ephemeral_nn), CPU backend |
done (v0.2.0) — see below |
wasinn: wasi-nn guest bindings for Go, examples/mnist |
done (v0.2.0) |
nn/webgpu: wasi-nn GPU backend (WebGPU WGSL kernels, browser) |
done — see below |
| wasi:webgpu compute (browser) | next |
| rendering to a dashboard pane | planned |
wasi-nn: nn (host) and wasinn (guest)
nn.NewModule() is a host module for the engine (v0.13.0+,
wazero.Options.HostModules / browser.Options.HostModules) serving the
preview 1 wasi-nn interface, wasi_ephemeral_nn (load, load_by_name,
init_execution_context, set_input, get_output, compute), as
Wasmtime and the wasi-nn crate's bindings lower it. A graph goes to the
backend of its encoding (ONNX: the pure-Go CPU backend). The execution
target is checked: gpu needs the container's gpu capability
(wctr run --gpu) and a GPU backend, and otherwise fails with
unsupported_operation — never runs on the CPU in its place. Model bytes,
graphs and contexts per process, tensor rank and data are bounded;
FuzzHostCalls drives arbitrary call sequences over arbitrary memory.
wasinn gives Go guests (GOOS=wasip1) Load, NewContext,
SetInputF32, Compute, OutputF32; elsewhere it builds and returns
ErrUnavailable.
examples/mnist is a container that reads a handwritten
digit: natively it classifies MNIST test pictures 0–3 (labels 7, 2, 1, 0)
correctly, p = 1.000 for the embedded one.
wasi-nn on the GPU: nn/webgpu
In the browser, webgpu.NewModule() adds a "gpu" backend to the CPU one:
each ONNX node becomes one WGSL compute dispatch on a WebGPU device of the
graph's own (in the engine's worker), shapes and constants coming from
onnx.Session.Plan (a CPU dry run, so both executors agree on every shape
by construction). Its calls wait on WebGPU promises, so nn runs them off
the host call (AsyncBackend). A fallback (software) adapter is refused,
and each graph's GPU memory is bounded (256 MiB).
Checked twice: natively, TestKernelsEmulated runs each kernel's index
arithmetic in Go on all 122 vendored ONNX node tests (no GPU needed, every
platform); in a browser, e2e/ runs the real WGSL on the same tests and
MNIST-12 through Playwright. Measured on an Apple Metal adapter (Chromium
143, new headless): 123/123 pass, MNIST-12 within 3.1e-5 of the
reference. The CI lanes: macOS (must get a GPU adapter, or fail) and Linux
on SwiftShader (labelled fallback: proves the WGSL, not a GPU).
internal/cmd/gpuprobe reports what adapter a browser offers.
onnx
Loads an ONNX model (onnx.proto, decoded by a bounded protobuf reader)
and runs it on the CPU. Operators: Add, AveragePool, Concat, Constant,
Conv, Div, Dropout (inference), Flatten, Gemm, Identity, MatMul, MaxPool,
Mul, Relu, Reshape, Softmax, Sub, Transpose — default domain, operator
sets 7 and later, float tensors (int64 for shapes).
- Refuses, precisely. A model using anything else fails at load with
every unsupported operator and its node count, e.g.
onnx: unsupported operators: LSTM (2 nodes), com.example:Foo (1 node); supported: Add, …. Unsupported variants (MaxPool indices, Dropout training mode, non-float graph values) are refused at load too, not mid-inference. - Checked op by op against the ONNX node tests (onnx v1.22.0,
onnx/backend/test/data/node): the 122 tests whose models load here are vendored byinternal/cmd/onnxvendor(a derived list, not a hand-written one) and pass with the ONNX backend's tolerances,|out − ref| ≤ 1e-7 + 1e-3·|ref|. MNIST-12 from the ONNX model zoo passes on its own test data. - Untrusted input. The model size, node count, rank, elements per
tensor and in total, string lengths, and the multiply-accumulates of one
inference are bounded (
Limits);FuzzRun(decode, load, run) andFuzzDecodeTensorrun in CI.
Performance has not been measured yet (the machine's load was above 40); convolutions are direct loops over precomputed taps, not im2col + GEMM.
License
Apache-2.0 (LICENSE). Test data: ONNX node tests, Apache-2.0 (onnx/testdata/node/LICENSE); MNIST-12, MIT (onnx/testdata/models).
Directories
¶
| Path | Synopsis |
|---|---|
|
examples
|
|
|
mnist
command
Command mnist is a WASI container that reads a handwritten digit: it embeds MNIST-12 (ONNX model zoo, MIT) and a 28x28 picture, runs the model through wasi-nn, and prints the digit with its probability.
|
Command mnist is a WASI container that reads a handwritten digit: it embeds MNIST-12 (ONNX model zoo, MIT) and a 28x28 picture, runs the model through wasi-nn, and prints the digit with its probability. |
|
internal
|
|
|
cmd/gpuprobe
command
Command gpuprobe asks a real browser, through Playwright, what WebGPU it offers a page: whether navigator.gpu exists (in a window and in a worker), and the adapter's vendor, architecture and whether it is a fallback (software) adapter.
|
Command gpuprobe asks a real browser, through Playwright, what WebGPU it offers a page: whether navigator.gpu exists (in a window and in a worker), and the adapter's vendor, architecture and whether it is a fallback (software) adapter. |
|
cmd/mnistpng
command
Command mnistpng extracts one picture of the MNIST test set (IDX files, gzipped, as published by Y. LeCun, C. Cortes and C. Burges; CC BY-SA 3.0) to a 28x28 grayscale PNG and prints its label, so the mnist example classifies a real handwritten digit whose answer is known.
|
Command mnistpng extracts one picture of the MNIST test set (IDX files, gzipped, as published by Y. LeCun, C. Cortes and C. Burges; CC BY-SA 3.0) to a 28x28 grayscale PNG and prints its label, so the mnist example classifies a real handwritten digit whose answer is known. |
|
cmd/onnxvendor
command
Command onnxvendor copies the ONNX node tests this repository's onnx package can run into onnx/testdata/node: from a checkout of github.com/onnx/onnx (v1.22.0 is the last tag that ships onnx/backend/test/data/node), every test whose model decodes, uses only supported operators and passes onnx.NewSession.
|
Command onnxvendor copies the ONNX node tests this repository's onnx package can run into onnx/testdata/node: from a checkout of github.com/onnx/onnx (v1.22.0 is the last tag that ships onnx/backend/test/data/node), every test whose model decodes, uses only supported operators and passes onnx.NewSession. |
|
cmd/webtest
command
Command webtest runs in a browser Web Worker (as the playground's engine worker does) and checks the WebGPU wasi-nn backend against the ONNX node tests and MNIST-12: for each case it fetches the model and its test data from the page's origin, runs it through webgpu.Backend, compares with the reference (rtol 1e-3, atol 1e-7, the ONNX backend's tolerances) and posts a JSON report: the adapter, and every case's result.
|
Command webtest runs in a browser Web Worker (as the playground's engine worker does) and checks the WebGPU wasi-nn backend against the ONNX node tests and MNIST-12: for each case it fetches the model and its test data from the page's origin, runs it through webgpu.Backend, compares with the reference (rtol 1e-3, atol 1e-7, the ONNX backend's tolerances) and posts a JSON report: the adapter, and every case's result. |
|
jsgpu
Package jsgpu is a thin Go layer over the browser's WebGPU API (https://www.w3.org/TR/webgpu/) through syscall/js, for code running in a Web Worker (the playground's engine worker): a device, storage buffers, compute pipelines from WGSL, dispatches and readback.
|
Package jsgpu is a thin Go layer over the browser's WebGPU API (https://www.w3.org/TR/webgpu/) through syscall/js, for code running in a Web Worker (the playground's engine worker): a device, storage buffers, compute pipelines from WGSL, dispatches and readback. |
|
reentry
Package reentry measures risk R1 of docs/DESIGN.md: can a host, while a Go wasip1 guest is inside a //go:wasmimport call, call the guest's //go:wasmexport cabi_realloc and write the result into the block it returns? The canonical ABI needs exactly that to return a list or a string from a host function.
|
Package reentry measures risk R1 of docs/DESIGN.md: can a host, while a Go wasip1 guest is inside a //go:wasmimport call, call the guest's //go:wasmexport cabi_realloc and write the result into the block it returns? The canonical ABI needs exactly that to return a list or a string from a host function. |
|
Package nn is the host side of wasi-nn for go-wasmcontainers: the preview 1 (witx) interface, import module "wasi_ephemeral_nn" (github.com/WebAssembly/wasi-nn, wasi-nn.witx), served by the engine's WASI host as a host module, natively and in the browser.
|
Package nn is the host side of wasi-nn for go-wasmcontainers: the preview 1 (witx) interface, import module "wasi_ephemeral_nn" (github.com/WebAssembly/wasi-nn, wasi-nn.witx), served by the engine's WASI host as a host module, natively and in the browser. |
|
webgpu
Package webgpu is a wasi-nn backend that runs ONNX graphs on the browser's GPU through WebGPU (WGSL compute kernels), for the "gpu" execution target.
|
Package webgpu is a wasi-nn backend that runs ONNX graphs on the browser's GPU through WebGPU (WGSL compute kernels), for the "gpu" execution target. |
|
Package onnx loads ONNX models (onnx/onnx.proto) and runs a bounded set of operators on the CPU, in pure Go.
|
Package onnx loads ONNX models (onnx/onnx.proto) and runs a bounded set of operators on the CPU, in pure Go. |
|
Package wasinn is the guest side of wasi-nn (preview 1, import module "wasi_ephemeral_nn") for Go programs built with GOOS=wasip1: load a model, set inputs, compute, read outputs.
|
Package wasinn is the guest side of wasi-nn (preview 1, import module "wasi_ephemeral_nn") for Go programs built with GOOS=wasip1: load a model, set inputs, compute, read outputs. |