README
¶
gpu
GPU access for go-wasmcontainers
WASI containers: wasi-nn inference, wasi:webgpu compute, and
rendering, on WebGPU in the browser (the playground) and on the CPU
natively. Pure Go, CGO_ENABLED=0.
Design and references: docs/DESIGN.md.
Status
| Part | State |
|---|---|
onnx: ONNX model loader and CPU operators |
done (v0.1.0) — see below |
nn: wasi-nn host module (wasi_ephemeral_nn), CPU backend |
done (v0.2.0) — see below |
wasinn: wasi-nn guest bindings for Go, examples/mnist |
done (v0.2.0) |
nn/webgpu: wasi-nn GPU backend (WebGPU WGSL kernels, browser) |
done (v0.3.0) — see below |
wit, webgpu, wasigpu: wasi:webgpu compute (host and Go guest), examples/compute |
done (v0.4.0) — see below |
rendering: wasi-gfx:surface + surface-webgpu, frames to the page, examples/triangle |
done (v0.5.0) — see below; the dashboard pane is the dashboard's part |
surface input: pointer and key streams (stream<…> and the Canonical ABI's stream and waitable-set built-ins) |
done — see below; forwarding events from the pane is the dashboard's part |
on-resize / on-frame streams, vertex buffers, depth-stencil |
not yet |
wasi:webgpu: wit, webgpu (host), wasigpu (guest)
Containers are WASI preview 1 core modules, and the engine has no
component-model runtime; wasi:webgpu@0.3.0-rc.2 is WIT. So:
witparses WIT and implements the Canonical ABI for a 32-bit memory (alignment, sizes, flattening, loads, stores, flat lifting and lowering; traps as errors), with the guest allocations of a result planned first and applied once the guest'scabi_reallocgave them (FuzzLoad).webgpu.NewModule(backend)is a host module serving every function of the interface (core names such as[method]gpu-device.create-buffer, signatures from the WIT), resources as per-process handles. Asynchronous functions (request-adapter,map-async, …) are lowered synchronously and run off the host call.webgpu.Browser{}maps them to WebGPU in the engine's worker; elsewhere there is no backend:get-gpuworks andrequest-adapteranswers none. A process without--gpugets no adapter. Quotas per process: 4096 handles, 256 MiB of buffers, 1 MiB per WGSL module, 256 pipelines (traps past them); a submission still running after 2 s has its device destroyed; fallback adapters are refused.wasigpuis the Go guest side (GOOS=wasip1), over the samewitcode; its imports are generated from the WIT byinternal/cmd/witgen(a test keeps them current).
examples/compute adds two 1M-float vectors and
multiplies two 256×256 matrices on the GPU, checking every element against
the CPU. e2e/engine_test.go runs it — and examples/mnist on the CPU and
on the GPU — as real containers in the engine's browser executor
(Chromium, a Web Worker per container): on an Apple Metal adapter every
element matches; matmul 15 ms on the GPU vs 42 ms on the guest's CPU, the
vector add 69 ms vs 3.5 ms (dominated by transfers; one run, not a
benchmark). Without --gpu the guest is told there is no adapter, and a
fifth 64 MiB buffer traps on the memory quota.
Rendering: surfaces
webgpu.HostModules(backend) serves four modules sharing one handle table
per process: wasi:webgpu/webgpu@0.3.0-rc.2,
wasi-gfx:surface/surface@0.2.0, wasi-gfx:surface/surface-webgpu@0.2.0
and $root (the waitable-set built-ins; a guest using wasigpu imports
all four). In the browser each surface
is an OffscreenCanvas in the engine's worker; present turns it into an
ImageBitmap handed to Browser.OnSurface with a sequence number (the
open event names the container), for the
page to draw (in the playground, a dashboard pane). The page acknowledges
each frame (webgpu.AckFrame); at most 2 frames are unacknowledged
(past that, present waits for an acknowledgement at most 1 s, so a page
that stops acknowledging slows the guest to a frame per second without
stopping it), frames are paced to 60
per second, surfaces are at most 1920×1080. Render pipelines take their
vertices from the shader (no vertex buffers yet) and have no depth-stencil.
Input. A surface's on-pointer-down, on-pointer-up,
on-pointer-move, on-key-down and on-key-up each return a
stream<pointer-event> or stream<key-event> (UI Events key codes, the
text the key produces, modifiers). A core module reads it with the
Canonical ABI built-ins as wit-component names them:
[stream-read-0]… (blocks until an event), [async-lower][stream-read-0]…
(returns BLOCKED, completes through a waitable set), cancel-read,
drop-readable, and $root's [waitable-set-new|poll|wait|drop] and
[waitable-join]. The page reports events on a pane with
webgpu.SurfaceInput(surface, InputEvent); each stream keeps at most 256
undelivered events (the oldest dropped), a surface serves at most 64
streams, coordinates must be finite and key text at most 64 bytes of
UTF-8. Guest side: (*Surface).Pointer(kind) and (*Surface).Keys(kind),
whose Poll returns what arrived without blocking (once per frame).
Guest side: (*wasigpu.Device).Surface(w, h),
(*Surface).RenderPipeline(wgsl, vs, fs) and (*Surface).Frame(pipeline, uniforms, clear, vertices). examples/triangle draws
a spinning triangle; e2e/engine_test.go runs it as a container on an Apple
Metal adapter and checks, at the page, 60 frames in order, the clear colour
in a corner, the triangle in the middle, frames that change, and a rate
within the 60 per second pacing (one run: 51 frames per second); with
input, the page clicks and types on the pane and the guest prints what it
read (pointer-down 10,20, key-down key-a "A" shift=true).
The same checks, pixels included, pass on the Linux SwiftShader lane.
Headless Linux Chromium can back a WebGPU canvas only with Skia on
Vulkan: launch it with --enable-features=Vulkan --use-vulkan=swiftshader --use-angle=swiftshader (or a real Vulkan driver), otherwise the WebGPU
instance is torn down at the first canvas configure and frames are blank
(docs/DESIGN.md, "Surfaces"). go run ./internal/cmd/gpuprobe --canvas
follows one canvas frame from the GPU to the page to check a browser.
wasi-nn: nn (host) and wasinn (guest)
nn.NewModule() is a host module for the engine (v0.13.0+,
wazero.Options.HostModules / browser.Options.HostModules) serving the
preview 1 wasi-nn interface, wasi_ephemeral_nn (load, load_by_name,
init_execution_context, set_input, get_output, compute), as
Wasmtime and the wasi-nn crate's bindings lower it. A graph goes to the
backend of its encoding (ONNX: the pure-Go CPU backend). The execution
target is checked: gpu needs the container's gpu capability
(wctr run --gpu) and a GPU backend, and otherwise fails with
unsupported_operation — never runs on the CPU in its place. Model bytes,
graphs and contexts per process, tensor rank and data are bounded;
FuzzHostCalls drives arbitrary call sequences over arbitrary memory.
wasinn gives Go guests (GOOS=wasip1) Load, NewContext,
SetInputF32, Compute, OutputF32; elsewhere it builds and returns
ErrUnavailable.
examples/mnist is a container that reads a handwritten
digit: natively it classifies MNIST test pictures 0–3 (labels 7, 2, 1, 0)
correctly, p = 1.000 for the embedded one.
wasi-nn on the GPU: nn/webgpu
In the browser, webgpu.NewModule() adds a "gpu" backend to the CPU one:
each ONNX node becomes one WGSL compute dispatch on a WebGPU device of the
graph's own (in the engine's worker), shapes and constants coming from
onnx.Session.Plan (a CPU dry run, so both executors agree on every shape
by construction). Its calls wait on WebGPU promises, so nn runs them off
the host call (AsyncBackend). A fallback (software) adapter is refused,
and each graph's GPU memory is bounded (256 MiB).
Checked twice: natively, TestKernelsEmulated runs each kernel's index
arithmetic in Go on all 122 vendored ONNX node tests (no GPU needed, every
platform); in a browser, e2e/ runs the real WGSL on the same tests and
MNIST-12 through Playwright. Measured on an Apple Metal adapter (Chromium
143, new headless): 123/123 pass, MNIST-12 within 3.1e-5 of the
reference. The CI lanes: macOS (must get a GPU adapter, or fail) and Linux
on SwiftShader (labelled fallback: proves the WGSL, not a GPU).
internal/cmd/gpuprobe reports what adapter a browser offers.
onnx
Loads an ONNX model (onnx.proto, decoded by a bounded protobuf reader)
and runs it on the CPU. Operators: Add, AveragePool, Concat, Constant,
Conv, Div, Dropout (inference), Flatten, Gemm, Identity, MatMul, MaxPool,
Mul, Relu, Reshape, Softmax, Sub, Transpose — default domain, operator
sets 7 and later, float tensors (int64 for shapes).
- Refuses, precisely. A model using anything else fails at load with
every unsupported operator and its node count, e.g.
onnx: unsupported operators: LSTM (2 nodes), com.example:Foo (1 node); supported: Add, …. Unsupported variants (MaxPool indices, Dropout training mode, non-float graph values) are refused at load too, not mid-inference. - Checked op by op against the ONNX node tests (onnx v1.22.0,
onnx/backend/test/data/node): the 122 tests whose models load here are vendored byinternal/cmd/onnxvendor(a derived list, not a hand-written one) and pass with the ONNX backend's tolerances,|out − ref| ≤ 1e-7 + 1e-3·|ref|. MNIST-12 from the ONNX model zoo passes on its own test data. - Untrusted input. The model size, node count, rank, elements per
tensor and in total, string lengths, and the multiply-accumulates of one
inference are bounded (
Limits);FuzzRun(decode, load, run) andFuzzDecodeTensorrun in CI.
Performance has not been measured yet (the machine's load was above 40); convolutions are direct loops over precomputed taps, not im2col + GEMM.
License
Apache-2.0 (LICENSE). Test data: ONNX node tests, Apache-2.0 (onnx/testdata/node/LICENSE); MNIST-12, MIT (onnx/testdata/models).
Directories
¶
| Path | Synopsis |
|---|---|
|
Package cabi is the guest side of the Canonical ABI's memory for Go programs built with GOOS=wasip1: the cabi_realloc export a host calls to return lists and strings, guest-side allocations for arguments, and the guest's own linear memory as a wit.Memory.
|
Package cabi is the guest side of the Canonical ABI's memory for Go programs built with GOOS=wasip1: the cabi_realloc export a host calls to return lists and strings, guest-side allocations for arguments, and the guest's own linear memory as a wit.Memory. |
|
examples
|
|
|
compute
command
Command compute is a WASI container that computes on the GPU through wasi:webgpu and checks every result against the CPU: a vector add of 1M floats and a 256x256 matrix product.
|
Command compute is a WASI container that computes on the GPU through wasi:webgpu and checks every result against the CPU: a vector add of 1M floats and a 256x256 matrix product. |
|
mnist
command
Command mnist is a WASI container that reads a handwritten digit: it embeds MNIST-12 (ONNX model zoo, MIT) and a 28x28 picture, runs the model through wasi-nn, and prints the digit with its probability.
|
Command mnist is a WASI container that reads a handwritten digit: it embeds MNIST-12 (ONNX model zoo, MIT) and a 28x28 picture, runs the model through wasi-nn, and prints the digit with its probability. |
|
triangle
command
Command triangle is a WASI container that draws a spinning triangle on a surface through wasi:webgpu and wasi-gfx:surface: in the playground the surface is a pane of the dashboard.
|
Command triangle is a WASI container that draws a spinning triangle on a surface through wasi:webgpu and wasi-gfx:surface: in the playground the surface is a pane of the dashboard. |
|
internal
|
|
|
cmd/enginetest
command
Command enginetest is a minimal engine worker, as the playground's: the engine's browser executor (one Web Worker per container, the SAB bridge) with this module's host modules, wasi:webgpu and wasi-nn (CPU and WebGPU backends).
|
Command enginetest is a minimal engine worker, as the playground's: the engine's browser executor (one Web Worker per container, the SAB bridge) with this module's host modules, wasi:webgpu and wasi-nn (CPU and WebGPU backends). |
|
cmd/gpuprobe
command
Command gpuprobe asks a real browser, through Playwright, what WebGPU it offers a page: whether navigator.gpu exists (in a window and in a worker), and the adapter's vendor, architecture and whether it is a fallback (software) adapter.
|
Command gpuprobe asks a real browser, through Playwright, what WebGPU it offers a page: whether navigator.gpu exists (in a window and in a worker), and the adapter's vendor, architecture and whether it is a fallback (software) adapter. |
|
cmd/mnistpng
command
Command mnistpng extracts one picture of the MNIST test set (IDX files, gzipped, as published by Y. LeCun, C. Cortes and C. Burges; CC BY-SA 3.0) to a 28x28 grayscale PNG and prints its label, so the mnist example classifies a real handwritten digit whose answer is known.
|
Command mnistpng extracts one picture of the MNIST test set (IDX files, gzipped, as published by Y. LeCun, C. Cortes and C. Burges; CC BY-SA 3.0) to a 28x28 grayscale PNG and prints its label, so the mnist example classifies a real handwritten digit whose answer is known. |
|
cmd/onnxvendor
command
Command onnxvendor copies the ONNX node tests this repository's onnx package can run into onnx/testdata/node: from a checkout of github.com/onnx/onnx (v1.22.0 is the last tag that ships onnx/backend/test/data/node), every test whose model decodes, uses only supported operators and passes onnx.NewSession.
|
Command onnxvendor copies the ONNX node tests this repository's onnx package can run into onnx/testdata/node: from a checkout of github.com/onnx/onnx (v1.22.0 is the last tag that ships onnx/backend/test/data/node), every test whose model decodes, uses only supported operators and passes onnx.NewSession. |
|
cmd/webtest
command
Command webtest runs in a browser Web Worker (as the playground's engine worker does) and checks the WebGPU wasi-nn backend against the ONNX node tests and MNIST-12: for each case it fetches the model and its test data from the page's origin, runs it through webgpu.Backend, compares with the reference (rtol 1e-3, atol 1e-7, the ONNX backend's tolerances) and posts a JSON report: the adapter, and every case's result.
|
Command webtest runs in a browser Web Worker (as the playground's engine worker does) and checks the WebGPU wasi-nn backend against the ONNX node tests and MNIST-12: for each case it fetches the model and its test data from the page's origin, runs it through webgpu.Backend, compares with the reference (rtol 1e-3, atol 1e-7, the ONNX backend's tolerances) and posts a JSON report: the adapter, and every case's result. |
|
cmd/witgen
command
Command witgen writes the core imports of a WIT interface for Go guests (GOOS=wasip1): one //go:wasmimport per function and resource drop, with its Canonical ABI core signature (package wit), and a table from import name to a call taking and returning core values as uint64.
|
Command witgen writes the core imports of a WIT interface for Go guests (GOOS=wasip1): one //go:wasmimport per function and resource drop, with its Canonical ABI core signature (package wit), and a table from import name to a call taking and returning core values as uint64. |
|
jsgpu
Package jsgpu is a thin Go layer over the browser's WebGPU API (https://www.w3.org/TR/webgpu/) through syscall/js, for code running in a Web Worker (the playground's engine worker): a device, storage buffers, compute pipelines from WGSL, dispatches and readback.
|
Package jsgpu is a thin Go layer over the browser's WebGPU API (https://www.w3.org/TR/webgpu/) through syscall/js, for code running in a Web Worker (the playground's engine worker): a device, storage buffers, compute pipelines from WGSL, dispatches and readback. |
|
reentry
Package reentry measures risk R1 of docs/DESIGN.md: can a host, while a Go wasip1 guest is inside a //go:wasmimport call, call the guest's //go:wasmexport cabi_realloc and write the result into the block it returns? The canonical ABI needs exactly that to return a list or a string from a host function.
|
Package reentry measures risk R1 of docs/DESIGN.md: can a host, while a Go wasip1 guest is inside a //go:wasmimport call, call the guest's //go:wasmexport cabi_realloc and write the result into the block it returns? The canonical ABI needs exactly that to return a list or a string from a host function. |
|
Package nn is the host side of wasi-nn for go-wasmcontainers: the preview 1 (witx) interface, import module "wasi_ephemeral_nn" (github.com/WebAssembly/wasi-nn, wasi-nn.witx), served by the engine's WASI host as a host module, natively and in the browser.
|
Package nn is the host side of wasi-nn for go-wasmcontainers: the preview 1 (witx) interface, import module "wasi_ephemeral_nn" (github.com/WebAssembly/wasi-nn, wasi-nn.witx), served by the engine's WASI host as a host module, natively and in the browser. |
|
webgpu
Package webgpu is a wasi-nn backend that runs ONNX graphs on the browser's GPU through WebGPU (WGSL compute kernels), for the "gpu" execution target.
|
Package webgpu is a wasi-nn backend that runs ONNX graphs on the browser's GPU through WebGPU (WGSL compute kernels), for the "gpu" execution target. |
|
Package onnx loads ONNX models (onnx/onnx.proto) and runs a bounded set of operators on the CPU, in pure Go.
|
Package onnx loads ONNX models (onnx/onnx.proto) and runs a bounded set of operators on the CPU, in pure Go. |
|
Package wasigpu is the guest side of wasi:webgpu (wasi:webgpu@0.3.0-rc.2) for Go programs built with GOOS=wasip1: the compute subset (adapter, device, buffers, WGSL modules, compute pipelines, bind groups, command encoders, queue, mapping).
|
Package wasigpu is the guest side of wasi:webgpu (wasi:webgpu@0.3.0-rc.2) for Go programs built with GOOS=wasip1: the compute subset (adapter, device, buffers, WGSL modules, compute pipelines, bind groups, command encoders, queue, mapping). |
|
Package wasinn is the guest side of wasi-nn (preview 1, import module "wasi_ephemeral_nn") for Go programs built with GOOS=wasip1: load a model, set inputs, compute, read outputs.
|
Package wasinn is the guest side of wasi-nn (preview 1, import module "wasi_ephemeral_nn") for Go programs built with GOOS=wasip1: load a model, set inputs, compute, read outputs. |
|
Package webgpu is the host side of wasi:webgpu (wasi-gfx, wasi:webgpu@0.3.0-rc.2) for core modules: every function of the WIT interface is a typed import of one host module, lowered and lifted with the Canonical ABI (package wit) by hand, without a component-model runtime.
|
Package webgpu is the host side of wasi:webgpu (wasi-gfx, wasi:webgpu@0.3.0-rc.2) for core modules: every function of the WIT interface is a typed import of one host module, lowered and lifted with the Canonical ABI (package wit) by hand, without a component-model runtime. |
|
Package wit reads WIT (the WebAssembly Interface Types language, component-model design/mvp/WIT.md) for the subset the wasi-gfx packages use, and implements the Canonical ABI (design/mvp/CanonicalABI.md) over a core module's linear memory: alignment, sizes, flattening, loads, stores, flat lifting and lowering.
|
Package wit reads WIT (the WebAssembly Interface Types language, component-model design/mvp/WIT.md) for the subset the wasi-gfx packages use, and implements the Canonical ABI (design/mvp/CanonicalABI.md) over a core module's linear memory: alignment, sizes, flattening, loads, stores, flat lifting and lowering. |