ociapps

README

ociapps

Stream WebAssembly apps from any OCI registry, with multi-registry cluster fallback.

A pure-Go (CGO=0) library + two CLI tools that pack a wasm app into an OCI image, push it to any OCI Distribution v2 registry, and load it back into a browser through a cluster of mirrors.

---

ociapps is the foundation the wasmdesk ecosystem uses to stream any client wasm app from any OCI Distribution v2 registry. An "app" is an OCI image whose manifest annotates each layer with the VFS-relative file name it represents:

ociapps.path/app.wasm      = sha256:<digest>
ociapps.path/worker.js     = sha256:<digest>
ociapps.path/wasm_exec.js  = sha256:<digest>

The loader walks the annotations, pulls every referenced blob, and returns an App struct whose Files map is ready to hand to the browser side of the wasm host. The resolver tries each registry in declaration order and returns the first one that responds, so a single logical app reference can be served by an entire cluster of mirrors transparently.

This package is the generalisation of go-quake1/engine/ociassets: the Quake loader streamed pak files keyed by quake.path/...; this one streams arbitrary apps keyed by ociapps.path/.... The fetch machinery, single-flight cache, and Distribution v2 wire shape are the same.

Library quick start

import "github.com/wasmdesk/ociapps"

r := &ociapps.Resolver{
    Registries: []ociapps.Registry{
        {URL: "https://ghcr.io"},                // primary
        {URL: "https://registry.example.com"},   // fallback
        {URL: "http://localhost:5000"},          // dev mirror
    },
}

app, err := r.LoadApp(ctx, "wasmdesk/terminal:latest")
if err != nil {
    log.Fatal(err)
}

wasmBytes := app.Files["app.wasm"]
workerSrc := app.Files["worker.js"]
// ... instantiate in the browser, e.g. through syscall/js

Manual cluster control

// Fetch just the manifest -- learn which mirror was healthy:
reg, manifest, err := r.FetchManifest(ctx, "wasmdesk/terminal", "latest")

// Fetch a single blob (content-addressed, cached across mirrors):
reg, body, err := r.FetchBlob(ctx, "wasmdesk/terminal", "sha256:abc...")

Persistent cache (browser builds)

On js/wasm an IndexedDBCache lets app loads survive a page reload without re-fetching every blob:

//go:build js && wasm

r := &ociapps.Resolver{
    Registries: []ociapps.Registry{{URL: "https://ghcr.io"}},
    Cache:      ociapps.NewIndexedDBCache("", ""), // defaults
}

On host builds the cache defaults to an in-memory map.

CLI quick start

Two binaries ship under cmd/:

ociapps-pack -- write an OCI image-layout directory

go install github.com/wasmdesk/ociapps/cmd/ociapps-pack@latest

# clients/terminal/manifest.toml
cat <<EOF > clients/terminal/manifest.toml
mediatype = "application/vnd.oci.image.manifest.v1+json"
[files]
"app.wasm"     = "terminal.wasm"
"worker.js"    = "worker.js"
"wasm_exec.js" = "wasm_exec.js"
EOF

ociapps-pack \
  -in clients/terminal \
  -manifest clients/terminal/manifest.toml \
  -out _oci \
  -ref terminal:latest

Both .toml (minimal subset) and .json manifests are accepted.

ociapps-push -- pure-Go OCI v2 pusher (no oras, no skopeo)

go install github.com/wasmdesk/ociapps/cmd/ociapps-push@latest

ociapps-push \
  -in _oci \
  -ref localhost:5000/wasmdesk/terminal:latest

HTTPS endpoints work transparently when -scheme=https is set. Token auth for push kicks in automatically when a credential is present: pass -username and set $GHCR_TOKEN (or $GITHUB_TOKEN), and the pusher wraps its HTTP client with a TokenAuthDoer, so it can publish straight to ghcr.io or any other token-gated registry. A local unauthenticated registry (localhost:5000) needs neither.

ociapps-static -- write a static Distribution v2 /v2 tree

go install github.com/wasmdesk/ociapps/cmd/ociapps-static@latest

ociapps-static -in _oci -repo hello -out site
# writes site/v2/hello/manifests/<tag> (+ /<digest>) and
#        site/v2/hello/blobs/sha256:<hex>

WriteStaticTree materialises an OCI image-layout directory as a static file tree matching the Distribution v2 GET API, so a plain static server — GitHub Pages, an S3 bucket, python -m http.server — can serve it with no registry process and no auth. It is the on-disk twin of ServeLayout: the browser-side OCIAppsLoader requests exactly these paths, so a tree written beside the page loads same-origin — no CORS, no token, no proxy. This is how wasmdesk ships the desktop and its apps from one Pages origin while ghcr stays the canonical upstream (public registries refuse cross-origin browser reads).

Layout

.
├── doc.go             package overview
├── manifest.go        Manifest + Descriptor + digest helpers
├── resolver.go        Registry + Resolver + multi-registry fallback
├── app.go             App + Resolver.LoadApp
├── layout.go          PackLayout + ServeLayout (OCI image-layout)
├── static.go          WriteStaticTree (static /v2 mirror for Pages/S3)
├── push.go            Pusher + Distribution v2 PUT machinery + TokenAuthDoer
├── cache_wasm.go      js/wasm IndexedDB cache (build tag)
└── cmd/
    ├── ociapps-pack/    packer CLI
    ├── ociapps-push/    pure-Go v2 pusher (token auth for ghcr)
    └── ociapps-static/  static /v2 tree writer

CI runs go vet, a 100% coverage gate, and a 6-arch cross-compile.

Conventions

• CGO=0 everywhere, on every supported arch.
• BSD-3-Clause licensed; all source files carry the SPDX header.
• Content-addressed cache: a blob fetched from any mirror is cached under its digest, so the next call for the same digest -- regardless of which mirror served it -- hits memory.
• Annotations-first: only layers that appear under ociapps.path/<vfs-name> are surfaced in App.Files. Other layers are ignored, which lets the same image carry auxiliary blobs (debug symbols, source maps) without polluting the runtime.

Sibling

The canonical consumer of ociapps is wasmbox, the wasmdesk window-manager + compositor. wasmbox uses ociapps to load each client (terminal, dock, files, ...) from any configured OCI registry instead of bundling them into the compositor binary.

Documentation

Overview

Package ociapps loads WebAssembly applications from OCI Distribution v2 registries, with multi-registry cluster fallback. Each "app" is an OCI image whose manifest annotates each layer with the VFS-relative file name it represents (typically: app.wasm, worker.js, wasm_exec.js), so a single image can carry every artifact a browser needs to instantiate the wasm module.

Three layers of API are exposed:

• Registry + Resolver: low-level multi-registry HTTP client. A Resolver tries its registries in order and returns the first one that responds successfully -- the cluster fallback the wasmdesk loader needs when one mirror is offline or geo-blocked.

• Manifest + Descriptor: thin OCI image-manifest model. Other OCI fields (history, config payload) are kept in the parsed JSON but not interpreted; the only annotation namespace this package reads is "ociapps.path/<name>" -> "sha256:..." per layer.

• App + Resolver.LoadApp: end-to-end loader that pulls a manifest + every referenced blob, content-addresses them through an in-memory LRU, and returns an App whose Files map is ready to hand off to the browser side of the wasm host.

On js/wasm builds an optional IndexedDB cache (file cache_wasm.go, build tag `js && wasm`) lets app loads survive a page reload without re-fetching every blob; on host builds the cache field defaults to a no-op so the loader is identical from the caller's POV.

This package is the generalisation of go-quake1/engine/ociassets: the Quake loader streamed pak files keyed by "quake.path/...", this one streams arbitrary apps keyed by "ociapps.path/...". The fetch machinery, single-flight cache, and Distribution v2 wire shape are the same.

Index

• Constants
• Variables
• func BuildFileMap(m *Manifest) (map[string]string, error)
• func EncodeManifest(m *Manifest) ([]byte, error)
• func PackLayout(outDir, reference string, files []FileEntry) (manifestDigest string, manifestSize int64, err error)
• func ServeLayout(layoutDir, repo, urlPath string) (body []byte, contentType string, status int, err error)
• func Sha256Digest(data []byte) string
• func VerifyDigest(data []byte, expected string) error
• func WriteStaticTree(layoutDir, repo, outRoot string) ([]string, error)
• type App
• type Cache
• type Descriptor
• type FileEntry
• type HTTPDoer
• type Manifest
  • func DecodeManifest(body []byte) (*Manifest, error)
• type Pusher
  • func (p *Pusher) PushLayout(layoutDir, repo, tag string) (manifestDigest string, err error)
• type Registry
• type Resolver
  • func (r *Resolver) FetchBlob(ctx context.Context, repo, digest string) (Registry, []byte, error)
  • func (r *Resolver) FetchManifest(ctx context.Context, repo, reference string) (Registry, *Manifest, error)
  • func (r *Resolver) LoadApp(ctx context.Context, ref string) (*App, error)
• type TokenAuthDoer
  • func (t *TokenAuthDoer) Do(req *http.Request) (*http.Response, error)

Constants

const (
	// MediaTypeManifest is the standard OCI image-manifest media type.
	// distribution:2 validates this against an internal allowlist and
	// rejects manifests with unrecognised mediaType values
	// (MANIFEST_INVALID). The app-specific identity is carried in the
	// manifest annotations + the config blob's mediaType -- both are
	// passed through verbatim by every spec-compliant registry.
	MediaTypeManifest = "application/vnd.oci.image.manifest.v1+json"

	// MediaTypeConfig is the descriptor.mediaType for the config blob.
	// Vendor suffix is allowed; the body itself is free-form JSON.
	MediaTypeConfig = "application/vnd.wasmdesk.ociapps.config.v1+json"

	// MediaTypeLayerWasm is the descriptor.mediaType for a .wasm layer.
	MediaTypeLayerWasm = "application/wasm"

	// MediaTypeLayerJS is the descriptor.mediaType for a JS layer
	// (worker.js, wasm_exec.js, etc.).
	MediaTypeLayerJS = "application/javascript"

	// MediaTypeLayerOctet is the catch-all descriptor.mediaType for
	// any layer that isn't .wasm or .js (assets, .json, etc.).
	MediaTypeLayerOctet = "application/octet-stream"

	// AnnotationPathPrefix is prepended to each VFS-relative file
	// name in the manifest's annotation map ("ociapps.path/app.wasm",
	// ...). The prefix lets the package coexist with other annotation
	// namespaces (org.opencontainers.image.*) without collision.
	AnnotationPathPrefix = "ociapps.path/"
)

Media types this package emits + accepts. Grouped in one place so the CLI packer and the runtime resolver agree on the wire vocabulary.

Variables

var ErrEmptyReference = errors.New("ociapps: empty reference")

ErrEmptyReference is returned by Resolver.LoadApp when the ref argument is the empty string. Surface separately from a parse error so callers can give a precise "you forgot to set --ref" diagnostic.

var ErrInvalidReference = errors.New("ociapps: invalid reference")

ErrInvalidReference is returned when LoadApp's ref argument can't be split into a repo + tag pair.

var ErrManifestNoAnnotations = errors.New("ociapps: manifest has no ociapps.path/* annotations")

ErrManifestNoAnnotations is returned when a manifest's annotations map carries no entries under AnnotationPathPrefix. Without the path->digest mapping the loader has no way to translate file names into blob fetches, so it fails fast at LoadApp time.

var ErrManifestNoLayers = errors.New("ociapps: manifest has no layers")

ErrManifestNoLayers is returned when a manifest carries an empty layers array. The loader can't usefully build an App from a manifest with no blobs so we surface this as a typed error rather than let later fetches fail one-by-one.

var ErrNoRegistries = errors.New("ociapps: resolver has no registries configured")

ErrNoRegistries is returned by Resolver methods when Registries is empty. The fix is for the caller to populate at least one entry before invoking the resolver.

Functions

func BuildFileMap

func BuildFileMap(m *Manifest) (map[string]string, error)

BuildFileMap walks m.Annotations and returns a VFS-name -> digest map keyed without the AnnotationPathPrefix. The result is what Resolver.LoadApp populates App.Files from.

Returns ErrManifestNoAnnotations when no annotations carry the ociapps.path/ prefix (likely a manifest from a different producer).

func EncodeManifest

func EncodeManifest(m *Manifest) ([]byte, error)

EncodeManifest is the round-trip companion of DecodeManifest. It emits canonical JSON (two-space indent, no HTML escaping) suitable for writing to an OCI image-layout `blobs/sha256/<digest>` file.

func PackLayout

func PackLayout(outDir, reference string, files []FileEntry) (manifestDigest string, manifestSize int64, err error)

PackLayout writes an OCI image-layout directory at outDir packing the given files as one layer each. Layout produced:

outDir/
  oci-layout
  index.json
  blobs/
    sha256/
      <hex of manifest digest>
      <hex of config digest>
      <hex of each layer digest>

The reference (e.g. "terminal:latest") is recorded as the org.opencontainers.image.ref.name annotation on the index entry -- that's the tag a pusher (incl. Pusher) reads when uploading.

Returns the manifest's digest + size so the caller can echo them (handy for "what did I just produce" CLI banners).

func ServeLayout

func ServeLayout(layoutDir, repo, urlPath string) (body []byte, contentType string, status int, err error)

ServeLayout maps an OCI image-layout directory onto the path fragments the Distribution v2 API exposes. Used by the test fake registry + by tools that want to serve an OCI layout directly without a real registry.

Returns the (body, contentType, status) triple for a given request path. The path forms recognised:

/v2/<repo>/manifests/<reference>  -> manifest bytes
/v2/<repo>/blobs/sha256:<hex>     -> blob bytes

repo is fixed to the value the caller passed in; the directory itself doesn't record a repo name so the caller decides what to map it under.

func Sha256Digest

func Sha256Digest(data []byte) string

Sha256Digest hashes data and returns the canonical "sha256:<hex>" digest string. Used by the CLI packer when emitting blob filenames.

func VerifyDigest

func VerifyDigest(data []byte, expected string) error

VerifyDigest recomputes sha256 over data and asserts it matches the expected "sha256:<hex>" string. Used after a blob fetch to detect cache / transport corruption -- a registry that returns the wrong bytes for a digest must never silently feed them to a wasm loader.

func WriteStaticTree

func WriteStaticTree(layoutDir, repo, outRoot string) ([]string, error)

WriteStaticTree materializes an OCI image-layout directory (as produced by PackLayout) as a static file tree matching the Distribution v2 GET API, so a plain static file server — GitHub Pages, an S3 bucket, `python -m http.server` — can serve it with no registry process and no auth:

outRoot/v2/<repo>/manifests/<tag>           (and .../manifests/<digest>)
outRoot/v2/<repo>/blobs/sha256:<hex>        (manifest, config, every layer)

It is the on-disk twin of ServeLayout: the same path vocabulary, written once instead of served dynamically. The browser-side OCIAppsLoader requests exactly these paths, so a tree written here is loadable same-origin — no CORS, no token, no proxy. That is the point: ghcr (and every other public registry) refuses cross-origin browser reads, but a same-origin static mirror beside the page needs none of that.

repo is the name to mount the layout under (e.g. "hello"); the layout directory itself records no repo name. The reference tag is taken from the index entry's org.opencontainers.image.ref.name annotation (the part after the last ':', so "hello:latest" -> "latest"); the manifest is always also written under its own digest. Returns the slash-separated relative paths written, sorted, for a CLI banner or a deploy manifest.

Types

type App

type App struct {
	Manifest    *Manifest
	Annotations map[string]string
	Files       map[string][]byte
}

App is a loaded wasm application: the parsed manifest, the union of its annotations, and every referenced blob keyed by the VFS-relative file name from the manifest's "ociapps.path/" entries.

Files is ready to hand off to the wasm host -- typical consumers pull "app.wasm" out and instantiate it directly, mounting the others ("worker.js", "wasm_exec.js") on the page through a script tag or a Blob URL.

type Cache

type Cache interface {
	Get(digest string) ([]byte, bool)
	Put(digest string, data []byte)
}

Cache is the optional blob-byte cache plugged into Resolver. On host builds the default is an in-memory LRU-ish map; on js/wasm builds callers can swap in an IndexedDB-backed cache so app loads survive a page reload.

Implementations must be safe for concurrent use.

type Descriptor

type Descriptor struct {
	MediaType string `json:"mediaType"`
	Digest    string `json:"digest"`
	Size      int64  `json:"size"`
}

Descriptor is the standard OCI content descriptor (mediaType + digest + size). Optional fields (urls, annotations, platform) are kept in the raw JSON but not modelled here; callers that need them can inspect Manifest via json.Unmarshal over the raw bytes.

type FileEntry

type FileEntry struct {
	// Name is the VFS-relative file name (e.g. "app.wasm",
	// "worker.js"). Stored verbatim in the manifest annotation under
	// [AnnotationPathPrefix].
	Name string

	// Path is the on-disk path the packer reads from.
	Path string

	// MediaType is recorded on the resulting layer descriptor.
	// Empty -> [MediaTypeLayerOctet].
	MediaType string
}

FileEntry pairs a VFS-relative name with the on-disk path holding its bytes + the media type to record on the resulting layer. The CLI packer feeds PackLayout a slice of these.

type HTTPDoer

type HTTPDoer interface {
	Do(*http.Request) (*http.Response, error)
}

HTTPDoer is the minimum surface Resolver needs from net/http. Tests inject an httptest.Server-backed http.Client; production uses http.DefaultClient (which on wasm transparently routes through the browser fetch() API).

type Manifest

type Manifest struct {
	SchemaVersion int               `json:"schemaVersion"`
	MediaType     string            `json:"mediaType,omitempty"`
	Config        Descriptor        `json:"config"`
	Layers        []Descriptor      `json:"layers"`
	Annotations   map[string]string `json:"annotations,omitempty"`
}

Manifest is the OCI v1 image-manifest JSON shape, narrowed to the fields this package reads or writes. Extra fields registries add on the wire are tolerated (json.Unmarshal drops unknown keys).

func DecodeManifest

func DecodeManifest(body []byte) (*Manifest, error)

DecodeManifest parses raw JSON into a Manifest. The schemaVersion is required to be 2 (the only OCI image-manifest version this package understands); any other value returns an error.

type Pusher

type Pusher struct {
	// Client is the HTTPDoer used for every request. nil ->
	// http.DefaultClient.
	Client HTTPDoer

	// BaseURL is the full origin (scheme + host) of the destination
	// registry, no trailing slash. Example: "http://localhost:5000".
	BaseURL string
}

Pusher uploads an OCI image-layout directory (produced by PackLayout) to one OCI Distribution v2 registry. Plain HTTP; no auth, no TLS by default -- mirrors what `registry:2` accepts on localhost:5000. A custom Client lets callers point at an https registry or attach auth headers via a wrapping HTTPDoer.

func (*Pusher) PushLayout

func (p *Pusher) PushLayout(layoutDir, repo, tag string) (manifestDigest string, err error)

PushLayout uploads every blob + the manifest from layoutDir under the given repo + tag. Returns the manifest digest the registry confirmed (always the same as the local manifest digest -- a mismatch indicates the registry mutated the manifest, which is outside spec).

type Registry

type Registry struct {
	URL string
}

Registry is one OCI Distribution v2 endpoint. URL is the scheme + host (no trailing slash); future fields (Auth, TLS) will land here without breaking existing call sites.

type Resolver

type Resolver struct {
	Registries []Registry
	Client     HTTPDoer
	Cache      Cache
	// contains filtered or unexported fields
}

Resolver picks among a cluster of registries. The default policy is sequential fallback in declaration order: each call tries Registries from index 0 and returns the first one whose request succeeds (HTTP 200 + body decode). On all-fail the last error is surfaced verbatim so callers can see which registry was tried last.

func (*Resolver) FetchBlob

func (r *Resolver) FetchBlob(ctx context.Context, repo, digest string) (Registry, []byte, error)

FetchBlob does GET /v2/<repo>/blobs/<digest> against the cluster. Bytes are content-addressed: a successful fetch from any registry is cached under the digest so the next call for the same blob hits the in-memory cache regardless of which mirror is healthy.

func (*Resolver) FetchManifest

func (r *Resolver) FetchManifest(ctx context.Context, repo, reference string) (Registry, *Manifest, error)

FetchManifest does GET /v2/<repo>/manifests/<reference> against the cluster. The first registry that responds 200 wins; the parsed manifest + the winning Registry are returned so callers can stick to the same mirror for the follow-up blob fetches.

func (*Resolver) LoadApp

func (r *Resolver) LoadApp(ctx context.Context, ref string) (*App, error)

LoadApp pulls a manifest + all referenced blobs from the cluster. ref is "<repo>:<tag>" -- the registry component is taken from Resolver.Registries, so the same logical reference can be served by every mirror without the caller having to embed a host name.

Convention: each layer carries a manifest annotation "ociapps.path/<file>" -> "<digest>"; the same digest must appear in the manifest's Layers array. LoadApp does NOT trust files outside that annotation -- a layer with no annotation is skipped.

type TokenAuthDoer

type TokenAuthDoer struct {
	// Base is the underlying doer (nil -> http.DefaultClient).
	Base HTTPDoer
	// Username / Password are sent as HTTP Basic auth to the token realm.
	Username string
	Password string
	// contains filtered or unexported fields
}

TokenAuthDoer wraps an HTTPDoer with OCI Distribution / Docker-registry Bearer-token auth. Registries such as ghcr.io answer an unauthenticated request with 401 + a `WWW-Authenticate: Bearer realm=...,service=..., scope=...` challenge; this fetches a token from the realm (anonymously, or with the configured credentials for push scopes) and retries the original request with `Authorization: Bearer <token>`. The last token is reused across subsequent requests (a push is single-repo, so its many blob uploads do not each re-auth); a stale token simply triggers one more challenge + refresh.

Empty Username+Password is valid — it does anonymous token fetches, which is all a public registry needs for read. A ghcr **push** needs a GitHub username plus a PAT / GITHUB_TOKEN with `packages:write`.

It is safe for sequential use (the pusher uploads blobs one at a time); the token cache is mutex-guarded so a shared doer does not race.

func (*TokenAuthDoer) Do

func (t *TokenAuthDoer) Do(req *http.Request) (*http.Response, error)

Do sends req, transparently handling a single Bearer-token challenge: it attaches any cached token, and on a 401 with a parseable Bearer challenge it fetches a fresh token and replays the request once.