opod-sdk

README

opod-sdk

The Go contract for Opod: the exact JSON an Opod leader and its workers speak, the environment variables they read, and the model catalog they share — as Go types you import instead of guessing.

opod.io · opod-core (the runtime) · Discussions · Contributing · Support · Apache-2.0

go get github.com/opod-io/opod-sdk@latest

---

Contents

• What is this, in one minute
• Do I need it?
• The Opod family
• Words you will meet
• What is in the box
• Install
• Recipes
  1. Check which leader you are talking to
  2. Read the load for an autoscaler
  3. Pull every usage row, never lose one
  4. Follow lifecycle events
  5. Write the auth file a leader watches
  6. Write the policy file a leader watches
  7. Check the environment you render
  8. Read the model catalog
  9. Speak the node protocol (sign a worker call)
  10. Hash prompt blocks the way a worker does
• Package reference
• Rules that keep the wire stable
• Versioning
• Common mistakes
• FAQ
• Contributing, security, licence

---

What is this, in one minute

Opod is a self-hosted inference runtime. You run a leader (opod up) and one or more workers (opod join). The leader serves an OpenAI-compatible API; the workers run the engines (vLLM, llama.cpp, SGLang, …) on your GPUs.

Something has to manage a leader: start it, give it API keys, read its load, collect its usage, scale its workers. Opod's own control plane does that. So can your script, your autoscaler, your dashboard or your CI check.

That manager and the leader must agree on every JSON field, every route, every environment variable. This module is where they agree. The leader marshals these exact Go types itself, so:

• a field in this module is a field on the wire — not a copy of it, not a guess of it;
• when you decode a leader's answer into these types, you read what the leader wrote;
• when you write a file the leader reads (keys, policy), the leader parses the same types.

It is types and constants only. There is no HTTP client, no retry logic and no business rule in it. You call the leader with net/http (the recipes show how) and decide everything yourself. It imports nothing but the Go standard library.

Do I need it?

You are writing…	Use	Why
A dashboard, a CLI or a CI check that reads a leader's /admin/v1 or /loadz	adminapi	Decode the answers without writing your own structs
An autoscaler (KEDA, a custom loop)	adminapi.Load	/loadz is built for it: in-flight, queue, KV cache, TTFT
A usage collector or a billing export	adminapi.UsageBatch	The cursor contract that never loses a row
A manager that mounts keys and policy into a leader	adminapi.AuthSnapshot, adminapi.PolicySnapshot	The leader watches these files and reloads them live
A pod template, a systemd unit or a script that starts opod	adminapi.Env()	Every variable the binary reads, with who reads it and since when
A tool that lists or validates models	catalog	The same 48 model files the leader embeds
Your own worker, or a test double of one	nodeapi	The register / heartbeat / load protocol, held by golden files

You do not need it to use Opod. A client that only chats with models talks to the leader's OpenAI-compatible /v1 API with any OpenAI SDK. This module is for code that manages Opod.

The Opod family

Part	What it is	Where
Opod core	The open runtime: opod up (leader) and opod join (worker). Apache-2.0, CLI only	github.com/opod-io/opod-core
Opod SDK (this repo)	The contract between core and anything that manages it. Apache-2.0	here
Opod Control Plane (opodcp)	The commercial manager on top of core: runs leaders and workers on Kubernetes, a per-GPU memory ledger, rollouts, usage export, a web console. It imports this module like any other manager	opod.io

Core never knows a manager exists. It reads files and environment variables, serves routes, and emits streams. Everything a manager can do goes through what this module describes. That is why the module exists: it is the whole wall between the two.

Words you will meet

Word	Meaning
Leader	The opod up process. One per endpoint. Serves /v1/* to clients and /admin/v1/* to managers. Holds no GPU of its own unless you ask it to
Worker	An opod join process beside one engine on one machine. Registers with the leader, heartbeats every 5 s, loads models when asked
Engine	The inference server a worker drives: vLLM, SGLang, llama.cpp, TensorRT-LLM, Ollama, MLX-LM
Manager	Anything that steers a leader from outside: the control plane, your script, your autoscaler
Gang	One model split across several workers (llama.cpp RPC parts, a vLLM + Ray cluster, or SGLang's launcher)
Plan / auth / policy file	JSON files a manager mounts into the leader's filesystem. The leader watches them and applies a change without restarting
Contract version	adminapi.ContractVersion ("v1"). Every change inside one version is additive
Feature key	A name in GET /admin/v1/capabilities → features ("usage_stream": true). Probe it before you rely on a newer behaviour
Admin key	The sk-orc-… key opod up prints once. It authenticates every /admin/v1 call except register and heartbeat
Node token	A key minted with opod token create --node. A worker signs its calls with it (HMAC); it never sends it in the clear

What is in the box

github.com/opod-io/opod-sdk
├── adminapi/   what a leader serves to a manager, and the files and variables a manager gives it
│   ├── types.go     Version, Capabilities, Load, UsageBatch, EventsBatch,
│   │                AuthSnapshot, PolicySnapshot, DoorRoutes, …
│   ├── env.go       the environment contract: Env(), Lookup(), Retired(), one Env* constant per variable
│   ├── overhead.go  EngineOverheadMiB: measured memory an engine holds outside its pool
│   └── fetch.go     FetchResult: what `opod fetch` reports once weights are on disk
├── nodeapi/    the node protocol: the JSON between a worker and its leader, with the paths as constants
│   └── testdata/    golden files: every body as the binary wrote it
└── catalog/    the model catalog: one YAML per model, embedded, plus its schema (Entry)

Install

go get github.com/opod-io/opod-sdk@latest

• Go version: the one in go.mod or newer.
• Dependencies: none. The module imports the standard library only. To parse the catalog YAML you bring your own YAML library (recipe 8).
• Which tag? Take the newest. Within one contract version every tag is additive, and a leader ignores fields it does not know, so a newer SDK against an older leader is safe — probe feature keys for anything new (recipe 1).

import (
	"github.com/opod-io/opod-sdk/adminapi"
	"github.com/opod-io/opod-sdk/catalog"
	"github.com/opod-io/opod-sdk/nodeapi"
)

You need a running leader for recipes 1–4. The quickest one:

curl -fsSL https://raw.githubusercontent.com/opod-io/opod-core/main/installer/install.sh | sh
opod up                     # prints the admin key once; it is also in ~/.opod/admin.key
export OPOD_URL=http://localhost:8080
export OPOD_ADMIN_KEY=$(cat ~/.opod/admin.key)

---

Recipes

Every recipe is a complete main package. They share one small helper for authenticated GETs; copy it once.

// getJSON calls a leader route with the admin key and decodes the answer into out.
func getJSON(ctx context.Context, base, key, path string, out any) error {
	req, err := http.NewRequestWithContext(ctx, http.MethodGet, base+path, nil)
	if err != nil {
		return fmt.Errorf("build %s: %w", path, err)
	}
	req.Header.Set("Authorization", "Bearer "+key)
	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		return fmt.Errorf("GET %s: %w", path, err)
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		body, _ := io.ReadAll(io.LimitReader(resp.Body, 4096))
		return fmt.Errorf("GET %s: %s: %s", path, resp.Status, body)
	}
	return json.NewDecoder(resp.Body).Decode(out)
}

1. Check which leader you are talking to

When: before anything else. A manager that talks to many leaders of many versions asks each one what it can do, instead of guessing from a version string.

var v adminapi.Version
if err := getJSON(ctx, base, key, "/admin/v1/version", &v); err != nil {
	log.Fatal(err)
}
if v.Contract != adminapi.ContractVersion {
	log.Fatalf("leader speaks contract %s, this SDK speaks %s", v.Contract, adminapi.ContractVersion)
}

var caps adminapi.Capabilities
if err := getJSON(ctx, base, key, "/admin/v1/capabilities", &caps); err != nil {
	log.Fatal(err)
}
fmt.Println("leader", v.Version, "serves", len(caps.Routes), "contract routes")
if !caps.Features["usage_stream"] {
	log.Fatal("this leader has no usage stream; upgrade it")
}

• caps.Routes is the frozen route list — every method and path a manager may call, with chi placeholders (/admin/v1/nodes/{id}/drain).
• caps.Features is the list of feature keys. A missing key means false. An older leader simply does not list a newer feature.
• Every EnvVar.Since in recipe 7 is a feature key (or an SDK version), so the same map tells you which environment variables the binary reads.

2. Read the load for an autoscaler

When: you scale workers up and down. /loadz is unauthenticated (a probe), cheap, and built for this.

resp, err := http.Get(base + "/loadz")
if err != nil {
	log.Fatal(err)
}
defer resp.Body.Close()
var l adminapi.Load
if err := json.NewDecoder(resp.Body).Decode(&l); err != nil {
	log.Fatal(err)
}
fmt.Printf("in flight %d · queue %d · KV %.0f%% · TTFT p95 %d ms · %d/%d workers reporting\n",
	l.InFlight, l.QueueDepth, l.KVUsedPct, l.TTFTP95Ms, l.Reporting, l.Workers)
if l.EnginesUnhealthy > 0 {
	fmt.Println("not capacity:", l.EnginesUnhealthy, "workers hold a card with a dead engine:", l.EngineIssue)
}

What to scale on, and why:

Field	Use it for	Watch out
InFlight, QueueDepth	"requests are waiting"	QueueDepth is the engines' own queue, summed
KVBusyWorkers	horizontal scaling — replicas = ceil(KVBusyWorkers ÷ target)	KVUsedPct is a maximum over workers and can never ask for more than two replicas; use KVBusyWorkers for a replica count
TTFTP50Ms, TTFTP95Ms	a latency budget	0 means nothing streamed in the last minute, never "instant"
EnginesUnhealthy	subtract from capacity	those workers heartbeat like healthy ones
RPM1m, LastRequestUnix	scale to zero after idle
pointer fields (KVCapacityTokens, Tier*, EngineTTFTP50Ms)	engine-reported detail	nil means the engine did not say — never read it as 0

A leader started with OPOD_PROBE_LISTEN also serves /loadz on a second, always-plain port, so a scraper needs no TLS certificate.

3. Pull every usage row, never lose one

When: you export usage (tokens per key, per model) to a database, a warehouse or a billing tool.

The stream is a cursor: you remember the last id you stored, and ask for the rows after it. The leader keeps rows on disk, so a crash on either side loses nothing — at worst you read a row twice, so dedupe by id.

after := loadCursor() // the last id you committed; 0 the first time
var boot int64
for {
	var page adminapi.UsageBatch
	path := fmt.Sprintf("/admin/v1/usage/stream?after=%d&limit=1000", after)
	if err := getJSON(ctx, base, key, path, &page); err != nil {
		log.Fatal(err)
	}
	if boot != 0 && page.Boot != 0 && page.Boot != boot {
		log.Println("leader restarted; ids may have restarted with it")
	}
	boot = page.Boot
	for _, e := range page.Events {
		u := e.Data
		store(e.ID, e.TS, u.APIKeyID, u.Model, u.PromptTokens, u.CompletionTokens, u.LatencyMS, u.Outcome)
	}
	after = page.Next
	saveCursor(after) // commit AFTER the rows are stored, never before
	if !page.More {
		time.Sleep(5 * time.Second)
	}
}

• limit is at most 1000; the default is 500.
• Next is the last id on the page; pass it back as after.
• More is true when another page is waiting right now.
• UsageData.APIKeyID of a key you mounted in an auth file starts with adminapi.SnapshotKeyPrefix (k_cp_). Strip it to find your own key id.
• UsageData.CostUSD is deprecated and always 0. Opod records facts, not prices; rate the tokens on your side.
• Request and response content is never in the stream. Only counts and timings.

4. Follow lifecycle events

When: you want to know when a node joined, a model loaded, a gang formed or healed.

Same cursor contract, different route and type:

var page adminapi.EventsBatch
if err := getJSON(ctx, base, key, fmt.Sprintf("/admin/v1/events/stream?after=%d", after), &page); err != nil {
	log.Fatal(err)
}
for _, e := range page.Events {
	switch e.Type {
	case "node.registered", "node.drained", "node.removed":
		fmt.Println("node", e.Subject, e.Type)
	case "shard.created", "shard.healed", "shard.heal_failed":
		fmt.Println("gang of", e.Subject, e.Type, e.Data)
	default:
		// New types appear over time. Ignore what you do not know — never fail on it.
	}
}

Data is a map[string]any on purpose: each event type carries its own fields, and new ones are added without a new SDK type.

5. Write the auth file a leader watches

When: you manage API keys from outside the leader. You mount a JSON file and point OPOD_AUTH_FILE at it (default /etc/opod-auth/auth.json). The leader re-reads it on change — no restart.

The file holds hashes, never keys: the SHA-256 of each key, in lowercase hex.

plaintext := "sk-orc-" + randomString() // give this to the user ONCE; never store it
sum := sha256.Sum256([]byte(plaintext))

snap := adminapi.AuthSnapshot{
	Revision:    "2026-10-31T12:00:00Z#7", // any string that changes when the content changes
	RequireKeys: true,                     // false = anonymous requests are allowed
	IssuedAt:    time.Now().UTC().Format(time.RFC3339),
	Keys: []adminapi.SnapshotKey{{
		ID:        "alice-laptop",                // leader records it as "k_cp_alice-laptop"
		Name:      "Alice's laptop",
		Hash:      hex.EncodeToString(sum[:]),
		ExpiresAt: "2027-01-31T00:00:00Z",        // optional, RFC 3339
		Class:     "standard",                    // critical | standard | sheddable; ranks when full
		Team:      "search",                      // keys of one team share fairly inside a class
	}},
	RevokedKeys: []string{"bob-old"}, // a tombstone: refused even if a row for it is still in Keys
}
b, err := json.MarshalIndent(snap, "", "  ")
if err != nil {
	log.Fatal(err)
}
// Write to a temp file and rename it, so the leader never reads half a file.
if err := os.WriteFile("/etc/opod-auth/auth.json.tmp", b, 0o600); err != nil {
	log.Fatal(err)
}
if err := os.Rename("/etc/opod-auth/auth.json.tmp", "/etc/opod-auth/auth.json"); err != nil {
	log.Fatal(err)
}

What each field does:

Field	Meaning
Revision	Names this version of the file. Change it whenever the content changes
RequireKeys	true = every gateway request needs a key
Keys[].Hash	hex(sha256(plaintext)). The plaintext never touches disk
Keys[].Class	Rank when the endpoint is full. A request may lower its own class with X-Opod-Class, never raise it
RevokedKeys	Ids the leader refuses forever. A leader never un-revokes an id it has seen
IssuedAt	When you wrote the file. Pairs with PolicyAuth.MaxSnapshotAgeSec (recipe 6)
NodeMTLS, NodeCertCA, RevokedCerts	Worker client certificates: off · allow · require, the CA PEM, revoked serials (lowercase hex)

A key carries no rate limit or quota. Limits belong to the gateway in front of Opod (LiteLLM, Envoy, your API gateway).

6. Write the policy file a leader watches

When: you change how a leader routes, admits and wakes, without restarting it. Mount it at OPOD_POLICY_FILE (default /etc/opod-auth/policy.json).

pol := adminapi.PolicySnapshot{
	Revision: "policy-12",
	Routing: adminapi.PolicyRouting{
		KVWeight:        2,    // a full KV cache counts as 2 extra requests
		KVSaturationPct: 90,   // a worker above 90% KV gets new work only when no other has room
		PrefixAffinity:  true, // a repeated prompt prefix goes back to the worker that has it cached
		CacheIsolation:  adminapi.CacheIsolationTeam,
		Revisions: []adminapi.RevisionWeight{ // a canary: 10% of traffic to plan revision 8
			{Revision: 7, Weight: 9},
			{Revision: 8, Weight: 1},
		},
	},
	Admission: adminapi.PolicyAdmission{
		HoldMs:  2000, // wait up to 2 s for a free slot, then 503 + Retry-After
		MaxHeld: 200,
		Classes: []adminapi.PolicyAdmissionClass{
			{Name: "critical", HoldMs: 10000},
			{Name: "sheddable", HoldMs: 0, MaxShare: 0.25}, // shed at once; at most a quarter of the slots
		},
	},
	Auth: adminapi.PolicyAuth{MaxSnapshotAgeSec: 0}, // 0 = keep serving from the last auth file forever
}

• Every zero value is the old behaviour. An empty policy changes nothing. Set only what you mean.
• Weight is a share, not a percent: {9, 1} and {90, 10} both mean 10%.
• MaxSnapshotAgeSec > 0 is a trade: the leader refuses keyed traffic once the auth file is older than that. You get fast revocation; you lose serving through a manager outage. You must then re-issue the auth file more often than the bound.
• Routing.SpeedRank is reserved; no leader implements it.

7. Check the environment you render

When: you write a pod template, a systemd unit or a Compose file that starts opod. Catch a typo before it ships.

func checkEnv(rendered map[string]string, caps adminapi.Capabilities) []string {
	var problems []string
	for name := range rendered {
		if v, ok := adminapi.Lookup(name); ok {
			// Since is a feature key ("vram_budget", "vertex") or an SDK version ("v0.3.0").
			sdkVersion := len(v.Since) > 1 && v.Since[0] == 'v' && v.Since[1] >= '0' && v.Since[1] <= '9'
			if v.Since != "" && !sdkVersion && !caps.Features[v.Since] {
				problems = append(problems, name+": this leader lacks feature "+v.Since)
			}
			continue
		}
		if r, ok := adminapi.LookupRetired(name); ok {
			problems = append(problems, name+": retired in "+r.RetiredIn+" — "+r.Note)
			continue
		}
		if strings.HasPrefix(name, "OPOD_") {
			problems = append(problems, name+": not in the contract (typo, or a config override)")
		}
	}
	return problems
}

Each row of adminapi.Env() has:

Field	Meaning
Name	The variable, e.g. OPOD_VRAM_BUDGET_GB. Also a constant: adminapi.EnvVRAMBudgetGB
Side	leader, worker or both — which process reads it
Doc	The leader's own one-line description
Since	A feature key to probe ("vram_budget"), or the SDK version that first listed it ("v0.3.0"). Empty = every binary on this contract

Print the whole table:

for _, v := range adminapi.Env() {
	fmt.Printf("%-30s %-7s %s\n", v.Name, v.Side, v.Doc)
}

Not in the table, on purpose: the OPOD_* spellings of config.yaml keys (listen address, data directory, …) and the container entrypoint's own launch variables (OPOD_ROLE, OPOD_LEADER_URL). The leader does not declare those as contract data, so the SDK does not list them.

8. Read the model catalog

When: you list models, validate a plan against a model's shape, or size a card.

The catalog is one YAML file per model, embedded in this module. The SDK stays stdlib-only, so you parse it with your own YAML library — catalog.Entry carries the yaml tags.

import (
	"github.com/opod-io/opod-sdk/catalog"
	"gopkg.in/yaml.v3"
)

for _, name := range catalog.Names() { // "llama-3.2-1b.yaml", … sorted
	b, err := catalog.Read(name)
	if err != nil {
		log.Fatal(err)
	}
	var e catalog.Entry
	if err := yaml.Unmarshal(b, &e); err != nil {
		log.Fatalf("%s: %v", name, err)
	}
	fmt.Printf("%-32s %6.1f GB  ctx %-7d engines %v  sharded=%v\n",
		e.ID, float64(e.SizeBytes)/1e9, e.ContextWindow, e.RecommendedEngines, e.Sharding.Required)
}

• catalog.FS is an fs.FS if you prefer fs.WalkDir or template.ParseFS.
• Entry.Architecture (layers, heads, KV heads, head dim, MLA and sliding-window fields) is what you need to compute a KV cache size. Zero means unknown: warn, never refuse.
• Entry.ToolParser names the tool-call parser each engine needs for this model.
• The full schema, with every field and examples: catalog/README.md.
• To add a model, add one YAML file and open a PR. No code change.

9. Speak the node protocol (sign a worker call)

When: you write your own worker, a worker test double, or a tool that talks to a worker's listener. Most people never need this — opod join does it for you.

Two directions, two listeners:

Direction	Listener	Paths
worker → leader	the leader's (:8080)	nodeapi.PathRegister once, then nodeapi.PathHeartbeat every 5 s
leader → worker	the worker's (:8081)	PathModelLoad, PathModelUnload, PathModelSleep, PathModelResume, PathAdapters*, PathProcess*, PathTokenize

Every call carries an HMAC header. The node token is the key; the token itself never travels:

X-Opod-Auth: v=1,id=<node id>,ts=<unix seconds>,sig=<hex>
sig = hex( HMAC-SHA256( token, "v1\n" + METHOD + "\n" + path + "\n" + ts ) )

• path is the URL path without the query string.
• METHOD is upper case.
• The receiver refuses a ts more than 5 minutes from its own clock. Keep NTP on.
• The body is not signed. Uploads stream gigabytes and are never buffered to hash. That is why only the key set and value types matter, and why the golden files freeze exactly those.

The SDK has no signer function; the scheme is six lines:

func sign(req *http.Request, nodeID, token string) {
	ts := strconv.FormatInt(time.Now().Unix(), 10)
	mac := hmac.New(sha256.New, []byte(token))
	mac.Write([]byte("v1\n" + strings.ToUpper(req.Method) + "\n" + req.URL.Path + "\n" + ts))
	req.Header.Set("X-Opod-Auth", "v=1,id="+nodeID+",ts="+ts+",sig="+hex.EncodeToString(mac.Sum(nil)))
}

Register, then heartbeat:

hw, err := nodeapi.EncodeHardware(nodeapi.Capabilities{
	Hostname: "gpu-box-1", OS: "linux", Arch: "amd64", CPUCores: 32, RAMGB: 128,
	GPUs:   []nodeapi.GPU{{Name: "NVIDIA RTX 4090", VRAMGB: 24}},
	Engine: "vllm",
})
if err != nil {
	log.Fatal(err)
}
reg := nodeapi.RegisterRequest{
	ID: "gpu-box-1", Hostname: "gpu-box-1", OS: "linux", Arch: "amd64", RAMGB: 128,
	Address:      "10.0.0.12:8081", // where the leader dials this worker back
	HardwareJSON: hw,               // JSON inside JSON, on purpose: see below
	BootID:       newBootID(),      // a fresh id every process start
}
post(nodeapi.PathRegister, reg) // build the request, sign(req, reg.ID, token), send

loaded := []string{"qwen3-coder-30b"}
post(nodeapi.PathHeartbeat, nodeapi.Heartbeat{
	ID: reg.ID, BootID: reg.BootID,
	LoadedModels: loaded, // nil = "engine did not answer"; []string{} = "holds nothing" — different words
	Slots:        16,     // how many requests the engine serves at once
	Engine:       &nodeapi.EngineState{State: nodeapi.EngineServing},
})

Things that surprise people:

• HardwareJSON is a string holding JSON. The leader stores it as it arrived. Use EncodeHardware / DecodeHardware.
• Capabilities and ProcessSpec have Capitalised JSON keys ("Hostname", "ID"). They predate JSON tags, and stored rows outlive any one binary.
• LoadedModels is never omitempty: null and [] mean different things. Never turn a failed engine query into [] — that tells the leader the worker holds nothing, and it stops routing to it.
• A heartbeat answered 404 means "unknown node, register first". 401/403 mean the token was refused.
• Worker errors are plain text with a status code, except three typed bodies: StartProcessError (502), SleepResponse with unsupported (501), UnloadModelResponse with unsupported (501).
• ProcessSpec's ReadyTimeout and RestartBackoff travel as integer nanoseconds (Go's time.Duration).

The golden files in nodeapi/testdata/ are the reference: each is a body exactly as the binary writes it.

10. Hash prompt blocks the way a worker does

When: you build prefix-cache-aware routing or analysis outside the leader. Workers report cache contents as block hashes, and both sides must compute them identically.

tokens := []int{151644, 872, 198 /* … the prompt's token ids … */}
hashes := nodeapi.BlockHashes(tokens, 16) // one hash per FULL block of 16 tokens; a partial tail is dropped
fmt.Println(hashes)                        // ["3f9c…", "a1b2…", …], each chains from the one before

A hash names a whole prefix, not a fragment: block n's hash depends on every block before it. No token ids or text leave the worker — only these hashes.

---

Package reference

Full API docs: pkg.go.dev/github.com/opod-io/opod-sdk. The map:

adminapi

Type / func	Wire	Notes
ContractVersion	"v1"	In Version.Contract and Capabilities.Contract
Version	GET /admin/v1/version	Binary version + contract
Capabilities, Route	GET /admin/v1/capabilities	Frozen routes + feature keys
Load, PrefixIndex, TierGhost, GhostBucket	GET /loadz (no auth)	AdmissionBoundSlots / AdmissionBoundKV for Load.AdmissionBound
UsageBatch, UsageEvent, UsageData	GET /admin/v1/usage/stream?after=&limit=	Cursor; dedupe by id
EventsBatch, LifecycleEvent	GET /admin/v1/events/stream?after=&limit=	Ignore unknown types
AuthSnapshot, SnapshotKey, SnapshotKeyPrefix	file at OPOD_AUTH_FILE	Hashes only
PolicySnapshot and Policy*, RevisionWeight, CacheIsolation*	file at OPOD_POLICY_FILE	Zero = old behaviour
DoorRoutes, DoorRoute, DoorCandidate	the routes file of opod up --role door --routes <file>	Name → endpoint lookup in front of many leaders
Env(), Lookup(), Retired(), LookupRetired(), EnvVar, RetiredEnvVar, Env*, Side*	process environment	The environment contract
EngineOverheadMiB(engine, vendor)	—	Measured MiB an engine holds outside its memory pool (vLLM: 837 NVIDIA, 1,103 AMD; llama.cpp: 640 NVIDIA). Unknown vendor → the engine's largest figure; unknown engine → 0
FetchResult, FetchSignature, Signature*	file at OPOD_FETCH_RESULT	unverified means a signature exists and nothing checked it — never a pass

nodeapi

Type / func	Wire
PathRegister, RegisterRequest, RegisterResponse, Capabilities, GPU, GivenDevice	worker → leader, once
PathHeartbeat, Heartbeat, HeartbeatResponse, EngineLoad, EngineState, KVBlocks	worker → leader, every 5 s
PathModelLoad / PathModelUnload, LoadModelRequest/Response, UnloadModelRequest/Response	leader → worker
PathModelSleep / PathModelResume, SleepResponse	leader → worker (vLLM sleep mode)
PathAdapters*, Adapter, LoadAdapterRequest, UnloadAdapterRequest, AdapterResponse, HeldAdapter	leader → worker (LoRA)
PathProcess*, ProcessSpec, ProcessInfo, StartProcessError, StopProcessRequest, UploadResponse, HeaderFilePath	leader → worker (gang parts)
PathTokenize, TokenizeRequest, TokenizeResponse	leader → worker
EncodeHardware, DecodeHardware, NormalizePCI, BlockHash, BlockHashes	helpers both sides call
Status*, Process*, Engine*, Role*, KVTier*, Vendor*	the status and enum words

catalog

Name	What
FS	fs.FS of every entry as <id>.yaml
Names()	Sorted file names
Read(name)	One file's bytes
Entry, Source, Sharding, Architecture, Hardware	The schema, yaml and json tags

Rules that keep the wire stable

1. JSON tags are the wire. The leader marshals these types itself; nothing mirrors them.
2. Additive only within a contract version. New optional fields, new rows, new types. A key is never renamed or re-typed.
3. Receivers ignore keys they do not know. So either side may be the newer one. Your code must do the same: never use DisallowUnknownFields on a leader's answer.
4. A pointer means "may be unstated". nil is "not said", never zero. Check before you dereference.
5. Golden files change only when the wire does. There is no -update flag, on purpose.
6. Standard library only. Nothing here imports anything else, and nothing here decides anything.

Both sides hold these with tests: core asserts its declared environment equals Env() and marshals its answers through these types; a manager asserts every variable it renders is in Env().

Versioning

Change	Tag
Additive — a new optional field (omitempty), a new row in a table, a new type or package, a doc fix. A consumer takes it without touching its code	patch: v0.x.Y
Anything a consumer must change for — a removed or renamed Go identifier, a field whose type changes, a key that starts or stops being omitted, a variable moved from Env() to Retired(), a new package a consumer is expected to decode into instead of its own types	minor: v0.X.0

A change to a key that is already on the wire is neither: it is a new ContractVersion, and both sides carry the old shape until every deployed binary is past it.

History: CHANGELOG.md.

Common mistakes

Mistake	What happens	Do this
Reading Load.KVUsedPct as a replica count	Your scaler never asks for more than 2 replicas	Use KVBusyWorkers
Treating a nil pointer field as 0	"0 tokens of KV capacity" when the engine just did not say	Check for nil
Treating TTFTP95Ms == 0 as fast	A quiet minute looks like a perfect one	0 = nothing streamed
Saving the cursor before storing the rows	A crash loses the page	Store, then save Next
Not deduping usage by id	Double-counted tokens after a retry	Upsert by id
Writing the auth file in place	The leader reads half a file	Write a temp file, then rename
Putting the plaintext key in Hash	Nobody can authenticate	hex(sha256(plaintext))
Signing the path with its query string	signature mismatch	Sign URL.Path only
Sending LoadedModels: [] when the engine timed out	The leader stops routing to a healthy worker	Send nil
DisallowUnknownFields on a leader answer	Breaks on the next additive release	Ignore unknown keys
Reading UsageData.CostUSD	Always 0	Price tokens on your side

FAQ

Is there an HTTP client in here? No. The module is the contract only. net/http plus the types is all a client needs (see the recipes). Keeping logic out is what lets core, the control plane and your code share it without sharing bugs.

Why does the control plane not have its own copy of these types? It did once. Copies drift. Now the leader marshals the SDK types and the control plane decodes the same ones, so a field cannot exist on one side only.

Can I use this from Python, TypeScript or Rust? Not directly; it is Go. The wire is plain JSON, so the types, the golden files in nodeapi/testdata/ and the pkg.go.dev page are the spec for a port.

My leader is older than my SDK. Is that safe? Yes, within one contract version. Fields the leader does not send decode as zero values (or nil pointers). Probe Capabilities.Features before relying on a newer behaviour.

My SDK is older than my leader? Also safe. Go's encoding/json drops keys your struct does not have. Upgrade when you want the new fields.

Where is the full list of /admin/v1 routes? Ask a leader: GET /admin/v1/capabilities → routes. The list is frozen and append-only. Core's README documents each route.

Where do I ask a question? Discussions. Bugs: an issue on this repo. Security: SECURITY.md.

Contributing, security, licence

• CONTRIBUTING.md — a catalog entry is a one-file PR; a wire change comes with its golden file and a CHANGELOG line
• SECURITY.md — report privately through a GitHub Security Advisory
• SUPPORT.md — where each kind of question goes
• CODE_OF_CONDUCT.md — Contributor Covenant 2.1
• LICENSE — Apache-2.0, no CLA, DCO sign-off on every commit