zen

package module
v1.4.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 18, 2026 License: BSD-3-Clause Imports: 23 Imported by: 0

Documentation

Overview

Copyright 2026 Hanzo AI Inc. All Rights Reserved.

Package zen is the Zen model family's serving layer: it maps a public "zen*" SKU onto an upstream provider, injects the Zen identity, folds the caller's reasoning intent into the upstream's vocabulary, and serves every modality the family offers — chat and embeddings (token I/O), plus image, audio (voice, music, foley), video, and rerank (media I/O) — behind OpenAI- and Anthropic-compatible endpoints. The catalog (catalog.yaml) is the single source of truth for the family; callers discover it via GET /v1/models.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AdminCatalog added in v1.3.1 

type AdminCatalog struct {
	Margin         string                   `json:"margin"`         // catalog-default resale multiple ("3" or "200%"); "" = 3×
	VisionFallback string                   `json:"visionFallback"` // SKU an image on a non-vision model routes to
	Providers      map[string]AdminProvider `json:"providers"`
	Models         map[string]AdminModel    `json:"models"`
	Aliases        map[string]string        `json:"aliases"`
}

AdminCatalog is the whole zen family as a SuperAdmin edits it: providers, models (with their upstream route + cost + derived retail), the resale margin, aliases, and the vision fallback. GET returns it fully populated (retail/profit derived); PUT accepts it and replaces the live catalog. It is the catalog value in JSON — the same shape catalog.yaml carries, so a round-trip is lossless.

type AdminModel added in v1.3.1 

type AdminModel struct {
	Mode            string      `json:"mode"`            // "" (chat) | embedding | image | audio | video | rerank
	Disabled        bool        `json:"disabled"`        // hidden from the tenant catalog
	Vision          bool        `json:"vision"`          // reads images
	Margin          string      `json:"margin"`          // per-SKU resale override; "" = catalog default
	EffectiveMargin string      `json:"effectiveMargin"` // resolved multiple actually applied (output-only)
	Route           []AdminTier `json:"route"`
}

AdminModel is one SKU as an admin sees it: its mode, whether it is hidden from the catalog, its vision capability, its resale margin (own override + the resolved effective multiple), and its upstream route ladder.

type AdminPrice added in v1.3.1 

type AdminPrice struct {
	In        string `json:"in"`
	Out       string `json:"out"`
	CacheRead string `json:"cacheRead"`
}

AdminPrice is a price pair as exact decimal STRINGS ("0.60"), never floats, so no value drifts crossing this surface. In/Out are $/MTok for chat, or the per-unit price in In (Out mirrors it) for media SKUs.

type AdminProvider added in v1.3.1 

type AdminProvider struct {
	Base       string `json:"base"`
	KeyEnv     string `json:"keyEnv"`
	Kind       string `json:"kind"` // "openai" (default) or "anthropic"
	KeyPresent bool   `json:"keyPresent,omitempty"`
}

AdminProvider is one upstream endpoint. KeyEnv is the NAME of the env var / KMS secret that holds the credential — never the credential itself. KeyPresent is a resolved boolean (is the credential wired?), so the UI can flag an uncredentialed provider without the secret ever crossing this surface.

type AdminTier added in v1.3.1 

type AdminTier struct {
	Provider       string      `json:"provider"`
	Upstream       string      `json:"upstream"` // the SKU → upstream mapping (SuperAdmin only)
	Ctx            int         `json:"ctx"`
	Cost           AdminPrice  `json:"cost"`                     // input: what the upstream charges us ($/MTok, or $/unit for media)
	RetailOverride *AdminPrice `json:"retailOverride,omitempty"` // optional fixed retail; nil = derive cost × margin
	Retail         AdminPrice  `json:"retail"`                   // DERIVED (output-only)
	Profit         AdminPrice  `json:"profit"`                   // DERIVED retail − cost (output-only)
}

AdminTier is one tier of a SKU's context ladder: which upstream serves it, at what context, at what cost, and — derived — at what retail and profit. Cost is the input (what the upstream charges us). Retail/Profit are computed from cost × margin and are output-only (ignored on PUT). RetailOverride sets a fixed retail instead of a cost-relative margin (e.g. to match a hyperscaler's list price); nil means "derive from margin".

type Caps 

type Caps struct {
	Vision bool `yaml:"vision"`
}

Caps declares model capabilities that callers route on (e.g. vision).

type Catalog 

type Catalog struct {
	Providers      map[string]Provider `yaml:"providers"`
	Models         map[string]Model    `yaml:"models"`
	Aliases        map[string]string   `yaml:"aliases"`
	VisionFallback string              `yaml:"vision_fallback"`
	Margin         margin              `yaml:"margin"` // default resale policy (retail = cost × margin)
}

Catalog is the whole zen family as a value: providers (upstream endpoints) and models (SKUs), plus bare-name aliases and the resale margin. It is immutable once loaded.

Margin is the ONE resale-policy knob: retail = wholesale × Margin. Each tier carries only its upstream WHOLESALE cost (what DO charges us); retail is derived at load, never hand-written. So a SKU can never be priced below cost — margin is structural, not a number someone might fat-finger — and raising the whole family's margin is a one-line change. A model may override with its own margin.

func LoadCatalog 

func LoadCatalog(data []byte) (*Catalog, error)

LoadCatalog parses catalog bytes, normalizes every model's route into ascending-context order (so Tier's "smallest that fits" is one scan), and derives each tier's retail from its cost and the resolved margin. Retail is NEVER read from the catalog — it is computed here — so no SKU can be priced below cost.

func (*Catalog) IDs 

func (c *Catalog) IDs() []string

IDs returns the canonical model ids in stable (sorted) order, skipping disabled.

func (*Catalog) Resolve 

func (c *Catalog) Resolve(name string) (id string, m Model, ok bool)

Resolve maps a requested name (SKU or alias) to its canonical id and model.

type Config 

type Config struct {
	Family  string                                       // which family to serve: "zen" | "enso"; "" → ZEN_FAMILY env, else "zen"
	Catalog []byte                                       // catalog bytes; nil → ZEN_CATALOG file if set, else the family's embedded catalog
	Key     func(ctx context.Context, env string) string // credential resolver; nil → os.Getenv
	Tenant  TenantResolver                               // billing identity resolver; nil → headers
	Meter   Meter                                        // usage sink; nil → logMeter (still records)
	Gate    Gate                                         // pre-serve authorization; nil → tenant-required only
	Cache   string                                       // path to persist a SuperAdmin override for warm restart; "" = memory only
	Logger  luxlog.Logger
}

Config configures Zen. Catalog defaults to the embedded catalog.yaml; Key defaults to os.Getenv; Meter defaults to a logging meter (usage is never silently free); Gate is optional (nil = require a valid tenant only). Tenant defaults to a header-reading resolver (X-Org-Id/X-User-Id/X-Project-Id) that bills the caller's own org — correct for a standalone host; a multi-tenant host overrides it to key billing on the validated home org + project scope.

type Escalate added in v1.3.10 

type Escalate struct {
	Mode  string              `yaml:"mode"`  // "adaptive" — the only mode (nil stanza = always fan out)
	Panel int                 `yaml:"panel"` // arms in the escalated panel, probe included; 0 → defaultPanel
	Rank  map[string][]string `yaml:"rank"`  // task → arm upstreams, best-first
}

Escalate is a fan-out SKU's ADAPTIVE-ESCALATION policy. The default fan-out spends every arm on every request (a fixed N× cost floor); adaptive escalation instead PROBES on one task-appropriate arm and escalates only on doubt, collapsing the common case to one arm. It is pure data — the WHEN (the confidence predicate) is code (escalate.go); this only carries WHICH arms and HOW MANY.

Rank is the task→arm-ranking table and the ONE source of both the probe and the panel, so they can never disagree: for a task shape ("default" is required; "code" and friends are optional), it lists the SKU's arm upstreams best-first. The probe is rank[task][0] (the single best arm); the panel is rank[task][:Panel] (the top-K). Every upstream named MUST be one of the SKU's arms — validated at load.

type Gate 

type Gate func(ctx context.Context, t Tenant, model string, est money.Amount) error

Gate authorizes a request BEFORE serving (credit balance, plan entitlement). Returning an error denies it. Composable: cloud wires commerce credit; the standalone server wires its own. A nil Gate means "tenant-required only". The estimate is an exact atto-USD Amount (money.USD), same unit as the ledger. The context is the request's, so a commerce-backed Gate can time out the balance check rather than block the whole upstream call on a slow ledger.

type Meter 

type Meter interface {
	Record(ctx context.Context, u Usage)
}

Meter records served usage for billing + attribution. zen always meters — a nil Meter is replaced by logMeter, so usage is never silently free. The context is the request's; a commerce-backed Meter uses it for timeout/cancellation and to carry the request id into the ledger.

type Model 

type Model struct {
	Caps     Caps   `yaml:"caps"`
	Mode     string `yaml:"mode"`     // "" (chat) | embedding | image | audio | video | rerank
	Disabled bool   `yaml:"disabled"` // true = hidden from catalog (not yet entitled)
	Gated    bool   `yaml:"gated"`    // true = LISTED but access-controlled (limited preview / waitlist); the fronting host enforces the grant
	MinTier  string `yaml:"min_tier"` // OPTIONAL subscription floor: "" (any, incl. free) | free | trial | paid. Rides discovery like Gated but ORTHOGONAL to it (the waitlist grant); the fronting host enforces the tier.
	Margin   margin `yaml:"margin"`   // per-model resale policy; unset = use the catalog default
	Route    []Tier `yaml:"route"`

	// Fan-out SKU (e.g. enso-ultra): when Arms is non-empty the SKU is served by
	// fanning the SAME request to every arm concurrently, then one Synth call folds
	// the candidate answers into a single best answer (see ultra.go). Route[0] stays
	// the SKU's public billing rate + advertised context — Arms/Synth are the path,
	// Route is the meter. An ordinary (single-upstream) SKU leaves both empty.
	Arms  []Tier `yaml:"arms"`
	Synth *Tier  `yaml:"synth"`

	// Escalate (optional, fan-out SKUs only) turns always-fan-out into ADAPTIVE
	// escalation: probe ONE arm first, and fan to a top-K panel + verify-then-select
	// ONLY when the probe is low-confidence. nil = always fan to every arm (the prior
	// behavior). It is the cost knob — probe-only requests bill a single arm — while
	// Route stays the meter. See escalate.go for the confidence gate + ranking.
	Escalate *Escalate `yaml:"escalate"`
}

Model is one zen SKU. route is an ascending-context ladder of upstreams (the per-model ROUTING policy: which backends, in what order, with what overflow); the serving rule picks the smallest tier that holds the prompt and advertises the largest as the SKU's context window. Cost lives on each tier, not the model, so a request bills at the tier that actually served it and an overflow can never underbill. Margin (optional) overrides the catalog default for THIS SKU — the per-model MARGIN control. Retail = cost × margin, derived at load (see setRetail).

func (Model) BasePrice 

func (m Model) BasePrice() Price

BasePrice is the SKU's headline RETAIL price: the cheapest (smallest-context) tier — what a typical in-window request pays. The ladder above it is the overflow.

func (Model) IsUltra added in v1.3.5 

func (m Model) IsUltra() bool

IsUltra reports whether the SKU is a fan-out (arms → synthesize) SKU.

func (Model) MaxCtx 

func (m Model) MaxCtx() int

MaxCtx is the largest context window the SKU can serve — what it advertises.

func (Model) Tier 

func (m Model) Tier(promptTokens int) Tier

Tier chooses the upstream for a prompt of promptTokens: the smallest-context tier that holds it, or the largest when nothing does (best effort — the upstream, not zen, refuses an over-cap prompt). route is ascending by ctx.

type Price 

type Price struct {
	In        decimal.Decimal
	Out       decimal.Decimal
	CacheRead decimal.Decimal
}

Price is $/MTok as EXACT decimals, never float (a float64 could not represent "0.05" and would drift across billions of tokens). Both a tier's Cost (what DO charges us) and its derived Retail (what we charge) are Prices.

CacheRead is the rate for an input token served from the upstream's prompt cache. Every upstream we resell publishes one, and it is cheaper than In; carrying it here is what lets a caller be quoted — and billed — the discount they actually get. A Price that could only say In and Out could not express it, so the public catalog invented cache numbers instead of reading them.

func (*Price) UnmarshalYAML 

func (p *Price) UnmarshalYAML(n *yaml.Node) error

UnmarshalYAML reads {in, out, cache_read: "<decimal>"} as exact decimals. Keeping the YAML seam here lets hanzoai/decimal stay a pure stdlib leaf (no yaml import). cache_read is optional: an upstream that does not cache (or a per-unit media SKU, where the concept does not apply) leaves it zero.

type Provider 

type Provider struct {
	Base   string `yaml:"base"`
	KeyEnv string `yaml:"key_env"`
	Kind   string `yaml:"kind"` // "openai" (default) or "anthropic"
}

Provider is one upstream: where to send, which env var holds the key, and the wire dialect (openai | anthropic).

type Req 

type Req struct {
	Model    string
	Stream   bool
	Messages []json.RawMessage // each a full chat message; content may be text or parts
	// contains filtered or unexported fields
}

Req is the neutral chat request — OpenAI-shaped, the pipeline's lingua franca. Anthropic inbound is translated INTO a Req; every upstream is encoded FROM one. raw carries every top-level field zen does not model, so an OpenAI→OpenAI proxy is byte-lossless (tools, tool_choice, response_format, temperature, … survive). think is lifted OUT of raw at decode and written back at encode by normalizeThinking, the sole place that knows the upstream's thinking vocabulary.

type Tenant 

type Tenant struct {
	Org        string
	User       string
	BillingOrg string
	Project    string
}

Tenant is the validated principal a request is served and billed to. zen does NOT authenticate — it TRUSTS the identity headers the host (cloud IAM) has already validated and re-minted (X-Org-Id / X-User-Id). A request with no Org is unattributable and therefore refused: there is no free, anonymous usage.

Billing is the ONE use of these fields — zen is a stateless proxy to upstreams and does no data-scoping on the org. The fields carry the three billing granularities the ledger supports:

  • BillingOrg — the org that PAYS (home org). The balance is always gated on this. It defaults to Org so a standalone host bills the caller's own org; a multi-tenant host (cloud) sets it from its billing-org header so an admin acting in another org bills the admin's home org, never the org acted on.
  • Project — the project scope WITHIN the org. A project may carry its own billing account (resolved server-side by commerce from the org's project binding); empty is the org-wide default. Spend caps bind on (Project, Service).
  • User — the actor (org/sub) for the audit trail.

The three are independent axes: org pays, project scopes, user is attributed. The same value flows to the Meter for the post-serve debit.

func (Tenant) Valid 

func (t Tenant) Valid() bool

Valid reports whether the tenant can be billed (has an org).

type TenantResolver added in v1.1.0 

type TenantResolver func(c *zip.Ctx) Tenant

TenantResolver derives the billing Tenant from a request. zen never authenticates — it trusts the host's identity middleware to have validated and re-minted the principal. The default reads the standard identity headers (X-Org-Id, X-User-Id, X-Project-Id) and treats the org as its own biller, which is correct for a standalone single-tenant host. A multi-tenant host (cloud) injects a resolver that keys billing on the HOME org (who pays) and resolves the project scope from a validated claim, so an admin acting in another org bills the admin's home org and a project with its own billing account is scoped correctly. One resolver, one billing identity per request.

type Tier 

type Tier struct {
	Provider  string `yaml:"provider"`
	Upstream  string `yaml:"upstream"`
	Ctx       int    `yaml:"ctx"`
	Cost      Price  `yaml:"cost"`   // upstream published price — our cost basis (the fact)
	RetailSet *Price `yaml:"retail"` // OPTIONAL fixed retail override (e.g. match a hyperscaler's list price)
	Retail    Price  `yaml:"-"`      // resolved: RetailSet if given, else cost × margin
}

Tier is one tier of a model's context ladder: a provider, its real model id, that upstream's context window, the upstream's COST ($/MTok DO charges us), and the RETAIL price we bill when THIS tier serves. Cost comes from the catalog; Retail is derived (cost × margin) at load — never hand-written — so no tier is ever priced below cost. Billing keys off the tier that served, so "bill what served" holds.

type Usage 

type Usage struct {
	Tenant           Tenant
	Model            string
	PromptTokens     int // the whole prompt: fresh + cached + cache-write
	CachedTokens     int // of PromptTokens: served from the upstream's prompt cache (billed at cache_read)
	CacheWriteTokens int // of PromptTokens: written INTO the cache this turn (billed at `in`)
	CompletionTokens int
	Charge           money.Amount // retail — what the caller pays (the debit)
	Cost             money.Amount // upstream cost — what we pay DO (for margin)
	Upstream         string
	RequestID        string // internal X-Request-Id (idempotency/audit)
	ResponseID       string // the response `id` the CLIENT sees — the routing/feedback join key
}

Usage is one served request's metered record — the unit of billing and attribution. Both money amounts are EXACT USD (money.USD), tokens × the served tier's rate: the Amount keeps full decimal precision (currency decimals only affect display, never the stored value), so a sub-cent per-token rate summed across billions of tokens never rounds away.

  • Charge is what the caller pays (retail = cost × margin) — the debit.
  • Cost is what WE pay the upstream (DO's published rate). Never surfaced to a caller; recorded so profit = Charge − Cost is exact and tracked per model.

Upstream is the served model id, kept for cost reconciliation. The Tenant carries the billing axes (org pays, project scopes, user attributed) so a Meter debits the same scope the Gate authorized. RequestID is the host's request id (X-Request-Id), carried so a Meter's debit is idempotent and attributable to the audited request.

The token counts describe HOW the prompt was served, which is what makes the prompt-cache discount auditable:

  • PromptTokens is the WHOLE prompt the upstream read, however it was served.
  • CachedTokens and CacheWriteTokens are PARTS OF IT — subsets, never additional to it. A consumer that reads only PromptTokens therefore sees exactly what it saw before these fields existed; one that reads all three can price the split and emit it on the gen_ai span.

PromptTokens − CachedTokens − CacheWriteTokens is the part billed at the full input rate. A nonzero CachedTokens in o11y is the proof that an upstream really does pass prompt-cache data through.

func (Usage) Margin added in v1.3.0 

func (u Usage) Margin() money.Amount

Margin is the exact profit on a usage record: what we charged minus what the upstream cost us. Negative is structurally impossible (retail = cost × margin, margin ≥ 1), so the subtraction's currency-mismatch error cannot arise here (both are USD) and is discarded.

type Zen 

type Zen struct {
	// contains filtered or unexported fields
}

Zen is the loaded serving state: the catalog (source of truth), identity prompts, credential resolver, and the billing seam (meter + gate). It is a value built once and shared read-only across requests.

The catalog is the one mutable cell: a SuperAdmin edit (see admin.go) swaps in a freshly validated *Catalog atomically, so pricing/margin/routing take effect on the running server without a redeploy. It is held behind an atomic.Pointer and read through catalog(); every request takes one consistent snapshot. A *Catalog is never mutated in place — an edit builds a new value and stores it — so a read is always a complete, self-consistent catalog.

func New 

func New(cfg Config) (*Zen, error)

New builds a Zen from config. It fails only if the catalog is invalid — a serving layer with a broken catalog must not start.

func (*Zen) AdminGuard added in v1.3.1 

func (z *Zen) AdminGuard(next zip.Handler) zip.Handler

AdminGuard gates a standalone admin route with a control token held in KMS (ZEN_ADMIN_TOKEN, resolved through the same credential seam as upstream keys). It fails CLOSED: no token configured → the route is invisible (404), a wrong or missing bearer → 403, and the compare is constant-time. A co-resident host does NOT use this — there cloud's edge-minted SuperAdmin bit is the gate, and the token is never set. One gate per deployment, never two live at once.

func (*Zen) AdminSnapshot added in v1.3.1 

func (z *Zen) AdminSnapshot(ctx context.Context) AdminCatalog

AdminSnapshot renders the live catalog as an AdminCatalog. ctx resolves each provider's KeyPresent flag through the credential resolver (never the value). This is the in-process read the host's admin handler returns as JSON.

func (*Zen) AdminUpdate added in v1.3.1 

func (z *Zen) AdminUpdate() zip.Handler

AdminUpdate returns the PUT handler: replace the live catalog with the posted one. A body that fails the resale law is rejected 400 with the exact reason and the live catalog is left untouched; on success it returns the new snapshot so the caller sees the derived retail/profit immediately.

func (*Zen) AdminView added in v1.3.1 

func (z *Zen) AdminView() zip.Handler

AdminView returns the GET handler: the full catalog as an admin sees it. The host gates it (cloud's SuperAdmin bit, or AdminGuard standalone).

func (*Zen) ApplyAdmin added in v1.3.1 

func (z *Zen) ApplyAdmin(in AdminCatalog) error

ApplyAdmin replaces the live catalog with an edited one: it ingests the DTO into a *Catalog, runs the resale law (normalize), swaps it in atomically, and persists it to the warm-restart cache if one is configured. A catalog that violates the law (a below-cost SKU, an unknown provider, an unpriced tier) is rejected whole — the live catalog is untouched. This is the one write path; the HTTP handler and the co-resident host both go through it.

func (*Zen) Claim 

func (z *Zen) Claim() zip.Handler

Claim returns the middleware a host mounts (app.Use or app.Group) to route zen* models to zen in-process and pass everything else through. It is zen's single decision point: read the model from the body, serve it if zen's catalog resolves it, else c.Next(). See proxy.go for the decision.

func (*Zen) Health 

func (z *Zen) Health() zip.Handler

Health returns the GET /health handler (liveness). Exported for a standalone host; co-resident hosts use the cloud binary's own liveness probe.

func (*Zen) Model 

func (z *Zen) Model() zip.Handler

Model returns the GET /v1/models/:id handler (one zen SKU or alias).

func (*Zen) Models 

func (z *Zen) Models() zip.Handler

Models returns the GET /v1/models handler (the zen family as an OpenAI-shaped list). Exported for a standalone host that wants zen's catalog as its model list; co-resident hosts do NOT mount this (ai owns /v1/models there).

func (*Zen) Reload added in v1.3.1 

func (z *Zen) Reload(data []byte) error

Reload replaces the live catalog from raw catalog YAML (the catalog.yaml format), for a gitops/file-driven edit. It runs the same resale law as boot (LoadCatalog) and swaps atomically. The JSON admin editor uses ApplyAdmin; this is its YAML sibling — one validation law behind both.

Source Files

View all Source files

Directories

Path Synopsis
cmd
zen command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.