compat

package
v0.32.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 5, 2026 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package compat is the CLI's ONLY turn transport: a stdlib-only HTTP client speaking OpenAI-compat chat/completions — streaming SSE, tools + forced tool_choice, image/file parts — against any base URL: the memcode gateway at {api}/v1 (Memcode=true: the optional extensions ride) or any arbitrary compat endpoint (Memcode=false: pure standard wire). The request shape is IDENTICAL either way — no memcode headers exist; model selection arrives already made (req.Pin, stamped by cli/internal/llm's policy). The transport itself carries no endpoint or routing knowledge.

Package compatwire is the OpenAI-compat chat/completions dialect — the ONE set of wire-type declarations (used by the CLI transport, the gateway's inbound surface, the lane client, and the conformance suite) and the ONE client engine for the dialect (encode/decode/SSE/retry, with the optional memcode extensions, tool-call salvage, and the lane error contract as configuration). Extracted from the gateway's internal/compat (types) + the CLI's transport + the gateway's Fireworks lane client — one implementation per protocol, shared by every consumer. Package compat is the OpenAI-compat wire: the chat-completions request/ response/chunk shapes the gateway's {prefix}/chat/completions surface speaks, plus the pure translation between that wire and the internal common.Request/ common.Response protocol (translate.go).

This IS the one-wire architecture (plans/flickering-soaring-falcon): the memcode base URL behaves exactly like an OpenAI-compatible endpoint — an ordinary OpenAI client pointed at https://api.memcode.ai/v1 works with zero memcode-specific transport branches. The turn surface is POST /v1/chat/completions + GET /v1/models; there is no memcode-shaped turn wire anymore. The memcode extensions are all optional and ignorable by third-party tooling:

  1. system-messages convention: first system message = cacheable stable prefix, second = volatile suffix (3+ concatenate into volatile);
  2. `memcode_billing` on the request — the billing-lane the gateway ENFORCES (byok_preferred | byok_only | credits; never chosen server-side). The standard `user` field carries session/cache affinity;
  3. assistant messages may carry a memcode_opaque array (vendor reasoning blocks round-tripped verbatim; the gateway re-expands them);
  4. a `memcode` object on the final response/chunk (byok, fallback_reason, search_count, context_window, input_budget, pool, session_phase);
  5. a `memcode` object per GET /v1/models entry + on the list itself — the routing CONTROL PLANE the CLI's selection policy runs on (vendor, capabilities, byok coverage, credits_exhausted, vendors, roles).

The same types serve both directions: the gateway decodes inbound requests with them, and the conformance suite (compat/conformance) marshals outbound requests with them against arbitrary endpoints — so a shape drift from the real ecosystem fails conformance, not production.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type ChatChunk 

type ChatChunk struct {
	ID      string        `json:"id"`
	Object  string        `json:"object"` // "chat.completion.chunk"
	Created int64         `json:"created"`
	Model   string        `json:"model"`
	Choices []ChunkChoice `json:"choices"`
	Usage   *Usage        `json:"usage,omitempty"`
	Memcode *MemcodeExt   `json:"memcode,omitempty"`
}

ChatChunk is one SSE `data:` payload of a streamed completion. The final usage chunk carries empty choices + Usage (+ the memcode extension), then the stream terminates with `data: [DONE]`.

type ChatMessage 

type ChatMessage struct {
	Role    string         `json:"role"`
	Content MessageContent `json:"content,omitzero"`

	// assistant
	ToolCalls []ToolCall `json:"tool_calls,omitempty"`
	// MemcodeOpaque is extension (3): vendor reasoning blocks (Anthropic
	// thinking signatures, OpenAI rs_ items) round-tripped verbatim. Each
	// element is one common.Block in its wire form; the gateway re-expands
	// them ahead of the message's text/tool_use blocks.
	MemcodeOpaque []json.RawMessage `json:"memcode_opaque,omitempty"`

	// tool
	ToolCallID string `json:"tool_call_id,omitempty"`
	Name       string `json:"name,omitempty"`
}

ChatMessage is one request-side message. Roles: system | developer (treated as system) | user | assistant | tool.

type ChatRequest 

type ChatRequest struct {
	Model    string        `json:"model"`
	Messages []ChatMessage `json:"messages"`

	Tools []Tool `json:"tools,omitempty"`
	// ToolChoice is the standard union: "auto" | "none" | "required" |
	// {"type":"function","function":{"name":…}} — kept raw and interpreted in
	// translate.go (the forced-tool form is what the classifiers depend on).
	ToolChoice json.RawMessage `json:"tool_choice,omitempty"`

	Stream        bool           `json:"stream,omitempty"`
	StreamOptions *StreamOptions `json:"stream_options,omitempty"`

	// MaxCompletionTokens is the current spelling; MaxTokens the deprecated one.
	// The newer field wins when both are set.
	MaxTokens           int `json:"max_tokens,omitempty"`
	MaxCompletionTokens int `json:"max_completion_tokens,omitempty"`

	// ReasoningEffort maps onto the abstract common.Effort (the thinking knob).
	ReasoningEffort string `json:"reasoning_effort,omitempty"`

	// User is the standard end-user/session affinity field — mapped onto
	// Request.Session (Fireworks sticky routing already keys on `user`).
	User string `json:"user,omitempty"`

	// MemcodeBilling is the billing-lane extension (memcode backend only):
	// "" | "byok_preferred" (default — the user's key serves when present, else
	// credits), "byok_only" (fail if the serving vendor isn't user-keyed —
	// never touch credits), "credits" (skip BYOK injection; an explicit,
	// consented, debited serve — the CLI's "retry this turn on credits" path).
	// The gateway ENFORCES the lane; it never chooses one: byok-preferred
	// serving still never falls back to credits server-side (the doctrine's
	// no-silent-billing invariant, now enforcement rather than policy).
	MemcodeBilling string `json:"memcode_billing,omitempty"`

	// Accepted-and-ignored standard fields (the gateway owns sampling; one
	// choice is always served). Declared so intent is documented, not enforced.
	Temperature *float64 `json:"temperature,omitempty"`
	TopP        *float64 `json:"top_p,omitempty"`
	N           int      `json:"n,omitempty"`
}

ChatRequest is POST {prefix}/chat/completions. Decode is deliberately loose: standard knobs the gateway doesn't honor (temperature, top_p, n, …) are accepted and ignored — the gateway owns sampling — rather than rejected.

type ChatResponse 

type ChatResponse struct {
	ID      string   `json:"id"`
	Object  string   `json:"object"` // "chat.completion"
	Created int64    `json:"created"`
	Model   string   `json:"model"`
	Choices []Choice `json:"choices"`
	Usage   *Usage   `json:"usage,omitempty"`
	// Memcode is extension (4) — loose decoders ignore it.
	Memcode *MemcodeExt `json:"memcode,omitempty"`
}

ChatResponse is the non-streamed chat completion.

type Choice 

type Choice struct {
	Index        int             `json:"index"`
	Message      ResponseMessage `json:"message"`
	FinishReason string          `json:"finish_reason"`
}

Choice is one completion choice (the gateway always serves exactly one).

type ChunkChoice 

type ChunkChoice struct {
	Index        int     `json:"index"`
	Delta        Delta   `json:"delta"`
	FinishReason *string `json:"finish_reason"`
}

ChunkChoice is one delta frame.

type Config 

type Config struct {
	// Compose, when set, renders a legacy-shaped side call (Mode stamped,
	// doctrine not yet composed) into the two-system form before encoding. The
	// CALLER owns doctrine — the CLI passes its renderer; server-side lane use
	// leaves it nil.
	Compose func(wire.Request) (wire.Request, error)
	// Salvage enables the tool-call salvage net for small open models that
	// emit tool calls as TEXT (wrapped or bare JSON) instead of the structured
	// envelope — plus the MiniMax leak strip. The gateway's cheap lane runs
	// with this on; leave off for well-behaved backends.
	Salvage bool
	// Lane selects the SERVER-SIDE lane error contract (LaneRequestError for
	// request-shaped 4xx, generic errors otherwise) instead of the client
	// sentinel mapping, plus the lane's model-conditional reasoning_effort
	// vocabulary and the legacy max_tokens spelling.
	Lane bool

	// BaseURL is the FULL compat base including any path prefix,
	// e.g. https://code.memcode.ai/v1 — {base}/chat/completions is the
	// turn endpoint.
	BaseURL string
	// Token is the bearer credential; "" sends no Authorization header (a
	// keyless local endpoint).
	Token string
	// Memcode enables the memcode-backend extensions: memcode_opaque reasoning
	// round-trip (attach AND re-extract) and the memcode response object.
	// False = pure standard wire (arbitrary endpoints).
	Memcode bool
	// Model is the endpoint's session model, used when a request doesn't pin
	// one (Request.Pin). Arbitrary endpoints have no Automatic — every request
	// must name a concrete model — so with Memcode false and neither a pin nor
	// this default, calls fail with an actionable error instead of sending the
	// gateway sentinel "auto" to an endpoint that can't serve it.
	Model string
	// Headers are extra request headers sent on every turn — the identity a
	// subscription backend requires to accept the request (e.g. a Copilot
	// endpoint's Editor-Version / Copilot-Integration-Id). Applied after the
	// standard headers. Empty for a plain endpoint.
	Headers map[string]string
	// HTTPClient overrides the default client (tests, custom timeouts).
	HTTPClient *http.Client
}

Config configures a Transport.

type ContentPart 

type ContentPart struct {
	Type     string        `json:"type"` // "text" | "image_url" | "file"
	Text     string        `json:"text,omitempty"`
	ImageURL *ImageURLPart `json:"image_url,omitempty"`
	File     *FilePart     `json:"file,omitempty"`
}

ContentPart is one element of the array content form.

func TextPart 

func TextPart(s string) ContentPart

TextPart builds a text content part.

type Delta 

type Delta struct {
	Role          string            `json:"role,omitempty"`
	Content       *string           `json:"content,omitempty"`
	ToolCalls     []ToolCallDelta   `json:"tool_calls,omitempty"`
	MemcodeOpaque []json.RawMessage `json:"memcode_opaque,omitempty"`
}

Delta is the incremental message fragment of a chunk.

type ErrorBody 

type ErrorBody struct {
	Message string `json:"message"`
	Type    string `json:"type,omitempty"`
	Code    string `json:"code,omitempty"`
}

ErrorBody is the standard error object. Code carries the machine-readable memcode codes ("unknown_model", "context_overflow", …) the CLI keys on.

type ErrorResponse 

type ErrorResponse struct {
	Error ErrorBody `json:"error"`
}

ErrorResponse is the standard error envelope: {"error":{...}}.

type FilePart 

type FilePart struct {
	FileData string `json:"file_data,omitempty"`
	FileID   string `json:"file_id,omitempty"`
	Filename string `json:"filename,omitempty"`
}

FilePart carries a document input (OpenAI's own `file` content part). The gateway accepts inline file_data data: URLs; file_id has no file store behind it and is rejected.

type FunctionCall 

type FunctionCall struct {
	Name      string `json:"name"`
	Arguments string `json:"arguments"`
}

FunctionCall is a call's name + JSON-encoded arguments string.

type FunctionDef 

type FunctionDef struct {
	Name        string         `json:"name"`
	Description string         `json:"description,omitempty"`
	Parameters  map[string]any `json:"parameters,omitempty"`
	Strict      *bool          `json:"strict,omitempty"`
}

FunctionDef is the function payload of a tool definition.

type ImageURLPart 

type ImageURLPart struct {
	URL    string `json:"url"`
	Detail string `json:"detail,omitempty"`
}

ImageURLPart carries a vision input. The gateway accepts data: URLs only (it never fetches remote images on the user's behalf).

type LaneRequestError 

type LaneRequestError struct {
	Status   int
	Message  string // full server message, never clipped
	Overflow bool   // true when the rejection is a context-length overflow
}

LaneRequestError is a non-retryable 4xx from an OpenAI-compatible lane: the request itself is the problem (malformed, or — most commonly — longer than the served context window), NOT the server. Distinguish from 5xx/timeout, which mean unhealthy.

func (*LaneRequestError) Error 

func (e *LaneRequestError) Error() string

type MemcodeExt 

type MemcodeExt struct {
	Byok           bool   `json:"byok,omitempty"`
	FallbackReason string `json:"fallback_reason,omitempty"`
	SearchCount    int    `json:"search_count,omitempty"`
	ContextWindow  int    `json:"context_window,omitempty"`
	InputBudget    int    `json:"input_budget,omitempty"`
	Pool           string `json:"pool,omitempty"`
}

MemcodeExt is extension (4): the response metadata the CLI footer/compaction feed on, attached to the final body/chunk. All fields optional; third-party clients never need it.

type MessageContent 

type MessageContent struct {
	Text    string
	Parts   []ContentPart
	IsParts bool
}

MessageContent is the standard `content` union: a plain string or an array of typed parts. IsParts records which form arrived (and which to emit).

func PartsContent 

func PartsContent(parts ...ContentPart) MessageContent

PartsContent builds the array form.

func StringContent 

func StringContent(s string) MessageContent

StringContent builds the plain-string form.

func (MessageContent) IsZero 

func (m MessageContent) IsZero() bool

IsZero makes `content` omittable (omitzero) for messages that carry only tool_calls.

func (MessageContent) MarshalJSON 

func (m MessageContent) MarshalJSON() ([]byte, error)

func (*MessageContent) UnmarshalJSON 

func (m *MessageContent) UnmarshalJSON(b []byte) error

type ModelEntry 

type ModelEntry struct {
	ID      string     `json:"id"`
	Object  string     `json:"object"` // "model"
	Created int64      `json:"created,omitempty"`
	OwnedBy string     `json:"owned_by,omitempty"`
	Memcode *ModelMeta `json:"memcode,omitempty"`
}

ModelEntry is one listed model. The ids are the catalog LABELS — raw provider ids never leave the server.

type ModelList 

type ModelList struct {
	Object string       `json:"object"` // "list"
	Data   []ModelEntry `json:"data"`
	// Memcode is the list-level extension: org/routing facts that aren't
	// per-model. Strict OpenAI clients decode {object,data} and ignore it.
	Memcode *ModelsExt `json:"memcode,omitempty"`
}

ModelList is GET {prefix}/models: the standard list shape, extended with an ignorable top-level `memcode` object (extension 5) so one call carries everything the CLI's /model picker needs.

type ModelMeta 

type ModelMeta struct {
	Name      string `json:"name,omitempty"`
	Desc      string `json:"desc,omitempty"`   // one-line picker description
	Group     string `json:"group,omitempty"`  // display family — presentation only
	Vendor    string `json:"vendor,omitempty"` // authoritative serving vendor — the selection/steering identity
	Window    int    `json:"window,omitempty"`
	Vision    bool   `json:"vision,omitempty"`
	PDF       bool   `json:"pdf,omitempty"`       // native PDF/document input — the document-turn pre-check
	Reasoning bool   `json:"reasoning,omitempty"` // exposes a thinking/reasoning knob
	Pinnable  bool   `json:"pinnable,omitempty"`  // offered in the /model picker (serving accepts every listed label)
	// Byok marks a model served by a vendor the requesting user brought their
	// own key for.
	Byok bool `json:"byok,omitempty"`
}

ModelMeta is the ignorable per-model extension. This is the hosted ROUTING CONTROL PLANE (all-policy-client-side): every server-side fact the CLI's selection policy reads must appear here — anything missing gets added explicitly, never smuggled back into gateway routing.

type ModelsExt 

type ModelsExt struct {
	// CreditsExhausted reports the org's empty-wallet state so the CLI can
	// frame BYOK-only routing honestly.
	CreditsExhausted bool `json:"credits_exhausted"`
	// Backend names the gateway's provider mode ("hybrid" in prod).
	Backend string `json:"backend,omitempty"`
	// Vendors lists the strong-tier vendors the gateway has keys for — the
	// /model vendor selector's roster.
	Vendors []string `json:"vendors,omitempty"`
}

ModelsExt is the list-level memcode extension on GET {prefix}/models.

type PromptTokensDetails 

type PromptTokensDetails struct {
	CachedTokens int `json:"cached_tokens"`
}

PromptTokensDetails carries the cached-token subset of prompt_tokens.

type ResponseMessage 

type ResponseMessage struct {
	Role          string            `json:"role"`
	Content       *string           `json:"content"`
	ToolCalls     []ToolCall        `json:"tool_calls,omitempty"`
	MemcodeOpaque []json.RawMessage `json:"memcode_opaque,omitempty"`
}

ResponseMessage is the assistant message of a completion. Content is null (not "") when the message is tool-calls only, per the standard shape.

type StreamOptions 

type StreamOptions struct {
	IncludeUsage bool `json:"include_usage,omitempty"`
}

StreamOptions is the standard stream_options object. The gateway always sends the final usage chunk (a superset of include_usage:false — clients that did not ask simply ignore the extra chunk).

type Tool 

type Tool struct {
	Type     string      `json:"type"` // "function"
	Function FunctionDef `json:"function"`
}

Tool is the standard function-tool definition envelope.

type ToolCall 

type ToolCall struct {
	ID       string       `json:"id"`
	Type     string       `json:"type"` // "function"
	Function FunctionCall `json:"function"`

	// MemcodeSignature carries opaque provider state that belongs to THIS call
	// and must come back verbatim when the call is replayed — Gemini issues a
	// thoughtSignature with every functionCall and rejects the replay with a 400
	// without it. The standard tool_calls shape has nowhere to put that, so it
	// rides a namespaced extension, the same way reasoning blocks ride
	// memcode_opaque. Ignored by any server that does not know it.
	MemcodeSignature string `json:"memcode_signature,omitempty"`
}

ToolCall is one function call (request-side history and response-side output).

type ToolCallDelta 

type ToolCallDelta struct {
	Index    int           `json:"index"`
	ID       string        `json:"id,omitempty"`
	Type     string        `json:"type,omitempty"`
	Function *FunctionCall `json:"function,omitempty"`

	// MemcodeSignature is the streaming half of ToolCall.MemcodeSignature —
	// sent once on the delta that opens the call.
	MemcodeSignature string `json:"memcode_signature,omitempty"`
}

ToolCallDelta is a streamed tool-call fragment, accumulated by Index.

type Transport 

type Transport struct {
	// contains filtered or unexported fields
}

Transport speaks the compat wire to one endpoint. Safe for concurrent use after construction (SetRetryNotify is wiring-time, like the SDK client's).

func New 

func New(cfg Config) *Transport

New returns a Transport for one compat endpoint.

func (*Transport) Complete 

func (t *Transport) Complete(ctx context.Context, r wire.Request) (wire.Response, error)

Complete runs one non-streamed turn.

func (*Transport) SetRetryNotify 

func (t *Transport) SetRetryNotify(fn func(attempt int, err error, delay time.Duration))

SetRetryNotify wires a retry-notify callback after construction — the same seam the runtime uses on the legacy client to surface "⊙ retrying…".

func (*Transport) Stream 

func (t *Transport) Stream(ctx context.Context, r wire.Request, h wire.StreamHandler) (wire.Response, error)

Stream runs one streamed turn, forwarding deltas to h and returning the assembled Response — the same contract as every provider's Stream.

type Usage 

type Usage struct {
	PromptTokens        int                  `json:"prompt_tokens"`
	CompletionTokens    int                  `json:"completion_tokens"`
	TotalTokens         int                  `json:"total_tokens"`
	PromptTokensDetails *PromptTokensDetails `json:"prompt_tokens_details,omitempty"`
}

Usage is the standard usage object. NOTE the semantics conversion: the internal protocol counts Anthropic-style (input_tokens EXCLUDES cache reads/writes), while prompt_tokens here INCLUDES them, with the cache-read subset reported under prompt_tokens_details.cached_tokens.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.