core

package
v0.1.56 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 21, 2026 License: MIT Imports: 20 Imported by: 0

Documentation

Overview

Package core provides core types and interfaces for the LLM gateway.

Package core defines the core interfaces and types for the LLM gateway.

Index

Constants

View Source 
const (
	// BatchActionCreate represents POST /v1/batches.
	BatchActionCreate = "create"
	// BatchActionList represents GET /v1/batches.
	BatchActionList = "list"
	// BatchActionGet represents GET /v1/batches/{id}.
	BatchActionGet = "get"
	// BatchActionCancel represents POST /v1/batches/{id}/cancel.
	BatchActionCancel = "cancel"
	// BatchActionResults represents GET /v1/batches/{id}/results.
	BatchActionResults = "results"
	// BatchActionDelete represents DELETE /v1/messages/batches/{id}
	// (Anthropic Message Batches dialect; the OpenAI dialect has no delete).
	BatchActionDelete = "delete"
)
View Source 
const (
	// FileActionCreate represents POST /v1/files.
	FileActionCreate = "create"
	// FileActionList represents GET /v1/files.
	FileActionList = "list"
	// FileActionGet represents GET /v1/files/{id}.
	FileActionGet = "get"
	// FileActionDelete represents DELETE /v1/files/{id}.
	FileActionDelete = "delete"
	// FileActionContent represents GET /v1/files/{id}/content.
	FileActionContent = "content"
)
View Source 
const (
	// ModelPricingSourceModelRegistry identifies pricing data from the model registry.
	ModelPricingSourceModelRegistry = "model_registry"
	// ModelPricingSourceConfigYAML identifies pricing data from config.yaml.
	ModelPricingSourceConfigYAML = "config_yaml"
)
View Source 
const ConversationDeletedObject = "conversation.deleted"

ConversationDeletedObject is the value of the "object" field returned by DELETE /v1/conversations/{id}.

View Source 
const ConversationObject = "conversation"

ConversationObject is the value of the "object" field on a conversation.

View Source 
const (
	// MaxConversationInitialItems caps the items array accepted by
	// POST /v1/conversations.
	MaxConversationInitialItems = 20
)

Conversation limits mirror the OpenAI Conversations API so the gateway keeps an OpenAI-compatible public contract.

View Source 
const UserPathHeader = "X-GoModel-User-Path"

Variables

View Source 
var ErrNativeBatchDeleteUnsupported = errors.New("native batch deletion is not supported by this provider")

ErrNativeBatchDeleteUnsupported reports that a provider's upstream batch API has no delete operation. Callers fall back to gateway-local deletion.

Functions

func ApplyBodySelectorHints 

func ApplyBodySelectorHints(env *WhiteBoxPrompt, model, provider string, stream bool)

ApplyBodySelectorHints records selector hints parsed from a request body. The hints are intentionally sparse and best-effort; canonical request decode remains authoritative for translated JSON requests.

func CacheFileRouteInfo 

func CacheFileRouteInfo(env *WhiteBoxPrompt, req *FileRouteInfo)

CacheFileRouteInfo stores sparse file route metadata on the request semantics.

func CachePassthroughRouteInfo 

func CachePassthroughRouteInfo(env *WhiteBoxPrompt, req *PassthroughRouteInfo)

CachePassthroughRouteInfo stores typed passthrough route metadata on the request semantics.

func CloneRawJSON 

func CloneRawJSON(raw json.RawMessage) json.RawMessage

func DecodeCanonicalSelector 

func DecodeCanonicalSelector(body []byte, env *WhiteBoxPrompt) (model, provider string, ok bool)

DecodeCanonicalSelector decodes a canonical request body using the codec resolved by canonicalOperationCodecFor for env, then extracts the model and provider via semanticSelectorFromCanonicalRequest. It returns ok=false for a nil env, missing codec, or decode failure.

func DispatchDecodedBatchItem 

func DispatchDecodedBatchItem[T any](decoded *DecodedBatchItemRequest, handlers DecodedBatchItemHandlers[T]) (T, error)

DispatchDecodedBatchItem routes a decoded batch item to the matching typed handler based on its canonical request payload.

func EnsureModel 

func EnsureModel(model *string, requested string)

EnsureModel sets *model to the requested model when a provider response omits it, keeping responses OpenAI-compatible.

func ExtractTextContent 

func ExtractTextContent(content any) string

ExtractTextContent returns the textual portion of request content. Structured content parts are reduced to their text components only.

func GetAuthKeyID 

func GetAuthKeyID(ctx context.Context) string

GetAuthKeyID retrieves the managed auth key id from the context.

func GetEffectiveUserPath 

func GetEffectiveUserPath(ctx context.Context) string

GetEffectiveUserPath retrieves the effective user path override from context.

func GetEnforceReturningUsageData 

func GetEnforceReturningUsageData(ctx context.Context) bool

GetEnforceReturningUsageData reports whether the request should ask providers to include usage in streaming responses when possible.

func GetFailoverUsed 

func GetFailoverUsed(ctx context.Context) bool

GetFailoverUsed reports whether the request was served by a failover model.

func GetGuardrailsHash 

func GetGuardrailsHash(ctx context.Context) string

GetGuardrailsHash retrieves the guardrails hash from the context. Returns empty string when no guardrails are active or the hash has not been set.

func GetRequestID 

func GetRequestID(ctx context.Context) string

GetRequestID retrieves the request ID from the context. Returns empty string if not found.

func HasStructuredContent 

func HasStructuredContent(content any) bool

HasStructuredContent reports whether the content uses the array form.

func IsCredentialHeader 

func IsCredentialHeader(name string) bool

IsCredentialHeader reports whether the header name carries credentials. Matching is case-insensitive and ignores surrounding whitespace.

func IsJSONNull 

func IsJSONNull(trimmed []byte) bool

CloneRawJSON returns a detached copy of a raw JSON value. IsJSONNull reports whether trimmed JSON data is empty or the null literal.

func IsModelInteractionPath 

func IsModelInteractionPath(path string) bool

IsModelInteractionPath reports whether a path is a model/provider interaction route.

func MergeLabels 

func MergeLabels(sets ...[]string) []string

MergeLabels combines label sets in order into one list, trimming whitespace and dropping empty values and duplicates. Returns nil when nothing remains.

func NormalizeEmbeddingEncoding 

func NormalizeEmbeddingEncoding(resp *EmbeddingResponse, encodingFormat string)

NormalizeEmbeddingEncoding reconciles a response's embedding encoding with the encoding_format the client requested, keeping responses OpenAI-compatible regardless of provider quirks.

The OpenAI Python and JS/LangChain SDKs request encoding_format="base64" by default and decode it client-side. Some OpenAI-compatible servers (notably LM Studio) ignore encoding_format and always return float arrays, which makes those SDKs mis-decode the floats as packed bytes and produce corrupted, wrong-dimension vectors. Following Postel's Law, GoModel accepts whatever the upstream returns and re-encodes each vector into the format the caller asked for: base64 (little-endian float32, matching OpenAI) or a float array.

An empty or unrecognized format is treated as "float" (the OpenAI default when the field is omitted). Vectors already in the requested form, and values that don't parse as either shape, are left untouched.

func NormalizeMessageContent 

func NormalizeMessageContent(content any) (any, error)

NormalizeMessageContent validates dynamic content and returns its canonical form.

func NormalizeOperationPath 

func NormalizeOperationPath(raw string) string

NormalizeOperationPath returns a stable path-only form for model-facing endpoints.

func NormalizeUserPath 

func NormalizeUserPath(raw string) (string, error)

NormalizeUserPath canonicalizes one user hierarchy path from request ingress.

func ParseProviderPassthroughPath 

func ParseProviderPassthroughPath(path string) (provider string, endpoint string, ok bool)

ParseProviderPassthroughPath extracts provider and endpoint from /p/{provider}/{endpoint...}.

func PrimaryRouteSaturated 

func PrimaryRouteSaturated(ctx context.Context) error

PrimaryRouteSaturated returns the rate-limit rejection recorded for the resolved primary route, or nil when the route has capacity.

func RequestLabelsFromContext 

func RequestLabelsFromContext(ctx context.Context) []string

RequestLabelsFromContext returns the labels extracted for this request. Callers must treat the returned slice as read-only.

func ResolveBatchItemEndpoint 

func ResolveBatchItemEndpoint(defaultEndpoint, itemURL string) string

ResolveBatchItemEndpoint prefers an inline item URL and otherwise falls back to the batch default endpoint.

func RewriteTokensSavedFromContext 

func RewriteTokensSavedFromContext(ctx context.Context) int

RewriteTokensSavedFromContext retrieves the request's rewrite savings estimate, or zero when no rewriter reported savings.

func SpeechResponseContentType 

func SpeechResponseContentType(format string) string

SpeechResponseContentType maps a text-to-speech response_format to its MIME type. An unset format defaults to mp3, matching OpenAI's default.

func TaggingStripHeadersFromContext 

func TaggingStripHeadersFromContext(ctx context.Context) map[string]struct{}

TaggingStripHeadersFromContext returns the canonical header names marked as do-not-pass by the tagging configuration. Callers must treat the returned map as read-only.

func TranscriptionResponseContentType 

func TranscriptionResponseContentType(format string) string

TranscriptionResponseContentType maps a transcription response_format to its MIME type. json and verbose_json (and an unset format) are JSON; text, srt and vtt are plain text.

func UnmarshalMessageContent 

func UnmarshalMessageContent(data []byte) (any, error)

UnmarshalMessageContent decodes supported chat message content payloads. Chat content accepts plain strings, null, or arrays of supported content parts.

func UserPathAncestors 

func UserPathAncestors(path string) []string

UserPathAncestors returns deepest-to-root path fallback candidates.

func UserPathFromContext 

func UserPathFromContext(ctx context.Context) string

UserPathFromContext returns the canonical request user path when available.

func UserPathHeaderName 

func UserPathHeaderName(raw string) string

UserPathHeaderName canonicalizes the configured user-path header name.

func UserPathHeaderNameFromContext 

func UserPathHeaderNameFromContext(ctx context.Context) string

UserPathHeaderNameFromContext returns the request-scoped user-path header name, falling back to the default public header.

func ValidInputAudioPayload 

func ValidInputAudioPayload(data, format string) bool

ValidInputAudioPayload reports whether an input_audio payload satisfies the contract: data is always required, and format may be omitted only when data is a data: URI that already carries an explicit media type (used by providers such as Xiaomi MiMo ASR).

func WithAuthKeyID 

func WithAuthKeyID(ctx context.Context, id string) context.Context

WithAuthKeyID returns a new context with the authenticated managed auth key id attached.

func WithBatchPreparationMetadata 

func WithBatchPreparationMetadata(ctx context.Context, metadata *BatchPreparationMetadata) context.Context

WithBatchPreparationMetadata returns a new context with batch preprocessing metadata attached.

func WithEffectiveUserPath 

func WithEffectiveUserPath(ctx context.Context, userPath string) context.Context

WithEffectiveUserPath returns a new context with an effective user path override attached.

func WithEnforceReturningUsageData 

func WithEnforceReturningUsageData(ctx context.Context, enforce bool) context.Context

WithEnforceReturningUsageData returns a new context with the streaming usage policy attached.

func WithFailoverUsed 

func WithFailoverUsed(ctx context.Context) context.Context

WithFailoverUsed returns a new context marked as having used a failover model.

func WithGuardrailsHash 

func WithGuardrailsHash(ctx context.Context, hash string) context.Context

WithGuardrailsHash returns a new context with the guardrails hash attached. The hash is the SHA-256 of all applied guardrail rule IDs and their versions, computed post-patch in the translated inference handlers.

func WithPrimaryRouteSaturated 

func WithPrimaryRouteSaturated(ctx context.Context, err error) context.Context

WithPrimaryRouteSaturated marks the resolved primary route as rate-saturated. The stored error is the 429 the client would have received; dispatch uses it as the synthetic primary failure that triggers the failover sweep, and it surfaces unchanged when no failover target can take the request.

func WithRequestID 

func WithRequestID(ctx context.Context, requestID string) context.Context

WithRequestID returns a new context with the request ID attached.

func WithRequestLabels 

func WithRequestLabels(ctx context.Context, labels []string) context.Context

WithRequestLabels returns a new context with the request labels attached. Labels are extracted at ingress from configured tagging headers.

func WithRequestOrigin 

func WithRequestOrigin(ctx context.Context, origin RequestOrigin) context.Context

WithRequestOrigin returns a new context with the logical request origin attached.

func WithRequestSnapshot 

func WithRequestSnapshot(ctx context.Context, snapshot *RequestSnapshot) context.Context

WithRequestSnapshot returns a new context with the request snapshot attached.

func WithRewriteTokensSaved 

func WithRewriteTokensSaved(ctx context.Context, tokensSaved int) context.Context

WithRewriteTokensSaved returns a new context carrying the total prompt tokens that applied request rewriters estimate they removed. Non-positive totals leave the context unchanged.

func WithTaggingStripHeaders 

func WithTaggingStripHeaders(ctx context.Context, headers map[string]struct{}) context.Context

WithTaggingStripHeaders returns a new context carrying the canonical tagging header names that must not be forwarded to upstream providers.

func WithUserPathHeaderName 

func WithUserPathHeaderName(ctx context.Context, headerName string) context.Context

WithUserPathHeaderName returns a new context with a non-default configured user-path request header name attached. The default header is intentionally a no-op and does not clear an existing value.

func WithWhiteBoxPrompt 

func WithWhiteBoxPrompt(ctx context.Context, prompt *WhiteBoxPrompt) context.Context

WithWhiteBoxPrompt returns a new context with the white-box prompt attached.

func WithWorkflow 

func WithWorkflow(ctx context.Context, workflow *Workflow) context.Context

WithWorkflow returns a new context with the workflow attached.

Types

type AudioProvider 

type AudioProvider interface {
	CreateSpeech(ctx context.Context, req *AudioSpeechRequest) (*AudioResponse, error)
	CreateTranscription(ctx context.Context, req *AudioTranscriptionRequest) (*AudioResponse, error)
}

AudioProvider is implemented by providers that support OpenAI-compatible audio endpoints: text-to-speech (CreateSpeech) and speech-to-text (CreateTranscription). It is optional so providers without audio support can omit it.

type AudioResponse 

type AudioResponse struct {
	ContentType string
	Data        []byte
}

AudioResponse wraps an opaque audio or transcription payload with its content type. Speech returns binary audio; transcription returns JSON or text depending on response_format. In both cases the gateway proxies the bytes verbatim.

type AudioSpeechRequest 

type AudioSpeechRequest struct {
	Model          string  `json:"model"`
	Input          string  `json:"input"`
	Voice          string  `json:"voice"`
	Instructions   string  `json:"instructions,omitempty"`
	ResponseFormat string  `json:"response_format,omitempty"`
	Speed          float64 `json:"speed,omitempty"`

	// Provider is gateway routing metadata, stripped before dispatching upstream.
	Provider string `json:"provider,omitempty"`
}

AudioSpeechRequest is an OpenAI-compatible POST /v1/audio/speech (text-to-speech) request.

func DecodeAudioSpeechRequest 

func DecodeAudioSpeechRequest(body []byte, _ *WhiteBoxPrompt) (*AudioSpeechRequest, error)

DecodeAudioSpeechRequest decodes a JSON text-to-speech request body. The semantic envelope is unused: audio responses are binary and not response-cached.

type AudioTranscriptionRequest 

type AudioTranscriptionRequest struct {
	Model                  string
	Filename               string
	FileContentType        string
	File                   []byte
	FileReader             io.Reader
	Language               string
	Prompt                 string
	ResponseFormat         string
	Temperature            string
	TimestampGranularities []string

	// Provider is gateway routing metadata, stripped before dispatching upstream.
	Provider string
}

AudioTranscriptionRequest is an OpenAI-compatible POST /v1/audio/transcriptions (speech-to-text) request. The upstream call is multipart/form-data, so the audio bytes and form fields are transport data rather than a JSON body.

type AvailabilityChecker 

type AvailabilityChecker interface {
	// CheckAvailability verifies the provider's backend service is reachable.
	// Returns nil if available, error otherwise. Initialization logs failures but
	// keeps the provider registered so later refreshes can retry discovery.
	CheckAvailability(ctx context.Context) error
}

AvailabilityChecker is an optional interface for providers that can report backend reachability during startup diagnostics.

type BatchCreateHintAwareProvider 

type BatchCreateHintAwareProvider interface {
	CreateBatchWithHints(ctx context.Context, req *BatchRequest) (*BatchResponse, map[string]string, error)
}

BatchCreateHintAwareProvider is an optional native batch extension for providers that need gateway persistence for per-item endpoint hints.

type BatchError 

type BatchError struct {
	Type    string `json:"type"`
	Message string `json:"message"`
}

BatchError represents a normalized error for a failed batch item.

type BatchFileTransport 

type BatchFileTransport interface {
	GetFileContent(ctx context.Context, providerType, id string) (*FileContentResponse, error)
	CreateFile(ctx context.Context, providerType string, req *FileCreateRequest) (*FileObject, error)
}

BatchFileTransport is the minimal provider-native file API surface needed to preprocess file-backed batch requests.

type BatchItemRewriteFunc 

type BatchItemRewriteFunc func(context.Context, BatchRequestItem, *DecodedBatchItemRequest) (json.RawMessage, error)

BatchItemRewriteFunc rewrites a decoded batch item body for provider submission. The original batch item is provided so callers can preserve non-semantic JSON structure when needed.

type BatchListResponse 

type BatchListResponse struct {
	Object  string          `json:"object"`
	Data    []BatchResponse `json:"data"`
	HasMore bool            `json:"has_more"`
	FirstID string          `json:"first_id,omitempty"`
	LastID  string          `json:"last_id,omitempty"`
}

BatchListResponse is returned by GET /v1/batches.

type BatchPreparationMetadata 

type BatchPreparationMetadata struct {
	OriginalInputFileID  string
	RewrittenInputFileID string
}

BatchPreparationMetadata captures request-scoped batch preprocessing effects that are useful for persistence and debugging but should not be exposed as public API fields automatically.

func GetBatchPreparationMetadata 

func GetBatchPreparationMetadata(ctx context.Context) *BatchPreparationMetadata

GetBatchPreparationMetadata retrieves batch preprocessing metadata from the context.

func (*BatchPreparationMetadata) RecordInputFileRewrite 

func (m *BatchPreparationMetadata) RecordInputFileRewrite(original, rewritten string)

RecordInputFileRewrite tracks the first user-supplied provider file id and the latest derived provider file id submitted upstream.

type BatchRequest 

type BatchRequest struct {
	InputFileID      string             `json:"input_file_id,omitempty"`
	Endpoint         string             `json:"endpoint,omitempty"`
	CompletionWindow string             `json:"completion_window,omitempty"`
	Metadata         map[string]string  `json:"metadata,omitempty"`
	Requests         []BatchRequestItem `json:"requests,omitempty"`
	ExtraFields      UnknownJSONFields  `json:"-" swaggerignore:"true"`
}

BatchRequest is OpenAI-compatible for core fields and extends with inline requests.

OpenAI-compatible fields:

  • input_file_id
  • endpoint
  • completion_window
  • metadata

Gateway extension:

  • requests (inline payloads for providers that support native inline batch bodies)

func DecodeBatchRequest 

func DecodeBatchRequest(body []byte, env *WhiteBoxPrompt) (*BatchRequest, error)

DecodeBatchRequest decodes and caches the canonical batch request for a semantic envelope.

func (BatchRequest) MarshalJSON 

func (r BatchRequest) MarshalJSON() ([]byte, error)

func (*BatchRequest) UnmarshalJSON 

func (r *BatchRequest) UnmarshalJSON(data []byte) error

type BatchRequestCounts 

type BatchRequestCounts struct {
	Total     int `json:"total"`
	Completed int `json:"completed"`
	Failed    int `json:"failed"`
}

BatchRequestCounts is OpenAI-compatible aggregate batch status.

type BatchRequestItem 

type BatchRequestItem struct {
	CustomID    string            `json:"custom_id,omitempty"`
	Method      string            `json:"method,omitempty"`
	URL         string            `json:"url"`
	Body        json.RawMessage   `json:"body" swaggertype:"object"`
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

BatchRequestItem represents one sub-request in an inline batch.

func (BatchRequestItem) MarshalJSON 

func (r BatchRequestItem) MarshalJSON() ([]byte, error)

func (*BatchRequestItem) UnmarshalJSON 

func (r *BatchRequestItem) UnmarshalJSON(data []byte) error

type BatchResponse 

type BatchResponse struct {
	ID               string             `json:"id"`
	Object           string             `json:"object"`
	Provider         string             `json:"provider,omitempty"`
	ProviderBatchID  string             `json:"provider_batch_id,omitempty"`
	Endpoint         string             `json:"endpoint"`
	InputFileID      string             `json:"input_file_id,omitempty"`
	CompletionWindow string             `json:"completion_window,omitempty"`
	Status           string             `json:"status"`
	CreatedAt        int64              `json:"created_at"`
	InProgressAt     *int64             `json:"in_progress_at,omitempty"`
	CompletedAt      *int64             `json:"completed_at,omitempty"`
	FailedAt         *int64             `json:"failed_at,omitempty"`
	CancellingAt     *int64             `json:"cancelling_at,omitempty"`
	CancelledAt      *int64             `json:"cancelled_at,omitempty"`
	RequestCounts    BatchRequestCounts `json:"request_counts"`
	Metadata         map[string]string  `json:"metadata,omitempty"`

	// Gateway extension: optional usage/result snapshots persisted by the gateway.
	Usage   BatchUsageSummary `json:"usage"`
	Results []BatchResultItem `json:"results,omitempty"`
}

BatchResponse uses OpenAI-compatible batch fields and includes provider mapping plus optional cached results.

type BatchResultHintAwareProvider 

type BatchResultHintAwareProvider interface {
	GetBatchResultsWithHints(ctx context.Context, id string, endpointByCustomID map[string]string) (*BatchResultsResponse, error)
	ClearBatchResultHints(batchID string)
}

BatchResultHintAwareProvider is an optional native batch extension for providers that need persisted per-item endpoint hints to shape results.

type BatchResultItem 

type BatchResultItem struct {
	Index      int         `json:"index"`
	CustomID   string      `json:"custom_id,omitempty"`
	URL        string      `json:"url"`
	StatusCode int         `json:"status_code"`
	Model      string      `json:"model,omitempty"`
	Provider   string      `json:"provider,omitempty"`
	Response   any         `json:"response,omitempty"`
	Error      *BatchError `json:"error,omitempty"`
}

BatchResultItem represents one sub-response in a batch.

type BatchResultsResponse 

type BatchResultsResponse struct {
	Object  string            `json:"object"`
	BatchID string            `json:"batch_id"`
	Data    []BatchResultItem `json:"data"`
}

BatchResultsResponse is returned by GET /v1/batches/{id}/results.

type BatchRewriteResult 

type BatchRewriteResult struct {
	Request              *BatchRequest
	RequestEndpointHints map[string]string
	OriginalInputFileID  string
	RewrittenInputFileID string
}

BatchRewriteResult captures the normalized request plus any gateway-only metadata derived while rewriting inline or file-backed batch sources.

func RewriteBatchSource 

func RewriteBatchSource(
	ctx context.Context,
	providerType string,
	req *BatchRequest,
	fileTransport BatchFileTransport,
	operations []Operation,
	rewrite BatchItemRewriteFunc,
) (*BatchRewriteResult, error)

RewriteBatchSource normalizes both inline and file-backed batch sources using the same typed per-item rewrite callback.

type BatchRouteInfo 

type BatchRouteInfo struct {
	Action   string
	BatchID  string
	After    string
	LimitRaw string
	Limit    int
	HasLimit bool
}

BatchRouteInfo is sparse canonical metadata the gateway can derive for /v1/batches* routes. The full create payload remains in BatchRequest when the gateway lazily decodes JSON bodies.

func BatchRouteMetadata 

func BatchRouteMetadata(env *WhiteBoxPrompt, method, path string, routeParams map[string]string, queryParams map[string][]string) (*BatchRouteInfo, error)

BatchRouteMetadata returns sparse batch route semantics, caching them on the envelope when present.

func DeriveBatchRouteInfoFromTransport 

func DeriveBatchRouteInfoFromTransport(method, path string, routeParams map[string]string, queryParams map[string][]string) *BatchRouteInfo

DeriveBatchRouteInfoFromTransport derives sparse batch route info from transport metadata.

type BatchUsageSummary 

type BatchUsageSummary struct {
	InputTokens  int `json:"input_tokens"`
	OutputTokens int `json:"output_tokens"`
	TotalTokens  int `json:"total_tokens"`

	InputCost  *float64 `json:"input_cost,omitempty"`
	OutputCost *float64 `json:"output_cost,omitempty"`
	TotalCost  *float64 `json:"total_cost,omitempty"`
}

BatchUsageSummary aggregates usage and cost for successful batch items.

type BodyMode 

type BodyMode string

BodyMode describes the transport shape expected for an endpoint. 

const (
	BodyModeNone      BodyMode = "none"
	BodyModeJSON      BodyMode = "json"
	BodyModeMultipart BodyMode = "multipart"
	BodyModeOpaque    BodyMode = "opaque"
)

type CapabilitySet 

type CapabilitySet struct {
	SemanticExtraction bool
	AliasResolution    bool
	Guardrails         bool
	RequestPatching    bool
	UsageTracking      bool
	ResponseCaching    bool
	Streaming          bool
	Passthrough        bool
}

CapabilitySet advertises the gateway behaviors that are valid for a request. This is intentionally small and pragmatic for the initial workflow slice.

func CapabilitiesForEndpoint 

func CapabilitiesForEndpoint(desc EndpointDescriptor) CapabilitySet

CapabilitiesForEndpoint returns the current capability set for one endpoint.

type ChatRequest 

type ChatRequest struct {
	Temperature       *float64          `json:"temperature,omitempty"`
	TopP              *float64          `json:"top_p,omitempty"`
	MaxTokens         *int              `json:"max_tokens,omitempty"`
	Model             string            `json:"model"`
	Provider          string            `json:"provider,omitempty"` // Gateway routing hint; stripped before upstream execution.
	Messages          []Message         `json:"messages"`
	Tools             []map[string]any  `json:"tools,omitempty"`
	ToolChoice        any               `json:"tool_choice,omitempty"` // string or object
	ParallelToolCalls *bool             `json:"parallel_tool_calls,omitempty"`
	Stream            bool              `json:"stream,omitempty"`
	StreamOptions     *StreamOptions    `json:"stream_options,omitempty"`
	Reasoning         *Reasoning        `json:"reasoning,omitempty"`
	User              string            `json:"user,omitempty"`
	ServiceTier       string            `json:"service_tier,omitempty"`
	ExtraFields       UnknownJSONFields `json:"-" swaggerignore:"true"`
}

ChatRequest represents the incoming chat completion request

func DecodeChatRequest 

func DecodeChatRequest(body []byte, env *WhiteBoxPrompt) (*ChatRequest, error)

DecodeChatRequest decodes and caches the canonical chat request for a semantic envelope.

func (ChatRequest) MarshalJSON 

func (r ChatRequest) MarshalJSON() ([]byte, error)

func (*ChatRequest) UnmarshalJSON 

func (r *ChatRequest) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes the typed fields via an alias (so new fields are picked up automatically) and captures every other member in ExtraFields.

func (*ChatRequest) WithStreaming 

func (r *ChatRequest) WithStreaming() *ChatRequest

WithStreaming returns a shallow copy of the request with Stream set to true. This avoids mutating the caller's request object.

type ChatResponse 

type ChatResponse struct {
	ID                string   `json:"id"`
	Object            string   `json:"object"`
	Model             string   `json:"model"`
	Provider          string   `json:"provider"`
	SystemFingerprint string   `json:"system_fingerprint,omitempty"`
	Choices           []Choice `json:"choices"`
	Usage             Usage    `json:"usage"`
	Created           int64    `json:"created"`
}

ChatResponse represents the chat completion response

type Choice 

type Choice struct {
	Message      ResponseMessage `json:"message"`
	FinishReason string          `json:"finish_reason"`
	Index        int             `json:"index"`
	Logprobs     json.RawMessage `json:"logprobs,omitempty" swaggertype:"object"`
	// StopSequence is the matched stop sequence when the provider reports one
	// natively (Anthropic stop_reason "stop_sequence"). OpenAI's finish_reason
	// "stop" conflates natural stops with stop-parameter hits, so this is an
	// extension field: present only when the provider knows the answer, in the
	// same spirit as the relayed reasoning_content extension.
	StopSequence string `json:"stop_sequence,omitempty"`
}

Choice represents a single completion choice

type CompletionTokensDetails 

type CompletionTokensDetails struct {
	ReasoningTokens          int `json:"reasoning_tokens"`
	AudioTokens              int `json:"audio_tokens"`
	AcceptedPredictionTokens int `json:"accepted_prediction_tokens"`
	RejectedPredictionTokens int `json:"rejected_prediction_tokens"`
}

CompletionTokensDetails holds extended output token breakdown (OpenAI/xAI).

type ContentPart 

type ContentPart struct {
	Type        string             `json:"type"`
	Text        string             `json:"text,omitempty"`
	ImageURL    *ImageURLContent   `json:"image_url,omitempty"`
	InputAudio  *InputAudioContent `json:"input_audio,omitempty"`
	ExtraFields UnknownJSONFields  `json:"-" swaggerignore:"true"`
}

ContentPart represents a single OpenAI-compatible multimodal chat content part.

func NormalizeContentParts 

func NormalizeContentParts(content any) ([]ContentPart, bool)

NormalizeContentParts converts dynamic JSON-decoded content into typed parts.

func (ContentPart) MarshalJSON 

func (p ContentPart) MarshalJSON() ([]byte, error)

func (*ContentPart) UnmarshalJSON 

func (p *ContentPart) UnmarshalJSON(data []byte) error

type Conversation 

type Conversation struct {
	ID        string            `json:"id"`
	Object    string            `json:"object"` // "conversation"
	CreatedAt int64             `json:"created_at"`
	Metadata  map[string]string `json:"metadata"`
}

Conversation is the OpenAI-compatible conversation resource returned by the /v1/conversations endpoints.

type ConversationCreateRequest 

type ConversationCreateRequest struct {
	Items    []json.RawMessage `json:"items,omitempty" swaggertype:"array,object"`
	Metadata map[string]string `json:"metadata,omitempty"`
}

ConversationCreateRequest is the accepted body for POST /v1/conversations. Items are stored as opaque JSON so the gateway accepts any item shape the client sends without constraining future item-list support.

func DecodeConversationCreateRequest 

func DecodeConversationCreateRequest(data []byte) (*ConversationCreateRequest, error)

DecodeConversationCreateRequest parses a conversation create body. An empty body is treated as an empty request (a conversation with no items/metadata).

type ConversationDeleteResponse 

type ConversationDeleteResponse struct {
	ID      string `json:"id"`
	Object  string `json:"object"` // "conversation.deleted"
	Deleted bool   `json:"deleted"`
}

ConversationDeleteResponse is returned by DELETE /v1/conversations/{id}.

type ConversationUpdateRequest 

type ConversationUpdateRequest struct {
	Metadata *map[string]string `json:"metadata" binding:"required"`
}

ConversationUpdateRequest is the accepted body for POST /v1/conversations/{id}. Metadata is a pointer so the handler can tell an absent field apart from an explicit empty object: OpenAI requires metadata on update.

func DecodeConversationUpdateRequest 

func DecodeConversationUpdateRequest(data []byte) (*ConversationUpdateRequest, error)

DecodeConversationUpdateRequest parses a conversation update body. An empty body decodes to a request with no metadata, which the handler rejects.

type DecodedBatchItemHandlers 

type DecodedBatchItemHandlers[T any] struct {
	Chat       func(*ChatRequest) (T, error)
	Responses  func(*ResponsesRequest) (T, error)
	Embeddings func(*EmbeddingRequest) (T, error)
	Default    func(*DecodedBatchItemRequest) (T, error)
}

DecodedBatchItemHandlers contains operation-specific handlers for a decoded batch item request. Downstream consumers can use this instead of switching on operation names directly.

type DecodedBatchItemRequest 

type DecodedBatchItemRequest struct {
	Endpoint  string
	Method    string
	Operation Operation
	Request   any
}

DecodedBatchItemRequest is the canonical decode result for known JSON batch subrequests.

func DecodeKnownBatchItemRequest 

func DecodeKnownBatchItemRequest(defaultEndpoint string, item BatchRequestItem) (*DecodedBatchItemRequest, error)

DecodeKnownBatchItemRequest normalizes and decodes a known JSON batch subrequest.

func MaybeDecodeKnownBatchItemRequest 

func MaybeDecodeKnownBatchItemRequest(defaultEndpoint string, item BatchRequestItem, operations ...Operation) (*DecodedBatchItemRequest, bool, error)

MaybeDecodeKnownBatchItemRequest selectively decodes a known JSON batch subrequest only when it targets one of the requested operations. Non-POST, body-less, or unmatched items are reported as not handled.

func (*DecodedBatchItemRequest) RequestedModelSelector 

func (decoded *DecodedBatchItemRequest) RequestedModelSelector() (RequestedModelSelector, error)

RequestedModelSelector returns the raw selector requested by the decoded batch item, preserving whether the provider came from the explicit field.

type EmbeddingData 

type EmbeddingData struct {
	Object    string          `json:"object"`
	Embedding json.RawMessage `json:"embedding" swaggertype:"object"`
	Index     int             `json:"index"`
}

EmbeddingData represents a single embedding data point. Embedding is json.RawMessage to support both float arrays and base64-encoded strings.

type EmbeddingRequest 

type EmbeddingRequest struct {
	Model          string            `json:"model"`
	Provider       string            `json:"provider,omitempty"` // Gateway routing hint; stripped before upstream execution.
	Input          any               `json:"input"`
	EncodingFormat string            `json:"encoding_format,omitempty"`
	Dimensions     *int              `json:"dimensions,omitempty"`
	ExtraFields    UnknownJSONFields `json:"-" swaggerignore:"true"`
}

EmbeddingRequest represents the incoming embeddings request (OpenAI-compatible).

func DecodeEmbeddingRequest 

func DecodeEmbeddingRequest(body []byte, env *WhiteBoxPrompt) (*EmbeddingRequest, error)

DecodeEmbeddingRequest decodes and caches the canonical embeddings request for a semantic envelope.

func (EmbeddingRequest) MarshalJSON 

func (r EmbeddingRequest) MarshalJSON() ([]byte, error)

func (*EmbeddingRequest) UnmarshalJSON 

func (r *EmbeddingRequest) UnmarshalJSON(data []byte) error

type EmbeddingResponse 

type EmbeddingResponse struct {
	Object   string          `json:"object"`
	Data     []EmbeddingData `json:"data"`
	Model    string          `json:"model"`
	Provider string          `json:"provider"`
	Usage    EmbeddingUsage  `json:"usage"`
}

EmbeddingResponse represents the embeddings response (OpenAI-compatible).

type EmbeddingUsage 

type EmbeddingUsage struct {
	PromptTokens int `json:"prompt_tokens"`
	TotalTokens  int `json:"total_tokens"`
}

EmbeddingUsage represents token usage information for embeddings.

type EndpointDescriptor 

type EndpointDescriptor struct {
	ModelInteraction bool
	IngressManaged   bool
	Dialect          string
	Operation        Operation
	BodyMode         BodyMode
}

EndpointDescriptor centralizes the transport-facing classification of model and provider routes.

func DescribeEndpoint 

func DescribeEndpoint(method, path string) EndpointDescriptor

DescribeEndpoint classifies a request path and method for ADR-0002 ingress handling.

func DescribeEndpointPath 

func DescribeEndpointPath(path string) EndpointDescriptor

DescribeEndpointPath classifies a request path for ADR-0002 ingress handling.

type ErrorType 

type ErrorType string

ErrorType represents the type of error that occurred 

const (
	// ErrorTypeProvider indicates an upstream provider error (5xx)
	ErrorTypeProvider ErrorType = "provider_error"
	// ErrorTypeRateLimit indicates a rate limit error (429)
	ErrorTypeRateLimit ErrorType = "rate_limit_error"
	// ErrorTypeInvalidRequest indicates a client error (4xx)
	ErrorTypeInvalidRequest ErrorType = "invalid_request_error"
	// ErrorTypeAuthentication indicates an authentication error (401)
	ErrorTypeAuthentication ErrorType = "authentication_error"
	// ErrorTypeNotFound indicates a not found error (404)
	ErrorTypeNotFound ErrorType = "not_found_error"
)

type ExecutionMode 

type ExecutionMode string

ExecutionMode describes how the gateway intends to execute a request. 

const (
	ExecutionModeTranslated  ExecutionMode = "translated"
	ExecutionModePassthrough ExecutionMode = "passthrough"
	ExecutionModeNativeBatch ExecutionMode = "native_batch"
	ExecutionModeNativeFile  ExecutionMode = "native_file"
)

type FileContentResponse 

type FileContentResponse struct {
	ID          string
	Filename    string
	ContentType string
	Data        []byte
}

FileContentResponse wraps raw file bytes with response metadata.

type FileCreateRequest 

type FileCreateRequest struct {
	Purpose       string    `json:"purpose"`
	Filename      string    `json:"filename,omitempty"`
	Content       []byte    `json:"-"`
	ContentReader io.Reader `json:"-"`
}

FileCreateRequest represents an OpenAI-compatible file upload request. The actual request is multipart/form-data; Content is not serialized.

type FileDeleteResponse 

type FileDeleteResponse struct {
	ID      string `json:"id"`
	Object  string `json:"object"`
	Deleted bool   `json:"deleted"`
}

FileDeleteResponse is returned by DELETE /v1/files/{id}.

type FileListResponse 

type FileListResponse struct {
	Object  string       `json:"object"`
	Data    []FileObject `json:"data"`
	HasMore bool         `json:"has_more,omitempty"`
}

FileListResponse is returned by GET /v1/files.

type FileMultipartMetadataReader 

type FileMultipartMetadataReader interface {
	Value(name string) string
	Filename(name string) (string, bool)
}

FileMultipartMetadataReader exposes the small subset of multipart form data needed for sparse file-create semantics.

type FileObject 

type FileObject struct {
	ID            string  `json:"id"`
	Object        string  `json:"object"`
	Bytes         int64   `json:"bytes"`
	CreatedAt     int64   `json:"created_at"`
	Filename      string  `json:"filename"`
	Purpose       string  `json:"purpose"`
	Status        string  `json:"status,omitempty"`
	StatusDetails *string `json:"status_details,omitempty"`

	// Gateway enrichment for multi-provider deployments.
	Provider string `json:"provider,omitempty"`
}

FileObject represents an OpenAI-compatible file object.

type FileRouteInfo 

type FileRouteInfo struct {
	Action   string
	Provider string
	Purpose  string
	Filename string
	FileID   string
	After    string
	LimitRaw string
	Limit    int
	HasLimit bool
}

FileRouteInfo is sparse canonical metadata the gateway can derive for /v1/files* routes. It intentionally excludes file bytes, which remain transport data rather than semantic data.

func DeriveFileRouteInfoFromTransport 

func DeriveFileRouteInfoFromTransport(method, path string, routeParams map[string]string, queryParams map[string][]string) *FileRouteInfo

DeriveFileRouteInfoFromTransport derives sparse file route info from transport metadata.

func EnrichFileCreateRouteInfo 

func EnrichFileCreateRouteInfo(req *FileRouteInfo, reader FileMultipartMetadataReader) *FileRouteInfo

EnrichFileCreateRouteInfo enriches req with provider, purpose, and filename metadata extracted from a multipart reader for file-create requests. It returns req unchanged when req is nil, req.Action is not FileActionCreate, or reader is nil.

func FileRouteMetadata 

func FileRouteMetadata(env *WhiteBoxPrompt, method, path string, routeParams map[string]string, queryParams map[string][]string) (*FileRouteInfo, error)

FileRouteMetadata returns sparse file route semantics, caching them on the envelope when present.

type FunctionCall 

type FunctionCall struct {
	Name        string            `json:"name"`
	Arguments   string            `json:"arguments"`
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

FunctionCall contains the function name and serialized arguments payload.

func (FunctionCall) MarshalJSON 

func (f FunctionCall) MarshalJSON() ([]byte, error)

FunctionCall.MarshalJSON marshals a FunctionCall to JSON, including unknown JSON members from ExtraFields. alias inherits FunctionCall's fields and tags but drops MarshalJSON so json.Marshal does not recurse; ExtraFields (json:"-") is merged separately.

func (*FunctionCall) UnmarshalJSON 

func (f *FunctionCall) UnmarshalJSON(data []byte) error

FunctionCall.UnmarshalJSON unmarshals a FunctionCall from JSON, preserving unknown JSON members in ExtraFields.

type GatewayError 

type GatewayError struct {
	Type       ErrorType `json:"type"`
	Message    string    `json:"message"`
	StatusCode int       `json:"status_code"`
	Provider   string    `json:"provider,omitempty"`
	Param      *string   `json:"param" extensions:"x-nullable"`
	Code       *string   `json:"code" extensions:"x-nullable"`
	// Original error for debugging (not exposed to clients)
	Err error `json:"-"`
	// ResponseBody and ResponseHeaders carry the raw upstream error response so
	// failed provider attempts can be audited. Never serialized to API clients.
	ResponseBody    []byte      `json:"-"`
	ResponseHeaders http.Header `json:"-"`
}

GatewayError is the base error type for all gateway errors

func NewAuthenticationError 

func NewAuthenticationError(provider string, message string) *GatewayError

NewAuthenticationError creates a new authentication error (401)

func NewEmptyProviderResponseError 

func NewEmptyProviderResponseError(provider string) *GatewayError

NewEmptyProviderResponseError reports that a provider returned no response body (502).

func NewInvalidRequestError 

func NewInvalidRequestError(message string, err error) *GatewayError

NewInvalidRequestError creates a new invalid request error (400)

func NewInvalidRequestErrorWithStatus 

func NewInvalidRequestErrorWithStatus(statusCode int, message string, err error) *GatewayError

NewInvalidRequestErrorWithStatus creates a new invalid request error with a specific status code

func NewModelNotFoundError 

func NewModelNotFoundError(model string) *GatewayError

NewModelNotFoundError reports a model the gateway cannot route. It mirrors OpenAI's contract for unknown models — HTTP 404 with code "model_not_found" — so clients that key on the status or code behave the same as against OpenAI.

func NewNotFoundError 

func NewNotFoundError(message string) *GatewayError

NewNotFoundError creates a new not found error (404)

func NewProviderError 

func NewProviderError(provider string, statusCode int, message string, err error) *GatewayError

NewProviderError creates a new provider error (upstream 5xx)

func NewRateLimitError 

func NewRateLimitError(provider string, message string) *GatewayError

NewRateLimitError creates a new rate limit error (429)

func ParseProviderError 

func ParseProviderError(provider string, statusCode int, body []byte, originalErr error) *GatewayError

ParseProviderError parses an error response from a provider and returns an appropriate GatewayError

func ValidateConversationMetadata 

func ValidateConversationMetadata(metadata map[string]string) *GatewayError

ValidateConversationMetadata enforces the OpenAI metadata limits (at most 16 pairs, keys up to 64 characters, values up to 512 characters). It returns nil when the metadata is acceptable.

func (*GatewayError) Error 

func (e *GatewayError) Error() string

Error implements the error interface

func (*GatewayError) HTTPStatusCode 

func (e *GatewayError) HTTPStatusCode() int

HTTPStatusCode returns the appropriate HTTP status code for this error

func (*GatewayError) ToJSON 

func (e *GatewayError) ToJSON() map[string]any

ToJSON converts the error to a JSON-compatible map

func (*GatewayError) Unwrap 

func (e *GatewayError) Unwrap() error

Unwrap implements the error unwrapping interface

func (*GatewayError) WithCode 

func (e *GatewayError) WithCode(code string) *GatewayError

WithCode annotates the error with a machine-readable error code.

func (*GatewayError) WithParam 

func (e *GatewayError) WithParam(param string) *GatewayError

WithParam annotates the error with the offending parameter name.

type ImageURLContent 

type ImageURLContent struct {
	URL         string            `json:"url"`
	Detail      string            `json:"detail,omitempty"`
	MediaType   string            `json:"media_type,omitempty"`
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

ImageURLContent contains an image reference for image_url parts.

func (ImageURLContent) MarshalJSON 

func (c ImageURLContent) MarshalJSON() ([]byte, error)

func (*ImageURLContent) UnmarshalJSON 

func (c *ImageURLContent) UnmarshalJSON(data []byte) error

type InputAudioContent 

type InputAudioContent struct {
	Data        string            `json:"data"`
	Format      string            `json:"format,omitempty"`
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

InputAudioContent contains inline audio payload metadata.

func (InputAudioContent) MarshalJSON 

func (a InputAudioContent) MarshalJSON() ([]byte, error)

func (*InputAudioContent) UnmarshalJSON 

func (a *InputAudioContent) UnmarshalJSON(data []byte) error

type Message 

type Message struct {
	Role        string `json:"role"`
	ToolCallID  string `json:"tool_call_id,omitempty"`
	ContentNull bool   `json:"-"`
	// Content accepts either a plain string or an array of ContentPart values.
	// This preserves OpenAI-compatible multimodal chat payloads.
	Content MessageContent `json:"content"`
	//nolint:govet // Intentional duplicate json tag for Swagger docs: content is null OR string OR []ContentPart.
	// ContentSchema documents that `content` accepts either a plain string
	// or an array of ContentPart values.
	ContentSchema []ContentPart     `` /* 166-byte string literal not displayed */
	ToolCalls     []ToolCall        `json:"tool_calls,omitempty"`
	ExtraFields   UnknownJSONFields `json:"-" swaggerignore:"true"`
}

Message represents a single message in the chat.

func (Message) MarshalJSON 

func (m Message) MarshalJSON() ([]byte, error)

Message.MarshalJSON emits validated chat request message content, preserves null handling, and includes unknown JSON members from ExtraFields.

func (*Message) UnmarshalJSON 

func (m *Message) UnmarshalJSON(data []byte) error

Message.UnmarshalJSON validates chat request message content, preserves unknown JSON members in ExtraFields, and keeps null content handling intact.

type MessageContent 

type MessageContent any

MessageContent stores message content as either text or structured parts.

type Model 

type Model struct {
	ID      string `json:"id"`
	Object  string `json:"object"`
	OwnedBy string `json:"owned_by"`
	Created int64  `json:"created"`
	// Metadata holds optional enrichment data (display name, pricing, capabilities, etc.).
	// May be nil if the model was not found in the external registry.
	Metadata *ModelMetadata `json:"metadata,omitempty"`
}

Model represents a single model in the models list

type ModelCategory 

type ModelCategory string

ModelCategory represents a model's functional category for UI grouping. 

const (
	CategoryAll            ModelCategory = "all"
	CategoryTextGeneration ModelCategory = "text_generation"
	CategoryEmbedding      ModelCategory = "embedding"
	CategoryImage          ModelCategory = "image"
	CategoryAudio          ModelCategory = "audio"
	CategoryVideo          ModelCategory = "video"
	CategoryUtility        ModelCategory = "utility"
)

func AllCategories 

func AllCategories() []ModelCategory

AllCategories returns the ordered list of categories for UI rendering.

func CategoriesForModes 

func CategoriesForModes(modes []string) []ModelCategory

CategoriesForModes returns deduplicated ModelCategory values for the given mode strings. Unrecognized modes are silently skipped.

type ModelLookup 

type ModelLookup interface {
	// Supports returns true if the registry has a provider for the given model
	Supports(model string) bool

	// GetProvider returns the provider for the given model, or nil if not found
	GetProvider(model string) Provider

	// GetProviderType returns the provider type string for the given model.
	// Returns empty string if the model is not found.
	GetProviderType(model string) string

	// ListModels returns all models in the registry
	ListModels() []Model

	// ModelCount returns the number of registered models
	ModelCount() int

	// GetProviderName maps a model selector back to the concrete configured
	// provider instance name. Implementations that have no such mapping return
	// an empty string. Same shape as the optional ProviderNameResolver
	// interface used elsewhere for provider-side type assertions.
	GetProviderName(model string) string

	// GetProviderNameForType maps a provider type such as "openai" to the
	// concrete configured instance name chosen for routing, e.g.
	// "openai-primary". Returns empty when no mapping exists.
	GetProviderNameForType(providerType string) string

	// GetProviderTypeForName maps a concrete configured instance name back to
	// its provider type. Returns empty when no mapping exists.
	GetProviderTypeForName(providerName string) string
}

ModelLookup defines the interface for looking up models and their providers. This abstraction allows the Router to be decoupled from the concrete ModelRegistry implementation.

type ModelMetadata 

type ModelMetadata struct {
	DisplayName     string                  `json:"display_name,omitempty" yaml:"display_name,omitempty"`
	Description     string                  `json:"description,omitempty" yaml:"description,omitempty"`
	Family          string                  `json:"family,omitempty" yaml:"family,omitempty"`
	Modes           []string                `json:"modes,omitempty" yaml:"modes,omitempty"`
	Categories      []ModelCategory         `json:"categories,omitempty" yaml:"categories,omitempty"`
	Tags            []string                `json:"tags,omitempty" yaml:"tags,omitempty"`
	ContextWindow   *int                    `json:"context_window,omitempty" yaml:"context_window,omitempty"`
	MaxOutputTokens *int                    `json:"max_output_tokens,omitempty" yaml:"max_output_tokens,omitempty"`
	Capabilities    map[string]bool         `json:"capabilities,omitempty" yaml:"capabilities,omitempty"`
	Rankings        map[string]ModelRanking `json:"rankings,omitempty" yaml:"rankings,omitempty"`
	Pricing         *ModelPricing           `json:"pricing,omitempty" yaml:"pricing,omitempty"`
	PricingSources  map[string]string       `json:"pricing_sources,omitempty" yaml:"-"`
}

ModelMetadata holds enriched metadata from the external model registry. YAML tags mirror the JSON field names so operators can declare metadata overrides in config.yaml in the same shape that appears in /v1/models output.

func (*ModelMetadata) Clone 

func (m *ModelMetadata) Clone() *ModelMetadata

Clone returns a deep copy so callers can safely mutate the result without affecting the original. Slices, maps, and pointer fields are re-allocated.

type ModelPricing 

type ModelPricing struct {
	Currency               string             `json:"currency" yaml:"currency"`
	InputPerMtok           *float64           `json:"input_per_mtok,omitempty" yaml:"input_per_mtok,omitempty"`
	OutputPerMtok          *float64           `json:"output_per_mtok,omitempty" yaml:"output_per_mtok,omitempty"`
	CachedInputPerMtok     *float64           `json:"cached_input_per_mtok,omitempty" yaml:"cached_input_per_mtok,omitempty"`
	CacheWritePerMtok      *float64           `json:"cache_write_per_mtok,omitempty" yaml:"cache_write_per_mtok,omitempty"`
	ReasoningOutputPerMtok *float64           `json:"reasoning_output_per_mtok,omitempty" yaml:"reasoning_output_per_mtok,omitempty"`
	BatchInputPerMtok      *float64           `json:"batch_input_per_mtok,omitempty" yaml:"batch_input_per_mtok,omitempty"`
	BatchOutputPerMtok     *float64           `json:"batch_output_per_mtok,omitempty" yaml:"batch_output_per_mtok,omitempty"`
	AudioInputPerMtok      *float64           `json:"audio_input_per_mtok,omitempty" yaml:"audio_input_per_mtok,omitempty"`
	AudioOutputPerMtok     *float64           `json:"audio_output_per_mtok,omitempty" yaml:"audio_output_per_mtok,omitempty"`
	PerImage               *float64           `json:"per_image,omitempty" yaml:"per_image,omitempty"`
	InputPerImage          *float64           `json:"input_per_image,omitempty" yaml:"input_per_image,omitempty"`
	PerSecondInput         *float64           `json:"per_second_input,omitempty" yaml:"per_second_input,omitempty"`
	PerSecondOutput        *float64           `json:"per_second_output,omitempty" yaml:"per_second_output,omitempty"`
	PerCharacterInput      *float64           `json:"per_character_input,omitempty" yaml:"per_character_input,omitempty"`
	PerRequest             *float64           `json:"per_request,omitempty" yaml:"per_request,omitempty"`
	PerPage                *float64           `json:"per_page,omitempty" yaml:"per_page,omitempty"`
	Tiers                  []ModelPricingTier `json:"tiers,omitempty" yaml:"tiers,omitempty"`
}

ModelPricing holds pricing information for cost calculation.

func (*ModelPricing) Clone 

func (p *ModelPricing) Clone() *ModelPricing

Clone returns a deep copy so callers can safely mutate the result without affecting the original. Pointer fields and Tiers are re-allocated.

func (*ModelPricing) FieldSources 

func (p *ModelPricing) FieldSources(source string) map[string]string

FieldSources returns non-empty pricing field names mapped to source. Callers should pass a non-empty source string. Tiered pricing is reported as the coarse "tiers" key rather than per-tier entries.

type ModelPricingTier 

type ModelPricingTier struct {
	UpToTokens    *float64 `json:"up_to_tokens,omitempty" yaml:"up_to_tokens,omitempty"`
	UpToMtok      *float64 `json:"up_to_mtok,omitempty" yaml:"up_to_mtok,omitempty"`
	InputPerMtok  *float64 `json:"input_per_mtok,omitempty" yaml:"input_per_mtok,omitempty"`
	OutputPerMtok *float64 `json:"output_per_mtok,omitempty" yaml:"output_per_mtok,omitempty"`
}

ModelPricingTier represents a volume-based pricing tier.

type ModelRanking 

type ModelRanking struct {
	Elo  *float64 `json:"elo,omitempty" yaml:"elo,omitempty"`
	Rank *int     `json:"rank,omitempty" yaml:"rank,omitempty"`
	AsOf string   `json:"as_of,omitempty" yaml:"as_of,omitempty"`
}

ModelRanking holds one benchmark or leaderboard entry for a model.

func CloneModelRanking 

func CloneModelRanking(r ModelRanking) ModelRanking

CloneModelRanking returns a deep copy so the caller can mutate pointer fields (Elo, Rank) without affecting the original.

type ModelSelector 

type ModelSelector struct {
	Model    string
	Provider string
}

ModelSelector is a normalized model routing selector. Model is always the raw upstream model ID (without provider prefix).

func ParseModelSelector 

func ParseModelSelector(model, provider string) (ModelSelector, error)

ParseModelSelector normalizes model/provider routing input.

Accepted forms:

  • model only: "gpt-4o"
  • model with prefix: "openai/gpt-4o"
  • explicit provider field: provider="openai", model="gpt-4o"
  • explicit provider with raw slash model: provider="groq", model="openai/gpt-oss-120b"

When provider is explicit, it is authoritative. A matching leading "provider/" prefix on the model is stripped once as redundant qualification.

func (ModelSelector) QualifiedModel 

func (s ModelSelector) QualifiedModel() string

QualifiedModel returns "provider/model" when Provider is set, or only model otherwise.

type ModelsResponse 

type ModelsResponse struct {
	Object string  `json:"object"`
	Data   []Model `json:"data"`
}

ModelsResponse represents the response from the /v1/models endpoint

type NativeBatchDeleteProvider added in v0.1.53 

type NativeBatchDeleteProvider interface {
	DeleteBatch(ctx context.Context, id string) error
}

NativeBatchDeleteProvider is an optional native batch extension for providers whose upstream supports deleting an ended batch (the Anthropic Message Batches dialect exposes DELETE; the OpenAI batch API does not).

type NativeBatchDeleteRoutableProvider added in v0.1.53 

type NativeBatchDeleteRoutableProvider interface {
	DeleteBatch(ctx context.Context, providerType, id string) error
}

NativeBatchDeleteRoutableProvider extends routing with native batch deletion.

type NativeBatchHintRoutableProvider 

type NativeBatchHintRoutableProvider interface {
	CreateBatchWithHints(ctx context.Context, providerType string, req *BatchRequest) (*BatchResponse, map[string]string, error)
	GetBatchResultsWithHints(ctx context.Context, providerType, id string, endpointByCustomID map[string]string) (*BatchResultsResponse, error)
	ClearBatchResultHints(providerType, batchID string)
}

NativeBatchHintRoutableProvider is an optional routing extension for providers that can consume persisted per-item endpoint hints.

type NativeBatchProvider 

type NativeBatchProvider interface {
	CreateBatch(ctx context.Context, req *BatchRequest) (*BatchResponse, error)
	GetBatch(ctx context.Context, id string) (*BatchResponse, error)
	ListBatches(ctx context.Context, limit int, after string) (*BatchListResponse, error)
	CancelBatch(ctx context.Context, id string) (*BatchResponse, error)
	GetBatchResults(ctx context.Context, id string) (*BatchResultsResponse, error)
}

NativeBatchProvider is implemented by providers that support native discounted batching. This is intentionally separate from Provider so unsupported providers can still implement regular synchronous APIs without batch capabilities.

type NativeBatchProviderTypeLister 

type NativeBatchProviderTypeLister interface {
	NativeBatchProviderTypes() []string
}

NativeBatchProviderTypeLister exposes registered provider types that support native batch operations.

type NativeBatchRoutableProvider 

type NativeBatchRoutableProvider interface {
	CreateBatch(ctx context.Context, providerType string, req *BatchRequest) (*BatchResponse, error)
	GetBatch(ctx context.Context, providerType, id string) (*BatchResponse, error)
	ListBatches(ctx context.Context, providerType string, limit int, after string) (*BatchListResponse, error)
	CancelBatch(ctx context.Context, providerType, id string) (*BatchResponse, error)
	GetBatchResults(ctx context.Context, providerType, id string) (*BatchResultsResponse, error)
}

NativeBatchRoutableProvider extends routing with native batch operations.

type NativeFileProvider 

type NativeFileProvider interface {
	CreateFile(ctx context.Context, req *FileCreateRequest) (*FileObject, error)
	ListFiles(ctx context.Context, purpose string, limit int, after string) (*FileListResponse, error)
	GetFile(ctx context.Context, id string) (*FileObject, error)
	DeleteFile(ctx context.Context, id string) (*FileDeleteResponse, error)
	GetFileContent(ctx context.Context, id string) (*FileContentResponse, error)
}

NativeFileProvider is implemented by providers that support OpenAI-compatible files APIs.

type NativeFileProviderTypeLister 

type NativeFileProviderTypeLister interface {
	NativeFileProviderTypes() []string
}

NativeFileProviderTypeLister exposes registered provider types that support native file operations. This is an internal capability inventory and must not depend on the public model catalog.

type NativeFileRoutableProvider 

type NativeFileRoutableProvider interface {
	CreateFile(ctx context.Context, providerType string, req *FileCreateRequest) (*FileObject, error)
	ListFiles(ctx context.Context, providerType, purpose string, limit int, after string) (*FileListResponse, error)
	GetFile(ctx context.Context, providerType, id string) (*FileObject, error)
	DeleteFile(ctx context.Context, providerType, id string) (*FileDeleteResponse, error)
	GetFileContent(ctx context.Context, providerType, id string) (*FileContentResponse, error)
}

NativeFileRoutableProvider extends routing with provider-native file operations.

type NativeResponseLifecycleProvider 

type NativeResponseLifecycleProvider interface {
	GetResponse(ctx context.Context, id string, params ResponseRetrieveParams) (*ResponsesResponse, error)
	ListResponseInputItems(ctx context.Context, id string, params ResponseInputItemsParams) (*ResponseInputItemListResponse, error)
	CancelResponse(ctx context.Context, id string) (*ResponsesResponse, error)
	DeleteResponse(ctx context.Context, id string) (*ResponseDeleteResponse, error)
}

NativeResponseLifecycleProvider is implemented by providers that support OpenAI-compatible Responses lifecycle operations.

type NativeResponseLifecycleRoutableProvider 

type NativeResponseLifecycleRoutableProvider interface {
	GetResponse(ctx context.Context, providerType, id string, params ResponseRetrieveParams) (*ResponsesResponse, error)
	ListResponseInputItems(ctx context.Context, providerType, id string, params ResponseInputItemsParams) (*ResponseInputItemListResponse, error)
	CancelResponse(ctx context.Context, providerType, id string) (*ResponsesResponse, error)
	DeleteResponse(ctx context.Context, providerType, id string) (*ResponseDeleteResponse, error)
}

NativeResponseLifecycleRoutableProvider extends routing with provider-native Responses lifecycle operations.

type NativeResponseProviderTypeLister 

type NativeResponseProviderTypeLister interface {
	NativeResponseProviderTypes() []string
}

NativeResponseProviderTypeLister exposes registered provider types that support native Responses lifecycle operations.

type NativeResponseUtilityProvider 

type NativeResponseUtilityProvider interface {
	CountResponseInputTokens(ctx context.Context, req *ResponsesRequest) (*ResponseInputTokensResponse, error)
	CompactResponse(ctx context.Context, req *ResponsesRequest) (*ResponseCompactResponse, error)
}

NativeResponseUtilityProvider is implemented by providers that support OpenAI-compatible Responses utility operations.

type NativeResponseUtilityRoutableProvider 

type NativeResponseUtilityRoutableProvider interface {
	CountResponseInputTokens(ctx context.Context, providerType string, req *ResponsesRequest) (*ResponseInputTokensResponse, error)
	CompactResponse(ctx context.Context, providerType string, req *ResponsesRequest) (*ResponseCompactResponse, error)
}

NativeResponseUtilityRoutableProvider extends routing with provider-native Responses utility operations.

type OpenAIErrorEnvelope 

type OpenAIErrorEnvelope struct {
	Error OpenAIErrorObject `json:"error" binding:"required"`
}

OpenAIErrorEnvelope documents the public OpenAI-compatible error response.

type OpenAIErrorObject 

type OpenAIErrorObject struct {
	Type    ErrorType `json:"type" binding:"required"`
	Message string    `json:"message" binding:"required"`
	Param   *string   `json:"param" binding:"required" extensions:"x-nullable"`
	Code    *string   `json:"code" binding:"required" extensions:"x-nullable"`
}

OpenAIErrorObject is the error object exposed in public API responses.

type Operation 

type Operation string

Operation identifies the gateway operation represented by an endpoint. 

const (
	OperationChatCompletions     Operation = "chat_completions"
	OperationResponses           Operation = "responses"
	OperationConversations       Operation = "conversations"
	OperationEmbeddings          Operation = "embeddings"
	OperationBatches             Operation = "batches"
	OperationFiles               Operation = "files"
	OperationAudioSpeech         Operation = "audio_speech"
	OperationAudioTranscriptions Operation = "audio_transcriptions"
	OperationRealtime            Operation = "realtime"
	OperationProviderPassthrough Operation = "provider_passthrough"
	OperationMCP                 Operation = "mcp"
)

type PassthroughProvider 

type PassthroughProvider interface {
	Passthrough(ctx context.Context, req *PassthroughRequest) (*PassthroughResponse, error)
}

PassthroughProvider supports opaque provider-native forwarding.

type PassthroughRequest 

type PassthroughRequest struct {
	Method       string
	Endpoint     string
	Body         io.ReadCloser
	Headers      http.Header
	ProviderName string // optional: concrete configured provider instance name for name-based routing
}

PassthroughRequest is the transport-oriented request for opaque provider-native forwarding.

type PassthroughResponse 

type PassthroughResponse struct {
	StatusCode int
	Headers    map[string][]string
	Body       io.ReadCloser
}

PassthroughResponse is the raw upstream response for opaque forwarding. Body is an io.ReadCloser returned by the upstream provider, and callers are responsible for closing it when they are finished with the response body.

type PassthroughRouteInfo 

type PassthroughRouteInfo struct {
	Provider           string // resolved provider type (e.g. "anthropic")
	ProviderName       string // original configured provider instance name (e.g. "teste")
	RawEndpoint        string
	NormalizedEndpoint string
	SemanticOperation  string
	AuditPath          string
	Model              string
}

PassthroughRouteInfo is typed passthrough metadata derived from ingress transport and later enrichment stages.

RawEndpoint reflects the route-relative provider endpoint from the inbound path. NormalizedEndpoint, SemanticOperation, and AuditPath are optional and may be filled by later gateway enrichment before execution. Once cached on WhiteBoxPrompt or Workflow, it should be treated as immutable by later request stages.

type PassthroughSemanticEnricher 

type PassthroughSemanticEnricher interface {
	ProviderType() string
	Enrich(snapshot *RequestSnapshot, prompt *WhiteBoxPrompt, info *PassthroughRouteInfo) *PassthroughRouteInfo
}

PassthroughSemanticEnricher derives provider-specific passthrough metadata from ingress transport and best-effort prompt state before execution workflow resolution runs.

type PromptTokensDetails 

type PromptTokensDetails struct {
	CachedTokens int `json:"cached_tokens"`
	AudioTokens  int `json:"audio_tokens"`
	TextTokens   int `json:"text_tokens"`
	ImageTokens  int `json:"image_tokens"`
}

PromptTokensDetails holds extended input token breakdown (OpenAI/xAI).

type Provider 

type Provider interface {
	// ChatCompletion executes a chat completion request
	ChatCompletion(ctx context.Context, req *ChatRequest) (*ChatResponse, error)

	// StreamChatCompletion returns a raw SSE stream (caller must close)
	StreamChatCompletion(ctx context.Context, req *ChatRequest) (io.ReadCloser, error)

	// ListModels returns the list of available models
	ListModels(ctx context.Context) (*ModelsResponse, error)

	// Responses executes a Responses API request (OpenAI-compatible)
	Responses(ctx context.Context, req *ResponsesRequest) (*ResponsesResponse, error)

	// StreamResponses returns a raw SSE stream for Responses API (caller must close)
	StreamResponses(ctx context.Context, req *ResponsesRequest) (io.ReadCloser, error)

	// Embeddings sends an embeddings request to the provider
	Embeddings(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)
}

Provider defines the interface for LLM providers

type ProviderNameResolver 

type ProviderNameResolver interface {
	GetProviderName(model string) string
}

ProviderNameResolver is an optional interface for components that can map a routed model selector back to the concrete configured provider instance name.

type ProviderNameTypeResolver 

type ProviderNameTypeResolver interface {
	GetProviderTypeForName(providerName string) string
}

ProviderNameTypeResolver is an optional interface for components that can map a concrete configured provider instance name such as "openai_primary" back to its provider type, such as "openai".

type ProviderTypeNameResolver 

type ProviderTypeNameResolver interface {
	GetProviderNameForType(providerType string) string
}

ProviderTypeNameResolver is an optional interface for components that can map a provider type such as "openai" to the concrete configured provider instance name used for routing, such as "openai_primary".

type RealtimeCallProvider 

type RealtimeCallProvider interface {
	RealtimeCallTarget(ctx context.Context, req *RealtimeRequest) (*RealtimeHTTPTarget, error)
	RealtimeClientSecretTarget(ctx context.Context, req *RealtimeRequest) (*RealtimeHTTPTarget, error)
}

RealtimeCallProvider is implemented by providers that expose OpenAI-compatible realtime HTTP signaling endpoints: SDP exchange for WebRTC calls and ephemeral client secrets for browser clients. It is optional, like RealtimeProvider; websocket-only realtime providers simply omit it.

type RealtimeCallRouter 

type RealtimeCallRouter interface {
	RealtimeCallTarget(ctx context.Context, req *RealtimeRequest) (*RealtimeHTTPTarget, error)
	RealtimeClientSecretTarget(ctx context.Context, req *RealtimeRequest) (*RealtimeHTTPTarget, error)
}

RealtimeCallRouter resolves realtime HTTP signaling targets for a request. The Router implements it by routing on the model, mirroring RealtimeRouter.

type RealtimeHTTPTarget 

type RealtimeHTTPTarget struct {
	URL     string
	Headers http.Header
}

RealtimeHTTPTarget describes an upstream HTTPS endpoint for realtime call signaling (WebRTC SDP exchange, ephemeral client secrets). Like the websocket target, it carries only the dial URL and the credential headers to inject; headers must never be logged.

type RealtimeProvider 

type RealtimeProvider interface {
	RealtimeTarget(ctx context.Context, req *RealtimeRequest) (*RealtimeTarget, error)
}

RealtimeProvider is implemented by providers that expose an OpenAI-compatible realtime websocket endpoint. It is optional, like AudioProvider, so providers without realtime support simply omit it.

type RealtimeRequest 

type RealtimeRequest struct {
	Model    string
	Provider string
	CallID   string
}

RealtimeRequest carries the resolved parameters for opening a realtime (speech-to-speech) websocket session. The model selects the provider; the optional Provider hint mirrors the audio endpoints. CallID, when set, attaches to an existing WebRTC/SIP call as a sideband websocket instead of opening a fresh model session.

type RealtimeRouter 

type RealtimeRouter interface {
	RealtimeTarget(ctx context.Context, req *RealtimeRequest) (*RealtimeTarget, error)
}

RealtimeRouter resolves a realtime target for a request. The Router implements it by routing on the model (optionally constrained by a provider hint), so it backs both the typed /v1/realtime route and the /p/{provider}/v1/realtime passthrough upgrade.

type RealtimeTarget 

type RealtimeTarget struct {
	URL          string
	Headers      http.Header
	Subprotocols []string
}

RealtimeTarget describes the upstream websocket a provider exposes for realtime sessions. Realtime is a transport concern, not a translation concern: the provider's event schema is the wire format, so the gateway only needs the dial URL and the credential headers to inject. Headers must never be logged.

type Reasoning 

type Reasoning struct {
	// Effort controls how much reasoning effort the model should use.
	// Valid values are "low", "medium", "high", "xhigh", and "max".
	// "xhigh" and "max" are supported by newer models such as Claude Opus 4.8;
	// providers downgrade unsupported levels to their nearest equivalent.
	Effort string `json:"effort,omitempty"`
}

Reasoning configures reasoning behavior for models that support extended thinking. This is used with OpenAI's o-series models and other reasoning-capable models.

type RequestModelResolution 

type RequestModelResolution struct {
	Requested        RequestedModelSelector
	ResolvedSelector ModelSelector
	ProviderType     string
	ProviderName     string
	AliasApplied     bool
}

RequestModelResolution captures the requested model selector at ingress and the concrete selector chosen for execution after alias resolution.

func (*RequestModelResolution) RequestedQualifiedModel 

func (r *RequestModelResolution) RequestedQualifiedModel() string

RequestedQualifiedModel returns the canonical requested selector.

func (*RequestModelResolution) ResolvedQualifiedModel 

func (r *RequestModelResolution) ResolvedQualifiedModel() string

ResolvedQualifiedModel returns the concrete qualified model selected for execution.

type RequestOrigin 

type RequestOrigin string

RequestOrigin identifies whether a request came from an external caller or an internal gateway-owned workflow. 

const (
	RequestOriginExternal  RequestOrigin = "external"
	RequestOriginGuardrail RequestOrigin = "guardrail"
)

func GetRequestOrigin 

func GetRequestOrigin(ctx context.Context) RequestOrigin

GetRequestOrigin retrieves the request origin from context. When unset, external traffic is assumed.

type RequestSnapshot 

type RequestSnapshot struct {
	// Method is the inbound HTTP method.
	Method string
	// Path is the request URL path as received at ingress.
	Path string
	// UserPath is the canonical business hierarchy path sourced from the
	// configured user-path request header when provided.
	UserPath string

	// ContentType is the inbound Content-Type header value.
	ContentType string

	// BodyNotCaptured reports that the request body exceeded the capture limit,
	// so CapturedBody is omitted and the live body stream remains on the request.
	BodyNotCaptured bool
	// RequestID is the canonical request id propagated through context, headers,
	// providers, and audit records for this request.
	RequestID string
	// contains filtered or unexported fields
}

RequestSnapshot is the transport-level capture of an inbound request. It preserves the request as received at the HTTP boundary so later stages can extract semantics without losing fidelity while keeping mutable state behind defensive-copy accessors by default.

func GetRequestSnapshot 

func GetRequestSnapshot(ctx context.Context) *RequestSnapshot

GetRequestSnapshot retrieves the request snapshot from the context.

func NewRequestSnapshot 

func NewRequestSnapshot(method, path string, routeParams map[string]string, queryParams, headers map[string][]string, contentType string, capturedBody []byte, bodyNotCaptured bool, requestID string, traceMetadata map[string]string, userPath ...string) *RequestSnapshot

NewRequestSnapshot constructs a RequestSnapshot and defensively copies its mutable map and byte-slice inputs.

func NewRequestSnapshotWithOwnedMaps 

func NewRequestSnapshotWithOwnedMaps(method, path string, routeParams map[string]string, queryParams, headers map[string][]string, contentType string, capturedBody []byte, bodyNotCaptured bool, requestID string, traceMetadata map[string]string, userPath ...string) *RequestSnapshot

NewRequestSnapshotWithOwnedMaps constructs a RequestSnapshot that takes ownership of routeParams, queryParams, traceMetadata, and capturedBody (callers must not mutate them afterwards) while still defensively cloning headers, which is typically the live request header map mutated downstream.

Use this on the ingress hot path, where the route/query/trace maps and body are freshly built for the snapshot and would otherwise be cloned for no benefit.

func (*RequestSnapshot) CapturedBody 

func (s *RequestSnapshot) CapturedBody() []byte

CapturedBody returns a defensive copy of the captured request body bytes.

func (*RequestSnapshot) CapturedBodyView 

func (s *RequestSnapshot) CapturedBodyView() []byte

CapturedBodyView returns the captured request body bytes without cloning. Callers must treat the returned slice as read-only.

func (*RequestSnapshot) GetHeaders 

func (s *RequestSnapshot) GetHeaders() map[string][]string

GetHeaders returns a defensive copy of the captured request headers.

func (*RequestSnapshot) GetQueryParams 

func (s *RequestSnapshot) GetQueryParams() map[string][]string

GetQueryParams returns a defensive copy of the captured query parameters.

func (*RequestSnapshot) GetRouteParams 

func (s *RequestSnapshot) GetRouteParams() map[string]string

GetRouteParams returns a defensive copy of the captured route parameters.

func (*RequestSnapshot) GetTraceMetadata 

func (s *RequestSnapshot) GetTraceMetadata() map[string]string

GetTraceMetadata returns a defensive copy of the captured trace metadata.

func (*RequestSnapshot) HeadersView 

func (s *RequestSnapshot) HeadersView() map[string][]string

HeadersView returns the captured request headers without cloning. Callers must treat the returned map as read-only.

func (*RequestSnapshot) WithOwnedCapturedBody 

func (s *RequestSnapshot) WithOwnedCapturedBody(capturedBody []byte, bodyNotCaptured bool) *RequestSnapshot

WithOwnedCapturedBody returns a shallow-cloned snapshot with request body capture state replaced. capturedBody is taken as owned by the snapshot and must not be mutated after this call.

func (*RequestSnapshot) WithUserPath 

func (s *RequestSnapshot) WithUserPath(userPath string) *RequestSnapshot

WithUserPath returns a shallow-cloned snapshot with UserPath and the captured default user-path header rewritten to the provided canonical value.

func (*RequestSnapshot) WithUserPathHeader 

func (s *RequestSnapshot) WithUserPathHeader(userPath, headerName string) *RequestSnapshot

WithUserPathHeader returns a shallow-cloned snapshot with UserPath and the captured configured user-path header rewritten to the provided canonical value.

type RequestedModelSelector 

type RequestedModelSelector struct {
	Model            string
	ProviderHint     string
	ExplicitProvider bool
}

RequestedModelSelector captures the raw selector as provided by a caller before alias resolution or provider routing.

func BatchItemRequestedModelSelector 

func BatchItemRequestedModelSelector(defaultEndpoint string, item BatchRequestItem) (RequestedModelSelector, error)

BatchItemRequestedModelSelector derives the raw requested selector for a known JSON batch subrequest.

func NewRequestedModelSelector 

func NewRequestedModelSelector(model, providerHint string) RequestedModelSelector

NewRequestedModelSelector normalizes raw selector input while preserving whether the provider came from an explicit field rather than model syntax.

func (RequestedModelSelector) Normalize 

func (s RequestedModelSelector) Normalize() (ModelSelector, error)

Normalize returns the canonical routing selector for this request input.

func (RequestedModelSelector) RequestedQualifiedModel 

func (s RequestedModelSelector) RequestedQualifiedModel() string

RequestedQualifiedModel returns the canonical requested selector string used for audit and workflow reporting.

type ResolvedWorkflowPolicy 

type ResolvedWorkflowPolicy struct {
	VersionID string
	Version   int
	// ScopeProvider is the configured provider instance name stored on the matched workflow.
	ScopeProvider  string
	ScopeModel     string
	ScopeUserPath  string
	Name           string
	WorkflowHash   string
	Features       WorkflowFeatures
	GuardrailsHash string
}

ResolvedWorkflowPolicy is the request-scoped runtime projection of one matched persisted workflow version.

type ResponseCompactRequest 

type ResponseCompactRequest ResponseInputTokensRequest

ResponseCompactRequest documents the request body accepted by POST /v1/responses/compact. It accepts exactly the same members as ResponseInputTokensRequest, so it is defined from that struct: the field set is written once and both utility endpoints stay in lockstep.

func (ResponseCompactRequest) MarshalJSON 

func (r ResponseCompactRequest) MarshalJSON() ([]byte, error)

MarshalJSON preserves the dynamic input payload while omitting Swagger-only schema fields.

func (*ResponseCompactRequest) UnmarshalJSON 

func (r *ResponseCompactRequest) UnmarshalJSON(data []byte) error

UnmarshalJSON preserves the dynamic input payload for gateway utility requests.

type ResponseCompactResponse 

type ResponseCompactResponse struct {
	ID        string                `json:"id"`
	Object    string                `json:"object"`
	CreatedAt int64                 `json:"created_at"`
	Output    []ResponsesOutputItem `json:"output"`
	Usage     *ResponsesUsage       `json:"usage,omitempty"`
	Error     *ResponsesError       `json:"error,omitempty"`
	Metadata  map[string]string     `json:"metadata,omitempty"`
	Provider  string                `json:"provider,omitempty"`
}

ResponseCompactResponse is returned by POST /v1/responses/compact.

type ResponseDeleteResponse 

type ResponseDeleteResponse struct {
	ID      string `json:"id"`
	Object  string `json:"object"`
	Deleted bool   `json:"deleted"`
}

ResponseDeleteResponse is returned by DELETE /v1/responses/{id}.

type ResponseInputItemListResponse 

type ResponseInputItemListResponse struct {
	Object  string            `json:"object"`
	Data    []json.RawMessage `json:"data" swaggertype:"array,object"`
	FirstID string            `json:"first_id,omitempty"`
	LastID  string            `json:"last_id,omitempty"`
	HasMore bool              `json:"has_more"`
}

ResponseInputItemListResponse is returned by GET /v1/responses/{id}/input_items.

type ResponseInputItemsParams 

type ResponseInputItemsParams struct {
	After   string
	Include []string
	Limit   int
	Order   string
}

ResponseInputItemsParams contains query parameters accepted by GET /v1/responses/{id}/input_items.

type ResponseInputTokensRequest 

type ResponseInputTokensRequest struct {
	Model              string            `json:"model,omitempty"`
	Provider           string            `json:"provider,omitempty"` // Gateway routing hint; stripped before upstream execution.
	Input              any               `json:"input,omitempty"`    // string or []ResponsesInputElement — see docs for array form
	Instructions       string            `json:"instructions,omitempty"`
	Tools              []map[string]any  `json:"tools,omitempty"`
	ToolChoice         any               `json:"tool_choice,omitempty"` // string or object
	ParallelToolCalls  *bool             `json:"parallel_tool_calls,omitempty"`
	Temperature        *float64          `json:"temperature,omitempty"`
	TopP               *float64          `json:"top_p,omitempty"`
	TopLogprobs        *int              `json:"top_logprobs,omitempty"`
	MaxOutputTokens    *int              `json:"max_output_tokens,omitempty"`
	Metadata           map[string]string `json:"metadata,omitempty"`
	Reasoning          *Reasoning        `json:"reasoning,omitempty"`
	Text               any               `json:"text,omitempty"`
	Include            []string          `json:"include,omitempty"`
	Truncation         string            `json:"truncation,omitempty"`
	Store              *bool             `json:"store,omitempty"`
	PreviousResponseID string            `json:"previous_response_id,omitempty"`
	// Conversation accepts either a conversation ID string or an object with id.
	Conversation         *ResponsesConversationRef `json:"conversation,omitempty"`
	Prompt               any                       `json:"prompt,omitempty"`
	PromptCacheRetention string                    `json:"prompt_cache_retention,omitempty"`
	ContextManagement    any                       `json:"context_management,omitempty"`
	User                 string                    `json:"user,omitempty"`
	ServiceTier          string                    `json:"service_tier,omitempty"`
	SafetyIdentifier     string                    `json:"safety_identifier,omitempty"`
	ExtraFields          UnknownJSONFields         `json:"-" swaggerignore:"true"`
}

ResponseInputTokensRequest documents the request body accepted by POST /v1/responses/input_tokens.

func (ResponseInputTokensRequest) MarshalJSON 

func (r ResponseInputTokensRequest) MarshalJSON() ([]byte, error)

MarshalJSON preserves the dynamic input payload while omitting Swagger-only schema fields.

func (*ResponseInputTokensRequest) UnmarshalJSON 

func (r *ResponseInputTokensRequest) UnmarshalJSON(data []byte) error

UnmarshalJSON preserves the dynamic input payload for gateway utility requests.

type ResponseInputTokensResponse 

type ResponseInputTokensResponse struct {
	Object      string `json:"object"`
	InputTokens int    `json:"input_tokens"`
}

ResponseInputTokensResponse is returned by POST /v1/responses/input_tokens.

type ResponseMessage 

type ResponseMessage struct {
	Role    string         `json:"role"`
	Content MessageContent `json:"content"`
	//nolint:govet // Intentional duplicate json tag for Swagger docs: content is null OR string OR []ContentPart.
	ContentSchema []ContentPart     `` /* 166-byte string literal not displayed */
	ToolCalls     []ToolCall        `json:"tool_calls,omitempty"`
	ExtraFields   UnknownJSONFields `json:"-" swaggerignore:"true"`
}

ResponseMessage represents a single assistant message in a chat response.

func (ResponseMessage) MarshalJSON 

func (m ResponseMessage) MarshalJSON() ([]byte, error)

ResponseMessage.MarshalJSON preserves OpenAI-compatible null content for tool-call response messages and includes unknown JSON members from ExtraFields.

func (*ResponseMessage) UnmarshalJSON 

func (m *ResponseMessage) UnmarshalJSON(data []byte) error

ResponseMessage.UnmarshalJSON validates chat response message content, preserves unknown JSON members in ExtraFields, and keeps tool-call null content handling intact.

type ResponseRetrieveParams 

type ResponseRetrieveParams struct {
	Include            []string
	IncludeObfuscation *bool
	StartingAfter      *int
	Stream             bool
}

ResponseRetrieveParams contains query parameters accepted by GET /v1/responses/{id}.

type ResponsesContentItem 

type ResponsesContentItem struct {
	Type       string             `json:"type"` // "output_text", "input_image", "input_audio", etc.
	Text       string             `json:"text,omitempty"`
	ImageURL   *ImageURLContent   `json:"image_url,omitempty"`
	InputAudio *InputAudioContent `json:"input_audio,omitempty"`
	// Providers can return structured annotation objects here (for example
	// citations from native tools), so keep the payload shape liberal.
	Annotations []json.RawMessage `json:"annotations,omitempty" swaggertype:"array,object"`
}

ResponsesContentItem represents a content item in the output.

type ResponsesConversationRef 

type ResponsesConversationRef struct {
	ID  string          `json:"id,omitempty"`
	Raw json.RawMessage `json:"-" swaggerignore:"true"`
}

ResponsesConversationRef represents the Responses API conversation request field. OpenAI accepts either a conversation ID string or an object with id. Raw preserves the original string/object shape across JSON round trips.

func (ResponsesConversationRef) MarshalJSON 

func (c ResponsesConversationRef) MarshalJSON() ([]byte, error)

MarshalJSON preserves whether the conversation was originally supplied as a string or object. The ID field is authoritative so callers can update or clear a decoded reference without leaking the original raw value.

func (*ResponsesConversationRef) UnmarshalJSON 

func (c *ResponsesConversationRef) UnmarshalJSON(data []byte) error

UnmarshalJSON accepts the documented Responses conversation union: a string ID or an object with an id field.

type ResponsesError 

type ResponsesError struct {
	Code    string `json:"code"`
	Message string `json:"message"`
}

ResponsesError represents an error in the response.

type ResponsesInputElement 

type ResponsesInputElement struct {
	Type string `json:"type,omitempty"` // "message", "function_call", "function_call_output"

	// Message fields (type="" or "message")
	Role    string `json:"role,omitempty"`
	Status  string `json:"status,omitempty"`
	Content any    `json:"content,omitempty"` // Can be string or []ContentPart
	//nolint:govet // Intentional duplicate json tag for Swagger docs: content is string OR []ContentPart.
	ContentSchema []ContentPart `` /* 146-byte string literal not displayed */

	// Function call fields (type="function_call")
	CallID    string `json:"call_id,omitempty"`
	Name      string `json:"name,omitempty"`
	Arguments string `json:"arguments,omitempty"`

	// Function call output fields (type="function_call_output") — CallID shared above
	Output      string            `json:"output,omitempty"`
	Raw         json.RawMessage   `json:"-" swaggerignore:"true"`
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

ResponsesInputElement represents a single item in the Responses API input array. It is a discriminated union keyed on Type:

  • "" or "message": a chat-style message with Role and Content
  • "function_call": a tool invocation with CallID, Name, and Arguments
  • "function_call_output": a tool result with CallID and Output

Unknown JSON members encountered during unmarshaling are preserved in ExtraFields (UnknownJSONFields) and marshaled back out unchanged so extensions can round-trip; Swagger ignores ExtraFields, and typed fields should be preferred when available.

func (ResponsesInputElement) MarshalJSON 

func (e ResponsesInputElement) MarshalJSON() ([]byte, error)

MarshalJSON serializes a ResponsesInputElement, emitting only the fields relevant to its Type variant.

func (*ResponsesInputElement) UnmarshalJSON 

func (e *ResponsesInputElement) UnmarshalJSON(data []byte) error

UnmarshalJSON deserializes a ResponsesInputElement, switching on the "type" field to populate variant-specific fields.

type ResponsesOutputItem 

type ResponsesOutputItem struct {
	ID        string                 `json:"id"`
	Type      string                 `json:"type"` // "message", "function_call", etc.
	Role      string                 `json:"role,omitempty"`
	Status    string                 `json:"status,omitempty"`
	CallID    string                 `json:"call_id,omitempty"`
	Name      string                 `json:"name,omitempty"`
	Arguments string                 `json:"arguments,omitempty"`
	Content   []ResponsesContentItem `json:"content,omitempty"`
}

ResponsesOutputItem represents an item in the output array.

type ResponsesRequest 

type ResponsesRequest struct {
	Model              string            `json:"model"`
	Provider           string            `json:"provider,omitempty"` // Gateway routing hint; stripped before upstream execution.
	Input              any               `json:"input"`              // string or []ResponsesInputElement — see docs for array form
	Instructions       string            `json:"instructions,omitempty"`
	Tools              []map[string]any  `json:"tools,omitempty"`
	ToolChoice         any               `json:"tool_choice,omitempty"` // string or object
	ParallelToolCalls  *bool             `json:"parallel_tool_calls,omitempty"`
	Temperature        *float64          `json:"temperature,omitempty"`
	TopP               *float64          `json:"top_p,omitempty"`
	TopLogprobs        *int              `json:"top_logprobs,omitempty"`
	MaxOutputTokens    *int              `json:"max_output_tokens,omitempty"`
	Stream             bool              `json:"stream,omitempty"`
	StreamOptions      *StreamOptions    `json:"stream_options,omitempty"`
	Metadata           map[string]string `json:"metadata,omitempty"`
	Reasoning          *Reasoning        `json:"reasoning,omitempty"`
	Text               any               `json:"text,omitempty"`
	Include            []string          `json:"include,omitempty"`
	Truncation         string            `json:"truncation,omitempty"`
	Store              *bool             `json:"store,omitempty"`
	PreviousResponseID string            `json:"previous_response_id,omitempty"`
	// Conversation accepts either a conversation ID string or an object with id.
	Conversation         *ResponsesConversationRef `json:"conversation,omitempty"`
	Prompt               any                       `json:"prompt,omitempty"`
	PromptCacheRetention string                    `json:"prompt_cache_retention,omitempty"`
	ContextManagement    any                       `json:"context_management,omitempty"`
	User                 string                    `json:"user,omitempty"`
	ServiceTier          string                    `json:"service_tier,omitempty"`
	SafetyIdentifier     string                    `json:"safety_identifier,omitempty"`
	ExtraFields          UnknownJSONFields         `json:"-" swaggerignore:"true"`
}

ResponsesRequest represents the request body for the Responses API. This is the OpenAI-compatible /v1/responses endpoint. Unknown JSON members encountered during unmarshaling are preserved in ExtraFields (UnknownJSONFields) and emitted again during marshaling so callers can round-trip extensions; Swagger ignores ExtraFields, and typed fields should be preferred when available.

func DecodeResponsesRequest 

func DecodeResponsesRequest(body []byte, env *WhiteBoxPrompt) (*ResponsesRequest, error)

DecodeResponsesRequest decodes and caches the canonical responses request for a semantic envelope.

func (*ResponsesRequest) CompactRequest 

func (r *ResponsesRequest) CompactRequest() *ResponseCompactRequest

CompactRequest reduces a full Responses request for the compact endpoint; see InputTokensRequest.

func (*ResponsesRequest) InputTokensRequest 

func (r *ResponsesRequest) InputTokensRequest() *ResponseInputTokensRequest

InputTokensRequest reduces a full Responses request to the field set shared with the utility endpoints (the streaming controls are dropped — utility endpoints never stream). ExtraFields are cloned. The reduction is defined here, next to the types, so the field knowledge stays in one file; a guard test asserts it covers every non-streaming ResponsesRequest field.

func (ResponsesRequest) MarshalJSON 

func (r ResponsesRequest) MarshalJSON() ([]byte, error)

MarshalJSON preserves dynamic input payloads while supporting Swagger-only schema fields. alias inherits every field and json tag from ResponsesRequest but drops the MarshalJSON method (so json.Marshal does not recurse); ExtraFields is json:"-" and merged in separately. New typed fields round-trip automatically.

func (*ResponsesRequest) UnmarshalJSON 

func (r *ResponsesRequest) UnmarshalJSON(data []byte) error

UnmarshalJSON preserves dynamic input payloads while supporting Swagger-only schema fields. Array inputs are deserialized as []ResponsesInputElement for type-safe downstream handling. The body decodes through an alias embedding so every typed field (present and future) is populated by the JSON package directly — only Input (a raw union) and ExtraFields need explicit handling.

func (*ResponsesRequest) WithStreaming 

func (r *ResponsesRequest) WithStreaming() *ResponsesRequest

WithStreaming returns a shallow copy of the request with Stream set to true. This avoids mutating the caller's request object.

type ResponsesResponse 

type ResponsesResponse struct {
	ID        string                `json:"id"`
	Object    string                `json:"object"` // "response"
	CreatedAt int64                 `json:"created_at"`
	Model     string                `json:"model"`
	Provider  string                `json:"provider"`
	Status    string                `json:"status"` // "completed", "failed", "in_progress"
	Output    []ResponsesOutputItem `json:"output"`
	Usage     *ResponsesUsage       `json:"usage,omitempty"`
	Error     *ResponsesError       `json:"error,omitempty"`
}

ResponsesResponse represents the response from the Responses API.

type ResponsesUsage 

type ResponsesUsage struct {
	InputTokens             int                      `json:"input_tokens"`
	OutputTokens            int                      `json:"output_tokens"`
	TotalTokens             int                      `json:"total_tokens"`
	PromptTokensDetails     *PromptTokensDetails     `json:"prompt_tokens_details,omitempty"`
	CompletionTokensDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
	RawUsage                map[string]any           `json:"raw_usage,omitempty"`
}

ResponsesUsage represents token usage for the Responses API.

func (ResponsesUsage) MarshalJSON 

func (u ResponsesUsage) MarshalJSON() ([]byte, error)

func (*ResponsesUsage) UnmarshalJSON 

func (u *ResponsesUsage) UnmarshalJSON(data []byte) error

type RoutablePassthrough 

type RoutablePassthrough interface {
	Passthrough(ctx context.Context, providerType string, req *PassthroughRequest) (*PassthroughResponse, error)
}

RoutablePassthrough resolves a provider type before issuing an opaque passthrough request.

type RoutableProvider 

type RoutableProvider interface {
	Provider

	Supports(model string) bool
	GetProviderType(model string) string
}

RoutableProvider extends Provider with routing capability. This is implemented by the Router which uses a model registry to determine if a model is supported.

type RouteHints 

type RouteHints struct {
	Model    string
	Provider string
	Endpoint string
}

RouteHints holds minimal routing-relevant request hints derived from the transport snapshot.

These hints are intentionally smaller than a full semantic interpretation.

Lifecycle:

  • DeriveWhiteBoxPrompt seeds these values directly from transport/body data.
  • Canonical JSON decode may refine them from a cached request object.
  • Selector normalization (ParseModelSelector / RequestedModelSelector.Normalize) canonicalizes model/provider values in place.

Consumers that require canonical selector state should prefer a cached canonical request or normalize the selector before relying on these fields.

type StreamOptions 

type StreamOptions struct {
	// IncludeUsage requests token usage information in streaming responses.
	// When true, the final streaming chunk will include usage statistics.
	IncludeUsage bool `json:"include_usage,omitempty"`
}

StreamOptions controls streaming behavior options. This is used to request usage data in streaming responses.

type ToolCall 

type ToolCall struct {
	ID          string            `json:"id"`
	Type        string            `json:"type"`
	Function    FunctionCall      `json:"function"`
	ExtraFields UnknownJSONFields `json:"-" swaggerignore:"true"`
}

ToolCall represents a single tool invocation emitted by a model.

func (ToolCall) MarshalJSON 

func (t ToolCall) MarshalJSON() ([]byte, error)

ToolCall.MarshalJSON marshals a ToolCall to JSON, including unknown JSON members from ExtraFields. alias inherits ToolCall's fields and tags but drops MarshalJSON so json.Marshal does not recurse; ExtraFields (json:"-") is merged separately.

func (*ToolCall) UnmarshalJSON 

func (t *ToolCall) UnmarshalJSON(data []byte) error

ToolCall.UnmarshalJSON unmarshals a ToolCall from JSON, preserving unknown JSON members in ExtraFields.

type UnknownJSONFields 

type UnknownJSONFields struct {
	// contains filtered or unexported fields
}

UnknownJSONFields stores unknown JSON object members as a single raw object. This avoids allocating a map for every decoded chat-family request while still allowing lookups and round-trip preservation when needed.

func CloneUnknownJSONFields 

func CloneUnknownJSONFields(fields UnknownJSONFields) UnknownJSONFields

CloneUnknownJSONFields returns a detached copy of a raw unknown-field object.

func MergeUnknownJSONFields 

func MergeUnknownJSONFields(base UnknownJSONFields, additions map[string]json.RawMessage) (UnknownJSONFields, error)

MergeUnknownJSONFields returns base with the given raw members added; additions override existing members on key conflict. It lets translation layers inject derived fields (such as a chat response_format mapped from a Responses text format) into a request's passthrough object without a dedicated typed field.

func UnknownJSONFieldsFromMap 

func UnknownJSONFieldsFromMap(fields map[string]json.RawMessage) UnknownJSONFields

UnknownJSONFieldsFromMap converts a raw field map into a compact JSON object.

func (UnknownJSONFields) IsEmpty 

func (fields UnknownJSONFields) IsEmpty() bool

IsEmpty reports whether the container has no stored fields.

func (UnknownJSONFields) Lookup 

func (fields UnknownJSONFields) Lookup(key string) json.RawMessage

Lookup returns the raw JSON value for key or nil when absent. It scans the stored object on demand so single-lookups stay allocation-light, but repeated lookups on the same value are linear in the raw JSON size.

type Usage 

type Usage struct {
	PromptTokens            int                      `json:"prompt_tokens"`
	CompletionTokens        int                      `json:"completion_tokens"`
	TotalTokens             int                      `json:"total_tokens"`
	PromptTokensDetails     *PromptTokensDetails     `json:"prompt_tokens_details,omitempty"`
	CompletionTokensDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`
	RawUsage                map[string]any           `json:"raw_usage,omitempty"`
}

Usage represents token usage information

func (Usage) MarshalJSON 

func (u Usage) MarshalJSON() ([]byte, error)

func (*Usage) UnmarshalJSON 

func (u *Usage) UnmarshalJSON(data []byte) error

type WhiteBoxPrompt 

type WhiteBoxPrompt struct {
	RouteType     string
	OperationType string
	RouteHints    RouteHints
	// StreamRequested reports that the inbound request explicitly asked for
	// streaming semantics. This is request intent, not endpoint capability.
	StreamRequested bool
	// JSONBodyParsed reports that the captured request body was parsed as JSON
	// (for selector peeking and/or canonical request decode).
	JSONBodyParsed bool
	// contains filtered or unexported fields
}

WhiteBoxPrompt is the gateway's best-effort semantic extraction from the transport snapshot. It may be partial and should not be treated as authoritative transport state.

The semantics are populated incrementally:

  • transport seeds RouteType/OperationType plus sparse RouteHints
  • route-specific metadata may be cached on demand
  • canonical request decode may cache a parsed request and refine RouteHints
  • selector normalization may rewrite selector hints into canonical form

func DeriveWhiteBoxPrompt 

func DeriveWhiteBoxPrompt(snapshot *RequestSnapshot) *WhiteBoxPrompt

DeriveWhiteBoxPrompt derives best-effort request semantics from the captured transport snapshot. Unknown or invalid bodies are tolerated; the returned envelope may be partial.

func GetWhiteBoxPrompt 

func GetWhiteBoxPrompt(ctx context.Context) *WhiteBoxPrompt

GetWhiteBoxPrompt retrieves the white-box prompt from the context.

func (*WhiteBoxPrompt) CachedBatchRequest 

func (env *WhiteBoxPrompt) CachedBatchRequest() *BatchRequest

CachedBatchRequest returns the cached canonical batch create request, if present.

func (*WhiteBoxPrompt) CachedBatchRouteInfo 

func (env *WhiteBoxPrompt) CachedBatchRouteInfo() *BatchRouteInfo

CachedBatchRouteInfo returns cached sparse batch route info, if present.

func (*WhiteBoxPrompt) CachedChatRequest 

func (env *WhiteBoxPrompt) CachedChatRequest() *ChatRequest

CachedChatRequest returns the cached canonical chat request, if present.

func (*WhiteBoxPrompt) CachedEmbeddingRequest 

func (env *WhiteBoxPrompt) CachedEmbeddingRequest() *EmbeddingRequest

CachedEmbeddingRequest returns the cached canonical embeddings request, if present.

func (*WhiteBoxPrompt) CachedFileRouteInfo 

func (env *WhiteBoxPrompt) CachedFileRouteInfo() *FileRouteInfo

CachedFileRouteInfo returns cached sparse file route info, if present.

func (*WhiteBoxPrompt) CachedPassthroughRouteInfo 

func (env *WhiteBoxPrompt) CachedPassthroughRouteInfo() *PassthroughRouteInfo

CachedPassthroughRouteInfo returns cached typed passthrough route info, if present.

func (*WhiteBoxPrompt) CachedResponsesRequest 

func (env *WhiteBoxPrompt) CachedResponsesRequest() *ResponsesRequest

CachedResponsesRequest returns the cached canonical responses request, if present.

func (*WhiteBoxPrompt) CanonicalSelectorFromCachedRequest 

func (env *WhiteBoxPrompt) CanonicalSelectorFromCachedRequest() (model, provider string, ok bool)

CanonicalSelectorFromCachedRequest returns model/provider selector hints from any cached canonical JSON request for the current operation kind.

type Workflow 

type Workflow struct {
	RequestID    string
	Endpoint     EndpointDescriptor
	Mode         ExecutionMode
	Capabilities CapabilitySet
	ProviderType string
	Passthrough  *PassthroughRouteInfo
	Resolution   *RequestModelResolution
	Policy       *ResolvedWorkflowPolicy
}

Workflow is the request-scoped control-plane result consumed by later execution stages. It carries the resolved execution mode, endpoint capabilities, and any model routing decision already made for the request.

func GetWorkflow 

func GetWorkflow(ctx context.Context) *Workflow

GetWorkflow retrieves the workflow from the context.

func (*Workflow) AuditEnabled 

func (p *Workflow) AuditEnabled() bool

AuditEnabled reports whether audit logging is enabled for the request.

func (*Workflow) BudgetEnabled 

func (p *Workflow) BudgetEnabled() bool

BudgetEnabled reports whether budget checks are enabled for the request.

func (*Workflow) CacheEnabled 

func (p *Workflow) CacheEnabled() bool

CacheEnabled reports whether response caching is enabled for the request.

func (*Workflow) FailoverEnabled 

func (p *Workflow) FailoverEnabled() bool

FailoverEnabled reports whether translated-route failover is enabled for the request.

func (*Workflow) GuardrailsEnabled 

func (p *Workflow) GuardrailsEnabled() bool

GuardrailsEnabled reports whether guardrail processing is enabled for the request.

func (*Workflow) GuardrailsHash 

func (p *Workflow) GuardrailsHash() string

GuardrailsHash returns the matched workflow's guardrails hash.

func (*Workflow) RequestedQualifiedModel 

func (p *Workflow) RequestedQualifiedModel() string

RequestedQualifiedModel returns the requested model selector when present.

func (*Workflow) ResolvedQualifiedModel 

func (p *Workflow) ResolvedQualifiedModel() string

ResolvedQualifiedModel returns the resolved model selector when present.

func (*Workflow) UsageEnabled 

func (p *Workflow) UsageEnabled() bool

UsageEnabled reports whether usage tracking is enabled for the request.

func (*Workflow) WorkflowVersionID 

func (p *Workflow) WorkflowVersionID() string

WorkflowVersionID returns the matched immutable workflow version id.

type WorkflowFeatures 

type WorkflowFeatures struct {
	Cache      bool `json:"cache"`
	Audit      bool `json:"audit"`
	Usage      bool `json:"usage"`
	Budget     bool `json:"budget"`
	Guardrails bool `json:"guardrails"`
	Failover   bool `json:"failover"`
}

WorkflowFeatures stores resolved per-request feature flags sourced from the matched persisted workflow.

func DefaultWorkflowFeatures 

func DefaultWorkflowFeatures() WorkflowFeatures

DefaultWorkflowFeatures returns the permissive runtime default used when no persisted workflow has been attached to the request.

func (WorkflowFeatures) ApplyUpperBound 

func (f WorkflowFeatures) ApplyUpperBound(caps WorkflowFeatures) WorkflowFeatures

ApplyUpperBound returns features with process-level caps applied.

type WorkflowSelector 

type WorkflowSelector struct {
	// Provider is the configured provider instance name used for workflow matching.
	Provider string
	Model    string
	UserPath string
}

WorkflowSelector contains the request facts used to match one persisted workflow version.

func NewWorkflowSelector 

func NewWorkflowSelector(provider, model string, userPath ...string) WorkflowSelector

NewWorkflowSelector trims selector inputs for deterministic matching.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.