server

package
v0.1.56 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 21, 2026 License: MIT Imports: 55 Imported by: 0

Documentation

Overview

Package server provides HTTP handlers and server setup for the LLM gateway.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AuthMiddlewareWithAuthenticator 

func AuthMiddlewareWithAuthenticator(masterKey string, authenticator BearerTokenAuthenticator, skipPaths []string, userPathHeader ...string) echo.MiddlewareFunc

AuthMiddlewareWithAuthenticator creates an Echo middleware that validates the legacy master key and, when configured, managed auth keys from the auth key service. If no auth mechanism is configured, no authentication is required. skipPaths is a list of paths that should bypass authentication.

func PassthroughSemanticEnrichment 

func PassthroughSemanticEnrichment(provider core.RoutableProvider, enrichers []core.PassthroughSemanticEnricher, allowPassthroughV1Alias bool) echo.MiddlewareFunc

PassthroughSemanticEnrichment applies provider-owned passthrough metadata enrichment before workflow resolution runs.

func RequestRewriteMiddleware 

func RequestRewriteMiddleware(rewriters []ext.RequestRewriter, auditLogger auditlog.LoggerInterface) echo.MiddlewareFunc

RequestRewriteMiddleware invokes registered ext.RequestRewriter extensions on the raw JSON body of inference requests. It must run after authentication (rewriters only see authenticated traffic and the final user path) and before workflow resolution (so body changes, including the "model" field, affect routing, failover, guardrails, budgets, and caching).

Rewriter errors fail the request (fail-closed). When a rewriter changes the body, the audit entry is pinned to the original client body first so operators always see what the client actually sent.

func RequestSnapshotCapture 

func RequestSnapshotCapture(userPathHeader ...string) echo.MiddlewareFunc

RequestSnapshotCapture captures immutable transport-level request data for model-facing endpoints. Known-small JSON bodies are captured once for the hot path; larger or unknown-size bodies only get a bounded selector peek and stay on the live request stream until the handler actually decodes them.

func SwaggerAvailable 

func SwaggerAvailable() bool

SwaggerAvailable reports whether this binary was built with Swagger UI support.

func TaggingCapture 

func TaggingCapture(service *tagging.Service) echo.MiddlewareFunc

TaggingCapture extracts request labels from the configured tagging headers and attaches them, together with the do-not-pass strip set, to the request context. It runs after RequestSnapshotCapture so audit logging still sees the original headers; stripping happens at the provider forwarding boundary.

func WorkflowResolutionWithResolverAndPolicy 

func WorkflowResolutionWithResolverAndPolicy(
	provider core.RoutableProvider,
	resolver RequestModelResolver,
	policyResolver RequestWorkflowPolicyResolver,
) echo.MiddlewareFunc

WorkflowResolutionWithResolverAndPolicy resolves the request-scoped workflow for model-facing routes and matches one persisted workflow policy when configured. The workflow centralizes endpoint capabilities, execution mode, resolved provider type, and any early model routing decision that downstream handlers or middleware need to consume. An explicit selector resolver lets workflow resolution own alias policy instead of depending on provider decorators.

Types

type BatchRequestPreparer 

type BatchRequestPreparer = gateway.BatchRequestPreparer

BatchRequestPreparer rewrites a native batch request before provider submission. This keeps batch-specific policy out of provider decorators.

func ComposeBatchRequestPreparers 

func ComposeBatchRequestPreparers(fileTransport core.NativeFileRoutableProvider, preparers ...BatchRequestPreparer) BatchRequestPreparer

ComposeBatchRequestPreparers runs explicit batch preparers in order and cleans up superseded rewritten input files between stages.

type BearerTokenAuthenticator 

type BearerTokenAuthenticator interface {
	Enabled() bool
	Authenticate(ctx context.Context, token string) (authkeys.AuthenticationResult, error)
}

BearerTokenAuthenticator authenticates managed bearer tokens and returns their internal auth key metadata on success.

type BudgetChecker 

type BudgetChecker interface {
	Check(ctx context.Context, userPath string, now time.Time) error
}

type Config 

type Config struct {
	BasePath                        string                                 // URL path prefix where the app is mounted (default: /)
	MasterKey                       string                                 // Optional: Master key for authentication
	Authenticator                   BearerTokenAuthenticator               // Optional: managed API key authenticator
	MetricsEnabled                  bool                                   // Whether to expose Prometheus metrics endpoint
	MetricsEndpoint                 string                                 // HTTP path for metrics endpoint (default: /metrics)
	BodySizeLimit                   string                                 // Max request body size (e.g., "10M", "1024K")
	PprofEnabled                    bool                                   // Whether to expose debug profiling routes at /debug/pprof/*
	AuditLogger                     auditlog.LoggerInterface               // Optional: Audit logger for request/response logging
	UsageLogger                     usage.LoggerInterface                  // Optional: Usage logger for token tracking
	BudgetChecker                   BudgetChecker                          // Optional: per-user-path budget checker
	RateLimiter                     RateLimiter                            // Optional: per-user-path rate limiter
	UsageSummarizer                 UsageSummarizer                        // Optional: usage aggregates for the self-service GET /v1/usage endpoint
	PricingResolver                 usage.PricingResolver                  // Optional: Resolves pricing for cost calculation
	ModelResolver                   RequestModelResolver                   // Optional: explicit model resolver used during workflow resolution
	ModelAuthorizer                 RequestModelAuthorizer                 // Optional: request-scoped concrete model access controller
	WorkflowPolicyResolver          RequestWorkflowPolicyResolver          // Optional: persisted workflow resolver used during workflow resolution
	FailoverResolver                RequestFailoverResolver                // Optional: translated-route failover resolver
	TranslatedRequestPatcher        TranslatedRequestPatcher               // Optional: request patcher for translated routes after workflow resolution
	BatchRequestPreparer            BatchRequestPreparer                   // Optional: batch request preparer before native provider submission
	ExposedModelLister              ExposedModelLister                     // Optional: additional public models to merge into GET /v1/models
	KeepOnlyAliasesAtModelsEndpoint bool                                   // Whether GET /v1/models should hide concrete provider models
	PassthroughSemanticEnrichers    []core.PassthroughSemanticEnricher     // Optional: provider-owned passthrough semantic enrichers before workflow resolution
	BatchStore                      batchstore.Store                       // Optional: Batch lifecycle persistence store
	FileStore                       filestore.Store                        // Optional: File provider mapping persistence store
	ResponseStore                   responsestore.Store                    // Optional: Responses lifecycle persistence store
	ConversationStore               conversationstore.Store                // Optional: Conversations lifecycle persistence store
	LogOnlyModelInteractions        bool                                   // Only log AI model endpoints (default: true)
	DisablePassthroughRoutes        bool                                   // Disable /p/{provider}/{endpoint} route registration
	RealtimeEnabled                 bool                                   // Enable realtime websocket route /v1/realtime and passthrough upgrades
	MCPEnabled                      bool                                   // Enable the MCP gateway routes /mcp and /mcp/{server}
	MCPGateway                      *mcpgateway.Service                    // MCP gateway service (nil if disabled or not wired)
	EnabledPassthroughProviders     []string                               // Provider types enabled on /p/{provider}/... passthrough routes
	AllowPassthroughV1Alias         *bool                                  // Allow /p/{provider}/v1/... aliases; nil defaults to true
	UserPathHeader                  string                                 // Header carrying the request user path (default: X-GoModel-User-Path)
	AdminEndpointsEnabled           bool                                   // Whether admin API endpoints are enabled
	AdminUIEnabled                  bool                                   // Whether admin dashboard UI is enabled
	AdminHandler                    *admin.Handler                         // Admin API handler (nil if disabled)
	DashboardHandler                *dashboard.Handler                     // Dashboard UI handler (nil if disabled)
	SwaggerEnabled                  bool                                   // Whether to expose the Swagger UI at /swagger/index.html
	ResponseCacheMiddleware         *responsecache.ResponseCacheMiddleware // Optional: response cache middleware for cacheable endpoints
	GuardrailsHash                  string                                 // Optional: SHA-256 hash of active guardrail rules; stored in context post-patch for semantic cache
	IPExtractor                     echo.IPExtractor                       // Optional: trusted client IP extraction strategy for proxied deployments
	StorageProbe                    ReadinessProbe                         // Optional: primary storage connectivity check; failure makes /health/ready report not_ready (503)
	CacheProbe                      ReadinessProbe                         // Optional: Redis cache connectivity check; failure makes /health/ready report degraded (200, non-blocking)
	RequestRewriters                []ext.RequestRewriter                  // Optional: raw-body rewriters invoked on inference ingress (post-auth, pre-workflow-resolution)
	ExtraMiddleware                 []echo.MiddlewareFunc                  // Optional: extension middleware registered after audit, before gateway auth
	ExtraRoutes                     []func(*echo.Echo)                     // Optional: extension route registration callbacks invoked after core routes
	ExtraAuthSkipPaths              []string                               // Optional: extension paths appended to the auth skip list ("/*" suffix matches a prefix)
	Tagging                         *tagging.Service                       // Optional: request labelling based on configured tagging headers
}

Config holds server configuration options

type ExposedModelLister 

type ExposedModelLister interface {
	ExposedModels() []core.Model
}

ExposedModelLister surfaces additional public models to include in GET /v1/models.

type FilteredExposedModelLister 

type FilteredExposedModelLister interface {
	ExposedModelsFiltered(allow func(core.ModelSelector) bool) []core.Model
}

FilteredExposedModelLister optionally filters exposed models using their concrete targets.

type Handler 

type Handler struct {
	// contains filtered or unexported fields
}

Handler holds the HTTP handlers

func (*Handler) AudioSpeech 

func (h *Handler) AudioSpeech(c *echo.Context) error

AudioSpeech handles POST /v1/audio/speech.

@Summary Create speech (text-to-speech) @Tags audio @Accept json @Produce application/octet-stream @Security BearerAuth @Param request body core.AudioSpeechRequest true "Text-to-speech request" @Success 200 {file} file "Binary audio in the requested response_format" @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/audio/speech [post]

func (*Handler) AudioTranscriptions 

func (h *Handler) AudioTranscriptions(c *echo.Context) error

AudioTranscriptions handles POST /v1/audio/transcriptions.

@Summary Create transcription (speech-to-text) @Tags audio @Accept mpfd @Produce json @Produce plain @Security BearerAuth @Param file formData file true "Audio file to transcribe" @Param model formData string true "Model ID" @Param language formData string false "Input language (ISO-639-1)" @Param prompt formData string false "Optional text to guide the model" @Param response_format formData string false "json, text, srt, verbose_json, or vtt" @Param temperature formData number false "Sampling temperature (0-1)" @Param timestamp_granularities[] formData []string false "Timestamp granularities to populate: word and/or segment" @Success 200 {object} map[string]interface{} "Transcription in the requested response_format: a JSON object for json/verbose_json, or a text/plain body for text/srt/vtt" @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/audio/transcriptions [post]

func (*Handler) BatchResults 

func (h *Handler) BatchResults(c *echo.Context) error

BatchResults handles GET /v1/batches/{id}/results.

@Summary Get batch results @Tags batch @Produce json @Security BearerAuth @Param id path string true "Batch ID" @Success 200 {object} core.BatchResultsResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 409 {object} core.OpenAIErrorEnvelope @Failure 500 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/batches/{id}/results [get]

func (*Handler) Batches 

func (h *Handler) Batches(c *echo.Context) error

Batches handles POST /v1/batches.

OpenAI-compatible fields are accepted (`input_file_id`, `endpoint`, `completion_window`, `metadata`). Inline `requests` are also accepted for providers with native inline batch support (for example Anthropic).

@Summary Create a native provider batch @Tags batch @Accept json @Produce json @Security BearerAuth @Param request body core.BatchRequest true "Batch request" @Success 200 {object} core.BatchResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/batches [post]

func (*Handler) CancelBatch 

func (h *Handler) CancelBatch(c *echo.Context) error

CancelBatch handles POST /v1/batches/{id}/cancel.

@Summary Cancel a batch @Tags batch @Accept json @Produce json @Security BearerAuth @Param id path string true "Batch ID" @Success 200 {object} core.BatchResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 500 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/batches/{id}/cancel [post]

func (*Handler) CancelMessagesBatch added in v0.1.53 

func (h *Handler) CancelMessagesBatch(c *echo.Context) error

CancelMessagesBatch handles POST /v1/messages/batches/{id}/cancel.

@Summary Cancel a message batch @Tags messages @Produce json @Security BearerAuth @Param id path string true "Message batch ID" @Success 200 {object} anthropicapi.MessageBatch @Failure 401 {object} anthropicapi.ErrorResponse @Failure 404 {object} anthropicapi.ErrorResponse @Router /v1/messages/batches/{id}/cancel [post]

func (*Handler) CancelResponse 

func (h *Handler) CancelResponse(c *echo.Context) error

CancelResponse handles POST /v1/responses/{id}/cancel.

@Summary Cancel a response @Tags responses @Produce json @Security BearerAuth @Param id path string true "Response ID" @Param provider query string false "Provider override for native cancellation" @Success 200 {object} core.ResponsesResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 501 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/responses/{id}/cancel [post]

func (*Handler) ChatCompletion 

func (h *Handler) ChatCompletion(c *echo.Context) error

ChatCompletion handles POST /v1/chat/completions

@Summary Create a chat completion @Tags chat @Accept json @Produce json @Produce text/event-stream @Security BearerAuth @Param request body core.ChatRequest true "Chat completion request" @Success 200 {object} core.ChatResponse "JSON response or SSE stream when stream=true" @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 429 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/chat/completions [post]

func (*Handler) CompactResponse 

func (h *Handler) CompactResponse(c *echo.Context) error

CompactResponse handles POST /v1/responses/compact.

@Summary Compact response input @Tags responses @Accept json @Produce json @Security BearerAuth @Param request body core.ResponseCompactRequest true "Response compact request" @Success 200 {object} core.ResponseCompactResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 501 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/responses/compact [post]

func (*Handler) CountMessageTokens 

func (h *Handler) CountMessageTokens(c *echo.Context) error

CountMessageTokens handles POST /v1/messages/count_tokens.

@Summary Count message tokens (Anthropic Messages API) @Description Returns a provider-agnostic heuristic estimate of the input token count. @Tags messages @Accept json @Produce json @Security BearerAuth @Param request body anthropicapi.MessagesRequest true "Anthropic Messages request" @Success 200 {object} anthropicapi.CountTokensResponse @Failure 400 {object} anthropicapi.ErrorResponse @Failure 401 {object} anthropicapi.ErrorResponse @Router /v1/messages/count_tokens [post]

func (*Handler) CreateConversation 

func (h *Handler) CreateConversation(c *echo.Context) error

CreateConversation handles POST /v1/conversations.

Conversations are a gateway-managed resource: GoModel generates the conversation id and stores the conversation locally, so the endpoint behaves identically regardless of which provider serves model traffic.

@Summary Create a conversation @Tags conversations @Accept json @Produce json @Security BearerAuth @Param request body core.ConversationCreateRequest false "Conversation create request" @Success 200 {object} core.Conversation @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 500 {object} core.OpenAIErrorEnvelope @Router /v1/conversations [post]

func (*Handler) CreateFile 

func (h *Handler) CreateFile(c *echo.Context) error

CreateFile handles POST /v1/files.

@Summary Upload a file @Tags files @Accept mpfd @Produce json @Security BearerAuth @Param provider query string false "Provider override when multiple providers are configured" @Param purpose formData string true "File purpose" @Param file formData file true "File to upload" @Success 200 {object} core.FileObject @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/files [post]

func (*Handler) DeleteConversation 

func (h *Handler) DeleteConversation(c *echo.Context) error

DeleteConversation handles DELETE /v1/conversations/{id}.

@Summary Delete a conversation @Tags conversations @Produce json @Security BearerAuth @Param id path string true "Conversation ID" @Success 200 {object} core.ConversationDeleteResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Router /v1/conversations/{id} [delete]

func (*Handler) DeleteFile 

func (h *Handler) DeleteFile(c *echo.Context) error

DeleteFile handles DELETE /v1/files/{id}.

@Summary Delete a file @Tags files @Produce json @Security BearerAuth @Param id path string true "File ID" @Param provider query string false "Provider override" @Success 200 {object} core.FileDeleteResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/files/{id} [delete]

func (*Handler) DeleteMessagesBatch added in v0.1.53 

func (h *Handler) DeleteMessagesBatch(c *echo.Context) error

DeleteMessagesBatch handles DELETE /v1/messages/batches/{id}.

@Summary Delete an ended message batch @Tags messages @Produce json @Security BearerAuth @Param id path string true "Message batch ID" @Success 200 {object} anthropicapi.DeletedMessageBatch @Failure 400 {object} anthropicapi.ErrorResponse @Failure 401 {object} anthropicapi.ErrorResponse @Failure 404 {object} anthropicapi.ErrorResponse @Router /v1/messages/batches/{id} [delete]

func (*Handler) DeleteResponse 

func (h *Handler) DeleteResponse(c *echo.Context) error

DeleteResponse handles DELETE /v1/responses/{id}.

@Summary Delete a response @Tags responses @Produce json @Security BearerAuth @Param id path string true "Response ID" @Param provider query string false "Provider override for native deletion" @Success 200 {object} core.ResponseDeleteResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 501 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/responses/{id} [delete]

func (*Handler) Embeddings 

func (h *Handler) Embeddings(c *echo.Context) error

Embeddings handles POST /v1/embeddings

@Summary Create embeddings @Tags embeddings @Accept json @Produce json @Security BearerAuth @Param request body core.EmbeddingRequest true "Embeddings request" @Success 200 {object} core.EmbeddingResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 429 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/embeddings [post]

func (*Handler) GetBatch 

func (h *Handler) GetBatch(c *echo.Context) error

GetBatch handles GET /v1/batches/{id}.

@Summary Get a batch @Tags batch @Produce json @Security BearerAuth @Param id path string true "Batch ID" @Success 200 {object} core.BatchResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 500 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/batches/{id} [get]

func (*Handler) GetConversation 

func (h *Handler) GetConversation(c *echo.Context) error

GetConversation handles GET /v1/conversations/{id}.

@Summary Get a conversation @Tags conversations @Produce json @Security BearerAuth @Param id path string true "Conversation ID" @Success 200 {object} core.Conversation @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Router /v1/conversations/{id} [get]

func (*Handler) GetFile 

func (h *Handler) GetFile(c *echo.Context) error

GetFile handles GET /v1/files/{id}.

@Summary Get file metadata @Tags files @Produce json @Security BearerAuth @Param id path string true "File ID" @Param provider query string false "Provider override" @Success 200 {object} core.FileObject @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/files/{id} [get]

func (*Handler) GetFileContent 

func (h *Handler) GetFileContent(c *echo.Context) error

GetFileContent handles GET /v1/files/{id}/content.

@Summary Download file content @Tags files @Produce application/octet-stream @Security BearerAuth @Param id path string true "File ID" @Param provider query string false "Provider override" @Success 200 {file} file "Raw file content" @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/files/{id}/content [get]

func (*Handler) GetMessagesBatch added in v0.1.53 

func (h *Handler) GetMessagesBatch(c *echo.Context) error

GetMessagesBatch handles GET /v1/messages/batches/{id}.

@Summary Get a message batch @Tags messages @Produce json @Security BearerAuth @Param id path string true "Message batch ID" @Success 200 {object} anthropicapi.MessageBatch @Failure 401 {object} anthropicapi.ErrorResponse @Failure 404 {object} anthropicapi.ErrorResponse @Router /v1/messages/batches/{id} [get]

func (*Handler) GetResponse 

func (h *Handler) GetResponse(c *echo.Context) error

GetResponse handles GET /v1/responses/{id}.

@Summary Get a response @Tags responses @Produce json @Security BearerAuth @Param id path string true "Response ID" @Param provider query string false "Provider override for native lookups" @Param include query []string false "Fields to include in the response" collectionFormat(multi) @Param include[] query []string false "Fields to include in the response" collectionFormat(multi) @Param include_obfuscation query bool false "Whether to include obfuscated response data" @Param starting_after query int false "Input item offset for providers that support it" @Success 200 {object} core.ResponsesResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 501 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/responses/{id} [get]

func (*Handler) Health 

func (h *Handler) Health(c *echo.Context) error

Health handles GET /health

@Summary Health check @Tags system @Produce json @Success 200 {object} map[string]string @Router /health [get]

func (*Handler) ListBatches 

func (h *Handler) ListBatches(c *echo.Context) error

ListBatches handles GET /v1/batches.

@Summary List batches @Tags batch @Produce json @Security BearerAuth @Param after query string false "Pagination cursor" @Param limit query int false "Maximum items to return (1-100, default 20)" @Success 200 {object} core.BatchListResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 500 {object} core.OpenAIErrorEnvelope @Router /v1/batches [get]

func (*Handler) ListFiles 

func (h *Handler) ListFiles(c *echo.Context) error

ListFiles handles GET /v1/files.

@Summary List files @Tags files @Produce json @Security BearerAuth @Param provider query string false "Provider filter" @Param purpose query string false "File purpose filter" @Param after query string false "Pagination cursor" @Param limit query int false "Maximum items to return (1-100, default 20)" @Success 200 {object} core.FileListResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/files [get]

func (*Handler) ListMessagesBatches added in v0.1.53 

func (h *Handler) ListMessagesBatches(c *echo.Context) error

ListMessagesBatches handles GET /v1/messages/batches.

@Summary List message batches @Tags messages @Produce json @Security BearerAuth @Param after_id query string false "Pagination cursor" @Param limit query int false "Maximum items to return (1-100, default 20)" @Success 200 {object} anthropicapi.MessageBatchList @Failure 401 {object} anthropicapi.ErrorResponse @Router /v1/messages/batches [get]

func (*Handler) ListModels 

func (h *Handler) ListModels(c *echo.Context) error

ListModels handles GET /v1/models

@Summary List available models @Tags models @Produce json @Security BearerAuth @Success 200 {object} core.ModelsResponse @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/models [get]

func (*Handler) ListResponseInputItems 

func (h *Handler) ListResponseInputItems(c *echo.Context) error

ListResponseInputItems handles GET /v1/responses/{id}/input_items.

@Summary List response input items @Tags responses @Produce json @Security BearerAuth @Param id path string true "Response ID" @Param provider query string false "Provider override for native lookups" @Param after query string false "Pagination cursor" @Param include query []string false "Fields to include in listed input items" collectionFormat(multi) @Param include[] query []string false "Fields to include in listed input items" collectionFormat(multi) @Param limit query int false "Maximum items to return (1-100, default 20)" @Param order query string false "Sort order: asc or desc" Enums(asc, desc) @Success 200 {object} core.ResponseInputItemListResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 501 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/responses/{id}/input_items [get]

func (*Handler) MCP 

func (h *Handler) MCP(c *echo.Context) error

MCP handles the aggregated MCP endpoint at /mcp.

@Summary MCP gateway (aggregated) @Description Streamable-HTTP MCP endpoint aggregating every configured upstream MCP server visible to the caller. Tools and prompts are namespaced as {slug}_{name}. POST carries JSON-RPC messages, GET opens the server-notification SSE stream, DELETE ends the session. The X-MCP-Servers request header optionally narrows the visible servers to a comma-separated subset of server slugs. @Tags mcp @Accept json @Produce json @Produce text/event-stream @Security BearerAuth @Success 200 {object} map[string]interface{} "JSON-RPC response or SSE stream" @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 429 {object} core.OpenAIErrorEnvelope @Failure 501 {object} core.OpenAIErrorEnvelope @Router /mcp [post] @Router /mcp [get] @Router /mcp [delete]

func (*Handler) MCPServer 

func (h *Handler) MCPServer(c *echo.Context) error

MCPServer handles the per-server MCP endpoints at /mcp/{server}.

@Summary MCP gateway (single server) @Description Streamable-HTTP MCP endpoint exposing one configured upstream MCP server with original (un-prefixed) tool names. @Tags mcp @Accept json @Produce json @Produce text/event-stream @Security BearerAuth @Param server path string true "Configured MCP server slug" @Success 200 {object} map[string]interface{} "JSON-RPC response or SSE stream" @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 429 {object} core.OpenAIErrorEnvelope @Failure 501 {object} core.OpenAIErrorEnvelope @Router /mcp/{server} [post] @Router /mcp/{server} [get] @Router /mcp/{server} [delete]

func (*Handler) Messages 

func (h *Handler) Messages(c *echo.Context) error

Messages handles POST /v1/messages.

It accepts the Anthropic Messages API request dialect, translates it to the canonical chat request, and runs it through the standard chat-completions pipeline so it routes to any configured provider with full workflow, budget, failover, cache, usage, and audit support. See ADR-0007.

@Summary Create a message (Anthropic Messages API) @Tags messages @Accept json @Produce json @Produce text/event-stream @Security BearerAuth @Param request body anthropicapi.MessagesRequest true "Anthropic Messages request" @Success 200 {object} anthropicapi.MessagesResponse "JSON response or SSE stream when stream=true" @Failure 400 {object} anthropicapi.ErrorResponse @Failure 401 {object} anthropicapi.ErrorResponse @Failure 429 {object} anthropicapi.ErrorResponse @Failure 502 {object} anthropicapi.ErrorResponse @Router /v1/messages [post]

func (*Handler) MessagesBatchResults added in v0.1.53 

func (h *Handler) MessagesBatchResults(c *echo.Context) error

MessagesBatchResults handles GET /v1/messages/batches/{id}/results.

@Summary Get message batch results (JSONL) @Tags messages @Produce application/x-jsonl @Security BearerAuth @Param id path string true "Message batch ID" @Success 200 {string} string "JSONL stream of batch results" @Failure 401 {object} anthropicapi.ErrorResponse @Failure 404 {object} anthropicapi.ErrorResponse @Failure 409 {object} anthropicapi.ErrorResponse @Router /v1/messages/batches/{id}/results [get]

func (*Handler) MessagesBatches added in v0.1.53 

func (h *Handler) MessagesBatches(c *echo.Context) error

MessagesBatches handles POST /v1/messages/batches.

@Summary Create a message batch (Anthropic Message Batches API) @Tags messages @Accept json @Produce json @Security BearerAuth @Param request body anthropicapi.BatchCreateRequest true "Anthropic Message Batches create request" @Success 200 {object} anthropicapi.MessageBatch @Failure 400 {object} anthropicapi.ErrorResponse @Failure 401 {object} anthropicapi.ErrorResponse @Failure 429 {object} anthropicapi.ErrorResponse @Failure 502 {object} anthropicapi.ErrorResponse @Router /v1/messages/batches [post]

func (*Handler) ProviderPassthrough 

func (h *Handler) ProviderPassthrough(c *echo.Context) error

ProviderPassthrough handles opaque provider-native requests under /p/{provider}/{endpoint}.

OpenAI and Anthropic are the first-class providers in this ADR-0002 slice. Other providers are intentionally deferred until they fit the same low-friction opaque path.

@Summary Provider passthrough @Description Runtime-configurable passthrough endpoint under /p/{provider}/{endpoint}; enabled by default via server.enable_passthrough_routes. The endpoint path is opaque and may proxy JSON, binary, or SSE responses with upstream status codes preserved. For multi-segment provider endpoints, clients that rely on OpenAPI-generated path handling should URL-encode embedded slashes in the endpoint parameter. A leading v1/ segment is normalized away by default so /p/{provider}/v1/... and /p/{provider}/... map to the same upstream path relative to the provider base URL. @Tags passthrough @Accept json @Accept mpfd @Produce json @Produce application/octet-stream @Produce text/event-stream @Security BearerAuth @Param provider path string true "Provider type" @Param endpoint path string true "Provider-native endpoint path relative to the provider base URL. URL-encode embedded / characters when using generated clients." @Success 200 {file} file "Opaque upstream response body" @Success 201 {file} file "Opaque upstream response body" @Success 202 {file} file "Opaque upstream response body" @Success 204 {string} string "No Content passthrough response" @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /p/{provider}/{endpoint} [get] @Router /p/{provider}/{endpoint} [post] @Router /p/{provider}/{endpoint} [put] @Router /p/{provider}/{endpoint} [patch] @Router /p/{provider}/{endpoint} [delete] @Router /p/{provider}/{endpoint} [head] @Router /p/{provider}/{endpoint} [options]

func (*Handler) Ready 

func (h *Handler) Ready(c *echo.Context) error

Ready handles GET /health/ready

Readiness reports whether this instance should receive traffic. It probes dependencies the gateway owns:

  • Storage is required: if it is unreachable the gateway cannot serve requests, so the response is not_ready (HTTP 503).
  • The Redis exact cache is a performance optimization: if it is unreachable the gateway still serves requests, so the response is degraded (HTTP 200).

Upstream provider reachability is deliberately excluded — a provider outage must not pull a healthy gateway out of rotation. Use GET /health for liveness.

@Summary Readiness check @Tags system @Produce json @Success 200 {object} map[string]interface{} "ready or degraded" @Failure 503 {object} map[string]interface{} "not ready" @Router /health/ready [get]

func (*Handler) Realtime 

func (h *Handler) Realtime(c *echo.Context) error

Realtime handles GET /v1/realtime.

@Summary Open a realtime session @Description Upgrades to a websocket and relays an OpenAI-compatible realtime (speech-to-speech) session to the provider that owns the model named in the ?model= query parameter. Provider credentials are injected by the gateway. Passing ?call_id= instead attaches to an existing WebRTC/SIP call as a sideband channel; calls created through this gateway instance are routed automatically, others need explicit model (and provider) parameters. @Tags realtime @Security BearerAuth @Param model query string false "Model that owns the realtime session (required unless call_id names a call created through this gateway instance)" @Param provider query string false "Optional provider hint" @Param call_id query string false "Existing WebRTC/SIP call to attach to as a sideband channel" @Success 101 {string} string "Switching Protocols" @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Failure 429 {object} core.OpenAIErrorEnvelope @Failure 501 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/realtime [get]

func (*Handler) RealtimeCalls 

func (h *Handler) RealtimeCalls(c *echo.Context) error

RealtimeCalls handles POST /v1/realtime/calls.

@Summary Create a realtime WebRTC call @Description OpenAI-compatible WebRTC SDP exchange. Accepts a raw application/sdp offer with the model in the ?model= query parameter, or a multipart form with sdp and session (JSON) fields. The gateway routes by model, injects provider credentials, and relays the SDP answer; the Location header carries the created call id. Media flows directly between the client and the provider, so usage is recorded by a gateway-side sideband observer when usage tracking is enabled. @Tags realtime @Accept application/sdp @Accept mpfd @Produce application/sdp @Produce json @Security BearerAuth @Param model query string false "Model that owns the call (required for application/sdp offers)" @Param provider query string false "Optional provider hint" @Param request body string true "SDP offer (raw application/sdp body), or a multipart form with sdp and session (JSON) fields" @Success 201 {string} string "SDP answer" @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 429 {object} core.OpenAIErrorEnvelope @Failure 501 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/realtime/calls [post]

func (*Handler) RealtimeClientSecrets 

func (h *Handler) RealtimeClientSecrets(c *echo.Context) error

RealtimeClientSecrets handles POST /v1/realtime/client_secrets.

@Summary Mint an ephemeral realtime client secret @Description OpenAI-compatible ephemeral credential minting for browser and mobile realtime clients. Routes by session.model (or the transcription model for transcription sessions), applies the same model-access, budget, and rate-limit gates as other model endpoints, and relays the provider response verbatim. The minted secret authenticates the client directly against the provider, bypassing the gateway for the session itself. @Tags realtime @Accept json @Produce json @Security BearerAuth @Param provider query string false "Optional provider hint" @Param request body object{session=object,expires_after=object} true "Client secret request: session config with the routing model (session.model, or the nested transcription model) plus optional expires_after; additional fields are relayed verbatim" @Success 200 {object} map[string]any "Provider client secret response" @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 429 {object} core.OpenAIErrorEnvelope @Failure 501 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/realtime/client_secrets [post]

func (*Handler) ResponseInputTokens 

func (h *Handler) ResponseInputTokens(c *echo.Context) error

ResponseInputTokens handles POST /v1/responses/input_tokens.

@Summary Count response input tokens @Tags responses @Accept json @Produce json @Security BearerAuth @Param request body core.ResponseInputTokensRequest true "Response input token request" @Success 200 {object} core.ResponseInputTokensResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 501 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/responses/input_tokens [post]

func (*Handler) Responses 

func (h *Handler) Responses(c *echo.Context) error

Responses handles POST /v1/responses

@Summary Create a model response (Responses API) @Tags responses @Accept json @Produce json @Produce text/event-stream @Security BearerAuth @Param request body core.ResponsesRequest true "Responses API request" @Success 200 {object} core.ResponsesResponse "JSON response or SSE stream when stream=true" @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 429 {object} core.OpenAIErrorEnvelope @Failure 502 {object} core.OpenAIErrorEnvelope @Router /v1/responses [post]

func (*Handler) SetBatchStore 

func (h *Handler) SetBatchStore(store batchstore.Store)

SetBatchStore replaces the batch store used by lifecycle endpoints. nil is ignored to keep an always-available fallback memory store.

func (*Handler) SetConversationStore 

func (h *Handler) SetConversationStore(store conversationstore.Store)

SetConversationStore replaces the conversation store used by the Conversations lifecycle endpoints and by /v1/responses conversation turns. nil is ignored to keep an always-available fallback memory store.

func (*Handler) SetFileStore 

func (h *Handler) SetFileStore(store filestore.Store)

SetFileStore replaces the file provider mapping store. nil is ignored to keep an always-available fallback memory store.

func (*Handler) SetResponseStore 

func (h *Handler) SetResponseStore(store responsestore.Store)

SetResponseStore replaces the response snapshot store used by lifecycle endpoints. nil is ignored to keep an always-available fallback memory store.

func (*Handler) UpdateConversation 

func (h *Handler) UpdateConversation(c *echo.Context) error

UpdateConversation handles POST /v1/conversations/{id}.

@Summary Update a conversation @Description Replaces the conversation metadata in full. @Tags conversations @Accept json @Produce json @Security BearerAuth @Param id path string true "Conversation ID" @Param request body core.ConversationUpdateRequest true "Conversation update request" @Success 200 {object} core.Conversation @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 404 {object} core.OpenAIErrorEnvelope @Router /v1/conversations/{id} [post]

func (*Handler) UsageStatus 

func (h *Handler) UsageStatus(c *echo.Context) error

UsageStatus handles GET /v1/usage.

@Summary Self-service usage, budget, and rate limit status @Description Returns recorded usage, budget statuses, and rate limit statuses for the caller's effective user path (the path bound to the managed API key, or the user-path header for master-key callers). @Tags usage @Produce json @Security BearerAuth @Param start_date query string false "Inclusive window start (YYYY-MM-DD, UTC); defaults to 29 days before end_date" @Param end_date query string false "Inclusive window end (YYYY-MM-DD, UTC); defaults to today; the whole range may span at most 365 days" @Param days query int false "Window length ending today when no explicit dates are given (default 30, max 365)" @Success 200 {object} usageStatusResponse @Failure 400 {object} core.OpenAIErrorEnvelope @Failure 401 {object} core.OpenAIErrorEnvelope @Failure 503 {object} core.OpenAIErrorEnvelope @Router /v1/usage [get]

type InternalChatCompletionExecutor 

type InternalChatCompletionExecutor struct {
	// contains filtered or unexported fields
}

InternalChatCompletionExecutor executes internal translated chat requests without synthesizing an HTTP request or Echo context.

func NewInternalChatCompletionExecutor 

func NewInternalChatCompletionExecutor(provider core.RoutableProvider, cfg InternalChatCompletionExecutorConfig) *InternalChatCompletionExecutor

NewInternalChatCompletionExecutor creates a transport-free translated chat executor that reuses workflow resolution, failover, usage, and audit logic.

func (*InternalChatCompletionExecutor) ChatCompletion 

func (e *InternalChatCompletionExecutor) ChatCompletion(ctx context.Context, req *core.ChatRequest) (resp *core.ChatResponse, err error)

ChatCompletion executes one internal translated chat request.

type InternalChatCompletionExecutorConfig 

type InternalChatCompletionExecutorConfig struct {
	ModelResolver          RequestModelResolver
	ModelAuthorizer        RequestModelAuthorizer
	WorkflowPolicyResolver RequestWorkflowPolicyResolver
	FailoverResolver       RequestFailoverResolver
	AuditLogger            auditlog.LoggerInterface
	UsageLogger            usage.LoggerInterface
	PricingResolver        usage.PricingResolver
	ResponseCache          *responsecache.ResponseCacheMiddleware
}

InternalChatCompletionExecutorConfig configures the transport-free translated chat execution path used by gateway-owned workflows such as guardrails.

type RateLimiter 

type RateLimiter interface {
	Acquire(subjects ratelimit.Subjects, now time.Time) (*ratelimit.Reservation, error)
	RouteAvailable(providerName, model string) bool
}

RateLimiter admits or rejects requests against configured rate limit rules. RouteAvailable additionally satisfies gateway.RouteGate so failover can skip saturated provider/model targets.

type ReadinessProbe 

type ReadinessProbe interface {
	Ping(ctx context.Context) error
}

ReadinessProbe verifies that a dependency the gateway owns is reachable. It is deliberately narrow so the server stays decoupled from concrete storage and cache types. Upstream provider reachability is intentionally NOT a probe: an external provider outage must not pull a healthy gateway out of rotation.

type RequestFailoverResolver 

type RequestFailoverResolver = gateway.FailoverResolver

RequestFailoverResolver resolves alternate concrete model selectors for a translated request after the primary selector has already been resolved.

type RequestModelAuthorizer 

type RequestModelAuthorizer = gateway.ModelAuthorizer

RequestModelAuthorizer validates request-scoped access to concrete models.

type RequestModelResolver 

type RequestModelResolver = gateway.ModelResolver

RequestModelResolver resolves raw request selectors into concrete model selectors before provider execution.

type RequestWorkflowPolicyResolver 

type RequestWorkflowPolicyResolver = gateway.WorkflowPolicyResolver

RequestWorkflowPolicyResolver matches persisted workflow versions for requests.

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server wraps the Echo server

func New 

func New(provider core.RoutableProvider, cfg *Config) *Server

New creates a new HTTP server

func (*Server) ServeHTTP 

func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP implements the http.Handler interface, allowing Server to be used with httptest

func (*Server) Shutdown 

func (s *Server) Shutdown(_ context.Context) error

Shutdown releases server resources. The HTTP server itself is stopped by cancelling the context passed to Start; this method drains any in-flight response cache writes, closes the cache store, and closes the response and conversation stores.

func (*Server) Start 

func (s *Server) Start(ctx context.Context, addr string) error

Start starts the HTTP server on the given address and exits when ctx is canceled.

func (*Server) StartWithListener 

func (s *Server) StartWithListener(ctx context.Context, listener net.Listener) error

StartWithListener starts the HTTP server using a pre-bound listener. This is useful in tests that need an already-reserved loopback port.

type TranslatedRequestPatcher 

type TranslatedRequestPatcher = gateway.TranslatedRequestPatcher

TranslatedRequestPatcher applies request-level transforms for translated routes after workflow resolution has resolved the concrete execution selector.

type UsageSummarizer 

type UsageSummarizer interface {
	GetSummary(ctx context.Context, params usage.UsageQueryParams) (*usage.UsageSummary, error)
}

UsageSummarizer aggregates recorded usage entries for the self-service usage endpoint. Usage readers satisfy it; the endpoint deliberately needs only the summary slice of the full reader interface.

type UserPathExposedModelLister 

type UserPathExposedModelLister interface {
	ExposedModelsForUserPath(userPath string, allow func(core.ModelSelector) bool) []core.Model
}

UserPathExposedModelLister optionally filters exposed models by the effective request user path in addition to their concrete targets, so a redirect scoped to user_paths is not listed to callers outside its scope.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.