server

Documentation

Overview

Package server implements the HTTP transport layer for the Gandalf gateway.

Index

• func New(deps Deps) http.Handler
• type Cache
• type Deps
• type KeyInvalidator
• type QuotaChecker
• type ReadyChecker
• type TokenCounter
• type UsageRecorder

Functions

func New

func New(deps Deps) http.Handler

New creates an http.Handler with all routes and middleware wired.

Types

type Cache

type Cache interface {
	Get(ctx context.Context, key string) ([]byte, bool)
	Set(ctx context.Context, key string, val []byte, ttl time.Duration)
	Delete(ctx context.Context, key string)
	Purge(ctx context.Context)
}

Cache is the interface for response caching used by the server.

type Deps

type Deps struct {
	Auth           gateway.Authenticator
	Proxy          *app.ProxyService
	Providers      *provider.Registry // needed for NativeProxy type assertion
	Router         *app.RouterService // needed for model -> provider routing
	Keys           *app.KeyManager
	Store          storage.Store       // nil = no admin CRUD (for tests)
	Metrics        *telemetry.Metrics  // nil = no Prometheus metrics
	MetricsHandler http.Handler        // nil = no /metrics endpoint
	Tracer         trace.Tracer        // nil = no distributed tracing
	ReadyCheck     ReadyChecker        // nil = always ready (for tests)
	Usage          UsageRecorder       // nil = no usage recording
	RateLimiter    *ratelimit.Registry // nil = no rate limiting
	TokenCounter   TokenCounter        // nil = fixed estimate
	Cache          Cache               // nil = no caching
	Quota          QuotaChecker        // nil = no quota enforcement
	KeyInvalidator KeyInvalidator      // nil = no auth cache invalidation
	DefaultRPM     int64               // fallback RPM when per-key is 0
	DefaultTPM     int64               // fallback TPM when per-key is 0
}

Deps holds all dependencies for the HTTP server.

type KeyInvalidator

type KeyInvalidator interface {
	InvalidateByKeyID(keyID string)
}

KeyInvalidator invalidates cached auth entries when keys are modified.

type QuotaChecker

type QuotaChecker interface {
	Check(keyID string, limit float64) bool
	Consume(keyID string, costUSD float64)
}

QuotaChecker verifies and tracks spend budgets.

type ReadyChecker

type ReadyChecker func(ctx context.Context) error

ReadyChecker reports whether the system is ready to serve traffic.

type TokenCounter

type TokenCounter interface {
	EstimateRequest(model string, messages []gateway.Message) int
}

TokenCounter estimates token counts for request messages.

type UsageRecorder

type UsageRecorder interface {
	Record(gateway.UsageRecord)
}

UsageRecorder records API usage asynchronously.