provider

package
v0.1.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 25, 2026 License: MIT Imports: 19 Imported by: 0

Documentation

Overview

Package provider defines the LLM provider abstraction for IBEX Harness. All LLM communication goes through this interface.

Phase 2: OpenAI (+ mock) implementation. Phase 2.5: Anthropic adapter (OpenAI-compatible wire translation; ADR-0040) and ModelCapability registry (ADR-0041). Later: Azure OpenAI, AWS Bedrock, self-hosted OpenAI-compatible backends.

Index

Constants

View Source 
const (
	CapabilityProviderOpenAI    = "openai"
	CapabilityProviderAnthropic = "anthropic"
)

CapabilityProvider* are vendor-family values for ModelCapability.Provider.

View Source 
const (
	TokenizerFamilyO200kBase  = "o200k_base"
	TokenizerFamilyCL100kBase = "cl100k_base"
	TokenizerFamilyClaude     = "claude"
	// TokenizerFamilyUnknown is allowed only on explicit ExtraModels overlays,
	// never on built-in curated rows.
	TokenizerFamilyUnknown = "unknown"
)

Tokenizer family keys consumed by the Phase 2.5.G2 tokenizer registry.

View Source 
const (
	ErrorReasonQueueFull   = "queue_full"
	ErrorReasonCircuitOpen = "circuit_open"
)

Well-known ProviderError.Reason values for self-hosted backends.

View Source 
const (
	// DefaultMaxRetryBackoff caps exponential retry sleep across provider clients.
	DefaultMaxRetryBackoff = 30 * time.Second
)

Variables

View Source 
var ErrDuplicateModel = errors.New("provider model conflict")

ErrDuplicateModel is returned by NewRegistry when two providers claim the same model ID.

View Source 
var ErrInvalidCapability = errors.New("invalid model capability")

ErrInvalidCapability is returned when a capability row fails validation (malformed fields, tokenizer allowlist, ModelID mismatch, etc.).

View Source 
var ErrMissingCapability = errors.New("missing model capability")

ErrMissingCapability is returned by NewRegistry when a registered model has no capability entry in the catalog.

View Source 
var ErrNoProviderForModel = errors.New("no provider configured for this model")

ErrNoProviderForModel is returned when no registered provider supports a model. Callers must detect it with errors.Is(err, ErrNoProviderForModel) and map it to the provider-not-configured HTTP response (501 PROVIDER_NOT_CONFIGURED).

Functions

func AttachStreamCancel added in v0.1.4 

func AttachStreamCancel(resp *http.Response, stream bool, cancel context.CancelFunc) *http.Response

AttachStreamCancel wraps a streaming response body so Close cancels the request. For non-stream responses the cancel func is invoked immediately.

func DoUpstream added in v0.1.4 

func DoUpstream(
	ctx context.Context,
	streamTimeout time.Duration,
	httpClient, streamClient *http.Client,
	build func(context.Context, UpstreamCall) (*http.Request, error),
	call UpstreamCall,
) (*http.Response, error)

DoUpstream builds and executes one upstream request with stream-aware context/cancel.

func IsEventStream added in v0.1.4 

func IsEventStream(contentType string) bool

IsEventStream reports whether Content-Type is text/event-stream.

func IsRetryableHTTPStatus added in v0.1.4 

func IsRetryableHTTPStatus(code int, extra ...int) bool

IsRetryableHTTPStatus reports statuses that are safe to retry for Complete.

func IsRetryableTransport added in v0.1.4 

func IsRetryableTransport(err error) bool

IsRetryableTransport reports pre-delivery connection failures worth retrying. Timeouts and context errors are not retried: for non-idempotent POSTs the request may already have been accepted upstream (duplicate completion risk).

func JoinBaseURL added in v0.1.4 

func JoinBaseURL(base, path string) string

JoinBaseURL joins base and path with a single slash boundary.

func MapError added in v0.1.2 

func MapError(err error) (mapped *apierror.Error, write bool)

MapError is the handler entrypoint: unwraps ProviderError / deadline / cancel. write is false only for context.Canceled (caller must not write a response).

func MapProviderError added in v0.1.2 

func MapProviderError(in MapInput) *apierror.Error

MapProviderError translates provider HTTP status or transport failure into the canonical IBEX apierror.Error. Never exposes API keys or raw bodies. ProviderName is for caller logs only — not copied into the client envelope.

func MergeSupportedModels added in v0.1.4 

func MergeSupportedModels(base, extra []string) []string

MergeSupportedModels concatenates base and extra model IDs with trim + dedupe.

func NewJSONPostRequest added in v0.1.4 

func NewJSONPostRequest(ctx context.Context, call UpstreamCall, headers map[string]string) (*http.Request, error)

NewJSONPostRequest builds a JSON POST with optional stream Accept header.

func NewPooledHTTPClient added in v0.1.4 

func NewPooledHTTPClient(timeout time.Duration) *http.Client

NewPooledHTTPClient builds an upstream client with connection pooling defaults.

func NoopCancel added in v0.1.4 

func NoopCancel()

NoopCancel is used when a call shares the parent context (no derived deadline).

func RecordSpanErr added in v0.1.4 

func RecordSpanErr(span trace.Span, err error)

RecordSpanErr marks a span as failed when err is non-nil.

func RetryAfterFromError added in v0.1.4 

func RetryAfterFromError(lastErr error) time.Duration

RetryAfterFromError returns Retry-After for rate-limit / overload / unavailable errors.

func RetryAfterHeader added in v0.1.4 

func RetryAfterHeader(hdr string) time.Duration

RetryAfterHeader parses the Retry-After response header when present.

func RetryDelay added in v0.1.4 

func RetryDelay(base time.Duration, attempt int, maxBackoff time.Duration) time.Duration

RetryDelay computes exponential backoff with jitter, capped at maxBackoff.

func StartCompleteSpan added in v0.1.4 

func StartCompleteSpan(ctx context.Context, tracer trace.Tracer, span CompleteSpan) (context.Context, trace.Span)

StartCompleteSpan starts the standard provider Complete span attributes.

func StatusClass added in v0.1.4 

func StatusClass(code int) string

StatusClass maps an HTTP status to a coarse metrics label.

func StreamHTTPClient added in v0.1.4 

func StreamHTTPClient(base *http.Client) *http.Client

StreamHTTPClient returns a client that shares Transport but has no Client.Timeout so long-lived SSE streams are bounded only by request context.

func StreamRequestContext added in v0.1.4 

func StreamRequestContext(ctx context.Context, stream bool, streamTimeout time.Duration) (context.Context, context.CancelFunc)

StreamRequestContext derives a timeout only for streaming calls.

func TracerOrNoop added in v0.1.4 

func TracerOrNoop(tracer trace.Tracer, name string) trace.Tracer

TracerOrNoop returns tracer, or a noop tracer named name when tracer is nil.

func ValidateBuiltinCapability added in v0.1.4 

func ValidateBuiltinCapability(cap ModelCapability) error

ValidateBuiltinCapability is ValidateCapability plus rejection of TokenizerFamilyUnknown (built-in curated rows must name a real family).

func ValidateCapability added in v0.1.4 

func ValidateCapability(cap ModelCapability) error

ValidateCapability checks required fields for a catalog row (overlay-safe). TokenizerFamily must be one of the declared family constants (including TokenizerFamilyUnknown for ExtraModels overlays).

func WaitBeforeRetry added in v0.1.4 

func WaitBeforeRetry(ctx context.Context, base time.Duration, attempt int, lastErr error) error

WaitBeforeRetry sleeps for backoff (honoring Retry-After hints) or returns ctx err.

func WithProvider added in v0.1.2 

func WithProvider(ctx context.Context, p Provider) context.Context

WithProvider attaches the selected LLM provider to ctx. Routing middleware selects a provider once so downstream handlers stay registry-independent and only consume the attached value.

Types

type AttemptOutcome added in v0.1.4 

type AttemptOutcome struct {
	Resp  Response
	Err   error
	Retry bool
}

AttemptOutcome is one Complete attempt inside WithRetries.

func NonEventStreamError added in v0.1.4 

func NonEventStreamError(name string, resp *http.Response) AttemptOutcome

NonEventStreamError closes the body and returns a Bad Gateway provider error.

func TimedHTTPOnce added in v0.1.4 

func TimedHTTPOnce(
	maxRetries int,
	attempt int,
	doRequest func() (*http.Response, error),
	incRequest func(statusClass string),
	isRetryableStatus func(int) bool,
	readErr func(*http.Response) *ProviderError,
	onOK func(*http.Response, time.Duration) AttemptOutcome,
) AttemptOutcome

TimedHTTPOnce is TryHTTPOnce with latency measured for the OK path.

func TryHTTPOnce added in v0.1.4 

func TryHTTPOnce(
	maxRetries int,
	attempt int,
	doRequest func() (*http.Response, error),
	incRequest func(statusClass string),
	isRetryableStatus func(int) bool,
	readErr func(*http.Response) *ProviderError,
	onOK func(*http.Response) AttemptOutcome,
) AttemptOutcome

TryHTTPOnce runs one HTTP attempt: transport/status classification + OK handler.

type CancelOnClose added in v0.1.4 

type CancelOnClose struct {
	io.ReadCloser
	Cancel context.CancelFunc
}

CancelOnClose cancels the request context when the response body is closed.

func (*CancelOnClose) Close added in v0.1.4 

func (c *CancelOnClose) Close() error

Close closes the underlying body then cancels the request context.

type CapabilityCatalog added in v0.1.4 

type CapabilityCatalog map[string]ModelCapability

CapabilityCatalog maps model ID → capability. Lookups are case-sensitive on the trimmed model ID used at registration time.

func BuiltInCapabilityCatalog added in v0.1.4 

func BuiltInCapabilityCatalog() CapabilityCatalog

func CatalogFromCapabilities added in v0.1.4 

func CatalogFromCapabilities(caps ...ModelCapability) CapabilityCatalog

CatalogFromCapabilities builds a catalog from capability rows. Empty ModelID entries are skipped. Duplicate IDs: last write wins. Provider and TokenizerFamily are trimmed so stored rows match ValidateCapability checks.

func MergeCapabilityCatalog added in v0.1.4 

func MergeCapabilityCatalog(base CapabilityCatalog, overlays ...CapabilityCatalog) CapabilityCatalog

MergeCapabilityCatalog returns a new catalog with base entries, then each overlay applied in order (later overlays win on ID collision).

func (CapabilityCatalog) Lookup added in v0.1.4 

func (c CapabilityCatalog) Lookup(model string) (ModelCapability, bool)

Lookup returns the capability for model, or (zero, false) if absent.

type CompleteSpan added in v0.1.4 

type CompleteSpan struct {
	Names CompleteSpanNames
	Req   Request
}

CompleteSpan describes a provider Complete span start.

type CompleteSpanNames added in v0.1.4 

type CompleteSpanNames struct {
	Span     string
	Provider string
}

CompleteSpanNames identifies a provider Complete span.

type HTTPClients added in v0.1.4 

type HTTPClients struct {
	Sync   *http.Client
	Stream *http.Client
}

HTTPClients holds the sync and streaming upstream clients that share a Transport.

func NewHTTPClients added in v0.1.4 

func NewHTTPClients(timeout time.Duration) HTTPClients

NewHTTPClients builds pooled sync + stream clients for a provider adapter.

type MapInput added in v0.1.2 

type MapInput struct {
	ProviderName string
	StatusCode   int // 0 when transport-only
	RetryAfter   time.Duration
	TransportErr error
	SafeMessage  string // optional sanitized 400 detail
	Reason       string // optional: queue_full, circuit_open
}

MapInput is the primitive input for MapProviderError (table-driven mapping).

type Message 

type Message struct {
	Role    string // "system", "user", "assistant", "tool"
	Content string
}

Message is a single turn in the conversation.

type ModelCapability added in v0.1.4 

type ModelCapability struct {
	ModelID           string `json:"model_id"`
	Provider          string `json:"provider"`
	ContextWindow     int    `json:"context_window"`
	MaxOutputTokens   int    `json:"max_output_tokens"`
	SupportsTools     bool   `json:"supports_tools"`
	SupportsVision    bool   `json:"supports_vision"`
	SupportsStreaming bool   `json:"supports_streaming"`
	TokenizerFamily   string `json:"tokenizer_family"`
}

ModelCapability describes static per-model limits and feature support. Provider is the vendor family for the model ID ("openai", "anthropic"), not necessarily the runtime adapter name (mock reuses OpenAI rows).

type Provider 

type Provider interface {
	// Complete sends a request to the LLM provider and returns the response.
	Complete(ctx context.Context, req Request) (Response, error)

	// Name returns the provider identifier (e.g. "openai", "anthropic").
	Name() string

	// SupportedModels returns the list of model IDs this provider handles.
	SupportedModels() []string
}

Provider is the interface all LLM provider implementations must satisfy. Implementations must be safe for concurrent use.

func ProviderFromContext added in v0.1.2 

func ProviderFromContext(ctx context.Context) (Provider, bool)

ProviderFromContext returns the provider attached by routing middleware. ok is false when no provider is present — valid outside the routed chain (tests, non-chat paths) and must be handled by callers that require one.

type ProviderError 

type ProviderError struct {
	ProviderName   string
	StatusCode     int
	ProviderBody   []byte
	ProviderErrMsg string
	RetryAfter     time.Duration
	// Reason is an optional machine-oriented cause (queue_full, circuit_open).
	Reason string
}

ProviderError is returned by Complete when the provider returns a non-2xx response. ProviderBody may be inspected by MapError sanitizers for retry metadata only — never log it or copy it into client envelopes.

func ReadProviderError added in v0.1.4 

func ReadProviderError(name string, resp *http.Response, extractMsg func([]byte) string) *ProviderError

ReadProviderError reads a capped error body and builds a ProviderError.

func (*ProviderError) Error 

func (e *ProviderError) Error() string

Error returns a redacted, caller-safe message: provider name, HTTP status, and ProviderErrMsg only. ProviderBody is never included in the string.

type Registry 

type Registry struct {
	// contains filtered or unexported fields
}

Registry maps model IDs to provider implementations and capability records. It is built once at service startup and is read-only thereafter.

func NewRegistry 

func NewRegistry(catalog CapabilityCatalog, providers ...Provider) (*Registry, error)

NewRegistry constructs a Registry from the given providers. catalog must supply a valid capability for every SupportedModels() ID. Returns ErrDuplicateModel when two providers claim the same model ID. Returns ErrMissingCapability when a model has no catalog entry. Returns ErrInvalidCapability when a catalog row fails validation or ModelID does not match the lookup key.

func (*Registry) Capability added in v0.1.4 

func (r *Registry) Capability(model string) (ModelCapability, bool)

Capability returns the capability record for the given model ID. Returns (ModelCapability{}, false) if no capability is registered. Callers must check ok; a zero ModelCapability is not a silent default.

func (*Registry) For 

func (r *Registry) For(model string) (Provider, error)

For returns the provider for the given model ID. Returns (nil, ErrNoProviderForModel) if no provider supports the model.

type Request 

type Request struct {
	// Model is the model identifier as requested by the client.
	Model string

	// Messages is the conversation history (including any injected directive).
	Messages []Message

	// Stream, if true, requests a streaming (SSE) response.
	Stream bool

	// MaxTokens is the maximum number of completion tokens. 0 = provider default.
	MaxTokens int

	// Temperature controls randomness. Nil = provider default.
	Temperature *float64

	// PassthroughFields contains client-supplied fields not explicitly modelled.
	PassthroughFields map[string]any
}

Request is a normalised LLM completion request. It is provider-agnostic; implementations translate to provider-specific format. Directive injection (Phase 2.3.3) mutates Messages before Complete is called.

type Response 

type Response struct {
	// Body is the response body from the provider.
	Body io.ReadCloser

	// StatusCode is the provider HTTP response status code.
	StatusCode int

	// Usage holds token counts extracted from the response.
	Usage *Usage

	// Latency is the time from sending the request to receiving the first byte.
	Latency time.Duration

	// ProviderRequestID is the request ID returned by the provider.
	ProviderRequestID string
}

Response is the outcome of a Complete call. For non-streaming requests, Body contains the complete provider JSON response. For streaming requests, Body is an SSE stream; the caller must read and forward it. The caller is responsible for closing Body.

func WithRetries added in v0.1.4 

func WithRetries(
	ctx context.Context,
	span trace.Span,
	maxRetries int,
	exhaustedMsg string,
	wait func(context.Context, int, error) error,
	onRetry func(),
	tryOnce func(context.Context, int) AttemptOutcome,
) (Response, error)

WithRetries runs tryOnce with exponential backoff until success, non-retry, or budget exhausted.

type UpstreamCall added in v0.1.4 

type UpstreamCall struct {
	URL    string
	Body   []byte
	Stream bool
}

UpstreamCall is the HTTP payload for one provider attempt.

type Usage 

type Usage struct {
	InputTokens  int
	OutputTokens int
	TotalTokens  int
}

Usage holds LLM token consumption data.

Source Files

View all Source files

Directories

Path Synopsis
Package mockllm provides an in-process LLM provider for CI and local smoke.
 Package mockllm provides an in-process LLM provider for CI and local smoke.
scripts
diff_capabilities command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.