gatewayclient

Documentation

Overview

Package gatewayclient is the daemon's HTTP client for kram-gateway. The daemon never talks to LLM providers directly — every model call goes through the gateway, so load balancing, fallback and compression apply uniformly regardless of which client drove the session.

Index

• func WithRunID(ctx context.Context, runID string) context.Context
• type Client
  • func New(baseURL string) *Client
  • func (c *Client) ChatCompletion(ctx context.Context, model string, messages []openai.ChatMessage, ...) (Result, error)
  • func (c *Client) ChatCompletionStream(ctx context.Context, model string, messages []openai.ChatMessage, ...) (<-chan StreamDelta, error)
  • func (c *Client) Status(ctx context.Context) (Status, error)
• type ComboStatus
• type GatewayError
  • func (e *GatewayError) Error() string
• type ProviderStatus
• type Result
• type Status
  • func (s Status) ComboSupportsImages(comboID string) bool
• type StreamDelta

Functions

func WithRunID

func WithRunID(ctx context.Context, runID string) context.Context

WithRunID attaches an opaque per-agent-run identifier to ctx. Every gateway call made with the returned context sends it as openai.RunIDHeader, which is what lets kram-gateway's Sticky routing tell one agent run's tool round-trips apart from a later, unrelated turn in the same session — see DECISIONS.md, "Sticky is run-scoped, not session-prefix-scoped". The agent loop calls this once per Service.Run/RunTask; nothing else needs to.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client calls a kram-gateway instance's OpenAI-compatible API.

func New

func New(baseURL string) *Client

New builds a client pointed at a running kram-gateway (e.g. http://127.0.0.1:20128).

func (*Client) ChatCompletion

func (c *Client) ChatCompletion(ctx context.Context, model string, messages []openai.ChatMessage, tools []openai.Tool) (Result, error)

ChatCompletion sends a non-streaming chat completion request — with the full message history and, if the caller wants tool calling, the tool definitions to offer the model — and returns whatever the gateway's chosen provider produced.

func (*Client) ChatCompletionStream

func (c *Client) ChatCompletionStream(ctx context.Context, model string, messages []openai.ChatMessage, tools []openai.Tool) (<-chan StreamDelta, error)

ChatCompletionStream is ChatCompletion's streaming counterpart: same request shape, but the gateway's response is relayed chunk by chunk instead of buffered. The daemon's agent loop runs entirely off this — tool-call turns are silent (no user-visible deltas until Done reveals ToolCalls), text-answering turns stream live to whoever is reading the channel.

func (*Client) Status

func (c *Client) Status(ctx context.Context) (Status, error)

Status fetches the gateway's current provider/combo status.

type ComboStatus

type ComboStatus struct {
	ID        string   `json:"id"`
	Strategy  string   `json:"strategy"`
	Providers []string `json:"providers"`
}

ComboStatus mirrors the gateway's combo summary.

type GatewayError

type GatewayError struct {
	Combo      string
	Retryable  bool
	RetryAfter time.Duration
	Cause      openai.FailureClass
	Attempts   []openai.AttemptInfo
	Message    string
}

GatewayError is what ChatCompletion returns when the gateway itself reports every candidate in a combo failed — the typed, daemon-side counterpart of openai.ErrorBody's extension fields, so a caller (internal/daemon/agent's Gateway Round retry) can decide whether retrying is worth it instead of pattern-matching an error string.

func (*GatewayError) Error

func (e *GatewayError) Error() string

type ProviderStatus

type ProviderStatus struct {
	ID             string `json:"id"`
	BreakerOpen    bool   `json:"breaker_open"`
	SupportsImages bool   `json:"supports_images"`
	SupportsTools  bool   `json:"supports_tools"`
}

ProviderStatus is the capability/health subset of the gateway's /admin/status the daemon needs to make routing decisions (e.g. whether any provider in a combo can accept images).

type Result

type Result struct {
	Content   string
	ToolCalls []openai.ToolCall
	Provider  string
	Attempts  []openai.AttemptInfo
	// Ranking and Strategy are the router's own full candidate ranking
	// and the combo's configured strategy name for this call — kram-
	// gateway extensions passed straight through from
	// openai.ChatCompletionResponse, used to build a RouteCall (see
	// internal/daemon/agent/route.go).
	Ranking  []openai.RankedProviderInfo
	Strategy string
	Usage    openai.Usage
}

Result is what a chat completion call actually produced: the reply (text and/or tool calls), which provider served it, the real fallback trail attempted to get there, and token usage for that request.

type Status

type Status struct {
	Providers []ProviderStatus `json:"providers"`
	Combos    []ComboStatus    `json:"combos"`
}

Status is the subset of GET /admin/status the daemon consumes.

func (Status) ComboSupportsImages

func (s Status) ComboSupportsImages(comboID string) bool

ComboSupportsImages reports whether at least one provider in the named combo accepts image input. Kram never assumes support — this is the check the agent loop makes before attaching an image to a request.

type StreamDelta

type StreamDelta struct {
	Content   string
	Done      bool
	ToolCalls []openai.ToolCall
	Provider  string
	Usage     openai.Usage
	Attempts  []openai.AttemptInfo
	// Ranking and Strategy mirror Result's — see its doc comment.
	Ranking  []openai.RankedProviderInfo
	Strategy string
	Err      error
}

StreamDelta is one normalized increment of a streaming chat completion — either a text fragment, or (on the final delta, Done set) the complete picture the non-streaming Result carries: which provider served it, the fallback trail, usage, and any tool calls the model is requesting.