llm

README

llm

Small, native, per-provider model adapters for agentcore / jess agents, behind one domain-named port: llm.LLM.

Why

agentcore.ChatModel is the interface an agent talks to. llm.LLM is that same contract under a name that belongs to your domain, so the rest of your code never imports a vendor SDK. Each provider adapter is:

• Native — it speaks the provider's own API, not an OpenAI-compatibility shim layered over a different provider.
• An anti-corruption layer — the only package allowed to import that provider's SDK, translating it to and from agentcore message / tool / stream-event types.

Add a provider by adding a subpackage; the rest of your agent doesn't change.

Install

go get github.com/guygrigsby/llm

Requires Go 1.26+.

The port

type LLM interface {
	Generate(ctx context.Context, messages []agentcore.Message, tools []agentcore.ToolSpec, opts ...agentcore.CallOption) (*agentcore.LLMResponse, error)
	GenerateStream(ctx context.Context, messages []agentcore.Message, tools []agentcore.ToolSpec, opts ...agentcore.CallOption) (<-chan agentcore.StreamEvent, error)
	SupportsTools() bool
}

Anything satisfying llm.LLM is an agentcore.ChatModel, so it drops straight into jess.WithModel. agentcore types cross the boundary freely — they are the ubiquitous language jess is built on, not an isolated vendor. The isolated vendor is each provider's SDK, which only that provider's adapter imports.

Adapters

anthropic

github.com/guygrigsby/llm/anthropic — native Anthropic adapter on the official anthropic-sdk-go. Adaptive thinking + effort; never sends temperature/top_p/top_k. Full streaming and tool support.

m, err := anthropic.New(anthropic.Config{APIKey: key, Model: "claude-sonnet-5"})
if err != nil {
	return err
}
agent := jess.New(jess.WithModel(m) /* , ... */)

deepseek

github.com/guygrigsby/llm/deepseek — native DeepSeek adapter over its OpenAI-format chat-completions API, using only net/http (no SDK dependency). Built for cheap one-shot work such as summaries and extraction; tool-calling and true token streaming are not wired yet (GenerateStream returns the whole result as one terminal event). Wire those before using it as a primary conversational model.

m, err := deepseek.New(deepseek.Config{APIKey: key, Model: "deepseek-chat"})

License

MIT. See LICENSE.

Documentation

Overview

Package llm is the LLM port for jess/agentcore-based agents, plus a home for native, per-provider adapters that satisfy it.

LLM is the ubiquitous term for "the model the agent talks to." It mirrors the agentcore ChatModel contract, so anything satisfying LLM plugs directly into jess.WithModel — but naming it here keeps the vendor type out of consumers' domain language.

Each provider adapter lives in its own subpackage (llm/anthropic, and later llm/openai, ...) and is native: it speaks that provider's own API/SDK, not an OpenAI-compatible translation. An adapter is an anti-corruption layer — the only package allowed to import its provider's SDK — translating that SDK to and from agentcore message / tool / stream-event types.

Index

• type LLM
• type Meter
• type MeterFunc
  • func (f MeterFunc) Observe(u Usage)
• type Usage

Types

type LLM

type LLM interface {
	Generate(ctx context.Context, messages []agentcore.Message, tools []agentcore.ToolSpec, opts ...agentcore.CallOption) (*agentcore.LLMResponse, error)
	GenerateStream(ctx context.Context, messages []agentcore.Message, tools []agentcore.ToolSpec, opts ...agentcore.CallOption) (<-chan agentcore.StreamEvent, error)
	SupportsTools() bool
}

LLM is the model port the agent consumes. It is the agentcore ChatModel contract under a domain name: any LLM is usable directly as an agent's model. agentcore types cross this boundary freely — they are the ubiquitous language jess is built on, not an isolated vendor. The isolated vendor is each provider's SDK, which only that provider's adapter imports.

type Meter

type Meter interface {
	Observe(Usage)
}

Meter observes per-call Usage. Set one on an adapter's Config to capture tokens and latency for every generation. Implementations must be safe for concurrent use — adapters may call Observe from multiple goroutines.

type MeterFunc

type MeterFunc func(Usage)

MeterFunc adapts a plain function to a Meter.

func (MeterFunc) Observe

func (f MeterFunc) Observe(u Usage)

Observe calls f.

type Usage

type Usage struct {
	Provider         string        `json:"provider"`
	Model            string        `json:"model"`
	PromptTokens     int           `json:"prompt_tokens"`
	CompletionTokens int           `json:"completion_tokens"`
	TotalTokens      int           `json:"total_tokens"`
	Latency          time.Duration `json:"latency"`
}

Usage reports the token counts and wall-clock latency of a single model call. Adapters populate it from the provider response and hand it to a Meter (when one is configured), so a consumer can record cost and latency without the adapter knowing anything about pricing.