coremodels

package module
v0.15.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 1, 2026 License: Apache-2.0 Imports: 12 Imported by: 0

README

core-models

One LLM provider layer for Go agents built on Google's Agent Development Kit. It is shared by mast and core-agent.

What's in it:

  • provider adapters for Gemini, Claude, OpenAI, the Vertex AI partner models, and self-hosted model servers (vLLM, SGLang, Ollama);
  • one usage record every adapter fills in, so tokens and cost look the same whichever model answered;
  • a thin adapter module for ADK v2.

Status: pre-release (v0.x), in use. mast and core-agent both run on it. Four wire formats ("dialects") are built:

Dialect Package Reaches
openai-chat dialect/openaichat Vertex AI partner models, xAI, vLLM, SGLang, Ollama and any OpenAI-compatible endpoint; SAP AI Core's orchestration service (sap-ai-core, not yet validated against a real tenant)
openai-responses dialect/openairesp OpenAI, and any server that serves /v1/responses (xAI, vLLM)
anthropic dialect/anthropic Claude on Anthropic's API and on Vertex AI
gemini dialect/gemini Gemini on the Developer API and Vertex AI

See the roadmap for what is still owed.

Modules

Module For
github.com/go-steer/core-models The contract (llm), the usage record (usage), profiles, the dialects above, the KV-cache sampler (kvmetrics) and the shared price catalog (pricing). Never imports ADK
github.com/go-steer/core-models/adkv2 Projects on google.golang.org/adk/v2

License

Apache 2.0.

Documentation

Overview

Package coremodels opens a provider profile: it resolves the profile against the environment, picks the dialect its server speaks, and hands back a Provider whose models are llm.LLMs.

p, err := coremodels.Open(ctx, prof, coremodels.Options{})
m, err := p.Model(ctx, "Qwen/Qwen3-Coder-Next")

Everything that can fail before a request — an unknown dialect, a missing variable, absent credentials, a model the profile does not serve — fails here, naming the profile (requirement R1).

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Options

type Options struct {
	// Resolve is passed to profile.Resolve.
	Resolve profile.Options
	// HTTPClient supplies the base transport and timeout. Nil means a
	// client with http.DefaultTransport.
	HTTPClient *http.Client
	// Retry overrides the HTTP-layer retry policy.
	Retry *retry.Policy
	// BuiltinTools turns on server-side tools by their provider-neutral
	// names (web_search, url_context, code_execution), for a dialect
	// that has them. Empty is none. Opening a profile whose dialect has
	// none with any set is an error, not a silently missing tool.
	BuiltinTools []string
	// Logf receives an adapter's operator-facing notices. Nil discards
	// them.
	Logf func(format string, args ...any)

	// PromptCache is the Anthropic prompt-caching policy. Nil means
	// anthropic.DefaultCacheOptions: the system block and the
	// conversation tail, at the five-minute TTL. Pass a zero value to
	// place no breakpoints. Ignored by other dialects.
	PromptCache *anthropic.CacheOptions
}

Options adjusts Open.

type Provider

type Provider interface {
	// Name is the profile's name.
	Name() string
	// Backend is the identity prices are keyed on, beside the model id.
	Backend() string
	// Model returns a model the profile serves, and refuses one it
	// does not.
	Model(ctx context.Context, id string) (llm.LLM, error)
	// DefaultSmallModel is the profile's small-tier model, or "".
	DefaultSmallModel() string
	// Capabilities are what the profile declares its server honors.
	Capabilities() profile.Capabilities
	// Profile is the resolved profile, extends applied.
	Profile() profile.Profile
}

Provider is an opened profile.

func Open

func Open(ctx context.Context, p profile.Profile, opts Options) (Provider, error)

Open resolves p and returns a Provider for it.

Directories

Path Synopsis
adkv1 module
adkv2 module
Package auth resolves the credential a provider profile declares.
Package auth resolves the credential a provider profile declares.
Package callctx is the per-call vocabulary adapters and products share through a context.Context.
Package callctx is the per-call vocabulary adapters and products share through a context.Context.
dev
regen-builtin-pricing command
Generator for pricing/builtin.go, the compiled-in table core-models' pricing package ships to mast and core-agent alike.
Generator for pricing/builtin.go, the compiled-in table core-models' pricing package ships to mast and core-agent alike.
dialect
anthropic
Package anthropic is the anthropic dialect: Claude through the Messages API, first-party and on Vertex AI (docs/design.md §7.3, phase L4).
Package anthropic is the anthropic dialect: Claude through the Messages API, first-party and on Vertex AI (docs/design.md §7.3, phase L4).
gemini
Package gemini is core-models' Gemini adapter: the Gemini Developer API (API key) and Gemini on Vertex AI (Application Default Credentials), over genai, as llm.LLM.
Package gemini is core-models' Gemini adapter: the Gemini Developer API (API key) and Gemini on Vertex AI (Application Default Credentials), over genai, as llm.LLM.
gemini/vertexcache
Package vertexcache owns the lifecycle of a single Vertex explicit context cache — Create at agent startup, Refresh on TTL pressure, Delete on session unregister.
Package vertexcache owns the lifecycle of a single Vertex explicit context cache — Create at agent startup, Refresh on TTL pressure, Delete on session unregister.
openaichat
Package openaichat is the openai-chat dialect: POST {base_url}/chat/completions, the wire shape Vertex AI's partner models, xAI, vLLM, SGLang, Ollama, llama.cpp, NVIDIA NIM and the managed long tail all speak (docs/design.md §7.1).
Package openaichat is the openai-chat dialect: POST {base_url}/chat/completions, the wire shape Vertex AI's partner models, xAI, vLLM, SGLang, Ollama, llama.cpp, NVIDIA NIM and the managed long tail all speak (docs/design.md §7.1).
openairesp
Package openairesp is the openai-responses dialect: POST {base_url}/responses, OpenAI's Responses API (docs/design.md §7.2).
Package openairesp is the openai-responses dialect: POST {base_url}/responses, OpenAI's Responses API (docs/design.md §7.2).
internal
litellm
Package litellm reads the parts of LiteLLM's model_prices_and_context_window.json that more than one caller in this module needs: the pricing package's Refresh and the dev/regen-builtin-pricing generator.
Package litellm reads the parts of LiteLLM's model_prices_and_context_window.json that more than one caller in this module needs: the pricing package's Refresh and the dev/regen-builtin-pricing generator.
openaishared
Package openaishared holds what the two OpenAI-shaped dialects (openai-chat and openai-responses) do identically: usage mapping, error-envelope parsing, tool-call id assignment and the small helpers around them.
Package openaishared holds what the two OpenAI-shaped dialects (openai-chat and openai-responses) do identically: usage mapping, error-envelope parsing, tool-call id assignment and the small helpers around them.
schemajson
Package schemajson renders genai's typed Schema as JSON Schema, for the dialects whose servers take JSON Schema (openai-chat and openai-responses).
Package schemajson renders genai's typed Schema as JSON Schema, for the dialects whose servers take JSON Schema (openai-chat and openai-responses).
Package kvmetrics samples a self-hosted server's KV-cache statistics from its Prometheus endpoint (docs/design.md §8): prefix-cache queries and hits, the hit rate, and KV-cache utilization.
Package kvmetrics samples a self-hosted server's KV-cache statistics from its Prometheus endpoint (docs/design.md §8): prefix-cache queries and hits, the hit rate, and KV-cache utilization.
Package llm is the contract every core-models provider adapter implements.
Package llm is the contract every core-models provider adapter implements.
Package pricing resolves a model's per-million-token rates across a layered set of sources, so usage costs stay accurate as new models ship and operators add overrides.
Package pricing resolves a model's per-million-token rates across a layered set of sources, so usage costs stay accurate as new models ship and operators add overrides.
Package profile declares how to reach a model: endpoint, credential, wire dialect, capabilities, and the models served.
Package profile declares how to reach a model: endpoint, credential, wire dialect, capabilities, and the models served.
Package retry retries model calls at the HTTP layer, where a provider's rate-limit headers are still visible.
Package retry retries model calls at the HTTP layer, where a provider's rate-limit headers are still visible.
Package toolwire states the invariant every provider adapter is held to: each tool a model is offered arrives on the wire with every argument it declares, with the required ones still required, and with each argument's own schema intact.
Package toolwire states the invariant every provider adapter is held to: each tool a model is offered arrives on the wire with every argument it declares, with the required ones still required, and with each argument's own schema intact.
Package usage is the normalized usage record a core-models adapter attaches to every Response, beside genai's UsageMetadata.
Package usage is the normalized usage record a core-models adapter attaches to every Response, beside genai's UsageMetadata.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL