llm

package
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 3, 2026 License: Apache-2.0 Imports: 24 Imported by: 0

Documentation

Overview

Package llm adapts LLM providers to the agentgo.ChatModel interface. It wraps litellm to reach OpenAI, Anthropic, Gemini, and other backends, and classifies provider errors onto agentgo's retry and overflow contracts. Construct a model with NewModel.

Index

Constants

View Source 
const ThinkingAuto agentgo.ThinkingLevel = ""

Variables

View Source 
var DefaultGenerationConfig = &GenerationConfig{
	Temperature:      0.7,
	TopP:             0.9,
	TopK:             0,
	MaxTokens:        65536,
	StopSequences:    []string{},
	PresencePenalty:  0.0,
	FrequencyPenalty: 0.0,
	Seed:             nil,
}
View Source 
var ThinkingLevelOrder = []agentgo.ThinkingLevel{
	agentgo.ThinkingOff,
	agentgo.ThinkingMinimal,
	agentgo.ThinkingLow,
	agentgo.ThinkingMedium,
	agentgo.ThinkingHigh,
	agentgo.ThinkingXHigh,
	agentgo.ThinkingMax,
}

Functions

func CalculateCost 

func CalculateCost(pricing *ModelPricing, usage *agentgo.Usage) *agentgo.Cost

CalculateCost computes the monetary cost from pricing rates and token usage. Returns nil if pricing or usage is nil.

Pricing semantic: usage.Input already includes usage.CacheRead per the underlying litellm convention. The cached portion must only be billed at the cache-read rate; charging full Input at input rate AND CacheRead at cache-read rate would double-bill.

func IsProviderRegistered 

func IsProviderRegistered(name string) bool

IsProviderRegistered reports whether the provider name is known to this adapter.

func RegisteredProviders 

func RegisteredProviders() []string

RegisteredProviders returns all provider names known to this adapter.

func TransformMessages 

func TransformMessages(messages []agentgo.Message, targetProvider string) []agentgo.Message

TransformMessages normalizes a message sequence for a target provider. Use this when switching models mid-conversation to avoid provider rejections.

Two-pass algorithm:

  1. Normalize tool call IDs (truncate >64 chars, sanitize to [a-zA-Z0-9_-]), handle thinking blocks based on target provider.
  2. Apply ID mapping to tool results, insert synthetic results for orphaned tool calls.

Types

type BaseModel 

type BaseModel struct {
	// contains filtered or unexported fields
}

BaseModel provides common model metadata and capability checks.

func NewBaseModel 

func NewBaseModel(info ModelInfo, config *GenerationConfig) *BaseModel

func (*BaseModel) GetConfig 

func (m *BaseModel) GetConfig() *GenerationConfig

func (*BaseModel) Info 

func (m *BaseModel) Info() ModelInfo

func (*BaseModel) ModelName 

func (m *BaseModel) ModelName() string

ModelName implements agentgo.ModelNamer.

func (*BaseModel) SupportsCapability 

func (m *BaseModel) SupportsCapability(capability ModelCapability) bool

func (*BaseModel) SupportsStreaming 

func (m *BaseModel) SupportsStreaming() bool

func (*BaseModel) SupportsTools 

func (m *BaseModel) SupportsTools() bool

type Capabilities 

type Capabilities struct {
	Provider string
	Model    string

	Thinking   ThinkingCapabilities
	Tools      ToolCapabilities
	Structured StructuredCapabilities
	Streaming  StreamingCapabilities
	Usage      UsageCapabilities
}

Capabilities is agentgo's provider-neutral view of model capabilities.

func (Capabilities) ThinkingPolicy 

func (c Capabilities) ThinkingPolicy() ThinkingPolicy

type CapabilityProvider 

type CapabilityProvider interface {
	Capabilities() Capabilities
}

CapabilityProvider is implemented by models that can expose provider/model capabilities for UI preflight and configuration validation. It is advisory: request execution remains the source of truth and should still fail loudly.

type GenerationConfig 

type GenerationConfig struct {
	Temperature      float64  `json:"temperature"`
	TopP             float64  `json:"top_p"`
	TopK             int      `json:"top_k"`
	MaxTokens        int      `json:"max_tokens"`
	StopSequences    []string `json:"stop_sequences"`
	PresencePenalty  float64  `json:"presence_penalty"`
	FrequencyPenalty float64  `json:"frequency_penalty"`
	Seed             *int64   `json:"seed"`
}

GenerationConfig defines sampling and length control parameters.

type LiteLLMAdapter 

type LiteLLMAdapter struct {
	*BaseModel
	// contains filtered or unexported fields
}

LiteLLMAdapter adapts litellm to the agentgo.ChatModel interface.

func NewLiteLLMAdapter 

func NewLiteLLMAdapter(model string, client *litellm.Client) *LiteLLMAdapter

NewLiteLLMAdapter wraps an existing litellm.Client as a ChatModel. Use this when you need to reuse a Client or inject a custom Provider instance; for the common case prefer NewModel.

func NewModel 

func NewModel(provider, model string, opts ...ModelOption) (*LiteLLMAdapter, error)

NewModel constructs a ChatModel by provider name. The provider must be registered in litellm (builtin or via litellm.RegisterProvider).

func (*LiteLLMAdapter) Capabilities 

func (l *LiteLLMAdapter) Capabilities() Capabilities

Capabilities returns the provider/model capability view exposed by litellm.

func (*LiteLLMAdapter) Generate 

func (l *LiteLLMAdapter) Generate(ctx context.Context, messages []agentgo.Message, tools []agentgo.ToolSpec, opts ...agentgo.CallOption) (*agentgo.LLMResponse, error)

Generate produces a synchronous response.

func (*LiteLLMAdapter) GenerateStream 

func (l *LiteLLMAdapter) GenerateStream(ctx context.Context, messages []agentgo.Message, tools []agentgo.ToolSpec, opts ...agentgo.CallOption) (<-chan agentgo.StreamEvent, error)

GenerateStream produces a streaming response with fine-grained events.

func (*LiteLLMAdapter) ProviderName 

func (l *LiteLLMAdapter) ProviderName() string

ProviderName returns the provider name (e.g. "openai", "anthropic"). Implements agentgo.ProviderNamer for per-provider API key resolution.

type ModelCapability 

type ModelCapability string

ModelCapability defines capability identifiers. 

const (
	CapabilityChat         ModelCapability = "chat"
	CapabilityCompletion   ModelCapability = "completion"
	CapabilityToolCalling  ModelCapability = "tool_calling"
	CapabilityStreaming    ModelCapability = "streaming"
	CapabilityMultimodal   ModelCapability = "multimodal"
	CapabilityFunctionCall ModelCapability = "function_call"
)

type ModelInfo 

type ModelInfo struct {
	Name         string        `json:"name"`
	Provider     string        `json:"provider"`
	Version      string        `json:"version"`
	MaxTokens    int           `json:"max_tokens"`
	ContextSize  int           `json:"context_size"`
	Capabilities []string      `json:"capabilities"`
	Pricing      *ModelPricing `json:"pricing,omitempty"`
}

ModelInfo contains basic model metadata.

type ModelOption 

type ModelOption func(*modelConfig)

ModelOption configures NewModel.

func WithAPIKey 

func WithAPIKey(key string) ModelOption

func WithBaseURL 

func WithBaseURL(url string) ModelOption

func WithClientOptions 

func WithClientOptions(opts ...litellm.ClientOption) ModelOption

WithClientOptions forwards litellm ClientOptions (e.g. litellm.WithHook) to the underlying client, letting callers attach observability or other cross-cutting behaviour without this package importing those concerns.

func WithExtra 

func WithExtra(extra map[string]any) ModelOption

WithExtra sets model-level, provider-specific request parameters merged into every request's Extra map (e.g. min_p, presence_penalty, or provider keys like chat_template_kwargs). OpenAI-compatible providers pass these through verbatim into the request body — the extra_body convention. Per-call Extra entries (e.g. session_id) are added alongside, not overwritten.

func WithProviderExtra 

func WithProviderExtra(extra map[string]any) ModelOption

WithProviderExtra sets provider-level configuration passed to litellm.ProviderConfig.Extra. Use it for HTTP headers or other provider client options, while WithExtra remains request-body Extra.

func WithRequestTimeout 

func WithRequestTimeout(d time.Duration) ModelOption

WithRequestTimeout sets an optional per-request timeout. Zero leaves timeout control to the caller context.

func WithResilience 

func WithResilience(rc ResilienceConfig) ModelOption

WithResilience replaces the entire resilience config; later With* options may still override specific fields.

func WithStreamIdleTimeout 

func WithStreamIdleTimeout(d time.Duration) ModelOption

WithStreamIdleTimeout aborts a streaming response if no chunk arrives within the window (default 120s). Pass 0 to disable the watchdog explicitly.

type ModelPricing 

type ModelPricing struct {
	InputPerToken      float64 `json:"input_per_token"`
	OutputPerToken     float64 `json:"output_per_token"`
	CacheReadPerToken  float64 `json:"cache_read_per_token"`
	CacheWritePerToken float64 `json:"cache_write_per_token"`
}

ModelPricing defines per-token cost rates in USD. Set rates to 0 for categories that don't apply.

type ProviderConfig 

type ProviderConfig struct {
	APIKey  string
	BaseURL string
	Timeout time.Duration
	Extra   map[string]any
	Retry   *retry.Policy
}

ProviderConfig is kept for source compatibility with older agentgo callers. New litellm providers expose explicit Config structs; this package maps the common subset that agentgo needs.

type ResilienceConfig 

type ResilienceConfig struct {
	StreamIdleTimeout time.Duration
	Retry             *retry.Policy
}

ResilienceConfig is the compatibility shape for retry and stream idle knobs.

type StreamingCapabilities 

type StreamingCapabilities struct {
	Supported       Support
	Usage           Support
	ReasoningDeltas Support
	ToolCallDeltas  Support
	NativeResponses Support
	IdleTimeout     Support
}

type StructuredCapabilities 

type StructuredCapabilities struct {
	JSONObject Support
	JSONSchema Support
	Strict     Support
	PromptOnly bool
}

type Support 

type Support int

Support describes whether a model/provider supports a capability. 

const (
	SupportUnknown Support = iota
	SupportNo
	SupportYes
	SupportPartial
)

type ThinkingCapabilities 

type ThinkingCapabilities struct {
	Supported     Support
	Disable       Support
	Efforts       []agentgo.ThinkingLevel
	BudgetTokens  Support
	IncludeOutput Support
	Notes         []string
}

func (ThinkingCapabilities) SupportsEffort 

func (c ThinkingCapabilities) SupportsEffort(level agentgo.ThinkingLevel) bool

type ThinkingPolicy 

type ThinkingPolicy struct {
	Available []agentgo.ThinkingLevel
}

func ThinkingPolicyFor 

func ThinkingPolicyFor(model any) ThinkingPolicy

func ThinkingPolicyFromCapabilities 

func ThinkingPolicyFromCapabilities(caps Capabilities) ThinkingPolicy

func (ThinkingPolicy) Allows 

func (p ThinkingPolicy) Allows(level agentgo.ThinkingLevel) bool

func (ThinkingPolicy) Resolve 

func (p ThinkingPolicy) Resolve(level agentgo.ThinkingLevel) (agentgo.ThinkingLevel, bool)

type ToolCapabilities 

type ToolCapabilities struct {
	Calls               Support
	ParallelCalls       Support
	StrictSchema        Support
	Choice              Support
	MultimodalResults   Support
	RequiresAdjacency   bool
	RoundTripSignatures Support
	HostedProviderTools Support
}

type UsageCapabilities 

type UsageCapabilities struct {
	InputTokens      Support
	OutputTokens     Support
	TotalTokens      Support
	ReasoningTokens  Support
	CacheReadTokens  Support
	CacheWriteTokens Support
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.