provider

package
v0.23.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 24, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var WellKnownPorts = []WellKnownPort{
	{1234, "LM Studio"},
	{11434, "Ollama"},
	{8080, "MLX/llama.cpp/LocalAI"},
	{8000, "vLLM"},
	{5000, "text-generation-webui"},
}

WellKnownPorts lists default ports for common local model servers, ordered by preference (first match wins in ScanLocalServers).

Functions

func AppendTextBuilder added in v0.19.0 

func AppendTextBuilder(msg *core.AssistantMessage, delta string, builders map[int]*strings.Builder)

AppendTextBuilder accumulates delta into a strings.Builder keyed by content index. If no TextContent exists in msg, one is appended.

func BestModel added in v0.22.3 

func BestModel(results []ProbeResult) string

BestModel returns the ID of the best chat-capable model from a list. Prefers loaded models over unloaded, skips embedding-only models. Returns "" if results is empty.

func ComputeCost added in v0.19.0 

func ComputeCost(u core.Usage, c core.ModelCost) float64

func ConvertMessageList added in v0.19.0 

func ConvertMessageList[T any](msgs []core.Message, conv MessageConverters[T]) []T

ConvertMessageList applies the appropriate converter for each message type.

func ConvertToolSchemas added in v0.19.0 

func ConvertToolSchemas[T any](tools []core.ToolSchema, build func(name, desc string, params any) T) []T

ConvertToolSchemas iterates tool schemas, normalises nil parameters, and calls build to produce provider-specific tool definitions.

func DecodeUserBlocks added in v0.19.0 

func DecodeUserBlocks[T any](msg *core.UserMessage, text func(string) T, image func(core.ImageContent) T) []T

DecodeUserBlocks converts a UserMessage's Content and Blocks into provider-specific typed slices using the supplied callbacks.

func DefaultModelsYAML added in v0.5.0 

func DefaultModelsYAML() string

func DeferredToolsNote added in v0.17.0 

func DeferredToolsNote() string

DeferredToolsNote returns the instruction shown when deferred tools are present.

func FinalizeTextBuilders added in v0.19.0 

func FinalizeTextBuilders(msg *core.AssistantMessage, builders map[int]*strings.Builder)

FinalizeTextBuilders writes accumulated text from builders into msg.Content.

func GenerateModelsYAML added in v0.16.8 

func GenerateModelsYAML(models []CuratedModel, overrides map[string]CuratedModelOverride) string

GenerateModelsYAML builds a models.yaml string from a curated model list. If overrides is non-nil, it maps "provider/id" to replacement ContextWindow and MaxTokens values (used by modelsdev to inject API data).

func IsLocalProvider added in v0.22.0 

func IsLocalProvider(provider string) bool

IsLocalProvider reports whether the provider name indicates a local server (auto-discovered or explicitly configured as local/lmstudio/ollama/vllm).

func IsLoopbackURL added in v0.22.0 

func IsLoopbackURL(rawURL string) bool

IsLoopbackURL reports whether rawURL points to a loopback or local address.

func LocalDefaultContextWindow added in v0.17.0 

func LocalDefaultContextWindow() int

LocalDefaultContextWindow returns the default context window for ad-hoc local models.

func LocalDefaultMaxTokens added in v0.17.0 

func LocalDefaultMaxTokens() int

LocalDefaultMaxTokens returns the default max tokens for ad-hoc local models.

func MapStopReasonFromTable added in v0.19.0 

func MapStopReasonFromTable(reason string, table map[string]core.StopReason) core.StopReason

MapStopReasonFromTable looks up a provider-specific stop reason string in the given table, returning core.StopReasonStop as default.

func MaxCompletionTokensPrefixes added in v0.17.0 

func MaxCompletionTokensPrefixes() []string

MaxCompletionTokensPrefixes returns model ID prefixes that require max_completion_tokens instead of max_tokens.

func RunStream added in v0.19.0 

func RunStream(ctx context.Context, req core.StreamRequest, p StreamPipeline) <-chan core.StreamEvent

RunStream is the shared Stream() goroutine template.

func ScanSSE added in v0.19.0 

func ScanSSE(ctx context.Context, reader io.Reader, ch chan<- core.StreamEvent, handler func(data []byte), opts ...ScanOption)

ScanSSE reads SSE lines from reader, calling handler for each non-empty data payload. Respects context cancellation (sends StreamError on cancel).

func SetupDefaults added in v0.17.0 

func SetupDefaults() map[string]string

SetupDefaults returns the per-provider default model IDs used during first-time setup.

func ToolResultText added in v0.19.0 

func ToolResultText(msg *core.ToolResultMessage) string

ToolResultText extracts joined text from a ToolResultMessage.

func WriteDefaultModels added in v0.5.0 

func WriteDefaultModels(path string) error

WriteDefaultModels writes the default models catalog to path.

func WriteModelsData added in v0.14.0 

func WriteModelsData(path string, data string) error

WriteModelsData writes the given YAML content to path using an atomic write (temp file + rename). It creates the parent directory if needed. Returns nil without writing if the file already exists.

Types

type BaseProvider added in v0.19.0 

type BaseProvider struct {
	Model      core.Model
	APIKeyFn   func() string
	HTTPClient *http.Client
	Logger     *slog.Logger // nil = no debug logging
}

BaseProvider holds fields shared by all provider implementations.

func NewBaseProvider added in v0.19.0 

func NewBaseProvider(model core.Model, apiKeyFn func() string) BaseProvider

func (*BaseProvider) DebugLog added in v0.19.0 

func (b *BaseProvider) DebugLog() *slog.Logger

func (*BaseProvider) DoHTTPRequest added in v0.19.0 

func (b *BaseProvider) DoHTTPRequest(ctx context.Context, url string, body []byte, setHeaders func(*http.Request)) (io.ReadCloser, error)

DoHTTPRequest handles the shared HTTP POST + status check logic.

func (*BaseProvider) ResolveMaxTokens added in v0.19.0 

func (b *BaseProvider) ResolveMaxTokens(req core.StreamRequest) int

func (*BaseProvider) SetLogger added in v0.19.0 

func (b *BaseProvider) SetLogger(l *slog.Logger)

type CuratedModel added in v0.16.8 

type CuratedModel struct {
	ID                  string         `yaml:"id"`
	Name                string         `yaml:"name"`
	Provider            string         `yaml:"provider"`
	API                 string         `yaml:"api"` // "openai", "anthropic", "google"
	BaseURL             string         `yaml:"baseUrl"`
	ContextWindow       int            `yaml:"contextWindow"`       // default when API data unavailable
	MaxTokens           int            `yaml:"maxTokens"`           // default when API data unavailable
	MaxCompletionTokens bool           `yaml:"maxCompletionTokens"` // true: use max_completion_tokens instead of max_tokens (OpenAI newer models)
	Cost                core.ModelCost `yaml:"cost"`
}

CuratedModel defines a model in the default catalog. This is the single source of truth for the model list — used by both the embedded fallback YAML and the modelsdev YAML generator.

func CuratedModels added in v0.16.8 

func CuratedModels() []CuratedModel

CuratedModels returns the default model catalog.

type CuratedModelOverride added in v0.16.8 

type CuratedModelOverride struct {
	Name          string
	ContextWindow int
	MaxTokens     int
}

CuratedModelOverride holds API-sourced values that replace defaults.

type DebugLogger added in v0.19.0 

type DebugLogger interface {
	DebugLog() *slog.Logger
}

type Debuggable added in v0.5.0 

type Debuggable interface {
	SetLogger(l *slog.Logger)
}

type MessageConverters added in v0.19.0 

type MessageConverters[T any] struct {
	User       func(*core.UserMessage) T
	Assistant  func(*core.AssistantMessage) T
	ToolResult func(*core.ToolResultMessage) T
}

MessageConverters holds per-type callbacks for converting core messages to a provider-specific wire type.

type OpenAI 

type OpenAI struct {
	BaseProvider
}

OpenAI implements core.StreamProvider for OpenAI-compatible APIs.

func NewOpenAI 

func NewOpenAI(model core.Model, apiKeyFn func() string) *OpenAI

NewOpenAI creates a provider for OpenAI-compatible APIs.

func (*OpenAI) BuildRequest added in v0.19.0 

func (p *OpenAI) BuildRequest(req core.StreamRequest) ([]byte, error)

func (*OpenAI) ParseResponse added in v0.19.0 

func (p *OpenAI) ParseResponse(ctx context.Context, reader io.Reader, ch chan<- core.StreamEvent) core.AssistantMessage

func (*OpenAI) SendRequest added in v0.19.0 

func (p *OpenAI) SendRequest(ctx context.Context, body []byte) (io.ReadCloser, error)

func (*OpenAI) Stream 

func (p *OpenAI) Stream(ctx context.Context, req core.StreamRequest) <-chan core.StreamEvent

Stream implements core.StreamProvider.

func (*OpenAI) StreamModel added in v0.19.0 

func (p *OpenAI) StreamModel() core.Model

type ProbeResult added in v0.16.26 

type ProbeResult struct {
	ModelID       string // discovered model ID (e.g. "qwen3.5-27b-mxfp8")
	ServerType    string // "lmstudio", "ollama", or "unknown"
	State         string // "loaded", "not-loaded", or "" (unknown/not reported)
	ContextWindow int    // if discoverable, else 0
}

ProbeResult holds the outcome of probing a local model server.

func ProbeModels added in v0.18.3 

func ProbeModels(baseURL string) ([]ProbeResult, error)

ProbeModels contacts baseURL + "/v1/models" and returns all available models.

func ProbeServer added in v0.16.26 

func ProbeServer(baseURL string) (ProbeResult, error)

ProbeServer contacts baseURL + "/v1/models" and returns the best available model, preferring loaded chat-capable models. For Ollama, it checks /api/ps for warm (in-memory) models first.

type Registry 

type Registry struct {
	// contains filtered or unexported fields
}

Registry holds the model catalog and creates providers.

func NewRegistry 

func NewRegistry() *Registry

NewRegistry creates a registry, loading models from ~/.config/piglet/models.yaml.

func NewRegistryFromData added in v0.16.11 

func NewRegistryFromData(data []byte) (*Registry, error)

NewRegistryFromData creates a registry from raw YAML model data without reading from disk.

func (*Registry) Create 

func (r *Registry) Create(m core.Model, apiKeyFn func() string) (core.StreamProvider, error)

Create creates a StreamProvider for the given model. All providers use the OpenAI-compatible protocol by default. Non-OpenAI protocols (Anthropic, Google) are provided by extensions.

func (*Registry) Models 

func (r *Registry) Models() []core.Model

Models returns all registered models sorted by provider then ID.

func (*Registry) ModelsByProvider 

func (r *Registry) ModelsByProvider(provider string) []core.Model

ModelsByProvider returns models for a specific provider.

func (*Registry) Register 

func (r *Registry) Register(m core.Model)

Register adds a model to the registry.

func (*Registry) RegisterLocalServers added in v0.18.3 

func (r *Registry) RegisterLocalServers(urls []string, contextWindow, maxTokens int) int

RegisterLocalServers probes each URL concurrently for available models and registers them. Returns the number of models discovered. Errors are silently skipped (servers may be offline).

func (*Registry) ReloadModels added in v0.16.11 

func (r *Registry) ReloadModels() (int, error)

ReloadModels re-reads models.yaml from disk, replacing in-memory entries.

func (*Registry) Resolve 

func (r *Registry) Resolve(query string) (core.Model, bool)

Resolve finds a model by provider/id string or just model id.

type RetryAfterError added in v0.21.0 

type RetryAfterError struct {
	Err      error
	Duration time.Duration
}

RetryAfterError wraps an API error that includes a Retry-After hint. Implements the retryAfterHint interface checked by core/stream.go.

func (*RetryAfterError) Error added in v0.21.0 

func (e *RetryAfterError) Error() string

func (*RetryAfterError) RetryAfter added in v0.21.0 

func (e *RetryAfterError) RetryAfter() time.Duration

func (*RetryAfterError) Unwrap added in v0.21.0 

func (e *RetryAfterError) Unwrap() error

type ScanOption added in v0.19.0 

type ScanOption func(*bufio.Scanner)

func WithLargeBuffer added in v0.19.0 

func WithLargeBuffer(initial, max int) ScanOption

type ScanResult added in v0.22.3 

type ScanResult struct {
	URL    string        // base URL of the responding server
	Models []ProbeResult // all models reported by that server
}

ScanResult holds the outcome of ScanLocalServers.

func ScanLocalServers added in v0.22.3 

func ScanLocalServers() (ScanResult, error)

ScanLocalServers probes well-known ports concurrently, returning the first responding server by preference order.

type StreamPipeline added in v0.19.0 

type StreamPipeline interface {
	BuildRequest(req core.StreamRequest) ([]byte, error)
	SendRequest(ctx context.Context, body []byte) (io.ReadCloser, error)
	ParseResponse(ctx context.Context, reader io.Reader, ch chan<- core.StreamEvent) core.AssistantMessage
	StreamModel() core.Model
}

StreamPipeline is implemented by each concrete provider to plug into RunStream.

type WellKnownPort added in v0.22.3 

type WellKnownPort struct {
	Port int
	Name string
}

WellKnownPort describes a default port for a local model server.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.