providers

package
v1.55.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 20, 2026 License: MIT Imports: 12 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AnthropicContentBlock 

type AnthropicContentBlock struct {
	Type      string                 `json:"type"`
	Text      string                 `json:"text,omitempty"`
	ID        string                 `json:"id,omitempty"`
	Name      string                 `json:"name,omitempty"`
	Input     map[string]interface{} `json:"input,omitempty"`
	ToolUseID string                 `json:"tool_use_id,omitempty"`
}

type AnthropicProvider 

type AnthropicProvider struct {
	// contains filtered or unexported fields
}

func NewAnthropicProvider 

func NewAnthropicProvider() *AnthropicProvider

func (*AnthropicProvider) GetRawStream 

func (p *AnthropicProvider) GetRawStream(messages []stream.Message, customTools []tools.ToolDefinition, builtinTools []interface{}) (io.ReadCloser, error)

func (*AnthropicProvider) IsConfigured 

func (p *AnthropicProvider) IsConfigured() bool

func (*AnthropicProvider) StreamChat 

func (p *AnthropicProvider) StreamChat(w http.ResponseWriter, message string) error

Keep the old method for backward compatibility during transition

type AnthropicRequest 

type AnthropicRequest struct {
	Model        string           `json:"model"`
	MaxTokens    int              `json:"max_tokens"`
	Messages     []stream.Message `json:"messages"`
	System       string           `json:"system,omitempty"`
	Stream       bool             `json:"stream"`
	Temperature  *float64         `json:"temperature,omitempty"`
	Tools        []interface{}    `json:"tools,omitempty"`         // Mixed: custom + built-in tools
	CacheControl *CacheControl    `json:"cache_control,omitempty"` // Automatic prompt caching
}

type AnthropicStreamResponse 

type AnthropicStreamResponse struct {
	Type  string       `json:"type"`
	Delta ContentDelta `json:"delta,omitempty"`
}

type AnthropicTranslator 

type AnthropicTranslator struct{}

AnthropicTranslator - Anthropic uses our universal format natively

func (*AnthropicTranslator) TranslateMessage 

func (t *AnthropicTranslator) TranslateMessage(msg stream.Message) (interface{}, error)

type CacheControl added in v1.43.5 

type CacheControl struct {
	Type string `json:"type"` // "ephemeral"
}

CacheControl enables automatic prompt caching for Anthropic API requests.

type CerebrasProvider added in v1.45.0 

type CerebrasProvider struct {
	*OpenAICompatibleProvider
}

CerebrasProvider wraps the OpenAI-compatible base provider with Cerebras-specific configuration Cerebras provides ultra-fast inference with their custom wafer-scale hardware API Documentation: https://inference-docs.cerebras.ai

func NewCerebrasProvider added in v1.45.0 

func NewCerebrasProvider() *CerebrasProvider

NewCerebrasProvider creates a new Cerebras provider instance Requires CEREBRAS_API_KEY environment variable to be set

Popular models: - gpt-oss-120b (OpenAI open-weight MoE, 128k context, 3000 tok/s) - llama-4-scout-17b-16e-instruct (latest, fast) - llama3.3-70b (recommended, high quality) - llama3.1-8b (ultra-fast)

type ContentDelta 

type ContentDelta struct {
	Type string `json:"type"`
	Text string `json:"text"`
}

type FireworksProvider 

type FireworksProvider struct {
	*OpenAICompatibleProvider
}

FireworksProvider wraps the OpenAI-compatible base provider with Fireworks-specific configuration Fireworks AI provides fast inference with a wide range of open-source and custom models API Documentation: https://docs.fireworks.ai/api-reference

func NewFireworksProvider 

func NewFireworksProvider() *FireworksProvider

NewFireworksProvider creates a new Fireworks provider instance Requires FIREWORKS_API_KEY environment variable to be set

Popular models: - accounts/fireworks/models/kimi-k2-thinking (Kimi K2 Thinking, reasoning model) - accounts/fireworks/models/llama-v3p3-70b-instruct (Llama 3.3 70B, recommended) - accounts/fireworks/models/llama-v3p1-405b-instruct (Llama 3.1 405B, largest) - accounts/fireworks/models/llama-v3p1-70b-instruct (Llama 3.1 70B) - accounts/fireworks/models/llama-v3p1-8b-instruct (Llama 3.1 8B, fast) - accounts/fireworks/models/mixtral-8x22b-instruct (Mixtral 8x22B) - accounts/fireworks/models/qwen2p5-72b-instruct (Qwen 2.5 72B) - accounts/fireworks/models/deepseek-v3 (DeepSeek V3) - accounts/fireworks/models/glm-4p7 (GLM-4 Plus 7B)

type GeminiContent 

type GeminiContent struct {
	Role  string       `json:"role"` // "user" or "model" - REQUIRED, no omitempty
	Parts []GeminiPart `json:"parts"`
}

GeminiContent represents a message in Gemini's format

type GeminiFunction 

type GeminiFunction struct {
	Name        string                 `json:"name"`
	Description string                 `json:"description"`
	Parameters  map[string]interface{} `json:"parameters"` // JSON Schema
}

GeminiFunction represents a tool definition

type GeminiFunctionCall 

type GeminiFunctionCall struct {
	Name string                 `json:"name"`
	Args map[string]interface{} `json:"args"`
}

GeminiFunctionCall for tool use

type GeminiFunctionResponse 

type GeminiFunctionResponse struct {
	Name     string                 `json:"name"`
	Response map[string]interface{} `json:"response"`
}

GeminiFunctionResponse for tool results

type GeminiGenerationConfig 

type GeminiGenerationConfig struct {
	Temperature     *float64              `json:"temperature,omitempty"`
	TopP            *float64              `json:"topP,omitempty"`
	TopK            *int                  `json:"topK,omitempty"`
	MaxOutputTokens int                   `json:"maxOutputTokens,omitempty"`
	ThinkingConfig  *GeminiThinkingConfig `json:"thinkingConfig,omitempty"`
}

GeminiGenerationConfig for model parameters

type GeminiInlineData 

type GeminiInlineData struct {
	MimeType string `json:"mimeType"`
	Data     string `json:"data"` // base64 encoded
}

GeminiInlineData for base64-encoded media

type GeminiPart 

type GeminiPart struct {
	Text             string                  `json:"text,omitempty"`
	InlineData       *GeminiInlineData       `json:"inlineData,omitempty"`
	FunctionCall     *GeminiFunctionCall     `json:"functionCall,omitempty"`
	FunctionResponse *GeminiFunctionResponse `json:"functionResponse,omitempty"`
	ThoughtSignature string                  `json:"thoughtSignature,omitempty"` // At Part level for Gemini 3
}

GeminiPart represents a single part of content (text, image, function call, etc.)

type GeminiProvider 

type GeminiProvider struct {
	// contains filtered or unexported fields
}

GeminiProvider implements the Provider interface for Google Gemini API

func NewGeminiProvider 

func NewGeminiProvider() *GeminiProvider

NewGeminiProvider creates a new Gemini provider instance

func (*GeminiProvider) GetRawStream 

func (p *GeminiProvider) GetRawStream(messages []stream.Message, customTools []tools.ToolDefinition, builtinTools []interface{}) (io.ReadCloser, error)

GetRawStream implements the Provider interface for streaming responses

func (*GeminiProvider) IsConfigured 

func (p *GeminiProvider) IsConfigured() bool

IsConfigured checks if the provider has an API key

func (*GeminiProvider) StreamChat 

func (p *GeminiProvider) StreamChat(w http.ResponseWriter, message string) error

StreamChat provides backward compatibility for simple streaming (deprecated, use GetRawStream)

type GeminiRequest 

type GeminiRequest struct {
	Contents          []GeminiContent         `json:"contents"`
	SystemInstruction *GeminiContent          `json:"system_instruction,omitempty"`
	GenerationConfig  *GeminiGenerationConfig `json:"generationConfig,omitempty"`
	Tools             []GeminiToolConfig      `json:"tools,omitempty"`
}

GeminiRequest is the request structure for Gemini API

type GeminiThinkingConfig 

type GeminiThinkingConfig struct {
	ThinkingBudget int `json:"thinkingBudget"` // 0 = disabled
}

GeminiThinkingConfig for thinking mode (2.5 models)

type GeminiToolConfig 

type GeminiToolConfig struct {
	FunctionDeclarations []GeminiFunction `json:"functionDeclarations"`
}

GeminiToolConfig wraps function declarations

type GeminiTranslator 

type GeminiTranslator struct {
	// contains filtered or unexported fields
}

GeminiTranslator - Convert to Gemini's format

func NewGeminiTranslator 

func NewGeminiTranslator() *GeminiTranslator

func (*GeminiTranslator) TranslateMessage 

func (t *GeminiTranslator) TranslateMessage(msg stream.Message) (interface{}, error)

type GroqProvider 

type GroqProvider struct {
	*OpenAICompatibleProvider
}

GroqProvider wraps the OpenAI-compatible base provider with Groq-specific configuration Groq provides ultra-fast inference with open-source models like Llama, Gemma, and Mixtral API Documentation: https://console.groq.com/docs/api-reference

func NewGroqProvider 

func NewGroqProvider() *GroqProvider

NewGroqProvider creates a new Groq provider instance Requires GROQ_API_KEY environment variable to be set

Popular models: - llama-3.3-70b-versatile (recommended, 128k context) - llama-3.1-8b-instant (ultra-fast, 131k context) - mixtral-8x7b-32768 (32k context) - gemma2-9b-it (Google, efficient)

type MessageTranslator 

type MessageTranslator interface {
	TranslateMessage(stream.Message) (interface{}, error)
}

MessageTranslator converts messages from universal format to provider-specific format

type MistralProvider added in v1.39.0 

type MistralProvider struct {
	*OpenAICompatibleProvider
}

MistralProvider wraps the OpenAI-compatible base provider with Mistral-specific configuration Mistral AI provides high-performance open and commercial models API Documentation: https://docs.mistral.ai/api/

func NewMistralProvider added in v1.39.0 

func NewMistralProvider() *MistralProvider

NewMistralProvider creates a new Mistral provider instance Requires MISTRAL_API_KEY environment variable to be set

Popular models (December 2025+): - mistral-large-3-25-12 (flagship, 675B MoE open-weight) - mistral-medium-3-1-25-08 (multimodal) - mistral-small-3-2-25-06 (fast, cost-effective) - magistral-medium-2509 (reasoning, 40K context) - devstral-2-25-12 (code agents) - codestral-2508 (code completion) - ministral-3-8b-25-12 (open-weight, fast)

type MoonshotProvider 

type MoonshotProvider struct {
	*OpenAICompatibleProvider
}

MoonshotProvider wraps the OpenAI-compatible base provider with Moonshot-specific configuration Moonshot AI (Kimi) provides advanced AI models with OpenAI-compatible API API Documentation: https://platform.moonshot.cn/docs

func NewMoonshotProvider 

func NewMoonshotProvider() *MoonshotProvider

NewMoonshotProvider creates a new Moonshot provider instance Requires MOONSHOT_API_KEY environment variable to be set

Popular models: - kimi-k2-turbo-preview (Latest turbo model, recommended) - kimi-k2.5 (Kimi K2.5) - kimi-k2-0711-preview (K2 preview) - moonshot-v1-128k (128K context) - moonshot-v1-32k (32K context) - moonshot-v1-8k (8K context, fast)

type NovitaProvider added in v1.37.0 

type NovitaProvider struct {
	*OpenAICompatibleProvider
}

NovitaProvider wraps the OpenAI-compatible base provider with Novita AI-specific configuration Novita AI provides a wide range of open-source models via API API Documentation: https://novita.ai/docs/model-api/reference/llm/llm.html

func NewNovitaProvider added in v1.37.0 

func NewNovitaProvider() *NovitaProvider

NewNovitaProvider creates a new Novita AI provider instance Requires NOVITA_API_KEY environment variable to be set

Popular models: - deepseek/deepseek-v3.2 (DeepSeek V3.2) - openai/gpt-oss-120b (GPT OSS 120B) - openai/gpt-oss-20b (GPT OSS 20B, fast) - qwen/qwen3-coder-480b-a35b-instruct (Qwen 3 Coder 480B) - zai-org/glm-4.7 (GLM 4.7)

type OllamaProvider added in v1.44.7 

type OllamaProvider struct {
	*OpenAICompatibleProvider
}

OllamaProvider wraps the OpenAI-compatible base provider for local Ollama instances. Ollama exposes an OpenAI-compatible API at /v1 and does not require an API key.

func NewOllamaProvider added in v1.44.7 

func NewOllamaProvider() *OllamaProvider

type OpenAIChoice 

type OpenAIChoice struct {
	Index        int         `json:"index"`
	Delta        OpenAIDelta `json:"delta"`
	FinishReason *string     `json:"finish_reason"`
}

type OpenAICompatibleProvider 

type OpenAICompatibleProvider struct {
	// contains filtered or unexported fields
}

OpenAICompatibleProvider provides a reusable base for all OpenAI-compatible APIs This includes: OpenAI, Groq, Together AI, Perplexity, Fireworks, and others

func NewOpenAICompatibleProvider 

func NewOpenAICompatibleProvider(apiKey, baseURL, authHeader, authPrefix, keyEnvVar string) *OpenAICompatibleProvider

NewOpenAICompatibleProvider creates a new OpenAI-compatible provider

func (*OpenAICompatibleProvider) GetLastCachedTokens added in v1.49.0 

func (p *OpenAICompatibleProvider) GetLastCachedTokens() int

GetLastCachedTokens returns cached prompt tokens from the last response (Fireworks header)

func (*OpenAICompatibleProvider) GetRawStream 

func (p *OpenAICompatibleProvider) GetRawStream(messages []stream.Message, customTools []tools.ToolDefinition, builtinTools []interface{}) (io.ReadCloser, error)

func (*OpenAICompatibleProvider) IsConfigured 

func (p *OpenAICompatibleProvider) IsConfigured() bool

func (*OpenAICompatibleProvider) SetBaseURL added in v1.44.7 

func (p *OpenAICompatibleProvider) SetBaseURL(baseURL string)

SetBaseURL updates the base URL for this provider

func (*OpenAICompatibleProvider) StreamChat 

func (p *OpenAICompatibleProvider) StreamChat(w http.ResponseWriter, message string) error

StreamChat provides backward compatibility for streaming

type OpenAIDelta 

type OpenAIDelta struct {
	Role    string `json:"role,omitempty"`
	Content string `json:"content,omitempty"`
}

type OpenAIFunctionDef 

type OpenAIFunctionDef struct {
	Name        string                 `json:"name"`
	Description string                 `json:"description"`
	Parameters  map[string]interface{} `json:"parameters"`
}

OpenAIFunctionDef represents a function definition in OpenAI format Note: OpenAI uses "parameters" instead of Anthropic's "input_schema"

type OpenAIMessage 

type OpenAIMessage struct {
	Role             string                   `json:"role"`
	Content          interface{}              `json:"content,omitempty"`           // Can be string or array of content parts
	ReasoningContent string                   `json:"reasoning_content,omitempty"` // For thinking models (Kimi K2, etc.) - MUST preserve in context
	ReasoningDetails []interface{}            `json:"reasoning_details,omitempty"` // For MiniMax M2.5 structured reasoning
	ToolCalls        []map[string]interface{} `json:"tool_calls,omitempty"`        // For assistant tool calls
	ToolCallID       string                   `json:"tool_call_id,omitempty"`      // For tool role messages
}

type OpenAIProvider 

type OpenAIProvider struct {
	*OpenAICompatibleProvider
}

OpenAIProvider wraps the OpenAI-compatible base provider with OpenAI-specific configuration

func NewOpenAIProvider 

func NewOpenAIProvider() *OpenAIProvider

type OpenAIRequest 

type OpenAIRequest struct {
	Model               string          `json:"model"`
	Messages            []OpenAIMessage `json:"messages"`
	MaxTokens           int             `json:"max_tokens,omitempty"`            // For older models and compatible APIs
	MaxCompletionTokens int             `json:"max_completion_tokens,omitempty"` // For newer OpenAI models (gpt-4o, gpt-5.x, o1, etc.)
	Temperature         *float64        `json:"temperature,omitempty"`
	Stream              bool            `json:"stream"`
	StreamOptions       *StreamOptions  `json:"stream_options,omitempty"` // Required for token usage in streaming mode
	Tools               []interface{}   `json:"tools,omitempty"`
	ReasoningEffort     string          `json:"reasoning_effort,omitempty"`  // For reasoning models (o1, o3, gpt-5.x)
	ReasoningHistory    string          `json:"reasoning_history,omitempty"` // For thinking models (Kimi K2): "interleaved" or "preserved"
	ReasoningSplit      *bool           `json:"reasoning_split,omitempty"`   // For MiniMax M2.5: true for structured reasoning_details
}

type OpenAIResponsesProvider added in v1.46.0 

type OpenAIResponsesProvider struct {
	// contains filtered or unexported fields
}

OpenAIResponsesProvider implements the Provider interface for OpenAI's Responses API. Used when operator mode is enabled with GPT-5.4+ for native computer use support.

func NewOpenAIResponsesProvider added in v1.46.0 

func NewOpenAIResponsesProvider() *OpenAIResponsesProvider

NewOpenAIResponsesProvider creates a new Responses API provider

func (*OpenAIResponsesProvider) APIKeyLen added in v1.46.0 

func (p *OpenAIResponsesProvider) APIKeyLen() int

APIKeyLen returns the length of the API key (for debugging without exposing the key)

func (*OpenAIResponsesProvider) GetPreviousResponseID added in v1.46.0 

func (p *OpenAIResponsesProvider) GetPreviousResponseID() string

GetPreviousResponseID returns the current response ID for chaining

func (*OpenAIResponsesProvider) GetRawStream added in v1.46.0 

func (p *OpenAIResponsesProvider) GetRawStream(messages []stream.Message, customTools []tools.ToolDefinition, builtinTools []interface{}) (io.ReadCloser, error)

GetRawStream implements the Provider interface for the Responses API

func (*OpenAIResponsesProvider) IsConfigured added in v1.46.0 

func (p *OpenAIResponsesProvider) IsConfigured() bool

func (*OpenAIResponsesProvider) NewSession added in v1.46.0 

func (p *OpenAIResponsesProvider) NewSession() *OpenAIResponsesProvider

NewSession creates a per-request copy that can track previousResponseID without state leaking between concurrent conversations.

func (*OpenAIResponsesProvider) SetPreviousResponseID added in v1.46.0 

func (p *OpenAIResponsesProvider) SetPreviousResponseID(id string)

SetPreviousResponseID implements ResponseIDTracker for conversation chaining

type OpenAIResponsesRequest added in v1.46.0 

type OpenAIResponsesRequest struct {
	Model              string              `json:"model"`
	Input              interface{}         `json:"input"` // Can be string or []ResponsesInputItem
	Tools              []interface{}       `json:"tools,omitempty"`
	PreviousResponseID string              `json:"previous_response_id,omitempty"`
	Stream             bool                `json:"stream"`
	Instructions       string              `json:"instructions,omitempty"`
	Reasoning          *ResponsesReasoning `json:"reasoning,omitempty"`
}

OpenAIResponsesRequest is the request structure for the Responses API

type OpenAIStreamResponse 

type OpenAIStreamResponse struct {
	ID      string         `json:"id"`
	Object  string         `json:"object"`
	Created int64          `json:"created"`
	Model   string         `json:"model"`
	Choices []OpenAIChoice `json:"choices"`
}

type OpenAIToolDef 

type OpenAIToolDef struct {
	Type     string            `json:"type"`
	Function OpenAIFunctionDef `json:"function"`
}

type OpenAITranslator 

type OpenAITranslator struct{}

OpenAITranslator - Convert to OpenAI's format

func (*OpenAITranslator) TranslateMessage 

func (t *OpenAITranslator) TranslateMessage(msg stream.Message) (interface{}, error)

type ResponsesReasoning added in v1.46.0 

type ResponsesReasoning struct {
	Effort string `json:"effort,omitempty"` // "low", "medium", "high"
}

ResponsesReasoning controls reasoning behavior for the Responses API

type StreamOptions added in v1.44.0 

type StreamOptions struct {
	IncludeUsage bool `json:"include_usage"`
}

StreamOptions controls streaming behavior (OpenAI-compatible APIs)

type TogetherProvider 

type TogetherProvider struct {
	*OpenAICompatibleProvider
}

TogetherProvider wraps the OpenAI-compatible base provider with Together AI-specific configuration Together AI provides fast inference for open-source models including Kimi K2 Thinking API Documentation: https://docs.together.ai/reference

func NewTogetherProvider 

func NewTogetherProvider() *TogetherProvider

NewTogetherProvider creates a new Together AI provider instance Requires TOGETHER_API_KEY environment variable to be set

Popular models: - moonshotai/Kimi-K2-Thinking (Kimi K2 Thinking, reasoning model) - moonshotai/Kimi-K2.5 (Kimi K2.5, reasoning model) - moonshotai/Kimi-K2-Instruct (Kimi K2 Instruct) - meta-llama/Llama-3.3-70B-Instruct-Turbo (Llama 3.3 70B, fast) - meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo (Llama 3.1 405B, largest) - meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo (Llama 3.1 70B) - meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo (Llama 3.1 8B, ultra-fast) - Qwen/Qwen2.5-72B-Instruct-Turbo (Qwen 2.5 72B) - deepseek-ai/DeepSeek-R1 (DeepSeek R1, reasoning) - deepseek-ai/DeepSeek-V3 (DeepSeek V3)

type VeniceProvider 

type VeniceProvider struct {
	*OpenAICompatibleProvider
}

VeniceProvider wraps the OpenAI-compatible base provider with Venice AI configuration Venice AI provides uncensored and private AI models API Documentation: https://docs.venice.ai/

func NewVeniceProvider 

func NewVeniceProvider() *VeniceProvider

NewVeniceProvider creates a new Venice AI provider instance Requires VENICE_API_KEY environment variable to be set

Popular models: - olafangensan-glm-4.7-flash-heretic (GLM 4.7 Flash Heretic, fast thinking/reasoning) - zai-org-glm-4.7 (GLM 4.7 Private, 203K context, $0.20/M in, $0.90/M out) - venice-uncensored (Uncensored model, no tool call support)

func (*VeniceProvider) GetRawStream 

func (p *VeniceProvider) GetRawStream(messages []stream.Message, customTools []tools.ToolDefinition, builtinTools []interface{}) (io.ReadCloser, error)

GetRawStream overrides the base provider to strip tools for models that don't support them

type XAIProvider 

type XAIProvider struct {
	*OpenAICompatibleProvider
}

XAIProvider wraps the OpenAI-compatible base provider with xAI-specific configuration xAI provides Grok models from X/Twitter API Documentation: https://docs.x.ai/api

func NewXAIProvider 

func NewXAIProvider() *XAIProvider

NewXAIProvider creates a new xAI provider instance Requires XAI_API_KEY environment variable to be set

Popular models: - grok-4-1-fast-reasoning (recommended, fast reasoning model) - grok-4-1-fast-non-reasoning (fast non-reasoning model) - grok-2-latest (latest Grok 2) - grok-2-1212 (Grok 2 December 2024) - grok-beta (original beta)

type ZAIProvider added in v1.37.0 

type ZAIProvider struct {
	*OpenAICompatibleProvider
}

ZAIProvider wraps the OpenAI-compatible base provider with Z.ai-specific configuration Z.ai (formerly Zhipu AI) provides GLM models API Documentation: https://docs.z.ai/guides/overview/quick-start

func NewZAIProvider added in v1.37.0 

func NewZAIProvider() *ZAIProvider

NewZAIProvider creates a new Z.ai provider instance Requires ZAI_API_KEY environment variable to be set

Popular models: - glm-5 (745B MoE, agentic, SOTA open-source coding) - glm-4.7 (flagship with reasoning, 200K context) - glm-4.6 (flagship, 200K context) - glm-4.5 (general purpose, 128K context) - glm-4.5v (vision, 128K context) - glm-4.5-flash (free tier, 128K context) - glm-4.5-air (lightweight, 128K context)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.