llm

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 24, 2026 License: MIT Imports: 17 Imported by: 0

README

thinwrap/llm (Go)

⚠️ Work in progress (v0). A faithful Go port of @thinwrap/llm (TypeScript) and thinwrap/llm (PHP): one unified Chat + Embeddings surface over 18 providers, with the same normalized shapes and error model.

Unified, SDK-free, zero-dependency Go wrapper for chat completions (with streaming) and embeddings across 18 LLM providers. Switch vendor by changing the provider id + model; the input and output shapes stay identical. Stateless — bring your own *http.Client, no vendor SDKs.

import "github.com/thinwrap/llm-go"

Requires Go ≥ 1.18. No third-party dependencies — Bedrock's SigV4 is hand-rolled on crypto/*, SSE streaming on bufio.

Two-minute chat

chat, err := llm.NewChat(llm.OpenAI, llm.OpenAICompatConfig{APIKey: os.Getenv("OPENAI_API_KEY")})
if err != nil { log.Fatal(err) }

res, err := chat.Complete(ctx, llm.ChatInput{
    Model:    "gpt-4o-mini",
    Messages: []llm.ChatMessage{{Role: "user", Content: "Say hi in one word."}},
})
if err != nil {
    var ce *llm.ConnectorError
    if errors.As(err, &ce) { log.Printf("%s: %s", ce.ProviderCode, ce.ProviderMessage) }
    return
}
fmt.Println(res.Message.Content, res.Usage.TotalTokens)

Streaming

stream, err := chat.Stream(ctx, in)
if err != nil { return }
defer stream.Close()
for {
    delta, err := stream.Recv()
    if err == io.EOF { break }
    if err != nil { return }
    fmt.Print(delta.ContentDelta)
}

Providers

The provider id selects the connector; ChatInput / ChatResult are identical across all of them.

Chat (18): the 15 first-class OpenAI-compatible providers — OpenAI, AzureOpenAI, OpenRouter, Groq, Together, Fireworks, DeepSeek, XAI, Mistral, Perplexity, DeepInfra, Cloudflare, VLLM, Ollama, LMStudio — all take OpenAICompatConfig; plus 3 natives: Anthropic (AnthropicConfig), Bedrock (BedrockConfig), Gemini (GeminiConfig).

Embeddings (11): OpenAI, AzureOpenAI, OpenRouter, Together, Fireworks, Mistral, DeepInfra, Cloudflare, VLLM, Ollama, LMStudio (the OpenAI-float subset).

emb, _ := llm.NewEmbeddings(llm.OpenAI, llm.OpenAICompatConfig{APIKey: key})
out, _ := emb.Create(ctx, llm.EmbeddingsInput{Model: "text-embedding-3-small", Input: []string{"hello", "world"}})
// out.Embeddings is [][]float64 in input order

Per-provider details (endpoint, auth, quirks, passthrough keys) live in docs/providers/ — one page per provider.

Switching providers

// Same Complete(ctx, in) call, same ChatResult — id + model change only.
chat, _ := llm.NewChat(llm.Anthropic, llm.AnthropicConfig{APIKey: key})
// then in.Model = "claude-sonnet-4-5", etc.

OpenAICompatConfig covers all 15 first-class providers (APIKey, BaseURL, Headers); BaseURL is required for AzureOpenAI / Cloudflare and any self-host provider on a non-default host.

The ≥90% baseline & passthrough

Only fields ≥90% of providers support normalizably are first-class on ChatInput / ChatResult. Everything else rides through Passthrough (request) / Raw (response) and is never emulated:

res, _ := chat.Complete(ctx, llm.ChatInput{
    Model:    "...",
    Messages: msgs,
    Passthrough: &llm.Passthrough{Body: map[string]any{"logprobs": true}},
})
_ = res.Raw // verbatim vendor body — reasoning CoT, cached-token counts, cost, ...

Errors

Every failure is a *ConnectorError (errors.As): ProviderCode is one of rate_limited, auth_failed, provider_unavailable, invalid_request, context_length_exceeded, content_filtered, unknown. The raw Retry-After rides in Cause["retryAfter"] (+ parsed Cause["retryAfterSeconds"]) — there is no top-level retry field.

Relationship to the siblings

One of four thinwrap llm libraries sharing a normalized surface and byte-identical provider ids / error codes: TypeScript (@thinwrap/llm), PHP (thinwrap/llm), Go (this package), Python (thinwrap-llm).

License

MIT © Dmitry Polyanovsky & contributors

Documentation

Overview

Package llm is a lightweight, SDK-free, zero-dependency Go wrapper for chat completions and embeddings across 18 LLM providers behind one normalized surface.

It is the Go sibling of @thinwrap/llm (npm) and thinwrap/llm (Packagist), sharing the same normalized surface: one ChatInput / ChatResult / ChatStreamDelta shape across every provider, an Embeddings operation, and a single typed *ConnectorError.

Fifteen first-class providers speak OpenAI Chat Completions and share one connector (parameterized per provider); three natives — Anthropic, Bedrock, Gemini — have bespoke wire adapters that emit the identical normalized shapes, so they stay drop-in swappable. 

chat, err := llm.NewChat(llm.OpenAI, llm.OpenAICompatConfig{APIKey: key})
if err != nil { ... }
res, err := chat.Complete(ctx, llm.ChatInput{
	Model:    "gpt-4o-mini",
	Messages: []llm.ChatMessage{{Role: "user", Content: "Hello"}},
})
if err != nil {
	var ce *llm.ConnectorError
	if errors.As(err, &ce) { /* ce.ProviderCode, ce.StatusCode */ }
}
fmt.Println(res.Message.Content, res.Usage.TotalTokens)

Streaming: 

stream, err := chat.Stream(ctx, in)
if err != nil { ... }
defer stream.Close()
for {
	delta, err := stream.Recv()
	if err == io.EOF { break }
	if err != nil { ... }
	fmt.Print(delta.ContentDelta)
}

Switching vendor is a change of provider id + model only. The wrapper holds no state (no caching, retries, or conversation memory); provider-specific request fields flow through Passthrough and are never emulated. Zero third-party dependencies: Bedrock's SigV4 is hand-rolled on the standard library.

Index

Constants

This section is empty.

Variables

View Source 
var ProviderIDs = []ProviderID{
	OpenAI, AzureOpenAI, OpenRouter, Groq, Together, Fireworks, DeepSeek, XAI,
	Mistral, Perplexity, DeepInfra, Cloudflare, VLLM, Ollama, LMStudio,
	Anthropic, Bedrock, Gemini,
}

ProviderIDs is the full list of implemented provider ids.

Functions

This section is empty.

Types

type AnthropicConfig 

type AnthropicConfig struct {
	// APIKey is sent as the x-api-key header (not Bearer).
	APIKey string
	// BaseURL overrides the default https://api.anthropic.com/v1.
	BaseURL string
	// AnthropicVersion is the anthropic-version header (default 2023-06-01).
	AnthropicVersion string
	// DefaultMaxTokens is used when ChatInput.MaxOutputTokens is unset (default 4096).
	DefaultMaxTokens int
	Headers          map[string]string
}

AnthropicConfig configures the Anthropic Messages API native adapter.

type AssistantMessage 

type AssistantMessage struct {
	Role      ChatRole
	Content   string
	ToolCalls []ToolCall
}

AssistantMessage is the assistant turn on a ChatResult.

type BedrockConfig 

type BedrockConfig struct {
	Region          string
	AccessKeyID     string
	SecretAccessKey string
	SessionToken    string
	// BaseURL overrides the default https://bedrock-runtime.<region>.amazonaws.com.
	BaseURL string
	// DefaultMaxTokens is used when ChatInput.MaxOutputTokens is unset (default 4096).
	DefaultMaxTokens int
	// Headers participate in the SigV4 signature.
	Headers map[string]string
}

BedrockConfig configures the AWS Bedrock Converse native adapter (SigV4).

type Chat 

type Chat struct {
	// contains filtered or unexported fields
}

Chat is the chat-completion facade. Construct it with NewChat.

func NewChat 

func NewChat(id ProviderID, cfg ChatConfig, opts ...Option) (*Chat, error)

NewChat builds a chat facade for the given provider id. For the 15 first-class providers pass an OpenAICompatConfig; for a native pass its config. Switching vendor is a change of id + model only; ChatInput/ChatResult are identical.

func (*Chat) Complete 

func (c *Chat) Complete(ctx context.Context, in ChatInput) (*ChatResult, error)

Complete performs a non-streaming chat completion.

func (*Chat) ProviderID 

func (c *Chat) ProviderID() ProviderID

ProviderID returns the provider this facade dispatches to.

func (*Chat) Stream 

func (c *Chat) Stream(ctx context.Context, in ChatInput) (*ChatStream, error)

Stream performs a streaming chat completion. Read deltas from the returned *ChatStream with Recv until it returns io.EOF, then Close it.

type ChatConfig 

type ChatConfig interface {
	// contains filtered or unexported methods
}

ChatConfig is implemented by every chat-provider config value (OpenAICompatConfig for the 15 first-class providers; AnthropicConfig / BedrockConfig / GeminiConfig for the natives).

type ChatInput 

type ChatInput struct {
	Model           string
	Messages        []ChatMessage
	Tools           []ToolDefinition
	ToolChoice      *ToolChoice
	ResponseFormat  *ResponseFormat
	Temperature     *float64
	TopP            *float64
	MaxOutputTokens *int
	Stop            []string
	Reasoning       *ReasoningOptions
	Passthrough     *Passthrough
}

ChatInput is the normalized request. Pointer fields distinguish unset from zero.

type ChatMessage 

type ChatMessage struct {
	Role         ChatRole
	Content      string
	ContentParts []ContentPart
	// ToolCalls is present on assistant turns that invoked tools.
	ToolCalls []ToolCall
	// ToolCallID is required on role:"tool" messages.
	ToolCallID string
	// Name is an optional author / tool name.
	Name string
}

ChatMessage is one turn. Use Content for plain text, or ContentParts for multimodal (ContentParts wins when non-empty).

type ChatResult 

type ChatResult struct {
	Message      AssistantMessage
	FinishReason FinishReason
	Usage        Usage
	// Model is the model id the provider reported serving.
	Model string
	// Raw is the verbatim decoded vendor response body.
	Raw any
}

ChatResult is the normalized non-streaming completion.

type ChatRole 

type ChatRole = string

ChatRole is one of "system", "user", "assistant", "tool".

type ChatStream 

type ChatStream struct {
	// contains filtered or unexported fields
}

ChatStream is a pull-based stream of ChatStreamDelta. Recv returns io.EOF when the stream is exhausted. Always Close it (safe to defer).

func (*ChatStream) Close 

func (s *ChatStream) Close() error

Close releases the underlying response body.

func (*ChatStream) Recv 

func (s *ChatStream) Recv() (*ChatStreamDelta, error)

Recv returns the next delta, or io.EOF at end of stream.

type ChatStreamDelta 

type ChatStreamDelta struct {
	ContentDelta  string
	ToolCallDelta *ToolCallDelta
	FinishReason  FinishReason
	Usage         *Usage
	Raw           any
}

ChatStreamDelta is one incremental streaming chunk.

type ConnectorError 

type ConnectorError struct {
	StatusCode      *int
	ProviderCode    ProviderCode
	ProviderMessage string
	Message         string
	Cause           any
}

ConnectorError is the single typed error every operation can return. Retrieve it with errors.As: 

var ce *llm.ConnectorError
if errors.As(err, &ce) { ... }

There is deliberately no top-level RetryAfterSeconds field: for a vendor HTTP error the raw Retry-After header rides in Cause["retryAfter"] and its parsed seconds in Cause["retryAfterSeconds"].

func (*ConnectorError) Error 

func (e *ConnectorError) Error() string

func (*ConnectorError) Unwrap 

func (e *ConnectorError) Unwrap() error

Unwrap exposes an underlying error cause (e.g. a transport error).

type ContentPart 

type ContentPart struct {
	Type string
	// Text is set when Type == "text".
	Text string
	// Base64 is the raw base64 image payload (no data: prefix) when Type == "image".
	Base64 string
	// MediaType e.g. "image/png" when Type == "image".
	MediaType string
}

ContentPart is one part of a multimodal message. Type is "text" or "image".

func ImagePart 

func ImagePart(base64, mediaType string) ContentPart

func TextPart 

func TextPart(text string) ContentPart

TextPart / ImagePart constructors for ergonomics.

type Embeddings 

type Embeddings struct {
	// contains filtered or unexported fields
}

Embeddings is the embeddings facade — a separate operation from Chat. Only the OpenAI-float subset of providers is supported.

func NewEmbeddings 

func NewEmbeddings(id ProviderID, cfg OpenAICompatConfig, opts ...Option) (*Embeddings, error)

NewEmbeddings builds an embeddings facade for a first-class provider that exposes an OpenAI-float /embeddings surface.

func (*Embeddings) Create 

func (e *Embeddings) Create(ctx context.Context, in EmbeddingsInput) (*EmbeddingsResult, error)

Create computes embeddings for the input(s).

func (*Embeddings) ProviderID 

func (e *Embeddings) ProviderID() ProviderID

ProviderID returns the provider this facade dispatches to.

type EmbeddingsInput 

type EmbeddingsInput struct {
	Model string
	Input []string
	// Dimensions requests a specific output dimensionality (Matryoshka), when supported.
	Dimensions  *int
	Passthrough *Passthrough
}

EmbeddingsInput is the normalized embeddings request. Input is one or more strings (embedded in input order).

type EmbeddingsResult 

type EmbeddingsResult struct {
	// Embeddings has one float vector per input, in input order.
	Embeddings [][]float64
	Usage      EmbeddingsUsage
	Model      string
	Raw        any
}

EmbeddingsResult is the normalized embeddings response.

type EmbeddingsUsage 

type EmbeddingsUsage struct {
	InputTokens int
	TotalTokens int
}

EmbeddingsUsage is normalized token accounting for embeddings.

type FinishReason 

type FinishReason = string

FinishReason is one of "stop", "length", "tool_calls", "content_filter", "unknown".

type GeminiConfig 

type GeminiConfig struct {
	// APIKey is sent as the x-goog-api-key header.
	APIKey string
	// BaseURL overrides the default https://generativelanguage.googleapis.com/v1beta.
	BaseURL          string
	DefaultMaxTokens int
	Headers          map[string]string
}

GeminiConfig configures the Google Gemini generateContent native adapter.

type HTTPClient 

type HTTPClient interface {
	Do(req *http.Request) (*http.Response, error)
}

HTTPClient is the bring-your-own HTTP seam, satisfied by *http.Client. Inject any implementation for tracing, retries, mocking, or proxying.

type OpenAICompatConfig 

type OpenAICompatConfig struct {
	APIKey  string
	BaseURL string
	// Headers are merged onto every request (e.g. OpenRouter HTTP-Referer / X-Title).
	Headers map[string]string
}

OpenAICompatConfig configures any of the 15 first-class OpenAI-compatible providers (the provider id selects which). APIKey is optional for self-host providers (vLLM / Ollama / LM Studio) that run without auth. BaseURL is required for providers with no fixed public default (azure-openai, cloudflare).

type Option 

type Option func(*facadeOptions)

Option configures a facade constructor.

func WithHTTPClient 

func WithHTTPClient(c HTTPClient) Option

WithHTTPClient injects a custom HTTP client (see HTTPClient). Without it, a non-redirect-following default is used.

type Passthrough 

type Passthrough struct {
	Body    map[string]any    `json:"body,omitempty"`
	Headers map[string]string `json:"headers,omitempty"`
	Query   map[string]string `json:"query,omitempty"`
}

Passthrough is the per-request escape hatch for sub-baseline / provider- specific fields. Body is shallow-merged onto (and overrides) the connector- built request body; Headers are merged onto request headers; Query is appended to the URL. Never emulated — forwarded verbatim.

type ProviderCode 

type ProviderCode string

ProviderCode is the normalized, cross-provider error classification surfaced on every *ConnectorError. The values are byte-identical across the thinwrap llm siblings (TypeScript, PHP, Go). 

const (
	CodeRateLimited           ProviderCode = "rate_limited"
	CodeAuthFailed            ProviderCode = "auth_failed"
	CodeProviderUnavailable   ProviderCode = "provider_unavailable"
	CodeInvalidRequest        ProviderCode = "invalid_request"
	CodeContextLengthExceeded ProviderCode = "context_length_exceeded"
	CodeContentFiltered       ProviderCode = "content_filtered"
	CodeUnknown               ProviderCode = "unknown"
)

type ProviderID 

type ProviderID string

ProviderID is the stable, user-facing identifier for an LLM provider. The string values are byte-identical across the thinwrap llm siblings (TypeScript, PHP, Go). 

const (
	// First-class OpenAI-compatible providers (one shared connector).
	OpenAI      ProviderID = "openai"
	AzureOpenAI ProviderID = "azure-openai"
	OpenRouter  ProviderID = "openrouter"
	Groq        ProviderID = "groq"
	Together    ProviderID = "together"
	Fireworks   ProviderID = "fireworks"
	DeepSeek    ProviderID = "deepseek"
	XAI         ProviderID = "xai"
	Mistral     ProviderID = "mistral"
	Perplexity  ProviderID = "perplexity"
	DeepInfra   ProviderID = "deepinfra"
	Cloudflare  ProviderID = "cloudflare"
	VLLM        ProviderID = "vllm"
	Ollama      ProviderID = "ollama"
	LMStudio    ProviderID = "lmstudio"

	// Native adapters (structurally different wire, identical normalized surface).
	Anthropic ProviderID = "anthropic"
	Bedrock   ProviderID = "bedrock"
	Gemini    ProviderID = "gemini"
)

type ReasoningOptions 

type ReasoningOptions struct {
	Effort string
}

ReasoningOptions is the minimal normalized reasoning control. Effort is "low", "medium", or "high". CoT output is not normalized — read it from ChatResult.Raw.

type ResponseFormat 

type ResponseFormat struct {
	Type           string
	JSONSchemaName string
	JSONSchema     map[string]any
}

ResponseFormat controls structured output. Type is "text", "json_object", or "json_schema" (with JSONSchemaName + JSONSchema set).

type ToolCall 

type ToolCall struct {
	ID   string
	Name string
	// Arguments is a JSON-encoded arguments string.
	Arguments string
}

ToolCall is a function call the assistant emitted.

type ToolCallDelta 

type ToolCallDelta struct {
	Index          int
	ID             string
	FunctionName   string
	ArgumentsDelta string
}

ToolCallDelta is an incremental tool-call fragment (concatenate ArgumentsDelta per Index).

type ToolChoice 

type ToolChoice struct {
	Mode         string
	FunctionName string
}

ToolChoice controls tool selection. Mode is "auto", "none", "required", or "function" (with FunctionName set).

type ToolDefinition 

type ToolDefinition struct {
	Name        string
	Description string
	// Parameters is a JSON Schema object describing the arguments.
	Parameters map[string]any
}

ToolDefinition describes a callable function (type is implicitly "function").

type Usage 

type Usage struct {
	InputTokens  int
	OutputTokens int
	TotalTokens  int
}

Usage is normalized token accounting.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.