llm

package module

v0.9.1-0...-f97c2f4 Latest Latest Go to latest Published: Aug 28, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/humbornjo/llm

Links

Open Source Insights

README ¶

llm

Communicate with any LLM provider using a single, unified interface. Switch between OpenAI, Anthropic, Mistral, Ollama, and more without changing your code.

Python SDK | Documentation | Platform (Beta)

[!NOTE] This repository is an independent downstream fork of mozilla-ai/any-llm-go, maintained under the module path github.com/humbornjo/llm. The fork preserves the upstream Git history and Apache License 2.0 attribution while allowing its API and provider abstractions to evolve for Fuss independently of upstream. It is not an official Mozilla AI distribution and is not endorsed by Mozilla AI. See LICENSE for the applicable terms.

Quickstart

go get github.com/humbornjo/llm
export OPENAI_API_KEY="YOUR_KEY_HERE"  # or ANTHROPIC_API_KEY, etc

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/humbornjo/llm"
    "github.com/humbornjo/llm/providers/openai"
)

func main() {
    ctx := context.Background()

    provider, err := openai.New()
    if err != nil {
        log.Fatal(err)
    }

    response, err := provider.Completion(ctx, llm.CompletionParams{
        Model: "gpt-4o-mini",
        Messages: []llm.Message{
            {Role: llm.ROLE_USER, Content: "Hello!"},
        },
    })
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(response.Choices[0].Message.Content)
}

That's it! To switch providers, change the import and constructor (e.g., anthropic.New() instead of openai.New()).

Installation

Requirements

Go 1.26 or newer
API keys for whichever LLM providers you want to use

Import the main package and the providers you need:

import (
    "github.com/humbornjo/llm"
    "github.com/humbornjo/llm/providers/openai"    // OpenAI
    "github.com/humbornjo/llm/providers/anthropic" // Anthropic
)

See our list of supported providers to choose which ones you need.

Setting Up API Keys

Set environment variables for your chosen providers:

export OPENAI_API_KEY="your-key-here"
export ANTHROPIC_API_KEY="your-key-here"
export MISTRAL_API_KEY="your-key-here"
# ... etc

Alternatively, pass API keys directly in your code:

provider, err := openai.New(llm.WithAPIKey("your-key-here"))

any-llm-gateway

any-llm-gateway is an optional FastAPI-based proxy server that adds enterprise-grade features on top of the core library:

Budget Management - Enforce spending limits with automatic daily, weekly, or monthly resets
API Key Management - Issue, revoke, and monitor virtual API keys without exposing provider credentials
Usage Analytics - Track every request with full token counts, costs, and metadata
Multi-tenant Support - Manage access and budgets across users and teams

The gateway sits between your applications and LLM providers, exposing an OpenAI-compatible API that works with any supported provider.

Quick Start

docker run \
  -e GATEWAY_MASTER_KEY="your-secure-master-key" \
  -e OPENAI_API_KEY="your-api-key" \
  -p 8000:8000 \
  ghcr.io/mozilla-ai/any-llm/gateway:latest

Note: You can use a specific release version instead of latest (e.g., 1.2.0). See available versions.

Managed Platform (Beta)

Prefer a hosted experience? The any-llm platform provides a managed control plane for keys, usage tracking, and cost visibility across providers, while still building on the same any-llm interfaces.

Usage

Create a provider instance and use it for requests:

import (
    "context"
    "fmt"
    "log"

    "github.com/humbornjo/llm"
    "github.com/humbornjo/llm/providers/openai"
)

provider, err := openai.New(llm.WithAPIKey("your-api-key"))
if err != nil {
    log.Fatal(err)
}

ctx := context.Background()

response, err := provider.Completion(ctx, llm.CompletionParams{
    Model: "gpt-4o-mini",
    Messages: []llm.Message{
        {Role: llm.ROLE_USER, Content: "Hello!"},
    },
})
if err != nil {
    log.Fatal(err)
}

fmt.Println(response.Choices[0].Message.Content)

Provider instances are reusable and recommended for production applications.

Streaming

chunks, errs := provider.CompletionStream(ctx, llm.CompletionParams{
    Model: "gpt-4o-mini",
    Messages: []llm.Message{
        {Role: llm.ROLE_USER, Content: "Write a short poem about Go."},
    },
})

for chunk := range chunks {
    if len(chunk.Choices) > 0 {
        fmt.Print(chunk.Choices[0].Delta.Content)
    }
}

if err := <-errs; err != nil {
    log.Fatal(err)
}

Tools / Function Calling

response, err := provider.Completion(ctx, llm.CompletionParams{
    Model: "gpt-4o-mini",
    Messages: []llm.Message{
        {Role: llm.ROLE_USER, Content: "What's the weather in Paris?"},
    },
    Tools: []llm.ToolInfo{
        {
            Type: "function",
            Function: llm.Function{
                Name:        "get_weather",
                Description: "Get the current weather for a location",
                Parameters: map[string]any{
                    "type": "object",
                    "properties": map[string]any{
                        "location": map[string]any{
                            "type":        "string",
                            "description": "The city name",
                        },
                    },
                    "required": []string{"location"},
                },
            },
        },
    },
    ToolChoice: "auto",
})

// Check for tool calls.
if len(response.Choices[0].Message.ToolCalls) > 0 {
    tc := response.Choices[0].Message.ToolCalls[0]
    fmt.Printf("Function: %s, Args: %s\n", tc.Function.Name, tc.Function.Arguments)
}

ToolInfo is the serializable declaration sent to an LLM provider. Tool is the executable interface used by agent loops and tool dispatchers. An implementation supplies its Info and Function, plus synchronous and streaming execution methods. Per-call metadata can be passed with WithToolMetadata; implementations apply each ToolOption directly to a zero-value ToolConfig before execution.

Extended Thinking (Reasoning)

For models that support extended thinking (like Claude):

response, err := provider.Completion(ctx, llm.CompletionParams{
    Model: "claude-sonnet-4-20250514",
    Messages: []llm.Message{
        {Role: llm.ROLE_USER, Content: "Solve this step by step: What is 15% of 80?"},
    },
    ReasoningEffort: llm.REASONING_EFFORT_MEDIUM,
})

if response.Choices[0].Message.Reasoning != nil {
    fmt.Println("Thinking:", response.Choices[0].Message.Reasoning.Content)
}
fmt.Println("Answer:", response.Choices[0].Message.Content)

Embeddings

provider, _ := openai.New()
result, err := provider.Embedding(ctx, llm.EmbeddingParams{
    Model: "text-embedding-3-small",
    Input: "Hello world",
})

Listing Models

provider, _ := openai.New()
models, err := provider.ListModels(ctx)
for _, model := range models.Data {
    fmt.Println(model.ID)
}

Moderation

The gateway provider supports OpenAI-compatible content moderation. Use errors.As with *llm.UnsupportedOperationError (or errors.Is with llm.ErrUnsupported) to detect providers that do not support moderation.

import (
    stderrors "errors"

    "github.com/humbornjo/llm"
    "github.com/humbornjo/llm/config"
    "github.com/humbornjo/llm/providers/gateway"
)

provider, err := gateway.New(config.WithBaseURL("https://gw.example.com"))
if err != nil {
    log.Fatal(err)
}

resp, err := provider.Moderation(ctx, llm.ModerationParams{
    Model: "openai:omni-moderation-latest",
    Input: "I want to hurt someone",
})
if err != nil {
    var unsup *llm.UnsupportedOperationError
    if stderrors.As(err, &unsup) {
        // Provider does not support moderation; pick another model.
        log.Printf("%s cannot do %s", unsup.Provider, unsup.Operation)
        return
    }
    log.Fatal(err)
}
if resp.Results[0].Flagged {
    // Handle flagged content.
}

Error Handling

All provider errors are normalized to common error types:

response, err := provider.Completion(ctx, params)
if err != nil {
    switch {
    case errors.Is(err, llm.ErrRateLimit):
        // Handle rate limiting - maybe retry with backoff.
    case errors.Is(err, llm.ErrAuthentication):
        // Handle auth errors - check API key.
    case errors.Is(err, llm.ErrContextLength):
        // Handle context too long - reduce input.
    default:
        // Handle other errors.
    }
}

You can also use type assertions for more details:

var rateLimitErr *llm.RateLimitError
if errors.As(err, &rateLimitErr) {
    fmt.Printf("Rate limited by %s: %s\n", rateLimitErr.Provider, rateLimitErr.Message)
}

Supported Providers

Provider	Completion	Streaming	Tools	Reasoning	Embeddings
Anthropic	✅	✅	✅	✅	❌
DeepSeek	✅	✅	✅	✅	❌
Gemini	✅	✅	✅	✅	✅
Groq	✅	✅	✅	❌	❌
llama.cpp	✅	✅	✅	❌	✅
Llamafile	✅	✅	✅	❌	✅
Mistral	✅	✅	✅	✅	✅
Ollama	✅	✅	✅	✅	✅
OpenAI	✅	✅	✅	✅	✅
z.ai	✅	✅	✅	✅	❌

Why choose `llm`?

Simple, unified interface - Same types and patterns across all providers, switch models with just a string change
Developer friendly - Full type definitions for better IDE support and clear, actionable error messages
Leverages official provider SDKs - Uses github.com/openai/openai-go and github.com/anthropics/anthropic-sdk-go for maximum compatibility
Stays framework-agnostic so it can be used across different projects and use cases
Idiomatic Go - Follows Go conventions with proper error handling and context support
Streaming support - Channel-based streaming that's natural in Go
Battle-tested - Based on the proven any-llm Python library

Development

make lint       # Run linter with auto-fix
make test       # Lint + run all tests
make test-only  # Run tests without linting
make test-unit  # Run unit tests only (skip integration)
make build      # Verify compilation

Documentation

Full Documentation - Complete guides and API reference
Supported Providers - List of all supported LLM providers
Gateway Documentation - Gateway setup and deployment
Python SDK - The full Python SDK with direct provider access
any-llm Platform (Beta) - Hosted control plane for key management, usage tracking, and cost visibility

Contributing

We welcome contributions from developers of all skill levels! Please see our Contributing Guide or open an issue to discuss changes.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Documentation ¶

Overview ¶

Package llm provides a unified interface for interacting with LLM providers.

This package re-exports common types and configuration options from subpackages, allowing most use cases to work with just two imports:

import (
    "github.com/humbornjo/llm"
    "github.com/humbornjo/llm/providers/openai"
)

provider, err := openai.New(llm.WithAPIKey("sk-..."))
response, err := provider.Completion(ctx, llm.CompletionParams{
    Model: "gpt-4o-mini",
    Messages: []llm.Message{
        {Role: llm.ROLE_USER, Content: "Hello!"},
    },
})

Index ¶

Constants
Variables
func NewToolsHandler(tools ...Tool) func(context.Context, FunctionCall, ...ToolOption) (string, error)
type AuthenticationError
type BaseError
type Batch
type BatchProvider
type BatchRequestCounts
type BatchRequestItem
type BatchResult
type BatchResultError
type BatchResultItem
type BatchStatus
type Capabilities
type CapabilityProvider
type ChatCompletion
type ChatCompletionChunk
type Choice
type ChunkChoice
type ChunkDelta
type CompletionParams
type CompletionTokensDetails
type Config
type Content
type ContentFilterError
type ContentPart
type ContentPartAudio
type ContentPartFile
type ContentPartImage
type ContentPartText
type ContentPartType
type ContentParts
type ContentString
type ContextLengthError
type CreateBatchParams
type EmbeddingData
type EmbeddingParams
type EmbeddingProvider
type EmbeddingResponse
type EmbeddingUsage
type File
type Function
type FunctionCall
type ImageURL
type InputAudio
type InsufficientFundsError
type InvalidRequestError
type JSONSchema
type ListBatchesOptions
type Message
type MissingAPIKeyError
type Model
type ModelLister
type ModelNotFoundError
type ModelsResponse
type ModerationParams
type ModerationProvider
type ModerationResponse
type ModerationResult
type Option
type PromptTokensDetails
type Provider
type ProviderError
type RateLimitError
type Reasoning
type ReasoningEffort
type RerankMeta
type RerankParams
type RerankProvider
type RerankResponse
type RerankResult
type RerankUsage
type ResponseFormat
type StreamOptions
type Tool
- func NewTool[T any](info ToolInfo, ...) Tool
type ToolCall
type ToolChoice
type ToolChoiceFunction
type ToolConfig
type ToolInfo
type ToolOption
- func WithToolMetadata(metadata map[string]any) ToolOption
type UnsupportedOperationError
type UnsupportedParamError
type UnsupportedProviderError
type Usage

Constants ¶

View Source

const (
	ROLE_ASSISTANT = providers.ROLE_ASSISTANT
	ROLE_SYSTEM    = providers.ROLE_SYSTEM
	ROLE_TOOL      = providers.ROLE_TOOL
	ROLE_USER      = providers.ROLE_USER
)

Message roles.

View Source

const (
	CONTENT_PART_TEXT        = providers.CONTENT_PART_TEXT
	CONTENT_PART_FILE        = providers.CONTENT_PART_FILE
	CONTENT_PART_IMAGE_URL   = providers.CONTENT_PART_IMAGE_URL
	CONTENT_PART_INPUT_AUDIO = providers.CONTENT_PART_INPUT_AUDIO
)

Content part type constants.

View Source

const (
	FINISH_REASON_CONTENT_FILTER = providers.FINISH_REASON_CONTENT_FILTER
	FINISH_REASON_LENGTH         = providers.FINISH_REASON_LENGTH
	FINISH_REASON_STOP           = providers.FINISH_REASON_STOP
	FINISH_REASON_TOOL_CALLS     = providers.FINISH_REASON_TOOL_CALLS
)

Finish reasons.

View Source

const (
	BATCH_STATUS_CANCELLED   = providers.BATCH_STATUS_CANCELLED
	BATCH_STATUS_CANCELLING  = providers.BATCH_STATUS_CANCELLING
	BATCH_STATUS_COMPLETED   = providers.BATCH_STATUS_COMPLETED
	BATCH_STATUS_EXPIRED     = providers.BATCH_STATUS_EXPIRED
	BATCH_STATUS_FAILED      = providers.BATCH_STATUS_FAILED
	BATCH_STATUS_FINALIZING  = providers.BATCH_STATUS_FINALIZING
	BATCH_STATUS_IN_PROGRESS = providers.BATCH_STATUS_IN_PROGRESS
	BATCH_STATUS_VALIDATING  = providers.BATCH_STATUS_VALIDATING
)

Batch status constants.

View Source

const (
	REASONING_EFFORT_AUTO   = providers.REASONING_EFFORT_AUTO
	REASONING_EFFORT_HIGH   = providers.REASONING_EFFORT_HIGH
	REASONING_EFFORT_LOW    = providers.REASONING_EFFORT_LOW
	REASONING_EFFORT_MEDIUM = providers.REASONING_EFFORT_MEDIUM
	REASONING_EFFORT_NONE   = providers.REASONING_EFFORT_NONE
)

ReasoningEffort levels.

Variables ¶

View Source

var (
	NewConfig         = config.New
	WithAPIKey        = config.WithAPIKey
	WithBaseURL       = config.WithBaseURL
	WithExtra         = config.WithExtra
	WithHTTPClient    = config.WithHTTPClient
	WithTimeout       = config.WithTimeout
	ContentFromParts  = providers.ContentFromParts
	ContentFromString = providers.ContentFromString
)

Configuration options.

View Source

var (
	ErrAuthentication      = errors.ErrAuthentication
	ErrContentFilter       = errors.ErrContentFilter
	ErrContextLength       = errors.ErrContextLength
	ErrInsufficientFunds   = errors.ErrInsufficientFunds
	ErrInvalidRequest      = errors.ErrInvalidRequest
	ErrMissingAPIKey       = errors.ErrMissingAPIKey
	ErrModelNotFound       = errors.ErrModelNotFound
	ErrProvider            = errors.ErrProvider
	ErrRateLimit           = errors.ErrRateLimit
	ErrUnsupported         = errors.ErrUnsupported
	ErrUnsupportedParam    = errors.ErrUnsupportedParam
	ErrUnsupportedProvider = errors.ErrUnsupportedProvider
)

Sentinel errors for type checking with errors.Is().

View Source

var ErrToolNotFound = errors.New("tool not found")

Functions ¶

func NewToolsHandler ¶

func NewToolsHandler(tools ...Tool) func(context.Context, FunctionCall, ...ToolOption) (string, error)

Types ¶

type AuthenticationError ¶

type AuthenticationError = errors.AuthenticationError

Error types.

type BaseError ¶

type BaseError = errors.BaseError

Error types.

type Batch ¶

type Batch = providers.Batch

Batch types.

type BatchProvider ¶

type BatchProvider = providers.BatchProvider

Provider types.

type BatchRequestCounts ¶

type BatchRequestCounts = providers.BatchRequestCounts

Batch types.

type BatchRequestItem ¶

type BatchRequestItem = providers.BatchRequestItem

Batch types.

type BatchResult ¶

type BatchResult = providers.BatchResult

Batch types.

type BatchResultError ¶

type BatchResultError = providers.BatchResultError

Batch types.

type BatchResultItem ¶

type BatchResultItem = providers.BatchResultItem

Batch types.

type BatchStatus ¶

type BatchStatus = providers.BatchStatus

Usage and model types.

type Capabilities ¶

type Capabilities = providers.Capabilities

Provider types.

type CapabilityProvider ¶

type CapabilityProvider = providers.CapabilityProvider

Provider types.

type ChatCompletion ¶

type ChatCompletion = providers.ChatCompletion

Request/Response types.

type ChatCompletionChunk ¶

type ChatCompletionChunk = providers.ChatCompletionChunk

Request/Response types.

type Choice ¶

type Choice = providers.Choice

Request/Response types.

type ChunkChoice ¶

type ChunkChoice = providers.ChunkChoice

Request/Response types.

type ChunkDelta ¶

type ChunkDelta = providers.ChunkDelta

Request/Response types.

type CompletionParams ¶

type CompletionParams = providers.CompletionParams

Request/Response types.

type CompletionTokensDetails ¶

type CompletionTokensDetails = providers.CompletionTokensDetails

Usage and model types.

type Config ¶

type Config = config.Config

Config types.

type Content ¶

type Content = providers.Content

Message types.

type ContentFilterError ¶

type ContentFilterError = errors.ContentFilterError

Error types.

type ContentPart ¶

type ContentPart = providers.ContentPart

Message types.

type ContentPartAudio ¶

type ContentPartAudio = providers.ContentPartAudio

Message types.

type ContentPartFile ¶

type ContentPartFile = providers.ContentPartFile

Message types.

type ContentPartImage ¶

type ContentPartImage = providers.ContentPartImage

Message types.

type ContentPartText ¶

type ContentPartText = providers.ContentPartText

Message types.

type ContentPartType ¶

type ContentPartType = providers.ContentPartType

Message types.

type ContentParts ¶

type ContentParts = providers.ContentParts

Message types.

type ContentString ¶

type ContentString = providers.ContentString

Message types.

type ContextLengthError ¶

type ContextLengthError = errors.ContextLengthError

Error types.

type CreateBatchParams ¶

type CreateBatchParams = providers.CreateBatchParams

Batch types.

type EmbeddingData ¶

type EmbeddingData = providers.EmbeddingData

Usage and model types.

type EmbeddingParams ¶

type EmbeddingParams = providers.EmbeddingParams

Request/Response types.

type EmbeddingProvider ¶

type EmbeddingProvider = providers.EmbeddingProvider

Provider types.

type EmbeddingResponse ¶

type EmbeddingResponse = providers.EmbeddingResponse

Request/Response types.

type EmbeddingUsage ¶

type EmbeddingUsage = providers.EmbeddingUsage

Usage and model types.

type File ¶

type File = providers.File

Message types.

type Function ¶

type Function = providers.Function

Tool types.

type FunctionCall ¶

type FunctionCall = providers.FunctionCall

Tool types.

type ImageURL ¶

type ImageURL = providers.ImageURL

Message types.

type InputAudio ¶

type InputAudio = providers.InputAudio

Message types.

type InsufficientFundsError ¶

type InsufficientFundsError = errors.InsufficientFundsError

Error types.

type InvalidRequestError ¶

type InvalidRequestError = errors.InvalidRequestError

Error types.

type JSONSchema ¶

type JSONSchema = providers.JSONSchema

Response format types.

type ListBatchesOptions ¶

type ListBatchesOptions = providers.ListBatchesOptions

Batch types.

type Message ¶

type Message = providers.Message

Message types.

type MissingAPIKeyError ¶

type MissingAPIKeyError = errors.MissingAPIKeyError

Error types.

type Model ¶

type Model = providers.Model

Usage and model types.

type ModelLister ¶

type ModelLister = providers.ModelLister

Provider types.

type ModelNotFoundError ¶

type ModelNotFoundError = errors.ModelNotFoundError

Error types.

type ModelsResponse ¶

type ModelsResponse = providers.ModelsResponse

Request/Response types.

type ModerationParams ¶

type ModerationParams = providers.ModerationParams

Request/Response types.

type ModerationProvider ¶

type ModerationProvider = providers.ModerationProvider

Provider types.

type ModerationResponse ¶

type ModerationResponse = providers.ModerationResponse

Request/Response types.

type ModerationResult ¶

type ModerationResult = providers.ModerationResult

Request/Response types.

type Option ¶

type Option = config.Option

Config types.

type PromptTokensDetails ¶

type PromptTokensDetails = providers.PromptTokensDetails

Usage and model types.

type Provider ¶

type Provider = providers.Provider

Provider types.

type ProviderError ¶

type ProviderError = errors.ProviderError

Error types.

type RateLimitError ¶

type RateLimitError = errors.RateLimitError

Error types.

type Reasoning ¶

type Reasoning = providers.Reasoning

Message types.

type ReasoningEffort ¶

type ReasoningEffort = providers.ReasoningEffort

Usage and model types.

type RerankMeta ¶

type RerankMeta = providers.RerankMeta

Rerank types.

type RerankParams ¶

type RerankParams = providers.RerankParams

Rerank types.

type RerankProvider ¶

type RerankProvider = providers.RerankProvider

Provider types.

type RerankResponse ¶

type RerankResponse = providers.RerankResponse

Rerank types.

type RerankResult ¶

type RerankResult = providers.RerankResult

Rerank types.

type RerankUsage ¶

type RerankUsage = providers.RerankUsage

Rerank types.

type ResponseFormat ¶

type ResponseFormat = providers.ResponseFormat

Response format types.

type StreamOptions ¶

type StreamOptions = providers.StreamOptions

Response format types.

type Tool ¶

type Tool interface {
	Info() ToolInfo
	Function() Function
	Execute(context.Context, string, ...ToolOption) (string, error)
	ExecuteStream(context.Context, string, ...ToolOption) iter.Seq2[string, error]
}

Tool describes and executes one function that an LLM may call.

func NewTool ¶

func NewTool[T any](
	info ToolInfo,
	execf func(ctx context.Context, args T, opts ...ToolOption) (string, error),
	streamf func(ctx context.Context, args T, opts ...ToolOption) iter.Seq2[string, error],
) Tool

type ToolCall ¶

type ToolCall = providers.ToolCall

Tool types.

type ToolChoice ¶

type ToolChoice = providers.ToolChoice

Tool types.

type ToolChoiceFunction ¶

type ToolChoiceFunction = providers.ToolChoiceFunction

Tool types.

type ToolConfig ¶

type ToolConfig struct {
	Metadata map[string]any
}

ToolConfig contains metadata associated with one tool execution.

type ToolInfo ¶

type ToolInfo = providers.ToolInfo

Tool types.

type ToolOption ¶

type ToolOption func(*ToolConfig)

ToolOption configures one tool execution.

func WithToolMetadata ¶

func WithToolMetadata(metadata map[string]any) ToolOption

WithToolMetadata attaches protocol-specific metadata to a tool execution.

type UnsupportedOperationError ¶

type UnsupportedOperationError = errors.UnsupportedOperationError

Error types.

type UnsupportedParamError ¶

type UnsupportedParamError = errors.UnsupportedParamError

Error types.

type UnsupportedProviderError ¶

type UnsupportedProviderError = errors.UnsupportedProviderError

Error types.

type Usage ¶

type Usage = providers.Usage

Usage and model types.

Source Files ¶

View all Source files

Directories ¶

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

Path	Synopsis
config
errors
examples
basic command Example: Basic completion request	Example: Basic completion request
multi-provider command Example: Multi-provider usage	Example: Multi-provider usage
streaming command Example: Streaming responses	Example: Streaming responses
tools command Example: Tool/Function calling	Example: Tool/Function calling
internal
testutil Package testutil provides testing utilities and fixtures for llm.	Package testutil provides testing utilities and fixtures for llm.
providers Package providers defines the core provider interface and related types.	Package providers defines the core provider interface and related types.
anthropic Package anthropic provides an Anthropic provider implementation for llm.	Package anthropic provides an Anthropic provider implementation for llm.
deepseek Package deepseek provides a DeepSeek provider implementation for llm.	Package deepseek provides a DeepSeek provider implementation for llm.
gateway Package gateway provides a gateway provider implementation for any-llm.	Package gateway provides a gateway provider implementation for any-llm.
gemini Package gemini provides a Google Gemini provider implementation for llm.	Package gemini provides a Google Gemini provider implementation for llm.
groq Package groq provides a Groq provider implementation for llm.	Package groq provides a Groq provider implementation for llm.
openai Package openai provides an OpenAI provider implementation for llm.	Package openai provides an OpenAI provider implementation for llm.
platform Package platform provides a platform provider implementation for llm.	Package platform provides a platform provider implementation for llm.
zai Package zai provides a z.ai provider implementation for llm.	Package zai provides a z.ai provider implementation for llm.
sdk Package sdk provides metadata for the llm library.	Package sdk provides metadata for the llm library.

README ¶

llm

Quickstart

Installation

Requirements

Setting Up API Keys

any-llm-gateway

Quick Start

Managed Platform (Beta)

Usage

Streaming

Tools / Function Calling

Extended Thinking (Reasoning)

Embeddings

Listing Models

Moderation

Error Handling

Supported Providers

Why choose llm?

Development

Documentation

Contributing

License

Documentation ¶

Overview ¶

Index ¶

Constants ¶

Variables ¶

Functions ¶

func NewToolsHandler ¶

Types ¶

type AuthenticationError ¶

type BaseError ¶

type Batch ¶

type BatchProvider ¶

type BatchRequestCounts ¶

type BatchRequestItem ¶

type BatchResult ¶

type BatchResultError ¶

type BatchResultItem ¶

type BatchStatus ¶

type Capabilities ¶

type CapabilityProvider ¶

type ChatCompletion ¶

type ChatCompletionChunk ¶

type Choice ¶

type ChunkChoice ¶

type ChunkDelta ¶

type CompletionParams ¶

type CompletionTokensDetails ¶

type Config ¶

type Content ¶

type ContentFilterError ¶

type ContentPart ¶

type ContentPartAudio ¶

type ContentPartFile ¶

type ContentPartImage ¶

type ContentPartText ¶

type ContentPartType ¶

type ContentParts ¶

type ContentString ¶

type ContextLengthError ¶

type CreateBatchParams ¶

type EmbeddingData ¶

type EmbeddingParams ¶

type EmbeddingProvider ¶

type EmbeddingResponse ¶

type EmbeddingUsage ¶

type File ¶

type Function ¶

type FunctionCall ¶

type ImageURL ¶

type InputAudio ¶

type InsufficientFundsError ¶

type InvalidRequestError ¶

type JSONSchema ¶

type ListBatchesOptions ¶

type Message ¶

type MissingAPIKeyError ¶

type Model ¶

Why choose `llm`?