golangllm

README

golangllm

A Go SDK for calling Claude via AWS Bedrock or the direct Anthropic API — session Chat (Ask / AskStream with a tool loop), thin Agent presets, bundled model catalog with estimated cost, SQL persistence, caller-supplied RAG, and Bedrock media support for embeddings, image generation (Paint), OCR, and moderation (Moderate); audio is deferred. golangllm is a Go library inspired by and derived from ruby_llm (MIT, Copyright Carmine Paolino). It covers Anthropic and AWS Bedrock only, not ruby_llm's full multi-provider surface.

Package tree

golangllm/                 # root aliases: Client, Media, NewChat, NewMedia, OCR, Moderate, …
  provider/                # Client, Request, Chat, ChatStream
  tool/                    # Tool, Registry, Run (tool loop)
  chat/                    # session: Ask, AskStream, history
  agent/                   # preset → *chat.Chat
  models/                  # catalog.json + named ids + Lookup + Cost
  persist/                 # Store on database/sql + EnsureSchema
  rag/                     # Retriever + Format
  media/                   # Bedrock Embed, Paint, OCR, Moderate implementations
  internal/noaws/          # test-only: block live AWS HTTP

Root golangllm re-exports NewBedrockClient, NewAnthropicClient, NewChat, Client, Request, Agent, Tool / ToolRegistry, RunToolLoop, and the media client, types, and helpers so callers need not import deep paths. Subpackages must not import the root package.

Install

go get github.com/ikizmet/golang_llm

Authentication

NewBedrockClient authenticates via the standard AWS credential chain (environment variables, shared config/profile, or an IAM role) — there is no way to pass a hardcoded key, by design. When the resolved config carries both SigV4 credentials and a bearer token (an AWS SSO session resolves its access token as one), SigV4 wins: Bedrock rejects an SSO access token as a malformed API key. Set AWS_BEARER_TOKEN_BEDROCK to authenticate with a real Bedrock bearer token instead. Attach a logger with client.SetLogger(logger) and client.SetLogParams(true) to log Chat/ChatStream request fields (model, token limits, counts) without message bodies. Set GOLANG_LLM_LOG to error, info, or debug to enable without code (default off). info logs every Chat/ChatStream call with elapsed time, stop reason, token counts, and tool names from that round. debug also logs request shape before the call. client.SetLogLevel / client.SetLogger override the env. option.WithDebugLog still dumps rewritten Bedrock HTTP if passed to NewBedrockClient. Note also that a profile without a region yields an unroutable endpoint, so set one in the profile or via AWS_REGION. For a direct Anthropic API connection instead of Bedrock, use NewAnthropicClient(apiKey), where apiKey comes from your own environment variable or secrets manager — never hardcode it.

Usage

Session chat (NewChat / Ask / AskStream)

NewChat builds a multi-turn *chat.Chat that owns history and runs the tool loop. *provider.Client satisfies chat.Streamer.

client := golangllm.NewBedrockClient(ctx)
c := golangllm.NewChat(client,
    chat.WithModel(models.USAnthropicClaudeSonnet4_5_20250929V1_0),
    chat.WithMaxTokens(1024),
    chat.WithInstructions("You are a helpful assistant."),
    chat.WithTools(tools), // optional
)

msg, err := c.Ask(ctx, "What's the weather in Berlin?")
if err != nil {
    return err
}
fmt.Println(msg.Content[0].AsText().Text)
fmt.Println(c.LastCost) // estimated USD for this Ask (0 if model unknown)

err = nil
msg, err = c.AskStream(ctx, "Tell me more.", func(chunk chat.Chunk) {
    // chunk.Type is "text" or "thinking"
    fmt.Print(chunk.Text)
})

Ask and AskStream both run the complete tool loop (including when no tools are registered — one provider round). After success they update in-memory history; with a store attached they also persist (see below).

Agent

In ruby_llm, an agent is a class that declares model, instructions, and tools once, then .chat / .ask. golangllm does the same job with a struct preset — there is no class DSL, no prompt-file lookup, and no Rails chat_model. There is also no router in the SDK: picking among several agents is application code. The sibling golang_agent CLI is a worked example: a tiny classifier names writer and/or mathematician, then each Agent runs with its own ToolRegistry (story outline / character / title, or add / multiply / divide) and a 1024-token ThinkingBudget. See that repo's README.md.

Use NewChat + chat.With* for a one-off conversation. Use Agent when the same instructions and tools should be reused:

a := golangllm.Agent{
    Model:          models.USAnthropicClaudeSonnet4_5_20250929V1_0,
    MaxTokens:      4096,
    ThinkingBudget: 1024, // optional; 0 omits extended thinking
    Instructions:   "You are a concise support assistant.",
    Tools:          tools, // optional; same Registry you would pass to chat.WithTools
}
c := a.Chat(client) // *chat.Chat with those options applied
msg, err := c.Ask(ctx, "How do I reset my API key?")

Agent.Chat is equivalent to NewChat with WithModel, WithMaxTokens, WithInstructions, WithTools, WithThinkingBudget, and WithOutputSchema for whichever fields are non-zero. Zero fields are omitted (same as not calling that With*).

Tool / function calling

Tools are Go functions the model may call. Build a tool.Registry (aliased as golangllm.ToolRegistry): name → {Param, Handler}. Param is the Anthropic tool definition; Handler receives json.RawMessage input and returns a string result (or an error, which is sent back as is_error: true without aborting sibling tools).

Attach them on a session with chat.WithTools or on an Agent. Ask / AskStream run the complete loop: model may return tool_use blocks, handlers run concurrently, results are appended, next round streams until end_turn.

tools := golangllm.ToolRegistry{
    "get_weather": golangllm.Tool{
        Param: anthropic.ToolUnionParam{OfTool: &anthropic.ToolParam{
            Name:        "get_weather",
            Description: param.NewOpt("Get current weather for a city"),
            InputSchema: anthropic.ToolInputSchemaParam{
                Properties: map[string]any{
                    "city": map[string]any{"type": "string"},
                },
                Required: []string{"city"},
            },
        }},
        Handler: func(ctx context.Context, input json.RawMessage) (string, error) {
            return getWeather(input)
        },
    },
}

c := golangllm.NewChat(client,
    chat.WithModel(models.USAnthropicClaudeSonnet4_5_20250929V1_0),
    chat.WithMaxTokens(1024),
    chat.WithTools(tools),
)
msg, err := c.Ask(ctx, "What's the weather in Berlin?")

RunToolLoop is the same loop on a one-shot *Request (pointer so appended history is visible). Prefer Chat.Ask / AskStream for session use.

req := &golangllm.Request{
    Model:     models.USAnthropicClaudeSonnet4_5_20250929V1_0,
    MaxTokens: 1024,
    Messages:  []anthropic.MessageParam{anthropic.NewUserMessage(anthropic.NewTextBlock("What's the weather in Berlin?"))},
}
resp, err := golangllm.RunToolLoop(ctx, client, req, tools)
// req.Messages now includes assistant + tool turns

Errors from Anthropic surface unwrapped — use var apiErr *anthropic.Error; errors.As(err, &apiErr).

Provider one-shot (Chat / ChatStream)

For a single request without session history:

resp, err := client.Chat(ctx, golangllm.Request{
    Model:     models.USAnthropicClaudeSonnet4_5_20250929V1_0,
    MaxTokens: 1024,
    System:    "You are a helpful assistant.",
    Messages: []anthropic.MessageParam{
        anthropic.NewUserMessage(anthropic.NewTextBlock("Hello!")),
    },
})

provider.Client.ChatStream still exists for raw SSE without a tool loop — drive the stream yourself when you do not want AskStream's complete-loop:

stream := client.ChatStream(ctx, golangllm.Request{
    Model:     models.USAnthropicClaudeSonnet4_5_20250929V1_0,
    MaxTokens: 1024,
    Messages:  []anthropic.MessageParam{anthropic.NewUserMessage(anthropic.NewTextBlock("Tell me a short story."))},
})
defer stream.Close()
for stream.Next() {
    event := stream.Current()
    if event.Type == "content_block_delta" {
        fmt.Print(event.Delta.Text)
    }
}
if err := stream.Err(); err != nil {
    return err
}

The caller owns closing the stream on every path.

Extended thinking

Set Request.ThinkingBudget, chat.WithThinkingBudget, or Agent.ThinkingBudget to a positive token budget; 0 (the zero value) omits it. The budget must be less than MaxTokens — thinking tokens count against that cap. AskStream yields thinking chunks as well as text.

Structured output

Set Request.OutputSchema / chat.WithOutputSchema to a JSON Schema object. Claude returns constrained JSON as a normal text block — decode it yourself. catalog.json records structured_output per id; models.SupportsStructuredOutput is false for unknown ids and for Bedrock profiles that reject output_config.format. Chat / Ask return models.ErrStructuredOutputUnsupported instead of sending that field.

Model catalog and cost

models embeds catalog.json with current Anthropic API ids (including aliases) and matching Bedrock us. / global. inference-profile ids. Named constants (generated into ids.go) and models.IDs() cover every entry — pass those to Agent.Model / chat.WithModel instead of raw strings. Rows are refreshed from models.dev (https://models.dev/api.json, Anthropic + Amazon Bedrock Claude entries — the same dump ruby_llm uses). Run go run ./models/internal/syncmodels then go generate ./models. Prices are estimates and may lag vendor pricing. Base input/output USD per million tokens only — prompt-cache, batch, and regional Bedrock premiums are not applied. structured_output follows models.dev; unknown ids and Bedrock profiles that omit the flag are treated as unsupported on this library's Messages API path.

cost, err := models.Cost(models.USAnthropicClaudeSonnet4_5_20250929V1_0, inputTokens, outputTokens)
info, ok := models.Lookup(id)

Chat.LastCost is set after each successful Ask / AskStream (sum of all provider rounds). Unknown models leave LastCost at 0 without error.

Persist

persist.Store uses database/sql with dialect "sqlite" or "postgres". Schema is ruby_llm-shaped defaults only — chats, messages, tool_calls (no session_id, no app model FKs):

store := persist.NewStore(db, "sqlite")
if err := store.EnsureSchema(ctx); err != nil {
    return err
}
chatID, err := store.CreateChat(ctx)
c.WithStore(store, chatID)

// resume:
msgs, err := store.LoadMessages(ctx, chatID)
c.SetMessages(msgs)

After each successful Ask / AskStream, history is replace-all saved for that chatID.

RAG

Supply your own retrieval; the library only formats and injects chunks:

c.WithRetriever(func(ctx context.Context, query string) ([]rag.Chunk, error) {
    return []rag.Chunk{{ID: "1", Text: "…", Source: "docs"}}, nil
})

Non-empty results are prepended as rag.Format(chunks) + "\n\n" + prompt on the user turn. Empty results leave the prompt unchanged. The library does not run a vector database: retrieval remains caller-owned. Use Media.Embed (or another embedder) inside your own Retriever.

Embeddings (Embed)

Titan Text Embeddings V2 on Bedrock InvokeModel. Same AWS credential chain and SigV4-over-SSO rule as NewBedrockClient. Default model amazon.titan-embed-text-v2:0. Dimensions 256, 512, or 1024 (zero means 1024). Normalize nil means true.

mc, err := golangllm.NewMedia(ctx)
if err != nil {
    return err
}
emb, err := mc.Embed(ctx, "Ruby is elegant", golangllm.EmbedOptions{})
// emb.Vector []float32, emb.InputTokens, emb.Model

This is Titan JSON only — a Cohere or OpenAI embedding id will not work.

Image generation (Paint)

Media.Paint invokes Stability AI Stable Image Core on Bedrock Runtime. It is available in us-west-2; configure that region in your AWS profile or environment before creating the media client.

mc, err := golangllm.NewMedia(ctx)
if err != nil {
    return err
}
image, err := mc.Paint(ctx, "a watercolor fox in a forest", golangllm.PaintOptions{})
if err != nil {
    return err
}
// image.Data contains PNG bytes by default.

OCR (OCR)

Pass a provider client (or another MediaCompleter) and image bytes to extract Markdown text with Claude.

client := golangllm.NewBedrockClient(ctx)
text, err := golangllm.OCR(ctx, client, golangllm.OCRInput{
    Data:      imageBytes,
    MediaType: "image/png",
}, golangllm.OCROptions{})
if err != nil {
    return err
}

Moderation (Moderate)

Moderate is Claude's probabilistic classification, not a security boundary. For regulatory or strict policy needs, use Bedrock Guardrails or another dedicated service.

client := golangllm.NewBedrockClient(ctx)
result, err := golangllm.Moderate(ctx, client, "text to classify", golangllm.ModerationOptions{})
if err != nil {
    return err
}
fmt.Println(result.Flagged, result.Categories, result.Explanation)

Tests and AWS

go test ./... must not call live AWS or Anthropic. Provider and chat tests use httptest. media tests point Bedrock Runtime at a local server. internal/noaws rejects *.amazonaws.com and instance-metadata HTTP if a test omits that fake. Production media.New / NewBedrockClient still use the real AWS endpoints.

Development

See CONTRIBUTING.md for how to send changes. CLAUDE.md / AGENTS.md describe the required workflow (TDD, linting, vulnerability scanning). .claude/skills/golang-best-practices/SKILL.md has Go conventions. Review agents: /code-reviewer, /security-reviewer, /golang-reviewer (.cursor/agents/ and .claude/agents/).

gofmt -s -l .
go build ./...
go test ./...
golangci-lint run
govulncheck ./...

License

MIT. Copyright iKizmet and Alexander Mamrenko. Includes copyright notice from ruby_llm (Carmine Paolino). See LICENSE.

Documentation

Overview

Package golangllm is a Go SDK for Claude on AWS Bedrock and the Anthropic Messages API: session chat with a tool loop, agent presets, a model catalog with estimated cost, SQL persistence, caller-supplied RAG, and Bedrock embeddings, image generation, OCR, and moderation.

The module path is github.com/ikizmet/golang_llm. This root package only re-exports the subpackages so callers need not import deep paths: provider (Client, Request, Chat, ChatStream), tool (Tool, Registry, RunToolLoop), chat (NewChat, Ask, AskStream), agent (Agent), and media (Media, Embed, Paint, OCR, Moderate). Subpackages never import this one.

Start with NewBedrockClient or NewAnthropicClient for a Client, then NewChat for a multi-turn session. See the README for worked examples and the authentication rules for AWS credentials.

It is inspired by and derived from ruby_llm (https://github.com/crmne/ruby_llm) and covers Anthropic and AWS Bedrock only.

Index

• Constants
• Variables
• func NewChat(s chat.Streamer, opts ...chat.Option) *chat.Chat
• func OCR(ctx context.Context, client MediaCompleter, input OCRInput, opts OCROptions) (string, error)
• func RunToolLoop(ctx context.Context, client *Client, req *Request, tools ToolRegistry, ...) (*anthropic.Message, error)
• type Agent
• type Client
  • func NewAnthropicClient(apiKey string, opts ...option.RequestOption) (*Client, error)
• type EmbedOptions
• type Embedding
• type Image
• type LeveledLogger
• type LogLevel
• type Logger
• type Media
• type MediaCompleter
• type Moderation
  • func Moderate(ctx context.Context, client MediaCompleter, text string, ...) (Moderation, error)
• type ModerationOptions
• type OCRInput
• type OCROptions
• type PaintOptions
• type Request
• type Tool
• type ToolHandler
• type ToolRegistry

Constants

const (
	LogOff   = provider.LogOff
	LogError = provider.LogError
	LogInfo  = provider.LogInfo
	LogDebug = provider.LogDebug
)

LogOff, LogError, LogInfo, and LogDebug match provider.LogLevel.

const DefaultMaxRounds = chat.DefaultMaxRounds

DefaultMaxRounds is chat.DefaultMaxRounds (100).

Variables

var NewBedrockClient = provider.NewBedrockClient

NewBedrockClient is provider.NewBedrockClient.

var NewMedia = media.New

NewMedia is media.New.

var NewMediaFromConfig = media.NewFromConfig

NewMediaFromConfig is media.NewFromConfig.

var NewTool = tool.New

NewTool is tool.New.

var ParseLogLevel = provider.ParseLogLevel

ParseLogLevel is provider.ParseLogLevel.

var WithMaxRounds = chat.WithMaxRounds

WithMaxRounds is chat.WithMaxRounds. 0 means no cap.

Functions

func NewChat

func NewChat(s chat.Streamer, opts ...chat.Option) *chat.Chat

NewChat is chat.New.

func OCR

func OCR(ctx context.Context, client MediaCompleter, input OCRInput, opts OCROptions) (string, error)

OCR extracts text from an image as Markdown.

func RunToolLoop

func RunToolLoop(ctx context.Context, client *Client, req *Request, tools ToolRegistry, opts ...tool.RunOption) (*anthropic.Message, error)

RunToolLoop is tool.Run.

Types

type Agent

type Agent = agent.Agent

Agent is an alias for agent.Agent.

type Client

type Client = provider.Client

Client is an alias for provider.Client.

func NewAnthropicClient

func NewAnthropicClient(apiKey string, opts ...option.RequestOption) (*Client, error)

NewAnthropicClient is provider.NewAnthropicClient.

type EmbedOptions

type EmbedOptions = media.EmbedOptions

EmbedOptions is an alias for media.EmbedOptions.

type Embedding

type Embedding = media.Embedding

Embedding is an alias for media.Embedding.

type Image

type Image = media.Image

Image is an alias for media.Image.

type LeveledLogger

type LeveledLogger = provider.LeveledLogger

LeveledLogger is an alias for provider.LeveledLogger.

type LogLevel

type LogLevel = provider.LogLevel

LogLevel is an alias for provider.LogLevel.

type Logger

type Logger = provider.Logger

Logger is an alias for provider.Logger.

type Media

type Media = media.Client

Media is an alias for media.Client.

type MediaCompleter

type MediaCompleter = media.Completer

MediaCompleter is an alias for media.Completer.

type Moderation

type Moderation = media.Moderation

Moderation is an alias for media.Moderation.

func Moderate

func Moderate(ctx context.Context, client MediaCompleter, text string, opts ModerationOptions) (Moderation, error)

Moderate classifies text against configured categories.

type ModerationOptions

type ModerationOptions = media.ModerationOptions

ModerationOptions is an alias for media.ModerationOptions.

type OCRInput

type OCRInput = media.OCRInput

OCRInput is an alias for media.OCRInput.

type OCROptions

type OCROptions = media.OCROptions

OCROptions is an alias for media.OCROptions.

type PaintOptions

type PaintOptions = media.PaintOptions

PaintOptions is an alias for media.PaintOptions.

type Request

type Request = provider.Request

Request is an alias for provider.Request.

type Tool

type Tool = tool.Tool

Tool is an alias for tool.Tool.

type ToolHandler

type ToolHandler = tool.ToolHandler

ToolHandler is an alias for tool.ToolHandler.

type ToolRegistry

type ToolRegistry = tool.Registry

ToolRegistry is an alias for tool.Registry.