ollama

README

ollama

ollama is a Go client for a local or remote Ollama server. It implements the github.com/goloop/ai interface, so it looks and works like every other goloop AI provider, and exposes Ollama's native endpoints on top.

Features

• Chat: Generate for a single response, Stream for token-by-token output through iter.Seq2.
• Tool use (function calling) and image input.
• Native ChatCompletion and ChatStream over /api/chat.
• Embeddings, the installed-model list and per-model details (Show).
• Retries on 429 and 5xx with backoff; normalized, typed API errors.
• Depends only on github.com/goloop/ai and the standard library.
• Structured output: ai.Format maps onto the server's format field ("json" or a schema); read the reply with resp.JSON(&v).
• Hosted capabilities: ai.Request.Hosted is refused with ai.ErrNoHosted, because a local model server has no search to run.

Installation

go get github.com/goloop/ollama

Quick start

package main

import (
	"context"
	"fmt"

	"github.com/goloop/ai"
	"github.com/goloop/ollama"
)

func main() {
	c := ollama.New("") // local server on http://localhost:11434

	resp, err := c.Generate(context.Background(), &ai.Request{
		Model:    ollama.ModelLlama32,
		Messages: []ai.Message{ai.UserText("Say hello in one word.")},
	})
	if err != nil {
		panic(err)
	}
	fmt.Println(resp.Text())
}

Remote server

c := ollama.New(apiKey, ollama.WithBaseURL("https://ollama.example.com"))

The API key is optional; it is sent as a bearer token when set, for authenticated proxies in front of a server.

Streaming

Ollama streams newline-delimited JSON rather than Server-Sent Events; Stream hides that difference and yields the same ai.Chunk values as every other provider.

for chunk, err := range c.Stream(ctx, req) {
	if err != nil {
		break
	}
	fmt.Print(chunk.Text)
}

Embeddings and models

vecs, err := c.Embed(ctx, ollama.ModelLlama32, "hello", "world")
models, err := c.Models(ctx) // what is installed locally
info, err := c.Show(ctx, ollama.ModelLlama32) // template, params, capabilities

Documentation

Full reference: DOC.md (Ukrainian: DOC.UK.md).

Contributing

See CONTRIBUTING.md.

License

MIT - see LICENSE.

Documentation

Overview

Package ollama is a client for a local or remote Ollama server, built on the goloop/ai interface.

The Client implements ai.Client, so Generate and Stream work the same as with any other goloop AI provider. On top of that it exposes Ollama's native endpoints: /api/chat (with tool use and image input), embeddings, the installed-model list and per-model details (Show). Ollama streams newline-delimited JSON rather than Server-Sent Events; Stream hides that difference.

c := ollama.New("") // local server, no API key
resp, err := c.Generate(ctx, &ai.Request{
    Model:    ollama.ModelLlama32,
    Messages: []ai.Message{ai.UserText("Say hello in one word.")},
})

Structured output

ai.Request.Format maps onto the server's format field - the bare word "json", or a schema directly - so a request for JSON is enforced rather than merely asked for, and ai.Response.JSON decodes the reply.

Hosted capabilities

Models here run on the machine that serves them, and that server has no search to run. ai.Hosted is refused with ai.ErrNoHosted.

The refusal is the documented behavior, not a gap waiting to be filled in silence: an answer produced without the search that was asked for looks exactly like one produced with it. A caller who would rather have the answer anyway asks again without ai.Request.Hosted.

Asking what this driver can do

Capabilities describes this driver for the decision taken before a call: whether to offer a feature at all, and whether it needs one request or two.

if ai.SupportsHosted(c, ai.Hosted{Kind: ai.HostedWebSearch}) { ... }

It is a hint and not a permission - support also depends on the model, the account and the region - so ai.ErrNoHosted and ai.ErrNoFormat remain the source of truth and a caller still handles them. What changes is that a refusal the provider only reports as a 400 now arrives as those same sentinels, wrapped around the original ai.APIError, so one errors.Is covers a limitation this driver knew in advance and one it learned over the wire.

It depends only on goloop/ai and the standard library.

Index

• Constants
• type ChatRequest
• type ChatResponse
• type Client
  • func New(apiKey string, opts ...Option) *Client
  • func (c *Client) Capabilities() ai.Capabilities
  • func (c *Client) ChatCompletion(ctx context.Context, req *ChatRequest) (*ChatResponse, error)
  • func (c *Client) ChatStream(ctx context.Context, req *ChatRequest) iter.Seq2[ChatResponse, error]
  • func (c *Client) Embed(ctx context.Context, model string, input ...string) ([][]float64, error)
  • func (c *Client) Generate(ctx context.Context, req *ai.Request) (*ai.Response, error)
  • func (c *Client) Models(ctx context.Context) ([]Model, error)
  • func (c *Client) Show(ctx context.Context, model string) (*ModelInfo, error)
  • func (c *Client) Stream(ctx context.Context, req *ai.Request) iter.Seq2[ai.Chunk, error]
• type Message
• type Model
• type ModelDetails
• type ModelInfo
• type Option
  • func WithBaseURL(u string) Option
  • func WithHTTPClient(c *http.Client) Option
  • func WithHeader(key, value string) Option
  • func WithMaxRetries(n int) Option
  • func WithTimeout(d time.Duration) Option
• type Options
• type Tool
• type ToolCall
• type ToolCallFunction
• type ToolFunction

Examples

• Client.Generate
• Client.Show
• New
• Tool

Constants

const (
	ModelLlama32 = "llama3.2"
	ModelLlama31 = "llama3.1"
	ModelMistral = "mistral"
	ModelGemma3  = "gemma3"
	ModelQwen25  = "qwen2.5"
)

Convenience model identifiers. Any model tag installed on the server is accepted; use Models to list what is available locally.

const DefaultBaseURL = "http://localhost:11434"

DefaultBaseURL is the base URL of a local Ollama server.

Types

type ChatRequest

type ChatRequest struct {
	Model     string          `json:"model"`
	Messages  []Message       `json:"messages"`
	Tools     []Tool          `json:"tools,omitempty"`
	Format    json.RawMessage `json:"format,omitempty"`
	Options   *Options        `json:"options,omitempty"`
	KeepAlive string          `json:"keep_alive,omitempty"`
	Stream    bool            `json:"stream"`
}

ChatRequest is the native /api/chat request body.

type ChatResponse

type ChatResponse struct {
	Model           string  `json:"model"`
	CreatedAt       string  `json:"created_at"`
	Message         Message `json:"message"`
	Done            bool    `json:"done"`
	DoneReason      string  `json:"done_reason,omitempty"`
	PromptEvalCount int     `json:"prompt_eval_count"`
	EvalCount       int     `json:"eval_count"`
}

ChatResponse is the native /api/chat response (and each streamed chunk).

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is an Ollama API client. It implements ai.Client and adds the provider's native endpoints. It talks to a local (or remote) Ollama server using its native API.

func New

func New(apiKey string, opts ...Option) *Client

New returns a Client. The API key may be empty for a local server; when set, it is sent as a bearer token for authenticated proxies. Shared options (WithBaseURL, WithHTTPClient, WithTimeout, WithMaxRetries, WithHeader) configure it.

Example

package main

import (
	"fmt"

	"github.com/goloop/ollama"
)

func main() {
	c := ollama.New("") // local server, no API key
	_ = c               // use c.Generate, c.Stream, c.ChatCompletion, ...
	fmt.Println(ollama.ModelLlama32)
}

Output:
llama3.2

func (*Client) Capabilities

func (c *Client) Capabilities() ai.Capabilities

Capabilities describes what this driver can be asked for. It is a hint for the decision taken before a call - whether to offer a feature, and whether it needs one request or two - and never a substitute for handling ai.ErrNoHosted, ai.ErrNoFormat or ai.ErrFormatWithHosted, because support also depends on the model, the account and the region.

func (*Client) ChatCompletion

func (c *Client) ChatCompletion(ctx context.Context, req *ChatRequest) (*ChatResponse, error)

ChatCompletion sends a native /api/chat request and returns the whole response. Use it for provider-specific options; use Generate for the shared, provider-agnostic path.

func (*Client) ChatStream

func (c *Client) ChatStream(
	ctx context.Context,
	req *ChatRequest,
) iter.Seq2[ChatResponse, error]

ChatStream sends a native streaming /api/chat request and yields each chunk as it arrives.

func (*Client) Embed

func (c *Client) Embed(
	ctx context.Context,
	model string,
	input ...string,
) ([][]float64, error)

Embed embeds one or more inputs with a model and returns their vectors in order, using the native /api/embed endpoint.

func (*Client) Generate

func (c *Client) Generate(ctx context.Context, req *ai.Request) (*ai.Response, error)

Generate implements ai.Client over /api/chat.

Example

ExampleClient_Generate builds a request. Sending it needs a running server, so this example only shows the shape.

package main

import (
	"fmt"

	"github.com/goloop/ai"
	"github.com/goloop/ollama"
)

func main() {
	req := &ai.Request{
		Model: ollama.ModelLlama32,
		Messages: []ai.Message{
			ai.UserText("Name the capital of France."),
		},
	}
	fmt.Println(req.Model, len(req.Messages))
}

Output:
llama3.2 1

func (*Client) Models

func (c *Client) Models(ctx context.Context) ([]Model, error)

Models lists the models installed on the server (the /api/tags endpoint).

func (*Client) Show

func (c *Client) Show(ctx context.Context, model string) (*ModelInfo, error)

Show returns detailed information about an installed model: its template, parameters, capabilities and architecture fields (the /api/show endpoint).

Example

ExampleClient_Show notes the per-model details endpoint. Show returns the model's template, parameters, capabilities and architecture fields (such as its context length). Calling it needs a running server, so this example only names the target model.

package main

import (
	"fmt"

	"github.com/goloop/ollama"
)

func main() {
	c := ollama.New("")
	_ = c // info, _ := c.Show(ctx, ollama.ModelLlama32)
	fmt.Println(ollama.ModelLlama32)
}

Output:
llama3.2

func (*Client) Stream

func (c *Client) Stream(
	ctx context.Context,
	req *ai.Request,
) iter.Seq2[ai.Chunk, error]

Stream implements ai.Client over /api/chat. Ollama streams newline- delimited JSON objects rather than Server-Sent Events.

type Message

type Message struct {
	Role      string     `json:"role"`
	Content   string     `json:"content"`
	Images    []string   `json:"images,omitempty"`
	ToolCalls []ToolCall `json:"tool_calls,omitempty"`
}

Message is one message in a chat request or response. Images are base64- encoded; ToolCalls are set on assistant messages that call a tool.

type Model

type Model struct {
	Name       string       `json:"name"`
	Model      string       `json:"model"`
	Size       int64        `json:"size"`
	ModifiedAt string       `json:"modified_at"`
	Details    ModelDetails `json:"details"`
}

Model describes a model installed on the server.

type ModelDetails

type ModelDetails struct {
	Family            string `json:"family"`
	ParameterSize     string `json:"parameter_size"`
	QuantizationLevel string `json:"quantization_level"`
}

ModelDetails is the family and quantization summary shared by Model and ModelInfo.

type ModelInfo

type ModelInfo struct {
	Modelfile    string         `json:"modelfile"`
	Parameters   string         `json:"parameters"`
	Template     string         `json:"template"`
	Details      ModelDetails   `json:"details"`
	ModelInfo    map[string]any `json:"model_info"`
	Capabilities []string       `json:"capabilities"`
}

ModelInfo describes an installed model in detail (the /api/show endpoint). ModelInfo holds architecture-specific fields with dynamic keys, such as "llama.context_length"; Capabilities lists features like "completion" or "tools".

type Option

type Option func(*settings)

Option configures a Client in New.

func WithBaseURL

func WithBaseURL(u string) Option

WithBaseURL overrides the server URL (a remote Ollama host, a proxy, a mock server). Defaults to a local server.

func WithHTTPClient

func WithHTTPClient(c *http.Client) Option

WithHTTPClient sets the HTTP client used for requests.

func WithHeader

func WithHeader(key, value string) Option

WithHeader adds a header sent with every request.

func WithMaxRetries

func WithMaxRetries(n int) Option

WithMaxRetries sets how many times a request is retried on 429 and 5xx.

func WithTimeout

func WithTimeout(d time.Duration) Option

WithTimeout sets the per-request timeout when no custom HTTP client is set.

type Options

type Options struct {
	Temperature *float64 `json:"temperature,omitempty"`
	TopP        *float64 `json:"top_p,omitempty"`
	NumPredict  int      `json:"num_predict,omitempty"`
	Stop        []string `json:"stop,omitempty"`
}

Options tunes generation. Temperature and TopP are pointers so an explicit zero is distinct from unset.

type Tool

type Tool struct {
	Type     string       `json:"type"`
	Function ToolFunction `json:"function"`
}

Tool declares a callable function.

Example

ExampleTool shows a tool definition passed with a request.

package main

import (
	"encoding/json"
	"fmt"

	"github.com/goloop/ai"
)

func main() {
	tool := ai.Tool{
		Name:        "get_weather",
		Description: "Get the current weather for a city.",
		Schema:      json.RawMessage(`{"type":"object","properties":{"city":{"type":"string"}}}`),
	}
	fmt.Println(tool.Name)
}

Output:
get_weather

type ToolCall

type ToolCall struct {
	Function ToolCallFunction `json:"function"`
}

ToolCall is a tool call the model produced.

type ToolCallFunction

type ToolCallFunction struct {
	Name      string          `json:"name"`
	Arguments json.RawMessage `json:"arguments"`
}

ToolCallFunction is the name and JSON arguments of a tool call. Ollama sends arguments as a JSON object.

type ToolFunction

type ToolFunction struct {
	Name        string          `json:"name"`
	Description string          `json:"description,omitempty"`
	Parameters  json.RawMessage `json:"parameters,omitempty"`
}

ToolFunction is a function's name, description and JSON Schema parameters.