ollama

README

Ollama Go Client Library

This package defines a partial Ollama client with rich feature set around chat requests with Ollama. It was created to experiment with the new tool calling functionality in Llama.cpp and Ollama.

See example/orderbot for a full example of how to bind a Go function and provide it as a tool. A simpler example of tool use can be found in example/tick.

Using the Library

First, you must install the client as a dependency of your project:

go install github.com/swdunlop/ollama-client

Then, in many circumstances, you can just use the ollama.Chat function.

import "github.com/swdunlop/ollama-client"

ret, _ := ollama.Chat(
	context.TODO(),
	chat.Model(`llama3.1:latest`),
	chat.Message(`user`, `what is the airspeed of an unladen swallow?`),
)

This will connect to the Ollama instance running locally and run the request. See the examples for more involved examples using tools and other features.

Documentation

Index

• Variables
• func Apply[T any](options ...func(*T)) func(*T)
• func Chat(ctx context.Context, options ...chat.Option) (*chat.Response, error)
• func Embed(ctx context.Context, options ...embed.Option) (*embed.Response, error)
• func With(ctx context.Context, options ...Option) context.Context
• type Client
  • func New(options ...Option) *Client
  • func (ct *Client) Apply(options ...Option) *Client
  • func (ct *Client) Do(ctx context.Context, rsp any, method string, req any, api string) error
• type Error
  • func (err *Error) Error() string
• type Option
  • func Host(host string) Option
  • func RequestHook(hook func(*http.Request) error) Option
  • func ResponseHook(hook func(*http.Response) error) Option
  • func TraceZerolog(logger zerolog.Logger) Option

Variables

var Default = defaultClient

Default is the default client for contexts without their own specialized client.

Functions

func Apply

func Apply[T any](options ...func(*T)) func(*T)

Apply constructs an option that applies the provided options.

func Chat

func Chat(ctx context.Context, options ...chat.Option) (*chat.Response, error)

Chat does a chat request with the provided context. If a toolkit is provided for the request, it will be used to handle any tool calls.

func Embed

func Embed(ctx context.Context, options ...embed.Option) (*embed.Response, error)

Embed returns a vector that describes the input in a dimensions understood by the model. This can be used to identify similar inputs or to find relevant inputs.

func With

func With(ctx context.Context, options ...Option) context.Context

With creates a new Ollama client or expands the previous one in a context.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

func New

func New(options ...Option) *Client

New constructs a new Client with the provided options.

func (*Client) Apply

func (ct *Client) Apply(options ...Option) *Client

Apply returns a client improved with new options.

func (*Client) Do

func (ct *Client) Do(ctx context.Context, rsp any, method string, req any, api string) error

Do exchanges a Request for a Response or an error.

type Error

type Error struct {
	URL        string
	StatusCode int
	Status     string
	Header     http.Header
	Content    []byte
}

func (*Error) Error

func (err *Error) Error() string

type Option

type Option func(*Client)

func Host

func Host(host string) Option

Host specifies the base URL of the Ollama server. This may be either a URL without a trailing "/" or a TCP/IP address, in which case, HTTP will be used. The default host is `http://localhost:11434` but if OLLAMA_HOST is present in the environment, it will be used instead.

func RequestHook

func RequestHook(hook func(*http.Request) error) Option

RequestHook adds a request hook to the client; each request hook is applied to a request by the Do client method. This is useful for injecting authentication headers or logging. They are applied in First In First Out (FIFO) order.

func ResponseHook

func ResponseHook(hook func(*http.Response) error) Option

ResponseHook adds a response hook to the client; like request hooks, these hooks are applied to responses by the Do client method, but unlike RequestHooks, they are applied in Last In First Out (LIFO) order.

func TraceZerolog

func TraceZerolog(logger zerolog.Logger) Option

TraceZerolog adds a zerolog trace using the provided logger that traces requests and responses.