gollama

package module
v1.0.30 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 23, 2025 License: MIT Imports: 14 Imported by: 0

README

gollama

Easy Ollama package for Golang

Example use

go get -u github.com/jonathanhecl/gollama

package main

import (
	"context"
	"fmt"

	"github.com/jonathanhecl/gollama"
)

func main() {
	ctx := context.Background()
	g := gollama.New("llama3.2") // Create a new Gollama with the default model
	g.Verbose = true // Enable verbose mode
	if err := g.PullIfMissing(ctx); err != nil { // Pull the model if it is not available
		fmt.Println("Error:", err)
		return
	}

	prompt := "what is the capital of Argentina?" // The prompt to send to the model

	type Capital struct {
		Capital string `required:"true"`
	}

	option := gollama.StructToStructuredFormat(Capital{}) // Convert the struct to a structured format

	fmt.Printf("Option: %+v\n", option)

	output, err := g.Chat(ctx, prompt, option) // Generate a response
	if err != nil {
		fmt.Println("Error:", err)
		return
	}
	fmt.Printf("\n%s\n", output.Content) // Print the response
}
Features
  • Support Vision models
  • Support Tools models
  • Support Structured format
  • Downloads model if missing
  • Chat with model
  • Generates embeddings with model
  • Get model details
  • Get list of available models
Functions
  • New(model string) *Gollama - Create a new Gollama
  • NewWithConfig(config Gollama) *Gollama - Create a new Gollama with a pre-populated config
  • Chat(prompt string, ...ChatOption) (*gollama.ChatOutput, error) - Generate a response
  • Embedding(prompt string) ([]float64, error) - Generate embeddings
  • ListModels() ([]ModelInfo, error) - List models available on ollama
  • HasModel(model string) (bool, error) - Check if model is available
  • ModelSize(model string) (int, error) - Get model size from ollama
  • PullModel(model string) error - Pull model
  • PullIfMissing(model ...string) error - Pull model if missing
  • GetModels() ([]string, error) - Get list of available models
  • GetDetails(model ...string) ([]ModelDetails, error) - Get model details from ollama
  • Version() (string, error) - Get ollama version
  • StructToStructuredFormat(interface{}) StructuredFormat - Converts a Go struct to a Gollama structured format
  • CosenoSimilarity(vector1, vector2 []float64) float64 - Calculates the cosine similarity between two vectors
  • output.DecodeContent(output interface{}) error - Decodes the content of a Gollama response

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CosenoSimilarity added in v1.0.13 

func CosenoSimilarity(vector1, vector2 []float64) float64

Types

type ChatOption added in v1.0.9 

type ChatOption interface{}

type ChatOuput added in v1.0.9 

type ChatOuput struct {
	Role           string     `json:"role"`
	Content        string     `json:"content"`
	ToolCalls      []ToolCall `json:"tool_calls"`
	PromptTokens   int        `json:"prompt_tokens"`
	ResponseTokens int        `json:"response_tokens"`
}

func (ChatOuput) DecodeContent added in v1.0.18 

func (o ChatOuput) DecodeContent(v interface{}) error

type FormatProperty added in v1.0.10 

type FormatProperty struct {
	Type        string       `json:"type"`
	Description string       `json:"description,omitempty"`
	Enum        []string     `json:"enum,omitempty"`
	Items       ItemProperty `json:"items,omitempty"`
}

type Gollama 

type Gollama struct {
	ServerAddr                string
	ModelName                 string
	SeedOrNegative            int
	TemperatureIfNegativeSeed float64
	TopK                      int
	TopP                      float64
	PullTimeout               time.Duration
	HTTPTimeout               time.Duration
	TrimSpace                 bool
	Verbose                   bool
	ContextLength             int64
	SystemPrompt              string
}

func New 

func New(model string) *Gollama

New is a constructor for Gollama that takes a model name as an argument. Any unset fields will be populated with default values.

This is useful for cases where you want to create a Gollama with a specific model name, but don't want to specify every single field.

If the provided model name is empty, the default model name will be used.

The following environment variables can be set to override the default values of the fields:

OLLAMA_HOST: the address of the Ollama server OLLAMA_MODEL: the model name OLLAMA_VERBOSE: whether to print debug information

The returned Gollama can be used to call other methods on the Ollama API.

func NewWithConfig 

func NewWithConfig(config Gollama) *Gollama

NewWithConfig is a constructor for Gollama that takes a pre-populated Gollama as an argument. Any unset fields will be populated with default values.

This is useful for cases where you want to create a Gollama with some custom configuration, but don't want to specify every single field.

func (*Gollama) Chat 

func (c *Gollama) Chat(ctx context.Context, prompt string, options ...ChatOption) (*ChatOuput, error)

Chat generates a response to a prompt using the Ollama API.

The first argument is the prompt to generate a response to.

The function takes a variable number of options as arguments. The options are:

  • A slice of strings representing the paths to images that should be passed as vision input.
  • A slice of Tool objects representing the tools that should be available to the model.

The function returns a pointer to a ChatOuput object, which contains the response to the prompt, as well as some additional information about the response. If an error occurs, the function returns nil and an error.

func (*Gollama) Embedding added in v1.0.8 

func (c *Gollama) Embedding(ctx context.Context, prompt string) ([]float64, error)

Embedding generates a vector embedding for a given string of text using the currently set model. The model must support the "embeddings" capability.

The function will return an error if the model does not support the "embeddings" capability. The function will also return an error if the request fails.

The function returns a slice of floats, representing the vector embedding of the input text.

func (*Gollama) GetDetails added in v1.0.8 

func (c *Gollama) GetDetails(ctx context.Context, model ...string) ([]ModelDetails, error)

GetDetails retrieves the details of specified models from the server.

The function accepts a variadic parameter of model names. If no model names are provided, it defaults to using the model name set in the Gollama object.

It returns a slice of ModelDetails for each requested model, or an error if the request fails.

func (*Gollama) GetModels added in v1.0.29 

func (c *Gollama) GetModels(ctx context.Context) ([]string, error)

GetModels retrieves a list of available models from the server.

It returns a slice of strings containing model names, or an error if the request fails.

func (*Gollama) HasModel 

func (c *Gollama) HasModel(ctx context.Context, model string) (bool, error)

HasModel checks if a given model is available on the server.

The function will return an error if the request fails.

The function will return false if the model is not found on the server.

func (*Gollama) ListModels 

func (c *Gollama) ListModels(ctx context.Context) ([]ModelInfo, error)

ListModels lists all available models on the server.

The function will return an error if the request fails.

func (*Gollama) ModelSize 

func (c *Gollama) ModelSize(ctx context.Context, model string) (int, error)

ModelSize returns the size of a model on the server.

The function will return an error if the model is not found.

The function will return 0 if the model is not found.

func (*Gollama) PullIfMissing 

func (c *Gollama) PullIfMissing(ctx context.Context, model ...string) error

PullIfMissing pulls a model from the server if it is not available locally.

The function will return an error if the request fails.

The function will return an error if the model is not found on the server.

If no model is specified, the model name set in the Gollama object is used.

func (*Gollama) PullModel 

func (c *Gollama) PullModel(ctx context.Context, model string) error

PullModel pulls a model from the server if it is not available locally.

The function will return an error if the request fails.

The function will return an error if the model is not found on the server.

func (*Gollama) SetContextLength 

func (c *Gollama) SetContextLength(contextLength int64) *Gollama

SetContextLength Sets the size of the context window used to generate the next token. (Default: 2048)

func (*Gollama) SetHTTPTimeout 

func (c *Gollama) SetHTTPTimeout(timeout time.Duration) *Gollama

func (*Gollama) SetModel added in v1.0.30 

func (c *Gollama) SetModel(model string)

SetModel sets the model to use for the Gollama object.

func (*Gollama) SetRandomSeed 

func (c *Gollama) SetRandomSeed() *Gollama

func (*Gollama) SetSeed 

func (c *Gollama) SetSeed(seed int) *Gollama

SetSeed Sets the random number seed to use for generation. Setting this to a specific number will make the model generate the same text for the same prompt. (Default: 0)

func (*Gollama) SetSystemPrompt added in v1.0.7 

func (c *Gollama) SetSystemPrompt(prompt string) *Gollama

func (*Gollama) SetTemperature 

func (c *Gollama) SetTemperature(temperature float64) *Gollama

SetTemperature The temperature of the model. Increasing the temperature will make the model answer more creatively. (Default: 0.8)

func (*Gollama) SetTopK added in v1.0.28 

func (c *Gollama) SetTopK(topK int) *Gollama

SetTopK Reduces the probability of generating nonsense. A higher value (e.g. 100) will give more diverse answers, while a lower value (e.g. 10) will be more conservative. (Default: 40)

func (*Gollama) SetTopP added in v1.0.28 

func (c *Gollama) SetTopP(topP float64) *Gollama

SetTopP Works together with top-k. A higher value (e.g., 0.95) will lead to more diverse text, while a lower value (e.g., 0.5) will generate more focused and conservative text. (Default: 0.9)

func (*Gollama) Version 

func (c *Gollama) Version(ctx context.Context) (string, error)

type ItemProperty added in v1.0.10 

type ItemProperty struct {
	Type string `json:"type"`
}

type ModelDetails added in v1.0.8 

type ModelDetails struct {
	License    string `json:"license"`
	Modelfile  string `json:"modelfile"`
	Parameters string `json:"parameters"`
	Template   string `json:"template"`
	Details    struct {
		ParentModel       string   `json:"parent_model"`
		Format            string   `json:"format"`
		Family            string   `json:"family"`
		Families          []string `json:"families"`
		ParameterSize     string   `json:"parameter_size"`
		QuantizationLevel string   `json:"quantization_level"`
	} `json:"details"`
	ModelInfo struct {
		GeneralArchitecture               string   `json:"general.architecture"`
		GeneralBasename                   string   `json:"general.basename"`
		GeneralFileType                   int      `json:"general.file_type"`
		GeneralFinetune                   string   `json:"general.finetune"`
		GeneralLanguages                  []string `json:"general.languages"`
		GeneralLicense                    string   `json:"general.license"`
		GeneralParameterCount             int64    `json:"general.parameter_count"`
		GeneralQuantizationVersion        int      `json:"general.quantization_version"`
		GeneralSizeLabel                  string   `json:"general.size_label"`
		GeneralTags                       []string `json:"general.tags"`
		GeneralType                       string   `json:"general.type"`
		LlamaAttentionHeadCount           int      `json:"llama.attention.head_count"`
		LlamaAttentionHeadCountKv         int      `json:"llama.attention.head_count_kv"`
		LlamaAttentionLayerNormRmsEpsilon float64  `json:"llama.attention.layer_norm_rms_epsilon"`
		LlamaBlockCount                   int      `json:"llama.block_count"`
		LlamaContextLength                int      `json:"llama.context_length"`
		LlamaEmbeddingLength              int      `json:"llama.embedding_length"`
		LlamaFeedForwardLength            int      `json:"llama.feed_forward_length"`
		LlamaRopeDimensionCount           int      `json:"llama.rope.dimension_count"`
		LlamaRopeFreqBase                 int      `json:"llama.rope.freq_base"`
		LlamaVocabSize                    int      `json:"llama.vocab_size"`
		TokenizerGgmlBosTokenID           int      `json:"tokenizer.ggml.bos_token_id"`
		TokenizerGgmlEosTokenID           int      `json:"tokenizer.ggml.eos_token_id"`
		TokenizerGgmlMerges               any      `json:"tokenizer.ggml.merges"`
		TokenizerGgmlModel                string   `json:"tokenizer.ggml.model"`
		TokenizerGgmlPre                  string   `json:"tokenizer.ggml.pre"`
		TokenizerGgmlTokenType            any      `json:"tokenizer.ggml.token_type"`
		TokenizerGgmlTokens               any      `json:"tokenizer.ggml.tokens"`
	} `json:"model_info"`
	ModifiedAt string `json:"modified_at"`
}

type ModelInfo 

type ModelInfo struct {
	Model string `json:"model"`
	Size  int    `json:"size"`
}

type PromptImage added in v1.0.10 

type PromptImage struct {
	Filename string `json:"filename"`
}

type StructuredFormat added in v1.0.10 

type StructuredFormat struct {
	Type       string                    `json:"type"`
	Properties map[string]FormatProperty `json:"properties"`
	Required   []string                  `json:"required,omitempty"`
}

func StructToStructuredFormat added in v1.0.12 

func StructToStructuredFormat(s interface{}) StructuredFormat

type Tool added in v1.0.9 

type Tool struct {
	Type     string       `json:"type"`
	Function ToolFunction `json:"function"`
}

type ToolCall added in v1.0.9 

type ToolCall struct {
	Function ToolCallFunction `json:"function"`
}

type ToolCallFunction added in v1.0.9 

type ToolCallFunction struct {
	Name      string         `json:"name"`
	Arguments map[string]any `json:"arguments"`
}

type ToolFunction added in v1.0.9 

type ToolFunction struct {
	Name        string           `json:"name"`
	Description string           `json:"description,omitempty"`
	Parameters  StructuredFormat `json:"parameters"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.