ollama

Documentation

Index

• Constants
• Variables
• func BuildCategories(categories []CategoryEmbedding) error
• func EmbedBatch(texts ...string) ([][]float32, error)
• func EmbedChunks(chunks ...vector.Chunk) ([][]float32, error)
• type Category
  • func Classify(content, contentURL string, categories []CategoryEmbedding, ...) ([]Category, error)
• type CategoryEmbedding
• type Config
• type DomainCategoryPrior
  • func BuildPriors() *DomainCategoryPrior
  • func NewDomainCategoryPrior() *DomainCategoryPrior
  • func (p *DomainCategoryPrior) Get(domain string) map[string]float32
  • func (p *DomainCategoryPrior) Observe(domain, label string, alpha float32)
  • func (p *DomainCategoryPrior) Seed(domain string, weights map[string]float32)
• type URLSignals
  • func ExtractURLSignals(rawURL string) (URLSignals, error)

Constants

const (
	// ConfigEnvPrefix is the prefix applied to environment variables for configuring ollama.
	ConfigEnvPrefix = "OLLAMA_"
)

Variables

var LoadConfig = sync.OnceValue(func() error {
	if err := config.Load(ConfigEnvPrefix, &cfg); err != nil {
		return fmt.Errorf("google: unable to load config: %w", err)
	}

	if err := validation.Validate.Struct(cfg); err != nil {
		return fmt.Errorf("google: unable to validate config: %w", err)
	}

	slog.Debug("Ollama config loaded.")
	return nil
})

LoadConfig loads the auth0 configuration and ensures this is only done one time, no matter how many times it is called.

Functions

func BuildCategories

func BuildCategories(categories []CategoryEmbedding) error

BuildCategories embeds each category description using an instruction tuned for classification.

func EmbedBatch

func EmbedBatch(texts ...string) ([][]float32, error)

EmbedBatch embeds up to BatchSize texts in one request against a local Ollama server. Ollama returns L2-normalized vectors already, so no further normalization is needed for cosine similarity.

func EmbedChunks

func EmbedChunks(chunks ...vector.Chunk) ([][]float32, error)

EmbedChunks embeds all chunks, batching requests according to c.BatchSize.

Types

type Category

type Category struct {
	Label string
	Score float32
}

func Classify

func Classify(
	content, contentURL string,
	categories []CategoryEmbedding,
	domainPriors *DomainCategoryPrior,
	alpha float32,
) ([]Category, error)

Classify embeds content and returns the best-matching category label along with its cosine similarity score (confidence proxy).

type CategoryEmbedding

type CategoryEmbedding struct {
	Label       string
	Description string
	Embedding   []float32
}

type Config

type Config struct {
	// URL is the URL to the ollama server.
	URL string `koanf:"url" validate:"required,url"`
	// Model is the model to use.
	Model string `koanf:"model" validate:"required"`
	// BatchSize is the number of input texts to process at once.
	BatchSize int `koanf:"batchsize" validate:"omitempty,gt=0"`
	// KeepAlive is how long to keep a request alive.
	KeepAlive config.Duration `koanf:"keepalive"`
}

Config contains the pubsub configuration options.

type DomainCategoryPrior

type DomainCategoryPrior struct {
	// contains filtered or unexported fields
}

DomainCategoryPrior tracks a learned or curated category distribution per domain. Safe for concurrent use.

func BuildPriors

func BuildPriors() *DomainCategoryPrior

func NewDomainCategoryPrior

func NewDomainCategoryPrior() *DomainCategoryPrior

func (*DomainCategoryPrior) Get

func (p *DomainCategoryPrior) Get(domain string) map[string]float32

func (*DomainCategoryPrior) Observe

func (p *DomainCategoryPrior) Observe(domain, label string, alpha float32)

Observe updates a domain's prior via exponential moving average each time a document from it gets classified -- so priors self-improve from your own pipeline's output without manual curation. Use a small alpha (0.05-0.15) so one mis-classified outlier can't swing a domain's prior.

func (*DomainCategoryPrior) Seed

func (p *DomainCategoryPrior) Seed(domain string, weights map[string]float32)

Seed manually associates a domain with known category weights, e.g. {"Technology & Computing": 0.9, "Business and Finance": 0.1} for a publisher you already know the beat of.

type URLSignals

type URLSignals struct {
	Domain       string   // "example.com", "www." stripped
	PathSegments []string // human-readable path tokens; numeric IDs/UUIDs dropped
}

URLSignals extracts classification-relevant features from a content URL.

func ExtractURLSignals

func ExtractURLSignals(rawURL string) (URLSignals, error)