llm

Documentation

Overview

Package llm implements Excise's v0.3 opt-in rerank against a local Ollama host. Everything in this package is reachable only when the user passes `--llm`; the v0.2 heuristic path never imports it.

The trust contract documented in the README is enforced here:

• the only outbound HTTP call is to the host configured in excise.toml (default http://localhost:11434), nothing else
• timeouts are bounded by the caller-supplied context (default 20s)
• any failure (network, HTTP status, JSON parse) is mapped to ErrLLMUnavailable so callers can fall back deterministically without having to recognise every failure mode

We intentionally use net/http with no third-party SDK — Ollama's /api/generate is a single endpoint, the JSON shape is tiny, and adding a dependency is more code to audit than the 30 lines below.

Index

• Variables
• type OllamaClient
  • func NewOllamaClient(host, model string, timeout time.Duration) *OllamaClient
  • func (c *OllamaClient) Generate(ctx context.Context, prompt string) (string, error)
• type OllamaReranker
  • func NewOllamaReranker(client *OllamaClient) *OllamaReranker
  • func (r *OllamaReranker) Rerank(ctx context.Context, s *session.Session, shortlist []suggest.TurnScore) ([]suggest.TurnScore, error)
• type Reranker
  • func NewStubReranker(err error, picks []struct{ ... }) Reranker

Variables

var ErrLLMUnavailable = errors.New("llm: backend unavailable")

ErrLLMUnavailable is the single sentinel error this package returns when anything goes wrong: HTTP failure, non-2xx response, timeout, body too short, JSON parse failure. Callers compare with errors.Is.

Types

type OllamaClient

type OllamaClient struct {
	Host    string        // base URL, e.g. http://localhost:11434
	Model   string        // model tag, e.g. llama3.2
	Timeout time.Duration // per-call timeout; 0 means no limit (not recommended)
	HTTP    *http.Client  // optional injection for tests
}

OllamaClient is the tiny HTTP shim around Ollama's /api/generate. Construction is cheap and concurrency-safe; one instance per rerank call is fine.

func NewOllamaClient

func NewOllamaClient(host, model string, timeout time.Duration) *OllamaClient

NewOllamaClient is the canonical constructor.

func (*OllamaClient) Generate

func (c *OllamaClient) Generate(ctx context.Context, prompt string) (string, error)

Generate sends a single prompt and returns the model's full text response. Any error path maps to ErrLLMUnavailable wrapped with the original cause for diagnostic logging, so callers can `errors.Is(err, ErrLLMUnavailable)` without checking every leaf.

type OllamaReranker

type OllamaReranker struct {
	Client *OllamaClient
}

OllamaReranker is the production implementation. It defers all transport to OllamaClient and all parsing to parseRerankReply, then merges the returned scores/reasons back onto the original TurnScore records.

If the LLM returns a turn id we don't recognise, that entry is dropped (we never invent turns). If the LLM omits a turn id from the input, we keep it at its heuristic position with an empty reason — better partial information than total fallback.

func NewOllamaReranker

func NewOllamaReranker(client *OllamaClient) *OllamaReranker

NewOllamaReranker is the canonical constructor.

func (*OllamaReranker) Rerank

func (r *OllamaReranker) Rerank(ctx context.Context, s *session.Session, shortlist []suggest.TurnScore) ([]suggest.TurnScore, error)

Rerank implements the Reranker contract.

type Reranker

type Reranker interface {
	Rerank(ctx context.Context, s *session.Session, shortlist []suggest.TurnScore) ([]suggest.TurnScore, error)
}

Reranker is the v0.3 interface — Rerank takes the v0.2 heuristic shortlist and returns a reordered copy with each entry's LLMReason populated.

The interface is intentionally tiny so adding a v0.4 remote backend (or a stub for tests) is a single method, not a re-wiring of the CLI.

func NewStubReranker

func NewStubReranker(err error, picks []struct {
	TurnID string
	Score  float64
	Reason string
}) Reranker

NewStubReranker is a test helper that returns a Reranker which:

• on err != nil, returns err immediately (use to exercise the fallback path)
• otherwise merges the supplied (turn_id, score, reason) tuples onto the incoming shortlist via the same mergeReplies path the real reranker uses.