localllm

Documentation

Index

• Constants
• type Client
  • func New(baseURL string) *Client
  • func (c *Client) ChatJSON(ctx context.Context, model string, messages []Message) (string, error)
  • func (c *Client) Ping(ctx context.Context, model string) error
• type Message

Constants

const DefaultBaseURL = "http://localhost:11434"

DefaultBaseURL mirrors localembed — Ollama listens on 11434 by default. GETDEBUG_OLLAMA_URL overrides for custom hosts.

const DefaultModel = "qwen2.5-coder:7b"

DefaultModel — Qwen2.5-Coder 7B is the best small-model code-aware option on Ollama today (Apache-2.0, ~4.7GB Q4, JSON-mode-friendly). Users can override with --local-llm-model; popular alternatives: deepseek-r1:7b, llama3.1:8b. We pick a code-specialized default so SAST-style prompts degrade gracefully on weak models.

Types

type Client

type Client struct {
	BaseURL string
	HTTP    *http.Client
}

Client is the minimal HTTP client we need. Stateless aside from the configured base URL + http.Client; safe to reuse across calls.

func New

func New(baseURL string) *Client

New returns a client with sane defaults. 10-minute per-request timeout — chat completions on CPU for a non-trivial source file can take 2-5 min on a 7B model; the outer context is still the wall-clock budget.

func (*Client) ChatJSON

func (c *Client) ChatJSON(ctx context.Context, model string, messages []Message) (string, error)

ChatJSON sends `messages` to Ollama in JSON-output mode and returns the assistant's content string. Caller json.Unmarshal's it against the expected shape. We do NOT parse here — keeping JSON parsing in the caller lets each prompt own its own response schema.

Temperature defaults to 0.1 (consistent classification, not creative).

func (*Client) Ping

func (c *Client) Ping(ctx context.Context, model string) error

Ping mirrors localembed.Ping — server reachable + model installed. Returned errors point at the fix (install Ollama / pull the model) because this IS the onboarding error message for new users.

type Message

type Message struct {
	Role    string `json:"role"`
	Content string `json:"content"`
}

Message is one turn in the chat request. Roles match OpenAI's: system / user / assistant — Ollama is OpenAI-API-compatible enough that the hosted SAST prompt format ports cleanly.