llm

Documentation

Index

• type ChatRequest
• type ChatResponse
• type ChatUsage
• type Client
  • func New(cfg config.LLMConfig) *Client
  • func (c *Client) Chat(ctx context.Context, req ChatRequest) (string, error)
  • func (c *Client) ChatJSON(ctx context.Context, messages []Message, out any) error
  • func (c *Client) Model() string
  • func (c *Client) SetTracker(t *usage.Tracker)
• type Embedder
  • func NewEmbedder(cfg config.EmbeddingConfig) *Embedder
  • func (e *Embedder) Dimensions() int
  • func (e *Embedder) Embed(ctx context.Context, text string) ([]float32, error)
  • func (e *Embedder) EmbedBatch(ctx context.Context, texts []string) ([][]float32, error)
  • func (e *Embedder) SetTracker(t *usage.Tracker)
• type Message

Types

type ChatRequest

type ChatRequest struct {
	Model       string    `json:"model"`
	Messages    []Message `json:"messages"`
	MaxTokens   *int      `json:"max_tokens,omitempty"`
	Temperature *float64  `json:"temperature,omitempty"`
}

ChatRequest is a non-streaming chat completion request.

type ChatResponse

type ChatResponse struct {
	Choices []struct {
		Message Message `json:"message"`
	} `json:"choices"`
	Usage *ChatUsage `json:"usage,omitempty"`
	Error *struct {
		Message string `json:"message"`
	} `json:"error,omitempty"`
}

ChatResponse mirrors the OpenAI chat completion response (non-streaming).

type ChatUsage

type ChatUsage struct {
	PromptTokens          int `json:"prompt_tokens"`
	CompletionTokens      int `json:"completion_tokens"`
	TotalTokens           int `json:"total_tokens"`
	PromptCacheHitTokens  int `json:"prompt_cache_hit_tokens"`
	PromptCacheMissTokens int `json:"prompt_cache_miss_tokens"`
}

ChatUsage is the token accounting block of an OpenAI-compatible response. PromptCacheHitTokens is the DeepSeek prompt_cache_hit_tokens field (input tokens served from the provider's prompt cache — the direct cost saving).

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is a fully-configurable OpenAI-compatible chat client.

func New

func New(cfg config.LLMConfig) *Client

New creates a chat client from config. apiKey is resolved from the config's declared env var at construction time.

func (*Client) Chat

func (c *Client) Chat(ctx context.Context, req ChatRequest) (string, error)

Chat performs a non-streaming completion and returns the assistant text.

func (*Client) ChatJSON

func (c *Client) ChatJSON(ctx context.Context, messages []Message, out any) error

ChatJSON asks the model to produce a JSON payload and unmarshals it into out.

func (*Client) Model

func (c *Client) Model() string

Model returns the configured model name.

func (*Client) SetTracker

func (c *Client) SetTracker(t *usage.Tracker)

SetTracker wires an optional usage tracker; token deltas are reported after every completion.

type Embedder

type Embedder struct {
	// contains filtered or unexported fields
}

Embedder produces vector embeddings for text via an OpenAI-compatible embeddings endpoint. Fully configurable (base_url/key/model/dimensions).

func NewEmbedder

func NewEmbedder(cfg config.EmbeddingConfig) *Embedder

NewEmbedder creates an embedding client from config.

func (*Embedder) Dimensions

func (e *Embedder) Dimensions() int

Dimensions returns the configured vector dimensionality.

func (*Embedder) Embed

func (e *Embedder) Embed(ctx context.Context, text string) ([]float32, error)

Embed computes a single embedding vector for text.

func (*Embedder) EmbedBatch

func (e *Embedder) EmbedBatch(ctx context.Context, texts []string) ([][]float32, error)

EmbedBatch computes embeddings for multiple texts in one request.

func (*Embedder) SetTracker

func (e *Embedder) SetTracker(t *usage.Tracker)

SetTracker wires an optional usage tracker; embedding input tokens are reported after every request.

type Message

type Message struct {
	Role    string `json:"role"` // user | assistant | system
	Content string `json:"content"`
}

Message is one chat turn in OpenAI format.