executor

Documentation

Overview

Package executor describes the supported model executors.

Index

• Variables
• type Address
• type Executor
  • func All() []Executor
  • func For(k Kind) (Executor, bool)
• type Kind
  • func Kinds() []Kind
  • func ParseKind(s string) (Kind, error)
• type Params
• type Setting

Variables

var ErrUnsupported = errors.New("engine kind is not supported")

ErrUnsupported marks an engine value without an executor.

Types

type Address

type Address struct {
	Host   Setting
	Port   Setting
	APIKey Setting
}

Address names the settings the engine reads back from a merged argv.

type Executor

type Executor interface {
	Kind() Kind
	Binary() string     // executable name resolved via PATH
	BackendDir() string // ggml backend plugins loaded at runtime
	ReadyPath() string  // HTTP path polled until 200
	Address() Address   // spellings the merged argv is read back through
	Args(p Params) []string
	Env(p Params) []string // extra env, appended to the parent env
}

Executor shapes the launch of one family.

func All

func All() []Executor

All lists executors in stable kind order.

func For

func For(k Kind) (Executor, bool)

For finds the executor of a kind.

type Kind

type Kind string

Kind is a supported executor family (the exedra.engine model field).

const (
	Llama      Kind = "llama"      // llama-server: llm | vlm | embeddings
	Transcribe Kind = "transcribe" // transcribe-server: asr
)

Supported kinds.

func Kinds

func Kinds() []Kind

Kinds lists supported families in stable order.

func ParseKind

func ParseKind(s string) (Kind, error)

ParseKind validates the exedra.engine value of a model.

type Params

type Params struct {
	ModelPath    string
	Alias        string
	Capabilities []string // model modalities (llm | vlm | embeddings | asr | tts)
	Host         string
	Port         int
	APIKey       string
	NoGPU        bool
	UI           bool // keep the web UI enabled
	Ctx          int  // 0 = executor default
	Threads      int  // 0 = executor default
}

Params is the flat launch data an executor turns into argv/env.

type Setting

type Setting struct {
	Flag string
	Env  string
}

Setting is one launch value: its flag and env twin.