rpc

Documentation

Overview

Package rpc is AutoDoc's msgpack-RPC API (ADR 0203 §4.4): a projection of core, with no business logic of its own. Every client speaks it: the TUI, the Web-UI, AutoVim's auto-core, the GUI.

A session must say sys.hello with a protocol this build serves (MinProtocol through Protocol) before any other verb answers; a hello with another refuses the session for good. The TUI is held to this build's Protocol exactly. The server only answers: it never sends a notification or a request, so a request/response-only client (auto-core's) can use every verb.

Index

• Constants
• Variables
• func ProbeOn(ctx context.Context, network, addr string) (string, error)
• func RegistrationsMap(t registrations.Tables) map[string]any
• func RegistrationsOf(caps any) (registrations.Tables, error)
• type Databases
• type Embeddings
• type Events
• type Option
  • func WithEmbeddings(e Embeddings) Option
  • func WithEvents(ev Events) Option
  • func WithFiles(d *docs.Docs) Option
  • func WithListener(ln net.Listener) Option
  • func WithLogger(l logger.Logger) Option
  • func WithPreferences(p Preferences) Option
  • func WithRankers(r Rankers) Option
  • func WithRegistrations(t registrations.Tables) Option
  • func WithStoreID(id string) Option
• type Preferences
• type ProbeInfo
  • func Probe(ctx context.Context, network, addr string) (ProbeInfo, error)
• type ProviderChoice
• type RankerList
• type Rankers
• type SchemaStatus
• type Server
  • func New(workspaces Workspaces, version string, opts ...Option) *Server
  • func (s *Server) Addr() string
  • func (s *Server) Instance() string
  • func (s *Server) RequestShutdown()
  • func (s *Server) Run(ctx context.Context) error
  • func (s *Server) Verbs() []string
• type Workspace
• type Workspaces
  • func Fixed(ws ...*Workspace) Workspaces

Constants

const (
	CodeProtocolMismatch  int64 = -32020 // upgrade
	CodeHandshakeRequired int64 = -32021 // say sys.hello
	CodeNoSuchWorkspace   int64 = -32060 // re-list the workspaces
	CodeNotFound          int64 = -32061 // refresh the listing
	CodeConflict          int64 = -32062 // doc.read, then merge or retry
	CodeCursorExpired     int64 = -32063 // re-list, and restart index.changes from the status cursor
	CodeUnsupported       int64 = -32065 // the workspace's driver cannot
	CodeCommitted         int64 = -32066 // the write landed: doc.read and compare, never re-send
	CodeEmbedFailed       int64 = -32067 // search lexically, or retry later
	CodeProviderRefused   int64 = -32068 // the provider refused: switch provider, or fix its key
	CodeSwitching         int64 = -32069 // a new model is filling: search lexically until it is ready
	CodeRankUnavailable   int64 = -32070 // a build that answers ranked results or none could not rank: retry later
)

The error codes, chosen by what the client does next (ADR 0203 §4.4). The protocol codes keep AutoDB's numbers.

const MaxMessage = 4 << 20

MaxMessage is the largest message either way: a document plus its envelope (docs.MaxSize).

const MinProtocol int64 = 12

MinProtocol is the oldest protocol this build still serves. A session keeps the protocol it declared: a verb added after it answers as an unknown method, as an older daemon would, and every request and result it knew keeps its shape and meaning; a result may only gain keys, which a client ignores. A change that is not an addition raises MinProtocol, or keeps the old behaviour for sessions below it.

const Protocol int64 = 19

Protocol is the API's version. A client declares the one it was written for, which this build serves from MinProtocol up; any change to the verbs, their parameters or their results bumps it (TestVerbsArePinned holds the list). sys.hello and sys.shutdown are frozen across protocols: a newer client stops an older daemon with them, by declaring the daemon's number, to start the installed one in its place.

Protocol 7 adds workspace.set_patterns for workspace admission changes. Protocol 8 adds frontmatter schemas (workspace.set_schema, doc.validate, search.query's facets, workspace.list's schema and index.status's diagnosed), doc.outline, workspace.set_text_extensions with workspace.list's text_extensions, sys.events (sys.hello answers the client's token and the log's head), and workspace.set_provider with workspace.list's provider. Protocol 9 adds workspace.configure, every setting saved at once, sys.capabilities, and workspace.list's databases (ADR 0214). Protocol 10 adds a build's registrations (ADR 0216): workspace.list's text_collisions; the documents a daemon holds, index.status's held, held_stale and held_unchecked, search.query's hold on a hit, and index.reindex's refusal of a held document; sys.capabilities' registrations, always there. Protocol 11 adds the ranker models (ADR 0215): the ranker.* verbs, search.query's rank and a ranked hit's rank_score, and sys.capabilities' ranker. Protocol 12 adds search.query's stages: the stages a search runs, asked for in place of a mode, and how its answer was made. Protocol 13 adds index.documents, search.query's line_start and line_end on a hit, sys.capabilities' verbs, and sys.hello's server_protocol, min_protocol and store_id. Protocol 14 adds links from frontmatter relations (the kinds supersedes, superseded_by, amends, related, sources and adr) to graph.links, graph.backlinks, graph.neighborhood and graph.unresolved, and their kinds option; a session below 14 sees the body links alone, its graph unchanged. It also adds graph.unresolved's reason cycle (a supersession loop), workspace.configure's abstract_chunk and demote_superseded with workspace.list's, search.query's retrieval option and a hit's superseded_by. Protocol 15 adds index.documents' diagnosed: the documents whose frontmatter the schema diagnoses, each with its diagnostics; a session below 15 is refused the option. Protocol 16 is ADR 1791284787: embedding.cancel_switch deletes the partial target's vectors and embedding.remove the removed provider's model's, so a session below 16 is refused those two verbs; index.purge_model refuses the active model and the target saying what to do instead. Protocol 17 is ADR 1791430651: file.read, file.write and file.locate, a local peer's files outside every workspace. Protocol 18 adds graph.resolve: where a link as written reaches, for an editor following one not yet indexed. Protocol 19 is ADR 1791474356: doc.read_raw, doc.write_raw, file.read_raw and file.write_raw, an HTML file's own bytes where doc.read and file.read give its derived text; a session below 19 sees them as unknown methods.

const ServerName = "autodoc"

ServerName is what sys.hello answers as "server", so a probe tells AutoDoc from another occupant.

const TUIName = "autodoc-tui"

TUIName is the name the TUI says hello with. It is admitted at this build's Protocol only: the TUI is the same binary as its daemon, and a mismatch is its cue to offer a restart.

Variables

var ErrNoRegistrations = errors.New("rpc: sys.capabilities has no registrations: a protocol error")

ErrNoRegistrations is a Protocol 10 sys.capabilities without its registrations: a protocol error, never read as none, or every client built with registrations would offer a restart.

var ErrNoSwitch = errors.New("rpc: no model switch is under way")

ErrNoSwitch is embedding.cancel_switch with no model filling to replace the active one.

var ErrNotAutodoc = errors.New("rpc: the endpoint's occupant is not a compatible autodoc")

ErrNotAutodoc is an endpoint whose occupant is not a compatible AutoDoc.

var ErrSuppliedRanker = errors.New("this build supplies its ranker")

ErrSuppliedRanker is a change refused while the build supplies its ranker: the one in use, the window, or the stored selection kept dormant for a build that does not.

var ErrWindow = fmt.Errorf("a ranker's window is %d to %d candidates", rank.MinWindow, rank.MaxWindow)

ErrWindow is a window outside the ranker's bounds.

Functions

func ProbeOn

func ProbeOn(ctx context.Context, network, addr string) (string, error)

ProbeOn asks the occupant of addr who it is, as Probe does, and returns its version when it is an AutoDoc of this Protocol. A dial error is returned as it is: nothing answers there.

func RegistrationsMap

func RegistrationsMap(t registrations.Tables) map[string]any

RegistrationsMap is a build's registrations as sys.capabilities reports them: each chunker's version by extension, each format's deriver by format, and their fingerprint.

func RegistrationsOf

func RegistrationsOf(caps any) (registrations.Tables, error)

RegistrationsOf reads the registrations of a sys.capabilities answer.

Types

type Databases

type Databases struct {
	UID, Destination, VectorIndex string
	ViewArgs                      map[string]any
	Connections                   []store.ConnectionSummary
}

Databases is a workspace's database settings as workspace.list reports them: never a DSN, only what each connection points at (store.ConnectionSummary).

type Embeddings

type Embeddings interface {
	Providers(ctx context.Context) (providers []store.ProviderInfo, active, lastErr string, err error)
	AddProvider(ctx context.Context, sp store.ProviderSpec) (store.ProviderInfo, error)
	UpdateProvider(ctx context.Context, name string, sp store.ProviderSpec) error
	RemoveProvider(ctx context.Context, name string) error
	Use(ctx context.Context, name string) error
	Models(ctx context.Context, name string, sp store.ProviderSpec) ([]string, error)
	Usage(ctx context.Context, name string, days int) ([]store.Usage, error)
	Log(ctx context.Context, name string, limit int) ([]store.LogEntry, error)
	CancelSwitch(ctx context.Context) (model string, err error)
}

Embeddings are the daemon's embedding providers. A server given none answers the embedding verbs that it cannot.

type Events

type Events interface {
	AppendEvent(ctx context.Context, e store.Event) (int64, error)
	Events(ctx context.Context, since int64, limit int) ([]store.Event, int64, bool, error)
}

Events is the daemon-wide event log (core/store's).

type Option

type Option func(*options)

Option configures a Server.

func WithEmbeddings

func WithEmbeddings(e Embeddings) Option

WithEmbeddings serves the daemon's embedding providers.

func WithEvents

func WithEvents(ev Events) Option

WithEvents keeps the event log in ev; without it, sys.events answers that the server keeps none.

func WithFiles

func WithFiles(d *docs.Docs) Option

WithFiles serves file.read and file.write on d, the documents of a vfs rooted at "/" (the daemon's own disk). Without it they answer that the server serves no outside files.

func WithListener

func WithListener(ln net.Listener) Option

WithListener serves on ln (the daemon's unix socket).

func WithLogger

func WithLogger(l logger.Logger) Option

WithLogger logs to l.

func WithPreferences

func WithPreferences(p Preferences) Option

WithPreferences keeps clients' preferences in p (the daemon's store).

func WithRankers

func WithRankers(r Rankers) Option

WithRankers serves the daemon's rankers.

func WithRegistrations

func WithRegistrations(t registrations.Tables) Option

WithRegistrations reports the build's registrations in sys.capabilities (ADR 0216 §1.4); without it, the empty ones of the community build.

func WithStoreID

func WithStoreID(id string) Option

WithStoreID reports the store's identity (store.Identity) in sys.hello and its probe, so a client that found this daemon through the store's lease-info can check it serves that store.

type Preferences

type Preferences interface {
	Preferences(ctx context.Context) (map[string]string, error)
	SetPreference(ctx context.Context, name, value string) error
}

Preferences are what a client keeps between runs, by name: the store's. A server given none answers the preference verbs that it cannot.

type ProbeInfo

type ProbeInfo struct {
	Version     string
	Protocol    int64 // the daemon's own protocol
	MinProtocol int64 // the oldest it serves; equal to Protocol for a daemon before ranges
	StoreID     string
	Instance    string
	Addr        string
	PID         int64 // the daemon's process; 0 when its hello names none
}

ProbeInfo is what an AutoDoc daemon says of itself to a probe: a hello that declares no protocol.

func Probe

func Probe(ctx context.Context, network, addr string) (ProbeInfo, error)

Probe asks the occupant of addr who it is, with a sys.hello that declares no protocol (so it admits nothing), and returns what any AutoDoc answers, whatever its protocol: the caller decides whether it will do. A dial error is returned as it is: nothing answers there.

type ProviderChoice

type ProviderChoice struct {
	Override string
	Err      string
}

ProviderChoice is which embedding provider a workspace uses: its own (Override, by name), or the daemon's when Override is "". Err says why its own is not set up, so it searches by words.

type RankerList

type RankerList struct {
	Rankers  []store.RankerInfo
	Active   string
	Window   int
	Err      string
	Supplied string
}

RankerList is ranker.list's answer: the stored rankers, the one in use, its window, why the one named is not in use, and the build's own ranker's model when it supplies one.

type Rankers

type Rankers interface {
	ListRankers(ctx context.Context) (RankerList, error)
	AddRanker(ctx context.Context, sp store.RankerSpec) (store.RankerInfo, error)
	UpdateRanker(ctx context.Context, name string, sp store.RankerSpec) error
	RemoveRanker(ctx context.Context, name string) error
	Use(ctx context.Context, name string) error
	SetWindow(ctx context.Context, n int) error
	Models(ctx context.Context, name string, sp store.RankerSpec) ([]string, error)
	Usage(ctx context.Context, name string, days int) ([]store.Usage, error)
	Log(ctx context.Context, name string, limit int) ([]store.LogEntry, error)
	// Supplied is the build's own ranker's model, and whether the build supplies one.
	Supplied() (string, bool)
}

Rankers are the daemon's rankers. A server given none answers the ranker verbs that it cannot.

type SchemaStatus

type SchemaStatus struct {
	Path   string
	Active bool
	Fields int
	Err    string
	Line   int
}

SchemaStatus is how a workspace's schema file stands: the stored path ("" for none), whether a valid schema is active and how many fields it declares, and why the file is not valid now (Err, on Line when it is one line's). An invalid edit leaves the last valid schema active.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server is the API over the daemon's workspaces.

func New

func New(workspaces Workspaces, version string, opts ...Option) *Server

New returns the server of workspaces, answering as version.

func (*Server) Addr

func (s *Server) Addr() string

Addr is the address the server listens on.

func (*Server) Instance

func (s *Server) Instance() string

Instance is this process's instance id, as sys.hello answers it.

func (*Server) RequestShutdown

func (s *Server) RequestShutdown()

RequestShutdown stops Run, as sys.shutdown does.

func (*Server) Run

func (s *Server) Run(ctx context.Context) error

Run serves until ctx ends or a client calls sys.shutdown, then drains.

func (*Server) Verbs

func (s *Server) Verbs() []string

Verbs lists the registered verbs, sorted.

type Workspace

type Workspace struct {
	Name, Root       string
	EmbeddingPolicy  string
	EmbeddingMode    func() string
	SectionTokens    int
	SectionSize      func() int // live workspace preference, when managed by the daemon
	Include, Exclude []string
	Err              error
	Index            *index.Indexer
	Docs             *docs.Docs
	Following        func() follow.Status
	// Warming is what the workspace waits on before it answers fully — a provider being set up, a
	// restart to take one — for index.status; nil or empty when it is ready.
	Warming        func() []string
	Searched       func() // any client searched this workspace
	EmbeddingQueue func() (state, behind string)
	// FrontmatterSchema is the workspace's frontmatter schema (ADR 0212 §5): the active one, nil for
	// none, and how its file stands. nil when the server keeps no schemas.
	FrontmatterSchema func() (*schema.Schema, SchemaStatus)
	// TextExtensions are the workspace's own plain-text extensions (ADR 0212 §3); nil for none.
	TextExtensions func() []string
	// TextCollisions are those of its text extensions a registration of the build reads instead
	// (ADR 0216 §1.3); nil for none.
	TextCollisions func() []string
	// Databases are the workspace's identity, destination and database connections (ADR 0214),
	// read from the store when listed; nil when the server keeps none.
	Databases func() Databases
	// Provider is the workspace's own embedding provider, when it has one (ADR 0212 §7).
	Provider ProviderChoice
}

Workspace is one workspace as the daemon serves it. Err set means it could not be opened (its root is gone); every verb naming it fails with Err, and workspace.list reports it.

type Workspaces

type Workspaces interface {
	// List is every workspace, by name.
	List() []*Workspace
	// Get is the workspace named name.
	Get(name string) (*Workspace, bool)
	// Add creates a workspace and starts serving it.
	Add(ctx context.Context, w config.Workspace) (*Workspace, error)
	// Rename gives a workspace a new name.
	Rename(ctx context.Context, name, to string) error
	// Remove stops serving a workspace and deletes its index; its files stay.
	Remove(ctx context.Context, name string) error
}

Workspaces is the daemon's set of workspaces: what the API serves, and what the workspace verbs change. The daemon's implementation keeps them in the store and starts and stops each one's indexer and follower.

func Fixed

func Fixed(ws ...*Workspace) Workspaces

Fixed is a set of workspaces that never changes: the workspace verbs answer that it cannot.