shadow

Documentation

Overview

Package shadow runs a request a second time on the local model, after the real answer has gone out, and records how the two compare. It is how a class of request is judged before a rule is trusted to send it local.

Nothing here can affect a client. A shadow starts only once the real exchange is finished, runs on its own goroutine, and its answer goes to the ledger and nowhere else.

Index

• func Compare(realStop string, realTools []string, realDuration time.Duration, ...) *ledger.Compare
• type Input
• type Output
  • func Run(ctx context.Context, client *http.Client, local *backend.Local, in Input) Output
• type Runner
  • func New(local *backend.Local, led *ledger.Ledger, captures *capture.Writer, ...) *Runner
  • func (r *Runner) Close()
  • func (r *Runner) Submit(ex proxy.Exchange, rec ledger.Record)

Functions

func Compare

func Compare(realStop string, realTools []string, realDuration time.Duration, local anthropic.Summary) *ledger.Compare

Compare sets a local answer beside the real one, which stopped for realStop having called realTools, and took realDuration.

Types

type Input

type Input struct {
	// ID names the run in its capture.
	ID string
	// Body is the request as the client originally sent it.
	Body             []byte
	AnthropicVersion string
	// Model is the local model's id in LM Studio; ContextTokens the largest
	// prompt to send it, or zero to use what it is loaded with.
	Model         string
	ContextTokens int
	Timeout       time.Duration
	// Reason is recorded as why the run happened.
	Reason string
}

Input is one request to run on the local model for comparison.

type Output

type Output struct {
	Skipped string
	Err     error
	// Exchange is what was sent to the local model and what came back. Nil
	// when Skipped. It carries no credentials, because none are sent.
	Exchange *proxy.Exchange
}

Output is what came of one run. Exactly one of these describes it: Skipped (it was never sent), Err (it was sent and the connection failed), or neither (an answer came back, which may itself be an error or cut short; Exchange.Summary says).

func Run

func Run(ctx context.Context, client *http.Client, local *backend.Local, in Input) Output

Run sends one request to the local model and reads the answer. It is the shared core of shadow mode and of replay.

type Runner

type Runner struct {
	// contains filtered or unexported fields
}

Runner runs shadows one at a time.

func New

func New(local *backend.Local, led *ledger.Ledger, captures *capture.Writer, logf func(string, ...any)) *Runner

New starts a Runner. Close it to stop.

func (*Runner) Close

func (r *Runner) Close()

Close stops the runner. A shadow in progress is abandoned, and ones still waiting are dropped: they are measurements, not work anyone is owed.

func (*Runner) Submit

func (r *Runner) Submit(ex proxy.Exchange, rec ledger.Record)

Submit asks for a shadow of a finished exchange, whose ledger record is rec. It returns at once. Exchanges that did not ask for a shadow, or have no complete real answer to compare with, are ignored.