narrowing

Documentation

Overview

Package narrowing decides, before a chat turn reaches the AI model, which of a project's tables the model needs to see.

A project can hold hundreds of tables. Sending every definition to a large generative model on each turn is slow and costly, and most of it is noise. A decision model such as Jev (TypeSafe AI's external model, reached through the cloud decider) scores each table's relevance to the question cheaply, and only the tables it selects are passed on as schema context.

The decision walks a ladder and stops at the first rung that answers:

1. Deterministic project knowledge (Rule): free, exact, and the engine is never called.
2. A decision engine (a decision.ScoredProvider) asked one relevance question over the candidate tables, judged by a decision.SelectionPolicy.
3. The full schema, which is exactly what the chat did before this package.

Every way the engine can fail to give a usable answer lands on rung 3: not configured, an error, a timeout, an incomplete or uncalibrated answer, an uncertain answer, none-of-these, or a stopped engine (quota, budget, misconfigured). A stop is never read as "ask a bigger paid model to decide": it is recorded in the Record, and the chat simply keeps its full context. After a failure the engine is not asked again for a cool-down, so a slow or refusing service costs one wait, not one per turn.

A wrong narrowing would be silent, so three things make it recoverable and visible: tables the engine judged only possibly relevant stay in the model's context; tables on the foreign-key path between selected tables are added, and the previous turn's tables are carried into a follow-up; and the model is told the names of the omitted tables and can read any one's definition with the describe_relation tool (Narrower.Describe). The user sees one line naming the tables the model was given (Record.Notice).

What leaves the machine when an engine is configured: the user's question and up to three earlier questions of the session, every table name and its column names (never types or row data), the interaction id and the client context the cloud client already sends. See the README.

Index

• Constants
• func ScorerOf(p decision.Provider) (decision.ScoredProvider, error)
• func WithHistory(ctx context.Context, h History) context.Context
• type Attempt
• type Config
• type History
• type Link
• type Mechanism
• type Narrower
  • func New(cfg Config) (*Narrower, error)
  • func (n *Narrower) Candidates() []string
  • func (n *Narrower) Describe(name string) (string, bool)
  • func (n *Narrower) Narrow(ctx context.Context, prompt string) Outcome
  • func (n *Narrower) Warnings() []string
• type Outcome
• type Record
  • func (r Record) DetectionSteps() []cloudproto.DetectionStep
  • func (r Record) Notice() string
  • func (r Record) Summary() string
• type Rule
• type Score
• type Settings
  • func LoadSettings(projectDir string) Settings

Constants

const (
	ReasonDisabled     = "disabled"
	ReasonNoCandidates = "no_candidates"
	ReasonNoReduction  = "no_reduction"
	// ReasonTooManyCandidates: more tables than one engine question may hold.
	ReasonTooManyCandidates = "too_many_candidates"
	// ReasonCoolingDown: the engine failed or refused recently and is left alone
	// for a while.
	ReasonCoolingDown = "cooling_down"
)

Fallback reasons recorded when the full schema was kept, besides the ones that name an engine outcome as the library reports it: an attempt outcome (timeout, unavailable, unsupported, auth, rejected, quota, budget, misconfigured, cancelled, error, invalid) or a selection outcome (uncertain, none, unscored). All are short identifiers, safe as telemetry dimensions.

const SettingsFile = "ai/table-rules.yaml"

SettingsFile is where a project keeps its table-narrowing settings and rules, relative to the project directory. The format is provisional until the DataTug decision layer is specified (plan task K-1):

decision: auto          # disabled (default) | auto | cloud
rules:
  - phrase: Sales by country
    tables: [Invoice, Customer]

"decision" opts the project in to the cloud decider (which forwards the question text and the table and column names to TypeSafe AI's Jev model, through the DataTug AI cloud); the environment variable DATATUG_AI_DECISION_PROVIDER overrides it. Rules are local and deterministic.

Functions

func ScorerOf

func ScorerOf(p decision.Provider) (decision.ScoredProvider, error)

ScorerOf returns p as a decision.ScoredProvider, which a decision engine must be to answer a relevance question. The cloud decider is one.

func WithHistory

func WithHistory(ctx context.Context, h History) context.Context

WithHistory returns ctx carrying the session's history for Narrow.

Types

type Attempt

type Attempt struct {
	Provider  string `json:"provider"`
	Outcome   string `json:"outcome"`
	Detail    string `json:"detail,omitempty"`
	LatencyMs int64  `json:"latencyMs"`
}

Attempt is one provider's part in the decision, in the library's vocabulary (decision.Attempt outcomes such as decided, timeout, quota, unavailable).

type Config

type Config struct {
	// Relations are the healthy tables and views of the active source, in the
	// order the full schema context lists them.
	Relations []api.CatalogRelation
	// Rules are the project's deterministic knowledge, tried before the engine.
	Rules []Rule
	// Links, when set, returns the schema's foreign keys, read each turn. They let
	// the decision keep the tables on the path between selected tables.
	Links func() []Link
	// Engine scores candidate tables. Nil means no engine: only rules narrow.
	Engine decision.ScoredProvider
	// Policy turns the engine's probabilities into a verdict. It must be valid
	// (decision.NarrowingPolicy is the intended one).
	Policy decision.SelectionPolicy
	// Format renders relations as the model's schema context (the chat's
	// FormatSchemaContext), so the byte counts compare like with like.
	Format func(*api.CatalogSchema) string
	// Product is sent to the engine; default "datatug".
	Product string
	// Timeout bounds the engine call; default 1.5s.
	Timeout time.Duration
	// CoolDown is how long the engine is left alone after a failure or a stop;
	// default 5 minutes.
	CoolDown time.Duration
	// Now is the clock; default time.Now.
	Now func() time.Time
}

Config assembles a Narrower. The engine, the clock and the formatter are injected so that tests need no network and no wall clock.

type History

type History struct {
	// Questions are the user's earlier questions in this session, oldest first.
	// Only the last few are used.
	Questions []string
	// Kept are the tables the previous narrowed turn put in the model's context.
	// Empty when the previous turn did not narrow.
	Kept []string
}

History is what the session already knows when a follow-up arrives. A question such as "and by genre?" is meaningless alone: it is scored together with the questions before it, and the tables the previous turn kept are carried over (see Narrow).

type Link

type Link struct {
	FromSchema, From string
	ToSchema, To     string
}

Link is one foreign-key relationship between two tables (either direction; the schema is optional and matched case-insensitively with the table name).

type Mechanism

type Mechanism string

Mechanism says which rung of the decision ladder produced the narrowing. The ladder is: deterministic project knowledge, then a decision engine; the full schema is the floor below both.

const (
	// MechanismNone: nothing decided, so the chat kept its full schema context.
	MechanismNone Mechanism = ""
	// MechanismDeterministic: a project rule answered, and no engine was called.
	MechanismDeterministic Mechanism = "deterministic"
	// MechanismEngine: a decision engine (Jev, through the cloud decider) scored
	// the candidate tables and the selection policy chose.
	MechanismEngine Mechanism = "engine"
)

type Narrower

type Narrower struct {
	// contains filtered or unexported fields
}

Narrower decides which tables a turn's model sees. Its zero value is not usable; build one with New. It is safe for concurrent use.

func New

func New(cfg Config) (*Narrower, error)

New builds a Narrower, checking the configuration once.

func (*Narrower) Candidates

func (n *Narrower) Candidates() []string

Candidates returns the ids the engine is asked about, in schema order.

func (*Narrower) Describe

func (n *Narrower) Describe(name string) (string, bool)

Describe returns the definition of one table of the schema, as it would appear in the model's context. It is the read-only lookup behind describe_relation: the model can recover a table the narrowing left out. The name is matched exactly, then case-insensitively.

func (*Narrower) Narrow

func (n *Narrower) Narrow(ctx context.Context, prompt string) Outcome

Narrow decides which tables the model sees for prompt. It never fails: every problem is a fallback to the full schema, recorded in the returned Record.

func (*Narrower) Warnings

func (n *Narrower) Warnings() []string

Warnings are problems found when the narrower was built, in words for the user: a rule that names a table the schema lacks never fires. They never stop a chat.

type Outcome

type Outcome struct {
	Context string
	Record  Record
}

Outcome is one decision. An empty Context means the narrowing did not apply and the caller keeps its full schema context.

type Record

type Record struct {
	DecidedAt time.Time `json:"decidedAt"`
	// Mechanism is the rung that decided; Engine and Model name the answering
	// engine and its model id when one was asked.
	Mechanism Mechanism `json:"mechanism,omitempty"`
	Engine    string    `json:"engine,omitempty"`
	Model     string    `json:"model,omitempty"`
	// Provenance is the decision.Provenance class of the answer that stood:
	// deterministic, calibrated or self_reported.
	Provenance string `json:"provenance,omitempty"`
	Policy     string `json:"policy,omitempty"`
	// DeciderEnabled is true when a decision engine was configured when the
	// question was asked. The question of a turn is only reused as history for a
	// later question if it was (a question typed before the user opted in must
	// never be sent after).
	DeciderEnabled bool `json:"deciderEnabled,omitempty"`
	// Verdict is the selection policy's judgement, for example "several" or
	// "uncertain: only_potential".
	Verdict string `json:"verdict,omitempty"`
	// Narrowed is true only when the model was given fewer tables than the
	// full schema. FallbackReason says why not, when the answer is false.
	Narrowed       bool   `json:"narrowed"`
	FallbackReason string `json:"fallbackReason,omitempty"`
	// StoppedBy is set when the engine refused for a reason that must not be
	// answered by asking a bigger, paid model: an exhausted allowance (quota), a
	// spent budget (budget) or a misconfigured endpoint (misconfigured).
	StoppedBy string `json:"stoppedBy,omitempty"`

	CandidatesBefore int `json:"candidatesBefore"`
	CandidatesAfter  int `json:"candidatesAfter"`
	// Selected are the tables the engine or rule selected; Strong is the subset
	// it was most sure of; Potential are tables it judged possibly relevant, kept
	// in the model's context so that ambiguity is preserved, not hidden.
	// Proposed holds the picks of an uncalibrated answer, which are never a
	// selection and are recorded only.
	Selected  []string `json:"selected,omitempty"`
	Strong    []string `json:"strong,omitempty"`
	Potential []string `json:"potential,omitempty"`
	Proposed  []string `json:"proposed,omitempty"`
	// Carried are tables kept because the previous turn kept them (a follow-up is
	// about the same data); Closure are tables kept because they lie on the
	// foreign-key path between selected tables.
	Carried []string `json:"carried,omitempty"`
	Closure []string `json:"closure,omitempty"`
	// Kept are the tables whose definitions reached the model, in schema order.
	Kept   []string `json:"kept,omitempty"`
	Scores []Score  `json:"scores,omitempty"`

	ContextBytesBefore int `json:"contextBytesBefore"`
	ContextBytesAfter  int `json:"contextBytesAfter"`

	Attempts     []Attempt `json:"attempts,omitempty"`
	LatencyMs    int64     `json:"latencyMs"`
	InputTokens  int       `json:"inputTokens,omitempty"`
	OutputTokens int       `json:"outputTokens,omitempty"`
}

Record is the inspectable provenance of one narrowing decision. It is stored with the chat session and summarised into telemetry. It holds table names, scores and counts, never row data.

func (Record) DetectionSteps

func (r Record) DetectionSteps() []cloudproto.DetectionStep

DetectionSteps renders the decision as the cloud interaction report's detection steps: one for the rung that decided or, when an engine was asked and the full schema was kept, one for that engine; none when no rule matched and no engine was asked (the decider is disabled). The steps carry counts, mechanism and the engine and model ids, but no table names or question text.

func (Record) Notice

func (r Record) Notice() string

Notice is the one line shown to the user when the model was given fewer tables than the schema holds ("" when nothing was narrowed). Names are quoted so that a hostile name cannot add lines.

func (Record) Summary

func (r Record) Summary() string

Summary is the record as one telemetry-safe identifier: it states the counts and the reason, never a table name.

type Rule

type Rule struct {
	Phrase string   `yaml:"phrase"`
	Tables []string `yaml:"tables"`
}

Rule is deterministic project knowledge: when the question is exactly Phrase (compared after rules.Normalize: case, spacing and trailing punctuation do not matter), the answer needs exactly Tables, and no decision engine is asked.

type Score

type Score struct {
	ID          string  `json:"id"`
	Probability float64 `json:"probability"`
}

Score is one candidate's probability as the engine reported it.

type Settings

type Settings struct {
	// Decision is the project's choice of decision provider ("" when unset).
	Decision string
	Rules    []Rule
	Warnings []string
}

Settings are a project's narrowing settings as read from SettingsFile. Reading never fails: a problem becomes a Warning and the offending part is ignored, so a bad file can never stop a chat from starting.

func LoadSettings

func LoadSettings(projectDir string) Settings

LoadSettings reads the project's SettingsFile. A project without one has no settings. The file is read only when it is a regular file inside the project (a symbolic link is never followed), is at most 64 KiB, and no warning echoes its content.