jev

Documentation

Overview

Package jev calls TypeSafe's System One model — a decision model that answers typed questions with calibrated probabilities instead of generating text.

Why this shape. The tool-call reviewer used to ask the conversation model for strict JSON, then parse it, then retry when the model wrote prose around the object. That costs a full generation, and every failure mode is a parsing failure. A decision model answers in one pass with no prose: the question types are the request schema, so there is nothing to parse and nothing to retry on formatting.

What this package is careful about:

• Concurrency. Tool batches run in parallel, so several reviews can be in flight at once. The upstream documents 429/529 replies under load, so calls pass a limiter and retry with backoff rather than stampeding.
• Failure is loud. Every error path returns an error; there is no "assume safe". A caller that cannot get a decision must fall back to a slower reviewer or ask the user — never proceed.
• The key never reaches a log. It travels in one header and is never formatted into an error, a warning, or a plan's Raw text.

Index

• Constants
• type Answer
  • func (a Answer) Margin() float64
  • func (a Answer) RunnerUp() (string, float64)
  • func (a Answer) Top() (string, float64)
• type AnswerType
• type Client
  • func New(cfg Config) *Client
  • func (c *Client) Endpoint() string
  • func (c *Client) Evaluate(ctx context.Context, state any, questions map[string]Question) (*Result, error)
• type Config
• type Error
  • func (e *Error) Error() string
• type Question
  • func (q Question) Validate() error
• type QuestionType
• type Result
• type Usage

Constants

const (
	// MaxChoiceOptions is the documented cap on a Choice's criteria.
	MaxChoiceOptions = 255
	// MinScoreLevels and MaxScoreLevels bound a Score's rubric.
	MinScoreLevels = 2
	MaxScoreLevels = 10
)

Limits from the upstream documentation, enforced here so a programming mistake becomes a local error instead of a 422 after a round trip.

const (
	// DefaultConcurrency keeps a batch from stampeding a rate-limited service.
	// Four is a compromise: enough that a five-call batch barely serializes,
	// low enough that a burst is not what causes the 429.
	DefaultConcurrency = 4
	// DefaultRetries is the number of retries AFTER the first attempt.
	DefaultRetries = 3
	// DefaultBackoff is the base of the exponential backoff.
	DefaultBackoff = 300 * time.Millisecond
	// DefaultTimeout bounds one evaluation.
	DefaultTimeout = 10 * time.Second
)

const DefaultEndpoint = "https://api.typesafe.ai/v1/systemone"

DefaultEndpoint is the documented evaluation endpoint.

const DefaultModel = "jev-latest"

DefaultModel is TypeSafe's flagship alias.

Types

type Answer

type Answer struct {
	Type AnswerType
	// Choice is the winning option.
	Choice string
	// Probabilities maps every option or level to its probability.
	Probabilities map[string]float64
	// Confidence is how certain the model is (Choice and Score only). For a
	// Noul the probability itself carries that meaning.
	Confidence float64
	// Score is the probability-weighted position on the rubric.
	Score float64
	// Noul is the probability that a yes/no proposition holds.
	Noul float64
}

Answer is one decoded answer. Exactly one of the three fields is meaningful, according to Type.

func (Answer) Margin

func (a Answer) Margin() float64

Margin is how far the winner is ahead of the runner-up.

func (Answer) RunnerUp

func (a Answer) RunnerUp() (string, float64)

RunnerUp returns the second-highest probability, which is what tells a caller whether the answer was decisive: 0.9/0.05 is a verdict, 0.45/0.40 is not.

func (Answer) Top

func (a Answer) Top() (string, float64)

Top returns the highest-probability option and its probability.

type AnswerType

type AnswerType string

AnswerType mirrors QuestionType on the response side.

const (
	AnswerChoice AnswerType = "choice"
	AnswerScore  AnswerType = "score"
	AnswerNoul   AnswerType = "noul"
)

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client calls the evaluation endpoint. It is safe for concurrent use.

func New

func New(cfg Config) *Client

New builds a client. A nil HTTPClient gets a default with Timeout applied.

func (*Client) Endpoint

func (c *Client) Endpoint() string

Endpoint reports the evaluation endpoint this client was built for. It is the key the process-wide circuit breaker uses, so a caller can tell which gateway a decision came from.

func (*Client) Evaluate

func (c *Client) Evaluate(ctx context.Context, state any, questions map[string]Question) (*Result, error)

Evaluate answers every question against one state, in one call.

questions must be non-empty; each is validated locally first. The returned Result carries every answer under the key it was asked with.

type Config

type Config struct {
	// Endpoint overrides the evaluation endpoint (tests, or a gateway).
	Endpoint string
	// APIKey authenticates the request. An empty key is refused by Evaluate
	// rather than sent, because a 401 wastes a round trip and reads as an
	// upstream fault.
	APIKey string
	// APIKeyFunc resolves the key per call, and takes precedence over APIKey.
	// It exists because the credential store refreshes tokens: reading the key
	// once at construction would pin a stale value for the life of the process.
	APIKeyFunc func(context.Context) string
	// Model is the model id or alias. Empty means DefaultModel.
	Model string
	// HTTPClient carries the transport. The caller builds it, which is how the
	// proxy routing stays in the provider package instead of being duplicated
	// here: this client only needs a client, not the knowledge of which
	// providers route through what.
	HTTPClient *http.Client
	// Timeout bounds one call, overriding HTTPClient.Timeout when non-zero.
	Timeout time.Duration
	// MaxConcurrent bounds simultaneous calls. Zero means DefaultConcurrency.
	// It exists because tool batches review in parallel and the upstream
	// documents rate limits under load.
	MaxConcurrent int
	// MaxRetries bounds retry attempts for a retryable failure. Zero means
	// DefaultRetries. Negative disables retrying (tests).
	MaxRetries int
	// Backoff overrides the base backoff. Zero means DefaultBackoff.
	Backoff time.Duration
	// Now and Sleep are seams for tests. Nil means the real clock.
	Now   func() time.Time
	Sleep func(context.Context, time.Duration) bool
}

Config configures a Client.

type Error

type Error struct {
	StatusCode int
	Message    string
	// Retryable reports whether the status is one the upstream documents as
	// transient (429 Too Many Requests, 529 Overloaded).
	Retryable bool
}

Error is an upstream failure with the parts a caller needs to decide whether to retry. It never carries the API key.

func (*Error) Error

func (e *Error) Error() string

type Question

type Question struct {
	Type QuestionType
	// Instructions is what to decide. It may be a string or, for a question
	// that needs data alongside it, a map/array — the upstream documents
	// structured instructions, and passing an object keeps the data next to the
	// question instead of concatenated into prose.
	Instructions any
	// Criteria is the option set for a Choice (option → description), the
	// ordered rubric for a Score, or {true, false} descriptions for a Noul.
	Criteria any
}

Question is one typed question. The zero value is invalid and Evaluate refuses it, so a programming mistake surfaces before a round trip.

func (Question) Validate

func (q Question) Validate() error

Validate reports a malformed question. It runs locally so a schema mistake costs nothing, rather than a 422 after a network round trip.

type QuestionType

type QuestionType string

QuestionType names the three primitives.

const (
	// Choice picks one option from a named set.
	Choice QuestionType = "choice"
	// Score places the state on an ordered rubric.
	Score QuestionType = "score"
	// Noul answers a yes/no question with a probability.
	Noul QuestionType = "noul"
)

type Result

type Result struct {
	Model   string
	Answers map[string]Answer
	Usage   Usage
	// Latency is how long the call took, for the telemetry the reviewer already
	// reports.
	Latency time.Duration
	// Attempts counts tries including the first, so a caller can see retries.
	Attempts int
}

Result is one evaluation: the answers plus what the call cost.

type Usage

type Usage struct {
	InputTokens  int `json:"input_tokens"`
	OutputTokens int `json:"output_tokens"`
}

Usage is the request's token accounting.