Documentation
¶
Overview ¶
Package harness decides how much scaffolding the agent loop applies to a given model. A strong model is driven leanly; a weaker or more heavily quantized one is given more help, such as grammar-constrained decoding, a tighter context, and extra verification, so it can still complete a task reliably. The decision is a pure function of a model's measured capability, so the policy is testable on its own; the assembly that builds the model client and executor applies the resulting plan, and a separate evaluation produces the profile.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ModelProfile ¶
type ModelProfile struct {
// ToolCallReliability is how reliably the model emits well-formed tool calls.
ToolCallReliability float64
// StructuredOutput is how reliably the model adheres to a required output shape.
StructuredOutput float64
// InstructionFollowing is how well the model follows long, detailed instructions.
InstructionFollowing float64
// EffectiveContext is the token span the model stays coherent over, which is often shorter
// than its advertised window. Zero means unknown, and imposes no extra cap.
EffectiveContext int
}
ModelProfile is a model's measured capability fingerprint, for a particular model, quant, and runtime. Each score is in [0,1] where higher is more capable. The zero value is the unknown, worst case on purpose: an unmeasured model is treated conservatively and given the most help.
type Plan ¶
type Plan struct {
// ConstrainToolCalls forces grammar-constrained decoding so a tool call cannot be malformed.
ConstrainToolCalls bool
// SimplifyToolSchemas presents smaller, flatter tool schemas a weaker model can follow.
SimplifyToolSchemas bool
// MaxContext caps the context window to the model's effective range; zero means no cap.
MaxContext int
// VerifyPasses is the number of extra self-check or repair passes before a result is trusted.
VerifyPasses int
}
Plan is how hard the loop works for a model: the adaptations the assembly applies.
func Adapt ¶
func Adapt(p ModelProfile, advertisedContext int) Plan
Adapt maps a profile to a plan. Weaker capability yields more scaffolding and never less: low tool-call or structured-output reliability forces constrained decoding; weak instruction-following simplifies schemas; a known-narrow effective context caps the window; and lower overall reliability adds verification passes. The mapping is monotonic, so a strictly weaker model is never given less help than a stronger one, and the zero-value profile yields the most conservative plan.
func PlanFor ¶
func PlanFor(src ProfileSource, model string, advertisedContext int) Plan
PlanFor resolves model's profile from src and maps it to a scaffolding plan against the advertised context window. A nil source, or a model the source has not measured, is treated as unknown: the zero profile, which Adapt turns into the fully scaffolded plan. This makes the safe default automatic, so a model is only driven leanly once it has been measured and found reliable, never on assumption.
type ProfileSource ¶
type ProfileSource interface {
Profile(model string) (ModelProfile, bool)
}
ProfileSource resolves the measured capability profile for a model, keyed by its selector (the provider:model or runtime:model id). A source that has no measurement for a model returns ok=false, and the caller treats the model as unknown, which yields the most conservative plan. It is the read side of the capability store: an evaluation harness writes profiles in, and the assembly reads them out here to decide how much to scaffold a run.
type StaticProfiles ¶
type StaticProfiles map[string]ModelProfile
StaticProfiles is a ProfileSource backed by a fixed map, for seeding known profiles ahead of a live evaluation store. A nil or absent entry reads as unmeasured.
func (StaticProfiles) Profile ¶
func (s StaticProfiles) Profile(model string) (ModelProfile, bool)
Profile returns the profile recorded for model, if any.
Source Files