Retracted: Withdrawn — every version published before this one.
The first two carried names belonging to the private installation this format
grew in. The rest carried its VOCABULARY: word lists baked into the engine and
the linter, and a shipped example built from one application's dictionary. A
library used by an agent about a kitchen, one about a car fleet and one about
a warehouse must know none of that, and a version that does is one an
embedder should not pick up.
The public module proxy caches a version's content immutably, so all of them
stay fetchable through it whatever happens to this repository. Retracting is
what tells the tooling not to select them and warns whoever already has one.
They are retracted rather than re-pointed at the cleaned commits on purpose:
a version whose content changes breaks go.sum for everyone who fetched it.
README
¶
skill-lint
Checks a skill against the rules of the format — the ones the engine cannot enforce at run time without having already burned the turn.
Every rule here was paid for by a broken turn, and each is about a defect that stays quiet: a loop that collects into a variable nobody writes runs all its iterations and gathers nothing; a typo in a variable's name resolves to an empty string; a required field the instruction allows to be empty sends the model into whitespace up to the token ceiling. None of it raises an error. All of it produces an answer that looks fine.
import (
"gopkg.in/yaml.v3"
"github.com/inhuman/skill-engine/lint"
)
rep, err := lint.Lint(raw, facts, lint.Options{Unmarshal: yaml.Unmarshal})
if err != nil {
return err // the check could not run — not a verdict on the skill
}
fmt.Println(rep.Text())
if rep.HasErrors() {
// your policy decides what that means
}
err is an infrastructure failure. "The skill is bad" is always findings in the
report.
The line with Flow.Validate
Validate refuses what cannot run. The linter reports what runs badly.
Do not move advice into Validate: a rule promoted there starts breaking skills
that are already written and already working.
The two are used together — the linter runs Validate itself (rule W1) and
reads the description in the shape execution will see it, profiles folded in.
Severity is ours, the gate is yours
The library says "this is a defect of the format". Whether that stops a save, fails a build, or is merely shown to the author is the embedder's policy — which is why there are no profiles here.
Facts: the rule knows what to compare, you say what with
Which servers are up, which tools they carry, which built-in tools exist — the engine cannot know any of it. Pass what you know:
facts := lint.Facts{
ServerNames: func() []string { return live.Servers() },
AllTools: func() map[string][]string { return live.Tools() },
ToolSchemas: func() map[string][]byte { return live.Schemas() },
BuiltinTools: func() []string { return registry.Names() },
WriteServers: func() []string { return live.Writing() },
SkillNames: func() []string { return catalogue.Names() },
}
Every field is optional. A missing one skips the rules that need it and
records the reason in Report.Skipped and as an X1 info finding. That is
deliberate: a linter that falls over because a dependency is down is a linter
people stop running, and a partial check that looks like a clean one is worse
than no check at all.
The same applies to the vocabulary in Options: the format deliberately leaves
asset kinds, roles and calling conventions to the application, so the rules
about them stay off — loudly — until you say what you call things.
The rules
| id | severity | catches | needs |
|---|---|---|---|
| S1 | error | the file parses, the header is legal, the format version is one the engine speaks | — |
| S3 | error | the playbook uses a construct you have removed | Options.StaleAPIs |
| S5 | warn | the playbook's size against the budget — it is context weight on every run | — |
| S6 | info | no trigger_examples: only reachable by being named outright |
— |
| W1 | error | the description does not pass the engine's own validation | — |
| W2 | error | a server the program names is declared by the skill and registered | Facts.ServerNames |
| W3 | error | a call step's tool exists on its server |
Facts.AllTools |
| W4 | warn | an asset is passed the way its kind implies — through the model's context or past it | Options.Assets |
| W5 | error | a call step carries the arguments its tool requires |
Facts.ToolSchemas |
| W6 | error | somebody writes into the variable a loop collects | — |
| W7 | error | a built-in tool called by a step is declared in builtin_tools |
— |
| W8 | error/warn | a wrapped call result is substituted whole where a field was meant | Options.Envelopes |
| W9 | error | an object in a response schema has at least one required field |
— |
| W10 | error | from: in a call's arguments receives a handle, not the value's text |
— |
| W11 | warn | an asset's params are keys the resolver actually reads |
Options.Assets |
| W12 | warn | an instruction names a tool without saying how tools are called | Options.CallProtocol, Facts.AllTools |
| W13 | warn | a free-text field of a response schema has a length ceiling | — |
| W14 | error | every reference names a variable that exists at that point in the flow | — |
| W15 | error | a declared built-in tool exists in the registry | Facts.BuiltinTools |
| W16 | error | a required field is not one the description beside it allows to be empty | — |
| W17 | error | switch.var is given a variable's name, not a {{template}} |
— |
| E1 | error | every server the skill declares is registered | Facts.ServerNames |
| E2 | error | a tool the playbook calls exists on the server it names | Options.CallProtocol, Facts.AllTools |
| E3 | warn | a read-only skill does not reach for a server that writes | Options.ReadOnlyRoles, Facts.WriteServers |
| E4 | warn | a delegate step names a skill that exists |
Facts.SkillNames |
| E5 | error | a built-in tool the playbook says to call is declared | Facts.BuiltinTools |
| X1 | info | a rule did not run, and why | — |
lint.Rules() returns the same catalogue as data. The numbers are owned by this
package: while rules were being added on both sides of the boundary they
collided, and a number meaning two things in two places is worse than no number.
What it does not check, and why
- Judgement by a model — do two skills claim the same requests, is the description precise enough, is the instruction well written. That needs an intent matcher, a corpus of live phrasings and a judge model: the installation's business, not the format's.
- Your own file wrapper — front matter, a markdown body, fences with attributes. The format is one YAML document; anything you wrap around it, you check yourself.
- Policy — which role may do what, what a schedule requires, what blocks a save. The format deliberately does not close those lists.
The limitation worth knowing before you rely on it
The linter only sees contradictions the author spelled out. W16 catches a required field because the instruction beside it says "not named — an empty string". Where the description says nothing about absence, there is nothing for statics to grab, and the same failure goes through unseen. Finding those needs a live run of the step against the real model with the real schema — not a rule.
Documentation
¶
Overview ¶
Package lint checks a skill against the rules of the format — the ones the engine cannot enforce at execution time without having already burned the turn.
Why it lives with the format. Every rule here was paid for by a broken turn in production, and each is about a defect that stays QUIET: a loop that collects into a variable nobody writes runs all its iterations and gathers nothing, a typo in a variable name resolves to an empty string, a required field the instruction allows to be empty sends the model into whitespace up to the token ceiling. None of it raises an error; all of it produces an answer that looks fine. Left in the embedding application, these rules are rewritten from scratch by the next embedder, along with the failures that taught them.
The split with Flow.Validate is deliberate: Validate refuses what CANNOT run, the linter reports what runs badly. Moving advice into Validate would start breaking skills that are already written and already working.
Degradation is built in: every fact about the installation is optional, a missing one skips its rules and records the skip in Report.Skipped AND as an info finding — a partial check must never look like a full one. A linter that falls over because a dependency is unavailable is a linter people stop running.
Index ¶
Constants ¶
const DefaultPlaybookBudget = 12288
DefaultPlaybookBudget — the playbook size above which S5 warns. Not a limit but a reminder: a playbook is the weight of the context of every single run.
const SkipRule = "X1"
SkipRule — the rule id carrying "a rule did not run". It is a FINDING and not only an entry in Report.Skipped, because a caller that reads findings and nothing else would otherwise see a partial check as a clean one.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AssetVocabulary ¶
type AssetVocabulary struct {
// CodeKinds — kinds whose content is a program: it belongs PAST the model,
// passed by reference into a call's arguments.
CodeKinds []string
// ReferenceKinds — kinds whose content is knowledge for the model: it
// belongs IN the instruction's text, or the model never sees it.
ReferenceKinds []string
// KnownParams — the param keys the host's resolver actually reads.
KnownParams []string
// LangParam — the key naming a code asset's language, if the host has one.
LangParam string
}
AssetVocabulary — the names the embedder uses for asset kinds and params.
The format deliberately does not close these lists (kinds are an application's vocabulary — that is why `params` is an open map), so the rules about them cannot be grounded without the host saying what it calls things. Left empty, the rules that need it skip with a reason. The vocabulary used by the shipped examples is returned by SchemaVocabulary.
func SchemaVocabulary ¶
func SchemaVocabulary() AssetVocabulary
SchemaVocabulary returns the asset vocabulary the shipped examples use. A starting point, not a default: an embedder with its own kinds passes its own.
type Envelope ¶
type Envelope struct {
// Server — whose results are wrapped.
Server string
// Fields — the wrapper's fields, named in the finding so the author
// recognises what they are looking at.
Fields []string
// Payload — the field holding what the call actually produced.
Payload string
}
Envelope describes a server whose call result WRAPS the payload instead of being it — an exit code, stdout and stderr around what the script printed.
The engine cannot know which server does that, and the difference is invisible: substituting the whole envelope where the payload is expected breaks nothing loudly. A loop honestly makes one iteration over the wrapper, a renderer honestly reports zero findings — because there was nothing to find.
type Facts ¶
type Facts struct {
// ServerNames — the registered MCP servers. W2, E1.
ServerNames func() []string
// AllTools — server name → the tools it carries. W3, W12, E2.
AllTools func() map[string][]string
// ToolSchemas — "server:tool" → the tool's input JSON Schema. W5 checks a
// call step's arguments against `required` in it.
ToolSchemas func() map[string][]byte
// BuiltinTools — the built-in tools the application actually has. W15, E5.
BuiltinTools func() []string
// WriteServers — the servers that CHANGE something. E3 uses it to spot a
// skill that calls itself read-only and reaches for one anyway.
WriteServers func() []string
// SkillNames — the catalogue a `delegate` step can name. E4.
SkillNames func() []string
}
Facts — what the embedder knows about ITS installation, and the engine cannot know: which servers are up, which tools they carry, which built-in tools exist. The rule knows WHAT to compare, the host says WHAT WITH.
Every field is optional. nil (or an empty answer) skips the rules that need it and records the reason — never an error, and never silence.
type Finding ¶
type Finding struct {
Rule string
Severity Severity
Skill string
Path string
Line int // 1-based; 0 = the finding is about the whole file
Message string
}
Finding — one rule's result on one skill.
type Options ¶
type Options struct {
// Unmarshal — how to parse YAML. Required, for the same reason the engine
// takes it as a parameter: a library that picks a YAML implementation picks
// it for everyone who embeds it.
Unmarshal skillengine.Unmarshal
// PlaybookBudget — the size of a playbook, in bytes, above which S5 warns.
// 0 = DefaultPlaybookBudget; negative = the rule is off.
PlaybookBudget int
// CallProtocol — the name of the tool an MCP tool is called THROUGH, if the
// application has one (a step is handed that one tool, not a function per
// tool name). Empty → W12 and E2 do not run.
CallProtocol string
// Envelopes — servers whose result wraps the payload. Empty → W8 does not run.
Envelopes []Envelope
// Assets — what the host calls its asset kinds and params. See AssetVocabulary.
Assets AssetVocabulary
// StaleAPIs — constructs the host has removed. Empty → S3 does not run.
StaleAPIs []StaleAPI
// ReadOnlyRoles — the role names that promise to change nothing. Empty →
// E3 does not run.
ReadOnlyRoles []string
// ImplicitBuiltins — built-in tools handed out WITHOUT being declared. A
// skill is not required to declare them, and E5 does not ask it to.
ImplicitBuiltins []string
// HostVars — variables the embedding application puts into the flow before
// the first step (the request, the history). A skill does not declare them
// and is free to read them; W14 would otherwise report every one of them as
// a typo. Leaving this empty is safe but noisy — the rule cannot tell an
// injected variable from a misspelled one.
HostVars []string
}
Options — the knobs and the host's vocabulary. The zero value is valid: the rules that need a name they were not given skip with a reason.
type Report ¶
type Report struct {
Findings []Finding
// Skipped — rules that did not run, with the reason. Read it before
// concluding a skill is clean.
Skipped []string
}
Report — the findings of one run plus a summary of what was skipped.
func Lint ¶
Lint checks one skill. An error means the check itself could not run (a missing parser, a broken option) — "the skill is bad" is findings in the report, never an error.
type Rule ¶
type Rule struct {
// ID — the rule's number, e.g. "W14". The numbers are owned by this
// package: while rules were being added on both sides of the boundary they
// collided, and a number that means two things in two places is worse than
// no number.
ID string
// Title — what it catches, in one line.
Title string
// Emits — the severities the rule can produce.
Emits []Severity
// Needs — the Facts and Options fields the rule cannot run without. Empty
// means it only needs the skill itself.
Needs []string
}
Rule — one entry of the catalogue: what a rule catches and what it needs to run at all.
The catalogue exists so an embedder can show the list without reading the source, and so that "which rules did not run, and why" has an answer before the run rather than after it. A test keeps it honest in both directions: a rule the code emits and the catalogue does not list is a rule nobody can look up, and a catalogued rule that fires nowhere is a rule that was renamed or quietly lost its call site.
type Severity ¶
type Severity string
Severity ranks a finding. What BLOCKS is decided by the caller: the library says "this is a defect of the format", the embedder decides whether that stops a save or fails a build. Without that split every embedder would need its own severity scale to express its own policy.
type StaleAPI ¶
type StaleAPI struct {
// Pattern — a regular expression matched against the playbook.
Pattern string
// What / Instead — how the finding reads: "<what> is no longer supported —
// use <instead>".
What string
Instead string
}
StaleAPI — a construct the embedder has REMOVED, with what replaced it.
The vocabulary belongs to the host: the engine knows nothing of the tools an application used to have. Empty list → rule S3 does not run.
Source Files