unswell

package module
v0.1.0-alpha.17 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 16, 2026 License: MIT Imports: 34 Imported by: 1

README

Unswell

A linter for lean English, built to reduce AI-sounding prose.

Unswell's main motivation is to keep formulaic AI-style wording from leaking into source code and documentation. AI-assisted work often leaves behind conversational preambles, generic claims of importance, inflated modifiers and repeated setup. Unswell finds these patterns in prose and explains what to revise before the text reaches a commit or a published document.

The checks apply to text regardless of its author. They enforce a chosen editorial policy through concrete findings and explainable scores. The checker runs locally as a Go library and CLI and sends source text nowhere. Three further commands, unswell review, unswell trim, and unswell compare, read a document with a model you name; see Review and trim with a model.

Unswell is not an authorship detector. A clean scan of a generated text is a correct result, and a finding on a human-written text is a finding about the wording. No score is a probability that a tool wrote the text; see ADR 0037 for the goal and its non-goals.

Development alpha. The published release is v0.1.0-alpha.16. Stage 2 features include custom rules, baselines, and checks of changed code. Go tools can train, calibrate, and load models.

Rules and defaults remain experimental. No editorial model has met the criteria for use in the product. See the acceptance audit for features, release evidence, and work still open. Revision probability is unavailable until a labeled corpus and compatible calibration model meet the later acceptance criteria.

What it does

unswell check applies fixed rules offline: announcements, stock phrases, repetition, a page that leans on em dashes. The strict profile adds wordy constructions, long sentences, and emoji in headings. It explains each finding, gives the same result on every run, and can fail a build on wording your project has decided to refuse.

unswell check also gives each Markdown page an AI-sounding index from 0 to 100 and lists the marks behind it: em dashes, emoji in headings and list items, arrows, stock words, bold labels in lists. On pages from before a commit that removes AI-sounding wording, set against README files written before November 2022, the index separates the two with an AUC of 0.87, and 3.9% of those README files reach its high level. unswell fix removes the marks that have a mechanical replacement, without a model. On 250 pages that no earlier measurement had read, blind model judges found the page after unswell fix the less AI-sounding one against the original on 231 of 233 decisive pages, and against a page fixed with the findings of a third-party linter on 146 of 209. A model that follows the report of unswell check gave 111 of 113 and 73 of 108 on 131 more such pages.

The index also reads the rhythm of the sentences, for a page that a model wrote under a style guide and that has none of those marks: plain statements of one length in a row, each opening with its subject. Against human blog posts written before November 2022 it separates such posts with an AUC of 0.87 to 0.99, where the marks alone gave 0.25 to 0.28. When gpt-6-astra revised 179 such pages by the report, blind model judges found the page more like a model's than its revision on 153 of 178 decisive pages.

unswell review gives a document to a model you name. The model reads it in context and flags sentences to tighten. Unswell binds each flag to the exact place in the source and drops a flag that does not quote its sentence.

unswell trim asks a model for sentences with empty words removed and keeps a proposal only if it deletes words and nothing else. It changes a file only when you ask.

On answers that chat models gave to instructions and that experts then edited, the words that unswell trim removes are words the expert also removed or replaced in 71.2% and 68.0% of cases with two trimming models, against 62.7% and 59.3% for random words of the same sentences. Removing them brings the text nearer to the expert's version than removing random words of the same sentences does, by 0.13 and 0.15 words per removed word.

How each part was measured is in the documentation of review and trim.

Quick start

Build with the minimum Go version declared in go.mod:

go build -o bin/unswell ./cmd/unswell
bin/unswell config init --profile technical
bin/unswell check README.md docs/
bin/unswell fix README.md docs/

Install the published alpha without Go through the Homebrew tap:

brew install stokaro/unswell/unswell

A separate GitHub Action runs pinned release archives and writes JSON/SARIF reports. See installation and release verification for availability and platform checks.

For example, this introduction:

It is important to note that the client retries a request only after a connection failure.

produces filler.announced-importance. A direct version preserves the condition:

The client retries a request only after a connection failure.

Use the strict profile to make the introductory phrase a hard policy violation:

printf '%s\n' 'It is important to note that the client may retry.' |
  bin/unswell check --stdin --filename draft.md --profile strict

Review and trim with a model

Fixed rules name constructions. unswell review asks a model you name to read each document in context, binds its flags to source bytes, and drops a flag that does not quote its item:

bin/unswell review README.md docs \
  --endpoint https://api.example.com/v1/chat/completions \
  --model example-model --api-key-env REVIEW_API_KEY

A local program that reads a prompt on standard input works as well, and a second model can shorten the list.

unswell trim proposes sentences with words removed:

docs/guide.md:3:1: trim
    - These features significantly improve the ability to type-check your code.
    + These features improve the ability to type-check your code.

A proposal is shown only if a deterministic check passes: no word is added, replaced, or moved, at least half of the sentence remains, and no negation, number, modal word, code, or link is removed. --write applies the trims to the files.

unswell compare checks a revision of a page. A judge reads the opening of the page and of the revision without a hint of which is which, in both orders. The revision is accepted only when the judge names the page both times, the statements and the sentences of the page are still in the revision, no number, code, or link is lost, and no em dash is added. The sentences are compared by the command itself, without a model:

bin/unswell compare docs/guide.md docs/guide.revised.md \
  --endpoint https://api.example.com/v1/chat/completions \
  --model example-model --api-key-env REVIEW_API_KEY

It exits with 1 for a rejected revision. The page index plays no part in it.

The three commands send the texts you select to the model you name and to nothing else. The output of review and trim is advisory and never fails a build. See review in context, trim, and compare.

Reports and gate

One analysis can write all five reports:

bin/unswell check README.md docs/ \
  --report text:- \
  --report json:artifacts/unswell.json \
  --report sarif:artifacts/unswell.sarif \
  --report html:artifacts/unswell.html \
  --report markdown:artifacts/unswell.md

Add --include-source to retain snippets and highlighted paragraphs in saved reports. Source text is omitted by default. Reports can contain internal prose; choose artifact access accordingly.

bin/unswell report artifacts/unswell.json --format html --output artifacts/review.html
bin/unswell explain README.md --at 3:1
bin/unswell rules list
bin/unswell rules show repetition.near-sentence
bin/unswell rules test
bin/unswell config validate
bin/unswell config explain --file README.md
bin/unswell doctor

Exit codes: 0 for a complete pass, 1 for a complete policy failure, 2 for operational errors or incomplete analysis, and 130 for cancellation. An empty scan fails unless explicitly allowed. --no-gate permits advisory checks while preserving operational errors. Display filtering never changes the gate. A rule that exhausts its candidate budget abstains on that document with the reason budget_exhausted; the report lists the abstention and the other rules' findings stay.

The index uses points from 0 to 100. Every local score includes its contributing rules, activations, deduplication decisions and caps. A clean paragraph cannot reduce another paragraph's score. The tool makes no authorship or factuality claim.

Input and configuration

Inputs include plain text, Markdown/GFM, MDX, and comments and string literals in Go, JavaScript, TypeScript/TSX, Python, Rust, Java, C/C++, C#, YAML, Bash/POSIX sh, Zsh's common shell syntax, Fish and PowerShell. Markdown uses block and inline grammars. See formats and extraction limits for extensions and dialect details. For stdin, provide --filename or a supported --format. Coordinates use original UTF-8 byte ranges and one-based Unicode code point columns.

Recursive Git scans select tracked files. Explicit files may be untracked and bypass recursive include/exclude patterns. Outside Git, directories use a bounded filesystem walk. Inputs must remain within the current project root. Symlinks are not followed during discovery. Unsupported explicit formats fail.

The CLI reads the nearest .unswell.yaml up to the project root, or the exact file named by --config. It reads no home configuration. Local inheritance, ordered file overrides, and shared term dictionaries use one configuration plan in the CLI, library, and MCP server. Add team policies with custom rule packs, load them through --ruleset or inline rule_sets, and execute their examples with unswell rules test. Select checked contexts globally or per language with the extraction policy. Use reasoned exceptions for intentional examples, dictionaries and other text that should remain outside an editorial check. Exceptions can select paths, languages, comments or strings, and named string owners.

AI assistants can call the same engine through the separate MCP server. Its tools check supplied drafts and expose the fixed policy. make dogfood-mcp verifies repository checks through a real MCP client and subprocess. Separate CLI and MCP containers support Linux AMD64 and ARM64, including repository checks through both interfaces in CI. Both images are public in GHCR, and Docker Hub mirrors the release with matching digests.

version: 1
extends: [builtin:technical-v1]
language: en
calibration:
  model: none
rules:
  filler.announced-importance:
    enabled: true
    severity: error
    gate: forbid
  policy.banned-phrases:
    parameters:
      phrases: ["leverage synergies"]
      positions: [any]

A phrase position is any, sentence-start, document-start, prose-start (the first sentence after the leading headings), or document-end.

Available profiles are technical, strict, minimal, business, reference and custom; each also accepts a -v1 suffix. Unknown settings, duplicate keys, unsupported rule parameters and unavailable model names fail before analysis.

Go library

engine, err := unswell.New(unswell.Options{})
if err != nil {
    return err
}
result, err := engine.Analyze(ctx, document.Source{
    Name: "draft.md",
    Format: document.Markdown,
    Bytes: draft,
})
if err != nil {
    return err
}
return report.Write(writer, "json", result, report.Options{})

The external consumer implements a custom rule using only public packages. See the API policy for supported imports and ownership contracts. Go rules are trusted code and must support concurrent calls.

Development and limits

Unswell checks its own README, documentation, Go code and Bash scripts with the committed strict repository policy. Run make dogfood, or make check for all checks. CI runs the same gate and uploads all five report formats. A negative CLI probe must fail with exit code 1, proving the policy is active. Third-party licenses and test-data documents are outside this editorial policy; inline code and fenced examples use the normal extractor's protected boundaries. The policy excludes deliberate fixture strings and catalog data with recorded reasons. Comments and runtime strings remain checked. Bash also passes syntax, ShellCheck and shfmt checks, including negative probes for each gate.

Reasoned source suppressions permit individual rule findings while retaining raw evidence, scores, and an audit record across CLI and MCP. Explicit baselines accept existing debt without removing findings or changing scores. The new gate checks new findings and changed scored units. Committed changed-unit checks compare merge-base to HEAD with clean-source verification and retain whole-document repetition context. Use trusted policy to apply merge-base rules and baseline while auditing candidate policy changes and new source permissions. The public Go analysis adapter and its prototype vet driver check comments and strings through the same engine. The contextual editorial catalog adds opt-in filler, hype, and rhetorical-pattern checks with explicit evidence and technical counterexamples. The repetition catalog adds bounded phrase, POS-template, paragraph, heading, and summary comparisons with measured overlap evidence. The surface catalog adds opt-in syntax, readability, and formatting measurements with explicit counting protocols and protected boundaries. The research pipeline that fits and evaluates those models is a separate developer workflow; see training and evaluation. The active roadmap improves contextual diagnostics and checks them on real technical prose. Human qualification and calibrated revision probabilities remain deferred under ADR 0037, with their original requirements preserved. Neither capability is claimed by this alpha.

Code, URLs, front matter, directives and quoted Markdown blocks are excluded from ordinary prose checks. POS-based chunks are surface candidates, not grammatical dependencies. Lexical overlap is not a claim of semantic equivalence. Thresholds are initial editorial policy, not measured precision or confidence.

Code is MIT licensed. Runtime dependencies and the embedded English model have their own notices. No scan downloads rules, sends prose to a service or requires Python, Node.js, a server, an API key or cgo.

Documentation

Overview

Package unswell analyzes English prose offline with an extensible rule engine. It never discovers files, reads environment variables, prints, exits, or uses the network. Style findings are results; operational failures are Go errors. A rule that exhausts its own candidate budget on one document abstains there instead: the result records the abstention and the other rules' findings stay.

Index

Constants

View Source
const (
	AIReadingMarks  = "marks"
	AIReadingRhythm = "rhythm"
)

Names of the two readings.

View Source
const (
	AISoundingLow      = "low"
	AISoundingElevated = "elevated"
	AISoundingHigh     = "high"
)

Levels of an AISounding index.

View Source
const FeatureBlockBindingContract = "mapped-block-v1"

FeatureBlockBindingContract identifies complete mapped-text block bindings.

View Source
const FeatureCollectionVersion = "unswell-feature-collection-v1"

FeatureCollectionVersion identifies the optional saved collection structure.

View Source
const PreparedFeatureCollectionVersion = "unswell-prepared-feature-collection-v1"

PreparedFeatureCollectionVersion identifies optional prepared measurements.

View Source
const SchemaVersion = "1.0.0-alpha.1"

SchemaVersion versions the saved result contract independently of CLI flags.

View Source
const Version = "0.1.0-alpha.17"

Version is the release identity used in canonical results.

Variables

View Source
var BuildCommit = "development"

BuildCommit identifies release builds. Set it with -ldflags at build time. Applications should not change this value while engines are running.

Functions

func AILevel

func AILevel(index int) string

AILevel returns the level of an index, or of one of its readings.

func RemoveAIMarks

func RemoveAIMarks(source []byte, marks []AIMark) ([]byte, int)

RemoveAIMarks applies the fixes of the marks to the source they were found in and returns the new source and the number of fixes applied. A fix that overlaps an earlier one, or that does not fit the source, is left out; a second analysis of the result finds what remains. Marks without a fix stay.

Types

type AIMark

type AIMark struct {
	// Signal names the signal that the mark counts for.
	Signal string `json:"signal"`
	// Span is the mark in the original bytes, and Start is its position.
	Span  document.Span     `json:"span"`
	Start document.Position `json:"start"`
	// Word is the matched word of the fixed list, in lower case, for a
	// stock-word mark. Other marks carry no text.
	Word string `json:"word,omitempty"`
	// Sentences is the length of a run of sentences, for a mark that is one.
	Sentences int `json:"sentences,omitempty"`
	// Fix is the replacement that removes the mark without a rewrite of the
	// sentence. A stock word has none: only a person or a model can say what
	// the thing does. A dash between two numbers has none either.
	Fix *AIMarkFix `json:"fix,omitempty"`
	// contains filtered or unexported fields
}

AIMark is one counted mark and its place in the source.

type AIMarkFix

type AIMarkFix struct {
	Span document.Span `json:"span"`
	Text string        `json:"text"`
}

AIMarkFix replaces a range of the original bytes with a text. The range can reach past the mark to the spaces around it.

type AIReadings

type AIReadings struct {
	Marks  int `json:"marks"`
	Rhythm int `json:"rhythm"`
}

AIReadings are the two readings of a page, each 0 to 100. Marks reads the characters, words, and labels that people remove when they clean a model's text. Rhythm reads the runs of sentences that look alike, which is what remains in a page that a model wrote under a style guide.

type AISignal

type AISignal struct {
	Name string `json:"name"`
	// Reading names the reading that the signal belongs to: "marks" or
	// "rhythm".
	Reading string `json:"reading"`
	Count   int    `json:"count"`
	// Sentences is set for a signal whose marks are runs of sentences: the
	// sentences in all its runs. Count is then the number of runs.
	Sentences int `json:"sentences,omitempty"`
	// PerThousand is the rate that the index weighs: the marks per 1,000
	// prose words, or for a signal of runs the sentences in its runs.
	PerThousand float64 `json:"per_thousand_words"`
	// Advice says how to remove a mark of this signal.
	Advice string `json:"advice"`
}

AISignal is one counted mark of an AISounding index.

type AISounding

type AISounding struct {
	// Status is "available", or says why there is no index: "too_short" for a
	// page with fewer prose words than the measurement covers.
	Status string `json:"status"`
	// Index is 0 to 100 and meaningful only when Status is "available". It
	// is the higher of the two readings.
	Index int `json:"index"`
	// Readings are the two readings of the page that the index is the higher
	// of.
	Readings *AIReadings `json:"readings,omitempty"`
	// Level is "low", "elevated", or "high". At most one in twenty pages that
	// people wrote before language models reach "high".
	Level string `json:"level,omitempty"`
	// Words is the number of prose words the rates are taken over.
	Words int `json:"prose_words"`
	// Signals lists every signal with its count, in a fixed order.
	Signals []AISignal `json:"signals,omitempty"`
	// Marks lists the counted marks in source order, so that a person or an
	// assistant can go to each one. A page with more than 200 marks lists the
	// first 200; Signals keep the full counts.
	Marks []AIMark `json:"marks,omitempty"`
}

AISounding is a page-level index of marks that people remove when they clean AI-sounding wording out of documentation. It is an index from 0 to 100, never a probability, and it says nothing about who wrote the page. ADR 0059 records the signals, the weights, and what the index was measured against.

func (AISounding) Listed

func (index AISounding) Listed() ([]AIMark, int)

Listed returns the marks that a report lists and the number of marks they stand for. A reading at the level low has too few marks to tell the page from one that a person wrote, so its marks are left out: a few plain sentences in a row are ordinary, and so is one em dash on a long page. The number counts the marks of the listed readings past the bound of Marks too.

func (AISounding) Reading

func (index AISounding) Reading(name string) int

Reading returns one reading of the page by its name, or 0 for a page without an index.

type Assessment

type Assessment struct {
	ChangeState            string         `json:"change_state,omitempty"`
	ChangeFingerprint      string         `json:"change_fingerprint,omitempty"`
	BaselineFingerprint    string         `json:"baseline_fingerprint,omitempty"`
	BaselineState          string         `json:"baseline_state,omitempty"`
	Path                   string         `json:"path"`
	Scope                  string         `json:"scope"`
	UnitID                 int            `json:"unit_id"`
	Span                   document.Span  `json:"span"`
	Words                  int            `json:"words"`
	SlopScore              float64        `json:"slop_score"`
	EffectiveSlopScore     float64        `json:"effective_slop_score"`
	SlopProbability        *float64       `json:"slop_probability"`
	ProbabilityStatus      string         `json:"probability_status"`
	ProbabilityDetail      string         `json:"probability_detail,omitempty"`
	OriginEstimate         *float64       `json:"origin_estimate,omitempty"`
	OriginStatus           string         `json:"origin_status,omitempty"`
	OriginDetail           string         `json:"origin_detail,omitempty"`
	Status                 string         `json:"status"`
	Contributions          []Contribution `json:"contributions"`
	EffectiveContributions []Contribution `json:"effective_contributions,omitempty"`
}

Assessment is a local, nondilutable index. A revision probability is present only when a compatible pack qualifies the unit; ProbabilityStatus always explains an unavailable estimate, and an index is never a probability.

type ChangeOptions

type ChangeOptions struct {
	TrustedPolicy bool
	PolicyChanges []PolicyChange
}

ChangeOptions selects comparison against a caller-declared trusted engine policy. PolicyChanges describe external resource differences; the engine never loads them.

type ChangeSelection

type ChangeSelection struct {
	Version       string            `json:"version"`
	Complete      bool              `json:"complete"`
	Documents     []ChangedDocument `json:"documents"`
	ComparedUnits int               `json:"compared_units"`
	SelectedUnits int               `json:"selected_units"`
}

ChangeSelection records structural comparison without granting baseline debt. Complete describes in-memory analysis; Git provenance belongs to the caller.

type ChangedDocument

type ChangedDocument struct {
	Path       string `json:"path"`
	BeforeHash string `json:"before_hash,omitempty"`
	AfterHash  string `json:"after_hash,omitempty"`
	Status     string `json:"status"`
	FullReason string `json:"full_reason,omitempty"`
}

ChangedDocument identifies both versions and any reason for selecting a full file. A deleted document has no AfterHash and contributes no current source locations.

type Contribution

type Contribution struct {
	FindingID  string  `json:"finding_id"`
	RuleID     string  `json:"rule_id"`
	Group      string  `json:"group"`
	Weight     int     `json:"weight"`
	Activation int     `json:"activation"`
	Raw        float64 `json:"raw"`
	Effective  float64 `json:"effective"`
	RuleCap    int     `json:"rule_cap"`
	GroupCap   int     `json:"group_cap"`
	Reason     string  `json:"reason"`
}

Contribution explains fixed-point scoring and the effects of caps and deduplication.

type DocumentResult

type DocumentResult struct {
	GateMode         string               `json:"gate_mode,omitempty"`
	Name             string               `json:"name"`
	Format           document.Format      `json:"format"`
	SourceHash       string               `json:"source_hash"`
	ConfigHash       string               `json:"config_hash,omitempty"`
	AppliedOverrides []string             `json:"applied_overrides,omitempty"`
	Bytes            int                  `json:"bytes"`
	ProseWords       int                  `json:"prose_words"`
	Blocks           int                  `json:"blocks"`
	Sentences        int                  `json:"sentences"`
	Excluded         []document.Exclusion `json:"excluded"`
	Source           string               `json:"source,omitempty"`
	Maximum          float64              `json:"paragraph_maximum"`
	Median           float64              `json:"paragraph_median"`
	P90              float64              `json:"paragraph_p90"`
	FlaggedFraction  float64              `json:"eligible_paragraph_flagged_fraction"`
	// AISounding is the page-level index for a Markdown page and absent for
	// other documents.
	AISounding *AISounding `json:"ai_sounding,omitempty"`
}

DocumentResult summarizes one source without including its prose by default.

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine is immutable after construction and supports concurrent calls. Custom rule and NLP implementations must uphold their documented concurrency contract.

func New

func New(options Options) (*Engine, error)

New validates the entire policy and required capabilities before reading prose.

func (*Engine) Analyze

func (e *Engine) Analyze(ctx context.Context, src document.Source) (Result, error)

Analyze runs the same pipeline used by AnalyzeAll on one in-memory source.

func (*Engine) AnalyzeAll

func (e *Engine) AnalyzeAll(ctx context.Context, sources []document.Source) (RunResult, error)

AnalyzeAll returns a deterministic, path-sorted result regardless of worker count or source order. It returns partial evidence and an error on incompleteness.

func (*Engine) AnalyzeChanged

func (e *Engine) AnalyzeChanged(ctx context.Context, before, after []document.Source) (RunResult, error)

AnalyzeChanged analyzes complete source versions under this engine's fixed policy. Only changed or ambiguous findings and scored units affect the gate. All current evidence remains visible. New paths are selected in full; identical duplicates cannot silently inherit an unchanged state. Callers own revision discovery, clean-source verification, and policy selection.

func (*Engine) AnalyzeChangedWithOptions

func (e *Engine) AnalyzeChangedWithOptions(
	ctx context.Context, before, after []document.Source, options ChangeOptions,
) (RunResult, error)

AnalyzeChangedWithOptions optionally uses this engine's policy and the before sources as trusted inputs. Only matching base permissions can suppress current findings. Resource changes trigger full selection under the trusted policy; they do not apply the candidate policy or automatically accept new baseline debt. Callers own resource loading, revision verification, and the trust assertion.

func (*Engine) Catalog

func (e *Engine) Catalog() []rule.Descriptor

Catalog returns owned descriptors for the complete registered rule catalog.

func (*Engine) FeatureIDs

func (e *Engine) FeatureIDs() []string

FeatureIDs returns owned, canonical IDs explicitly requested for collection.

func (*Engine) OriginModelIdentity

func (e *Engine) OriginModelIdentity() *ProbabilityModel

OriginModelIdentity returns the origin pack's declarations, or nil. An origin estimate is a separate experimental channel that no gate consults.

func (*Engine) PolicyForFile

func (e *Engine) PolicyForFile(name string) (config.Policy, error)

PolicyForFile returns an owned effective policy for a project-relative logical filename. An empty filename returns the base discovery and run-wide policy.

func (*Engine) PreparedFeatureIDs

func (e *Engine) PreparedFeatureIDs() []string

PreparedFeatureIDs returns owned, canonical IDs selected for prepared targets.

func (*Engine) PreparedUnitKinds

func (e *Engine) PreparedUnitKinds() []string

PreparedUnitKinds returns owned, canonical kinds selected for preparation.

func (*Engine) ProbabilityModelIdentity

func (e *Engine) ProbabilityModelIdentity() *ProbabilityModel

ProbabilityModelIdentity returns the revision pack's declarations, or nil.

func (*Engine) ResolvedPolicy

func (e *Engine) ResolvedPolicy(name string) (config.Policy, error)

ResolvedPolicy returns the shared policy resolved for a source name. It costs one lookup per call after the first file of its override combination, where PolicyForFile copies the whole policy. The result is read-only: the engine and every other caller of the same combination share its maps.

type FeatureBlockBinding

type FeatureBlockBinding struct {
	Contract        string          `json:"contract"`
	TextSHA256      string          `json:"text_sha256"`
	Segments        []document.Span `json:"segments"`
	TrimmedSHA256   string          `json:"trimmed_sha256"`
	TrimmedSegments []document.Span `json:"trimmed_segments"`
}

FeatureBlockBinding identifies every mapped byte of an original extracted block. TextSHA256 includes whitespace, punctuation, and protected separators. Segments cover mapped text, including the source ranges of protected placeholders, not only counted tokens. Missing bindings cannot be inferred from token segments. Trimmed fields apply the same outer-whitespace trim as corpus unit preparation; original fields retain the actual input used by rules.

type FeatureCollection

type FeatureCollection struct {
	ActivationContract string          `json:"activation_contract,omitempty"`
	Version            string          `json:"version"`
	BlockContract      string          `json:"block_contract"`
	Requested          []string        `json:"requested"`
	Sources            []FeatureSource `json:"sources"`
}

FeatureCollection contains requested descriptive measurements, not model scores. The enclosing RunResult completion state applies: partial runs are not complete training or inference inputs. Values never include normalized words or keys.

type FeatureSource

type FeatureSource struct {
	RulesetHash    string           `json:"ruleset_hash,omitempty"`
	Path           string           `json:"path"`
	SourceHash     string           `json:"source_hash"`
	PolicyHash     string           `json:"policy_hash"`
	VocabularyHash string           `json:"vocabulary_hash"`
	Preprocessing  string           `json:"preprocessing"`
	NLP            nlp.Identity     `json:"nlp"`
	Capabilities   []nlp.Capability `json:"capabilities"`
	Units          []FeatureUnit    `json:"units"`
}

FeatureSource identifies the actual representation and policy used for a source.

type FeatureUnit

type FeatureUnit struct {
	Excluded    bool                 `json:"excluded,omitempty"`
	Binding     *FeatureBlockBinding `json:"binding,omitempty"`
	Scope       string               `json:"scope"`
	UnitID      int                  `json:"unit_id"`
	Kind        string               `json:"kind"`
	ContextHash string               `json:"context_hash"`
	Span        document.Span        `json:"span"`
	Segments    []document.Span      `json:"segments"`
	InputHash   string               `json:"input_hash,omitempty"`
	Values      []feature.Value      `json:"values"`
}

FeatureUnit records one original block and its requested numeric values. Segments contain counted source tokens, without filling protected gaps. An unsupported measurement kind has no input hash or counted segments. Rule activations retain the source, context, and rule identities instead. Excluded units retain their binding but have no counted segments or numbers. Their source exclusion reason remains in DocumentResult.Excluded.

type Finding

type Finding struct {
	ChangeState         string        `json:"change_state,omitempty"`
	ChangeFingerprint   string        `json:"change_fingerprint,omitempty"`
	ID                  string        `json:"id"`
	RuleID              string        `json:"rule_id"`
	RuleVersion         string        `json:"rule_version"`
	Severity            string        `json:"severity"`
	Gate                string        `json:"gate"`
	Group               string        `json:"group"`
	Scope               string        `json:"scope"`
	Message             string        `json:"message"`
	Primary             Location      `json:"primary"`
	Related             []Location    `json:"related"`
	Evidence            rule.Evidence `json:"evidence"`
	Fingerprint         string        `json:"fingerprint"`
	Suppressed          bool          `json:"suppressed"`
	SuppressionIDs      []string      `json:"suppression_ids,omitempty"`
	BaselineState       string        `json:"baseline_state"`
	BaselineFingerprint string        `json:"baseline_fingerprint,omitempty"`
	Derived             bool          `json:"derived"`
}

Finding is a rule activation enriched with policy, identity, and source locations.

type GateDecision

type GateDecision struct {
	Unchanged []GateReason `json:"unchanged,omitempty"`
	Passed    bool         `json:"passed"`
	Reasons   []GateReason `json:"reasons"`
	Accepted  []GateReason `json:"accepted,omitempty"`
}

GateDecision reports policy separately from operational completion.

type GateReason

type GateReason struct {
	Code      string `json:"code"`
	Path      string `json:"path"`
	FindingID string `json:"finding_id"`
	Message   string `json:"message"`
}

GateReason identifies the exact local condition behind a policy failure.

type GitSelection

type GitSelection struct {
	RequestedRef string `json:"requested_ref"`
	BaseCommit   string `json:"base_commit"`
	HeadCommit   string `json:"head_commit"`
	Clean        bool   `json:"clean"`
}

GitSelection records application-verified committed source provenance. The engine never invokes Git. Clean describes the selected inputs, not every file.

type Location

type Location struct {
	Path     string            `json:"path"`
	Span     document.Span     `json:"span"`
	Start    document.Position `json:"start"`
	End      document.Position `json:"end"`
	Segments []document.Span   `json:"segments"`
	Snippet  string            `json:"snippet,omitempty"`
}

Location contains original byte spans and Unicode code point positions.

type Manifest

type Manifest struct {
	Git             *GitSelection             `json:"git,omitempty"`
	GateMode        string                    `json:"gate_mode,omitempty"`
	ToolVersion     string                    `json:"tool_version"`
	ToolCommit      string                    `json:"tool_commit"`
	ConfigHash      string                    `json:"config_hash"`
	ConfigIdentity  string                    `json:"config_identity,omitempty"`
	ConfigSources   []config.SourceIdentity   `json:"config_sources,omitempty"`
	ConfigOverrides []config.OverrideIdentity `json:"config_overrides,omitempty"`
	RulesetHash     string                    `json:"ruleset_hash"`
	ScoringProfile  string                    `json:"scoring_profile"`
	FeatureContract string                    `json:"feature_contract"`
	Probability     *ProbabilityModel         `json:"probability,omitempty"`
	Origin          *ProbabilityModel         `json:"origin,omitempty"`
	NLP             nlp.Identity              `json:"nlp"`
	Rules           []rule.Descriptor         `json:"rules"`
	SelectionMode   string                    `json:"selection_mode"`
	Complete        bool                      `json:"complete"`
	NoGate          bool                      `json:"no_gate"`
	IncludeSource   bool                      `json:"include_source"`
	SkippedRules    []string                  `json:"skipped_rules"`
	// AbstainedRules names each rule that abstained on at least one document,
	// sorted and unique. Such a rule did not cover every document of the run;
	// RunResult.Abstentions lists the documents and reasons.
	AbstainedRules []string `json:"abstained_rules,omitempty"`
}

Manifest records the exact execution policy and feature versions, without time.

type Options

type Options struct {
	// PreparedFeatures and PreparedKinds select descriptive target measurements.
	// Both sets are required when either is nonempty; collection is off by default.
	PreparedFeatures []string
	PreparedKinds    []string
	// Features selects block measurements for the result. IDs form a set; unknown
	// or repeated IDs are errors. Nil or empty disables result collection.
	Features []string
	// Model contains one explicitly selected revision-probability pack. It
	// requires calibration.model: pack and is rejected without it.
	Model []byte
	// OriginModel contains one explicitly selected origin pack. It requires
	// origin.model: pack, is rejected without it, and never affects the gate.
	OriginModel []byte
	// Baseline contains an explicitly selected artifact; nil disables comparison.
	Baseline []byte
	// CollectBaseline requests a validated snapshot for explicit create or update.
	CollectBaseline bool
	// GateMode overrides configuration with all or new; empty follows the policy.
	GateMode      string
	Config        []byte
	ConfigBundle  *config.Bundle
	Rules         []rule.Rule
	RuleSets      [][]byte
	NLP           nlp.Provider
	Jobs          int
	IncludeSource bool
	AllowEmpty    bool
	NoGate        bool
}

Options injects all inputs. A nil Rules slice selects the builtin catalog; a nonnil slice replaces that catalog. RuleSets adds declarative YAML packs to the registry. Config and RuleSets contain bytes, never filenames.

type OriginalPreparedUnit

type OriginalPreparedUnit struct {
	Target  []PreparedSourceSegment
	Context []PreparedSourceSegment
}

OriginalPreparedUnit contains the original source segments for one prepared target and its enclosing prose. Separate segments must not be joined across gaps: those gaps can contain protected code or other excluded source text.

type PolicyChange

type PolicyChange struct {
	Path       string `json:"path"`
	Kind       string `json:"kind"`
	BeforeHash string `json:"before_hash,omitempty"`
	AfterHash  string `json:"after_hash,omitempty"`
}

PolicyChange identifies a resource or structural permission change. Hashes are SHA-256 identities; an empty side means that the resource or permission is absent.

type PolicyComparison

type PolicyComparison struct {
	Version  string         `json:"version"`
	Complete bool           `json:"complete"`
	FullScan bool           `json:"full_scan"`
	Changes  []PolicyChange `json:"changes"`
}

PolicyComparison audits trusted-policy use. FullScan selects all current units for resource changes; permission changes can select individual complete files. This record proves neither organizational approval nor the checker's integrity.

type PreparedFeatureCollection

type PreparedFeatureCollection struct {
	Version         string                  `json:"version"`
	FeatureContract string                  `json:"feature_contract"`
	UnitContract    string                  `json:"unit_contract"`
	Requested       []string                `json:"requested"`
	Kinds           []string                `json:"kinds"`
	Sources         []PreparedFeatureSource `json:"sources"`
}

PreparedFeatureCollection contains descriptive values, never model scores. The enclosing run's completion state applies to every collection consumer.

type PreparedFeatureSource

type PreparedFeatureSource struct {
	TargetCount          int                   `json:"target_count"`
	Path                 string                `json:"path"`
	SourceHash           string                `json:"source_hash"`
	PolicyHash           string                `json:"policy_hash"`
	VocabularyHash       string                `json:"vocabulary_hash"`
	ExtractionPolicyHash string                `json:"extraction_policy_hash"`
	PreparationHash      string                `json:"preparation_hash"`
	IncludeQuotes        bool                  `json:"include_quotes"`
	IncludeStructure     bool                  `json:"include_structure"`
	NLP                  nlp.Identity          `json:"nlp"`
	Capabilities         []nlp.Capability      `json:"capabilities"`
	Units                []PreparedFeatureUnit `json:"units"`
}

PreparedFeatureSource binds actual source, policy, extraction, and NLP inputs. PreparationHash covers UnitContract, extraction policy, and both switches. PolicyHash and VocabularyHash conservatively bind the entire effective policy.

func (PreparedFeatureSource) OriginalUnit

func (s PreparedFeatureSource) OriginalUnit(source document.Source, index int) (OriginalPreparedUnit, error)

OriginalUnit returns exact source segments for one prepared target. It checks the source name and SHA-256, the unit contract, UTF-8 boundaries, and the target/context relationship before exposing any text. It does not re-extract prose or reconstruct the normalized text used by the NLP provider.

type PreparedFeatureUnit

type PreparedFeatureUnit struct {
	Binding   nlp.UnitBinding `json:"binding"`
	InputHash string          `json:"input_hash"`
	Segments  []document.Span `json:"segments"`
	Values    []feature.Value `json:"values"`
}

PreparedFeatureUnit keeps complete target/context mappings separate from the counted token segments. Its parent block ID is not a unique target identifier.

type PreparedSourceSegment

type PreparedSourceSegment struct {
	Span document.Span
	Text string
}

PreparedSourceSegment is an exact, owned slice of the original UTF-8 source. Its span uses the original source's half-open byte offsets. Text may contain markup or escapes that differ from the extracted text used for measurements.

type ProbabilityModel

type ProbabilityModel struct {
	PackID         string `json:"pack_id"`
	SHA256         string `json:"sha256"`
	Version        string `json:"version"`
	Task           string `json:"task"`
	Rubric         string `json:"rubric"`
	Kind           string `json:"kind"`
	DeclaredStatus string `json:"declared_status"`
	HumanCorpus    string `json:"human_corpus"`
	Evaluation     string `json:"evaluation,omitempty"`
	MinWords       int    `json:"min_words"`
}

ProbabilityModel repeats the declarations of the pack behind reported estimates. The engine checks compatibility and consistency, never the truth of an acceptance, corpus, or evaluation statement.

type Result

type Result = RunResult

Result is the same report contract restricted to one source by Analyze.

type RuleAbstention

type RuleAbstention struct {
	Path        string `json:"path"`
	RuleID      string `json:"rule_id"`
	RuleVersion string `json:"rule_version"`
	Reason      string `json:"reason"`
	Detail      string `json:"detail,omitempty"`
}

RuleAbstention records one rule that declined to judge one document for a declared reason, such as an exhausted candidate budget. The document stays measured by every other rule and the run stays complete; the abstaining rule contributes no findings and no activation values for that document.

type RunError

type RunError struct {
	Path    string `json:"path"`
	Message string `json:"message"`
}

RunError records an operational failure in a partial result.

type RunResult

type RunResult struct {
	PreparedFeatures *PreparedFeatureCollection `json:"prepared_features,omitempty"`
	Features         *FeatureCollection         `json:"features,omitempty"`
	PolicyComparison *PolicyComparison          `json:"policy_comparison,omitempty"`
	Changes          *ChangeSelection           `json:"changes,omitempty"`
	BaselineSnapshot *baseline.Snapshot         `json:"baseline_snapshot,omitempty"`
	Baseline         *baseline.Comparison       `json:"baseline,omitempty"`
	SchemaVersion    string                     `json:"schema_version"`
	Status           string                     `json:"status"`
	Manifest         Manifest                   `json:"manifest"`
	Documents        []DocumentResult           `json:"documents"`
	Findings         []Finding                  `json:"findings"`
	Assessments      []Assessment               `json:"assessments"`
	Gate             GateDecision               `json:"gate"`
	Errors           []RunError                 `json:"errors"`
	Suppressions     []Suppression              `json:"suppressions,omitempty"`
	// Abstentions lists each rule that declined one document, sorted by path
	// and rule ID. Manifest.AbstainedRules names the affected rules once.
	Abstentions []RuleAbstention `json:"abstentions,omitempty"`
}

RunResult is the immutable input shared by every reporter. Returned slices are owned by the caller and do not alias the engine.

type Suppression

type Suppression struct {
	TrustState       string              `json:"trust_state,omitempty"`
	TrustFingerprint string              `json:"trust_fingerprint,omitempty"`
	ID               string              `json:"id"`
	Kind             string              `json:"kind"`
	RuleIDs          []string            `json:"rule_ids"`
	Reason           string              `json:"reason"`
	Directive        Location            `json:"directive"`
	End              *Location           `json:"end,omitempty"`
	Targets          []SuppressionTarget `json:"targets"`
	FindingIDs       []string            `json:"finding_ids"`
	UsedRules        []string            `json:"used_rules"`
	Status           string              `json:"status"`
}

Suppression records an explicit permission and links it to retained raw findings. A partially used or unused permission is an error when reject_unused is true.

type SuppressionTarget

type SuppressionTarget struct {
	Scope  string        `json:"scope"`
	UnitID int           `json:"unit_id"`
	Span   document.Span `json:"span"`
}

SuppressionTarget identifies one complete unit permitted by a source directive.

Directories

Path Synopsis
Package baseline records explicit acceptance of existing editorial debt.
Package baseline records explicit acceptance of existing editorial debt.
Package builtin constructs the versioned alpha rule catalog without init hooks.
Package builtin constructs the versioned alpha rule catalog without init hooks.
cmd
docscatalog command
Command docscatalog renders the builtin rule catalog as the JSON data file that the documentation site's rules page reads.
Command docscatalog renders the builtin rule catalog as the JSON data file that the documentation site's rules page reads.
genschema command
Command genschema derives the saved-result JSON schema from the public types.
Command genschema derives the saved-result JSON schema from the public types.
repocheck command
Command repocheck validates the repository's source and CI contracts.
Command repocheck validates the repository's source and CI contracts.
unswell command
Command unswell checks English prose against a local editorial policy.
Command unswell checks English prose against a local editorial policy.
Package config compiles strict, offline YAML policy against an explicit catalog.
Package config compiles strict, offline YAML policy against an explicit catalog.
Package document defines source coordinates and the neutral prose model.
Package document defines source coordinates and the neutral prose model.
Package extract provides source-preserving prose extraction from text and syntax trees.
Package extract provides source-preserving prose extraction from text and syntax trees.
Package feature computes versioned descriptive measurements from extracted prose.
Package feature computes versioned descriptive measurements from extracted prose.
goanalysis module
internal
appconfig
Package appconfig loads local policy resources at the CLI and MCP startup boundary.
Package appconfig loads local policy resources at the CLI and MCP startup boundary.
cli
Package cli owns command parsing, filesystem discovery, streams, and exit codes.
Package cli owns command parsing, filesystem discovery, streams, and exit codes.
glyph
Package glyph classifies single characters that rules and page-level measurements treat alike.
Package glyph classifies single characters that rules and page-level measurements treat alike.
mapping
Package mapping builds source-preserving normalized prose.
Package mapping builds source-preserving normalized prose.
pathglob
Package pathglob compiles bounded project-relative path patterns.
Package pathglob compiles bounded project-relative path patterns.
repopolicy
Package repopolicy checks repository contracts against the actual source tree.
Package repopolicy checks repository contracts against the actual source tree.
suppress
Package suppress binds explicit source permissions to complete prose units.
Package suppress binds explicit source permissions to complete prose units.
terms
Package terms matches explicit vocabulary at complete token boundaries.
Package terms matches explicit vocabulary at complete token boundaries.
textutil
Package textutil shares text boundaries without changing source mappings.
Package textutil shares text boundaries without changing source mappings.
Package model trains numerical classifiers and separate calibration mappings in Go.
Package model trains numerical classifiers and separate calibration mappings in Go.
nlp
Package nlp defines backend capabilities and prepares mapped prose units.
Package nlp defines backend capabilities and prepares mapped prose units.
english
Package english adapts Prose's tokenizer, Punkt segmenter, and POS tagger.
Package english adapts Prose's tokenizer, Punkt segmenter, and POS tagger.
Package probability decodes explicitly supplied editorial revision-probability packs and decides per-unit applicability before any estimate is reported.
Package probability decodes explicitly supplied editorial revision-probability packs and decides per-unit applicability before any estimate is reported.
Package report renders a completed result without analyzing or reading sources.
Package report renders a completed result without analyzing or reading sources.
Package review prepares a document for review in context and binds a reviewer's flags to original source bytes.
Package review prepares a document for review in context and binds a reviewer's flags to original source bytes.
Package rule defines explicit rule registration, evidence, and configuration.
Package rule defines explicit rule registration, evidence, and configuration.
Package ruleset compiles bounded, declarative editorial rules from YAML bytes.
Package ruleset compiles bounded, declarative editorial rules from YAML bytes.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL