policylearn

package
v1.0.217 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 25, 2026 License: MIT Imports: 20 Imported by: 0

Documentation

Overview

Package policylearn is the Security Policy Learning Mode engine (ADR-0025): the Learning Session lifecycle (M1) with schema-versioned, bounded, fail-closed node-local persistence, plus the observation TRANSPORT (M2) — a bounded drop-on-full queue with a single drain goroutine (see observe.go). Aggregation, recommendations, and any API/GUI surface are deliberately absent until M3+.

Design contract (ADR-0025 — every clause is test-pinned):

  • ADVISORY ONLY. Learning has no authority over active policy or any enforcement subsystem. This package's import surface is walled to the standard library + internal/fileutil + internal/obs (root wall test); it cannot import or mutate the policy store, TLS/decryption, PAC, CDR, default-action, or any other enforcement state. Everything it needs from the outside arrives through injected function values on Config.
  • INJECTED CLOCK. The engine never reads the wall clock; Config.Now is required and every time-dependent decision (expiry, stamps, gaps) is a pure function of it — deterministic under test.
  • NODE-LOCAL, OFF EVERY CONFIG SURFACE. Session state persists to one JSON document via fileutil.AtomicWrite (0600): never exported/imported, never on the config-version rollback surface, never CP→DP synced.
  • FAIL-CLOSED PERSISTENCE. A corrupt store is quarantined through the injected seam and the engine starts empty (never half-loaded). A store written by a NEWER schema is left untouched and the engine refuses to write (read-only) — a downgrade can never clobber newer state.
  • BOUNDED. Terminal sessions are pruned FIFO to MaxRetainedSessions; an active session auto-completes at MaxSessionDuration (lazily, on the next engine operation — no timers). The engine's ONLY goroutine is the M2 observation drain, started at construction and stopped deterministically by Close.
  • EXPLICIT OFF STATE. Disabled means this engine is simply never constructed: no file, no goroutine, no allocation (the root singleton stays nil).

Index

Constants

View Source 
const (
	ScopeUnauth    = "s:unauth"
	ScopeGroupless = "s:groupless"
)

Scope-key construction. The "g:"/"s:" prefixes are the collision wall.

View Source 
const (
	DefaultMaxRetainedSessions = 8

	DefaultMaxSessionDuration = 90 * 24 * time.Hour
)

Defaults and clamps for the engine bounds.

View Source 
const (
	RecStateGenerated  = "generated"
	RecStateSuperseded = "superseded"
	RecStateAccepting  = "accepting"
	RecStateAccepted   = "accepted"
	RecStateRejected   = "rejected"
)

Recommendation lifecycle states. Staleness is deliberately COMPUTED, not latched: a recommendation's pins (Baseline + SubjectKeyID) are compared against current identities via the pure StaleReasons helper, so there is no mutation path that could rot or race. Superseded IS latched — it records the historical fact that a later generation replaced this content, which no current-state comparison could recover.

M5B adds the DECISION states. The engine records lifecycle + acceptance INTENT only — translation into a policy rule and every draft interaction happen at the root trust boundary OUTSIDE this package (the wall): 

generated ──BeginAccept──► accepting ──FinalizeAccept──► accepted (terminal)
    │            ▲              │
    │            └──────────────┘ AbortAccept (safe reconcile: rule absent + fences stale)
    └───Reject──► rejected (terminal)

accepting is the durable CROSS-STORE intent: it carries the preallocated TargetRuleID so a crash at any instruction boundary leaves enough state to reconcile deterministically (finalize if the exact rule exists, redo or abort under current fences if it does not). generated→superseded remains the only other latch; regeneration NEVER touches accepting/accepted/rejected.

View Source 
const (
	ConfidenceHigh   = "high"
	ConfidenceMedium = "medium"
	ConfidenceLow    = "low"
)

Confidence levels — a closed three-value set (never a score).

View Source 
const (
	StateLearning  = "learning"
	StateCompleted = "completed"
	StateCancelled = "cancelled"
)

Session states — the minimal explicit machine (ADR-0025 §2): 

(none) ──Start──► Learning ──Stop──────────────► Completed (terminal)
                     │  ──Cancel───────────────► Cancelled (terminal)
                     └──max-duration overrun───► Completed (StoppedBy=system:max-duration)

Terminal states never transition again. "Inactive" is the absence of a Learning session, not a stored state. Degradation (restart gaps) is a recorded fact on the session, not a state.

View Source 
const DefaultActionRuleKey = "_default_"

DefaultActionRuleKey is the RuleHits attribution key recorded when a POLICY decision carried no RuleID (the default action decided). Exported so evidence readers can distinguish default-action attribution from any real rule ULID — a ULID can never equal this sentinel.

View Source 
const (
	// MaxObservationGroups bounds the groups carried per observation
	// (deterministic truncation — a pathological IdP claim set must not become
	// an unbounded copy on the request path).
	MaxObservationGroups = 16
)
View Source 
const PreDispatchRuleKey = "_predispatch_"

PreDispatchRuleKey is the RuleHits attribution key for pre-dispatch blocks (threat feed / blocklist / plugin / global file-extension gate) — decisions made BEFORE the policy evaluator ran (Codex round 26: they carry no RuleID, and folding them under DefaultActionRuleKey claimed the default action decided traffic the evaluator never saw). A ULID can never equal this sentinel either.

View Source 
const SchemaVersion = 9

SchemaVersion is the session-store document schema. Bump on any shape change that an OLDER binary's strict decoder would reject (added fields count — DisallowUnknownFields makes them corruption to an old reader); a store carrying a HIGHER version puts the engine into the read-only fail-closed posture (ErrStoreReadOnly) so a downgraded binary can never clobber newer state. Version history: 1 = M1 sessions; 2 = M3 aggregation (subject_key_id / category_churn / transport / agg / baseline category_epoch); 3 = M4 recommendations (top-level recommendations array + baseline guardrails_hash); 4 = M4.1 recommendation-policy identity (policy / policy_hash embedded per recommendation); 5 = M5A baseline policy_content_hash (canonical access-policy content identity); 6 = M5B decision lifecycle (accepting/accepted/rejected states + target_rule_id / accepted_* / rejected_* / reject_reason); 7 = M5B.1 group-truncation loss accounting (transport groups_truncated); 8 = policy-content churn (session policy_churn + aggregate policy_churn_overflow — Codex round 13: transient A→B→A policy changes latched as they happen); 9 = late-loss invalidation (recommendation late_loss_invalidated — Codex round 29/30: the field is only ever WRITTEN as true, and a version-8 binary strict-decodes unknown fields as CORRUPTION, so persisting it under version 8 made a rollback quarantine the whole store instead of entering the newer-schema read-only posture). Older documents load cleanly on a newer binary (pure field additions, all omitempty); saves always write the current version.

Variables

View Source 
var (
	// ErrActiveSession: StartSession while a session is already Learning
	// (the one-active-session invariant).
	ErrActiveSession = errors.New("policylearn: a learning session is already active")
	// ErrNoActiveSession: Stop/Cancel with no session in Learning.
	ErrNoActiveSession = errors.New("policylearn: no active learning session")
	// ErrStoreReadOnly: the persisted store was written by a newer schema;
	// the engine refuses every mutation so a downgrade can never clobber it.
	ErrStoreReadOnly = errors.New("policylearn: session store schema is newer than this binary (read-only)")
	// ErrEngineClosed: a lifecycle mutation reached an engine whose transport
	// has shut down (Codex fix — the engine closes at shutdown order 67 while
	// the admin UI stops at 70, so a session start can still arrive; arming
	// learningActive on a closed transport would count drops AFTER the final
	// save, loss the store can no longer persist).
	ErrEngineClosed = errors.New("policylearn: engine is closed (shutting down)")
)

Sentinel errors returned by lifecycle operations.

View Source 
var (
	ErrSessionNotFound = errors.New("policylearn: session not found")
	// ErrSessionNotCompleted: only a terminal COMPLETED session generates —
	// never Learning (evidence still moving) and never Cancelled
	// (non-authoritative by definition).
	ErrSessionNotCompleted = errors.New("policylearn: recommendations require a completed session")
	// ErrNoGuardrailBaseline: the session predates guardrail pinning (schema v2
	// store) — without a pinned allowlist identity the evidence cannot be
	// interpreted against ANY guardrail set; fail closed, never guess.
	ErrNoGuardrailBaseline = errors.New("policylearn: session has no pinned guardrails baseline (pre-M4 session) — start a new session")
	// ErrGuardrailsChanged: the current recommendable-category allowlist differs
	// from the one pinned at session start. Never silently regenerate or
	// reinterpret old evidence under new guardrails.
	ErrGuardrailsChanged = errors.New("policylearn: recommendable-category guardrails changed since this session was pinned — evidence is stale for generation; start a new session")
	// ErrSubjectKeyChanged: the pseudonym key rotated mid-session, so the same
	// user may hold two disjoint tokens — distinct-subject counts can OVERSTATE
	// diversity, the one direction evidence must never err in (ADR-0025).
	// Ineligible (the safer disposition vs. a capped-LOW recommendation, whose
	// subject figures would still be presented while known-unsound).
	ErrSubjectKeyChanged = errors.New("policylearn: subject pseudonym key changed mid-session — distinct-subject evidence may double-count; session ineligible for recommendations")

	// M5B decision-lifecycle sentinels — one per state so every invalid
	// transition has an explicit, deterministic refusal.
	ErrRecommendationNotFound   = errors.New("policylearn: recommendation not found")
	ErrRecommendationSuperseded = errors.New("policylearn: recommendation superseded by a later generation — act on the current object")
	ErrRecommendationAccepted   = errors.New("policylearn: recommendation already accepted")
	ErrRecommendationRejected   = errors.New("policylearn: recommendation already rejected")
	ErrRecommendationAccepting  = errors.New("policylearn: recommendation has an unresolved acceptance in progress")

	// ErrAcceptInvalidatedByLateLoss (Codex round 29): the accepting intent's
	// owning session was charged late transport loss after the intent latched —
	// the evidence understates loss, so the finalize latch refuses and the root
	// resolves the intent to superseded (regeneration produces honestly-degraded
	// replacements).
	ErrAcceptInvalidatedByLateLoss = errors.New("policylearn: acceptance invalidated — late transport loss was charged to the learning session after the intent; evidence understates loss")
)

Sentinel errors for generation.

Functions

func CellKey 

func CellKey(scope, category string) string

CellKey builds/splits the aggregate map key.

func GuardrailsHashForCategories 

func GuardrailsHashForCategories(cats []string) string

GuardrailsHashForCategories computes the guardrail identity a config with the given RecommendableCategories would carry — the same canonicalization + hash New applies. Pure; lets root wiring detect a no-op allowlist change without constructing an engine.

func SplitCellKey 

func SplitCellKey(key string) (scope, category string)

SplitCellKey returns (scope, category).

func StaleReasons 

func StaleReasons(r *Recommendation, cur StaleInputs) []string

StaleReasons is the pure staleness computation: the named pin mismatches between a recommendation's generation-time identities and the supplied current ones. Empty ⇒ fresh. Computed on demand — never latched, never persisted — so it cannot rot or race. Policy-identity precedence (M5B §5, the exact contract):

  1. When the caller asserts a CURRENT PolicyContentHash (cur non-empty) AND the recommendation carries a content pin, the CONTENT comparison is the sole policy-identity check — the generation counter is IGNORED, so a generation-only change (same enforced content: counter churn, meta resets, add-then-remove round trips) is NOT stale, while any actual rule/default-action content difference IS (policy_content_changed).
  2. When the caller asserts content but the recommendation has NO content pin (pre-M5A object), it cannot prove content identity: stale, fail closed (policy_content_changed) — generation again irrelevant.
  3. When the caller does NOT assert content (legacy caller), the generation comparison is the defense-in-depth fallback (policy_generation_changed).

Types

type Aggregate 

type Aggregate struct {
	Cells map[string]*Cell `json:"cells,omitempty"` // key: scope + \x1f + category

	// Loss/degradation accounting (evidence can only weaken on overflow).
	CellsDropped        int64 `json:"cells_dropped,omitempty"`       // contributions refused at the cell cap
	SubjectBudgetUsed   int64 `json:"subject_budget_used,omitempty"` // global token budget consumption
	ChurnOverflow       int64 `json:"churn_overflow,omitempty"`
	PolicyChurnOverflow int64 `json:"policy_churn_overflow,omitempty"` // policy-content changes past the bounded list (schema v8)
	SubjectKeyChanged   bool  `json:"subject_key_changed,omitempty"`   // key rotated/lost mid-session — token populations before/after are disjoint
}

Aggregate is the per-session bounded aggregation state.

type AggregateOverview 

type AggregateOverview struct {
	Cells             int   `json:"cells"`
	CellsDropped      int64 `json:"cells_dropped"`
	SubjectBudgetUsed int64 `json:"subject_budget_used"`
	ChurnOverflow     int64 `json:"churn_overflow"`
	SubjectKeyChanged bool  `json:"subject_key_changed"`
}

AggregateOverview is the bounded factual summary of one session's aggregation state, safe for operator surfaces: counts and degradation flags only — no cell contents, no subject tokens, no hosts (M5A API boundary).

type AttributionCount 

type AttributionCount struct {
	Key   string `json:"key"`
	Count int64  `json:"count"`
}

AttributionCount is one bounded attribution row (rule-hit or tier-hit breakdown) copied by value into recommendation evidence.

type Baseline 

type Baseline struct {
	PolicyGeneration int64  `json:"policy_generation"`
	DefaultAction    string `json:"default_action,omitempty"`
	CapturedAt       string `json:"captured_at,omitempty"`     // RFC3339 UTC
	CategoryEpoch    string `json:"category_epoch,omitempty"`  // opaque category-generation identity pinned at Start (M3)
	GuardrailsHash   string `json:"guardrails_hash,omitempty"` // recommendable-category allowlist identity pinned at Start (M4)
	// PolicyContentHash is the canonical CONTENT identity of the running access
	// policy at session start (M5A): unlike PolicyGeneration (a persisted
	// counter — robust to restarts but not to counter resets or same-content
	// re-imports), this is a deterministic hash of the policy content itself,
	// captured by root wiring. Opaque to the engine; compared for staleness.
	PolicyContentHash string `json:"policy_content_hash,omitempty"`
}

Baseline pins the configuration generations a session's future evidence is valid against (ADR-0025 §6). M1 carries the minimal set; M2+ extends it via the same injected capture seam without touching the engine.

type Cell 

type Cell struct {
	// Factual counters, structurally split by evidence direction. Requests =
	// Allowed + Blocked + ThreatBlocked for THIS cell (an observation carrying
	// N groups contributes to N cells — per-cell counts are per-population
	// views, deliberately not summable across cells; session-level totals live
	// on the session transport counters).
	Requests      int64 `json:"requests"`
	Allowed       int64 `json:"allowed"`
	Blocked       int64 `json:"blocked"`        // policy-plane blocks (POLICY_*/FILE_BLOCKED/…)
	ThreatBlocked int64 `json:"threat_blocked"` // pre-dispatch threat/blocklist/plugin blocks

	// Allowed-evidence sets (updated by ALLOWED observations only).
	Subjects        map[string]bool  `json:"subjects,omitempty"` // pseudonymous tokens — never raw subjects
	SubjectOverflow int64            `json:"subject_overflow,omitempty"`
	Days            map[string]bool  `json:"days,omitempty"` // distinct UTC dates (YYYY-MM-DD)
	DayOverflow     int64            `json:"day_overflow,omitempty"`
	FirstSeen       int64            `json:"first_seen,omitempty"` // unix seconds, allowed evidence
	LastSeen        int64            `json:"last_seen,omitempty"`
	TopHosts        map[string]int64 `json:"top_hosts,omitempty"` // admission-bounded representative destinations
	OtherHosts      int64            `json:"other_hosts,omitempty"`

	// Attribution breakdowns (all evidence directions; bounded).
	RuleHits   map[string]int64 `json:"rule_hits,omitempty"` // ruleID → count ("" = default action, keyed "_default_")
	OtherRules int64            `json:"other_rules,omitempty"`
	TierHits   map[string]int64 `json:"tier_hits,omitempty"` // category tier → count
	OtherTiers int64            `json:"other_tiers,omitempty"`
}

Cell is one bounded Group × Category evidence cell. Maps marshal with sorted keys, so persistence is deterministic.

func (*Cell) DistinctSubjects 

func (c *Cell) DistinctSubjects() int

DistinctSubjects returns the exact tokenized count for a cell (callers must pair it with SubjectOverflow — ">= N" semantics once overflowed).

type Config 

type Config struct {
	// StorePath is the session-store JSON document. Empty = memory-only
	// (tests); the engine then never touches the filesystem.
	StorePath string
	// Now is the injected clock (required — New rejects nil).
	Now func() time.Time
	// Baseline captures the generation pins at session start. nil = zero
	// Baseline (M1 root wiring supplies the real capture).
	Baseline func() Baseline
	// Quarantine is invoked when the store fails to parse: the root wires
	// the process quarantine (rename + alert + readiness row). nil = the
	// engine still starts empty; the corrupt file is left in place and will
	// be overwritten by the next successful save.
	Quarantine func(path string, err error)
	// MaxRetainedSessions bounds retained TERMINAL sessions (FIFO prune).
	// 0 ⇒ DefaultMaxRetainedSessions; clamped to [1, 64].
	MaxRetainedSessions int
	// MaxSessionDuration auto-completes an overdue active session (lazily).
	// 0 ⇒ DefaultMaxSessionDuration; clamped up to minSessionDuration.
	MaxSessionDuration time.Duration
	// Sink consumes drained observations (M2: tests / the M3 aggregator).
	// nil = validated observations are counted and discarded. Called ONLY from
	// the single drain goroutine, with per-event panic containment. Fixed at
	// construction; never called after Close returns.
	Sink func(Observation)
	// SubjectKeyPath is the durable pseudonymization key (M3), stored
	// SEPARATELY from StorePath. Empty = ephemeral in-memory key (memory-only
	// engines; tokens are not restart-stable there by construction).
	SubjectKeyPath string
	// Categories resolves a destination host to (category, tier). Called ONLY
	// from the drain goroutine (never the request hot path). nil = everything
	// aggregates as uncategorized ("", tier "none").
	Categories func(host string) (category, tier string)
	// CategoryEpoch returns the current opaque category-generation identity
	// (feed generation + overrides revision + admin taxonomy sequence). Pinned
	// into the session baseline at Start; a mid-session change is recorded as
	// churn. nil = epoch tracking disabled.
	CategoryEpoch func() string
	// PolicyContent returns the current canonical policy CONTENT identity
	// (schema v8, Codex round 13) — compared per consumed observation against
	// the Baseline.PolicyContentHash pinned at Start, so evidence collected
	// under a TRANSIENT policy change (A→B→A) is latched as churn even though
	// the restored hash matches the baseline again at generation time. Must
	// be cheap (the root memoizes by policy generation). nil = policy-churn
	// tracking disabled.
	PolicyContent func() string
	// TaxonomyKey returns a MONOTONIC change token for the taxonomy the
	// Categories resolver consults (Codex round 24): the content-derived
	// CategoryEpoch is deliberately ABA-blind — a taxonomy A→B→A round trip
	// restores the same epoch string — so the consume-time brackets alone
	// cannot witness a round trip completing WITHIN one observation's
	// resolution. The drain reads this token on both sides of the resolution
	// and latches a churn witness when it moved. Must be cheap (atomic
	// loads). nil = resolution-bracket witnessing disabled.
	TaxonomyKey func() TaxonomyToken
	// RecommendableCategories is the fail-closed ALLOWLIST of categories the
	// M4 generator may recommend (never a denylist): a cell whose category is
	// not on this list can never produce a recommendation, and an EMPTY list
	// means NOTHING is recommendable. Canonicalized deterministically at New
	// (trim/dedupe/sort, exact-match semantics); its identity (GuardrailsHash)
	// is pinned into every session Baseline at Start.
	RecommendableCategories []string
	// Recommend holds the explicit confidence-predicate thresholds (M4). Zero
	// fields take the package defaults.
	Recommend Thresholds
}

Config wires the engine. Now is REQUIRED; everything else has safe defaults.

type CoverageEvidence 

type CoverageEvidence struct {
	ObservedSubjects           int             `json:"observed_subjects"`
	SubjectsIsLowerBound       bool            `json:"subjects_is_lower_bound,omitempty"`
	ObservationDays            int             `json:"observation_days"`
	DaysIsLowerBound           bool            `json:"days_is_lower_bound,omitempty"`
	SessionWindowDays          int             `json:"session_window_days,omitempty"` // whole session span (calendar days, inclusive)
	TransportLoss              TransportWindow `json:"transport_loss,omitempty"`      // session-window loss accounting (deltas)
	TransportDegraded          bool            `json:"transport_degraded,omitempty"`
	MembershipDenominatorKnown bool            `json:"membership_denominator_known"` // constant false in M4 — no membership input exists
}

CoverageEvidence is facts about how much of the population/window was observed — DISTINCT from Confidence. No fabricated denominators: there is no percentage field, and MembershipDenominatorKnown is structurally false in M4 because no IdP membership input exists ("Confidence: HIGH / membership denominator unavailable" is a legitimate, honest pairing).

type Engine 

type Engine struct {
	// contains filtered or unexported fields
}

Engine owns the Learning Session lifecycle and (M2) the observation transport. Session state is guarded by one mutex; accessors return copies, never internal pointers. learningActive mirrors "a session is Learning" as an atomic so the request-path Observe gate is lock-free.

func New 

func New(cfg Config) (*Engine, error)

New constructs the engine and loads the store (fail-closed; see load).

func (*Engine) AbortAccept 

func (e *Engine) AbortAccept(id string) (Recommendation, error)

AbortAccept reverts an unresolved intent: accepting → generated, clearing the TargetRuleID (the safe reconcile when the target rule does not exist and the current fences refuse re-execution). Idempotent on generated; refuses on accepted/rejected/superseded — an intent whose rule EXISTS must finalize, never abort.

func (*Engine) ActiveSession 

func (e *Engine) ActiveSession() (Session, bool)

ActiveSession returns a copy of the Learning session, if any. Lazy expiry is applied in memory (flagged dirty; persisted on the next mutation or Close) so reads stay side-effect-free on disk while never reporting an overdue session as active.

func (*Engine) BeginAccept 

func (e *Engine) BeginAccept(id, targetRuleID string) (Recommendation, error)

BeginAccept persists the durable acceptance INTENT: generated → accepting with the root-preallocated TargetRuleID. Idempotent on an already-accepting recommendation: the EXISTING intent (and its TargetRuleID) is returned and the caller's newly minted ID is discarded — retries and crash-recovery must converge on ONE target rule identity. Every other state refuses.

func (*Engine) CancelSession 

func (e *Engine) CancelSession(actor string) (Session, error)

CancelSession cancels the active session (terminal; retained but marked non-authoritative — M2+ never generates recommendations from it).

func (*Engine) CaptureWindow 

func (e *Engine) CaptureWindow() (uint64, bool)

CaptureWindow returns the generation of the currently OPEN acceptance window, coherently with the learning gate (Codex round 28): reading the gate and the generation separately let a session stop and rotate between the two reads, capturing an unowned closing generation whose later observation could only fall back to the global drop counter. The generation is monotonic (no ABA), so gate-on bracketed by two equal generation reads proves the generation was current — and therefore owned — at the gate read. ok=false ⇒ no session is Learning.

func (*Engine) Close 

func (e *Engine) Close() error

Close stops the observation transport (draining everything already queued — deterministic shutdown), then flushes any lazily-flipped session state. Safe to call more than once and on a read-only engine.

func (*Engine) CurrentRecommendationPolicy 

func (e *Engine) CurrentRecommendationPolicy() RecommendationPolicy

CurrentRecommendationPolicy returns a copy of the canonical decision-policy snapshot the engine generates under.

func (*Engine) FinalizeAccept 

func (e *Engine) FinalizeAccept(id, actor string) (Recommendation, error)

FinalizeAccept latches accepting → accepted (recording actor + instant). Idempotent: already-accepted returns the stored object unchanged. Only an accepting recommendation can finalize — the root boundary calls this ONLY after verifying the exact target draft rule exists.

func (*Engine) GenerateRecommendations 

func (e *Engine) GenerateRecommendations(sessionID string) (GenerateResult, error)

GenerateRecommendations deterministically generates recommendations from the COMPLETED session sessionID. Eligibility gates (each named, each fail-closed; note blocked traffic is deliberately NOT a gate — negative evidence never rejects a cell, it only appears as factual counts): 

session_completed    — State == Completed (Learning/Cancelled refuse)
guardrails_pinned    — Baseline.GuardrailsHash present (pre-M4 sessions refuse)
guardrails_match     — pinned hash == current hash (stale ⇒ refuse, never reinterpret)
subject_key_stable   — no mid-session pseudonym-key change (see ErrSubjectKeyChanged)
real_group_scope     — cell scope is g:<group> (synthetic scopes are evidence-only)
category_allowlisted — cell category non-empty AND on the canonical allowlist
allowed_evidence     — cell.Allowed > 0 (something was actually observed allowed)

Repeated generation is idempotent: identical content (same EvidenceHash) is kept, not duplicated; changed content supersedes the prior object. Persists before returning (rollback on failure).

func (*Engine) GuardrailsHash 

func (e *Engine) GuardrailsHash() string

GuardrailsHash exposes the engine's current guardrail identity (read-only).

func (*Engine) LearningActive 

func (e *Engine) LearningActive() bool

LearningActive reports whether a session is currently Learning (the same lock-free gate Observe checks). The M5A adapters consult it BEFORE building an Observation, so the enabled-but-idle posture does no per-request work beyond two atomic loads — no DTO construction, no group copy, no enqueue.

func (*Engine) ObservationStats 

func (e *Engine) ObservationStats() ObservationStats

ObservationStats snapshots the transport counters.

func (*Engine) Observe 

func (e *Engine) Observe(o Observation)

Observe emits one observation. Non-blocking under every condition; safe from any goroutine. Ignored (uncounted — "not learning" is not loss) when no session is Learning.

func (*Engine) ReadOnly 

func (e *Engine) ReadOnly() bool

ReadOnly reports the newer-schema fail-closed posture.

func (*Engine) RecommendableCategories 

func (e *Engine) RecommendableCategories() []string

RecommendableCategories returns the canonical allowlist (copy).

func (*Engine) RecommendationByID 

func (e *Engine) RecommendationByID(id string) (Recommendation, bool)

RecommendationByID returns a copy of one stored recommendation.

func (*Engine) RecommendationPolicyHash 

func (e *Engine) RecommendationPolicyHash() string

RecommendationPolicyHash exposes the engine's current decision-policy identity (read-only; immutable after New).

func (*Engine) Recommendations 

func (e *Engine) Recommendations() []Recommendation

Recommendations returns deep copies of every retained recommendation in durable-store order.

func (*Engine) Reject 

func (e *Engine) Reject(id, actor, reason string) (Recommendation, error)

Reject latches generated → rejected with a bounded, control-char-stripped reason. Idempotent: already-rejected returns the stored object (the original reason is kept — a retry never rewrites history). Accepted, accepting, and superseded recommendations refuse. Reject mutates NOTHING outside this recommendation's state.

func (*Engine) SessionOverview 

func (e *Engine) SessionOverview(id string) (AggregateOverview, bool)

SessionOverview returns the aggregate overview for one session by ID.

func (*Engine) Sessions 

func (e *Engine) Sessions() []Session

Sessions returns copies of all retained sessions, creation-ordered.

func (*Engine) Snapshot 

func (e *Engine) Snapshot() Stats

Snapshot returns the engine posture.

func (*Engine) StartSession 

func (e *Engine) StartSession(actor string) (Session, error)

StartSession opens a new Learning session. Enforces the one-active-session invariant and the read-only posture; persists before returning.

func (*Engine) StopSession 

func (e *Engine) StopSession(actor string) (Session, error)

StopSession completes the active session. Persists before returning.

func (*Engine) SubjectKeyID 

func (e *Engine) SubjectKeyID() string

SubjectKeyID exposes the pseudonym key's stable identity (hex of a hash prefix — reveals nothing of the key). Server-side staleness input; not for client DTOs.

func (*Engine) SupersedeInvalidatedAccept 

func (e *Engine) SupersedeInvalidatedAccept(id string) (Recommendation, error)

SupersedeInvalidatedAccept resolves an accepting intent whose evidence was invalidated by a late loss charge (round 29): accepting+flag → superseded, clearing TargetRuleID — the root calls this ONLY after compensating away any candidate draft rule the intent created, so the linkage is dead. Idempotent on superseded; every other state (including an UN-flagged accepting intent) refuses with its state sentinel.

func (*Engine) WindowGeneration 

func (e *Engine) WindowGeneration() uint64

WindowGeneration returns the current acceptance-window generation, for producers that capture their decision context BEFORE dispatch and stamp it via Observation.WindowGen (Codex round 24).

type EpochChurn 

type EpochChurn struct {
	At string `json:"at"` // RFC3339 UTC
	To string `json:"to"` // the new epoch value (opaque)
}

EpochChurn records one category-generation change observed mid-session.

type Evidence 

type Evidence struct {
	AllowedRequests       int64 `json:"allowed_requests"`
	PolicyBlockedRequests int64 `json:"policy_blocked_requests,omitempty"` // request count — NOT a user count
	ThreatBlockedRequests int64 `json:"threat_blocked_requests,omitempty"` // request count — NOT a user count

	ObservedAllowedSubjects int   `json:"observed_allowed_subjects"`         // exact distinct pseudonymous tokens admitted
	SubjectsIsLowerBound    bool  `json:"subjects_is_lower_bound,omitempty"` // overflow occurred: true count ≥ the figure
	SubjectOverflow         int64 `json:"subject_overflow,omitempty"`

	AllowedObservationDays int   `json:"allowed_observation_days"`
	DaysIsLowerBound       bool  `json:"days_is_lower_bound,omitempty"`
	DayOverflow            int64 `json:"day_overflow,omitempty"`

	AllowedFirstSeen int64 `json:"allowed_first_seen,omitempty"` // unix seconds
	AllowedLastSeen  int64 `json:"allowed_last_seen,omitempty"`

	TopAllowedHosts   []HostCount `json:"top_allowed_hosts,omitempty"` // admission-bounded representatives
	OtherAllowedHosts int64       `json:"other_allowed_hosts,omitempty"`

	// RuleHits attributes the WHOLE cell (all evidence directions) by matched
	// RuleID; DefaultActionRuleKey marks default-action decisions and can never
	// collide with a real ULID. Factual only — no redundancy/obsolescence
	// inference is derived or stored.
	RuleHits   []AttributionCount `json:"rule_hits,omitempty"`
	OtherRules int64              `json:"other_rules,omitempty"`
	TierHits   []AttributionCount `json:"tier_hits,omitempty"` // category-resolution tiers (whole cell)
	OtherTiers int64              `json:"other_tiers,omitempty"`
}

Evidence is the by-value copy of one cell's facts at generation time, independent of the live aggregate. Field naming carries the evidence direction: every subject/day/host/first-last figure is ALLOWED-only by M3 construction. Blocked traffic appears as request counts ONLY — there is no field for blocked users/subjects because that fact is never collected.

type Gap 

type Gap struct {
	At     string `json:"at"` // RFC3339 UTC
	Reason string `json:"reason"`
}

Gap records a window in which observations were (or would have been) lost — M1: process restarts while a session was active. Never silent (ADR-0025 §8).

type GenerateResult 

type GenerateResult struct {
	SessionID                string
	Recommendations          []Recommendation
	EligibleCells            int // cells that passed every eligibility gate
	TruncatedCells           int // eligible cells beyond maxRecommendationsPerGeneration
	SkippedSyntheticScope    int // s:unauth / s:groupless — evidence-only populations
	SkippedCategory          int // empty or non-allowlisted category (fail-closed)
	SkippedNoAllowedEvidence int // no positive allowed evidence in the cell
	SupersededCount          int // prior recommendations replaced by changed content
	UnchangedCount           int // idempotent hits (identical content already stored)
}

GenerateResult is the per-call accounting for one generation pass. Every bound and skip is COUNTED (no silent truncation — the no-silent-caps house rule); the recommendations returned are deep copies of the CURRENT generated set for the session, sorted by (group, category).

type HostCount 

type HostCount struct {
	Host  string `json:"host"`
	Count int64  `json:"count"`
}

HostCount / AttributionCount are the deterministic (sorted-slice) forms of the aggregate's bounded maps — the recommendation DTOs carry no maps so marshal order is fixed.

type Observation 

type Observation struct {
	At         int64    // unix seconds; stamped by the engine at accept when zero
	Subject    string   // resolved identity; "" = unauthenticated
	AuthSource string   // verbatim provenance ("local"/"exempt"/"unauth"/IdP source)
	Groups     []string // bounded copy of the resolved identity's groups
	Host       string   // normalized destination host only — never a URL
	Method     string
	RuleID     string // matched access-rule ULID; "" = default action
	Action     string // rule action, or "default:allow"/"default:deny"
	Status     string // the request-log Status taxonomy value for this decision
	SSLAction  string // "Inspect"/"Bypass" when resolved; "" on blocked branches

	// WindowGen, when non-zero, is the acceptance-window generation the
	// producer captured WITH its decision (Codex round 24): a decision made
	// while session A was active must never aggregate into a session B whose
	// window opened mid-dispatch — A's finish barrier cannot wait for a
	// producer that has not registered yet. Observe stamps the captured
	// generation instead of the current one, so a rotated-window event
	// resolves through the existing gen-mismatch machinery as a COUNTED drop,
	// never as another session's evidence. Zero = stamp the current window
	// (paths with no decision-time capture).
	WindowGen uint64

	// PolicyID/CatEpoch are the caller-supplied DECISION-TIME identity stamps
	// (Codex rounds 20/22): the producer captured (or derived a change
	// witness from) the state that actually determined enforcement/category
	// resolution, so the stamp cannot be a later value that a
	// flip-and-restore already re-baselined. When empty, Observe stamps the
	// corresponding seam's current value at enqueue instead (the prior
	// behavior; still used by paths with no decision-time capture). Opaque to
	// the engine — only compared for equality by the churn latch.
	PolicyID string
	CatEpoch string
	// contains filtered or unexported fields
}

Observation is the normalized per-decision event the runtime emits. Every field is SERVER-DERIVED (typed auth/policy state — F6); nothing here may originate from a client-controlled header or request field. Deliberately absent: URLs/paths/query strings, request headers/bodies, cookies, credentials, client IP (not needed by the M2 transport; M3 decides if distinct-subject evidence needs an address token), rule NAME (RuleID is the stable rename-safe attribution), and any open-ended metadata map.

Subject carries the authoritative resolved identity TRANSIENTLY (queue + sink only — observations are never persisted in M2); M3 decides the durable representation (transformation/pseudonymization) before any identifier is stored. AuthSource is verbatim opaque provenance — its syntax is not canonical and must not be parsed or normalized. Empty Subject with AuthSource "unauth"/"exempt" is the explicit unauthenticated marker and must never be folded into group evidence (Groups is nil there by construction).

type ObservationStats 

type ObservationStats struct {
	Accepted       int64 // enqueued
	Dropped        int64 // whole observation lost: queue full at enqueue, transport closed, or no attributable session window at consume (closed/rotated window) — always counted, never silent
	Rejected       int64 // invalid (empty Host) — discarded
	ConsumerPanics int64 // sink panicked — event lost, drain continued
	Delivered      int64 // handed to the sink (or discarded clean when no sink)
	// GroupsTruncated counts ACCEPTED observations whose identity carried more
	// than MaxObservationGroups groups (M5B.1): the observation was kept but
	// attributes to only the first 16 groups, so group context is INCOMPLETE
	// for those events. Surfaced (never silent) so evidence can never imply
	// complete group coverage; deliberately NOT a Degraded() trigger — the
	// omitted groups simply receive no evidence (undercount, the safe
	// direction), while the retained cells' counts are exact.
	GroupsTruncated int64
}

ObservationStats are the monotonic transport counters (loss accounting — future readiness computations must be unable to lie about transport loss).

type ProposedRule 

type ProposedRule struct {
	Action       string `json:"action"`     // always "Allow"
	SSLAction    string `json:"ssl_action"` // always "Inspect"
	Enabled      bool   `json:"enabled"`    // always false (born disabled)
	SourceGroup  string `json:"source_group"`
	DestCategory string `json:"dest_category"`
}

ProposedRule is the engine-owned rule-shape DTO — deliberately NOT the root policy-rule type (the wall makes importing it impossible). Fixed semantics: Action=Allow, SSLAction=Inspect, Enabled=false; only the group/category pair varies, and both are copied exactly from the observed evidence.

type Recommendation 

type Recommendation struct {
	ID                string           `json:"id"` // content-derived (deterministic); see recID
	SessionID         string           `json:"session_id"`
	State             string           `json:"state"`
	Group             string           `json:"group"`    // exact observed real group (no scope prefix)
	Category          string           `json:"category"` // exact allowlisted category
	ProposedRule      ProposedRule     `json:"proposed_rule"`
	Confidence        string           `json:"confidence"`
	ConfidenceReasons []string         `json:"confidence_reasons,omitempty"` // named predicates supporting the level
	ConfidenceLimits  []string         `json:"confidence_limits,omitempty"`  // named caps/limitations constraining it
	Coverage          CoverageEvidence `json:"coverage"`
	Evidence          Evidence         `json:"evidence"`
	// Generation-identity pins (by-value): Baseline carries PolicyGeneration,
	// DefaultAction, CategoryEpoch and GuardrailsHash as pinned at session
	// start; SubjectKeyID pins the pseudonym-key identity the subject evidence
	// was minted under; EngineSchema pins the store schema at generation.
	Baseline     Baseline `json:"baseline"`
	SubjectKeyID string   `json:"subject_key_id,omitempty"`
	// Policy/PolicyHash pin the recommendation DECISION POLICY (M4.1): the
	// snapshot is embedded by value so the decision stays explainable after the
	// runtime configuration changes; the hash is the identity a future
	// acceptance surface compares (a mismatch is an explicit stale reason and
	// must refuse acceptance).
	Policy       RecommendationPolicy `json:"policy"`
	PolicyHash   string               `json:"policy_hash"`
	EvidenceHash string               `json:"evidence_hash"` // canonical content hash (identity/idempotency anchor)
	EngineSchema int                  `json:"engine_schema"`
	GeneratedAt  string               `json:"generated_at"` // RFC3339 UTC (injected clock)

	// M5B decision lifecycle (schema v6). TargetRuleID is preallocated by the
	// root trust boundary at BeginAccept and persisted WITH the accepting
	// intent — the crash-safe linkage between this recommendation and the one
	// draft rule its acceptance may create. Kept on accepted (the created rule)
	// and cleared by AbortAccept. Evidence fields above are never touched by
	// any decision transition.
	TargetRuleID string `json:"target_rule_id,omitempty"`
	AcceptedAt   string `json:"accepted_at,omitempty"`
	AcceptedBy   string `json:"accepted_by,omitempty"`
	RejectedAt   string `json:"rejected_at,omitempty"`
	RejectedBy   string `json:"rejected_by,omitempty"`
	RejectReason string `json:"reject_reason,omitempty"` // bounded, control-chars stripped

	// LateLossInvalidated marks an ACCEPTING intent whose owning session was
	// charged late transport loss AFTER the intent latched (Codex round 29):
	// the evidence's loss accounting changed post-intent, so FinalizeAccept
	// refuses (ErrAcceptInvalidatedByLateLoss) and the root trust boundary
	// resolves the intent to superseded, compensating away any candidate draft
	// rule it created. Generated recommendations are superseded directly by
	// the charge and never carry this flag; the flag survives on superseded as
	// history. Set only by the charge path, under the same e.mu FinalizeAccept
	// mutates under — there is no interleaving in which a flagged intent
	// latches accepted.
	LateLossInvalidated bool `json:"late_loss_invalidated,omitempty"`
}

Recommendation is the immutable durable advisory object. Evidence, coverage, and baseline are copied BY VALUE at generation — later reads show generation- time facts regardless of live-cell churn. The only field that ever changes after creation is State (generated → superseded, the supersession latch).

type RecommendationPolicy 

type RecommendationPolicy struct {
	AlgorithmVersion         int      `json:"algorithm_version"`
	HighMinAllowedRequests   int64    `json:"high_min_allowed_requests"`
	HighMinSubjects          int      `json:"high_min_subjects"`
	HighMinDays              int      `json:"high_min_days"`
	MediumMinAllowedRequests int64    `json:"medium_min_allowed_requests"`
	MediumMinSubjects        int      `json:"medium_min_subjects"`
	MediumMinDays            int      `json:"medium_min_days"`
	CommunityTiers           []string `json:"community_tiers"` // canonical form (trimmed/deduped/sorted)
}

RecommendationPolicy is the by-value snapshot of the DECISION POLICY a recommendation was generated under (M4.1): every engine-configuration value that can change eligibility, confidence, or the confidence caps, plus the algorithm version standing in for the non-configurable logic and bounds. Deliberately DISJOINT from GuardrailsHash: guardrails say WHICH categories are eligible; this says HOW evidence becomes a recommendation and a confidence tier. Embedded by value in every Recommendation so historical decisions stay fully explainable without the runtime configuration that produced them.

type Session 

type Session struct {
	ID        string   `json:"id"`
	State     string   `json:"state"`
	CreatedAt string   `json:"created_at"`
	StartedAt string   `json:"started_at"`
	StoppedAt string   `json:"stopped_at,omitempty"`
	CreatedBy string   `json:"created_by"`
	StoppedBy string   `json:"stopped_by,omitempty"`
	Baseline  Baseline `json:"baseline"`
	Gaps      []Gap    `json:"gaps,omitempty"`

	// M3 — bounded factual aggregation + degradation metadata.
	SubjectKeyID  string          `json:"subject_key_id,omitempty"` // pseudonym-key identity the tokens were minted under
	CategoryChurn []EpochChurn    `json:"category_churn,omitempty"` // bounded mid-session category-generation changes
	PolicyChurn   []EpochChurn    `json:"policy_churn,omitempty"`   // bounded mid-session policy-content changes (schema v8, Codex round 13: an A→B→A round trip during the session collects evidence under B that the restored baseline hash alone cannot reveal)
	Transport     TransportWindow `json:"transport,omitempty"`      // session-window transport-counter deltas (loss accounting)
	Agg           *Aggregate      `json:"agg,omitempty"`            // bounded Group × Category cells
	// contains filtered or unexported fields
}

Session is one learning session. All timestamps are RFC3339 UTC strings derived from the injected clock.

type StaleInputs 

type StaleInputs struct {
	PolicyGeneration int64
	CategoryEpoch    string
	GuardrailsHash   string
	SubjectKeyID     string
	// RecommendationPolicyHash is the CURRENT decision-policy identity
	// (Engine.RecommendationPolicyHash). Empty ⇒ the caller is not asserting
	// it (no claim); non-empty ⇒ compared fail-closed, so a recommendation
	// with a missing pin (pre-M4.1 object) is stale, never assumed current.
	RecommendationPolicyHash string
	// PolicyContentHash is the CURRENT canonical content identity of the
	// running access policy (root-computed). Same claim semantics: empty ⇒ no
	// claim; asserted ⇒ compared fail-closed against the pinned baseline.
	PolicyContentHash string
}

StaleInputs are the CURRENT identities a recommendation's pins are compared against. The caller (M5's surface, eventually) supplies them explicitly — this package cannot and does not read live state.

type Stats 

type Stats struct {
	Sessions        int
	Recommendations int
	Active          bool
	ReadOnly        bool
	MaxRetained     int
	MaxDuration     time.Duration
	SchemaVersion   int
}

Stats is the M1 posture snapshot (bounded scalars only).

type TaxonomyToken 

type TaxonomyToken struct {
	View any
	Rev  uint64
}

TaxonomyToken is an opaque MONOTONIC change token for the taxonomy the Categories resolver consults, produced by the Config.TaxonomyKey seam. View holds the root's immutable taxonomy view object (holding it prevents pointer reuse, so equality proves the same view); Rev is a monotonic mutation counter for the mutable store. Compared only for equality.

type Thresholds 

type Thresholds struct {
	HighMinAllowedRequests   int64 // default 30
	HighMinSubjects          int   // default 5
	HighMinDays              int   // default 5
	MediumMinAllowedRequests int64 // default 5
	MediumMinSubjects        int   // default 2
	MediumMinDays            int   // default 2
	// CommunityTiers names the category-resolution tiers treated as
	// community/UT1-sourced for the community-majority confidence cap. Tier
	// strings are opaque to the engine; the default matches the root resolver's
	// taxonomy. Empty ⇒ default.
	CommunityTiers []string
}

Thresholds are the explicit, testable confidence predicates (spec: named deterministic predicates, no composite scores). Zero fields take the defaults below. HIGH structurally requires distinct-subject AND distinct-day diversity in addition to request volume — volume alone can never reach HIGH.

type TransportWindow 

type TransportWindow struct {
	Accepted       int64 `json:"accepted,omitempty"`
	Dropped        int64 `json:"dropped,omitempty"`
	Rejected       int64 `json:"rejected,omitempty"`
	ConsumerPanics int64 `json:"consumer_panics,omitempty"`
	// GroupsTruncated (M5B.1) counts accepted observations whose group list
	// exceeded MaxObservationGroups: those events carry INCOMPLETE group
	// context (only the first 16 groups received evidence). Carried on every
	// recommendation via Coverage.TransportLoss so evidence can never imply
	// complete group coverage.
	GroupsTruncated int64 `json:"groups_truncated,omitempty"`
}

TransportWindow is the persisted per-session accumulation of transport- counter DELTAS (pinned at session start / restart-load and advanced at every flush) — never lifetime process totals.

func (TransportWindow) Degraded 

func (w TransportWindow) Degraded() bool

Degraded reports whether the session window LOST whole observations. GroupsTruncated is deliberately excluded: a truncated observation was delivered and its retained cells are exact — the omitted groups simply received no evidence (an undercount, the direction evidence is allowed to err in), so it is surfaced as a coverage fact rather than capping confidence for every cell in the session.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.