scheduler

package
v1.10.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 5, 2026 License: MIT Imports: 9 Imported by: 0

Documentation

Overview

Package scheduler owns extraction lifecycle: initial-walk on workspace activation, incremental scheduling on file changes (Phase 60 fills body), state tracking, and the centralized RequireReady consumer-side gate.

D-04 hard invariants:

  • One active extraction job per workspace at a time.
  • ScheduleInitialExtraction is idempotent — concurrent calls return same JobID.
  • File changes during in-flight initial extraction are queued for incremental.
  • Jobs are cancellable on workspace deactivation.
  • RequireReady is the ONLY semantic readiness API for consumers — no time.Sleep polling.

Allowed imports:

  • internal/semantic/extract (consumes Registry, fact types)
  • internal/repomap (consumes PageRank scores for initial-walk priority)
  • internal/semantic/store (writes ExtractedFile facts via Phase 59 surface)

Forbidden imports:

  • internal/kernel (scheduler must not depend on kernel — daemon is the wiring point that bridges them via callback)

Index

Constants

This section is empty.

Variables

View Source 
var ErrSemanticFailed = errors.New("semantic index failed")

ErrSemanticFailed is returned by RequireReady when the workspace's semantic index is in the SemanticFailed state.

Functions

func ClassifyByExtension 

func ClassifyByExtension(path string) string

ClassifyByExtension maps a file path to ("go" | "typescript" | "python" | "other"). Bounded label for the helix_semantic_extraction_total metric (matches the allowlist primed in internal/obs/metrics.go for the "language" label of SemanticExtraction). Closed enum: do not extend without updating the allowlist test.

Types

type ExtractionScheduler 

type ExtractionScheduler interface {
	ScheduleInitialExtraction(workspaceID semantic.WorkspaceID, req InitialExtraction) JobID
	ScheduleIncremental(workspaceID semantic.WorkspaceID, changes []FileChange) JobID
	Status(workspaceID semantic.WorkspaceID) SemanticStatus
	Subscribe(workspaceID semantic.WorkspaceID) <-chan SemanticStatus
	// RequireReady is the SOLE semantic readiness API. Implementation lives in
	// ready.go; consumers MUST use this method instead of polling Status().
	RequireReady(ctx context.Context, ws semantic.WorkspaceID, policy ReadyPolicy) (ReadyResult, error)
}

ExtractionScheduler is the consumer-facing surface owned by P03. The daemon (P05) constructs a *Scheduler and exposes it through this interface to keep callers free of the concrete type's wiring deps.

D-04 invariants:

  • ScheduleInitialExtraction is idempotent per workspace: concurrent calls return the same in-flight JobID.
  • ScheduleIncremental is a Phase 60 stub here (returns a sentinel JobID).
  • Status reads the latest published SemanticStatus.
  • Subscribe returns a buffered channel (cap 8) that receives every transition; slow consumers are dropped non-blockingly (T-59-03-02).
  • RequireReady is the ONLY semantic readiness API — no time.Sleep polling anywhere in this package (D-04 acceptance #9).

type FileChange 

type FileChange struct {
	Path string
	Kind string // "modified" | "created" | "deleted"
}

FileChange describes a single workspace file mutation passed to ScheduleIncremental. Phase 60 fills the body; Phase 59 ships the type so consumers can compile against the interface today.

type FilePriority 

type FilePriority int

FilePriority is the scheduling priority for a single file in the initial-walk queue. Lower numeric value = higher priority (the underlying container is a min-heap). 

const (
	// PriorityInflightToolReferenced — tier 1: file is being looked at by a
	// tool call right now. Caller (P05 daemon) supplies the inflight set.
	PriorityInflightToolReferenced FilePriority = 0
	// PriorityRepomapImportant — tier 2: top-quartile PageRank from the
	// repomap engine. Phase 59 ships the tier; P05 wires the actual set.
	PriorityRepomapImportant FilePriority = 1
	// PriorityFirstClassSource — tier 3: remaining first-class language
	// source (go / typescript / python).
	PriorityFirstClassSource FilePriority = 2
	// PriorityNonFirstClass — tier 4: file-row-only emit (D-05 unsupported).
	PriorityNonFirstClass FilePriority = 3
)

type IndexError 

type IndexError struct {
	File    string
	Reason  string // matches extract.PartialReason values
	Message string
}

IndexError describes a per-file extraction failure surfaced through SemanticStatus.Errors. Reason should match one of the extract.PartialReason values when the failure originates from the extractor; free-form otherwise.

type InitialExtraction 

type InitialExtraction struct {
	Reason string
	Mode   InitialExtractionMode
}

InitialExtraction is the request payload for ScheduleInitialExtraction. Reason is recorded for telemetry (e.g., "workspace_activation", "require_ready_cold"); Mode selects the extraction strategy.

type InitialExtractionMode 

type InitialExtractionMode int

InitialExtractionMode controls whether ScheduleInitialExtraction prefers an incremental rebuild (when a prior snapshot exists) or always walks the full tree. Phase 59 ships the enum; the daemon-wired implementation in P05 picks the actual strategy. 

const (
	ModeAuto InitialExtractionMode = iota
	ModeIncrementalIfPossible
	ModeFullRebuild
)

type JobID 

type JobID string

JobID is the opaque handle returned by ScheduleInitialExtraction / ScheduleIncremental. Idempotent: concurrent ScheduleInitialExtraction calls for the same workspace return the same JobID until the in-flight job completes (D-04 invariant).

type PoppedFile 

type PoppedFile struct {
	Path     string
	Language string // bounded label: "go" | "typescript" | "python" | "other"
	Priority FilePriority
}

PoppedFile is the consumer-visible payload of a priority-queue Pop. The internal heap entry is unexported; PopFile returns this struct so callers outside the package can drain the queue without leaking heap internals.

type PriorityQueue 

type PriorityQueue struct {
	// contains filtered or unexported fields
}

PriorityQueue is the consumer-facing priority queue handle. Construct via BuildPriorityQueue; drain via Len() + PopFile().

func BuildPriorityQueue 

func BuildPriorityQueue(files []string, inflightToolReferenced, repomapImportant map[string]struct{}) *PriorityQueue

BuildPriorityQueue constructs a priority queue from a list of files using the 4-tier order in CONTEXT.md D-04:

  1. PriorityInflightToolReferenced — caller passes the set of files referenced by the currently-handled MCP tool call (may be nil/empty).
  2. PriorityRepomapImportant — caller passes the top-quartile PageRank file set (may be nil if repomap is disabled or cold).
  3. PriorityFirstClassSource — anything classified as go/typescript/python that wasn't already promoted to tier 1 or 2.
  4. PriorityNonFirstClass — everything else; emits file-row-only via D-05.

The function does NOT import internal/repomap directly — the caller (P05 daemon wiring) computes the repomap-important set and passes it in. This keeps priority.go testable without a running repomap.

I10 (DEFERRED): Phase 59 ships the priority queue infrastructure but does NOT wire repomap PageRank into Scheduler.ScheduleInitialExtraction. P05 passes a nil/empty repomapImportant set in Phase 59; the PriorityRepomapImportant tier is structurally reachable but unused. Phase 60+ adds the actual PageRank-set construction in the daemon callback path when the live-update pipeline lands. Intentional scope deferral per the phase 59 must_haves derivation review (planner I10).

func (*PriorityQueue) Len 

func (pq *PriorityQueue) Len() int

Len returns the number of items remaining in the queue.

func (*PriorityQueue) PopFile 

func (pq *PriorityQueue) PopFile() PoppedFile

PopFile removes and returns the highest-priority entry. Panics if the queue is empty — callers must check Len() > 0.

type ReadyPolicy 

type ReadyPolicy struct {
	// Timeout caps the maximum wait. Default 30s
	// (cfg.SemanticIndex.Extraction.ExtractionReadyTimeout). On timeout the
	// caller receives the latest known status with Ready=true if any files
	// were indexed (best-partial) and Ready=false otherwise.
	Timeout time.Duration

	// AllowPartial: if true, SemanticPartial counts as ready. Default true.
	AllowPartial bool

	// MinState is the minimum index state that satisfies the wait. Default
	// SemanticPartial. Implementations rank states via stateRank().
	MinState SemanticIndexState

	// TriggerIfCold: if true and the workspace is in SemanticNotStarted,
	// RequireReady kicks ScheduleInitialExtraction before waiting. Default true.
	TriggerIfCold bool
}

ReadyPolicy controls how RequireReady waits for the workspace's semantic index to reach a usable state. The zero value is NOT the documented default — callers that want the documented behavior call DefaultReadyPolicy().

D-04 documented defaults (from CONTEXT.md): 

Timeout=30s, AllowPartial=true, MinState=SemanticPartial, TriggerIfCold=true.

func DefaultReadyPolicy 

func DefaultReadyPolicy() ReadyPolicy

DefaultReadyPolicy returns the D-04 documented defaults. Callers that want tighter policy override fields explicitly; do not mutate the returned struct.

type ReadyResult 

type ReadyResult struct {
	State      SemanticIndexState
	Partial    bool
	Ready      bool
	IndexedAt  time.Time
	FilesTotal int
	FilesDone  int
	Errors     []IndexError
}

ReadyResult is the consumer-facing return of RequireReady. Ready==true means the wait was satisfied (state met MinState or partial-with-AllowPartial). On timeout, Ready==true iff at least one file was indexed (best-partial).

type Scheduler 

type Scheduler struct {
	// contains filtered or unexported fields
}

Scheduler is the concrete in-memory implementation of ExtractionScheduler. Phase 59 P03 ships the orchestration surface (state, subscribers, idempotent job admission). Phase 59 P05 wires the actual extraction loop (provider dispatch, store writes) onto this scaffold; the public API does not change.

func NewScheduler 

func NewScheduler(registry *extract.Registry) *Scheduler

NewScheduler constructs a Scheduler with the given extract.Registry. The registry is consumed read-only and may be nil in tests that do not exercise the per-language dispatch path; production daemon wiring (P05) always passes a non-nil registry.

func (*Scheduler) RequireReady 

func (s *Scheduler) RequireReady(ctx context.Context, ws semantic.WorkspaceID, policy ReadyPolicy) (ReadyResult, error)

RequireReady is the SOLE semantic readiness API. Consumers MUST use this; any time.Sleep-based polling violates D-04 acceptance #9.

Callers MUST be on a goroutine that does not hold any scheduler-internal lock — RequireReady can wait up to ReadyPolicy.Timeout.

Behavior:

  • State == ready → return immediately (Ready=true).
  • State == partial && AllowPartial → return immediately (Ready=true, Partial=true).
  • State == failed → return ErrSemanticFailed (Ready=false).
  • State == not_started && TriggerIfCold → kick scheduler then wait.
  • State == indexing|stale → wait until ready/partial/failed or Timeout.
  • Timeout → return best-partial (Ready=true if FilesDone > 0, else false, no error — partial readiness is not an error per D-04).
  • ctx.Done() → return ctx.Err() with the latest known status.

T-59-03-01: Timer + ctx.Done() escape paths bound the wait; the default 30s ceiling caps DoS exposure even if the caller forgets to set Timeout.

func (*Scheduler) ScheduleIncremental 

func (s *Scheduler) ScheduleIncremental(ws semantic.WorkspaceID, changes []FileChange) JobID

ScheduleIncremental is a Phase 60 stub — returns a sentinel JobID and does not mutate state. Wave 2 ships the interface so consumers compile today.

func (*Scheduler) ScheduleInitialExtraction 

func (s *Scheduler) ScheduleInitialExtraction(ws semantic.WorkspaceID, req InitialExtraction) JobID

ScheduleInitialExtraction kicks an initial-walk extraction for ws. Idempotent per workspace: if a job is already in-flight, the existing JobID is returned and req is ignored (the first call's reason wins). The state transitions to SemanticIndexing on the first admission and Subscribers receive the new status.

func (*Scheduler) Status 

func (s *Scheduler) Status(ws semantic.WorkspaceID) SemanticStatus

Status returns the latest published SemanticStatus for ws. Workspaces that were never scheduled return a zero-value SemanticStatus with State = SemanticNotStarted.

func (*Scheduler) Subscribe 

func (s *Scheduler) Subscribe(ws semantic.WorkspaceID) <-chan SemanticStatus

Subscribe returns a buffered channel (cap 8) that receives every state transition for ws. The channel is owned by the scheduler — callers MUST NOT close it. P05 may add an explicit Unsubscribe when wiring deactivation; Wave 2 ships scheduler-lifetime-bound subscriptions.

type SemanticIndexState 

type SemanticIndexState string

SemanticIndexState is the lifecycle state of a workspace's semantic index. Closed enum — values are written into log fields, surfaced as bounded-label metric values, and consumed by RequireReady's ordering rank. 

const (
	SemanticNotStarted SemanticIndexState = "not_started"
	SemanticIndexing   SemanticIndexState = "indexing"
	SemanticReady      SemanticIndexState = "ready"
	SemanticPartial    SemanticIndexState = "partial"
	SemanticFailed     SemanticIndexState = "failed"
	SemanticStale      SemanticIndexState = "stale"
)

type SemanticStatus 

type SemanticStatus struct {
	State      SemanticIndexState
	Partial    bool
	FilesTotal int
	FilesDone  int
	IndexedAt  time.Time
	Errors     []IndexError
}

SemanticStatus is the consumer-facing snapshot of a workspace's extraction progress. Returned by Scheduler.Status and pushed to Subscribe channels on every state transition.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.