scan

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 10, 2026 License: MIT Imports: 31 Imported by: 0

Documentation

Overview

Package scan orchestrates artifact discovery: walks the repo, dispatches adapters, and upserts artifacts/revisions/todos/criteria into the store.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func SourceTypeDisplayLabel 

func SourceTypeDisplayLabel(sourceType string) string

SourceTypeDisplayLabel returns human-facing copy for `ds scan` summaries. Internal pipeline IDs remain markdown / openspec / adr in config and the DB; this layer is display-only.

Types

type EvidenceConceptExample added in v0.1.0 

type EvidenceConceptExample struct {
	Kind              string `json:"kind"`
	Canonical         string `json:"canonical"`
	DocumentFrequency int    `json:"document_frequency"`
	Reason            string `json:"reason"`
}

EvidenceConceptExample explains one concept skipped from edge materialization.

type EvidenceEdgeExample added in v0.1.0 

type EvidenceEdgeExample struct {
	EdgeType      string  `json:"edge_type"`
	Source        string  `json:"source"`
	Target        string  `json:"target"`
	SourceSignal  string  `json:"source_signal"`
	Explanation   string  `json:"explanation"`
	Confidence    float64 `json:"confidence"`
	EvidenceCount int     `json:"evidence_count"`
}

EvidenceEdgeExample gives a compact receipt-like example for diagnostics.

type EvidenceGraphDiagnostics added in v0.1.0 

type EvidenceGraphDiagnostics struct {
	RichTypedIndex       bool                     `json:"rich_typed_index,omitempty"`
	ConceptsIndexed      int                      `json:"concepts_indexed"`
	MentionsIndexed      int                      `json:"mentions_indexed"`
	EdgesIndexed         int                      `json:"edges_indexed"`
	ConceptsByKind       map[string]int           `json:"concepts_by_kind,omitempty"`
	MentionsByField      map[string]int           `json:"mentions_by_field,omitempty"`
	EdgesByType          map[string]int           `json:"edges_by_type,omitempty"`
	NoisyConceptsSkipped int                      `json:"noisy_concepts_skipped,omitempty"`
	TopNoisyConcepts     []EvidenceConceptExample `json:"top_noisy_concepts,omitempty"`
	TopEdges             []EvidenceEdgeExample    `json:"top_edges,omitempty"`
}

EvidenceGraphDiagnostics is a scan-time summary of persisted graph evidence.

type GitEvidenceDiagnostics added in v0.1.0 

type GitEvidenceDiagnostics struct {
	Enabled                    bool                     `json:"enabled"`
	HistoryShape               string                   `json:"history_shape"`
	IsShallow                  bool                     `json:"is_shallow,omitempty"`
	Branch                     string                   `json:"branch,omitempty"`
	MaxCommits                 int                      `json:"max_commits"`
	MaxFilesPerCommit          int                      `json:"max_files_per_commit"`
	TotalCommits               int                      `json:"total_commits,omitempty"`
	CommitsRead                int                      `json:"commits_read,omitempty"`
	CommitsStored              int                      `json:"commits_stored,omitempty"`
	FilesStored                int                      `json:"files_stored,omitempty"`
	BodiesTruncated            int                      `json:"commit_bodies_truncated,omitempty"`
	EdgesIndexed               int                      `json:"edges_indexed,omitempty"`
	EdgesByType                map[string]int           `json:"edges_by_type,omitempty"`
	SkippedLargeCommits        int                      `json:"skipped_large_commits,omitempty"`
	SkippedMergeCommits        int                      `json:"skipped_merge_commits,omitempty"`
	SkippedLockfileOnlyCommits int                      `json:"skipped_lockfile_only_commits,omitempty"`
	SkippedNoMappedArtifacts   int                      `json:"skipped_no_mapped_artifacts,omitempty"`
	PathPairsEvaluated         int                      `json:"path_pairs_evaluated,omitempty"`
	PathPairsMaterialized      int                      `json:"path_pairs_materialized,omitempty"`
	CappedSourcePaths          int                      `json:"capped_source_paths,omitempty"`
	HighDensitySourcePaths     int                      `json:"high_density_source_paths,omitempty"`
	CappedCommitPathPairs      int                      `json:"capped_commit_path_pairs,omitempty"`
	CappedPathPairs            int                      `json:"capped_path_pairs,omitempty"`
	CappedArtifactEdges        int                      `json:"capped_artifact_edges,omitempty"`
	GitError                   string                   `json:"git_error,omitempty"`
	TopEdges                   []GitEvidenceEdgeExample `json:"top_edges,omitempty"`
}

GitEvidenceDiagnostics is emitted by ds scan --json when git evidence is enabled.

type GitEvidenceEdgeExample added in v0.1.0 

type GitEvidenceEdgeExample struct {
	EdgeType          string   `json:"edge_type"`
	Source            string   `json:"source"`
	Target            string   `json:"target"`
	SourceTitle       string   `json:"source_title,omitempty"`
	TargetTitle       string   `json:"target_title,omitempty"`
	SourceKind        string   `json:"source_kind,omitempty"`
	TargetKind        string   `json:"target_kind,omitempty"`
	SourceSubtype     string   `json:"source_subtype,omitempty"`
	TargetSubtype     string   `json:"target_subtype,omitempty"`
	SourcePath        string   `json:"source_path,omitempty"`
	TargetPath        string   `json:"target_path,omitempty"`
	SourceSignal      string   `json:"source_signal"`
	Explanation       string   `json:"explanation"`
	Confidence        float64  `json:"confidence"`
	EvidenceCount     int      `json:"evidence_count"`
	Commits           []string `json:"commits,omitempty"`
	FileExamples      []string `json:"file_examples,omitempty"`
	HistoryShape      string   `json:"history_shape,omitempty"`
	ConfidenceRule    string   `json:"confidence_rule,omitempty"`
	SourcePathDensity int      `json:"source_path_density,omitempty"`
	TargetPathDensity int      `json:"target_path_density,omitempty"`
}

GitEvidenceEdgeExample is a compact receipt for manual edge audit.

type ProgressEvent added in v0.1.0 

type ProgressEvent struct {
	Phase                string         `json:"phase"`
	Event                string         `json:"event,omitempty"`
	ElapsedMS            int64          `json:"elapsed_ms,omitempty"`
	FilesTotal           int            `json:"files_total,omitempty"`
	FilesScanned         int            `json:"files_scanned,omitempty"`
	CurrentAdapter       string         `json:"current_adapter,omitempty"`
	CandidatesTotal      int            `json:"candidates_total,omitempty"`
	CandidatesProcessed  int            `json:"candidates_processed,omitempty"`
	CandidatesDiscovered map[string]int `json:"candidates_discovered,omitempty"`
	CandidatesParsed     map[string]int `json:"candidates_parsed,omitempty"`
	ArtifactsUpserted    map[string]int `json:"artifacts_upserted,omitempty"`
	ParseDurationMS      int64          `json:"parse_duration_ms,omitempty"`
	ClassifierDurationMS int64          `json:"classifier_duration_ms,omitempty"`
	WriterDurationMS     int64          `json:"writer_duration_ms,omitempty"`
	FlushDurationMS      int64          `json:"flush_duration_ms,omitempty"`
	FTSDurationMS        int64          `json:"fts_duration_ms,omitempty"`
	RowsWritten          map[string]int `json:"rows_written,omitempty"`
	ChunksFlushed        map[string]int `json:"chunks_flushed,omitempty"`
	DeferredFTSRows      int            `json:"deferred_fts_rows,omitempty"`
	InventoryFiles       int            `json:"inventory_files,omitempty"`
	SharedDiscovery      bool           `json:"shared_discovery,omitempty"`
	ParallelWorkers      int            `json:"parallel_workers,omitempty"`
	CappedDiscovery      bool           `json:"capped_discovery,omitempty"`
	TransactionEnabled   bool           `json:"transaction_enabled,omitempty"`
	SkipAuthoredAtLookup bool           `json:"skip_authored_at_lookup,omitempty"`
}

ProgressEvent is a coarse scan heartbeat for long eval/index runs.

type Result 

type Result struct {
	Found              map[string]int                       `json:"Found"`
	SourcesBreakdown   []SourceBreakdownRow                 `json:"sources_breakdown"`
	New                int                                  `json:"New"`
	Updated            int                                  `json:"Updated"`
	Unchanged          int                                  `json:"Unchanged"`
	Hints              []ScanHint                           `json:"hints,omitempty"`
	OpenSpec           *openspecmetrics.Metrics             `json:"openspec,omitempty"`
	EvidenceGraph      *EvidenceGraphDiagnostics            `json:"evidence_graph,omitempty"`
	GitEvidence        *GitEvidenceDiagnostics              `json:"git_evidence,omitempty"`
	WorkstreamEvidence *WorkstreamEvidenceDiagnostics       `json:"workstream_evidence,omitempty"`
	SourceCompanions   *SourceCompanionAdmissionDiagnostics `json:"source_companion_admission,omitempty"`
	SourceManifest     *SourceManifestDiagnostics           `json:"source_manifest,omitempty"`
	// contains filtered or unexported fields
}

Result holds scan summary counts and per-source breakdown for CLI output.

JSON shape (see ds scan --json):

  • "Found": map of adapter/source pipeline name → count of successfully indexed artifacts
  • "sources_breakdown": array of { source_type, label, count, formats }
  • "New", "Updated", "Unchanged": revision outcomes
  • "hints": optional; only when all adapters indexed zero artifacts AND at least one hint candidate exists. Empty candidate list omits the key (encoding/json omitempty on []ScanHint).

type RunOptions added in v0.1.0 

type RunOptions struct {
	MaxCandidatesByAdapter    map[string]int
	FileWorkerCount           int
	UseTransaction            bool
	SkipAuthoredAtLookup      bool
	FreshIndex                bool
	IncludeGitEvidence        bool
	IncludeWorkstreamEvidence bool
	RichTypedIndex            bool
	RecentSourceContext       bool
	FirstPartySourceContext   bool
	SourceManifest            bool
	GitMaxCommits             int
	GitMaxFilesPerCommit      int
	IgnoreRules               bool
	Progress                  func(ProgressEvent)
	ProgressInterval          time.Duration
}

type ScanHint 

type ScanHint struct {
	Path           string `json:"path"`
	SourceType     string `json:"source_type,omitempty"`
	SuggestCommand string `json:"suggest_command,omitempty"`
}

ScanHint is one recovery hint when a scan finds zero artifacts (`hints` in ds scan --json).

type Scanner 

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner runs adapters against a repo and persists results.

func New 

func New(db *store.DB, ids *idgen.Factory, adpts []adapters.Adapter) *Scanner

New creates a Scanner with the given store and adapters.

func (*Scanner) Run 

func (s *Scanner) Run(ctx context.Context, repoRoot string, cfg *config.RepoConfig) (*Result, error)

Run scans the repo at repoRoot, using config if available.

func (*Scanner) RunWithOptions added in v0.1.0 

func (s *Scanner) RunWithOptions(ctx context.Context, repoRoot string, cfg *config.RepoConfig, opts RunOptions) (*Result, error)

RunWithOptions scans the repo with optional eval/runtime controls.

type SourceBreakdownRow 

type SourceBreakdownRow struct {
	SourceType string         `json:"source_type"`
	Label      string         `json:"label"`
	Count      int            `json:"count"`
	Formats    map[string]int `json:"formats"`
}

SourceBreakdownRow is one element of `ds scan --json` field "sources_breakdown".

type SourceCompanionAdmissionDiagnostics added in v0.1.0 

type SourceCompanionAdmissionDiagnostics struct {
	Enabled                  bool                              `json:"enabled"`
	TestFiles                int                               `json:"test_files,omitempty"`
	ExistingSourceCandidates int                               `json:"existing_source_candidates,omitempty"`
	CandidatesConsidered     int                               `json:"candidates_considered,omitempty"`
	Admitted                 int                               `json:"admitted,omitempty"`
	AlreadyPresent           int                               `json:"already_present,omitempty"`
	SkippedByCap             int                               `json:"skipped_by_cap,omitempty"`
	RejectedByReason         map[string]int                    `json:"rejected_by_reason,omitempty"`
	TopAdmitted              []SourceCompanionAdmissionExample `json:"top_admitted,omitempty"`
	TopRejected              []SourceCompanionRejectionExample `json:"top_rejected,omitempty"`
}

SourceCompanionAdmissionDiagnostics summarizes bounded source files admitted because indexed tests pointed at likely implementation companions.

type SourceCompanionAdmissionExample added in v0.1.0 

type SourceCompanionAdmissionExample struct {
	Path       string   `json:"path"`
	Signals    []string `json:"signals,omitempty"`
	Confidence string   `json:"confidence,omitempty"`
	TestPaths  []string `json:"test_paths,omitempty"`
}

SourceCompanionAdmissionExample is a compact receipt for one admitted source companion.

type SourceCompanionRejectionExample added in v0.1.0 

type SourceCompanionRejectionExample struct {
	Path   string `json:"path"`
	Reason string `json:"reason"`
	Signal string `json:"signal,omitempty"`
}

SourceCompanionRejectionExample is a compact receipt for one rejected source companion.

type SourceManifestDiagnostics added in v1.0.0 

type SourceManifestDiagnostics struct {
	Enabled         bool           `json:"enabled"`
	InventoryFiles  int            `json:"inventory_files,omitempty"`
	SourceLikeFiles int            `json:"source_like_files,omitempty"`
	TestLikeFiles   int            `json:"test_like_files,omitempty"`
	IndexedFiles    int            `json:"indexed_files,omitempty"`
	IndexedTests    int            `json:"indexed_tests,omitempty"`
	SymbolRows      int            `json:"symbol_rows,omitempty"`
	TestRows        int            `json:"test_rows,omitempty"`
	ImportRows      int            `json:"import_rows,omitempty"`
	FTSRows         int            `json:"fts_rows,omitempty"`
	IgnoredByReason map[string]int `json:"ignored_by_reason,omitempty"`
	RowsByRoot      map[string]int `json:"rows_by_root,omitempty"`
	RowsByLanguage  map[string]int `json:"rows_by_language,omitempty"`
	RowsByRole      map[string]int `json:"rows_by_role,omitempty"`
}

SourceManifestDiagnostics summarizes the hidden compact first-party source/test manifest.

type WorkstreamArtifactExample added in v0.1.0 

type WorkstreamArtifactExample struct {
	ID      string `json:"id"`
	Title   string `json:"title,omitempty"`
	Kind    string `json:"kind,omitempty"`
	Subtype string `json:"subtype,omitempty"`
	Path    string `json:"path,omitempty"`
}

WorkstreamArtifactExample identifies one artifact in a cluster receipt.

type WorkstreamClusterExample added in v0.1.0 

type WorkstreamClusterExample struct {
	Anchor           string                      `json:"anchor"`
	AnchorType       string                      `json:"anchor_type"`
	Dialect          string                      `json:"dialect,omitempty"`
	PackStrength     string                      `json:"pack_strength,omitempty"`
	Confidence       float64                     `json:"confidence"`
	ConfidenceRule   string                      `json:"confidence_rule"`
	ArtifactCount    int                         `json:"artifact_count"`
	EvidenceCount    int                         `json:"evidence_count"`
	RoleMix          map[string]int              `json:"role_mix,omitempty"`
	RoleFamilyMix    map[string]int              `json:"role_family_mix,omitempty"`
	EvidenceSources  []string                    `json:"evidence_sources,omitempty"`
	ExampleArtifacts []WorkstreamArtifactExample `json:"example_artifacts,omitempty"`
	ExampleCommits   []string                    `json:"example_commits,omitempty"`
	Capped           bool                        `json:"capped,omitempty"`
	Caveats          []string                    `json:"caveats,omitempty"`
}

WorkstreamClusterExample gives a compact receipt for a materialized anchor cluster.

type WorkstreamDialectProfileDiagnostics added in v0.1.0 

type WorkstreamDialectProfileDiagnostics struct {
	DialectCounts   map[string]int            `json:"dialect_counts,omitempty"`
	CrossRoleCounts map[string]int            `json:"cross_role_counts,omitempty"`
	TrustedDialects map[string]string         `json:"trusted_dialects,omitempty"`
	RiskFlags       map[string]int            `json:"risk_flags,omitempty"`
	EvidenceSources map[string]map[string]int `json:"evidence_sources,omitempty"`
	RoleFamilies    map[string]map[string]int `json:"role_families,omitempty"`
}

WorkstreamDialectProfileDiagnostics summarizes repo-local work-reference conventions.

type WorkstreamEvidenceDiagnostics added in v0.1.0 

type WorkstreamEvidenceDiagnostics struct {
	Enabled               bool                                 `json:"enabled"`
	AnchorsSeen           int                                  `json:"anchors_seen,omitempty"`
	AnchorsMaterialized   int                                  `json:"anchors_materialized,omitempty"`
	MentionsIndexed       int                                  `json:"mentions_indexed,omitempty"`
	EdgesIndexed          int                                  `json:"edges_indexed,omitempty"`
	EdgesByType           map[string]int                       `json:"edges_by_type,omitempty"`
	EdgesByPackStrength   map[string]int                       `json:"edges_by_pack_strength,omitempty"`
	DialectCounts         map[string]int                       `json:"dialect_counts,omitempty"`
	CappedDialectCounts   map[string]int                       `json:"capped_dialect_counts,omitempty"`
	ClustersCapped        int                                  `json:"clusters_capped,omitempty"`
	PackStrengthCounts    map[string]int                       `json:"pack_strength_counts,omitempty"`
	CappedPackStrengths   map[string]int                       `json:"capped_pack_strength_counts,omitempty"`
	CappedRoleFamilies    map[string]int                       `json:"capped_role_family_counts,omitempty"`
	StrongOrCrossClusters int                                  `json:"strong_or_cross_clusters,omitempty"`
	StrongOrCrossEdges    int                                  `json:"strong_or_cross_edges,omitempty"`
	PRFossilsSeen         int                                  `json:"pr_fossils_seen,omitempty"`
	PRFossilsMaterialized int                                  `json:"pr_fossils_materialized,omitempty"`
	PRFossilsByDialect    map[string]int                       `json:"pr_fossils_by_dialect,omitempty"`
	DialectProfile        *WorkstreamDialectProfileDiagnostics `json:"dialect_profile,omitempty"`
	NoisyAnchorsRejected  int                                  `json:"noisy_anchors_rejected,omitempty"`
	RejectedByReason      map[string]int                       `json:"rejected_by_reason,omitempty"`
	TopRejectedAnchors    []WorkstreamRejectedAnchorExample    `json:"top_rejected_anchors,omitempty"`
	TopClusters           []WorkstreamClusterExample           `json:"top_clusters,omitempty"`
	TopCappedClusters     []WorkstreamClusterExample           `json:"top_capped_clusters,omitempty"`
}

WorkstreamEvidenceDiagnostics is emitted by ds scan --json when workstream evidence is enabled.

type WorkstreamRejectedAnchorExample added in v0.1.0 

type WorkstreamRejectedAnchorExample struct {
	Anchor  string `json:"anchor"`
	Type    string `json:"type,omitempty"`
	Dialect string `json:"dialect,omitempty"`
	Reason  string `json:"reason"`
	Source  string `json:"source,omitempty"`
}

WorkstreamRejectedAnchorExample explains a suppressed anchor.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.