scan

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 29, 2026 License: MIT Imports: 30 Imported by: 0

Documentation

Overview

Package scan orchestrates artifact discovery: walks the repo, dispatches adapters, and upserts artifacts/revisions/todos/criteria into the store.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func SourceTypeDisplayLabel 

func SourceTypeDisplayLabel(sourceType string) string

SourceTypeDisplayLabel returns human-facing copy for `ds scan` summaries. Internal pipeline IDs remain markdown / openspec / adr in config and the DB; this layer is display-only.

Types

type EvidenceConceptExample added in v0.1.0 

type EvidenceConceptExample struct {
	Kind              string `json:"kind"`
	Canonical         string `json:"canonical"`
	DocumentFrequency int    `json:"document_frequency"`
	Reason            string `json:"reason"`
}

EvidenceConceptExample explains one concept skipped from edge materialization.

type EvidenceEdgeExample added in v0.1.0 

type EvidenceEdgeExample struct {
	EdgeType      string  `json:"edge_type"`
	Source        string  `json:"source"`
	Target        string  `json:"target"`
	SourceSignal  string  `json:"source_signal"`
	Explanation   string  `json:"explanation"`
	Confidence    float64 `json:"confidence"`
	EvidenceCount int     `json:"evidence_count"`
}

EvidenceEdgeExample gives a compact receipt-like example for diagnostics.

type EvidenceGraphDiagnostics added in v0.1.0 

type EvidenceGraphDiagnostics struct {
	RichTypedIndex       bool                     `json:"rich_typed_index,omitempty"`
	ConceptsIndexed      int                      `json:"concepts_indexed"`
	MentionsIndexed      int                      `json:"mentions_indexed"`
	EdgesIndexed         int                      `json:"edges_indexed"`
	ConceptsByKind       map[string]int           `json:"concepts_by_kind,omitempty"`
	MentionsByField      map[string]int           `json:"mentions_by_field,omitempty"`
	EdgesByType          map[string]int           `json:"edges_by_type,omitempty"`
	NoisyConceptsSkipped int                      `json:"noisy_concepts_skipped,omitempty"`
	TopNoisyConcepts     []EvidenceConceptExample `json:"top_noisy_concepts,omitempty"`
	TopEdges             []EvidenceEdgeExample    `json:"top_edges,omitempty"`
}

EvidenceGraphDiagnostics is a scan-time summary of persisted graph evidence.

type GitEvidenceDiagnostics added in v0.1.0 

type GitEvidenceDiagnostics struct {
	Enabled                    bool                     `json:"enabled"`
	HistoryShape               string                   `json:"history_shape"`
	IsShallow                  bool                     `json:"is_shallow,omitempty"`
	Branch                     string                   `json:"branch,omitempty"`
	MaxCommits                 int                      `json:"max_commits"`
	MaxFilesPerCommit          int                      `json:"max_files_per_commit"`
	TotalCommits               int                      `json:"total_commits,omitempty"`
	CommitsRead                int                      `json:"commits_read,omitempty"`
	CommitsStored              int                      `json:"commits_stored,omitempty"`
	FilesStored                int                      `json:"files_stored,omitempty"`
	BodiesTruncated            int                      `json:"commit_bodies_truncated,omitempty"`
	EdgesIndexed               int                      `json:"edges_indexed,omitempty"`
	EdgesByType                map[string]int           `json:"edges_by_type,omitempty"`
	SkippedLargeCommits        int                      `json:"skipped_large_commits,omitempty"`
	SkippedMergeCommits        int                      `json:"skipped_merge_commits,omitempty"`
	SkippedLockfileOnlyCommits int                      `json:"skipped_lockfile_only_commits,omitempty"`
	SkippedNoMappedArtifacts   int                      `json:"skipped_no_mapped_artifacts,omitempty"`
	PathPairsEvaluated         int                      `json:"path_pairs_evaluated,omitempty"`
	PathPairsMaterialized      int                      `json:"path_pairs_materialized,omitempty"`
	CappedSourcePaths          int                      `json:"capped_source_paths,omitempty"`
	HighDensitySourcePaths     int                      `json:"high_density_source_paths,omitempty"`
	CappedCommitPathPairs      int                      `json:"capped_commit_path_pairs,omitempty"`
	CappedPathPairs            int                      `json:"capped_path_pairs,omitempty"`
	CappedArtifactEdges        int                      `json:"capped_artifact_edges,omitempty"`
	GitError                   string                   `json:"git_error,omitempty"`
	TopEdges                   []GitEvidenceEdgeExample `json:"top_edges,omitempty"`
}

GitEvidenceDiagnostics is emitted by ds scan --json when git evidence is enabled.

type GitEvidenceEdgeExample added in v0.1.0 

type GitEvidenceEdgeExample struct {
	EdgeType          string   `json:"edge_type"`
	Source            string   `json:"source"`
	Target            string   `json:"target"`
	SourceTitle       string   `json:"source_title,omitempty"`
	TargetTitle       string   `json:"target_title,omitempty"`
	SourceKind        string   `json:"source_kind,omitempty"`
	TargetKind        string   `json:"target_kind,omitempty"`
	SourceSubtype     string   `json:"source_subtype,omitempty"`
	TargetSubtype     string   `json:"target_subtype,omitempty"`
	SourcePath        string   `json:"source_path,omitempty"`
	TargetPath        string   `json:"target_path,omitempty"`
	SourceSignal      string   `json:"source_signal"`
	Explanation       string   `json:"explanation"`
	Confidence        float64  `json:"confidence"`
	EvidenceCount     int      `json:"evidence_count"`
	Commits           []string `json:"commits,omitempty"`
	FileExamples      []string `json:"file_examples,omitempty"`
	HistoryShape      string   `json:"history_shape,omitempty"`
	ConfidenceRule    string   `json:"confidence_rule,omitempty"`
	SourcePathDensity int      `json:"source_path_density,omitempty"`
	TargetPathDensity int      `json:"target_path_density,omitempty"`
}

GitEvidenceEdgeExample is a compact receipt for manual edge audit.

type ProgressEvent added in v0.1.0 

type ProgressEvent struct {
	Phase                string         `json:"phase"`
	Event                string         `json:"event,omitempty"`
	ElapsedMS            int64          `json:"elapsed_ms,omitempty"`
	FilesTotal           int            `json:"files_total,omitempty"`
	FilesScanned         int            `json:"files_scanned,omitempty"`
	CurrentAdapter       string         `json:"current_adapter,omitempty"`
	CandidatesTotal      int            `json:"candidates_total,omitempty"`
	CandidatesProcessed  int            `json:"candidates_processed,omitempty"`
	CandidatesDiscovered map[string]int `json:"candidates_discovered,omitempty"`
	CandidatesParsed     map[string]int `json:"candidates_parsed,omitempty"`
	ArtifactsUpserted    map[string]int `json:"artifacts_upserted,omitempty"`
	ParseDurationMS      int64          `json:"parse_duration_ms,omitempty"`
	ClassifierDurationMS int64          `json:"classifier_duration_ms,omitempty"`
	WriterDurationMS     int64          `json:"writer_duration_ms,omitempty"`
	FlushDurationMS      int64          `json:"flush_duration_ms,omitempty"`
	FTSDurationMS        int64          `json:"fts_duration_ms,omitempty"`
	RowsWritten          map[string]int `json:"rows_written,omitempty"`
	ChunksFlushed        map[string]int `json:"chunks_flushed,omitempty"`
	DeferredFTSRows      int            `json:"deferred_fts_rows,omitempty"`
	InventoryFiles       int            `json:"inventory_files,omitempty"`
	SharedDiscovery      bool           `json:"shared_discovery,omitempty"`
	ParallelWorkers      int            `json:"parallel_workers,omitempty"`
	CappedDiscovery      bool           `json:"capped_discovery,omitempty"`
	TransactionEnabled   bool           `json:"transaction_enabled,omitempty"`
	SkipAuthoredAtLookup bool           `json:"skip_authored_at_lookup,omitempty"`
}

ProgressEvent is a coarse scan heartbeat for long eval/index runs.

type Result 

type Result struct {
	Found              map[string]int                       `json:"Found"`
	SourcesBreakdown   []SourceBreakdownRow                 `json:"sources_breakdown"`
	New                int                                  `json:"New"`
	Updated            int                                  `json:"Updated"`
	Unchanged          int                                  `json:"Unchanged"`
	Hints              []ScanHint                           `json:"hints,omitempty"`
	OpenSpec           *openspecmetrics.Metrics             `json:"openspec,omitempty"`
	EvidenceGraph      *EvidenceGraphDiagnostics            `json:"evidence_graph,omitempty"`
	GitEvidence        *GitEvidenceDiagnostics              `json:"git_evidence,omitempty"`
	WorkstreamEvidence *WorkstreamEvidenceDiagnostics       `json:"workstream_evidence,omitempty"`
	SourceCompanions   *SourceCompanionAdmissionDiagnostics `json:"source_companion_admission,omitempty"`
	// contains filtered or unexported fields
}

Result holds scan summary counts and per-source breakdown for CLI output.

JSON shape (see ds scan --json):

  • "Found": map of adapter/source pipeline name → count of successfully indexed artifacts
  • "sources_breakdown": array of { source_type, label, count, formats }
  • "New", "Updated", "Unchanged": revision outcomes
  • "hints": optional; only when all adapters indexed zero artifacts AND at least one hint candidate exists. Empty candidate list omits the key (encoding/json omitempty on []ScanHint).

type RunOptions added in v0.1.0 

type RunOptions struct {
	MaxCandidatesByAdapter    map[string]int
	FileWorkerCount           int
	UseTransaction            bool
	SkipAuthoredAtLookup      bool
	FreshIndex                bool
	IncludeGitEvidence        bool
	IncludeWorkstreamEvidence bool
	RichTypedIndex            bool
	GitMaxCommits             int
	GitMaxFilesPerCommit      int
	Progress                  func(ProgressEvent)
	ProgressInterval          time.Duration
}

type ScanHint 

type ScanHint struct {
	Path           string `json:"path"`
	SourceType     string `json:"source_type,omitempty"`
	SuggestCommand string `json:"suggest_command,omitempty"`
}

ScanHint is one recovery hint when a scan finds zero artifacts (`hints` in ds scan --json).

type Scanner 

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner runs adapters against a repo and persists results.

func New 

func New(db *store.DB, ids *idgen.Factory, adpts []adapters.Adapter) *Scanner

New creates a Scanner with the given store and adapters.

func (*Scanner) Run 

func (s *Scanner) Run(ctx context.Context, repoRoot string, cfg *config.RepoConfig) (*Result, error)

Run scans the repo at repoRoot, using config if available.

func (*Scanner) RunWithOptions added in v0.1.0 

func (s *Scanner) RunWithOptions(ctx context.Context, repoRoot string, cfg *config.RepoConfig, opts RunOptions) (*Result, error)

RunWithOptions scans the repo with optional eval/runtime controls.

type SourceBreakdownRow 

type SourceBreakdownRow struct {
	SourceType string         `json:"source_type"`
	Label      string         `json:"label"`
	Count      int            `json:"count"`
	Formats    map[string]int `json:"formats"`
}

SourceBreakdownRow is one element of `ds scan --json` field "sources_breakdown".

type SourceCompanionAdmissionDiagnostics added in v0.1.0 

type SourceCompanionAdmissionDiagnostics struct {
	Enabled                  bool                              `json:"enabled"`
	TestFiles                int                               `json:"test_files,omitempty"`
	ExistingSourceCandidates int                               `json:"existing_source_candidates,omitempty"`
	CandidatesConsidered     int                               `json:"candidates_considered,omitempty"`
	Admitted                 int                               `json:"admitted,omitempty"`
	AlreadyPresent           int                               `json:"already_present,omitempty"`
	SkippedByCap             int                               `json:"skipped_by_cap,omitempty"`
	RejectedByReason         map[string]int                    `json:"rejected_by_reason,omitempty"`
	TopAdmitted              []SourceCompanionAdmissionExample `json:"top_admitted,omitempty"`
	TopRejected              []SourceCompanionRejectionExample `json:"top_rejected,omitempty"`
}

SourceCompanionAdmissionDiagnostics summarizes bounded source files admitted because indexed tests pointed at likely implementation companions.

type SourceCompanionAdmissionExample added in v0.1.0 

type SourceCompanionAdmissionExample struct {
	Path       string   `json:"path"`
	Signals    []string `json:"signals,omitempty"`
	Confidence string   `json:"confidence,omitempty"`
	TestPaths  []string `json:"test_paths,omitempty"`
}

SourceCompanionAdmissionExample is a compact receipt for one admitted source companion.

type SourceCompanionRejectionExample added in v0.1.0 

type SourceCompanionRejectionExample struct {
	Path   string `json:"path"`
	Reason string `json:"reason"`
	Signal string `json:"signal,omitempty"`
}

SourceCompanionRejectionExample is a compact receipt for one rejected source companion.

type WorkstreamArtifactExample added in v0.1.0 

type WorkstreamArtifactExample struct {
	ID      string `json:"id"`
	Title   string `json:"title,omitempty"`
	Kind    string `json:"kind,omitempty"`
	Subtype string `json:"subtype,omitempty"`
	Path    string `json:"path,omitempty"`
}

WorkstreamArtifactExample identifies one artifact in a cluster receipt.

type WorkstreamClusterExample added in v0.1.0 

type WorkstreamClusterExample struct {
	Anchor           string                      `json:"anchor"`
	AnchorType       string                      `json:"anchor_type"`
	Dialect          string                      `json:"dialect,omitempty"`
	PackStrength     string                      `json:"pack_strength,omitempty"`
	Confidence       float64                     `json:"confidence"`
	ConfidenceRule   string                      `json:"confidence_rule"`
	ArtifactCount    int                         `json:"artifact_count"`
	EvidenceCount    int                         `json:"evidence_count"`
	RoleMix          map[string]int              `json:"role_mix,omitempty"`
	RoleFamilyMix    map[string]int              `json:"role_family_mix,omitempty"`
	EvidenceSources  []string                    `json:"evidence_sources,omitempty"`
	ExampleArtifacts []WorkstreamArtifactExample `json:"example_artifacts,omitempty"`
	ExampleCommits   []string                    `json:"example_commits,omitempty"`
	Capped           bool                        `json:"capped,omitempty"`
	Caveats          []string                    `json:"caveats,omitempty"`
}

WorkstreamClusterExample gives a compact receipt for a materialized anchor cluster.

type WorkstreamDialectProfileDiagnostics added in v0.1.0 

type WorkstreamDialectProfileDiagnostics struct {
	DialectCounts   map[string]int            `json:"dialect_counts,omitempty"`
	CrossRoleCounts map[string]int            `json:"cross_role_counts,omitempty"`
	TrustedDialects map[string]string         `json:"trusted_dialects,omitempty"`
	RiskFlags       map[string]int            `json:"risk_flags,omitempty"`
	EvidenceSources map[string]map[string]int `json:"evidence_sources,omitempty"`
	RoleFamilies    map[string]map[string]int `json:"role_families,omitempty"`
}

WorkstreamDialectProfileDiagnostics summarizes repo-local work-reference conventions.

type WorkstreamEvidenceDiagnostics added in v0.1.0 

type WorkstreamEvidenceDiagnostics struct {
	Enabled               bool                                 `json:"enabled"`
	AnchorsSeen           int                                  `json:"anchors_seen,omitempty"`
	AnchorsMaterialized   int                                  `json:"anchors_materialized,omitempty"`
	MentionsIndexed       int                                  `json:"mentions_indexed,omitempty"`
	EdgesIndexed          int                                  `json:"edges_indexed,omitempty"`
	EdgesByType           map[string]int                       `json:"edges_by_type,omitempty"`
	EdgesByPackStrength   map[string]int                       `json:"edges_by_pack_strength,omitempty"`
	DialectCounts         map[string]int                       `json:"dialect_counts,omitempty"`
	CappedDialectCounts   map[string]int                       `json:"capped_dialect_counts,omitempty"`
	ClustersCapped        int                                  `json:"clusters_capped,omitempty"`
	PackStrengthCounts    map[string]int                       `json:"pack_strength_counts,omitempty"`
	CappedPackStrengths   map[string]int                       `json:"capped_pack_strength_counts,omitempty"`
	CappedRoleFamilies    map[string]int                       `json:"capped_role_family_counts,omitempty"`
	StrongOrCrossClusters int                                  `json:"strong_or_cross_clusters,omitempty"`
	StrongOrCrossEdges    int                                  `json:"strong_or_cross_edges,omitempty"`
	PRFossilsSeen         int                                  `json:"pr_fossils_seen,omitempty"`
	PRFossilsMaterialized int                                  `json:"pr_fossils_materialized,omitempty"`
	PRFossilsByDialect    map[string]int                       `json:"pr_fossils_by_dialect,omitempty"`
	DialectProfile        *WorkstreamDialectProfileDiagnostics `json:"dialect_profile,omitempty"`
	NoisyAnchorsRejected  int                                  `json:"noisy_anchors_rejected,omitempty"`
	RejectedByReason      map[string]int                       `json:"rejected_by_reason,omitempty"`
	TopRejectedAnchors    []WorkstreamRejectedAnchorExample    `json:"top_rejected_anchors,omitempty"`
	TopClusters           []WorkstreamClusterExample           `json:"top_clusters,omitempty"`
	TopCappedClusters     []WorkstreamClusterExample           `json:"top_capped_clusters,omitempty"`
}

WorkstreamEvidenceDiagnostics is emitted by ds scan --json when workstream evidence is enabled.

type WorkstreamRejectedAnchorExample added in v0.1.0 

type WorkstreamRejectedAnchorExample struct {
	Anchor  string `json:"anchor"`
	Type    string `json:"type,omitempty"`
	Dialect string `json:"dialect,omitempty"`
	Reason  string `json:"reason"`
	Source  string `json:"source,omitempty"`
}

WorkstreamRejectedAnchorExample explains a suppressed anchor.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.