pipeline

Documentation

Overview

Package pipeline wires the stages together: parsed blob → recognize → for each match (authenticate → recon → summarize) → notes. Each credential is isolated so one module's failure never aborts the others.

Index

• func IsGitRepo(dir string) bool
• func LooksLikeGitleaks(path string) bool
• func SortBySeverity(rs []Result, ctx score.Context)
• type Batch
  • func NewBatch(reg *module.Registry, opts Options) *Batch
  • func (bt *Batch) AnnotateDuplicates(results []Result)
  • func (bt *Batch) Run(b parse.Blob) []Result
  • func (bt *Batch) RunConcurrent(srcs []Source, emit func(Result), progress func(done int)) []Result
• type Options
• type Result
  • func ResultFromNote(n module.Note) Result
  • func Run(b parse.Blob, reg *module.Registry, opts Options) []Result
  • func RunSources(srcs []Source, reg *module.Registry, opts Options) []Result
• type Source
  • func FromGitleaks(path string) ([]Source, error)
  • func FromKingfisher(path string) ([]Source, error)
  • func FromNuclei(path string) ([]Source, error)
  • func FromTrufflehog(path string) ([]Source, error)
  • func GitHistorySources(dir string, onFile func(scanned int)) ([]Source, error)
  • func SourcesFromArchive(path string) ([]Source, bool)
  • func WalkDir(dir string, onFile func(scanned int)) ([]Source, error)

Functions

func IsGitRepo

func IsGitRepo(dir string) bool

IsGitRepo reports whether dir is inside a git working tree.

func LooksLikeGitleaks

func LooksLikeGitleaks(path string) bool

LooksLikeGitleaks reports whether a file is probably a gitleaks JSON report.

func SortBySeverity

func SortBySeverity(rs []Result, ctx score.Context)

SortBySeverity orders results by composite blast-radius score so the highest- impact (and any crown-jewel context match) surface first, invalids last.

Types

type Batch

type Batch struct {
	// contains filtered or unexported fields
}

Batch triages many sources sharing one harvestState, so a credential appearing in several files is reconned ONCE (saving network/OPSEC cost and de-noising output); the extra locations are recorded for AnnotateDuplicates.

func NewBatch

func NewBatch(reg *module.Registry, opts Options) *Batch

NewBatch creates a batch runner with fresh shared dedupe state.

func (*Batch) AnnotateDuplicates

func (bt *Batch) AnnotateDuplicates(results []Result)

AnnotateDuplicates appends an "also exposed in" finding to each result whose secret was also found in other source files (deduped away). The value groups those locations by exposure class — so N auto-saved history snapshots of one file read as one source, not N leaks — and the full path list rides in Detail (shown with -v / in JSON). Call it after all sources run, before sorting. (Not usable with streaming, where earlier results are already printed.)

func (*Batch) Run

func (bt *Batch) Run(b parse.Blob) []Result

Run triages one source within the batch (deduping against earlier sources).

func (*Batch) RunConcurrent

func (bt *Batch) RunConcurrent(srcs []Source, emit func(Result), progress func(done int)) []Result

RunConcurrent triages every source through a bounded worker pool and returns all results. emit (may be nil) is invoked for each result as it is produced; progress (may be nil) is invoked with the running count of completed sources. Both callbacks are serialized, so they may touch shared state and write output safely. Result order is not meaningful on the concurrent path — the caller sorts. Duplicate locations are still recorded for AnnotateDuplicates.

type Options

type Options struct {
	Live         bool
	Intrusive    bool // permit read-only-but-invasive actions (DB connect, k8s live, harvest)
	MinFootprint bool // OPSEC: run only the identity call, skip inventory fan-out
	Correlate    bool // read local hints to correlate SSH keys to candidate hosts
	Trace        bool // capture masked request/response bodies
	Endpoint     string
	Proxy        string // SOCKS5/HTTP proxy URL for HTTP recon egress
	Timeout      time.Duration
	Concurrency  int       // max credentials reconned at once on the live path (0 = default)
	StartedAt    time.Time // run start, stamped on live-validated findings (zero = now)
	// Select, when set, scopes the run: only recognized credentials whose module
	// name passes are reconned (the rest are skipped entirely, not just hidden).
	// Backs --only/--skip so a second, deeper pass needn't re-exercise everything.
	Select func(moduleName string) bool
}

Options controls a run.

type Result

type Result struct {
	Note    module.Note
	Planned []recon.PlannedCall
	// contains filtered or unexported fields
}

Result pairs a note with the calls that were planned/made for it.

func ResultFromNote

func ResultFromNote(n module.Note) Result

ResultFromNote wraps a pre-built Note (e.g. from the browser scanner, which produces capability/blast-radius findings rather than recognized credentials) as a Result so it flows through the same sort/tier/print path.

func Run

func Run(b parse.Blob, reg *module.Registry, opts Options) []Result

Run executes the pipeline over a blob and returns one result per credential, recursively triaging any secrets harvested from a secrets store (under --live --intrusive only).

func RunSources

func RunSources(srcs []Source, reg *module.Registry, opts Options) []Result

RunSources runs the pipeline over many sources sharing one dedupe state (so a credential present in several files is reconned once) through a bounded worker pool. Results are annotated with the other files each secret appeared in.

type Source

type Source struct {
	Label string
	Blob  parse.Blob
}

Source is one parsed input plus where it came from (for batch reporting).

func FromGitleaks

func FromGitleaks(path string) ([]Source, error)

FromGitleaks ingests a gitleaks JSON report and yields one Source per finding, so a prior scanner run can feed Geiger's triage directly.

func FromKingfisher

func FromKingfisher(path string) ([]Source, error)

FromKingfisher ingests a Kingfisher report (JSON envelope, JSONL records, or a JSON array) and yields one Source per finding. path "-" reads stdin, so the intended use is a streaming pipe that never lands secrets on disk.

Kingfisher's own fingerprint rides along on each Source: geiger cannot recompute it (it hashes the matched bytes with the origin label and the byte offsets, which don't survive re-typing) so it is carried verbatim, letting Kingfisher's viewer line geiger's findings up with its own.

func FromNuclei

func FromNuclei(path string) ([]Source, error)

FromNuclei ingests nuclei JSONL output (the `-j`/`-jsonl` stream, or a JSON array) and yields one Source per extracted credential value. nuclei casts the wide net — its templates extract any value that *looks* like a secret from an exposed endpoint — and geiger is the authority: each value flows through the same recognizer as every other source, so over-matches that aren't real credentials are dropped here. path "-" reads stdin, so the intended use is a streaming pipe (`nuclei … -j | geiger --from-nuclei - --live`) that never lands secrets on disk. The matched-at URL becomes the Source label, which drives the title provenance ("from https://host/.env"), the cross-source dedup/"also exposed in" rollup (one key exposed at many URLs collapses to one finding), and the internet-exposed-endpoint exposure class.

func FromTrufflehog

func FromTrufflehog(path string) ([]Source, error)

FromTrufflehog ingests a TruffleHog v3 JSON report (newline-delimited, the default `trufflehog ... --json` output, or a JSON array) and yields one Source per verified/unverified finding. TruffleHog over a home dir + git history is exactly what supply-chain malware runs, so this lets a responder triage that same dump.

func GitHistorySources

func GitHistorySources(dir string, onFile func(scanned int)) ([]Source, error)

GitHistorySources yields a Source per blob reachable from any ref, so a credential that was committed and later deleted still surfaces. The working tree is walked separately; a value present in both dedupes to one finding with the other location rolled up, which is exactly the signal you want — "removed from the tree, still in history".

This is opt-in: history is unbounded, and reading all of it would change the cost of pointing geiger at a repo.

func SourcesFromArchive

func SourcesFromArchive(path string) ([]Source, bool)

SourcesFromArchive expands path as an archive, reporting whether it was handled as one. A file named directly on the command line goes through here so that `geiger host.tar.gz` behaves like a walk of the tree inside it — being handed a tarball of a compromised host is the common IR case.

A file whose name says archive but whose contents don't parse as one is reported as unhandled, so the caller falls back to reading it as plain text.

func WalkDir

func WalkDir(dir string, onFile func(scanned int)) ([]Source, error)

WalkDir returns a Source per regular file under dir, skipping the obvious dependency / cache / build directories and generated noise files. IR works in volume: point Geiger at a tree of leaked files and triage them all — but a vendored dependency tree or a lockfile full of hashes is pure false-positive fuel, so we don't descend into it. onFile, if non-nil, is called with the running count of accepted files after each is added, for progress reporting.